Re: Changeset Extraction v7.6.1

On Tue, Feb 18, 2014 at 4:07 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> 2. I think the snapshot-export code is fundamentally misdesigned. As
>> I said before, the idea that we're going to export one single snapshot
>> at one particular point in time strikes me as extremely short-sighted.
>
> I don't think so. It's precisely what you need to implement a simple
> replication solution. Yes, there are usecases that could benefit from
> more possibilities, but that's always the case.
>
>> For example, consider one-to-many replication where clients may join
>> or depart the replication group at any time. Whenever somebody joins,
>> we just want a <snapshot, LSN> pair such that they can apply all
>> changes after the LSN except for XIDs that would have been visible to
>> the snapshot.
>
> And? They need to create individual replication slots, which each will
> get a snapshot.

So we have to wait for startup N times, and transmit the change stream
N times, instead of once? Blech.

>> And in fact, we don't even need any special machinery
>> for that; the client can just make a connection and *take a snapshot*
>> once decoding is initialized enough.
>
> No, they can't. Two reasons: For one the commit order between snapshots
> and WAL isn't necessarily the same.

So what?

> For another, clients now need logic
> to detect whether a transaction's contents has already been applied or
> has not been applied yet, that's nontrivial.

My point is, I think we should be trying to *make* that trivial,
rather than doing this.

>> 3. As this feature is proposed, the only plugin we'll ship with 9.4 is
>> a test_decoding plugin which, as its own documentation says, "doesn't
>> do anything especially useful." What exactly do we gain by forcing
>> users who want to make use of these new capabilities to write C code?
>
> It gains us to have a output plugin in which we can easily demonstrate
> features so they can be tested in the regression tests. Which I find to
> be rather important.
> Just like e.g. the test_shm_mq stuff doesn't do anything really useful.

It definitely doesn't, but this patch is a lot closer to being done
than parallel query is, so I'm not sure it's a fair comparison.

>> The test_decoding plugin doesn't seem tremendously
>> much simpler than something that someone could actually use, so why
>> not make that the goal?
>
> For one, it being a designated toy plugin allows us to easily change it,
> to showcase/test new features. For another, I still don't agree that
> it's easy to agree to an output format. I think we should include some
> that matured into 9.5.

I regret that more effort has not been made in that area.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company