Re: [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

On Thu, Jun 28, 2012 at 12:04 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On Thursday, June 28, 2012 06:01:10 PM Robert Haas wrote:
>> On Tue, Jun 26, 2012 at 8:13 PM, Andres Freund <andres(at)2ndquadrant(dot)com>
> wrote:
>> > It even can be significantly higher than max_connections because
>> > subtransactions are only recognizable as part of their parent transaction
>> > uppon commit.
>>
>> I've been wondering whether sub-XID assignment was going to end up on
>> the list of things that need to be WAL-logged to enable logical
>> replication. It would be nicer to avoid that if we can, but I have a
>> feeling that we may not be able to.
> I don't think it needs to. We only need that information during commit and we
> have it there. If a subtxn aborts a separate abort is logged, so thats no
> problem. The 'merging' of the transactions would be slightly easier if we had
> the knowledge from the get go but that would add complications again in the
> case of rollbacks.
>
> What do you think we need it?

Well, I don't know for sure that we do. But, ultimately, I think
we're going to find that applying the whole transaction at
transaction-end is something that we need to optimize - so that we can
apply the transaction as it happens and commit it at the end. For
small transactions this is no big deal, but for big transactions it
could lead to hours and hours of replication delay. I'm not sure
whether it's practical to fix that in version 1 - certainly there are
a lot of issues there - but ultimately I think it's something that our
users are going to demand. And it seems like that might require
knowing the transaction tree at some point before the commit record
shows up.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company