Re: [HACKERS] logical decoding of two-phase transactions

>> - Modified HeapTupleSatisfiesVacuum to return HEAPTUPLE_RECENTLY_DEAD
>> if the transaction id is newer than OldestXmin. Doing this only for
>> CATALOG tables (htup->t_tableOid < (Oid) FirstNormalObjectId).
>
>
> Because logical decoding supports user-catalog relations, we need to use the
> same sort of logical that GetOldestXmin uses instead of a simple oid-range
> check. See RelationIsAccessibleInLogicalDecoding() and the
> user_catalog_table reloption.
>

Unfortunately, HeapTupleSatisfiesVacuum does not have the Relation
structure handily available to allow for these checks..

> Otherwise pseudo-catalogs used by logical decoding output plugins could
> still suffer issues with needed tuples getting vacuumed, though only if the
> txn being decoded made changes to those tables than ROLLBACKed. It's a
> pretty tiny corner case for decoding of 2pc but a bigger one when we're
> addressing streaming decoding.
>

We disallow rewrites on user_catalog_tables, so they cannot change
underneath. Yes, DML can be carried out on them inside a 2PC
transaction which then gets ROLLBACK'ed. But if it's getting aborted,
then we are not interested in that data anyways. Also, now that we
have the "filter_decode_txn_cb_wrapper()" function, we will stop
decoding by the next change record cycle because of the abort.

So, I am not sure if we need to track user_catalog_tables in
HeapTupleSatisfiesVacuum explicitly.

> Otherwise I'm really, really happy with how this is progressing and want to
> find time to play with it.

Yeah, I will do some more testing and add a few more test cases in the
test_decoding plugin. It might be handy to have a DELAY of a few
seconds after every change record processing, for example. That ways,
we can have a TAP test which can do a few WAL activities and then we
introduce a concurrent rollback midways from another session in the
middle of that delayed processing. I have done debugger based testing
of this concurrent rollback functionality as of now.

Another test (actually, functionality) that might come in handy, is to
have a way for DDL to be actually carried out on the subscriber. We
will need something like pglogical.replicate_ddl_command to be added
to the core for this to work. We can add this functionality as a
follow-on separate patch after discussing how we want to implement
that in core.

Regards,
Nikhils
--
Nikhil Sontakke http://www.2ndQuadrant.com/
PostgreSQL/Postgres-XL Development, 24x7 Support, Training & Services