Re: Logical Decoding and HeapTupleSatisfiesVacuum assumptions

On 22 January 2018 at 20:03, Petr Jelinek <petr(dot)jelinek(at)2ndquadrant(dot)com> wrote:
> On 22/01/18 20:21, Robert Haas wrote:

>>> The other option would be to make sure 2PC decoding/tx streaming does
>>> not read aborted transaction but that would mean locking the transaction
>>> every time we give control to output plugin. Given that output plugin
>>> may do network write, this would really mean locking the transaction for
>>> and unbounded period of time. That does not strike me as something we
>>> want to do, decoding should not interact with frontend transaction
>>> management, definitely not this badly.
>>
>> Yeah, I don't think it's acceptable to postpone abort indefinitely.
>>
>
> Hmm, maybe less bad and potentially acceptable version of this would be
> to make TransactionIdIsInProgress() treat transaction as running if it's
> being decoded, that might solve most of the issues. There is still
> potential interlocking issue with UPDATEs but they have to change same
> tuples as the aborted tx did.
>
> What I mean concretely is that decoding would check if tx has aborted,
> if not then it would put somewhere in shmem info about it working on the
> tx and once plugin is done with current call it would reset the shmem
> info to invalid tx. Then the TransactionIdIsInProgress() would check
> this shmem info before checking about abort. If the first check would
> show that transaction is aborted then decoding would not bother decoding
> it further.

I must be missing something in this discussion, cos I don't see any
problems with this "other option".

Surely we prepare the 2PCxact and then it is persistent. Yes,
potentially for an unbounded period of time. And it is (usually) up to
the XA manager to resolve that. 2PC interacts with transaction
management and yes, it can be slow. But the choice is slow and
consistent, or not. This would only be used with the full choice of
the user, just like synchronous_commit.

In this case, we call the decoding plugin's precommit hook which would
then prepare the 2PCxact and set a non-persistent flag saying it is
being decoded. If decoding completes normally we release the lock and
commit. If decoding fails or the DBA has another reason to do so, we
provide a function that allows the flag to be unlocked. While it is
locked the 2PCxact cannot be aborted or committed.

There is no danger of accidental abort because the prepare has persistent state.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services