Re: Skipping logical replication transactions on subscriber side

On Mon, Jan 24, 2022 at 7:40 PM Peter Eisentraut
<peter(dot)eisentraut(at)enterprisedb(dot)com> wrote:
>
> On 22.01.22 10:41, Amit Kapila wrote:
> >> Additionally, the description for pg_stat_subscription_workers should describe what happens once the transaction represented by last_error_xid has either been successfully processed or skipped. Does this "last error" stick around until another error happens (which is hopefully very rare) or does it reset to blanks?
> >>
> > It will be reset only on subscription drop, otherwise, it will stick
> > around until another error happens.
>
> Is this going to be a problem with transaction ID wraparound?
>

I think to avoid this we can send a message to clear this (at least to
clear XID in the view) after skipping the xact but there is no
guarantee that it will be received by the stats collector.
Additionally, the worker can periodically (say after every N (100,
500, etc) successful transaction) send a clear message after
successful apply. This will ensure that eventually the error entry
will be cleared.

> Do we
> need to use 64-bit xids for this?
>

For 64-bit XIds, as this reported XID is for the remote transactions,
I think we need to add 4-bytes to each transaction message(say Begin)
and that could be costly for small transactions. We also probably need
to make logical decoding aware of 64-bit XID? Note that XIDs in WAL
records are still 32-bit XID. I don't think this feature deserves such
a big (in terms of WAL and network message size) change.

--
With Regards,
Amit Kapila.