Re: logical decoding and replication of sequences, take 2

On Wed, Dec 6, 2023 at 7:09 PM Tomas Vondra
<tomas(dot)vondra(at)enterprisedb(dot)com> wrote:
>
> Yes, if something like this happens, that'd be a problem:
>
> 1) decoding starts, with
>
> SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT
>
> 2) transaction that creates a new refilenode gets decoded, but we skip
> it because we don't have the correct snapshot
>
> 3) snapshot changes to SNAPBUILD_FULL_SNAPSHOT
>
> 4) we decode sequence change from nextval() for the sequence
>
> This would lead to us attempting to apply sequence change for a
> relfilenode that's not visible yet (and may even get aborted).
>
> But can this even happen? Can we start decoding in the middle of a
> transaction? How come this wouldn't affect e.g. XLOG_HEAP2_NEW_CID,
> which is also skipped until SNAPBUILD_FULL_SNAPSHOT. Or logical
> messages, where we also call the output plugin in non-transactional cases.

It's not a problem for logical messages because whether the message is
transaction or non-transactional is decided while WAL logs the message
itself. But here our problem starts with deciding whether the change
is transactional vs non-transactional, because if we insert the
'relfilenode' in hash then the subsequent sequence change in the same
transaction would be considered transactional otherwise
non-transactional. And XLOG_HEAP2_NEW_CID is just for changing the
snapshot->curcid which will only affect the catalog visibility of the
upcoming operation in the same transaction, but that's not an issue
because if some of the changes of this transaction are seen when
snapbuild state < SNAPBUILD_FULL_SNAPSHOT then this transaction has to
get committed before the state change to SNAPBUILD_CONSISTENT_SNAPSHOT
i.e. the commit LSN of this transaction is going to be <
start_decoding_at.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com