Re: Logical Replication of sequences

On Wed, Jun 5, 2024 at 8:45 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:

> On Tue, Jun 4, 2024 at 7:40 PM Ashutosh Bapat
> <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> wrote:
> >
> > On Tue, Jun 4, 2024 at 4:27 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
> wrote:
> >>
> >>
> >> 3. Replicate published sequences via walsender at the time of shutdown
> >> or incrementally while decoding checkpoint record. The two ways to
> >> achieve this are: (a) WAL log a special NOOP record just before
> >> shutting down checkpointer. Then allow the WALsender to read the
> >> sequence data and send it to the subscriber while decoding the new
> >> NOOP record. (b) Similar to the previous idea but instead of WAL
> >> logging a new record directly invokes a decoding callback after
> >> walsender receives a request to shutdown which will allow pgoutput to
> >> read and send required sequences. This approach has a drawback that we
> >> are adding more work at the time of shutdown but note that we already
> >> waits for all the WAL records to be decoded and sent before shutting
> >> down the walsender during shutdown of the node.
> >>
> >> Any other ideas?
> >>
> >
> > In case of primary crash the sequence won't get replicated. That is true
> even with the previous approach in case walsender is shut down because of a
> crash, but it is more serious with this approach.
> >
>
> Right, but if we just want to support a major version upgrade scenario
> then this should be fine because upgrades require a clean shutdown.
>
> >
> How about periodically sending this information?
> >
>
> Now, if we want to support some sort of failover then probably this
> will help. Do you have that use case in mind?

Regular failover was a goal for supporting logical replication of
sequences. That might be more common than major upgrade scenario.

> If we want to send
> periodically then we can do it when decoding checkpoint
> (XLOG_CHECKPOINT_ONLINE) or some other periodic WAL record like
> running_xacts (XLOG_RUNNING_XACTS).
>
>
Yeah. I am thinking along those lines.

It must be noted, however, that none of those optional make sure that the
replicated sequence's states are consistent with the replicated object
state which use those sequences. E.g. table t1 uses sequence s1. By last
sequence replication, as of time T1, let's say t1 had consumed values upto
vl1 from s1. But later, by time T2, it consumed values upto vl2 which were
not replicated but the changes to t1 by T2 were replicated. If failover
happens at that point, INSERTs on t1 would fail because of duplicate keys
(values between vl1 and vl2). Previous attempt to support logical sequence
replication solved this problem by replicating a future state of sequences
(current value +/- log count). Similarly, if the sequence was ALTERed
between T1 and T2, the state of sequence on replica would be inconsistent
with the state of t1. Failing over at this stage, might end t1 in an
inconsistent state.

--
Best Wishes,
Ashutosh Bapat