Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

On Mon, Jan 6, 2025 at 3:20 AM Ashutosh Bapat
<ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> wrote:
>
> On Sat, Jan 4, 2025 at 6:03 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> >
> > On Fri, Jan 3, 2025 at 6:31 AM Euler Taveira <euler(at)eulerto(dot)com> wrote:
> > >
> > > On Fri, Jan 3, 2025, at 10:14 AM, Bertrand Drouvot wrote:
> > >
> > > If we don't want to force wal_level = logical to enable logical decoding (your
> > > second idea) then I think that it would be better to "hide" everything behind
> > > logical replication slot creation / deletion. That would mean that having at
> > > least one logical replication slot created would be synonym to "activate logical
> > > decoding" and zero logical replication slot created would be synonym to "deactivate
> > > logical decoding".
> > >
> > >
> > > I like this idea. The logical replication slot existence already has the
> > > required protections and guarantees (no running transactions from the past while
> > > creating) for logical decoding.
> >
> > I agree that it's better behavior.
> >
> > >
> > > Having said that, you are basically folding 'logical' machinery into 'replica'.
> > > The 'logical' option can still exists but it will be less attractive because it
> > > increases the WAL volume even if you are not using logical replication. I don't
> > > know if the current 'logical' behavior (WAL records for logical decoding even
> > > if there is no active logical replication) has any use case (maybe someone
> > > inspects these extra records for analysis) but one suggestion (separate patch)
> > > is to make 'logical' synonymous with the new 'replica' behavior (logical
> > > decoding capability). This opens the door to remove 'logical' in future
> > > releases (accepted as synonym but not documented).

FYI one possible use case of the 'logical' level would be that if we
support retrieving WAL records for logical decoding from somewhere
like restore_command in the future, users would like to keep the
wal_level 'logical' or keep the logical decoding active.

> >
> > To enable the logical decoding automatically when the first slot is
> > created, probably we need to do something like:
> >
> > 1. enable WAL-logging logical info.
> > 2. wait for all running transactions to finish.
> > 3. enable logical decoding, and write a XLOG_LOGICAL_DECODING_STATUS record,
> > 4. create a logical slot
> > 4-1. reserve the WAL and write an XLOG_RUNNING_XACTS record.
> > 4-2. wait for all transactions in the XLOG_RUNNING_XACTS record to
> > finish during creating the initial snapshot.
>
> A naive question: Can we combine step 2 and 4-2 - may be we need to
> combine 4-1 and 3 too.

Yeah, maybe we can somewhat optimize this process. Probably we can do like:

1. create a logical slot.
2. enable WAL-logging logical info.
3. reserve the WAL and write an XLOG_RUNNING_XACTS record.
4. wait for all transactions in the XLOG_RUNNING_XACTS record to
finish during creating the initial snapshot.
4-1. write a XLOG_LOGICAL_DECODING_STATUS record if a process
detects all transactions in the XLOG_RUNNING_XACTS record to finish.

However, we need to note that 4-1 should be done by only one process.
Also, it's a bit weird and concerns to me that the snapbuild
initialization contains the process of activating logical decoding.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com