Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

On Fri, Jan 3, 2025 at 6:31 AM Euler Taveira <euler(at)eulerto(dot)com> wrote:
>
> On Fri, Jan 3, 2025, at 10:14 AM, Bertrand Drouvot wrote:
>
> If we don't want to force wal_level = logical to enable logical decoding (your
> second idea) then I think that it would be better to "hide" everything behind
> logical replication slot creation / deletion. That would mean that having at
> least one logical replication slot created would be synonym to "activate logical
> decoding" and zero logical replication slot created would be synonym to "deactivate
> logical decoding".
>
>
> I like this idea. The logical replication slot existence already has the
> required protections and guarantees (no running transactions from the past while
> creating) for logical decoding.

I agree that it's better behavior.

>
> Having said that, you are basically folding 'logical' machinery into 'replica'.
> The 'logical' option can still exists but it will be less attractive because it
> increases the WAL volume even if you are not using logical replication. I don't
> know if the current 'logical' behavior (WAL records for logical decoding even
> if there is no active logical replication) has any use case (maybe someone
> inspects these extra records for analysis) but one suggestion (separate patch)
> is to make 'logical' synonymous with the new 'replica' behavior (logical
> decoding capability). This opens the door to remove 'logical' in future
> releases (accepted as synonym but not documented).

To enable the logical decoding automatically when the first slot is
created, probably we need to do something like:

1. enable WAL-logging logical info.
2. wait for all running transactions to finish.
3. enable logical decoding, and write a XLOG_LOGICAL_DECODING_STATUS record,
4. create a logical slot
4-1. reserve the WAL and write an XLOG_RUNNING_XACTS record.
4-2. wait for all transactions in the XLOG_RUNNING_XACTS record to
finish during creating the initial snapshot.

That is, we could need to wait for different sets of transactions to
finish (at 2 and 4-2). Which could surprise users since the first slot
creation could have users wait for a longer time. Using the 'logical'
level would avoid such waits.

Also, if we make the 'logical' level synonymous with the new 'replica'
behavior, we need to adjust regression tests as well. In some
regression tests (e.g., ondisk_startup.spec), we create a logical slot
while a transaction is concurrently running. We expect we wait for the
concurrent transaction during the slot creation but with the above
algorithm, we would wait for them to activate logical decoding. So I
agree that making the 'logical' level synonymous with the new
'replica' behavior is done in a separate patch. It would be a good
start to implement the new 'replica' level behavior.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com