From: | Craig Ringer <craig(dot)ringer(at)enterprisedb(dot)com> |
---|---|
To: | pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Logical decoding without slots: decoding in lockstep with recovery |
Date: | 2020-12-23 06:56:07 |
Message-ID: | CAGRY4nzdD0t3t6TNAMGydvt=_GSVcN90fMiw6uLBRu6m7V8=YQ@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi all
I want to share an idea I've looked at a few times where I've run into
situations where logical slots were inadvertently dropped, or where it
became necessary to decode changes in the past on a slot.
As most of you will know you can't just create a logical slot in the past.
Even if it was permitted, it'd be unsafe due to catalog_xmin retention
requirements and missing WAL.
But if we can arrange a physical replica to replay the WAL of interest and
decode each commit as soon as it's replayed by the startup process, we know
the needed catalog rows must all exist, so it's safe to decode the change.
So it should be feasible to run logical decoding in standby, even without a
replication slot, so long as we:
* pause startup process after each xl_xact_commit
* wake the walsender running logical decoding
* decode and process until ReorderBufferCommit for the just-committed xact
returns
* wake the startup process to decode the up to the next commit
Can anyone see any obvious problem with this?
I don't think the potential issues with WAL commit visibility order vs
shmem commit visibility order should be a concern.
I see this as potentially useful in data recovery, where you might want to
be able to extract a change stream for a subset of tables from PITR
recovery, for example. Also for audit use.
From | Date | Subject | |
---|---|---|---|
Next Message | Craig Ringer | 2020-12-23 07:11:58 | TAP PostgresNode function to gdb stacks and optional cores for all backends |
Previous Message | Bharath Rupireddy | 2020-12-23 06:37:25 | Re: Better client reporting for "immediate stop" shutdowns |