Re: Logical Replication of sequences

From: vignesh C <vignesh21(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Peter Eisentraut <peter(at)eisentraut(dot)org>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, Euler Taveira <euler(at)eulerto(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, Hou, Zhijie/侯 志杰 <houzj(dot)fnst(at)fujitsu(dot)com>, "Jonathan S(dot) Katz" <jkatz(at)postgresql(dot)org>
Subject: Re: Logical Replication of sequences
Date: 2024-06-19 15:03:37
Message-ID: CALDaNm1xAfB7C+YVWtHOa941Yfi-cHYrGvQPh-SGgAmay9x3Og@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, 18 Jun 2024 at 16:10, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
>
> Agreed and I am not sure which is better because there is a value in
> keeping the state name the same for both sequences and tables. We
> probably need more comments in code and doc updates to make the
> behavior clear. We can start with the sequence state as 'init' for
> 'needs-to-be-sycned' and 'ready' for 'synced' and can change if others
> feel so during the review.

Here is a patch which does the sequence synchronization in the
following lines from the above discussion:
This commit introduces sequence synchronization during 1) creation of
subscription for initial sync of sequences 2) refresh publication to
synchronize the sequences for the newly created sequences 3) refresh
publication sequences for synchronizing all the sequences.
1) During subscription creation with CREATE SUBSCRIPTION (no syntax change):
- The subscriber retrieves sequences associated with publications.
- Sequences are added in the 'init' state to the pg_subscription_rel table.
- Sequence synchronization worker will be started if there are any
sequences to be synchronized
- A new sequence synchronization worker handles synchronization in
batches of 100 sequences:
a) Retrieves sequence values using pg_sequence_state from the publisher.
b) Sets sequence values accordingly.
c) Updates sequence state to 'READY' in pg_susbcripion_rel
d) Commits batches of 100 synchronized sequences.
2) Refreshing sequences with ALTER SUBSCRIPTION ... REFRESH
PUBLICATION (no syntax change):
- Stale sequences are removed from pg_subscription_rel.
- Newly added sequences in the publisher are added in 'init' state
to pg_subscription_rel.
- Sequence synchronization will be done by sequence sync worker as
listed in subscription creation process.
- Sequence synchronization occurs for newly added sequences only.
3) Introduce new command ALTER SUBSCRIPTION ... REFRESH PUBLICATION
SEQUENCES for refreshing all sequences:
- Removes stale sequences and adds newly added sequences from the
publisher to pg_subscription_rel.
- Resets all sequences in pg_subscription_rel to 'init' state.
- Initiates sequence synchronization for all sequences by sequence
sync worker as listed in subscription creation process.

Regards,
Vignesh

Attachment Content-Type Size
v20240619-0001-Introduce-pg_sequence_state-and-SetSequenc.patch text/x-patch 9.5 KB
v20240619-0002-Introduce-ALL-SEQUENCES-support-for-Postgr.patch text/x-patch 90.8 KB
v20240619-0003-Enhance-sequence-synchronization-during-su.patch text/x-patch 55.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Nathan Bossart 2024-06-19 15:09:02 Re: Report runtimes in pg_upgrade verbose mode
Previous Message Pavel Borisov 2024-06-19 15:02:19 Re: Missing docs for new enable_group_by_reordering GUC