Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

Hi Shveta,

Thanks for reviewing.

shveta malik <shveta(dot)malik(at)gmail(dot)com>, 25 Oca 2023 Çar, 16:02 tarihinde
şunu yazdı:

> On Mon, Jan 23, 2023 at 6:30 PM Melih Mutlu <m(dot)melihmutlu(at)gmail(dot)com>
> wrote:
> --I see initial data copied, but new catalog columns srrelslotname
> and srreloriginname are not updated:
> postgres=# select sublastusedid from pg_subscription;
> sublastusedid
> ---------------
> 2
>
> postgres=# select * from pg_subscription_rel;
> srsubid | srrelid | srsubstate | srsublsn | srrelslotname |
> srreloriginname
>
> ---------+---------+------------+-----------+---------------+-----------------
> 16409 | 16384 | r | 0/15219E0 | |
> 16409 | 16389 | r | 0/15219E0 | |
> 16409 | 16396 | r | 0/15219E0 | |
>
> When are these supposed to be updated? I thought the slotname created
> will be updated here. Am I missing something here?
>

If a relation is currently being synced by a tablesync worker and uses a
replication slot/origin for that operation, then srrelslotname and
srreloriginname fields will have values.
When a relation is done with its replication slot/origin, their info gets
removed from related catalog row, so that slot/origin can be reused for
another table or dropped if not needed anymore.
In your case, all relations are in READY state so it's expected that
srrelslotname and srreloriginname are empty. READY relations do not need a
replication slot/origin anymore.

Tables are probably synced so quickly that you're missing the moments when
a tablesync worker copies a relation and stores its rep. slot/origin in the
catalog.
If initial sync is long enough, then you should be able to see the columns
get updated. I follow [1] to make it longer and test if the patch really
updates the catalog.

> Also the v8 patch does not apply on HEAD, giving merge conflicts.
>

Rebased and resolved conflicts. Please check the new version

---
[1]
publisher:
SELECT 'CREATE TABLE table_'||i||'(i int);' FROM generate_series(1, 100)
g(i) \gexec
SELECT 'INSERT INTO table_'||i||' SELECT x FROM generate_series(1, 10000)
x' FROM generate_series(1, 100) g(i) \gexec
CREATE PUBLICATION mypub FOR ALL TABLES;

subscriber:
SELECT 'CREATE TABLE table_'||i||'(i int);' FROM generate_series(1, 100)
g(i) \gexec
CREATE SUBSCRIPTION mysub CONNECTION 'dbname=postgres port=5432 '
PUBLICATION mypub;
select * from pg_subscription_rel where srrelslotname <> ''; \watch 0.5

Thanks,
--
Melih Mutlu
Microsoft