| From: | Melih Mutlu <m(dot)melihmutlu(at)gmail(dot)com> |
|---|---|
| To: | pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
| Cc: | Melanie Plageman <melanieplageman(at)gmail(dot)com>, "wangw(dot)fnst(at)fujitsu(dot)com" <wangw(dot)fnst(at)fujitsu(dot)com>, "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, shveta malik <shveta(dot)malik(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com> |
| Subject: | Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication |
| Date: | 2023-06-23 13:32:47 |
| Message-ID: | CAGPVpCRnMK-MvkmUDBZ=utRU=wuN6Wp1XGELnCMR0cekWh3kpg@mail.gmail.com |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi hackers,
You can find the updated patchset attached.
I worked to address the reviews and made some additional changes.
Let me first explain the new patchset.
0001: Refactors the logical replication code, mostly worker.c and
tablesync.c. Although this patch makes it easier to reuse workers, I
believe that it's useful even by itself without other patches. It does
not improve performance or anything but aims to increase readability
and such.
0002: This is only to reuse worker processes, everything else stays
the same (replication slots/origins etc.).
0003: Adds a new command for streaming replication protocol to create
a snapshot by an existing replication slot.
0004: Reuses replication slots/origins together with workers.
Even only 0001 and 0002 are enough to improve table sync performance
at the rates previously shared on this thread. This also means that
currently 0004 (reusing replication slots/origins) does not improve as
much as I would expect, even though it does not harm either.
I just wanted to share what I did so far, while I'm continuing to
investigate it more to see what I'm missing in patch 0004.
Thanks,
--
Melih Mutlu
Microsoft
| Attachment | Content-Type | Size |
|---|---|---|
| v14-0004-Reuse-Replication-Slot-and-Origin-in-Tablesync.patch | application/octet-stream | 55.8 KB |
| 0001-Refactor-to-split-Apply-and-Tablesync-Workers.patch | application/octet-stream | 21.1 KB |
| v3-0002-Reuse-Tablesync-Workers.patch | application/octet-stream | 10.4 KB |
| v11-0003-Add-replication-protocol-cmd-to-create-a-snapshot.patch | application/octet-stream | 21.0 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Melih Mutlu | 2023-06-23 13:39:57 | Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication |
| Previous Message | Ashutosh Bapat | 2023-06-23 13:24:08 | Re: logical decoding and replication of sequences, take 2 |