Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

Hi,

This patch is not applying on the HEAD. Please rebase and share the
updated patch.

Thanks and Regards
Shlok Kyal

On Wed, 10 Jan 2024 at 14:55, Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
>
> Oops - now with attachments
>
> On Mon, Aug 21, 2023 at 5:56 PM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
>>
>> Hi Melih,
>>
>> Last week we revisited your implementation of design#2. Vignesh rebased it, and then made a few other changes.
>>
>> PSA v28*
>>
>> The patch changes include:
>> * changed the logic slightly by setting recv_immediately(new variable), if this variable is set the main apply worker loop will not wait in this case.
>> * setting the relation state to ready immediately if there are no more incremental changes to be synced.
>> * receive the incremental changes if applicable and set the relation state to ready without waiting.
>> * reuse the worker if the worker is free before trying to start a new table sync worker
>> * restarting the tablesync worker only after wal_retrieve_retry_interval
>>
>> ~
>>
>> FWIW, we just wanted to share with you the performance measurements seen using this design#2 patch set:
>>
>> ======
>>
>> RESULTS (not busy tests)
>>
>> ------
>> 10 empty tables
>> 2w 4w 8w 16w
>> HEAD: 125 119 140 133
>> HEAD+v28*: 92 93 123 134
>> %improvement: 27% 22% 12% -1%
>> ------
>> 100 empty tables
>> 2w 4w 8w 16w
>> HEAD: 1037 843 1109 1155
>> HEAD+v28*: 591 625 2616 2569
>> %improvement: 43% 26% -136% -122%
>> ------
>> 1000 empty tables
>> 2w 4w 8w 16w
>> HEAD: 15874 10047 9919 10338
>> HEAD+v28*: 33673 12199 9094 9896
>> %improvement: -112% -21% 8% 4%
>> ------
>> 2000 empty tables
>> 2w 4w 8w 16w
>> HEAD: 45266 24216 19395 19820
>> HEAD+v28*: 88043 21550 21668 22607
>> %improvement: -95% 11% -12% -14%
>>
>> ~~~
>>
>> Note - the results were varying quite a lot in comparison to the HEAD
>> e.g. HEAD results are very consistent, but the v28* results observed are not
>> HEAD 1000 (2w): 15861, 15777, 16007, 15950, 15886, 15740, 15846, 15740, 15908, 15940
>> v28* 1000 (2w): 34214, 13679, 8792, 33289, 31976, 56071, 57042, 56163, 34058, 11969
>>
>> ------
>> Kind Regards,
>> Peter Smith.
>> Fujitsu Australia