Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

Oops - now with attachments

On Mon, Aug 21, 2023 at 5:56 PM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:

> Hi Melih,
>
> Last week we revisited your implementation of design#2. Vignesh rebased
> it, and then made a few other changes.
>
> PSA v28*
>
> The patch changes include:
> * changed the logic slightly by setting recv_immediately(new variable), if
> this variable is set the main apply worker loop will not wait in this case.
> * setting the relation state to ready immediately if there are no more
> incremental changes to be synced.
> * receive the incremental changes if applicable and set the relation state
> to ready without waiting.
> * reuse the worker if the worker is free before trying to start a new
> table sync worker
> * restarting the tablesync worker only after wal_retrieve_retry_interval
>
> ~
>
> FWIW, we just wanted to share with you the performance measurements seen
> using this design#2 patch set:
>
> ======
>
> RESULTS (not busy tests)
>
> ------
> 10 empty tables
> 2w 4w 8w 16w
> HEAD: 125 119 140 133
> HEAD+v28*: 92 93 123 134
> %improvement: 27% 22% 12% -1%
> ------
> 100 empty tables
> 2w 4w 8w 16w
> HEAD: 1037 843 1109 1155
> HEAD+v28*: 591 625 2616 2569
> %improvement: 43% 26% -136% -122%
> ------
> 1000 empty tables
> 2w 4w 8w 16w
> HEAD: 15874 10047 9919 10338
> HEAD+v28*: 33673 12199 9094 9896
> %improvement: -112% -21% 8% 4%
> ------
> 2000 empty tables
> 2w 4w 8w 16w
> HEAD: 45266 24216 19395 19820
> HEAD+v28*: 88043 21550 21668 22607
> %improvement: -95% 11% -12% -14%
>
> ~~~
>
> Note - the results were varying quite a lot in comparison to the HEAD
> e.g. HEAD results are very consistent, but the v28* results observed are
> not
> HEAD 1000 (2w): 15861, 15777, 16007, 15950, 15886, 15740, 15846, 15740,
> 15908, 15940
> v28* 1000 (2w): 34214, 13679, 8792, 33289, 31976, 56071, 57042, 56163,
> 34058, 11969
>
> ------
> Kind Regards,
> Peter Smith.
> Fujitsu Australia
>