Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

Hi Melih,

Last week we revisited your implementation of design#2. Vignesh rebased it,
and then made a few other changes.

PSA v28*

The patch changes include:
* changed the logic slightly by setting recv_immediately(new variable), if
this variable is set the main apply worker loop will not wait in this case.
* setting the relation state to ready immediately if there are no more
incremental changes to be synced.
* receive the incremental changes if applicable and set the relation state
to ready without waiting.
* reuse the worker if the worker is free before trying to start a new table
sync worker
* restarting the tablesync worker only after wal_retrieve_retry_interval

~

FWIW, we just wanted to share with you the performance measurements seen
using this design#2 patch set:

======

RESULTS (not busy tests)

------
10 empty tables
2w 4w 8w 16w
HEAD: 125 119 140 133
HEAD+v28*: 92 93 123 134
%improvement: 27% 22% 12% -1%
------
100 empty tables
2w 4w 8w 16w
HEAD: 1037 843 1109 1155
HEAD+v28*: 591 625 2616 2569
%improvement: 43% 26% -136% -122%
------
1000 empty tables
2w 4w 8w 16w
HEAD: 15874 10047 9919 10338
HEAD+v28*: 33673 12199 9094 9896
%improvement: -112% -21% 8% 4%
------
2000 empty tables
2w 4w 8w 16w
HEAD: 45266 24216 19395 19820
HEAD+v28*: 88043 21550 21668 22607
%improvement: -95% 11% -12% -14%

~~~

Note - the results were varying quite a lot in comparison to the HEAD
e.g. HEAD results are very consistent, but the v28* results observed are not
HEAD 1000 (2w): 15861, 15777, 16007, 15950, 15886, 15740, 15846, 15740,
15908, 15940
v28* 1000 (2w): 34214, 13679, 8792, 33289, 31976, 56071, 57042, 56163,
34058, 11969

------
Kind Regards,
Peter Smith.
Fujitsu Australia