Re: Logical replication timeout

On Fri, 27 Dec 2024 at 09:41, vignesh C <vignesh21(at)gmail(dot)com> wrote:
>
> On Wed, 25 Dec 2024 at 13:55, Hayato Kuroda (Fujitsu)
> <kuroda(dot)hayato(at)fujitsu(dot)com> wrote:
> >
> > Dear Marc,
> >
> > > Thanks again for this new patch.
> > >
> > > Unfortunately it does not compile (17.2 source):
> >
> > Right, because of the reason I posted [1].
> >
> > I updated the patch which did the same approach. It could pass my CI.
>
> Let's conduct some performance tests with varying numbers of spill
> files (e.g., small ones like 1, 5, and 10, and larger ones like 100,
> 1000, and 10,000) along with different levels of concurrent
> transactions. We can then compare the results with the current HEAD.
>

Hi Vignesh,
I did the performance testing with the patch and compared the time
taken for Cleanup of spill files with HEAD and with Patch. Following
are the results:
(All timing are average of 3 runs)

No. of Spill Files | Head (nano sec) | Avg Patch (nano sec) | No. to
times Patch is faster than HEAD
--------------------------------------------------------------------------------------------------------------------------------------
10000 | 68057452 | 442923.3333
| 153.6551518
1000 | 737519.6667 | 46049.33333
| 16.0158598
500 | 199108 | 19401.66667
| 10.26241732
100 | 12566.66667 | 3921
| 3.20496472
10 | 576 | 466.3333333
| 1.235167977
1 | 60 | 56.33333333
| 1.065088757

I have made a patch to log the time for cleanup for the transaction. I
have attached the test perl script here as well.
To get the time to clean up the spill files, I run the perl script and
then check the publisher log for the time of cleanup of the very first
transaction.

Thanks and Regards,
Shlok Kyal