Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

On Thu, Feb 20, 2025 at 3:08 PM Hayato Kuroda (Fujitsu)
<kuroda(dot)hayato(at)fujitsu(dot)com> wrote:
>
> Dear Ajin,
>
> > I compared the patch 1 which does not employ a hash cache and has the
> > overhead of starting a transaction every time the filter is checked.
> >
> > I created a test setup of 10 million inserts in 3 different scenarios:
> > 1. All inserts on unpublished tables
> > 2. Half of the inserts on unpublished table and half on pupblished table
> > 3. All inserts on published tables.
> >
> > The percentage improvement in the new optimized patch compared to the
> > old patch is:
> >
> > No transactions in publication: 85.39% improvement
> > Half transactions in publication: 72.70% improvement
> > All transactions in publication: 48.47% improvement
> >
> > Attaching a graph to show the difference.
>
> I could not find any comparisons with HEAD. Can you clarify the throughput/latency/memory
> usage with HEAD?

Here's the difference in latency with head. Again 10 million inserts
in 3 scenarios: All transactions on unpublished tables, half of the
transactions on unpublished tables and all transactions on published
tables
Conclusion:
The patched code with 100 transaction throttling significantly
improves performance, reducing execution time by ~69% when no
published transactions are involved, ~43% with partial published
transactions, and ~15% in all published transactions.
Attaching a graph showing the performance differences.

I will run tests comparing memory and throughput as well.

regards,
Ajin Cherian
Fujitsu Australia