Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

From: Ajin Cherian <itsajin(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding
Date: 2025-02-25 04:43:26
Message-ID: CAFPTHDad=XG1vbTSJ1wwVD2vnuSxfqUFc2mimC8=KYaMoTO4kQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Feb 25, 2025 at 3:26 PM Ajin Cherian <itsajin(at)gmail(dot)com> wrote:
>
> On Fri, Feb 21, 2025 at 2:24 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Fri, Feb 21, 2025 at 7:57 AM Ajin Cherian <itsajin(at)gmail(dot)com> wrote:
> > > In these tests, I also see an increased performance with the patch
> > > even when all transactions are published. I will investigate why this
> > > happens and update.
> > >
> >
> > Yes, it is important to investigate this because in the best case, it
> > should match with HEAD. One thing you can verify is whether the
> > changes processed on the server are exactly for the published table,
> > it shouldn't happen that it is processing both published and
> > unpublished changes. If the server is processing for both tables then
> > it is expected that the patch performs better. I think you can verify
> > before starting each test and after finishing each test whether the
> > slot is pointing at the appropriate location for the next test or
> > create a new slot for each with the required location.
>
> Yes, you are right, I modified the tests to drop the slot and create a
> new slot advance to current_lsn and now I see a fractionally better
> performance in head code when all transactions are published.
> Graph attached.
>

Just to summarize and clarify:

The patched code significantly improves performance when no
transactions are published, reducing execution time from ~54,100 ms
(head code) to ~17,700 ms—a nearly 70% improvement.
When half of the transactions are published, the patched code also
shows a notable performance gain, reducing execution time from ~62,800
ms to ~44,500 ms (~29% faster).
When all transactions are published, the patched code is only 0.53%
slower than the head code, indicating a negligible performance
degradation.

regards,
Ajin Cherian
Fujitsu Australia

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Kyotaro Horiguchi 2025-02-25 05:09:53 Fix untranslatable split message
Previous Message Ashutosh Bapat 2025-02-25 04:33:07 Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints