Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

From: Ajin Cherian <itsajin(at)gmail(dot)com>
To: Peter Smith <smithpb2250(at)gmail(dot)com>
Cc: vignesh C <vignesh21(at)gmail(dot)com>, "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding
Date: 2025-02-17 10:50:11
Message-ID: CAFPTHDZQmjwbhLTD10DO8dNXqE-5FhhF5VJeShiooOu=4JzEPg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Feb 17, 2025 at 10:08 AM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
>
> Hi Ajin.
>
> FYI - Patch set v13* no longer applies cleanly. Needs rebasing.
>
I've rebased the patch. I've also merged patch 1 into patch 2 as the
functionality of the changes in patch 1 are only n patch 2.

On Mon, Feb 17, 2025 at 10:08 AM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
>
> Hi Ajin.
>
> FYI - Patch set v13* no longer applies cleanly. Needs rebasing.
>
I've rebased the patch. I've also merged patch 1 into patch 2 as the
functionality of the changes in patch 1 are only in patch 2. So only 2
patches in this version.

On Fri, Feb 14, 2025 at 6:18 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> Again, can you share the link of performance data for this overhead
> and if you have not published then please share it and also mention it
> in commit message?

I will run the performance tests and post an update with the results.

> Can we try different thresholds for this like 10, 50, 100, 200, etc.
> to decide what is a good threshold value to skip filtering changes?
>
Ok, will do this. For this patch, I reset the count to 0, else the
test case fails as filtering could be skipped due to throttling. I
think we need a way for the user to set this threshold via a GUC and
that can be used for testing.

regards,
Ajin Cherian
Fujitsu Australia

Attachment Content-Type Size
v14-0001-Filter-transactions-that-need-not-be-published.patch application/octet-stream 24.5 KB
v14-0002-Introduce-a-output-plugin-callback-to-filter-cha.patch application/octet-stream 23.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message PG Bug reporting form 2025-02-17 10:52:32 BUG #18815: Logical replication worker Segmentation fault
Previous Message Amit Kapila 2025-02-17 10:36:34 Re: Log a warning in pg_createsubscriber for max_slot_wal_keep_size