RE: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

From: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>
To: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, 'Ajin Cherian' <itsajin(at)gmail(dot)com>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: RE: Proposal: Filter irrelevant change before reassemble transactions during logical decoding
Date: 2025-02-04 03:32:55
Message-ID: OS0PR01MB57160A7487F0E12B0BD043DB94F42@OS0PR01MB5716.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Friday, January 31, 2025 9:43 PM Kuroda, Hayato/黒田 隼人 <kuroda(dot)hayato(at)fujitsu(dot)com>
>
> Dear Ajin, Hou,
>
> > - Snapshot construction
>
> I understand the approach that we do not try to filter for all the workloads; do
> just best-effort.
>
> 3.
> Both ReorderBuffer and RelFileLocatorFilterCache have the same lifetime but
> RelFileLocatorFilterCache is provided as the global variable; it is quite strange.
> Can you somehow avoid this? I considered idea but could not because we do
> not have APIs to Unregister the relcacheCallback.

I think we already have precedent of this (e.g., RelationSyncCache in pgoutput.c),
so, I am OK for it unless we come up with better ideas.

>
> 4.
> For output plugins which does not cache the catalog, it may not able to do
> filtering without the transaction. For them, the early filter may degrade the
> performance because it requires to open transactions for every changes.
> Is my understanding correct?

Partially correct. To be precise, I think we should recommend that output
plugin that cannot filter without transaction can avoid implementing this
callback in the first place, so that it would not cause degradation.

Best Regards,
Hou zj

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2025-02-04 03:50:51 Re: injection points for hash aggregation
Previous Message vignesh C 2025-02-04 03:19:17 Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding