RE: Proposal: Deferred Replica Filtering for PostgreSQL Logical Replication

From: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>
To: Dean <ds(dot)blue797(at)gmail(dot)com>
Cc: "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Subject: RE: Proposal: Deferred Replica Filtering for PostgreSQL Logical Replication
Date: 2025-03-19 02:00:27
Message-ID: OS0PR01MB5716E377AC9EFF613995F0B194D92@OS0PR01MB5716.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 2025/03/19 10:00:00 </o=ExchangeLabs/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Recipients/cn=3da34e28d9d240d4abadbbb549f9ff21-houzj.fnst> wrote:
On Wed, Mar 19, 2025 at 8:56 AM Dean wrote:
> Unfortunately, neither column lists nor row filters can provide the level of
> control I'm proposing. These revised examples might help illustrate the use
> case for DRF:
>
> Alice, Bob, and Eve subscribe to changes on a `friend_requests` table.
> Row-level security ensures CRUD access based on user IDs.
>
> 1. Per-subscriber column control: Bob makes a change to the table. Alice
> should receive the entire record, while Eve should only receive the
> timestamp - no other columns. Why DRF is needed: Column lists are static
> and apply equally to all subscribers, meaning we can't distinguish Alice's
> subscription from Eve's.

I would like to confirm the concept to ensure I understand it correctly. If
your goal is for the two subscriptions, Alice and Eve, to utilize different
column lists, you can achieve this by creating two separate publications:
Alice_pub and Eve_pub. You can specify the desired column list for the table in
Eve_pub. Consequently, the subscription Alice will subscribe to Alice_pub,
receiving the full record, while Eve will subscribe to Eve_pub, receiving a
limited set of columns. If this does not fully meet your requirements, could
you please provide additional details or clarify your specific needs ?

> 2. Bob DELETEs a row from the table. Alice should see the DELETE event, while
> Eve should not even be aware of an event. Why DRF is needed: The
> deterministic nature of row filters makes them unsuitable for
> per-subscriber filtering based on session data.
>
>
> The goal of DRF is to allow per-subscriber variations in change broadcasts,
> enabling granular control over what data is sent to each subscriber based on
> their session context.

Same as above, it would be helpful to elaborate a bit more.

Best Regards,
Hou zj

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Euler Taveira 2025-03-19 02:01:02 Re: Proposal: Deferred Replica Filtering for PostgreSQL Logical Replication
Previous Message Xuneng Zhou 2025-03-19 01:53:16 Re: Fwd: [BUG]: the walsender does not update its IO statistics until it exits