Proposal: Deferred Replica Filtering for PostgreSQL Logical Replication

From: Dean <ds(dot)blue797(at)gmail(dot)com>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Cc: Dean <ds(dot)blue797(at)gmail(dot)com>
Subject: Proposal: Deferred Replica Filtering for PostgreSQL Logical Replication
Date: 2025-03-15 19:28:50
Message-ID: CALWmXttn=GUwmDeUOKapL-EU4ax2Wutr0OgCG2kAZcLgep1MZQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi hackers,

I'd like to propose an enhancement to PostgreSQL's logical replication
system: Deferred Replica Filtering (DRF). The goal of this feature is to
provide more granular control over which rows are replicated by applying
publication filters *after* the WAL decoding process, before sending data
to subscribers.

Currently, PostgreSQL's logical replication filters apply
deterministically. Deferred filtering, however, operates after the WAL has
been decoded, giving it access to the complete row data and making
filtering decisions based on mutable values. Additionally, record columns
may be omitted by the filter.

This opens up several possibilities for granular control. Consider the
following examples:
Alice and Bob subscribe to changes on a table with RLS enabled, allowing
CRUD operations based on user's IDs.
1. Alice needs to know the timestamp at which Bob updated the table. With
DRF, we can omit all columns except for the timestamp.
2. Bob wants to track DELETEs on the table. Without DRF, Bob can see all
columns on any deleted row, potentially exposing complete records he
shouldn't be authorized to view. DRF can filter these rows out.

Deferred replica filtering allows for session-specific, per-row, and
per-column filtering - features currently not supported by existing
replication filters, enhancing security and data privacy.

I look forward to hearing your thoughts!

Best,
Dean S

Browse pgsql-hackers by date

  From Date Subject
Next Message Jeff Davis 2025-03-15 20:07:44 Re: Optimization for lower(), upper(), casefold() functions.
Previous Message Jeff Davis 2025-03-15 17:49:44 Re: Update Unicode data to Unicode 16.0.0