RE: Selectively invalidate caches in pgoutput module

From: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>
To: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
Cc: "ShlokKumar(dot)Kyal(at)fujitsu(dot)com" <ShlokKumar(dot)Kyal(at)fujitsu(dot)com>, 'Amit Kapila' <amit(dot)kapila16(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: RE: Selectively invalidate caches in pgoutput module
Date: 2025-03-10 11:07:23
Message-ID: OS3PR01MB5718B8EFFBB856C8B5749E8B94D62@OS3PR01MB5718.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Monday, March 10, 2025 7:00 PM Kuroda, Hayato <kuroda(dot)hayato(at)fujitsu(dot)com> wrote:
>
> I did a self-reviewing and updated a patch. PSA new version. What's new:

Thanks for updating the patch.

I tested the behavior for partitioned table and have a comment on this.

> + relids = GetPublicationRelations(pubform->oid,
> + PUBLICATION_PART_LEAF);

Currently, only the leaf partition is invalidated when the published table is
partitioned. However, I think pgoutput could cache both the partitioned table
and the leaf partition table as relsync entries.

For INSERT/UPDATE/DELETE on a partitioned table, only the leaf partition's
relsync entry is used in pgoutput, but the TRUNCATE references the parent
table's relsync entry.

For example[1], if the parent table's relsync entry is not invalidated after a
RENAME operation, it results in the TRUNCATE to be missed. So I think we should
Invalidate all the tables in the partition tree by passing
PUBLICATION_PART_ALL, which is also consistent with ALTER PUB ADD/SET/DROP
TABLE.

[1]--- Example ---
create table test(a int primary key) PARTITION BY RANGE (a);
CREATE TABLE test_1 PARTITION OF test
FOR VALUES FROM (0) TO (2);
CREATE TABLE test_2 PARTITION OF test
FOR VALUES FROM (2) TO (4);

CREATE PUBLICATION pub;
CREATE PUBLICATION pub2 FOR TABLE test WITH (PUBLISH_VIA_PARTITION_ROOT);

SELECT 'init' FROM pg_create_logical_replication_slot('isolation_slot', 'pgoutput');

TRUNCATE test;
ALTER PUBLICATION pub RENAME TO pub3;
ALTER PUBLICATION pub2 RENAME TO pub;
TRUNCATE test;

-- I can consume some changes using the following function on HEAD, but got
-- nothing after applying the patch.
SELECT * FROM pg_logical_slot_get_binary_changes('isolation_slot', NULL, NULL, 'proto_version', '4', 'publication_names', 'pub', 'streaming', 'on');

---

Best Regards,
Hou zj

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Álvaro Herrera 2025-03-10 11:12:33 Re: pgsql: reindexdb: Add the index-level REINDEX with multiple jobs
Previous Message Ilia Evdokimov 2025-03-10 11:01:41 Re: track generic and custom plans in pg_stat_statements