Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

From: Ajin Cherian <itsajin(at)gmail(dot)com>
To: li jie <ggysxcq(at)gmail(dot)com>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding
Date: 2024-03-19 02:49:57
Message-ID: CAFPTHDaLCL-CkrHeP5NEik4GFwVPEm8rewcvPNruwjHSaF1gNw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

>
> Of course you can, but this will only convert disk space into memory space.
> For details, please see the case in Email [1].
>
> [1]
> https://www.postgresql.org/message-id/CAGfChW51P944nM5h0HTV9HistvVfwBxNaMt_s-OZ9t%3DuXz%2BZbg%40mail.gmail.com
>
> Regards, lijie
>
>
Hi lijie,

Overall, I think the patch is a good improvement. Some comments from first
run through of patch:
1. The patch no longer applies cleanly, please rebase.

2. While testing the patch, I saw something strange. If I try to truncate a
table that is published. I still see the message:
2024-03-18 22:25:51.243 EDT [29385] LOG: logical filter change by table
pg_class

This gives the impression that the truncate operation on the published
table has been filtered but it hasn't. Also the log message needs to be
reworded. Maybe, "Logical filtering change by non-published table
<relation_name>"

3. Below code:
@@ -1201,11 +1343,14 @@ DecodeMultiInsert(LogicalDecodingContext *ctx,
XLogRecordBuffer *buf)
+
+ if (FilterByTable(ctx, change))
+ continue;;

extra semi-colon after continue.

4. I am not sure if this is possible, but is there a way to avoid the
overhead in the patch if the publication publishes "ALL TABLES"?

5. In function: pgoutput_table_filter() - this code appears to be filtering
out not just unpublished tables but also applying row based filters on
published tables as well. Is this really within the scope of the feature?

regards,
Ajin Cherian
Fujitsu Australia

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message John Naylor 2024-03-19 03:03:36 Re: add AVX2 support to simd.h
Previous Message Richard Guo 2024-03-19 02:47:49 Re: A problem about partitionwise join