Re: Add an option to skip loading missing publication to avoid logical replication failure

From: vignesh C <vignesh21(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Add an option to skip loading missing publication to avoid logical replication failure
Date: 2025-02-18 08:54:23
Message-ID: CALDaNm2VTjKe8mQsTXFvrbgzcSO4OiuyOdw4VDBK_Ntn=ay-bQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, 14 Feb 2025 at 15:36, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> Did you try to measure the performance impact of this change? We can
> try a few cases where DDL and DMLs are involved, missing publication
> (drop publication and recreate after a varying number of records to
> check the impact).

Since we don't have an exact scenario to compare with the patch
(because, in the current HEAD, when the publication is missing, an
error is thrown and the walsender/worker restarts), I compared the
positive case, where records are successfully replicated to the
subscriber, as shown below. For the scenario with the patch, I ran the
same test, where the publication is dropped before the insert,
allowing the walsender to check whether the publication is present.
The test results, which represent the median of 7 runs and the
execution run is in milliseconds, are provided below:

Brach/records | 100 | 1000 | 10000 | 100000 | 1000000
Head | 1.214 | 2.548 | 10.823 | 90.3 | 951.833
Patch | 1.215 | 2.5485 | 10.8545 | 90.94 | 955.134
% diff | 0.082 | 0.020 | 0.291 | 0.704 | 0.347

I noticed that the test run with patches is very negligible. The
scripts used for execution are attached.

> The other names for the option could be:
> skip_notexistant_publications, or ignore_nonexistant_publications. Can
> we think of any others?

How about using ignore_missing_publications or skip_missing_publications?

Regards,
Vignesh

Attachment Content-Type Size
v3-0001-Skip-loading-the-publication-if-the-publication-d.patch application/octet-stream 3.0 KB
v3-0002-Added-an-option-skip_not_exist_publication-which-.patch application/octet-stream 65.8 KB
test_skip_pub_1702.sh text/x-sh 904 bytes
insert.sh text/x-sh 386 bytes

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Evgeny Voropaev 2025-02-18 09:02:42 Re: Elimination of the repetitive code at the SLRU bootstrap functions.
Previous Message Jelte Fennema-Nio 2025-02-18 08:51:12 Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs