Re: speed up a logical replica setup

From: vignesh C <vignesh21(at)gmail(dot)com>
To: "Andrey M(dot) Borodin" <x4mmm(at)yandex-team(dot)ru>
Cc: Shlok Kyal <shlok(dot)kyal(dot)oss(at)gmail(dot)com>, Euler Taveira <euler(at)eulerto(dot)com>, "kuroda(dot)hayato(at)fujitsu(dot)com" <kuroda(dot)hayato(at)fujitsu(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Michael Paquier <michael(at)paquier(dot)xyz>, Peter Eisentraut <peter(at)eisentraut(dot)org>, Andres Freund <andres(at)anarazel(dot)de>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Fabrízio de Royes Mello <fabriziomello(at)gmail(dot)com>
Subject: Re: speed up a logical replica setup
Date: 2024-03-11 03:56:52
Message-ID: CALDaNm093n+XZsJ1_QqsYD9o-e1MG9xADHaw-u0GJ_ekshzPsg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, 8 Mar 2024 at 15:31, Andrey M. Borodin <x4mmm(at)yandex-team(dot)ru> wrote:
>
>
>
> > On 8 Mar 2024, at 12:03, Shlok Kyal <shlok(dot)kyal(dot)oss(at)gmail(dot)com> wrote:
> >
> > <v27-0004-Add-additional-testcases.patch><v27-0001-pg_createsubscriber-creates-a-new-logical-replic.patch><v27-0005-Fix-error-for-windows.patch><v27-0002-Use-latest-replication-slot-position-as-replicat.patch><v27-0003-port-replace-int-with-string.patch>
>
> I haven't digged into the thread, but recent version fails some CFbot's tests.
>
> http://commitfest.cputube.org/euler-taveira.html
> https://cirrus-ci.com/task/4833499115421696
> ==29928==ERROR: AddressSanitizer: heap-use-after-free on address 0x61a000001458 at pc 0x7f3b29fdedce bp 0x7ffe68fcf1c0 sp 0x7ffe68fcf1b8

This is because of disconnect_database called twice in the error flow:
+ PQclear(res);
+
+ disconnect_database(conn, false);
+
+ if (max_repslots < num_dbs)
+ {
+ pg_log_error("subscriber requires %d replication
slots, but only %d remain",
+ num_dbs, max_repslots);
+ pg_log_error_hint("Consider increasing
max_replication_slots to at least %d.",
+ num_dbs);
+ disconnect_database(conn, true);
+ }
+
+ if (max_lrworkers < num_dbs)
+ {
+ pg_log_error("subscriber requires %d logical
replication workers, but only %d remain",
+ num_dbs, max_lrworkers);
+ pg_log_error_hint("Consider increasing
max_logical_replication_workers to at least %d.",
+ num_dbs);
+ disconnect_database(conn, true);
+ }

This is handled in the attached patch set. Apart from this there are a
couple of test failures which will be handled in the upcoming version.

Regards,
Vignesh

Attachment Content-Type Size
v27-0005-Fix-error-for-windows.patch text/x-patch 1.1 KB
v27-0004-Add-additional-testcases.patch text/x-patch 8.5 KB
v27-0002-Use-latest-replication-slot-position-as-replicat.patch text/x-patch 6.1 KB
v27-0001-pg_createsubscriber-creates-a-new-logical-replic.patch text/x-patch 92.4 KB
v27-0003-port-replace-int-with-string.patch text/x-patch 3.8 KB
v27-0006-Fix-double-free-issue.patch text/x-patch 1.5 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message vignesh C 2024-03-11 04:12:06 Re: speed up a logical replica setup
Previous Message Anton A. Melnikov 2024-03-11 03:43:54 Re: May be BUG. Periodic burst growth of the checkpoint_req counter on replica.