Re: Data loss on logical replication, 12.12 to 14.5, ALTER SUBSCRIPTION

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Michail Nikolaev <michail(dot)nikolaev(at)gmail(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Data loss on logical replication, 12.12 to 14.5, ALTER SUBSCRIPTION
Date: 2022-12-27 04:57:55
Message-ID: CAA4eK1LweFTFs9Q2qK6ubkkJp+os1_TeX-4Rb1=q8t1jKXvZhw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Dec 26, 2022 at 8:50 PM Michail Nikolaev
<michail(dot)nikolaev(at)gmail(dot)com> wrote:
>
> Hello again.
>
> Just small a fix for:
>
> > 2022-12-14 09:21:25.705 to
> > 2022-12-14 09:49:20.664 (after synchronization start, but before finish).
>
> Correct values are:
>
> 2022-12-14 09:49:31.340
> 2022-12-14 09:49:41.683
>
> So, it looks like we lost about 10s of one of the tables WAL.
>

IIUC, this is the time when only table B's initial sync was
in-progress. Table A's initial sync was finished by that time and for
Table C, it is yet not started. We have some logic where in such
overlapping phases where we allow apply worker to skip the data which
will be later applied by the initial sync worker. During the time of
the initial sync of B, are there any other operations on that table
apart from the missing ones? You may want to see the LOGs of
subscribers during the initial sync time for any errors or other
messages.

--
With Regards,
Amit Kapila.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dilip Kumar 2022-12-27 05:06:29 Re: Perform streaming logical transactions by background workers and parallel apply
Previous Message wangw.fnst@fujitsu.com 2022-12-27 04:57:00 RE: Perform streaming logical transactions by background workers and parallel apply