Re: Missing rows after migrating from postgres 11 to 12 with logical replication

On 12/21/20 12:26 PM, Lars Vonk wrote:
> Hi Adrian,
>
> Thanks for taking the time to reply!

>
> 2) Are the logs still available for inspection to see if there where
> any
> errors thrown?
>
>
> Yes, and we dug into those. And we also found some indications that
> something went wrong.
>

> 4) How did you determine the rows where missing?
>
>
> We were alerted by a bug later that day and found that some rows were
> missing in the new primary. We did a compare based on primary key and
> found that several tables were missing rows. Before the switch we
> unfortunately only checked max(id) and did some counts on tables and
> those all checked out. We didn't do a count on all tables...
>
> So to come back at the logs:
>
> We dug a little deeper and we did found ERROR logs around the time we
> ran the initial copies. During a period of several hours that day we see
> a couple of messages like:
>
> ERROR: requested WAL segment 00000001000001F10000001D has already
> been removed

What was being run when the above ERROR was triggered?

>
> Regards,
> Lars
>
> On Sun, Dec 20, 2020 at 6:58 PM Adrian Klaver <adrian(dot)klaver(at)aklaver(dot)com

--
Adrian Klaver
adrian(dot)klaver(at)aklaver(dot)com