Re: 12.3 replicas falling over during WAL redo

On 2020-Aug-03, Ben Chobot wrote:

> Alvaro Herrera wrote on 8/3/20 2:34 PM:
> > On 2020-Aug-03, Ben Chobot wrote:

> > dd if=16605/16613/60529051 bs=8192 count=1 seek=6501 of=/tmp/page.6501
>
> If I use skip instead of seek....

Argh, yes, I did correct that in my test and forgot to copy and paste.

>      lsn      | checksum | flags | lower | upper | special | pagesize |
> version | prune_xid
> --------------+----------+-------+-------+-------+---------+----------+---------+-----------
>  A0A/99BA11F8 |     -215 |     0 |   180 |  7240 |    8176 |     8192
> |       4 |         0
>
> As I understand what we're looking at, this means the WAL stream was
> assuming this page was last touched by A0A/AB2C43D0, but the page itself
> thinks it was last touched by A0A/99BA11F8, which means at least one write
> to the page is missing?

Yeah, that's exactly what we're seeing. Somehow an older page version
was resurrected. Of course, this should never happen.

So my theory has been proved. What now?

--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services