Re: Bug in visibility map WAL-logging

On 2014-01-07 21:36:31 +0200, Heikki Linnakangas wrote:
> >2088220 ... Heap2 ... 24/ 192, ... lsn: 2BC/46AB8B20 ... desc: clean:
> >rel 1663/883916/151040222; blk 1073 remxid 107409880
> >2088221 ... Heap2 ... 20/ 52, ... lsn: 2BC/46AB8BE0 ... desc: visible:
> >rel 1663/883916/151040222; blk 1074
> >2088222 ... Heap2 ... 24/ 128, ... lsn: 2BC/46AB8C18 ... desc: clean:
> >rel 1663/883916/151040222; blk 1074 remxid 107409880
> > ...
>
> Hmm. The xlogdump indeed shows that the order of 'clean' and 'visi ble' is
> incorrect, but I don't immediately see how that could cause the PANIC. Why
> is the page uninitialized in the standby? If VACUUM is removing some dead
> tuples from it, it certainly should exist and be correctly initialized.

Yea, that's strange. I first thought it might be the PageIsNew() branch
in lazy_scan_heap(). That initializes the page without WAL logging which
would explain it being uninitialized on the standby. But that wouldn't
explain why we found something to clean on the primary while the page is
still empty on the standby...

> How did you set up the standby? Did you initialize it from an offline backup
> of the master's data directory, perhaps? The log shows that the startup took
> the the "crash recovery first, then start archive recovery" path, because
> there was no backup label file

Good point.

>In particular, I have to ask because I've seen it before: you
> didn't delete backup_label from the backup, did you?

I have seen that repeatedly too.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services