Re: Hot standby 9.2.6 -> 9.2.6 PANIC: WAL contains references to invalid pages

Hi,

On 2013-12-19 14:37:04 -0800, Sergey Konoplev wrote:
> 2013-12-19 20:51:22 MSK 19938 @ from [vxid:1/0 txid:0] [] WARNING:
> page 14833 of relation base/16436/3321003988 is uninitialized
> 2013-12-19 20:51:22 MSK 19938 @ from [vxid:1/0 txid:0] [] CONTEXT:
> xlog redo vacuum: rel 1663/16436/3321003988; blk 38538,
> lastBlockVacuumed 0
> 2013-12-19 20:51:22 MSK 19938 @ from [vxid:1/0 txid:0] [] PANIC: WAL
> contains references to invalid pages
> 2013-12-19 20:51:22 MSK 19938 @ from [vxid:1/0 txid:0] [] CONTEXT:
> xlog redo vacuum: rel 1663/16436/3321003988; blk 38538,
> lastBlockVacuumed 0
> 2013-12-19 20:51:22 MSK 19935 @ from [vxid: txid:0] [] LOG: startup
> process (PID 19938) was terminated by signal 6: Aborted
> 2013-12-19 20:51:22 MSK 19935 @ from [vxid: txid:0] [] LOG:
> terminating any other active server processes

There just was another case of this reported on IRC by MatheusOl and for
some reason in his case I noticed the pertinent details and it quickly
clicked:
* page 14833 is the one with the error
* we're actually vacuuming page 38538
* lastBlockVacuumed is 0

In btree_xlog_vacuum() we scan all the pages between lastBlockVacuumed
and the page vacuumed and acquire a cleanup lock on it. But there isn't
any guarantee that the intermediate pages are valid, filled pages,
afaics. ISTM we can just use RBM_ZERO_ON_ERROR instead of RBM_NORMAL.

Alternatively we could devise a trivial routine that just does a
BufTableLookup() and the required hangups around that. But that seems
more invasive than necessary.

Neither makes me really happy, but I don't see a nicer solution right
now.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services