Re: Hot standby 9.2.6 -> 9.2.6 PANIC: WAL contains references to invalid pages

On 2014-01-06 16:35:42 +0200, Heikki Linnakangas wrote:
> >In btree_xlog_vacuum() we scan all the pages between lastBlockVacuumed
> >and the page vacuumed and acquire a cleanup lock on it. But there isn't
> >any guarantee that the intermediate pages are valid, filled pages,
> >afaics.
>
> Hmm. So the problem arises if there's an uninitialized page in the middle of
> the b-tree relation for some reason. It's unusual for an uninitialized page
> to be left in the middle of the relation, but it's certainly possible, if
> e.g you crash just after extending the relation. In a heap, vacuum will
> initialize such pages and emit a WARNING like "page %u is uninitialized ---
> fixing", but we don't do that for b-tree.

Yes. There's probably some resource exhaustion scenarios that can cause
this as well. Triggering *_timeout or out-of-memory errors at least...

> >ISTM we can just use RBM_ZERO_ON_ERROR instead of RBM_NORMAL.
>
> That'd be horrendously dangerous. It would silently zap any page with any
> error on it.

Well, the number of scenarios in which you can hit that is somewhat
limited - it's not going to be ENOSPC since the file is guaranteed to be
longer, and we don't create holes - so the scenario pretty much is disk
reads not returning (pretty rare) and checksums failing.

> But we could add a new ReadBufferMode that returns
> InvalidBuffer on error, without zeroing the page.

But yes, that's much better.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services