Re: strange nbtree corruption report

Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> writes:
> We got a very strange nbtree corruption report some time ago. This was
> a btree index on a vey high churn table -- entries are updated and
> deleted very quickly, so the index grows very large and also shrinks
> quickly (AFAICT this is a work queue of sorts).

> The most strange thing of all is that there was this error:

> ERROR: left link changed unexpectedly in block 3378 of index "index_name"
> CONTEXT: automatic vacuum of table "table_name"

> This was reported not once, but several dozens of times, by each new
> autovacuum worker that tried to vacuum the table.

> As far as I can see, there is just no way for this to happen ... much
> less happen repeatedly.

It's not hard to believe that that would happen repeatedly given a
corrupted set of sibling links, eg deletable page A links left to page
B, which links right to C, which links right to A. The question is how
the index got into such a state. A dropped update during a page split
would explain it (ie, B used to be A's left sibling, then at some point
B got split into B and C, but A's left-link never got updated on disk).
I wonder how reliable their disk+filesystem is ...

regards, tom lane