Recovering data from table show corruption with "invalid page header in block X"

From: Erik Jones <ejones(at)engineyard(dot)com>
To: pgsql-general(at)postgresql(dot)org
Subject: Recovering data from table show corruption with "invalid page header in block X"
Date: 2010-02-09 23:28:14
Message-ID: 8A324DF3-9D5C-48A2-9A93-29CBCBDE06A8@engineyard.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Greetings,

We've recently had database server crash due to a heavy duty disk failure and upon rebooting we now have a table showing corruption via the "invalid page header in block X" message when querying one table in particular, the rest have all checked out via full table dumps with no errors.

Here is what we've done so far to try to fix the table and/or recover the good data from it:

* REINDEX -- no good.

* Set zero_damaged_pages=on, run query that originally showed the corruption. This reports 3 different blocks with invalid page headers and reports that they are being zero'd out. Unfortunately, subsequently querying the table the same blocks show as corrupt. Well, after running the query twice with zero_damaged_pages=on the first one did go away but the other two remain.

* Set zero_damaged_pages=on for pg_dump. Same results as above.

I'm unsure what to do next to get those damaged pages cleared out...

Erik Jones, Database Administrator
Engine Yard
Support, Scalability, Reliability
866.518.9273 x 260
Location: US/Pacific
IRC: mage2k

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Erik Jones 2010-02-09 23:54:11 Re: Recovering data from table show corruption with "invalid page header in block X"
Previous Message Justin Graf 2010-02-09 23:05:21 Re: Best way to handle multi-billion row read-only table?