Re: vacuumdb: PANIC: corrupted item pointer

Richard Huxton <dev(at)archonet(dot)com> writes:
> Alain Peyrat wrote:
>> Initial problem:
>>
>> # pg_dump -O dbname -Ft -f /tmp/database.tar
>> pg_dump: query to get table columns failed: ERROR: invalid memory alloc
>> request size 9000688640
>>
>> After some research, it seems to be related to a corruption of the
>> database. Running a vacum crashes the db:
>>
>> -bash-3.00$ vacuumdb -z -a
>> vacuumdb: vacuuming database "template1"
>> vacuumdb: vacuuming of database "template1" failed: PANIC: corrupted
>> item pointer: offset = 3336, size = 20

> It would be nice if it's just template1 that is damaged, but I'm not
> sure that's the case.

This looks pretty bad, because the above already proves corruption in two
different databases --- there is something wrong somewhere in template1,
and something else wrong somewhere in "dbname" (unless that is actually
template1). It seems likely that there's been widespread damage from
some hardware or filesystem-level misfortune.

FWIW, a look in the source code shows that the 'corrupted item pointer'
message comes only from PageIndexTupleDelete, so that indicates a
damaged index which should be fixable by reindexing. But the other one
looks more like heap damage, since it's apparently trying to copy a
damaged variable-width field and I don't think index entries get copied
anywhere in a normal query. The query pg_dump is complaining about
looks at pg_attribute and pg_type entries for a particular table ---
hmm, I notice that it's not fetching any variable-width fields, so it's
not entirely clear where the error is coming from. If you are really
lucky it could be that the corruption is actually in pg_statistic,
and it's failing when the planner tries to estimate row counts for the
query. That would be really lucky because that's all discardable data.
Try "DELETE FROM pg_statistic" as superuser and see if you can dump
then.

> 1. Have you had crashes or other hardware problems recently?

Indeed.

regards, tom lane