Re: [GENERAL] Undetected corruption of table files

Tom Lane wrote:
>>>> Would it be an option to have a checksum somewhere in each
>>>> data block that is verified upon read?
>
>>> That's been proposed before and rejected before. See the
>>> archives ...
>
> I think
> the prior discussions were around the same time WAL was initially put
> in, and/or when we dropped the WAL CRC width from 64 to 32 bits.
> The very measurable overhead of WAL CRCs are the main thing that's
> discouraged us from having page CRCs. (Well, that and the lack of
> evidence that they'd actually gain anything.)

Hmmm - silence me if I'm misunderstanding this, but the most
conclusive hit I had was a mail by you:

http://archives.postgresql.org/pgsql-general/2001-10/msg01142.php

which only got affirmative feedback.

Also, there's a TODO entry:

- Add optional CRC checksum to heap and index pages

This seems to me to be exactly what I wish for...

To the best of my knowledge, the most expensive thing in databases
today is disk I/O, because CPU speed is increasing faster. Although
calculating a checksum upon writing a block to disk will
certainly incur CPU overhead, what may have seemed too expensive
a couple of years ago could be acceptable today.

I understand the argument that it's the task of hardware and
OS to see that data don't get corrupted, but it would improve
PostgreSQL's reliabitity if it can detect such errors and at
least issue a warning.
This wouldn't fix the underlying problem, but it would tell you
to not overwrite last week's backup tape...

Not all databases are on enterprise scale storage systems, and
there's also the small possibility of PostgreSQL bugs that could
be detected that way.

Yours,
Laurenz Albe