Re: Enabling Checksums

On 10 April 2013 09:01, Ants Aasma <ants(at)cybertec(dot)at> wrote:

>
> > Using SIMD for WAL is not a requirement at all; I just thought it might
> > be a nice benefit for non-checksum-enabled users in some later release.
>
> I think we should first deal with using it for page checksums and if
> future versions want to reuse some of the code for WAL checksums then
> we can rearrange the code.

We have essentially the same problem in both cases: we want to calculate a
checksum of BLCKSZ chunks of data, plus some smaller header data. We
currently use the same code for both cases and it makes sense to do the
same thing with any new code. This is also the *same* issue: when we make a
new hint we need to issue a full page write in WAL, so we are calculating
checksums in two new places: XLOG_HINT records and data blocks.

Few technical points:

* We're taking a copy of the buffer, so Jeff's zero trick works safely, I
think.
* We can use a different algorithm for big and small blocks, we just need a
way to show we've done that, for example setting the high order bit of the
checksum.
* We might even be able to calculate CRC32 checksum for normal WAL records,
and use Ants' checksum for full page writes (only). So checking WAL
checksum would then be to confirm header passes CRC32 and then re-check the
Ants checksum of each backup block.

This work needs to happen now, since once the checksum algorithm is set we
won't easily be able to change it.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services