Re: Enabling Checksums

From: Simon Riggs <simon(at)2ndQuadrant(dot)com>
To: Ants Aasma <ants(at)cybertec(dot)at>
Cc: Jeff Davis <pgsql(at)j-davis(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, Greg Smith <greg(at)2ndquadrant(dot)com>, Heikki Linnakangas <hlinnakangas(at)vmware(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Enabling Checksums
Date: 2013-04-10 19:17:08
Message-ID: CA+U5nM+kty+8_1MX6Y0FjM=6-MLcj9McSz69O5znFaEzXzytxQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 10 April 2013 11:15, Ants Aasma <ants(at)cybertec(dot)at> wrote:

> > * We might even be able to calculate CRC32 checksum for normal WAL
> records,
> > and use Ants' checksum for full page writes (only). So checking WAL
> checksum
> > would then be to confirm header passes CRC32 and then re-check the Ants
> > checksum of each backup block.
>
> If we ensure that the checksum on the page is correct when we do a
> full page write then we could only include the checksum field in the
> WAL CRC. When reading WAL we would first check that the CRC is correct
> and then verify the the page checksum.

OK, so we have a single combined "calculate a checksum for a block"
function. That uses Jeff's zeroing trick and Ants' bulk-oriented
performance optimization.

For buffer checksums we simply calculate for the block.

For WAL full page writes, we first set the checksums for all defined
buffers, then calculate the checksum of remaining data plus the pd_checksum
field from each block using the normal WAL CRC32.

Seems good to me. One set of fast code. And it avoids the weirdness that
the checksum stored on the full page is actually wrong.

It also means that the WAL checksum calculation includes the hole, yet we
do not include the data for the hole. So we have to do an extra copy when
restoring the backuo block.

Comments?

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2013-04-10 19:59:13 Re: [GSOC] questions about idea "rewrite pg_dump as library"
Previous Message Alvaro Herrera 2013-04-10 19:11:52 Re: SIGHUP not received by custom bgworkers if postmaster is notified