Re: Enabling Checksums

On Tue, 2013-04-09 at 05:35 +0300, Ants Aasma wrote:
> And here you go. I decided to be verbose with the comments as it's
> easier to delete a comment to write one. I also left in a huge jumble
> of macros to calculate the contents of a helper var during compile
> time. This can easily be replaced with the calculated values once we
> settle on specific parameters.

Great, thank you.

Is it possible to put an interface over it that somewhat resembles the
CRC checksum (INIT/COMP/FIN)? It looks a little challenging because of
the nature of the algorithm, but it would make it easier to extend to
other places (e.g. WAL). It doesn't have to match the INIT/COMP/FIN
pattern exactly.

Regardless, we should have some kind of fairly generic interface and
move the code to its own file (e.g. checksum.c).

To make the interface more generic, would it make sense to require the
caller to save the page's stored checksum and zero it before
calculating? That would avoid the awkwardness of avoiding the
pd_checksum field. For example (code for illustration only):

PageCalcChecksum16(Page page, BlockNumber blkno)
{
PageHeader phdr = (PageHeader)page;
uint16 stored_checksum = phdr->pd_checksum;
uint16 calc_checksum;

phdr->pd_checksum = 0;
calc_checksum = SIMD_CHECKSUM(page, BLCKSZ);
phdr->pd_checksum = stored_checksum;
return calc_checksum;
}

That would make it possible to use a different word size -- is uint16
optimal or would a larger word be more efficient?

It looks like the block size needs to be an even multiple of
sizeof(uint16)*NSUMS. And it also look like it's hard to combine
different regions of memory into the same calculation (unless we want to
just calculate them separately and XOR them or something). Does that
mean that this is not suitable for WAL at all?

Using SIMD for WAL is not a requirement at all; I just thought it might
be a nice benefit for non-checksum-enabled users in some later release.

Regards,
Jeff Davis