Re: Enabling Checksums

From: Florian Pflug <fgp(at)phlo(dot)org>
To: Ants Aasma <ants(at)cybertec(dot)at>
Cc: Andres Freund <andres(at)2ndquadrant(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Bruce Momjian <bruce(at)momjian(dot)us>, Jeff Davis <pgsql(at)j-davis(dot)com>, Heikki Linnakangas <hlinnakangas(at)vmware(dot)com>, Simon Riggs <simon(at)2ndquadrant(dot)com>, Greg Smith <greg(at)2ndquadrant(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Enabling Checksums
Date: 2013-04-18 18:11:28
Message-ID: 730D0717-66D8-4096-A60E-2D0055A34CCA@phlo.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 18.04.2013, at 20:02, Ants Aasma <ants(at)cybertec(dot)at> wrote:
> On Thu, Apr 18, 2013 at 8:24 PM, Ants Aasma <ants(at)cybertec(dot)at> wrote:
>> On Thu, Apr 18, 2013 at 8:15 PM, Florian Pflug <fgp(at)phlo(dot)org> wrote:
>>> So either the CRC32-C polynomial isn't irreducible, or there something
>>> fishy going on. Could there be a bug in your CRC implementation? Maybe
>>> a mixup between big and little endian, or something like that?
>>
>> I'm suspecting an implementation bug myself. I already checked the
>> test harness and that was all sane, compiler hadn't taken any
>> unforgivable liberties there. I will crosscheck the output with other
>> implementations to verify that the checksum is implemented correctly.
>
> Looks like the implementation is correct. I cross-referenced it
> against a bitwise algorithm for crc32 with the castagnoli polynomial.
> This also rules out any endianness issues as the bitwise variant
> consumes input byte at a time.
>
> What ever it is, it is something specific to PostgreSQL page layout.
> If I use /dev/urandom as the source the issue disappears. So much for
> CRC32 being proven good.

Weird. Is the code of your test harness available publicly, or could you post it? I'd like to look into this...

best regard,
Florian Pflug

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Heikki Linnakangas 2013-04-18 18:11:29 Recovery target 'immediate'
Previous Message Ants Aasma 2013-04-18 18:02:49 Re: Enabling Checksums