Re: Re: [RFC][PATCH]: CRC32 is limiting at COPY/CTAS/INSERT ... SELECT + speeding it up

Florian Pflug wrote:
> On Jun 7, 2010, at 12:45 , Andres Freund wrote:
>
>> On Monday 07 June 2010 12:37:13 Pierre C wrote:
>>
>>>> On Sunday 30 May 2010 18:29:31 Greg Stark wrote:
>>>>
>>>>> On Sun, May 30, 2010 at 4:54 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>>>>
>>>>>> I read through that thread and couldn't find much discussion of
>>>>>> alternative CRC implementations --- we spent all our time on arguing
>>>>>> about whether we needed 64-bit CRC or not.
>>>>>>
>>> SSE4.2 has a hardware CRC32 instruction, this might be interesting to
>>> use...
>>>
>> Different polynom unfortunately...
>>
>
> Since only the WAL uses CRC, I guess the polynomial could be changed though. pg_upgrade for example shouldn't care.
>
> RFC3385 compares different checksumming methods for use in iSCSI, and CRC32c (which uses the same polynomial as the SSE4.2 instruction) wins. Here's
> a link: http://www.faqs.org/rfcs/rfc3385.html
>
The linux kernel also uses it when it's availabe, see e.g.
http://tomoyo.sourceforge.jp/cgi-bin/lxr/source/arch/x86/crypto/crc32c-intel.c

regards,
Yeb Havinga