Re: What exactly is our CRC algorithm?

From: Abhijit Menon-Sen <ams(at)2ndQuadrant(dot)com>
To: Heikki Linnakangas <hlinnakangas(at)vmware(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org, Andres Freund <andres(at)2ndquadrant(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>
Subject: Re: What exactly is our CRC algorithm?
Date: 2014-12-30 16:06:19
Message-ID: 20141230160619.GA6441@toroid.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

At 2014-12-30 16:05:50 +0200, hlinnakangas(at)vmware(dot)com wrote:
>
> A couple of quick comments:
>
> bswap32 is unused on on little-endian systems. That will give a
> compiler warning.

Huh. I don't get a warning, even when I add -Wunused to the build flags.
But since you mention it, it would be better to write the function thus:

static inline uint32 cpu_to_le32(uint32 x)
{
#ifndef WORDS_BIGENDIAN
return x;
#elif defined(__GNUC__) || defined(__clang__)
return __builtin_bswap32(x);
#else
return ((x << 24) & 0xff000000) |
((x << 8) & 0x00ff0000) |
((x >> 8) & 0x0000ff00) |
((x >> 24) & 0x000000ff);
#endif
}

> pg_comp_crc32c_sse […] fetches the 8-byte chunks from only 4-byte
> aligned addresses. Is that intentional?

Thanks for spotting that. I had meant to change the test to "& 7". But
again, now that you mention it, I'm not sure it's necessary. The CRC32*
instructions don't have the usual SSE alignment requirements, and I see
that the Linux kernel (among other implementations) process eight bytes
at a time from the start of the buffer and then process the remaining
bytes one at a time. I'll do a bit more research and post an update.

Thanks for having a look.

-- Abhijit

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Merlin Moncure 2014-12-30 16:15:15 Re: BUG #12330: ACID is broken for unique constraints
Previous Message Adrian Klaver 2014-12-30 15:54:04 Re: [HACKERS] ON_ERROR_ROLLBACK