Re: Cost of XLogInsert CRC calculations

> -----Original Message-----
> From: Tom Lane [mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us]
> Sent: 18 May 2005 06:12
> To: Greg Stark
> Cc: Bruce Momjian; Manfred Koizar; Mark Cave-Ayland
> (External); pgsql-hackers(at)postgresql(dot)org
> Subject: Re: [HACKERS] Cost of XLogInsert CRC calculations
>
>
> Greg Stark <gsstark(at)mit(dot)edu> writes:
> > Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:
> >> Do we know that? The results I showed put at least one fundamentally
> >> 32bit platform (the PowerBook I'm typing this on) at dead par for
> >> 32bit and 64bit CRCs.
>
> > Wait, par for 32-bit CRCs? Or for 64-bit CRCs calculated using 32-bit
> > ints?
>
> Right, the latter. We haven't actually tried to measure the
> cost of plain 32bit CRCs... although I seem to recall that
> when we originally decided to use 64bit, someone put up some
> benchmarks purporting to show that there wasn't much difference.

If all goes to plan, I shall post a test program for CRC32 tomorrow along
with my results for other people to benchmark. The one thing this exercise
has shown me is that you can't necessarily trust the compiler to make the
right optimisation choices 100% of time, and so the only real way to find
out about the performance is to test out what you are trying to do with some
sample data. To quote Tom on several occasions: "Let's stop the hand
waving...."

Looking at the asm code produced by gcc, my feeling is that most of the time
would be spent reading/writing to memory rather than doing the shift and
xor. Using a 64-bit CRC, there aren't enough registers under x86-32 to store
the result of each iteration of the algorithm and so it gets pushed out of
the registers into memory at the end of each iteration and then read back in
at the beginning of the next iteration.

With a 32-bit CRC, the entire calculation could potentially be done entirely
in the registers, with the final result being written to memory at the end.
Combined with the fact that less cycles will be required for the shift (and
that main memory is only read rather than written) then I would expect a
32-bit CRC to be significantly faster. I also think that since the last
tests for 32-bit vs 64-bit CRC were done, compilers will have improved by
several orders of magnitude making the difference between the two more
noticeable. However, I shall wait until I have the code completed and
working before I report on the results :)

Kind regards,

Mark.

------------------------
WebBased Ltd
17 Research Way
Plymouth
PL6 8BT

T: +44 (0)1752 797131
F: +44 (0)1752 791023
W: http://www.webbased.co.uk