Re: What exactly is our CRC algorithm?

At 2014-11-20 09:52:01 +0530, ams(at)2ndQuadrant(dot)com wrote:
>
> To address (a), I am timing a standby restoring the same 11GB of WAL
> via restore_command with and without the CRC patch.

I ran "perf record -F 100 --call-graph=dwarf bin/postgres -D backup",
where:

(a) bin/postgres was compiled, as before, with CFLAGS="-O3 -msse4.2" and
without --enable-debug.
(b) backup is a basebackup of the original data directory before I ran
pgbench; it has a recovery.conf with a restore_command to fetch the
WAL generated during the pgbench run.

I ran the test "frice" (as my daughter would say) with HEAD and the
slice-by-8 patch, and counted the time between the first and last
«restored log file "NNN" from archive» log messages. The number in
parentheses shows the order in which I ran the tests.

HEAD: 6m47s (1), 6m23s (2), 3m12s (6), 3m04s (7)
8CRC: 5m16s (3), 3m13s (4), 2m57s (5), 2m46s (8)

In the unpatched server, the profile shows ValidXLogRecord at >50% in
every case (54.81%, 59.41%, 58.82%, 59.05%).

In the patched server, pg_comp_crc32c was at the top of the profile in
three cases, with 10.29%, 12.01%, and 10.99%. In test (4), however, it
was at 6.38%, and first place went to copy_user_enhanced_fast_string
with 10.03% (which was in second place the other three times).

I believe the wallclock runtime in these tests was influenced more by IO
than CPU, but that the profile shows a worthwhile improvement anyway. I
have the perf.data from each run, and I can rerun the tests if anyone
wants to suggest a different strategy.

> Suggestions for how to address (b) are welcome.

I'll think about how to do this, and work on the SSE4.2 patch meanwhile.

-- Abhijit