| From: | Andres Freund <andres(at)anarazel(dot)de> |
|---|---|
| To: | John Naylor <johncnaylorls(at)gmail(dot)com> |
| Cc: | Bruce Momjian <bruce(at)momjian(dot)us>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Nathan Bossart <nathandbossart(at)gmail(dot)com>, "Shankaran, Akash" <akash(dot)shankaran(at)intel(dot)com>, "Devulapalli, Raghuveer" <raghuveer(dot)devulapalli(at)intel(dot)com> |
| Subject: | Re: Proposal for Updating CRC32C with AVX-512 Algorithm. |
| Date: | 2024-12-12 15:45:29 |
| Message-ID: | 5rnt26xcnd6flvnahyr37kozdiagfj5cczsmolu722ccfgeyy3@3yj5crjnf3o5 |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi,
On 2024-12-12 18:32:20 +0700, John Naylor wrote:
> I went and looked at the Chromium source, and found the following
> snippet that uses the same technique, but only requires 128-bit CLMUL
> and has a minimum input size of 64 bytes, rather than 256. This seems
> like it might be better suited for shorter inputs. Also seems much
> easier than trying to get the AVX-512 hippo to dance. It uses the IEEE
> polynomial, so would need new constants calculated for ours, but that
> had to be done for the shared patch, too.
Frankly, we should just move away from using CRCs. They're good for cases
where short runs of bit flips are much more likely than other kinds of errors
and where the amount of data covered by them has a low upper bound. That's not
at all the case for WAL records. It'd not matter too much if CRCs were cheap
to compute - but they aren't. We should instead move to some more generic
hashing algorithm, decent ones are much faster.
Greetings,
Andres
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Nathan Bossart | 2024-12-12 15:53:00 | Re: .ready and .done files considered harmful |
| Previous Message | Tom Lane | 2024-12-12 15:35:15 | Re: pg_createsubscriber TAP test wrapping makes command options hard to read. |