Re: Enabling Checksums

On 3/8/13 3:38 AM, Heikki Linnakangas wrote:
> See https://www.kernel.org/doc/Documentation/block/data-integrity.txt

That includes an interesting comment that's along the lines of the MySQL
checksum tests already mentioned:

"The 16-bit CRC checksum mandated by both the SCSI and SATA specs
is somewhat heavy to compute in software. Benchmarks found that
calculating this checksum had a significant impact on system
performance for a number of workloads. Some controllers allow a
lighter-weight checksum to be used when interfacing with the operating
system. Emulex, for instance, supports the TCP/IP checksum instead."

The TCP/IP checksum spec is at https://tools.ietf.org/html/rfc793 ; its
error detection limitations are described at
http://www.noahdavids.org/self_published/CRC_and_checksum.html ; and a
good article about optimizing its code is at
http://www.locklessinc.com/articles/tcp_checksum/ I'll take a longer
look at whether it's an improvement on the Fletcher-16 used by the
current patch. All of these 16 bit checksums are so much better than
nothing. I don't think some shift toward prioritizing computation speed
over detection rate is a problem. In the long run really sensitive 32
bit checksums will become more practical.

As Heikki pointed out, the direction this whole area seems to be going
is that one day you might get checksums all the way from application to
hardware. That's another possible future where having some field tested
checksum feature in the database will be valuable.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com