Re: [REVIEW] Re: Compression of full-page-writes

On Fri, Sep 12, 2014 at 11:17:12PM +0300, Ants Aasma wrote:
> On Fri, Sep 12, 2014 at 10:38 PM, Heikki Linnakangas
> <hlinnakangas(at)vmware(dot)com> wrote:
> > I don't mean that we should abandon this patch - compression makes the WAL
> > smaller which has all kinds of other benefits, even if it makes the raw TPS
> > throughput of the system worse. But I'm just saying that these TPS
> > comparisons should be taken with a grain of salt. We probably should
> > consider switching to a faster CRC algorithm again, regardless of what we do
> > with compression.
>
> CRC is a pretty awfully slow algorithm for checksums. We should
> consider switching it out for something more modern. CityHash,
> MurmurHash3 and xxhash look like pretty good candidates, being around
> an order of magnitude faster than CRC. I'm hoping to investigate
> substituting the WAL checksum algorithm 9.5.
>
> Given the room for improvement in this area I think it would make
> sense to just short-circuit the CRC calculations for testing this
> patch to see if the performance improvement is due to less data being
> checksummed.
>
> Regards,
> Ants Aasma

+1 for xxhash -

version speed on 64-bits speed on 32-bits
------- ---------------- ----------------
XXH64 13.8 GB/s 1.9 GB/s
XXH32 6.8 GB/s 6.0 GB/s

Here is a blog about its performance as a hash function:

http://fastcompression.blogspot.com/2014/07/xxhash-wider-64-bits.html

Regards,
Ken