Re: libpq compression

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> On Mon, Jun 25, 2012 at 4:25 PM, ktm(at)rice(dot)edu <ktm(at)rice(dot)edu> wrote:
>> Here is the benchmark list from the Google lz4 page:
>>
>> Name Ratio C.speed D.speed
>> LZ4 (r59) 2.084 330 915
>> LZO 2.05 1x_1 2.038 311 480
>> QuickLZ 1.5 -1 2.233 257 277
>> Snappy 1.0.5 2.024 227 729
>> LZF 2.076 197 465
>> FastLZ 2.030 190 420
>> zlib 1.2.5 -1 2.728 39 195
>> LZ4 HC (r66) 2.712 18 1020
>> zlib 1.2.5 -6 3.095 14 210

>> lz4 absolutely dominates on compression/decompression speed. While fastlz
>> is faster than zlib(-1) on compression, lz4 is almost 2X faster still.

> At the risk of making everyone laugh at me, has anyone tested pglz?

Another point here is that those Google numbers (assuming that they
apply to our use-cases, a point not in evidence) utterly fail to make
the argument that zlib is not the thing to use. zlib is beating all
the others on compression ratio, often by 50%. Before making any
comparisons, you have to make some assumptions about what the network
speed is ... and unless it's pretty damn fast relative to your CPU speed
getting the better compression ratio is going to be more attractive than
saving some cycles.

regards, tom lane