Re: [REVIEW] Re: Compression of full-page-writes

From: Merlin Moncure <mmoncure(at)gmail(dot)com>
To: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
Cc: Andres Freund <andres(at)2ndquadrant(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, Robert Haas <robertmhaas(at)gmail(dot)com>, Rahila Syed <rahilasyed90(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [REVIEW] Re: Compression of full-page-writes
Date: 2014-12-16 15:12:49
Message-ID: CAHyXU0xxAfhBBALgDknMTFf=y+LA53rwMXcb=YmffMRG+Q=H3w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Dec 15, 2014 at 5:37 PM, Michael Paquier
<michael(dot)paquier(at)gmail(dot)com> wrote:
> On Tue, Dec 16, 2014 at 5:14 AM, Merlin Moncure <mmoncure(at)gmail(dot)com> wrote:
>> OTOH, Our built in compressor as we all know is a complete dog in
>> terms of cpu when stacked up against some more modern implementations.
>> All that said, as long as there is a clean path to migrating to
>> another compression alg should one materialize, that problem can be
>> nicely decoupled from this patch as Robert pointed out.
> I am curious to see some numbers about that. Has anyone done such
> comparison measurements?

I don't, but I can make some. There are some numbers on the web but
it's better to make some new ones because IIRC some light optimization
had gone into plgz of late.

Compressing *one* file with lz4 and a quick/n/dirty plgz i hacked out
of the source (borrowing heavily from
https://github.com/maropu/pglz_bench/blob/master/pglz_bench.cpp) I
tested the results:

lz4 real time: 0m0.032s
pglz real time: 0m0.281s

mmoncure(at)mernix2 ~/src/lz4/lz4-r125 $ ls -lh test.*
-rw-r--r-- 1 mmoncure mmoncure 2.7M Dec 16 09:04 test.lz4
-rw-r--r-- 1 mmoncure mmoncure 2.5M Dec 16 09:01 test.pglz

A better test would examine all manner of different xlog files in a
fashion closer to how your patch would need to compress them but the
numbers here tell a fairly compelling story: similar compression
results for around 9x the cpu usage. Be advised that compression alg
selection is one of those types of discussions that tends to spin off
into outer space; that's not something you have to solve today. Just
try and make things so that they can be switched out if things
change....

merlin

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2014-12-16 15:15:12 Re: On partitioning
Previous Message Mark Cave-Ayland 2014-12-16 15:06:43 Re: Commitfest problems