From: | Daniel Farina <daniel(at)heroku(dot)com> |
---|---|
To: | pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Faster compression, again |
Date: | 2012-03-14 18:06:16 |
Message-ID: | CAAZKuFb59sABSa7gCG0vnVnGb-mJCUBBbrKiyPraNXHnis7KMw@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
For 9.3 at a minimum.
The topic of LZO became mired in doubts about:
* Potential Patents
* The author's intention for the implementation to be GPL
Since then, Google released "Snappy," also an LZ77-class
implementation, and it has been ported to C (recently, and with some
quirks, like no LICENSE file...yet, although it is linked from the
original Snappy project). The original Snappy (C++) has a BSD license
and a patent grant (which shields you from Google, at least). Do we
want to investigate a very-fast compression algorithm inclusion again
in the 9.3 cycle?
I've been using the similar implementation "LZO" for WAL archiving and
it is a significant savings (not as much as pg_lesslog, but also less
invasive). It is also fast enough that even if one were not to uproot
TOAST's compression that it would probably be very close to a complete
win for protocol traffic, whereas SSL's standardized zlib can
definitely be a drag in some cases.
This idea resurfaces often, but the reason why I wrote in about it is
because I have a table which I categorized as "small" but was, in
fact, 1.5MB, which made transferring it somewhat slow over a remote
link. zlib compression takes it down to about 550K and lzo (similar,
but not identical) 880K. If we're curious how it affects replication
traffic, I could probably gather statistics on LZO-compressed WAL
traffic, of which we have a pretty huge amount captured.
--
fdr
From | Date | Subject | |
---|---|---|---|
Next Message | ktm@rice.edu | 2012-03-14 18:13:27 | Re: Faster compression, again |
Previous Message | Marco Nenciarini | 2012-03-14 18:03:08 | Re: [PATCH] Support for foreign keys with arrays |