Re: Compression of full-page-writes

On Thu, Oct 24, 2013 at 8:37 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Mon, Oct 21, 2013 at 11:52 PM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>> So, our consensus is to introduce the hooks for FPW compression so that
>> users can freely select their own best compression algorithm?
>> Also, probably we need to implement at least one compression contrib module
>> using that hook, maybe it's based on pglz or snappy.
>
> I don't favor making this pluggable. I think we should pick snappy or
> lz4 (or something else), put it in the tree, and use it.

The reason why the discussion went towards making it pluggable (or at
least what made me to think like that) was because of below reasons:
a. what somebody needs to do to make snappy or lz4 in the tree, is it
only performance/compression data for some of the scenario's or some
other legal
stuff as well, if it is only performance/compression then what
will be the scenario's (is pgbench sufficient?).
b. there can be cases where one or the other algorithm can be better
or not doing compression is better. For example in one of the other
patches where
we were trying to achieve WAL reduction in Update operation
(http://www.postgresql.org/message-id/8977CB36860C5843884E0A18D8747B036B9A4B04@szxeml558-mbs.china.huawei.com)
Heikki has came up with a test (where data is not much
compressible), in such a case, the observation was that LZ was better
than native
compression method used in that patch and Snappy was better than
LZ and not doing compression could be considered preferable in such a
scenario because all the algorithm's were reducing TPS for that case.

Now I think it is certainly better if we could choose one of the
algorithms (snappy or lz4) and test them for most used scenario's for
compression and performance and call it done, but I think giving at
least an option to user to make compression altogether off should be
still considered.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com