Re: Compression of full-page-writes

On Thu, Jan 1, 2015 at 2:10 PM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> On Thu, Jan 1, 2015 at 2:39 AM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>> > So why are you bringing it up? That's not an argument for anything,
>> > except not doing it in such a simplistic way.
>>
>> I still don't understand the value of adding WAL compression, given the
>> high CPU usage and minimal performance improvement. The only big
>> advantage is WAL storage, but again, why not just compress the WAL file
>> when archiving.
When doing some tests with pgbench for a fixed number of transactions,
I also noticed a reduction in replay time as well, see here for
example some results here:
http://www.postgresql.org/message-id/CAB7nPqRv6RaSx7hTnp=g3dYqOu++FeL0UioYqPLLBdbhAyB_jQ@mail.gmail.com

>> I thought we used to see huge performance benefits from WAL compression,
>> but not any more?
>
> I think there can be performance benefit for the cases when the data
> is compressible, but it would be loss otherwise. The main thing is
> that the current compression algorithm (pg_lz) used is not so
> favorable for non-compresible data.
Yes definitely. Switching to a different algorithm would be the next
step forward. We have been discussing mainly about lz4 that has a
friendly license, I think that it would be worth studying other things
as well once we have all the infrastructure in place.

>>Has the UPDATE WAL compression removed that benefit?
>
> Good question, I think there might be some impact due to that, but in
> general for page level compression still there will be much more to
> compress.
That may be a good thing to put a number on. We could try to patch a
build with a revert of a3115f0d and measure a bit that the difference
in WAL size that it creates. Thoughts?

> In general, I think this idea has merit with respect to compressible data,
> and to save for the cases where it will not perform well, there is a on/off
> switch for this feature and in future if PostgreSQL has some better
> compression method, we can consider the same as well. One thing
> that we need to think is whether user's can decide with ease when to
> enable this global switch.
The opposite is true as well, we shouldn't force the user to have data
compressed even if the switch is disabled.
--
Michael