Re: Compression of full-page-writes

On Thu, Jan 1, 2015 at 2:39 AM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>
> On Tue, Dec 30, 2014 at 01:27:44PM +0100, Andres Freund wrote:
> > On 2014-12-30 21:23:38 +0900, Michael Paquier wrote:
> > > On Tue, Dec 30, 2014 at 6:21 PM, Jeff Davis <pgsql(at)j-davis(dot)com> wrote:
> > > > On Fri, 2013-08-30 at 09:57 +0300, Heikki Linnakangas wrote:
> > > >> Speeding up the CRC calculation obviously won't help with the WAL
volume
> > > >> per se, ie. you still generate the same amount of WAL that needs
to be
> > > >> shipped in replication. But then again, if all you want to do is to
> > > >> reduce the volume, you could just compress the whole WAL stream.
> > > >
> > > > Was this point addressed?
> > > Compressing the whole record is interesting for multi-insert records,
> > > but as we need to keep the compressed data in a pre-allocated buffer
> > > until WAL is written, we can only compress things within a given size
> > > range. The point is, even if we define a lower bound, compression is
> > > going to perform badly with an application that generates for example
> > > many small records that are just higher than the lower bound...
> > > Unsurprisingly for small records this was bad:
> >
> > So why are you bringing it up? That's not an argument for anything,
> > except not doing it in such a simplistic way.
>
> I still don't understand the value of adding WAL compression, given the
> high CPU usage and minimal performance improvement. The only big
> advantage is WAL storage, but again, why not just compress the WAL file
> when archiving.
>
> I thought we used to see huge performance benefits from WAL compression,
> but not any more?

I think there can be performance benefit for the cases when the data
is compressible, but it would be loss otherwise. The main thing is
that the current compression algorithm (pg_lz) used is not so
favorable for non-compresible data.

>Has the UPDATE WAL compression removed that benefit?

Good question, I think there might be some impact due to that, but in
general for page level compression still there will be much more to
compress.

In general, I think this idea has merit with respect to compressible data,
and to save for the cases where it will not perform well, there is a on/off
switch for this feature and in future if PostgreSQL has some better
compression method, we can consider the same as well. One thing
that we need to think is whether user's can decide with ease when to
enable this global switch.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com