Re: Compression of full-page-writes

On Mon, Oct 21, 2013 at 4:40 PM, KONDO Mitsumasa
<kondo(dot)mitsumasa(at)lab(dot)ntt(dot)co(dot)jp> wrote:
> (2013/10/19 14:58), Amit Kapila wrote:
>> On Tue, Oct 15, 2013 at 11:41 AM, KONDO Mitsumasa
>> <kondo(dot)mitsumasa(at)lab(dot)ntt(dot)co(dot)jp> wrote:
>> In general, my thinking is that we should prefer compression to reduce
>> IO (WAL volume), because reducing WAL volume has other benefits as
>> well like sending it to subscriber nodes. I think it will help cases
>> where due to less n/w bandwidth, the disk allocated for WAL becomes
>> full due to high traffic on master and then users need some
>> alternative methods to handle such situations.
> Do you talk about archiving WAL file?

One of the points what I am talking about is sending data over
network to subscriber nodes for streaming replication and another is
WAL in pg_xlog. Both scenario's get benefited if there is is WAL
volume.

> It can easy to reduce volume that we
> set and add compression command with copy command at archive_command.

Okay.

>> I think many users would like to use a method which can reduce WAL
>> volume and the users which don't find it enough useful in their
>> environments due to decrease in TPS or not significant reduction in
>> WAL have the option to disable it.
> I favor to select compression algorithm for higher performance. If we need
> to compress WAL file more, in spite of lessor performance, we can change
> archive copy command with high compression algorithm and add documents that
> how to compress archive WAL files at archive_command. Does it wrong?

No, it is not wrong, but there are scenario's as mentioned above
where less WAL volume can be beneficial.

> In
> actual, many of NoSQLs use snappy for purpose of higher performance.

Okay, you can also check the results with snappy algorithm, but don't
just rely completely on snappy for this patch, you might want to think
of another alternative for this patch.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com