Re: Compression of full-page-writes

On 8 December 2014 at 11:46, Michael Paquier <michael(dot)paquier(at)gmail(dot)com> wrote:

>> * ideally we'd like to be able to differentiate the types of usage.
>> which then allows the user to control the level of compression
>> depending upon the type of action. My first cut at what those settings
>> should be are ALL > LOGICAL > PHYSICAL > VACUUM.
>> VACUUM - only compress while running vacuum commands
>> PHYSICAL - only compress while running physical DDL commands (ALTER
>> TABLE set tablespace, CREATE INDEX), i.e. those that wouldn't
>> typically be used for logical decoding
>> LOGICAL - compress FPIs for record types that change tables
>> ALL - all user commands
>> (each level includes all prior levels)
>
> Well, that's clearly an optimization so I don't think this should be
> done for a first shot but those are interesting fresh ideas.

It is important that we offer an option that retains user performance.
I don't see that as an optimisation, but as an essential item.

The current feature will reduce WAL volume, at the expense of
foreground user performance. Worse, that will all happen around time
of new checkpoint, so I expect this will have a large impact.
Presumably testing has been done to show the impact on user response
times? If not, we need that.

The most important distinction is between foreground and background tasks.

If you think the above is too complex, then we should make the
parameter into a USET, but set it to on in VACUUM, CLUSTER and
autovacuum.

> Technically speaking, note that we would need to support such things
> with a new API to switch a new context flag in registered_buffers of
> xloginsert.c for each block, and decide if the block is compressed
> based on this context flag, and the compression level wanted.
>
>> * name should not be wal_compression - we're not compressing all wal
>> records, just fpis. There is no evidence that we even want to compress
>> other record types, nor that our compression mechanism is effective at
>> doing so. Simple => keep name as compress_full_page_writes
>> Though perhaps we should have it called wal_compression_level
>
> I don't really like those new names, but I'd prefer
> wal_compression_level if we go down that road with 'none' as default
> value. We may still decide in the future to support compression at the
> record level instead of context level, particularly if we have an API
> able to do palloc_return_null_at_oom, so the idea of WAL compression
> is not related only to FPIs IMHO.

We may yet decide, but the pglz implementation is not effective on
smaller record lengths. Nor has any testing been done to show that is
even desirable.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services