Re: Different compression methods for FPI

On Sun, Jun 13, 2021 at 08:24:12PM -0500, Justin Pryzby wrote:
> I think it's more nuanced than just finding the algorithm with the least CPU
> use. The GUC is PGC_USERSET, and it's possible that a data-loading process
> might want to use zlib for better compress ratio, but an interactive OLTP
> process might want to use lz4 or no compression for better responsiveness.

It seems to me that this should be a PGC_SUSET, at least? We've had
our share of problems with assumptions behind data leaks depending on
data compressibility (see ssl_compression and the kind).

> In this patch series, I added compression information to the errcontext from
> xlog_block_info(), and allow specifying compression levels like zlib-2. I'll
> rearrange that commit earlier if we decide that's desirable to include.

The compression level may be better if specified with a different
GUC. That's less parsing to have within the GUC machinery.

So, how does the compression level influence those numbers? The level
of compression used by LZ4 here is the fastest-CPU/least-compression,
same for zlib and zstd? Could we get some data with getrusage()? It
seems to me that if we can get the same amount of compression and CPU
usage just by tweaking the compression level, there is no need to
support more than one extra compression algorithm, easing the life of
packagers and users.
--
Michael