Re: Introduce pg_receivewal gzip compression tests

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Tuesday, July 13th, 2021 at 09:37, Michael Paquier <michael(at)paquier(dot)xyz> wrote:

> On Tue, Jul 13, 2021 at 06:36:59AM +0000, gkokolatos(at)pm(dot)me wrote:
>
> > I am sorry this was not so clear. It is indeed running twice the binary
> > with different flags. However the goal is not to check the flags, but
> > to make certain that the partial file has now been completed. That is
> > why there was code asserting that the previous FILENAME.gz.partial file
> > after the second invocation is converted to FILENAME.gz.
>
> The first run you are adding checks the same thing thanks to
> pg_switch_wal(), where pg_receivewal completes the generation of
> 000000010000000000000002.gz and finishes with
> 000000010000000000000003.gz.partial.

This is correct. It is the 000000010000000000000003 awl that the rest
of the tests are targeting.

>
> > Additionally the second invocation of pg_receivewal is extending the
> > coverage of FindStreamingStart().
>
> Hmm. It looks like a waste in runtime once we mix LZ4 in that as that
> would mean 5 runs of pg_receivewal, but we really need only three of
> them with --endpos:
> - One with ZLIB compression.
> - One with LZ4 compression.
> - One without compression.
>
> Do you think that we could take advantage of what is now the only run
> of pg_receivewal --endpos for that? We could make the ZLIB checks run
> first, conditionally, and then let the last command with --endpos
> perform a full scan of the contents in $stream_dir with the .gz files
> already in place. The addition of LZ4 would be an extra conditional
> block similar to what's introduced in ZLIB, running before the last
> command without compression.

I will admit that for the current patch I am not taking lz4 into account as
at the moment I have little idea as to how the lz4 patch will advance with the
review rounds. I simply accepted that it will be rebased on top of the patch
in the current thread and probably need to modify the current then.

But I digress. I would like have some combination of .gz and .gz.parial but
I will not take too strong of a stance. I am happy to go with your suggestion.

Cheers,
//Georgios

>
> --
>
> Michael