Re: refactoring basebackup.c

Hi,

Similar to LZ4 server-side compression, I have also tried to add a ZSTD
server-side compression in the attached patch. I have done some initial
testing and things seem to be working.

Example run:
pg_basebackup -t server:/tmp/data_zstd -Xnone --server-compression=zstd

The patch surely needs some grooming, but I am expecting some initial
review, specially in the area where we are trying to close the zstd stream
in bbsink_zstd_end_archive(). We need to tell the zstd library to end the
compression by calling ZSTD_compressStream2() thereby sending a
ZSTD_e_end flag. But, this also needs some input string, which per
example[1] line # 686, I have taken as an empty ZSTD_inBuffer.

Thanks, Tushar for testing the LZ4 patch. I have added the LZ4 option in
the pg_basebackup help now.

Note: Before applying these patches please apply Robert's v10 version
of patches 0002, 0003, and 0004.

[1]
https://fuchsia.googlesource.com/fuchsia/+/refs/heads/main/zircon/tools/zbi/zbi.cc

Regards,
Jeevan Ladhe

On Wed, Jan 5, 2022 at 10:24 PM tushar <tushar(dot)ahuja(at)enterprisedb(dot)com>
wrote:

>
>
> On Tue, Dec 28, 2021 at 1:12 PM Jeevan Ladhe <
> jeevan(dot)ladhe(at)enterprisedb(dot)com> wrote:
>
>> Hi Tushar,
>>
>> You need to apply Robert's v10 version patches 0002, 0003 and 0004,
>> before applying the lz4 patch(v8 version).
>> Please let me know if you still face any issues.
>>
>
> Thanks, Jeevan.
> I tested —server-compression option using different other options of
> pg_basebackup, also checked -t/—server-compression from pg_basebackup of
> v15 will
> throw an error if the server version is v14 or below. Things are looking
> good to me.
> Two open issues -
> 1)lz4 value is missing for --server-compression in pg_basebackup --help
> 2)Error messages need to improve if using -t server with -z/-Z
>
> regards,
>