Re: Introduce pg_receivewal gzip compression tests

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: gkokolatos(at)pm(dot)me
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Introduce pg_receivewal gzip compression tests
Date: 2021-07-15 12:35:52
Message-ID: YPArqBhngZZkzs1x@paquier.xyz
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Jul 15, 2021 at 08:35:27PM +0900, Michael Paquier wrote:
> 1) bowerbird on Windows/MSVC:
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bowerbird&dt=2021-07-15%2010%3A30%3A36
> pg_receivewal: fatal: could not fsync existing write-ahead log file
> "000000010000000000000002.partial": Permission denied
> not ok 20 - streaming some WAL using ZLIB compression
> I don't think the existing code can be blamed for that as this means a
> failure with gzflush(). Likely a concurrency issue as that's an
> EACCES. If that's repeatable, that could point to an actual issue
> with pg_receivewal --compress.

For this one, I'll try to test harder on my own host. I am curious to
see if the other Windows members running the TAP tests have anything
to say. Looking at the code of zlib, this would come from gz_zero()
in gzflush(), which could blow up on a write() in gz_comp().

> 2) curculio:
>
> Looking at the OpenBSD code (usr.bin/compress/main.c), long options
> are supported, where --version does exit(0) without printing
> anything, and --test is supported even if that's not on the man pages.
> set_outfile() is doing a discard of the file suffixes it does not
> recognize, and I think that their implementation bumps on .gz.partial
> and generates an exit code of 512 to map with WARNING. I still wish
> to keep this test, and I'd like to think that the contents of
> @zlib_wals are enough in terms of coverage. What do you think?

After thinking more about this one, I have taken the course to just
remove the .gz.partial segment from the check, a full segment should
be enough in terms of coverage. I prefer this simplification over a
rename of the .partial segment or a tweak of the error code to map
with WARNING.
--
Michael

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Ranier Vilela 2021-07-15 12:38:26 Re: [PATCH] Use optimized single-datum tuplesort in ExecSort
Previous Message Ranier Vilela 2021-07-15 12:31:15 Re: Signed vs Unsigned (take 2) (src/backend/storage/ipc/procarray.c)