Re: zstd compression for pg_dump

On 4/1/23 15:36, Justin Pryzby wrote:
>
> ...
>
>> If there are no concerns, why disable it outside Windows? I don't have a
>> good idea how beneficial the multi-threaded compression is, so I can't
>> quite judge the risk/benefits tradeoff.
>
> Because it's a minor/fringe feature, and it's annoying to have platform
> differences (would we plan on relaxing the restriction in v17, or is it
> more likely we'd forget ?).
>
> I realized how little I've tested with zstd workers myself. And I think
> on cirrusci, the macos and freebsd tasks have zstd libraries with
> threading support, but it wasn't being exercised (because using :workers
> would cause the patch to fail unless it's supported everywhere). So I
> updated the "for CI only" patch to 1) use meson wraps to compile zstd
> library with threading on linux and windows; and, 2) use zstd:workers=3
> "opportunistically" (but avoid failing if threads are not supported,
> since the autoconf task still doesn't have access to a library with
> thread support). That's a great step, but it still seems bad that the
> thread stuff has been little exercised until now. (Also, the windows
> task failed; I think that's due to a transient network issue).
>

Agreed, let's leave the threading for PG17, depending on how beneficial
it turns out to be for pg_dump.

> Feel free to mess around with threads (but I'd much rather see the patch
> progress for zstd:long).

OK, understood. The long mode patch is pretty simple. IIUC it does not
change the format, i.e. in the worst case we could leave it for PG17
too. Correct?

regards

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company