Re: refactoring basebackup.c (zstd workers)

From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Justin Pryzby <pryzby(at)telsasoft(dot)com>
Cc: Dipesh Pandit <dipesh(dot)pandit(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Jeevan Ladhe <jeevanladhe(dot)os(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, "Shinoda, Noriyoshi (PN Japan FSIP)" <noriyoshi(dot)shinoda(at)hpe(dot)com>, Abhijit Menon-Sen <ams(at)toroid(dot)org>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, Jeevan Ladhe <jeevan(dot)ladhe(at)enterprisedb(dot)com>, Mark Dilger <mark(dot)dilger(at)enterprisedb(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>, tushar <tushar(dot)ahuja(at)enterprisedb(dot)com>
Subject: Re: refactoring basebackup.c (zstd workers)
Date: 2022-03-14 17:02:20
Message-ID: CA+TgmoaYrNxoGB8x_7Fk5uKXfbyMVAdSXTnfQN-1zu_3gB_v=Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Mar 14, 2022 at 12:35 PM Justin Pryzby <pryzby(at)telsasoft(dot)com> wrote:
> I suggest to use a syntax that's more general than that, maybe something like
>
> :[level=]N,parallel=N,flag,flag,...
>
> For example, someone may want to use zstd "long" mode or (when it's released)
> rsyncable mode, or specify fine-grained compression parameters (strategy,
> windowLog, hashLog, etc).

That's an interesting idea. I wonder what the replication protocol
ought to look like in that case. Should we have a COMPRESSION_DETAIL
argument that is just a string, and let the server parse it out? Or
separate protocol-level options? It does feel reasonable to have both
COMPRESSION_LEVEL and COMPRESSION_WORKERS as first-class options, but
I don't know that we want COMPRESSION_HASHLOG true as part of our
first-class grammar.

> I hope the same syntax will be shared with wal_compression and pg_dump.
> And libpq, if that patch progresses.
>
> BTW, I think this may be better left for PG16.

Possibly so ... but if we're thinking of any revisions to the
newly-added grammar, we had better take care of that now, before it's
set in stone.

--
Robert Haas
EDB: http://www.enterprisedb.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Nathan Bossart 2022-03-14 17:04:03 Re: add checkpoint stats of snapshot and mapping files of pg_logical dir
Previous Message Andres Freund 2022-03-14 17:00:23 Re: Allow async standbys wait for sync replication