Re: libpq compression (part 3)

On Tue, Dec 19, 2023 at 11:41 AM Jacob Burroughs
<jburroughs(at)instructure(dot)com> wrote:
> The compression "handshaking" process now looks as follows:
> 1. Client sends startup packet with `_pq_.libpq_compression = alg1;alg2`
> 2. At this point, the server can immediately begin compressing packets
> to the client with any of the specified algorithms it supports if it
> so chooses
> 3. Server includes `libpq_compression` in the automatically sent
> `ParameterStatus` messages before handshaking
> 4. At this point, the client can immediately begin compressing packets
> to the server with any of the supported algorithms
> Both the server and client will prefer to compress using the first
> algorithm in their list that the other side supports, and we
> explicitly support `none` in the algorithm list. This allows e.g. a
> client to use `none;gzip` and a server to use `zstd;gzip;lz4`, and
> then the client will not compress its data but the server will send
> its data using gzip.

I'm having difficulty understanding the details of this handshaking
algorithm from this description. It seems good that the handshake
proceeds in each direction somewhat separately from the other, but I
don't quite understand how the whole thing fits together. If the
client tells the server that 'none,gzip' is supported, and I elect to
start using gzip, how does the client know that I picked gzip rather
than none? Are the compressed packets self-identifying?

It's also slightly odd to me that the same parameter seems to specify
both what we want to send, and what we're able to receive. I'm not
really sure we should have separate parameters for those things, but I
don't quite understand how this works without it. The "none" thing
seems like a bit of a hack. It lets you say "I'd like to receive
compressed data but send uncompressed data" ... but what about the
reverse? How do you say "don't bother compressing what you receive
from the server, but please lz4 everything you send to the server"? Or
how do you say "use zstd from server to client, but lz4 from client to
server"? It seems like you can't really say that kind of thing.

What if we had, on the server side, a GUC saying what compression to
accept and a GUC saying what compression to be willing to do? And then
let the client request whatever it wants for each direction.

> Also please let me know if I have made any notable mailing list/patch
> etiquette/format/structure errors. This is my first time submitting a
> patch to a mailing-list driven open source project and while I have
> tried to carefully review the various wiki guides I'm sure I didn't
> take everything in perfectly.

Seems fine to me.

--
Robert Haas
EDB: http://www.enterprisedb.com