Re: libpq compression

On 15.02.2019 15:42, Peter Eisentraut wrote:
> On 2018-06-19 09:54, Konstantin Knizhnik wrote:
>> The main drawback of streaming compression is that you can not
>> decompress some particular message without decompression of all previous
>> messages.
> It seems this would have an adverse effect on protocol-aware connection
> proxies: They would have to uncompress everything coming in and
> recompress everything going out.
>
> The alternative of compressing each packet individually would work much
> better: A connection proxy could peek into the packet header and only
> uncompress the (few, small) packets that it needs for state and routing.
>
Individual compression of each message depreciate all idea of libpq
compression.
Messages are two small to efficiently compress each of them separately.
So using streaming compression algorithm is absolutely necessary here.

Concerning possible problem with proxies I do not think that it is
really a problem.
Proxy is very rarely located somewhere in the "middle" between client
and database servers.
It is usually launched either in the same network as DBMS client (for
example, if client is application server) either in the same network
with database server.
In both cases there is not so much sense to pass compressed traffic
through the proxy.
If proxy and DBMS server are located in the same network, then proxy
should perform decompression and send
decompressed messages to the database server.

Thank you very much for noticing this problem with compatibility
compression and protocol-aware connection proxies.
I have wrote that current compression implementation (zpq_stream.c) can
be used not only for libpq backend/frontend, but
also for compression any other streaming data. But I could not imaging
what other data sources can require compression.
And proxy is exactly such case: it also needs to compress/decompress
messages.
It is one more argument to make interface of zpq_stream as simple as
possible and encapsulate all inflating/deflating logic in this code.
It can be achieved by passing arbitrary rx/tx function to zpq_create
function.

--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company