Re: Wire protocol compression

On 21 April 2016 at 14:19, Shay Rojansky <roji(at)roji(dot)org> wrote:

> Does it make sense to you guys to discuss compression outside of TLS?
>

Yes.

> There are potentially huge bandwidth savings which could benefit both WAN
> and non-WAN scenarios, and decoupling this problem from TLS would make it
> both accessible to everyone (assuming PostgreSQL clients follow). It would
> be a protocol change though.
>

It would, but not necessarily a drastic one.

The only reason it has to be any drama at all is that we can't securely use
some GUC like allow_compression=on set in the startup message. As Tom has
pointed out in the past, users may be able to set these "through" a client
library that is not expecting compressed responses, causing the client
library to fail in a possibly-exploitable manner. I'm not nearly as
concerned about that, but I don't deny that it's theoretically possible.

So we'd have to do a protocol version bump from the 3.0 protocol, allowing
clients to send 3.1 (or 4.0 or whatever) protocol requests. Newer clients
connecting to older servers would get rejected and have to reconnect with
3.0 protocol, or be configured with protocol=3.0 in their setup (JDBC URI,
libpq connstring, etc). Much like we already do for SSL when the server
doesn't support it.

The problem there is that suddenly everyone wants to get their desired
protocol features in, since we're changing things anyway, and "enabling
protocol compression" becomes ... rather more.

https://wiki.postgresql.org/wiki/Todo#Wire_Protocol_Changes

It'd be pretty stupid not to add some form of capabilities negotiation, at
least, so we can avoid having the same drama next time. In the process we'd
probably want to get proper STARTTLS-like support in place to avoid the
disconnect/reconnect dance when connecting to a host that doesn't support
SSL.

So this isn't quite as simple as just adding compression if both client and
server support it. But I do think it's very worthwhile, and that we've been
relying on a pretty sad cop-out for some time by telling people to use SSL
protocol layer compression.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services