Re: Extensibility of the PostgreSQL wire protocol

> On 19 Feb 2021, at 14:48, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:
>
> For example, there has been discussion elsewhere about integrating connection pooling into the server itself. For that, you want to have a custom process that listens for incoming connections, and launches backends independently of the incoming connections. These hooks would not help with that.
>

Not clear how the connection polling in the core is linked to discussing pluggable wire protocols.

> Similarly, if you want to integrate a web server into the database server, you probably also want some kind of connection pooling. A one-to-one relationship between HTTP connections and backend processes doesn't seem nice.
>

HTTP/2 is just a protocol, not unlike fe/be that has a one-to-one relationship to backend processes as it stands. It shuttles data back and forth in query/response exchanges, and happens to be used by web servers and web browsers, among other things. My mentioning of it was simply an example I can speak of from experience, as opposed to speculating. Could have brought up any other wire protocol if I had experience with it, say MQTT.

To make it clear, “a pluggable wire protocol” as discussed here is a set of rules that defines how data is transmitted: what the requests and responses are, and how is the data laid out on the wire, what to do in case of error, etc. Nothing to do with a web server; why would one want to integrate it in the database, anyway?

The intended contribution to the discussion of big picture of pluggable wire protocols is that there are significant use cases where the protocol choice is restricted on the client side, and allowing a pluggable wire protocol on the server side brings tangible benefits in performance and architectural simplification. That’s all. The rest were supporting facts that hopefully can also serve as a counterpoint to “pluggable wire protocol is primarily useful to make Postgres pretend to be Mysql."

Protocol conversion HTTP/2<—>FE/BE on the connection pooler already brings a lot of the mentioned benefits, and I’m satisfied with it. Beyond that I’m simply supporting the idea of pluggable protocols as experience so far allows me to see advantages that might sound theoretical to someone who never tried this scenario in production.

Glad to offer a couple of examples where I see potential for performance gains for having such a wire protocol pluggable in the core. Let me know if you want me to elaborate.

> Querying multiple databases over a single connection is not possible with the approach taken here.

Indeed, querying multiple databases over a single connection is something you need a proxy for and a different client protocol from fe/be. No need to mix that with the talk about pluggable wire protocol.

My mentioning of it was in the sense “a lot of LoB backend code is nothing more than a bloated protocol converter that happens to also allow connecting to multiple databases from a single client connection => letting the client speak to the database [trough a proxy in this case] removed the bloated source of latency but kept the advantages.”

--
Damir