Re: Proposal: http2 wire format

> On 26 Mar 2018, at 16:56, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> Damir Simunic <damir(dot)simunic(at)wa-research(dot)ch> writes:
>>> On 26 Mar 2018, at 11:06, Vladimir Sitnikov <sitnikov(dot)vladimir(at)gmail(dot)com> wrote:
>>>> If anyone finds the idea of Postgres speaking http2 appealing
>
> TBH, this sounds like a proposal to expend a whole lot of work (much of it
> outside the core server, and thus not under our control) in order to get
> from a state of affairs where there are things we'd like to do but can't
> because of protocol compatibility worries, to a different state of affairs
> where there are things we'd like to do but can't because of protocol
> compatibility worries.

What do you mean by compatibility worries? Is it backward compatibility?

If so, I’m not suggesting we get rid of FEBE, but leave it as is and complement it with a widely understood and supported protocol, that in fact takes compatibility way more seriously than FEBE. Just leave v3 frozen. Seems like ultimate backward compatibility, no? Or am I missing something?

You likely know every possible use case for Postgres, which makes you believe that the status quo is the right way. Or maybe I didn’t flesh out my proposal enough for you to give it a chance. Either way, I just can’t figure out where would HTTP2 be the same as status quo or a step backward compared to FEBE. I can see you’re super-busy and dedicated, but if you can find the time to enlighten me beyond just waving the “compatibility” and “engineering” banners, I’d appreciate you endlessly.

> Why would forcing our data into a protocol
> designed for a completely different purpose, and which we have no control
> over, be a step forward?

What purpose do you see HTTP2 being designed for that is completely different from FEBE? Not being cynical, genuinely want to learn. (Oh, it’s my data, too; presently held hostage to the v3 protocol).

You mention twice loss of control--what exactly is the fear?

> How would that address the fundamental issue of
> inertia in multiple chunks of software (ie, client libraries and
> applications as well as the server)?
>

Is this inertia as in "our TODO list is years old and nobody’s doing anything about it"? If so, I posit here that using HTTP2 as the v4 protocol will lead to significant reduction of inertia. And that just because we’re talking HTTP2 and not some new obscure thing we invented.

The psychological and social aspects are not to be underestimated.

>> This proposal takes the stance that having HTTP2 wire protocol in place will enable wide experimentation with and implementation of many new features and content types, but is not concerned with the specifics of those.
>
> That reads to me as pie in the sky, and uninformed by any engineering
> reality. As an example, it's not the protocol's fault that database
> server processes are expensive to spin up; changing to a different
> protocol will do nothing to make them more lightweight. We've thought
> about various ways to amortize that cost, but they tend to fall foul of
> the fact that sessions are associated with TCP connections, which we can't
> transparently remake or reattach to a different endpoint process. HTTP2
> is not going to fix that, because it's still TCP based.

That reads to me as uninformed engineering reality. Just because you are encumbered with the worries of compatibility and stuck in the world of TCP, doesn’t mean it can’t be done.

You know what? HTTP2 just might fix it. Getting a new protocol into the core will force enough adjustments to the code to open the door for the next protocol on the horizon: QUIC, which happens to be UDP based, and might just be the ticket. At a minimum it will get significantly more people thinking about the possibility of reattaching sessions and doing all kinds of other things. Allowing multiple protocols is not very different from allowing a multitude of pl implementations.

Help me put HTTP2 in place, and I’ll bet you, within a few months someone will come up with a patch for QUIC. And then someone else will remember your paragraph above and say “hmm, let’s see…"

> I realize that
> webservers manage to have pretty lightweight sessions, but that's not a
> property of the protocol they use, it's a property of their internal
> architectures. We can't get there without a massive rewrite of the PG
> server --- one that would be largely independent of any particular way of
> representing data on the wire, anyway.
>

A smart outsider might come along, look at an ultra-fast web server, then look at Postgres and think, “Hmm, both speak HTTP2, but one is blazing fast, the other slow. Can I learn anything from the former to apply to the latter? Maybe I'll add another type of a backend that serves only a very very narrow use case, but makes it blazing fast?” Pie in the sky? Maybe. But isn’t it how it works today: lots of smart people chipping away in small increments?

Let’s not underestimate the effect of possibilities on mobilizing minds. Innovation is fueled by the power of possibilities. “Engineering reality” is not enough. HTTP2 is at least as good as FEBE, but it has infinitely more cachet than anything we can come up with. And that is super-important.

> We've certainly got issues that can't be solved without protocol changes.
> But starting from the assumption that HTTP2 solves our problems seems to
> me to be "Here's a hammer. I'm sure your problem must be a nail, because
> all problems are nails”.

Or maybe starting from the assumption that a small change will get a lot of people excited about solving those issues seems to me to be “ideas help start revolutions”?

Yes, I do happen to believe HTTP2 can solve a slice of current problems, and open the possibilities you didn’t have the time to think of. And yes, a protocol designed to transport data happens to look like a good hammer to nail data transfer problems. What are the odds of coming up with a better one?

Look, I keep trying to limit this to the smallest possible increment that I could think of. The choice is simply pragmatic. But that doesn’t make me a hipster fanboi of the protocol du jour, just because they are all doing it we should too.

There are three alternatives to the proposal: do nothing, make a few anemic changes to v3, or start a multiyear discussion on the design of the next protocol. And you’ll still converge to something like HTTP2 or QUIC.

It’s hard to move forward if you’re not focused. Doubly hard when you’re an outsider, and extra frustrating when you have the idea and the intuition, but it takes forever to learn everything. Someone with your experience and skills would get HTTP2 done in a couple of days, and have a ton of people well on their way to resolving all these issues that can’t be solved today. If I could have pulled off the coding all by myself already, i would have already done it. But I need you and everyone else here to help.

What would it take to convince you, or at least lend enough support to the idea to give it a chance?

Thanks,
Damir

>
> regards, tom lane
>