Re: HTTP Frontend? (and a brief thought on materialized views)

On Fri, Mar 30, 2012 at 10:55 AM, Daniel Farina <daniel(at)heroku(dot)com> wrote:

> On Thu, Mar 29, 2012 at 6:37 PM, Dobes Vandermeer <dobesv(at)gmail(dot)com>
> wrote:
> > On Fri, Mar 30, 2012 at 3:59 AM, Daniel Farina <daniel(at)heroku(dot)com>
> wrote:
> >>
> >> On Thu, Mar 29, 2012 at 12:57 PM, Daniel Farina <daniel(at)heroku(dot)com>
> >> wrote:
> >>
> >> More technical concerns:
> >> > * Protocol compression -- but a bit of sand in the gears is *which*
> >> > compression -- for database workloads, the performance of zlib can be
> >> > a meaningful bottleneck.
> >
> > I think if performance is the issue, people should use the native
> protocol.
>
> No. I do not think so. I think a reasonable solution (part of what MS
> is actually proposing to the IETF) is to make compression optional, or
> have clients support an LZ77 family format like Snappy. I get the
> impression that SPDY is waffling a little on this fact, it mandates
> compression, and definitely zlib, but is less heavy handed about
> pushing, say Snappy. Although I can understand why a
> Google-originated technology probably doesn't want to push another
> Google-originated implementation so hard, it would have been
> convenient for me for Snappy to have become a more common format.
>
> > Isn't the URL good enough (/databases/<dbname>) or are you talking about
> > having some some of "virtual host" setup where you have multiple sets of
> > databases available on the same port?
>
> Virtual hosts. Same port.
>

In that case, the frontend would not be tied to a specific PostgreSQL
server, then? I think initially this might complicate things a bit, and
you could solve it by putting an HTTP proxy in front to do the virtual
hosts for you.

> >> > * A standard frame extension format. For example, last I checked
> >> > Postgres-XC, it required snapshot information to be passed, and it'd
> >> > be nice that instead of having to hack the protocol that they could
> >> > attach an X-Snapshot-Info header, or whatever. This could also be a
> >> > nice way to pass out-of-band hint information of all sorts.
> >
> >
> > I am sorry to admit I don't understand the terms "frame extension
> format" or
> > "Postgres-XC" in this paragraph ... help?
>
> It'd be nice if it wasn't
> necessary to do that and they had a much easier path to multiplex
> additional information into the connection.
>

Ah, I get it - you want a way to add some extra information to the protocol
in a backwards compatible way. HTTP (and SPDY) provides a "standard" way
to do that. Makes sense.

>
> For my own purposes, I'm intensely interest in lacing the connection with:
>
> * EXPLAIN ANALYZE
> * Partition IDs
> * Read-only vs. Write workload

I'll make a note of these and hash out the details a bit more once there's
something working to add them to.

> > As for SPDY I can see how it may be helpful but as it is basically a

> different way to send HTTP requests
>
> I think SPDY or like-protocols [...] give a crisp treatment to
> interactive, stateful workloads involving
> back-and-forth between client and server with multiplexing, fixing
> some problems with the hacks in HTTP-land from before.
>

It sounds like at some level you're really talking about replacing the
built-in protocol with SPDY because SPDY is possibly a better baseline than
updating the existing protocol. That's an interesting idea, I think this
project could evolve in that direction if there's demand for it.