From: | Dave Cramer <davecramer(at)gmail(dot)com> |
---|---|
To: | PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Binary support for pgoutput plugin |
Date: | 2019-06-08 23:41:34 |
Message-ID: | CADK3HHLr-iQGiG0aFE_kNAd_R0obvSUAGMf71kEmAt-kz+2T7A@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
This should have gone to hackers as well
---------- Forwarded message ---------
From: Dave Cramer <davecramer(at)gmail(dot)com>
Date: Sat, Jun 8, 2019, 6:41 PM
Subject: Re: Binary support for pgoutput plugin
To: Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
On Sat, Jun 8, 2019, 6:27 PM Tomas Vondra, <tomas(dot)vondra(at)2ndquadrant(dot)com>
wrote:
> On Fri, Jun 07, 2019 at 06:01:12PM -0700, Andres Freund wrote:
> >Hi,
> >
> >On 2019-06-07 20:52:38 -0400, Chapman Flack wrote:
> >> It seems they had ended up designing a whole 'nother "protocol level"
> >> involving queries wrapping their results as JSON and an app layer that
> >> unwraps again, after trying a simpler first approach that was foiled by
> the
> >> inability to see into arrays and anonymous record types in the
> 'describe'
> >> response.
> >
> >I suspect quite a few people would have to have left the projectbefore
> >this would happen.
> >
> >
> >> I thought, in a new protocol rev, why not let the driver send additional
> >> 'describe' messages after the first one, to drill into structure of
> >> individual columns mentioned in the first response, before sending the
> >> 'execute' message?
> >>
> >> If it doesn't want the further detail, it doesn't have to ask.
> >>
> >> > And then we suddenly need tracking for all these, so we don't always
> >> > send out that information when we previously already did
> >>
> >> If it's up to the client driver, it can track what it needs or already
> has.
> >
> >> I haven't looked too deeply into the replication protocol ... it happens
> >> under a kind of copy-both, right?, so maybe there's a way for the
> receiver
> >> to send some inquiries back, but maybe in a windowed, full-duplex way
> where
> >> it might have to buffer some incoming messages before getting the
> response
> >> to an inquiry message it sent.
> >
> >That'd be a *lot* of additional complexity, and pretty much prohibitive
> >from a performance POV. We'd have to not continue decoding on the server
> >side *all* the time to give the client a chance to inquire additional
> >information.
> >
>
> I kinda agree with this, and I think it's an argument why replication
> solutions that need such additional metadata (e.g. because they have no
> database to query) should not rely on pgoutput but should invent their own
> decoding plugin. Which is why it's a plugin.
>
So the reason we are discussing using pgoutput plugin is because it is part
of core and guaranteed to be in cloud providers solutions. I'm trying to
find a balance here of using what we have as opposed to burdening core to
take on additional code to take care of. Not sending the metadata is not a
deal breaker but i can see some value in it.
Dave
>
>
From | Date | Subject | |
---|---|---|---|
Next Message | Andres Freund | 2019-06-09 00:09:44 | Re: Binary support for pgoutput plugin |
Previous Message | Tomas Vondra | 2019-06-08 22:27:34 | Re: Binary support for pgoutput plugin |