Re: GUC_REPORT for protocol tunables was: Re: Optimize binary serialization format of arrays with fixed size elements

On Wed, Jan 25, 2012 at 2:29 PM, Marko Kreen <markokr(at)gmail(dot)com> wrote:
>> well, I see the following cases:
>> 1) Vserver > Vapplication: server downgrades wire formats to
>> applications version
>> 2) Vapplication > Vlibpq > Vserver: since the application is
>> reading/writing formats the server can't understand, an error should
>> be raised if they are used in either direction
>> 3) Vlibpq >= VApplication > Vserver: same as above, but libpq can
>> 'upconvert' low version wire format to application's wire format or
>> error otherwise.
>
> I don't see why you special-case libpq here.  There is no reason
> libpq cannot pass older/newer formats through.  Only thing that
> matters it parser/formatter version.  If that is done in libpq,
> then app version does not matter.  If it's done in app, then
> libpq version does not matter.

Only because if the app is targeting wire format N, but the server can
only handle N-1, libpq has the opportunity to fix it up. That's could
be just over thinking it though.

>> By far, the most common cause of problems (both in terms of severity
>> and frequency) is case #1.  #3 allows a 'compatibility mode' via
>> libpq, but that comes at significant cost of complexity since libpq
>> needs to be able to translate wire formats up (but not down).  #2/3 is
>> a less common problem though as it's more likely the application can
>> be adjusted to get up to speed: so to keep things simple we can maybe
>> just error out in those scenarios.
>
> I don't like the idea of "conversion".  Instead either client
> writes values through API that picks format based on server version,
> or it writes them for specific version only.  In latter case it cannot
> work with older server.  Unless the fixed version is the baseline.

ok. another point about that: libpq isn't really part of the solution
anyways since there are other popular fully native protocol consumers,
including (and especially) jdbc, but also python, node.js etc etc.

that's why I was earlier insisting on a protocol bump, so that we
could in the new protocol force application version to be advertised.
v3 would remain caveat emptor for wire formats but v4 would not.

>> In the database, we need to maintain outdated send/recv functions
>> basically forever and as much as possible try and translate old wire
>> format data to and from newer backend structures (maybe in very
>> specific cases that will be impossible such that the application is
>> SOL, but that should be rare).  All send/recv functions, including
>> user created ones need to be stamped with a version token (database
>> version?).  With the versions of the application, libpq, and all
>> server functions, we can determine all wire formats as long as we
>> assume the application's targeted database version represents all the
>> wire formats it was using.
>>
> My good ideas stop there: the exact mechanics of how the usable set of
>> functions are determined, how exactly the adjusted type look ups will
>> work, etc. would all have to be sorted out.  Most of the nastier parts
>> though (protocol changes notwithstanding) are not in libpq, but the
>> server.  There's just no quick fix on the client side I can see.
>
> It does not need to be complex - just bring the version number to
> i/o function and let it decide whether it cares about it or not.
> Most functions will not..  Only those that we want to change in
> compatible manner need to look at it.

well, maybe instead of passing version number around, the server
installs the proper compatibility send/recv functions just once on
session start up so your code isn't littered with stuff like
if(version > n) do this; else do this;?

> But seriously - on-wire compatibility is good thing, do not fear it...

sure -- but for postgres I just don't think it's realistic, especially
for the binary wire formats. a json based data payload could give it
to you (and I'm only half kidding) :-).

merlin