Re: GUC_REPORT for protocol tunables was: Re: Optimize binary serialization format of arrays with fixed size elements

On 01/25/2012 06:40 PM, Tom Lane wrote:
> Marko Kreen<markokr(at)gmail(dot)com> writes:
>> On Wed, Jan 25, 2012 at 10:23:14AM -0500, Tom Lane wrote:
>>> Huh? How can that work? If we decide to change the representation of
>>> some other "well known type", say numeric, how do we decide whether a
>>> client setting that bit is expecting that change or not?
>
>> It sets that bit *and* version code - which means that it is
>> up-to-date with all "well-known" type formats in that version.
>
> Then why bother with the bit in the format code? If you've already done
> some other negotiation to establish what datatype formats you will
> accept, this doesn't seem to be adding any value.
>
>> Basically, I see 2 scenarios here:
>
>> 1) Client knows the result types and can set the
>> text/bin/version code safely, without further restrictions.
>
>> 2) There is generic framework, that does not know query contents
>> but can be expected to track Postgres versions closely.
>> Such framework cannot say "binary" for results safely,
>> but *could* do it for some well-defined subset of types.
>
> The hole in approach (2) is that it supposes that the client side knows
> the specific datatypes in a query result in advance. While this is
> sometimes workable for application-level code that knows what query it's
> issuing, it's really entirely untenable for a framework or library.
> The only way that a framework can deal with arbitrary queries is to
> introduce an extra round trip (Describe step) to see what datatypes
> the query will produce so it can decide what format codes to issue
> ... and that will pretty much eat up any time savings you might get
> from a more efficient representation.

This is pretty much what jdbc driver already does, since it does not have
100% coverage of even current binary formats. First time you execute a
query it requests text encoding, but caches the Describe results. Next
time it sets the binary bits on all return columns that it knows how to
decode.

> You really want to do the negotiation once, at connection setup, and
> then be able to process queries without client-side prechecking of what
> data types will be sent back.

I think my original minor_version patch tried to do that. It introduced a
per-connection setting for version. Server GUC_REPORTED the maximum supported
minor_version but defaulted to the baseline wire format.
The jdbc client could bump the minor_version to supported higher
value (error if value larger than what server advertised).

A way was provided for the application using jdbc driver to
override the requested minor_version in the rare event that something
broke (rare, because jdbc driver generally does not expose the
wire-encoding to applications).

Now if pgbounce and other pooling solutions would reset the minor_version
to 0 then it should work.

Scenarios where other end is too old to know about the minor_version:
Vserver>>Vlibpq => client does nothing -> use baseline version
Vlibpq>>Vserver => no supported_minor_version in GUC_REPORT -> use baseline

Normal 9.2+ scenarios:
Vserver>Vlibpq => libpg sets minor_version to largest that is supports
-> libpq requested version used
Vlibpq>Vserver => libpg notices that server supported value is lower than
its so it sets minor_version to server supported value
-> server version used

For perl driver that exposes the wire format to application by default
I can envision that the driver needs to add a new API that applications
need to use to explicitly bump the minor_version up instead of defaulting
to the largest supported by the driver as in jdbc/libpg.

The reason why I proposed a incrementing minor_version instead of bit flags
of new encodings was that it takes less space and is easier to document and
understand so that exposing it to applications is possible.

But how to handle postgres extensions that change their wire-format?
Maybe we do need to have "oid:minor_version,oid:ver,oid_ver" as the
negotiated version variable?

-Mikko