Re: [PATCHES] libpq type system 0.9a

On Tue, Apr 8, 2008 at 10:15 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> The biggie is floating-point format. IEEE standard is not quite
> universal ... and even for platforms that fully adhere to that standard,
> it's not entirely clear that we get the endianness issues correct.
> There used to be platforms where FP and integer endianness were
> different; is anyone sure that's no longer the case?

If somebody can provide a machine to test this, we will code for
this...and hope to catch any other cases with the regression test. As
it happens, we have a fairly large array of old hardware to test on,
pa-risc, power, mips, etc. We are still setting them up to test some
of our own stuff, but we will put things through the ringer, no doubt
(we will also put some machines on the buildfarm).

> But I'll agree that cross-version hazards are a much more clear and
> present danger. We've already broken binary compatibility at least once
> since the current binary-I/O system was instituted (intervals now have
> three fields not two) and there are obvious candidates for future
> breakage, such as text locale/encoding support.

There are other cases, for example money getting promoted to 64 bit.
Here is an even better example. When looking at the array_send/recv
functions, I noticed that a good candidate for wire format
optimization would be to trim out the length columns for arrays where
the type is fixed length and has no nulls (great win for arrays of
ints and such). This change would involve modifying some of the most
complicated parts of the proposed patch...these things are going to
happen..

So, this problem has to be dealt with, so lets look at some ways of handling it:
*) only allow direct binary mapping between client/server between same
version, or 'trusted' version. (lame, but easy).

*) as above, but break it down to type level. similar to catalog bump
for types. basically, if the client is not the same version as the
server, there is a negotiation so that the client and server agree
which types if any are unsafe to send. Better, and still fairly easy,
since we are still sidestepping backwards compatibility, but in a more
fine-grained way.

*) as above, but start (from here on in), building in logical to grok
older versions of binary formats, to a certain suitable time frame.
This is the biggest headache in terms of long term maintenance, since
it puts lots of checks into the code, but is the only way of
completely isolating clients from the issue. We have already chosen
this path for a couple of times (money for example), as things evolved
over the 8.3 cycle.

While format changes do happen, this is a farily uncommon thing. I
suspect that on average there about 3-5 format changes per major
release...this would ad up to about 20 or so version specific checks
in the in/out procedures should we go down the compatibility road.
Not great, but not completely terrible either. In terms of everything
else we had to do, this was a minor nuisance at worst. We are
completely open to how this particular issue should be tackled. We
didn't feel particularly rushed, since the biggest problems wouldn't
manifest until 8.5 at the earliest...

Another issue that you have not addressed, is that (rightly or
wrongly), we are adding another non-trivial step in terms of
introducing new built in types. It's already a fairly large checklist
as it is (enums acquired send/recv almost too late for the 8.3 cycle).
One of my least favorite things about the proposal is the amount code
duplication in libpq and the server...I think that this can be fixed
in the long run, eliminating this 'extra' step by consolidating the
client and server in/out routines into a consolidated library.

merlin