7.4 COPY BINARY Format Change

From: Lee Kindness <lkindness(at)csl(dot)co(dot)uk>
To: pgsql-hackers(at)postgresql(dot)org
Cc: Lee Kindness <lkindness(at)csl(dot)co(dot)uk>, pgsql-patches(at)postgresql(dot)org
Subject: 7.4 COPY BINARY Format Change
Date: 2003-08-01 15:12:37
Message-ID: 16170.33637.772495.67561@kelvin.csl.co.uk
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches

Guys,

I've been testing 7.4 against an application today. Recompiled
everything against the new libraries. However the application includes
code which builds up a bulkload file in the documented 7.1 format. The
documentation for COPY goes on at length about the format being
forward compatible...

However of course it's changed in 7.4 for the following minor reasons:

1. Header is now PGCOPY\n\377\r\n\0 rather than PGBCOPY\n\377\r\n\0

2. Integers within the header (but not the data itself) are now in
network order, rather than native order

3. The "integer layout field" has disappeared.

4. typlen is now an int32 rather than an int16 plus an additional
int32 if a varlen type.

I've attached a patch which lets COPY read in the 7.1 format. However
i'm not convinced this is the right way to go - I think the format
which is output by 7.4 should be identical to the 7.1 format. The
"integer layout field" could be used to determine if byteswapping is
required on reading. The other changes seem to be unnecessary? If the
typlen change is kept it should be flagged in the flags field rather
than requiring a completely new format - this would allow old readers
to gracefully fail and old & new files to be read in by 7.4...

It's extremely counter-productive to break backward compatibility for
such whimsical changes! This will hurt those updating to 7.4 once it
is released...

So, I expect the patch below to be rejected - I'll happily rework the
patch to revert 7.4 to a version of the 7.1 format which results in
the same feature gain but without forfeiting backward
compatibility. Let me know.

Thanks. Lee.

Attachment Content-Type Size
unknown_filename text/plain 9.1 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2003-08-01 15:52:26 Re: contrib compilation probs
Previous Message scott.marlowe 2003-08-01 15:09:53 Re: ACCESSING POST GRESQL DATABASE THRU MFCOBOL

Browse pgsql-patches by date

  From Date Subject
Next Message Peter Eisentraut 2003-08-01 16:19:11 Re: [pgsql-rus] Russian NLS update: pg_dump, libpq,
Previous Message Gavin Sherry 2003-08-01 03:31:20 Re: Numeric version of factorial()