pg_upgrade: convert on read is dead end

We discussed many times what on-line storage upgrade approach is best. Two of
them were still in the game. One is "read all formats, write new only" and
second is "convert on read". The first one is general and complex solution which
needs lot of changes in the code. Second one is not so intrusive, but it has
another problems.

The main problem with second solution until yesterday was that data could
overlap a new page size. Second much bigger problem which I found yesterday is
TOAST table and TOASTed arrays and composite data types.

PostgreSQL stores all table's external data in one TOAST table and stored data
does not contains any clue about datatype. When postgreSQL reads TOSTEed value
then there is not possible detect what type it is and perform varlena conversion
on composite datatypes or arrays.

It could be converted in detoast_datum function but it means that datum have to
be retoasted and store back on new pages. The old page MUST keep in old format
because any page conversion lost information about version and different
datatypes can be store on one page PosgreSQL.

By my opinion, this issue completely kill convert on read solution and only
"read all formats..." solution is right one.

Comments, ideas?

Zdenek

--
Zdenek Kotala Sun Microsystems
Prague, Czech Republic http://sun.com/postgresql