Re: Reduce heap tuple header size

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Tom, I have reviewed your objections:
>
> Thanks.
>
> > Is there something I am missing?
>
> My principal objection to this is that I do not like piecemeal breakage
> of pg_upgrade, disk page examination tools, etc. There are other things

What do we have, pg_upgrade and Red Hat's disk dump tool, right? (I
forgot the name.)

> that have been proposed that would require changes of page header or
> tuple header format, and I would prefer to see them bundled up and done
> as one big change (in some future revision; it's probably too late for
> 7.3). "Disk format of the week" is not an idea that appeals to me.

This is part of the "do it perfect or don't do it" logic that I usually
disagree with. We all agree we have a tuple header that is too large.
Here we have a chance to reduce it by 11% on most platforms, and as a
downside, we will not have a working pg_upgrade. What do you think the
average user will choose?

> I know as well as you do that pg_upgrade compatibility is only
> interesting if there *is* a pg_upgrade, which very possibly won't
> happen for 7.3 anyway --- but I don't like foreclosing the possibility

I am not sure what needs to be done for pg_upgrade for 7.3 (does anyone
else?), but if you suspect it will not work anyway, why are you trying
to save it for 7.3? (I think the problem with pg_upgrade in 7.2 was the
inability to create pg_clog files to match the new transaction counter.)

> for a marginal win. And this is definitely a marginal win. Let's
> try to batch up enough changes to make it worth the pain.

Marginal? Seems like a pretty concrete win to me in disk space.

Also, he has offered to write a conversion tool. If he does that, maybe
he can improve pg_upgrade if needed.

> In case you are wondering what I am talking about, here is an
> off-the-cuff list of things that I'd prefer not to do one at a time:
>
> * Version identifier words in page headers

Yes, we need that, and in fact should do that in 7.3 if we accept this
patch. This may make future upgrades easier. In fact, I wonder if we
could place the page version number in such a way that old pre-7.3 pages
could be easily identified, i.e. place version number 0xAE in an offset
that used to hold a values that were always less than 0x80.

> * CRCs in page headers
> * Replication and/or point-in-time recovery might need additional
> header fields similar to those used by WAL
> * Restructuring of line pointers
> * Making OIDs optional (no wasted space if no OID)
> * Adding a tuple version identifier to tuples (for DROP COLUMN etc)
> * Restructuring index tuple headers (remove length field)
> * Fixing array storage to support NULLs inside arrays
> * Restoring time travel on some optional basis
>
> Some of these may never happen (I'm not even in favor of all of them)
> but it's certain that we will want to do some of them. I don't want to
> take the same hit over and over when some intelligent project management
> would let us take it just once or twice.

Yes, there are some good ones there, but the idea that somehow they are
all going to hit in the same release seems unlikely. I say let's do
some now, some later, and move ahead.

Ultimately, I think we need to add smarts to the backend to read old
formats. I know it is a pain, but I see no other options. pg_upgrade
always will have trouble with certain changes.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026