Re: Prototype: In-place upgrade v02

Heikki Linnakangas wrote:
> > 4) This patch contains more topics for decision. First is general if
> > this approach is acceptable.
>
> I don't like the invasiveness of this approach. It's pretty invasive
> already, and ISTM you'll need similar switch-case handling of all data
> types that have changed the internal representation as well.
>
> We've talked about this before, so you'll remember that I favor teh
> approach is to convert the page format, page at a time, when the pages
> are read in. I grant you that there's non-trivial issues with that as
> well, like if the converted data takes more space and don't fit in the
> page anymore.

I 100% agree with Heikki here; having the conversion spill out into the
main backend is very expensive and adds lots of complexity. The only
argument for the Zdenek's conversion spill appoach is that it allows
conversion to happen at a more natural time than when the page is read
in, but frankly I think the conversion needs are going to be pretty
limited and are better done in a localized way at page read-in time.

As far as the page not fitting after conversion, what about some user
command that will convert an entire table to the new format if page
expansion fails.

> I wonder if we could go with some sort of a hybrid approach? Convert the
> whole page when it's read in, but if it doesn't fit, fall back to
> tricks like loosening the alignment requirements on platforms that can
> handle non-aligned data, or support a special truncated page header,
> without pd_tli and pd_prune_xid fields. Just a thought, not sure how
> feasible those particular tricks are, but something along those lines..
>
> All in all, though. I find it a bit hard to see the big picture. For
> upgrade-in-place, what are all the pieces that we need? To keep this
> concrete, let's focus on PG 8.2 -> PG 8.3 (or are you focusing on PG 8.3
> -> 8.4? That's fine with me as well, but let's pick one) and forget
> about hypothetical changes that might occur in a future version. I can see:
> 1. Handling page layout changes (pd_prune_xid, pd_flags)
> 2. Handling tuple header changes (infomask2, HOT bits, combocid)
> 3. Handling changes in data type representation (packed varlens)
> 4. Toast chunk size
> 5. Catalogs
>
> After putting all those together, how large a patch are we talking
> about, and what's the performance penalty then? How much of all that
> needs to be in core, and how much can live in a pgfoundry project or an
> extra binary in src/bin or contrib? I realize that none of us have a
> crystal ball, and one has to start somewhere, but I feel uneasy
> committing to an approach until we have a full plan.

Yes, another very good point.

I am ready to focus on these issues for 8.4; all this needs to be
fleshed out, perhaps on a wiki. As a starting point, what would be
really nice is to start a wiki that lists all data format changes for
every major release.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +