Re: Prototype: In-place upgrade v02

Zdenek Kotala wrote:
> Heikki Linnakangas napsal(a):
>> The patch seems to be missing the new htup.c file.
>
> I'm sorry. I attached new version which is synchronized with current
> head. I would like to say more comments as well.
>
> 1) The patch contains also changes which was discussed during July
> commit fest. - PageGetTempPage modification suggested by Tom
> - another hash.h backward compatible cleanup

It might be a good idea to split that into a separate patch. The sheer
size of this patch is quite daunting, even though the bulk of it is
straightforward search&replace.

> 2) I add tuplimits.h header file which contains tuple limits for
> different access method. It is not finished yet, but idea is to keep all
> limits in one file and easily add limits for different page layout
> version - for example replace static computing with dynamic based on
> relation (maxtuplesize could be store in pg_class for each relation).
>
> I need this header also because I fallen in a cycle in header dependency.
>
> 3) I already sent Page API performance result in
> http://archives.postgresql.org/pgsql-hackers/2008-08/msg00398.php
>
> I replaced call sequence PagetGetItemId, PageGetItemId with
> PageGetIndexTuple and PageGetHeapTuple function. It is main difference
> in this patch. PAgeGetHeap Tuple fills t_ver in HeapTuple to identify
> correct tupleheader version.
>
> It would be good to mention that PageAPI (and tuple API) implementation
> is only prototype without any performance optimization.

You mentioned 5% performance degradation in that thread. What test case
was that? What would be a worst-case scanario, and how bad is it?

5% is a pretty hefty price, especially when it's paid by not only
upgraded installations, but also freshly initialized clusters. I think
you'll need to pursue those performance optimizations.

> 4) This patch contains more topics for decision. First is general if
> this approach is acceptable.

I don't like the invasiveness of this approach. It's pretty invasive
already, and ISTM you'll need similar switch-case handling of all data
types that have changed the internal representation as well.

We've talked about this before, so you'll remember that I favor teh
approach is to convert the page format, page at a time, when the pages
are read in. I grant you that there's non-trivial issues with that as
well, like if the converted data takes more space and don't fit in the
page anymore.

I wonder if we could go with some sort of a hybrid approach? Convert the
whole page when it's read in, but if it doesn't fit, fall back to
tricks like loosening the alignment requirements on platforms that can
handle non-aligned data, or support a special truncated page header,
without pd_tli and pd_prune_xid fields. Just a thought, not sure how
feasible those particular tricks are, but something along those lines..

All in all, though. I find it a bit hard to see the big picture. For
upgrade-in-place, what are all the pieces that we need? To keep this
concrete, let's focus on PG 8.2 -> PG 8.3 (or are you focusing on PG 8.3
-> 8.4? That's fine with me as well, but let's pick one) and forget
about hypothetical changes that might occur in a future version. I can see:
1. Handling page layout changes (pd_prune_xid, pd_flags)
2. Handling tuple header changes (infomask2, HOT bits, combocid)
3. Handling changes in data type representation (packed varlens)
4. Toast chunk size
5. Catalogs

After putting all those together, how large a patch are we talking
about, and what's the performance penalty then? How much of all that
needs to be in core, and how much can live in a pgfoundry project or an
extra binary in src/bin or contrib? I realize that none of us have a
crystal ball, and one has to start somewhere, but I feel uneasy
committing to an approach until we have a full plan.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com