Re: RPMS for 7.3 beta.

On Tuesday 17 September 2002 11:51 pm, Tom Lane wrote:
> Lamar Owen <lamar(dot)owen(at)wgcr(dot)org> writes:
> >> How does pg_upgrade work?
> > [ pretty good description ]

> You missed a key point, which is that pg_upgrade does not even try to
> cope with version-to-version system catalog changes. It assumes it can
> use pg_dump to dump and reload the database schema. So there is no
> hope, ever, that it will be more reliable than pg_dump. All pg_upgrade
> tries to do is short-circuit the moving of the bulk data.

Yes, this is a key point and one that shouldn't be overlooked. If the
metadata belonging to the user's data didn't have to be pg_dumped, but was
decoupled somewhat from the system metadata about types, operators, classes,
and the like, the schema (great, another overloaded term) wouldn't need
dumping but would travel with its data.

> The bald fact of the matter is that we are still a good ways away from
> the point where we might be willing to freeze the system catalogs.

Not talking about a freeze. Talking about separation of system/feature
metadata from user metadata that wouldn't change in the upgrade anyway --
table names, fields, user types, views, triggers, etc, that belong to this
database and not to the installation as a whole. If columns need changed or
added to the user data's metadata, have the upgrade script run the
appropriate ALTER commands and UPDATES necessary. The hard parts, I know,
are the details behind the broad 'appropriate'.

> PG
> is evolving and improving by a substantial amount with every release,
> and the implication of that is that there *will* be some upgrade pain.

Why is it a given conclusion? It should not be axiomatic that 'there *will*
be upgrade pain if we improve our features.' That's fatalistic.

We have innovative solutions in PostgreSQL that solve some pretty hairy
problems. WAL. MVCC. The subselect code (made my day when I heard about
that one -- but then had to wait seven months before Red Hat saw fit to
provide an RPM that I wasn't expecting.....the other reason I began RPM
building, even though it was two cycles later before I got up the nerve to
tackle it...). The PL's. Foreign keys. TOAST (now that's a prime example of
a 'sideways' solution to a head-on problem).

This is just a different challenge: how to keep the loosely dynamic system
catalog structure while at the same time allowing the possibility of smooth
data migration so people can more easily take advantage of the improved
system catalog structure. And yes I know that such a change is not for 7.3.
Too late for that, and maybe too late for 7.4 too.

But unlike Bruce I winced at Oliver's last line -- it hit a little too close
to home and to many multitudes of bug reports and nastygrams directed my way
for something I have tried to kludge around in the past. Yes, nastygrams, in
the grand old alt.flame tradition. When you maintain RPM's, you find
yourself the point man for the entire project in some people's eyes. The bug
report about my RPM's trashing a fellow's RPM database was an extreme example
of that. I get two-three dozen e-mails a week that I redirect to the web
site and/or the mailing lists. I'm sure Oliver is nodding his head in
understanding on this one.

I don't think seamless upgrading is a pipe dream. And I think that dismissing
it out of hand as 'impossible' is a self-fulfilling prophecy.

But I do think it won't work well if it's just tacked-on.

But, like Tom, I really don't have more of an answer than that. I do
understand pg_upgrade much better now, though.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11