Re: pg_upgrade project: high-level design proposal of in-place upgrade facility

"Serguei A. Mokhov" <mokhov(at)cs(dot)concordia(dot)ca> writes:
> Comments are very welcome, especially _*CONSTRUCTIVE*_...

This is fundamentally wrong, because you are assigning the storage
manager functionality that it does not have. In particular, the
storage manager knows nothing of the contents or format of the files
it is managing, and so you can't realistically expect to use the smgr
switch as a way to support access to tables with different internal
formats. The places that change in interesting ways across versions
are usually far above the smgr switch.

I don't believe in the idea of incremental "lazy" upgrades very much
either. It certainly will not work on the system catalogs --- you have
to convert those in a big-bang fashion, because how are you going to
find the other stuff otherwise? And the real problem with it IMHO is
that if something goes wrong partway through the process, you're in
deep trouble because you have no way to back out. You can't just revert
to the old version because it won't understand your data, and your old
backups that are compatible with it are now out of date. If there are
going to be any problems, you really need to find out about them
immediately while your old backups are still current, not in a "lazy"
fashion.

The design we'd pretty much all bought into six months ago involved
being able to do in-place upgrades when the format/contents of user
relations and indexes is not changing. All you'd have to do is dump
and restore the schema data (system catalogs) which is a reasonably
short process even on a large DB, so the big-bang nature of the
conversion isn't a problem. Admittedly this will not work for every
single upgrade, but we had agreed that we could schedule upgrades
so that the majority of releases do not change user data. Historically
that's been mostly true anyway, even without any deliberate attempt to
group user-data-changing features together.

I think the last major discussion about it started here:
http://archives.postgresql.org/pgsql-hackers/2003-12/msg00379.php
(I got distracted by other stuff and never did the promised work,
but I still think the approach is sound.)

regards, tom lane