In-place upgrade: catalog side

From: Greg Smith <gsmith(at)gregsmith(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Cc: Zdenek Kotala <Zdenek(dot)Kotala(at)sun(dot)com>
Subject: In-place upgrade: catalog side
Date: 2008-12-03 07:21:53
Message-ID: Pine.GSO.4.64.0812022349130.28660@westnet.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Since this whole in-place upgrade thing is going nowhere until there's
also a good solution for ye olde "The database cluster was initialized
with CATALOG_VERSION_NO ..." error, I spent some time today touring
through what everybody else has done there and have some initial review
commentary and questions.

Zdenek provided the following code:
http://src.opensolaris.org/source/xref/sfw/usr/src/cmd/postgres/postgresql-upgrade/

The main thing you'll find there is a ksh script that handles most of the
upgrade, presuming there's no page format changes. It looks like it was
originally aimed at 8.1->8.2 upgrades (easy as long as you don't use
INET/CIDR in your database) and still has some hard-coded stuff from that
in there to clean up.

I spent some time reading the code to figure out exactly what it's doing,
but come to find there's an easier way to ramp up. It looks like this
script is a fairly straight shell port (presumably to make things more
flexible) of EDB's pg_migrator:
http://pgfoundry.org/projects/pg-migrator/

If you want to understand the basics of what happens in either program,
the pg_migrator download has a nice outline in doc/intro.txt The basic
concept seems workable, albeit a bit more brute-force than I was
expecting: don't bother trying to figure out what actually changed in the
catalog, just start with a new cluster, restore schema, then slam
renumbered data pages into it. At a high level it works like this:

-Run pg_dumpall against the old database to grab its schema
-Extract some catalog information from the old database
-Export the pg_control information
-Create a new cluster
-Copy the old pg_clog over
-With the help of pg_resetxlog, move over the timeline and other ids
-Restore the schema dump
-Update TOAST info
-Join the new database relid information against the old set so you can
easily map old and new relids for each relation
-Move the underlying database files from their old location (the original
relid) to the new one
-Adjust tablespace information

I'd like to start doing something constructive to help move this part
forward, so here's an initial set of questions I've got for mapping that
work out:

-A ksh script for something this critical is a non-starter on Windows in
particular, so a good plan would be to use Zdenek's script as a malleable
prototype, confirm it works, then update pg_migrator with anything it's
missing, right?

-Are there already any specific tasks done by Zdenek's script that are
already known to be unique to only its implementation? Eventually I
expect I'll figure that out for sure myself just by comparing the code,
was curious what the already known divergences were.

-There are 10 TODO items listed for the pg_migrator project, most or all
of which look like should be squashed before this is really complete.
Any chance somebody (Korry?) has an improved version of this floating
around beyond what's in the pgfoundry CVS already?

-Am I really the only person who is frantic that this program isn't being
worked on actively?

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Guillaume Smet 2008-12-03 07:36:53 Re: Simple postgresql.conf wizard
Previous Message Koichi Suzuki 2008-12-03 07:18:06 Re: Hot Standby (commit fest version - v5)