Re: State of Beta 2

Manfred Koizar wrote:
> On Thu, 18 Sep 2003 12:11:18 -0400, Lamar Owen <lowen(at)pari(dot)edu> wrote:
>>Marc G. Fournier wrote:
>>>[...] upgrading is a key feature [...]

>>a migration tool
>>that could read the old format _without_a_running_old_backend_ [...]
>>the new backend is powerless to recover the old data.
>> OS upgrades [...], FreeBSD ports upgrades, and RPM
>>upgrades are absolutely horrid at this point. [...]
>>[censored] has a better system than we
>>[...] the pain of upgrading [...]
>>*I* should complain about a ramble? :-)

> Lamar, I *STRONGLY* agree with almost everything you say here and in
> other posts, except perhaps ...

> You et al. seem to think that system catalog changes wouldn't be a
> problem if only we could avoid page format changes. This is not
> necessarily so. Page format changes can be handled without much
> effort, if

No, I'm aware of the difference, and I understand the issues with
catalog changes. Tom and I, among others, have discussed this. We
talked about reorganizing the system catalog to separate the data that
typically changes with a release from the data that describes the user's
tables. It is a hard thing to do, separating this data.

> Oh, that's me, I think. I am to blame for the heap tuple header
> changes between 7.2 and 7.3;

It has happened at more than one version change, not just 7.2->7.3. I
actually was thinking about a previous flag day. So the plural still
stands.

> Later, in your "Upgrading rant" thread, I even posted some code
> (http://archives.postgresql.org/pgsql-hackers/2003-01/msg00294.php)
> Unfortunately this went absolutely unnoticed, probably because it
> looked so long because I fat-fingered the mail and included the code
> twice. :-(

I don't recall that, but I believe you. My antivirus software may have
flagged it if it had more than one . in the file name. But I may go
back and look at it. Again, I wasn't fingering 7.2->7.3 -- it has
happened more than once prior to that.

> A working pg_upgrade is *not* the first thing we need. What we need
> first is willingness to not break backwards compatibility.

To this I agree. But it must be done in stages, as Tom, Marc, and
others have already said (I read the rest of the thread before replying
to this message). We can't simply declare a catalog freeze (which you
didn't do, I know), nor can we declare an on-disk format change freeze.
We need to think about what is required to make upgrades easy, not
what is required to write a one-off upgrade tool (which each version of
pg_upgrade ends up being). Can the system catalog be made more
friendly? Is upgrading by necessity a one-step process (that is, can we
stepwise migrate tables as they are used/upgraded individually)? Can we
decouple the portions of the system catalogs that change from the
portions that give basic access to the user's data? That is, what would
be required to allow a backend to read old data tables? An upgrade tool
is redundant if the backend is version agnostic and version aware.

Look, my requirements are simple. I should be able to upgrade the
binaries and not lose access to my data. That's the bottom line.
--
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute