Re: Page-level version upgrade (was: Block-level CRC checks)

On Tue, Dec 01, 2009 at 10:34:11PM -0500, Bruce Momjian wrote:
> Robert Haas wrote:
> > The key issue, as I think Heikki identified at the time, is to
> > figure out how you're eventually going to get rid of the old
> > pages. He proposed running a pre-upgrade utility on each page to
> > reserve the right amount of free space.
> >
> > http://archives.postgresql.org/pgsql-hackers/2008-11/msg00208.php
>
> Right. There were two basic approaches to handling a patch that
> would expand when upgraded to the new version --- either allow the
> system to write the old format, or have a pre-upgrade script that
> moved tuples so there was guaranteed enough free space in every page
> for the new format. I think we agreed that the later was better
> than the former, and it was easy because we don't have any need for
> that at this time. Plus the script would not rewrite every page,
> just certain pages that required it.

Please forgive me for barging in here, but that approach simply is
untenable if it requires that the database be down while those pages
are being found, marked, moved around, etc.

The data volumes that really concern people who need an in-place
upgrade are such that even

dd if=$PGDATA of=/dev/null bs=8192 # (or whatever the optimal block size would be)

would require *much* more time than such people would accept as a down
time window, and while that's a lower bound, it's not a reasonable
lower bound on the time.

If this re-jiggering could kick off in the background at start and
work on a running PostgreSQL, the whole objection goes away.

A problem that arises for any in-place upgrade system we do is that if
someone's at 99% storage capacity, we can pretty well guarantee some
kind of catastrophic failure. Could we create some way to get an
estimate of space needed, given that the system needs to stay up while
that's happening?

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate