Re: [GENERAL] pg_upgrade problem

From: daveg <daveg(at)sonic(dot)net>
To: Bruce Momjian <bruce(at)momjian(dot)us>
Cc: hubert depesz lubaczewski <depesz(at)depesz(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [GENERAL] pg_upgrade problem
Date: 2011-09-06 02:17:28
Message-ID: 20110906021728.GK24583@sonic.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

On Mon, Sep 05, 2011 at 08:19:21PM -0400, Bruce Momjian wrote:
> daveg wrote:
> > > Can you tell me what table is showing this error? Does it happen during
> > > vacuum? Can you run a vacuum verbose to see what it is throwing the
> > > error on? Thanks.
> >
> > This was upgrading from 8.4.8 to 9.0.4. I don't have the running cluster
> > anymore, but I do have tar.gz archives of it and could probably find
> > 2TB free somewhere to restore it to if there is something useful to extract.
> >
> > However, I don't think this was toast related. Most of our rows are short and
> > have only int, float, and short text columns. These errors hit over 60
> > different tables mostly during the analyzes we ran immediately after the
> > upgrade. It also hit during select, insert and delete statements. We did not
> > run the db more than a few minutes as the damage was so extensive.
> >
> > As far as I can tell pg_upgrade never copied any pg_clog files from the
> > old cluster to the new cluster. I wish I had detected that before running
> > the remove_old_cluster.sh script.
>
> Wow, no clogs? That would make the system very confused. You can pull
> the clogs out of the old backup and move them over if the files don't
> already exist.

We don't have the old cluster after running delete_old_cluster.ch. We use
pg_dump for backup, so no clogs. We ended up restored 20 odd dbs totalling
2.1TB from the previous days pg_dumps.

If you review my original report I mentioned that there were only 2 clog
files in the new cluster both with ctime after the start of postgresql
after the upgrade. I did the upgrade for three hosts at the same time, the
others were fine. They have dozens of clogs dating back days before the
upgrade. The failing system had only 2 recent clog.

-dg

> --
> Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
> EnterpriseDB http://enterprisedb.com
>
> + It's impossible for everything to be true. +
>

--
David Gould daveg(at)sonic(dot)net 510 536 1443 510 282 0869
If simplicity worked, the world would be overrun with insects.

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Bruce Momjian 2011-09-06 02:35:03 Re: [GENERAL] pg_upgrade problem
Previous Message Bruce Momjian 2011-09-06 00:19:21 Re: [GENERAL] pg_upgrade problem

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2011-09-06 02:18:24 Re: tolower() identifier downcasing versus multibyte encodings
Previous Message Bruce Momjian 2011-09-06 02:14:42 Re: pg_ctl restart - behaviour based on wrong instance