| From: | "Reuven M(dot) Lerner" <reuven(at)lerner(dot)co(dot)il> | 
|---|---|
| To: | pgsql-general(at)postgresql(dot)org | 
| Cc: | eli-d(at)nova(dot)co(dot)il | 
| Subject: | Hanging with pg_restore and large objects | 
| Date: | 2010-12-07 13:51:24 | 
| Message-ID: | 4CFE3BDC.9070206@lerner.co.il | 
| Views: | Whole Thread | Raw Message | Download mbox | Resend email | 
| Thread: | |
| Lists: | pgsql-general | 
Hi, everyone.  I'm working on a project that is using 8.3.0; among other 
things, I'm helping them to move to 9.0.  The project is running on 
Windows XP.
Someone from this project asked me earlier today why a particular 
database restore was taking a long time.  How long?  Well, it has been 
running for 1.5 days (yes, that's 36 hours).  The restore is running 
under Windows XP, and the backup was done using pg_dump into the 
custom/binary format.  The dumpfile was about 140 MB in size.
Using the Windows process monitor, we saw that pg_restore was using 
about 50 percent of the CPU, doing an enormous (about 60 billion, by 
this point) reads from the disk, but zero writes.  The dumpfile does 
contain a number of large (binary) objects, as well as a number of 
regular tables with integer and textual content.  The restore was run 
with the -a (data only) flag, on an empty database schema.
We tried to replicate this problem on another, similarly equipped 
machine, adding the -c (clean before restoring), -e (exit upon error), 
and -v (verbose) flags.  We saw that the restore hung (for about 30 
minutes, as of this writing) while loading one of the large objects from 
the restore.
We tried to use pg_restore on the dumpfile, but found that it hung when 
restoring the same large object.  It's not even close to the first large 
object, and I don't believe that it's the last one, either.
My guess is pg_dump in 8.3 is somehow causing a problem in the dumpfile 
on or around that large object.
So:
    * Is this a known problem on PostgreSQL 8.3, Windows, or the
      combination?
    * Is there an easy way to identify problems, corruption, and the
      like in our pg_dump file?
    * Should we be using a different type of dumpfile, such as text, to
      get around this problem for now?
    * Is there any obvious way to diagnose or work around this problem?
    * I don't believe that there's a way to tell either pg_dump or
      pg_restore to ignore objects with particular OIDs.  Am I right?
Thanks in advance for any help you can offer,
Reuven
-- 
Reuven M. Lerner -- Web development, consulting, and training
Mobile: +972-54-496-8405 * US phone: 847-230-9795
Skype/AIM: reuvenlerner
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Daniel Verite | 2010-12-07 14:12:38 | Re: Do we want SYNONYMS? | 
| Previous Message | Vick Khera | 2010-12-07 13:35:52 | Re: Do we want SYNONYMS? |