Re: pg_dump incredibly slow dumping a single schema from a large db

From: Mike Roest <mike(dot)roest(at)replicon(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: pg_dump incredibly slow dumping a single schema from a large db
Date: 2012-03-30 16:40:44
Message-ID: CAE7ByhiseyD+Qk+oKYr_S==-hV5yrNoVYUnqAkoKPa7GKvssgg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

For sure I'll work on that now. One thing I noticed looking through the
pg_dump code based on the messages and the code one thing I noticed it
seems to be grabbing the full dependency graph for the whole db rather then
limiting it by the schema (not sure if limiting this would be possible)

This query returns 9843923 rows from the DB. So processing this seems like
it'll take quite a while.

I'll get a -pg build of pg_dump going here on a dev box so I can get you a
profile.

On Fri, Mar 30, 2012 at 10:18 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Mike Roest <mike(dot)roest(at)replicon(dot)com> writes:
> > This dump is currently taking around 8 minutes. While dumping the
> pg_dump
> > process is using 100% of one core in the server (24 core machine).
> Doing a
> > -v pg_dump I found that the following stages are taking the majority of
> the
> > time
>
> > reading user_defined tables (2 minutes and 20 seconds)
> > reading dependency data (5 minutes and 30 seconds)
>
> Can you get an execution profile with oprofile or gprof or similar tool?
> It doesn't surprise me a lot that pg_dump might have some issues with
> large numbers of objects, but guessing which inefficiencies are hurting
> you is difficult without more info.
>
> regards, tom lane
>

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Prashant Bharucha 2012-03-30 17:00:31 ERROR: invalid byte sequence for encoding "UTF8": 0xc325
Previous Message Tom Lane 2012-03-30 16:18:25 Re: pg_dump incredibly slow dumping a single schema from a large db

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2012-03-30 17:16:59 Re: Uppercase tab completion keywords in psql?
Previous Message Alvaro Herrera 2012-03-30 16:31:56 checkpointer code behaving strangely on postmaster -T