Re: pg_dump and thousands of schemas

From: Jeff Janes <jeff(dot)janes(at)gmail(dot)com>
To: Tatsuo Ishii <ishii(at)postgresql(dot)org>
Cc: tgl(at)sss(dot)pgh(dot)pa(dot)us, pgsql-performance(at)postgresql(dot)org
Subject: Re: pg_dump and thousands of schemas
Date: 2012-06-10 23:47:41
Message-ID: CAMkU=1zdr7eOEcbopM6c-+zT1aTaWXsTyA_5ZkZ4rgG7EkxMPQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-performance

On Wed, May 30, 2012 at 2:06 AM, Tatsuo Ishii <ishii(at)postgresql(dot)org> wrote:
>> Yeah, Jeff's experiments indicated that the remaining bottleneck is lock
>> management in the server.  What I fixed so far on the pg_dump side
>> should be enough to let partial dumps run at reasonable speed even if
>> the whole database contains many tables.  But if psql is taking
>> AccessShareLock on lots of tables, there's still a problem.
>
> Ok, I modified the part of pg_dump where tremendous number of LOCK
> TABLE are issued. I replace them with single LOCK TABLE with multiple
> tables. With 100k tables LOCK statements took 13 minutes in total, now
> it only takes 3 seconds. Comments?

Could you rebase this? I tried doing it myself, but must have messed
it up because it got slower rather than faster.

Thanks,

Jeff

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2012-06-11 00:55:13 Re: [COMMITTERS] pgsql: Run pgindent on 9.2 source tree in preparation for first 9.3
Previous Message Jeff Frost 2012-06-10 23:19:15 Re: Backends stalled in 'startup' state: index corruption

Browse pgsql-performance by date

  From Date Subject
Next Message Rural Hunter 2012-06-11 04:46:41 Re: how to change the index chosen in plan?
Previous Message Robert Klemme 2012-06-10 11:16:40 Re: partitioning performance question