Re: pg_dump and thousands of schemas

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Tatsuo Ishii <ishii(at)postgresql(dot)org>
Cc: pgsql-performance(at)postgresql(dot)org, pgsql-hackers(at)postgresql(dot)org
Subject: Re: pg_dump and thousands of schemas
Date: 2012-05-31 03:54:30
Message-ID: 26386.1338436470@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-performance

Tatsuo Ishii <ishii(at)postgresql(dot)org> writes:
>> Ok, I modified the part of pg_dump where tremendous number of LOCK
>> TABLE are issued. I replace them with single LOCK TABLE with multiple
>> tables. With 100k tables LOCK statements took 13 minutes in total, now
>> it only takes 3 seconds. Comments?

> Shall I commit to master and all supported branches?

I'm not excited by this patch. It dodges the O(N^2) lock behavior for
the initial phase of acquiring the locks, but it does nothing for the
lock-related slowdown occurring in all pg_dump's subsequent commands.
I think we really need to get in the server-side fix that Jeff Janes is
working on, and then re-measure to see if something like this is still
worth the trouble. I am also a tad concerned about whether we might not
have problems with parsing memory usage, or some such, with thousands of
tables being listed in a single command.

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Stephen Frost 2012-05-31 04:01:49 Re: pg_dump and thousands of schemas
Previous Message Jeff Davis 2012-05-31 02:52:30 Re: temporal support patch

Browse pgsql-performance by date

  From Date Subject
Next Message Stephen Frost 2012-05-31 04:01:49 Re: pg_dump and thousands of schemas
Previous Message Tatsuo Ishii 2012-05-31 00:20:43 Re: pg_dump and thousands of schemas