Re: [HACKERS] pg_dump and thousands of schemas

From: Claudio Freire <klaussfreire(at)gmail(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Robert Klemme <shortcutter(at)googlemail(dot)com>, Tatsuo Ishii <ishii(at)postgresql(dot)org>, pgsql-performance(at)postgresql(dot)org, pgsql-hackers(at)postgresql(dot)org
Subject: Re: [HACKERS] pg_dump and thousands of schemas
Date: 2012-05-31 15:01:42
Message-ID: CAGTBQpYdNN_Q3Ksvr7tN_6aXqp_gc0KOsbrWLW_QZE=c22QjUg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-performance

On Thu, May 31, 2012 at 11:50 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> The performance patches we applied to pg_dump over the past couple weeks
> were meant to relieve pain in situations where the big server-side
> lossage wasn't the dominant factor in runtime (ie, partial dumps).
> But this one is targeting exactly that area, which is why it looks like
> a band-aid and not a fix to me.

No, Tatsuo's patch attacks a phase dominated by latency in some
setups. That it's also becoming slow currently because of the locking
cost is irrelevant, with locking sped up, the patch should only
improve the phase even further. Imagine the current timeline:

* = locking
. = waiting

*.*.**.**.***.***.****.****.*****.****

Tatsuo's patch converts it to:

*.**************

The locking fix would turn the timeline into:

*.*.*.*.*.*.*

Tatsuo's patch would turn that into:

*******

And, as noted before, pg_dump --schema-only is a key bottleneck in pg_upgrade.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2012-05-31 15:04:12 Re: [HACKERS] pg_dump and thousands of schemas
Previous Message Bruce Momjian 2012-05-31 15:00:54 Re: [HACKERS] pg_dump and thousands of schemas

Browse pgsql-performance by date

  From Date Subject
Next Message Robert Haas 2012-05-31 15:04:12 Re: [HACKERS] pg_dump and thousands of schemas
Previous Message Bruce Momjian 2012-05-31 15:00:54 Re: [HACKERS] pg_dump and thousands of schemas