From: | Scott Carey <scott(at)richrelevance(dot)com> |
---|---|
To: | Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc>, Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov> |
Cc: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Matthew Wakeling <matthew(at)flymine(dot)org>, "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org> |
Subject: | Re: PostgreSQL 8.4 performance tuning questions |
Date: | 2009-07-30 20:01:26 |
Message-ID: | C6974A26.DD73%scott@richrelevance.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-performance |
On 7/30/09 11:24 AM, "Stefan Kaltenbrunner" <stefan(at)kaltenbrunner(dot)cc> wrote:
> Kevin Grittner wrote:
>> Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov> writes:
>>>> Since the dump to custom format ran longer than the full pg_dump
>>>> piped directly to psql would have taken, the overall time to use
>>>> this technique is clearly longer for our databases on our hardware.
>>> Hmmm ... AFAIR there isn't a good reason for dump to custom format
>>> to take longer than plain text dump, except for applying
>>> compression. Maybe -Z0 would be worth testing? Or is the problem
>>> that you have to write the data to a disk file rather than just
>>> piping it?
>>
>> I did some checking with the DBA who normally copies these around for
>> development and test environments. He confirmed that when the source
>> and target are on the same machine, a pg_dump piped to psql takes
>> about two hours. If he pipes across the network, it runs more like
>> three hours.
>>
>> My pg_dump to custom format ran for six hours. The single-transaction
>> restore from that dump file took two hours, with both on the same
>> machine. I can confirm with benchmarks, but this guy generally knows
>> what he's talking about (and we do create a lot of development and
>> test databases this way).
>>
>> Either the compression is tripling the dump time, or there is
>> something inefficient about how pg_dump writes to the disk.
>
> seems about right - compression in pg_dump -Fc is a serious bottleneck
> and unless can significantly speed it up or make it use of multiple
> cores (either for the dump itself - which would be awsome - or for the
> compression) I would recommend to not use it at all.
>
That's not an option when a dump compressed is 200GB and uncompressed is
1.3TB, for example.
>
> Stefan
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2009-07-30 20:15:44 | Re: PostgreSQL 8.4 performance tuning questions |
Previous Message | Scott Carey | 2009-07-30 19:59:46 | Re: PostgreSQL 8.4 performance tuning questions |