Re: 8.3 / 8.2.6 restore comparison

From: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
To: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Greg Smith <gsmith(at)gregsmith(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: 8.3 / 8.2.6 restore comparison
Date: 2008-02-20 01:36:45
Message-ID: 20080219173645.2aac1bd2@commandprompt.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 19 Feb 2008 13:36:48 -0800
"Joshua D. Drake" <jd(at)commandprompt(dot)com> wrote:

Hello,

Some more testing on this. This time (using 8.3) I modified my restore
process to use multiple processes by manipulating TOC files. I used
three processes for the data copies, two processes for the pk creation,
two process for normal indexes and two processes for constraint
creation.

The machine averaged 40-60MB/s write versus the pathetic ~ 2/3 MB/s on
a single thread. It had an average I/O wait of < 10%. Lastly it
restored 57G of data 1.25 hours. Under my single thread testing 57G
would have taken ~ 3 hours.

I am pretty sure I can make it faster too as I wasn't balancing with
tablespaces nor did I move the xlogs off.

IMO this pretty much proves that we need to seriously look at a multi
connection restores. I can't imagine a situation where we look at
people that have dual and quad cores on their desktops and say... sorry
we can't use that to help you get your data quicker.

Sincerely,

Joshua D. Drake

- --
The PostgreSQL Company since 1997: http://www.commandprompt.com/
PostgreSQL Community Conference: http://www.postgresqlconference.org/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL SPI Liaison | SPI Director | PostgreSQL political pundit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHu4QvATb/zqfZUUQRAhyZAJ9U468fVDm8ww/2TrjDt6gM2wtlhwCffYYq
KJEsKpvRm6efiMQ+uAn/cs4=
=ZEGc
-----END PGP SIGNATURE-----

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Dunstan 2008-02-20 01:37:51 Re: Including PL/PgSQL by default
Previous Message Marc G. Fournier 2008-02-20 01:30:35 gateway test ...