Re: Best COPY Performance

From: "Luke Lonergan" <llonergan(at)greenplum(dot)com>
To: "Worky Workerson" <worky(dot)workerson(at)gmail(dot)com>
Cc: "Merlin Moncure" <mmoncure(at)gmail(dot)com>, "Markus Schaber" <schabi(at)logix-tt(dot)com>, pgsql-performance(at)postgresql(dot)org
Subject: Re: Best COPY Performance
Date: 2006-10-25 20:13:25
Message-ID: C1651375.523A%llonergan@greenplum.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

Mr. Worky,

On 10/25/06 11:26 AM, "Worky Workerson" <worky(dot)workerson(at)gmail(dot)com> wrote:

> And here are the dd results for 16GB RAM, i.e. 4,000,000 8K blocks:

So, if we divide 32,000 MB by the real time, we get:

/home (WAL):
53 MB/s write
84 MB/s read

/data (data):
89 MB/s write
38 MB/s read

The write and read speeds on /home look like a single disk drive, which is
not good if you have more drives in a RAID. OTOH, it should be sufficient
for WAL writing and you should think that the COPY speed won't be limited by
WAL.

The read speed on your /data volume is awful to the point where you should
consider it broken and find a fix. A quick comparison: the same number on a
16 drive internal SATA array with 7200 RPM disks gets 950 MB/s read, about
25 times faster for about 1/4 the price.

But again, this may not have anything to do with the speed of your COPY
statements.

Can you provide about 10 seconds worth of "vmstat 1" while running your COPY
so we can get a global view of the I/O and CPU?

- Luke

In response to

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Brian Hurt 2006-10-25 20:18:51 Re: commit so slow program looks frozen
Previous Message Carlo Stonebanks 2006-10-25 20:07:38 Re: commit so slow program looks frozen