Re: Consumer-grade vs enterprise-grade disk drives

From: "Luke Lonergan" <llonergan(at)greenplum(dot)com>
To: "Hannu Krosing" <hannu(at)skype(dot)net>, "Mark Kirkwood" <markir(at)paradise(dot)net(dot)nz>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Consumer-grade vs enterprise-grade disk drives
Date: 2005-06-01 19:40:31
Message-ID: BEC35D3F.6BE4%llonergan@greenplum.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hannu,

> RAID10
>
> "Throughput report Y-axis is type of test X-axis is number of processes"
> "Record size = 8 Kbytes "
> "Output is in ops/sec"
> " Initial write " 1352.90
> " Rewrite " 413.31

> RAID5
>
> "Throughput report Y-axis is type of test X-axis is number of processes"
> "Record size = 8 Kbytes "
> "Output is in ops/sec"
> " Initial write " 1178.55
> " Rewrite " 145.91
>
> Seems to have either very low overhead for RAID5 or something else is
> keeping RAID10 speed down.

The rewrite number is telling - the FPGA on the 3Ware card isn't doing XOR
while reading very well.

I've found that synthetic benchmarks aren't necessarily predictive for
Postgres performance. The issue is that the bottlenecks for writing data
and/or scans are not currently in the I/O subsystem. There are many
improvements to the executor and to the load/copy and DML paths necessary
before this is the case.

<begin flame bait :-), answers on bizgres-general>

So, I find that for data warehousing workloads (sequential access) if the
I/O subsystem runs faster than 100MB/s on sequential writes and about
120MB/s on scans, it will outrun Postgres anyway.

We're working to fix this, but right now it appears that high performance
I/O on one table (faster than 120MB/s) is not possible with a single
instance of Postgres.

<end flame bait>

Luke

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Simon Riggs 2005-06-01 20:17:39 Re: NOLOGGING option, or ?
Previous Message Jonah H. Harris 2005-06-01 19:11:18 Re: Google's Summer of Code ...