Re: disk I/O problems and Solutions

----- "Alan McKay" <alan(dot)mckay(at)gmail(dot)com> escreveu:
> CentOS / PostgreSQL shop over here.
>
> Our system
> IBM 3650 - quad 2Ghz e5405 Xeon
> 8K SAS RAID Controller
> 6 x 300G 15K/RPM SAS Drives
> /dev/sda - 2 drives configured as a RAID 1 for 300G for the OS
> /dev/sdb - 3 drives configured as RAID5 for 600G for the DB
> 1 drive as a global hot spare
>
> /dev/sdb is the one that is maxing out.

What are you calling "maxing out"? Excess IOPS, MB/s or high response times?
Each of these have different approaches when trying to find out a solution.

> Is there a public library somewhere of disk subsystems and their
> performance figures? Done with some semblance of a standard
> benchmark?

you should try using iostat or sar utilities. Both can give you complete reports of your online disk activity and probably were the tools in the backend used by your tool as the frontend.

It's very important to figure out that the percentage seen is all about CPU time used when in an I/O operation. If you have 100% you have to worry but not too desperatelly.
What matters most for me is the disk operation response time and queue size. If you have these numbers increasing then your database performance will suffer.

Always check the man pages for iostat to understand what those numbers are all about.

> One thing I am thinking of in the immediate term is taking the RAID5
> +
> hot spare and converting it to RAID10 with the same amount of
> storage.
> Will that perform much better?

Usually yes for write operations because the raid controller doesn't have to calculate parity for the spare disk. You'll have some improvements in the disk seek time and your database will be snapier if you have an OLTP application.

RAID5 can handle more IOPS, otherwise. It can be good for your pg_xlog directory, but the amount of disk space needed for WAL is just a small amount.

> In general we are planning to move away from RAID5 toward RAID10.
>
> We also have on order an external IBM array (don't have the exact
> name
> on hand but model number was 3000) with 12 drive bays. We ordered it
> with just 4 x SATAII drives, and were going to put it on a different
> system as a RAID10. These are just 7200 RPM drives - the goal was
> cheaper storage because the SAS drives are about twice as much per
> drive, and it is only a 300G drive versus the 1T SATA2 drives. IIRC
> the SATA2 drives are about $200 each and the SAS 300G drives about
> $500 each.

I think it's a good choice.

> So I have 2 thoughts with this 12 disk array. 1 is to fill it up
> with 12 x cheap SATA2 drives and hope that even though the spin-rate
> is a lot slower, that the fact that it has more drives will make it
> perform better. But somehow I am doubtful about that. The other
> thought is to bite the bullet and fill it up with 300G SAS drives.
>
> any thoughts here? recommendations on what to do with a tight
> budget?

Take you new storage system when it arrives, make it RAID10 and administer it using LVM in Linux.
If you need greater performance later you will be able to make stripes between raid arrays.

Regards

Flavio Henrique A. Gurgel
Consultor -- 4Linux
tel. 55-11-2125.4765
fax. 55-11-2125.4777
www.4linux.com.br