Re: RAID controllers for Postgresql on large setups

On Mon, May 12, 2008 at 8:04 PM, Francisco Reyes <lists(at)stringsutils(dot)com> wrote:
> Inheritted a number of servers and I am starting to look into the hardware.
>
> So far what I know from a few of the servers
> Redhat servers.
> 15K rpm disks, 12GB to 32GB of RAM.
> Adaptec 2120 SCSI controller (64MB of cache).

Considering the generally poor performance of adaptec RAID
controllers, you'd probably be better off with 12 SATA drives hooked
up to an escalade or Areca card (or cards). Since you seem to want a
lot of storage, a large array of SATA disks may be a better balance
between performance and economy.

> The servers have mostly have 12 drives in RAID 10.
> We are going to redo one machine to compare RAID 10 vs RAID 50. Mostly to
> see if the perfomance is close, the space gain may be usefull.

See the remark about SATA drives above. With 12 750Gig drives, you'd
have 6*750G of storage in RAID-10, or about 4.5 Terabytes of redundant
storage.

> The usage pattern is mostly large set of transactions ie bulk loads of
> millions of rows, queries involving tens of millions of rows.
> A few questions.
> Will it pay to go to a controller with higher memory for existing machines?

Then no matter how big your cache on your controller, it's likely NOT
big enough to ever hope to just swallow the whole set at once. Bigger
might be better for a lot of things, but for loading, a good
controller is more important. An increase from 64M to 256M is not
that big in comparison to how big your datasets are likely to be.

> The one machine I am about to redo has PCI which seems to somewhat limit our
> options.

You do know that you can plug a PCI-X card into a PCI slot, right?
(see the second paragraph here:
http://en.wikipedia.org/wiki/PCI-X#Technical_description) So, you can
get a nice card today, and if needs be, a better server to toss it in
tomorrow.

Where I work we have a nice big machine in production with a very nice
PCI-X card (Not sure which one, my cohort ordered it) and we wanted
out in house testing machine to have the same card, but that machine
is much less powerful. Same card fit, so we can get some idea about
I/O patterns on the test box before beating on production.

> A few questions.
> Will it pay to go to a controller with higher memory for existing machines?

Pay more attention to the performance metrics the card gets from
people testing it here. Areca, Escalade / 3Ware, and LSI get good
reviews, with LSI being solid but a little slower than the other two
for most stuff.

> For large setups (ie 500GB+ per server) does it make sense to try to get a
> controller in a machine or do SANs have better throughput even if at a much
> higher cost?

SANs generally don't have much better performance, and cost MUCH more
per meg stored. They do however have some nice management options.
If a large number of disks in discrete machines presents a problem of
maintenance, the SAN might help, but given the higher cost, it's often
just cheaper to keep a box of disks handy and have a hardware person
replace them.

> For future machines I plan to look into controllers with at least 512MB,
> which likely will be PCI-X/PCI-e.. not seen anything with large caches for
> PCI.

See remark about PCI-X / PCI

> Also the machines in question have SCSI drives, not SAS. I believe the
> most recent machine has SAS, but the others may be 15K rpm scsi
> Whether a SAN or just an external enclosure is 12disk enough to substain 5K
> inserts/updates per second on rows in the 30 to 90bytes territory?

You'll only know by testing, and a better RAID controller can make a
WORLD of difference here. Just make sure whatever controller you get
has battery backed cache, and preferably a fair bit of it. Some
controllers can handle 1G+ of memory.