SAN/NAS options

Hello all,

It seems that I'm starting to outgrow our current Postgres setup. We've been
running a handful of machines as standalone db servers. This is all in a
colocation environment, so everything is stuffed into 1U Supermicro boxes. Our
standard build looks like this:

Supermicro 1U w/SCA backplane and 4 bays
2x2.8 GHz Xeons
Adaptec 2015S "zero channel" RAID card
2 or 4 x 73GB Seagate 10K Ultra 320 drives (mirrored+striped)
2GB RAM
FreeBSD 4.11
PGSQL data from 5-10GB per box

Recently I started studying what we were running up against in our nightly runs
that do a ton of updates/inserts to prep things for the tasks the db does
during the business day (light mix of selects/inserts/updates). While we have
plenty of disk bandwidth (according to bonnie), we are really dying on IOPS.
I'm guessing this is a mix of a rather anemic RAID controller (ever notice how
adaptec doesn't publish any real performance specs on raid cards?) and having
only two or four spindles (effectively 1 or 2 on writes).

So that's where we are...

I'm new to the whole SAN thing, but did recently pick up a few used NetApp
shelves and a Fibre Channel RAID HBA (Mylex ExtremeRAID 3000, also used) to toy
with. I started wondering if I could put something together to both get our
storage on one set of boxes and allow me to get data striped across more
drives. Our budget is not huge and we are not adverse to getting used gear
where appropriate.

What do you folks recommend? I'm just starting to look at what's out there for
SANs and NAS, and from what I've seen, our options are:

NetApp Filers - the pluses with these are that if we use NFS, we don't have to
worry about either large filesystem support in FreeBSD (2TB practical limit),
or limitation on "growing" partitions as the NetApp just deals with that. I
also understand these make backups a bit simpler. I have a great, trusted,
spare-stocking source for these.

Apple X-Serve RAID - well, it's pretty cheap. Honestly, that's all I know
about it - they don't talk about IOPS numbers, and I have no idea what lurks in
that box as a RAID controller.

SAN box w/integrated RAID - it seems like this might not be a good choice since
the RAID hardware in the box may be where I hit any limits. I also imagine I'm
probably overpaying for some OEM RAID controller integrated into the box. No
idea where to look for used gear.

SAN box, JBOD - this seems like it might be affordable as well. A few big
shelves full of drives a SAN "switch" to plug all the shelves and hosts into
and a FC RAID card in each host. No idea where to look for used gear here
either.

You'll note that I'm being somewhat driven by my OS of choice, FreeBSD. Unlike
Solaris or other commercial offerings, there is no nice volume management
available. While I'd love to keep managing a dozen or so FreeBSD boxes, I
could be persuaded to go to Solaris x86 if the volume management really shines
and Postgres performs well on it.

Lastly, one thing that I'm not yet finding in trying to educate myself on SANs
is a good overview of what's come out in the past few years that's more
affordable than the old big-iron stuff. For example I saw some brief info on
this list's archives about the Dell/EMC offerings. Anything else in that vein
to look at?

I hope this isn't too far off topic for this list. Postgres is the main
application that I'm looking to accomodate. Anything else I can do with
whatever solution we find is just gravy...

Thanks!

Charles