Re: Request for feedback on hardware for a new database server

On Thu, Mar 17, 2011 at 6:51 PM, Oliver Charles
<postgresql-perf(at)ocharles(dot)org(dot)uk> wrote:
> Hello,
>
> At MusicBrainz we're looking to get a new database server, and are
> hoping to buy this in the next couple of days. I'm mostly a software
> guy, but I'm posting this on behalf of Rob, who's actually going to be
> buying the hardware. Here's a quote of what we're looking to get:
>
>    I'm working to spec out a bad-ass 1U database server with loads of
>    cores (12), RAM (24GB) and drives (4 SAS) in a hardware RAID-1,0
>    configuration:
>
>    1 * SuperMicro 2016R-URF, 1U, redundant power supply, 4 SATA/SAS
>    drive bays 2
>    2 * Intel Xeon X5650 Westmere 2.66GHz 12MB L3 Cache LGA 1366 95W
>    Six-Core Server Processor 2
>    2 * Crucial 24GB (3 x 4GB) DDR3 SDRAM ECC Registered DDR3 1333,
>    CT3KIT51272BV1339 1
>    1 * LSI MegaRAID SATA/SAS 9260-4i ($379) (linux support [1])
>    or
>    1 * HighPoint RocketRAID 4320 PCI-Express x8 ($429)
>    or
>    1 * Adaptec RAID 3405 controller ($354)
>    4 * Fujitsu MBA3147RC 147GB 15000 RPM
>
>    SuperMicro machines have treated us really well over time (better
>    than Dell or Sun boxes), so I am really happy to throw more money in
>    their direction.  Redundant power supplies seem like a good idea for
>    a database server.
>
>    For $400 more we can get hexa core processors as opposed to quad
>    core processors at 2.66Ghz. This seems like a really good deal --
>    any thoughts on this?
>
>    Crucial memory has also served us really well, so that is a
>    no-brainer.
>
>    The RAID controller cards are where I need to most feedback! Of the
>    LSI, Highpoint or Adaptec cards, which one is likely to have native
>    linux support that does not require custom drivers to be installed?
>    The LSI card has great specs at a great price point with Linux
>    support, but installing the custom driver sounds like a pain. Does
>    anyone have any experience with these cards?
>
>    We've opted to not go for SSD drives in the server just yet -- it
>    doesn't seem clear how well SSDs do in a driver environment.
>
>    That's it -- anyone have any feedback?
>
> Just a quick bit more information. Our database is certainly weighted
> towards being read heavy, rather than write heavy (with a read-only web
> service accounting for ~90% of our traffic). Our tables vary in size,
> with the upperbound being around 10mil rows.
>
> I'm not sure exactly what more to say - but any feedback is definitely
> appreciated. We're hoping to purchase this server on Monday, I
> believe. Any questions, ask away!

I order my boxes from a white box builder called Aberdeen. They'll
test whatever hardware you want with whatever OS you want to make sure
it works before sending it out. As far as I know the LSI card should
just work with linux, if not, the previous rev should work fine (the
LSI 8888). I prefer Areca RAID 1680/1880 cards, they run cooler and
faster than the LSIs.

Another point. My experience with 1U chassis and cooling is that they
don't move enough air across their cards to make sure they stay cool.
You'd be better off ordering a 2U chassis with 8 3.5" drive bays so
you can add drives later if you need to, and it'll provide more
cooling air across the card.

Our current big 48 core servers are running plain LSI SAS adapters
without HW RAID because the LSI 8888s we were using overheated and
cooked themselves to death after about 3 months. Those are 1U chassis
machines, and our newer machines are all 2U boxes now. BTW, if you
ever need more than 2 sockets, right now the Magny Cours AMDs are the
fastest in that arena. For 2 sockets the Nehalem based machines are
about equal to them.

The high point RAID controllers are toys (or at least they were last I checked).

If you have to go with 4 drives just make it one big RAID-10 array and
then partition that out into 3 or 4 partitions. It's important to put
pg_xlog on a different partition even if it's on the same array, as it
allows the OS to fsync it separately.