IBM P-series machines (was: Excessive context switching on SMP Xeons)

On Tue, Oct 05, 2004 at 09:47:36AM -0700, Josh Berkus wrote:
> As long as you're on x86, scaling outward is the way to go. If you want to
> continue to scale upwards, ask Andrew Sullivan about his experiences running
> PostgreSQL on big IBM boxes. But if you consider an quad-Opteron server
> expensive, I don't think that's an option for you.

Well, they're not that big, and both Chris Browne and Andrew Hammond
are at least as qualified to talk about this as I. But since Josh
mentioned it, I'll put some anecdotal rablings here just in case
anyone is interested.

We used to run our systems on Solaris 7, then 8, on Sun E4500s. We
found the performance on those boxes surprisingly bad under certain
pathological loads. I ultimately traced this to some remarkably poor
shared memory handling by Solaris: during relatively heavy load
(in particular, thousands of selects per second on the same set of
tuples) we'd see an incredible number of semaphore operations, and
concluded that the buffer handling was killing us.

I think we could have tuned this away, but for independent reasons we
decided to dump Sun gear (the hardware had become unreliable, and we
were not satisfied with the service we were getting). We ended up
choosing IBM P650s as a replacement.

The 650s are not cheap, but boy are they fast. I don't have any
numbers I can share, but I can tell you that we recently had a few
days in which our write load was as large as the entire write load
for last year, and you couldn't tell. It is too early for us to say
whether the P series lives up to its billing in terms of relibility:
the real reason we use these machines is reliability, so if
approaching 100% uptime isn't important to you, the speed may not be
worth it.

We're also, for the record, doing experiments with Opterons. So far,
we're impressed, and you can buy a lot of Opteron for the cost of one
P650.

A

--
Andrew Sullivan | ajs(at)crankycanuck(dot)ca
I remember when computers were frustrating because they *did* exactly what
you told them to. That actually seems sort of quaint now.
--J.D. Baldwin