Re: Hardware recommendations to scale to silly load

> Don't know how "cheap" they are.
>
> I have an app that does large batch updates. I found that if I dropped
> the indexes, did the updates and recreated the indexes, it was faster
> than doing the updates while the indexes were intact.

Yeah, unfortunately it's not batch work, but real time financial work.
If I drop all the indexes my select performance goes through the floor,
as you'd expect.

> Does noatime make much difference on a PostgreSQL database? I haven't
> tested that yet.

Yup, it does. In fact it should probably be in the standard install
documentation (unless someone has a reason why it shouldn't). Who
*cares* when PG last looked at the tables? If 'nomtime' was available
that would probably be a good thing too.

> Can you split it onto multiple boxes? Some database layouts lend themselves
> to this, others don't. Obviously you can't do joins from one server to
> another, so you may lose more in multiple queries than you gain by having
> multiple servers. It's worth looking into though.

I'm considering that. There are some tables which I might be able to
split out. There amy even be some things I can pull from the DB
altogether (session info in particular, so long as I can reliably send a
given user's requests to the same app server each time, bearing in mind
I can't see the cookies too easily because 50% of the requests are over
SSL)

> I know my answers aren't quite the ones you were looking for, but my
> experience is that many people try to solve poor application design
> by simply throwing bigger hardware at the problem. It appears as though
> you've already done your homework, though.

Well, I *hope* that's the case! The core issue is simply that we have
to deal with an insane load for 1 hour a week, and there's just no
avoiding it.

Maybe I can get Sun/HP/IBM to lend some gear (it's a pretty high-profile
site).