Re: RAMFS with Postgres

On Fri, 2005-07-22 at 09:56, Alex Stapleton wrote:
> On 21 Jul 2005, at 17:02, Scott Marlowe wrote:

> >
> > My feeling is that you may be going about this the wrong way. Most
> > likely the issue so far has been I/O contention. Have you tested your
> > application using a fast, battery backed caching RAID controller on
> > top
> > of, say, a 10 disk RAID 1+0 array? Or even RAID 0 with another
> > machine
> > as the slony slave?
>
> Isn't that slightly cost prohibitive? Even basic memory has
> enormously fast access/throughput these days, and for a fraction of
> the price.

I don't know. How expensive is your data? If you lose it all, is saving
a few grand on hardware worth it? And I'm not being sarcastic. Some
data isn't really worth getting too worked up over, some data is a big
deal.

Choose two: Fast, Cheap, Reliable.

Plus, this really depends on your storage requirements. If you only
need a couple of gigabytes, then you don't need that big of drives,
because you'll be using a fair number of them. Let's say you pick up 10
9 gig Ultra SCSI drives off ebay, using 8 in a RAID1+0 with two spares.

The going price for used 9 gig drives is about $20.00 or so.

A good used RAID card with battery backed cache can be had on ebay for
<$300, sometimes around $100 if you shop around.

So, that's an 8 disk RAID 1+0, with two spares, and a controller, for
about $500 or so. And, while it should have performance similar to what
you're wanting to do with the ramfs setup, it should be able to survive
someone tripping over the power cable with no loss of data. (I'd test
it to be sure, not all RAID controllers / hard drive combos tell the
truth about caching and fsync.)

> > Slony, by the way, is quite capable, but using a RAMFS master and a
> > Disk
> > drive based slave is kind of a recipe for disaster in ANY replication
> > system under heavy load, since it is quite possible that the master
> > could get very far ahead of the slave, since Slony is asynchronous
> > replication. At some point you could have more data waiting to be
> > replicated than your ramfs can hold and have some problems.
> >
> > If a built in RAID controller with battery backed caching isn't
> > enough,
> > you might want to look at a large, external storage array then. many
> > hosting centers offer these as a standard part of their package, so
> > rather than buying one, you might want to just rent one, so to speak.
>
> Again with the *money* RAM = Cheap. Disks = Expensive. At least when
> you look at speed/$. Your right about replicating to disk and to ram
> though, that is pretty likely to result in horrible problems if you
> don't keep load down. For some workloads though, I can see it
> working. As long as the total amount of data doesn't get larger than
> your RAMFS it could probably survive.

Right, for certain bursty situations it would work just fine. For
something that just kept tossing huge amounts of data at the system it
could turn ugly fast.