Re: Reliability with RAID 10 SSD and Streaming Replication

On 5/22/13 12:56 PM, Shaun Thomas wrote:
> Well, you may not be able to make that claim, but I can. While we don't
> use Intel SSDs, our first-gen FusinoIO cards can deliver about 20k
> PostgreSQL TPS of our real-world data right off the device before
> caching effects start boosting the numbers.

I've seen FusionIO hit that 20K commit number, as well as hitting 75K
IOPS on random reads (600MB/s). They are roughly 5 to 10X faster than
the Intel 320/710 drives. There's a corresponding price hit though, and
having to provision PCI-E cards is a pain in some systems.

A claim that a FusionIO drive in particular is capable of 100X the
performance of a spinning drive, that I wouldn't dispute. I even made
that claim myself with some benchmark numbers to back it up:
http://www.fusionio.com/blog/fusion-io-boosts-postgresql-performance/
That's not just a generic SSD anymore though.

> An 8-drive 15k RPM RAID-10 gave us about 1800 TPS back when we switched
> to FusionIO about two years ago. So, while Intel drives themselves may
> not be able to hit sustained 100x speeds over spindles, it's pretty
> clear that that's a firmware or implementation limitation.

1800 TPS to 20K TPS is just over a 10X speedup.

As for Intel vs. FusionIO, rather than implementation quality it's more
what architecture you're willing to pay for. If you test a few models
across Intel's product line, you can see there's a rough size vs. speed
correlation. The larger units have more channels of flash going at the
same time. FusionIO has architected such that there is a wide write
path even on their smallest cards. That 75K IOPS number I got even out
of their little 80GB card. (since dropped from the product line)

I can buy a good number of Intel DC S3700 drives for what a FusionIO
card costs though.

> The main "issue" is that the sustained sequence scan speeds are
> generally less than an order of magnitude faster than drives. So as soon
> as you hit something that isn't limited by random IOPS, spindles get a
> chance to catch up.

I have some moderately fast SSD based transactional systems that are
still using traditional drives with battery-backed cache for the
sequential writes of the WAL volume, where the data volume is on Intel
710 disks. WAL writes really burn through flash cells, too, so keeping
them on traditional drives can be cost effective in a few ways. That
approach is lucky to hit 10K TPS though, so it can't compete against
what a PCI-E card like the FusionIO drives are capable of.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com