Re: compact flash disks?

At 09:11 AM 3/8/2007, Merlin Moncure wrote:
>On 3/8/07, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>>On Thu, Mar 08, 2007 at 06:24:35AM -0000, James Mansion wrote:
>> >
>> > In the long run, we are going to have to seriously rethink pg's use
>> > of WAL as the way we implement MVCC as it becomes more and more of a
>> > performance bottleneck.
>> > We have WAL because Stonebreaker made an assumption about the future
>> > dominance of optical media that has turned out to be false.
>> > ...and it's been one of pg's big issues every since.
>>
>>Uh. pg didn't *have* WAL back when Stonebreaker was working on it. It
>>was added in PostgreSQL 7.1, by Vadim. And it significantly increased
>>performance at the time, since we no longer had to sync the datafiles
>>after every transaction commit.
>>(We also didn't have MVCC back in the Stonebreaker days - it was added
>>in 6.5)

Huh. I have to go re-read my references. Seems I've misremembered history.
Thanks for correcting my mistake.

>Exactly, and WAL services other purposes than minimizing the penalty
>from writing to high latency media. WAL underlies PITR for example.

Never argued with any of this.

>Near-zero latency media is coming, eventually...and I don't think the
>issue is reliability (catastrophic failure is extremely unlikely) but
>cost. I think the poor write performance is not an issue because you
>can assemble drives in a giant raid 0 (or even 00 or 000) which will
>blow away disk based raid 10 systems at virtually everything.
Have you considered what the $cost$ of that much flash RAM would be?

>Solid State Drives consume less power (a big deal in server farms) and
>the storage density and life-span will continue to improve. I give it
>five years (maybe less) before you start to see SSD penetration in a
>big way. It will simply become cheaper to build a box with SSD than
>without since you won't need to buy as much RAM, draws less power, and
>is much more reliable.
Don't bet on it. HDs 2x in density faster than RAM does, even flash
RAM, and have a -much- lower cost per bit.
...and it's going to stay that way for the foreseeable future.

ATM, I can buy a 500GB 7200 rpm SATA II HD w/ a 5 yr warranty for
~$170 US per HD.
1TB HDs of that class cost ~$350-$400 US per.
(...and bear in mind the hybrid HDs coming out later this year that
offer the best of both HD and flash technologies at very close to
current HD costs per bit.)

The 128GB flash RAM SSDs coming out later this year are going to cost
4x - 10x those HD prices...

4+ decades of history shows that initial acquisition cost is =by far=
the primary deciding factor in IT spending.
QED: SSDs are going to remain a niche product unless or until
Something Drastic (tm) happens to the economics and storage requirements of IT.

>Disk drives will displace tape as low speed archival storage but will
>probably live on in super high storage enterprise environments.

Again, don't bet on it. Tape 2x in density even faster than HD does
and has an even lower cost per bit.

>my 0.02$, as usual,
>merlin