Re: ext4 finally doing the right thing

* Greg Smith <greg(at)2ndquadrant(dot)com> [100121 09:49]:
> Aidan Van Dyk wrote:
>> Sure, if your WAL is on the same FS as your data, you're going to get
>> hit, and *especially* on ext3...
>>
>> But, I think that's one of the reasons people usually recommend putting
>> WAL separate.
>
> Separate disks can actually concentrate the problem. The writes to the
> data disk by checkpoints will also have fsync behind them eventually, so
> splitting out the WAL means you just push the big write backlog to a
> later point. So less frequently performance dives, but sometimes
> bigger. All of the systems I was mentioning seeing >10 second pauses on
> had a RAID-1 pair of WAL disks split from the main array.

That's right, so with the WAL split off on it's own disk, you don't wait
on "WAL" for your checkpoint/data syncs, but you can build up a huge
wait in the queue for main data (which can even block reads).

Having WAL on the main disk means that (for most ext3), you sometimes
have WAL writes taking longer, but the WAL fsyncs are keeping the
backlog "down" in the main data area too.

Now, with ext4 moving to full barrier/fsync support, we could get to the
point where WAL in the main data FS can mimic the state where WAL is
seperate, namely that WAL writes can "jump the queue" and be written
without waiting for the data pages to be flushed down to disk, but also
that you'll get the big backlog of data pages to flush when
the first fsyncs on big data files start coming from checkpoints...

a.
--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.