Re: [Testperf-general] Re: ExclusiveLock

> From: Kenneth Marshall [mailto:ktm(at)is(dot)rice(dot)edu]
[snip]
> The simplest idea I had was to pre-layout the WAL logs in a
> contiguous fashion
> on the disk. Solaris has this ability given appropriate FS
> parameters and we
> should be able to get close on most other OSes. Once that has
> happened, use
> something like the FSM map to show the allocated blocks. The
> CPU can keep track
> of its current disk rotational position (approx. is okay)
> then when we need to
> write a WAL block start writing at the next area that the
> disk head will be
> sweeping. Give it a little leaway for latency in the system
> and we should be
> able to get very low latency for the writes. Obviously, there
> would be wasted
> space but you could intersperse writes to the granularity of
> space overhead
> that you would like to see. As far as implementation, I was reading an
> interesting article that used a simple theoretical model to
> estimate disk head
> position to avoid latency.
>

Ken,

That's a neat idea, but I'm not sure how much good it will do. As bad as
rotational latency is, seek time is worse. Pre-allocation isn't going to do
much for rotational latency if the heads also have to seek back to the WAL.

OTOH, pre-allocation could help two other performance aspects of the WAL:
First, if the WAL was pre-allocated, steps could be taken (by the operator,
based on their OS) to make the space allocated to the WAL contiguous.
Statistics on how much WAL is needed in 24 hours would help with that
sizing. This would reduce seeks involved in writing the WAL data.

The other thing it would do is reduce seeks and metadata writes involved in
extending WAL files.

All of this is moot if the WAL doesn't have its own spindle(s).

This almost leads back to the old-fashioned idea of using a raw partition,
to avoid the overhead of the OS and file structure.

Or I could be thoroughly demonstrating my complete lack of understanding of
PostgreSQL internals. :-)

Maybe I'll get a chance to try the flash drive WAL idea in the next couple
of weeks. Need to see if the hardware guys have a spare flash drive I can
abuse.

Paul