Re: Analysis of ganged WAL writes

> "Curtis Faith" <curtis(at)galtair(dot)com> writes:
> > Successive writes would write different NON-OVERLAPPING sections of the
> > same log buffer. It wouldn't make sense to send three separate
> copies of
> > the entire block. That could indeed cause problems.
>
> So you're going to undo the code's present property that all writes are
> block-sized? Aren't you worried about incurring page-in reads because
> the kernel can't know that we don't care about data beyond what we've
> written so far in the block?

Yes, I'll try undoing the current behavior.

I'm not really worried about doing page-in reads because the disks internal
buffers should contain most of the blocks surrounding the end of the log
file. If the successive partial writes exceed a block (which they will in
heavy use) then most of the time this won't be a problem anyway since the
disk will gang the full blocks before writing.

If the inserts are not coming fast enough to fill the log then the disks
cache should contain the data from the last time that block (or the block
before) was written. Disks have become pretty good at this sort of thing
since writing sequentially is a very common scenario.

It may not work, but one doesn't make significant progress without trying
things that might not work.

If it doesn't work, then I'll make certain that commit log records always
fill the buffer they are written too, with variable length commit records
and something to identify the size of the padding used to fill the rest of
the block.