Re: Potential Large Performance Gain in WAL synching

"Zeugswetter Andreas SB SD" <ZeugswetterA(at)spardat(dot)at> writes:
> To make this competitive, the WAL writes would need to be improved to
> do more than one block (up to 256k or 512k per write) with one write call
> (if that much is to be written for this tx to be able to commit).
> This should actually not be too difficult since the WAL buffer is already
> contiguous memory.

Hmmm ... if you were willing to dedicate a half meg or meg of shared
memory for WAL buffers, that's doable. I was originally thinking of
having the (still hypothetical) background process wake up every time a
WAL page was completed and available to write. But it could be set up
so that there is some "slop", and it only wakes up when the number of
writable pages exceeds N, for some N that's still well less than the
number of buffers. Then it could write up to N sequential pages in a
single write().

However, this would only be a win if you had few and large transactions.
Any COMMIT will force a write of whatever we have so far, so the idea of
writing hundreds of K per WAL write can only work if it's hundreds of K
between commit records. Is that a common scenario? I doubt it.

If you try to set it up that way, then it's more likely that what will
happen is the background process seldom awakens at all, and each
committer effectively becomes responsible for writing all the WAL
traffic since the last commit. Wouldn't that lose compared to someone
else having written the previous WAL pages in background?

We could certainly build the code to support this, though, and then
experiment with different values of N. If it turns out N==1 is best
after all, I don't think we'd have wasted much code.

regards, tom lane