Re: Pre-allocating WAL files

On 10/6/21, 5:20 AM, "Maxim Orlov" <m(dot)orlov(at)postgrespro(dot)ru> wrote:
> We've looked through the code and everything looks good except few minor things:
> 1). Using dedicated bg worker seems not optimal, it introduces a lot of redundant code for little single action.
> We'd join initial proposal of Andres to implement it as an extra fuction of checkpointer.

Thanks for taking a look!

I have been thinking about the right place to put this logic. My
first thought is that it sounds like something that ought to go in the
WAL writer process, but as Andres noted upthread, that is undesirable
because it'll add latency for asynchronous commits. The checkpointer
process seems to be another candidate, but I'm not totally sure how
it'll fit in. My patch works by maintaining a small pool of pre-
allocated segments that is quickly replenished whenever one is used.
If the checkpointer is spending most of its time checkpointing, this
small pool could remain empty for long periods of time. To keep pre-
allocating WAL while we're checkpointing, perhaps we could add another
function like CheckpointWriteDelay() that is called periodically.
There still might be several operations in CheckPointGuts() that take
a while and leave the segment pool empty, but maybe that's okay for
now.

Nathan