Re: bgwriter and checkpoints

On Fri, Aug 12, 2011 at 7:09 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> The bgwriter has been responsible for two main activities: incremental
> page cleaning and checkpointing.
>
> We've worked out the code to smooth checkpointing, but that now means
> we have periods where we do both page cleaning and checkpointing, and
> other times when we do just page cleaning. That means the processing
> loops are complex and that makes latch based processing more difficult
> to introduce.

I've thought about this before, and I think it's probably a good idea.
At the very least that code needs some refactoring, and breaking it
into two completely separate processes may be the way to go.

One possible trouble spot is that this will have the effect of
increasing traffic on BgWriterCommLock. Right now, page-cleaning
activity doesn't need to do that, because the bgwriter is both the
source of all the fsync requests and the guy who eventually executes
them, so it can just update its local state directly. If the two
functions are separated, that gets a bit more complicated: the
checkpoint process, even when not checkpointing, will need to absorb
fsync requests every so often (maybe that could be driven off a latch,
so we wake it up when the request queue is 25% full, or something like
that?). Or else the cleaning process will have to be in charge of
issuing the fsyncs and the checkpoint process will have to request
that it do so at the appropriate time. Or maybe there's some third
solution I'm not thinking of. Anyway, it may not be a big problem in
practice, just something I was worrying about...

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company