Re: Load Distributed Checkpoints, final patch

Heikki Linnakangas <heikki(at)enterprisedb(dot)com> writes:
> Here's latest revision of Itagaki-sans Load Distributed Checkpoints patch:

Applied with some minor revisions to make some of the internal APIs a
bit cleaner; mostly, it seemed like a good idea to replace all those
bool parameters with a flag-bits approach, so that you could have
something like "CHECKPOINT_FORCE | CHECKPOINT_WAIT" instead of
"false, true, true, false" ...

For the moment I removed all the debugging elog's in the patch.
We still have Greg Smith's checkpoint logging patch to look at
(which I suppose needs adjustment now), and that seems like the
appropriate venue to consider what to put in.

Also, the question of redesigning the bgwriter's LRU scan is
still open. I believe that's on Greg's plate, too.

One other closely connected item that might be worth looking at is the
code for creating new future xlog segments (PreallocXlogFiles). Greg
was griping upthread about xlog segment creation being a real
performance drag. I realized that as we currently have it set up, the
checkpoint code is next to useless for high-WAL-volume installations,
because it only considers making *one* future XLOG segment. Once you've
built up enough XLOG segments, the system isn't too bad about recycling
them, but there will be a nasty startup transient where foreground
processes have to stop and make the things. I wonder whether it would
help if we (a) have the bgwriter call PreallocXlogFiles during its
normal loop, and (b) back the slop in PreallocXlogFiles way off, so that
it will make a future segment as soon as we start using the last
existing segment, instead of only when we're nearly done. This would at
least make it more likely that the bgwriter does the work instead of a
foreground process. I'm hesitant to go much further than that, because
I don't want to bloat the minimum disk footprint for low-volume
installations, but the minimum footprint is really 2 xlog files anyway...

regards, tom lane