Re: Final background writer cleanup for 8.3

Greg Smith <gsmith(at)gregsmith(dot)com> writes:
> In the interest of closing work on what's officially titled the "Automatic
> adjustment of bgwriter_lru_maxpages" patch, I wanted to summarize where I
> think this is at ...

> 2) Having backends write their own buffers out does not significantly
> degrade performance, as those turn into cached OS writes which generally
> execute fast enough to not be a large drag on the backend.

[ itch... ] That assumption scares the heck out of me. It is doubtless
true in a lightly loaded system, but once the kernel is under any kind
of memory pressure I think it's completely wrong. I think designing the
system around this assumption will lead to something that performs great
as long as you're not pushing it hard.

However, your actual specific proposals do not seem to rely on this
assumption extensively, so I wonder why you are emphasizing it.

The only parts of your specific proposals that I find a bit dubious are

> ... It estimates how many
> buffers are probably clean in the space between the strategy point and
> where it's starting at, based on how far ahead it is combined with
> historical data about how many buffers are scanned on average per reusable
> buffer found (the exact computation of this number is the main thing I'm
> still fiddling with).

If you're still fiddling with it then you probably aren't going to get
it right in the next few days. Perhaps you should think about whether
this can be left out entirely for 8.3 and revisited later.

> F) A moving average of buffer allocations is used to predict how many
> clean buffers are expected to be needed in the next delay cycle. The
> original patch from Itagaki doubled the recent allocations to pad this
> out; (3) suggests that's too much.

Maybe you need to put back the eliminated tuning parameter in the form
of the scaling factor to be used here. I don't like 1.0, mainly because
I don't believe your assumption (2). I'm willing to concede that 2.0
might be too much, but I don't know where in between is the sweet spot.

Also, we might need a tuning parameter for the reaction speed of the
moving average --- what are you using for that?

regards, tom lane