Re: Bgwriter LRU cleaning: we've been going at this all wrong

Heikki Linnakangas <heikki(at)enterprisedb(dot)com> wrote:

> Tom Lane wrote:
> > In fact, the notion of the bgwriter's cleaning scan being "in front of"
> > the clock sweep is entirely backward. It should try to be behind the
> > sweep, ie, so far ahead that it's lapped the clock sweep and is trailing
> > along right behind it, cleaning buffers immediately after their
> > usage_count falls to zero. All the rest of the buffer arena is either
> > clean or has positive usage_count.
>
> That will vary widely depending on your workload, of course, but keeping
> 1/4 of the buffer cache clean seems like overkill to me. If any of those
> buffers are re-dirtied after we write them, the write was a waste of time.

Agreed intuitively, but I don't know how offen backends change usage_count
0 to 1. If the rate is high, backward-bgwriter would not work. It seems to
happen frequently when we use large shared buffers.

I read Tom is changing the bgwriter LRU policy from "clean dirty pages
recycled soon" to "clean dirty pages just when they turn out to be less
frequently used", right? I have another thought -- advancing bgwriter's
sweep-startpoint a little ahead.

[buf] 0 lru X bgw-start N
|-----|----------->|-----------------------------|

I think X=0 is in the current behavior and X=N is in the backward-bgwriter.
Are there any other appropriate values for X? It might be good to use
statistics information about buffer usage to modify X runtime.

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center