Re: [Testperf-general] BufferSync and bgwriter

Josh Berkus wrote:
> Simon,
>
>
>>Clearly, OSDL-DBT2 is not a real world test! That is its benefit, since
>>it is heavily instrumented and we are able to re-run it many times
>>without different parameter settings. The application is well known and
>>doesn't suffer that badly from factors that would allow certain effects
>>to be swamped. If it had too much randomness or variation, it would be
>>difficult to interpret.
>
>
> I don't think you followed me. The issue is that for parameters designed to
> "smooth out spikes" like bgwriter and vacuum delay, it helps to have really
> bad spikes to begin with. There's a possibility that the parameters (and
> calculations) that work well for for a "steady-state" OLTP application are
> actually bad for an application with much more erratic usage, just as a high
> sort_mem is good for DSS and bad for OLTP.

I'm a little concerned that in an erratic, or even just a changing
environment there isn't going to be any set of values that are "correct".

If I've got this right, the behaviour we're trying to get is:
1. Starting from the oldest dirty block,
2. Write as many dirty blocks as you can, but don't...
3. Re-write frequently used blocks too much (wasteful)

So, can we not just keep track of two numbers:
1. Change in the number of dirty blocks this time vs last
2. Number of re-writes we perform (count collisions in a hash or
similar - doesn't need to be perfect).

If #1 is increasing, then we need to become more active (reduce
bgwriter_delay, increase bgwriter_maxpages).
If #2 starts to go up, or goes past some threshold then we reduce
activity (increase bgwriter_delay, decrease bgwriter_maxpages).
If of the last N blocks written, C have been collisions then assume
we've run out of low-activity blocks to write, stop and sleep.

This has a downside that the figures will never be completely accurate,
but has the advantage that it will automatically track activity.

I'm clearly beyond my technical knowledge here, so if I haven't
understood / it's impractical / will never work, then don't be afraid to
step up and let me know. If it helps, you could always think of me as an
idiot savant who failed his savant exams :-)

--
Richard Huxton
Archonet Ltd