Re: Just-in-time Background Writer Patch+Test Results

>>> On Wed, Sep 5, 2007 at 10:31 PM, in message
<Pine(dot)GSO(dot)4(dot)64(dot)0709052324020(dot)25284(at)westnet(dot)com>, Greg Smith
<gsmith(at)gregsmith(dot)com> wrote:
>
> -There are two magic constants in the code:
>
> int smoothing_samples = 16;
> float scan_whole_pool_seconds = 120.0;
>

> I personally
> don't feel like these constants need to be exposed for tuning purposes;

> Determining
> whether these should be exposed as GUC tunables is certainly an open
> question though.

If you exposed the scan_whole_pool_seconds as a tunable GUC, that would
allay all of my concerns about this patch. Basically, our problems were
resolved by getting all dirty buffers out to the OS cache within two
seconds; any longer than that and the OS cache didn't reach its trigger
point for pushing out to the controller cache in time to prevent the glut
which locks everything up. I also suspect that this interval kept the OS
cache more aware of frequently updated pages, so that it could avoid
unnecessary physical writes under its own logic.

While I'm hoping that the new checkpoint techniques will be a better
solution, I can't count on that without significant testing in our
environment, and I really want a fall-back. The metric you emphasized was
the percentage of PostgreSQL writes to the OS cache which were handled by
the background writer, which doesn't necessarily correspond to a solution
to the glut, which is based on the peak number of total writes presented
to the controller by the OS within a small window of time.

-Kevin