Re: Just-in-time Background Writer Patch+Test Results

Greg Smith <gsmith(at)gregsmith(dot)com> writes:
> Tom gets credit for naming the attached patch, which is my latest attempt to
> finalize what has been called the "Automatic adjustment of
> bgwriter_lru_maxpages" patch for 8.3; that's not what it does anymore but
> that's where it started.

I've applied this patch with some revisions.

> -The way I'm getting the passes number back from the freelist.c
> strategy code seems like it will eventually overflow

Yup ... I rewrote that. I also revised the collection of backend-write
count events, which didn't seem to me to be something the freelist.c
code should have anything to do with. It turns out that we can count
them with essentially no overhead by attaching the counter to
the existing fsync-request reporting machinery.

> -Heikki didn't like the way I pass information back from SyncOneBuffer
> back to the background writer.

I didn't either --- it was too complicated and not actually doing
anything useful. I simplified it down to the two bits that were being
used. We can always add more as needed, but since this routine isn't
even exported, I see no need to make it do more than the known callers
need it to do.

I did some marginal tweaking to the way you were doing the moving
averages --- in particular, use a float to avoid strange roundoff
behavior and force the smoothed_alloc average up when a new peak
occurs, instead of only letting it affect the behavior for one
cycle.

Also, I set the default value of bgwriter_lru_multiplier to 2.0,
as 1.0 seemed to be leaving too many writes to the backends in my
testing. That's something we can play with during beta when we'll
have more testing resources available.

I did some other cleanup in BgBufferSync too, like trying to reduce
the chattiness of the debug output, but I don't believe I made any
fundamental change in your algorithm.

Nice work --- thanks for seeing it through!

regards, tom lane