Re: [HACKERS] Bgwriter behavior

On Sat, 2005-01-01 at 06:20, Bruce Momjian wrote:
> This change isn't going to make it for RC3, and it probably not
> something we want to rush.

OK. Thank you.

> I think there are a few issues involved:
>
> o everyone agrees the current meaning of bgwriter_percent is
> useless (percent of dirty buffers)
> o removal of bgwriter_percent will cause problems because
> postgresql.conf is only installed via initdb, so beta users
> will have to have some workaround so their existing
> postgresql.conf files work.
> o bgwriter_percent and bgwriter_maxpages are duplicate for a
> given number of buffers and it isn't clear which one takes
> precedence.
> o 8.1 might use these variables with different meanings,
> causing slight upgrade confusion.
> o Another idea is for bgwriter_percent to control how much of
> the buffer is scanned.
>

Agreed.

Would add as item #1: current behaviour of bgwriter causes sub-optimal
performance for 8.0, for systems with a high write workload, more CPUs
and higher shared_buffers.

> Tom feels bgwriter_maxpages is good because it allows the user to
> specify the I/O traffic, while bgwriter_percent as total pages (not just
> dirty ones) is perhaps easier to set a default (I/O load varies based on
> buffer cache size) and perhaps easier to understand.
>

Agreed.

> I am not sure what to suggest at this point but whatever solution we use
> should take the above issues into account.

Well, I think we're saying: its not in 8.0 now, and we take our time to
consider patches for 8.1 and accept the situation that the parameter
names/meaning will change in next release.

The patch is there if that decision changes, but I'll say no more on it.

> ---------------------------------------------------------------------------
>
> Simon Riggs wrote:
> > On Fri, 2004-12-31 at 01:14, Bruce Momjian wrote:
> > > Simon Riggs wrote:
> > > > On Mon, 2004-12-27 at 22:21, Bruce Momjian wrote:
> > > > > Should we consider at least adjusting the meaning of bgwriter_percent?
> > > >
> > > > Yes. As things stand, this is the only change that seems safe.
> > > >
> > > > Here's a very short patch that implements this change within BufferSync
> > > > in bufmgr.c
> > > >
> > > > - No algorithm changes
> > > > - No error message changes
> > > > - Only change is the call to StrategyDirtyBufferList is made using the
> > > > maximum number of buffers that will be cleaned, rather than uselessly
> > > > trawling through all of shared_buffers
> > > >
> > > > This changes the meaning of bgwriter_percent from "percent of dirty
> > > > buffers" to "percent of shared_buffers". The default settings of 1% of
> > > > 1000 buffers gives up to 10 dirty block writes every 250ms
> > > >
> > > > Benefit: allows performance tuning by increases options for setting
> > > > bgwriter_delay which would otherwise have an ineffectually high minimum
> > > > setting
> > > >
> > > > Risk: low
> > > >
> > > > 1-line doc patch to follow, if this is approved.
> > >
> > > I am not objecting to the patch, but what value is there in having both
> > > bgwriter_percent and bgwriter_maxpages? Seems both are redundant and
> > > that one would be enough.
> >
> > In brief:
> > i) for now: as little change as possible is good
> > ii) the two parameters are OK
> > iii) trying to decide an alternative takes time, which we do not have
> > iv) what is presented here is simply a performance bug fix, not the best
> > long term alternative...
> >
> > I'd like to move quickly: if we do this (or an alternative), it has to
> > be done soon and it would be easy to discuss this until we run out of
> > time. Could we vote: in RC3, or not?
> >
> > In more detail...
> >
> > The value of having both is:
> > i) as little change as possible at this stage of RC - the main one
> > ...which gives us stability
> > ...and also avoids having to re-discuss what they *should* be
> >
> > ii) Having two isn't that bad. bgwriter_percent auto adjusts the length
> > of the to-be-cleaned-list, so it is roughly useful anywhere between 500
> > and 10000 shared_buffers. That is IMHO slightly more useful than a hard
> > definition set via bgwriter_maxpages, since that is likely to be set
> > wrong anyway - but has some value as an outside limit on the number of
> > pages. [You may wish to set shared_buffers > 10000 even on smaller
> > servers, since many now have 2GB RAM and yet a relatively poor I/O
> > subsystem. Having maxpages set separately allows the majority of people
> > to set shared_buffers higher without swamping their I/O subsystems
> > because they didn't know about the r8.0 bgwriter feature/parameters]
> >
> > iii) changing the parameters might tempt us towards changing the
> > algorithm, which is not a topic we have reached agreement on
> >
> > iv) I see it as a goal to remove all of those parameters anyway, as well
> > as explore some of the many options and ideas everybody has presented,
> > so further change is likely at the next release whatever is done now.
> >
> > The patch is as simple as I can make it and yet remove the unnecessary
> > performance effect in the existing code. Thanks to Neil and others for
> > showing that this was possible...I see this patch as a team effort.
> >
> > I've already spoken against larger change and would do so again now: if
> > we don't agree this change, then I would vote for no-change.... simply
> > because this patch is minimal change. We *suspect* further change is
> > beneficial but we have no evidence to support what that change should
> > be, amongst the large range of possible solutions proposed.
> >

--
Best Regards, Simon Riggs