Re: [HACKERS] Bgwriter behavior

This change isn't going to make it for RC3, and it probably not
something we want to rush.

I think there are a few issues involved:

o everyone agrees the current meaning of bgwriter_percent is
useless (percent of dirty buffers)
o removal of bgwriter_percent will cause problems because
postgresql.conf is only installed via initdb, so beta users
will have to have some workaround so their existing
postgresql.conf files work.
o bgwriter_percent and bgwriter_maxpages are duplicate for a
given number of buffers and it isn't clear which one takes
precedence.
o 8.1 might use these variables with different meanings,
causing slight upgrade confusion.
o Another idea is for bgwriter_percent to control how much of
the buffer is scanned.

Tom feels bgwriter_maxpages is good because it allows the user to
specify the I/O traffic, while bgwriter_percent as total pages (not just
dirty ones) is perhaps easier to set a default (I/O load varies based on
buffer cache size) and perhaps easier to understand.

I am not sure what to suggest at this point but whatever solution we use
should take the above issues into account.

---------------------------------------------------------------------------

Simon Riggs wrote:
> On Fri, 2004-12-31 at 01:14, Bruce Momjian wrote:
> > Simon Riggs wrote:
> > > On Mon, 2004-12-27 at 22:21, Bruce Momjian wrote:
> > > > Should we consider at least adjusting the meaning of bgwriter_percent?
> > >
> > > Yes. As things stand, this is the only change that seems safe.
> > >
> > > Here's a very short patch that implements this change within BufferSync
> > > in bufmgr.c
> > >
> > > - No algorithm changes
> > > - No error message changes
> > > - Only change is the call to StrategyDirtyBufferList is made using the
> > > maximum number of buffers that will be cleaned, rather than uselessly
> > > trawling through all of shared_buffers
> > >
> > > This changes the meaning of bgwriter_percent from "percent of dirty
> > > buffers" to "percent of shared_buffers". The default settings of 1% of
> > > 1000 buffers gives up to 10 dirty block writes every 250ms
> > >
> > > Benefit: allows performance tuning by increases options for setting
> > > bgwriter_delay which would otherwise have an ineffectually high minimum
> > > setting
> > >
> > > Risk: low
> > >
> > > 1-line doc patch to follow, if this is approved.
> >
> > I am not objecting to the patch, but what value is there in having both
> > bgwriter_percent and bgwriter_maxpages? Seems both are redundant and
> > that one would be enough.
>
> In brief:
> i) for now: as little change as possible is good
> ii) the two parameters are OK
> iii) trying to decide an alternative takes time, which we do not have
> iv) what is presented here is simply a performance bug fix, not the best
> long term alternative...
>
> I'd like to move quickly: if we do this (or an alternative), it has to
> be done soon and it would be easy to discuss this until we run out of
> time. Could we vote: in RC3, or not?
>
> In more detail...
>
> The value of having both is:
> i) as little change as possible at this stage of RC - the main one
> ...which gives us stability
> ...and also avoids having to re-discuss what they *should* be
>
> ii) Having two isn't that bad. bgwriter_percent auto adjusts the length
> of the to-be-cleaned-list, so it is roughly useful anywhere between 500
> and 10000 shared_buffers. That is IMHO slightly more useful than a hard
> definition set via bgwriter_maxpages, since that is likely to be set
> wrong anyway - but has some value as an outside limit on the number of
> pages. [You may wish to set shared_buffers > 10000 even on smaller
> servers, since many now have 2GB RAM and yet a relatively poor I/O
> subsystem. Having maxpages set separately allows the majority of people
> to set shared_buffers higher without swamping their I/O subsystems
> because they didn't know about the r8.0 bgwriter feature/parameters]
>
> iii) changing the parameters might tempt us towards changing the
> algorithm, which is not a topic we have reached agreement on
>
> iv) I see it as a goal to remove all of those parameters anyway, as well
> as explore some of the many options and ideas everybody has presented,
> so further change is likely at the next release whatever is done now.
>
> The patch is as simple as I can make it and yet remove the unnecessary
> performance effect in the existing code. Thanks to Neil and others for
> showing that this was possible...I see this patch as a team effort.
>
> I've already spoken against larger change and would do so again now: if
> we don't agree this change, then I would vote for no-change.... simply
> because this patch is minimal change. We *suspect* further change is
> beneficial but we have no evidence to support what that change should
> be, amongst the large range of possible solutions proposed.
>
> --
> Best Regards, Simon Riggs
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
> (send "unregister YourEmailAddressHere" to majordomo(at)postgresql(dot)org)
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073