Re: [Lsf-pc] Linux kernel impact on PostgreSQL performance

On Tue, Jan 14, 2014 at 05:38:10PM -0700, Jonathan Corbet wrote:
> On Wed, 15 Jan 2014 09:23:52 +1100
> Dave Chinner <david(at)fromorbit(dot)com> wrote:
>
> > It appears to me that we are seeing large memory machines much more
> > commonly in data centers - a couple of years ago 256GB RAM was only
> > seen in supercomputers. Hence machines of this size are moving from
> > "tweaking settings for supercomputers is OK" class to "tweaking
> > settings for enterprise servers is not OK"....
> >
> > Perhaps what we need to do is deprecate dirty_ratio and
> > dirty_background_ratio as the default values as move to the byte
> > based values as the defaults and cap them appropriately. e.g.
> > 10/20% of RAM for small machines down to a couple of GB for large
> > machines....
>
> I had thought that was already in the works...it hits people on far
> smaller systems than those described here.
>
> http://lwn.net/Articles/572911/
>
> I wonder if anybody ever finished this work out for 3.14?

Not that I know of. This patch was suggested as the solution to the
slow/fast drive issue that started the whole thread:

http://thread.gmane.org/gmane.linux.kernel/1584789/focus=1587059

but I don't see it in a current kernel. It might be in Andrew's tree
for 3.14, but I haven't checked.

However, most of the discussion in that thread about dirty limits
was a side show that rehashed old territory. Rate limiting and
throttling in a generic, scalable manner is a complex problem. We've
got some of the infrastructure we need to solve the problem, but
there was no conclusion as to the correct way to connect all the
dots. Perhaps it's another topic for the LSFMM conf?

Cheers,

Dave.
--
Dave Chinner
david(at)fromorbit(dot)com