Re: [Lsf-pc] Linux kernel impact on PostgreSQL performance

On 14/01/14 22:23, Dave Chinner wrote:
> On Tue, Jan 14, 2014 at 11:40:38AM -0800, Kevin Grittner wrote:
>> To quantify that, in a production setting we were seeing pauses of
>> up to two minutes with shared_buffers set to 8GB and default dirty
> ^^^^^^^^^^^^^
>> page settings for Linux, on a machine with 256GB RAM and 512MB
> ^^^^^^^^^^^^^
> There's your problem.
>
> By default, background writeback doesn't start until 10% of memory
> is dirtied, and on your machine that's 25GB of RAM. That's way to
> high for your workload.
>
> It appears to me that we are seeing large memory machines much more
> commonly in data centers - a couple of years ago 256GB RAM was only
> seen in supercomputers. Hence machines of this size are moving from
> "tweaking settings for supercomputers is OK" class to "tweaking
> settings for enterprise servers is not OK"....
>
> Perhaps what we need to do is deprecate dirty_ratio and
> dirty_background_ratio as the default values as move to the byte
> based values as the defaults and cap them appropriately. e.g.
> 10/20% of RAM for small machines down to a couple of GB for large
> machines....

<whisper> Perhaps the kernel needs a dirty-amount control measured
in time units rather than pages (it being up to the kernel to
measure the achievable write rate)...
--
Cheers,
Jeremy