Re: Large PostgreSQL servers

On Thu, Mar 22, 2012 at 9:29 AM, Merlin Moncure <mmoncure(at)gmail(dot)com> wrote:
> On Thu, Mar 22, 2012 at 10:02 AM, Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com> wrote:
>> There's other issues you run into with large shared_buffers as well.
>> If you've got a large shared_buffers setting, but only regularly hit a
>> small subset of your db (say 32GB shared_buffers but only hit 4G or so
>> regularly in your app) then it's quite possible that older
>> shared_buffer segments will get swapped out because they're not being
>> used.  Then, when the db goes to hit a page in shared_buffers, the OS
>> will have to swap it back in.  What was supposed to make your db much
>> faster has now made it much slower.
>>
>> With Linux, the OS tends to swap out unused memory to make room for
>> file buffers.  While you can change the swappiness settings to 0 to
>> slow it down, the OS will eventually swap out the least used segments
>> anyway.  The only solution on large memory servers is often to just
>> turn off swap.
>
> Right -- but my take on that is that hacking the o/s to disable swap
> is dealing with symptoms of problem related to server
> misconfiguration.

You can configure a big memory linux server anyway you want. After a
while, they seem to go crazy anyway and start swapping even when
you've told them not to.

> In particular it probably means shared_buffers is set too high...the
> o/s thinks it needs that memory more than you do and it may very well
> be right.

I've had machines with 128GB RAM and a 4G shared_buffers start
swapping for no apparent reason and just fall over. There's no memory
pressure etc, just kswapd decides to go nuts and start swapping.

This was on Ubuntu 10.04 and 12.04 and RHEL 5.2 through 5.latest with
all updates. These machines typically had ~90GB+ of kernel cache and
zero memory pressure.