Re: Big trouble with memory !!

Hervé Piedvache wrote:
> On Wednesday 06 April 2005 17:03, Richard Huxton wrote:
>
>>Hervé Piedvache wrote:
>>
>>>Hi,
>>>
>>>We have switched to kernel 2.6.11.6 from kernel 2.4.26 ... since this
>>>date we have many troubles with PostgreSQL and most of them seems to be
>>>memory troubles.
>>>
>>>As far as we can see, kernel kills the postmaster process when it begins
>>>to use swap. You can see the output from dmesg at the bottom of the
>>>message. The first thing I am not sure to understand is that the kernel
>>>should kill processes to reallocate memory only when physical memory and
>>>swap memory are exhausted, shouldn't it ?
>>>Second thing: it seems to be related to our kernel switch as it did not
>>>happen before that.
>>>
>>>This can occur when queries / a vacuum require too much memory to run.
>>>I have configured my kernel with these options:
>>># shared mem
>>>kernel.shmmax= 641604096
>>># semaphore
>>>kernel.sem = 250 32000 100 400
>>>fs.file-max=65536
>>># overcommit
>>>vm.overcommit_memory=2
>>>
>>>Does anyone can explain me why I have this problem and how to resolve it
>>>?
>>>
>>>This server is a dedicated PostgreSQL server with 4Gb of RAM.
>>
>>You might want to try vm.overcommit_memory=1. You don't appear to be the
>>only one suffering from an over-zealous oom-killer.
>>
>
>
> But if you look at the documentation of PostgreSQL :
> -------------------------------------------------------------------------------------------------
> 16.5.3. Linux Memory Overcommit
>
> In Linux 2.4 and later, the default virtual memory behavior is not optimal for
> PostgreSQL. Because of the way that the kernel implements memory overcommit,
> the kernel may terminate the PostgreSQL server (the postmaster process) if
> the memory demands of another process cause the system to run out of virtual
> memory.
>
> If this happens, you will see a kernel message that looks like this (consult
> your system documentation and configuration on where to look for such a
> message):
>
> Out of Memory: Killed process 12345 (postmaster).
>
> This indicates that the postmaster process has been terminated due to memory
> pressure. Although existing database connections will continue to function
> normally, no new connections will be accepted. To recover, PostgreSQL will
> need to be restarted.
>
> One way to avoid this problem is to run PostgreSQL on a machine where you can
> be sure that other processes will not run the machine out of memory.
>
> On Linux 2.6 and later, a better solution is to modify the kernel's behavior
> so that it will not "overcommit" memory. This is done by selecting strict
> overcommit mode via sysctl:
>
> sysctl -w vm.overcommit_memory=2
>
> or placing an equivalent entry in /etc/sysctl.conf. You may also wish to
> modify the related setting vm.overcommit_ratio. For details see the kernel
> documentation file Documentation/vm/overcommit-accounting.
>
> Some vendors' Linux 2.4 kernels are reported to have early versions of the 2.6
> overcommit sysctl. However, setting vm.overcommit_memory to 2 on a kernel
> that does not have the relevant code will make things worse not better. It is
> recommended that you inspect the actual kernel source code (see the function
> vm_enough_memory in the file mm/mmap.c) to verify what is supported in your
> copy before you try this in a 2.4 installation. The presence of the
> overcommit-accounting documentation file should not be taken as evidence that
> the feature is there. If in any doubt, consult a kernel expert or your kernel
> vendor.
> -------------------------------------------------------------------------------------------------

Ah, but I believe there have been some issues with the Linux kernel
recently:
http://www.kerneltraffic.org/kernel-traffic/kt20050212_296.html#6

If the oom-killer is running riot on your system that would suggest the
bug is still there in some form.

--
Richard Huxton
Archonet Ltd