Re: Kernel kills postgres process - help need

On Wed, 2008-01-09 at 22:57 +0100, Hervé Piedvache wrote:
> Hi,
>
> I have a big trouble with a PostgreSQL server ... regulary since I have added
> 8 Gb of memory, on a server having already 8Gb of memory, I have troubles.
> Nothing else have changed ... I'm on a Dell server, and all the memory
> diagnostics from Dell seems to be good ...
> When I have a lot of connexions (persistante connexions from 6 web apache/php
> serveurs using PDO, about 110 process on each web servers) on the server, or
> long request, it's difficult for me to know when it's appening, the kernel
> seems to kill my postgresql process then the server become completly
> instable, and most of the time need a reboot ...
>
> I'm on Linux kernel 2.6.15 with a version 8.1.10 of PostgreSQL.
> My database is a size of 56G
> RAM = 16 Gb
>

[snip]

> Jan 9 20:30:47 db2 kernel: oom-killer: gfp_mask=0x84d0, order=0

It looks like the Out Of Memory Killer was invoked, and you need to find
out why it was invoked.

I posted to LKML here:

http://kerneltrap.org/mailarchive/linux-kernel/2007/2/12/54202

because linux has a behavior -- which in my opinion is a bug -- that
causes the OOM killer to almost always kill PostgreSQL first, regardless
of whether it was truly the offending process or not.

So, find out which process truly caused the memory pressure that lead to
the OOM being invoked, and fix that problem.

You may also consider some other linux configuration options that make
invocation of OOM killer less likely.

Regards,
Jeff Davis