Re: select on 22 GB table causes "An I/O error occured while sending to the backend." exception

On Wed, 27 Aug 2008, david(at)lang(dot)hm wrote:
> if memory overcommit is disabled, the kernel checks to see if you have an
> extra 1G of ram available, if you do it allows the process to continue, if
> you don't it tries to free memory (by throwing away cache, swapping to disk,
> etc), and if it can't free the memory will return a memroy allocation error
> (which I believe will cause firefox to exit).

Remember that the memory overcommit check is checking against the amount
of RAM + swap you have - not just the amount of RAM. When a fork occurs,
hardly any extra actual RAM is used (due to copy on write), but the
potential is there for the process to use it. If overcommit is switched
off, then you just need to make sure there is *plenty* of swap to convince
the kernel that it can actually fulfil all of the memory requests if all
the processes behave badly and all shared pages become unshared. Then the
consequences of processes actually using that memory are that the machine
will swap, rather than the OOM killer having to act.

Of course, it's generally bad to run a machine with more going on than
will fit in RAM.

Neither swapping nor OOM killing are particularly good - it's just a
consequence of the amount of memory needed being unpredictable.

Probably the best solution is to just tell the kernel somehow to never
kill the postmaster.

Matthew

--
<Taking apron off> And now you can say honestly that you have been to a
lecture where you watched paint dry.
- Computer Graphics Lecturer