OperationalError: FATAL: could not open file <filename>: Too many open files in system

From: Ben Welsh <ben(dot)welsh(at)gmail(dot)com>
To: pgsql-general(at)postgresql(dot)org
Subject: OperationalError: FATAL: could not open file <filename>: Too many open files in system
Date: 2009-04-23 20:39:04
Message-ID: a5ad29010904231339s759393e2l8f33f41eb5489cf4@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hello,

I'm managing a pgsql->django->mod_python->apache machine that serves no
media, but handles postgres, memcached and apache all by itself.

Recently I've begun drawing "too many open files in the system" errors, as
seen in the subject line.

Browsing the postgres documentation, I suspect that I need to lower my
`max_files_per_process` from the default 100. But the support team at my
host is pointing the finger at memcached.

Before I do anything, I thought I'd consult the experts here on the list and
see whether you can had any opinions. Here are some of the basic config
settings that, from cruising around the web and talking to listservs, I've
come to suspect are crucial to this sort of problem. Really, any advice is
appreciated, because I've been getting competiting advice from different
sources and I'm not sure exactly where to go with this.

And, of course, thanks listers and developers for such a great product and
community.

$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
pending signals (-i) 1024
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 137215
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

postgres.conf (max_files_per_process is commented out)
max_connections = 100
shared_buffers = 1000

$ httpd -l

Compiled in modules:
core.c
prefork.c
http_core.c
mod_so.c

httpd.conf...

Timeout 20
KeepAlive Off
MaxKeepAliveRequests 100
KeepAliveTimeout 5
<IfModule prefork.c>
StartServers 8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000
</IfModule>

$ cat /proc/sys/fs/file-nr
73320 0 262144

$ lsof | awk '{print $1}' | sort | uniq -c
1 COMMAND
10 awk
49 bash
17 crond
5154 httpd
9 init
25 lsof
39 memcached
807 postmaste
62 saslauthd
69 sendmail
28 sh
7 sort
183 sshd
13 syslogd
8 uniq
19 xinetd

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Peter Eisentraut 2009-04-23 21:11:03 Re: utf8 encoding for template0 and template1?
Previous Message Conrad Lender 2009-04-23 19:33:11 standard_conforming_strings and pg_escape_string()