Re: Re: Too many open files (was Re: spinlock problems reported earlier)

Department of Things that Fell Through the Cracks:

Back in August we had concluded that it is a bad idea to trust
"sysconf(_SC_OPEN_MAX)" as an indicator of how many files each backend
can safely open. FreeBSD was reported to return 4136, and I have
since noticed that LinuxPPC returns 1024. Both of those are
unreasonably large fractions of the actual kernel file table size.
A few dozen backends opening hundreds of files apiece will fill the
kernel file table on most Unix platforms.

I'm not sure why this didn't get dealt with, but I think it's a "must
fix" kind of problem for 7.1. The dbadmin has *got* to be able to
limit Postgres' appetite for open file descriptors.

I propose we add a new configuration parameter, MAX_FILES_PER_PROCESS,
with a default value of about 100. A new backend would set its
max-files setting to the smaller of this parameter or
sysconf(_SC_OPEN_MAX).

An alternative approach would be to make the parameter be total open files
across the whole installation, and divide it by MaxBackends to arrive at
the per-backend limit. However, it'd be much harder to pick a reasonable
default value if we did it that way.

Comments?

regards, tom lane