Re: Too many open files (was Re: spinlock problems reported earlier)

Tom Lane wrote:
>
> Department of Things that Fell Through the Cracks:
>
> Back in August we had concluded that it is a bad idea to trust
> "sysconf(_SC_OPEN_MAX)" as an indicator of how many files each backend
> can safely open. FreeBSD was reported to return 4136, and I have
> since noticed that LinuxPPC returns 1024. Both of those are
> unreasonably large fractions of the actual kernel file table size.
> A few dozen backends opening hundreds of files apiece will fill the
> kernel file table on most Unix platforms.
>
> I'm not sure why this didn't get dealt with, but I think it's a "must
> fix" kind of problem for 7.1. The dbadmin has *got* to be able to
> limit Postgres' appetite for open file descriptors.
>
> I propose we add a new configuration parameter, MAX_FILES_PER_PROCESS,
> with a default value of about 100. A new backend would set its
> max-files setting to the smaller of this parameter or
> sysconf(_SC_OPEN_MAX).
>
> An alternative approach would be to make the parameter be total open files
> across the whole installation, and divide it by MaxBackends to arrive at
> the per-backend limit. However, it'd be much harder to pick a reasonable
> default value if we did it that way.
>
> Comments?

On Linux, at least, the 1024 file limit is a per process limit, the
system wide limit defaults to 4096 and can be easily changed by

echo 16384 > /proc/sys/fs/file-max

(16384 is arbitrary and can be much larger)

I am all for having the ability to tune behavior over the system
reported values, but I think it should be an option which defaults to
the previous behavior.

--
http://www.mohawksoft.com