Re: Bump soft open file limit (RLIMIT_NOFILE) to hard limit on startup

On 2/11/25 20:20, Tom Lane wrote:
> Jelte Fennema-Nio <postgres(at)jeltef(dot)nl> writes:
>> The default open file limit of 1024 is extremely low, given modern
>> resources and kernel architectures. The reason that this hasn't changed
>> change is because doing so would break legacy programs that use the
>> select(2) system call in hard to debug ways. So instead programs that
>> want to opt-in to a higher open file limit are expected to bump their
>> soft limit to their hard limit on startup. Details on this are very well
>> explained in a blogpost by the systemd author[1].
>
> On a handy Linux machine (running RHEL9):
>
> $ ulimit -n
> 1024
> $ ulimit -n -H
> 524288
>
> I'm okay with believing that 1024 is unreasonably small, but that
> doesn't mean I think half a million is a safe value. (Remember that
> that's *per backend*.) Postgres has run OSes out of FDs in the past
> and I don't believe we couldn't do it again.
>
> Also, the argument you cite is completely recent-Linux-centric
> and does not consider the likely effects on other platforms.
> To take one example, on current macOS:
>
> $ ulimit -n
> 4864
> $ ulimit -n -H
> unlimited
>
> (Hm, so Apple wasn't impressed by the "let's not break select(2)"
> argument. But I digress.)
>
> I'm afraid this patch will replace "you need to tune ulimit -n
> to get best performance" with "you need to tune ulimit -n to
> avoid crashing your system". Does not sound like an improvement.
>
> Maybe a sanity limit on how high we'll try to raise the ulimit
> would help.
>

I agree the defaults may be pretty low for current systems, but do we
want to get into the business of picking a value and overriding whatever
value is set by the sysadmin? I don't think a high hard limit should be
seen as an implicit permission to just set is as the soft limit.

Imagine you're a sysadmin / DBA who picks a low soft limit (for whatever
reason - there may be other stuff running on the system, ...). And then
postgres starts and just feels like bumping the soft limit. Sure, the
sysadmin can lower the hard limit and then we'll respect that, but I
don't recall any other tool requiring this approach, and it would
definitely be quite surprising to me.

I did run into bottlenecks due to "too few file descriptors" during a
recent experiments with partitioning, which made it pretty trivial to
get into a situation when we start trashing the VfdCache. I have a
half-written draft of a blog post about that somewhere.

But my conclusion was that it's damn difficult to even realize that's
happening, especially if you don't have access to the OS / perf, etc. So
my takeaway was we should improve that first, so that people have a
chance to realize they have this issue, and can do the tuning. The
improvements I thought about were:

- track hits/misses for the VfdCache (and add a system view for that)

- maybe have wait event for opening/closing file descriptors

- show max_safe_fds value somewhere, not just max_files_per_process
(which we may silently override and use a lower value)

Even if we decide to increase the soft limit somehow, I think it'd still
be useful to make this info available (or at least some of it).

regards

--
Tomas Vondra