Re: pgsql: Add kqueue(2) support to the WaitEventSet API.

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Cc: Rémi Zara <remi_zara(at)mac(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: pgsql: Add kqueue(2) support to the WaitEventSet API.
Date: 2020-02-21 04:30:07
Message-ID: 18091.1582259407@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-committers pgsql-hackers

I wrote:
> Clearly, we gotta do something about that too. Maybe bumping up
> NUM_RESERVED_FDS would be a good idea, but I feel like more-honest
> accounting for permanently-eaten FDs would be a better idea. And
> in any case we can't suppose that there's a fixed upper limit on
> the number of postgres_fdw connections. I'm liking the idea I floated
> earlier of letting postgres_fdw forcibly close the oldest LRU entry.

A late-night glimmer of an idea: suppose we make fd.c keep count,
not just of the open FDs under its control, but also of open FDs
not under its control. The latter count would include the initial
FDs (stdin/stdout/stderr), and FDs allocated by OpenTransientFile
et al, and we could provide functions for other callers to report
that they just allocated or released a FD. So postgres_fdw could
report, when it opens or closes a PGconn, that the count of external
FDs should be increased or decreased. fd.c would then forcibly
close VFDs as needed to keep NUM_RESERVED_FDS worth of slop. We
still need slop, to provide some daylight for code that isn't aware
of this mechanism. But we could certainly get all these known
long-lived FDs to be counted, and that would allow fd.c to reduce
the number of open VFDs enough to ensure that some slop remains.

This idea doesn't fix every possible problem. For instance, if you
have a plperlu function that wants to open a bunch of files, I don't
see any easy way to ensure we release VFDs to make that possible.
But it's sure an improvement on the status quo.

regards, tom lane

In response to

Responses

Browse pgsql-committers by date

  From Date Subject
Next Message Peter Eisentraut 2020-02-21 08:28:06 pgsql: Require stdint.h
Previous Message Michael Paquier 2020-02-21 03:08:13 pgsql: Doc: Fix instructions to control build environment with MSVC

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2020-02-21 06:07:12 Re: [Patch] Make pg_checksums skip foreign tablespace directories
Previous Message Michael Paquier 2020-02-21 04:18:00 Re: client-side fsync() error handling