Re: Escaping from blocked send() reprised.

On 2014-09-28 00:54:21 +0200, Andres Freund wrote:
> I've invested some more time in this:

And yet round of time spent.

The major change since last round is that I've introduced a local latch
that exists in every process. If InitProcess() is run, that latch is
replaced by the shared process latch and the reverse happens in
ProcKill. To implement this, I decided to remove some code duplication
around process initialization.

The major reason to do is that this allows us to get rid of the special
case where MyProc isn't available yet during early process startup,
where the socket still had to be in blocking mode. Instead we can now
rely on the latch all the time.

Other than that I've significantly cleaned up and tested the
patchset. Unfortunately that testing has brought a significant number of
SSL bugs to light, but they all turned out to be independent of these
changes. I'll try to detail them in a separate email early next week.

I've done some performance measurements, to verify this doesn't cause a
regression. When testing with 'SELECT 1' as a trivial statement I
couldn't measure any regression at 32 pgbench clients/threads on my 4
core (i7-4800MQ) laptop. At 390 clients I saw about 1.6%. With statement
that actually does something that difference vanished. Personally I'm
ok with that.

The patches are:

0001-Allow-latches-to-wait-for-socket-writability-without.patch
Imo pretty close to commit and can be committed independently.

0002-Commonalize-process-startup-code.patch
The above mentioned deduplication. Needs a review (completely new),
but it's imo a clear improvement and allows for a fair amount of
future/further deduplication.

0003-Add-a-default-local-latch-for-use-in-signal-handlers.patch
Adds the default latch + the logic to switch to shared latch while
attached. I think it's a good idea, but I'd like some feedback.

0004-Use-a-nonblocking-socket-for-FE-BE-communication-and.patch
The earlier patch that converts the communication to use latches
with some added cleanup/improvements. Specifically we don't have to
rely on MyProc->procLatch anymore due to 0003 which gets rid of
some ugly code in be-secure.c and postgres.c

I think this patch might not be safe without 0005 because I can't
convince myself that it's safe to interrupt latch waits with work
that actually might also use the same latch (sinval
interrupts). But it's easier to review this way.

0005-Introduce-and-use-infrastructure-for-interrupt-proce.patch
Actually move most of sinval.c/async.c's interrupt handling out of
the signal handlers and use the latches. This is the cleaned up
version of the earlier commit.

0006-WIP-Process-die-interrupts-while-reading-writing-fro.patch
Not much has changed, needs at least some comments. I want to get
the other stuff done first.

There remains one 'FIXME' after all the patches, which is
interactive_getc() won't react to catchup/notify interrupts - which, as
far as I can see, is fine, as there are none.

Comments?

Greetings,

Andres Freund