Testing LISTEN/NOTIFY more effectively

Since we have three or four different NOTIFY improvement proposals
floating around in the current CF, I got a bit distressed at the lack
of test coverage for that functionality. While the code coverage
report makes it look like commands/async.c isn't so badly covered,
that's all coming from src/test/regress/sql/async.sql and
src/test/isolation/specs/async-notify.spec. A look at those files
shows that nowhere is there any actual verification that "NOTIFY foo"
results in a report of "foo" being received; let alone any
more-advanced questions such as whether de-duplication of reports
happens.

The reason for this is that psql's report of a notification event
includes the sending backend's PID, making it impossible for the
test output to be stable; neither the core nor isolation regression
test frameworks can cope with unpredictable output.

We've occasionally batted around ideas for making it possible for
these test frameworks to verify not-entirely-fixed output, and that
would be a good thing to do, but I'm not volunteering for that today.

So, if we'd like to have more thorough NOTIFY coverage without going
to that much work, what to do? I thought of a few alternatives:

1. Write a TAP test instead of using the old test frameworks, and
use regexps to check the expected output. But this seems ugly and
hard to get right. In particular, our TAP infrastructure doesn't
seem to be (easily?) capable of running concurrent psql sessions,
so it doesn't seem like there's any good way to test cross-session
notifies that way.

2. Change psql so that there's a way to get NOTIFY messages without
the sending PID. For testing purposes, it'd be sufficient to know
whether the sending PID is our own backend's PID or not. This idea
is not horrible, and it might even be useful for outside purposes
if we made it flexible enough; which leads to thoughts like allowing
the psql user to set a format-style string, similar to the PROMPT
strings but with escapes for channel name, payload, etc. I foresee
bikeshedding, but we could probably come to an agreement on a feature
like that.

3. On the other hand, that doesn't help much for the isolation tester
because it doesn't go through psql. In fact, AFAICS it doesn't have
any provision for dealing with notify messages at all; probably,
in the async-notify.spec test, the listening session builds up a
queue of notifies that it never reads. So we could imagine addressing
the testing gap strictly inside the isolation-tester framework, if we
added the ability for it to detect and print notifications in a
test-friendly format (no explicit PIDs).

I'm finding alternative #3 the most attractive, because we really
want isolation-style testing for LISTEN/NOTIFY, and this solution
doesn't require designing a psql feature that we'd need to get
consensus on.

Before I start coding that, any thoughts or better ideas?

regards, tom lane