Re: Listen / Notify - what to do when the queue is full

Josh Berkus <josh(at)agliodbs(dot)com> writes:
> (4) drop *old* notifications if the queue is full.

> Since everyone has made the point that LISTEN is not meant to be a full
> queueing system, I have no problem dropping notifications LRU-style.

NO, NO, NO, a thousand times no!

That turns NOTIFY into an unreliable signaling system, and if I haven't
made this perfectly clear yet, any such change will be committed over my
dead body.

If we are unable to insert a new message into the queue, the correct
recourse is to fail the transaction that is trying to insert the *new*
message. Not to drop messages from already-committed transactions.
Failing the current transaction still leaves things in a consistent
state, ie, you don't get messages from aborted transactions but that's
okay because they didn't change the database state.

I think Greg has a legitimate concern about whether this redesign
reduces the usefulness of NOTIFY for existing use-cases, though.
Formerly, since pg_listener would effectively coalesce notifies
across multiple sending transactions instead of only one, it was
impossible to "overflow the queue", unless maybe you managed to
bloat pg_listener to the point of being out of disk space, and
even that was pretty hard. There will now be a nonzero chance
of transactions failing at commit because of queue full. If the
chance is large this will be an issue. (Is it sane to wait for
the queue to be drained?)

BTW, did we discuss the issue of 2PC transactions versus notify?
The current behavior of 2PC with notify is pretty cheesy and will
become more so if we make this change --- you aren't really
guaranteed that the notify will happen, even though the prepared
transaction did commit. I think it might be better to disallow
NOTIFY inside a prepared xact.

regards, tom lane