Re: Listen/Notify feedback

From: Andrew Smith <laconical(at)gmail(dot)com>
To: Rita <rmorgan466(at)gmail(dot)com>
Cc: Brian Dunavant <dunavant(at)gmail(dot)com>, "pgsql-general(at)postgresql(dot)org >> PG-General Mailing List" <pgsql-general(at)postgresql(dot)org>
Subject: Re: Listen/Notify feedback
Date: 2020-07-12 12:52:30
Message-ID: CALr=acEZ4Xggp7AUDKwuRVFy=9SyF-kmLtYSpM89HnZ1MN_j2Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

On Sun, 12 Jul 2020 at 21:39, Rita <rmorgan466(at)gmail(dot)com> wrote:

> Thats good to know. Are there some standard patterns or best practices I
> should follow when using messaging and with listen/notify?
>
> On Sat, Jul 11, 2020 at 1:44 PM Brian Dunavant <dunavant(at)gmail(dot)com> wrote:
>
>> One aspect is if there is no one listening when a notify happens, the
>> message is lost (e.g. no durability). If this is important to you, it can
>> be addressed by writing the messages to a table as well when you NOTIFY,
>> and the listener deletes messages after they are processed. On connection
>> the listener can query the table to catch up on any missed messages, or
>> messages that were mid-process during a crash. This is trickier with more
>> than one listener. This isn't a whole lot more efficient than just using
>> the table alone, but it saves you from having to poll so better response
>> times.
>>
>> On Sat, Jul 11, 2020 at 8:58 AM Rita <rmorgan466(at)gmail(dot)com> wrote:
>>
>>> I am investigating various pub/sub tools such as ActiveMQ, Rabbit,
>>> Redis, etc.I came across Postgresql Listen/Notify and was easily able to
>>> write code to listen to messages. For the people who have been using this
>>> for a while: what are its downsides, things to consider when writing good
>>> code that use pub/sub, how do you deal with large messages, can I have
>>> subscribers listen to replica nodes?
>>>
>>> Thanks
>>> --
>>> --- Get your facts first, then you can distort them as you please.--
>>>
>>
A couple of years ago I started looking into listen/notify in PG10 and
found that the throughput decreased quite a bit as I added more and more
listeners. Given the number of apps I needed to have listening and the
number of messages that I expected to be consuming, I ended up writing a
single listener app which then republished the messages via MQTT. Not sure
if the performance has improved in subsequent versions (or whether this
will affect you at all) but it's something to keep in mind.

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2020-07-12 14:16:52 Re: Listen/Notify feedback
Previous Message Rita 2020-07-12 11:39:09 Re: Listen/Notify feedback