Re: Lock/deadlock issues with priority queue in Postgres - possible VACUUM conflicts

On Tue, Jan 31, 2012 at 08:17:57AM +1100, Chris Angelico wrote:
> On Tue, Jan 31, 2012 at 4:12 AM, Marko Kreen <markokr(at)gmail(dot)com> wrote:
> > On Mon, Jan 9, 2012 at 5:58 AM, Chris Angelico <rosuav(at)gmail(dot)com> wrote:
> >> http://wiki.postgresql.org/wiki/PGQ_Tutorial
> >>
> >> PGQ looks promising, but I can't afford the risk of losing calls in
> >> the event that there are no workers to process them (the correct
> >> action is for them simply to languish in the database until one is
> >> started up).
> > PGQ does not lose events - after consumer registers
> > on the queue it is guaranteed to see all events.
> >
> > So it's a matter of registering your consumers
> > before anything interesting happens in database.
> > The actual consumers do not need to be running
> > at that moment.
>
> Ah, I think I understand. So registering a consumer simply means
> registering its textual name.

Exactly.

> I was under the impression that it
> registered the session/connection it was on. PGQ may still be
> unsuitable (it's more geared toward replication than a shared-workload
> scenario), but that's my primary concern solved.

Calling it replication-specific is wrong, there is nothing replication
specific there. But it *is* geared towards batch-processing in a
situtation where few seconds of latencty is OK. Idea being that
if you are getting >100 events/sec, you cannot care about subsecond
latency anymore.

If that does not describe your use-case - eg. you need to process single
event at a time - then yeah, pgq is wrong tool.

Note that we do have shared processing framework (pgq_coop), but that
also is geared towards batch-processing - it shares batches between
workers, not events.

--
marko