PG-MQ?

From: Chris Browne <cbbrowne(at)acm(dot)org>
To: pgsql-hackers(at)postgresql(dot)org
Subject: PG-MQ?
Date: 2007-06-19 21:45:16
Message-ID: 60ir9jd3f7.fsf@dba2.int.libertyrms.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I'm seeing some applications where it appears that there would be
value in introducing asynchronous messaging, ala "message queueing."
<http://en.wikipedia.org/wiki/Message_queue>

The "granddaddy" of message queuing systems is IBM's MQ-Series, and I
don't see particular value in replicating its functionality.

On the other side, the "big names" these days are:

a) The Java Messaging Service, which seems to implement *way* more
options than I'm even vaguely interested in having (notably, lots
that involve data stores or lack thereof that I do not care to use);

b) AMPQ
<http://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol> which
again is a big, complex, protocol-oriented thing that seems mighty big
and complex;

c) There are lesser names, like isectd <http://isectd.sf.net> and the
(infamous?) Spread Toolkit which both implement memory-based messaging
systems.

There's a simple + free system called MQS that is implemented in Perl
and uses XML-RPC; <http://www.foo.be/mqs/> that seems perhaps a little
*too* primitive.

FYI, here's its simple schema (filtered into PG):

create table mqname (
name varchar (255) not null default '',
id serial,
priority integer not null default 10,
primary key (id));

create table message (
id serial,
timestamp timestamptz NOT NULL
cid varchar(255),
queue integer not null references mqname(id),
body text not null,
priority integer not null default 10,
flag integer default 0,
primary key(id)
);

My bias would be to have something that can basically run as a thin
set of stored procedures atop PostgreSQL :-). It would be trivial to
extend that to support SOAP/XML-RPC, if desired.

It would be nice to achieve 'higher availability' by having queues
where you might replicate the contents (probably using the MQ system
itself ;-)) to other servers.

There tend to be varying semantics out there:

- Some queues may represent "subscriptions" where a whole bunch of
listeners want to get all the messages;

- Sometimes you have the semantics where:
- messages need to be delivered at least once
- messages need to be delivered no more than once
- messages need to be delivered exactly once

Is there any existing work out there on this? Or should I maybe be
looking at prototyping something?
--
(format nil "~S(at)~S" "cbbrowne" "linuxfinances.info")
http://linuxfinances.info/info/lsf.html
Q: How many Newtons does it take to change a light bulb?
A: Faux! There to eat lemons, axe gravy soup!

Responses

  • Re: PG-MQ? at 2007-06-19 22:01:20 from Steve Atkins
  • Re: PG-MQ? at 2007-06-20 07:45:57 from Jeroen T. Vermeulen
  • Re: PG-MQ? at 2007-06-20 10:37:42 from Markus Schiltknecht

Browse pgsql-hackers by date

  From Date Subject
Next Message Steve Atkins 2007-06-19 22:01:20 Re: PG-MQ?
Previous Message Heikki Linnakangas 2007-06-19 20:09:44 Re: Maximum reasonable bgwriter_delay