From: | Hannu Krosing <hannu(at)2ndQuadrant(dot)com> |
---|---|
To: | pgsql-hackers(at)postgresql(dot)org, Andres Freund <andres(at)2ndquadrant(dot)com>, Simon Riggs <simon(at)2ndQuadrant(dot)com>, Marko Kreen <markokr(at)gmail(dot)com> |
Subject: | [RFC] CREATE QUEUE (log-only table) for londiste/pgQ ccompatibility |
Date: | 2012-10-16 08:56:43 |
Message-ID: | 507D214B.601@2ndQuadrant.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hallo postgresql and replication hackers
This mail is an additional RFC which proposes a simple way to extend the
new logical replication feature so it can cover most usages of
skytools/pgq/londiste
While the current work for BDR/LCR (bi-directional replication/logical
replication)
using WAL is theoretically enought to cover _replication_ offered by
Londiste it
falls short in one important way - there is currently no support for
pure queueing,
that is for "streams" of data which does not need to be stored in the
source database.
Fortunately there is a simple solution - do not store it in the source
database :)
The only thing needed for adding this is to have a table type which
a) generates a INSERT record in WAL
and
b) does not actually store the data in a local file
If implemented in userspace it would be a VIEW (or table) with a
before/instead
trigger which logs the inserted data and then cancels the insert.
I'm sure this thing could be implemented, but I leave the tech
discussion to those
who are currently deep in WAL generation/reconstruction .
If we implement logged only tables / queues we would not only enable a more
performant pgQ replacement for implementing full Londiste / skytools
functionality
but would also become a very strong player to be used as persistent
basis for
message queueing solutions like ActiveMQ, StorMQ, any Advanced Message
Queuing Protocol (AMQP) and so on.
comments ?
Hannu Krosing
From | Date | Subject | |
---|---|---|---|
Next Message | Simon Riggs | 2012-10-16 09:18:44 | Re: [RFC] CREATE QUEUE (log-only table) for londiste/pgQ ccompatibility |
Previous Message | Shigeru HANADA | 2012-10-16 08:14:36 | Re: proposal - assign result of query to psql variable |