Re: [RFC] CREATE QUEUE (log-only table) for londiste/pgQ ccompatibility

From: Christopher Browne <cbbrowne(at)gmail(dot)com>
To: Greg Stark <stark(at)mit(dot)edu>
Cc: Josh Berkus <josh(at)agliodbs(dot)com>, PostgreSQL Mailing Lists <pgsql-hackers(at)postgresql(dot)org>, Hannu Krosing <hannu(at)2ndquadrant(dot)com>
Subject: Re: [RFC] CREATE QUEUE (log-only table) for londiste/pgQ ccompatibility
Date: 2012-10-17 18:48:45
Message-ID: CAFNqd5X8XXHC+zsL6wn7qAwAtCa1F0jCpBdqs2mArji7LHx0GA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Well, replication is arguably a relevant case.

For Slony, the origin/master node never cares about logged changes - that
data is only processed on replicas. Now, that's certainly a little
weaselly - the log data (sl_log_*) has got to get read to get to the
replica.

This suggests, nonetheless, a curiously different table structure than is
usual, and I could see this offering interesting possibilities.

The log tables are only useful to read in transaction order, which is
pretty well the order data gets written to WAL, so perhaps we could have
savings by only writing data to WAL...

It occurs to me that this notion might exist as a special sort of table,
interesting for pgq as well as Slony, which consists of:

- table data is stored only in WAL
- an index supports quick access to this data, residing in WAL
- TOASTing perhaps unneeded?
- index might want to be on additional attributes
- the triggers-on-log-tables thing Slony 2.2 does means we want these
tables to support triggers
- if data is only held in WAL, we need to hold the WAL until (mumble,
later, when known to be replicated)
- might want to mix local updates with updates imported from remote nodes

I think it's a misnomer to think this is about having the data not locally
accessible. Rather, it has a pretty curious access and storage pattern.

And a slick pgq queue would likely make a good Slony log, too.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message John R Pierce 2012-10-17 18:50:31 Re: Deprecating RULES
Previous Message Simon Riggs 2012-10-17 18:41:51 Re: Deprecating RULES