Re: GDQ iimplementation (was: Re: Clustering features for upcoming developer meeting -- please claim yours!)

On Tue, 2010-05-11 at 08:33 -0400, Jan Wieck wrote:

> What are the advantages of anything proposed over the current
> implementations used by Londiste and Slony?

It would be good to have a core technology that provided a generic
transport to other remote databases.

We already have WALSender and WALReceiver, which uses the COPY protocol
as a transport mechanism. It would be easy to extend that so we could
send other forms of data.

We can do that in two ways:

* Alter triggers so that Slony/Londiste write directly to WAL rather
than log tables, using a new WAL record for custom data blobs.

* Alter WALSender so it can read Slony/Londiste log tables for
consumption by an architecture similar to WALreceiver/Startup. Probably
easier.

We can also alter the WAL format itself to include the information in
WAL that is required to do what Slony/Londiste already do, so we don't
need to specifically write anything at all, just read WAL at other end.
Even more efficient.

The advantages of these options would be

* integration of core technologies
* greater efficiency for trigger based logging via WAL

In other RDBMS "replication" has long meant "data transport, either for
HA or application use". We should be looking beyond the pure HA aspects,
as pgq does.

I would certainly like to see a system that wrote data on master and
then constructed the SQL on receiver-side (i.e. on slave), so the
integration was less tight. That would allow data to be sent and for it
to be consumed to a variety of purposes, not just HA replay.

--
Simon Riggs www.2ndQuadrant.com