Re: [PATCH 13/16] Introduction of pair of logical walreceiver/sender

On Friday, June 29, 2012 05:16:11 PM Heikki Linnakangas wrote:
> On 13.06.2012 14:28, Andres Freund wrote:
> > A logical WALReceiver is started directly by Postmaster when we enter
> > PM_RUN state and the new parameter multimaster_conninfo is set. For now
> > only one of those is started, but the code doesn't rely on that. In
> > future multiple ones should be allowed.
>
> Could the receiver-side of this be handled as an extra daemon:
> http://archives.postgresql.org/message-id/CADyhKSW2uyrO3zx-tohzRhN5-vaBEfKN
> HyvLG1yp7=cx_YH9UA(at)mail(dot)gmail(dot)com
Well, I think it depends on what the protocol turns out to be. In the
prototype we used the infrastructure from walreceiver which reduced the
required code considerably.

> In general, I feel that the receiver-side could live outside core.
I think it should be possible to write receivers outside core, but one
sensible implementation should be in-core.

> The sender-side needs to be at least somewhat integrated into the walsender
> stuff, and there are changes to the WAL records etc. that are hard to do
> outside, but AFAICS the stuff to receive changes is pretty high-level
> stuff.
> None of that needs to be in implemented inside a PostgreSQL server.
If you want robust and low-overhead crash recovery you need (at least I think
so) tighter integration into postgres. To be sure that you pick of where you
stopped after a crash you need to have a state synchronized to the commits
into the receiving side. So you either always write to another table and
analyze that afterwards - which imo sucks - or you integrate it with the
commit record. Which needs integration into pg.

Greetings,

Andres
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services