Re: [RFC][PATCH] wal decoding, attempt #2 - Design Documents (really attached)

On Monday, October 15, 2012 08:19:54 PM Bruce Momjian wrote:
> On Mon, Oct 15, 2012 at 09:57:19AM +0200, Andres Freund wrote:
> > On Monday, October 15, 2012 04:54:20 AM Robert Haas wrote:
> > > On Thu, Oct 11, 2012 at 3:15 AM, Heikki Linnakangas
> > >
> > > <hlinnakangas(at)vmware(dot)com> wrote:
> > > > IMHO that's a good thing, and I'd hope this new logical replication
> > > > to live outside core as well, as much as possible. But whether or
> > > > not something is in core is just a political decision, not a reason
> > > > to implement something new.
> > > >
> > > > If the only meaningful advantage is reducing the amount of WAL
> > > > written, I can't help thinking that we should just try to address
> > > > that in the existing solutions, even if it seems "easy to solve at a
> > > > first glance, but a solution not using a normal transactional table
> > > > for its log/queue has to solve a lot of problems", as the document
> > > > says. Sorry to be a naysayer, but I'm pretty scared of all the new
> > > > code and complexity these patches bring into core.
> > >
> > > I do not personally believe that a WAL decoding solution adequate to
> > > drive logical replication can live outside of core, at least not
> > > unless core exposes a whole lot more interface than we do now, and
> > > probably not even then. Even if it could, I don't see the case for
> > > making every replication solution reinvent that wheel. It's a big
> > > wheel to be reinventing, and everyone needs pretty much the same
> > > thing.
> >
> > Unsurprisingly I aggree.
> >
> > > That having been said, I have to agree that the people working on this
> > > project seem to be wearing rose-colored glasses when it comes to the
> > > difficulty of implementing a full-fledged solution in core.
> >
> > That very well might be true. Sometimes rose-colored glasses can be quite
> > productive in getting something started...
> >
> > Note at this point were only want wal decoding, background workers and
> > related things to get integrated...
>
> Well, TODO does have:
>
> Move pgfoundry's xlogdump to /contrib and have it rely more closely on
> the WAL backend code

Uhm. How does that relate to my statement?

The xlogreader code I submitted does contain a very small POC xlogdump with
almost no code duplication. It needs some work to be really useful though.

> I think Robert is right that if Slony can't use the API, it is unlikely
> any other replication system could use it.

I aggree and I don't think I have ever said something contrary. I just don't
want to be the only one working on slony integration. I am ready to do a good
part of that, but somebody with slony experience needs to help, especially on
consuming those changes.

Greetings,

Andres
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services