Re: Logical decoding & exported base snapshot

On 2012-12-13 21:40:43 +0100, Andres Freund wrote:
> On 2012-12-13 11:02:06 -0500, Steve Singer wrote:
> > On 12-12-12 06:20 AM, Andres Freund wrote:
> > >>Possible solutions:
> > >>1) INIT_LOGICAL_REPLICATION waits for an answer from the client that
> > >>confirms that logical replication initialization is finished. Before
> > >>that the walsender connection cannot be used for anything else.
> > >>
> > >>2) we remove the snapshot as soon as any other commend is received, this
> > >>way the replication connection stays usable, e.g. to issue a
> > >>START_LOGICAL_REPLICATION in parallel to the initial data dump. In that
> > >>case the snapshot would have to be imported *before* the next command
> > >>was received as SET TRANSACTION SNAPSHOT requires the source transaction
> > >>to be still open.
> > >>Option 2 sounds more flexible. Is it more difficult to implement?
> > >No, I don't think so. It's a bit more intrusive in that it requires
> > >knowledge about logical replication in more parts of walsender, but it
> > >should be ok.
> > >
> > >Note btw, that my description of 1) was easy to misunderstand. The
> > >"that" in "Before that the walsender connection cannot be used for
> > >anything else." is the answer from the client, not the usage of the
> > >exported snapshot. Once the snapshot has been iimported into other
> > >session(s) the source doesn't need to be alive anymore.
> > >Does that explanation change anything?
> >
> > I think I understood you were saying the walsender connection can't be used
> > for anything else (ie streaming WAL) until the exported snapshot has been
> > imported. I think your clarification is still consistent with this?
>
> Yes, thats correct.
>
> > WIth option 2 I can still get the option 1 behaviour by not sending the next
> > command to the walsender until I am done importing the snapshot. However if
> > I want to start processing WAL before the snapshot has been imported I can
> > do that with option 2.
> >
> > I am not sure I would need to do that, I'm just saying having the option is
> > more flexible.
>
> True.
>
> Still not sure whats better, but since youre slightly leaning towards 2)
> I am going to implement that.

Pushed and lightly tested.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services