Re: logical replication empty transactions

On Tue, Mar 3, 2020 at 9:35 AM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> On Mon, Mar 2, 2020 at 4:56 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> >
> > One thing that is not clear to me is how will we advance restart_lsn
> > if we don't send any empty xact in a system where there are many such
> > xacts? IIRC, the restart_lsn is advanced based on confirmed_flush lsn
> > sent by subscriber. After this change, the subscriber won't be able
> > to send the confirmed_flush and for a long time, we won't be able to
> > advance restart_lsn. Is that correct, if so, why do we think that is
> > acceptable? One might argue that restart_lsn will be advanced as soon
> > as we send the first non-empty xact, but not sure if that is good
> > enough. What do you think?
>
> It seems like a valid point. One idea could be that we can track the
> last commit LSN which we streamed and if the confirmed flush location
> is already greater than that then even if we skip the sending the
> commit message we can increase the confirm flush location locally.
> Logically, it should not cause any problem because once we have got
> the confirmation for whatever we have streamed so far. So for other
> commits(which we are skipping), we can we advance it locally because
> we are sure that we don't have any streamed commit which is not yet
> confirmed by the subscriber.
>

Will this work after restart? Do you want to persist the information
of last streamed commit LSN?

> This is just my thought, but if we
> think from the code and design perspective then it might complicate
> the things and sounds hackish.
>

Another idea could be that we stream the transaction after some
threshold number (say 100 or anything we think is reasonable) of empty
xacts. This will reduce the traffic without tinkering with the core
design too much.

--
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com