Re: Skipping logical replication transactions on subscriber side

On Mon, Jan 24, 2022 at 8:24 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> On Fri, Jan 21, 2022 at 9:13 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Fri, Jan 21, 2022 at 5:25 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > >
> > > On Fri, Jan 21, 2022 at 10:10 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> > > >
> > >
> > > Few things that I think we can improve in 028_skip_xact.pl are as follows:
> > >
> > > After CREATE SUBSCRIPTION, wait for initial sync to be over and
> > > two_phase state to be enabled. Please see 021_twophase. For the
> > > streaming case, we might be able to ensure streaming even with lesser
> > > data. Can you please try that?
> > >
> >
> > I noticed that the newly added test by this patch takes time is on the
> > upper side. See comparison with the subscription test that takes max
> > time:
> > [17:38:49] t/028_skip_xact.pl ................. ok 9298 ms
> > [17:38:59] t/100_bugs.pl ...................... ok 11349 ms
> >
> > I think we can reduce time by removing some stream tests without much
> > impacting on coverage, possibly related to 2PC and streaming together,
> > and if you do that we probably don't need a subscription with both 2PC
> > and streaming enabled.
>
> Agreed.
>
> In addition to that, after some tests, I realized that the two tests
> of ROLLBACK PREPARED are not stable. If the walsender detects a
> concurrent abort of the transaction that is being decoded, it’s
> possible that it sends only beigin_prepare and prepare messages, and
> consequently. If this happens before setting skip_xid, a unique key
> constraint violation doesn’t occur on the subscription, and
> consequently, skip_xid is not cleared. We can reduce the possibility
> by setting a very high value to wal_retrieve_retry_interval but I
> think it’s better to remove them.
>

+1.

--
With Regards,
Amit Kapila.