Re: Skipping logical replication transactions on subscriber side

On Fri, Jan 21, 2022 at 9:13 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Fri, Jan 21, 2022 at 5:25 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Fri, Jan 21, 2022 at 10:10 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> > >
> >
> > Few things that I think we can improve in 028_skip_xact.pl are as follows:
> >
> > After CREATE SUBSCRIPTION, wait for initial sync to be over and
> > two_phase state to be enabled. Please see 021_twophase. For the
> > streaming case, we might be able to ensure streaming even with lesser
> > data. Can you please try that?
> >
>
> I noticed that the newly added test by this patch takes time is on the
> upper side. See comparison with the subscription test that takes max
> time:
> [17:38:49] t/028_skip_xact.pl ................. ok 9298 ms
> [17:38:59] t/100_bugs.pl ...................... ok 11349 ms
>
> I think we can reduce time by removing some stream tests without much
> impacting on coverage, possibly related to 2PC and streaming together,
> and if you do that we probably don't need a subscription with both 2PC
> and streaming enabled.

Agreed.

In addition to that, after some tests, I realized that the two tests
of ROLLBACK PREPARED are not stable. If the walsender detects a
concurrent abort of the transaction that is being decoded, it’s
possible that it sends only beigin_prepare and prepare messages, and
consequently. If this happens before setting skip_xid, a unique key
constraint violation doesn’t occur on the subscription, and
consequently, skip_xid is not cleared. We can reduce the possibility
by setting a very high value to wal_retrieve_retry_interval but I
think it’s better to remove them. What do you think?

Regards,

--
Masahiko Sawada
EDB: https://www.enterprisedb.com/