Re: Skipping logical replication transactions on subscriber side

On Tue, Oct 19, 2021 at 8:23 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> On Mon, Oct 18, 2021 at 6:07 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Mon, Oct 11, 2021 at 1:00 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> > >
> > > On Sun, Oct 10, 2021 at 11:04 PM Peter Eisentraut
> > > <peter(dot)eisentraut(at)enterprisedb(dot)com> wrote:
> > > >
> > > > On 04.10.21 02:31, Masahiko Sawada wrote:
> > > > > I guess disabling subscriptions on error/conflict and skipping the
> > > > > particular transactions are somewhat different types of functions.
> > > > > Disabling subscriptions on error/conflict seems likes a setting
> > > > > parameter of subscriptions. The users might want to specify this
> > > > > option at creation time. Whereas, skipping the particular transaction
> > > > > is a repair function that the user might want to use on the spot in
> > > > > case of a failure. I’m concerned a bit that combining these functions
> > > > > to one syntax could confuse the users.
> > > >
> > > > Also, would the skip option be dumped and restored using pg_dump? Maybe
> > > > there is an argument for yes, but if not, then we probably need a
> > > > different path of handling it separate from the more permanent options.
> > >
> > > Good point. I don’t think the skip option should be dumped and
> > > restored using pg_dump since the utilization of transaction ids in
> > > another installation is different.
> > >
> >
> > This is a xid of publisher which subscriber wants to skip. So, even if
> > one restores the subscriber data in a different installation why would
> > it matter till it points to the same publisher?
> >
> > Either way, can't we handle this in pg_dump?
>
> Because of backups (dumps), I think we cannot expect that the user
> restore it somewhere soon. If the dump is restored several months
> later, the publisher could be a different installation (by rebuilding
> from scratch) or XID of the publisher could already be wrapped around.
> It might be useful to dump the skip_xid by pg_dump in some cases, but
> I think it should be optional if we want to do that.
>

Agreed, I think it depends on the use case, so we can keep it
optional, or maybe in the initial version let's not dump it, and only
if we later see the use case then we can add an optional parameter in
pg_dump. Do you think we need any special handling if we decide not to
dump it? I think if we decide to dump it either optionally or
otherwise, then we do need changes in pg_dump.

--
With Regards,
Amit Kapila.