Re: Skipping logical replication transactions on subscriber side

On Tue, Oct 19, 2021 at 12:38 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Tue, Oct 19, 2021 at 8:23 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> >
> > On Mon, Oct 18, 2021 at 6:07 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > >
> > > On Mon, Oct 11, 2021 at 1:00 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> > > >
> > > > On Sun, Oct 10, 2021 at 11:04 PM Peter Eisentraut
> > > > <peter(dot)eisentraut(at)enterprisedb(dot)com> wrote:
> > > > >
> > > > > On 04.10.21 02:31, Masahiko Sawada wrote:
> > > > > > I guess disabling subscriptions on error/conflict and skipping the
> > > > > > particular transactions are somewhat different types of functions.
> > > > > > Disabling subscriptions on error/conflict seems likes a setting
> > > > > > parameter of subscriptions. The users might want to specify this
> > > > > > option at creation time. Whereas, skipping the particular transaction
> > > > > > is a repair function that the user might want to use on the spot in
> > > > > > case of a failure. I’m concerned a bit that combining these functions
> > > > > > to one syntax could confuse the users.
> > > > >
> > > > > Also, would the skip option be dumped and restored using pg_dump? Maybe
> > > > > there is an argument for yes, but if not, then we probably need a
> > > > > different path of handling it separate from the more permanent options.
> > > >
> > > > Good point. I don’t think the skip option should be dumped and
> > > > restored using pg_dump since the utilization of transaction ids in
> > > > another installation is different.
> > > >
> > >
> > > This is a xid of publisher which subscriber wants to skip. So, even if
> > > one restores the subscriber data in a different installation why would
> > > it matter till it points to the same publisher?
> > >
> > > Either way, can't we handle this in pg_dump?
> >
> > Because of backups (dumps), I think we cannot expect that the user
> > restore it somewhere soon. If the dump is restored several months
> > later, the publisher could be a different installation (by rebuilding
> > from scratch) or XID of the publisher could already be wrapped around.
> > It might be useful to dump the skip_xid by pg_dump in some cases, but
> > I think it should be optional if we want to do that.
> >
>
> Agreed, I think it depends on the use case, so we can keep it
> optional, or maybe in the initial version let's not dump it, and only
> if we later see the use case then we can add an optional parameter in
> pg_dump.

Agreed. I prefer not to dump it in the first version since it's
difficult to remove the option once it's introduced.

> Do you think we need any special handling if we decide not to
> dump it? I think if we decide to dump it either optionally or
> otherwise, then we do need changes in pg_dump.

Yeah, if we don't dump the skip_xid (which is the current patch
behavior), any special handling is not required for pg_dump. On the
other hand, if we do that in any way, we need changes for pg_dump.

Regards,

--
Masahiko Sawada
EDB: https://www.enterprisedb.com/