Re: Conflict detection for update_deleted in logical replication

On Mon, Feb 10, 2025 at 2:45 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Mon, Feb 10, 2025 at 10:26 AM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> >
> > On Fri, Feb 7, 2025 at 11:17 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > >
> > > > I'm not really sure that these behaviors are the expected behavior of
> > > > users who set max_conflict_retention_duration to some subscriptions.
> > > > Or I might have set the wrong expectation or assumption on this
> > > > parameter. I'm fine with a subscription-level
> > > > max_conflict_retention_duration if it's clear this option works as
> > > > expected by users who want to use it.
> > > >
> > >
> > > It seems you are not convinced to provide this parameter at the
> > > subscription level and anyway providing it as GUC will simplify the
> > > implementation and it would probably be easier for users to configure
> > > rather than giving it at the subscription level for all subscriptions
> > > that have set retain_conflict_info set to true. I guess in the future
> > > we can provide it at the subscription level as well if there is a
> > > clear use case for it. Does that make sense to you?
> >
> > Would it make sense to introduce a GUC parameter for this value, where
> > subscribers can overwrite it for their specific subscriptions, but
> > only up to the limit set by the GUC? This would allow flexibility in
> > certain cases --subscribers could opt to wait for a shorter duration
> > than the GUC value if needed.
> >
> > Although a concrete use case isn't immediately clear, consider a
> > hypothetical scenario: Suppose a subscriber connected to Node1 must
> > wait for long period to avoid an incorrect conflict decision. In such
> > cases, it would rely on the default high value set by the GUC.
> > However, since Node1 is generally responsive and rarely has
> > long-running transactions, this long wait would only be necessary in
> > rare cases. On the other hand, a subscriber pulling from Node2 may not
> > require as stringent conflict detection. If Node2 frequently has
> > long-running transactions, waiting too long could lead to excessive
> > delays.
> >
> > The idea here is that the Node1 subscriber can wait for the full
> > max_conflict_retention_duration set by the GUC when necessary, while
> > the Node2 subscriber can choose a shorter wait time to avoid
> > unnecessary delays caused by frequent long transactions.
> >
>
> I see that users can have some cases like this where it can be helpful
> to provide the option to set max_conflict_retention_duration both at
> GUC as well as a subscription parameter. However, I suggest let's go a
> bit slower in adding more options for this particular stuff. In the
> first version of this work, let's add a GUC and then let it bake for
> some time after which we can discuss again adding a subscription
> option based on some feedback from the field.

I am fine with that.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com