Re: Conflict detection for update_deleted in logical replication

On Mon, Feb 10, 2025 at 10:26 AM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> On Fri, Feb 7, 2025 at 11:17 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> >
> > > I'm not really sure that these behaviors are the expected behavior of
> > > users who set max_conflict_retention_duration to some subscriptions.
> > > Or I might have set the wrong expectation or assumption on this
> > > parameter. I'm fine with a subscription-level
> > > max_conflict_retention_duration if it's clear this option works as
> > > expected by users who want to use it.
> > >
> >
> > It seems you are not convinced to provide this parameter at the
> > subscription level and anyway providing it as GUC will simplify the
> > implementation and it would probably be easier for users to configure
> > rather than giving it at the subscription level for all subscriptions
> > that have set retain_conflict_info set to true. I guess in the future
> > we can provide it at the subscription level as well if there is a
> > clear use case for it. Does that make sense to you?
>
> Would it make sense to introduce a GUC parameter for this value, where
> subscribers can overwrite it for their specific subscriptions, but
> only up to the limit set by the GUC? This would allow flexibility in
> certain cases --subscribers could opt to wait for a shorter duration
> than the GUC value if needed.
>
> Although a concrete use case isn't immediately clear, consider a
> hypothetical scenario: Suppose a subscriber connected to Node1 must
> wait for long period to avoid an incorrect conflict decision. In such
> cases, it would rely on the default high value set by the GUC.
> However, since Node1 is generally responsive and rarely has
> long-running transactions, this long wait would only be necessary in
> rare cases. On the other hand, a subscriber pulling from Node2 may not
> require as stringent conflict detection. If Node2 frequently has
> long-running transactions, waiting too long could lead to excessive
> delays.
>
> The idea here is that the Node1 subscriber can wait for the full
> max_conflict_retention_duration set by the GUC when necessary, while
> the Node2 subscriber can choose a shorter wait time to avoid
> unnecessary delays caused by frequent long transactions.
>

I see that users can have some cases like this where it can be helpful
to provide the option to set max_conflict_retention_duration both at
GUC as well as a subscription parameter. However, I suggest let's go a
bit slower in adding more options for this particular stuff. In the
first version of this work, let's add a GUC and then let it bake for
some time after which we can discuss again adding a subscription
option based on some feedback from the field.

--
With Regards,
Amit Kapila.