Re: Conflict Detection and Resolution

On Wed, Jul 3, 2024 at 4:04 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> On Wed, Jul 3, 2024 at 3:35 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Wed, Jul 3, 2024 at 2:16 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> > >
> > > On Wed, Jul 3, 2024 at 12:30 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > > >
> > > > On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
> > >
> > > > But waiting after applying the operations and before applying the
> > > > commit would mean that we need to wait with the locks held. That could
> > > > be a recipe for deadlocks in the system. I see your point related to
> > > > performance but as we are not expecting clock skew in normal cases, we
> > > > shouldn't be too much bothered on the performance due to this. If
> > > > there is clock skew, we expect users to fix it, this is just a
> > > > worst-case aid for users.
> > >
> > > But if we make it wait at the very first operation that means we will
> > > not suck more decoded data from the network and wouldn't that make the
> > > sender wait for the network buffer to get sucked in by the receiver?
> > >
> >
> > That would be true even if we wait just before applying the commit
> > record considering the transaction is small and the wait time is
> > large.
>
> What I am saying is that if we are not applying the whole transaction,
> it means we are not receiving it either unless we plan to spill it to
> a file. If we don't spill it to a file, the network buffer will fill
> up very quickly. This issue wouldn't occur if we waited right before
> the commit because, by that time, we would have already received all
> the data from the network.
>

We would have received the transaction data but there could be other
transactions that need to wait because the apply worker is waiting
before the commit. So, the situation will be the same. We can even
decide to spill the data to files if the decision is that we need to
wait to avoid network buffer-fill situations. But note that the wait
in apply worker has consequences that the subscriber won't be able to
confirm the flush position and publisher won't be able to vacuum the
dead rows and we won't be remove WAL as well. Last time when we
discussed the delay_apply feature, we decided not to proceed because
of such issues. This is the reason I proposed a cap on wait time.

--
With Regards,
Amit Kapila.