Re: Conflict detection for update_deleted in logical replication

On Thu, Dec 19, 2024 at 4:34 PM Zhijie Hou (Fujitsu)
<houzj(dot)fnst(at)fujitsu(dot)com> wrote:
>
> On Sunday, December 15, 2024 9:39 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
>
> >
> > 5. The apply worker needs to at least twice get the publisher status message to
> > advance oldest_nonremovable_xid once. It then uses the remote_lsn of the last
> > such message to ensure that it has been applied locally. Such a remote_lsn
> > could be a much later value than required leading to delay in advancing
> > oldest_nonremovable_xid. How about if while first time processing the
> > publisher_status message on walsender, we get the
> > latest_transaction_in_commit by having a function
> > GetLatestTransactionIdInCommit() instead of
> > GetOldestTransactionIdInCommit() and then simply wait till that proc has
> > written commit WAL (aka wait till it clears DELAY_CHKPT_IN_COMMIT)?
> > Then get the latest LSN wrote and send that to apply worker waiting for the
> > publisher_status message. If this is feasible then we should be able to
> > advance oldest_nonremovable_xid with just one publisher_status message.
> > Won't that be an improvement over current? If so, we can even further try to
> > improve it by just using commit_LSN of the transaction returned by
> > GetLatestTransactionIdInCommit(). One idea is that we can try to use
> > MyProc->waitLSN which we are using in synchronous replication for our
> > purpose. See SyncRepWaitForLSN.
>
> I will do more performance tests on this and address if it improves
> the performance.
>

Did you check this idea? Again, thinking about this, I see a downside
to the new proposal. In the new proposal, the walsender needs to
somehow wait for the transactions in the commit which essentially
means that it may lead delay in decoding and sending the decoded WAL.
But it is still worth checking the impact of such a change, if nothing
else, we can add a short comment in the code to suggest such an
improvement is not worthwhile.

--
With Regards,
Amit Kapila.