Re: Conflict detection for update_deleted in logical replication

On Tue, Dec 17, 2024 at 8:54 AM Zhijie Hou (Fujitsu)
<houzj(dot)fnst(at)fujitsu(dot)com> wrote:
>
> On Monday, December 16, 2024 7:21 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> > So IIUC in step 2) we send the message and get the list of all the
> > transactions which are in the commit phase? What do you exactly mean by a
> > transaction which is in the commit phase?
>
> I was referring to transactions calling RecordTransactionCommit() and have
> entered the commit critical section. In the patch, we checked if the proc has
> marked the new flag DELAY_CHKPT_IN_COMMIT in 'MyProc->delayChkptFlags'.
>
> > Can I assume transactions which are currently running on the publisher?
>
> I think it's a subset of the running transactions. We only get the transactions
> in commit phase with the intention to avoid delays caused by waiting for
> long-running transactions to complete, which can result in the long retention
> of dead tuples.

Ok

> We decided to wait for running(committing) transactions due to the WAL/LSN
> inversion issue[1]. The original idea is to directly return the latest WAL
> write position without checking running transactions. But since there is a gap
> between when we acquire the commit_timestamp and the commit LSN, it's possible
> the transactions might have been assigned an earlier commit timestamp but have
> not yet written the commit WAL record.

Yes, that makes sense.

> > And in step 3) we wait for all the transactions to get committed which we saw
> > running (or in the commit phase) and we anyway don't worry about the newly
> > started transactions as they would not be problematic for us. And in step 4)
> > we would wait for all the flush location to reach "last received WAL
> > position", here my question is what exactly will be the "last received WAL
> > position" I assume it would be the position somewhere after the position of
> > the commit WAL of all the transaction we were interested on the publisher?
>
> Yes, your understanding is correct. It's a position after the position of all
> the interesting transactions. In the patch, we get the latest WAL write
> position(GetXLogWriteRecPtr()) in walsender after all interesting transactions
> have finished and reply it to apply worker.

Got it, thanks.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com