Re: Conflict detection and logging in logical replication

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Michail Nikolaev <michail(dot)nikolaev(at)gmail(dot)com>
Cc: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>, "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, shveta malik <shveta(dot)malik(at)gmail(dot)com>, Nisha Moond <nisha(dot)moond412(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Jan Wieck <jan(at)wi3ck(dot)info>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>
Subject: Re: Conflict detection and logging in logical replication
Date: 2024-08-16 04:23:25
Message-ID: CAA4eK1K1fZyxLTDQUYEzWmH4gbsZPyM1QAFmft96g4Nds=pXSg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Aug 14, 2024 at 7:45 PM Michail Nikolaev
<michail(dot)nikolaev(at)gmail(dot)com> wrote:
>
> > This is as expected, and we have documented this in the code comments. We don't
> > need to report a conflict if the conflicting tuple has been removed or updated
> > due to concurrent transaction. The same is true if the transaction that
> > inserted the conflicting tuple is rolled back before CheckAndReportConflict().
> > We don't consider such cases as a conflict.
>
> That seems a little bit strange to me.
>
> From the perspective of a user, I expect that if a change from publisher is not applied - I need to know about it from the logs.
>

In the above conditions where a concurrent tuple insertion is removed
or rolled back before CheckAndReportConflict, the tuple inserted by
apply will remain. There is no need to report anything in such cases
as apply was successful.

> But in that case, I will not see any information about conflict in the logs in SOME cases. But in OTHER cases I will see it.
> However, in both cases the change from publisher was not applied.
> And these cases are just random and depend on the timing of race conditions. It is not something I am expecting from the database.
>
> Maybe it is better to report about the fact that event from publisher was not applied because of conflict and then try to
> provide additional information about the conflict itself?
>
> Or possibly in case we were unable to apply the event and not able to find the conflict, we should retry the event processing?
>

Per my understanding, we will apply or the conflict will be logged and
retried where required (unique key violation).

--
With Regards,
Amit Kapila.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Yugo Nagata 2024-08-16 04:31:11 Re: define PG_REPLSLOT_DIR
Previous Message vignesh C 2024-08-16 04:15:33 Re: CREATE SUBSCRIPTION - add missing test case