Re: Conflict Detection and Resolution

From: shveta malik <shveta(dot)malik(at)gmail(dot)com>
To: Nisha Moond <nisha(dot)moond412(at)gmail(dot)com>
Cc: Ajin Cherian <itsajin(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Jan Wieck <jan(at)wi3ck(dot)info>, Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>, shveta malik <shveta(dot)malik(at)gmail(dot)com>
Subject: Re: Conflict Detection and Resolution
Date: 2024-07-17 06:01:12
Message-ID: CAJpy0uDAMtrqXWFtxrXSwTNO8awFAoENtbo47Czj-67k3OVMyQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Jul 9, 2024 at 3:09 PM shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:
>
> On Fri, Jul 5, 2024 at 5:12 PM Nisha Moond <nisha(dot)moond412(at)gmail(dot)com> wrote:
> >
> > Thank you Ajin for reporting the issue, This is now fixed with the
> > v4-0003 patch.
>
> Please find v5 patch-set. Changes are:
>
> 1) patch003:
> Added test cases for all resolvers (034_conflict_resolver.pl).
>
> 2) Patch004:
> a) Emit error while resolving conflict if conflict resolver is default
> 'last_update_wins' but track_commit_timetsamp is not enabled.
> b) Emit Warning during create and alter subscription when
> 'detect_conflict' is ON but 'track_commit_timetsamp' is not enabled.
> c) Restrict start of pa worker if either max-clock-skew is configured
> or conflict detection and resolution is enabled for a subscription.
> d) Implement clock-skew delay/error when changes are applied from a
> file (apply_spooled_messages).
> e) Implement clock-skew delay while applying prepared changes (two
> phase txns). The prepare-timestamp to be considered as base for
> clock-skew handling as well as for last_update_win resolver.
> <TODO: This needs to be analyzed and tested further to see if there is
> any side effect of taking prepare-timestamp as base.>
>
> Thanks Ajin fo working on 1.
> Thanks Nisha for working on 2a,2b.
>

Please find v6 patch-set. Changes are:

1) patch003:
1a) Improved log and restructured code around it.
1b) added test case for delete_differ.

2) patch004:
2a) Local and remote timestamps were logged incorrectly due to a bug,
corrected that.
2b) Added tests for last_update_wins.
2c) Added a cap on wait time; introduced a new GUC for this. Apply
worker will now error out without waiting if the computed wait exceeds
this GUC's value.
2d) Restricted enabling two_phase and detect_conflict together for a
subscription. This is because the time based resolvers may result in
data divergence for two phase commit transactions if prepare-timestamp
is used for comparison.

Thanks Nisha for working on 1a to 2b.

thanks
Shveta

Attachment Content-Type Size
v6-0003-Conflict-resolvers-for-insert-update-and-delete.patch application/octet-stream 55.8 KB
v6-0004-Manage-Clock-skew-and-implement-last_update_wins.patch application/octet-stream 49.3 KB
v6-0001-Detect-and-log-conflicts-in-logical-replication.patch application/octet-stream 95.5 KB
v6-0002-DDL-command-to-configure-Global-Conflict-Resolver.patch application/octet-stream 26.5 KB
v6-0005-Configure-table-level-conflict-resolvers.patch application/octet-stream 48.9 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2024-07-17 06:24:45 Re: long-standing data loss bug in initial sync of logical replication
Previous Message Ashutosh Bapat 2024-07-17 05:34:48 Re: SQL Property Graph Queries (SQL/PGQ)