RE: Conflict detection and logging in logical replication

From: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: shveta malik <shveta(dot)malik(at)gmail(dot)com>, Nisha Moond <nisha(dot)moond412(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Jan Wieck <jan(at)wi3ck(dot)info>, Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>
Subject: RE: Conflict detection and logging in logical replication
Date: 2024-07-29 06:14:30
Message-ID: OS0PR01MB57167251931CCFD703ADCF1694B72@OS0PR01MB5716.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Friday, July 26, 2024 7:34 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Thu, Jul 25, 2024 at 4:12 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
> wrote:
> > >
> A few more comments:

Thanks for the comments.

> 1.
> For duplicate key, the patch reports conflict as following:
> ERROR: conflict insert_exists detected on relation "public.t1"
> 2024-07-26 11:06:34.570 IST [27800] DETAIL: Key (c1)=(1) already exists in
> unique index "t1_pkey", which was modified by origin 1 in transaction 770 at
> 2024-07-26 09:16:47.79805+05:30.
> 2024-07-26 11:06:34.570 IST [27800] CONTEXT: processing remote data for
> replication origin "pg_16387" during message type "INSERT" for replication
> target relation "public.t1" in transaction 742, finished at 0/151A108
>
> In detail, it is better to display the origin name instead of the origin id. This will
> be similar to what we do in CONTEXT information.

Agreed. Before modifying this, I'd like to confirm the message style in the
cases where origin id may not have a corresponding origin name (e.g., if the
data was modified locally (id = 0), or if the origin that modified the data has
been dropped). I thought of two styles:

1)
- for local change: "xxx was modified by a different origin \"(local)\" in transaction 123 at 2024.."
- for dropped origin: "xxx was modified by a different origin \"(unknown)\" in transaction 123 at 2024.."

One issue for this style is that user may create an origin with the same name
here (e.g. "(local)" and "(unknown)").

2)
- for local change: "xxx was modified locally in transaction 123 at 2024.."
- for dropped origin: "xxx was modified by an unknown different origin 1234 in transaction 123 at 2024.."

This style slightly modifies the message format. I personally feel 2) maybe
better but am OK for other options as well.

What do you think ?

Here is the V7 patch set that addressed all the comments so far[1][2][3].
The subscription option part is splitted into the separate patch 0002 and
we will decide whether to drop this patch after finishing the perf tests.
Note that I didn't display the tuple value in the message as the discussion
is still ongoing[4].

[1] https://www.postgresql.org/message-id/CAJpy0uDhCnzvNHVYwse%3DKxmOB%3DqtXr6twnDP9xqdzT-oU0OWEQ%40mail.gmail.com
[2] https://www.postgresql.org/message-id/CAA4eK1%2BCJXKK34zJdEJZf2Mpn5QyMyaZiPDSNS6%3Dkvewr0-pdg%40mail.gmail.com
[3] https://www.postgresql.org/message-id/CAA4eK1Lmu%3DoVySfGjxEUykCT3FPnL1YFDHKr1ZMwFy7WUgfc6g%40mail.gmail.com
[4] https://www.postgresql.org/message-id/CAA4eK1%2BaK4MLxbfLtp%3DEV5bpvJozKhxGDRS6T9q8sz_s%2BLK3vw%40mail.gmail.com

Best Regards,
Hou zj

Attachment Content-Type Size
v7-0003-Collect-statistics-about-conflicts-in-logical-rep.patch application/octet-stream 23.7 KB
v7-0001-Detect-and-log-conflicts-in-logical-replication.patch application/octet-stream 44.5 KB
v7-0002-Add-a-detect_conflict-option-to-subscriptions.patch application/octet-stream 82.0 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Anthonin Bonnefoy 2024-07-29 07:11:52 Re: Use pgBufferUsage for block reporting in analyze
Previous Message vignesh C 2024-07-29 05:54:25 Re: Logical Replication of sequences