Re: Horizontal scalability/sharding

From: Etsuro Fujita <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp>
To: Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>, Albe Laurenz <laurenz(dot)albe(at)wien(dot)gv(dot)at>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Bruce Momjian <bruce(at)momjian(dot)us>, Josh Berkus <josh(at)agliodbs(dot)com>, Mason S <masonlists(at)gmail(dot)com>, Oleg Bartunov <obartunov(at)gmail(dot)com>, Simon Riggs <simon(at)2ndquadrant(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Horizontal scalability/sharding
Date: 2015-09-02 08:07:23
Message-ID: 55E6AE3B.8090908@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 2015/09/02 16:40, Amit Langote wrote:
> On 2015-09-02 PM 04:07, Albe Laurenz wrote:
>> Amit Langote wrote:
>>> On 2015-09-02 PM 03:25, Amit Kapila wrote:
>>>> Will it handle deadlocks across different table partitions. Consider
>>>> a case as below:
>>>>
>>>> T1
>>>> 1. Updates row R1 of T1 on shard S1
>>>> 2. Updates row R2 of T2 on shard S2
>>>>
>>>> T2
>>>> 1. Updates row R2 of T2 on shard S2
>>>> 2. Updates row R1 of T1 on shard S1
>>
>>> As long as shards are processed in the same order in different
>>> transactions, ISTM, this issue should not arise? I can imagine it becoming
>>> a concern if parallel shard processing enters the scene. Am I missing
>>> something?
>>
>> That would only hold for a single query, right?
>>
>> If 1. and 2. in the above example come from different queries within one
>> transaction, you cannot guarantee that shards are processed in the same order.
>>
>> So T1 and T2 could deadlock.

> Sorry, I failed to see why that would be the case. Could you elaborate?

I think Laurenz would assume that the updates 1. and 2. in the above
transactions are performed *in a non-inherited manner*. If that's
right, T1 and T2 could deadlock, but I think we assume here to run
transactions over shards *in an inherited manner*.

Best regards,
Etsuro Fujita

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Langote 2015-09-02 08:18:37 Re: ON CONFLICT DO UPDATE using EXCLUDED.column gives an error about mismatched types
Previous Message Michael Paquier 2015-09-02 07:49:14 Re: Fwd: Core dump with nested CREATE TEMP TABLE