Re: Horizontal scalability/sharding

From: Etsuro Fujita <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Pavan Deolasee <pavan(dot)deolasee(at)gmail(dot)com>
Cc: Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>, Albe Laurenz <laurenz(dot)albe(at)wien(dot)gv(dot)at>, Robert Haas <robertmhaas(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Bruce Momjian <bruce(at)momjian(dot)us>, Josh Berkus <josh(at)agliodbs(dot)com>, Mason S <masonlists(at)gmail(dot)com>, Oleg Bartunov <obartunov(at)gmail(dot)com>, Simon Riggs <simon(at)2ndquadrant(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Horizontal scalability/sharding
Date: 2015-09-02 12:55:04
Message-ID: 55E6F1A8.7010202@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 2015/09/02 20:42, Amit Kapila wrote:
> On Wed, Sep 2, 2015 at 4:19 PM, Pavan Deolasee <pavan(dot)deolasee(at)gmail(dot)com
> <mailto:pavan(dot)deolasee(at)gmail(dot)com>> wrote:
> > On Wed, Sep 2, 2015 at 3:55 PM, Amit Langote
> <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp <mailto:Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>>
> wrote:
> >> On 2015-09-02 PM 06:41, Amit Langote wrote:
> >> Which, I now realize, is not the worry Amit Kapila's expresses.

> >> The deadlock was *indeed detected* in this case, with all the locks
> in the
> >> same PG instance. In a sharded environment with multiple PG instances,
> >> that becomes tricky. DLM (distributed lock manager/deadlock detector)
> >> seems indeed necessary as Amit K. suspects.

Ah, you are right.

> > Right. XC/XL did not address this issue and they rely on statement
> timeouts to break distributed deadlocks.

> I think that will be difficult for application to decide and then it
> needs to decide the same for all statements which is tricky because
> different statements could take different time. I think it is better to
> have solution for this problem and deadlock should be detected.

+1

Best regards,
Etsuro Fujita

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Etsuro Fujita 2015-09-02 12:55:42 Hooking at standard_join_search (Was: Re: Foreign join pushdown vs EvalPlanQual)
Previous Message Magnus Hagander 2015-09-02 12:48:12 Re: commitfest does not see my real latest patch