Re: HACKERS[PROPOSAL] split ProcArrayLock into multiple parts

From: Sokolov Yura <y(dot)sokolov(at)postgrespro(dot)ru>
To: Jim Van Fleet <vanfleet(at)us(dot)ibm(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: HACKERS[PROPOSAL] split ProcArrayLock into multiple parts
Date: 2017-06-05 20:11:13
Message-ID: 317cb5e8-7ca4-4010-b2dd-8413f9b9858b@email.android.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

<div dir='auto'><div dir="auto">Hi, Jim.</div><div dir="auto"><br></div><div>How do you ensure of transaction order?<div dir="auto"><br></div><div dir="auto">Example:</div><div dir="auto">- you lock shard A and gather info. You find transaction T1 in-progress.</div><div dir="auto">- Then you unlock shard A.</div><div dir="auto">- T1 completes. T2, that depends on T1, also completes. But T2 was on shard B.</div><div dir="auto">- you lock shard B, and gather info from.</div><div dir="auto">- You didn't saw T2 as in progress, so you will lookup into clog then and will find it as commited.</div><div dir="auto"><br></div><div dir="auto">Now you see T2 as commited, but T1 as in-progress - clear violation of transaction order.</div><div dir="auto"><br></div><div dir="auto">Probably you've already solved this issue. If so it would be great to learn the solution.</div><br><div class="gmail_extra"><br><div class="gmail_quote">5 июня 2017 г. 10:30 PM пользователь Jim Van Fleet &lt;vanfleet(at)us(dot)ibm(dot)com&gt; написал:<br type="attribution"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><font size="2" face="sans-serif">Hi,</font><br><br><font size="2" face="sans-serif">I have been experimenting with splitting
&nbsp;the ProcArrayLock into parts. &nbsp;That is, to Acquire the ProcArrayLock
in shared mode, it is only necessary to acquire one of the parts in shared
mode; to acquire the lock in exclusive mode, all of the parts must be acquired
in exclusive mode. For those interested, I have attached a design description
of the change.</font><br><br><font size="2" face="sans-serif">This approach has been quite successful
on large systems with the hammerdb benchmark.</font><font size="3" face="Times New Roman">With a prototype based on 10 master source and running on power8 (model
8335-GCA with 2sockets, 20 core)</font><br><font size="3" face="Times New Roman">&nbsp;hammerdb &nbsp;improved by
16%; On intel (Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz, 2 socket, 44
core) with 9.6 base and prototype hammerdb improved by 4%. (attached is
a set of spreadsheets for power8.</font><br><br><font size="3" face="Times New Roman">The down side is that on smaller
configurations (single socket) where there is less "lock thrashing"
in the storage subsystem and there are multiple Lwlocks to take for an
exclusive acquire, there is a decided downturn in performance. On &nbsp;hammerdb,
the prototype was 6% worse than the base on a single socket power configuration.</font><br><br><font size="3" face="Times New Roman">If there is interest in this approach,
I will submit a patch.</font><br><br><font size="3" face="Times New Roman">Jim Van Fleet</font><br><br><br></blockquote></div><br></div></div></div>

Attachment Content-Type Size
unknown_filename text/html 2.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Sokolov Yura 2017-06-05 20:26:46 Re: HACKERS[PROPOSAL] split ProcArrayLock into multiple parts
Previous Message Tom Lane 2017-06-05 19:34:33 Re: Make ANALYZE more selective about what is a "most common value"?