From: | Robert Haas <robertmhaas(at)gmail(dot)com> |
---|---|
To: | Peter Geoghegan <pg(at)heroku(dot)com> |
Cc: | Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, Corey Huinker <corey(dot)huinker(at)gmail(dot)com> |
Subject: | Re: Parallel tuplesort (for parallel B-Tree index creation) |
Date: | 2016-09-26 17:58:30 |
Message-ID: | CA+TgmoYQYtMYqiudsqmi78qJ=r9M0A5c5Z9uSUGk9C5Y+WYMDA@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Sat, Sep 24, 2016 at 9:07 AM, Peter Geoghegan <pg(at)heroku(dot)com> wrote:
> On Thu, Sep 22, 2016 at 8:57 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> On Thu, Sep 22, 2016 at 3:51 PM, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:
>>> It'd be good if you could overlap the final merges in the workers with the
>>> merge in the leader. ISTM it would be quite straightforward to replace the
>>> final tape of each worker with a shared memory queue, so that the leader
>>> could start merging and returning tuples as soon as it gets the first tuple
>>> from each worker. Instead of having to wait for all the workers to complete
>>> first.
>>
>> If you do that, make sure to have the leader read multiple tuples at a
>> time from each worker whenever possible. It makes a huge difference
>> to performance. See bc7fcab5e36b9597857fa7e3fa6d9ba54aaea167.
>
> That requires some kind of mutual exclusion mechanism, like an LWLock.
No, it doesn't. Shared memory queues are single-reader, single-writer.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
From | Date | Subject | |
---|---|---|---|
Next Message | Tomas Vondra | 2016-09-26 18:48:55 | Re: Speed up Clog Access by increasing CLOG buffers |
Previous Message | Robert Haas | 2016-09-26 17:51:21 | Re: [PATCH] pgpassfile connection option |