Re: Support Parallel Query Execution in Executor

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "Jonah H(dot) Harris" <jonah(dot)harris(at)gmail(dot)com>
Cc: "Markus Schiltknecht" <markus(at)bluegap(dot)ch>, josh(at)agliodbs(dot)com, pgsql-hackers(at)postgresql(dot)org, "Mike Rylander" <mrylander(at)gmail(dot)com>, "Qingqing Zhou" <zhouqq(at)cs(dot)toronto(dot)edu>
Subject: Re: Support Parallel Query Execution in Executor
Date: 2006-04-08 22:02:46
Message-ID: 28812.1144533766@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches

"Jonah H. Harris" <jonah(dot)harris(at)gmail(dot)com> writes:
> On 4/8/06, Markus Schiltknecht <markus(at)bluegap(dot)ch> wrote:
>> Ehm.. which additional I/O cost? Or do you count inter-process
>> communication to I/O?

> Inter-process will add a minimal amount, but if it's done correctly,
> you'll still end up ahead.

This is exactly the bit of optimism I was questioning. We've already
been sweating blood trying to reduce multiprocessor contention on data
structures in which collisions ought to be avoidable (ie, buffer arrays
where you hope not everyone is hitting the same buffer at once). I
think passing large volumes of data between different processes is going
to incur quite a lot of locking overhead, pipeline stalls for cache line
transfers, etc, etc, because heavy contention for the transfer buffer is
simply not going to be avoidable.

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David Fetter 2006-04-08 22:07:37 Re: How to implement oracle like rownum(function or seudocolumn)
Previous Message Tom Lane 2006-04-08 21:38:00 Re: FOUND not set by EXECUTE?

Browse pgsql-patches by date

  From Date Subject
Next Message Jonah H. Harris 2006-04-08 22:21:38 Re: Support Parallel Query Execution in Executor
Previous Message Markus Schiltknecht 2006-04-08 18:54:35 Re: Support Parallel Query Execution in Executor