Re: Support Parallel Query Execution in Executor

On 4/8/06, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> passing large volumes of data between different processes is going
> to incur quite a lot of locking overhead, pipeline stalls for cache line
> transfers, etc, etc, because heavy contention for the transfer buffer is
> simply not going to be avoidable.

I don't think anyone believes there isn't going to be contention
somewhere. Anyone that's developed a parallel system acknowledges
there's a price to pay with parallelism (in complexity) which requires
good algorithms, strategy, and design.

I may certainly be misunderstanding you, but it seems like you want to
avoid parallelism altogether. Aside from PostgreSQL, the only systems
I know of which don't support parallelism in query execution are MySQL
and Firebird, but I don't see competing at the low end as a compelling
argument. Similarly, I don't think every major vendor put money into
developing something that isn't useful.

I do agree that there are many times you won't want to have parallel
query. The reasons for employing parallelism is very dependent on the
environment and application it's used in, so it's certainly going to
be up to the user to decide whether they want to use it or not.

We're currently playing with a proof-of-concept to find issues before
proposing a design. This doesn't affect the community in any way.
Rather than negativity, I'd really like to see your suggestions on
avoiding contention as much as possible; your ideas may certainly get
us past several obstacles more quickly.

--
Jonah H. Harris, Database Internals Architect
EnterpriseDB Corporation
732.331.1324