Re: Threaded Sorting

tom lane writes:
>The notion of a sort process pool seems possibly attractive. I'm
>unconvinced that it's going to be a win though because of the cost of
>shoving data across address-space boundaries.

What about splitting out parsing, optimization and plan generation from
execution and having a separate pool of exececutor processes.

As an optimizer finished with a query plan it would initiate execution
by grabbing an executor from a pool and passing it the plan.

This would allow the potential for passing partial plans to multiple
executors so a given query might be split up into three or four pieces
and then executed in parallel with the results passed through a
shared memory area owned by each executor process.

This would allow for potential optimization of sorts without threads or
incurring the overhead problems you mentioned for a separate sorter
process. The optimizer could do things like split a scan into 3 or 4
pieces before sending it off for execution and then join the pieces
back together.

It could also make complex queries much faster if there are idling CPUs
if the optimizer was updated to take advantage of this.

If we are going to split things apart, then this should be done at a
natural communication boundary right? The code has this logical split
right now anyway so the change would be more natural.

OTOH, there are much bigger fish to fry at the moment, I suspect.

- Curtis