Re: Sorting Improvements for 8.4

On Tue, 2007-12-18 at 09:31 +0000, Simon Riggs wrote:
> On Mon, 2007-12-17 at 16:34 -0800, Ron Mayer wrote:
>
> > PS: Yeah, I know multi-threading is a hot-button on these
> > lists; but sorting seems a relatively isolated of the code
> > and I'd wonder if it'd be isolate-able enough that multiple
> > CPUs could be used there.
>
> I'm not sure multi-threading is the issue you think. Threads is, but
> only for architectural reasons. Using multiple processes to complete a
> task seems very sensible to me.

My first thought would be that we would need a new executor node (e.g.
"ParallelSort") that would only be chosen when the cost of the sort is
large enough to outweigh other factors (such as process creation time,
dividing available work_mem, and any necessary IPC).

It seems to me the simplest way to do it would be to allow each sub
process to allocate work_mem/P where P is the degree of parallelization.
However, that somewhat works against our schemes for dynamic run
handling and forecasting, and may lead to more random reading from disk.
Any other scheme I can think of would involve more IPC, which might make
the idea just too complex.

Regards,
Jeff Davis