Re: Threaded Sorting

On 4 Oct 2002 at 9:46, Hans-Jürgen Schönig wrote:

> Did anybody think about threaded sorting so far?
> Assume an SMP machine. In the case of building an index or in the case
> of sorting a lot of data there is just one backend working. Therefore
> just one CPU is used.
> What about starting a thread for every temporary file being created?
> This way CREATE INDEX could use many CPUs.
> Maybe this is worth thinking about because it will speed up huge
> databases and enterprise level computing.

I have a better plan. I have a thread architecture ready which acts as generic
thread templates. Even the function pointers in the thread can be altered on
the fly.

I suggest we use some such architecture for threading. It can be used in any
module without hardcoding things. Like say in sorting we assign exclusive
jobs/data ranges to threads then there would be minimum locking and one thread
could merge the results.. Something like that.

All it takes to change entry functions to accept one more parameter that
indicates range of values to act upon. In non-threaded version, it's not there
because the function acts on entire data set.

Further more, with this model threading support can be turned off easily. In
non-threaded model, a wrapper function can call the entry point in series with
necessary arguments. So postgresql does not have to deal with not-so-good-
enough thread implementations. Keeping tradition to conservative defaults we
can set default threads to off..

The code is in C++ but it's hardly couple of pages. I can convert it to C and
post it if required..

Let me know..

Bye
Shridhar

--
Parkinson's Fourth Law: The number of people in any working group tends to
increase regardless of the amount of work to be done.