Re: Support Parallel Query Execution in Executor

Hi,

On Sat, 2006-04-08 at 13:16 -0400, Jonah H. Harris wrote:
> On 4/8/06, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> > ... but I'm failing to follow where it says that parallel processing
> > will fix that. All I can foresee in that direction is extra data
> > transfer costs, bought at the price of portability and locking headaches.
>
> I don't think it's any less portable than the system is now;

ACK. As long as processes, signals and shared memory are used this could
be as portable as PostgreSQL is now.

> It's just
> enabling multiple slave processes to participate in scans and
> processing (parallel query, parallel index builds, parallel sorts,
> ...) Likewise, the additional I/O cost isn't that much of an issue
> because systems which really take advantage of this type of parallel
> processing have large bandwidth I/O arrays anyway.

Ehm.. which additional I/O cost? Or do you count inter-process
communication to I/O?

I'd like to help teaching PostgreSQL the art of parallel query
execution. I have rawly implemented an internal message passing system
on shared memory basis. This allows all pg-processes to send messages to
each other identified by PID. Uppon receiving of a message a process
gets a SIGUSR2 (AFAIC remember now).

On the positive side are:
- portable as I'm using only shmem and signals
- fast (not much copying around of the data in memory)

One downside I see with my approach is:
- the shared memory buffer is limited in size, so it can simply be
'full' and no more messages can be sent before another process reads
its messages and frees the memory for other messages.

In case you're interested I'll compile a patch for review.

Regards

Markus