Re: Support Parallel Query Execution in Executor

Greg,

On 4/8/06 5:43 PM, "Gregory Maxwell" <gmaxwell(at)gmail(dot)com> wrote:

> For example, one case made in this thread involved bursty performance
> with seqscans presumably because the I/O was stalling while processing
> was being performed. In general this can be avoided without parallel
> execution through the use of non-blocking I/O and making an effort to
> keep the request pipeline full.

I agree - there is a real continuing need for overlapped scan and execution,
though I'm not sure quite how to get it efficiently using inter-process
communication. The AIO interface is nicely proven at this point, but to
Tom's point, it's non-portable.

What we see now is that the executor is CPU bound at about 300MB/s scan rate
using modern Opteron CPUs. Using some kind of overlap strategy would help
this a lot - using programmable readahead to stage I/O requests
asynchronously from the scan node would be better than OS hints like
fadvise(), but the tuning would be tricky IMO.

- Luke