Re: Parallel Seq Scan

On Thu, Sep 17, 2015 at 11:44 PM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> Okay, but I think the same can be achieved with this as well. Basic idea
> is that each worker will work on one planned statement at a time and in
> above case there will be two different planned statements and they will
> store partial seq scan related information in two different loctions in
> toc, although the key (PARALLEL_KEY_SCAN) would be same and I think this
> will quite similar to what we are already doing for response queues.
> The worker will work on one of those keys based on planned statement
> which it chooses to execute. I have explained this in somewhat more details
> in one of my previous mails [1].

shm_toc keys are supposed to be unique. If you added more than one
with the same key, there would be no look up the second one. That was
intentional, and I don't want to revise it.

I don't want to have multiple PlannedStmt objects in any case. That
doesn't seem like the right approach. I think passing down an Append
tree with multiple Partial Seq Scan children to be run in order is
simple and clear, and I don't see why we would do it any other way.
The master should be able to generate a plan and then copy the part of
it below the Funnel and send it to the worker. But there's clearly
never more than one PlannedStmt in the master, so where would the
other ones come from in the worker? There's no reason to introduce
that complexity.

>> Each partial sequential scan needs to have a *separate* key, which
>> will need to be stored in either the Plan or the PlanState or both
>> (not sure exactly). Each partial seq scan needs to get assigned a
>> unique key there in the master, probably starting from 0 or 100 or
>> something and counting up, and then this code needs to extract that
>> value and use it to look up the correct data for that scan.
>
> In that case also, multiple workers can worker on same key, assuming
> in your above example, multiple workers will be required to execute
> each partial seq scan. In this case we might need to see how to map
> instrumentation information for a particular execution.

That was discussed on the nearby thread about numbering plan nodes.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company