Re: Parallel Seq Scan

On Wed, Apr 1, 2015 at 6:03 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
> On Tue, Mar 31, 2015 at 8:53 AM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
wrote:
> >> It looks to me like the is an InitPlan, not a subplan. There
> >> shouldn't be any problem with a Funnel node having an InitPlan; it
> >> looks to me like all of the InitPlan stuff is handled by common code
> >> within the executor (grep for initPlan), so it ought to work here the
> >> same as it does for anything else. What I suspect is failing
> >> (although you aren't being very clear about it here) is the passing
> >> down of the parameters set by the InitPlan to the workers.
> >
> > It is failing because we are not passing InitPlan itself (InitPlan is
> > nothing but a list of SubPlan) and I tried tried to describe in previous
> > mail [1] what we need to do to achieve the same, but in short, it is not
> > difficult to pass down the required parameters (like plan->InitPlan or
> > plannedstmt->subplans), rather the main missing part is the handling
> > of such parameters in worker side (mainly we need to provide support
> > for all plan nodes which can be passed as part of InitPlan in
readfuncs.c).
> > I am not against supporting InitPlan's on worker side, but just wanted
to
> > say that if possible why not leave that for first version.
>
> Well, if we *don't* handle it, we're going to need to insert some hack
> to ensure that the planner doesn't create plans. And that seems
> pretty unappealing. Maybe it'll significantly compromise plan
> quality, and maybe it won't, but at the least, it's ugly.
>

I also think changing anything in planner related to this is not a
good idea, but what about detecting this during execution (into
ExecFunnel) and then just run the plan locally (by master backend)?

> > [1]
> > I have tried to evaluate what it would take us to support execution
> > of subplans by parallel workers. We need to pass the sub plans
> > stored in Funnel Node (initPlan) and corresponding subplans stored
> > in planned statement (subplans) as subplan's stored in Funnel node
> > has reference to subplans in planned statement. Next currently
> > readfuncs.c (functions to read different type of nodes) doesn't support
> > reading any type of plan node, so we need to add support for reading all
> > kind
> > of plan nodes (as subplan can have any type of plan node) and similarly
> > to execute any type of Plan node, we might need more work
(infrastructure).
>
> I don't think you need to do anything that complicated. I'm not
> proposing to *run* the initPlan in the workers, just to pass the
> parameter values down.
>

Sorry, but I am not able to understand how it will help if just parameters
(If I understand correctly you want to say about Bitmapset *extParam;
Bitmapset *allParam; in Plan structure) are passed to workers and I
think they are already getting passed only initPlan and related Subplan
in planned statement is not passed and the reason is that ss_finalize_plan()
attaches initPlan to top node (which in this case is Funnel node and not
PartialSeqScan)

By any chance, do you mean that we run the part of the statement in
workers and then run initPlan in master backend?

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com