Re: Foreign join pushdown vs EvalPlanQual

> On 2015/08/25 10:18, Kouhei Kaigai wrote:
> > How about your opinion towards the solution?
>
> >> Likely, what you need to do are...
> >> 1. Save the alternative path on fdw_paths when foreign join push-down.
> >> GetForeignJoinPaths() may be called multiple times towards a particular
> >> joinrel according to the combination of innerrel/outerrel.
> >> RelOptInfo->fdw_private allows to avoid construction of same remote
> >> join path multiple times. On the second or later invocation, it may be
> >> a good tactics to reference cheapest_startup_path and replace the saved
> >> one if later invocation have cheaper one, prior to exit.
>
> I'm not sure that the tactics is a good one. I think you probably
> assume that GetForeignJoinPaths executes set_cheapest each time that
> gets called, but ISTM that that would be expensive. (That is one of the
> reason why I think it would be better to hook that routine in
> standard_join_search.)
>
Here is two different problems. I'd like to identify whether the problem
is "must be solved" or "nice to have". Obviously, failure on EPQ check
is a problem must be solved, however, hook location is nice to have.

In addition, you may misunderstand the proposition of mine above.
You can check RelOptInfo->fdw_private on top of the GetForeignJoinPaths,
then, if it is second or later invocation, you can check cost of the
alternative path kept in the ForeignPath node previously constructed.
If cheapest_total_path at the moment of GetForeignJoinPaths invocation
is cheaper than the saved alternative path, you can adjust the node to
replace the alternative path node.

> >> 2. Save the alternative Plan nodes on fdw_plans or lefttree/righttree
> >> somewhere you like at the GetForeignPlan()
> >> 3. Makes BeginForeignScan() to call ExecInitNode() towards the plan node
> >> saved at (2), then save the PlanState on fdw_ps, lefttree/righttree,
> >> or somewhere private area if not displayed on EXPLAIN.
> >> 4. Implement ForeignRecheck() routine. If scanrelid==0, it kicks the
> >> planstate node saved at (3) to generate tuple slot. Then, call the
> >> ExecQual() to check qualifiers being pushed down.
> >> 5. Makes EndForeignScab() to call ExecEndNode() towards the PlanState
> >> saved at (3).
>
> >> I never think above steps are "too" complicated for people who can write
> >> FDW drivers. It is what developer usually does.
>
> Sorry, my explanation was not accurate, but the design that you proposed
> looks complicated beyond necessity. I think we should add an FDW API
> for doing something if FDWs have more knowledge about doing that than
> the core, but in your proposal, instead of the core, an FDW has to
> eventually do a lot of the core's work: ExecInitNode, ExecProcNode,
> ExecQual, ExecEndNode and so on.
>
It is a trade-off problem between interface flexibility and code smallness
of FDW extension if it fits scope of the core support.
I stand on the viewpoint that gives highest priority on the flexibility,
especially, in case when unpredictable type of modules are expected.
Your proposition is comfortable to FDW on behalf of RDBMS, however, nobody
can promise it is beneficial to FDW on behalf of columnar-store for example.

If you stick on the code smallness of FDW on behalf of RDBMS, we can add
utility functions on foreign.c or somewhere. It will be able to provide
equivalent functionality, but FDW can determine whether it use the routines.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>