Re: Foreign join pushdown vs EvalPlanQual

On 2015/08/26 13:49, Kouhei Kaigai wrote:
>> On 2015/08/25 10:18, Kouhei Kaigai wrote:
>>>> Likely, what you need to do are...
>>>> 1. Save the alternative path on fdw_paths when foreign join push-down.
>>>> GetForeignJoinPaths() may be called multiple times towards a particular
>>>> joinrel according to the combination of innerrel/outerrel.
>>>> RelOptInfo->fdw_private allows to avoid construction of same remote
>>>> join path multiple times. On the second or later invocation, it may be
>>>> a good tactics to reference cheapest_startup_path and replace the saved
>>>> one if later invocation have cheaper one, prior to exit.

>> I'm not sure that the tactics is a good one. I think you probably
>> assume that GetForeignJoinPaths executes set_cheapest each time that
>> gets called, but ISTM that that would be expensive. (That is one of the
>> reason why I think it would be better to hook that routine in
>> standard_join_search.)

> Here is two different problems. I'd like to identify whether the problem
> is "must be solved" or "nice to have". Obviously, failure on EPQ check
> is a problem must be solved, however, hook location is nice to have.

OK I'll focus on the "must be solved" problem at least on this thread.

> In addition, you may misunderstand the proposition of mine above.
> You can check RelOptInfo->fdw_private on top of the GetForeignJoinPaths,
> then, if it is second or later invocation, you can check cost of the
> alternative path kept in the ForeignPath node previously constructed.
> If cheapest_total_path at the moment of GetForeignJoinPaths invocation
> is cheaper than the saved alternative path, you can adjust the node to
> replace the alternative path node.

To get the (probably unparameterized) cheapest_total_path, IIUC, we need
to do set_cheapest during GetForeignJoinPaths in each subsequent
invocation of that routine, don't we? And set_cheapest is expensive,
isn't it?

>>>> 2. Save the alternative Plan nodes on fdw_plans or lefttree/righttree
>>>> somewhere you like at the GetForeignPlan()
>>>> 3. Makes BeginForeignScan() to call ExecInitNode() towards the plan node
>>>> saved at (2), then save the PlanState on fdw_ps, lefttree/righttree,
>>>> or somewhere private area if not displayed on EXPLAIN.
>>>> 4. Implement ForeignRecheck() routine. If scanrelid==0, it kicks the
>>>> planstate node saved at (3) to generate tuple slot. Then, call the
>>>> ExecQual() to check qualifiers being pushed down.
>>>> 5. Makes EndForeignScab() to call ExecEndNode() towards the PlanState
>>>> saved at (3).

>> but the design that you proposed
>> looks complicated beyond necessity. I think we should add an FDW API
>> for doing something if FDWs have more knowledge about doing that than
>> the core, but in your proposal, instead of the core, an FDW has to
>> eventually do a lot of the core's work: ExecInitNode, ExecProcNode,
>> ExecQual, ExecEndNode and so on.

> It is a trade-off problem between interface flexibility and code smallness
> of FDW extension if it fits scope of the core support.
> I stand on the viewpoint that gives highest priority on the flexibility,
> especially, in case when unpredictable type of modules are expected.
> Your proposition is comfortable to FDW on behalf of RDBMS, however, nobody
> can promise it is beneficial to FDW on behalf of columnar-store for example.

Maybe I'm missing something, but why do we need such a flexiblity for
the columnar-stores?

> If you stick on the code smallness of FDW on behalf of RDBMS, we can add
> utility functions on foreign.c or somewhere. It will be able to provide
> equivalent functionality, but FDW can determine whether it use the routines.

That might be an idea, but I'd like to hear the opinions of others.

Best regards,
Etsuro Fujita