Re: Foreign join pushdown vs EvalPlanQual

Hello,

At Thu, 26 Nov 2015 05:04:32 +0000, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote in <9A28C8860F777E439AA12E8AEA7694F801176205(at)BPXM15GP(dot)gisp(dot)nec(dot)co(dot)jp>
> > On 2015/11/24 2:41, Robert Haas wrote:
> > > On Fri, Nov 20, 2015 at 12:11 AM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
> > >> One subplan means FDW driver run an entire join sub-tree with local
> > >> alternative sub-plan; that is my expectation for the majority case.
> >
> > > What I'm imagining is that we'd add handling that allows the
> > > ForeignScan to have inner and outer children. If the FDW wants to
> > > delegate the EvalPlanQual handling to a local plan, it can use the
> > > outer child for that. Or the inner one, if it likes. The other one
> > > is available for some other purposes which we can't imagine yet. If
> > > this is too weird, we can only add handling for an outer subplan and
> > > forget about having an inner subplan for now. I just thought to make
> > > it symmetric, since outer and inner subplans are pretty deeply baked
> > > into the structure of the system.
> >
> > I'd vote for only allowing an outer subplan.
> >
> The attached patch adds: Path *fdw_outerpath field to ForeignPath node.
> FDW driver can set arbitrary but one path-node here.

It is named "outerpath/plan". Surely we used the term 'outer' in
association with other nodes for disign decision but is it valid
to call it outer? Addition to that, there's no innerpath in this
patch and have "path" instead.

> After that, this path-node shall be transformed to plan-node by
> createplan.c, then passed to FDW driver using GetForeignPlan callback.
> We expect FDW driver set this plan-node on lefttree (a.k.a outerPlan).
> The Plan->outerPlan is a common field, so patch size become relatively

Plan->outerPlan => Plan->lefttree?

> small. FDW driver can initialize this plan at BeginForeignScan, then
> execute this sub-plan-tree on demand.
>
> Remaining portions are as previous version. ExecScanFetch is revised
> to call recheckMtd always when scanrelid==0, then FDW driver can get
> control using RecheckForeignScan callback.

Perhaps we need a comment about foreignscan as a fake join for
the case with scanrelid == 0 in ExecScanReScan.

> It allows FDW driver to handle (1) EPQ recheck on underlying scan nodes,
> (2) reconstruction of joined tuple, and (3) EPQ recheck on join clauses,
> by its preferable implementation - including execution of an alternative
> sub-plan.

In ForeignRecheck, ExecQual on fdw_recheck_quals is executed if
RecheckForeignScan returns true, which I think the signal that
the returned tuple matches the recheck qual. Whether do you think
have the responsibility to check the reconstructed tuple when
RecheckCoreignScan is registered, RecheckForeignScan or ExecQual?

The documentation says as following so I think the former has.

# I don't understhad what 'can or must' means, though... 'can and
# must'?

+ Also, this callback can or must recheck scan qualifiers and join
+ conditions which are pushed down. Especially, it needs special

> > There seems to be no changes to make_foreignscan. Is that OK?
> >
> create_foreignscan_path(), not only make_foreignscan().
>
> This patch is not tested by actual FDW extensions, so it is helpful
> to enhance postgres_fdw to run the alternative sub-plan on EPQ recheck.

regardes,

--
Kyotaro Horiguchi
NTT Open Source Software Center