Re: Foreign join pushdown vs EvalPlanQual

> -----Original Message-----
> From: pgsql-hackers-owner(at)postgresql(dot)org
> [mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Robert Haas
> Sent: Saturday, September 12, 2015 1:39 AM
> To: Etsuro Fujita
> Cc: Kaigai Kouhei(海外 浩平); PostgreSQL-development; 花田茂
> Subject: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual
>
> On Thu, Sep 10, 2015 at 11:36 PM, Etsuro Fujita
> <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp> wrote:
> > I've proposed the following API changes:
> >
> > * I modified create_foreignscan_path, which is called from
> > postgresGetForeignJoinPaths/postgresGetForeignPaths, so that a path,
> > subpath, is passed as the eighth argument of the function. subpath
> > represents a local join execution path if scanrelid==0, but NULL if
> > scanrelid>0.
>
> OK, I see now. But I don't much like the way
> get_unsorted_unparameterized_path() looks.
>
> First, it's basically praying that MergePath, NodePath, and NestPath
> can be flat-copied without breaking anything. In general, we have
> copyfuncs.c support for nodes that we need to be able to copy, and we
> use copyObject() to do it. Even if what you've got here works today,
> it's not very future-proof.
>
> Second, what guarantee do we have that we'll find a path with no
> pathkeys and a NULL param_info? Why can't all of the paths for a join
> relation have pathkeys? Why can't they all be parameterized? I can't
> think of anything that would guarantee that.
>
> Third, even if such a guarantee existed, why is this the right
> behavior? Any join type will produce the same output; it's just a
> question of performance. And if you have only one tuple on each side,
> surely a nested loop would be fine.
>
> It seems to me that what you ought to be doing is using data hung off
> the fdw_private field of each RelOptInfo to cache a NestPath that can
> be used for EPQ rechecks at that level. When you go to consider
> pushing down another join, you can build up a new NestPath that's
> suitable for the new level. That seems much cleaner than groveling
> through the list of surviving paths and hoping you find the right kind
> of thing.
>
> And all that having been said, I still don't really understand why you
> are resisting the idea of providing a callback so that the FDW can
> execute arbitrary code in the recheck path. There doesn't seem to be
> any reason not to let the FDW take control of the rechecks if it
> wishes, and there's no real cost in complexity that I can see.
>
The discussion has been pending for two weeks, even though we put this
problem on the open item towards v9.5; that means we recognize it is
a problem to be fixed by the v9.5 release.

The attached patch allows FDW driver to handle EPQ recheck by its own
preferable way, even if it is alternative local join or ExecQual to
the expression being pushed down.

Regarding to the alternative join path selection, I initially thought
it is valuable to choose the best path from performance standpoint,
however, what we need to do here is visibility check towards all the
EPQ tuples already loaded to EState. So, unparametalized NestLoop is
sufficient to execute qualifier across relations.
(What happen if HashJoin is chosen? It's probably problematic.)

So, if your modified postgres_fdw keeps an alternative path, what
we need to do is construction of dummy NestPath with no param_info,
no pathkeys, and dummy cost. Then, give this path on fdw_paths of
ForeignPath. It shall be transformed to plan-nodes, then eventually
transformed to plan-state-node by postgres_fdw itself.
I cannot find out something difficult to do any more.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>