Re: Foreign join pushdown vs EvalPlanQual

> -----Original Message-----
> From: Etsuro Fujita [mailto:fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp]
> Sent: Thursday, November 12, 2015 2:54 PM
> To: Robert Haas
> Cc: Kaigai Kouhei(海外 浩平); Tom Lane; Kyotaro HORIGUCHI;
> pgsql-hackers(at)postgresql(dot)org; Shigeru Hanada
> Subject: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual
>
> On 2015/11/12 2:53, Robert Haas wrote:
> > On Sun, Nov 8, 2015 at 11:13 PM, Etsuro Fujita
> > <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp> wrote:
> >> To test this change, I think we should update the postgres_fdw patch so as
> >> to add the RecheckForeignScan.
> >>
> >> Having said that, as I said previously, I don't see much value in adding the
> >> callback routine, to be honest. I know KaiGai-san considers that that would
> >> be useful for custom joins, but I don't think that that would be useful even
> >> for foreign joins, because I think that in case of foreign joins, the
> >> practical implementation of that routine in FDWs would be to create a
> >> secondary plan and execute that plan by performing ExecProcNode, as my patch
> >> does [1]. Maybe I'm missing something, though.
>
> > I really don't see why you're fighting on this point. Making this a
> > generic feature will require only a few extra lines of code for FDW
> > authors. If this were going to cause some great inconvenience for FDW
> > authors, then I'd agree it isn't worth it. But I see zero evidence
> > that this is actually the case.
>
> Really? I think there would be not a little burden on an FDW author;
> when postgres_fdw delegates to the subplan to the remote server, for
> example, it would need to create a remote join query by looking at
> tuples possibly fetched and stored in estate->es_epqTuple[], send the
> query and receive the result during the callback routine.
>
I cannot understand why it is the only solution.
Our assumption is, FDW driver knows the best way to do. So, you can
take the best way for your FDW driver - including what you want to
implement in the built-in feature.

> Furthermore,
> what I'm most concerned about is that wouldn't be efficient. So, my
>
You have to add "because ..." sentence here because I and Robert
think a little inefficiency is not a problem. If you try to
persuade other parsons who have different opinion, you need to
introduce WHY you have different conclusion. (Of course, we might
oversight something)
Please don't start the sentence from "I think ...". We all knows
your opinion, but what I've wanted to see is "the reason why my
approach is valuable is ...".

I never suggest something technically difficult, but it is
a problem on communication.

> question about that approach is whether FDWs really do some thing like
> that during the callback routine, instead of performing a secondary join
> plan locally.
>
Nobody prohibits postgres_fdw performs a secondary join here.
All you need to do is, picking up a sub-plan tree from FDW's private
field then call ExecProcNode() inside the callback.

> As I said before, I know that KaiGai-san considers that
> that approach would be useful for custom joins. But I see zero evidence
> that there is a good use-case for an FDW.
>
> > From my point of view I'm now
> > thinking this solution has two parts:
> >
> > (1) Let foreign scans have inner and outer subplans. For this
> > purpose, we only need one, but it's no more work to enable both, so we
> > may as well. If we had some reason, we could add a list of subplans
> > of arbitrary length, but there doesn't seem to be an urgent need for
> > that.
> >
> > (2) Add a recheck callback.
> >
> > If the foreign data wrapper wants to adopt the solution you're
> > proposing, the recheck callback can call
> > ExecProcNode(outerPlanState(node)). I don't think this should end up
> > being more than a few lines of code, although of course we should
> > verify that. So no problem: postgres_fdw and any other FDWs where the
> > remote side is a database can easily delegate to a subplan, and
> > anybody who wants to do something else still can.
> >
> > What is not to like about that?
> >

--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>