Re: Foreign join pushdown vs EvalPlanQual

> -----Original Message-----
> From: pgsql-hackers-owner(at)postgresql(dot)org
> [mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Etsuro Fujita
> Sent: Tuesday, September 29, 2015 4:36 PM
> To: Kaigai Kouhei(海外 浩平); Robert Haas
> Cc: PostgreSQL-development; 花田茂
> Subject: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual
>
> On 2015/09/29 13:55, Kouhei Kaigai wrote:
> >> From: pgsql-hackers-owner(at)postgresql(dot)org
> >> [mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Etsuro Fujita
> >> On 2015/09/29 9:13, Kouhei Kaigai wrote:
>
> >>>> From: pgsql-hackers-owner(at)postgresql(dot)org
> >>>> [mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Robert Haas
> >>>> On Mon, Sep 28, 2015 at 3:34 AM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:
>
> >>>>> The attached patch allows FDW driver to handle EPQ recheck by its own
> >>>>> preferable way, even if it is alternative local join or ExecQual to
> >>>>> the expression being pushed down.
>
> >>>> Thanks. I was all set to commit this, or at least part of it, when I
> >>>> noticed that we already have an FDW callback called RefetchForeignRow.
> >>>> We seem to be intending that this new callback should refetch the row
> >>>> from the foreign server and verify that any pushed-down quals apply to
> >>>> it. But why can't RefetchForeignRow do that? That seems to be what
> >>>> it's for.
>
> >> I thought the same thing [1]. While I thought it was relatively easy to
> >> make changes to RefetchForeignRow that way for the foreign table case
> >> (scanrelid>0), I was not sure how hard it would be to do so for the
> >> foreign join case (scanrelid==0). So, I proposed to leave that changes
> >> for 9.6. I'll have a rethink on this issue along the lines of that
> >> approach.
>
> > Even if base relation case, is it really easy to do?
> >
> > RefetchForeignRow() does not take ForeignScanState as its argument,
> > so it is not obvious to access its private field, isn't it?
> > ExecRowMark contains "rti" field, so it might be feasible to find out
> > the target PlanState using walker routine recently supported, although
> > it is not a simple enough.
> > Unless we don't have reference to the private field, it is not feasible
> > to access expression that was pushed down to the remote-side, therefore,
> > it does not allow to apply proper rechecks here.
> >
> > In addition, it is problematic when scanrelid==0 because we have no
> > relevant ForeignScanState which represents the base relations, even
> > though ExecRowMark is associated with a particular base relation.
> > In case of scanrelid==0, EPQ recheck routine also have to ensure
> > the EPQ tuple is visible towards the join condition in addition to
> > the qualifier of base relation. These information is also stored within
> > private data field, so it has to have a reference to the private data
> > of ForeignScanState of the remote join (scanrelid==0) which contains
> > the target relation.
> >
> > Could you introduce us (1) how to access private data field of
> > ForeignScanState from the RefetchForeignRow callback? (2) why it
> > is reasonable to implement than the callback on ForeignRecheck().
>
> For the foreign table case (scanrelid>0), I imagined an approach
> different than yours. In that case, I thought the issue would be
> probably addressed by just modifying the remote query performed in
> RefetchForeignRow, which would be of the form "SELECT ctid, * FROM
> remote table WHERE ctid = $1", so that the modified query would be of
> the form "SELECT ctid, * FROM remote table WHERE ctid = $1 AND *remote
> quals*".
>
My question is how to pull expression of the remote query.
It shall be stored at somewhere private field of ForeignScanState,
however, RefetchForeignRow does not have direct access to the
relevant ForeignScanState node.
It is what I asked at the question (1).

Also note that EvalPlanQualFetchRowMarks() will raise an error
if RefetchForeignRow callback returned NULL tuple.
Is it right or expected behavior?
It looks to me this callback is designed to pull out a particular
tuple identified by the remote-row-id, regardless of the qualifier
checks based on the latest value.

> For the foreign join case (scanrelid==0), in my vision, I think we would
> need some changes not only to RefetchForeignRow but to the existing
> EvalPlanQual machinery in the core. I've not had a clear image yet, though.
>
If people agree with FDW remote join is incomplete feature in v9.5,
the attached fix-up is the minimum requirement from the standpoint
of custom-scan/join.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>