Re: Foreign join pushdown vs EvalPlanQual

On 2015/09/29 13:55, Kouhei Kaigai wrote:
>> From: pgsql-hackers-owner(at)postgresql(dot)org
>> [mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Etsuro Fujita
>> On 2015/09/29 9:13, Kouhei Kaigai wrote:

>>>> From: pgsql-hackers-owner(at)postgresql(dot)org
>>>> [mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Robert Haas
>>>> On Mon, Sep 28, 2015 at 3:34 AM, Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com> wrote:

>>>>> The attached patch allows FDW driver to handle EPQ recheck by its own
>>>>> preferable way, even if it is alternative local join or ExecQual to
>>>>> the expression being pushed down.

>>>> Thanks. I was all set to commit this, or at least part of it, when I
>>>> noticed that we already have an FDW callback called RefetchForeignRow.
>>>> We seem to be intending that this new callback should refetch the row
>>>> from the foreign server and verify that any pushed-down quals apply to
>>>> it. But why can't RefetchForeignRow do that? That seems to be what
>>>> it's for.

>> I thought the same thing [1]. While I thought it was relatively easy to
>> make changes to RefetchForeignRow that way for the foreign table case
>> (scanrelid>0), I was not sure how hard it would be to do so for the
>> foreign join case (scanrelid==0). So, I proposed to leave that changes
>> for 9.6. I'll have a rethink on this issue along the lines of that
>> approach.

> Even if base relation case, is it really easy to do?
>
> RefetchForeignRow() does not take ForeignScanState as its argument,
> so it is not obvious to access its private field, isn't it?
> ExecRowMark contains "rti" field, so it might be feasible to find out
> the target PlanState using walker routine recently supported, although
> it is not a simple enough.
> Unless we don't have reference to the private field, it is not feasible
> to access expression that was pushed down to the remote-side, therefore,
> it does not allow to apply proper rechecks here.
>
> In addition, it is problematic when scanrelid==0 because we have no
> relevant ForeignScanState which represents the base relations, even
> though ExecRowMark is associated with a particular base relation.
> In case of scanrelid==0, EPQ recheck routine also have to ensure
> the EPQ tuple is visible towards the join condition in addition to
> the qualifier of base relation. These information is also stored within
> private data field, so it has to have a reference to the private data
> of ForeignScanState of the remote join (scanrelid==0) which contains
> the target relation.
>
> Could you introduce us (1) how to access private data field of
> ForeignScanState from the RefetchForeignRow callback? (2) why it
> is reasonable to implement than the callback on ForeignRecheck().

For the foreign table case (scanrelid>0), I imagined an approach
different than yours. In that case, I thought the issue would be
probably addressed by just modifying the remote query performed in
RefetchForeignRow, which would be of the form "SELECT ctid, * FROM
remote table WHERE ctid = $1", so that the modified query would be of
the form "SELECT ctid, * FROM remote table WHERE ctid = $1 AND *remote
quals*".

For the foreign join case (scanrelid==0), in my vision, I think we would
need some changes not only to RefetchForeignRow but to the existing
EvalPlanQual machinery in the core. I've not had a clear image yet, though.

Best regards,
Etsuro Fujita