Re: Foreign join pushdown vs EvalPlanQual

Fujita-san,

> > BTW, if you try newer version of postgres_fdw foreign join patch,
> > please provide me to reproduce the problem/
>
> OK
>
Did you forget to attach the patch, or v17 is in use?

> > I'd like to suggest a solution that re-construct remote tuple according
> > to the fdw_scan_tlist on ExecScanFetch, if given scanrelid == 0.
> > It enables to run local qualifier associated with the ForeignScan node,
> > and it will also work for the case when tuple in es_epqTupleSet[] was
> > local heap.
>
> Maybe I'm missing something, but I don't think your proposal works
> properly because we don't have any component ForeignScan state node or
> subsidiary join state node once we've replaced the entire join with the
> ForeignScan performing the join remotely, IIUC. So, my image was to
> have another subplan for EvalPlanQual as well as the ForeignScan, to do
> the entire join locally for the component test tuples if we are inside
> an EvalPlanQual recheck.
>
Hmm... Probably, we have two standpoints to tackle the problem.

The first standpoint tries to handle the base foreign table as
a prime relation for locking. Thus, we have to provide a way to
fetch a remote tuple identified with the supplied ctid.
The advantage of this approach is the way to fetch tuples from
base relation is quite similar to the existing form, however,
its disadvantage is another side of the same coin, because the
ForeignScan node with scanrelid==0 (that represents remote join
query) may have local qualifiers which shall run on the tuple
according to fdw_scan_tlist.

One other standpoint tries to handle a bunch of base foreign
tables as a unit. That means, if any of base foreign table is
the target of locking, it prompts FDW driver to fetch the latest
"joined" tuple identified by "ctid", even if this join contains
multiple base relations to be locked.
The advantage of this approach is that we can use qualifiers of
the ForeignScan node with scanrelid==0 and no need to pay attention
of remote qualifier and/or join condition individually.
Its disadvantage is, we may extend EState structure to keep the
"joined" tuples, in addition to es_epqTupleSet[].

I'm inclined to think the later standpoint works well, because
it does not need to reproduce an alternative execution path in
local side again, even if a ForeignScan node represents much
complicated remote query.
If we would fetch tuples of individual base relations, we need
to reconstruct identical join path to be executed on remote-
side, don't it?

IIUC, the purpose of EvalPlanQual() is to ensure the tuples to
be locked is still visible, so it is not an essential condition
to fetch base tuples individually.

Just an aside, please tell me if someone know, does EvalPlanQual
logic work correctly even if the tuple to be locked located in
the right tree of HashJoin?
In this case, it seems to me ExecHashJoin does not refresh Hash
table again even if ExecProcNode() is invoked with es_epqTupleSet[],
thus, old tuple is already visible and checked, isn't it?

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>

> -----Original Message-----
> From: pgsql-hackers-owner(at)postgresql(dot)org
> [mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Etsuro Fujita
> Sent: Thursday, June 25, 2015 3:12 PM
> To: Kaigai Kouhei(海外 浩平)
> Cc: PostgreSQL-development
> Subject: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual
>
> Hi KaiGai-san,
>
> I'd like to work on this issue with you!
>
> On 2015/06/25 10:48, Kouhei Kaigai wrote:
> > I'd like to suggest a solution that re-construct remote tuple according
> > to the fdw_scan_tlist on ExecScanFetch, if given scanrelid == 0.
> > It enables to run local qualifier associated with the ForeignScan node,
> > and it will also work for the case when tuple in es_epqTupleSet[] was
> > local heap.
>
> Maybe I'm missing something, but I don't think your proposal works
> properly because we don't have any component ForeignScan state node or
> subsidiary join state node once we've replaced the entire join with the
> ForeignScan performing the join remotely, IIUC. So, my image was to
> have another subplan for EvalPlanQual as well as the ForeignScan, to do
> the entire join locally for the component test tuples if we are inside
> an EvalPlanQual recheck.
>
> > BTW, if you try newer version of postgres_fdw foreign join patch,
> > please provide me to reproduce the problem/
>
> OK
>
> > Also, as an aside, postgres_fdw does not implement RefetchForeignRow()
> > at this moment. Does it make a problem?
>
> I don't think so, though I think it would be better to test that the
> foreign join pushdown API patch also allows late row locking using the
> postgres_fdw.
>
> Best regards,
> Etsuro Fujita
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers