Re: Custom Scan APIs (Re: Custom Plan node)

Hi Stephen,

Thanks for your comments.

> * Kouhei Kaigai (kaigai(at)ak(dot)jp(dot)nec(dot)com) wrote:
> > Is somebody available to volunteer to review the custom-scan patch?
>
> I looked through it a bit and my first take away from it was that the patches
> to actually use the new hooks were also making more changes to the backend
> code, leaving me with the impression that the proposed interface isn't
> terribly stable. Perhaps those changes should have just been in the first
> patch, but they weren't and that certainly gave me pause.
>
Yes, the part-1 patch provides a set of interface portion to interact
between the backend code and extension code. Rest of part-2 and part-3
portions are contrib modules that implements its feature on top of
custom-scan API.

> I'm also not entirely convinced that this is the direction to go in when
> it comes to pushing down joins to FDWs. While that's certainly a goal that
> I think we all share, this seems to be intending to add a completely different
> feature which happens to be able to be used for that. For FDWs, wouldn't
> we only present the FDW with the paths where the foreign tables for that
> FDW, or perhaps just a given foreign server, are being joined?
>
FDW's join pushing down is one of the valuable use-cases of this interface,
but not all. As you might know, my motivation is to implement GPU acceleration
feature on top of this interface, that offers alternative way to scan or join
relations or potentially sort or aggregate.
Probably, it is too stretch interpretation if we implement radix-sort on top
of FDW. I'd like you to understand the part-3 patch (FDW's join pushing-down)
is a demonstration of custom-scan interface for application, but not designed
for a special purpose.

Right now, I put all the logic to interact CSI and FDW driver on postgres_fdw
side, it might be an idea to have common code (like a logic to check whether
the both relations to be joined belongs to same foreign server) on the backend
side as something like a gateway of them.

As an aside, what should be the scope of FDW interface?
In my understanding, it allows extension to implement "something" on behalf of
a particular data structure being declared with CREATE FOREIGN TABLE.
In other words, extension's responsibility is to generate a view of "something"
according to PostgreSQL' internal data structure, instead of the object itself.
On the other hands, custom-scan interface allows extensions to implement
alternative methods to scan or join particular relations, but it is not a role
to perform as a target being referenced in queries. In other words, it is methods
to access objects.
It is natural both features are similar because both of them intends extensions
to hook the planner and executor, however, its purpose is different.

Thanks,
--
NEC OSS Promotion Center / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>