Re: Custom Scan APIs (Re: Custom Plan node)

* Kouhei Kaigai (kaigai(at)ak(dot)jp(dot)nec(dot)com) wrote:
> Do you think it makes sense if my submission was only interface portion
> without working example?

No, we're pretty strongly against putting in interfaces which don't have
working examples in contrib- for one thing, we want to know when we
break it.

> The purpose of ctidscan module is, similar to
> postgres_fdw, to demonstrate the usage of custom-scan interface with
> enough small code scale. If tons of code example were attached, nobody
> will want to review the patch.

I gathered that's why it was included. Is the plan to eventually submit
something larger to go into -contrib which will use this interface? Or
will it always be external?

> The "cache_scan" module that I and Haribabu are discussing in another
> thread also might be a good demonstration for custom-scan interface,
> however, its code scale is a bit larger than ctidscan.

That does sound interesting though I'm curious about the specifics...

> > For one thing, an example where you could have this CustomScan node calling
> > other nodes underneath would be interesting. I realize the CTID scan can't
> > do that directly but I would think your GPU-based system could; after all,
> > if you're running a join or an aggregate with the GPU, the rows could come
> > from nearly anything. Have you considered that, or is the expectation that
> > users will just go off and access the heap and/or whatever indexes directly,
> > like ctidscan does? How would such a requirement be handled?
> >
> In case when custom-scan node has underlying nodes, it shall be invoked using
> ExecProcNode as built-in node doing, then it will be able to fetch tuples
> come from underlying nodes. Of course, custom-scan provider can perform the
> tuples come from somewhere as if it came from underlying relation. It is
> responsibility of extension module. In some cases, it shall be required to
> return junk system attribute, like ctid, for row-level locks or table updating.
> It is also responsibility of the extension module (or, should not add custom-
> path if this custom-scan provider cannot perform as required).

Right, tons of work to do to make it all fit together and play nice-
what I was trying to get at is: has this actually been done? Is the GPU
extension that you're talking about as the use-case for this been
written? How does it handle all of the above? Or are we going through
all these gyrations in vain hope that it'll actually all work when
someone tries to use it for something real?

Thanks,

Stephen