Re: using CustomScan to inject nodes into the plan

On 11.3.2015 21:53, Tom Lane wrote:
> Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com> writes:
>> I've been experimenting with using CustomScan to inject nodes into the
>> plan - I'm experimenting a bit, and this seemed like a nice way to do
>> that in an extension, outside the tree.
>
>> Sadly set_rel_pathlist_hook is not flexible enough, because it only
>> allows overriding paths for base relations, while I'd like to inject
>> nodes above joins, for example, so instead of
>
> It's Custom*Scan*, not Custom-anything-you-want. It's only supposed
> to exist at the bottom level of the plan tree, and that's why it
> does not have fields that would allow using it somewhere else. Quite
> aside from the scanrelid thing, haven't you noticed that it lacks any
> link to child plan nodes?

As I mentioned, I'm aware this is a misuse of a Scan. So yes, I'm well
aware it's not Custom-anything.

Regarding the child plan nodes, those are in the Plan. I got this whole
thing working as an experiment, and yes - it seems to work, at least
when the node is injected right above a seqscan for example.

> I'm not terribly on board with this whole concept of adding plan
> nodes from extensions; I do not believe that that approach will ever
> lead to anything except horrid kluges, bad planning decisions, and
> requests for ever-more-outre hooks to shift the kluginess somewhere
> else. But as Robert says, the custom join thing is more likely to be
> usable for this, and it would be interesting to hear just how bad the
> impedance mismatch is if you try that.

FWIW, I've been experimenting with some ideas, and simply injecting a
node through CustomScan and planner_hook seemed like an easier way to
see if it makes sense and develop a proper in-core plan node. And also
looking at CustomScan, which I haven't really followed before.

Aside from this different would that be from what CustomScan (and
CustomJoin) do? Apparently, there are uses for that - e.g. say you know
how to perform Scan and Join nodes on a specialized hardware, but moving
the tuples back and forth is rather expensive. Why not to offload a
larger part of the plan at once? I know people are working on such
things (e.g. on GPUs).

Clearly, CustomAnything would be a nightmare, no doubt about that.

--
Tomas Vondra http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services