Re: Custom Plan node

2013/10/3 Robert Haas <robertmhaas(at)gmail(dot)com>:
>>> I am a little less sanguine about the chances of a CustomJoin node
>>> working out well. I agree that we need something to handle join
>>> pushdown, but it seems to me that might be done by providing a Foreign
>>> Scan path into the joinrel rather than by adding a concept of foreign
>>> joins per se.
>>>
>> Indeed, if we have a hook on add_paths_to_joinrel(), it also makes
>> sense for foreign tables; probably, planner will choose foreign-path
>> instead of existing join node including foreign-scans.
>
> Yes, I think it's reasonable to think about injecting a scan path into
> a join node.
>
>>> And I think that lumping everything else together under "not a scan or
>>> join" has the least promise of all. Is replacing Append really the
>>> same as replacing Sort? I think we'll need to think harder here about
>>> what we're trying to accomplish and how to get there.
>>>
>> As long as extension modifies PlannedStmt on the planner_hook,
>> I don't think it is not difficult so much, as I demonstrate on the
>> previous patch.
>> Unlike scan or join, existing code is not designed to compare
>> multiple possible paths, so it seems to me a feature to adjust
>> a plan-tree already construct is sufficient for most usage
>> because extension can decide which one can offer more cheap
>> path than built-in ones.
>
> Well, there were a lot of problems with your demonstration, which have
> already been pointed out upthread. I'm skeptical about the idea of
> simply replacing planner nodes wholesale, and Tom is outright opposed.
> I think you'll do better to focus on a narrower case - I'd suggest
> custom scan nodes - and leave the rest as a project for another time.
>
Thanks, it makes me clear what we should target on v9.4 development.
Towards the next commitfest, I'm planning to develop the following
features:
* CustomScan node that can run custom code instead of built-in
scan nodes.
* Join-pushdown of postgres_fdw using the hook to be located on
the add_paths_to_joinrel(), for demonstration purpose.
* Something new way to scan a relation; probably, your suggested
ctid scan with less or bigger qualifier is a good example, also for
demonstration purpose.

Probably, above set of jobs will be the first chunk of this feature.
Then, let's do other stuff like Append, Sort, Aggregate and so on
later. It seems to me a reasonable strategy.

Even though it is an off-topic in this thread....

>> Regarding to the "guest module" of this framework, another
>> idea that I have is, built-in query cache module that returns
>> previous scan result being cached if table contents was not
>> updated from the previous run. Probably, it makes sense in
>> case when most of rows are filtered out in this scan.
>> Anyway, I'd like to consider something useful to demonstrate
>> this API.
>
> I doubt that has any chance of working well. Supposing that query
> caching is a feature we want, I don't think that plan trees are the
> right place to try to install it. That sounds like something that
> ought to be done before we plan and execute the query in the first
> place.
>
The query cache I mention above is intended to cache a part of
query tree. For example, if an ad-hoc query tries to join A,B and
C, and data analyst modifies qualifiers attached on C but it joins
A and B as usual. I though, it makes sense if we can cache the
result of the statically joined result. Idea is similar to materialized
view. If executor can pick up cached result on AxB, all it needs
to do is joining C and the cached one.
Of course, I know it may have difficulty like cache validation. :-(

Thanks,
--
KaiGai Kohei <kaigai(at)kaigai(dot)gr(dot)jp>