Re: upper planner path-ification

Hello, this topic lured me on..

At Wed, 13 May 2015 23:43:57 -0400, Robert Haas <robertmhaas(at)gmail(dot)com> wrote in <CA+TgmoaQa6BcASGCL8tOXWmmOom-D7EbesAdZ4y58Cb+TJQ79g(at)mail(dot)gmail(dot)com>
> On Wed, May 13, 2015 at 10:27 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> > Both of those are problems all right, but there is more context here.
>
> Thanks for providing the context.
>
> >> I'm inclined to think that it would be useful to solve the first
> >> problem even if we didn't solve the second one right away (but that
> >> might be wrong). As a preparatory step, I'm thinking it would be
> >> sensible to split grouping_planner() into an outer function that would
> >> handle the addition of Limit and LockRows nodes and maybe planning of
> >> set operations, and an inner function that would handle GROUP BY,
> >> DISTINCT, and possibly window function planning.
> >
> > For the reasons I mentioned, I'd like to get to a point where
> > subquery_planner's output is Paths not Plans as soon as possible. But the
> > idea of coarse representation of steps that we aren't trying to be smart
> > about might be useful to save some labor in the short run.
> >
> > The zero-order version of that might be a single Path node type that
> > represents "do whatever grouping_planner would do", which we'd start to
> > break down into multiple node types once we had the other APIs fixed.
>
> The problem I'm really interested in solving is gaining the ability to
> add additional aggregation strategies, such as letting an FDW do it
> remotely, or doing it in parallel. It seems to me that your proposed
> zero-order version of that wouldn't really get us terribly far toward
> that goal - it would be more oriented towards solving the other
> problems you mention, specifically adding more intelligence to setops
> and allowing parameterization of subqueries. Those things certainly
> have some value, but I think supporting alternate aggregation
> strategies is a lot more interesting.

I don't know which you are planning the interface between the
split layers to be path or plan, I guess simply splitting
grouping_planner results in the latter. Although Tom's "path
representation up to higher layer" does not seem to clash with,
or have anything to do directly with your layer splitting,
complicating the existing complication should make it more
difficult to refactor the planner to have more intelligence.

For example, Limit could be pushed down into bottom of a path
tree ideally and would be also effective for remote node greately
like remote aggregations would be. The zero-order version would
eliminate subqery scan (or frozen plan) path or such nodes (if
any) which obstruct possible global optimization. But simple
splitting of the current grouping_planner looks make it more
difficult. It is so frustrating not to be able to express my
thought well but IMHO, shortly, I wish the planner not to grow to
be in such form.

Certainly it seems not so easy to get over the zero-order
version, but..

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center