Re: upper planner path-ification

Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
> On 18 May 2015 at 14:50, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> So for the moment, let's assume that we still rigidly follow the sequence
>> of upper-level steps currently embodied in grouping_planner. (I'm not
>> sure if it even makes sense to consider other orderings of those
>> processing steps, but in any case we don't need to allow it on day zero.)
>> Then, make a dummy RelOptInfo corresponding to the result of each step,
>> and insert links to those in new fields in PlannerInfo. (We create these
>> *before* starting scan/join planning, so that FDWs, custom scans, etc, can
>> inject paths into these RelOptInfos if they want, so as to represent cases
>> like remote aggregation.) Then just use add_path with the appropriate
>> target RelOptInfo when producing different ways to do grouping etc.
>>
>> This is a bit ad-hoc but it would be a place to start.

> My thinking was to push aggregation down to the lowest level possible in
> the plan, hopefully a single relation.

In this sketch, it's basically up to an FDW to recognize when remote
aggregation is possible, and put a suitable ForeignScan path into the
RelOptInfo that corresponds to "aggregate output". It could do that for
either a single foreign relation or a foreign join, if it had enough
smarts. This is more or less where we are today for foreign queries of
all types: the responsibility is totally on the FDW to figure out whether
it can optimize the query. I have hopes that we can later provide some
infrastructure that would assist FDWs in that, so they're not all
reinventing the wheel. But I don't yet have much idea what such
infrastructure ought to look like, and it's probably hopeless to try to
design it until things have stabilized more.

Most of the other stuff you're talking about is entirely off the table
until we think about ways that aggregation could be done below the top
level, which this sketch isn't pretending to address. I agree with
Robert that that's probably best left till later. There's a fair amount
of work to do before we even have this much done.

regards, tom lane