Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

Hi David.

On 2018/01/18 12:14, David Rowley wrote:
> On 18 January 2018 at 00:13, David Rowley <david(dot)rowley(at)2ndquadrant(dot)com> wrote:
>> On 17 January 2018 at 23:48, Amit Langote <amitlangote09(at)gmail(dot)com> wrote:
>>> I'm concerned that after your patch to remove
>>> match_clauses_to_partkey(), we'd be doing more work than necessary in
>>> some cases. For example, consider the case of using run-time pruning
>>> for nested loop where the inner relation is a partitioned table. With
>>> the old approach, get_partitions_from_clauses() would only be handed
>>> the clauses that are known to match the partition keys (which most
>>> likely is fewer than all of the query's clauses), so
>>> get_partitions_from_clauses() doesn't have to do the work of filtering
>>> non-partition clauses every time (that is, for every outer row).
>>> That's why I had decided to keep that part in the planner.
>>
>> That might be better served by splitting
>> classify_partition_bounding_keys() into separate functions, the first
>> function would be in charge of building keyclauses_all. That way the
>> remaining work during the executor would never need to match clauses
>> to a partition key as they'd be in lists dedicated to each key.
>
> I've attached another delta against your v20 patch which does this.
> It's very rough for now and I've only checked that it passes the
> regression test so far.

Thanks!

> It will need some cleanup work, but I'd be keen to know what you think
> of the general idea.

This one looks in a much better shape.

> I've not fully worked out how run-time pruning
> will use this as it'll need another version of
> get_partitions_from_clauses but passes in a PartScanClauseInfo
> instead, and does not call extract_partition_key_clauses. That area
> probably needs some shuffling around so that does not end up a big
> copy and paste of all that new logic.
So, I've been assuming that the planner changes in the run-time pruning
patch have to do with selecting clauses (restriction clauses not
containing Consts and/or join clauses) to be passed to the executor by
recording them in the Append node. Will they be selected by the planner
calling into partition.c?

Meanwhile, here is a revised version (v21) that incorporates your changes.
I added you as the author in 0002 and 0005 patches. I guess a v22 will
have to follow very soon...

Thanks,
Amit