Re: [HACKERS] Runtime Partition Pruning

From: Beena Emerson <memissemerson(at)gmail(dot)com>
To: David Rowley <david(dot)rowley(at)2ndquadrant(dot)com>
Cc: Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>, Robert Haas <robertmhaas(at)gmail(dot)com>, amul sul <sulamul(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Subject: Re: [HACKERS] Runtime Partition Pruning
Date: 2017-12-21 12:43:31
Message-ID: CAOG9ApGH8AQRUhpvEaC=hgMKut7UPcz9dhyx3QhdQGLYjgT8cg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello,

On Thu, Dec 21, 2017 at 6:01 PM, David Rowley
<david(dot)rowley(at)2ndquadrant(dot)com> wrote:
> On 21 December 2017 at 23:51, Beena Emerson <memissemerson(at)gmail(dot)com> wrote:
>> On Thu, Dec 21, 2017 at 2:31 PM, David Rowley
>> <david(dot)rowley(at)2ndquadrant(dot)com> wrote:
>>> On 19 December 2017 at 21:54, Beena Emerson <memissemerson(at)gmail(dot)com> wrote:
>>
>>> The problem is down to the logic in choose_custom_plan() only choosing
>>> a generic plan if the average cost of the generic plan is less than
>>> the average custom plan cost. The problem is that the generic plan can
>>> have many extra Append subnodes in comparison to the custom plan, all
>>> of which are taken into account in the total plan cost, but these may
>>> be pruned during execution. The logic in choose_custom_plan() has no
>>> idea about this..
>>
>> I had mentioned this in the first mail on this thread that the generic
>> plan is always preferred and Robert said that it is not in scope of
>> this patch. Maybe we can start a new thread for this.
>
> Sorry, I missed that. I knew you had modified choose_custom_plan()
> too, but I didn't know the diagnosis of the problem had made its way
> here yet.
>
> I wonder if Robert understands the extent of the problem. The patch
> will be useful to prune away partitions when the partitioned table is
> on the inside of a parameterised nested loop join, but I think we'll
> likely get some complaints about PREPARE statements always using a
> custom plan. It's going to make this pretty hard to write regression
> tests for unless we went and invent some GUC "prefer_generic_plan", or
> "generic_plan_cost_multiplier" to provide a way to coax
> choose_custom_plan() into not choosing a custom plan. I admit to not
> having the answer to this but I don't think we should rule out
> discussing possible fixes.

I have attached the regression tests I am currently using to check my
patch. It has few prepared statements using runtime pruning.

--

Beena Emerson

EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Attachment Content-Type Size
runtime_pruning_testcases.patch application/octet-stream 24.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Fabien COELHO 2017-12-21 12:44:59 Re: General purpose hashing func in pgbench
Previous Message David Rowley 2017-12-21 12:31:04 Re: [HACKERS] Runtime Partition Pruning