Re: speeding up planning with partitions

On 2018/08/30 10:09, Tsunakawa, Takayuki wrote:
> From: Amit Langote [mailto:Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp]
>> I measured the gain in performance due to each patch on a modest virtual
>> machine. Details of the measurement and results follow.
>
> Amazing!

Thanks.

> UPDATE
>> nparts master 0001 0002 0003
>> ====== ====== ==== ==== ====
>> 0 2856 2893 2862 2816
>> 8 507 1115 1447 1872
>
> SELECT
>> nparts master 0001 0002 0003
>> ====== ====== ==== ==== ====
>> 0 2290 2329 2319 2268
>> 8 1058 1077 1414 1788
>
> Even a small number of partitions still introduces a not-small overhead (UPDATE:34%, SELECT:22%).

Yeah, that's true.

> Do you think this overhead can be reduced further?

We can definitely try, but I'm not immediately sure if the further
improvements will come from continuing to fix the planner. Maybe, the
overhead of partitioning could be attributed to other parts of the system.

> What part do you guess would be relevant? This one?
>
>> that it is now performed after pruning. However, it doesn't do anything
>> about the fact that partitions are all still locked in the earlier phase.

Actually, I wrote that for patch 0002. The next patch (0003) is meant to
fix that. So, the overhead you're seeing is even after making sure that
only the selected partitions are locked.

Thanks,
Amit