Re: Increasing parallel workers at runtime

On Tue, May 23, 2017 at 4:41 PM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:

> Isn't this point contradictory? Basically, on one side you are
> suggesting to calculate additional workers (work_remaining) based on
> selectivity and on another side you are saying that it will fix
> estimation errors. IIUC, then you are talking about some sort of
> executor feedback to adjust a number of workers which might be a good
> thing to do but I that is a different problem to solve. As of now, I
> don't think we have that type of mechanism even for non-parallel
> execution.
>
Yes, I am talking of a executor feedback mechanism sort of a thing.
For non parallel execution it's a lot more challenging since the
execution plan might need to be changed in the runtime, but in
parallel query case only addition/subtraction of workers is to be
dealt, which appears to be a less complex thing.
Why I am thinking in this direction is because in my experience it
seems very easy to regress with more workers when over-estimation is
done and this runtime calculation of workers can mitigate it largely.
However, I understand that this might require more proof than my
experience alone. Let's see if anybody else shares my gut feeling. :)

--
Regards,
Rafia Sabih
EnterpriseDB: http://www.enterprisedb.com/