Re: parallel joins, and better parallel explain

On Wed, Dec 9, 2015 at 11:51 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
> On Fri, Dec 4, 2015 at 3:07 AM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
wrote:
>
> > I think the problem is at Gather node, the number of buffers (read +
hit)
> > are greater than the number of pages in relation. The reason why it
> > is doing so is that in Workers (ParallelQueryMain()), it starts the
buffer
> > usage accumulation before ExecutorStart() whereas in master backend
> > it always calculate it for ExecutorRun() phase, on changing it to
accumulate
> > for ExecutorRun() phase above problem is fixed. Attached patch fixes
the
> > problem.
>
> Why is it a bad thing to capture the cost of doing ExecutorStart() in
> the worker? I can see there's an argument that changing this would be
> more consistent, but I'm not totally convinced. The overhead of
> ExecutorStart() in the leader isn't attributable to any specific
> worker, but the overhead of ExecutorStart() in the worker can fairly
> be blamed on Gather, I think.
>

This boils down to the question why currently buffer usage or other
similar stats (time for node execution) for a node doesn't include
the cost for ExecutorStart(). I think the reason is that ExecutorStart()
does some other miscellaneous works like accessing system tables
or initialization of nodes which we might not even execute, so
accumulating the cost for such work doesn't seems to be meaningful.
Looking at references of InstrStartNode() which actually marks the
beginning of buffer usage stats, it is clear that buffer usage stats are
not counted for ExecutorStart() phase, so I think following the same
for worker stats seems to be more accurate.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com