Re: [HACKERS] [POC] Faster processing at Gather node

On Thu, Nov 16, 2017 at 6:42 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> The problem here is that we have no idea how big the queue needs to
> be. The workers will always be happy to generate tuples faster than
> the leader can read them, if that's possible, but it will only
> sometimes help performance to let them do so. I think in most cases
> we'll end up allocating the local queue - because the workers can
> generate faster than the leader can read - but only occasionally will
> it make anything faster.

For the Gather Merge driven by Parallel Index Scan case it seems to me
that the correct queue size is one that can store two index pages
worth of tuples. Additional space will always help buffer any
performance variations, but there should be a step function somewhere
around 1+1/n_workers pages. I wonder if the queue could be dynamically
sized based on the driving scan. With some limits of course as parent
nodes to the parallel index scan can increase the row count by
arbitrary amounts.

Regards,
Ants Aasma
--
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26, A-2700 Wiener Neustadt
Web: http://www.postgresql-support.de, http://www.cybertec.at