Re: Parallel grouping sets

From: Pengzhou Tang <ptang(at)pivotal(dot)io>
To: Jesse Zhang <sbjesse(at)gmail(dot)com>
Cc: Richard Guo <riguo(at)pivotal(dot)io>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Parallel grouping sets
Date: 2020-02-10 03:37:19
Message-ID: CAG4reASx+p3z5W59O57xhPC57Z4MW3mSnE=s-MJGromnmgg2fA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Thanks to reviewing those patches.

Ha, I believe you meant to say a "normal aggregate", because what's
> performed above gather is no longer "grouping sets", right?
>
> The group key idea is clever in that it helps "discriminate" tuples by
> their grouping set id. I haven't completely thought this through, but my
> hunch is that this leaves some money on the table, for example, won't it
> also lead to more expensive (and unnecessary) sorting and hashing? The
> groupings with a few partials are now sharing the same tuplesort with
> the groupings with a lot of groups even though we only want to tell
> grouping 1 *apart from* grouping 10, not neccessarily that grouping 1
> needs to come before grouping 10. That's why I like the multiplexed pipe
> / "dispatched by grouping set id" idea: we only pay for sorting (or
> hashing) within each grouping. That said, I'm open to the criticism that
> keeping multiple tuplesort and agg hash tabes running is expensive in
> itself, memory-wise ...
>
> Cheers,
> Jesse

That's something we need to testing, thanks. Meanwhile, for the approach to
use "normal aggregate" with grouping set id, one concern is that it cannot
use
Mixed Hashed which means if a grouping sets contain both non-hashable or
non-sortable sets, it will fallback to one-phase aggregate.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Langote 2020-02-10 03:54:10 Re: Identifying user-created objects
Previous Message nuko yokohama 2020-02-10 03:34:32 Re: Implementing Incremental View Maintenance