Re: Memory-Bounded Hash Aggregation

On Mon, 2020-02-03 at 18:24 -0800, Jeff Davis wrote:
> On Mon, 2020-02-03 at 10:29 -0800, Jeff Davis wrote:
> > I ended up converting the freelist to a min heap.
> >
> > Attached is a patch which makes three changes to better support
> > HashAgg:
>
> And now I'm attaching another version of the main Hash Aggregation
> patch to be applied on top of the logtape.c patch.
>
> Not a lot of changes from the last version; mostly some cleanup and
> rebasing. But it's faster now with the logtape.c changes.

Attaching latest version (combined logtape changes along with main
HashAgg patch).

I believe I've addressed all of the comments, except for Heikki's
question about changing the logtape.c API. I think big changes to the
API (such as Heikki's proposal) are out of scope for this patch,
although I do favor the changes in general. This patch just includes
the LogicalTapeSetExtend() API by Adam Lee, which is less intrusive.

I noticed (and fixed) a small regression for some in-memory hashagg
queries due to the way I was choosing the number of buckets when
creating the hash table. I don't think that it is necessarily worse in
general, but given that there is at least one case of a regression, I
made it more closely match the old behavior, and the regression
disappared.

I improved costing by taking into account the actual number of
partitions and the memory limits, at least for the first pass (in
recursive passes the number of partitions can change).

Aside from that, just some cleanup and rebasing.

Regards,
Jeff Davis