| From: | Jeff Davis <pgsql(at)j-davis(dot)com> |
|---|---|
| To: | pgsql-hackers(at)postgresql(dot)org |
| Cc: | Andres Freund <andres(at)anarazel(dot)de> |
| Subject: | AllocSetEstimateChunkSpace() |
| Date: | 2020-03-25 01:12:03 |
| Message-ID: | e74ab197868a0095edafe3bdcacf632ddb130a52.camel@j-davis.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Attached is a small patch that introduces a simple function,
AllocSetEstimateChunkSpace(), and uses it to improve the estimate for
the size of a hash entry for hash aggregation.
Getting reasonable estimates for the hash entry size (including the
TupleHashEntryData, the group key tuple, the per-group state, and by-
ref transition values) is important for multiple reasons:
* It helps the planner know when hash aggregation is likely to spill,
and how to cost it.
* It helps hash aggregation regulate itself by setting a group limit
(separate from the memory limit) to account for growing by-ref
transition values.
* It helps choose a good initial size for the hash table. Too large of
a hash table will crowd out space that could be used for the group keys
or the per-group state.
Each group requires up to three palloc chunks: one for the group key
tuple, one for the per-group states, and one for a by-ref transition
value. Each chunk can have a lot of overhead: in addition to the chunk
header (16 bytes overhead), it also rounds the value up to a power of
two (~25% overhead). So, it's important to account for this chunk
overhead.
I considered making it a method of a memory context, but the planner
will call this function before the hash agg memory context is created.
It seems to make more sense for this to just be an AllocSet-specific
function for now.
Regards,
Jeff Davis
| Attachment | Content-Type | Size |
|---|---|---|
| estimatechunkspace.patch | text/x-patch | 3.3 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | James Coleman | 2020-03-25 02:08:54 | Re: [PATCH] Incremental sort (was: PoC: Partial sort) |
| Previous Message | James Coleman | 2020-03-25 00:58:31 | Re: [PATCH] Incremental sort (was: PoC: Partial sort) |