From: | Jeff Davis <pgsql(at)j-davis(dot)com> |
---|---|
To: | Andres Freund <andres(at)anarazel(dot)de>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Justin Pryzby <pryzby(at)telsasoft(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, David Rowley <dgrowleyml(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Melanie Plageman <melanieplageman(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org |
Subject: | Re: Default setting for enable_hashagg_disk |
Date: | 2020-06-25 16:42:33 |
Message-ID: | 3e07d41d5104466cb46e3fd076c47989fe65f085.camel@j-davis.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-docs pgsql-hackers |
On Wed, 2020-06-24 at 12:31 -0700, Andres Freund wrote:
> nodeAgg.c already treats those separately:
>
> void
> hash_agg_set_limits(double hashentrysize, uint64 input_groups, int
> used_bits,
> Size *mem_limit, uint64
> *ngroups_limit,
> int *num_partitions)
> {
> int npartitions;
> Size partition_mem;
>
> /* if not expected to spill, use all of work_mem */
> if (input_groups * hashentrysize < work_mem * 1024L)
> {
> if (num_partitions != NULL)
> *num_partitions = 0;
> *mem_limit = work_mem * 1024L;
> *ngroups_limit = *mem_limit / hashentrysize;
> return;
> }
The reason this code exists is to decide how much of work_mem to set
aside for spilling (each spill partition needs an IO buffer).
The alternative would be to fix the number of partitions before
processing a batch, which didn't seem ideal. Or, we could just ignore
the memory required for IO buffers, like HashJoin.
Granted, this is an example where an underestimate can give an
advantage, but I don't think we want to extend the concept into other
areas.
Regards,
Jeff Davis
From | Date | Subject | |
---|---|---|---|
Next Message | Jeff Davis | 2020-06-25 17:15:52 | Re: Default setting for enable_hashagg_disk |
Previous Message | Andres Freund | 2020-06-25 16:37:46 | Re: Default setting for enable_hashagg_disk |
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2020-06-25 16:43:12 | Re: CUBE_MAX_DIM |
Previous Message | Andres Freund | 2020-06-25 16:37:46 | Re: Default setting for enable_hashagg_disk |