Re: Memory Accounting v11

On 14 June 2015 at 23:51, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com> wrote:

> The current state, where HashAgg just blows up the memory, is just not
>>> reasonable, and we need to track the memory to fix that problem.
>>>
>>
>> Meh. HashAgg could track its memory usage without loading the entire
>> system with a penalty.
>>
>
> +1 to a solution like that, although I don't think that's doable without
> digging the info from memory contexts somehow.
>
>>
I am sorry to ask questions unrelated to the subject, but how is tracking
memory going to fix the HashAgg blow up problem? Is there a plan to make
HashAgg not blow up (i.e. spill the hash table)?

Thanks,
-cktan

On Thu, Jul 2, 2015 at 4:19 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:

> On 14 June 2015 at 23:51, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
> wrote:
>
>
>> The current state, where HashAgg just blows up the memory, is just not
>>>> reasonable, and we need to track the memory to fix that problem.
>>>>
>>>
>>> Meh. HashAgg could track its memory usage without loading the entire
>>> system with a penalty.
>>>
>>
>> +1 to a solution like that, although I don't think that's doable without
>> digging the info from memory contexts somehow.
>>
>>>
> Jeff is right, we desperately need a solution and this is the place to
> start.
>
> Tom's concern remains valid: we must not load the entire system with a
> penalty.
>
>
> The only questions I have are:
>
> * If the memory allocations adapt to the usage pattern, then we expect to
> see few memory chunk allocations. Why are we expecting "the entire system"
> to experience a penalty?
>
> * If we do not manage our resources, how are we certain this does not
> induce a penalty? Not tracking memory could be worse than tracking it.
>
> --
> Simon Riggs http://www.2ndQuadrant.com/
> <http://www.2ndquadrant.com/>
> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
>