From: | Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | pgsql-hackers(at)postgresql(dot)org, Stefan Kaltenbrunner <stefan(at)kaltenbrunner(dot)cc> |
Subject: | Re: (full) Memory context dump considered harmful |
Date: | 2015-08-22 04:25:50 |
Message-ID: | 55D7F9CE.3040904@2ndquadrant.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On 08/21/2015 08:37 PM, Tom Lane wrote:
> Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com> writes:
>>
>> I also don't think logging just subset of the stats is a lost case.
>> Sure, we can't know which of the lines are important, but for example
>> logging just the top-level contexts with a summary of the child contexts
>> would be OK.
>
> I thought a bit more about this. Generally, what you want to know about
> a given situation is which contexts have a whole lot of allocations
> and/or a whole lot of child contexts. What you suggest above won't work
> very well if the problem is buried more than about two levels down in
> the context tree. But suppose we add a parameter to memory context stats
> collection that is the maximum number of child contexts to print *per
> parent context*. If there are more than that, summarize the rest as per
> your suggestion. So any given recursion level might look like
>
> FooContext: m total in n blocks ...
> ChildContext1: m total in n blocks ...
> possible grandchildren...
> ChildContext2: m total in n blocks ...
> possible grandchildren...
> ChildContext3: m total in n blocks ...
> possible grandchildren...
> k more child contexts containing m total in n blocks ...
>
> This would require a fixed amount of extra state per recursion level,
> so it could be done with a few more parameters/local variables in
> MemoryContextStats and no need to risk a malloc().
>
> The case where you would lose important data is where the serious
> bloat is in some specific child context that is after the first N
> children of its direct parent. I don't believe I've ever seen a case
> where that was critical information as long as N isn't too tiny.
Couldn't we make it a bit smarter to handle even cases like this? For
example we might first count/sum the child contexts, and then print
either all child contexts (if there are only a few of them) or just
those that are >5% of the total, 2x the average or something like that.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
From | Date | Subject | |
---|---|---|---|
Next Message | Amit Kapila | 2015-08-22 04:33:51 | Re: Make HeapTupleSatisfiesMVCC more concurrent |
Previous Message | Jim Nasby | 2015-08-22 03:34:21 | Re: Declarative partitioning |