Re: Using ProcSignal to get memory context stats from a running backend

On 20 December 2017 at 08:46, Craig Ringer <craig(at)2ndquadrant(dot)com> wrote:

> On 20 December 2017 at 02:35, Andres Freund <andres(at)anarazel(dot)de> wrote:
>
>
>> > Yeah. But please don't mess with MemoryContextStats per se ---
>> > I dunno about you guys but "call MemoryContextStats(TopMemoryContext)"
>> > is kinda wired into my gdb reflexes. I think what'd make sense
>> > is a new function "MemoryContextStatsTo(context, function_pointer)".
>> > It's okay to redefine the APIs of the per-context-type functions
>> > these would call, though, because nobody calls those functions directly.
>>
>> We already have MemoryContextStatsDetail() - it seems to make sense to
>> expand that API and leave MemoryContextStats() alone. I don't think
>> there's a need for a third variant?
>>
>
> Cool, can do.
>
> I'll have to expose a typedef for the printf-wrapper callback in
> memnodes.h and add it to the stats method, which I thought would be more
> likely to get complaints than the global hook. I'm actually happier to do
> it with a passed callback.
>
> Will revise when I get a chance in the next couple of days.
>

Done.

It uses vfprintf unconditionally, even on Windows where we'd usually use
write_stderr, so it doesn't change the current MemoryContextStats behaviour.

2017-12-21 14:39:20.045 AWST [10588] pg_regress/misc_functions LOG:
diagnostic dump requested; memory context info: TopMemoryContext: 67440
total in 5 blocks; 13376 free (6 chunks); 54064 used
pgstat TabStatusArray lookup hash table: 8192 total in 1 blocks;
1376 free (0 chunks); 6816 used
TopTransactionContext: 8192 total in 1 blocks; 7728 free (1
chunks); 464 used
...

To verify that stderr output still works properly I ran:

SELECT
repeat('spamspamspam', 20000000),
repeat('spamspamspam', 20000000),
repeat('spamspamspam', 20000000),
repeat('spamspamspam', 20000000),
... lots ...;

and got the expected

+ERROR: out of memory
+DETAIL: Failed on request of size 240000004.

and memory context dump to stderr. I didn't add a TAP test for that because
I'd rather avoid exercising OOM in tests where we don't know what the
host's swap config is like, how its ulimit behaves, whether it has
craziness like the OOM killer, etc. But it might make sense to add a TAP
test to set a low ulimit and exercise OOM recovery at some point.

I've added a separate vfprintf wrapper for memory context use that doesn't
try to use the windows error log like write_stderr(...) does. If we want to
change OOM behaviour on Windows it can be done in a followup.

--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services