Re: excessive amounts of consumed memory (RSS), triggering OOM killer

On 2.12.2014 00:31, Andrew Dunstan wrote:
>
> On 12/01/2014 05:39 PM, Tomas Vondra wrote:
>> Hi all,
>>
>> while working on the patch decreasing amount of memory consumed by
>> array_agg [1], I've ran into some strange OOM issues. Reproducing them
>> using the attached SQL script is rather simple.
>>
>> [1] https://commitfest.postgresql.org/action/patch_view?id=1652
>>
>> At first I thought there's some rare hidden memory leak, but I'm pretty
>> sure that's not the case. I've even put some explicit counters into
>> aset.c counting allocated/freed blocks, and it seems to be working just
>> fine (and matching the context tree perfectly). So no memory leak.
>
>
> Doesn't this line:
>
> TopMemoryContext: 136614192 total in 16678 blocks; 136005936 free
> (500017 chunks); 608256 used
>
>
> look pretty suspicious?

It certainly looks a bit large, especially considering this is a fresh
cluster with a single DB, containing a single table (created by user),
no doubt about that.

For comparison, this is a new backend:

TopMemoryContext: 78128 total in 11 blocks; 8528 free (16 chunks); 69600
used

OTOH, it's "just" 130MB, and the RSS values are ~6GB when the OOM hits.

For the record, I only tweaked these settings in the config file:

test=# select name, setting from pg_settings where source like
'%configuration file%';
name | setting
----------------------------+-------------------
checkpoint_segments | 32
DateStyle | ISO, DMY
default_text_search_config | pg_catalog.simple
dynamic_shared_memory_type | posix
lc_messages | cs_CZ.UTF-8
lc_monetary | cs_CZ.UTF-8
lc_numeric | cs_CZ.UTF-8
lc_time | cs_CZ.UTF-8
log_timezone | Europe/Prague
maintenance_work_mem | 524288
max_connections | 100
shared_buffers | 65536
TimeZone | Europe/Prague
work_mem | 1048576
(14 rows)

Some of those are set by the initdb, of course.

regards
Tomas