Out of memory: postgres killed after changing "checkoint_completion_target" and reloading postgres configuration file

From: Pierre Slania <pierre(dot)slania(at)lampiris(dot)be>
To: "pgsql-bugs(at)postgresql(dot)org" <pgsql-bugs(at)postgresql(dot)org>
Subject: Out of memory: postgres killed after changing "checkoint_completion_target" and reloading postgres configuration file
Date: 2014-12-08 10:47:26
Message-ID: 3D8F759DF4D90F45A696149A03E61FEB95D6E732@GELATINE.lampiris.local
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Hi,

Problem :
------------
Dec 4 10:42:41 "myserver" kernel: [7604991.147522] Out of memory: Kill process 2904 (postgres) score 217 or sacrifice child

How to cause the error:
-----------------------------
I changed the parameter "checkpoint_completion_target" from 0.5 to 0.9 in the postgresql.conf file unless the production is up and used by multiple users. After that I made a postgres reload the 4th dec 2014 at 10:20AM

(Note that same problem has occurred several weeks ago when I changed "checkpoint_segment" from 3 to 10 in the postgresql.conf file after reloading postgres)

Application that are running
-----------------------------------
AS JBoss 7.1.1.Final with a connection pool

Postgres version running in production
------------------------------------------------
PostgreSQL 9.1.8 on x86_64-unknown-linux-gnu, compiled by gcc-4.4.real (Debian 4.4.5-8) 4.4.5, 64-bit

OS Server production
--------------------------
Linux "myserver" 3.2.0-4-amd64 #1 SMP Debian 3.2.51-1 x86_64 GNU/Linux

The following memories value was done today:

postgres@"myserver":/var/log/postgresql$ free -m
total used free shared buffers cached
Mem: 32115 29498 2617 0 110 20128
-/+ buffers/cache: 9258 22857
Swap: 0 0 0

ð No swap allowed in this server

Observation:
----------------
The database seems to run well from the 4th dec between 10:20AM to 10:42 AM before falling back suddently to out of memory.
It switched to recovery mode and restart automatically.
There were no problem encountered like that before making any change to the checkpoint parameters.

Attached files
-----------------
Syslog file and postgresql.log file are attached. I made some clean up in the postgres log file.

Question:
------------
Is it not recommended to change any "checkpoint_xxx" parameters in the postgresql.conf file when the production is running ?

Kind Regards,
SLANIA Pierre

Attachment Content-Type Size
syslog.log application/octet-stream 32.8 KB
postgresql-9.1-main.log application/octet-stream 3.0 KB
parameters.log application/octet-stream 3.2 KB

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Ondrej Chaloupka 2014-12-08 11:26:46 Incorrect return code for error on call of commit during 2PC
Previous Message zmokdad 2014-12-08 10:39:25 BUG #12181: Low Disk Space