Calculating mem requirements for parallel queries and Docker and coredumps!

From: Jorge Torralba <jorge(dot)torralba(at)gmail(dot)com>
To: pgsql-admin(at)postgresql(dot)org
Subject: Calculating mem requirements for parallel queries and Docker and coredumps!
Date: 2019-02-11 17:30:33
Message-ID: CACut7uQNzP0RfTcoiEeCe8O59cTKf9XwDo=E-rxBD2rkNW5s9A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

So this weekend we ran into the the docker limit of 64m for /dev/shm in one
of our highest volume db's in production which we just upgraded to 11 from
9.5. The typical ERROR: could not resize shared memory segment .... was
flooding our log files and we were core dumping constantly. We had to set
max_parallel_workers_per = 0 in order to disable the feature. This resolved
the problem but . took away a feature were looking forward to use.

Our system is 44 cores with 300 gigs of ram, 75 gig shared_buffers, 250mb
work_mem and 600 max _connections although we use 270 consistent connection
remainders are for redeploy that consume additional connections while
others expire.

Having said all this, the parallel query options where all default values
and we were brought to our knees.

So, I cannot find any documentation anywhere that would help me calculate
the size for shm_size and have docker started with that value. Are there
any tios or formula to help with this ?

Lastly, is there a way to turn off postgres core dump on segfaults ? Every
time we generated the could not resize error, it generated a core dump
which killed IO on the system.

Thanks

Thanks,

Jorge Torralba
----------------------------

Note: This communication may contain privileged or other confidential
information. If you are not the intended recipient, please do not print,
copy, retransmit, disseminate or otherwise use the information. Please
indicate to the sender that you have received this email in error and
delete the copy you received. Thank You.

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Pavan Kumar 2019-02-11 18:37:32 installation of postgres 10.6
Previous Message Guillaume Lelarge 2019-02-11 11:16:10 Re: [Junk released by User action] Re: [Junk released by User action] Re: [External] Postgres Database Disk Usage