Re: Service outage: each postgres process use the exact amount of the configured work_mem

From: Moreno Andreo <moreno(dot)andreo(at)evolu-s(dot)it>
To: Melvin Davidson <melvin6925(at)gmail(dot)com>
Cc: "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org>
Subject: Re: Service outage: each postgres process use the exact amount of the configured work_mem
Date: 2017-04-14 18:28:43
Message-ID: 6218eb9b-0c94-97f3-1633-5c7f5a85bfc7@evolu-s.it
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Melvin,<br>
    Sorry for top-posting.    <br>
I'm going ahead in troubleshooting. As Jeff said, there's probably
nothing wrong with my values (at the end of the message you can
find minimal info you requested).<br>
I tried running some queries against psql server and response
times are good, so I'm moving my attentions to Windows server,
which hosts a WCF service, that is the one that actually server
customers.<br>
<br>
Thanks for now<br>
Moreno<br>
<br>
Il 14/04/2017 20:01, Melvin Davidson ha scritto:<br>
</div>
<blockquote
cite="mid:CANu8FixXw3g-fuuPRtyWjDywpe+w+TpO=VfcySo8JXSEOHRmvA(at)mail(dot)gmail(dot)com"
type="cite">
<div dir="ltr">
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Apr 14, 2017 at 1:50 PM,
Moreno Andreo <span dir="ltr">&lt;<a moz-do-not-send="true"
href="mailto:moreno(dot)andreo(at)evolu-s(dot)it" target="_blank">moreno(dot)andreo(at)evolu-s(dot)it</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<div class="gmail-m_-6686708562138806043moz-cite-prefix">Sorry,<br>
 my mistake (I'm a bit nervous...)<br>
<br>
that's not work_mem, but shared_buffers<br>
<br>
Thanks
<div>
<div class="gmail-h5"><br>
<br>
Il 14/04/2017 19:33, Melvin Davidson ha scritto:<br>
</div>
</div>
</div>
<div>
<div class="gmail-h5">
<blockquote type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Apr 14, 2017
at 1:12 PM, Moreno Andreo <span dir="ltr">&lt;<a
moz-do-not-send="true"
href="mailto:moreno(dot)andreo(at)evolu-s(dot)it"
target="_blank">moreno(dot)andreo(at)evolu-s(dot)it</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> Hi all,<br>
About 2 hours and half ago, suddenly
(and on the late afternoon of the Easter
Friday), customers reported failing
connections to our server, or even very
slow.<br>
After a bit of checking (that also
involved server reboot) I noticed (using
top) that every process regarding
postgres is using exactly the amout I
configured as work_mem (3 GB). And AFAIK
it's not good.<br>
<br>
30085 postgres 20 0 <b>3370048</b>
156656 153876 S 6.7 0.3 0:00.20 postgres<br>
29833 postgres 20 0 <b>3370000</b>
65260 62416 S 1.7 0.1 0:00.17 postgres <br>
29632 postgres 20 0 <b>3372468</b>
11712 6028 S 0.7 0.0 0:00.60 postgres <br>
<br>
What can be happened?<br>
Nothing has been touched....<br>
postgresql 9.5.6 on debian 8 just
apt-get upgrade'd<br>
<br>
Any help would be appreciated.<span
class="gmail-m_-6686708562138806043gmail-HOEnZb"><font
color="#888888"><br>
Moreno.<br>
</font></span></div>
</blockquote>
</div>
<br>
<b>&gt;using exactly the amout I configured as
work_mem (3 GB).<br>
<br>
</b></div>
<div class="gmail_extra"><b>You are right, that
is bad, but that is your own fault. 3GB of
work_mem is very bad, Try lowing in to
something more reasonable, like 20GB.<br>
<br clear="all">
</b></div>
<div class="gmail_extra"><b><a
moz-do-not-send="true"
href="https://www.postgresql.org/docs/9.5/static/runtime-config-resource.html#RUNTIME-CONFIG-RESOURCE-MEMORY"
target="_blank">https://www.postgresql.org/<wbr>docs/9.5/static/runtime-<wbr>config-resource.html#RUNTIME-<wbr>CONFIG-RESOURCE-MEMORY</a><br>
"several running sessions could be doing
such operations concurrently. <span
style="color:rgb(255,0,0)">Therefore, the
total memory used could be many times the
value of <tt
class="gmail-m_-6686708562138806043gmail-VARNAME">work_mem</tt>;</span>
it is necessary to keep this fact in mind
when choosing the value."</b></div>
<div class="gmail_extra"><br>
-- <br>
<div
class="gmail-m_-6686708562138806043gmail_signature">
<div dir="ltr"><font size="4"><b><span
style="font-family:courier
new,monospace">Melvin Davidson</span></b></font><br>
<font style="font-weight:bold" size="3"><span
style="color:rgb(128,0,255)">I reserve
the right to fantasize.  Whether or
not you </span><br
style="color:rgb(128,0,255)">
<span style="color:rgb(128,0,255)">wish
to share my fantasy is entirely up to
you. </span><img
moz-do-not-send="true" style="color:
rgb(128, 0, 255);"
src="http://us.i1.yimg.com/us.yimg.com/i/mesg/tsmileys2/01.gif"></font><br>
</div>
</div>
</div>
</div>
</blockquote>
<p><br>
</p>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<div>
<div>
<div>Moreno,<br>
</div>
we are working with minimal information here.<br>
</div>
Perhaps if you provided us with the following information it
would be more useful,<br>
<br>
</div>
A. Total <u>SYSTEM MEMORY</u><br clear="all">
</div>
</div>
</blockquote>
52 GB<br>
<blockquote
cite="mid:CANu8FixXw3g-fuuPRtyWjDywpe+w+TpO=VfcySo8JXSEOHRmvA(at)mail(dot)gmail(dot)com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">B. shared_memory<br>
</div>
</div>
</blockquote>
3 GB (was 13 GB)<br>
<blockquote
cite="mid:CANu8FixXw3g-fuuPRtyWjDywpe+w+TpO=VfcySo8JXSEOHRmvA(at)mail(dot)gmail(dot)com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">C. work_memory<br>
</div>
</div>
</blockquote>
default<br>
<blockquote
cite="mid:CANu8FixXw3g-fuuPRtyWjDywpe+w+TpO=VfcySo8JXSEOHRmvA(at)mail(dot)gmail(dot)com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">D. max_connections<br>
</div>
</div>
</blockquote>
1000<br>
<blockquote
cite="mid:CANu8FixXw3g-fuuPRtyWjDywpe+w+TpO=VfcySo8JXSEOHRmvA(at)mail(dot)gmail(dot)com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">E. How many users were connected when
the problem occurred?<br>
</div>
</div>
</blockquote>
About 350 connections<br>
<br>
Thanks<br>
<br>
<blockquote
cite="mid:CANu8FixXw3g-fuuPRtyWjDywpe+w+TpO=VfcySo8JXSEOHRmvA(at)mail(dot)gmail(dot)com"
type="cite">
<div dir="ltr">
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">-- <br>
<div class="gmail_signature">
<div dir="ltr"><font size="4"><b><span
style="font-family:courier new,monospace">Melvin
Davidson</span></b></font><br>
<font style="font-weight:bold" size="3"><span
style="color:rgb(128,0,255)">I reserve the right to
fantasize.  Whether or not you </span><br
style="color:rgb(128,0,255)">
<span style="color:rgb(128,0,255)">wish to share my
fantasy is entirely up to you. </span><img
moz-do-not-send="true" style="color: rgb(128, 0,
255);"
src="http://us.i1.yimg.com/us.yimg.com/i/mesg/tsmileys2/01.gif"></font><br>
</div>
</div>
</div>
</div>
</blockquote>
<p><br>
</p>
</body>
</html>

Attachment Content-Type Size
unknown_filename text/html 10.2 KB

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Melvin Davidson 2017-04-14 18:36:57 Re: Service outage: each postgres process use the exact amount of the configured work_mem
Previous Message Vincent Elschot 2017-04-14 18:23:09 Re: dynamic schema modeling and performance