Re: Out of Memory errors are frustrating as heck!

From: Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
To: Gunther <raj(at)gusw(dot)net>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Justin Pryzby <pryzby(at)telsasoft(dot)com>, Jeff Janes <jeff(dot)janes(at)gmail(dot)com>, pgsql-performance(at)lists(dot)postgresql(dot)org, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
Subject: Re: Out of Memory errors are frustrating as heck!
Date: 2019-10-06 21:06:18
Message-ID: 20191006210618.vvt7hvuqw4ygv6u6@development
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

On Sat, Aug 24, 2019 at 11:40:09AM -0400, Gunther wrote:
>Thanks Tom, yes I'd say it's using a lot of memory, but wouldn't call
>it "leak" as it doesn't grow during the 30 min or so that this query
>runs. It explodes to 4GB and then stays flat until done.
>

Well, the memory context stats you've shared however show this:

total: 1345345736 bytes in 1209 blocks; 4529600 free (270 chunks); 1340816136 used

That's only ~1.3GB, and ~1.1GB of that is the expression context. So
when you say 4GB, when does that happen and can you share stats showing
state at that point?

>Yes, and this time the query is super complicated with many joins and
>tables involved. The query plan has 100 lines. Not easy to share for
>reproduce and I have my issue under control by adding some swap just
>in case. The swap space was never actually used.
>

Still, without the query plan we can hardly do any guesses about what
might be the issue.

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

In response to

Browse pgsql-performance by date

  From Date Subject
Next Message Tomas Vondra 2019-10-06 21:11:31 Re: Out of Memory errors are frustrating as heck!
Previous Message Tomas Vondra 2019-10-06 20:54:52 Re: Delete huge Table under XFS