Re: Out of Memory errors are frustrating as heck!

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Gunther <raj(at)gusw(dot)net>
Cc: pgsql-performance(at)lists(dot)postgresql(dot)org, Jeff Janes <jeff(dot)janes(at)gmail(dot)com>, Justin Pryzby <pryzby(at)telsasoft(dot)com>
Subject: Re: Out of Memory errors are frustrating as heck!
Date: 2019-04-15 03:24:59
Message-ID: 12064.1555298699@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

Gunther <raj(at)gusw(dot)net> writes:
> ExecutorState: 2234123384 total in 266261 blocks; 3782328 free (17244 chunks); 2230341056 used

Oooh, that looks like a memory leak right enough. The ExecutorState
should not get that big for any reasonable query.

Your error and stack trace show a failure in HashBatchContext,
which is probably the last of these four:

> HashBatchContext: 57432 total in 3 blocks; 16072 free (6 chunks); 41360 used
> HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
> HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
> HashBatchContext: 100711712 total in 3065 blocks; 7936 free (0 chunks); 100703776 used

Perhaps that's more than it should be, but it's silly to obsess over 100M
when there's a 2.2G problem elsewhere. I think it's likely that it was
just coincidence that the failure happened right there. Unfortunately,
that leaves us with no info about where the actual leak is coming from.

The memory map shows that there were three sorts and four hashes going
on, so I'm not sure I believe that this corresponds to the query plan
you showed us before.

Any chance of extracting a self-contained test case that reproduces this?

regards, tom lane

In response to

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Gunther 2019-04-15 03:59:45 Re: Out of Memory errors are frustrating as heck!
Previous Message Gunther 2019-04-15 03:04:18 Re: Out of Memory errors are frustrating as heck!