Re: Out of Memory errors are frustrating as heck!

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Gunther <raj(at)gusw(dot)net>
Cc: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Jeff Janes <jeff(dot)janes(at)gmail(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, pgsql-performance(at)lists(dot)postgresql(dot)org, Justin Pryzby <pryzby(at)telsasoft(dot)com>
Subject: Re: Out of Memory errors are frustrating as heck!
Date: 2019-04-16 02:28:03
Message-ID: 32096.1555381683@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

Gunther <raj(at)gusw(dot)net> writes:
> Tom (BTW, your mail server rejects my direct mail,

[ raised eyebrow ] It's coming through fine AFAICS.

>> I'm pretty sure that's not the droid we're looking for.
>> ExecHashJoinGetSavedTuple does palloc a new tuple, but it immediately
>> sticks it into a TupleTableSlot that will be responsible for freeing
>> it (when the next tuple is stuck into the same slot).

> I did continue a "few times", but few as in a dozen, it's always the same

Well, I still don't believe that ExecHashJoinGetSavedTuple is the issue.
It has a mechanism for freeing the allocation at the right time, and
if that were broken then all hash joins would be leaking. It's easy
to prove that that's not so, both by experiment and by the lack of
other reports.

It's barely conceivable that in your particular query, there's something
acting to break that which doesn't manifest typically; but I think it's
much more likely that you simply haven't found the culprit allocation.
It's quite feasible that many many ExecHashJoinGetSavedTuple calls would
go by in between problem allocations.

Another line of thought is that maybe the problem is with realloc'ing
something larger and larger? You could try trapping AllocSetRealloc
to see.

(BTW, it looks like we *do* have a leak with simple hash joins in
HEAD. But not v11.)

regards, tom lane

In response to

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Alvaro Herrera 2019-04-16 02:39:12 Re: Out of Memory errors are frustrating as heck!
Previous Message Mark Kirkwood 2019-04-16 02:08:45 Re: PostgreSQL upgrade.