Re: BUG #16112: large, unexpected memory consumption

Hi,

On 2019-11-13 17:53:28 -0500, Jeff Janes wrote:
> On Wed, Nov 13, 2019 at 9:50 AM Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
> wrote:
>
> >
> > Yeah, I can reproduce this pretty easily. It seems like a memory leak in
> > ExecMakeTableFunctionResult. a9c35cf85ca reworked FunctionCallInfo to be
> > variable-length, but it gets allocated in ExecutorState context directly
> > and so until the end of the query.
> >
>
> I find the leak was introduced much earlier than that, in:
>
> commit 763f2edd92095b1ca2f4476da073a28505c13820
> Author: Andres Freund <andres(at)anarazel(dot)de>
> Date: Thu Nov 15 14:26:14 2018 -0800
>
> Rejigger materializing and fetching a HeapTuple from a slot.
>
> I have no idea if this info is useful to informing the best solution,
> though.
>
> You patch applied to REL_12_STABLE does fix it for me.

I think that's likely two overlapping issues.

Possibly
commit 88e6ad3054ddd5aa0dee12e5def2c335fe92a414
Author: Andres Freund <andres(at)anarazel(dot)de>
Date: 2019-04-19 11:33:37 -0700

Fix two memory leaks around force-storing tuples in slots.

> > The attached trivial patch fixes that by adding a pfree() at the end of
> > the function. I wonder if we have the same issue elsewhere ...
> >
> >
> Is there an easy way to assess if the "make check" regression tests are
> taking more memory than they used to?

Not easily, I think. I also suspect that you'd not have seen a
meaningful increase in memory usage due to this bug - it "only" was
noticable for the query at hand, because the FunctionScan node was
reached so many times, due to the lateral join forcing it to be scanned
once for each value in the source temp table. I don't think we have a
query in the tests doing so often enough (partially because we don't,
partially because it'd take a lot of time on slow machines).

Greetings,

Andres Freund