Re: json(b)_array_elements use causes very large memory usage when also referencing entire json document

On 2017-10-06 20:28:21 -0400, Tom Lane wrote:
> Andres Freund <andres(at)anarazel(dot)de> writes:
> > Yea, that should be quite doable, slightly annoying to need more than
> > one expr context, but that seems unavoidable. Should even be doable for
> > SFRM_ValuePerCall? SFRM_Materialize isn't a "central" problem, given it
> > properly manages memory for the tuplestore and filling the tuplestore is
> > an internal matter.
>
> I'm not really convinced whether there's an issue for ValuePerCall SRFs.
> We've not had complaints about them, and I would think we would if that
> feature alone would create an issue. But I've not thought about it hard,
> nor tested.

I've just played around with this. ValuePerCall SRFs are fine with
called in a short-lived context (they're required to be able to, as
documented in xfunc.sgml), so is SFRM_Materialize. The only thing to be
careful about is the *arguments* to the function, those need to live
long enough in the ValuePerCall case.

I'm not entirely sure what the best way to code the memory allocation
is. I've just prototyped this by switching to
econtext->ecxt_per_query_memory around ExecEvalFuncArgs, and resetting
econtext->ecxt_per_tuple_memory before every ExecProjectSRF call. But
obviously we don't want to leak the arguments this way. Easiest seems
to be to just pass in a separate econtext for the arguments, and reset
that explicitly. Unless you've a better suggestion, I'll clean up my
hack and propose it.

Greetings,

Andres Freund