Re: json(b)_array_elements use causes very large memory usage when also referencing entire json document

From: Andres Freund <andres(at)anarazel(dot)de>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Lucas Fairchild-Madar <lucas(dot)madar(at)gmail(dot)com>, pgsql-bugs(at)postgresql(dot)org
Subject: Re: json(b)_array_elements use causes very large memory usage when also referencing entire json document
Date: 2017-10-07 00:21:16
Message-ID: 20171007002116.2fqr6z26byhsjwoe@alap3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

On 2017-10-06 15:37:03 -0400, Tom Lane wrote:
> So a leak would occur in any case ... but it's particularly awful in
> this case, because data->'id' involves detoasting the rather wide
> value of "data", which is then promptly leaked.

While not tackling the problem in a systematic manner, it seems like
it'd be a good idea to free the detoasted column in this and related
functions?

> As of v10, it might be possible to fix this for the tlist case
> as well, by doing something like using a separate short-lived
> context for the non-SRF tlist items.

Yea, that should be quite doable, slightly annoying to need more than
one expr context, but that seems unavoidable. Should even be doable for
SFRM_ValuePerCall? SFRM_Materialize isn't a "central" problem, given it
properly manages memory for the tuplestore and filling the tuplestore is
an internal matter.

Greetings,

Andres Freund

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Tom Lane 2017-10-07 00:28:21 Re: json(b)_array_elements use causes very large memory usage when also referencing entire json document
Previous Message Tom Lane 2017-10-06 23:21:10 Re: pg_logical_slot_peek_changes crashes postgres when called from inside pl/pgsql