From: | Andres Freund <andres(at)anarazel(dot)de> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | Lucas Fairchild-Madar <lucas(dot)madar(at)gmail(dot)com>, pgsql-bugs(at)postgresql(dot)org |
Subject: | Re: json(b)_array_elements use causes very large memory usage when also referencing entire json document |
Date: | 2017-10-07 04:08:43 |
Message-ID: | 20171007040843.zqf4oerahz5mxrp3@alap3.anarazel.de |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-bugs |
On 2017-10-06 19:42:05 -0700, Andres Freund wrote:
> On 2017-10-06 22:35:37 -0400, Tom Lane wrote:
> > Andres Freund <andres(at)anarazel(dot)de> writes:
> > > I've just played around with this. ValuePerCall SRFs are fine with
> > > called in a short-lived context (they're required to be able to, as
> > > documented in xfunc.sgml), so is SFRM_Materialize. The only thing to be
> > > careful about is the *arguments* to the function, those need to live
> > > long enough in the ValuePerCall case.
> >
> > Isn't there already code to deal with that? See around line 175
> > in execSRF.c.
>
> Well, that's for nodeFunctionscan.c, not nodeProjectSet.c. But it seems
> quite sensible to model this very similarly.
Patch attached. The business with having to switch the memory context
tuplestore_gettupleslot() ain't pretty, but imo is tolerable.
I'm kinda tempted to put this, after a bit more testing, into v10.
After this Lucas' testcase doesn't leak memory anymore - it's still slow
in execution, decompressing the same datum over and over. But that feels
like something that should be fixed separately.
- Andres
Attachment | Content-Type | Size |
---|---|---|
0001-Reduce-memory-usage-of-targetlist-SRFs.patch | text/x-diff | 6.9 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Ben Chobot | 2017-10-07 04:24:34 | Re: pg_logical_slot_peek_changes crashes postgres when called from inside pl/pgsql |
Previous Message | Andres Freund | 2017-10-07 02:42:05 | Re: json(b)_array_elements use causes very large memory usage when also referencing entire json document |