Re: json(b)_array_elements use causes very large memory usage when also referencing entire json document

On 2017-10-06 20:44:20 -0400, Tom Lane wrote:
> Andres Freund <andres(at)anarazel(dot)de> writes:
> > Hm. We've a bunch of places where we free detoasted data, just out of
> > efficiency concerns. And since the jsonb functions are quite likely to
> > detoast a lot, it doesn't seem unreasonable to do so for the most likely
> > offenders. I mean if you've a bit more complex expression involving a
> > few fields accessed, freeing in the accesses will reduce maximum memory
> > usage by quite a bit. I'm not suggesting to work towards leak free, just
> > towards reducing the lifetime of a few potentially large allocations.
>
> Dunno, for the common case of not-so-large values, this would just be
> a net loss. pfree'ing a value we can afford to ignore till the next
> per-tuple context reset is not a win.
>
> Maybe we need some kind of PG_FREE_IF_LARGE_COPY macro? Where it
> would kick in for values over 8K or thereabouts?

Yea, that might be worthwhile.

> (You might argue that any value that got detoasted at all would be
> large enough to be worth worrying about; but I think that falls down
> because we folded short-header unshorting into the detoast mechanism.)

Hm. Unrelated to the topic at hand, but I wonder how much it'd take to
make some of the hot-path jsonb functionality tolerant of unaligned
datums.

Greetings,

Andres Freund