Re: BUG #15321: XMLTABLE leaks memory like crazy

From: Andrew Gierth <andrew(at)tao11(dot)riddles(dot)org(dot)uk>
To: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
Cc: pgsql-bugs(at)lists(dot)postgresql(dot)org, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
Subject: Re: BUG #15321: XMLTABLE leaks memory like crazy
Date: 2018-08-12 10:25:39
Message-ID: 87bma77v67.fsf@news-spur.riddles.org.uk
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

>>>>> "Pavel" == Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com> writes:

>> No, tuplestore_putvalues is only responsible for the memory it
>> allocates itself, which belongs to the tuplestore, it has nothing to
>> do with the memory allocated by its caller - and XmlTableGetValue
>> does quite a few allocations.

Pavel> Maybe I used wrong words. Sorry, my English lang is not good.

Pavel> In master tfuncLoadRows switch to tstate->perValueCxt. You
Pavel> change it by switch econtext->ecxt_per_tuple_memory what is
Pavel> wrong I thing.

Pavel> If we don't change memory context, we will stay inside
Pavel> perValueCxt. And this context will be cleaned outside.

Yes, but it won't be cleaned up until _all_ the rows from a single call
have been generated, which means that the allocations from GetValue have
been (uselessly) accumulating during this time, wasting memory.

We do need a context that is reset for every output row, as perValueCxt
used to be. I don't see why this is even in question.

>> Before, you were using perValueCxt and resetting it once per GetValue
>> call. Since my patch takes perValueCxt to use for another purpose
>> instead, it needs to be replaced, and econtext->ecxt_per_tuple_memory
>> is suitable for the job (and consistent with functionscan).

Pavel> I think so using cxt_per_tuple_memory is not necessary - and my
Pavel> patch is working too.

Working, just using more memory than it needs to.

Pavel> Just I removed MemoryContextReset(tstate->perValueCxt) after
Pavel> tuplestore_putvalues. It is possible, because
Pavel> tstate->perValueCxt is cleaned immediately in caller
Pavel> tfuncFetchRows function.

But that's not "immediately" because tfuncLoadRows is looping over the
FetchRow call, and calling GetValue for each column in that row, and in
your version it is _not cleaning up memory in that loop_.

--
Andrew (irc:RhodiumToad)

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Pavel Stehule 2018-08-12 10:39:44 Re: BUG #15321: XMLTABLE leaks memory like crazy
Previous Message Pavel Stehule 2018-08-12 10:02:31 Re: BUG #15321: XMLTABLE leaks memory like crazy