Re: Memory leak with PL/Python trigger

On Fri, Jul 31, 2015 at 12:13 PM, Haribabu Kommi
<kommi(dot)haribabu(at)gmail(dot)com> wrote:
>
>
> On Fri, Jul 31, 2015 at 9:15 AM, Behn, Edward (EBEHN) <EBEHN(at)arinc(dot)com>
> wrote:
>>
>> I believe that I have discovered a memory leak bug that affects PostgreSQL
>> 9.5 Alpha -1. (It also affects 9.4.1.)
>>
>>
>>
>> The leak appears in the server process when a PL/Python trigger function
>> is used to populate a composite type that contains a field that is a domain
>> type. This leak is worse if the domain type has a constraint.
>>
>> The attached files create a database that has this property (Schema.sql)
>> and add rows to the table causing the leak (Test.py).
>>
>>
>>
>> If the trigger function is implemented in PL/pgSQL, the leak disappears.
>>
>>
>>
>> After 1,000,000 rows, the server process has a memory footprint of 544 MiB
>> (~570 bytes/row) if the constraint is present on the domain type.
>>
>>
>>
>> After 1,000,000 rows, the server process has a memory footprint of 144 MiB
>> (~150 bytes/row) if the constraint is not present on the domain type.
>>
>>
>>
>> I am running PostgreSQL 9.5 Alpha-1 on Fedora 21. My installation is “out
>> of the box” from the YUM repository. I have not altered any configuration
>> files. I have tested the code as user/role postgres.
>>
>>
>>
>>
>
> Thanks for the defect details.
> I am able to reproduce the issue on Head.
>
> This is because of many unnecessary "Expr Context" that are created in
> function "domain_check_input"
> using CreateStandaloneExprContext function under TopMemoryContext.
>
> This memory context is reset for future use by storing the context pointer
> in "fcinfo->flinfo->fn_extra".
> But this pointer always set as NULL for every call. Because of this reason
> the memory was leaked.

Adding to this point, In "PLyObject_ToComposite" function the context pointer
which was allocated earlier is freed without removing the context.
Because of this
reason for the next record, it again allocates the context and thus it
leading to a
memory leak.

Instead of reset the context in "domain_check_input" function, we can free that
context, which can solve this problem. But I am not sure, is it right fix?

Instead of the above fix, either we need to teach plpython functions to taking
care of reusing the pointer or something?

Regards,
Hari Babu
Fujitsu Australia