From: | Mat Arye <mat(at)timescale(dot)com> |
---|---|
To: | pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Memory leak in plpython3u (with testcase and patch) |
Date: | 2025-01-10 18:43:00 |
Message-ID: | CADsUR0DvVgnZYWwnmKRK65MZg7YLUSTDLV61qdnrwtrAJgU6xw@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hi all,
I found a memory leak in plpython3u. It happens when converting the
argument for an spi plan execution (via plpy.execute() or similar). I've
attached a sql file to create two plpython3u functions to reproduce the
issue (memory_leak_test.sql).
When running
```
set client_min_messages = 'debug';
select test_mem(1000);
```
You'll see that the memory goes to 720MB.
In contrast the test_mem_query function you won't see the leak (it uses
~70MB):
```
set client_min_messages = 'debug';
select test_mem_query(1000);
```
This is because test_mem_query executes the query using a sql string
instead of a spi plan with parameters.
I believe the issue is in the call to `PLy_output_convert` inside
`PLy_spi_execute_plan`. I've attached a patch that reduces the memory usage
to ~70MB from ~720 MB when using test_mem. I based what I did on what
`PLy_input_convert` does. But I am not an expert in this part of the code
so I am not sure the patch is correct. In particular, I don't fully grasp
the comment in `PLy_input_convert` about why the scratch is reset before
not after the conversion cycle and what that has to do with python
refcounts. A review by an expert would be appreciated.
Thanks,
Mat
--
Mat Arye, Technical lead of all thing AI @ Timescale
Attachment | Content-Type | Size |
---|---|---|
memory_leak_test.sql | application/octet-stream | 2.5 KB |
memory_leak.patch | application/octet-stream | 1.0 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Andres Freund | 2025-01-10 18:43:14 | Re: Reorder shutdown sequence, to flush pgstats later |
Previous Message | Andres Freund | 2025-01-10 18:42:11 | Re: Reorder shutdown sequence, to flush pgstats later |