From: | Damiano Albani <damiano(dot)albani(at)gmail(dot)com> |
---|---|
To: | psycopg(at)postgresql(dot)org |
Subject: | Understanding memory usage |
Date: | 2013-10-28 21:28:56 |
Message-ID: | CAKys952SOUe+Xu+0UOUTcGBfQHZP_rKGfNKc_4EBk1hkTDQpJg@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | psycopg |
Hello,
I'm trying to understand how Psycopg behaves memory-wise.
Basically, in my use case, it looks like memory is not released as I
thought it would be.
I've taken the script mentioned in bug report
#78<http://psycopg.lighthouseapp.com/projects/62710/tickets/78-memory-issue>and
run on my system (Ubuntu 12.04/x64, PostgreSQL 9.3, Python 2.7.3,
Psycopg 2.5.1).
Using the original query:
SELECT pagecontent FROM test WHERE siteid = 3;
loaded 0
VmSize: 3251264 kB
cleared 0
VmSize: 112924 kB
loaded 1
VmSize: 3250496 kB
cleared 1
VmSize: 112928 kB
So far, so good -- incidentally, the lowest memory usage seems to 112 MB,
where it is reported at 78 MB in bug report #78. Any idea why?
Now, using this very simple query (column1 being a NUMBER, around 2
millions rows):
SELECT column1 FROM table1
loaded 0
VmSize: 259392 kB
cleared 0
VmSize: 232960 kB
loaded 1
VmSize: 259392 kB
cleared 1
VmSize: 226676 kB
So why isn't the memory reclaimed as in the first case?
I've also done some tests where the fetchall() call is commented
altogether. The results are :
SELECT pagecontent FROM test WHERE siteid = 3;
loaded 0
VmSize: 2226860 kB
cleared 0
VmSize: 112920 kB
loaded 1
VmSize: 2226488 kB
cleared 1
VmSize: 112924 kB
SELECT column1 FROM table1
loaded 0
VmSize: 159772 kB
cleared 0
VmSize: 113036 kB
loaded 1
VmSize: 162356 kB
cleared 1
VmSize: 112920 kB
In that case, the memory is effectively reclaimed for the second query --
getting to the lowest 112 MB number I have mentioned before.
But what is the reason why Psycopg needs memory to just run a query, *
without* fetching results?
Thanks for your help!
--
Damiano Albani
From | Date | Subject | |
---|---|---|---|
Next Message | Daniele Varrazzo | 2013-10-28 23:23:11 | Re: Understanding memory usage |
Previous Message | Peter Eisentraut | 2013-10-17 19:40:58 | Re: psycopg2 doesn't use LDFLAGS from pg_config |