| From: | Julien Rouhaud <julien(dot)rouhaud(at)dalibo(dot)com> |
|---|---|
| To: | pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
| Cc: | Marc Cousin <marc(dot)cousin(at)dalibo(dot)com> |
| Subject: | Memory leak in GIN index build |
| Date: | 2016-04-16 17:31:09 |
| Message-ID: | 571276DD.5050303@dalibo.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hello,
Another colleague provided a report of memory leak, during a GIN index
build. Test case to reproduce the attached (need to create a gin index
on the val column after loading). Sorry, it generates a 24GB table, and
memory start leaking with a 1GB maintenance_work_mem after reaching 8 or
9 times the m_w_m setting, leading to ~ 9GB used in Gin build temporary
context.
After some digging, the leak comes from walbufbegin palloc in
registerLeafRecompressWALData().
IIUC, walbufbegin isn't pfree-d and can't be before XLogInsert() is
called, which happens in ginPlaceToPage().
I don't see a simple way to fix that. My first idea would be to change
GinBtreeData's placeToPage (and all other needed) functions signature to
pass a pointer to pfree in ginPlaceToPage() if needed, but it doesn't
really seem appealing. In any case, I'd be happy to work on a patch if
needed.
Also, in dataPlaceToPageLeaf() and ginVacuumPostingTreeLeaf(), shouldn't
the START_CRIT_SECTION() calls be placed before the xlog code?
Regards.
--
Julien Rouhaud
http://dalibo.com - http://dalibo.org
| Attachment | Content-Type | Size |
|---|---|---|
| generate_data.pl | application/x-perl | 411 bytes |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tom Lane | 2016-04-16 18:45:07 | Re: Memory leak in GIN index build |
| Previous Message | Tom Lane | 2016-04-16 17:06:36 | Spinlocks and semaphores in 9.2 and 9.3 |