From: | Julien Rouhaud <julien(dot)rouhaud(at)dalibo(dot)com> |
---|---|
To: | pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Cc: | Marc Cousin <marc(dot)cousin(at)dalibo(dot)com> |
Subject: | Memory leak in GIN index build |
Date: | 2016-04-16 17:31:09 |
Message-ID: | 571276DD.5050303@dalibo.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Hello,
Another colleague provided a report of memory leak, during a GIN index
build. Test case to reproduce the attached (need to create a gin index
on the val column after loading). Sorry, it generates a 24GB table, and
memory start leaking with a 1GB maintenance_work_mem after reaching 8 or
9 times the m_w_m setting, leading to ~ 9GB used in Gin build temporary
context.
After some digging, the leak comes from walbufbegin palloc in
registerLeafRecompressWALData().
IIUC, walbufbegin isn't pfree-d and can't be before XLogInsert() is
called, which happens in ginPlaceToPage().
I don't see a simple way to fix that. My first idea would be to change
GinBtreeData's placeToPage (and all other needed) functions signature to
pass a pointer to pfree in ginPlaceToPage() if needed, but it doesn't
really seem appealing. In any case, I'd be happy to work on a patch if
needed.
Also, in dataPlaceToPageLeaf() and ginVacuumPostingTreeLeaf(), shouldn't
the START_CRIT_SECTION() calls be placed before the xlog code?
Regards.
--
Julien Rouhaud
http://dalibo.com - http://dalibo.org
Attachment | Content-Type | Size |
---|---|---|
generate_data.pl | application/x-perl | 411 bytes |
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2016-04-16 18:45:07 | Re: Memory leak in GIN index build |
Previous Message | Tom Lane | 2016-04-16 17:06:36 | Spinlocks and semaphores in 9.2 and 9.3 |