Memory leak in XLOG reader facility when decoding records (caused by WAL refactoring)

From: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
To: PostgreSQL mailing lists <pgsql-bugs(at)postgresql(dot)org>
Cc: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>
Subject: Memory leak in XLOG reader facility when decoding records (caused by WAL refactoring)
Date: 2015-07-23 13:50:12
Message-ID: CAB7nPqQFqCDXX_R9qde3bjmzXVvuYnnXgTf-u5K3oAqk0VbG0Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Hi all,

While running valgring on pg_rewind, I have noticed that each new call
to XLogReadRecord leaks XLogReaderState->main_data and
XLogReaderState->blocks[block_id].data because each one of them is
palloc'd to store respectively the main data of the record and the
data attached to each block. I think that the most correct fix to
prevent the leak is to free those pointers when calling ResetDecoder()
which is used to reset a reader state between two records.
==46805== 154 bytes in 1 blocks are definitely lost in loss record 103 of 167
==46805== at 0x6B1B: malloc (in
/usr/local/Cellar/valgrind/3.10.1/lib/valgrind/vgpreload_memcheck-amd64-darwin.so)
==46805== by 0x10000DB2D: pg_malloc_internal (fe_memutils.c:30)
==46805== by 0x10000DE79: palloc (fe_memutils.c:118)
==46805== by 0x1000052A0: DecodeXLogRecord (xlogreader.c:1203)
==46805== by 0x100003AA9: XLogReadRecord (xlogreader.c:461)
==46805== by 0x1000022F0: extractPageMap (parsexlog.c:78)
==46805== by 0x10000188C: main (pg_rewind.c:280)

This leak is rather bad particularly when a node is in recovery,
because the recovering node can fail with an OOM after replaying a
certain amount of records, and even worse for a standby node which is
aimed to replay records indefinitely.

Attached is a patch fixing the leak, and I checked that it got fixed
after re-running valgrind. I am adding an entry in the open item list
for 9.5 as well.
Regards,
--
Michael

Attachment Content-Type Size
20150723_xlogreader_leak.patch text/x-diff 816 bytes

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Sushant Sinha 2015-07-23 15:41:02 Re: Stalled post to pgsql-bugs
Previous Message sushant 2015-07-23 04:56:17 BUG #13515: Much higher disk writes after postgres upgrade 9.4.3->9.4.4