Re: Memory use in 8.3 plpgsql with heavy use of xpath()

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "Matt Magoffin" <postgresql(dot)org(at)msqr(dot)us>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: Memory use in 8.3 plpgsql with heavy use of xpath()
Date: 2008-07-03 00:10:32
Message-ID: 18771.1215043832@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

"Matt Magoffin" <postgresql(dot)org(at)msqr(dot)us> writes:
> Later, I added a large set of plpgsql trigger functions that operate on
> that new xml column data, using the xpath() function to extract bits of
> XML and populate them into normal tables. The server has been running in
> this fashion for many months now, and there is a noticeable difference in
> how Postgres is using memory now, in that over time it's non-shared memory
> use is climbing higher and higher. Right now I'm tracking this from data
> captured by Munin on the system. The memory creep is very slight, but over
> many months is easy to discern.

Looking back, I'm not sure that what we just found explains that part
of your original report.

> If I run some of these plpgsql functions using a lot of xpath() calls on
> large sets of data, huge amounts of memory are consumed (gigabytes) and
> the memory seems to accumulate until the transaction the functions are
> running in completes.

This part seems to match the bug though --- the leak is approximately
the same size as all the text returned by xpath() within the current
transaction.

So there may be a second issue remaining to be found. Can you put
together a test case for the long-term small leak?

regards, tom lane

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2008-07-03 00:35:22 Re: changing text search treatment of puncutation
Previous Message Tom Lane 2008-07-03 00:06:31 Re: Memory use in 8.3 plpgsql with heavy use of xpath()