Re: Memory use in 8.3 plpgsql with heavy use of xpath()

>> Gregory Stark <stark(at)enterprisedb(dot)com> writes:

>>> That's just a special case of what would be expected to happen with
>>> memory
>>> allocation anyways though. Few allocators return memory to the OS
>>> anyways.
>>
>> Well, that does happen on Linux for instance. Since Matt knew in his
>> original report that the xpath leak was intra-transaction, I assumed
>> he must be using a platform where malloc/free can release memory back
>> to the OS --- else he couldn't have seen that behavior from outside
>> the backend.
>>
>> Still, it's entirely possible that some sort of high-water-mark is
>> involved somewhere, perhaps in malloc's internal data structures.
>
> I was really going on a hunch, as I noticed a definite trend of postgres
> processes using more and more memory over time, and it only started after
> switching to 8.3 and starting to use xpath() quite heavily. Most of the
> memory data I have comes from Linux x64 systems with Postgres compiled as
> 64-bit. But I did also notice what appeared to be a similar trend on an OS
> X PPC system.
>
> In any event, I'm sorry I can't provide any more helpful tests, but I'll
> report back how the system changes after I've patched these systems.

I'm following up on this thread from a month ago on PG 8.3 memory use. I'm
afraid even after updating to 8.3.3 + this patch, I still see the same
overall memory trend. You can see what I'm looking at here with a couple
of memory graphs.

The first comes from our old primary database server, which started on 8.1
and then we switched to 8.3. I've added a callout on the chart to when we
switched to 8.3. Before the switch, the application memory use (bright
green) was pretty consistent, around 250MB. After the switch, you can see
the application memory trends upwards over time.

http://msqr.us/pg83-mem/pg83-memlead-1.png

Around June, we then switched to some new hardware. It was on this machine
about a month ago I updated to 8.3.3 + the xml.c patch referred to in this
thread. I've pointed this out on this chart:

http://msqr.us/pg83-mem/pg83-memlead-2.png

Here the application memory continues to trend upwards over time since the
8.3.3 server was started, up to 1.2GB as of now.

Am I right in my understanding that in general I should not see this
every-increasing use of app memory by PostgreSQL? Or will PostgreSQL
slowly work towards consuming all the shared memory we've allocated (4GB
in this case) and retain that? The only apps running on this box are
PostgreSQL and Slony for replication.

-- m@