Re: BUG #4996: postgres.exe memory consumption keeps going up

On 26/08/2009 9:25 AM, WANGRUNGVICHAISRI, SHIVESH wrote:
> Hi,
>
> I have uploaded everything that you should need here:
>
> http://www.filefactory.com/file/ah334c6/n/Pgsql_Leak_zip

I notice that you used the Windows Task Manager to monitor memory usage,
rather than Process Explorer. There's a reason I suggested that you
monitor with Process Explorer: it provides more detail about the
process's memory usage. In particular, it lets you tell how much of the
memory usage is shared memory and how much is private.

The Task Manager does not. Because PostgreSQL makes heavy use of shared
memory this information is important.

That said, the "VM size" looks like the private size, making the rest
shared. The growth you're observing is in shared memory, suggesting it's
not in fact a leak at all, but the backend touching more and more of
shared_buffers .

I don't see any sign of a leak in postgres.exe . Given your
configuration, I'd expect to see postgres.exe's memory use grow up to
between 256 and say 290-ish MB (given that you have shared_buffers =
256MB) and none of your screenshots show it exceeding that. It all looks
quite normal to me from what you've posted.

You need to understand that PostgreSQL uses shared memory. The
postmaster allocates this memory *once*. All postgres.exe backends
attach to this shared memory block, which is already allocated. The way
Windows accounts for memory use means that they don't immediately show
up as using that memory, but it does get added to their reported memory
usage as they read from and write to the shared memory block. Despite it
being shown as part of each backend's memory use, the memory is actually
ONLY USED ONCE - by the postmaster.

You haven't let the test run long enough for the crash you've reported
to take place or even for postgres.exe to grow beyond normal size (if it
does). You also haven't provided the contents of postgresql's log file
after the crash you described. As a result so I can't tell what happens
when growth hits the ~2GB limit you mentioned in your initial post.

I can't reproduce the crash myself since shared usage on my postgres.exe
process(es) stabilizes and ceases growing after a while, as it should.

I'm running your test program on my sandbox machine now to see if I can
reproduce the issue, but so far everything looks quite normal. Both
active backends have stabilized at 37,965K of shared memory - one has a
total working set of 41,697K and the other 41,620K (so each is using
about 3.6MB of real system RAM). I'll come back tomorrow and see if
there are any signs of anything odd going on, but right now it looks
sensible.

SandBox.exe is growing in the same way my locally built test program is.
Weird. I think I'm going to have to recompile PostgreSQL's libpq using
VC++8 in memory debugging mode. Sigh.

--
Craig Ringer