Re: heavy swapping, not sure why

On Mon, Aug 29, 2011 at 4:45 PM, Lonni J Friedman <netllama(at)gmail(dot)com> wrote:
>> using any C code in the backend? this includes 3rd party libraries
>> which link in C, including postgis, pljava, xml2, etc.  Any features
>> being used not included in the standard core distribution are
>> interesting.
>
> Nope, nothing like that.  They're fairly generic setups, with nothing
> added that isn't part of the core distribution.
>
>>
>> How long do your database connections stay open? forever? If yes, is
>> memory distributed semi-evenly across all postgres processes or only
>> to particular ones? If no, do you see excessive consumption with the
>> non user backends like the stats collector, etc?
>
> Nope, nothing is forever, everything is a fairly brief connection (a
> few seconds, tops, with most under 1s). Although I do have pgbouncer
> sitting in front of systemA to serve as a connection pooler.

hm. well iirc pgbouncer tries to dump server connections older than
an hour or so. this plus your other statements makes me very
suspension the problem is in postgres itself. since postgres process
dies when the connection dies, long term memory accumulation is just
not possible except in the processes that aren't serving client
sessions (the very first thing you need to do is rule those processes
out). pgbouncer itself could be the issue, but i doubt it.
obviously, a full memory profile during your problem times is a
critical piece of evidence (a 'top' sorted by memory usage should to
the trick nicely).

it's possible you've unhappily tripped a leak in the o/s -- is
everything properly updated? running any funky hardware (like fiber
san drivers)? anything else interesting or out of the ordinary to
report?

merlin