Re: FW: Java Memory Issue while Loading Postgres library

On 29/04/2010 7:34 PM, A(dot)Bhattacharya(at)sungard(dot)com wrote:

> We have a java exe making a call to a postgres function. This postgres
> function internally makes a call to a dll (which is written using
> Postgres extended C).

If I understand you correctly, you have a dll that uses ecpg to
communicate with postgresql. It is loaded by a JVM via JNI, and that JVM
then uses JNI calls to use your ecpg-based DLL to talk to the database.

If I've understood you right: Why this strange architecture? Usually
Java apps just talk to the database via JDBC.

As far as I know there's no particular reason you *can't* do calls to
your ecpg-using dll via JNI, though. I'd be surprised if your problems
didn't turn out to be misuse/misunderstanding of the
notoriously-hard-to-get-right JNI interface, or issues with your DLL
and/or its use of ecpg.

If I have NOT understood how you are doing things correctly, then please
be MUCH MORE SPECIFIC. Making people guess what you're doing doesn't
help you get accurate, useful answers.

I strongly recommend that you try to produce a self-contained test case.
Write a minimalist ecpg dll that does only one simple thing. Write a
minimalist, simple JNI-using Java program that loads and uses that DLL.
See if it leaks memory. If this minimalist test case demonstrates the
leak, post the source code to BOTH the dll and your simple Java program
here. Make sure both can be compiled without extra libraries and include
any project files etc required to compile them.

If your new test case doesn't leak, then you know the basic JNI + ECPG
combination isn't the issue. You need to start investigating what in
your code is causing the leak. If you just can't figure it out, then
start chopping things out and disabling things until the leak stops
happening to narrow it down.

Without a lot more detail than you have provided, it is unlikely that
anybody here can help you.

> Now the issue is that, when we make a call to this dll, it consumes a
> lot of memory and this memory is getting consumed from the heap space of
> the original java process causing an out of memory exception in Java.
>
> Is this a known issue.

I rather doubt that anybody else anywhere in the world is doing what
you're doing ;-)

Not as far as I know it's not, no. Nor have you provided any evidence
it's an issue with PostgreSQL not your own code yet.

> Do we have a way to keep these processes disjoint
> and not eat up heap space of the original process?

http://google.com/search?q=inter+process+communication

... but you should probably fix the problem that's causing the excessive
memory use/leakage rather than just working around it by running the
code out-of-process.

--
Craig Ringer