Re: FW: Java Memory Issue while Loading Postgres library

Hi Craig,

Sorry for creating confusion. Let me (I work with Ambarish, the original
author of the mail) try to be more specific now.

We have a library (written in C) which helps us in doing phonetic based
name search. We want to use this library inside a postgres DB function.
To achieve this we wrote a small C code (we referred as wrapper class)
which uses the library. This C code is an ECPG which is bundled as a dll
and placed in postgres's lib dir. The original postgres function is
supposed to be called from a java program using JDBC. And the postgres
function should call the C function of the wrapper class. At runtime we
observed that when the postgres DB function calls the C function of the
wrapper class (in the dll), the java heap memory start increasing and
reached to the max level resulted in crashing of JVM. Then we commented
out the call to ECPG C function from postgres DB function and realized
that everything went well.

We were surprised why the loading and execution of the ECPG is taking
JVM memory. Ideally it should use the postgres.exe memory and CPU
utilization. We observed all these using windows task manager.

I hope this will help you in understanding our problem.

Thanks and Regards,

Ashish Arya

-----Original Message-----
From: pgsql-general-owner(at)postgresql(dot)org
[mailto:pgsql-general-owner(at)postgresql(dot)org] On Behalf Of Craig Ringer
Sent: Thursday, April 29, 2010 6:34 PM
To: Bhattacharya, A
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: FW: [GENERAL] Java Memory Issue while Loading Postgres
library

On 29/04/2010 8:48 PM, A(dot)Bhattacharya(at)sungard(dot)com wrote:

> Your understanding is slightly incorrect. Actually we required to uses
a
> special library from postgres.

That mystery library being? From "postgres"? Do you mean a library
supplied by the PostgreSQL project its self? Libpq? If not, what?

C'mon, specifics. Please. Try to read your message after you have
written it while pretending you are somebody who does not work with your

code and does not know the things you know about it. Try to notice
assumptions and clarify them.

> For this we write a wrapper around the
> library and composed a dll out of it (which uses ECPG to communicate
to
> postgres). From the application front (Java) we make a JDBC call to
> postgres function. This postgres function then makes call to the dll.

This does not make sense. Your earlier mail said that the DLL is loaded
into the Java runtime.

How does a "postgres function" make a call to a DLL running in the JRE?
You can't make a call from the postgresql backend to code on a client
connected to ecpg.

Please be SPECIFIC. What is a "postgres function?" How does it "make a
call to the dll?" Etc. Provide links to PostgreSQL and Java
documentation for the APIs you use, use the precise terms for those
APIs, and generally be SPECIFIC and DETAILED. Post diagrams or code
samples if you have to!

> But the strange thing we observed that at runtime, somehow the
execution
> of dll is attached with java.exe. In other words, when we monitor the
> performance of the application we realized that the execution of dll
is
> consuming java heap memory and is shown as a thread in java.exe
process.
> We expected that it should be linked with postgres.exe.

Ah, ok, so you are TRYING to load code a C extension to PostgreSQL into
the server backend?

If so, why are you using ECPG? Or are you in fact *not* using ECPG, but
the PostgreSQL backend extension interface?

How is the dll "shown as a thread in [the] java.exe process" ? Using a
Java debugger attached to the JRE? Using Process Explorer or a C-level
debugger examining the loaded DLL list of the JRE? Using the Java
monitoring APIs?

I really do not understand what you are doing, or what you are trying to

do. I doubt anybody can help you without a much more detailed and
specific explanation of what you're doing.

I think it'd be a really good idea for you to write a minimalist test
case for this and post it. Either you'll figure out what's wrong in the
process of making the test case, or you'll have something to post that
people can play with to see what you are trying to do.

--
Craig Ringer