Re: Storage Manager crash at mdwrite()

On Thu, Mar 15, 2012 at 7:58 PM, Jeff Davis <pgsql(at)j-davis(dot)com> wrote:

> On Thu, 2012-03-15 at 19:36 -0400, Tareq Aljabban wrote:
> > When configuring postgreSQL, I'm adding the libraries needed to run
> > HDFS C API (libhdfs).
> >
>
> >From the information below, it looks like C++.
>
> >
> > ./configure --prefix=/diskless/myUser/Workspace/EclipseWS1/pgsql
> > --enable-depend --enable-cassert --enable-debug CFLAGS="$CFLAGS
> > -I/diskless/myUser/Workspace/HDFS_Append/hdfs/src/c++/libhdfs
> > -I/usr/lib/jvm/default-java/include" LDFLAGS="$LDFLAGS
> > -L/diskless/myUser/Workspace/HDFS_Append/hdfs/src/c++/libhdfs
> > -L/diskless/myUser/Workspace/HDFS_Append/build/c++/Linux-i386-32/lib
> > -L/usr/lib/jvm/default-java/jre/lib/i386/server -ljvm -lhdfs"
> >
> >
> >
> >
> >
> > I have done lots of changes so far on how the storage manager works.
> > In fact, I changed smgr.c so instead of calling regular md.c
> > functions, that it would call my functions .
> > For simplicity, you can say that whenever mdwrite() was supposed to be
> > called, another function is also called beside it. so now what is
> > being called is:
> > mdwrite(param1, param2, buffer, ....);
> > hdfswrite(param1, param2, buffer, ....);
> >
> >
> > where hdfswrite() is the function where I write the buffer to HDFS.
> > I changed hdfswrite() so it will always write only the same (dummy)
> > buffer down to HDFS storage. Let's say "dummyBufferFile". After
> > writing this file 3 times to HDFS, I'm getting the message that I've
> > shown in my first email.
> > The same hdfswrite() code works without any issues when I run it in a
> > separate application.
> >
> >
> > Hope it's clear now.
>
> Well, it's clear that there's a lot going on ;)
>
> In other words, is there a reason you think that these would all work
> together nicely?
>
> I don't know the specifics, but I understand there are numerous perils
> to linking complex C++ code into complex C code, particularly around
> exception handling. Look in the archives for previous discussions around
> C++.
>

Thanks for your response Jeff..
It's true that libhdfs code resides under the c++ folder, but in all of the
documentation, libhdfs is referred to as the C interface of HDFS.
Now what you're saying makes sense.. that nothing guarantees this will work
well.. but in the phase of initDB, the function calls are done nicely
without any exceptions.. when starting the postmaster, something wrong
happens after 3 calls to libhdfs.. that's what I'm confused about..
What it's the difference between the two processes (initDB, start
postmaster), that might cause this crash to happen?

Regards

>
>