Re: FreeBSD 5.3: Sharedlibs using sharedlibs (and Tcl)

On Wed, Mar 16, 2005 at 03:35:16PM +0100, Palle Girgensohn wrote:
!
! --On onsdag, mars 16, 2005 11.43.31 +0100 Peter Much
! <pmc(at)citylink(dot)dinoex(dot)sub(dot)org> wrote:
!
! >! So, you're saying that pctclsh *can* access, but pgaccess *cannot*?
! >Odd... ! I would expect they'd both use the same lib to connect, no?
! >I'll have to
! >
! >They use the same libraries, yes. But Tcl interpreter seem to need more
! >advice on where to find sub-functions in other libraries. It looks
! >like this:
! >
! >pgtclsh -finds-> libpgtcl.so -finds-> libpq.so -finds-> libkrb5.so
! >pgaccess -loads-> libpgtcl.so -finds-> libpq.so -fails-> libkrb5.so
!
! Uh, OK. I'm not qualified enough with linkers to answer this, I'm afraid.
! Did you try the pgsql-interfaces mailing list?

Oh well, same with me. I sent a copy of one of my reports to that
list, yes. But only got feedback that it will be evaluated by
moderator, as I am not signed on that list.
I'm actually no professional psql user - the database is just a small
part of my installation, mainly logging the lowlevel error counts
from my exabyte drives and providing reports about tape wearout.
And the kerberos is just there for fun, as a reference installation.

Nevertheless, I would think this is not a matter for the postgres
community. Because this would happen the same way with any other
application that provides Tcl support and kerberos support (or maybe
also with other components of the system, if they are used from Tcl).

So it seems either a Tcl problem or a linker/loader problem. Which,
I cannot say - maybe both.

! >And then I found that it is enough to place into libpq.so the explicit
! >references to libkrb5 and the other kerberos libs. That is what the
! >"readelf -a" output in my other mail shows.
!
! sounds like a better solution, yes... Shouldn't they always be there?
! Sounds like a bug to me?

Thats the question.
I just did a little more investigation (like reading manpages)
and found out _WHY_ it does work for pgtclsh but not for pgaccess.

There is a command "ldd" that shows nested library dependencies
for any program.
For pgtclsh it shows all the kerberos libs:

bash-3.00# ldd /usr/local/bin/pgtclsh
/usr/local/bin/pgtclsh:
libpgtcl.so.2 => /usr/local/lib/libpgtcl.so.2 (0x28075000)
libpq.so.3 => /usr/local/lib/libpq.so.3 (0x2807d000)
libtcl84.so.1 => /usr/local/lib/libtcl84.so.1 (0x28097000)
libm.so.3 => /lib/libm.so.3 (0x28135000)
libkrb5.so.7 => /usr/lib/libkrb5.so.7 (0x2814f000)
libasn1.so.7 => /usr/lib/libasn1.so.7 (0x28186000)
libcrypto.so.3 => /lib/libcrypto.so.3 (0x281a6000)
libroken.so.7 => /usr/lib/libroken.so.7 (0x2829b000)
libcrypt.so.2 => /lib/libcrypt.so.2 (0x282a9000)
libcom_err.so.2 => /usr/lib/libcom_err.so.2 (0x282c1000)
libz.so.2 => /lib/libz.so.2 (0x282c3000)
libreadline.so.5 => /lib/libreadline.so.5 (0x282d3000)
libutil.so.4 => /lib/libutil.so.4 (0x282ff000)
libc.so.5 => /lib/libc.so.5 (0x2830b000)
libintl.so.6 => /usr/local/lib/libintl.so.6 (0x283e4000)
libssl.so.3 => /usr/lib/libssl.so.3 (0x283ed000)
libncurses.so.5 => /lib/libncurses.so.5 (0x2841b000)
libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x2845a000)

But for libpgtcl.so (this is the first elf binary that pgaccess gets
to see) it does not show these kerberos libraries (I use the old
libpq.so here, not the one that I have modified):

bash-3.00# ldd /usr/local/lib/libpgtcl.so
/usr/local/lib/libpgtcl.so:
libpq.so.3 => /usr/local/lib/libpq.so.3 (0x2815a000)
libtcl84.so.1 => /usr/local/lib/libtcl84.so.1 (0x28174000)
libm.so.3 => /lib/libm.so.3 (0x28212000)
libintl.so.6 => /usr/local/lib/libintl.so.6 (0x2822c000)
libssl.so.3 => /usr/lib/libssl.so.3 (0x28235000)
libcrypto.so.3 => /lib/libcrypto.so.3 (0x28263000)
libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x28358000)

Then the explanation became simple: these kerberos libraries get
just LITERALLY LISTED WITHIN THE pgtclsh BINARY! And this is an
impossible method for a Tcl script.

bash-3.00# readelf -d /usr/local/bin/pgtclsh | grep krb5
0x00000001 (NEEDED) Shared library: [libkrb5.so.7]

So now we have a full explanation for the behaviour, but not really
a solution.
Instead, this looks like a fundamental question about how to load
nested elf sharedlibs from interpreter languages.

From my technical viewpoint, the only solution that makes sense
would be: every shared library must reference all other shared
libraries from which it uses functions. The shared library cannot
rely on the executable to do this job, because the executable
may be an interpreter script, which neither is able to do this
nor would it want to know them all.

From this viewpoint, the linker command that creates libpq.so
is defective. So You were right and its a problem for the
postgresql developers.

But as I am not competent with shared libraries and development
systems and such stuff, I would very much appreciate the opinion
of somebody with profound knowledge of these things. And anyway,
perl should have similar problems.

Ok, lets see - which nice mailinglists do we crosspost today?
I go with questions and ports, and add hackers. I think we're now
deep enough in the rabbithole for them. ;-)
And psql-interfaces.

PMc