Can't load shared library (but it's right there!)

brazzle · 02-15-2011, 02:52 PM

Hello all,

I have visited these boards a few times, but never posted.

Here's my problem: I was given the source to a program and asked to get it running on a 64-bit Debian 2.6.26 machine. Currently it is working on 2 64-bit OpenSUSE machines.

The application uses TCL TK for a GUI and everything compiles just fine; however, on startup, the user must enter one of three possible modules to load; when attempting to load these modules (tcl 'load' function), I receive this error:

Code:

Error in startup script: couldn't load file "../Build/libMpf.so": libTransReaders.so: cannot open shared object file: No such file or directory
    while executing
"load ../Build/libMpf.so Mpf"
    ("eval" body line 1)
    invoked from within
"eval load ../Build/${px}${i}${sx} $i"
    (procedure "load_lib" line 10)
    invoked from within
...

When I do an 'ldd' on the library, I get "not found" for every shared object that was built during the make:

Code:

ldd libMpf.so
        linux-vdso.so.1 =>  (0x00007fff86dff000)
        libTransReaders.so => not found
        libCore_Core.so => not found
        libRei_Core_rei_lib.so => not found
        libRei_Core_SummaryValues.so => not found
        libRei_Core_Reks.so => not found
        libRei_Core_Therm.so => not found
        libIncinerators_Core_Mpfgrid.so => not found
        libIncinerators_Core_Streamlines.so => not found
        libRei_Core_Solver.so => not found
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f1196463000)
        libm.so.6 => /lib/libm.so.6 (0x00007f11961e0000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f1195fc8000)
        libc.so.6 => /lib/libc.so.6 (0x00007f1195c75000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f11969c8000)

I did a 'file' on each of the files and they are all 64-bit (here's a few):

Code:

libMpf.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
libTransReaders.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
cowtip: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.8, not stripped

The only thing I can see that is different is that one machine uses PGI Fortran Compiler 6.2 and this one I'm trying to get it working on uses 10.6; however, the problematic modules are not written in fortran (it is used for one other module).

Any ideas on what the problem could be?

Thanks

corp769 · 02-15-2011, 03:02 PM

Try doing either a locate or whereis on those files and try to track them down. Most likely, the files need to be either copied or sym linked into /lib64

johnsfine · 02-15-2011, 03:40 PM

Quote:

Originally Posted by corp769

Try doing either a locate or whereis on those files and try to track them down.

The OP clearly indicated the files are in a known/intended location, so using locate or whereis is not appropriate.

Quote:

Most likely, the files need to be either copied or sym linked into /lib64

That might be the appropriate answer. But too much use of that answer leads to a lot of clutter in /lib64.

In many situations that may be similar to the OP's, I need to make certain .so files available for specific uses without cluttering /lib64.

I usually have the script that does the surrounding operation modify LD_LIBRARY_PATH so it includes the correct extra directories at the correct time. Web searches on that approach turn up lots of pages that seem to advise against it (though it works well for me in many cases). Also, this approach depends on having some script or other parent in the appropriate point above/before the relevant load operation to do that change to LD_LIBRARY_PATH. If you push that all the way up to something like .bashrc or the user's login script, then LD_LIBRARY_PATH may be too cluttered.

Whether or not my approach is poor, such sites give some extra insight and alternatives. The following link is for Solaris, but many other sites say similar things for Linux but not quite as clearly.
http://prefetch.net/articles/linkers.badldlibrary.html

Another approach involves modifying the ld command used to produce your executable (libMpf.so). For any .so file that libMpf.so uses, you can specify the full path and filename (including the lib and .so part) instead of using a library option with a partial name. Then the load time dependency will point to that exact file, rather than using a load time search.

That method is often used when you are building and testing an experimental version of an executable with related (also experimental) .so file. It requires the .so file you depend on be in the same place at build time as at final run time. Usually searching for .so files at load time is better than hard wired locations. But needs vary and sometimes hard wired (chosen at link time) locations are better.

Quote:

Originally Posted by brazzle

to get it running on a 64-bit Debian 2.6.26 machine. Currently it is working on 2 64-bit OpenSUSE machines.

I don't think the following difference between Debian and Suse is relevant to your problem, but it is close enough to relevant that I wonder whether you left out some detail that makes it relevant:

In Debian, the 64 bit lib directory is /lib and /lib64 is a sym link to /lib while /lib32 is a sym link to a 32 bit lib directory elsewhere.

In Suse, the 64 bit lib directory actually is /lib64 while /lib is a directory of 32 bit libs.

So any direct use of /lib can make your code non portable across the Debian/Suse boundary.

corp769 · 02-15-2011, 03:53 PM

Quote:

Originally Posted by johnsfine

The OP clearly indicated the files are in a known/intended location, so using locate or whereis is not appropriate.

I know this, I just included it because that's how I normally issues like this. I always locate libraries just to verify their presence.

brazzle · 02-16-2011, 01:10 PM

Quote:

Originally Posted by johnsfine

I usually have the script that does the surrounding operation modify LD_LIBRARY_PATH so it includes the correct extra directories at the correct time. Web searches on that approach turn up lots of pages that seem to advise against it (though it works well for me in many cases). Also, this approach depends on having some script or other parent in the appropriate point above/before the relevant load operation to do that change to LD_LIBRARY_PATH. If you push that all the way up to something like .bashrc or the user's login script, then LD_LIBRARY_PATH may be too cluttered.

Yeah, this is what I've read also. I went ahead and added the path for progress' sake and was able to make it past this error. I have new errors, but they're less confusing.

Quote:

Originally Posted by johnsfine

Another approach involves modifying the ld command used to produce your executable (libMpf.so). For any .so file that libMpf.so uses, you can specify the full path and filename (including the lib and .so part) instead of using a library option with a partial name. Then the load time dependency will point to that exact file, rather than using a load time search.

Interesting. I will give that a try and see how it goes. All the libraries and the executable all reside in the same directory, so it is very odd that something like that would be necessary, but it is worth a shot.

Quote:

Originally Posted by johnsfine

In Debian, the 64 bit lib directory is /lib and /lib64 is a sym link to /lib while /lib32 is a sym link to a 32 bit lib directory elsewhere.

In Suse, the 64 bit lib directory actually is /lib64 while /lib is a directory of 32 bit libs.

So any direct use of /lib can make your code non portable across the Debian/Suse boundary.

That's a great point, but we are not directly using /lib (or any equivalent) to build.

Thanks again for all the help! If I can't modify ld to make things work, I will make an exception to the 'no modifying LD path' rule.

johnsfine · 02-16-2011, 03:28 PM

Quote:

Originally Posted by brazzle

I will give that a try and see how it goes.

You probably already know to use ldd to understand the results of the ld command (rather than testing real use of the .so and inferring the results from the behavior). But in case you had forgotten that, I'm suggesting it.

Quote:

All the libraries and the executable all reside in the same directory, so it is very odd that something like that would be necessary

I don't fully understand the Windows rules for searching for DLLs, but I think Windows generally includes the directory of the requesting binary in the places to look for the requested DLL.

I'm even less sure of details of the Linux rules in that situation, but I think Linux generally does not automatically include the directory of the requesting binary when searching for a requested .so.

paulsm4 · 02-16-2011, 03:38 PM

Hi, brazzle -

Thank you for the "ldd" output - extremely useful.

Q: Have you tried setting LD_LIBRARY_PATH?

ta0kira · 02-16-2011, 07:18 PM

When you qualify a library with a relative path during linking, the dynamic linker checks for it relative to the execution path at run-time. You should therefore link using the full path name, or place the library in a standard lib directory and link without a directory qualification. You might also look at the -rpath linker option. Setting LD_LIBRARY_PATH will likely be of no help because of the directory qualification.
Kevin Barry

brazzle · 02-16-2011, 11:12 PM

Thanks again all for the replies. Setting LD_LIBRARY_PATH did work around the problem. I'm primarily a Windows developer, so I am trying to quickly learn the nuances of building on Linux. I will try out your suggestions and see what I can come up with and post back here--hopefully with a solution.

B