LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 01-10-2011, 12:15 AM   #1
cwhite102
LQ Newbie
 
Registered: Jan 2011
Posts: 3

Rep: Reputation: 0
How to change (add to) library search path at runtime on Linux


I'm writing an application with a plugin architecture and would like modify (specifically add to) the dynamic library search path while the main executable is running.

The plugin paths are not known until the application is running so I can't set LD_LIBRARY_PATH ahead of time.

My understanding (although I haven't tested it) is that the executable will only parse the LD_LIBRARY_PATH once, early on, so modifying this environment variable at runtime will have no effect.

(Note - on Windows the solution is to modify the PATH environment variable)

I know it's possible to specify the full path to load a dynamic library, and this would work if the plugin(s) only had a single library to load, but some of them will have a bunch of libraries with their own inter-dependencies, so I'd like the plugin folder(s) to get added to the search path to pick up the dependant libs.

Is there a proper technique to accomplish this on Linux?
Any guidance is appreciated.
 
Old 01-10-2011, 01:15 AM   #2
paulsm4
Guru
 
Registered: Mar 2004
Distribution: SusE 8.2
Posts: 5,863
Blog Entries: 1

Rep: Reputation: Disabled
Q: How is modifying %PATH% on Windows any different from modifying $LD_LIBRARY_PATH on Linux?

Q: What's wrong with "wrapping" your executable in a script that sets LD_LIBRARY_PATH, then runs the binary?
 
1 members found this post helpful.
Old 01-10-2011, 01:19 AM   #3
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 453Reputation: 453Reputation: 453Reputation: 453Reputation: 453
Quote:
Originally Posted by paulsm4 View Post
... Q: What's wrong with "wrapping" your executable in a script that sets LD_LIBRARY_PATH, then runs the binary?
Which is the way, say, FireFox runs and which is the way I am using building stuff from scratch - in the latter case the script is autogenerated.
 
Old 01-10-2011, 02:00 PM   #4
cwhite102
LQ Newbie
 
Registered: Jan 2011
Posts: 3

Original Poster
Rep: Reputation: 0
Thanks for the responses thus far.

Quote:
Originally Posted by paulsm4 View Post
Q: How is modifying %PATH% on Windows any different from modifying $LD_LIBRARY_PATH on Linux?
On Windows, the PATH variable will be reparsed just prior to each load library call. On Linux, (please correct me if I'm wrong) - my understanding is that LD_LIBRARY_PATH will only be parsed at the application startup, and changing it at runtime will have no effect. I'm also hoping to find the proper solution, perhaps various Unix flavors will have different behaviour with respect to reparsing LD_LIBRARY_PATH. I'd like to know if there is a sanctioned way to accomplish this goal.

Quote:
Originally Posted by paulsm4 View Post
Q: What's wrong with "wrapping" your executable in a script that sets LD_LIBRARY_PATH, then runs the binary?
Wrapping the application is a partial solution. It adds a fair amount of unwanted complexity as there is more logic to loading the plugins than just checking a folder for their existence. This logic is in the application and is already cross-platform. Having to pull that logic out into a pre-run script isn't ideal. In addition, a wrapper will prevent the ability to load additional plugins while the application is running without requiring a restart (eg. a user action causes a download of a new plugin).
 
Old 01-10-2011, 02:36 PM   #5
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 453Reputation: 453Reputation: 453Reputation: 453Reputation: 453
Quote:
Originally Posted by cwhite102 View Post
Thanks for the responses thus far.



On Windows, the PATH variable will be reparsed just prior to each load library call. On Linux, (please correct me if I'm wrong) - my understanding is that LD_LIBRARY_PATH will only be parsed at the application startup, and changing it at runtime will have no effect. I'm also hoping to find the proper solution, perhaps various Unix flavors will have different behaviour with respect to reparsing LD_LIBRARY_PATH. I'd like to know if there is a sanctioned way to accomplish this goal.
...
You can use 'dlopen' (wrapped into GNU libtool functions if desired) - 'man 3 dlopen'. You can load library from whatever file using 'dlopen'.
 
1 members found this post helpful.
Old 01-10-2011, 10:43 PM   #6
pet5800
LQ Newbie
 
Registered: Dec 2009
Posts: 7

Rep: Reputation: 0
add your search path in /etc/ld.so.conf
and then run the command ldconfig as root
 
Old 01-11-2011, 01:22 AM   #7
Nominal Animal
Senior Member
 
Registered: Dec 2010
Location: Finland
Distribution: Xubuntu, CentOS, LFS
Posts: 1,723
Blog Entries: 3

Rep: Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943Reputation: 943
If your application has per-user libraries or modules, they should be put in the application directory in the users home directory, $HOME/.myapp/lib/ for example. If there are ancillary libraries in strange places, run your application with $HOME/.myapp/lib/ appended to LD_LIBRARY_PATH, and let the user decide how to manage their libraries.

In Linux, handling shared libraries is quite well standardized. There's ldconfig utility managing /etc/ld.so* files to configure exactly where the libraries may reside, and of course LD_LIBRARY_PATH to add special cases. Dynamic libraries are versioned, with the default symlinked to the latest version. (Okay, some libraries have goofed in not bumping the version number at the proper time, but that's another matter.) Applications are expected to adhere to this information, and not go out of their way to do something else.

How does this work, then?

Assume the user has a special module in $HOME/.myapp/lib/, which requires a number of libraries scattered all over, none of them in the expected places. If your application was executed with $HOME/.myapp/lib/ in LD_LIBRARY_PATH, the user only needs to add symlinks in $HOME/.myapp/lib/ to point to the wayward libraries.

Your application does nothing special. It uses just the library filenames without paths in the dlopen() call.

And please, do not write an application which dynamically edits those symlinks; it's just weird and suspect behaviour. I'd personally never use such a badly behaving application.

You could, however, write a separate configurator tool, which uses e.g. ldd, file and readelf tools with e.g. find, the user can use to locate the libraries and manage the needed symlinks. But I don't believe the users will actually need one.
Nominal Animal

Last edited by Nominal Animal; 03-21-2011 at 02:57 AM.
 
1 members found this post helpful.
Old 01-11-2011, 09:35 PM   #8
cwhite102
LQ Newbie
 
Registered: Jan 2011
Posts: 3

Original Poster
Rep: Reputation: 0
Thanks for all the help, I'm understanding the platform a little better.

I tested altering the LD_LIBRARY_PATH in the running executable, and as suspected that doesn't work. (Setting LD_LIBRARY_PATH ahead of time by hand does allow the libs to load ok).

Quote:
Originally Posted by Sergei Steshenko View Post
You can use 'dlopen' (wrapped into GNU libtool functions if desired) - 'man 3 dlopen'. You can load library from whatever file using 'dlopen'.
Calling dlopen with a full path worked fine unless the library has a dependency that is not on the search path. For example, if I have a plugin with libs A and B, where A depends on B, and I try to dlopen /fullpath/A - the dlopen will fail even if I had already done a dlopen /fullpath/B. dlopen is only happy if B is on the search path.

I took a quick look into libtool, and noticed some promising functions in libltdl that may address this issue. I haven't had a chance to try it yet, but thanks for the pointer there.

The non-linux-standard plugin architecture is actually a result of the application not really being a single-user installed application at all. It is actually a distributed, cross-platform, Java based service. (With an additional end-user component to it). Some of the plugins are trivially cross-platform because they are pure Java. Some plugins (for performance and/or 3rd party library integration) have a native C/C++ component to them, and thus bundle native libraries in the plugin. We have Web-service based plugin repositories from which the service can download plugins. For simplicity, this is done the same on each platform, hence standard linux install techniques of the libraries are not used. Everything works well on all platforms (Linux, Windows, Mac, and Solaris) except for this issue of dependant libraries on the Unix platforms.
 
Old 01-12-2011, 01:58 AM   #9
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 453Reputation: 453Reputation: 453Reputation: 453Reputation: 453
Quote:
Originally Posted by cwhite102 View Post
... Calling dlopen with a full path worked fine unless the library has a dependency that is not on the search path. For example, if I have a plugin with libs A and B, where A depends on B, and I try to dlopen /fullpath/A - the dlopen will fail even if I had already done a dlopen /fullpath/B. dlopen is only happy if B is on the search path. ...
On UNIXish sytem there is a notion of TMPDIR - if needed, you can first find all the needed libraries, then symlink them to a directory under TMPDIR, then use the the full path with that directory.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How to set runtime library path in Makefile ???? narendra1310 Linux - Newbie 3 06-09-2010 04:45 AM
How to force preferred library search path for Apache Larry James Linux - Server 7 03-04-2008 03:15 PM
How to (temporarily) add a library search path? G00fy Programming 4 07-03-2007 09:06 AM
Can I Reset Xserver Library File Search Path? Daniel N. Santiago Linux - Newbie 1 08-18-2006 01:45 PM


All times are GMT -5. The time now is 06:15 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration