Altering PATH search in Bash

Dark_Helmet · 02-12-2007, 01:12 PM

Is anyone here familiar with the internals of Bash? I want to hack at it some and change how the shell searches for executable commands in PATH. The goal is to make a flag in PATH (or perhaps a new env. variable altogether) that contains a list of directories. Executables would be found in each path entry such as:
path_entry1/*/bin
path_entry1/*/sbin
path_entry2/*/bin
path_entry2/*/sbin
etc.

The reason is this: I'm building a new LFS system and I really dislike the "put everything in /bin, /usr/bin/, /usr/local/bin, etc." mentality. I want each software package to reside in it's own directory (e.g. samba would be installed in /usr/local/packages/samba-X.Y/{bin,sbin,etc,var,share}).

Yes, I know this is not Unix-like, but that's why it's Linux From Scratch, right?

Yes, I know there will be other issues to deal with given this organization.

Yes, yes, yes, I have thought about it a great deal and want to try.

I've pulled up the bash source and have found the section of the code that does the hash searching. I don't imagine it will be too difficult to locate where the hash table is populated. I'm looking for anyone that has experience poking around in Bash that can point out any pitfalls or suggest files/line numbers to look at, or explain some of the internals.

Thanks.

gnashley · 02-13-2007, 02:56 AM

Actually, what you want to do is exactly the Unix way, but not the 'usual' linux way.
What you want to do has been allowed for by the creation of the /opt directory where you can install each program in its' own folder and arrange things inside the folder any way you want.
BASH already does what you want by using wild-card expansion. After installing programs under /opt/progname/bin, just do this:

export PATH=/opt/*/bin:/opt/*/sbin:$PATH

Dark_Helmet · 02-13-2007, 09:57 AM

That's interesting. I'll admit, I never tried that because I had never seen it used before. Also, this LFS page (section 6.3.2.2) led me to believe that wildcards/globbing would not work in environment variables (namely PATH).

It raises some interesting questions, such as when does the wildcard expansion occur? at initial login? Each time a command is searched for?

I'll definitely give it a shot next time I sit down at my box. Thanks.

gnashley · 02-13-2007, 11:06 AM

Oh those are good questions indeed. While writing, I was going to say that it might work. Then I thought I'd see, so I created a fake prog called
/opt/test/test
and did a 'which test' and got /opt/test/test
Perhaps this doesn't work with 'env', but it seems to be fine for BASH itself.

anomie · 02-13-2007, 01:08 PM

Quote:

It raises some interesting questions, such as when does the wildcard expansion occur?

Expansion should occur when you assign the values to PATH. Try to echo $PATH afterwards and you'll see real paths rather than the wildcard character.

Dark_Helmet · 02-14-2007, 02:22 AM

I'm at a loss guys... I assume everything is working fine for both of you with that wildcard in the path, but it's not for me.

For reference:
GNU bash, version 3.1.17(1)-release (i686-pc-linux-gnu)

Code:

$ export PATH=/bin:/usr/bin:/packages/installed/*/bin
$ echo $PATH
/bin:/usr/bin:/packages/installed/*/bin
$ emacs
sh: emacs: command not found
$ ls -l /packages/installed/emacs/
drwxr-xr-x 2 compile compile 4096 2007-02-14 02:00 bin
drwxr-xr-x 2 compile compile 4096 2007-02-14 02:00 info
drwxr-xr-x 2 compile compile 4096 2007-02-14 02:00 man
drwxr-xr-x 2 compile compile 4096 2007-02-14 02:00 sbin
drwxr-xr-x 2 compile compile 4096 2007-02-14 02:00 share
$ ls -l /packages/installed/emacs/bin/emacs
-rwxr-xr-t 2 compile compile 6071862 2007-02-14 02:00 /packages/installed/emacs/bin/emacs
$ hash -r
$ emacs
sh: emacs: command not found
$ whoami
compile
$ ls -ld /packages
drwxr-xr-x 6 root root 4096 2007-02-14 01:43 /packages
$ ls -ld /packages/installed
drwxr-xr-x 2 root root 4096 2007-02-14 01:47 /packages/installed

Any ideas on why this isn't working? Maybe something added/changed in each of your distributions? I've still got a bare-minimum install of LFS at this point.

gnashley · 02-14-2007, 04:34 AM

Try declaring the PATH and exporting separately:
PATH=/bin:/usr/bin:/packages/installed/*/bin
export PATH
Also try enclosing in quotes:
PATH="/bin:/usr/bin:/packages/installed/*/bin"
export PATH

anomie · 02-14-2007, 08:46 AM

Dark_Helmet, that is very strange. On my CentOS box, the paths are being expanded when I assign them to the variable. Also, remember that paths should be delimited by a colon (':'). Have a look at this example:

Code:

[hector@troy ~]$ export EXPPATH=/usr/*/bin
[lala@shasha ~]$ echo $EXPPATH
/usr/kerberos/bin /usr/local/bin /usr/ofed/bin /usr/X11R6/bin
[hector@troy ~]$ export PATH=$(echo $EXPPATH | tr ' ' ':')
[hector@troy ~]$ echo $PATH
/usr/kerberos/bin:/usr/local/bin:/usr/ofed/bin:/usr/X11R6/bin

GNU bash, version 3.00.15(1)-release (i686-redhat-linux-gnu)

You can see the behavior on my session was a different.

Dark_Helmet · 02-14-2007, 11:46 AM

I'm not sure what the hang-up is, but it's still there. I have another LFS system built with a previous release of LFS that does not exhibit the same behavior--it expands the wildcard.

So, for the moment, I'll chalk it up as a problem with a configuration file (or two) that I haven't set up yet.

If it's not a config file issue, I've figured a way around it (very similar to anomie's method). When I have more time, I'll experiment.

gnashley · 02-15-2007, 02:25 AM

It may have to do with the BASH version. Did you try quoting the wildcard?

jr1 · 02-16-2007, 02:55 AM

Quote:

Originally Posted by Dark_Helmet

I'm not sure what the hang-up is, but it's still there. I have another LFS system built with a previous release of LFS that does not exhibit the same behavior--it expands the wildcard.

Bash does filename (wildcard) expansion on the string that is assigned to a variable, for example

Code:

export FOO=/opt/*/bin

It does not do filename expansion on colon-delimited elements of a such a string, such as

Code:

export FOO=/bin:/usr/bin:opt/*/bin

Nor does bash know to replace spaces with colons, which would also be necessary for this to have the desired result.

So, one solution is to use something like anomie's approach of assigning a single expandable string to one variable and then using it to help build PATH. Perhaps a simpler method would be to use process substitution each time a wild card is needed within the assignment to PATH, such as

Code:

export PATH="/bin:/usr/bin:$(echo /opt/*/bin | tr ' ' :)"

But note that this will fail if any of the directory names have spaces in them. A more general solution could be

Code:

export PATH="/bin:/usr/bin:$(perl -e '$" = ":"; print "@{[</opt/*/bin>]}";')"

This should work as long you haven't got colons in your directory names (but I'd have no idea how to get bash's PATH to properly understand those directories anyway). You could also do this using find in instead of perl.

matthewg42 · 02-16-2007, 03:09 AM

I think GoboLinux uses a layout like this. It may help to check out how they do it.

Dark_Helmet · 02-17-2007, 01:25 PM

@gnashley:
Yeah, I had originally tried quoting the string with the wildcard - no luck.

@matthewg42:
I had come across GoboLinux before, was interested by it, but lost the link. From their page, it sounds as if their solution is fairly complex. I'll look into it some more when I have time. Thanks for helping me find their website again.

@jr1:
I'm using a temporary method similar to what you provided at the moment. I'll give it a test drive and see how it works. Basically, in /etc/profile I have this:

Code:

for directory in $( ls -d /pkgs/inst/* ) ; do
  if [ -d ${directory}/bin ] ; then
    export PATH=${PATH}:${directory}/bin
  fi

  if [ -d ${directory}/sbin -a ${EUID} -eq 0 ] ; then
    export PATH=${PATH}:${directory}/sbin
  fi

  ...

done

There's a little more involved -- I handle other environment variables similarly (LD_LIBRARY_PATH, MANPATH, INFOPATH, PKG_CONFIG_PATH). I decided to go with this (for the moment) because te environment variable is a regular char *, meaning it can be arbitrary in length.

The primary drawback of this method is that it takes a re-login to "see" any packages installed while the user was logged in.

It also takes a little experimentation to get a package installed and live neatly in its own space, but that's expected.