Using shell aliases in make

Dark_Helmet · 02-23-2007, 03:01 PM

I'm having trouble getting make to recognize shell aliases to commands and I don't know why. Any help would be appreciated. Here's what I've tried so far.

First, the simple Makefile:

Code:

install:
<tab>alias
<tab>which install
<tab>@echo "SHELL = $(SHELL)"
<tab>install -d /home/compile/install_test

Whenever I execute make, I never see that install is aliased elsewhere. Of the methods I've tried:

1. First attempt -- encapsulated shell script (plain jane aliases)

contents of ~/scripts/soft_install.bash

Code:

#!/bin/bash

alias install='sudo /path/to/special/install'

make SHELL=/bin/bash install

unalias install

2. Second attempt -- startup files and environment variables
Because make starts a sub-shell for each command, I thought to try putting the alias in startup files (triggered by an environment variable).

contents of ~/scripts/soft_install2.bash:

Code:

#!/bin/bash

SOFTWARE_INSTALL=yes

make SHELL=/bin/bash install

unset SOFTWARE_INSTALL

abbreviated contents of ~/.bashrc:

Code:

if [ -n ${SOFTWARE_INSTALL} ] ; then
  alias install='sudo /path/to/special/install'
fi

3. Third attempt -- more startup files
Started grasping a straws here. I worried about how make was starting each sub-shell (regarding POSIX compliance). So I linked ~/.profile to ~/.bash_profile. ~/.bash_profile in turn sources ~/.bashrc above.

4. Fourth attempt -- more straw-grasping
I set the ENV environment variable to point to ~/.bashrc thinking (again) that a POSIX problem might be involved.

cotents of ~/scripts/soft_install3.bash:

Code:

#!/bin/bash

ENV=/home/compile/.bashrc

make SHELL=/bin/bash install

unset ENV

In ALL cases, I get the following output from make:

Code:

alias
which install
/usr/bin/install
SHELL = /bin/bash
install -d /home/compile/install_test

The directory is created, but not with the proper install executable. The "special" version of install displays a debug message when invoked.

I'm at wit's end here. I need to have make see the aliases and I want it to be transparent--that is, no modification of the Makefile necessary. My bag of tricks is empty, and unless someone here saves me, I may have to open the make source code to see why in the world it absolutely refuses to take the aliases.

For those that must know, I've hacked some of the coreutils programs (chgrp, chmod, chown, cp, install, mkdir, mv, and rm) to accept new arguments: --allow-src-path=<arg>, --allow-dst-path=<arg>, --allow-path=<arg>. If any of the programs that support these arguments try to operate on a file outside the allowed path(s), the program fails. I install each package into its own directory, and I don't want anything to go outside that directory: config files, "share" files, executables, etc. Those arguments ensure that the files get put in the proper directory or the make install fails.

And then I get to gripe at the developers for not honoring the --prefix option to the configure script.

jr1 · 02-23-2007, 03:35 PM

Quote:

Originally Posted by Dark_Helmet

I'm having trouble getting make to recognize shell aliases to commands and I don't know why.

Bash aliases aren't normally expanded in non-interactive shells, and I think it's safe to assume that shells started by make are non-interactive. Try putting "shopt -s expand_aliases" in the script, before the alias is used.

Dark_Helmet · 02-23-2007, 04:11 PM

Thanks for the reply. Interactive vs. non-interactive shells has never been something I've felt comfortable with.

Anyway, I gave your suggestion a try, but the shopt option isn't preserved when I invoke make. I tried inserting "shopt -s expand_aliases" in the script of my original message and in my ~/.bashrc. I also added this line to Makefile:

Code:

<tab>shopt | grep expand_aliases

The output:

Code:

$ ~/scripts/soft_install.bash
shopt | grep expand_aliases
expand_aliases  off
aliases
which install
/usr/bin/install
...
$ shopt | grep expand_aliases
expand_aliases  on

Any ideas?

EDIT: Wow... I decided to add "shopt -s expand_aliases" to Makefile. It changed nothing.

contents of Makefile:

Code:

install:
<tab>shopt -s expand_aliases
<tab>shopt | grep expand_aliases
<tab>alias
<tab>which install
<tab>install -d /home/compile/install_test

output:

Code:

shopt -s expand_aliases
shopt | grep expand_aliases
expand_aliases  off
alias
which install
/usr/bin/install
install -d /home/compile/install_test

EDIT2:
After reading the bash man page, I may try to change the aliases into functions and export them with "export -f". That'll have to wait until later though.

Dark_Helmet · 02-23-2007, 06:58 PM

I met with success by export'ing the functions. Basically the setup is this:

Contents of ~/scripts/softinstall.bash

Code:

#!/bin/bash

# ... Do some argument processing ...

ALLOW_PATH_BIN="/root/software_install"
SRC_ALLOW="--allow-src-path=${parsed_src_arg}"
DST_ALLOW="--allow-dst-path=${parsed_dst_arg}"
export ALLOW_PATH_BIN SRC_ALLOW DST_ALLOW

source ~/.install_functions
export -f install

make ${make_args} ${make_target}

unset ALLOW_PATH_BIN SRC_ALLOW DST_ALLOW
unset install

Contents of ~/.install_functions

Code:

function install ()
{
  sudo ${ALLOW_PATH_BIN}/install ${SRC_ALLOW} ${DST_ALLOW} $*
}

I'm sure they'll need some refinement, but it appears to work.

tuxdev · 02-23-2007, 07:08 PM

Yuck. Why do you *have* to do it this way, and not just the normal approach of using enviroment variables to hold various aspects of the build configuration and use make -e so that the enviroment overrides whatever is in the Makefile?

Dark_Helmet · 02-23-2007, 07:46 PM

The problem is this: the traditional software compile/install routine wants root to execute "make install". When root does this, files can go anywhere, regardless of what the --prefix argument was set to with the configure script. I don't want that. I want every piece of software tucked into its own directory. Installing as a non-root user will give a "permission denied" error if any file tries to stray from the specified --prefix path.

Then the custom coreutils commands are what allow the install to set appropriate permissions and ownerships to installed files. I give the non-root user the ability to execute those commands as root with sudo, but only with appropriate limits: the destination and source directories must be specified and file operations are limited to only those directories.

There are very respectable software packages that don't honor --prefix completely. The ntp client being one (it wants to install its config file in /etc regardless of --prefix value).

I'll take a look at the -e option, but if memory serves, I think it's possible for the Makefile to override the -e. And I'm not sure what the scope of the environment settings imported are.

Dark_Helmet · 02-24-2007, 07:53 PM

tuxdev,

Sorry, I don't think I understood what you were asking at first. I re-read the make manual, and I must have been confused. There doesn't seem to be any indication that the Makefile can override the -e option (except maybe the SHELL environment variable).

So I experimented, and I can get the setup I need (I think) by using the -e, but it's still a two-step process.

I have to set environment variables for the functions to work properly. That becomes tedious quickly because I need to set three, sometimes four, environment variables. And that has to be done each time I want to install a new package. So I created an alias that sources a file to adjust the variables. So it becomes something like this:

Code:

$ setup_install
exporting ALLOW_PATH_BIN=/path/to/special
exporting SRC_ALLOW=--allow-src-path=/path/to/compile/dir
exporting DST_ALLOW=--allow-dst-path=/path/to/install/dir
done
$ make -e SHELL=/bin/bash <other_args> install

I may keep the script-based method since it calls make anyway. In that case it would be:

Code:

$ soft_install.bash <src_arg> <dst_arg> <make_args>

In that version, the environment for the compile user isn't modified, giving regular access to the traditional versions of the chgrp, chmod, chown, cp, install, mv, and rm commands.

If I still haven't answered your question (i.e. don't understand what you were getting at), or if you have an alternative, please let me know.

tuxdev · 02-24-2007, 09:00 PM

Hmm, there may be a way for a Makefile to override -e, call itself recursively. This sounds like a yucky problem, so I suppose a yucky solution fits. How would you handle a Makefile calling install directly by /usr/bin/install or whatever? The ultimate solution to this problem may be a chroot jail-like thing.

Dark_Helmet · 02-24-2007, 09:41 PM

I'd thought of trying the chroot thing, but ultimately decided it wasn't efficient. The chroot environment must be updated after any system change--whether it be a newly installed piece of software (a dependency) or a configuration file change (such as ld.so.conf). Essentially your system occupies twice the normal space--one for each copy because you can't symlink outside the chroot. So there's aggravation in keeping the chroot environment current and in reduced disk space.

It's funny you mention the /usr/bin/install thing, because my first prime-time test was compiling and installing rsync, and its Makefile uses /usr/bin/install. Fortunately, they made variables for it (broken into INSTALLCMD and INSTALLMAN) and I could change those values from the command line. But I imagine I'll have to inspect the Makefiles of each package to see if any changes are needed. I'll likely start making modifications to the Makefile and save the changes as patches. The grand idea being that I might be able to use the same patch on a later release. Or I might send in the patch to the devs and attempt to convince them that they should change their style, or use variables like rsync.

tuxdev · 02-24-2007, 10:11 PM

If you've got to inspect and modify the Makefiles themselves, that's kind of game over.

Hmm, it might be possible to do something like installwatch and/or use unionfs to do the chroot-like thing without the efficiency issues.

Dark_Helmet · 02-25-2007, 11:28 AM

I only plan to inspect the Makefile on failures. I'm about to wrap all the above stuff into yet another script. The script will extract the source, configure it, compile, install to a directory inside the compile user's home directory, and run some checks. Specifically, did the install exit with non-zero result and are any of the installed files not owned by root. If so, then something didn't go according to plan and the Makefile needs to be modified. If all is successful, it re-configures, re-makes, and installs in the proper location.

At least, that's how it will work in theory...

I'll look into installwatch and unionfs. I hadn't heard of either. Thanks.

tuxdev · 02-25-2007, 03:09 PM

Hmm, this kind of work would be really, really useful in src2pkg to do the whole packaging process as a normal user, than only install the package as superuser. src2pkg doesn't support generating RPMs (yet), so it's not so useful to a RPM distro user as yourself.

(mumbles and grumbles about pains with RPM-based distros...)

Dark_Helmet · 02-25-2007, 03:32 PM

Well this is for my LFS system. In fact, I need to update my profile because I don't have FC4 installed anywhere. So yeah, this is all source-based installation. I'll take a look at what gnashley has come up with.

In fact, he was helping me out before. The problem I'm trying to solve here is step 1 in a larger goal of a general change in filesystem structure. The next step is to hack at bash to modify how executables are found. Instead of PATH pointing to each bin directory, it will point to directories containing a collection of installed packages--each package having a bin subdirectory.

Who knows, I may actually be able to contribute something useful to his project

jr1 · 02-26-2007, 03:16 AM

Quote:

Originally Posted by Dark_Helmet

Code:

$ ~/scripts/soft_install.bash
shopt | grep expand_aliases
expand_aliases  off
aliases
which install
/usr/bin/install
...

The "which" command is a program that is external to bash and doesn't know anything about aliases or shell functions. In bash, you should always use "type" instead of "which" if you want to see whether a command is aliased.

Quote:

Originally Posted by Dark_Helmet

EDIT: Wow... I decided to add "shopt -s expand_aliases" to Makefile. It changed nothing.

contents of Makefile:

Code:

install:
<tab>shopt -s expand_aliases
<tab>shopt | grep expand_aliases
...

As far as I can tell, the reason this fails is that make basically starts a new shell at each line. For example,

Code:

$ cat Makefile
SHELL=/bin/bash

foo:
        shopt -s expand_aliases; shopt expand_aliases
        shopt expand_aliases
$
$ make foo
shopt -s expand_aliases; shopt expand_aliases
expand_aliases  on
shopt expand_aliases
expand_aliases  off
make: *** [foo] Error 1

One solution would be to invoke bash as an interactive shell by using the -i option, and also perhaps use the --rcfile option to specify a startup file that contains your aliases. In this case, you'd have something like

Code:

SHELL=/bin/bash --rcfile my_aliases -i

in your Makefile.

Or you could just set BASH_ENV to the value of a file that contains aliases and the "shopt -s expand_aliases" command. BASH_ENV is used to to determine the startup file for a non-interactive shell; by default no startup files are used (don't forget to export BASH_ENV). Check the "Bash Startup Files" section of the bash reference manual for more info. I think that this approach is a little cleaner than forcing the shell into interactive mode, because the latter might have some unintended consequences.

Quote:

Originally Posted by Dark_Helmet

EDIT2:
After reading the bash man page, I may try to change the aliases into functions and export them with "export -f". That'll have to wait until later though.

If functions work for you, then I'd strongly recommend them over using aliases. Aliases just perform very simple substitutions and are rarely the best choice for scripting.