Bash scripting and && operator

Dark_Helmet · 07-09-2003, 11:25 PM

I'm running into a problem that I do not understand in the slightest. In a script, I have a command like so:

mkdir ../build-dir && cd ../build-dir

When the script executes that line, it errors with this:

mkdir: cannot create directory `../build-dir': File exists

I look and sure enough, the directory does exist, but what's causing this error? Is the shell trying to execute the command twice?

I know, I could break the command into two separate statements, but being able to use the && is key to the way I want to implement the script. I don't want to break it up unless I absolutely have to since it will add a significant amount of complexity.

Tinkster · 07-09-2003, 11:33 PM

Certainly not ...

You might, however, want to check whether
the directory exists before you create it :)

Or did you do rm -Rf ../build-dir before
running the script?

Cheers,
Tink

DrOzz · 07-09-2003, 11:39 PM

why are you making the directory a level up from your current location? couldn't you just go to the location and issue the command with just mkdir build-dir && cd build-dir? anyways, i don't know if i understand...you say you issue this command and you get that error, and you look and the directory is there...so wouldn't that explain why your getting the error?

Dark_Helmet · 07-10-2003, 12:24 AM

Sorry, I guess I didn't clarify. The script is trying to automate compiling packages. It un-tar's the source, goes into the source tree, applies any patches, and then wants to execute the command I gave:

mkdir ../build-dir && cd ../build-dir

The directory does not exist prior to this command being executed. I've verified that more than once in two different ways:

1) verifying prior to the script running
2) exiting from the script immediately before the command is executed

In both cases, the directory does not exist. So, the mkdir portion does in fact create the directory as expected, but I get the error mentioned and I can't explain it.

For full disclosure, the above command resides in a shell variable. assigned similar to:

command="mkdir ../build-dir && cd ../build-dir"

and then executed later simply as:

${command}

I'm wondering if the ampersands need to be escaped, or if they simply cannot be used in this fashion. I've looked online for a script that does something like this, but all I can find is the && operator being used to combine test cases; not actual commands.

Edit: Hmmmm... I guess I have to take that back... tests ARE commands...

moses · 07-10-2003, 12:50 AM

Code:

if [ \! -d ../goobers ]; then mkdir ../goobers && \
echo "mkdired ../goobers" && cd ../goobers && pwd; \
else cd ../goobers && pwd; fi

or

Code:

command='if [ \! -d ../goobers ]; then mkdir ../goobers && echo 
"mkdired ../goobers" && cd ../goobers && pwd; else cd ../goobers && 
pwd; fi'
eval ${command}

Where everything after "command=" up to just before "eval" is on one line. . .

Dark_Helmet · 07-16-2003, 11:22 PM

Ok, it's been a while, but hopefully we can hammer this out. I would REALLY like to keep the command in the form "mkdir ../build-dir && cd ../build-dir". At this point, I've played with the code some and just cannot figure it out. So, I'm going to post what I think are the important parts and hope SOMEBODY can see an error or at least try it out on their system to see if they get the same problem. You are warned, this is a LOT of code, and I would be VERY appreciative if someone can identify my problem or confirm the possibility of a bug in bash's parsing. I can also email the whole script if someone thinks it would be necessary.. it's too long to post though.

Code:

#!/bin/bash

<snip>

TRUE=1
FALSE=0

EMPTY_CMD=" "
KEEP_LOGS=1
TEST_RUN=$FALSE

<snip>

#========================================================================
# binutils package information and commands
#========================================================================
BINUTILS_PKG="binutils-2.13.2"
BINUTILS_STATIC_PRE="mkdir ../binutils-build && cd ../binutils-build"
BINUTILS_STATIC_CFG="../binutils-2.13.2/configure --prefix=$LFS/static --disable-nls"
BINUTILS_STATIC_MAKE="make LDFLAGS='-all-static'"
BINUTILS_STATIC_INSTALL="make install"
BINUTILS_STATIC_POST="${EMPTY_CMD}"
BINUTILS_STATIC_PATCHES="${EMPTY_CMD}"

<snip>

#========================================================================
# Function: gen_cmd()
# GENeral COMmand is just that, it reduces the number of keystrokes needed
# when implementing the package case statement found below by creating a
# general structure for executing compilation commands. Since error chacking
# is a must, that's what this function's main purpose is: to remove error
# checking from the main body of the script and to implement it in a uniform
# manner.
function gen_cmd ()
{
  local logging_enabled
  local package_name
  local package_command
  local log_file
  local command_status

  # Yes, I'm anal. Put each argument passed into a MEANINGFUL name rather than
  # using $1, $2, etc. in the functional script commands later.
  logging_enabled=$1
  package_name=$2
  package_command=$3
  log_file=$4
  command_status=0            # default to a good exit status

  # The commands for each package are sent to this function regardless of
  # whether there's any meaningful work to be done. So we check to see if
  # we have to do anything here...
  if [ "${package_command}" != "${EMPTY_CMD}" ] ; then

    # Ok, we have something to do. Do we need to log the output?
    if [ ${logging_enabled} -eq 1 ] ; then

      # We create the log file in advance. That allows us to use the append
      # (>>) redirection operator. We want to use append in case a
      # a package has multiple patches to apply
      if [ ! -e "${log_file}" ] ; then
        echo "[Command] touch ${log_file}"

        if [ ${TEST_RUN} -ne ${TRUE} ] ; then
          touch ${log_file}
        fi
      fi

      echo "[Command] time ${package_command} >> ${log_file}"

      # Using time is not necessary, but I thought it would be cool to add its
      # output if logging is enabled. Then I can claim how bad-ass my computer
      # is by quoting compile times... until someone else's computer smokes
      # my times...
      if [ ${TEST_RUN} -ne ${TRUE} ] ; then
        time ${package_command} >> ${log_file}
        command_status=$?
      fi
    else
      echo "[Command] ${package_command}"

      # Plain vanilla version. Just execute the command.
      if [ ${TEST_RUN} -ne ${TRUE} ] ; then
        ${package_command}
        command_status=$?
      fi
    fi

    # Alright, the command has run, and we put the exit status into
    # command_status. Check that the command executed successfully. If not
    # display a message and exit
    # NOTE: This assumes an exit status of 0 is always good. For most cases
    # this is correct, but it might be better to pass in an expected exit
    # status per command
    if [ ${command_status} -ne 0 ] ; then
      echo "!!!Failure working on package ${package_name}"
      echo "Command returned a status of: ${command_status}"
      echo "Script is exiting with same status..."
      exit ${command_status}
    fi
  fi

}

<snip ; now inside actual functional portion of script>

  case "${package}" in

    <snip>

    ${BINUTILS_PKG})
      if [ ${KEEP_LOGS} -eq 1 ] ; then
        if [ ! -d ${STATIC_LOG}/${BINUTILS_PKG} ] ; then
          gen_cmd 0 "${BINUTILS_PKG}" "mkdir ${STATIC_LOG}/${BINUTILS_PKG}" "/dev/null"
        fi
        LOG_FILE="${STATIC_LOG}/${BINUTILS_PKG}/${BINUTILS_PKG}"
      fi

      if [ "${BINUTILS_STATIC_PATCHES}" != "${EMPTY_CMD}" ] ; then
        for pkg_patch in ${BINUTILS_STATIC_PATCHES} ;
        do
          patch_file="${STATIC_PKG}/${pkg_patch}"
          gen_cmd ${KEEP_LOGS} "${BINUTILS_PKG}" "${PATCH_CMD} ${patch_file}" "${LOG_FILE}.patches"
        done
      fi

      gen_cmd 0 "${BINUTILS_PKG}" "${BINUTILS_STATIC_PRE}" "${LOG_FILE}.pre"
      gen_cmd ${KEEP_LOGS} "${BINUTILS_PKG}" "${BINUTILS_STATIC_CFG}" "${LOG_FILE}.cfg"
      gen_cmd ${KEEP_LOGS} "${BINUTILS_PKG}" "${BINUTILS_STATIC_MAKE}" "${LOG_FILE}.make"
      gen_cmd ${KEEP_LOGS} "${BINUTILS_PKG}" "${BINUTILS_STATIC_INSTALL}" "${LOG_FILE}.install"
      gen_cmd ${KEEP_LOGS} "${BINUTILS_PKG}" "${BINUTILS_STATIC_POST}" "${LOG_FILE}.post"
      ;;

moses · 07-17-2003, 02:03 AM

I think it's a misunderstanding in the way the shell expands variables/parameters (I'm not saying I completely understand it either). My solution, for now, would be, if you really don't want to use if statements and the "eval" built-in, to define BINUTILS_STATIC_PRE as a command rather than a parameter:

Code:

BINUTILS_STATIC_PRE () { mkdir ../build-dir && cd ../build-dir; }
gen_cmd 0 "${BINUTILS_PKG}" BINUTILS_STATIC_PRE "${LOG_FILE}.pre"

Dark_Helmet · 07-17-2003, 08:11 PM

Well, the short of it is, I got it to work.

I tried the function definition like you suggested, but I couldn't get it to work. I understood the concept, but maybe I didn't code it up properly.

Anyway, here's the "solution" that I'll probably keep:

Code:

if [ ${TEST_RUN} -ne ${TRUE} ] ; then
  remaining_cmd="${package_command}"
  while [ ${command_status} -eq 0 ] && [ -n "${remaining_cmd}" ] ; do
    current_cmd="`echo ${remaining_cmd} | cut -f 1 -d '&'`"
    remaining_cmd="`echo ${remaining_cmd} | cut -s -f 3- -d '&'`"

    time ${current_cmd} >> ${log_file}
    command_status=$?
  done
fi

archtoad6 · 09-16-2005, 01:14 AM

I was poking around old posts about scripting & this piqued my interest. You have probably long since discovered this answer &/or no longer have the problem. In any case, for the benefit of those who might find it useful, I believe the answer to the orig. problem is "eval".

Here is the orig. problem, as I understand it:

Code:

[test]$ X="mkdir ../build-dir && cd ../build-dir"

[test]$ ${X}
mkdir: cannot create directory `&&': File exists
mkdir: cannot create directory `cd': File exists
mkdir: cannot create directory `../build-dir': File exists

[test]$ ls ..
build-dir  test

The described behavior has been verified -- the "mkdir" works but fails, & the "cd" doesn't happen.

Now clean up & verify it:

Code:

[test]$ cd ../test ;  rm -rf ../build-dir;  ls ..  
test

Then try it w/ "eval":

Code:

[test]$ eval ${X}
[build-dir]$

No error msgs. & we have cd'd to the created directory.

I can't explain exactly why this works when a raw '$X' does not. I am sure it has something to do w/ the subtle details of how bash parses & expands the various tokens.

gnashley · 09-16-2005, 02:56 PM

What about:
(mkdir ../build-dir) && (cd ../build-dir)

Dark_Helmet · 09-16-2005, 03:54 PM

As I understand it, that's not quite the same. The parentheses create a subshell to execute the command in. That will work fine for the mkdir command, but it won't achieve the cd correctly. The subshell will change working directory and then get destroyed. The parent script's execution never changes directory; just the subshell which no longer exists.

Honestly though, I'm not at a computer to check. So I may be completely wrong.

As a side note, I never explicitly said what this script was for. Anyone familiar with the LFS project would likely recognize the steps/commands. It was an attempt to automate the LFS build process, but has since been abandoned. So there's no need to "solve" the problem any more except from a curiosity standpoint.

Dave Kelly · 09-17-2005, 02:23 PM

Suggest you check here:
http://www.tldp.org/LDP/abs/abs-guide.txt.gz

There a lot of reference to the double ampersand and you may get some new insights into its usage.

eddiebaby1023 · 09-18-2005, 06:04 AM

Quote:

I would REALLY like to keep the command in the form "mkdir ../build-dir && cd ../build-dir".

The problem is that you want to change directory even if the mkdir fails, so you're using the wrong construct (execute the second command if the first command returns true). Instead, try something like

Code:

[ ! -d ../build_dir ] && mkdir ../build_dir ; cd ../build_dir

That still gets you Brownie points for using &&, but is logically correct (I think).

Dark_Helmet · 09-18-2005, 11:01 AM

Nope, the command conditionally changes directory if the mkdir exits successfully. Bash implements logical "short circuits" when it comes to that operator. To illustrate, let me use a few examples (not shell syntax, just regular logic):

Logical AND
if A & B then C
The system evaluates whether A is true or false. If A is true, then B needs to be evaluated. If A is false, there's nothing that B can equal to make the "A & B" statement true. So there's no need to evaluate B; the statement as a whole is false.

Logical OR
if A | B then C
The system evaluates whether A is true or false. If true, then B is skipped. Again, because if A is true, there's nothing that B can equal to make "A | B" false. If A is false, then B is evaluated.

In the command above, the equivalent of a logical AND is used. If it's picked apart, we get:
if A & B then C
A = "mkdir ../build-dir"
B = "cd ../build-dir"
There is no C, because "A && B" is interested only in accomplishing the "side effects" of the commands from a logical standpoint

Evaluate A: "mkdir ../build-dir"
If that command returns true, then the script needs to evaluate B ("cd ../build-dir"). If the mkdir command returns false, we skip evaluation of B. In this case, there's an error for A, and we don't want to cd into the directory because it may not exist. So by skipping evaluation of B (since A was false) the script avoids commiting an error, which is what's desired.

Clear as mud, I know... sorry.

eddiebaby1023 · 09-18-2005, 03:40 PM

Quote:

Nope, the command conditionally changes directory if the mkdir exits successfully.

That's what I said. Your post seemed to be saying that you wanted to change directory because the directory existed, even though the mkdir failed.

BTW, thanks for the patronising logic lesson - I've been writing shellscripts since 1980, and I think I've already got the hang of the shell's "short-circuiting" by now.