[SOLVED] bash script runs perfectly when started manually

jonasbdk · 08-23-2014, 02:07 AM

Hi guys,

First post here - but long time reader.

I have a small bash problem, which seems to be something I have done wrong in the scripts - but I can't identify the problem.

I am using Raspbian on my Raspberry Pi, and I wish to create a mirror of two harddrives using the RSYNC function via a cron job. And so far it has been working, but after I made some amendments to the bash, it will no longer run correctly as a cron job.

The cron job looks like this:

Code:

05 07 * * * cd /home/pi/.scripts && sh rsync.sh

And the bash looks like this:

Code:

#!/bin/bash

#Get timestamp
time="$(date +"%k%M")"

#Geting date
datestamp="$(date +"%Y%m%d")"
year="$(date +"%Y")"

#Script runs from this time
time_begin="658"

#Script runs before this time
time_end="1200"

# The Sync

        echo 'Syncronizing files'  >> /media/slave/.rsync-log/rsync-$datestamp.log
if [[ "$time" -ge "$time_begin"  ]] && [[ "$time" -le  "$time_end" ]];then

        rsync -av --log-file=/media/slave/.rsync-log/rsync-$datestamp.log --delete --exclude=".*" /media/slave/files/          /media/master/files/
        sleep 5

else

        echo 'It is not time to sync the files' >> /media/slave/.rsync-log/rsync-$datestamp.log

fi

As you can see if the time is between 6:58 and 12:00 hrs, then the sync should run - and if not, it returns "It is not time to sync the files" and but that in the log file.

When I run the script manually between the said times it works like a charm. The sync starts and the info is put in the log file as requested.

However, when CRON runs the script it returns the "It is not time to sync the files" information, no matter what time of the day it's run including the previously mentioned time intervals. The script still puts the info successfully into the log file though, which tells me that part works OK.

Can you guys identify the problem as I seem to be at a dead end right now.

Thanks in advances.

Jonas

wpeckham · 08-23-2014, 06:19 AM

There is not enough here to be sure we can identify the problem, but....
Why not call the script from crontab using the fulel path name and avoid stacking two commands?
ie, instead of 'cd path && sh script', make the script executable and just use the command 'path/script'.

Check the PATH used by cron as opposed to the PATH in your session. It is very likely that they differ, and that can cause problems. To resolve, se the PATH environment variable correctly early in the script.

Once you have experimented with these factors, return to describe your results.

pwalden · 08-23-2014, 05:58 PM

Also, Why bother testing for the time window when you know cron will run the rsync everyday at 7:05 AM?

You can also add the set -x command in the script so that it spits out the word expansions and trace.

jonasbdk · 08-24-2014, 03:41 AM

Quote:

Originally Posted by wpeckham

There is not enough here to be sure we can identify the problem, but....
Why not call the script from crontab using the fulel path name and avoid stacking two commands?
ie, instead of 'cd path && sh script', make the script executable and just use the command 'path/script'.

Check the PATH used by cron as opposed to the PATH in your session. It is very likely that they differ, and that can cause problems. To resolve, se the PATH environment variable correctly early in the script.

Once you have experimented with these factors, return to describe your results.

Thanks for your swift reply. But how do I check which PATH is used by cron?

Beryllos · 08-24-2014, 04:28 AM

My guess: The interpreter is ignoring the

Code:

#!/bin/bash

and the script is running under a different shell. Then the "if" command may fail due to a syntax error, perhaps because double brackets [[ ]] are not interpreted in the way you expected.

Note added in a later edit: A syntax error within an if command doesn't necessarily give an error message because the if command will still do something. It just won't do what you expected. Thus it might be correct to say the if command fails due to a logic error, but the logic error is due to syntax.

There may be several ways to correct this.

The method that wpeckham suggested will probably work because (I think) it uses bash by default (Edit: actually the reason is because this method of invocation uses the #! directive):

Quote:

Originally Posted by wpeckham

Why not call the script from crontab using the full path name and avoid stacking two commands?
ie, instead of 'cd path && sh script', make the script executable and just use the command 'path/script'.

That's what I would have done. In other words, your cron line would look like this:

Code:

05 07 * * * /home/pi/.scripts/rsync.sh

And that should work as long as the script file rsync.sh has its permission set to executable for the user in whose account cron is running.

Another possible solution is to run the script (in cron) with bash instead of sh.

I agree with pwalden that you could eliminate the time test altogether, since cron would only ever run the script at the specified time.

wpeckham · 08-24-2014, 11:21 AM

Actually, the cron simply uses the default system path, often very minimal.
To be safe, check the location of every executable you will call using the script (date for example, is an external, as is rsync) and make sure you set a path in the script to includes all of those locations.

I also agree with the above comment about the time checks. If you are using crontab to CONTROL when the script fires off, you do not really need a second check within the script.

There are other considerations here, involving effiency and CPU cycles, but they are trivial. The more important thing is what goes on in your mind as you read the script. Keep it simple, clear, and elegant for ease of maintenance over the long term.

pwalden · 08-24-2014, 12:54 PM

Quote:

Originally Posted by jonasbdk

Thanks for your swift reply. But how do I check which PATH is used by cron?

Add this line to your script

echo "PATH = $PATH";
echo "SHELL = $SHELL";

It should print something in your log like:

PATH = /bin:/usr/bin:/usr/sbin
SHELL = /bin/bash

You could also print out your other script variables as well.

Alternatively, you could just execute the env command in the script.

catkin · 08-25-2014, 03:36 AM

The most robust solution is for the script to set the PATH it needs; that works regardless of how cron is configured.

For example (including some other defensive settings):

Code:

    # Configure shell environment
    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~
    export PATH=/usr/sbin:/sbin:/usr/bin:/bin
    IFS=$' \n\t'
    unset CDPATH                # Ensure cd behaves as expected
    umask 022

jonasbdk · 08-26-2014, 01:25 AM

Quote:

Originally Posted by pwalden

Add this line to your script

echo "PATH = $PATH";
echo "SHELL = $SHELL";

It should print something in your log like:

PATH = /bin:/usr/bin:/usr/sbin
SHELL = /bin/bash

You could also print out your other script variables as well.

Alternatively, you could just execute the env command in the script.

The output from env was:

Code:

TERM=xterm
SHELL=/bin/bash
SSH_CLIENT=192.168.1.120 60644 22
SSH_TTY=/dev/pts/1
JRE_HOME=/usr/lib/jvm/jdk1.7.0/jre
USER=pi
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:
MAIL=/var/mail/pi
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games:/usr/lib/jvm/jdk1.7.0/:/usr/lib/jvm/jdk1.7.0/jre
PWD=/home/pi/rpi-dkelly.smb
JAVA_HOME=/usr/lib/jvm/jdk1.7.0/
LANG=en_GB.UTF-8
SHLVL=1
HOME=/home/pi
LOGNAME=pi
SSH_CONNECTION=192.168.1.120 60644 192.168.1.190 22
OLDPWD=/home/pi
_=/usr/bin/env

I tried this.

Quote:

Originally Posted by catkin

The most robust solution is for the script to set the PATH it needs; that works regardless of how cron is configured.

For example (including some other defensive settings):

Code:

    # Configure shell environment
    # ~~~~~~~~~~~~~~~~~~~~~~~~~~~
    export PATH=/usr/sbin:/sbin:/usr/bin:/bin
    IFS=$' \n\t'
    unset CDPATH                # Ensure cd behaves as expected
    umask 022

Unfortunately this input failed. I could not install a new crontab.

Code:

crontab: installing new crontab
"/tmp/crontab.ZCmdYs/crontab":7: bad command
errors in crontab file, can't install.
Do you want to retry the same edit? (y/n)

jonasbdk · 08-26-2014, 02:12 AM

Quote:

Originally Posted by Beryllos

My guess: The interpreter is ignoring the

Code:

#!/bin/bash

and the script is running under a different shell. Then the "if" command may fail due to a syntax error, perhaps because double brackets [[ ]] are not interpreted in the way you expected.

Note added in a later edit: A syntax error within an if command doesn't necessarily give an error message because the if command will still do something. It just won't do what you expected. Thus it might be correct to say the if command fails due to a logic error, but the logic error is due to syntax.

There may be several ways to correct this.

The method that wpeckham suggested will probably work because (I think) it uses bash by default (Edit: actually the reason is because this method of invocation uses the #! directive):

That's what I would have done. In other words, your cron line would look like this:

Code:

05 07 * * * /home/pi/.scripts/rsync.sh

And that should work as long as the script file rsync.sh has its permission set to executable for the user in whose account cron is running.

Another possible solution is to run the script (in cron) with bash instead of sh.

I agree with pwalden that you could eliminate the time test altogether, since cron would only ever run the script at the specified time.

This worked. I just removed the sh in front of the path to the script, as it was executable already.

Man, it's hard to be a newbie to this bash thing

Anyway - thank you very much for your help.

Beryllos · 08-26-2014, 02:45 AM

Quote:

Originally Posted by jonasbdk

This worked. I just removed the sh in front of the path to the script, as it was executable already.

Man, it's hard to be a newbie to this bash thing

Anyway - thank you very much for your help.

Glad to hear it!

If it's no trouble, go back to my previous post and click where it says Did you find this post helpful? Thanks.
(Pardon me, since you're a new member, that option may not be available. Nevermind!
but you can still mark the thread as solved.)

Beryllos · 08-26-2014, 03:07 AM

It looks like you have solved your original problem, but would you like to understand what went wrong here?

Quote:

Originally Posted by jonasbdk

The output from env was:

...

I tried this.

...

Unfortunately this input failed. I could not install a new crontab.

...

Code:

crontab: installing new crontab
"/tmp/crontab.ZCmdYs/crontab":7: bad command
errors in crontab file, can't install.
Do you want to retry the same edit? (y/n)

I think you did something different from what they suggested.

First you put the env command in the script, and redirect its output to an output file:

Code:

#!/bin/bash
env >> temp.txt

Check that the script is executable, or make it so.

Edit crontab so that it will execute the script:

Code:

00 04 * * * /home/beryllos/temp.sh

Wait until the appointed time and then inspect the output:

Code:

beryllos@op745:~$ cat temp.txt
SHELL=/bin/sh
PATH=/usr/bin:/bin
PWD=/home/beryllos
LANG=en_US.UTF-8
SHLVL=1
HOME=/home/beryllos
LOGNAME=beryllos
_=/usr/bin/env
beryllos@op745:~$

That's funny! It shows SHELL=/bin/sh, not /bin/bash. Is that significant? On my system, apparently not. My scripts running under crontab seem to be interpreted according to bash syntax, regardless of this SHELL=/bin/sh environment variable. I am not sure why. Perhaps, I guess, it is in fact following the #!/bin/bash directive, without regard to the SHELL variable.

Edit: Confirm that. There is another method to determine which command interpreter is being used for a script:

See this from Stack Exchange: Does the shebang determine the shell which runs the script?

If I use this in my script:

Code:

readlink /proc/$$/exe

and run it by crontab, the output is:

Code:

/bin/bash

I'm not exactly a newbie, but I still have a lot to learn about Linux...

catkin · 08-27-2014, 05:54 AM

Quote:

Originally Posted by jonasbdk

I tried this.

Unfortunately this input failed. I could not install a new crontab.

Code:

crontab: installing new crontab
"/tmp/crontab.ZCmdYs/crontab":7: bad command
errors in crontab file, can't install.
Do you want to retry the same edit? (y/n)

The lines were intended for the script, not for the crontab.