[SOLVED] Cronjob doesn't run daily as expected

PatrickDickey · 12-23-2011, 02:01 AM

Hi everyone,

I set up a bash script, which is supposed to download two csv files for me. And I want it to run every day under cron. However, no matter what I try (even putting it under the sudo crontab -e or sudo nano /etc/crontab files), it doesn't work. Also no errors are reported, that I know of.

My distro is Ubuntu 11.10 default setup. Here is the crontab -l for my personal user (patrickdickey).

Code:

MAILTO=myemail@gmail.com
# Edit this file to introduce tasks to be run by cron.
# 
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
# 
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').# 
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
# 
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
# 
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
# 
# For more information see the manual pages of crontab(5) and cron(8)
# 
# m h  dom mon dow   command
40 01 * * * /home/patrickdickey/Documents/StockAdvisor/getdata

the results when I do "sudo crontab -l"

Code:

# Edit this file to introduce tasks to be run by cron.
# 
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
# 
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').# 
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
# 
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
# 
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
# 
# For more information see the manual pages of crontab(5) and cron(8)
# 
44 01 * * * /home/patrickdickey/Documents/StockAdvisor/getdata
# m h  dom mon dow   command

(one thing to note is that it's in the wrong place. I *#(*&&*($#*(## hate vi with a passion)

and cat /etc/crontab

Code:

# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=myemail@gmail.com

# m h dom mon dow user	command
17 *	* * *	root    cd / && run-parts --report /etc/cron.hourly
25 6	* * *	root	test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 6	* * 7	root	test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 6	1 * *	root	test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
47 01 * * *     patrickdickey /home/patrickdickey/Documents/StockAdvisor/getdata
#

I don't know if I need to restart cron.d or if I'm doing something wrong. So, if I want this to run at 1:00 every day (0100 daily is it set up right (technically I've got them set up to run at 0140 0144 or 0147 daily, so I want to know if the order is correct--not so much the time)?

I guess my three main questions are these:

1. Which crontab should I set it up in?
2. What is the correct format of the command to run /home/patrickdickey/Documents/StockAdvisor/getdata.sh at 1:00 a.m. daily?
3. What do I need to have configured to have the mailto send the email out to my gmail.com address? (for example user@gmail.com)

I should note that the script does work if I run it manually. One other question that I have is this: Do I need to do a cd /home/patrickdickey/Documents/StockAdvisor in my script, if it's running from that directory? That's the directory that the script and the csv files are located in.

Thanks for any help, and have a great day

Patrick.

fukawi1 · 12-23-2011, 02:20 AM

Quote:

Originally Posted by PatrickDickey

I guess my three main questions are these:

1. Which crontab should I set it up in?
2. What is the correct format of the command to run /home/patrickdickey/Documents/StockAdvisor/getdata.sh at 1:00 a.m. daily?
3. What do I need to have configured to have the mailto send the email out to my gmail.com address? (for example user@gmail.com)

1. Depends on which user you want it to run as, in this instance i would suggest running it as the user, rather than root. By adding it to cron as the user. The script needs to be executable by the user running it.

Code:

$ crontab -e

2. I notice here (in bold) the script has a .sh extention, where in your crontab's it is missing the .sh extention.

Also is crond running?

Code:

ps | grep crond

PatrickDickey · 12-23-2011, 02:50 AM

Quote:

Originally Posted by fukawi1

1. Depends on which user you want it to run as, in this instance i would suggest running it as the user, rather than root. By adding it to cron as the user. The script needs to be executable by the user running it.

Code:

$ crontab -e

2. I notice here (in bold) the script has a .sh extention, where in your crontab's it is missing the .sh extention.

Also is crond running?

Code:

ps | grep crond

I tried it with the .sh, but was reading somewhere that cron doesn't like the '.', so I took that off. When i did a ps | grep cron, it came up with two entries (root and the grep search), but crond has nothing. So, I'm going to assume that crond isn't running (and will start it now).

So, in short, I need to get rid of the lines from /etc/crontab and the sudo crontab files, but leave the one in my crontab. And I should have the .sh at the end of it. Also, make sure crond is running.

Thanks, and have a great day

Patrick.

fukawi1 · 12-23-2011, 03:00 AM

Cron and crond should be the same thing. Some distro's call it crond, ubuntu obviously isnt one of them

You need to specify the path to the file, as it is named in the filesystem, any time i use cron, i use a script with a .sh extension, and have never had a problem.

PatrickDickey · 12-23-2011, 03:11 AM

Quote:

Originally Posted by fukawi1

Cron and crond should be the same thing. Some distro's call it crond, ubuntu obviously isnt one of them

You need to specify the path to the file, as it is named in the filesystem, any time i use cron, i use a script with a .sh extension, and have never had a problem.

Ok, so here's my latest changes. I removed all entries of the command from crontabs (except for my user's). My user's crontab looks like this

Code:

MAILTO=pdickey043@gmail.com
# Edit this file to introduce tasks to be run by cron.
# 
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
# 
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').# 
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
# 
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
# 
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
# 
# For more information see the manual pages of crontab(5) and cron(8)
# 
# m h  dom mon dow   command
05 03 * * * /home/patrickdickey/Documents/StockAdvisor/getdata.sh

I restarted cron at 0300, and it's now 0307. When I do an ls, it doesn't show the csv files (I have two in there that are renamed, so the script *should* delete them, and download the new ones).

Just for clarification, here's the script that it's running

Code:

#!/bin/bash
# set -xv

#Declaration of variables

strtmo=$(date --date='350 days ago' '+%m')
if [ "$strtmo" -eq 1 ]; then
    startmo=0$(($strtmo - 1))
else
    startmo=$(($strtmo -1))
fi
startday=$(date --date='350 days ago' '+%d')
startyr=$(date --date='350 days ago' '+%y')
endm=$(date '+ %m')
if [ "$endm" -eq 1 ]; then
    endmo=0$(($endm - 1))
else
    endmo=$(($endm - 1))
fi
endday=$(date '+%d')
endyr=$(date '+%y')

# Remove any csv files prior to running the script.

rm *.csv

# Get the files.

sturl='http://ichart.finance.yahoo.com/table.csv?s=%5EGSPC&a='$startmo'&b='$startday'&c='$startyr'&d='$endmo'&e='$endday'&f='$endyr'&g=d&ignore=.csv'
sturl2="http://ichart.finance.yahoo.com/table.csv?s=%5EIXIC&a="$startmo"&b="$startday"&c="$startyr"&d="$endmo"&e="$endday"&f="$endyr"&g=d&ignore=.csv"
wget $sturl
wget $sturl2

# Rename the csv files so my macro can find them.

mv table.csv?s=^GSPC* GSPC.csv
mv table.csv?s=^IXIC* IXIC.csv

So, I'm not sure where to go with this. I could always try rebooting, and see if that clears things up. Also, should I put a cd /home/patrickdickey/Documents/StockAdvisor into my getdata.sh script, just to make sure it's in the right directory?

Have a great day, and thanks again.

Patrick.

fukawi1 · 12-23-2011, 03:26 AM

Quote:

Originally Posted by PatrickDickey

Also, should I put a cd /home/patrickdickey/Documents/StockAdvisor into my getdata.sh script, just to make sure it's in the right directory?

Yup, give that a shot. To see if it is working, but downloading to a different directory than expected,

Code:

updatedb && locate *.csv

Its also possible to set a destination file by using the "-O" option with wget.

PatrickDickey · 12-23-2011, 03:51 AM

Yep, it was the cd command. When I put that in the beginning of my script, everything started working. So, apparently I had the csv files somewhere (I got an error when I ran updatedb && locate *.csv about not being able to write to locatedb).

Thanks for your help, and Merry Christmas/Happy Holidays/Seasons Greetings/etc.

Patrick.

fukawi1 · 12-23-2011, 03:59 AM

Oh, i neglected to mention that should be done as root.
Glad it works.