Correct config for cron jobs

mrgordonz · 02-11-2013, 11:09 PM

Hi there,

I am pretty new at creating/configuring cron jobs, so please excuse any glaring errors in my terminology or setup. The distro/version of Linux I am running is Oracle Linux Server release 6.3.

I have three shell scripts which need to be executed each evening, 5 minutes apart (to export three separate Oracle databases).

I can execute the scripts manually, and they work perfectly. For example:

Code:

$ sh ./expdp_caldev.sh

So I added three entries to crontab for the oracle user:

Code:

[oracle@oracle-lp01 dba-tasks]$ crontab -l
45 20 * * * /home/oracle/dba-tasks/expdp_calprod.sh
50 20 * * * /home/oracle/dba-tasks/expdp_caltest.sh
55 20 * * * /home/oracle/dba-tasks/expdp_caldev.sh

I have confirmed that crond is running

Code:

[oracle@oracle-lp01 ~]$ service crond status
crond (pid  2523) is running...

So my expectation is that the scripts would get executed at 8:45 PM, 8:50 PM, and 8:55 PM. But nothing is happening.

Do I need to change the crontab entries as follows:

Code:

[oracle@oracle-lp01 dba-tasks]$ crontab -l
45 20 * * * sh /home/oracle/dba-tasks/expdp_calprod.sh
50 20 * * * sh /home/oracle/dba-tasks/expdp_caltest.sh
55 20 * * * sh /home/oracle/dba-tasks/expdp_caldev.sh

Cheers,

Paul

mrgordonz · 02-12-2013, 12:08 AM

Just a follow-up question:

Does it matter that the crontab entries belong to the oracle user, and not root?

Cheers,

Paul

chrism01 · 02-12-2013, 12:56 AM

1. show the scripts.
More to the point, the default env in cron is minimal/unreliable. ALWAYS use full absolute paths to cmds and files used, or start by sourcing a file that sets up the correct env.
Incidentally, if want to use bash, say bash and not sh; they aren't necessarily the same.
On my system I'd use

Code:

#!/bin/bash

as the first line of each script, in which case no need for external call to bash (or sh

)

For debugging, add this as the 2nd line

Code:

set -xv

then capture all stdout & stderr to a log

Code:

45 20 * * * /home/oracle/dba-tasks/expdp_calprod.sh >/tmp/expdp_calprod.log 2>&1

Note that if cron has a problem it normally emails the cron owner or root

2. owner: given you are backing up an ora db, I'd normally expect oracle to have sufficient privs, but you might need to use root's crontab if it needs extra privs.

HTH
http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/
http://www.adminschoice.com/crontab-quick-reference

mrgordonz · 02-12-2013, 04:56 AM

Here is one of the scripts that needs to get executed. The three scripts are all identical except for the Oracle SID and usernames/passwords.

Code:

#!/bin/bash

# set the environment
export ORACLE_HOME=/opt/oracle/oracle/product/11.2.0/dbhome_1
export ORACLE_BASE=/opt/oracle/oracle
export ORACLE_SID=******
export DMPDIR=/usr/oracle/export
export EXPORT_DIR_NAME=export
export DUMP_FILE=expdp_${ORACLE_SID}.dmp
export LOG_FILE=expdp_${ORACLE_SID}.log
export SYSTEM_USER=system
export SYSTEM_PASSWD=******
export DB_SYSTEM_USER_PASS=${SYSTEM_USER}/${SYSTEM_PASSWD}@${ORACLE_SID}
export INTERNAL_USER=******
export REPORT_USER=******


# change to the working directory
cd ${DMPDIR}

# check for existing logfile and
# remove if found.
#echo Removing log file
if [ -e ${DMPDIR}/${LOG_FILE} ]; then
    rm ${DMPDIR}/${LOG_FILE}
fi

# check for existing dump file and
# remove if found.
#echo Removing old dump file
if [ -e ${DMPDIR}/${DUMP_FILE} ]; then
    rm ${DMPDIR}/${DUMP_FILE}
fi

# call expdp to export DB
expdp ${DB_SYSTEM_USER_PASS} directory=${EXPORT_DIR_NAME} dumpfile=${DUMP_FILE} schemas=${INTERNAL_USER},${REPORT_USER} exclude=statistics logfile=${LOG_FILE}

# Get the log status for email.
export STATUS=`grep Job ${DMPDIR}/${LOG_FILE}`

# mail results.
mailx -s "DB Export Log: ${ORACLE_SID} - ${STATUS}" ******@******.com < ${DMPDIR}/${LOG_FILE}

# rename the log file to preserve it.
if [ -e ${DMPDIR}/${LOG_FILE} ]; then
    mv ${DMPDIR}/${LOG_FILE} ${DMPDIR}/${LOG_FILE}.`date +%y%m%d`
fi

Is there anything in the script which screams "you idiot, what are you doing?" When I run the script manually, it works perfectly - the DB is exported; I get the email with the log results; and the log file is renamed with the date stamp. I execute the script manually as the oracle user.

Regarding bash vs sh, I am not enough of a Linux person to really understand the difference (I'll have to consult Dr Google to find out).

Thanks for the help,

Cheers,

Paul

linosaurusroot · 02-12-2013, 06:03 AM

If you want them all run each evening I'd put them in one cron job. That way if one job takes over 5 minutes the next one will wait for it to finish.

unSpawn · 02-12-2013, 07:02 AM

Quote:

Originally Posted by mrgordonz

Is there anything in the script which screams "you idiot, what are you doing?"

No, more like "why aren't you listening?" as chrism01 said you should export the full path. Taking shell scripting basics and linosaurusroot' suggestion into account here is a rewritten version (one script to rule all them databases) exporting PATH, allowing for per-SID credentials, checking if binaries are found, etc, etc:

Code:

#!/bin/bash --
# Set debug mode while testing:
set -vx
# Process all SIDs:
for CURRENT_SID in orc1 orc2 orc3; do
 export ORACLE_SID="${CURRENT_SID}"
 # Set per-SID credentials below:
 case $ORACLE_SID in
  orc1) export SYSTEM_PASSWD=******; export INTERNAL_USER=******; export REPORT_USER=******
        ;;
  orc2) export SYSTEM_PASSWD=******; export INTERNAL_USER=******; export REPORT_USER=******
        ;;
  orc3) export SYSTEM_PASSWD=******; export INTERNAL_USER=******; export REPORT_USER=******
        ;;
 esac
 # Fill the rest of the environment
 export PATH=$PATH:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/oracle/bin:/home/oracle/app/product/11.2.0/bin:/opt/oracle/bin:/opt/oracle/oracle/product/11.2.0/bin
 export ORACLE_HOME="/opt/oracle/oracle/product/11.2.0/dbhome_1"
 export ORACLE_BASE="/opt/oracle/oracle"
 export DMPDIR="/usr/oracle/export"
 export EXPORT_DIR_NAME="export"
 export DUMP_FILE="expdp_${ORACLE_SID}.dmp"
 export LOG_FILE="expdp_${ORACLE_SID}.log"
 export SYSTEM_USER="system"
 export DB_SYSTEM_USER_PASS="${SYSTEM_USER}/${SYSTEM_PASSWD}@${ORACLE_SID}"
 # Pre-flight check if we can find stuff in current path:
 for BINARY in expdp mailx; do
  which $BINARY >/dev/null 2>&1 || { echo "Cant find "$BINARY" in PATH=\"${PATH}\", exiting."; exit 1; }
 done
 # Pre-flight check CWD and move items:
 cd ${DMPDIR} || { echo "Cant cd, exiting."; exit 1; }
 for ITEM in "${LOG_FILE}" "${DUMP_FILE}"; do
  [ -f "${DMPDIR}/${ITEM}" ] && rm -f "${DMPDIR}/${ITEM}" || { echo "Epic fail removing "${ITEM}", exiting."; exit 1; }
 done
 # Now call expdp to export current DB
 expdp "${DB_SYSTEM_USER_PASS}" directory="${EXPORT_DIR_NAME}" dumpfile="${DUMP_FILE}" schemas="${INTERNAL_USER},${REPORT_USER}" exclude=statistics logfile="${LOG_FILE}"
 [ $? -ne 0 ] && { echo "Error while processing SID: ${CURRENT_SID}."; }
 # Get the log status for email.
 export STATUS=$(grep -m1 Job "${DMPDIR}/${LOG_FILE}")
 # mail results.
 mailx -s "DB Export Log: ${ORACLE_SID} - ${STATUS}" ******@******.com < "${DMPDIR}/${LOG_FILE}"
 # rename the log file to preserve it.
 [ -f ${DMPDIR}/${LOG_FILE} ] && mv -f $"{DMPDIR}/${LOG_FILE}" "${DMPDIR}/${LOG_FILE}.$(/bin/date +'%Y%m%d')"
# Ready to process next SID now.
done
# All done, exit nicely.
exit 0

As always YMMV(VM) so change the exported PATH to include the directories you use and test it on a non-production database first.

mrgordonz · 02-13-2013, 08:57 AM

Hi unSpawn,

My apologies - I thought chrism01 meant to export the full paths to whatever commands or files I was referencing; I didn't realise chrism01 meant to export the PATH variable.

Many thanks for re-writing the shell script - I really appreciate it. Just reading through it has taught me some neat tricks I didn't know which I'm sure will come in handy one day.

I note that you've added the line

Code:

set -vx

which was what chrism01 also recommended. I gather this will generate a log file - where will the log be created? In the same folder as the script?

Cheers,

Paul

unSpawn · 02-13-2013, 12:02 PM

No need to apologize. If you run the script manually it'll output to stdout. If you run the script as cron job then it'll be emailed to the cron job owner or whomever the non-empty MAILTO cron job variable is set to. Next to testing it on a non-production database it may be beneficial to run the script from the CLI first.

mrgordonz · 03-01-2013, 06:49 AM

Hi unSpawn,

Many thanks for your revised shell script. A couple of minor tweaks, and it works nicely (from the CLI). I have configured the crontab as follows:

Code:

45 20 * * * sh /home/oracle/dba-tasks/expdp_databases.sh >/home/oracle/dba-tasks/expdp_databases.log 2>&1

I will find out tomorrow evening whether it works as a cron job.

Cheers,

Paul