I am trying to download, using ftp, a set of files on a nightly basis. The download runs like a champ when executed from the command line. I do have a .netrc file that works when I run it from the command line, but cron doesn't seem to use it. When cron tries to run the script, it hangs. I included the pertinent information below. Any thoughts on why the script hangs when being initiated from cron?

Your help is appreciated.

Here is the crontab statement that I am using:

0 22 * * * /<user_home_dir>/scripts/nightly_ftp.sh &>/<user_home_dir>/log/ftp.log

Here is a copy of the script that I am executing.

#!/bin/bash
# clear directory for new files
#rm /<user_home_dir>/log/ftp.log
rm /<user_home_dir>/data/download/*.txt
rm /<user_home_dir>/data/download/*.gz

# get the value of yesterday's date
. /<user_home_dir>/scripts/yesterday.sh `date +%m`/`date +%d`/`date +%Y`

# determine file name of image files
landFile=pics-land-$yesterday.tar
resFile=pics-residential-$yesterday.tar
comFile=pics-commercial-$yesterday.tar
multiFile=pics-multi-family-$yesterday.tar

# download latest listing files
ftp -n <ftpsite> <<End-of-Session
user <uid> <pwd>
get listings-commercial.txt.gz /<user_home_dir>/data/listings-commercial.txt.gz
get listings-land.txt.gz /<user_home_dir>/data/listings-land.txt.gz
get listings-multi-family.txt.gz /<user_home_dir>/data/listings-multi-family.txt.gz
get listings-residential.txt.gz /<user_home_dir>/data/listings-residential.txt.gz
get $landFile /<user_home_dir>/data/images/$landFile
get $resFile /<user_home_dir>/data/images/$resFile
get $comFile /<user_home_dir>/data/images/$comFile
get $multiFile /<user_home_dir>/data/images/$multiFile
bye
End-of-Session

mv listings* /<user_home_dir>/data/download

exit 0


And here is the information in the log file.

rm: cannot remove `/home/highland/data/download/*.gz': No such file or directory
09/14/2004
Connected to <remote addr> (<remote addr>).
220 FTP Server ready.

The file not found is no big deal, the date value shows that the yesterday script is executing correctly, and you can see that I'm connecting successfully to the remote site, but then the process just hangs.

I'm open to suggestions.

Thanks again.

The fix should be simple enough:

for I in /<user_home_dir>/data/download/*.txt do
rm $I
done

if that causes problems, you can always try and escape the * character with \* (just a thought).

Aaron

Thanks for the reply, but I'm not sure how that addresses the issue.

The issue is not in clearing up the .txt files on the local machine.

The issue is getting the ftp script to execute from cron. As you can see from the log file excerpt, it connects to the remote machine, but for some reason it does not issue the "get" commands in the ftp script. I'm curious why those commands are not being issued after a successful login.

When I run it from the command line, the script runs just fine, and retrieves all of the files that I am requesting.

So why, when I run the script through cron, does it not issue the "get" statements in the ftp script?

All help appreciated.

try inserting
set -xv
before it gets to the ftp section. That will tell you in detail exactly what it's trying to do.

Or did you try to send command from a cat pipe like :

cat <<END | ftp -n <adress>
user <uid> <pwd>
get listings-commercial.txt.gz /<user_home_dir>/data/listings-commercial.txt.gz
get listings-land.txt.gz /<user_home_dir>/data/listings-land.txt.gz
...
END

Just an idea, did you use the username and password in front of your crontab


I'm looking at doing the same thing, using FTP to upload a group of files to another Server.

TT

a cron environment issue?
have you tried
. ~/.profile in the script?

try it on the at daemon.
batch
this works with the current environment.

No I haven't try it, U kind of lost me there.unless U were taking to the other guy

TT

well, talking to the OP of course!

Well, I guess, if the log is showing that the connection was successful, the requirement of username password etc. shouldn't be the issue.

Probabaly you can increase the verbosity of the ftp session by adding some "-v" flags.

Billy's idea of trying this using "at" is also useful.

HTH

cron and ftp work fine for me.

Are you relying on .netrc?

man ftp

Hi guys/gals

any of you people have a sample of a working FTP .sh script, I've just finshed working with FTP comands from the command line.

I connected to my brothers machine 270km's from were I am so I know that FTP from the command line works.

TT

read the man page

make a .netrc file in HOME.
chmod .netrc to 600. (It won't work with go+r read permissions)

here you put a list of machines, users and passwords.
e.g:
macdef init is a macro that's run on each login.
with this done, you can simply write a file of commands and
cat then into ftp. Or you can write more macros with macdef.