LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 09-16-2004, 12:10 PM   #1
grhansen
LQ Newbie
 
Registered: Sep 2004
Posts: 2

Rep: Reputation: 0
Question FTP Script will not run from Cron


I am trying to download, using ftp, a set of files on a nightly basis. The download runs like a champ when executed from the command line. I do have a .netrc file that works when I run it from the command line, but cron doesn't seem to use it. When cron tries to run the script, it hangs. I included the pertinent information below. Any thoughts on why the script hangs when being initiated from cron?

Your help is appreciated.

Here is the crontab statement that I am using:

0 22 * * * /<user_home_dir>/scripts/nightly_ftp.sh &>/<user_home_dir>/log/ftp.log

Here is a copy of the script that I am executing.

#!/bin/bash
# clear directory for new files
#rm /<user_home_dir>/log/ftp.log
rm /<user_home_dir>/data/download/*.txt
rm /<user_home_dir>/data/download/*.gz

# get the value of yesterday's date
. /<user_home_dir>/scripts/yesterday.sh `date +%m`/`date +%d`/`date +%Y`

# determine file name of image files
landFile=pics-land-$yesterday.tar
resFile=pics-residential-$yesterday.tar
comFile=pics-commercial-$yesterday.tar
multiFile=pics-multi-family-$yesterday.tar

# download latest listing files
ftp -n <ftpsite> <<End-of-Session
user <uid> <pwd>
get listings-commercial.txt.gz /<user_home_dir>/data/listings-commercial.txt.gz
get listings-land.txt.gz /<user_home_dir>/data/listings-land.txt.gz
get listings-multi-family.txt.gz /<user_home_dir>/data/listings-multi-family.txt.gz
get listings-residential.txt.gz /<user_home_dir>/data/listings-residential.txt.gz
get $landFile /<user_home_dir>/data/images/$landFile
get $resFile /<user_home_dir>/data/images/$resFile
get $comFile /<user_home_dir>/data/images/$comFile
get $multiFile /<user_home_dir>/data/images/$multiFile
bye
End-of-Session

mv listings* /<user_home_dir>/data/download

exit 0


And here is the information in the log file.

rm: cannot remove `/home/highland/data/download/*.gz': No such file or directory
09/14/2004
Connected to <remote addr> (<remote addr>).
220 FTP Server ready.

The file not found is no big deal, the date value shows that the yesterday script is executing correctly, and you can see that I'm connecting successfully to the remote site, but then the process just hangs.

I'm open to suggestions.

Thanks again.
 
Old 09-16-2004, 01:45 PM   #2
orgcandman
Member
 
Registered: May 2002
Location: dracut MA
Distribution: Ubuntu; PNE-LE; LFS (no book)
Posts: 594

Rep: Reputation: 102Reputation: 102
The fix should be simple enough:

for I in /<user_home_dir>/data/download/*.txt do
rm $I
done

if that causes problems, you can always try and escape the * character with \* (just a thought).

Aaron
 
Old 09-16-2004, 02:43 PM   #3
grhansen
LQ Newbie
 
Registered: Sep 2004
Posts: 2

Original Poster
Rep: Reputation: 0
Thanks for the reply, but I'm not sure how that addresses the issue.

The issue is not in clearing up the .txt files on the local machine.

The issue is getting the ftp script to execute from cron. As you can see from the log file excerpt, it connects to the remote machine, but for some reason it does not issue the "get" commands in the ftp script. I'm curious why those commands are not being issued after a successful login.

When I run it from the command line, the script runs just fine, and retrieves all of the files that I am requesting.

So why, when I run the script through cron, does it not issue the "get" statements in the ftp script?

All help appreciated.
 
Old 09-16-2004, 03:54 PM   #4
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.6, Centos 5.10
Posts: 16,324

Rep: Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041
try inserting
set -xv
before it gets to the ftp section. That will tell you in detail exactly what it's trying to do.
 
Old 09-16-2004, 05:32 PM   #5
Cedrik
Senior Member
 
Registered: Jul 2004
Distribution: Slackware
Posts: 2,140

Rep: Reputation: 242Reputation: 242Reputation: 242
Or did you try to send command from a cat pipe like :

cat <<END | ftp -n <adress>
user <uid> <pwd>
get listings-commercial.txt.gz /<user_home_dir>/data/listings-commercial.txt.gz
get listings-land.txt.gz /<user_home_dir>/data/listings-land.txt.gz
...
END
 
Old 01-12-2006, 05:59 AM   #6
tommytomato
Member
 
Registered: Nov 2003
Location: Narrogin Western Australia
Distribution: GUI Ubuntu 12.04 - Server 12.04.4 LTS
Posts: 935

Rep: Reputation: 32
Quote:
0 22 * * * /<user_home_dir>/scripts/nightly_ftp.sh &>/<user_home_dir>/log/ftp.log
Just an idea, did you use the username and password in front of your crontab


I'm looking at doing the same thing, using FTP to upload a group of files to another Server.

TT
 
Old 01-12-2006, 06:46 AM   #7
bigearsbilly
Senior Member
 
Registered: Mar 2004
Location: england
Distribution: FreeBSD, Debian, Mint, Puppy
Posts: 3,314

Rep: Reputation: 175Reputation: 175
a cron environment issue?
have you tried
. ~/.profile in the script?

try it on the at daemon.
batch
this works with the current environment.
 
Old 01-12-2006, 06:51 AM   #8
tommytomato
Member
 
Registered: Nov 2003
Location: Narrogin Western Australia
Distribution: GUI Ubuntu 12.04 - Server 12.04.4 LTS
Posts: 935

Rep: Reputation: 32
Quote:
~/.profile in the script
No I haven't try it, U kind of lost me there.unless U were taking to the other guy

TT

Last edited by tommytomato; 01-12-2006 at 06:52 AM.
 
Old 01-12-2006, 07:04 AM   #9
bigearsbilly
Senior Member
 
Registered: Mar 2004
Location: england
Distribution: FreeBSD, Debian, Mint, Puppy
Posts: 3,314

Rep: Reputation: 175Reputation: 175
well, talking to the OP of course!
 
Old 01-12-2006, 08:08 AM   #10
dustu76
Member
 
Registered: Sep 2004
Distribution: OpenSuSe
Posts: 153

Rep: Reputation: 30
Well, I guess, if the log is showing that the connection was successful, the requirement of username password etc. shouldn't be the issue.

Probabaly you can increase the verbosity of the ftp session by adding some "-v" flags.

Code:
ftp -v -v -n <remote_ip_addr>
Billy's idea of trying this using "at" is also useful.

HTH
 
Old 01-12-2006, 09:00 AM   #11
bigearsbilly
Senior Member
 
Registered: Mar 2004
Location: england
Distribution: FreeBSD, Debian, Mint, Puppy
Posts: 3,314

Rep: Reputation: 175Reputation: 175
cron and ftp work fine for me.

Are you relying on .netrc?

man ftp
Code:
 -n    Does not attempt "auto-login" upon initial connection.
           If  auto-login  is not disabled, ftp checks the .netrc
           file  in  the  user's  home  directory

Last edited by bigearsbilly; 01-12-2006 at 09:01 AM.
 
Old 01-12-2006, 09:58 PM   #12
tommytomato
Member
 
Registered: Nov 2003
Location: Narrogin Western Australia
Distribution: GUI Ubuntu 12.04 - Server 12.04.4 LTS
Posts: 935

Rep: Reputation: 32
Hi guys/gals

any of you people have a sample of a working FTP .sh script, I've just finshed working with FTP comands from the command line.

Code:
ftp> cd uploading
250 CWD command successful.
ftp> put database.sql
local: database.sql remote: database.sql
227 Entering Passive Mode (xxx,xx,xxx,xxx,62,95)
125 Data connection already open; Transfer starting.
226 Transfer complete.
607177 bytes sent in 19 seconds (31 Kbytes/s)
ftp> bye
421 Timeout (120 seconds): closing control connection.
I connected to my brothers machine 270km's from were I am so I know that FTP from the command line works.

TT
 
Old 01-13-2006, 05:13 AM   #13
bigearsbilly
Senior Member
 
Registered: Mar 2004
Location: england
Distribution: FreeBSD, Debian, Mint, Puppy
Posts: 3,314

Rep: Reputation: 175Reputation: 175
read the man page

make a .netrc file in HOME.
chmod .netrc to 600. (It won't work with go+r read permissions)

here you put a list of machines, users and passwords.
e.g:
Code:
machine ftp.vim.org
login anonymous
password billgates@google.com
macdef init
hash
prompt

machine blah
...
macdef init is a macro that's run on each login.
with this done, you can simply write a file of commands and
cat then into ftp. Or you can write more macros with macdef.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
run shell script on cron varunbihani Linux - Newbie 5 07-08-2005 02:50 AM
can't get cron to run my script. dr_zayus69 Linux - Software 6 04-12-2005 07:34 PM
howto run a script from cron every 5 minutes except 11pm-8am ? cccc Programming 2 03-20-2005 04:52 AM
How to write a script to run bitdefender antivirus as a cron job? ginda Linux - Software 1 03-10-2005 09:08 PM
shell script fo run auto job in cron JolynnMarie LinuxQuestions.org Member Intro 0 04-28-2004 12:21 PM


All times are GMT -5. The time now is 03:44 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration