LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 12-06-2001, 12:57 PM   #1
w0rmh0l3
Member
 
Registered: Nov 2001
Distribution: Mandrake 8.1
Posts: 30

Rep: Reputation: 15
Crontab Help


I have the following crontab job setup to run a certain stats program, and then have my upload script run about 30 minutes later. As seen below..

My crontab:
30 * * * * nice /usr/local/halflife/psychostats1.7/stats
0 * * * * /bin/stat_upload

Where stat_upload is defined as:
#!/bin
cd /var/www/html/stats
ftp -i -n -v <<-EOD
open #my server IP#
user #my username# #my password#
cd /var/www/html/stats
passive
ascii
mput *.html
quit
EOD

The stats script is executed perfectly, the stat_upload script is not. I can verify this by retrieving the stats from the webserver running on that machine (/var/www/html/). The webserver it should be transferring the stats info to however never receives them. It's as if the crontab job is skipping stat_upload entirely. Anyone know what the problem could be?

Also: is it possible to create a link under /etc/cron.hourly/ to /bin/stat_upload to have it run that every hour? I might try this alternative if crontab refuses to do it the other way.

Last edited by w0rmh0l3; 12-06-2001 at 01:08 PM.
 
Old 12-06-2001, 09:51 PM   #2
w0rmh0l3
Member
 
Registered: Nov 2001
Distribution: Mandrake 8.1
Posts: 30

Original Poster
Rep: Reputation: 15
BTW - what does that EOD mean in the script?
 
Old 12-07-2001, 06:25 AM   #3
Mik
Senior Member
 
Registered: Dec 2001
Location: The Netherlands
Distribution: Ubuntu
Posts: 1,316

Rep: Reputation: 46
Well what your doing is passing commands to the standard input of the ftp program. In a shell you can tell it where to start the commands and where to end the commands which will be sent to standard input by using a limitstring. The limit string can basically be anything but it's best to choose something pretty unique.

I have it something like:

ftp -n servername <<End-Of-Section
user name password
binary
cd directory
put filename
quit
End-Of-Section

Adding a - like <<-End-Of-Section would make it strip off the tabs which might be in front of the command, it wouldn't affect spaces though. Like if all your commands would have a leading tab because you like indenting in your scripts, the leading tabs wouldn't get sent to the standard in of the ftp program.

Anyways you either have to remove the first line or define it properly, change it to something like:
#!/bin/bash

Or change bash to whatever shell script you use. Leaving it at #!/bin will make it try to run the script using /bin which is a directory so it will give you a permission denied error.
Another thing which isn't necessary but you could remove the -v option in the ftp command so it doesn't display every step it takes but just does it's work quietly.

Any executable script you place in /etc/cron.hourly will be run every hour.
 
Old 12-07-2001, 08:57 AM   #4
w0rmh0l3
Member
 
Registered: Nov 2001
Distribution: Mandrake 8.1
Posts: 30

Original Poster
Rep: Reputation: 15
Ahh okay, I will try that. It's going to be interesting, I just fileted my pinky finger open w/ a knife on accident.
 
Old 12-08-2001, 08:17 AM   #5
w0rmh0l3
Member
 
Registered: Nov 2001
Distribution: Mandrake 8.1
Posts: 30

Original Poster
Rep: Reputation: 15
Okay I tried all of that and nothing happened.

Apparently from what people have told me, the FTP upload script works only with 1 file and not multiple ones. Something is weird there.. can someone maybe help me out with a different transfer program that could do the same thing just w/o using the command line built in FTP?
 
Old 12-08-2001, 09:13 AM   #6
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,397

Rep: Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963Reputation: 1963
as opposed to doing it on purpose....

erm, you might want to try looking at ftpfs, should do what you want, in a different way. there's a thread here called 'How to mount an ftp site' i think if you need any points
 
Old 12-08-2001, 12:06 PM   #7
DavidPhillips
Guru
 
Registered: Jun 2001
Location: South Alabama
Distribution: Fedora / RedHat / SuSE
Posts: 7,154

Rep: Reputation: 56
ncftp will get multiple files and can also be run in batch mode which will continue to connect and retrieve until it gets the files.
 
Old 12-10-2001, 10:36 AM   #8
w0rmh0l3
Member
 
Registered: Nov 2001
Distribution: Mandrake 8.1
Posts: 30

Original Poster
Rep: Reputation: 15
Does ncftp come with Mandrake 8.1, or is it something separate that I have to download?
 
Old 12-10-2001, 08:12 PM   #9
w0rmh0l3
Member
 
Registered: Nov 2001
Distribution: Mandrake 8.1
Posts: 30

Original Poster
Rep: Reputation: 15
I downloaded ncftp and started using it.. seems to work pretty nicely, except that it uploads files very very slowly. I have at least a thousand or more files that need to transfer, and even though I get about 850-1000 KB/sec to the server, each file takes about 1 second. Any ideas for how to speed it up? This is my command line:

/usr/local/bin/ncftpput -u <user> -p <password> -a -E <server> html/stats/ *.html
 
Old 12-11-2001, 09:37 AM   #10
Mik
Senior Member
 
Registered: Dec 2001
Location: The Netherlands
Distribution: Ubuntu
Posts: 1,316

Rep: Reputation: 46
Uploading multiple files works fine. I copied your script and it uploaded all the html files. Have you tried testing the script without using crontab. Maybe it's because I tested it with 3 files instead of a thousand. But I don't see why that should really make a difference.
I don't really know how ncftp works but is it by any chance opening a new connection for every file it needs to transfer? You could try the -b or -bb options.
 
Old 12-11-2001, 10:09 AM   #11
w0rmh0l3
Member
 
Registered: Nov 2001
Distribution: Mandrake 8.1
Posts: 30

Original Poster
Rep: Reputation: 15
Question

Yes, the upload script using /usr/bin/ftp works fine when I run it from a console - but crontab will not run it. It's a problem with crontab + /usr/bin/ftp.

The ncftpput upload works fine (haven't tested it w/ crontab) but it is extremely slow. Yes, it opens a new port for each connection - but so does /usr/bin/ftp. It seems to me though that ncftp is purposely controlling each connection, making a maximum of 1 connection/upload per second. The files themselves are transferring at anywhere from 850 KB/sec to 1 MB/sec, but the transfers all still take 1 second a piece (I see this from the verbose output of ncftpput.)

Anyone know how to speed up ncftpput?
 
Old 12-11-2001, 05:06 PM   #12
w0rmh0l3
Member
 
Registered: Nov 2001
Distribution: Mandrake 8.1
Posts: 30

Original Poster
Rep: Reputation: 15
Date: Tue, 11 Dec 2001 16:24:22 -0600
From: omitted <omitted@ncftp.com>
To: omitted
Subject: Re: ncftpput problem

Yes. There's a bug in ncftpput which we'll fix for the next release
(next week).

On Tuesday, December 11, 2001, at 02:09 PM, omitted wrote:

Quote:
I was wondering - when I use the ncftpput program to transfer a few hundred files, it seems to be choking - the transfer rate is about 850 KB/sec, and yet each file takes exactly 1 second to transfer. Any idea why? This is my command line:

/usr/local/bin/ncftpput -u <user> -p <password> -a -E <server> html/stats/ *.html

Any help is appreciated!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
crontab x2??? killahsmurf Slackware 2 05-24-2005 10:53 PM
Crontab wormraper Debian 4 05-15-2005 01:01 AM
How to use crontab ? Dakkar Linux - Newbie 2 01-22-2004 10:49 AM
system-wide crontab in /etc/crontab ner Linux - General 2 11-18-2003 12:35 PM
HELP!! crontab karleong Linux - General 9 07-12-2003 10:54 AM


All times are GMT -5. The time now is 09:15 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration