w0rmh0l3

[Log in to get rid of this advertisement]

I have the following crontab job setup to run a certain stats program, and then have my upload script run about 30 minutes later. As seen below..

My crontab:
30 * * * * nice /usr/local/halflife/psychostats1.7/stats
0 * * * * /bin/stat_upload

Where stat_upload is defined as:
#!/bin
cd /var/www/html/stats
ftp -i -n -v <<-EOD
open #my server IP#
user #my username# #my password#
cd /var/www/html/stats
passive
ascii
mput *.html
quit
EOD

The stats script is executed perfectly, the stat_upload script is not. I can verify this by retrieving the stats from the webserver running on that machine (/var/www/html/). The webserver it should be transferring the stats info to however never receives them. It's as if the crontab job is skipping stat_upload entirely. Anyone know what the problem could be?

Also: is it possible to create a link under /etc/cron.hourly/ to /bin/stat_upload to have it run that every hour? I might try this alternative if crontab refuses to do it the other way.

w0rmh0l3

BTW - what does that EOD mean in the script?

Mik

Well what your doing is passing commands to the standard input of the ftp program. In a shell you can tell it where to start the commands and where to end the commands which will be sent to standard input by using a limitstring. The limit string can basically be anything but it's best to choose something pretty unique.

I have it something like:

ftp -n servername <<End-Of-Section
user name password
binary
cd directory
put filename
quit
End-Of-Section

Adding a - like <<-End-Of-Section would make it strip off the tabs which might be in front of the command, it wouldn't affect spaces though. Like if all your commands would have a leading tab because you like indenting in your scripts, the leading tabs wouldn't get sent to the standard in of the ftp program.

Anyways you either have to remove the first line or define it properly, change it to something like:
#!/bin/bash

Or change bash to whatever shell script you use. Leaving it at #!/bin will make it try to run the script using /bin which is a directory so it will give you a permission denied error.
Another thing which isn't necessary but you could remove the -v option in the ftp command so it doesn't display every step it takes but just does it's work quietly.

Any executable script you place in /etc/cron.hourly will be run every hour.

w0rmh0l3

Ahh okay, I will try that. It's going to be interesting, I just fileted my pinky finger open w/ a knife on accident.

w0rmh0l3

Okay I tried all of that and nothing happened.

Apparently from what people have told me, the FTP upload script works only with 1 file and not multiple ones. Something is weird there.. can someone maybe help me out with a different transfer program that could do the same thing just w/o using the command line built in FTP?

acid_kewpie

as opposed to doing it on purpose....

erm, you might want to try looking at ftpfs, should do what you want, in a different way. there's a thread here called 'How to mount an ftp site' i think if you need any points

DavidPhillips

ncftp will get multiple files and can also be run in batch mode which will continue to connect and retrieve until it gets the files.

w0rmh0l3

Does ncftp come with Mandrake 8.1, or is it something separate that I have to download?

w0rmh0l3

I downloaded ncftp and started using it.. seems to work pretty nicely, except that it uploads files very very slowly. I have at least a thousand or more files that need to transfer, and even though I get about 850-1000 KB/sec to the server, each file takes about 1 second. Any ideas for how to speed it up? This is my command line:

/usr/local/bin/ncftpput -u <user> -p <password> -a -E <server> html/stats/ *.html

Mik

Uploading multiple files works fine. I copied your script and it uploaded all the html files. Have you tried testing the script without using crontab. Maybe it's because I tested it with 3 files instead of a thousand. But I don't see why that should really make a difference.
I don't really know how ncftp works but is it by any chance opening a new connection for every file it needs to transfer? You could try the -b or -bb options.

w0rmh0l3

Yes, the upload script using /usr/bin/ftp works fine when I run it from a console - but crontab will not run it. It's a problem with crontab + /usr/bin/ftp.

The ncftpput upload works fine (haven't tested it w/ crontab) but it is extremely slow. Yes, it opens a new port for each connection - but so does /usr/bin/ftp. It seems to me though that ncftp is purposely controlling each connection, making a maximum of 1 connection/upload per second. The files themselves are transferring at anywhere from 850 KB/sec to 1 MB/sec, but the transfers all still take 1 second a piece (I see this from the verbose output of ncftpput.)

Anyone know how to speed up ncftpput?

w0rmh0l3

Date: Tue, 11 Dec 2001 16:24:22 -0600
From: omitted <omitted@ncftp.com>
To: omitted
Subject: Re: ncftpput problem

Yes. There's a bug in ncftpput which we'll fix for the next release
(next week).

On Tuesday, December 11, 2001, at 02:09 PM, omitted wrote: