Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Hello all, I've visited here for years, just never got around to posting.
I'm looking for a little help writing a script to run on my server (CentOS), and its something I've never done before. I've got these text files and full permission to grab them off a Microsoft-IIS/7.5 server. I'm trying to automate the process with cron jobs.
So far I'm striking out trying to do this with php and I keep wondering if doesn't make sense to at least try a bash or something like it.
Here is the php code that gives this error; Can't open data connection Googling that everyone seems to say add this; ftp_pasv($conn_id, true);
Code:
<?php
ini_set('display_errors',1);
error_reporting(E_ALL);
// define some variables
$folder_path = "/home/name/public_html/test/";
$local_file = "file.txt"; //
$server_file = "file.txt";
//-- Connection Settings
$ftp_server = "msserver"; // Address of FTP server.
$ftp_user_name = "u"; // Username
$ftp_user_pass = "p"; // Password
// set up basic connection
$conn_id = ftp_connect($ftp_server); // I checked, port is 21
// login with username and password
$login_result = ftp_login($conn_id, $ftp_user_name, $ftp_user_pass);
ftp_pasv($conn_id, true);
if(ftp_chdir($conn_id, "folder/webfiles/")) {
echo "Current directory is now: " . ftp_pwd($conn_id) . "\n";
}
// ftp_pasv($conn_id, true);
$lf = fopen($local_file, 'w');
if (!$lf){
die('Failed to create local file.');
}
if (!ftp_fget($conn_id, $lf, $server_file, FTP_ASCII)) {
die('Failed to download file from FTP server.');
}
// ftp_get($conn_id, $local_file, $server_file, FTP_ASCII);
// close the connection
ftp_close($conn_id);
fclose($lf);
?>
Anyway I keep looking at #!/bin/bash scripts ... do you use pico or what?
I'm still interested in any comments about the wisdom of dealing with this issue, or whether it's good or bad form to use a bash script where you just can't get it working in php.
Seems to me there's an argument that server to server, I'm just getting closer to the heart of the matter.
Here goes;
#!/bin/bash
HOST=ftp.server.com #This is the FTP servers host or IP address.
USER=ftpuser #This is the FTP user that has access to the server.
PASS=password #This is the password for the FTP user.
# Call 1. Uses the ftp command with the -inv switches. -i turns off interactive prompting. -n Restrains FTP from attempting the auto-login feature. -v enables verbose and progress.
ftp -inv $HOST << EOF
# Call 2. Here the login credentials are supplied by calling the variables.
user $USER $PASS
# Call 3. Here you will change to the directory where you want to put or get
cd /path/to/file
# Call4. Here you will tell FTP to put or get the file.
put test.txt
# or
get test.txt
bye
EOF
I'm assuming call 3 is for changing directory on the remote server
Last edited by motorcity; 02-05-2013 at 05:01 AM.
Reason: found a start
!/bin/bash
HOST=00.000.000.000 #This is the FTP servers host or IP address.
USER=user #This is the FTP user that has access to the server.
PASS=password #This is the password for the FTP user.
ftp -invd $HOST << EOF
user $USER $PASS
cd /ftp_server/webfiles
lcd /home/name/public_html/admin/test
passive
get skudescadd.txt
bye
EOF
Trying to figure out how to specify the port...
<<<<<<< Got It >>>>>>>>>>>
this line above should be
ftp -invd $HOST 21 << EOF #Got lucky adding the known port
Thanks to all
Last edited by motorcity; 02-05-2013 at 02:47 PM.
Reason: forgot the code
Second issue is closing the ftp session on the remote server. It doesn't seem to understand; bye or EOF
Trying;
disconnect
close
exit
500 'ňABOR': command not understood
Last edited by motorcity; 02-06-2013 at 06:14 AM.
Reason: 500 quote
i would not use the old, deprecated and insecure ftp protocol.
i would turn on ssh (which comes with scp (for automated file transfers) and sftp (for an ftp like interactive prompt (read: babysitting <@)). i would also install sshfs for extra ease-of-use of drag-and-drop local file copy emulation.
i would disable root login and create public keys for each of my clients.
i would not use the old, deprecated and insecure ftp protocol.
i would turn on ssh (which comes with scp (for automated file transfers) and sftp (for an ftp like interactive prompt (read: babysitting <@)). i would also install sshfs for extra ease-of-use of drag-and-drop local file copy emulation.
i would disable root login and create public keys for each of my clients.
Turns out the bash script only works randomly, about once out of maybe a dozen tries. The only thing I can see that changes is the port identifier which is amended to the ip address so I didn't include it in any posting here. I took the problem over to my server admin and they're taking a look at it.
But the thing is I have no way to influence the msII server, that belongs to a huge company and I'm just one of many thousands of their customers. They determine the active connection and plain old FTP on port 21.
I'm pretty sure I was mistaken about closing the ftp session now. It's all about my server not finding a proper address or blocking the ftp port address, something along that line.
I should first thank you all for the help and let you know that the problem came down to white listing the remote server. Why didn't I think of that? We're fortunate to have the hosting support we have.
Now perhaps it's time to end this thread. I've got 11 crons and all tested good the first time through. That covers a bunch of automation for all the text. But I have something of an interesting dilemma with the images.
I need to FTP and rename multiple images, from weekly updates, and I'm trying to come up with a script for this.
The interesting part is the folder names change every week, so at this early point in the year the WebImages folder on the remote server has Week1 - Week5. It wouldn't be so bad to use a wildcard like Week* if it's only five folders with maybe 200 images each, but what about later in the year when there might be Week1 - Week48 (200 * 48 = 9600 images) and only 200 images are actually new at the time?
A suggestion I got elsewhere was to look into using brace expansion running a cron job and/or a separate script to clear old folders and specify a brace number range.
Any thoughts or links or suggestions are greatly appreciated. Especially any that may direct me to where this type of question, (sort of a tricky bash requirement), might find the best response on this website.
%U
week number of year, with Sunday as first day of week (00..53)
%V
ISO week number, with Monday as first day of week (01..53)
%W
week number of year, with Monday as first day of week (00..53)
or write your own algo if remote uses a funky calcn.
However(!), now you've solved the underlying issue (whitelisting) you may want to revert to php.
Imho, if this is a standalone cron job, stick with bash; if its called from Apache, go php.
Totally awesome links there Chris!
Thank you very much!
Quote:
if this is a standalone cron job
It's still sort of a toss up; Eight of the eleven crons are already php but this one (images) and the other 3 FTP updates make sense as bash because they don't need to be called from the website, they're following some other entities schedule where we just take what they give.
On the other hand, this script with renaming the files & changing folder numbers is more complex than the simple bash files I'm currently testing.
This project of automating what I've been doing very manually has been long overdue. I'm looking for the easiest path to running on autopilot here. So once again I thank you for the consideration, and appreciate hearing your opinion.
~Motorcity
If its time based, use cron, if event based (based on something happening in the website), use Apache to call php.
I believe there is a pure cli (non-apache) way to use php, but I've rarely seen it used (& I'm not a php guy anyway...)
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.