LinuxQuestions.org - FTP script Microsoft-IIS/7.5 to Unix

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - FTP script Microsoft-IIS/7.5 to Unix (https://www.linuxquestions.org/questions/linux-newbie-8/ftp-script-microsoft-iis-7-5-to-unix-4175448620/)

motorcity

02-04-2013 08:37 PM

FTP script Microsoft-IIS/7.5 to Unix

Hello all, I've visited here for years, just never got around to posting.

I'm looking for a little help writing a script to run on my server (CentOS), and its something I've never done before. I've got these text files and full permission to grab them off a Microsoft-IIS/7.5 server. I'm trying to automate the process with cron jobs.
So far I'm striking out trying to do this with php and I keep wondering if doesn't make sense to at least try a bash or something like it.
Here is the php code that gives this error;
Can't open data connection Googling that everyone seems to say add this; ftp_pasv($conn_id, true);

Code:

<?php

ini_set('display_errors',1);

error_reporting(E_ALL);

        // define some variables

        $folder_path = "/home/name/public_html/test/";

        $local_file = "file.txt"; //

        $server_file = "file.txt";

        //-- Connection Settings

        $ftp_server = "msserver"; // Address of FTP server.

        $ftp_user_name = "u"; // Username

        $ftp_user_pass = "p"; // Password

        // set up basic connection

        $conn_id = ftp_connect($ftp_server); // I checked, port is 21

        // login with username and password

        $login_result = ftp_login($conn_id, $ftp_user_name, $ftp_user_pass);

ftp_pasv($conn_id, true);

if(ftp_chdir($conn_id, "folder/webfiles/")) {

echo "Current directory is now: " . ftp_pwd($conn_id) . "\n";

}

// ftp_pasv($conn_id, true);

$lf = fopen($local_file, 'w');

if (!$lf){

die('Failed to create local file.');

}

        if (!ftp_fget($conn_id, $lf, $server_file, FTP_ASCII)) {

        die('Failed to download file from FTP server.');

}

// ftp_get($conn_id, $local_file, $server_file, FTP_ASCII);

        // close the connection

        ftp_close($conn_id);

fclose($lf);

?>

Anyway I keep looking at #!/bin/bash scripts ... do you use pico or what?

joshp

02-04-2013 08:59 PM

Hello,

Here is a super basic setup for a ftp transfer using Bash.

Code:

#!/bin/bash



HOST=localhost

USER=someuser

PASS=password



ftp -inv $HOST  << EOF

user $USER $PASS



[COMMANDS HERE]



EOF

motorcity

02-05-2013 04:37 AM

Thanks joshp.
Sorry to the forum for putting the dreaded ms word in my question title.

Found a super simpleton youtubehttp://www.youtube.com/watch?v=QGvvJO5UIs4 that directs me to use nano. So I'm on my way.

I'm still interested in any comments about the wisdom of dealing with this issue, or whether it's good or bad form to use a bash script where you just can't get it working in php.

Seems to me there's an argument that server to server, I'm just getting closer to the heart of the matter.

Here goes;

#!/bin/bash

HOST=ftp.server.com #This is the FTP servers host or IP address.
USER=ftpuser #This is the FTP user that has access to the server.
PASS=password #This is the password for the FTP user.

# Call 1. Uses the ftp command with the -inv switches. -i turns off interactive prompting. -n Restrains FTP from attempting the auto-login feature. -v enables verbose and progress.

ftp -inv $HOST << EOF

# Call 2. Here the login credentials are supplied by calling the variables.

user $USER $PASS

# Call 3. Here you will change to the directory where you want to put or get
cd /path/to/file

# Call4. Here you will tell FTP to put or get the file.
put test.txt

# or
get test.txt
bye
EOF

I'm assuming call 3 is for changing directory on the remote server

chrism01

02-05-2013 06:11 AM

Have a good read of this http://linux.die.net/man/1/ftp

motorcity

02-05-2013 01:24 PM

Thanks Chris.

This sure is a tough nut to crack!

Code:

!/bin/bash



HOST=00.000.000.000  #This is the FTP servers host or IP address.

USER=user        #This is the FTP user that has access to the server.

PASS=password        #This is the password for the FTP user.

ftp -invd $HOST << EOF

user $USER $PASS

cd /ftp_server/webfiles

lcd /home/name/public_html/admin/test

passive

get skudescadd.txt

bye

EOF

Trying to figure out how to specify the port...
<<<<<<< Got It >>>>>>>>>>>
this line above should be
ftp -invd $HOST 21 << EOF #Got lucky adding the known port

Thanks to all

Habitual

02-05-2013 04:11 PM

Code:

MYPORT=21

ftp -invd $HOST $MYPORT

may do it. :)

You should quote variables. :)
See http://mywiki.wooledge.org/Quotes

and http://www.google.com/cse/home?cx=01...1bdfg5ga&hl=en will search about 15 linux-specific sites
that deal with scripting and/or shell coding.

Good Luck and let us know!

motorcity

02-06-2013 06:12 AM

Thanks for the links!

Working on two issues;
I'm not finding the correct method to list different file names where a wildcard isn't going to work.

FILES='namename_snamec.txt', 'snamead.txt', 'skudescadd.txt', 'OTHER123.TXT'

mget $FILES

Second issue is closing the ftp session on the remote server. It doesn't seem to understand; bye or EOF
Trying;
disconnect
close
exit
500 'òABOR': command not understood

chrism01

02-06-2013 05:51 PM

Might be easier to use one of these tools

http://linux.die.net/man/1/ncftp
http://linux.die.net/man/1/lftp

http://www.ibm.com/developerworks/ai...lex/index.html
http://russbrooks.com/2010/11/19/lftp-cheetsheet

schneidz

02-06-2013 06:01 PM

i would not use the old, deprecated and insecure ftp protocol.

i would turn on ssh (which comes with scp (for automated file transfers) and sftp (for an ftp like interactive prompt (read: babysitting <@:))). i would also install sshfs for extra ease-of-use of drag-and-drop local file copy emulation.

i would disable root login and create public keys for each of my clients.

motorcity

02-06-2013 09:03 PM

Quote:

Originally Posted by schneidz (Post 4885876)

Turns out the bash script only works randomly, about once out of maybe a dozen tries. The only thing I can see that changes is the port identifier which is amended to the ip address so I didn't include it in any posting here. I took the problem over to my server admin and they're taking a look at it.

But the thing is I have no way to influence the msII server, that belongs to a huge company and I'm just one of many thousands of their customers. They determine the active connection and plain old FTP on port 21.

I'm pretty sure I was mistaken about closing the ftp session now. It's all about my server not finding a proper address or blocking the ftp port address, something along that line.

motorcity

02-07-2013 07:49 PM

I should first thank you all for the help and let you know that the problem came down to white listing the remote server. Why didn't I think of that? We're fortunate to have the hosting support we have.

Now perhaps it's time to end this thread. I've got 11 crons and all tested good the first time through. That covers a bunch of automation for all the text. But I have something of an interesting dilemma with the images.

I need to FTP and rename multiple images, from weekly updates, and I'm trying to come up with a script for this.

The interesting part is the folder names change every week, so at this early point in the year the WebImages folder on the remote server has Week1 - Week5. It wouldn't be so bad to use a wildcard like Week* if it's only five folders with maybe 200 images each, but what about later in the year when there might be Week1 - Week48 (200 * 48 = 9600 images) and only 200 images are actually new at the time?

A suggestion I got elsewhere was to look into using brace expansion running a cron job and/or a separate script to clear old folders and specify a brace number range.

Any thoughts or links or suggestions are greatly appreciated. Especially any that may direct me to where this type of question, (sort of a tricky bash requirement), might find the best response on this website.

Thanks again to all.
~Motorcity

chrism01

02-07-2013 08:32 PM

You need to know the definition/calculation of the week number at the remote end, then use the date cmd to give you the current week num

http://linux.die.net/man/1/date

Code:

%U

    week number of year, with Sunday as first day of week (00..53) 

%V

    ISO week number, with Monday as first day of week (01..53) 

%W

    week number of year, with Monday as first day of week (00..53)

or write your own algo if remote uses a funky calcn.

Target week num is then

Code:

weekdir=Week${weeknum}

Some good bash links
http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/

However(!), now you've solved the underlying issue (whitelisting) you may want to revert to php.
Imho, if this is a standalone cron job, stick with bash; if its called from Apache, go php.

motorcity

02-08-2013 06:41 PM

Totally awesome links there Chris!
Thank you very much!

Quote:

if this is a standalone cron job

It's still sort of a toss up; Eight of the eleven crons are already php but this one (images) and the other 3 FTP updates make sense as bash because they don't need to be called from the website, they're following some other entities schedule where we just take what they give.
On the other hand, this script with renaming the files & changing folder numbers is more complex than the simple bash files I'm currently testing.

This project of automating what I've been doing very manually has been long overdue. I'm looking for the easiest path to running on autopilot here. So once again I thank you for the consideration, and appreciate hearing your opinion.
~Motorcity

chrism01

02-10-2013 07:45 PM

If its time based, use cron, if event based (based on something happening in the website), use Apache to call php.
I believe there is a pure cli (non-apache) way to use php, but I've rarely seen it used (& I'm not a php guy anyway...)

All times are GMT -5. The time now is 01:54 AM.