LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-04-2013, 09:37 PM   #1
motorcity
LQ Newbie
 
Registered: Feb 2013
Posts: 16

Rep: Reputation: Disabled
FTP script Microsoft-IIS/7.5 to Unix


Hello all, I've visited here for years, just never got around to posting.

I'm looking for a little help writing a script to run on my server (CentOS), and its something I've never done before. I've got these text files and full permission to grab them off a Microsoft-IIS/7.5 server. I'm trying to automate the process with cron jobs.
So far I'm striking out trying to do this with php and I keep wondering if doesn't make sense to at least try a bash or something like it.
Here is the php code that gives this error;
Can't open data connection Googling that everyone seems to say add this; ftp_pasv($conn_id, true);
Code:
<?php
ini_set('display_errors',1);
error_reporting(E_ALL);
         // define some variables
         $folder_path = "/home/name/public_html/test/";
         $local_file = "file.txt"; //
         $server_file = "file.txt";
         //-- Connection Settings
         $ftp_server = "msserver"; // Address of FTP server.
         $ftp_user_name = "u"; // Username
         $ftp_user_pass = "p"; // Password
         // set up basic connection
         $conn_id = ftp_connect($ftp_server); // I checked, port is 21
         // login with username and password
         $login_result = ftp_login($conn_id, $ftp_user_name, $ftp_user_pass);
ftp_pasv($conn_id, true);
if(ftp_chdir($conn_id, "folder/webfiles/")) {
echo "Current directory is now: " . ftp_pwd($conn_id) . "\n";
}
// ftp_pasv($conn_id, true);
$lf = fopen($local_file, 'w');
if (!$lf){
die('Failed to create local file.');
}
         if (!ftp_fget($conn_id, $lf, $server_file, FTP_ASCII)) {
         die('Failed to download file from FTP server.');
}
// ftp_get($conn_id, $local_file, $server_file, FTP_ASCII);
         // close the connection
         ftp_close($conn_id);
fclose($lf);
?>
Anyway I keep looking at #!/bin/bash scripts ... do you use pico or what?
 
Old 02-04-2013, 09:59 PM   #2
joshp
LQ Newbie
 
Registered: Aug 2006
Location: Chicago IL
Distribution: To many to list.
Posts: 27

Rep: Reputation: 1
Hello,

Here is a super basic setup for a ftp transfer using Bash.

Code:
#!/bin/bash

HOST=localhost
USER=someuser
PASS=password

ftp -inv $HOST  << EOF
user $USER $PASS

[COMMANDS HERE]

EOF
 
1 members found this post helpful.
Old 02-05-2013, 05:37 AM   #3
motorcity
LQ Newbie
 
Registered: Feb 2013
Posts: 16

Original Poster
Rep: Reputation: Disabled
Thanks joshp.
Sorry to the forum for putting the dreaded ms word in my question title.

Found a super simpleton youtubehttp://www.youtube.com/watch?v=QGvvJO5UIs4 that directs me to use nano. So I'm on my way.

I'm still interested in any comments about the wisdom of dealing with this issue, or whether it's good or bad form to use a bash script where you just can't get it working in php.

Seems to me there's an argument that server to server, I'm just getting closer to the heart of the matter.

Here goes;

#!/bin/bash

HOST=ftp.server.com #This is the FTP servers host or IP address.
USER=ftpuser #This is the FTP user that has access to the server.
PASS=password #This is the password for the FTP user.

# Call 1. Uses the ftp command with the -inv switches. -i turns off interactive prompting. -n Restrains FTP from attempting the auto-login feature. -v enables verbose and progress.

ftp -inv $HOST << EOF

# Call 2. Here the login credentials are supplied by calling the variables.

user $USER $PASS

# Call 3. Here you will change to the directory where you want to put or get
cd /path/to/file

# Call4. Here you will tell FTP to put or get the file.
put test.txt

# or
get test.txt
bye
EOF

I'm assuming call 3 is for changing directory on the remote server

Last edited by motorcity; 02-05-2013 at 06:01 AM. Reason: found a start
 
Old 02-05-2013, 07:11 AM   #4
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,417

Rep: Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397
Have a good read of this http://linux.die.net/man/1/ftp
 
1 members found this post helpful.
Old 02-05-2013, 02:24 PM   #5
motorcity
LQ Newbie
 
Registered: Feb 2013
Posts: 16

Original Poster
Rep: Reputation: Disabled
Thanks Chris.

This sure is a tough nut to crack!
Code:
!/bin/bash

HOST=00.000.000.000  #This is the FTP servers host or IP address.
USER=user        #This is the FTP user that has access to the server.
PASS=password        #This is the password for the FTP user.
ftp -invd $HOST << EOF
user $USER $PASS
cd /ftp_server/webfiles
lcd /home/name/public_html/admin/test
passive
get skudescadd.txt
bye
EOF
Trying to figure out how to specify the port...
<<<<<<< Got It >>>>>>>>>>>
this line above should be
ftp -invd $HOST 21 << EOF #Got lucky adding the known port

Thanks to all

Last edited by motorcity; 02-05-2013 at 03:47 PM. Reason: forgot the code
 
Old 02-05-2013, 05:11 PM   #6
Habitual
LQ 5k Club
 
Registered: Jan 2011
Location: Nowhere near you, thank God.
Distribution: High Sierra
Posts: 8,598
Blog Entries: 15

Rep: Reputation: Disabled
Code:
MYPORT=21
ftp -invd $HOST $MYPORT
may do it.

You should quote variables.
See http://mywiki.wooledge.org/Quotes

and http://www.google.com/cse/home?cx=01...1bdfg5ga&hl=en will search about 15 linux-specific sites
that deal with scripting and/or shell coding.

Good Luck and let us know!

Last edited by Habitual; 02-05-2013 at 05:13 PM.
 
1 members found this post helpful.
Old 02-06-2013, 07:12 AM   #7
motorcity
LQ Newbie
 
Registered: Feb 2013
Posts: 16

Original Poster
Rep: Reputation: Disabled
Thanks for the links!

Working on two issues;
I'm not finding the correct method to list different file names where a wildcard isn't going to work.

FILES='namename_snamec.txt', 'snamead.txt', 'skudescadd.txt', 'OTHER123.TXT'

mget $FILES

Second issue is closing the ftp session on the remote server. It doesn't seem to understand; bye or EOF
Trying;
disconnect
close
exit
500 'ňABOR': command not understood

Last edited by motorcity; 02-06-2013 at 07:14 AM. Reason: 500 quote
 
Old 02-06-2013, 06:51 PM   #8
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,417

Rep: Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397
Might be easier to use one of these tools

http://linux.die.net/man/1/ncftp
http://linux.die.net/man/1/lftp

http://www.ibm.com/developerworks/ai...lex/index.html
http://russbrooks.com/2010/11/19/lftp-cheetsheet
 
1 members found this post helpful.
Old 02-06-2013, 07:01 PM   #9
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,167

Rep: Reputation: 890Reputation: 890Reputation: 890Reputation: 890Reputation: 890Reputation: 890Reputation: 890
i would not use the old, deprecated and insecure ftp protocol.

i would turn on ssh (which comes with scp (for automated file transfers) and sftp (for an ftp like interactive prompt (read: babysitting <@)). i would also install sshfs for extra ease-of-use of drag-and-drop local file copy emulation.

i would disable root login and create public keys for each of my clients.

Last edited by schneidz; 02-06-2013 at 07:04 PM.
 
Old 02-06-2013, 10:03 PM   #10
motorcity
LQ Newbie
 
Registered: Feb 2013
Posts: 16

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by schneidz View Post
i would not use the old, deprecated and insecure ftp protocol.

i would turn on ssh (which comes with scp (for automated file transfers) and sftp (for an ftp like interactive prompt (read: babysitting <@)). i would also install sshfs for extra ease-of-use of drag-and-drop local file copy emulation.

i would disable root login and create public keys for each of my clients.
Turns out the bash script only works randomly, about once out of maybe a dozen tries. The only thing I can see that changes is the port identifier which is amended to the ip address so I didn't include it in any posting here. I took the problem over to my server admin and they're taking a look at it.

But the thing is I have no way to influence the msII server, that belongs to a huge company and I'm just one of many thousands of their customers. They determine the active connection and plain old FTP on port 21.

I'm pretty sure I was mistaken about closing the ftp session now. It's all about my server not finding a proper address or blocking the ftp port address, something along that line.
 
1 members found this post helpful.
Old 02-07-2013, 08:49 PM   #11
motorcity
LQ Newbie
 
Registered: Feb 2013
Posts: 16

Original Poster
Rep: Reputation: Disabled
I should first thank you all for the help and let you know that the problem came down to white listing the remote server. Why didn't I think of that? We're fortunate to have the hosting support we have.

Now perhaps it's time to end this thread. I've got 11 crons and all tested good the first time through. That covers a bunch of automation for all the text. But I have something of an interesting dilemma with the images.

I need to FTP and rename multiple images, from weekly updates, and I'm trying to come up with a script for this.

The interesting part is the folder names change every week, so at this early point in the year the WebImages folder on the remote server has Week1 - Week5. It wouldn't be so bad to use a wildcard like Week* if it's only five folders with maybe 200 images each, but what about later in the year when there might be Week1 - Week48 (200 * 48 = 9600 images) and only 200 images are actually new at the time?

A suggestion I got elsewhere was to look into using brace expansion running a cron job and/or a separate script to clear old folders and specify a brace number range.

Any thoughts or links or suggestions are greatly appreciated. Especially any that may direct me to where this type of question, (sort of a tricky bash requirement), might find the best response on this website.

Thanks again to all.
~Motorcity
 
Old 02-07-2013, 09:32 PM   #12
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,417

Rep: Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397
You need to know the definition/calculation of the week number at the remote end, then use the date cmd to give you the current week num

http://linux.die.net/man/1/date
Code:
%U
    week number of year, with Sunday as first day of week (00..53) 
%V
    ISO week number, with Monday as first day of week (01..53) 
%W
    week number of year, with Monday as first day of week (00..53)
or write your own algo if remote uses a funky calcn.

Target week num is then
Code:
weekdir=Week${weeknum}
Some good bash links
http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/

However(!), now you've solved the underlying issue (whitelisting) you may want to revert to php.
Imho, if this is a standalone cron job, stick with bash; if its called from Apache, go php.
 
2 members found this post helpful.
Old 02-08-2013, 07:41 PM   #13
motorcity
LQ Newbie
 
Registered: Feb 2013
Posts: 16

Original Poster
Rep: Reputation: Disabled
Totally awesome links there Chris!
Thank you very much!
Quote:
if this is a standalone cron job
It's still sort of a toss up; Eight of the eleven crons are already php but this one (images) and the other 3 FTP updates make sense as bash because they don't need to be called from the website, they're following some other entities schedule where we just take what they give.
On the other hand, this script with renaming the files & changing folder numbers is more complex than the simple bash files I'm currently testing.

This project of automating what I've been doing very manually has been long overdue. I'm looking for the easiest path to running on autopilot here. So once again I thank you for the consideration, and appreciate hearing your opinion.
~Motorcity
 
Old 02-10-2013, 08:45 PM   #14
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,417

Rep: Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397
If its time based, use cron, if event based (based on something happening in the website), use Apache to call php.
I believe there is a pure cli (non-apache) way to use php, but I've rarely seen it used (& I'm not a php guy anyway...)
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
MS IIs & Plesk: ftp user cannot login xtendz General 3 11-22-2010 09:27 AM
squid as reverse proxy for microsoft IIS with Advanced Digest Authentication mikmok Linux - Networking 0 10-19-2006 10:43 AM
LXer: Microsoft's IIS 7 will aid PHP developers' Linux deployments LXer Syndicated Linux News 0 09-26-2006 09:54 AM
Is there an alternate to microsoft iis indexing liquidghondi Linux - Software 3 02-12-2004 08:49 PM
How to schedule unix script periodically from unix os level??? gopi_20us Programming 2 03-11-2002 07:45 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 01:01 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration