LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices



Reply
 
Search this Thread
Old 10-19-2005, 02:32 PM   #1
unreal128
Member
 
Registered: Jun 2003
Distribution: SuSE, Slackware, Gentoo
Posts: 207

Rep: Reputation: 30
Mirror an FTP site.


I need to mirror the file contents of an FTP site but the only access method is FTP (not SSH.) Currently I have an expect script that automates the download process via ncftp (which can copy whole directories automatically.) The script is rather dumb though since I have to manually terminate once all the files complete. Expect doesn't like the way ncftp displays its screens and I can never get it to properly expect the end of the file transfer. What is the best approach or even software out there that can do this and exit cleanly. I need to attach the mirroring function to a website action form that is using PHP.
 
Old 10-19-2005, 02:37 PM   #2
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,028
Blog Entries: 5

Rep: Reputation: 791Reputation: 791Reputation: 791Reputation: 791Reputation: 791Reputation: 791Reputation: 791
Check out "rsync". Its designed for keeping directories in sync between remote hosts.
 
Old 10-19-2005, 02:47 PM   #3
unreal128
Member
 
Registered: Jun 2003
Distribution: SuSE, Slackware, Gentoo
Posts: 207

Original Poster
Rep: Reputation: 30
This was actually my first thought but doesn't the remote host have to be rsync enabled (acting as a server?) The only service I can use to the remote host is ftp. Can rsync function over FTP?
 
Old 10-19-2005, 02:48 PM   #4
amitsharma_26
Member
 
Registered: Sep 2005
Location: New delhi
Distribution: RHEL 3.0/4.0
Posts: 777

Rep: Reputation: 31
Re: Mirror an FTP site.

Quote:
Originally posted by unreal128
I need to mirror the file contents of an FTP site but the only access method is FTP (not SSH.) Currently I have an expect script that automates the download process via ncftp (which can copy whole directories automatically.) The script is rather dumb though since I have to manually terminate once all the files complete. Expect doesn't like the way ncftp displays its screens and I can never get it to properly expect the end of the file transfer. What is the best approach or even software out there that can do this and exit cleanly. I need to attach the mirroring function to a website action form that is using PHP.
Go for RSYNC.
I have my own tutorial on that. I have explained some easily understandable examples over there also.
GOTO : www.amitsharma.linuxbloggers.com

And if you got any problem in implementation of that, mail me or message me or revert to this thread.

Last edited by amitsharma_26; 08-06-2006 at 05:33 PM.
 
Old 10-19-2005, 03:12 PM   #5
amitsharma_26
Member
 
Registered: Sep 2005
Location: New delhi
Distribution: RHEL 3.0/4.0
Posts: 777

Rep: Reputation: 31
Quote:
Originally posted by unreal128
This was actually my first thought but doesn't the remote host have to be rsync enabled (acting as a server?) The only service I can use to the remote host is ftp. Can rsync function over FTP?
Dear i got a trick for you.

Switch off your FTP service at remote host.

In /etc/services at remote host comment out the line
#rsync 873/tcp
#rsync 873/udp.

Restart your box.

Then start rysnc at port 21 with command
rsync --port=21 --daemon

Now you are actually using port 21 i.e. FTP port for RSYNC.

On the remote side run rsync to access through port 21
rsync -avz --port=21 --progress -partial <username>@<remote-ip>::<share-name>/* /<local-location>

I TESTED THAT, IT WORKS.

For more detailed info regarding RSYNC goto www.amitsharma.linuxbloggers.com

FEEDBACK for any further query.

Last edited by amitsharma_26; 08-06-2006 at 05:34 PM.
 
Old 10-19-2005, 03:46 PM   #6
unreal128
Member
 
Registered: Jun 2003
Distribution: SuSE, Slackware, Gentoo
Posts: 207

Original Poster
Rep: Reputation: 30
Unfortunately my only account access is as a limited rights FTP user. I am unable to edit the remote host's services.
 
Old 10-20-2005, 02:54 PM   #7
amitsharma_26
Member
 
Registered: Sep 2005
Location: New delhi
Distribution: RHEL 3.0/4.0
Posts: 777

Rep: Reputation: 31
FTP SYNC may help you in this.
http://freshmeat.net/projects/ftpsync/

Cheers.
 
Old 10-20-2005, 03:13 PM   #8
unreal128
Member
 
Registered: Jun 2003
Distribution: SuSE, Slackware, Gentoo
Posts: 207

Original Poster
Rep: Reputation: 30
This looks like a nice perl script for handling synchronization with FTP, thanks.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Subversion (client) -- site is down, need a mirror Josh1billion Linux - Software 1 10-02-2005 11:22 AM
help creatin own mirror site... jsheffie Suse/Novell 1 04-13-2005 09:44 AM
Howto mirror a web site ivanatora Linux - Networking 4 12-17-2003 05:47 PM
linuxquestions mirror site sub_slack General 5 08-31-2003 08:53 PM
Mandrake Mirror Dload site? iJasonT Linux - Software 1 03-29-2003 12:42 AM


All times are GMT -5. The time now is 06:25 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration