Mirror an FTP site.

unreal128 · 10-19-2005, 01:32 PM

I need to mirror the file contents of an FTP site but the only access method is FTP (not SSH.) Currently I have an expect script that automates the download process via ncftp (which can copy whole directories automatically.) The script is rather dumb though since I have to manually terminate once all the files complete. Expect doesn't like the way ncftp displays its screens and I can never get it to properly expect the end of the file transfer. What is the best approach or even software out there that can do this and exit cleanly. I need to attach the mirroring function to a website action form that is using PHP.

MensaWater · 10-19-2005, 01:37 PM

Check out "rsync". Its designed for keeping directories in sync between remote hosts.

unreal128 · 10-19-2005, 01:47 PM

This was actually my first thought but doesn't the remote host have to be rsync enabled (acting as a server?) The only service I can use to the remote host is ftp. Can rsync function over FTP?

amitsharma_26 · 10-19-2005, 01:48 PM

Quote:

Originally posted by unreal128
I need to mirror the file contents of an FTP site but the only access method is FTP (not SSH.) Currently I have an expect script that automates the download process via ncftp (which can copy whole directories automatically.) The script is rather dumb though since I have to manually terminate once all the files complete. Expect doesn't like the way ncftp displays its screens and I can never get it to properly expect the end of the file transfer. What is the best approach or even software out there that can do this and exit cleanly. I need to attach the mirroring function to a website action form that is using PHP.

Go for RSYNC.
I have my own tutorial on that. I have explained some easily understandable examples over there also.
GOTO : www.amitsharma.linuxbloggers.com

And if you got any problem in implementation of that, mail me or message me or revert to this thread.

amitsharma_26 · 10-19-2005, 02:12 PM

Quote:

Originally posted by unreal128
This was actually my first thought but doesn't the remote host have to be rsync enabled (acting as a server?) The only service I can use to the remote host is ftp. Can rsync function over FTP?

Dear i got a trick for you.

Switch off your FTP service at remote host.

In /etc/services at remote host comment out the line
#rsync 873/tcp
#rsync 873/udp.

Restart your box.

Then start rysnc at port 21 with command
rsync --port=21 --daemon

Now you are actually using port 21 i.e. FTP port for RSYNC.

On the remote side run rsync to access through port 21
rsync -avz --port=21 --progress -partial <username>@<remote-ip>::<share-name>/* /<local-location>

I TESTED THAT, IT WORKS.

For more detailed info regarding RSYNC goto www.amitsharma.linuxbloggers.com

FEEDBACK for any further query.

unreal128 · 10-19-2005, 02:46 PM

Unfortunately my only account access is as a limited rights FTP user. I am unable to edit the remote host's services.

amitsharma_26 · 10-20-2005, 01:54 PM

FTP SYNC may help you in this.
http://freshmeat.net/projects/ftpsync/

Cheers.

unreal128 · 10-20-2005, 02:13 PM

This looks like a nice perl script for handling synchronization with FTP, thanks.