LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices

Reply
 
Search this Thread
Old 06-24-2013, 06:19 AM   #1
kikinovak
Senior Member
 
Registered: Jun 2011
Location: Montpezat (South France)
Distribution: ElementaryOS, Ubuntu LTS, Slackware
Posts: 1,549

Rep: Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721
Closest thing to rsync emulation with FTP client?


Hi,

I have 100 GB space on a mutualized web server, e. g. a machine on which I only have an FTP account, but no shell access. This machine hosts my new package repository. Unfortunately I have no rsync access, so synchronizing the local file tree with the remote file tree is a bit of a headache.

To make things worse, I have a slow DSL connection with a ridiculous upload bandwidth of about 40 kB/s. That's the price you pay for living in the south French countryside.

So far, GFTP and Filezilla don't really behave like they should. I guess the solution will be a short script using lftp, ncftp or whatever command-line client will be apt for the job.

Here's how the script should behave, preferably.
  1. If a remote file is identical to the local copy, proceed to the next file.
  2. If a remote file is present but different than the local file (in content), re-upload and squash it.
  3. If a remote file exists where there is no local equivalent, delete the file.

In short, the whole thing should be equivalent to:

Code:
$ rsync -av --delete [localstuff] [remote_server]
Any suggestions?
 
Old 06-24-2013, 06:33 AM   #2
cendryon
Member
 
Registered: Aug 2005
Location: France
Distribution: Slackware64 current
Posts: 78

Rep: Reputation: 26
Hi

The "mirror" command from lftp is what you're looking for:
Code:
# lftp -c "open ftp://remote.host.example ; mirror /path/to/local/source path/to/remote/target ;"
Eric uses it in his mirror-alien-repository.sh for instance.
I use it to keep my local Slackware mirrors up to date.

Cheers
 
4 members found this post helpful.
Old 06-24-2013, 08:04 AM   #3
GazL
Senior Member
 
Registered: May 2008
Posts: 3,380

Rep: Reputation: 913Reputation: 913Reputation: 913Reputation: 913Reputation: 913Reputation: 913Reputation: 913Reputation: 913
I believe you'll need "mirror -R" in lftp when uploading.
 
1 members found this post helpful.
Old 06-24-2013, 08:19 AM   #4
kikinovak
Senior Member
 
Registered: Jun 2011
Location: Montpezat (South France)
Distribution: ElementaryOS, Ubuntu LTS, Slackware
Posts: 1,549

Original Poster
Rep: Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721
I've been playing around with this for a little while, and here's the result, which works as expected:

Code:
#!/bin/sh
LFTP=$(which lftp)
$LFTP -c "set ftp:list-options -a;
open ftp://user:password@remoteserver;
set net:limit-rate 25000
lcd localdir;
cd remotedir;
mirror --continue --reverse --dereference --parallel=5 --delete --verbose "
  • --continue allows to resume partial uploads after an interruption
  • --reverse means mirror from here to there and not to there from here
  • --dereference allows handling of symlinks in the local file tree
  • I've decided to limit the upload speed so I can continue to surf the web while uploading

I'll test this some more, but first results are quite encouraging. I guess I'll write a blog post about this.

Cheers,

Niki
 
2 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Closest thing to Slackware but with PAM and systemd deadbeat Linux - Distributions 6 02-24-2013 07:42 AM
What linux FTP program is closest to FlashFXP for windows? eBopBob Linux - Software 7 05-21-2009 12:11 AM
ProFTPd ... FTP client fails to connect: timeout after client sends 'LIST' nutnut Linux - Software 2 01-01-2006 07:09 PM
what about an AS/400 5250 client emulation? Morz Linux - Enterprise 6 05-20-2004 03:39 PM
FTP restart the whole thing Henry Linux - Networking 22 07-15-2003 03:27 PM


All times are GMT -5. The time now is 09:42 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration