Lftp mirroring multiple folders
 

I need to backup several folders,files in my linux box.. I have read about lftp but the thing is every example I find is about local/path to remote/path.. ONE folder and SUBfolders..

is there any option to copy multiple folders ?

the idea is to execute a script..

lftp -u xx,yy ftpserver -e "mirror -R file/ remotefile/"

thanks

rsync is your friend. lftp will not be secure and does not offer the flexibility that rsync will provide. use the right tool for the job, rsync is the tool for this job.

enjoy.

the think is that for the moment we have to do the backup to a ftp server running windows. that is why I would like to use lftp. in the past I have used rsync but using a 2nd linux server.

thanks

hmm try something along this line:

if you want to pass more then one command via -e option you have to use && to separate them.

p.s. also as you are dealing with MS FTP from a linux box, adding the passive can not hurt.

thanks.

I am creating something like this:
in the log file lftp.out only register the successful transfer if there is any.. but if there is any error because the path or something else, it doesnt show the failed transfer..

is there any way to show the failed transfer?

thanks

you should be able to call both stderr and stdout with 2>&1 at the end of your lines that contain

thanks, that works...

this thing never end...now, 3 of the servers has 90MB in /etc and I would like to compress it before uploading to the ftp server...

I will try something meanwhile. if anyone know how, feel free to post it.


thanks

actually, I can add this line to the script

and then use put test1.tar.gz

the thing is...

if this directory has a big size where it could take some seconds or minutes while it get compressed...does the script will wait until this tar finish to upload or will try to upload the tar file even while it is still compressing?

yes to both. bash is linear. in other words it goes line by line in order from top to bottom. it will not move onto the next line until the last is completed. few other things to keep in mind if you are doing this for backups you might want to both encrypt and create a md5sum for the file to be tested before/after of each of the steps as well as a file check after compression.

things are starting to get a bit larger, but if you wish to do that i can help to an extent. ive done a few of those types of scripts in the past. typically though ill just compress, verify the compression, encrypt and transmit. that is if i am backing up daily. if i am only backing up weekly or less often, ill add the md5sum check after encryption and verify it after transmission.

thanks for your help, I appreciate it.

I think it wont go further. let see how this works. maybe I could try using samba to share the folders we need to backup..

thanks

instead of samba, create the share on the MS system and mount it locally on the Linux system as a cifs mount point. you can move the files around much easier that way.

remember samba is not native for linux. cifs is not much better, but if you mount it locally then the linux box will treat it as a local drive not a network drive.

then you would be able to use rsync, but look up in the man page howto point to a local /tmp instead of the remote /tmp that rsync would typically create.

Thanks.

I am still trying something with lftp..

I would like to keep some old backups in the server. so the idea is to create some variable in the file and use it for creating several directories in the ftp server. i.e backup_552013 backup_542013 ....

I can't get the variables to work.

What I do is create a script similar to this:



then invoke it using:

lftp -f script in the console or cron

and get this error:

how could I create a folder like backup_05-05-05-2013 in the ftp server with lftp? I am stuck with this right now..

thanks

now I solved this issue about the date..

I'm just running the script.sh and no invoking it with lftp -f...


But the issue now is removing folders older than X day..

and it doesnt work...

the other issue is if I connect directly to the ftp server I can list with ls the folder.. then type rm -rf folderName and it delete some of the files but not ALL.. onley some of them are hidden files but other arent....not sure why it doesnt delete ALL...

try something like this instead:

the find command will continue to execute until it runs out of items to remove. instead of removing first (fyi rm -rf is the same as rf -r -f) you might want to first try it with an ls -laF:

This way you can print out the files and see if there are any that meet the date restrictions. If that works, then replace ls -laF with rm -rf

I was doing that..

it get me the list of files.. but at the end I get this:

If you're trying to tidy up the remote system, the problem you have there is that FTP is a File Transfer(!) Protocol, not a File Sharing one, so you can't use any old cmd (eg find), you can only use those provided internally by ftp http://linux.die.net/man/1/ftp.

Normally, I'd use a combo of scp (or sftp) to do file txfr, then ssh to do remote work.

hmm running as root not a really good idea.

also i think i found my typo. try it with /test/ instead of just test/

i typically use this to clean up log files. so mine looks like the following:

note the specific file type not just path.using rm -rf as root on a directory can be dangerous just FYI.

Shouldn't there be a space between the dir to start from and the file pattern
Also a good idea to single-quote the wildcard pattern to prevent interpolation by the shell; we want find to handle that.

no need for the space, but the '*.log' is probably a good idea. not sure why i didnt do it there when i did later on with the {}

the path for my log files in $HOME/logs/foo.log, thus the $HOMEDIR/logs/*.log in my find command. i will modify my script to place the ' ' around the wildcard though

No: to get a sane list returned find requires a dirpath as the 'where to start' param and then you'd need -name (or -iname) to specify the file pattern to search for

oh, now thats new to me. thank you. ill read up on that.