LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-12-2009, 08:57 AM   #1
laurens
Member
 
Registered: Mar 2009
Posts: 65

Rep: Reputation: 15
Bash script to transfer folders recursively without overwriting via FTP


Hi all,

I would need a bash script to transfer folders recursively via FTP. Username/password/IP - static defined.

They are all Linux servers (of course), and 3 different servers
1. source
2. destination - Samba
3. Server where the script runs on

As the files that have to be transferred are big database files (.bak, etc) overwriting should be avoided. Also no prompting because the script has to be 'cronned', to run at night.
For the 'not-overwriting' issue, is there an option to select this or should the files be compared first?

(I'm not a hero in bash, but I have the impression it's the easiest for this situation)

Code solutions are my favourite...!

Thanks a bunch in advance!
 
Old 11-12-2009, 08:59 AM   #2
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 244Reputation: 244Reputation: 244
this is your work, what have you tried?
 
Old 11-12-2009, 07:09 PM   #3
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,241

Rep: Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325Reputation: 2325
If you really want a different copy every night, you can rename the target with today's date

date '+%Y%m%d'
 
Old 11-13-2009, 04:34 AM   #4
laurens
Member
 
Registered: Mar 2009
Posts: 65

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by chrism01 View Post
If you really want a different copy every night, you can rename the target with today's date

date '+%Y%m%d'
Thanks for your replies.

To sketch the situation; Now there is a remote samba backup server containing lots of SQL database files: .bak, .mdf files etc.
Here, on location, we have a server that should be synced with this samba via FTP. File wise.
On the remote samba server the files are constantly updated and already have the filename %data%_%hour%_%databasename%.bak
Something like:

Code:
BELGsdsM_TRAININGS_23-10-2009_10h01.bak
BELGsdsM_TRAININGS_23-10-2009_18h01.bak
etc
The script runs on yet another remote server.

The actions that I would need are

1. Connect via FTP to source server
2. Connect via FTP to dest. server
3. Possibly compare the files between source and dest.
4. transfer the files recursively (no overwriting)
5. close connections

What I have are the 1st, 2nd and 5th step. I would really appreciate a solution. Didn't find a lot on the web so I'm sure others will be helped by it too.

Thanks in advance!

Last edited by laurens; 11-13-2009 at 04:37 AM.
 
Old 11-13-2009, 04:44 AM   #5
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 244Reputation: 244Reputation: 244
you can retrieve a directory listing of the remote server first, do some manipulation to compare dates, then decide whether to copy over or not. or you can see if your FTP client support options of not overwriting when there are similar files..etc..
 
Old 11-13-2009, 05:47 AM   #6
laurens
Member
 
Registered: Mar 2009
Posts: 65

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by ghostdog74 View Post
you can retrieve a directory listing of the remote server first, do some manipulation to compare dates, then decide whether to copy over or not. or you can see if your FTP client support options of not overwriting when there are similar files..etc..
Thx. I would prefer comparing the filenames, they also contain the data as a matter a fact. Could some code hero provide me some code for this. The programming logics are not the problem, it's the (bash!) programming itself! Thanks in advance
 
Old 11-13-2009, 06:21 AM   #7
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,753

Rep: Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288
This is a task that is crying out for the use of rsync. Unfortunately, you say it needs to be done with FTP. Is using FTP realy set in stone? If your machines were running ssh, it would be a breeze to use 'rsync -au -e ssh'.

Evo2.
 
Old 11-13-2009, 06:21 AM   #8
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 244Reputation: 244Reputation: 244
to automate FTP
Code:
ftp -n $server <<EOF
user anonymous password
dir
EOF
 
Old 11-13-2009, 08:36 AM   #9
laurens
Member
 
Registered: Mar 2009
Posts: 65

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by evo2 View Post
This is a task that is crying out for the use of rsync. Unfortunately, you say it needs to be done with FTP. Is using FTP realy set in stone? If your machines were running ssh, it would be a breeze to use 'rsync -au -e ssh'.

Evo2.
oh no it's not obliged to use FTP only for the solution! If you have a proposition with RSYNC, be my welcome guest.
I already had the impression it was a job for RSYNC (not that it's 'my cup of tea'...)

Thanks in advance!
 
Old 11-13-2009, 08:40 AM   #10
laurens
Member
 
Registered: Mar 2009
Posts: 65

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by ghostdog74 View Post
to automate FTP
Code:
ftp -n $server <<EOF
user anonymous password
dir
EOF
Thanks
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
find -exec command to recursively delete files and folders in folders with X name Joan Murt Linux - Newbie 2 07-08-2009 05:35 PM
Help needed to transfer list of files to FTP server, to different folders amazon Programming 1 06-26-2009 02:12 AM
Entering folders recursively to run script on files nested inside, ubuntu 7.10 aidansmoker Linux - Newbie 4 11-12-2007 12:39 AM
FTP need to transfer entire folder recursively rolster Linux - Newbie 6 02-02-2006 09:54 AM
Using BASH script to transfer a file via (s)ftp senther Linux - Newbie 2 07-19-2005 05:47 PM


All times are GMT -5. The time now is 01:34 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration