LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 12-04-2004, 09:15 AM   #1
eradrix
LQ Newbie
 
Registered: May 2004
Location: Switzerland
Distribution: whitebox
Posts: 12

Rep: Reputation: 0
Question local rsync shell script


Hi folks,

this is supposed not to be complicated but I'm on a level in shell programming

I have several folders to backup regularily so (using search on the web) I created this script, wich is actually ok (this is just a part)
Code:
source="/something"
cible="/mnt/backup"
exclu="excluded"
options="-avzx --delete"
options2="--exclude $exclu"
nom_backup=$(basename "${source}")
	
rsync ${options} "${source}" ${options2} "${cible}" >> $rapport
This is perfect for one folder or two but as I have more folders to backup I'd prefer if the script could read variables in a file like this:
Quote:
source1 cible1
source2 cible2 exclu-a exclu-b
source3 cible3 exclu-c
source4 cible4 exclu-d exclu-e exclu-f
(there would be zero, one or more excluded files by rsync command )


Do you have a tip to do this?

<Edit> you guessed, "cible" means "target" </Edit>

Last edited by eradrix; 12-04-2004 at 04:24 PM.
 
Old 12-04-2004, 11:34 AM   #2
dustu76
Member
 
Registered: Sep 2004
Distribution: OpenSuSe
Posts: 153

Rep: Reputation: 30
Create a file as:

List_of_sources|List_of_exclusions
source1a,source1b,...|excl1a,excl1b,...
source2a,source2b,...|excl2a,excl2b,...

Read the file in a for loop & split each line into list of sources & exclusions, and then execute your rsync.
 
Old 12-04-2004, 04:28 PM   #3
eradrix
LQ Newbie
 
Registered: May 2004
Location: Switzerland
Distribution: whitebox
Posts: 12

Original Poster
Rep: Reputation: 0
I suppose you meant "source1a,target1a|excl1a,excl1b,..." or I don't understand

The matter is that I never did something like that (this is the first time I try to create such a script), can you tell me more?
 
Old 12-05-2004, 01:48 AM   #4
dustu76
Member
 
Registered: Sep 2004
Distribution: OpenSuSe
Posts: 153

Rep: Reputation: 30
I dont know rsync syntax. I presume there is one target, multiple sources, multiple exclusions. In that case its like:

target1|source1a,source1b,source1c..|excl1a,excl1b,excl1c...
target2|source2a,source2b,source2c..|excl2a,excl2b,excl2c...

A simple for loop & using "cut" to extract the values from "delimited" columns should work..

Presuming my presumptions are correct, I already have the solution... I'm just making you get it... You'll thank me for that

HTH.
 
Old 12-05-2004, 02:51 AM   #5
eradrix
LQ Newbie
 
Registered: May 2004
Location: Switzerland
Distribution: whitebox
Posts: 12

Original Poster
Rep: Reputation: 0
sure, I'll try and report result here...
 
Old 12-05-2004, 06:13 PM   #6
eradrix
LQ Newbie
 
Registered: May 2004
Location: Switzerland
Distribution: whitebox
Posts: 12

Original Poster
Rep: Reputation: 0
I'm just all lost

To be clear: in a line, there's only one source and only one target and there's possibly one or more excluded.

Then, as I read, "cut" works on columns: but I don't need a separate source list or target list??? I didn't find any option to select one line...

Maybe awk would be able to do the job, as I can print $1 $2 etc... per line, but I didn't find how to assign them on variables, nor how to create a loop inside awk


deep...

Last edited by eradrix; 12-05-2004 at 06:16 PM.
 
Old 12-06-2004, 03:30 AM   #7
dustu76
Member
 
Registered: Sep 2004
Distribution: OpenSuSe
Posts: 153

Rep: Reputation: 30
Code:
SF1B:/supmis/soumen/tmp> cat rs
#!/usr/bin/bash

cfg_file=rsync.cfg
options="-avzx --delete"

cat > $cfg_file <<-END
/something1|/mnt/backup|
/something2|/mnt/backup|excl1,excl2
/something3|/mnt/backup|excl3
END

### This presumes that you dont have any filename with spaces
for i in $(cat $cfg_file |tr -d " ") ; do
        src=$(echo $i |cut -d"|" -f1)
        target=$(echo $i |cut -d"|" -f2)
        excl_list=$(echo $i |cut -d"|" -f3 |tr "," " ")
        if [ "${excl_list}X" != "X" ] ; then
                for i in $(echo $excl_list |tr "," " ") ; do
                        excl_list_opt="$excl_list_opt --exclude $i"
                done
        fi
        echo "rsync $options $src $excl_list_opt $target"
done
SF1B:/supmis/soumen/tmp> rs
rsync -avzx --delete /something1  /mnt/backup
rsync -avzx --delete /something2  --exclude excl1 --exclude excl2 /mnt/backup
rsync -avzx --delete /something3  --exclude excl1 --exclude excl2 --exclude excl3 /mnt/backup
SF1B:/supmis/soumen/tmp>
You should be able to tweak this in case it doesn't meet your exact requirement...

Out of all this, I might end up learning rsync

HTH
 
Old 12-06-2004, 05:02 PM   #8
eradrix
LQ Newbie
 
Registered: May 2004
Location: Switzerland
Distribution: whitebox
Posts: 12

Original Poster
Rep: Reputation: 0
Many thanks!
takes a little time to assimilate all this stuff

I didn't realized that with your format (/something3|/mnt/backup|excl3) all the informations were taken like one word.



Something I didn't found: would you explain what [ "${excl_list}X" != "X" ] does?

The double-space in result may cause problem (does it come from the line <excl_list_opt="$excl_list_opt --exclude $i">?)
Anyway, if I change the last command to <echo "rsync $options $src$excl_list_opt $target"> this is OK


You saved me a lot of trouble, I missed kind knowledge. Can you give me some good weblinks for learning more in shell scripting?
 
Old 12-06-2004, 06:10 PM   #9
Dave Kelly
Member
 
Registered: Aug 2004
Location: Todd Mission Texas
Distribution: Linspire
Posts: 213

Rep: Reputation: 31
Quote:
Can you give me some good weblinks for learning more in shell scripting
http://rpmseek.com/rpm/abs-guide-3.1...:1822515:0:2:0

Also, Machtelt Garrels has a new book out. "bash guide for beginners" available on amazon.com.
 
Old 12-06-2004, 10:16 PM   #10
dustu76
Member
 
Registered: Sep 2004
Distribution: OpenSuSe
Posts: 153

Rep: Reputation: 30
Quote:
Something I didn't found: would you explain what [ "${excl_list}X" != "X" ] does?
Checks whether excl_list is non-empty i.e. whether there is anything to exclude.

Quote:
The double-space in result may cause problem (does it come from the line <excl_list_opt="$excl_list_opt --exclude $i">?)
Anyway, if I change the last command to <echo "rsync $options $src$excl_list_opt $target"> this is OK
I don't see why extra spaces in a *nix command (generally speaking) should cause problems... Have you tried rsync with the the leading space intact? Has it failed?

You can kill that space by using this:

Code:
excl_list_opt=$(echo $excl_list_opt |sed -e 's/^ *//g')
echo "rsync $options $src $excl_list_opt $target"
[preaching_hat_on]
I'll say something OT here. A few years ago I knew shell command & syntax etc. But it's when I was faced with a very similar problem, I learned how to apply them. That's why I was insistent upon you doing it. No exercise from a guide book asking you to reverse a string can provide you with the kind of insight you'll gain by solving real-life problems life this.
[preaching_hat_off]

HTH
 
Old 12-06-2004, 10:21 PM   #11
dustu76
Member
 
Registered: Sep 2004
Distribution: OpenSuSe
Posts: 153

Rep: Reputation: 30
Btw, there is a very efficient & elegant way of eliminating extra spaces:

Code:
SF1B:/supmis> val="    hello      world             "
SF1B:/supmis> echo "--$val--"
--    hello      world             --
SF1B:/supmis> val=$(echo $val)
SF1B:/supmis> echo "--$val--"
--hello world--
SF1B:/supmis>
Wonder why I didn't use this...

HTH.
 
Old 12-07-2004, 03:59 AM   #12
eradrix
LQ Newbie
 
Registered: May 2004
Location: Switzerland
Distribution: whitebox
Posts: 12

Original Poster
Rep: Reputation: 0
Quote:
Originally posted by dustu76
I don't see why extra spaces in a *nix command (generally speaking) should cause problems... Have you tried rsync with the the leading space intact? Has it failed?
I must confess I didn't try

Quote:
I'll say something OT here. A few years ago I knew shell command & syntax etc. But it's when I was faced with a very similar problem, I learned how to apply them. That's why I was insistent upon you doing it. No exercise from a guide book asking you to reverse a string can provide you with the kind of insight you'll gain by solving real-life problems life this.
I won't blame you for preaching Just difficult for me to solve such problems as I start from near zero, never followed any computing course, and computing isn't my job (in addition, english isn't my mother tongue). But I'm persevering...


Quote:
Originally posted by Dave Kelly
http://rpmseek.com/rpm/abs-guide-3.1...:1822515:0:2:0
Also, Machtelt Garrels has a new book out. "bash guide for beginners" available on amazon.com.
I'll have a look, thanks
 
Old 07-07-2005, 11:44 AM   #13
eradrix
LQ Newbie
 
Registered: May 2004
Location: Switzerland
Distribution: whitebox
Posts: 12

Original Poster
Rep: Reputation: 0
Quote:
Originally posted by dustu76
Code:
### This presumes that you dont have any filename with spaces
for i in $(cat $cfg_file |tr -d " ") ; do
        src=$(echo $i |cut -d"|" -f1)
        target=$(echo $i |cut -d"|" -f2)
        excl_list=$(echo $i |cut -d"|" -f3 |tr "," " ")
        if [ "${excl_list}X" != "X" ] ; then
                for i in $(echo $excl_list |tr "," " ") ; do
                        excl_list_opt="$excl_list_opt --exclude $i"
                done
        fi
        echo "rsync $options $src $excl_list_opt $target"
done
Hi, just a little UP, many months later

I just found there was a little bug, with a missing 'excl_list_opt=""', which caused excluded list growing and growing after each loop...

Should be:
Code:
### This presumes that you dont have any filename with spaces
for i in $(cat $cfg_file |tr -d " ") ; do
        src=$(echo $i |cut -d"|" -f1)
        target=$(echo $i |cut -d"|" -f2)
        excl_list_opt=""
        excl_list=$(echo $i |cut -d"|" -f3 |tr "," " ")
        if [ "${excl_list}X" != "X" ] ; then
                for i in $(echo $excl_list |tr "," " ") ; do
                        excl_list_opt="$excl_list_opt --exclude $i"
                done
        fi
        echo "rsync $options $src $excl_list_opt $target"
done

Last edited by eradrix; 07-07-2005 at 11:46 AM.
 
Old 07-09-2005, 12:57 AM   #14
dustu76
Member
 
Registered: Sep 2004
Distribution: OpenSuSe
Posts: 153

Rep: Reputation: 30
Yeah from the code grave
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
rsync syntax for local copy hoover93 Linux - Software 3 12-21-2012 02:39 AM
shell script problem, want to use shell script auto update IP~! singying304 Programming 4 11-29-2005 05:32 PM
rsync & scponlyc shell: not working hamish Linux - Software 3 05-28-2005 07:15 AM
Using rsync for local server backup. Some issues. enygma Linux - Networking 0 02-03-2005 01:46 PM
Tightening shell accounts? SSH & RSync gian2oo1 Slackware 2 01-07-2005 02:15 PM


All times are GMT -5. The time now is 02:27 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration