LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-25-2013, 10:55 AM   #1
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,630

Rep: Reputation: 495Reputation: 495Reputation: 495Reputation: 495Reputation: 495
unexpected results with --include-from= for rsync help please.


I am messing around again with my backup script for my web server. I am only wanting to backup the /www directory (yes I know this is not the default for apache) and a select group of files in one users home directory. In order to test the command before adding it to my script I have created the include.txt and used the following command:
Code:
$ rsync -aviS --include-from=/home/user/include.txt /www/ user@<home_URL>:/home/user/Monday/
my include.txt looks as follows:
Code:
-rw-rw-r-- 1 user  group    322 Feb 25 10:35 include.txt

$ cat include.txt
+ /home/user/hosts
+ /home/user/ifcfg-eth0
+ /home/user/ifcfg-eth1
+ /home/user/httpd.conf
+ /home/user/httpd.conf.dist
+ /home/user/httpd.conf.rpmnew
+ /home/user/php.conf
+ /home/user/proxy_ajp.conf
+ /home/user/webalizer.conf
+ /home/user/welcome.conf
+ /home/user/resolv.conf
+ /home/user/mega-file
+ /home/user/z_local_aliases.sh
from what I have read on the man page as well as looking at a few
examples online this should work, yet the rsync command is not grabbing
any of the files from the include.txt

what did i screw up?
 
Old 02-25-2013, 11:03 AM   #2
TobiSGD
Moderator
 
Registered: Dec 2009
Location: Germany
Distribution: Whatever fits the task best
Posts: 17,130
Blog Entries: 2

Rep: Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825
Just a guess, but none of the files in your include-file is in /www, so it seems natural for me that they are not picked up. try what happens if you add /home/user to your rsync command:
Code:
$ rsync -aviS --include-from=/home/user/include.txt /home/user /www/ user@<home_URL>:/home/user/Monday/
Alternatively try it with the --files-from= option, which looks more like what you want.

Last edited by TobiSGD; 02-25-2013 at 11:05 AM.
 
1 members found this post helpful.
Old 02-25-2013, 11:09 AM   #3
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,800
Blog Entries: 4

Rep: Reputation: 286Reputation: 286Reputation: 286
Beside files mentioned in /home/user/include.txt, is it taking backup of /www/?

Can you try:
Code:
$ rsync -aviS --files-from=/home/user/include.txt /www/ user@<home_URL>:/home/user/Monday/
 
1 members found this post helpful.
Old 02-25-2013, 11:26 AM   #4
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,630

Original Poster
Rep: Reputation: 495Reputation: 495Reputation: 495Reputation: 495Reputation: 495
shivaa, yes the /www directory is backing up flawlessly.

TobiSGD/shivaa when i replaced --include-from= with files-from= i got the following results:

Code:
(user@mine) ~ $ rsync -aviS --files-from=/home/user/include.txt /www/ user@<home_URL>:/home/user/Monday/
building file list ... rsync: link_stat "/www/+ /home/user/hosts" failed: No such file or directory (2)
rsync: link_stat "/www/+ /home/user/ifcfg-eth0" failed: No such file or directory (2)
rsync: link_stat "/www/+ /home/user/ifcfg-eth1" failed: No such file or directory (2)
rsync: link_stat "/www/+ /home/user/httpd.conf" failed: No such file or directory (2)
rsync: link_stat "/www/+ /home/user/httpd.conf.dist" failed: No such file or directory (2)
rsync: link_stat "/www/+ /home/user/httpd.conf.rpmnew" failed: No such file or directory (2)
rsync: link_stat "/www/+ /home/user/php.conf" failed: No such file or directory (2)
rsync: link_stat "/www/+ /home/user/proxy_ajp.conf" failed: No such file or directory (2)
rsync: link_stat "/www/+ /home/user/webalizer.conf" failed: No such file or directory (2)
rsync: link_stat "/www/+ /home/user/welcome.conf" failed: No such file or directory (2)
rsync: link_stat "/www/+ /home/user/resolv.conf" failed: No such file or directory (2)
rsync: link_stat "/www/+ /home/user/mega-file" failed: No such file or directory (2)
rsync: link_stat "/www/+ /home/user/z_local_aliases.sh" failed: No such file or directory (2)
done

sent 18 bytes  received 12 bytes  20.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1039) [sender=3.0.6]
(user@mine) ~ $ d
total 268
drwx------ 5 user group   4096 Feb 25 11:18 ./
drwxr-xr-x 4 root root  4096 Jul  6  2012 ../
-rw-rw-r-- 1 user group     37 Feb 25 10:35 hosts
-rw-r--r-- 1 user group  11942 Feb 25 10:35 httpd.conf
-rw-rw-r-- 1 user group  33677 Feb 25 10:35 httpd.conf.dist
-rw-r--r-- 1 user group 33726 Feb 25 10:35 httpd.conf.rpmnew
-rw-rw-r-- 1 user group    132 Feb 25 10:35 ifcfg-eth0
-rw-rw-r-- 1 user group    112 Feb 25 10:35 ifcfg-eth1
-rw-rw-r-- 1 user group    322 Feb 25 10:35 include.txt
drwxr-xr-x 2 user group  12288 Feb 25 00:00 logs/
-rw-r--r-- 1 user group    771 Feb 25 10:35 php.conf
-rw-r--r-- 1 user group    567 Feb 25 10:35 proxy_ajp.conf
-rw-rw-r-- 1 user group    45 Feb 25 10:35 resolv.conf
-rw-r--r-- 1 user group    352 Feb 25 10:35 webalizer.conf
-rw-r--r-- 1 user group    299 Feb 25 10:35 welcome.conf
-rwxr-xr-x 1 user group   2307 Jan  5 21:39 wwwbackup*
-rw-r--r-- 1 user group    36 Feb 25 10:35 z_local_aliases.sh
(user@mine) ~ $ pwd
/home/user
why would rsync say those files that are clearly in the path /home/user say no such file?
 
Old 02-25-2013, 11:36 AM   #5
TobiSGD
Moderator
 
Registered: Dec 2009
Location: Germany
Distribution: Whatever fits the task best
Posts: 17,130
Blog Entries: 2

Rep: Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825Reputation: 4825
It seems that rsync mixes the /www-path with the paths given in the include-file. Try it without the /www path, if that works you maybe have to use two rsync commands.
 
1 members found this post helpful.
Old 02-25-2013, 12:03 PM   #6
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,800
Blog Entries: 4

Rep: Reputation: 286Reputation: 286Reputation: 286
Try it:
Code:
~$ rsync -avz /www/ user@<home_URL>:/home/user/Monday/
~$ rsync -avz -F $(cat /home/user/include.txt) user@<home_URL>:/home/user/Monday/
 
1 members found this post helpful.
Old 02-25-2013, 01:41 PM   #7
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,630

Original Poster
Rep: Reputation: 495Reputation: 495Reputation: 495Reputation: 495Reputation: 495
interesting, i tried a combination of what you two were saying with the following results:

Code:
(user@mine) ~ $ rsync -aviS -F $(cat /home/user/include.txt) /www/ user@<home_URL>:/home/user/Monday/
sending incremental file list
rsync: link_stat "/home/user/+" failed: No such file or directory (2)
rsync: link_stat "/home/user/+" failed: No such file or directory (2)
rsync: link_stat "/home/user/+" failed: No such file or directory (2)
rsync: link_stat "/home/user/+" failed: No such file or directory (2)
rsync: link_stat "/home/user/+" failed: No such file or directory (2)
rsync: link_stat "/home/user/+" failed: No such file or directory (2)
rsync: link_stat "/home/user/+" failed: No such file or directory (2)
rsync: link_stat "/home/user/+" failed: No such file or directory (2)
rsync: link_stat "/home/user/+" failed: No such file or directory (2)
rsync: link_stat "/home/user/+" failed: No such file or directory (2)
rsync: link_stat "/home/user/+" failed: No such file or directory (2)
rsync: link_stat "/home/user/+" failed: No such file or directory (2)
rsync: link_stat "/home/user/+" failed: No such file or directory (2)
<f+++++++++ hosts
<f+++++++++ httpd.conf
<f+++++++++ httpd.conf.dist
<f+++++++++ httpd.conf.rpmnew
<f+++++++++ ifcfg-eth0
<f+++++++++ ifcfg-eth1
<f+++++++++ mega-file
<f+++++++++ php.conf
<f+++++++++ proxy_ajp.conf
<f+++++++++ resolv.conf
<f+++++++++ webalizer.conf
<f+++++++++ welcome.conf
<f+++++++++ z_local_aliases.sh

sent 251699 bytes  received 495 bytes  168129.33 bytes/sec
total size is 12028156741  speedup is 47694.06
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1039) [sender=3.0.6]
If I have to run 2x rsync that is fine, but there should be a way for rsync to follow the include/exclude path it is provided. FYI, other then the error msg. this did work and the files that are in the include.txt did get transfered to the backup server properly.
 
Old 02-25-2013, 06:13 PM   #8
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,630

Original Poster
Rep: Reputation: 495Reputation: 495Reputation: 495Reputation: 495Reputation: 495
ive chosen to ignore those errors as the files get transfer but i did make some minor modifications to my script:

[code[### -c Checksum comparison (takes longer, but recommended for backups)
### -H Preserve hard linkes
### -X Preserve Extended Attributes (ACLs, SELinux contexts, etc...)
### -i "in-place" and dropped the -S "sparse" option

rsync -aciHXv -F $(cat /home/user/include.txt) /www/ .....[code]

that seems to work rather well. it does drastically increase the time to complete the backup, but having the checksum to me is more then worth it.

Thanks again for the help both of you. I have hit the "yes" for this post helped from you both.
 
Old 02-25-2013, 08:40 PM   #9
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,800
Blog Entries: 4

Rep: Reputation: 286Reputation: 286Reputation: 286
Quote:
Originally Posted by lleb
...If I have to run 2x rsync that is fine, but there should be a way for rsync to follow the include/exclude path it is provided. FYI, other then the error msg. this did work and the files that are in the include.txt did get transfered to the backup server properly.
I guess, in that case, following should work fine (copying both in a single rsync):
Code:
~$ rsync -aciHXv -F $(cat /home/user/include.txt) /www/ user@<home_URL>:/home/user/Monday/
 
Old 02-25-2013, 09:58 PM   #10
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,630

Original Poster
Rep: Reputation: 495Reputation: 495Reputation: 495Reputation: 495Reputation: 495
for now its a good enough work around. ill learn more and fix it as i go.

i do like what i learned about the other options ive been looking for a Checksum of sorts for rsync. good to find that now.
 
Old 02-25-2013, 10:39 PM   #11
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,800
Blog Entries: 4

Rep: Reputation: 286Reputation: 286Reputation: 286
Yeah..! Good learning experience for me as well.
 
Old 02-25-2013, 10:48 PM   #12
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,630

Original Poster
Rep: Reputation: 495Reputation: 495Reputation: 495Reputation: 495Reputation: 495
shivaa, i really enjoy your responces and posts. they are very educational and extremely helpful. Please keep posting to LQ.

and Thank you again.
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] sed loop gives unexpected results jgombos Programming 13 09-10-2011 01:33 AM
find -mtime gives unexpected results amchargue Linux - Newbie 3 08-12-2010 05:19 PM
rsync can not rsync files with include filter... xiutuo Linux - Server 2 07-23-2010 03:10 AM
[SOLVED] Sort generates unexpected results danielbmartin Linux - Newbie 5 06-08-2010 03:39 AM
find command displays unexpected results helptonewbie Linux - Newbie 5 08-12-2008 03:25 AM


All times are GMT -5. The time now is 08:55 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration