Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place! |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
07-16-2012, 10:03 AM
|
#1
|
Senior Member
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora/Pop!_OS
Posts: 2,983
|
an other BASH question. trying to put a '- ' at the front of all lines in a txt file
i am trying to take advantage of the excludes.conf file that resides in a directory we backup nightly/hourly (for rsync) to get the --exclude-from=excludes.txt. i am trying to create the excludes.txt in my rsync script.
I found a bit of code, but I was unable to make it work. mainly due to the fact I dont know enough about coding.
Code:
for excludes.txt in /usr/rx30
do
while read -r line
do
echo "- ${line}"
done < ${excludes.txt} >temp
mv temp ${excludes.txt}
done
This provided the error on line 9, the last line of the code, excludes.txt is invalid. sorry i lost the error.
Code:
[rx30@rx30 ~]$ cat excludes.txt
*.viminfo*
*exclude*
*lifeline*.que
*lifeline*.lo*
*lifeline.sta*
*.bash_history*
*.tar*
*.bz2*
*.zip*
*.gz*
*core*
*.kde*
*.ICEauthority*
*rr_moved*
*.gnome*
*.mcop*
*.sawfish*
*.metacity*
*.nautilus*
*backup.log*
database-20120502.tar
datidx-20120521.tgz
*.bz
*.google*
*.mozilla*
*.openoffice*
*.DCOP*
*.adobe*
*.gconf*
*.thumbnail
in order to use this for rsync i would have to have a line that looks like:
from what i have read.
Thanks for the help.
|
|
|
07-16-2012, 11:25 AM
|
#2
|
LQ Guru
Registered: Sep 2009
Location: Perth
Distribution: Arch
Posts: 10,031
|
Just to confirm, the file you have shown is the input that you wish to change to now have dashes at the front of each line?
|
|
|
07-16-2012, 11:40 AM
|
#3
|
Senior Member
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora/Pop!_OS
Posts: 2,983
Original Poster
|
yes that is 100% correct. in the --exclude-from=foo.txt the file format needs to be a '-' for ignore and '+' to include in the rsync.
that is if i am reading that portion of the rsync correctly.
|
|
|
07-16-2012, 11:45 AM
|
#4
|
Moderator
Registered: Dec 2009
Location: Germany
Distribution: Whatever fits the task best
Posts: 17,148
|
Then you did get it wrong. The exclude file has to contain the bare file/folder names, one per line, without any additional prefixes or suffixes.
|
|
|
07-16-2012, 12:03 PM
|
#5
|
Senior Member
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora/Pop!_OS
Posts: 2,983
Original Poster
|
correct, I understand I did it wrong with my above bit of code. Again I am trying to take the existing excludes.conf and create excludes.txt to be used with the rsync command.
The excludes.conf contains the proper list of files/types/directories I would like to 'ignore' in the rsync. The format is slightly different between the two files as well. the .conf does not require the '_' or '+' while the .txt will.
I am looking for a bit of code to take the excludes.conf mv excludes.txt and at the same time add the '- ' to the beginning of each line without the ''.
|
|
|
07-16-2012, 12:05 PM
|
#6
|
Senior Member
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora/Pop!_OS
Posts: 2,983
Original Poster
|
this is some of what I have read on the --exclude-from=foo
Code:
The"exclude-from" file
The exclude file is a list of directory and file names to be excluded from the rsync destination e.g.
+ /source1/.fileA
- /source1/.*
- /source1/junk/
The format of the exclude file is one pattern per line. The pattern can be a literal string, wildcard, or character range.
In UNIX, hidden files start with a dot (.fileA is a hidden file), and * is a wildcard (.* means all hidden files).
A leading "+" means include the pattern. A leading "-" means exclude the pattern.
A path with a leading-slash must start with the source directory name (not the entire path).
Trailing slash is a directory (not a file). No trailing slash can be a directory or a file.
Lines in an exclude file are read verbatim. One frequent error is leaving some extra whitespace after a file name.
thus allowing for either a path and or a wildcard as well as hidden files.
|
|
|
07-16-2012, 12:48 PM
|
#7
|
Senior Member
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora/Pop!_OS
Posts: 2,983
Original Poster
|
ahh figured it out:
Code:
sed 's/^/- /' excludes.conf > excludes.txt
This did the trick:
this is the testing that i did, now i can add the little sed command to my backup scripts:
Code:
[rx30@rx30 ~]$ ls -l excl*
-rwxrwxr-x 1 rx30 group 208 Oct 21 2004 exclude.conf
-rwxrwxr-x 1 rx30 group 335 Jul 3 16:59 excludes.conf
[rx30@rx30 ~]$ chmod +x EXCLUDE
[rx30@rx30 ~]$ EXCLUDE
[rx30@rx30 ~]$ ls -l excl*
-rwxrwxr-x 1 rx30 group 208 Oct 21 2004 exclude.conf
-rwxrwxr-x 1 rx30 group 335 Jul 3 16:59 excludes.conf
-rw-rw-rw- 1 rx30 group 395 Jul 16 12:47 excludes.txt
[rx30@rx30 ~]$ cat excludes.txt
- *.viminfo*
- *exclude*
- *lifeline*.que
- *lifeline*.lo*
- *lifeline.sta*
- *.bash_history*
- *.tar*
- *.bz2*
- *.zip*
- *.gz*
- *core*
- *.kde*
- *.ICEauthority*
- *rr_moved*
- *.gnome*
- *.mcop*
- *.sawfish*
- *.metacity*
- *.nautilus*
- *backup.log*
- database-20120502.tar
- datidx-20120521.tgz
- *.bz
- *.google*
- *.mozilla*
- *.openoffice*
- *.DCOP*
- *.adobe*
- *.gconf*
- *.thumbnail
[rx30@rx30 ~]$ cat EXCLUDE
#!/bin/bash
sed 's/^/- /' excludes.conf > excludes.txt
exit;
Thanks for the help.
|
|
|
07-16-2012, 12:55 PM
|
#8
|
LQ Newbie
Registered: Mar 2006
Posts: 5
Rep:
|
Quote:
Originally Posted by lleb
ahh figured it out:
Code:
sed 's/^/- /' excludes.conf > excludes.txt
This did the trick:
|
You can also use simply:
Code:
sed -i 's/^/- /gm' excludes.conf
Sed will process the file inplace (-i), thus avoiding the need to move the temporary .txt file afterwards.
|
|
|
07-16-2012, 02:02 PM
|
#9
|
Senior Member
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora/Pop!_OS
Posts: 2,983
Original Poster
|
psyhe, are you saying it will modify the excludes.conf? i dont want that, i still need to have the output in excludes.txt as each of the two files is used for different things. the excludes.conf is for tar and the .txt is for rsync.
|
|
|
07-16-2012, 08:36 PM
|
#10
|
Member
Registered: Sep 2009
Location: Perth, W.A.
Distribution: Slackware, Debian, Gentoo, FreeBSD, OpenBSD
Posts: 208
Rep:
|
By the way, the original error is likely from these lines:
Code:
done < ${excludes.txt} >temp
mv temp ${excludes.txt}
By using ${...}, you specifying a variable named excludes.txt, when you really mean just the file itself which would just be plain excludes.txt.
The error comes because the shell can't find a variable called excludes.txt.
|
|
1 members found this post helpful.
|
07-16-2012, 09:02 PM
|
#11
|
Senior Member
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora/Pop!_OS
Posts: 2,983
Original Poster
|
ahh thank you padeen, that i understand. just trying to get in good coding habit and when i see $string, i automatically put {} around it.
helps to understand the difference in this case. thank you.
|
|
|
07-16-2012, 09:08 PM
|
#12
|
Moderator
Registered: Dec 2009
Location: Germany
Distribution: Whatever fits the task best
Posts: 17,148
|
I can only state again that this whole script is not necessary. This is my exclude-file, which I use with rsync to backup my Slackware system on my main computer:
Code:
/data
/server
/mnt
/tmp
/media
/proc
/dev
/sys
/lost+found
/win_d
You can use the notation with -/+ prefix, if you want to, but it is not necessary. You can just use plain text, respective your exclude.conf file.
|
|
|
All times are GMT -5. The time now is 12:40 AM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|