BASH Script --> read a file & cp -r to my ~/ directory

manwithaplan · 10-05-2009, 01:40 AM

I would like to read a file output from a find, and a grep. This is a horrendous grep...!(but works) This finds the folders I need copied, and outputs to a tmp file

I would like to then read the file and use cp -r to copy the directories specified in the file. I have tried xargs, and many other ways, and this is what I could only come with. I'm new to learning this... sooo.... If I could get some pointers on shortening the code, or a way of a better grep

for future references .. that would be great...

Code:

#!/bin/bash

builds="/media/ARCH/var/abs"
log1="/tmp/log.txt"

if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi

if [ -e $log1 ]; then
     rm /tmp/log.txt
fi
   
 find $builds -maxdepth 2 -type d | grep -i -e 'core/acl' -e 'core/attr' \
	-e 'core/fakeroot' -e 'core/pacman$' -e 'extra/rsync$' -e 'core/libfetch' \
	-e 'core/libarchive' -e 'core/openssl' -e 'core/wget$' >> /tmp/log.txt  
               
exec xargs -a /tmp/log.txt cp -r ~/builds/test 

exit

Here's the output of the grep:

Code:

/media/ARCH/var/abs/core/wget
/media/ARCH/var/abs/core/fakeroot
/media/ARCH/var/abs/core/acl
/media/ARCH/var/abs/core/pacman
/media/ARCH/var/abs/core/attr
/media/ARCH/var/abs/core/libfetch
/media/ARCH/var/abs/core/openssl
/media/ARCH/var/abs/core/libarchive
/media/ARCH/var/abs/extra/rsync

chrism01 · 10-05-2009, 01:49 AM

This line doesn't make sense to me

Code:

exec xargs -a /tmp/log.txt cp -r ~/builds/test

I can't tell if you want to cp output from find to or from ~/builds/test.
Also, you use 'builds' as part of an absolute path and and defined as a var at the top of your prog.
You also define
log1="/tmp/log.txt"
yet use the value instead of the variable twice later in the prog.

Anyway, once you've got a list of dirs in a file, you can

Code:

for dir in `cat $log1`
do
    cp -r $dir ${builds}/test/
done

or similar; but as I say, it's not clear to me what you actually want to do.

manwithaplan · 10-05-2009, 02:16 AM

Quote:

Originally Posted by chrism01

This line doesn't make sense to me

Code:

exec xargs -a /tmp/log.txt cp -r ~/builds/test

I can't tell if you want to cp output from find to or from ~/builds/test.
Also, you use 'builds' as part of an absolute path and and defined as a var at the top of your prog.
You also define
log1="/tmp/log.txt"
yet use the value instead of the variable twice later in the prog.

Anyway, once you've got a list of dirs in a file, you can

Code:

for dir in `cat $log1`
do
    cp -r $dir ${builds}/test/
done

or similar; but as I say, it's not clear to me what you actually want to do.

Better explanation:

Code:

log1="/tmp/log.txt"

if [ -e $log1 ]; then
     rm /tmp/log.txt
fi

I was using this to check if /tmp/log.txt existed, and if so, remove it before writing to the log file again.

Code:

xargs -a /tmp/log.txt cp -r ~/builds/test

I read the man pages of "xargs" /w the -a option ... and I was trying to "cp -r" all the directories in the list I generated in the log file from the "find & "grep" to my test directory.

The "$builds" variable was a mistake, I changed it... It was the dir path to where "find" would search for my directories to be "grepped".

I am new with "for" loops ... and I didn't know I could use 'cat' to read and copy directories.

Here is the revision that works:

Code:

#!/bin/bash

blds="/media/ARCH/var/abs"
log1="/tmp/log.txt"

if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root" 1>&2
   exit 1
fi

if [ -e $log1 ]; then
     rm /tmp/log.txt
fi
   
 find ${blds} -maxdepth 2 -type d | grep -i -e 'core/acl' -e 'core/attr' \
	-e 'core/fakeroot' -e 'core/pacman$' -e 'extra/rsync$' -e 'core/libfetch' \
	-e 'core/libarchive' -e 'core/openssl' -e 'core/wget$' >> /tmp/log.txt  
               
for dir in `cat $log1`
do
    cp -r $dir ~/builds/test
done             

exit

This reads the log and copies the folders to my test dir....

Thanks ... Though is there any way I can shorten my grep ...? or an easier way doing this...?

slakmagik · 10-05-2009, 02:27 AM

I'm not really looking at the script itself and, on the grep, I don't know how precise you need to be - the following might work:

Code:

egrep '(core|extra)/(acl|attr|fakeroot|pacman|rsync|libfetch|libarchive|openssl|wget)'

but it would allow 'extra/acl' as well as 'core'acl', so this would be a little more restrictive:

Code:

egrep 'core/(acl|attr|fakeroot|pacman|rsync|libfetch|libarchive|openssl|wget)|extra/rsync'

but I see you have some match EOL and some not. So that might have to be accounted for. But the (parenthetical|pipe) constructs with egrep should help give the idea.

-- actually, looking at the script, some initial thoughts are (most of which chrism01 already noted):

Code:

if [ -e $log1 ]; then
     rm /tmp/log.txt # you should use your variable
fi
   
 find ${blds} -maxdepth 2 -type d | grep -i -e 'core/acl' -e 'core/attr' \
	-e 'core/fakeroot' -e 'core/pacman$' -e 'extra/rsync$' -e 'core/libfetch' \
	-e 'core/libarchive' -e 'core/openssl' -e 'core/wget$' >> /tmp/log.txt  # ditto
               
for dir in `cat $log1` # use the $(cat log) form

and back on the regex, find has a '-regex' flag so you probably can get rid of the grep altogether.

-- Oh, and exit is not needed unless you're accounting for an error condition and exiting non-zero.

-- Wait. Why are you doing it this way? Why not have 'find' find the directories and use the '-exec' arg to cp them?

Code:

find SRC -regextype posix-egrep -regex '(core|extra)/(acl|attr|fakeroot|pacman|rsync|libfetch|libarchive|openssl|wget)' -exec cp -r {} DEST \;

Something like that, anyway.

konsolebox · 10-05-2009, 02:51 AM

You probably won't need grep if you use -path, -ipath, -regex or -iregex in find. Do you really need to save the list to a temporary file? How 'bout using command substitution or process substitution? Also after 'exec', 'exit' probably will never be executed.

for reference, do:

# man find && man bash

manwithaplan · 10-05-2009, 05:16 AM

Quote:

Originally Posted by slakmagik

I'm not really looking at the script itself and, on the grep, I don't know how precise you need to be - the following might work:

Code:

egrep '(core|extra)/(acl|attr|fakeroot|pacman|rsync|libfetch|libarchive|openssl|wget)'

but it would allow 'extra/acl' as well as 'core'acl', so this would be a little more restrictive:

Code:

egrep 'core/(acl|attr|fakeroot|pacman|rsync|libfetch|libarchive|openssl|wget)|extra/rsync'

but I see you have some match EOL and some not. So that might have to be accounted for. But the (parenthetical|pipe) constructs with egrep should help give the idea.

-- actually, looking at the script, some initial thoughts are (most of which chrism01 already noted):

Code:

if [ -e $log1 ]; then
     rm /tmp/log.txt # you should use your variable
fi
   
 find ${blds} -maxdepth 2 -type d | grep -i -e 'core/acl' -e 'core/attr' \
	-e 'core/fakeroot' -e 'core/pacman$' -e 'extra/rsync$' -e 'core/libfetch' \
	-e 'core/libarchive' -e 'core/openssl' -e 'core/wget$' >> /tmp/log.txt  # ditto
               
for dir in `cat $log1` # use the $(cat log) form

and back on the regex, find has a '-regex' flag so you probably can get rid of the grep altogether.

-- Oh, and exit is not needed unless you're accounting for an error condition and exiting non-zero.

-- Wait. Why are you doing it this way? Why not have 'find' find the directories and use the '-exec' arg to cp them?

Code:

find SRC -regextype posix-egrep -regex '(core|extra)/(acl|attr|fakeroot|pacman|rsync|libfetch|libarchive|openssl|wget)' -exec cp -r {} DEST \;

Something like that, anyway.

Thanks....

I wasn't familiar with regex ... & a side note is that I had tried using -exec to cp to my directories... I just couldn't get it to copy.
That's why I was using xargs at first.

So that is why I decided to write to a temp file and then read the contents to copy the folders.

I am having issues with your egreps filtering the EOL's $ and coping the dir... Since this being the first time I had ever used egrep, I'll have to look closer at the man pages.

Here's what I am using now ... NOTE, I can't remember the escape sequence in a case statement.(I thought it was 255, I want the loop to break once the user hits the escape key).

Code:

#!/bin/bash

blds="/media/ARCH/var/abs"
log1="/tmp/log.txt"

clear
leave=no  
while [ $leave = no ]; do 
  cat << END
This script will copy all packages needed from ABS to a build directory 
that you specify. From there, makepkg will build each package needed to install
pacman into your CLFS installation. 
     
In addition, this script will copy the contents from the build diectories onto you
CLFS partition. It is recommended that you mount your CLFS partition before 
continuing with this script.
     
Press Enter to continue or X to exit....
END

read selection
  case $selection in
   
       *) clear && break ;;

     255) break && exit;;
  
  esac
done    

if [ -e ${log1} ]; then
  rm ${log1}
fi
   
 find ${blds} -maxdepth 2 -type d | grep -i -e 'core/acl' -e 'core/attr' \
	-e 'core/fakeroot' -e 'core/pacman$' -e 'extra/rsync$' -e 'core/libfetch' \
	-e 'core/libarchive' -e 'core/openssl' -e 'core/wget$' >> ${log1}  
               
for dir in $(cat ${log1})
do
    cp -r ${dir} ~/builds/test
done             
    rm ${log1}

chrism01 · 10-05-2009, 11:25 PM

You don't need to use ${var} all the time, only if 'var' is embedded inside string eg if you have

var1=string1

and actual filename is string1string2, then you would use

rm ${var1}string2

so that the shell parser can figure out what's a var and what's a literal.
However, if the filename is just string1, then

rm $var1

is sufficient.

You should read

http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/

and play around until you're happy with bash concepts.