LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 04-27-2013, 12:57 PM   #1
amejiag
LQ Newbie
 
Registered: Apr 2013
Posts: 5

Rep: Reputation: Disabled
script using grep to filter 10 tar files


I have to make a script using grep filtering a pattern in a 10 files that have been compressed tar format, this is what I have done until now.

#!/bin/sh

cd ../Data

ls *.tar #list file with extention tar

tar – xzvf *.tar > ../Result/file01.tar #decompress tar file and send to directory Result as
file01.tar

for i = 1; $i <=10; $i=++; #for i as the first file until the 10 th file find a pattern
from tar files in Data and send to Result directory as fileresult.out

do

grep “pattern” Data/*.tar > Result/fileresult.out
echo “\n”

done

Also I have to improve this script

#!/bin/sh

# ../Data/subject01.fa ../Data/query01.fa ../Data/subject02.fa ../Data/query02.fa


cd ../Data

formatdb -i subject01.fa -p F
formatdb -i subject02.fa -p F


cd ../Code

blastall -p blastn -d $1 -i $2 -F F -e 1e-5 -o ../Results/blast01.raw -m 9
blastall -p blastn -d $3 -i $4 -F F -e 1e-5 -o ../Results/blast02.raw -m 9


for i in ../Results/b*.raw;

do

echo $i >> ../Results/filtro100.out
grep 100.00 $i >> ../Results/filtro100.out
echo "\n" >> ../Results/filtro100.out

done

I modified as:

#!/bin/sh

# ../Data/subject01.fa ../Data/query01.fa ../Data/subject02.fa ../Data/query02.fa

cd ../Data

formatdb -i subject01.fa -p F -i subject02.fa -p F

cd ../Code



blastall -p blastn -d ../Data/subject01.fa -i ../Data/query01.fa -d ../Data/subject02.fa -i ../Data/query02.fa -F F -e 1e-5 -o ../Results/blast01.raw -m 9 ../Results/blast02.raw -m 9 #without blast02.raw it run as only blast01.raw

#blastall -p blastn -d -F F -e 1e-5 -o ../Results/blast02.raw -m 9
cat ../Results/blast01.raw ../Results/blast02.raw > ../Results/prueba.raw

for i in ../Results/p*.raw;
do

echo $i >> ../Results/filtro100.out
grep 100.00 $i >> ../Results/filtro100.out
echo "\n" >> ../Results/filtro100.out

done

when I run this script come out this error
[blastall] ERROR: Arguments must start with '-' (the offending argument #19 was: '../Results/blast02.raw')

as I what to incluide not only filter 100 I change the last part as

do

echo $i >> ../Results/filtro100.out
grep ["$i" -le 100.00] || ["$i" -ge 90.00] $i >> ../Results/filtro100.out
echo "\n" >> ../Results/filtro100.out

done

but is send an error, how I can to fix this and it is something that I can improve of this script

Thanks you for the help

Last edited by amejiag; 04-28-2013 at 11:16 PM.
 
Old 04-27-2013, 03:55 PM   #2
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 18,316

Rep: Reputation: 3878Reputation: 3878Reputation: 3878Reputation: 3878Reputation: 3878Reputation: 3878Reputation: 3878Reputation: 3878Reputation: 3878Reputation: 3878Reputation: 3878
Quote:
Originally Posted by amejiag View Post
My name is Adriana, and I have to make a script using grep filtering a pattern in a 10 files where are compress with tar, I hope to heard from you soon. thanks your very much
We are not going to write scripts for you, but we will be glad to HELP you. So, post what you've written/tried so far, and where you're stuck. Otherwise, you can read the man pages on the commands, and look up any of the VERY easily-found bash scripting tutorials on Google, and start there. Also, be aware that we aren't going to give you answers to homework questions, either.
 
Old 04-28-2013, 08:07 AM   #3
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,254

Rep: Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328
In addition, tar is not a compression tool; it just collects all input into one big file archive.
If you created with with the z (gzip) or j (bzip2) flag, then it will be compressed as well.
If the archive is compressed, you will need to uncompress it first.
 
Old 04-28-2013, 08:52 AM   #4
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,800
Blog Entries: 4

Rep: Reputation: 286Reputation: 286Reputation: 286
What you've done so far and let's know if you're stuck somewhere. Following would be helpful for you:
http://tldp.org/LDP/abs/html/
http://linux.die.net/man/1/tar
http://linux.die.net/man/1/gzip
 
Old 04-28-2013, 11:35 AM   #5
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,630

Rep: Reputation: 495Reputation: 495Reputation: 495Reputation: 495Reputation: 495
Quote:
Originally Posted by amejiag View Post
My name is Adriana, and I have to make a script using grep filtering a pattern in a 10 files where are compress with tar, I hope to heard from you soon. thanks your very much
sounds an awful lot like homework to me.

as stated post what you have done, we will be glad to help you fix your errors, but not do you homework for you as that is against forum rules.
 
Old 04-28-2013, 06:26 PM   #6
amejiag
LQ Newbie
 
Registered: Apr 2013
Posts: 5

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by lleb View Post
sounds an awful lot like homework to me.

as stated post what you have done, we will be glad to help you fix your errors, but not do you homework for you as that is against forum rules.
Sorry about this impression, It is my fault about my message, but I do not try that someone do the "homework" for my, how I can learn from it if I do not do an effort. Thanks you for take your time and answer my message
 
Old 04-28-2013, 08:51 PM   #7
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,332
Blog Entries: 55

Rep: Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533
Quote:
Originally Posted by amejiag View Post
(..) I do not try that someone do the "homework" for my, how I can learn from it if I do not do an effort.
Then best start by posting what you tried.

Here's some BASH scripting guides should you need them:
http://www.tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html
http://www.tldp.org/LDP/Bash-Beginne...tml/index.html
http://www.gnu.org/software/bash/man...ode/index.html
http://www.grymoire.com/Unix/Sh.html
http://www.tldp.org/LDP/abs/html/
http://mywiki.wooledge.org/BashFAQ
http://mywiki.wooledge.org/BashPitfalls

*Note the Advanced BASH Scripting Guide is not the first link to read (and for good reasons).
 
Old 04-29-2013, 10:27 PM   #8
amejiag
LQ Newbie
 
Registered: Apr 2013
Posts: 5

Original Poster
Rep: Reputation: Disabled
grep a patter in 10 file comprised in a tar file

I made this script

#!/bin/sh

cd ../Data_taller_eval/Data_preg2.tar

#list file with extention tar

tar –xzvf Data_preg2.tar > ../Result_taller_eval #decompress tar file and send to directory Result as file01.tar


#for (my $i=1;$i<=10;$i++ #for i as the first tar file until the 10 th file find a pattern from tar files Data and send to Result directory fileresult.out

#do

#grep “agcct” Data/*.sh > Result/fileresult.out
#echo “\n”

#done

I try to run the first part but I get this error

tar: opción incorrecta -- «�»
Intente`tar --help' o `tar --usage' para mayor información.
I when to man tar but I do not what to do
when I run all the script give me an error
./scriptgrep.sh: 10: Syntax error: Bad for loop variable
How I make an good for loop variable

Thanks you for your help
 
Old 04-29-2013, 11:39 PM   #9
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,800
Blog Entries: 4

Rep: Reputation: 286Reputation: 286Reputation: 286
Fisrt thing is to add debug mode i.e. set -xv in your script and check that where scripts stuck.

Second, in order to check content of a tar file, use -t flag, as:
Code:
~$ tar -tvf foo.tar
Note that you cannot simply do cd on a tar file, because a .tar is not a directory, but a file.

However, you can grep a pattern simply, as:
Code:
~$ cd ../Data_taller_eval/
~$ grep <pattern> <(tar -tvf Data_preg2.tar)
 
Old 04-30-2013, 12:45 AM   #10
amejiag
LQ Newbie
 
Registered: Apr 2013
Posts: 5

Original Poster
Rep: Reputation: Disabled
Thanks you for your answer I did as

#!/bin/sh

set -xv

cd ../Data_taller_eval/


grep "agcct" <(tar -tvf Data_preg2.tar) #list file with extention tar

#tar –xzvf Data_preg2.tar > ../Result_taller_eval #decompress tar file and send to directory Result as file01.tar


#for (my $i=1;$i<=10;$i++ #for i as the first tar file until the 10 th file find a pattern from tar files Data and send to Result directory fileresult.out

#do

#grep “agcct” Data/*.sh > Result/fileresult.out
#echo “\n”

#done

and the answer is cd ../Data_taller_eval/
+ cd ../Data_taller_eval/


grep "agcct" <(tar -tvf Data_preg2.tar) #list file with extention tar
./scriptgrep.sh: 8: Syntax error: "(" unexpected
I do not recognized what is the syntax error and how I make this with 10 file the last part I used for (my $i=1;$i<=10;$i++ but it also get error

Thanks you
 
Old 04-30-2013, 01:03 AM   #11
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,800
Blog Entries: 4

Rep: Reputation: 286Reputation: 286Reputation: 286
Use /bin/bash instead of /bin/sh.

You can also try like:
Code:
#!/bin/bash
while read -r file
do
grep 'agcct' $file
done < <(tar -tvf Data_preg2.tar | awk -F' ' '{print $6}')

Last edited by shivaa; 04-30-2013 at 08:51 AM. Reason: Command substitution added
 
Old 04-30-2013, 01:27 AM   #12
amejiag
LQ Newbie
 
Registered: Apr 2013
Posts: 5

Original Poster
Rep: Reputation: Disabled
Thanks, I try to run this

#!/bin/bash


set -xv

cd ../Data_taller_eval/

while read -r file
do
grep 'agcct' $file
done < $(tar -tvf Data_preg2.tar)

and I get this

cd ../Data_taller_eval/
+ cd ../Data_taller_eval/

while read -r file
do
grep 'agcct' $file
done < $(tar -tvf Data_preg2.tar)
tar -tvf Data_preg2.tar)
tar -tvf Data_preg2.tar
++ tar -tvf Data_preg2.tar
tar: Data_preg2.tar: No se puede read: Es un directorio
tar: Al principio de la cinta, se terminará ahora
tar: Error is not recoverable: exiting now
tar -tvf Data_preg2.tar)
tar -tvf Data_preg2.tar
++ tar -tvf Data_preg2.tar
tar: Data_preg2.tar: No se puede read: Es un directorio
tar: Al principio de la cinta, se terminará ahora
tar: Error is not recoverable: exiting now
./scriptgrep1.sh: línea 11: $(tar -tvf Data_preg2.tar): redireccionamiento ambiguo

line 11 is ambiguos and how I can to make this to 10 file ?
 
Old 04-30-2013, 08:52 AM   #13
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,800
Blog Entries: 4

Rep: Reputation: 286Reputation: 286Reputation: 286
Just try it now:
Code:
#!/bin/bash
while read -r file
do
grep 'agcct' $file
done < <(tar -tvf Data_preg2.tar | awk -F' ' '{print $6}')
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Need a small script to grep file details from around 2000 files. v11pandey Linux - Newbie 4 04-19-2013 04:56 AM
Need a shell script to filter out unique hostnames from two text files. Thaidog Programming 6 11-01-2011 02:58 PM
How to filter script files in bash programming Marcopredador Linux - Newbie 8 02-12-2010 01:01 PM
Script to grep log files based on system date cyclegar Programming 2 05-05-2009 07:08 PM
How to filter files in files and files which are in a sub-directory with "grep"? Piero Linux - Newbie 9 08-29-2003 03:38 AM


All times are GMT -5. The time now is 08:10 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration