LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-13-2011, 10:29 AM   #1
emmanouelaaaaa
LQ Newbie
 
Registered: Oct 2011
Posts: 3

Rep: Reputation: Disabled
Problem with while real line and awk


Hello,

Sorry for the simplicity of this question, but the errors are getting me nowhere.
I have a (tab separated) file (my_snps) that looks like this:
rs1 chr21 9555555
rs2 chr21 9555556
rs3 chr21 9555557
rs5 chr22 9555558
what Im trying to do is read each line in, store each value in a variable and then extract the line number (3rd column) from the files named chrxx.conservation.

So far I have this script:

#!/bin/bash
cat my_snps | while read line
do
rs = `awk '{print $1;}' $line`
chr = `awk '{print $2;}' $line`
pos = `awk '{print $3;}' $line`
sed "$posq;d" $chr.conservation
done

but Im getting this error:
awk: can't open file rs1
source line number 1
rs2 chr21 9555556 rs3 chr21 9555557 rs5 chr21 9555558
rs9 chr21 9555559
awk: can't open file rs1
source line number 1
./get_scores_unix: line 6: chr: command not found
./get_scores_unix: command substitution: line 7: unexpected EOF while looking for matching `''
./get_scores_unix: command substitution: line 8: syntax error: unexpected end of file
./get_scores_unix: line 7: pos: command not found

sed: .conservation: No such file or directory

I also tried this for the variables:
rs = `echo $line | awk '{ print $1; }'`
chr = echo $line | awk '{ print $2; }'`
pos = `echo $line | awk '{ print $3; }'`
but its not working either, with the error:

rs2 chr21 9555556 rs3 chr21 9555557 rs5 chr21 9555558
rs9 chr21 9555559
./get_scores_unix: line 7: chr: command not found
./get_scores_unix: line 8: pos: command not found


Any help would be much appreciated!
 
Old 10-13-2011, 10:50 AM   #2
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,245

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
Firstly, please use [code][/code] tags to make what you have written more legible.

First issue is that spaces are not allowed in assignments within bash:
Code:
rs = `awk '{print $1;}' $line`

# Should be

rs=`awk '{print $1;}' $line`

# or the more readable form

rs=$(awk '{print $1;}' $line)
Second, variable names can only be expanded if they exist or else they will be null:
Code:
sed "$posq;d" $chr.conservation
You do not have any variables called 'posq'

Third, you have used cat to provide the while loop with data. This is not so much an error but should you require access
to any of your variables outside the loop they will not exist as this format creates a subshell. If this is an issue
try the form:
Code:
while read -r var
do
done<your_file_here
Finally, either go with awk or go with bash but this poor use of both is not helping you. If you want to put each value
in its own variable have a look at what your while loop is really capable of.
 
Old 10-13-2011, 11:07 AM   #3
crts
Senior Member
 
Registered: Jan 2010
Posts: 1,606

Rep: Reputation: 448Reputation: 448Reputation: 448Reputation: 448Reputation: 448
Hi,

As mentioned by grail, you might want to do it all with awk or do not use it at all.
Here is a solution without awk:
Code:
while read line; do
 set -- $line
 sed "${3} d" "${2}.conservation"
done <file
I did not see you using the 'rs' variable at all in your example so I am not sure if your actual script is more complex or if the variable is simply redundant.
 
Old 10-13-2011, 12:35 PM   #4
emmanouelaaaaa
LQ Newbie
 
Registered: Oct 2011
Posts: 3

Original Poster
Rep: Reputation: Disabled
Thanks crts and grail!
Sorry but Im a really newbie in unix and scripting so I tend to go with whatever works for me (ie patches from other posts).
The rs variable is sort of an index for my files so I was planning to use it as the first column of my output (I want to use these for quite large files).
I have changed my code according to ctrs's code but I only get back the line from the my_snps file and an empty line for the sed, i.e.
rs1 chr21 9555555

rs2 chr21 9555556

rs3 chr21 9555557

But when I try it from outside the script it works:
Code:
sed '9555555q;d' chr21.conservation
That's what the q was for, it wasn't the variable posq but the variable pos with the q afterwards. Again, sorry if its too basic but I need to make this work soon so don't have time to immerse myself in the unix tutorials.
 
Old 10-13-2011, 01:04 PM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,245

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
Could you show us exactly what code you are using? I ask as there is no echo statement in crts' code so I do not see how you can get back the line from the file??

Also, my reference to posq was because the way you wrote it that is what bash sees, ie there is a big difference between the following:
Code:
echo $posq

echo ${pos}q
The first looks to expand the variable posq while the second appends the letter q to whatever the variable pos expands to.

Maybe if you also explained what the chrXX.conservation files look like and what you are trying to do to them it may help (as this seems to be the actual question)
 
Old 10-13-2011, 02:44 PM   #6
crts
Senior Member
 
Registered: Jan 2010
Posts: 1,606

Rep: Reputation: 448Reputation: 448Reputation: 448Reputation: 448Reputation: 448
[QUOTE=emmanouelaaaaa;4497541
That's what the q was for, it wasn't the variable posq but the variable pos with the q afterwards. Again, sorry if its too basic but I need to make this work soon so don't have time to immerse myself in the unix tutorials.[/QUOTE]
I thought that the 'posq' was a typo. Since you want to quit after the line the sed needs to be changed:
Code:
sed "${3} q;d" "${2}.conservation"
You can run your script with 'set -x' to see what is happening:
Code:
#!/bin/bash

set -x

<rest of the script>
 
Old 10-14-2011, 06:33 AM   #7
emmanouelaaaaa
LQ Newbie
 
Registered: Oct 2011
Posts: 3

Original Poster
Rep: Reputation: Disabled
Thanks, it finally worked! Putting the {} in the right places does miracles! Thank you both!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Insert line using sed or awk at line using line number as variable sunilsagar Programming 11 02-03-2012 11:48 AM
filter source line based on results line in log using awk and sed samanp Programming 5 04-06-2011 10:42 AM
awk command line: blank line record sep, new line field sep robertmarkbram Programming 4 02-21-2010 06:25 AM
how to replace line of file with another line using awk in shell script amit_pansuria Programming 3 03-29-2009 10:43 AM
Problem changing line with sed. Should i use AWK? RattleSn@ke Linux - General 7 11-12-2007 02:03 PM


All times are GMT -5. The time now is 02:47 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration