LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 03-11-2011, 07:02 AM   #1
gtirsmiley
LQ Newbie
 
Registered: Mar 2011
Posts: 7

Rep: Reputation: 0
[BASH] How to read multiple lines from a text file?


For example, I have a text file with data which lists numerical values from two separate individuals

Code:
Person A
100
200
300
400
500
600
700
800
900
1000
1100
1200

Person B
1200
1100
1000
900
800
700
600
500
400
300
200
100
How would I go about reading the values for each Person, then being able to perform mathematical equations for each Person (finding the sum for example) ?

Thanks.
 
Old 03-11-2011, 07:27 AM   #2
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
Are the number of lines for each person fixed? Are there any other lines in the file or other variations possible compared to the above?

One possibility would be to simply load the whole file into an array. Bash v.4 even has a new mapfile (syn: readarray) built-in command for doing just that. Or for older versions you can use cat:
Code:
mapfile arrayname <file
or
arrayname=( $(cat file) )
And call each array variable with "${arrayname[n]}"; n being the number of the line (starting with 0).

Simple integer arithmetic can then be done with $((...)).
Code:
echo "$(( arrayname[1] + arrayname[20] ))"
For more advanced stuff, such as working with decimal amounts, you'll have to use bc, awk, or a similar external tool.

The only difficulty is that you need to know which index numbers to use, which is why I asked if the input is predictable.

Last edited by David the H.; 03-11-2011 at 07:29 AM. Reason: removed incorrect quotes
 
2 members found this post helpful.
Old 03-11-2011, 07:33 AM   #3
gtirsmiley
LQ Newbie
 
Registered: Mar 2011
Posts: 7

Original Poster
Rep: Reputation: 0
The text file is fixed. Meaning there will be just 2 persons A and B, and they will each have 12 values under them.

I noticed that you are Location is listed in Japan .. and Im hoping that you and any of your family and friends are okay in this time.. my Prayers going out to you and everybody in this tough time..
 
Old 03-11-2011, 07:36 AM   #4
XavierP
Moderator
 
Registered: Nov 2002
Location: Kent, England
Distribution: Lubuntu
Posts: 19,176
Blog Entries: 4

Rep: Reputation: 430Reputation: 430Reputation: 430Reputation: 430Reputation: 430
Moved: This thread is more suitable in Programming and has been moved accordingly to help your thread/question get the exposure it deserves.
 
Old 03-11-2011, 07:57 AM   #5
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
I'd suggest this then:
Code:
IFS=$'\n'
p1array=( $(grep -A 12 "Person A" file) )
p2array=( $(grep -A 12 "Person B" file) )
unset IFS
This will give you a separate array for each person, with element 0 being the title line and 1-12 being the numbered entries. The IFS needs to be set to newline first though, in order to keep it from breaking at spaces; something I failed to note that in my first post. Or you can just shift the index numbers to account for the extra entry instead if you want.

Edit: For bash v4's mapfile, you can do this:
Code:
mapfile -t p1array < <(grep -A 12 "Person A" file)
mapfile -t p2array < <(grep -A 12 "Person B" file)
No need to futz around with IFS when using mapfile, but it seems you need to use -t to strip the newlines from the end of each line.

And thank you for your concern. But I'm half the country away from the quake and wasn't directly affected at all. The strength was only about a 4 here. In fact, I was on my bike at the time heading for a job and didn't notice a thing. I did feel the large aftershock 30 minutes later, however, which set everything rattling again for some time.

I'm sitting at home now watching the aftermath on the news and those tsunami images are scary stuff.

Last edited by David the H.; 03-11-2011 at 08:07 AM. Reason: As stated
 
1 members found this post helpful.
Old 03-11-2011, 08:08 AM   #6
gtirsmiley
LQ Newbie
 
Registered: Mar 2011
Posts: 7

Original Poster
Rep: Reputation: 0
I am a bit confused. Sorry. I am quite new to programming and Bash. Ive had around 2 days worth of experience.

I am fairly average with arrays, and I have not heard of the IFS command..

Also, I maybe did not mention clearly that both the PersonA and PersonB would be in one text file..

Sorry if I have set confusion upon you !
 
Old 03-11-2011, 08:08 AM   #7
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,686

Rep: Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987
So not sure what else you might need to do, but as a sum example:
Code:
awk '/Person/{a=$0}/[0-9]/{sum[a]+=$0}END{for(x in sum)print "Total for",x,"is",sum[x]}' file
 
Old 03-11-2011, 08:26 AM   #8
gtirsmiley
LQ Newbie
 
Registered: Mar 2011
Posts: 7

Original Poster
Rep: Reputation: 0
Sorry, I just had noticed you updated your post to include the mapfile and its really great!

I just have one more question in regards to this.

How would I go about creating a NEW text file, using the above p1array and p2array, which would show the total of each line..

For example the first value for PersonA is 100, while the first value of PersonB is 1200 .. the sum would be 1300

How can I get it to show that Line 1 = 1300 into a new file..
 
Old 03-11-2011, 08:27 AM   #9
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
Quote:
Originally Posted by gtirsmiley View Post
I am a bit confused. Sorry. I am quite new to programming and Bash. Ive had around 2 days worth of experience.

I am fairly average with arrays, and I have not heard of the IFS command..

Also, I maybe did not mention clearly that both the PersonA and PersonB would be in one text file..

Sorry if I have set confusion upon you !
No worries.

IFS is the internal field separator environmental variable. It controls what characters bash views as "word" separators. It's set for space/tab/newline by default. But when you want it to ignore spaces, you can set it to newline only and it will break up the text based on lines instead.

And yes, I realized they're in the same file. That's why I used grep to grab each individual entry. the -A 12 option means that grep will output the matching string you give it, plus the 12 lines following it.
 
1 members found this post helpful.
Old 03-11-2011, 08:33 AM   #10
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
Quote:
Originally Posted by gtirsmiley View Post
How can I get it to show that Line 1 = 1300 into a new file..
Code:
echo "The sum of the first lines is $(( p1array[1] + p2array[1] ))" >> newfile
Use >> to append lines to an existing file, or > to overwrite the file or create a new one.

Here are a few useful bash scripting references:
http://www.linuxcommand.org/index.php
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/index.html
http://www.gnu.org/software/bash/manual/bashref.html

Last edited by David the H.; 03-11-2011 at 08:35 AM. Reason: forgot the code tags + fixed typos
 
2 members found this post helpful.
Old 03-11-2011, 08:42 AM   #11
gtirsmiley
LQ Newbie
 
Registered: Mar 2011
Posts: 7

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by David the H. View Post
Code:
echo "The sum of the first lines is $(( p1array[1] + p2array[1] ))" >> newfile
Use >> to append lines to an existing file, or > to overwrite the file or create a new one.

Here are a few useful bash scripting references:
http://www.linuxcommand.org/index.php
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/index.html
http://www.gnu.org/software/bash/manual/bashref.html
Thank you so much. I've actually read 2 of those sites already and its helping me alot.

But you have helped me much more!

I do have one more question but I will try to solve that on my own before bugging you again!
 
Old 03-11-2011, 09:03 AM   #12
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,686

Rep: Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987
And just for the alternative:
Code:
awk -ventry=1 '/Person/{a++;i=0}/[0-9]/{_[a,++i]=$0}END{print "The sum for entries",entry,"is",_[1,entry]+_[2,entry]}' file >> newfile
Change the value of 'entry' each time you want a different line.
 
1 members found this post helpful.
Old 03-11-2011, 10:09 AM   #13
gtirsmiley
LQ Newbie
 
Registered: Mar 2011
Posts: 7

Original Poster
Rep: Reputation: 0
I've tried some googling and reading tutorials, but can't seem to answer my problem..

I understand to use decimals in BASH is not possible .. and we have to use bc, awk or dc ..

I've tried to implement it but without success, I keep getting :

syntax error: invalid arithmetic operator (error token is ".50")

I won't have any decimals, but it would be nice to add to my knowledge incase in the future I would have to do such a task.
 
Old 03-11-2011, 11:18 AM   #14
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950Reputation: 1950
It would help if we could see the command that's generating the error.


I only know the most basic use of bc, but that's all you need here. Simply build the expression you want to evaluate using echo and pipe it through bc.
Code:
echo "${p1array[1]} + ${p2array[1]}" | bc

or using an alternate syntax:

bc < <( echo "${p1array[1]} + ${p2array[1]}" )
awk can be used in a similar way, but just feed it the values, not the operator, separated by spaces. Then use $1, $2, etc to refer to them.
Code:
echo "${p1array[1]} ${p2array[1]}" | awk '{print $1 + $2}'
Usually awk needs some kind of input from a file or or stdin like this to work on, but another way to do it is to use a BEGIN command block to have it print arbitrary strings without input. The awk language has it's own separate variable system though, so to use it this way you have to first transfer your bash variables into the command using the -v option.
Code:
awk -v p1="${p1array[1]}" -v p2="${p2array[1]}" 'BEGIN{ print p1 + p2 }'
Of course, as grail demonstrated above, awk is a powerful scripting language of its own, and it can do everything you want without needing bash at all. When you have time, I recommend the awk tutorial here.

Finally, you can redirect the output of any of the above directly to a file as before, or you can capture the output of any command into a variable using $(..).
Code:
result="$( echo "${p1array[1]} + ${p2array[1]}" | bc )"

echo "$result" > newfile
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Extract multiple lines of data from a text file. shawnamiller Programming 8 04-30-2010 12:46 PM
[SOLVED] edit multiple lines of a text file into 1 line: schneidz Programming 2 04-09-2009 12:22 PM
bash: read multiple lines in a file 300zxkyle Programming 7 07-29-2007 05:38 AM
Read specific lines from a text file chobin Programming 8 06-14-2006 12:14 PM
bash script that can read lines of text palceksmuk Programming 1 12-25-2005 04:49 AM


All times are GMT -5. The time now is 01:16 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration