LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (http://www.linuxquestions.org/questions/programming-9/)
-   -   [BASH] How to read multiple lines from a text file? (http://www.linuxquestions.org/questions/programming-9/%5Bbash%5D-how-to-read-multiple-lines-from-a-text-file-867901/)

gtirsmiley 03-11-2011 06:02 AM

[BASH] How to read multiple lines from a text file?
 
For example, I have a text file with data which lists numerical values from two separate individuals

Code:

Person A
100
200
300
400
500
600
700
800
900
1000
1100
1200

Person B
1200
1100
1000
900
800
700
600
500
400
300
200
100

How would I go about reading the values for each Person, then being able to perform mathematical equations for each Person (finding the sum for example) ?

Thanks.

David the H. 03-11-2011 06:27 AM

Are the number of lines for each person fixed? Are there any other lines in the file or other variations possible compared to the above?

One possibility would be to simply load the whole file into an array. Bash v.4 even has a new mapfile (syn: readarray) built-in command for doing just that. Or for older versions you can use cat:
Code:

mapfile arrayname <file
or
arrayname=( $(cat file) )

And call each array variable with "${arrayname[n]}"; n being the number of the line (starting with 0).

Simple integer arithmetic can then be done with $((...)).
Code:

echo "$(( arrayname[1] + arrayname[20] ))"
For more advanced stuff, such as working with decimal amounts, you'll have to use bc, awk, or a similar external tool.

The only difficulty is that you need to know which index numbers to use, which is why I asked if the input is predictable.

gtirsmiley 03-11-2011 06:33 AM

The text file is fixed. Meaning there will be just 2 persons A and B, and they will each have 12 values under them.

I noticed that you are Location is listed in Japan .. and Im hoping that you and any of your family and friends are okay in this time.. my Prayers going out to you and everybody in this tough time..

XavierP 03-11-2011 06:36 AM

Moved: This thread is more suitable in Programming and has been moved accordingly to help your thread/question get the exposure it deserves.

David the H. 03-11-2011 06:57 AM

I'd suggest this then:
Code:

IFS=$'\n'
p1array=( $(grep -A 12 "Person A" file) )
p2array=( $(grep -A 12 "Person B" file) )
unset IFS

This will give you a separate array for each person, with element 0 being the title line and 1-12 being the numbered entries. The IFS needs to be set to newline first though, in order to keep it from breaking at spaces; something I failed to note that in my first post. Or you can just shift the index numbers to account for the extra entry instead if you want.

Edit: For bash v4's mapfile, you can do this:
Code:

mapfile -t p1array < <(grep -A 12 "Person A" file)
mapfile -t p2array < <(grep -A 12 "Person B" file)

No need to futz around with IFS when using mapfile, but it seems you need to use -t to strip the newlines from the end of each line. :)

And thank you for your concern. But I'm half the country away from the quake and wasn't directly affected at all. The strength was only about a 4 here. In fact, I was on my bike at the time heading for a job and didn't notice a thing. I did feel the large aftershock 30 minutes later, however, which set everything rattling again for some time.

I'm sitting at home now watching the aftermath on the news and those tsunami images are scary stuff.

gtirsmiley 03-11-2011 07:08 AM

I am a bit confused. Sorry. I am quite new to programming and Bash. Ive had around 2 days worth of experience.

I am fairly average with arrays, and I have not heard of the IFS command..

Also, I maybe did not mention clearly that both the PersonA and PersonB would be in one text file..

Sorry if I have set confusion upon you !

grail 03-11-2011 07:08 AM

So not sure what else you might need to do, but as a sum example:
Code:

awk '/Person/{a=$0}/[0-9]/{sum[a]+=$0}END{for(x in sum)print "Total for",x,"is",sum[x]}' file

gtirsmiley 03-11-2011 07:26 AM

Sorry, I just had noticed you updated your post to include the mapfile and its really great!

I just have one more question in regards to this.

How would I go about creating a NEW text file, using the above p1array and p2array, which would show the total of each line..

For example the first value for PersonA is 100, while the first value of PersonB is 1200 .. the sum would be 1300

How can I get it to show that Line 1 = 1300 into a new file..

David the H. 03-11-2011 07:27 AM

Quote:

Originally Posted by gtirsmiley (Post 4286858)
I am a bit confused. Sorry. I am quite new to programming and Bash. Ive had around 2 days worth of experience.

I am fairly average with arrays, and I have not heard of the IFS command..

Also, I maybe did not mention clearly that both the PersonA and PersonB would be in one text file..

Sorry if I have set confusion upon you !

No worries.

IFS is the internal field separator environmental variable. It controls what characters bash views as "word" separators. It's set for space/tab/newline by default. But when you want it to ignore spaces, you can set it to newline only and it will break up the text based on lines instead.

And yes, I realized they're in the same file. That's why I used grep to grab each individual entry. the -A 12 option means that grep will output the matching string you give it, plus the 12 lines following it.

David the H. 03-11-2011 07:33 AM

Quote:

Originally Posted by gtirsmiley (Post 4286880)
How can I get it to show that Line 1 = 1300 into a new file..

Code:

echo "The sum of the first lines is $(( p1array[1] + p2array[1] ))" >> newfile
Use >> to append lines to an existing file, or > to overwrite the file or create a new one.

Here are a few useful bash scripting references:
http://www.linuxcommand.org/index.php
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/index.html
http://www.gnu.org/software/bash/manual/bashref.html

gtirsmiley 03-11-2011 07:42 AM

Quote:

Originally Posted by David the H. (Post 4286889)
Code:

echo "The sum of the first lines is $(( p1array[1] + p2array[1] ))" >> newfile
Use >> to append lines to an existing file, or > to overwrite the file or create a new one.

Here are a few useful bash scripting references:
http://www.linuxcommand.org/index.php
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/index.html
http://www.gnu.org/software/bash/manual/bashref.html

Thank you so much. I've actually read 2 of those sites already and its helping me alot.

But you have helped me much more!

I do have one more question but I will try to solve that on my own before bugging you again! :hattip:

grail 03-11-2011 08:03 AM

And just for the alternative:
Code:

awk -ventry=1 '/Person/{a++;i=0}/[0-9]/{_[a,++i]=$0}END{print "The sum for entries",entry,"is",_[1,entry]+_[2,entry]}' file >> newfile
Change the value of 'entry' each time you want a different line.

gtirsmiley 03-11-2011 09:09 AM

I've tried some googling and reading tutorials, but can't seem to answer my problem..

I understand to use decimals in BASH is not possible .. and we have to use bc, awk or dc ..

I've tried to implement it but without success, I keep getting :

syntax error: invalid arithmetic operator (error token is ".50")

I won't have any decimals, but it would be nice to add to my knowledge incase in the future I would have to do such a task.

David the H. 03-11-2011 10:18 AM

It would help if we could see the command that's generating the error.


I only know the most basic use of bc, but that's all you need here. Simply build the expression you want to evaluate using echo and pipe it through bc.
Code:

echo "${p1array[1]} + ${p2array[1]}" | bc

or using an alternate syntax:

bc < <( echo "${p1array[1]} + ${p2array[1]}" )

awk can be used in a similar way, but just feed it the values, not the operator, separated by spaces. Then use $1, $2, etc to refer to them.
Code:

echo "${p1array[1]} ${p2array[1]}" | awk '{print $1 + $2}'
Usually awk needs some kind of input from a file or or stdin like this to work on, but another way to do it is to use a BEGIN command block to have it print arbitrary strings without input. The awk language has it's own separate variable system though, so to use it this way you have to first transfer your bash variables into the command using the -v option.
Code:

awk -v p1="${p1array[1]}" -v p2="${p2array[1]}" 'BEGIN{ print p1 + p2 }'
Of course, as grail demonstrated above, awk is a powerful scripting language of its own, and it can do everything you want without needing bash at all. When you have time, I recommend the awk tutorial here.

Finally, you can redirect the output of any of the above directly to a file as before, or you can capture the output of any command into a variable using $(..).
Code:

result="$( echo "${p1array[1]} + ${p2array[1]}" | bc )"

echo "$result" > newfile



All times are GMT -5. The time now is 01:45 AM.