[BASH] How to read multiple lines from a text file?
ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
[BASH] How to read multiple lines from a text file?
For example, I have a text file with data which lists numerical values from two separate individuals
Code:
Person A
100
200
300
400
500
600
700
800
900
1000
1100
1200
Person B
1200
1100
1000
900
800
700
600
500
400
300
200
100
How would I go about reading the values for each Person, then being able to perform mathematical equations for each Person (finding the sum for example) ?
Are the number of lines for each person fixed? Are there any other lines in the file or other variations possible compared to the above?
One possibility would be to simply load the whole file into an array. Bash v.4 even has a new mapfile (syn: readarray) built-in command for doing just that. Or for older versions you can use cat:
Code:
mapfile arrayname <file
or
arrayname=( $(cat file) )
And call each array variable with "${arrayname[n]}"; n being the number of the line (starting with 0).
Simple integer arithmetic can then be done with $((...)).
Code:
echo "$(( arrayname[1] + arrayname[20] ))"
For more advanced stuff, such as working with decimal amounts, you'll have to use bc, awk, or a similar external tool.
The only difficulty is that you need to know which index numbers to use, which is why I asked if the input is predictable.
Last edited by David the H.; 03-11-2011 at 06:29 AM.
Reason: removed incorrect quotes
The text file is fixed. Meaning there will be just 2 persons A and B, and they will each have 12 values under them.
I noticed that you are Location is listed in Japan .. and Im hoping that you and any of your family and friends are okay in this time.. my Prayers going out to you and everybody in this tough time..
IFS=$'\n'
p1array=( $(grep -A 12 "Person A" file) )
p2array=( $(grep -A 12 "Person B" file) )
unset IFS
This will give you a separate array for each person, with element 0 being the title line and 1-12 being the numbered entries. The IFS needs to be set to newline first though, in order to keep it from breaking at spaces; something I failed to note that in my first post. Or you can just shift the index numbers to account for the extra entry instead if you want.
Edit: For bash v4's mapfile, you can do this:
Code:
mapfile -t p1array < <(grep -A 12 "Person A" file)
mapfile -t p2array < <(grep -A 12 "Person B" file)
No need to futz around with IFS when using mapfile, but it seems you need to use -t to strip the newlines from the end of each line.
And thank you for your concern. But I'm half the country away from the quake and wasn't directly affected at all. The strength was only about a 4 here. In fact, I was on my bike at the time heading for a job and didn't notice a thing. I did feel the large aftershock 30 minutes later, however, which set everything rattling again for some time.
I'm sitting at home now watching the aftermath on the news and those tsunami images are scary stuff.
Last edited by David the H.; 03-11-2011 at 07:07 AM.
Reason: As stated
I am a bit confused. Sorry. I am quite new to programming and Bash. Ive had around 2 days worth of experience.
I am fairly average with arrays, and I have not heard of the IFS command..
Also, I maybe did not mention clearly that both the PersonA and PersonB would be in one text file..
Sorry if I have set confusion upon you !
No worries.
IFS is the internal field separator environmental variable. It controls what characters bash views as "word" separators. It's set for space/tab/newline by default. But when you want it to ignore spaces, you can set it to newline only and it will break up the text based on lines instead.
And yes, I realized they're in the same file. That's why I used grep to grab each individual entry. the -A 12 option means that grep will output the matching string you give it, plus the 12 lines following it.
It would help if we could see the command that's generating the error.
I only know the most basic use of bc, but that's all you need here. Simply build the expression you want to evaluate using echo and pipe it through bc.
Code:
echo "${p1array[1]} + ${p2array[1]}" | bc
or using an alternate syntax:
bc < <( echo "${p1array[1]} + ${p2array[1]}" )
awk can be used in a similar way, but just feed it the values, not the operator, separated by spaces. Then use $1, $2, etc to refer to them.
Usually awk needs some kind of input from a file or or stdin like this to work on, but another way to do it is to use a BEGIN command block to have it print arbitrary strings without input. The awk language has it's own separate variable system though, so to use it this way you have to first transfer your bash variables into the command using the -v option.
Of course, as grail demonstrated above, awk is a powerful scripting language of its own, and it can do everything you want without needing bash at all. When you have time, I recommend the awk tutorial here.
Finally, you can redirect the output of any of the above directly to a file as before, or you can capture the output of any command into a variable using $(..).
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.