LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 04-23-2011, 12:54 AM   #1
akelly
LQ Newbie
 
Registered: Aug 2010
Location: Dallas
Distribution: Fedora 14 x86-64 GNOME
Posts: 6

Rep: Reputation: 0
Loop through records in a file


I want to loop through the records in the below file (homedir.temp)
/home/user1
/home/user2
/home/user3

I want to do the following activities with each record
1. du -s - to get the total usage for that directory (my variable name is SIZE)
2. divide SIZE by du -c for /home to get the percentage of usage. (my variable name is PER)
3. write the directory, SIZE, PER to a file

PROBLEM
I am using the below for loop:
for record in homedir.temp
do
the mentioned activities
done

The above is not looping through the records. It does the first record perfectly and exits the loop.

Can someone please help me? If I need to provide more information, please let me know.

Thanks.

Last edited by akelly; 04-23-2011 at 12:57 AM. Reason: I wasn't finished. i hit submit too soon. Ooops!
 
Old 04-23-2011, 01:04 AM   #2
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297
Hello and Welcome to LinuxQuestions,

This feels very much like homework to me. You only provide a basic 'skeleton' of what you think is needed to be done. If you want assistance from us, then you need to realize that we will not do your homework for you (if it is homework) because you'll learn a lot more doing it yourself. Also the more information you provide the better help you get. Post what you've got (completely) and I'm sure we can get it to work. What's also important to know in case this is a school task, is that your teacher might be looking over your shoulder too.

Kind regards,

Eric
 
1 members found this post helpful.
Old 04-23-2011, 01:12 AM   #3
Telengard
Member
 
Registered: Apr 2007
Location: USA
Distribution: Kubuntu 8.04
Posts: 579
Blog Entries: 8

Rep: Reputation: 148Reputation: 148
Quote:
Originally Posted by EricTRA View Post
Post what you've got (completely) and I'm sure we can get it to work.
+1 to everything EricTRA said, and please put your code inside code tags.

Your for loop looks wrong to me. This will not read a file. It will only have one iteration because you only provided a single item in the list.

Code:
for record in homedir.temp # only one item to iterate here?
do
# whatever
done
If you want your loop to read a file then you need to redirect its standard input stream.

Also it might help to actually have a read command in there somewhere :-/

Last edited by Telengard; 04-23-2011 at 01:15 AM.
 
1 members found this post helpful.
Old 04-23-2011, 01:24 AM   #4
akelly
LQ Newbie
 
Registered: Aug 2010
Location: Dallas
Distribution: Fedora 14 x86-64 GNOME
Posts: 6

Original Poster
Rep: Reputation: 0
This is not homework...I just need help...

I am sorry I am a little bit new at scripting, so I am stuck. If you could just bare the basic question and help me, I would appreciate it.

Below is exactly what I have written so far:

#LIST greps for the users with home directories and writes them to homedir.temp
#FINDGRAND gets the total disk usage of the home directory as a whole
#DIR holds the first record which is a directory in homedir.temp
#SIZE gets the size of that directory
#GTOTAL extracts the number from gtotal.temp because GTOTAL1 has a bunch of spaces then a period...it causes issues.
#PER1 divides SIZE by GTOTAL with a scale 2
#PER2 multiplies PER1 to make it a whole number

awk -F":" '{print $6}' /etc/passwd > passwdhome.temp
LIST=$(grep -i "home" passwdhome.temp)
echo "$LIST" > homedir.temp
FINDGRAND=$(du -s /home)
echo "${FINDGRAND}" > gtotal.temp
for record in homedir.temp
do
DIR=$(awk -F" " '{print $1} homedir.temp)
SIZE=$(du -s $DIR)
echo "${SIZE}" > size.temp
GTOTAL=$(awk -F" " '{print $1}' gtotal.temp)
PER1=$(echo "scale=2; $SIZE/$GTOTAL" | bc)
PER2=$(echo "scale=0; $PER1*100" | bc)
echo -e "${DIR}\t${TOTAL}\t${PER2}" >> diskhogs1.file
done


If I need to be referred to somewhere else to ask a Linux Question, please tell me where to go.

Thanks.
 
Old 04-23-2011, 01:36 AM   #5
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297
Hello,

You came to the right place for answers in regards to Linux but you have to admit you made us doubt by only providing a small part of what you've got. Like pointed out by Telengard your for loop is incorrect if homedir.tmp contains the directories you want to process on. You need to iterate through that list and in order to do so you need to provide it to the for loop as input. The way you called it you only get one iteration. Try this:
Code:
cat homedir.tmp | while read record; 
do
your thing;
done
or
Code:
for record in $(cat homedir.tmp); 
do
your thing;
done
and see where that gets you.

Also as indicated by Telengard please use code tags to post your code. It makes it more readable. To use them click on the # in the icon bar of the editor when you're posting.

Kind regards,

Eric
 
2 members found this post helpful.
Old 04-23-2011, 01:57 AM   #6
akelly
LQ Newbie
 
Registered: Aug 2010
Location: Dallas
Distribution: Fedora 14 x86-64 GNOME
Posts: 6

Original Poster
Rep: Reputation: 0
One more question on the variable SIZE

Thank you guys for such a speedy reply!!!

I'm sorry to bother you guys again...One more tiny question..

In regards to my variable SIZE, the way it is now its not working. I am using the while loop from above. Why can't I say SIZE=$(du -s record)? If I say SIZE=$(du -s homedir.temp), it does nothing. I want SIZE to get the du -s for each record in homedir.temp which contains a directory on each line/record.

I appreciate you guys help more than you know!!!

Thanks,
 
Old 04-23-2011, 01:57 AM   #7
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,007

Rep: Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192
Quote:
If I need to be referred to somewhere else to ask a Linux Question, please tell me where to go.
No, you are in the right place. It is just that the way you phrased the question it looked like it came from a book or a teacher
and hence part of the LQ rules are that homework is not to be done in this way.

So firstly, you may have missed it but please place code in [code][/code] tags so we can see what is going on better and the indentation will not be lost.

Second, Telengard did provide valuable information regarding your for loop. Effectively the loop only passes values to your variable based on the information delivered.
In your example you present only a single item:
Code:
for record in homedir.temp
Here you have presented the string 'homedir.temp' and seeing that you never use the variable record within the loop itself, everything in the loop is performed once.

That being said, maybe you would like to identify more of what it is you want to do rather than how you are trying to do it?

Let me step through your code and make some observations:
Code:
awk -F":" '{print $6}' /etc/passwd > passwdhome.temp
LIST=$(grep -i "home" passwdhome.temp)
echo "$LIST" > homedir.temp
So the use of awk and grep here is not really required seeing as awk has both skills already.
Code:
FINDGRAND=$(du -s /home)
echo "${FINDGRAND}" > gtotal.temp
I left this until here but it also goes for the first piece of code as well, as you do not use the variable anywhere else you do not need to assign
it and then echo the variable. Just simply echo what you have between $() and redirect to the file:
Code:
echo "$(du -s /home)" > gtotal.temp
Code:
for record in homedir.temp
do
So this was my point above in that record is never used below and you are passing the string and not the contents of homedir.temp (How would you normally display the contents
of a file at the command prompt??)
Code:
    DIR=$(awk -F" " '{print $1} homedir.temp)
I think you need to understand what is the contents of homedir.temp? Based on earlier code it is not space separated information.
Code:
    SIZE=$(du -s $DIR)
    echo "${SIZE}" > size.temp
    GTOTAL=$(awk -F" " '{print $1}' gtotal.temp)
    PER1=$(echo "scale=2; $SIZE/$GTOTAL" | bc)
    PER2=$(echo "scale=0; $PER1*100" | bc)
    echo -e "${DIR}\t${TOTAL}\t${PER2}" >> diskhogs1.file
done
My question to you here would be, why use bc when you are using awk everywhere else and it can do decimal calculations??

I hope you find some of this useful. I would add that you could do it all as an awk script if you really wanted to
 
2 members found this post helpful.
Old 04-23-2011, 02:00 AM   #8
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,007

Rep: Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192
Quote:
Why can't I say SIZE=$(du -s record)?
record is a variable and needs to be told to bash that it is. So how do you recognise a variable?
 
Old 04-23-2011, 02:12 AM   #9
akelly
LQ Newbie
 
Registered: Aug 2010
Location: Dallas
Distribution: Fedora 14 x86-64 GNOME
Posts: 6

Original Poster
Rep: Reputation: 0
A reply

Honestly, I don't know awk that well yet. I am making a career move to a beginner Sys Admin, and I am just starting my Linux Career. I am practicing writing a few real world scripts, so I won't look like an idiot. I am googling to see what you guys do and preparing myself with some challenges on my own. I am just going with what i know.

I am just trying anything to make this work. I am sure there is a much better and efficient way of going about this, but my experience is limited. I am doing the best I can with what I know.

I know that homedir.temp has the following contents:
/home/user1
/home/user2
/home/user3

I was just trying anything to get the output of du -s for each line. I guess it made me look like an idiot...sorry.

Please be understanding... This is my obsession and my new career, and I want to learn and practice as much as I can.

Thanks,
 
Old 04-23-2011, 03:23 AM   #10
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,007

Rep: Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192
Quote:
I was just trying anything to get the output of du -s for each line. I guess it made me look like an idiot...sorry.
No need to be sorry and no one here who has started at the ground level would ever call you an idiot

My normal rule of thumb is to perform the task on the command line and then move it into a script.
By looking at your code this would seem to be where you are coming from.

Three sites I would recommend, based on what you have so far are:

http://www.gnu.org/software/gawk/man...ode/index.html
http://tldp.org/LDP/abs/html/ (Do not let the word advanced throw you)
http://mywiki.wooledge.org/TitleIndex (Once you have a reasonable understanding of what the previous site has to teach you, this is the site that
you will live by as far as the correct way to do a lot of things in bash)

Let us make things a little simpler and do away with everything except the for loop.
We will assume you have manually created the file homedir.temp with the data contained in post #9

My question from post #7 stands, how would you display the contents of this file if you were on the command line?

The next thing to understand is that a for loop simply takes input, via 'in', and places it into the variable.
If we again look at your original loop start:
Code:
for record in homedir.temp
As there are no commands being used, 'in' will place the string "homedir.temp" into record. As there are no other strings on the line
it will loop through once and perform your commands and then exit the loop.

However, if you redirect or output some information, 'in' will now grab that data and using word splitting based on the IFS variable (might want to look this one up)
it will assign each block of data into record and therefore loop as many times as pieces of data are assigned.

So your test here is, using the file you created are you able to loop through the lines and echo each of them one at a time,
ie. the inside of the loop should be only:
Code:
echo "$record"
See how you go with that and it should lead you toward what you want to do.
 
1 members found this post helpful.
Old 04-24-2011, 12:14 AM   #11
akelly
LQ Newbie
 
Registered: Aug 2010
Location: Dallas
Distribution: Fedora 14 x86-64 GNOME
Posts: 6

Original Poster
Rep: Reputation: 0
Yahoo

I got it to work perfectly!!

You guys are geniuses! I learned a lot from you guys! Thank you so much for your help!

One day, I hope to be in your shoes knowing what you guys know.

I am going to figure out the last thing for this script for me to be satisfied.
I want to make sure the user is in the right directory
I want to make sure they are in root
If not, I want the script to exit.

Thats my next challenge, but I have a few ideas before I scream for help. I promise I want to yell for help until I am frustrated and have exhausted my brain...
 
Old 04-24-2011, 12:45 AM   #12
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 10,007

Rep: Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192Reputation: 3192
Glad you got there Mark as SOLVED if your happy and raise a new question when you get stuck.

Maybe show your solution to as it may help others on the learning process
 
Old 04-24-2011, 12:54 AM   #13
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297
Hi,

Great help given by grail! Glad you've got your solution. Have fun with Linux.

Kind regards,

Eric
 
Old 04-24-2011, 01:29 AM   #14
akelly
LQ Newbie
 
Registered: Aug 2010
Location: Dallas
Distribution: Fedora 14 x86-64 GNOME
Posts: 6

Original Poster
Rep: Reputation: 0
Yahoo yahoo

I GOT MY TWO TASKS DONE THAT I MENTIONED EARLIER. The script is commented. I am happy camper.

Thank you guys for helping and teaching me some things.

I wish I had another challenging script to write, so I can learn some new things.

I am sure you will probably be seeing more of me.
 
Old 04-24-2011, 01:31 AM   #15
EricTRA
LQ Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297Reputation: 1297
Hello,

Glad to hear so! Challenges are easily found, so look what you can automate and write a script for it. Trying is the best practice you can have and the best teacher. Don't forget to mark your thread as solved.

Kind regards,

Eric
 
  


Reply

Tags
loop, shell script



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
how to loop over text file lines within bash script for loop? johnpaulodonnell Linux - Newbie 9 07-28-2015 03:49 PM
[SOLVED] MX Records / A Records / CNAME Records - Advice Please fusion1275 Linux - Newbie 15 01-18-2011 04:06 AM
for loop or while loop to read the fields of a file.. visitnag Linux - Newbie 10 09-02-2010 08:47 PM
awk: swapping fields and records and for loop sebelk Programming 5 05-10-2010 07:41 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 10:51 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration