LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (http://www.linuxquestions.org/questions/programming-9/)
-   -   Loop through records in a file (http://www.linuxquestions.org/questions/programming-9/loop-through-records-in-a-file-876616/)

akelly 04-23-2011 01:54 AM

Loop through records in a file
 
I want to loop through the records in the below file (homedir.temp)
/home/user1
/home/user2
/home/user3

I want to do the following activities with each record
1. du -s - to get the total usage for that directory (my variable name is SIZE)
2. divide SIZE by du -c for /home to get the percentage of usage. (my variable name is PER)
3. write the directory, SIZE, PER to a file

PROBLEM
I am using the below for loop:
for record in homedir.temp
do
the mentioned activities
done

The above is not looping through the records. It does the first record perfectly and exits the loop.

Can someone please help me? If I need to provide more information, please let me know.

Thanks.

EricTRA 04-23-2011 02:04 AM

Hello and Welcome to LinuxQuestions,

This feels very much like homework to me. You only provide a basic 'skeleton' of what you think is needed to be done. If you want assistance from us, then you need to realize that we will not do your homework for you (if it is homework) because you'll learn a lot more doing it yourself. Also the more information you provide the better help you get. Post what you've got (completely) and I'm sure we can get it to work. What's also important to know in case this is a school task, is that your teacher might be looking over your shoulder too.

Kind regards,

Eric

Telengard 04-23-2011 02:12 AM

Quote:

Originally Posted by EricTRA (Post 4333328)
Post what you've got (completely) and I'm sure we can get it to work.

+1 to everything EricTRA said, and please put your code inside code tags.

Your for loop looks wrong to me. This will not read a file. It will only have one iteration because you only provided a single item in the list.

Code:

for record in homedir.temp # only one item to iterate here?
do
# whatever
done

If you want your loop to read a file then you need to redirect its standard input stream.

Also it might help to actually have a read command in there somewhere :-/

akelly 04-23-2011 02:24 AM

This is not homework...I just need help...
 
I am sorry I am a little bit new at scripting, so I am stuck. If you could just bare the basic question and help me, I would appreciate it.

Below is exactly what I have written so far:

#LIST greps for the users with home directories and writes them to homedir.temp
#FINDGRAND gets the total disk usage of the home directory as a whole
#DIR holds the first record which is a directory in homedir.temp
#SIZE gets the size of that directory
#GTOTAL extracts the number from gtotal.temp because GTOTAL1 has a bunch of spaces then a period...it causes issues.
#PER1 divides SIZE by GTOTAL with a scale 2
#PER2 multiplies PER1 to make it a whole number

awk -F":" '{print $6}' /etc/passwd > passwdhome.temp
LIST=$(grep -i "home" passwdhome.temp)
echo "$LIST" > homedir.temp
FINDGRAND=$(du -s /home)
echo "${FINDGRAND}" > gtotal.temp
for record in homedir.temp
do
DIR=$(awk -F" " '{print $1} homedir.temp)
SIZE=$(du -s $DIR)
echo "${SIZE}" > size.temp
GTOTAL=$(awk -F" " '{print $1}' gtotal.temp)
PER1=$(echo "scale=2; $SIZE/$GTOTAL" | bc)
PER2=$(echo "scale=0; $PER1*100" | bc)
echo -e "${DIR}\t${TOTAL}\t${PER2}" >> diskhogs1.file
done


If I need to be referred to somewhere else to ask a Linux Question, please tell me where to go.

Thanks.

EricTRA 04-23-2011 02:36 AM

Hello,

You came to the right place for answers in regards to Linux but you have to admit you made us doubt by only providing a small part of what you've got. Like pointed out by Telengard your for loop is incorrect if homedir.tmp contains the directories you want to process on. You need to iterate through that list and in order to do so you need to provide it to the for loop as input. The way you called it you only get one iteration. Try this:
Code:

cat homedir.tmp | while read record;
do
your thing;
done

or
Code:

for record in $(cat homedir.tmp);
do
your thing;
done

and see where that gets you.

Also as indicated by Telengard please use code tags to post your code. It makes it more readable. To use them click on the # in the icon bar of the editor when you're posting.

Kind regards,

Eric

akelly 04-23-2011 02:57 AM

One more question on the variable SIZE
 
Thank you guys for such a speedy reply!!!

I'm sorry to bother you guys again...One more tiny question..

In regards to my variable SIZE, the way it is now its not working. I am using the while loop from above. Why can't I say SIZE=$(du -s record)? If I say SIZE=$(du -s homedir.temp), it does nothing. I want SIZE to get the du -s for each record in homedir.temp which contains a directory on each line/record.

I appreciate you guys help more than you know!!!

Thanks,

grail 04-23-2011 02:57 AM

Quote:

If I need to be referred to somewhere else to ask a Linux Question, please tell me where to go.
No, you are in the right place. It is just that the way you phrased the question it looked like it came from a book or a teacher
and hence part of the LQ rules are that homework is not to be done in this way.

So firstly, you may have missed it but please place code in [code][/code] tags so we can see what is going on better and the indentation will not be lost.

Second, Telengard did provide valuable information regarding your for loop. Effectively the loop only passes values to your variable based on the information delivered.
In your example you present only a single item:
Code:

for record in homedir.temp
Here you have presented the string 'homedir.temp' and seeing that you never use the variable record within the loop itself, everything in the loop is performed once.

That being said, maybe you would like to identify more of what it is you want to do rather than how you are trying to do it?

Let me step through your code and make some observations:
Code:

awk -F":" '{print $6}' /etc/passwd > passwdhome.temp
LIST=$(grep -i "home" passwdhome.temp)
echo "$LIST" > homedir.temp

So the use of awk and grep here is not really required seeing as awk has both skills already.
Code:

FINDGRAND=$(du -s /home)
echo "${FINDGRAND}" > gtotal.temp

I left this until here but it also goes for the first piece of code as well, as you do not use the variable anywhere else you do not need to assign
it and then echo the variable. Just simply echo what you have between $() and redirect to the file:
Code:

echo "$(du -s /home)" > gtotal.temp
Code:

for record in homedir.temp
do

So this was my point above in that record is never used below and you are passing the string and not the contents of homedir.temp (How would you normally display the contents
of a file at the command prompt??)
Code:

    DIR=$(awk -F" " '{print $1} homedir.temp)
I think you need to understand what is the contents of homedir.temp? Based on earlier code it is not space separated information.
Code:

    SIZE=$(du -s $DIR)
    echo "${SIZE}" > size.temp
    GTOTAL=$(awk -F" " '{print $1}' gtotal.temp)
    PER1=$(echo "scale=2; $SIZE/$GTOTAL" | bc)
    PER2=$(echo "scale=0; $PER1*100" | bc)
    echo -e "${DIR}\t${TOTAL}\t${PER2}" >> diskhogs1.file
done

My question to you here would be, why use bc when you are using awk everywhere else and it can do decimal calculations??

I hope you find some of this useful. I would add that you could do it all as an awk script if you really wanted to :)

grail 04-23-2011 03:00 AM

Quote:

Why can't I say SIZE=$(du -s record)?
record is a variable and needs to be told to bash that it is. So how do you recognise a variable?

akelly 04-23-2011 03:12 AM

A reply
 
Honestly, I don't know awk that well yet. I am making a career move to a beginner Sys Admin, and I am just starting my Linux Career. I am practicing writing a few real world scripts, so I won't look like an idiot. I am googling to see what you guys do and preparing myself with some challenges on my own. I am just going with what i know.

I am just trying anything to make this work. I am sure there is a much better and efficient way of going about this, but my experience is limited. I am doing the best I can with what I know.

I know that homedir.temp has the following contents:
/home/user1
/home/user2
/home/user3

I was just trying anything to get the output of du -s for each line. I guess it made me look like an idiot...sorry.

Please be understanding... This is my obsession and my new career, and I want to learn and practice as much as I can.

Thanks,

grail 04-23-2011 04:23 AM

Quote:

I was just trying anything to get the output of du -s for each line. I guess it made me look like an idiot...sorry.
No need to be sorry and no one here who has started at the ground level would ever call you an idiot :)

My normal rule of thumb is to perform the task on the command line and then move it into a script.
By looking at your code this would seem to be where you are coming from.

Three sites I would recommend, based on what you have so far are:

http://www.gnu.org/software/gawk/man...ode/index.html
http://tldp.org/LDP/abs/html/ (Do not let the word advanced throw you)
http://mywiki.wooledge.org/TitleIndex (Once you have a reasonable understanding of what the previous site has to teach you, this is the site that
you will live by as far as the correct way to do a lot of things in bash)

Let us make things a little simpler and do away with everything except the for loop.
We will assume you have manually created the file homedir.temp with the data contained in post #9

My question from post #7 stands, how would you display the contents of this file if you were on the command line?

The next thing to understand is that a for loop simply takes input, via 'in', and places it into the variable.
If we again look at your original loop start:
Code:

for record in homedir.temp
As there are no commands being used, 'in' will place the string "homedir.temp" into record. As there are no other strings on the line
it will loop through once and perform your commands and then exit the loop.

However, if you redirect or output some information, 'in' will now grab that data and using word splitting based on the IFS variable (might want to look this one up)
it will assign each block of data into record and therefore loop as many times as pieces of data are assigned.

So your test here is, using the file you created are you able to loop through the lines and echo each of them one at a time,
ie. the inside of the loop should be only:
Code:

echo "$record"
See how you go with that and it should lead you toward what you want to do.

akelly 04-24-2011 01:14 AM

Yahoo
 
I got it to work perfectly!!

You guys are geniuses! I learned a lot from you guys! Thank you so much for your help!

One day, I hope to be in your shoes knowing what you guys know.

I am going to figure out the last thing for this script for me to be satisfied.
I want to make sure the user is in the right directory
I want to make sure they are in root
If not, I want the script to exit.

Thats my next challenge, but I have a few ideas before I scream for help. I promise I want to yell for help until I am frustrated and have exhausted my brain...

grail 04-24-2011 01:45 AM

Glad you got there :) Mark as SOLVED if your happy and raise a new question when you get stuck.

Maybe show your solution to as it may help others on the learning process

EricTRA 04-24-2011 01:54 AM

Hi,

Great help given by grail! Glad you've got your solution. Have fun with Linux.

Kind regards,

Eric

akelly 04-24-2011 02:29 AM

Yahoo yahoo
 
I GOT MY TWO TASKS DONE THAT I MENTIONED EARLIER. The script is commented. I am happy camper.

Thank you guys for helping and teaching me some things.

I wish I had another challenging script to write, so I can learn some new things.

I am sure you will probably be seeing more of me.

EricTRA 04-24-2011 02:31 AM

Hello,

Glad to hear so! Challenges are easily found, so look what you can automate and write a script for it. Trying is the best practice you can have and the best teacher. Don't forget to mark your thread as solved.

Kind regards,

Eric


All times are GMT -5. The time now is 08:38 PM.