LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (http://www.linuxquestions.org/questions/programming-9/)
-   -   Script to read line by line from a file (http://www.linuxquestions.org/questions/programming-9/script-to-read-line-by-line-from-a-file-408359/)

kushalkoolwal 01-26-2006 04:46 PM

Script to read line by line from a file
 
Hi guys,

I have a file called test.txt which I would like to read line by line in a while loop and store the contents of every line in a variable, do some processing and again loop back to the next line until I reach at the end of the file(test.txt) through a shell script.

Basically, here is what I am looking for:

Read line of the file
store it in a temp variable
do some operation/testing
loop till the end of the file.





Thanks

gilead 01-26-2006 05:01 PM

That's not the most efficient way to process a text file. Can you post what you want to do with the text in the file? There are plenty of tools which will operate on all of the lines in a file rather than being called line by line (e.g. sed, grep, awk, etc.).

kushalkoolwal 01-26-2006 05:15 PM

Quote:

Originally Posted by gilead
That's not the most efficient way to process a text file. Can you post what you want to do with the text in the file? There are plenty of tools which will operate on all of the lines in a file rather than being called line by line (e.g. sed, grep, awk, etc.).

I would like to run the command cat /dev/null on the text (for example #cat /dev/null > file1.txt), since the text on each line is a file on the disk. I would really appreciate if you tell me how to read line by line through a shell script. I have tired the following command but it does not work, that's why I am asking this question
Code:

find /home/user/test -type f -exec cat /dev/null > {} \;
I have spent two entire days to get this working but I was not able to.

Thanks in advance

perfect_circle 01-26-2006 05:33 PM

Quote:

Originally Posted by kushalkoolwal
Hi guys,

I have a file called test.txt which I would like to read line by line in a while loop and store the contents of every line in a variable, do some processing and again loop back to the next line until I reach at the end of the file(test.txt) through a shell script.

Basically, here is what I am looking for:

Read line of the file
store it in a temp variable
do some operation/testing
loop till the end of the file.





Thanks

You need to pipe the read command to read line by line. Something like this:
Code:

cat test.txt | while read a_line
do
    echo "Now I have read the line: $a_line"
done

In every execution of the while loop the $a_line variable will contain the next line.

Tinkster 01-26-2006 05:35 PM

Quote:

I would like to run the command cat /dev/null on the text (for example #cat /dev/null > file1.txt), since the text on each line is a file on the disk. I would really appreciate if you tell me how to read line by line through a shell script. I have tired the following command but it does not work, that's why I am asking this question
If I understand you correctly you want to empty all the
files whose name & path are stored in list.txt ...
To achieve that you could just:

cat list.txt|xargs -i cat /dev/null > {}


Cheers,
Tink

kushalkoolwal 01-26-2006 05:55 PM

Quote:

Originally Posted by Tinkster
If I understand you correctly you want to empty all the
files whose name & path are stored in list.txt ...
To achieve that you could just:

cat list.txt|xargs -i cat /dev/null > {}

Tink, You are exactly right, I want to achieve what you said, also I tried the command you suggested above but it did not work.

Any ideas why?

Kushal

Hko 01-26-2006 06:05 PM

I assume you are trying to create empty files with "cat /dev/null > somefile.txt". Do you realize that if a file already exists will be made empty (i.e. all content deleted) as well? If this is not what you want, or if it doesn't matter, just do:
Code:

xargs touch <test.txt
The code above will not truncate already existing files (but it will update the date/time as if they were modified).

If you do want to truncate an existing file:
Code:

while read L; do echo -n >$L; done <test.txt
This will cause an error if an empty line exists in "test.txt", also if the last line is empty. To prevent that:
Code:

while read L; do test $L && echo -n >$L; done <test.txt

kushalkoolwal 01-26-2006 06:09 PM

Quote:

Originally Posted by perfect_circle
You need to pipe the read command to read line by line. Something like this:
Code:

cat test.txt | while read a_line
do
    echo "Now I have read the line: $a_line"
done

In every execution of the while loop the $a_line variable will contain the next line.


Hey Perfect Circle, thanks a lot. You made my day. You are really perfect. The suggestion you gave worked perfectly for me. Thanks a lot dude.

I wonder why
find /home/user/test -type f -exec cat /dev/null > {} \;
and
cat list.txt|xargs -i cat /dev/null > {}

do not work although they look ok to me..

Isn;t that strange?

Tinkster 01-26-2006 06:13 PM

Not without more input :}

What does the file look like internally, are the paths
to the files you want to empty absolute?


Cheers,
Tink

kushalkoolwal 01-26-2006 06:22 PM

Quote:

Originally Posted by Tinkster
Not without more input :}

What does the file look like internally, are the paths
to the files you want to empty absolute?


Cheers,
Tink

The file looks something like this:

Code:

./file1.txt
./file2.txt
./file3.doc
...and  so on


Hko 01-26-2006 06:28 PM

Quote:

Originally Posted by kushalkoolwal
I wonder why
find /home/user/test -type f -exec cat /dev/null > {} \;
and
cat list.txt|xargs -i cat /dev/null > {}

do not work although they look ok to me..

Isn;t that strange?

They are not OK. Here's why:
Code:

find /home/user/test -type f
This will will search for any file called "/home/user/test". There will be only one file found: /home/user/test.

So you could expect the the file containing the list of file ("list.txt") would be truncate to zero-length! But you were lucky this did not happen, because of another error:

When redirecting to {} , the -exec option does not replace {} with the files found by "find". So /dev/null is catted to a file called "{}". I bet you have a an empty file now in your directory called "{}".

Code:

xargs -i cat /dev/null > {}
The same problem here with redirecting to {}.

Also, "xargs" will put as many arguments from its standard input (stdin) as possible. So the command actually executed will be something like:
Code:

cat /dev/null > file1 file2 file3 file4
...which isn't what you were expecting.

Tinkster 01-26-2006 06:29 PM

Bummer - it's my bad, the redirect within xargs doesn't work; the
output of xargs is being redirected to {} rather than redirecting the
output to the substituted filename. Sorry to get your hopes up :/

Alternatively create a script
#!/bin/bash
cat /dev/null > $1

and feed the cat to that ... :}

cat list.txt|xargs -i empty {}
works ...



Cheers,
Tink

Hko 01-26-2006 06:35 PM

Quote:

Originally Posted by Tinkster
cat list.txt|xargs -i empty {}
works ...

I've never heard of the "empty" command. And it it's not available on my computer (Debian sarge). What distro do you use?

Tinkster 01-26-2006 06:36 PM

Quote:

Originally Posted by Hko
I've never heard of the "empty" command. And it it's not available on my computer (Debian sarge). What distro do you use?

Quote:

Alternatively create a script
#!/bin/bash
cat /dev/null > $1
That's empty ;)


Cheers,
Tink

Hko 01-26-2006 06:44 PM

Quote:

Originally Posted by Tinkster
That's empty ;)

Right.
I missed that part.
I should have read your post less quickly.


All times are GMT -5. The time now is 04:03 AM.