LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 01-26-2006, 04:46 PM   #1
kushalkoolwal
Senior Member
 
Registered: Feb 2004
Location: Middle of nowhere
Distribution: Debian Squeeze
Posts: 1,249

Rep: Reputation: 48
Script to read line by line from a file


Hi guys,

I have a file called test.txt which I would like to read line by line in a while loop and store the contents of every line in a variable, do some processing and again loop back to the next line until I reach at the end of the file(test.txt) through a shell script.

Basically, here is what I am looking for:

Read line of the file
store it in a temp variable
do some operation/testing
loop till the end of the file.





Thanks
 
Old 01-26-2006, 05:01 PM   #2
gilead
Senior Member
 
Registered: Dec 2005
Location: Brisbane, Australia
Distribution: Slackware64 14.0
Posts: 4,123

Rep: Reputation: 151Reputation: 151
That's not the most efficient way to process a text file. Can you post what you want to do with the text in the file? There are plenty of tools which will operate on all of the lines in a file rather than being called line by line (e.g. sed, grep, awk, etc.).
 
Old 01-26-2006, 05:15 PM   #3
kushalkoolwal
Senior Member
 
Registered: Feb 2004
Location: Middle of nowhere
Distribution: Debian Squeeze
Posts: 1,249

Original Poster
Rep: Reputation: 48
Quote:
Originally Posted by gilead
That's not the most efficient way to process a text file. Can you post what you want to do with the text in the file? There are plenty of tools which will operate on all of the lines in a file rather than being called line by line (e.g. sed, grep, awk, etc.).
I would like to run the command cat /dev/null on the text (for example #cat /dev/null > file1.txt), since the text on each line is a file on the disk. I would really appreciate if you tell me how to read line by line through a shell script. I have tired the following command but it does not work, that's why I am asking this question
Code:
find /home/user/test -type f -exec cat /dev/null > {} \;
I have spent two entire days to get this working but I was not able to.

Thanks in advance

Last edited by kushalkoolwal; 01-26-2006 at 05:17 PM.
 
Old 01-26-2006, 05:33 PM   #4
perfect_circle
Senior Member
 
Registered: Oct 2004
Location: Athens, Greece
Distribution: Slackware, arch
Posts: 1,783

Rep: Reputation: 52
Quote:
Originally Posted by kushalkoolwal
Hi guys,

I have a file called test.txt which I would like to read line by line in a while loop and store the contents of every line in a variable, do some processing and again loop back to the next line until I reach at the end of the file(test.txt) through a shell script.

Basically, here is what I am looking for:

Read line of the file
store it in a temp variable
do some operation/testing
loop till the end of the file.





Thanks
You need to pipe the read command to read line by line. Something like this:
Code:
cat test.txt | while read a_line
do 
    echo "Now I have read the line: $a_line" 
done
In every execution of the while loop the $a_line variable will contain the next line.

Last edited by perfect_circle; 01-26-2006 at 05:36 PM.
 
Old 01-26-2006, 05:35 PM   #5
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,973
Blog Entries: 11

Rep: Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879
Quote:
I would like to run the command cat /dev/null on the text (for example #cat /dev/null > file1.txt), since the text on each line is a file on the disk. I would really appreciate if you tell me how to read line by line through a shell script. I have tired the following command but it does not work, that's why I am asking this question
If I understand you correctly you want to empty all the
files whose name & path are stored in list.txt ...
To achieve that you could just:

cat list.txt|xargs -i cat /dev/null > {}


Cheers,
Tink
 
Old 01-26-2006, 05:55 PM   #6
kushalkoolwal
Senior Member
 
Registered: Feb 2004
Location: Middle of nowhere
Distribution: Debian Squeeze
Posts: 1,249

Original Poster
Rep: Reputation: 48
Quote:
Originally Posted by Tinkster
If I understand you correctly you want to empty all the
files whose name & path are stored in list.txt ...
To achieve that you could just:

cat list.txt|xargs -i cat /dev/null > {}
Tink, You are exactly right, I want to achieve what you said, also I tried the command you suggested above but it did not work.

Any ideas why?

Kushal
 
Old 01-26-2006, 06:05 PM   #7
Hko
Senior Member
 
Registered: Aug 2002
Location: Groningen, The Netherlands
Distribution: ubuntu
Posts: 2,530

Rep: Reputation: 108Reputation: 108
I assume you are trying to create empty files with "cat /dev/null > somefile.txt". Do you realize that if a file already exists will be made empty (i.e. all content deleted) as well? If this is not what you want, or if it doesn't matter, just do:
Code:
xargs touch <test.txt
The code above will not truncate already existing files (but it will update the date/time as if they were modified).

If you do want to truncate an existing file:
Code:
while read L; do echo -n >$L; done <test.txt
This will cause an error if an empty line exists in "test.txt", also if the last line is empty. To prevent that:
Code:
while read L; do test $L && echo -n >$L; done <test.txt

Last edited by Hko; 01-26-2006 at 06:10 PM.
 
Old 01-26-2006, 06:09 PM   #8
kushalkoolwal
Senior Member
 
Registered: Feb 2004
Location: Middle of nowhere
Distribution: Debian Squeeze
Posts: 1,249

Original Poster
Rep: Reputation: 48
Quote:
Originally Posted by perfect_circle
You need to pipe the read command to read line by line. Something like this:
Code:
cat test.txt | while read a_line
do 
    echo "Now I have read the line: $a_line" 
done
In every execution of the while loop the $a_line variable will contain the next line.

Hey Perfect Circle, thanks a lot. You made my day. You are really perfect. The suggestion you gave worked perfectly for me. Thanks a lot dude.

I wonder why
find /home/user/test -type f -exec cat /dev/null > {} \;
and
cat list.txt|xargs -i cat /dev/null > {}

do not work although they look ok to me..

Isn;t that strange?
 
Old 01-26-2006, 06:13 PM   #9
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,973
Blog Entries: 11

Rep: Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879
Not without more input :}

What does the file look like internally, are the paths
to the files you want to empty absolute?


Cheers,
Tink
 
Old 01-26-2006, 06:22 PM   #10
kushalkoolwal
Senior Member
 
Registered: Feb 2004
Location: Middle of nowhere
Distribution: Debian Squeeze
Posts: 1,249

Original Poster
Rep: Reputation: 48
Quote:
Originally Posted by Tinkster
Not without more input :}

What does the file look like internally, are the paths
to the files you want to empty absolute?


Cheers,
Tink
The file looks something like this:

Code:
./file1.txt
./file2.txt
./file3.doc
...and  so on
 
Old 01-26-2006, 06:28 PM   #11
Hko
Senior Member
 
Registered: Aug 2002
Location: Groningen, The Netherlands
Distribution: ubuntu
Posts: 2,530

Rep: Reputation: 108Reputation: 108
Quote:
Originally Posted by kushalkoolwal
I wonder why
find /home/user/test -type f -exec cat /dev/null > {} \;
and
cat list.txt|xargs -i cat /dev/null > {}

do not work although they look ok to me..

Isn;t that strange?
They are not OK. Here's why:
Code:
find /home/user/test -type f
This will will search for any file called "/home/user/test". There will be only one file found: /home/user/test.

So you could expect the the file containing the list of file ("list.txt") would be truncate to zero-length! But you were lucky this did not happen, because of another error:

When redirecting to {} , the -exec option does not replace {} with the files found by "find". So /dev/null is catted to a file called "{}". I bet you have a an empty file now in your directory called "{}".

Code:
xargs -i cat /dev/null > {}
The same problem here with redirecting to {}.

Also, "xargs" will put as many arguments from its standard input (stdin) as possible. So the command actually executed will be something like:
Code:
cat /dev/null > file1 file2 file3 file4
...which isn't what you were expecting.

Last edited by Hko; 01-26-2006 at 06:30 PM.
 
Old 01-26-2006, 06:29 PM   #12
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,973
Blog Entries: 11

Rep: Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879
Bummer - it's my bad, the redirect within xargs doesn't work; the
output of xargs is being redirected to {} rather than redirecting the
output to the substituted filename. Sorry to get your hopes up :/

Alternatively create a script
#!/bin/bash
cat /dev/null > $1

and feed the cat to that ... :}

cat list.txt|xargs -i empty {}
works ...



Cheers,
Tink
 
Old 01-26-2006, 06:35 PM   #13
Hko
Senior Member
 
Registered: Aug 2002
Location: Groningen, The Netherlands
Distribution: ubuntu
Posts: 2,530

Rep: Reputation: 108Reputation: 108
Quote:
Originally Posted by Tinkster
cat list.txt|xargs -i empty {}
works ...
I've never heard of the "empty" command. And it it's not available on my computer (Debian sarge). What distro do you use?
 
Old 01-26-2006, 06:36 PM   #14
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,973
Blog Entries: 11

Rep: Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879
Quote:
Originally Posted by Hko
I've never heard of the "empty" command. And it it's not available on my computer (Debian sarge). What distro do you use?
Quote:
Alternatively create a script
#!/bin/bash
cat /dev/null > $1
That's empty ;)


Cheers,
Tink
 
Old 01-26-2006, 06:44 PM   #15
Hko
Senior Member
 
Registered: Aug 2002
Location: Groningen, The Netherlands
Distribution: ubuntu
Posts: 2,530

Rep: Reputation: 108Reputation: 108
Quote:
Originally Posted by Tinkster
That's empty
Right.
I missed that part.
I should have read your post less quickly.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
bash shell script read file line by line. Darren[UoW] Programming 51 07-04-2013 08:16 PM
C: fread to read a file line by line until the end Blue_muppet Programming 2 09-19-2008 09:42 AM
C++ text file line by line/each line to string/array Dimitris Programming 15 03-11-2008 08:22 AM
linux scripting help needed read from file line by line exc commands each line read atokad Programming 4 12-26-2003 10:24 PM
Display/Read line 'N' in a text file using script ganninu Linux - Newbie 2 10-13-2003 05:28 AM


All times are GMT -5. The time now is 05:04 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration