LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-28-2012, 01:48 PM   #1
vicky007aggrwal
Member
 
Registered: Aug 2012
Posts: 95

Rep: Reputation: Disabled
read file line by line


right way to read file contents line by line

---------- Post added 10-28-12 at 11:18 PM ----------

http://en.kioskea.net/faq/1757-how-t...e-line-by-line
 
Old 10-28-2012, 02:25 PM   #2
Didier Spaier
LQ Addict
 
Registered: Nov 2008
Location: Paris, France
Distribution: Slint64-14.2 on Lenovo Thinkpad W520
Posts: 7,867

Rep: Reputation: 2780Reputation: 2780Reputation: 2780Reputation: 2780Reputation: 2780Reputation: 2780Reputation: 2780Reputation: 2780Reputation: 2780Reputation: 2780Reputation: 2780
Better use awk for that.

To print each line of a file:

Code:
awk '{print}' file.txt
To print all users and their home directories from /etc/passwd, separated by a dot:

Code:
awk -F : '{print $1 "." $5}' /etc/passwd
To know more: man awk.
 
Old 10-28-2012, 02:55 PM   #3
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,837

Rep: Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981
Are you the author of this? And why exactly did you post it? What context? Do you have a question, are you looking for comments, or are you just doing this to help others?

In any case, this is incorrect:

Quote:
Although the while loop is the easiest method, it has its side effects. It obliterates the formatting of lines including spaces and tabs.
A while+read loop saves everything up to the next line delimiter exactly as it appears in the original. All whitespace other than the newline itself (the line delimiter) is preserved. It only ever breaks anything else up if you supply read with multiple variables as arguments, in which case it divides the line up according to the setting of IFS.

In fact, it's the for loop that comes out worse here. Preserving line formatting requires awkwardly working around the natural behavior of the shell, and even then you can never preserve completely blank lines unless you also forgo using the newline as a delimiter.

The for loop is also less efficient, as the entire file has to be read into memory before the loop is ever run. Not to mention that it generally relies on the external cat command, which means even more overhead (bash allows you to replace it with $(<file) but that's a non-portable extension).


These two pages cover the reading of files in the shell in much more detail than the above:

Don't Read Lines With For
How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?


In short, a while+read loop is always the preferred way to read a file or any other input that consists of lines of complex text.

Last edited by David the H.; 10-28-2012 at 02:56 PM.
 
2 members found this post helpful.
Old 10-28-2012, 03:15 PM   #4
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 3,618

Rep: Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577
Quote:
Originally Posted by David the H. View Post
A while+read loop saves everything up to the next line delimiter exactly as it appears in the original. All whitespace other than the newline itself (the line delimiter) is preserved. It only ever breaks anything else up if you supply read with multiple variables as arguments, in which case it divides the line up according to the setting of IFS.
Even with just a single variable as an argument, the read command looks for the first word on the line and will discard any leading IFS characters. If you want to preserve leading white space, you have to first set IFS to a null string.

Last edited by rknichols; 10-28-2012 at 03:21 PM. Reason: Grammar fix
 
1 members found this post helpful.
Old 10-28-2012, 07:23 PM   #5
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,837

Rep: Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981
Yeah, ok, my mistake. I guess I kind of jumped the gun. I was probably thinking more of bash's mapfile (a.k.a. readarray), which doesn't have that problem.

But it's still easier, cleaner, and more efficient to handle it with while than with for, since you can simply set IFS directly for read alone, rather than globally.
 
Old 10-28-2012, 09:50 PM   #6
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 3,618

Rep: Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577Reputation: 1577
Quote:
Originally Posted by David the H. View Post
But it's still easier, cleaner, and more efficient to handle it with while than with for, since you can simply set IFS directly for read alone, rather than globally.
Oh, definitely! No disagreement there. I've just been bitten by that leading white space issue enough times to remember the pain well.
 
Old 10-30-2012, 12:32 PM   #7
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,837

Rep: Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981
Yeah. I haven't had much need to worry about preserving leading whitespace much myself, or I suppose I would've caught it myself too.
 
Old 10-31-2012, 02:26 PM   #8
vicky007aggrwal
Member
 
Registered: Aug 2012
Posts: 95

Original Poster
Rep: Reputation: Disabled
Thanks GUys for suggesting
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
bash : read every line from text file starting at given line number quadmore Programming 4 02-20-2009 01:29 PM
help with c program to read each line from text file, split line , process and output gkoumantaris Programming 12 07-01-2008 01:38 PM
php - Read file line by line and change a specific line. anrea Programming 2 01-28-2007 02:43 PM
linux scripting help needed read from file line by line exc commands each line read atokad Programming 4 12-26-2003 11:24 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 05:53 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration