LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 02-12-2011, 04:10 AM   #1
ted_chou12
Member
 
Registered: Aug 2010
Location: Zhongli, Taoyuan
Distribution: slackware, windows, debian (armv4l GNU/Linux)
Posts: 421
Blog Entries: 28

Rep: Reputation: 2
bash reads one less line


Hi, I am using this code:
Quote:
while read line ; do
...
done
to read file line by line, but there is a problem when the text file lacks an empty line:
Code:
1
2
3
4
5
//extra spaced line
This the script reads fine up to line 5
but with:
Code:
1
2
3
4
5
without the last spaced line, script only reads to line 4. fifth line is ignored. Is there anyway to force it read the last line?
Thanks
 
Old 02-12-2011, 04:17 AM   #2
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,502

Rep: Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893
Seeing the windows icon on your status makes me wonder if the file being read is created under Windows or linux?
If windows you need to remove the extraneous newline format from the end.
 
Old 02-12-2011, 04:33 AM   #3
ted_chou12
Member
 
Registered: Aug 2010
Location: Zhongli, Taoyuan
Distribution: slackware, windows, debian (armv4l GNU/Linux)
Posts: 421
Blog Entries: 28

Original Poster
Rep: Reputation: 2
yeah, I am using windows os, but im using winscp ascii to edit the text files. But if I remove those extra lines at the end, the script doesn't read up to the last non-empty line.
 
Old 02-12-2011, 05:08 AM   #4
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,502

Rep: Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893Reputation: 1893
Try running od -c on the file and see what characters are there or if you have it try running dos2unix over the file.
The action of not reading the last line is not in any way to do with the while loop.
 
Old 02-12-2011, 10:05 AM   #5
Kenhelm
Member
 
Registered: Mar 2008
Location: N. W. England
Distribution: Mandriva
Posts: 329

Rep: Reputation: 140Reputation: 140
This reproduces the problem
Code:
echo -n '1
2
3
4
5' | while read line; do
       echo "$line"
     done
1
2
3
4
grep '^' will add a newline to the end of a file if there isn't one already there. Try
Code:
grep '^' file | while read line; do
                  echo "$line"
                done
# Or

while read line; do
  echo "$line"
done < <(grep '^' file)

Last edited by Kenhelm; 02-12-2011 at 10:10 AM.
 
Old 02-12-2011, 10:25 AM   #6
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,498

Rep: Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956
The line remains in the input buffer when the loop terminates, so that you can process it after the loop itself:
Code:
while read line
do
  echo $line
done < file
echo $line
but you have to find a way to check if the file has an ending newline before you attempt to process the last line (otherwise it will be processed twice, one in the loop and one after). In alternative follow the suggestion by Kenhelm. Another option is awk:
Code:
while read line
do
  echo $line
done < <(awk 1 file)
 
  


Reply

Tags
shell script


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
shell script reads all file names passed on the command line path_finder Linux - Kernel 1 04-01-2010 04:01 PM
Bash script that reads files in directory and processes them KayakJim Linux - Newbie 2 01-22-2010 09:53 AM
need to delete a line if a field of that line matches using awf in bash scripting accesskarthi Linux - Newbie 8 06-29-2009 03:15 AM
bash : read every line from text file starting at given line number quadmore Programming 4 02-20-2009 12:29 PM


All times are GMT -5. The time now is 08:49 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration