LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 03-16-2006, 12:59 PM   #1
moo-cow
Member
 
Registered: Mar 2006
Distribution: Debian
Posts: 105

Rep: Reputation: 25
Shell Scripting: Strange Problem with Text Files


I've got a problem with the following (simplified) script:

while read line
do
echo $line
done < MYFILE

When I create MYFILE with Kate or KWrite like this

One
Two
Three

I get the following output from my script:

One
Two

-> One line is missing! However, when I create the same file in Vim, the output is OK:

One
Two
Three

When I open the file I created in Vim with KWrite or Kate, it shows an empty line at the end that should not be there:

1: One
2: Two
3: Three
4:

It seems that when I use Kate or Kwrite, I have to end every file with an empty line for my shell script to read it. Why is this so? Please enlighten me!
The problem is I have this huge stack of text files created with Kate that I need to process line by line with my script...

Thanks,
MOO-COW
 
Old 03-16-2006, 01:16 PM   #2
jim mcnamara
Member
 
Registered: May 2002
Posts: 964

Rep: Reputation: 34
The "extra line" should be there - it is not a line, it's just the way vi shows that a newline exists at the end of the previous line.

Your bash problem is that the last line does not end with a newline.

This can kill you when you edit files like /etc/fstab - every line HAS to have a newline in order to be seen. Lots of new unix users run afoul of this.
 
Old 03-16-2006, 03:24 PM   #3
moo-cow
Member
 
Registered: Mar 2006
Distribution: Debian
Posts: 105

Original Poster
Rep: Reputation: 25
OK, then I have to check for each file whether it ends with a newline character and, if not, append one. Any idea how to do that in Bash?
 
Old 03-20-2006, 03:59 AM   #4
/bin/bash
Senior Member
 
Registered: Jul 2003
Location: Indiana
Distribution: Mandrake Slackware-current QNX4.25
Posts: 1,802

Rep: Reputation: 46
You would first check the last character of the file to see if it is a newline:
tail -c1 file|grep "^$"
If it is not a newline then you can add newline at the end of a file like this:
echo >>file

So you could put it all together like this:
Code:
#!/bin/bash
for i in *;do
  tail -c1 $i|grep "^$"
  if [ "$?" = "1" ]; then #If $? = 1 then grep failed
    echo >>$i             #So we add a newline to the end of the file
  fi
done
The above checks all files in the current directory, you may need to modify that.

Last edited by /bin/bash; 03-20-2006 at 04:00 AM.
 
Old 03-28-2006, 03:12 AM   #5
moo-cow
Member
 
Registered: Mar 2006
Distribution: Debian
Posts: 105

Original Poster
Rep: Reputation: 25
That's exactly what I needed... and so short & elegant!
Thanks very much
 
Old 03-28-2006, 06:25 PM   #6
/bin/bash
Senior Member
 
Registered: Jul 2003
Location: Indiana
Distribution: Mandrake Slackware-current QNX4.25
Posts: 1,802

Rep: Reputation: 46
No prob, HTH.
 
Old 03-30-2006, 09:49 AM   #7
Gins
Senior Member
 
Registered: Jul 2004
Location: Germany
Distribution: open SUSE 11.0, Fedora 7 and Mandriva 2007
Posts: 1,638

Rep: Reputation: 46
#!/bin/bash
for i in *;do
tail -c1 $i|grep "^$"
if [ "$?" = "1" ]; then #If $? = 1 then grep failed
echo >>$i #So we add a newline to the end of the file
fi
done

I ran the above program. Nothing happened.

What does it check?

What is the line '' tail -c1 $1 | grep "$" " is doing here ?
I can't understand it.
---------------------------------------------------------------------------

I created a file using the '' vi '' editor and named '' unwanted1 ''.


Some of the output is here.

[nissanka@c83-250-107-194 ~]$ ./unwanted1

tail: error reading `Applications': Is a directory
./unwanted1: line 10: Applications: Is a directory


tail: cannot open `The' for reading: No such file or directory
tail: cannot open `Holy' for reading: No such file or directory
tail: cannot open `Family\'s' for reading: No such file or directory
tail: cannot open `flight' for reading: No such file or directory
tail: cannot open `to' for reading: No such file or directory
tail: cannot open `Egypt' for reading: No such file or directory
./unwanted1: line 10: 1: ambiguous redirect
tail: cannot open `The' for reading: No such file or directory
tail: cannot open `Portugese' for reading: No such file or directory
tail: cannot open `advent' for reading: No such file or directory
tail: cannot open `in' for reading: No such file or directory
tail: cannot open `Sri' for reading: No such file or directory
tail: cannot open `Lanka' for reading: No such file or directory
./unwanted1: line 10: 1: ambiguous redirect


tail: error reading `tmp': Is a directory
./unwanted1: line 10: tmp: Is a directory

tail: error reading `ussh': Is a directory
./unwanted1: line 10: ussh: Is a directory

tail: error reading `Varansi_files': Is a directory
./unwanted1: line 10: Varansi_files: Is a directory

tail: error reading `Video': Is a directory
./unwanted1: line 10: Video: Is a directory

[nissanka@c83-250-107-194 ~]$

Last edited by Gins; 03-30-2006 at 09:50 AM.
 
Old 03-30-2006, 10:53 AM   #8
/bin/bash
Senior Member
 
Registered: Jul 2003
Location: Indiana
Distribution: Mandrake Slackware-current QNX4.25
Posts: 1,802

Rep: Reputation: 46
Quote:
What is the line '' tail -c1 $1 | grep "$" " is doing here ?
I can't understand it.
It should be:
tail -c1 $i|grep "^$"
tail -c1 returns the last character of a file ($i in this case.)
grep "^$" is looking for a newline character. Probably should have used something else to make it more readable:
grep "\n" would do the same thing.

As for the directory errors, that could be avoided by first checking that $i is a file.

Code:
#!/bin/bash
for i in *;do
  if [ -f "$i" ];then
    tail -c1 $i|grep "\n"
    if [ "$?" = "1" ]; then #If $? = 1 then grep failed
      echo >>$i             #So we add a newline to the end of the file
    fi
  fi
done
As posted earlier this just checks the last character of a file to see if it is a newline. If it isn't a newline then it adds one.
 
Old 03-30-2006, 01:30 PM   #9
Gins
Senior Member
 
Registered: Jul 2004
Location: Germany
Distribution: open SUSE 11.0, Fedora 7 and Mandriva 2007
Posts: 1,638

Rep: Reputation: 46
bin/bash suggested the following program. I ran it. I named it ' unwanted2 '.

#!/bin/bash
for i in *;do
if [ -f "$i" ];then
tail -c1 $i|grep "\n"
if [ "$?" = "1" ]; then #If $? = 1 then grep failed
echo >>$i #So we add a newline to the end of the file
fi
fi
done
---------------------------------------------------


Some of the output is as follows:

[nissanka@c83-250-107-194 ~]$ chmod 755 unwanted2
[nissanka@c83-250-107-194 ~]$ ./unwanted2
./unwanted2: line 15: Important_Command: Permission denied
./unwanted2: line 15: klamav-0.32-2mdk.x86_64.rpm: Permission denied
tail: cannot open `Nepal\'s' for reading: No such file or directory
tail: cannot open `Buddha' for reading: No such file or directory
tail: cannot open `boy' for reading: No such file or directory
tail: cannot open `goes' for reading: No such file or directory
tail: cannot open `missing' for reading: No such file or directory
./unwanted2: line 15: 1: ambiguous redirect
./unwanted2: line 15: NTI: Permission denied
tail: cannot open `Portugese' for reading: No such file or directory
tail: cannot open `in' for reading: No such file or directory
tail: cannot open `Sri' for reading: No such file or directory
tail: cannot open `Lanka' for reading: No such file or directory
./unwanted2: line 15: 1: ambiguous redirect
tail: cannot open `Scientists' for reading: No such file or directory
tail: cannot open `to' for reading: No such file or directory
tail: cannot open `check' for reading: No such file or directory
tail: cannot open `Nepal' for reading: No such file or directory
tail: cannot open `Buddha' for reading: No such file or directory
tail: cannot open `boy.stm' for reading: No such file or directory
./unwanted2: line 15: 1: ambiguous redirect
tail: cannot open `Sri' for reading: No such file or directory
tail: cannot open `Lanka' for reading: No such file or directory
tail: cannot open `ceasefire' for reading: No such file or directory
./unwanted2: line 15: 1: ambiguous redirect


----------------------------------------------------------------------
----------------------------------------------------------------------

[nissanka@c83-250-107-194 ~]$ ./unwanted2 teco8

./unwanted2: line 15: Important_Command: Permission denied
./unwanted2: line 15: klamav-0.32-2mdk.x86_64.rpm: Permission denied
tail: cannot open `Nepal\'s' for reading: No such file or directory
tail: cannot open `Buddha' for reading: No such file or directory
tail: cannot open `boy' for reading: No such file or directory
tail: cannot open `goes' for reading: No such file or directory
tail: cannot open `missing' for reading: No such file or directory
./unwanted2: line 15: 1: ambiguous redirect
./unwanted2: line 15: NTI: Permission denied
tail: cannot open `Portugese' for reading: No such file or directory
tail: cannot open `in' for reading: No such file or directory
tail: cannot open `Sri' for reading: No such file or director



It didn't do anything worthwhile. I have a file called 'teco8'. It is just a text file I wrote using the 'vi' editor

What is the problem?
 
Old 03-31-2006, 03:00 AM   #10
Gins
Senior Member
 
Registered: Jul 2004
Location: Germany
Distribution: open SUSE 11.0, Fedora 7 and Mandriva 2007
Posts: 1,638

Rep: Reputation: 46
I urge someone to look at this post. I badly need your help.
 
Old 03-31-2006, 03:55 AM   #11
/bin/bash
Senior Member
 
Registered: Jul 2003
Location: Indiana
Distribution: Mandrake Slackware-current QNX4.25
Posts: 1,802

Rep: Reputation: 46
I forgot to quote $i. Try this.

Code:
#!/bin/bash
for i in *;do
  if [ -f "$i" ];then
    tail -c1 "$i"|grep "\n"
    if [ "$?" = "1" ]; then #If $? = 1 then grep failed
      echo >>"$i"           #So we add a newline to the end of the file
    fi
  fi
done
 
Old 03-31-2006, 04:27 AM   #12
Gins
Senior Member
 
Registered: Jul 2004
Location: Germany
Distribution: open SUSE 11.0, Fedora 7 and Mandriva 2007
Posts: 1,638

Rep: Reputation: 46
Thanks bin/bash for taking time again to help me.
By the way, I like the name you invented. You are bin/bash!

#!/bin/bash
for i in *;do
if [ -f "$i" ];then
tail -c1 "$i"|grep "\n"
if [ "$?" = "1" ]; then #If $? = 1 then grep failed
echo >>"$i" #So we add a newline to the end of the file
fi
fi
done


---------------------------------------------

Yes, I ran the above. I named it 'unwanted3'.

echo >>$i

tail -c1 $i|grep "\n"

Only the above two lines are different than the previous program
-----------------------------------------------------

The output is as follows:

[nissanka@c83-250-107-194 ~]$ chmod 755 unwanted3
[nissanka@c83-250-107-194 ~]$ ./unwanted3
./unwanted3: line 12: Important_Command: Permission denied
./unwanted3: line 12: klamav-0.32-2mdk.x86_64.rpm: Permission denied
./unwanted3: line 12: NTI: Permission denied
[nissanka@c83-250-107-194 ~]$

Is it fine? I am not satisfied

Last edited by Gins; 03-31-2006 at 04:29 AM.
 
Old 03-31-2006, 02:09 PM   #13
Gins
Senior Member
 
Registered: Jul 2004
Location: Germany
Distribution: open SUSE 11.0, Fedora 7 and Mandriva 2007
Posts: 1,638

Rep: Reputation: 46
I hope /bin/bash or someone will read this and tell me something more.
 
Old 03-31-2006, 07:15 PM   #14
/bin/bash
Senior Member
 
Registered: Jul 2003
Location: Indiana
Distribution: Mandrake Slackware-current QNX4.25
Posts: 1,802

Rep: Reputation: 46
Permission denied

You probably do not have write permission for those files.
 
Old 04-01-2006, 03:24 AM   #15
Gins
Senior Member
 
Registered: Jul 2004
Location: Germany
Distribution: open SUSE 11.0, Fedora 7 and Mandriva 2007
Posts: 1,638

Rep: Reputation: 46
/bin/bash

First of all I must thank you for taking time to reply me.

You say that I probably don't have permission to write those files.
In my experience, definetly I don't have permission to write those files unless I logged in as a root user.

For example, I tried to use the 'find' command to search files which are written using the Open office wordprocessor. The funny message of 'no permission' surfaced. Then I logged in as a root user. Everything went smoothly.

However, my question was a different one.

Has the program worked fine? I am not sure whether it yielded the right output.

As I said before I am a newbie to these scripting, sed awk. I am comfortable with 'grep commands', 'find commands' and other UNIX related commands. I am not an expert , however. There are more things to learn.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Combine files using shell scripting ccray Linux - Newbie 14 01-12-2006 11:14 AM
Shell Scripting and Files With Spaces Matir Programming 12 08-17-2005 01:43 AM
Reading text from a file using shell scripting. mrobertson Programming 11 06-29-2005 12:12 PM
Shell scripting problem for importing Firefox files montylee Programming 6 06-03-2005 01:52 AM
Shell Scripting: using <printf> to align text need HELP!!! naomi Programming 2 01-06-2005 01:06 PM


All times are GMT -5. The time now is 11:06 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration