LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 04-21-2010, 07:28 AM   #1
SharpyWarpy
Member
 
Registered: Feb 2003
Location: Florida
Distribution: Fedora 18
Posts: 862

Rep: Reputation: 90
bash script is reprinting part of a field from an awk statement ??


Hi, everybody. Here's the bash script:
Code:
FILES="/usr/sbin/accept
/usr/sbin/pwck
/usr/sbin/chroot
/usr/bin/fakefile
/sbin/badblocks
/sbin/ypbind"
for file in $FILES
do
	if [ ! -e "$file" ]       # Check if file exists.
	then 	
	 echo "$file does not exist.";
	 continue                # On to next.
	fi
ls -l $file | awk '{ print $8 "   file size "  $5 }';  # Print 2 fields.
whatis `basename $file`   # File info.
	# Note that the whatis database needs to have been set up for this to work.
	# To do this, as root run /usr/bin/makewhatis.
	echo
done
And here's the output:

Code:
/usr/sbin/accept   file size 10
   file size 
accept               (2)  - accept a connection on a socket
accept               (3p)  - accept a new connection on a socket

/usr/sbin/pwck   file size 35496
   file size 
pwck                 (8)  - verify integrity of password files

/usr/sbin/chroot   file size 28216
   file size 
chroot               (1)  - run command or interactive shell with special root directory
chroot               (2)  - change root directory

/usr/bin/fakefile does not exist.
/sbin/badblocks   file size 20608
   file size 
badblocks            (8)  - search a device for bad blocks

/sbin/ypbind   file size 41384
   file size 
ypbind               (8)  - NIS binding process
Notice the extra " file size" lines in there? What's causing that? I'm trying to learn more bash skills. I have no experience with awk because I have been unable to understand it's basic necessity. But I thought maybe if I try it with some test scripts I might become more interested in using it more and expand my very limited capabilities. Thanks in advance, fellow Linux geeks.
 
Old 04-21-2010, 07:37 AM   #2
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,551
Blog Entries: 28

Rep: Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176
Is ls -l $file producing two lines of output on your system, the last of which doe not have 5 or more fields? You could try ls -l $file | awk '{ print $0 }'
 
Old 04-21-2010, 08:09 AM   #3
SharpyWarpy
Member
 
Registered: Feb 2003
Location: Florida
Distribution: Fedora 18
Posts: 862

Original Poster
Rep: Reputation: 90
Quote:
Originally Posted by catkin View Post
Is ls -l $file producing two lines of output on your system, the last of which doe not have 5 or more fields? You could try ls -l $file | awk '{ print $0 }'
Thank you for your reply, catkin.
No, putting the line
ls -l $file
in there after the "for" construct produces one line with 8 fields -- all of which are normal in appearance. No double lines there.
Using the awk statement you provide displays all 8 fields.
 
Old 04-21-2010, 08:36 AM   #4
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,551
Blog Entries: 28

Rep: Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176
I can't help you any further because it works for me (Slackware 13.0, bash 3.1.17, GNU Awk 3.1.6)
Code:
c@CW8:/var/log$ ls -l messages | awk '{ print $8 "   file size "  $5 }'
messages   file size 299251
 
Old 04-21-2010, 09:27 AM   #5
SharpyWarpy
Member
 
Registered: Feb 2003
Location: Florida
Distribution: Fedora 18
Posts: 862

Original Poster
Rep: Reputation: 90
Thanks, catkin. I have gawk version 3.1.7, it might have a bug of sorts but I don't use it enough to justify downgrading to find out. It might be possible to use something besides awk to achieve the same or similar results, I'll research that.
 
Old 04-21-2010, 10:28 AM   #6
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,551
Blog Entries: 28

Rep: Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176
Quote:
Originally Posted by SharpyWarpy View Post
Thanks, catkin. I have gawk version 3.1.7, it might have a bug of sorts but I don't use it enough to justify downgrading to find out. It might be possible to use something besides awk to achieve the same or similar results, I'll research that.
I very much doubt that gawk 3.1.7 has a defect when doing such a routine task but it would be interesting to try the same thing with any other versions of awk that you have installed and maybe try sed and read
Code:
c@CW8:/var/log$ ls -l messages | cut --delimiter=' ' --fields=5,8
300869 messages
c@CW8:/var/log$ while read x x x x size x x name x; do echo $size $name; done <<EoF
> $(ls -l messages)
> EoF
300900 messages
Note: the "> " prompts appear when entering this at the command prompt; they would not be necessary in a script.
 
Old 04-21-2010, 04:48 PM   #7
SharpyWarpy
Member
 
Registered: Feb 2003
Location: Florida
Distribution: Fedora 18
Posts: 862

Original Poster
Rep: Reputation: 90
Quote:
Originally Posted by catkin View Post
I very much doubt that gawk 3.1.7 has a defect when doing such a routine task but it would be interesting to try the same thing with any other versions of awk that you have installed and maybe try sed and read
Code:
c@CW8:/var/log$ ls -l messages | cut --delimiter=' ' --fields=5,8
300869 messages
c@CW8:/var/log$ while read x x x x size x x name x; do echo $size $name; done <<EoF
> $(ls -l messages)
> EoF
300900 messages
Note: the "> " prompts appear when entering this at the command prompt; they would not be necessary in a script.
Hey, those are some excellent alternatives, thanks very much. They are much simpler and are more likely to find themselves in my script files.
 
Old 04-21-2010, 05:02 PM   #8
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,551
Blog Entries: 28

Rep: Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176
Quote:
Originally Posted by SharpyWarpy View Post
Hey, those are some excellent alternatives, thanks very much. They are much simpler and are more likely to find themselves in my script files.
Did either of them fix the problem though?
 
Old 04-21-2010, 06:12 PM   #9
SharpyWarpy
Member
 
Registered: Feb 2003
Location: Florida
Distribution: Fedora 18
Posts: 862

Original Poster
Rep: Reputation: 90
Yes, the "while" statement did. After I added "ls -l $file"
to the front of the line. I wanted to add something I found. I have an alias set for ls, "alias ls='ls --color=always'" . When I changed that to "alias ls='ls --color=auto'" it fixed the problem using awk. I should mention I had been running the script without executable bit applied with ". script_file". When I made it executable and run it with "./script_file" it runs fine, only without file colors. Also with "alias ls='ls --color=auto'" it shows no color. No problem, just odd. I read the man page on "ls" and it says the "--color=auto" option causes ls to output colors only if standard output is connected to a terminal (tty). I thought that is what I'm using, since I am doing all this in CLI, X server is not running.
 
Old 04-22-2010, 01:57 AM   #10
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,551
Blog Entries: 28

Rep: Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176
Quote:
Originally Posted by SharpyWarpy View Post
Yes, the "while" statement did. After I added "ls -l $file"
to the front of the line. I wanted to add something I found. I have an alias set for ls, "alias ls='ls --color=always'" . When I changed that to "alias ls='ls --color=auto'" it fixed the problem using awk. I should mention I had been running the script without executable bit applied with ". script_file". When I made it executable and run it with "./script_file" it runs fine, only without file colors. Also with "alias ls='ls --color=auto'" it shows no color. No problem, just odd. I read the man page on "ls" and it says the "--color=auto" option causes ls to output colors only if standard output is connected to a terminal (tty). I thought that is what I'm using, since I am doing all this in CLI, X server is not running.
Ah ha! Sorry for not considering an alias; I usually do and advise people to use /bin/ls just to rule it out. It's a good idea for scripts to initialise with unalias -a as part of setting up a known environment, same as it's a good idea to set $PATH.

If the output of ls is being sent down a pipeline (as it was, to awk) then stdout is not connected to a terminal.
 
Old 04-22-2010, 03:10 AM   #11
SharpyWarpy
Member
 
Registered: Feb 2003
Location: Florida
Distribution: Fedora 18
Posts: 862

Original Poster
Rep: Reputation: 90
Quote:
Ah ha! Sorry for not considering an alias; I usually do and advise people to use /bin/ls just to rule it out. It's a good idea for scripts to initialise with unalias -a as part of setting up a known environment, same as it's a good idea to set $PATH.

If the output of ls is being sent down a pipeline (as it was, to awk) then stdout is not connected to a terminal.
"...use /bin/ls just to rule it out...". Also good for that is to use backward slash immediately before the command to cancel the alias property.

Enlightening it is to hear sending a command output to pipe disables stdout. I did not know that. I love learning this stuff, it is filling my toolbox fast. Hope I can learn to apply it effectively.
I have another question that applies to your statement:
"...same as it's a good idea to set $PATH...".
Whenever I start a script with a shabang -- #!/bin/bash -- and end the script with "exit 0" it exits the entire tty and I have to log in again. Everything I have read on the subject says it is good practice to do it this way so there are no environment variables set in the normal environment after the script closes but I can't implement that without having to log back in to the console every time. So what am I doing wrong? How can I do this the correct way? I really want to write good scripts that work the way they are supposed to.
 
Old 04-22-2010, 03:47 AM   #12
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,551
Blog Entries: 28

Rep: Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176
Yes -- \ls works, too

Are you running your scripts by the "." command? Assuming you don't have "." in your path then the way to run a script in the current directory is
Code:
./my_script.sh
where the "." signifies "this directory". If you are running your script by
Code:
. my_script.sh
then it runs the script in your current shell and exit terminates the shell. For more info on this see the ". (a period)" here and "source" here (but it's just a synonym for "." so there's not much to see).

The usefulness of the "." and source commands is because the called script _does_ affect the environment which is useful for setting variables (including environment variables) and aliases; hence they are often used in bash startup files. Another use is as a simple way of setting configuration variables; this is commonly used in boot scripts.
 
Old 04-22-2010, 03:48 AM   #13
SharpyWarpy
Member
 
Registered: Feb 2003
Location: Florida
Distribution: Fedora 18
Posts: 862

Original Poster
Rep: Reputation: 90
Okay never mind about the script using "exit 0" logging out of the console. I was using ". script_file" to execute instead of making the script executable first and running it with "./script_file". Making it executable causes the script to exit without logging completely out of the console and it also unsets any environment variables set within the script. So I have made mistakes and learned the hard way, which is the best way to retain that knowledge. Now if I can just stop kicking myself. I swear, I am my own worst enemy. But I persevere to the bitter end, maties. To the bitter end.
 
Old 04-22-2010, 03:58 AM   #14
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,551
Blog Entries: 28

Rep: Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176
Quote:
Originally Posted by SharpyWarpy View Post
Okay never mind about the script using "exit 0" logging out of the console. I was using ". script_file" to execute instead of making the script executable first and running it with "./script_file". Making it executable causes the script to exit without logging completely out of the console and it also unsets any environment variables set within the script. So I have made mistakes and learned the hard way, which is the best way to retain that knowledge. Now if I can just stop kicking myself. I swear, I am my own worst enemy. But I persevere to the bitter end, maties. To the bitter end.
Good spirit! It will take you a long way.

Do you know the origin of "to the bitter end"? It comes from "to the biter end", a nautical reference to letting the anchor out so far that all the chain is used and the "biter" end -- where the chain bites into part of the ship, usually the keel -- to prevent losing the anchor and chain overboard. Thus is is a nice way of saying "as far as possible, right to the very end".
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] awk: how to print a field when field position is unknown? elfoozo Programming 12 08-18-2010 03:52 AM
how to keep Field Separator in AWK when using a sub statement tmcguinness Programming 4 02-09-2009 02:24 PM
awk run script on field before output Geneset Programming 3 08-26-2008 04:59 AM
Strange if statement behaviour when using bash/bash script freeindy Programming 7 08-04-2008 06:00 AM
Need help adding an IF / ELSE statement to my Bash Script Relix Linux - General 1 08-01-2002 02:03 PM


All times are GMT -5. The time now is 10:42 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration