LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - General (https://www.linuxquestions.org/questions/linux-general-1/)
-   -   bash script is reprinting part of a field from an awk statement ?? (https://www.linuxquestions.org/questions/linux-general-1/bash-script-is-reprinting-part-of-a-field-from-an-awk-statement-803272/)

SharpyWarpy 04-21-2010 08:28 AM

bash script is reprinting part of a field from an awk statement ??
 
Hi, everybody. Here's the bash script:
Code:

FILES="/usr/sbin/accept
/usr/sbin/pwck
/usr/sbin/chroot
/usr/bin/fakefile
/sbin/badblocks
/sbin/ypbind"
for file in $FILES
do
        if [ ! -e "$file" ]      # Check if file exists.
        then       
        echo "$file does not exist.";
        continue                # On to next.
        fi
ls -l $file | awk '{ print $8 "  file size "  $5 }';  # Print 2 fields.
whatis `basename $file`  # File info.
        # Note that the whatis database needs to have been set up for this to work.
        # To do this, as root run /usr/bin/makewhatis.
        echo
done

And here's the output:

Code:

/usr/sbin/accept  file size 10
  file size
accept              (2)  - accept a connection on a socket
accept              (3p)  - accept a new connection on a socket

/usr/sbin/pwck  file size 35496
  file size
pwck                (8)  - verify integrity of password files

/usr/sbin/chroot  file size 28216
  file size
chroot              (1)  - run command or interactive shell with special root directory
chroot              (2)  - change root directory

/usr/bin/fakefile does not exist.
/sbin/badblocks  file size 20608
  file size
badblocks            (8)  - search a device for bad blocks

/sbin/ypbind  file size 41384
  file size
ypbind              (8)  - NIS binding process

Notice the extra " file size" lines in there? What's causing that? I'm trying to learn more bash skills. I have no experience with awk because I have been unable to understand it's basic necessity. But I thought maybe if I try it with some test scripts I might become more interested in using it more and expand my very limited capabilities. Thanks in advance, fellow Linux geeks.

catkin 04-21-2010 08:37 AM

Is ls -l $file producing two lines of output on your system, the last of which doe not have 5 or more fields? You could try ls -l $file | awk '{ print $0 }'

SharpyWarpy 04-21-2010 09:09 AM

Quote:

Originally Posted by catkin (Post 3942588)
Is ls -l $file producing two lines of output on your system, the last of which doe not have 5 or more fields? You could try ls -l $file | awk '{ print $0 }'

Thank you for your reply, catkin.
No, putting the line
ls -l $file
in there after the "for" construct produces one line with 8 fields -- all of which are normal in appearance. No double lines there.
Using the awk statement you provide displays all 8 fields.

catkin 04-21-2010 09:36 AM

I can't help you any further because it works for me (Slackware 13.0, bash 3.1.17, GNU Awk 3.1.6)
Code:

c@CW8:/var/log$ ls -l messages | awk '{ print $8 "  file size "  $5 }'
messages  file size 299251


SharpyWarpy 04-21-2010 10:27 AM

Thanks, catkin. I have gawk version 3.1.7, it might have a bug of sorts but I don't use it enough to justify downgrading to find out. It might be possible to use something besides awk to achieve the same or similar results, I'll research that.

catkin 04-21-2010 11:28 AM

Quote:

Originally Posted by SharpyWarpy (Post 3942756)
Thanks, catkin. I have gawk version 3.1.7, it might have a bug of sorts but I don't use it enough to justify downgrading to find out. It might be possible to use something besides awk to achieve the same or similar results, I'll research that.

I very much doubt that gawk 3.1.7 has a defect when doing such a routine task but it would be interesting to try the same thing with any other versions of awk that you have installed and maybe try sed and read
Code:

c@CW8:/var/log$ ls -l messages | cut --delimiter=' ' --fields=5,8
300869 messages
c@CW8:/var/log$ while read x x x x size x x name x; do echo $size $name; done <<EoF
> $(ls -l messages)
> EoF
300900 messages

Note: the "> " prompts appear when entering this at the command prompt; they would not be necessary in a script.

SharpyWarpy 04-21-2010 05:48 PM

Quote:

Originally Posted by catkin (Post 3942799)
I very much doubt that gawk 3.1.7 has a defect when doing such a routine task but it would be interesting to try the same thing with any other versions of awk that you have installed and maybe try sed and read
Code:

c@CW8:/var/log$ ls -l messages | cut --delimiter=' ' --fields=5,8
300869 messages
c@CW8:/var/log$ while read x x x x size x x name x; do echo $size $name; done <<EoF
> $(ls -l messages)
> EoF
300900 messages

Note: the "> " prompts appear when entering this at the command prompt; they would not be necessary in a script.

Hey, those are some excellent alternatives, thanks very much. They are much simpler and are more likely to find themselves in my script files.

catkin 04-21-2010 06:02 PM

Quote:

Originally Posted by SharpyWarpy (Post 3943197)
Hey, those are some excellent alternatives, thanks very much. They are much simpler and are more likely to find themselves in my script files.

Did either of them fix the problem though?

SharpyWarpy 04-21-2010 07:12 PM

Yes, the "while" statement did. After I added "ls -l $file"
to the front of the line. I wanted to add something I found. I have an alias set for ls, "alias ls='ls --color=always'" . When I changed that to "alias ls='ls --color=auto'" it fixed the problem using awk. I should mention I had been running the script without executable bit applied with ". script_file". When I made it executable and run it with "./script_file" it runs fine, only without file colors. Also with "alias ls='ls --color=auto'" it shows no color. No problem, just odd. I read the man page on "ls" and it says the "--color=auto" option causes ls to output colors only if standard output is connected to a terminal (tty). I thought that is what I'm using, since I am doing all this in CLI, X server is not running.

catkin 04-22-2010 02:57 AM

Quote:

Originally Posted by SharpyWarpy (Post 3943282)
Yes, the "while" statement did. After I added "ls -l $file"
to the front of the line. I wanted to add something I found. I have an alias set for ls, "alias ls='ls --color=always'" . When I changed that to "alias ls='ls --color=auto'" it fixed the problem using awk. I should mention I had been running the script without executable bit applied with ". script_file". When I made it executable and run it with "./script_file" it runs fine, only without file colors. Also with "alias ls='ls --color=auto'" it shows no color. No problem, just odd. I read the man page on "ls" and it says the "--color=auto" option causes ls to output colors only if standard output is connected to a terminal (tty). I thought that is what I'm using, since I am doing all this in CLI, X server is not running.

Ah ha! Sorry for not considering an alias; I usually do and advise people to use /bin/ls just to rule it out. It's a good idea for scripts to initialise with unalias -a as part of setting up a known environment, same as it's a good idea to set $PATH.

If the output of ls is being sent down a pipeline (as it was, to awk) then stdout is not connected to a terminal.

SharpyWarpy 04-22-2010 04:10 AM

Quote:

Ah ha! Sorry for not considering an alias; I usually do and advise people to use /bin/ls just to rule it out. It's a good idea for scripts to initialise with unalias -a as part of setting up a known environment, same as it's a good idea to set $PATH.

If the output of ls is being sent down a pipeline (as it was, to awk) then stdout is not connected to a terminal.
"...use /bin/ls just to rule it out...". Also good for that is to use backward slash immediately before the command to cancel the alias property.

Enlightening it is to hear sending a command output to pipe disables stdout. I did not know that. I love learning this stuff, it is filling my toolbox fast. Hope I can learn to apply it effectively.
I have another question that applies to your statement:
"...same as it's a good idea to set $PATH...".
Whenever I start a script with a shabang -- #!/bin/bash -- and end the script with "exit 0" it exits the entire tty and I have to log in again. Everything I have read on the subject says it is good practice to do it this way so there are no environment variables set in the normal environment after the script closes but I can't implement that without having to log back in to the console every time. So what am I doing wrong? How can I do this the correct way? I really want to write good scripts that work the way they are supposed to.

catkin 04-22-2010 04:47 AM

Yes -- \ls works, too :)

Are you running your scripts by the "." command? Assuming you don't have "." in your path then the way to run a script in the current directory is
Code:

./my_script.sh
where the "." signifies "this directory". If you are running your script by
Code:

. my_script.sh
then it runs the script in your current shell and exit terminates the shell. For more info on this see the ". (a period)" here and "source" here (but it's just a synonym for "." so there's not much to see).

The usefulness of the "." and source commands is because the called script _does_ affect the environment which is useful for setting variables (including environment variables) and aliases; hence they are often used in bash startup files. Another use is as a simple way of setting configuration variables; this is commonly used in boot scripts.

SharpyWarpy 04-22-2010 04:48 AM

Okay never mind about the script using "exit 0" logging out of the console. I was using ". script_file" to execute instead of making the script executable first and running it with "./script_file". Making it executable causes the script to exit without logging completely out of the console and it also unsets any environment variables set within the script. So I have made mistakes and learned the hard way, which is the best way to retain that knowledge. Now if I can just stop kicking myself. I swear, I am my own worst enemy. But I persevere to the bitter end, maties. To the bitter end.

catkin 04-22-2010 04:58 AM

Quote:

Originally Posted by SharpyWarpy (Post 3943747)
Okay never mind about the script using "exit 0" logging out of the console. I was using ". script_file" to execute instead of making the script executable first and running it with "./script_file". Making it executable causes the script to exit without logging completely out of the console and it also unsets any environment variables set within the script. So I have made mistakes and learned the hard way, which is the best way to retain that knowledge. Now if I can just stop kicking myself. I swear, I am my own worst enemy. But I persevere to the bitter end, maties. To the bitter end.

Good spirit! :) It will take you a long way.

Do you know the origin of "to the bitter end"? It comes from "to the biter end", a nautical reference to letting the anchor out so far that all the chain is used and the "biter" end -- where the chain bites into part of the ship, usually the keel -- to prevent losing the anchor and chain overboard. Thus is is a nice way of saying "as far as possible, right to the very end". :study:


All times are GMT -5. The time now is 12:42 PM.