ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Here is a program I had to write for work that I really enjoyed writing. I figured I would see if anyone else would like to try.
RULES: There are no rules. Just make it as clean as possible.
GUIDELINES: You are given a file with 4 wireless interfaces. You must monitor both of the overruns values for it's respective iface and check if the values are incrementing. For each time you check and the value is indeed incrementing for either overrun, you need to keep track. Once the counter value for its respective iface reaches 5, restart the iface. Also, the files must be hidden so it looks clean.
TEST FILE:
Code:
eth0 Link encap:Ethernet HWaddr 78:24:af:6e:a6:60
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:337 errors:0 dropped:0 overruns:0 frame:0
TX packets:337 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:25551 (25.5 KB) TX bytes:25551 (25.5 KB)
wlan0 Link encap:Ethernet HWaddr 68:a3:c4:85:24:b4
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:21 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth1 Link encap:Ethernet HWaddr 78:24:af:6e:a6:60
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:337 errors:0 dropped:0 overruns:0 frame:0
TX packets:337 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:25551 (25.5 KB) TX bytes:25551 (25.5 KB)
wlan1 Link encap:Ethernet HWaddr 68:a3:c4:85:24:b4
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:10 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth2 Link encap:Ethernet HWaddr 78:24:af:6e:a6:60
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:337 errors:0 dropped:0 overruns:0 frame:0
TX packets:337 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:25551 (25.5 KB) TX bytes:25551 (25.5 KB)
wlan2 Link encap:Ethernet HWaddr 68:a3:c4:85:24:b4
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:10 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth3 Link encap:Ethernet HWaddr 78:24:af:6e:a6:60
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:337 errors:0 dropped:0 overruns:0 frame:0
TX packets:337 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:25551 (25.5 KB) TX bytes:25551 (25.5 KB)
wlan3 Link encap:Ethernet HWaddr 68:a3:c4:85:24:b4
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
MY SOLUTION:
Code:
#!/bin/bash
#
# Grabs each iface and its respective overruns variables, ties them all together
# and pushes them into the res variable. Resulting in a line for every iface.
res=$(readarray -t mon < <(cat ifconfig | \
awk '/w[a-z]*[0-9]/,/TX.*overruns:[0-9]/{print}'); echo "${mon[@]}" | \
egrep -o "(w[a-z]*[0-9]|overruns:[0-9]{1,99})" | \
sed -e 'N;N;s/\n/ /g');
readarray -t fin < <(echo "$res");
counter=0
for i in "${fin[@]}"; do
# Joins both overrun numbers from same iface together and pushes them into the don variable.
# Each iface has 2 overruns counters. One for TX and one for RX.
don=$(echo "$i" | egrep -o ":[0-9]{1,99}" | sed ':a;N;$!ba;s/\n//g;s/://g');
# Grabs just the iface name and stores it in the if variable.
iface=`echo "$i" | grep -o "w[a-z]*[0-9]\{1,99\}"`;
# Cretes necessary files if they're not present.
if [[ ! -f ".${iface}" || ! -f ".${iface}counter" ]]; then
echo -e "File does not exit. Creating now.\n";
touch .${iface};
touch .${iface}counter;
fi
# Checks to see if the overruns are greater than zero for each iface in the iteration.
if [[ $don == '00' ]]; then
echo -e "$iface remains clean\n";
else
echo -e "!!!!! $iface is not clean.\n";
echo $don >> ".${iface}"; # Pushes the current overrun numbers into a file.
# Checks if the last line in the respective iface file is greater than the second
# to last. If that's the case, then it will increment the counter inside that ifaces file.
if [[ `cat .${iface} | sed -n '$p'` > `cat .${iface} | sed -n 'x;$p'` ]]; then
echo -e "$iface has an incrementing overrun.\n";
echo $((`cat .${iface}counter`+1)) > .${iface}counter;
# If counter is equal to 5, then reset the respective server.
if [[ `cat .${iface}counter` -eq 5 ]]; then
echo -e "Restarting Server - ${iface}.\n";
echo > .${iface}; # Reset the file and counter after the server has been reset.
echo > .${iface}counter
fi
else
echo -e "Incrementing overruns on $iface has stopped.\n";
fi
fi
done
0) The kernel exposes counters via /proc and, amongst other tools, netstat relies on those. So in /proc/net/dev you have your device name in ARRAY[0] and (IIRC ARRAY[4] and ARRAY[14]) for RX drops and TX drops respectively. Point is you don't need to rely on external binaries like 'netstat' or 'ifconfig', you can read that file as unprivileged user and there's no need to massage the hell out of data using awk / egrep / sed. 1) Biggest no no IMHO is not defining a directory for dumping in data first, now it just ends up in $PWD which can (but shouldn't) be anywhere. 2) Personally I like uppercase variable names as it makes reading easier and I'd use sqlite for storing / querying data but that's prolly just me.
You’ve read that file already? Why not re-use the value? This will also prevent overwriting the counter file twice.
Code:
echo $don >> ".${iface}"; # Pushes the current overrun numbers into a file.
So here’s a thing: You are interested only in current values (stored in ‘$don’) and previous ones (stored in the file). There’s really no need to keep [i]all[/ir of the values in the file. Consider:
Code:
prev=$(cat ".${iface}")
if [ "$don" != "$prev" ]; then
echo "$don" >."${iface}"
counter=$(cat ".${face}counter")
if [ $counter -eq 4 ]; then
echo -e "Restarting Server - ${iface}.\n";
echo >.${iface}
echo >.${iface}counter
else
echo $(($counter + 1)) >.${iface}counter
fi
fi
In fact, I’d keep a single file:
Code:
if ! read prev counter <.${iface}; then
echo "$don 0" >.${iface}
elif [ "$don" != "$prev" ]; then
if [ $counter -eq 4 ]; then
echo -e "Restarting Server - ${iface}.\n";
: >.${iface}
else
echo "$don $(( $counter + 1))" >.${iface}
fi
fi
Personally though, I would transform ifconfig output into something along the lines of:
Code:
<if-name> <rx-overruns> <tx-overruns> 0
and then using ‘join’ merge output from previous and current run into:
From what I can see '.${iface}counter' file contains overrun numbers stored in 'don' variable. After placing all the awk/sed/grep/cat's together, I see for wlan0 the first entry would be 210.
Whilst I am not sure how this is a valid value or number, when I run the left side of the redirect I get the following error:
Code:
echo $((`cat .wlan0counter` + 1))
211
Now assuming it did return output plus 1, I still fail to see how '211' is a useful figure??
I am not understanding how we are getting the 5 rounds here?
[QUOTE=mina86;5306972]I use upper-case for exported environment variable while lower-case for variables local to the script.[\QUOTE]
Agreed. I do the same.
Quote:
Originally Posted by mina86
Anyway, back to OP’s post:
3) Use $(…) instead of backticks.
Is there a specific reason? They behave the same. The only difference is syntactical. It is easier to nest $() than it is to nest ``.
Quote:
Originally Posted by mina86
Code:
cat ifconfig |
Code:
cat .${iface} |
Useless Use of Cat™
It really isn't. If you think I wrote all of this without knowing that I cannot use cat in conjunction with a command like ifconfig, you my friend are either oblivious or ignorant. It's obviously a file. Take a look at my OP and you will see the words test file in bold. It includes output from the ifconfig command. I guess I wouldn't notice the error my script wold throw if this weren't the case.
Quote:
Originally Posted by mina86
Code:
sed -n '$p'
That’s IMO better written as ‘tail -n1’.
It's a matter of choice. There is no real difference here. Only your opinion. Thank you though.
Much safer? In what way? Please explain. In my approach, I am taking the output of my counter file for it's respective iface and overwriting the old counter number in one pass.
Quote:
Originally Posted by mina86
Code:
if [[ `cat .${iface}counter` -eq 5 ]]; then
You’ve read that file already? Why not re-use the value? This will also prevent overwriting the counter file twice.
Separate functions. One increments and one checks to see if the counter has reached 5. I'm not overwritting the file twice. Only once. I guess I could combine the 2 functions in an if loop.
Quote:
Originally Posted by mina86
Code:
echo $don >> ".${iface}"; # Pushes the current overrun numbers into a file.
So here’s a thing: You are interested only in current values (stored in ‘$don’) and previous ones (stored in the file). There’s really no need to keep [i]all[/ir of the values in the file. Consider:
Code:
prev=$(cat ".${iface}")
if [ "$don" != "$prev" ]; then
echo "$don" >."${iface}"
counter=$(cat ".${face}counter")
if [ $counter -eq 4 ]; then
echo -e "Restarting Server - ${iface}.\n";
echo >.${iface}
echo >.${iface}counter
else
echo $(($counter + 1)) >.${iface}counter
fi
fi
In fact, I’d keep a single file:
Code:
if ! read prev counter <.${iface}; then
echo "$don 0" >.${iface}
elif [ "$don" != "$prev" ]; then
if [ $counter -eq 4 ]; then
echo -e "Restarting Server - ${iface}.\n";
: >.${iface}
else
echo "$don $(( $counter + 1))" >.${iface}
fi
fi
Personally though, I would transform ifconfig output into something along the lines of:
Code:
<if-name> <rx-overruns> <tx-overruns> 0
and then using ‘join’ merge output from previous and current run into:
UPDATE: Of course by ‘I would’ I mean ‘if I had to use ifconfig, I would’. In reality, /proc is your friend.
Personally, I think you should not be such a jerk! If you're going to try and rip my code apart, at least be constructive. Explain yourself or give me some tips. Not sure if I am being sensitive but I am a bit offended. I don't believe I am because I wasn't offended when unspawn replied with "constructive" criticism.
From what I can see '.${iface}counter' file contains overrun numbers stored in 'don' variable. After placing all the awk/sed/grep/cat's together, I see for wlan0 the first entry would be 210.
Whilst I am not sure how this is a valid value or number, when I run the left side of the redirect I get the following error:
Code:
echo $((`cat .wlan0counter` + 1))
211
Now assuming it did return output plus 1, I still fail to see how '211' is a useful figure??
I am not understanding how we are getting the 5 rounds here?
Sorry that command is actually a file.
The parsing takes the overruns for it's respective iface and pushes it into a file. If the overruns for the given iface were 0 for both rx and tx, then the result would be 00. If either or were to increment, it would still be incremented with 01 or 10. It grabs the over runs and pushes them into its respective file and compares the last 2 lines and checks if the last value is greater than the previous value. Then acts accordingly.
Off: Imho the problem with shell scripts is that they are easy to write but hard to understand/debug/maintain/reuse. Also they are slow and prone to be platform-dependent. That's why I suggest writing programs in some actual programming language (script or otherwise).
Is there a specific reason? They behave the same. The only difference is syntactical. It is easier to nest $() than it is to nest ``.
Isn’t that reason enough? $(…) is easier to use than backticks so why not use it? [UPDATE: At unSpawn’s request, here’s a whole article about it: Why is $(...) preferred over `...` (backticks)?]
Quote:
Originally Posted by amboxer21
It really isn't.
cat ifconfig | … is equivalent to <ifconfig … and cat .${iface} | … is equivalent to <.$iface … so yes, it is a Useless Use of Cat™.
Quote:
Originally Posted by amboxer21
Much safer? In what way? Please explain. In my approach, I am taking the output of my counter file for it's respective iface and overwriting the old counter number in one pass.
You are relying on the fact that $((`cat .${iface}counter`+1)) will be fully evaluated prior to >.${iface}counter. Maybe ‘much safer’ is exaggeration but your approach to me looks at least suspicious.
Quote:
Originally Posted by amboxer21
Separate functions. One increments and one checks to see if the counter has reached 5. I'm not overwritting the file twice. Only once. I guess I could combine the 2 functions in an if loop.
If counter reaches five, you first execute echo $don >> ".${iface}"; (which is always run) and then echo "$don" >."${iface}" (which zeroes the data once limit is reached). You can simply move appending $don to the file, i.e.:
Code:
echo -e "!!!!! $iface is not clean.\n";
# Checks if the last line in the respective iface file is
# greater than the second to last. If that's the case, then it
# will increment the counter inside that ifaces file.
if [ "$don" != "$(tail -n1 <.${iface})" ]; then
echo -e "$iface has an incrementing overrun.\n";
echo $((`cat .${iface}counter`+1)) > .${iface}counter;
# If counter is equal to 5, then reset the respective
# server.
if [[ `cat .${iface}counter` -eq 5 ]]; then
echo -e "Restarting Server - ${iface}.\n";
# Reset the file and counter after the server
# has been reset.
echo > .${iface}
echo > .${iface}counter
else
# Pushes the current overrun numbers into
# a file.
echo $don >> ".${iface}"
fi
else
echo -e "Incrementing overruns on $iface has stopped.\n"
fi
to save on some I/O and simplify the condition in the first if. The next obvious step is to stop appending to the file, since the history of the overrun values is irrelevant:
Code:
echo -e "!!!!! $iface is not clean.\n";
# Checks if the last line in the respective iface file is
# greater than the second to last. If that's the case, then it
# will increment the counter inside that ifaces file.
if [ "$don" != "$(cat ".${iface")" ]; then
echo -e "$iface has an incrementing overrun.\n";
echo $((`cat .${iface}counter`+1)) > .${iface}counter;
# If counter is equal to 5, then reset the respective
# server.
if [[ `cat .${iface}counter` -eq 5 ]]; then
echo -e "Restarting Server - ${iface}.\n";
# Reset the file and counter after the server
# has been reset.
echo > .${iface}
echo > .${iface}counter
else
# Pushes the current overrun numbers into
# a file.
echo "$don" >.${iface}
fi
else
echo -e "Incrementing overruns on $iface has stopped.\n"
fi
Quote:
Originally Posted by amboxer21
If you're going to try and rip my code apart, at least be constructive.
I gave you code samples of what I believe would be better solution, I don’t know how to be more constructive then that.
-- UPDATE --
Solution with join:
Code:
#!/bin/sh
file=/tmp/watch-for-overruns
lock=/tmp/watch-for-overruns.lock
set -eu
# Are we the only ones?
exec 9>$lock
flock -n 9 || exit 1
# Make sure we have a file to read from, /dev/null will do if $file
# does not exist
in_file=$file
[ -f "$in_file" ] || in_file=/dev/null
temp=$(mktemp "$file.temp.XXXXXXXXXX")
trap 'rm -f -- "$temp"' 0
# gawk produces lines of the following form:
# <interface-name> <rx-overruns> <tx-overruns>
<ifconfig gawk -e '
$0 ~ /^[^ ]/ {
if (iface ~ /^wlan/) print iface, cnt["RX"], cnt["TX"]
iface = $1
cnt["RX"] = cnt["TX"] = 0
}
match($0, /^ *[TR]X .*overruns:([0-9]+)/, arr) {
cnt[$1] = arr[1]
}
END {
if (iface ~ /^wlan/) print iface, cnt["RX"], cnt["TX"]
}
' | sort | join -a 1 -- - "$in_file" | while read iface rx tx prx ptx cnt; do
# For lines successfully joined (an old interface), we get:
# <if-name> <rx> <tx> <previous-rx> <previous-tx> <counter>
# For lines that were only in output of gawk (new interface), we get:
# <if-name> <rx> <tx>
# We ignore lines that were in $in_file only (disappearing interface)
if [ -z "$prx" ]; then
echo "$iface $rx $tx 0"
elif [ "$rx" -le "$prx" ] || [ "$tx" -le "$ptx" ]; then
echo "$iface $rx $tx $cnt"
elif [ "$cnt" -eq 4 ]; then
echo "!!! restart $iface" >&2
else
echo "$iface $rx $tx $(($cnt + 1))"
fi
done >$temp
mv -f -- "$temp" "$file"
One thing I don’t like about it is that it uses gawk, but if that’s a problem, it can be fairly simply fixed by first treating output of ifconfig with sed.
UPDATE: Renamed ‘read’ variable to ‘in_file’ as per grail’s suggestion.
Well it makes more sense now at least that 'ifconfig' is a file. May I suggest in future it might be nice to not call variables and files by well known command names to keep it clearer
For me this would extend mina86's code as well where he uses 'read' for a variable name ... I just think these types of things can make the code a little unclear at first glance (just my opinion of course)
I think mina86 has shown a good way forward, but thought I might just throw in a little bash solution for storing the data from your file or in fact a command:
Code:
w_regex='^(wlan[0-9]+)'
o_regex='overruns:([0-9]+)'
# Put our file / command data in an array for easy access
while read -r line
do
if [[ "$line" =~ $w_regex ]]
then
ifaces+=( ${BASH_REMATCH[1]} )
elif (( ${#ifaces[@]} > 0 )) && \
[[ "${ifaces[${#ifaces[@]}-${#ifaces[@]}%3]}" =~ wlan && \
"$line" =~ $o_regex ]]
then
ifaces+=( ${BASH_REMATCH[1]} )
fi
done<file_name # or you could go with <(command)
It's not rocket science but just a nice little bash alternative to the grep/sed/awk scenario
Having some lengthy experience with seeing others, such as grail, unSpawn, and Mina86 as well as others offer BASH improvements, I'd say they are imminently more qualified than I.
I do agree with their comments both about organization, use of resources and names and style. For instance I would use variables within the script as opposed to creating files, the comment about that the files needing to be hidden made me wonder what you meant until I read what you were doing.
grail's latest post probably is 100% on, but you'll have to test it. Clearly very brief and on the mark. I have little to add insofar as BASH script recommendations.
But, just my personal style, and in conjunction with NevemTeve's comment about using a language. Half on/half off that bandwagon. Yes, I might use C to do this. But also what I'd do if I went down the script option was to segregate the problem and make functions, because you can do that in BASH. I tend to do brute force stuff because I'm not big with sed, awk, or regular expressions, so I use the fundamental stuff and extend mainly when I have to via looking the details up for a complex command.
I've found that you can organize a lengthy, and involved BASH script by using functions and then further, re-use the functions if properly segmented. That's really my point here. A combination of K.I.S.S. along with some re-checking of how you're doing. Try to be simple. Sure some complex problems make for complex solutions, one thing to do is ask for style suggestions as you've done here. The other thing to do is break down your effort in verbiage as well as within code so that you do not end up having a highly involved set of instructions where:
You possibly could look at it in a year and not understand it because it was a one-of which you looked up in order to complete
You can't re-use it either for a similar but different effort because the complex commands are so tightly coupled to a very particular search/replace/action solution
You can't re-use it on a different machine because you chose highly specific command options which aren't always universal from machine to machine.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.