Shell script to check if a port is in netstat then act on it

kayasaman · 01-06-2012, 02:12 PM

Quote:

Quote:
Originally Posted by kayasaman View Post
Thanks UnSpawn for that but Monit unfortunately takes memory!

Heh, not using any memory wasn't a posted requirement ;-p

LOL

Quote:

Quote:
Originally Posted by kayasaman View Post
I had to disable (..) basically any other non-essential service to make room for Tomcat (..) Bind, NTP, ZFS (..) Squid. (..) file server over NFS (..) FreeBSD Jail'ed environment (..) nearly 2GB of swap used

Once processes start swapping constantly things spin out of control. Java eats RAM like it's 1999, Squid needs RAM for caching (expensive I/O), too many services on that machine IMHO. Unless you're infatuated with Java or require specific Xwiki functionality you can't get otherwise why not try Wiki SW that doesn't hog memory?..

Mine too but with limited resources and as someone who wants to know all about their systems and network kit. My network plan was to have a few servers, maybe 5 - 10 and use them for seperate things; just like you find in the 'real' world. However after buying a few Cisco switches and routers I quickly ran out of finances then lost my job and things went downhill from there..... <sniffle>

Xwiki is good for me as it does what I want and need. I've had a look at things like twiki and run a demo but didn't like the UI or the limited customization of it either. Besides I've started to build up information in Xwiki (UNIX/Linux and Cisco based howto's) which would be annoying to migrate and also got used to the way Xwiki works too which kinda suites my weirdo style of doing things

.

Quote:

Quote:
Originally Posted by kayasaman View Post
Solaris would have been cool but OpenSolaris is DEAD

Ever heard of OpenIndiana?

Yep! I thought that died too after the microsystems part of Sun got replaced by oracle??? I know the last thing that happened was that SXCE and SXDE merged but I didn't know that development was still going on with OSOL spin-off projects (ok I haven't been around that long! I became an OSOL group leader then got made redundant out of that role too after that project died).

Anyway I think I'll change my profession to washing dishes I might be able to save up for that server I want in oh what - like 30-40 years ;-P

kayasaman · 01-07-2012, 09:15 AM

Just to come back here quickly I further adapted my script as due to PATH not being set properly crontab wasn't restarting tomcat:

Code:

#!/usr/local/bin/bash

export PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin
#set -v -x

ntstat=`netstat -ap tcp | grep 8180 | sed -n '1p'`
port="8180"

sock=`sockstat | grep java`
sock_name="java"

#echo $ntstat
#echo $port

if [[ $sock =~ $sock_name ]]; then
   echo "Output of Sockstat command" > /root/java_restart/java_restart.log;
   echo "" >> /root/java_restart/java_restart.log;
   echo "$sock" >> /root/java_restart/java_restart.log;
   echo "" >> /root/java_restart/java_restart.log;
   echo "Socket type $sock_name" >> /root/java_restart/java_restart.log;
   echo "" >> /root/java_restart/java_restart.log;
   echo "Output of Netstat command" >> /root/java_restart/java_restart.log;
   echo "" >> /root/java_restart/java_restart.log;
   echo "$ntstat" >> /root/java_restart/java_restart.log;
   echo "" >> /root/java_restart/java_restart.log;
   echo "Port number is:" >> /root/java_restart/java_restart.log;
   echo "$port" >> /root/java_restart/java_restart.log;
else
   sleep 60; /usr/local/etc/rc.d/tomcat6 restart;
fi

Then run from crontab as so:

Code:

0,30 * * * * /usr/local/bin/bash /root/java_restart/java_restart.sh

Hopefully this will be useful to someone running out of memory or getting major segfaults from services in FreeBSD or any UNIX for that matter

Thanks to EVERYONE for helping me out BTW!!

Cedrik · 01-07-2012, 10:37 AM

Looks good

Maybe you could also add a date field in the java_restart.log lines

(edit)
I just realized you log things when tomcat6 is running ok, but not when it has stoped ?

kayasaman · 01-07-2012, 10:56 AM

Quote:

Looks good Maybe you could also add a date field in the java_restart.log lines

(edit)
I just realized you log things when tomcat6 is running ok, but not when it has stoped ?

Yeah am working in reverse!

Reason being that the log is not a 'real' log but a sudo log just to basically make sure that the scipt is getting run ok!

If one looks closely each time the script is run the log file gets overwritten. It's just a simple check and not meant for debug purposes.

For debug I have included the:

Code:

set -v -x

at the top of the script which will output what the script does to root's mail. That in turn can either get email'ed or whatever but as it is it's fine.

The date portion could be added by doing something like:

Code:

if [[ $sock =~ $sock_name ]]; then
   date > /root/java_restart/java_restart.log
   echo "Output of Sockstat command" >> /root/java_restart/java_restart.log;

with default formatting.

I mean with this setup at least if I don't see that ls -l /root/java_restart/java_restart.log has been updated in a while there's something hinky. Either tomcat had a Java Heap Space error or got killed due to 'out of swap space' error.

Consider this as being a poor mans way to keep dying services alive

Highly un-ideal but having no other choice at present it's all I can do to continue my work in expanding my knowledge and self.

kayasaman · 01-07-2012, 11:13 AM

Hmm....

I guess the whole script could possibly be resiprocated to something like:

Code:

#!/usr/local/bin/bash

export PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/local/sbin
#set -v -x

sock=`sockstat | grep java`
sock_name="java"


if [[ $sock !=~ $sock_name ]]; then
   sleep 60; /usr/local/etc/rc.d/tomcat6 restart;
else
   exit
fi

????

David the H. · 01-07-2012, 07:01 PM

I'd like to make a few comments on the above scripts.

This line in particular:

Code:

ntstat=`netstat -ap tcp | grep 8180 | sed -n '1p'`

1) $(..) is highly recommended over `..`

2) grep and sed|awk should almost never need to be used together. The latter programs both have grep-like matching built in.

In this case, since all you need is the first match, you can just use grep's "-m" option, assuming your version supports it. If not, you can also tell sed to quit after the first match.

Code:

ntstat=$( netstat -ap tcp | grep -m 1 '8180' )

ntstat=$( netstat -ap tcp | sed -n '/8180/{p;q}' )

Here are a few useful sed references.
http://www.grymoire.com/Unix/Sed.html
http://sed.sourceforge.net/grabbag/
http://sed.sourceforge.net/sedfaq.html
http://sed.sourceforge.net/sed1line.txt

Then grail said this above:

Quote:

Lastly, the posix shell does not support regular expression matching, =~, but bash does although you would also need to use double square brackets.

While this is true, don't forget that even in a posix shell you can use a case statement to do globbing matches.

Code:

case $ntstat in

	*$port*) echo "Output of Netstat command $ntstat port number $port" ;;

	*) wait 60; /usr/local/etc/rc.d/tomcat6 restart	;;

esac

Although come to think of it, why can't you just grep directly for the exact "$port" pattern you want in the first place, and skip the complex test entirely? You can just test the variable for a null/non-null string after that, or you could even substitute the grep command directly in place of the test in the if statement.

kayasaman · 01-07-2012, 07:18 PM

Thanks for the tips

Instead of using netstat I decided to go with sockstat but your definition of mixing and matching grep/sed/awk is still valid. Luckily with sockstat I only need to match a simple expression 'java' nothing else and since nothing related to 'java' will ever come up as I'm in a minimual process BSD jail with only Postgresql, Tomcat and the kernel (Buildworld environment) running.

Quote:

Although come to think of it, why can't you just grep directly for the exact "$port" pattern you want in the first place, and skip the complex test entirely? You can just test the variable for a null/non-null string after that, or you could even substitute the grep command directly in place of the test in the if statement.

I should have tested for null/non-null string which probably would have been better practice but as my scripting ability isn't that far down the line I decided to go with what I know and script according to that; as I'd have less chance hosing something later on down the line and could get somewhere without much help as my own work would be considered 'more' valid during this case.

As mentioned I need to get better/more dynamic at scripting but there's also a specific mentality that goes with it rather then doing things the long way which I have unfortunatley done in the past when scripting for other things. It's hard to develop this and needs time and experience. I mean my primary focus is more on networking in terms of Cisco - where majority of my time and effort is being concentrated right now and deploying core infrastructure services on UNIX based OS's rather then say 'long term' admining. I will get there eventually but infront of me I have Cisco CCNP studies and even RHEL certification studies; but it will come as I do enjoy scripting and just wish it would come naturally.

David the H. · 01-07-2012, 07:50 PM

Not to worry. Everyone works at their own level of proficiency. I certainly understand what it's like to be a beginner. A few years ago I was making similar mistakes. But thanks to the helpful folks here and some good web resources, I got better.

So now it's my turn to help others improve.

Keep up the good work!

kayasaman · 01-07-2012, 08:39 PM

Quote:

Not to worry. Everyone works at their own level of proficiency. I certainly understand what it's like to be a beginner. A few years ago I was making similar mistakes. But thanks to the helpful folks here and some good web resources, I got better.

So now it's my turn to help others improve.

Keep up the good work!

Thanks for the vote of confidence

I must admit that LQ is great for a broad range of things, and hekped me a lot while I was learning how to use Linux/UNIX at the beginning. Now I have round 10+ years (non-scripting) Linux experience and 3+ years UNIX (BSD + SOLARIS + Cisco). The real experience I need is in the field as in enterprise but that's the most difficult thing and also depends on who you know and where your located to an extent.

To be honest configuring services is quite easy for me as I've got the most experience with that or learning how to build and implement new ones that I haven't got any or much experience in such as Sun's GlassFish server or Twiki which I checked out the other day. One of the really cool things I did was setup a 3-way site-to-site VPN between my internet connections (I was living and working abroad at the time) to my parents house then used remotely deployed TFTP server to boot Polycom IP phones which would connect to another remote Trixbox server on one hand and the other was to remote boot and hook-up my Sun Ray 1 DTU. That was quite a nice usage of OSPF and multiple VLANs with both tagged and non-tagged traffic.

Ok I'm gona get told off by a moderator now for going off-topic in a tangential fashion so I'll quite rambling

kayasaman · 01-07-2012, 08:44 PM

Oh BTW, Tomcat exited as my system ran out of swap space and my new nicely built automated script just kicked in and restarted.

Now swap is showing up for ~80% usage to just 5%

- my robots.txt file may also have contributed a bit to that too???