[Solved] removing specific pattern from output file

pedropt · 06-23-2017, 12:23 PM

Hi everyone , i am struggling with a difficulty on my code here , this next script will provide the user :
-the current ip
-the conected interface
-the ip range from the router

The script will do a scan over the network for live hosts .

My objective is to remove from final display output :
-gateway ip
-current user ip

from the list .
I was trying with sed but i get no output .

Quote:

#!/bin/bash
router=`route -n | sed -n 3p | awk '{print $2}'`
iprange=`ip route ls | sed -n 2p | awk '{print $1}'`
device=`ip route ls | sed -n 2p | awk '{print $3}'`
ipadd=`ip route ls | sed -n 2p | awk '{print $9}'`
if [ -z $router ]
then
echo "You are not connected to any network"
exit 1
fi

echo "Gateway : $router"
echo "Router Ip Range : $iprange"
echo "Connected Device : $device"
echo "Current Ip address : $ipadd"
echo ""
echo "Scanning for live hosts on $iprange"
echo ""
nmap -n -sn $iprange -oG - > scan
echo "Discovered Live hosts on your network"
cat scan | awk '/Up$/{print $2}'

Any ideas ?

danielbmartin · 06-23-2017, 12:32 PM

Help us to help you. Provide a sample input file (10-15 lines will do). Construct a sample output file which corresponds to your sample input and post both samples here. With "InFile" and "OutFile" examples we can better understand your needs and also judge if our proposed solution fills those needs.

Daniel B. Martin

BW-userx · 06-23-2017, 12:46 PM

this is what I am getting out of your script, maybe because I am using wifi its not getting everything?

Code:

userx%slackwhere ⚡ testing ⚡> sudo ./IP-Stuff
Gateway : 172.31.98.1
Router Ip Range : 127.0.0.0/8
Connected Device : lo
Current Ip address : 

Scanning for live hosts on 127.0.0.0/8

to remove gateway from showing would you just not have it echo it?

found why I am not getting my IP off your script

Code:

userx%slackwhere ⚡ testing ⚡> sudo ip route ls | sed -n 3p | awk '{print $9}'
172.31.99.xx

Changed sed -n 2p to sed -n 3p

I changed all of your sed -n 2p to sed -n 3p

Code:

router=$(route -n | sed -n 3p | awk '{print $2}')
iprange=$(ip route ls | sed -n 3p | awk '{print $1}')
device=$(ip route ls | sed -n 3p | awk '{print $3}')
ipadd=$(ip route ls | sed -n 3p | awk '{print $9}')

now my output is this

Code:

userx%slackwhere ⚡ testing ⚡> sudo ./IP-Stuff                                                                 
Router Ip Range : 172.31.98.0/23
Connected Device : wlan0
Current Ip address : 172.31.99.123

Scanning for live hosts on 172.31.98.0/23

Discovered Live hosts on your network
172.31.98.1
172.31.99.123

what you want to remove from that information?
Because it is all directed by echo command. yes?

I commended out Gateway.

Turbocapitalist · 06-23-2017, 12:54 PM

Here are some adjustments that I'd suggest:

Code:

#!/bin/sh

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin

router=$(route -n     | awk 'NR==3 {print $2}');
iprange=$(ip route ls | awk 'NR==2 {print $1}');
device=$(ip route ls  | awk 'NR==2 {print $3}');
ipadd=$(ip route ls   | awk 'NR==2 {print $9}');

if [ -z $router ]; then
        echo "You are not connected to any network"
        exit 1
fi

echo "Gateway : $router"
echo "Router Ip Range : $iprange"
echo "Connected Device : $device"
echo "Current Ip address : $ipadd"
echo ""
echo "Scanning for live hosts on $iprange"
echo ""

scan=$(tempfile --prefix 'scan.');

nmap -n -sn $iprange -oG - > $scan
echo "Discovered Live hosts on your network"
awk '/Up$/{print $2}' $scan

if [ -f $scan ]; then
        rm $scan;
fi

If running as an unprivileged user, it is necessary to set the $PATH in order to find route.

The command substitution is better as $( ... ) instead of backticks.

sed is not needed if you are using awk.

cat is usually unnecessary.

If you make a temporary file, it is good to be sure it is unique and then clean up afterward.

BW-userx · 06-23-2017, 01:13 PM

Quote:

Originally Posted by Turbocapitalist

The command substitution is better as $( ... ) instead of backticks.

OP
as such with your old code as I did change it.

Code:

router=$(route -n | sed -n 3p | awk '{print $2}')
iprange=$(ip route ls | sed -n 3p | awk '{print $1}')
device=$(ip route ls | sed -n 3p | awk '{print $3}')
ipadd=$(ip route ls | sed -n 3p | awk '{print $9}')

What @Turbocapitalist did makes that script run a LOT faster too - I might add.

Code:

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin

router=$(route -n     | awk 'NR==3 {print $2}');
iprange=$(ip route ls | awk 'NR==3 {print $1}');
device=$(ip route ls  | awk 'NR==3 {print $3}');
ipadd=$(ip route ls   | awk 'NR==3 {print $9}');

Code:

userx%slackwhere ⚡ testing ⚡> ./IP-Stuff
Gateway : 172.31.98.1
Router Ip Range : 172.31.98.0/23
Connected Device : wlan0
Current Ip address : 172.31.99.123

Scanning for live hosts on 172.31.98.0/23

./IP-Stuff: line 35: scan: Permission denied
Discovered Live hosts on your network
172.31.98.1
172.31.99.123

Line 35

Code:

nmap -n -sn $iprange -oG - > scan

pedropt · 06-23-2017, 01:22 PM

Ok , on this next image :

https://s16.postimg.org/87s6hthnp/script.png

i get from output these ips :

1.1.2.1 -> this is my gateway
1.1.2.171 -> other computer in my network
1.1.2.148 -> this computer

My objective is to display only the remote computers and not the gateway and this machine current ip .

I expect at the end only this ip :
1.1.2.171

The way to this is problably using a routine that read each ip from output file (scan) , and if it reads the gateway ip or this computer ip then it wont add it to the end .
The variables i want to get rid at the end of the Script is the $router and $ipadd .
Using sed to search for those variables and remove the lines where those variables are stored could solve the issue before cat display the final output , but i have no idea how to do it .

One of the options i try without success it is on this next link :
https://stackoverflow.com/questions/...pecific-string

which in my case would be applied as this :

Quote:

#!/bin/bash
router=`route -n | sed -n 3p | awk '{print $2}'`
iprange=`ip route ls | sed -n 2p | awk '{print $1}'`
device=`ip route ls | sed -n 2p | awk '{print $3}'`
ipadd=`ip route ls | sed -n 2p | awk '{print $9}'`
if [ -z $router ]
then
echo "You are not connected to any network"
exit 1
fi

echo "Gateway : $router"
echo "Router Ip Range : $iprange"
echo "Connected Device : $device"
echo "Current Ip address : $ipadd"
echo ""
echo "Scanning for live hosts on $iprange"
echo ""
nmap -n -sn $iprange -oG - > scan
echo "Discovered Live hosts on your network"
sed '/$router/d' scan
sed '/$ipadd/d' scan
cat scan | awk '/Up$/{print $2}'

but it does not work properly .

BW-userx · 06-23-2017, 01:49 PM

command using Turbocapitalist method of obtaining scan file in post #4

Code:

awk '/Up$/{print}' $scan

get this

Code:

Host: 172.31.98.1 ()    Status: Up
Host: 172.31.99.123 ()  Status: Up

I do not see any 'key' word to use to get just the one I want. So if it is always returning 2nd IP that you only want to see
your\

Quote:

1.1.2.1 -> this is my gateway
1.1.2.171 -> other computer in my network
1.1.2.148 -> this computer

2nd is other PC

Code:

echo "Scanning for live hosts on $iprange"
echo ""

scan=$(tempfile --prefix 'scan.');

nmap -n -sn $iprange -oG - > $scan

echo "Discovered Live hosts on your network"
awk 'FNR==3 {print $2}' $scan
echo

awk '{print}' $scan

if [ -f $scan ]; then
        rm $scan;
fi

gets me this

Code:

Discovered Live hosts on your network
172.31.99.123

# Nmap 7.12 scan initiated Fri Jun 23 14:04:32 2017 as: nmap -n -sn -oG - 172.31.98.0/23
Host: 172.31.98.1 ()    Status: Up
Host: 172.31.99.123 ()  Status: Up <--- 3rd line column 2
# Nmap done at Fri Jun 23 14:04:36 2017 -- 512 IP addresses (2 hosts up) scanned in 4.01 seconds
userx%slackwhere ⚡ testing ⚡>

pedropt · 06-23-2017, 02:17 PM

to test this script you can not have only your computer connected on the network , because if it is that way then there should not be any output ips at the end the way i want it .
To test this script properly you must have at least one other computer on the same network and output only that remote computer ip .

The objective is simple and i already explained before .

Quote:

Ok , on this next image :

https://s16.postimg.org/87s6hthnp/script.png

i get from output these ips :

1.1.2.1 -> this is my gateway
1.1.2.171 -> other computer in my network
1.1.2.148 -> this computer

My objective is to display only the remote computers and not the gateway and this machine current ip .

I expect at the end only this ip :
1.1.2.171

eventually if no one here is able to figure out what i need then i have to use a different way to approach this issue .

The adjustments to the code you all suggested i will leave it to the end , right now the script is working perfectly as it is .

Turbocapitalist · 06-23-2017, 02:23 PM

The script from #4 above works for me when I test it on a small LAN. It gives me approximately the following output:

Code:

Gateway : 192.168.x.a
Router Ip Range : 192.168.x.0/24
Connected Device : wlan0
Current Ip address : 192.168.x.b

Scanning for live hosts on 192.168.x.0/24

Discovered Live hosts on your network
192.168.x.1
192.168.x.c
192.168.x.d

What else are you looking for?

BW-userx · 06-23-2017, 02:23 PM

Quote:

Originally Posted by pedropt

to test this script you can not have only your computer connected on the network , because if it is that way then there should not be any output ips at the end the way i want it .
To test this script properly you must have at least one other computer on the same network and output only that remote computer ip .

The objective is simple and i already explained before .

eventually if no one here is able to figure out what i need then i have to use a different way to approach this issue .

The adjustments to the code you all suggested i will leave it to the end , right now the script is working perfectly as it is .

if you changed your method to Turbocapitalist method of obtaining scan file in post #4

then look at post #7 in seeing how that scan file has the information inside of it, then see how I figured out to get whatever line is needed to get the proper IP then print only that one and not all of them.

but that last line - is confusing to me , nevertheless I think it is nicer to put it all inside of a file then work off of that ...

pedropt · 06-23-2017, 02:45 PM

The solution :

Quote:

#!/bin/bash
router=`route -n | sed -n 3p | awk '{print $2}'`
iprange=`ip route ls | sed -n 2p | awk '{print $1}'`
device=`ip route ls | sed -n 2p | awk '{print $3}'`
ipadd=`ip route ls | sed -n 2p | awk '{print $9}'`
if [ -z $router ]
then
echo "You are not connected to any network"
exit 1
fi

echo "Gateway : $router"
echo "Router Ip Range : $iprange"
echo "Connected Device : $device"
echo "Current Ip address : $ipadd"
echo ""
echo "Scanning for live hosts on $iprange"
echo ""
rm scan
nmap -n -sn $iprange -oG - > scan
echo "Discovered Live hosts on your network"
ips=`cat scan | awk '/Up$/{print $2}'`
echo $ips > output
rm scan
tr " " "\n" <output >scan
sed "s/\<$router\>//g" < scan > 1
rm scan
sed "s/\<$ipadd\>//g" < 1 > scan
sed -i '/^$/d' scan
result=`cat scan`
if [ -z $result ]; then
echo "No live hosts detected"
else
echo $result
fi

If are only using your computer on your network then it will popup a message that there is no live hosts .

scasey · 06-23-2017, 03:45 PM

OK -- but the there where only three things about your first attempts that were problematic...
1) The single quotes around the sed command prevented the $router and $ipadd from being expanded to their values, so the script was using the literal "$router" as the pattern, not the value in the variable. Solved by using double quotes instead of single quotes. [was very puzzled myself until I remembered to set -x in the script, which showed me what those sed commands were doing]
2) At least on my network,

Code:

sed "/192.168.1.1/d"

also removed 192.168.1.13 and 192.168.1.150. Solved by adding a trailing space to the pattern

Code:

sed "/$router /d"

3) The results of the sed commands weren't being saved, so they had no effect on the final output of the file scan.

Code:

sed "/$router /d" scan > scan1
sed "/$ipadd /d" scan1 > scan2
cat scan2 | awk '/Up$/{print $2}'

saved the results of each sed command, so I finally got:

Code:

# ./test.sh
Gateway : 192.168.1.1
Router Ip Range : 192.168.1.0/24
Connected Device : 192.168.1.1
Current Ip address : 192.168.1.150

Scanning for live hosts on 192.168.1.0/24

Discovered Live hosts on your network
192.168.1.2
192.168.1.3
192.168.1.5
192.168.1.13
192.168.1.112
192.168.1.201

A very useful script! Thank you for sharing!
PS
turbocapital's script in #4 didn't work on my system: don't have tempfile. His and BW-userx's comments are all valid, tho.

pedropt · 06-23-2017, 04:04 PM

Final adjustment to the script for those that may need .
In case no live host detected then will ask if user want to scan again .
And also because i may need to consult this page in future .

Quote:

#!/bin/bash
router=`route -n | sed -n 3p | awk '{print $2}'`
iprange=`ip route ls | sed -n 2p | awk '{print $1}'`
device=`ip route ls | sed -n 2p | awk '{print $3}'`
ipadd=`ip route ls | sed -n 2p | awk '{print $9}'`
if [ -z $router ]
then
echo "You are not connected to any network"
exit 1
fi
lhost () {
echo "Scanning for live hosts on $iprange"
echo ""
rm -rf scan >/dev/null
nmap -n -sn $iprange -oG - > scan
echo "Discovered Live hosts on your network"
ips=`cat scan | awk '/Up$/{print $2}'`
echo $ips > output
rm -rf scan >/dev/null
tr " " "\n" <output >scan
rm -rf output >/dev/null
sed "s/\<$router\>//g" < scan > 1
rm -rf scan >/dev/null
sed "s/\<$ipadd\>//g" < 1 > scan
rm -rf 1 >/dev/null
sed -i '/^$/d' scan
result=`cat scan`
if [ -z $result ]; then
echo "No live hosts detected"
echo ""
echo "Do You want to rescan the network again? (y/n)"
read opt
case $opt in
y|Y|yes|YES|Yes)
clear
lhost
;;
n|N|No|NO)
exit 1
;;
*)
exit 1
;;
esac
else
echo $result
fi
}

echo "Gateway : $router"
echo "Router Ip Range : $iprange"
echo "Connected Device : $device"
echo "Current Ip address : $ipadd"
echo ""
echo "Scanning for live hosts on $iprange"
echo ""
rm -rf scan >/dev/null
nmap -n -sn $iprange -oG - > scan
echo "Discovered Live hosts on your network"
ips=`cat scan | awk '/Up$/{print $2}'`
echo $ips > output
rm -rf scan >/dev/null
tr " " "\n" <output >scan
rm -rf output >/dev/null
sed "s/\<$router\>//g" < scan > 1
rm -rf scan >/dev/null
sed "s/\<$ipadd\>//g" < 1 > scan
rm -rf 1 >/dev/null
sed -i '/^$/d' scan
result=`cat scan`
if [ -z $result ]; then
echo "No live hosts detected"
echo ""
echo "Do You want to rescan the network again? (y/n)"
read opt
case $opt in
y|Y|yes|YES|Yes)
clear
lhost
;;
n|N|No|NO)
exit 1
;;
*)
exit 1
;;
esac
else
echo $result
fi

Turbocapitalist · 06-23-2017, 10:28 PM

Quote:

Originally Posted by scasey

don't have tempfile

Curious. Which distro do you have?

Turbocapitalist · 06-23-2017, 10:30 PM

Quote:

Originally Posted by pedropt

Final adjustment to the script for those that may need .
In case no live host detected then will ask if user want to scan again .
And also because i may need to consult this page in future .

I'd recommend taking a look at the points mentioned at the bottom of post #4 above. Also avoid using -r with rm as -f alone should suffice and -r has the potential for trouble.