LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 10-03-2017, 07:50 AM   #1
pedropt
Member
 
Registered: Aug 2014
Distribution: Devuan
Posts: 89

Rep: Reputation: Disabled
[SOLVED] Writing multiples variables on same line


Hi , everyone , i am doing a small script that uses nmap , and i want to write the filtered data on line by line .

Example : this nmap scan : scan.log
Quote:
# Nmap 7.60 scan initiated Tue Oct 3 12:34:51 2017 as: nmap -sV -oN /root/scan/scan.log google.pt
Nmap scan report for google.pt (216.58.211.227)
Host is up (0.020s latency).
Other addresses for google.pt (not scanned): 2a00:1450:4003:804::2003
rDNS record for 216.58.211.227: mad01s24-in-f3.1e100.net
Not shown: 998 filtered ports
PORT STATE SERVICE VERSION
80/tcp open http gws
443/tcp open ssl/https gws
2 services unrecognized despite returning data. If you know the service/version, please submit the following fingerprints at https://nmap.org/cgi-bin/submit.cgi?new-service :

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
# Nmap done at Tue Oct 3 12:36:13 2017 -- 1 IP address (1 host up) scanned in 81.79 seconds
From this log i want to retrieve the ports , the services and version .

for that i use :

Code:
ports=$(cat "scan.log" | grep 'open' | awk -F '\\/tcp' '{print$1}')
popen=$(cat "scan.log" | grep 'open' | awk -F '\\/tcp' '{print$1}' | wc -l)
servn=$(cat "scan.log" | grep 'open' | awk '{print$4}')
serv=$(cat "scan.log" | grep 'open' | awk '{print$3}')
Variables :

ports = to get only the open port without the "/tcp" ahead
popen = to know how many open ports were detected
servn = Service name running at that port
serv = Protocol of that port

now , i want to write each port on sinle line each one like this :
example :

"Open port : 80 Protocol : http Service Running : gws"

to try to do this , i did a loop :

Code:
for i in $(seq 0 "$popen")
do
echo -n "Open port : " "$ports" " Protocol : " "$serv" " Service : " "$servn" >> popen ;
done
However , i am unable to write the output file properly , because i get :
Quote:
Open Port : 80
443 Protocol : http
ssl/https Service : gws
gwsOpen Port : 80
443 Protocol : http
ssl/https Service : gws
gwsOpen Port : 80
443 Protocol : http
ssl/https Service : gws
gws
instead :

Quote:
Open port : 80 Protocol : http Service : gws
Open port : 443 Protocol : ssl/https Service : gws
Any ideas ?

Last edited by pedropt; 10-04-2017 at 12:49 PM.
 
Old 10-03-2017, 07:57 AM   #2
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,907

Rep: Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921
first of all you can simplify it:
Code:
instead of:
cat file | grep pattern | awk 'script'
you can simply write:
awk /pattern/ 'script' file
You need to check I think your variables contain newlines.
You also may need to reorganize your variables (to use arrays instead of single string variables)
 
Old 10-04-2017, 01:13 AM   #3
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,566

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
I agree with pan64 and would add that you could just do the whole lot with awk (unless this is a smaller part of a much bigger script where you want to use the variables again).

pan64 is also on the mark about your variables too. Simply run your first awk:
Code:
cat "scan.log" | grep 'open' | awk -F '\\/tcp' '{print$1}'
And check the output. I would assume from your example the above would return multiple lines, hence you would get all entries stored in each variable and all returned when called.
Arrays would fix this, but I would still recommend the all awk solution first.
 
Old 10-04-2017, 03:06 AM   #4
pedropt
Member
 
Registered: Aug 2014
Distribution: Devuan
Posts: 89

Original Poster
Rep: Reputation: Disabled
Yes i agree with him too , but using it as an array it will be more difficult because of this .

You can use grep to search for the pattern "open" & awk to remove from that pattern what i need , which is "/tcp" , and then output the variable , but doing it like that will not able to write like i want .

Quote:
Open port : 80 Protocol : http Service : gws
I know a way to get over this issue , which is grabbing all the variables one-by one , write only the ports in one temp file , the protocols in other temp file ad the services in other temp file , and then use a loop to read all files line by line and write them in the final output file .

However , i created here the topic with a little faith that this process can be done without ll that extensive process .
 
Old 10-04-2017, 03:42 AM   #5
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,907

Rep: Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921
a single awk can do that easily.
From the other hand storing the ports in an array is:
Code:
ports=( $(awk -F '\\/tcp' '/open/ {print$1}' scan.log) )
# size is:
${#ports[@]}
# do not need to grep/awk again
and you can construct a for loop do go thru
Code:
for port in "${ports[@]}"
do
    echo -n "Open port : " "$port" " Protocol : " "$serv" " Service : " "$servn" >> popen ;
done
 
Old 10-04-2017, 05:57 AM   #6
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,566

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
How about we make it less complicated:
Code:
#!/usr/bin/env bash

while read -r port _ service version
do
  [[ "$port" =~ tcp ]] || continue

  echo "Open port: ${port%/*} Protocol: $service Service: $version"
done< <(nmap <your options here>)

Last edited by grail; 10-04-2017 at 11:24 AM.
 
Old 10-04-2017, 10:41 AM   #7
pedropt
Member
 
Registered: Aug 2014
Distribution: Devuan
Posts: 89

Original Poster
Rep: Reputation: Disabled
Writing multiples variables on same line

I think i have to approach this in another perspective .
The thing is that i retrieve every line to filter with the word "open" , but in an extensive nmap scan , the word "open" may appear in other lines that i dont need .
Example :
Quote:
# Nmap 7.60 scan initiated Wed Oct 4 15:28:58 2017 as: nmap -sV -O -oN /root/scan/scan.log goog.pt
Nmap scan report for goog.pt (185.53.178.6)
Host is up (0.059s latency).
Not shown: 998 filtered ports
PORT STATE SERVICE VERSION
80/tcp open http nginx
443/tcp open ssl/https
1 service unrecognized despite returning data. If you know the service/version, please submit the following fingerprint at https://nmap.org/cgi-bin/submit.cgi?new-service :
SF-Port443-TCP:V=7.60%T=SSL%I=7%D=10/4%Time=59D544A2%P=x86_64-pc-linux-gnu
SF:%r(GetRequest,CF,"HTTP/1\.0\x20302\x20Found\r\nLocation:\x20http://park
......
Warning: OSScan results may be unreliable because we could not find at least 1 open and 1 closed port
Device type: general purpose|specialized
Running (JUST GUESSING): FreeBSD 12.X|9.X|8.X (88%), Crestron 2-Series (85%)
OS CPE: cpe:/o:freebsd:freebsd:12.0 cpe:/o:freebsd:freebsd:9.1 cpe:/o:freebsd:freebsd:8.2 cpe:/o:crestron:2_series
Aggressive OS guesses: FreeBSD 12.0-CURRENT (88%), FreeBSD 9.1-STABLE (85%), FreeBSD 8.2-STABLE (85%), Crestron XPanel control system (85%)
No exact OS matches for host (test conditions non-ideal).

OS and Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
# Nmap done at Wed Oct 4 15:31:03 2017 -- 1 IP address (1 host up) scanned in 125.75 seconds
I will think on how i will deal with this issue .

Last edited by pedropt; 10-04-2017 at 12:48 PM.
 
Old 10-04-2017, 10:50 AM   #8
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,907

Rep: Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921Reputation: 2921
Code:
awk -F/ '$3 == "open" { print $1 }'
 
Old 10-04-2017, 10:51 AM   #9
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian, Arch
Posts: 3,260

Rep: Reputation: 1421Reputation: 1421Reputation: 1421Reputation: 1421Reputation: 1421Reputation: 1421Reputation: 1421Reputation: 1421Reputation: 1421Reputation: 1421
Use -oG instead of -oN.

https://nmap.org/book/man-output.html

Quote:
-oG <filespec> (grepable output)

This output format is covered last because it is deprecated. The XML output format is far more powerful, and is nearly as convenient for experienced users. XML is a standard for which dozens of excellent parsers are available, while grepable output is my own simple hack. XML is extensible to support new Nmap features as they are released, while I often must omit those features from grepable output for lack of a place to put them.

Nevertheless, grepable output is still quite popular. It is a simple format that lists each host on one line and can be trivially searched and parsed with standard Unix tools such as grep, awk, cut, sed, diff, and Perl. Even I usually use it for one-off tests done at the command line. Finding all the hosts with the SSH port open or that are running Solaris takes only a simple grep to identify the hosts, piped to an awk or cut command to print the desired fields.
 
Old 10-04-2017, 11:24 AM   #10
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,566

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
My solution does not worry about the 'open' string but can be easily adapted to add it. Was it not fit for the solution?
 
Old 10-04-2017, 11:29 AM   #11
pedropt
Member
 
Registered: Aug 2014
Distribution: Devuan
Posts: 89

Original Poster
Rep: Reputation: Disabled
I already found a way .
All i need to do is to search exactly for the string i need to remove , which is "/tcp"
that string only appears in the region of log i want to filter .
Using my old code :
Code:
cat scan.log | grep "\/\tcp" | awk -F '\\/tcp' '{print$1,$2,$3,$4}'
This will popup the output :

Quote:
80 open http gws
443 open ssl/https gws
Based on all you guys suggestions , i will find a solution .
 
Old 10-04-2017, 11:50 AM   #12
pedropt
Member
 
Registered: Aug 2014
Distribution: Devuan
Posts: 89

Original Poster
Rep: Reputation: Disabled
The Solution :

Code:
cat scan.log | grep "\/\tcp" | sed 's/\/tcp//g' | awk '{print"Open Port: "$1,"Protocol: ",$3,"Service: ",$4}'
Resuming :
Code:
var=$(cat scan.log | grep "\/\tcp" | sed 's/\/tcp//g' | awk '{print"Open Port: "$1,"Protocol: ",$3,"Service: ",$4}')
echo "$var" >> openp
This will popup this output :

Quote:
Open Port: 80 Protocol: http Service: gws
Open Port: 443 Protocol: ssl/https Service: gws
So , from the beginning for all he others that may need this in nmap :

Code:
#!/bin/bash
path=$(pwd)
rm -f "$path/scan.log" >/dev/null 2>&1
rm -f "$path/popen" >/dev/null 2>&1
echo -n "Enter IP or Hostname to scan : "
read ip
nmap -sV -O "$ip" -oN "$path/scan.log" >/dev/null 2>&1
ports=$(cat scan.log | grep "\/\tcp" | sed 's/\/tcp//g' | awk '{print"Open Port: "$1,"Protocol: ",$3,"Service: ",$4}')
echo "$ports" >> popen
cat popen

Last edited by pedropt; 10-04-2017 at 12:50 PM.
 
Old 10-04-2017, 03:34 PM   #13
MadeInGermany
Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 547

Rep: Reputation: 253Reputation: 253Reputation: 253
awk can even do more. sub() does a substitution and returns true on success, and this can be used as a condition for the following { block }.
Code:
ports=$( < scan.log awk 'sub("/tcp","") {print"Open Port: "$1,"Protocol: ",$3,"Service: ",$4}' )
echo "$ports" >> popen
In one stroke
Code:
awk 'sub("/tcp","") {print"Open Port: "$1,"Protocol: ",$3,"Service: ",$4}' < scan.log >> popen
Last but not least you can run the sub() on $1 only, that is more precise.
Code:
awk 'sub("/tcp","",$1) {print"Open Port: "$1,"Protocol: ",$3,"Service: ",$4}' < scan.log >> popen

Last edited by MadeInGermany; 10-04-2017 at 03:39 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] bash script to get userinput and save it line by line to variables dr.x Linux - Server 7 04-25-2015 08:47 AM
[SOLVED] Writing aliases with strings and variables disciple218 Linux - Newbie 2 11-19-2013 02:01 AM
How to read in line-by-line from a file and store them in several variables ncjlee Programming 1 09-28-2008 07:20 PM
BASH not writing variables to file. snowman81 Programming 7 05-15-2008 05:11 PM
Newbie: Ruby and Writing Variables In Strings lmcilwain Programming 4 10-18-2006 02:19 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 08:00 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration