script to extract IP address from a honeypot log txt file

newtolinux2020 · 04-05-2019, 04:21 PM

grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}" */pentbox/other/log_honeypot.txt > IP.txt

I'm using the above code to extract the IP addresses off pentbox honeypot but it creates the file and nothing is in it. Can anyone help me please?

syg00 · 04-05-2019, 05:12 PM

The regex looks ok - look at the rest of the command. Any messages ?.

scasey · 04-05-2019, 05:15 PM

Quote:

Originally Posted by newtolinux2020

grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}" */pentbox/other/log_honeypot.txt > IP.txt

I'm using the above code to extract the IP addresses off pentbox honeypot but it creates the file and nothing is in it. Can anyone help me please?

The redirect is going to create the output file even if there's nothing to redirect

Please post a few lines...10 or so...of the log_honeypot.txt file.

Also, please post

Code:

ls -l */pentbox/other/log_honeypot.txt

...that leading * could be the problem. What is the actual path to the file?

(tested the grep regex...that works great!) -- so it's not finding the file.
You should be getting

Code:

grep: */pentbox/other/log_honeypot.txt : No such file or directory

on the screen when you run that line of code. Yes?

newtolinux2020 · 04-05-2019, 06:04 PM

thank you for replying.

the full path to the file is /root/pentbox-1.8/other/log_honeypot.txt

the first 10 lines are:

INTRUSION ATTEMPT DETECTED! from 000.00.00.00:46940 (2019-03-24 17:15:56 +0000)
-----------------------------
SSH-2.0-PUTTY

INTRUSION ATTEMPT DETECTED! from 000.00.00.00:40953 (2019-03-24 17:16:39 +0000)
-----------------------------
SSH-2.0-PUTTY

INTRUSION ATTEMPT DETECTED! from 000.00.00.00:61033 (2019-03-24 17:17:37 +0000)
-----------------------------
SSH-2.0-PUTTY

INTRUSION ATTEMPT DETECTED! from 000.00.00.00:22734 (2019-03-24 17:18:35 +0000)
-----------------------------
SSH-2.0-PUTTY

INTRUSION ATTEMPT DETECTED! from 000.00.00.00:56706 (2019-03-24 17:19:36 +0000)
-----------------------------
SSH-2.0-PUTTY

INTRUSION ATTEMPT DETECTED! from 000.00.00.00:35205 (2019-03-24 17:20:35 +0000)
-----------------------------
SSH-2.0-PUTTY

INTRUSION ATTEMPT DETECTED! from 000.00.00.00:22192 (2019-03-24 17:21:37 +0000)
-----------------------------
SSH-2.0-PUTTY

INTRUSION ATTEMPT DETECTED! from 000.00.00.00:33720 (2019-03-24 17:22:36 +0000)
-----------------------------
SSH-2.0-PUTTY

INTRUSION ATTEMPT DETECTED! from 000.00.00.00:42042 (2019-03-24 17:23:05 +0000)
-----------------------------
SSH-2.0-PUTTY

INTRUSION ATTEMPT DETECTED! from 000.00.00.00:43272 (2019-03-24 17:23:20 +0000)
-----------------------------
SSH-2.0-PUTTY

this is what happens when i run the ls -l

ls -l /root/pentbox-1.8/other/log_honeypot.txt
-rw-r--r-- 1 root root 1377443 Apr 5 21:01 /root/pentbox-1.8/other/log_honeypot.txt

newtolinux2020 · 04-05-2019, 06:16 PM

I managed to get it to work, now my next question is, how could i get the script to count duplicate IP addresses and eliminate 172 and 192 addresses within it? also i'm getting alot of 00.00.00.0.0 addresses.

scasey · 04-05-2019, 08:20 PM

Quote:

Originally Posted by newtolinux2020

I managed to get it to work, now my next question is, how could i get the script to count duplicate IP addresses and eliminate 172 and 192 addresses within it? also i'm getting alot of 00.00.00.0.0 addresses.

Please use [code] tags when posting code or output. Thanks!
I found this regexp on the 'net

Code:

(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])

it only matches 'valid' IP addresses, so that should elminate the 00.00.00.00

Pipe your output to

Code:

sort -u

to get a unique list. See man sort. That won't give you a count, tho. Pipe to just sort then iterate the resulting list and count the duplicates. You could also eliminate those beginning with 172 and 192 then, too.

So something like

Code:

grep -E -o (([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]) | \
sort | grep -v ^172 | grep -v ^192 > sortedIP.txt

to get the sorted, cleaned up list.
Then, loop sortedIP.txt to get the counts you want.

syg00 · 04-05-2019, 09:09 PM

@newtolinux2020, you need to understand any solutions you find - use them to learn if you don't. Use it as a base for your own solutions as shown above.
You also need to think about what you care about in your data - better to just toss the zero records before complicating things unnecessarily IMHO. Personally I would use a tool that has the logic to handle the summing as it processes the data - awk, perl, python, ... pick your favourite

newtolinux2020 · 04-05-2019, 09:25 PM

Thank you all very much for the help. I'm still learning all this and you have given me a lot of new ideas so I can make my own script.

newtolinux2020 · 04-06-2019, 12:43 PM

Quote:

Originally Posted by scasey

Please use [code] tags when posting code or output. Thanks!
I found this regexp on the 'net

Code:

(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])

it only matches 'valid' IP addresses, so that should elminate the 00.00.00.00

Pipe your output to

Code:

sort -u

to get a unique list. See man sort. That won't give you a count, tho. Pipe to just sort then iterate the resulting list and count the duplicates. You could also eliminate those beginning with 172 and 192 then, too.

So something like

Code:

grep -E -o (([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5]) | \
sort | grep -v ^172 | grep -v ^192 > sortedIP.txt

to get the sorted, cleaned up list.
Then, loop sortedIP.txt to get the counts you want.

I was wondering if you could tell me where the file location goes in here as I'm just using it for learning and testing.

scasey · 04-06-2019, 01:33 PM

Quote:

Originally Posted by newtolinux2020

I was wondering if you could tell me where the file location goes in here as I'm just using it for learning and testing.

Oops. Typo. It goes after the regexp.

Code:

grep <regexp> <sourcefile> | sort | grep -v <regexp> | grep -v <regexp> > outputfile

See man grep and man sort

newtolinux2020 · 04-06-2019, 03:05 PM

Quote:

Originally Posted by scasey

Oops. Typo. It goes after the regexp.

Code:

grep <regexp> <sourcefile> | sort | grep -v <regexp> | grep -v <regexp> > outputfile

See man grep and man sort

So I should type that and it will sort it? I wont need the ((05) etc next to it.

scasey · 04-06-2019, 03:43 PM

In #6 I left out the name of your input file. Add it after the regexp in that post.
Yes, you still need the regexp.

Code:

grep -E -o (([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4] [0-9]|25[0-5]) /root/pentbox-1.8/other/log_honeypot.txt | sort | grep -v ^172 | grep -v ^192 > sortedIP.txt

newtolinux2020 · 04-06-2019, 04:22 PM

Quote:

Originally Posted by scasey

In #6 I left out the name of your input file. Add it after the regexp in that post.
Yes, you still need the regexp.

Code:

grep -E -o (([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4] [0-9]|25[0-5]) /root/pentbox-1.8/other/log_honeypot.txt | sort | grep -v ^172 | grep -v ^192 > sortedIP.txt

Code:

grep <regexp> /root/pentbox-1.8/other/log_honeypot.txt | sort -u | grep -v <regexp> | grep -v <regexp> > test.txt

keeps giving this error

bash: syntax error near unexpected token `|'

scasey · 04-06-2019, 04:32 PM

Put delimiters around the regexp.

Code:

grep -E -o "(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])" file...etc.

Yes, I'm not being good about my copy/paste, but as syg00 pointed out, you need to understand what's being given you, too.
Sorry.

newtolinux2020 · 04-06-2019, 04:49 PM

I managed to get it working now. It has stopped the duplicate IP addresses and forgets the 192, 178 and 00 addresses.

Thank you so much for helping me learn from this and I'll head on to make it count them up now