Tricky parse logfiles question

zepsweden · 07-22-2015, 12:58 PM

Hi,

I run a chat (which is clean) which outputs an entry when a user connects to the chat. The problem is that i encounter "trolls" where we have people loggin in with 50+ different nicknames with 1 IP-address, and i need to be able to find the ones who is abusing the service.

So i made a simple script with a variable "IP" to search within the log files. Now i want to be able to fetch the unique usernames that has been used to connect with that IP (no duplicates) and do NOT have all the "dates, other info" in the results apart from "username".

The contents of the log file is as follows :

Code:

2015-07-22 12:03:37 AM GMT+2 - Username-- e8f3b769-d2b0-4149-8cad-660b4bcf3b58 -- 79.102.227.163 -- SWEDEN -- connect

What i want to do is :
a) Grep/Select out the ip address 79.102.227.163 and do a "grep search" for that ip address in all log files and pick out the "unique" Username above (between the - "UserName" -) above. I want to return / redirect all usernames into an own file.

b) Have a script for the above where "Ip-Address" is the input variable, like ./findusernames 79.102.227.163 (Which will * wildcard search on all files within the log folder) and then the script does the above in a) and output 1 file with all usernames that has been used to login to the chat with the ip above.

Below is how far i've reached, but i do not know how to use uniq | sort | and fetch the unique usernames in the result which is between the "--" Username "--" in the log file.

Code:

# Search for IP and results show all nicknames logged in with that IP
#!/bin/bash
cd /var/www/sexchatta.nu/ipl0gg
# Search for 
/bin/grep -F -h -i "$1" * >/log/vaff4.html

Dumping some additional log rows to show how it looks like. The below dump can be used to find users logging in with same IP but with different usernames. Below for example "FakeUser1" / "FakeUser2" | "FakeDavid35", "David35" should be returned if you search on their individual IP's.

Code:

2015-07-22 12:25:36 AM GMT+2 - Onanilover -- 72cc07fe-d55c-48c3-b3df-a56a715f2213 -- 12.71.41.177 -- SWEDEN
 -- enter room - Room name: Bild- & filmrummet
2015-07-22 12:25:46 AM GMT+2 - swe_coolio -- b2898a6b-3346-451b-b1bb-e5f0b85be164 -- 83.55.120.57 -- SWED
EN -- quit room - Room name: Rollspelsrummet
2015-07-22 12:25:51 AM GMT+2 - FakeUser1-- 6c3bcfb3-c2a7-4ec5-80ec-be426c8ac564 -- 211.112.166.103 -- SWE
DEN -- connect
2015-07-22 12:25:52 AM GMT+2 - FakeUser2 -- 6c3bcfb3-c2a7-4ec5-80ec-be426c8ac564 -- 211.112.166.103 -- SWE
DEN -- connect
2015-07-22 12:25:51 AM GMT+2 - KilleXCool -- 818584a2-c9dc-4d39-9256-2f27a0367003 -- 211.66.80.162 -- SWEDEN -
- enter room - Room name: Bild- & filmrummet
2015-07-22 12:25:54 AM GMT+2 - David35 -- 6c3bcfb3-c2a7-4ec5-80ec-be426c8ac564 -- 211.112.166.103 -- SWEDE
N -- login
2015-07-22 12:25:57 AM GMT+2 - FakeDavid35 -- 6c3bcfb3-c2a7-4ec5-80ec-be426c8ac564 -- 211.112.166.103 -- SWEDE
N -- enter room - Room name: Entrn
2015-07-22 12:25:57 AM GMT+2 - FakeDavid35 -- 6c3bcfb3-c2a7-4ec5-80ec-be426c8ac564 -- 211.112.166.103 -- SWEDE
N -- login
2015-07-22 12:25:57 AM GMT+2 - FakeDavid35 -- 6c3bcfb3-c2a7-4ec5-80ec-be426c8ac564 -- 211.112.166.103 -- SWEDE
N -- enter room - Room name: Entrn

Big Thanks in advance!

grail · 07-23-2015, 12:29 PM

Firstly, please place code / data in [code][/code] tags to make it more legible and break it up from the question.

As for your script, what have you tried and where are you stuck? We are happy to assist but need to see some effort from you first.

zepsweden · 07-24-2015, 02:08 AM

Quote:

Originally Posted by grail

Firstly, please place code / data in [code][/code] tags to make it more legible and break it up from the question.

As for your script, what have you tried and where are you stuck? We are happy to assist but need to see some effort from you first.

Hi,

Thanks for you feedback, i have now updated my original thread. I am completely stuck and this has terrorized me for quite some time now with my poor combinations of uniq | sort | awk | grep to try to find a solution

NevemTeve · 07-24-2015, 03:45 AM

Please start to learn how to write shell-scripts. (Of course, this could be solved in any programming language.) Here is a start:

Code:

#!/bin/sh

cat testfile | while IFS=' -' read -r D1 D2 D3 D4 D5 D6 USER_ X X X X X IP X; do
    DATE="$D1-$D2-$D3 $D4 $D5 $D6"
    printf 'DATE="%s" USER="%s" IP="%s"\n' "$DATE" "$USER_" "$IP"
    echo 'Now please do something with these variables'
done

output:

Code:

DATE="2015-07-22 12:25:36 AM GMT+2" USER="Onanilover" IP="12.71.41.177"
DATE="2015-07-22 12:25:46 AM GMT+2" USER="swe_coolio" IP="83.55.120.57"
DATE="2015-07-22 12:25:51 AM GMT+2" USER="FakeUser1" IP="211.112.166.103"
DATE="2015-07-22 12:25:52 AM GMT+2" USER="FakeUser2" IP="211.112.166.103"
DATE="2015-07-22 12:25:51 AM GMT+2" USER="KilleXCool" IP="211.66.80.162"
DATE="2015-07-22 12:25:54 AM GMT+2" USER="David35" IP="211.112.166.103"
DATE="2015-07-22 12:25:57 AM GMT+2" USER="FakeDavid35" IP="211.112.166.103"
DATE="2015-07-22 12:25:57 AM GMT+2" USER="FakeDavid35" IP="211.112.166.103"
DATE="2015-07-22 12:25:57 AM GMT+2" USER="FakeDavid35" IP="211.112.166.103"

zepsweden · 07-24-2015, 06:30 AM

Quote:

Originally Posted by NevemTeve

Please start to learn how to write shell-scripts. (Of course, this could be solved in any programming language.) Here is a start:

Code:

#!/bin/sh

cat testfile | while IFS=' -' read -r D1 D2 D3 D4 D5 D6 USER_ X X X X X IP X; do
    DATE="$D1-$D2-$D3 $D4 $D5 $D6"
    printf 'DATE="%s" USER="%s" IP="%s"\n' "$DATE" "$USER_" "$IP"
    echo 'Now please do something with these variables'
done

output:

Code:

DATE="2015-07-22 12:25:36 AM GMT+2" USER="Onanilover" IP="12.71.41.177"
DATE="2015-07-22 12:25:46 AM GMT+2" USER="swe_coolio" IP="83.55.120.57"
DATE="2015-07-22 12:25:51 AM GMT+2" USER="FakeUser1" IP="211.112.166.103"
DATE="2015-07-22 12:25:52 AM GMT+2" USER="FakeUser2" IP="211.112.166.103"
DATE="2015-07-22 12:25:51 AM GMT+2" USER="KilleXCool" IP="211.66.80.162"
DATE="2015-07-22 12:25:54 AM GMT+2" USER="David35" IP="211.112.166.103"
DATE="2015-07-22 12:25:57 AM GMT+2" USER="FakeDavid35" IP="211.112.166.103"
DATE="2015-07-22 12:25:57 AM GMT+2" USER="FakeDavid35" IP="211.112.166.103"
DATE="2015-07-22 12:25:57 AM GMT+2" USER="FakeDavid35" IP="211.112.166.103"

WOW, Thanks! you really now got me thrilled and started to play around further with these variables! Thanks a TON!