LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-26-2016, 10:58 AM   #1
reginah72
LQ Newbie
 
Registered: Feb 2016
Posts: 16

Rep: Reputation: Disabled
Need help with log files and browser statistics


I am an extremely newbie to Linux. I'm taking a class on it right now but I am hopelessly lost in this project we are doing with log files. Our teacher never said anything within the class on log files so I have been basically looking them up online.

I have to find the number of visitors using MSIE, Firefox broken down by the version. Then I have to get number of people using any other browser. Now I can get it to the field that it's listed in by doing cut -f6 example.log I have tried several other things using grep,tr,cut, uniq, sort, and wc but I still can't get just the count of people by version of the browser. Can anyone please help me?

Here is a few lines of the log files we are using:

192.168.28.168 user143 [08/May/2010:09:52:52] "GET /NoAuth/js/scriptaculous/scriptaculous.js?load=effects,controls HTTP/1.1" "http://www.example.com/index.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0"

192.168.28.168 user147 [08/May/2010:09:52:52] "GET /NoAuth/js/prototype/prototype.js HTTP/1.1" "http://www.example.com/index.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0"

192.168.28.168 user174 [08/May/2010:09:52:52] "GET /NoAuth/js/ahah.js HTTP/1.1" "http://www.example.com/index.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0"

192.168.28.168 user82 [08/May/2010:09:52:52] "GET /NoAuth/js/titlebox-state.js HTTP/1.1" "http://www.example.com/index.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0"

192.168.28.168 user14 [08/May/2010:09:52:52] "GET /NoAuth/css/validation.css HTTP/1.1" "http://www.example.com/index.html" "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0"
 
Old 02-26-2016, 11:08 AM   #2
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 19.1 MATE
Posts: 8,018
Blog Entries: 5

Rep: Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869
The idea with anything like this is to cut it down into steps, starting simple, and then make it increasingly complex.

First of all, ignoring MSIE, how would you determine how many people had used Firefox?
 
Old 02-26-2016, 11:23 AM   #3
reginah72
LQ Newbie
 
Registered: Feb 2016
Posts: 16

Original Poster
Rep: Reputation: Disabled
Well I know I would have to get the field to Firefox first and I'm pretty sure I would use something like cut -d I know the single ' ' but I have no clue what I would put in there. All the symbols confuse me even when I am sitting there with the book opened up to it. So I figure there would probably be 2 cut codes in there one that is to the field cut -f6 and the other cut -d. I started with grep example.log | cut -f6 | cut -d but I have no clue what else to put there besides sort and uniq -c.
 
Old 02-26-2016, 11:32 AM   #4
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 19.1 MATE
Posts: 8,018
Blog Entries: 5

Rep: Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869
Quote:
Originally Posted by reginah72 View Post
Well I know I would have to get the field to Firefox first and I'm pretty sure I would use something like cut -d I know the single ' ' but I have no clue what I would put in there. All the symbols confuse me even when I am sitting there with the book opened up to it. So I figure there would probably be 2 cut codes in there one that is to the field cut -f6 and the other cut -d. I started with grep example.log | cut -f6 | cut -d but I have no clue what else to put there besides sort and uniq -c.
To find out how many people used Firefox, it is a simple grep, or a simple grep piped through a wc. There are no cuts involved.

Can you figure out either of these?

As I said, we will start simple and *then* make things more complex. Always try to break any complex task into simpler steps.
 
1 members found this post helpful.
Old 02-26-2016, 11:38 AM   #5
reginah72
LQ Newbie
 
Registered: Feb 2016
Posts: 16

Original Poster
Rep: Reputation: Disabled
okay so I used grep Firefox example.log | wc -l and it came back with the number 7871
 
Old 02-26-2016, 11:48 AM   #6
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 19.1 MATE
Posts: 8,018
Blog Entries: 5

Rep: Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869
Quote:
Originally Posted by reginah72 View Post
okay so I used grep Firefox example.log | wc -l and it came back with the number 7871
Splendid. The other way of doing it is by using grep -c. It's always useful to have a quick read through the options for the main commands like grep to see if there are any that might prove useful.

Another thing to bear in mind is that there are often many ways to achieve the same result in Linux, due to the strength and versatility of the commands.

Ok. Next. Do a simple grep for Firefox but produce only the word Firefox on each line that is produced (i.e. the search term), not the whole line (hint: look at the grep options).
 
Old 02-26-2016, 12:23 PM   #7
reginah72
LQ Newbie
 
Registered: Feb 2016
Posts: 16

Original Poster
Rep: Reputation: Disabled
Okay finally got that one it's grep -o Firefox example.log
 
Old 02-26-2016, 12:36 PM   #8
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 19.1 MATE
Posts: 8,018
Blog Entries: 5

Rep: Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869
Quote:
Originally Posted by reginah72 View Post
Okay finally got that one it's grep -o Firefox example.log
Great. Ok, now we make it slightly more complex. Instead of just outputting "Firefox" for each grep hit, you want it to output the version too e.g. Firefox/3.7.3.

There may well be an easier way to do it, but using regular expressions in grep is a good way of going about this. In English, instead of just looking for "Firefox", you want to look for Firefox, followed by a forward slash /, followed by a group of characters that can be a number or a dot.

Grep uses regular expressions by default. However it's a better idea to put double quotes around the search term if things get more complicated than a simple alphanumeric expression (it's not necessary in *this particular* case but it is good practice). So,

Code:
grep "Firefox/"
finds Firefox followed by a forward slash. Try it.

All you have to do now is find out how to use a regular expression search for the number/dot group of characters as well.

Last edited by hydrurga; 02-26-2016 at 12:39 PM.
 
1 members found this post helpful.
Old 02-26-2016, 12:49 PM   #9
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 19.1 MATE
Posts: 8,018
Blog Entries: 5

Rep: Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869
A simple but useful tutorial:

https://www.digitalocean.com/communi...terns-in-linux
 
Old 02-26-2016, 12:57 PM   #10
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 19.1 MATE
Posts: 8,018
Blog Entries: 5

Rep: Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869
A simplified cheat sheet of the regexp operators:

http://ryanstutorials.net/linuxtutor...tsheetgrep.php
 
Old 02-26-2016, 01:29 PM   #11
reginah72
LQ Newbie
 
Registered: Feb 2016
Posts: 16

Original Poster
Rep: Reputation: Disabled
got it
code:
grep "Firefox/.*" example.log
or I can still do the grep -o Firefox/.* example.log

now I have to sort it right?

Think I have it : grep "Firefox/.*" example.log | sort | uniq -c

Last edited by reginah72; 02-26-2016 at 01:47 PM.
 
Old 02-26-2016, 01:52 PM   #12
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 19.1 MATE
Posts: 8,018
Blog Entries: 5

Rep: Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869
Quote:
Originally Posted by reginah72 View Post
got it
code:
grep "Firefox/.*" example.log
or I can still do the grep -o Firefox/.* example.log

now I have to sort it right?
Almost there. Have you tried it to see what output it gives?

The regular expression you have used looks for Firefox, followed by a forward slash, followed by 0 or more characters. So, it will output everything until the end of the line. Not what we wanted.

If you want to search for a collection of different characters, but don't care what order they're in, you use square brackets [].

For example, to search for any alphabetical character, lower or upper case, you use [a-zA-Z] (i.e. 2 ranges of characters).

If you want to search for any numerical digit, you use [0-9]

If you want to search for one of a combination of lowercase characters and digits, you use [a-z0-9]

You can also have other characters in there e.g. to search for a % or & symbol or the lowercase f, you can use [%&f] (it doesn't matter what order you put these in the square brackets). You can also mix ranges and individual characters in the same square brackets.

But these only look for *one* character from within the square brackets. To look for 0 or more, you put a * afterwards. so []*

Finally, a dot is one of the special characters in regular expressions, so if you want to look for an actual dot then you must "escape" it by putting a backslash in front of it e.g. \.

You have what you need. Can you come up with the grep command to find Firefox, followed by a forward slash /, followed by a group of characters that can be a number or a dot?
 
Old 02-26-2016, 01:54 PM   #13
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 19.1 MATE
Posts: 8,018
Blog Entries: 5

Rep: Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869
P.S. Excellent job with the

Code:
sort | uniq -c
!!!
 
Old 02-26-2016, 02:20 PM   #14
reginah72
LQ Newbie
 
Registered: Feb 2016
Posts: 16

Original Poster
Rep: Reputation: Disabled
your right some of them have the (net clr and numbers after it) but others have Firefox and the version number.

So I did this one instead grep "Firefox/.[0-9.]* example.log | sort | uniq -c

So the MSIE would be about the same right? answered my own question I guess this one is harder

was that a , after the /? Firefox/,

Last edited by reginah72; 02-26-2016 at 02:25 PM.
 
Old 02-26-2016, 02:25 PM   #15
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 19.1 MATE
Posts: 8,018
Blog Entries: 5

Rep: Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869Reputation: 2869
Quote:
Originally Posted by reginah72 View Post
your right some of them have the (net clr and numbers after it) but others have Firefox and the version number.

So I did this one instead grep "Firefox/.[0-9]* example.log | sort | uniq -c

So the MSIE would be about the same right?
Did you test the output? Can you please tail the last 5 lines of your

Code:
grep "Firefox/.[0-9]* example.log
and cut and paste it here?

To properly develop Linux commands and scripts, you need to be sitting in front of a keyboard and screen, testing what you're developing as you go along to see that it works correctly.

Edit: I see that you've now changed the grep command in your previous post. By all means cut and paste the output from that revised command.

Last edited by hydrurga; 02-26-2016 at 02:29 PM. Reason: Change in details
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
need help with getting browser statistics reginah72 Linux - Newbie 2 02-26-2016 07:30 PM
how to check browser log files on centos 5.3 mohanj.linux Linux - Newbie 3 03-27-2013 03:15 AM
Squid statistics via browser - connection aborted noigan *BSD 3 12-29-2010 10:07 AM
Apache Browser Log Files soulestream Linux - Software 1 12-14-2005 03:59 AM
Are there any browser statistics for LQ? MartinN LQ Suggestions & Feedback 3 09-21-2004 09:34 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 05:19 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration