LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 06-19-2013, 03:05 PM   #1
jaa1180
Member
 
Registered: Oct 2003
Location: USA, Tennessee
Distribution: Ubuntu
Posts: 307

Rep: Reputation: 30
Need to read log file line to certain string then grep out 400


On my web server I need to see the HTTP 400 requests. Or other bad requests.

Example: Log entry...

[18/Jun/2013:00:00:28 -0400] "GET /your-overthere/?/12072/order HTTP/1.1" 200 261335 "-" "Mozilla/5.0 (compatible; Blekkobot; ScoutJet; +http://blekko.com/about/blekkobot)"


How can I read up to the HTTP/1.1, then grep based off of the 200, 400, 404 or whatever is after.

I have tried:
egrep "400" access.log | sed 's/^.\{,45\}//' | grep "HTTP"

Of course it is still reading incorrectly.
Ideas?
 
Old 06-19-2013, 03:54 PM   #2
Kustom42
Senior Member
 
Registered: Mar 2012
Distribution: Red Hat
Posts: 1,565

Rep: Reputation: 410Reputation: 410Reputation: 410Reputation: 410Reputation: 410
Ok well if I read your post right you are trying to grep for the "400" or other error codes and then get x amount of info before or after the result?


grep has this built in,

You can do a -A for lines after or -B for lines before. So egrep "400" /var/log/httpd/access.log -B 2 -A 2 would give you the two lines before and after the line it matches.
 
Old 06-19-2013, 04:08 PM   #3
jaa1180
Member
 
Registered: Oct 2003
Location: USA, Tennessee
Distribution: Ubuntu
Posts: 307

Original Poster
Rep: Reputation: 30
Quote:
Originally Posted by Kustom42 View Post
Ok well if I read your post right you are trying to grep for the "400" or other error codes and then get x amount of info before or after the result?


grep has this built in,

You can do a -A for lines after or -B for lines before. So egrep "400" /var/log/httpd/access.log -B 2 -A 2 would give you the two lines before and after the line it matches.
Sort of. I am looking to grep or some other command to output only the lines with the 400 error or whatever error.
So if I could search for [HTTP/1.1" 400] and only show the lines in the log that match that, it would be great. I don't know how to get grep to do that.
 
Old 06-19-2013, 04:31 PM   #4
Kustom42
Senior Member
 
Registered: Mar 2012
Distribution: Red Hat
Posts: 1,565

Rep: Reputation: 410Reputation: 410Reputation: 410Reputation: 410Reputation: 410
Quote:
Originally Posted by jaa1180 View Post
Sort of. I am looking to grep or some other command to output only the lines with the 400 error or whatever error.
So if I could search for [HTTP/1.1" 400] and only show the lines in the log that match that, it would be great. I don't know how to get grep to do that.
Ok so you are probably having issues with double/single quotes interpreting the backslash here.


If you use double quotes, ", / will be interpreted as a backslash and " would also be interpreted. Basically double quotes still allow for bash interpretation and expansion.

If you use single quotes, ', it will treat it as a literal string and ignore special characters. For example, if you did the following:

Code:
#!/bin/bash

MYVARIABLE="Hello World!"

echo "$MYVARIABLE" # This would print the string "Hello World!" to stdout.
echo '$MYVARIABLE' # Would print $MYVARIABLE
 
Old 06-19-2013, 04:46 PM   #5
jaa1180
Member
 
Registered: Oct 2003
Location: USA, Tennessee
Distribution: Ubuntu
Posts: 307

Original Poster
Rep: Reputation: 30
Quote:
Originally Posted by Kustom42 View Post
Ok so you are probably having issues with double/single quotes interpreting the backslash here.


If you use double quotes, ", / will be interpreted as a backslash and " would also be interpreted. Basically double quotes still allow for bash interpretation and expansion.

If you use single quotes, ', it will treat it as a literal string and ignore special characters. For example, if you did the following:

Code:
#!/bin/bash

MYVARIABLE="Hello World!"

echo "$MYVARIABLE" # This would print the string "Hello World!" to stdout.
echo '$MYVARIABLE' # Would print $MYVARIABLE
Okay, forgot about the single quotes. I will see if that does the trick. Thank you sir!
 
Old 06-19-2013, 04:47 PM   #6
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,329

Rep: Reputation: 400Reputation: 400Reputation: 400Reputation: 400Reputation: 400
Quote:
Originally Posted by jaa1180 View Post
Sort of. I am looking to grep or some other command to output only the lines with the 400 error or whatever error.
So if I could search for [HTTP/1.1" 400] and only show the lines in the log that match that, it would be great. I don't know how to get grep to do that.

Code:
grep "HTTP/1\.1\" 4.." access.log
I'm sure I'm missing something
 
1 members found this post helpful.
Old 06-19-2013, 05:38 PM   #7
Kustom42
Senior Member
 
Registered: Mar 2012
Distribution: Red Hat
Posts: 1,565

Rep: Reputation: 410Reputation: 410Reputation: 410Reputation: 410Reputation: 410
Quote:
Originally Posted by Firerat View Post
Code:
grep "HTTP/1\.1\" 4.." access.log
I'm sure I'm missing something
Although this will work it would be nice to explain why. Putting a backslash infront of a special character "escapes" the character. However, in this case there really is no need to do it and

Code:
grep "HTTP/1\.1\" 4.." access.log
and

Code:
grep 'HTTP/1.1" 4' access.log
Would accomplish the same thing.
 
Old 06-19-2013, 05:43 PM   #8
jaa1180
Member
 
Registered: Oct 2003
Location: USA, Tennessee
Distribution: Ubuntu
Posts: 307

Original Poster
Rep: Reputation: 30
Cool, thanks Kustom42.
 
Old 06-19-2013, 06:16 PM   #9
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,329

Rep: Reputation: 400Reputation: 400Reputation: 400Reputation: 400Reputation: 400
true

and
Code:
grep 'HTTP/1\.1" 4[0-9][0-9] ' access.log
would be more exact, not that it makes much difference with this type of log
 
Old 06-21-2013, 08:57 AM   #10
jaa1180
Member
 
Registered: Oct 2003
Location: USA, Tennessee
Distribution: Ubuntu
Posts: 307

Original Poster
Rep: Reputation: 30
Code:
grep 'HTTP/1.1" 404'
That worked. It is getting me the lines I need. Thanks all!!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
read line from a file and check the string rajachan Programming 2 01-18-2013 03:23 AM
[SOLVED] Won't boot: /etc/rc.d/init.d/rc: line 193: /var/log/boot.log: Read-only file system liquidkaleidoscopes Linux From Scratch 6 11-11-2011 08:53 PM
How to grep lines containing a certain string PLUS the line following that line? kmkocot Linux - Newbie 5 09-01-2009 03:54 PM
read files from a folder and grep a string bhagirathi Programming 6 07-06-2009 06:27 AM
linux scripting help needed read from file line by line exc commands each line read atokad Programming 4 12-26-2003 10:24 PM


All times are GMT -5. The time now is 11:50 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration