LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 01-05-2008, 05:11 PM   #1
matyu
LQ Newbie
 
Registered: Jun 2006
Posts: 5

Rep: Reputation: 0
Smile how to use sed to delete mutiple string patterns from apache access log


hi all,

Happy New Year !

I have been asked to remove additional lines from the Apache internet logfiles in an attempt to reduce the amount of non-human traffic (web bots / spiders) that's being analyzed in WebTrends. So, I am trying to write a script to filter out the following string patterns in the
access.log and generate a new output file:

Yahoo! Slurp
help.yahoo.com/help/us/ysearch/slurp
msnbot-media
MSRBOT
Googlebot

My question is how to use Multiple commands in sed to delete the above string patterns from the access.log and pipe it to a new file ?

I am able to delete a specified string pattern, but don't how to delete mutiple .

cat access.log |sed '/Googlebot/d' > newfile

Thanks .
 
Old 01-05-2008, 05:55 PM   #2
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,000
Blog Entries: 11

Rep: Reputation: 893Reputation: 893Reputation: 893Reputation: 893Reputation: 893Reputation: 893Reputation: 893
Code:
sed -e '/Googlebot/d' -e '/MSRBOT/d' -e .... access.log > newfile


Cheers,
Tink
 
Old 01-05-2008, 07:31 PM   #3
matyu
LQ Newbie
 
Registered: Jun 2006
Posts: 5

Original Poster
Rep: Reputation: 0
thanks for your help

thanks for your help

Quote:
Originally Posted by Tinkster View Post
Code:
sed -e '/Googlebot/d' -e '/MSRBOT/d' -e .... access.log > newfile


Cheers,
Tink
 
Old 01-06-2008, 12:42 AM   #4
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
for sed that supports alternation
Code:
# sed -e '/Googlebot\|MSRBOT/d' file
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Shell Script: Delete lines til string found or until particular string. bhargav_crd Linux - General 3 12-21-2007 12:14 AM
Remembering patterns and printing only those patterns using sed bernie82 Programming 5 05-26-2005 06:18 PM
Strange results in /var/log/apache/access.log subt13 Linux - Security 2 08-03-2004 02:21 PM
apache access log mindcry Linux - Security 6 02-12-2003 01:17 PM
Apache Access Log Crashed_Again Linux - Security 2 01-24-2003 03:57 PM


All times are GMT -5. The time now is 12:14 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration