LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 01-13-2013, 09:51 PM   #1
cmartz
LQ Newbie
 
Registered: Apr 2011
Posts: 15

Rep: Reputation: 0
In need of command or script to grep events for last n days


Hello All,

Trying to come up with some command which will grep or egrep (or some other way) the last n days events in a log file. I've already previously grep'd the original file for all the events I'm interested in. Now the final step is just to get the last 7 days events so that I can email the report off. The lines have the following date format:

2013/01/13 15:38:04

So the date will be the variable since obviously this will change every 7 days.

Any help would be much appreciated.

Chris

Last edited by cmartz; 01-13-2013 at 09:55 PM.
 
Old 01-13-2013, 10:31 PM   #2
sag47
Senior Member
 
Registered: Sep 2009
Location: Philly, PA
Distribution: Kubuntu x64, RHEL, Fedora Core, FreeBSD, Windows x64
Posts: 1,429
Blog Entries: 33

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357
Quote:
Originally Posted by cmartz View Post
Hello All,

Trying to come up with some command which will grep or egrep (or some other way) the last n days events in a log file. I've already previously grep'd the original file for all the events I'm interested in. Now the final step is just to get the last 7 days events so that I can email the report off. The lines have the following date format:

2013/01/13 15:38:04

So the date will be the variable since obviously this will change every 7 days.

Any help would be much appreciated.

Chris
Giving a couple of sample lines from the log file would be more useful if you're trying to get comments on a script. Also, what have you tried so far? This isn't something which can simply be done with a single 'grep' command.

One way you could go about doing it is going line by line, extracting the date, and then getting a numerical comparison for the date.

e.g. dategrepscript which I just made up (script mixed with pseudo code)
Code:
#!/bin/bash
#By Sam Gleske
#Created Sun Jan 13 23:34:34 EST 2013
#the following commented line converts the log date format into seconds since 1970-01-01 00:00:00 UTC
#date -d "2013/01/13 15:38:04" +%s

#Calculate seven days ago based on the current time in seconds...
seven_days_ago=$(($(date +%s) - 7*3600*24))

while read line;do
  #extract date from log line
  log_date=$(echo "$line" | awk '{print $1 " " $2}')
  log_date_seconds=$(date -d "$log_date" +%s)
  if [ "$log_date_seconds" -gt "$seven_days_ago" ];then
    echo $line
  fi
done
Calling the script.
Code:
dategrepscript < somelog.log
In a nutshell, that's a rough way for how you *could* do it. I make no guarantees for quality as you did give limited information. In the future, it's best to attempt the script yourself first and see how far you can go. Then ask questions posting your own script.

Is this in a single log file or multiple files? If the logs are rotated daily like they should be then it should be as easy as a find command.
Code:
find . -type f -name '*.log' -mtime -7
I'm curious how this works out for you so failure or not it's appreciated if you report back on your findings.

SAM

Last edited by sag47; 01-13-2013 at 10:40 PM.
 
Old 01-13-2013, 10:55 PM   #3
cmartz
LQ Newbie
 
Registered: Apr 2011
Posts: 15

Original Poster
Rep: Reputation: 0
Wow - that was quick

sag47,

Thanks for the quick response - I haven't tried anything on my own yet nor your script here but I will give it a shot and let you know.

To answer your question: these files are daily files in the form of somefilename.<filedate>.log

So what I'm doing is just grep'ing every few days the string I'm interested in and concatenating to a file like so: grep "string" somefilename.* >> dest.file

From there i'm sorting and getting rid of dups like so: sort -u dest.file > uniq-dest.file

Then from there, I'll try and use your script (thank you) and hope it works. I need to learn how to do script.

Chris
 
Old 01-13-2013, 11:07 PM   #4
sag47
Senior Member
 
Registered: Sep 2009
Location: Philly, PA
Distribution: Kubuntu x64, RHEL, Fedora Core, FreeBSD, Windows x64
Posts: 1,429
Blog Entries: 33

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357
Well if they're in separate files then you could try something like this as well.
Code:
find . -type f -name 'somefilename.*' -mtime -7 -exec grep 'string' {} \;
In English,
Recursively find in the current directory (.) a file of type file (-type f) which matches a name (-name ...) that is younger than the past seven days (-mtime -7). Then when a said file is found it will then execute the grep command on that particular file. In the case of multiple files, it will execute grep individually on each one as it encounters them searching for 'string' (-exec ...). NOTE: Not all versions of find can handle plus/minus seven with mtime (-mtime -7) so YMMV depending on your Unix/Linux flavor and version.

Bash scripting is a good skill to learn and I'd recommend it. I also recommend reading and fully understanding any script (and all of the options of the commands therein) a forum user gives you. While it is not usually intended, hack-a-day scripts can damage a production system if not properly reviewed for errors or malicious code.

SAM

Last edited by sag47; 01-13-2013 at 11:13 PM.
 
Old 01-13-2013, 11:16 PM   #5
cmartz
LQ Newbie
 
Registered: Apr 2011
Posts: 15

Original Poster
Rep: Reputation: 0
thank you again

sag47 - you are a good Samaritan and thank you again. Do you have any good bash reference tutorial you would recommend online or a book ?
 
Old 01-13-2013, 11:51 PM   #6
sag47
Senior Member
 
Registered: Sep 2009
Location: Philly, PA
Distribution: Kubuntu x64, RHEL, Fedora Core, FreeBSD, Windows x64
Posts: 1,429
Blog Entries: 33

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357
Quote:
Originally Posted by cmartz View Post
sag47 - you are a good Samaritan and thank you again. Do you have any good bash reference tutorial you would recommend online or a book ?
Bash Beginners Guide and Advanced Bash-Scripting Guide. Also read the bash man page. Every time I read the man page I learn something new about bash.

A quick tip most text books fail at teaching people... One thing you should note to yourself when checking out scripts and remembering how the "if" conditionals work is that [ is a program (namely /usr/bin/[) which is also called test (man test). /usr/bin/[ outputs a zero if the arguments are evaluated true and a non-zero if the evaluated expression is false.

Since /usr/bin/[ or just [ is a program this is why it requires spaces for the expression because they're all arguments (which is why [5 -gt 4] doesn't work).

e.g.
Code:
/usr/bin/[ 5 -gt 4 ]
[ 5 -gt 4 ]
test 5 -gt 4
All three above examples can be evaluated on the command line and return a zero (echo $? after running the command) upon evaluation. Each can be used with an "if" conditional. Any program that returns a zero upon success and non-zero upon failure can be used as a test for a conditional.

e.g.
Code:
if curl --connect-timeout 1 http://derpserver.com;then
  echo "server exists."
else
  echo "server does not exist or is down!
fi
At any rate, it's time for me to go to bed.

SAM

Last edited by sag47; 01-13-2013 at 11:58 PM.
 
Old 01-14-2013, 06:05 PM   #7
cmartz
LQ Newbie
 
Registered: Apr 2011
Posts: 15

Original Poster
Rep: Reputation: 0
man - you're awfully generous with you time - thanks again for your help.
 
Old 01-15-2013, 01:20 AM   #8
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,261

Rep: Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028
Good bash tutorial http://rute.2038bug.com/index.html.gz
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How do I grep my /var/log/secure file for the past 7 days or so many days? johnmccarthy Linux - Newbie 5 01-04-2013 09:43 PM
[SOLVED] run ps|grep command by script/command line ... ERROR: Unsupported option (BSD syntax) masuch Programming 4 05-23-2012 04:13 AM
generating the keyboard events in shell script anudeepshetty Programming 2 11-24-2010 01:52 AM
how to grep the process running more than 30 days only? ahpin Programming 8 08-24-2009 05:48 AM
Command line program to send X events? drj000 Linux - Software 0 09-07-2005 05:48 PM


All times are GMT -5. The time now is 09:33 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration