LinuxQuestions.org - In need of command or script to grep events for last n days

- Linux - Server (https://www.linuxquestions.org/questions/linux-server-73/)

- - In need of command or script to grep events for last n days (https://www.linuxquestions.org/questions/linux-server-73/in-need-of-command-or-script-to-grep-events-for-last-n-days-4175445477/)

In need of command or script to grep events for last n days

Hello All,

Trying to come up with some command which will grep or egrep (or some other way) the last n days events in a log file. I've already previously grep'd the original file for all the events I'm interested in. Now the final step is just to get the last 7 days events so that I can email the report off. The lines have the following date format:

2013/01/13 15:38:04

So the date will be the variable since obviously this will change every 7 days.

Any help would be much appreciated.

Chris

Quote:

Originally Posted by cmartz (Post 4869389)

Giving a couple of sample lines from the log file would be more useful if you're trying to get comments on a script. Also, what have you tried so far? This isn't something which can simply be done with a single 'grep' command.

One way you could go about doing it is going line by line, extracting the date, and then getting a numerical comparison for the date.

e.g. dategrepscript which I just made up (script mixed with pseudo code)

Code:

#!/bin/bash

#By Sam Gleske

#Created Sun Jan 13 23:34:34 EST 2013

#the following commented line converts the log date format into seconds since 1970-01-01 00:00:00 UTC

#date -d "2013/01/13 15:38:04" +%s



#Calculate seven days ago based on the current time in seconds...

seven_days_ago=$(($(date +%s) - 7*3600*24))



while read line;do

  #extract date from log line

  log_date=$(echo "$line" | awk '{print $1 " " $2}')

  log_date_seconds=$(date -d "$log_date" +%s)

  if [ "$log_date_seconds" -gt "$seven_days_ago" ];then

    echo $line

  fi

done

Calling the script.

Code:

dategrepscript < somelog.log

In a nutshell, that's a rough way for how you *could* do it. I make no guarantees for quality as you did give limited information. In the future, it's best to attempt the script yourself first and see how far you can go. Then ask questions posting your own script.

Is this in a single log file or multiple files? If the logs are rotated daily like they should be then it should be as easy as a find command.

Code:

find . -type f -name '*.log' -mtime -7

I'm curious how this works out for you so failure or not it's appreciated if you report back on your findings.

SAM

Wow - that was quick

sag47,

Thanks for the quick response - I haven't tried anything on my own yet nor your script here but I will give it a shot and let you know.

To answer your question: these files are daily files in the form of somefilename.<filedate>.log

So what I'm doing is just grep'ing every few days the string I'm interested in and concatenating to a file like so: grep "string" somefilename.* >> dest.file

From there i'm sorting and getting rid of dups like so: sort -u dest.file > uniq-dest.file

Then from there, I'll try and use your script (thank you) and hope it works. I need to learn how to do script.

Chris

Well if they're in separate files then you could try something like this as well.

Code:

find . -type f -name 'somefilename.*' -mtime -7 -exec grep 'string' {} \;

In English,
Recursively find in the current directory (.) a file of type file (-type f) which matches a name (-name ...) that is younger than the past seven days (-mtime -7). Then when a said file is found it will then execute the grep command on that particular file. In the case of multiple files, it will execute grep individually on each one as it encounters them searching for 'string' (-exec ...). NOTE: Not all versions of find can handle plus/minus seven with mtime (-mtime -7) so YMMV depending on your Unix/Linux flavor and version.

Bash scripting is a good skill to learn and I'd recommend it. I also recommend reading and fully understanding any script (and all of the options of the commands therein) a forum user gives you. While it is not usually intended, hack-a-day scripts can damage a production system if not properly reviewed for errors or malicious code.

SAM

sag47 - you are a good Samaritan and thank you again. Do you have any good bash reference tutorial you would recommend online or a book ?

Quote:

Originally Posted by cmartz (Post 4869424)

sag47 - you are a good Samaritan and thank you again. Do you have any good bash reference tutorial you would recommend online or a book ?

Bash Beginners Guide and Advanced Bash-Scripting Guide. Also read the bash man page. Every time I read the man page I learn something new about bash.

A quick tip most text books fail at teaching people... One thing you should note to yourself when checking out scripts and remembering how the "if" conditionals work is that [ is a program (namely /usr/bin/[) which is also called test (man test). /usr/bin/[ outputs a zero if the arguments are evaluated true and a non-zero if the evaluated expression is false.

Since /usr/bin/[ or just [ is a program this is why it requires spaces for the expression because they're all arguments (which is why [5 -gt 4] doesn't work).

e.g.

Code:

/usr/bin/[ 5 -gt 4 ]

[ 5 -gt 4 ]

test 5 -gt 4

All three above examples can be evaluated on the command line and return a zero (echo $? after running the command) upon evaluation. Each can be used with an "if" conditional. Any program that returns a zero upon success and non-zero upon failure can be used as a test for a conditional.

e.g.

Code:

if curl --connect-timeout 1 http://derpserver.com;then

  echo "server exists."

else

  echo "server does not exist or is down!

fi

At any rate, it's time for me to go to bed.

SAM

man - you're awfully generous with you time - thanks again for your help.