LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
Reload this Page Grep between certain times
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
Page 1 of 2 1 2
  Search this Thread
Old 03-16-2012, 08:56 AM   #1
mikes88
Member
 
Registered: Jan 2012
Posts: 61

Rep: Reputation: Disabled
Grep between certain times

Hello,

wondering if there was a way to grep my file that contains thousands of lines like this:
Quote:
Thu Feb 02 01:30:44 EST 2012,Thu Feb 02 01:31:04 EST 2012,169
Thu Feb 02 01:31:04 EST 2012,Thu Feb 02 01:31:24 EST 2012,206.4
Thu Feb 02 01:31:24 EST 2012,Thu Feb 02 01:31:44 EST 2012,146.2
Thu Feb 02 01:31:44 EST 2012,Thu Feb 02 01:32:04 EST 2012,134.35
taking out all lines that contain a time before 09:00:00 and after 17:00:00 ??

Tried using grep -v '(08:[0-5][0-9]:*|00:*:*)' but isnt working..
 
Old 03-16-2012, 09:38 AM   #2
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,399
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
Managing, parsing and interpreting times and dates is tricky at the best of times. There are so many formats used, and the formats don't lend themselves well to sorting and other natural ordering. I doubt that a pure grep solution is practical. What other programming languages are you acquainted with?
--- rod.
 
Old 03-16-2012, 09:49 AM   #3
mikes88
Member
 
Registered: Jan 2012
Posts: 61

Original Poster
Rep: Reputation: Disabled
just basic shell, vbscript.
 
Old 03-16-2012, 10:47 AM   #4
makyo
Member
 
Registered: Aug 2006
Location: Saint Paul, MN, USA
Distribution: {Free,Open}BSD, CentOS, Debian, Fedora, Solaris, SuSE
Posts: 735

Rep: Reputation: 76
Hi.

If I understand the request, you want everything inside [09 - 17] hours, which is the other way of saying you want to omit everything outside of those hours.

If either one of the times is outside the bounds, you want to exclude it, correct? As it is now, it looks like all these lines would be excluded.

Please post enough of a sample that shows what should be included and excluded, and what you expect the results to be. Please post that sample inside CODE brackets, not inside QUOTE brackets.

Best wishes ... cheers, makyo
Last edited by makyo; 03-16-2012 at 10:49 AM.
 
Old 03-16-2012, 10:51 AM   #5
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,881

Rep: Reputation: 660Reputation: 660Reputation: 660Reputation: 660Reputation: 660Reputation: 660
Quote:
Originally Posted by mikes88 View Post
... taking out all lines that contain a time before 09:00:00 and after 17:00:00 ??
I think awk can do the job.

As a newbie, my skill with awk is limited. A knowledgeable LQer can tighten up this code.

Input file:
Code:
Thu Feb 02 01:31:24 EST 2012,Thu Feb 02 06:31:44 EST 2012,146.2
Thu Feb 02 01:31:44 EST 2012,Thu Feb 02 07:32:04 EST 2012,134.35 
Thu Feb 02 01:30:44 EST 2012,Thu Feb 02 08:31:04 EST 2012,169
Thu Feb 02 01:31:04 EST 2012,Thu Feb 02 09:31:24 EST 2012,206.4
Thu Feb 02 01:31:24 EST 2012,Thu Feb 02 10:31:44 EST 2012,146.2
Thu Feb 02 01:31:24 EST 2012,Thu Feb 02 11:31:44 EST 2012,146.2
Thu Feb 02 01:31:44 EST 2012,Thu Feb 02 12:32:04 EST 2012,134.35
Thu Feb 02 01:31:24 EST 2012,Thu Feb 02 13:31:44 EST 2012,146.2
Thu Feb 02 01:31:44 EST 2012,Thu Feb 02 14:32:04 EST 2012,134.35
Thu Feb 02 01:31:24 EST 2012,Thu Feb 02 15:31:44 EST 2012,146.2
Thu Feb 02 01:31:44 EST 2012,Thu Feb 02 16:32:04 EST 2012,134.35
Thu Feb 02 01:30:44 EST 2012,Thu Feb 02 17:31:04 EST 2012,169
Thu Feb 02 01:31:04 EST 2012,Thu Feb 02 18:31:24 EST 2012,206.4
Thu Feb 02 01:31:24 EST 2012,Thu Feb 02 19:31:44 EST 2012,146.2
Thu Feb 02 01:31:44 EST 2012,Thu Feb 02 20:32:04 EST 2012,134.35

Code:
Code:
awk '{print substr($9,1,2) " " $0}' $InFile \
|awk '{if ( 8 < $1) {print $0}}' \
|awk '{if (17 > $1) {print $0}}' \
|cut -d' ' -f2-

Output file:
Code:
Thu Feb 02 01:31:04 EST 2012,Thu Feb 02 09:31:24 EST 2012,206.4
Thu Feb 02 01:31:24 EST 2012,Thu Feb 02 10:31:44 EST 2012,146.2
Thu Feb 02 01:31:24 EST 2012,Thu Feb 02 11:31:44 EST 2012,146.2
Thu Feb 02 01:31:44 EST 2012,Thu Feb 02 12:32:04 EST 2012,134.35
Thu Feb 02 01:31:24 EST 2012,Thu Feb 02 13:31:44 EST 2012,146.2
Thu Feb 02 01:31:44 EST 2012,Thu Feb 02 14:32:04 EST 2012,134.35
Thu Feb 02 01:31:24 EST 2012,Thu Feb 02 15:31:44 EST 2012,146.2
Thu Feb 02 01:31:44 EST 2012,Thu Feb 02 16:32:04 EST 2012,134.35
Daniel B. Martin
 
Old 03-16-2012, 02:34 PM   #6
mikes88
Member
 
Registered: Jan 2012
Posts: 61

Original Poster
Rep: Reputation: Disabled
yes i want to keep everything within the timeframe of 09:00:00 -> 17:00:00 anything outside of this get rid of.

Code:
Tue Feb 21 08:55:04 EST 2012,Tue Feb 21 08:55:24 EST 2012,324.7,
Tue Feb 21 08:55:24 EST 2012,Tue Feb 21 08:55:44 EST 2012,356.65,
Tue Feb 21 08:55:44 EST 2012,Tue Feb 21 08:56:04 EST 2012,528.5,
Tue Feb 21 08:56:04 EST 2012,Tue Feb 21 08:56:24 EST 2012,489.2,
Tue Feb 21 08:56:24 EST 2012,Tue Feb 21 08:56:44 EST 2012,471.8,
Tue Feb 21 08:56:44 EST 2012,Tue Feb 21 08:57:04 EST 2012,281.6,
Tue Feb 21 08:57:04 EST 2012,Tue Feb 21 08:57:24 EST 2012,344.45,
Tue Feb 21 08:57:24 EST 2012,Tue Feb 21 08:58:04 EST 2012,430.05,
Tue Feb 21 08:58:04 EST 2012,Tue Feb 21 08:58:24 EST 2012,360.05,
Tue Feb 21 08:58:24 EST 2012,Tue Feb 21 08:58:44 EST 2012,384.75,
Tue Feb 21 08:58:44 EST 2012,Tue Feb 21 08:59:04 EST 2012,446.55,
Tue Feb 21 08:59:04 EST 2012,Tue Feb 21 08:59:24 EST 2012,312.4,
Tue Feb 21 08:59:24 EST 2012,Tue Feb 21 08:59:44 EST 2012,337.5,
Tue Feb 21 08:59:44 EST 2012,Tue Feb 21 09:00:04 EST 2012,379.45,
Tue Feb 21 09:00:04 EST 2012,Tue Feb 21 09:00:24 EST 2012,1182.75,
Tue Feb 21 09:00:24 EST 2012,Tue Feb 21 09:00:44 EST 2012,1193.75,
Tue Feb 21 09:00:44 EST 2012,Tue Feb 21 09:01:04 EST 2012,768.55,
Tue Feb 21 09:01:04 EST 2012,Tue Feb 21 09:01:24 EST 2012,700.55,
Tue Feb 21 09:01:24 EST 2012,Tue Feb 21 09:01:44 EST 2012,814.7,
Tue Feb 21 09:01:44 EST 2012,Tue Feb 21 09:02:04 EST 2012,682.9,
Tue Feb 21 09:02:04 EST 2012,Tue Feb 21 09:02:24 EST 2012,585.3,
Tue Feb 21 09:02:24 EST 2012,Tue Feb 21 09:03:04 EST 2012,506.55,
Tue Feb 21 09:03:04 EST 2012,Tue Feb 21 09:03:24 EST 2012,393.95,
Tue Feb 21 09:03:24 EST 2012,Tue Feb 21 09:03:44 EST 2012,940.1,
Tue Feb 21 09:03:44 EST 2012,Tue Feb 21 09:04:04 EST 2012,994.5,
Tue Feb 21 09:04:04 EST 2012,Tue Feb 21 09:04:24 EST 2012,792.65,
Tue Feb 21 16:56:24 EST 2012,Tue Feb 21 16:56:44 EST 2012,90.65,
Tue Feb 21 16:56:44 EST 2012,Tue Feb 21 16:57:04 EST 2012,110.5,
Tue Feb 21 16:57:04 EST 2012,Tue Feb 21 16:57:44 EST 2012,120.0,
Tue Feb 21 16:57:44 EST 2012,Tue Feb 21 16:58:04 EST 2012,97.3,
Tue Feb 21 16:58:04 EST 2012,Tue Feb 21 16:58:24 EST 2012,116.45,
Tue Feb 21 16:58:24 EST 2012,Tue Feb 21 16:58:44 EST 2012,107.25,
Tue Feb 21 16:58:44 EST 2012,Tue Feb 21 16:59:24 EST 2012,94.45,
Tue Feb 21 16:59:24 EST 2012,Tue Feb 21 16:59:44 EST 2012,115.3,
Tue Feb 21 16:59:44 EST 2012,Tue Feb 21 17:00:04 EST 2012,95.4,
Tue Feb 21 17:00:04 EST 2012,Tue Feb 21 17:00:24 EST 2012,236.4,
Tue Feb 21 17:00:24 EST 2012,Tue Feb 21 17:00:44 EST 2012,104.75,
Tue Feb 21 17:00:44 EST 2012,Tue Feb 21 17:01:24 EST 2012,114.15,
Tue Feb 21 17:01:24 EST 2012,Tue Feb 21 17:02:24 EST 2012,103.75,
Tue Feb 21 17:02:24 EST 2012,Tue Feb 21 17:03:04 EST 2012,91.45,
Tue Feb 21 17:03:04 EST 2012,Tue Feb 21 17:03:24 EST 2012,122.75,
Tue Feb 21 17:03:24 EST 2012,Tue Feb 21 17:03:44 EST 2012,115.95,
This is only a snippet at the 09:00:00 point and 17:00:00 point./ Theres just way to many lines to paste here. Also i need to remove the first section. As you can see it places the time and date twice. The first time and date is really void. im just owrried about the 2nd time and date.
Last edited by mikes88; 03-16-2012 at 02:36 PM.
 
Old 03-16-2012, 03:13 PM   #7
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,399
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
A one-liner Perl solution, if I understand the requirements correctly:
Code:
perl -e 'while(<>){$tod=(split)[8]; ($hr,$min,$sec)=split(/:/,$tod); if( $hr>=9 && $hr<=16){ print "$_";}}' LQmikes88.dat
Assumes the input data file is named 'LQmikes88.dat'.
--- rod.
Last edited by theNbomr; 03-16-2012 at 03:16 PM.
 
Old 03-16-2012, 03:23 PM   #8
mikes88
Member
 
Registered: Jan 2012
Posts: 61

Original Poster
Rep: Reputation: Disabled
Im not familiar with perl. is there a shell code for something like this?
 
Old 03-16-2012, 03:30 PM   #9
mikes88
Member
 
Registered: Jan 2012
Posts: 61

Original Poster
Rep: Reputation: Disabled
That worked like a charm! . Im not usre if you can do this or not but can you combine that line inside a shell script?
 
Old 03-16-2012, 03:41 PM   #10
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,399
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
Certainly. That's what makes it a 'one-liner'. Just enter that line at a commandline, or in a script. It prints the result to standard output, so redirection to a file will capture it for subsequent use.

You can also make it into a proper perl script, if you like:
Code:
#! /usr/bin/ perl -w
#
#  I'll call this LQmikes88.pl
#
use strict;
    while(<>){
        my $tod=(split)[8]; 
        my ($hr,$min,$sec)=split(/:/,$tod); 
        if( $hr>=9 && $hr<=16 ){ 
            print $_;
        }
    }
Then, you would just do
Code:
./LQmikes88.pl <yourDataFile>
(after making it executable, with chmod +x)

It doesn't read as well as it might if it had been born as a full-on script. My one-liners tend to emphasize terseness.

--- rod.
Last edited by theNbomr; 03-16-2012 at 03:43 PM.
 
Old 03-16-2012, 03:42 PM   #11
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405
Hi,

Here's an awk version:
Code:
awk '$9 ~ /(09|1[0-6]):[0-5][0-9]:[0-5][0-9]/ { print }' LQmikes88.dat
 
1 members found this post helpful.
Old 03-16-2012, 03:57 PM   #12
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,399
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
druuna's solution is nice, elegant, and concise. I believe it can be reduced even further, by ignoring the minutes and seconds parts of the time field. It only needs to test the hours component:
Code:
awk '$9 ~ /^(09|1[0-6]):/ { print }' LQmikes88.dat
Hard to imagine anything smaller than that.

--- rod.
Last edited by theNbomr; 03-16-2012 at 04:03 PM. Reason: Added '^' to anchor beginning-of-field
 
1 members found this post helpful.
Old 03-16-2012, 04:01 PM   #13
mikes88
Member
 
Registered: Jan 2012
Posts: 61

Original Poster
Rep: Reputation: Disabled
You guys work wonders!! i appreciate all the help!
 
Old 03-16-2012, 04:01 PM   #14
Karl Godt
Member
 
Registered: Mar 2010
Location: Kiel , Germany
Distribution: once:SuSE6.2,Debian3.1, aurox9.2+3,Mandrake?,DSL? then:W7st,WVHB, #!8.10.02,PUPPY4.3.1 now:Macpup
Posts: 314

Rep: Reputation: 45
Quote:
Tried using grep -v '(08:[0-5][0-9]:*|00:*:*)' but isnt working..
should be " grep -vE " , if your are using the staff"|" as an OR argument .
 
Old 03-16-2012, 04:04 PM   #15
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405Reputation: 2405
Hi,
Quote:
Originally Posted by theNbomr View Post
druuna's solution is nice, elegant, and concise. I believe it can be reduced even further, by ignoring the minutes and seconds parts of the time field. It only needs to test the hours component:
Code:
awk '$9 ~ /(09|1[0-6])/ { print }' LQmikes88.dat
Hard to imagine anything smaller than that.

--- rod.
LOL, your end result is what I started with!

I do believe you need the extra bit. This time: 23:09:16 would otherwise be matched (twice), same would be true for 05:11:13.

So you do need to use this:
Code:
awk '$9 ~ /(09|1[0-6]):[0-5][0-9]:[0-5][0-9]/ { print }' LQmikes88.dat

Just read theNbomr's edit: That is indeed working and even better then mine!
Last edited by druuna; 03-16-2012 at 04:06 PM.
 
  


Reply
Page 1 of 2 1 2



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Grep -p for Linux, Trying to grep a paragraph. ohijames Linux - Newbie 5 07-22-2010 02:09 PM
Trying to understand pipes - Can't pipe output from tail -f to grep then grep again lostjohnny Linux - Newbie 15 03-12-2009 10:31 PM
Data transfer online is slow "at times" or "stops at times" Balarabay1 SUSE / openSUSE 14 04-30-2006 10:00 AM
System hangs; Atheros Madwifi-ping times out every 15/16 times james 456 Linux - Networking 0 01-12-2006 06:55 PM
ps -ef|grep -v root|grep apache<<result maelstrombob Linux - Newbie 1 09-24-2003 11:38 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 01:42 AM.

Contact Us - Advertising Info - Rules - Privacy - LQ Merchandise - Donations - Contributing Member - LQ Sitemap -
Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration