LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 08-29-2012, 04:20 PM   #1
lcliver
LQ Newbie
 
Registered: Sep 2009
Posts: 3

Rep: Reputation: 0
Smile Need directory listing of file names with ranges that contain dates and times


I have a directory with the following example file names: they are named based on date/time. exampleYYYYMMDDHHMMSSsyslog.log

example20120826184847syslog.log
example20120826184848syslog.log
example20120826185226syslog.log
example20120826193642syslog.log
example20120827153241syslog.log

I need to get a list of files that are in the range of a user entered START_TIME and END_TIME; for example 20120826184848-20120827153241 (the 26th of August @ x time through the 27th of August @ y time) and I can not figure out how to do this. This directory listing I want to cat into a file so at the end I will have a file (syslog.out) that contains the requested file names.

Does this make sense?

Thank you for any help...I have a feeling this is relatively easy but I am stuck!!! so any help would be greatly appreciated!

Thanks!
 
Old 08-29-2012, 05:20 PM   #2
casualfred
Member
 
Registered: Aug 2012
Location: USA
Distribution: Slackware 14.1
Posts: 73

Rep: Reputation: 15
I think this should probably work..
There's most likely a very simple elegant solution, but I didn't think of one. Just set this script in your directory of dated log files, make it executable, and run it.

Code:
#!/bin/bash

echo "Usage:"
echo "cd /desired/directory"
echo "listdates.sh [start date] [end date]"
echo ""

# put all the .log files in the current directory in a list
ls -1 *.log > filelist

# find the line number of the highest date in the list
TOPDATELNUM=`grep -n "$1" filelist | cut -f1 -d:`

# find the line number of the lowest date in the list
BOTTOMDATELNUM=`grep -n "$2" filelist | cut -f1 -d:`

# output only files between and including those two dates
sed -n -e $TOPDATELNUM,$BOTTOMDATELNUM"p" filelist > syslog.out

# remove our temporary file
rm filelist

echo "syslog.out created"
Good luck!
 
Old 08-29-2012, 05:38 PM   #3
byannoni
Member
 
Registered: Aug 2012
Location: /home/byannoni
Distribution: Arch
Posts: 128

Rep: Reputation: 36
Here is a nice little awk script:
Code:
#!/bin/bash
ls -1 | awk -v begin=$1 -v end=$2 '
match( $0, /.*([12][0-9][0-9][0-9][01][0-9][0-3][0-9][0-5][0-9][0-5][0-9][0-5][0-9])syslog\.log/, matches )
{
	if( matches[1] >= begin && matches[1] <= end ) { print }
}'
Usage:
Code:
cd dir_with_syslogs
sh name.sh begin_time end_time > output_file

Last edited by byannoni; 08-30-2012 at 09:41 AM. Reason: Simplified the script
 
Old 08-29-2012, 05:38 PM   #4
kakaka
Member
 
Registered: Sep 2003
Posts: 382

Rep: Reputation: 86
casualfred's shell script approach is a clever one. You might need to adjust it a bit in case a file name with the exact date and time the user entered, doesn't exist. If you don't mind including an awk program as part of the solution, and if I understood what you wanted, this appears to work do a comparison, whether or not a file exists with a name containing the exact date/time the User enters.

I put the list of file names you provided in a file named data.txt and the following program in a file named date_list.gawk:

Code:
(  NR  ==  1  )  {  start_date = $1 + 0 ;  end_date = $2 + 0 ; }
# File name pattern: example20120826184847syslog.log
(  NR  >  1  )  {  match( $0 ,  /([0-9]+)/ )  ;  file_date = substr( $0 , RSTART , RLENGTH ) + 0 ; if ( ( file_date >= start_date )  &&  ( file_date <= end_date ) )  print $0 ; }
Then when I run this command sequence, to input a date range and the list of file names to the awk program:

Code:
( echo 20120826184848 20120827153241 ; cat data.txt ) | gawk -f date_list.gawk
I get this output:

Code:
example20120826184848syslog.log
example20120826185226syslog.log
example20120826193642syslog.log
example20120827153241syslog.log
Hope this helps.

Last edited by kakaka; 08-29-2012 at 05:41 PM.
 
Old 08-29-2012, 06:58 PM   #5
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,311

Rep: Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040
casualfred's soln is a nice shell script one, but for paranoia mode, I'd sort the list
Code:
ls -1 *.log |sort > filelist

# or, if you know the files mtimes haven't been tampered with

ls -1t *.log > filelist

# add -r switch if you want the opposite order
 
Old 08-30-2012, 08:16 AM   #6
lcliver
LQ Newbie
 
Registered: Sep 2009
Posts: 3

Original Poster
Rep: Reputation: 0
Question What about this solution

Thank you for the feedback!

I came up with this late last night but it takes a few seconds to run so maybe it is clean on the eye, but not clean running.

for i in `ls *2012082919{5923...5963}* 2>/dev/null`;
do
echo $i >> syslog.out;
done

Is there some reason not to do it this way?
 
Old 08-30-2012, 08:38 AM   #7
lcliver
LQ Newbie
 
Registered: Sep 2009
Posts: 3

Original Poster
Rep: Reputation: 0
Smile Chose the date_list.gawk

The date_list.gawk works much faster than mine! Thanks!
 
Old 08-30-2012, 09:32 AM   #8
byannoni
Member
 
Registered: Aug 2012
Location: /home/byannoni
Distribution: Arch
Posts: 128

Rep: Reputation: 36
Here is one that is completely bash:
Code:
#!/bin/bash
for file in *; do
	time=${file:$(expr match $file '[^0-9]*'):14}
	if [[ $time -ge $1 && $time -le $2 ]]; then
		echo $file
	fi
done
Same usage as before:
Code:
cd dir_with_syslogs
sh name.sh begin_time end_time > output_file
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] how to get a file listing including subdirs, with sizes plus fully-qualified names anadem Linux - Newbie 3 05-31-2012 01:49 PM
proftpd server. If connect wirh windows explorer, directory names listing with dates richmondas Linux - Server 1 05-23-2012 02:09 PM
[SOLVED] Listing file names side by side with directory names anon84 Linux - Newbie 3 04-03-2012 11:13 AM
File names listing question Quest ion Programming 4 10-03-2011 08:36 PM
Extracting File Names from Long Listing of Directory in UNIX Hi_This_is_Dev Programming 15 09-01-2010 12:32 AM


All times are GMT -5. The time now is 08:10 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration