-   Linux - Newbie (
-   -   Need directory listing of file names with ranges that contain dates and times (

lcliver 08-29-2012 04:20 PM

Need directory listing of file names with ranges that contain dates and times
I have a directory with the following example file names: they are named based on date/time. exampleYYYYMMDDHHMMSSsyslog.log


I need to get a list of files that are in the range of a user entered START_TIME and END_TIME; for example 20120826184848-20120827153241 (the 26th of August @ x time through the 27th of August @ y time) and I can not figure out how to do this. This directory listing I want to cat into a file so at the end I will have a file (syslog.out) that contains the requested file names.

Does this make sense?

Thank you for any help...I have a feeling this is relatively easy but I am stuck!!! so any help would be greatly appreciated!


casualfred 08-29-2012 05:20 PM

I think this should probably work..
There's most likely a very simple elegant solution, but I didn't think of one. Just set this script in your directory of dated log files, make it executable, and run it.



echo "Usage:"
echo "cd /desired/directory"
echo " [start date] [end date]"
echo ""

# put all the .log files in the current directory in a list
ls -1 *.log > filelist

# find the line number of the highest date in the list
TOPDATELNUM=`grep -n "$1" filelist | cut -f1 -d:`

# find the line number of the lowest date in the list
BOTTOMDATELNUM=`grep -n "$2" filelist | cut -f1 -d:`

# output only files between and including those two dates
sed -n -e $TOPDATELNUM,$BOTTOMDATELNUM"p" filelist > syslog.out

# remove our temporary file
rm filelist

echo "syslog.out created"

Good luck!

byannoni 08-29-2012 05:38 PM

Here is a nice little awk script:

ls -1 | awk -v begin=$1 -v end=$2 '
match( $0, /.*([12][0-9][0-9][0-9][01][0-9][0-3][0-9][0-5][0-9][0-5][0-9][0-5][0-9])syslog\.log/, matches )
        if( matches[1] >= begin && matches[1] <= end ) { print }


cd dir_with_syslogs
sh begin_time end_time > output_file

kakaka 08-29-2012 05:38 PM

casualfred's shell script approach is a clever one. You might need to adjust it a bit in case a file name with the exact date and time the user entered, doesn't exist. If you don't mind including an awk program as part of the solution, and if I understood what you wanted, this appears to work do a comparison, whether or not a file exists with a name containing the exact date/time the User enters.

I put the list of file names you provided in a file named data.txt and the following program in a file named date_list.gawk:


(  NR  ==  1  )  {  start_date = $1 + 0 ;  end_date = $2 + 0 ; }
# File name pattern: example20120826184847syslog.log
(  NR  >  1  )  {  match( $0 ,  /([0-9]+)/ )  ;  file_date = substr( $0 , RSTART , RLENGTH ) + 0 ; if ( ( file_date >= start_date )  &&  ( file_date <= end_date ) )  print $0 ; }

Then when I run this command sequence, to input a date range and the list of file names to the awk program:


( echo 20120826184848 20120827153241 ; cat data.txt ) | gawk -f date_list.gawk
I get this output:



Hope this helps.

chrism01 08-29-2012 06:58 PM

casualfred's soln is a nice shell script one, but for paranoia mode, I'd sort the list

ls -1 *.log |sort > filelist

# or, if you know the files mtimes haven't been tampered with

ls -1t *.log > filelist

# add -r switch if you want the opposite order

lcliver 08-30-2012 08:16 AM

What about this solution
Thank you for the feedback!

I came up with this late last night but it takes a few seconds to run so maybe it is clean on the eye, but not clean running.

for i in `ls *2012082919{5923...5963}* 2>/dev/null`;
echo $i >> syslog.out;

Is there some reason not to do it this way?

lcliver 08-30-2012 08:38 AM

Chose the date_list.gawk
The date_list.gawk works much faster than mine! Thanks!

byannoni 08-30-2012 09:32 AM

Here is one that is completely bash:

for file in *; do
        time=${file:$(expr match $file '[^0-9]*'):14}
        if [[ $time -ge $1 && $time -le $2 ]]; then
                echo $file

Same usage as before:

cd dir_with_syslogs
sh begin_time end_time > output_file

All times are GMT -5. The time now is 01:02 AM.