selecting specific variables from log file

slayer_1994 · 07-13-2017, 08:19 AM

Hi All

I am trying to create a script which will allow me to tell the last execution of a specific process within the log file.

I created a script previously which would allow me take away the date of the last execution from todays date but what I noticed is every time that file is submitted it will change number.

An example below:

Quote:

7/13/17 02:10:09.318 PM BST [INFO] [TimPollThreadPool.Thread1] [Manager.com.timestock.tess.services.tim.TimIo] File 'Wynyard_MTP_Primary-defect-14999514040000899961.xml' read, length=20417

My script before:

Code:

CURRENTDATE=$(date +%s)
OTHERDATE=$(date -d @1499951404 +%s);
DIFFDATE=$((CURRENTDATE-OTHERDATE));
echo -n "Seconds: "$DIFFDATE" - "
#date -d @$DIFFDATE +%Y%m%d-%H:%M:%S
date -d @$DIFFDATE +%c
exit 0

This would bring back what I wanted but if that specific file is submitted again the "1499951404" would change.

Is their away of my selecting the specific variable of that number so when it changes the script will pick it up and I can see when it was last actioned?

Thanks in advance

Alex

Turbocapitalist · 07-13-2017, 08:28 AM

Maybe some more detail is needed in regards to what data you are trying to extract or track. One problem is that the date is in a non-standard format that is particularly hard to sort and needs extra parsing to do so. Can you fix the logger so that it writes the date in ISO 8601?

Otherwise you could try awk or perl. Scan the log for a pattern and each time it is found save the date-time along with the line it came from. Then at the end of the log, print out the saved date-time and line.

BW-userx · 07-13-2017, 09:26 AM

if the string and pattern itself stays the same and the date string part stays at N digits long then you could do something like this first.

remove the RED part.

Code:

userx%slackwhere ⚡ ~ ⚡> echo $string                                                                                                                                                                                       
7/13/17 02:10:09.318 PM BST [INFO] [TimPollThreadPool.Thread1] [Manager.com.timestock.tess.services.tim.TimIo] File 'Wynyard_MTP_Primary-defect-14999514040000899961.xml' read, length=20417

#chop off left end
userx%slackwhere ⚡ ~ ⚡> newL=${string##*defect-}   
#chop off right end to period                                                                                                                                                                        
userx%slackwhere ⚡ ~ ⚡> newR=${newL%%.*}    
#chop down the number to get date                                                                                                                                                                               
userx%slackwhere ⚡ ~ ⚡> Date=${newR:0:10}
userx%slackwhere ⚡ ~ ⚡> echo $Date                                                                                                                                                                                         
1499951404

I do not see how you're getting that string off the file in this block of code but, I'll let you work that out?
Just add what I did to your code: (if you want to)

Code:

CURRENTDATE=$(date +%s)

#get last date or Other Date
newL=${string##*defect-} 
newR=${newL%%.*} 
Date=${newR:0:10}

OTHERDATE="$Date";

DIFFDATE=$((CURRENTDATE-OTHERDATE));
echo -n "Seconds: "$DIFFDATE" - "
#date -d @$DIFFDATE +%Y%m%d-%H:%M:%S
date -d @$DIFFDATE +%c
exit 0

so now it does not matter if that date changes as long as the string has the key 'pattern' defect- before the date needed and that date keeps the same amount of digits - it will get whatever it is so you can do the math on it.

this worked too using egrep
Line 1:

Code:

7/13/17 02:10:09.318 PM BST [INFO] [TimPollThreadPool.Thread1] [Manager.com.timestock.tess.services.tim.TimIo] File 'Wynyard_MTP_Primary-defect-14999514040000899961.xml' read, length=20417
userx%slackwhere ⚡ ~ ⚡> gerped=$( echo $string | egrep -o [0-9]{12})
userx%slackwhere ⚡ ~ ⚡> echo $gerped
149995140400

but too many 0 zeros?
changing t to 10 got me this
Line 2

Code:

userx%slackwhere ⚡ ~ ⚡> gerped=$( echo $string | egrep -o [0-9]{10})
userx%slackwhere ⚡ ~ ⚡> echo $gerped
1499951404 0000899961

so if using egrep with {12} just chop off the zeros on the right side before using it.

with line 1 do this

Code:

userx%slackwhere ⚡ ~ ⚡> NewGerp=${gerped%*00}
userx%slackwhere ⚡ ~ ⚡> echo "$NewGerp"
1499951404

with line 2: do this,

Code:

userx%slackwhere ⚡ ~ ⚡> gerped=$( echo $string | egrep -o [0-9]{10})
userx%slackwhere ⚡ ~ ⚡> echo $gerped
1499951404 0000899961
userx%slackwhere ⚡ ~ ⚡> NewGerp=${gerped%%* }
userx%slackwhere ⚡ ~ ⚡> echo "$NewGerp"                                                                                                                                                                                    
1499951404
0000899961

well that is just weird. has to be a new line hidden in there?

Fixed it:

Code:

7/13/17 02:10:09.318 PM BST [INFO] [TimPollThreadPool.Thread1] [Manager.com.timestock.tess.services.tim.TimIo] File 'Wynyard_MTP_Primary-defect-14999514040000899961.xml' read, length=20417

#is suppose to get first 10 digits but splits them 
userx%slackwhere ⚡ ~ ⚡> NewDate=$(echo $string | egrep -o [0-9]{10})                                                                                                                                                       
userx%slackwhere ⚡ ~ ⚡> echo $NewDate
1499951404 0000899961
#just strip it again
userx%slackwhere ⚡ ~ ⚡> NewNewGerp=${gerped:0:11}
userx%slackwhere ⚡ ~ ⚡> echo $NewNewGerp
1499951404

used 11 to be sure to get the space out of there as well.

http://tldp.org/LDP/abs/html/string-manipulation.html