LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-13-2017, 08:19 AM   #1
slayer_1994
Member
 
Registered: Feb 2017
Posts: 38

Rep: Reputation: Disabled
selecting specific variables from log file


Hi All

I am trying to create a script which will allow me to tell the last execution of a specific process within the log file.

I created a script previously which would allow me take away the date of the last execution from todays date but what I noticed is every time that file is submitted it will change number.

An example below:
Quote:
7/13/17 02:10:09.318 PM BST [INFO] [TimPollThreadPool.Thread1] [Manager.com.timestock.tess.services.tim.TimIo] File 'Wynyard_MTP_Primary-defect-14999514040000899961.xml' read, length=20417
My script before:
Code:
CURRENTDATE=$(date +%s)
OTHERDATE=$(date -d @1499951404 +%s);
DIFFDATE=$((CURRENTDATE-OTHERDATE));
echo -n "Seconds: "$DIFFDATE" - "
#date -d @$DIFFDATE +%Y%m%d-%H:%M:%S
date -d @$DIFFDATE +%c
exit 0
This would bring back what I wanted but if that specific file is submitted again the "1499951404" would change.

Is their away of my selecting the specific variable of that number so when it changes the script will pick it up and I can see when it was last actioned?

Thanks in advance

Alex
 
Old 07-13-2017, 08:28 AM   #2
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,655
Blog Entries: 3

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
Maybe some more detail is needed in regards to what data you are trying to extract or track. One problem is that the date is in a non-standard format that is particularly hard to sort and needs extra parsing to do so. Can you fix the logger so that it writes the date in ISO 8601?

Otherwise you could try awk or perl. Scan the log for a pattern and each time it is found save the date-time along with the line it came from. Then at the end of the log, print out the saved date-time and line.
 
Old 07-13-2017, 09:26 AM   #3
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (current), FreeBSD, Win10, It varies
Posts: 9,952

Rep: Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148Reputation: 2148
if the string and pattern itself stays the same and the date string part stays at N digits long then you could do something like this first.

remove the RED part.
Code:
userx%slackwhere ⚡ ~ ⚡> echo $string                                                                                                                                                                                       
7/13/17 02:10:09.318 PM BST [INFO] [TimPollThreadPool.Thread1] [Manager.com.timestock.tess.services.tim.TimIo] File 'Wynyard_MTP_Primary-defect-14999514040000899961.xml' read, length=20417

#chop off left end
userx%slackwhere ⚡ ~ ⚡> newL=${string##*defect-}   
#chop off right end to period                                                                                                                                                                        
userx%slackwhere ⚡ ~ ⚡> newR=${newL%%.*}    
#chop down the number to get date                                                                                                                                                                               
userx%slackwhere ⚡ ~ ⚡> Date=${newR:0:10}
userx%slackwhere ⚡ ~ ⚡> echo $Date                                                                                                                                                                                         
1499951404
I do not see how you're getting that string off the file in this block of code but, I'll let you work that out?
Just add what I did to your code: (if you want to)
Code:
CURRENTDATE=$(date +%s)

#get last date or Other Date
newL=${string##*defect-} 
newR=${newL%%.*} 
Date=${newR:0:10}

OTHERDATE="$Date";

DIFFDATE=$((CURRENTDATE-OTHERDATE));
echo -n "Seconds: "$DIFFDATE" - "
#date -d @$DIFFDATE +%Y%m%d-%H:%M:%S
date -d @$DIFFDATE +%c
exit 0
so now it does not matter if that date changes as long as the string has the key 'pattern' defect- before the date needed and that date keeps the same amount of digits - it will get whatever it is so you can do the math on it.

this worked too using egrep
Line 1:
Code:
7/13/17 02:10:09.318 PM BST [INFO] [TimPollThreadPool.Thread1] [Manager.com.timestock.tess.services.tim.TimIo] File 'Wynyard_MTP_Primary-defect-14999514040000899961.xml' read, length=20417
userx%slackwhere ⚡ ~ ⚡> gerped=$( echo $string | egrep -o [0-9]{12})
userx%slackwhere ⚡ ~ ⚡> echo $gerped
149995140400
but too many 0 zeros?
changing t to 10 got me this
Line 2
Code:
userx%slackwhere ⚡ ~ ⚡> gerped=$( echo $string | egrep -o [0-9]{10})
userx%slackwhere ⚡ ~ ⚡> echo $gerped
1499951404 0000899961
so if using egrep with {12} just chop off the zeros on the right side before using it.

with line 1 do this
Code:
userx%slackwhere ⚡ ~ ⚡> NewGerp=${gerped%*00}
userx%slackwhere ⚡ ~ ⚡> echo "$NewGerp"
1499951404
with line 2: do this,
Code:
userx%slackwhere ⚡ ~ ⚡> gerped=$( echo $string | egrep -o [0-9]{10})
userx%slackwhere ⚡ ~ ⚡> echo $gerped
1499951404 0000899961
userx%slackwhere ⚡ ~ ⚡> NewGerp=${gerped%%* }
userx%slackwhere ⚡ ~ ⚡> echo "$NewGerp"                                                                                                                                                                                    
1499951404
0000899961
well that is just weird. has to be a new line hidden in there?

Fixed it:
Code:
7/13/17 02:10:09.318 PM BST [INFO] [TimPollThreadPool.Thread1] [Manager.com.timestock.tess.services.tim.TimIo] File 'Wynyard_MTP_Primary-defect-14999514040000899961.xml' read, length=20417

#is suppose to get first 10 digits but splits them 
userx%slackwhere ⚡ ~ ⚡> NewDate=$(echo $string | egrep -o [0-9]{10})                                                                                                                                                       
userx%slackwhere ⚡ ~ ⚡> echo $NewDate
1499951404 0000899961
#just strip it again
userx%slackwhere ⚡ ~ ⚡> NewNewGerp=${gerped:0:11}
userx%slackwhere ⚡ ~ ⚡> echo $NewNewGerp
1499951404
used 11 to be sure to get the space out of there as well.

http://tldp.org/LDP/abs/html/string-manipulation.html

Last edited by BW-userx; 07-13-2017 at 10:29 AM.
 
  


Reply

Tags
log files, script, shell, time, variable


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] BASH - Log specific output to a log.txt file Rownzy Linux - Software 2 06-17-2015 04:21 AM
Selecting a specific record from a file ultraclassic Linux - Newbie 3 10-02-2013 10:09 AM
how can I monitor a specific log file using zenoss lynos Linux - Software 1 10-10-2010 07:32 PM
Text file manipulation: selecting specific lines/columns using awk and print CHARL0TTE Linux - Newbie 2 02-27-2010 02:40 AM
Monitoring a Specific Port and Exporting to a log file chrisfirestar Linux - General 0 10-27-2003 04:17 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 07:54 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration