LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-07-2012, 07:28 AM   #1
sysmicuser
Member
 
Registered: Mar 2010
Posts: 332

Rep: Reputation: 0
Unhappy Need help with parsing log file


Hi Guru's,

I need some assistance from you in regards to parsing a log file.

What my aim or objective is to have a "Report" which would report files which have been added by user and not submitted to repository(Perforce) or any such file which is edited and changes not submitted to repository.

I am not writing here actual script as it is pretty staright forward and command it executes is pretty much p4 diff -f <file_name>...

What I am definitely writing is my grep command of script(set -xv)

Code:
echo "Following files are differing from repository, please submit them to repository"
++ echo 'Following files are differing from repository, please submit them to repository'
Following files are differing from repository, please submit them to repository
grep -E -v "===" -E "file(s) not on client" $LOGS_DIR/p4_opened_files.sh.log > /tmp/$$
++ grep -E -v === -E 'file(s) not on client' test/logs/p4_opened_files.sh.log
grep: file(s) not on client: No such file or directory
mailx -s "Opened files Report" test@testme.com < /tmp/$$
++ mailx -s 'Opened files Report' test@testme.com
Email which I get has following output.
Code:
test/logs/p4_opened_files.sh.log:batch - file(s) not on client.
test/logs/p4_opened_files.sh.log:build_essentials - file(s) not on client.
test/logs/p4_opened_files.sh.log:ccb_batch_restart_worker.sh.20120302 - file(s) not on client.
test/logs/p4_opened_files.sh.log:ccb_spl_ear_undeploy.sh - file(s) not on client.
test/logs/p4_opened_files.sh.log:orig_split_spl_ear.xml - file(s) not on client.
test/logs/p4_opened_files.sh.log:archive - file(s) not on client.
test/logs/p4_opened_files.sh.log:build_essentials - file(s) not on client.
testl/ogs/p4_opened_files.sh.log:check_disk_space.sh_BACKUP - file(s) not on client.
test/logs/p4_opened_files.sh.log:password.txt - file(s) not on client.
test/logs/p4_opened_files.sh.log:549c549
test/logs/p4_opened_files.sh.log:<         ant clean build deploy 2>&1  | tee -a ${IGL_LOGFILE}
test/logs/p4_opened_files.sh.log:---
test/logs/p4_opened_files.sh.log:>         ant -v clean build deploy 2>&1  | tee -a ${IGL_LOGFILE}
test/logs/p4_opened_files.sh.log:560c560
test/logs/p4_opened_files.sh.log:<         ant clean build 2>&1 | tee -a ${IGL_LOGFILE}
test/logs/p4_opened_files.sh.log:---
test/logs/p4_opened_files.sh.log:>         ant -v clean build 2>&1 | tee -a ${IGL_LOGFILE}
test/logs/p4_opened_files.sh.log:* - file(s) not on client.
test/logs/p4_opened_files.sh.log:oas_opmn_start.sh - file(s) not on client.
test/logs/p4_opened_files.sh.log:oas_opmn_stop.sh - file(s) not on client.
test/logs/p4_opened_files.sh.log:cdc_jobs - file(s) not on client.
test/logs/p4_opened_files.sh.log:set_bld02.sh - file(s) not on client.
The actual logfile output is.

Code:
batch - file(s) not on client.
build_essentials - file(s) not on client.
ccb_batch_restart_worker.sh.20120302 - file(s) not on client.
ccb_spl_ear_undeploy.sh - file(s) not on client.
orig_split_spl_ear.xml - file(s) not on client.
archive - file(s) not on client.
build_essentials - file(s) not on client.
check_disk_space.sh_BACKUP - file(s) not on client.
password.txt - file(s) not on client.
==== //glide/main/scm/ahscripts/common/build_and_deploy.sh#23 - test/ahscripts/common/build_and_deploy.sh ====
549c549
<         ant clean build deploy 2>&1  | tee -a ${IGL_LOGFILE}
---
>         ant -v clean build deploy 2>&1  | tee -a ${IGL_LOGFILE}
560c560
<         ant clean build 2>&1 | tee -a ${IGL_LOGFILE}
---
>         ant -v clean build 2>&1 | tee -a ${IGL_LOGFILE}
* - file(s) not on client.
oas_opmn_start.sh - file(s) not on client.
oas_opmn_stop.sh - file(s) not on client.
cdc_jobs - file(s) not on client.
set_bld02.sh - file(s) not on client.
==== //glide/main/scm/ahscripts/tomcat/tomcat_stop.sh#1 - test/ahscripts/tomcat/tomcat_stop.sh ====
The above output(in email) I do care, issue with it is.
1. it should only output file name and not whole path.
for example
Instead of
test/logs/p4_opened_files.sh.logassword.txt - file(s) not on client.
It should say
password.txt

2. Second Issue is I believe more complex to script

Have a look at actual log file output.
Code:
==== //glide/main/scm/ahscripts/common/build_and_deploy.sh#23 - test/ahscripts/common/build_and_deploy.sh ====
549c549
<         ant clean build deploy 2>&1  | tee -a ${IGL_LOGFILE}
---
>         ant -v clean build deploy 2>&1  | tee -a ${IGL_LOGFILE}
560c560
<         ant clean build 2>&1 | tee -a ${IGL_LOGFILE}
---
>         ant -v clean build 2>&1 | tee -a ${IGL_LOGFILE}
* - file(s) not on client.
oas_opmn_start.sh - file(s) not on client.
But in email I get some non-friendly output...

Code:
test/logs/p4_opened_files.sh.log:password.txt - file(s) not on client.
test/logs/p4_opened_files.sh.log:549c549
test/logs/p4_opened_files.sh.log:<         ant clean build deploy 2>&1  | tee -a ${IGL_LOGFILE}
test/logs/p4_opened_files.sh.log:---
test/logs/p4_opened_files.sh.log:>         ant -v clean build deploy 2>&1  | tee -a ${IGL_LOGFILE}
test/logs/p4_opened_files.sh.log:560c560
test/logs/p4_opened_files.sh.log:<         ant clean build 2>&1 | tee -a ${IGL_LOGFILE}
test/logs/p4_opened_files.sh.log:---
test/logs/p4_opened_files.sh.log:>         ant -v clean build 2>&1 | tee -a ${IGL_LOGFILE}
test/logs/p4_opened_files.sh.log:* - file(s) not on client.
Instead it should give only name of file that is build_and_deploy.sh


How can I achieve my objective?

Please assist me here.

Many Thanks.
 
Old 03-08-2012, 07:58 PM   #2
kbp
Senior Member
 
Registered: Aug 2009
Posts: 3,790

Rep: Reputation: 650Reputation: 650Reputation: 650Reputation: 650Reputation: 650Reputation: 650
For the first issue you could just install biabam and send the log file as an attachment instead. The path is being shown because of it's relative location to the current working directory of the script, so you'd need to change dir.

In regard to the second issue, you need to add extra filters to your grep to take out the other stuff you don't want

Code:
grep -v xxx | grep -v yyy | grep -v zzz
BTW .. is Glide going to go live some year soon?

Last edited by kbp; 03-09-2012 at 01:42 AM. Reason: typo
 
Old 03-08-2012, 11:08 PM   #3
sysmicuser
Member
 
Registered: Mar 2010
Posts: 332

Original Poster
Rep: Reputation: 0
hhaha! Glide is already live mate !

Ok, I shall try the technique you described.
 
Old 03-09-2012, 05:15 AM   #4
sysmicuser
Member
 
Registered: Mar 2010
Posts: 332

Original Poster
Rep: Reputation: 0
@kbp

Code:
grep -v xxx | grep -v yyy | grep -v zzz
Instead I performed
Code:
grep -e -v '===' -e "not on client" p4_opened_files.sh.log
but it gives error such as
Code:
grep: ===: No such file or directory
not so cool about it
 
Old 03-09-2012, 08:45 AM   #5
kbp
Senior Member
 
Registered: Aug 2009
Posts: 3,790

Rep: Reputation: 650Reputation: 650Reputation: 650Reputation: 650Reputation: 650Reputation: 650
The pattern needs to be directly after '-e', try this:

Code:
grep -v -e '===' -e "not on client" p4_opened_files.sh.log
 
1 members found this post helpful.
Old 03-10-2012, 06:50 PM   #6
sysmicuser
Member
 
Registered: Mar 2010
Posts: 332

Original Poster
Rep: Reputation: 0
@kbp

Thanks for your assistance.
I am still working on it.

Interesting enough !
Code:
grep -v -e '===' -e "not on client" p4_opened_files.sh.log
doesn't give any output!

The final grep which at the moment is working is something like this.
Code:
grep -h "not on client" $LOG_FILE >> /tmp/$$
The out gives name of files but I am want name of directory where file is located. May be I need to use awk some where.
Code:
test.sh - file(s) not on client.

working and still in cooking phase
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Script for parsing a log file pepepapa82 Linux - Newbie 3 10-04-2011 02:24 AM
Perl: Parsing a log file MKSrivastava Programming 3 11-10-2010 09:59 AM
Parsing log file with awk sebelk Programming 1 08-31-2009 09:47 AM
Parsing a log file jpostma Linux - Newbie 5 12-05-2008 04:58 PM
Help on parsing a log file in BASH globemast Programming 5 01-11-2007 02:56 AM


All times are GMT -5. The time now is 01:01 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration