LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
LinkBack Search this Thread
Old 07-21-2010, 05:52 AM   #46
GrapefruiTgirl
Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 542Reputation: 542Reputation: 542Reputation: 542Reputation: 542Reputation: 542

Code:
st_time=`expr $((start_hour - 12))`
st_time=$(( start_hour - 12 ))
Either of the above two methods works instead of:
Code:
st_time=`echo expr{$start_hour - 12}`
 
Old 07-21-2010, 12:20 PM   #47
simon.sweetman
Member
 
Registered: Mar 2009
Posts: 32

Rep: Reputation: 22
Sorry real life took me away from this for a while - looks like your going well.

Remember back a couple of pages grail suggested using sed to simplify the input, that was a realy good idea.

I've done up a little bourne shell script that uses this idea. I also defined a couple of internal functions getfield() and timeap() to make the code more readable.

The sed scripts just cut out the required fields and put them in a : seperated list.

msg1 is name:status:end_date:end_h:end_m:end_s
msg is name:status:start_date:start_h:start_m:start_s

After this is pretty easy to extract your required fields (I don't have access to a bourne shell so it's untested):

Code:
#!/bin/sh
dir=/home/input/test.log
type=Report

getfield () {
   # echo Fields from : list
   echo $1 | cut -d: -f$2 | tr ':' ' '
}

timeap () {
    # Convert 24 hour time to AM/PM
    if [ $1 -le 12 ]
    then
        hours=$1
       ap=AM
    else
       hours=$(($1 - 12))
       ap=PM
    fi
    printf "%02d:%02d:%02d %s" $hours $2 $3 $ap
}

msg1=`tail -1 $dir | sed -n -r -e '$ {/successfully/s/(.*) successfully ran on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:success:\3 \2 \5:\4/p;/ERROR/s/.*R: (.*) is Failed on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:failed:\3 \2 \5:\4/p}'`
msg=`tail -3 $dir | head -1 | sed -n -r -e 's/(.*) started at [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:status:\3 \2 \5:\4/p'`

name=`getfield "$msg1" 1`
status=`getfield "$msg1" 2`
logdate=`getfield "$msg1" 3 | sed -r -e 's/^([0-9]) /0\1 /'`
time_end=`getfield "$msg1" 4,5,6`
time_end=`timeap $time_end`
time_start=`getfield "$msg" 4,5,6`
time_start=`timeap $time_start`

echo "$name $logdate $time_start $time_end $type $status"

Last edited by simon.sweetman; 07-21-2010 at 01:02 PM.
 
Old 07-21-2010, 12:44 PM   #48
GrapefruiTgirl
Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 542Reputation: 542Reputation: 542Reputation: 542Reputation: 542Reputation: 542
I just tested the script using Dash shell, but got some errors, as follows:

Code:
sasha@reactor: ./logfilerun
sed: -e expression #1, char 124: unterminated `s' command
[: 37: -le: unexpected operator
printf: 37: PM: expected numeric value
[: 39: -le: unexpected operator
printf: 39: PM: expected numeric value
  -12:00:00  -12:00:00  Report 
sasha@reactor:
Maybe this could be caused by my input file, which I made to contain the following stuff from Post #43 above:

Code:
report_name started at Fri Jul 9 00:01:00 SST 2010.
PL/SQL procedure successfully completed.
report_name successfully ran on Fri Jul 19 15:13:00 SST 2010.
I started messing with the sed stuff, but thought better of that because I don't know what the input file is exactly supposed to look like.

If OP can post a real sample of what the input file should contain, I can test again in a bit. Also, please verify that dir=/home/input/test.log is actually supposed to point directly at the logfile, and not a directory, right?


EDIT:
I have changed the input file to contain only:
Code:
report_name successfully ran on Tue Jul 11 18:00:01 SST 2010.
and now the results on screen are:
Code:
sasha@reactor: ./logfilerun
sed: -e expression #1, char 124: unterminated `s' command
[: 31: -le: unexpected operator
printf: 31: PM: expected numeric value
  00:01:00 AM -12:00:00  Report 
sasha@reactor:
So a little better - but still something amiss.

Last edited by GrapefruiTgirl; 07-21-2010 at 01:07 PM. Reason: corrected to default logfile location
 
Old 07-21-2010, 01:31 PM   #49
GrapefruiTgirl
Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 542Reputation: 542Reputation: 542Reputation: 542Reputation: 542Reputation: 542
Code:
#!/bin/sh
dir=/home/sasha/logfile
type=Report

getfield () {
   # echo Fields from : list
   echo $1 | cut -d: -f $2 | tr ':' ' '
}

timeap () {
    # Convert 24 hour time to AM/PM
    if [ $1 -le 12 ]
    then
        hours=$1
       ap='AM'
    else
       hours=$(( ${1} - 12))
       ap='PM'
    fi
    printf "%02d:%02d:%02d %s" $hours $2 $3 $ap

}

msg1=`tail -1 $dir | sed -n -r -e '$ {/successfully/s/(.*) successfully ran on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:success:\3 \2 \5:\4/p;
                                             /ERROR/s/(.*)R: (.*) is Failed on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:failed:\3 \2 \5:\4/p}'`
msg=`tail -3 $dir | head -1 | sed -n -r -e 's/(.*) started at [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:status:\3 \2 \5:\4/p'`

name=`getfield "$msg1" 1`
status=`getfield "$msg1" 2`
logdate=`getfield "$msg1" 3 | sed -r -e 's/^([0-9]) /0\1 /'`
time_end=`getfield "$msg1" 4,5,6`
time_end=`timeap $time_end`
time_start=`getfield "$msg" 4,5,6`
time_start=`timeap $time_start`

echo "$name $logdate $time_start $time_end $type $status"
Back to 3 lines in the input file (after I noticed the `head -3` ) and now, with some small edits, it works. I dunno if it works "as it should", but it works error free in Dash shell. Here's what it outputs:
Code:
sasha@reactor: ./logfilerun
report_name 19 Jul 2010 00:01:00 AM 03:13:00 PM Report success
sasha@reactor:

Last edited by GrapefruiTgirl; 07-21-2010 at 01:35 PM. Reason: Show output..
 
Old 07-21-2010, 06:51 PM   #50
simon.sweetman
Member
 
Registered: Mar 2009
Posts: 32

Rep: Reputation: 22
Nice work sasha, like the reformatted sed line makes it much easier to understand the two lines processed. I've learnt something new as well, as I never used dash before, looks like a nice simple/fast shell.
 
Old 07-21-2010, 07:02 PM   #51
GrapefruiTgirl
Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 542Reputation: 542Reputation: 542Reputation: 542Reputation: 542Reputation: 542
Thanks, nice work to you too, for coming up with those sed's - they can be mind-bending at times.

Yes, breaking up long lines really makes things easier to read. And since I began using nano editor a lot more, it's beneficial because unless you enlarge nano across multiple monitors, it doesn't show long lines properly, and is known to screw them up if one is not careful.

As for the sed's in your script before I edited it, there was only one minor problem with the first operation: a missing set of ()'s - otherwise it was fine. I can't recall now if there were any other things wrong, or if that one thing contributed to the other errors.

As for Dash, yes, it is noticeably faster than Bash, and significantly smaller. Plus, if you're interested in portability of your scripts, if they run in Dash, they'll run in most any shell, as Dash is POSIX compliant.

Hopefully OP is happy.

Best regards!
 
Old 07-21-2010, 07:12 PM   #52
simon.sweetman
Member
 
Registered: Mar 2009
Posts: 32

Rep: Reputation: 22
A quick check of the error line processing and I found a slight issue. Revise the sed lines as below

Code:
msg1=`tail -1 $dir | sed -n -r -e '$ {/successfully/s/(.*) .*lly ran on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:success:\3 \2 \5:\4/p;
                                      /ERROR/s/ERROR: (.*) is Failed on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:failed:\3 \2 \5:\4/p}'`
msg=`tail -3 $dir | head -1 | sed -n -r -e 's/(.*) started at [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:status:\3 \2 \5:\4/p'`

name=`getfield "$msg1" 1`
This is how the OP reported the error line appear:

Code:
report_name started at Fri Jul 9 00:01:00 SST 2010.
PL/SQL procedure successfully completed.
ERROR: report_name is Failed on Sat Jul 10 02:15:00 SST 2010. ErrorCode : 500

Last edited by simon.sweetman; 07-21-2010 at 07:17 PM.
 
Old 07-21-2010, 07:49 PM   #53
GrapefruiTgirl
Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 542Reputation: 542Reputation: 542Reputation: 542Reputation: 542Reputation: 542
After revising the sed's as above, we have a problem. Here's the logfile & output for success:

Code:
sasha@reactor: cat logfile
report_name started at Fri Jul 9 00:01:00 SST 2010.
PL/SQL procedure successfully completed.
report_name successfully ran on Fri Jul 19 15:13:00 SST 2010.

sasha@reactor: ./logfilerun
report_name 19 Jul 2010 00:01:00 AM 03:13:00 PM Report success
But with a "Failed" logfile, we've got some problem:
Code:
sasha@reactor: cat logfile_error
report_name started at Fri Jul 9 00:01:00 SST 2010.
PL/SQL procedure successfully completed.
ERROR: report_name is Failed on Sat Jul 10 02:15:00 SST 2010. ErrorCode : 500

sasha@reactor: ./logfilerun
[: 33: -le: unexpected operator
printf: 33: PM: expected numeric value
[: 35: -le: unexpected operator
printf: 35: PM: expected numeric value
  -12:00:00  -12:00:00  Report
I suspect it's something small, but I probably won't be up much longer tonight so will see where this is at tomorrow.
 
Old 07-21-2010, 08:10 PM   #54
GrapefruiTgirl
Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 542Reputation: 542Reputation: 542Reputation: 542Reputation: 542Reputation: 542
EDIT: That was my fault - I had an extra newline/blank-line at the end of my error log!

Here's the actual failure output:

Code:
sasha@reactor: ./logfilerun
report_name 10 Jul 2010 00:01:00 AM 02:15:00 AM Report failed
And here's a successful one:
Code:
sasha@reactor: ./logfilerun
report_name 19 Jul 2010 00:01:00 AM 03:13:00 PM Report success
sasha@reactor:
And here's the whole script:
Code:
#!/bin/sh
dir=/home/sasha/logfile_error
type=Report

getfield () {
   # echo Fields from : list
   echo $1 | cut -d: -f $2 | tr ':' ' '
}

timeap () {
    # Convert 24 hour time to AM/PM
    if [ ${1} -le 12 ]
    then
        hours=$1
       ap='AM'
    else
       hours=$(( ${1} - 12))
       ap='PM'
    fi
    printf "%02d:%02d:%02d %s" $hours $2 $3 $ap

}

msg1=`tail -1 $dir | sed -n -r -e '$ {/successfully/s/(.*) .*lly ran on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:success:\3 \2 \5:\4/p;
                                      /ERROR/s/ERROR: (.*) is Failed on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:failed:\3 \2 \5:\4/p}'`
msg=`tail -3 $dir | head -1 |           sed -n -r -e 's/(.*) started at [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:status:\3 \2 \5:\4/p'`


name=`getfield "$msg1" 1`
status=`getfield "$msg1" 2`
logdate=`getfield "$msg1" 3 | sed -r -e 's/^([0-9]) /0\1 /'`
time_end=`getfield "$msg1" 4,5,6`
time_end=`timeap $time_end`
time_start=`getfield "$msg" 4,5,6`
time_start=`timeap $time_start`

echo "$name $logdate $time_start $time_end $type $status"

Last edited by GrapefruiTgirl; 07-21-2010 at 08:13 PM.
 
Old 07-21-2010, 08:46 PM   #55
tedy2808
Member
 
Registered: Jul 2010
Posts: 34

Original Poster
Rep: Reputation: 15
thanks guys for helping me..

but i've ran sasha code but tis is what i got:


sed: illegal option -- r
sed: illegal option -- r
sed: illegal option -- r
./sasha.sh[2]: test: Specify a parameter with this command.
./sasha.sh[2]: test: Specify a parameter with this command.
Usage Usage: sed [-n] [-e script] [-f source_file] [file...] -12:80:00 -12:80:00 sed [-n] [-e script] [-f source_file] [file...]
Usage: sed [-n] [-e script] [-f source_file] [file...]
Usage: sed [-n] [-e script] [-f source_file] [file...]



i dun have any idea on how to solve the issue..can someone explain to me what should i do to solve it?
 
Old 07-21-2010, 09:06 PM   #56
simon.sweetman
Member
 
Registered: Mar 2009
Posts: 32

Rep: Reputation: 22
OK your sed dosn't support extended regular expressions. This update should work using standard regular expressions:

(edit) Oops left a -r on the logdate line - fixed now

Code:
#!/bin/sh
dir=/home/input/test.log
type=Report

getfield () {
   # echo Fields from : list
   echo $1 | cut -d: -f $2 | tr ':' ' '
}

timeap () {
    # Convert 24 hour time to AM/PM
    if [ ${1} -le 12 ]
    then
        hours=$1
       ap='AM'
    else
       hours=$(( ${1} - 12))
       ap='PM'
    fi
    printf "%02d:%02d:%02d %s" $hours $2 $3 $ap

}

msg1=`tail -1 $dir | sed -n -e '$ {/successfully/s/\(.*\) .*lly ran on [^ ]* \(.*\) \(.*\) \(.*\) SST \([0-9][0-9]*\).*/\1:success:\3 \2 \5:\4/p;
                                   /ERROR/s/ERROR: \(.*\) is Failed on [^ ]* \(.*\) \(.*\) \(.*\) SST \([0-9][0-9]*\).*/\1:failed:\3 \2 \5:\4/p}'`
msg=`tail -3 $dir | head -1 |           sed -n -e 's/\(.*\) started at [^ ]* \(.*\) \(.*\) \(.*\) SST \([0-9][0-9]*\).*/\1:status:\3 \2 \5:\4/p'`


name=`getfield "$msg1" 1`
status=`getfield "$msg1" 2`
logdate=`getfield "$msg1" 3 | sed -e 's/^\([0-9]\) /0\1 /'`
time_end=`getfield "$msg1" 4,5,6`
time_end=`timeap $time_end`
time_start=`getfield "$msg" 4,5,6`
time_start=`timeap $time_start`

echo "$name $logdate $time_start $time_end $type $status

Last edited by simon.sweetman; 07-21-2010 at 09:19 PM.
 
Old 07-21-2010, 09:26 PM   #57
simon.sweetman
Member
 
Registered: Mar 2009
Posts: 32

Rep: Reputation: 22
Of course we see this script is very dependant on the log file having the correct format. If your considering putting this onto a production system it should verify the format is OK and gracefully terminate with an error.

For example insert the following saftey test just after the 2 big sed lines. It tests for blank msg and msg1 and reports an error.
Code:
   if [ -z "$msg" ] 
   then
        echo "Invalid start message format: " `tail -3 $dir | head -1`
        exit 1
    fi

    if [ -z "$msg1" ]
    then
        echo "Invalid finished message format: " `tail -1 $dir`
        exit 2
    fi

Last edited by simon.sweetman; 07-21-2010 at 09:36 PM.
 
Old 07-22-2010, 11:29 PM   #58
tedy2808
Member
 
Registered: Jul 2010
Posts: 34

Original Poster
Rep: Reputation: 15
hi simon,sasha and all,

need your help again..

i really apologize because not using the latest script that you guys suggested.This is because i encountered several problem in running the script..thus,i stayed with my original script.Currently my original script runs well as desired.But i wanted to upgrade my script,what i meant is that..based on the script, it will only get the specific line assigned in (voicemsg and voicemsg1)..thus,it is very risky if the line is not the exact line it want to take..i'm planning to use grep(the grep function will identify unique value which what i can see here is the date, but the date is not in exact format)i've tried and i couldn't solve the error..i need your help to guide me on how to edit the script in order for it to read the exact line using grep function, not tail that had been assigned specific number on it.

below are my script,

Code:

#!/bin/sh
voicedir=/home/input/test.log
voicemsg=`tail -3 $voicedir`
voicemsg1=`tail -1 $voicedir`
voicetype=Report
voicestatus=successfully

voicetest_name=`echo "$voicemsg1"|awk ' { print $3 } '`
#identify the report status
if [ $voicetest_name = $voicestatus ]; then
voicename=`echo "$voicemsg1"|awk ' { print $2 } '`
voicee_time=`echo "$voicemsg1"|awk ' { print $9 } '`
voicemonth=`echo "$voicemsg1"|awk ' { print $7 } '`
voiceyear=`echo "$voicemsg1"| tr -d '.'|awk ' { print $NF } ' |tr ":" " "`
voiceday=`echo $voicemsg1|awk ' { print $8 } '`
voicedate=`echo "$voiceday" "$voicemonth" "$voiceyear"`
voicestat=Success
else
voicename=`echo "$voicemsg1"|awk ' { print $3 } '`
voicee_time=`echo "$voicemsg1"|awk ' { print $10 } '`
voicemonth=`echo "$voicemsg1"|awk ' { print $7 } '`
voiceyear=`echo $voicemsg|awk '{ print $11 }' |tr -d '.'`
voiceday=`echo "$voicemsg1"|awk ' { print $9 } '`
voicedate=`echo "$voiceday" "$voicemonth" "$voiceyear"`
voicestat=Failed
fi

#to assign time in 12 hours format
voices_time=`echo $voicemsg|awk '{ print $9 }'`
voicestart_hour=`echo "$voices_time" |cut -c1-2`
voicestart_min=`echo $voices_time |cut -c4-5`
voicestart_sec=`echo $voices_time |cut -c7-8`
if [ $voicestart_hour -le 12 ] ;then
voicest_hour="$voicestart_hour"
voice_sap='AM'
else
voicest_hour=`expr $((voicestart_hour - 12))`
if [ $voicest_hour -lt 10 ] ;then
#voicest_hour=`echo $voicest_hour |cut -c1

voicest_hour=0$voicest_hour
fi

voice_sap='PM'
fi

voicehour=`echo $voicee_time |cut -c1-2`
voicemin=`echo $voicee_time |cut -c4-5`
voicesec=`echo $voicee_time |cut -c7-8`
if [ $voicehour -le 12 ] ;then
voiceen_hour=$voicehour
voiceap='AM'
else
voiceen_hour=`expr $((voicehour - 12))`
if [ $voiceen_hour -lt 10 ]; then
voiceen_hour=0$voiceen_hour
fi
voiceap='PM'
fi

if [ $voiceday -lt 10 ]
then
voicef_day="0$voiceday"
else
voicef_day="$voiceday"
fi

voicenew_start_time="$voicest_hour:$voicestart_min:$voicestart_sec $voice_sap"
voicenew_end_time="$voiceen_hour:$voicemin:$voicesec $voiceap"
voicenew_date=`echo "$voicef_day" "$voicemonth" "$voiceyear"`

#echo $voicest_hour
echo $voicename \ \ \ \ \ $voicenew_date \ \ \ \ \ $voicenew_start_time \ \ \ \ \ $voicenew_end_time \ \ \ \ \ $voicetype \ \ \ \ \ $voicestat


test.log (beware of the setting,i just copy and paste from the command prompt)

exec report_name started at Wed Mar 24 00:10:00 SST 2010.

KL/SSL procedure successfully completed.

exec report_name successfully ran on Wed Mar 24 00:10:00 SST 2010.

exec report_name started at Thu Mar 25 00:10:01 SST 2010.

KL/SSL procedure successfully completed.

exec report_name successfully ran on Thu Mar 25 00:10:01 SST 2010.

exec report_name started at Fri Mar 26 00:10:00 SST 2010.

KL/SSL procedure successfully completed.

exec report_name successfully ran on Fri Mar 26 00:10:00 SST 2010.

exec report_name started at Sat Mar 27 00:10:00 SST 2010.

KL/SSL procedure successfully completed.

exec report_name successfully ran on Sat Mar 27 00:10:00 SST 2010.




i did the script based on my understanding,sorry if it makes you guys headache understanding it..

i need your reply asap...thanks in advanced
 
Old 07-22-2010, 11:47 PM   #59
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,695
Blog Entries: 5

Rep: Reputation: 239Reputation: 239Reputation: 239
Quote:
Originally Posted by tedy2808 View Post
i need your reply asap...thanks in advanced
the way you code is very inefficient. A lot of cuts/awk/tr on each line..
I haven't looked at all the posts, but since you provided test.log, can you show me again clearly what is your expected output ?
 
Old 07-23-2010, 01:17 AM   #60
tedy2808
Member
 
Registered: Jul 2010
Posts: 34

Original Poster
Rep: Reputation: 15
ok..below are the output i got from the script..i want the same output but need to change the script using grep instead of assigning specific line to print the output.


report_name 27 Mar 2010 00:10:00 AM 00:10:00 AM Report Success

thanks.
 
  


Reply

Tags
date, day, manipulating, sed, set


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] BASH: if Variable -eq String not working worm5252 Programming 2 01-24-2010 03:07 PM
Get value for variable name defined as string webquinty Programming 4 11-18-2009 05:30 AM
Help: removing a variable substring from a string variable in sh script gnparsons Programming 2 06-04-2008 05:21 PM
variable to string x2000koh Programming 4 07-30-2003 02:23 AM
Getting a variable name based on a string. jtshaw Programming 7 10-08-2002 02:06 PM


All times are GMT -5. The time now is 12:19 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration