[SOLVED] getting sub-string in a variable

GrapefruiTgirl · 07-21-2010, 05:52 AM

Code:

st_time=`expr $((start_hour - 12))`
st_time=$(( start_hour - 12 ))

Either of the above two methods works instead of:

Code:

st_time=`echo expr{$start_hour - 12}`

simon.sweetman · 07-21-2010, 12:20 PM

Sorry real life took me away from this for a while - looks like your going well.

Remember back a couple of pages grail suggested using sed to simplify the input, that was a realy good idea.

I've done up a little bourne shell script that uses this idea. I also defined a couple of internal functions getfield() and timeap() to make the code more readable.

The sed scripts just cut out the required fields and put them in a : seperated list.

msg1 is name:status:end_date:end_h:end_m:end_s
msg is name:status:start_date:start_h:start_m:start_s

After this is pretty easy to extract your required fields (I don't have access to a bourne shell so it's untested):

Code:

#!/bin/sh
dir=/home/input/test.log
type=Report

getfield () {
   # echo Fields from : list
   echo $1 | cut -d: -f$2 | tr ':' ' '
}

timeap () {
    # Convert 24 hour time to AM/PM
    if [ $1 -le 12 ]
    then
        hours=$1
       ap=AM
    else
       hours=$(($1 - 12))
       ap=PM
    fi
    printf "%02d:%02d:%02d %s" $hours $2 $3 $ap
}

msg1=`tail -1 $dir | sed -n -r -e '$ {/successfully/s/(.*) successfully ran on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:success:\3 \2 \5:\4/p;/ERROR/s/.*R: (.*) is Failed on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:failed:\3 \2 \5:\4/p}'`
msg=`tail -3 $dir | head -1 | sed -n -r -e 's/(.*) started at [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:status:\3 \2 \5:\4/p'`

name=`getfield "$msg1" 1`
status=`getfield "$msg1" 2`
logdate=`getfield "$msg1" 3 | sed -r -e 's/^([0-9]) /0\1 /'`
time_end=`getfield "$msg1" 4,5,6`
time_end=`timeap $time_end`
time_start=`getfield "$msg" 4,5,6`
time_start=`timeap $time_start`

echo "$name $logdate $time_start $time_end $type $status"

GrapefruiTgirl · 07-21-2010, 12:44 PM

I just tested the script using Dash shell, but got some errors, as follows:

Code:

sasha@reactor: ./logfilerun
sed: -e expression #1, char 124: unterminated `s' command
[: 37: -le: unexpected operator
printf: 37: PM: expected numeric value
[: 39: -le: unexpected operator
printf: 39: PM: expected numeric value
  -12:00:00  -12:00:00  Report 
sasha@reactor:

Maybe this could be caused by my input file, which I made to contain the following stuff from Post #43 above:

Code:

report_name started at Fri Jul 9 00:01:00 SST 2010.
PL/SQL procedure successfully completed.
report_name successfully ran on Fri Jul 19 15:13:00 SST 2010.

I started messing with the sed stuff, but thought better of that because I don't know what the input file is exactly supposed to look like.

If OP can post a real sample of what the input file should contain, I can test again in a bit. Also, please verify that dir=/home/input/test.log is actually supposed to point directly at the logfile, and not a directory, right?

EDIT:
I have changed the input file to contain only:

Code:

report_name successfully ran on Tue Jul 11 18:00:01 SST 2010.

and now the results on screen are:

Code:

sasha@reactor: ./logfilerun
sed: -e expression #1, char 124: unterminated `s' command
[: 31: -le: unexpected operator
printf: 31: PM: expected numeric value
  00:01:00 AM -12:00:00  Report 
sasha@reactor:

So a little better - but still something amiss.

GrapefruiTgirl · 07-21-2010, 01:31 PM

Code:

#!/bin/sh
dir=/home/sasha/logfile
type=Report

getfield () {
   # echo Fields from : list
   echo $1 | cut -d: -f $2 | tr ':' ' '
}

timeap () {
    # Convert 24 hour time to AM/PM
    if [ $1 -le 12 ]
    then
        hours=$1
       ap='AM'
    else
       hours=$(( ${1} - 12))
       ap='PM'
    fi
    printf "%02d:%02d:%02d %s" $hours $2 $3 $ap

}

msg1=`tail -1 $dir | sed -n -r -e '$ {/successfully/s/(.*) successfully ran on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:success:\3 \2 \5:\4/p;
                                             /ERROR/s/(.*)R: (.*) is Failed on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:failed:\3 \2 \5:\4/p}'`
msg=`tail -3 $dir | head -1 | sed -n -r -e 's/(.*) started at [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:status:\3 \2 \5:\4/p'`

name=`getfield "$msg1" 1`
status=`getfield "$msg1" 2`
logdate=`getfield "$msg1" 3 | sed -r -e 's/^([0-9]) /0\1 /'`
time_end=`getfield "$msg1" 4,5,6`
time_end=`timeap $time_end`
time_start=`getfield "$msg" 4,5,6`
time_start=`timeap $time_start`

echo "$name $logdate $time_start $time_end $type $status"

Back to 3 lines in the input file (after I noticed the `head -3`

) and now, with some small edits, it works. I dunno if it works "as it should", but it works error free in Dash shell. Here's what it outputs:

Code:

sasha@reactor: ./logfilerun
report_name 19 Jul 2010 00:01:00 AM 03:13:00 PM Report success
sasha@reactor:

simon.sweetman · 07-21-2010, 06:51 PM

Nice work sasha, like the reformatted sed line makes it much easier to understand the two lines processed. I've learnt something new as well, as I never used dash before, looks like a nice simple/fast shell.

GrapefruiTgirl · 07-21-2010, 07:02 PM

Thanks, nice work to you too, for coming up with those sed's - they can be mind-bending at times.

Yes, breaking up long lines really makes things easier to read. And since I began using nano editor a lot more, it's beneficial because unless you enlarge nano across multiple monitors, it doesn't show long lines properly, and is known to screw them up if one is not careful.

As for the sed's in your script before I edited it, there was only one minor problem with the first operation: a missing set of ()'s - otherwise it was fine. I can't recall now if there were any other things wrong, or if that one thing contributed to the other errors.

As for Dash, yes, it is noticeably faster than Bash, and significantly smaller. Plus, if you're interested in portability of your scripts, if they run in Dash, they'll run in most any shell, as Dash is POSIX compliant.

Hopefully OP is happy.

Best regards!

simon.sweetman · 07-21-2010, 07:12 PM

A quick check of the error line processing and I found a slight issue. Revise the sed lines as below

Code:

msg1=`tail -1 $dir | sed -n -r -e '$ {/successfully/s/(.*) .*lly ran on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:success:\3 \2 \5:\4/p;
                                      /ERROR/s/ERROR: (.*) is Failed on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:failed:\3 \2 \5:\4/p}'`
msg=`tail -3 $dir | head -1 | sed -n -r -e 's/(.*) started at [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:status:\3 \2 \5:\4/p'`

name=`getfield "$msg1" 1`

This is how the OP reported the error line appear:

Code:

report_name started at Fri Jul 9 00:01:00 SST 2010.
PL/SQL procedure successfully completed.
ERROR: report_name is Failed on Sat Jul 10 02:15:00 SST 2010. ErrorCode : 500

GrapefruiTgirl · 07-21-2010, 07:49 PM

After revising the sed's as above, we have a problem. Here's the logfile & output for success:

Code:

sasha@reactor: cat logfile
report_name started at Fri Jul 9 00:01:00 SST 2010.
PL/SQL procedure successfully completed.
report_name successfully ran on Fri Jul 19 15:13:00 SST 2010.

sasha@reactor: ./logfilerun
report_name 19 Jul 2010 00:01:00 AM 03:13:00 PM Report success

But with a "Failed" logfile, we've got some problem:

Code:

sasha@reactor: cat logfile_error
report_name started at Fri Jul 9 00:01:00 SST 2010.
PL/SQL procedure successfully completed.
ERROR: report_name is Failed on Sat Jul 10 02:15:00 SST 2010. ErrorCode : 500

sasha@reactor: ./logfilerun
[: 33: -le: unexpected operator
printf: 33: PM: expected numeric value
[: 35: -le: unexpected operator
printf: 35: PM: expected numeric value
  -12:00:00  -12:00:00  Report

I suspect it's something small, but I probably won't be up much longer tonight so will see where this is at tomorrow.

GrapefruiTgirl · 07-21-2010, 08:10 PM

EDIT: That was my fault - I had an extra newline/blank-line at the end of my error log!

Here's the actual failure output:

Code:

sasha@reactor: ./logfilerun
report_name 10 Jul 2010 00:01:00 AM 02:15:00 AM Report failed

And here's a successful one:

Code:

sasha@reactor: ./logfilerun
report_name 19 Jul 2010 00:01:00 AM 03:13:00 PM Report success
sasha@reactor:

And here's the whole script:

Code:

#!/bin/sh
dir=/home/sasha/logfile_error
type=Report

getfield () {
   # echo Fields from : list
   echo $1 | cut -d: -f $2 | tr ':' ' '
}

timeap () {
    # Convert 24 hour time to AM/PM
    if [ ${1} -le 12 ]
    then
        hours=$1
       ap='AM'
    else
       hours=$(( ${1} - 12))
       ap='PM'
    fi
    printf "%02d:%02d:%02d %s" $hours $2 $3 $ap

}

msg1=`tail -1 $dir | sed -n -r -e '$ {/successfully/s/(.*) .*lly ran on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:success:\3 \2 \5:\4/p;
                                      /ERROR/s/ERROR: (.*) is Failed on [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:failed:\3 \2 \5:\4/p}'`
msg=`tail -3 $dir | head -1 |           sed -n -r -e 's/(.*) started at [^ ]* (.*) (.*) (.*) SST ([0-9]+).*/\1:status:\3 \2 \5:\4/p'`


name=`getfield "$msg1" 1`
status=`getfield "$msg1" 2`
logdate=`getfield "$msg1" 3 | sed -r -e 's/^([0-9]) /0\1 /'`
time_end=`getfield "$msg1" 4,5,6`
time_end=`timeap $time_end`
time_start=`getfield "$msg" 4,5,6`
time_start=`timeap $time_start`

echo "$name $logdate $time_start $time_end $type $status"

tedy2808 · 07-21-2010, 08:46 PM

thanks guys for helping me..

but i've ran sasha code but tis is what i got:

sed: illegal option -- r
sed: illegal option -- r
sed: illegal option -- r
./sasha.sh[2]: test: Specify a parameter with this command.
./sasha.sh[2]: test: Specify a parameter with this command.
Usage Usage: sed [-n] [-e script] [-f source_file] [file...] -12:80:00 -12:80:00 sed [-n] [-e script] [-f source_file] [file...]
Usage: sed [-n] [-e script] [-f source_file] [file...]
Usage: sed [-n] [-e script] [-f source_file] [file...]

i dun have any idea on how to solve the issue..can someone explain to me what should i do to solve it?

simon.sweetman · 07-21-2010, 09:06 PM

OK your sed dosn't support extended regular expressions. This update should work using standard regular expressions:

(edit) Oops left a -r on the logdate line - fixed now

Code:

#!/bin/sh
dir=/home/input/test.log
type=Report

getfield () {
   # echo Fields from : list
   echo $1 | cut -d: -f $2 | tr ':' ' '
}

timeap () {
    # Convert 24 hour time to AM/PM
    if [ ${1} -le 12 ]
    then
        hours=$1
       ap='AM'
    else
       hours=$(( ${1} - 12))
       ap='PM'
    fi
    printf "%02d:%02d:%02d %s" $hours $2 $3 $ap

}

msg1=`tail -1 $dir | sed -n -e '$ {/successfully/s/\(.*\) .*lly ran on [^ ]* \(.*\) \(.*\) \(.*\) SST \([0-9][0-9]*\).*/\1:success:\3 \2 \5:\4/p;
                                   /ERROR/s/ERROR: \(.*\) is Failed on [^ ]* \(.*\) \(.*\) \(.*\) SST \([0-9][0-9]*\).*/\1:failed:\3 \2 \5:\4/p}'`
msg=`tail -3 $dir | head -1 |           sed -n -e 's/\(.*\) started at [^ ]* \(.*\) \(.*\) \(.*\) SST \([0-9][0-9]*\).*/\1:status:\3 \2 \5:\4/p'`


name=`getfield "$msg1" 1`
status=`getfield "$msg1" 2`
logdate=`getfield "$msg1" 3 | sed -e 's/^\([0-9]\) /0\1 /'`
time_end=`getfield "$msg1" 4,5,6`
time_end=`timeap $time_end`
time_start=`getfield "$msg" 4,5,6`
time_start=`timeap $time_start`

echo "$name $logdate $time_start $time_end $type $status

simon.sweetman · 07-21-2010, 09:26 PM

Of course we see this script is very dependant on the log file having the correct format. If your considering putting this onto a production system it should verify the format is OK and gracefully terminate with an error.

For example insert the following saftey test just after the 2 big sed lines. It tests for blank msg and msg1 and reports an error.

Code:

   if [ -z "$msg" ] 
   then
        echo "Invalid start message format: " `tail -3 $dir | head -1`
        exit 1
    fi

    if [ -z "$msg1" ]
    then
        echo "Invalid finished message format: " `tail -1 $dir`
        exit 2
    fi

tedy2808 · 07-22-2010, 11:29 PM

hi simon,sasha and all,

need your help again..

i really apologize because not using the latest script that you guys suggested.This is because i encountered several problem in running the script..thus,i stayed with my original script.Currently my original script runs well as desired.But i wanted to upgrade my script,what i meant is that..based on the script, it will only get the specific line assigned in (voicemsg and voicemsg1)..thus,it is very risky if the line is not the exact line it want to take..i'm planning to use grep(the grep function will identify unique value which what i can see here is the date, but the date is not in exact format)i've tried and i couldn't solve the error..i need your help to guide me on how to edit the script in order for it to read the exact line using grep function, not tail that had been assigned specific number on it.

below are my script,

Code:


#!/bin/sh
voicedir=/home/input/test.log
voicemsg=`tail -3 $voicedir`
voicemsg1=`tail -1 $voicedir`
voicetype=Report
voicestatus=successfully

voicetest_name=`echo "$voicemsg1"|awk ' { print $3 } '`
#identify the report status
if [ $voicetest_name = $voicestatus ]; then
voicename=`echo "$voicemsg1"|awk ' { print $2 } '`
voicee_time=`echo "$voicemsg1"|awk ' { print $9 } '`
voicemonth=`echo "$voicemsg1"|awk ' { print $7 } '`
voiceyear=`echo "$voicemsg1"| tr -d '.'|awk ' { print $NF } ' |tr ":" " "`
voiceday=`echo $voicemsg1|awk ' { print $8 } '`
voicedate=`echo "$voiceday" "$voicemonth" "$voiceyear"`
voicestat=Success
else
voicename=`echo "$voicemsg1"|awk ' { print $3 } '`
voicee_time=`echo "$voicemsg1"|awk ' { print $10 } '`
voicemonth=`echo "$voicemsg1"|awk ' { print $7 } '`
voiceyear=`echo $voicemsg|awk '{ print $11 }' |tr -d '.'`
voiceday=`echo "$voicemsg1"|awk ' { print $9 } '`
voicedate=`echo "$voiceday" "$voicemonth" "$voiceyear"`
voicestat=Failed
fi

#to assign time in 12 hours format
voices_time=`echo $voicemsg|awk '{ print $9 }'`
voicestart_hour=`echo "$voices_time" |cut -c1-2`
voicestart_min=`echo $voices_time |cut -c4-5`
voicestart_sec=`echo $voices_time |cut -c7-8`
if [ $voicestart_hour -le 12 ] ;then
voicest_hour="$voicestart_hour"
voice_sap='AM'
else
voicest_hour=`expr $((voicestart_hour - 12))`
if [ $voicest_hour -lt 10 ] ;then
#voicest_hour=`echo $voicest_hour |cut -c1

voicest_hour=0$voicest_hour
fi

voice_sap='PM'
fi

voicehour=`echo $voicee_time |cut -c1-2`
voicemin=`echo $voicee_time |cut -c4-5`
voicesec=`echo $voicee_time |cut -c7-8`
if [ $voicehour -le 12 ] ;then
voiceen_hour=$voicehour
voiceap='AM'
else
voiceen_hour=`expr $((voicehour - 12))`
if [ $voiceen_hour -lt 10 ]; then
voiceen_hour=0$voiceen_hour
fi
voiceap='PM'
fi

if [ $voiceday -lt 10 ]
then
voicef_day="0$voiceday"
else
voicef_day="$voiceday"
fi

voicenew_start_time="$voicest_hour:$voicestart_min:$voicestart_sec $voice_sap"
voicenew_end_time="$voiceen_hour:$voicemin:$voicesec $voiceap"
voicenew_date=`echo "$voicef_day" "$voicemonth" "$voiceyear"`

#echo $voicest_hour
echo $voicename \ \ \ \ \ $voicenew_date \ \ \ \ \ $voicenew_start_time \ \ \ \ \ $voicenew_end_time \ \ \ \ \ $voicetype \ \ \ \ \ $voicestat

test.log (beware of the setting,i just copy and paste from the command prompt)

exec report_name started at Wed Mar 24 00:10:00 SST 2010.

KL/SSL procedure successfully completed.

exec report_name successfully ran on Wed Mar 24 00:10:00 SST 2010.

exec report_name started at Thu Mar 25 00:10:01 SST 2010.

KL/SSL procedure successfully completed.

exec report_name successfully ran on Thu Mar 25 00:10:01 SST 2010.

exec report_name started at Fri Mar 26 00:10:00 SST 2010.

KL/SSL procedure successfully completed.

exec report_name successfully ran on Fri Mar 26 00:10:00 SST 2010.

exec report_name started at Sat Mar 27 00:10:00 SST 2010.

KL/SSL procedure successfully completed.

exec report_name successfully ran on Sat Mar 27 00:10:00 SST 2010.

i did the script based on my understanding,sorry if it makes you guys headache understanding it..

i need your reply asap...thanks in advanced

ghostdog74 · 07-22-2010, 11:47 PM

Quote:

Originally Posted by tedy2808

i need your reply asap...thanks in advanced

the way you code is very inefficient. A lot of cuts/awk/tr on each line..
I haven't looked at all the posts, but since you provided test.log, can you show me again clearly what is your expected output ?

tedy2808 · 07-23-2010, 01:17 AM

ok..below are the output i got from the script..i want the same output but need to change the script using grep instead of assigning specific line to print the output.

report_name 27 Mar 2010 00:10:00 AM 00:10:00 AM Report Success

thanks.