[SOLVED] getting sub-string in a variable

tedy2808 · 07-13-2010, 09:27 PM

Hi all,

i really new in linux and just heard about shell scripting couple days ago..

i did exercises on linux in online tutorial but as a beginner, i'm facing problems in developing the script as there are errors that sometimes i dun have any idea on how to solve it.What i'm doing now is not homework, but i had been assigned to develop some system in linux.

i appreciate if there are people that willing to guide me on this matter.

my problem currently is on sub-string matter, where i need to read the line from file/directory and based on the line retrieved,i need to seperate the information in the line and assign as variable..

below are my script:

(the script reside in test.sh)

!/bin/sh
dir=/home/user/input/test.log #(this is the input file)

msg=`tail -1 $dir`
#(it will take only the last line as log file will be updated every day)

echo "$msg"

thus, the output something like this:

file_name successfully ran on Mon Jul 12 23:15:00 SST 2009.

now, what i need to do is to extract certain information from that line which is the name, date, time and the status

the desired output is:

name date time status
file_name Mon Jul 12 23:15:00 success

so,my next step is to identify the sub-string and assigned as variable first in order to parse the info and output it.

thus,my script is:

!/bin/sh
dir=/home/user/input/test.log #(this is the input file)

msg=`tail -1 $dir`
#(it will take only the last line as log file will be updated every day)

sub= $msg
name=${msg:0:8}

echo "$msg"
echo "name"

but i received error message which is

./test.sh[10]: name=${msg:0:8}: The specified substitution is not valid for this command.

guide me please..

jlinkels · 07-13-2010, 09:54 PM

The command to extract the substring is valid:

Code:

jlinkels@jlinkels-lt:~$ msg="hello world"
jlinkels@jlinkels-lt:~$ part=${msg:0:5}
jlinkels@jlinkels-lt:~$ echo $part
hello

Are you sure you are running Bash?
Try this to test:

Code:

jlinkels@jlinkels-lt:~$ which sh
/bin/sh
jlinkels@jlinkels-lt:~$ file /bin/sh
/bin/sh: symbolic link to `bash'
jlinkels@jlinkels-lt:~$

If it is not Bash, run your script:

Code:

bash test.sh

Or even better:

Code:

bash -x test.sh

to see every line of code executed and echoed.

jlinkels

simon.sweetman · 07-13-2010, 11:09 PM

As jlinkels points out it's probably because you aren't using the bash shell (substrings are specific to bash).

Your first line should be:

Code:

#!/bin/bash

To ensure script is run with bash

If you don't have a space in the file_name, time or status strings, using shell pattern removal should work quite well:

Code:

name=${msg%% *}      # remove 1st space add all following
status=${msg#* }     # remove 1st space and all proceeding
status=${status%% *} # remove 1st space add all following
datetime=${msg##*ran on }   # remove "ran on " and all proceeding
datetime=${datetime%% SST *} # remove " SST " and all following
time=${datetime##* } # remove last space and all proceeding
date=${datetime% *}  # remove last space and all following

tedy2808 · 07-14-2010, 12:15 AM

ok...i've edited my script and i got my desired output..

but the position of the string is always changes..Thus the script is no longer efficient.It may output wrong result. i'm thinking of creating a script that can read the exact sub-string that i wanted...let say from the whole line, i want to capture only "my file name"..instead of i have to set the position.

maybe in the log file, the last line has long/many words,so i just wanted to capture certain word in order for me to assign it as variable..

i've tried to search and did some try an error but still doesn't work.. any idea?

#!/bin/sh
dir=/home/user/input/test.log

#cat $dir

msg=`tail -1 $dir`
sub=`echo ${msg:6:23}`
echo "$sub"

#name=${msg:0:21}

echo "$msg"
#echo "$name"

grail · 07-14-2010, 12:59 AM

By position I presume you mean where the string starts or finishes but not where it is contained in the string?
If yhis is the case I would suggest using awk to return the required field.
Based on your first example this could look like:

Code:

awk 'END{print $1}' $dir

This will return the file_name

tedy2808 · 07-14-2010, 05:33 AM

thanks grails,
but what if the file name has several words,
for example: daily product tested that include the space??

grail · 07-14-2010, 07:06 AM

So I am guessing you mean if your first example changed and the actual file name is "file name".
If this is the case, none of your other options were going to work any better. You will need one of the following:

1. valid delimeter so as to parse the information
2. prior knowledge of the exact length of each item

simon.sweetman · 07-14-2010, 11:39 AM

You might be able to use the status sting to align things for example if the string is always "successfully ran on" or "unsuccesfully run on" the following should work:

Code:

name=${msg%% unsuccesfully *}
status=failed
if [ ${#name} -eq ${#msg} ]
then
   # length of string is same as original message
   #  ie replace matched nothing - assume success string
   status=success
   name=${msg%% successfully *}
fi
datetime=${msg##*ran on }   # remove "ran on " and all proceeding
datetime=${datetime%% SST *} # remove " SST " and all following
time=${datetime##* } # remove last space and all proceeding
date=${datetime% *}  # remove last space and all following

Andrew Benton · 07-14-2010, 01:05 PM

Bash arrays can be useful

Code:

#!/bin/bash
set -e

dir=/home/user/input/test.log
msg=($(tail -1 $dir))
echo ${msg[0]} ${msg[1]} ran on ${msg[4]} ${msg[5]} ${msg[6]} ${msg[7]} ${msg[8]} ${msg[9]}

grail · 07-14-2010, 09:17 PM

@Andy - I think arrays will still fail in this scenario as the user is now indicating the input file string may look like:

file name successfully ran on Mon Jul 12 23:15:00 SST 2009

Which I believe your suggestion will place "name successfully" in ${msg[1]} which would be wrong

vikas027 · 07-14-2010, 09:47 PM

Dear tedy2808,
Please try to use codes in shell scripts/commands/outputs in the thread. It makes the thread a lot more readable.

tedy2808 · 07-14-2010, 11:47 PM

simon.sweetman,
i don't really understand one of the line;

if [ ${#name} -eq ${#msg} ] in

name=${msg%% unsuccesfully *}
status=failed
if [ ${#name} -eq ${#msg} ]
then
# length of string is same as original message
# ie replace matched nothing - assume success string
status=success
name=${msg%% successfully *}
fi

can anyone explain clearly to me?

simon.sweetman · 07-15-2010, 02:26 AM

Ok I'll try and explain (numbered for ease of reference):

Code:

 1 name=${msg%% unsuccesfully *}
 2 status=failed
 3 if [ ${#name} -eq ${#msg} ]
 4 then
 5   # length of string is same as original message
 6   # ie replace matched nothing - assume success string
 7   status=success
 8   name=${msg%% successfully *}
 9 fi

Line 1 removes everything from just before " unsuccesfully " to the end of the string. So after 1 name will have the filename if the string contains " unsuccesfully", or the whole value of msg if the string dosn't (nothing being removed because of nonmatch).

line 3 now compares the length of name with the length of msg, they will be equal if nothing was replaced in line 1 (ie string doesn't contain " unsuccesfully ").

We could also have used if [ "$name" = "$msg" ] here (note -eq is the numeric compare and = is the string compare). Lines 3 and 4 deduce if msg doesn't contain "unsuccesfully" then it must contain "sucessfully" (assumption is that there must be one or the other).

Try playing around from within the shell, just assign msg and then echo anything you want to try out eg:

Code:

$ msg="file name successfully ran on Mon Jul 12 23:15:00 SST 2009"
$ name=${msg%% unsuccessfully *}
$ echo ${#name}  "?" ${#msg}
60 ? 60
$ echo ${msg%% successfully *}
file name

tedy2808 · 07-15-2010, 03:48 AM

below are the scripts that i've edited based on my understanding:

#!/bin/sh
dir=/home/user/input/test.log

msg=`tail -1 $dir`
name=${msg%% Failed *}

status=Failed
if [ ${#name} -eq ${#msg} ]
then
# length of string is same as original message
# ie replace matched nothing - assume success string
# status=success
name=${msg%% successfully *}
status=Success
fi
datetime=${msg##*ran on } # remove "ran on " and all proceeding
datetime=${datetime%% SST *} # remove " SST " and all following
time=${datetime##* } # remove last space and all proceeding

date=${datetime% *} # remove last space and all following
echo "$msg"
echo "$name"
echo "$status"

the script output:

if success:

report_name successfully ran on Tue Jul 11 23:15:00 SST 2010.
report_name
Success
00:15:00

if failed:

ERROR: report_name is Failed on Tue Jul 11 23:15:00 SST 2010. ErrorCode : 500

ERROR: report_name is
Failed
00:15:00

BUT the output for name should be only "report_name" instead of "ERROR: report_name is" if failed

grail · 07-15-2010, 04:10 AM

This is back to my point that using a word somewhere down the string as a delimeter is not going to return what you want.

Are you able to show us several lines from the file you are looking at.

Also several of the lines you are showing actually do nothing to help the output:

Code:

datetime=${msg##*ran on } # remove "ran on " and all proceeding
datetime=${datetime%% SST *} # remove " SST " and all following
time=${datetime##* } # remove last space and all proceeding

date=${datetime% *} # remove last space and all following

None of these are output anywhere so not sure why you bothered?