How to match strings inside 2 lists?

yanik · 07-08-2011, 03:26 PM

Hello, I have that script that checks the nfs mount points:

Code:

#!/bin/ksh
#set -xv

test="DO_NOT_DELETE"
rc=0

nfs=`ping teslltidcnas01`
nfs_reply="teslltidcnas01 is alive"
tmp=/tmp/nfs_monitor.txt
df_file=/tmp/df.$$
df -h|grep nas|awk '{print $6}' > $df_file
fs_share=`cat /etc/vfstab|grep nas|grep -v "#"|awk '{print $3}'`
share_num=`cat /etc/vfstab|grep nas|grep -v "#"|awk '{print $3}'|wc -l`

# -----------------------
# Check if can ping NAS
# -----------------------

nas_available ()
{
if [ "$nfs" = "$nfs_reply" ]; then
        na_rc=0
else
        na_rc=1
fi
}

# ------------------------------------------
# Check if df can find mount point of vfstab
# ------------------------------------------

df_mount ()
{
if [ -f $tmp ];then
     :
  else
     cat /dev/null    > $tmp
fi

while read -u3 df_shr; do
        for shr in $fs_share; do
                if [ "$df_shr" = "$shr" ]; then
                        echo $df_shr >> $tmp
                fi
        done
done 3<$df_file

tmp_num=`cat $tmp|wc -l`

if [ "$tmp_num" -eq "$share_num" ]; then
        df_rc=0
        else
        df_rc=1
fi
}

# -------------------------------
# Check if test file can be found
# -------------------------------

chk_file ()
{
for shr in `cat $tmp`; do
        if [ -e "$shr/$test" ]; then
                cf_rc=0
        else
                cf_rc=1
        fi
done
}

# -------------------------------
# Main
# -------------------------------

nas_available
if [ "$na_rc" -eq "0" ]; then
        echo "  INFO: NAS is available"
        rc=0
else
        echo "  ERROR: NAS is not available"
        rc=1
fi

if [ "$rc" -eq "0" ]; then
        df_mount
        if [ "$df_rc" -eq "0" ]; then
                echo "  INFO: All NFS share(s) specified in vfstab are mounted according to df"
                rc=0
        else
                echo "  ERROR: One or more NFS share not mounted"
                rc=1
        fi
fi

if [ "$rc" -eq "0" ]; then
        chk_file
        if [ "$cf_rc" -eq "0" ]; then
                echo "  INFO: All file access test(s) successful"
                rc=0
        else
                echo "  ERROR: Cannot acccess one or more test file"
                rc=1
        fi
fi

rm -f $tmp $df_file

return $rc

It works but I'd like to add something so it could tell which mount point is not mounted in the df_mount function.

The output when successful is like this:

Code:

bash-2.05# nfs_monitor.sh
  INFO: NAS is available
  INFO: All NFS share(s) specified in vfstab are mounted according to df
  INFO: All file access test(s) successful

Any ideas?

I am no expert in loops and it took me all day to write that. I couldn't really tell how to match the string in $df_file and $fs_share, so I did a little workaround with a count.

I would appreciate to be shown a little fishing trick, instead of being given the fish :-P

Thanks

pafoo · 07-09-2011, 12:07 AM

No idea how to compare strings in 2 lists in korne shell. But its easy in python.

Code:

#!/usr/bin/python

file1 = open('/tmp/list1', 'r')
list1 = file1.read()
list1 = list1.split()
file1.close()

file2 = open('/tmp/list2', 'r')
list2 = file2.read()
list2 = list2.split()
file2.close()

for item in list1:
  if item in list2:
    print "yay"
  else:
    print "boo"

A hint, just like you asked! =)

catkin · 07-09-2011, 12:08 AM

Beautifully written code but non-trivial to understand. COuld you either explain what the code is doing with data instance examples or provide a minimal script which illustrates your desire?

Meanwhile, a couple of things you might like to tidy up ...

Code:

if [ -f $tmp ];then
     :
  else
     cat /dev/null    > $tmp
fi

First off [[ ... ]] is preferred over [ ... ] and the code can be simplified to

Code:

if [[ ! -f $tmp ]]; then
     cat /dev/null    > $tmp
fi

or, less comprehensibly to

Code:

[[ -f $tmp ]] || cat /dev/null > $tmp

or, less comprehensibly and faster

Code:

[[ -f $tmp ]] || : > $tmp

Secondly, also with arguably a loss of comprehensibility

Code:

if [ "$tmp_num" -eq "$share_num" ]; then
        df_rc=0
        else
        df_rc=1
fi

can be reduced to

Code:

 df_rc=$(( tmp_num != share_num ))

EDIT: if it has to be ksh then forget most of that

grail · 07-09-2011, 12:43 AM

Well i have some questions / suggestions that may help lead you in the right direction:

1. Do you use a non-standard copy / version of ping? I ask this as the following line:

Code:

nfs=`ping teslltidcnas01`

Hangs any script I have it in as ping has not been told to finish.

2. The prolific use of cat and grep with awk are not required:

Code:

df -h | grep nas|awk '{print $6}' > $df_file

# becomes

df -h | awk '/nas/{print $6}' > $df_file

fs_share=`cat /etc/vfstab|grep nas|grep -v "#"|awk '{print $3}'`

# becomes

fs_share=`awk '/nas/ && !/^#/{print $3}' /etc/vfstab`

Also here you will see I added '^' which means that we do not wish to see lines that are completely commented. This is different to 'grep -v "#"' or '!/#/'
as the last examples would get rid of something in the following format:

Code:

blah ablha blah # This is a comment

3. To make the code easier to read and to save you headaches later on with nesting, I would recommend the use of $() over ``

4. Instead of using : so that your 'if' has somewhere to go, you can negate the result using !:

Code:

if [ -f $tmp ];then
     :
  else
     cat /dev/null    > $tmp
fi

# becomes

if [ ! -f $tmp ]; then
    cat /dev/null > $tmp
fi

This cat could also be simplified:

Code:

: > $tmp

5. Not sure if this one is a personal choice, but there is no need to use a different file descriptor in your while loop

6. Instead of using 'wc -l' to get a count, change the while loop to use a counter:

Code:

#example
counter=0

(( counter++ ))

echo $counter

7. I am not sure the following function is exactly what you were looking for (or at least it is a bit vague):

Code:

chk_file ()
{
    for shr in `cat $tmp`; do
        if [ -e "$shr/$test" ]; then
                cf_rc=0
        else
                cf_rc=1
        fi
    done
}

Based on the test you use later:

Code:

if [ "$rc" -eq "0" ]; then
        chk_file
        if [ "$cf_rc" -eq "0" ]; then
                echo "  INFO: All file access test(s) successful"
                rc=0
        else
                echo "  ERROR: Cannot acccess one or more test file"
                rc=1
        fi
fi

Assuming we get in to run chk_file, as long as the last entry in $tmp has the require file (DO_NOT_DELETE) you will always see:

Code:

INFO: All file access test(s) successful

Which is not necessarily the case if the first entry in the $tmp file does not have the required file

8. You perform many numeric comparisons but do not use the associated arithmetic operators:

Code:

if [ "$na_rc" -eq "0" ]; then

# becomes

if (( na_rc == 0 )); then

# or if you change the if / else structure

if (( na_rc )); then

I suggest testing this in an alternate small script to understand how it works.

9. Lastly, you finish the script with 'return' which is used to return the status of a function whereas 'exit' should be used at the end of a script (or anywhere in between as well)

Let me know how you go with all the information and I will see if I can help further with your actual query.

grail · 07-09-2011, 12:44 AM

Quote:

EDIT: if it has to be ksh then forget most of that

@catkin - These things work in ksh as well

catkin · 07-09-2011, 12:55 AM

Quote:

Originally Posted by grail

@catkin - These things work in ksh as well

Thanks grail; that's good to know

There are many bash features which are not in ksh but I cannot recall which and AFAIK there's no list of them to check.

Regards the advice seen recently on LQ to use numeric tests instead of general purpose tests with numeric comparison operators, such as

Code:

if [ "$na_rc" -eq "0" ]; then

# becomes

if (( na_rc == 0 )); then

what is the advantage (apart from the relatively trivial being able to drop the $ prefix operator, which is anyway arguably less legible)?

grail · 07-09-2011, 04:11 AM

Quote:

what is the advantage (apart from the relatively trivial being able to drop the $ prefix operator, which is anyway arguably less legible)?

Well if you are asking if it is anymore efficient ... I am not sure.

If legibility requires the $ you can still use it, but I think I also like being able to use the normal mathematical symbols for comparisons.
From that point of view I think it is much clearer

catkin · 07-09-2011, 05:10 AM

Thanks grail

So we are in "personal choice" territory rather than functional advantage ... OK

EDIT: and, in that sense, it's not like $( ... ) instead of ` ... ` or [[ ... ]]] instead of [ .. ]

yanik · 07-11-2011, 08:36 AM

You guys are just great.

I forgot to mention some informations. It is to be ran on a Solaris box we have here, in a ksh shell. So that explains the ping command, which is different in solaris.

Thanks for all the inputs. I will make the code clearer and will try some ideas from this thread.

grail · 07-11-2011, 10:12 AM

Quote:

It is to be ran on a Solaris box we have here, in a ksh shell

If you can, if there is anything unusual about your setup, like being on Solaris, try to always make mention otherwise you get dummies like me ranting on about irrelevant stuff

Let us know how you get on? I believe most if not all the suggestions should be relatively easy to move to ksh.