How to search multiple files w/ SED then echo back the filenames and results???

hgate73 · 04-03-2009, 02:06 AM

I am tearing my hair out with this one.

I am writing a script to let me search a variable number of files for a string, then output the results like this (bold/italics formatting is unimportant):

filename: search result

An example command would look like this:

# powersearch -f searchterm file1 file2 file3 ...
results:
name of file where match was found1: searchterm
name of file where match was found3: searchterm
name of file where match was found9: searchterm
...(etc)

I cannot for the life of me figure out how to do this. Here is what I have written so far. The first case (-w|w) works fine. But on the second case (-f|f) I cannot figure out how to search through all the files and get output like above. I appreciate any help...

Code:

while test -n "$1"; do
  case "$1" in
   -w|w) # Case to search a single file using a window size
	# custom variable names for readability
        windowSize=$2; searchTerm=$3; filename=$4
	a=`grep -n "$searchTerm" $filename | cut -d":" -f 1`
	((b=a+$windowSize))
	((c=a-$windowSize))
	echo; sed -n ""$c","$b" p " $filename; echo
	exit
      ;;
   -f|f) # Case to search multiple files
	searchTerm=$2; 

	for var in "$@"
	do
		result=`sed /$searchTerm/!d $var`
		filename=`$var`
		echo $filename ": ""$result"
	done
	exit
      ;;
   *)	# Default search in a file if no argument was specified
	searchTerm=$1
	filename=$2
	sed -n /$searchTerm/!d $filename
      ;;
  esac
done

jan61 · 04-03-2009, 03:27 PM

Moin,

Do you mean this?

Code:

...
   -f|f) # Case to search multiple files
	searchTerm=$2; 

	for var in "$@"
	do
          sed -n "/$searchTerm/s/^/$var: /p" $var
	done
	exit
      ;;

Jan

David the H. · 04-03-2009, 03:31 PM

Just how is it failing? What kind of output do you get when you try the code as is?

Is this a bash script, or do you want it to be shell-agnostic? In bash you can use "#!/bin/bash -x" on the first line to turn on verbose output for debugging.

I'm away from home and don't have access to a linux machine right now, so I can't do much to help you directly, but I do notice one thing that you probably want to fix. Since '$@' represents all the command line arguments together, using it here means that you're also including your '$1' and '$2' entries as file names.

Also, is there a chance that your file names will have spaces in them? That could throw off the search. A good way to get around that is to change the IFS environment variable to ignore spaces as separators in the script.

hgate73 · 04-05-2009, 06:12 PM

Quote:

Originally Posted by David the H.

Just how is it failing? What kind of output do you get when you try the code as is?

Is this a bash script, or do you want it to be shell-agnostic? In bash you can use "#!/bin/bash -x" on the first line to turn on verbose output for debugging.

I'm away from home and don't have access to a linux machine right now, so I can't do much to help you directly, but I do notice one thing that you probably want to fix. Since '$@' represents all the command line arguments together, using it here means that you're also including your '$1' and '$2' entries as file names.

Also, is there a chance that your file names will have spaces in them? That could throw off the search. A good way to get around that is to change the IFS environment variable to ignore spaces as separators in the script.

I used the above code snippet, and it works, but the problem you mentioned does show up: it treats the search term as a file name for the first iteration of the loop. This what the output looked like:

Code:

user@PN-SLEETSTONE ~/scripts $ ./myGrep_v2 -f beth phoneNumbers doc
sed: option requires an argument -- f
Usage: sed [OPTION]... {script-only-if-no-other-script} [input-file]...


## etc etc.... (it prints out the default SED help text here)...
## output omitted


sed: can't read beth: No such file or directory
phoneNumbers: beth (888) 786-5432
doc: beth (888) 786-5432

Here I tried searching for the term "beth" in the files "phoneNumbers" and "doc."

The file names on my personal machine have spaces, but the files at work don't, so that's not an issue. This (above) is just to test it out.

Is there anyway to improve that loop so that it ignores $1 as a file name? Or a way to make SED completely suppress any error output?

Sergei Steshenko · 04-05-2009, 06:17 PM

And why would one need such a script when 'grep' is around ?

hgate73 · 04-06-2009, 11:34 AM

Quote:

Originally Posted by hgate73

I used the above code snippet, and it works, but the problem you mentioned does show up: it treats the search term as a file name for the first iteration of the loop. This what the output looked like:



Here I tried searching for the term "beth" in the files "phoneNumbers" and "doc."

Is there anyway to improve that loop so that it ignores $1 as a file name? Or a way to make SED completely suppress any error output?

Okay, I solved it. I just redirected error output to /dev/null, like this. Here's the finished code block.

Code:

#!/bin/bash
# Title:          ps (power search)
# Purpose:        Finds a window of text in a file or multiple files
# Requirements:   BASH

# Arguments:
# -w    specify a window size. I.E. -w3
# -f    tell ps that you are searching inside multiple files. Syntax is "ps [search string] [filename1] [filename2] [etc]"
#       no argument, tells ps to simply search the file for the string.

RETVAL=0

while test -n "$1"; do
  case "$1" in

  -w|w) # Case to search a single file using a window size
    windowSize=$2; searchTerm=$3; filename=$4
    a=`grep -n "$searchTerm" $filename | cut -d":" -f 1`
    ((b=a+$windowSize))
    ((c=a-$windowSize))
    echo; sed -n ""$c","$b" p " $filename; echo
    exit
      ;;

  -f|f) # Case to search multiple files
    searchTerm=$2;
    echo
    echo "FILE NAME: SEARCH TERM"
    echo "----------------------"
    for var in "$@"
    do
          sed -n "/$searchTerm/s/^/$var: /p" $var 2> /dev/null
    done; echo
    exit
      ;;
   *)  # Default search in a file if no argument was specified
    searchTerm=$1
    filename=$2
    sed -n /$searchTerm/!d $filename
      ;;
  esac
done

exit $RETVAL

theNbomr · 04-06-2009, 01:02 PM

Without studying your code in detail, it looks like you've re-invented a grep search piped to more. Beyond that, may I humbly suggest a different name for your script than ps, which is already the name of a commonly used program to display a list of processes on the host?
--- rod.

hgate73 · 04-06-2009, 04:11 PM

Quote:

Originally Posted by theNbomr

Without studying your code in detail, it looks like you've re-invented a grep search piped to more. Beyond that, may I humbly suggest a different name for your script than ps, which is already the name of a commonly used program to display a list of processes on the host?
--- rod.

Now that I look at it, I realize you're right :-P. Oh well...I guess I learned something though.

Good idea on the script name - thanks. How about mySearch? That sounds like a spyware program :-P.