LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 11-28-2010, 06:59 AM   #1
threeonethree
LQ Newbie
 
Registered: Oct 2010
Posts: 24

Rep: Reputation: 0
Bash read each file in a directory and match with pattern


I have n files in a directory. I want to read them each line by line and match a certain pattern

psudocode

while there are files in directory
read file
if there is a pattern in file like "apple" set match=1 exit
if there is a pattern in file like "banana" set match=2 exit
if there is a pattern in file like "banana" set match=3 exit
else match=4 exit

how do i do this in bash?
 
Old 11-28-2010, 07:15 AM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

Can you tell a bit more about what it is you try to do?

About your pseudo-code:
- If a file contains both apple and banana, match will be set to 3 (and apple is "forgotten").
- match holds only the token set for the last file you process and "forgets" all the previous files.

Is this what you want/need?
 
Old 11-28-2010, 07:15 AM   #3
carltm
Member
 
Registered: Jan 2007
Location: Canton, MI
Distribution: CentOS, SuSE, Red Hat, Debian, etc.
Posts: 700

Rep: Reputation: 96
The grep command looks for patterns, and you can use it to
search through a file. "man grep" for more info.

Something like this might work:
Code:
if grep apple $File >/dev/null
then match=1
fi
Here is one way to process files in a directory.
Code:
for File in `ls 2>/dev/null`
do
  echo $File
done

Last edited by carltm; 11-28-2010 at 07:16 AM. Reason: fixed typo
 
Old 11-28-2010, 07:30 AM   #4
threeonethree
LQ Newbie
 
Registered: Oct 2010
Posts: 24

Original Poster
Rep: Reputation: 0
i am trying something like

Code:
temp="$HOME/Desktop/aad/temp"
while read line
do
apple=`grep apple $line`
banana=`grep banana $line`
none=`grep V3.90 $line`
done < `ls $temp`
but i get a ambiguous redirect error...


if i get this working.. i can have something like

if apple is not null
then match=1

else if banana is not null
then match =2

else
then match=0



each file will only match one word not more.. but it needs to remember the file because then for each file i need create another file of the same name in a different directory and put the match variable in it
 
Old 11-28-2010, 07:39 AM   #5
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,563
Blog Entries: 29

Rep: Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179
Code:
temp="$HOME/Desktop/aad/temp"
while read file
do
    apple=`grep apple $file`
    banana=`grep banana $file`
    none=`grep V3.90 $file`
done <<< $(ls $temp)
I see you are working quickly so will post now and add explanation later,

EDIT:
  1. Changed variable name line to file for legibility.
  2. The < did not work because it must be followed by a single file name.
  3. <<< introduces a here string
  4. $( <commands> ) is equivalent to ` <commands> ` for simple commands but easier to read. For more complex commands it is easier to use. More on Greg's WIKI.
  5. Indenting the lines between do and done makes it easier to see the logic of the code -- and that each do is matched by a done.

Last edited by catkin; 11-28-2010 at 07:48 AM.
 
Old 11-28-2010, 07:51 AM   #6
threeonethree
LQ Newbie
 
Registered: Oct 2010
Posts: 24

Original Poster
Rep: Reputation: 0
yeah i am working so fast i cant even think of a metaphor to relate this sentence with..

didnt work

i get a no such file or directory error

grep: /home/noc/Desktop/autotel/temp/10.169.247.34 10.173.73.1 10.173.73.254 71.11.162.253 71.11.162.254: No such file or directory


<<< is redirecting all of the file names at once and < is giving me an ambiguous redirect error


Code:
while read file
do
maipu=`grep Maipu "$temp/$file"`
cisco=`grep Cisco "$temp/$file"`
zyxel=`grep V3.90 "$temp/$file"`
done <<<$(ls -1 $temp)

Last edited by threeonethree; 11-28-2010 at 08:09 AM.
 
Old 11-28-2010, 08:03 AM   #7
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,692

Rep: Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987
Well I see a number of problems:

1. No test to see if 'temp' directory exists
2. The use of ls is generally a nightmare, but here its only problem is that you may return items other than files (ie directories, block device, etc)

Work on these prior to moving on to what you need to do with the data.
 
Old 11-28-2010, 08:05 AM   #8
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,563
Blog Entries: 29

Rep: Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179
Quote:
Originally Posted by threeonethree View Post
yeah i am working so fast i cant even think of a metaphor to relate this sentence with..

didnt work

i get a no such file or directory error
Always helpful to copy and paste what you did and the output into replies -- it's a quick and easy way to let us know precisely what happened.

Guessing without that info, try:
Code:
temp="$HOME/Desktop/aad/temp"
cd "$temp" || exit 1
while read file
do
    apple=`grep apple $file`
    banana=`grep banana $file`
    none=`grep V3.90 $file`
done <<< $(/bin/ls -1)
That -1 is a number.
 
Old 11-28-2010, 08:20 AM   #9
threeonethree
LQ Newbie
 
Registered: Oct 2010
Posts: 24

Original Poster
Rep: Reputation: 0
Code:
cd "$temp" || exit 1
while read file
do
    cisco=`grep apple $file`
    Maipu=`grep banana $file`
    zyxel=`grep V3.90 $file`
done <<< $(ls -1)


echo "$maipu"
echo "$cisco"
echo "$zyxel"
thanks

now how do i get

if cisco is not null
then match=1

else if maipu is not null
then match =2

else if zaixl is not null
match =3
else
then match=0



worked good...
 
Old 11-28-2010, 08:44 AM   #10
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,563
Blog Entries: 29

Rep: Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179
Quote:
Originally Posted by threeonethree View Post
worked good...
What grail said is good advice but I thought you would prefer to get it working first and then refine it.

One way of refining it is not to bother getting the matching lines back from grep -- all you want to know is whether the search pattern exists in the file (OK until you add pineapple when you already have apple!). For this, grep's -q, --quiet or --silent option (they all do the same thing) can be used along with grep's return code:
Code:
cd "$temp" || exit 1
while read file
do
    if [[ grep --silent apple $file ]]; then
        <whatever you want to do with the apple file>
    elif [[ grep --silent banana $file ]];then
        <whatever you want to do with the banana file>
    elif [[ grep --silent V3.90 $file ]];then
        <whatever you want to do with the V3.90 file>
    else
        echo "Nothing found in $file!" >&2
    fi
done <<< $(ls -1)
 
Old 11-29-2010, 03:43 AM   #11
threeonethree
LQ Newbie
 
Registered: Oct 2010
Posts: 24

Original Poster
Rep: Reputation: 0
thanks so much thats really elegant.


but grep --silent returns nothing if a match is found

if [ ! `grep --silent Cisco "$temp/$file"` | echo "$?" ]; then
echo "$line cisco"

gives a missing ] error

Last edited by threeonethree; 11-29-2010 at 05:37 AM.
 
Old 11-29-2010, 05:35 AM   #12
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,692

Rep: Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987
Quote:
but grep --silent returns nothing if a match is found
That is the idea as you are testing the success or failure of finding something with grep, not what has been found.
 
Old 11-29-2010, 05:43 AM   #13
threeonethree
LQ Newbie
 
Registered: Oct 2010
Posts: 24

Original Poster
Rep: Reputation: 0
Code:
noc@noc-desktop:~/autotel/temp$ grep V3.90 10.173.73.254
  Current ZyNOS version: V3.90(BBD.3) | 08/19/2010
noc@noc-desktop:~/autotel/temp$ grep -q V3.90 10.173.73.254
noc@noc-desktop:~/autotel/temp$ grep -q V3.90 10.173.73.254 | echo "$?"
0
noc@noc-desktop:~/autotel/temp$ grep -q bang 10.173.73.254 | echo "$?"
0
noc@noc-desktop:~/autotel/temp$ grep --silent bang 10.173.73.254
noc@noc-desktop:~/autotel/temp$ grep --silent V3.90 10.173.73.254
\noc@noc-desktop:~/autotel/temp$ grep --silent V3.90 10.173.73.254
but it always prints 0 whether or not there is a match or not?? V3.90 matches but bang does not
 
Old 11-29-2010, 06:35 AM   #14
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,692

Rep: Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987Reputation: 1987
I have obviously missed something here?? How are you grepping a pattern of an IP address? Or is this just a strange file naming convention??

Until I have this in context I am unable to assist.
 
Old 11-29-2010, 07:30 AM   #15
threeonethree
LQ Newbie
 
Registered: Oct 2010
Posts: 24

Original Poster
Rep: Reputation: 0
lets make this simpler

Code:
#! /bin/bash

cd "$HOME/Desktop/pingtest" || exit 1
while read line; 
do 
    if [[ grep --silent hello $file ]]; then
        echo "found hello in $line"
    elif [[ grep --silent apple $file ]]; then
        echo "found apple in $line"
    elif [[ grep --silent mango $file ]]; then
        echo "found mango in $line"
    else
        echo "found nothing"
    fi
done <<< `ls -1`
Quote:
noc@noc-desktop:~/Desktop/pingtest$ ./scr
./scr: line 6: conditional binary operator expected
./scr: line 6: syntax error near `--silent'
./scr: line 6: ` if [[ grep --silent hello $file ]]; then'
noc@noc-desktop:~/Desktop/pingtest$
ping test folder contains 3 files named 1 2 and 3 .. which have text apple mango banana respectively.

Last edited by threeonethree; 11-29-2010 at 08:09 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
BASH Script --> read a file & cp -r to my ~/ directory manwithaplan Linux - Newbie 6 10-06-2009 12:25 AM
Bash test if pattern match of file exists bioinformatics_guy Linux - Newbie 4 04-17-2009 07:42 AM
How to delete the file using pattern match? nishanthhampali Programming 3 04-16-2008 01:50 PM
Select the files of a directory that match a specific pattern jianelisj Linux - Newbie 2 03-17-2008 01:25 PM
how to use the sed w option to redirect pattern match to file nickleus Linux - General 11 04-18-2006 09:34 AM


All times are GMT -5. The time now is 11:26 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration