LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-13-2011, 07:22 PM   #1
kristo5747
Member
 
Registered: Jul 2010
Location: Earth
Distribution: Ubuntu 11.04 (Natty Narwhal)
Posts: 31

Rep: Reputation: 0
Grep: Return a user message when no hits


I have ASCII files to parse that 48 hours old or more ; I can identify them like so

Code:
    find . -name "FILE*TXT"  -mtime +1 -exec ls -ltas '{}' ';'
Some files have a list of hardware errors (we test electronic components), some have none. If the file name has no errors, I still want to display a message like so

Code:
        grep ^err R*VER && echo "No error"
    FILEA.TXT:err	->USB	3910	err	
    FILED.TXT:err No Error
This grep statement works but it seemingly overrides the find() statement above if I run both at the same time... How can I combine the two statements to create a report that lists the filename and error(s) like so

Code:
    FILEA.TXT Button	3320	err
    FILEB.TXT USB	3235	err
    FILEC.TXT IR Remote	2436	err
    FILED.TXT No error
Is it possible to return "No error" with the file name without error? Thanks in advance for your help.
 
Old 06-13-2011, 08:09 PM   #2
neonsignal
Senior Member
 
Registered: Jan 2005
Location: Melbourne, Australia
Distribution: Debian Stretch (Fluxbox WM)
Posts: 1,389
Blog Entries: 52

Rep: Reputation: 355Reputation: 355Reputation: 355Reputation: 355
You may be better off writing a script, but you can do something like this by starting a shell command from the find so that you can run the grep each time:
Code:
find . -name "FILE*TXT" -mtime +1 -exec sh -c 'grep -H ^err "{}" || echo "{}:No error"' \;
 
1 members found this post helpful.
Old 06-13-2011, 08:18 PM   #3
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,753

Rep: Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288
Hi,

not sure I really understand the problem, but I suspect the following might do what you want:
Code:
find . -name 'FILE*TXT' -mtime +1 -print0 | xargs -0 sh -c 'grep -H ^err $@ && echo "$@: No error"'
One problem I have is that you have tow different shell globs for the file name: FILE*TXT and R*VER.

Also you seem to be searching for lines beginning with "err" (grep ^err), yet the output indicates that "err" is at the end of the line.

I think if I understood better what the input files looked like I could give a better solution.

Evo2.
 
Old 06-14-2011, 12:48 PM   #4
kristo5747
Member
 
Registered: Jul 2010
Location: Earth
Distribution: Ubuntu 11.04 (Natty Narwhal)
Posts: 31

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by neonsignal View Post
You may be better off writing a script, but you can do something like this by starting a shell command from the find so that you can run the grep each time:
Code:
find . -name "FILE*TXT" -mtime +1 -exec sh -c 'grep -H ^err "{}" || echo "{}:No error"' \;
Beautiful!! It accomplished exactly what I was looking for. The challenge for me was to run grep each time: I could not run figure out a way.

My plan is put the code into a script so the data can be loaded into a mySQL database table. Then, I am going to use the fulltext feature to create a search engine (form based) in PHP.

Thanks again.
 
Old 06-14-2011, 07:15 PM   #5
kristo5747
Member
 
Registered: Jul 2010
Location: Earth
Distribution: Ubuntu 11.04 (Natty Narwhal)
Posts: 31

Original Poster
Rep: Reputation: 0
After running the command

Code:
find . -name "R*VER" -mtime +1 -exec sh -c 'grep -H ^err "{}" || echo "{}:No error"' \;
my files look like this

Code:
RRR1~COS~COSMETICS~40048~jgmdtv113~1~P~R22-200~029053662549~20110607~102151.VER No error
RRR1~COS~COSMETICS~ETT03~jgm14652.~3~F~R16-500~000907009757~20110607~085109.VER err 3922 	
...
Using the tilde (~) symbol as delimiter, can I extract the fields I want so I get an output like this

Code:
RRR1~COS~COSMETICS~40048~jgmdtv113~1~P~R22-200~029053662549~20110607~102151.VER No error RRR1 COS P
RRR1~COS~COSMETICS~ETT03~jgm14652.~3~F~R16-500~000907009757~20110607~085109.VER err 3922 RRR1 COS F 	
...
I tried the following

Code:
find . -name "R*VER" -mtime +1 -exec sh -c 'grep -H ^err "{}" || echo "{}:No error"' \;|awk -F~ '{print $0}{print $1"\t"$2"\t"$7"\t"$8"\t"$9"\t"$10}'
but it does not work; Instead, it produces this output (not all fields shown here...)

Code:
RRR1~COS~COSMETICS~40048~jgmdtv113~1~P~R22-200~029053662549~20110607~102151.VER No error
RRR1~COS~COSMETICS~40048~jgmdtv113~1~P~R22-200~029053662549~20110607~102151.VER RRR1 COS P
RRR1~COS~COSMETICS~ETT03~jgm14652.~3~F~R16-500~000907009757~20110607~085109.VER err 3922
RRR1~COS~COSMETICS~ETT03~jgm14652.~3~F~R16-500~000907009757~20110607~085109.VER RRR1 COS F
I'd like to do this is one pass so I can generate a SQL script with INSERT statements...Can it be done?

Last edited by kristo5747; 06-14-2011 at 07:21 PM.
 
Old 06-14-2011, 09:26 PM   #6
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,552

Rep: Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899
Quote:
my files look like this

Code:
RRR1~COS~COSMETICS~40048~jgmdtv113~1~P~R22-200~029053662549~20110607~102151.VER No error
RRR1~COS~COSMETICS~ETT03~jgm14652.~3~F~R16-500~000907009757~20110607~085109.VER err 3922
...
They only look like this after the find and grep. I think what evo2 was asking is what does the inside of each
of the files listed above look like?

I would add that if you are going down the road of using awk, why not simply employ it for the whole task,
ie get rid of grep as it is superfluous when awk can do its task as well. Again to help you do this we would
need to see some of the file contents (can be made up if sensitive, but must still include the main items to be searched for).
 
Old 06-15-2011, 07:19 PM   #7
kristo5747
Member
 
Registered: Jul 2010
Location: Earth
Distribution: Ubuntu 11.04 (Natty Narwhal)
Posts: 31

Original Poster
Rep: Reputation: 0
I came up with this (with help)

Code:
for myfile in `find . -name "R*VER" -mtime +1`
do
     somestr=`grep -H ^err $myfile || echo -e "$myfile\tNo error"`
     echo "$somestr" |sed 's:./::;s:tmp/::;s/:/\t/;'
     echo $somestr | awk -F~ '{print $1"\t"$2"\t"$7"\t"$8"\t"$9"\t"$10}' | sed 's:./::;s:tmp/::;'
done
It is more concise but still outputs on two lines i.e.

Code:
RRR1~COS~COSMETICS~99537~jgmdtv132~1~P~R22-200~029053252648~20110607~094718.VER    No error
RRR1    COS    P    R22-200    029053252648    20110607
RRR1~COS~COSMETICS~ETT03~jgm14652.~1~F~R16-300~000894980523~20110607~084053.VER    err    ->IR Remote Key 1    3310    err   
RRR1    COS    F    R16-300    000894980523    20110607
I am looking to generate dynamically a SQL script with insert statements like so:

Code:
INSERT INTO MYTABLE (COL1,COL2,COL3,COL4,COL5,COL6,COL7,COL8)
VALUES ('RRR1~COS~COSMETICS~99537~jgmdtv132~1~P~R22-200~029053252648~20110607~094718.VER','No error','RRR1','COS','P','R22-200','029053252648','20110607');
INSERT INTO MYTABLE (COL1,COL2,COL3,COL4,COL5,COL6,COL7,COL8)
VALUES  ('RRR1~COS~COSMETICS~ETT03~jgm14652.~1~F~R16-300~000894980523~20110607~084053.VER',' err    ->IR Remote Key 1    3310    err','RRR1','COS','P','R16-300','000894980523','20110607');
All I would need to do is wrap both lines but I am not clear how to do it (still learning awk/sed). Is it possible?

Thanks for your help.

Last edited by kristo5747; 06-15-2011 at 07:52 PM.
 
Old 06-15-2011, 07:58 PM   #8
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,412

Rep: Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397
TBH, I'd write a program (in my case using Perl, but awk would work) to do the whole thing ie right from analysing the orig log files to creating the INSERT statements file, all in one go/program.
It's got to be easier/more flexible/more robust in the long run than the current approach?
 
Old 06-15-2011, 07:59 PM   #9
kristo5747
Member
 
Registered: Jul 2010
Location: Earth
Distribution: Ubuntu 11.04 (Natty Narwhal)
Posts: 31

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by evo2 View Post
Hi,
...
I think if I understood better what the input files looked like I could give a better solution.

Evo2.
I am not permitted by my boss (re: "trade secret") to show you a complete example of a (DOS) file but the section where the error is looks like this in case of an error

Code:
hdr     DTV Error Code List             hdr     ^M
err     ->BIST Login Fail       3922    err     ^M
64      Clean Slot Global Data  00:00:00        Passed  20101110        0^M
and like this in case of no error
Code:
hdr     DTV Error Code List             hdr     ^M
64      Clean Slot Global Data  00:00:00        Passed  0       0^M
No fixed layout, as you can see.

Last edited by kristo5747; 06-15-2011 at 08:01 PM.
 
Old 06-16-2011, 01:24 AM   #10
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,552

Rep: Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899
First I would have to agree with Chris that the current method does not seem optimal.
Quote:
Is it possible?
Of course Like my previous post where grep was not required, your current example does not need sed.
Quote:
I am not permitted by my boss (re: "trade secret") to show you a complete example
The actual data can be useless information, ie 111 222 aaa ... whatever. What we require is to know what IS important,
ie like:

1. the err on the line - Is it always at the start of the line?
2. What separates one record from the next? ie a blank line a word a symbol??
3. You seem to show - file_name (no)error {if error then show it} and then pieces of the file name. Does this now mean that
there will be only one error per file?
4. Have you taken into account that the DOS based file with ^M at the end of lines will throw you bogus information from
awk and sed unless correctly looked for?

I am sure there will be other things, but these are the few that appear to me based on what is presented so far.
 
Old 06-16-2011, 07:41 PM   #11
kristo5747
Member
 
Registered: Jul 2010
Location: Earth
Distribution: Ubuntu 11.04 (Natty Narwhal)
Posts: 31

Original Poster
Rep: Reputation: 0
With the help, this is what I got

Code:
find . -name "R*VER" -exec sh -c 'grep -H ^err "{}" || echo "{}:No error"' \; |
        awk -F: '
                BEGIN { q="\047" }
                {
                        file=$1
                        result=$2
                        sub(/^.*\//,"",file)
                        split(file,a,/~/)
                        print "INSERT INTO MYTABLE (COL1,COL2,COL3,COL4,COL5,COL6,COL7,COL8) VALUES (" q file q "," q result q "," q a[1] q "," q a[2] q "," q a[7] q "," q a[8] q "," q a[9] q "," q a[10] q ")"
                }
        '
It works great. Thanks for taking the time. Thread marked as SOLVED.
 
Old 06-16-2011, 09:54 PM   #12
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,552

Rep: Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899
Great to see you have a solution

I thought as I keep going on about your seds and greps not being required that it would now be rude not to demonstrate,
of course you may need to test it a little bit as I do not have the input files as you do:
Code:
#!/usr/bin/awk -f

BEGIN { OFS = "\047,\047" 
		insert = "INSERT INTO MYTABLE (COL1,COL2,COL3,COL4,COL5,COL6,COL7,COL8) VALUES (\047"
		noerr = "No error"
		}

file && file != FILENAME{ print insert file,err,a[1],a[2],a[7],a[8],a[9],a[10]"\047)" }

FNR == 1{
	file = FILENAME
	err = noerr
	split(file,a,/~/)
	}

/^err/{
	err = $0
	nextfile
}

END{ print insert file,err,a[1],a[2],a[7],a[8],a[9],a[10]"\047)" }
Please this in a file and make it executable and then run it like so:
Code:
./script.awk R*VER
Now I guessed that there are no directories with this name and your find is only targeting the current directory (even
though you have not set the maxdepth)

If there truly are files in directories below the current one, this can easily be added (let me know)

Also if these are windows files being read we may need to look at the record endings, but your grep makes no allowance so I have not either.

Last edited by grail; 06-16-2011 at 09:55 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
grep detecting carriage return, how ? Grafbak Programming 22 01-13-2010 02:19 PM
How to return the entire path using PERL grep WindozBytes Programming 8 01-02-2009 08:48 AM
How to check grep status when find elements return 0 ahjiefreak Programming 18 12-18-2007 06:06 PM
How do you return a value with grep ? pppaaarrrkkk Linux - Newbie 2 11-30-2007 07:37 PM
Multi-line return from grep into an array? rose_bud4201 Programming 2 06-14-2005 05:11 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 02:03 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration