LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices



Reply
 
Search this Thread
Old 05-07-2007, 12:31 PM   #1
micksul
LQ Newbie
 
Registered: May 2007
Posts: 9

Rep: Reputation: 0
awk Question? Search by line, using 2 files?


Hi guys,

I need some help searching 2 files if possible?
I need to make sure that the sequence of characters in each line contained in in File1, is also contained somewhere in a line in File2.
If a line in File1 does not appear in File2, I need to redirect the output containging the contents of that line to a new file.
The file formats do not match, so line 1 in File1 may not appear at the leftmost of File 2, ie it could be 20 characters in. Anyone have any ideas.

I was thinking swk, but I do not know how to achieve this?


File1 is in this format:
0000HHGJ47P49e71ULCKQN6BSO0
0017RKQEU5CSAe7PVH9PDDSMDQL
002D9856TGG6De4A5JEP9KA5RDR
002Q68CL8D6JCe0DHO6SJU13K2K
003OFNNNLSL8IeFN253UP0LFSK1

File2 is in this format:
[root@c001n01 IC]# head -10 Enumeration.reflections.0.F1.uniq
0000HHGJ47P49e71ULCKQN6BSO0~MD5_R_2ELR14GQH1SAI9J3HAU9B6UVN6~MD5~M2
0000Q4AJ7MEKGe5Q5BAEN71KMIV~MD5_R_832BFUCEFE9RD39MJKAPTUTJ1H~MD5~M2
0001CAMSNRL8ReC39IBSLOR2573~MD5_R_14AV9PVVOIGOEC6LAV8UJT77I2~MD5~M2
.
.
(Thousands of lines)
.
.
G4128H5E2040J65A40T16HQNF679QFHFP3EDS3Te3AHRHJH969EOP~GM_R_CRSM79KOGN5H7887OKVMQCQQIQ~MD5~M2
G4128H5E54N0CANSU6B8411JHUD4MK8AUNLKBFKe2O1OPBHHCK09T~GM_R_46GD5OOKCMALDF4FBHSP2TFBVS~MD5~M2
G4128HAGN3S0I4QKB67Q6CI55VBDSTCQ5OLH08Ne5TTVV4UBFKOV7~GM_R_6HBKI5A645S9JEI0V56LKJ3061~MD5~M2

Thanks,
Mick
 
Old 05-07-2007, 12:52 PM   #2
osor
HCL Maintainer
 
Registered: Jan 2006
Distribution: (H)LFS, Gentoo
Posts: 2,450

Rep: Reputation: 70
You might use awk, but I would use simple bash:
Code:
#!/bin/bash

for line in `cat file1`; do
	if ! grep -q "$line" file2; then
		echo $line >> file3
	fi
done
 
Old 05-07-2007, 01:55 PM   #3
micksul
LQ Newbie
 
Registered: May 2007
Posts: 9

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by osor
You might use awk, but I would use simple bash:
Code:
#!/bin/bash

for line in `cat file1`; do
	if ! grep -q "$line" file2; then
		echo $line >> file3
	fi
done
Excellent, will give this a try, thanks!
 
Old 06-06-2007, 05:38 AM   #4
micksul
LQ Newbie
 
Registered: May 2007
Posts: 9

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by osor
You might use awk, but I would use simple bash:
Code:
#!/bin/bash

for line in `cat file1`; do
	if ! grep -q "$line" file2; then
		echo $line >> file3
	fi
done

This worked a treat, thanks.

Is there anyway that I could alter the script slightly so that after 1 instance of the line in file1 is found in file2, then the next line in file1 is tested (rather than have the script look for a 2nd instance). This would make the search a lot faster as file 2 is huge and there is only ever going to be 1 instance of the line in file2 anyway?

Thanks in advance!
 
Old 06-06-2007, 07:34 AM   #5
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.6, Centos 5.10
Posts: 16,324

Rep: Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041
According to the man page, i think it's doing that already:

-q, --quiet, --silent
Quiet; do not write anything to standard output. Exit immedi-
ately with zero status if any match is found, even if an error
was detected. Also see the -s or --no-messages option.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
removing first line with AWK koobi Programming 7 08-03-2006 02:48 AM
how to select a line using awk sharad Linux - Software 5 04-05-2006 10:26 AM
Awk - get a parameter from the command line benjalien Programming 1 01-24-2006 10:06 AM
Deleting a line with gawk/awk caps_phisto Linux - General 4 11-06-2004 03:31 PM
search for specific text in fields using awk Helene Programming 2 04-23-2004 01:13 AM


All times are GMT -5. The time now is 01:08 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration