LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-15-2012, 06:26 AM   #1
t.othoneos
LQ Newbie
 
Registered: Oct 2011
Location: Athens, Greece
Posts: 16

Rep: Reputation: Disabled
Merging text files based on Pattern


Hei everybody,

I have 2 text files like so:

file1.txt

101.63.121.142 email1@domain.com
101.63.129.87 email2@domain.com
105.130.100.194 email2@domain.com
108.167.112.229 email3@domain.com
108.167.112.229 email5@domain.com
108.204.74.14 email4@domain.com
108.27.234.42 email7@domain.com
109.100.20.55 email10@domain.com
109.107.14.233 email135@domain.com
.
.
.about 6000 Lines

and file2.txt

111 168.122.13.218 US
156 155.33.225.201 US
174 38.127.112.130 US
209 63.231.92.16 US
209 70.56.123.222 US
278 132.248.173.102 MX
278 132.248.44.85 MX
278 132.248.84.79 MX
286 92.68.196.78 NL
513 128.141.145.169 CH
513 128.141.145.203 CH
513 128.141.226.236 CH
513 128.141.86.21 CH
680 194.94.224.254 DE
702 195.124.9.41 DE
702 195.124.9.42 DE
766 161.111.163.120 EU
786 129.215.5.255 EU
.
.
.About 3000 lines

Just to clarify, file1.txt contains IP addresses and e-mails and file2.txt has AS Numbers, IP Addresses and Country Codes.

What I need is to read the column of the IP Addresses on file1.txt and search file2.txt, in order to have a result like this:


101.63.121.142 email1@domain.com 111 US
101.63.129.87 email2@domain.com 156 US
105.130.100.194 email2@domain.com 278 MX
108.167.112.229 email3@domain.com 1241 GR
108.167.112.229 email5@domain.com 1248 FI
108.204.74.14 email4@domain.com 1680 IL
108.27.234.42 email7@domain.com 2529 GB
109.100.20.55 email10@domain.com 3268 CY
109.107.14.233 email135@domain.com 197295 PL

* The above are not correct, just to make clear the pattern.

Any Help?
 
Old 03-15-2012, 06:31 AM   #2
Markus Franke
LQ Newbie
 
Registered: Nov 2010
Distribution: Ubuntu 11.04
Posts: 13

Rep: Reputation: 2
find, grep and cut are your friends. ;-)

Hope this helps
 
Old 03-15-2012, 06:34 AM   #3
t.othoneos
LQ Newbie
 
Registered: Oct 2011
Location: Athens, Greece
Posts: 16

Original Poster
Rep: Reputation: Disabled
I know i can use these, maybe even awk. I don't know HOW, in order to get the result I need.
 
Old 03-15-2012, 06:39 AM   #4
Markus Franke
LQ Newbie
 
Registered: Nov 2010
Distribution: Ubuntu 11.04
Posts: 13

Rep: Reputation: 2
1. for each line in file1.txt (while loop)
2. get IP address from line (cut)
3. search this IP address in file2.txt (grep)
4. if found then get number and country code from this line (cut)
5. put all information together and write to a new output file

Which step of the algorithm you need code for?
 
Old 03-15-2012, 07:18 AM   #5
t.othoneos
LQ Newbie
 
Registered: Oct 2011
Location: Athens, Greece
Posts: 16

Original Poster
Rep: Reputation: Disabled
I think I got it :

while read ip mail
do
var=$(grep -o $ip file2.txt)
asn=$(echo $var | awk '{print $1}')
cod=$(echo $var | awk '{print $NF}')
printf "%15s %s %s %s\n" $ip $mail $asn $cod
done < file1.txt > final.txt;
 
Old 03-15-2012, 07:25 AM   #6
Markus Franke
LQ Newbie
 
Registered: Nov 2010
Distribution: Ubuntu 11.04
Posts: 13

Rep: Reputation: 2
Looks good. I would have used cut instead of awk as I am not very familiar with the sytax of awk.
But if it's working - Ok.
 
Old 03-15-2012, 08:06 AM   #7
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,243

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
Why not ditch the awk lines all together and just use a bash array:
Code:
var=($(grep -o "$ip" file2.txt))
printf "%15s %s %s %s\n" $ip $mail ${var[0]} ${var[2]}
If second file has unknown number of values per line then use the length of the array for final value.
 
Old 03-15-2012, 08:19 AM   #8
t.othoneos
LQ Newbie
 
Registered: Oct 2011
Location: Athens, Greece
Posts: 16

Original Poster
Rep: Reputation: Disabled
@grail: Works too

My code had the mistake of grep -o instead of the correct grep -m1.

SOLVED. Thanks guys
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Merging files together based on latex syntax. batonac Linux - Software 2 03-05-2010 09:15 AM
join 2 text files based on first number present in every line of the 2 text files markraem Linux - Software 4 01-25-2010 07:26 AM
replace a text pattern with the reverse of another text pattern lothario Linux - Software 5 07-25-2008 03:43 PM
renaming text files based upon a pattern in their content Spacepup Linux - General 1 07-28-2005 02:43 PM
Find string pattern in directory of text files magnum818 Linux - Newbie 2 10-15-2003 09:19 PM


All times are GMT -5. The time now is 12:57 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration