How to get output file from a master file

secondchanti · 08-03-2010, 10:12 AM

Frieds,

I am having following problem in linux environment.

I have a following basic file with name " BASEFILE", as shown example below :

sl.no pol.no name status loan
1 123 rama FORCE 500
2 234 jama LAPSE 800
3 345 kama FORCE 900
4 456 lama FORCE 550
5 567 nama WDRAW 650
6 678 sama SURRD 750
7 789 pama FORCE 850

Now I want outputfile with name "OUTPUTFILE" for only few pol nos
like
123
789
678

ie,1. MY OUTPUT FILE IS TO BE WITH ALL THE COLUMNS for THOSE ONLY 3 POL
2. MY OUTPUT FILE IS TO BE WITH SELECTED COLUMNS SAY name and status ,ONLY FOR 3 POLICIES.

( in my work environment, iam having a basic file with 195000 rows and i want output file for only 57000 policies, such the case )

I request you to suggest the command for drawing such output file.
, if awk programme is necessary, pl tell me where, and how to write such
programme.
Pl guide me

Rao

aka_norm · 08-03-2010, 10:21 AM

just use grep

so like:

grep "123" BASEFILE > OUTPUTFILE

would give you all the lines containing the 123 on them and then

grep "789" BASEFILE >> OUTPUTFILE

would append all the lines with 789 on them to the OUTPUTFILE.

etc..

I hope this helps.
Norm

druuna · 08-03-2010, 10:24 AM

Hi,

Here are 2 ways of doing just that:

Awk:
awk '$2==123||$2==789||$2==678' infile > outputfile

Grep:
egrep " 123 | 789 | 678 " infile > outputfile

Hope this helps.

druuna · 08-03-2010, 10:26 AM

Hi,

Quote:

Originally Posted by aka_norm

grep "123" BASEFILE > OUTPUTFILE

Be carefull with this, it will probably inlcude false hits!!

With a 195000 entries there is probably an entry like 123, but also one like 1230. Both will be shown with the above example.

Include the space/tab in front and behind the number (see my post #3).

Hope this clears things up.

aka_norm · 08-03-2010, 10:30 AM

Thanks Druuna... Great points.. After I posted, I noticed that the input file might also have " 123 " in the loan field so the awk method is the best way so that you are only looking at second field.

druuna · 08-03-2010, 10:33 AM

@aka_norm: LOL!! I overlooked the loan part (shame on me.....)

The awk one is indeed the one to go with.

After taking a good look: If the loan part is the last word on the line, the grep will work (I included a leading and a trailing space).

secondchanti · 08-03-2010, 11:12 AM

friends,

like 123 678, i have to at a stretch i need outputfile with 57000 entries, like 123,678,789,...... (570000).
It is little bit difficult to write all the nos in command line. pl help me in getting output file for all those 57000 entries from base file of 195000.
Pl guide me.
Rao

druuna · 08-03-2010, 11:20 AM

Hi,

- Where are these 57000 stored?
- How are they stored?

Both not mentioned in your original post and how are we to know that this and i want output file for only 57000 policies means you have 57000 policies floating around somewhere and that is the input? I see no good reason to flag all answers as unhelpful if your initial description of the problem is (partially) incorrect.....

Please be precise if you describe your problem.

secondchanti · 08-03-2010, 11:34 AM

Dear friend, I have in my first posting also i have noted that out of a base file with 195000 i want output file for 57000 entries. ( pl see in brackets in my first posting.)

Further my problem is like this,

in my office policy data master file is with 195000 policy numbers with various details like loan amount, status, name of policyholder, address, commencement date like so.
Now another data file with 57000 i have recd from our higher office in linux format ie policy numbers are in the filed 1 to 9 , ie 123456789
325645893
256321456
.... like so (570000 policy nos)

Now my higher office requires a out put file with the following fields for all the 57000 policies
field like status, loanamount,name, address etc.

That is the reason, i have a given a test example for my above situation.

ie for few nos , output file is required with mentioned fields from the base file.

now pl guide me

Rao

secondchanti · 08-03-2010, 11:46 AM

Dear friend, I have in my first posting also i have noted that out of a base file with 195000 i want output file for 57000 entries. ( pl see in brackets in my first posting.)

Further my problem is like this,

in my office policy data master file is with 195000 policy numbers with various details like loan amount, status, name of policyholder, address, commencement date like so.
Now another data file with 57000 i have recd from our higher office in linux format ie policy numbers are in the filed 1 to 9 , ie 123456789
325645893
256321456
.... like so (570000 policy nos)

Now my higher office requires a out put file with the following fields for all the 57000 policies
field like status, loanamount,name, address etc.

That is the reason, i have a given a test example for my above situation.

ie for few nos , output file is required with mentioned fields from the base file.

now pl guide me

druuna · 08-03-2010, 11:56 AM

Hi again,

First of all: Both aka_norm and me never understood from your first post what is was you wanted. And I did read what's between the braces (I even quoted from it and told you it was not clear). Please be as precise as possible when describing your problem and keep in mind that you might not have been understood before pushing Thanks/Yes/No buttons.......

Code:

#!/bin/bash

policyFile="policyfile"
inFile="INFILE"
outFile="OUTPUTFILE"

> $OUTPUTFILE

while read polNo
do
  awk '$2=='$polNo'' $inFile >> $outFile
done < $policyFile

Example run:

Code:

$ cat INFILE 
sl.no pol.no name status loan
1 123 rama FORCE 500
2 234 jama LAPSE 800
3 345 kama FORCE 900
4 456 lama FORCE 550
5 567 nama WDRAW 650
6 678 sama SURRD 750
7 789 pama FORCE 850

$ cat  policyfile
123
789
678

$ ./extract.sh
$ cat OUTPUTFILE 
1 123 rama FORCE 500
7 789 pama FORCE 850
6 678 sama SURRD 750

Mara · 08-03-2010, 12:00 PM

Please post your thread in only one forum. Posting a single thread in the most relevant forum will make it easier for members to help you and will keep the discussion in one place. Your threads have been merged.

jthill · 08-04-2010, 12:12 AM

To select from text files based on key fields, you want the 'join' command - which can also do other stuff, but this is what you need it for now.

join works on files sorted in key field order. Just to be safe, make a sorted policies file:

Code:

sort -nk2 pol >sorted.pol

and then selecting your records is easy:

Code:

sort -n sel|join -1 1 -2 2 - sorted.pol

you can get join to print a list of fields; see its info page.