LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-09-2012, 04:09 PM   #1
cooker97
LQ Newbie
 
Registered: Jul 2012
Posts: 5

Rep: Reputation: Disabled
Exclamation Finding data in large no. of files


I need to find some data in a large no. of files. The data is in the following format :

VALUE A VALUE B VALUE C VALUE D
10 4 65 1
12 4.5 65.5 2
10.75 5.1 87 3
9.8 4 67 4

All the files have data is the same format (above). I need to write a script that copies those files (to a subdirectory, which also the script should create) that have ANY row satisfying the search criteria that : 10.5<VALUE A<11.5 && 4.5<VALUE B<5.5 && 80<VALUE C<90, and then also displays the VALUE D for those particular rows in the selected file which fulfill the above criteria.
The files are in "bin/models" which has two subdirectories,"model 1"and "model 2", each of which contain 10 data files. The files end in ".track". The new subdirectories are to be named "new_sub", and are to be created both in "model 1" and "model 2".

Thanks a TON in advance; I really need to know this one quick for a project!! Please help!
 
Old 07-09-2012, 06:00 PM   #2
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.2
Posts: 18,307

Rep: Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743
Why don't you show us what you've done so far and we'll help.
 
Old 07-10-2012, 12:41 AM   #3
cooker97
LQ Newbie
 
Registered: Jul 2012
Posts: 5

Original Poster
Rep: Reputation: Disabled
Actually nothing so far. I'm a complete noob, but need to write that script for a project.
 
Old 07-10-2012, 12:48 AM   #4
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 681Reputation: 681Reputation: 681Reputation: 681Reputation: 681Reputation: 681
Since the files are organized into fields, and contain floating point numbers, look at using awk.
 
Old 07-10-2012, 01:51 AM   #5
cooker97
LQ Newbie
 
Registered: Jul 2012
Posts: 5

Original Poster
Rep: Reputation: Disabled
@jschiwal - yeah but how exactly? I couldn't glean much from the man pages. Could you please post a sample script?
 
Old 07-10-2012, 02:32 AM   #6
cooker97
LQ Newbie
 
Registered: Jul 2012
Posts: 5

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by jschiwal View Post
Since the files are organized into fields, and contain floating point numbers, look at using awk.
@jschiwal - yeah but how exactly? I couldn't glean much from the man pages. Could you please post a sample script?
 
Old 07-10-2012, 03:02 AM   #7
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 681Reputation: 681Reputation: 681Reputation: 681Reputation: 681Reputation: 681
Something like:
Code:
 $1 > 10.5 && $1 < 11.5 && $2 < 5.5 && $2 > 4.5  && $3 < 90 && $3 > 80  { exit}
The END block will be executed, and there is a variable called FILENAME which you can use to print out the filename with a match, or even a command with the cp or mv command to move the file into the subdirectory.

I didn't follow what you want to do with the 4th field. That seems to imply that you may also want a file produced containing all matching lines, in which case, don't use exit, but set a flag, and print out the 4th column. Check this flag in the END block to determine if there were any matches.

Look at a user guide such as http://www.gnu.org/software/gawk/manual/gawk.html#top instead of the man page. Most distros also have a gawk doc package that supplies a book "Gawk: Effective AWK Programming".

It may be best to simply print out only matching filenames to stdout, and use this output in a bash for loop to move the files.

Last edited by jschiwal; 07-10-2012 at 03:06 AM.
 
Old 07-10-2012, 03:48 AM   #8
cooker97
LQ Newbie
 
Registered: Jul 2012
Posts: 5

Original Poster
Rep: Reputation: Disabled
Thanks a ton.

I think your idea of simply using the std o/p in a for loop makes better sense. I followed the link, and I think it'll prove helpful.
By the way, what is 'END block' and how do I put my filename in the FILENAME variable?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Finding large files investmentbnker75 Linux - Newbie 3 07-21-2008 05:57 PM
need help processing large data files lothario Linux - Software 2 05-27-2008 09:16 PM
Finding the large files on a linux box antken Linux - General 1 08-31-2004 05:06 PM
need help on processing large data files eph Programming 3 03-11-2004 04:56 AM
Large data files on CD dema Linux - Newbie 1 01-26-2002 10:30 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 07:14 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration