LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-15-2009, 11:08 AM   #1
Spinoz
LQ Newbie
 
Registered: Sep 2009
Posts: 10

Rep: Reputation: 0
extracting data and putting in new file


Hello

I am new to linux. I would like some assistance in writing a script that allows me to extract my data. I will illustrate my problem with an example.
Consider the following: octoberdata1, octoberdat12, octoberdata3 & octoberdata4 ( basically a text file found within the directory tpl) contains important parameters and a collection of data points:
** Parameter1
....

** Parameter2
...

...
..
** cell contour outline
Under here it contains the following
x y dx dy ddx ddy
1 ... ... ... ... ... ...
2 ... ... ... .. .. ...

..

..
etc...

Basically I want a script that tells it to go under "cell contour outline" and extract the data there and put it in a new file, which will be saved in the directory tpl.

This is what I was thinking:

#!/bin/bash
#

cd ..../tpl
printf octoberdata1"\t"cell contour outline" > "c2c12_cell_1.txt"
printf octoberdata2"\t"cell contour outline" > "c2c12_cell_2.txt"
printf octoberdata3"\t"cell contour outline" > "c2c12_cell_3.txt"
printf octoberdata4"\t"cell contour outline" > "c2c12_cell_4.txt"

done
exit

Can anyone kindly look over this and tell me whether it is correct or can show me a much efficient way.

Most appreciated
Spinoz
 
Old 10-15-2009, 01:20 PM   #2
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960
First of all, please use [code][/code] tags to enclose your code and data. It will help keep things readable.

Second, I'm still not 100% sure what you want to do exactly. You want to find the line in a file that says "cell contour outline", then print the line following it to a new file? Just one line? Can there be multiple lines, or multiple instances in one file? If there are multiple lines, how can you tell when that section ends?

Next, this doesn't appear right at all.
Code:
printf octoberdata1"\t"cell contour outline" > "c2c12_cell_1.txt"
printf can be viewed as an extended echo command, and > redirects the output to the specified filename, overwriting any previous contents. So basically you're creating (or overwriting) a file named "c2c12_cell_1.txt" with the literal line "octoberdata1 <tab> cell contour outline". This seems to have nothing to do with any "extracted" data.

(But actually, the line as given above doesn't work at all because there's an odd number of quotation marks.)

What you probably need to do is use sed or awk or some other tool to match the proper line and extract the data you want, then redirect it to a new file. But I think you should clarify your requirements for us first so we can figure out exactly how to match what you need.

Last edited by David the H.; 10-15-2009 at 01:22 PM.
 
Old 10-15-2009, 04:14 PM   #3
Spinoz
LQ Newbie
 
Registered: Sep 2009
Posts: 10

Original Poster
Rep: Reputation: 0
Hello David,

Yes that is what I want to do. I want to find the line in a file that says "BEGIN CELL EDGE VECTOR FIELD". Yes you are correct that there are multiple lines:
F0r example the 17th line reads "BEGIN CELL EDGE VECTOR FIELD", then it terminates by having a line of stars ********** then another line starts with a different title containing other data points. So basically I want something where I can go into that file and tell it to look under the line that says BEGIN CELL EDGE VECTOR FIELD and extract all the data there and put it in a new file and in the same directory.
Attached is a sample of what I am talking about in txt format.

Thank you so much for your time, and my apologies for any in coherencies.

Spinoz
Attached Files
File Type: txt octoberdata.txt (9.3 KB, 9 views)

Last edited by Spinoz; 10-15-2009 at 04:15 PM.
 
Old 10-16-2009, 09:20 AM   #4
Spinoz
LQ Newbie
 
Registered: Sep 2009
Posts: 10

Original Poster
Rep: Reputation: 0
Anyone plz
 
Old 10-16-2009, 09:37 AM   #5
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738
You might want to look at SED---in particular address ranges.

Example:

Code:
sed -n '/START/,/STOP/p' sourcefile >destfile
Reads from sourcefile and returns everything from a line containing "START" to a line containing "STOP", then writes this data to destfile.

Excellent SED tutorial here:
http://www.grymoire.com/Unix/Sed.html
 
Old 10-16-2009, 02:51 PM   #6
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960Reputation: 1960
You should try to be patient. We are all volunteers here, after all.

Yes, sed should do it. But while it's fairly easy to get a range of lines with it, it's a bit harder to extract a range from between two lines. I had to work a bit with nested expressions to get it to output only the lines you want (I suppose I could've done it quicker with multiple piped sed commands, but I enjoy the challenge ).

However, there's still one thing that's not clear to me. The data field you want looks like this:
Code:
   75 BEGIN CELL EDGE VECTOR FIELD
*****        Xfr        Yfr        Xto        Yto      BLANK      ETYPE
    1  3.227E-05 -1.630E-06  3.896E-05 -9.291E-06  0.000E+00  0.000E+00
    2  3.896E-05 -9.291E-06  4.222E-05 -1.043E-05  0.000E+00  0.000E+00
***** ********** ********** ********** ********** ********** **********
Do you want to print the header line too? or just numbered data lines between the ***** lines? I did it both ways.

For the former, this seems to work.
Code:
sed -n '0,/CELL EDGE/! {/Xfr/,/^\*/ {/\*$/d;p}}' octoberdata-1.txt > outputfile.txt
To explain:
-n silences normal output.
'0,/CELL EDGE/! says to ignore everything from the beginning of the file up to and including the line containing "CELL EDGE".

Then the remaining text is filtered by a nested expression...
/Xfr/,/^\*/ , which says to match the lines from "Xfr" to the next line starting with a *.

Then in another nested expression...
/\*$/d;p deletes the second starred line (actually, any line that has a star as the final character), and prints the remainder, which is piped into the output file.


Getting just the numbered lines is pretty much the same. I just changed the final expression to remove all lines that begin with a star instead:
Code:
sed -n '0,/CELL EDGE/! {/Xfr/,/^\*/ {/^\*/d;p}}' octoberdata-1.txt >outputfile.txt
Caveat: This probably only works right with gnu sed. Other versions may not be supported.
Also, you may want to modify the matching expressions, if you think they may conflict with other parts of the file. Just choose a string that's unique to that line.

Now just set up a scripting loop to cycle it through your files and you should have it.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
AWK/Perl for extracting data from txt file to numerous other files briana.paige Linux - Newbie 2 05-05-2009 09:53 AM
Extracting chunks of data based on variables stored in another file (Perl?) mchriste Linux - Software 2 03-12-2009 12:44 PM
extracting data from html files into one text file adityavpratap Slackware 9 05-10-2007 10:30 AM
Extracting data from file using sed EneWolverine Programming 7 12-29-2006 09:23 AM
help extracting data from csv file willinusf Linux - General 10 10-27-2006 09:10 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 12:00 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration