LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 05-26-2011, 05:14 AM   #1
mauludi
LQ Newbie
 
Registered: Jan 2010
Posts: 21

Rep: Reputation: 0
awk: how to read dan manipulate data in multi files from a file


Dear All,

I wonder capability of awk to manipulate data in consecutive multi files by read one batch file.
for example I have files: data1.dat, data2.dat,data3.dat and listfile.txt

cat data1.dat
23 34 54
43 45 76

cat data2.dat
76 64 92
43 76 89

cat data3.dat
65 83 92
37 94 90


cat listfile.txt
data1.dat
data2.dat
data3.dat

how to make awk script which can manipulate/manage data in data1,data2,data3 by read listfile.txt . for exampale we want to sum up all column 1 of data1.dat,data2.dat and data3.dat

thank you in advance
 
Old 05-26-2011, 05:31 AM   #2
Guttorm
Senior Member
 
Registered: Dec 2003
Location: Trondheim, Norway
Distribution: Debian and Ubuntu
Posts: 1,261

Rep: Reputation: 297Reputation: 297Reputation: 297
Hi

You can use many files with awk, and the shell can use cat to get the file list. For example:

Code:
awk ... $(cat listfile.txt)
 
Old 05-26-2011, 05:44 AM   #3
sycamorex
LQ Veteran
 
Registered: Nov 2005
Location: London
Distribution: Slackware64-current
Posts: 5,811
Blog Entries: 1

Rep: Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191Reputation: 1191
Or even without creating a separate file:

Code:
awk .... $(ls -1 /path/to/data*)
Please note that in "ls -1" it's number 1, not lower case L.
 
Old 05-26-2011, 06:52 AM   #4
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,251

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
Assuming that the files in listfile.txt are not all in the same location:
Code:
awk '{f = $0;while(getline < f)sum+=$1}END{print sum}' listfile.txt
 
Old 05-27-2011, 02:52 AM   #5
mauludi
LQ Newbie
 
Registered: Jan 2010
Posts: 21

Original Poster
Rep: Reputation: 0
thank you for your nice explanation

sorry for ask more question, for example in each data file (data1.dat,data2.dat,data3.dat),there are not only one list of data they consist several list of data separated by some sentence. It look like:

cat data1.dat
temperature pressure stress
23 34 54
43 45 76
type charge
34 54
45 76

data2.dat and data3.dat are the same format.

If we only one to take data of type and charge, how to extract it. thank you very much.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
want to open & manipulate column data from excel file. rinu budhbhatti LinuxQuestions.org Member Intro 1 02-01-2011 09:50 AM
Get data from multi lined text file using awk, sed or perl - grep & cut not upto par cam34 Programming 4 07-02-2010 04:10 AM
AWK/Perl for extracting data from txt file to numerous other files briana.paige Linux - Newbie 2 05-05-2009 10:53 AM
manipulate dcop data for File commandS? carl0ski Programming 2 07-02-2005 09:46 PM


All times are GMT -5. The time now is 07:02 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration