LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 02-23-2010, 05:03 PM   #1
AlexYZ
LQ Newbie
 
Registered: Sep 2009
Posts: 3

Rep: Reputation: 0
AWK - combining multiple columns


Hi

This is probably something really simple but my brains gone to mush :-(

I have a folder with only 24 files named <number>.dat (i.e. 4.dat, 6.dat and so on) where <number> is between 0 and 256. Each file has just two columns of data and nothing else.

I'm trying to combine all the second columns ($2) together. I've been fiddling around with getline and so far have

awk '{ getline ln < "6.dat" ; print ln" "$2 }' 4.dat

which takes file 4.dat and adds $2 from 6.dat, but I want a single command to take each $2 from every file and add them to (for example) 4.dat (having $1 from 4.dat is no problem). A command that takes every file in the folder and grabs $2 and places them in a common file would be ideal. Frankly I can work around if you combine both columns from every file.

Thanks,

Al
 
Old 02-23-2010, 05:19 PM   #2
AnanthaP
Member
 
Registered: Jul 2004
Location: Chennai, India
Distribution: UBUNTU 5.10 since Jul-18,2006 on Intel 820 DC
Posts: 644

Rep: Reputation: 153Reputation: 153
Student?

awk can input all *.dat files in 1 shot - see the syntax. You can print the second variable and redirect the output to a file.

What have you got so far?

End
 
Old 02-23-2010, 11:10 PM   #3
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,587

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
how about:

cd into directory

for file in *; do
awk '{ print $2 }' $file >> combine.dat;
done

Last edited by grail; 02-24-2010 at 11:22 PM. Reason: Should remember to type what I wrote <doh>
 
Old 02-24-2010, 05:47 AM   #4
AlexYZ
LQ Newbie
 
Registered: Sep 2009
Posts: 3

Original Poster
Rep: Reputation: 0
Thanks for responses, eventually I found a workaround using paste, but it would be good to see what I'm missing. That said I'm out of time to look at this myself

AnanthaP - I've tried various combinations of the appropriate syntax (starting with wild card and moving on to [0-9]). For example


Code:
awk '{ getline ln < /^[1-9].dat/ ; print ln" "$2 }' 4.dat > combined.dat
which only takes $2 from 4.dat, while


Code:
awk '{ getline ln < /^[1-9].dat/ ; print ln" "$2 }' *.dat > combined.dat
combined $2 but all into a single column.

grail, thanks, although for me

Code:
for file in *; do
awk '{ print $2 }' $f >> combine.dat;
done
just hangs.



Alex
 
Old 02-24-2010, 06:38 AM   #5
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
Quote:
That said I'm out of time to look at this myself
Does that mean we are supposed to finish it?

In the suggestion from grail, I think the "$f" is supposed to be "$file"
 
Old 02-24-2010, 07:09 AM   #6
AlexYZ
LQ Newbie
 
Registered: Sep 2009
Posts: 3

Original Poster
Rep: Reputation: 0
Quote:
Does that mean we are supposed to finish it?
nono not at all, quite the opposite, I don't want people to think I'd just given up trying and was waiting for an answer - I don't like leaving things but equally I only have a few days left with access to the system I'm working on and considering I've found a work around I'm just going to use that.

Replacing $f with $file makes it run, but it oupts everything to a single column rather than multiple ones (which is why I was using getline).

Many thanks
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
printing multiple columns with awk kdelover Programming 16 12-16-2009 09:10 AM
Using awk to switch columns bioinformatics_guy Linux - Newbie 3 10-30-2008 09:50 AM
awk print correct lines when certain columns are blank schneidz Programming 11 04-04-2008 04:06 PM
Moving columns with sed or awk? btm Linux - Newbie 4 09-27-2007 02:03 PM
awk command to merge columns from two separate files into single file? johnpaulodonnell Linux - Newbie 4 01-23-2007 10:10 AM


All times are GMT -5. The time now is 08:04 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration