LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 03-09-2009, 05:30 PM   #1
Firebar
Member
 
Registered: Feb 2005
Location: Southampton (UK)
Distribution: Debian, RHEL and SuSE
Posts: 69

Rep: Reputation: 15
awk and csv file


Hi everyone,

Hoping someone can help me with my basic awk script. I have a csv file containing lines like:

alias,hostname,interface,ipaddr,netmask,gateway,mac,profile
alias2,hostname2,interface2,ipaddr2,netmask2,gateway2,mac2,profile2
alias3,hostname3,inerface3,ipaddr3,netmask3,gateway3,mac3,profile3
(and so on....)

I'm wanting to read each line and put the different columns into variables to use in a script, e.g:

Code:
alias=$( awk -F, '{ print $1 }' /root/systems.csv )
hostname=$( awk -F, '{ print $2 }' /root/systems.csv )
interface=$( awk -F, '{ print $3 }' /root/systems.csv )
ipaddr=$( awk -F, '{ print $4 }' /root/systems.csv )
netmask=$( awk -F, '{ print $5 }' /root/systems.csv )
gateway=$( awk -F, '{ print $6 }' /root/systems.csv )
mac=$( awk -F, '{ print $7 }' /root/systems.csv )
profile=$( awk -F, '{ print $8 }' /root/systems.csv )

/usr/bin/systemadd $alias $hostname $interface [....]
However, it will print all the entires in each column rather than just the first. I'd rather like it to do the above for one line at a time if possible:

Code:
while read line
do
awk -F .....
/usr/bin/systemadd $alias [....]
done
However I can't seem to make any headway. Can anyone help? Does awk have a function whereby it can read a line at a time (however I expect I wont be able to do that as I'll be breaking out of awk to run the systemadd commands..?)?

Thanks for any assistance that can be given.
 
Old 03-09-2009, 05:52 PM   #2
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora, Lubuntu, FreeBSD
Posts: 3,930
Blog Entries: 5

Rep: Reputation: Disabled
I don't know of a way to bend awk to do what you're describing, but I'll offer a simple solution: use awk to iterate through the file, and instead of actually running any commands, have awk write output to another file (which will eventually become a shell script).

In other words, the output file will contain lots of entries that look like:
Code:
/usr/bin/systemadd alias1 [....]
/usr/bin/systemadd alias2 [....]
/usr/bin/systemadd alias3 [....]
... and so on.

Afterwards, briefly preview the output file, and then run it like a shell script:
# sh /output/file

Probably much more elegant ways to accomplish this task -- i.e. with bash or with a more featured language -- but I've used what I just described on many occasions.
 
Old 03-09-2009, 06:13 PM   #3
Firebar
Member
 
Registered: Feb 2005
Location: Southampton (UK)
Distribution: Debian, RHEL and SuSE
Posts: 69

Original Poster
Rep: Reputation: 15
Thanks for your input and that makes sense. However, I think the issue I'm hitting is I'm not sure how to print the field in each record (iterate through the file) with awk, rather than just the entire column. Or do you mean perhaps grep each line out, awk it and then form the shell script?
 
Old 03-09-2009, 06:27 PM   #4
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,503

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
awk can do many things and offer the possibility to run system commands (and interact with their result). However, following your direction I'd suggest to stick with bash, by parsing the file line by line using a while loop.
Code:
while read alias hostname interface ipaddr netmask gateway mac profile
do
  /usr/bin/systemadd $alias $hostname $interface $ipaddr $netmask $gateway $mac $profile
done < <(cat file | tr "," " ")
The input to the while loop uses process substitution to pass the output of the tr command as input to the loop. The tr command just translates commas to blank spaces. The read statement can read multiple variables/fields separated by blank spaces (or better by the IFS, Input Field Separator).

If the arguments are passed to the /usr/bin/systemadd command in the same order as the input, you can just use read line and the eval statement to build and execute the command line. Something like (not tested)
Code:
while read line
do
  eval /usr/bin/systemadd $line
done < <(cat file | tr "," " ")
 
Old 03-09-2009, 09:25 PM   #5
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora, Lubuntu, FreeBSD
Posts: 3,930
Blog Entries: 5

Rep: Reputation: Disabled
@Firebar: You got a good answer from colucix already, but for the sake of posterity I'll give an example of what I was referring to.

Given a file called "data" that contains comma-separated columns value and the output file (which value should be echoed to)...
Code:
$ cat data
Hello,file1
How are you?,file2
Bye,file3

$ awk -F, '{ OFS=" " ; print "/bin/echo " $1 " > " $2 }' data > my-script

$ cat my-script 
/bin/echo Hello > file1
/bin/echo How are you? > file2
/bin/echo Bye > file3

$ sh my-script 

$ grep '.' file*
file1:Hello
file2:How are you?
file3:Bye
Rather crude, but I am sure you get the idea. awk writes the otherwise monotonous script for you. (To be fair, three rows of data is not monotonous; but three-thousand rows of data could be.)
 
Old 03-09-2009, 09:37 PM   #6
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,695
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Code:
awk -F"," '{cmd = "/usr/bin/systemadd "$1" "$2" "$3 ......; system(cmd)}' file
 
Old 03-10-2009, 01:58 AM   #7
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,503

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
If all the fields are passed as arguments to the command using the same order:
Code:
awk '{gsub(","," "); cmd ="ls -l "$0; system(cmd)}' file
 
Old 03-10-2009, 03:33 AM   #8
Firebar
Member
 
Registered: Feb 2005
Location: Southampton (UK)
Distribution: Debian, RHEL and SuSE
Posts: 69

Original Poster
Rep: Reputation: 15
Quote:
The input to the while loop uses process substitution to pass the output of the tr command as input to the loop. The tr command just translates commas to blank spaces. The read statement can read multiple variables/fields separated by blank spaces (or better by the IFS, Input Field Separator).
Wow, a very elegant way of doing it without awk. I was too busy munging some bash with awk

Quote:
Given a file called "data" that contains comma-separated columns value and the output file (which value should be echoed to)...
Thank you, that explains it clearly to me.

Thanks for the input all. I'm going to go away and give all the methods a go to try and explore some more.
 
Old 11-18-2010, 12:45 AM   #9
pavani.ventrapaati
LQ Newbie
 
Registered: Nov 2010
Posts: 1

Rep: Reputation: 0
how do i grep an entire column from a file using awk command??

how do i grep an entire column from a file using awk command??


for example i had a text file called name.txt


pavani praveena sireesha
pavithra nayanaswitha spandana
kusuma susmitha chandana


in that i want to grep entire second column

my output should be like thisraveena
nayanaswitha
susmintha


please help me with clear explanation,Thank you
 
Old 11-18-2010, 05:00 AM   #10
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,518

Rep: Reputation: 1896Reputation: 1896Reputation: 1896Reputation: 1896Reputation: 1896Reputation: 1896Reputation: 1896Reputation: 1896Reputation: 1896Reputation: 1896Reputation: 1896
As stated you could easily do all in awk, but depending on how many operations you wish to perform for each line I would follow colucix's suggestion to just keep it in bash.
As such, here is another alternative:
Code:
#!/bin/bash

while read -r
do
    set - ${REPLY//,/ }

    #each item from the file is now a positional parameter
    #ie $1 equals alias based on your post
    /usr/bin/systemadd $1 $2 ...
done<file
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
awk question on handling *.CSV "text fields" in awk jschiwal Programming 8 05-27-2010 06:23 AM
AWK: change a particular field in a csv file help help help!!!! haydar68 Programming 20 08-03-2008 01:10 AM
Comparing two csv files and write different record in third CSV file irfanb146 Linux - Newbie 3 06-30-2008 09:15 PM
CSV File AMMullan Programming 2 11-10-2003 12:49 AM


All times are GMT -5. The time now is 09:36 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration