LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-20-2011, 01:42 AM   #1
nandi
LQ Newbie
 
Registered: Oct 2011
Posts: 7

Rep: Reputation: Disabled
parse particular column in csv


Hi

can anyone give me idea how i can parse particular column in a csv file. say file has 50 column and i need to check column number 1 5 7 8 if they contain value or not.
i'm not getting on conclusion that what i should use.
give me some idea how i can do that...

Thanks....
 
Old 12-20-2011, 03:23 AM   #2
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 14,833

Rep: Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820
Probably awk, with the comma specified as field separator - then you can just test $1, $5, $7 and $8 (say).
KISS.
 
Old 12-20-2011, 05:46 AM   #3
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,576
Blog Entries: 31

Rep: Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195
Awk would be a good tool. It gets more complex if the CSV file includes fields with values including the field separator and a lot more complex if the values include line ends.
 
Old 12-20-2011, 06:02 AM   #4
nandi
LQ Newbie
 
Registered: Oct 2011
Posts: 7

Original Poster
Rep: Reputation: Disabled
thanks guys...

csv just contains values only..
as i'm totally in learning phase of unix scripting so dn't know much...can you please tell me how to test particular column using awk.....give me a hint or somthing like....
thanks....
 
Old 12-20-2011, 06:04 AM   #5
aazkan
Member
 
Registered: Jan 2008
Posts: 72

Rep: Reputation: 5
*hint*

cat somefile.txt|awk '{print $1}'

should show the 1st column
 
Old 12-20-2011, 06:17 AM   #6
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 14,833

Rep: Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820
Look at the awk doco - and try this - gets a few recommendations, although maybe a bit old.

This may get you started
Code:
awk -F"," '$2== two && $5 == 5 {print}' file.csv
 
Old 12-20-2011, 06:44 AM   #7
nandi
LQ Newbie
 
Registered: Oct 2011
Posts: 7

Original Poster
Rep: Reputation: Disabled
its printing nothing......i tried after putting error in csv....but no result
 
Old 12-20-2011, 06:46 AM   #8
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 14,833

Rep: Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820Reputation: 1820
Read the first sentence. First.
 
1 members found this post helpful.
Old 12-20-2011, 12:19 PM   #9
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
If you would post an actual example of the input text, and spell out exactly what you want to test for, we could probably help you come up with the solution you need.

And don't forget to use [code][/code] tags around your code and data, to preserve formatting and to improve readability.
 
Old 12-21-2011, 05:42 AM   #10
nandi
LQ Newbie
 
Registered: Oct 2011
Posts: 7

Original Poster
Rep: Reputation: Disabled
Hi!

spelling out my requirements correctly:
There is one prop file which is like a constraint file. which contains keys to tell which are mandatory columns in a csv file.
Say MAN_COL=1,2,5,6

Now a script need to check the prop file first then to pick the value about mandatory columns.
Then it will go to csv file. Read it line by line. Check the mandatory columns if they have some value in it means they are not empty or null.
if they are null script will send a mail to some address.
otherwise it will return 0.

As m new n learning from ground i tried commands here m posting those one.
These are just commands need to combine them to make a working script for my requirement.
I know many things are missing in the code....
help needed..
Here is the code tried by me:
Code:
#!/bin/bash
#while loop to read line by line
N=0
While read LINE ; do
N=$((N+1))
echo “Line $N = $LINE
done

# cut command to extract columns
cut –f 1,3,4 –d ‘|’ Jewson.csv

#if else clause to check the columns if they contain something or empty
if ($1==[a-zA-Z][0-9] && $2==[a-zA-Z][0-9] && $3==$1==[a-zA-Z][0-9])
then
return 0
else
#send mail to admin {code for sendmail server}
fi

Thanks a lot....please lemme know
I need to know how i will send output of one command to next one in above scenario. How i will i make it a working script....
n do lemme know Am I a little bit on right track or am just on a diff track in diff world.....
 
Old 12-21-2011, 07:25 AM   #11
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,576
Blog Entries: 31

Rep: Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195
This requirement can be broken up and addressed in digestible pieces. A bash script can be seen as automating a sequence of commands that could be entered at the command prompt. The "digestible pieces" can be explored at the command prompt first and then a working solution built into the script.
Quote:
Originally Posted by nandi View Post
There is one prop file which is like a constraint file. which contains keys to tell which are mandatory columns in a csv file.
Say MAN_COL=1,2,5,6

Now a script need to check the prop file first then to pick the value about mandatory columns.

...

if they are null script will send a mail to some address.
Does the prop file contain a line containing only something like MAN_COL=1,2,5,6? Are there always four mandatory fields?

For the mailing part, are you able to send mails from the system "to some address" at the command line?
Quote:
Originally Posted by nandi View Post
As m new n learning from ground i tried commands here m posting those one.
These are just commands need to combine them to make a working script for my requirement.
I know many things are missing in the code....
help needed..
Here is the code tried by me:
Code:
#!/bin/bash
#while loop to read line by line
N=0
While read LINE ; do
N=$((N+1))
echo “Line $N = $LINE
done
That loop has no stdin supplied so will try to read user input from the terminal. Could be fixed with something like done < input.txt. N=$((N+1)) is not wrong but the same thing can be done more simply by ((N++)). Strings started with " must be ended with a matching ".

Quote:
Originally Posted by nandi View Post
Code:
# cut command to extract columns
cut –f 1,3,4 –d ‘|’ Jewson.csv

#if else clause to check the columns if they contain something or empty
if ($1==[a-zA-Z][0-9] && $2==[a-zA-Z][0-9] && $3==$1==[a-zA-Z][0-9])
then
return 0
else
#send mail to admin {code for sendmail server}
fi
The cut hopefully gets the fields you want. Running it like that (in the script or at the command prompt) you will be able to see what it does generate. How the output can be assigned to script variables depends on what the output is so it would help if you posted it.

The emptiness test is not valid bash for several reasons. The == comparison operator must be surrounded by whitespace; spaces are the usual choice. What are $1, 2 1nd 3 intended to be? In bash, except in a function they are the arguments given when the script is called.

It looks as if you are not just checking for emptiness but for valid values. What do you actually need to do? If it's an emptiness test, use something like $field == "". If you want to test for valid values, use $field =~ <some regular expression>

Either way, the [[ ... ]] test is easier to work with than the old [ ... ] test for reasons explained here.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] validating mandatory column value in csv file using script sukhdip Linux - Newbie 2 01-31-2012 03:45 AM
awk - rearrange column data in csv file to match columns wolverene13 Programming 9 12-21-2011 05:55 AM
[SOLVED] using awk to parse csv and exclude field range rylincoln Linux - Software 3 09-13-2011 12:26 AM
How to redirect the output to a different column in .csv file Sayan Acharjee Linux - General 2 04-25-2011 12:43 PM
Rearrange the column in CSV file.. govi1234 Linux - Newbie 9 08-27-2010 08:15 PM


All times are GMT -5. The time now is 10:29 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration