LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 09-10-2007, 01:01 PM   #1
kkjegan
LQ Newbie
 
Registered: Sep 2007
Location: New Jersey
Posts: 5

Rep: Reputation: 0
Delete lines using awk


Hi all
I am looking for the solution to delete the entire row from a file if it matches a 'null' value in a particular column.

suppose for example , I have a file

one two three four
five six eight
nine ten eleven
one two four

from the above file, i have to delete the row whose 3rd column is 'null'. i.e. in the above example, it has to remove 2nd and 4th row as it contains null in 3rd column. not the 3rd row as it contains null in 4th column.

please help me to do it

Thanks in advance
 
Old 09-10-2007, 01:52 PM   #2
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
the way you describe the file, there does not appear to be any difference between the last 3 rows (lines)--ie what character is there where say the "null" is. (maybe ascii 0x00?).

Awk reads into records using a delimiter to define the boundaries. In this case, it would presumably default to "space" as the default. After reading a line, simply run a test on the four records: $1, $2, $3, and $4

My favorite AWK tutorial (and SED) is here: http://www.grymoire.com/Unix/
 
Old 09-10-2007, 03:00 PM   #3
trashbird1240
Member
 
Registered: Sep 2006
Location: Durham, NC
Distribution: Slackware, Ubuntu (yes, both)
Posts: 463

Rep: Reputation: 31
I'm surprised you didn't find an example of exactly what you want in the awk reference you're using.

You need some kind of field delimiter, other than whitespace, if
you're going to delete an "empty" field, unless one of your words represents "null."

I use commas. Many system administration files use colons.

Joel
 
Old 09-10-2007, 03:33 PM   #4
frenchn00b
Senior Member
 
Registered: Jun 2007
Location: E.U., Mountains :-)
Distribution: Debian, Etch, the greatest
Posts: 2,546

Rep: Reputation: 51
Quote:
Originally Posted by trashbird1240 View Post
I'm surprised you didn't find an example of exactly what you want in the awk reference you're using.

You need some kind of field delimiter, other than whitespace, if
you're going to delete an "empty" field, unless one of your words represents "null."

I use commas. Many system administration files use colons.

Joel
awk with some if stuffs and NR ...http://www.gnu.org/software/gawk/manual/gawk.pdf
 
Old 09-10-2007, 03:45 PM   #5
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,150

Rep: Reputation: 330Reputation: 330Reputation: 330Reputation: 330
If your columns occur at fixed positions in the record, you might want to look at using the gawk FIELDWIDTHS variable to split the data into columns.

See the "Constant Size" subsection of the "Reading Files" section in info gawk for details.
 
Old 09-10-2007, 04:07 PM   #6
kkjegan
LQ Newbie
 
Registered: Sep 2007
Location: New Jersey
Posts: 5

Original Poster
Rep: Reputation: 0
Thanks for your reply...

Actually I have given space in between the columns. But it is combined with the next column. Let me explain the problem clearly.
I have file with 50 columns and value of many columns are null(it is not in order).I need to delete the whole row whereever the value of the 30th column is null.Many columns may have null value.But i want to delete the row which has the 30th column null value. Hope everyone got the problem now.
I dont have gawk in my system.
please help me to solve this with awk.

Tahnks in advance
Jegan
 
Old 09-10-2007, 04:41 PM   #7
frenchn00b
Senior Member
 
Registered: Jun 2007
Location: E.U., Mountains :-)
Distribution: Debian, Etch, the greatest
Posts: 2,546

Rep: Reputation: 51
example:
Code:
cat /tmp/myfile | awk '  BEGIN {   # hello
i=1 ; 
j=1 ;
parameters="";
middlepart=1; 
}

{ 

if ( NR == 1)  {  parameters=$0  }

if (( index($1,"thisisthemiddlepart") == 0 ) && (NR > 1) ) { 

if ( middlepart == 1  )  { 

n=split($1, vk , "=" ) ; 
a[i,1]=vk[1] ;
a[i,2]=vk[2] ;
i++ ; 
 }

bla bla bla
 
Old 09-10-2007, 05:02 PM   #8
kkjegan
LQ Newbie
 
Registered: Sep 2007
Location: New Jersey
Posts: 5

Original Poster
Rep: Reputation: 0
Thanks.
Hope this will work.
Let me try this

Jegan
 
Old 09-10-2007, 06:44 PM   #9
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,695
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Quote:
Originally Posted by frenchn00b View Post
example:
Code:
cat /tmp/myfile | awk '  BEGIN {   # hello
i=1 ; 
j=1 ;
parameters="";
middlepart=1; 
}

{ 

if ( NR == 1)  {  parameters=$0  }

if (( index($1,"thisisthemiddlepart") == 0 ) && (NR > 1) ) { 

if ( middlepart == 1  )  { 

n=split($1, vk , "=" ) ; 
a[i,1]=vk[1] ;
a[i,2]=vk[2] ;
i++ ; 
 }

bla bla bla
no need for cat!!
Code:
awk 'BEGIN{}...' /tmp/myfile
 
Old 09-10-2007, 06:49 PM   #10
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,695
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Quote:
Originally Posted by kkjegan View Post
Hi all
I am looking for the solution to delete the entire row from a file if it matches a 'null' value in a particular column.

suppose for example , I have a file

one two three four
five six eight
nine ten eleven
one two four

from the above file, i have to delete the row whose 3rd column is 'null'. i.e. in the above example, it has to remove 2nd and 4th row as it contains null in 3rd column. not the 3rd row as it contains null in 4th column.

please help me to do it

Thanks in advance
if your delimiter is a blank space(or tab), awk would not know where where null value is. The only way i can think of now is getting the correct number of fields, check the number of fields in each row and compare against the actual value. However, this method, you will only know which rows have null values, but would not know which columns. ( i may be wrong though ).
Its best if you could get your source to change to some other delimiters, such as commas...
 
Old 09-11-2007, 10:28 AM   #11
kkjegan
LQ Newbie
 
Registered: Sep 2007
Location: New Jersey
Posts: 5

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by ghostdog74 View Post
if your delimiter is a blank space(or tab), awk would not know where where null value is. The only way i can think of now is getting the correct number of fields, check the number of fields in each row and compare against the actual value. However, this method, you will only know which rows have null values, but would not know which columns. ( i may be wrong though ).
Its best if you could get your source to change to some other delimiters, such as commas...

Yes.That is the problem now.If it is other delimeter, it is very easy to make it. also each row may have null value in different columns.so it is very diffult to use number of columns also.
 
Old 09-11-2007, 10:31 AM   #12
kkjegan
LQ Newbie
 
Registered: Sep 2007
Location: New Jersey
Posts: 5

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by ghostdog74 View Post
no need for cat!!
Code:
awk 'BEGIN{}...' /tmp/myfile
Is it need to have this big coding?. Can't we do it in a single line command? becos awk is meant for it. is it not?
 
Old 09-11-2007, 10:53 AM   #13
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,695
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Quote:
Originally Posted by kkjegan View Post
Is it need to have this big coding?. Can't we do it in a single line command? becos awk is meant for it. is it not?
it depends on the complexity of your problem. Also, cramming code that is meant to solve complex problems into single line does not equate to being "cool". It makes code unreadable and difficult to troubleshoot.
 
Old 09-11-2007, 07:36 PM   #14
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,269

Rep: Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028
Hey ghostdog74, you're d*mn right. I really, really hate people who do that.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
find awk sed.. something along these lines citrus Linux - General 1 08-21-2006 03:04 PM
awk/gawk/sed - read lines from file1, comment out or delete matching lines in file2 rascal84 Linux - General 1 05-24-2006 09:19 AM
awk print lines that doesn't have a pattern huynguye Programming 5 05-04-2006 11:08 AM
awk text that is on several lines homey Programming 2 10-31-2004 09:27 AM
counting the commented lines using awk [ /* */] itsjvivek Linux - General 8 01-17-2003 08:30 AM


All times are GMT -5. The time now is 12:54 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration