Delete lines using awk

kkjegan · 09-10-2007, 01:01 PM

Hi all
I am looking for the solution to delete the entire row from a file if it matches a 'null' value in a particular column.

suppose for example , I have a file

one two three four
five six eight
nine ten eleven
one two four

from the above file, i have to delete the row whose 3rd column is 'null'. i.e. in the above example, it has to remove 2nd and 4th row as it contains null in 3rd column. not the 3rd row as it contains null in 4th column.

please help me to do it

Thanks in advance

pixellany · 09-10-2007, 01:52 PM

the way you describe the file, there does not appear to be any difference between the last 3 rows (lines)--ie what character is there where say the "null" is. (maybe ascii 0x00?).

Awk reads into records using a delimiter to define the boundaries. In this case, it would presumably default to "space" as the default. After reading a line, simply run a test on the four records: $1, $2, $3, and $4

My favorite AWK tutorial (and SED) is here: http://www.grymoire.com/Unix/

trashbird1240 · 09-10-2007, 03:00 PM

I'm surprised you didn't find an example of exactly what you want in the awk reference you're using.

You need some kind of field delimiter, other than whitespace, if
you're going to delete an "empty" field, unless one of your words represents "null."

I use commas. Many system administration files use colons.

Joel

frenchn00b · 09-10-2007, 03:33 PM

Quote:

Originally Posted by trashbird1240

I'm surprised you didn't find an example of exactly what you want in the awk reference you're using.

You need some kind of field delimiter, other than whitespace, if
you're going to delete an "empty" field, unless one of your words represents "null."

I use commas. Many system administration files use colons.

Joel

awk with some if stuffs and NR ...http://www.gnu.org/software/gawk/manual/gawk.pdf

PTrenholme · 09-10-2007, 03:45 PM

If your columns occur at fixed positions in the record, you might want to look at using the gawk FIELDWIDTHS variable to split the data into columns.

See the "Constant Size" subsection of the "Reading Files" section in info gawk for details.

kkjegan · 09-10-2007, 04:07 PM

Thanks for your reply...

Actually I have given space in between the columns. But it is combined with the next column. Let me explain the problem clearly.
I have file with 50 columns and value of many columns are null(it is not in order).I need to delete the whole row whereever the value of the 30th column is null.Many columns may have null value.But i want to delete the row which has the 30th column null value. Hope everyone got the problem now.
I dont have gawk in my system.
please help me to solve this with awk.

Tahnks in advance
Jegan

frenchn00b · 09-10-2007, 04:41 PM

example:

Code:

cat /tmp/myfile | awk '  BEGIN {   # hello
i=1 ; 
j=1 ;
parameters="";
middlepart=1; 
}

{ 

if ( NR == 1)  {  parameters=$0  }

if (( index($1,"thisisthemiddlepart") == 0 ) && (NR > 1) ) { 

if ( middlepart == 1  )  { 

n=split($1, vk , "=" ) ; 
a[i,1]=vk[1] ;
a[i,2]=vk[2] ;
i++ ; 
 }

bla bla bla

kkjegan · 09-10-2007, 05:02 PM

Thanks.
Hope this will work.
Let me try this

Jegan

ghostdog74 · 09-10-2007, 06:44 PM

Quote:

Originally Posted by frenchn00b

example:

Code:

cat /tmp/myfile | awk '  BEGIN {   # hello
i=1 ; 
j=1 ;
parameters="";
middlepart=1; 
}

{ 

if ( NR == 1)  {  parameters=$0  }

if (( index($1,"thisisthemiddlepart") == 0 ) && (NR > 1) ) { 

if ( middlepart == 1  )  { 

n=split($1, vk , "=" ) ; 
a[i,1]=vk[1] ;
a[i,2]=vk[2] ;
i++ ; 
 }

bla bla bla

no need for cat!!

Code:

awk 'BEGIN{}...' /tmp/myfile

ghostdog74 · 09-10-2007, 06:49 PM

Quote:

Originally Posted by kkjegan

Hi all
I am looking for the solution to delete the entire row from a file if it matches a 'null' value in a particular column.

suppose for example , I have a file

one two three four
five six eight
nine ten eleven
one two four

from the above file, i have to delete the row whose 3rd column is 'null'. i.e. in the above example, it has to remove 2nd and 4th row as it contains null in 3rd column. not the 3rd row as it contains null in 4th column.

please help me to do it

Thanks in advance

if your delimiter is a blank space(or tab), awk would not know where where null value is. The only way i can think of now is getting the correct number of fields, check the number of fields in each row and compare against the actual value. However, this method, you will only know which rows have null values, but would not know which columns. ( i may be wrong though ).
Its best if you could get your source to change to some other delimiters, such as commas...

kkjegan · 09-11-2007, 10:28 AM

Quote:

Originally Posted by ghostdog74

if your delimiter is a blank space(or tab), awk would not know where where null value is. The only way i can think of now is getting the correct number of fields, check the number of fields in each row and compare against the actual value. However, this method, you will only know which rows have null values, but would not know which columns. ( i may be wrong though ).
Its best if you could get your source to change to some other delimiters, such as commas...

Yes.That is the problem now.If it is other delimeter, it is very easy to make it. also each row may have null value in different columns.so it is very diffult to use number of columns also.

kkjegan · 09-11-2007, 10:31 AM

Quote:

Originally Posted by ghostdog74

no need for cat!!

Code:

awk 'BEGIN{}...' /tmp/myfile

Is it need to have this big coding?. Can't we do it in a single line command? becos awk is meant for it. is it not?

ghostdog74 · 09-11-2007, 10:53 AM

Quote:

Originally Posted by kkjegan

Is it need to have this big coding?. Can't we do it in a single line command? becos awk is meant for it. is it not?

it depends on the complexity of your problem. Also, cramming code that is meant to solve complex problems into single line does not equate to being "cool". It makes code unreadable and difficult to troubleshoot.

chrism01 · 09-11-2007, 07:36 PM

Hey ghostdog74, you're d*mn right. I really, really hate people who do that.