[SOLVED] Script/Command to delete a line from a text file when a column value is null

Buddhike G · 06-11-2014, 09:45 PM

Hi All,

sorry if i'm posting in a wrong forum.

I have a text file with ';' delimited entries as follows

1;2;3;4
234;234;45;5757;454;5335
2;3;4;5;6;7
33;44;55;66;77;88;99
345;553;33;55
3;e;4;6
4;5;7;8

I need a command/script to delete all the lines which dont have a value at 5th column.

so the command/script should delete following lines

1;2;3;4
3;e;4;6
4;5;7;8
345;553;33;55

so after running the command/script the remaining entries in the file should be

234;234;45;5757;454;5335
2;3;4;5;6;7
33;44;55;66;77;88;99

Kindly help to get the desired output

Thank You in Advance !

evo2 · 06-11-2014, 10:36 PM

Hi,

lots of ways to do this. Here is one with sed that also checks to make sure the entries are integers.

Code:

sed -in '/[0-9]*;[0-9]*;[0-9]*;[0-9]*;[0-9]*/p' foo.txt

NB.
- This will actually modify foo.txt (as you requested)- Test carefully first
- The sample output you posted is wronge: the last line has five entries

Evo2.

syg00 · 06-11-2014, 10:39 PM

And there is also no null 5th "column" anywhere.

Would have been better IMHO for the OP to provide his/her tries that we could help correct in need.

Buddhike G · 06-11-2014, 11:18 PM

Hi All,

Thank You for the quick responses

Hi evo2

sorry for putting the 345;553;33;55 line in output. I've edited the original post to correct the error now.

I tried your suggested method and its working fine for my requirement.
Actually I've created this as a example to simplify my issue. In actual scenario I have around 35 columns in one line.And I want to remove the lines whit the empty values in column 35. so with that my command will be very long. anyway I will try that one with the real text file too

Hi syg00

I thought we can say the value of the column is null when there is no fifth column in a line.
So does that mean the value of the 5th column="0" for those lines which doesn't have 5th column ?

Kindly advise

syg00 · 06-11-2014, 11:29 PM

I would expect something like the following for a null field

Code:

2;3;4;5;;6;7

awk has good support for this sort of work - it has builtin field names/numbers that can be easily tested for null in this case.

grail · 06-11-2014, 11:57 PM

In addition to syg00's advise on awk, have a look at the NF built-in variable as it makes this a simple test expression and print scenario.

And in case you haven't got it, this is the main resource to find everything awk related

Buddhike G · 06-12-2014, 09:30 PM

Thank You all for your replies

Quote:

Originally Posted by syg00

I would expect something like the following for a null field

Code:

2;3;4;5;;6;7

awk has good support for this sort of work - it has builtin field names/numbers that can be easily tested for null in this case.

so if the column values are as follows

Code:

3;e;4;6

then what is the value of column 5 ? will it be zero ? or null ?

i've tried the following command

Code:

awk -F";" '$5>0' test.txt

i get the correct output for the test scenario as follows

Code:

2;3;4;5;6;7
33;44;55;66;77;88;99
234;234;45;5757;454;5335

still wondering $5>0 is the correct comparison method to use

kindly advise

syg00 · 06-12-2014, 09:45 PM

"correct" is usually debatable - that could hide some obscure cases. Personally I prefer to make tests definitive - in this case "at least 5 fields", and (maybe) "5th field not null" as you specified in the subject line. Have a look at this

Code:

awk -F";" 'NF > 4 && $5 != "" {print}' test.txt

Buddhike G · 06-13-2014, 02:03 AM

Thank you for all your replies,

syg00's solution is much suitable for serving my requirement.

I will close this thread as solved.

All your thoughts are highly appreciated !