[SOLVED] awk: filter out fields that are blank or are spaces

schneidz · 04-08-2016, 07:34 AM

here are the results:

Code:

[schneidz@hyper ~]$ awk 'NF == 2' p12345.tab #1
hello	world
all3r	sp1ca
l33t	h4x0rz
boston	celtics
[schneidz@hyper ~]$ awk -F"\t" 'NF != 1 && $2 !~ /          |^$/' p12345.tab #2
hello	world
all3r	sp1ca
l33t	h4x0rz
boston	celtics
[schneidz@hyper ~]$ awk -F '\t' '{ picker = $2; gsub(" .*", ""); if(picker != "") print $1 FS $2 }' p12345.tab #3
hello	world
stanley	
all3r	sp1ca
l33t	h4x0rz
boston	celtics
[schneidz@hyper ~]$ awk -F '\t' '(index($2, "        ") == 0) && (index($2, "") == 0) {print $1 "\t" $2}' p12345.tab #4
[schneidz@hyper ~]$ awk -F '\t' '$2 != "        " && $2 != "" {print $1 "\t" $2}' p12345.tab #5
hello	world
stanley	          
all3r	sp1ca
l33t	h4x0rz
boston	celtics

in retrospeck, my filtered down example was not representative of the original data so #1 and #2 will not filter out some appropriate lines.

edit: this seems to work in the original data that ocntains 9 fields but not the test i created for some reason:

Code:

[schneidz@hyper ~]$ awk -F '\t' '(index($2, "        ") == 0) && (index($2, "") != 0) {print $1 "\t" $2}' p12345.tab
hello	world
all3r	sp1ca
l33t	h4x0rz
88888888	
boston	celtics

grail · 04-08-2016, 05:19 PM

Quote:

my filtered down example was not representative of the original data so #1 and #2 will not filter out some appropriate lines.

Obviously difficult to tailor a solution if you do not show the type of data. I am curious if the red information shown are errors, ie not supposed to be shown, or are more
correctly showing what you were looking for?

It is also unclear if #4 is not working as that also seems to be the same as the final script shown which does produce output?

schneidz · 04-08-2016, 07:23 PM

Quote:

Originally Posted by grail

Obviously difficult to tailor a solution if you do not show the type of data. I am curious if the red information shown are errors, ie not supposed to be shown, or are more
correctly showing what you were looking for?

It is also unclear if #4 is not working as that also seems to be the same as the final script shown which does produce output?

red are errors; in #4 this didnt work :

Code:

(index($2, "") == 0)

but i changed it to this:

Code:

(index($2, "") != 0)

its healthcare information so i am anxious to not just copy-pasta it.

grail · 04-08-2016, 08:09 PM

ahh ... the old not equal to switcheroo .. hehe

Yes I certainly don't mean to place the exact data if it is sensitive, just that making a script to do the job would need to see the same type of fields

You have a working solution though, so all good