LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-28-2016, 08:24 PM   #1
Mike_Brown
Member
 
Registered: May 2015
Posts: 37

Rep: Reputation: Disabled
How to remove the columns which contains NA in linux


I would like to remove the column which contains any number of NA. I used this command

Code:
awk ' $0 !="NA" {print $0}' file
But it does not work.
For example, the file is as following

1 2 3 NA 6 male
4 6 2 1 NA female
NA 2 2 NA 3 male
7 2 2 7 NA male

I want to the output file as

2 3 male
6 2 female
2 2 male
2 2 male
 
Old 03-28-2016, 08:56 PM   #2
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,844

Rep: Reputation: 1478Reputation: 1478Reputation: 1478Reputation: 1478Reputation: 1478Reputation: 1478Reputation: 1478Reputation: 1478Reputation: 1478Reputation: 1478
Well, if you read the documentation, $0 is the input record. $1 is the first field, $2 is the second...

So instead of print $0, try print $2 $3 $7
 
1 members found this post helpful.
Old 03-28-2016, 09:02 PM   #3
Mike_Brown
Member
 
Registered: May 2015
Posts: 37

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by jpollard View Post
Well, if you read the documentation, $0 is the input record. $1 is the first field, $2 is the second...

So instead of print $0, try print $2 $3 $7
I do not know which columns contains NA. Then, I do not know $2 $3 $7

Last edited by Mike_Brown; 03-28-2016 at 09:03 PM.
 
Old 03-28-2016, 10:26 PM   #4
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 18,481

Rep: Reputation: 3097Reputation: 3097Reputation: 3097Reputation: 3097Reputation: 3097Reputation: 3097Reputation: 3097Reputation: 3097Reputation: 3097Reputation: 3097Reputation: 3097
This is not a trivial exercise - you would do well to take note and read the doco.
For example, you cannot know all the columns that contain the string until you have read the entire file. Having saved a list of those columns you will have to re-read the entire file to ascertain which fields you still want. Or you could keep each record in an array for later processing.

In the doco you will find an inbuilt variable that tells you the number of fields in the current record - which you can loop through to test each for the string.
 
Old 03-29-2016, 09:43 AM   #5
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware FreeBSD Win10 MX
Posts: 9,593

Rep: Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069
awk
http://www.cyberciti.biz/faq/howto-d...-bsd-appleosx/

http://how-to.linuxcareer.com/learni...x-commands-awk

grep
http://www.cyberciti.biz/faq/howto-u...in-linux-unix/

sed
http://www.cyberciti.biz/faq/howto-d...-bsd-appleosx/

that should be enough to get you started down the right path. Look closely at the methodology of each function.

Last edited by BW-userx; 03-29-2016 at 09:53 AM.
 
Old 03-29-2016, 12:31 PM   #6
HMW
Member
 
Registered: Aug 2013
Location: Sweden
Distribution: Debian, Arch, Red Hat, CentOS
Posts: 773
Blog Entries: 3

Rep: Reputation: 369Reputation: 369Reputation: 369Reputation: 369
I have to confess that I don't understand the description of the problem. Nevertheless, given your INPUT
Code:
1 2 3 NA 6 male
4 6 2 1 NA female
NA 2 2 NA 3 male
7 2 2 7 NA male
And your expected OUTPUT
Code:
2 3 male
6 2 female
2 2 male
2 2 male
I solved this using sed:
Code:
echo "1 2 3 NA 6 male
4 6 2 1 NA female
NA 2 2 NA 3 male
7 2 2 7 NA male" | sed '<hidden>'
2 3 male
6 2 female
2 2 male
2 2 male
Will be more than happy to share my solution once OP gets back with a renewed effort and/or clarification of the problem at hand.

Best regards,
HMW
 
Old 03-29-2016, 12:56 PM   #7
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware FreeBSD Win10 MX
Posts: 9,593

Rep: Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069Reputation: 2069
Quote:
Originally Posted by HMW View Post
Best regards,
HMW
off topic while you wait. But does HMW mean "Her Majesty's Wardrobe"? I was just trying to figure out that acronym. NHI
 
Old 03-29-2016, 01:58 PM   #8
HMW
Member
 
Registered: Aug 2013
Location: Sweden
Distribution: Debian, Arch, Red Hat, CentOS
Posts: 773
Blog Entries: 3

Rep: Reputation: 369Reputation: 369Reputation: 369Reputation: 369
Quote:
Originally Posted by BW-userx View Post
But does HMW mean "Her Majesty's Wardrobe"? I was just trying to figure out that acronym. NHI
HAHA!

No, nothing fancy at all. It's just my initials, inspired by RMS.
 
Old 03-29-2016, 02:22 PM   #9
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 19.1 MATE
Posts: 8,018
Blog Entries: 5

Rep: Reputation: 2872Reputation: 2872Reputation: 2872Reputation: 2872Reputation: 2872Reputation: 2872Reputation: 2872Reputation: 2872Reputation: 2872Reputation: 2872Reputation: 2872
Quote:
Originally Posted by HMW View Post
I have to confess that I don't understand the description of the problem.
I think what OP wants to do is (i) only generate data for columns which do not contain NA for any of the records in the entire record set. (ii) always output the 6th column.

In the example they gave, only columns 2 and 3 did not contain NA in any of the records.

As syg00 pointed out, all the records therefore need to be read before any output can be generated (unless you determine earlier on that no columns can be printed ).
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
merge columns from multiple files in a directory based on match of two columns prasanthi yanamala Linux - Newbie 2 11-12-2015 11:11 AM
Match file1 to file2 and remove non-matched lines by columns from file3 aori Linux - General 1 06-28-2014 03:33 AM
[SOLVED] bash suggestions to convert horizontal columns to vertical columns Guyverix Programming 14 01-24-2013 12:03 PM
SQL statements howto -- 3 columns input but 2 columns output fhleung Programming 3 11-29-2012 11:45 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 07:39 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration