LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-29-2012, 01:14 PM   #1
arn2025
LQ Newbie
 
Registered: Feb 2012
Posts: 26
Blog Entries: 1

Rep: Reputation: Disabled
awk


how do i search for ALL lines with field 5 having "0" in a huge txt file
 
Old 03-29-2012, 01:18 PM   #2
jhwilliams
Senior Member
 
Registered: Apr 2007
Location: Portland, OR
Distribution: Debian, Android, LFS
Posts: 1,168

Rep: Reputation: 210Reputation: 210Reputation: 210
Code:
awk '($5==0)' huge.txt
 
Old 03-29-2012, 01:20 PM   #3
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,682
Blog Entries: 14

Rep: Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022
awk '$5 ~ 0' filename
 
Old 03-29-2012, 01:22 PM   #4
Linux_Kidd
Member
 
Registered: Jan 2006
Location: USA
Posts: 540

Rep: Reputation: 60
awk '$5==0{print $0}' file
same as post #2, but demonstrates you can print out the line or any fields you like, etc.

replace , or ,, or ,,, with , then pipe it to awk
sed -i 's/,\+/,/g' file | awk -F "," '$5==0{print $0}'

Last edited by Linux_Kidd; 03-29-2012 at 01:45 PM.
 
Old 03-29-2012, 01:22 PM   #5
arn2025
LQ Newbie
 
Registered: Feb 2012
Posts: 26
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
This is part of the file, however the command awk '($5==0)' huge.txt did not return anything
Quote:
29,,0,,0,0,,0,,,22,,0,0,,25677212881000,-1,,8,,0,0,-1,256784456293,,281,0,60,0,4
CCN_MOD16,68687,1543,63974,0,658,0,283,,185,59,256777547728,05-FEB-12,79.285714,30,,,U:726120535:3526164791328448271::840750662,52430,05-FEB-12,,N,,506,320,,0,,0,0,,0,,,22,,0,0,,25677030212541000,-1,,8,,0,0,-1,256778647313,,192,17.07692,59,25.61538,4
CCN_MOD16,93222,2905,63225,0,658,0,287,,0,60,256800200028,05-FEB-12,0,0,,,U:1009230930:3526164791328448271::141632229,52430,05-FEB-12,,N,,656,656,,0,,0,0,,0,,,22,,0,0,,25677208352000,-1,,256,,0,0,-1,256783258338,,-1,0,60,0,4
CCN_MOD16,68688,1543,62873,0,658,0,283,,63,20,256782834338,05-FEB-12,27,30,,,U:1120701893:3526164791328448271::411062659,52430,05-FEB-12,,N,,576,513,,0,,0,0,,0,,,22,,0,0,,25677030207203000,-1,,8,,0,0,-1,256784647114,,192,5.81538,20,8.72308,4
CCN_MOD16,66249,2905,61915,0,658,0,283,,295,4,250788298313,05-FEB-12,0,0,,,U:79861
 
Old 03-29-2012, 01:23 PM   #6
jhwilliams
Senior Member
 
Registered: Apr 2007
Location: Portland, OR
Distribution: Debian, Android, LFS
Posts: 1,168

Rep: Reputation: 210Reputation: 210Reputation: 210
@arn2025 You need to specify the correct field separator. ',' is not in IFS. Consult the Awk man page.
 
1 members found this post helpful.
Old 03-29-2012, 01:35 PM   #7
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,682
Blog Entries: 14

Rep: Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022
Quote:
Originally Posted by jhwilliams View Post
@arn2025 You need to specify the correct field separator. ',' is not in IFS. Consult the Awk man page.
awk -F, will tell it to use comma as the field separator.
 
1 members found this post helpful.
Old 03-29-2012, 01:47 PM   #8
Linux_Kidd
Member
 
Registered: Jan 2006
Location: USA
Posts: 540

Rep: Reputation: 60
Quote:
Originally Posted by MensaWater View Post
awk -F, will tell it to use comma as the field separator.
exactly.
i assumed the extra comma's were an issue, hence why my sed.

so if the comma counts are ok then
awk -F "," '$5==0{print $0}'
or
awk -F, '$5==0{print $0}'
 
1 members found this post helpful.
Old 03-29-2012, 01:54 PM   #9
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,682
Blog Entries: 14

Rep: Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022
Quote:
Originally Posted by Linux_Kidd View Post
exactly.
i assumed the extra comma's were an issue, hence why my sed.
Your original post with sed was before the OP put his example in so how could you have known he had extra commas?

Anyway I don't think there are EXTRA commas based on what the OP put as example - just lines that don't have something in every field so the , delimiter is next to each other for the blank field.
 
Old 03-29-2012, 01:57 PM   #10
Linux_Kidd
Member
 
Registered: Jan 2006
Location: USA
Posts: 540

Rep: Reputation: 60
Quote:
Originally Posted by MensaWater View Post
Your original post with sed was before the OP put his example in so how could you have known he had extra commas?

Anyway I don't think there are EXTRA commas based on what the OP put as example - just lines that don't have something in every field so the , delimiter is next to each other for the blank field.
huh, his sample is on post #4, my answer in post #5.
 
Old 03-29-2012, 02:33 PM   #11
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,682
Blog Entries: 14

Rep: Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022
Odd - On my display its the opposite. He is 4 and you are 5. But both show 1:22 PM (ET) so presumably you both posted at near the same time.

I was thinking maybe you were psychic.
 
Old 04-03-2012, 09:46 AM   #12
Linux_Kidd
Member
 
Registered: Jan 2006
Location: USA
Posts: 540

Rep: Reputation: 60
Quote:
Originally Posted by MensaWater View Post
Odd - On my display its the opposite. He is 4 and you are 5. But both show 1:22 PM (ET) so presumably you both posted at near the same time.

I was thinking maybe you were psychic.
i am still lost. you say here exactly as i did, his on 4 mine on 5, so mine came after his.

but his post has a weird date/time in the post header, its on 2 lines while all others are on one line.....?

Last edited by Linux_Kidd; 04-03-2012 at 09:49 AM.
 
Old 04-03-2012, 01:13 PM   #13
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,682
Blog Entries: 14

Rep: Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022
I'm saying the post where you first mentioned sed on my screen shows up BEFORE the one where he gave sample output. So to me your later claim to have mentioned sed implied you knew what his output looked like before he provided it and I questioned your prescience.

On looking at it after you later reply I saw that both posts have a 1:22 PM ET time stamp so see that while it appears to me yours came before his it is quite possible it actually came after his and it is just the vagaries of the order the messages are displayed that makes it appear the opposite on my screen.

I won't go into 4 and 5 as for all I know you're seeing something completely different for those numbers than I am.

Not a big deal.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
parsing a text file - to awk or not to awk ? rollyah Programming 9 08-18-2011 03:20 PM
[SOLVED] call awk from bash script behaves differently to awk from CLI = missing newlines titanium_geek Programming 4 05-26-2011 10:06 PM
[SOLVED] awk: how can I assign value to a shell variable inside awk? quanba Programming 6 03-23-2010 03:18 AM
shell command using awk fields inside awk one71 Programming 6 06-26-2008 05:11 PM
Some comments on awk and awk scripts makyo Programming 4 03-02-2008 06:39 PM


All times are GMT -5. The time now is 09:19 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration