LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-22-2012, 01:54 PM   #1
alirz
LQ Newbie
 
Registered: Jun 2011
Posts: 19

Rep: Reputation: Disabled
Filtering out some lines from a text file


Hi,
i attempted this in excel but couldnt figure it out. Im guessing it might be a bit simpler in linux?

In a file i have some lines like:
MMS25201-49001111000701 14:57:32,737 MO to MT fwd to MSISDN
MMS25201-49001111000701 14:57:32,776 [PASS] Testcase status: All steps validated successfully..
MMS25201-49001111000702 14:57:33,121 MO to MT with delivery report and fwd to MSISDN
MMS25201-49001111000702 14:57:33,158 [PASS] Testcase status: All steps validated successfully..
MMS25201-49001111000703 14:57:33,504 MO to MT with read report and fwd to MSISDN
MMS25201-49001111000703 14:57:33,542 [FAIL] Testcase status: Step: 005 - Field: "X-Mms-Read-Report" - Expected: Yes, received: No. .
MMS25201-49001111000704 14:57:33,877 MO to MT with delivery & read reports and fwd to MSISDN
MMS25201-49001111000704 14:57:33,912 [FAIL] Testcase status: Step: 005 - Field: "X-Mms-Read-Report" - Expected: Yes, received: No. .
MMS25201-49001111000705 14:57:34,100 MO to MT fwd to email
MMS25201-49001111000705 14:57:34,137 [PASS] Testcase status: All steps validated successfully..
MMS25201-49001111000706 14:57:34,342 MO to MT fwd to email with delivery report
MMS25201-49001111000706 14:57:34,388 [FAIL] Testcase status: Step: 006 - Field: "X-Mms-Status" - Expected: (-127=RETRIEVED), received: (-122=FORWARDED). .
MMS25201-49001111000707 14:57:34,591 MO to MT fwd to email with read report
MMS25201-49001111000707 14:57:34,625 [PASS] Testcase status: All steps validated successfully..
MMS25201-49001111000708 14:57:35,085 MO to MT fwd to email with read & delivery reports
MMS25201-49001111000708 14:57:35,122 [PASS] Testcase status: All steps validated successfully..

This is a test result file.each test has two result lines. Im trying to remove all the lines which have [FAIL] in column 3 and also its corresponding line above for which column 3 is blank.
I hope my question is clear. Does anyone have any idea?
thanks
 
Old 02-22-2012, 02:03 PM   #2
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,256

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
I didn't understand the last part:
Quote:
and also its corresponding line above for which column 3 is blank.
On looking at the data presented, the line above a FAIL seems to look no different than to a PASS except for what is written, eg:
Quote:
#Above PASS
MMS25201-49001111000702 14:57:33,121 MO to MT with delivery report and fwd to MSISDN
# Above FAIL
MMS25201-49001111000703 14:57:33,504 MO to MT with read report and fwd to MSISDN
 
Old 02-22-2012, 02:09 PM   #3
alirz
LQ Newbie
 
Registered: Jun 2011
Posts: 19

Original Poster
Rep: Reputation: Disabled
well the idea is to remove the test cases that failed. So as each test case seems to have two lines.
First the test case itself and then the second line with the status in the 3rd column. I noticed the pattern and thought maybe the way to get what i want is to remove the line with the word fail (grep -i fail)and then somehow also exclude the line above that line.

cat TC_LIST.txt |grep -a -i -B 1 pass |more seems to be give me pretty much the result i want but for some lines its skipping some lines and putting a "--"

MMS25201-49001111000726 14:57:35,668 MO to MT cpy to MSISDN
MMS25201-49001111000726 14:57:35,690 [PASS] Testcase status: All steps validated successfully..
MMS25201-49001111000727 14:57:35,894 MO to MT with delivery report and cpy to MSISDN
MMS25201-49001111000727 14:57:35,931 [PASS] Testcase status: All steps validated successfully..
--
MMS25201-49001111000730 14:57:37,078 MO to MT cpy to email
MMS25201-49001111000730 14:57:37,101 [PASS] Testcase status: All steps validated successfully..
MMS25201-49001111000731 14:57:37,375 MO to MT cpy to email with delivery report
 
Old 02-22-2012, 02:11 PM   #4
alirz
LQ Newbie
 
Registered: Jun 2011
Posts: 19

Original Poster
Rep: Reputation: Disabled
i think i might have figured it out..Will comfirm and the close the thread. thanks
 
Old 02-22-2012, 02:50 PM   #5
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,186

Rep: Reputation: 346Reputation: 346Reputation: 346Reputation: 346
This lists all lines with PASS in them and the line before the PASS. (test.dat is your posted sample data.)
Code:
$ grep -B 1 'PASS' test.dat
MMS25201-49001111000701 14:57:32,737 MO to MT fwd to MSISDN
MMS25201-49001111000701 14:57:32,776 [PASS] Testcase status: All steps validated successfully..
MMS25201-49001111000702 14:57:33,121 MO to MT with delivery report and fwd to MSISDN
MMS25201-49001111000702 14:57:33,158 [PASS] Testcase status: All steps validated successfully..
--
MMS25201-49001111000705 14:57:34,100 MO to MT fwd to email
MMS25201-49001111000705 14:57:34,137 [PASS] Testcase status: All steps validated successfully..
--
MMS25201-49001111000707 14:57:34,591 MO to MT fwd to email with read report
MMS25201-49001111000707 14:57:34,625 [PASS] Testcase status: All steps validated successfully..
MMS25201-49001111000708 14:57:35,085 MO to MT fwd to email with read & delivery reports
MMS25201-49001111000708 14:57:35,122 [PASS] Testcase status: All steps validated successfully..
Or, if you don't want the "--" lines,
Code:
$ gawk '/fwd to/ {prior_line=$0}/\[FAIL]/{next}/\[/{print prior_line;print $0}' test.dat 
MMS25201-49001111000701 14:57:32,737 MO to MT fwd to MSISDN
MMS25201-49001111000701 14:57:32,776 [PASS] Testcase status: All steps validated successfully..
MMS25201-49001111000702 14:57:33,121 MO to MT with delivery report and fwd to MSISDN
MMS25201-49001111000702 14:57:33,158 [PASS] Testcase status: All steps validated successfully..
MMS25201-49001111000705 14:57:34,100 MO to MT fwd to email
MMS25201-49001111000705 14:57:34,137 [PASS] Testcase status: All steps validated successfully..
MMS25201-49001111000707 14:57:34,591 MO to MT fwd to email with read report
MMS25201-49001111000707 14:57:34,625 [PASS] Testcase status: All steps validated successfully..
MMS25201-49001111000708 14:57:35,085 MO to MT fwd to email with read & delivery reports
MMS25201-49001111000708 14:57:35,122 [PASS] Testcase status: All steps validated successfully..

Last edited by PTrenholme; 02-22-2012 at 02:51 PM.
 
Old 02-23-2012, 12:42 AM   #6
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,256

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
Awk can be a little simpler:
Code:
awk '/fwd to/{keep=$0}/PASS/{print keep"\n"$0}' file
 
Old 02-23-2012, 11:41 AM   #7
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
gnu grep now has an option for not printing the "--" separators, and another for defining a different string.

Code:
grep -B 1 --no-group-separator 'PASS' test.dat

grep -B 1 --group-separator=#####' 'PASS' test.dat
They're not mentioned in the man page yet, but they are in the info page.
 
Old 02-23-2012, 01:07 PM   #8
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,186

Rep: Reputation: 346Reputation: 346Reputation: 346Reputation: 346
Quote:
Originally Posted by David the H. View Post
gnu grep now has an option for not printing the "--" separators, and another for defining a different string.
...
Nice. I seldom look at the info file, and whoever added that option neglected to update the --help output and man page.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
copy last 10000+ lines of large text file to a temporary file emilyg Linux - Newbie 3 06-24-2009 03:43 PM
Remove lines in a text file based on another text file asiandude Programming 10 01-29-2009 11:59 AM
Adding lines of text to beginning of a text file BillKat Programming 2 01-19-2009 11:40 AM
Filtering an ugly text file bingo Programming 17 11-11-2006 04:01 PM
Grab text lines in text file LULUSNATCH Programming 1 12-02-2005 11:55 AM


All times are GMT -5. The time now is 08:18 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration