LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 03-14-2013, 09:24 AM   #1
Alvin88
LQ Newbie
 
Registered: Mar 2012
Posts: 13

Rep: Reputation: Disabled
Extracting and printing only part of the line with awk or sed


Hi all,

I got an issue with regular expression. Here is the deal:

Log file on my server looks like this - let's call it LOGFILE.log:
2013-03-14 02:14:18 GMT comment server INFO 200 - ModuleMediaCasterStreamMonitorAdvanced.onValidateMediaCaster[rtpinput1/_definst_] Stream not healthy [stream startup timeout]: flv:STREAM--NAME.stream - - - 55485.773 - - - - - - - -- - - - - - - - - - - - - - - - -
2013-03-14 02:14:18 GMT comment server INFO 200 - ModuleMediaCasterStreamMonitorAdvanced.onValidateMediaCaster[rtpinput1/_definst_] Stream not healthy [stream startup timeout]: flv:STREAM--NAME.stream - - - 55485.773 - - - - - - -- - - - - - - - - - - - - - - - - -
2013-03-14 02:14:18 GMT comment server INFO 200 - ModuleMediaCasterStreamMonitorAdvanced.onValidateMediaCaster[rtpinput1/_definst_] Stream not healthy [stream startup timeout]: flv:STREAM--NAME.stream - - - 55485.774 - - - - - - - -- - - - - - - - - - - - - - - - -

I would like to extract the part STREAM--NAME, how can I do this?

I try to write something like: flv:.*\.stream, but it doesn't help much.

My solution so far:
egrep "Stream not healthy " LOGFILE.log | cut -d ':' -f5 | awk -F"." '{print $1}'
as I need to do this only for the lines which contains string "Stream not healthy ", hence the egrep as the first filter.

The issue is that this pattern --> STREAM--NAME is not always in the same place in the log file - 99% the command above will work fine, but I would like to match this pattern, wherever it will be in the line in this LOGFILE.log.

Question no 1: How can I write a regular expression to print only the part STREAM--NAME from any place in the log line?

Question no 2: How can also extract, in the same one-liner - date and time? So, the output will look like:
2013-03-14 02:14:18 STREAM--NAME
For this one I have completly no idea how to shorten the line for LOGFILE.log. Date and time will always be the first characters in the line - like above, but the location of the pattern STREAM--NAME might be in a diffrent place - (but after the date and time for sure) hence the need for the command which will extract it and print it.

Question no 3: Could you recommend any good pages, books or magazines where I can read about it? I do not have much experience with regular expresion, especailly connected with awk or sed. So i can learn and help others form time to time?

Can I ask you chaps for help on this one, or some hints how to achieve my goal?
 
Old 03-14-2013, 11:58 AM   #2
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Ubuntu
Posts: 1,067

Rep: Reputation: 284Reputation: 284Reputation: 284
This code ...
Code:
 sed 's/GMT/~/;s/flv:/~/' $InFile  \
|awk -F~ '{print $1 $3}'           \
> $OutFile
... produced this output ...
Code:
2013-03-14 02:14:18 STREAM--NAME.stream - - - 55485.773 - - - - - - - -- - - - - - - - - - - - - - - - -
2013-03-14 02:14:18 STREAM--NAME.stream - - - 55485.773 - - - - - - -- - - - - - - - - - - - - - - - - -
2013-03-14 02:14:18 STREAM--NAME.stream - - - 55485.774 - - - - - - - -- - - - - - - - - - - - - - - - -
Daniel B. Martin
 
1 members found this post helpful.
Old 03-14-2013, 02:45 PM   #3
Alvin88
LQ Newbie
 
Registered: Mar 2012
Posts: 13

Original Poster
Rep: Reputation: Disabled
Many thanks, works like a charm.

I found something like:
grep -E -o 'v:.*\.s' | awk -F"[:.]" '{print $2}'

but i really didn't know how to do the trick with date, time, and STREAMNAME.

Your solution is much better.

After a moment, and a cup of tea I understood the trick here:
sed 's/GMT/~/;s/flv:/~/;s/\.stream/~/' $InFile | awk -F~ '{print $1 $3}'
gave me:
2013-03-14 18:23:06 STREAMNAME
and this one can be marked as solved.

Mr Daniel B. Martin, can I buy you a beer, if you live in London or somewhere near by?

Once again - many thank.

Last edited by Alvin88; 03-14-2013 at 02:59 PM.
 
1 members found this post helpful.
Old 03-14-2013, 03:35 PM   #4
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Ubuntu
Posts: 1,067

Rep: Reputation: 284Reputation: 284Reputation: 284
Quote:
Originally Posted by Alvin88 View Post
After a moment, and a cup of tea I understood the trick ...
Solutions are usually offered with the assumption the Original Poster will understand. However, there is no shame in making a follow-up post to ask for an explanation.
Quote:
Mr Daniel B. Martin, can I buy you a beer, if you live in London or somewhere near by?
Apex NC USA is some distance from London, so I thank you for the offer but respectfully decline.

Daniel B. Martin
 
Old 03-15-2013, 01:25 AM   #5
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,442

Rep: Reputation: 1880Reputation: 1880Reputation: 1880Reputation: 1880Reputation: 1880Reputation: 1880Reputation: 1880Reputation: 1880Reputation: 1880Reputation: 1880Reputation: 1880
Another alternative:
Code:
grep -Po '^.*(?= GMT)|(?<=flv:)[^.]*' file | awk 'ORS=NR%2?" ":"\n"'
 
1 members found this post helpful.
Old 03-15-2013, 02:38 AM   #6
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,247

Rep: Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025Reputation: 2025
Re qn 3: http://www.grymoire.com/Unix/
Enjoy
 
1 members found this post helpful.
Old 03-15-2013, 10:39 AM   #7
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
Please use ***[code][/code]*** tags around your code and data, to preserve the original formatting and to improve readability. Do not use quote tags, bolding, colors, "start/end" lines, or other creative techniques.

Here's my addition to the list of options:
Code:
sed -rn '/not healthy/ s/^([0-9-]+[ ][0-9:]+).*flv:(.+)[.]stream.*/\1 \2/p' logfile
This first matches lines containing "not healthy". The substitution then captures the date string to the first backreference, and the part between "flv:" and ".stream" to the second backreference. The rest of the line is discarded.

Naturally it assumes there will be only one instance of each pattern on the line.
 
1 members found this post helpful.
  


Reply

Tags
awk, extracting, sed


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Sed and printing only part of a line jrdioko Linux - Software 12 05-17-2012 07:15 PM
[SOLVED] Insert line using sed or awk at line using line number as variable sunilsagar Programming 11 02-03-2012 10:48 AM
filter source line based on results line in log using awk and sed samanp Programming 5 04-06-2011 09:42 AM
extract part of a line with sed or awk alirezan1 Linux - Newbie 2 10-01-2008 09:44 PM
extracting part of a string (sed, bash, ...) rama76 Programming 3 01-11-2008 08:14 AM


All times are GMT -5. The time now is 08:43 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration