LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 02-08-2008, 05:01 PM   #1
nabmufti
LQ Newbie
 
Registered: Feb 2008
Posts: 13

Rep: Reputation: 0
Unhappy how to extract paragraphs from file in BASH script followed by prefix ! , !! and !!!


hi all
i am in confusion since last 2 days
i posted thraed yesterday and some friends did help but still i couldnt get solution to my probleml

let it be very clear


i have a long log file of alkatel switch and i have to seperate the minor major and critical alarms shown by ! , !! and !!! respectively

the file format is like this


>> ALAIL;

CMD ALAIL 06095 08-01-30 11-58-23
INP ALAIL 06095 08-01-30 11-58-23
OPTEX=



RES ALAIL 06095 08-01-30 11-58-24
CEN=1/08-01-30/11 H 58 MN 19/STORED ALARMS LIST
PROCESSING TPFFIR ACC


! *A0628/540 /07-12-17/15 H 58/N=7598/TYP=ICT/CAT=SI/EVENT=MAL
/NCEN=MULCT /AM =SMTA1/AGEO=S1-TR01-B03-A085-R000
/TEXAL=AIS/COMPL.INF:
/AF=URMA1
/ ICTRQ AGCA=S1-TR01-B03-A112-R065
/AMET=01-07-02
/AFLR=222-09/CRC=NACT
!!! *A0628/303 /07-12-17/15 H 46/N=7501/TYP=COM/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 4/AGEO=BUDLASAN-B01
/TEXAL=SINGLE JUNCT/UT OOS/COMPL.INF:
/CN=3


!!! *A0628/306 /08-01-23/07 H 20/N=9619/TYP=COM/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFCN =CNLM3-4/AGEO=MKRASHID-B02-A003
/TEXAL=CN ISOLATED
!!! *A0628/303 /07-12-17/15 H 46/N=7503/TYP=COM/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 12/AGEO=KHOTYWAL-B01
/TEXAL=SINGLE JUNCT/UT OOS/COMPL.INF:
/CN=0
!!! *A0628/089 /08-01-29/22 H 14/N=1760/TYP=ENP/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 11/AGEO=QADIRPUR-B01
/TEXAL=GENERATING UNIT


!! *A0628/320 /08-01-17/13 H 47/N=8062/TYP=COM/CAT=ID/EVENT=MAL
/NCEN=MULCT /AFUR =URAL- 15/AGEO=S1-TR01-B04
/TEXAL=FAULTY UNIT/COMPL.INF:
* #F0612/T11F14/NCEN=MULCT /08-01-30/11 H 33/NAM=ODCAB /TDA=0001
/N=7064/NIV=2/ENS=006/SENS=013-000-000/P='0228'H/CN=02/PAR='03'H
/EM: AFUR =URAL- 15
!!! *A0628/303 /07-12-17/15 H 46/N=7506/TYP=COM/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 7/AGEO=ADABOSAN-B01
/TEXAL=SINGLE JUNCT/UT OOS/COMPL.INF:
/CN=2


!!! *A0628/303 /07-12-17/15 H 46/N=7507/TYP=COM/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 11/AGEO=QADIRPUR-B01
/TEXAL=SINGLE JUNCT/UT OOS/COMPL.INF:
/CN=2
!! *A0628/320 /08-01-26/06 H 39/N=0566/TYP=COM/CAT=ID/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 14/AGEO=KHOKARAN-B01
/TEXAL=FAULTY UNIT/COMPL.INF:

how wil i extract these paragraphs of alarms in respective alarm file??
i need to extract this log on the basis of prefix ! , !! , !!!

i am using the code to take input from file into script as


Code:
Code:
#!/bin/bash
alarm= `cat ~/alkatel.txt`
echo $alarm | while read line
do 
{
echo $line
}done
i want to have this code in bash only not in perl and any othe language please do me this favor
 
Old 02-08-2008, 08:29 PM   #2
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
Your script simply prints out the file which is what cat does.

You might try sed or awk.

The formatting of the log makes it difficult. If a line where to separate log entries, it would be easier. Here you can insert a line before a log entry and then filter out the critical parts. Instead of a proper sed program that does it all in one, I'll get lazy and cheat. The first sed command simply inserts the lines, while the second filters the entries:

Code:
 sed  '/^!/i\
' log | sed -n '/^!!! /,/^$/p'
The second sed command can write to three respective logs, minor.log, major.log & critical.log.
I found that I couldn't use the form "sed 'sedcommand1;sedcommand2;sedcommand3'" so I needed to use the "-e" argument to separate the three sed command lines.

Code:
 sed  '/^!/i\
' log | sed -n -e '/^!!! /,/^$/w critical.log' -e '/^!! /,/^$/w major.log' -e'/^! /,/^$/w minor.log'
Code:
jschiwal@lmax:~> cat minor.log 
! *A0628/540 /07-12-17/15 H 58/N=7598/TYP=ICT/CAT=SI/EVENT=MAL
/NCEN=MULCT /AM =SMTA1/AGEO=S1-TR01-B03-A085-R000
/TEXAL=AIS/COMPL.INF:
/AF=URMA1
/ ICTRQ AGCA=S1-TR01-B03-A112-R065
/AMET=01-07-02
/AFLR=222-09/CRC=NACT

jschiwal@lmax:~> cat major.log 
!! *A0628/320 /08-01-17/13 H 47/N=8062/TYP=COM/CAT=ID/EVENT=MAL
/NCEN=MULCT /AFUR =URAL- 15/AGEO=S1-TR01-B04
/TEXAL=FAULTY UNIT/COMPL.INF:
* #F0612/T11F14/NCEN=MULCT /08-01-30/11 H 33/NAM=ODCAB /TDA=0001
/N=7064/NIV=2/ENS=006/SENS=013-000-000/P='0228'H/CN=02/PAR='03'H
/EM: AFUR =URAL- 15

!! *A0628/320 /08-01-26/06 H 39/N=0566/TYP=COM/CAT=ID/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 14/AGEO=KHOKARAN-B01
/TEXAL=FAULTY UNIT/COMPL.INF:

jschiwal@lmax:~> cat critical.log 
!!! *A0628/303 /07-12-17/15 H 46/N=7501/TYP=COM/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 4/AGEO=BUDLASAN-B01
/TEXAL=SINGLE JUNCT/UT OOS/COMPL.INF:
/CN=3

!!! *A0628/306 /08-01-23/07 H 20/N=9619/TYP=COM/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFCN =CNLM3-4/AGEO=MKRASHID-B02-A003
/TEXAL=CN ISOLATED

!!! *A0628/303 /07-12-17/15 H 46/N=7503/TYP=COM/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 12/AGEO=KHOTYWAL-B01
/TEXAL=SINGLE JUNCT/UT OOS/COMPL.INF:
/CN=0

!!! *A0628/089 /08-01-29/22 H 14/N=1760/TYP=ENP/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 11/AGEO=QADIRPUR-B01
/TEXAL=GENERATING UNIT

!!! *A0628/303 /07-12-17/15 H 46/N=7506/TYP=COM/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 7/AGEO=ADABOSAN-B01
/TEXAL=SINGLE JUNCT/UT OOS/COMPL.INF:
/CN=2

!!! *A0628/303 /07-12-17/15 H 46/N=7507/TYP=COM/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 11/AGEO=QADIRPUR-B01
/TEXAL=SINGLE JUNCT/UT OOS/COMPL.INF:
/CN=2
I haven't thouroughly tested this. You might need to add a blank line to the end of the log or test it with a sample where the last entry doesn't end with a blank line. That could cause the ",/^$/" end of range test to fail.

Last edited by jschiwal; 02-08-2008 at 08:34 PM.
 
Old 02-09-2008, 04:21 AM   #3
nabmufti
LQ Newbie
 
Registered: Feb 2008
Posts: 13

Original Poster
Rep: Reputation: 0
Question thanx JSCHIWAL

thanx a lot
the code follwing gave me a way to extract the paragraphs follwing prefix ! , !! , !!! from log file


Code:
 sed  '/^!/i\
' log | sed -n -e '/^!!! /,/^$/w critical.log' -e '/^!! /,/^$/w major.log' -e'/^! /,/^$/w minor.log'
i need your help again

please guide me that how can i trim the required character string in paragraph from log file?????????????

to make you more clear
see

! *A0628/540 /07-12-17/15 H 58/N=7598/TYP=ICT/CAT=SI/EVENT=MAL
/NCEN=MULCT /AM =SMTA1/AGEO=S1-TR01-B03-A085-R000
/TEXAL=AIS/COMPL.INF:
/AF=URMA1
/ ICTRQ AGCA=S1-TR01-B03-A112-R065
/AMET=01-07-02
/AFLR=222-09/CRC=NACT

from the above paragraph i want to extract

date & time i.e. /07-12-17/15 H 58/
NCEN=MULCT
AGEO=S1-TR01-B03-A085-R000
EVENT=MAL
TEXAL=AIS/COMPL.INF:

please guide me that how to deal with these substring extraction from a paragraph in a file
 
Old 02-09-2008, 07:05 AM   #4
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
This can give you an idea on extracting the date on the first line:
Code:
 sed '/^!/s/.*\/\([[:digit:]][[:digit:]]-[[:digit:]][[:digit:]]-[[:digit:]][[:digit:]]\)\/.*/\1/' minor.log
The general pattern is like this:
The first part "/^!/" selects the first line with the date. On the other lines, you may need more patterns to work with.
Are the positions fixed? Is NCEN= a constant or one of numerous possibilities?

Code:
sed '/NCEN=/s/.*\(NCEN=[^ ]*\) .*/\1/'
So far we have:
Code:
 sed -e '/^!/s/.*\/\([[:digit:]][[:digit:]]-[[:digit:]][[:digit:]]-[[:digit:]][[:digit:]]\)\/.*/\1/' -e '/NCEN=/s/.*\(NCEN=[^ ]*\) .*/\1/' minor.log 
07-12-17
NCEN=MULCT
/TEXAL=AIS/COMPL.INF:
/AF=URMA1
/ ICTRQ AGCA=S1-TR01-B03-A112-R065
/AMET=01-07-02
/AFLR=222-09/CRC=NACT
I've been writing it a part at a time and testing it as I go on. It might make sense to create a file with a sed program and call sed this way:
sed -f extract.sed minor.log

And I just noticed looking on the next lines in your sample output that some of the data you want later is contained on the first line I thought I was done with. That means that it is necessary to either use the more advanced commands or sed or use another tool like awk.

Awk might be a better choice. You can extract the information you want from these lines and put them into variables. Then when it reaches a blank line you can print out the variables in the order that you want.

A test awk program like this might help you identify the fields you want to keep:
Code:
awk 'BEGIN { FS="/" }
/^! / { print "1:"$1 " 2:"$2 "3:"$3 " 4:"$4 " 5:"$5 " 6:" $6}' minor.log
1:! *A0628 2:540 3:07-12-17 4:15 H 58 5:N=7598 6:TYP=ICT
The FS is the builtin "Field Separator" variable that awk uses to separate a line into fields. If the values you want are always located in the same fields on each line, you might consider changing the RS (Record Separator) to an empty line. Then you could simply have a line like:
Code:
 awk 'BEGIN {FS="/"; RS=""} { printf "%s/%s/\n%s\n%s\n%s\n%s/%s\n\n", $3, $4, $10, $12, $8, $14, $15}' minor.log 
07-12-17/15 H 58/
NCEN=MULCT 
AGEO=S1-TR01-B03-A085-R000
EVENT=MAL
TEXAL=AIS/COMPL.INF:
Whether this will work depends on how regular the sample pattern you gave is.

Last edited by jschiwal; 02-09-2008 at 07:09 AM.
 
Old 02-10-2008, 09:23 AM   #5
nabmufti
LQ Newbie
 
Registered: Feb 2008
Posts: 13

Original Poster
Rep: Reputation: 0
Smile thanx JSCHIWAL

i got the exact solution i was looking for
u r genius
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
bash script extract name transfer & to right folder gizmo1007 Programming 2 12-15-2007 05:12 AM
file or directory? bash script efus Programming 3 04-26-2007 06:11 PM
Trying to write a BASH script which allows input of paragraphs ChrisScott Linux - General 6 11-26-2006 05:32 PM
cannot execute a script unless prefix ./ 1kyle Suse/Novell 2 09-11-2006 06:00 AM
Can't get lines of a file with a Bash script.. barisdemiray Programming 2 08-11-2004 12:42 PM


All times are GMT -5. The time now is 03:39 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration