LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 02-13-2008, 03:03 AM   #1
nabmufti
LQ Newbie
 
Registered: Feb 2008
Posts: 13

Rep: Reputation: 0
Question sort date in paragraphs within file


hi all
i want help in sortng date in paragraphs within file ,
i want to ask as if there any option to sort a certain pattern of file not the rest of file.i.e the data of file become sorted with respect to date
i have a log file as follows


!! *A0628/081 /08-01-10/13 H 52/N=5524/TYP=INC/CAT=ID/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 13/AGEO=BAGRIAN -B01
/TEXAL=FIRE DETECT FAIL
!! *A0628/320 /08-01-15/14 H 33/N=7444/TYP=COM/CAT=ID/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 2/AGEO=CONTAIN2-B01
/TEXAL=FAULTY UNIT/COMPL.INF:
* #F0612/T11F14/NCEN=MULCT /08-01-30/11 H 41/NAM=ODCAB /TDA=0001
/N=7444/NIV=2/ENS=006/SENS=016-000-000/P='0018'H/CN=05
/EM: AFUR =URAD- 2


!! *A0628/538 /07-12-17/15 H 52/N=7576/TYP=ICT/CAT=ID/EVENT=MAL
/NCEN=MULCT /AM =SMTA1/AGEO=S1-TR01-B03-A085-R000
/TEXAL=LCL MFM SYN/COMPL.INF:
/AF=URMA1
/ ICTRQ AGCA=S1-TR01-B03-A085-R133
/AMET=01-26-03
/AFLR=217-06/CRC=NACT
!!! *A0628/294 /07-12-17/15 H 46/N=7512/TYP=SRE/CAT=IM/EVENT=MAL
/NCEN=MULCT /OBJCT=PS/AGEO=CDC
/TEXAL=SP INACCESSIBLE/COMPL.INF:
/PS =00009999/TYR=RN


!! *A0628/361 /07-12-17/15 H 46/N=7513/TYP=COM/CAT=ID/EVENT=MAL
/NCEN=MULCT /AM =SMMA/AGEO=S1-TR02-B05-A109-R000
/TEXAL=SAI-HARDWARE/COMPL.INF:
/N=3485/ALARMA MATERIAL /R=00000/CU1A
/CB 200 BLOS
! *A0628/359 /07-12-17/15 H 46/N=7514/TYP=COM/CAT=SI/EVENT=MAL
/NCEN=MULCT /AM =SMMA/AGEO=S1-TR02-B05-A109-R000
/TEXAL=SAI-SOFTWARE/COMPL.INF:
/N=4543/ABNT X25 INDISPO. /R=00083/CU1A
/SNPA=A13381/PHYSLINE=LIGP13F1


!!! *A0628/080 /08-01-13/07 H 30/N=6540/TYP=INC/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 10/AGEO=RANGPUR -B01
/TEXAL=FIRE
!! *A0628/081 /07-12-29/16 H 22/N=5052/TYP=INC/CAT=ID/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 12/AGEO=KHOTYWAL-B01
/TEXAL=FIRE DETECT FAIL
!! *A0628/081 /08-01-10/13 H 52/N=5526/TYP=INC/CAT=ID/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 9/AGEO=BASTGLZR-B01
/TEXAL=FIRE DETECT FAIL


!!! *A0628/080 /08-01-10/13 H 52/N=5527/TYP=INC/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 7/AGEO=ADABOSAN-B01
/TEXAL=FIRE
!!! *A0628/087 /08-01-16/01 H 28/N=7648/TYP=CLI/CAT=IM/EVENT=MAL
/NCEN=MULCT /AFUR =URAD- 10/AGEO=RANGPUR -B01

i want to extract the date , time , NCEN , EVENT , TAXAL & AGEO
i used the code following
Code:
#!/bin/bash
sed  '/^!/i\
' log | sed -n -e '/^!!! /,/^$/w critical.log' -e '/^!! /,/^$/w major.log' -e'/^! /,/^$/w minor.log'
awk 'BEGIN {FS="/"; RS=""} { printf "%s/%s/%s/\n%s\n%s\n%s\n%s/%s\n\n", $1, $3, $4, $10, $12, $8, $14, $15}' minor.log 
exit 0
this code does seperate the paragraphs follwing ! , !! , !!! (i.e. minor , major & critical alarms of the log file respectively ,in their respective log files i.e. minor.log ,major.log & critical.log )
and 'awk' gives me output as following


! *A0628/07-12-17/15 H 58/
NCEN=MULCT
AGEO=S1-TR01-B03-A085-R000
EVENT=MAL
TEXAL=AIS/COMPL.INF:/ /AF=URMA1

but i want the output to be sorted by date
(date pattern=/07-12-17/)
i tried to use the sort -k but i coldnt understand the right pattern to use it
will u plz help me to solve my problem

Last edited by nabmufti; 02-13-2008 at 01:33 PM.
 
Old 02-13-2008, 10:54 PM   #2
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 677Reputation: 677Reputation: 677Reputation: 677Reputation: 677Reputation: 677
One plan of action could be to use awk to do the sorting. Another thing you could try is to replace the double returns to \0 and pipe the output to sort with the -z option and use the "/" character as the field separator.
Code:
barely tested example.  May not be the best.
awk 'BEGIN { FS="\n";RS=""} { printf "%s\0\0",$0 }' critical | sort -gt/ -z | tr '\0' '\n' | sed '/^!/i\
I think stepping back a bit and deciding how you need to use the output might be a good idea. What do you want to do with this data and how do you want to process it. It seems that some lines have one field while others have 3 or 4. Why not process these logs into tab separated files and input that into a spread sheet or database? That would make various lookups and analysis easier.
 
Old 02-14-2008, 04:01 AM   #3
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 244Reputation: 244Reputation: 244
if Python can be alternative:
Code:
#!/usr/bin/env python
import re
pat=re.compile("\n(?=!)",re.M|re.DOTALL)
data=open("file").read()
storage={}
for i in  pat.split(data):
    line= ' '.join(i.split())
    line=line.split()
    for n,items in enumerate(line):
        if "NCEN" in items:
            ncen= items[items.index("NCEN"):]
        elif "AGEO" in items:
            ageo = items[items.index("AGEO"):]
        elif "EVENT" in items:
            event = items[items.index("EVENT"):]
    date = line[2].split("/")[1].replace("-","")
    storage[date] = "%s %s %s %s" % ( line[2],ncen,ageo,event)    
for keys in sorted(storage.keys()):
    print "%s" % (storage[keys])
output:
Code:
# ./test1.py
/07-12-17/15 NCEN=MULCT AGEO=S1-TR02-B05-A109-R000 EVENT=MAL
/07-12-29/16 NCEN=MULCT AGEO=KHOTYWAL-B01 EVENT=MAL
/08-01-10/13 NCEN=MULCT AGEO=ADABOSAN-B01 EVENT=MAL
/08-01-13/07 NCEN=MULCT AGEO=RANGPUR EVENT=MAL
/08-01-15/14 NCEN=MULCT AGEO=CONTAIN2-B01 EVENT=MAL
/08-01-16/01 NCEN=MULCT AGEO=RANGPUR EVENT=MAL
 
Old 02-14-2008, 10:13 AM   #4
angrybanana
Member
 
Registered: Oct 2003
Distribution: Archlinux
Posts: 147

Rep: Reputation: 21
Quote:
Originally Posted by jschiwal View Post
I think stepping back a bit and deciding how you need to use the output might be a good idea. What do you want to do with this data and how do you want to process it.
I'm quoting jschiwal because I think this is the best advice.

nabmufti,
I noticed you're doing a lot of manipulating of the same data. Having a whole bunch of different awk/sed scripts for different small tasks is probably a bad way of approaching this problem. Depending on what you need to do with the data, a different file format or a perl/python approach might be better.

ghostdog74,
Code:
    line= ' '.join(i.split())
    line=line.split()
is the same as
Code:
line = i.split()
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
how to extract paragraphs from file in BASH script followed by prefix ! , !! and !!! nabmufti Programming 4 02-10-2008 09:23 AM
Trying to write a BASH script which allows input of paragraphs ChrisScott Linux - General 6 11-26-2006 05:32 PM
bash sort files by date in file name thedude2010 Programming 6 05-12-2006 11:07 AM
sort by date via Terminal itz2000 Linux - Newbie 3 01-22-2006 11:42 AM
How to sort files in a directory by date in KDE? Ace_Azzameen Linux - Software 7 05-24-2004 02:49 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 11:29 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration