LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-03-2013, 06:03 AM   #16
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444

@all

if you are confused as to what samasara wants/needs check their thread history.
 
Old 10-03-2013, 07:29 AM   #17
samasara
Member
 
Registered: Aug 2013
Posts: 34

Original Poster
Rep: Reputation: Disabled
Hi dear,
These two blocks are not the same. you can see little differences in each part of them( i mean part A-Z of each part). Actually the whole log file contains more block(not just two). this is an another example:

Quote:
--25814763-A--
[21/Jul/2013:10:46:31 --0400] Uev0R38AAAEAAAeHdbsAAABb 192.168.153.128 46943 192.168.153.128 80
--25814763-B--
GET /inssgtz7ieltdSstbw7e/neQhmsdwu7imdb0etet/eT/hsvegbff/EH/niRAvLwGK_L/osLnBWcHRk5oGMI/tmLJFqSww/sSjS6KRJB.html?Settotzeertnl=%27pn+&8nafitm=74LuKUC5t0J&4ttNe=Anmsyusi6&Mf1g-vYqyx=elTTsw&Euoytxprthane=e229evoY%401&tu=aht7hhgettiuy&hol=74723&7Emgo0il=aahri&ocLupha=ocark01n&d itduesn4on=vS%24&prfclmdeIaEEd=N0drn7genterci&metucwotoiwr=5N%2BWp4jo+%5Dp&emt=11ZefSA2P HTTP/1.0
Host: 28.99.169.38
Connection: close
Accept: */*
Accept-Charset: *
Accept-Encoding: *;q=0.7
Accept-Language: ondtmsih-rqjk1;q=0.6
Cache-Control: upy='aels'
Cookie2: $Version="388"
Date: Thu, 27 Apr 06 24:36:50 UTC
ETag: "kAPqiaNm18b1MKgZ"
If-Modified-Since: Thu, 18 May 06 01:58:48 GMT
If-Unmodified-Since: Fri, 19 Sep 08 10:20:47 CET
If-Match: *
If-None-Match: "mGN7VPxNspBNWF50vLeI"
Pragma: vthu=ifdlS4il
Authorization: Basic TnJ3MDY6UWVzc29sY2U=
Referer: http://www.hao8.de/0ehnqceo/segfto2z...f/tmdpe2En.avi
TE: deflate;q=0.7,deflate;q=0.1,deflate;q=0.2
User-Agent: rfno5rdhfNnssmascs
UA-CPU: MIPS
Via: FTP/1.8 200.207.173.114
Transfer-Encoding: wria
Warning: 540 www.plsshi.jpeg "rdomaoAwf2ahhud2t" "Fri, 15 Sep 06 13:24:25 CET"

--25814763-F--
HTTP/1.1 501 Method Not Implemented
Allow: TRACE
Connection: close
Content-Type: text/html; charset=iso-8859-1

--25814763-H--
Message: Access denied with code 406 (phase 2). Pattern match "(^[\"'`\xc2\xb4\xe2\x80\x99\xe2\x80\x98;]+|[\"'`\xc2\xb4\xe2\x80\x99\xe2\x80\x98;]+$)" at ARGS:Settotzeertnl. [file "/usr/local/apache/conf/samane_rules/SpiderLabs-owasp-modsecurity-crs-33612c6/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "64"] [id "981318"] [rev "2"] [msg "SQL Injection Attack: Common Injection Testing Detected"] [data "Matched Data: ' found within ARGS:Settotzeertnl: 'pn "] [severity "CRITICAL"] [ver "OWASP_CRS/2.2.7"] [maturity "9"] [accuracy "8"] [tag "OWASP_CRS/WEB_ATTACK/SQL_INJECTION"] [tag "WASCTC/WASC-19"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE1"] [tag "PCI/6.5.2"]
Apache-Error: [file "http_filters.c"] [line 262] [level 3] Unknown Transfer-Encoding: wria, referer: http://www.hao8.de/0ehnqceo/segfto2z...f/tmdpe2En.avi
Action: Intercepted (phase 2)
Stopwatch: 1374417991011250 5572 (- - -)
Stopwatch2: 1374417991011250 5572; combined=867, p1=10, p2=830, p3=0, p4=0, p5=26, sr=0, sw=1, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.7.2 (http://www.modsecurity.org/).
Server: Apache/2.2.23 (Unix) mod_ssl/2.2.23 OpenSSL/1.0.0-fips DAV/2 PHP/5.4.12
Engine-Mode: "ENABLED"

--25814763-Z-
--25814763-A--
[21/Jul/2013:10:46:31 --0400] Uev0R38AAAEAAAfNy6IAAABy 192.168.153.128 46947 192.168.153.128 80
--25814763-B--
GET /suGdrVj.cfm?seivste3c=804252514&Gxnc1atstirc3eR=eA&ioe9ogl=%3B%7E&J7etc88Op=7%24gr+stx&HWlTW=c2.2mwQ &anE5=TAistyle&usi=la+&md=dNt%40&zs=%5Dt%7ChmEh HTTP/1.0
Host: 104.88.31.54
Connection: msii
Accept: text/*, text/plain;q=0.7
Accept-Charset: *;q=0.3
Accept-Encoding: *;q=0.0
Accept-Language: *
Cache-Control: max-age=56191
Client-ip: 198.201.173.41
Cookie: oarraatavd=tisrP3n5mopsy;C.psdnB=87378
Cookie2: $Version="382"
Date: Mon, 15 Mar 04 14:00:38 CET
ETag: W/"8iZ39RWZewgvp0wTVy"
Expect: 100-continue
From: 4atoe@ehNulvLa.net
If-Modified-Since: Thu, 08 Jul 04 03:25:10 GMT
If-Unmodified-Since: Wed, 29 Nov 06 01:48:48 CET
If-Match: "zv_rZaStdUmXCxRi0SvS"
If-None-Match: *
If-Range: *
Max-Forwards: 415
Pragma: vthu=ifdlS4il
Authorization: Digest username="raqt"
Range: 91030-90141
Referer: /eIepao/wli3/aiguta.sh
TE: gzip;q=0.1,trailers,trailers
Trailer: Host
User-Agent: pxptea0aj/2.2.8
UA-CPU: MIPS
Via: HTTP/8.5 38.114.179.225
Transfer-Encoding: compress
Warning: 092 www.bnp6yotc.jpg "It5ru0hZb2OxuMoerlh" "Fri, 11 Dec 09 19:58:51 GMT"
X-Forwarded-For: 20.155.227.227
X-Serial-Number: 157557837076027

--25814763-F--
HTTP/1.1 501 Method Not Implemented
Allow: TRACE
Connection: close
Content-Type: text/html; charset=iso-8859-1
--25814763-H--
Message: Access denied with code 406 (phase 2). Pattern match "(^[\"'`\xc2\xb4\xe2\x80\x99\xe2\x80\x98;]+|[\"'`\xc2\xb4\xe2\x80\x99\xe2\x80\x98;]+$)" at ARGS:ioe9ogl. [file "/usr/local/apache/conf/samane_rules/SpiderLabs-owasp-modsecurity-crs-33612c6/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "64"] [id "981318"] [rev "2"] [msg "SQL Injection Attack: Common Injection Testing Detected"] [data "Matched Data: ; found within ARGS:ioe9ogl: ;~"] [severity "CRITICAL"] [ver "OWASP_CRS/2.2.7"] [maturity "9"] [accuracy "8"] [tag "OWASP_CRS/WEB_ATTACK/SQL_INJECTION"] [tag "WASCTC/WASC-19"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE1"] [tag "PCI/6.5.2"]
Apache-Error: [file "http_filters.c"] [line 262] [level 3] Unknown Transfer-Encoding: compress, referer: /eIepao/wli3/aiguta.sh
Action: Intercepted (phase 2)
Stopwatch: 1374417991258030 4293 (- - -)
Stopwatch2: 1374417991258030 4293; combined=1009, p1=9, p2=981, p3=0, p4=0, p5=18, sr=0, sw=1, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.7.2 (http://www.modsecurity.org/).
Server: Apache/2.2.23 (Unix) mod_ssl/2.2.23 OpenSSL/1.0.0-fips DAV/2 PHP/5.4.12
Engine-Mode: "ENABLED"

--25814763-Z--
--25814763-A--
[21/Jul/2013:10:46:31 --0400] Uev0R38AAAEAAAczRX8AAAAR 192.168.153.128 46951 192.168.153.128 80
--25814763-B--
GET /rNWuny/mvcHeMK-k-aem/reahtnonn/dayAsrgdlnerattoiPn/olert/gcbeole/i1megdotg/U7TEY2v/arOui.png?iTtoaQitnt=ftxe&eal=+%24optObaee%29rscriptcnid%3Bcc%29&pINfCoH8aMV1=6&84k=68&oPtrhlpc=i+&f nadtT9ao=sYjZp4Sf8&gem5neenha=3esgNemkuti HTTP/1.0
Host: www.ooki.be:4884
Connection: tS58ln
Accept: */*
Accept-Charset: koi8, macintosh;q=0.0, windows-1250;q=0.8, us-ascii;q=0.6, x-mac-hebrew
Accept-Encoding: *
Accept-Language: Za-5dOiioi;q=0.3, oe-h, wrgoLrA-Ydyutlel
Cache-Control: only-if-cached
Client-ip: 198.201.173.41
Cookie: oarraatavd=tisrP3n5mopsy;C.psdnB=87378
Cookie2: $Version="58"
Date: Fri, 23 Nov 07 22:59:51 GMT
ETag: W/"8iZ39RWZewgvp0wTVy"
Expect: 100-continue
From: 4atoe@ehNulvLa.net
If-Modified-Since: Mon, 14 Jan 08 23:43:42 UTC
If-Unmodified-Since: Tue, 19 Oct 04 06:30:05 CET
If-Match: "Ea4h_iuEGBpuhik0E"
If-None-Match: "LpnoLelb7twMFrynvu"
If-Range: Sat, 06 Nov 04 17:13:37 UTC
Max-Forwards: 235
MIME-Version: 6.7
Pragma: no-cache
Authorization: NTLM MHRtRGV0ZWF2bFRhZXJlRXRkZXNubmhzaXRtaG1kdGVxOHNlc2E=
Range: -8,-6
Referer: /ura9/nrhgapm4/9Gnp4eL/anHytt/obknr.asp
TE: deflate,gzip,gzip;q=0.7
Trailer: Host
User-Agent: Mozilla/1.1 (X11; U; Linux i386 2.4; Nb-ei; rv:1.0.8) Gecko/66238372
UA-CPU: MIPS
UA-OS: Mac OS X
UA-Color: color8
Via: HTTP/8.1 172.254.186.232:6108, FTP/2.6 222.98.171.160:50
Transfer-Encoding: compress
Warning: 092 www.bnp6yotc.jpg "It5ru0hZb2OxuMoerlh" "Fri, 11 Dec 09 19:58:51 GMT"
X-Forwarded-For: 20.155.227.227
X-Serial-Number: 98610483750174
~~~~~: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

--25814763-F--
HTTP/1.1 501 Method Not Implemented
Allow: TRACE
Connection: close
Content-Type: text/html; charset=iso-8859-1
dayAsrgdlnerattoiPn/olert/gcbeole/i1megdotg/U7TEY2v/arOui.png could not be found on this server.</p>

--25814763-H--
Message: Access denied with code 406 (phase 2). Pattern match "([\\~\\!\\@\\#\\$\\%\\^\\&\\*\\(\\)\\-\\+\\=\\{\\}\\[\\]\\|\\:\\;\"\\'\\\xc2\xb4\\\xe2\x80\x99\\\xe2\x80\x98\\`\\<\\>].*?){4,}" at ARGS:eal. [file "/usr/local/apache/conf/samane_rules/SpiderLabs-owasp-modsecurity-crs-33612c6/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "159"] [id "981173"] [rev "2"] [msg "Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded"] [data "Matched Data: ) found within ARGS:eal: $optObaee)rscriptcnid;cc)"] [ver "OWASP_CRS/2.2.7"] [maturity "9"] [accuracy "8"] [tag "OWASP_CRS/WEB_ATTACK/SQL_INJECTION"]
Apache-Error: [file "http_filters.c"] [line 262] [level 3] Unknown Transfer-Encoding: compress, referer: /ura9/nrhgapm4/9Gnp4eL/anHytt/obknr.asp
Action: Intercepted (phase 2)
Stopwatch: 1374417991413461 7231 (- - -)
Stopwatch2: 1374417991413461 7231; combined=3514, p1=12, p2=3481, p3=0, p4=0, p5=20, sr=0, sw=1, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.7.2 (http://www.modsecurity.org/).
Server: Apache/2.2.23 (Unix) mod_ssl/2.2.23 OpenSSL/1.0.0-fips DAV/2 PHP/5.4.12
Engine-Mode: "ENABLED"

--25814763-Z--

--c13bff4c-A--
[21/Jul/2013:10:46:33 --0400] Uev0SX8AAAEAAAdYN9wAAAA1 192.168.153.128 46961 192.168.153.128 80
--c13bff4c-B--
GET /9Zwindow.openk6eL@qspdJT/ePjaJ5IXKvcHfjU/bzAD3FTcSVB-_exec/c_2vxQ12Azn0YO/Bid/rBh1.Fh5Y/efE94ee06ohbnuA/emeaaphlluhy/p8oHnO/nAexecC_GFAO15zS/3SXKdl8qW.swf?rw2hitehtehaeo=eatar&9Lote0hto3as=sfpup%40&fhdnfNa=8&hse=rtlftnfwd4bhEnaasu&nhp8ap=%7C bod&bgsoundAnxKcEUN=54&aemitataehxv=lTl9rFXIDxa&l7xnoSelei=leo%3Coe%3A%26graon&scniverce=i3itccopyg4 fieeieZ&eRrrn=4&bYBkP=n%2Bbkfhm0cfesamittT+s&R@A_h=n%28uintcopyhC&ygo=65561774 HTTP/1.1
Host: www.ejCedtX0s.de:80
Connection: dufNei5a
Accept: audio/*, audio/x-wav;q=0.9, video/mpeg
Accept-Charset: x-mac-turkish;q=0.0, iso-8859-2
Accept-Encoding:
Accept-Language: mYo-s;q=0.5
Cache-Control: max-age=2925
Client-ip: 198.201.173.41
Cookie: i5uditwC=4r9XmC;eHsh5Lh=2;tNtD0vn0mols=73g7;temep=emE%7E;tEqajrmobtenoi=%7Ed;eaytsOepg=73828178
Cookie2: $Version="52"
Date: Wed, 27 Jun 07 05:58:50 GMT
ETag: W/"1hX6T0c-CbBwPLabz"
Expect: 100-continue
From: ogeatRw@t7e8ttf.it
If-Modified-Since: Sat, 26 Aug 06 06:04:38 UTC
If-Unmodified-Since: Wed, 11 Jun 08 02:48:36 GMT
If-Match: "q.4hO5T.h7vocxB"
If-None-Match: "NQ40hdzMS@ClRJgPuDH"
If-Range: Mon, 29 May 06 21:00:32 GMT
Max-Forwards: 2
MIME-Version: 6.7
Pragma: 0ra='al'
Proxy-Authorization: Digest response="9A4225299AdfbC29Fc0D3BFfa178C4E8"
Authorization: Digest cnonce="qeoheteq"
Range: -923117,3050-297658
Referer: http://www.setet.biz/utwi9/hnlou.php3
TE: gzip;q=0.1,trailers
Trailer: From
User-Agent: IEir/1.2.8.0.2
UA-CPU: PowerPC
UA-OS: FreeBSD
UA-Color: color8
UA-Pixels: 2292x8708
Via: FTP/6.0 www.ydee.css, 8.0 117.205.152.141
Transfer-Encoding: deflate
Upgrade: sroo/1.1
Warning: 927 www.uEPrbtt.js:5292 "ywqs8f"
X-Forwarded-For: 20.155.227.227
X-Serial-Number: 579346705

--c13bff4c-F--
HTTP/1.1 501 Method Not Implemented
Allow: TRACE
Connection: close
Content-Type: text/html; charset=iso-8859-1

--c13bff4c-H--
Message: Access denied with code 406 (phase 2). Pattern match "(?i[\\s'\"`\xc2\xb4\xe2\x80\x99\xe2\x80\x98\\(\\)]*?)([\\d\\w]++)([\\s'\"`\xc2\xb4\xe2\x80\x99\xe2\x80\x98\\(\\)]*?)(??:=|<=>|r?like|sounds\\s+like|regexp)([\\s'\"`\xc2\xb4\xe2\x80\x99\xe2\x80\x98\\(\\)]*?)\\2|(?:!=|<=|>=|<>|<|>|\\^|is\\s+not|not\\ ..." at ARGS:l7xnoSelei. [file "/usr/local/apache/conf/samane_rules/SpiderLabs-owasp-modsecurity-crs-33612c6/base_rules/modsecurity_crs_41_sql_injection_attacks.conf"] [line "77"] [id "950901"] [rev "2"] [msg "SQL Injection Attack: SQL Tautology Detected."] [data "Matched Data: leo<oe found within ARGS:l7xnoSelei: leo<oe:&graon"] [severity "CRITICAL"] [ver "OWASP_CRS/2.2.7"] [maturity "9"] [accuracy "8"] [tag "OWASP_CRS/WEB_ATTACK/SQL_INJECTION"] [tag "WASCTC/WASC-19"] [tag "OWASP_TOP_10/A1"] [tag "OWASP_AppSensor/CIE1"] [tag "PCI/6.5.2"]
Apache-Error: [file "http_filters.c"] [line 262] [level 3] Unknown Transfer-Encoding: deflate, referer: http://www.setet.biz/utwi9/hnlou.php3
Action: Intercepted (phase 2)
Stopwatch: 1374417993387304 4992 (- - -)
Stopwatch2: 1374417993387304 4992; combined=2425, p1=3, p2=2404, p3=0, p4=0, p5=17, sr=0, sw=1, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.7.2 (http://www.modsecurity.org/).
Server: Apache/2.2.23 (Unix) mod_ssl/2.2.23 OpenSSL/1.0.0-fips DAV/2 PHP/5.4.12
Engine-Mode: "ENABLED"

--c13bff4c-Z--
my log file contain lots of logs. in this example i show you my log file contains 4 different logs. I want to get from this log file italic log and Bold log. just two last logs(Bold and Italic ones).My logfile is in var/log/access_audit.log path. how can i just get two last log from this file(Bold and Italic in this example). I hope my meaning to be clear.
Really thanks for your kind and help
Best Regards
 
Old 10-03-2013, 09:54 AM   #18
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
After having a good look at the examples you provided I'm not sure there is enough uniqueness to create a script that will give you the last 2 blocks.

At first I thought that the --c13bff4c-A-- to --c13bff4c-Z-- parts might be unique identifiers, but the first three blocks in the above example all use the same identifier (--25814763-A-- to --25814763-Z--).

The string following the date/time part might be used to separate the individual blocks. I'll have to think about how that might be used, not sure if I can come up with a solution though.

I might have a possible alternate solution to the problem: You mention that all the output is stored in /var/log/access_audit.log, wouldn't it be simpler to create individual logs for each run? I'm not sure how this log is created and if this can be (easily) implemented but life would be easier if you had something like this to work with (simple example):
Code:
/var/log/access_audit/ -> directory for the access_audit logs

20130721.104631.Uev0R38AAAEAAAeHdbsAAABb.log
20130721.104631.Uev0R38AAAEAAAfNy6IAAABy.log
20130721.104631.Uev0R38AAAEAAAczRX8AAAAR.log
20130721.104633.Uev0SX8AAAEAAAdYN9wAAAA1.log
Each run has its own log file, which consists of: date.time.identifier.log (yyyymmdd.hhmmss.identifier.log).

I'll have another look at the provided example when time permits and try to come up with a solution.
 
Old 10-03-2013, 10:04 AM   #19
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444
why the last two?

surly you want to process each log as it arrives

and this is where I go back to saying that your c++ program is what should be monitoring the logs 'from start to finish'
 
Old 10-03-2013, 07:08 PM   #20
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,240

Rep: Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324
Its not entirely clear to me if this logs issue is a separate qn from the original, although it looks like it and in which case it should have been a new thread.
In any case, I'm going to treat it as such for clarity.

If you want to keep track of the latest log blocks, then (as mentioned above) the easiest thing to do is to constantly tail the log file and output/process each new block as it arrives.
I would use this module http://search.cpan.org/~mgrabnar/Fil...0.99.3/Tail.pm

HTH
 
Old 10-04-2013, 12:17 AM   #21
samasara
Member
 
Registered: Aug 2013
Posts: 34

Original Poster
Rep: Reputation: Disabled
Quote:
why the last two?

surly you want to process each log as it arrives
Hi firerat, the problem is not by log processing. i can now process my logs. but the main problem is get the last two logs as the log file is being updated.
regards
 
Old 10-04-2013, 05:18 AM   #22
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Although I'm still not 100% sure what it is you want, this seems to do what you want (show last 2 blocks):
Code:
#!/bin/bash

INFILE="infile"                                            # infile

NoIn="$( grep -c '\-\-[a-z0-9][a-z0-9]*\-A\-' $INFILE )"   # last instance
SlIn=$(( NoIn -1 ))                                        # second to last

echo " ------> Second to last instance <------"
awk -v n="$SlIn" '/--[a-z0-9][a-z0-9]*-Z-/{p=0}; p && c == n; /--[a-z0-9][a-z0-9]*-A-/ && !p {p=1; c++}' $INFILE

echo -e "\n ------> Last instance <------\n"
awk -v n="$NoIn" '/--[a-z0-9][a-z0-9]*-Z-/{p=0}; p && c == n; /--[a-z0-9][a-z0-9]*-A-/ && !p {p=1; c++}' $INFILE

echo -e "\n ------> Done processing <------\n"
An example run with a simplified input file (removed most of the "in-between" stuff):
Code:
$ cat infile
--25814763-A--
[21/Jul/2013:10:46:31 --0400] Uev0R38AAAEAAAeHdbsAAABb 192.168.153.128 46943 192.168.153.128 80
--25814763-B--
GET /inssgtz7ieltdSstbw7e/neQhmsdwu7imdb0etet/eT/hsvegb ...........

--25814763-F--
HTTP/1.1 501 Method Not Implemented

--25814763-H--
Action: Intercepted (phase 2)

--25814763-Z-
--25814763-A--
[21/Jul/2013:10:46:31 --0400] Uev0R38AAAEAAAfNy6IAAABy 192.168.153.128 46947 192.168.153.128 80
--25814763-B--
GET /suGdrVj.cfm?seivste3c=804252514&Gxnc1atstirc3eR=eA&ioe9ogl=%3 ......

--25814763-F--
HTTP/1.1 501 Method Not Implemented
--25814763-H--
Engine-Mode: "ENABLED"

--25814763-Z--
--25814763-A--
[21/Jul/2013:10:46:31 --0400] Uev0R38AAAEAAAczRX8AAAAR 192.168.153.128 46951 192.168.153.128 80
--25814763-B--
GET /rNWuny/mvcHeMK-k-aem/reahtnonn/dayAsrgdln .....

--25814763-F--
HTTP/1.1 501 Method Not Implemented
Allow: TRACE

--25814763-H--
Engine-Mode: "ENABLED"

--25814763-Z--
--c13bff4c-A--
[21/Jul/2013:10:46:33 --0400] Uev0SX8AAAEAAAdYN9wAAAA1 192.168.153.128 46961 192.168.153.128 80
--c13bff4c-B--
GET /9Zwindow.openk6eL@qspdJT/ePjaJ ......

--c13bff4c-F--
HTTP/1.1 501 Method Not Implemented
Allow: TRACE

--c13bff4c-H--
Engine-Mode: "ENABLED"

--c13bff4c-Z--
Code:
$ ./test.sh
 ------> Second to last instance <------
[21/Jul/2013:10:46:31 --0400] Uev0R38AAAEAAAczRX8AAAAR 192.168.153.128 46951 192.168.153.128 80
--25814763-B--
GET /rNWuny/mvcHeMK-k-aem/reahtnonn/dayAsrgdln .....

--25814763-F--
HTTP/1.1 501 Method Not Implemented
Allow: TRACE

--25814763-H--
Engine-Mode: "ENABLED"


 ------> Last instance <------

[21/Jul/2013:10:46:33 --0400] Uev0SX8AAAEAAAdYN9wAAAA1 192.168.153.128 46961 192.168.153.128 80
--c13bff4c-B--
GET /9Zwindow.openk6eL@qspdJT/ePjaJ ......

--c13bff4c-F--
HTTP/1.1 501 Method Not Implemented
Allow: TRACE

--c13bff4c-H--
Engine-Mode: "ENABLED"


 ------> Done processing <------
This isn't perfect, the --*-A- and --*-Z- entries itself aren't printed in the output, but that shouldn't be too much of an issue.

BTW: You really need to have a good look at the way the log file is generated. This, seemingly faulty entry appears in it: --25814763-Z- That should be --25814763-Z-- (mind the extra - at the end).
 
Old 10-04-2013, 10:25 AM   #23
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444
Quote:
Originally Posted by samasara View Post
Hi firerat, the problem is not by log processing. i can now process my logs. but the main problem is get the last two logs as the log file is being updated.
regards
Yes I know, I feel I understand what you are doing better than you do.,

currently you have a script ( or two ) read a log and output 3 numbers which you then give to a c++ program, which goes off and does fuzzy logic on them.

Now, the problem you have is that since you ran your initial scripts more logs have been added.
if you run your script(s) again you have duplicate data.
Solution
Code:
tail -f YourLog | YourScript | FuzzyCPP
Alternate
in C++ FuzzyCPP
  • Open Logfile
  • if line is "GET" Var1="result of function"
    if line is "MESSAGE" Var2="result of other function"
  • when we have Var1 and Var2 make Var3 ( with yet another Function )
  • now we have the 3 Vars, put into the FuzzyLogic Function
  • unset Var1,2 and 3
  • return to the start
in effect you never close the logfile you just keep reading it ( you will need a loop )
if your FuzzyCPP is asked to terminate , then send any unprocessed lines to newfile, close log, more or unlink logfile, move newfile to logfile

at the end of the day , building that functionality into your C++ program is much easier than trying to do it with your scripts
 
Old 10-04-2013, 10:35 AM   #24
samasara
Member
 
Registered: Aug 2013
Posts: 34

Original Poster
Rep: Reputation: Disabled
process the output of one shell script in another one

Quote:
Although I'm still not 100% sure what it is you want, this seems to do what you want (show last 2 blocks):
Hi druuna. I show two last blocks in bold and italic.
I tried to solve my problem with tail --line=50. it shows me the last 50 lines and i tried to work with that. Now i am trying to do something like you said. i hope to be successful. i would be in contact with you.
Really thanks for your kind and help

Last edited by samasara; 10-04-2013 at 10:39 AM.
 
Old 10-11-2013, 04:18 PM   #25
samasara
Member
 
Registered: Aug 2013
Posts: 34

Original Poster
Rep: Reputation: Disabled
Thanks

Hi dear users,
Really thanks for your kind and help. I solved my problem with your guidance and help. The only thing is how can i know if my file is updated or not?I mean i want to know if my log file is updated and then process the logs? Except periodic time for doing this job, what would be any other way to know if our log file is updated or not?

thanks a lot
Regards
 
Old 10-11-2013, 08:09 PM   #26
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444
keep a record of the size or md5sum, if it changes the log changed

OR do it the way I suggested with "Alternate" in post 23

This is why I questioned the "last two" in post 19, really you want anything not yet processed.

another option is to keep a log of the --25814763.. --c13bff4c..
I assume these are unique markers for each data set, log what you have done and ignore them if you see them again.
that will slow things down over time, better to process a log, archive it and wait for new logs

Going back to doing it in C, I'm sure you can process a 'record' dump it to an archive log and remove it from the original, new records should still be appended
maybe the original logger needs some tweaks
maybe the original logger should be putting things in a database instead of a file


Sooo many ways to do what you ask, just use a little imagination
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
shell script ssh: how to eliminate the login process output figo Programming 1 06-01-2009 03:05 PM
Shell script for reading a particular process contineously from process table vinaykori Linux - General 2 05-29-2009 07:52 AM
Shell Script : Kill a running process when another process starts ashmew2 Linux - General 3 08-20-2008 04:47 AM
shell script to read ps -e output and determine process double processes. dr_zayus69 Programming 1 09-21-2005 06:37 PM
Redirecting process output to shell mhelles Linux - Newbie 1 04-30-2003 10:30 AM


All times are GMT -5. The time now is 10:39 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration