LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 02-01-2009, 05:11 PM   #1
pobman
Member
 
Registered: Jun 2005
Location: Wellington, New Zealand
Distribution: Fedora 9
Posts: 31

Rep: Reputation: 16
Question Shell Script or perl help. to write sections of a log to a tmp file for mailing


Hi,

I am using tripwire to report on my servers.

Some of my reports are over 2MB in size, I need to extract on the sections of the report that are relevant to me, and have them mailed.

Eg: Extract of current report.
Code:
Report generated by:          root
Report created on:            Mon Feb  2 07:30:00 2009
Database last updated on:     Never

===============================================================================
Report Summary:
===============================================================================

Host name:                    erato
Host IP address:              19.101.13.134
Host ID:                      None
Policy file used:             /usr/local/secure/tw/etc/tw.pol
Configuration file used:      /usr/local/secure/tw/etc/tw.cfg
Database file used:           /usr/local/secure/tw/db/tw.db_erato
Command line used:            /usr/local/secure/tw/bin/tripwire -m c -n -c 
/usr/local/secure/tw/etc/tw.cfg -P ***** -r 
/usr/local/security/logs/tw_erato_20090202:0730.twr

===============================================================================
Rule Summary:
===============================================================================

-------------------------------------------------------------------------------
  Section: Unix File System
-------------------------------------------------------------------------------

  Rule Name                       Severity Level    Added    Removed 
Modified
  ---------                       --------------    -----    -------  --------
  Invariant Directories           66                0        0        0
  Tripwire Data Files             100               0        0        0
  Temporary directories           33                0        0        0
* Critical devices                100               0        0        1
  Tripwire Binaries               100               0        0        0
* User binaries                   66                130      62       781
* Libraries                       66                120      47       7455
* OS executables and libraries    100               10       2        167
* File System and Disk Administraton Programs
                                  100               0        0        38
* Networking Programs             100               0        0        16
* System Administration Programs  100               0        0        16
* Operating System Utilities      100               0        0        33
* Critical Utility Sym-Links      100               0        0        25
* Shell Binaries                  100               0        0        6
* Security Control                100               4        1        25
  Login Scripts                   100               0        0        0
* System boot changes             100               3088     0        19
* Critical configuration files    100               87       16       137
* Kernel Administration Programs  100               0        0        10
* Hardware and Device Control Programs
                                  100               0        0        5
* System Information Programs     100               0        0        2
* Application Information Programs
                                  100               0        0        3
* Shell Releated Programs         100               0        0        1
  (/sbin/getkey)
* Critical system boot files      100               13       0        6
* Root config files               100               11       10902    11

Total objects scanned:  21039
Total violations found:  23250

===============================================================================
Object Detail:
===============================================================================

-------------------------------------------------------------------------------
  Section: Unix File System
-------------------------------------------------------------------------------

-------------------------------------------------------------------------------
Rule Name: Critical devices (/proc/mdstat)
Severity Level: 100
-------------------------------------------------------------------------------
  ----------------------------------------
  Modified Objects: 1
  ----------------------------------------

Modified object name:  /proc/mdstat

  Property:            Expected                    Observed
  -------------        -----------                 -----------
* Mode                 -r--r--r--                  -rw-r--r--



-------------------------------------------------------------------------------
Rule Name: User binaries (/usr/local/bin)
Severity Level: 66
-------------------------------------------------------------------------------
  ----------------------------------------
  Added Objects: 1
  ----------------------------------------

Added object name:  /usr/local/bin/sudo
I would like to see only the following in my email report:
Code:
===============================================================================
Rule Summary:
===============================================================================

-------------------------------------------------------------------------------
  Section: Unix File System
-------------------------------------------------------------------------------

  Rule Name                       Severity Level    Added    Removed 
Modified
  ---------                       --------------    -----    -------  --------
  Invariant Directories           66                0        0        0
  Tripwire Data Files             100               0        0        0
  Temporary directories           33                0        0        0
* Critical devices                100               0        0        1
  Tripwire Binaries               100               0        0        0
* User binaries                   66                130      62       781
* Libraries                       66                120      47       7455
* OS executables and libraries    100               10       2        167
* File System and Disk Administraton Programs
                                  100               0        0        38
* Networking Programs             100               0        0        16
* System Administration Programs  100               0        0        16
* Operating System Utilities      100               0        0        33
* Critical Utility Sym-Links      100               0        0        25
* Shell Binaries                  100               0        0        6
* Security Control                100               4        1        25
  Login Scripts                   100               0        0        0
* System boot changes             100               3088     0        19
* Critical configuration files    100               87       16       137
* Kernel Administration Programs  100               0        0        10
* Hardware and Device Control Programs
                                  100               0        0        5
* System Information Programs     100               0        0        2
* Application Information Programs
                                  100               0        0        3
* Shell Releated Programs         100               0        0        1
  (/sbin/getkey)
* Critical system boot files      100               13       0        6
* Root config files               100               11       10902    11

Total objects scanned:  21039
Total violations found:  23250
Using grep -n and wc I managed to get the line number of Rule summary then -1 of the wc to get the line number for the title, I then used tail -n of total number of lines - title
this gives me everything to the end of the log.

I would then have to do a similar process again to find the section I do not want and use head.

But there must be an easier way to do this or a more efficient to get the data as there are several sections I want and this method seems cumbersome.

Code:
#!/bin/bash

TRIPWIRE=`/opt/tripwire/sbin/twprint`
REPORT=/opt/tripwire/reports
HOSTNAME=`hostname`.twr
LOG=/tmp/tripwire.tmp

/opt/tripwire/sbin/twprint -m r -r ${REPORT}/${HOSTNAME} | grep "Total" >${LOG}

NUM_LINES=`/opt/tripwire/sbin/twprint -m r -r ${REPORT}/${HOSTNAME} | wc -l`

START_NUM=`/opt/tripwire/sbin/twprint -m r -r ${REPORT}/${HOSTNAME} | grep -n "Rule Summary:"|awk -F\: '{ print $1 }' `

TOTAL_LINES=`expr ${NUM_LINES} - ${START_NUM} + 2`
#echo $NUM_LINES $START_NUM $TOTAL_LINES
/opt/tripwire/sbin/twprint -m r -r ${REPORT}/${HOSTNAME} |tail -n ${TOTAL_LINES} >> ${LOG}
mail -s "${HOSTNAME} Report" user@domain < ${LOG}


Cheers
 
Old 02-02-2009, 03:51 AM   #2
blackhole54
Senior Member
 
Registered: Mar 2006
Posts: 1,896

Rep: Reputation: 61
I am not familiar with perl, but I would think sed might be able to help out.

If you can do without that first line of equal signs, I think the following would work for you

Code:
sed -n "/^Rule Summary:/,/^Total violations found:/p"
(Pipe, redirect and/or name the input file on the command line as necessary.)

If you really need that first line of equal signs I would suspect a little sed script could be devised. You might want to look in particular at sed's h command (which copies current line into the hold space).
 
Old 02-02-2009, 03:30 PM   #3
pobman
Member
 
Registered: Jun 2005
Location: Wellington, New Zealand
Distribution: Fedora 9
Posts: 31

Original Poster
Rep: Reputation: 16
Thanks blackhole64.

I can not believe I did not know that one.

that is going to save me heaps of time
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
shell script to monitor log file calipryss Linux - Newbie 14 08-05-2008 10:46 PM
Filtering a CSV file from web log with shell script? Micro420 Programming 8 08-22-2007 03:13 AM
Trying to write a perl script that will print shell variable ohcarol Programming 2 04-16-2007 08:02 AM
How to write a shell script to download a file via FTP? guarriman Linux - General 4 12-21-2004 09:31 AM


All times are GMT -5. The time now is 10:23 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration