LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (https://www.linuxquestions.org/questions/programming-9/)
-   -   Shell Script or perl help. to write sections of a log to a tmp file for mailing (https://www.linuxquestions.org/questions/programming-9/shell-script-or-perl-help-to-write-sections-of-a-log-to-a-tmp-file-for-mailing-701497/)

pobman 02-01-2009 06:11 PM

Shell Script or perl help. to write sections of a log to a tmp file for mailing
 
Hi,

I am using tripwire to report on my servers.

Some of my reports are over 2MB in size, I need to extract on the sections of the report that are relevant to me, and have them mailed.

Eg: Extract of current report.
Code:

Report generated by:          root
Report created on:            Mon Feb  2 07:30:00 2009
Database last updated on:    Never

===============================================================================
Report Summary:
===============================================================================

Host name:                    erato
Host IP address:              19.101.13.134
Host ID:                      None
Policy file used:            /usr/local/secure/tw/etc/tw.pol
Configuration file used:      /usr/local/secure/tw/etc/tw.cfg
Database file used:          /usr/local/secure/tw/db/tw.db_erato
Command line used:            /usr/local/secure/tw/bin/tripwire -m c -n -c
/usr/local/secure/tw/etc/tw.cfg -P ***** -r
/usr/local/security/logs/tw_erato_20090202:0730.twr

===============================================================================
Rule Summary:
===============================================================================

-------------------------------------------------------------------------------
  Section: Unix File System
-------------------------------------------------------------------------------

  Rule Name                      Severity Level    Added    Removed
Modified
  ---------                      --------------    -----    -------  --------
  Invariant Directories          66                0        0        0
  Tripwire Data Files            100              0        0        0
  Temporary directories          33                0        0        0
* Critical devices                100              0        0        1
  Tripwire Binaries              100              0        0        0
* User binaries                  66                130      62      781
* Libraries                      66                120      47      7455
* OS executables and libraries    100              10      2        167
* File System and Disk Administraton Programs
                                  100              0        0        38
* Networking Programs            100              0        0        16
* System Administration Programs  100              0        0        16
* Operating System Utilities      100              0        0        33
* Critical Utility Sym-Links      100              0        0        25
* Shell Binaries                  100              0        0        6
* Security Control                100              4        1        25
  Login Scripts                  100              0        0        0
* System boot changes            100              3088    0        19
* Critical configuration files    100              87      16      137
* Kernel Administration Programs  100              0        0        10
* Hardware and Device Control Programs
                                  100              0        0        5
* System Information Programs    100              0        0        2
* Application Information Programs
                                  100              0        0        3
* Shell Releated Programs        100              0        0        1
  (/sbin/getkey)
* Critical system boot files      100              13      0        6
* Root config files              100              11      10902    11

Total objects scanned:  21039
Total violations found:  23250

===============================================================================
Object Detail:
===============================================================================

-------------------------------------------------------------------------------
  Section: Unix File System
-------------------------------------------------------------------------------

-------------------------------------------------------------------------------
Rule Name: Critical devices (/proc/mdstat)
Severity Level: 100
-------------------------------------------------------------------------------
  ----------------------------------------
  Modified Objects: 1
  ----------------------------------------

Modified object name:  /proc/mdstat

  Property:            Expected                    Observed
  -------------        -----------                -----------
* Mode                -r--r--r--                  -rw-r--r--



-------------------------------------------------------------------------------
Rule Name: User binaries (/usr/local/bin)
Severity Level: 66
-------------------------------------------------------------------------------
  ----------------------------------------
  Added Objects: 1
  ----------------------------------------

Added object name:  /usr/local/bin/sudo

I would like to see only the following in my email report:
Code:

===============================================================================
Rule Summary:
===============================================================================

-------------------------------------------------------------------------------
  Section: Unix File System
-------------------------------------------------------------------------------

  Rule Name                      Severity Level    Added    Removed
Modified
  ---------                      --------------    -----    -------  --------
  Invariant Directories          66                0        0        0
  Tripwire Data Files            100              0        0        0
  Temporary directories          33                0        0        0
* Critical devices                100              0        0        1
  Tripwire Binaries              100              0        0        0
* User binaries                  66                130      62      781
* Libraries                      66                120      47      7455
* OS executables and libraries    100              10      2        167
* File System and Disk Administraton Programs
                                  100              0        0        38
* Networking Programs            100              0        0        16
* System Administration Programs  100              0        0        16
* Operating System Utilities      100              0        0        33
* Critical Utility Sym-Links      100              0        0        25
* Shell Binaries                  100              0        0        6
* Security Control                100              4        1        25
  Login Scripts                  100              0        0        0
* System boot changes            100              3088    0        19
* Critical configuration files    100              87      16      137
* Kernel Administration Programs  100              0        0        10
* Hardware and Device Control Programs
                                  100              0        0        5
* System Information Programs    100              0        0        2
* Application Information Programs
                                  100              0        0        3
* Shell Releated Programs        100              0        0        1
  (/sbin/getkey)
* Critical system boot files      100              13      0        6
* Root config files              100              11      10902    11

Total objects scanned:  21039
Total violations found:  23250

Using grep -n and wc I managed to get the line number of Rule summary then -1 of the wc to get the line number for the title, I then used tail -n of total number of lines - title
this gives me everything to the end of the log.

I would then have to do a similar process again to find the section I do not want and use head.

But there must be an easier way to do this or a more efficient to get the data as there are several sections I want and this method seems cumbersome.

Code:

#!/bin/bash

TRIPWIRE=`/opt/tripwire/sbin/twprint`
REPORT=/opt/tripwire/reports
HOSTNAME=`hostname`.twr
LOG=/tmp/tripwire.tmp

/opt/tripwire/sbin/twprint -m r -r ${REPORT}/${HOSTNAME} | grep "Total" >${LOG}

NUM_LINES=`/opt/tripwire/sbin/twprint -m r -r ${REPORT}/${HOSTNAME} | wc -l`

START_NUM=`/opt/tripwire/sbin/twprint -m r -r ${REPORT}/${HOSTNAME} | grep -n "Rule Summary:"|awk -F\: '{ print $1 }' `

TOTAL_LINES=`expr ${NUM_LINES} - ${START_NUM} + 2`
#echo $NUM_LINES $START_NUM $TOTAL_LINES
/opt/tripwire/sbin/twprint -m r -r ${REPORT}/${HOSTNAME} |tail -n ${TOTAL_LINES} >> ${LOG}
mail -s "${HOSTNAME} Report" user@domain < ${LOG}



Cheers

blackhole54 02-02-2009 04:51 AM

I am not familiar with perl, but I would think sed might be able to help out.

If you can do without that first line of equal signs, I think the following would work for you

Code:

sed -n "/^Rule Summary:/,/^Total violations found:/p"
(Pipe, redirect and/or name the input file on the command line as necessary.)

If you really need that first line of equal signs I would suspect a little sed script could be devised. You might want to look in particular at sed's h command (which copies current line into the hold space).

pobman 02-02-2009 04:30 PM

Thanks blackhole64.

I can not believe I did not know that one.

that is going to save me heaps of time :D


All times are GMT -5. The time now is 01:18 PM.