LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - General (http://www.linuxquestions.org/questions/linux-general-1/)
-   -   grepping, zgrepping in python (http://www.linuxquestions.org/questions/linux-general-1/grepping-zgrepping-in-python-770938/)

suran 11-23-2009 04:32 AM

grepping, zgrepping in python
 
Hi guys,

im writing a program/script in python to read data from mail logs the gist is this,

i want the python to grep for an email address and then output the results from the logs to a file as a csv and then email it as an attachment:

the logs in question are .gz files and reside in /scratch/syslog-ng/maillogs

my python code so far is as follows:

#!/usr/bin/env python
import subprocess
import optparse
import re
import os
import gzip

LOGDIR="/scratch/syslog-ng/maillogs"
os.chdir(LOGDIR)
email = raw_input('Please enter email address: ')

the files under maillogs are in. gz

1) how do i read a gz file and do a grep based on the value of email from python?

a) can i use a zgrep from the python? will that work if so is there some special command to allow python to read unix shell commands?


this whole idea stemmed from the following shell script i wrote, but i wanted to be able to process the data in a better method and also email the results in an attachment:

#!/bin/ksh
#script:email_logs
#date:16/10/2009
#purpose: to find emails from a specific address and generate a report that is legible

#Define variables

logdir="/scratch/syslog-ng/maillogs"
email_address=$1
output="/tmp/$1.log"

function usage
{
echo "Usage: email_logs <email_address>"
echo "email_logs sabrix@malca.custhelp.com"
}

#####check for correct usage

if [ $# -ne 1 ]

then
usage
exit 1
fi

#####main program#######

cd ${logdir}

zgrep -E "$1" 10.2[0-9].0.1[0-100]/maillog.[0-99].gz | awk '{print " To: "$13,"From: "$12, "Subject: "$21,$22,$23,$24,$25, "Date & Time: " $1,$2,$3}' \
| sed 's%10.2[0-9].0.1[0-99]/maillog.[0-99].gz:%%g' >>${output} #checks on the 10.20. address range

zgrep -E "$1" 172.2[0-9].0.5[0-9]/maillog.[0-99].gz | awk '{print " To: "$13,"From: "$12, "Subject: "$21,$22,$23,$24,$25, "Date & Time: " $1,$2,$3}' \
| sed 's%10.2[0-9].0.1[0-99]/maillog.[0-99].gz:%%g' >>${output}

voyciz 11-23-2009 06:56 AM

Just gunzip the log first. There is an re.search() method.
Also you could read each line of the log and do
Code:

if 'johndoe@aol.com' in $line:
    MY CODE

To use shell commands there is os.system() and os.popen()


All times are GMT -5. The time now is 03:38 PM.