LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 11-23-2009, 04:32 AM   #1
suran
LQ Newbie
 
Registered: Jan 2009
Posts: 12

Rep: Reputation: 0
grepping, zgrepping in python


Hi guys,

im writing a program/script in python to read data from mail logs the gist is this,

i want the python to grep for an email address and then output the results from the logs to a file as a csv and then email it as an attachment:

the logs in question are .gz files and reside in /scratch/syslog-ng/maillogs

my python code so far is as follows:

#!/usr/bin/env python
import subprocess
import optparse
import re
import os
import gzip

LOGDIR="/scratch/syslog-ng/maillogs"
os.chdir(LOGDIR)
email = raw_input('Please enter email address: ')

the files under maillogs are in. gz

1) how do i read a gz file and do a grep based on the value of email from python?

a) can i use a zgrep from the python? will that work if so is there some special command to allow python to read unix shell commands?


this whole idea stemmed from the following shell script i wrote, but i wanted to be able to process the data in a better method and also email the results in an attachment:

#!/bin/ksh
#script:email_logs
#date:16/10/2009
#purpose: to find emails from a specific address and generate a report that is legible

#Define variables

logdir="/scratch/syslog-ng/maillogs"
email_address=$1
output="/tmp/$1.log"

function usage
{
echo "Usage: email_logs <email_address>"
echo "email_logs sabrix@malca.custhelp.com"
}

#####check for correct usage

if [ $# -ne 1 ]

then
usage
exit 1
fi

#####main program#######

cd ${logdir}

zgrep -E "$1" 10.2[0-9].0.1[0-100]/maillog.[0-99].gz | awk '{print " To: "$13,"From: "$12, "Subject: "$21,$22,$23,$24,$25, "Date & Time: " $1,$2,$3}' \
| sed 's%10.2[0-9].0.1[0-99]/maillog.[0-99].gz:%%g' >>${output} #checks on the 10.20. address range

zgrep -E "$1" 172.2[0-9].0.5[0-9]/maillog.[0-99].gz | awk '{print " To: "$13,"From: "$12, "Subject: "$21,$22,$23,$24,$25, "Date & Time: " $1,$2,$3}' \
| sed 's%10.2[0-9].0.1[0-99]/maillog.[0-99].gz:%%g' >>${output}
 
Old 11-23-2009, 06:56 AM   #2
voyciz
Member
 
Registered: Mar 2004
Distribution: Slackware
Posts: 425

Rep: Reputation: 40
Just gunzip the log first. There is an re.search() method.
Also you could read each line of the log and do
Code:
if 'johndoe@aol.com' in $line:
    MY CODE
To use shell commands there is os.system() and os.popen()

Last edited by voyciz; 11-23-2009 at 06:57 AM.
 
  


Reply

Tags
linux, python


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Python Python Python (aka Python 3) LXer Syndicated Linux News 0 08-05-2009 08:30 PM
grepping lines... visitnag Linux - Newbie 4 06-26-2008 01:22 PM
grepping "'s and .'s and the such secretlydead Linux - Software 6 11-30-2007 03:49 AM
grepping all manfiles pgrodt Linux - Software 8 12-01-2006 10:40 AM
question about grepping something Grafbak Programming 6 03-01-2005 01:28 PM


All times are GMT -5. The time now is 07:19 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration