[SOLVED] text manipulating ... the more efficient way to grab a string

grail · 11-01-2013, 02:00 AM

Twist on Daniel's solution:

Code:

awk -F"[ ,:]+" '/^Oct 31 14/ {print $8,$9}' file

socalheel · 11-01-2013, 07:51 AM

Quote:

Originally Posted by grail

Twist on Daniel's solution:

Code:

awk -F"[ ,:]+" '/^Oct 31 14/ {print $8,$9}' file

could you explain what the + is used for in that line?

danielbmartin · 11-01-2013, 08:20 AM

Quote:

Originally Posted by socalheel

could you explain what the + is used for in that line?

-F"[ ,:]" says that any one of the specified characters is taken as one field separator.

-F"[ ,:]+" says that any cluster of the specified characters is taken as one field separator.

Illustrative examples:

str="abc@#def@ghi#$jkl"

This ... awk -F"[@#$]" '{print $2}' <<< $str
... produces a null because there is nothing in str between the @ and the #.

This ... awk -F"[@#$]+" '{print $2}' <<< $str
produces def because @# is taken as the first field separator.

Daniel B. Martin

socalheel · 11-01-2013, 09:48 AM

so i'm not sure how to resolve what i want to do now.

so my script gives me exactly what i want it to.

this:

Code:

#!/bin/bash
STRING=`grep "Oct 31 11" /var/log/maillog | grep -i spam | egrep -v "avast.com|to=<spam"`

if
test -n "$STRING"
then
MAILID=`grep "Oct 31 11" /var/log/maillog | grep -i spam | egrep -v "avast.com|to=<spam" | awk -F ": " '{print $2}' | sort -u`
for i in $MAILID
 do
  echo "Postfix ID Number $i"
  grep $i /var/log/maillog | grep "sasl_username" | awk -F ", " '{print $3}'; echo -e '\n'
  echo "From:  "; grep $i /var/log/maillog | grep "from=<" | sed -e 's/from=<>/<NO DATA FOR THIS FIELD>/g' |  awk -F "<" '{print $2}' | awk -F ">" '{print $1}'; echo -e '\n'
  echo "To:  ";   grep $i /var/log/maillog | grep "to=<" | sed -e 's/to=<>/<NO DATA FOR THIS FIELD>/g' | awk -F "<" '{print $2}' | awk -F ">" '{print $1}'; echo -e '\n'
 done
fi

will produce this:

Code:

Postfix ID Number EFA161FF9F8
sasl_username=first.last@work.com


From:
admin@aroundeugene.com


To:
email1@hotmail.com
email2@hotmail.com
email3@hotmail.com

my goal now is to run this every hour, and if spam is present, send all that output to my email. if it's not present, don't do anything.

can someone direct me down the right path?

socalheel · 11-01-2013, 11:59 AM

nevermind, figured it out.

this is designed to put in cron at the 59th minute of each hour to scan the maillog for any instance of the word spam for the current hour. so at 1159, it would grep maillog for every entry for the current day and every entry for the 11th hour (11:00 to 11:59)

Code:

#!/bin/bash

DATE=`date -d today +'%b %e %H'`
STRING=`grep "$DATE" /var/log/maillog | grep -i spam | egrep -v "avast.com|to=<spam"`

if
test -n "$STRING"
then
  MAILID=`grep "$DATE" /var/log/maillog | grep -i spam | egrep -v "avast.com|to=<spam" | awk -F ": " '{print $2}' | sort -u`
  for i in $MAILID
  {
        echo "Postfix ID Number $i"
        grep $i /var/log/maillog | grep "sasl_username" | awk -F ", " '{print $3}'
        echo "From:  "; grep $i /var/log/maillog | grep "from=<" | sed -e 's/from=<>/<NO DATA FOR THIS FIELD>/g' |  awk -F "<" '{print $2}' | awk -F ">" '{print $1}'
        echo "To:  ";   grep $i /var/log/maillog | grep "to=<" | sed -e 's/to=<>/<NO DATA FOR THIS FIELD>/g' | awk -F "<" '{print $2}' | awk -F ">" '{print $1}'; echo -e '\n'
 } > /spam_entries.txt
mail -s "Spam entries have been found in $(hostname)'s maillog" < /spam_entries.txt first.last@work.com
  fi

grail · 11-02-2013, 02:01 AM

Glad you found a solution

You could do the whole thing in awk which would remove the need for bash, grep, date and echo and use a single solution. Of course it may well be suited to Perl, Ruby or Python too

socalheel · 11-02-2013, 02:31 PM

i figured as much

i've only been doing red hat admin for 2 years now and i just started "scripting" about 3 months ago, i'm so far from where i want to be in terms of knowing what i'm doing.

in the meantime, i'll just post my questions here when i need guidance and hopefully you seasoned veterans can give me the right direction.

i do appreciate all the input you guys give me ... it is well appreciated.