Scripting Question

lowcalspam · 06-09-2010, 09:14 AM

Hey everyone,

So I've been writing a script that does a Lynx grab from a list of port numbers and IP's from SANS Internet Storm website. I then sed it down to the format that I want and dump each of those into text files. (See code below)

for i in `lynx -source "http://isc.sans.edu/feeds/topips.txt"|sed -e "s/\t/,/g"|cut -d "," -f1`; do echo "$i"; done > TOP_IPS_`date +20%y-%m-%d`.txt

for i in `lynx -source "http://isc.sans.edu/trendascii.html"|sed '/SANS/d'|sed '/Commons ShareAlike/d'|sed '/#/d'|sed '/Creative/d'|sed '/(c)/d'| sed -e "s/\t/,/g"|cut -d "," -f1`; do echo "$i"; done > TOP_PORTS`date +20%y-%m-%d`.txt

So, then what I'm wanting to do is have a loop go through and pull each of the IP's from the file(s) above and do a grep on certain log files for each IP, and likewise do the same thing from the port listing file that I have created. I have been thinking that creating an array in bash would be the way to go, but I'm not having any luck. Anyone have any ideas how to accomplish this?

Thanks a ton in advance.

grail · 06-09-2010, 09:41 AM

Ok ... so as your new the first thing you need to use when asking your questions are the code tags [code] (see advanced button to help with these). Just helps for us to read what code you have written.

Secondly, I find it helps if you give us the following:

1. What the input files look like that we need to get data from
2. What have you tried so far?

You mentioned you haven't had any luck so maybe one of the people here will see where you went wrong

lowcalspam · 06-09-2010, 10:07 AM

Sorry about that, I will correct my actions

thanks for the pointers.

The input file is merely a text file with port numbers listed for instance:

304
1010
1020

There are no quotes, commas, tabs, etc. Each line merely has a new port number in the ports file. The source file is named: 1TOP_PORTS__`date +20%y-%m-%d`.txt

So, here is what I have tried:

Code:

for i in `lynx -source "http://isc.sans.edu/trendascii.html"|sed '/SANS/d'|sed '/Commons ShareAlike/d'|sed '/#/d'|sed '/Creative/d'|sed '/(c)/d'| sed -e "s/\t/,/g"|cut -d "," -f1`; do echo "$i"; done > 1TOP_PORTS__`date +20%y-%m-%d`.txt


IFS='
'
file=( $( < 1TOP_*.txt) )
echo $file

cat global-*-ipt.log | grep "DPT\=$file"

Right now, it's grep'ing fine for the first line for the port number form the source file, but I'm drawing a blank on how to get it to iterate to the next port number from the source file.

crts · 06-09-2010, 12:38 PM

Quote:

Originally Posted by lowcalspam

Code:

for i in `lynx -source "http://isc.sans.edu/trendascii.html"|sed '/SANS/d'|sed '/Commons ShareAlike/d'|sed '/#/d'|sed '/Creative/d'|sed '/(c)/d'| sed -e "s/\t/,/g"|cut -d "," -f1`; do echo "$i"; done > 1TOP_PORTS__`date +20%y-%m-%d`.txt


IFS='
'
file=( $( < 1TOP_*.txt) )
echo $file

cat global-*-ipt.log | grep "DPT\=$file"

Right now, it's grep'ing fine for the first line for the port number form the source file, but I'm drawing a blank on how to get it to iterate to the next port number from the source file.

Hi,

if I understand correctly then the for loop works fine and you just need to step through every line in the generated file, right? Well, the standard approach for such a task would be

Code:

while read line
do
echo $line
# do other stuff like
grep "DPT\=$line" global-*-ipt.log # don't need the cat to grep
done < /path/to/file

Is this what you need?

onebuck · 06-09-2010, 03:41 PM

Hi,

Welcome to LQ!

As you have been instructed that we can aid you when you help yourself to a solution. You should provide us with what you have attempted along with relative data, code and then maybe someone will be able to assist. Plus an additional reminder to use the vbcode tags # or quote.

'How to Ask Questions the Smart Way' would be one link you should look at to help us to help you in the future. Below are several good reference links that may aid you;

Linux Documentation Project
Rute Tutorial & Exposition
Linux Command Guide
Utimate Linux Newbie Guide
LinuxSelfHelp
Getting Started with Linux
Bash Reference Manual <<<<<<

Look here

Advanced Bash-Scripting Guide <<<<<<

Look here

Linux Home Networking
Virtualiation- Top 10

The above links and others can be found at 'Slackware-Links'. More than just Slackware® links!