I monitor my networks and jot down things I need based on reports such as I've posted below which I receive via email.

However, I very badly need to convert these outages into meaningful times/dates/patterns/stats which I can show the provider so that they can get things fixed.

I have scoured the net but simply don't quite understand enough about scripting to do something this complex.

Below is a snippet of what the emails looks like. What I need to do is to read the start times when failures occur and when the connection was reestablished.

So for example, I need to read only the lines which have a Track IP start and stop date/time. So based on the snippet below, the first one would read something like;

Dn 2013-01-15 18:06:53
Up 2013-01-15 18:06:55

Then, I need to be able to turn the above into a statistic showing the amount of down time and when so that I can force the provider to fix their services.

I sure would appreciate some help on this and again, I'm not a programmer which is why I'm asking for help.

Thanks.


[00001] 2013-01-15 18:10:12 [Root]system-alert-00008: IP spoofing! From 192.168.0.10 to 224.0.0.1, proto 2 (zone Untrust, int ethernet0/0). Occurred 1 times.
[00002] 2013-01-15 18:10:06 [Root]system-alert-00008: IP spoofing! From 192.168.0.1:1024 to 239.255.255.250:1900, proto UDP (zone Untrust, int ethernet0/0). Occurred 8 times.
[00003] 2013-01-15 18:09:06 [Root]system-alert-00008: IP spoofing! From 192.168.0.1:1024 to 239.255.255.250:1900, proto UDP (zone Untrust, int ethernet0/0). Occurred 8 times.
[00004] 2013-01-15 18:08:06 [Root]system-alert-00008: IP spoofing! From 192.168.0.1:1024 to 239.255.255.250:1900, proto UDP (zone Untrust, int ethernet0/0). Occurred 8 times.
[00005] 2013-01-15 18:08:02 [Root]system-alert-00008: IP spoofing! From 192.168.0.10 to 224.0.0.1, proto 2 (zone Untrust, int ethernet0/0). Occurred 1 times.
[00006] 2013-01-15 18:07:06 [Root]system-alert-00008: IP spoofing! From 192.168.0.1:1024 to 239.255.255.250:1900, proto UDP (zone Untrust, int ethernet0/0). Occurred 8 times.
[00007] 2013-01-15 18:06:55 [Root]system-critical-00062: Track IP IP address 24.116.2.50 succeeded.
[00008] 2013-01-15 18:06:53 [Root]system-critical-00062: Track IP failure reached threshold.
[00009] 2013-01-15 18:06:52 [Root]system-critical-00062: Track IP IP address 24.116.2.50 failed.
[00010] 2013-01-15 18:06:43 [Root]system-critical-00062: Track IP IP address 24.116.2.50 succeeded.
[00011] 2013-01-15 18:06:42 [Root]system-critical-00062: Track IP failure reached threshold.
[00012] 2013-01-15 18:06:41 [Root]system-critical-00062: Track IP IP address 24.116.2.50 failed.
[00013] 2013-01-15 18:06:40 [Root]system-critical-00062: Track IP IP address 4.2.2.2 succeeded.
[00014] 2013-01-15 18:06:40 [Root]system-critical-00062: Track IP failure reached threshold.
[00015] 2013-01-15 18:06:39 [Root]system-critical-00062: Track IP IP address 4.2.2.2 failed.
[00016] 2013-01-15 18:06:33 [Root]system-critical-00062: Track IP IP address 4.2.2.2 succeeded.
[00017] 2013-01-15 18:06:31 [Root]system-critical-00062: Track IP IP address 24.116.2.50 succeeded.

1. I note that the rec nums are increasing, but the timestamps are decreasing ... ???

2. if you wanted to take a purely textual approach, I'd use Perl for this. Data/Text munging (fold/spindle/mutilate/re-format etc) is something Perl is very good at.
http://perldoc.perl.org/
http://www.tizag.com/perlT/index.php
http://www.perlmonks.org/?node=Tutorials

3. if you can't guarantee that you won't get rec pairs split over 2 emails (eg down in one email, up in the next), it might be better to save all the data to a DB (MYSQL) and use SQL to pull out pairs and stats.
http://dev.mysql.com/doc/refman/5.1/en/

4. A graphical approach might be to install Cacti and then send screen shots of the graphs or even give them a read-only access and just email a link.
http://www.cacti.net/downloads/docs/html/

5. Nagios is an alerting system that often goes with Cacti and you should be able to pull out that sort of data from the logs or the DB (if you choose to use that with Nagios)
http://www.nagios.org/documentation/

The reports are coming from my juniper firewalls which do allow for remote syslog. I had thought of feeding syslog to a proper logging server but that didn't help me with past problems where I need to show a curve.

In the example I posted above, I was showing how the output might look after it's been taken from the text log.

The logs look like;
[00007] 2013-01-15 18:06:55 [Root]system-critical-00062: Track IP IP address 24.116.2.50 succeeded.
[00008] 2013-01-15 18:06:53 [Root]system-critical-00062: Track IP failure reached threshold.

This means the connection went down at 18:06:53 and recovered at 18:06:55.

I think I just need to find a decent Linux based syslog of which I've tried several over the years so know they aren't hard to find. I just need to find something however that will give me the stats of the past logs which I've kept.

Thanks.

I'd still like to know why the timestamps are in reverse?

As for syslog tools; they are for logging.
If you want to produce text stats or graphical representation, you're going to have to process the log recs (emails).

Speaking of Graphs, Cacti is based on snmpd, which even firewalls do (usually) have, so you'd get real-time graphs.

>I'd still like to know why the timestamps are in reverse?

I explained it above.

>As for syslog tools; they are for logging.

They can also be used for statistics, alerts, and much more than simply logging.

>If you want to produce text stats or graphical representation, you're going to have to
>process the log recs (emails).

What I asked is how can I do that with the logs I already have. Moving forward, I need to get some proper syslogs working again. That fell behind because the network is new so not everything is built just yet.

>Speaking of Graphs, Cacti is based on snmpd, which even firewalls do (usually) have, so you'd get real-time graphs

Yes, most do but this question is about the logs I already gave .

Thanks.