I have been seeing regular spikes in my bandwidth graphs over the last 3 days at the same time. This is happening for lots of machine in my network.

I am monitoring with cacti so can only see layer 2 data which I collect via snmp from the cisco 2950 port. I can see that the 5 minuted average between 10:00 and 10:05 GMT spikes up really high for the last few days. I checked the /var/log/messages but could see anything out of the ordinary.

Some of the boxes are an asterisk box doing lots of VOIP traffic. So I checked the CDR's to see if there was any kind of spike in minutes but there wasn't. I also checked the asterisk log files to see if there was anything unusual but that was also a negative.

So apart from sitting in front of the computer and running a live iftop at that time. is there anyway that I can log and then lookback to find out what is going on?

I do have bandwidthd running on the some machines, but the obviously just give daily, weekly and monthly breakdowns which wont help me narrow down on the 5 minute windows that the spike happened in.

Most of my machines are running Centos, my switches are 2950's and my router is a 7200vxr.


Thanks

well, saiyen2002

these are from the top of my mind.

1. How many machines do you have? If it's not significant bandwidth that you have(not over 100Mbit/s) you could monitor the network with wireshark. Adding filters to sniffer could help you peal off the traffic that you consider more likely not to be relevant.

2. I don't want to frighten you, but if you truly haven't been doing anything that you know could produce that kind of traffic, then this sudden "verbosity" of your network could be indication of presence of bot-nets. They usually have exact time when they should contact the server. This peak might mean they all rushed to contact their "boss".
You could check the presence of bot-nets with NIDS such as Snort, though I think it should be last to do, because installation might be "kind of an adventure"(as someone said), and it is certainly killing a fly with a gun, if it's only a service that somehow cut loose and gone wild.
If you decide to run Snort, try downloading Insta-Snorby, because it suppose to work out of the box(at least producers say so). It's free, too...

Hope this helped.
Greetings
Hogar