Hello, all.

I'm trying to pull certain information out of a log file, looking back some period of time (e.g. the past hour). I've found the following awk construct that works, capturing the data as a variable:

myvar="$(awk '$0>=from&&$0<=to' from="$(date +%b" "%d" "%H:%M:%S -d -10minute)" to="$(date +%b" "%d" "%H:%M:%S)" /var/log/messages)"

I'd like to be able to get certain information from that variable, for use to create an alarm, but am not sure how. Grep? Something else? If anyone knows the solution, and can show me how, I'd very much appreciate it.

Can we see what is in that variable $myvar, and elaborate on what is "certain information".

Also I would recommend to split that complicated one-liner in various statements and variables. It makes debugging easier.

jlinkels

Some suggestions are to post representative data of what you expect to see and cite the desired patterns you'd like to detect. Then by using a regular expression you can attain filtering to detect or edit data using various techniques. For those, and for my experiences, it would be grep to search and sed to edit, but there are others.

Bear further in mind that people will likely be more willing to work at it and offer a better solution if you at least demonstrate some script or code where you've tried but not yet attained your desired outcome.

Please use [code] tags to include any code clips. If you don't know how to, there's a link to show how in my signature.

I'm actually trying to redo a script that's looking over a bind log for anomalies that would trigger an email alert. As a start, I worked on the awk construct above to at least give me all data from a log for the past x time (e.g. 1 hour). I ran this against syslog, creating a variable. The variable returned the all of the data in syslog for the specified period. Now, I'd like to go the next step, and parse out an entire entry from the variable. For example, of all the data captured in the variable, this information will be used to trigger an email alert. An example from syslog: "Apr 16 12:32:26 satest01 kdump: kexec: loaded kdump kernel". I'd like to use "kdump" as the searched word, then send the entire line in the body of the email message.

Hope the above is understandable as you try to assist me.

Best assistance I can provide at this point would be to state that "anything you can do via typing on the command line, you can script", there are links for BASH scripting guides in my signature as well as a blog entry. You save this information to a variable, you feel you wish to use search that variable to find matching entries. Step (1) is to save the information to a variable, that will stay with the shell as a variable so long as you don't clear it or exit that command shell. Step (2) would be to determine what search statement works best for you. And so forth, and then you can put those steps into a BASH script. When you've proceeded to a point where you have some form of script and are stuck, people will certainly offer some suggestions.

A thing to note here is actually that grep will not work in this case, you're searching an environment variable for a substring, therefore you'll need to either research in BASH how you match sub-strings out of a larger string, or you should consider that when you perform that awk statement, or similar that you process better the output so that you construct an array or list of information already in the form you wish.

Thanks, rtmisler. The reason I'm using that awk statement is to gather data for a specific time period (for example, as mentioned earlier, the last hour). I kind of sussed that I couldn't use grep against that variable; a lot of tries seem to confirm that. The awk statement works fine. If I run it with syslog as the input, I get back an e.g. hour's worth of log data as output. I'm trying to find a way to filter the data even more by using a keyword or phrase. Say the awk piece returns the following:

Apr 16 13:28:05 satest01 puppet-agent[15689]: Finished catalog run in 0.20 seconds
Apr 16 13:36:31 satest01 yum[16120]: Installed: 2:xinetd-2.3.14-39.el6_4.x86_64
Apr 16 13:36:54 satest01 xinetd[16150]: xinetd Version 2.3.14 started with libwrap loadavg labeled-networking options compiled in.
Apr 16 13:36:54 satest01 xinetd[16150]: Started working: 1 available service
Apr 16 13:58:05 satest01 puppet-agent[16923]: Finished catalog run in 0.25 seconds

I only want the lines with "xinetd" returned, and only if they occurred during the past hour.

I'm actually not an AWK person, SED somewhat, but not AWK.

An initial recommendation though would be to use GREP on the file to extract the lines with "xinetd" and then use your AWK statement to limit the information to the time range you select.

Ok..can you post what you have written/tried of your own so far?? Have you checked the bash scripting tutorials (one link in my posting signature), for help in getting going??

If I had to write this, I'd first figure out how to get just the 1 hours worth of data. From there, I'd run it through grep to bring out whatever lines match a rough pattern, and from there, process them further if needed.

All that aside...what are you trying to accomplish? There are MANY existing logfile 'watchers' already, logwatch being a common one:
http://linux.die.net/man/8/logwatch

If you have an existing monitoring system in place (Nagios/Zabbix/Big Brother), ALL of them have log file plugins available.

Thanks to you both.

The awk statement grabs one clock-hour's worth of data. If I tried grepping for a time period from within a log itself, I think that would be much more complicated, and much less accurate.

The awk statement does capture an hour's worth of data. It's how to grep (or whatever) specific data from within that set that I'm having a problem with.

Well, how you get the data is up to you. If this method works for you, then go for it.
Why is that an issue? Have you looked at the man page for grep?? Once you have the data you want to look for (and you say you DO), and you know what you want to look for (and you say you do..the xinetd string), then you're almost done. Shove the array/variable/whatever-you're-doing (you still don't post your script), through grep and look for xinetd.

Again, post what you've written/done/tried of your own. And, why doesn't logwatch work for you, since it already does EXACTLY what you're after??