Bash script to put log files into single file and email
 

I am looking for a bash script that will take my log files, put them into a single file, and email that file to my email account. This is what I have so far, but it doesn't work, and besides I know there is a more effective way to do the same thing.

If I run this file I get the errors:
If somebody knows of a more efficient way to do this (which I'm sure there is) please, do tell.

Thanks,
DragonM15

There is a typo in your script. You did not define
MYFILENAME, instead, you defined MYFILEMAME --

MYFILEMAME="$HOME/systats"
!(MAME should be NAME)

Ok, I admit I feel like a complete and utter moron for that one. But now for the next hump:
If I run with -v (for verbose) I get:
If that helps in figuring out why it wont work... I know it has to do with the number 8 in the date... but why does it work if you simply put in in quotes outside this script?

Thanks,
DragonM15

If you read this post before, I DID NOT fix the problem. I only eliminated the error messages by putting '''s around the $LOGDATE 's. It still wont output the data to the file. Any ideas?

Thanks,
DragonM15

Without changing the structure of the script, a proper construct is
Piping the output from cat to the grep command is unuseful because grep itself is able to process the content of a file (not only the standard input as the one coming from the cat command).
The errors you obtain are related to the interpretation of the LOGDATE and APACHEACDATE variables. As you already discovered the quotes are useful to let the shell interpret a string literally, despite of the presence of special characters or blank spaces, but you should put every variable inside quotes when you use it. In other words, it is not enough to put quotes only in the variable assigments at the beginning of the script. Every time the shell encounter a variable, expands it whitout preserving the surrounding quotes. So, the line
is expanded as
where the grep command tries to look for the pattern "Oct" in the file "8". Instead, in the following line you have another typo
since you defined APACHEACDATE at the beginning. In this case the unassigned APACHEACDATA is expanded as a null string and the resulting command is grep without any pattern to search.
Just a note about debugging: executing the script by "bash -x" will give the execution trace with all the variable expansions performed by the shell. The output can be difficult to read, but it gives many more debugging info. Another hint is to look for differences in the usage of single quotes ' and double quotes ".

As stated above you should run the script with bash -x scriptname.sh and you will see a lot of errors (if there is any). I used too think that using the -x option didn't help me much...but now I use it all the time for debugging...


-custangro
ps I like using ksh...just my preferance... ;)

Hi,

There should be double quotes around the $LOGDATE part.

This: cat /var/log/apache/error_log | $GREP $LOGDATE >> $MYFILENAME

Should be: cat /var/log/apache/error_log | $GREP "$LOGDATE" >> $MYFILENAME

Grep sees this: grep oct 09 infile as grep oct in files 09 and infile, so the oct 09 part should be: grep "oct 09" infile.

Hope this helps.

Worked like a charm colucix, thanks for the help...

cat file file2 file3 | mailx -vs "logs" user@blah.com

The wait statements in the OP are not necessary. wait will pause until background jobs have completed, but you are not running the cat commands in the background.

i.e. this might make sense:
...it will allow command1 and command2 to operate in parallel, and the wait statement will halt the progress of the script until they are both done, and only then proceed with command3.

In the case where you are appending files somewhere, running in parallel is probably not what you want to do, unless you want all the data mixed up.

Something else which may be of interest to you: the more command, when called with multiple input files named as arguments, and re-directed to a file will prepend the content of each file with
So, as long as you don't mind the ::::::: bits, you can accomplish much the same thing as the intent of the OP, just like this:
The shell also allows for some flexibility in formatting of your code - you can put a backslash (\) as the last character on a line, and the shell will ignore the line break. This can greatly help the readability and maintainability of your scripts. For example, the command above is equivalent to this:
Note: for this to work there must be no characters after the \ even spaces or tabs.

Yup, Colucix suggested that before. Plus a preferable invocation (saving the waste of typing and forking 'cat').

DragonM15,
to make the script more generic, you may want to generate the date you're looking for, e.g., This way you shouldn't need to change it daily.

Also beware that when the log is rolled, you might miss a part. E.g., my /var/log/messages gets rolled every Sunday at 4:02; presumably the FC2 default. In that case, some relevant messages may be in /var/log/messages.1. You could add that as another argument to grep , e.g.,
I'd put it as the first file, right after the pattern, to preserve chronological order.

If you're willing to change the structure, you could reduce code duplication by using a loop (and curly braces):
The apache access log and dmesg would have to be done separately.


BTW, if you'd like a ready-made solution, look at LogWatch. It's running on my machine, and it does a decent job of summarizing all noteworthy log entries in a daily e-mail.

This wouldnt do what I want it to do.... yes it would give me the log files, but it would give me ALL of the log files... I just want it a day at a time. Isn't that an organized way of doing things?

Thanks,
DragonM15

Did you read Quigi's post?

This might work, too:
Notes:

• I used single letter variables for brevity, you can make yours as long as you like.
• You cannot put variables inside a brace expansion.
• On my system the (not-yet-compressed) previous log file file has a '.0' extension.

well thanks for the appreciation.

Awfiully sorry I haven't got time to sit there and work out a perfect solution,
but it was a pointer to an easy way to mail log files.
surely you could work out how to limit it to
today's log file only?

phew