[SOLVED] NEWBIE: Find a date just after a string?

hristo77 · 11-03-2015, 06:12 AM

Hi,

Im looking to build a simple script that looks in a log after RMAN-errors. The log looks like this:

Code:

--------------------------------------------------
20151102 23:47 Completed rman INC backup of testdb01
20151102 23:47 Starting rman INC backup of testdb02
20151103 00:10 Completed rman INC backup of testdb02
20151103 00:10 Starting rman INC backup of testdb03

The ---------- indicates a new backup. How can I look for the first date after --------
Date in this case will be yesterday. When I find ----- + yesterday I would like to grep after RMAN-errors.

This is what I have so far:


log_file=/appl/app/oracle/admin/scripts/backup/log/logfile_rman.log

day_to_check=$(date +Y%m%d -d yesterday)

#expdp_errors=`grep -e "$day_to_check" $log_file |  grep -e 'ORA'`
#find . -name "logfile_rman.log" -exec grep "RMAN-" {} +

expdp_errors=`grep "$day_to_check" -C 1 $log_file | grep -C 1 'RMAN-'`

if [ "$expdp_errors" != 0 ];
then
echo $expdp_errors
exit 2
else
echo "No errors!"
exit 0
fi

Which gives me this output:

./check_rman.sh
20151102 01:12 Starting rman FULL backup of testdb01 RMAN-00571: =========================================================== -- RMAN-03002: failure of backup plus archivelog command at 11/02/2015 01:12:06 20151102 01:12 Error: rman returned Error for testdb01

Regards
H

hristo77 · 11-03-2015, 06:40 AM

I want two look after ------ because otherwise I have to check for two diffrent dates in the logfile. If I look for errors only after ---- and a date I dont have to

Regards
H

rtmistler · 11-03-2015, 06:46 AM

To make your code more read-able to others, please use [code] tags. There are instructions in the FAQ and I have a link to instructions about the topic in my signature. Note that you can edit your original to do that. It helps to take care of indentations and spacings.

This is your main body of work:

Code:

expdp_errors=`grep "$day_to_check" -C 1 $log_file | grep -C 1 'RMAN-'`

It appears that your example file chunk does not include an error, which I believe would include the string "RMAN-" is that a correct assumption?

My few guidances here are:

Remove the -C 1 term from the first part of the grep.

The reason being is that it causes you to find only one instance of yesterday's logs and that's not your intention. You instead need to first find all log entries from yesterday, and then when you pipe it to your next grep, where you search for the RMAN- string.
Use set -xv after the top of your script to enable debug and what this will do is show much more output to stdout as your script runs and let you know things like how each grep statement ends up completing.
I feel that for the second part of that grep, you need not use the CAPITAL -C but instead you can use the LOWERCASE -c flag to get a "count" of instances where grep finds the string RMAN- in your yesterday's logs. Then you can output how many log entries you encountered.

rtmistler · 11-03-2015, 06:48 AM

Quote:

Originally Posted by hristo77

I want two look after ------ because otherwise I have to check for two diffrent dates in the logfile. If I look for errors only after ---- and a date I dont have to

Regards
H

This confuses me because it does not appear as if your script is looking after "-----" but instead it is looking for "yesterday's" dated logs and then looking for the string RMAN-

Perhaps you can post a more inclusive example of your log file.

hristo77 · 11-03-2015, 07:07 AM

Hi,

This is how the log file looks:

Code:

20151102 01:42 Starting rman FULL backup of testdb3
20151102 01:44 Completed rman FULL backup of testdb3
-------------------------------------------------
20151102 23:00 Starting rman INC backup of test01ddb
20151102 23:25 Completed rman INC backup of test01ddb
20151102 23:25 Starting rman INC backup of test02ddb
20151102 23:47 Completed rman INC backup of test02ddb
20151102 23:47 Starting rman INC backup of test03ddb
20151103 00:10 Completed rman INC backup of test03ddb
20151103 00:10 Starting rman INC backup of test04ddb
20151103 00:21 Completed rman INC backup of test04ddb
20151103 00:21 Starting rman INC backup of test07ddb
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of backup plus archivelog command at 11/03/2015 00:22:00
20151103 00:22 Error: rman returned Error for test07ddb
20151103 00:22 Starting rman INC backup of test08ddb

So I would like to look after errors after the ------ (which indicates a new backup). The code I posted is my fatal attempt to script something...

So I want to search for ---- on yesterdays date and then scan the rest of the rows for RMAN-errors.

One other solution would be to just have the script look for RMAN-errors during the last 6 or so hours as there are timestamps in the log. But not sure on how to format date for that.

Regards
H

rtmistler · 11-03-2015, 07:28 AM

I'll have to put some thought into it. Likely someone else with much greater regular expressions experience will offer a better solution.

The issues here are multiple.

Firstly, the logs containing the string "RMAN-" do not have day-time stamps, so you cannot use the "yesterday" convention to even locate those logs.

Next, the logs containing the string "RMAN-" do not contain information of any substance, so why bother grabbing them, it would seem that the logs following those are really the ones you could use. Otherwise this is just a bell ringer where you know that "something" happened, but not why.

Finally, the "-----" convention is there, but confusing. There are several backup starts and completions for 11/2 and they aren't all delimited by that notation. And then there are a few backup starts and completions for 11/3 also without that notation. Further, the notation does not also delimit from day to day. It would seem instead that it's best to grab the chunk of file delimited between the start of a day and either the end of the log or the end of a particular day. For instance, 11/2 is now complete, so grab the chunk of the file contain all logs from 11/2. Bear in mind that you cannot just use your "yesterday" convention because some of the logs do not contain the day-time stamp. But actually the one log which you can make use out of does:

Code:

20151103 00:22 Error: rman returned Error for test07ddb

Why not modify your script to search for yesterday's time stamped logs (already doing that), but then pipe it to grep for the string "Error"? And then you'll get that log which at least tells you something specific, that you can deal with, if possible. There may be other logs that do not fit that model, but for now there's only that example for me to observe and recommend.

hristo77 · 11-04-2015, 02:29 AM

Hi,

I used this:

Code:

grep "\-*" logfile_rman.log -A 100| grep "20151103" -A 500 | grep -C 1 'RMAN-'

20151103 00:21 Starting rman INC backup of tcu07d
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of backup plus archivelog command at 11/03/2015 00:22:00
20151103 00:22 Error: rman returned Error for tcu07d

Which is what I want

Regards
H

rtmistler · 11-04-2015, 07:12 AM

Quote:

Originally Posted by hristo77

Hi,

I used this:

Code:

grep "\-*" logfile_rman.log -A 100| grep "20151103" -A 500 | grep -C 1 'RMAN-'

20151103 00:21 Starting rman INC backup of tcu07d
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of backup plus archivelog command at 11/03/2015 00:22:00
20151103 00:22 Error: rman returned Error for tcu07d

Which is what I want

Regards
H

Excellent work

and I'm glad you attained what you wish.

Having had the experience of developing complex things like that at times, I find I very rarely remember them. So as a result I find it may be more useful to create a script such that the argument is the date you wish to grep for. And thus the remainder of the script contains all the rest of the stuff and just uses your passing argument to insert the date desired. This gives flexibility, puts it somewhere in a script file so you can revisit it, or refine it as you need, or learn more, and also makes it easier to use and remember because the interface for a script would be way easier than retyping all the grammar shown above.

You may also want to consider marking the thread as solved for future seekers to notice that you had a problem to solve and ultimately did solve it.