I want to lookup for error in running logs which are rotating after reaching file size of 5MB. Rotation of logs will be as follows :
default.log (Current logs)
default.log.1 (after reaching 5MB of default.log)
default.log.2 and so on.

Error log which is coming would be like follows:
28 Jul 2010 10:54:29,968 [com.bea.wlcp.wlng.edr.publisher.EdrBatchingPublisher$BatchProcessor@8d91a93] DEBUG com.bea.wlcp.wlng.snmp.SNMPInternalAlarmListener - Send SNMP v2 trap: ( 18507593 0 0 ( 1.3.6.1.2.1.1.3.0 96527164 )
( 1.3.6.1.6.3.1.1.4.1.0 1.3.6.1.4.1.140.627.0.110616 )
( 1.3.6.1.6.3.1.1.4.3.0 1.3.6.1.4.1.140.627 )
( 1.3.6.1.4.1.140.627.100.1 3 )
( 1.3.6.1.4.1.140.627.100.2 n/a )
( 1.3.6.1.4.1.140.627.100.3 3940649740689060 )
( 1.3.6.1.4.1.140.627.100.4 110616 )
( 1.3.6.1.4.1.140.627.100.5 Failed to bind to SMSC. )
( 1.3.6.1.4.1.140.627.100.6 10.64.0.7_11 )
( 1.3.6.1.4.1.140.627.100.7 9081 )
( 1.3.6.1.4.1.140.627.100.8 WLNG_NT1 )
( 1.3.6.1.4.1.140.627.100.9 Plugin_px21_short_messaging_smpp_56300_north )

Previously i am using code as follows:
#####Code:
tail -F default.log | grep "Failed to bind to SMSC" >> file.txt &
######END of code

Output it produces in file is only one line:
( 1.3.6.1.4.1.140.627.100.5 Failed to bind to SMSC. )

Now i want my output will be written in file as:
date(28 Jul 2010 10:54:29,968 which is in starting of error line)| reason(Failed to bind to SMSC which is serach keyword) | Plugin Name(Plugin_px21_short_messaging_smpp_56300_north which is on 4th line
after the error line)

and continue to append.

For example
28 Jul 2010 10:54:29,968| Failed to bind to SMSC |Plugin_px21_short_messaging_smpp_56300_north
29 Jul 2010 10:54:29,968| Failed to bind to SMSC |Plugin_px21_short_messaging_smpp_56300_south

Thanks in advance for help.

Well that all sounds fine and dandy ... what distinguishes any of these lines from what I am guessing is potentially thousands of lines?
Is the example above the same format everytime? ie for every error?
Have you not shown us the continuing lines after the last one as it is just more of the same?

From looking at your data provided I can see the words:

Plugin
Failed
SNMP

If these are unique items then yes you can get them, but the issue I see is how to know when to get them???

Sorry to not provide a solution but as you can see there are some other questions that need to be answered first.

0 members found this post helpful.

Hi,

I don't think regular grep is the way to go. You would be better of with your favorite script language.

Personally I would go for awk (most installations have it). As my skills are rusty I'm not going to create a solution, sorry. But take a look at this site and search for awk+parse+file and I'm sure you will be able to figure it out.

Best wishes.

0 members found this post helpful.

4th line or 4th column? You can use head and tail to extract individual lines. You have to use something like awk to extract parts of a line. Or one of the many other tools out there.

0 members found this post helpful.

Hi Grail,

Find your answers inline:
1. what distinguishes any of these lines from what I am guessing is potentially thousands of lines?
Answer : There are 1000 or even more lines and this line is generated and format is same when any plugin is down from pludins that are running....

2.Is the example above the same format everytime? ie for every error?
Have you not shown us the continuing lines after the last one as it is just more of the same?
Answer : The format is same but generated limited when any of running plugin is down.If all plugin is up there is no such type of logs.

3.From looking at your data provided I can see the words:

Plugin
Failed
SNMP

If these are unique items then yes you can get them, but the issue I see is how to know when to get them???
Answer : Whenever we found 'Failed to bind to SMSC' in logs these words are come automatically with them and this means that one or more plugin is down from list of plugins configured.

Hope above answers clears your queries.

Yes I am with you so far It is good that the format is the same as I agree with others that have mentioned awk as a solution.
Would it be possible to see at least 2 or 3 errors from the log?
Also, are there other items in the log file? (ie not errors)

If yes to last question, then it would be helpful to see a portion of the log containing both other data and the error data.

A good solution in terms of tracking and re-formatting would be Perl, using the File::Tail module http://search.cpan.org/~mgrabnar/Fil...0.99.3/Tail.pm. It automatically handles issues such as log rotation.

Hi Grail,

This log file contains error as well as general logs for debugging also.
Find the sample as follows:
04 Aug 2010 09:43:24,742 [[ACTIVE] ExecuteThread: '108' for queue: 'weblogic.kernel.Default (self-tuning)'] DEBUG org.apache.axis.encoding.SerializationContext - [dci_plugin_InstanceFour#wlng_nt_payment_dci#1.0] Start element [http://schemas.xmlsoap.org/soap/envelope/]:Body
04 Aug 2010 09:43:24,742 [[ACTIVE] ExecuteThread: '108' for queue: 'weblogic.kernel.Default (self-tuning)'] DEBUG org.apache.axis.i18n.ProjectResourceBundle - [dci_plugin_InstanceFour#wlng_nt_payment_dci#1.0] org.apache.axis.i18n.resource::handleGetObject(register00)
04 Aug 2010 09:43:24,742 [[ACTIVE] ExecuteThread: '108' for queue: 'weblogic.kernel.Default (self-tuning)'] DEBUG org.apache.axis.encoding.SerializationContext - [dci_plugin_InstanceFour#wlng_nt_payment_dci#1.0] register 'a1' - 'http://telcordia.com/cvas/rcs/SAM/BalanceRetrieve/schemas/BalanceRetrAll/v1_0'

Hope it will clear your requirement and i can't use awk and i have to catch logs at runtime.

Then I would look into chrism01's suggestion in post #7

Reading the man page for grep, I see that (at least on OpenSuSE using bash) you can use grep -A x <searchstring> to print the next x lines after each hit:

Ex:
========================
$ dmesg | grep -A 3 memory
init_memory_mapping
0000000000 - 0077e00000 page 2M
0077e00000 - 0077e70000 page 4k
kernel direct mapping tables up to 77e70000 @ 10000-14000
--
PM: Registered nosave memory: 000000000009d000 - 000000000009e000
PM: Registered nosave memory: 000000000009e000 - 00000000000a0000
PM: Registered nosave memory: 00000000000a0000 - 00000000000d0000
PM: Registered nosave memory: 00000000000d0000 - 0000000000100000
Allocating PCI resources starting at 88000000 (gap: 80000000:60000000)
PERCPU: Allocating 61472 bytes of per cpu data
NR_CPUS: 512, nr_cpu_ids: 2, nr_node_ids 1
--
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
--
Freeing initrd memory: 6201k freed
audit: initializing netlink socket (disabled)
type=2000 audit(1280954570.909:1): initialized
HugeTLB registered 2 MB page size, pre-allocated 0 pages
========================

In this case, I used grep to find any references to "memory" in the output of the dmesg program. Using the -A x option allows me to specify how many lines of output to include after each instance of the search string being found. Using the -B x option allows you to print x number of lines before each match. This allows you to get some context surrounding your match.

Thanks ahedler,

this really works.