I implemented an echo server(in c++) that mainly forks and execs multiple iperf commands(linux) and I am getting a kernel crash. I'm pretty new to kernel programming and don't really know how to read the kernel crash log and was hoping someone would help me. The linux box crashes so randomly that it is hard to pinpoint where it is crashing. Most of the time it works for about 2 hours(about 20-30 iperf commands) then the kernel freezes and crashes and needs a reboot. I caught the crash once and it seemed to crash in the middle of an iperf command.
Here is the version of linux I am running.
Quote:
|
Linux 5NetSim08 2.6.35-22-generic #35-Ubuntu SMP Sat Oct 16 20:36:48 UTC 2010 i686 GNU/Linux
|
And here is the crash log
Quote:
[ 1883.081940] BUG: unable to handle kernel NULL pointer dereference at 000002c0
[ 1883.089141] IP: [<c153d24f>] __udp4_lib_rcv+0x16f/0x680
[ 1883.089141] *pdpt = 0000000000000000 *pde = f000eef3f000eef3
[ 1883.089141] Oops: 0000 [#1] SMP
[ 1883.089141] Modules linked in: coretemp gpio_ich snd_hda_codec_realtek microcode parport_pc rfcomm bnep ppdev bluetooth snd_hda_intel psmouse snd_hda_codec snd_hwdep nfsd nfs binfmt_misc serio_raw lockd fscache auth_rpcgss nfs_acl snd_pcm sunrpc i915 lpc_ich drm_kms_helper snd_seq_midi snd_rawmidi snd_seq_midi_event drm i2c_algo_bit snd_seq snd_timer snd_seq_device mac_hid snd soundcore video snd_page_alloc lp parport r8169
[ 1883.089141]
[ 1883.089141] Pid: 0, comm: swapper/0 Not tainted 3.5.0-25-generic #39~precise1-Ubuntu /i945GSEx-QS R2.00 May.24.2010
[ 1883.089141] EIP: 0060:[<c153d24f>] EFLAGS: 00010282 CPU: 0
[ 1883.089141] EIP is at __udp4_lib_rcv+0x16f/0x680
[ 1883.089141] EAX: 00000000 EBX: f68609c0 ECX: f1367d80 EDX: f2ad344e
[ 1883.089141] ESI: f2ad3462 EDI: 00000011 EBP: f4c0be08 ESP: f4c0bdb4
[ 1883.089141] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 1883.089141] CR0: 8005003b CR2: 000002c0 CR3: 019a1000 CR4: 000007e0
[ 1883.089141] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
[ 1883.089141] DR6: ffff0ff0 DR7: 00000400
[ 1883.089141] Process swapper/0 (pid: 0, ti=f4c0a000 task=c1869240 task.ti=c185c000)
[ 1883.089141] Stack:
[ 1883.089141] 00004e3c f6860c00 000069c0 00000440 01000000 f68609c0 00004b28 c154f851
[ 1883.089141] f596ac00 f6860c00 00004e28 0b09a8c0 c18d8c88 f1368b17 000042c3 c18c5b00
[ 1883.089141] 0b09a8c0 0b0aa8c0 f68609c0 c165c59c 00000011 f4c0be10 c153d777 f4c0be34
[ 1883.089141] Call Trace:
[ 1883.089141] [<c154f851>] ? inet_frag_destroy+0xc1/0x100
[ 1883.089141] [<c153d777>] udp_rcv+0x17/0x20
[ 1883.089141] [<c15148a7>] ip_local_deliver_finish+0xa7/0x290
[ 1883.089141] [<c1514bdf>] ip_local_deliver+0x3f/0x80
[ 1883.089141] [<c1514567>] ip_rcv_finish+0xf7/0x390
[ 1883.089141] [<c1514e41>] ip_rcv+0x221/0x320
[ 1883.089141] [<c1143567>] ? kmem_cache_alloc+0x77/0x140
[ 1883.089141] [<c14e91d7>] __netif_receive_skb+0x437/0x4c0
[ 1883.089141] [<c14e927f>] netif_receive_skb+0x1f/0x80
[ 1883.089141] [<c14e965b>] ? dev_gro_receive+0x16b/0x240
[ 1883.089141] [<c14e941f>] napi_skb_finish+0x4f/0x70
[ 1883.089141] [<c14ea299>] napi_gro_receive+0xe9/0x110
[ 1883.089141] [<f8438c6c>] rtl_rx+0x9c/0x300 [r8169]
[ 1883.089141] [<f843bc56>] rtl8169_poll+0xc6/0xd0 [r8169]
[ 1883.089141] [<c14e99fd>] net_rx_action+0x10d/0x1e0
[ 1883.089141] [<c104e190>] ? local_bh_enable_ip+0x90/0x90
[ 1883.089141] [<c104e211>] __do_softirq+0x81/0x1a0
[ 1883.089141] [<c104e190>] ? local_bh_enable_ip+0x90/0x90
[ 1883.089141] <IRQ>
[ 1883.089141] [<c104e566>] ? irq_exit+0x76/0xa0
[ 1883.089141] [<c15eb01b>] ? do_IRQ+0x4b/0xc0
[ 1883.089141] [<c109953a>] ? tick_notify+0x11a/0x1d0
[ 1883.089141] [<c15eae70>] ? common_interrupt+0x30/0x38
[ 1883.089141] [<c104007b>] ? post_kmmio_handler+0x4b/0xc0
[ 1883.089141] [<c132eaf2>] ? intel_idle+0xc2/0x120
[ 1883.089141] [<c14a8435>] ? cpuidle_enter+0x15/0x20
[ 1883.089141] [<c14a89cc>] ? cpuidle_idle_call+0x9c/0x260
[ 1883.089141] [<c10195fa>] ? cpu_idle+0xaa/0xe0
[ 1883.089141] [<c15afe65>] ? rest_init+0x5d/0x68
[ 1883.089141] [<c18daa1f>] ? start_kernel+0x375/0x37b
[ 1883.089141] [<c18da62b>] ? pass_bootoption.constprop.3+0xaf/0xaf
[ 1883.089141] [<c18da303>] ? i386_start_kernel+0xa6/0xad
[ 1883.089141] Code: 0f b7 4e 02 0f b7 06 66 89 4d e0 8b 4b 10 85 c9 0f 85 93 04 00 00 8b 4b 48 0f b7 c0 89 45 e4 8b 42 0c 83 e1 fe 89 45 d8 8b 41 0c <8b> 80 c0 02 00 00 89 45 d0 8b 45 dc 89 44 24 0c 8b 41 70 8b 4d
[ 1883.089141] EIP: [<c153d24f>] __udp4_lib_rcv+0x16f/0x680 SS:ESP 0068:f4c0bdb4
[ 1883.089141] CR2: 00000000000002c0
[ 1883.561261] ---[ end trace a7bdb48fae24ea76 ]---
[ 1883.565953] Kernel panic - not syncing: Fatal exception in interrupt
[ 1883.581374] panic occurred, switching back to text console
|
Any help diagnosing the problem would be greatly appreciated. Thank you
