Kernel panic 2.6.32.60

rpgiri2002 · 09-11-2017, 11:26 PM

Hi
I am seeing kernel panic with Linux kernel 2.6.32.60 (i know its quite old). Was there a known bug/commit fixed for this issue?

Hardware config:

CPU1 : Intel(R) Xeon(R) E5-2680 - 8 Cores, 2.7 GHz
CPU2 : Intel(R) Xeon(R) E5-2680 - 8 Cores, 2.7 GHz
DIMM : 16GB DDR3-1600-MHz RDIMM/PC3-12800/dual rank/1.35v
Network: Cisco VIC 1240
Storage: RAID Controller LSI MegaRAID SAS 2004 ROMB
Storage: Fusion IO 365GB

Note, i have seen the crash in RHEL 2.6.32-71, 2.6.32-131.0.15, 2.6.32-220 and 2.6.32-279. But 2.6.32-358+ (on wards) the kernel panic is not seen

[May 30 09:50:24.043] [ 1.735000] Time: 13:51:16 Date: 05/29/17
[ 1.736000] NET: Registered protocol family 16
[ 1.738000] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
[ 1.739000] ACPI: bus type pci registered
[ 1.740000] PCI: MCFG configuration 0: base 80000000 segment 0 buses 0 - 255
[ 1.741000] PCI: MCFG area at 80000000 reserved in E820
[ 1.772000] PCI: Using MMCONFIG at 80000000 - 8fffffff
[ 1.773000] PCI: Using configuration type 1 for base access
[ 1.787000] bio: create slab <bio-0> at 0
[ 2.114000] ACPI: Interpreter enabled
[ 2.115000] ACPI: (supports S0 S5)
[ 2.118000] ACPI: Using IOAPIC for interrupt routing
[ 2.158000] ACPI: No dock devices found.
[ 2.160000] ACPI: PCI Root Bridge [PCI0] (0000:00)
[ 2.161000] pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
[ 2.162000] pci 0000:00:00.0: PME# disabled
[ 2.163000] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
[ 2.164000] pci 0000:00:01.0: PME# disabled
[ 2.165000] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold
[ 2.166000] pci 0000:00:02.0: PME# disabled
[ 2.167000] pci 0000:00:11.0: PME# supported from D0 D3hot D3cold
[ 2.168000] pci 0000:00:11.0: PME# disabled
[ 2.169000] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold
[ 2.170000] pci 0000:00:1a.0: PME# disabled
[ 2.171000] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
[ 2.172000] pci 0000:00:1c.0: PME# disabled
[ 2.173000] pci 0000:00:1c.7: PME# supported from D0 D3hot D3cold
[ 2.174000] pci 0000:00:1c.7: PME# disabled
[ 2.175000] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold
[ 2.176000] pci 0000:00:1d.0: PME# disabled
[ 2.178000] pci 0000:02:00.0: PME# supported from D0 D3hot
[ 2.179000] pci 0000:02:00.0: PME# disabled
[ 2.180000] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold
[ 2.181000] pci 0000:03:00.0: PME# disabled
[ 2.182000] pci 0000:04:00.0: PME# supported from D0 D3hot D3cold
[ 2.183000] pci 0000:04:00.0: PME# disabled
[ 2.185000] pci 0000:05:00.0: PME# supported from D0 D3hot D3cold
[ 2.186000] pci 0000:05:00.0: PME# disabled
[ 2.187000] pci 0000:05:01.0: PME# supported from D0 D3hot D3cold
[ 2.188000] pci 0000:05:01.0: PME# disabled
[ 2.190000] pci 0000:00:1e.0: transparent bridge
[ 2.204000] ACPI: PCI Root Bridge [UNC0] (0000:7f)
[ 2.210000] ACPI: PCI Root Bridge [PCI1] (0000:80)
[ 2.212000] pci 0000:80:02.0: PME# supported from D0 D3hot D3cold
[ 2.213000] pci 0000:80:02.0: PME# disabled
[ 2.2
[May 30 09:50:25.001] 14000] pci 0000:81:00.0: PME# supported from D0 D1 D2 D3hot
[ 2.215000] pci 0000:81:00.0: PME# disabled
[ 2.218000] ACPI: PCI Root Bridge [UNC1] (0000:ff)
[ 2.224000] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 *11 12 14 15)
[ 2.232000] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 *10 11 12 14 15)
[ 2.240000] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 *5 6 10 11 12 14 15)
[ 2.248000] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 10 *11 12 14 15)
[ 2.256000] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 10 11 12 14 15) *0
[ 2.265000] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11 12 14 15) *0
[ 2.274000] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 10 11 12 14 15) *0
[ 2.283000] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 *7 10 11 12 14 15)
[ 2.291000] vgaarb: device added: PCI:0000:0a:00.0,decodes=io+mem,owns=io+mem,locks=none
[ 2.292000] vgaarb: loaded
[ 2.293000] SCSI subsystem initialized
[ 2.295000] usbcore: registered new interface driver usbfs
[ 2.297000] usbcore: registered new interface driver hub
[ 2.298000] usbcore: registered new device driver usb
[ 2.300000] PCI: Using ACPI for IRQ routing
[ 2.302000] initcall pci_subsys_init+0x0/0x10a returned with preemption imbalance
[ 2.303000] BUG: unable to handle kernel paging request at 000000038da1e618
[ 2.303000] IP: [<ffffffff8102f849>] task_tick_fair+0x22/0xb1
[ 2.303000] PGD 0
[ 2.303000] Thread overran stack, or stack corrupted
[ 2.303000] Oops: 0000 [#1] SMP
[ 2.303000] last sysfs file:
[ 2.303000] CPU 0
[ 2.303000] Modules linked in:
[ 2.303000] Pid: 1, comm: swapper Not tainted 2.6.32.60 #2 UCSB-B200-M3
[ 2.303000] RIP: 0010:[<ffffffff8102f849>] [<ffffffff8102f849>] task_tick_fair+0x22/0xb1
[ 2.303000] RSP: 0018:ffff880028203ec8 EFLAGS: 00010082
[ 2.303000] RAX: 0000000081841b5f RBX: 0000000000000000 RCX: 0000000000000004
[ 2.303000] RDX: 0000000000000000 RSI: ffff886067dc0000 RDI: ffff880028211480
[ 2.303000] RBP: ffff886067dc0038 R08: 0000000000000005 R09: 0000000000000400
[ 2.303000] R10: 000f424000000000 R11: 000f424000000000 R12: 0000000000011480
[ 2.303000] R13: 0000000000011480 R14: ffff886067dc0000 R15: 0000000000000000
[ 2.303000] FS: 0000000000000000(0000) GS:ffff880028200000(0000) knlGS:0000000000000000
[ 2.303000] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
[ 2.303000] CR2: 000000038da1e618 CR3: 0000000001001000 CR4: 00000000000406f0
[ 2.303000] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 2.303000] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 2.303000] Process swapper (pid: 1, threadinfo ffff883067eb6000, task ffff886067dc0000)
[ 2.303000] Stack:
[ 2.303000] ffffffff81402580 0000000000000000 ffff880028211480 000000008944fdc0
[ 2.303000] <0> 0000000000011480 ffffffff810338f0 0000000000000000 0000000000000000
[ 2.303000] <0> ffffffff8106726d 0000000000000000 ffff88002820b980 0000000000000000
[ 2.303000] Call Trace:
[ 2.303000] <IRQ>
[ 2.303000] [<ffffffff810338f0>] ? scheduler_tick+0x106/0x15a
[ 2.303000] [<ffffffff8106726d>] ? __rcu_pending+0x3e/0xaf
[ 2.303000] [<ffffffff810421ad>] ? update_process_times+0x3c/0x49
[ 2.303000] [<ffffffff81052caa>] ? tick_periodic+0x5b/0x5d
[ 2.303000] [<ffffffff81052cc4>] ? tick_handle_periodic+0x18/0x5f
[ 2.303000] [<ffffffff8101ca25>] ? smp_apic_timer_interrupt+0x80/0x92
[ 2.303000] [<ffffffff8100c52e>] ? apic_timer_interrupt+0xe/0x20
[ 2.303000] [<ffffffff8100c533>] ? apic_timer_interrupt+0x13/0x20
[ 2.303000] <EOI>
[ 2.303000] [<ffffffff811741f1>] ? vgacon_cursor+0x0/0x1b1
[ 2.303000] [<ffffffff81153660>] ? delay_loop+0x0/0x29
[ 2.303000] [<ffffffff81153660>] ? delay_loop+0x0/0x29
[ 2.303000] [<ffffffff8103a65b>] ? vprintk+0x335/0x34a
[ 2.303000] [<ffffffff81841b5f>] ? __reserve_region_with_split+0xa1/0xb0
[ 2.303000] [<ffffffff81859f9a>] ? pci_subsys_init+0x0/0x10a
[ 2.303000] [<ffffffff8103a6fd>] ? printk+0x8d/0x95
[ 2.303000] [<ffffffff81859f9a>] ? pci_subsys_init+0x0/0x10a
[ 2.303000] [<ffffffff8103e88c>] ? insert_resource+0x21/0x3c
[ 2.303000] [<ffffffff81839a63>] ? ioapic_insert_resources+0x43/0x51
[ 2.303000] [<ffffffff81859f9a>] ? pci_subsys_init+0x0/0x10a
[ 2.303000] [<ffffffff8185a9ba>] ? pcibios_init+0x65/0x78
[ 2.303000] [<ffffffff8185a09b>] ? pci_subsys_init+0x101/0x10a
[ 2.303000] [<ffffffff81856608>] ? md_init+0x0/0xcd
[ 2.303000] [<ffffffff81856608>] ? md_init+0x0/0xcd
[ 2.303000] [<ffffffff81859f9a>] ? pci_subsys_init+0x0/0x10a
[ 2.303000] [<ffffffff8100913d>] ? do_one_initcall+0x13c/0x147
[ 2.303000] [<ffffffff8182bc38>] ? kernel_init+0x141/0x18c
[

[May 30 09:50:26.006] 2.303000] [<ffffffff8100ca3a>] ? child_rip+0xa/0x20
[ 2.303000] [<ffffffff8182baf7>] ? kernel_init+0x0/0x18c
[ 2.303000] [<ffffffff8100ca30>] ? child_rip+0x0/0x20
[ 2.303000] Code: fa 73 00 58 c3 eb 99 eb 97 41 55 41 54 55 48 89 f5 48 83 c5 38 53 50 0f 84 95 00 00 00 48 8b 45 d0 49 c7 c4 80 14 01 00 8b 40 18 <4c> 03 24 c5 20 0b 81 81 49 8d 5c 24 58 48 89 df e8 05 f3 ff ff
[ 2.303000] RIP [<ffffffff8102f849>] task_tick_fair+0x22/0xb1
[ 2.303000] RSP <ffff880028203ec8>
[ 2.303000] CR2: 000000038da1e618
[ 2.303000] ---[ end trace ff9d59f7e5fdbc66 ]---
[ 2.303000] Kernel panic - not syncing: Fatal exception in interrupt
[ 2.303000] Pid: 1, comm: swapper Tainted: G D 2.6.32.60 #2
[ 2.303000] Call Trace:
[ 2.303000] <IRQ> [<ffffffff810394eb>] ? panic+0xc4/0x185
[ 2.303000] [<ffffffff81153660>] ? delay_loop+0x0/0x29
[ 2.303000] [<ffffffff81153660>] ? delay_loop+0x0/0x29
[ 2.303000] [<ffffffff811b19ae>] ? extract_entropy+0x4d/0x94
[ 2.303000] [<ffffffff81153660>] ? delay_loop+0x0/0x29
[ 2.303000] [<ffffffff81153660>] ? delay_loop+0x0/0x29
[ 2.303000] [<ffffffff81040000>] ? do_proc_doulongvec_minmax+0x2a2/0x341
[ 2.303000] [<ffffffff81039796>] ? init_oops_id+0x1e/0x2f
[ 2.303000] [<ffffffff810397ad>] ? print_oops_end_marker+0x6/0x1c
[ 2.303000] [<ffffffff813ee87e>] ? oops_end+0xa1/0xae
[ 2.303000] [<ffffffff81027e36>] ? no_context+0x1f1/0x1fe
[ 2.303000] [<ffffffff813ede4f>] ? page_fault+0x1f/0x30
[ 2.303000] [<ffffffff8102f849>] ? task_tick_fair+0x22/0xb1
[ 2.303000] [<ffffffff810338f0>] ? scheduler_tick+0x106/0x15a
[ 2.303000] [<ffffffff8106726d>] ? __rcu_pending+0x3e/0xaf
[ 2.303000] [<ffffffff810421ad>] ? update_process_times+0x3c/0x49
[ 2.303000] [<ffffffff81052caa>] ? tick_periodic+0x5b/0x5d
[ 2.303000] [<ffffffff81052cc4>] ? tick_handle_periodic+0x18/0x5f
[ 2.303000] [<ffffffff8101ca25>] ? smp_apic_timer_interrupt+0x80/0x92
[ 2.303000] [<ffffffff8100c52e>] ? apic_timer_interrupt+0xe/0x20
[ 2.303000] [<ffffffff8100c533>] ? apic_timer_interrupt+0x13/0x20
[ 2.303000] <EOI> [<ffffffff811741f1>] ? vgacon_cursor+0x0/0x1b1
[ 2.303000] [<ffffffff81153660>] ? delay_loop+0x0/0x29
[ 2.303000] [<ffffffff81153660>] ? delay_loop+0x0/0x29
[ 2.303000] [<ffffffff8103a65b>] ? vprintk+0x335/0x34a
[ 2.303000] [<ffffffff81841b5f>] ? __reserve_region_with_split+0xa1/0xb0
[ 2.303000] [<ffffffff81859f9a>] ? pci_subsys_init+0x0/0x10a
[ 2.303000] [<ffffffff8103a6fd>] ? printk+0x8d/0x95
[ 2.303000] [<ffffffff81859f9a>] ? pci_subsys_init+0x0/0x10a
[ 2.303000] [<ffffffff8103e88c>] ? insert_resource+0x21/0x3c
[ 2.303000] [<ffffffff81839a63>] ? ioapic_insert_resources+0x43/0x51
[ 2.303000] [<ffffffff81859f9a>] ? pci_subsys_init+0x0/0x10a
[ 2.303000] [<ffffffff8185a9ba>] ? pcibios_init+0x65/0x78
[ 2.303000] [<ffffffff8185a09b>] ? pci_subsys_init+0x101/0x10a
[ 2.303000] [<ffffffff81856608>] ? md_init+0x0/0xcd
[ 2.303000] [<ffffffff81856608>] ? md_init+0x0/0xcd
[ 2.303000] [<ffffffff81859f9a>] ? pci_subsys_init+0x0/0x10a
[ 2.303000] [<ffffffff8100913d>] ? do_one_initcall+0x13c/0x147
[ 2.303000] [<ffffffff8182bc38>] ? kernel_init+0x141/0x18c
[ 2.303000] [<ffffffff8100ca3a>] ? child_rip+0xa/0x20
[ 2.303000] [<ffffffff8182baf7>] ? kernel_init+0x0/0x18c
[ 2.303000] [<ffffffff8100ca30>] ? child_rip+0x0/0x20

MensaWater · 09-13-2017, 09:01 AM

As I recall FusionIO required you to recompile when changing kernel versions. (i.e. The driver is not included in the stock kernel but one you add from Fusion [now owned by SanDisk]). I'm wondering if you compiled for later kernel and tried to apply to earlier system?

MegaRAID and MegaSAS are provided by the stock kernels and I've never seen any issues with them on LSI (now owned by Avago) cards in the 2.6.x kernels. Dell RAID cards from about PERC2 on are OEMed from LSI (Avago).