LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)
-   -   Anyone running Java on 2.6.* and got kernel crashes (https://www.linuxquestions.org/questions/linux-software-2/anyone-running-java-on-2-6-%2A-and-got-kernel-crashes-298278/)

aufomechanic 03-06-2005 09:13 AM

Anyone running Java on 2.6.* and got kernel crashes
 
The oddest bug I am overloading the system to about a average 5 over 15min. I keep on pressing the buttons on my Tomcat server and watchimg the process. Simply put the memory goes to about 44MRSS and 251M SWAP.(Yes thats allot but my understanding is java grabe 250 when it starts and tries to hang on to it). But here is the odd part as the system gets more underload. The RSS starts reducing from 44 to 0 then it crashes the kernel. I am running 1.4.2 java linux 2.6.10. I cant give you a dmesg as it dies out before I can get it. I have 512Mb RAM on a 2.8 P. Thats possible not important.

Before it crashes the kernel the systems starts a kind of backoff where it goes kill all the process. I know the detail in this is slim. But /var/log/* does not seem to give me anymore detail. I will try grab it the next time it crashes.

Thanks

aufomechanic 03-06-2005 10:36 AM

Here is the crash stage by stage

oom-killer: gfp_mask=0xd2
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1
cpu 0 cold: low 0, high 2, batch 1
Normal per-cpu:
cpu 0 hot: low 32, high 96, batch 16
cpu 0 cold: low 0, high 32, batch 16
HighMem per-cpu: empty

Free pages: 2848kB (0kB HighMem)
Active:94 inactive:419 dirty:0 writeback:0 unstable:0 free:712 slab:124877 mapped:58 pagetables:180
DMA free:88kB min:88kB low:108kB high:132kB active:0kB inactive:20kB present:16384kB pages_scanned:5 all_unreclaimable? yes
protections[]: 0 0 0
Normal free:2760kB min:2780kB low:3472kB high:4168kB active:376kB inactive:1656kB present:499712kB pages_scanned:770 all_unreclaimable? yes
protections[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
protections[]: 0 0 0
DMA: 0*4kB 1*8kB 1*16kB 0*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 88kB
Normal: 0*4kB 1*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2760kB
HighMem: empty
Swap cache: add 46598, delete 46456, find 6062/9298, race 0+1
Out of Memory: Killed process 816 (java).
oom-killer: gfp_mask=0xd2
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1
cpu 0 cold: low 0, high 2, batch 1
Normal per-cpu:
cpu 0 hot: low 32, high 96, batch 16
cpu 0 cold: low 0, high 32, batch 16
HighMem per-cpu: empty

Free pages: 2848kB (0kB HighMem)
Active:112 inactive:280 dirty:0 writeback:13 unstable:0 free:712 slab:125007 mapped:39 pagetables:180
DMA free:88kB min:88kB low:108kB high:132kB active:0kB inactive:20kB present:16384kB pages_scanned:1 all_unreclaimable? yes
protections[]: 0 0 0
Normal free:2696kB min:2780kB low:3472kB high:4168kB active:448kB inactive:1100kB present:499712kB pages_scanned:475 all_unreclaimable? yes
protections[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
protections[]: 0 0 0
DMA: 0*4kB 1*8kB 1*16kB 0*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 88kB
Normal: 0*4kB 1*8kB 0*16kB 0*32kB 0*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2696kB
HighMem: empty
Swap cache: add 47794, delete 47655, find 6172/9607, race 0+3
Out of Memory: Killed process 828 (java).
oom-killer: gfp_mask=0xd2
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1
cpu 0 cold: low 0, high 2, batch 1
Normal per-cpu:
cpu 0 hot: low 32, high 96, batch 16
cpu 0 cold: low 0, high 32, batch 16
HighMem per-cpu: empty

Free pages: 2848kB (0kB HighMem)
Active:88 inactive:175 dirty:0 writeback:10 unstable:0 free:712 slab:125151 mapped:14 pagetables:180
DMA free:88kB min:88kB low:108kB high:132kB active:0kB inactive:44kB present:16384kB pages_scanned:15 all_unreclaimable? yes
protections[]: 0 0 0
Normal free:2760kB min:2780kB low:3472kB high:4168kB active:352kB inactive:656kB present:499712kB pages_scanned:430 all_unreclaimable? yes
protections[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
protections[]: 0 0 0
DMA: 0*4kB 1*8kB 1*16kB 0*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 88kB
Normal: 0*4kB 1*8kB 0*16kB 0*32kB 0*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2696kB
HighMem: empty
Swap cache: add 49056, delete 48953, find 6236/9869, race 0+6
Out of Memory: Killed process 829 (java).
oom-killer: gfp_mask=0xd2
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1
cpu 0 cold: low 0, high 2, batch 1
Normal per-cpu:
cpu 0 hot: low 32, high 96, batch 16
cpu 0 cold: low 0, high 32, batch 16
HighMem per-cpu: empty

Free pages: 2720kB (0kB HighMem)
Active:7 inactive:167 dirty:0 writeback:3 unstable:0 free:680 slab:125254 mapped:11 pagetables:180
DMA free:88kB min:88kB low:108kB high:132kB active:4kB inactive:0kB present:16384kB pages_scanned:13 all_unreclaimable? yes
protections[]: 0 0 0
Normal free:2632kB min:2780kB low:3472kB high:4168kB active:24kB inactive:668kB present:499712kB pages_scanned:346 all_unreclaimable? yes
protections[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
protections[]: 0 0 0
DMA: 0*4kB 1*8kB 1*16kB 0*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 88kB
Normal: 0*4kB 1*8kB 2*16kB 1*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 2568kB
HighMem: empty
Swap cache: add 49974, delete 49948, find 6278/10072, race 0+14
Out of Memory: Killed process 830 (java).
oom-killer: gfp_mask=0x1d2
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1
cpu 0 cold: low 0, high 2, batch 1
Normal per-cpu:
cpu 0 hot: low 32, high 96, batch 16
cpu 0 cold: low 0, high 32, batch 16
HighMem per-cpu: empty

Free pages: 2144kB (0kB HighMem)
Active:31 inactive:2 dirty:0 writeback:0 unstable:0 free:536 slab:125466 mapped:0 pagetables:180
DMA free:88kB min:88kB low:108kB high:132kB active:4kB inactive:0kB present:16384kB pages_scanned:91 all_unreclaimable? yes
protections[]: 0 0 0
Normal free:1992kB min:2780kB low:3472kB high:4168kB active:120kB inactive:8kB present:499712kB pages_scanned:944 all_unreclaimable? yes
protections[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
protections[]: 0 0 0
DMA: 0*4kB 1*8kB 1*16kB 0*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 88kB
Normal: 12*4kB 19*8kB 10*16kB 7*32kB 0*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1992kB
HighMem: empty
Swap cache: add 50367, delete 50365, find 6316/10176, race 0+15
Out of Memory: Killed process 831 (java).
oom-killer: gfp_mask=0x1d2
DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1
cpu 0 cold: low 0, high 2, batch 1
Normal per-cpu:
cpu 0 hot: low 32, high 96, batch 16
cpu 0 cold: low 0, high 32, batch 16
HighMem per-cpu: empty

Free pages: 1288kB (0kB HighMem)
Active:22 inactive:71 dirty:0 writeback:0 unstable:0 free:322 slab:125635 mapped:6 pagetables:180
DMA free:64kB min:88kB low:108kB high:132kB active:4kB inactive:0kB present:16384kB pages_scanned:194 all_unreclaimable? yes
protections[]: 0 0 0
Normal free:1224kB min:2780kB low:3472kB high:4168kB active:84kB inactive:284kB present:499712kB pages_scanned:1500 all_unreclaimable? yes
protections[]: 0 0 0
HighMem free:0kB min:128kB low:160kB high:192kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
protections[]: 0 0 0
DMA: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 64kB
Normal: 0*4kB 23*8kB 11*16kB 5*32kB 1*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1224kB
HighMem: empty
Swap cache: add 50666, delete 50656, find 6352/10259, race 0+15
Out of Memory: Killed process 832 (java).
swapper: page allocation failure. order:0, mode:0x20
[<c0131891>] __alloc_pages+0x1c1/0x370
[<c0131a65>] __get_free_pages+0x25/0x40
[<c0134d52>] kmem_getpages+0x22/0xc0
[<c013595b>] cache_grow+0x9b/0x130
[<c0135aad>] cache_alloc_refill+0xbd/0x200
[<c0135e99>] __kmalloc+0x69/0x70
[<c0237807>] alloc_skb+0x47/0xf0
[<c010ff22>] activate_task+0x62/0x80
[<e00a24d4>] e100_poll+0x184/0x5c0 [e100]
[<c023e14a>] net_rx_action+0x6a/0xf0
[<c0117af5>] __do_softirq+0x85/0x90
[<c0117b27>] do_softirq+0x27/0x30
[<c0103f9b>] do_IRQ+0x3b/0x70
[<c0117af5>] __do_softirq+0x85/0x90
[<c0100450>] default_idle+0x0/0x30
[<c01025fa>] common_interrupt+0x1a/0x20
[<c0100450>] default_idle+0x0/0x30
[<c010c7ce>] apm_bios_call_simple+0x8e/0xc0
[<c0100450>] default_idle+0x0/0x30
[<c010c9e7>] apm_do_busy+0x27/0x40
[<c01004f0>] cpu_idle+0x30/0x40
[<c031c9a1>] start_kernel+0x161/0x1a0
[<c031c510>] unknown_bootoption+0x0/0x200

aufomechanic 03-06-2005 03:21 PM

As I have now taken to writing my own replies :). I think the problem is with DMA. Do not know how to go about proving that and I cannot give the exact dmesg line. In short a message says that DMA is being reset some error code then it is being turned off. If I can recreate It I will post. But in my logic. Java is after grabbing 250M Virtual memory when the DMA switchs off that makes swap pretty useless. But still why oh why would this crash the kernel. I have put this stuff through stability tests and it is rock hard. Its an intel 845GV chipset. Here the important part of my kernel conf.

Any Clever folks wanna throw me a bone!

#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.10
# Mon Mar 7 02:08:23 2005
#
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_UID16=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y

#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_CLEAN_COMPILE=y
CONFIG_BROKEN_ON_SMP=y

#
# General setup
#
CONFIG_LOCALVERSION=""
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
# CONFIG_POSIX_MQUEUE is not set
CONFIG_BSD_PROCESS_ACCT=y
# CONFIG_BSD_PROCESS_ACCT_V3 is not set
CONFIG_SYSCTL=y
# CONFIG_AUDIT is not set
CONFIG_LOG_BUF_SHIFT=16
# CONFIG_HOTPLUG is not set
CONFIG_KOBJECT_UEVENT=y
# CONFIG_IKCONFIG is not set
# CONFIG_EMBEDDED is not set
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_FUTEX=y
CONFIG_EPOLL=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_SHMEM=y
CONFIG_CC_ALIGN_FUNCTIONS=0
CONFIG_CC_ALIGN_LABELS=0
CONFIG_CC_ALIGN_LOOPS=0
CONFIG_CC_ALIGN_JUMPS=0
# CONFIG_TINY_SHMEM is not set

#
# Loadable module support
#
CONFIG_MODULES=y
# CONFIG_MODULE_UNLOAD is not set
CONFIG_OBSOLETE_MODPARM=y
CONFIG_MODVERSIONS=y
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_KMOD=y

#
# Processor type and features
#
CONFIG_X86_PC=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_NUMAQ is not set
# CONFIG_X86_SUMMIT is not set
# CONFIG_X86_BIGSMP is not set
# CONFIG_X86_VISWS is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_ES7000 is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
CONFIG_MPENTIUM4=y
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MEFFICEON is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_X86_GENERIC is not set
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_L1_CACHE_SHIFT=7
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
# CONFIG_HPET_TIMER is not set
# CONFIG_SMP is not set
# CONFIG_PREEMPT is not set
# CONFIG_X86_UP_APIC is not set
CONFIG_X86_TSC=y
CONFIG_X86_MCE=y
# CONFIG_X86_MCE_NONFATAL is not set
# CONFIG_TOSHIBA is not set
# CONFIG_I8K is not set
CONFIG_MICROCODE=m
CONFIG_X86_MSR=m
CONFIG_X86_CPUID=m

#
# Firmware Drivers
#
CONFIG_EDD=m
# CONFIG_NOHIGHMEM is not set
CONFIG_HIGHMEM4G=y
# CONFIG_HIGHMEM64G is not set
CONFIG_HIGHMEM=y
# CONFIG_HIGHPTE is not set
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
# CONFIG_REGPARM is not set

#
# Power management options (ACPI, APM)
#
CONFIG_PM=y
# CONFIG_PM_DEBUG is not set
# CONFIG_SOFTWARE_SUSPEND is not set

#
# ACPI (Advanced Configuration and Power Interface) Support
#
# CONFIG_ACPI is not set
CONFIG_ACPI_BLACKLIST_YEAR=0

#
# APM (Advanced Power Management) BIOS Support
#
CONFIG_APM=y
# CONFIG_APM_IGNORE_USER_SUSPEND is not set
# CONFIG_APM_DO_ENABLE is not set
CONFIG_APM_CPU_IDLE=y
# CONFIG_APM_DISPLAY_BLANK is not set
CONFIG_APM_RTC_IS_GMT=y
# CONFIG_APM_ALLOW_INTS is not set
# CONFIG_APM_REAL_MODE_POWER_OFF is not set

#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set

#
# Bus options (PCI, PCMCIA, EISA, MCA, ISA)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
# CONFIG_PCI_LEGACY_PROC is not set
CONFIG_PCI_NAMES=y
CONFIG_ISA=y
CONFIG_EISA=y
# CONFIG_EISA_VLB_PRIMING is not set
CONFIG_EISA_PCI_EISA=y
CONFIG_EISA_VIRTUAL_ROOT=y
CONFIG_EISA_NAMES=y
# CONFIG_MCA is not set
# CONFIG_SCx200 is not set

#
# Executable file formats
#
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_AOUT=m
CONFIG_BINFMT_MISC=m

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
# CONFIG_DEBUG_DRIVER is not set

#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set

#
# Parallel port support
#
CONFIG_PARPORT=m
CONFIG_PARPORT_PC=m
CONFIG_PARPORT_PC_CML1=m
CONFIG_PARPORT_SERIAL=m
# CONFIG_PARPORT_PC_FIFO is not set
# CONFIG_PARPORT_PC_SUPERIO is not set
# CONFIG_PARPORT_OTHER is not set
CONFIG_PARPORT_1284=y

#
# Plug and Play support
#
# CONFIG_PNP is not set

#
# Block devices
#
# CONFIG_BLK_DEV_FD is not set
# CONFIG_BLK_DEV_XD is not set
# CONFIG_PARIDE is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
CONFIG_BLK_DEV_LOOP=m
# CONFIG_BLK_DEV_CRYPTOLOOP is not set
CONFIG_BLK_DEV_NBD=m
# CONFIG_BLK_DEV_SX8 is not set
# CONFIG_BLK_DEV_UB is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=4096
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
CONFIG_LBD=y
# CONFIG_CDROM_PKTCDVD is not set

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y


All times are GMT -5. The time now is 05:39 PM.