Kernel panic - not syncing: No init found. Try passing init= option to kernel.

I'm a newbie to embedded Linux and I'm to the point where I'm stumped at why this system will not boot on the AT91SAM9263EK development board. I'm using a board with a Rev A part so part of the boot environment is loaded in dataflash and the root file system is loaded in the NAND Flash.

bootstrap loader image - dataflash
u-boot environment - dataflash
u-boot image - dataflash
kernel image - dataflash
root file system (ext2) - NAND Flash (0x400000)

The file system mounts but then the messages state the system is unable to open an intial console and then the next message is a panic stating that no init was found.

I've read some posts about an init script but I'm unable to find that on my root file system. I mounted the file system file as a loop device on my linux development system. I see scripts in the /etc/init.d directory but no init script in the root directory. There is a symbolic link the root directory named linuxrc that points to /bin/busybox.

The console device in the kernel argument list is a device that exists in the root file system at /dev/ttyS0. I'm not sure if this is the correct device but it was the same device used in the linux demo for this board.

I've done some more digging since I posted this originally in Linux Newbie. I found the procedure init_post in init/main.c where the console open and the init exec occurs. The errors being returned from the exec calls for each of the init files is

/sbin/init - EIO - I/O error
/etc/init - EIO - I/O error
/bin/init - ENOENT - no such file or directory
/bin/sh - EIO - I/O error

As far as I know the only file that actually exists in the root file system in /sbin/init

It appears that all procedures called to mount the root file system are not returning errors. create_dev, mount_block_root, sys_mount, sys_chroot all completed without errors.

Here's the output of the boot process.

RomBOOT
>

U-Boot 1.1.5 (Jun 12 2008 - 16:22:11)

DRAM: 64 MB
NAND: NAND device: Manufacturer ID: 0xec, Chip ID: 0xda (Samsung NAND 256MiB 3,3V 8-bit)
NAND: Pagesize: 2048, Blocksize: 128K, OOBsize: 64
256 MiB
DataFlash:AT45DB642
Nb pages: 8192
Page Size: 1056
Size= 8650752 bytes
Logical address: 0xC0000000
Area 0: C0000000 to C0003FFF (RO)
Area 1: C0004000 to C0007FFF
Area 2: C0008000 to C0037FFF (RO)
Area 3: C0038000 to C083FFFF
In: serial
Out: serial
Err: serial
DM9161A PHY Detected

No link

MAC: error during RMII initialization
Hit any key to stop autoboot: 3  2  1  0
## Booting image at 22200000 ...
Image Name: Linux-2.6.24
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 1318548 Bytes = 1.3 MB
Load Address: 20008000
Entry Point: 20008000
Verifying Checksum ... OK
OK

Starting kernel ...

Uncompressing Linux...................................................................................... done, booting the kernel.
Linux version 2.6.24 (john@iguana4) (gcc version 4.2.1) #4 Mon Oct 6 18:04:09 PDT 2008
CPU: ARM926EJ-S [41069265] revision 5 (ARMv5TEJ), cr=00053177
Machine: Atmel AT91SAM9263-EK
Ignoring unrecognised tag 0x54410008
Memory policy: ECC disabled, Data cache writeback
Clocks: CPU 199 MHz, master 99 MHz, main 16.367 MHz
CPU0: D VIVT write-back cache
CPU0: I cache: 16384 bytes, associativity 4, 32 byte lines, 128 sets
CPU0: D cache: 16384 bytes, associativity 4, 32 byte lines, 128 sets
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 16256
Kernel command line: mem=64M console=ttyS0,115200 root=/dev/mtdblock1 rw rootfstype=ext2
AT91: 160 gpio irqs in 5 banks
PID hash table entries: 256 (order: 8, 1024 bytes)
Console: colour dummy device 80x30
console [ttyS0] enabled
Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
Memory: 64MB = 64MB total
Memory: 62116KB available (2404K code, 230K data, 116K init)
Mount-cache hash table entries: 512
CPU: Testing write buffer coherency: ok
net_namespace: 64 bytes
NET: Registered protocol family 16
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
NET: Registered protocol family 2
Time: pit clocksource has been installed.
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 2048 (order: 2, 16384 bytes)
TCP bind hash table entries: 2048 (order: 1, 8192 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
TCP reno registered
NetWinder Floating Point Emulator V0.97 (double precision)
JFFS2 version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
io scheduler noop registered
io scheduler anticipatory registered (default)
atmel_lcdfb atmel_lcdfb.0: backlight control is not available
atmel_lcdfb atmel_lcdfb.0: 150KiB frame buffer at 23d40000 (mapped at ffc00000)
atmel_lcdfb atmel_lcdfb.0: fb0: Atmel LCDC at 0x00700000 (mapped at c485c000), irq 26
atmel_usart.0: ttyS0 at MMIO 0xfeffee00 (irq = 1) is a ATMEL_SERIAL
atmel_usart.1: ttyS1 at MMIO 0xfff8c000 (irq = 7) is a ATMEL_SERIAL
RAMDISK driver initialized: 16 RAM disks of 8192K size 1024 blocksize
loop: module loaded
MACB_mii_bus: probed
eth0: Atmel MACB at 0xfffbc000 irq 21 (00:00:00:00:00:00)
eth0: attached PHY driver [Generic PHY] (mii_bushy_addr=ffffffff:00, irq=-1)
Driver 'sd' needs updating - please use bus_type methods
NAND device: Manufacturer ID: 0xec, Chip ID: 0xda (Samsung NAND 256MiB 3,3V 8-bit)
AT91 NAND: 8-bit, Software ECC
Scanning device for bad blocks
Bad eraseblock 90 at 0x00b40000
Creating 3 MTD partitions on "at91_nand":
0x00000000-0x00400000 : "Bootstrap"
0x00400000-0x04000000 : "Partition 1"
0x04000000-0x10000000 : "Partition 2"
atmel_spi atmel_spi.0: Atmel SPI Controller at 0xfffa4000 (irq 14)
mtd_dataflash spi0.0: AT45DB642x (8448 KBytes)
usbmon: debugfs is not available
at91_ohci at91_ohci: AT91 OHCI
at91_ohci at91_ohci: new USB bus registered, assigned bus number 1
at91_ohci at91_ohci: irq 29, io mem 0x00a00000
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
Initializing USB Mass Storage driver...
usbcore: registered new interface driver usb-storage
USB Mass Storage support registered.
udc: at91_udc version 3 May 2006
mice: PS/2 mouse device common for all mice
input: gpio-keys as /class/input/input0
ads7846 spi0.3: touchscreen, irq 31
input: ADS784x Touchscreen as /class/input/input1
rtc-at91sam9 at91_rtt.0: rtc core: registered at91_rtt as rtc0
rtc-at91sam9 at91_rtt.0: rtc0: SET TIME!
i2c /dev entries driver
i2c-gpio i2c-gpio: using pins 68 (SDA) and 69 (SCL)
at91sam9_wdt: invalid timeout (must be between 1 and 16)
TCP cubic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
rtc-at91sam9 at91_rtt.0: hctosys: unable to read the hardware clock
VFS: Mounted root (ext2 filesystem).
Freeing init memory: 116K
Warning: unable to open an initial console.
Kernel panic - not syncing: No init found. Try passing init= option to kernel.

Originally Posted by john_schimandle The console device in the kernel argument list is a device that exists in the root file system at /dev/ttyS0. I'm not sure if this is the correct device but it was the same device used in the linux demo for this board. Kernel command line: mem=64M console=ttyS0,115200 root=/dev/mtdblock1 rw rootfstype=ext2 console [ttyS0] enabled atmel_usart.0: ttyS0 at MMIO 0xfeffee00 (irq = 1) is a ATMEL_SERIAL atmel_usart.1: ttyS1 at MMIO 0xfff8c000 (irq = 7) is a ATMEL_SERIAL Warning: unable to open an initial console.

According to the source code ttyS0 is correct:

http://svn.dd-wrt.com:8000/dd-wrt/br...rd-sam9263ek.c


You could try /dev/ttyS0:

Kernel command line: mem=64M console=/dev/ttyS0,115200 root=/dev/mtdblock1 rw rootfstype=ext2

Another possibility is that you may have the speed incorrect:

Kernel command line: mem=64M console=/dev/ttyS0,9600 root=/dev/mtdblock1 rw rootfstype=ext2


Also I would try /dev/ttyS1 if /dev/ttyS0 doesn't work.

-----------------------
Steve Stites

Thanks for your post. I tried changing the console kernel parameter but that doesn't help. Changing it to /dev/ttyS0 didn't change anything and changing it to ttyS1 caused the messgages to go to the other serial port.

It seems to me the console= kernel boot argument is for the boot operation. At the very end of the boot process the code in init_post clearly tries to open /dev/console and does not succeed even though this file exists. Also, during the search for an init executable in init_post the system is unable to locate files and gets I/O errors. The file system can be mounted as a loop device on my development system and there are no errors seen there. All files and directories are visible.

It seems like there is some sort of corruption on the root file system after it is downloaded to the NAND Flash. The root file system header must be intact because the kernel can identify and mount the root file system. For some reason the kernel cannot access the root file system once it is mounted.

I've read back the NAND Flash and compared it to the contents of the file and there are no compare errors.

Found the problem:

Problem turned out to be a bad block on the NAND as exposed by this message in the boot sequence.

Scanning device for bad blocks
Bad eraseblock 90 at 0x00b40000

This spot also happens to be in the middle of the root file system area. SAM-BA also reported a problem with this area of the NAND when loading the file system into the NAND but did not generate an error. You have to inspect the log file for the message. This is a hidden error.

-I- Writing: 0x20000 bytes at 0xB40000 (buffer addr : 0x200047E4)
-I- 0x0 bytes written by applet

I spent some time tracking down the problem and for some reason the ext2 file system reports an I/O error (EIO) from the inode permission procedure in vfs_permissions. So the reason that /dev/console and /sbin/init could not open was the permission check failed. The code found the files on the root file system in the directory structure but could not open then due to permission issues. Somehow the bad block interfered with the permission check.

When I moved the root file system to partition 2 (mtdblock2) and then rebooted the system came up without a problem. The following boot messages show the partitions and their respective addresses in NAND. So I changed the SAM-BA load script to load the file system into partition 2 at 0x04000000 and then changed the bootargs argument string to contain root=/dev/mtdblock2.

Creating 3 MTD partitions on "at91_nand":
0x00000000-0x00400000 : "Bootstrap"
0x00400000-0x04000000 : "Partition 1"
0x04000000-0x10000000 : "Partition 2"

The interesting thing is the angstrom demo for the AT91SAM9263EK has the same problem with the bad block and the file system was placed in partition 1 but was able to boot. It uses the jffs2 file system instead of ext2. I had a reply on linuxquestions.org that jffs2 was a better choice of file system for NAND but I didn't know it handled this bad NAND block problem. I'll have to look into this further.

The at91sam9263ek_defconfig file has the file system set to ext2 so that is why I used it.