SAN disk problem

ronanio · 01-25-2006, 03:58 AM

I'm running slamd64-current(slackware for 64bit) current as of yesterday! on a dual opteron 2G hardwareraid SCSI powered sun box.

It has 2 qloggic cards in it

kernel 2.6.15.1 w/ qla2300 in as a module.

There is only 1 fibre running from the first card to the SAN. I'm just trying to get it working before I plug the other in. There is only 1 disk presented.

So it boots up fine and when i modprobe the driver (after i present a 5 gig disk to it)
then following appears in the ring buffer...

kobject_register failed for scsi_transport_fc (-17)

Call Trace:<ffffffff80233636>{kobject_register+70} <ffffffff8014f444>{load_module+5492}
<ffffffff80157959>{__generic_file_aio_read+425} <ffffffff80157b12>{generic_file_read+178}
<ffffffff80148ff0>{autoremove_wake_function+0} <ffffffff801751a5>{vfs_read+197}
<ffffffff8014f7f7>{sys_init_module+119} <ffffffff8010dc26>{system_call+126}

firmware_class_init: class_register failed
QLogic Fibre Channel HBA Driver
GSI 19 sharing vector 0xC1 and IRQ 19
ACPI: PCI Interrupt 0000:02:05.0[A] -> GSI 24 (level, low) -> IRQ 193
qla2300 0000:02:05.0: Found an ISP2312, irq 193, iobase 0xffffc2000001c000
qla2300 0000:02:05.0: Configuring PCI space...
qla2300 0000:02:05.0: Configure NVRAM parameters...
qla2300 0000:02:05.0: Verifying loaded RISC code...
qla2300 0000:02:05.0: LIP reset occured (f7f7).
qla2300 0000:02:05.0: Waiting for LIP to complete...
qla2300 0000:02:05.0: LOOP UP detected (2 Gbps).
qla2300 0000:02:05.0: Topology - (F_Port), Host Loop address 0xffff
scsi1 : qla2xxx
qla2300 0000:02:05.0:
QLogic Fibre Channel HBA Driver: 8.01.03-k
QLogic QLA2340 -
ISP2312: PCI-X (66 MHz) @ 0000:02:05.0 hdma-, host#=1, fw=3.03.18 IPX
GSI 20 sharing vector 0xC9 and IRQ 20
ACPI: PCI Interrupt 0000:03:01.0[A] -> GSI 28 (level, low) -> IRQ 201
qla2300 0000:03:01.0: Found an ISP2312, irq 201, iobase 0xffffc2000001e000
qla2300 0000:03:01.0: Configuring PCI space...
qla2300 0000:03:01.0: Configure NVRAM parameters...
scsi: unknown device type 12
Vendor: COMPAQ Model: HSV110 (C)COMPAQ Rev: 3025
Type: RAID ANSI SCSI revision: 02
1:0:0:0: Attached scsi generic sg1 type 12
Vendor: COMPAQ Model: HSV110 (C)COMPAQ Rev: 3025
Type: Direct-Access ANSI SCSI revision: 02
SCSI device sdb: 10485760 512-byte hdwr sectors (5369 MB)
sdb: asking for cache data failed
sdb: assuming drive cache: write through
SCSI device sdb: 10485760 512-byte hdwr sectors (5369 MB)
sdb: asking for cache data failed
sdb: assuming drive cache: write through
sdb:<6>sd 1:0:0:1: Device not ready.
end_request: I/O error, dev sdb, sector 0
Buffer I/O error on device sdb, logical block 0
sd 1:0:0:1: Device not ready.
end_request: I/O error, dev sdb, sector 0
Buffer I/O error on device sdb, logical block 0
unable to read partition table
sd 1:0:0:1: Attached scsi disk sdb
sd 1:0:0:1: Attached scsi generic sg2 type 0
scsi: unknown device type 12
Vendor: COMPAQ Model: HSV110 (C)COMPAQ Rev: 3025
Type: RAID ANSI SCSI revision: 02
1:0:1:0: Attached scsi generic sg3 type 12
Vendor: COMPAQ Model: HSV110 (C)COMPAQ Rev: 3025
Type: Direct-Access ANSI SCSI revision: 02
SCSI device sdc: 10485760 512-byte hdwr sectors (5369 MB)
SCSI device sdc: drive cache: write through
SCSI device sdc: 10485760 512-byte hdwr sectors (5369 MB)
SCSI device sdc: drive cache: write through
sdc: unknown partition table
sd 1:0:1:1: Attached scsi disk sdc
sd 1:0:1:1: Attached scsi generic sg4 type 0
scsi: unknown device type 12
Vendor: COMPAQ Model: HSV110 (C)COMPAQ Rev: 3025
Type: RAID ANSI SCSI revision: 02
1:0:2:0: Attached scsi generic sg5 type 12
scsi: unknown device type 12
Vendor: COMPAQ Model: HSV110 (C)COMPAQ Rev: 3025
Type: RAID ANSI SCSI revision: 02
1:0:3:0: Attached scsi generic sg6 type 12
qla2300 0000:03:01.0: Verifying loaded RISC code...
qla2300 0000:03:01.0: Waiting for LIP to complete...
qla2300 0000:03:01.0: Cable is unplugged...
scsi2 : qla2xxx
qla2300 0000:03:01.0:
QLogic Fibre Channel HBA Driver: 8.01.03-k
QLogic QLA2340 -
ISP2312: PCI-X (133 MHz) @ 0000:03:01.0 hdma-, host#=2, fw=3.03.18 IPX

lsscsi produces

root@scarab:/home/ronan/lsscsi-0.16# lsscsi
[0:0:0:0] disk LSILOGIC 1030 IM IM 1000 /dev/sda
[1:0:0:0] storage COMPAQ HSV110 (C)COMPAQ 3025 -
[1:0:0:1] disk COMPAQ HSV110 (C)COMPAQ 3025 /dev/sdb
[1:0:1:0] storage COMPAQ HSV110 (C)COMPAQ 3025 -
[1:0:1:1] disk COMPAQ HSV110 (C)COMPAQ 3025 /dev/sdc
[1:0:2:0] storage COMPAQ HSV110 (C)COMPAQ 3025 -
[1:0:3:0] storage COMPAQ HSV110 (C)COMPAQ 3025 -

fdisk /dev/sdb

Unable to read /dev/sdb

and this entry in the ring buffer

sd 1:0:0:1: Device not ready.
end_request: I/O error, dev sdb, sector 0
Buffer I/O error on device sdb, logical block 0
Buffer I/O error on device sdb, logical block 1
Buffer I/O error on device sdb, logical block 2
Buffer I/O error on device sdb, logical block 3
Buffer I/O error on device sdb, logical block 4
Buffer I/O error on device sdb, logical block 5
Buffer I/O error on device sdb, logical block 6
Buffer I/O error on device sdb, logical block 7
sd 1:0:0:1: Device not ready.
end_request: I/O error, dev sdb, sector 0
Buffer I/O error on device sdb, logical block 0
sd 1:0:0:1: Device not ready.
end_request: I/O error, dev sdb, sector 0
sd 1:0:0:1: Device not ready.
end_request: I/O error, dev sdb, sector 0
Buffer I/O error on device sdb, logical block 0
Buffer I/O error on device sdb, logical block 1
Buffer I/O error on device sdb, logical block 2
Buffer I/O error on device sdb, logical block 3
Buffer I/O error on device sdb, logical block 4
Buffer I/O error on device sdb, logical block 5
sd 1:0:0:1: Device not ready.
end_request: I/O error, dev sdb, sector 0
sd 1:0:0:1: Device not ready.
end_request: I/O error, dev sdb, sector 2883808
sd 1:0:0:1: Device not ready.
end_request: I/O error, dev sdb, sector 3933120
sd 1:0:0:1: Device not ready.
end_request: I/O error, dev sdb, sector 4722648
...

any ideas as to how i can get the drives working?

Ronan

**EDIT**
seems im half asleep ths morning

the /dev/sdc drive is the actual drive and works fine...
sdb must be the failover path...

Ronan

chen23547 · 03-02-2006, 07:43 AM

I have the similar problem, were there any one can deal this?

ronanio · 03-02-2006, 07:49 AM

yes i resolved it, what is your specific problem though?
Ronan

bret · 03-02-2006, 03:49 PM

Hi Ronanio,

I recently installed dual QLogic 2340 Fibre channel cards in my Dell 1850 Poweredge server. I am running SuSE 9 64-bit, Service Pack 2 (Linux 2.6 Kernel). My backend SAN is a Hitachi 9570. Everything worked smoothly for me. I got the most recent Linux installer off their website. The QLogic linux driver release is 8.01.00-4. After I downloaded the driver, there was an install script that I ran. I then rebooted the server, and was ready for LVM. Things went real smooth for me.

Regards,

Bret Goodfellow
SLC, UT

chen23547 · 03-03-2006, 03:34 AM

Quote:

Originally Posted by ronanio

yes i resolved it, what is your specific problem though?
Ronan

thanx for your reply

the detail is: I have a HP DL585 server that pluged in a Qlogic HBA card. And it is connected to a disk array. When I finished installing CentOS 4.2 in and boot it, several error messages like "buffer i/o error on device sdb (address)xxxxxxxx" came into screen just like the messages you post. I tried to search the resolution in google, but found nothing that it useful. How did you resolve this problem?

chen23547

ronanio · 03-03-2006, 03:58 AM

chen, when i got those error messages I deduced that the module for the qla drivers were by default loadin in failover mode. So the machine will always see 2 disks presented to it.. (only one read/writeable though ). EG i presented one disk from the SAN and an lsscsi (<- get this package.. very useful) shows me sdb AND sdc i get the buffer i/o error on one device, but the other device can be formatted and partitioned as a normal scsi attached disk.

have a look at the module parameters and Try load ing the module without failover enabled and see if that removes the (extra) disk error. I have yet to try this because for my purpouses the san disk in question is working so im happy.

my current setup:
i have only 2 disks presented from the san but because of failover and the fact that i have two cards in the machine going to two seperate switches... i see the following 8 drives. (sda is my local scsi disk)
lsscsi
[0:0:0:0] disk LSILOGIC 1030 IM IM 1000 /dev/sda
[1:0:0:0] storage COMPAQ HSV110 (C)COMPAQ 3025 -
[1:0:0:1] disk COMPAQ HSV110 (C)COMPAQ 3025 /dev/sdb
[1:0:1:0] storage COMPAQ HSV110 (C)COMPAQ 3025 -
[1:0:1:1] disk COMPAQ HSV110 (C)COMPAQ 3025 /dev/sdc
[1:0:2:0] storage COMPAQ HSV110 (C)COMPAQ 3025 -
[1:0:2:1] disk COMPAQ HSV110 (C)COMPAQ 3025 /dev/sdd
[1:0:3:0] storage COMPAQ HSV110 (C)COMPAQ 3025 -
[1:0:3:1] disk COMPAQ HSV110 (C)COMPAQ 3025 /dev/sde
[2:0:0:0] storage COMPAQ HSV110 (C)COMPAQ 3025 -
[2:0:0:1] disk COMPAQ HSV110 (C)COMPAQ 3025 /dev/sdf
[2:0:1:0] storage COMPAQ HSV110 (C)COMPAQ 3025 -
[2:0:1:1] disk COMPAQ HSV110 (C)COMPAQ 3025 /dev/sdg
[2:0:2:0] storage COMPAQ HSV110 (C)COMPAQ 3025 -
[2:0:2:1] disk COMPAQ HSV110 (C)COMPAQ 3025 /dev/sdh
[2:0:3:0] storage COMPAQ HSV110 (C)COMPAQ 3025 -
[2:0:3:1] disk COMPAQ HSV110 (C)COMPAQ 3025 /dev/sdi

HTH

Ronan

chen23547 · 03-05-2006, 01:39 AM

hello, ronan. Below is my dmesg info. Could you check out some infomation from it?

I haven't check my kernel config yet, I will do it tommorow. Maybe the incorrect kernel config cause these faults.

Best regards
chen23547

Code:

Bootdata ok (command line is ro root=/dev/VolGroup00/LogVol00 rhgb quiet)
Linux version 2.6.9-22.0.2.ELsmp (buildcentos@x8664-build.centos.org) (gcc version 3.4.4 20050721 (Red Hat 3.4.4-2)) #1 SMP Tue Jan 17 06:12:06 CST 2006
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009f400 (usable)
 BIOS-e820: 000000000009f400 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000f57f6800 (usable)
 BIOS-e820: 00000000f57f6800 - 00000000f5800000 (ACPI data)
 BIOS-e820: 00000000fdc00000 - 00000000fdc01000 (reserved)
 BIOS-e820: 00000000fdc10000 - 00000000fdc11000 (reserved)
 BIOS-e820: 00000000fdc20000 - 00000000fdc21000 (reserved)
 BIOS-e820: 00000000fdc30000 - 00000000fdc31000 (reserved)
 BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
 BIOS-e820: 00000000fec10000 - 00000000fec11000 (reserved)
 BIOS-e820: 00000000fec20000 - 00000000fec21000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved)
 BIOS-e820: 00000000ff800000 - 0000000100000000 (reserved)
ACPI: RSDP (v002 HP                                    ) @ 0x00000000000f4f20
ACPI: XSDT (v001 HP     A01      0x00000002 ? 0x0000162e) @ 0x00000000f57f6c00
ACPI: FADT (v003 HP     A01      0x00000002 ? 0x0000162e) @ 0x00000000f57f6c80
ACPI: MADT (v001 HP     00000083 0x00000002  0x00000000) @ 0x00000000f57f6900
ACPI: SPCR (v001 HP     SPCRRBSU 0x00000001 ? 0x0000162e) @ 0x00000000f57f6a00
ACPI: SRAT (v001 HP     A01      0x00000001  0x00000000) @ 0x00000000f57f6a80
ACPI: DSDT (v001 HP         DSDT 0x00000001 MSFT 0x02000001) @ 0x0000000000000000
Scanning NUMA topology in Northbridge 24
Number of nodes 2 (10010)
Node 0 MemBase 0000000000000000 Limit 000000007fffffff
Node 1 MemBase 0000000080000000 Limit 00000000f57f6000
Using 24 for the hash shift. Max addr is f57f6000 
Using node hash shift of 24
Bootmem setup node 0 0000000000000000-000000007fffffff
Bootmem setup node 1 0000000080000000-00000000f57f6000
No mptable found.
On node 0 totalpages: 524287
  DMA zone: 4096 pages, LIFO batch:1
  Normal zone: 520191 pages, LIFO batch:16
  HighMem zone: 0 pages, LIFO batch:1
On node 1 totalpages: 481270
  DMA zone: 0 pages, LIFO batch:1
  Normal zone: 481270 pages, LIFO batch:16
  HighMem zone: 0 pages, LIFO batch:1
DMI 2.3 present.
ACPI: PM-Timer IO Port: 0x908
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
Processor #1 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] disabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] disabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] disabled)
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] disabled)
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] disabled)
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] disabled)
ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
Setting APIC routing to flat
ACPI: IOAPIC (id[0x04] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 4, version 17, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x05] address[0xfec10000] gsi_base[24])
IOAPIC[1]: apic_id 5, version 17, address 0xfec10000, GSI 24-27
ACPI: IOAPIC (id[0x06] address[0xfec20000] gsi_base[28])
IOAPIC[2]: apic_id 6, version 17, address 0xfec20000, GSI 28-31
ACPI: IOAPIC (id[0x07] address[0xfdc00000] gsi_base[32])
IOAPIC[3]: apic_id 7, version 17, address 0xfdc00000, GSI 32-35
ACPI: IOAPIC (id[0x08] address[0xfdc10000] gsi_base[36])
IOAPIC[4]: apic_id 8, version 17, address 0xfdc10000, GSI 36-39
ACPI: IOAPIC (id[0x09] address[0xfdc20000] gsi_base[40])
IOAPIC[5]: apic_id 9, version 17, address 0xfdc20000, GSI 40-43
ACPI: IOAPIC (id[0x0a] address[0xfdc30000] gsi_base[44])
IOAPIC[6]: apic_id 10, version 17, address 0xfdc30000, GSI 44-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
Checking aperture...
CPU 0: aperture @ 390000000 size 32 MB
Aperture from northbridge cpu 0 too small (32 MB)
No AGP bridge found
Built 2 zonelists
Kernel command line: ro root=/dev/VolGroup00/LogVol00 rhgb quiet console=tty0
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 131072 bytes)
time.c: Using 3.579545 MHz PM timer.
time.c: Detected 2197.207 MHz processor.
Console: colour VGA+ 80x25
Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
Memory: 3953436k/4022232k available (2077k kernel code, 0k reserved, 1276k data, 188k init)
Calibrating delay loop... 4308.99 BogoMIPS (lpj=2154496)
Security Scaffold v1.0.0 initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
There is already a security framework initialized, register_security failed.
selinux_register_security:  Registering secondary module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 256 (order: 0, 4096 bytes)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 0(1) -> Node 0
Using local APIC NMI watchdog using perfctr0
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 0(1) -> Node 0
CPU0: AMD Opteron (tm) Processor 848 stepping 0a
per-CPU timeslice cutoff: 1023.75 usecs.
task migration cache decay timeout: 2 msecs.
Booting processor 1/1 rip 6000 rsp 10037e03f58
Initializing CPU#1
Calibrating delay loop... 4390.91 BogoMIPS (lpj=2195456)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 1(1) -> Node 1
AMD Opteron (tm) Processor 848 stepping 0a
Total of 2 processors activated (8699.90 BogoMIPS).
Using local APIC timer interrupts.
Detected 12.484 MHz APIC timer.
checking TSC synchronization across 2 CPUs: passed.
Brought up 2 CPUs
Disabling vsyscall due to use of PM timer
time.c: Using PM based timekeeping.
checking if image is initramfs... it is
NET: Registered protocol family 16
PCI: Using configuration type 1
mtrr: v2.0 (20020519)
mtrr: your CPUs had inconsistent fixed MTRR settings
mtrr: probably your BIOS does not setup all CPUs.
mtrr: corrected configuration.
ACPI: Subsystem revision 20040816
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [CFG0] (00:00)
PCI: Probing PCI hardware (bus 00)
ACPI: PCI Interrupt Routing Table [\_SB_.CFG0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.CFG0.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.CFG0.PCI1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.CFG0.PCI2._PRT]
ACPI: PCI Root Bridge [CFG1] (00:04)
PCI: Probing PCI hardware (bus 04)
ACPI: PCI Interrupt Routing Table [\_SB_.CFG1.PCI3._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.CFG1.PCI4._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.CFG1.PCI5._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.CFG1.PCI6._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 5 *7 10 11)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 5 7 *10 11)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 5 7 10 *11)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 *5 7 10 11)
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Using ACPI for IRQ routing
ACPI: PCI interrupt 0000:01:00.0[D] -> GSI 19 (level, low) -> IRQ 169
ACPI: PCI interrupt 0000:01:00.1[D] -> GSI 19 (level, low) -> IRQ 169
ACPI: PCI interrupt 0000:01:02.0[A] -> GSI 16 (level, low) -> IRQ 177
ACPI: PCI interrupt 0000:01:02.2[B] -> GSI 17 (level, low) -> IRQ 185
ACPI: PCI interrupt 0000:02:04.0[A] -> GSI 18 (level, low) -> IRQ 193
ACPI: PCI interrupt 0000:02:06.0[A] -> GSI 25 (level, low) -> IRQ 201
ACPI: PCI interrupt 0000:02:06.1[B] -> GSI 24 (level, low) -> IRQ 209
ACPI: PCI interrupt 0000:08:0c.0[A] -> GSI 46 (level, low) -> IRQ 217
PCI-DMA: Disabling IOMMU.
IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak Exp $
audit: initializing netlink socket (disabled)
audit(1141325868.631:1): initialized
Total HugeTLB memory allocated, 0
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
SELinux:  Registering netfilter hooks
Initializing Cryptographic API
ksign: Installing public key data
Loading keyring
- Added public key 656B17C76C9B4227
- User ID: CentOS (Kernel Module GPG key)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
ACPI: Processor [CPU0] (supports C1)
ACPI: Processor [CPU1] (supports C1)
Real Time Clock Driver v1.12
Linux agpgart interface v0.100 (c) Dave Jones
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
Serial: 8250/16550 driver $Revision: 1.90 $ 8 ports, IRQ sharing enabled
ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize
divert: not allocating divert_blk for non-ethernet device lo
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
AMD8111: IDE controller at PCI slot 0000:00:04.1
AMD8111: chipset revision 3
AMD8111: not 100% native mode: will probe irqs later
AMD8111: 0000:00:04.1 (rev 03) UDMA133 controller
    ide0: BM-DMA at 0x2000-0x2007, BIOS settings: hda:pio, hdb:pio
Probing IDE interface ide0...
hda: HL-DT-ST GCR-8240N, ATAPI CD/DVD-ROM drive
Using cfq io scheduler
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Probing IDE interface ide1...
Probing IDE interface ide2...
Probing IDE interface ide3...
Probing IDE interface ide4...
Probing IDE interface ide5...
hda: ATAPI 24X CD-ROM drive, 128kB Cache, DMA
Uniform CD-ROM driver Revision: 3.20
ide-floppy driver 0.99.newide
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.0:USB HID core driver
mice: PS/2 mouse device common for all mice
input: AT Translated Set 2 keyboard on isa0060/serio0
input: ImPS/2 Generic Wheel Mouse on isa0060/serio1
md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
NET: Registered protocol family 2
IP: routing cache hash table of 16384 buckets, 256Kbytes
TCP: Hash tables configured (established 262144 bind 65536)
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
powernow-k8: Found 2 AMD Athlon 64 / Opteron processors (version 1.39.04)
powernow-k8:    0 : fid 0xe (2200 MHz), vid 0x2 (1500 mV)
powernow-k8:    1 : fid 0xc (2000 MHz), vid 0x6 (1400 mV)
powernow-k8:    2 : fid 0xa (1800 MHz), vid 0xa (1300 mV)
powernow-k8: cpu_init done, current fid 0xe, vid 0x2
powernow-k8:    0 : fid 0xe (2200 MHz), vid 0x2 (1500 mV)
powernow-k8:    1 : fid 0xc (2000 MHz), vid 0x6 (1400 mV)
powernow-k8:    2 : fid 0xa (1800 MHz), vid 0xa (1300 mV)
powernow-k8: cpu_init done, current fid 0xe, vid 0x2
ACPI: (supports S0 S4 S5)
ACPI wakeup devices: 

Freeing unused kernel memory: 188k freed
SCSI subsystem initialized
HP CISS Driver (v 2.6.8)
cciss: Device 0xb178 has been found at bus 2 dev 4 func 0
ACPI: PCI interrupt 0000:02:04.0[A] -> GSI 18 (level, low) -> IRQ 193
cciss: using DAC cycles
      blocks= 860216490 block_size= 512
      heads= 255, sectors= 63, cylinders= 53546

 cciss/c0d0: p1 p2
QLogic Fibre Channel HBA Driver
ACPI: PCI interrupt 0000:08:0c.0[A] -> GSI 46 (level, low) -> IRQ 217
qla2300 0000:08:0c.0: Found an ISP2312, irq 217, iobase 0xffffff0000020000
qla2300 0000:08:0c.0: Configuring PCI space...
qla2300 0000:08:0c.0: Configure NVRAM parameters...
qla2300 0000:08:0c.0: Verifying loaded RISC code...
qla2300 0000:08:0c.0: LIP reset occured (f7f7).
qla2300 0000:08:0c.0: Waiting for LIP to complete...
qla2300 0000:08:0c.0: LOOP UP detected (2 Gbps).
qla2300 0000:08:0c.0: Topology - (F_Port), Host Loop address 0xffff
scsi0 : qla2xxx
qla2300 0000:08:0c.0: 
 QLogic Fibre Channel HBA Driver: 8.01.00b5-rh2
  QLogic QLA2340 - 133MHz PCI-X to 2Gb FC, Single Channel
  ISP2312: PCI-X (100 MHz) @ 0000:08:0c.0 hdma+, host#=0, fw=3.03.15 IPX
  Vendor: DFT   ES  Model: 1600              Rev: 7903
  Type:   Direct-Access                      ANSI SCSI revision: 02
qla2300 0000:08:0c.0: scsi(0:0:0:0): Enabled tagged queuing, queue depth 32.
SCSI device sda: 2906234880 512-byte hdwr sectors (1487992 MB)
SCSI device sda: drive cache: write back
 sda: sda1
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
  Vendor: DFT   ES  Model: 1600     PSEUDO   Rev: 7903
  Type:   Direct-Access                      ANSI SCSI revision: 02
qla2300 0000:08:0c.0: scsi(0:0:1:0): Enabled tagged queuing, queue depth 32.
sdb: Unit Not Ready, sense:
Current : sense key Illegal Request
Additional sense: Logical unit not supported
sdb : READ CAPACITY failed.
sdb : status=1, message=00, host=0, driver=08 
Current sd: sense key Illegal Request
Additional sense: Logical unit not supported
sdb: asking for cache data failed
sdb: assuming drive cache: write through
 sdb:end_request: I/O error, dev sdb, sector 0
Buffer I/O error on device sdb, logical block 0
end_request: I/O error, dev sdb, sector 0
Buffer I/O error on device sdb, logical block 0
end_request: I/O error, dev sdb, sector 0
Buffer I/O error on device sdb, logical block 0
 unable to read partition table
Attached scsi disk sdb at scsi0, channel 0, id 1, lun 0
device-mapper: 4.4.0-ioctl (2005-01-12) initialised: dm-devel@redhat.com
cdrom: open failed.
end_request: I/O error, dev sdb, sector 0
Buffer I/O error on device sdb, logical block 0
Buffer I/O error on device sdb, logical block 1
Buffer I/O error on device sdb, logical block 2
Buffer I/O error on device sdb, logical block 3
Buffer I/O error on device sdb, logical block 4
Buffer I/O error on device sdb, logical block 5
Buffer I/O error on device sdb, logical block 6
Buffer I/O error on device sdb, logical block 7
Buffer I/O error on device sdb, logical block 8
Buffer I/O error on device sdb, logical block 9
Buffer I/O error on device sdb, logical block 10
Buffer I/O error on device sdb, logical block 11
Buffer I/O error on device sdb, logical block 12
Buffer I/O error on device sdb, logical block 13
Buffer I/O error on device sdb, logical block 14
Buffer I/O error on device sdb, logical block 15
end_request: I/O error, dev sdb, sector 0
Buffer I/O error on device sdb, logical block 0
end_request: I/O error, dev sdb, sector 2097024
end_request: I/O error, dev sdb, sector 0
end_request: I/O error, dev sdb, sector 0
kjournald starting.  Commit interval 5 seconds

kuman2k · 01-13-2007, 06:50 AM

hi. newbie questions here, appreciate everyone's helps.
a) how do i know/what command to issue to find out whether qla2300 failover mode is loadin/not?
b) with failover mode load in, 1 card dual port installed and 1 disk presented in SAN then lsscsi should list 4 drives, shouldnt it?
c) for the case failover loaded in, 1 card 1 port and 1 SAN disk, we should get 2 drives. say /dev/sdb and /dev/sdc. if we fdisk /dev/sdc and set the partition, then does that setting reflected to /dev/sdb?
d) additional to (c), which is the right way to be choosen for fdisk - /dev/sdb or /dev/sdc?
tqvm.
-kuman2k-

xjlittle · 01-14-2007, 11:07 AM

khuman2k-you need to start a new thread for your question and not hijack this one

chen23547-did you register the machine with the array? Did you create a storage group and add the host to it along with the lun?

You didn't give many details about the SAN that you are using but when I see those errors on ours that is generally what is going on.

in.rishi · 02-15-2007, 03:34 AM

I installed RHEL AS 4.0 and when I did dmesg I saw buffer I/O errors on sd* devices.

I am using qlogic card.

lspci | grep QLogic
06:01.0 Fibre Channel: QLogic Corp. QLA2312 Fibre Channel Adapter (rev 02)
06:02.0 Fibre Channel: QLogic Corp. QLA2312 Fibre Channel Adapter (rev 02)

dmesg
Buffer I/0 errors.

CSI device sda: 314572800 512-byte hdwr sectors (161061 MB)
sda: asking for cache data failed
sda: assuming drive cache: write through
SCSI device sda: 314572800 512-byte hdwr sectors (161061 MB)
sda: asking for cache data failed
sda: assuming drive cache: write through
sda:<6>Device sda not ready.
end_request: I/O error, dev sda, sector 0
Buffer I/O error on device sda, logical block 0
Device sda not ready.
end_request: I/O error, dev sda, sector 0
Buffer I/O error on device sda, logical block 0
Device sda not ready.
end_request: I/O error, dev sda, sector 0
Buffer I/O error on device sda, logical block 0
unable to read partition table
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
Vendor: DGC Model: RAID 5 Rev: 0219
Type: Direct-Access ANSI SCSI revision: 04

==================================================================================================== ================

SCSI device sdc: 272629760 512-byte hdwr sectors (139586 MB)
sdc: asking for cache data failed
sdc: assuming drive cache: write through
SCSI device sdc: 272629760 512-byte hdwr sectors (139586 MB)
sdc: asking for cache data failed
sdc: assuming drive cache: write through
sdc:<6>Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
unable to read partition table
Attached scsi disk sdc at scsi0, channel 0, id 0, lun 2
Vendor: DGC Model: RAID 5 Rev: 0219
Type: Direct-Access ANSI SCSI revision: 04

==================================================================================================== ==========
sde: asking for cache data failed
sde: assuming drive cache: write through
SCSI device sde: 157286400 512-byte hdwr sectors (80531 MB)
sde: asking for cache data failed
sde: assuming drive cache: write through
sde:<6>Device sde not ready.
end_request: I/O error, dev sde, sector 0
Buffer I/O error on device sde, logical block 0
Device sde not ready.
end_request: I/O error, dev sde, sector 0
Buffer I/O error on device sde, logical block 0
Device sde not ready.
end_request: I/O error, dev sde, sector 0
Buffer I/O error on device sde, logical block 0
unable to read partition table
Attached scsi disk sde at scsi0, channel 0, id 1, lun 1
Vendor: DGC Model: RAID 5 Rev: 0219
Type: Direct-Access ANSI SCSI revision: 04

==================================================================================================== =============
cdrom: open failed.
================================================================
Device sda not ready.
end_request: I/O error, dev sda, sector 0
Buffer I/O error on device sda, logical block 0
Buffer I/O error on device sda, logical block 1
Buffer I/O error on device sda, logical block 2
Buffer I/O error on device sda, logical block 3
Buffer I/O error on device sda, logical block 4
Buffer I/O error on device sda, logical block 5
Buffer I/O error on device sda, logical block 6
Buffer I/O error on device sda, logical block 7
Buffer I/O error on device sda, logical block 8
Buffer I/O error on device sda, logical block 9
Buffer I/O error on device sda, logical block 10
Buffer I/O error on device sda, logical block 11
Buffer I/O error on device sda, logical block 12
Buffer I/O error on device sda, logical block 13
Buffer I/O error on device sda, logical block 14
Buffer I/O error on device sda, logical block 15
Device sda not ready.
end_request: I/O error, dev sda, sector 0
Buffer I/O error on device sda, logical block 0
Device sda not ready.
end_request: I/O error, dev sda, sector 314572672
Device sda not ready.
end_request: I/O error, dev sda, sector 0
==================================================================================================== =====

Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Buffer I/O error on device sdc, logical block 1
Buffer I/O error on device sdc, logical block 2
Buffer I/O error on device sdc, logical block 3
Buffer I/O error on device sdc, logical block 4
Buffer I/O error on device sdc, logical block 5
Buffer I/O error on device sdc, logical block 6
Buffer I/O error on device sdc, logical block 7
Buffer I/O error on device sdc, logical block 8
Buffer I/O error on device sdc, logical block 9
Buffer I/O error on device sdc, logical block 10
Buffer I/O error on device sdc, logical block 11
Buffer I/O error on device sdc, logical block 12
Buffer I/O error on device sdc, logical block 13
Buffer I/O error on device sdc, logical block 14
Buffer I/O error on device sdc, logical block 15
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Buffer I/O error on device sdc, logical block 0
Device sdc not ready.
end_request: I/O error, dev sdc, sector 272629632
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0

==================================================================================================== =

Device sde not ready.
end_request: I/O error, dev sde, sector 0
Buffer I/O error on device sde, logical block 0
Buffer I/O error on device sde, logical block 1
Buffer I/O error on device sde, logical block 2
Buffer I/O error on device sde, logical block 3
Buffer I/O error on device sde, logical block 4
Buffer I/O error on device sde, logical block 5
Buffer I/O error on device sde, logical block 6
Buffer I/O error on device sde, logical block 7
Buffer I/O error on device sde, logical block 8
Buffer I/O error on device sde, logical block 9
Buffer I/O error on device sde, logical block 10
Buffer I/O error on device sde, logical block 11
Buffer I/O error on device sde, logical block 12
Buffer I/O error on device sde, logical block 13
Buffer I/O error on device sde, logical block 14
Buffer I/O error on device sde, logical block 15
Device sde not ready.
end_request: I/O error, dev sde, sector 0
Buffer I/O error on device sde, logical block 0
Device sde not ready.
end_request: I/O error, dev sde, sector 157286272
Device sde not ready.
end_request: I/O error, dev sde, sector 0
Device sda not ready.
end_request: I/O error, dev sda, sector 0
Device sdc not ready.
end_request: I/O error, dev sdc, sector 0
Device sde not ready.
end_request: I/O error, dev sde, secto

why these errors and out there.
Pls. anyone help me out from this.
I need a stable OS and HW env and we need to deploy Oracle DB on real env.

Thanks
Regards
Rishi kapur

MuFTy · 03-27-2007, 07:26 PM

I have the same situation, when booting CentOS4.4 on a Diskless Host from a SAN.

Host: HP DL320g3
HBA: Emulex LP1050-E
SAN: EMC CX500

Issue is that CentOS4.4 has installed onto sdb, and these same errors are appearing on sda:

Two disks are detected, identical size.

CentOS is installed on sdb which does not sound safe.

Code:

Linux version 2.6.9-42.ELsmp (buildcentos@build-i386) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-3)) #1 SMP Sat Aug 12 09:39:11 CDT 2006
...
...
SCSI subsystem initialized
libata version 1.20 loaded.
ata_piix 0000:00:1f.2: version 1.05
ata_piix 0000:00:1f.2: MAP [ P0 P2 P1 P3 ]
ACPI: PCI interrupt 0000:00:1f.2[B] -> GSI 17 (level, low) -> IRQ 177
PCI: Setting latency timer of device 0000:00:1f.2 to 64
ata1: SATA max UDMA/133 cmd 0x1080 ctl 0x108A bmdma 0x10A0 irq 177
ata2: SATA max UDMA/133 cmd 0x1090 ctl 0x109A bmdma 0x10A8 irq 177
ata1: SATA port has no device.
scsi0 : ata_piix
ata2: SATA port has no device.
scsi1 : ata_piix
Emulex LightPulse Fibre Channel SCSI driver 8.0.16.27
Copyright(c) 2003-2006 Emulex.  All rights reserved.
ACPI: PCI interrupt 0000:09:01.0[A] -> GSI 48 (level, low) -> IRQ 217
lpfc 0000:09:01.0: 0:1303 Link Up Event x1 received Data: x1 xf7 x8 x0
scsi2 : Emulex LP1050 2Gb PCI-X Fibre Channel Adapter on PCI bus 09 device 08 irq 217
  Vendor: DGC       Model: RAID 5            Rev: 0219
  Type:   Direct-Access                      ANSI SCSI revision: 04
SCSI device sda: 42965504 512-byte hdwr sectors (21998 MB)
sda: asking for cache data failed
sda: assuming drive cache: write through
SCSI device sda: 42965504 512-byte hdwr sectors (21998 MB)
sda: asking for cache data failed
sda: assuming drive cache: write through
 sda:<6>Device sda not ready.
end_request: I/O error, dev sda, sector 0
Buffer I/O error on device sda, logical block 0
Device sda not ready.
end_request: I/O error, dev sda, sector 0
Buffer I/O error on device sda, logical block 0
Device sda not ready.
end_request: I/O error, dev sda, sector 0
Buffer I/O error on device sda, logical block 0
 unable to read partition table
Attached scsi disk sda at scsi2, channel 0, id 0, lun 0
  Vendor: DGC       Model: RAID 5            Rev: 0219
  Type:   Direct-Access                      ANSI SCSI revision: 04
SCSI device sdb: 42965504 512-byte hdwr sectors (21998 MB)
SCSI device sdb: drive cache: write through
SCSI device sdb: 42965504 512-byte hdwr sectors (21998 MB)
SCSI device sdb: drive cache: write through
 sdb: sdb1 sdb2
Attached scsi disk sdb at scsi2, channel 0, id 1, lun 0
device-mapper: 4.5.0-ioctl (2005-10-04) initialised: dm-devel@redhat.com
cdrom: open failed.
Device sda not ready.
end_request: I/O error, dev sda, sector 0
Buffer I/O error on device sda, logical block 0
Buffer I/O error on device sda, logical block 1
Buffer I/O error on device sda, logical block 2
Buffer I/O error on device sda, logical block 3
Buffer I/O error on device sda, logical block 4
Buffer I/O error on device sda, logical block 5
Buffer I/O error on device sda, logical block 6
Buffer I/O error on device sda, logical block 7
Buffer I/O error on device sda, logical block 8
Buffer I/O error on device sda, logical block 9
Buffer I/O error on device sda, logical block 10
Buffer I/O error on device sda, logical block 11
Buffer I/O error on device sda, logical block 12
Buffer I/O error on device sda, logical block 13
Buffer I/O error on device sda, logical block 14
Buffer I/O error on device sda, logical block 15
Device sda not ready.
end_request: I/O error, dev sda, sector 0
Buffer I/O error on device sda, logical block 0

ronanio · 03-28-2007, 05:07 AM

sda not ready means that you dont have failover switched on. You have only presented one disk havent you
? I also think you only have one card in the machine. But im guessing as per normal SAN conditions you have two routes/fabrics to the disk. This in itself isnt really a problem

MuFTy · 03-29-2007, 01:52 AM

I have figured out what was happening in my case, and likely in the other cases similar.

With the following setup:
Single Fabric
1xLUN 20Gb
SAN with 2x Storage Processors
1xHBA

There will be two paths from the HBA to the LUN, via each of the SP's. Each path is detected as a seperate disk.

The solution i am trying to put in place is a VMware Server 1.02 that boots from SAN.
With just one LUN exposed to the box which i was booting from, sda and sdb were being detected.

Depending on which path was active, say sdb, only sdb would mount, and sda would recive the I/O errors. This is ok aslong as there is no failover or tresspass in which case the system would likeley crash.

When using a SAN and mutliple paths, you need to use multipath tool to logicaly group the paths into a single device, or use the Path Management s/w supplied by the SAN Vendor ie PowerPath for EMC SAN. multipath is included from RHEL4 update 4

Booting from SAN and linux is basicly not supported with multiple paths with multipath because multipath cant group a device which is in use (ie system disk)

If you really want to boot linux from san, and want failover on the paths this guy has a hack to get it going:

basicly he forces multipath to load and mount the logical device before the filesystem.

I have not tested this myself, and have decided to use local disks for my VMware solution.