LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Server (http://www.linuxquestions.org/questions/linux-server-73/)
-   -   dm-linear: Device lookup failed (http://www.linuxquestions.org/questions/linux-server-73/dm-linear-device-lookup-failed-4175457766/)

SukkoPera 04-11-2013 02:16 AM

dm-linear: Device lookup failed
 
Hi everybody,

here at work we have a few virtual servers, running on Proxmox. There are nightly backups scheduled for each of them, through Proxmox itself.

One day, backups on one physical node started failing with the following error:

Code:

command 'lvcreate --size 1024M --snapshot --name 'vzsnap-ve2-0' '/dev/ve2ds1-sync/vm-201-disk-1'' failed: exit code 5
Note: /dev/ve2ds1-sync is an LVM VG that is replicated on another machine through drbd for redundancy. vm-201-disk-1 is the LV that is used as virtual disk for one of the virtual machines, but the same happens for all VGs.

Some more info is available in the logs:

Code:

Mar 11 01:00:03 INFO:  device-mapper: create ioctl on ve2ds1--sync-vzsnap--ve2--0 failed: Device or resource busy
Mar 11 01:00:03 INFO:  Aborting. Failed to activate snapshot exception store.

So it seems LVM cannot create LVs anymore. dmesg reports a few errors:

Code:

device-mapper: ioctl: device doesn't appear to be in the dev hash table.
device-mapper: table: 253:9: linear: dm-linear: Device lookup failed
device-mapper: ioctl: error adding target to table

The LV shows up in dmsetup ls:

Code:

root@ve2:~# dmsetup ls
ve2ds1--sync-vzsnap--ve2--0    (253:9)

I can remove it with dmsetup remove but this doesn't solve anything. What can I do? I might try rebooting the server, but that is really the last resort, since these servers are production ones.

Any suggestions? Thanks in advance!

smallpond 04-12-2013 12:56 PM

Exit code 5 probably means EIO - an I/O error occurred.
Check the status of your LVM with
Code:

lvdisplay /dev/ve2ds1-sync/vm-201-disk-1
Also check your log for disk errors.

SukkoPera 04-15-2013 02:41 AM

Thanks for the help, smallpond. The LVM is ok, AFAIK:

Code:

root@ve2:~# lvdisplay /dev/ve2ds1-sync/vm-201-disk-1
  --- Logical volume ---
  LV Path                /dev/ve2ds1-sync/vm-201-disk-1
  LV Name                vm-201-disk-1
  VG Name                ve2ds1-sync
  LV UUID                v4HJgO-ce1E-uEtm-OlKH-149O-PsGK-TRnnTq
  LV Write Access        read/write
  LV Creation host, time ve2, 2012-12-02 17:28:10 +0100
  LV Status              available
  # open                1
  LV Size                20.00 GiB
  Current LE            5120
  Segments              1
  Allocation            inherit
  Read ahead sectors    auto
  - currently set to    256
  Block device          253:5

Also, the disks are OK, as they are hardware RAID1 (MegaRAID) and we have no failures ATM.

whizje 04-15-2013 03:06 AM

test the disks with smartctl -t long /dev/sda

SukkoPera 04-15-2013 06:58 AM

I can't use smartctl, as my disks are logical disks and the physical disks are hidden by the MegaRAID controller. Anyway I have started a RAID consistency check with MegaRAID's utility, and all my disks are OK.

SukkoPera 05-08-2013 07:20 AM

Just for reference: we finally solved this rebooting everything.


All times are GMT -5. The time now is 07:18 AM.