A few questions but the main one is:
How to debug this modprobe hung task?
Background:
This started to happen when /etc/ was accidentally removed. We recovered the folder by copying over from another host and modifying a bunch of files including the networking, hosts, resolv.conf any any other that had references to the previous host.
Also modified the UUID in /etc/fstab for /boot.
However we may have missed something still.
One of the other issues is also that udevadm trigger causes the 70-persistent-net rules to be regenerated, which doesn't happen on other hosts. ( Have not rebooted yet since recovering. Hoping to trace the remainder of the issues before rebooting to ensure highest chance of recovery. It's an important server. ) These issues are on a CentOS 6.7 distribution.
Hoping for any tips.
Cheers,
Code:
Apr 6 23:59:39 devlan-p01 kernel: INFO: task modprobe:17780 blocked for more than 120 seconds.
Apr 6 23:59:39 devlan-p01 kernel: Not tainted 2.6.32-504.16.2.el6.x86_64 #1
Apr 6 23:59:39 devlan-p01 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 6 23:59:39 devlan-p01 kernel: modprobe D 0000000000000020 0 17780 17668 0x00000080
Apr 6 23:59:39 devlan-p01 kernel: ffff88142329be18 0000000000000086 0000000000000000 ffff88142329be40
Apr 6 23:59:39 devlan-p01 kernel: ffff88161be0d520 7fffffffffffffff 000367dbc5652eba ffffffff8152a9cd
Apr 6 23:59:39 devlan-p01 kernel: ffffffff8100bb8e 0000000139289b3d ffff88161be0dad8 ffff88142329bfd8
Apr 6 23:59:39 devlan-p01 kernel: Call Trace:
Apr 6 23:59:39 devlan-p01 kernel: [<ffffffff8152a9cd>] ? wait_for_common+0x14d/0x180
Apr 6 23:59:39 devlan-p01 kernel: [<ffffffff8100bb8e>] ? apic_timer_interrupt+0xe/0x20
Apr 6 23:59:39 devlan-p01 kernel: [<ffffffff810585cf>] ? mutex_spin_on_owner+0x9f/0xc0
Apr 6 23:59:39 devlan-p01 kernel: [<ffffffff8152b7a6>] __mutex_lock_slowpath+0x96/0x210
Apr 6 23:59:39 devlan-p01 kernel: [<ffffffff8152b2cb>] mutex_lock+0x2b/0x50
Apr 6 23:59:39 devlan-p01 kernel: [<ffffffff81457b8d>] unregister_pernet_subsys+0x1d/0x40
Apr 6 23:59:39 devlan-p01 kernel: [<ffffffffa017c252>] iptable_mangle_fini+0x26/0x28 [iptable_mangle]
Apr 6 23:59:39 devlan-p01 kernel: [<ffffffff810bd004>] sys_delete_module+0x194/0x260
Apr 6 23:59:39 devlan-p01 kernel: [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
