Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I have a VMware VM running CentOS 5.11 (I know), where the ESXi host lost the VLAN for storage traffic, and now the VM dies on boot -
Code:
EXT3-fs error (device dm-): ext3_lookup: unlinked inode 2424833 in dir #2
EXT3-fs error (device dm-): ext3_lookup: unlinked inode 2785281 in dir #2
exec of init (/sbin/init) failed!!!: No such file or directory
Kernel panic - not syncing: Attempted to kill init!
This VM has one drive with two partitions. I grabbed the 5.11 CentOS rescue CD & booted into 'linux rescue' mode. fsck reports sda1 as /boot and clean, and I can mount it without issue. The second partition is an LVM with a few volumes, but is unable to detect anything on sda2-
Code:
fsck 1.38 (30-Jun-2005)
fsck: WARNING: couldn't open /etc/fstab: No such file or directory
e2fsck 1.38 (30-Jun-2005)
Couldn't find ext2 superblock, trying backup blocks...
fsck.ext2: while trying to open /dev/sda2
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
fsck.ext2:
So I grabbed the list of superblocks with -
Code:
mke2fs -n /dev/sda2
Then for each superblock I tried running -
Code:
e2fsck -b <SuperBlock> /dev/sda2
And every time it errored with -
Code:
e2fsck 1.38 (30-Jun-2005)
e2fsck: while trying to open /dev/sda2
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
e2fsck:
/dev/sda2 is apparently your LVM pv - that means it is a (logical) device; it does not directly contain a filesystem, so it can't be fsck'd. You will have a selection of lv's within that pv, and these each do contain a filesystem. The CentOS liveCD should find and mount them all if possible at boot. Let's see what this returns - might need sudo.
Thanks for your response. Looks like lsblk wasn't available until CentOS/RHEL 6, was getting command not found. I mounted a CentOS 7 ISO I had readily available and ran it that way -
So at least they are recognisable. Before you start, note my sigline - get a backup of the current state of affairs.
I would simply activate the lvs, and fsck them. If you're lucky good things might happen and once they are mounted you can check them and hope the contents are valid.
FWIW, I rarely trust a filesystem after a major "event" - I reformat and restore from most recent valid backup. Given that we have progressed this far I'm presuming you don't have such.
I do not have a good backup, no. Unfortunately I thought it had been backing up daily, found a set of VMs that weren't. I had snapshotted the machine in a working but broken state. You can't SSH/console into it because it throws inode errors, it doesn't even let you try. But the services on the box are still responding. So I had cloned that machine into a new one and snapshotted it's basic broken state and have been working on that cloned box and presented this thread as such for simplicity. But I can at least reset it to the basic broken image if need be.
So, before I continue, a heads up that I've found a separate backup and so I'll be able to fully recover and have started going that route.
For the sake of this thread and continued learning, I'd like to see if it's still possible to fix this VM. Continuing with what you had said to try.
I ran -
Code:
# lvm vgscan
Reading volume groups from cache.
Found volume group "VolGroup00" using metadata type lvm2
# vchange -a y
2 logical volume(s) in volume group "VolGroup00" now active
Then I mounted /dev/VolGroup00/LogVol00 to /tmp, and did an ls, which showed a few directories with '?' in place of values, including for the /etc directory. So I unmounted it and ran 'fsck /dev/VolGroup00/LogVol00 -y' against it. When that was done, /etc was no longer even listed. LogVol01 is swap. I'm going to revert to snapshot and prod at it some more, any other thoughts or does this look like the original was thoroughly boned?
I'm sorry, but it was definitely the vlan. The event was correlated to user/time on top of them coming forward about it. You can't just drop the vlan passing storage traffic, it's similar to just pulling out SSD from a normal powered on machine- bad things can happen.
I haven't tried testdisk yet though, will let you know if that has any luck.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.