LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-17-2015, 04:16 PM   #1
cbtshare
Member
 
Registered: Jul 2009
Posts: 645

Rep: Reputation: 42
EXT4-fs group descriptors corrupted


Hello All,

My hard drive recently failed on my raid and got a new drive and got the server booting again, but now my NFS 2TB drive won't mount.Can you help please.

Thank you

Code:
 lvs
  LV      VG     Attr     LSize  Pool Origin Data%  Move Log Copy%  Convert
  lv_root vg00   -wi-ao-- 35.57g
  lv_swap vg00   -wi-ao--  3.94g
  lvdata  vgdata -wi-a---  2.00t
Code:
mount -t ext4 /dev/vgdata/lvdata /data/
mount: wrong fs type, bad option, bad superblock on /dev/mapper/vgdata-lvdata,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so


EXT4-fs (dm-2): group descriptors corrupted!
device-mapper: ioctl: device doesn't appear to be in the dev hash table.
EXT4-fs (dm-2): ext4_check_descriptors: Checksum for group 3968 failed (16234!=0)
EXT4-fs (dm-2): group descriptors corrupted!
Code:
 mke2fs -n /dev/sdb1
mke2fs 1.41.12 (17-May-2010)
/dev/sdb1 is apparently in use by the system; will not make a filesystem here!

lsof /dev/mapper/vgdata-lvdata
[root@nfs01 ~]#
Code:
 fsck /dev/vgdata/lvdata
fsck from util-linux-ng 2.17.2
e2fsck 1.41.12 (17-May-2010)
fsck.ext4: Group descriptors look bad... trying backup blocks...
fsck.ext4: Bad magic number in super-block when using the backup blocks
fsck.ext4: going back to original superblock
fsck.ext4: Device or resource busy while trying to open /dev/mapper/vgdata-lvdata
Filesystem mounted or opened exclusively by another program?


lsof /dev/mapper/vgdata-lvdata
 fuser /dev/mapper/vgdata-lvdata
[root@nfs01 ~]#
Code:
--- Volume group ---
  VG Name               vgdata
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               2.00 TiB
  PE Size               4.00 MiB
  Total PE              524286
  Alloc PE / Size       524286 / 2.00 TiB
  Free  PE / Size       0 / 0
  VG UUID               sq4utn-JLOM-j0Ho-h2io-l8hY-JmGO-vjNCLu

  --- Logical volume ---
  LV Path                /dev/vgdata/lvdata
  LV Name                lvdata
  VG Name                vgdata
  LV UUID                ilqGXq-wjf9-lXaR-WTce-SEgJ-T8wb-4NaArS
  LV Write Access        read/write
  LV Creation host, time nfs01.domain.com, 2014-08-29 13:46:16 -0400
  LV Status              available
  # open                 0
  LV Size                2.00 TiB
  Current LE             524286
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:2
Code:
lvm lvs -a -o+devices
  LV      VG     Attr     LSize  Pool Origin Data%  Move Log Copy%  Convert Devices
  lv_root vg00   -wi-ao-- 35.57g                                            /dev/sda2(0)
  lv_swap vg00   -wi-ao--  3.94g                                            /dev/sda2(9106)
  lvdata  vgdata -wi-a---  2.00t                                            /dev/sdb1(0)
 
Old 07-18-2015, 12:12 AM   #2
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
Did you google the error message "device doesn't appear to be in the dev hash table"?

Are you sure that lvdata has an ext filesystem? Not xfs or something else?

Try running blkid; it tries to figure out what is on that volume.

Also, this:

Code:
mke2fs -n /dev/sdb1
If you want to find the superblocks on lvdata, run this command with lvdata, not sdb1.

Finally, to clarify, is lvdata located on your RAID? Are there other volumes on the RAID that have problems? What kind of RAID, hardware or software? Any messages referring to the RAID and lvdata in the message buffer (dmesg or /var/log/[syslog|messages])?

Last edited by berndbausch; 07-18-2015 at 12:15 AM.
 
Old 07-19-2015, 01:55 AM   #3
cbtshare
Member
 
Registered: Jul 2009
Posts: 645

Original Poster
Rep: Reputation: 42
Thank you for your assistance.Yes, I did an extensive search in google.

Yes i am sure it is EXT4
Code:
dev/mapper/vgdata-lvdata: UUID="f4803c5a-8149-438b-a16e-fbd2a73c9ffb" TYPE="ext4

I have hardware RAID, there is no reference to the device in dmesg

Code:
XT4-fs (dm-2): ext4_check_descriptors: Checksum for group 3992 failed (18287!=0)
EXT4-fs (dm-2): ext4_check_descriptors: Checksum for group 3993 failed (43630!=0)
EXT4-fs (dm-2): ext4_check_descriptors: Checksum for group 3994 failed (56686!=0)
EXT4-fs (dm-2): ext4_check_descriptors: Checksum for group 3995 failed (12399!=0)
EXT4-fs (dm-2): ext4_check_descriptors: Checksum for group 3996 failed (13166!=0)
EXT4-fs (dm-2): ext4_check_descriptors: Checksum for group 3997 failed (56943!=0)
EXT4-fs (dm-2): ext4_check_descriptors: Checksum for group 3998 failed (43375!=0)
EXT4-fs (dm-2): ext4_check_descriptors: Checksum for group 3999 failed (17518!=0)
EXT4-fs (dm-2): ext4_check_descriptors: Inode bitmap for group 4000 not in group (block 536868864)!
EXT4-fs (dm-2): group descriptors corrupted!
root@nfs01 ~]# mke2fs -n /dev/mapper/vgdata-lvdata
m
Code:
ke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
134217728 inodes, 536868864 blocks
26843443 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
16384 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        102400000, 214990848, 512000000
I'm not sure what else to try, any other suggestions?
 
Old 07-19-2015, 05:52 AM   #4
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
Quote:
Originally Posted by cbtshare View Post
I have hardware RAID, there is no reference to the device in dmesg
You mention it in your original posting. Is it relevant to the problem with lvdata?

Code:
/dev/sdb1 is apparently in use by the system; will not make a filesystem here!
That's because it's a physical volume, I would think.
Code:
fsck /dev/vgdata/lvdata
(...)
fsck.ext4: Device or resource busy while trying to open /dev/mapper/vgdata-lvdata
Filesystem mounted or opened exclusively by another program?
It seems that fsck never attempts to repair anything; is this correct? Perhaps you are luckier when running fsck in single-user mode.
You can also try fsck -b <one of the backup superblocks displayed by mkfs>, but it seems that it does it by itself already, with little success.
 
Old 07-21-2015, 09:15 AM   #5
cbtshare
Member
 
Registered: Jul 2009
Posts: 645

Original Poster
Rep: Reputation: 42
Yes it is relevant because the system is on a raid array and on that raid I use vcenter to make a host then from that create a vm, and assign hard drives.There are two hard drives in the system, /dev/sda the root and boot partition and /dev/sdb the nfs storage.

You are correct, fsck, never attempt to repair anything it just throws an error.I did try in single user mode, but will try again after the drive has finished being imaged.
 
Old 07-21-2015, 09:33 AM   #6
cbtshare
Member
 
Registered: Jul 2009
Posts: 645

Original Poster
Rep: Reputation: 42
I tried running FSCK in single user mode now, and I am getting this error in the screen shot.
Attached Thumbnails
Click image for larger version

Name:	VM.jpg
Views:	904
Size:	48.9 KB
ID:	18976  
 
Old 07-26-2015, 08:49 PM   #7
cbtshare
Member
 
Registered: Jul 2009
Posts: 645

Original Poster
Rep: Reputation: 42
I just ran fsck again, and again it deletes my data.why is that? there is nothing in lost+found except 2 files that are 400mb in total.The size of data on there was 2.TB.There is nothing in the logs either
 
Old 07-28-2015, 10:49 AM   #8
cbtshare
Member
 
Registered: Jul 2009
Posts: 645

Original Poster
Rep: Reputation: 42
I guess nothing?
 
Old 07-29-2015, 03:01 AM   #9
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
Quote:
Originally Posted by cbtshare View Post
I guess nothing?
Sorry I am on leave and spend most of the time in the real world rather than the usual cyberspace

It's also hard to say what's going on. Why lvdata is "busy" although you are in single user mode is a mystery to me.
Regarding the data: fsck repairs filesystem structures, not data. It identifies inconsistencies like wrong summary counters (benign), files that have no directory entries (will be moved to lost+found) and data blocks that belong to no files but are not marked free either (will be marked free). File content is not considered. So you may say that fsck has removed your data, but what it has really done is marking unaccounted-for data blocks as free, as it is unable to guess what files they might belong to.

So after fsck has done its work, you may end up with an empty, but consistent filesystem.

Your data is probably still there. With the help of a filesystem debugger like debugfs you might be able to salvage something, but it would be very tedious.

My feeling is that something is wrong with the way you defined the physical and/or logical volume(s) on top of your RAID, but I would have to play around with the disks to get clues.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
group descriptors corrupted! illuminate0 Linux - Newbie 1 11-16-2011 05:52 PM
Block bitmap for group 416 not in group (block 0) + group descriptors corrupted? quanta Linux - Server 1 12-08-2010 10:40 AM
EXT3-fs: group descriptors corrupted! secretlydead Linux - Hardware 1 01-31-2010 08:34 AM
EXT3-fs: group descriptors corrupted dadriver Linux - Hardware 2 02-04-2006 03:16 AM
EXt3-fs:group descriptors corrupted. anjanesh Fedora 0 11-15-2004 12:26 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 09:02 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration