Share your knowledge at the LQ Wiki.
Go Back > Forums > Linux Forums > Linux - Software
User Name
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.


  Search this Thread
Old 04-26-2008, 05:30 AM   #1
Registered: Apr 2008
Posts: 114

Rep: Reputation: 15
Disaster with LVM


I had an LVM with two PV, md1 and md2 (both RAID1).
The VG was called "archive", and the LVM was "MaxtorLVM".
File system was ext3.
Cause of my inexperience with LVM, I deleted all the stuff (with lvremove, pvremove and vgremove command) but without overwriting the content of md devices or reuse them.
I thought that he was possible rebuild all the content of LVM, by recreate PV, VG and LV with the same size - parameter - uuid and so on..

Unfortunately, I get an error when I try to mount the LVM device:

# mount /dev/mapper/archive-MaxtorLVM /lvm/
mount: you must specify the filesystem type

# mount -t ext3 /dev/mapper/archive-MaxtorLVM /lvm/
mount: wrong fs type, bad option, bad superblock on /dev/mapper/archive-MaxtorLVM,
missing codepage or other error
In some cases useful info is found in syslog - try
dmesg | tail or so

dmesg says:

Apr 26 12:13:22 backup kernel: VFS: Can't find a valid ext3 filesystem on dev dm-0.

cat /proc/mdstat is ok.
Other info:
Debian stable, kernel

Having the backup in /etc/lvm/archive and /etc/lvm/backup, It's possibile to recover the content of the old LVM?

Old 04-27-2008, 05:11 PM   #2
Registered: Feb 2008
Location: Texas
Distribution: Fedora, RHEL, CentOS
Posts: 488

Rep: Reputation: 78
If only the LVM metadata has been modified and the actual filesystem data has been left in-tact (meaning no process has caused any modification) and you have an available config from archive or backup then you might be able to recover the LV by using vgcfgrestore. Consult the man page for its use.

I've used this to "repair" corrupt on-disk LVM metadata but never tried it after lv-, vg-, pvremove.

Hope this helps.

Edit 1
Ok, so you got me curious.
A quick run-down of what I did and my results.

presented a ~20GB iscsi target to my system (sdd)
created single LinuxLVM primary partition sdd1
pvcreate /dev/sdd1
vgcreate testVG /dev/sdd1
lvcreate -l 100%VG -n testlV /dev/testVG
mke2fs -j /dev/testVG/testLV
mount /dev/testVG/testLV /mnt/test
cp /tmp/lvm.tgz /mnt/test/
umount /mnt/test
lvremove /dev/testVG/testLV
vgremove /dev/testVG
pvremove /dev/sdd1
pvcreate /dev/sdd1
vgcreate /dev/test2VG
lvcreate -l 100%VG -n test2LV /dev/test2VG
mount /dev/test2VG/test2LV /mnt/test
get the same fs error you are seeing
lvremove /dev/test2VG/test2LV
vgremove /dev/test2VG
pvremove /dev/sdd1
back to a partition (sdd1) with no LVM metadata on it
at this point i reviewed the testVG_00*.vg in /etc/lvm/archive to find the one that has the correct information that i want/need.
mine happened to be "", yours may (and will likely) be different.

grab the pv uuid from there for the next command
pvcreate --restorefile /etc/lvm/archive/ --uuid N9iohj-6Rhd-Tv6S-8Wo9-NtnZ-dpj3-JVcF0G  /dev/sdd1
proceeding, restore the vg metadata, confirm the vg info, activate the lv, and attempt mount
vgcfgrestore  -f  /etc/lvm/archive/ -v testVG
vgdisplay -v /dev/testVG
vgchange -ay /dev/testVG
mount /dev/testVG/testLV /mnt/test
mount fails again with your error

run an fsck on it (i used ext3 for my fs)
e2fsck /dev/testVG/testLV
you will in all likely-hood encounter "problems" with the filesystem, i said yes to all questions to fix or correct a problem;
ran an fsck a 2nd time, came back clean
#mount /dev/testVG/testLV /mnt/test
#sha1sum /mnt/test/lvm.tgz /tmp/lvm.tgz
  a73297ecb368a1b0aeea98b4f39c37363fd7f49f  /mnt/test/lvm.tgz
  a73297ecb368a1b0aeea98b4f39c37363fd7f49f  /tmp/lvm.tgz
the file on the testVG that i just restored matches the original when output compared of sha1sum of the two files.

I can only hope you are as successful.

Hope this helps even more.

Last edited by rayfordj; 04-27-2008 at 06:36 PM.
Old 05-01-2008, 02:46 AM   #3
Registered: Apr 2008
Posts: 114

Original Poster
Rep: Reputation: 15

sorry I'm late, your answer was very useful for me
I did the same procedure and all data have been recovered!

Thanks a lot!!
Old 05-01-2008, 07:11 AM   #4
Registered: Feb 2008
Location: Texas
Distribution: Fedora, RHEL, CentOS
Posts: 488

Rep: Reputation: 78
Great news!
I'm glad it worked out for you!
Old 04-18-2010, 01:09 PM   #5
LQ Newbie
Registered: May 2009
Posts: 4

Rep: Reputation: 0
Thumbs up Thank you!


Thank you for your wonderful post! With your example I was able to successfully recover ~ 650 MB of family photos, music, etc.

Truly a life saver!
( I keep a compressed archive of my /etc/lvm directory in the cloud for easy access )

Thanks again.


..for any others that may be in need, there is also a similar solution involving multiple physical volumes here:
Old 06-28-2011, 03:57 AM   #6
LQ Newbie
Registered: Jun 2011
Posts: 1

Rep: Reputation: Disabled

0 members found this post helpful.
Old 11-26-2011, 08:16 AM   #7
LQ Newbie
Registered: Nov 2011
Posts: 2

Rep: Reputation: Disabled
I don't suppose by some miracle you guys are still following this thread...and I'm sorry for bumping an old thread...

I feel like I'm so close to being able to get my data back, but I'm stuck. Everything goes smoothly

vgcfgrestore  -f  /etc/lvm/archive/ -v testVG
vgdisplay -v /dev/testVG
vgchange -ay /dev/testVG
but with

mount -t ext4 /dev/group0/home /home
mount: wrong fs type, bad option, bad superblock on /dev/mapper/group0-home,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
It's the just the modified version of
mount /dev/testVG/testLV /mnt/test
It's definitely ext4 and I've tried to restore the superblock with
e2fsck -b
but without any luck and now I'm at the point that I'm not sure what to do. Any help would be so massively appreciated.
Old 11-26-2011, 09:33 AM   #8
Registered: Aug 2006
Location: Saint Paul, MN, USA
Distribution: {Free,Open}BSD, CentOS, Debian, Fedora, Solaris, SuSE
Posts: 735

Rep: Reputation: 76
Hi, jtholb.

Welcome to LQ.

Yes, some people are subscribed, but this is a very old thread.

In general, you are far more likely to receive more views of your question (and so more answers and discussion) if it is in a new thread. It's rarely the case that 2 problems are exactly the same, so a fresh thread is almost always your best bet. If you think the problems are close, you could refer to this thread with a link.

Best wishes ... cheers, makyo
Old 12-03-2011, 06:45 AM   #9
Registered: Jan 2010
Location: Velveeta, USA
Distribution: Xen, Gentoo,Ubuntu,openSUSE,Debian,pfSense
Posts: 98

Rep: Reputation: 9
At this point I would suggest buying one of the commercial data recovery apps.

I had a similar error with my volume group, with a successful vgcfgrestore, except I was using xfs, not ext3/4

Writing the VG metadata apparently touches a block or two shared by filesystem metadata.

I used one of the "deep scan" data recovery apps (wasn't expensive either, less than $100) and the deep scan found all of my data (missing some/most filenames).
Old 12-03-2011, 07:13 AM   #10
LQ Newbie
Registered: Nov 2011
Posts: 2

Rep: Reputation: Disabled
Sorry for bumping an old thread. In the end I figured it out and all my data was still intact--lucky me. For a more thorough explanation of it all you can go to (I didn't actually get any replies, so it ended up being entirely my own posts of what was happening). In the end it was basically three commands that fixed things for me.

mdadm --create --metadata=0.9 --raid-devices=3 --chunk=64 --level=5 --assume-clean /dev/md127 /dev/sdb1 /dev/sdc1 missing --uuid=d6bfa8db:624bcebf:cd0fe2e0:41039747

pvcreate -u sftW7n-JaLc-l85n-IcJg-3fqW-Dd9k-NxyKjK /dev/md127

vgcfgrestore -f /etc/lvm/archive/ group0
Thanks you guys.


lvm, lvm2, recover, rescue

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Managing LVM with the LVM Manager LXer Syndicated Linux News 0 03-16-2008 11:10 AM
LXer: Back Up (And Restore) LVM Partitions With LVM Snapshots LXer Syndicated Linux News 0 04-17-2007 11:16 AM
disaster plan: raid + lvm nmayotte Linux - Software 2 04-18-2006 02:23 PM
One newb's disaster Showtime8000 Linux - General 1 10-24-2004 05:32 PM
kERNEL DISASTER nelsonnery Linux - Newbie 1 10-09-2003 04:46 AM > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 02:02 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration