Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place! |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
|
11-11-2016, 11:00 AM
|
#16
|
Senior Member
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,820
|
You have damaged the xfs filesystem, and recovery attempts may fail. The fatal step was the resizing of the expanded root filesystem (" lvextend -r -l+100%FREE /dev/mapper/centos_sfvm03-root"). That has written some filesystem metadata into the new space, essentially scribbling garbage over parts of the old home filesystem.
Usually, the first step in recovery would be to undo the resizing of the root LV and filesystem, putting it back to its original 12800 extents. Unfortunately, your root filesystem appears to be xfs, and there is no way to shrink an xfs filesystem in place. The only way to do that is to back up the entire root filesystem elsewhere. I was going to suggest using pvmove to do that, but that would conflict with restoring the LVM configuration with vgcfgrestore and just make things even more complicated. So however you do it, make sure that the backup is stored somewhere not in the centos_sfvm03 volume group.
Once you have the root filesystem backed up, you can use vgcfgrestore to put the LVM configuration bck the way it was. Look in the most recent centos_sfvm03_*.vg files in /etc/lvm/archive (they are ASCII files) for one that contains the description, "Created *before* executing 'lvreduce -L 70G /dev/mapper/centos_sfvm03-home'", and run
Code:
vgcfgrestore -f /path/to/that/file centos_sfvm03
You can include the "--test" and "--verbose" options if you like to see what it will do.
Then, you can make a new xfs filesystem in /dev/mapper/centos_sfvm03-root and restore that root filesystem from its backup.
Finally, you can attempt your xfs_repair on /dev/mapper/centos_sfvm03-home. Due to the "scribbling" above, there is no guarantee that it will be successful, but that's your best shot. If it fails and you can't recover you data, ... well ..., running a server for "200-300 websites" with no backups leaves you vulnerable to that.
|
|
1 members found this post helpful.
|
11-11-2016, 02:15 PM
|
#17
|
LQ Newbie
Registered: Nov 2016
Posts: 15
Original Poster
Rep: 
|
Quote:
Originally Posted by rknichols
Usually, the first step in recovery would be to undo the resizing of the root LV and filesystem, putting it back to its original 12800 extents. Unfortunately, your root filesystem appears to be xfs, and there is no way to shrink an xfs filesystem in place.
|
Hello rknichols, thanks so much for your detailed answer.
I'm not sure about type of file system of root. but now I tried
Code:
lvreduce -t -l 12800 /dev/mapper/centos_sfvm03-root
and got "size of logical volume centos-sfvm03/root changed from 77.51 GiB (19842 extents to 50.00 GiB (12800 extents)." message in test mode.
And I didn't get the part which you said I need to take backup from root to somewhere. What is outcome out of that? and if I could resize root, do I need to go with backup etc.?
On the other hand, the .vg file just before changes is /etc/lvm/archive/centos_sfvm03_00001-1621702387.vg there are 2 more files there, one very old, and one with 2 second after which the name is centos_sfvm03_00002-1249077736.vg so in
Code:
vgcfgrestore -f /path/to/that/file centos_sfvm03
you meant path to this file, isn't that?
Again thanks for your replies which will help me solving this issue
|
|
|
11-11-2016, 05:27 PM
|
#18
|
Senior Member
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,820
|
Quote:
Originally Posted by sahruk
Hello rknichols, thanks so much for your detailed answer.
I'm not sure about type of file system of root. but now I tried
Code:
lvreduce -t -l 12800 /dev/mapper/centos_sfvm03-root
and got "size of logical volume centos-sfvm03/root changed from 77.51 GiB (19842 extents to 50.00 GiB (12800 extents)." message in test mode.
|
Do NOT do that! That would damage the root filesystem as well. Any time you reduce the size of an LVM logical volume, you must shrink the filesystem, either as part of the LVM operation (" lvreduce -r ..."), or else prior to the LVM operation. Otherwise, you are left with a broken filesystem that is larger than the volume that is holding it. Since there is no way to shrink an xfs filesystem, neither option is possible.
Quote:
And I didn't get the part which you said I need to take backup from root to somewhere. What is outcome out of that? and if I could resize root, do I need to go with backup etc.?
|
If (and that's a very big "if") you could resize root, you wouldn't need to do the backup. You can try it. Just run " lvreduce -r -l 12800 centos_sfvm03-root", but that is going to fail if root is an xfs filesystem. Do NOT leave out the "-r".
Quote:
On the other hand, the .vg file just before changes is /etc/lvm/archive/centos_sfvm03_00001-1621702387.vg there are 2 more files there, one very old, and one with 2 second after which the name is centos_sfvm03_00002-1249077736.vg so in
Code:
vgcfgrestore -f /path/to/that/file centos_sfvm03
you meant path to this file, isn't that?
|
Yes. There indeed should be a later file with the description "Created *before* executing 'lvextend -r -l+100%FREE /dev/mapper/centos_sfvm03-root'". That first file is the one you want. It will restore the volume group back to the way it was before you did any of this. That will give you a home filesystem that fits properly in its container. Now the issue is whatever unknown damage was done by the metadata written over that filesystem when you expanded the root filesystem. Running xfs_repair will either fix that or give you the bad news.
Last edited by rknichols; 11-11-2016 at 05:29 PM.
|
|
1 members found this post helpful.
|
11-11-2016, 11:08 PM
|
#19
|
LQ Newbie
Registered: Nov 2016
Posts: 15
Original Poster
Rep: 
|
Quote:
Originally Posted by rknichols
If (and that's a very big "if") you could resize root, you wouldn't need to do the backup. You can try it. Just run "lvreduce -r -l 12800 centos_sfvm03-root", but that is going to fail if root is an xfs filesystem. Do NOT leave out the "-r".
|
Once again thanks for your reply.
Is there any way to find out type of partitions at the moment while home damaged?
|
|
|
11-12-2016, 01:06 AM
|
#20
|
LQ Newbie
Registered: Nov 2016
Posts: 15
Original Poster
Rep: 
|
I copied VM Disk to somewhere else to deal with data recovery. I used 2 different software for data recovery. Geeksnerd XFS data recovery, which found even old sized partition on hard and I tried to scan files, almost 100% are found recoverable, and second software is UFS Explorer which showing me disks with new partition but I can travel live over home partition and almost 100% are recoverable. Just curious how fast UFS Explorer could find files and folder structures lively while partition is damaged without even scanning! Just curious why it's not possible to heal partition while a software like UFS Explorer makes traveling lively possible?
|
|
|
11-12-2016, 08:13 AM
|
#21
|
Senior Member
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,820
|
Quote:
Originally Posted by sahruk
Is there any way to find out type of partitions at the moment while home damaged?
|
Run " lsblk -f" to see the filesystem types on all existing devices. You can also run " file -sL /dev/mapper/centos_sfvm03-root".
|
|
|
11-12-2016, 08:54 AM
|
#22
|
Senior Member
Registered: Nov 2011
Location: London, UK
Distribution: Debian
Posts: 1,959
|
I'm wondering why you are using a non-resizable file system with LVM (one benefit of which is the simple resizing of volumes)?
|
|
|
11-12-2016, 09:58 AM
|
#23
|
LQ Guru
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware (15 current), Slack15, Ubuntu studio, MX Linux, FreeBSD 13.1, WIn10
Posts: 10,342
|
Quote:
Originally Posted by JeremyBoden
I'm wondering why you are using a non-resizable file system with LVM (one benefit of which is the simple resizing of volumes)?
|
his answer from an earlier post.
Quote:
..I'm just managing the server, not the first creator.
|
|
|
|
11-12-2016, 11:00 AM
|
#24
|
Senior Member
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,820
|
Quote:
Originally Posted by JeremyBoden
I'm wondering why you are using a non-resizable file system with LVM (one benefit of which is the simple resizing of volumes)?
|
The default filesystem type for RHEL/CentOS 7 is xfs, and the default storage arrangement is LVM, so blame Red Hat. There is no problem with expanding an xfs filesystem. It's just shrinking that can't be done.
Last edited by rknichols; 11-12-2016 at 11:01 AM.
|
|
1 members found this post helpful.
|
11-12-2016, 05:59 PM
|
#25
|
LQ Newbie
Registered: Nov 2016
Posts: 15
Original Poster
Rep: 
|
Quote:
Originally Posted by rknichols
The default filesystem type for RHEL/CentOS 7 is xfs, and the default storage arrangement is LVM, so blame Red Hat. There is no problem with expanding an xfs filesystem. It's just shrinking that can't be done.
|
Now that I'm almost done with recovering data, would you please instruct me how exactly should I expand?
Server is running as a virtual machine (VMWare) and I have ability to extend hard disk, this makes a new unformated raw partition. Shall I first create a physical partition? then add it to Volume Group or what?
|
|
|
11-12-2016, 07:30 PM
|
#26
|
Senior Member
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,820
|
Making a virtual disk larger typically gives you unallocated space, not an "unformatted raw partition." Use any partitioning tool to make a new partition in that space, add it to the existing LVM volume group, and then enlarge whichever LVs need more space. You can do a pvcreate on the new partition prior to the vgextend, but a pvcreate will be done automatically (with default parameters) if you don't. Use the "-r" (--resizefs) option on the lvextend command as you did before so that the filesystem gets enalrged as well.
If your added space is on a new virtual disk rather than enlarging the present one, then it is your choice whether to partition it or just use the entire unpartitioned device as an LVM physical volume. I always recommend making a partition first. That is so that someone doesn't come along later, see that a partitioning tool reports the drive as having no partition table, and conclude that the drive is unused.
|
|
1 members found this post helpful.
|
11-13-2016, 06:54 AM
|
#27
|
LQ Newbie
Registered: Nov 2016
Posts: 15
Original Poster
Rep: 
|
Any one has any idea about issue below?
Code:
[root@sfvm08 mapper]# ls
centos_sfvm03-home centos_sfvm03-root centos_sfvm03-swap control
[root@sfvm08 mapper]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/centos_sfvm03-root 78G 69G 9.1G 89% /
/dev/loop0 2.2G 29M 2.1G 2% /tmp
home and swap doesn't show up in list above, but I already mounted it and have access to /home !!
|
|
|
11-13-2016, 08:42 AM
|
#28
|
Senior Member
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,820
|
Let's see the output from "lsblk" and "df /home".
Last edited by rknichols; 11-13-2016 at 08:47 AM.
|
|
|
11-13-2016, 09:07 AM
|
#29
|
LQ Newbie
Registered: Nov 2016
Posts: 15
Original Poster
Rep: 
|
Quote:
Originally Posted by rknichols
Let's see the output from "lsblk" and "df /home".
|
Code:
[root@sfvm08 mapper]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
fd0 2:0 1 4K 0 disk
sda 8:0 0 200G 0 disk
├─sda1 8:1 0 500M 0 part
└─sda2 8:2 0 149.5G 0 part
├─centos_sfvm03-root 253:0 0 77.5G 0 lvm /
├─centos_sfvm03-swap 253:1 0 2G 0 lvm [SWAP]
└─centos_sfvm03-home 253:2 0 50G 0 lvm /home
sr0 11:0 1 603M 0 rom
loop0 7:0 0 2.3G 0 loop /tmp
|
|
|
11-13-2016, 09:33 AM
|
#30
|
Senior Member
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,820
|
I have no idea why /home isn't showing up in the df output. Swap devices aren't mounted like regular filesystems, and thus will never be reported by df.
Again, what happens when you run "df /home" specifically asking about that filesystem?
|
|
|
All times are GMT -5. The time now is 11:01 AM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|