LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware
User Name
Password
Linux - Hardware This forum is for Hardware issues.
Having trouble installing a piece of hardware? Want to know if that peripheral is compatible with Linux?

Notices


Reply
  Search this Thread
Old 08-29-2020, 11:28 AM   #1
dezix
Member
 
Registered: Sep 2017
Location: Frog's Land
Distribution: Debian
Posts: 98

Rep: Reputation: Disabled
LVM2 : Extending underlying PV's partition


Hi!

On a system (Debian) freshly installed on a msdos PT disk with full single ext4 part sda1 (no having other choice),
I start resizing sda1 to 2G and then creating with fdisk a primary sda2 on the free space.

Assuming fdisk creates sda2 with Linux type (83) by default,
then
I create a PV on sda2 (without creating ext4 fs on sda2 first),
after what,
I create a VG with a full set of LV's full-fitting my needs with ext4 fs on them.

At this point, I move system files toward LV's, excepted /boot content moved on sda1.

fstab is edited to mount sda1 on /boot and LV's where they have to.

grub-install in a chroot

reboot ⇒ works fine !

Now as /boot doesn't need more than 200M, sda1 is reduced to that size.

This freed (1.8G) space I want to add it to my PV on sda2
(to avoid a miss-numbered and unnecessary sda3 between sda1 & sda2).

First I've done the job on a training VBox with GParted in a Live Session => it works !

But in the true life (not VBox) I can't use GParted so I have to do it only with commands.

GParted before to move/resize a part used as PV,
needs "desactivate" it in part's options then there is no difficulty to resize as usual.

Seems that is achieved with :

lvchange -a n vg0

I've tried that way it works following with GParted

I've tried vgchange -a n vg0 too, getting an issue with GParted

GParted doesn't report detailed commands about how it extends/moves sda2
the only I can see, is that it spends time copying/moving datas.

GParted reports only the following commands at the end of its process :

Quote:
lvm pvck -v '/dev/sda2'

lvm pvck -v '/dev/sda2'

lvm pvresize -v '/dev/sda2'


So I've tried :

Code:
lvchange -a n vg0

parted /dev/sda resizepart 2 100%

lvm pvck -v '/dev/sda2' 

lvm pvresize -v '/dev/sda2'

but it doesn't change nothing at all !

So, should be kind if someone could show me where I'm wrong with that commands, thanks

Last edited by dezix; 08-29-2020 at 12:18 PM.
 
Old 08-30-2020, 01:47 AM   #2
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
There are a few things that confuse me. You say that you can't use Gparted, then you talk about Gparted at great length.
The sequence of tasks is creating a 2GB sda1, then sda2 with the remaining space, then shrinking sda1 and attempting to merge the free space with sda2. Why? Why not simply creating the right sizes from the beginning?
You say that vgchange creates an issue with Gparted. Which issue?

While I am not certain, I believe that the free space you want to add to sda2 is before the beginning of sda2, and sda2 occupies space at the end of sda. This would explain why resizepart doesn't change anything, since it only moves the end of the partition.
Also, I don't think you can non-destructively add space to the beginning of a PV, only to its end, but perhaps Gparted has a trick to achieve this.

Last edited by berndbausch; 08-30-2020 at 01:48 AM.
 
Old 08-30-2020, 02:10 AM   #3
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,340

Rep: Reputation: 4177Reputation: 4177Reputation: 4177Reputation: 4177Reputation: 4177Reputation: 4177Reputation: 4177Reputation: 4177Reputation: 4177Reputation: 4177Reputation: 4177
And why do you care about adding an extra pv before the current pv. LVM won't care, and neither should you - LVM is designed to adstract these sort of concerns away.
 
Old 08-30-2020, 07:02 AM   #4
dezix
Member
 
Registered: Sep 2017
Location: Frog's Land
Distribution: Debian
Posts: 98

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by berndbausch View Post
You say that you can't use Gparted, then you talk about Gparted at great length.
Sorry for poor explanations: For tests, I'm using GParted in a live-session of SystemRescueCD on a VBox.
First just to see if GParted was able to do what I expected,
and now because I was hoping to know from the actions report what commands GParted uses to do it.

But the end goal is to use that commands in a script I will use over ssh to setup storage on server whithout X.



Quote:
Originally Posted by berndbausch View Post
The sequence of tasks is creating a 2GB sda1, then sda2 with the remaining space, then shrinking sda1 and attempting to merge the free space with sda2. Why? Why not simply creating the right sizes from the beginning?
Because in my first real situation I can't do that,
I'm using a VPS which the only install option is from an image of the provider with system on one part over the whole disk.
2GB is the near minimum space for the system.


Quote:
Originally Posted by berndbausch View Post
You say that vgchange creates an issue with Gparted. Which issue?

Code:
GParted 1.1.0
configuration --enable-libparted-dmraid --enable-online-resize
libparted 3.3
========================================
Device:/dev/sdaModel:ATA VBOX HARDDISKSerial:VB398f5397-a2fa431eSector size:512Total sectors:41943040 Heads:255Sectors/track:2Cylinders:82241 
Partition table:msdos PartitionTypeStartEndFlagsPartition NameFile SystemLabelMount Point/dev/sda1Primary2048411647

ext4

/dev/sda2Primary409804841943039

lvm2 pv
vg0
========================================
Move /dev/sda2 to the left and grow it from 18.05 GiB to 19.80 GiB  00:37:25    ( ERROR )      
calibrate /dev/sda2  00:00:00    ( SUCCESS )     path: /dev/sda2 (partition)
start: 4098048
end: 41943039
size: 37844992 (18.05 GiB) check file system on /dev/sda2 for errors and (if possible) fix them  00:00:01    ( SUCCESS )
lvm pvck -v '/dev/sda2'  00:00:01    ( SUCCESS )
Found label on /dev/sda2, sector 1, type=LVM2 001
  Found text metadata area: offset=4096, size=1044480
    Found LVM2 metadata record at offset=47616, size=4096, offset2=0 size2=0
    Found LVM2 metadata record at offset=43520, size=4096, offset2=0 size2=0
    Found LVM2 metadata record at offset=39424, size=4096, offset2=0 size2=0
    Found LVM2 metadata record at offset=35328, size=4096, offset2=0 size2=0
    Found LVM2 metadata record at offset=31232, size=4096, offset2=0 size2=0
    Found LVM2 metadata record at offset=27136, size=4096, offset2=0 size2=0
    Found LVM2 metadata record at offset=23552, size=3584, offset2=0 size2=0
    Found LVM2 metadata record at offset=20480, size=3072, offset2=0 size2=0
    Found LVM2 metadata record at offset=17408, size=3072, offset2=0 size2=0
    Found LVM2 metadata record at offset=14848, size=2560, offset2=0 size2=0
    Found LVM2 metadata record at offset=12288, size=2560, offset2=0 size2=0
    Found LVM2 metadata record at offset=10240, size=2048, offset2=0 size2=0
grow partition from 18.05 GiB to 19.80 GiB  00:00:15    ( SUCCESS )     old start: 4098048
old end: 41943039
old size: 37844992 (18.05 GiB)new start: 411648
new end: 41943039
new size: 41531392 (19.80 GiB) libparted messages    
( INFO )     
Partition(s) 2 on /dev/sda have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use.  
As a result, the old partition(s) will remain in use.  You should reboot now before making further changes. move file system to the left  00:04:28    ( SUCCESS )
using internal algorithm  copy 18.05 GiB  finding optimal block size      
copy 16.00 MiB using a block size of 1.00 MiB  00:00:00    ( SUCCESS )     
16.00 MiB of 16.00 MiB copied0.427894 seconds copy 16.00 MiB using a block size of 2.00 MiB  00:00:01    ( SUCCESS )     
16.00 MiB of 16.00 MiB copied0.446412 seconds copy 16.00 MiB using a block size of 4.00 MiB  00:00:00    ( SUCCESS )     
16.00 MiB of 16.00 MiB copied0.464252 seconds copy 16.00 MiB using a block size of 8.00 MiB  00:00:01    ( SUCCESS )     
16.00 MiB of 16.00 MiB copied0.442224 seconds copy 16.00 MiB using a block size of 16.00 MiB  00:00:00    ( SUCCESS )     
16.00 MiB of 16.00 MiB copied0.460796 seconds optimal block size is 1.00 MiB  copy 17.97 GiB using a block size of 1.00 MiB  00:04:26    ( SUCCESS )     
17.97 GiB of 17.97 GiB copied 18.05 GiB (19376635904 B) copied    ( SUCCESS )  shrink partition from 19.80 GiB to 18.05 GiB  00:32:41    ( SUCCESS )     
old start: 411648
old end: 41943039
old size: 41531392 (19.80 GiB)new start: 411648
new end: 38256639
new size: 37844992 (18.05 GiB) libparted messages    
( INFO )     
Partition(s) 2 on /dev/sda have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use.  
As a result, the old partition(s) will remain in use.  You should reboot now before making further changes. 
check file system on /dev/sda2 for errors and (if possible) fix them  00:00:00    ( ERROR )     
lvm pvck -v '/dev/sda2'  00:00:00    ( ERROR )       Could not find LVM label on /dev/sda2

NEW!

Now, i've tried again with GParted after reboot => it shows sda2 moved "left" with no gap after sda1 and allows successfully to extend sda2 to the end of the disk.

The system reboots and everything is allright!



Quote:
Originally Posted by berndbausch View Post
I believe that the free space you want to add to sda2 is before the beginning of sda2 and sda2 occupies space at the end of sda
Yes, it is.

Quote:
Originally Posted by berndbausch View Post
This would explain why resizepart doesn't change anything, since it only moves the end of the partition
That's my problem




Quote:
Originally Posted by berndbausch View Post
but perhaps Gparted has a trick to achieve this
Yes, it has

Do you know a way to record what a graphical app is doing in the background?
I didn't found any log with that in SystemRescueCD session.



Quote:
Originally Posted by sig00
And why do you care about adding an extra pv before the current pv.
LVM won't care, and neither should you - LVM is designed to adstract these sort of concerns away.
In absolute you're right : why not simply to create a new PV on the gap and extend the VG?
Only because I care to achieve the cleaner as possible... might be silly?
 
Old 08-30-2020, 07:34 AM   #5
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
Quote:
Originally Posted by dezix View Post
Do you know a way to record what a graphical app is doing in the background?
You could watch the process tree spawned by the app, but it's not clear if that can help. You could also trace system calls of the app and its descendants, in particular exec system calls. In general, there is no way to look into a program except with a debugger. All these methods are overkill for your task, I think.

The Gparted error only indicates that it can't update live kernel structures. A reboot is required.

Last edited by berndbausch; 08-30-2020 at 07:35 AM.
 
1 members found this post helpful.
Old 08-30-2020, 01:15 PM   #6
dezix
Member
 
Registered: Sep 2017
Location: Frog's Land
Distribution: Debian
Posts: 98

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by berndbausch View Post
All these methods are overkill for your task, I think.
I agree, it's too much, I'm on a wrong way.

I will go back towards FDISK to extend partition and then try to move datas with DD to leftest sectors and RESIZE2FS to extend the files system to entire partition.

I'm not very sure about the DD command to use, but I will give some try on a VM.

Thanks.
 
Old 08-30-2020, 11:17 PM   #7
computersavvy
Senior Member
 
Registered: Aug 2016
Posts: 3,345

Rep: Reputation: 1486Reputation: 1486Reputation: 1486Reputation: 1486Reputation: 1486Reputation: 1486Reputation: 1486Reputation: 1486Reputation: 1486Reputation: 1486
Quote:
Originally Posted by dezix View Post
I agree, it's too much, I'm on a wrong way.

I will go back towards FDISK to extend partition and then try to move datas with DD to leftest sectors and RESIZE2FS to extend the files system to entire partition.

I'm not very sure about the DD command to use, but I will give some try on a VM.

Thanks.
dd is not the way to go.

Starting from the way you described with sda1 as 2G and sda2 as LVM spanning the rest of the disk.

First boot from live media and use gparted, fdisk, or whatever to reduce the size of sda1 to what you wish.
first reduce the filesystem size to the end size (with resize2fs) then do the partition resize to match.
You might want to do a backup first to be sure everything is still as needed when you are done.

then you can use vgextend to grow sda2 (which you said was LVM and a volume group) to add the free space taken from sda1.

The only part that absolutely must be done while booted from the live media is reducing the size of the /boot filesystem and then sda1. the VG operations can be done either from live media or booted into the normal OS.

Note that changing physical partitions with gparted, fdisk, etc can ONLY be done when the drive is idle and no filesystems mounted. Changing VGs and LVs to increase their size can be done (usually) while the device is online. OTOH, reducing a filesystem, VG, or LV also requires the device be idle (unmounted).

I suggest reading the man pages for resize2fs, vgextend as well as those related to the LV operations needed to resize logical volumes. IIRC each of the VG and LV operations has a test mode that you can use to make sure your commands don't screw something up before actually committing the changes.
 
Old 08-31-2020, 10:42 AM   #8
dezix
Member
 
Registered: Sep 2017
Location: Frog's Land
Distribution: Debian
Posts: 98

Original Poster
Rep: Reputation: Disabled
Thanks to your hints, I was able to find my way.

FDISK + DD + PVRESIZE

Here is the whole record of a test :
Code:
Initial state
°°°°°°°°°°°°°

# pvs
  PV         VG  Fmt  Attr PSize  PFree
  /dev/sda2  vg0 lvm2 a--  18.04g 6.32g

# vgs
  VG  #PV #LV #SN Attr   VSize  VFree
  vg0   1   6   0 wz--n- 18.04g 6.32g

# lvscan
  ACTIVE            '/dev/vg0/lv_racine' [<3.91 GiB] inherit
  ACTIVE            '/dev/vg0/lv_home' [300.00 MiB] inherit
  ACTIVE            '/dev/vg0/lv_var_cache' [400.00 MiB] inherit
  ACTIVE            '/dev/vg0/lv_var_log' [400.00 MiB] inherit
  ACTIVE            '/dev/vg0/lv_tmp' [400.00 MiB] inherit
  ACTIVE            '/dev/vg0/lv_swap' [1.46 GiB] inherit


Desactivation of volumes
°°°°°°°°°°°°°°°°°°°°°°°°
# lvchange -a n vg0


Extending sda2
°°°°°°°°°°°°°°

# fdisk /dev/sda

Command (m for help): p
Disk /dev/sda: 20 GiB, 21474836480 bytes, 41943040 sectors
Disk model: VBOX HARDDISK   
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xa1b183e9

Device     Boot   Start      End  Sectors  Size Id Type
/dev/sda1          2048   411647   409600  200M 83 Linux
/dev/sda2       4098048 41943039 37844992   18G 83 Linux

Command (m for help): v
Remaining 3686400 unallocated 512-byte sectors.

Command (m for help): d
Partition number (1,2, default 2): 

Partition 2 has been deleted.

Command (m for help): n
Partition type
   p   primary (1 primary, 0 extended, 3 free)
   e   extended (container for logical partitions)
Select (default p): 

Using default response p.
Partition number (2-4, default 2): 
First sector (411648-41943039, default 411648): 
Last sector, +/-sectors or +/-size{K,M,G,T,P} (411648-41943039, default 41943039): 

Created a new partition 2 of type 'Linux' and of size 19.8 GiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.



Moving raw datas to the new beginning of sda2
°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°°

# dd if=/dev/sda2 of=/dev/sda2 bs=512 skip=3686400 count=37844992 seek=0 conv=noerror,notrunc,fsync status=progress
19372057088 bytes (19 GB, 18 GiB) copied, 1527 s, 12.7 MB/s
37844992+0 records in
37844992+0 records out
19376635904 bytes (19 GB, 18 GiB) copied, 1528.7 s, 12.7 MB/s




PV is still 18G
°°°°°°°°°°°°°°°°

# pvs
  PV         VG  Fmt  Attr PSize  PFree
  /dev/sda2  vg0 lvm2 a--  18.04g 6.32g

# pvscan
  PV /dev/sda2   VG vg0             lvm2 [18.04 GiB / 6.32 GiB free]
  Total: 1 [18.04 GiB] / in use: 1 [18.04 GiB] / in no VG: 0 [0   ]


Resizing PV to partition size
°°°°°°°°°°°°°°°°°°°°°°°°°°°°°

# pvresize /dev/sda2
  Physical volume "/dev/sda2" changed
  1 physical volume(s) resized or updated / 0 physical volume(s) not resized

# pvscan
  PV /dev/sda2   VG vg0             lvm2 [19.80 GiB / 8.08 GiB free]
  Total: 1 [19.80 GiB] / in use: 1 [19.80 GiB] / in no VG: 0 [0   ]

# vgscan
  Reading volume groups from cache.
  Found volume group "vg0" using metadata type lvm2

# lvscan
  inactive          '/dev/vg0/lv_racine' [<3.91 GiB] inherit
  inactive          '/dev/vg0/lv_home' [300.00 MiB] inherit
  inactive          '/dev/vg0/lv_var_cache' [400.00 MiB] inherit
  inactive          '/dev/vg0/lv_var_log' [400.00 MiB] inherit
  inactive          '/dev/vg0/lv_tmp' [400.00 MiB] inherit
  inactive          '/dev/vg0/lv_swap' [1.46 GiB] inherit


Activation of LV
°°°°°°°°°°°°°°°°
# lvchange -a y vg0

# lvscan
  ACTIVE            '/dev/vg0/lv_racine' [<3.91 GiB] inherit
  ACTIVE            '/dev/vg0/lv_home' [300.00 MiB] inherit
  ACTIVE            '/dev/vg0/lv_var_cache' [400.00 MiB] inherit
  ACTIVE            '/dev/vg0/lv_var_log' [400.00 MiB] inherit
  ACTIVE            '/dev/vg0/lv_tmp' [400.00 MiB] inherit
  ACTIVE            '/dev/vg0/lv_swap' [1.46 GiB] inherit

# reboot

  =>  It works!

DONE!



@computersavvy

About vgextend what I understand (I've readen the man again) :
its purpose is to extend the VG adding a (new) PV for the group.
This would be the way creating a new (sda3) PV in the gap between sda1/sda2, but it is what I try to avoid.

Thanks.
 
Old 08-31-2020, 09:54 PM   #9
computersavvy
Senior Member
 
Registered: Aug 2016
Posts: 3,345

Rep: Reputation: 1486Reputation: 1486Reputation: 1486Reputation: 1486Reputation: 1486Reputation: 1486Reputation: 1486Reputation: 1486Reputation: 1486Reputation: 1486
Quote:
Originally Posted by dezix View Post
Thanks to your hints, I was able to find my way.

About vgextend what I understand (I've readen the man again) :
its purpose is to extend the VG adding a (new) PV for the group.
This would be the way creating a new (sda3) PV in the gap between sda1/sda2, but it is what I try to avoid.

Thanks.
Glad you were able to make it work.

Doing it by adding a pv then using vgextend would have only required resizing sda1, creating a new partition (sda3) in the now free space, then adding that as a new pv and using vgextend to expand the volume group to include the free space.
It would have eliminated the use of dd and its associated risks including what may have happened if your skip argument was off by even one block, as well as eliminating the time required for copying the entire partition bit-by-bit as dd does, and would have also eliminated the pvscan, vgscan, lvscan, and lvchange steps you were required to perform.

The change in the vg would not have even been seen other than administratively until the size of an lv needed to be changed and that would have been totally transparent as LVs are not required to be contiguous in allocation.

All that said, your way worked and without using LVM the new partition may have been orphaned unless you followed the steps you did.
Since there are many ways of doing the same task in linux I can see that your expertise paid off.

Good job!

Last edited by computersavvy; 08-31-2020 at 09:57 PM.
 
1 members found this post helpful.
Old 09-01-2020, 05:12 AM   #10
dezix
Member
 
Registered: Sep 2017
Location: Frog's Land
Distribution: Debian
Posts: 98

Original Poster
Rep: Reputation: Disabled
For information about what moves me to do so.
As described in the 1srt post it is part of the partition setup after OS installation because there is no way to do it during OS install.

Other way I tried successfully, consists in :
  • shrink sda1 to 2G (in the same way)
  • create small (no matter the size) sda2 and sda3 on the end of disk
  • sda3 => PV
  • setup VG/LV
  • move everything where it has to
  • shrink sda1 to 200M for /boot
  • extend sda2 with 1.8G freed by sda1
  • sda2 => PV
  • extend VG with sda2

I'm ending to feel this former way was better than what I found here above ;-))


Quote:
Good job!
Thanks

Last edited by dezix; 09-01-2020 at 05:14 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
lvm2 to non lvm2 L1nuxn00b703 Red Hat 5 12-18-2013 05:32 PM
extending volume group vs extending physical volume which is better ? Gil@LQ Linux - Server 2 08-19-2013 11:13 AM
LVM PV UUID Disappeared After Resizing Underlying Partition intrados Linux - Software 3 11-11-2008 04:12 AM
extending an ext2 fs on an lvm2 lv GameboyHippo Debian 6 09-15-2004 04:42 PM
need help resizing an ext3 partition with an underlying partition... spiroth10 Linux - Software 1 07-30-2004 01:21 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware

All times are GMT -5. The time now is 10:21 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration