LVM and Resizing with VMWare

jeg1972 · 12-23-2009, 02:11 PM

I wonder if anyone can help me with this LVM resizing problem I have. I've not really used LVM to any great extent before, but I thought that I'd try it out now that we're moving to a VSphere environment.

So, I created a virtual disc with thin provisioning of 20GB and installed Linux on it using 100MB for /boot (on ext3) and a Volume Group taking up the rest of the drive with the logical volumes created within it. Everything OK so far.

Now, what I did next was to increase the size of the disc in VSphere to 30GB and rescanned the drive so the 30GB was available in Linux as you can see below;

Code:

fdisk -l

Disk /dev/sda: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14        2610    20860402+  8e  Linux LVM

Now, what I want to do is increase the size of the Volume Group so that it uses the extra space on /dev/sda, ready to extend one or more of the logical volumes but I have no idea how to do this, as you can see the Volume Group is still the original size, eventhough the device is now larger;

Code:

vgdisplay
  --- Volume group ---
  VG Name               rootvg
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  7
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                6
  Open LV               6
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               19.88 GB
  PE Size               32.00 MB
  Total PE              636
  Alloc PE / Size       636 / 19.88 GB
  Free  PE / Size       0 / 0
  VG UUID               rWAxdp-PEki-WQko-eLK9-OX5M-jRny-QHsDpN

Can anyone enlighten me on how to do this?

Thanks

John

MensaWater · 12-23-2009, 02:43 PM

Review of the lvm man page suggests pvresize.

Review of the pvresize man page suggests doing "pvresize /dev/sda2" will make the physical volume recognize the fdisk (partition) changes.

When working with LVM typing "man lvm" will give you an overview of what each of the commands done. Then typing man on the command (e.g. "man pvresize") will give you details as to what the specific command does.

jeg1972 · 12-23-2009, 03:06 PM

Quote:

Originally Posted by jlightner

Review of the lvm man page suggests pvresize.

Review of the pvresize man page suggests doing "pvresize /dev/sda2" will make the physical volume recognize the fdisk (partition) changes.

When working with LVM typing "man lvm" will give you an overview of what each of the commands done. Then typing man on the command (e.g. "man pvresize") will give you details as to what the specific command does.

Thanks for that, I had seen that and tried it, but this is what I get;

Code:

 pvresize /dev/sda2
  Physical volume "/dev/sda2" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized

That certainly looks positive... but;

Code:

 vgdisplay
  --- Volume group ---
  VG Name               rootvg
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  11
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                6
  Open LV               6
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               19.88 GB
  PE Size               32.00 MB
  Total PE              636
  Alloc PE / Size       636 / 19.88 GB
  Free  PE / Size       0 / 0
  VG UUID               rWAxdp-PEki-WQko-eLK9-OX5M-jRny-QHsDpN

VG Size hasn't changed to take up the extra ~10GB

ArfaSmif · 12-23-2009, 05:57 PM

I think Oracle Enterprise Linux is Based on Redhat Enterprise Linux, in which case you probably have a very useful gui called "system-config-lvm". Have you tried using this?

If you don't have the gui, you will need to use the lvm shell. The command vgextend comes to mind.

This link http://tldp.org/HOWTO/LVM-HOWTO/is the one you need for all of your answers. It will probably have examples of your scenario.

Of course before you do anything like this, backup all of your data and have a contingency plan in place in case anything _bad_ happens. Good luck.

jeg1972 · 12-24-2009, 12:12 AM

Quote:

Originally Posted by ArfaSmif

I think Oracle Enterprise Linux is Based on Redhat Enterprise Linux, in which case you probably have a very useful gui called "system-config-lvm". Have you tried using this?

If you don't have the gui, you will need to use the lvm shell. The command vgextend comes to mind.

This link http://tldp.org/HOWTO/LVM-HOWTO/is the one you need for all of your answers. It will probably have examples of your scenario.

Of course before you do anything like this, backup all of your data and have a contingency plan in place in case anything _bad_ happens. Good luck.

Indeed it is based on RHEL, but the graphical tool doesn't seem to offer anything to resize a Volume Group.

I'm doing this on a scratch VM which I set up to specifically to test this out, so I don't have to worry about losing data... thanks for the reminder though! :-)

I've been looking for days to find a solution to a simple problem, and there really doesn't seem to be anything on this, so I think I must be misunderstanding something... It seems really simple, I set up a Volume Group at 20Gig, created some logical volumes in it, increased the underlying drive size to 30Gig, and now want to extend one of the lvs to take up the remaining space. I thought I would have to increase the VG first, before increasing the size of the LV... but perhaps not?

Any help appreciated.

Thanks

John

jeg1972 · 12-24-2009, 01:39 AM

OK, I've re-read the docs about LVM and there is definitely something not working here.

We know that the OS can see the extra space (highlighted in red);

Code:

Disk /dev/sda: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14        2610    20860402+  8e  Linux LVM

So, rather than increase the VG size, I think I need to increase the size of Physical Volume first, as this is underlying physical storage unit of an LVM logical volume... So, I'll do that;

Code:

pvresize /dev/sda2
  Physical volume "/dev/sda2" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized

This looks promising, it has told me that 1 physical volume has resized. Lets take a look at what it now looks like;

Code:

pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               rootvg
  PV Size               19.89 GB / not usable 19.30 MB
  Allocatable           yes (but full)
  PE Size (KByte)       32768
  Total PE              636
  Free PE               0
  Allocated PE          636
  PV UUID               QA2yD3-U05c-ZIGG-tcdD-ayBJ-sikC-zj7S9e

and there lies the problem. Nothing has changed, the PV Size should be approx 30GB not 20GB, so why can't it resize to the new space available? Is it something to do with the fact that I have 0 Free Physical Extents available?

jeg1972 · 12-24-2009, 02:09 AM

OK, so I deleted one of LVs and then resized the PV

Code:

 pvresize -v /dev/sda2
    Using physical volume(s) on command line
    Archiving volume group "rootvg" metadata (seqno 16).
    No change to size of physical volume /dev/sda2.
    Resizing volume "/dev/sda2" to 41720421 sectors.
    Updating physical volume "/dev/sda2"
    Creating volume group backup "/etc/lvm/backup/rootvg" (seqno 17).
  Physical volume "/dev/sda2" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized

So in this case something has definitely been resized, and now we can see;

Code:

pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               rootvg
  PV Size               19.89 GB / not usable 19.30 MB
  Allocatable           yes
  PE Size (KByte)       32768
  Total PE              636
  Free PE               96
  Allocated PE          540
  PV UUID               QA2yD3-U05c-ZIGG-tcdD-ayBJ-sikC-zj7S9e

Allocatable is now set to 'yes', not 'yes (but full)' as it was and looking at the VG;

Code:

vgdisplay
  --- Volume group ---
  VG Name               rootvg
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  18
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                5
  Open LV               5
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               19.88 GB
  PE Size               32.00 MB
  Total PE              636
  Alloc PE / Size       540 / 16.88 GB
  Free  PE / Size       96 / 3.00 GB
  VG UUID               rWAxdp-PEki-WQko-eLK9-OX5M-jRny-QHsDpN

The only space is available is the space I've made available by deleting the LV. It's driving me mad! There is an extra 10GB available on the device, but LVM can't see it!

syg00 · 12-24-2009, 03:07 AM

There have been murmurings in the past about vgresize - doesn't seem to be in the "standard" LVM2 command set.
Set the PV back and create a new PV and add that to the VG. Then all should come out in the wash.

(not a LVM user BTW - for a truckload of reasons - just musing aloud).

ongte · 12-24-2009, 03:33 AM

syg00 is right, you don't normally resize PVs, instead, you should use fdisk to create another partition (sda3) taking up the extra 10Gig and changed it's partition ID to 8e (for LVM).

Then you run:

Code:

# pvcreate /dev/sda3
# vgextend rootvg /dev/sda3

To extend the LV:

Code:

# lvextend -L +10G /dev/rootvg/rootlv

Finally resize the filesystem in the LV:

Code:

# resize2fs -p /dev/rootvg/rootlv

jeg1972 · 12-24-2009, 03:40 AM

Quote:

Originally Posted by syg00

There have been murmurings in the past about vgresize - doesn't seem to be in the "standard" LVM2 command set.
Set the PV back and create a new PV and add that to the VG. Then all should come out in the wash.

(not a LVM user BTW - for a truckload of reasons - just musing aloud).

It looks like you're right about vgresize not being in LVM2 (http://sources.redhat.com/cgi-bin/cv...n&cvsroot=lvm2)

What do you mean about "Set the PV back and create a new PV and add that to the VG"? Do you mean set the size of the disc back to 20GB in VSphere, rescan the device in Linux so it can only see 20GB and not 30GB and then create a new 10GB Virtual Disc in VSphere which will appear that as a new device in Linux and add that to the VG?

jeg1972 · 12-24-2009, 06:49 AM

Quote:

Originally Posted by ongte

syg00 is right, you don't normally resize PVs, instead, you should use fdisk to create another partition (sda3) taking up the extra 10Gig and changed it's partition ID to 8e (for LVM).

Then you run:

Code:

# pvcreate /dev/sda3
# vgextend rootvg /dev/sda3

To extend the LV:

Code:

# lvextend -L +10G /dev/rootvg/rootlv

Finally resize the filesystem in the LV:

Code:

# resize2fs -p /dev/rootvg/rootlv

Thanks, I'll have a go at this... but this leaves a question regarding how useful thin provisioning is in VSphere. It's supposed to allow you to resize the disc on the fly, I could just create another another virtual disc and mount that... one for the VMWare forum I guess. Anyway, thanks for all of your help everyone!

Cheers

John

twk · 01-01-2010, 09:11 PM

I think once you run pvresize, you will need to reboot and server will pick up the new PV size.

However, normally on a VM (hosted by ESX), I simply add another disk to the VM - run rescan-scsi-bus.sh so it's visible to the VM, make it a PV then add it to the VG (vgextend). Finally I resize the LV (lvextend/resize2fs - RHEL5). This way there's no reboot involved.

gbroadbent · 02-19-2019, 10:35 AM

You CAN resize a vmware volume (this is for RHEL 7)

1. change the size of the volume using vsphere client

2. find the device name in /sys/class/scsi_disk (usually 0:0:0:0 is /dev/sda, 0:0:1:0 is /dev/sdb, etc.)

3. echo "1" > /sys/class/scsi_disk/0\:0\:x\:0/device/rescan
for ubuntu use echo "- - -" > /sys/class/scsi_host/host#/scan (find host # with ls)

4. parted /dev/sdx
(parted) resizepart 1 322GB
(parted) p
Model: VMware Virtual disk (scsi)
Disk /dev/sdd: 322GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number Start End Size Type File system Flags
1 1049kB 322GB 322GB primary

(parted) quit
Information: You may need to update /etc/fstab.

5. lvextend -L+xG /dev/(lvm logical volume)

syg00 · 02-19-2019, 07:40 PM

Yes, a lot of things have changed in the intervening 8 or 9 years since the last post in this thread. I suspect the OP no longer cares.
However LVM has added a good deal of functionality in the last couple of years - even I champion its use these days ...

MensaWater · 02-20-2019, 10:21 AM

Quote:

Originally Posted by syg00

even I champion its use these days ...

I fell in love with LVM when I first encountered it on HP-UX back in the mid 90s after I understood the concepts. My initial reaction was negative but only because I hadn't grok'ked the beauty of LVM yet.

I hated Sun's Solstice Disk Suite and thought Veritas Volume Manager (VxVM) was a bit annoying though better than Solstice. Later I used VxVM for some layouts on HP-UX as well as LVM.

When I began working more on Linux around 2004 I was happy to find LVM there and it has gotten better over time especially since the advent of LVM2. I'm always mystified by folks that continue to use partitioning even for single drive systems. The flexibility of LVM beats the pants off partitioning.