[help] How to boot 2 or more linux OS on one same harddisk with LVM?
 

Hi all,

I have questions need to be advised with solutions.

I have a 60GB hardisk, and try to install CENOS 5.5 RHEL5.5 and Fedora13 on it.

I partitioned the harddisk to 4 primary partitions like below:

Partition MountPoint Type

sda1 /boot ext3
sda2 VGRHEL LVM PV
sda3 VGCENT LVM PV
sda4 VGFEDORA LVM PV


I created 3 Volume Groups as you see up: VGRHEL, VGCENT, VGFEDORA;

under every VG, i created 2 logical volumes ,just like below:

VGRHEL=LVRHEL00(swap) + LVRHEL01(MountPoint: /)

Below VG try to partition them the same as VGRHEL too.

VGCENT=LVCEN00(swap) + LVCEN01(MountPoint: /)

VGFEDORA=LVFEDORA00(swap) + LVFEDORA01(MountPoint: /)


I successed installed RHEL5.5 on VGRHEL, but when i try to use the same way to install CentOS 5.5, i got below error infromation:

"Bootable Partitions cannot be on a logical volume"

I don't why this happened, VGRHEL has logical volume too, but why i can install RHEL on it successfully.

I realy confused on this and need your help, pls let me know the reason and advise the correct solutions.

Thank you so much, if i can fix this issue.

I suggest you do this:
First of all, let all distros use same swap - it saves space and is no problem at all.
Also let all systems use same /boot - as long as you use grub legacy not grub 2 that works fine.

All you need do is:
1) Install grub once in sda1
2) When you have installed a a new system, you will find the kernel and initrd under / or /boot (in my experience it's depending on distro).
Copy kernel & initrd to sda1 and mount it as /boot.
3) Add your new distro to /boot/grub/menu.lst or /boot/grub/grub.conf.
You're done!

(Edit)
Thinking about it, I'm not sure you actually get a kernel & initrd if you don't install grub.
You can easily check that, if so:
1) Each installation installs grub to sda1, mounted on /boot.
2) After each installation copy kernel & initrd from /boot to / (since new installation should be done reformatting sda1 or there is a smal risk for trouble)
Maybe also copy the boot-lines from /boot/grub/menu.lst or /boot/grub/grub.conf, not necessary but can be convenient.
3) After each new installation copy kernel & initrd from the older distros to /boot and add lines to grub.conf / menu.lst.

Thanks Pingu!
I'm really a newbie on linux, in fact i don't know how to let all distros use the same /boot and the same swap.

I created /dev/sda1 separatly and used as /boot, but i'm not sure if the grub will install on this partition or it will be installed on each distros.

could you tell me how to install grub on /dev/sda1 and don't let it install to each distro again?

1. Don't use sda1 during install.
2. You will be asked where you want grub installed, choose root-partition.

Now concerning your LVM-setup: Why 3 volumegroups?
Do you have a specific reason for that, do you know the benefits of LVM? You do say you're a newbie so my guess is you have misunderstood LVM a little - sorry if that's incorrect!

The biggest benefit of LVM is the possibility to resize partitions on-the-fly. With your layout you have limited this possibility since you can not use space across volumegroups.

My suggestion:
Use one volumegroup, then create a bunch of logical volumes:
one swap-partition, 3 partitions for your distros root and one for /home.

Best would be not to use any ext# filesystem, I'd suggest reiserfs - but unfortunately at least CentOS doesn't support that out of the box.
VERY annoying!
The reason for my saying this is that before you resize an ext3-partition it has to be unmounted.
So to change size of root-partition you have to unmount it...
With reiserfs you don't need to unmount - same for at least xfs & jfs, but I have no personal experience of those.

To use reiserfs in CentOS I believe you have to install it using ext3/4, install reiserfs support, format a partition with reiserfs and copy the whole system to it.
Maybe a little too much fuss for you - anyway I recommend sticking to one filesystem not mix! That is, don't use reiserfs for one distro and ext3 /ext4 for another.

Hi Pingu,

I almost understand all you say, but i still confuse on one thing.

As your suggestion:

Use one volumegroup, then create a bunch of logical volumes:
one swap-partition, 3 partitions for your distros root and one for /home.

If i install 3 distros in different LV, how can i let the different distros to use the same /home partition(in fact, here is a lv)? And also how can i control grub use the correct kenerl to start up different distros(here i use a single boot partition /dev/sda),because the linux kenerl version of the distros are different.
I am fussed on this.

I already know how to let all of the distros use the same swap partition(in fact, here is a lv), i can edit the /etc/fstab file in each distros, am i doing this right?
Thanks in advance.

Let's be frank!

Grub1 cannot read a LVM. That is why Red Hat distros put a /boot separately so that Grub can boot the kernel inside.


When a kerenl is booted it has the drive to read/write a LVM.

So to put several Linux in the same PC you need to put either all the their Linux kernel in the same /boot or create a different /boot1, /boot2, /boot3 etc to house different kernels and tell Grub which one is which. Then you pile the rest of every Linux in the LVM by slotting them in different logical volume. You have to do house keeping of which all the logical volumes for each Linux.

Here are the bad news

(1) Not every installer can see a LVM. If an installer sees it then it may has its own idea of how to put the installation inside and mess up your arrangement of LVM.

(2) By not using LVM your life is a lot simpler by putting every Linux in each partition and one swap for all of them. If you go for a gpt partitioning scheme you can put 128 Linux in one hard disk. Linux odes not have a partition limit on a hard disk but all distros support 15 partitions and most allow you to install and boot from up to 60 partitions in a disk with the normal Msdos partition table.

(3) You are wasting your time with LVM unless you are doing a server for multiusers applications.

(4) I would say if it take you 100% effort to boot several Linux in a hard disk it will be 300% to 400% if you use LVM.

Hi Saikee,

You're right, this may be waste time.
I got it, i can let all the distros use the same swap and install distros in different physical partitions.
I finally understand the function of grub and LVM.
Thanks Pingu and Saikee.

Let's enjoy the World Cup, i'm watching the match: France against Uruguay.

Let's see now:
1) You want to use LVM - good choice in my opinion, makes it easy to handle out-of-diskspace situations.
2) You need to boot from a partition outside lvm
3) The question of /home - one for all or one each?

For booting, as explained earlier, install grub to sda1 and copy all your distro's kernel + initrd to sda1.
Then edit /boot/grub/menu.lst or /boot/grub/grub.conf (which you have depends on distro)
Also good to add boot partition to each /etc/fstab (you don't have to do this, system works anyway. It's completely for your own control.)
You will have a partition /dev/sda1 containing several kernels & inits, but they all have different names.
(I have installed hundreds of distros always using one /boot, it never happened that two distros had the same namne for kernel/initrd.)
So in your grub config-file you enter name of distro and what kernel, root, init etc to use.
Here's part of my /boot/grub/menu.lst:
3) /home
Now this is worth thinking about:
* Problem 1: You want to access all your files from every distro.
* Solution 1: Use only one /home partition, same for all distros
* Problem 2: All configuration is stored under the home-directory - but different distros don't use same version of every program.
* Solution 2: Use one /home partition for each distro
.... See what I mean? ...
There are 2 ways to handle this:
Either use one home-partition, then you have to put configuration somewhere else & link it in. (That's what I do today)
Or use several partitions and link all personal directories (like Documents, My Music etc)
OH Yes, important!
How could I forget? Make sure your username has the same uid & gid in all distros!
You're a Linux newbie but an IT Pro, so I think you realize this - users are identified to system not with username but with userid.

Now I started writing this before Saikee replied and you to him.
I don't quite agree with Saikee, LVM is not that hard & it has benefits even for a home user.
Of course it is a little bit easier not to use it, but as soon as you run out of space somewhere you'll understand...
I have installed lots of distros using lvm, as long as the installer supports lvm there has never been a problem.
2) We're not talking about 15+ installations, we're talking about handling diskspace.
3) Well, maybe you'll never really need to reallocate space - todays disks are HUGE! - but then, in my experience you always run out of disk space one day.
Apart from that, it's pretty much fun to fiddle & learn!
4) Completely disagree.
But then Saikee, you & I have different methods of booting. My method is always use one partition for /boot.

PS I'm not a sports fan so not watching World Cup, enjoy & hope your favvo team wins!

As to the /home problem, there is of course a third possibility:
1) Use one /home for each distro. Needn't be big, you don't need a separate partition for it.
2) Create a separate partition where you store your user data. Mount that partition somewhere. Let's call it /usr/local/myhome here.
3) Disciplin needed! Always store everything under /usr/local/myhome, never in homedir!

pingu,

I always admire a Linux user able to make a good use of its commands. You are pretty slick with the LVM and I can see you have done a good job, especially advising the /home for different distros, not forgetting making sure of the same set of the uid and gid etc. As for myself I gave up after I discovered each distro has its own way of assigning the uid and uig and the amount of work to correct everyone seemed not worthwhile. For a single user it is a lot easier to have the same data in its own partition that can be mounted by every operating system.

As for the problem of running out of hard disk space you have already confirmed that the hard disk space isn't an issue nowadays due to the huge hard disks cheaply available. If LVM is just for giving bottomless limit to hard disk space then there are at least other methods to cure the space limitation.

(1) Linux now has very good resizer programs so altering partition size isn't a challenge at all.

(2) Linux can be freely moved around between partitions and between hard disks even with server type distros. One can move a Linux from a Msdos partition to a gpt partition, to different sizes, using different filing systems etc.

(3) If a Linux occupies one partition then the space between the directories are self expanding and self contracting. Only the total size of thepartition needs to be worried. This is much easier to maintain than by splitting it in many small partitions each has a space limitation.

(4) Space relieved by a normal partition can be available to other operating systems sharing the same hard disk. This is more difficult in the case of a LVM.

I do think LVM has its place but Linux has moved on and its importance is no longer as it was before, especially for normal desktop users who could do away with an extra manager layer for the hard disk. To many of them having a LVM does not bring the real benefit but the unwarranted maintenance headaches.

When it comes to booting a user using LVM can only have one boot loader and each Linux must be booted "directly". Without a LVM if a user installing 10 Linux can have 10 boot loaders all able to boot both "directly" as well as "indirectly" (using "chainloader" command in Grub or "other" in Lilo). The freedom, flexibility and excitement can not be compared. As an example I put Grub in a Dos partition with NTLDR and bootmgr. It boots a Dos 4 MS Windows (Win2k, Xp, Vista and Win7) and another 35 Linux.

i'm glad to see the helpful comments between you two, both of you idea do benefit me shot out my problem, though I am not very perfessional on linux, I am expecting to see further of your comments.

Thanks a million!!!

Please don't get me wrong that I am against LVM which is a valuable asset in Linux.

I like to understand what I am doing so sometimes I don't go along with every installer's wish if I think there is a better way for my own arrangement.

Red Hat family distros are perhaps the biggest supporter of LVM and their installer will sell us a LVM every time. On top of that being a server-base distro their installers by default split the Linux into several partitions for easy control of resources and security. This all makes sense with a LVM in a server.

As I run a desktop I prefer simplicity and a single partition installation suits me. If I need more space I can resize the partition, move it to another partition or a bigger hard disk. Unlike a MS Windows which can be moved similarly but with restrictions Linux is the easiest to migrate, move, resize, reduce and expand.

What I have found is that no Linux installer would refuse being installed into a single partition and none of the Red Hat distros would insist on the LVM if the user opts it out. It is all down to the user's choice.

I think users can get a lot out of Linux if they try to understand it. Take Grub for example. We use it as a boot loader in the Linux that provides it. However Grub can be installed on its own to boot nothing but MS systems too.

I did spend time to understand LVM and had it worked in the various ways. Being a desktop user I just couldn't get much out of it. Ditching it I gain possibly more from the advances in the hard disk technology, RAIDs, partitioning improvements and boot loaders for the same amount of my time I put in. My situation would change if I must have a system operational all the time and can't power down it easily to do alterations.

The question of using LVM or not is of course not a question of "right or wrong", it's about what suits you.

Personally I use lvm because I find it so easy to use, easier than raid0 which I also tried.
Also both easier & faster than resizing or moving partitions - with lvm & reiserfs adding space takes less than a minute!
Lvm has 2 downsides though: One thing is, as Saikee mentioned earlier, that Windows can't use lvm. Thus it's more difficult to give space to Windows if needed.
Not a problem for me since I don't use Windows any more.
Second thing is that I believe it's a lot more difficult to rescue data from a broken disk if you use lvm - I haven't actually tried or even researched this, but logically it must be harder.
Regular backup's takes care of that.

And finally: I would definitely not advice a complete newbie to start with lvm, however mr zerocoffee wanted to try it.
Being an IT Pro I think he can handle it!

However Saikee, I'm a bit interrested in this, you wrote earlier in this thread:So how do you handle the uid issue? No matter what distro you run you still need access to same personal files, and then you run into permission problems?

(What I do is I never create my personal user "pingu" during installation, I create it manually with uid 501 (never used by installers) & gid 100 = users after installation. )

pingu,

I put my own personal data on their own partitions and disks so that I can mount them with any operating system. MS Windows have free driver to read write Ext2/3 partitions same as Linux using ntfs-3g to do theirs.

As you are already aware it requires root privilege to mount a partition in Linux so moving files in a root terminal never has any restriction. I seldom do a drag and drop in GUI as I think that is a bad practice and only do it for small files occasionally. I do create my personal data and assign my standard ownership setting which I apply to every distro I install so I rarely need to revise the uid and gid.

I separate my personal information from the operating systems as it is far more important than any of them. To me it is pointless to back up files from an operating system. A data-ony partition is fasterer to run and quicker to backup. Any file I leave inside an operating system partition is dispensable to me.