Moving my install from single disk to raid 1

Keldek · 10-27-2009, 10:40 AM

If you want a quick run down as to WHY I want to do this, read here

Basically, my ISP could not get my server running stable on a simple raid 1 (or raid 5) so what it came down to was having them install my system on a single disk. I don't exactly like this, main reason being, if the system (or HDD) crashes, I'll end up with another several hours of down time... So here is my proposal:

Please Note: This will have to be accomplished on a live System (full backups!) over ssh as I don't trust my ISP to do things right as described in my post above.

Current Setup:
CentOS 5.3 x86_64
Dual Xeon 5410
16GB RAM
4x 750GB HDDs

/boot - /dev/sda1 - 512MB (real 471MB) ext3
/ - /dev/sda2 - 50GB (real 46GB) ext3
/home - /dev/sda3 - remaining space (real 642GB) ext3

swap - /dev/sdb1 - 32GB
/backup - /dev/sdb2 - remaining space (real 659GB) ext3

/dev/md0 - /dev/sdc + /dev/sdd - raid 1 (NOTE: uses entire disk, no partitions!)

Proposed Solution:

Code:

vgcreate -s 64M vg0 /dev/md0

lvcreate -L 512M -n lvboot vg0
lvcreate -L 50G -n lvroot vg0
lvcreate -L 5G -n lvtmp vg0
lvcreate --extent 100%FREE -n lvhome vg0

mkfs -t ext3 -m 1 /dev/vg0/lvboot
mkfs -t ext3 -m 1 /dev/vg0/lvroot
mkfs -t ext3 -m 1 /dev/vg0/lvtmp
mkfs -t ext3 -m 1 /dev/vg0/lvhome

Now, I'd like to 'init 1' at this stage but I can't, so I won't (possible solutions?? Possible to umount the / partition??)

Assuming I'd have to do this on a fully live system, I'd disable all services that I can

Code:

/etc/init.d/sendmail stop
/etc/init.d/postfix stop
/etc/init.d/saslauthd stop
/etc/init.d/httpd stop
/etc/init.d/mysql stop
/etc/init.d/courier-authlib stop
/etc/init.d/courier-imap stop
/etc/init.d/amavisd stop
/etc/init.d/clamd stop
/etc/init.d/pure-ftpd stop
/etc/init.d/fail2ban stop
/etc/init.d/syslogd stop

Then we copy all of our data from the single partitions to the raid disks

Code:

mkdir /mnt/newroot
mkdir /mnt/newroot/boot
mkdir /mnt/newroot/tmp
mkdir /mnt/newroot/home

Code:

mount /dev/vg0/lvboot /mnt/newroot/boot
mount /dev/vg0/lvroot /mnt/newroot/root
mount /dev/vg0/lvtmp /mnt/newroot/tmp
mount /dev/vg0/lvhome /mnt/newroot/home

(I think I covered everything)

Code:

umount -l /dev/sda1 (/boot)
umount -l /dev/sda3 (/home)
cp -dpRx /* /mnt/newroot/
mount /dev/sda1 /boot
cp -dpRx /boot/* /mnt/newroot/boot/
mount /dev/sda3 /home
cp -dpRx /home/* /mnt/newroot/home/

Once we have everything copied, update /etc/fstab and /etc/mtab to reflect the changes we made:
vi /etc/fstab

Code:

LABEL=/home             /home                   ext3    defaults        0 3
LABEL=/                 /                       ext3    defaults,usrquota,grpquota        1 1
LABEL=/boot             /boot                   ext3    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
LABEL=SWAP-sdb1         swap                    swap    defaults        0 0

To

Code:

/dev/vg0/lvhome   /home  ext3  defaults  0 3
/dev/vg0/lvroot  /  ext3  defaults,usrquota,grpquota  1 1
/dev/vg0/lvboot  /boot  ext3  defaults  1 2
tmpfs  /dev/shm  tmpfs  defaults  0 0
devpts  /dev/pts  devpts  gid=5,mode=620  0 0
sysfs  /sys  sysfs  defaults  0 0
proc  /proc  proc  defaults  0 0
LABEL=SWAP-sdb1  swap  swap  defaults  0 0

Moving on, we make a fail safe in the event this doesn't work out:

Code:

vi /boot/grub/menu.lst

find

Code:

default=0

add after

Code:

fallback=1

Then we find (in the same file):

Code:

title CentOS (2.6.18-164.el5)
        root (hd1,0)
        kernel /vmlinuz-2.6.18-164.el5 ro root=/dev/vg0/vgroot
        initrd /initrd-2.6.18-164.el5.img

Add before:

Code:

title CentOS (2.6.18-164.el5)
        root (hd3,0)
        kernel /vmlinuz-2.6.18-164.el5 ro root=/dev/sda2
        initrd /initrd-2.6.18-164.el5.img

Where (hd3,0) is /dev/sdc. If the system fails to boot to the raid then it'll auto boot to the single disk (/dev/sda)

then update my ramdisk:

Code:

mv /boot/initrd-`uname -r`.img /boot/initrd-`uname -r`.img_bak
mkinitrd /boot/initrd-`uname -r`.img `uname -r`

And now to set up grub...

Code:

grub
> root (hd0,0)
> setup (hd0)

we should see something like this:

Code:

Checking if "/boot/grub/stage1" exists... no
Checking if "/grub/stage1" exists... yes
Checking if "/grub/stage2" exists... yes
Checking if "/grub/e2fs_stage1_5" exists... yes
Running "embed /grub/e2fs_stage1_5 (hd0)"... 15 sectors are embedded.
succeeded
Running "install /grub/stage1 (hd0) (hd0)1+15 p (hd0,0)/grub/stage2 /grub/grub.conf"... succeeded
Done.

Code:

> root (hd3,0)
> setup (hd3)

Again, we should see something like this:

Code:

Checking if "/boot/grub/stage1" exists... no
Checking if "/grub/stage1" exists... yes
Checking if "/grub/stage2" exists... yes
Checking if "/grub/e2fs_stage1_5" exists... yes
Running "embed /grub/e2fs_stage1_5 (hd1)"... 15 sectors are embedded.
succeeded
Running "install /grub/stage1 (hd1) (hd1)1+15 p (hd1,0)/grub/stage2 /grub/grub.conf"... succeeded
Done.

Code:

> quit

From here I think we're ready to reboot, can't see where I missed anything. If all goes well then I should see my volume groups listed in 'df- h'

Now, the big question is... have I missed anything?

Any and all input is greatly appreciated.

jlinkels · 10-28-2009, 08:11 PM

Man... I feel with you. I read thru the first page of your post on the other forum. It must drive you insane if the "support" people only seem to follow a flowchart to answer you. No matter how many times you call...

Anyway, since you are using CentOS, I am not quite familiar with your OS and the tools it provides. What I miss in your sequence is the usual sequence of creating a new array with a missing disk.

Usually you would have two disks in you machine, one running the OS, the other one part of the raid-to-be. You would create the raid from the empty disk, telling the second one is missing. Once you have that incomplete array, you would copy your old one onto the new incomplete array. As a last step you would boot from the incomplete array, nuke the original disk and add it to the array to become complete now.

When I did this, I followed this guide: http://www200.pair.com/mecham/raid/r...aded-etch.html and as a reference this one: http://wiki.yobi.be/wiki/Debian_Soft_Raid

I know it is Debian, but I hope at least the steps taken will help you as a checklist, and maybe provide you with some ideas.

The author of the www200.pair.com page is very careful and takes a number of intermediate testing steps. Switching to single user is not necessary despite of what the author writes. You only have the chance that you miss some data which is added after you start the RAID for the first time, and the copying of the contents of the old disk. Your idea to switch off services during this time is excellent, because it prevents data corruption. But it only need to be the service which might be writing on the disk.

Your step to install grub once on each disk instead of a dd copy action is also mandatory. If you don't do that the RAID driver will complain.

Just for fun I did the entire RAIDing over ssh from my desk instead of going into the server room.

Last but not least, you should try this once on a server which sits on your desk before you do this on your co-located server. Just have the same partition scheme, but of course with the smaller hard disks.

good luck
jlinkels

Keldek · 10-28-2009, 11:19 PM

Hi, thanks for your feedback. Being I had 4 disks already I decided to skip the broken array part, and would be starting with a fully working array. I wonder if this would make things more difficult however. Basically, the idea follows similar suit to those guides you listed but skips a few steps.

Quote:

Originally Posted by jlinkels

Man... I feel with you. I read thru the first page of your post on the other forum. It must drive you insane if the "support" people only seem to follow a flowchart to answer you. No matter how many times you call...

Anyway, since you are using CentOS, I am not quite familiar with your OS and the tools it provides. What I miss in your sequence is the usual sequence of creating a new array with a missing disk.

Usually you would have two disks in you machine, one running the OS, the other one part of the raid-to-be. You would create the raid from the empty disk, telling the second one is missing. Once you have that incomplete array, you would copy your old one onto the new incomplete array. As a last step you would boot from the incomplete array, nuke the original disk and add it to the array to become complete now.

When I did this, I followed this guide: http://www200.pair.com/mecham/raid/r...aded-etch.html and as a reference this one: http://wiki.yobi.be/wiki/Debian_Soft_Raid

I know it is Debian, but I hope at least the steps taken will help you as a checklist, and maybe provide you with some ideas.

The author of the www200.pair.com page is very careful and takes a number of intermediate testing steps. Switching to single user is not necessary despite of what the author writes. You only have the chance that you miss some data which is added after you start the RAID for the first time, and the copying of the contents of the old disk. Your idea to switch off services during this time is excellent, because it prevents data corruption. But it only need to be the service which might be writing on the disk.

Your step to install grub once on each disk instead of a dd copy action is also mandatory. If you don't do that the RAID driver will complain.

Just for fun I did the entire RAIDing over ssh from my desk instead of going into the server room.

Last but not least, you should try this once on a server which sits on your desk before you do this on your co-located server. Just have the same partition scheme, but of course with the smaller hard disks.

good luck
jlinkels