[SOLVED] Advice on replacing boot disk, non-identical partitions

kaza · 12-18-2021, 11:39 AM

Hello!

It had been quite a few years since I had to replace a disk on my PC and
the pair I replaced wasn't the boot disk. Now I have to replace the boot disk,
something I never did before (PC built in 2009) so in addition to googling
I seek some advice to make sure I won't ruin my system.

I have 4 SAS disks controlled by the Adaptec ICP5165BR RAID controller.
The boot disk is a RAID-0 array of two 147GB disks (12 years old)
and two newer disks (2TB each) are also RAID-0 array for most of the data (the old array also
has the /home partition with not so large data).

The boot array "/dev/sda" contains 10GB unallocated space (used to be a partition I no longer
need).

I bought a pair of 600GB SAS disks, connected them, run the ADAPTEC utility during boot
to create a new RAID-1 array and initialized the array so now it's visible as /dev/sdc.

Searching the web I found more than one suggested way. Some suggest a simple

dd if=/dev/sda of=/dev/sdc bs=1M

(after booting from some other source which in my case would be the DVD from which I installed
the Fedora).
Others suggest a more complex way of re-creating a bootable partitions on the new disk,
reinstalling grub2 and copying boot image files.
One of the problems with googling for solutions is they're not necessarily up to date
and not necessarily relevant to my system (PC built in 2009, before UEFI, RAID, running FC32).

I would also like to remove the unallocated 10GB space existing on the old boot array
and to increase the /home (while leaving all other partitions the same size they're now).
And more importantly, I don't think I can just "reconnect the new disk instead of old one"
as suggested after using the "dd if=/dev/sda of=/dev/sdc bs=1M" because, AFAIR, the RAID
controller remembers the IDs of disks.

The contents of my /etc/fstab are:

Code:

/dev/mapper/VolGroup-lv_root /                       ext4    defaults        1 1
UUID=b9823434-e9c3-4aa7-90fd-0048b558a79b /boot                   ext4    defaults        1 2
/dev/mapper/VolGroup-lv_home /home                   ext4    defaults        1 2
/dev/VolGroup/lv_swap        swap                    swap    defaults        0 0
/dev/mapper/VolGroup_graphics-lv_graphics /home/user/GRAPHICS     ext4    defaults        1 2

What should be my process of copying (or re-creating) my boot disk and changing the /home
partition size? At the controller side, I can change the order of the disks (arrays),
the first in the list is the boot disk.

TIA,
kaza.

mrmazda · 12-18-2021, 03:35 PM

Yikes! If it was me with a system as complicated, I'd disconnect everything, then build fresh boot/OS with the new disk RAID as sda if that's where I wanted boot and OS to live. Once that's running and defaults adjusted to preferences as desired, I'd add back the rest, taking as many steps as required to avoid mental or physical boo-boos. I wouldn't try to dd anything involving RAID or LVM. Note though that I've never had a hardware RAID controller, so am completely unschooled in their nuances.

I quit putting OS on RAID when I started using SSDs. I have only /home and other non-system data on RAID any more. Installing an OS or dd'ing an image backup has gotten too quick and simple over the years to worry over something on / getting lost. I never did put /boot or swap on RAID.

Brains · 12-18-2021, 04:28 PM

Quote:

the RAID controller remembers the IDs of disks

I've never used this version of this application, but I did go through parts of the manual and it appears it can copy (mirror) your RAID 0 boot disk to the RAID 1 disk. Once booted, then use LVM tools to expand/alter the LVM structure ie: expand /home.

Quote:

Image for Linux works seamlessly with on-board hardware-RAID

You can use this application for 30 days trial, full feature.

syg00 · 12-18-2021, 04:53 PM

Caveat: never used a hardware RAID configuration. Been times I wish I had ...

As far as Linux is concerned they are just (real) disks. Being able to change the boot disk easily is a boon. I never us dd except for forensics - dd'ing an LVM setup is just asking for problems with UUIDs and duplicate names.
I would simply create a new boot partition and copy that over. Then create a pv on the rest of /dev/sdc and add that to the vg and use LVM to move the data over, then re-install grub. After that you should be able to test boot /dev/sdc. That way you use facilities you already have, and transparently get to use all the new space on /dev/sdc - which might become /dev/sda after being set as the "first" disk ??.
What are your plans for the current /dev/sda in future - LVM or "normal" partitions ?.

Brains · 12-18-2021, 08:45 PM

Well
I've been using Image for UEFI to trim VMWare snapshot data from my virtual machines. By taking an image of the entire drive which only captures OS data, no tmp stuff or unused blocks. Then create a new virtual machine with same drive size, boot Image for UEFI and apply the image, most will boot as if they were born there, some need grub reinstalled. But it cannot do LVM so I could not trim Fedora, CentOS default installations, I was able to trim over 3 GB off Ubuntu once.

I finally tried this Image for Linux. Since I use VMWare in Windows I clicked on the makedisk.exe and created an ISO on the data drive and booted it in CentOS VM with LVM and a USB key attached. Created a backup image of the drive and transferred it to a new CentOS VM drive and it booted as if it was born there. It took 10GB of it's 30GB drive, the image file was 6 GB.

Creating a backup image or copying to another drive is the same process except you do part of it manually, typically this software can do all this stuff over networks also.

Brains · 12-19-2021, 01:09 AM

So I did another trial with CentOS.
Should of mentioned I'm using the GUI version of IFL.
Using same steps as before I created a backup image of CentOS on a 30 GB drive but restored it to a 100GB drive this time.
Again it booted no issues and this is what showed:

Quote:

[root@localhost jo]# fdisk -l
Disk /dev/sda: 100 GiB, 107374182400 bytes, 209715200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 84A767EB-EFA4-4981-A801-557907281054

Device Start End Sectors Size Type
/dev/sda1 2048 1230847 1228800 600M EFI System
/dev/sda2 1230848 3327999 2097152 1G Linux filesystem
/dev/sda3 3328000 62912511 59584512 28.4G Linux LVM

Disk /dev/mapper/cs-root: 25.4 GiB, 27283947520 bytes, 53288960 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mapper/cs-swap: 3 GiB, 3221225472 bytes, 6291456 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

[root@localhost jo]# lvdisplay
--- Logical volume ---
LV Path /dev/cs/swap
LV Name swap
VG Name cs
LV UUID 3I4A82-vfwU-Xqs2-WI1c-89co-ldPM-I8VKyX
LV Write Access read/write
LV Creation host, time localhost.localdomain, 2021-10-20 18:00:04 -0600
LV Status available
# open 2
LV Size 3.00 GiB
Current LE 768
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 8192
Block device 253:1

--- Logical volume ---
LV Path /dev/cs/root
LV Name root
VG Name cs
LV UUID 6rWcZ5-TZKa-yxxz-SL2X-p9Gw-VijK-SRdhD6
LV Write Access read/write
LV Creation host, time localhost.localdomain, 2021-10-20 18:00:05 -0600
LV Status available
# open 1
LV Size 25.41 GiB
Current LE 6505
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 8192
Block device 253:0

I then booted an ISO of Gparted live and expanded /dev/sda3 (/dev/sda3 3328000 62912511 59584512 28.4G Linux LVM) to use up the free space and this is what I got upon reboot:

Quote:

[root@localhost jo]# fdisk -l
Disk /dev/sda: 100 GiB, 107374182400 bytes, 209715200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 84A767EB-EFA4-4981-A801-557907281054

Device Start End Sectors Size Type
/dev/sda1 2048 1230847 1228800 600M EFI System
/dev/sda2 1230848 3327999 2097152 1G Linux filesystem
/dev/sda3 3328000 209713151 206385152 98.4G Linux LVM

Disk /dev/mapper/cs-root: 25.4 GiB, 27283947520 bytes, 53288960 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mapper/cs-swap: 3 GiB, 3221225472 bytes, 6291456 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

I then ran commands:

Quote:

[root@localhost jo]# pvresize /dev/sda3
Physical volume "/dev/sda3" changed
1 physical volume(s) resized or updated / 0 physical volume(s) not resized

[root@localhost jo]# vgdisplay
--- Volume group ---
VG Name cs
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 4
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 98.41 GiB
PE Size 4.00 MiB
Total PE 25193
Alloc PE / Size 7273 / 28.41 GiB
Free PE / Size 17920 / 70.00 GiB
VG UUID I34RyM-2JJl-HnSZ-iOG4-FxjL-uMxd-ldc5pI

[root@localhost jo]# lvextend -r -l +100%FREE /dev/cs/root
Size of logical volume cs/root changed from 25.41 GiB (6505 extents) to 95.41 GiB (24425 extents).
Logical volume cs/root successfully resized.
meta-data=/dev/mapper/cs-root isize=512 agcount=4, agsize=1665280 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=0
= reflink=1 bigtime=0 inobtcount=0
data = bsize=4096 blocks=6661120, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=3252, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
data blocks changed from 6661120 to 25011200

[root@localhost jo]# vgdisplay
--- Volume group ---
VG Name cs
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 6
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 98.41 GiB
PE Size 4.00 MiB
Total PE 25193
Alloc PE / Size 25193 / 98.41 GiB
Free PE / Size 0 / 0
VG UUID I34RyM-2JJl-HnSZ-iOG4-FxjL-uMxd-ldc5pI

kaza · 12-19-2021, 12:47 PM

Thanks for all the replies!

I'll re-read them again and probably wait to see if more replies will come in.
I see that my "gut feeling" that using "dd" seems to not be suitable for my system was correct
so I'll have to build the bootable disk from the beginning. Since my mainboard had been bought in 2009,
it's before the UEFI era, just BIOS so it seems the UEFI tools are not for me.
Are there any more tools like Image For Linux (it's OK if they're just CLI)?
I see I'll have to read more about creating a bootable HDD (google results have tons of "bootable USB drive creation"
but very little about HDDs).

Regarding "What are your plans for the current /dev/sda in future - LVM or "normal" partitions ?.":
do you mean what use I plan for the old disk which is currently /dev/sda or how would I like
to arrange the new disk which'll become /dev/sda?

Since the old disk is 12 years old and I'm beginning to doubt its reliability, my plans for it
are not involving it being in a PC (maybe I'll try showing my daughter how old, mechanical disks are made
inside).

As for the new disk: I would like to preserve exactly my current setup with LVM.

TIA,
kaza.

business_kid · 12-19-2021, 02:02 PM

One of the last jobs I did as an Electronic hardware guru was on a server with Raid 0. I got him going but it could have been an absolute disaster.

Raid 0 has striped layers, so writes are performed thusly: fill the disk cache in disk A, fill the disk cache in disk B while disk A finishes writing, then back to disk A, etc. There's somewhere where all these stripes are catalogued (a third disk in my case).

Positive: It gets a fast read/write from slow disks.
Negative: If a disk goes, you lose the lot.

Review your situation. PCIE speeds have gone up, disk speeds have gone up, SSDs are here. I hear nothing about backups. Are there Backups?

I don't have half enough information to call it, but you only need raid 0 on things where massive amounts of data are being thrown around in time-critical fashion. Somebody has done the sums and your box isn't fast enough. The installation I rescued was driving a laser on a printing plate machine in a large printing company. And if you have raid 0, you need an offline backup or on a different network where you have the most recent copy of data. So when smoke starts rising from your box, you have a copy of the data. If you don't like SSDs, there's some other quite fast and expensive server drives with a fast throughput. The name escapes me.

syg00 · 12-19-2021, 04:04 PM

If you' re going to toss the old disks no point worrying about the orphan space.
If you're worried about their reliability, get your data off - now. LVM was designed for exactly this type of scenario. Like I said, pre=allocate the partitions on current /dev/sdc and move the LVM components over using pvmove. Can be done online. Then remove the pv on the current /dev/sda. You can run that way until you are confident you can handle the swap of the boot disk.

kaza · 12-23-2021, 01:57 PM

Hi again,
thanks for the replies.

To answer the backup questions: there is a tape (HP StorageWorks Ultrium 1760 SAS, LTO-4)
on which I do the backups. Obviously, I'll do a backup before I'll start (had few days
of thunderstorms, PC+UPS+router were most of the time OFF and disconnected, both power and
network, waiting for calm days as power outages might last longer than 20-30 minutes of UPS
capability. Also waiting for a free weekend as 12+ hours working days leave my brain
in a condition that dealing with disks would be "ill advised").

12 years ago, when I built my PC I didn't know exactly what would be the throughput demands so I
decided on RAID 0, this time I set the pair of new disks as RAID 1.

I considered SSDs but their price (of the enterprise class, not the cheap ones) is too high for me
so I decided on HDDs.

Thanks,
kaza.

kaza · 12-25-2021, 03:02 PM

OK, the backup has completed (~25 hours), backup verification is running (ETA: +9 h),
meanwhile I started with the simple task: using GParted created on the /dev/sdc (new RAID) a partition table of
the MSDOS type (the 2009 mainboard is of BIOS type, before the UEFI era), created 1024 MB primary partition,
formatted it with ext4 and set the "boot" flag.

it had been currently mounted as

/dev/sdc1 999320 1284 929224 1% /run/media/user/4bfd7509-3c77-4ebf-ab39-fe6f7bf4ae22

Now, if I understand correctly, I need to move "grub2" onto it to be able to boot from it in the future, when I'll swap order of RAID
arrays in the controller.
Will the command

pvmove /dev/sda1 /dev/sdc1

do what I need? Or was it a mistake to format the partition and the "pvmove" should run on unformatted partition?

TIA,
kaza.

business_kid · 12-26-2021, 03:53 AM

Great that you have a backup.

Now do you really need raid 0 at all? And if you're not regularly using all that data, do you need it on your pc? For most folks (not including me) this is an alcoholiday season; Time to relax with family and mull things over.

kaza · 12-27-2021, 12:56 PM

OK, I attempted to run pvmove and it didn't work:

Code:


<root localhost.localdomain>.../root>df
Filesystem                                 1K-blocks       Used  Available Use% Mounted on
devtmpfs                                     8173400          0    8173400   0% /dev
tmpfs                                        8194952        180    8194772   1% /dev/shm
tmpfs                                        8194952       1540    8193412   1% /run
/dev/mapper/VolGroup-lv_root               102009824   19314864   77470104  20% /
tmpfs                                        8194952          8    8194944   1% /tmp
/dev/sda1                                     999320     235808     694700  26% /boot
/dev/mapper/VolGroup-lv_home               134948080  115753016   12317064  91% /home
/dev/mapper/VolGroup_graphics-lv_graphics 3838422480 1788402848 1854968320  50% /home/user/GRAPHICS
tmpfs                                        1638988         56    1638932   1% /run/user/1000
/dev/sdc1                                     999320       1284     929224   1% /run/media/user/4bfd7509-3c77-4ebf-ab39-fe6f7bf4ae22

<root localhost.localdomain>.../root>pvmove /dev/sda1 /dev/sdc1
  WARNING: PV /dev/sda2 in VG VolGroup is using an old PV header, modify the VG to update.
  Failed to find physical volume "/dev/sda1".
  Run `pvmove --help' for more information.

From some searching I see that I should update PV metadata so I run:

Code:

<root localhost.localdomain>.../root>vgck --updatemetadata VolGroup
  WARNING: PV /dev/sda2 in VG VolGroup is using an old PV header, modify the VG to update.
  WARNING: updating PV header on /dev/sda2 for VG VolGroup.

and after it:

Code:

<root localhost.localdomain>.../root>pvmove /dev/sda1 /dev/sdc1
  Failed to find physical volume "/dev/sda1".
  Run `pvmove --help' for more information.

Why "pvmove" can't find the physical volume "/dev/sda1" when I see it on the "df" output?
Should I, instead, use "grub2-install"?

TIA,
kaza.

michaelk · 12-27-2021, 02:21 PM

Code:

/dev/sda1                                     999320     235808     694700  26% /boot
pvmove /dev/sda1 /dev/sdc1

/dev/sda1 is a regular partition and not a LVM PV. To migrate the /boot partition cp or rsync is sufficient.

kaza · 12-28-2021, 03:17 PM

Thanks, michaelk.

So, to make the things clearer to me: the fact that I set the "boot" flag on the /dev/sdc1 partition using GParted has created the boot code in the MBR
and just by copying into /dev/sdc1 all the files from /dev/sda1 will make the boot process successfull (after I'll switch the order of disks in the
RAID controller so what's currently /dev/sdc will become /dev/sda)?

When copying files using a simple "cp" the "inodes" on the destination might not be the same as on the source (I think), does the boot code cares about such things
or it's sophisticated enough to find the files just by their names?

TIA,
kaza.