raid-5 simulators?

eantoranz · 05-07-2005, 10:42 AM

I little step forward!

Thanks for that piece of information, Ekectro.

This is losetup output:

Code:

root@ttyp3[knoppix]# losetup -a
/dev/loop0: [1641]:11 (/mnt/hdd1/discoa.dat) offset 0, no encryption
/dev/loop1: [1641]:12 (/mnt/hdd1/discob.dat) offset 0, no encryption
/dev/loop2: [1641]:13 (/mnt/hdd1/discoc.dat) offset 0, no encryption
/dev/loop3: [1641]:14 (/mnt/hdd1/discod.dat) offset 0, no encryption

I have set up all 4 images.... but now I get messages about wrong raid superblock magic. Perhaps it's because I'm not ordering the disks in the array the right way?

dmesg:

Code:

 [events: 00000000]
md: invalid raid superblock magic on [dev 07:00]
md: [dev 07:00] has invalid sb, not importing!
md: could not import [dev 07:00]!
md: autostart [dev 07:00] failed!
 [events: 00000000]
md: invalid raid superblock magic on [dev 07:01]
md: [dev 07:01] has invalid sb, not importing!
md: could not import [dev 07:01]!
md: autostart [dev 07:01] failed!
 [events: 00000000]
md: invalid raid superblock magic on [dev 07:02]
md: [dev 07:02] has invalid sb, not importing!
md: could not import [dev 07:02]!
md: autostart [dev 07:02] failed!
 [events: 00000000]
md: invalid raid superblock magic on [dev 07:03]
md: [dev 07:03] has invalid sb, not importing!
md: could not import [dev 07:03]!
md: autostart [dev 07:03] failed!

Any ideas here? Maybe there's a way to figure out the order of the images in the array by reading certain binary data from them.

eantoranz · 05-07-2005, 12:18 PM

I have being taking a look at the separate devices with mdadm. It says there is no dm superblock in any of the images.

So I decided to take a look with mdadm with a very simple software raid 5 I started from scratch. When I query the separate devices, it says they are not md devices either. What's going on?

Code:

md: marking sb clean...
md: md1 stopped.
md: unbind<[dev 07:06],2>
md: export_rdev([dev 07:06])
md: unbind<[dev 07:05],1>
md: export_rdev([dev 07:05])
md: unbind<[dev 07:04],0>
md: export_rdev([dev 07:04])
 [events: 00000000]
md: invalid raid superblock magic on [dev 07:04]
md: [dev 07:04] has invalid sb, not importing!
md: could not import [dev 07:04]!
md: autostart [dev 07:04] failed!
 [events: 00000000]
md: invalid raid superblock magic on [dev 07:05]
md: [dev 07:05] has invalid sb, not importing!
md: could not import [dev 07:05]!
md: autostart [dev 07:05] failed!
 [events: 00000000]
md: invalid raid superblock magic on [dev 07:06]
md: [dev 07:06] has invalid sb, not importing!
md: could not import [dev 07:06]!
md: autostart [dev 07:06] failed!

eantoranz · 05-07-2005, 04:41 PM

For your own confort, I have prepared a script so you can do what I do here in a single step.

It will create three 10mb files that will be the images of a brand new raid (disk0.dat, disk1.dat and disk2.dat) in the local directory.

It will "loop" them to /dev/loop0, 1 and 2.

It will create a config file in the local directory.

Will make the raid device (/dev/md0) with the local config file the script just created.

Will format it (ext3)

Will mount it on /mnt/tmp

Will copy there the config file

Will umount the raid, stop it, start it again, cat the file from there, umount, stop the raid again, and finally remove the loop devices.

The problem is that it always fails when reatsrting the raid after having stoped it the first time. Why is that?

script:

Code:

echo creation of the images and the loop devices
for i in 0 1 2; do
        echo Image $i
        dd if=/dev/zero count=10 bs=1M of=disk$i.dat
        losetup /dev/loop$i $PWD/disk$i.dat
done;

echo creation of the conf file
echo raiddev /dev/md0 > raidtab
echo $'\t'raid-level$'\t\t'5 >> raidtab
echo $'\t'nr-raid-disks$'\t\t'3 >> raidtab
echo $'\t'nr-spare-disks$'\t\t'0 >> raidtab
echo $'\t'persistent-superblock$'\t'0 >> raidtab
echo $'\t'chunk-size$'\t'64 >> raidtab
echo >> raidtab
for i in 0 1 2; do
        echo $'\t'device$'\t\t'/dev/loop$i >> raidtab
        echo $'\t'raid-disk$'\t\t'$i >> raidtab
done
echo building the raid device
mkraid -c raidtab /dev/md0 && \
echo formatting to ext3 &&
mkfs.ext3 /dev/md0 && \
echo mounting && \
mount /dev/md0 /mnt/tmp && \
echo copying a file to the raid && \
cp raidtab /mnt/tmp/ && \
echo output of the file: && \
cat /mnt/tmp/raidtab && \
echo umounting &&
umount /mnt/tmp && \
echo shuting down the raid && \
raidstop /dev/md0 && \
echo starting the raid again && \
raidstart -c raidtab /dev/md0 && \
echo mounting again && \
mount /dev/md0 /mnt/tmp && \
echo show the file inside && \
echo the raidtab is && \
cat /mnt/tmp/raidtab && \
echo "Great! We could do it!"


echo
echo one way or the other, bring the raid down
raidstop /dev/md0
echo .... and remove the loop devices
for i in 0 1 2; do
        echo device $i
        losetup -d /dev/loop$i
done;

What is my mistake?

Electro · 05-07-2005, 06:53 PM

A few months ago, I have used a loop device to make a FAT filesystem with those instructions on the site that I posted. You have to specify "-o" option in order to make it feel like it is a disk. If you do not, you will have an error. I have never seen an "-a" option for losetup.

The script will not work. When using dd that way, you have to use "dd if=/dev/zero of=disk1.dat bs=516096 count=21". This syntax just make the file be almost near hard drive specs. After it is done, you have to use losetup to bring up the loop device and make a partition. Then specify partition type like RAID. Next detach the loop device. Then re-connect the loop device but specify the -o with value of 32256. Now you can setup it up the partition on the loop device in RAID. Yes, you have to always use -o32256 when bring up the loop devices for hard drive disk images. Again, the MBR information starts on the front of the image from 0 to 32256.

eantoranz · 05-07-2005, 07:14 PM

losetup -a shows the devices already configured.

eantoranz · 05-07-2005, 07:19 PM

What is the effect of the -o option... I already read what it does.... but I mean from the stand point of a "theoretical" HD device

eantoranz · 05-07-2005, 07:35 PM

Oh, man! You got me SOOOOOO close to solving this problem.

I tried the -o option with the images of the RAID I'm trying to get the data from. It just complained about ONE superblock!

This is the output of dmesg:

Code:

md: invalid raid superblock magic on loop0
md: loop0 has invalid sb, not importing!
md: could not import loop0!
md: autostart unknown-block(0,1792) failed!

What a difference from when I started. Any idea how to get rid of THIS single error message?

eantoranz · 05-07-2005, 07:44 PM

I'm taking a look at byte 21 (15h) of the images. What sould they be?

I have them like this:
Disk 1: 83h
Disk 2: A7h
Disk 3: B9h
Disk 4: ECh

But I think Disk 1 is wrong. I guess it's F2h. What do you think?

eantoranz · 05-08-2005, 07:36 AM

I can be so stupid sometimes..... I was forgetting about the slightest of details in my script:

/dev/md0 is not a partition I can mount.... but a HD. Oops!

I'll have to correct it!

eantoranz · 05-08-2005, 07:45 AM

Quote:

Originally posted by Electro
Again, the MBR information starts on the front of the image from 0 to 32256.

Man.... you sure it's 32256? Maybe it's 32768 (2^15), right?

eantoranz · 05-08-2005, 07:57 AM

But hold on a second. In the software raid tutorial, they treat raids like partitions.... not HD.

This is driving me crazy right now.

Matir · 05-08-2005, 10:25 AM

You can, and should, mount /dev/md0 as a raid device.

eantoranz · 05-08-2005, 11:26 AM

I know... but when I try to restart the RAID, I can't. :'(

modification of the script (remommended by Electro):

Code:

for i in 0 1 2; do
        echo Image $i
        dd if=/dev/zero count=10 bs=1M of=disk$i.dat
        losetup -o 32768 /dev/loop$i $PWD/disk$i.dat
done;

Give it a shot and tell me if it works for you, please!

Matir · 05-08-2005, 12:08 PM

That would completely delete the first 10 megabytes of each of your .dat files! (Well, technically, it would zero it)

What was this supposed to accomplish?

eantoranz · 05-08-2005, 01:54 PM

I know.... it's a brand new test everytime you run the script. That's why I dd in the very beginning, to make sure I start from scratch.