LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 11-16-2009, 03:31 AM   #1
exvor
Senior Member
 
Registered: Jul 2004
Location: Phoenix, Arizona
Distribution: Gentoo, LFS, Debian,Ubuntu
Posts: 1,537

Rep: Reputation: 87
Post Usb FlashDrive Raid System


I have been working on this for a little while and I finally have the system stable working and booting normally. This posting is really a sort of how to as well as a question. If you are a moderator and you feel this should be somewhere else please move it to the correct place.

<<WARNING>>
I feel I should mention that this is not a very hard project but its also not something a Linux beginner would not want to tackle. It requires learning about mdadm commands and recompiling kernels and manipulating the boot system. It does however give you a pretty responsive system with solid state memory

Anyway let me introduce you to our testbed system named darkfathom. This system is a older pc that is capable of booting from usb.

SYSTEM SPECS
Code:
HARDWARE
------------------------
AMD Duron 1299.936 Mhz
768MB of DDR333 memory 
80gb western digital IDE hard drive (Houses old system and /boot) 
8gb segate IDE drive (Not used for any system stuff used to load recovery)
KT4 MSI MS-6590 Mainboard 
4x 4gb CENTON usb 2.0 flash drives (raid0 /dev/md0 16gb main system drive)
These drives have a lifetime warranty (Well see how long that is exactly :P ) 
Nvidia 6400 

SOFTWARE 
-------------------------
Linux version 2.6.30-gentoo-r6 
gcc version 4.3.2 
gentoo 4.3.2
I made this system after scrounging around in our garage that is full of old computer crap and I am pretty pleased at how well its working out. It does not have a regular monitor but rather uses a 32inch LCD tv as a display via HDMI.

I bought the flash drives at Frys Electronics during a sale and got them for around 10 bucks a piece. It was a little cheaper then that but tax evened it out to about $38.00 US.

Anyway I was curious one day and started looking around the net for info on a usb thumb drive raid array as I figured it would be a cheep experiment to see if it actually worked. Lucky for me someone else had done it and had even gone further and made the primary drive the raid itself. Here is the link to the original document http://analogbit.com/node/13

I thought this was a smashing idea and went to work getting everything installed on the system. This quickly turned from a weekend project into a month long one since school keeps me busy most of the time. I finally however got the system up with X installed and with enough tools to do the conversion. A quick rebuild of the kernel to add some raid stuff and off we went.
You would be surprised how hard it is to get usb raid array to boot. You have to contend with the usb devices not coming up fast enough and Linux just booting too quickly to allow the raid to have enough time to assemble. Like in the original document I decided to put the /boot onto a external IDE drive for convenience.
I followed there document pretty much to the letter but did make some changes and some pitfalls. Ill illustrate them below in case there ever is another person crazy enough to try this as well.

The tools I used was a Systemrescue disk and mdadm.

In the tutorial they talk how you should setup your partitions on the usb keys but they leave out a step that may confuse some people. After you create the raid using mdadm you need to make partition(s) in the /dev/md0 device you can do this by using cfdisk /dev/md0.


Also in the tutorial they say to do
Code:
# tune2fs -o  journal_data_writeback /dev/md0
I did this the first time and the raid became unstable and halfway though coping over data I got errors saying that the destination file system was readonly. After further investigating I found I had lost contact with the usb drives and had to reboot. The second time though I did not use this option and was able to git it to complete.

Some things to watch out for is if you decide to make a larger raid later you will have to make sure md0 does not get created or you will get device in use errors when trying to assemble the new raid. I found this out the hard way after being confused why my /dev/sdd drive was in use. I had created a smaller 2 drive raid on my netbook to test out some things before hand. This was even after repartitioning the drives. The raid info gets saved somewhere else probably the MBR of each drive.



Kernel options- This is another place where I found that the instructions lacked some as I was unable to get the system to boot the way they say. the options I used to get it working are below

Code:
kernel /boot/usbraid root=/dev/md0 usb-storage.delay_use=1 rootdelay=10 rootfstype=ext3
Some of the options I did not use was rootflags=data=writeback and I added rootdelay=10. The value 10 here may be a bit too long but I find it better safe then sorry. With these options the raid array gets mounted and created successfully.

All and all the amount of work I put into it was moderate and I am more then pleased with the result. The system is a bit slower when accessing information but not so much as you would think. It is definitely faster then a single usb thumb drive system.

Some questions for you guys are.
====================================
1. What do you think I could do besides add devices to the raid to make it work even faster?
NOTE: The drives are currently on 2 different USB controllers on the Mobo 2 drives on each. At least I think they are :P

2. What types of torture tests you want me to run? Here is your chance to smoke a system and I am more then willing to try and break the box.

3. Is there anything you think I left out here that would be helpful if others attempted to do this same project?

4. Any other thoughts or suggestions?

Last edited by exvor; 11-16-2009 at 03:51 AM. Reason: Even more grammer (some formating)
 
Old 11-16-2009, 11:40 AM   #2
akolff
Member
 
Registered: Jan 2002
Location: Milwaukee
Distribution: Ubuntu Server 12.04.1 LTS & Mint 17
Posts: 38

Rep: Reputation: 4
Hi,
This may be a real boost in extremely harsh environments that are abrasive. It will be interesting to see your results.

Use what ever test you prefer (One that uses prime numbers can really heat up the CPU.) Run the test a few times and see if there are drive failures or data loss.

After running a few regular tests and finding the system reliable, I would suggest hooking up a timer and plugging the power strip into it or just yank the plug.

Nothing like a good old power outage to cause data corruption/loss.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
usb flashdrive help l3it Red Hat 1 07-19-2009 04:16 AM
usb HD vs. usb flashdrive? fstab entry? bioe007 Linux - Hardware 11 05-07-2007 03:52 PM
How to format USB flashdrive tux_addict Slackware 4 11-13-2006 10:04 PM
USB FlashDrive -- Again kadalz Linux - Newbie 1 10-12-2003 11:20 AM
USB Flashdrive disto cli_man General 7 02-23-2003 04:56 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 08:23 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration