If I wish to put /var on the same hard drive as / (the OS), a seperate partition on the 60GB hDD can be created, correct? Like I said, the 750GB HDDs I only want for data storage and mirroring. I figured the 60GB HDD can be for keeping the OS files organized and seperate from the the 750GB drives.

actually i would put all 3 on the 750 gig, depending on what you are doing with the machine, a LAMP server (linux apache mysql php for instance) will use /var for mysql and can be configured to have the Document root (the top level folder where the your are stored) either as /var/www (default) or somewhere in individual user's /home directory

hmm.. ok... First and foremost, it will be a web (LAMP) server. If I put /var and /etc on the 750GB HDDs with /home, will this be an issue trying to setup a file server and/or other server packages? Or, if I do put them on the 60GB OS drive will that be a problem for the web server package? Also, do I have to designate a certain amount of space for these partitions/directories, and if so, how much would you suggest?

the file server part depends on how you wish to serve the files? with other linux machines, or with windows machines or both, however usually files are served from within individual home directories, or a common share point, which could be also simply another 'dummy' user's home directory.

I am going to suggest something completely different. Why not set your machine up as a bare metal hypervisor and then install a bunch of VMs to play with? This way you can segregate tasks, experiment, use snapshotting, etc. I have done that and it works really really well. I am using VMware ESXi 5.0 on my home server. You could also look into XenServer. Have a look.

ok.. well I'll have to look into that part as well. Basically, I'm imagining a scenario would be like this (whether it be an office document or mp3 or video)...

I sit down and decide I want to watch a movie on my windows 7 computer (everyday use computer)
login to the server from the windows 7 machine
locate and select a file (move.avi) on the /home partition/directory and execute
it plays it on my windows 7 machine

I haven't looked into or heard anything really about "how to serve the files" as I thought once you select a file path on the linux machine it will play it on the device I'm selecting it with, lol. Probably COMPLETELY wrong though.. I'd like them to be accessible on both Windows and Linux. I don't run any MACs so that platform isn't a big deal I guess.

Thanks. I'll check it out and see if it may fit my needs better.

You would want RAID1.

RAID0 is stripped, so anything you write is split between the two drives (half on each). This means that if you lose either drive, you lose everything on the entire array. There is no redundancy, however your array will be the sum of all parts, so two 750GB drives will give you a 1.5TB array.

RAID1 is mirrored, so anything you write is copied to both drives. This means that if you lose either drive, the system will fall back on the other seamlessly. The unfortunate part is because you're mirroring, the array will be the size of one drive (750GB).

You would probably just set up a SAMBA share on the Linux box. On Windows it will just show up as a shared drive, which you can map to a local drive letter (eg: P:\ will be a remote mount of the Linux SAMBA share, which might be /home/winshare on the Linux system). At that point, you can play it on the Windows machine as if it was local. I have my home set up this way. One Linux server hosting all of the media and serving it through both SAMBA and NFS. I then have several client machines, both Windows and Linux, who remote mount the shares and can play the media as if it's local. Even iTunes has its archive set to the shared location, so any time I buy music through iTunes it automatically moves it onto the Linux box through the share.

Your ISP may provide this information. I found out about an unpublished, unadvertised way of accessing this information using some backdoor access to, I think, my cable modem. I assume this is also where the ISP gets it's data. At any rate, shortly after my service was cut off, I implemented a primitive system in my server/firewall that makes use of iptables' throughput logging to keep tabs on my usage. It wasn't too difficult and is pretty low overhead. If you have a stand-alone firewall box, it might record such data.

--- rod.

Cool thanks. Yeah, I know most people are usually running at least 1TB drives, but I've been running on a 160GB drive for the last 4-5 years and have filled up maybe 50GB of it, lol. That is OS and all, hahah. I'm basically only going to be starting out with around 20GB of data to transfer to the server when it's set up. (perhaps 50GB if I keep the 25-30GB of music I just backed up for a friend of mine, hahah). But, that is also the reason I grabbed the 60GB drive so I can leave as much as possible of the 750GB intact. Right now, my windows machine is only seeing like 690-700GB on the HDDs... not sure if Linux will read them differently or not.

THIS! lol. This sounds exactly like what I want to do for "file sharing" on my home network. SAMBA has been mentioned in another post I had online. Will this setup work with smart phones, consoles (such as ps3, or xbox), laptops, etc... they would all be connected wirelessly to the network.

I may want to look into this at some point (depending on what the ISP says). I have no stand alone firewall box (unless my router acts as one, lol) so I don't think I'm recording this info anywhere.

I also thought about setting up a domain and trying to do something like an intranet. This way, users (my girlfriend, lol) can go to our OurMedia.com and browse the files on the server. Of course, I need to look into security for this so the world can't see all of my files and damage, steal, or plant viruses in anything.

That's normal. Hard drive manufacturers rate them in base 10 (1000). Capacity is generally measured in base 2 (1024).
The notation is technically different, GB for base 10, GiB for base 2, but very few people actually use GiB when referencing it.

So 750GB is actually 698 GiB, which is what your OS is reporting as the capacity. Linux will be the same way.

Laptops yes, never tried smart phones or consoles.

I knew HDDs never actually have the amount they state on them. I knew you always lose a few GBs of storage, but I didn't know why (the base thing). I just didn't realize you lose so much, lol. I only lost like 10GB on my 160GB drive while my 320GB drive (in my windows machine) loses like 30GB.

It comes out to about 7% when you're in the GB range, closer to 9% in the TB range.

Ok. I figured laptops, because they can recognize it as a "local drive" like the desktop. But, I'm not sure if smart devices and consoles are able to have that same recognition. That's where the intranet like setup comes to mind because all of these devices can access the web.

I would shy away from this configuration for a couple of reasons;

a) If the 750GB drives come up after the OS drive the system might not boot as the partitions are not mounted yet, especially /etc.

b) It is pretty straightforward to redirect MySQL and apache root dirs to places with more space; /home/mysql and /home/www

c) Backup - if two of the OS partitions are on the RAID'd disks and the rest are on the 60G drive it gives Clonezilla a harder job if you need to reinstate the drive

d) /etc changes very little in size over the life of an OS - my web server /etc currently stands at 28M

Mirroring - there a number of packages in Linux that can RAID drives without the need for a hardware RAID solution; dmraid, LVM. They will use more of your CPU to do the RAID function than dedicated hardware.

I'll have to look into the directing of PHP and mySQL directories. I like the idea of everything for the OS on the one drive...

As for the mirroring using the CPU... would my 3.2 dual core hold up? or would hardware for the RAID be needed? If the CPU can handle it well, that'd be great, lol.

Hardware RAID is only really beneficial (from a performance standpoint) for the architectures that require parity calculation, namely RAID levels 5 and 6. For RAID levels 0, 1, and 10 it really won't make much of a performance difference.

mdadm is the standard software RAID controller in Linux, I would just stick with that.

Is RAID 1 the only RAID "level" that does the mirroring? Above, someone mentioned RAID 0 will split my saved file information between the two 750GB HDDs so I won't use it. What can I do with RAID 10?

RAID 0 - data is stripped across all drives in the array. Total array capacity will be the sum of the individual drives. Speed goes up, but reliability goes down since losing just a single drive will wipe the entire array and everything on it. Processor overhead is low.

RAID 1 - data is mirrored across all drives in the array. Total array capacity will be the size of ONE of the drives. Write speed stays the same as single, read speed goes up, reliability goes up since you won't lose any data until you lose ALL of the drives in the array. Processor overhead is low.

RAID 10 - combination of 0 and 1. It requires 4+ drives, essentially you're setting up two 2-drive RAID 0 arrays (for a 4-drive RAID 10), and then putting those RAID 0 arrays in RAID 1. Total array capacity will be the sum of half of the drives (2TB for four 1TB drives, for example). Write speed goes up, read speed goes up, reliability goes up. You can lose at least one drive, possibly two (depending on which one it is) without losing any data. Processor overhead is low.

RAID 5 - basically like RAID 0, except one drive in the array is reserved for parity calculations. Total array capacity will be the sum of the individual drives minus one (3TB for four 1TB drives, for example). Write speed goes up, read speed goes up, reliability goes up. You can lose any ONE drive from the array without losing any data, but as soon as you lose a second drive you lose everything. Processor overhead is high.

RAID 6 - basically the same as RAID 5, except TWO drives are used for parity. Total array capacity will be the sum of the individual drives minus two (6TB for eight 1TB drives, for example). Write speed goes up, read speed goes up, reliability goes up. You can lose any TWO drives without losing any data, but as soon as you lose a third drive you lose everything. Processor overhead is very high.


All of my RAID systems run levels 1, 10, or 6, depending on the capacity and speed required (1 being the smallest, 6 the biggest). For levels 1 and 10 I usually just use mdadm, for level 6 I use a dedicated hardware controller because of all of the parity calculations involved...usually Adaptec since they're stupid fast, at least compared to LSI.

Thanks for the break down. I was wondering if there was another RAID level for mirroring where one might be better to do than the other... Looks like I'll be sticking with RAID 1 when I begin to set up to the hard drives.

Couple of pages that go into RAID in detail with diagrams & calculations.
https://en.wikipedia.org/wiki/RAID
https://en.wikipedia.org/wiki/Standard_RAID_levels

Thanks!

So, I'm having problems with the installation of Debian. I grabbed the "netinstall" CD and booted off that. I'm going through the standard setup procedure, and it is allowing me to setup my RAID configuration now, but I'm having trouble getting the /home partition assigned to one of the hardrives..

basically, the "automatic" install wants to split the partitions like this...

60GB HDD = /home, /usr, /var, /tmp (no mention of the /etc, but it does create the /swap without mentioning it)
750GB HDD = ? (I can use as "physical volume for RAID" but I'm not sure if only or both 750GBs are too be set at that?)
750GB HDD = ?

Should I be doing this through the command line? lol... I'm trying to go off the instructions on their website. This is the page I'm using for the RAID setup... http://www.debian.org/releases/stabl...ml.en#di-setup

I'm also not sure if I should be setting up the other partitions first before assigning /home to the 750GB...

My recommendation is to ignore the 750GB drives entirely during installation. Install everything on the 60GB, setting up all of your partitions as you want them (except for /home, leave that on /). Wait to set up the RAID 1 until the installation is complete. You'll have a lot more flexibility to get the RAID up and running when you're in the actual OS versus at the installation command line.

Once you get the OS installed and running, set up the RAID. Once you get the RAID up and running and mounted somewhere (eg: /mnt/raid/), log out all users, use Ctrl+Alt+F1-7 to switch to a terminal, log in as root, rsync your /home/ directory onto the RAID, move your /home/ directory to a backup (eg: /home.bak), mount your RAID to /home via /etc/fstab, then switch back to the GUI, log in as a regular user and make sure everything works. Once you're sure it does, remove /home.bak and you should be good to go.

what about partition sizes then? I need to either fill in my GB amounts for each partition I create manually, or they will automatically create the disk amount giving the /home directory most of the disk space. Although I'll be moving the /home directory...

---------- Post added 08-26-12 at 04:35 PM ----------

Any suggestions on a good tutorial for this? haha

Remember that you don't HAVE to create multiple mount pts; you could just put everything under '/' and then move /home as shown by suicidaleggroll.
That leaves maximum flexibility for disk usage on the 60GB; it will be shared by all the system except whatever you put on the 750GB RAID.
If you're paranoid about /var (logs) filling up, give that a separate partition.
Other than that, on a home system, you don't really need to bother... I do recommend a swap partition though.

Not that I know of, but it's pretty straight forward. Basically you're just transferring the contents of /home onto the RAID, then swapping the RAID in its place. You just want to make sure no regular users are logged in at the time so none of the files in /home are in use. The root user's home directory is in /root/, so any maintenance you want on /home can be done from the root account without negative consequences (since root's home directory is not located in /home/).

I do this all the time when setting up a system using a hardware RAID controller. A lot of hardware RAID controllers aren't recognized by the kernel, so it's infinitely easier to just set up the OS normally, then install the RAID driver, get the array up and running, then swap out whatever directory you want to put on the RAID...rather than doing everything during the OS install.

Something like this should do the trick (assuming the RAID is /dev/md127, and you've already logged out as any regular user and logged into one of the Ctrl+Alt+F# TTYs as root):
If everything look kosher in /home, you'd want to add the md127 mount to /etc/fstab, then run:
and re-check /home to make sure everything looks good.

Finally, you should reboot the system to make sure everything comes up like normal.

Instead of selecting the Automatic partitioning you may find it better to boot from a Parted Magic CD (using the Parted Magic GUI) and create your partition that way prior to trying to install with the Debian CD (the partitioning tool can be a bit daunting if you have not used this before)

I knew I could leave them mounted together, but if I mounted them separately I thought maybe the system would perform better. I also thought it would make access easier if I ever needed to go through the files in that directory/partition. Would this be correct?

Makes sense. I'd rather do it this way because it allows me to nit pick through the packages I want to run on the server. The CD installation basically tries to configure all of this stuff for me. I also figured installing the packages one by one would help me to learn and familiarize myself with a linux environment.

My problem with the installation manual on the Debian.org website is that it seems to be catered to the people who want a simple OS and are using the "regular or graphical" install methods. It's for the people who just want to run through the setup and learn nothing about the system (like windows, lol). It doesn't seem to go into much detail about installing the OS with multiple HDDs, RAID, or manually partitioning the HDD(s).


Also, there is currently no (and has never been) data on the 750GB HDDs. So, I'm not sure what I'd be backing up when moving the /home partition to the RAID setup. So will the backup still be necessary?

Also, the installation CD asks me to "configure my network" and when it asks for a name it has "debian" typed in already. I assumed this wouldn't be a big deal to change as it will be the name that appears on my home network. What confuses me, is it asks for a domain name. I don't have one (yet) and I don't want to just make one up because if it cannot be changed later, I can't know that domain name will be available later once I get the server running.

Also, it automatically enters "wowway.com" (my ISP) as the domain name. Should I just leave it? The installation manual seems to completely skip this step.

You can leave the domain blank, and you can give the machine any name you want to instead of "debian" - during installation or after.

The reason is that during the installation you will be asked to create a user account. That users configuration files will go into /home/aunnix (for example). So, in order that user aunnix can log in after the RAID is in place those configuration files must be copied to the new /home/aunnix (ie the one on the RAID disks)

Ok. I didn't actually think about there being "system (configuration)" files on the home partition once it is created. For the user name, I basically want all data saved to the one "account." So, would I be able to get by with naming it "all" (or something) and then just accessing the /home/all directory when manipulating files?

This is good to know.

The data doesn't need its own account, you can just make a directory called /home/all or /home/data to put your data, without creating a dedicated user for it.

So the way I understand it, is that you have to setup an admin (in order to access /) and the user is created so you can use the files and such that are stored on /home. If I don't create a user, will I be accessing everything as the admin? If so, this shouldn't be a problem since I will be the only using the server, but would it be better to go ahead and create the user?

Yes you should have a regular user. Doing all work on the system (including web browsing, etc) as root is not only dangerous in regards to accidentally running a command that will wipe out the entire system, it's also a security risk.

Are there any types of online chats I could use during my installation attempt(s)? lol. This way, if I run into a problem I could try to get advice relatively quickly rather than waiting forum replies?

Why don't you just run an installation using the default settings, and then you will have an opportunity to see what it does, and make a list of the things you want changed? Many things are fairly simple to fix after the installation is complete, such as hostname, domain name, etc. Don't sweat the small stuff until you get a feel for what is small stuff, and what isn't. It only takes a few minutes to repeat an installation, and you'll still learn things along the way.
--- rod.

I have thought about this... but I also wondered how well I'll see how the system runs with nothing on it, lol. There will be no data and if I do the basic install it seems like a waste of time trying to install any packages because I'm most likely going to have reinstall the OS. Can I do the install, and still move partitions around and change the amount of space (GB) assigned to them? If this is possible, then going through the install would be good. It gives me something to play with, lol...

At least this way, I can do the basic/default settings then tune them up the way I want them once the OS is operational..

Not easily

hahaha, so I assumed..

Another issue I have is I'm not sure what partitions are essential. Like I said, the installer will separate partitions, but it shows partitions that no one has mentioned in this thread, and people have mentioned partitions in this thread that the installer is not showing...

For the partitions mentioned in this thread and not shown by the installer, would I be safe to assume that those partitions are just going to be included in the "/" after the install is completed? I kind of feel like my "/home" partition doesn't need to be very large since I will be moving(mounting) it to my 750GB HDDs in the RAID setup.

Anything not explicitly partitioned out and mounted will be included in the / partition.

You do not want to separate your /home partition during installation, otherwise that will just be a wasted partition after you migrate onto the RAID. Leave /home on / during the install.