Windows is not able to read ext4. there are some drivers for that, but I never used them, so I can't comment on their quality. IMHO formatting the data drives in NTFS would be the best option.

Ok. So, how do I save files to those drives if my /home is on my 60GB OS drive?

Just mount it in a way that your user has permissions to access the disk.

I just thought of this... I won't necessarily NEED to hook it up to a windows computer unless it is a computer not on my network. Everything will be connected, so if I need files from the server on my Windows computer I can just transfer it, haha. I'm dumb, lol..

What if a friend wanted some files? Could I hook his NTFS formatted external HDD to the PCI USB 3.0 card and transfer files from my server straight to his external HDD? I guess that's a more realistic scenario for how to format my drives..

Yes, that is totally possible.
Keep in mind that you have to make your files accessible to other users anyways, most likely the streaming service, your webserver, ... will not run under your users account, but have their own accounts.

hmm... so when I installed Debian I had to setup a user account (called derek) for which I use to log into the server (derek@server). If I'm logged in under "derek" and save files it goes to my "user/derek/home" correct?

Are you saying I will have to create many "dereks" for all of the services I want to run? IE: the web server will have its own account = "user/webserver/home", streaming service with its own account = "user/streaming/home"?

If so, how would I go about saving files? Still save them under "user/derek/home" and just point "user/webserver" to the website files (or directory(ies)) that are stored in "user/derek/home"? And, the same with "user/streaming" as the "user/webserver" example?


Sorry if I'm asking too many questions and wasting your time, haha. I'm just trying to get the best grasp on how best to setup things.. (namely the backing up / RAID situation). I do appreciate the chat!

No, by default your files are saved somewhere in "/home/derek".

No, the different services will automatically create their own users when they are installed. Those services usually don't have their own home-directory. For example, on my Debian server I have the automatically created users mail, www-data, sshd, ftp, ntp, dovecot, fetchmail, ..., just because all these services run on that machine.

Where you save those files is usually not the point, since you can configure your services which directories to use. The point is that a service that needs to read a file (for example a streaming server) must obviously have the permissions to read that file. The Linux file permission system is flexible enough to handle that situation.

Ok. I think I get the idea. I'm straying from the backup issue though, haha. I will figure out the services once I get a backup system running.

I will see if I can do some testing over the weekend. I'm still having people suggest a RAID 1 setup for this project.

http://www.tomshardware.com/forum/fo...nojs=0#t873598

They are saying the only problem I will have with losing files is if I accidently delete it. They say my data will still be intact (if the HDDs are not the issue anyway) if I have a hardware or software failure... It seems like it is coming down to whether I want the file writes to be instant or if I want them to be synced every so often. I'm a bit torn, lol, as both methods seem like they will do what I need. I'm not sure which would be more efficient though..

Have a look at the recent found bug in the ext4 file-system driver. That bug turned out to be not as critical as thought at first, but a RAID-1 can not withstand such a bug if it occurs. The backup solution we discussed here can do that easily.
So, no, there can occur other problems than just accidental file deletion, those errors can occur suddenly and without any warning.
This why anyone with some experience in the field will tell you the same: A RAID is never in any form a substitute for a backup solution.
A RAID is only a solution for you if you make daily (hourly? whatever timespan of data loss seems reasonable to you) backup to a external drive, which usually has to be done manually. You explicitly stated that this is not what you want.

Ok... I assume a RAID setup that encounters that bug will lose the data from both HDDs because they act as one? So, of course, whatever is deleted from HDD A is deleted from HDD B...

Then, with our discussed method, I'm assuming that all of the data copied to HDD B before the bug occurred will still be safely sitting on B? Would this bug stop by from syncing to A? Because if not, what happens if B syncs to A before I see the error and / or data loss? If files are changed or missing from A, will B make the same changes?

And you are right about the external drive bit. This method discussed does appear to be better suited for my desires.


Also, would you recommend ext3 over ext4?

Correct.

Correct.

It depends on how you configure the syncing application. If you use rsync for that you can either specify the --delete option (which will delete all files on disk B that are not longer present on disk A, so that the bug would affect you if it would have gone unnoticed) or you don't specify that option (so that deleted files will still be on disk B).
Since you want to sync the disks automatically in a relative short interval not specifying the option makes actually more sense, so that you can recover accidentally removed files. You can simply run once a month (or whatever is convenient for you), after making sure that all files are in place, the same backup script as usual just with the --delete option to clean up disk B.

I use ext4 for quite some time now and never had a problem with that, but which file-system you use should be determined by the usage of that partition. For example, if you have are storing mostly large files on the disk you may get better performance with using XFS.

Awesome. I think this method of actually backing up makes more sense to use.

I will need to do some research on the file systems I guess. I'd say the largest file I'd be storing would be like a 2GB movie... is this considered a large file to consider XFS? I'd say of the 1200GB of data, 1100 or so are movies. So, if it's not considered a large file would the number of files this large be a concern when deciding what filesystem to use? Everything is going to be stored in sub directories in my /home or /home/derek.. I'm thinking /home so it is easier to point to for assigning permissions for files.

I'm wondering now how saving files in /home/derek affects the access of them for another user such as /home/tom? hmm...

I am not sure about that, so doing some research is in fact the best way to determine which file-system you should use.

Handling situations like that is actually pretty easy and flexible in Linux:
For example, assuming that derek should be the main user with full read/write access and all other users (here for example tom)should have only read access to the movies:
- create a new usergroup, for example called movies
- change the ownership of the movies to the user derek and the group movies
- change the permission on the movies to read/write for the user and read for the group
- add tom to the group movies

If you later want to give another user read permissions for the movies (for example the streaming server) just add him to the movies group

Nice. Sounds simple enough. Basically I would have derek as the "admin" user and I want to create a "guest" user that everyone can use to read files.. no one should need write permissions on this machine besides me. If they save something to the machine, it will be through a computer or laptop on the network as I plan to save everything directly to the server rather than locally.