LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware
User Name
Password
Linux - Hardware This forum is for Hardware issues.
Having trouble installing a piece of hardware? Want to know if that peripheral is compatible with Linux?

Notices

Reply
 
Search this Thread
Old 10-26-2012, 01:17 PM   #16
TobiSGD
Moderator
 
Registered: Dec 2009
Location: Hanover, Germany
Distribution: Gentoo
Posts: 15,364
Blog Entries: 2

Rep: Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981

Windows is not able to read ext4. there are some drivers for that, but I never used them, so I can't comment on their quality. IMHO formatting the data drives in NTFS would be the best option.
 
Old 10-26-2012, 01:34 PM   #17
Aunnix
Member
 
Registered: Aug 2012
Posts: 126

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by TobiSGD View Post
Windows is not able to read ext4. there are some drivers for that, but I never used them, so I can't comment on their quality. IMHO formatting the data drives in NTFS would be the best option.

Ok. So, how do I save files to those drives if my /home is on my 60GB OS drive?
 
Old 10-26-2012, 01:36 PM   #18
TobiSGD
Moderator
 
Registered: Dec 2009
Location: Hanover, Germany
Distribution: Gentoo
Posts: 15,364
Blog Entries: 2

Rep: Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981
Just mount it in a way that your user has permissions to access the disk.
 
Old 10-26-2012, 01:42 PM   #19
Aunnix
Member
 
Registered: Aug 2012
Posts: 126

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by TobiSGD View Post
Just mount it in a way that your user has permissions to access the disk.

I just thought of this... I won't necessarily NEED to hook it up to a windows computer unless it is a computer not on my network. Everything will be connected, so if I need files from the server on my Windows computer I can just transfer it, haha. I'm dumb, lol..

What if a friend wanted some files? Could I hook his NTFS formatted external HDD to the PCI USB 3.0 card and transfer files from my server straight to his external HDD? I guess that's a more realistic scenario for how to format my drives..
 
Old 10-26-2012, 01:52 PM   #20
TobiSGD
Moderator
 
Registered: Dec 2009
Location: Hanover, Germany
Distribution: Gentoo
Posts: 15,364
Blog Entries: 2

Rep: Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981
Yes, that is totally possible.
Keep in mind that you have to make your files accessible to other users anyways, most likely the streaming service, your webserver, ... will not run under your users account, but have their own accounts.
 
Old 10-26-2012, 02:15 PM   #21
Aunnix
Member
 
Registered: Aug 2012
Posts: 126

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by TobiSGD View Post
Yes, that is totally possible.
Keep in mind that you have to make your files accessible to other users anyways, most likely the streaming service, your webserver, ... will not run under your users account, but have their own accounts.
hmm... so when I installed Debian I had to setup a user account (called derek) for which I use to log into the server (derek@server). If I'm logged in under "derek" and save files it goes to my "user/derek/home" correct?

Are you saying I will have to create many "dereks" for all of the services I want to run? IE: the web server will have its own account = "user/webserver/home", streaming service with its own account = "user/streaming/home"?

If so, how would I go about saving files? Still save them under "user/derek/home" and just point "user/webserver" to the website files (or directory(ies)) that are stored in "user/derek/home"? And, the same with "user/streaming" as the "user/webserver" example?


Sorry if I'm asking too many questions and wasting your time, haha. I'm just trying to get the best grasp on how best to setup things.. (namely the backing up / RAID situation). I do appreciate the chat!
 
Old 10-26-2012, 02:34 PM   #22
TobiSGD
Moderator
 
Registered: Dec 2009
Location: Hanover, Germany
Distribution: Gentoo
Posts: 15,364
Blog Entries: 2

Rep: Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981
Quote:
Originally Posted by Aunnix View Post
hmm... so when I installed Debian I had to setup a user account (called derek) for which I use to log into the server (derek@server). If I'm logged in under "derek" and save files it goes to my "user/derek/home" correct?
No, by default your files are saved somewhere in "/home/derek".

Quote:
Are you saying I will have to create many "dereks" for all of the services I want to run? IE: the web server will have its own account = "user/webserver/home", streaming service with its own account = "user/streaming/home"?
No, the different services will automatically create their own users when they are installed. Those services usually don't have their own home-directory. For example, on my Debian server I have the automatically created users mail, www-data, sshd, ftp, ntp, dovecot, fetchmail, ..., just because all these services run on that machine.

Quote:
If so, how would I go about saving files? Still save them under "user/derek/home" and just point "user/webserver" to the website files (or directory(ies)) that are stored in "user/derek/home"? And, the same with "user/streaming" as the "user/webserver" example?
Where you save those files is usually not the point, since you can configure your services which directories to use. The point is that a service that needs to read a file (for example a streaming server) must obviously have the permissions to read that file. The Linux file permission system is flexible enough to handle that situation.
 
Old 10-26-2012, 03:00 PM   #23
Aunnix
Member
 
Registered: Aug 2012
Posts: 126

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by TobiSGD View Post
No, by default your files are saved somewhere in "/home/derek".

No, the different services will automatically create their own users when they are installed. Those services usually don't have their own home-directory. For example, on my Debian server I have the automatically created users mail, www-data, sshd, ftp, ntp, dovecot, fetchmail, ..., just because all these services run on that machine.

Where you save those files is usually not the point, since you can configure your services which directories to use. The point is that a service that needs to read a file (for example a streaming server) must obviously have the permissions to read that file. The Linux file permission system is flexible enough to handle that situation.
Ok. I think I get the idea. I'm straying from the backup issue though, haha. I will figure out the services once I get a backup system running.

I will see if I can do some testing over the weekend. I'm still having people suggest a RAID 1 setup for this project.

http://www.tomshardware.com/forum/fo...nojs=0#t873598

They are saying the only problem I will have with losing files is if I accidently delete it. They say my data will still be intact (if the HDDs are not the issue anyway) if I have a hardware or software failure... It seems like it is coming down to whether I want the file writes to be instant or if I want them to be synced every so often. I'm a bit torn, lol, as both methods seem like they will do what I need. I'm not sure which would be more efficient though..
 
Old 10-26-2012, 04:05 PM   #24
TobiSGD
Moderator
 
Registered: Dec 2009
Location: Hanover, Germany
Distribution: Gentoo
Posts: 15,364
Blog Entries: 2

Rep: Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981
Have a look at the recent found bug in the ext4 file-system driver. That bug turned out to be not as critical as thought at first, but a RAID-1 can not withstand such a bug if it occurs. The backup solution we discussed here can do that easily.
So, no, there can occur other problems than just accidental file deletion, those errors can occur suddenly and without any warning.
This why anyone with some experience in the field will tell you the same: A RAID is never in any form a substitute for a backup solution.
A RAID is only a solution for you if you make daily (hourly? whatever timespan of data loss seems reasonable to you) backup to a external drive, which usually has to be done manually. You explicitly stated that this is not what you want.

Last edited by TobiSGD; 10-26-2012 at 04:10 PM.
 
Old 10-26-2012, 05:24 PM   #25
Aunnix
Member
 
Registered: Aug 2012
Posts: 126

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by TobiSGD View Post
Have a look at the recent found bug in the ext4 file-system driver. That bug turned out to be not as critical as thought at first, but a RAID-1 can not withstand such a bug if it occurs. The backup solution we discussed here can do that easily.
So, no, there can occur other problems than just accidental file deletion, those errors can occur suddenly and without any warning.
This why anyone with some experience in the field will tell you the same: A RAID is never in any form a substitute for a backup solution.
A RAID is only a solution for you if you make daily (hourly? whatever timespan of data loss seems reasonable to you) backup to a external drive, which usually has to be done manually. You explicitly stated that this is not what you want.
Ok... I assume a RAID setup that encounters that bug will lose the data from both HDDs because they act as one? So, of course, whatever is deleted from HDD A is deleted from HDD B...

Then, with our discussed method, I'm assuming that all of the data copied to HDD B before the bug occurred will still be safely sitting on B? Would this bug stop by from syncing to A? Because if not, what happens if B syncs to A before I see the error and / or data loss? If files are changed or missing from A, will B make the same changes?

And you are right about the external drive bit. This method discussed does appear to be better suited for my desires.


Also, would you recommend ext3 over ext4?

Last edited by Aunnix; 10-26-2012 at 05:25 PM.
 
Old 10-26-2012, 05:41 PM   #26
TobiSGD
Moderator
 
Registered: Dec 2009
Location: Hanover, Germany
Distribution: Gentoo
Posts: 15,364
Blog Entries: 2

Rep: Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981
Quote:
Originally Posted by Aunnix View Post
Ok... I assume a RAID setup that encounters that bug will lose the data from both HDDs because they act as one?
Correct.

Quote:
Then, with our discussed method, I'm assuming that all of the data copied to HDD B before the bug occurred will still be safely sitting on B?
Correct.

Quote:
Would this bug stop by from syncing to A? Because if not, what happens if B syncs to A before I see the error and / or data loss? If files are changed or missing from A, will B make the same changes?
It depends on how you configure the syncing application. If you use rsync for that you can either specify the --delete option (which will delete all files on disk B that are not longer present on disk A, so that the bug would affect you if it would have gone unnoticed) or you don't specify that option (so that deleted files will still be on disk B).
Since you want to sync the disks automatically in a relative short interval not specifying the option makes actually more sense, so that you can recover accidentally removed files. You can simply run once a month (or whatever is convenient for you), after making sure that all files are in place, the same backup script as usual just with the --delete option to clean up disk B.

Quote:
Also, would you recommend ext3 over ext4?
I use ext4 for quite some time now and never had a problem with that, but which file-system you use should be determined by the usage of that partition. For example, if you have are storing mostly large files on the disk you may get better performance with using XFS.
 
Old 10-26-2012, 07:08 PM   #27
Aunnix
Member
 
Registered: Aug 2012
Posts: 126

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by TobiSGD View Post
It depends on how you configure the syncing application. If you use rsync for that you can either specify the --delete option (which will delete all files on disk B that are not longer present on disk A, so that the bug would affect you if it would have gone unnoticed) or you don't specify that option (so that deleted files will still be on disk B).
Since you want to sync the disks automatically in a relative short interval not specifying the option makes actually more sense, so that you can recover accidentally removed files. You can simply run once a month (or whatever is convenient for you), after making sure that all files are in place, the same backup script as usual just with the --delete option to clean up disk B.
Awesome. I think this method of actually backing up makes more sense to use.

Quote:
Originally Posted by TobiSGD View Post
I use ext4 for quite some time now and never had a problem with that, but which file-system you use should be determined by the usage of that partition. For example, if you have are storing mostly large files on the disk you may get better performance with using XFS.
I will need to do some research on the file systems I guess. I'd say the largest file I'd be storing would be like a 2GB movie... is this considered a large file to consider XFS? I'd say of the 1200GB of data, 1100 or so are movies. So, if it's not considered a large file would the number of files this large be a concern when deciding what filesystem to use? Everything is going to be stored in sub directories in my /home or /home/derek.. I'm thinking /home so it is easier to point to for assigning permissions for files.

I'm wondering now how saving files in /home/derek affects the access of them for another user such as /home/tom? hmm...
 
Old 10-26-2012, 07:44 PM   #28
TobiSGD
Moderator
 
Registered: Dec 2009
Location: Hanover, Germany
Distribution: Gentoo
Posts: 15,364
Blog Entries: 2

Rep: Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981Reputation: 3981
Quote:
Originally Posted by Aunnix View Post
I will need to do some research on the file systems I guess. I'd say the largest file I'd be storing would be like a 2GB movie... is this considered a large file to consider XFS? I'd say of the 1200GB of data, 1100 or so are movies. So, if it's not considered a large file would the number of files this large be a concern when deciding what filesystem to use? Everything is going to be stored in sub directories in my /home or /home/derek.. I'm thinking /home so it is easier to point to for assigning permissions for files.
I am not sure about that, so doing some research is in fact the best way to determine which file-system you should use.

Quote:
I'm wondering now how saving files in /home/derek affects the access of them for another user such as /home/tom? hmm...
Handling situations like that is actually pretty easy and flexible in Linux:
For example, assuming that derek should be the main user with full read/write access and all other users (here for example tom)should have only read access to the movies:
- create a new usergroup, for example called movies
- change the ownership of the movies to the user derek and the group movies
- change the permission on the movies to read/write for the user and read for the group
- add tom to the group movies

If you later want to give another user read permissions for the movies (for example the streaming server) just add him to the movies group
 
Old 10-26-2012, 07:50 PM   #29
Aunnix
Member
 
Registered: Aug 2012
Posts: 126

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by TobiSGD View Post
I am not sure about that, so doing some research is in fact the best way to determine which file-system you should use.

Handling situations like that is actually pretty easy and flexible in Linux:
For example, assuming that derek should be the main user with full read/write access and all other users (here for example tom)should have only read access to the movies:
- create a new usergroup, for example called movies
- change the ownership of the movies to the user derek and the group movies
- change the permission on the movies to read/write for the user and read for the group
- add tom to the group movies

If you later want to give another user read permissions for the movies (for example the streaming server) just add him to the movies group


Nice. Sounds simple enough. Basically I would have derek as the "admin" user and I want to create a "guest" user that everyone can use to read files.. no one should need write permissions on this machine besides me. If they save something to the machine, it will be through a computer or laptop on the network as I plan to save everything directly to the server rather than locally.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
setup hardware raid on Debian machine mposwal Debian 4 05-20-2012 11:30 PM
Hardware Mirrored Raid Setup niteshadw Linux - General 1 01-11-2011 01:26 PM
Hardware RAID 5 Setup on Debian 3.01 bullium Linux - Hardware 2 02-24-2006 07:45 AM
Setup Hardware Raid ust Linux - Distributions 2 05-09-2005 02:28 PM
Hardware raid setup Mavin Linux - Hardware 2 02-08-2004 01:33 PM


All times are GMT -5. The time now is 06:47 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration