[SOLVED] Advanced file server

We are several families that shares an internet connection. There is a server room with a rack and I want to make a file server that is available for everyone to store data like photos and video. I also have a Dell PowerVault filled With harddisks. However I find it a little hard to find out where I should begin. I will number the specifications and if someone wants to comment one or more of the questions I would appreciate it. I plan on using Debian.

1. Users from ActiveDirectory
------------------------------------
I would like to use Samba since all of the users have Windows. There is also a domain server (ActiveDirectory) and the question is: Is there a way to use that ActiveDirectory for the login?

2. Quota
------------------------------------
Assuming that no.1 is possible, can I give every user a certain amount of data?

3. File System
------------------------------------
I'm going to use the PowerVault with RAID for maximum protection of the data. There will also be the possibility to connect more disks. Is there a way to handle several disks as one directory? I imagine there would be a system that can handle all of the details in the background and just present a directory.

4. Backup
------------------------------------
To protect data from fire and theft I would like to regulary backup the system to another building through the network. I would need some kind of mirror server and synchronization between the servers. There should be a limited speed to not clog the network and maybe full speed during night hours. Any suggestions on what to use?

Thanks!

2. Quota is possible
3. Database: Raid 5 or Raid 1

Have you choosen a Distro like Centos?

Welcome. Sounds like a useful and interesting project.

For 1, maybe consider SFTP instead of or along side Samba. SSHFS is available to legacy users and all FOSS users have built-in SFTP support in their file managers. SFTP can be safely used over the Internet in contrast to Samba (or anything from M$ or implementing M$ protocols) which would need a VPN (e.g. OpenVPN) for that. Also, SSH / SFTP can use PAM which has a lot of authentication options. There are also some "cloud" packages available to Debian if people prefere to access everything through a browser with javascript.

For 2, Samba does quotas, so do other files systems though they may have to be set explicitly in the mount options.

For 3 I'd at least look at ZFS level raidZ2 instead of RAID6 or RAID10

For 4 I'd plan for having at least two off-site backups, but they can all be in the one other building. The reason for multiple backups is that I wouldn't really consider anything a backup during the time it is also connected to the net. During the time these are online, they are only caches not backups. Either way, though, rsync is good at that and can even do incremental backups. It automates easily, too.

Also, I'd aim to run a few trial set ups first and plan to erase everything a few times before officially going into production. "plan to throw one away; you will, anyhow" If there are any in the builing that would like to help test after you've narrowed down the choices a bit they can really find the rough edges that you might have missed. Then the second or third time you set it up it should be quite nice and, if you've kept notes along the way, well documented.

Originally Posted by jorx2 1. Users from ActiveDirectory ------------------------------------ I would like to use Samba since all of the users have Windows. There is also a domain server (ActiveDirectory) and the question is: Is there a way to use that ActiveDirectory for the login?

I've never used it, but "realmd" seems to be what you want for this. This how-to is for the previous version of Debian (Debian 8), but it may work for the current version (Debian 9) unchanged:

http://www.alandmoore.com/blog/2015/...ive-directory/

Note that by default, Debian does not use sudo. If you see a how-to that has "sudo" in it, log in as root (using the command "su" at a command line) instead and remove "sudo " from any commands before entering them.

2. Quota ------------------------------------ Assuming that no.1 is possible, can I give every user a certain amount of data?

Yes. See:

https://debian-handbook.info/browse/...ct.quotas.html

3. File System ------------------------------------ I'm going to use the PowerVault with RAID for maximum protection of the data. There will also be the possibility to connect more disks. Is there a way to handle several disks as one directory? I imagine there would be a system that can handle all of the details in the background and just present a directory.

LVM. Unfortunately, the Debian installer is not the most intuitive thing to use for creating LVM, but if you want to handle several disks as one then it's really the best option.

LVM lets you add or remove drives to/from a "volume group", and manage partitions (logical volumes) within it. You'll probably want two logical volumes - one for the root OS "/" (4GB will be plenty) and one for the user data "/home". Splitting it up this way will make it easier later on if you decide to reinstall the OS or install a different OS.

4. Backup ------------------------------------ To protect data from fire and theft I would like to regulary backup the system to another building through the network. I would need some kind of mirror server and synchronization between the servers. There should be a limited speed to not clog the network and maybe full speed during night hours. Any suggestions on what to use?

rsnapshot is a good efficient method. It regularly makes incremental backups, and does so in an efficient way so that unchanged unmoved files are stored only once. That way, it keeps snapshots of previous days and weeks without consuming excessive amounts of space.

It fundamentally uses rsync to do the backups, and you can optionally set it to limit bandwidth use. As such, it is suitable for doing continuous synchronization. Personally, I use rsync synchronization instead of RAID on my home systems. For my use, it's kind of like a poor man's RAID1, but without automatic failover. That means that if the primary fails, it fails. I can't remember the last time I had a hard drive failure on my file server, so I don't worry about the potential for downtime (there are a lot of other causes of downtime, which hit me more often than hard drive failure...power failure, motherboard failure, PSU failure...). But rsync synchronization works great for laptops, since laptops often can only have one drive anyway and network shares are good for the backup.

Obviously, the backup does indeed consume MORE space than the primary. Changed/moved files are stored in the backup snapshots, in addition to the current files.

Therefore, you should be thinking about the backup first. It has to be bigger, so you need to think about how much storage you want to devote to that system. OTOH, depending on what sort of RAID you intend to set up on your primary, this might not be such a big deal. Obviously, if you use RAID1 on the primary, but don't use RAID at all on the backup, then you could have twice as much storage on the backup even with the same drives involved.

Peek at the stuff from 2x.com while you are looking.

Many web articles on how to integrate linux with AD. There are a number of steps but all tend to be easy.

Types of support on media. Advanced file systems like Btrfs/zfs. Two is LVM. Three is software raid and lastly hardware raid options. Any or all might be used. The choice may depend on your OS.

Originally Posted by IsaacKuo LVM, but if you want to handle several disks as one then it's really the best option. LVM lets you add or remove drives to/from a "volume group", and manage partitions (logical volumes) within it.

But make the LVM a raid volume. Otherwise, if you lose one drive, you've lost all your data.

Originally Posted by AwesomeMachine But make the LVM a raid volume. Otherwise, if you lose one drive, you've lost all your data.

But if I set up a hardware RAID, LVM will see it as one disk but it will be protected. Is that true?