I'll be setting up a home file server and am looking for tips, feedback, and critiques. The target hardware is a single-core P4 with 2GB RAM. It's got onboard RAID on 2 of the 3 IDE headers (VIA VT6410 IDE RAID) as well as on the 4 SATA headers (Intel ICH6R SATA RAID). It has an onboard gigabit NIC. Blazingly fast read/write speed on this system will not be a really major concern--which will actually be an update/upgrade from a less capable P4 I set up some years ago to run NAS4Free.

On that latter note, while there's a lot to like about NAS4Free, I'm moving away from it because they no longer support 32-bit systems. I also find it's a bit of overkill for my needs: it's really nice to have all the options it provides (ZFS, software RAID, SMB/CIFS, AFP, NFS, ProFTPD, RSYNC, Unison, iSCSI target, BitTorrent client, iTunes/DAAP server, Network Bandwidth measure tool, Syncthing, Webserver, VirtualBox with WebSocket VNC server, phpVirtualBox and noVNC for VM management, etc, etc) but I've really only found myself using the file sharing features like FTP and/or NFS. And I can set up my own up-to-date 32-bit Linux distro of choice (probably Void) on this machine to do that limited task set.

I'd like to devote the various SATA/IDE connections on this machine to some hard drives I have around, so I'm looking into booting the OS from USB (I have a moderate amount of experience setting up and using Linuces that boot from USB drives). I would then use a few 100-250Gb IDE HD's to create a JBOD that would represent one file storage directory, while a 750GB SATA drive I recently inherited would represent another. The total 1.5GB drive capacity would be augmented by a 2TB drive that would act as a back-up for the 2 storage directories using rsync or some such in, say, a cron job.

1.5TB might sound like a paltry amount of space in this age of multi-terabyte HD's, but the fairly complete cache of data I am now storing on the aging NAS4Free system occupies about 20% of that space. So, for the immediate future, I think 1.5TB should be plenty for my needs.

Those are my considerations at the moment, in any case. Major questions that still need addressing are what file system to use, whether to incorporate LVM somehow (no direct experience with LVM apart from having read about it), whether ZFS might somehow be employed (I currently understand that it does not work well on 32-bit systems), and what sorts of distros/OSes might be suitable. On the latter note I have a lot of experience with Debian/Ubuntu and Arch, a moderate amount with Gentoo and Void, and a smattering of BSD experience (mainly under NAS4Free/FreeNAS).

Anyone here have any input on the project as described? OS recommendations? LVM/ZFS thoughts? Etc? Thanks in advance for any feedback.

PS As much as being an effective server/back-up solution, this project is also intended to provide me with some learning opportunities that will allow me to deepen my knowledge of various aspects of computer technology.

Backup is a relative term. Is it acceptable to you the second copy of the data is stored in the same physical frame? For me, I moved my main server to the basement, and my NAS which is the backup is on the first floor, so they are not right beside each other.

I recommend having your backup location be mounted only when you are performing the backup/rsync. In the backup script, do some sha256sums against files you have that won't change (anti-malware consideration), and then mount the destination drive.

LVM is useful for when storage capacity changes. If you have any possibilities of swapping out drives, use LVM.

ZFS is a separate option from LVM, because ZFS includes its own volume management. I have almost 0 experience with ZFS but it is probably worth more research for you. Its inclusion in Linux specifically is experimental, as I understand it. Some caveats the FZX on Linux team provides: https://github.com/zfsonlinux/zfs/wiki/FAQ

Use whatever distro you want. Any of the basic distro (Ubuntu, CentOS, Arch) can serve nfs/cifs/ftp with ease. If you like full control of the service, with a lot of manual building/config, use a generic distro. Any of the specialized distros could provide you a web ui or other "friendlier" methods of installation/admin but probably wouldn't give you the full control some people desire.

I personally use Centos 7 with just nfs, samba, and freeipa authentication. The rest of my environment is Centos/Fedora (oh, and some devuan) so the file server maintenance is identical to my environment. That was one of my considerations.

Thanks for your feedback, bgstack15. Great idea about leaving the back-up drive unmounted except when back-up is in progress--hadn't thought of that. The sha256sums idea also provides food for thought. I don't believe there's anything absolutely mission-critical on this file server but I do need to review that: it wouldn't be wise to realize only after I'd lost it how important one or other piece of data might have been. I tend to prefer doing tasks via command line as opposed to through some GUI overlay, so I doubt I'll pursue those sorts of front-end solutions. Continuing to look into viability/feasibility of LVM and/or ZFS for this project . . .

Regarding ZFS, from https://github.com/zfsonlinux/zfs/wi...64-bit-systems
So it looks like I'll need to use XFS, ext4, or some other. And maybe LVM underneath.

If I/O isn't a huge concern you can use MergerFS to pool the drives rather than using a traditional raid setup or lvm. I do it at home for my media server. Haven't had an issue yet, albeit only have 2 boxes at any given time running off it.

Feature of MergerFS that drew me to it > LVM. If a drive dies I only lose that drive, not the whole LVM setup. Depending on how often the data changes you can use Snapraid to back it up. Again, I do it at home but it's all media files that nearly never change. Snapraid also provides bitrot protection.

I run all my stuff at home on Ubuntu LTS, currently 16.04. Haven't had any issues in quite a long time. I rely on it and it hasn't let me down yet.

*EDIT* Personal note. To some 1.5tb might seem like nothing. My total storage capacity is only 2tb. I'm only using 54% of it, 500+ movies, 1264 episodes of various tv shows, along with all my other media. Depending on your purpose, there is nothing wrong with a 1.5tb. Don't overbuild if you don't have to. And don't be ashamed of a smaller storage than some other posters may have.

Dump the on-board RAID. If you ever have a mainboard failure (or a processor failure or a memory failure you are not able to buy the same board anymore either!) your RAID is gone and inaccessible. Repeat: Dump the on-board RAID.

Use mdadm RAID. It is virtually indestructible and recoverable regardless of mainboard type.

I don't know ZFS. If you decide to use Ext4 or XFS, go with LVM. Put LVM on top of the RAID. LVM is a great tool. If you have ever used it, you'll never install a server without it.

Put your backup separately from the box. If your data is worth anything to you, use 2 backup devices and store them alternatingly off-site. At this moment, my data is worth more than my complete household goods and office inventory together.

Any distro will do. Debian is perfect, but I recommend the distro in which you have the most experience.

jlinkels

1 members found this post helpful.

Thanks for the additional feedback, jmgibson1981 and jlinkels. I hadn't heard of either MergerFS or Snapraid. I've done some reading up on those and will need to do a bit more. As to having an additional remote back-up, jlinkels, I do, of course, see the sense in that. I think the files I'm storing are not quite as mission-critical as yours (these are almost all personal, as opposed to business, files) but, once I get this server set up and have gigabit networking operational, I will have better ability to back up to some remote storage location as you suggest.

As to dumping the onboard RAID, I have to confess that I really have to date no experience with hardware RAID (only a bit with software RAID). The mobo comes with the RAID integrated, so I am uncertain whether I can use the hard drive headers on this board without implementing RAID of some sort. I'll have to look into whether I can, effectively, bypass those RAID chips and thus entertain a suggestion such as you are proposing of "dumping" the RAID. If I can't, then what's really being proposed is dumping the motherboard, is it not?

One other thing about this project I neglected to mention is up-time. Under the old NAS4Free set up, the file server was powered off at least 90% of the time, and I hope to do implement that same scenario on the new file server. What made that feasible was enabling wake-on-LAN, which allowed me to power up the server remotely whenever I wanted to do a back up (the server is physically located on a different floor of the building in which I live). Then I would just power down the server through the web admin interface when finished. I hope to be able to implement the same sort of wake-on-LAN feature in this case, though I will likely need to ssh in to do the power down when I finish back-up tasks.

The mainboards with VIA chipset I know can simply bypass the RAID. On an additional note, this is not true hardware RAID. It is also called fake raid because the RAID is executed by software. The problem is not the software execution but the incompatibility.

About your power issue: maybe you should calculate the power consumption of your P4 against the power consumption of an Atom during 2 years and its purchase price. I run my headless server on a 1.6 GHz Atom, a powered down (from 12V to 5V) chassis fan and powered down power supply fan.

I use this as a file server and for some more other simple services like radio recording and playback, router/firewall. No GUI, no virtual machines or so. Moving large movie files from one files system to the other is exactly as fast as writing to a single hard disk. And I use mdadm RAID1

jlinkels

What you say about this board not having true hardware RAID makes sense now that you point that out: in fact, I recall reading in the manual about how some Windows software manages the RAID that runs on the VIA chip. Thanks for clarifying that point for me, jlinkels.

I know what you're driving at in terms of power consumption: P4 boards were known, IIRC, as some of the worst power hogs yet. But since the server will be located in an area of this apartment building where I can mooch off the building's common power system, running it will not result in any additional cost to me. So I'm not too concerned about that aspect of the file server in this particular incarnation.

I'll look into mdadm as well though, at the moment, I'm leaning toward LVM. Since I plan regularly to back up data to a mostly-unmounted hard drive within the same case and am entertaining the idea of additionally doing occassional further back-ups to some remote storage device, I'm not sure it will be expedient to implement RAID1 here. But like I've mentioned, I'm still in the planning/investigation phase of the project, so nothing definite has been decided yet on any of this.

Thanks.

Now that I've got all the data copied onto intermediate storage from the old file/back-up server, I'm ready to begin setting up the new server. I've formulated some plans for file structure and disk topography and I'd like to ask for some additional feedback prior to implementation.

First, I envision, in addition to the basic operating system files on this machine that will reside on a small additional drive (perhaps on a USB stick), two main storage directories: let's call one of those directories "personal" and the other "technical." Underlying each directory will be one or more largish hard drives with a mount point at, say /mnt/personal and /mnt/technical (there will, of course, be sub-directories under each). Yet another, even larger hard drive, will be attached but will remain unmounted most of the time (per bgstack15's suggestion). Its purpose will be to act as a place to back up, at regular intervals, the entire content of both /mnt/personal and /mnt/technical. When mounted in preparation for doing a back up, it will be mounted at, say, /mnt/back-up.

An additional drive of similar size, such as an external USB drive, may be added in the future to act as an additional back up of /mnt/back-up, and could be attached to some other machine on my LAN that will be located in a place remote to the file server (per jlinkels's suggestion). But that is a potential future scenario that is only being contemplated at the moment and that will likely not be implemented in the near future.

As to the available drives, I currently have at my disposal a 2TB SATA drive I intend to use as the /mnt/back-up drive, a 750GB SATA drive I intend to mount at /mnt/technical, and 4 IDE drives--2 at 250GB and 2 at 100GB (700GB total)--that I intend to pool together and mount at /mnt/personal. Again, blazingly fast transfer speeds to/from this file server are a secondary concern: this will act as long-term storage with see only occassional transfers to/from the unit. And, in fact, I hope to keep it powered off about 90% of the time.

Apart from soliciting input on the overall plan just presented, one particular aspect of it I'd like to ask about is the pooling of the 4 IDE drives. So far as I understand it, I have a few options at my disposal for pooling them: 1) use the onboard, built-in software RAID to create a JBOD and to put on top of that a file system such as ext4; 2) use a Linux software RAID utility like mdadm to similarly join them together, then format; 3) use LVM to make a pool of them and then format; or 4) a combination 2) and 3). Does it seem I've covered all possible methods for turning the 4 IDE disks into a single pool?

I am inclined at the moment to opt for 1) because it appears the simplest solution. LVM, while otherwise attractive, looks to me not especially appropriate because I foresee no need for expanding storage space here--mainly because the underlying hardware of this file server is essentially EOL (by the time it will be necessary to do the next upgrade, it seems likely that 32-bit operating systems may no longer be around). It looks to me that the main advantage of using mdadm would be increased read/write speed: but, as mentioned, the speed at which the data can be accessed on this machine is a decidedly secondary concern for me. So I'm not sure the added complexity of implementing mdadm on this system will be justified.

Finally, as to backing up to /mnt/back-up, I am currently envisioning a script that will run each time the machine will be shut down, one that will first mount the drive at its mount point, then run rsync to do the back up. I didn't yet work out details for doing that but I believe my quite modest bash skills, augmented with a bit of additional advice, should be sufficient to the task.

Any further input on the proposed scheme? Have I understood the available options for the attached hard drives? Any that I've overlooked? Thanks.