I want to set up an inexpensive but highly reliable file server for my photos and movies/TV shows using a stable and reliable Linux distro. The photos CANNOT be allowed to disappear, while the movies and TV shows are replaceable. The "server" will (due to poor economy) be a dirt cheap PC (~US$150 for MB+CPU) with two high quality SATA disks in a RAID 1 configuration for storing the photos. I do not have the money to buy a RAID controller, so I will have to set up Linux to do software-based RAID. The operating system partitions will be on a separate HDD to keep things simple. To keep my photos as safe as possible, I wish to use a journalled file system. Backup of the photos will be made to an external USB2 disk - this disk being perhaps the most important component of my system. The movies/TV shows are, like I said earlier, expendable, so I will use two ordinary, big external HDDs (USB2, FireWire or eSATA) for these files. These disks should preferably have as high throughput as possible.

To recap: A cheap Celeron D or Athlon 64-based PC with one small HDD for Linux installation, two big SATA HDDs in a SW RAID 1 configuration, one big, external USB2 disk for photo backup and two big, external USB2/FireWire/eSATA disks for storing movies/TV shows (with no backup or RAID).

The computer will be used as a SAMBA domain controller and file server ONLY. No other applications will be running on this system. No GUI will be running (although as a newbie I might be better off setting up/configuring the server using a GUI?). I will be buying all new hardware for this system (with the exception of the small HDD for OS installation, the USB2 backup disk and possibly casing/PSU).

I am new to Linux and love the simplicity (and HW support) of Ubuntu (I am posting this from an Ubuntu 6.06.1 LTS installation on a PowerMac G3 with an old MS wireless mouse - it just works). At the same time I want a STABLE distro, where focus is on the server just WORKING. The distro must have support for software RAID and support eSATA and/or the FireWire card I own (SafeWay FireWire-PCI) - I assume "all" distros support SATA and USB2? Obviously, it needs to support the motherboard I'm planning to buy as well (cheap MicroATX-board - I will ask the Linux HW forum for advise). No wireless support is needed.

So, what say ye? Is this a sound approach, or have I completely missed something? Remember: the two most important factors for this project is file integrity (for the photos) and low cost.

What distro should I use? Is Ubuntu Server sufficiently stable? Am I better off with a "pure" server distro such as Trustix Secure Linux? Or perhaps Slackware or Debian Etch?

I want to use ext3 for the photo disks because of it's stable journaling properties (both metadata and file contents are written to the journal before being committed to the main filesystem) as opposed to XFS and ReiserFS, which is metadata-journaling only. Any comments on this? File size on these disks will be big; between 512kB-15MB, so ReiserFS is basically out of the question. On the media disks the files will be even bigger; 100MB-1GB each.

Is it possible to use ext3 on my RAID disks and a faster (XFS? ext2?) file system on my two media disks? Since I am new to Linux and this will be a learning project it is critical that the photo backup disk is functioning - preferably in a multitude of systems. Can I use MS Windows-readable NTFS on my backup disk?

Best regards
Tord Stromdal, Linux newbie

I think it all boils down to the old formula; do you want speed or security?

I have one machine which I use for critical stuff, instead of using RAID, I mounted two identical disks which I mirror manually to have the second one as a online backup. I also backup the initial drive with a cronjob to another machine (this would then be the external drive in your case).

I use ReiserFS in debug mode on the initial drive, mainly because of previous experiences. (I once mounted a bad drive in my fileserver, it wasn't even usable in windows due to many bad blocks, but was rather large at the time, so I wanted to see if I could use it as a temp-drive or something, and while running ReiserFS in debug mode it has still never dropped a file in it's three-year long service after it got unusable by windows, it just keeps getting smaller...

If you require speed, large files and at the same time want to access your drives from windows, I would setup ext2 or ext3 all the way. You can find a nice little piece of software here: http://www.fs-driver.org/ which will allow you to use your ext2/ext3 drive from windows. The choice between ext2/ext3 would in my opinion depend on how much you use windows, ext2fs cannot write journal changes on ext3 from windows, so it gets treated as a ext2 system. ext2 supports large files in new kernels anyway.

When it comes to your distro choice, I would experiment a little. If you keep your files (photo/video/etc) on the drive configuration you specify, you can easily format your OS drive without being afraid of losing data (just unplug them if you get paranoid... My personal choice is Gentoo (hardened edition), which I configure with only the software I need. With Gentoo you can compile everything to suit your hardware, which gives you extra speed and configurability. It is very small upon installation (the word empty comes to mind.... I have found that I can make a very efficient server configuration/maintainability by running this software only:

-----
Gentoo-hardened (Distro)
SAMBA (Win Support for Net)
VSFTPD (Fast and easy transfer to unshared directories, started when needed)
WebMin (Web-based GUI for configuration, comes with own mini-webserver, runs ssl.)
SSH (Certificate-based)
Apache (Webserver, for you an idea might be webbased catalogues of your data..?)
-----

The only thing I actually keep running all the time is SAMBA and SSH (I mainly use rsync and ssh for file transfers and backups, SAMBA shares my printers and controls my windows-machines and "guests"). If I need FTP, I ssh into the box and start VSFTPD, and shut it down when I am done. Same thing with WebMin, and any other services. Never keep them running if you don't need them. If you want to experiment with your server configuration, and you are new to it, I cannot express how good I think WebMin is. It gives you a GUI-based approach to configuration, but still, it not a replacement for good config-file editing....

If you need X you should use the standard edition of Gentoo though, and steer clear of the hardened-edition, X doesn't like the extra scrutiny put on it's code, as I understand it. Also, if you are new to configuring Linux, I would split your OS drive in two between Slackware and Gentoo. Both are excellent OS's which you will learn a lot from (you'll have to...), and in the end you will have a system which is tailored to suit your needs, which in the end is the only way to make a system truly stable; cut away what you don't need....

Also, Slackware is in my experience EXTREMELY stable. You might still get a lot of packages/services you might not want. If you experiment too much when installing, you might break dependencies, etc. Install the whole DVD/CDSet, and use it to get your Gentoo system up and running, then you have your Slackware-boot option as a rescue-OS.

When it comes to hardware-support, this comes down to your kernel, so this is almost identical for all the distros out there, as long as you configure your own kernel. (And I think you should anyway, especially on your server)

My 2 cents, anyway.
Eller skulle jeg sagt 2 øre?

I never really considered Gentoo as an option. The concept of "configuring my own kernel" scares the hell out of me – but I suppose this is the time to start learning. This project is in part a result of me wanting to learn how to use Linux (beyond just creating an OO document or surfing the web). This is why I need the external backup drive to work – I think it will be a long time before the server is working the way I want it to; I have a lot to learn about the Linux kernel, file systems, Samba and RAID configuration. Until then, I'm counting on the backup drive to be my life saver.

Using a web interface for the bulk of the configuration instead of X is fine by me – keeping things as simple as possible. I realize that I will eventually get down and dirty with config-file editing, but I want to take it one step at a time.

On the eternal question of speed vs security, this server is all about security. the server WILL be fast enough for my needs, the main objective is to keep the photo files as safe as possible without not bankrupting myself in the process. I might set up a second, faster file server for media files in a year or two if I feel the need.

Accessing the files from windows is, if I'm not entirely mistaken about the role of Samba, only an issue with the backup drive. NTFS would be preferable, but I understand that NTFS (write) support in Linux is somewhat shaky? Perhaps using ext2 with a Windows driver is a better choice in terms of stability? I am planning on migrating to Linux entirely, but for now, Windows is where I feel at home. Having something that is guaranteed to work in windows makes me feel comfortable.

Choice in distro: yes, I think I would like to experiment a little; keeping the data separate from the OS would allow me to boot multiple distros without risking the data. simply unmounting the data HDDs would let me experiment quite a bit. The USB2 backup would even allow me to experiment with the RAID.

Again; thanx for your input

Regards
Tord Strømdal

P.S: Hvordan får jeg fram "ø" i Linux? Alt+0248 funker ikke... D.S.

Hehe, I remember my first kernel build......I thought "What the h*ll have I gotten myself into?!" .......But it all gets quite simple once you get the grip of it. As long as you have a good overview of what hardware and software you are going to run, it actually is quite easy.

The things that can throw you off is not knowing which modules to use for different types of hardware, but this has actually been one of the most valuable learning experiences for me since I switched to Linux, simply knowing what is inside all of the different types of gizmos I buy and use makes the world of a difference to me. I did for example never know how many manufacturers of network cards who just buy a Realtek-chip and jam it onto a printcard until I started configuring the kernels for them to work. Many things are actually much easier to set up using the kernel in Linux than it is using for example Windows.

Instead of checking the card, finding manufacturers website, navigating to find the correct driver and installing it, I check the HCL here at linuxquestions, almost everything supported in the kernel is in there I think, and if not, Google (read: www.scroogle.org) is your best friend. Once you know what module you need, it is a simple matter of "make menuconfig", checking the mark for the module you need, and "make && make modules_install", and all you have to do is copy them to your /boot and restart. Ok, it might not be so GUI, but it works like a charm. And I REALLY like the idea of being able to un-select all the stuff I don't need..

For you, I can imagine one of the biggest problems in the beginning will be filenames and all the different ways on the web they define how you should do it....I've lost a lot of Norwegian characters the last years... One hint (just my personal experience): steer clear of Unicode.... Missing my ø's æ's and å's has been largely due to my (failed) experiments with Unicode, FAT and Samba. I've actually landed on using ISO-8859-1 all the way, and dropping all special characters in my filenames. It was just the easy way to go. ("slocate ø æ å" and start editing)

NTFS under Linux works perfect when reading, but if write is enabled, it is not able to change the size of the file, just change the contents. (Should be called edit-able, not write-able... So my choice has been ext2 under windows. I have not found any problems yet, and use this on a portable USB/SATA2 drive I carry with me all day at work. (One unformatted Win-partition, and a ext2-partition, works perfect for me, and if someone else plugs it into their computer, they get a "The disk is not formatted, would you like to format" message which at least would stop me from continuing to mess with the drive....

Getting ø's to work in Linux usually has to do with your keypage setting, try "loadkeys no" or "loadkeys no-latin-1" on/in the CLI. I've read 1000 different ways to do this, the important thing is to do the same on all your boxes, or you could end up losing your øæå's.

Which distribution are you running? The setup is different from distro to distro. If you run KDE, you can set it all up from in there, but it will not change your system configuration.

I also come from a windows-only background, and work mainly on windows-systems during my day at work, but I have made a personal choice: If stuff is going to be broken anyway, I want to be the one to break it, instead of having it broken for me at the absolutely worst time I could imagine. You don't update your critical server right before you go to a meeting to show it off, you do it a day later. I felt as if I had no control over when my windows-box would go bust, and felt like my linux-boxes only broke when I started to fiddle 'round with them.... That was the thing that made the choice easy...

If you really want to learn, go for Gentoo or Slackware, they are true to the "Linux way" both of them, and don't wrap you up in a protective layer of cotton wool.... I mean, if a pretty, non-configurable system was what you wanted, you'd buy a mac, right?

You could have a system with Gentoo, Redhat, Slackware and XP/2003server running on the same box with no problems, but in the end, I'd stick to a specially configured Gentoo-box, given that it can be configured so very nicely up to your hardware, and you can leave out all the bits and pieces you really don't need. I personally started with Slackware (STABLE STUFF), and moved on to Gentoo (HIGHLY CONFIGURABLE STUFF).

Hope the thoughts help, in the end Linux is all about making choices, and nobody can make them but you..

Another thing that made my life easier in this respect was vmware.....being able to test things in a virtual environment without having to reboot all the time is worth all the money they ask...

Cheers and regards,
-F