Starting a server build
 

Hi Everyone,

I will be starting to install Linux on my previous computer tomorrow evening, and was looking for any ideas, help, advice, etc. that anyone has to offer.

Firstly, the specs of the server machine:
MSI k9n SLI ready motherboard
8GB 800mhz OCZ RAM
Dell 300W (maybe 350W) power supply
AMD Athlon 64 X2 6000+ 3.1 Dual Core processor
2 X 750GB Western Digital Caviar Black SATA hard drives (plan to run in a raid setup to mirror)
Radeon ATI HD 4650 Video Card (mobo has no on board video)


So far, my intentions for this server are:
1 - website hosting (my few personal websites, and maybe a few "pro bono" sites for friends/family members

2 - website development / testing (php and mysql, etc.)

3 - file sharing on my home network (not looking to share online with strangers)

4 - music and video streaming (not necessarily streaming from various online sources... I want to be able to stream my own music and video files that I've downloaded to devices on my own home network

5 - of course, security... I need all of this to be secured so no one can have access to my data outside of my home network. I don't want my server to download random files, or allow anyone outside of the home network to access the server and download random (potentially threatening) files to my server (I don't know if this one is possible anyway).

6 - once I have the OS installed and running smoothly, I plan to begin accessing the server remotely from main, everyday use, computer.



I will be heading to Micro Center in a couple hours to pick up a few SATA cables and prepare to do the install and raid setup. The functions above are the basic / main functions I can think of and / or care about right now. I'm sure once I start diving into the world of servers I'll find more for it to do?

Please, any advice on the system, how to run it, issues I may run into, Linux distributions, or anything that comes to your mind, share it. I figured I'd throw up this post to see if it will help me with keeping the installation(s) as smooth and simple as possible. It could also help me find some other interesting server functions to run / install..

* I've had a pretty basic introduction to Linux, but haven't played around with it much at all.. So, I'm not too positive of what all I'm capable of doing with it and how to go about it.


Thanks much!!

Aunnix,

I would heartily recommend Debian Stable. Does all the things you are after, plus a lot more.

Steve

Thanks. I've also been told to check out Ubuntu (12.04 or something?) as it is supposed to be very easy to install and setup, and I've been told about Cent OS or something. Any comments on these builds?

ks8
 

I have debian squeeze (stable) running on my older single core desktop with one GB of ram. I use it for my photo web server. Debian is easy to install. Just download the netinstall. When it comes to partitions, I recommend using one single partition, or 2 (/root with 15-20 GB and /home for the balance of the space in your HDD). You also need to have a small swap (e.g., 1 GB).

Ubuntu is based on debian, so most commands are alike.

Put a little time into developing your firewall , if your going to be exposing the server to the world your going to get whacked as soon as it connects to the world. figure out what ports you need and how to control access . learn what a "host.deny" file is and how to use it. Learn how to use the log files to ferret out problems. running a server is a lot of fun , I've had a linux server for my home running for the last 15 years. Have a back up plan and do your data backups. For each service you run there is going to be some type of configuration file , learn to read them and how to tighten them up , as they all come with default settings that need to be tailored to your own situation.

and don't forget a backup power supply UPS , and consider using detachable drives instead of mirroring , physically rotating the drives with spares pre configured and ready. RAID is great but a pain in the ass to restore from.

have fun

Greg

Yes, secure server is important once it is open to the internet. I followed this tutorial to install the apache server:
http://aboutdebian.com/internet.htm

There are other useful how-to in there.

Care to elaborate on the partitioning? I assume the /root would be what houses the OS/Linux distro? does the /home partition just the rest of the HDD space where I'll be storing my files? I'm confused about the "balancing of space" comment, lol.

Yes, /root is for the system files. 20GB is plenty. You can back up the image with GParted-Clonzilla easily and fast (clonzilla is a package inside GParted). /home is where the user data are. You can back it up with grsync which can do incremental backup(come as debian package).

I also use ext3 (partition type) for /root, and ext4 for /home. You can read about them in aboutdebian.com

I assume I should develop the firewall before setting up the configurations / functions? So far, I'm under the impression that once I install the OS I'm basically ready to go online. Also, can you refer me to any good resources on reading up / figuring out the ports issue(s)? I assume for a backup power supply, there is some type of external PSU? I can't run a second in the ATX tower I'm using, lol, and I don't want to spend any more money on this, hahaha.

I will probably start out mirroring (so I can get everything running for as little cost as possible) and once I start moving data around significantly I'll have to look into the detachable drives. Is this the same thing as "hot swappable"?

Cool. I plan to use as little partitions as possible. I figured it will make it easier for storing files and mirroring. Incremental backup?


What do you mean small swap? I need a third hard drive?

---------- Post added 08-15-12 at 04:32 PM ----------

Thanks! I do plan to use apache so this should come in handy.

Incremental backup = backing up only the new files, and files that has been changed since the last backup. This speeds up the backup process.

Swap - is a third partition that is required. It is like a page file in MS. Since you have so much memory, you will probably never use swap, so make it small, 1 GB is plenty.

Definitely check out the Security forum.
I'd also recommend looking at Centos as the OS (free rebuild of RHEL).
Lots of manuals here www.linuxtopia.org; you probably want the SysAdmin list http://www.linuxtopia.org/online_boo...ion_index.html

You might find this tutorial useful as a good starter http://rute.2038bug.com/index.html.gz

Ok, so I'll actually have 3 partitions? Above (in earlier post) you said you recommend only one or two partitions...

---------- Post added 08-15-12 at 08:50 PM ----------


Thanks!

If it was me, I wouldn't bother about separate partitions under /, especially since you will be serving large multi-media files, so the size of, say, /var or /srv could be quite big.

I also would use a smaller swap partition than 1GB, that seems big to me. You will likely never use it, and 512MB would be plenty.

I run Debian Squeeze on two servers, FreeBSD on another, and RHEL on another. I've tried CentOS, and found Debian easier to administer for a server that is for my own use. FreeBSD is great as a server (and OpenBSD even more so) , but if you're a linux newbie you may find the BSDs a bit intimidating.

I run a number of servers (Debian, Ubuntu and CentOS), but I chose to run Debian at home because of the huge number of packages available, the fact that it a rolling-update install (ie no need to reinstall when the next version comes out) and it just keeps going with little (no) maintenance.

My preference would be to go for a 5 partitions;

10G /
5G /tmp
5G /var
RAM+2G swap
/home "the rest"

Given that you have 750G drives the amount taken up by the OS is small. You might even want to consider using an SSD drive for the OS then use the two 750G drives for /home mirror.

I would have a good long "play" with setting things up. Try different setups and just enjoy the experience.
I don't have a firewall running on my server as I have a NAT firewall on my DSL router, so only WWW and SSH ports are directed to that machine (and SSH is on a none-default port, away from prying eyes)

Saying that, I do use fail2ban to drop connections from people trying too hard to access my server.

Don't put all your data on the machine until you've had a GOOD LONG PLAY!

and BACKUP your data!!!!! (rdiff-backup is great for incremental backups, and clonezilla (mentioned previously) is great for taking a snapshot image of the OS).

padeen - I have never used a /srv partition. Today was the first day I've ever looked in that dir and there is nothing in it.

It's the new hierarchy for served files, such as web, etc. RH are moving to it and Debian looks like they are too. I recently installed a Debian Squeeze package of a web-server and it set up /srv (but admittedly with soft links to /var/www).

padeen - thanks for that info.

It sounds like Debian is the way I should go, being a newbie? I've been told Ubuntu is pretty easy to learn and get the hang of, and since it is based of Debian they are similar. Then, per sjreilly above, Debian has alot of packages available and I like the idea of the rolling update.

I have actually considered the third hard drive to house the OS and supporting files, but my I don't think my power supply has enough connections for all of them, lol. And, I'd really like to avoid spending more money. It may not matter much, but I'm already using a molex splitter to run an extra fan and I'm trying to keep the splitter adapters to a minimum, lol. I don't know how much my power supply can handle because I've been told the Radeon 4650 is a power hungry card. I did actually come across an old video card, so maybe it still works and can be used to consume less power. I don't plan to keep a monitor attached to it once everything is setup (maintenance will be done remotely) but I'd like to keep a video card in it in case I do need the monitor.

I didn't really plan to transfer any of the data until all packages/functions are setup and secure. Figured there is no point in wasting time transferring everything if I can't get the system functioning, lol. So far, I only have about 25GB of data (I'm not much of a file sharer / donwloader, lol) and it is all on my windows machine. I will keep this in a backup folder until everything is functioning and can be safely retained on my 750GB hard drive.

In the event I can hook up a 3rd hard drive... what would be your suggestions on the size of hard drive which would be enough to house the OS and system files?

I think a 40G SSD would work great. You don't need that much space, but you don't want to be running at capacity either, especially on an SSD.

I would steer clear of Ubuntu as well. Yes it's based on Debian, just like Fedora is based on RHEL, but that doesn't mean it's a good alternative for the same applications.

Thanks! I don't know if I'll have the cash to spend on a SSD, but I do have an IDE 40GB drive that I could throw in.

I do believe I'll start with Debian and see what happens. Does Debian have a desktop feature? I planned to install a desktop feature just to have it for learning purposes (and in case I can't figure out something in command line, maybe I can going through the desktop). I assume since it will be a server that no one is physically using everyday, I'd want to keep it pretty basic? Sounds like Debian is my best bet so far...

Great Installation How-tos at http://debian-handbook.info/browse/s...ion-steps.html and http://www.howtoforge.com/perfect-se...nx-ispconfig-3 (replacing nginx with apache - s'up to you)

Steve

You get an option to install the desktop packages (along with those for File, Web and other services) during the install.

A useful tool for server Admin is Webmin (http://www.webmin.com) which provide a web interface to many of the services that your server will run - its one of the first packages I install (not, however, available in the repositories but you can add the Webmin repo or install the .deb package).

I jut purchased a 60G SSD for £40 - which is pretty much all you would need for the OS (more than enough actually assuming you move the web site onto your main drives)

Your IDE drive would also fit the bill.

Steve

Thanks for the links! And yes, that should definitely be enough space for the OS... I don't plan on having anything stored on the same drive/partition that the OS is installed on. Basically, I'd like the OS drive to house the OS and any installed packages I may use and the 750GB drives would be for only for storing data.

As you are planning to allow web access to your machine you should also get a domain name from your ISP, or if you have a fixed IP address from your ISP you could buy a domain name.
Or you can register free with DynDNS (http://dyn.com/dns/), use one of their domains and use the ddclient package to update the global DNS tables if your IP address changes.

Also consider running rkhunter(Rootkit Hunter) in order to check if you have been hacked.

Steve

Thanks! I will be calling my ISP today, so I will ask about this as well.

Of course, your ISP will try to get you to use space on their server, which isn't what you are after.

One more limiting factor for a web site on a home server is upload speed - I have 12M download at home but 1M upload. Just remember that if you expect people to be able to download big files or pictures from your site(s).

hmm.. I have the "hi speed" connection or whatever for residential(home) services which I believe is at 8M. I'm not sure what the upload speed is. Shouldn't I only need to be worried about the download speed if I have files for viewers to download? I would assume the upload speed will only matter for the files I upload to my server through a website or for files that users/viewers can upload to my server/network (which I definitely do NOT plan to do, lol).

Their "download" is your "upload".
External viewers won't be able to dowload from your site faster than you can upload content.
Within your home network you are, of course, only limited by your home network speed - wireless, 100M or 1G LAN.

Ah, ok.. makes sense. I will ask my ISP what my upload speed is to see if I need to increase it. What would be recommended as an upload speed? Starting out and in terms of long run (say for the next year or so)?

erm, /root is for USER root's files, it should be the same partition as /

as for partitions, i would stick with as few as possible
say
/
/home
/var
/etc

that covers user accounts, most global config files, and many common daemon's files such as mysql, and apache etc..

No need to call the ISP, just stop all Internet traffic and run a bandwidth test (google "bandwidth test" and run any of the top 5 results). That'll tell you what your actual upload speed is, not what the ISP says it should be.

Yeah, I'd like to use a few as possible. It seems like it would be easier to keep organized, and since it's going to be basically a "storage" server, I don't want to have to search through several partitions to access some data.

So / (or /root) will house the OS and packages installed?
/home will house my data being stored?
what will /var and /etc be used for?
Where does this "swap" partition (mentioned earlier in the thread) come into play?

How do I stop all internet traffic? Am I just making sure that no one is accessing the internet while the test is performed? Thanks for the info. I do hate talking to the ISP because they'll say I have the 8M connection but they don't tell me what my connection speed actually runs at and that the 8M is the cap for my connection, not my actual connection... lol

Just make sure nobody is downloading/uploading anything, shut off the web server, ftp server, stop all torrents, etc. You don't have to do anything special, just make sure the bandwidth tester has as much bandwidth as possible so you get the best reading, otherwise it'll read lower than actual.

You might want to check what your ISP has already published regarding terms of use. My ISP says 'No servers allowed', but this is not enforced unless they have a reason to. I've been running a web server on my home internet connection for years, and never had an issue with my ISP about it, but then, I never told them I was doing so. I would think that if you keep your overall throughput within your quota, and possibly some ratio of upload/download reasonable (my ISP does monitor this, apparently to monitor torrent servers violating copyright infringement), you should be able to fly under the radar. If you ask them about the spec's related to running a server, they will probably tell you that you need to pay for a commercial service.

--- rod.

/ holds the OS and packages

/root is the root user's home directory, and looks pretty much like any regular user's home directory.

/home holds the home directory for all users (except root), along with all of the data they place in their home directory, their preferences/settings for web browsers and email, any files they have on their desktop, their desktop settings, etc.

/var holds misc OS log files and some other things

/etc holds misc OS settings files

swap is used when your RAM fills up, it's like Windows' page file. When you max out the RAM, the OS starts transferring data from RAM to swap to keep the OS from locking up. It's used a bit here and there for some RAM optimization as well, but the big purpose is to give yourself some breathing room if you run out of RAM, so the computer doesn't come to a screeching halt.

"data" can be placed anywhere you like.

When you say "keep your overall throughput within your quota..." what exactly do you mean? My quota of what? Connection speed(s)? Since I'm not much of a file sharer, I would think that I can keep within this "quota" of whatever it is for... And the websites hosted will basically be portfolio/resume type websites.. maybe a blog or something.. I mainly want a web server where I can test PHP and mySQL development without having to login to godaddy and then login to five other admin areas to complete some testing. I doubt I'll be offering very many files for users/viewers to download.

When I called my ISP a week ago, they understood I wanted to setup a server and didn't say anything about needing to upgrade my service. They transferred me to their tech team, who apparently couldn't understand what I was wanting to do. I told him I wanted to run a small server to host a personal website or two and wanted to make sure I was allowed to do so and asked about them scanning for open ports and closing them on me. He sounded confused and basically told me to call them back when I set it up and they will help me as much as they can.. haha.

Ok, got ya. So, I'll end up with 4 or 5 partitions, but everyone is recommending that I only use or two partitions.. are they referring to the partitions outstide of the OS partitions (/, /root, /var, /etc)? So only have like 1 or 2 "/home" partitions where I will be storing my data?

How difficult will it be to setup the 3rd HDD for the OS files so I can leave the 2 750GB completely intact for the /home partition/data storage?

Many ISP's limit your data throughput to some number. My ISP has different levels of service with different speeds and different
monthly GB transfer limits, for different prices, of course. I once had my service cut off temporarily for exceeding my quota. My provider currently has limits of 125, 200, 400, 500 & 1000 GB per month for their various plans. These numbers were originally much smaller (I think as low as 2GB per month). Now that I've checked those numbers, I see that they also publish both their upload and download speeds, so maybe they've changed their policies regarding servers.
--- rod.

You need to remember that partitions in Linux are not like partitions in Windows. In Windows, every drive/partition is completely separate from the rest. You have your C: drive, you have your D: drive, E:, F:, G:, and so on. Each drive/partition is completely independent, and the kicker is, each one has its own directory structure. If your OS is on C:, it will only ever be on C:. Getting your C:\Program Files to actually live on D: is not exactly easy, for example.

Linux is different in that there is ONE directory structure, which can be distributed across multiple drives if you so desire. The root of the directory structure is "/", EVERYTHING, no matter what physical device/partition it lives on, is in a subdirectory of "/" (or a subdirectory of a subdirectory, etc). You will always have an /etc directory, you will always have a /var directory, /home directory, /root directory, and many others. It is up to you whether you want these directories to lie on their OWN partition, or to simply be subdirectories of / on /'s partition.

What the above people are saying is that you might want to place /home on its own drive, separate from everything else in /. Whether you want to do the same with /etc or /var is up to you. If you don't put them on their own partition, they will still exist, they will just live on the same partition as / rather than their own, separate partition.

Setting up the OS on a third drive outside the RAID is quite easy, and in many ways it's actually easier than trying to install everything on the RAID. The reason is if in the future you decide you want to wipe the OS and install a new one, getting the new OS to recognize and correctly configure the RAID DURING the installation process can often be tricky. However, you can easily install the OS on the single drive, and then rebuild the RAID once you're in the OS and re-mount it where it's supposed to be.

I did this exact thing recently. I had a Fedora 10 system, OS on one drive, /home on a 4-drive 3TB RAID 10. I wiped the OS and installed CentOS 6.3 in its place. CentOS did NOT recognize or configure the RAID properly during install. When I was up and running everything was on the one boot drive, and fdisk told me I had four 1.5TB drives just sitting there. However, once I was in the OS I was able to use mdadm to scan and rebuild the RAID 10, edit /etc/fstab to mount it back in /home, rebooted, and my home directory, all of its contents, and all of its settings was exactly how I had left them on Fedora 10.

This is the advantage of separating parts of the filesystem onto other partitions. /home is usually the first one to make it onto its own partition, and depending on what you do with the system you may decide to move others as well.

I would go with grabbing a download ISO of parted magic (http://partedmagic.com/doku.php?id=downloads), boot from the burned CD with all the drives fitted then you can use gparted to partion each drive before installing the OS. Parted Magic also has Clonzilla on the CD so that you can image your OS drive to provide a backup once you are happy with your setup.

Yes, these are all just our own ways of doing things - serving suggestions.

Like I said before - Have a play. Make mistakes (I've made plenty) but you can start again and learn from the experience.

a standard desktop install can get away with 2 partitions, / and swap, though keeping separate partitions for /home, /var and /etc, although not necessary can make any persistent changes to OS settings, personal files, and logs and home directories able to survive a wipe and reinstall of the OS in case something gets hosed. this is why separate partitions is often seen, though a bare minimum of / and swap is all that is technically necessary on modern hardware, on older units a separate /boot partition was needed at the beginning of the drive to hold the kernel and second stage boot loader in a section of hard drive accessible by the BIOS, but that is irrelevant nowadays.

I am more concerned with runaway processes creating log files that fill partitions - that's why I have three partions on servers - to help prevent the server from falling over.

And on desktops / and /home

indeed, if /var is i'ts own separate partition and fills with logs, it won't crash your system, but if /var is under the same partition as /, and / fills, it can bring your server to a grinding halt.

Ok, I will see if I can find out this information in the TOS or something before I call them. Is there a way I can check the history of the data throughput to see how close I am coming to my quota? Or, is that something the ISP can provide?

Makes sense. I just went to Micro Center and grabbed a used 60GB IDE drive to run the OS on, so it looks like I'm planning the same setup as you basically. /, /root (if it's seperate from /), /var, /etc will all be on the 60GB while my 2 750GB HDDs will be /home in Raid 0 or 1 (I still need to do some research on the raid setup, but Micro Center told me I don't need any Raid cards for 0 or 1 so I figured I'd go that route).

I see. So, I plan to use the /var and /etc partitions or directories because it seems like (although they are not required) the system may run a little bit smoother so it can fall back and utilize those directories when needed. Perhaps it may save me some troubles later?

I did grab a 60GB HDD to install outside of the 750GB HDDs, so the OS will be on the 60GB. Would you suggest putting the /var and /etc directories on the 750GB where the /home will be so they are on seperate partitions from the OS? At the moment, I'm thinking only /home on the 750GB HDDs and ALL OS files/partitions/directories on the 60GB for a clean OS reinstall later (if needed).