Hello all,

I have a webapplication that is used by 40 clients. (number will increase a lot in the next few months)
2 weeks back I had a sudden server failure, and it took me 2 days to get a different server back up, and pick up the application.

I want to prevent this from ever happening again, and I was toying with the idea of splitting the functions of the server onto several other servers.

In detail, I was thinking of a mail server, a load balancer, 2 frontend servers, a user content server and a database server.

Next, I was thinking of adding a slave/replication database server after the first one with a failover ip.
same thing for the user content server, and the mail server.

Now I have no idea what system specs are needed for each of those functionalities...
What are good distributions to use for each, and what programs are helpfull for accomplishing this.

Right now, my server is running ubuntu 12.04 with postfix for mails, mysql for db, and apache with php for the code.
Everything is hosted on the same server, and a backup server syncs the data each hour.

I've read some articles on a nginx load balancer to apache. But can anyone help fill in the blanks for this setup? I'll need to install and configure everything in this setup, so any and all help or information will be much appreciated.

Thx in advance

You'll never prevent it from ever happening again, however you can mitigate against a lot of the common causes of outage. Anyway.....

Without knowing exactly what your app is / does it's difficult to say, but yeah, that does seem sensible. I'm assuming this is a high-revenue application? Otherwise you'll need to think about a cost/benefit analysis to see if it needs the relevant investment / hosting, or if some form of simpler "warm swap" would be better.

If you've no idea then neither have we! A lot depends on exactly what the load is going to be. You mention you've 40 users at present but will increase "a lot", define "a lot".... doubling to 80 users? 400? 4000?

Any of the "server" distros would be good, personally I'd favour CentOS.

So far all standard, that's good!

Have you looked at HA-PROXY that may give you the functionality you require and would be able to treat your various protocols the same.


Back to my comments about cost/benefit, at the "day job" we have a layered web/app/database structure that's replicated across multiple physical locations with cross-location functionality. This requires a rather large $XXX,000 investment in terms of hardware and hosting.


My "hobby" site which does around 60,000 page displays a day runs on a much simpler platform where the database is replicated from the "primary" to the "warm swap secondary" site in real-time, user uploaded files are synced every 15 minutes and the "warm swap" site takes an hourly database dump and sends that to a small VPS in a third location. Total cost, including hosting, of under $300 a month. Hardware and O/S are identical between "primary" and "secondary" so it's a relatively simple matter of a couple of manual DNS changes to redirect the traffic in the event of a failure. This works fine for my needs, so it does all depend on the budget you have.

Well to give some more information.
The 40 clients aren't 40 users.
It's a management system for schools, so the 40 clients are 40 different virtual hosts, with each their own database, and user files. Teachers, personnel and students can log onto the system, so it's fairly well used. (about 500 users a day, with an average of 100-150k page views / day. The downtime was during the weekend, so the user base wasn't as impacted as it would have been during the week, but still.

At the moment I have 3 servers, of which one is a dev server, and one used to be a fallback server, but in the past 8 months (the time I wasn't in charge of the application) it was removed, and a empty backup was set in it's place. (that was a fun moment when I figured that one out)

the 3 servers are identical and have a cost of only € 50 / month (each). But they are all managed by me (at least they are now)

It is very important that we can remain active to draw in more schools. which we expect to increase to 200-300 schools within 2 years. which means a lot more server load, so the load balancing might be a good idea.

The application in itself allows the generation of dynamic word and pdf documents (which is quite CPU intensive) but it also contains loads of data for each school (official data exports that need to be send the the government and such.)

so average disk usage of each school is about 1-2GB / year (average).

Are there any specific specs that I need to keep in mind when deciding on a mail server, a database server, a user-content/data server and the frontend server(s)?

if any additional tips can be giving according to the added data, it'll be more than welcome.

Thx again