Hi

I have a running, live webserver running PHP and mysql, and it cannot be taken down except for very short periods in the middle of the night.

The PHP and mysql versions are now getting quite old, and I want to upgrade them, however I can only think of a very clumsy way to do it, which would take a few hours and which may indeed be irreversible if things go wrong.

I've heard about setting up a virtualised environment and setting up the new PHP and new MYSQL in it and then hot-changing them over. Ehem, but that all sounds like marketing speak.

I'm looking for some practices here though.

Probably I'd need to replicate the content environment of the current website and try to rerun it under the new php and mysql, and so have a decent test environment.

Any advice from anybody?

I recently was contemplating this, but it was an ancient server, so I upgraded the hardware as well.

Do you have physical access to the production machine? If so, have you considered installing a hot-spare, and switching over to that during the upgrade?

The biggest thing I would worry about is that there were some major changes in both MySQL5 and PHP5. I had a few PHP4 scripts that wouldn't run under 5, specifically anything using mcal.

What works realistically for you may be different from what works for me. How I would go about it:

First setup a "test" server running the same OS distro, with the new PHP and MySQL. Copy everything over (database dump and copy the web root) and test all of the PHP scripts to be sure they work with PHP5.

Assuming all goes OK, sync them, bring the production server offline and switch over to the "test" server, upgrade the production machine, test that everything works, and then switch back to the production machine. If databases may change, you should lock them when the "test" machine is serving content.

If you have a distro that uses packages, upgrading MySQL and PHP and all of their dependencies is pretty quick (using RPM, apt, whatever).

Really your best bet is to do everything on another machine and then swap on over. You will probably want to script a export->import of you DB from the old machine to the new machine, and run the script immediately before switching over.

If you try to modify your running system, you will break it. Murphy is in full effect here.

Ok, you can't.

Are you building MySQL and PHP/Apache by hand or are you installing from RPMS? If you are doing the RPM thing, then make a machine that is a clone of your old one and use that to test the upgrade. Just try the rpm -U <RPMS> and see what breaks.

If you are building by hand, you can always build them in a separate directory using non standard ports. Once you get it working, use iptables and port forwarding to the new port 80 replacement.
Ideally you would build on a machine that is identical to you live machine WRT OS version, and then copy the resultant files on over.

Another option would be to run VMWare and use that as the web server. You can do a complete install within VMWare. This is how the big ISPs give everyone their own login without giving them their own hardware. Xen might work for you as well, but I am assuming that your Linux box is old enough to not support it.

Hi jantman and admiyo,

Many thanks for taking this one up! Appreciate it!

No, I don't have physical access, it's a hosted server. I only have one, and its Red Hat 7.2.

Actually I know that's already bad news.

Yes, admiyo, you're right. I shouldn't even try to tempt Murphy. I hadn't thought of the non-standard ports, many thanks. Though I'll admt, I'm very weak on iptables. Yes. I'd build by hand, mainly out of habit.

I could try to replicate at home, or suscribe to another server, a testing one. I presume that's what a pro would have, a second, testing server. Maybe that's a recognised best practice in these cases. I don't know. that's why I'm posting up here in the Enterprise Forum.

How important is it that the test server be the exact same distro as the production server? I have to say: not much (I don't think). Can any one verify? It's an oscommerce website I'm running. I don't reckon the PHP and mySQL are making version specific calls to the OS. I wouldn't bother upgrading Apache. That, at least, would stay the same.

I suppose I can test all this by replicating the current environment on my Slack 11 and seeing if it works OK. That should give me the answer to the different OS question. I think I should look into moving out of Red Hat 7.2 into Slack 11.

As it's an oscommerce app, there is the area of transactions. That could be tricky, esepcially when testing.

Thanks for the heads-up re. mcal. Yes, I heard of some incompatibilities. I should check out for those before hand. Ref. db dump and all that, yes, fully agree.

In short, the idea of setting up a test environment is totally uncontested: it has to be done. I expected as much. The next problem is when all the testing is done, how to do the changeover in a smooth a way as possible.

In relation to the virtualisation. I was thinking of setting up a UNIONFS partition and chrooting into it. How does that sound? Could I set up a virtual environment within that chroot? Just surmising.

But yes, VMWare would sound like the way to go. Everybody is doing it, right, I know. Interesting that so many SPs are doing it now, I don't think they advertise the fact though. But it does explain some of the cheaper dedicated server offerings coming up now. I wish they'd come straight and say it was VMWare.

Honestly, I think the safest way is to inform users a few days ahead of time that the system will be down for such and such a time, and lock the tables when you start the upgrade. If there's money changing hands, you don't want anything to go wrong...

Personally, I have a spare machine for my *personal* web site, ready to bring up with backups in a few minutes... where I work, as a programmer for a University, we have a stack of spares in another room, ready to bring online, as well as hot spares.

In terms of testing, it's a Murphy situation. It wouldn't be the first time that I had two installs of the same PHP and MySQL on only slightly different OS versions, perhaps patched slightly differently, and things weren't the same on the test and production machines. For a real test, you want things to be as similar as humanly (or mechanically) possible....

I'm just starting to roll out a new backup system on my home network, which I use for hosting my personal site, my personal files and (yikes!) *all* of my development work... I bought two identical servers, same batch, same controllers, and used Ubuntu LiveCDs to do a dd from one to the other. I'm now mostly confident that the machines will produce adequate test results...

OK, I've been there before. Again, VMWare is your friend.

On your home machine, install a VMWare instance that is Identical to the setup on the hosted machine: RH7.2, MySQL etc. Take a snapshot of it so you can easily roll back.

Now make a copy of this snap shot and get your app upgraded. Once you have it working the way you want, script a deploy from one VMWare instance to another. This way you can test not only the app, but how it will look on the live system. Since it won't work right the first time, you can run it time and again against a restored copy of the snap shot.

If you do upgrade the OS, make sure your hardware is supported. lspci is your friend. You don't wnat to be halfway through an install and find that your video card or NIC is no longer supported.


The ideal solution is not only to set up a duplicate server and get it working on there, but then to swap the stagin and live servers.

Best of luck.

Many thanks both of you for your comments. I really appreciate the words when they backed up by experience. Cheers!