LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Enterprise Linux Forums > Linux - Enterprise
User Name
Password
Linux - Enterprise This forum is for all items relating to using Linux in the Enterprise.

Notices


Reply
  Search this Thread
Old 08-23-2018, 05:38 PM   #1
Linux-cipher
LQ Newbie
 
Registered: Aug 2018
Posts: 8

Rep: Reputation: Disabled
Wink Patch Management Advise


Hello everyone!

I'm new to this forum and hope to provide you guys with advise as much as I could do with it!
I've currently supporting a environment which hosts a mixture of Ubuntu and Centos Linux Servers, which need patching!!

Most of the servers are running websites, which are running out of date versions of ubuntu! Every time I've gone to do an system update and distribution upgrade MySQL or PHP get updated in the process which breaks the website which is bring hosted So I end up restoring from check point.

I could really do with some advise on how to upgrade or update linux servers the safe way?? (Any advise would be helpful)

Also is there a way to exclude certain updates from being applied?

Thanks in advance

Cipher
 
Old 08-23-2018, 09:58 PM   #2
mralk3
Slackware Contributor
 
Registered: May 2015
Distribution: Slackware
Posts: 1,900

Rep: Reputation: 1050Reputation: 1050Reputation: 1050Reputation: 1050Reputation: 1050Reputation: 1050Reputation: 1050Reputation: 1050
Patch Management Advise

How many machines are we talking about here? I have never been in your position before but I suggest you set up those web sites in containers in the future so that you can easily maintain your systems. Then set up ansible to maintain everything all at once. The containers will make it simple to tear down and bring up new websites.

For the containers you can use Docker or lxc.

Last edited by mralk3; 08-23-2018 at 10:02 PM.
 
1 members found this post helpful.
Old 08-26-2018, 11:42 AM   #3
upnort
Senior Member
 
Registered: Oct 2014
Distribution: Slackware
Posts: 1,893

Rep: Reputation: 1161Reputation: 1161Reputation: 1161Reputation: 1161Reputation: 1161Reputation: 1161Reputation: 1161Reputation: 1161Reputation: 1161
Linux-cipher,

You did not share exactly what you meant by "out of date versions."

I manage several systems running Proxmox/Debian, CentOS, and Ubuntu Server. Proxmox is a virtualization platform for running KVM and LXC containers. Most of our systems are LXC containers.

Proxmox is free/libre and costs nothing to use. A community subscription gains a tad bit better tech support on the forum but is not required.

We have bare metal rack servers running CentOS 7, used primarily as backup systems.

Back at the beginning of 2017 we updated the CentOS 6 systems to CentOS 7. Basically we installed fresh and then slowly tweaked each system until fully functional again like the CentOS 6 system.

At the end of 2017 we updated all of the Proxmox systems from version 4 to 5. We did that one system at a time and let everything settle for a couple of weeks before updating the next system. As Proxmox uses Debian as a base, we just did a typical apt dist-upgrade. As might be expected there were some minor bumps but we have a dedicated single Proxmox system that we use for testing to discover those bumps.

I have no problems updating any of our systems, but all systems are using a supported LTS release. With trivial updates I update all systems at once. With serious updates, such as kernels or systemd, I update in a rolling manner with only a few systems each day. That allows me to watch for issues. I use the standard apt or yum commands as appropriate.

If your servers are running really old releases, then a fresh install probably is the only sane avenue. Converting to containers would save a lot of time and overhead. A single rack server can host many containers. If that sounds promising then ask the owner to buy a refurbished server. Something with dual sockets multi-core Xeons and 32 GB of RAM with a hardware RAID controller will cost less than $1,000.

If the owner of the environment does not want to go that route then likely you are stuck with prototyping the old systems on a spare system until you get everything tweaked.

I hope that helps.
 
1 members found this post helpful.
Old 08-30-2018, 05:47 PM   #4
Linux-cipher
LQ Newbie
 
Registered: Aug 2018
Posts: 8

Original Poster
Rep: Reputation: Disabled
Thumbs up Updating Older Releases!

Hi Guys,

Sorry! I should of maybe put a bit more information than I did!

So I work as an Infrastructure engineer! We have over 30 Linux Servers sitting on VMware Servers which are hosted in-house, so the servers are already virtualised. Most of the servers a running older versions of Ubuntu. (12.04)!! The servers run internal websites which uses MySQL Database and apache etc etc...

I've been tasked to update these linux servers to the latest release (18.04)! I just needed to know some tips and ticks you guys may know or have learnt along your linux careers.

I've already tried to upgrade a Ubuntu 12.04 to 16.04 but for what ever reason MySQL broke and Apache didn't want to know. Is there any way to maybe upgrade OS without updating MySQL, Apache, or even PHP? As these Linux Server has custom PHP applications which require older version of PHP.

My vision is to try keep current applications on the same versions and to update the Ubuntu to the latest release. Is there such a thing?

Any help would be grateful.

Regards

Cipher
 
Old 08-31-2018, 12:37 AM   #5
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 19,872
Blog Entries: 12

Rep: Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053
Quote:
Originally Posted by Linux-cipher View Post
I've already tried to upgrade a Ubuntu 12.04 to 16.04 but for what ever reason MySQL broke and Apache didn't want to know.
as a debianite i can say that dist-upgrades only from one major version to the next, so on ubuntu that would be 12.04 => 14.04 (carefully check that everything went ok and works) 14.04 => 16.04
etc.
captain hindsight says: you should've done this much earlier.

Quote:
Is there any way to maybe upgrade OS without updating MySQL, Apache, or even PHP? As these Linux Server has custom PHP applications which require older version of PHP.
good question. i don't know if it's possible within apt. "apt pinning" comes to mind, read up on that.

Quote:
My vision is to try keep current applications on the same versions and to update the Ubuntu to the latest release.
it's still a security issue.
 
1 members found this post helpful.
Old 08-31-2018, 10:15 PM   #6
upnort
Senior Member
 
Registered: Oct 2014
Distribution: Slackware
Posts: 1,893

Rep: Reputation: 1161Reputation: 1161Reputation: 1161Reputation: 1161Reputation: 1161Reputation: 1161Reputation: 1161Reputation: 1161Reputation: 1161
I am not familiar with VMWare, but as these systems are all virtualized, you should be able to clone them and test updating on the clone.

As ondoho noted, do a dist-upgrade. Do not try to jump releases. Slow but more likely to succeed.

With respect to Apache, I think 12.04 started with version 2.2 and then along the way the version changed to 2.4. There are some config and syntax differences but overall you should be able to transition.

PHP has gone through many releases since 12.04. You'll have to look into compatibility issues to resolve any breakage.

I don't know why MySQL broke. MariaDB is the new package name. A fork of the original MySQL but 100% compatible with the old MySQL.

With respect to the CentOS systems, 6.x reaches EOL in more than two years. Don't bother trying any kind of distro upgrade. Install CentOS 7 fresh and methodically migrate services and config files. There was a lot of discussion about this when 7 was released and the bottom line was do a fresh install. But the 6.x EOL provides plenty of breathing room to act as opposed to Ubuntu 12.04.

Ubuntu 14.04 reaches EOL April 2019. That provides some breathing room to shake down issues. Update to 14.04, stabilize for a month or two and then update 16.04. Rinse, repeat.

If the servers are internal and do not face the public web in any way, you do not need to panic. If they face the public, then consider external firewalls to prevent penetration.

Trying to update the OS while holding or pinning the apps is asking for bats out of Hell. You will age quickly and likely go nuts trying to maintain everything.
 
2 members found this post helpful.
Old 09-03-2018, 07:58 AM   #7
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 26,553

Rep: Reputation: 7946Reputation: 7946Reputation: 7946Reputation: 7946Reputation: 7946Reputation: 7946Reputation: 7946Reputation: 7946Reputation: 7946Reputation: 7946Reputation: 7946
Quote:
Originally Posted by Linux-cipher View Post
Hi Guys,
Sorry! I should of maybe put a bit more information than I did!

So I work as an Infrastructure engineer! We have over 30 Linux Servers sitting on VMware Servers which are hosted in-house, so the servers are already virtualised. Most of the servers a running older versions of Ubuntu. (12.04)!! The servers run internal websites which uses MySQL Database and apache etc etc...

I've been tasked to update these linux servers to the latest release (18.04)! I just needed to know some tips and ticks you guys may know or have learnt along your linux careers. I've already tried to upgrade a Ubuntu 12.04 to 16.04 but for what ever reason MySQL broke and Apache didn't want to know. Is there any way to maybe upgrade OS without updating MySQL, Apache, or even PHP? As these Linux Server has custom PHP applications which require older version of PHP.

My vision is to try keep current applications on the same versions and to update the Ubuntu to the latest release. Is there such a thing?
Aside from the wise "don't wait so long again" advice given previously, there are two bits of advice I'll throw in:
  1. Do *NOT* do in-place upgrades on production systems.
  2. Do *NOT* keep old versions of software around if you can help it.
To the first point: I, and many others I'm sure, have been bitten by this numerous times in the past. Old versions of libraries kept laying around, multiple copies of things, corrupted package databases, and a gazillion other things can crop up. If you're in a VM environment already, spinning up a new VM and doing a fresh install should be trivial. Copy your configs from the old server to new, TEST IT thoroughly, and then move production to it. Step through one by one, and get it done, until you're up to date. You will be in a FAR more stable environment then. As an added bonus, you need to use this time to document everything, clean up old junk that gets left around (as it always does), and you'll know your environment inside and out when you're finished.

To the second: update the software, if you can. Keeping old software around because "it just works", or "migration will be hard and cause downtime" is plain foolish. Your existing software/server **WILL ABSOLUTELY DIE** at some point, and there's no getting around it. Right now, you may be a short way behind the current release, and there may be an upgrade path. If you wait....that path may be GONE, and upgrading will be far worse. And would you rather upgrade while the existing system is still available in production, or be forced to do it at 2 AM one night, when the server dies, and your company is down until you get it going again? Take the time to do things right...upgrade the older software. Don't keep the same versions of applications, if there are newer ones available.
 
2 members found this post helpful.
Old 09-09-2018, 04:11 PM   #8
Linux-cipher
LQ Newbie
 
Registered: Aug 2018
Posts: 8

Original Poster
Rep: Reputation: Disabled
Hello everyone,

First of all, just wanted to say thanks for your feedback.
The feed back gives me a plan to work towards! I've only recently started at this new place. So I have a lot to work towards to bring them up to speed.

Plan moving forward.
1. Create an update schedule
2. Apply all security updates and system patches.
3. Migrate older linux boxes to new VMs.

If anyone could recommend any videos/training guides/youtube links to help bring me up to speed with linux migrations please do so.

Thanks again people.

Linux-Cipher
 
Old 11-12-2018, 08:00 AM   #9
Mark_667
Member
 
Registered: Aug 2005
Location: Manchester, England
Distribution: Ubuntu 20.04
Posts: 383

Rep: Reputation: 30
As you're only using a LAMP stack you may want to consider using Docker containers in the future, there'll be far less to upgrade.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
patch management solution - Ivanti patch management for Linux, Unix and Mac pranesh.annamalai Linux - Enterprise 5 02-29-2020 03:50 PM
Redhat patch management QWE123 Linux - Newbie 6 03-31-2017 07:12 AM
Patch management and security hokie1999 Linux - Software 9 05-29-2016 10:31 AM
Patch Management Sathish Hemadhri Linux - Server 5 11-15-2013 05:06 AM

LinuxQuestions.org > Forums > Enterprise Linux Forums > Linux - Enterprise

All times are GMT -5. The time now is 03:55 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration