LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 09-12-2021, 09:41 PM   #1
TBotNik
Member
 
Registered: May 2016
Location: Greenville, TX
Distribution: Kubuntu 18.04
Posts: 796

Rep: Reputation: Disabled
Purge Script


All,

With me having so many issues with the whole LAMP stack, I decided I needed to purge all of it and start over, so looked online and found a partial code set to get it done. I've put all that into a script to round out and automate the process, which I've posted at:

https://pastebin.com/2A020262

What I need here, since this is a "drastic action" script, is review from you gurus, to see if my script is on point, accurate and any comments or tweaks needed. As you can read from the comments I put in, the functions "ilamp_ins" and "ipgdb_ins" are where, I feel, the attention is needed.

I am writing this script to actually solve all the problems I posted at:
linuxquestions.org/questions/showthread.php?p=6250772#post6250772
linuxquestions.org/questions/linux-software-2/apache-not-sure-4175689230/
linuxquestions.org/questions/linux-software-2/mysql-not-working-4175678774/

Additionally I did not get a function defined for data restore, but can even put that in another script or write a PHP script for that, especially since some data sync is needed.

Other than the code source I put in the comments, I pulled most o the rest of the code from other scripts I had already written.

So your help needed here!

Thanks!

TBNK
 
Old 09-12-2021, 09:44 PM   #2
wpeckham
LQ Guru
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, VSIDO, tinycore, Q4OS, Manjaro
Posts: 5,787

Rep: Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773
I never purge. I scrub the hardware and do a clean install of JUST the things I REALLY want. (Make sure to back up any special configurations, data, or settings first. You DO have backups, right?)
 
Old 09-13-2021, 07:24 AM   #3
TBotNik
Member
 
Registered: May 2016
Location: Greenville, TX
Distribution: Kubuntu 18.04
Posts: 796

Original Poster
Rep: Reputation: Disabled
ThanX!

Quote:
Originally Posted by wpeckham View Post
I never purge. I scrub the hardware and do a clean install of JUST the things I REALLY want. (Make sure to back up any special configurations, data, or settings first. You DO have backups, right?)
wpeckham,

Too much going on, on this particular server, so would need "clonezilla" or another full recovery dup/clone app, to do so and over 6 years of trying to install clonezilla and/or 2 other clone apps have always failed, so absolutely cannot ever consider a full out hardware level redo. A new server install takes me over 6 weeks to install, there is so much going on, and that 6 weeks is from scripts built to automate the process.

Thanks for the idea, but sorry cannot use it, especially on this box as it is the Primary Network and Domain server. I'd be totally out of commission, on the entire network, for weeks with none of the cron processes working during that down time.

I'm working toward my own install DVD, so all the automated scripts can run automated, so I start a new machine install and can let it run unattended, the whole time. But still perfecting some of those install scripts.

Cheers!

TBNK

Last edited by TBotNik; 09-13-2021 at 07:29 AM.
 
Old 09-13-2021, 09:13 AM   #4
wpeckham
LQ Guru
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, VSIDO, tinycore, Q4OS, Manjaro
Posts: 5,787

Rep: Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773
It sounds like you have carefully loaded all of your eggs into one basket to make sure that you have a truly disastrous single point of failure!

Can you prep a second server as a replacement/failover server for a fast replacement swap?

If you absolutely MUST adjust in-place, then it is even more critical that you DO have a backup! You do not "install" clonezilla for a full clone backup operation, you BOOT form clonezilla media and back up your main storage to backup media. Installing onto local storage that you clone WHILE IT IS ACTIVE results in a very high chance of a corrupt or incomplete backup.

There are other backup options, the critical thing is that you have more than one backup, and that you have tested them by restore on other (perhaps isolated) hardware to make sure that the recovery process works.

Taking a backup might mean a little downtime. Doing a major change with NO backup or replacement could mean the slightest thing going wrong might mean LOTS of downtime while you completely recreate the server and services. That is to be avoided.
 
Old 09-14-2021, 10:59 AM   #5
TBotNik
Member
 
Registered: May 2016
Location: Greenville, TX
Distribution: Kubuntu 18.04
Posts: 796

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by wpeckham View Post
It sounds like you have carefully loaded all of your eggs into one basket to make sure that you have a truly disastrous single point of failure!

Can you prep a second server as a replacement/failover server for a fast replacement swap?

If you absolutely MUST adjust in-place, then it is even more critical that you DO have a backup! You do not "install" clonezilla for a full clone backup operation, you BOOT form clonezilla media and back up your main storage to backup media. Installing onto local storage that you clone WHILE IT IS ACTIVE results in a very high chance of a corrupt or incomplete backup.

There are other backup options, the critical thing is that you have more than one backup, and that you have tested them by restore on other (perhaps isolated) hardware to make sure that the recovery process works.

Taking a backup might mean a little downtime. Doing a major change with NO backup or replacement could mean the slightest thing going wrong might mean LOTS of downtime while you completely recreate the server and services. That is to be avoided.
wpeckham,

I can always try loading another clone package. Clonezilla always stops somewhere with a failed download key, so will be looking for another.

Cheers!

TBNK
 
Old 09-14-2021, 02:13 PM   #6
TBotNik
Member
 
Registered: May 2016
Location: Greenville, TX
Distribution: Kubuntu 18.04
Posts: 796

Original Poster
Rep: Reputation: Disabled
wpeckham,

So went to:

https://www.tecmint.com/linux-disk-cloning-tools/

and downloaded the following files:

clonezilla-live-20160210-wily-amd64.iso
doclone-0.8.0.tar.gz
g4l-v0.59.iso
partclone-dbg_0.2.89-1_amd64.deb
systemimager-boot-i386-standard_4.1.6_all.deb

I'll trying installing partclone as it matches my system best being both amd64 and .deb.

Let you know how that goes. Hope it backs up the "/3T/" drive as most active data now on that disk.

None installed! Again all failed!

Cheers!

TBNK

Last edited by TBotNik; 09-14-2021 at 06:32 PM.
 
Old 09-14-2021, 03:38 PM   #7
computersavvy
Senior Member
 
Registered: Aug 2016
Posts: 3,345

Rep: Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484
Your issues seem to be with the configs thus on the system file systems and not on the data file systems.

Yes, it is good to have backups of the data, but a purge, reconfig, reinstall of apps should not directly affect the data so the OS file system seems the really critical one to be concerned about. Hopefully it is not on a 3TB partition.

Cloning with clonezilla does not require nearly the same space as is occupied by the partition but ANY CLONING PROCEDURE will require the file system being cloned to be inactive and not mounted in order to prevent corruption of the clone image.

In my experience with clonezilla and the like they will not even allow you to select a mounted partition as the source when making the clone image.
 
Old 09-14-2021, 06:36 PM   #8
TBotNik
Member
 
Registered: May 2016
Location: Greenville, TX
Distribution: Kubuntu 18.04
Posts: 796

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by computersavvy View Post
Your issues seem to be with the configs thus on the system file systems and not on the data file systems.

Yes, it is good to have backups of the data, but a purge, reconfig, reinstall of apps should not directly affect the data so the OS file system seems the really critical one to be concerned about. Hopefully it is not on a 3TB partition.

Cloning with clonezilla does not require nearly the same space as is occupied by the partition but ANY CLONING PROCEDURE will require the file system being cloned to be inactive and not mounted in order to prevent corruption of the clone image.

In my experience with clonezilla and the like they will not even allow you to select a mounted partition as the source when making the clone image.
computersavvy,

As I understand clonezilla, the .iso file is supposed to be on a DVD, so you boot the DVD to actually clone the system. Please correct me if this is not correct.

Cheers!

TBNK
 
Old 09-14-2021, 07:26 PM   #9
michaelk
Moderator
 
Registered: Aug 2002
Posts: 25,822

Rep: Reputation: 5960Reputation: 5960Reputation: 5960Reputation: 5960Reputation: 5960Reputation: 5960Reputation: 5960Reputation: 5960Reputation: 5960Reputation: 5960Reputation: 5960
Quote:
to see if my script is on point, accurate and any comments or tweaks needed. As you can read from the comments I put in, the functions "ilamp_ins" and "ipgdb_ins" are where, I feel, the attention is needed.
On point and accurate is difficult to ascertain from an outsider's perspective because it appears that your ilamp_ins function contains seemingly unrelated services like ssh.
 
Old 09-15-2021, 08:32 AM   #10
SlowCoder
Senior Member
 
Registered: Oct 2004
Location: Southeast, U.S.A.
Distribution: Debian based
Posts: 1,250

Rep: Reputation: 164Reputation: 164
Quote:
Originally Posted by TBotNik View Post
Too much going on, on this particular server, so would need "clonezilla" or another full recovery dup/clone app, to do so and over 6 years of trying to install clonezilla and/or 2 other clone apps have always failed, so absolutely cannot ever consider a full out hardware level redo. A new server install takes me over 6 weeks to install, there is so much going on, and that 6 weeks is from scripts built to automate the process.
I'm genuinely curious what the nature of this server is. 1 machine, 6 weeks to install/configure, and no backups? 8O wpeckham is right, you need to assess your configuration, and split up your processes to better deal with resource management, maintenance, backups, etc.

Ol' Murphy is gonna come knocking eventually.
 
1 members found this post helpful.
Old 09-15-2021, 01:08 PM   #11
computersavvy
Senior Member
 
Registered: Aug 2016
Posts: 3,345

Rep: Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484Reputation: 1484
Quote:
Originally Posted by TBotNik View Post
computersavvy,

As I understand clonezilla, the .iso file is supposed to be on a DVD, so you boot the DVD to actually clone the system. Please correct me if this is not correct.

Cheers!

TBNK
Yes, DVD or USB
 
Old 09-20-2021, 10:51 AM   #12
TBotNik
Member
 
Registered: May 2016
Location: Greenville, TX
Distribution: Kubuntu 18.04
Posts: 796

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by michaelk View Post
On point and accurate is difficult to ascertain from an outsider's perspective because it appears that your ilamp_ins function contains seemingly unrelated services like ssh.
michaelk,

That is why I'm asking for the review, because most of the code past the 3 purge functions were copied from other install script that were previously written, so want to prune this code to be specific to the actual installs needed. Also since some code is version specific, wanting code that deals only with the latest versions.

Cheers!

TBNK
 
Old 09-20-2021, 10:52 AM   #13
TBotNik
Member
 
Registered: May 2016
Location: Greenville, TX
Distribution: Kubuntu 18.04
Posts: 796

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by computersavvy View Post
Yes, DVD or USB
Thanks! Thought so!

TBNK
 
Old 09-24-2021, 04:49 PM   #14
TBotNik
Member
 
Registered: May 2016
Location: Greenville, TX
Distribution: Kubuntu 18.04
Posts: 796

Original Poster
Rep: Reputation: Disabled
All,

Well got forced in this situation as I needed to install a media card, but it messed the IRQ, so the 3T drive was not seen. I added it to the fstab as it was not longer sdb, but sdc, and still it would not work. So found that the IRQ problem had totally destroyed the entire boot section, rewriting the boot sequence. So had to re-install with format entire drive for the boot disk.

Now I have the over 1,000 changes that have to be made to the desktop and the over 2,000 apps that have to be added to the system. Since each app takes about 90 minutes to install and that means approx 180,000 min or 3,000 hrs. And since there are only 8-10 hours a day that an install can be monitored, that is 300-375 days on installation and it would take that long if BASH install files did not exist to bring it down to somewhere around 6-8 weeks.

Now understand? I'm a perfectionist, so it has to be perfect!

Cheers!

TBNK

Last edited by TBotNik; 09-24-2021 at 04:52 PM.
 
Old 09-25-2021, 08:55 AM   #15
wpeckham
LQ Guru
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, VSIDO, tinycore, Q4OS, Manjaro
Posts: 5,787

Rep: Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773Reputation: 2773
Quote:
Originally Posted by TBotNik View Post
All,

Well got forced in this situation as I needed to install a media card, but it messed the IRQ, so the 3T drive was not seen. I added it to the fstab as it was not longer sdb, but sdc, and still it would not work. So found that the IRQ problem had totally destroyed the entire boot section, rewriting the boot sequence. So had to re-install with format entire drive for the boot disk.

Now I have the over 1,000 changes that have to be made to the desktop and the over 2,000 apps that have to be added to the system. Since each app takes about 90 minutes to install and that means approx 180,000 min or 3,000 hrs. And since there are only 8-10 hours a day that an install can be monitored, that is 300-375 days on installation and it would take that long if BASH install files did not exist to bring it down to somewhere around 6-8 weeks.

Now understand? I'm a perfectionist, so it has to be perfect!

Cheers!

TBNK
1. Ever since the udev revolution in 2008 the approved way of identifying devices for mounting in fstab has been either by uuid or label. The device path can move from one mount to the next and is therefor unreliable, so using a more reliable identifier is the common default for the last 13 years. I suspect it would have avoided this problem.

2. You are clearly not really a perfectionist, or you would have made and verified perfect backups, but I think I do understand what you mean. Take this as an educational event and make sure that if (for whatever reason) this happens again you can deal with it in a faster and easier manner.

3. I cannot verify your script. It does not appear faulty, but I have no environment that is enough like yours to supply any valid testing information. I advise that you fully back up or image the storage involved FIRST, so you can quickly and easily get back to where you are now with little lost time and no loss of data.
If I were creating automation for this, I might make additional tests to verify steps and abort if something was clearly not right. In fact, I might reduce it to three or four different scripts so that I could do some manual checking at critical points along the way. That way you could avoid taking a step that might cause corruption before it executes.

4. I would never again build a system that depends upon 2000 applications that require 90 minutes each for installation for production. In fact I would not even create that monster for testing! I might create that as a template for creating systems for test, development, and production so that those systems could be built, rebuilt, and replace rapidly, but I would never set myself such a booby trap! That is setting yourself up for catastrophic failure only mitigated by regular, generational, complete backups or images.
 
  


Reply

Tags
bash, lamp, purge, review



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
apt-get purge or --purge is false advertising... jmgibson1981 Linux - Newbie 12 07-01-2015 08:04 AM
Backup Purge Script Help Please!!! nooby210 Linux - Newbie 14 05-31-2011 11:20 AM
[SOLVED] Using bash script to purge rotating tcpdump files merixon Programming 5 10-20-2010 11:19 AM
Script: purge files more than N days old. Need Non Recursive Find explore.s AIX 3 12-04-2008 07:24 AM
How do I purge Linux kickstart from my box? UnderSiege Linux - Security 7 04-17-2003 02:35 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 11:20 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration