LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 06-01-2016, 03:10 PM   #1
biosboy4
Member
 
Registered: Aug 2015
Distribution: *DEB, Centos, NXOS
Posts: 242

Rep: Reputation: 38
Question Hot Imaging Software


Hello,

I find using Acronis to automatically image production machines and then using rsync to collect the images/archive them to be a fantastic backup solution.

Is there any way to do this with Open Source software? The only open source software I can find for linux or windows only support cold imaging.

Thanks,

biosboy4
 
Old 06-02-2016, 08:40 AM   #2
Emerson
LQ Guru
 
Registered: Nov 2004
Location: Saint Amant, Acadiana
Distribution: Gentoo ~arch
Posts: 6,317

Rep: Reputation: Disabled
I realize this is not what you are asking, but it never crossed my mind to image for backup purposes. I use rsnapshot, very happy with it.
 
Old 06-02-2016, 08:52 AM   #3
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,140
Blog Entries: 13

Rep: Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205
I use dd. Yes it needs to be done off-line. It is however "free". Sorry, while I agree that their product is a good one, and I'll also concede that it my be irrelevant to backup live because the ephemeral data on a system is throwaway anyways, I just find it better to image the disk fully using dd.

I create custom embedded Linux systems. The FIRST thing I do once I establish my first stable install point on a new system design is to dd off a copy. I sort of have to grab hold of my fast progress and halt myself to do that, but I manage; .... or I go to far, screw up the system, swear .... actually make that all caps, I SWEAR, and then I start from scratch.

Eventually I make an actual install process which is different than disk duplication. However having the full disk backup is critical when things get dicey and I have to "go back" to my last successful point, several times. Because if I were just doing rote, replication of known things, then I wouldn't be doing it. Not tooting my own horn, my point is that if you could just grab a regular distro and toss it on a system and ship it for our purposes, then no client would've engaged us, and nor would we punish them by making them pay for a custom, we'd instead say, "Here, put Mint, Debian, or Ubuntu on it ... done!" Therefore each system is unique in configuration and application, and thus I am mostly learning what works and what doesn't for the most part.

For one-time cost you can get a NAS type of storage which is very much larger than your system and back up the system to that regularly.

I'd separate system versus data and back each up independently. You may find that the system part may rarely change and while you may make daily or weekly backups, you may not need to keep them all since they're the same a lot.
 
Old 06-02-2016, 09:49 AM   #4
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 19,378

Rep: Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422
The problem child in "hot" backups is your database system. They don't want to use system cache, they don't want to use normal filesystem APIs - it has to be o_direct and AIO. I don't know how Acronis would handle this for all the open system databases.
Linux filesystems (all ?, dunno) honour freeze requests - so for instance snapshot (LVM and btrfs) can get point-in-time consistency. I use this almost exclusively to make backups - take a snap, back the snap up (not the active filesystem), delete the snap. If you are (very) lucky there is no extra disk usage due to CoW. And even if there is, you reclaim it at the end of the backup. With btrfs you can even do incremental snaps and just send that to your backup server.
 
Old 06-02-2016, 03:34 PM   #5
biosboy4
Member
 
Registered: Aug 2015
Distribution: *DEB, Centos, NXOS
Posts: 242

Original Poster
Rep: Reputation: 38
I need a way to to live, hot backups to physical windows and linux machines. Shutting down is not an option since this is a 25/8/366 operation.

Is there really no open source software that does this? There are plenty of proprietary solutions like Acronis.

Boy this is a tough pill to swallow..
 
Old 06-02-2016, 08:43 PM   #6
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 19,378

Rep: Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422Reputation: 3422
Have you ever tested one of these backups ?.
I mean a real disaster test - instant failure with no shutdown, restore of public facing system(s) and continue ?.
Everybody you deal with will have a different view of the world because they committed the transactions, and you are in the past with no record of those intervening transactions. Depending on backup interval this may not be trivial.
Even the bosses spreadsheet updates are gone.
 
1 members found this post helpful.
Old 06-03-2016, 08:32 AM   #7
rtmistler
Moderator
 
Registered: Mar 2011
Location: USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 9,140
Blog Entries: 13

Rep: Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205Reputation: 4205
Quote:
Originally Posted by biosboy4 View Post
I need a way to to live, hot backups to physical windows and linux machines. Shutting down is not an option since this is a 25/8/366 operation.

Is there really no open source software that does this? There are plenty of proprietary solutions like Acronis.

Boy this is a tough pill to swallow..
Spending money for a business purpose is a tough pill to swallow ...
Quote:
Originally Posted by syg00 View Post
Have you ever tested one of these backups ?.
I mean a real disaster test - instant failure with no shutdown, restore of public facing system(s) and continue ?.
Everybody you deal with will have a different view of the world because they committed the transactions, and you are in the past with no record of those intervening transactions. Depending on backup interval this may not be trivial.
Even the bosses spreadsheet updates are gone.
While you are expecting the environment to be up 99.999% of the time, syg00 is correctly pointing out that even incremental backups of "everything" don't occur each and every second of these 25 hour days you have.

Consider if you have a user who kicked the cord out on their system? What would "you" tell them about their lost data?
 
Old 06-03-2016, 10:43 AM   #8
biosboy4
Member
 
Registered: Aug 2015
Distribution: *DEB, Centos, NXOS
Posts: 242

Original Poster
Rep: Reputation: 38
rsync incrementally versions everything at the file level actively. The script runs once every 20-30 minutes. The images are made daily, and backup at the same time/just like any other file.

I don't mind spending money for this, I'm just looking for something Open Source since that is my religion.

Many of these machines are physical windows boxes with crazy plc hardware, so I'm limited to the amount of things I can do. All the vms in the server room, get backed up with xsibackup, and are fault tolerant (at least the critical ones. Some are HA.) where there are real backup/HA/FT possibilities.

Last edited by biosboy4; 06-03-2016 at 10:45 AM.
 
Old 06-05-2016, 09:53 AM   #9
danjfk
LQ Newbie
 
Registered: Aug 2015
Posts: 5

Rep: Reputation: Disabled
You should try XSIBackup, you can download it at: https://sourceforge.net/projects/xsibackup/

- It performs hot backups, so your VMs will always be available.
- It also performs cold backups, shutting down your VM prior to the backup and turns it on when the backup has finished.
- It is cron programmable.
- Can automatically make room for your new backup in the backup device once it is full based on the full size of the device or a configurable space limit.
- It is capable of using Rsync to perform differential backups, mirrored to a second ESXI box or to a local datastore.
- Can overwrite backups or create a new folder for each new backup.
- It's self capable, will run in the hypervisor without the need of additional hardware.
- It's totally free to use for personal and commercial use.
- Will work on the free version of the ESXi hypervisor.
- Will create and use it's own SSL key pair allowing to stablish trust relationships between different servers that will be preserved across reboots.
- It's super easy to install.

You can find the man page at: http://33hops.com/xsibackup-help-man-page.html
And a number of useful articles at: http://33hops.com/blog-main.asp?lang=EN
 
Old 06-05-2016, 04:03 PM   #10
biosboy4
Member
 
Registered: Aug 2015
Distribution: *DEB, Centos, NXOS
Posts: 242

Original Poster
Rep: Reputation: 38
Hot Imaging Software

I use xsibackup for my vms, I'm looking for an open source way to do hot images of physical machines.
 
Old 06-05-2016, 09:56 PM   #11
danjfk
LQ Newbie
 
Registered: Aug 2015
Posts: 5

Rep: Reputation: Disabled
For Windows systems you can use Disk2VHD

https://technet.microsoft.com/en-us/.../ee656415.aspx

In case of Linux systems you can use dd. To avoid inconsistencies in case of very active systems during the backup process, you should backup at LVM level after taking a snapshot.

https://www.howtoforge.com/linux_lvm_snapshots

Regards
 
Old 06-06-2016, 08:33 AM   #12
biosboy4
Member
 
Registered: Aug 2015
Distribution: *DEB, Centos, NXOS
Posts: 242

Original Poster
Rep: Reputation: 38
Quote:
Originally Posted by danjfk View Post
For Windows systems you can use Disk2VHD

https://technet.microsoft.com/en-us/.../ee656415.aspx

In case of Linux systems you can use dd. To avoid inconsistencies in case of very active systems during the backup process, you should backup at LVM level after taking a snapshot.

https://www.howtoforge.com/linux_lvm_snapshots

Regards
Thanks, I'll look into dd.

Disk2VHD and/or vmware's converter might be suitable as backups if many of these machine's didn't have crazy, proprietary PLC hardware.
 
Old 06-06-2016, 03:31 PM   #13
JimMcCall
LQ Newbie
 
Registered: Jul 2007
Location: North West PA
Distribution: Slackware
Posts: 17

Rep: Reputation: 0
be careful with dd. it is commonly called Disk Destroyer. Not because of any problems with the program, but, because it does exactly what you tell it to do, not necessarily what you want it to do. Check and double check your command before you hit enter with this one. Many people have wiped out entire drives without realizing it until it's too late.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Software for imaging linux SlacUser Linux - General 3 02-23-2008 12:01 AM
Imaging Software custangro Linux - Software 1 11-21-2007 12:42 PM
imaging software jazztown Linux - Software 2 06-20-2005 06:54 PM
Imaging software john8675309 Linux - Software 2 06-20-2005 12:58 PM
Imaging Software unwrittenLaww Linux - Software 1 04-20-2004 01:30 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 07:15 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration