LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 12-04-2011, 07:59 AM   #16
grvsaxena419
LQ Newbie
 
Registered: Dec 2011
Location: India
Distribution: ubuntu 11.10
Posts: 12

Original Poster
Rep: Reputation: Disabled

Hello Proud , Thanks for your reply.
Quote:
Originally Posted by Proud View Post
I don't know why you'd assume I'm more experienced or intelligent, I'm just raising what seem to be the key considerations.
That was just a guess ! It seemed to me from your answers.

Quote:
Originally Posted by Proud View Post
As for updating a read-only medium, I'd look to differential/delta snapshots, perhaps similar to how UnionFS allows trial users of Linux LiveCDs to seamlessly write to locations that appear to be a part of the disc image but end up being stored on a harddrive.
Ok I will have a look at UnionFS to see its working.
Quote:
Originally Posted by Proud View Post
The use of checksums is reasonable, but they're just data, technically you need to still be able to rationally trust that the OS and checksum tool binary are not also subverted. This may sound paranoid, but if you plan for your system to work against malicious activity or even just subtle memory corruption without filesystem-based checksums(such as ZFS) and ECC, you need to bootstrap your new restored system from the bare minimum to build a chain of trust.
For a related read, see this breakdown of the classic example.
Thanks for the link and idea, I will try my best to have such a system, I could keep all the basic required binaries on my secured partition and then use them to bootstrap the system.
Quote:
Originally Posted by Proud View Post
I would think that if you take the malicious aspect out of the equation, you could produce a simple enough system to snapshot and/or checksum a whole system. The thing is, what do you expect the source of the need to restore to be? If it's a hardware issue such as failing storage or memory, you will perhaps need the user to use additional tools such as memcheck86+ to determine such things, or to add/replace the storage device with a blank new one while your /abc may now be on the unreliable one.
My system would be able to restore system from like accidental deletion of files, broken upgrades (the system will be for ubuntu linux initially), wrong editing to conffiles, wrong changes to files like fstab. The need to restore would be specified by the user himself. If the user feels that the system is wrong he can restore it to a last known working state.
Quote:
Originally Posted by Proud View Post
Also hardware issues are more typically handled by things like RAID now, and filesystems that have built-in transparent integrity checking (again ZFS is a common example but not the only contender). A manually invoked restore system with a dedicate single location of its tools, data and perhaps snapshots seems to require a not-insignificant tradeoff in the end user's time to ensure it is current enough to be of benefit.
Are you familiar with version control tools such as subversion and git, where versions are a state of the whole repository rather than per path/filename?
Ok. I understand the problem with my system, it would have user to do much , I could have it create restore points on everyday basis so that the backup is current for a day atleast. But ofcourse there is a dedicated single location of backup which might not save the system in situations of major crashes. I would need to work upon that, may be I provide an option to export the backups to a portable drive.
Quote:
Originally Posted by Proud View Post
AFAIK it's best practice for people to separately back up their important data as frequently and granularly as required, and to keep their OS image with current configuration settings to hand should they need to rebuild the setup from scratch. I understand Windows has a system restore mechanism, but I think the unix tradition is to checksum your existing system if you suspect a specific transient integrity problem, perhaps use the package management tools to repair applications, or just start again from a fresh trusted OS image + data backups in the worst case.
Yes , i am intending to work on a package state saving tool, which could save state of the installed packages and then restore them when needed. yes that's true, but this system would be useful in cases of small failures, I need to think of more ways to make it robust to severe damages.ce.

Quote:
Originally Posted by Proud View Post
Commercially, I am unaware of the supply & demand of such mechanisms.
I am a computer engineering student I am working for open source so it could be just released in open source to benefit simple end users.
 
Old 12-04-2011, 08:58 AM   #17
resetreset
Senior Member
 
Registered: Mar 2008
Location: Cyberspace
Distribution: Dynebolic, Ubuntu 10.10
Posts: 1,340

Rep: Reputation: 62
just out of curiosity, what language are you programming in?

grvsaxena, I'm in India too, and interested in things like this. Where are you??!!
 
Old 12-04-2011, 10:14 AM   #18
Proud
Senior Member
 
Registered: Dec 2002
Location: England
Distribution: Used to use Mandrake/Mandriva
Posts: 2,794

Rep: Reputation: 116Reputation: 116
Quote:
Originally Posted by grvsaxena419 View Post
My system would be able to restore system from like accidental deletion of files, broken upgrades (the system will be for ubuntu linux initially), wrong editing to conffiles, wrong changes to files like fstab. The need to restore would be specified by the user himself. If the user feels that the system is wrong he can restore it to a last known working state.
Consider how your system will work if the user has made a typo in fstab and then rebooted, such that / or /abc are no longer correctly mounted. Or if they have broken a file or permission in key locations such as /boot, /bin, /lib, /usr/sbin, /usr/bin or /usr/lib.
I'm not sure you would build a reliable restore system by running it within the unstable OS it is restoring. I think only the snapshotting/restore point creation/trigger system could live there, and is then still as vulnerable to software and hardware issues as the OS it protects.

As for backups every day, would your system need all other processes to stop, i.e. be run right before shutdown or right after booting? If you plan to create restore points of a live system, how will you handle integrity issues such that many applications may require all files to be snapshotted at a single instance in time, while in reality a recursive per-file diff takes time and could well version some files before an app makes a change, and some after, such that restoring the mismatched pair could be an invalid state for the app and useless? A simple case would be you catch one copy of a file in one location before a move, then another copy of it after a move, but at any one time there was (and should be) only one copy of the file. It could be an app-specific backup/archive, a lock file, logfiles, there are many such examples.
You may also need to consider data not yet flushed to disk.
Ideally I think you want atomic snapshots for useful versions/restore points, without filesystem/kernel support I'm not sure how you achieve this.

Late edit:
Quote:
Originally Posted by grvsaxena419 View Post
I am a computer engineering student I am working for open source so it could be just released in open source to benefit simple end users.
I was not directly addressing how you would popularise your creation but rather what rivals surely already exist in the marketplace for the whole range of users from single installs to enterprises. A quick google for 'linux system restore' reveals many as well as discussions here at LQ from years ago. A key assumption in your mind seems to be that Windows system restore works well enough in the first place that replicating it to Linux is of benefit, and that the same conditions which make it useful there also carry over. Again, modern Linux tends to have robust package management with built in dependancy and repair mechanisms, this and the use of tested official package repositories allows for far more stable systems than Windows and the risk of 3rd party driver installs & downloaded utilities.

Last edited by Proud; 12-04-2011 at 12:45 PM.
 
Old 12-05-2011, 09:19 AM   #19
grvsaxena419
LQ Newbie
 
Registered: Dec 2011
Location: India
Distribution: ubuntu 11.10
Posts: 12

Original Poster
Rep: Reputation: Disabled
Hello Nbomr,
Quote:
Originally Posted by theNbomr View Post
I don't know a great deal about SElinux, but I think it has some capacity to distinguish what applications can/cannot read/write specified files and directories. Perhaps it is worth looking at as a ready-made solution.
--- rod.
I will have a look at SElinux so that I could get such functionality.
Thank you very much.
 
Old 12-05-2011, 09:23 AM   #20
grvsaxena419
LQ Newbie
 
Registered: Dec 2011
Location: India
Distribution: ubuntu 11.10
Posts: 12

Original Poster
Rep: Reputation: Disabled
Hello resetreset,
Quote:
Originally Posted by resetreset View Post
just out of curiosity, what language are you programming in?

grvsaxena, I'm in India too, and interested in things like this. Where are you??!!
I am making changes in linux kernel, its written mostly in C and some assembly language(low level). I am in NIT, Jaipur.
 
Old 12-05-2011, 11:22 AM   #21
grvsaxena419
LQ Newbie
 
Registered: Dec 2011
Location: India
Distribution: ubuntu 11.10
Posts: 12

Original Poster
Rep: Reputation: Disabled
Hello Proud,
Quote:
Originally Posted by Proud View Post
Consider how your system will work if the user has made a typo in fstab and then rebooted, such that / or /abc are no longer correctly mounted. Or if they have broken a file or permission in key locations such as /boot, /bin, /lib, /usr/sbin, /usr/bin or /usr/lib.
I'm not sure you would build a reliable restore system by running it within the unstable OS it is restoring. I think only the snapshotting/restore point creation/trigger system could live there, and is then still as vulnerable to software and hardware issues as the OS it protects.
Yes, you are right my system would not work in that situations, it will fail. I think I could have a system that could initiate a recovery using bootstrap which could be called using grub or initramfs. I will surely try that or have an option for using program using live cd.
My approach for the backup and restore is to save the packages of the system, and related files like dpkg-status, source.list, pakage-selection and then restore the package selection and all the files.
Quote:
Originally Posted by Proud View Post
As for backups every day, would your system need all other processes to stop, i.e. be run right before shutdown or right after booting? If you plan to create restore points of a live system, how will you handle integrity issues such that many applications may require all files to be snapshotted at a single instance in time, while in reality a recursive per-file diff takes time and could well version some files before an app makes a change, and some after, such that restoring the mismatched pair could be an invalid state for the app and useless? A simple case would be you catch one copy of a file in one location before a move, then another copy of it after a move, but at any one time there was (and should be) only one copy of the file. It could be an app-specific backup/archive, a lock file, logfiles, there are many such examples.
You may also need to consider data not yet flushed to disk.
Ideally I think you want atomic snapshots for useful versions/restore points, without filesystem/kernel support I'm not sure how you achieve this.
I am using dpkg-repack for packing up a package files and creating deb packages for them, this doesn't need to take backup right before shutdown. Yes my approach would definitely suffer from this issue. I think I could like pause a process using kernel support and then create a package for it and then resume it, maybe I could do that. Yes that's really a difficult thing to solve I suppose because I had not thought about such issues, actually I had not imagined about such issues. I am agree with you about the need of support of filesystem/kernel for the atomic snapshot thing. Actually what problem I thought about snapshot thing is that its too much complex for users to handle. But I think it maybe a better thing to design such a robust and reliable system instead of my approach.
Quote:
Originally Posted by Proud View Post
Late edit:
I was not directly addressing how you would popularise your creation but rather what rivals surely already exist in the marketplace for the whole range of users from single installs to enterprises. A quick google for 'linux system restore' reveals many as well as discussions here at LQ from years ago. A key assumption in your mind seems to be that Windows system restore works well enough in the first place that replicating it to Linux is of benefit, and that the same conditions which make it useful there also carry over. Again, modern Linux tends to have robust package management with built in dependancy and repair mechanisms, this and the use of tested official package repositories allows for far more stable systems than Windows and the risk of 3rd party driver installs & downloaded utilities.
Yes I agree with you , and also I think that a system for just saving the package state and then restoring it is not worth the effort.
 
Old 12-06-2011, 07:56 AM   #22
resetreset
Senior Member
 
Registered: Mar 2008
Location: Cyberspace
Distribution: Dynebolic, Ubuntu 10.10
Posts: 1,340

Rep: Reputation: 62
Quote:
Originally Posted by grvsaxena419 View Post
Hello resetreset,


I am making changes in linux kernel, its written mostly in C and some assembly language(low level). I am in NIT, Jaipur.
hey that's great, I'm not a C guy, but I *do* love asm! we should become friends can I send you a PM?
 
Old 12-06-2011, 11:00 AM   #23
grvsaxena419
LQ Newbie
 
Registered: Dec 2011
Location: India
Distribution: ubuntu 11.10
Posts: 12

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by resetreset View Post
hey that's great, I'm not a C guy, but I *do* love asm! we should become friends can I send you a PM?
Hey , sure that's a nice idea really, I have sent you a friend request but i don't have much idea of this forum quite new.
 
Old 12-09-2011, 08:58 AM   #24
resetreset
Senior Member
 
Registered: Mar 2008
Location: Cyberspace
Distribution: Dynebolic, Ubuntu 10.10
Posts: 1,340

Rep: Reputation: 62
grvsaxena, did you send me an EMAIL by any chance? the email addr that LQ has is not what I use - I lost the password to that ages ago. i'm sending you a *PM*, read it



oh sorry I can't - have you turned that off? ok, sending you an EMAIL then - pleas read.

Last edited by resetreset; 12-09-2011 at 09:00 AM.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Get error code from child process? C++arl Programming 13 03-30-2011 10:37 AM
Source code for the init process? jhwilliams Programming 4 10-11-2009 03:04 AM
Child Process code help anik18 Programming 12 09-03-2009 12:34 AM
How to wait for the process and know if the exit code of the sub process williamhomanchun Linux - General 10 08-11-2008 01:32 PM
Getting the return code of a backgrounded process 0ddba11 Linux - General 7 07-02-2007 05:31 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 07:30 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration