LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 10-01-2014, 02:58 PM   #1
mikecolley
LQ Newbie
 
Registered: Apr 2014
Location: All Over USA
Distribution: Mint16 Cinnamon
Posts: 29

Rep: Reputation: Disabled
Has anybody written a disk or directory rebuild program?


Hi All:

I'm trying to prepare for the future disk crash that hasn't happened yet. I'm not talking about an unerase program, I'm talking about a total loss of disk data including directory structure type of rebuild program.

Sooner or later it happens, your PC crashes just because it can. I have had crashes where the disk/directory is no longer recognizable by a CD booted OS.

Question 1 - Has anyone written a program that will look in raw disk space to find a ext/2/3/4 (or other) directory structure and then recover it?

Question 2 - Or has anybody written a program that would look in raw disk space to see if there are any files out there and then recover them?

I keep poor backups so when it happens, I'm not DIW, but still it would be nice to get my up-to-date data back. It would be nicer if a utility could re-create the entire disk so I could even boot from it, but I think that is asking too much.

Has anyone written anything similar?

Thanks! - Mike
 
Old 10-01-2014, 03:39 PM   #2
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 26,480

Rep: Reputation: 7939Reputation: 7939Reputation: 7939Reputation: 7939Reputation: 7939Reputation: 7939Reputation: 7939Reputation: 7939Reputation: 7939Reputation: 7939Reputation: 7939
Quote:
Originally Posted by mikecolley View Post
Hi All:
I'm trying to prepare for the future disk crash that hasn't happened yet. I'm not talking about an unerase program, I'm talking about a total loss of disk data including directory structure type of rebuild program. Sooner or later it happens, your PC crashes just because it can. I have had crashes where the disk/directory is no longer recognizable by a CD booted OS.

Question 1 - Has anyone written a program that will look in raw disk space to find a ext/2/3/4 (or other) directory structure and then recover it?
Question 2 - Or has anybody written a program that would look in raw disk space to see if there are any files out there and then recover them?

I keep poor backups so when it happens, I'm not DIW, but still it would be nice to get my up-to-date data back. It would be nicer if a utility could re-create the entire disk so I could even boot from it, but I think that is asking too much.
Short answer: yes
Longer answer, based on your EXACT question: no.

Why? If the disk space is 'raw', then there is no data or directories to recover. If the disk is TOTALLY unreadable (as you say), even when booting from CD, then you can't recover data from it, period. There are no programs that can recover from a physical, mechanical failure.

There are MANY bare-metal recovery programs, ranging in size and expense from free/personal use (systemimager, mondoarchive, or mkcdrec), to $10k+/enterprise solutions (Tivoli Storage Manager, Netbackup, and others all have clients to do just this).

Mondoarchive creates a 'snapshot' ISO image...burn it to DVD/USB, boot from it, and you get back EXACTLY what was on your original system. Then again, if you back up your data, you should ALREADY have your directories/files, and can reload them back to where they went easily, on a freshly installed system.
 
Old 10-01-2014, 03:49 PM   #3
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,278

Rep: Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693
At least you are looking for this ahead of time.

Let me say this. Looking for a magic disk fixer program is going to take a lot more time and effort than making a good backup strategy.

If you use one of the many forensics tools (Caine being a good one) you will certainly be able to pull some things off of a disk (as long as it isn't mechanically broken as Tb0ne mentioned) but the time it is going to take will be exponential compared to making a good backup plan and restoring files.

For reference, on the laptop, I use a combination of Github for code, a remote NFS mount for remote backups, a flash-drive for encrypted important backups, and an external HDD for weekly full/daily incremental backups. If my laptop catches fire, I can lose no more than a day's worth of work. Usually less with version control considered.

If you aren't making backups, then you WILL have an emergency eventually that might cost you your job, or worse, money.
 
Old 10-01-2014, 04:31 PM   #4
mikecolley
LQ Newbie
 
Registered: Apr 2014
Location: All Over USA
Distribution: Mint16 Cinnamon
Posts: 29

Original Poster
Rep: Reputation: Disabled
Fantastic answers TBOne and szboardstretcher!

Yes, backups are king here. I have finally found a backup plan that works for me.

I have 3 really large drives for long term data storage and retreival, the biggest two are backups of everything else. They almost never spin-up and I keep them dis-connected from USB and seldom connect them to the production machine until I need a big file or a backup.

My production machine doesn't even have spinning HD. I run live session cinnamon with persistence on 32GB thumb flash drive. I keep taking the large seldom used files off the flash and keep putting them on the seldom used multi-terrabyte drives. That way I can keep all my daily use files on the 32GB and then when I backup, I do a full 100% weekly backup of the 32GB flash, quick, easy, doesn't take much space and I never have to combine multiple incremental backups to get back up to speed. Those big drives get backed up to the biggest one about once per year (I'm lazy). I find even old magnetic disks to make the least errors over time, particularly if they aren't used much.

I have several times helped others that blow up their HD by unplugging at the wrong time/running a virus/m$-dump/whatever and they ask me for help. I am asking so I can have one more tool in my toolbox to help others.

I will keep this as unsolved for a while so others can contribute. Thank You ALL!
 
Old 10-02-2014, 08:13 AM   #5
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.2
Posts: 18,340

Rep: Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745
As above a dead disk is unrecoverable.
OTOH, if its just corrupt or accidentally deleted, try photorec (recovers all sorts of stuff, but realistically its going to be a lot of work and you may only get bits back...)

If you're helping others out, the best you can do (at least before the fact) is to bore them rigid about backups..
 
Old 10-02-2014, 11:47 AM   #6
mikecolley
LQ Newbie
 
Registered: Apr 2014
Location: All Over USA
Distribution: Mint16 Cinnamon
Posts: 29

Original Poster
Rep: Reputation: Disabled
Thanks chrism01 - and yes I already do bore people stiff telling them how important backups are, funny you should put it that way, exactly what I do.

I don't know anything about MBR or bootloader or etc. Sometimes a drive just won't automount after experiencing a crash for whatever reason. Yes I know hardware failure is catastrophic and not fixable. I was looking for a utility that would search bare metal and if it found a directory structure, it would fix up what is needed so the drive could be mounted again. I don't know much about this stuff so I need a smart turnkey utility for a dummy like me.

I'm aware of a file recovery program but I'm looking for something more macro in approach. I'm not certain if I'm even asking the right question.

I'm asking in advance before the next crash, rather than after the fact.

Thanks! - Mike
 
Old 10-02-2014, 11:58 AM   #7
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,278

Rep: Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693Reputation: 1693
Check out the 'forensic' distros, there are plenty of utilities to do this as Tb0ne and I mentioned.

Here is a link to CAINE, which AFAIK is the more popular distro:

http://www.caine-live.net/

And also ... Backups. Backups. Backups. Etc.
 
Old 10-03-2014, 06:52 AM   #8
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.2
Posts: 18,340

Rep: Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745Reputation: 2745
The problem with " I was looking for a utility that would search bare metal and if it found a directory structure, it would fix up what is needed so the drive could be mounted again" is that it would have to be psychic
Although, in a sense, that's roughly what fsck attempts ...

If you want a filesystem that's impervious to just about all SW errors (not inc deliberate deletes) and a lot of HW errors, try ZFS http://zfsonlinux.org/ & https://en.wikipedia.org/wiki/ZFS
 
Old 10-03-2014, 07:23 AM   #9
Germany_chris
Senior Member
 
Registered: Jun 2011
Location: NOVA
Distribution: Debian 12
Posts: 1,068

Rep: Reputation: 496Reputation: 496Reputation: 496Reputation: 496Reputation: 496
I never thought of using something like this because I keep clones of my drives but something like this could be quite handy especially for USB sticks and memory cards.
 
Old 10-03-2014, 01:59 PM   #10
mikecolley
LQ Newbie
 
Registered: Apr 2014
Location: All Over USA
Distribution: Mint16 Cinnamon
Posts: 29

Original Poster
Rep: Reputation: Disabled
ZFS -- !! -- I'm impressed, almost enough to try it on my LM17 Cinnamon. Any user comments about it, like slow...? or difficult...? or ... , I'm just asking. This sounds like a good way to head off the need for bare metal recovery routines.

Hi szboardstretcher, can you give me a hint about which caine utility to use? No, I didn't look closely at the distribution, maybe I should have before asking this question.

- Mike
 
Old 10-07-2014, 11:28 PM   #11
mikecolley
LQ Newbie
 
Registered: Apr 2014
Location: All Over USA
Distribution: Mint16 Cinnamon
Posts: 29

Original Poster
Rep: Reputation: Disabled
(Lost the first post here is a duplicate)

A friend found a small $ solution. He had a drive FAT32 or NTFS I don't know which, he couldn't mount anywhere. He couldn't mount it on linux or anywhere, he couldn't get his files. He paid a small $ for Restorer-Ultimate and got all his files back. He said he found the reference to Restorer-Ultimate on the Caine-CD-Disk mentioned above.

Yes, it would be nice if someone wrote this for linux and made it public domain. That is what I was asking about initially.

I guess this is the type of solution I was asking for. Good discussion, I am going to mark this thread solved

The discussion was great, but now I'm wondering if I could make a ZFS USB Flash drive with 2X redundancy using LM17 Cinnamon? I suppose if I didn't use the live session I could do it. Yes writing to flash is slow and 2X writing will be even slower, but I like reliability. I guess I would have to turn off the virtual memory file.

Thank You All!

Last edited by mikecolley; 10-07-2014 at 11:35 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How was the first program ever written? thelinuxist Programming 31 08-19-2013 05:03 AM
[SOLVED] A compiled C program copied using a self-written program, failed to work trainee Programming 8 06-27-2013 08:53 PM
running a written program judoka Linux - Newbie 14 07-17-2009 02:43 AM
How to get last file written to directory kd5giv Linux - Networking 4 03-29-2004 03:26 PM
have you ever written a program for linux? today53 General 33 12-05-2003 07:54 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 08:04 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration