LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 07-24-2007, 11:58 AM   #1
Daws
Member
 
Registered: May 2006
Location: UK
Distribution: Debian, Linux From Scratch
Posts: 416
Blog Entries: 1

Rep: Reputation: 30
Overwriting data with zeroes prior to deletion.


First some background (this may seem a little odd otherwise). Back in March I made my first shell script, and with some help from the forums it was working nicely.

Code:
#!/bin/sh

for file in $@; do
    if [ -f "$file" ]; then
        shred -zvf "$file"
        rm -fv "$file"
    elif [ -d "$file" ]; then
        cd "$file"
        find -type f -exec shred -zvf {} \;
        cd ..
        rm -rfv "$file"
    else
        echo "File $file Does Not Exist"
    fi
done
I haven't really used it much, but that is besides the point.

Recently, after a few experiments that went awry, I have taken to backing up my entire hard drive image using dd, and compressing it with gzip.

Using shred with the -z option means the data to be shredded is overwritten with zeroes on the final pass, as you can imagine a string of zeroes compresses quite nicely and keeps the backup image nice and small.

You may see where I am going with this... At the moment if I wanted to, I could call the script myself or set up an alias etc. But, for example, when I remove some unneeded software via apt. dpkg calls rm on its own, it doesn't heed aliases etc. and therefore it does not overwrite the old data with zeroes. All that previously removed (ghostly?) data ends up on my backup image.

I have a feeling that the answer does not lie with the script. So: is it possible to setup a system that always overwrites data with zeroes before removing it?

Thanks for your attention.

PS: using Ext2.
 
Old 07-24-2007, 12:08 PM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 26,990
Blog Entries: 54

Rep: Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743Reputation: 2743
The "adventurous" way would be to rename 'rm' and replace it with a script called 'rm'. For compatibility reasons you'll have to honour all the switches your version of 'rm' uses and on top of that using it will make your system significantly slower :-]


Recently, after a few experiments that went awry, I have taken to backing up my entire hard drive image using dd, and compressing it with gzip.
If you don't mind, what's *so* important "regular" backups aren't good enough?
 
Old 07-24-2007, 07:06 PM   #3
Junior Hacker
Senior Member
 
Registered: Jan 2005
Location: North America
Distribution: Debian testing Mandriva Ubuntu
Posts: 2,687

Rep: Reputation: 59
In post #6 of the thread link below is a suggestion, I had a hard time believing it and did show someone else this post and asked him to report back if he tried it. Still got no response, and I can't find the time to test this, at least not yet.
http://www.linuxquestions.org/questi...d.php?t=557866
EDIT: Would be nice if you were to try it and report back.

Last edited by Junior Hacker; 07-24-2007 at 07:08 PM.
 
Old 07-25-2007, 11:48 AM   #4
Daws
Member
 
Registered: May 2006
Location: UK
Distribution: Debian, Linux From Scratch
Posts: 416
Blog Entries: 1

Original Poster
Rep: Reputation: 30
Thank you for the replies. Well this was fun.

Quote:
Originally Posted by unSpawn
The "adventurous" way would be to rename 'rm' and replace it with a script called 'rm'. For compatibility reasons you'll have to honour all the switches your version of 'rm' uses and on top of that using it will make your system significantly slower :-]
This was the first one I tried. I moved rm to rm-old and modified the script in my first post accordingly (I couldn't be bothered with honouring the switches, I just put up with the occasional "File -rf does not exist"). And ... it seems to work, for some programs...

Going back to the dpkg thing, dpkg only calls rm when it is installing a package, and even then it only uses it to remove some temporary files it creates. It doesn't use rm when removing a package. I think in this case it bypasses the coreutils rm and does it's own thing, don't know what though.

Quote:
Originally Posted by unSpawn
If you don't mind, what's *so* important "regular" backups aren't good enough?
There's nothing really important, this is more a "can I" exercise, rather than something I actually need to do. A few weeks ago I was trying to get wine to work, I chose the wrong gl libraries and bang, major crash, that somehow scrambled the mbr. This, combined with the fact that the Debian netinstallers seem to have difficulty directly installing Sid at the moment, lead me onto backing up entire disk images rather than separate files.

Quote:
Originally Posted by Junior Hacker
In post #6 of the thread link below is a suggestion
Now this was slightly more successful. I didn't know loop could be used on ext2 images like that, I thought it was for cds/dvds only. The results are rather impressive, I tried backing up my / partiton (only around 300Mb, didn't take very long) and well see for yourself:

Code:
Daws:/home/tmp.lPPhmK4191# ls -lh
-rw-r--r-- 1 root root 283M 2007-07-25 16:49 hda1.img
-rw-r--r-- 1 root root 192M 2007-07-25 16:49 hda1-notzeroed.img.gz
-rw-r--r-- 1 root root  14M 2007-07-25 16:51 hda1-zeroed.img.gz
Daws:/home/tmp.lPPhmK4191#
Only one drawback, it is a pain in the rear if you want to back up on a regular basis.

Thanks again.

Last edited by Daws; 07-25-2007 at 11:51 AM.
 
Old 07-25-2007, 03:56 PM   #5
Junior Hacker
Senior Member
 
Registered: Jan 2005
Location: North America
Distribution: Debian testing Mandriva Ubuntu
Posts: 2,687

Rep: Reputation: 59
Have you verified the image by decompressing it back into a clean partition to see if it's exactly like the partition it was taken from?
 
Old 07-25-2007, 05:10 PM   #6
Junior Hacker
Senior Member
 
Registered: Jan 2005
Location: North America
Distribution: Debian testing Mandriva Ubuntu
Posts: 2,687

Rep: Reputation: 59
Quote:
Originally Posted by Daws
Only one drawback, it is a pain in the rear if you want to back up on a regular basis.
Glad you mentioned this, because I love bragging about my easy method.
I use bootitng as my boot manager, it comes with all the partitioning tools you need, including imaging partitions without un-allocated data (deleted or removed stuff). When using bootitng for the multi-boot manager features, one must install it to the hard drive and maybe eventually pay for it. In this scenario, it only creates primary partitions (up to 255 on one drive or up to eight drives, SATA or PATA), below is my current partition scheme:
Code:
Bootitng = 8MB
Windows Vista = 15GB
Windows XP Media Center 2005 = 15GB
Playground (Windows XP Pro testing OS) = 15GB
Mandriva = 10GB
Linux Swap = 2GB
Debian Testing (Loaded up with the works for playing) = 10GB
Debian Testing (Slimmed down for working, faster) = 10GB
VM XP (Windows XP Pro as virtual host) = 30GB
Free space = 21.5GB
Ntfs Shared Data Partition = 110GB
When I used to have Fedora 6 & 7, I would make a backup of it's partition once a week just before applying updates if it seemed to be running good, and delete the previous image. Because allot of times when you applied updates to Fedora, you had major issues upon a re-boot, so it was nice to have an image of it prior to applying the bad updates, sometimes all one needed to do was wait another week and do updates again and the bad update had a fix already, other times I would resolve the issue in various ways.
All my operating systems are backed up as an image file in the data partition and a copy in an external USB backup drive. The typical Linux partition would be approximately 55% used up, because bootitng does not include un-allocated space in it's images, the image of a typical Linux is 1.5GB in size compressed. The Windows and Mac OS X (which is on a PATA disk) are between 2GB & 2.6GB in size.
Here's the punch line:
I boot up the computer and write zeros across the bad Fedora (example), then re-create it's partition either in the same spot or in some other free space on the drive, then I paste a copy of the good Fedora image from the data partition in the new / partition and configure the boot menu to look at the new partition and boot it up. This entire procedure of wiping it out and loading an image takes 8 minutes on my Dell XPS Gen 5 with a Pentium 650 (3.4GHz processor). When creating an image of a 10GB Linux partition, takes only 4 minutes "tops".
And it's all done with a few clicks of the mouse, the only typing needed is when you give your partition/image a name.
The 30GB VM XP partition in my layout was derived from the 15GB Windows XP Pro image, then the partition was resized to 30GB in about a minute using the same boot manager/partition god. And if you don't install it in it's own 8MB partition to the drive, you'll have to stick to the traditional partitioning scheme with 3 primary, one extended, and 59 or so logical partitions. But you can use it's powerful features without obligation.

Last edited by Junior Hacker; 07-25-2007 at 05:16 PM.
 
Old 07-25-2007, 05:13 PM   #7
Daws
Member
 
Registered: May 2006
Location: UK
Distribution: Debian, Linux From Scratch
Posts: 416
Blog Entries: 1

Original Poster
Rep: Reputation: 30
Quote:
Originally Posted by Junior Hacker
Have you verified the image by decompressing it back into a clean partition to see if it's exactly like the partition it was taken from?
I just did, it mounts normally, no errors. All the md5sums checkout, it appears normal. It looks good, I cant think of any other way to test it meaningfully at present.
 
Old 07-25-2007, 05:23 PM   #8
Junior Hacker
Senior Member
 
Registered: Jan 2005
Location: North America
Distribution: Debian testing Mandriva Ubuntu
Posts: 2,687

Rep: Reputation: 59
Quote:
Originally Posted by Daws
I just did, it mounts normally, no errors. All the md5sums checkout, it appears normal. It looks good, I cant think of any other way to test it meaningfully at present.
That's excellent, now we have two working solutions in one thread. One is GUI based, the other is CLI, to satisfy any preference.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Precautionary question prior to install rhomp2002 Slackware 4 05-13-2007 08:18 PM
Looking for some advice prior to Installation DarkeStryke Mandriva 6 11-30-2006 10:37 AM
pflogsumm generates zeroes for results rioguia Linux - Software 1 09-29-2004 03:29 PM
Questions prior to Install jupitercore Debian 3 12-10-2003 02:07 PM
Vm Prior. purelific Linux - General 2 03-31-2002 10:24 PM


All times are GMT -5. The time now is 01:54 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration