LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices

Reply
 
Search this Thread
Old 11-13-2009, 10:55 AM   #16
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728

Quote:
Originally Posted by lewc View Post
why is smeezekitty's contribution unnacceptable, I think it is very fair. Overwriting data although lengthy is a service I offer professionally.
No-one said it was unacceptable----in most situations, it's just overkill.
 
Old 11-13-2009, 11:35 AM   #17
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,533
Blog Entries: 28

Original Poster
Rep: Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176
Quote:
Originally Posted by lewc View Post
why is smeezekitty's contribution unnacceptable, I think it is very fair. Overwriting data although lengthy is a service I offer professionally.
AIUI smeezekitty's technique does not address data written to blocks that have subsequently identified as bad and hence mapped out. The dd command, working inter alia through the hardware interface cannot write to such blocks so their contents remain as readable as the degree of actual badness permits. This could allow recovery of significant data.
 
Old 11-14-2009, 05:43 AM   #18
Guttorm
Senior Member
 
Registered: Dec 2003
Location: Trondheim, Norway
Distribution: Debian and Ubuntu
Posts: 1,134

Rep: Reputation: 218Reputation: 218Reputation: 218
Hi

I talked about this with an expert some time ago. He also said that recovering overwritten data was a myth. Maybe it was possible with very old disks, but nowadays the density in disks in so high, that it simply is impossible. If it was possible, don't you think the disk manufacturers would use it to increase disk space?

Utilities like shred can be bad in the sense that it can fool people to think a file is gone, when the contents could be in swap, journal files, automatic backups or at the end of sectors. You need to overwrite the entire disk, and that you cannot do unless you use a live cd or similar.

Instead of using dd, you can delete all partitions on a disk, make a new one that uses the entire disk and then use mkfs.ext2 with the -c option twice. Then everything is overwritten, and you also get the benefit of a health check of the disk. If no sectors are bad, use the disk for something else. If not, destroy and recycle it.
 
Old 11-14-2009, 06:38 AM   #19
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
Quote:
Originally Posted by catkin View Post
AIUI smeezekitty's technique does not address data written to blocks that have subsequently identified as bad and hence mapped out. The dd command, working inter alia through the hardware interface cannot write to such blocks so their contents remain as readable as the degree of actual badness permits. This could allow recovery of significant data.
Really!!
Are you saying that--if I read the whole disk with dd--I will get an output which does not include bad blocks, but still is in a contiguous sequence? This does not sound right to me---I had thought that fencing off of bad blocks was a function of the filesystem---which dd does not use.
 
Old 11-14-2009, 07:30 AM   #20
H_TeXMeX_H
Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269
Quote:
Originally Posted by pixellany View Post
Really!!
Are you saying that--if I read the whole disk with dd--I will get an output which does not include bad blocks, but still is in a contiguous sequence? This does not sound right to me---I had thought that fencing off of bad blocks was a function of the filesystem---which dd does not use.
I thought so too. But then, bad blocks are bad blocks ... you can't read them anyway, and you expect to get data from them ?
 
Old 11-14-2009, 08:32 AM   #21
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,533
Blog Entries: 28

Original Poster
Rep: Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176
Quote:
Originally Posted by H_TeXMeX_H View Post
I thought so too. But then, bad blocks are bad blocks ... you can't read them anyway, and you expect to get data from them ?
AIUI, AFAIK (and other expressions indicating tentative knowledge) bad block mapping is done in hardware and not visible to the OS except when running HDD OEM utilities. This link seems to support that concept. A single 512 byte block could contain enough data to be significant; if bad blocks develop in clusters, more than 512 bytes of contiguous data could be preserved<delete>, hidden from the OS' file system calls and thus from the dd command. I guess they might be available via the HDD hardware driver, reading the firmware, but not via the file system calls</delete>.

EDIT:

Scrub the red text. Stupidly I was forgetting that the dd command does not access the file system but the block device files -- either for the whole HDD or for a partition.

It is impractical to manufacture HDDs without bad blocks. From this StorageReview page: "On modern hard disks, a small number of sectors are reserved as substitutes for any bad sectors discovered in the main data storage area. During testing, any bad sectors that are found on the disk are programmed into the controller. When the controller receives a read or write for one of these sectors, it uses its designated substitute instead, taken from the pool of extra reserves. This is called spare sectoring. In fact, some drives have entire spare tracks available, if they are needed. This is all done completely transparently to the user, and the net effect is that all of the drives of a given model have the exact same capacity and there are no visible errors. This means that the operating system never sees the bad areas, and therefore never reports "bad sectors"".

The testing mentioned above is part of the manufacturing process but blocks that were good during that testing may become defective over the lifetime of the drive. From the same page: "These will normally be detected either during a routine scan of the hard disk for errors (the easy way) or when a read error is encountered trying access a program or data file (the hard way). When this happens, it is possible to tell the system to avoid using that bad area of the disk. Again, this can be done two ways. At the high level, the operating system can be told to mark the area as bad and avoid it (creating "bad sector" reports at the operating system level.). Alternately, the disk itself can be told at a low level to remap the bad area and use one its spares instead".

From this StorageReview page: "Many drives are smart enough to realize that if a sector can only be read after retries, the chances are good that something bad may be happening to that sector, and the next time it is read it might not be recoverable. For this reason, the drive will usually do something when it has to use retries to read a sector (but usually not when ECC will correct the problem on the fly). What the drive does depends on how it is designed.

[snip some S.M.A.R.T. stuff]

Today's hard disks will also often take corrective action on their own if they detect that errors are occurring. The occasional difficulty reading a sector would typically be ignored as a random occurrence, but if multiple retries or other advanced error correction procedures were needed to read a sector, many drives would automatically mark the sector bad and relocate its contents to one of the drive's spare sectors. In doing so, the drive would avoid the possibility of whatever problem caused the trouble worsening, and thereby not allow the data to be read at all on the next attempt
".

It is the above behaviour which means dd cannot be used to erase all data from an HDD. The bad blocks are still on the HDD, their contents may not have been erased (there's no reason for the HDD manufacturer to design their HDDs to do so) and they can no longer be addressed (any routine attempt to do so would be diverted to the appropriate spare sectors) but it may be possible to interface with the HDD firmware using special diagnostic commands and retrieve the contents of those sectors which could "only be read after retries".

Last edited by catkin; 11-14-2009 at 11:28 AM.
 
  


Reply

Tags
data, hdd, multiple, write


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How Linux writes/deletes from a hdd tkbonito Linux - General 2 07-29-2009 11:21 AM
Reduce HDD writes? Cyberman Linux - Hardware 1 09-29-2008 02:49 PM
FC6 Server Randomly Writes Data To HDD jmoschetti45 Linux - Server 6 05-22-2007 03:16 PM
Using OpenOffice source to render data for charts into images ? WhiskeyTangoFoxtrot Programming 4 01-26-2007 05:25 AM
Gnome 2.6, odd hdd writes. slackMeUp Slackware 11 10-07-2004 02:42 AM


All times are GMT -5. The time now is 03:04 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration