LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-04-2008, 02:06 AM   #16
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE 42.1_64+Tumbleweed-KDE, Mint 17.3
Posts: 4,033

Original Poster
Rep: Reputation: Disabled

Yep, thanks for the clarification. I was wrongly assuming that I really could write zeros and ones (low level) to the disk. The readmes of "wipe" and "dban" disabused me of that notion. You are right in the use of urandom "noise" for encryption and zeros for compression of course.
 
Click here to see the post LQ members have rated as the most helpful post in this thread.
Old 06-29-2009, 10:11 AM   #17
H_TeXMeX_H
LQ Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287
Quote:
Originally Posted by Matir View Post
FYI, binary one is 0x01, unless you mean every bit binary one, which would be 0xFF/binary 11111111/decimal 255/octal 377.

And no, no such virtual device exists.

You could do something like:
Code:
tr '\000' '\377' < /dev/zero
Thanks for this, it works great and is a great idea. As far as I can see there is no performance hit from piping to tr. For example if I wanted to overwrite free space with ones instead of zeros in order to clear out files I deleted I would do:

Code:
dd if=/dev/zero | tr '\000' '\377' > file; rm -f file
Possibly useful in wiping a disk with /dev/zero then with /dev/one for a bit of added paranoia relief.
 
Old 06-21-2010, 07:48 AM   #18
jpantano88
LQ Newbie
 
Registered: Jun 2010
Posts: 6

Rep: Reputation: 0
Quote:
Originally Posted by H_TeXMeX_H View Post
Thanks for this, it works great and is a great idea. As far as I can see there is no performance hit from piping to tr. For example if I wanted to overwrite free space with ones instead of zeros in order to clear out files I deleted I would do:

Code:
dd if=/dev/zero | tr '\000' '\377' > file; rm -f file
Possibly useful in wiping a disk with /dev/zero then with /dev/one for a bit of added paranoia relief.
Why are you using '\377' instead of '\777' ?

Translated from hex to binary these are 011111111 and 111111111, respectably. So isn't the second one the proper way to write all ones?
 
Old 06-21-2010, 08:01 AM   #19
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389Reputation: 2389
@jpantano88:

\NNN -> octal representation, not hex (see man tr).

0xFF/binary 11111111/decimal 255/octal 377 (see post #3).

Hope this clears things up.
 
Old 06-21-2010, 10:50 AM   #20
H_TeXMeX_H
LQ Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287
Well, maybe this will clear it up:

Code:
bash-3.1$ dd if=/dev/zero count=1000 | tr '\000' '\377' | hexdump
0000000 ffff ffff ffff ffff ffff ffff ffff ffff
*
1000+0 records in
1000+0 records out
512000 bytes (512 kB) copied, 0.0477885 s, 10.7 MB/s
007d000
 
Old 06-22-2010, 06:54 AM   #21
jpantano88
LQ Newbie
 
Registered: Jun 2010
Posts: 6

Rep: Reputation: 0
Ah, thanks guys, that does make sense now.
 
Old 08-30-2011, 07:32 AM   #22
Havin_it
LQ Newbie
 
Registered: May 2005
Location: Edinburgh, UK
Distribution: Gentoo
Posts: 18

Rep: Reputation: 0
Sorry for the Night of the Living Thread, but this was a top search result for "/dev/one" which I was searching for the same reason as OP (HDD wiping), and I thought this might be worth adding.

I'm doing a thorough wipe on an HDD that's starting to fail, so I need to use ddrescue instead of dd so it won't puke on write errors. The /dev/one kernel patch above would make this easier, but I think it's out-of-date now and I haven't the skill to update it myself. Sidenote: I think it'd be great to have this included in the kernel, but maybe this has already been suggested and rejected?

Anyway, to get the required result I created a file full of ones (using dd and tr as shown above). For faster operation I placed it in tmpfs, so as I only have 1GB RAM I made only a 50MB file. The more free RAM you have, the bigger you can make the file.

Code:
dd if=/dev/zero bs=512 count=100KB | tr '\000' '\377' > /tmp/ones.dat
Then I wrote a small bash script to loop runs of ddrescue, writing to consecutive 50MB chunks of the target drive.

Code:
#!/bin/sh

CNT=0
while ddrescue -f -o $CNT /tmp/ones.dat /dev/sdb; do
	CNT=$(( CNT + 51200000 ))
	echo "$CNT bytes written."
done
This doesn't compare too badly against "ddrescue /dev/zero /dev/sdb" for speed; the main bottleneck is the stop/start of ddrescue (each iteration of the loop took about 2.5s on an Atom N270 cpu) so the bigger the "ones file" you can afford to use, the faster it'll go.

Hope this is some help to somebody somewhere at some point
 
Old 08-30-2011, 07:55 AM   #23
rodrifra
Member
 
Registered: Mar 2007
Location: Spain
Distribution: Debian
Posts: 201

Rep: Reputation: 36
shred is another program to erase your data in a secure way.
 
Old 08-30-2011, 09:27 AM   #24
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE 42.1_64+Tumbleweed-KDE, Mint 17.3
Posts: 4,033

Original Poster
Rep: Reputation: Disabled
Thanks for your input and solution, Havin_it. Side note: I wouldn't choose too big a file with ones, since you want to wipe a failing drive. Normal sector size is 512 bytes. You leave 50 MB gaps (minus the broken sectors) in wiping even now. If you chose larger chunks of data, the gaps would increase accordingly...

Last edited by JZL240I-U; 08-31-2011 at 12:50 AM.
 
Old 08-30-2011, 09:50 AM   #25
H_TeXMeX_H
LQ Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287
Best way to wipe a failing drive:

Take the platters out, run a big magnet over them many times, then melt them down or smash them.
 
Old 08-31-2011, 12:07 PM   #26
Havin_it
LQ Newbie
 
Registered: May 2005
Location: Edinburgh, UK
Distribution: Gentoo
Posts: 18

Rep: Reputation: 0
Quote:
Originally Posted by JZL240I-U View Post
Thanks for your input and solution, Havin_it. Side note: I wouldn't choose too big a file with ones, since you want to wipe a failing drive. Normal sector size is 512 bytes. You leave 50 MB gaps (minus the broken sectors) in wiping even now. If you chose larger chunks of data, the gaps would increase accordingly...
Can you explain this? ddrescue is still writing using a 512B block size, so I'd assumed that it would write to every sector it can physically do so. How are there 50MB gaps?
 
Old 09-02-2011, 02:03 AM   #27
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE 42.1_64+Tumbleweed-KDE, Mint 17.3
Posts: 4,033

Original Poster
Rep: Reputation: Disabled
My bad . At least I assume ddrescue writes its blocks until it encounters a bad block so my warning is moot. Thanks for pointing this out .
 
Old 01-04-2012, 08:11 PM   #28
snmcdonald
Member
 
Registered: Jul 2011
Location: Canada
Distribution: Debian, Arch
Posts: 55

Rep: Reputation: 0
Quote:
Originally Posted by jschiwal View Post
Why not wipe the drive with the urandom device a few times and then zero's. Actually, if you are going encrypt your partitions, you want to start with the partition filled with psuedo-random junk, rather than all zero's or all ones. If on the other hand you are going to reuse the drive for a fresh install and want to create an image backup, having zeroed out free space will allow the image to compress nicely.
Random data is still the best but many government methodologies for data sanitization still require a few passes of ones and zeros before random data:

Quote:
For more security you can use 7-pass algorithms to wipe free space.
- the US Department of Defense DoD 5220.22-M(ECE) (7 passes)
DoD 5220.22-M(ECE) is seven pass overwriting algorithm: first and second passes - with certain bytes and with its compliment, then two passes with random character, then two passes with character and its complement and the last pass - with random character.

- Canadian RCMP TSSIT OPS-II (7 passes)
RCMP TSSIT OPS-II is seven pass overwriting algorithm with three alternating patterns of zeroes and ones and the last pass - with random character (with last pass verification).

- German VSITR (7 passes)
The German standard calls for each sector to be overwritten with three alternating patterns of zeroes and ones and in the last pass with character.

- Bruce Schneier (7 passes)
The Bruce Schneier wiping algorithm has seven passes: first pass - with ones, the second pass - with zeroes and then five times with random characters.
-Wiping Data
 
Old 09-04-2012, 07:36 AM   #29
Havin_it
LQ Newbie
 
Registered: May 2005
Location: Edinburgh, UK
Distribution: Gentoo
Posts: 18

Rep: Reputation: 0
FWIW, I've updated FreeBSoD's patch from page 1 against my Gentoo 3.5.2 kernel:
Code:
--- linux/drivers/char/mem.c	2012-07-21 21:58:29.000000000 +0100
+++ linux-3.5.2-gentoo/drivers/char/mem.c	2012-09-04 12:36:19.744029788 +0100
@@ -668,6 +668,41 @@
 	return written ? written : -EFAULT;
 }
 
+static ssize_t read_one(struct file *file, char __user *buf,
+			 size_t count, loff_t *ppos)
+{
+	size_t written;
+
+	if (!count)
+		return 0;
+
+	if (!access_ok(VERIFY_WRITE, buf, count))
+		return -EFAULT;
+
+	written = 0;
+	while (count) {
+		unsigned long unwritten = 0;
+		unsigned long chunkIterator;
+		size_t chunk = count;
+		static int one = ~0;
+
+		if (chunk > PAGE_SIZE)
+			chunk = PAGE_SIZE;	/* Just for latency reasons */
+		for(chunkIterator = 0; chunkIterator < chunk; chunkIterator++) {
+			unwritten += copy_to_user(&buf[written + chunkIterator], &one, 1);
+		}
+		written += chunk - unwritten;
+		if (unwritten)
+			break;
+		if (signal_pending(current))
+			return written ? written : -ERESTARTSYS;
+		buf += chunk;
+		count -= chunk;
+		cond_resched();
+	}
+	return written ? written : -EFAULT;
+}
+
 static int mmap_zero(struct file *file, struct vm_area_struct *vma)
 {
 #ifndef CONFIG_MMU
@@ -739,6 +774,9 @@
 #define open_mem	open_port
 #define open_kmem	open_mem
 #define open_oldmem	open_mem
+#define one_lseek	null_lseek
+#define write_one	write_null
+#define mmap_one	mmap_zero
 
 static const struct file_operations mem_fops = {
 	.llseek		= memory_lseek,
@@ -782,6 +820,13 @@
 	.write		= write_zero,
 	.mmap		= mmap_zero,
 };
+ 
+static const struct file_operations one_fops = {
+	.llseek		= one_lseek,
+	.read		= read_one,
+	.write		= write_one,
+	.mmap		= mmap_one,
+};
 
 /*
  * capabilities for /dev/zero
@@ -831,6 +876,7 @@
 #ifdef CONFIG_CRASH_DUMP
 	[12] = { "oldmem", 0, &oldmem_fops, NULL },
 #endif
+	[13] = { "one", 0666, &one_fops, &zero_bdi },
 };
 
 static int memory_open(struct inode *inode, struct file *filp)
It seems to work OK but I'm not a C coder (the adaptation was done by cut, paste, guesswork and a bit of research) so please tell me if the code could be more "elegant".

I'd like to see this in the kernel; for all that has been said above about the value of a pass of ones in disk-erasure, the fact remains that a number of govt-endorsed algorithms do rely on it, which gives it value I think. Who knows what other uses might be thought of?
 
Old 09-04-2012, 08:02 AM   #30
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE 42.1_64+Tumbleweed-KDE, Mint 17.3
Posts: 4,033

Original Poster
Rep: Reputation: Disabled
Hmmm. I've never done work like this, i.e. programming kernel modules. But I guess, this is not the appropriate place to get it inserted into the kernel. Perhaps asking in the kernel mailing list whether the kernel chiefs are interested?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
using flash drive changes device /dev/sr0 to /dev/sr1 for mapping to /dev/pktcdvd/0? lugoteehalt Linux - Software 3 10-24-2007 10:27 AM
/dev/audio, /dev/dsp and /dev/mixer missing in Debian Etch 1337_penguin Debian 2 04-11-2007 11:33 AM
/dev/cdrom links to /dev/sg0 instead of /dev/sr0 -why? Yalla-One Slackware 3 10-01-2006 07:02 PM
/dev/tty0, /dev/tty1, /dev/tty10...and so on...which should be used for a device ??? b0nd Slackware 2 04-02-2006 08:14 AM
I cannot access EITHER CD drive! And there's no /dev/hdc or /dev/hdd or /dev/cdrom! Dmalic Linux - Hardware 13 11-18-2005 07:11 PM


All times are GMT -5. The time now is 08:45 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration