LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-04-2012, 12:50 PM   #31
Havin_it
LQ Newbie
 
Registered: May 2005
Location: Edinburgh, UK
Distribution: Gentoo
Posts: 18

Rep: Reputation: 0

Quote:
Originally Posted by JZL240I-U View Post
Hmmm. I've never done work like this, i.e. programming kernel modules. But I guess, this is not the appropriate place to get it inserted into the kernel.
Haha, yes I agree; despite my low post-count here I've been knocking around long enough to know the protocols (even if I don't follow all of them!). Just an opinion, not a call to arms.

Quote:
Originally Posted by JZL240I-U View Post
Perhaps asking in the kernel mailing list whether the kernel chiefs are interested?
Weeelll, I sorta jumped a bit past that stage (I can't stand mailing lists, the forum paradigm should have buried the damn archaic things by now) but that was a short conversation.

I already figured this must have been considered before at some point, and I'm satisfied with the reasons given. I mainly just updated the thread in case anyone else was looking for a /dev/one for their own reasons.
 
Click here to see the post LQ members have rated as the most helpful post in this thread.
Old 09-05-2012, 02:11 AM   #32
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE 42.1_64+Tumbleweed-KDE, Mint 17.3
Posts: 3,950

Original Poster
Rep: Reputation: Disabled
Understood. Thanks for coming back and clearing up the debris .
 
Old 04-11-2013, 02:43 PM   #33
BruceFerjulian
LQ Newbie
 
Registered: Apr 2013
Posts: 5

Rep: Reputation: Disabled
Swap Bytes

Read this thread months ago and since found something that works easier for me.

dd if=/dev/zero of=/dev/xxxx conv=swab

The key for me was the ( convert = swap bytes ) in the ( dd ) command. Read zeros ( /dev/zero ), swap the zeros to ones, then write out to ( /dev/xxx_whatever ).
 
Old 04-11-2013, 03:19 PM   #34
273
LQ Addict
 
Registered: Dec 2011
Location: UK
Distribution: Debian Sid AMD64, Raspbian Wheezy, various VMs
Posts: 5,995

Rep: Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611
I'm not sure why anyone would go to the trouble of writing more than one pass of /dev/urandom. All you're doing is wasting computer and disk-write cycles to stop the NSA from reading your drive and if you're really keeping information on your drive that the NSA want then you'll be too busy half-drowning in an orange jumpsuit every day for the rest of your life to care whether they got it or not.
Cyber criminals and rival businesses that can afford an electron microscope probably couldn't care less about your data.
The Gutmann paper was written around seven years ago about a possible method of recovering data from the hard drives at the time using an electron microscope and some luck. Drives have become massively more dense since then and nobody has ever recovered data from a wiped drive in practice that I have read about.
 
Old 04-12-2013, 02:01 AM   #35
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE 42.1_64+Tumbleweed-KDE, Mint 17.3
Posts: 3,950

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by BruceFerjulian View Post
...
dd if=/dev/zero of=/dev/xxxx conv=swab

...Read zeros ( /dev/zero ), swap the zeros to ones, then write out to ( /dev/xxx_whatever ).
The man page says
Code:
swab     swap every pair of input bytes
In my understanding that means bytes A and B get to be bytes B and A. What I was aiming at would be inverting at the bit level, i.e. byte A and B coming from /dev/zero having the values of "00000000" and "00000000" get to be "11111111" and "11111111". That is not what "swab" is about as far as I can see.
 
Old 09-18-2013, 07:48 AM   #36
gsatish10
LQ Newbie
 
Registered: Sep 2009
Posts: 1

Rep: Reputation: 0
Thanks to all for great support.
I learnet today "/dev/zero" and "/dev/one"

thank you
 
Old 10-11-2013, 01:08 PM   #37
hpfeil
Member
 
Registered: Nov 2010
Location: Tucson, Arizona US
Distribution: Slackware Current, custom kernel, amd64, Beyond LinuxFromScratch
Posts: 130
Blog Entries: 1

Rep: Reputation: Disabled
Due to popular demand, a practical use of /dev/one:

In order to erase a USB memory key (thumb drive, memory stick), you need to write all ones, not zeros. A /dev/one would come in handy for erasing memory sticks before discarding or selling them. With Kingston Technologies' terabyte thumb drive, erasing a large NAND flash device would be tedious using data manipulation with cpu cycles.

https://en.wikipedia.org/wiki/Flash_memory "Block Erasure"
https://en.wikipedia.org/wiki/Field_electron_emission

The memory chip is programmed using tunnel injection (a field electron emission effect) specifically a quantum process called Fowler–Nordheim tunneling. The process used for erasing is called tunnel release.
 
Old 10-11-2013, 04:21 PM   #38
273
LQ Addict
 
Registered: Dec 2011
Location: UK
Distribution: Debian Sid AMD64, Raspbian Wheezy, various VMs
Posts: 5,995

Rep: Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611
Why would /dev/urandom not work?
 
Old 10-12-2013, 12:42 PM   #39
hpfeil
Member
 
Registered: Nov 2010
Location: Tucson, Arizona US
Distribution: Slackware Current, custom kernel, amd64, Beyond LinuxFromScratch
Posts: 130
Blog Entries: 1

Rep: Reputation: Disabled
/dev/urandom and /dev/random will write bytes of ones and zeros. That will not trigger a "block erasure". You have no control over the location that the front-end controller will place those bytes. There may be blocks that are not even touched, or part of blocks. Only writing 0xFF onto a block will trigger the erasure of the entire block. Forensic analysis of the flash chip can read the entire block of parts that received a random-pattern byte. Refer to https://en.wikipedia.org/wiki/File:USB_flash_drive.JPG, it is the controller that is the interface between the USB hardware and the flash chip. Please refer to http://pdf.datasheetcatalog.com/data.../389202_DS.pdf, Page 9 (you may find other datasheets for the same product, look for the "Product Information" page). Erasure is executed on a block basis. You cannot execute a bit-by-bit erasure. Random data reads and writes could be anywhere, the controller chip keeps track of that. The chip has a 2-cycle block erase command: 0x60 - erase setup, then 0xD0 - erase confirm. With lots of writes of random bytes, most likely the bytes are written to cache registers while the data registers are being programmed into memory cells for speed.

Disclaimer: It has been twenty years since I've had to read a data sheet for timing diagrams. Forensic analysis starts by removing the flash chip and setting it up to apply pulses to the control pins and get the output pins on a digital storage scope (controlled and data stored on a VXI system). Then it's just a matter of scanning the pages for data.
 
Old 10-12-2013, 12:44 PM   #40
273
LQ Addict
 
Registered: Dec 2011
Location: UK
Distribution: Debian Sid AMD64, Raspbian Wheezy, various VMs
Posts: 5,995

Rep: Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611Reputation: 1611
So how is writing ones better than random? Why does ones tell the controller to write to all blocks and random doesn't?
 
Old 10-19-2013, 01:13 PM   #41
hpfeil
Member
 
Registered: Nov 2010
Location: Tucson, Arizona US
Distribution: Slackware Current, custom kernel, amd64, Beyond LinuxFromScratch
Posts: 130
Blog Entries: 1

Rep: Reputation: Disabled
"USB sticks, CompactFlash cards and other removable flash media are not MTD devices. They are block devices. They do contain flash chip inside, but they also contain some translation layer above which emulates block device. This translation layer is implemented in hardware. So for outside world these devices look exactly as hard drives, not like MTD devices.
...
"Also note, these devices are "black boxes". The way they implement this flash-to-block device translation layer is not usually published. And in many cases the algorithms used at this layer are far from brilliant. For example, many USB sticks and other cards lose data in case of unclean reboots/power cuts."

I was taught to read at least 3 times before thinking about using the data therein (timing diagrams, etc.). This site has lots of information about "Memory Technology Devices" and why they are not block devices. Recall the wiki image of the innards of a USB stick. Two chips, one is the flash device, which the datasheet describes in excruciating detail, the other is the mysterious controller device that sits between the USB interface and the flash chip, or MTD. It is the inner workings of the controller chip which makes the MTD appear like a block device to the OS. This FAQ explains the three operations that MTDs understand: read from eraseblock, write to eraseblock, and erase eraseblock. You invoke the "erase block" operation by sending 0xFF to the device (sixteen bits). Writing anything else gets mysteriously translated by the controller chip into MTD blocks in a way that simulates a block device.

http://www.linux-mtd.infradead.org/faq/general.html
 
Old 10-21-2013, 02:58 AM   #42
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE 42.1_64+Tumbleweed-KDE, Mint 17.3
Posts: 3,950

Original Poster
Rep: Reputation: Disabled
Thank you, hpfeil, for your thorough explanation(s) and the enlightening statements concerning flash .
 
Old 10-21-2013, 11:06 AM   #43
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,600

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
Most application that attempt such things will create a designated buffer with the values desired...

Then just write that buffer out as many times as it takes.

This is faster than actually READING buffers.... which does nothing but make your overhead larger.

/dev/zero gets used a good bit - creating swap files (not partitions), virtual disks, empty data files (preallocated files with nothing in it - makes for faster updates).
 
Old 06-05-2014, 03:00 PM   #44
metaschima
Senior Member
 
Registered: Dec 2013
Distribution: Slackware
Posts: 1,982

Rep: Reputation: 490Reputation: 490Reputation: 490Reputation: 490Reputation: 490
A simple C program to fill with any character passed in as hex:

Code:
#include <stdio.h>
#include <stdint.h>

#define BUFFER_MAX 61440

int main (int argc, char * argv[])
{
	uint8_t buffer[BUFFER_MAX];
	unsigned int fill;
	unsigned int i;

	if (2 == argc)
	{
		sscanf (argv[1], "%x", &fill);
	}
	else
	{
		printf ("Usage: %s fill_char\n", argv[0]);
		return 1;
	}

	setvbuf (stdout, NULL, _IONBF, 0);

	for (i = 0; i < BUFFER_MAX; i++)
	{
		buffer[i] = fill;
	}

	while (fwrite (buffer, 1, BUFFER_MAX, stdout))
	{
	}

	return 0;
}
Adjust the buffer size as needed (check cpuinfo for your cache size).

Example:

Code:
bash-4.2$ fill '0xff' | dd bs=6M count=1000 of=/dev/null iflag=fullblock
1000+0 records in
1000+0 records out
6291456000 bytes (6.3 GB) copied, 1.18898 s, 5.3 GB/s
bash-4.2$ dd if=/dev/zero bs=6M count=1000 | tr '\000' '\377' > /dev/null
1000+0 records in
1000+0 records out
6291456000 bytes (6.3 GB) copied, 4.20534 s, 1.5 GB/s
bash-4.2$ fill '0xff' | hexdump
0000000 ffff ffff ffff ffff ffff ffff ffff ffff
*
bash-4.2$ fill '0xa5' | hexdump
0000000 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5
*
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
using flash drive changes device /dev/sr0 to /dev/sr1 for mapping to /dev/pktcdvd/0? lugoteehalt Linux - Software 3 10-24-2007 11:27 AM
/dev/audio, /dev/dsp and /dev/mixer missing in Debian Etch 1337_penguin Debian 2 04-11-2007 12:33 PM
/dev/cdrom links to /dev/sg0 instead of /dev/sr0 -why? Yalla-One Slackware 3 10-01-2006 08:02 PM
/dev/tty0, /dev/tty1, /dev/tty10...and so on...which should be used for a device ??? b0nd Slackware 2 04-02-2006 09:14 AM
I cannot access EITHER CD drive! And there's no /dev/hdc or /dev/hdd or /dev/cdrom! Dmalic Linux - Hardware 13 11-18-2005 08:11 PM


All times are GMT -5. The time now is 02:14 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration