[SOLVED] Dude necessary time to do dd if=/dev/urandom of=/dev/sdx

Gugur · 11-18-2017, 06:02 AM

I want to encrypt my slackware in my notebook (i5 2520, 1TB HDD 5400rpm , 8 GB ram).

How approximate time it is necesarry to urandom 1 TB of disk?

Thanks.

jlinkels · 11-18-2017, 06:10 AM

I hope you understand you don't encrypt the disk, but overwrite it with random characters.

The specific answer to your question is "long", likely "very long", and maybe even "extremely long".

Why don't you just try to fill, say a 100 MB, file with random characters. You measure that period of time and multiply it with 10,000.

Really, LQ is not the GUI version of dc.

jlinkels

tazza · 11-18-2017, 08:01 AM

With dd ram is pretty much irrelevant- and your question is open ended... You've left out BS= so are you writing 1kb, 1MB?

Guessing you are using kali and trying to figure things out for a question like this... sorry but it won't cut it around these parts. Maybe go to Ubuntu forums.

Edit to add: yes I know the default size before anyone jumps on me. But no one... I mean no one uses default size. 1TB drive would take an eternity.

GazL · 11-18-2017, 08:37 AM

Using /dev/urandom shouldn't be too bad, the bottleneck will most likely be the speed of your disk. 5400rpm disks tend not to be the fastest performers: my guess would be around 4-5 hours.

Use /dev/random and you'll see the heat-death of the universe before it completes.

P.S. Prefilling with random bytes is an extra step, that's not strictly necessary. You might want to consider skipping it.

rknichols · 11-18-2017, 09:10 AM

Quote:

Originally Posted by GazL

Using /dev/urandom shouldn't be too bad, the bottleneck will most likely be the speed of your disk. 5400rpm disks tend not to be the fastest performers: my guess would be around 4-5 hours.

No, /dev/urandom is actually pretty slow. You'll get maybe 15 MB/s on a fast machine. I get 13 MB/s with an Intel Core i5 @ 3.5 GHz. That works out to over 20 hours to fill 1 TB.

You want to go faster? Encrypt zeros with a random key and use that to fill the disk.

Code:

cryptsetup open --type plain -d /dev/urandom /dev/<block-device> to_be_wiped
dd if=/dev/zero of=/dev/mapper/to_be_wiped bs=1M
cryptsetup close to_be_wiped

That will run at the speed of your disk.

ntubski · 11-18-2017, 09:47 AM

Quote:

Originally Posted by rknichols

No, /dev/urandom is actually pretty slow. You'll get maybe 15 MB/s on a fast machine. I get 13 MB/s with an Intel Core i5 @ 3.5 GHz.

Hmm, I see 224 MB/s, am I counting wrong?

Code:

$ grep 'model name' /proc/cpuinfo 
model name	: Intel(R) Core(TM)2 Duo CPU     P8600  @ 2.40GHz
model name	: Intel(R) Core(TM)2 Duo CPU     P8600  @ 2.40GHz
$ time dd if=/dev/urandom of=/dev/null bs=1MB count=1000
1000+0 records in
1000+0 records out
1000000000 bytes (1.0 GB, 954 MiB) copied, 4.46424 s, 224 MB/s

real	0m4.466s
user	0m0.000s
sys	0m4.440s

GazL · 11-18-2017, 10:05 AM

Intel(R) Core(TM) i3-5157U CPU @ 2.50GHz

Code:

test@ws1:~$ time dd if=/dev/urandom of=/dev/null bs=1MB count=1000
1000+0 records in
1000+0 records out
1000000000 bytes (1.0 GB, 954 MiB) copied, 6.05859 s, 165 MB/s

real    0m6.065s
user    0m0.005s
sys     0m6.060s

My machine isn't quite that fast, but 165MB/s is still much faster than the write speed of my disk.

Gerard Lally · 11-18-2017, 10:27 AM

hmmmm -- 8-core AMD?

GazL · 11-18-2017, 10:44 AM

Quote:

Originally Posted by Gerard Lally

hmmmm -- 8-core AMD?

Very old kernel. /dev/urandom has probably undergone much change since 4.4.

rknichols · 11-18-2017, 12:27 PM

Quote:

Originally Posted by GazL

Very old kernel. /dev/urandom has probably undergone much change since 4.4.

That must be the explanation. Yes, the CentOS 6 machine I used for my test is based on a really old kernel (2.6.32).

bassmadrigal · 11-18-2017, 12:48 PM

That's odd... maybe it's an AMD thing? Ryzen 7 1800x with an NVMe drive only getting around 75MB/s

Code:

jbhansen@craven-moorhead:/share/tv/documentary$ grep 'model name' /proc/cpuinfo
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
model name      : AMD Ryzen 7 1800X Eight-Core Processor
jbhansen@craven-moorhead:/share/tv/documentary$ time dd if=/dev/urandom of=/dev/null bs=1MB count=1000
1000+0 records in
1000+0 records out
1000000000 bytes (1.0 GB, 954 MiB) copied, 13.6742 s, 73.1 MB/s

real    0m13.675s
user    0m0.000s
sys     0m13.663s

In comparison to writing a tempfile with 8GB of zeros:

Code:

jbhansen@craven-moorhead:~$ time dd if=/dev/zero of=tempfile bs=1M count=8096 conv=fdatasync,notrunc status=progress
8474591232 bytes (8.5 GB, 7.9 GiB) copied, 5.00032 s, 1.7 GB/s
8096+0 records in
8096+0 records out
8489271296 bytes (8.5 GB, 7.9 GiB) copied, 5.33874 s, 1.6 GB/s

real    0m5.340s
user    0m0.000s
sys     0m5.037s

kjhambrick · 11-18-2017, 02:43 PM

bassmadrigal --

I am jealous ... My Intel i7-6700K at 4GHz is ~5 x slower than your AMD Ryzen 7 1800X

-- kjh

Code:

$ uname -a

Linux kjhlt6 4.4.99.kjh #1 SMP Sat Nov 18 07:55:01 CST 2017 x86_64 Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz GenuineIntel GNU/Linux

$ count-proc

8 x Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz

$ time dd if=/dev/urandom of=/dev/null bs=1MB count=1000

1000+0 records in
1000+0 records out
1000000000 bytes (1.0 GB, 954 MiB) copied, 68.1852 s, 14.7 MB/s

real    1m8.187s
user    0m0.001s
sys     1m8.180s

This is count-proc if you want it ...

Code:

#!/bin/sh

PrgNam="`basename $0`"
DirNam="`dirname  $0`"

if [ "$DirNam" = "." ]
then
   DirNam="`pwd`"
elif [ "$DirNam" = ".." ]
then
   DirNam="`pwd`"
   DirNam="`dirname $DirNam`"
elif [ "$DirNam" = "" ]
then
   DirNam="`whichone $PrgNam`"
   DirNam="`dirname $DirNam`"
fi

TabStop=0

Usage ()
{
   ErrNum=$1
   shift

   [ $# -gt 0 ] && echo -e "\n$*" >&2

   echo "usage:  $PrgNam [ -t TabOver ] " >&2
   cat  <<Usage_EOF >&2

$PrgNam - count 'model name' lines in /proc/cpuinfo and 
          write a summary to stdout

Options:

   -t <<TabStop>> - tab over <<TabStop>> spaces before printing summary.
                    default = $TabStop
 Usage_EOF
 
   exit $ErrNum

}
while  getopts ht: junk 2>/dev/null
do
   case $junk in
      t)    TabStop="$OPTARG"
            ;;
      h)    Usage 0
            ;;
      *)    Usage 1
            ;;
   esac
done

shift `expr $OPTIND - 1`

[ ! -f /proc/cpuinfo ] && echo '' && exit 1

gawk '
BEGIN {

   TabStop = "'"$TabStop"'"  +0

   Ind = ( TabStop > 0 ) ? sprintf( "%" TabStop "s", "" ) : ""

   FS = ":" ;

   LookFor = "^ *model name" ;
}
{ 

   if ( ! match( $0, LookFor ))
   {
      next ;
   }
   CPU = $2 ; 

   gsub( /  */, " ", CPU )

   if ( ! ( CPU in CPUAry ))
   { 
      CPUAry[ CPU ] = 0 ;
   } 
   CPUAry[ CPU ] ++ ; 
}
END{ 

   for ( CPU in CPUAry )
   { 
      printf( "%s%d x%s\n", Ind, CPUAry[ CPU ], CPU ) ;
   } 
   exit( 0 )

}' /proc/cpuinfo

exit $?

Nille_kungen · 11-18-2017, 04:08 PM

Strange with the AMD times posted in this threads.
I tried on my AMD laptop from 2013 with AMD A10-5750M Richland 4-core APU.

Code:

time dd if=/dev/urandom of=/dev/null bs=1MB count=1000
1000+0 poster in
1000+0 poster ut
1000000000 byte (1,0 GB, 954 MiB) kopierade, 6,08544 s, 164 MB/s

real    0m6,140s
user    0m0,005s
sys     0m6,047s

kjhambrick · 11-19-2017, 06:27 AM

All --

The common denominator for the speed of the dd on /dev/urandom seems to be the Kernel Version ?

My 4.4.99 kernel consistently gets only 14.7 MB/sec while other users with newer kernels are getting much greater rates.

I've done a little reading and this Article on lwn.net: Replacing /dev/urandom was a good jumping off point.

Some of the links were especially handy ( https://lwn.net/Kernel/Index/#Random_numbers, and https://lkml.org/lkml/2016/5/2/52 )

Anyhow, more than I wanted to know

but interesting none-the-less and reading between the lines, maybe it was the 4.7 Kernel that introduced the improvements in /dev/urandom ?

-- kjh

273 · 11-19-2017, 07:14 AM

Doesn't /dev/urandom (and /dev/ramdom for that matter) run faster depending upon how much entropy there is to call upon, ie. how mnay "random" things the system is doing rather than being particularly processor or anything else bound?

Code:

$ grep 'model name' /proc/cpuinfo 
model name	: AMD FX-8370 Eight-Core Processor
model name	: AMD FX-8370 Eight-Core Processor
model name	: AMD FX-8370 Eight-Core Processor
model name	: AMD FX-8370 Eight-Core Processor
model name	: AMD FX-8370 Eight-Core Processor
model name	: AMD FX-8370 Eight-Core Processor
model name	: AMD FX-8370 Eight-Core Processor
model name	: AMD FX-8370 Eight-Core Processor
$ time dd if=/dev/urandom of=/dev/null bs=1MB count=1000
1000+0 records in
1000+0 records out
1000000000 bytes (1.0 GB, 954 MiB) copied, 3.27047 s, 306 MB/s

real	0m3.273s
user	0m0.001s
sys	0m3.271s

If there is no data on the drive there's no need to overwrite with urandom. Even then I'd be tempted just to overwrite with zeros as, to my knowledge, nobody has been able to recover data overwritten with zeros on a modern hard drive.