LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 12-14-2014, 11:22 AM   #1
replica9000
Senior Member
 
Registered: Jul 2006
Distribution: Debian Unstable
Posts: 1,068
Blog Entries: 2

Rep: Reputation: 239Reputation: 239Reputation: 239
cryptsetup low write speeds.


CPU: i7-2600K @ 4300MHz - supports aesni.
RAM: 32GB
Kernel: 3.16-3-amd64
HDDs: WD Passport Ultra 2TB USB 3.0 x 3
LVM2: 2.0.2.111 -- pvcreate -M2 --dataalignment 512K /dev/sd[ijk]1
--------------- vgcreate wdultra /dev/sd[ijk]1
--------------- lvcreate -n wdbackup -i 3 -I 512 -l +1430769 wdultra

cryptsetup: 1.6.6 -- cryptsetup -v luksFormat /dev/wdultra/wdbackup
Filesystem: btrfs 3.17 -- mount -t btrfs -o noatime /dev/mapper/<device> /mnt/backup

I've created a luks container on top of LVM2, which is striped across the 3 HDDs. According to the benchmark, I can encrypt data at over 2000 MiB/s. The benchmark only uses a single core.

Code:
# cryptsetup benchmark            
# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1      1242388 iterations per second
PBKDF2-sha256     732245 iterations per second
PBKDF2-sha512     595781 iterations per second
PBKDF2-ripemd160  708497 iterations per second
PBKDF2-whirlpool  267493 iterations per second
#  Algorithm | Key |  Encryption |  Decryption
     aes-cbc   128b   738.2 MiB/s  2525.3 MiB/s
 serpent-cbc   128b    93.6 MiB/s   364.0 MiB/s
 twofish-cbc   128b   209.1 MiB/s   394.1 MiB/s
     aes-cbc   256b   544.2 MiB/s  1929.9 MiB/s
 serpent-cbc   256b    93.6 MiB/s   364.8 MiB/s
 twofish-cbc   256b   210.6 MiB/s   395.9 MiB/s
     aes-xts   256b  2198.6 MiB/s  2199.0 MiB/s
 serpent-xts   256b   378.0 MiB/s   358.6 MiB/s
 twofish-xts   256b   390.9 MiB/s   389.3 MiB/s
     aes-xts   512b  1704.8 MiB/s  1713.5 MiB/s
 serpent-xts   512b   378.2 MiB/s   359.1 MiB/s
 twofish-xts   512b   389.2 MiB/s   390.1 MiB/s
hdparm shows the throughput of the 3 drives together can easily hit at least 300 MiB/s.

Code:
# hdparm -tT /dev/wdultra/wdbackup

/dev/wdultra/wdbackup:
 Timing cached reads:   24850 MB in  2.00 seconds = 12435.30 MB/sec
 Timing buffered disk reads: 896 MB in  3.00 seconds = 298.34 MB/sec
When I run rsync to backup to the 3 HDDs without cryptsetup, I can regularly write data at 320 MiB/s. When I use cryptsetup using the options I mentioned above, that throughput seems to be cut in half, and all 8 cores seems to be maxed with I/O traffic.

I would think the CPU was waiting for the slower I/O of the HDDs, instead of the HDDs waiting for the CPU to process the data.
 
Old 12-14-2014, 01:55 PM   #2
metaschima
Senior Member
 
Registered: Dec 2013
Distribution: Slackware
Posts: 1,982

Rep: Reputation: 492Reputation: 492Reputation: 492Reputation: 492Reputation: 492
What encryption mode are you using ? Note that CBC encrypts much slower than it decrypts because it can't be threaded.
 
Old 12-14-2014, 02:40 PM   #3
replica9000
Senior Member
 
Registered: Jul 2006
Distribution: Debian Unstable
Posts: 1,068

Original Poster
Blog Entries: 2

Rep: Reputation: 239Reputation: 239Reputation: 239
Quote:
Originally Posted by metaschima View Post
What encryption mode are you using ? Note that CBC encrypts much slower than it decrypts because it can't be threaded.
I believe the defaults for luks is --cipher aes-xts-plain64 --key-size 256 --hash sha1, which is also the fastest encryption/decryption according the cryptsetup's benchmark.
 
Old 12-14-2014, 03:34 PM   #4
replica9000
Senior Member
 
Registered: Jul 2006
Distribution: Debian Unstable
Posts: 1,068

Original Poster
Blog Entries: 2

Rep: Reputation: 239Reputation: 239Reputation: 239
Appears the problem was the filesystem. I decided to format with ext4 instead (mkfs.ext4 m 0 -b 4096 -E stripe-width=384,stride=128 /dev/mapper/<device>)

With ext4, I'm getting average write speeds of 270 MiB/s, and peaks around 330 MiB/s. And the CPU seems to be doing less than a 1/4th of the work.

Maybe I didn't optimize btrfs properly for a striped LVM setup (I didn't see any options in the manpages), or btrfs and cryptsetup just isn't a good combo at this time.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Filesharing between Windows 7 x64 and Debian 6 x86 - very low speeds Tralce Linux - Networking 2 04-28-2013 11:09 PM
can't force low speeds when burning cd's bennethos Linux - Software 1 09-15-2010 04:15 AM
NSLU2, Samba and Low Transfer speeds purelithium Linux - Networking 0 01-23-2006 11:43 PM
Low SCSI disk speeds Brainer Linux - Hardware 3 01-16-2006 09:01 PM
upgrading from 10mbps HD to 100mbps FD, transfer speeds at an all time low Sir Dwain Linux - Networking 7 10-06-2004 11:01 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 03:22 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration