[dm-crypt] AES-XTS performance

[dm-crypt] AES-XTS performance

[Igor Novgorodov]

Hello!
I've   got   a   question   regarding   encryption   performance  with
XTS mode  in  dm-crypt,  which is too low for what i
get.

Test system:
- Supermicro X8DTH-6F
- 1 x 4-Core Xeon E5620 with HyperThreading & AES-NI
- 12Gb RAM DDR3 Reg ECC

Preparation:
# mount -t tmpfs tmpfs -o size=4G /mnt/tmpfs
# dd if=/dev/zero of=/mnt/tmpfs/image
# losetup /dev/loop0 /mnt/tmpfs/image

AES-XTS-ESSIV:SHA256 + 512bit key
READ: 220 MB/s
WRITE: 79.6 MB/s

AES-XTS-ESSIV:SHA256 + 256bit key
READ: 259 MB/s
WRITE: 86.6 MB/s

And with CBC mode we get reasonable read performance (for AES-NI), but
writing is as almost slow as before:

AES-CBC-ESSIV:SHA256
READ: 495 MB/s
WRITE: 107 MB/s

AES-XTS-PLAIN64
READ: 271 MB/s
WRTITE: 90.2 MB/s

What is the problem here?
With aes-cbc-plain64 i get ~560Mb read, and another slow write ~110Mb.

Is that a limitation of a benchmark with loopback devices & tmpfs?
Write speed is very slow, as i want to saturate a very fast RAID
array (24 x 1Tb RAID6), which can deliver up to 800Mb/sec read:
# echo 3 > /proc/sys/vm/drop_caches
# dd if=/dev/sda of=/dev/null bs=1M count=20480
20480+0 records in
20480+0 records out
21474836480 bytes (21 GB) copied, 27.3376 s, 786 MB/s

Any suggestions? Why write speed is so low?
And why with XTS i get 50% speed drop, is that normal?
Thank you!

P.S.
In Windows with Trucrypt internal benchmark i get 1.6Gb/s
AES encryption speed with AES-NI even on low-end Core i5-620.

Re: [dm-crypt] AES-XTS performance

[Milan Broz]

On 11/15/2010 01:36 PM, Igor Novgorodov wrote:
> Hello!
> I've   got   a   question   regarding   encryption   performance  with
> XTS mode  in  dm-crypt,  which is too low for what i
> get.
> 
> Test system:
> - Supermicro X8DTH-6F
> - 1 x 4-Core Xeon E5620 with HyperThreading & AES-NI
> - 12Gb RAM DDR3 Reg ECC
> 
> Preparation:
> # mount -t tmpfs tmpfs -o size=4G /mnt/tmpfs
> # dd if=/dev/zero of=/mnt/tmpfs/image
> # losetup /dev/loop0 /mnt/tmpfs/image

Loop is not ideal device to test but it is not the problem.

The main problem is that dm-crypt uses only one core per device.

If you want to do some tests, try this patch
http://lkml.org/lkml/2010/11/12/344

(but there is still some issues and it will not help much
if only one process generates IOs.)

> And with CBC mode we get reasonable read performance (for AES-NI), but
> writing is as almost slow as before:

I think the write slowdown is partially loop problem here.

> What is the problem here?
> With aes-cbc-plain64 i get ~560Mb read, and another slow write ~110Mb.

Nice to benchmarking, but do not use plain/plain64 in CBC mode for data.
(It is vulnerable.)

> Any suggestions? Why write speed is so low?

Can you please try patch above? Will it help here?
(There are more possible fixes but stability is the No.1 here,
and we have still some unresolved problems with that.)

> And why with XTS i get 50% speed drop, is that normal?

In principle, XTS run 2x AES operation in comparison to CBC mode,
so I would say it is expected.

> In Windows with Trucrypt internal benchmark i get 1.6Gb/s
> AES encryption speed with AES-NI even on low-end Core i5-620.

You cannot compare internal benchmark to dm-crypt over loop.
dm-crypt uses 512b sectors and mainly block layer limits it here.

If you use device-mapper zero target as backing device you can get
better numbers but still it is comparing something different.

Milan

Re: [dm-crypt] AES-XTS performance

[Arno Wagner]

The asymmetry strongly suggests that the cipher is not the 
bottleneck in writes.

For reads, take into account that hyper-threaded cores can be 
vastly slower than real ones. I have seen this admittedly only
on an Atom CPU so far, but there I had to disable hyperthreading
to get reasonable performance. 

Arno

On Mon, Nov 15, 2010 at 03:36:04PM +0300, Igor Novgorodov wrote:
> Hello!
> I've   got   a   question   regarding   encryption   performance  with
> XTS mode  in  dm-crypt,  which is too low for what i
> get.
> 
> Test system:
> - Supermicro X8DTH-6F
> - 1 x 4-Core Xeon E5620 with HyperThreading & AES-NI
> - 12Gb RAM DDR3 Reg ECC
> 
> Preparation:
> # mount -t tmpfs tmpfs -o size=4G /mnt/tmpfs
> # dd if=/dev/zero of=/mnt/tmpfs/image
> # losetup /dev/loop0 /mnt/tmpfs/image
> 
> AES-XTS-ESSIV:SHA256 + 512bit key
> READ: 220 MB/s
> WRITE: 79.6 MB/s
> 
> AES-XTS-ESSIV:SHA256 + 256bit key
> READ: 259 MB/s
> WRITE: 86.6 MB/s
> 
> And with CBC mode we get reasonable read performance (for AES-NI), but
> writing is as almost slow as before:
> 
> AES-CBC-ESSIV:SHA256
> READ: 495 MB/s
> WRITE: 107 MB/s
> 
> AES-XTS-PLAIN64
> READ: 271 MB/s
> WRTITE: 90.2 MB/s
> 
> What is the problem here?
> With aes-cbc-plain64 i get ~560Mb read, and another slow write ~110Mb.
> 
> Is that a limitation of a benchmark with loopback devices & tmpfs?
> Write speed is very slow, as i want to saturate a very fast RAID
> array (24 x 1Tb RAID6), which can deliver up to 800Mb/sec read:
> # echo 3 > /proc/sys/vm/drop_caches
> # dd if=/dev/sda of=/dev/null bs=1M count=20480
> 20480+0 records in
> 20480+0 records out
> 21474836480 bytes (21 GB) copied, 27.3376 s, 786 MB/s
> 
> Any suggestions? Why write speed is so low?
> And why with XTS i get 50% speed drop, is that normal?
> Thank you!
> 
> P.S.
> In Windows with Trucrypt internal benchmark i get 1.6Gb/s
> AES encryption speed with AES-NI even on low-end Core i5-620.
> 
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] AES-XTS performance

[Igor Novgorodov]

> On 11/15/2010 01:36 PM, Igor Novgorodov wrote:
>> Hello!
>> I've   got   a   question   regarding   encryption   performance  with
>> XTS mode  in  dm-crypt,  which is too low for what i
>> get.
>> 
>> Test system:
>> - Supermicro X8DTH-6F
>> - 1 x 4-Core Xeon E5620 with HyperThreading & AES-NI
>> - 12Gb RAM DDR3 Reg ECC
>> 
>> Preparation:
>> # mount -t tmpfs tmpfs -o size=4G /mnt/tmpfs
>> # dd if=/dev/zero of=/mnt/tmpfs/image
>> # losetup /dev/loop0 /mnt/tmpfs/image

> Loop is not ideal device to test but it is not the problem.

> The main problem is that dm-crypt uses only one core per device.

> If you want to do some tests, try this patch
> http://lkml.org/lkml/2010/11/12/344

> (but there is still some issues and it will not help much
> if only one process generates IOs.)

Yes, i know about synchronous nature of dm-crypt,
but i thought that using AES-NI somehow alleviates
this problem, as it as i remember uses asynchronous crypto api (AEAD?),
but i may be wrong, just read somewhere.

Thanks, i'll try your patch and will report of any problems found.

>> And with CBC mode we get reasonable read performance (for AES-NI), but
>> writing is as almost slow as before:

> I think the write slowdown is partially loop problem here.
Yes, it looks so, as AES is quite symmetrical on
encryption/decryption. Tests on real filesystem shows same enc/dec
speed.

>> What is the problem here?
>> With aes-cbc-plain64 i get ~560Mb read, and another slow write ~110Mb.

> Nice to benchmarking, but do not use plain/plain64 in CBC mode for data.
> (It is vulnerable.)

Of course, i've read about watermarking attacks.
If XTS mode will remain too slow, i'll switch to CBC-ESSIV.

>> Any suggestions? Why write speed is so low?

> Can you please try patch above? Will it help here?
> (There are more possible fixes but stability is the No.1 here,
> and we have still some unresolved problems with that.)

>> And why with XTS i get 50% speed drop, is that normal?

> In principle, XTS run 2x AES operation in comparison to CBC mode,
> so I would say it is expected.

Hmm. It looks correct.

>> In Windows with Trucrypt internal benchmark i get 1.6Gb/s
>> AES encryption speed with AES-NI even on low-end Core i5-620.

> You cannot compare internal benchmark to dm-crypt over loop.
> dm-crypt uses 512b sectors and mainly block layer limits it here.
Accoding to http://www.truecrypt.org/docs/?s=modes-of-operation
truecrypt uses 512b unit size too, but, of course, raw in-memory
encryption is not compared to multiple-layer model of linux, but i
thought that difference of hunderds of percents is too big.
But it may be due to a multithreaded nature of TC.

> If you use device-mapper zero target as backing device you can get
> better numbers but still it is comparing something different.

I'll try that, thanks, i didn't even thought of this target before :)

> Milan

Re: [dm-crypt] AES-XTS performance

[Igor Novgorodov]

> Loop is not ideal device to test but it is not the problem.

> The main problem is that dm-crypt uses only one core per device.

> If you want to do some tests, try this patch
> http://lkml.org/lkml/2010/11/12/344

> (but there is still some issues and it will not help much
> if only one process generates IOs.)

I've tried to apply this patch against 2.6.36 and 2.6.37-rc2
but without success:

# patch -p1 -i ./dm-crypt-patch.diff
patching file drivers/md/dm-crypt.c
Hunk #2 FAILED at 78.
Hunk #3 succeeded at 91 with fuzz 2.
Hunk #4 FAILED at 133.
Hunk #5 FAILED at 152.
Hunk #7 FAILED at 227.
Hunk #8 FAILED at 239.
Hunk #9 FAILED at 259.
Hunk #10 FAILED at 336.
Hunk #11 FAILED at 358.
Hunk #12 FAILED at 381.
Hunk #13 FAILED at 501.
Hunk #14 FAILED at 546.
Hunk #15 FAILED at 562.
Hunk #16 succeeded at 574 with fuzz 2.
Hunk #17 FAILED at 583.
Hunk #19 succeeded at 1065 with fuzz 1.
Hunk #20 FAILED at 1096.
Hunk #21 FAILED at 1136.
Hunk #22 FAILED at 1155.
Hunk #23 FAILED at 1174.
Hunk #24 FAILED at 1209.
Hunk #25 FAILED at 1240.
Hunk #26 FAILED at 1259.
Hunk #27 FAILED at 1323.
Hunk #28 FAILED at 1354.
Hunk #29 FAILED at 1365.
Hunk #30 FAILED at 1378.
Hunk #31 FAILED at 1405.
25 out of 31 hunks FAILED -- saving rejects to file drivers/md/dm-crypt.c.rej

What version of kernel should i patch? GIT?

Re: [dm-crypt] AES-XTS performance

[Milan Broz]

On 11/16/2010 08:31 AM, Igor Novgorodov wrote:

> I've tried to apply this patch against 2.6.36 and 2.6.37-rc2
> but without success:
> 
> # patch -p1 -i ./dm-crypt-patch.diff

It applies cleanly to 2.6.37-rc1 and above.

For 2.6.36 small change is needed.

You can download both versions here for now:
http://mbroz.fedorapeople.org/dm-crypt/

Milan

Re: [dm-crypt] AES-XTS performance

[Igor Novgorodov]

Здравствуйте, Milan.

Вы писали 16 ноября 2010 г., 12:25:22:

> On 11/16/2010 08:31 AM, Igor Novgorodov wrote:

>> I've tried to apply this patch against 2.6.36 and 2.6.37-rc2
>> but without success:
>> 
>> # patch -p1 -i ./dm-crypt-patch.diff

> It applies cleanly to 2.6.37-rc1 and above.
It does not. I've tried 2.6.37-rc2 as i mentioned.
Copied & pasted the patch from your LKML post.

> For 2.6.36 small change is needed.

> You can download both versions here for now:
> http://mbroz.fedorapeople.org/dm-crypt/
Thanks, this patch for 2.6.36 applied fine.

Setup:
# echo '0 1000000000 zero' | dmsetup create zero1
# cryptsetup --cipher aes-xts-essiv:sha256 --key-size 512 --hash sha512 create c /dev/mapper/zero1

Benchmark with pure 2.6.36:
# dd if=/dev/mapper/c of=/dev/null bs=1M count=30000
30000+0 records in
30000+0 records out
31457280000 bytes (31 GB) copied, 128.674 s, 244 MB/s
# dd of=/dev/mapper/c if=/dev/zero bs=1M count=30000
30000+0 records in
30000+0 records out
31457280000 bytes (31 GB) copied, 129.512 s, 243 MB/s

Benchmark with your patch:
Read speed increased ~20%
# dd if=/dev/mapper/c of=/dev/null bs=1M count=30000
30000+0 records in
30000+0 records out
31457280000 bytes (31 GB) copied, 108.363 s, 290 MB/s

And according to `top` all kworker threads were runnning,
but they were using only a portion CPUs time:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2691 root      20   0     0    0    0 S   36  0.0   0:37.11 kworker/1:2
 2784 root      20   0     0    0    0 S   26  0.0   0:04.59 kworker/5:1
 2746 root      20   0     0    0    0 S   19  0.0   0:41.36 kworker/2:0
 2780 root      20   0  9304 1400  516 D   19  0.0   0:07.91 dd
 2781 root      20   0     0    0    0 S   17  0.0   0:07.36 kworker/6:1
 2039 root      20   0     0    0    0 R   17  0.0   1:01.45 kworker/4:2
 2350 root      20   0     0    0    0 S   15  0.0   0:43.15 kworker/0:2
 2783 root      20   0     0    0    0 S    2  0.0   0:00.38 kworker/1:0
 2750 root      20   0     0    0    0 S    2  0.0   0:22.71 kworker/5:0
 2686 root      20   0     0    0    0 S    1  0.0   0:43.79 kworker/2:2
 2695 root      20   0     0    0    0 S    1  0.0   0:42.46 kworker/6:2
 2782 root      20   0     0    0    0 S    1  0.0   0:02.38 kworker/4:0
 2752 root      20   0     0    0    0 S    1  0.0   0:20.96 kworker/0:1

But with the write speed we get a VERY HUGE speed improvement!:
# dd of=/dev/mapper/c if=/dev/zero bs=1M count=30000
30000+0 records in
30000+0 records out
31457280000 bytes (31 GB) copied, 37.2707 s, 844 MB/s

This time the threads are loaded much more:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2690 root      20   0     0    0    0 R  101  0.0   0:28.17 kworker/7:2
 2750 root      20   0     0    0    0 R  101  0.0   0:43.14 kworker/5:0
 2752 root      20   0     0    0    0 R   99  0.0   0:43.01 kworker/0:1
 2695 root      20   0     0    0    0 R   91  0.0   1:05.82 kworker/6:2
 2791 root      20   0  9304 1404  516 R   46  0.0   0:17.12 dd
 2748 root      20   0     0    0    0 S   34  0.0   0:19.76 kworker/3:0
 2746 root      20   0     0    0    0 R   28  0.0   1:08.36 kworker/2:0
 2039 root      20   0     0    0    0 S   10  0.0   1:33.78 kworker/4:2
 2783 root      20   0     0    0    0 S   10  0.0   0:28.28 kworker/1:0
 2686 root      20   0     0    0    0 R    2  0.0   0:47.27 kworker/2:2

Such big difference between read and write speed is by design?
Or it is a problem of my setup? Maybe i should try to disable
hyperthreading and check again?

Anyway, it's a huge step forward from vanilla kernel's performance,
thanks! I'll keep testing this, and if any new patches will come out,
i'll gladly try them out too.

Re: [dm-crypt] AES-XTS performance

[Milan Broz]

On 11/16/2010 12:32 PM, Igor Novgorodov wrote:
> Such big difference between read and write speed is by design?
> Or it is a problem of my setup? Maybe i should try to disable
> hyperthreading and check again?

That should be probably improved later.
(btw I just fixed stacking there, it was written by Andi Kleen).

Anyway, now I need to check that there is no known data corruption
with this approach and then we can add another patches on top of it.
(Targeted for 2.6.38, it missed 2.6.37 merge window already.)

(If anyone see some reproducible data corruption when using this patch,
please let me know.)

Milan

Re: [dm-crypt] AES-XTS performance

[Igor Novgorodov]

> On 11/16/2010 12:32 PM, Igor Novgorodov wrote:
>> Such big difference between read and write speed is by design?
>> Or it is a problem of my setup? Maybe i should try to disable
>> hyperthreading and check again?

> That should be probably improved later.
> (btw I just fixed stacking there, it was written by Andi Kleen).
Okay, will be waiting for improved versions, thanks.

> Anyway, now I need to check that there is no known data corruption
> with this approach and then we can add another patches on top of it.
> (Targeted for 2.6.38, it missed 2.6.37 merge window already.)

> (If anyone see some reproducible data corruption when using this patch,
> please let me know.)

I've just been testing it on a loopback device with a portion of our
maildir, about 50G, 70k files. Copying in/out and comparing MD5 sums
showed no evidence of data corruption so far.

Re: [dm-crypt] AES-XTS performance

[Jakob-Tobias Winter]

[-- Attachment #1.1: Type: text/plain, Size: 1274 bytes --]

Hey Milan,

thank you for spending time on the long awaited multi-core-crypto-support!

On 11/15/2010 03:25 PM, Milan Broz wrote:
> If you want to do some tests, try this patch
> http://lkml.org/lkml/2010/11/12/344

I started out giving the patch a try on my desktop box but quickly ran
into disk IO limitations of just one HDD and so decided to grab some
spare metal and do this on a more performant system.

The test results are included as text-file as I fear the syntax of the
bonnie output will suffer if included directly in the mail, so please
exuse using an attachement. I hope most MUAs will end up displaying it
directly.

> (but there is still some issues and it will not help much
> if only one process generates IOs.)
Are you sure about that? Actually I also observed quite some performance
gain for single process IO.

In case you would like me to do some other comparisons, feel free to
point me in the right direction.

So far I also did not take time to check if it ends up eating the data,
but if it fails, I guess my desktop will let me know. Hooray for the
backup and black-magic-kernel-patch-users. ;)

By the way: As it seems, compiling the patched kernel breaks, if trying
to compile the AES stuff as modules.

Tobias

[-- Attachment #1.2: results.txt --]
[-- Type: text/plain, Size: 6060 bytes --]

Dell PowerEdge 1950 Raid10 Hardware Raid 4x 72Gb 15k SAS 8G RAM model Intel Xeon CPU E5405 @ 2.00GHz (Quadcore w/o hyperthreading)

Linux testkiste 2.6.36-wintix-unpatched-crypto #1 SMP Wed Nov 17 17:08:56 CET 2010 x86_64 GNU/Linux

testkiste:/mnt#  bonnie++ -d ./ -c 4 -u root
Using uid:0, gid:0.
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   4     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
testkiste       16G   820  99 193744  20 95649  13  2449  95 191911  11  1049  10
Latency              9951us    3592ms    1178ms   45264us   24906us   53383us
Version  1.96       ------Sequential Create------ --------Random Create--------
testkiste           -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 25935  93 +++++ +++ 25837  94 25378  91 +++++ +++ 21993  81
Latency              8491us     128us     244us    1013us      12us    7994us
1.96,1.96,testkiste,4,1290012956,16G,,820,99,193744,20,95649,13,2449,95,191911,11,1049,10,16,,,,,25935,93,+++++,+++,25837,94,25378,91,+++++,+++,21993,81,9951us,3592ms,1178ms,45264us,24906us,53383us,8491us,128us,244us,1013us,12us,7994us
testkiste:/mnt# 

testkiste:/mnt_crypt#  bonnie++ -d ./ -c 4 -u root
Using uid:0, gid:0.
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   4     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
testkiste       16G   837  98 67741   6 36038   4  2571  98 101504   5 852.2   8
Latency              9562us   14773ms   11487ms   14082us     511ms   57357us
Version  1.96       ------Sequential Create------ --------Random Create--------
testkiste           -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  4570  17 +++++ +++  4281  15  4666  17 +++++ +++  3890  15
Latency             13172us     133us    8949us   12061us      38us   41450us
1.96,1.96,testkiste,4,1290018063,16G,,837,98,67741,6,36038,4,2571,98,101504,5,852.2,8,16,,,,,4570,17,+++++,+++,4281,15,4666,17,+++++,+++,3890,15,9562us,14773ms,11487ms,14082us,511ms,57357us,13172us,133us,8949us,12061us,38us,41450us
testkiste:/mnt_crypt# 

Linux testkiste 2.6.36-wintix-patched-crypto #2 SMP Wed Nov 17 23:21:41 CET 2010 x86_64 GNU/Linux

testkiste:/mnt_crypt#  bonnie++ -d ./ -c 4 -u root
Using uid:0, gid:0.
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   4     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
testkiste       16G   809  98 136538  14 49701   5  1958  76 107338   4  1040  10
Latency              9889us    6065ms    5466ms     363ms     530ms   47273us
Version  1.96       ------Sequential Create------ --------Random Create--------
testkiste           -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  5279  19 +++++ +++  4669  16  5234  19 +++++ +++  3899  14
Latency             13465us     136us   15558us   12742us      91us   30820us
1.96,1.96,testkiste,4,1290031937,16G,,809,98,136538,14,49701,5,1958,76,107338,4,1040,10,16,,,,,5279,19,+++++,+++,4669,16,5234,19,+++++,+++,3899,14,9889us,6065ms,5466ms,363ms,530ms,47273us,13465us,136us,15558us,12742us,91us,30820us
testkiste:/mnt_crypt# 

testkiste:/mnt_crypt# cryptsetup luksDump /dev/mapper/vg00-test 
LUKS header information for /dev/mapper/vg00-test

Version:        1
Cipher name:    aes
Cipher mode:    cbc-essiv:sha256
Hash spec:      sha1
Payload offset: 2056
MK bits:        256
MK digest:      94 20 85 7b fe a6 5e b3 e5 4c dc 11 36 e0 94 2c 63 2f e2 80 
MK salt:        88 86 48 63 3c 6b 46 b7 dc 67 2f 89 02 df 78 cf 
                7e 50 f7 90 ef dd 8f 55 a6 7a 2b ea 0c d9 63 c4 
MK iterations:  35625
UUID:           9b4e45b1-602b-4c1f-be36-10d0a98bbea5

Key Slot 0: ENABLED
        Iterations:             142950
        Salt:                   ba b6 9b f4 34 e4 4e 30 37 27 6b 7c 2a c2 de 17 
                                f0 90 36 ae ca ab ec c3 7a 2b b5 de ff ed b2 15 
        Key material offset:    8
        AF stripes:             4000
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED
testkiste:/mnt_crypt# 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]