linux-crypto.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* xts fuzz testing and lack of ciphertext stealing support
@ 2019-07-16 17:46 Horia Geanta
  2019-07-16 22:16 ` Eric Biggers
  0 siblings, 1 reply; 65+ messages in thread
From: Horia Geanta @ 2019-07-16 17:46 UTC (permalink / raw)
  To: Herbert Xu, Eric Biggers; +Cc: linux-crypto

Hi,

With fuzz testing enabled, I am seeing xts(aes) failures on caam drivers.

Below are several failures, extracted from different runs:

[    3.921654] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=40 klen=64"; expected_error=-22, cfg="random: inplace use_finup nosimd src_divs=[57.93%@+11, 37.18%@+164, <reimport>0.68%@+4, 0.50%@+305, 3.71%@alignmask+3975]" 

[    3.726698] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=369 klen=64"; expected_error=-22, cfg="random: inplace may_sleep use_digest src_divs=[100.0%@alignmask+584]" 

[    3.741082] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=2801 klen=64"; expected_error=-22, cfg="random: inplace may_sleep use_digest src_divs=[100.0%@+6] iv_offset=18"

It looks like the problem is not in CAAM driver.
More exactly, fuzz testing is generating random test vectors and running
them through both SW generic (crypto/xts.c) and CAAM implementation:
-SW generic implementation of xts(aes) does not support ciphertext stealing
and throws -EINVAL when input is not a multiple of AES block size (16B)
-caam has support for ciphertext stealing, and allows for any input size
which results in "unexpectedly succeeded" error messages.

Any suggestion how this should be fixed?

Thanks,
Horia

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-16 17:46 xts fuzz testing and lack of ciphertext stealing support Horia Geanta
@ 2019-07-16 22:16 ` Eric Biggers
  2019-07-17 17:09   ` Horia Geanta
  0 siblings, 1 reply; 65+ messages in thread
From: Eric Biggers @ 2019-07-16 22:16 UTC (permalink / raw)
  To: Horia Geanta; +Cc: Herbert Xu, linux-crypto

Hi Horia,

On Tue, Jul 16, 2019 at 05:46:29PM +0000, Horia Geanta wrote:
> Hi,
> 
> With fuzz testing enabled, I am seeing xts(aes) failures on caam drivers.
> 
> Below are several failures, extracted from different runs:
> 
> [    3.921654] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=40 klen=64"; expected_error=-22, cfg="random: inplace use_finup nosimd src_divs=[57.93%@+11, 37.18%@+164, <reimport>0.68%@+4, 0.50%@+305, 3.71%@alignmask+3975]" 
> 
> [    3.726698] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=369 klen=64"; expected_error=-22, cfg="random: inplace may_sleep use_digest src_divs=[100.0%@alignmask+584]" 
> 
> [    3.741082] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=2801 klen=64"; expected_error=-22, cfg="random: inplace may_sleep use_digest src_divs=[100.0%@+6] iv_offset=18"
> 
> It looks like the problem is not in CAAM driver.
> More exactly, fuzz testing is generating random test vectors and running
> them through both SW generic (crypto/xts.c) and CAAM implementation:
> -SW generic implementation of xts(aes) does not support ciphertext stealing
> and throws -EINVAL when input is not a multiple of AES block size (16B)
> -caam has support for ciphertext stealing, and allows for any input size
> which results in "unexpectedly succeeded" error messages.
> 
> Any suggestion how this should be fixed?
> 
> Thanks,
> Horia

I don't think drivers should allow inputs the generic implementation doesn't,
since those inputs aren't tested by the crypto self-tests (so how do you know
it's even correct?), and people could accidentally rely on the driver-specific
behavior and then be unable to migrate to another platform or implementation.

So for now I recommend just updating the caam driver to return -EINVAL on XTS
inputs not evenly divisible by the block size.

Of course, if there are actual use cases for XTS with ciphertext stealing in the
kernel, we could add it to all the other implementations too.  But I'm not aware
of any currently.  Don't all XTS users in the kernel pass whole blocks?

- Eric

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-16 22:16 ` Eric Biggers
@ 2019-07-17 17:09   ` Horia Geanta
  2019-07-17 17:28     ` Eric Biggers
  0 siblings, 1 reply; 65+ messages in thread
From: Horia Geanta @ 2019-07-17 17:09 UTC (permalink / raw)
  To: Eric Biggers; +Cc: Herbert Xu, linux-crypto, dm-devel

On 7/17/2019 1:16 AM, Eric Biggers wrote:
> Hi Horia,
> 
> On Tue, Jul 16, 2019 at 05:46:29PM +0000, Horia Geanta wrote:
>> Hi,
>>
>> With fuzz testing enabled, I am seeing xts(aes) failures on caam drivers.
>>
>> Below are several failures, extracted from different runs:
>>
>> [    3.921654] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=40 klen=64"; expected_error=-22, cfg="random: inplace use_finup nosimd src_divs=[57.93%@+11, 37.18%@+164, <reimport>0.68%@+4, 0.50%@+305, 3.71%@alignmask+3975]" 
>>
>> [    3.726698] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=369 klen=64"; expected_error=-22, cfg="random: inplace may_sleep use_digest src_divs=[100.0%@alignmask+584]" 
>>
>> [    3.741082] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=2801 klen=64"; expected_error=-22, cfg="random: inplace may_sleep use_digest src_divs=[100.0%@+6] iv_offset=18"
>>
>> It looks like the problem is not in CAAM driver.
>> More exactly, fuzz testing is generating random test vectors and running
>> them through both SW generic (crypto/xts.c) and CAAM implementation:
>> -SW generic implementation of xts(aes) does not support ciphertext stealing
>> and throws -EINVAL when input is not a multiple of AES block size (16B)
>> -caam has support for ciphertext stealing, and allows for any input size
>> which results in "unexpectedly succeeded" error messages.
>>
>> Any suggestion how this should be fixed?
>>
>> Thanks,
>> Horia
> 
> I don't think drivers should allow inputs the generic implementation doesn't,
> since those inputs aren't tested by the crypto self-tests (so how do you know
How about implementation adding static test vectors and using them to validate
the standard feature set that's not supported by the generic implementation?

> it's even correct?), and people could accidentally rely on the driver-specific
> behavior and then be unable to migrate to another platform or implementation.
> 
People could also *intentionally* rely on a driver offering an implementation
that is closer to the spec / standard.

> So for now I recommend just updating the caam driver to return -EINVAL on XTS
> inputs not evenly divisible by the block size.
> 
I was hoping for something more constructive...

> Of course, if there are actual use cases for XTS with ciphertext stealing in the
> kernel, we could add it to all the other implementations too.  But I'm not aware
> of any currently.  Don't all XTS users in the kernel pass whole blocks?
> 
That's my guess too.

What about user space relying on offloading and xts working
according to the spec?

Thanks,
Horia

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-17 17:09   ` Horia Geanta
@ 2019-07-17 17:28     ` Eric Biggers
  2019-07-17 18:08       ` Ard Biesheuvel
  0 siblings, 1 reply; 65+ messages in thread
From: Eric Biggers @ 2019-07-17 17:28 UTC (permalink / raw)
  To: Horia Geanta; +Cc: Herbert Xu, linux-crypto, dm-devel

On Wed, Jul 17, 2019 at 05:09:31PM +0000, Horia Geanta wrote:
> On 7/17/2019 1:16 AM, Eric Biggers wrote:
> > Hi Horia,
> > 
> > On Tue, Jul 16, 2019 at 05:46:29PM +0000, Horia Geanta wrote:
> >> Hi,
> >>
> >> With fuzz testing enabled, I am seeing xts(aes) failures on caam drivers.
> >>
> >> Below are several failures, extracted from different runs:
> >>
> >> [    3.921654] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=40 klen=64"; expected_error=-22, cfg="random: inplace use_finup nosimd src_divs=[57.93%@+11, 37.18%@+164, <reimport>0.68%@+4, 0.50%@+305, 3.71%@alignmask+3975]" 
> >>
> >> [    3.726698] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=369 klen=64"; expected_error=-22, cfg="random: inplace may_sleep use_digest src_divs=[100.0%@alignmask+584]" 
> >>
> >> [    3.741082] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=2801 klen=64"; expected_error=-22, cfg="random: inplace may_sleep use_digest src_divs=[100.0%@+6] iv_offset=18"
> >>
> >> It looks like the problem is not in CAAM driver.
> >> More exactly, fuzz testing is generating random test vectors and running
> >> them through both SW generic (crypto/xts.c) and CAAM implementation:
> >> -SW generic implementation of xts(aes) does not support ciphertext stealing
> >> and throws -EINVAL when input is not a multiple of AES block size (16B)
> >> -caam has support for ciphertext stealing, and allows for any input size
> >> which results in "unexpectedly succeeded" error messages.
> >>
> >> Any suggestion how this should be fixed?
> >>
> >> Thanks,
> >> Horia
> > 
> > I don't think drivers should allow inputs the generic implementation doesn't,
> > since those inputs aren't tested by the crypto self-tests (so how do you know
> How about implementation adding static test vectors and using them to validate
> the standard feature set that's not supported by the generic implementation?
> 
> > it's even correct?), and people could accidentally rely on the driver-specific
> > behavior and then be unable to migrate to another platform or implementation.
> > 
> People could also *intentionally* rely on a driver offering an implementation
> that is closer to the spec / standard.
> 
> > So for now I recommend just updating the caam driver to return -EINVAL on XTS
> > inputs not evenly divisible by the block size.
> > 
> I was hoping for something more constructive...
> 
> > Of course, if there are actual use cases for XTS with ciphertext stealing in the
> > kernel, we could add it to all the other implementations too.  But I'm not aware
> > of any currently.  Don't all XTS users in the kernel pass whole blocks?
> > 
> That's my guess too.
> 
> What about user space relying on offloading and xts working
> according to the spec?
> 

Sure, AF_ALG users could expect ciphertext stealing to work.  I don't know of
any actual examples of people saying they want it, but it's possible.

My point is simply that we add this, we need to find a way to support it in all
implementations.  It's not helpful to add it to only one specific driver, as
then it's inconsistent and is untestable with the common tests.

- Eric

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-17 17:28     ` Eric Biggers
@ 2019-07-17 18:08       ` Ard Biesheuvel
  2019-07-18  6:52         ` Herbert Xu
  0 siblings, 1 reply; 65+ messages in thread
From: Ard Biesheuvel @ 2019-07-17 18:08 UTC (permalink / raw)
  To: Horia Geanta, Herbert Xu, linux-crypto, dm-devel

On Wed, 17 Jul 2019 at 19:28, Eric Biggers <ebiggers@kernel.org> wrote:
>
> On Wed, Jul 17, 2019 at 05:09:31PM +0000, Horia Geanta wrote:
> > On 7/17/2019 1:16 AM, Eric Biggers wrote:
> > > Hi Horia,
> > >
> > > On Tue, Jul 16, 2019 at 05:46:29PM +0000, Horia Geanta wrote:
> > >> Hi,
> > >>
> > >> With fuzz testing enabled, I am seeing xts(aes) failures on caam drivers.
> > >>
> > >> Below are several failures, extracted from different runs:
> > >>
> > >> [    3.921654] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=40 klen=64"; expected_error=-22, cfg="random: inplace use_finup nosimd src_divs=[57.93%@+11, 37.18%@+164, <reimport>0.68%@+4, 0.50%@+305, 3.71%@alignmask+3975]"
> > >>
> > >> [    3.726698] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=369 klen=64"; expected_error=-22, cfg="random: inplace may_sleep use_digest src_divs=[100.0%@alignmask+584]"
> > >>
> > >> [    3.741082] alg: skcipher: xts-aes-caam encryption unexpectedly succeeded on test vector "random: len=2801 klen=64"; expected_error=-22, cfg="random: inplace may_sleep use_digest src_divs=[100.0%@+6] iv_offset=18"
> > >>
> > >> It looks like the problem is not in CAAM driver.
> > >> More exactly, fuzz testing is generating random test vectors and running
> > >> them through both SW generic (crypto/xts.c) and CAAM implementation:
> > >> -SW generic implementation of xts(aes) does not support ciphertext stealing
> > >> and throws -EINVAL when input is not a multiple of AES block size (16B)
> > >> -caam has support for ciphertext stealing, and allows for any input size
> > >> which results in "unexpectedly succeeded" error messages.
> > >>
> > >> Any suggestion how this should be fixed?
> > >>
> > >> Thanks,
> > >> Horia
> > >
> > > I don't think drivers should allow inputs the generic implementation doesn't,
> > > since those inputs aren't tested by the crypto self-tests (so how do you know
> > How about implementation adding static test vectors and using them to validate
> > the standard feature set that's not supported by the generic implementation?
> >
> > > it's even correct?), and people could accidentally rely on the driver-specific
> > > behavior and then be unable to migrate to another platform or implementation.
> > >
> > People could also *intentionally* rely on a driver offering an implementation
> > that is closer to the spec / standard.
> >
> > > So for now I recommend just updating the caam driver to return -EINVAL on XTS
> > > inputs not evenly divisible by the block size.
> > >
> > I was hoping for something more constructive...
> >
> > > Of course, if there are actual use cases for XTS with ciphertext stealing in the
> > > kernel, we could add it to all the other implementations too.  But I'm not aware
> > > of any currently.  Don't all XTS users in the kernel pass whole blocks?
> > >
> > That's my guess too.
> >
> > What about user space relying on offloading and xts working
> > according to the spec?
> >
>
> Sure, AF_ALG users could expect ciphertext stealing to work.  I don't know of
> any actual examples of people saying they want it, but it's possible.
>
> My point is simply that we add this, we need to find a way to support it in all
> implementations.  It's not helpful to add it to only one specific driver, as
> then it's inconsistent and is untestable with the common tests.
>

IIRC there are different ways to implement CTS, and the cts template
we have in the kernel (which is used for CBC in some cases) implements
the flavor where the last two blocks are reordered if the input size
is an exact multiple of the block size. If your CTS implementation
does not implement this reordering (which seems to be the case since
it is compatible with plain XTS), it implements CTS in an incompatible
way.

Since the kernel does not support CTS for XTS any way, and since no
AF_ALG users can portably rely on this, I agree with Eric that the
only sensible way to address this is to disable this functionality in
the driver.

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-17 18:08       ` Ard Biesheuvel
@ 2019-07-18  6:52         ` Herbert Xu
  2019-07-18  7:15           ` Ard Biesheuvel
  0 siblings, 1 reply; 65+ messages in thread
From: Herbert Xu @ 2019-07-18  6:52 UTC (permalink / raw)
  To: Ard Biesheuvel; +Cc: Horia Geanta, linux-crypto, dm-devel

On Wed, Jul 17, 2019 at 08:08:27PM +0200, Ard Biesheuvel wrote:
>
> Since the kernel does not support CTS for XTS any way, and since no
> AF_ALG users can portably rely on this, I agree with Eric that the
> only sensible way to address this is to disable this functionality in
> the driver.

But the whole point of XTS is that it supports sizes that are
not multiples of the block size.  So implementing it without
supporting ciphertext stealing is just wrong.

So let's fix the generic implementation rather than breaking
the caam driver.

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18  6:52         ` Herbert Xu
@ 2019-07-18  7:15           ` Ard Biesheuvel
  2019-07-18  7:21             ` Herbert Xu
  0 siblings, 1 reply; 65+ messages in thread
From: Ard Biesheuvel @ 2019-07-18  7:15 UTC (permalink / raw)
  To: Herbert Xu; +Cc: Horia Geanta, linux-crypto, dm-devel

On Thu, 18 Jul 2019 at 08:52, Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> On Wed, Jul 17, 2019 at 08:08:27PM +0200, Ard Biesheuvel wrote:
> >
> > Since the kernel does not support CTS for XTS any way, and since no
> > AF_ALG users can portably rely on this, I agree with Eric that the
> > only sensible way to address this is to disable this functionality in
> > the driver.
>
> But the whole point of XTS is that it supports sizes that are
> not multiples of the block size.  So implementing it without
> supporting ciphertext stealing is just wrong.
>
> So let's fix the generic implementation rather than breaking
> the caam driver.
>

Not just the generic implementation: there are numerous synchronous
and asynchronous implementations of xts(aes) in the kernel that would
have to be fixed, while there are no in-kernel users that actually
rely on CTS. Also, in the cbc case, we support CTS by wrapping it into
another template, i.e., cts(cbc(aes)).

So retroactively redefining what xts(...) means seems like a bad idea
to me. If we want to support XTS ciphertext stealing for the benefit
of userland, let's do so via the existing cts template, and add
support for wrapping XTS to it.

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18  7:15           ` Ard Biesheuvel
@ 2019-07-18  7:21             ` Herbert Xu
  2019-07-18  7:28               ` Ard Biesheuvel
  2019-07-18  7:40               ` Milan Broz
  0 siblings, 2 replies; 65+ messages in thread
From: Herbert Xu @ 2019-07-18  7:21 UTC (permalink / raw)
  To: Ard Biesheuvel; +Cc: Horia Geanta, linux-crypto, dm-devel

On Thu, Jul 18, 2019 at 09:15:39AM +0200, Ard Biesheuvel wrote:
>
> Not just the generic implementation: there are numerous synchronous
> and asynchronous implementations of xts(aes) in the kernel that would
> have to be fixed, while there are no in-kernel users that actually
> rely on CTS. Also, in the cbc case, we support CTS by wrapping it into
> another template, i.e., cts(cbc(aes)).
> 
> So retroactively redefining what xts(...) means seems like a bad idea
> to me. If we want to support XTS ciphertext stealing for the benefit
> of userland, let's do so via the existing cts template, and add
> support for wrapping XTS to it.

XTS without stealing should be renamed as XEX.  Sure you can then
wrap it inside cts to form xts but the end result needs to be called
xts.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18  7:21             ` Herbert Xu
@ 2019-07-18  7:28               ` Ard Biesheuvel
  2019-07-18  7:50                 ` Herbert Xu
  2019-07-18  7:40               ` Milan Broz
  1 sibling, 1 reply; 65+ messages in thread
From: Ard Biesheuvel @ 2019-07-18  7:28 UTC (permalink / raw)
  To: Herbert Xu; +Cc: Horia Geanta, linux-crypto, dm-devel

On Thu, 18 Jul 2019 at 09:22, Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> On Thu, Jul 18, 2019 at 09:15:39AM +0200, Ard Biesheuvel wrote:
> >
> > Not just the generic implementation: there are numerous synchronous
> > and asynchronous implementations of xts(aes) in the kernel that would
> > have to be fixed, while there are no in-kernel users that actually
> > rely on CTS. Also, in the cbc case, we support CTS by wrapping it into
> > another template, i.e., cts(cbc(aes)).
> >
> > So retroactively redefining what xts(...) means seems like a bad idea
> > to me. If we want to support XTS ciphertext stealing for the benefit
> > of userland, let's do so via the existing cts template, and add
> > support for wrapping XTS to it.
>
> XTS without stealing should be renamed as XEX.  Sure you can then
> wrap it inside cts to form xts but the end result needs to be called
> xts.
>

If we were adding XTS to the kernel today, then I would agree with
you. But xts() has an established meaning now, and I don't think it
makes sense to update all implementations for a theoretical use case,
given that no portable userland code can rely on the correct semantics
today, since CAAM is the only one that implements them correctly.

In any case, I won't have time to fix the ARM or arm64 implementations
(or review the changes if someone else steps up) until the end of
September.

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18  7:21             ` Herbert Xu
  2019-07-18  7:28               ` Ard Biesheuvel
@ 2019-07-18  7:40               ` Milan Broz
  2019-07-18 10:40                 ` Pascal Van Leeuwen
  1 sibling, 1 reply; 65+ messages in thread
From: Milan Broz @ 2019-07-18  7:40 UTC (permalink / raw)
  To: Herbert Xu, Ard Biesheuvel; +Cc: Horia Geanta, linux-crypto, dm-devel

On 18/07/2019 09:21, Herbert Xu wrote:
> On Thu, Jul 18, 2019 at 09:15:39AM +0200, Ard Biesheuvel wrote:
>>
>> Not just the generic implementation: there are numerous synchronous
>> and asynchronous implementations of xts(aes) in the kernel that would
>> have to be fixed, while there are no in-kernel users that actually
>> rely on CTS. Also, in the cbc case, we support CTS by wrapping it into
>> another template, i.e., cts(cbc(aes)).
>>
>> So retroactively redefining what xts(...) means seems like a bad idea
>> to me. If we want to support XTS ciphertext stealing for the benefit
>> of userland, let's do so via the existing cts template, and add
>> support for wrapping XTS to it.
> 
> XTS without stealing should be renamed as XEX.  Sure you can then
> wrap it inside cts to form xts but the end result needs to be called
> xts.

While I fully agree here from the technical point of view,
academically XEX, XEX* is a different mode.
It would create even more confusion.

Couldn't resist, but this is a nice example of what happens when academic,
standardization, and reality meets in one place :)

XTS is already implemented in gcrypt and OpenSSL.
IMO all the implementation should be exactly the same.

I agree with Herbert that the proper way is to implement ciphertext stealing.
Otherwise, it is just incomplete implementation, not a redefining XTS mode!

See the reference in generic code - the 3rd line - link to the IEEE standard.
We do not implement it properly - for more than 12 years!

Reality check - nobody in block layer needs ciphertext stealing, we are always
aligned to block. AF_ALG is a different story, though.

Milan


^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18  7:28               ` Ard Biesheuvel
@ 2019-07-18  7:50                 ` Herbert Xu
  0 siblings, 0 replies; 65+ messages in thread
From: Herbert Xu @ 2019-07-18  7:50 UTC (permalink / raw)
  To: Ard Biesheuvel; +Cc: Horia Geanta, linux-crypto, dm-devel

On Thu, Jul 18, 2019 at 09:28:03AM +0200, Ard Biesheuvel wrote:
>
> If we were adding XTS to the kernel today, then I would agree with
> you. But xts() has an established meaning now, and I don't think it
> makes sense to update all implementations for a theoretical use case,
> given that no portable userland code can rely on the correct semantics
> today, since CAAM is the only one that implements them correctly.
> 
> In any case, I won't have time to fix the ARM or arm64 implementations
> (or review the changes if someone else steps up) until the end of
> September.

I'm not asking you or anyone to fix this right away.  I'm just
saying that this is the direction we should be moving in.

After all, there is no immediate crisis as all that is broken
today is a fuzz test.

It should be possible to do this without causing performance
regressions for ARM.  We could rename the existing xts to a
new name (xek perhaps) and add xts into the cts template as
a wrapper around xek.

That way you don't have to touch the ARM code at all and it
should just work.

PS should we mark xek or whatever it's called as internal so
it isn't visible to user-space?

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18  7:40               ` Milan Broz
@ 2019-07-18 10:40                 ` Pascal Van Leeuwen
  2019-07-18 11:19                   ` Milan Broz
  2019-07-18 15:29                   ` Herbert Xu
  0 siblings, 2 replies; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-18 10:40 UTC (permalink / raw)
  To: Milan Broz, Herbert Xu, Ard Biesheuvel
  Cc: Horia Geanta, linux-crypto, dm-devel

> -----Original Message-----
> From: linux-crypto-owner@vger.kernel.org <linux-crypto-owner@vger.kernel.org> On Behalf Of Milan Broz
> Sent: Thursday, July 18, 2019 9:40 AM
> To: Herbert Xu <herbert@gondor.apana.org.au>; Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Horia Geanta <horia.geanta@nxp.com>; linux-crypto@vger.kernel.org; dm-devel@redhat.com
> Subject: Re: xts fuzz testing and lack of ciphertext stealing support
> 
> On 18/07/2019 09:21, Herbert Xu wrote:
> > On Thu, Jul 18, 2019 at 09:15:39AM +0200, Ard Biesheuvel wrote:
> >>
> >> Not just the generic implementation: there are numerous synchronous
> >> and asynchronous implementations of xts(aes) in the kernel that would
> >> have to be fixed, while there are no in-kernel users that actually
> >> rely on CTS. Also, in the cbc case, we support CTS by wrapping it into
> >> another template, i.e., cts(cbc(aes)).
> >>
> >> So retroactively redefining what xts(...) means seems like a bad idea
> >> to me. If we want to support XTS ciphertext stealing for the benefit
> >> of userland, let's do so via the existing cts template, and add
> >> support for wrapping XTS to it.
> >
> > XTS without stealing should be renamed as XEX.  Sure you can then
> > wrap it inside cts to form xts but the end result needs to be called
> > xts.
> 
> While I fully agree here from the technical point of view,
> academically XEX, XEX* is a different mode.
> It would create even more confusion.
> 
> Couldn't resist, but this is a nice example of what happens when academic,
> standardization, and reality meets in one place :)
> 
> XTS is already implemented in gcrypt and OpenSSL.
> IMO all the implementation should be exactly the same.
> 
> I agree with Herbert that the proper way is to implement ciphertext stealing.
> Otherwise, it is just incomplete implementation, not a redefining XTS mode!
> 
> See the reference in generic code - the 3rd line - link to the IEEE standard.
> We do not implement it properly - for more than 12 years!
> 

Full XTS is XEX-TCB-CTS so the proper terminology for "XTS without CTS" would be XEX-TCB.
But the problem there is that TCB and CTS are generic terms that do not imply a specific 
implementation for generating the tweak -or- performing the ciphertext stealing.
Only the *full* XTS operation is standardized (as IEEE Std P1619).

In fact, using the current cts template around the current xts template actually does NOT
implement standards compliant XTS at all, as the CTS *implementation* for XTS is 
different from the one for CBC as implemented by the current CTS template.
The actual implementation of the ciphertext stealing has (or may have) a security impact,
so the *combined* operation must be cryptanalyzed and adding some random CTS scheme
to some random block cipher mode would be a case of "roll your own crypto" (i.e. bad).

From that perspective - to prevent people from doing cryptographically stupid things -
IMHO it would be better to just pull the CTS into the XTS implementation i.e. make
xts natively support blocks that are not a multiple of (but >=) the cipher blocksize ...

> Reality check - nobody in block layer needs ciphertext stealing, we are always
> aligned to block. AF_ALG is a different story, though.

So you don't support odd sector sizes like 520 , 528, 4112, 4160 or 4224 bytes?
> 
> Milan

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18 10:40                 ` Pascal Van Leeuwen
@ 2019-07-18 11:19                   ` Milan Broz
  2019-07-18 15:27                     ` Herbert Xu
  2019-07-20  6:58                     ` [dm-devel] " Eric Biggers
  2019-07-18 15:29                   ` Herbert Xu
  1 sibling, 2 replies; 65+ messages in thread
From: Milan Broz @ 2019-07-18 11:19 UTC (permalink / raw)
  To: Pascal Van Leeuwen, Herbert Xu, Ard Biesheuvel
  Cc: Horia Geanta, linux-crypto, dm-devel

On 18/07/2019 12:40, Pascal Van Leeuwen wrote:
...
>> See the reference in generic code - the 3rd line - link to the IEEE standard.
>> We do not implement it properly - for more than 12 years!
>>
> 
> Full XTS is XEX-TCB-CTS so the proper terminology for "XTS without CTS" would be XEX-TCB.
> But the problem there is that TCB and CTS are generic terms that do not imply a specific 
> implementation for generating the tweak -or- performing the ciphertext stealing.
> Only the *full* XTS operation is standardized (as IEEE Std P1619).

Yes. Also XTS is allowed in FIPS now. Because the current code cannot submit
anything else than aligned blocks, we are ok.
(I hope. Speaking for disk encryption, dm-crypt, only).

> In fact, using the current cts template around the current xts template actually does NOT
> implement standards compliant XTS at all, as the CTS *implementation* for XTS is 
> different from the one for CBC as implemented by the current CTS template.
> The actual implementation of the ciphertext stealing has (or may have) a security impact,
> so the *combined* operation must be cryptanalyzed and adding some random CTS scheme
> to some random block cipher mode would be a case of "roll your own crypto" (i.e. bad).

> From that perspective - to prevent people from doing cryptographically stupid things -
> IMHO it would be better to just pull the CTS into the XTS implementation i.e. make
> xts natively support blocks that are not a multiple of (but >=) the cipher blocksize ...

I would definitely prefer adding CTS directly to XTS (as it is in gcrypt or OpenSSL now)
instead of some new compositions.

Also, I would like to avoid another "just because it is nicer" module dependence (XTS->XEX->ECB).
Last time (when XTS was reimplemented using ECB) we have many reports with initramfs
missing ECB module preventing boot from AES-XTS encrypted root after kernel upgrade...
Just saying. (Despite the last time it was keyring what broke encrypted boot ;-)

(That said, I will try to find some volunteer to help with CTS in XTS implementation, if needed.)

>> Reality check - nobody in block layer needs ciphertext stealing, we are always
>> aligned to block. AF_ALG is a different story, though.
> 
> So you don't support odd sector sizes like 520 , 528, 4112, 4160 or 4224 bytes?

No. Dm-crypt supports only power of two blocks, up to 4k (IOW: 512, 1024, 2048, 4096 bytes).
(Not more, because of compatible page size - this could be fixed in future though.)

The 520 hw sector is usually 512 + 8 bytes for DIF (data integrity field).
We can emulate something similar with dm-integrity, but the data section (input to encryption)
must be always as specified above (rest is in integrity bio section).

Milan

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18 11:19                   ` Milan Broz
@ 2019-07-18 15:27                     ` Herbert Xu
  2019-07-20  6:58                     ` [dm-devel] " Eric Biggers
  1 sibling, 0 replies; 65+ messages in thread
From: Herbert Xu @ 2019-07-18 15:27 UTC (permalink / raw)
  To: Milan Broz
  Cc: Pascal Van Leeuwen, Ard Biesheuvel, Horia Geanta, linux-crypto, dm-devel

On Thu, Jul 18, 2019 at 01:19:41PM +0200, Milan Broz wrote:
>
> Also, I would like to avoid another "just because it is nicer" module dependence (XTS->XEX->ECB).
> Last time (when XTS was reimplemented using ECB) we have many reports with initramfs
> missing ECB module preventing boot from AES-XTS encrypted root after kernel upgrade...
> Just saying. (Despite the last time it was keyring what broke encrypted boot ;-)
> 
> (That said, I will try to find some volunteer to help with CTS in XTS implementation, if needed.)

Well the main advantage of doing it on top of the existing xts is
that you can retain the existing ARM implementations without any
changes.  This would also apply to any existing xts drivers that
also don't implement CTS (I'm not aware of the status on these so
someone will need to check them one by one).

But if you were going to volunteer to change them all in one swoop
then it wouldn't matter.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18 10:40                 ` Pascal Van Leeuwen
  2019-07-18 11:19                   ` Milan Broz
@ 2019-07-18 15:29                   ` Herbert Xu
  2019-07-18 15:43                     ` Pascal Van Leeuwen
  1 sibling, 1 reply; 65+ messages in thread
From: Herbert Xu @ 2019-07-18 15:29 UTC (permalink / raw)
  To: Pascal Van Leeuwen
  Cc: Milan Broz, Ard Biesheuvel, Horia Geanta, linux-crypto, dm-devel

On Thu, Jul 18, 2019 at 10:40:54AM +0000, Pascal Van Leeuwen wrote:
>
> In fact, using the current cts template around the current xts template actually does NOT
> implement standards compliant XTS at all, as the CTS *implementation* for XTS is 
> different from the one for CBC as implemented by the current CTS template.

The template is just a name.  The implementation can do whatever it
wants for each instance.  So obviously we would employ a different
implementation for xts compared to cbc.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18 15:29                   ` Herbert Xu
@ 2019-07-18 15:43                     ` Pascal Van Leeuwen
  2019-07-18 15:51                       ` Herbert Xu
  0 siblings, 1 reply; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-18 15:43 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Milan Broz, Ard Biesheuvel, Horia Geanta, linux-crypto, dm-devel

> > In fact, using the current cts template around the current xts template actually does NOT
> > implement standards compliant XTS at all, as the CTS *implementation* for XTS is
> > different from the one for CBC as implemented by the current CTS template.
> 
> The template is just a name.  The implementation can do whatever it
> wants for each instance.  So obviously we would employ a different
> implementation for xts compared to cbc.
>
Hmmm ... so the generic CTS template would have to figure out whether it is wrapped 
around ECB, CBC, XTS or whatever and then adjust to that?

For ECB and CBC I suppose that's techically possible. But then what do I get when I
try to wrap CTS around some block cipher mode it doesn't recognise? Tricky ...

For XTS, you have this additional curve ball being thrown in called the "tweak".
For encryption, the underlying "xts" would need to be able to chain the tweak,
from what I've seen of the source the implementation cannot do that.

For decryption, you actually first need to decrypt the last block with the last
tweak before you can decrypt the 2nd last block with the 2nd last tweak.

Not sure how you intend to handle that with some generic wrapper around "xts".

> 
> Cheers,
> --
> Email: Herbert Xu <herbert@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18 15:43                     ` Pascal Van Leeuwen
@ 2019-07-18 15:51                       ` Herbert Xu
  2019-07-18 16:19                         ` Ard Biesheuvel
  2019-07-18 16:35                         ` Pascal Van Leeuwen
  0 siblings, 2 replies; 65+ messages in thread
From: Herbert Xu @ 2019-07-18 15:51 UTC (permalink / raw)
  To: Pascal Van Leeuwen
  Cc: Milan Broz, Ard Biesheuvel, Horia Geanta, linux-crypto, dm-devel

On Thu, Jul 18, 2019 at 03:43:28PM +0000, Pascal Van Leeuwen wrote:
>
> Hmmm ... so the generic CTS template would have to figure out whether it is wrapped 
> around ECB, CBC, XTS or whatever and then adjust to that?

That's not hard to do.  Right now cts only supports cbc.  IOW
if you pass it anything else it will refuse to instantiate.

> For XTS, you have this additional curve ball being thrown in called the "tweak".
> For encryption, the underlying "xts" would need to be able to chain the tweak,
> from what I've seen of the source the implementation cannot do that.

You simply use the underlying xts for the first n - 2 blocks and
do the last two by hand.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18 15:51                       ` Herbert Xu
@ 2019-07-18 16:19                         ` Ard Biesheuvel
  2019-07-18 16:22                           ` Herbert Xu
  2019-07-18 17:03                           ` Pascal Van Leeuwen
  2019-07-18 16:35                         ` Pascal Van Leeuwen
  1 sibling, 2 replies; 65+ messages in thread
From: Ard Biesheuvel @ 2019-07-18 16:19 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Pascal Van Leeuwen, Milan Broz, Horia Geanta, linux-crypto, dm-devel

On Thu, 18 Jul 2019 at 17:51, Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> On Thu, Jul 18, 2019 at 03:43:28PM +0000, Pascal Van Leeuwen wrote:
> >
> > Hmmm ... so the generic CTS template would have to figure out whether it is wrapped
> > around ECB, CBC, XTS or whatever and then adjust to that?
>
> That's not hard to do.  Right now cts only supports cbc.  IOW
> if you pass it anything else it will refuse to instantiate.
>
> > For XTS, you have this additional curve ball being thrown in called the "tweak".
> > For encryption, the underlying "xts" would need to be able to chain the tweak,
> > from what I've seen of the source the implementation cannot do that.
>
> You simply use the underlying xts for the first n - 2 blocks and
> do the last two by hand.
>

OK, so it appears the XTS ciphertext stealing algorithm does not
include the peculiar reordering of the 2 final blocks, which means
that the kernel's implementation of XTS already conforms to the spec
for inputs that are a multiple of the block size.

The reason I am not a fan of making any changes here is that there are
no in-kernel users that require ciphertext stealing for XTS, nor is
anyone aware of any reason why we should be adding it to the userland
interface. So we are basically adding dead code so that we are
theoretically compliant in a way that we will never exercise in
practice.

Note that for software algorithms such as the bit sliced NEON
implementation of AES, which can only operate on 8 AES blocks at a
time, doing the final 2 blocks sequentially is going to seriously
impact performance. This means whatever wrapper we invent around xex()
(or whatever we call it) should go out of its way to ensure that the
common, non-CTS case does not regress in performance, and the special
handling is only invoked when necessary (which will be never).

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18 16:19                         ` Ard Biesheuvel
@ 2019-07-18 16:22                           ` Herbert Xu
  2019-07-18 17:03                           ` Pascal Van Leeuwen
  1 sibling, 0 replies; 65+ messages in thread
From: Herbert Xu @ 2019-07-18 16:22 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Pascal Van Leeuwen, Milan Broz, Horia Geanta, linux-crypto, dm-devel

On Thu, Jul 18, 2019 at 06:19:24PM +0200, Ard Biesheuvel wrote:
>
> Note that for software algorithms such as the bit sliced NEON
> implementation of AES, which can only operate on 8 AES blocks at a
> time, doing the final 2 blocks sequentially is going to seriously
> impact performance. This means whatever wrapper we invent around xex()
> (or whatever we call it) should go out of its way to ensure that the
> common, non-CTS case does not regress in performance, and the special
> handling is only invoked when necessary (which will be never).

Agreed.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18 15:51                       ` Herbert Xu
  2019-07-18 16:19                         ` Ard Biesheuvel
@ 2019-07-18 16:35                         ` Pascal Van Leeuwen
  2019-07-19  1:47                           ` Herbert Xu
  1 sibling, 1 reply; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-18 16:35 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Milan Broz, Ard Biesheuvel, Horia Geanta, linux-crypto, dm-devel

> >
> > Hmmm ... so the generic CTS template would have to figure out whether it is wrapped
> > around ECB, CBC, XTS or whatever and then adjust to that?
> 
> That's not hard to do.  Right now cts only supports cbc.  IOW
> if you pass it anything else it will refuse to instantiate.
> 

Ah, I see it now:

if (strncmp(alg->base.cra_name, "cbc(", 4))
		goto err_drop_spawn;

So you cannot even do cts(xts) at the moment.

> > For XTS, you have this additional curve ball being thrown in called the "tweak".
> > For encryption, the underlying "xts" would need to be able to chain the tweak,
> > from what I've seen of the source the implementation cannot do that.
> 
> You simply use the underlying xts for the first n - 2 blocks and
> do the last two by hand.
> 

Possible, but then you're basically pulling a generic XTS implementation into cts.c,
since XTS is nothing more than the repeated application of just that.
Same thing would apply to any other mode you eventually need cts for.

But I suppose it would save you the trouble of having to add cts to all individual
optimized software implementations of xts-which-is-not-really-xts, so I can see 
WHY you would want to do it that way ...

Tthen there's still the issue of advancing that last tweak. From what I've seen,
the xts implementation does not output the last tweak so you would have to 
recompute it locally in cts.c as block_cipher_enc(iv) * alpha^j. Which is wasteful.
Of course this could be solved by redefining xts to output the last tweak
like CBC/CTR output their last IV ... Which would probably give us HW guys
trouble again because our HW cannot do *that* ... (While the HW very likely 
*does* implement the cipher text stealing properly, just to be able to boast
about IEEE P1619 compliance ...)

> Cheers,
> --
> Email: Herbert Xu <herbert@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18 16:19                         ` Ard Biesheuvel
  2019-07-18 16:22                           ` Herbert Xu
@ 2019-07-18 17:03                           ` Pascal Van Leeuwen
  2019-07-19  5:34                             ` Ard Biesheuvel
  1 sibling, 1 reply; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-18 17:03 UTC (permalink / raw)
  To: Ard Biesheuvel, Herbert Xu
  Cc: Milan Broz, Horia Geanta, linux-crypto, dm-devel

> > > For XTS, you have this additional curve ball being thrown in called the "tweak".
> > > For encryption, the underlying "xts" would need to be able to chain the tweak,
> > > from what I've seen of the source the implementation cannot do that.
> >
> > You simply use the underlying xts for the first n - 2 blocks and
> > do the last two by hand.
> >
> 
> OK, so it appears the XTS ciphertext stealing algorithm does not
> include the peculiar reordering of the 2 final blocks, which means
> that the kernel's implementation of XTS already conforms to the spec
> for inputs that are a multiple of the block size.
> 
Yes, for XTS you effectively don't do CTS if it's a 16 byte multiple ...

> The reason I am not a fan of making any changes here is that there are
> no in-kernel users that require ciphertext stealing for XTS, nor is
> anyone aware of any reason why we should be adding it to the userland
> interface. So we are basically adding dead code so that we are
> theoretically compliant in a way that we will never exercise in
> practice.
> 
You know, having worked on all kinds of workarounds for silly irrelevant
(IMHO) corner cases in  the inside-secure hardware driver over the past
months just to keep testmgr happy, this is kind of ironic ...

Cipher text stealing happens to be a *major* part of the XTS specification
(it's not actually XTS without the CTS part!), yet you are suggesting not 
to implement it because *you* don't have or know a use case for it.
That seems like a pretty bad argument to me. It's not some minor corner 
case that's not supported.The implementation is just *incomplete*
without it.

> Note that for software algorithms such as the bit sliced NEON
> implementation of AES, which can only operate on 8 AES blocks at a
> time, doing the final 2 blocks sequentially is going to seriously
> impact performance. This means whatever wrapper we invent around xex()
> (or whatever we call it) should go out of its way to ensure that the
> common, non-CTS case does not regress in performance, and the special
> handling is only invoked when necessary (which will be never).
>
I pretty much made the same argument about all these driver workarounds
slowing down my driver fast path but that was considered a non-issue.

In this particular case, it should not need to be more than:

if (unlikely(size & 15)) {
  xts_with_partial_last_block();
} else {
  xts_with_only_full_blocks();
}

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com


^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18 16:35                         ` Pascal Van Leeuwen
@ 2019-07-19  1:47                           ` Herbert Xu
  0 siblings, 0 replies; 65+ messages in thread
From: Herbert Xu @ 2019-07-19  1:47 UTC (permalink / raw)
  To: Pascal Van Leeuwen
  Cc: Milan Broz, Ard Biesheuvel, Horia Geanta, linux-crypto, dm-devel

On Thu, Jul 18, 2019 at 04:35:26PM +0000, Pascal Van Leeuwen wrote:
>
> Tthen there's still the issue of advancing that last tweak. From what I've seen,
> the xts implementation does not output the last tweak so you would have to 
> recompute it locally in cts.c as block_cipher_enc(iv) * alpha^j. Which is wasteful.
> Of course this could be solved by redefining xts to output the last tweak
> like CBC/CTR output their last IV ... Which would probably give us HW guys
> trouble again because our HW cannot do *that* ... (While the HW very likely 
> *does* implement the cipher text stealing properly, just to be able to boast
> about IEEE P1619 compliance ...)

If your hardware supports XTS properly then you wouldn't even need
to use this new template.  You would directly export the xts
interface.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-18 17:03                           ` Pascal Van Leeuwen
@ 2019-07-19  5:34                             ` Ard Biesheuvel
  2019-07-19  7:29                               ` Pascal Van Leeuwen
  0 siblings, 1 reply; 65+ messages in thread
From: Ard Biesheuvel @ 2019-07-19  5:34 UTC (permalink / raw)
  To: Pascal Van Leeuwen
  Cc: Herbert Xu, Milan Broz, Horia Geanta, linux-crypto, dm-devel

On Thu, 18 Jul 2019 at 19:03, Pascal Van Leeuwen
<pvanleeuwen@verimatrix.com> wrote:
>
> > > > For XTS, you have this additional curve ball being thrown in called the "tweak".
> > > > For encryption, the underlying "xts" would need to be able to chain the tweak,
> > > > from what I've seen of the source the implementation cannot do that.
> > >
> > > You simply use the underlying xts for the first n - 2 blocks and
> > > do the last two by hand.
> > >
> >
> > OK, so it appears the XTS ciphertext stealing algorithm does not
> > include the peculiar reordering of the 2 final blocks, which means
> > that the kernel's implementation of XTS already conforms to the spec
> > for inputs that are a multiple of the block size.
> >
> Yes, for XTS you effectively don't do CTS if it's a 16 byte multiple ...
>
> > The reason I am not a fan of making any changes here is that there are
> > no in-kernel users that require ciphertext stealing for XTS, nor is
> > anyone aware of any reason why we should be adding it to the userland
> > interface. So we are basically adding dead code so that we are
> > theoretically compliant in a way that we will never exercise in
> > practice.
> >
> You know, having worked on all kinds of workarounds for silly irrelevant
> (IMHO) corner cases in  the inside-secure hardware driver over the past
> months just to keep testmgr happy, this is kind of ironic ...
>
> Cipher text stealing happens to be a *major* part of the XTS specification
> (it's not actually XTS without the CTS part!), yet you are suggesting not
> to implement it because *you* don't have or know a use case for it.
> That seems like a pretty bad argument to me. It's not some minor corner
> case that's not supported.The implementation is just *incomplete*
> without it.
>

I would argue that these cases are diametrically opposite: you
proposed to remove support for zero length input vectors from the
entire crypto API to prevent your driver from having to deal with
inputs that the hardware cannot handle.

I am proposing not to add support for cases that we have no need for.
XTS without CTS is indistinguishable from XTS with CTS if the inputs
are always a multiple of the block size, and in 12 years, nobody has
ever raised the issue that our support is limited to that. So what
problem are we fixing by changing this? dm-crypt does not care,
fscrypt does not care, userland does not care (given that it does not
work today and we are only finding out now due to some fuzz test
failing on CAAM)


> > Note that for software algorithms such as the bit sliced NEON
> > implementation of AES, which can only operate on 8 AES blocks at a
> > time, doing the final 2 blocks sequentially is going to seriously
> > impact performance. This means whatever wrapper we invent around xex()
> > (or whatever we call it) should go out of its way to ensure that the
> > common, non-CTS case does not regress in performance, and the special
> > handling is only invoked when necessary (which will be never).
> >
> I pretty much made the same argument about all these driver workarounds
> slowing down my driver fast path but that was considered a non-issue.
>
> In this particular case, it should not need to be more than:
>
> if (unlikely(size & 15)) {
>   xts_with_partial_last_block();
> } else {
>   xts_with_only_full_blocks();
> }
>

Of course. But why add this at all if it is known to be dead code?

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: xts fuzz testing and lack of ciphertext stealing support
  2019-07-19  5:34                             ` Ard Biesheuvel
@ 2019-07-19  7:29                               ` Pascal Van Leeuwen
  2019-07-19 17:14                                 ` Ard Biesheuvel
  0 siblings, 1 reply; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-19  7:29 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Herbert Xu, Milan Broz, Horia Geanta, linux-crypto, dm-devel

> -----Original Message-----
> From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Sent: Friday, July 19, 2019 7:35 AM
> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> Cc: Herbert Xu <herbert@gondor.apana.org.au>; Milan Broz <gmazyland@gmail.com>; Horia Geanta <horia.geanta@nxp.com>; linux-
> crypto@vger.kernel.org; dm-devel@redhat.com
> Subject: Re: xts fuzz testing and lack of ciphertext stealing support
 > 
> I would argue that these cases are diametrically opposite: you
> proposed to remove support for zero length input vectors from the
> entire crypto API to prevent your driver from having to deal with
> inputs that the hardware cannot handle.
> 
I did not propose any such thing - I just proposed to make zero length hash support *optional*
(i.e. don't fail and disable the driver on it) as it's totally irrelevant for 99.99999% of use cases.
(including *all* use cases I consider relevant for HW acceleration)

> I am proposing not to add support for cases that we have no need for.
>
While you are proposing to stick with an implementation that can only deal with 6.25% (1/16th) of 
*legal* input data for XTS and fails on the remaining 93.75%. That's hardly a corner case anymore.
 
> XTS without CTS is indistinguishable from XTS with CTS if the inputs
> are always a multiple of the block size, and in 12 years, nobody has
> ever raised the issue that our support is limited to that. So what
> problem are we fixing by changing this? dm-crypt does not care,
> fscrypt does not care, userland does not care (given that it does not
> work today and we are only finding out now due to some fuzz test
> failing on CAAM)
> 
If it's not supported, then it cannot be used. Most people would not start complaining about that, 
they would just roll their own locally or they'd give up and/or use something else. 
So the fact that it's currently not being used does not mean a whole lot. Also, it does not mean 
that there will not be a relevant use case tomorrow. (and I can assure you there *are* definitely
real-life use cases, so I have to assume these are currently handled outside of the base kernel)

In any case, if you try to use XTS you would *expect* it to work for any input >= 16 bytes as that's
how the algorithm is *specified*. Without the CTS part it's simply not XTS.

> > I pretty much made the same argument about all these driver workarounds
> > slowing down my driver fast path but that was considered a non-issue.
> >
> > In this particular case, it should not need to be more than:
> >
> > if (unlikely(size & 15)) {
> >   xts_with_partial_last_block();
> > } else {
> >   xts_with_only_full_blocks();
> > }
> >
> 
> Of course. But why add this at all if it is known to be dead code?
>
But that's just an assumption and assumptions are the root of all evil ;-)


Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: xts fuzz testing and lack of ciphertext stealing support
  2019-07-19  7:29                               ` Pascal Van Leeuwen
@ 2019-07-19 17:14                                 ` Ard Biesheuvel
  2019-07-19 20:07                                   ` Pascal Van Leeuwen
  0 siblings, 1 reply; 65+ messages in thread
From: Ard Biesheuvel @ 2019-07-19 17:14 UTC (permalink / raw)
  To: Pascal Van Leeuwen
  Cc: Herbert Xu, Milan Broz, Horia Geanta, linux-crypto, dm-devel

On Fri, 19 Jul 2019 at 09:29, Pascal Van Leeuwen
<pvanleeuwen@verimatrix.com> wrote:
>
> > -----Original Message-----
> > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > Sent: Friday, July 19, 2019 7:35 AM
> > To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> > Cc: Herbert Xu <herbert@gondor.apana.org.au>; Milan Broz <gmazyland@gmail.com>; Horia Geanta <horia.geanta@nxp.com>; linux-
> > crypto@vger.kernel.org; dm-devel@redhat.com
> > Subject: Re: xts fuzz testing and lack of ciphertext stealing support
>  >
> > I would argue that these cases are diametrically opposite: you
> > proposed to remove support for zero length input vectors from the
> > entire crypto API to prevent your driver from having to deal with
> > inputs that the hardware cannot handle.
> >
> I did not propose any such thing - I just proposed to make zero length hash support *optional*
> (i.e. don't fail and disable the driver on it) as it's totally irrelevant for 99.99999% of use cases.
> (including *all* use cases I consider relevant for HW acceleration)
>

Fair enough. But it did involve making modifications to the generic
layer, since there are known users of the AF_ALG interface that may
pass zero length inputs (e.g., sha1sum).

> > I am proposing not to add support for cases that we have no need for.
> >
> While you are proposing to stick with an implementation that can only deal with 6.25% (1/16th) of
> *legal* input data for XTS and fails on the remaining 93.75%. That's hardly a corner case anymore.
>

I never said it was a corner case, nor does it make a lot of sense to
reason about fractional compliance, given that 100% of the inputs we
ever encounter are covered by your 6.25% of legal input data.

What i did say was that the moving parts we will add to the code will
never be put into motion, while they do increase the validation space,
and so the value of the contribution will be negative.

Perhaps I should emphasize that my concern is mainly about in-kernel
usage of the sync software ciphers, since they typically have no use
for userland, given that they can simply issue the same instructions
directly. For AF_ALG, I agree that exposing a non-compliant XTS
implementation is a bad idea.

> > XTS without CTS is indistinguishable from XTS with CTS if the inputs
> > are always a multiple of the block size, and in 12 years, nobody has
> > ever raised the issue that our support is limited to that. So what
> > problem are we fixing by changing this? dm-crypt does not care,
> > fscrypt does not care, userland does not care (given that it does not
> > work today and we are only finding out now due to some fuzz test
> > failing on CAAM)
> >
> If it's not supported, then it cannot be used. Most people would not start complaining about that,
> they would just roll their own locally or they'd give up and/or use something else.
> So the fact that it's currently not being used does not mean a whole lot. Also, it does not mean
> that there will not be a relevant use case tomorrow. (and I can assure you there *are* definitely
> real-life use cases, so I have to assume these are currently handled outside of the base kernel)
>
> In any case, if you try to use XTS you would *expect* it to work for any input >= 16 bytes as that's
> how the algorithm is *specified*. Without the CTS part it's simply not XTS.
>

I really don't care what we call it. My point is that we don't need
functionality that we will not use, regardless of how it is called.

> > > I pretty much made the same argument about all these driver workarounds
> > > slowing down my driver fast path but that was considered a non-issue.
> > >
> > > In this particular case, it should not need to be more than:
> > >
> > > if (unlikely(size & 15)) {
> > >   xts_with_partial_last_block();
> > > } else {
> > >   xts_with_only_full_blocks();
> > > }
> > >
> >
> > Of course. But why add this at all if it is known to be dead code?
> >
> But that's just an assumption and assumptions are the root of all evil ;-)
>

I think it was premature optimization that is the root of all evil, no?

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: xts fuzz testing and lack of ciphertext stealing support
  2019-07-19 17:14                                 ` Ard Biesheuvel
@ 2019-07-19 20:07                                   ` Pascal Van Leeuwen
  0 siblings, 0 replies; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-19 20:07 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Herbert Xu, Milan Broz, Horia Geanta, linux-crypto, dm-devel

Hi Ard,

> -----Original Message-----
> From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Sent: Friday, July 19, 2019 7:15 PM
> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> Cc: Herbert Xu <herbert@gondor.apana.org.au>; Milan Broz <gmazyland@gmail.com>; Horia Geanta <horia.geanta@nxp.com>; linux-
> crypto@vger.kernel.org; dm-devel@redhat.com
> Subject: Re: xts fuzz testing and lack of ciphertext stealing support
> 
> On Fri, 19 Jul 2019 at 09:29, Pascal Van Leeuwen
> <pvanleeuwen@verimatrix.com> wrote:
> >
> > > -----Original Message-----
> > > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > Sent: Friday, July 19, 2019 7:35 AM
> > > To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> > > Cc: Herbert Xu <herbert@gondor.apana.org.au>; Milan Broz <gmazyland@gmail.com>; Horia Geanta <horia.geanta@nxp.com>; linux-
> > > crypto@vger.kernel.org; dm-devel@redhat.com
> > > Subject: Re: xts fuzz testing and lack of ciphertext stealing support
> >  >
> > > I would argue that these cases are diametrically opposite: you
> > > proposed to remove support for zero length input vectors from the
> > > entire crypto API to prevent your driver from having to deal with
> > > inputs that the hardware cannot handle.
> > >
> > I did not propose any such thing - I just proposed to make zero length hash support *optional*
> > (i.e. don't fail and disable the driver on it) as it's totally irrelevant for 99.99999% of use cases.
> > (including *all* use cases I consider relevant for HW acceleration)
> >
> 
> Fair enough. But it did involve making modifications to the generic
> layer, since there are known users of the AF_ALG interface that may
> pass zero length inputs (e.g., sha1sum).
> 
Which is why I gave up and grudgingly implemented those workarounds ;-)

But they complicate the driver by orders of a magnitude and for that the
same validation argument you give below applies: there's now a lot more
- and worse: a lot more complicated! - code to validate in this driver.

While I seriously question the value of that contribution as well: I don't
expect this driver to ever be used for these corner cases, as you would 
simply never *select* it for the applications that might hit them (or, if
you try it an run into trouble, just select another implementation). So 
its really just there to keep testmgr happy and nothing else (IMHO).

Also: hardware may have a lot more "problematic" limitations that are not
currently hit by testmgr. The limitations that are hit are kind of arbitrary.
(and I'm not going to shoot myself in the foot by going into detail there ;-)

> > > I am proposing not to add support for cases that we have no need for.
> > >
> > While you are proposing to stick with an implementation that can only deal with 6.25% (1/16th) of
> > *legal* input data for XTS and fails on the remaining 93.75%. That's hardly a corner case anymore.
> >
> 
> I never said it was a corner case, nor does it make a lot of sense to
> reason about fractional compliance, given that 100% of the inputs we
> ever encounter are covered by your 6.25% of legal input data.
> 
> What i did say was that the moving parts we will add to the code will
> never be put into motion, while they do increase the validation space,
> and so the value of the contribution will be negative.
> 
How can you be so sure that it will never be used?

> Perhaps I should emphasize that my concern is mainly about in-kernel
> usage of the sync software ciphers, since they typically have no use
> for userland, given that they can simply issue the same instructions
> directly. For AF_ALG, I agree that exposing a non-compliant XTS
> implementation is a bad idea.
> 
And my concern is that I want to accelerate xts but I will now have to 
cripple my driver to return -EINVAL on non-multiple-of-16 inputs while
my hardware can actually handle that just fine, just because the actual
"reference" implementation is not compliant. And if I don't match its
(incorrect!) behavior *I* will be the one being failed by testmgr.
Which seems rather unfair :-(

Note that I'm not just interested in providing support to existing 
implementations, I may want to support additional features to provide
to my customers that they can use from their own drivers/applications
they build on top of it (which do not exist yet at this moment)

> > > XTS without CTS is indistinguishable from XTS with CTS if the inputs
> > > are always a multiple of the block size, and in 12 years, nobody has
> > > ever raised the issue that our support is limited to that. So what
> > > problem are we fixing by changing this? dm-crypt does not care,
> > > fscrypt does not care, userland does not care (given that it does not
> > > work today and we are only finding out now due to some fuzz test
> > > failing on CAAM)
> > >
> > If it's not supported, then it cannot be used. Most people would not start complaining about that,
> > they would just roll their own locally or they'd give up and/or use something else.
> > So the fact that it's currently not being used does not mean a whole lot. Also, it does not mean
> > that there will not be a relevant use case tomorrow. (and I can assure you there *are* definitely
> > real-life use cases, so I have to assume these are currently handled outside of the base kernel)
> >
> > In any case, if you try to use XTS you would *expect* it to work for any input >= 16 bytes as that's
> > how the algorithm is *specified*. Without the CTS part it's simply not XTS.
> >
> 
> I really don't care what we call it. My point is that we don't need
> functionality that we will not use, regardless of how it is called.
> 
If you make an implementation that is considered (e.g. by testmgr for fuzz testing)
to be the golden reference, then it'd better be fully compliant with the relevant
specification(s). I guess that's the real point I'm trying to make. 
Now a fully compliant implementation would get penalized for being fully compliant.

> > > > I pretty much made the same argument about all these driver workarounds
> > > > slowing down my driver fast path but that was considered a non-issue.
> > > >
> > > > In this particular case, it should not need to be more than:
> > > >
> > > > if (unlikely(size & 15)) {
> > > >   xts_with_partial_last_block();
> > > > } else {
> > > >   xts_with_only_full_blocks();
> > > > }
> > > >
> > >
> > > Of course. But why add this at all if it is known to be dead code?
> > >
> > But that's just an assumption and assumptions are the root of all evil ;-)
> >
> 
> I think it was premature optimization that is the root of all evil, no?

You are talking to a guy who used to prefer to use hand-optimized assembly 
for *everything* :-D There's no such thing as time wasted on optimization!
(if it wasn't useful, at least it was fun to do! :-P)

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com


^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-18 11:19                   ` Milan Broz
  2019-07-18 15:27                     ` Herbert Xu
@ 2019-07-20  6:58                     ` Eric Biggers
  2019-07-20  7:35                       ` Milan Broz
  1 sibling, 1 reply; 65+ messages in thread
From: Eric Biggers @ 2019-07-20  6:58 UTC (permalink / raw)
  To: Milan Broz
  Cc: Pascal Van Leeuwen, Herbert Xu, Ard Biesheuvel, dm-devel,
	linux-crypto, Horia Geanta

On Thu, Jul 18, 2019 at 01:19:41PM +0200, Milan Broz wrote:
> 
> > From that perspective - to prevent people from doing cryptographically stupid things -
> > IMHO it would be better to just pull the CTS into the XTS implementation i.e. make
> > xts natively support blocks that are not a multiple of (but >=) the cipher blocksize ...
> 
> I would definitely prefer adding CTS directly to XTS (as it is in gcrypt or OpenSSL now)
> instead of some new compositions.
> 
> Also, I would like to avoid another "just because it is nicer" module dependence (XTS->XEX->ECB).
> Last time (when XTS was reimplemented using ECB) we have many reports with initramfs
> missing ECB module preventing boot from AES-XTS encrypted root after kernel upgrade...
> Just saying. (Despite the last time it was keyring what broke encrypted boot ;-)
> 

Can't the "missing modules in initramfs" issue be solved by using a
MODULE_SOFTDEP()?  Actually, why isn't that being used for xts -> ecb already?

(There was also a bug where CONFIG_CRYPTO_XTS didn't select CONFIG_CRYPTO_ECB,
but that was simply a bug, which was fixed.)

Or "xts" and "xex" could go in the same kernel module xts.ko, which would make
this a non-issue.

Anyway, I agree that the partial block support, if added, should just be made
available under the name "xts", as people would expect.  It doesn't need a new
name.

- Eric

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-20  6:58                     ` [dm-devel] " Eric Biggers
@ 2019-07-20  7:35                       ` Milan Broz
  2019-07-21  9:50                         ` Ard Biesheuvel
  0 siblings, 1 reply; 65+ messages in thread
From: Milan Broz @ 2019-07-20  7:35 UTC (permalink / raw)
  To: Pascal Van Leeuwen, Herbert Xu, Ard Biesheuvel, dm-devel,
	linux-crypto, Horia Geanta

On 20/07/2019 08:58, Eric Biggers wrote:
> On Thu, Jul 18, 2019 at 01:19:41PM +0200, Milan Broz wrote:
>> Also, I would like to avoid another "just because it is nicer" module dependence (XTS->XEX->ECB).
>> Last time (when XTS was reimplemented using ECB) we have many reports with initramfs
>> missing ECB module preventing boot from AES-XTS encrypted root after kernel upgrade...
>> Just saying. (Despite the last time it was keyring what broke encrypted boot ;-)
>>
> 
> Can't the "missing modules in initramfs" issue be solved by using a
> MODULE_SOFTDEP()?  Actually, why isn't that being used for xts -> ecb already?
> 
> (There was also a bug where CONFIG_CRYPTO_XTS didn't select CONFIG_CRYPTO_ECB,
> but that was simply a bug, which was fixed.)

Sure, and it is solved now. (Some systems with a hardcoded list of modules
have to be manually updated etc., but that is just bad design).
It can be done properly from the beginning.

I just want to say that that switching to XEX looks like wasting time to me
for no additional benefit.

Fully implementing XTS does make much more sense for me, even though it is long-term
the effort and the only user, for now, would be testmgr.

So, there are no users because it does not work. It makes no sense
to implement it, because there are no users... (sorry, sounds like catch 22 :)

(Maybe someone can use it for keyslot encryption for keys not aligned to
block size, dunno. Actually, some filesystem encryption could have use for it.) 

> Or "xts" and "xex" could go in the same kernel module xts.ko, which would make
> this a non-issue.

If it is not available for users, I really see no reason to introduce XEX when
it is just XTS with full blocks.

If it is visible to users, it needs some work in userspace - XEX (as XTS) need two keys,
people are already confused enough that 256bit key in AES-XTS means AES-128...
So the examples, hints, man pages need to be updated, at least.

Milan

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-20  7:35                       ` Milan Broz
@ 2019-07-21  9:50                         ` Ard Biesheuvel
  2019-07-22  9:44                           ` Pascal Van Leeuwen
  0 siblings, 1 reply; 65+ messages in thread
From: Ard Biesheuvel @ 2019-07-21  9:50 UTC (permalink / raw)
  To: Milan Broz
  Cc: Pascal Van Leeuwen, Herbert Xu, dm-devel, linux-crypto, Horia Geanta

On Sat, 20 Jul 2019 at 10:35, Milan Broz <gmazyland@gmail.com> wrote:
>
> On 20/07/2019 08:58, Eric Biggers wrote:
> > On Thu, Jul 18, 2019 at 01:19:41PM +0200, Milan Broz wrote:
> >> Also, I would like to avoid another "just because it is nicer" module dependence (XTS->XEX->ECB).
> >> Last time (when XTS was reimplemented using ECB) we have many reports with initramfs
> >> missing ECB module preventing boot from AES-XTS encrypted root after kernel upgrade...
> >> Just saying. (Despite the last time it was keyring what broke encrypted boot ;-)
> >>
> >
> > Can't the "missing modules in initramfs" issue be solved by using a
> > MODULE_SOFTDEP()?  Actually, why isn't that being used for xts -> ecb already?
> >
> > (There was also a bug where CONFIG_CRYPTO_XTS didn't select CONFIG_CRYPTO_ECB,
> > but that was simply a bug, which was fixed.)
>
> Sure, and it is solved now. (Some systems with a hardcoded list of modules
> have to be manually updated etc., but that is just bad design).
> It can be done properly from the beginning.
>
> I just want to say that that switching to XEX looks like wasting time to me
> for no additional benefit.
>
> Fully implementing XTS does make much more sense for me, even though it is long-term
> the effort and the only user, for now, would be testmgr.
>
> So, there are no users because it does not work. It makes no sense
> to implement it, because there are no users... (sorry, sounds like catch 22 :)
>
> (Maybe someone can use it for keyslot encryption for keys not aligned to
> block size, dunno. Actually, some filesystem encryption could have use for it.)
>
> > Or "xts" and "xex" could go in the same kernel module xts.ko, which would make
> > this a non-issue.
>
> If it is not available for users, I really see no reason to introduce XEX when
> it is just XTS with full blocks.
>
> If it is visible to users, it needs some work in userspace - XEX (as XTS) need two keys,
> people are already confused enough that 256bit key in AES-XTS means AES-128...
> So the examples, hints, man pages need to be updated, at least.
>

OK, consider me persuaded. We are already exposing xts(...) to
userland, and since we already implement a proper subset of true XTS,
it will be simply a matter of making sure that the existing XTS
implementations don't regress in performance on the non-CTS code
paths.

It would be useful, though, to have some generic helper functions,
e.g., like the one we have for CBC, or the one I recently proposed for
CTS, so that existing implementations (such as the bit sliced AES) can
easily be augmented with a CTS code path (but performance may not be
optimal in those cases). For the ARM implementations based on AES
instructions, it should be reasonably straight forward to implement it
close to optimally by reusing some of the code I added for CBC-CTS
(but I won't get around to doing that for a while). If there are any
volunteers for looking into the generic or x86/AES-NI implementations,
please come forward :-) Also, if any of the publications that were
quoted in this thread have suitable test vectors, that would be good
to know.

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-21  9:50                         ` Ard Biesheuvel
@ 2019-07-22  9:44                           ` Pascal Van Leeuwen
  2019-07-22 16:43                             ` Ard Biesheuvel
  0 siblings, 1 reply; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-22  9:44 UTC (permalink / raw)
  To: Ard Biesheuvel, Milan Broz
  Cc: Herbert Xu, dm-devel, linux-crypto, Horia Geanta

> -----Original Message-----
> From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Sent: Sunday, July 21, 2019 11:50 AM
> To: Milan Broz <gmazyland@gmail.com>
> Cc: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> On Sat, 20 Jul 2019 at 10:35, Milan Broz <gmazyland@gmail.com> wrote:
> >
> > On 20/07/2019 08:58, Eric Biggers wrote:
> > > On Thu, Jul 18, 2019 at 01:19:41PM +0200, Milan Broz wrote:
> > >> Also, I would like to avoid another "just because it is nicer" module dependence (XTS->XEX->ECB).
> > >> Last time (when XTS was reimplemented using ECB) we have many reports with initramfs
> > >> missing ECB module preventing boot from AES-XTS encrypted root after kernel upgrade...
> > >> Just saying. (Despite the last time it was keyring what broke encrypted boot ;-)
> > >>
> > >
> > > Can't the "missing modules in initramfs" issue be solved by using a
> > > MODULE_SOFTDEP()?  Actually, why isn't that being used for xts -> ecb already?
> > >
> > > (There was also a bug where CONFIG_CRYPTO_XTS didn't select CONFIG_CRYPTO_ECB,
> > > but that was simply a bug, which was fixed.)
> >
> > Sure, and it is solved now. (Some systems with a hardcoded list of modules
> > have to be manually updated etc., but that is just bad design).
> > It can be done properly from the beginning.
> >
> > I just want to say that that switching to XEX looks like wasting time to me
> > for no additional benefit.
> >
> > Fully implementing XTS does make much more sense for me, even though it is long-term
> > the effort and the only user, for now, would be testmgr.
> >
> > So, there are no users because it does not work. It makes no sense
> > to implement it, because there are no users... (sorry, sounds like catch 22 :)
> >
> > (Maybe someone can use it for keyslot encryption for keys not aligned to
> > block size, dunno. Actually, some filesystem encryption could have use for it.)
> >
> > > Or "xts" and "xex" could go in the same kernel module xts.ko, which would make
> > > this a non-issue.
> >
> > If it is not available for users, I really see no reason to introduce XEX when
> > it is just XTS with full blocks.
> >
> > If it is visible to users, it needs some work in userspace - XEX (as XTS) need two keys,
> > people are already confused enough that 256bit key in AES-XTS means AES-128...
> > So the examples, hints, man pages need to be updated, at least.
> >
> 
> OK, consider me persuaded. We are already exposing xts(...) to
> userland, and since we already implement a proper subset of true XTS,
> it will be simply a matter of making sure that the existing XTS
> implementations don't regress in performance on the non-CTS code
> paths.
> 
> It would be useful, though, to have some generic helper functions,
> e.g., like the one we have for CBC, or the one I recently proposed for
> CTS, so that existing implementations (such as the bit sliced AES) can
> easily be augmented with a CTS code path (but performance may not be
> optimal in those cases). For the ARM implementations based on AES
> instructions, it should be reasonably straight forward to implement it
> close to optimally by reusing some of the code I added for CBC-CTS
> (but I won't get around to doing that for a while). If there are any
> volunteers for looking into the generic or x86/AES-NI implementations,
> please come forward :-) Also, if any of the publications that were
> quoted in this thread have suitable test vectors, that would be good
> to know.

Unfortunately, these algorithm & protocol specifications tend to be very frugal when it
comes to providing test vectors, barely scratching the surface of any corner cases, but
at least there is one non-multiple-of-16 vector in the original IEEE P1619 / D16 
specification in Annex B Test Vectors (last vector, "XTS-AES-128 applied for a data unit 
that is not a multiple of 16 bytes")

Besides that, I'd be happy to generate some testvectors from our defacto-standard
implementation ;-)

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com


^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-22  9:44                           ` Pascal Van Leeuwen
@ 2019-07-22 16:43                             ` Ard Biesheuvel
  2019-07-22 22:46                               ` Pascal Van Leeuwen
                                                 ` (2 more replies)
  0 siblings, 3 replies; 65+ messages in thread
From: Ard Biesheuvel @ 2019-07-22 16:43 UTC (permalink / raw)
  To: Pascal Van Leeuwen
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto, Horia Geanta

On Mon, 22 Jul 2019 at 12:44, Pascal Van Leeuwen
<pvanleeuwen@verimatrix.com> wrote:
>
> > -----Original Message-----
> > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > Sent: Sunday, July 21, 2019 11:50 AM
> > To: Milan Broz <gmazyland@gmail.com>
> > Cc: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> > crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> >
> > On Sat, 20 Jul 2019 at 10:35, Milan Broz <gmazyland@gmail.com> wrote:
> > >
> > > On 20/07/2019 08:58, Eric Biggers wrote:
> > > > On Thu, Jul 18, 2019 at 01:19:41PM +0200, Milan Broz wrote:
> > > >> Also, I would like to avoid another "just because it is nicer" module dependence (XTS->XEX->ECB).
> > > >> Last time (when XTS was reimplemented using ECB) we have many reports with initramfs
> > > >> missing ECB module preventing boot from AES-XTS encrypted root after kernel upgrade...
> > > >> Just saying. (Despite the last time it was keyring what broke encrypted boot ;-)
> > > >>
> > > >
> > > > Can't the "missing modules in initramfs" issue be solved by using a
> > > > MODULE_SOFTDEP()?  Actually, why isn't that being used for xts -> ecb already?
> > > >
> > > > (There was also a bug where CONFIG_CRYPTO_XTS didn't select CONFIG_CRYPTO_ECB,
> > > > but that was simply a bug, which was fixed.)
> > >
> > > Sure, and it is solved now. (Some systems with a hardcoded list of modules
> > > have to be manually updated etc., but that is just bad design).
> > > It can be done properly from the beginning.
> > >
> > > I just want to say that that switching to XEX looks like wasting time to me
> > > for no additional benefit.
> > >
> > > Fully implementing XTS does make much more sense for me, even though it is long-term
> > > the effort and the only user, for now, would be testmgr.
> > >
> > > So, there are no users because it does not work. It makes no sense
> > > to implement it, because there are no users... (sorry, sounds like catch 22 :)
> > >
> > > (Maybe someone can use it for keyslot encryption for keys not aligned to
> > > block size, dunno. Actually, some filesystem encryption could have use for it.)
> > >
> > > > Or "xts" and "xex" could go in the same kernel module xts.ko, which would make
> > > > this a non-issue.
> > >
> > > If it is not available for users, I really see no reason to introduce XEX when
> > > it is just XTS with full blocks.
> > >
> > > If it is visible to users, it needs some work in userspace - XEX (as XTS) need two keys,
> > > people are already confused enough that 256bit key in AES-XTS means AES-128...
> > > So the examples, hints, man pages need to be updated, at least.
> > >
> >
> > OK, consider me persuaded. We are already exposing xts(...) to
> > userland, and since we already implement a proper subset of true XTS,
> > it will be simply a matter of making sure that the existing XTS
> > implementations don't regress in performance on the non-CTS code
> > paths.
> >
> > It would be useful, though, to have some generic helper functions,
> > e.g., like the one we have for CBC, or the one I recently proposed for
> > CTS, so that existing implementations (such as the bit sliced AES) can
> > easily be augmented with a CTS code path (but performance may not be
> > optimal in those cases). For the ARM implementations based on AES
> > instructions, it should be reasonably straight forward to implement it
> > close to optimally by reusing some of the code I added for CBC-CTS
> > (but I won't get around to doing that for a while). If there are any
> > volunteers for looking into the generic or x86/AES-NI implementations,
> > please come forward :-) Also, if any of the publications that were
> > quoted in this thread have suitable test vectors, that would be good
> > to know.
>
> Unfortunately, these algorithm & protocol specifications tend to be very frugal when it
> comes to providing test vectors, barely scratching the surface of any corner cases, but
> at least there is one non-multiple-of-16 vector in the original IEEE P1619 / D16
> specification in Annex B Test Vectors (last vector, "XTS-AES-128 applied for a data unit
> that is not a multiple of 16 bytes")
>

Actually, that spec has a couple of test vectors. Unfortunately, they
are all rather short (except the last one in the 'no multiple of 16
bytes' paragraph, but unfortunately, that one is in fact a multiple of
16 bytes)

I added them here [0] along with an arm64 implementation for the AES
instruction based driver. Could you please double check that these
work against your driver? That would establish a ground truth against
which we can implement the generic version as well.

[0] https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=xts-cts

> Besides that, I'd be happy to generate some testvectors from our defacto-standard
> implementation ;-)
>

One or two larger ones would be useful, yes.

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-22 16:43                             ` Ard Biesheuvel
@ 2019-07-22 22:46                               ` Pascal Van Leeuwen
  2019-07-24 12:23                               ` Pascal Van Leeuwen
  2019-07-24 16:10                               ` Pascal Van Leeuwen
  2 siblings, 0 replies; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-22 22:46 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto, Horia Geanta

> -----Original Message-----
> From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Sent: Monday, July 22, 2019 6:43 PM
> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> On Mon, 22 Jul 2019 at 12:44, Pascal Van Leeuwen
> <pvanleeuwen@verimatrix.com> wrote:
> >
> > > -----Original Message-----
> > > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > Sent: Sunday, July 21, 2019 11:50 AM
> > > To: Milan Broz <gmazyland@gmail.com>
> > > Cc: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com;
> linux-
> > > crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> > > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> > >
> > > On Sat, 20 Jul 2019 at 10:35, Milan Broz <gmazyland@gmail.com> wrote:
> > > >
> > > > On 20/07/2019 08:58, Eric Biggers wrote:
> > > > > On Thu, Jul 18, 2019 at 01:19:41PM +0200, Milan Broz wrote:
> > > > >> Also, I would like to avoid another "just because it is nicer" module dependence (XTS->XEX->ECB).
> > > > >> Last time (when XTS was reimplemented using ECB) we have many reports with initramfs
> > > > >> missing ECB module preventing boot from AES-XTS encrypted root after kernel upgrade...
> > > > >> Just saying. (Despite the last time it was keyring what broke encrypted boot ;-)
> > > > >>
> > > > >
> > > > > Can't the "missing modules in initramfs" issue be solved by using a
> > > > > MODULE_SOFTDEP()?  Actually, why isn't that being used for xts -> ecb already?
> > > > >
> > > > > (There was also a bug where CONFIG_CRYPTO_XTS didn't select CONFIG_CRYPTO_ECB,
> > > > > but that was simply a bug, which was fixed.)
> > > >
> > > > Sure, and it is solved now. (Some systems with a hardcoded list of modules
> > > > have to be manually updated etc., but that is just bad design).
> > > > It can be done properly from the beginning.
> > > >
> > > > I just want to say that that switching to XEX looks like wasting time to me
> > > > for no additional benefit.
> > > >
> > > > Fully implementing XTS does make much more sense for me, even though it is long-term
> > > > the effort and the only user, for now, would be testmgr.
> > > >
> > > > So, there are no users because it does not work. It makes no sense
> > > > to implement it, because there are no users... (sorry, sounds like catch 22 :)
> > > >
> > > > (Maybe someone can use it for keyslot encryption for keys not aligned to
> > > > block size, dunno. Actually, some filesystem encryption could have use for it.)
> > > >
> > > > > Or "xts" and "xex" could go in the same kernel module xts.ko, which would make
> > > > > this a non-issue.
> > > >
> > > > If it is not available for users, I really see no reason to introduce XEX when
> > > > it is just XTS with full blocks.
> > > >
> > > > If it is visible to users, it needs some work in userspace - XEX (as XTS) need two keys,
> > > > people are already confused enough that 256bit key in AES-XTS means AES-128...
> > > > So the examples, hints, man pages need to be updated, at least.
> > > >
> > >
> > > OK, consider me persuaded. We are already exposing xts(...) to
> > > userland, and since we already implement a proper subset of true XTS,
> > > it will be simply a matter of making sure that the existing XTS
> > > implementations don't regress in performance on the non-CTS code
> > > paths.
> > >
> > > It would be useful, though, to have some generic helper functions,
> > > e.g., like the one we have for CBC, or the one I recently proposed for
> > > CTS, so that existing implementations (such as the bit sliced AES) can
> > > easily be augmented with a CTS code path (but performance may not be
> > > optimal in those cases). For the ARM implementations based on AES
> > > instructions, it should be reasonably straight forward to implement it
> > > close to optimally by reusing some of the code I added for CBC-CTS
> > > (but I won't get around to doing that for a while). If there are any
> > > volunteers for looking into the generic or x86/AES-NI implementations,
> > > please come forward :-) Also, if any of the publications that were
> > > quoted in this thread have suitable test vectors, that would be good
> > > to know.
> >
> > Unfortunately, these algorithm & protocol specifications tend to be very frugal when it
> > comes to providing test vectors, barely scratching the surface of any corner cases, but
> > at least there is one non-multiple-of-16 vector in the original IEEE P1619 / D16
> > specification in Annex B Test Vectors (last vector, "XTS-AES-128 applied for a data unit
> > that is not a multiple of 16 bytes")
> >
> 
> Actually, that spec has a couple of test vectors. Unfortunately, they
> are all rather short (except the last one in the 'no multiple of 16
> bytes' paragraph, but unfortunately, that one is in fact a multiple of
> 16 bytes)
> 
Yes, as usual it's very limited and does not cover all the interesting cases.
It's still better than nothing, though.
The fact that the last one actually *is* a multiple of 16 is quite hilarious, I
had not spotted that myself yet ...

> I added them here [0] along with an arm64 implementation for the AES
> instruction based driver. Could you please double check that these
> work against your driver? That would establish a ground truth against
> which we can implement the generic version as well.
> 
> [0] https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=xts-cts
> 
I'd be happy to do that except that adding XTS support to the driver is still
on my todo list. I was actually waiting for some of my earlier patches to be
acked as the situation is becoming rather unmanageable for me ...

But I suppose I can give it a shot anyway - adding XTS should be relatively
simple compare to, say, adding AES-GCM. But I'm sure I'll somehow regret
saying that.

> > Besides that, I'd be happy to generate some testvectors from our defacto-standard
> > implementation ;-)
> >
> 
> One or two larger ones would be useful, yes.
>
It was more or less a joke as how would you know them to be correct?
(not that I don't trust my hardware, of course ...)
If it's just for playing around, I can provide some larger vectors, no problem.

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com


^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-22 16:43                             ` Ard Biesheuvel
  2019-07-22 22:46                               ` Pascal Van Leeuwen
@ 2019-07-24 12:23                               ` Pascal Van Leeuwen
  2019-07-24 12:50                                 ` Pascal Van Leeuwen
  2019-07-24 16:10                               ` Pascal Van Leeuwen
  2 siblings, 1 reply; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-24 12:23 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto, Horia Geanta

Ard, 

> -----Original Message-----
> From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Sent: Monday, July 22, 2019 6:43 PM
> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> On Mon, 22 Jul 2019 at 12:44, Pascal Van Leeuwen
> <pvanleeuwen@verimatrix.com> wrote:
> >
> > > -----Original Message-----
> > > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > Sent: Sunday, July 21, 2019 11:50 AM
> > > To: Milan Broz <gmazyland@gmail.com>
> > > Cc: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> > > crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> > > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> > >
> > > On Sat, 20 Jul 2019 at 10:35, Milan Broz <gmazyland@gmail.com> wrote:
> > > >
> > > > On 20/07/2019 08:58, Eric Biggers wrote:
> > > > > On Thu, Jul 18, 2019 at 01:19:41PM +0200, Milan Broz wrote:
> > > > >> Also, I would like to avoid another "just because it is nicer" module dependence (XTS->XEX->ECB).
> > > > >> Last time (when XTS was reimplemented using ECB) we have many reports with initramfs
> > > > >> missing ECB module preventing boot from AES-XTS encrypted root after kernel upgrade...
> > > > >> Just saying. (Despite the last time it was keyring what broke encrypted boot ;-)
> > > > >>
> > > > >
> > > > > Can't the "missing modules in initramfs" issue be solved by using a
> > > > > MODULE_SOFTDEP()?  Actually, why isn't that being used for xts -> ecb already?
> > > > >
> > > > > (There was also a bug where CONFIG_CRYPTO_XTS didn't select CONFIG_CRYPTO_ECB,
> > > > > but that was simply a bug, which was fixed.)
> > > >
> > > > Sure, and it is solved now. (Some systems with a hardcoded list of modules
> > > > have to be manually updated etc., but that is just bad design).
> > > > It can be done properly from the beginning.
> > > >
> > > > I just want to say that that switching to XEX looks like wasting time to me
> > > > for no additional benefit.
> > > >
> > > > Fully implementing XTS does make much more sense for me, even though it is long-term
> > > > the effort and the only user, for now, would be testmgr.
> > > >
> > > > So, there are no users because it does not work. It makes no sense
> > > > to implement it, because there are no users... (sorry, sounds like catch 22 :)
> > > >
> > > > (Maybe someone can use it for keyslot encryption for keys not aligned to
> > > > block size, dunno. Actually, some filesystem encryption could have use for it.)
> > > >
> > > > > Or "xts" and "xex" could go in the same kernel module xts.ko, which would make
> > > > > this a non-issue.
> > > >
> > > > If it is not available for users, I really see no reason to introduce XEX when
> > > > it is just XTS with full blocks.
> > > >
> > > > If it is visible to users, it needs some work in userspace - XEX (as XTS) need two keys,
> > > > people are already confused enough that 256bit key in AES-XTS means AES-128...
> > > > So the examples, hints, man pages need to be updated, at least.
> > > >
> > >
> > > OK, consider me persuaded. We are already exposing xts(...) to
> > > userland, and since we already implement a proper subset of true XTS,
> > > it will be simply a matter of making sure that the existing XTS
> > > implementations don't regress in performance on the non-CTS code
> > > paths.
> > >
> > > It would be useful, though, to have some generic helper functions,
> > > e.g., like the one we have for CBC, or the one I recently proposed for
> > > CTS, so that existing implementations (such as the bit sliced AES) can
> > > easily be augmented with a CTS code path (but performance may not be
> > > optimal in those cases). For the ARM implementations based on AES
> > > instructions, it should be reasonably straight forward to implement it
> > > close to optimally by reusing some of the code I added for CBC-CTS
> > > (but I won't get around to doing that for a while). If there are any
> > > volunteers for looking into the generic or x86/AES-NI implementations,
> > > please come forward :-) Also, if any of the publications that were
> > > quoted in this thread have suitable test vectors, that would be good
> > > to know.
> >
> > Unfortunately, these algorithm & protocol specifications tend to be very frugal when it
> > comes to providing test vectors, barely scratching the surface of any corner cases, but
> > at least there is one non-multiple-of-16 vector in the original IEEE P1619 / D16
> > specification in Annex B Test Vectors (last vector, "XTS-AES-128 applied for a data unit
> > that is not a multiple of 16 bytes")
> >
> 
> Actually, that spec has a couple of test vectors. Unfortunately, they
> are all rather short (except the last one in the 'no multiple of 16
> bytes' paragraph, but unfortunately, that one is in fact a multiple of
> 16 bytes)
> 
> I added them here [0] along with an arm64 implementation for the AES
> instruction based driver. Could you please double check that these
> work against your driver? That would establish a ground truth against
> which we can implement the generic version as well.
> 
> [0] https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=xts-cts
> 
I'm working on my XTS implementation now and I noticed something funny with the test
vectors. The new CTS ones you added here, I can perfectly trace back to the IEEE spec,
they match byte-for-byte.

However, the ones that already existed puzzle me. The input data matches vectors from
the IEEE spec, however the expected output cipher text does NOT ????

Case in point, the very first vector, which has a key of all zeroes, a sector number (IV)
or all zeroes and an all zeroes plaintext of 32 bytes, which matches the 1st spec vector:
testmgr.h expects:
 "\x4b\xc9\x44\x4a\x11\xa3\xef\xac"
 "\x30\x74\xe4\x44\x52\x77\x97\x43"
  "\xa7\x60\xb2\x45\x2e\xf9\x00\x90"
  "\x9f\xaa\xfd\x89\x6e\x9d\x4a\xe0"

But the specification expects:
917cf69ebd68b2ec9b9fe9a3eadda692cd43d2f59598ed858c02c2652fbf922e

Which also happens to be what our hardware does ...

Did you notice the same thing with your implementation? Am I missing something??

> > Besides that, I'd be happy to generate some testvectors from our defacto-standard
> > implementation ;-)
> >
> 
> One or two larger ones would be useful, yes.

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com




^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-24 12:23                               ` Pascal Van Leeuwen
@ 2019-07-24 12:50                                 ` Pascal Van Leeuwen
  0 siblings, 0 replies; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-24 12:50 UTC (permalink / raw)
  To: Pascal Van Leeuwen, Ard Biesheuvel
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto, Horia Geanta

Ah, ugh, my bad. Ignore my previous mail below. Turns out the tf_xts_tv_template I was looking
at is for xts(twofish) not xts(aes) ... which was not immediately obvious from the name, for me.

Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

> -----Original Message-----
> From: linux-crypto-owner@vger.kernel.org <linux-crypto-owner@vger.kernel.org> On Behalf Of Pascal Van Leeuwen
> Sent: Wednesday, July 24, 2019 2:23 PM
> To: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> Subject: RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> Ard,
> 
> > -----Original Message-----
> > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > Sent: Monday, July 22, 2019 6:43 PM
> > To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> > Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> > crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> >
> > On Mon, 22 Jul 2019 at 12:44, Pascal Van Leeuwen
> > <pvanleeuwen@verimatrix.com> wrote:
> > >
> > > > -----Original Message-----
> > > > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > > Sent: Sunday, July 21, 2019 11:50 AM
> > > > To: Milan Broz <gmazyland@gmail.com>
> > > > Cc: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com;
> linux-
> > > > crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> > > > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> > > >
> > > > On Sat, 20 Jul 2019 at 10:35, Milan Broz <gmazyland@gmail.com> wrote:
> > > > >
> > > > > On 20/07/2019 08:58, Eric Biggers wrote:
> > > > > > On Thu, Jul 18, 2019 at 01:19:41PM +0200, Milan Broz wrote:
> > > > > >> Also, I would like to avoid another "just because it is nicer" module dependence (XTS->XEX->ECB).
> > > > > >> Last time (when XTS was reimplemented using ECB) we have many reports with initramfs
> > > > > >> missing ECB module preventing boot from AES-XTS encrypted root after kernel upgrade...
> > > > > >> Just saying. (Despite the last time it was keyring what broke encrypted boot ;-)
> > > > > >>
> > > > > >
> > > > > > Can't the "missing modules in initramfs" issue be solved by using a
> > > > > > MODULE_SOFTDEP()?  Actually, why isn't that being used for xts -> ecb already?
> > > > > >
> > > > > > (There was also a bug where CONFIG_CRYPTO_XTS didn't select CONFIG_CRYPTO_ECB,
> > > > > > but that was simply a bug, which was fixed.)
> > > > >
> > > > > Sure, and it is solved now. (Some systems with a hardcoded list of modules
> > > > > have to be manually updated etc., but that is just bad design).
> > > > > It can be done properly from the beginning.
> > > > >
> > > > > I just want to say that that switching to XEX looks like wasting time to me
> > > > > for no additional benefit.
> > > > >
> > > > > Fully implementing XTS does make much more sense for me, even though it is long-term
> > > > > the effort and the only user, for now, would be testmgr.
> > > > >
> > > > > So, there are no users because it does not work. It makes no sense
> > > > > to implement it, because there are no users... (sorry, sounds like catch 22 :)
> > > > >
> > > > > (Maybe someone can use it for keyslot encryption for keys not aligned to
> > > > > block size, dunno. Actually, some filesystem encryption could have use for it.)
> > > > >
> > > > > > Or "xts" and "xex" could go in the same kernel module xts.ko, which would make
> > > > > > this a non-issue.
> > > > >
> > > > > If it is not available for users, I really see no reason to introduce XEX when
> > > > > it is just XTS with full blocks.
> > > > >
> > > > > If it is visible to users, it needs some work in userspace - XEX (as XTS) need two keys,
> > > > > people are already confused enough that 256bit key in AES-XTS means AES-128...
> > > > > So the examples, hints, man pages need to be updated, at least.
> > > > >
> > > >
> > > > OK, consider me persuaded. We are already exposing xts(...) to
> > > > userland, and since we already implement a proper subset of true XTS,
> > > > it will be simply a matter of making sure that the existing XTS
> > > > implementations don't regress in performance on the non-CTS code
> > > > paths.
> > > >
> > > > It would be useful, though, to have some generic helper functions,
> > > > e.g., like the one we have for CBC, or the one I recently proposed for
> > > > CTS, so that existing implementations (such as the bit sliced AES) can
> > > > easily be augmented with a CTS code path (but performance may not be
> > > > optimal in those cases). For the ARM implementations based on AES
> > > > instructions, it should be reasonably straight forward to implement it
> > > > close to optimally by reusing some of the code I added for CBC-CTS
> > > > (but I won't get around to doing that for a while). If there are any
> > > > volunteers for looking into the generic or x86/AES-NI implementations,
> > > > please come forward :-) Also, if any of the publications that were
> > > > quoted in this thread have suitable test vectors, that would be good
> > > > to know.
> > >
> > > Unfortunately, these algorithm & protocol specifications tend to be very frugal when it
> > > comes to providing test vectors, barely scratching the surface of any corner cases, but
> > > at least there is one non-multiple-of-16 vector in the original IEEE P1619 / D16
> > > specification in Annex B Test Vectors (last vector, "XTS-AES-128 applied for a data unit
> > > that is not a multiple of 16 bytes")
> > >
> >
> > Actually, that spec has a couple of test vectors. Unfortunately, they
> > are all rather short (except the last one in the 'no multiple of 16
> > bytes' paragraph, but unfortunately, that one is in fact a multiple of
> > 16 bytes)
> >
> > I added them here [0] along with an arm64 implementation for the AES
> > instruction based driver. Could you please double check that these
> > work against your driver? That would establish a ground truth against
> > which we can implement the generic version as well.
> >
> > [0] https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=xts-cts
> >
> I'm working on my XTS implementation now and I noticed something funny with the test
> vectors. The new CTS ones you added here, I can perfectly trace back to the IEEE spec,
> they match byte-for-byte.
> 
> However, the ones that already existed puzzle me. The input data matches vectors from
> the IEEE spec, however the expected output cipher text does NOT ????
> 
> Case in point, the very first vector, which has a key of all zeroes, a sector number (IV)
> or all zeroes and an all zeroes plaintext of 32 bytes, which matches the 1st spec vector:
> testmgr.h expects:
>  "\x4b\xc9\x44\x4a\x11\xa3\xef\xac"
>  "\x30\x74\xe4\x44\x52\x77\x97\x43"
>   "\xa7\x60\xb2\x45\x2e\xf9\x00\x90"
>   "\x9f\xaa\xfd\x89\x6e\x9d\x4a\xe0"
> 
> But the specification expects:
> 917cf69ebd68b2ec9b9fe9a3eadda692cd43d2f59598ed858c02c2652fbf922e
> 
> Which also happens to be what our hardware does ...
> 
> Did you notice the same thing with your implementation? Am I missing something??
> 
> > > Besides that, I'd be happy to generate some testvectors from our defacto-standard
> > > implementation ;-)
> > >
> >
> > One or two larger ones would be useful, yes.
> 
> Regards,
> Pascal van Leeuwen
> Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
> www.insidesecure.com
> 
> 


^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-22 16:43                             ` Ard Biesheuvel
  2019-07-22 22:46                               ` Pascal Van Leeuwen
  2019-07-24 12:23                               ` Pascal Van Leeuwen
@ 2019-07-24 16:10                               ` Pascal Van Leeuwen
  2019-07-25  6:22                                 ` Ard Biesheuvel
  2 siblings, 1 reply; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-24 16:10 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto, Horia Geanta

Ard,

> -----Original Message-----
> From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Sent: Monday, July 22, 2019 6:43 PM
> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> On Mon, 22 Jul 2019 at 12:44, Pascal Van Leeuwen
> <pvanleeuwen@verimatrix.com> wrote:
> >
> > > -----Original Message-----
> > > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > Sent: Sunday, July 21, 2019 11:50 AM
> > > To: Milan Broz <gmazyland@gmail.com>
> > > Cc: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> > > crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> > > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> > >
> > > On Sat, 20 Jul 2019 at 10:35, Milan Broz <gmazyland@gmail.com> wrote:
> > > >
> > > > On 20/07/2019 08:58, Eric Biggers wrote:
> > > > > On Thu, Jul 18, 2019 at 01:19:41PM +0200, Milan Broz wrote:
> > > > >> Also, I would like to avoid another "just because it is nicer" module dependence (XTS->XEX->ECB).
> > > > >> Last time (when XTS was reimplemented using ECB) we have many reports with initramfs
> > > > >> missing ECB module preventing boot from AES-XTS encrypted root after kernel upgrade...
> > > > >> Just saying. (Despite the last time it was keyring what broke encrypted boot ;-)
> > > > >>
> > > > >
> > > > > Can't the "missing modules in initramfs" issue be solved by using a
> > > > > MODULE_SOFTDEP()?  Actually, why isn't that being used for xts -> ecb already?
> > > > >
> > > > > (There was also a bug where CONFIG_CRYPTO_XTS didn't select CONFIG_CRYPTO_ECB,
> > > > > but that was simply a bug, which was fixed.)
> > > >
> > > > Sure, and it is solved now. (Some systems with a hardcoded list of modules
> > > > have to be manually updated etc., but that is just bad design).
> > > > It can be done properly from the beginning.
> > > >
> > > > I just want to say that that switching to XEX looks like wasting time to me
> > > > for no additional benefit.
> > > >
> > > > Fully implementing XTS does make much more sense for me, even though it is long-term
> > > > the effort and the only user, for now, would be testmgr.
> > > >
> > > > So, there are no users because it does not work. It makes no sense
> > > > to implement it, because there are no users... (sorry, sounds like catch 22 :)
> > > >
> > > > (Maybe someone can use it for keyslot encryption for keys not aligned to
> > > > block size, dunno. Actually, some filesystem encryption could have use for it.)
> > > >
> > > > > Or "xts" and "xex" could go in the same kernel module xts.ko, which would make
> > > > > this a non-issue.
> > > >
> > > > If it is not available for users, I really see no reason to introduce XEX when
> > > > it is just XTS with full blocks.
> > > >
> > > > If it is visible to users, it needs some work in userspace - XEX (as XTS) need two keys,
> > > > people are already confused enough that 256bit key in AES-XTS means AES-128...
> > > > So the examples, hints, man pages need to be updated, at least.
> > > >
> > >
> > > OK, consider me persuaded. We are already exposing xts(...) to
> > > userland, and since we already implement a proper subset of true XTS,
> > > it will be simply a matter of making sure that the existing XTS
> > > implementations don't regress in performance on the non-CTS code
> > > paths.
> > >
> > > It would be useful, though, to have some generic helper functions,
> > > e.g., like the one we have for CBC, or the one I recently proposed for
> > > CTS, so that existing implementations (such as the bit sliced AES) can
> > > easily be augmented with a CTS code path (but performance may not be
> > > optimal in those cases). For the ARM implementations based on AES
> > > instructions, it should be reasonably straight forward to implement it
> > > close to optimally by reusing some of the code I added for CBC-CTS
> > > (but I won't get around to doing that for a while). If there are any
> > > volunteers for looking into the generic or x86/AES-NI implementations,
> > > please come forward :-) Also, if any of the publications that were
> > > quoted in this thread have suitable test vectors, that would be good
> > > to know.
> >
> > Unfortunately, these algorithm & protocol specifications tend to be very frugal when it
> > comes to providing test vectors, barely scratching the surface of any corner cases, but
> > at least there is one non-multiple-of-16 vector in the original IEEE P1619 / D16
> > specification in Annex B Test Vectors (last vector, "XTS-AES-128 applied for a data unit
> > that is not a multiple of 16 bytes")
> >
> 
> Actually, that spec has a couple of test vectors. Unfortunately, they
> are all rather short (except the last one in the 'no multiple of 16
> bytes' paragraph, but unfortunately, that one is in fact a multiple of
> 16 bytes)
> 
> I added them here [0] along with an arm64 implementation for the AES
> instruction based driver. Could you please double check that these
> work against your driver? 
>
I got XTS working with the inside-secure driver and these (and all other) vectors pass :-)

> That would establish a ground truth against
> which we can implement the generic version as well.
> 
> [0] https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=xts-cts
> 
> > Besides that, I'd be happy to generate some testvectors from our defacto-standard
> > implementation ;-)
> >
> 
> One or two larger ones would be useful, yes.
>
I'll see if I can extract some suitable vectors from our verification suite ...

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-24 16:10                               ` Pascal Van Leeuwen
@ 2019-07-25  6:22                                 ` Ard Biesheuvel
  2019-07-25  7:49                                   ` Pascal Van Leeuwen
  0 siblings, 1 reply; 65+ messages in thread
From: Ard Biesheuvel @ 2019-07-25  6:22 UTC (permalink / raw)
  To: Pascal Van Leeuwen
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto, Horia Geanta

On Wed, 24 Jul 2019 at 19:10, Pascal Van Leeuwen
<pvanleeuwen@verimatrix.com> wrote:
>
> Ard,
>
> > -----Original Message-----
> > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > Sent: Monday, July 22, 2019 6:43 PM
> > To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> > Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> > crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> >
> > On Mon, 22 Jul 2019 at 12:44, Pascal Van Leeuwen
> > <pvanleeuwen@verimatrix.com> wrote:
> > >
> > > > -----Original Message-----
> > > > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > > Sent: Sunday, July 21, 2019 11:50 AM
> > > > To: Milan Broz <gmazyland@gmail.com>
> > > > Cc: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> > > > crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> > > > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> > > >
> > > > On Sat, 20 Jul 2019 at 10:35, Milan Broz <gmazyland@gmail.com> wrote:
> > > > >
> > > > > On 20/07/2019 08:58, Eric Biggers wrote:
> > > > > > On Thu, Jul 18, 2019 at 01:19:41PM +0200, Milan Broz wrote:
> > > > > >> Also, I would like to avoid another "just because it is nicer" module dependence (XTS->XEX->ECB).
> > > > > >> Last time (when XTS was reimplemented using ECB) we have many reports with initramfs
> > > > > >> missing ECB module preventing boot from AES-XTS encrypted root after kernel upgrade...
> > > > > >> Just saying. (Despite the last time it was keyring what broke encrypted boot ;-)
> > > > > >>
> > > > > >
> > > > > > Can't the "missing modules in initramfs" issue be solved by using a
> > > > > > MODULE_SOFTDEP()?  Actually, why isn't that being used for xts -> ecb already?
> > > > > >
> > > > > > (There was also a bug where CONFIG_CRYPTO_XTS didn't select CONFIG_CRYPTO_ECB,
> > > > > > but that was simply a bug, which was fixed.)
> > > > >
> > > > > Sure, and it is solved now. (Some systems with a hardcoded list of modules
> > > > > have to be manually updated etc., but that is just bad design).
> > > > > It can be done properly from the beginning.
> > > > >
> > > > > I just want to say that that switching to XEX looks like wasting time to me
> > > > > for no additional benefit.
> > > > >
> > > > > Fully implementing XTS does make much more sense for me, even though it is long-term
> > > > > the effort and the only user, for now, would be testmgr.
> > > > >
> > > > > So, there are no users because it does not work. It makes no sense
> > > > > to implement it, because there are no users... (sorry, sounds like catch 22 :)
> > > > >
> > > > > (Maybe someone can use it for keyslot encryption for keys not aligned to
> > > > > block size, dunno. Actually, some filesystem encryption could have use for it.)
> > > > >
> > > > > > Or "xts" and "xex" could go in the same kernel module xts.ko, which would make
> > > > > > this a non-issue.
> > > > >
> > > > > If it is not available for users, I really see no reason to introduce XEX when
> > > > > it is just XTS with full blocks.
> > > > >
> > > > > If it is visible to users, it needs some work in userspace - XEX (as XTS) need two keys,
> > > > > people are already confused enough that 256bit key in AES-XTS means AES-128...
> > > > > So the examples, hints, man pages need to be updated, at least.
> > > > >
> > > >
> > > > OK, consider me persuaded. We are already exposing xts(...) to
> > > > userland, and since we already implement a proper subset of true XTS,
> > > > it will be simply a matter of making sure that the existing XTS
> > > > implementations don't regress in performance on the non-CTS code
> > > > paths.
> > > >
> > > > It would be useful, though, to have some generic helper functions,
> > > > e.g., like the one we have for CBC, or the one I recently proposed for
> > > > CTS, so that existing implementations (such as the bit sliced AES) can
> > > > easily be augmented with a CTS code path (but performance may not be
> > > > optimal in those cases). For the ARM implementations based on AES
> > > > instructions, it should be reasonably straight forward to implement it
> > > > close to optimally by reusing some of the code I added for CBC-CTS
> > > > (but I won't get around to doing that for a while). If there are any
> > > > volunteers for looking into the generic or x86/AES-NI implementations,
> > > > please come forward :-) Also, if any of the publications that were
> > > > quoted in this thread have suitable test vectors, that would be good
> > > > to know.
> > >
> > > Unfortunately, these algorithm & protocol specifications tend to be very frugal when it
> > > comes to providing test vectors, barely scratching the surface of any corner cases, but
> > > at least there is one non-multiple-of-16 vector in the original IEEE P1619 / D16
> > > specification in Annex B Test Vectors (last vector, "XTS-AES-128 applied for a data unit
> > > that is not a multiple of 16 bytes")
> > >
> >
> > Actually, that spec has a couple of test vectors. Unfortunately, they
> > are all rather short (except the last one in the 'no multiple of 16
> > bytes' paragraph, but unfortunately, that one is in fact a multiple of
> > 16 bytes)
> >
> > I added them here [0] along with an arm64 implementation for the AES
> > instruction based driver. Could you please double check that these
> > work against your driver?
> >
> I got XTS working with the inside-secure driver and these (and all other) vectors pass :-)
>

Excellent, thanks for the report. May I add your Tested-by when I post
the patch? (just the one that adds the test vectors)

> > That would establish a ground truth against
> > which we can implement the generic version as well.
> >
> > [0] https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=xts-cts
> >
> > > Besides that, I'd be happy to generate some testvectors from our defacto-standard
> > > implementation ;-)
> > >
> >
> > One or two larger ones would be useful, yes.
> >
> I'll see if I can extract some suitable vectors from our verification suite ...
>

Great. Once available, I'll run them against my implementations and report back.

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-25  6:22                                 ` Ard Biesheuvel
@ 2019-07-25  7:49                                   ` Pascal Van Leeuwen
  2019-07-25  8:01                                     ` Ard Biesheuvel
  0 siblings, 1 reply; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-25  7:49 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto, Horia Geanta


> -----Original Message-----
> From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Sent: Thursday, July 25, 2019 8:22 AM
> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> > >
> > > Actually, that spec has a couple of test vectors. Unfortunately, they
> > > are all rather short (except the last one in the 'no multiple of 16
> > > bytes' paragraph, but unfortunately, that one is in fact a multiple of
> > > 16 bytes)
> > >
> > > I added them here [0] along with an arm64 implementation for the AES
> > > instruction based driver. Could you please double check that these
> > > work against your driver?
> > >
> > I got XTS working with the inside-secure driver and these (and all other) vectors pass :-)
> >
> 
> Excellent, thanks for the report. May I add your Tested-by when I post
> the patch? (just the one that adds the test vectors)
> 
Sure, feel free

> > > That would establish a ground truth against
> > > which we can implement the generic version as well.
> > >
> > > [0] https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=xts-cts
> > >
> > > > Besides that, I'd be happy to generate some testvectors from our defacto-standard
> > > > implementation ;-)
> > > >
> > >
> > > One or two larger ones would be useful, yes.
> > >
> > I'll see if I can extract some suitable vectors from our verification suite ...
> >
> 
> Great. Once available, I'll run them against my implementations and report back.
>
Just wondering ... do you have any particular requirements on the sizes?
From my implementation's perspective, it doesn't make a whole lot of sense to test vectors 
of more than 3 times the cipher block size, but then I realized that you probably need
larger vectors due to the loop unrolling you do for the vector implementations?
You also don't want them to be too big as they take up space in the kernel image ...

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-25  7:49                                   ` Pascal Van Leeuwen
@ 2019-07-25  8:01                                     ` Ard Biesheuvel
  2019-07-26 10:31                                       ` Pascal Van Leeuwen
  0 siblings, 1 reply; 65+ messages in thread
From: Ard Biesheuvel @ 2019-07-25  8:01 UTC (permalink / raw)
  To: Pascal Van Leeuwen
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto, Horia Geanta

On Thu, 25 Jul 2019 at 10:49, Pascal Van Leeuwen
<pvanleeuwen@verimatrix.com> wrote:
>
>
> > -----Original Message-----
> > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > Sent: Thursday, July 25, 2019 8:22 AM
> > To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> > Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> > crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> >
> > > >
> > > > Actually, that spec has a couple of test vectors. Unfortunately, they
> > > > are all rather short (except the last one in the 'no multiple of 16
> > > > bytes' paragraph, but unfortunately, that one is in fact a multiple of
> > > > 16 bytes)
> > > >
> > > > I added them here [0] along with an arm64 implementation for the AES
> > > > instruction based driver. Could you please double check that these
> > > > work against your driver?
> > > >
> > > I got XTS working with the inside-secure driver and these (and all other) vectors pass :-)
> > >
> >
> > Excellent, thanks for the report. May I add your Tested-by when I post
> > the patch? (just the one that adds the test vectors)
> >
> Sure, feel free
>

Thanks

> > > > That would establish a ground truth against
> > > > which we can implement the generic version as well.
> > > >
> > > > [0] https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/log/?h=xts-cts
> > > >
> > > > > Besides that, I'd be happy to generate some testvectors from our defacto-standard
> > > > > implementation ;-)
> > > > >
> > > >
> > > > One or two larger ones would be useful, yes.
> > > >
> > > I'll see if I can extract some suitable vectors from our verification suite ...
> > >
> >
> > Great. Once available, I'll run them against my implementations and report back.
> >
> Just wondering ... do you have any particular requirements on the sizes?
> From my implementation's perspective, it doesn't make a whole lot of sense to test vectors
> of more than 3 times the cipher block size, but then I realized that you probably need
> larger vectors due to the loop unrolling you do for the vector implementations?
> You also don't want them to be too big as they take up space in the kernel image ...
>

We have code that operates on 1 block, 3 blocks (ARM), 4-5 blocks
(arm64) or 8 blocks (ARM,arm64) at a time. However, the most important
thing is to test the handover between the block based loop and the
epilogue that operates on the final 17-31 bytes when ciphertext
stealing is being done.

So ideally, we'd have 1 full block + 1 full/1 partial, 3 full blocks +
1 full/1 partial, and so on for 4, 5 and 8 blocks, to cover all the
code flows, but since the unrolled routines all support arbitrary
block counts (and so the handover between the multiblock and the
single block handling is already covered), just having the first two
would  be sufficient IMO.

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-25  8:01                                     ` Ard Biesheuvel
@ 2019-07-26 10:31                                       ` Pascal Van Leeuwen
  2019-07-26 19:59                                         ` Horia Geanta
       [not found]                                         ` <20f4832e-e3af-e3c2-d946-13bf8c367a60@nxp.com>
  0 siblings, 2 replies; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-26 10:31 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto, Horia Geanta

Ard,

> -----Original Message-----
> From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Sent: Thursday, July 25, 2019 10:02 AM
> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> crypto@vger.kernel.org; Horia Geanta <horia.geanta@nxp.com>
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> > > > > One or two larger ones would be useful, yes.
> > > > >
> > > > I'll see if I can extract some suitable vectors from our verification suite ...
> > > >
> > >
> > > Great. Once available, I'll run them against my implementations and report back.
> > >
> > Just wondering ... do you have any particular requirements on the sizes?
> > From my implementation's perspective, it doesn't make a whole lot of sense to test vectors
> > of more than 3 times the cipher block size, but then I realized that you probably need
> > larger vectors due to the loop unrolling you do for the vector implementations?
> > You also don't want them to be too big as they take up space in the kernel image ...
> >
> 
> We have code that operates on 1 block, 3 blocks (ARM), 4-5 blocks
> (arm64) or 8 blocks (ARM,arm64) at a time. However, the most important
> thing is to test the handover between the block based loop and the
> epilogue that operates on the final 17-31 bytes when ciphertext
> stealing is being done.
> 
> So ideally, we'd have 1 full block + 1 full/1 partial, 3 full blocks +
> 1 full/1 partial, and so on for 4, 5 and 8 blocks, to cover all the
> code flows, but since the unrolled routines all support arbitrary
> block counts (and so the handover between the multiblock and the
> single block handling is already covered), just having the first two
> would  be sufficient IMO.
>
Ok, find below a patch file that adds your vectors from the specification
plus my set of additional vectors covering all CTS alignments combined
with the block sizes you desired. Please note though that these vectors
are from our in-house home-grown model so no warranties.
They do run fine on the inside-secure driver + HW though, and I hereby
donate them to the public domain i.e. feel free to use them as you see fit.
(in case Outlook 365 messed up the patch below, it's also available from
my public Git: https://github.com/pvanleeuwen/linux-cryptodev.git,
branch is_driver_patch2)

--

This patch adds testvectors for AES-XTS that cover data inputs that are
not a multiple of 16 bytes and therefore require cipher text stealing
(CTS) to be applied. Vectors were added to cover all possible alignments
combined with various interesting (i.e. for vector implementations working
on 3,4,5 or 8 AES blocks in parallel) lengths.

Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
---
 crypto/testmgr.h | 368 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 368 insertions(+)

diff --git a/crypto/testmgr.h b/crypto/testmgr.h
index 105f2ce..1046e47 100644
--- a/crypto/testmgr.h
+++ b/crypto/testmgr.h
@@ -15594,6 +15594,374 @@ struct len_range_sel {
 			  "\xc4\xf3\x6f\xfd\xa9\xfc\xea\x70"
 			  "\xb9\xc6\xe6\x93\xe1\x48\xc1\x51",
 		.len	= 512,
+	}, { /* XTS-AES 15 */
+		.key    = "\xff\xfe\xfd\xfc\xfb\xfa\xf9\xf8"
+			  "\xf7\xf6\xf5\xf4\xf3\xf2\xf1\xf0"
+			  "\xbf\xbe\xbd\xbc\xbb\xba\xb9\xb8"
+			  "\xb7\xb6\xb5\xb4\xb3\xb2\xb1\xb0",
+		.klen   = 32,
+		.iv     = "\x9a\x78\x56\x34\x12\x00\x00\x00"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\x00\x01\x02\x03\x04\x05\x06\x07"
+			  "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f"
+			  "\x10",
+		.ctext	= "\x6c\x16\x25\xdb\x46\x71\x52\x2d"
+			  "\x3d\x75\x99\x60\x1d\xe7\xca\x09"
+			  "\xed",
+		.len	= 17,
+	}, { /* XTS-AES 16 */
+		.key    = "\xff\xfe\xfd\xfc\xfb\xfa\xf9\xf8"
+			  "\xf7\xf6\xf5\xf4\xf3\xf2\xf1\xf0"
+			  "\xbf\xbe\xbd\xbc\xbb\xba\xb9\xb8"
+			  "\xb7\xb6\xb5\xb4\xb3\xb2\xb1\xb0",
+		.klen   = 32,
+		.iv     = "\x9a\x78\x56\x34\x12\x00\x00\x00"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\x00\x01\x02\x03\x04\x05\x06\x07"
+			  "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f"
+			  "\x10\x11",
+		.ctext	= "\xd0\x69\x44\x4b\x7a\x7e\x0c\xab"
+			  "\x09\xe2\x44\x47\xd2\x4d\xeb\x1f"
+			  "\xed\xbf",
+		.len	= 18,
+	}, { /* XTS-AES 17 */
+		.key    = "\xff\xfe\xfd\xfc\xfb\xfa\xf9\xf8"
+			  "\xf7\xf6\xf5\xf4\xf3\xf2\xf1\xf0"
+			  "\xbf\xbe\xbd\xbc\xbb\xba\xb9\xb8"
+			  "\xb7\xb6\xb5\xb4\xb3\xb2\xb1\xb0",
+		.klen   = 32,
+		.iv     = "\x9a\x78\x56\x34\x12\x00\x00\x00"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\x00\x01\x02\x03\x04\x05\x06\x07"
+			  "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f"
+			  "\x10\x11\x12",
+		.ctext	= "\xe5\xdf\x13\x51\xc0\x54\x4b\xa1"
+			  "\x35\x0b\x33\x63\xcd\x8e\xf4\xbe"
+			  "\xed\xbf\x9d",
+		.len	= 19,
+	}, { /* XTS-AES 18 */
+		.key    = "\xff\xfe\xfd\xfc\xfb\xfa\xf9\xf8"
+			  "\xf7\xf6\xf5\xf4\xf3\xf2\xf1\xf0"
+			  "\xbf\xbe\xbd\xbc\xbb\xba\xb9\xb8"
+			  "\xb7\xb6\xb5\xb4\xb3\xb2\xb1\xb0",
+		.klen   = 32,
+		.iv     = "\x9a\x78\x56\x34\x12\x00\x00\x00"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\x00\x01\x02\x03\x04\x05\x06\x07"
+			  "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f"
+			  "\x10\x11\x12\x13",
+		.ctext	= "\x9d\x84\xc8\x13\xf7\x19\xaa\x2c"
+			  "\x7b\xe3\xf6\x61\x71\xc7\xc5\xc2"
+			  "\xed\xbf\x9d\xac",
+		.len	= 20,
+	/* Additional vectors to increase CTS coverage */
+	}, { /* 1 block + 21 bytes */
+		.key    = "\xa1\x34\x0e\x49\x38\xfd\x8b\xf6"
+			  "\x45\x60\x67\x07\x0f\x50\xa8\x2b"
+			  "\xa8\xf1\xfe\x7e\xf4\xf0\x47\xcd"
+			  "\xfd\x91\x78\xf9\x14\x8b\x7d\x27"
+			  "\x0e\xdc\xca\xe6\xf4\xfc\xd7\x4f"
+			  "\x19\x8c\xd0\xe6\x9e\x2f\xf8\x75"
+			  "\xb5\xe2\x48\x00\x4f\x07\xd9\xa1"
+			  "\x42\xbc\x9d\xfc\x17\x98\x00\x48",
+		.klen   = 64,
+		.iv     = "\xcb\x35\x47\x5a\x7a\x06\x28\xb9"
+			  "\x80\xf5\xa7\xe6\x8a\x23\x42\xf8",
+		.ptext	= "\x04\x52\xc8\x7f\xb0\x5a\x12\xc5"
+			  "\x96\x47\x6b\xf4\xbc\x2e\xdb\x74"
+			  "\xd2\x20\x24\x32\xe5\x84\xb6\x25"
+			  "\x4c\x2f\x96\xc7\x55\x9c\x90\x6f"
+			  "\x0e\x96\x94\x68\xf4",
+		.ctext	= "\x6a\x2d\x57\xb8\x72\x49\x10\x6b"
+			  "\x5b\x5a\xc9\x92\xab\x59\x79\x36"
+			  "\x7a\x01\x95\xf7\xdd\xcb\x3f\xbf"
+			  "\xb2\xe3\x7e\x35\xe3\x11\x04\x68"
+			  "\x28\xc3\x70\x6a\xe1",
+		.len	= 37,
+	}, { /* 3 blocks + 22 bytes */
+		.key    = "\xf7\x87\x75\xdf\x36\x20\xe7\xcb"
+			  "\x20\x5d\x49\x96\x81\x3d\x1d\x80"
+			  "\xc7\x18\x7e\xbf\x2a\x0f\x79\xba"
+			  "\x06\xb5\x4b\x63\x03\xfb\xb8\x49"
+			  "\x93\x2d\x85\x5b\x95\x1f\x78\xea"
+			  "\x7c\x1e\xf5\x5d\x02\xc6\xec\xb0"
+			  "\xf0\xaa\x3d\x0a\x04\xe1\x67\x80"
+			  "\x2a\xbe\x4e\x73\xc9\x11\xcc\x6c",
+		.klen   = 64,
+		.iv     = "\xeb\xba\x55\x24\xfc\x8f\x25\x7c"
+			  "\x66\xf9\x04\x03\xcc\xb1\xf4\x84",
+		.ptext	= "\x40\x75\x1b\x72\x2a\xc8\xbf\xef"
+			  "\x0c\x92\x3e\x19\xc5\x09\x07\x38"
+			  "\x4d\x87\x5c\xb8\xd6\x4f\x1a\x39"
+			  "\x8c\xee\xa5\x22\x41\x12\xe1\x22"
+			  "\xb5\x4b\xd7\xeb\x02\xfa\xaa\xf8"
+			  "\x94\x47\x04\x5d\x8a\xb5\x40\x12"
+			  "\x04\x62\x3d\xe4\x19\x8a\xeb\xb3"
+			  "\xf9\xa3\x7d\xb6\xeb\x57\xf9\xb8"
+			  "\x7f\xa8\xfa\x2d\x75\x2d",
+		.ctext	= "\x46\x6d\xe5\x35\x5d\x22\x42\x33"
+			  "\xf7\xb8\xfb\xc0\xcb\x18\xad\xa4"
+			  "\x75\x6c\xc6\x38\xbb\xd4\xa1\x32"
+			  "\x00\x05\x06\xd9\xc9\x17\xd9\x4f"
+			  "\x1a\xf6\x24\x64\x27\x8a\x4a\xad"
+			  "\x88\xa0\x86\xb7\xf9\x33\xaf\xa8"
+			  "\x0e\x83\xd8\x0e\x88\xa2\x81\x79"
+			  "\x65\x2e\x3e\x84\xaf\xa1\x46\x7d"
+			  "\xa9\x91\xf8\x17\x82\x8d",
+		.len	= 70,
+	}, { /* 4 blocks + 23 bytes */
+		.key    = "\x48\x09\xab\x48\xd6\xca\x7d\xb1"
+			  "\x90\xa0\x00\xd8\x33\x8a\x20\x79"
+			  "\x7c\xbc\x0c\x0c\x5f\x41\xbc\xbc"
+			  "\x82\xaf\x41\x81\x23\x93\xcb\xc7"
+			  "\x61\x7b\x83\x13\x16\xb1\x3e\x7c"
+			  "\xcc\xae\xda\xca\x78\xc7\xab\x18"
+			  "\x69\xb6\x58\x3e\x5c\x19\x5f\xed"
+			  "\x7b\xcf\x70\xb9\x76\x00\xd8\xc9",
+		.klen   = 64,
+		.iv     = "\x2e\x20\x36\xf4\xa3\x22\x5d\xd8"
+			  "\x38\x49\x82\xbf\x6c\x56\xd9\x3b",
+		.ptext	= "\x79\x3c\x73\x99\x65\x21\xe1\xb9"
+			  "\xa0\xfd\x22\xb2\x57\xc0\x7f\xf4"
+			  "\x7f\x97\x36\xaf\xf8\x8d\x73\xe1"
+			  "\x0d\x85\xe9\xd5\x3d\x82\xb3\x49"
+			  "\x89\x25\x30\x1f\x0d\xca\x5c\x95"
+			  "\x64\x31\x02\x17\x11\x08\x8f\x32"
+			  "\xbc\x37\x23\x4f\x03\x98\x91\x4a"
+			  "\x50\xe2\x58\xa8\x9b\x64\x09\xe0"
+			  "\xce\x99\xc9\xb0\xa8\x21\x73\xb7"
+			  "\x2d\x4b\x19\xba\x81\x83\x99\xce"
+			  "\xa0\x7a\xd0\x9f\x27\xf6\x8a",
+		.ctext	= "\xf9\x12\x76\x21\x06\x1e\xe4\x4b"
+			  "\xf9\x94\x38\x29\x0f\xee\xcb\x13"
+			  "\xa3\xc3\x50\xe3\xc6\x29\x9d\xcf"
+			  "\x6f\x6a\x0a\x25\xab\x44\xf6\xe4"
+			  "\x71\x29\x75\x3b\x07\x1c\xfc\x1a"
+			  "\x75\xd4\x84\x58\x7f\xc4\xf3\xf7"
+			  "\x8f\x7c\x7a\xdc\xa2\xa3\x95\x38"
+			  "\x15\xdf\x3b\x9c\xdd\x24\xb4\x0b"
+			  "\xa8\x97\xfa\x5f\xee\x58\x00\x0d"
+			  "\x23\xc9\x8d\xee\xc2\x3f\x27\xd8"
+			  "\xd4\x43\xa5\xf8\x25\x71\x3f",
+		.len	= 87,
+	}, { /* 5 blocks + 24 bytes */
+		.key    = "\x8c\xf4\x4c\xe5\x91\x8f\x72\xe9"
+			  "\x2f\xf8\xc0\x3c\x87\x76\x16\xa4"
+			  "\x20\xab\x66\x39\x34\x10\xd6\x91"
+			  "\xf1\x99\x2c\xf1\xd6\xc3\xda\x38"
+			  "\xed\x2a\x4c\x80\xf4\xa5\x56\x28"
+			  "\x1a\x1c\x79\x72\x6c\x93\x08\x86"
+			  "\x8f\x8a\xaa\xcd\xf1\x8c\xca\xe7"
+			  "\x0a\xe8\xee\x0c\x1c\xc2\xa8\xea",
+		.klen   = 64,
+		.iv     = "\x9a\x9e\xbc\xe4\xc9\xf3\xef\x9f"
+			  "\xff\x82\x0e\x22\x8f\x80\x42\x76",
+		.ptext	= "\xc1\xde\x66\x1a\x7e\x60\xd3\x3b"
+			  "\x66\xd6\x29\x86\x99\xc6\xd7\xc8"
+			  "\x29\xbf\x00\x57\xab\x21\x06\x24"
+			  "\xd0\x92\xef\xe6\xb5\x1e\x20\xb9"
+			  "\xb7\x7b\xd7\x18\x88\xf8\xd7\xe3"
+			  "\x90\x61\xcd\x73\x2b\xa1\xb5\xc7"
+			  "\x33\xef\xb5\xf2\x45\xf6\x92\x53"
+			  "\x91\x98\xf8\x5a\x20\x75\x4c\xa8"
+			  "\xf1\xf6\x01\x26\xbc\xba\x4c\xac"
+			  "\xcb\xc2\x6d\xb6\x2c\x3c\x38\x61"
+			  "\xe3\x98\x7f\x3e\x98\xbd\xec\xce"
+			  "\xc0\xb5\x74\x23\x43\x24\x7b\x7e"
+			  "\x3f\xed\xcb\xda\x88\x67\x6f\x9a",
+		.ctext	= "\xeb\xdc\x6a\xb7\xd9\x5f\xa7\xfc"
+			  "\x48\x75\x10\xef\xca\x65\xdc\x88"
+			  "\xd0\x23\xde\x17\x5f\x3b\x61\xa2"
+			  "\x15\x13\x81\x81\xf8\x57\x8b\x2a"
+			  "\xe2\xc8\x49\xd1\xba\xed\xd6\xcb"
+			  "\xed\x6f\x26\x69\x9b\xd2\xd2\x91"
+			  "\x4e\xd7\x81\x20\x66\x38\x0c\x62"
+			  "\x60\xcd\x01\x36\x97\x22\xf0\x5c"
+			  "\xcf\x53\xc6\x58\xf5\x8b\x48\x0c"
+			  "\xa5\x50\xc2\x73\xf9\x70\x60\x09"
+			  "\x22\x69\xf3\x71\x74\x5d\xc9\xa0"
+			  "\x9c\x79\xf9\xc4\x87\xac\xd7\x4b"
+			  "\xac\x3c\xc6\xda\x81\x7a\xdd\x14",
+		.len	= 104,
+	}, { /* 8 blocks + 25 bytes */
+		.key    = "\x70\x18\x09\x93\x10\x3a\x0c\xa9"
+			  "\x02\x0b\x11\x10\xae\x34\x98\xdb"
+			  "\x10\xb5\xee\x8c\x49\xbc\x52\x8e"
+			  "\x4b\xf7\x0a\x36\x16\x8a\xf7\x06"
+			  "\xb5\x94\x52\x54\xb9\xc1\x4d\x20"
+			  "\xa2\xf0\x6e\x19\x7f\x67\x1e\xaa"
+			  "\x94\x6c\xee\x54\x19\xfc\x96\x95"
+			  "\x04\x85\x00\x53\x7c\x39\x5f\xeb",
+		.klen   = 64,
+		.iv     = "\x36\x87\x8f\x9d\x74\xe9\x52\xfb"
+			  "\xe1\x76\x16\x99\x61\x86\xec\x8f",
+		.ptext	= "\x95\x08\xee\xfe\x87\xb2\x4f\x93"
+			  "\x01\xee\xf3\x77\x0d\xbb\xfb\x26"
+			  "\x3e\xb3\x34\x20\xee\x51\xd6\x40"
+			  "\xb1\x64\xae\xd9\xfd\x71\x8f\x93"
+			  "\xa5\x85\xff\x74\xcc\xd3\xfd\x5e"
+			  "\xc2\xfc\x49\xda\xa8\x3a\x94\x29"
+			  "\xa2\x59\x90\x34\x26\xbb\xa0\x34"
+			  "\x5d\x47\x33\xf2\xa8\x77\x90\x98"
+			  "\x8d\xfd\x38\x60\x23\x1e\x50\xa1"
+			  "\x67\x4d\x8d\x09\xe0\x7d\x30\xe3"
+			  "\xdd\x39\x91\xd4\x70\x68\xbb\x06"
+			  "\x4e\x11\xb2\x26\x0a\x85\x73\xf6"
+			  "\x37\xb6\x15\xd0\x77\xee\x43\x7b"
+			  "\x77\x13\xe9\xb9\x84\x2b\x34\xab"
+			  "\x49\xc1\x27\x91\x2e\xa3\xca\xe5"
+			  "\xa7\x79\x45\xba\x36\x97\x49\x44"
+			  "\xf7\x57\x9b\xd7\xac\xb3\xfd\x6a"
+			  "\x1c\xd1\xfc\x1c\xdf\x6f\x94\xac"
+			  "\x95\xf4\x50\x7a\xc8\xc3\x8c\x60"
+			  "\x3c",
+		.ctext	= "\xb6\xc8\xf9\x5d\x35\x5a\x0a\x33"
+			  "\x2b\xd3\x5a\x18\x09\x1c\x1b\x0b"
+			  "\x2a\x0e\xde\xf6\x0d\x04\xa6\xb3"
+			  "\xa8\xe8\x1b\x86\x29\x58\x75\x56"
+			  "\xab\xab\xbf\xbe\x1f\xb4\xc4\xf3"
+			  "\xde\x1a\xb0\x87\x69\xac\x5b\x0c"
+			  "\x1b\xb7\xc7\x24\xa4\x47\xe7\x81"
+			  "\x2c\x0a\x82\xf9\x18\x5d\xe6\x09"
+			  "\xe3\x65\x36\x54\x3d\x8a\x3a\x64"
+			  "\x34\xf4\x34\x7f\x26\x3c\x1e\x3b"
+			  "\x5a\x13\xdf\x7f\xa8\x2d\x81\xce"
+			  "\xfa\xad\xd0\xb1\xca\xfa\xc3\x55"
+			  "\x94\xc8\xb8\x16\x7e\xff\x44\x88"
+			  "\xb4\x47\x4b\xfe\xda\x60\x68\x2e"
+			  "\xfc\x70\xb5\xe3\xf3\xe9\x46\x22"
+			  "\x1d\x98\x66\x09\x0f\xed\xbb\x20"
+			  "\x7b\x8c\x2a\xff\x45\x62\xde\x9b"
+			  "\x20\x2e\x6c\xb4\xe4\x26\x03\x72"
+			  "\x8a\xb4\x19\xc9\xb1\xcf\x9d\x86"
+			  "\xa3",
+		.len	= 153,
+	}, { /* 0 blocks + 26 bytes */
+		.key    = "\x5a\x38\x3f\x9c\x0c\x53\x17\x6c"
+			  "\x60\x72\x23\x26\xba\xfe\xa1\xb7"
+			  "\x03\xa8\xfe\xa0\x7c\xff\x78\x4c"
+			  "\x7d\x84\x2f\x24\x84\x77\xec\x6f"
+			  "\x88\xc8\x36\xe2\xcb\x52\x3c\xb4"
+			  "\x39\xac\x37\xfa\x41\x8b\xc4\x59"
+			  "\x24\x03\xe1\x51\xc9\x54\x7d\xb7"
+			  "\xa3\xde\x91\x44\x8d\x16\x97\x22",
+		.klen   = 64,
+		.iv     = "\xfb\x7f\x3d\x60\x26\x0a\x3a\x3d"
+			  "\xa5\xa3\x45\xf2\x24\x67\xfa\x6e",
+		.ptext	= "\xfb\x56\x97\x65\x7c\xd8\x6c\x3c"
+			  "\x5d\xd3\xea\xa6\xa4\x83\xf7\x9d"
+			  "\x9d\x89\x2c\x85\xb8\xd9\xd4\xf0"
+			  "\x1a\xad",
+		.ctext	= "\xc9\x9b\x4b\xf2\xf7\x0f\x23\xfe"
+			  "\xc3\x93\x88\xa1\xb3\x88\xab\xd6"
+			  "\x26\x78\x82\xa6\x6b\x0b\x76\xad"
+			  "\x21\x5e",
+		.len	= 26,
+	}, { /* 0 blocks + 27 bytes */
+		.key    = "\xc0\xcf\x57\xa2\x3c\xa2\x4b\xf6"
+			  "\x5d\x36\x7b\xd7\x1d\x16\xc3\x2f"
+			  "\x50\xc6\x0a\xb2\xfd\xe8\x24\xfc"
+			  "\x33\xcf\x73\xfd\xe0\xe9\xa5\xd1"
+			  "\x98\xfc\xd6\x16\xdd\xfd\x6d\xab"
+			  "\x44\xbc\x37\x9d\xab\x5b\x1d\xf2"
+			  "\x6f\x5d\xbe\x6b\x14\x14\xc7\x74"
+			  "\xbb\x91\x24\x4b\x52\xcb\x78\x31",
+		.klen   = 64,
+		.iv     = "\x5c\xc1\x3d\xb6\xa1\x6a\x2d\x1f"
+			  "\xee\x75\x19\x4b\x04\xfa\xe1\x7e",
+		.ptext	= "\x02\x95\x3a\xab\xac\x3b\xcd\xcd"
+			  "\x63\xc7\x4c\x7c\xe5\x75\xee\x03"
+			  "\x94\xc7\xff\xe8\xe0\xe9\x86\x2a"
+			  "\xd3\xc7\xe4",
+		.ctext	= "\x8e\x84\x76\x8b\xc1\x47\x55\x15"
+			  "\x5e\x51\xb3\xe2\x3f\x72\x4d\x20"
+			  "\x09\x3f\x4f\xb1\xce\xf4\xb0\x14"
+			  "\xf6\xa7\xb3",
+		.len	= 27,
+	}, { /* 0 blocks + 28 bytes */
+		.key    = "\x0b\x5b\x1d\xc8\xb1\x3f\x8f\xcd"
+			  "\x87\xd2\x58\x28\x36\xc6\x34\xfb"
+			  "\x04\xe8\xf1\xb7\x91\x30\xda\x75"
+			  "\x66\x4a\x72\x90\x09\x39\x02\x19"
+			  "\x62\x2d\xe9\x24\x95\x0e\x87\x43"
+			  "\x4c\xc7\x96\xe4\xc9\x31\x6a\x13"
+			  "\x16\x10\xef\x34\x9b\x98\x19\xf1"
+			  "\x8b\x14\x38\x3f\xf8\x75\xcc\x76",
+		.klen   = 64,
+		.iv     = "\x0c\x2c\x55\x2c\xda\x40\xe1\xab"
+			  "\xa6\x34\x66\x7a\xa4\xa3\xda\x90",
+		.ptext	= "\xbe\x84\xd3\xfe\xe6\xb4\x29\x67"
+			  "\xfd\x29\x78\x41\x3d\xe9\x81\x4e"
+			  "\x3c\xf9\xf4\xf5\x3f\xd8\x0e\xcd"
+			  "\x63\x73\x65\xf3",
+		.ctext	= "\xd0\xa0\x16\x5f\xf9\x85\xd0\x63"
+			  "\x9b\x81\xa1\x15\x93\xb3\x62\x36"
+			  "\xec\x93\x0e\x14\x07\xf2\xa9\x38"
+			  "\x80\x33\xc0\x20",
+		.len	= 28,
+	}, { /* 0 blocks + 29 bytes */
+		.key    = "\xdc\x4c\xdc\x20\xb1\x34\x89\xa4"
+			  "\xd0\xb6\x77\x05\xea\x0c\xcc\x68"
+			  "\xb1\xd6\xf7\xfd\xa7\x0a\x5b\x81"
+			  "\x2d\x4d\xa3\x65\xd0\xab\xa1\x02"
+			  "\x85\x4b\x33\xea\x51\x16\x50\x12"
+			  "\x3b\x25\xba\x13\xba\x7c\xbb\x3a"
+			  "\xe4\xfd\xb3\x9c\x88\x8b\xb8\x30"
+			  "\x7a\x97\xcf\x95\x5d\x69\x7b\x1d",
+		.klen   = 64,
+		.iv     = "\xe7\x69\xed\xd2\x54\x5d\x4a\x29"
+			  "\xb2\xd7\x60\x90\xa0\x0b\x0d\x3a",
+		.ptext	= "\x37\x22\x11\x62\xa0\x74\x92\x62"
+			  "\x40\x4e\x2b\x0a\x8b\xab\xd8\x28"
+			  "\x8a\xd2\xeb\xa5\x8e\xe1\x42\xc8"
+			  "\x49\xef\x9a\xec\x1b",
+		.ctext	= "\x7c\x66\x72\x6b\xe3\xc3\x57\x71"
+			  "\x37\x13\xce\x1f\x6b\xff\x13\x87"
+			  "\x65\xa7\xa1\xc5\x23\x7f\xca\x40"
+			  "\x82\xbf\x2f\xc0\x2a",
+		.len	= 29,
+	}, { /* 0 blocks + 30 bytes */
+		.key    = "\x72\x9a\xf5\x53\x55\xdd\x0f\xef"
+			  "\xfc\x75\x6f\x03\x88\xc8\xba\x88"
+			  "\xb7\x65\x89\x5d\x03\x86\x21\x22"
+			  "\xb8\x42\x87\xd9\xa9\x83\x9e\x9c"
+			  "\xca\x28\xa1\xd2\xb6\xd0\xa6\x6c"
+			  "\xf8\x57\x42\x7c\x73\xfc\x7b\x0a"
+			  "\xbc\x3c\x57\x7b\x5a\x39\x61\x55"
+			  "\xb7\x25\xe9\xf1\xc4\xbb\x04\x28",
+		.klen   = 64,
+		.iv     = "\x8a\x38\x22\xba\xea\x5e\x1d\xa4"
+			  "\x31\x18\x12\x5c\x56\x0c\x12\x50",
+		.ptext	= "\x06\xfd\xbb\xa9\x2e\x56\x05\x5f"
+			  "\xf2\xa7\x36\x76\x26\xd3\xb3\x49"
+			  "\x7c\xe2\xe3\xbe\x1f\x65\xd2\x17"
+			  "\x65\xe2\xb3\x0e\xb1\x93",
+		.ctext	= "\xae\x1f\x19\x7e\x3b\xb3\x65\xcb"
+			  "\x14\x70\x6b\x3c\xa0\x63\x95\x94"
+			  "\x56\x52\xe1\xb4\x14\xca\x21\x13"
+			  "\xb5\x03\x3f\xfe\xc9\x9f",
+		.len	= 30,
+	}, { /* 0 blocks + 31 bytes */
+		.key    = "\xce\x06\x45\x53\x25\x81\xd2\xb2"
+			  "\xdd\xc9\x57\xfe\xbb\xf6\x83\x07"
+			  "\x28\xd8\x2a\xff\x53\xf8\x57\xc6"
+			  "\x63\x50\xd4\x3e\x2a\x54\x37\x51"
+			  "\x07\x3b\x23\x63\x3c\x31\x57\x0d"
+			  "\xd3\x59\x20\xf2\xd0\x85\xac\xc5"
+			  "\x3f\xa1\x74\x90\x0a\x3f\xf4\x10"
+			  "\x12\xf0\x1b\x2b\xef\xcb\x86\x74",
+		.klen   = 64,
+		.iv     = "\x6d\x3e\x62\x94\x75\x43\x74\xea"
+			  "\xed\x4a\xa6\xde\xba\x55\x83\x38",
+		.ptext	= "\x6a\xe6\xa3\x66\x7e\x78\xef\x42"
+			  "\x8b\x28\x08\x24\xda\xd4\xd6\x42"
+			  "\x3d\xb6\x48\x7e\x51\xa6\x92\x65"
+			  "\x98\x86\x26\x98\x37\x42\xa5",
+		.ctext	= "\x64\xc6\xfc\x60\x21\x87\x7a\xf5"
+			  "\xc3\x1d\xba\x41\x3c\x9c\x8c\xe8"
+			  "\x2d\x93\xf0\x02\x95\x6d\xfe\x8d"
+			  "\x68\x17\x05\x75\xc0\xd3\xa8",
+		.len	= 31,
 	}
 };
 
-- 
1.8.3.1

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com


^ permalink raw reply related	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-26 10:31                                       ` Pascal Van Leeuwen
@ 2019-07-26 19:59                                         ` Horia Geanta
  2019-07-26 21:43                                           ` Pascal Van Leeuwen
       [not found]                                         ` <20f4832e-e3af-e3c2-d946-13bf8c367a60@nxp.com>
  1 sibling, 1 reply; 65+ messages in thread
From: Horia Geanta @ 2019-07-26 19:59 UTC (permalink / raw)
  To: Pascal Van Leeuwen, Ard Biesheuvel
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto

On 7/26/2019 1:31 PM, Pascal Van Leeuwen wrote:
> Ok, find below a patch file that adds your vectors from the specification
> plus my set of additional vectors covering all CTS alignments combined
> with the block sizes you desired. Please note though that these vectors
> are from our in-house home-grown model so no warranties.
I've checked the test vectors against caam (HW + driver).

Test vectors from IEEE 1619-2007 (i.e. up to and including "XTS-AES 18")
are fine.

caam complains when /* Additional vectors to increase CTS coverage */
section starts:
alg: skcipher: xts-aes-caam encryption test failed (wrong result) on test vector 9, cfg="in-place"

(Unfortunately it seems that testmgr lost the capability of dumping
the incorrect output.)

IMO we can't rely on test vectors if they are not taken
straight out of a spec, or cross-checked with other implementations.

Horia

> They do run fine on the inside-secure driver + HW though, and I hereby
> donate them to the public domain i.e. feel free to use them as you see fit.
> (in case Outlook 365 messed up the patch below, it's also available from
> my public Git: https://github.com/pvanleeuwen/linux-cryptodev.git,
> branch is_driver_patch2)
> 
> --
> 
> This patch adds testvectors for AES-XTS that cover data inputs that are
> not a multiple of 16 bytes and therefore require cipher text stealing
> (CTS) to be applied. Vectors were added to cover all possible alignments
> combined with various interesting (i.e. for vector implementations working
> on 3,4,5 or 8 AES blocks in parallel) lengths.
> 
> Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
> ---
>  crypto/testmgr.h | 368 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 368 insertions(+)
> 
> diff --git a/crypto/testmgr.h b/crypto/testmgr.h
> index 105f2ce..1046e47 100644
> --- a/crypto/testmgr.h
> +++ b/crypto/testmgr.h
> @@ -15594,6 +15594,374 @@ struct len_range_sel {
>  			  "\xc4\xf3\x6f\xfd\xa9\xfc\xea\x70"
>  			  "\xb9\xc6\xe6\x93\xe1\x48\xc1\x51",
>  		.len	= 512,
> +	}, { /* XTS-AES 15 */
> +		.key    = "\xff\xfe\xfd\xfc\xfb\xfa\xf9\xf8"
> +			  "\xf7\xf6\xf5\xf4\xf3\xf2\xf1\xf0"
> +			  "\xbf\xbe\xbd\xbc\xbb\xba\xb9\xb8"
> +			  "\xb7\xb6\xb5\xb4\xb3\xb2\xb1\xb0",
> +		.klen   = 32,
> +		.iv     = "\x9a\x78\x56\x34\x12\x00\x00\x00"
> +			  "\x00\x00\x00\x00\x00\x00\x00\x00",
> +		.ptext	= "\x00\x01\x02\x03\x04\x05\x06\x07"
> +			  "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f"
> +			  "\x10",
> +		.ctext	= "\x6c\x16\x25\xdb\x46\x71\x52\x2d"
> +			  "\x3d\x75\x99\x60\x1d\xe7\xca\x09"
> +			  "\xed",
> +		.len	= 17,
[...]
> +	}, { /* XTS-AES 18 */
> +		.key    = "\xff\xfe\xfd\xfc\xfb\xfa\xf9\xf8"
> +			  "\xf7\xf6\xf5\xf4\xf3\xf2\xf1\xf0"
> +			  "\xbf\xbe\xbd\xbc\xbb\xba\xb9\xb8"
> +			  "\xb7\xb6\xb5\xb4\xb3\xb2\xb1\xb0",
> +		.klen   = 32,
> +		.iv     = "\x9a\x78\x56\x34\x12\x00\x00\x00"
> +			  "\x00\x00\x00\x00\x00\x00\x00\x00",
> +		.ptext	= "\x00\x01\x02\x03\x04\x05\x06\x07"
> +			  "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f"
> +			  "\x10\x11\x12\x13",
> +		.ctext	= "\x9d\x84\xc8\x13\xf7\x19\xaa\x2c"
> +			  "\x7b\xe3\xf6\x61\x71\xc7\xc5\xc2"
> +			  "\xed\xbf\x9d\xac",
> +		.len	= 20,
> +	/* Additional vectors to increase CTS coverage */
> +	}, { /* 1 block + 21 bytes */
> +		.key    = "\xa1\x34\x0e\x49\x38\xfd\x8b\xf6"
> +			  "\x45\x60\x67\x07\x0f\x50\xa8\x2b"
> +			  "\xa8\xf1\xfe\x7e\xf4\xf0\x47\xcd"
> +			  "\xfd\x91\x78\xf9\x14\x8b\x7d\x27"
> +			  "\x0e\xdc\xca\xe6\xf4\xfc\xd7\x4f"
> +			  "\x19\x8c\xd0\xe6\x9e\x2f\xf8\x75"
> +			  "\xb5\xe2\x48\x00\x4f\x07\xd9\xa1"
> +			  "\x42\xbc\x9d\xfc\x17\x98\x00\x48",
> +		.klen   = 64,
> +		.iv     = "\xcb\x35\x47\x5a\x7a\x06\x28\xb9"
> +			  "\x80\xf5\xa7\xe6\x8a\x23\x42\xf8",
> +		.ptext	= "\x04\x52\xc8\x7f\xb0\x5a\x12\xc5"
> +			  "\x96\x47\x6b\xf4\xbc\x2e\xdb\x74"
> +			  "\xd2\x20\x24\x32\xe5\x84\xb6\x25"
> +			  "\x4c\x2f\x96\xc7\x55\x9c\x90\x6f"
> +			  "\x0e\x96\x94\x68\xf4",
> +		.ctext	= "\x6a\x2d\x57\xb8\x72\x49\x10\x6b"
> +			  "\x5b\x5a\xc9\x92\xab\x59\x79\x36"
> +			  "\x7a\x01\x95\xf7\xdd\xcb\x3f\xbf"
> +			  "\xb2\xe3\x7e\x35\xe3\x11\x04\x68"
> +			  "\x28\xc3\x70\x6a\xe1",
> +		.len	= 37,
> +	}, { /* 3 blocks + 22 bytes */
[...]

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-26 19:59                                         ` Horia Geanta
@ 2019-07-26 21:43                                           ` Pascal Van Leeuwen
  2019-07-27  5:39                                             ` Ard Biesheuvel
  0 siblings, 1 reply; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-26 21:43 UTC (permalink / raw)
  To: Horia Geanta, Ard Biesheuvel
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto

> -----Original Message-----
> From: Horia Geanta <horia.geanta@nxp.com>
> Sent: Friday, July 26, 2019 9:59 PM
> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> crypto@vger.kernel.org
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> On 7/26/2019 1:31 PM, Pascal Van Leeuwen wrote:
> > Ok, find below a patch file that adds your vectors from the specification
> > plus my set of additional vectors covering all CTS alignments combined
> > with the block sizes you desired. Please note though that these vectors
> > are from our in-house home-grown model so no warranties.
> I've checked the test vectors against caam (HW + driver).
> 
> Test vectors from IEEE 1619-2007 (i.e. up to and including "XTS-AES 18")
> are fine.
> 
> caam complains when /* Additional vectors to increase CTS coverage */
> section starts:
> alg: skcipher: xts-aes-caam encryption test failed (wrong result) on test vector 9, cfg="in-place"
> 
> (Unfortunately it seems that testmgr lost the capability of dumping
> the incorrect output.)
> 
> IMO we can't rely on test vectors if they are not taken
> straight out of a spec, or cross-checked with other implementations.
> 

First off, I fully agree with your statement, which is why I did not post this as a straight
patch. The problem is that specification vectors usually (or actuaclly, always) don't cover
all the relevant corner cases needed for verification. And "reference" implementations 
by academics are usually shady at best as well.

In this particular case, the reference vectors only cover 5 out of 16 possible alignment
cases and the current situation proves that this is not sufficient. As we have 2 imple-
mentations (or actually more, if you count the models used for vector generation)
that are considered to be correct that disagree on results.

Which is very interesting, because which one is correct? I know that our model and
hardware implementation were independently developed (by 2 different engineers)
from the IEEE spec and match on results. And our hardware has been used "out in
the field" for many years already in disk controllers from major silicon vendors.
But that's still not a guarantee .... So how do we resolve this? Majority vote? ;-)

> Horia
> 

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com


^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-26 21:43                                           ` Pascal Van Leeuwen
@ 2019-07-27  5:39                                             ` Ard Biesheuvel
  2019-07-27 12:56                                               ` Pascal Van Leeuwen
  2019-07-27 16:04                                               ` Milan Broz
  0 siblings, 2 replies; 65+ messages in thread
From: Ard Biesheuvel @ 2019-07-27  5:39 UTC (permalink / raw)
  To: Pascal Van Leeuwen
  Cc: Horia Geanta, Milan Broz, Herbert Xu, dm-devel, linux-crypto

On Sat, 27 Jul 2019 at 00:43, Pascal Van Leeuwen
<pvanleeuwen@verimatrix.com> wrote:
>
> > -----Original Message-----
> > From: Horia Geanta <horia.geanta@nxp.com>
> > Sent: Friday, July 26, 2019 9:59 PM
> > To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> > crypto@vger.kernel.org
> > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> >
> > On 7/26/2019 1:31 PM, Pascal Van Leeuwen wrote:
> > > Ok, find below a patch file that adds your vectors from the specification
> > > plus my set of additional vectors covering all CTS alignments combined
> > > with the block sizes you desired. Please note though that these vectors
> > > are from our in-house home-grown model so no warranties.
> > I've checked the test vectors against caam (HW + driver).
> >
> > Test vectors from IEEE 1619-2007 (i.e. up to and including "XTS-AES 18")
> > are fine.
> >
> > caam complains when /* Additional vectors to increase CTS coverage */
> > section starts:
> > alg: skcipher: xts-aes-caam encryption test failed (wrong result) on test vector 9, cfg="in-place"
> >
> > (Unfortunately it seems that testmgr lost the capability of dumping
> > the incorrect output.)
> >
> > IMO we can't rely on test vectors if they are not taken
> > straight out of a spec, or cross-checked with other implementations.
> >
>
> First off, I fully agree with your statement, which is why I did not post this as a straight
> patch. The problem is that specification vectors usually (or actuaclly, always) don't cover
> all the relevant corner cases needed for verification. And "reference" implementations
> by academics are usually shady at best as well.
>
> In this particular case, the reference vectors only cover 5 out of 16 possible alignment
> cases and the current situation proves that this is not sufficient. As we have 2 imple-
> mentations (or actually more, if you count the models used for vector generation)
> that are considered to be correct that disagree on results.
>
> Which is very interesting, because which one is correct? I know that our model and
> hardware implementation were independently developed (by 2 different engineers)
> from the IEEE spec and match on results. And our hardware has been used "out in
> the field" for many years already in disk controllers from major silicon vendors.
> But that's still not a guarantee .... So how do we resolve this? Majority vote? ;-)
>

Thanks for the additional test vectors. They work fine with my SIMD
implementations for ARM [0], so this looks like it might be a CAAM
problem, not a problem with the test vectors.

I will try to find some time today to run them through OpenSSL to double check.


[0] https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/commit/?h=xts-cts

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-27  5:39                                             ` Ard Biesheuvel
@ 2019-07-27 12:56                                               ` Pascal Van Leeuwen
  2019-07-27 16:04                                               ` Milan Broz
  1 sibling, 0 replies; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-07-27 12:56 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Horia Geanta, Milan Broz, Herbert Xu, dm-devel, linux-crypto


> -----Original Message-----
> From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Sent: Saturday, July 27, 2019 7:39 AM
> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> Cc: Horia Geanta <horia.geanta@nxp.com>; Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-
> devel@redhat.com; linux-crypto@vger.kernel.org
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> On Sat, 27 Jul 2019 at 00:43, Pascal Van Leeuwen
> <pvanleeuwen@verimatrix.com> wrote:
> >
> > > -----Original Message-----
> > > From: Horia Geanta <horia.geanta@nxp.com>
> > > Sent: Friday, July 26, 2019 9:59 PM
> > > To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-devel@redhat.com; linux-
> > > crypto@vger.kernel.org
> > > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> > >
> > > On 7/26/2019 1:31 PM, Pascal Van Leeuwen wrote:
> > > > Ok, find below a patch file that adds your vectors from the specification
> > > > plus my set of additional vectors covering all CTS alignments combined
> > > > with the block sizes you desired. Please note though that these vectors
> > > > are from our in-house home-grown model so no warranties.
> > > I've checked the test vectors against caam (HW + driver).
> > >
> > > Test vectors from IEEE 1619-2007 (i.e. up to and including "XTS-AES 18")
> > > are fine.
> > >
> > > caam complains when /* Additional vectors to increase CTS coverage */
> > > section starts:
> > > alg: skcipher: xts-aes-caam encryption test failed (wrong result) on test vector 9, cfg="in-place"
> > >
> > > (Unfortunately it seems that testmgr lost the capability of dumping
> > > the incorrect output.)
> > >
> > > IMO we can't rely on test vectors if they are not taken
> > > straight out of a spec, or cross-checked with other implementations.
> > >
> >
> > First off, I fully agree with your statement, which is why I did not post this as a straight
> > patch. The problem is that specification vectors usually (or actuaclly, always) don't cover
> > all the relevant corner cases needed for verification. And "reference" implementations
> > by academics are usually shady at best as well.
> >
> > In this particular case, the reference vectors only cover 5 out of 16 possible alignment
> > cases and the current situation proves that this is not sufficient. As we have 2 imple-
> > mentations (or actually more, if you count the models used for vector generation)
> > that are considered to be correct that disagree on results.
> >
> > Which is very interesting, because which one is correct? I know that our model and
> > hardware implementation were independently developed (by 2 different engineers)
> > from the IEEE spec and match on results. And our hardware has been used "out in
> > the field" for many years already in disk controllers from major silicon vendors.
> > But that's still not a guarantee .... So how do we resolve this? Majority vote? ;-)
> >
> 
> Thanks for the additional test vectors. They work fine with my SIMD
> implementations for ARM [0], so this looks like it might be a CAAM
> problem, not a problem with the test vectors.
> 
Thanks for the heads up! As the engineer actually responsible for our hardware
implementation, I can now sleep at night again :-)

Not that I was too worried - the design has been in active use for nearly 12 years
already without any known issues, but still ...

> I will try to find some time today to run them through OpenSSL to double check.
> 
> 
> [0] https://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git/commit/?h=xts-cts

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-27  5:39                                             ` Ard Biesheuvel
  2019-07-27 12:56                                               ` Pascal Van Leeuwen
@ 2019-07-27 16:04                                               ` Milan Broz
  2019-08-04  8:36                                                 ` Ard Biesheuvel
  1 sibling, 1 reply; 65+ messages in thread
From: Milan Broz @ 2019-07-27 16:04 UTC (permalink / raw)
  To: Ard Biesheuvel, Pascal Van Leeuwen
  Cc: Horia Geanta, Milan Broz, Herbert Xu, dm-devel, linux-crypto

On 27/07/2019 07:39, Ard Biesheuvel wrote:
> Thanks for the additional test vectors. They work fine with my SIMD
> implementations for ARM [0], so this looks like it might be a CAAM
> problem, not a problem with the test vectors.
> 
> I will try to find some time today to run them through OpenSSL to double check.

I shamelessly copied your test vectors to my vector test for cryptsetup backend.

Both OpenSSL and gcrypt XTS implementation passed all tests here!

If interested - this is copy of backend we have in cryptsetup, vectors added in crypto-vectors.c
(there are some hard defines in Makefile, cryptsetup uses autoconf instead).
  OpenSSL: https://github.com/mbroz/cryptsetup_backend_test
  gcrypt branch: https://github.com/mbroz/cryptsetup_backend_test/tree/gcrypt

Once kernel AF_ALG supports it, I can easily test it the same way.

Thanks,
Milan

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-07-27 16:04                                               ` Milan Broz
@ 2019-08-04  8:36                                                 ` Ard Biesheuvel
  0 siblings, 0 replies; 65+ messages in thread
From: Ard Biesheuvel @ 2019-08-04  8:36 UTC (permalink / raw)
  To: Milan Broz
  Cc: Pascal Van Leeuwen, Horia Geanta, Herbert Xu, dm-devel, linux-crypto

On Sat, 27 Jul 2019 at 19:04, Milan Broz <gmazyland@gmail.com> wrote:
>
> On 27/07/2019 07:39, Ard Biesheuvel wrote:
> > Thanks for the additional test vectors. They work fine with my SIMD
> > implementations for ARM [0], so this looks like it might be a CAAM
> > problem, not a problem with the test vectors.
> >
> > I will try to find some time today to run them through OpenSSL to double check.
>
> I shamelessly copied your test vectors to my vector test for cryptsetup backend.
>
> Both OpenSSL and gcrypt XTS implementation passed all tests here!
>
> If interested - this is copy of backend we have in cryptsetup, vectors added in crypto-vectors.c
> (there are some hard defines in Makefile, cryptsetup uses autoconf instead).
>   OpenSSL: https://github.com/mbroz/cryptsetup_backend_test
>   gcrypt branch: https://github.com/mbroz/cryptsetup_backend_test/tree/gcrypt
>
> Once kernel AF_ALG supports it, I can easily test it the same way.
>

Thanks for confirming. So we can be reasonably confident that the test
vectors contributed by Pascal are sound.

I'll try to send out my ARM/arm64 changes shortly. However, I won't
have any access to hardware until end of next month, so they are
tested on QEMU only, which means I won't be able to provide any
performance numbers.

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
       [not found]                                         ` <20f4832e-e3af-e3c2-d946-13bf8c367a60@nxp.com>
@ 2019-08-07 15:51                                           ` Horia Geanta
  2019-08-07 20:57                                             ` Pascal Van Leeuwen
  2019-08-08 13:43                                             ` Pascal Van Leeuwen
  0 siblings, 2 replies; 65+ messages in thread
From: Horia Geanta @ 2019-08-07 15:51 UTC (permalink / raw)
  To: Pascal Van Leeuwen, Ard Biesheuvel
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto

On 7/26/2019 10:59 PM, Horia Geantă wrote:
> On 7/26/2019 1:31 PM, Pascal Van Leeuwen wrote:
>> Ok, find below a patch file that adds your vectors from the specification
>> plus my set of additional vectors covering all CTS alignments combined
>> with the block sizes you desired. Please note though that these vectors
>> are from our in-house home-grown model so no warranties.
> I've checked the test vectors against caam (HW + driver).
> 
> Test vectors from IEEE 1619-2007 (i.e. up to and including "XTS-AES 18")
> are fine.
> 
> caam complains when /* Additional vectors to increase CTS coverage */
> section starts:
> alg: skcipher: xts-aes-caam encryption test failed (wrong result) on test vector 9, cfg="in-place"
> 
I've nailed this down to a caam hw limitation.
Except for lx2160a and ls1028a SoCs, all the (older) SoCs allow only for
8-byte wide IV (sector index).
Will follow up with 16-byte IV support for the above-mentioned SoCs.

Pascal,

Could you also generate a few test vectors covering CTS with 8-byte IV?

Thanks,
Horia

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-07 15:51                                           ` Horia Geanta
@ 2019-08-07 20:57                                             ` Pascal Van Leeuwen
  2019-08-08 14:50                                               ` Horia Geanta
  2019-08-08 13:43                                             ` Pascal Van Leeuwen
  1 sibling, 1 reply; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-08-07 20:57 UTC (permalink / raw)
  To: Horia Geanta, Ard Biesheuvel
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto

> -----Original Message-----
> From: Horia Geanta <horia.geanta@nxp.com>
> Sent: Wednesday, August 7, 2019 5:52 PM
> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Ard Biesheuvel
> <ard.biesheuvel@linaro.org>
> Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-
> devel@redhat.com; linux-crypto@vger.kernel.org
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> On 7/26/2019 10:59 PM, Horia Geantă wrote:
> > On 7/26/2019 1:31 PM, Pascal Van Leeuwen wrote:
> >> Ok, find below a patch file that adds your vectors from the specification
> >> plus my set of additional vectors covering all CTS alignments combined
> >> with the block sizes you desired. Please note though that these vectors
> >> are from our in-house home-grown model so no warranties.
> > I've checked the test vectors against caam (HW + driver).
> >
> > Test vectors from IEEE 1619-2007 (i.e. up to and including "XTS-AES 18")
> > are fine.
> >
> > caam complains when /* Additional vectors to increase CTS coverage */
> > section starts:
> > alg: skcipher: xts-aes-caam encryption test failed (wrong result) on test vector 9,
> cfg="in-place"
> >
> I've nailed this down to a caam hw limitation.
> Except for lx2160a and ls1028a SoCs, all the (older) SoCs allow only for
> 8-byte wide IV (sector index).
>
I guess it's easy to say now, but I already suspected a problem with full 16 
byte random IV's. A problem with CTS itself seemed implausible due to the base
vectors from the spec running fine and I did happen to notice that all 
vectors from the spec only use up to the lower 40 bits of the sector number.
While my vectors randomize all 16 bytes.

So I guess that means that 16 byte multiples (i.e. not needing CTS) with
full 16 byte sector numbers will probably also fail on caam HW ...
	
As for the tweak size, with very close scrutiny of the IEEE spec I actually
noticed some inconsistencies:

- the text very clearly defines the tweak as 128 bit and starting from an 
*arbitrary* non-negative integer, this is what I based my implementation on

- all text examples and test vectors max out at 40 bits ... just examples,
but odd nonetheless (why 40 anyway?)

- the example code fragment in Annex C actually has the S data unit number
input as an u64b, further commented as "64 bits" (but then loops 16 times to
convert it to a byte string ...)

So I guess from this specification, a true HW engineer might implement a
full 128 bit tweak, while a SW engineer looking at the example code might
just implement 64 bits. And the latter would pass all provided test vectors.

> Will follow up with 16-byte IV support for the above-mentioned SoCs.
> 
> Pascal,
> 
> Could you also generate a few test vectors covering CTS with 8-byte IV?
> 
I can generate them in a format similar to what's in the IEEE spec, but I
don't feel much like typing them over into testmgr.h format again.
If you don't mind.

> Thanks,
> Horia

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-07 15:51                                           ` Horia Geanta
  2019-08-07 20:57                                             ` Pascal Van Leeuwen
@ 2019-08-08 13:43                                             ` Pascal Van Leeuwen
  2019-08-08 18:01                                               ` Horia Geanta
  1 sibling, 1 reply; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-08-08 13:43 UTC (permalink / raw)
  To: Horia Geanta, Ard Biesheuvel
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto

[-- Attachment #1: Type: text/plain, Size: 2082 bytes --]

Hi Horia,

This is the best I can do on short notice w.r.t vectors with 8 byte IV.
Format is actually equivalent to that of the XTS specification, with
the sector number being referred to as "H".

Actually, the input keys, plaintext and IV should be the same as before,
with the exception of the IV being truncated to 64 bits, so that should
give you some reference regarding byte order etc.

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

> -----Original Message-----
> From: Horia Geanta <horia.geanta@nxp.com>
> Sent: Wednesday, August 7, 2019 5:52 PM
> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Ard Biesheuvel
> <ard.biesheuvel@linaro.org>
> Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-
> devel@redhat.com; linux-crypto@vger.kernel.org
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> On 7/26/2019 10:59 PM, Horia Geantă wrote:
> > On 7/26/2019 1:31 PM, Pascal Van Leeuwen wrote:
> >> Ok, find below a patch file that adds your vectors from the specification
> >> plus my set of additional vectors covering all CTS alignments combined
> >> with the block sizes you desired. Please note though that these vectors
> >> are from our in-house home-grown model so no warranties.
> > I've checked the test vectors against caam (HW + driver).
> >
> > Test vectors from IEEE 1619-2007 (i.e. up to and including "XTS-AES 18")
> > are fine.
> >
> > caam complains when /* Additional vectors to increase CTS coverage */
> > section starts:
> > alg: skcipher: xts-aes-caam encryption test failed (wrong result) on test vector 9,
> cfg="in-place"
> >
> I've nailed this down to a caam hw limitation.
> Except for lx2160a and ls1028a SoCs, all the (older) SoCs allow only for
> 8-byte wide IV (sector index).
> Will follow up with 16-byte IV support for the above-mentioned SoCs.
> 
> Pascal,
> 
> Could you also generate a few test vectors covering CTS with 8-byte IV?
> 
> Thanks,
> Horia

[-- Attachment #2: linuxdrv_256enc2.out --]
[-- Type: application/octet-stream, Size: 11229 bytes --]

// Start of Record : 1
// AES XTS operation with CTS
// Encrypt
// Key
// Key2
// i
// Text
// No j input
// Increment value 00000001
// Output text
K               : a1340e4938fd8bf6456067070f50a82b
                : a8f1fe7ef4f047cdfd9178f9148b7d27
K2              : 0edccae6f4fcd74f198cd0e69e2ff875
                : b5e248004f07d9a142bc9dfc17980048
H               : cb35475a7a0628b90000000000000000
P               : 0452c87fb05a12c596476bf4bc2edb74
                : d2202432e584b6254c2f96c7559c906f
                : 0e969468f4
C               : 73fa690b1c213a6183885e57e3f2791f
                : 6a6479ffa2aaf27067f506f948b296a4
                : d7d54826c9
IV res          : 17e9f2c5faaa24711ad1de59178c596c
                : 2ed2e58bf55549e234a2bdb32e18b3d8
                : dba4cb17ebab92c469447b675d3066b1
// End of Record
//////////////////////////////////////////////////
// Start of Record : 2
// AES XTS operation with CTS
// Encrypt
// Key
// Key2
// i
// Text
// No j input
// Output text
K               : f78775df3620e7cb205d4996813d1d80
                : c7187ebf2a0f79ba06b54b6303fbb849
K2              : 932d855b951f78ea7c1ef55d02c6ecb0
                : f0aa3d0a04e167802abe4e73c911cc6c
H               : ebba5524fc8f257c0000000000000000
P               : 40751b722ac8bfef0c923e19c5090738
                : 4d875cb8d64f1a398ceea5224112e122
                : b54bd7eb02faaaf89447045d8ab54012
                : 04623de4198aebb3f9a37db6eb57f9b8
                : 7fa8fa2d752d
C               : e69e4b1b27f7c00bf939a6fa42514b4f
                : 72baeebf3ae697216429cd30ae3866ca
                : 2bff1df3d53de1f5a98b7a3ada111371
                : 2c1ed32b4373533b6fc7fdfc1f599e99
                : 399042cd0c38
IV res          : 28f256bc0aec463c7b872383a58fccb4
                : d7e4ad7815d88d78f60e47064b1f9969
                : aec95bf12ab01bf1ec1d8e0c963e32d3
                : db93b7e2556037e2d93b1c192c7d64a6
                : 31276fc5abc06ec4b377383258fac84c
// End of Record
//////////////////////////////////////////////////
// Start of Record : 3
// AES XTS operation with CTS
// Encrypt
// Key
// Key2
// i
// Text
// No j input
// Output text
K               : 4809ab48d6ca7db190a000d8338a2079
                : 7cbc0c0c5f41bcbc82af41812393cbc7
K2              : 617b831316b13e7cccaedaca78c7ab18
                : 69b6583e5c195fed7bcf70b97600d8c9
H               : 2e2036f4a3225dd80000000000000000
P               : 793c73996521e1b9a0fd22b257c07ff4
                : 7f9736aff88d73e10d85e9d53d82b349
                : 8925301f0dca5c956431021711088f32
                : bc37234f0398914a50e258a89b6409e0
                : ce99c9b0a82173b72d4b19ba818399ce
                : a07ad09f27f68a
C               : 1a87622b05095e069416d1a5aed83486
                : 1f1280efb97c007ff8da89d1850c0f79
                : 14969a545c0f11e1d82b2028b8e58b73
                : 8390b3c61e0007228bc80c5a1d74f1fc
                : 31fd80dbfd63c8d88027ccb63b7058c2
                : ef52594de49e4e
IV res          : 67d7ee744dffd5532bba98d4eac06a0a
                : ceaedde99afeaba7567431a9d581d514
                : 9c5dbbd335fd574fade86252ab03ab29
                : 38bb76a76bfaaf9e5ad1c5a456075653
                : 7076ed4ed7f45f3db5a28b49ad0eaca6
                : 67ecda9daee9bf7a6a4517935a1d584d
// End of Record
//////////////////////////////////////////////////
// Start of Record : 4
// AES XTS operation with CTS
// Encrypt
// Key
// Key2
// i
// Text
// No j input
// Output text
K               : 8cf44ce5918f72e92ff8c03c877616a4
                : 20ab66393410d691f1992cf1d6c3da38
K2              : ed2a4c80f4a556281a1c79726c930886
                : 8f8aaacdf18ccae70ae8ee0c1cc2a8ea
H               : 9a9ebce4c9f3ef9f0000000000000000
P               : c1de661a7e60d33b66d6298699c6d7c8
                : 29bf0057ab210624d092efe6b51e20b9
                : b77bd71888f8d7e39061cd732ba1b5c7
                : 33efb5f245f692539198f85a20754ca8
                : f1f60126bcba4caccbc26db62c3c3861
                : e3987f3e98bdeccec0b5742343247b7e
                : 3fedcbda88676f9a
C               : e5b402ac013ba8739e5ba4729e41850f
                : 60136bc57edd329c2f955e953ebc7a65
                : 1cf60d6158871eff96b801865fe23684
                : 61f3992c068d00c3ef07f524f76dac11
                : 0d401fe794cd023ed2d8677108ad8c71
                : ac21c709923c59d891fb43f22a67ca97
                : 3cc37844a990d84b
IV res          : ff181a1420e109d2b9f195819af7c2bd
                : 7931342840c213a473e32b0335ef857b
                : f262685080842748e7c657066ade0bf7
                : 63c5d0a000094f90ce8daf0cd4bc17ee
                : 418aa14101129e209d1b5f19a8792fdc
                : 0514438302243c413a37be3250f35eb8
                : 8d28860605487882746e7c65a0e6bd70
// End of Record
//////////////////////////////////////////////////
// Start of Record : 5
// AES XTS operation with CTS
// Encrypt
// Key
// Key2
// i
// Text
// No j input
// Output text
K               : 70180993103a0ca9020b1110ae3498db
                : 10b5ee8c49bc528e4bf70a36168af706
K2              : b5945254b9c14d20a2f06e197f671eaa
                : 946cee5419fc9695048500537c395feb
H               : 36878f9d74e952fb0000000000000000
P               : 9508eefe87b24f9301eef3770dbbfb26
                : 3eb33420ee51d640b164aed9fd718f93
                : a585ff74ccd3fd5ec2fc49daa83a9429
                : a259903426bba0345d4733f2a8779098
                : 8dfd3860231e50a1674d8d09e07d30e3
                : dd3991d47068bb064e11b2260a8573f6
                : 37b615d077ee437b7713e9b9842b34ab
                : 49c127912ea3cae5a77945ba36974944
                : f7579bd7acb3fd6a1cd1fc1cdf6f94ac
                : 95f4507ac8c38c603c
C               : 91e535f272cc15bb2ea36d8025adb214
                : 3e0eb83368c6c03f212b02bc9097eef0
                : 720ba95efc95af30145c83ed974fda61
                : f353ff6fcd0ab3f55fdd46a9f08a5a8b
                : eb992a079f1649a9be4e3b93a3be17a1
                : a35187492594278e49387bfbe4aa6ab9
                : 85a23aa30e8c0d03a2bfe47d711b4b4f
                : ae4ebdff94ee22bcb9470e7c2cd4a8b5
                : e2aaa9bfe7addc69fa51950b901053cb
                : c48bc9cfec0ec6c34f
IV res          : ca72233f2444e1bbcd867b98fc3a9d27
                : 94e5467e4888c2779b0df730f9753a4f
                : 28cb8dfc901085ef361bee61f2eb749e
                : d7961bf921210adf6d36dcc3e4d7e93c
                : ae2d37f2434214bedb6cb887c9afd379
                : 5c5b6ee48784287cb7d9700f935fa7f3
                : 3fb6dcc80f0951f86eb3e11e26bf4ee7
                : f96cb9911f12a2f0dd66c33d4c7e9dce
                : 75d972233f2444e1bbcd867b98fc3a9d
                : 6db2e5467e4888c2779b0df730f9753a
// End of Record
//////////////////////////////////////////////////
// Start of Record : 6
// AES XTS operation with CTS
// Encrypt
// Key
// Key2
// i
// Text
// No j input
// Output text
K               : 5a383f9c0c53176c60722326bafea1b7
                : 03a8fea07cff784c7d842f248477ec6f
K2              : 88c836e2cb523cb439ac37fa418bc459
                : 2403e151c9547db7a3de91448d169722
H               : fb7f3d60260a3a3d0000000000000000
P               : fb5697657cd86c3c5dd3eaa6a483f79d
                : 9d892c85b8d9d4f01aad
C               : 764f628f465093dceeb2920048580f0b
                : b047b9701aa3811934ef
IV res          : 09be008c8ac82a48549550805db8252d
                : 127c011815915590a82aa100bb704b5a
// End of Record
//////////////////////////////////////////////////
// Start of Record : 7
// AES XTS operation with CTS
// Encrypt
// Key
// Key2
// i
// Text
// No j input
// Output text
K               : c0cf57a23ca24bf65d367bd71d16c32f
                : 50c60ab2fde824fc33cf73fde0e9a5d1
K2              : 98fcd616ddfd6dab44bc379dab5b1df2
                : 6f5dbe6b1414c774bb91244b52cb7831
H               : 5cc13db6a16a2d1f0000000000000000
P               : 02953aabac3bcdcd63c74c7ce575ee03
                : 94c7ffe8e0e9862ad3c7e4
C               : d508321a5d93bc62d61ec48f4212e3c1
                : 0eb5990c6b641c40e30f4f
IV res          : 5c0d843696fc718285f5b04ed814f4dd
                : 3f1a086d2cf9e3040beb619db029e8bb
// End of Record
//////////////////////////////////////////////////
// Start of Record : 8
// AES XTS operation with CTS
// Encrypt
// Key
// Key2
// i
// Text
// No j input
// Output text
K               : 0b5b1dc8b13f8fcd87d2582836c634fb
                : 04e8f1b79130da75664a729009390219
K2              : 622de924950e87434cc796e4c9316a13
                : 1610ef349b9819f18b14383ff875cc76
H               : 0c2c552cda40e1ab0000000000000000
P               : be84d3fee6b42967fd2978413de9814e
                : 3cf9f4f53fd80ecd637365f3
C               : 5ca1215396dceb13f991318a65c6324f
                : eaa63e70d2fa37cf9bcbc34a
IV res          : 5772f4c7a47f94f8b78819c8825c1204
                : aee4e88f49ff28f16f11339005b92408
// End of Record
//////////////////////////////////////////////////
// Start of Record : 9
// AES XTS operation with CTS
// Encrypt
// Key
// Key2
// i
// Text
// No j input
// Output text
K               : dc4cdc20b13489a4d0b67705ea0ccc68
                : b1d6f7fda70a5b812d4da365d0aba102
K2              : 854b33ea511650123b25ba13ba7cbb3a
                : e4fdb39c888bb8307a97cf955d697b1d
H               : e769edd2545d4a290000000000000000
P               : 37221162a0749262404e2b0a8babd828
                : 8ad2eba58ee142c849ef9aec1b
C               : 51649d924664e1c6b738262b7414243d
                : ba17c48f72c2a4013948197469
IV res          : 2b7305025e693c4880b5a3ed6235ae1a
                : 56e60a04bcd27890006b47dbc56a5c35
// End of Record
//////////////////////////////////////////////////
// Start of Record : 10
// AES XTS operation with CTS
// Encrypt
// Key
// Key2
// i
// Text
// No j input
// Output text
K               : 729af55355dd0feffc756f0388c8ba88
                : b765895d03862122b84287d9a9839e9c
K2              : ca28a1d2b6d0a66cf857427c73fc7b0a
                : bc3c577b5a396155b725e9f1c4bb0428
H               : 8a3822baea5e1da40000000000000000
P               : 06fdbba92e56055ff2a7367626d3b349
                : 7ce2e3be1f65d21765e2b30eb193
C               : 8583e08c0a4e68b1e43f64030bf87276
                : dd9ee092e6ed7efddd8648b18e2d
IV res          : 6f838935b62409ed853f411e18d35a2b
                : de06136b6c4912da0b7f823c30a6b556
// End of Record
//////////////////////////////////////////////////
// Start of Record : 11
// AES XTS operation with CTS
// Encrypt
// Key
// Key2
// i
// Text
// No j input
// Output text
K               : ce0645532581d2b2ddc957febbf68307
                : 28d82aff53f857c66350d43e2a543751
K2              : 073b23633c31570dd35920f2d085acc5
                : 3fa174900a3ff41012f01b2befcb8674
H               : 6d3e6294754374ea0000000000000000
P               : 6ae6a3667e78ef428b280824dad4d642
                : 3db6487e51a69265988626983742a5
C               : 844b141f8ebcede395087d6e5b62f9be
                : 82085da2feac39f305fa2478f3966e
IV res          : 8da9535ac23b8da12513a56f5fc799b7
                : 9d53a7b484771a434b264adfbe8e336f
// End of Record
//////////////////////////////////////////////////

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-07 20:57                                             ` Pascal Van Leeuwen
@ 2019-08-08 14:50                                               ` Horia Geanta
  2019-08-09  8:35                                                 ` Pascal Van Leeuwen
  0 siblings, 1 reply; 65+ messages in thread
From: Horia Geanta @ 2019-08-08 14:50 UTC (permalink / raw)
  To: Pascal Van Leeuwen, Ard Biesheuvel
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto

On 8/7/2019 11:58 PM, Pascal Van Leeuwen wrote:
>> -----Original Message-----
>> From: Horia Geanta <horia.geanta@nxp.com>
>> Sent: Wednesday, August 7, 2019 5:52 PM
>> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Ard Biesheuvel
>> <ard.biesheuvel@linaro.org>
>> Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-
>> devel@redhat.com; linux-crypto@vger.kernel.org
>> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
>>
>> On 7/26/2019 10:59 PM, Horia Geantă wrote:
>>> On 7/26/2019 1:31 PM, Pascal Van Leeuwen wrote:
>>>> Ok, find below a patch file that adds your vectors from the specification
>>>> plus my set of additional vectors covering all CTS alignments combined
>>>> with the block sizes you desired. Please note though that these vectors
>>>> are from our in-house home-grown model so no warranties.
>>> I've checked the test vectors against caam (HW + driver).
>>>
>>> Test vectors from IEEE 1619-2007 (i.e. up to and including "XTS-AES 18")
>>> are fine.
>>>
>>> caam complains when /* Additional vectors to increase CTS coverage */
>>> section starts:
>>> alg: skcipher: xts-aes-caam encryption test failed (wrong result) on test vector 9,
>> cfg="in-place"
>>>
>> I've nailed this down to a caam hw limitation.
>> Except for lx2160a and ls1028a SoCs, all the (older) SoCs allow only for
>> 8-byte wide IV (sector index).
>>
> I guess it's easy to say now, but I already suspected a problem with full 16 
> byte random IV's. A problem with CTS itself seemed implausible due to the base
> vectors from the spec running fine and I did happen to notice that all 
> vectors from the spec only use up to the lower 40 bits of the sector number.
> While my vectors randomize all 16 bytes.
> 
> So I guess that means that 16 byte multiples (i.e. not needing CTS) with
> full 16 byte sector numbers will probably also fail on caam HW ...
> 	
Yes, the limitation applies for all input sizes.

It's actually mentioned in the commit that added xts support few years back:
c6415a6016bf ("crypto: caam - add support for acipher xts(aes)")

    sector index - HW limitation: CAAM device supports sector index of only
    8 bytes to be used for sector index inside IV, instead of whole 16 bytes
    received on request. This represents 2 ^ 64 = 16,777,216 Tera of possible
    values for sector index.

> As for the tweak size, with very close scrutiny of the IEEE spec I actually
> noticed some inconsistencies:
> 
> - the text very clearly defines the tweak as 128 bit and starting from an 
> *arbitrary* non-negative integer, this is what I based my implementation on
> 
> - all text examples and test vectors max out at 40 bits ... just examples,
> but odd nonetheless (why 40 anyway?)
> 
> - the example code fragment in Annex C actually has the S data unit number
> input as an u64b, further commented as "64 bits" (but then loops 16 times to
> convert it to a byte string ...)
> 
The input I received from our HW design team was something like:

- some P1619 drafts used LRW (instead of XTS), where the tweak "T"
was 16B-wide

- at some point P1619 drafts switched (and eventually standardized) XTS,
where "T" is no longer the tweak - "i" is the (public) tweak, "T" being
an intermediate (hidden) result in the encryption scheme

- since for XTS "i" is supposed to be the sector number,
there is no need to support 16B values - 8B being deemed sufficient

Agree, limiting "i" (XTS tweak) to 8B is out-of-spec - irrespective of the
usefulness of the full 16B.
That's why latest Freescale / NXP SoCs support 16B tweaks.

Thanks,
Horia

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-08 13:43                                             ` Pascal Van Leeuwen
@ 2019-08-08 18:01                                               ` Horia Geanta
  2019-08-09  2:48                                                 ` Herbert Xu
  0 siblings, 1 reply; 65+ messages in thread
From: Horia Geanta @ 2019-08-08 18:01 UTC (permalink / raw)
  To: Pascal Van Leeuwen, Ard Biesheuvel
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto

On 8/8/2019 4:43 PM, Pascal Van Leeuwen wrote:
> Hi Horia,
> 
> This is the best I can do on short notice w.r.t vectors with 8 byte IV.
> Format is actually equivalent to that of the XTS specification, with
> the sector number being referred to as "H".
> 
> Actually, the input keys, plaintext and IV should be the same as before,
> with the exception of the IV being truncated to 64 bits, so that should
> give you some reference regarding byte order etc.
> 
Thanks.

I've converted the test vectors and generated patch below,
which is based on a previous patch you've sent:
https://lore.kernel.org/linux-crypto/MN2PR20MB29739591E1A3E54E7A8A8E18CAC00@MN2PR20MB2973.namprd20.prod.outlook.com/

-- >8 --

Subject: [PATCH] crypto: testmgr - Add additional AES-XTS vectors for covering
 CTS (part II)

Add more test vectors (provided by Pascal Van Leeuwen)
for xts ciphertext stealing checking.

They are derived from the test vectors added in
commit "crypto: testmgr - Add additional AES-XTS vectors for covering CTS"
where IV / tweak was truncated to 8 bytes (and obviously
different ciphertext).

Link: https://lore.kernel.org/linux-crypto/MN2PR20MB2973127E4C159A8F5CFDD0C9CAD70@MN2PR20MB2973.namprd20.prod.outlook.com
Signed-off-by: Horia Geantă <horia.geanta@nxp.com>
---
 crypto/testmgr.h | 310 ++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 309 insertions(+), 1 deletion(-)

diff --git a/crypto/testmgr.h b/crypto/testmgr.h
index 717b9fcb9bfa..453bb6473f13 100644
--- a/crypto/testmgr.h
+++ b/crypto/testmgr.h
@@ -15351,7 +15351,315 @@ static const struct cipher_testvec aes_xts_tv_template[] = {
 			  "\x7b\xe3\xf6\x61\x71\xc7\xc5\xc2"
 			  "\xed\xbf\x9d\xac",
 		.len	= 20,
-	/* Additional vectors to increase CTS coverage */
+	/* Additional vectors to increase CTS coverage - 8B IV */
+	}, { /* 1 block + 21 bytes */
+		.key    = "\xa1\x34\x0e\x49\x38\xfd\x8b\xf6"
+			  "\x45\x60\x67\x07\x0f\x50\xa8\x2b"
+			  "\xa8\xf1\xfe\x7e\xf4\xf0\x47\xcd"
+			  "\xfd\x91\x78\xf9\x14\x8b\x7d\x27"
+			  "\x0e\xdc\xca\xe6\xf4\xfc\xd7\x4f"
+			  "\x19\x8c\xd0\xe6\x9e\x2f\xf8\x75"
+			  "\xb5\xe2\x48\x00\x4f\x07\xd9\xa1"
+			  "\x42\xbc\x9d\xfc\x17\x98\x00\x48",
+		.klen   = 64,
+		.iv     = "\xcb\x35\x47\x5a\x7a\x06\x28\xb9"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\x04\x52\xc8\x7f\xb0\x5a\x12\xc5"
+			  "\x96\x47\x6b\xf4\xbc\x2e\xdb\x74"
+			  "\xd2\x20\x24\x32\xe5\x84\xb6\x25"
+			  "\x4c\x2f\x96\xc7\x55\x9c\x90\x6f"
+			  "\x0e\x96\x94\x68\xf4",
+		.ctext	= "\x73\xfa\x69\x0b\x1c\x21\x3a\x61"
+			  "\x83\x88\x5e\x57\xe3\xf2\x79\x1f"
+			  "\x6a\x64\x79\xff\xa2\xaa\xf2\x70"
+			  "\x67\xf5\x06\xf9\x48\xb2\x96\xa4"
+			  "\xd7\xd5\x48\x26\xc9",
+		.len	= 37,
+	}, { /* 3 blocks + 22 bytes */
+		.key    = "\xf7\x87\x75\xdf\x36\x20\xe7\xcb"
+			  "\x20\x5d\x49\x96\x81\x3d\x1d\x80"
+			  "\xc7\x18\x7e\xbf\x2a\x0f\x79\xba"
+			  "\x06\xb5\x4b\x63\x03\xfb\xb8\x49"
+			  "\x93\x2d\x85\x5b\x95\x1f\x78\xea"
+			  "\x7c\x1e\xf5\x5d\x02\xc6\xec\xb0"
+			  "\xf0\xaa\x3d\x0a\x04\xe1\x67\x80"
+			  "\x2a\xbe\x4e\x73\xc9\x11\xcc\x6c",
+		.klen   = 64,
+		.iv     = "\xeb\xba\x55\x24\xfc\x8f\x25\x7c"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\x40\x75\x1b\x72\x2a\xc8\xbf\xef"
+			  "\x0c\x92\x3e\x19\xc5\x09\x07\x38"
+			  "\x4d\x87\x5c\xb8\xd6\x4f\x1a\x39"
+			  "\x8c\xee\xa5\x22\x41\x12\xe1\x22"
+			  "\xb5\x4b\xd7\xeb\x02\xfa\xaa\xf8"
+			  "\x94\x47\x04\x5d\x8a\xb5\x40\x12"
+			  "\x04\x62\x3d\xe4\x19\x8a\xeb\xb3"
+			  "\xf9\xa3\x7d\xb6\xeb\x57\xf9\xb8"
+			  "\x7f\xa8\xfa\x2d\x75\x2d",
+		.ctext	= "\xe6\x9e\x4b\x1b\x27\xf7\xc0\x0b"
+			  "\xf9\x39\xa6\xfa\x42\x51\x4b\x4f"
+			  "\x72\xba\xee\xbf\x3a\xe6\x97\x21"
+			  "\x64\x29\xcd\x30\xae\x38\x66\xca"
+			  "\x2b\xff\x1d\xf3\xd5\x3d\xe1\xf5"
+			  "\xa9\x8b\x7a\x3a\xda\x11\x13\x71"
+			  "\x2c\x1e\xd3\x2b\x43\x73\x53\x3b"
+			  "\x6f\xc7\xfd\xfc\x1f\x59\x9e\x99"
+			  "\x39\x90\x42\xcd\x0c\x38",
+		.len	= 70,
+	}, { /* 4 blocks + 23 bytes */
+		.key    = "\x48\x09\xab\x48\xd6\xca\x7d\xb1"
+			  "\x90\xa0\x00\xd8\x33\x8a\x20\x79"
+			  "\x7c\xbc\x0c\x0c\x5f\x41\xbc\xbc"
+			  "\x82\xaf\x41\x81\x23\x93\xcb\xc7"
+			  "\x61\x7b\x83\x13\x16\xb1\x3e\x7c"
+			  "\xcc\xae\xda\xca\x78\xc7\xab\x18"
+			  "\x69\xb6\x58\x3e\x5c\x19\x5f\xed"
+			  "\x7b\xcf\x70\xb9\x76\x00\xd8\xc9",
+		.klen   = 64,
+		.iv     = "\x2e\x20\x36\xf4\xa3\x22\x5d\xd8"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\x79\x3c\x73\x99\x65\x21\xe1\xb9"
+			  "\xa0\xfd\x22\xb2\x57\xc0\x7f\xf4"
+			  "\x7f\x97\x36\xaf\xf8\x8d\x73\xe1"
+			  "\x0d\x85\xe9\xd5\x3d\x82\xb3\x49"
+			  "\x89\x25\x30\x1f\x0d\xca\x5c\x95"
+			  "\x64\x31\x02\x17\x11\x08\x8f\x32"
+			  "\xbc\x37\x23\x4f\x03\x98\x91\x4a"
+			  "\x50\xe2\x58\xa8\x9b\x64\x09\xe0"
+			  "\xce\x99\xc9\xb0\xa8\x21\x73\xb7"
+			  "\x2d\x4b\x19\xba\x81\x83\x99\xce"
+			  "\xa0\x7a\xd0\x9f\x27\xf6\x8a",
+		.ctext	= "\x1a\x87\x62\x2b\x05\x09\x5e\x06"
+			  "\x94\x16\xd1\xa5\xae\xd8\x34\x86"
+			  "\x1f\x12\x80\xef\xb9\x7c\x00\x7f"
+			  "\xf8\xda\x89\xd1\x85\x0c\x0f\x79"
+			  "\x14\x96\x9a\x54\x5c\x0f\x11\xe1"
+			  "\xd8\x2b\x20\x28\xb8\xe5\x8b\x73"
+			  "\x83\x90\xb3\xc6\x1e\x00\x07\x22"
+			  "\x8b\xc8\x0c\x5a\x1d\x74\xf1\xfc"
+			  "\x31\xfd\x80\xdb\xfd\x63\xc8\xd8"
+			  "\x80\x27\xcc\xb6\x3b\x70\x58\xc2"
+			  "\xef\x52\x59\x4d\xe4\x9e\x4e",
+		.len	= 87,
+	}, { /* 5 blocks + 24 bytes */
+		.key    = "\x8c\xf4\x4c\xe5\x91\x8f\x72\xe9"
+			  "\x2f\xf8\xc0\x3c\x87\x76\x16\xa4"
+			  "\x20\xab\x66\x39\x34\x10\xd6\x91"
+			  "\xf1\x99\x2c\xf1\xd6\xc3\xda\x38"
+			  "\xed\x2a\x4c\x80\xf4\xa5\x56\x28"
+			  "\x1a\x1c\x79\x72\x6c\x93\x08\x86"
+			  "\x8f\x8a\xaa\xcd\xf1\x8c\xca\xe7"
+			  "\x0a\xe8\xee\x0c\x1c\xc2\xa8\xea",
+		.klen   = 64,
+		.iv     = "\x9a\x9e\xbc\xe4\xc9\xf3\xef\x9f"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\xc1\xde\x66\x1a\x7e\x60\xd3\x3b"
+			  "\x66\xd6\x29\x86\x99\xc6\xd7\xc8"
+			  "\x29\xbf\x00\x57\xab\x21\x06\x24"
+			  "\xd0\x92\xef\xe6\xb5\x1e\x20\xb9"
+			  "\xb7\x7b\xd7\x18\x88\xf8\xd7\xe3"
+			  "\x90\x61\xcd\x73\x2b\xa1\xb5\xc7"
+			  "\x33\xef\xb5\xf2\x45\xf6\x92\x53"
+			  "\x91\x98\xf8\x5a\x20\x75\x4c\xa8"
+			  "\xf1\xf6\x01\x26\xbc\xba\x4c\xac"
+			  "\xcb\xc2\x6d\xb6\x2c\x3c\x38\x61"
+			  "\xe3\x98\x7f\x3e\x98\xbd\xec\xce"
+			  "\xc0\xb5\x74\x23\x43\x24\x7b\x7e"
+			  "\x3f\xed\xcb\xda\x88\x67\x6f\x9a",
+		.ctext	= "\xe5\xb4\x02\xac\x01\x3b\xa8\x73"
+			  "\x9e\x5b\xa4\x72\x9e\x41\x85\x0f"
+			  "\x60\x13\x6b\xc5\x7e\xdd\x32\x9c"
+			  "\x2f\x95\x5e\x95\x3e\xbc\x7a\x65"
+			  "\x1c\xf6\x0d\x61\x58\x87\x1e\xff"
+			  "\x96\xb8\x01\x86\x5f\xe2\x36\x84"
+			  "\x61\xf3\x99\x2c\x06\x8d\x00\xc3"
+			  "\xef\x07\xf5\x24\xf7\x6d\xac\x11"
+			  "\x0d\x40\x1f\xe7\x94\xcd\x02\x3e"
+			  "\xd2\xd8\x67\x71\x08\xad\x8c\x71"
+			  "\xac\x21\xc7\x09\x92\x3c\x59\xd8"
+			  "\x91\xfb\x43\xf2\x2a\x67\xca\x97"
+			  "\x3c\xc3\x78\x44\xa9\x90\xd8\x4b",
+		.len	= 104,
+	}, { /* 8 blocks + 25 bytes */
+		.key    = "\x70\x18\x09\x93\x10\x3a\x0c\xa9"
+			  "\x02\x0b\x11\x10\xae\x34\x98\xdb"
+			  "\x10\xb5\xee\x8c\x49\xbc\x52\x8e"
+			  "\x4b\xf7\x0a\x36\x16\x8a\xf7\x06"
+			  "\xb5\x94\x52\x54\xb9\xc1\x4d\x20"
+			  "\xa2\xf0\x6e\x19\x7f\x67\x1e\xaa"
+			  "\x94\x6c\xee\x54\x19\xfc\x96\x95"
+			  "\x04\x85\x00\x53\x7c\x39\x5f\xeb",
+		.klen   = 64,
+		.iv     = "\x36\x87\x8f\x9d\x74\xe9\x52\xfb"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\x95\x08\xee\xfe\x87\xb2\x4f\x93"
+			  "\x01\xee\xf3\x77\x0d\xbb\xfb\x26"
+			  "\x3e\xb3\x34\x20\xee\x51\xd6\x40"
+			  "\xb1\x64\xae\xd9\xfd\x71\x8f\x93"
+			  "\xa5\x85\xff\x74\xcc\xd3\xfd\x5e"
+			  "\xc2\xfc\x49\xda\xa8\x3a\x94\x29"
+			  "\xa2\x59\x90\x34\x26\xbb\xa0\x34"
+			  "\x5d\x47\x33\xf2\xa8\x77\x90\x98"
+			  "\x8d\xfd\x38\x60\x23\x1e\x50\xa1"
+			  "\x67\x4d\x8d\x09\xe0\x7d\x30\xe3"
+			  "\xdd\x39\x91\xd4\x70\x68\xbb\x06"
+			  "\x4e\x11\xb2\x26\x0a\x85\x73\xf6"
+			  "\x37\xb6\x15\xd0\x77\xee\x43\x7b"
+			  "\x77\x13\xe9\xb9\x84\x2b\x34\xab"
+			  "\x49\xc1\x27\x91\x2e\xa3\xca\xe5"
+			  "\xa7\x79\x45\xba\x36\x97\x49\x44"
+			  "\xf7\x57\x9b\xd7\xac\xb3\xfd\x6a"
+			  "\x1c\xd1\xfc\x1c\xdf\x6f\x94\xac"
+			  "\x95\xf4\x50\x7a\xc8\xc3\x8c\x60"
+			  "\x3c",
+		.ctext	= "\x91\xe5\x35\xf2\x72\xcc\x15\xbb"
+			  "\x2e\xa3\x6d\x80\x25\xad\xb2\x14"
+			  "\x3e\x0e\xb8\x33\x68\xc6\xc0\x3f"
+			  "\x21\x2b\x02\xbc\x90\x97\xee\xf0"
+			  "\x72\x0b\xa9\x5e\xfc\x95\xaf\x30"
+			  "\x14\x5c\x83\xed\x97\x4f\xda\x61"
+			  "\xf3\x53\xff\x6f\xcd\x0a\xb3\xf5"
+			  "\x5f\xdd\x46\xa9\xf0\x8a\x5a\x8b"
+			  "\xeb\x99\x2a\x07\x9f\x16\x49\xa9"
+			  "\xbe\x4e\x3b\x93\xa3\xbe\x17\xa1"
+			  "\xa3\x51\x87\x49\x25\x94\x27\x8e"
+			  "\x49\x38\x7b\xfb\xe4\xaa\x6a\xb9"
+			  "\x85\xa2\x3a\xa3\x0e\x8c\x0d\x03"
+			  "\xa2\xbf\xe4\x7d\x71\x1b\x4b\x4f"
+			  "\xae\x4e\xbd\xff\x94\xee\x22\xbc"
+			  "\xb9\x47\x0e\x7c\x2c\xd4\xa8\xb5"
+			  "\xe2\xaa\xa9\xbf\xe7\xad\xdc\x69"
+			  "\xfa\x51\x95\x0b\x90\x10\x53\xcb"
+			  "\xc4\x8b\xc9\xcf\xec\x0e\xc6\xc3"
+			  "\x4f",
+		.len	= 153,
+	}, { /* 0 blocks + 26 bytes */
+		.key    = "\x5a\x38\x3f\x9c\x0c\x53\x17\x6c"
+			  "\x60\x72\x23\x26\xba\xfe\xa1\xb7"
+			  "\x03\xa8\xfe\xa0\x7c\xff\x78\x4c"
+			  "\x7d\x84\x2f\x24\x84\x77\xec\x6f"
+			  "\x88\xc8\x36\xe2\xcb\x52\x3c\xb4"
+			  "\x39\xac\x37\xfa\x41\x8b\xc4\x59"
+			  "\x24\x03\xe1\x51\xc9\x54\x7d\xb7"
+			  "\xa3\xde\x91\x44\x8d\x16\x97\x22",
+		.klen   = 64,
+		.iv     = "\xfb\x7f\x3d\x60\x26\x0a\x3a\x3d"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\xfb\x56\x97\x65\x7c\xd8\x6c\x3c"
+			  "\x5d\xd3\xea\xa6\xa4\x83\xf7\x9d"
+			  "\x9d\x89\x2c\x85\xb8\xd9\xd4\xf0"
+			  "\x1a\xad",
+		.ctext	= "\x76\x4f\x62\x8f\x46\x50\x93\xdc"
+			  "\xee\xb2\x92\x00\x48\x58\x0f\x0b"
+			  "\xb0\x47\xb9\x70\x1a\xa3\x81\x19"
+			  "\x34\xef",
+		.len	= 26,
+	}, { /* 0 blocks + 27 bytes */
+		.key    = "\xc0\xcf\x57\xa2\x3c\xa2\x4b\xf6"
+			  "\x5d\x36\x7b\xd7\x1d\x16\xc3\x2f"
+			  "\x50\xc6\x0a\xb2\xfd\xe8\x24\xfc"
+			  "\x33\xcf\x73\xfd\xe0\xe9\xa5\xd1"
+			  "\x98\xfc\xd6\x16\xdd\xfd\x6d\xab"
+			  "\x44\xbc\x37\x9d\xab\x5b\x1d\xf2"
+			  "\x6f\x5d\xbe\x6b\x14\x14\xc7\x74"
+			  "\xbb\x91\x24\x4b\x52\xcb\x78\x31",
+		.klen   = 64,
+		.iv     = "\x5c\xc1\x3d\xb6\xa1\x6a\x2d\x1f"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\x02\x95\x3a\xab\xac\x3b\xcd\xcd"
+			  "\x63\xc7\x4c\x7c\xe5\x75\xee\x03"
+			  "\x94\xc7\xff\xe8\xe0\xe9\x86\x2a"
+			  "\xd3\xc7\xe4",
+		.ctext	= "\xd5\x08\x32\x1a\x5d\x93\xbc\x62"
+			  "\xd6\x1e\xc4\x8f\x42\x12\xe3\xc1"
+			  "\x0e\xb5\x99\x0c\x6b\x64\x1c\x40"
+			  "\xe3\x0f\x4f",
+		.len	= 27,
+	}, { /* 0 blocks + 28 bytes */
+		.key    = "\x0b\x5b\x1d\xc8\xb1\x3f\x8f\xcd"
+			  "\x87\xd2\x58\x28\x36\xc6\x34\xfb"
+			  "\x04\xe8\xf1\xb7\x91\x30\xda\x75"
+			  "\x66\x4a\x72\x90\x09\x39\x02\x19"
+			  "\x62\x2d\xe9\x24\x95\x0e\x87\x43"
+			  "\x4c\xc7\x96\xe4\xc9\x31\x6a\x13"
+			  "\x16\x10\xef\x34\x9b\x98\x19\xf1"
+			  "\x8b\x14\x38\x3f\xf8\x75\xcc\x76",
+		.klen   = 64,
+		.iv     = "\x0c\x2c\x55\x2c\xda\x40\xe1\xab"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\xbe\x84\xd3\xfe\xe6\xb4\x29\x67"
+			  "\xfd\x29\x78\x41\x3d\xe9\x81\x4e"
+			  "\x3c\xf9\xf4\xf5\x3f\xd8\x0e\xcd"
+			  "\x63\x73\x65\xf3",
+		.ctext	= "\x5c\xa1\x21\x53\x96\xdc\xeb\x13"
+			  "\xf9\x91\x31\x8a\x65\xc6\x32\x4f"
+			  "\xea\xa6\x3e\x70\xd2\xfa\x37\xcf"
+			  "\x9b\xcb\xc3\x4a",
+		.len	= 28,
+	}, { /* 0 blocks + 29 bytes */
+		.key    = "\xdc\x4c\xdc\x20\xb1\x34\x89\xa4"
+			  "\xd0\xb6\x77\x05\xea\x0c\xcc\x68"
+			  "\xb1\xd6\xf7\xfd\xa7\x0a\x5b\x81"
+			  "\x2d\x4d\xa3\x65\xd0\xab\xa1\x02"
+			  "\x85\x4b\x33\xea\x51\x16\x50\x12"
+			  "\x3b\x25\xba\x13\xba\x7c\xbb\x3a"
+			  "\xe4\xfd\xb3\x9c\x88\x8b\xb8\x30"
+			  "\x7a\x97\xcf\x95\x5d\x69\x7b\x1d",
+		.klen   = 64,
+		.iv     = "\xe7\x69\xed\xd2\x54\x5d\x4a\x29"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\x37\x22\x11\x62\xa0\x74\x92\x62"
+			  "\x40\x4e\x2b\x0a\x8b\xab\xd8\x28"
+			  "\x8a\xd2\xeb\xa5\x8e\xe1\x42\xc8"
+			  "\x49\xef\x9a\xec\x1b",
+		.ctext	= "\x51\x64\x9d\x92\x46\x64\xe1\xc6"
+			  "\xb7\x38\x26\x2b\x74\x14\x24\x3d"
+			  "\xba\x17\xc4\x8f\x72\xc2\xa4\x01"
+			  "\x39\x48\x19\x74\x69",
+		.len	= 29,
+	}, { /* 0 blocks + 30 bytes */
+		.key    = "\x72\x9a\xf5\x53\x55\xdd\x0f\xef"
+			  "\xfc\x75\x6f\x03\x88\xc8\xba\x88"
+			  "\xb7\x65\x89\x5d\x03\x86\x21\x22"
+			  "\xb8\x42\x87\xd9\xa9\x83\x9e\x9c"
+			  "\xca\x28\xa1\xd2\xb6\xd0\xa6\x6c"
+			  "\xf8\x57\x42\x7c\x73\xfc\x7b\x0a"
+			  "\xbc\x3c\x57\x7b\x5a\x39\x61\x55"
+			  "\xb7\x25\xe9\xf1\xc4\xbb\x04\x28",
+		.klen   = 64,
+		.iv     = "\x8a\x38\x22\xba\xea\x5e\x1d\xa4"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\x06\xfd\xbb\xa9\x2e\x56\x05\x5f"
+			  "\xf2\xa7\x36\x76\x26\xd3\xb3\x49"
+			  "\x7c\xe2\xe3\xbe\x1f\x65\xd2\x17"
+			  "\x65\xe2\xb3\x0e\xb1\x93",
+		.ctext	= "\x85\x83\xe0\x8c\x0a\x4e\x68\xb1"
+			  "\xe4\x3f\x64\x03\x0b\xf8\x72\x76"
+			  "\xdd\x9e\xe0\x92\xe6\xed\x7e\xfd"
+			  "\xdd\x86\x48\xb1\x8e\x2d",
+		.len	= 30,
+	}, { /* 0 blocks + 31 bytes */
+		.key    = "\xce\x06\x45\x53\x25\x81\xd2\xb2"
+			  "\xdd\xc9\x57\xfe\xbb\xf6\x83\x07"
+			  "\x28\xd8\x2a\xff\x53\xf8\x57\xc6"
+			  "\x63\x50\xd4\x3e\x2a\x54\x37\x51"
+			  "\x07\x3b\x23\x63\x3c\x31\x57\x0d"
+			  "\xd3\x59\x20\xf2\xd0\x85\xac\xc5"
+			  "\x3f\xa1\x74\x90\x0a\x3f\xf4\x10"
+			  "\x12\xf0\x1b\x2b\xef\xcb\x86\x74",
+		.klen   = 64,
+		.iv     = "\x6d\x3e\x62\x94\x75\x43\x74\xea"
+			  "\x00\x00\x00\x00\x00\x00\x00\x00",
+		.ptext	= "\x6a\xe6\xa3\x66\x7e\x78\xef\x42"
+			  "\x8b\x28\x08\x24\xda\xd4\xd6\x42"
+			  "\x3d\xb6\x48\x7e\x51\xa6\x92\x65"
+			  "\x98\x86\x26\x98\x37\x42\xa5",
+		.ctext	= "\x84\x4b\x14\x1f\x8e\xbc\xed\xe3"
+			  "\x95\x08\x7d\x6e\x5b\x62\xf9\xbe"
+			  "\x82\x08\x5d\xa2\xfe\xac\x39\xf3"
+			  "\x05\xfa\x24\x78\xf3\x96\x6e",
+		.len	= 31,
+	/* Additional vectors to increase CTS coverage - 16B IV */
 	}, { /* 1 block + 21 bytes */
 		.key    = "\xa1\x34\x0e\x49\x38\xfd\x8b\xf6"
 			  "\x45\x60\x67\x07\x0f\x50\xa8\x2b"
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-08 18:01                                               ` Horia Geanta
@ 2019-08-09  2:48                                                 ` Herbert Xu
  2019-08-09  6:45                                                   ` Ard Biesheuvel
  0 siblings, 1 reply; 65+ messages in thread
From: Herbert Xu @ 2019-08-09  2:48 UTC (permalink / raw)
  To: Horia Geanta
  Cc: Pascal Van Leeuwen, Ard Biesheuvel, Milan Broz, dm-devel, linux-crypto

On Thu, Aug 08, 2019 at 06:01:49PM +0000, Horia Geanta wrote:
>
> -- >8 --
> 
> Subject: [PATCH] crypto: testmgr - Add additional AES-XTS vectors for covering
>  CTS (part II)

Patchwork doesn't like it when you do this and it'll discard
your patch.  To make it into patchwork you need to put the new
Subject in the email headers.

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-09  2:48                                                 ` Herbert Xu
@ 2019-08-09  6:45                                                   ` Ard Biesheuvel
  2019-08-09  7:44                                                     ` Horia Geanta
  0 siblings, 1 reply; 65+ messages in thread
From: Ard Biesheuvel @ 2019-08-09  6:45 UTC (permalink / raw)
  To: Herbert Xu
  Cc: Horia Geanta, Pascal Van Leeuwen, Milan Broz, dm-devel, linux-crypto

On Fri, 9 Aug 2019 at 05:48, Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> On Thu, Aug 08, 2019 at 06:01:49PM +0000, Horia Geanta wrote:
> >
> > -- >8 --
> >
> > Subject: [PATCH] crypto: testmgr - Add additional AES-XTS vectors for covering
> >  CTS (part II)
>
> Patchwork doesn't like it when you do this and it'll discard
> your patch.  To make it into patchwork you need to put the new
> Subject in the email headers.
>

IMO, pretending that your XTS implementation is compliant by only
providing test vectors with the last 8 bytes of IV cleared is not the
right fix for this issue. If you want to be compliant, you will need
to provide a s/w fallback for these cases.

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-09  6:45                                                   ` Ard Biesheuvel
@ 2019-08-09  7:44                                                     ` Horia Geanta
  2019-08-09 17:49                                                       ` Ard Biesheuvel
  0 siblings, 1 reply; 65+ messages in thread
From: Horia Geanta @ 2019-08-09  7:44 UTC (permalink / raw)
  To: Ard Biesheuvel, Herbert Xu
  Cc: Pascal Van Leeuwen, Milan Broz, dm-devel, linux-crypto

On 8/9/2019 9:45 AM, Ard Biesheuvel wrote:
> On Fri, 9 Aug 2019 at 05:48, Herbert Xu <herbert@gondor.apana.org.au> wrote:
>>
>> On Thu, Aug 08, 2019 at 06:01:49PM +0000, Horia Geanta wrote:
>>>
>>> -- >8 --
>>>
>>> Subject: [PATCH] crypto: testmgr - Add additional AES-XTS vectors for covering
>>>  CTS (part II)
>>
>> Patchwork doesn't like it when you do this and it'll discard
>> your patch.  To make it into patchwork you need to put the new
>> Subject in the email headers.
>>
> 
> IMO, pretending that your XTS implementation is compliant by only
I've never said that.
Some parts are compliant, some are not.

> providing test vectors with the last 8 bytes of IV cleared is not the
> right fix for this issue. If you want to be compliant, you will need
It's not a fix.
It's adding test vectors which are not provided in the P1619 standard,
where "data unit sequence number" is at most 5B.

> to provide a s/w fallback for these cases.
> 
Yes, the plan is to:

-add 16B IV support for caam versions supporting it - caam Era 9+,
currently deployed in lx2160a and ls108a

-remove current 8B IV support and add s/w fallback for affected caam versions
I'd assume this could be done dynamically, i.e. depending on IV provided
in the crypto request to use either the caam engine or s/w fallback.

Horia

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-08 14:50                                               ` Horia Geanta
@ 2019-08-09  8:35                                                 ` Pascal Van Leeuwen
  0 siblings, 0 replies; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-08-09  8:35 UTC (permalink / raw)
  To: Horia Geanta, Ard Biesheuvel
  Cc: Milan Broz, Herbert Xu, dm-devel, linux-crypto

> -----Original Message-----
> From: Horia Geanta <horia.geanta@nxp.com>
> Sent: Thursday, August 8, 2019 4:50 PM
> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Ard Biesheuvel
> <ard.biesheuvel@linaro.org>
> Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-
> devel@redhat.com; linux-crypto@vger.kernel.org
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> On 8/7/2019 11:58 PM, Pascal Van Leeuwen wrote:
> >> -----Original Message-----
> >> From: Horia Geanta <horia.geanta@nxp.com>
> >> Sent: Wednesday, August 7, 2019 5:52 PM
> >> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>; Ard Biesheuvel
> >> <ard.biesheuvel@linaro.org>
> >> Cc: Milan Broz <gmazyland@gmail.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-
> >> devel@redhat.com; linux-crypto@vger.kernel.org
> >> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> >>
> >> On 7/26/2019 10:59 PM, Horia Geantă wrote:
> >>> On 7/26/2019 1:31 PM, Pascal Van Leeuwen wrote:
> >>>> Ok, find below a patch file that adds your vectors from the specification
> >>>> plus my set of additional vectors covering all CTS alignments combined
> >>>> with the block sizes you desired. Please note though that these vectors
> >>>> are from our in-house home-grown model so no warranties.
> >>> I've checked the test vectors against caam (HW + driver).
> >>>
> >>> Test vectors from IEEE 1619-2007 (i.e. up to and including "XTS-AES 18")
> >>> are fine.
> >>>
> >>> caam complains when /* Additional vectors to increase CTS coverage */
> >>> section starts:
> >>> alg: skcipher: xts-aes-caam encryption test failed (wrong result) on test vector 9,
> >> cfg="in-place"
> >>>
> >> I've nailed this down to a caam hw limitation.
> >> Except for lx2160a and ls1028a SoCs, all the (older) SoCs allow only for
> >> 8-byte wide IV (sector index).
> >>
> > I guess it's easy to say now, but I already suspected a problem with full 16
> > byte random IV's. A problem with CTS itself seemed implausible due to the base
> > vectors from the spec running fine and I did happen to notice that all
> > vectors from the spec only use up to the lower 40 bits of the sector number.
> > While my vectors randomize all 16 bytes.
> >
> > So I guess that means that 16 byte multiples (i.e. not needing CTS) with
> > full 16 byte sector numbers will probably also fail on caam HW ...
> >
> Yes, the limitation applies for all input sizes.
> 
> It's actually mentioned in the commit that added xts support few years back:
> c6415a6016bf ("crypto: caam - add support for acipher xts(aes)")
> 
>     sector index - HW limitation: CAAM device supports sector index of only
>     8 bytes to be used for sector index inside IV, instead of whole 16 bytes
>     received on request. This represents 2 ^ 64 = 16,777,216 Tera of possible
>     values for sector index.
> 
> > As for the tweak size, with very close scrutiny of the IEEE spec I actually
> > noticed some inconsistencies:
> >
> > - the text very clearly defines the tweak as 128 bit and starting from an
> > *arbitrary* non-negative integer, this is what I based my implementation on
> >
> > - all text examples and test vectors max out at 40 bits ... just examples,
> > but odd nonetheless (why 40 anyway?)
> >
> > - the example code fragment in Annex C actually has the S data unit number
> > input as an u64b, further commented as "64 bits" (but then loops 16 times to
> > convert it to a byte string ...)
> >
> The input I received from our HW design team was something like:
> 
> - some P1619 drafts used LRW (instead of XTS), where the tweak "T"
> was 16B-wide
> 
> - at some point P1619 drafts switched (and eventually standardized) XTS,
> where "T" is no longer the tweak - "i" is the (public) tweak, "T" being
> an intermediate (hidden) result in the encryption scheme
> 
> - since for XTS "i" is supposed to be the sector number,
> there is no need to support 16B values - 8B being deemed sufficient
> 
Actually, the specification does NOT define i as a straight sector
number, it specifies i to start from some "arbitrary non-negative integer"
with further values assigned "consecutively". (which does not even mandate
a binary increment, as long as it can be seen as some unbroken ordered 
sequence).

A predictable sector number being a potential attack vector, I can imagine
not wanting to start i from zero (although that's probably what most simple
implementations will do?).

> Agree, limiting "i" (XTS tweak) to 8B is out-of-spec - irrespective of the
> usefulness of the full 16B.
> That's why latest Freescale / NXP SoCs support 16B tweaks.
> 
> Thanks,
> Horia

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-09  7:44                                                     ` Horia Geanta
@ 2019-08-09 17:49                                                       ` Ard Biesheuvel
  2019-08-09 20:57                                                         ` Pascal Van Leeuwen
  0 siblings, 1 reply; 65+ messages in thread
From: Ard Biesheuvel @ 2019-08-09 17:49 UTC (permalink / raw)
  To: Horia Geanta
  Cc: Herbert Xu, Pascal Van Leeuwen, Milan Broz, dm-devel, linux-crypto

On Fri, 9 Aug 2019 at 10:44, Horia Geanta <horia.geanta@nxp.com> wrote:
>
> On 8/9/2019 9:45 AM, Ard Biesheuvel wrote:
> > On Fri, 9 Aug 2019 at 05:48, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> >>
> >> On Thu, Aug 08, 2019 at 06:01:49PM +0000, Horia Geanta wrote:
> >>>
> >>> -- >8 --
> >>>
> >>> Subject: [PATCH] crypto: testmgr - Add additional AES-XTS vectors for covering
> >>>  CTS (part II)
> >>
> >> Patchwork doesn't like it when you do this and it'll discard
> >> your patch.  To make it into patchwork you need to put the new
> >> Subject in the email headers.
> >>
> >
> > IMO, pretending that your XTS implementation is compliant by only
> I've never said that.
> Some parts are compliant, some are not.
>
> > providing test vectors with the last 8 bytes of IV cleared is not the
> > right fix for this issue. If you want to be compliant, you will need
> It's not a fix.
> It's adding test vectors which are not provided in the P1619 standard,
> where "data unit sequence number" is at most 5B.
>

Indeed. But I would prefer not to limit ourselves to 5 bytes of sector
numbers in the test vectors. However, we should obviously not add test
vectors that are known to cause breakages on hardware that works fine
in practice.

> > to provide a s/w fallback for these cases.
> >
> Yes, the plan is to:
>
> -add 16B IV support for caam versions supporting it - caam Era 9+,
> currently deployed in lx2160a and ls108a
>
> -remove current 8B IV support and add s/w fallback for affected caam versions
> I'd assume this could be done dynamically, i.e. depending on IV provided
> in the crypto request to use either the caam engine or s/w fallback.
>

Yes. If the IV received from the caller has bytes 8..15 cleared, you
use the limited XTS h/w implementation, otherwise you fall back to
xts(ecb-aes-caam..).

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-09 17:49                                                       ` Ard Biesheuvel
@ 2019-08-09 20:57                                                         ` Pascal Van Leeuwen
  2019-08-10  4:39                                                           ` Ard Biesheuvel
  0 siblings, 1 reply; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-08-09 20:57 UTC (permalink / raw)
  To: Ard Biesheuvel, Horia Geanta
  Cc: Herbert Xu, Milan Broz, dm-devel, linux-crypto

> -----Original Message-----
> From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Sent: Friday, August 9, 2019 7:49 PM
> To: Horia Geanta <horia.geanta@nxp.com>
> Cc: Herbert Xu <herbert@gondor.apana.org.au>; Pascal Van Leeuwen
> <pvanleeuwen@verimatrix.com>; Milan Broz <gmazyland@gmail.com>; dm-devel@redhat.com; linux-
> crypto@vger.kernel.org
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> On Fri, 9 Aug 2019 at 10:44, Horia Geanta <horia.geanta@nxp.com> wrote:
> >
> > On 8/9/2019 9:45 AM, Ard Biesheuvel wrote:
> > > On Fri, 9 Aug 2019 at 05:48, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> > >>
> > >> On Thu, Aug 08, 2019 at 06:01:49PM +0000, Horia Geanta wrote:
> > >>>
> > >>> -- >8 --
> > >>>
> > >>> Subject: [PATCH] crypto: testmgr - Add additional AES-XTS vectors for covering
> > >>>  CTS (part II)
> > >>
> > >> Patchwork doesn't like it when you do this and it'll discard
> > >> your patch.  To make it into patchwork you need to put the new
> > >> Subject in the email headers.
> > >>
> > >
> > > IMO, pretending that your XTS implementation is compliant by only
> > I've never said that.
> > Some parts are compliant, some are not.
> >
> > > providing test vectors with the last 8 bytes of IV cleared is not the
> > > right fix for this issue. If you want to be compliant, you will need
> > It's not a fix.
> > It's adding test vectors which are not provided in the P1619 standard,
> > where "data unit sequence number" is at most 5B.
> >
> 
> Indeed. But I would prefer not to limit ourselves to 5 bytes of sector
> numbers in the test vectors. However, we should obviously not add test
> vectors that are known to cause breakages on hardware that works fine
> in practice.
> 
Well, obviously, the full 16 byte sector number vectors fail on existing
CAAM hardware, which I do assume to work fine in practice. And you know
I'm not in favor of building all kinds of workarounds into the drivers.

Fact is, we know there are no current users that need more than 64 bits
of IV. Fact is also that having 64 bits of IV in the vectors is already
an improvement over the 40 bits in the original vectors. And unlike CTS, 
I am not aware of any real use case for more than 64 bits.
Finally, another fact is that limiting the *vectors* to 64 bits of IV
does not prohibit anyone from *using* a full 128 bit IV on an 
implementation that *does* support this. I would think most users of 
XTS, like dmcrypt, would allow you to specify the cra_drivername
explictly anyway, so just don't select legacy CAAM if you need that.
(heck, if it would be reading and writing its own data, and not need
compatibility with other implementations, it wouldn't even matter)

So yes, the specs are quite clear on the sector number being a full
128 bits. But that doesn't prevent us from specifying that the 
crypto API implementation currently only supports 64 bits, with the
remaining bits being forced to 0. We can always revisit that when
an actual use case for more than 64 bits arises ...

> > > to provide a s/w fallback for these cases.
> > >
> > Yes, the plan is to:
> >
> > -add 16B IV support for caam versions supporting it - caam Era 9+,
> > currently deployed in lx2160a and ls108a
> >
> > -remove current 8B IV support and add s/w fallback for affected caam versions
> > I'd assume this could be done dynamically, i.e. depending on IV provided
> > in the crypto request to use either the caam engine or s/w fallback.
> >
> 
> Yes. If the IV received from the caller has bytes 8..15 cleared, you
> use the limited XTS h/w implementation, otherwise you fall back to
> xts(ecb-aes-caam..).

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com


^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-09 20:57                                                         ` Pascal Van Leeuwen
@ 2019-08-10  4:39                                                           ` Ard Biesheuvel
  2019-08-11 11:12                                                             ` Milan Broz
  2019-08-11 21:15                                                             ` Pascal Van Leeuwen
  0 siblings, 2 replies; 65+ messages in thread
From: Ard Biesheuvel @ 2019-08-10  4:39 UTC (permalink / raw)
  To: Pascal Van Leeuwen
  Cc: Horia Geanta, Herbert Xu, Milan Broz, dm-devel, linux-crypto

On Fri, 9 Aug 2019 at 23:57, Pascal Van Leeuwen
<pvanleeuwen@verimatrix.com> wrote:
>
> > -----Original Message-----
> > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > Sent: Friday, August 9, 2019 7:49 PM
> > To: Horia Geanta <horia.geanta@nxp.com>
> > Cc: Herbert Xu <herbert@gondor.apana.org.au>; Pascal Van Leeuwen
> > <pvanleeuwen@verimatrix.com>; Milan Broz <gmazyland@gmail.com>; dm-devel@redhat.com; linux-
> > crypto@vger.kernel.org
> > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> >
> > On Fri, 9 Aug 2019 at 10:44, Horia Geanta <horia.geanta@nxp.com> wrote:
> > >
> > > On 8/9/2019 9:45 AM, Ard Biesheuvel wrote:
> > > > On Fri, 9 Aug 2019 at 05:48, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> > > >>
> > > >> On Thu, Aug 08, 2019 at 06:01:49PM +0000, Horia Geanta wrote:
> > > >>>
> > > >>> -- >8 --
> > > >>>
> > > >>> Subject: [PATCH] crypto: testmgr - Add additional AES-XTS vectors for covering
> > > >>>  CTS (part II)
> > > >>
> > > >> Patchwork doesn't like it when you do this and it'll discard
> > > >> your patch.  To make it into patchwork you need to put the new
> > > >> Subject in the email headers.
> > > >>
> > > >
> > > > IMO, pretending that your XTS implementation is compliant by only
> > > I've never said that.
> > > Some parts are compliant, some are not.
> > >
> > > > providing test vectors with the last 8 bytes of IV cleared is not the
> > > > right fix for this issue. If you want to be compliant, you will need
> > > It's not a fix.
> > > It's adding test vectors which are not provided in the P1619 standard,
> > > where "data unit sequence number" is at most 5B.
> > >
> >
> > Indeed. But I would prefer not to limit ourselves to 5 bytes of sector
> > numbers in the test vectors. However, we should obviously not add test
> > vectors that are known to cause breakages on hardware that works fine
> > in practice.
> >
> Well, obviously, the full 16 byte sector number vectors fail on existing
> CAAM hardware, which I do assume to work fine in practice. And you know
> I'm not in favor of building all kinds of workarounds into the drivers.
>
> Fact is, we know there are no current users that need more than 64 bits
> of IV. Fact is also that having 64 bits of IV in the vectors is already
> an improvement over the 40 bits in the original vectors. And unlike CTS,
> I am not aware of any real use case for more than 64 bits.
> Finally, another fact is that limiting the *vectors* to 64 bits of IV
> does not prohibit anyone from *using* a full 128 bit IV on an
> implementation that *does* support this. I would think most users of
> XTS, like dmcrypt, would allow you to specify the cra_drivername
> explictly anyway, so just don't select legacy CAAM if you need that.
> (heck, if it would be reading and writing its own data, and not need
> compatibility with other implementations, it wouldn't even matter)
>
> So yes, the specs are quite clear on the sector number being a full
> 128 bits. But that doesn't prevent us from specifying that the
> crypto API implementation currently only supports 64 bits, with the
> remaining bits being forced to 0. We can always revisit that when
> an actual use case for more than 64 bits arises ...
>

You have got it completely backwards:

CTS has never worked in any kernel implementation, so regardless of
what the spec says, supporting it in the kernel is not a high priority
issue whichever way you put it. Now is the first time anyone has asked
for it in 12 years, and only because someone spotted a deviation
between a h/w and a s/w implementation, not because anyone tried to
use it and failed. (Note that passing anything other than a multiple
of the block size will cause an error rather than fail silently)

Truncated IVs are a huge issue, since we already expose the correct
API via AF_ALG (without any restrictions on how many of the IV bits
are populated), and apparently, if your AF_ALG request for xts(aes)
happens to be fulfilled by the CAAM driver and your implementation
uses more than 64 bits for the IV, the top bits get truncated silently
and your data might get eaten.

In my experience, users tend to care more about the latter than the former.


> > > > to provide a s/w fallback for these cases.
> > > >
> > > Yes, the plan is to:
> > >
> > > -add 16B IV support for caam versions supporting it - caam Era 9+,
> > > currently deployed in lx2160a and ls108a
> > >
> > > -remove current 8B IV support and add s/w fallback for affected caam versions
> > > I'd assume this could be done dynamically, i.e. depending on IV provided
> > > in the crypto request to use either the caam engine or s/w fallback.
> > >
> >
> > Yes. If the IV received from the caller has bytes 8..15 cleared, you
> > use the limited XTS h/w implementation, otherwise you fall back to
> > xts(ecb-aes-caam..).
>
> Regards,
> Pascal van Leeuwen
> Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
> www.insidesecure.com
>

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-10  4:39                                                           ` Ard Biesheuvel
@ 2019-08-11 11:12                                                             ` Milan Broz
  2019-08-11 20:34                                                               ` Eric Biggers
  2019-08-11 21:29                                                               ` Pascal Van Leeuwen
  2019-08-11 21:15                                                             ` Pascal Van Leeuwen
  1 sibling, 2 replies; 65+ messages in thread
From: Milan Broz @ 2019-08-11 11:12 UTC (permalink / raw)
  To: Ard Biesheuvel, Pascal Van Leeuwen
  Cc: Horia Geanta, Herbert Xu, dm-devel, linux-crypto

On 10/08/2019 06:39, Ard Biesheuvel wrote:
> Truncated IVs are a huge issue, since we already expose the correct
> API via AF_ALG (without any restrictions on how many of the IV bits
> are populated), and apparently, if your AF_ALG request for xts(aes)
> happens to be fulfilled by the CAAM driver and your implementation
> uses more than 64 bits for the IV, the top bits get truncated silently
> and your data might get eaten.

Actually, I think we have already serious problem with in in kernel (no AF_ALG needed).

I do not have the hardware, but please could you check that dm-crypt big-endian IV
(plain64be) produces the same output on CAAM?

It is 64bit IV, but big-endian and we use size of cipher block (16bytes) here,
so the first 8 bytes are zero in this case.

I would expect data corruption in comparison to generic implementation,
if it supports only the first 64bit...

Try this:

# create small null device of 8 sectors,  we use zeroes as fixed ciphertext
dmsetup create zero --table "0 8 zero"

# create crypt device on top of it (with some key), using plain64be IV
dmsetup create crypt --table "0 8 crypt aes-xts-plain64be e8cfa3dbfe373b536be43c5637387786c01be00ba5f730aacb039e86f3eb72f3 0 /dev/mapper/zero 0"

# and compare it with and without your driver, this is what I get here:
# sha256sum /dev/mapper/crypt 
532f71198d0d84d823b8e410738c6f43bc3e149d844dd6d37fa5b36d150501e1  /dev/mapper/crypt
# dmsetup remove crypt

You can try little-endian version (plain64), this should always work even with CAAM
dmsetup create crypt --table "0 8 crypt aes-xts-plain64 e8cfa3dbfe373b536be43c5637387786c01be00ba5f730aacb039e86f3eb72f3 0 /dev/mapper/zero 0"

# sha256sum /dev/mapper/crypt 
f17abd27dedee4e539758eabdb6c15fa619464b509cf55f16433e6a25da42857  /dev/mapper/crypt
# dmsetup remove crypt

# dmsetup remove zero


If you get different plaintext in the first case, your driver is actually creating
data corruption in this configuration and it should be fixed!
(Only the first sector must be the same, because it has IV == 0.)

Milan

p.s.
If you ask why we have this IV, it was added per request to allow map some chipset-based
encrypted drives directly. I guess it is used for some data forensic things.

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-11 11:12                                                             ` Milan Broz
@ 2019-08-11 20:34                                                               ` Eric Biggers
  2019-08-11 21:39                                                                 ` Pascal Van Leeuwen
  2019-08-11 21:29                                                               ` Pascal Van Leeuwen
  1 sibling, 1 reply; 65+ messages in thread
From: Eric Biggers @ 2019-08-11 20:34 UTC (permalink / raw)
  To: Milan Broz
  Cc: Ard Biesheuvel, Pascal Van Leeuwen, dm-devel, Herbert Xu,
	Horia Geanta, linux-crypto

On Sun, Aug 11, 2019 at 01:12:56PM +0200, Milan Broz wrote:
> On 10/08/2019 06:39, Ard Biesheuvel wrote:
> > Truncated IVs are a huge issue, since we already expose the correct
> > API via AF_ALG (without any restrictions on how many of the IV bits
> > are populated), and apparently, if your AF_ALG request for xts(aes)
> > happens to be fulfilled by the CAAM driver and your implementation
> > uses more than 64 bits for the IV, the top bits get truncated silently
> > and your data might get eaten.
> 
> Actually, I think we have already serious problem with in in kernel (no AF_ALG needed).
> 
> I do not have the hardware, but please could you check that dm-crypt big-endian IV
> (plain64be) produces the same output on CAAM?
> 
> It is 64bit IV, but big-endian and we use size of cipher block (16bytes) here,
> so the first 8 bytes are zero in this case.
> 
> I would expect data corruption in comparison to generic implementation,
> if it supports only the first 64bit...
> 
> Try this:
> 
> # create small null device of 8 sectors,  we use zeroes as fixed ciphertext
> dmsetup create zero --table "0 8 zero"
> 
> # create crypt device on top of it (with some key), using plain64be IV
> dmsetup create crypt --table "0 8 crypt aes-xts-plain64be e8cfa3dbfe373b536be43c5637387786c01be00ba5f730aacb039e86f3eb72f3 0 /dev/mapper/zero 0"
> 
> # and compare it with and without your driver, this is what I get here:
> # sha256sum /dev/mapper/crypt 
> 532f71198d0d84d823b8e410738c6f43bc3e149d844dd6d37fa5b36d150501e1  /dev/mapper/crypt
> # dmsetup remove crypt
> 
> You can try little-endian version (plain64), this should always work even with CAAM
> dmsetup create crypt --table "0 8 crypt aes-xts-plain64 e8cfa3dbfe373b536be43c5637387786c01be00ba5f730aacb039e86f3eb72f3 0 /dev/mapper/zero 0"
> 
> # sha256sum /dev/mapper/crypt 
> f17abd27dedee4e539758eabdb6c15fa619464b509cf55f16433e6a25da42857  /dev/mapper/crypt
> # dmsetup remove crypt
> 
> # dmsetup remove zero
> 
> 
> If you get different plaintext in the first case, your driver is actually creating
> data corruption in this configuration and it should be fixed!
> (Only the first sector must be the same, because it has IV == 0.)
> 
> Milan
> 
> p.s.
> If you ask why we have this IV, it was added per request to allow map some chipset-based
> encrypted drives directly. I guess it is used for some data forensic things.
> 

Also, if the CAAM driver is really truncating the IV for "xts(aes)", it should
already be failing the extra crypto self-tests, since the fuzz testing in
test_skcipher_vs_generic_impl() uses random IVs.

- Eric

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-10  4:39                                                           ` Ard Biesheuvel
  2019-08-11 11:12                                                             ` Milan Broz
@ 2019-08-11 21:15                                                             ` Pascal Van Leeuwen
  2019-08-11 22:24                                                               ` Ard Biesheuvel
  1 sibling, 1 reply; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-08-11 21:15 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Horia Geanta, Herbert Xu, Milan Broz, dm-devel, linux-crypto

> -----Original Message-----
> From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Sent: Saturday, August 10, 2019 6:40 AM
> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> Cc: Horia Geanta <horia.geanta@nxp.com>; Herbert Xu <herbert@gondor.apana.org.au>; Milan Broz
> <gmazyland@gmail.com>; dm-devel@redhat.com; linux-crypto@vger.kernel.org
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> On Fri, 9 Aug 2019 at 23:57, Pascal Van Leeuwen
> <pvanleeuwen@verimatrix.com> wrote:
> >
> > > -----Original Message-----
> > > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > Sent: Friday, August 9, 2019 7:49 PM
> > > To: Horia Geanta <horia.geanta@nxp.com>
> > > Cc: Herbert Xu <herbert@gondor.apana.org.au>; Pascal Van Leeuwen
> > > <pvanleeuwen@verimatrix.com>; Milan Broz <gmazyland@gmail.com>; dm-devel@redhat.com;
> linux-
> > > crypto@vger.kernel.org
> > > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> > >
> > > On Fri, 9 Aug 2019 at 10:44, Horia Geanta <horia.geanta@nxp.com> wrote:
> > > >
> > > > On 8/9/2019 9:45 AM, Ard Biesheuvel wrote:
> > > > > On Fri, 9 Aug 2019 at 05:48, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> > > > >>
> > > > >> On Thu, Aug 08, 2019 at 06:01:49PM +0000, Horia Geanta wrote:
> > > > >>>
> > > > >>> -- >8 --
> > > > >>>
> > > > >>> Subject: [PATCH] crypto: testmgr - Add additional AES-XTS vectors for covering
> > > > >>>  CTS (part II)
> > > > >>
> > > > >> Patchwork doesn't like it when you do this and it'll discard
> > > > >> your patch.  To make it into patchwork you need to put the new
> > > > >> Subject in the email headers.
> > > > >>
> > > > >
> > > > > IMO, pretending that your XTS implementation is compliant by only
> > > > I've never said that.
> > > > Some parts are compliant, some are not.
> > > >
> > > > > providing test vectors with the last 8 bytes of IV cleared is not the
> > > > > right fix for this issue. If you want to be compliant, you will need
> > > > It's not a fix.
> > > > It's adding test vectors which are not provided in the P1619 standard,
> > > > where "data unit sequence number" is at most 5B.
> > > >
> > >
> > > Indeed. But I would prefer not to limit ourselves to 5 bytes of sector
> > > numbers in the test vectors. However, we should obviously not add test
> > > vectors that are known to cause breakages on hardware that works fine
> > > in practice.
> > >
> > Well, obviously, the full 16 byte sector number vectors fail on existing
> > CAAM hardware, which I do assume to work fine in practice. And you know
> > I'm not in favor of building all kinds of workarounds into the drivers.
> >
> > Fact is, we know there are no current users that need more than 64 bits
> > of IV. Fact is also that having 64 bits of IV in the vectors is already
> > an improvement over the 40 bits in the original vectors. And unlike CTS,
> > I am not aware of any real use case for more than 64 bits.
> > Finally, another fact is that limiting the *vectors* to 64 bits of IV
> > does not prohibit anyone from *using* a full 128 bit IV on an
> > implementation that *does* support this. I would think most users of
> > XTS, like dmcrypt, would allow you to specify the cra_drivername
> > explictly anyway, so just don't select legacy CAAM if you need that.
> > (heck, if it would be reading and writing its own data, and not need
> > compatibility with other implementations, it wouldn't even matter)
> >
> > So yes, the specs are quite clear on the sector number being a full
> > 128 bits. But that doesn't prevent us from specifying that the
> > crypto API implementation currently only supports 64 bits, with the
> > remaining bits being forced to 0. We can always revisit that when
> > an actual use case for more than 64 bits arises ...
> >
> 
> You have got it completely backwards:
> 
> CTS has never worked in any kernel implementation, so regardless of
> what the spec says, supporting it in the kernel is not a high priority
> issue whichever way you put it. 
>
I never said it was a high priority, I merely pointed out it's not spec 
compliant. Apparently, you feel that that's only important insofar that
it matches current kernel use cases?

Anyway, as far as I understand, there are no users that need more than 
64 bits of IV in the kernel (i.e. dmcrypt uses only 64 bits), so I see
no fundamental difference with CTS except that most(?) implementations
possibly already "accidentally" (since unverified!) did it correctly.

Not that I have any interest in restricting the IV size: "my" hardware
handles full 128 bit IV's just fine. So why do I even bother ... :-)

> Now is the first time anyone has asked
> for it in 12 years, and only because someone spotted a deviation
> between a h/w and a s/w implementation, not because anyone tried to
> use it and failed. (Note that passing anything other than a multiple
> of the block size will cause an error rather than fail silently)
> 
Yes, failing silently is not such a good idea, I think we agree on that.  
Although we also need  to keep in mind that that's exactly what the CAAM 
driver has been doing all those years, and, before my vectors, nobody
noticed or cared. Without my involvement, this would have probably gone
unnoticed for many years to come (so I feel some responsibility ;-).

> Truncated IVs are a huge issue, since we already expose the correct
> API via AF_ALG (without any restrictions on how many of the IV bits
> are populated), and apparently, if your AF_ALG request for xts(aes)
> happens to be fulfilled by the CAAM driver and your implementation
> uses more than 64 bits for the IV, the top bits get truncated silently
> and your data might get eaten.
> 
Apparently, not such a "huge" issue at all, see previous remark.

As a precaution, the CAAM driver could return -EINVAL if the upper IV
bytes are non-zero. But then testmgr would have to do less strict error 
return code checking so we don't force this upon drivers that CAN do it.

Implementing a full SW fallback for that in the driver just seems like
massive overkill, as you normally specify the driver for dmcrypt explictly
anyway (or at least, you can do that if the default fails).

I don't like the idea of HW drivers doing SW fallbacks because it clouds
the whole picture of what is actually done by a certain HW device.

> In my experience, users tend to care more about the latter than the former.
> 
> 
> > > > > to provide a s/w fallback for these cases.
> > > > >
> > > > Yes, the plan is to:
> > > >
> > > > -add 16B IV support for caam versions supporting it - caam Era 9+,
> > > > currently deployed in lx2160a and ls108a
> > > >
> > > > -remove current 8B IV support and add s/w fallback for affected caam versions
> > > > I'd assume this could be done dynamically, i.e. depending on IV provided
> > > > in the crypto request to use either the caam engine or s/w fallback.
> > > >
> > >
> > > Yes. If the IV received from the caller has bytes 8..15 cleared, you
> > > use the limited XTS h/w implementation, otherwise you fall back to
> > > xts(ecb-aes-caam..).
> >
> > Regards,
> > Pascal van Leeuwen
> > Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
> > www.insidesecure.com
> >

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com


^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-11 11:12                                                             ` Milan Broz
  2019-08-11 20:34                                                               ` Eric Biggers
@ 2019-08-11 21:29                                                               ` Pascal Van Leeuwen
  2019-08-12  4:51                                                                 ` Herbert Xu
  1 sibling, 1 reply; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-08-11 21:29 UTC (permalink / raw)
  To: Milan Broz, Ard Biesheuvel
  Cc: Horia Geanta, Herbert Xu, dm-devel, linux-crypto

> -----Original Message-----
> From: Milan Broz <gmazyland@gmail.com>
> Sent: Sunday, August 11, 2019 1:13 PM
> To: Ard Biesheuvel <ard.biesheuvel@linaro.org>; Pascal Van Leeuwen
> <pvanleeuwen@verimatrix.com>
> Cc: Horia Geanta <horia.geanta@nxp.com>; Herbert Xu <herbert@gondor.apana.org.au>; dm-
> devel@redhat.com; linux-crypto@vger.kernel.org
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> On 10/08/2019 06:39, Ard Biesheuvel wrote:
> > Truncated IVs are a huge issue, since we already expose the correct
> > API via AF_ALG (without any restrictions on how many of the IV bits
> > are populated), and apparently, if your AF_ALG request for xts(aes)
> > happens to be fulfilled by the CAAM driver and your implementation
> > uses more than 64 bits for the IV, the top bits get truncated silently
> > and your data might get eaten.
> 
> Actually, I think we have already serious problem with in in kernel (no AF_ALG needed).
> 
> I do not have the hardware, but please could you check that dm-crypt big-endian IV
> (plain64be) produces the same output on CAAM?
> 
> It is 64bit IV, but big-endian and we use size of cipher block (16bytes) here,
> so the first 8 bytes are zero in this case.
> 
> I would expect data corruption in comparison to generic implementation,
> if it supports only the first 64bit...
> 
> Try this:
> 
> # create small null device of 8 sectors,  we use zeroes as fixed ciphertext
> dmsetup create zero --table "0 8 zero"
> 
> # create crypt device on top of it (with some key), using plain64be IV
> dmsetup create crypt --table "0 8 crypt aes-xts-plain64be
> e8cfa3dbfe373b536be43c5637387786c01be00ba5f730aacb039e86f3eb72f3 0 /dev/mapper/zero 0"
> 
> # and compare it with and without your driver, this is what I get here:
> # sha256sum /dev/mapper/crypt
> 532f71198d0d84d823b8e410738c6f43bc3e149d844dd6d37fa5b36d150501e1  /dev/mapper/crypt
> # dmsetup remove crypt
> 
> You can try little-endian version (plain64), this should always work even with CAAM
> dmsetup create crypt --table "0 8 crypt aes-xts-plain64
> e8cfa3dbfe373b536be43c5637387786c01be00ba5f730aacb039e86f3eb72f3 0 /dev/mapper/zero 0"
> 
> # sha256sum /dev/mapper/crypt
> f17abd27dedee4e539758eabdb6c15fa619464b509cf55f16433e6a25da42857  /dev/mapper/crypt
> # dmsetup remove crypt
> 
> # dmsetup remove zero
> 
> 
> If you get different plaintext in the first case, your driver is actually creating
> data corruption in this configuration and it should be fixed!
> (Only the first sector must be the same, because it has IV == 0.)
> 
It will very likely fail with that CAAM h/w, but that only proves that you
should not use plain64be IV's together with CAAM h/w. Which should be
entirely avoidable in real life unless you have some unlikely requirement
to move physical encrypted diks from one system (without CAAM h/w and needing
these plain64be IV's for some reason) to another system (with CAAM h/w) and be 
able to decrypt them there.

Formally, these plain64be IV's are actually WRONG, since the XTS specification is
very clear on the byte order of the sector number ("little endian byte array").


> Milan
> 
> p.s.
> If you ask why we have this IV, it was added per request to allow map some chipset-based
> encrypted drives directly. I guess it is used for some data forensic things.
>
Sounds like someone got the endianness wrong ;-) 

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-11 20:34                                                               ` Eric Biggers
@ 2019-08-11 21:39                                                                 ` Pascal Van Leeuwen
  0 siblings, 0 replies; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-08-11 21:39 UTC (permalink / raw)
  To: Eric Biggers, Milan Broz
  Cc: Ard Biesheuvel, dm-devel, Herbert Xu, Horia Geanta, linux-crypto

> -----Original Message-----
> From: Eric Biggers <ebiggers@kernel.org>
> Sent: Sunday, August 11, 2019 10:34 PM
> To: Milan Broz <gmazyland@gmail.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>; Pascal Van Leeuwen
> <pvanleeuwen@verimatrix.com>; dm-devel@redhat.com; Herbert Xu <herbert@gondor.apana.org.au>;
> Horia Geanta <horia.geanta@nxp.com>; linux-crypto@vger.kernel.org
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> On Sun, Aug 11, 2019 at 01:12:56PM +0200, Milan Broz wrote:
> > On 10/08/2019 06:39, Ard Biesheuvel wrote:
> > > Truncated IVs are a huge issue, since we already expose the correct
> > > API via AF_ALG (without any restrictions on how many of the IV bits
> > > are populated), and apparently, if your AF_ALG request for xts(aes)
> > > happens to be fulfilled by the CAAM driver and your implementation
> > > uses more than 64 bits for the IV, the top bits get truncated silently
> > > and your data might get eaten.
> >
> > Actually, I think we have already serious problem with in in kernel (no AF_ALG needed).
> >
> > I do not have the hardware, but please could you check that dm-crypt big-endian IV
> > (plain64be) produces the same output on CAAM?
> >
> > It is 64bit IV, but big-endian and we use size of cipher block (16bytes) here,
> > so the first 8 bytes are zero in this case.
> >
> > I would expect data corruption in comparison to generic implementation,
> > if it supports only the first 64bit...
> >
> > Try this:
> >
> > # create small null device of 8 sectors,  we use zeroes as fixed ciphertext
> > dmsetup create zero --table "0 8 zero"
> >
> > # create crypt device on top of it (with some key), using plain64be IV
> > dmsetup create crypt --table "0 8 crypt aes-xts-plain64be
> e8cfa3dbfe373b536be43c5637387786c01be00ba5f730aacb039e86f3eb72f3 0 /dev/mapper/zero 0"
> >
> > # and compare it with and without your driver, this is what I get here:
> > # sha256sum /dev/mapper/crypt
> > 532f71198d0d84d823b8e410738c6f43bc3e149d844dd6d37fa5b36d150501e1  /dev/mapper/crypt
> > # dmsetup remove crypt
> >
> > You can try little-endian version (plain64), this should always work even with CAAM
> > dmsetup create crypt --table "0 8 crypt aes-xts-plain64
> e8cfa3dbfe373b536be43c5637387786c01be00ba5f730aacb039e86f3eb72f3 0 /dev/mapper/zero 0"
> >
> > # sha256sum /dev/mapper/crypt
> > f17abd27dedee4e539758eabdb6c15fa619464b509cf55f16433e6a25da42857  /dev/mapper/crypt
> > # dmsetup remove crypt
> >
> > # dmsetup remove zero
> >
> >
> > If you get different plaintext in the first case, your driver is actually creating
> > data corruption in this configuration and it should be fixed!
> > (Only the first sector must be the same, because it has IV == 0.)
> >
> > Milan
> >
> > p.s.
> > If you ask why we have this IV, it was added per request to allow map some chipset-based
> > encrypted drives directly. I guess it is used for some data forensic things.
> >
> 
> Also, if the CAAM driver is really truncating the IV for "xts(aes)", it should
> already be failing the extra crypto self-tests, since the fuzz testing in
> test_skcipher_vs_generic_impl() uses random IVs.
> 
> - Eric
>
Yes, good point. Although that is only seen during development and not 
during "normal" use ...


Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-11 21:15                                                             ` Pascal Van Leeuwen
@ 2019-08-11 22:24                                                               ` Ard Biesheuvel
  2019-08-12  1:04                                                                 ` Pascal Van Leeuwen
  0 siblings, 1 reply; 65+ messages in thread
From: Ard Biesheuvel @ 2019-08-11 22:24 UTC (permalink / raw)
  To: Pascal Van Leeuwen
  Cc: Horia Geanta, Herbert Xu, Milan Broz, dm-devel, linux-crypto

On Mon, 12 Aug 2019 at 00:15, Pascal Van Leeuwen
<pvanleeuwen@verimatrix.com> wrote:
>
> > -----Original Message-----
> > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > Sent: Saturday, August 10, 2019 6:40 AM
> > To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> > Cc: Horia Geanta <horia.geanta@nxp.com>; Herbert Xu <herbert@gondor.apana.org.au>; Milan Broz
> > <gmazyland@gmail.com>; dm-devel@redhat.com; linux-crypto@vger.kernel.org
> > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> >
> > On Fri, 9 Aug 2019 at 23:57, Pascal Van Leeuwen
> > <pvanleeuwen@verimatrix.com> wrote:
> > >
> > > > -----Original Message-----
> > > > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > > Sent: Friday, August 9, 2019 7:49 PM
> > > > To: Horia Geanta <horia.geanta@nxp.com>
> > > > Cc: Herbert Xu <herbert@gondor.apana.org.au>; Pascal Van Leeuwen
> > > > <pvanleeuwen@verimatrix.com>; Milan Broz <gmazyland@gmail.com>; dm-devel@redhat.com;
> > linux-
> > > > crypto@vger.kernel.org
> > > > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> > > >
> > > > On Fri, 9 Aug 2019 at 10:44, Horia Geanta <horia.geanta@nxp.com> wrote:
> > > > >
> > > > > On 8/9/2019 9:45 AM, Ard Biesheuvel wrote:
> > > > > > On Fri, 9 Aug 2019 at 05:48, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> > > > > >>
> > > > > >> On Thu, Aug 08, 2019 at 06:01:49PM +0000, Horia Geanta wrote:
> > > > > >>>
> > > > > >>> -- >8 --
> > > > > >>>
> > > > > >>> Subject: [PATCH] crypto: testmgr - Add additional AES-XTS vectors for covering
> > > > > >>>  CTS (part II)
> > > > > >>
> > > > > >> Patchwork doesn't like it when you do this and it'll discard
> > > > > >> your patch.  To make it into patchwork you need to put the new
> > > > > >> Subject in the email headers.
> > > > > >>
> > > > > >
> > > > > > IMO, pretending that your XTS implementation is compliant by only
> > > > > I've never said that.
> > > > > Some parts are compliant, some are not.
> > > > >
> > > > > > providing test vectors with the last 8 bytes of IV cleared is not the
> > > > > > right fix for this issue. If you want to be compliant, you will need
> > > > > It's not a fix.
> > > > > It's adding test vectors which are not provided in the P1619 standard,
> > > > > where "data unit sequence number" is at most 5B.
> > > > >
> > > >
> > > > Indeed. But I would prefer not to limit ourselves to 5 bytes of sector
> > > > numbers in the test vectors. However, we should obviously not add test
> > > > vectors that are known to cause breakages on hardware that works fine
> > > > in practice.
> > > >
> > > Well, obviously, the full 16 byte sector number vectors fail on existing
> > > CAAM hardware, which I do assume to work fine in practice. And you know
> > > I'm not in favor of building all kinds of workarounds into the drivers.
> > >
> > > Fact is, we know there are no current users that need more than 64 bits
> > > of IV. Fact is also that having 64 bits of IV in the vectors is already
> > > an improvement over the 40 bits in the original vectors. And unlike CTS,
> > > I am not aware of any real use case for more than 64 bits.
> > > Finally, another fact is that limiting the *vectors* to 64 bits of IV
> > > does not prohibit anyone from *using* a full 128 bit IV on an
> > > implementation that *does* support this. I would think most users of
> > > XTS, like dmcrypt, would allow you to specify the cra_drivername
> > > explictly anyway, so just don't select legacy CAAM if you need that.
> > > (heck, if it would be reading and writing its own data, and not need
> > > compatibility with other implementations, it wouldn't even matter)
> > >
> > > So yes, the specs are quite clear on the sector number being a full
> > > 128 bits. But that doesn't prevent us from specifying that the
> > > crypto API implementation currently only supports 64 bits, with the
> > > remaining bits being forced to 0. We can always revisit that when
> > > an actual use case for more than 64 bits arises ...
> > >
> >
> > You have got it completely backwards:
> >
> > CTS has never worked in any kernel implementation, so regardless of
> > what the spec says, supporting it in the kernel is not a high priority
> > issue whichever way you put it.
> >
> I never said it was a high priority, I merely pointed out it's not spec
> compliant. Apparently, you feel that that's only important insofar that
> it matches current kernel use cases?
>
> Anyway, as far as I understand, there are no users that need more than
> 64 bits of IV in the kernel (i.e. dmcrypt uses only 64 bits), so I see
> no fundamental difference with CTS except that most(?) implementations
> possibly already "accidentally" (since unverified!) did it correctly.
>

Well, as Milan points out, dm-crypt may use the upper bits as well,
but the point I was making was about the userland interface. If you
care about spec compliance for the sake of it, I don't understand why
you dismiss a broken userland->kernel interface that silently fails
and corrupts your data as something we should just live with, and just
assume nobody will ever be silly enough to use it the way the spec
describes it, and use all the available IV bits.

> Not that I have any interest in restricting the IV size: "my" hardware
> handles full 128 bit IV's just fine. So why do I even bother ... :-)
>
> > Now is the first time anyone has asked
> > for it in 12 years, and only because someone spotted a deviation
> > between a h/w and a s/w implementation, not because anyone tried to
> > use it and failed. (Note that passing anything other than a multiple
> > of the block size will cause an error rather than fail silently)
> >
> Yes, failing silently is not such a good idea, I think we agree on that.
> Although we also need  to keep in mind that that's exactly what the CAAM
> driver has been doing all those years, and, before my vectors, nobody
> noticed or cared.

True. But now that the cat is out of the bag, we have an obligation to
our users to ensure that a known problem does not corrupt their data.

> Without my involvement, this would have probably gone
> unnoticed for many years to come (so I feel some responsibility ;-).
>
> > Truncated IVs are a huge issue, since we already expose the correct
> > API via AF_ALG (without any restrictions on how many of the IV bits
> > are populated), and apparently, if your AF_ALG request for xts(aes)
> > happens to be fulfilled by the CAAM driver and your implementation
> > uses more than 64 bits for the IV, the top bits get truncated silently
> > and your data might get eaten.
> >
> Apparently, not such a "huge" issue at all, see previous remark.
>
> As a precaution, the CAAM driver could return -EINVAL if the upper IV
> bytes are non-zero.

Then we'd still have two implementations that behave differently, and
this is not how abstractions work. If a driver implements xts(aes),
then it should produce the same output (and return value) for every
input.

> But then testmgr would have to do less strict error
> return code checking so we don't force this upon drivers that CAN do it.
>
> Implementing a full SW fallback for that in the driver just seems like
> massive overkill, as you normally specify the driver for dmcrypt explictly
> anyway (or at least, you can do that if the default fails).
>

How would you find out if the default fails? Either it corrupts your
data, or it fails with an error that is specific to CAAM, and so we
would have to update the dm-crypt code to deal with en/decryption
failures that can only occur in this particular case. This is
*exactly* the thing I argued against before, when we discussed zero
length inputs: the reason we want the workaround in the driver is
because it is contained, and its peculiarities don't leak into other
layers.

> I don't like the idea of HW drivers doing SW fallbacks because it clouds
> the whole picture of what is actually done by a certain HW device.
>

Well, the reality is that we OS guys get to clean up after the h/w
guys all the time. I agree that explicit failure is better than silent
corruption, but that still means there are corner cases where things
just don't work. So the only acceptable way to me is to fix this with
a s/w workaround, and as I already pointed out, this could simply be
the xts template wrapped around the accelerated ECB implementation
exposed by the driver.


> > In my experience, users tend to care more about the latter than the former.
> >
> >
> > > > > > to provide a s/w fallback for these cases.
> > > > > >
> > > > > Yes, the plan is to:
> > > > >
> > > > > -add 16B IV support for caam versions supporting it - caam Era 9+,
> > > > > currently deployed in lx2160a and ls108a
> > > > >
> > > > > -remove current 8B IV support and add s/w fallback for affected caam versions
> > > > > I'd assume this could be done dynamically, i.e. depending on IV provided
> > > > > in the crypto request to use either the caam engine or s/w fallback.
> > > > >
> > > >
> > > > Yes. If the IV received from the caller has bytes 8..15 cleared, you
> > > > use the limited XTS h/w implementation, otherwise you fall back to
> > > > xts(ecb-aes-caam..).
> > >
> > > Regards,
> > > Pascal van Leeuwen
> > > Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
> > > www.insidesecure.com
> > >
>
> Regards,
> Pascal van Leeuwen
> Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
> www.insidesecure.com
>

^ permalink raw reply	[flat|nested] 65+ messages in thread

* RE: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-11 22:24                                                               ` Ard Biesheuvel
@ 2019-08-12  1:04                                                                 ` Pascal Van Leeuwen
  0 siblings, 0 replies; 65+ messages in thread
From: Pascal Van Leeuwen @ 2019-08-12  1:04 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Horia Geanta, Herbert Xu, Milan Broz, dm-devel, linux-crypto

> -----Original Message-----
> From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Sent: Monday, August 12, 2019 12:24 AM
> To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> Cc: Horia Geanta <horia.geanta@nxp.com>; Herbert Xu <herbert@gondor.apana.org.au>; Milan Broz
> <gmazyland@gmail.com>; dm-devel@redhat.com; linux-crypto@vger.kernel.org
> Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> 
> On Mon, 12 Aug 2019 at 00:15, Pascal Van Leeuwen
> <pvanleeuwen@verimatrix.com> wrote:
> >
> > > -----Original Message-----
> > > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > Sent: Saturday, August 10, 2019 6:40 AM
> > > To: Pascal Van Leeuwen <pvanleeuwen@verimatrix.com>
> > > Cc: Horia Geanta <horia.geanta@nxp.com>; Herbert Xu <herbert@gondor.apana.org.au>; Milan
> Broz
> > > <gmazyland@gmail.com>; dm-devel@redhat.com; linux-crypto@vger.kernel.org
> > > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> > >
> > > On Fri, 9 Aug 2019 at 23:57, Pascal Van Leeuwen
> > > <pvanleeuwen@verimatrix.com> wrote:
> > > >
> > > > > -----Original Message-----
> > > > > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > > > Sent: Friday, August 9, 2019 7:49 PM
> > > > > To: Horia Geanta <horia.geanta@nxp.com>
> > > > > Cc: Herbert Xu <herbert@gondor.apana.org.au>; Pascal Van Leeuwen
> > > > > <pvanleeuwen@verimatrix.com>; Milan Broz <gmazyland@gmail.com>; dm-devel@redhat.com;
> > > linux-
> > > > > crypto@vger.kernel.org
> > > > > Subject: Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
> > > > >
> > > > > On Fri, 9 Aug 2019 at 10:44, Horia Geanta <horia.geanta@nxp.com> wrote:
> > > > > >
> > > > > > On 8/9/2019 9:45 AM, Ard Biesheuvel wrote:
> > > > > > > On Fri, 9 Aug 2019 at 05:48, Herbert Xu <herbert@gondor.apana.org.au> wrote:
> > > > > > >>
> > > > > > >> On Thu, Aug 08, 2019 at 06:01:49PM +0000, Horia Geanta wrote:
> > > > > > >>>
> > > > > > >>> -- >8 --
> > > > > > >>>
> > > > > > >>> Subject: [PATCH] crypto: testmgr - Add additional AES-XTS vectors for covering
> > > > > > >>>  CTS (part II)
> > > > > > >>
> > > > > > >> Patchwork doesn't like it when you do this and it'll discard
> > > > > > >> your patch.  To make it into patchwork you need to put the new
> > > > > > >> Subject in the email headers.
> > > > > > >>
> > > > > > >
> > > > > > > IMO, pretending that your XTS implementation is compliant by only
> > > > > > I've never said that.
> > > > > > Some parts are compliant, some are not.
> > > > > >
> > > > > > > providing test vectors with the last 8 bytes of IV cleared is not the
> > > > > > > right fix for this issue. If you want to be compliant, you will need
> > > > > > It's not a fix.
> > > > > > It's adding test vectors which are not provided in the P1619 standard,
> > > > > > where "data unit sequence number" is at most 5B.
> > > > > >
> > > > >
> > > > > Indeed. But I would prefer not to limit ourselves to 5 bytes of sector
> > > > > numbers in the test vectors. However, we should obviously not add test
> > > > > vectors that are known to cause breakages on hardware that works fine
> > > > > in practice.
> > > > >
> > > > Well, obviously, the full 16 byte sector number vectors fail on existing
> > > > CAAM hardware, which I do assume to work fine in practice. And you know
> > > > I'm not in favor of building all kinds of workarounds into the drivers.
> > > >
> > > > Fact is, we know there are no current users that need more than 64 bits
> > > > of IV. Fact is also that having 64 bits of IV in the vectors is already
> > > > an improvement over the 40 bits in the original vectors. And unlike CTS,
> > > > I am not aware of any real use case for more than 64 bits.
> > > > Finally, another fact is that limiting the *vectors* to 64 bits of IV
> > > > does not prohibit anyone from *using* a full 128 bit IV on an
> > > > implementation that *does* support this. I would think most users of
> > > > XTS, like dmcrypt, would allow you to specify the cra_drivername
> > > > explictly anyway, so just don't select legacy CAAM if you need that.
> > > > (heck, if it would be reading and writing its own data, and not need
> > > > compatibility with other implementations, it wouldn't even matter)
> > > >
> > > > So yes, the specs are quite clear on the sector number being a full
> > > > 128 bits. But that doesn't prevent us from specifying that the
> > > > crypto API implementation currently only supports 64 bits, with the
> > > > remaining bits being forced to 0. We can always revisit that when
> > > > an actual use case for more than 64 bits arises ...
> > > >
> > >
> > > You have got it completely backwards:
> > >
> > > CTS has never worked in any kernel implementation, so regardless of
> > > what the spec says, supporting it in the kernel is not a high priority
> > > issue whichever way you put it.
> > >
> > I never said it was a high priority, I merely pointed out it's not spec
> > compliant. Apparently, you feel that that's only important insofar that
> > it matches current kernel use cases?
> >
> > Anyway, as far as I understand, there are no users that need more than
> > 64 bits of IV in the kernel (i.e. dmcrypt uses only 64 bits), so I see
> > no fundamental difference with CTS except that most(?) implementations
> > possibly already "accidentally" (since unverified!) did it correctly.
> >
> 
> Well, as Milan points out, dm-crypt may use the upper bits as well,
> but the point I was making was about the userland interface. If you
> care about spec compliance for the sake of it, I don't understand why
> you dismiss a broken userland->kernel interface that silently fails
> and corrupts your data as something we should just live with, and just
> assume nobody will ever be silly enough to use it the way the spec
> describes it, and use all the available IV bits.
> 
Pfff ... taking my words waaay out of context ...

I believe I agreed that such a thing should not be silent.
And my argument was never that everyone should support everything, just
that a driver should be able to expose it's CTS capability if available.
If you don't support CTS, just return -EINVAL on non-16 byte multiples,
but don't FORCE a driver that DOES support it to do so for no reason.

> > Not that I have any interest in restricting the IV size: "my" hardware
> > handles full 128 bit IV's just fine. So why do I even bother ... :-)
> >
> > > Now is the first time anyone has asked
> > > for it in 12 years, and only because someone spotted a deviation
> > > between a h/w and a s/w implementation, not because anyone tried to
> > > use it and failed. (Note that passing anything other than a multiple
> > > of the block size will cause an error rather than fail silently)
> > >
> > Yes, failing silently is not such a good idea, I think we agree on that.
> > Although we also need  to keep in mind that that's exactly what the CAAM
> > driver has been doing all those years, and, before my vectors, nobody
> > noticed or cared.
> 
> True. But now that the cat is out of the bag, we have an obligation to
> our users to ensure that a known problem does not corrupt their data.
> 
Again, it should indeed not fail silently. Returning an error should
prevent users from continuing to actually use it, as just building or
mounting the filesystem will already fail. 

> > Without my involvement, this would have probably gone
> > unnoticed for many years to come (so I feel some responsibility ;-).
> >
> > > Truncated IVs are a huge issue, since we already expose the correct
> > > API via AF_ALG (without any restrictions on how many of the IV bits
> > > are populated), and apparently, if your AF_ALG request for xts(aes)
> > > happens to be fulfilled by the CAAM driver and your implementation
> > > uses more than 64 bits for the IV, the top bits get truncated silently
> > > and your data might get eaten.
> > >
> > Apparently, not such a "huge" issue at all, see previous remark.
> >
> > As a precaution, the CAAM driver could return -EINVAL if the upper IV
> > bytes are non-zero.
> 
> Then we'd still have two implementations that behave differently, and
> this is not how abstractions work. If a driver implements xts(aes),
> then it should produce the same output (and return value) for every
> input.
> 
Well, that's where I will continue to disagree with you. 

I'm allergic to dogma, I like bending the rules a little. I believe in
flexibility and providing options, where possible and within reason.
I don't see why crypto drivers can't limit their functionality, so long
as these limitations are clear. I've argued this many times before, but
most HW crypto drivers are NOT selected by default, they are selected
EXPLICITLY (you  can control this by keeping the priority relatively low)
by the user through some config file or commandline switch.
You can choose  NOT to use them for a certain task (say dmcrypt or ipsec)
if they have  known issues for that use case. Vendors can document this. 
That's what we do all the time, document use cases and limitations for
our hardware.

But obviously, they should not fail silently, but provide meaningful
error responses instead, so it's immediately clear they don't work.

> > But then testmgr would have to do less strict error
> > return code checking so we don't force this upon drivers that CAN do it.
> >
> > Implementing a full SW fallback for that in the driver just seems like
> > massive overkill, as you normally specify the driver for dmcrypt explictly
> > anyway (or at least, you can do that if the default fails).
> >
> 
> How would you find out if the default fails? Either it corrupts your
> data, or it fails with an error that is specific to CAAM, and so we
> would have to update the dm-crypt code to deal with en/decryption
> failures that can only occur in this particular case. 
>
It would just return some error and dmcrypt would fail on that, as
it would have to do on any returned error code anyway.
Then you just configure dmcrypt to use some other driver for xts(aes).
It should not need to be more complex than that.

> This is
> *exactly* the thing I argued against before, when we discussed zero
> length inputs: the reason we want the workaround in the driver is
> because it is contained, and its peculiarities don't leak into other
> layers.
> 
And I still very fundamentally disagree with you on that ... ;-)

> > I don't like the idea of HW drivers doing SW fallbacks because it clouds
> > the whole picture of what is actually done by a certain HW device.
> >
> 
> Well, the reality is that we OS guys get to clean up after the h/w
> guys all the time. 
> I agree that explicit failure is better than silent
> corruption, but that still means there are corner cases where things
> just don't work. 
>
Corner cases relevant to specific use cases that you could document.
Many of these h/w drivers are only relevant to embedded systems, which
usually come with large stacks of documentation on how to use them.
It's not a problem unless it is the default selection and there is no
override. You could do stricter checks on implementations with the
highest priority that may end up becoming the default. Just a thought.

> So the only acceptable way to me is to fix this with
> a s/w workaround, and as I already pointed out, this could simply be
> the xts template wrapped around the accelerated ECB implementation
> exposed by the driver.
> 
You could do that. It's just no fun writing and maintaining all that 
code that you fundamentally don't want to be executed in the first
place. It seems like a huge waste of effort.
Also, for every XTS TFM, you always need the software implementation
on standby, consuming memory, with the key configured and the key 
scheduler  likely executed etc., just in case the upper 8 iv bytes are
nonzero. Which probably never happens anyway.

> 
> > > In my experience, users tend to care more about the latter than the former.
> > >
> > >
> > > > > > > to provide a s/w fallback for these cases.
> > > > > > >
> > > > > > Yes, the plan is to:
> > > > > >
> > > > > > -add 16B IV support for caam versions supporting it - caam Era 9+,
> > > > > > currently deployed in lx2160a and ls108a
> > > > > >
> > > > > > -remove current 8B IV support and add s/w fallback for affected caam versions
> > > > > > I'd assume this could be done dynamically, i.e. depending on IV provided
> > > > > > in the crypto request to use either the caam engine or s/w fallback.
> > > > > >
> > > > >
> > > > > Yes. If the IV received from the caller has bytes 8..15 cleared, you
> > > > > use the limited XTS h/w implementation, otherwise you fall back to
> > > > > xts(ecb-aes-caam..).
> > > >
> > > > Regards,
> > > > Pascal van Leeuwen
> > > > Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
> > > > www.insidesecure.com
> > > >
> >
> > Regards,
> > Pascal van Leeuwen
> > Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
> > www.insidesecure.com
> >

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

^ permalink raw reply	[flat|nested] 65+ messages in thread

* Re: [dm-devel] xts fuzz testing and lack of ciphertext stealing support
  2019-08-11 21:29                                                               ` Pascal Van Leeuwen
@ 2019-08-12  4:51                                                                 ` Herbert Xu
  0 siblings, 0 replies; 65+ messages in thread
From: Herbert Xu @ 2019-08-12  4:51 UTC (permalink / raw)
  To: Pascal Van Leeuwen
  Cc: Milan Broz, Ard Biesheuvel, Horia Geanta, dm-devel, linux-crypto

On Sun, Aug 11, 2019 at 09:29:38PM +0000, Pascal Van Leeuwen wrote:
>
> It will very likely fail with that CAAM h/w, but that only proves that you
> should not use plain64be IV's together with CAAM h/w. Which should be

It doesn't matter whether it's wrong or not.

The fact is that this is an interface that we export to user-space
and we must NEVER break that.  So if your driver is breaking this
interface then your driver is broken and needs to be fixed.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 65+ messages in thread

end of thread, other threads:[~2019-08-12  4:51 UTC | newest]

Thread overview: 65+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-16 17:46 xts fuzz testing and lack of ciphertext stealing support Horia Geanta
2019-07-16 22:16 ` Eric Biggers
2019-07-17 17:09   ` Horia Geanta
2019-07-17 17:28     ` Eric Biggers
2019-07-17 18:08       ` Ard Biesheuvel
2019-07-18  6:52         ` Herbert Xu
2019-07-18  7:15           ` Ard Biesheuvel
2019-07-18  7:21             ` Herbert Xu
2019-07-18  7:28               ` Ard Biesheuvel
2019-07-18  7:50                 ` Herbert Xu
2019-07-18  7:40               ` Milan Broz
2019-07-18 10:40                 ` Pascal Van Leeuwen
2019-07-18 11:19                   ` Milan Broz
2019-07-18 15:27                     ` Herbert Xu
2019-07-20  6:58                     ` [dm-devel] " Eric Biggers
2019-07-20  7:35                       ` Milan Broz
2019-07-21  9:50                         ` Ard Biesheuvel
2019-07-22  9:44                           ` Pascal Van Leeuwen
2019-07-22 16:43                             ` Ard Biesheuvel
2019-07-22 22:46                               ` Pascal Van Leeuwen
2019-07-24 12:23                               ` Pascal Van Leeuwen
2019-07-24 12:50                                 ` Pascal Van Leeuwen
2019-07-24 16:10                               ` Pascal Van Leeuwen
2019-07-25  6:22                                 ` Ard Biesheuvel
2019-07-25  7:49                                   ` Pascal Van Leeuwen
2019-07-25  8:01                                     ` Ard Biesheuvel
2019-07-26 10:31                                       ` Pascal Van Leeuwen
2019-07-26 19:59                                         ` Horia Geanta
2019-07-26 21:43                                           ` Pascal Van Leeuwen
2019-07-27  5:39                                             ` Ard Biesheuvel
2019-07-27 12:56                                               ` Pascal Van Leeuwen
2019-07-27 16:04                                               ` Milan Broz
2019-08-04  8:36                                                 ` Ard Biesheuvel
     [not found]                                         ` <20f4832e-e3af-e3c2-d946-13bf8c367a60@nxp.com>
2019-08-07 15:51                                           ` Horia Geanta
2019-08-07 20:57                                             ` Pascal Van Leeuwen
2019-08-08 14:50                                               ` Horia Geanta
2019-08-09  8:35                                                 ` Pascal Van Leeuwen
2019-08-08 13:43                                             ` Pascal Van Leeuwen
2019-08-08 18:01                                               ` Horia Geanta
2019-08-09  2:48                                                 ` Herbert Xu
2019-08-09  6:45                                                   ` Ard Biesheuvel
2019-08-09  7:44                                                     ` Horia Geanta
2019-08-09 17:49                                                       ` Ard Biesheuvel
2019-08-09 20:57                                                         ` Pascal Van Leeuwen
2019-08-10  4:39                                                           ` Ard Biesheuvel
2019-08-11 11:12                                                             ` Milan Broz
2019-08-11 20:34                                                               ` Eric Biggers
2019-08-11 21:39                                                                 ` Pascal Van Leeuwen
2019-08-11 21:29                                                               ` Pascal Van Leeuwen
2019-08-12  4:51                                                                 ` Herbert Xu
2019-08-11 21:15                                                             ` Pascal Van Leeuwen
2019-08-11 22:24                                                               ` Ard Biesheuvel
2019-08-12  1:04                                                                 ` Pascal Van Leeuwen
2019-07-18 15:29                   ` Herbert Xu
2019-07-18 15:43                     ` Pascal Van Leeuwen
2019-07-18 15:51                       ` Herbert Xu
2019-07-18 16:19                         ` Ard Biesheuvel
2019-07-18 16:22                           ` Herbert Xu
2019-07-18 17:03                           ` Pascal Van Leeuwen
2019-07-19  5:34                             ` Ard Biesheuvel
2019-07-19  7:29                               ` Pascal Van Leeuwen
2019-07-19 17:14                                 ` Ard Biesheuvel
2019-07-19 20:07                                   ` Pascal Van Leeuwen
2019-07-18 16:35                         ` Pascal Van Leeuwen
2019-07-19  1:47                           ` Herbert Xu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).