testmgr fuzzing for AEAD ciphers

testmgr fuzzing for AEAD ciphers

[Pascal Van Leeuwen]

Hi,

I was attempting to get some fuzzing going for the RFC3686 AEAD ciphers I'm adding to the 
inside-secure driver, and I noticed some more things besides what I mentioned below:

1) If there is no test suite, but the entry does point to something other then alg_test_null,
then fuzzing is still not performed if there is no test suite, as all of the alg_test_xxx routines
first check for suite->count being > 0 and exit due to count being 0 in this case.
I would think that if there are no reference vectors, then fuzzing against the generic 
implementation (if enabled) is the very least you can do?

2) The AEAD fuzzing routine attempts to determine the maximum key size by actually
scanning the test suite. So if there is no test suite, this will remain at zero and the AEAD
fuzzing routine will still exit without performing any tests because of this.
Isn't there a better way to determine the maximum key size for AEAD ciphers?

3) The AEAD fuzzing vector generation generates fully random keydata that is <= maxlen.
However, for AEAD ciphers, the key blob is actually some RTA struct containing length
fields and types. Which means that most of the time, it will simply be generating illegal
key blobs and you are merely testing whether both implementations correctly flag the
key as illegal. (for which they likely use the same crypto_authenc_extractkeys
subroutine, so that check probably/likely always passes - and therefore is not very useful)

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

> -----Original Message-----
> From: linux-crypto-owner@vger.kernel.org <linux-crypto-owner@vger.kernel.org> On Behalf Of Pascal Van Leeuwen
> Sent: Wednesday, July 3, 2019 11:51 PM
> To: linux-crypto@vger.kernel.org
> Subject: testmgr question
> 
> Hi,
> 
> I'm currently busy adding support for some AES-CTR based authenticated encryption suites to the
> inside-secure driver, and I got the distinct impression they were silently not getting tested at all.
> 
> Looking at testmgr.c, I noticed that they point to alg_test_null, confirming my suspicion.
> I was wondering whether this was intentional and, if so, why that is the case?
> 
> I get that there are no reference vectors for these ciphersuites yet, but that shouldn't stop
> testmgr from at least fuzzing the ciphersuite against the reference implementation?
> 
> Regards,
> Pascal van Leeuwen
> Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
> www.insidesecure.com

RE: testmgr fuzzing for AEAD ciphers

[Pascal Van Leeuwen]

> -----Original Message-----
> From: linux-crypto-owner@vger.kernel.org <linux-crypto-owner@vger.kernel.org> On Behalf Of Pascal Van Leeuwen
> Sent: Thursday, July 4, 2019 10:37 AM
> To: linux-crypto@vger.kernel.org
> Cc: Eric Biggers <ebiggers@kernel.org>; Herbert Xu <herbert@gondor.apana.org.au>
> Subject: testmgr fuzzing for AEAD ciphers
> 
> Hi,
> 
> I was attempting to get some fuzzing going for the RFC3686 AEAD ciphers I'm adding to the
> inside-secure driver, and I noticed some more things besides what I mentioned below:
> 
> 1) If there is no test suite, but the entry does point to something other then alg_test_null,
> then fuzzing is still not performed if there is no test suite, as all of the alg_test_xxx routines
> first check for suite->count being > 0 and exit due to count being 0 in this case.
> I would think that if there are no reference vectors, then fuzzing against the generic
> implementation (if enabled) is the very least you can do?
> 
> 2) The AEAD fuzzing routine attempts to determine the maximum key size by actually
> scanning the test suite. So if there is no test suite, this will remain at zero and the AEAD
> fuzzing routine will still exit without performing any tests because of this.
> Isn't there a better way to determine the maximum key size for AEAD ciphers?
> 
> 3) The AEAD fuzzing vector generation generates fully random keydata that is <= maxlen.
> However, for AEAD ciphers, the key blob is actually some RTA struct containing length
> fields and types. Which means that most of the time, it will simply be generating illegal
> key blobs and you are merely testing whether both implementations correctly flag the
> key as illegal. (for which they likely use the same crypto_authenc_extractkeys
> subroutine, so that check probably/likely always passes - and therefore is not very useful)
> 
I just confirmed the last point by adding some pr_info statements:
Even though I advertise 10 AEAD cipher suites, I have the fuzzing tests enabled
with the default iter count and the fuzzing tests are actually started, NONE of
the generated vectors actually hits the point where an actual encryption is being
performed, they ALL fail on illegal keys.
In other words: nothing relevant is tested at all by the fuzzing tests.

Actually I tried to be "smart" and loop the random key generation until a valid
key was generated. Which caused some nice softlock :-)
Which is when I realized that with 8 bytes of RTA header (i.e. 2^64 combo's)
and only a handful of legal values thereof, the odds of hitting a legal value
are far less than winning the main price in the local lottery ;-)

Ánd generating legal values is actually not so trivial if you don't know the
actual cipher and authentication key sizes supported ...

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com

Re: testmgr fuzzing for AEAD ciphers

[Eric Biggers]

Hi Pascal,

On Thu, Jul 04, 2019 at 08:37:11AM +0000, Pascal Van Leeuwen wrote:
> Hi,
> 
> I was attempting to get some fuzzing going for the RFC3686 AEAD ciphers I'm adding to the 
> inside-secure driver, and I noticed some more things besides what I mentioned below:
> 
> 1) If there is no test suite, but the entry does point to something other then alg_test_null,
> then fuzzing is still not performed if there is no test suite, as all of the alg_test_xxx routines
> first check for suite->count being > 0 and exit due to count being 0 in this case.
> I would think that if there are no reference vectors, then fuzzing against the generic 
> implementation (if enabled) is the very least you can do?
> 
> 2) The AEAD fuzzing routine attempts to determine the maximum key size by actually
> scanning the test suite. So if there is no test suite, this will remain at zero and the AEAD
> fuzzing routine will still exit without performing any tests because of this.
> Isn't there a better way to determine the maximum key size for AEAD ciphers?
> 
> 3) The AEAD fuzzing vector generation generates fully random keydata that is <= maxlen.
> However, for AEAD ciphers, the key blob is actually some RTA struct containing length
> fields and types. Which means that most of the time, it will simply be generating illegal
> key blobs and you are merely testing whether both implementations correctly flag the
> key as illegal. (for which they likely use the same crypto_authenc_extractkeys
> subroutine, so that check probably/likely always passes - and therefore is not very useful)
> 

Yes, these are real issues; we need to make the testing code smarter and perhaps
add some more test vectors too.  But just to clarify (since you keep using the
more general phrase "AEAD ciphers"), these issues actually only apply to RFC3686
ciphers, a.k.a. algorithms with "authenc" in their name, not to other AEADs in
the crypto API such as GCM, ChaCha20-Poly1305, and AEGIS128.

There's no way to easily determine the max key size of an arbitrary AEAD
currently, since it's not stored in struct aead_alg.  That's why the current
code is scanning the test vectors.  Instead, we probably should store
information about the supported key sizes and formats directly in struct
alg_test_desc, independent of the test vectors themselves.  That would make it
possible to solve all three issues you've identified.

- Eric

RE: testmgr fuzzing for AEAD ciphers

[Pascal Van Leeuwen]

> 
> Hi Pascal,
> 
> On Thu, Jul 04, 2019 at 08:37:11AM +0000, Pascal Van Leeuwen wrote:
> > Hi,
> >
> > I was attempting to get some fuzzing going for the RFC3686 AEAD ciphers I'm adding to the
> > inside-secure driver, and I noticed some more things besides what I mentioned below:
> >
> > 1) If there is no test suite, but the entry does point to something other then alg_test_null,
> > then fuzzing is still not performed if there is no test suite, as all of the alg_test_xxx routines
> > first check for suite->count being > 0 and exit due to count being 0 in this case.
> > I would think that if there are no reference vectors, then fuzzing against the generic
> > implementation (if enabled) is the very least you can do?
> >
> > 2) The AEAD fuzzing routine attempts to determine the maximum key size by actually
> > scanning the test suite. So if there is no test suite, this will remain at zero and the AEAD
> > fuzzing routine will still exit without performing any tests because of this.
> > Isn't there a better way to determine the maximum key size for AEAD ciphers?
> >
> > 3) The AEAD fuzzing vector generation generates fully random keydata that is <= maxlen.
> > However, for AEAD ciphers, the key blob is actually some RTA struct containing length
> > fields and types. Which means that most of the time, it will simply be generating illegal
> > key blobs and you are merely testing whether both implementations correctly flag the
> > key as illegal. (for which they likely use the same crypto_authenc_extractkeys
> > subroutine, so that check probably/likely always passes - and therefore is not very useful)
> >
> 
> Yes, these are real issues; we need to make the testing code smarter and perhaps
> add some more test vectors too.  But just to clarify (since you keep using the
> more general phrase "AEAD ciphers"), these issues actually only apply to RFC3686
> ciphers, a.k.a. algorithms with "authenc" in their name, not to other AEADs in
> the crypto API such as GCM, ChaCha20-Poly1305, and AEGIS128.
> 
Ok, since I was just working on authenc ciphersuites I assumed that the "other"
(real) AEAD's would work the same way, good to know that's not the case.
AES-GCM and AES-CCM are next on my list (after my holiday though) ...

> There's no way to easily determine the max key size of an arbitrary AEAD
> currently, since it's not stored in struct aead_alg.  That's why the current
> code is scanning the test vectors.  Instead, we probably should store
> information about the supported key sizes and formats directly in struct
> alg_test_desc, independent of the test vectors themselves.  That would make it
> possible to solve all three issues you've identified.
> 
> - Eric
>
Yes, that should work I guess. That would also allow it to be an actual *list* of 
correct values instead of assuming a range between 0 and max, where only a
few values are truly valid, which is wasting a lot of testvectors. This would
apply to skciphers as well.

Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Verimatrix
www.insidesecure.com