[U-Boot] [PATCH v2 1/3] rsa: Fix build with OpenSSL 1.1.x

[Mario Six <mario.six@gdsys.cc>]

Hi Jelle,

On Tue, Apr 4, 2017 at 11:59 PM, Jelle van der Waa <jelle@vdwaa.nl> wrote:
> @@ -20,6 +20,19 @@
>  #define HAVE_ERR_REMOVE_THREAD_STATE
>  #endif
>
> +#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER)
> +void RSA_get0_key(const RSA *r,
> +                 const BIGNUM **n, const BIGNUM **e, const BIGNUM **d)
> +{
> +   if (n != NULL)
> +       *n = r->n;
> +   if (e != NULL)
> +       *e = r->e;
> +   if (d != NULL)
> +       *d = r->d;
> +}
> +#endif
> +

Like in the other patch, this function should be static (also, missing #include
<openssl/bn.h> in this file as well).

> @@ -548,7 +568,8 @@ int rsa_get_params(RSA *key, uint64_t *exponent, uint32_t *n0_invp,
>         if (0 != rsa_get_exponent(key, exponent))
>                 ret = -1;
>
> -       if (!BN_copy(n, key->n) || !BN_set_word(big1, 1L) ||
> +       RSA_get0_key(key, NULL, &key_n, NULL);
> +       if (!BN_copy(n, key_n) || !BN_set_word(big1, 1L) ||
>             !BN_set_word(big2, 2L) || !BN_set_word(big32, 32L))
>                 ret = -1;
>

Your're loading the parameter e into key_n here! It should be

RSA_get0_key(key, &key_n, NULL, NULL);

instead.

Like I said in the previous patch, you will have to #ifdef out more functions
in this file:

* SSL_load_error_strings
* OpenSSL_add_all_algorithms
* OpenSSL_add_all_digests
* OpenSSL_add_all_ciphers
* ENGINE_cleanup
* CRYPTO_cleanup_all_ex_data
* ERR_free_strings();
* EVP_cleanup

And you'll also have to replace SSL_library_init() with
OPENSSL_init_ssl(0, NULL).

After making all these changes, I was able to build a working U-Boot (for our
controlcenterdc board) against OpenSSL 1.1 that loaded a signed FIT-Image that
a previous U-Boot also loaded.

Best regards,

Mario