[Buildroot] [v3 0/2] openssl: parallel build

[Buildroot] [v3 0/2] openssl: parallel build

[Ryan Barnett]

I have adopted Thomas P's patches that we submitted but in May in order
enable parallel build of openssl. I have updated the patches to use the
latest patches available from gentoo and have tested builds only on i386,
ARM, and MIPS.

Original Patches:

http://patchwork.ozlabs.org/patch/473086/
http://patchwork.ozlabs.org/patch/473087/

Ryan Barnett (2):
  openssl: use git formatted patches
  openssl: enable parallel build and installation

 ...time-building-manpages-if-we-re-not-going.patch | 27 ++++++++
 ...odev-Fix-issue-with-signature-generation.patch} | 71 +++++++++++++---------
 package/openssl/001-do-not-build-docs.patch        | 13 ----
 package/openssl/openssl.hash                       |  2 +
 package/openssl/openssl.mk                         | 12 ++--
 5 files changed, 78 insertions(+), 47 deletions(-)
 create mode 100644 package/openssl/0001-Dont-waste-time-building-manpages-if-we-re-not-going.patch
 rename package/openssl/{003-cryptodev-Fix-issue-with-signature-generation.patch => 0002-cryptodev-Fix-issue-with-signature-generation.patch} (82%)
 delete mode 100644 package/openssl/001-do-not-build-docs.patch

-- 
1.9.1

[Buildroot] [v3 1/2] openssl: use git formatted patches

[Ryan Barnett]

Using Git formatted patches makes it easier to adjust the patches when
needed.

Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com>
CC: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
CC: Gustavo Zacarias <gustavo@zacarias.com.ar>
CC: Arnout Vandecappelle <arnout@mind.be>
---
Changes v2 to v3:
 - Added Gustavo's SOB back to his patch after accidently droping it
   (suggested by Baurch)

Changes v1 to v2:
 - Move changes from patch 2/2 to here (suggested by Arnout)
 - Note: git automatically detects a delete and new file for
   patch 001 to 0001. So there is no rename since the changes
   to the file are greater than 50%.

Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com>
---
 ...time-building-manpages-if-we-re-not-going.patch | 26 ++++++++
 ...odev-Fix-issue-with-signature-generation.patch} | 69 +++++++++++++---------
 package/openssl/001-do-not-build-docs.patch        | 13 ----
 3 files changed, 67 insertions(+), 41 deletions(-)
 create mode 100644 package/openssl/0001-Dont-waste-time-building-manpages-if-we-re-not-going.patch
 rename package/openssl/{003-cryptodev-Fix-issue-with-signature-generation.patch => 0002-cryptodev-Fix-issue-with-signature-generation.patch} (82%)
 delete mode 100644 package/openssl/001-do-not-build-docs.patch

diff --git a/package/openssl/0001-Dont-waste-time-building-manpages-if-we-re-not-going.patch b/package/openssl/0001-Dont-waste-time-building-manpages-if-we-re-not-going.patch
new file mode 100644
index 0000000..8a0081e
--- /dev/null
+++ b/package/openssl/0001-Dont-waste-time-building-manpages-if-we-re-not-going.patch
@@ -0,0 +1,26 @@
+From 389efb564fa1453a9da835393eec9006bfae2a52 Mon Sep 17 00:00:00 2001
+From: Mike Frysinger <vapier@gentoo.org>
+Date: Sat, 16 May 2015 18:53:51 +0200
+Subject: [1/2] Dont waste time building manpages if we're not going to use em.
+
+Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com>
+---
+ Makefile.org | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/Makefile.org b/Makefile.org
+index 60f07cc..976ceaf 100644
+--- a/Makefile.org
++++ b/Makefile.org
+@@ -527,7 +527,7 @@ dist:
+ dist_pem_h:
+ 	(cd crypto/pem; $(MAKE) -e $(BUILDENV) pem.h; $(MAKE) clean)
+ 
+-install: all install_docs install_sw
++install: all install_sw
+ 
+ install_sw:
+ 	@$(PERL) $(TOP)/util/mkdir-p.pl $(INSTALL_PREFIX)$(INSTALLTOP)/bin \
+-- 
+1.9.1
+
diff --git a/package/openssl/003-cryptodev-Fix-issue-with-signature-generation.patch b/package/openssl/0002-cryptodev-Fix-issue-with-signature-generation.patch
similarity index 82%
rename from package/openssl/003-cryptodev-Fix-issue-with-signature-generation.patch
rename to package/openssl/0002-cryptodev-Fix-issue-with-signature-generation.patch
index 19ed5e7..829443d 100644
--- a/package/openssl/003-cryptodev-Fix-issue-with-signature-generation.patch
+++ b/package/openssl/0002-cryptodev-Fix-issue-with-signature-generation.patch
@@ -1,3 +1,8 @@
+From 90fd7e8f1a316cda86ee442b43fcd7d5e5baeede Mon Sep 17 00:00:00 2001
+From: Gustavo Zacarias <gustavo@zacarias.com.ar>
+Date: Sat, 16 May 2015 18:55:08 +0200
+Subject: [2/2] cryptodev: Fix issue with signature generation
+
 Forward port of 0001-cryptodev-Fix-issue-with-signature-generation.patch
 from http://rt.openssl.org/Ticket/Display.html?id=2770&user=guest&pass=guest
 It was originally targetted at 1.0.2-beta3.
@@ -5,10 +10,15 @@ It was originally targetted at 1.0.2-beta3.
 Without this patch digest acceleration via cryptodev is broken.
 
 Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
+Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com>
+---
+ crypto/engine/eng_cryptodev.c | 195 +++++++++++++++++++++++++++++++-----------
+ 1 file changed, 146 insertions(+), 49 deletions(-)
 
-diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto/engine/eng_cryptodev.c
---- openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c	2015-01-28 14:59:58.146682462 -0300
-+++ openssl-1.0.2/crypto/engine/eng_cryptodev.c	2015-01-28 15:29:25.107649077 -0300
+diff --git a/crypto/engine/eng_cryptodev.c b/crypto/engine/eng_cryptodev.c
+index 926d95c..7021d9a 100644
+--- a/crypto/engine/eng_cryptodev.c
++++ b/crypto/engine/eng_cryptodev.c
 @@ -2,6 +2,7 @@
   * Copyright (c) 2002 Bob Beck <beck@openbsd.org>
   * Copyright (c) 2002 Theo de Raadt
@@ -17,7 +27,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
   * All rights reserved.
   *
   * Redistribution and use in source and binary forms, with or without
-@@ -72,7 +73,6 @@
+@@ -72,7 +73,6 @@ struct dev_crypto_state {
      struct session_op d_sess;
      int d_fd;
  # ifdef USE_CRYPTODEV_DIGESTS
@@ -25,7 +35,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      unsigned char digest_res[HASH_MAX_LEN];
      char *mac_data;
      int mac_len;
-@@ -189,8 +189,10 @@
+@@ -189,8 +189,10 @@ static struct {
  static struct {
      int id;
      int nid;
@@ -37,7 +47,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      {
          CRYPTO_MD5_HMAC, NID_hmacWithMD5, 16
      },
-@@ -198,15 +200,15 @@
+@@ -198,15 +200,15 @@ static struct {
          CRYPTO_SHA1_HMAC, NID_hmacWithSHA1, 20
      },
      {
@@ -57,7 +67,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      {
          CRYPTO_MD5, NID_md5, 16
      },
-@@ -214,6 +216,15 @@
+@@ -214,6 +216,15 @@ static struct {
          CRYPTO_SHA1, NID_sha1, 20
      },
      {
@@ -73,7 +83,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
          0, NID_undef, 0
      },
  };
-@@ -288,13 +299,14 @@
+@@ -288,13 +299,14 @@ static int get_cryptodev_ciphers(const int **cnids)
      static int nids[CRYPTO_ALGORITHM_MAX];
      struct session_op sess;
      int fd, i, count = 0;
@@ -89,7 +99,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
  
      for (i = 0; ciphers[i].id && count < CRYPTO_ALGORITHM_MAX; i++) {
          if (ciphers[i].nid == NID_undef)
-@@ -327,18 +339,19 @@
+@@ -327,18 +339,19 @@ static int get_cryptodev_digests(const int **cnids)
      static int nids[CRYPTO_ALGORITHM_MAX];
      struct session_op sess;
      int fd, i, count = 0;
@@ -111,7 +121,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
          sess.cipher = 0;
          if (ioctl(fd, CIOCGSESSION, &sess) != -1 &&
              ioctl(fd, CIOCFSESSION, &sess.ses) != -1)
-@@ -424,14 +437,14 @@
+@@ -424,14 +437,14 @@ cryptodev_cipher(EVP_CIPHER_CTX *ctx, unsigned char *out,
      cryp.ses = sess->ses;
      cryp.flags = 0;
      cryp.len = inl;
@@ -129,7 +139,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
          if (!ctx->encrypt) {
              iiv = in + inl - ctx->cipher->iv_len;
              memcpy(save_iv, iiv, ctx->cipher->iv_len);
-@@ -483,7 +496,7 @@
+@@ -483,7 +496,7 @@ cryptodev_init_key(EVP_CIPHER_CTX *ctx, const unsigned char *key,
      if ((state->d_fd = get_dev_crypto()) < 0)
          return (0);
  
@@ -138,7 +148,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      sess->keylen = ctx->key_len;
      sess->cipher = cipher;
  
-@@ -749,16 +762,6 @@
+@@ -749,16 +762,6 @@ static int digest_nid_to_cryptodev(int nid)
      return (0);
  }
  
@@ -155,7 +165,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
  static int cryptodev_digest_init(EVP_MD_CTX *ctx)
  {
      struct dev_crypto_state *state = ctx->md_data;
-@@ -769,7 +772,6 @@
+@@ -769,7 +772,6 @@ static int cryptodev_digest_init(EVP_MD_CTX *ctx)
          printf("cryptodev_digest_init: Can't get digest \n");
          return (0);
      }
@@ -163,7 +173,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      memset(state, 0, sizeof(struct dev_crypto_state));
  
      if ((state->d_fd = get_dev_crypto()) < 0) {
-@@ -777,8 +779,8 @@
+@@ -777,8 +779,8 @@ static int cryptodev_digest_init(EVP_MD_CTX *ctx)
          return (0);
      }
  
@@ -174,7 +184,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      sess->mac = digest;
  
      if (ioctl(state->d_fd, CIOCGSESSION, sess) < 0) {
-@@ -794,8 +796,8 @@
+@@ -794,8 +796,8 @@ static int cryptodev_digest_init(EVP_MD_CTX *ctx)
  static int cryptodev_digest_update(EVP_MD_CTX *ctx, const void *data,
                                     size_t count)
  {
@@ -184,7 +194,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      struct session_op *sess = &state->d_sess;
  
      if (!data || state->d_fd < 0) {
-@@ -804,7 +806,7 @@
+@@ -804,7 +806,7 @@ static int cryptodev_digest_update(EVP_MD_CTX *ctx, const void *data,
      }
  
      if (!count) {
@@ -193,7 +203,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      }
  
      if (!(ctx->flags & EVP_MD_CTX_FLAG_ONESHOT)) {
-@@ -828,9 +830,9 @@
+@@ -828,9 +830,9 @@ static int cryptodev_digest_update(EVP_MD_CTX *ctx, const void *data,
      cryp.ses = sess->ses;
      cryp.flags = 0;
      cryp.len = count;
@@ -205,7 +215,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      if (ioctl(state->d_fd, CIOCCRYPT, &cryp) < 0) {
          printf("cryptodev_digest_update: digest failed\n");
          return (0);
-@@ -844,8 +846,6 @@
+@@ -844,8 +846,6 @@ static int cryptodev_digest_final(EVP_MD_CTX *ctx, unsigned char *md)
      struct dev_crypto_state *state = ctx->md_data;
      struct session_op *sess = &state->d_sess;
  
@@ -214,7 +224,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      if (!md || state->d_fd < 0) {
          printf("cryptodev_digest_final: illegal input\n");
          return (0);
-@@ -859,7 +859,7 @@
+@@ -859,7 +859,7 @@ static int cryptodev_digest_final(EVP_MD_CTX *ctx, unsigned char *md)
          cryp.len = state->mac_len;
          cryp.src = state->mac_data;
          cryp.dst = NULL;
@@ -223,7 +233,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
          if (ioctl(state->d_fd, CIOCCRYPT, &cryp) < 0) {
              printf("cryptodev_digest_final: digest failed\n");
              return (0);
-@@ -870,7 +870,7 @@
+@@ -870,7 +870,7 @@ static int cryptodev_digest_final(EVP_MD_CTX *ctx, unsigned char *md)
  
      memcpy(md, state->digest_res, ctx->digest->md_size);
  
@@ -232,7 +242,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
  }
  
  static int cryptodev_digest_cleanup(EVP_MD_CTX *ctx)
-@@ -921,8 +921,8 @@
+@@ -921,8 +921,8 @@ static int cryptodev_digest_copy(EVP_MD_CTX *to, const EVP_MD_CTX *from)
  
      digest = digest_nid_to_cryptodev(to->digest->type);
  
@@ -243,7 +253,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      sess->mac = digest;
  
      dstate->d_fd = get_dev_crypto();
-@@ -947,32 +947,116 @@
+@@ -947,32 +947,116 @@ static int cryptodev_digest_copy(EVP_MD_CTX *to, const EVP_MD_CTX *from)
  
  const EVP_MD cryptodev_sha1 = {
      NID_sha1,
@@ -367,7 +377,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
  };
  
  # endif                         /* USE_CRYPTODEV_DIGESTS */
-@@ -992,6 +1076,18 @@
+@@ -992,6 +1076,18 @@ cryptodev_engine_digests(ENGINE *e, const EVP_MD **digest,
      case NID_sha1:
          *digest = &cryptodev_sha1;
          break;
@@ -386,7 +396,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      default:
  # endif                         /* USE_CRYPTODEV_DIGESTS */
          *digest = NULL;
-@@ -1022,7 +1118,7 @@
+@@ -1022,7 +1118,7 @@ static int bn2crparam(const BIGNUM *a, struct crparam *crp)
          return (1);
      memset(b, 0, bytes);
  
@@ -395,7 +405,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      crp->crp_nbits = bits;
  
      for (i = 0, j = 0; i < a->top; i++) {
-@@ -1277,7 +1373,7 @@
+@@ -1277,7 +1373,7 @@ static DSA_SIG *cryptodev_dsa_do_sign(const unsigned char *dgst, int dlen,
      kop.crk_op = CRK_DSA_SIGN;
  
      /* inputs: dgst dsa->p dsa->q dsa->g dsa->priv_key */
@@ -404,7 +414,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      kop.crk_param[0].crp_nbits = dlen * 8;
      if (bn2crparam(dsa->p, &kop.crk_param[1]))
          goto err;
-@@ -1317,7 +1413,7 @@
+@@ -1317,7 +1413,7 @@ cryptodev_dsa_verify(const unsigned char *dgst, int dlen,
      kop.crk_op = CRK_DSA_VERIFY;
  
      /* inputs: dgst dsa->p dsa->q dsa->g dsa->pub_key sig->r sig->s */
@@ -413,7 +423,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
      kop.crk_param[0].crp_nbits = dlen * 8;
      if (bn2crparam(dsa->p, &kop.crk_param[1]))
          goto err;
-@@ -1398,9 +1494,10 @@
+@@ -1398,9 +1494,10 @@ cryptodev_dh_compute_key(unsigned char *key, const BIGNUM *pub_key, DH *dh)
          goto err;
      kop.crk_iparams = 3;
  
@@ -426,7 +436,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
  
      if (ioctl(fd, CIOCKEY, &kop) == -1) {
          const DH_METHOD *meth = DH_OpenSSL();
-@@ -1470,7 +1567,7 @@
+@@ -1470,7 +1567,7 @@ void ENGINE_load_cryptodev(void)
      put_dev_crypto(fd);
  
      if (!ENGINE_set_id(engine, "cryptodev") ||
@@ -435,3 +445,6 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto
          !ENGINE_set_ciphers(engine, cryptodev_engine_ciphers) ||
          !ENGINE_set_digests(engine, cryptodev_engine_digests) ||
          !ENGINE_set_ctrl_function(engine, cryptodev_ctrl) ||
+-- 
+1.9.1
+
diff --git a/package/openssl/001-do-not-build-docs.patch b/package/openssl/001-do-not-build-docs.patch
deleted file mode 100644
index 135a9d3..0000000
--- a/package/openssl/001-do-not-build-docs.patch
+++ /dev/null
@@ -1,13 +0,0 @@
-Dont waste time building manpages if we're not going to use em.
-
---- openssl/Makefile.org
-+++ openssl/Makefile.org
-@@ -792,7 +792,7 @@
- dist_pem_h:
- 	(cd crypto/pem; $(MAKE) CC='${CC}' SDIRS='${SDIRS}' CFLAG='${CFLAG}' pem.h; $(MAKE) clean)
- 
--install: all install_docs install_sw
-+install: all install_sw
- 
- install_sw:
- 	@$(PERL) $(TOP)/util/mkdir-p.pl $(INSTALL_PREFIX)$(INSTALLTOP)/bin \
-- 
1.9.1

[Buildroot] [v3 2/2] openssl: enable parallel build and installation

[Ryan Barnett]

This is a patch that is originally based on a patch Thomas P.
submitted for an earlier version of this package. I have adopted this
patch to use the latest available Gentoo parallel patch. I have also
seen about a minute improvement on my build times of openssl.

Part of Thomas P's original message:

On my build server, the current build of OpenSSL takes 1 minutes and
20 seconds. With this commit applied, enabling parallel build and
installation, the build only takes 28 seconds.

All the patches are downloaded from Gentoo.

There is apparently some interest in upstream OpenSSL to enable
parallel build, see for example commit
https://github.com/openssl/openssl/commit/c3f22253b139793ff3b91ff7e6969e180cf06815. This
commit is not part of any OpenSSL release, but we can hope that the
problem will resolved in the future.

Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com>
CC: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
CC: Gustavo Zacarias <gustavo@zacarias.com.ar>
CC: Arnout Vandecappelle <arnout@mind.be>

---
Changes from v2 to v3:
 - No changes

Changes from v1 to v2:
 - Removed changes to existing patches in openssl (suggested by Arnout)
 - Updated parallel patch URL to use Gentoo gitweb (suggested by Mike)

Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com>
---
 package/openssl/openssl.hash |  2 ++
 package/openssl/openssl.mk   | 12 +++++++-----
 2 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/package/openssl/openssl.hash b/package/openssl/openssl.hash
index 355be79..9eaa363 100644
--- a/package/openssl/openssl.hash
+++ b/package/openssl/openssl.hash
@@ -1,2 +1,4 @@
 # From https://www.openssl.org/source/openssl-1.0.2d.tar.gz.sha256
 sha256	671c36487785628a703374c652ad2cebea45fa920ae5681515df25d9f2c9a8c8	openssl-1.0.2d.tar.gz
+# Locally computed
+sha256	deaf6f3af41874ecc6d63841ea14b8e6c71cea81d4a511a754bc90c9a993147f	openssl-1.0.2d-parallel-build.patch?id=56bd759df1d0c750a065b8c845e93d5dfa6b549d
diff --git a/package/openssl/openssl.mk b/package/openssl/openssl.mk
index 3d87919..c1fc420 100644
--- a/package/openssl/openssl.mk
+++ b/package/openssl/openssl.mk
@@ -13,6 +13,8 @@ OPENSSL_DEPENDENCIES = zlib
 HOST_OPENSSL_DEPENDENCIES = host-zlib
 OPENSSL_TARGET_ARCH = generic32
 OPENSSL_CFLAGS = $(TARGET_CFLAGS)
+OPENSSL_PATCH = \
+	https://gitweb.gentoo.org/repo/gentoo.git/plain/dev-libs/openssl/files/openssl-1.0.2d-parallel-build.patch?id=56bd759df1d0c750a065b8c845e93d5dfa6b549d
 
 ifeq ($(BR2_USE_MMU),)
 OPENSSL_CFLAGS += -DHAVE_FORK=0
@@ -99,23 +101,23 @@ OPENSSL_POST_CONFIGURE_HOOKS += OPENSSL_FIXUP_STATIC_MAKEFILE
 endif
 
 define HOST_OPENSSL_BUILD_CMDS
-	$(MAKE1) -C $(@D)
+	$(MAKE) -C $(@D)
 endef
 
 define OPENSSL_BUILD_CMDS
-	$(MAKE1) -C $(@D)
+	$(MAKE) -C $(@D)
 endef
 
 define OPENSSL_INSTALL_STAGING_CMDS
-	$(MAKE1) -C $(@D) INSTALL_PREFIX=$(STAGING_DIR) install
+	$(MAKE) -C $(@D) INSTALL_PREFIX=$(STAGING_DIR) install
 endef
 
 define HOST_OPENSSL_INSTALL_CMDS
-	$(MAKE1) -C $(@D) install
+	$(MAKE) -C $(@D) install
 endef
 
 define OPENSSL_INSTALL_TARGET_CMDS
-	$(MAKE1) -C $(@D) INSTALL_PREFIX=$(TARGET_DIR) install
+	$(MAKE) -C $(@D) INSTALL_PREFIX=$(TARGET_DIR) install
 	rm -rf $(TARGET_DIR)/usr/lib/ssl
 	rm -f $(TARGET_DIR)/usr/bin/c_rehash
 endef
-- 
1.9.1

[Buildroot] [v3 1/2] openssl: use git formatted patches

[Thomas Petazzoni]

Ryan,

On Wed, 16 Sep 2015 07:06:24 -0500, Ryan Barnett wrote:

> +Subject: [1/2] Dont waste time building manpages if we're not going to use em.

Arnout also suggested to get rid of [1/2] (and [2/2] for the second
patch), since this information becomes obsolete when we add a new patch.

I've applied after doing this change. Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [v3 2/2] openssl: enable parallel build and installation

[Thomas Petazzoni]

Dear Ryan Barnett,

On Wed, 16 Sep 2015 07:06:25 -0500, Ryan Barnett wrote:
> This is a patch that is originally based on a patch Thomas P.
> submitted for an earlier version of this package. I have adopted this
> patch to use the latest available Gentoo parallel patch. I have also
> seen about a minute improvement on my build times of openssl.
> 
> Part of Thomas P's original message:
> 
> On my build server, the current build of OpenSSL takes 1 minutes and
> 20 seconds. With this commit applied, enabling parallel build and
> installation, the build only takes 28 seconds.
> 
> All the patches are downloaded from Gentoo.
> 
> There is apparently some interest in upstream OpenSSL to enable
> parallel build, see for example commit
> https://github.com/openssl/openssl/commit/c3f22253b139793ff3b91ff7e6969e180cf06815. This
> commit is not part of any OpenSSL release, but we can hope that the
> problem will resolved in the future.
> 
> Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com>
> CC: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> CC: Gustavo Zacarias <gustavo@zacarias.com.ar>
> CC: Arnout Vandecappelle <arnout@mind.be>
> 
> ---
> Changes from v2 to v3:
>  - No changes

Applied, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [v3 0/2] openssl: parallel build

[Thomas Petazzoni]

Ryan,

On Wed, 16 Sep 2015 07:06:23 -0500, Ryan Barnett wrote:
> I have adopted Thomas P's patches that we submitted but in May in order
> enable parallel build of openssl. I have updated the patches to use the
> latest patches available from gentoo and have tested builds only on i386,
> ARM, and MIPS.
> 
> Original Patches:
> 
> http://patchwork.ozlabs.org/patch/473086/
> http://patchwork.ozlabs.org/patch/473087/
> 
> Ryan Barnett (2):
>   openssl: use git formatted patches
>   openssl: enable parallel build and installation

I have the feeling that the parallel build patches for OpenSSL are not
working properly. Look at the recent build failures we've got for the
openssl and host-openssl builds:

  http://autobuild.buildroot.org/?reason=openssl-1.0.2d
  http://autobuild.buildroot.org/?reason=host-openssl-1.0.2d

They are pretty much all due to a similar error:

  ln: failed to create symbolic link 'libssl.so': File exists
  ln: creating symbolic link `libcrypto.so': File exists

I could spot only one that is different:

In file included from apps.h:117:0,
                 from req.c:74:
../include/openssl/bio.h:521:34: error: expected ';' before 'BIO_ctrl'
 # define BIO_set_fp(b,fp,c)      BIO_ctrl(b,BIO_C_SET_FILE_PTR,c,(char *)fp)

Do you have some ideas? Does Gentoo has some more patches? Should we
revert back to non-parallel build for OpenSSL ?

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [v3 0/2] openssl: parallel build

[Ryan Barnett]

Hi Thomas,

On Wed, Nov 18, 2015 at 6:57 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
>
> I have the feeling that the parallel build patches for OpenSSL are not
> working properly. Look at the recent build failures we've got for the
> openssl and host-openssl builds:
>
>   http://autobuild.buildroot.org/?reason=openssl-1.0.2d
>   http://autobuild.buildroot.org/?reason=host-openssl-1.0.2d
>
> They are pretty much all due to a similar error:
>
>   ln: failed to create symbolic link 'libssl.so': File exists
>   ln: creating symbolic link `libcrypto.so': File exists
>
> I could spot only one that is different:
>
> In file included from apps.h:117:0,
>                  from req.c:74:
> ../include/openssl/bio.h:521:34: error: expected ';' before 'BIO_ctrl'
>  # define BIO_set_fp(b,fp,c)      BIO_ctrl(b,BIO_C_SET_FILE_PTR,c,(char *)fp)
>
> Do you have some ideas?

I agree with you in that it looks like are some issues with the
parallel build. It appears there is a hard to track down race
condition due a symbolic being created twice during the build but as
to way I don't have any clue.

>Does Gentoo has some more patches?

No this is the only patch for openssl 1.0.2d. There also hasn't been
any updates to this patch other then the one that is currently there.

> Should we revert back to non-parallel build for OpenSSL ?

I guess I will leave that up to either yourself or Peter if you feel
that it should. I don't have time in the next week or two to
investigate the problem further so I'm OK with reverting this patch.

Thanks,
-Ryan

-- 
Ryan Barnett / Sr Software Engineer
Airborne Information Systems / Secure Platforms
MS 131-100, C Ave NE, Cedar Rapids, IA, 52498, USA
ryan.barnett at rockwellcollins.com
www.rockwellcollins.com

[Buildroot] [v3 0/2] openssl: parallel build

[Arnout Vandecappelle]

On 18-11-15 20:41, Ryan Barnett wrote:
> Hi Thomas,
> 
> On Wed, Nov 18, 2015 at 6:57 AM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
>>
>> I have the feeling that the parallel build patches for OpenSSL are not
>> working properly. Look at the recent build failures we've got for the
>> openssl and host-openssl builds:
>>
>>   http://autobuild.buildroot.org/?reason=openssl-1.0.2d
>>   http://autobuild.buildroot.org/?reason=host-openssl-1.0.2d
>>
>> They are pretty much all due to a similar error:
>>
>>   ln: failed to create symbolic link 'libssl.so': File exists
>>   ln: creating symbolic link `libcrypto.so': File exists
>>
>> I could spot only one that is different:
>>
>> In file included from apps.h:117:0,
>>                  from req.c:74:
>> ../include/openssl/bio.h:521:34: error: expected ';' before 'BIO_ctrl'
>>  # define BIO_set_fp(b,fp,c)      BIO_ctrl(b,BIO_C_SET_FILE_PTR,c,(char *)fp)
>>
>> Do you have some ideas?
> 
> I agree with you in that it looks like are some issues with the
> parallel build. It appears there is a hard to track down race
> condition due a symbolic being created twice during the build but as
> to way I don't have any clue.

 I can see where the problem is, but not what the solution is... The
build-shared target depends on do_crypto and link-shared, which will be executed
in parallel. do_crypto calls link_a.linux_shared -> link_a.gnu which does
SYMLINK_SO; in parallel, link-shared calls symlink.linux_shared which also does
SYMLINK_SO. Before the symlink is created, it is rm'ed, but there is a tiny
chance that the second one is created after the rm has been called. Bam.

 Maybe just replace the rm with an ln -sf? Don't know if upstream will like that
though.

 Regards,
 Arnout


> 
>> Does Gentoo has some more patches?
> 
> No this is the only patch for openssl 1.0.2d. There also hasn't been
> any updates to this patch other then the one that is currently there.
> 
>> Should we revert back to non-parallel build for OpenSSL ?
> 
> I guess I will leave that up to either yourself or Peter if you feel
> that it should. I don't have time in the next week or two to
> investigate the problem further so I'm OK with reverting this patch.
> 
> Thanks,
> -Ryan
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [v3 0/2] openssl: parallel build

[Yann E. MORIN]

Thomas, All,

On 2015-11-18 13:57 +0100, Thomas Petazzoni spake thusly:
> On Wed, 16 Sep 2015 07:06:23 -0500, Ryan Barnett wrote:
> > I have adopted Thomas P's patches that we submitted but in May in order
> > enable parallel build of openssl. I have updated the patches to use the
> > latest patches available from gentoo and have tested builds only on i386,
> > ARM, and MIPS.
> > 
> > Original Patches:
> > 
> > http://patchwork.ozlabs.org/patch/473086/
> > http://patchwork.ozlabs.org/patch/473087/
> > 
> > Ryan Barnett (2):
> >   openssl: use git formatted patches
> >   openssl: enable parallel build and installation
> 
> I have the feeling that the parallel build patches for OpenSSL are not
> working properly. Look at the recent build failures we've got for the
> openssl and host-openssl builds:
> 
>   http://autobuild.buildroot.org/?reason=openssl-1.0.2d
>   http://autobuild.buildroot.org/?reason=host-openssl-1.0.2d
[--SNIP--]
> Do you have some ideas? Does Gentoo has some more patches? Should we
> revert back to non-parallel build for OpenSSL ?

I believe we should just revert the patch.

It's unfortunate that we can't buid OpenSSL in parallel, but we can't
seriously go and patch their broken buildsystem, not at least until they
are willing to review and apply those patches upstream first.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [v3 0/2] openssl: parallel build

[Thomas Petazzoni]

Hello,

On Mon, 23 Nov 2015 14:05:28 +0100, Yann E. MORIN wrote:

> > Do you have some ideas? Does Gentoo has some more patches? Should we
> > revert back to non-parallel build for OpenSSL ?
> 
> I believe we should just revert the patch.
> 
> It's unfortunate that we can't buid OpenSSL in parallel, but we can't
> seriously go and patch their broken buildsystem, not at least until they
> are willing to review and apply those patches upstream first.

Agreed, we should revert the patch.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [v3 0/2] openssl: parallel build

[Mike Frysinger]

On 18 Nov 2015 13:41, Ryan Barnett wrote:
> On Wed, Nov 18, 2015 at 6:57 AM, Thomas Petazzoni wrote:
> >Does Gentoo has some more patches?
> 
> No this is the only patch for openssl 1.0.2d. There also hasn't been
> any updates to this patch other then the one that is currently there.

that's incorrect.  you have to look at the ebuild and then see that we apply
four parallel patches for 1.0.2d, one of which specifically for this issue.

https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-libs/openssl/openssl-1.0.2d-r2.ebuild?id=c8abcbe8de5d3b6cdd68c162f398c011ff6e2d9d
		epatch "${FILESDIR}"/${PN}-1.0.2d-parallel-build.patch
		epatch "${FILESDIR}"/${PN}-1.0.2a-parallel-obj-headers.patch
		epatch "${FILESDIR}"/${PN}-1.0.2a-parallel-install-dirs.patch
		epatch "${FILESDIR}"/${PN}-1.0.2a-parallel-symlinking.patch #545028
-mike
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20151124/dccb3f2f/attachment.asc>