* [Buildroot] [v3 0/2] openssl: parallel build
@ 2015-09-16 12:06 Ryan Barnett
2015-09-16 12:06 ` [Buildroot] [v3 1/2] openssl: use git formatted patches Ryan Barnett
` (2 more replies)
0 siblings, 3 replies; 11+ messages in thread
From: Ryan Barnett @ 2015-09-16 12:06 UTC (permalink / raw)
To: buildroot
I have adopted Thomas P's patches that we submitted but in May in order
enable parallel build of openssl. I have updated the patches to use the
latest patches available from gentoo and have tested builds only on i386,
ARM, and MIPS.
Original Patches:
http://patchwork.ozlabs.org/patch/473086/
http://patchwork.ozlabs.org/patch/473087/
Ryan Barnett (2):
openssl: use git formatted patches
openssl: enable parallel build and installation
...time-building-manpages-if-we-re-not-going.patch | 27 ++++++++
...odev-Fix-issue-with-signature-generation.patch} | 71 +++++++++++++---------
package/openssl/001-do-not-build-docs.patch | 13 ----
package/openssl/openssl.hash | 2 +
package/openssl/openssl.mk | 12 ++--
5 files changed, 78 insertions(+), 47 deletions(-)
create mode 100644 package/openssl/0001-Dont-waste-time-building-manpages-if-we-re-not-going.patch
rename package/openssl/{003-cryptodev-Fix-issue-with-signature-generation.patch => 0002-cryptodev-Fix-issue-with-signature-generation.patch} (82%)
delete mode 100644 package/openssl/001-do-not-build-docs.patch
--
1.9.1
^ permalink raw reply [flat|nested] 11+ messages in thread* [Buildroot] [v3 1/2] openssl: use git formatted patches 2015-09-16 12:06 [Buildroot] [v3 0/2] openssl: parallel build Ryan Barnett @ 2015-09-16 12:06 ` Ryan Barnett 2015-09-16 20:22 ` Thomas Petazzoni 2015-09-16 12:06 ` [Buildroot] [v3 2/2] openssl: enable parallel build and installation Ryan Barnett 2015-11-18 12:57 ` [Buildroot] [v3 0/2] openssl: parallel build Thomas Petazzoni 2 siblings, 1 reply; 11+ messages in thread From: Ryan Barnett @ 2015-09-16 12:06 UTC (permalink / raw) To: buildroot Using Git formatted patches makes it easier to adjust the patches when needed. Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com> CC: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> CC: Gustavo Zacarias <gustavo@zacarias.com.ar> CC: Arnout Vandecappelle <arnout@mind.be> --- Changes v2 to v3: - Added Gustavo's SOB back to his patch after accidently droping it (suggested by Baurch) Changes v1 to v2: - Move changes from patch 2/2 to here (suggested by Arnout) - Note: git automatically detects a delete and new file for patch 001 to 0001. So there is no rename since the changes to the file are greater than 50%. Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com> --- ...time-building-manpages-if-we-re-not-going.patch | 26 ++++++++ ...odev-Fix-issue-with-signature-generation.patch} | 69 +++++++++++++--------- package/openssl/001-do-not-build-docs.patch | 13 ---- 3 files changed, 67 insertions(+), 41 deletions(-) create mode 100644 package/openssl/0001-Dont-waste-time-building-manpages-if-we-re-not-going.patch rename package/openssl/{003-cryptodev-Fix-issue-with-signature-generation.patch => 0002-cryptodev-Fix-issue-with-signature-generation.patch} (82%) delete mode 100644 package/openssl/001-do-not-build-docs.patch diff --git a/package/openssl/0001-Dont-waste-time-building-manpages-if-we-re-not-going.patch b/package/openssl/0001-Dont-waste-time-building-manpages-if-we-re-not-going.patch new file mode 100644 index 0000000..8a0081e --- /dev/null +++ b/package/openssl/0001-Dont-waste-time-building-manpages-if-we-re-not-going.patch @@ -0,0 +1,26 @@ +From 389efb564fa1453a9da835393eec9006bfae2a52 Mon Sep 17 00:00:00 2001 +From: Mike Frysinger <vapier@gentoo.org> +Date: Sat, 16 May 2015 18:53:51 +0200 +Subject: [1/2] Dont waste time building manpages if we're not going to use em. + +Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com> +--- + Makefile.org | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/Makefile.org b/Makefile.org +index 60f07cc..976ceaf 100644 +--- a/Makefile.org ++++ b/Makefile.org +@@ -527,7 +527,7 @@ dist: + dist_pem_h: + (cd crypto/pem; $(MAKE) -e $(BUILDENV) pem.h; $(MAKE) clean) + +-install: all install_docs install_sw ++install: all install_sw + + install_sw: + @$(PERL) $(TOP)/util/mkdir-p.pl $(INSTALL_PREFIX)$(INSTALLTOP)/bin \ +-- +1.9.1 + diff --git a/package/openssl/003-cryptodev-Fix-issue-with-signature-generation.patch b/package/openssl/0002-cryptodev-Fix-issue-with-signature-generation.patch similarity index 82% rename from package/openssl/003-cryptodev-Fix-issue-with-signature-generation.patch rename to package/openssl/0002-cryptodev-Fix-issue-with-signature-generation.patch index 19ed5e7..829443d 100644 --- a/package/openssl/003-cryptodev-Fix-issue-with-signature-generation.patch +++ b/package/openssl/0002-cryptodev-Fix-issue-with-signature-generation.patch @@ -1,3 +1,8 @@ +From 90fd7e8f1a316cda86ee442b43fcd7d5e5baeede Mon Sep 17 00:00:00 2001 +From: Gustavo Zacarias <gustavo@zacarias.com.ar> +Date: Sat, 16 May 2015 18:55:08 +0200 +Subject: [2/2] cryptodev: Fix issue with signature generation + Forward port of 0001-cryptodev-Fix-issue-with-signature-generation.patch from http://rt.openssl.org/Ticket/Display.html?id=2770&user=guest&pass=guest It was originally targetted at 1.0.2-beta3. @@ -5,10 +10,15 @@ It was originally targetted at 1.0.2-beta3. Without this patch digest acceleration via cryptodev is broken. Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar> +Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com> +--- + crypto/engine/eng_cryptodev.c | 195 +++++++++++++++++++++++++++++++----------- + 1 file changed, 146 insertions(+), 49 deletions(-) -diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto/engine/eng_cryptodev.c ---- openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c 2015-01-28 14:59:58.146682462 -0300 -+++ openssl-1.0.2/crypto/engine/eng_cryptodev.c 2015-01-28 15:29:25.107649077 -0300 +diff --git a/crypto/engine/eng_cryptodev.c b/crypto/engine/eng_cryptodev.c +index 926d95c..7021d9a 100644 +--- a/crypto/engine/eng_cryptodev.c ++++ b/crypto/engine/eng_cryptodev.c @@ -2,6 +2,7 @@ * Copyright (c) 2002 Bob Beck <beck@openbsd.org> * Copyright (c) 2002 Theo de Raadt @@ -17,7 +27,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto * All rights reserved. * * Redistribution and use in source and binary forms, with or without -@@ -72,7 +73,6 @@ +@@ -72,7 +73,6 @@ struct dev_crypto_state { struct session_op d_sess; int d_fd; # ifdef USE_CRYPTODEV_DIGESTS @@ -25,7 +35,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto unsigned char digest_res[HASH_MAX_LEN]; char *mac_data; int mac_len; -@@ -189,8 +189,10 @@ +@@ -189,8 +189,10 @@ static struct { static struct { int id; int nid; @@ -37,7 +47,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto { CRYPTO_MD5_HMAC, NID_hmacWithMD5, 16 }, -@@ -198,15 +200,15 @@ +@@ -198,15 +200,15 @@ static struct { CRYPTO_SHA1_HMAC, NID_hmacWithSHA1, 20 }, { @@ -57,7 +67,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto { CRYPTO_MD5, NID_md5, 16 }, -@@ -214,6 +216,15 @@ +@@ -214,6 +216,15 @@ static struct { CRYPTO_SHA1, NID_sha1, 20 }, { @@ -73,7 +83,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto 0, NID_undef, 0 }, }; -@@ -288,13 +299,14 @@ +@@ -288,13 +299,14 @@ static int get_cryptodev_ciphers(const int **cnids) static int nids[CRYPTO_ALGORITHM_MAX]; struct session_op sess; int fd, i, count = 0; @@ -89,7 +99,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto for (i = 0; ciphers[i].id && count < CRYPTO_ALGORITHM_MAX; i++) { if (ciphers[i].nid == NID_undef) -@@ -327,18 +339,19 @@ +@@ -327,18 +339,19 @@ static int get_cryptodev_digests(const int **cnids) static int nids[CRYPTO_ALGORITHM_MAX]; struct session_op sess; int fd, i, count = 0; @@ -111,7 +121,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto sess.cipher = 0; if (ioctl(fd, CIOCGSESSION, &sess) != -1 && ioctl(fd, CIOCFSESSION, &sess.ses) != -1) -@@ -424,14 +437,14 @@ +@@ -424,14 +437,14 @@ cryptodev_cipher(EVP_CIPHER_CTX *ctx, unsigned char *out, cryp.ses = sess->ses; cryp.flags = 0; cryp.len = inl; @@ -129,7 +139,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto if (!ctx->encrypt) { iiv = in + inl - ctx->cipher->iv_len; memcpy(save_iv, iiv, ctx->cipher->iv_len); -@@ -483,7 +496,7 @@ +@@ -483,7 +496,7 @@ cryptodev_init_key(EVP_CIPHER_CTX *ctx, const unsigned char *key, if ((state->d_fd = get_dev_crypto()) < 0) return (0); @@ -138,7 +148,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto sess->keylen = ctx->key_len; sess->cipher = cipher; -@@ -749,16 +762,6 @@ +@@ -749,16 +762,6 @@ static int digest_nid_to_cryptodev(int nid) return (0); } @@ -155,7 +165,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto static int cryptodev_digest_init(EVP_MD_CTX *ctx) { struct dev_crypto_state *state = ctx->md_data; -@@ -769,7 +772,6 @@ +@@ -769,7 +772,6 @@ static int cryptodev_digest_init(EVP_MD_CTX *ctx) printf("cryptodev_digest_init: Can't get digest \n"); return (0); } @@ -163,7 +173,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto memset(state, 0, sizeof(struct dev_crypto_state)); if ((state->d_fd = get_dev_crypto()) < 0) { -@@ -777,8 +779,8 @@ +@@ -777,8 +779,8 @@ static int cryptodev_digest_init(EVP_MD_CTX *ctx) return (0); } @@ -174,7 +184,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto sess->mac = digest; if (ioctl(state->d_fd, CIOCGSESSION, sess) < 0) { -@@ -794,8 +796,8 @@ +@@ -794,8 +796,8 @@ static int cryptodev_digest_init(EVP_MD_CTX *ctx) static int cryptodev_digest_update(EVP_MD_CTX *ctx, const void *data, size_t count) { @@ -184,7 +194,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto struct session_op *sess = &state->d_sess; if (!data || state->d_fd < 0) { -@@ -804,7 +806,7 @@ +@@ -804,7 +806,7 @@ static int cryptodev_digest_update(EVP_MD_CTX *ctx, const void *data, } if (!count) { @@ -193,7 +203,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto } if (!(ctx->flags & EVP_MD_CTX_FLAG_ONESHOT)) { -@@ -828,9 +830,9 @@ +@@ -828,9 +830,9 @@ static int cryptodev_digest_update(EVP_MD_CTX *ctx, const void *data, cryp.ses = sess->ses; cryp.flags = 0; cryp.len = count; @@ -205,7 +215,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto if (ioctl(state->d_fd, CIOCCRYPT, &cryp) < 0) { printf("cryptodev_digest_update: digest failed\n"); return (0); -@@ -844,8 +846,6 @@ +@@ -844,8 +846,6 @@ static int cryptodev_digest_final(EVP_MD_CTX *ctx, unsigned char *md) struct dev_crypto_state *state = ctx->md_data; struct session_op *sess = &state->d_sess; @@ -214,7 +224,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto if (!md || state->d_fd < 0) { printf("cryptodev_digest_final: illegal input\n"); return (0); -@@ -859,7 +859,7 @@ +@@ -859,7 +859,7 @@ static int cryptodev_digest_final(EVP_MD_CTX *ctx, unsigned char *md) cryp.len = state->mac_len; cryp.src = state->mac_data; cryp.dst = NULL; @@ -223,7 +233,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto if (ioctl(state->d_fd, CIOCCRYPT, &cryp) < 0) { printf("cryptodev_digest_final: digest failed\n"); return (0); -@@ -870,7 +870,7 @@ +@@ -870,7 +870,7 @@ static int cryptodev_digest_final(EVP_MD_CTX *ctx, unsigned char *md) memcpy(md, state->digest_res, ctx->digest->md_size); @@ -232,7 +242,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto } static int cryptodev_digest_cleanup(EVP_MD_CTX *ctx) -@@ -921,8 +921,8 @@ +@@ -921,8 +921,8 @@ static int cryptodev_digest_copy(EVP_MD_CTX *to, const EVP_MD_CTX *from) digest = digest_nid_to_cryptodev(to->digest->type); @@ -243,7 +253,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto sess->mac = digest; dstate->d_fd = get_dev_crypto(); -@@ -947,32 +947,116 @@ +@@ -947,32 +947,116 @@ static int cryptodev_digest_copy(EVP_MD_CTX *to, const EVP_MD_CTX *from) const EVP_MD cryptodev_sha1 = { NID_sha1, @@ -367,7 +377,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto }; # endif /* USE_CRYPTODEV_DIGESTS */ -@@ -992,6 +1076,18 @@ +@@ -992,6 +1076,18 @@ cryptodev_engine_digests(ENGINE *e, const EVP_MD **digest, case NID_sha1: *digest = &cryptodev_sha1; break; @@ -386,7 +396,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto default: # endif /* USE_CRYPTODEV_DIGESTS */ *digest = NULL; -@@ -1022,7 +1118,7 @@ +@@ -1022,7 +1118,7 @@ static int bn2crparam(const BIGNUM *a, struct crparam *crp) return (1); memset(b, 0, bytes); @@ -395,7 +405,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto crp->crp_nbits = bits; for (i = 0, j = 0; i < a->top; i++) { -@@ -1277,7 +1373,7 @@ +@@ -1277,7 +1373,7 @@ static DSA_SIG *cryptodev_dsa_do_sign(const unsigned char *dgst, int dlen, kop.crk_op = CRK_DSA_SIGN; /* inputs: dgst dsa->p dsa->q dsa->g dsa->priv_key */ @@ -404,7 +414,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto kop.crk_param[0].crp_nbits = dlen * 8; if (bn2crparam(dsa->p, &kop.crk_param[1])) goto err; -@@ -1317,7 +1413,7 @@ +@@ -1317,7 +1413,7 @@ cryptodev_dsa_verify(const unsigned char *dgst, int dlen, kop.crk_op = CRK_DSA_VERIFY; /* inputs: dgst dsa->p dsa->q dsa->g dsa->pub_key sig->r sig->s */ @@ -413,7 +423,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto kop.crk_param[0].crp_nbits = dlen * 8; if (bn2crparam(dsa->p, &kop.crk_param[1])) goto err; -@@ -1398,9 +1494,10 @@ +@@ -1398,9 +1494,10 @@ cryptodev_dh_compute_key(unsigned char *key, const BIGNUM *pub_key, DH *dh) goto err; kop.crk_iparams = 3; @@ -426,7 +436,7 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto if (ioctl(fd, CIOCKEY, &kop) == -1) { const DH_METHOD *meth = DH_OpenSSL(); -@@ -1470,7 +1567,7 @@ +@@ -1470,7 +1567,7 @@ void ENGINE_load_cryptodev(void) put_dev_crypto(fd); if (!ENGINE_set_id(engine, "cryptodev") || @@ -435,3 +445,6 @@ diff -Nura openssl-1.0.2.orig/crypto/engine/eng_cryptodev.c openssl-1.0.2/crypto !ENGINE_set_ciphers(engine, cryptodev_engine_ciphers) || !ENGINE_set_digests(engine, cryptodev_engine_digests) || !ENGINE_set_ctrl_function(engine, cryptodev_ctrl) || +-- +1.9.1 + diff --git a/package/openssl/001-do-not-build-docs.patch b/package/openssl/001-do-not-build-docs.patch deleted file mode 100644 index 135a9d3..0000000 --- a/package/openssl/001-do-not-build-docs.patch +++ /dev/null @@ -1,13 +0,0 @@ -Dont waste time building manpages if we're not going to use em. - ---- openssl/Makefile.org -+++ openssl/Makefile.org -@@ -792,7 +792,7 @@ - dist_pem_h: - (cd crypto/pem; $(MAKE) CC='${CC}' SDIRS='${SDIRS}' CFLAG='${CFLAG}' pem.h; $(MAKE) clean) - --install: all install_docs install_sw -+install: all install_sw - - install_sw: - @$(PERL) $(TOP)/util/mkdir-p.pl $(INSTALL_PREFIX)$(INSTALLTOP)/bin \ -- 1.9.1 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [Buildroot] [v3 1/2] openssl: use git formatted patches 2015-09-16 12:06 ` [Buildroot] [v3 1/2] openssl: use git formatted patches Ryan Barnett @ 2015-09-16 20:22 ` Thomas Petazzoni 0 siblings, 0 replies; 11+ messages in thread From: Thomas Petazzoni @ 2015-09-16 20:22 UTC (permalink / raw) To: buildroot Ryan, On Wed, 16 Sep 2015 07:06:24 -0500, Ryan Barnett wrote: > +Subject: [1/2] Dont waste time building manpages if we're not going to use em. Arnout also suggested to get rid of [1/2] (and [2/2] for the second patch), since this information becomes obsolete when we add a new patch. I've applied after doing this change. Thanks! Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [v3 2/2] openssl: enable parallel build and installation 2015-09-16 12:06 [Buildroot] [v3 0/2] openssl: parallel build Ryan Barnett 2015-09-16 12:06 ` [Buildroot] [v3 1/2] openssl: use git formatted patches Ryan Barnett @ 2015-09-16 12:06 ` Ryan Barnett 2015-09-16 20:22 ` Thomas Petazzoni 2015-11-18 12:57 ` [Buildroot] [v3 0/2] openssl: parallel build Thomas Petazzoni 2 siblings, 1 reply; 11+ messages in thread From: Ryan Barnett @ 2015-09-16 12:06 UTC (permalink / raw) To: buildroot This is a patch that is originally based on a patch Thomas P. submitted for an earlier version of this package. I have adopted this patch to use the latest available Gentoo parallel patch. I have also seen about a minute improvement on my build times of openssl. Part of Thomas P's original message: On my build server, the current build of OpenSSL takes 1 minutes and 20 seconds. With this commit applied, enabling parallel build and installation, the build only takes 28 seconds. All the patches are downloaded from Gentoo. There is apparently some interest in upstream OpenSSL to enable parallel build, see for example commit https://github.com/openssl/openssl/commit/c3f22253b139793ff3b91ff7e6969e180cf06815. This commit is not part of any OpenSSL release, but we can hope that the problem will resolved in the future. Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com> CC: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> CC: Gustavo Zacarias <gustavo@zacarias.com.ar> CC: Arnout Vandecappelle <arnout@mind.be> --- Changes from v2 to v3: - No changes Changes from v1 to v2: - Removed changes to existing patches in openssl (suggested by Arnout) - Updated parallel patch URL to use Gentoo gitweb (suggested by Mike) Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com> --- package/openssl/openssl.hash | 2 ++ package/openssl/openssl.mk | 12 +++++++----- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/package/openssl/openssl.hash b/package/openssl/openssl.hash index 355be79..9eaa363 100644 --- a/package/openssl/openssl.hash +++ b/package/openssl/openssl.hash @@ -1,2 +1,4 @@ # From https://www.openssl.org/source/openssl-1.0.2d.tar.gz.sha256 sha256 671c36487785628a703374c652ad2cebea45fa920ae5681515df25d9f2c9a8c8 openssl-1.0.2d.tar.gz +# Locally computed +sha256 deaf6f3af41874ecc6d63841ea14b8e6c71cea81d4a511a754bc90c9a993147f openssl-1.0.2d-parallel-build.patch?id=56bd759df1d0c750a065b8c845e93d5dfa6b549d diff --git a/package/openssl/openssl.mk b/package/openssl/openssl.mk index 3d87919..c1fc420 100644 --- a/package/openssl/openssl.mk +++ b/package/openssl/openssl.mk @@ -13,6 +13,8 @@ OPENSSL_DEPENDENCIES = zlib HOST_OPENSSL_DEPENDENCIES = host-zlib OPENSSL_TARGET_ARCH = generic32 OPENSSL_CFLAGS = $(TARGET_CFLAGS) +OPENSSL_PATCH = \ + https://gitweb.gentoo.org/repo/gentoo.git/plain/dev-libs/openssl/files/openssl-1.0.2d-parallel-build.patch?id=56bd759df1d0c750a065b8c845e93d5dfa6b549d ifeq ($(BR2_USE_MMU),) OPENSSL_CFLAGS += -DHAVE_FORK=0 @@ -99,23 +101,23 @@ OPENSSL_POST_CONFIGURE_HOOKS += OPENSSL_FIXUP_STATIC_MAKEFILE endif define HOST_OPENSSL_BUILD_CMDS - $(MAKE1) -C $(@D) + $(MAKE) -C $(@D) endef define OPENSSL_BUILD_CMDS - $(MAKE1) -C $(@D) + $(MAKE) -C $(@D) endef define OPENSSL_INSTALL_STAGING_CMDS - $(MAKE1) -C $(@D) INSTALL_PREFIX=$(STAGING_DIR) install + $(MAKE) -C $(@D) INSTALL_PREFIX=$(STAGING_DIR) install endef define HOST_OPENSSL_INSTALL_CMDS - $(MAKE1) -C $(@D) install + $(MAKE) -C $(@D) install endef define OPENSSL_INSTALL_TARGET_CMDS - $(MAKE1) -C $(@D) INSTALL_PREFIX=$(TARGET_DIR) install + $(MAKE) -C $(@D) INSTALL_PREFIX=$(TARGET_DIR) install rm -rf $(TARGET_DIR)/usr/lib/ssl rm -f $(TARGET_DIR)/usr/bin/c_rehash endef -- 1.9.1 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [Buildroot] [v3 2/2] openssl: enable parallel build and installation 2015-09-16 12:06 ` [Buildroot] [v3 2/2] openssl: enable parallel build and installation Ryan Barnett @ 2015-09-16 20:22 ` Thomas Petazzoni 0 siblings, 0 replies; 11+ messages in thread From: Thomas Petazzoni @ 2015-09-16 20:22 UTC (permalink / raw) To: buildroot Dear Ryan Barnett, On Wed, 16 Sep 2015 07:06:25 -0500, Ryan Barnett wrote: > This is a patch that is originally based on a patch Thomas P. > submitted for an earlier version of this package. I have adopted this > patch to use the latest available Gentoo parallel patch. I have also > seen about a minute improvement on my build times of openssl. > > Part of Thomas P's original message: > > On my build server, the current build of OpenSSL takes 1 minutes and > 20 seconds. With this commit applied, enabling parallel build and > installation, the build only takes 28 seconds. > > All the patches are downloaded from Gentoo. > > There is apparently some interest in upstream OpenSSL to enable > parallel build, see for example commit > https://github.com/openssl/openssl/commit/c3f22253b139793ff3b91ff7e6969e180cf06815. This > commit is not part of any OpenSSL release, but we can hope that the > problem will resolved in the future. > > Signed-off-by: Ryan Barnett <ryanbarnett3@gmail.com> > CC: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> > CC: Gustavo Zacarias <gustavo@zacarias.com.ar> > CC: Arnout Vandecappelle <arnout@mind.be> > > --- > Changes from v2 to v3: > - No changes Applied, thanks. Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [v3 0/2] openssl: parallel build 2015-09-16 12:06 [Buildroot] [v3 0/2] openssl: parallel build Ryan Barnett 2015-09-16 12:06 ` [Buildroot] [v3 1/2] openssl: use git formatted patches Ryan Barnett 2015-09-16 12:06 ` [Buildroot] [v3 2/2] openssl: enable parallel build and installation Ryan Barnett @ 2015-11-18 12:57 ` Thomas Petazzoni 2015-11-18 19:41 ` Ryan Barnett 2015-11-23 13:05 ` Yann E. MORIN 2 siblings, 2 replies; 11+ messages in thread From: Thomas Petazzoni @ 2015-11-18 12:57 UTC (permalink / raw) To: buildroot Ryan, On Wed, 16 Sep 2015 07:06:23 -0500, Ryan Barnett wrote: > I have adopted Thomas P's patches that we submitted but in May in order > enable parallel build of openssl. I have updated the patches to use the > latest patches available from gentoo and have tested builds only on i386, > ARM, and MIPS. > > Original Patches: > > http://patchwork.ozlabs.org/patch/473086/ > http://patchwork.ozlabs.org/patch/473087/ > > Ryan Barnett (2): > openssl: use git formatted patches > openssl: enable parallel build and installation I have the feeling that the parallel build patches for OpenSSL are not working properly. Look at the recent build failures we've got for the openssl and host-openssl builds: http://autobuild.buildroot.org/?reason=openssl-1.0.2d http://autobuild.buildroot.org/?reason=host-openssl-1.0.2d They are pretty much all due to a similar error: ln: failed to create symbolic link 'libssl.so': File exists ln: creating symbolic link `libcrypto.so': File exists I could spot only one that is different: In file included from apps.h:117:0, from req.c:74: ../include/openssl/bio.h:521:34: error: expected ';' before 'BIO_ctrl' # define BIO_set_fp(b,fp,c) BIO_ctrl(b,BIO_C_SET_FILE_PTR,c,(char *)fp) Do you have some ideas? Does Gentoo has some more patches? Should we revert back to non-parallel build for OpenSSL ? Thanks, Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [v3 0/2] openssl: parallel build 2015-11-18 12:57 ` [Buildroot] [v3 0/2] openssl: parallel build Thomas Petazzoni @ 2015-11-18 19:41 ` Ryan Barnett 2015-11-18 23:14 ` Arnout Vandecappelle 2015-11-24 16:01 ` Mike Frysinger 2015-11-23 13:05 ` Yann E. MORIN 1 sibling, 2 replies; 11+ messages in thread From: Ryan Barnett @ 2015-11-18 19:41 UTC (permalink / raw) To: buildroot Hi Thomas, On Wed, Nov 18, 2015 at 6:57 AM, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote: > > I have the feeling that the parallel build patches for OpenSSL are not > working properly. Look at the recent build failures we've got for the > openssl and host-openssl builds: > > http://autobuild.buildroot.org/?reason=openssl-1.0.2d > http://autobuild.buildroot.org/?reason=host-openssl-1.0.2d > > They are pretty much all due to a similar error: > > ln: failed to create symbolic link 'libssl.so': File exists > ln: creating symbolic link `libcrypto.so': File exists > > I could spot only one that is different: > > In file included from apps.h:117:0, > from req.c:74: > ../include/openssl/bio.h:521:34: error: expected ';' before 'BIO_ctrl' > # define BIO_set_fp(b,fp,c) BIO_ctrl(b,BIO_C_SET_FILE_PTR,c,(char *)fp) > > Do you have some ideas? I agree with you in that it looks like are some issues with the parallel build. It appears there is a hard to track down race condition due a symbolic being created twice during the build but as to way I don't have any clue. >Does Gentoo has some more patches? No this is the only patch for openssl 1.0.2d. There also hasn't been any updates to this patch other then the one that is currently there. > Should we revert back to non-parallel build for OpenSSL ? I guess I will leave that up to either yourself or Peter if you feel that it should. I don't have time in the next week or two to investigate the problem further so I'm OK with reverting this patch. Thanks, -Ryan -- Ryan Barnett / Sr Software Engineer Airborne Information Systems / Secure Platforms MS 131-100, C Ave NE, Cedar Rapids, IA, 52498, USA ryan.barnett at rockwellcollins.com www.rockwellcollins.com ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [v3 0/2] openssl: parallel build 2015-11-18 19:41 ` Ryan Barnett @ 2015-11-18 23:14 ` Arnout Vandecappelle 2015-11-24 16:01 ` Mike Frysinger 1 sibling, 0 replies; 11+ messages in thread From: Arnout Vandecappelle @ 2015-11-18 23:14 UTC (permalink / raw) To: buildroot On 18-11-15 20:41, Ryan Barnett wrote: > Hi Thomas, > > On Wed, Nov 18, 2015 at 6:57 AM, Thomas Petazzoni > <thomas.petazzoni@free-electrons.com> wrote: >> >> I have the feeling that the parallel build patches for OpenSSL are not >> working properly. Look at the recent build failures we've got for the >> openssl and host-openssl builds: >> >> http://autobuild.buildroot.org/?reason=openssl-1.0.2d >> http://autobuild.buildroot.org/?reason=host-openssl-1.0.2d >> >> They are pretty much all due to a similar error: >> >> ln: failed to create symbolic link 'libssl.so': File exists >> ln: creating symbolic link `libcrypto.so': File exists >> >> I could spot only one that is different: >> >> In file included from apps.h:117:0, >> from req.c:74: >> ../include/openssl/bio.h:521:34: error: expected ';' before 'BIO_ctrl' >> # define BIO_set_fp(b,fp,c) BIO_ctrl(b,BIO_C_SET_FILE_PTR,c,(char *)fp) >> >> Do you have some ideas? > > I agree with you in that it looks like are some issues with the > parallel build. It appears there is a hard to track down race > condition due a symbolic being created twice during the build but as > to way I don't have any clue. I can see where the problem is, but not what the solution is... The build-shared target depends on do_crypto and link-shared, which will be executed in parallel. do_crypto calls link_a.linux_shared -> link_a.gnu which does SYMLINK_SO; in parallel, link-shared calls symlink.linux_shared which also does SYMLINK_SO. Before the symlink is created, it is rm'ed, but there is a tiny chance that the second one is created after the rm has been called. Bam. Maybe just replace the rm with an ln -sf? Don't know if upstream will like that though. Regards, Arnout > >> Does Gentoo has some more patches? > > No this is the only patch for openssl 1.0.2d. There also hasn't been > any updates to this patch other then the one that is currently there. > >> Should we revert back to non-parallel build for OpenSSL ? > > I guess I will leave that up to either yourself or Peter if you feel > that it should. I don't have time in the next week or two to > investigate the problem further so I'm OK with reverting this patch. > > Thanks, > -Ryan > -- Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [v3 0/2] openssl: parallel build 2015-11-18 19:41 ` Ryan Barnett 2015-11-18 23:14 ` Arnout Vandecappelle @ 2015-11-24 16:01 ` Mike Frysinger 1 sibling, 0 replies; 11+ messages in thread From: Mike Frysinger @ 2015-11-24 16:01 UTC (permalink / raw) To: buildroot On 18 Nov 2015 13:41, Ryan Barnett wrote: > On Wed, Nov 18, 2015 at 6:57 AM, Thomas Petazzoni wrote: > >Does Gentoo has some more patches? > > No this is the only patch for openssl 1.0.2d. There also hasn't been > any updates to this patch other then the one that is currently there. that's incorrect. you have to look at the ebuild and then see that we apply four parallel patches for 1.0.2d, one of which specifically for this issue. https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-libs/openssl/openssl-1.0.2d-r2.ebuild?id=c8abcbe8de5d3b6cdd68c162f398c011ff6e2d9d epatch "${FILESDIR}"/${PN}-1.0.2d-parallel-build.patch epatch "${FILESDIR}"/${PN}-1.0.2a-parallel-obj-headers.patch epatch "${FILESDIR}"/${PN}-1.0.2a-parallel-install-dirs.patch epatch "${FILESDIR}"/${PN}-1.0.2a-parallel-symlinking.patch #545028 -mike -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20151124/dccb3f2f/attachment.asc> ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [v3 0/2] openssl: parallel build 2015-11-18 12:57 ` [Buildroot] [v3 0/2] openssl: parallel build Thomas Petazzoni 2015-11-18 19:41 ` Ryan Barnett @ 2015-11-23 13:05 ` Yann E. MORIN 2015-11-23 16:13 ` Thomas Petazzoni 1 sibling, 1 reply; 11+ messages in thread From: Yann E. MORIN @ 2015-11-23 13:05 UTC (permalink / raw) To: buildroot Thomas, All, On 2015-11-18 13:57 +0100, Thomas Petazzoni spake thusly: > On Wed, 16 Sep 2015 07:06:23 -0500, Ryan Barnett wrote: > > I have adopted Thomas P's patches that we submitted but in May in order > > enable parallel build of openssl. I have updated the patches to use the > > latest patches available from gentoo and have tested builds only on i386, > > ARM, and MIPS. > > > > Original Patches: > > > > http://patchwork.ozlabs.org/patch/473086/ > > http://patchwork.ozlabs.org/patch/473087/ > > > > Ryan Barnett (2): > > openssl: use git formatted patches > > openssl: enable parallel build and installation > > I have the feeling that the parallel build patches for OpenSSL are not > working properly. Look at the recent build failures we've got for the > openssl and host-openssl builds: > > http://autobuild.buildroot.org/?reason=openssl-1.0.2d > http://autobuild.buildroot.org/?reason=host-openssl-1.0.2d [--SNIP--] > Do you have some ideas? Does Gentoo has some more patches? Should we > revert back to non-parallel build for OpenSSL ? I believe we should just revert the patch. It's unfortunate that we can't buid OpenSSL in parallel, but we can't seriously go and patch their broken buildsystem, not at least until they are willing to review and apply those patches upstream first. Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [v3 0/2] openssl: parallel build 2015-11-23 13:05 ` Yann E. MORIN @ 2015-11-23 16:13 ` Thomas Petazzoni 0 siblings, 0 replies; 11+ messages in thread From: Thomas Petazzoni @ 2015-11-23 16:13 UTC (permalink / raw) To: buildroot Hello, On Mon, 23 Nov 2015 14:05:28 +0100, Yann E. MORIN wrote: > > Do you have some ideas? Does Gentoo has some more patches? Should we > > revert back to non-parallel build for OpenSSL ? > > I believe we should just revert the patch. > > It's unfortunate that we can't buid OpenSSL in parallel, but we can't > seriously go and patch their broken buildsystem, not at least until they > are willing to review and apply those patches upstream first. Agreed, we should revert the patch. Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2015-11-24 16:01 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-09-16 12:06 [Buildroot] [v3 0/2] openssl: parallel build Ryan Barnett 2015-09-16 12:06 ` [Buildroot] [v3 1/2] openssl: use git formatted patches Ryan Barnett 2015-09-16 20:22 ` Thomas Petazzoni 2015-09-16 12:06 ` [Buildroot] [v3 2/2] openssl: enable parallel build and installation Ryan Barnett 2015-09-16 20:22 ` Thomas Petazzoni 2015-11-18 12:57 ` [Buildroot] [v3 0/2] openssl: parallel build Thomas Petazzoni 2015-11-18 19:41 ` Ryan Barnett 2015-11-18 23:14 ` Arnout Vandecappelle 2015-11-24 16:01 ` Mike Frysinger 2015-11-23 13:05 ` Yann E. MORIN 2015-11-23 16:13 ` Thomas Petazzoni
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.