* [PATCH] openssl: use nativeperl to build
@ 2018-10-11 9:55 Hongxu Jia
2018-10-11 12:00 ` richard.purdie
0 siblings, 1 reply; 7+ messages in thread
From: Hongxu Jia @ 2018-10-11 9:55 UTC (permalink / raw)
To: openembedded-core, richard.purdie
If host perl is old version (such as perl v5.16.3 on CentOS-7.5),
the openssl-native failed at do_install, the reason is its Makefile
is not correct which is generated at do_configure by perl script.
Use nativeperl to assure latest perl on host for openssl build.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/recipes-connectivity/openssl/openssl_1.1.1.bb | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/meta/recipes-connectivity/openssl/openssl_1.1.1.bb b/meta/recipes-connectivity/openssl/openssl_1.1.1.bb
index 5a43a15..52d4e60 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.1.1.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.1.1.bb
@@ -23,7 +23,7 @@ SRC_URI_append_class-nativesdk = " \
SRC_URI[md5sum] = "7079eb017429e0ffb9efb42bf80ccb21"
SRC_URI[sha256sum] = "2836875a0f89c03d0fdf483941512613a50cfb421d6fd94b9f41d7279d586a3d"
-inherit lib_package multilib_header ptest
+inherit lib_package multilib_header ptest perlnative
#| ./libcrypto.so: undefined reference to `getcontext'
#| ./libcrypto.so: undefined reference to `setcontext'
@@ -105,7 +105,9 @@ do_configure () {
fi
# WARNING: do not set compiler/linker flags (-I/-D etc.) in EXTRA_OECONF, as they will fully replace the
# environment variables set by bitbake. Adjust the environment variables instead.
- perl ./Configure ${EXTRA_OECONF} ${PACKAGECONFIG_CONFARGS} --prefix=$useprefix --openssldir=${libdir}/ssl-1.1 --libdir=${libdir} $target
+ export PERL="nativeperl"
+ /usr/bin/env nativeperl ./Configure ${EXTRA_OECONF} ${PACKAGECONFIG_CONFARGS} \
+ --prefix=$useprefix --openssldir=${libdir}/ssl-1.1 --libdir=${libdir} $target
}
do_install () {
--
2.7.4
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] openssl: use nativeperl to build
2018-10-11 9:55 [PATCH] openssl: use nativeperl to build Hongxu Jia
@ 2018-10-11 12:00 ` richard.purdie
2018-10-11 14:58 ` Hongxu Jia
0 siblings, 1 reply; 7+ messages in thread
From: richard.purdie @ 2018-10-11 12:00 UTC (permalink / raw)
To: Hongxu Jia, openembedded-core
On Thu, 2018-10-11 at 02:55 -0700, Hongxu Jia wrote:
> If host perl is old version (such as perl v5.16.3 on CentOS-7.5),
> the openssl-native failed at do_install, the reason is its Makefile
> is not correct which is generated at do_configure by perl script.
>
> Use nativeperl to assure latest perl on host for openssl build.
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
> meta/recipes-connectivity/openssl/openssl_1.1.1.bb | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
I think this will have a noticeable effect on built times unfortunately
since it means openssl-native would then depend upon perl-native...
Cheers,
Richard
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] openssl: use nativeperl to build
2018-10-11 12:00 ` richard.purdie
@ 2018-10-11 14:58 ` Hongxu Jia
2018-10-12 7:54 ` Hongxu Jia
0 siblings, 1 reply; 7+ messages in thread
From: Hongxu Jia @ 2018-10-11 14:58 UTC (permalink / raw)
To: richard.purdie, openembedded-core
On 2018年10月11日 20:00, richard.purdie@linuxfoundation.org wrote:
> On Thu, 2018-10-11 at 02:55 -0700, Hongxu Jia wrote:
>> If host perl is old version (such as perl v5.16.3 on CentOS-7.5),
>> the openssl-native failed at do_install, the reason is its Makefile
>> is not correct which is generated at do_configure by perl script.
>>
>> Use nativeperl to assure latest perl on host for openssl build.
>>
>> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
>> ---
>> meta/recipes-connectivity/openssl/openssl_1.1.1.bb | 6 ++++--
>> 1 file changed, 4 insertions(+), 2 deletions(-)
> I think this will have a noticeable effect on built times unfortunately
> since it means openssl-native would then depend upon perl-native...
Hi RP,
Yes, I do the test to build openssl-native from scratch, it requires
extra 4 minutes
1. Use host perl
18 task
real 1m33.978s
user 0m0.728s
sys 0m0.080s
2. Use nativeperl
114 tasks
real 5m49.911s
user 0m1.372s
sys 0m0.140s
But for a image build, the cost is not much
(To exclude network affect, we assure all tarball/git exist in downloads)
But build a image from scratch:
time bitbake core-image-minimal
1. Use host perl
NOTE: Tasks Summary: Attempted 1655 tasks of which 5 didn't need to be
rerun and all succeeded.
real 28m29.625s
user 0m7.296s
sys 0m0.988s
2. Use native perl
NOTE: Tasks Summary: Attempted 1655 tasks of which 5 didn't need to be
rerun and all succeeded.
real 28m42.619s
user 0m7.496s
sys 0m0.952s
As upstream said, it indeed require specific PERL version
`If you are building OpenSSL 1.1.0 and above, then you will also need
PERL 5.10 or high'
https://wiki.openssl.org/index.php/Compilation_and_Installation
And I found at least PERL 5.16.3 not work for OpenSSL 1.1.1
Or add a perl version detecting in insane check? We should report a lowest
workable perl version number when detect failed. But we may update the
version
number in insane check when openssl is upgraded.
//Hongxu
> Cheers,
>
> Richard
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] openssl: use nativeperl to build
2018-10-11 14:58 ` Hongxu Jia
@ 2018-10-12 7:54 ` Hongxu Jia
2018-10-12 8:57 ` richard.purdie
0 siblings, 1 reply; 7+ messages in thread
From: Hongxu Jia @ 2018-10-12 7:54 UTC (permalink / raw)
To: richard.purdie, openembedded-core
On 2018年10月11日 22:58, Hongxu Jia wrote:
> On 2018年10月11日 20:00, richard.purdie@linuxfoundation.org wrote:
>> On Thu, 2018-10-11 at 02:55 -0700, Hongxu Jia wrote:
>>> If host perl is old version (such as perl v5.16.3 on CentOS-7.5),
>>> the openssl-native failed at do_install, the reason is its Makefile
>>> is not correct which is generated at do_configure by perl script.
>>>
>>> Use nativeperl to assure latest perl on host for openssl build.
>>>
>>> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
>>> ---
>>> meta/recipes-connectivity/openssl/openssl_1.1.1.bb | 6 ++++--
>>> 1 file changed, 4 insertions(+), 2 deletions(-)
>> I think this will have a noticeable effect on built times unfortunately
>> since it means openssl-native would then depend upon perl-native...
Hi RP,
With some digging on upstream, more than perl version,
the building also requires to install perl Text::Template module
which does not exist in core perl, I think we have to add the recipe
[snip]
Notes on Perl modules we use
----------------------------
We make increasing use of Perl modules, and do our best to limit
ourselves to core Perl modules to keep the requirements down. There
are just a few exceptions:
Test::More We require the minimum version to be 0.96, which
appeared in Perl 5.13.4, because that version was
the first to have all the features we're using.
This module is required for testing only! If you
don't plan on running the tests, you don't need to
bother with this one.
Text::Template This module is not part of the core Perl modules.
As a matter of fact, the core Perl modules do not
include any templating module to date.
This module is absolutely needed, configuration
depends on it.
[snip]
https://github.com/openssl/openssl/blob/master/NOTES.PERL
https://github.com/openssl/openssl/commit/4e351ca92e3a1f447cef3d2e330f13941f9412c6
//Hongxu
> Hi RP,
>
> Yes, I do the test to build openssl-native from scratch, it requires
> extra 4 minutes
>
> 1. Use host perl
> 18 task
> real 1m33.978s
> user 0m0.728s
> sys 0m0.080s
>
> 2. Use nativeperl
> 114 tasks
> real 5m49.911s
> user 0m1.372s
> sys 0m0.140s
>
> But for a image build, the cost is not much
> (To exclude network affect, we assure all tarball/git exist in downloads)
> But build a image from scratch:
>
> time bitbake core-image-minimal
>
> 1. Use host perl
> NOTE: Tasks Summary: Attempted 1655 tasks of which 5 didn't need to be
> rerun and all succeeded.
>
> real 28m29.625s
> user 0m7.296s
> sys 0m0.988s
>
> 2. Use native perl
> NOTE: Tasks Summary: Attempted 1655 tasks of which 5 didn't need to be
> rerun and all succeeded.
>
> real 28m42.619s
> user 0m7.496s
> sys 0m0.952s
>
>
> As upstream said, it indeed require specific PERL version
>
> `If you are building OpenSSL 1.1.0 and above, then you will also need
> PERL 5.10 or high'
>
> https://wiki.openssl.org/index.php/Compilation_and_Installation
>
> And I found at least PERL 5.16.3 not work for OpenSSL 1.1.1
>
> Or add a perl version detecting in insane check? We should report a
> lowest
> workable perl version number when detect failed. But we may update the
> version
> number in insane check when openssl is upgraded.
>
> //Hongxu
>
>> Cheers,
>>
>> Richard
>>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] openssl: use nativeperl to build
2018-10-12 7:54 ` Hongxu Jia
@ 2018-10-12 8:57 ` richard.purdie
2018-10-12 9:03 ` Hongxu Jia
0 siblings, 1 reply; 7+ messages in thread
From: richard.purdie @ 2018-10-12 8:57 UTC (permalink / raw)
To: Hongxu Jia, openembedded-core
On Fri, 2018-10-12 at 15:54 +0800, Hongxu Jia wrote:
> On 2018年10月11日 22:58, Hongxu Jia wrote:
> > On 2018年10月11日 20:00, richard.purdie@linuxfoundation.org wrote:
> > > On Thu, 2018-10-11 at 02:55 -0700, Hongxu Jia wrote:
> > > > If host perl is old version (such as perl v5.16.3 on CentOS-
> > > > 7.5),
> > > > the openssl-native failed at do_install, the reason is its
> > > > Makefile
> > > > is not correct which is generated at do_configure by perl
> > > > script.
> > > >
> > > > Use nativeperl to assure latest perl on host for openssl build.
> > > >
> > > > Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> > > > ---
> > > > meta/recipes-connectivity/openssl/openssl_1.1.1.bb | 6 ++++--
> > > > 1 file changed, 4 insertions(+), 2 deletions(-)
> > >
> > > I think this will have a noticeable effect on built times
> > > unfortunately
> > > since it means openssl-native would then depend upon perl-
> > > native...
>
> Hi RP,
>
> With some digging on upstream, more than perl version,
> the building also requires to install perl Text::Template module
> which does not exist in core perl, I think we have to add the recipe
>
> [snip]
> Notes on Perl modules we use
> ----------------------------
> We make increasing use of Perl modules, and do our best to limit
> ourselves to core Perl modules to keep the requirements down. There
> are just a few exceptions:
> Test::More We require the minimum version to be 0.96, which
> appeared in Perl 5.13.4, because that version
> was
> the first to have all the features we're using.
> This module is required for testing only! If
> you
> don't plan on running the tests, you don't need
> to
> bother with this one.
>
> Text::Template This module is not part of the core Perl
> modules.
> As a matter of fact, the core Perl modules do
> not
> include any templating module to date.
> This module is absolutely needed, configuration
> depends on it.
> [snip]
>
> https://github.com/openssl/openssl/blob/master/NOTES.PERL
>
https://github.com/openssl/openssl/commit/4e351ca92e3a1f447cef3d2e330f13941f9412c6
That is a good find. Perhaps we could check for that in sanity.bbclass?
Is it commonly available in distributions like centos?
I am worried about the build time impact of a nativeperl dependency as
whilst your test doesn't show it in the image build, it will show up on
slower build machines with fewer threads.
Cheers,
Richard
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] openssl: use nativeperl to build
2018-10-12 8:57 ` richard.purdie
@ 2018-10-12 9:03 ` Hongxu Jia
2018-10-12 9:26 ` Hongxu Jia
0 siblings, 1 reply; 7+ messages in thread
From: Hongxu Jia @ 2018-10-12 9:03 UTC (permalink / raw)
To: richard.purdie, openembedded-core
[-- Attachment #1: Type: text/plain, Size: 2226 bytes --]
On 2018年10月12日 16:57, richard.purdie@linuxfoundation.org wrote:
>> Hi RP,
>>
>> With some digging on upstream, more than perl version,
>> the building also requires to install perl Text::Template module
>> which does not exist in core perl, I think we have to add the recipe
>>
>> [snip]
>> Notes on Perl modules we use
>> ----------------------------
>> We make increasing use of Perl modules, and do our best to limit
>> ourselves to core Perl modules to keep the requirements down. There
>> are just a few exceptions:
>> Test::More We require the minimum version to be 0.96, which
>> appeared in Perl 5.13.4, because that version
>> was
>> the first to have all the features we're using.
>> This module is required for testing only! If
>> you
>> don't plan on running the tests, you don't need
>> to
>> bother with this one.
>>
>> Text::Template This module is not part of the core Perl
>> modules.
>> As a matter of fact, the core Perl modules do
>> not
>> include any templating module to date.
>> This module is absolutely needed, configuration
>> depends on it.
>> [snip]
>>
>> https://github.com/openssl/openssl/blob/master/NOTES.PERL
>>
> https://github.com/openssl/openssl/commit/4e351ca92e3a1f447cef3d2e330f13941f9412c6
>
> That is a good find. Perhaps we could check for that in sanity.bbclass?
> Is it commonly available in distributions like centos?
I am afraid the perl module (Text::Template) is not common,
available on most distro, since they do not need to build
openssl :(
But I am curious that without that perl module, the build seem
ok with newer perl (perl v5.22.1 in ubuntu 1804 and without Text::Template)
I am going on digging the root cause why the build works well
without the perl module
//Hongxu
> I am worried about the build time impact of a nativeperl dependency as
> whilst your test doesn't show it in the image build, it will show up on
> slower build machines with fewer threads.
>
> Cheers,
>
> Richard
>
[-- Attachment #2: Type: text/html, Size: 3231 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] openssl: use nativeperl to build
2018-10-12 9:03 ` Hongxu Jia
@ 2018-10-12 9:26 ` Hongxu Jia
0 siblings, 0 replies; 7+ messages in thread
From: Hongxu Jia @ 2018-10-12 9:26 UTC (permalink / raw)
To: richard.purdie, openembedded-core
[-- Attachment #1: Type: text/plain, Size: 2061 bytes --]
On 2018年10月12日 17:03, Hongxu Jia wrote:
>> https://github.com/openssl/openssl/commit/4e351ca92e3a1f447cef3d2e330f13941f9412c6
>>
>> That is a good find. Perhaps we could check for that in sanity.bbclass?
>> Is it commonly available in distributions like centos?
> I am afraid the perl module (Text::Template) is not common,
> available on most distro, since they do not need to build
> openssl :(
>
> But I am curious that without that perl module, the build seem
> ok with newer perl (perl v5.22.1 in ubuntu 1804 and without
> Text::Template)
>
> I am going on digging the root cause why the build works well
> without the perl module
>
I know the reason
It will try to use Text::Template, if failed it fallback to load the old
one in perl source
[snip util/dofile.pl]
use with_fallback "Text::Template 1.46";
[snip util/dofile.pl]
[snip util/perl/with_fallback.pm]
sub import {
shift;
use File::Basename;
use File::Spec::Functions;
foreach (@_) {
eval "use $_";
if ($@) {
unshift @INC, catdir(dirname(__FILE__),
"..", "..", "external", "perl");
my $transfer = "transfer::$_";
eval "use $transfer";
shift @INC;
warn $@ if $@;
}
}
}
[snip util/perl/with_fallback.pm]
https://github.com/openssl/openssl/commit/8ff2af548303d311ce3591406111f77862875a60
[comments]
Bundle the non core Perl module Text::Template
Because we're using Text::Template and we know it's a non core Perl
module, we choose to bundle it into our source, for convenience.
external/perl/Downloaded.txt document what modules we choose to bundle
this way and exactly where we downloaded it from.
With this changes comes the transfer module for with_fallback.
Reviewed-by: Rich Salz <rsalz@openssl.org>
[comments]
//Hongxu
> //Hongxu
[-- Attachment #2: Type: text/html, Size: 3547 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2018-10-12 9:26 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-10-11 9:55 [PATCH] openssl: use nativeperl to build Hongxu Jia
2018-10-11 12:00 ` richard.purdie
2018-10-11 14:58 ` Hongxu Jia
2018-10-12 7:54 ` Hongxu Jia
2018-10-12 8:57 ` richard.purdie
2018-10-12 9:03 ` Hongxu Jia
2018-10-12 9:26 ` Hongxu Jia
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.