Re: [OE-core] [PATCH] openssl: update from 3.0.8 to 3.1.0

Re: [OE-core] [PATCH] openssl: update from 3.0.8 to 3.1.0

[Randy MacLeod]

[-- Attachment #1: Type: text/plain, Size: 9589 bytes --]

On 2023-03-15 00:48, Randy MacLeod via lists.openembedded.org wrote:
> >From the NEWS.md file:
>
>    ### Major changes between OpenSSL 3.0 and OpenSSL 3.1.0 [14 Mar 2023]

I know we're in feature freeze but this openssl update that came out on 
March 14th looks interesting.

I've sent it in before local tests complete so the Europeans / Brits can
put the update through the YP autobuilder tests.


The update was pretty simple and the code change isn't huge.

There's a likely very desirable FIPS 140 updated mentioned in:

https://www.openssl.org/blog/blog/2023/03/07/OpenSSL3.1Release/

     OpenSSL 3.1 is mostly a small increment over the functionality 
available in OpenSSL 3.0.
     The main changes are:

  *

    A FIPS 140-3 compliant FIPS Provider.

    The FIPS provider has been upgraded to be FIPS 140-3 compliant. ...

Local builds are still underway but going as well as expected.
See below for what I have on hand so far.

../Randy


$ cat ../b/ossl-3.1.0/openssl-buildall.log
BUILDALL-QEMU LOG FOR openssl
START TIME: 2023-03-14_18:09:37
HOSTNAME: yow-fedora-builder1
HOST OS: Fedora Linux 37 (Server Edition)
HOST KERNEL: 6.0.14-300.fc37.x86_64
===============
BUILD RESULTS:
[glibc]
PASS: qemumips
PASS: qemumips64
PASS: qemuppc
PASS: qemuriscv32
PASS: qemuriscv64
PASS: qemux86
PASS: qemux86-64
PASS: qemuarm
PASS: qemuarm64
PASS: qemuarmv5
FAIL: qemuloongarch64
PASS: qemuppc64
[musl]
PASS: qemumips
PASS: qemumips64
PASS: qemuppc
FAIL: qemuriscv32
PASS: qemuriscv64
PASS: qemux86
PASS: qemux86-64

...



ptest are also still executing but seem to be in good shape:

Poky (Yocto Project Reference Distro) 
4.1+snapshot-c9c1d6af8f00ad11cadd400c860a439349ce5b7d qemux86-64 /dev/ttyS0

qemux86-64 login: root
root@qemux86-64:~# ptest-runner openssl
START: ptest-runner
2023-03-15T04:46
BEGIN: /usr/lib/openssl/ptest
SKIP: 00-prep_fipsmodule_cnf.t .. skipped: FIPS module config file only 
supported in a fips build
Files=1, Tests=0,  0 wallclock secs ( 0.02 usr  0.00 sys +  0.38 cusr  
0.11 csys =  0.51 CPU)
Result: NOTESTS
PASS: 01-test_abort.t .................... ok
SKIP: 01-test_fipsmodule_cnf.t ........... skipped: Test only supported 
in a fips build
PASS: 01-test_sanity.t ................... ok
PASS: 01-test_symbol_presence.t .......... ok
PASS: 01-test_test.t ..................... ok
PASS: 02-test_errstr.t ................... ok
PASS: 02-test_internal_context.t ......... ok
PASS: 02-test_internal_ctype.t ........... ok
PASS: 02-test_internal_exts.t ............ ok
PASS: 02-test_internal_keymgmt.t ......... ok
PASS: 02-test_internal_provider.t ........ ok
PASS: 02-test_lhash.t .................... ok
PASS: 02-test_localetest.t ............... ok
PASS: 02-test_ordinals.t ................. ok
PASS: 02-test_sparse_array.t ............. ok
PASS: 02-test_stack.t .................... ok
PASS: 03-test_exdata.t ................... ok
SKIP: 03-test_fipsinstall.t .............. skipped: Test only supported 
in a fips build
PASS: 03-test_internal_asn1.t ............ ok
PASS: 03-test_internal_asn1_dsa.t ........ ok
PASS: 03-test_internal_bn.t .............. ok
PASS: 03-test_internal_chacha.t .......... ok
PASS: 03-test_internal_curve448.t ........ ok
PASS: 03-test_internal_ec.t .............. ok
PASS: 03-test_internal_ffc.t ............. ok
PASS: 03-test_internal_mdc2.t ............ ok
PASS: 03-test_internal_modes.t ........... ok
PASS: 03-test_internal_namemap.t ......... ok
PASS: 03-test_internal_poly1305.t ........ ok
PASS: 03-test_internal_rsa_sp800_56b.t ... ok
PASS: 03-test_internal_siphash.t ......... ok
PASS: 03-test_internal_sm2.t ............. ok
PASS: 03-test_internal_sm3.t ............. ok
PASS: 03-test_internal_sm4.t ............. ok
PASS: 03-test_internal_ssl_cert_table.t .. ok
PASS: 03-test_internal_x509.t ............ ok
PASS: 03-test_params_api.t ............... ok
PASS: 03-test_property.t ................. ok
PASS: 03-test_ui.t ....................... ok
PASS: 04-test_asn1_decode.t .............. ok
PASS: 04-test_asn1_encode.t .............. ok
PASS: 04-test_asn1_string_table.t ........ ok
PASS: 04-test_bio_callback.t ............. ok
PASS: 04-test_bio_core.t ................. ok
PASS: 04-test_bioprint.t ................. ok
PASS: 04-test_conf.t ..................... ok
PASS: 04-test_encoder_decoder.t .......... ok
PASS: 04-test_encoder_decoder_legacy.t ... ok
PASS: 04-test_err.t ...................... ok
PASS: 04-test_hexstring.t ................ ok
PASS: 04-test_nodefltctx.t ............... ok
PASS: 04-test_param_build.t .............. ok
PASS: 04-test_params.t ................... ok
PASS: 04-test_params_conversion.t ........ ok
PASS: 04-test_pem_read_depr.t ............ ok
PASS: 04-test_pem_reading.t .............. ok
PASS: 04-test_provfetch.t ................ ok
PASS: 04-test_provider.t ................. ok
PASS: 04-test_provider_fallback.t ........ ok
PASS: 04-test_provider_pkey.t ............ ok
PASS: 04-test_punycode.t ................. ok
PASS: 04-test_upcalls.t .................. ok
PASS: 05-test_bf.t ....................... ok
PASS: 05-test_cast.t ..................... ok
PASS: 05-test_cmac.t ..................... ok
PASS: 05-test_des.t ...................... ok
PASS: 05-test_hmac.t ..................... ok
PASS: 05-test_idea.t ..................... ok
PASS: 05-test_pbe.t ...................... ok
PASS: 05-test_rand.t ..................... ok
PASS: 05-test_rc2.t ...................... ok
PASS: 05-test_rc4.t ...................... ok
SKIP: 05-test_rc5.t ...................... skipped: rc5 is not supported 
by this OpenSSL build
PASS: 06-test_algorithmid.t .............. ok
PASS: 06-test_rdcpu_sanity.t ............. ok
PASS: 10-test_bn.t ....................... ok
PASS: 10-test_exp.t ...................... ok
PASS: 15-test_dh.t ....................... ok
PASS: 15-test_dsa.t ...................... ok
PASS: 15-test_dsaparam.t ................. ok
PASS: 15-test_ec.t ....................... ok
PASS: 15-test_ecdsa.t .................... ok
PASS: 15-test_ecparam.t .................. ok
PASS: 15-test_gendh.t .................... ok
PASS: 15-test_gendhparam.t ............... ok
PASS: 15-test_gendsa.t ................... ok
PASS: 15-test_genec.t .................... ok
PASS: 15-test_genrsa.t ................... ok
PASS: 15-test_mp_rsa.t ................... ok
PASS: 15-test_out_option.t ............... ok
PASS: 15-test_rsa.t ...................... ok
PASS: 15-test_rsaoaep.t .................. ok
PASS: 15-test_rsapss.t ................... ok
PASS: 15-test_sha.t ...................... ok
PASS: 20-test_app.t ...................... ok
SKIP: 20-test_cli_fips.t ................. skipped: Test only supported 
in a fips build with security checks
PASS: 20-test_dgst.t ..................... ok
PASS: 20-test_dhparam.t .................. ok
PASS: 20-test_dhparam_check.t ............ ok
PASS: 20-test_enc.t ...................... ok
PASS: 20-test_enc_more.t ................. ok
PASS: 20-test_kdf.t ...................... ok
PASS: 20-test_legacy_okay.t .............. ok
PASS: 20-test_mac.t ...................... ok
PASS: 20-test_passwd.t ................... ok
PASS: 20-test_pkeyutl.t .................. ok
PASS: 20-test_rand_config.t .............. ok
PASS: 20-test_spkac.t .................... ok
PASS: 25-test_crl.t ...................... ok
PASS: 25-test_d2i.t ...................... ok
PASS: 25-test_eai_data.t ................. ok
PASS: 25-test_pkcs7.t .................... ok
PASS: 25-test_req.t ...................... ok
PASS: 25-test_rusext.t ................... ok
PASS: 25-test_sid.t ...................... ok
PASS: 25-test_verify.t ................... ok
PASS: 25-test_verify_store.t ............. ok
PASS: 25-test_x509.t ..................... ok
SKIP: 30-test_acvp.t ..................... skipped: ACVP is not 
supported by this test
PASS: 30-test_aesgcm.t ................... ok
PASS: 30-test_afalg.t .................... ok
PASS: 30-test_defltfips.t ................ ok
PASS: 30-test_engine.t ................... ok
PASS: 30-test_evp.t ...................... ok
PASS: 30-test_evp_extra.t ................ ok
PASS: 30-test_evp_fetch_prov.t ........... ok
PASS: 30-test_evp_kdf.t .................. ok
PASS: 30-test_evp_libctx.t ............... ok
PASS: 30-test_evp_pkey_dparam.t .......... ok
PASS: 30-test_evp_pkey_provided.t ........ ok
PASS: 30-test_pbelu.t .................... ok
PASS: 30-test_pkey_meth.t ................ ok
PASS: 30-test_pkey_meth_kdf.t ............ ok
PASS: 30-test_prov_config.t .............. ok
PASS: 30-test_provider_status.t .......... ok
PASS: 40-test_rehash.t ................... ok
PASS: 60-test_x509_check_cert_pkey.t ..... ok
PASS: 60-test_x509_dup_cert.t ............ ok
PASS: 60-test_x509_store.t ............... ok
PASS: 60-test_x509_time.t ................ ok
PASS: 61-test_bio_prefix.t ............... ok
PASS: 61-test_bio_readbuffer.t ........... ok
PASS: 65-test_cmp_asn.t .................. ok
PASS: 65-test_cmp_client.t ............... ok
PASS: 65-test_cmp_ctx.t .................. ok
PASS: 65-test_cmp_hdr.t .................. ok
PASS: 65-test_cmp_msg.t .................. ok
PASS: 65-test_cmp_protect.t .............. ok
PASS: 65-test_cmp_server.t ............... ok
PASS: 65-test_cmp_status.t ............... ok
PASS: 65-test_cmp_vfy.t .................. ok
PASS: 66-test_ossl_store.t ............... ok
PASS: 70-test_asyncio.t .................. ok
...



[-- Attachment #2: Type: text/html, Size: 35957 bytes --]

Re: [OE-core] [PATCH] openssl: update from 3.0.8 to 3.1.0

[Randy MacLeod]

[-- Attachment #1: Type: text/plain, Size: 5877 bytes --]

The ptests all passed or were skipped as you'd hope and expect from 
openssl.
Results below.

../Randy


On 2023-03-15 00:58, Randy MacLeod via lists.openembedded.org wrote:
> On 2023-03-15 00:48, Randy MacLeod via lists.openembedded.org wrote:
>> >From the NEWS.md file:
>>
>>
> PASS: 65-test_cmp_ctx.t .................. ok
> PASS: 65-test_cmp_hdr.t .................. ok
> PASS: 65-test_cmp_msg.t .................. ok
> PASS: 65-test_cmp_protect.t .............. ok
> PASS: 65-test_cmp_server.t ............... ok
> PASS: 65-test_cmp_status.t ............... ok
> PASS: 65-test_cmp_vfy.t .................. ok
> PASS: 66-test_ossl_store.t ............... ok
> PASS: 70-test_asyncio.t .................. ok
> ...
>
>
PASS: 70-test_bad_dtls.t ................. ok
PASS: 70-test_clienthello.t .............. ok
PASS: 70-test_comp.t ..................... ok
PASS: 70-test_key_share.t ................ ok
PASS: 70-test_packet.t ................... ok
PASS: 70-test_recordlen.t ................ ok
PASS: 70-test_renegotiation.t ............ ok
PASS: 70-test_servername.t ............... ok
PASS: 70-test_sslcbcpadding.t ............ ok
PASS: 70-test_sslcertstatus.t ............ ok
PASS: 70-test_sslextension.t ............. ok
PASS: 70-test_sslmessages.t .............. ok
PASS: 70-test_sslrecords.t ............... ok
PASS: 70-test_sslsessiontick.t ........... ok
PASS: 70-test_sslsigalgs.t ............... ok
PASS: 70-test_sslsignature.t ............. ok
PASS: 70-test_sslskewith0p.t ............. ok
PASS: 70-test_sslversions.t .............. ok
PASS: 70-test_sslvertol.t ................ ok
PASS: 70-test_tls13alerts.t .............. ok
PASS: 70-test_tls13cookie.t .............. ok
PASS: 70-test_tls13downgrade.t ........... ok
PASS: 70-test_tls13hrr.t ................. ok
PASS: 70-test_tls13kexmodes.t ............ ok
PASS: 70-test_tls13messages.t ............ ok
PASS: 70-test_tls13psk.t ................. ok
PASS: 70-test_tlsextms.t ................. ok
PASS: 70-test_verify_extra.t ............. ok
PASS: 70-test_wpacket.t .................. ok
PASS: 71-test_ssl_ctx.t .................. ok
PASS: 79-test_http.t ..................... ok
PASS: 80-test_ca.t ....................... ok
PASS: 80-test_cipherbytes.t .............. ok
PASS: 80-test_cipherlist.t ............... ok
PASS: 80-test_ciphername.t ............... ok
PASS: 80-test_cmp_http.t ................. ok
PASS: 80-test_cms.t ...................... ok
PASS: 80-test_cmsapi.t ................... ok
PASS: 80-test_ct.t ....................... ok
PASS: 80-test_dane.t ..................... ok
PASS: 80-test_dtls.t ..................... ok
PASS: 80-test_dtls_mtu.t ................. ok
PASS: 80-test_dtlsv1listen.t ............. ok
PASS: 80-test_ocsp.t ..................... ok
PASS: 80-test_pkcs12.t ................... ok
PASS: 80-test_ssl_new.t .................. ok
PASS: 80-test_ssl_old.t .................. ok
PASS: 80-test_ssl_test_ctx.t ............. ok
PASS: 80-test_sslcorrupt.t ............... ok
PASS: 80-test_tsa.t ...................... ok
PASS: 80-test_x509aux.t .................. ok
PASS: 81-test_cmp_cli.t .................. ok
PASS: 90-test_asn1_time.t ................ ok
PASS: 90-test_async.t .................... ok
PASS: 90-test_bio_enc.t .................. ok
PASS: 90-test_bio_memleak.t .............. ok
PASS: 90-test_constant_time.t ............ ok
PASS: 90-test_fatalerr.t ................. ok
SKIP: 90-test_fipsload.t ................. skipped: Test is disabled 
with disabled fips
PASS: 90-test_gmdiff.t ................... ok
PASS: 90-test_ige.t ...................... ok
PASS: 90-test_includes.t ................. ok
PASS: 90-test_memleak.t .................. ok
PASS: 90-test_overhead.t ................. ok
PASS: 90-test_secmem.t ................... ok
PASS: 90-test_shlibload.t ................ ok
PASS: 90-test_srp.t ...................... ok
PASS: 90-test_sslapi.t ................... ok
PASS: 90-test_sslbuffers.t ............... ok
PASS: 90-test_store.t .................... ok
PASS: 90-test_sysdefault.t ............... ok
PASS: 90-test_threads.t .................. ok
PASS: 90-test_time_offset.t .............. ok
PASS: 90-test_tls13ccs.t ................. ok
PASS: 90-test_tls13encryption.t .......... ok
PASS: 90-test_tls13secrets.t ............. ok
PASS: 90-test_traceapi.t ................. ok
PASS: 90-test_v3name.t ................... ok
PASS: 91-test_pkey_check.t ............... ok
SKIP: 95-test_external_gost_engine.t ..... skipped: No external tests in 
this configuration
SKIP: 95-test_external_krb5.t ............ skipped: No external tests in 
this configuration
SKIP: 95-test_external_oqsprovider.t ..... skipped: No external tests in 
this configuration
SKIP: 95-test_external_pyca.t ............ skipped: No external tests in 
this configuration
SKIP: 95-test_external_tlsfuzzer.t ....... skipped: No external tests in 
this configuration
PASS: 99-test_ecstress.t ................. ok
PASS: 99-test_fuzz_asn1.t ................ ok
PASS: 99-test_fuzz_asn1parse.t ........... ok
PASS: 99-test_fuzz_bignum.t .............. ok
PASS: 99-test_fuzz_bndiv.t ............... ok
PASS: 99-test_fuzz_client.t .............. ok
PASS: 99-test_fuzz_cmp.t ................. ok
PASS: 99-test_fuzz_cms.t ................. ok
PASS: 99-test_fuzz_conf.t ................ ok
PASS: 99-test_fuzz_crl.t ................. ok
PASS: 99-test_fuzz_ct.t .................. ok
PASS: 99-test_fuzz_server.t .............. ok
PASS: 99-test_fuzz_x509.t ................ ok
All tests successful.
Files=250, Tests=3354, 588 wallclock secs (11.78 usr  1.17 sys + 465.74 
cusr 114.46 csys = 593.15 CPU)
Result: PASS
DURATION: 589
END: /usr/lib/openssl/ptest
2023-03-15T04:56
STOP: ptest-runner
TOTAL: 1 FAIL: 0


-- 
# Randy MacLeod
# Wind River Linux

[-- Attachment #2: Type: text/html, Size: 10282 bytes --]

Re: [OE-core] [PATCH] openssl: update from 3.0.8 to 3.1.0

[Randy MacLeod]

[-- Attachment #1: Type: text/plain, Size: 474 bytes --]

On 2023-03-15 09:58, Randy MacLeod via lists.openembedded.org wrote:
> The ptests all passed or were skipped as you'd hope and expect from 
> openssl.
> Results below.


The autobuilder shows that the updated openssl isn't reproducible:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2552/steps/12/logs/stdio

I have something more urgent to deal with next so if anyone wants to 
pick this up,
just say so, so that we don't duplicate effort.

../Randy


[-- Attachment #2: Type: text/html, Size: 1045 bytes --]

Re: [OE-core] [PATCH] openssl: update from 3.0.8 to 3.1.0

[Richard Purdie]

On Wed, 2023-03-15 at 20:02 -0400, Randy MacLeod wrote:
> On 2023-03-15 09:58, Randy MacLeod via lists.openembedded.org wrote:
>  
> > The ptests all passed or were skipped as you'd hope and expect from
> > openssl. 
> >  Results below.
> 
> The autobuilder shows that the updated openssl isn't reproducible:
> https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2552/steps/12/logs/stdio
> I have something more urgent to deal with next so if anyone wants to
> pick this up,
>  just say so, so that we don't duplicate effort.

I'm not saying I'll fix this but I did take a look just to get an idea
of what is wrong.

The diffoscope output is here:

http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230315-7znox1mp/packages/diff-html/

and we see an assembler file has random strings in it.

In the openssl source this is generated by a perl script,
rypto/modes/asm/aes-gcm-avx512.pl

Wearing suitable safety equipment to stop it burning, inside the perl
script you can see references like:

my $rndsuffix = &random_string();

I think this is done to avoid symbol clashes but I couldn't see
anything that would actually clash so more understanding of that this
is doing is needed. I wondered if we could set some kind of perl
seeding to make it deterministic.

I'll reply again if I learn anything more.

Cheers,

Richard

Re: [OE-core] [PATCH] openssl: update from 3.0.8 to 3.1.0

[Alexander Kanavin]

On Thu, 16 Mar 2023 at 09:30, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> In the openssl source this is generated by a perl script,
> rypto/modes/asm/aes-gcm-avx512.pl
>
> Wearing suitable safety equipment to stop it burning, inside the perl
> script you can see references like:
>
> my $rndsuffix = &random_string();
>
> I think this is done to avoid symbol clashes but I couldn't see
> anything that would actually clash so more understanding of that this
> is doing is needed. I wondered if we could set some kind of perl
> seeding to make it deterministic.
>
> I'll reply again if I learn anything more.

Even if it's not a real fix, avx-512 support can be more or less
disabled by default, as support for that is limited to high end server
grade chips. We haven't enabled avx/avx2 yet :)

Alex

Re: [OE-core] [PATCH] openssl: update from 3.0.8 to 3.1.0

[Richard Purdie]

On Thu, 2023-03-16 at 09:36 +0100, Alexander Kanavin wrote:
> On Thu, 16 Mar 2023 at 09:30, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > In the openssl source this is generated by a perl script,
> > rypto/modes/asm/aes-gcm-avx512.pl
> > 
> > Wearing suitable safety equipment to stop it burning, inside the perl
> > script you can see references like:
> > 
> > my $rndsuffix = &random_string();
> > 
> > I think this is done to avoid symbol clashes but I couldn't see
> > anything that would actually clash so more understanding of that this
> > is doing is needed. I wondered if we could set some kind of perl
> > seeding to make it deterministic.
> > 
> > I'll reply again if I learn anything more.
> 
> Even if it's not a real fix, avx-512 support can be more or less
> disabled by default, as support for that is limited to high end server
> grade chips. We haven't enabled avx/avx2 yet :)

I think the fix might be straight forward and just to add something
like:

# Ensure the local labels are reproduicble
srand(10000);

I'll put that into testing, see how it looks.

Cheers,

Richard