From: "Sundeep KOKKONDA" <sundeep.kokkonda@gmail.com>
To: Randy MacLeod <randy.macleod@windriver.com>,
openembedded-core@lists.openembedded.org,
"steve@sakoman.com" <steve@sakoman.com>
Subject: Re: [OE-core] [kirkstone 00/26] Patch review
Date: Mon, 20 Jun 2022 08:39:25 +0530 [thread overview]
Message-ID: <4b1d26ac-6fe2-50cf-4cbd-23fcee49b0b8@gmail.com> (raw)
In-Reply-To: <e449115f-00a6-e11a-cd8d-f502db8a0b0d@windriver.com>
[-- Attachment #1.1: Type: text/plain, Size: 2860 bytes --]
Hello Randy,
On 16-06-2022 19:49, Randy MacLeod wrote:
> On 2022-06-15 22:19, Sundeep KOKKONDA wrote:
>> Hello,
>>
>> GCC regression test performed on /Kirkstone/ with gcc-11.3 &
>> gcc-11.2. The results are *NOT* identical. Below is test summary.
>>
>> *gcc-11.3:*
>> / === gcc Summary ===/
>> /# of expected passes 126552/
>> /# of unexpected failures 24295/
>> /# of unexpected successes 77/
>> /# of expected failures 782/
>> /# of unresolved testcases 10/
>> /# of unsupported tests 2760/
>> //home/bft1/skokkonda/kirkstone/poky/build/sdk/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc
>> version 11.3.0 (GCC)/
>>
>> *gcc-11.2:*
>> / === gcc Summary ===/
>> /# of expected passes 125933/
>> /# of unexpected failures 24215/
>> /# of unexpected successes 76/
>> /# of expected failures 773/
>> /# of unresolved testcases 10/
>> /# of unsupported tests 2779/
>> //home/bft1/skokkonda/kirkstone-gcc-11.2/poky/build/sdk/sysroots/x86_64-pokysdk-linux/usr/bin/x86_64-poky-linux/x86_64-poky-linux-gcc
>> version 11.2.0 (GCC)
>>
>>
>> /
>
> /11.3: # of expected passes 126552/
> //11.2: # of expected passes 125933//
>
> So >= 619 additional tests were added and they mostly pass (as you'd
> expect).
> That's good news.
>
>
> /11.3: # of unexpected failures 24295/
> /11.2: # of unexpected failures 24215/
> /
> /
> /So 80 additional unexpected failures - can you summarize what these
> failures are?
> /
There are 83 tests failed with GCC-11.3 and out of them most of the
failed cases (75 tests) are /execution /tests and the remaining are
/test for excess errors/ (Tests with excess messages output to stderr).
The Failed tests list is attached.
> /Were any of them fixed after the 11.3 tag?
> /
Does it mean, do you want me to test on 11.4 and compare?
> /
> Are they related to one part of the toolchain or spread across various
> features?
> Are they reproducible when buiding outside of bitbake?/
The test failures are from gcc.dg/torture, gcc.c-torture & gcc.target
testsuites.
> /
> /
> /The other differences are less significant.
> If you post similar data in future emails, please do the math for people!
> /
> /
> /
> //
>> gcc-11.3 & gcc-11.2 test summaries are attached.
>
> I didn't look at these yet but thanks for providing them.
>
> Since we're not seeing any runtime regression in our Yocto
> test suite, I suspect that we shouldn't panic but should instead
> just work to understand, fix, upstream and monitor for future regressions.
>
> ../Randy
>
>>
>>
>> --
>> Thanks,
>> Sundeep K.
>>
>>
>>
>
> --
> # Randy MacLeod
> # Wind River Linux
[-- Attachment #1.2: Type: text/html, Size: 6852 bytes --]
[-- Attachment #2: FailedTests_GCC-11.3 --]
[-- Type: text/plain, Size: 5983 bytes --]
FAIL: gcc.c-torture/execute/bitfld-10.c -O0 execution test
FAIL: gcc.c-torture/execute/bitfld-10.c -O1 execution test
FAIL: gcc.c-torture/execute/bitfld-10.c -O2 execution test
FAIL: gcc.c-torture/execute/bitfld-10.c -O3 -g execution test
FAIL: gcc.c-torture/execute/bitfld-10.c -Os execution test
FAIL: gcc.c-torture/execute/bitfld-10.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.c-torture/execute/bitfld-10.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.c-torture/execute/pr103052.c -O0 execution test
FAIL: gcc.c-torture/execute/pr103052.c -O1 execution test
FAIL: gcc.c-torture/execute/pr103052.c -O2 execution test
FAIL: gcc.c-torture/execute/pr103052.c -O3 -g execution test
FAIL: gcc.c-torture/execute/pr103052.c -Os execution test
FAIL: gcc.c-torture/execute/pr103052.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.c-torture/execute/pr103052.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.c-torture/execute/pr103255.c -O0 execution test
FAIL: gcc.c-torture/execute/pr103255.c -O1 execution test
FAIL: gcc.c-torture/execute/pr103255.c -O2 execution test
FAIL: gcc.c-torture/execute/pr103255.c -O3 -fomit-frame-pointer -funroll-loops -fpeel-loops -ftracer -finline-functions execution test
FAIL: gcc.c-torture/execute/pr103255.c -O3 -g execution test
FAIL: gcc.c-torture/execute/pr103255.c -Os execution test
FAIL: gcc.c-torture/execute/pr103255.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.c-torture/execute/pr103255.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.c-torture/execute/pr104814.c -O0 execution test
FAIL: gcc.c-torture/execute/pr104814.c -O1 execution test
FAIL: gcc.c-torture/execute/pr104814.c -O2 execution test
FAIL: gcc.c-torture/execute/pr104814.c -O3 -fomit-frame-pointer -funroll-loops -fpeel-loops -ftracer -finline-functions execution test
FAIL: gcc.c-torture/execute/pr104814.c -O3 -g execution test
FAIL: gcc.c-torture/execute/pr104814.c -Os execution test
FAIL: gcc.c-torture/execute/pr104814.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.c-torture/execute/pr104814.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.dg/analyzer/data-model-1.c (test for warnings, line 254)
FAIL: gcc.dg/analyzer/strndup-1.c (test for warnings, line 11)
FAIL: c-c++-common/cpp/pr104147.c -Wc++-compat execution test
FAIL: gcc.dg/pr102224.c execution test
FAIL: gcc.dg/pr102798.c execution test
FAIL: gcc.dg/pr103860.c execution test
FAIL: gcc.dg/sso-16.c execution test
FAIL: gcc.dg/ipa/pr103083-1.c execution test
FAIL: gcc.dg/lto/pr101868 c_lto_pr101868_0.o-c_lto_pr101868_3.o execute -O2 -fno-strict-aliasing -flto
FAIL: gcc.dg/lto/pr101949 c_lto_pr101949_0.o-c_lto_pr101949_1.o execute -O2 -fipa-pta -flto -flto-partition=1to1
FAIL: gcc.dg/torture/pr102139.c -O0 execution test
FAIL: gcc.dg/torture/pr102139.c -O1 execution test
FAIL: gcc.dg/torture/pr102139.c -O2 execution test
FAIL: gcc.dg/torture/pr102139.c -O3 -g execution test
FAIL: gcc.dg/torture/pr102139.c -Os execution test
FAIL: gcc.dg/torture/pr102139.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.dg/torture/pr102139.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.dg/torture/pr103181.c -O0 execution test
FAIL: gcc.dg/torture/pr103181.c -O1 execution test
FAIL: gcc.dg/torture/pr103181.c -O2 execution test
FAIL: gcc.dg/torture/pr103181.c -O3 -g execution test
FAIL: gcc.dg/torture/pr103181.c -Os execution test
FAIL: gcc.dg/torture/pr103181.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.dg/torture/pr103181.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.dg/torture/pr103237.c -O0 execution test
FAIL: gcc.dg/torture/pr103237.c -O1 execution test
FAIL: gcc.dg/torture/pr103237.c -O2 execution test
FAIL: gcc.dg/torture/pr103237.c -O3 -fomit-frame-pointer -funroll-loops -fpeel-loops -ftracer -finline-functions execution test
FAIL: gcc.dg/torture/pr103237.c -O3 -g execution test
FAIL: gcc.dg/torture/pr103237.c -Os execution test
FAIL: gcc.dg/torture/pr103237.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.dg/torture/pr103237.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.dg/torture/pr105198.c -O0 execution test
FAIL: gcc.dg/torture/pr105198.c -O1 execution test
FAIL: gcc.dg/torture/pr105198.c -O2 execution test
FAIL: gcc.dg/torture/pr105198.c -O3 -fomit-frame-pointer -funroll-loops -fpeel-loops -ftracer -finline-functions execution test
FAIL: gcc.dg/torture/pr105198.c -O3 -g execution test
FAIL: gcc.dg/torture/pr105198.c -Os execution test
FAIL: gcc.dg/torture/pr105198.c -O2 -flto -fno-use-linker-plugin -flto-partition=none execution test
FAIL: gcc.dg/torture/pr105198.c -O2 -flto -fuse-linker-plugin -fno-fat-lto-objects execution test
FAIL: gcc.dg/vect/bb-slp-pr101615-2.c execution test
FAIL: gcc.dg/vect/bb-slp-pr101615-2.c -flto -ffat-lto-objects execution test
FAIL: gcc.target/i386/avx-pr102224.c execution test
FAIL: gcc.target/i386/avx512f-pr101471.c (test for excess errors)
FAIL: gcc.target/i386/avx512f-pr101471.c execution test
FAIL: gcc.target/i386/pr102498.c execution test
FAIL: gcc.target/i386/pr102627.c execution test
FAIL: gcc.target/i386/pr104971.c execution test
FAIL: gcc.target/i386/pr105000-1.c (test for excess errors)
FAIL: gcc.target/i386/pr105000-2.c (test for excess errors)
FAIL: gcc.target/i386/pr105000-3.c (test for excess errors)
FAIL: gcc.target/i386/pr99754-1.c execution test
FAIL: gcc.target/i386/pr99754-2.c execution test
next prev parent reply other threads:[~2022-06-20 3:09 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-02 16:51 [OE-core][kirkstone 00/26] Patch review Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 01/26] vim: Upgrade 8.2.4912 -> 8.2.5034 to fix 9 CVEs Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 02/26] tiff: Add jbig PACKAGECONFIG and clarify CVE-2022-1210 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 03/26] libxslt: Mark CVE-2022-29824 as not applying Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 04/26] cve-check.bbclass: Added do_populate_sdk[recrdeptask] Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 05/26] cve-check: Add helper for symlink handling Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 06/26] cve-check: Only include installed packages for rootfs manifest Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 07/26] cve-extra-exclusions: Add kernel CVEs Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 08/26] cve-check: Allow warnings to be disabled Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 09/26] linux-yocto/5.15: update to v5.15.37 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 10/26] linux-yocto/5.10: update to v5.10.113 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 11/26] linux-yocto/5.15: update to v5.15.38 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 12/26] linux-yocto/5.10: update to v5.10.114 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 13/26] libpcre2: upgrade 10.39 -> 10.40 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 14/26] ncurses: update to patchlevel 20220423 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 15/26] mesa.inc: package 00-radv-defaults.conf Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 16/26] staging.bbclass: process direct dependencies in deterministic order Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 17/26] libseccomp: Add missing files for ptests Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 18/26] insane.bbclass: make sure to close .patch files Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 19/26] pciutils: avoid lspci conflict with busybox Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 20/26] ovmf: Fix native build with gcc-12 Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 21/26] rust-common: Fix sstate signatures between arm hf and non-hf Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 22/26] rust-common: Ensure sstate signatures have correct dependencues for do_rust_gen_targets Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 23/26] rust-common: Fix for target definitions returning 'NoneType' for arm Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 24/26] rust-common: Drop LLVM_TARGET and simplify Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 25/26] rust-common: Fix native signature dependency issues Steve Sakoman
2022-06-02 16:51 ` [OE-core][kirkstone 26/26] gcc: Upgrade to 11.3 release Steve Sakoman
2022-06-10 8:39 ` [kirkstone 00/26] Patch review Sundeep KOKKONDA
2022-06-10 9:12 ` [OE-core] " Martin Jansa
2022-06-10 9:13 ` Martin Jansa
2022-06-10 14:19 ` Steve Sakoman
2022-06-16 2:19 ` Sundeep KOKKONDA
2022-06-16 14:19 ` [OE-core] " Randy MacLeod
2022-06-20 3:09 ` Sundeep KOKKONDA [this message]
2022-06-27 12:12 ` Randy MacLeod
2022-08-19 2:42 [OE-core][kirkstone " Steve Sakoman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4b1d26ac-6fe2-50cf-4cbd-23fcee49b0b8@gmail.com \
--to=sundeep.kokkonda@gmail.com \
--cc=openembedded-core@lists.openembedded.org \
--cc=randy.macleod@windriver.com \
--cc=steve@sakoman.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.