* QA cycle report for 2.4 M2 rc2
@ 2017-08-03 0:55 Cruz, Libertad
2017-08-03 10:19 ` Richard Purdie
0 siblings, 1 reply; 2+ messages in thread
From: Cruz, Libertad @ 2017-08-03 0:55 UTC (permalink / raw)
To: yocto, Purdie, Richard, Eggleton, Paul, Jolley, Stephen K, Zhang,
Jessica
Cc: Cruz Alcaraz, Juan M
[-- Attachment #1: Type: text/plain, Size: 7206 bytes --]
Hello All,
Here is the report for the 2.4 M2 rc2 full point release test cycle.
Full Report : https://wiki.yoctoproject.org/wiki/WW31_-_2017-07-31_-_Full_Test_Cycle_2.4_M2_rc2
======= Summary ========
The QA cycle for release 2.4 M2 rc2 is complete.
Performance
Measurements were mostly the same levels as of 2.4 M1 rc1, there was a little improvement of 5% on rootfs build time for Fedora machine.
Ubuntu Test 2.4 M1 rc1 2.4 M2 rc2 %
sato 1:12:11 1:13:58 2.47
rootfs 2:24 2:31 4.86
rmwork 1:08:27 1:07:22 -1.58
kernel 5:26 5:39 3.99
eSDK 2:30 2:30 0.00
Fedora Test 2.4 M1 rc1 2.4 M2 rc2 %
sato 1:13:49 1:13:52 0.07
rootfs 2:38 2:29 -5.70
rmwork 1:08:22 1:08:42 0.49
kernel 6:10 6:04 -1.62
eSDK 2:36 2:34 -1.28
ptest
There was an improvement of the pass rate for gdk-pixbuf of 5% , an improvement of 1.30% on glib-2.0, an improvement of 2.9% on openssh, an improvement of 15.2% on util-linux. There was a regression on the pass rate for libxml2 of 4.20% .
Compliance
For posix test there is no mayor change in the number of failures on 2.4m2 rc2 and 2.4 m1 rc1.
Failures 2.4 M2 RC2 2.4 M1 RC1
Generic86-64 46 45
Beaglebone 45 45
Edgerouter 46 46
Mpc8315e-rdb 45 45
Results are found here: https://wiki.yoctoproject.org/wiki/Posix_result
Detail analysis found here: https://wiki.yoctoproject.org/wiki/POSIX-results
For LTP testing we had 2 additional failures in comparison to 2.4 M1 rc1
Genericx86-64 mpc8315e-rdb
2.4 M2 rc2 Total 1384 1348
Failures 24 30
Genericx86-64 mpc8315e-rdb
2.4 M1 rc1 Total 1384 1348
Failures 22 30
The LPT results are found: https://wiki.yoctoproject.org/wiki/LTP_result .
======= QA-Hints========
QA does not approve that the workarounds provided for 2.4 m2 rc2 were on different commits. Testing was not limited to 2.4 m2 rc2 commit "8e15e9b6e478f6368034519b2a8fd3c7ea71d23b" it was performed by following each commit on the provided workarounds described below.
--11850[1] was opened to track the fact that Eclipse plugins were not publish under 2.4 m2 rc2 so, the plugins were taken from previous build 2.4 m2 rc1. These files need to be copied in to 2.4m2 rc2 if and before release is approved.
--11834[10] describes that if you add a bad layer to bblayers bitbake server will hang in connecting and although is resolved & verified this bug was linked to this release candidate because the commit is not integrated to 2.4 m2 rc2. By linking this issue in the test report users can then find the correct commit that will fix it.
--11857[9] was opened to track that there are some problems creating wic images on CentOS is also resolved & verified, but it is still linked to 2.4 m2 rc2 because the commit is not integrated to this release candidate. Users can then find the correct commit that will fix it.
--11855[4] this bug reports the fact that libgcc-initial can't build on fedora i686, but again when trying with latest master commit it passes. Users can then know which commit to work on.
--11889[11] Toaster does not build on 2.4 m2 rc2 but when changing to latest master commit it works fine.
This testing cycle report does not reflects the results exclusively of 2.4 m2 rc2 but of all the other commits that were applied so that eclipse, toaster, bitbake and ADT on Fedora i686 would not be blocked. We recommend a 2.4 m2 rc3 where all the commits are integrated.
On the wiki a new section was created for Compliance results. For LTP testing there is currently an issue with the hardware infrastructure of Wind River hence beaglebone and edgerouter have not been tested.
======= Bugs ========
New Bugs
-11850[1] [Autobuilder] Eclipse-plugin files are not found in autobuilder
-11873[2] These TCs need to be added to testopia Distrodata.test_checkpkg and Wic.test_mkfs_extraopts
-11879[3] Testimage: multiple "uvesafb" errors present in log in minnowmax on genericx86/genericx86-64
-11855[4] Failed to build libgcc-initial on Fedora26 i686
-11859[5] [Test Case 1883] TestImage.test_testimage_dnf Centos7
-11868[6] wic.Wic.test_mkfs_extraopts - Testcase -1 opensuse422
-11869[7] FAILED runqemu.RunqemuTests.test_boot_deploy - Testcase 2007: Ubuntu1604
-11871[8] distrodata.Distrodata.test_checkpkg - Testcase -1: ubuntu1604
-11857[9] FAIL [23.106s]: test_mkfs_extraopts (wic.Wic) centos7
-11834[10] if you have a bad layer in bblayers, bitbake server connection hangs
Full Bug Report : https://wiki.yoctoproject.org/wiki/WW31_-_2017-07-31_-_Full_Test_Cycle_2.4_M2_rc2#Bugs_Found_during_QA_Test
======== Links =========
1. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11850 [1]
2. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11873 <https://bugzilla.yoctoproject.org/show_bug.cgi?id=10477%20> [2]
3. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11879 <https://bugzilla.yoctoproject.org/show_bug.cgi?id=10832%20> [3]
4. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11855 <https://bugzilla.yoctoproject.org/show_bug.cgi?id=11811%20> [4]
5. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11859 <https://bugzilla.yoctoproject.org/show_bug.cgi?id=11812%20> [5]
6. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11868<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11813> [6]
7. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11869 [7]
8. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11871 [8]
9. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11857 [9]
10. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11834 [10]
11. https://bugzilla.yoctoproject.org/show_bug.cgi?id=11889 [11]
Regards
Libertad G.
[-- Attachment #2: Type: text/html, Size: 32993 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: QA cycle report for 2.4 M2 rc2
2017-08-03 0:55 QA cycle report for 2.4 M2 rc2 Cruz, Libertad
@ 2017-08-03 10:19 ` Richard Purdie
0 siblings, 0 replies; 2+ messages in thread
From: Richard Purdie @ 2017-08-03 10:19 UTC (permalink / raw)
To: Cruz, Libertad, yocto, Eggleton, Paul, Jolley, Stephen K, Zhang, Jessica
Cc: Cruz Alcaraz, Juan M
On Wed, 2017-08-02 at 17:55 -0700, Cruz, Libertad wrote:
> QA does not approve that the workarounds provided for 2.4 m2 rc2
> were on different commits. Testing was not limited to 2.4 m2 rc2
> commit “8e15e9b6e478f6368034519b2a8fd3c7ea71d23b” it was performed by
> following each commit on the provided workarounds described below.
Thanks Libertad.
The different commits wasn't intentional, there were bugs which were
reported and fixed after rc2 was built. It made sense to resolve those
bugs as fixed in master but obviously they were still present in M2.
Given the high volume of issues and other things that went on with rc2
I've decided to build an rc3. We'll have to take current master as
there are some many fixes it only makes sense to use that rather than
create any special branch. I've set that build away.
Cheers,
Richard
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2017-08-03 10:19 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-08-03 0:55 QA cycle report for 2.4 M2 rc2 Cruz, Libertad
2017-08-03 10:19 ` Richard Purdie
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.