QA Test Report for Yocto Project Full Cycle Release 2.4 M1 rc1

QA Test Report for Yocto Project Full Cycle Release 2.4 M1 rc1

[Cruz, Libertad]

[-- Attachment #1: Type: text/plain, Size: 3280 bytes --]



Hello All,



Here is the report for the 2.4 M1 rc1 full test cycle release.



Full Report : https://wiki.yoctoproject.org/wiki/WW25_-_2017-06-22_-_Full_Test_Cycle_2.4_M1_rc1



======== Summary ========



The QA cycle for release 2.4 M1 rc1 is complete, there were 3 new issues.

Most important one is 11687[3] Cannot run QEMU machines under UNFS since it is blocking 13 test suites. The second issue we are encountering is that we have yet to build a correct wic image for poky-tiny distribution.



Performance suffered a 5% regression from with 2.3_rc4 when building rootfs. Performance improvement 8% on extracting eSDK (Ubuntu) and 13% on fedora.





======== Details ========



QA Hints



Eclipse plugin is  blocked by 11687 and it also blocks 11 testruns with 4% each. Also there is trouble trying to build a correct poky-tiny image this last issue is still under analysis to determine if it is a bug. QA recommends a rc2 so that the blocked test runs can be executed.





Bugs

      * New



           - Bug 11681<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11598> - poky-tiny fails to build because of resource unavailability [1]

           - Bug 11699<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11604> - [megamanual] dnf using rpm: 5.18.4.3.1. Using RPM section changes required [2]

           - Bug 11687<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11607> - Can not run QEMU machines under UNFS [3]





      * High / M+ Not New



           - Bug 10477<https://bugzilla.yoctoproject.org/show_bug.cgi?id=10456> - yocto-bsp: tool should output a conf file to be consume by the qemu runner [4]





Full Bug Report : https://wiki.yoctoproject.org/wiki/WW25_-_2017-06-22_-_Full_Test_Cycle_2.4_M1_rc1#Bugs_Found_during_QA_Test







====== Performance ======



Performance measurements showed a regression compared with 2.3_rc4 when building rootfs of around 5% in both machines and improvement on extracting eSDK of around 8% (Ubuntu) and 13% fedora on extracting eSDK. All other metrics are in the normal ranges.



          eSDK    2:35         2:41         3.87



Ubuntu    Test    2.3 rc4    2.4 M1 rc1    %

          sato    1:12:18    1:12:11       -0.16

          rootfs  2:16       2:23          5.15

          rmwork  1:06:17    1:08:27       3.27

          kernel  5:13       5:26          4.15

          eSDK    2:44       2:30          -8.54





Fedora    Test    2.3 rc4    2.4 M1 rc1    %

          sato    1:12:42    1:13:49       1.54

          rootfs  2:29       2:38          6.04

          rmwork  1:06:24    1:08:22       2.96

          kernel  5:58       6:10          3.35

          eSDK    3:01       2:36          -13.81







Performance Charts :  https://wiki.yoctoproject.org/charts/perf_milestone_GDC/performance_test.html





======== Links =========



    1.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11681

    2.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11599

    3.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11687

    4.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=10477



Regards
Libertad G.


[-- Attachment #2: Type: text/html, Size: 16496 bytes --]