QA Test Report for Yocto Project Point Release 2.2.2. rc1

QA Test Report for Yocto Project Point Release 2.2.2. rc1

[Cruz, Libertad]

[-- Attachment #1: Type: text/plain, Size: 4853 bytes --]

Hello All,



Here is the report for the 2.2.2 rc1 full point release test cycle.



Full Report : https://wiki.yoctoproject.org/wiki/WW22_-_2017-06-05_-_Full_Point_Release_Test_Cycle_2.2.2_rc1





======== Summary ========



The QA cycle for release 2.2.2 rc1 is complete, there were 6 new issues.

Two of this are related  to testimage, one to compliance test, one to self-test, one to OE-Core and finally "11604 [Yocto 2.2.2] crosstap doesn't work" [1] this is a recurring issue that has been fixed on 10990[9]. Also the bugs 5948[10], 6016[11], 6118[12], 6122[13]  are just a few that have been marked duplicated of 10990[9].



Performance metrics showed a 16% improvement on Ubuntu machine, Fedora machine showed an improvement of 9%.





======== Details ========



QA Hints



QA test cycle seems to have been executed smoothly no apparent blockers or high priority bugs.  It would be worthwhile that before deciding if this build can be release, the following bugs have been reviewed and assigned priority 11611[4], 11607[3], 11604[1], 11606[6]. We are concern about the recurring issue on Crosstap not working, not sure if there can be a strategy so that this issue is fixed on all environments. Even though it seems that 11604[1] is not a blocker it does require reporting, retesting and development analysis time whenever its present.





Bugs

      * New



           - Bug 11598<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11598> - [morty]testimage quemux86-64 lsb genericx86-64 getty[1309]: tcgetattr: Input/output error^M[2]

           - Bug 11604<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11604> - [Yocto 2.2.2] crosstap doesn't work[1]

           - Bug 11607<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11607> - [morty] Testimage lsb image trow multiple smart errors[3]

           - Bug 11611<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11611> - [morty] Compliance test lsb-setup-4.1.0-1.noarch.rpm download fail[4]

           - Bug 11565<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11565> - morty self-test errors[5]

           - Bug 11606<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11606> - [morty] OE-Core test_recipetool_create (oeqa.selftest.recipetool.RecipetoolTests)[6]





      * High / M+ Not New



           - Bug 10456<https://bugzilla.yoctoproject.org/show_bug.cgi?id=10456> - X can not launch on qemumips64[7]

           - Bug 11064<https://bugzilla.yoctoproject.org/show_bug.cgi?id=11064> - [Yocto 2.2.1] kernel version mismatched when using yocto-bsp to create custom qemu bsp[8]



Full Bug Report : https://wiki.yoctoproject.org/wiki/WW22_-_2017-06-05_-_Full_Point_Release_Test_Cycle_2.2.2_rc1#Bugs_Found_during_QA_Test





====== Performance ======



All performance metrics showed improvements on Ubuntu machine, exceling 16% on rootfs creation, Fedora machine showed an improvement of 9% on extracting eSDK the other metrics where on the same level compared to 2.2.1 release.





Ubuntu      Test      2.2.1_rc1     2.2.2_rc1     %

            sato      1:02:41       1:00:42       -3.16

            rootfs    2:43          2:16          -16.56

            rmwork    1:01:13       0:59:29       -2.83

            kernel    5:05          5:01          -1.31

            eSDK      3:28          3:22          -2.88



Fedora      Test      2.2.1_rc1     2.2.2_rc1     %

            sato      1:00:26       1:00:43       0.47

            rootfs    2:18          2:16          -1.45

            rmwork    0:59:17       0:59:29       0.34

            kernel    5:22          5:01          -6.52

            eSDK      3:42          3:22          -9.01







Performance Charts :  https://wiki.yoctoproject.org/charts/perf_milestone_GDC/performance_test.html





======== Links =========



    1.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11604

    2.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11598

    3.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11607

    4.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11611

    5.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11565

    6.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11606

    7.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=10456

    8.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=11064

    9.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=10990

    10.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=5948

    11.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=6016

    12.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=6118

    13.    https://bugzilla.yoctoproject.org/show_bug.cgi?id=6122



Regards
Libertad G.


[-- Attachment #2: Type: text/html, Size: 22903 bytes --]