QA cycle report for 2.2.4 RC1

QA cycle report for 2.2.4 RC1

[Yeoh, Ee Peng]

Hello All,
 
This is the full report for 2.2.4.rc1:  
https://wiki.yoctoproject.org/wiki/WW28_-_2018-07-13_-_Full_Test_Cycle_2.2.4_rc1
 
======= Summary ========
 
All planned tests were executed.  There were zero high milestone defect.  Team had found 3 new defects [1] [2] [3] where runtime test_parselogs testcase failed with error message inside dmesg log.  While executing the runtime smart package manager tests on target platform to install psplash, team discovered that the target platform screen was splash with Yocto logo and the Yocto logo remain after the tests completed, need further investigation to verify if this was caused by post installation script from psplash.  For performance test, regression on 2.2.3.rc2 faced with error where SRCREV defined for distcc was no longer available.  For pTest, both glib-2.0 and util-linux passrate decreased as compared to 2.2.3.rc2 as some new tests available in 2.2.4.rc1 were failing.   
 
======= QA-Hints========
 
Found 3 non critical defects.  
 
======= Bugs ========
 
New Bugs
[1] Bug 12854 - [2.2.4.rc1][BSP Auto] [genericx86-64 on MMAX64bit] [test_parselogs] blk_update_request: I/O error, dev loop0
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12854

[2] Bug 12853 - [2.2.4.rc1][BSP Auto] [genericx86-64 on NUC6][test_parselogs] drm:intel_dp_link_training_clock_recovery] *ERROR* too many voltage retries, give up
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12853

[3] Bug 12852 - [2.2.4.rc1][BSP Auto] [genericx86-64-WIC on NUC6][test_parselogs] snd_hda_intel 0000:00:1f.3: failed to add i915 component master  
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12852

Regards
Ee Peng

Re: QA cycle report for 2.2.4 RC1

[richard.purdie]

On Thu, 2018-07-19 at 09:55 +0000, Yeoh, Ee Peng wrote:
> This is the full report for 2.2.4.rc1:  
> https://wiki.yoctoproject.org/wiki/WW28_-_2018-07-13_-_Full_Test_Cycl
> e_2.2.4_rc1
>  
> ======= Summary ========
>  
> All planned tests were executed.  There were zero high milestone
> defect.  Team had found 3 new defects [1] [2] [3] where runtime
> test_parselogs testcase failed with error message inside dmesg
> log.  While executing the runtime smart package manager tests on
> target platform to install psplash, team discovered that the target
> platform screen was splash with Yocto logo and the Yocto logo remain
> after the tests completed, need further investigation to verify if
> this was caused by post installation script from psplash.  For
> performance test, regression on 2.2.3.rc2 faced with error where
> SRCREV defined for distcc was no longer available.  For pTest, both
> glib-2.0 and util-linux passrate decreased as compared to 2.2.3.rc2
> as some new tests available in 2.2.4.rc1 were failing.   
>  
> ======= QA-Hints========
>  
> Found 3 non critical defects.  
>  
> ======= Bugs ========
>  
> New Bugs
> [1] Bug 12854 - [2.2.4.rc1][BSP Auto] [genericx86-64 on MMAX64bit]
> [test_parselogs] blk_update_request: I/O error, dev loop0
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12854
> 
> [2] Bug 12853 - [2.2.4.rc1][BSP Auto] [genericx86-64 on
> NUC6][test_parselogs] drm:intel_dp_link_training_clock_recovery]
> *ERROR* too many voltage retries, give up
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12853
> 
> [3] Bug 12852 - [2.2.4.rc1][BSP Auto] [genericx86-64-WIC on
> NUC6][test_parselogs] snd_hda_intel 0000:00:1f.3: failed to add i915
> component master  
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=12852

Thanks. One of these was resolved by the triage team as WONTFIX since
whilst there is a fix in master, backporting it has potential side
effects and in their view would be too risky for a stable release.

The other two issues were identified as being due to changes in QA team
hardware used for the testing and therefore are not regressions, just
error messages that appear on the newer testing hardware.

As such, I believe we should release 2.2.4.

Cheers,

Richard