* QA cycle report for 2.5.3 RC3
@ 2019-04-18 4:58 Jain, Sangeeta
2019-04-23 14:31 ` richard.purdie
2019-04-23 14:35 ` richard.purdie
0 siblings, 2 replies; 7+ messages in thread
From: Jain, Sangeeta @ 2019-04-18 4:58 UTC (permalink / raw)
To: 'yocto@yoctoproject.org',
Eggleton, Paul, 'richard.purdie@linuxfoundation.org',
'Michael Halstead',
Erway, Tracey M, 'sjolley.yp.pm@gmail.com',
'openembedded-core-request@lists.openembedded.org'
[-- Attachment #1.1: Type: text/plain, Size: 1794 bytes --]
QA cycle report for 2.5.3 RC3:
1. No high milestone defects.
2. Test results are available at following location:
* For results of all automated tests, refer to results at public AB [1].
* For other test results, refer to attachment [2].
* For test report for test cases run by Intel and WR team, refer attachment [3]
* For full test report, refer attachment [4]
* For ptest results, please refer to results at public AB [5]
* For ptest report, refer to attachment [6]
1. No new defects are found in this cycle.
2. Number of existing issues observed in this release is 2- toaster [7] and Build-appliance [8]
3. For ptest, regression data is not available for this release. No timeout issues.
4. Test result report on Public AB shows no failures.
======= Links ========
Link to testresults:
[1] - https://autobuilder.yocto.io/pub/releases/yocto-2.5.3.rc3/testresults/
[2] - 2.5.3_RC3_results_merged.zip
[3] - 2. 5.3_rc3_intel_wr_merged_report
[4] - 2. 5.3_rc3_full_report
[5] - https://autobuilder.yocto.io/pub/non-release/20190328-7/testresults/qemux86-64-ptest/
[6] - 2. 5.3_rc3_ptest_report
Previous Bugs observed in this release:
[7] Bug 13191 - [Test Case 1439] Build Failure after adding or removing packages with and without dependencies
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13191
[8] Bug 12991<https://bugzilla.yoctoproject.org/show_bug.cgi?id=12991> - [Bug] [2.6 M4 RC1][Build-Appliance] Bitbake build-appliance-image getting failed during building image due to webkitgtk package.
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12991
Thanks & Regards,
Sangeeta Jain
[-- Attachment #1.2: Type: text/html, Size: 17824 bytes --]
[-- Attachment #2: 2. 5.3_rc3_ptest_report.txt --]
[-- Type: text/plain, Size: 54573 bytes --]
==============================================================================================================
Test Result Status Summary (Counts/Percentages sorted by testseries, ID)
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
Test Series | ID | Passed | Failed | Skipped
--------------------------------------------------------------------------------------------------------------
ptest_results | runtime_core-image-sato-sdk-ptest_qemux86-64_20190328103248 | 15982 (91%) | 775 (4%) | 755 (4%)
--------------------------------------------------------------------------------------------------------------
==============================================================================================================
PTest Result Summary
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
Recipe | Passed | Failed | Skipped | Time(s)
--------------------------------------------------------------------------------------------------------------
acl | 221 | 0 | 0 | -
attr | 109 | 0 | 0 | -
bash | 75 | 4 | 0 | -
bluez5 | 7 | 0 | 0 | -
busybox | 509 | 14 | 36 | -
bzip2 | 6 | 0 | 0 | -
dbus-test | 18 | 3 | 2 | -
diffstat | 439 | 0 | 0 | -
diffutils | 20 | 0 | 0 | -
e2fsprogs | 304 | 5 | 27 | -
flex | 114 | 0 | 0 | -
gawk | 342 | 0 | 0 | -
gdbm | 19 | 0 | 0 | -
gdk-pixbuf | 20 | 0 | 0 | -
glib-2 | 252 | 1 | 0 | -
gzip | 21 | 0 | 0 | -
libevent | 1 | 0 | 0 | -
libpcre | 3 | 0 | 0 | -
libxml-parser-perl | 15 | 0 | 0 | -
libxml2 | 134 | 0 | 0 | -
lttng-tools | 1087 | 0 | 0 | -
lzo | 39 | 6 | 0 | -
nettle | 158 | 0 | 2 | -
openssh | 840 | 72 | 367 | -
openssl | 54 | 0 | 0 | -
pango | 12 | 0 | 0 | -
perl | 2282 | 3 | 120 | -
python | 8331 | 45 | 69 | -
quilt | 57 | 0 | 0 | -
sed | 86 | 0 | 0 | -
slang | 97 | 0 | 0 | -
strace | 160 | 233 | 80 | -
tcl | 148 | 0 | 0 | -
util-linux | 0 | 389 | 52 | -
--------------------------------------------------------------------------------------------------------------
==============================================================================================================
Failed test cases (sorted by testseries, ID)
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
testseries | result_id : ptest_results | runtime_core-image-sato-sdk-ptest_qemux86-64_20190328103248
ptestresult.strace.unshare.gen.test
ptestresult.openssh.ust/overlap/test_overlap_140_-_Trace_match
ptestresult.strace.unix-pair-send-recv.test
ptestresult.strace.printstrn-umoven-legacy.test
ptestresult.util-linux.hexdump:_1b_char-1__hexdump/highlighting-1b_char-1_
ptestresult.openssh.ust/overlap/test_overlap_8_-_Trace_match
ptestresult.openssh.ust/overlap/test_overlap_254_-_Trace_match
ptestresult.strace.nsyscalls.test
ptestresult.strace.sigaction.gen.test
ptestresult.util-linux.blkid:_hfs__blkid/low-probe-hfs_
ptestresult.util-linux.blkid:_ntfs__blkid/low-probe-ntfs_
ptestresult.strace.timer_create.gen.test
ptestresult.util-linux.findmnt:_types__findmnt/filter-types_
ptestresult.util-linux.blkid:_sgi__blkid/lowprobe-pt-sgi_
ptestresult.busybox.hostname-d-works
ptestresult.busybox.wget-retrieves-google-index
ptestresult.strace.sendfile64.gen.test
ptestresult.python.test_checkSetMinor__email.test.test_email.TestMIMEAudio_
ptestresult.strace.ioctl_nsfs.test
ptestresult.util-linux.blkid:_udf-hdd-udfclient-0.7.7__blkid/low-probe-udf-hdd-udfclient-0.7.7_
ptestresult.util-linux.mount:_by_label__mount/label_
ptestresult.util-linux.hexdump:_2b_dec-8__hexdump/highlighting-2b_dec-8_
ptestresult.strace.umount.test
ptestresult.python.test_script_compiled__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmpglztCm/script.pyc':
ptestresult.util-linux.getopt:_longopts__getopt/options-longopts_
ptestresult.util-linux.cal:_ysjw__cal/weeknum-ysjw_
ptestresult.busybox.hostname-s-works
ptestresult.util-linux.misc:_uname26__unsupported_--uname-2.6:_echo_true_
ptestresult.strace.prctl-tsc.test
ptestresult.util-linux.cal:_1sj-month__cal/sep1752-1sj-month_
ptestresult.strace.rt_sigaction.test
ptestresult.util-linux.blkid:_bsd__blkid/lowprobe-pt-bsd_
ptestresult.util-linux.cal:_1s__cal/month-1s_
ptestresult.strace.seccomp_get_action_avail.gen.test
ptestresult.util-linux.blkid:_tuxonice__blkid/low-probe-tuxonice_
ptestresult.util-linux.blkid:_minix-LE__blkid/low-probe-minix-LE_
ptestresult.util-linux.cal:_3s-month__cal/bigyear-3s-month_
ptestresult.strace.xetpriority.gen.test
ptestresult.strace.mmap.test
ptestresult.strace.quotactl-xfs.gen.test
ptestresult.util-linux.fdisk:_align_512/512__fdisk/align-512-512_
ptestresult.util-linux.cal:_3sw__cal/month-3sw_
ptestresult.strace.seccomp-filter-v.gen.test
ptestresult.util-linux.blkid:_udf-cd-mkudfiso-20100208__blkid/low-probe-udf-cd-mkudfiso-20100208_
ptestresult.strace.count-f.test
ptestresult.util-linux.isosize:_divisor_output__isosize/print-size-divisor_output_
ptestresult.util-linux.cal:_1sjw__cal/month-1sjw_
ptestresult.util-linux.hexdump:_1b_octal-6__hexdump/highlighting-1b_octal-6_
ptestresult.util-linux.libsmartcols:_wrapnl__libsmartcols/fromfile-wrapnl_
ptestresult.strace.ioctl_dm-v.test
ptestresult.python.test_encoding__email.test.test_email.TestMIMEAudio_
ptestresult.strace.seccomp-filter.gen.test
ptestresult.strace.statx.gen.test
ptestresult.util-linux.findmnt:_options-nameval-multimkdir:_cannot_create_directory_'/usr/lib/util-linux/ptest/tests/output':_No_space_left_on_device
ptestresult.lzo.tools/tracefile-limits/test_tracefile_count_79_-_Trace_validation_-_intfield
ptestresult.util-linux.libfdisk:_gpt-err-overlap__libfdisk/mkpart-full-gpt-err-overlap_
ptestresult.busybox.unzip__subdir_only_
ptestresult.util-linux.blkid:_atari-xgm__blkid/lowprobe-pt-atari-xgm_
ptestresult.strace.sigsuspend.gen.test
ptestresult.util-linux.findmnt:_options-no__findmnt/filter-options-no_
ptestresult.util-linux.hexdump:_2b_dec-5__hexdump/highlighting-2b_dec-5_
ptestresult.util-linux.hexdump:_2b_dec-1__hexdump/highlighting-2b_dec-1_
ptestresult.util-linux.cal:_3m-month__cal/bigyear-3m-month_
ptestresult.util-linux.bitops:_swap_bytes__bitops/swapbytes_
ptestresult.openssh.ust/overlap/test_overlap_53_-_Trace_match
ptestresult.strace.readlink.gen.test
ptestresult.util-linux.cal:_1mj-month__cal/sep1752-1mj-month_
ptestresult.openssh.ust/overlap/test_overlap_154_-_Trace_match
ptestresult.busybox.cpio_lists_hardlinks
ptestresult.strace.sock_filter-v.gen.test
ptestresult.util-linux.findmnt:_kernel__findmnt/outputs-kernel_
ptestresult.util-linux.cal:_1sjw-year__cal/sep1752-1sjw-year_
ptestresult.strace.unlink.gen.test
ptestresult.util-linux.cal:_3sjw__cal/weeknum-3sjw_
ptestresult.util-linux.getopt:_weird_quoting_bash__getopt/options-weird_quoting_bash_
ptestresult.util-linux.hexdump:_1b_char-2__hexdump/highlighting-1b_char-2_
ptestresult.strace.xetpgid.gen.test
ptestresult.strace.sync.gen.test
ptestresult.openssh.ust/overlap/test_overlap_74_-_Trace_match
ptestresult.strace.sigreturn.gen.test
ptestresult.strace.pwritev.gen.test
ptestresult.util-linux.cal:_3mj-week40__cal/weekarg-3mj-week40_
ptestresult.util-linux.blkid:_mdraid__blkid/low-probe-mdraid_
ptestresult.python.test_guess_minor_type__email.test.test_email.TestMIMEAudio_
ptestresult.util-linux.libmount:_loop_overlay__libmount/loop-overlay_
ptestresult.openssh.ust/overlap/test_overlap_109_-_Trace_match
ptestresult.util-linux.mount:_by_uuid__fstab_label___mount/fstab-uuid2label_
ptestresult.python.test_fcntl_bad_file_overflow__test.test_fcntl.TestFcntl_
ptestresult.strace.trace_statfs.gen.test
ptestresult.lzo.tools/tracefile-limits/test_tracefile_count_69_-_Trace_validation_-_intfield
ptestresult.strace.rt_sigprocmask.gen.test
ptestresult.strace.setresuid.gen.test
ptestresult.util-linux.colcrt:_crash2__colcrt/regressions-crash2_
ptestresult.strace.uname.test
ptestresult.strace.caps-abbrev.test
ptestresult.util-linux.mount:_fs_lists__mount/fslists_
ptestresult.util-linux.blkid:_udf-hdd-mkudffs-1.0.0-2__blkid/low-probe-udf-hdd-mkudffs-1.0.0-2_
ptestresult.util-linux.findmnt:_root__findmnt/target-root_
ptestresult.strace.clone_ptrace.test
ptestresult.util-linux.hexdump:_4b_dec-1__hexdump/highlighting-4b_dec-1_
ptestresult.util-linux.look:_separator__look/separator_
ptestresult.util-linux.blkid:_lsi-raid__blkid/low-probe-lsi-raid_
ptestresult.openssh.ust/overlap/test_overlap_52_-_Trace_match
ptestresult.util-linux.blkid:_zfs__blkid/low-probe-zfs_
ptestresult.util-linux.dmesg:_delta__dmesg/delta_
ptestresult.util-linux.blkid:_udf-hdd-mkudffs-1.0.0-1__blkid/low-probe-udf-hdd-mkudffs-1.0.0-1_
ptestresult.util-linux.libsmartcols:_noextremes__libsmartcols/fromfile-noextremes_
ptestresult.openssh.ust/overlap/test_overlap_55_-_Trace_match
ptestresult.util-linux.blkid:_adaptec-raid__blkid/low-probe-adaptec-raid_
ptestresult.util-linux.blkid:_ddf-raid__blkid/low-probe-ddf-raid_
ptestresult.strace.oldselect.test
ptestresult.openssh.ust/overlap/test_overlap_173_-_Trace_match
ptestresult.strace.prlimit64.gen.test
ptestresult.python.test_no_optimize_flag__distutils.tests.test_bdist_rpm.BuildRpmTestCase_
ptestresult.util-linux.colcrt:_no-options__colcrt/functional-no-options_
ptestresult.util-linux.cal:_1m-month__cal/bigyear-1m-month_
ptestresult.openssh.ust/overlap/test_overlap_21_-_Trace_match
ptestresult.strace.truncate.gen.test
ptestresult.strace.xet_robust_list.gen.test
ptestresult.openssh.ust/overlap/test_overlap_231_-_Trace_match
ptestresult.strace.setresuid32.gen.test
ptestresult.busybox.sed_NUL_in_command
ptestresult.util-linux.cal:_3m-month__cal/sep1752-3m-month_
ptestresult.strace.readahead.gen.test
ptestresult.strace.printstrn-umoven-peekdata.gen.test
ptestresult.util-linux.getopt:_invocation_model_one__getopt/options-invocation_model_one_
ptestresult.util-linux.dmesg:_colors__dmesg/colors_
ptestresult.strace.umovestr2.test
ptestresult.python.test_directory_compiled__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmpPbRffL':
ptestresult.openssh.tools/save-load/test_load_3_-_Destroy_session_load-42
ptestresult.util-linux.libsmartcols:_wrap-tree__libsmartcols/fromfile-wrap-tree_
ptestresult.util-linux.cal:_1mw-year__cal/bigyear-1mw-year_
ptestresult.python.testDelItem__test.test_class.ClassTests_
ptestresult.openssh.ust/overlap/test_overlap_208_-_Trace_match
ptestresult.util-linux.libsmartcols:_strictwidth__libsmartcols/fromfile-strictwidth_
ptestresult.util-linux.getopt:_sh__getopt/options-sh_
ptestresult.glib-2.0.glib/slice.test__Child_process_killed_by_signal_9_
ptestresult.util-linux.colcrt:_crash1__colcrt/regressions-crash1_
ptestresult.strace.pselect6.gen.test
ptestresult.util-linux.mount:_by_devname__fstab___mount/fstab-devname_
ptestresult.util-linux.hexdump:_1b_char__hexdump/format-strings-1b_char_
ptestresult.util-linux.libfdisk:_mbr-err-primary__libfdisk/mkpart-full-mbr-err-primary_
ptestresult.openssh.ust/overlap/test_overlap_76_-_Trace_match
ptestresult.openssh.ust/overlap/test_overlap_118_-_Trace_match
ptestresult.util-linux.colcrt:_hang1__colcrt/regressions-hang1_
ptestresult.strace.read-write.gen.test
ptestresult.python.test_encoding__email.test.test_email_renamed.TestMIMEAudio_
ptestresult.strace.recvmsg.gen.test
ptestresult.util-linux.cal:_1m-week53-color__cal/weekarg-1m-week53-color_
ptestresult.util-linux.cal:_3m-week40-color__cal/weekarg-3m-week40-color_
ptestresult.util-linux.libsmartcols:_title__libsmartcols/title_
ptestresult.util-linux.blkid:_reiser4__blkid/low-probe-reiser4_
ptestresult.dbus-test.test/test-names
ptestresult.util-linux.cal:_sj-week40__cal/weekarg-sj-week40_
ptestresult.strace.trace_fstat.gen.test
ptestresult.util-linux.hexdump:_1b_octal-3__hexdump/highlighting-1b_octal-3_
ptestresult.util-linux.dmesg:_indentation__dmesg/indentation_
ptestresult.strace.shmxt.gen.test
ptestresult.strace.net.test
ptestresult.strace.setreuid.gen.test
ptestresult.dbus-test.test/test-dbus-daemon
ptestresult.util-linux.blkid:_sun__blkid/lowprobe-pt-sun_
ptestresult.util-linux.cal:_m3w__cal/jan1753-m3w_
ptestresult.strace.rename.gen.test
ptestresult.python.testSettingException__test.test_exceptions.ExceptionTests_
ptestresult.util-linux.getopt:_no-arguments__getopt/options-no-arguments_
ptestresult.strace.sched_xetattr.gen.test
ptestresult.util-linux.fsck:_is_mounted__fsck/ismounted_
ptestresult.openssh.ust/overlap/test_overlap_218_-_Trace_match
ptestresult.python.test_import_initless_directory_warning__test.test_import.ImportTests_
ptestresult.util-linux.cal:_1mjw__cal/month-1mjw_
ptestresult.util-linux.hexdump:_canon__hexdump/format-strings-canon_
ptestresult.util-linux.fdisk:_MBR_-_sort__fdisk/mbr-sort_
ptestresult.util-linux.cal:_1s-year__cal/sep1752-1s-year_
ptestresult.strace.recvfrom.gen.test
ptestresult.util-linux.getopt:_weird_quoting_tcsh__getopt/options-weird_quoting_tcsh_
ptestresult.python.test_module_in_package_in_zipfile__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmponkXjC/launch.py':
ptestresult.strace.net-yy-netlinkmkdir:_cannot_create_directory_'net-yy-unix.dir':_No_space_left_on_device
ptestresult.strace.uio.test
ptestresult.strace.qual_inject-signal.test
ptestresult.util-linux.libfdisk:_mbr-primary__libfdisk/mkpart-full-mbr-primary_
ptestresult.strace.sethostname.gen.test
ptestresult.util-linux.findmnt:_options__findmnt/filter-options_
ptestresult.python.test_directory__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmpf498ii':
ptestresult.util-linux.blkid:_lvm2__blkid/low-probe-lvm2_
ptestresult.openssh.ust/overlap/test_overlap_176_-_Trace_match
ptestresult.util-linux.libsmartcols:_wrapnl-tree__libsmartcols/fromfile-wrapnl-tree_
ptestresult.util-linux.cal:_ymjw__cal/year-ymjw_
ptestresult.util-linux.hexdump:_1b_octal__hexdump/format-strings-1b_mkdir:_cannot_create_directory_'/usr/lib/util-linux/ptest/tests/output':_No_space_left_on_device
ptestresult.strace.clone_parent.test
ptestresult.util-linux.getopt:_alternative_option_long__getopt/options-alternative_option_long_
ptestresult.python.test_sizeof_without_buffer__test.test_array.UnsignedByteTest_
ptestresult.util-linux.blkid:_udf-hdd-mkudffs-1.3-6__blkid/low-probe-udf-hdd-mkudffs-1.3-6_
ptestresult.util-linux.cal:_ysw__cal/year-ysw_
ptestresult.util-linux.isosize:_default_output__isosize/print-size-default_output_
ptestresult.e2fsprogs.m_hugefile:_create_a_hugefile_fs_with_a_single_huge_file:_failed
ptestresult.bash.run-trap
ptestresult.openssh.ust/overlap/test_overlap_209_-_Trace_match
ptestresult.util-linux.colrm:_basic_check__colrm/rm2-2_
ptestresult.util-linux.hexdump:_2b_dec-7__hexdump/highlighting-2b_dec-7_
ptestresult.util-linux.blkid:_udf__blkid/low-probe-udf_
ptestresult.util-linux.hexdump:_canon-1__hexdump/highlighting-canon-1_
ptestresult.strace.qual_syscall.test
ptestresult.util-linux.mount:_move__mount/move_
ptestresult.util-linux.ipcs:_mk-rm-msg__ipcs/mk-rm-msg_
ptestresult.strace.vhangup.gen.test
ptestresult.util-linux.blkid:_iso-rr-joliet__blkid/low-probe-iso-rr-joliet_
ptestresult.util-linux.getopt:_test_for_enhanced_getopt__getopt/options-test_for_enhanced_getopt_
ptestresult.util-linux.findmnt:_force-tree__findmnt/outputs-force-tree_
ptestresult.util-linux.colcrt:_half-lines__colcrt/functional-half-lines_
ptestresult.strace.setfsgid.gen.test
ptestresult.strace.tee.gen.test
ptestresult.util-linux.cal:_1mw__cal/month-1mw_
ptestresult.util-linux.libsmartcols:_trunc__libsmartcols/fromfile-trunc_
ptestresult.util-linux.misc:_whereis__misc/whereis_
ptestresult.util-linux.cal:_3sw-month__cal/sep1752-3sw-month_
ptestresult.util-linux.cal:_1sw-year__cal/sep1752-1sw-year_
ptestresult.util-linux.cal:_1mj-month__cal/bigyear-1mj-month_
ptestresult.strace.rt_sigpending.gen.test
ptestresult.openssh.ust/overlap/test_overlap_54_-_Trace_match
ptestresult.openssh.ust/overlap/test_overlap_221_-_Trace_match
ptestresult.strace.quotactl-xfs-v.gen.test
ptestresult.util-linux.getopt:_quiet_output_option_short__getopt/options-quiet_output_option_short_
ptestresult.strace.rt_sigreturn.gen.test
ptestresult.openssh.tools/crash/test_crash_76_-_Extraction_of_crashed_buffers_to_path_/tmp/tmp.SWeObPrQge/extract
ptestresult.util-linux.fdisk:_GPT__fdisk/gpt_
ptestresult.python.test_package_recursion__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmp5II5qJ/launch.py':
ptestresult.strace.unix-pair-sendto-recvfrom.test
ptestresult.strace.filter-unavailable.test
ptestresult.util-linux.blkid:_f2fs__blkid/low-probe-f2fs_
ptestresult.util-linux.hexdump:_2b_dec-3__hexdump/highlighting-2b_dec-3_
ptestresult.openssh.tools/crash/test_crash_63_-_Extraction_of_crashed_buffers_to_path
ptestresult.util-linux.blkid:_minix-BE__blkid/low-probe-minix-BE_
ptestresult.util-linux.getopt:_non-option__getopt/options-non-option_
ptestresult.python.test_directory_error__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmpT9OqZ5':
ptestresult.util-linux.hexdump:_canon-2__hexdump/highlighting-canon-2_
ptestresult.openssh.ust/overlap/test_overlap_119_-_Trace_match
ptestresult.util-linux.findmnt:_submounts__findmnt/outputs-submounts_
ptestresult.strace.sigaltstack.test
ptestresult.util-linux.blkid:_nilfs2__blkid/low-probe-nilfs2_
ptestresult.util-linux.libfdisk:_mbr__libfdisk/mkpart-mbr_
ptestresult.strace.prctl-dumpable.test
ptestresult.util-linux.ipcs:_mk-rm-sem__ipcs/mk-rm-sem_
ptestresult.strace.sched_xetparam.gen.test
ptestresult.util-linux.blkid:_via-raid__blkid/low-probe-via-raid_
ptestresult.strace.unlinkat.gen.test
ptestresult.strace.ipc_msgbuf.test
ptestresult.python.test_sizeof_without_buffer__test.test_array.UnsignedShortTest_
ptestresult.util-linux.col:_multibyte_input_mkdir:_cannot_create_directory_'/usr/lib/util-linux/ptest/tests/output':_No_space_left_on_device
ptestresult.strace.swap.gen.test
ptestresult.util-linux.cal:_3mw-month__cal/bigyear-3mw-month_
ptestresult.util-linux.findmnt:_options-name__findmnt/filter-options-name_
ptestresult.busybox.wget-supports--P
ptestresult.python.test_module_in_subpackage_in_zipfile__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmpo0tvhm/launch.py':
ptestresult.strace.sun_path.test
ptestresult.strace.execve-v.test
ptestresult.util-linux.md5:_md5__md5/md5_
ptestresult.strace.printpath-umovestr.gen.test
ptestresult.util-linux.fdisk:_sunlabel_tests__fdisk/sunlabel_
ptestresult.util-linux.lsmem:_x86_64-16g__lsmem/lsmem-x86_64-16g_
ptestresult.util-linux.hexdump:_2b_dec-4__hexdump/highlighting-2b_dec-4_
ptestresult.util-linux.cal:_3mj-week40-color__cal/weekarg-3mj-week40-color_
ptestresult.openssh.ust/overlap/test_overlap_174_-_Trace_match
ptestresult.openssh.ust/overlap/test_overlap_152_-_Trace_match
ptestresult.openssh.ust/overlap/test_overlap_63_-_Trace_match
ptestresult.util-linux.blkid:_jfs__blkid/low-probe-jfs_
ptestresult.busybox.unzip__bad_archive_
ptestresult.python.test_checkSetMinor__email.test.test_email_renamed.TestMIMEAudio_
ptestresult.util-linux.mount:_call_mount.<type>__mount/special_
ptestresult.util-linux.mount:_by_devname__fstab_label___mount/fstab-devname2label_
ptestresult.util-linux.column:_table__ul_cd:_/usr/lib/util-linux/ptest/tests/output/column:_no_such_directory_
ptestresult.strace.renameat.gen.test
ptestresult.util-linux.blkid:_udf-bdr-2.60-nero__blkid/low-probe-udf-bdr-2.60-nero_
ptestresult.util-linux.blkid:_ubifs__blkid/low-probe-ubifs_
ptestresult.strace.wait4.gen.test
ptestresult.util-linux.findmnt:_messy-mountinfo__findmnt/outputs-messy-mountinfo_
ptestresult.util-linux.blkid:_udf-hdd-udfclient-0.7.5__blkid/low-probe-udf-hdd-udfclient-0.7.5_
ptestresult.util-linux.dmesg:_levels__dmesg/console-levels_
ptestresult.util-linux.colcrt:_no-underlining__colcrt/functional-no-underlining_
ptestresult.util-linux.cal:_ysw__cal/weeknum-ysw_
ptestresult.busybox.sed_nonexistent_label
ptestresult.util-linux.colcrt:_short-options__colcrt/functional-short-options_
ptestresult.util-linux.cal:_3mjw__cal/month-3mjw_
ptestresult.strace.rt_sigtimedwait.gen.test
ptestresult.util-linux.getopt:_alternative_option_short__getopt/options-alternative_option_short_
ptestresult.util-linux.cal:_3sjw-month__cal/sep1752mkdir:_cannot_create_directory_'/usr/lib/util-linux/ptest/tests/output':_No_space_left_on_device
ptestresult.util-linux.blkid:_bfs__blkid/low-probe-bfs_
ptestresult.strace.ioctl.test
ptestresult.strace.xettimeofday.gen.test
ptestresult.strace.setfsgid32.gen.test
ptestresult.strace.statfs.gen.test
ptestresult.strace.setresgid.gen.test
ptestresult.openssh.ust/overlap/test_overlap_75_-_Trace_match
ptestresult.strace.attach-p-cmd.test
ptestresult.python.test_sizeof__test.test_deque.TestBasic_
ptestresult.util-linux.blkid:_luks1__blkid/low-probe-luks1_
ptestresult.strace.trace_question.gen.test
ptestresult.util-linux.ipcs:_headers__ipcs/headers_
ptestresult.strace.set_mempolicy.gen.test
ptestresult.strace.utimensat.gen.test
ptestresult.util-linux.cal:_1m-year__cal/sep1752-1m-year_
ptestresult.util-linux.libsmartcols:_tree-middle__libsmartcols/fromfile-tree-middle_
ptestresult.strace.process_vm_writev.gen.test
ptestresult.strace.shutdown.gen.test
ptestresult.strace.symlink.gen.test
ptestresult.strace.ioctl_sock_gifconf.test
ptestresult.util-linux.blkid:_ocfs2__blmkdir:_cannot_create_directory_'/usr/lib/util-linux/ptest/tests/output':_No_space_left_on_device
ptestresult.util-linux.blkid:_swap0__blkid/low-probe-swap0_
ptestresult.util-linux.blkid:_fat16_noheads__blkid/low-probe-fat16_noheads_
ptestresult.strace.rmdir.gen.test
ptestresult.strace.sched_xetscheduler.gen.test
ptestresult.util-linux.cal:_1mw-month__cal/bigyear-1mw-month_
ptestresult.util-linux.blkid:_xfs-log__blkid/low-probe-xfs-log_
ptestresult.strace.umask.gen.test
ptestresult.util-linux.cal:_1sw-month__cal/bigyear-1sw-month_
ptestresult.openssh.ust/overlap/test_overlap_210_-_Trace_match
ptestresult.util-linux.blkid:_small-fat32__blkid/low-probe-small-fat32_
ptestresult.strace.wait4-v.gen.test
ptestresult.util-linux.blkid:_bcache-C__blkid/low-probe-bcache-C_
ptestresult.util-linux.cal:_3mj-month__cal/bigyear-3mj-month_
ptestresult.python.test_guess_minor_type__email.test.test_email_renamed.TestMIMEAudio_
ptestresult.busybox.wget-handles-empty-path
ptestresult.strace.splice.gen.test
ptestresult.strace.statfs64.gen.test
ptestresult.util-linux.blkid:_udf-hdd-mkudffs-1.3-1__blkid/low-probe-udf-hdd-mkudffs-1.3-1_
ptestresult.strace.btrfs-v.test
ptestresult.util-linux.blkid:_hpt45x-raid__blkid/low-probe-hpt45x-raid_
ptestresult.strace.vmsplice.gen.test
ptestresult.perl.lib/warnings
ptestresult.util-linux.getopt:_quiet_option_short__getopt/options-quiet_option_short_
ptestresult.strace.execve.test
ptestresult.strace.semop.gen.test
ptestresult.strace.vfork-f.gen.test
ptestresult.util-linux.blkid:_hpt37x-raid__blkid/low-probe-hpt37x-raid_
ptestresult.python.test_package__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmpI8OCId/launch.py':
ptestresult.util-linux.getopt:_long_option_ambiguous_1__getopt/options-long_option_ambiguous_1_
ptestresult.util-linux.misc:_options__misc/setarch-options_
ptestresult.strace.xetitimer.gen.test
ptestresult.util-linux.cal:_1mjw-month__cal/bigyear-1mjw-month_
ptestresult.strace.ptrace.gen.test
ptestresult.util-linux.findmnt:_options-no-multi__findmnt/filter-options-no-multi_
ptestresult.strace.umovestr3.gen.test
ptestresult.strace.so_linger.gen.test
ptestresult.util-linux.findmnt:_types-multi__findmnt/filter-types-multi_
ptestresult.util-linux.hexdump:_2b_hex__hexdump/format-strings-2b_hex_
ptestresult.strace.count.test
ptestresult.strace.time.gen.test
ptestresult.openssh.ust/overlap/test_overlap_30_-_Trace_match
ptestresult.util-linux.blkid:_iso-joliet__blkid/low-probe-iso-joliet_
ptestresult.util-linux.blkid:_mpool__blkid/low-probe-mpool_
ptestresult.util-linux.cal:_3s-week40__cal/weekarg-3s-week40_
ptestresult.util-linux.cal:_3mjw-month__cal/bigyear-3mjw-month_
ptestresult.strace.setgroups.gen.test
ptestresult.util-linux.cal:_3mj-month__cal/sep1752-3mj-month_
ptestresult.openssh.ust/overlap/test_overlap_229_-_Trace_match
ptestresult.strace.rt_tgsigqueueinfo.gen.test
ptestresult.util-linux.libfdisk:_mbr-logical__libfdisk/mkpart-full-mbr-logical_
ptestresult.openssh.ust/overlap/test_overlap_151_-_Trace_match
ptestresult.util-linux.hexdump:_1b_octal-7__hexdump/highlighting-1b_octal-7_
ptestresult.util-linux.fdisk:_invalid_input_tests__fdisk/oddinput_
ptestresult.python.testInvalidFd_overflow__test.test_fileio.OtherFileTests_
ptestresult.perl.t/op/stat
ptestresult.util-linux.blkid:_vdo__blkid/low-probe-vdo_
ptestresult.util-linux.cal:_1sj-year__cal/sep1752-1sj-year_
ptestresult.strace.siginfo.gen.test
ptestresult.strace.futex.test
ptestresult.util-linux.losetup:_losetup__losetup/losetup_
ptestresult.util-linux.cal:_ymw__cal/weeknum-ymw_
ptestresult.util-linux.cal:_1m-week53__cal/weekarg-1m-week53_
ptestresult.util-linux.libmount:_losetup-loop__libmount/loop_
ptestresult.util-linux.misc:_ul__misc/ul_
ptestresult.strace.setresgid32.gen.test
ptestresult.openssh.ust/overlap/test_overlap_31_-_Trace_match
ptestresult.strace.getuid.test
ptestresult.util-linux.blkid:_atari-icd__blkid/lowprobe-pt-atari-icd_
ptestresult.python.test_zipfile_error__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmp242HIX/test_zip.zip':
ptestresult.strace.detach-stopped.test
ptestresult.util-linux.blkid:_gfs2__blkid/low-probe-gfs2_
ptestresult.util-linux.findmnt:_default__findmnt/outputs-default_
ptestresult.strace.fadvise64.test
ptestresult.util-linux.cal:_1mj-year__cal/bigyear-1mj-year_
ptestresult.strace.qual_signal.test
ptestresult.util-linux.blkid:_ext3__blkid/low-probe-ext3_
ptestresult.util-linux.blkid:_fat__blkid/low-probe-fat_
ptestresult.strace.sched_xetaffinity.gen.test
ptestresult.strace.timerfd_xettime.gen.test
ptestresult.util-linux.cal:_1sw-year__cal/bigyear-1sw-year_
ptestresult.util-linux.cal:_color__cal/color_
ptestresult.util-linux.cal:_3sw__cal/weeknum-3sw_
ptestresult.util-linux.cal:_ysj__cal/year-ysj_
ptestresult.openssh.ust/overlap/test_overlap_230_-_Trace_match
ptestresult.strace.prctl-pdeathsig.test
ptestresult.util-linux.cal:_3mw__cal/month-3mw_
ptestresult.strace.select.gen.test
ptestresult.strace.net-yy-inet.test
ptestresult.util-linux.hexdump:_4b_dec-6__hexdump/highlighting-4b_dec-6_
ptestresult.util-linux.cal:_3sj__cal/month-3sj_
ptestresult.perl.t/op/pack
ptestresult.openssh.ust/overlap/test_overlap_77_-_Trace_match
ptestresult.util-linux.blkid:_isw-raid__blkid/low-probe-isw-raid_
ptestresult.strace.qual_fault-syntax.test
ptestresult.openssh.ust/overlap/test_overlap_97_-_Trace_match
ptestresult.util-linux.misc:_rev__misc/rev_
ptestresult.util-linux.hexdump:_4b_dec-2__hexdump/highlighting-4b_dec-2_
ptestresult.util-linux.blkid:_udf-hdd-mkudffs-1.3-3__blkid/low-probe-udf-hdd-mkudffs-1.3-3_
ptestresult.python.test_zipfile_compiled__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmpMLbD9C/test_zip.zip':
ptestresult.util-linux.findmnt:_options-neg__findmnt/filter-options-neg_
ptestresult.python.test_quiet__distutils.tests.test_bdist_rpm.BuildRpmTestCase_
ptestresult.strace.userfaultfd.gen.test
ptestresult.util-linux.cal:_ym__cal/year-ym_
ptestresult.util-linux.blkid:_netware__blkid/low-probe-netware_
ptestresult.util-linux.blkid:_hpfs__blkid/low-probe-hpfs_
ptestresult.strace.setfsuid32.gen.test
ptestresult.util-linux.libfdisk:_gpt-nopartno__libfdisk/mkmkdir:_cannot_create_directory_'/usr/lib/util-linux/ptest/tests/output':_No_space_left_on_device
ptestresult.util-linux.hexdump:_4b_dec-3__hexdump/highlighting-4b_dec-3_
ptestresult.strace.fflush.test
ptestresult.python.test_sizeof_with_buffer__test.test_array.UnsignedByteTest_
ptestresult.strace.setgid.gen.test
ptestresult.strace.utimes.gen.test
ptestresult.python.test_sizeof_without_buffer__test.test_array.UnsignedLongTest_
ptestresult.strace.setregid32.gen.test
ptestresult.strace.pc.test
ptestresult.strace.sysinfo.gen.test
ptestresult.strace.remap_file_pages.gen.test
ptestresult.strace.trace_fstatfs.gen.test
ptestresult.openssh.ust/overlap/test_overlap_22_-_Trace_match
ptestresult.util-linux.cal:_1sw-month__cal/sep1752-1sw-month_
ptestresult.util-linux.fdisk:_MBR_-_id__fdisk/id_
ptestresult.busybox.ls-s-works
ptestresult.util-linux.misc:_flock__misc/flock_
ptestresult.util-linux.blkid:_udf-cd-nero-6__blkid/low-probe-udf-cd-nero-6_
ptestresult.util-linux.cal:_1sjw-month__cal/sep1752-1sjw-month_
ptestresult.util-linux.libfdisk:_gpt__libfdisk/mkpart-full-gpt_
ptestresult.util-linux.mount:_by_label__fstab___mount/fstab-label_
ptestresult.util-linux.blkid:_nvidia-raid__blkid/low-probe-nvidia-raid_
ptestresult.util-linux.blkid:_fat32_label_64MB__blkid/low-probe-fat32_label_64MB_
ptestresult.util-linux.getopt:_unquoted_option_bash__getopt/options-unquoted_option_bash_
ptestresult.util-linux.getopt:_name_option_long__getopt/options-name_option_long_
ptestresult.util-linux.cal:_3sj-month__cal/sep1752-3sj-month_
ptestresult.util-linux.cal:_1mj__cal/month-1mj_
ptestresult.openssh.tools/crash/test_crash_64_-_Trace_match
ptestresult.python.test_sizeof_with_buffer__test.test_array.UnsignedShortTest_
ptestresult.python.test_package_error__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmpeKyhzb/launch.py':
ptestresult.util-linux.cal:_1mjw-year__cal/bigyear-1mjw-year_
ptestresult.strace.setdomainname.gen.test
ptestresult.strace.printpath-umovestr-legacy.test
ptestresult.strace.syslog.gen.test
ptestresult.util-linux.getopt:_alternative_option_clash__getopt/options-alternative_option_clash_
ptestresult.openssh.ust/overlap/test_overlap_197_-_Trace_match
ptestresult.strace.sxetmask.gen.test
ptestresult.util-linux.cal:_ysjw__cal/year-ysjw_
ptestresult.util-linux.libfdisk:_mbr-err-logical__libfdisk/mkpart-full-mbr-err-logical_
ptestresult.strace.utime.gen.test
ptestresult.util-linux.hexdump:_2b_dec-6__hexdump/highlighting-2b_dec-6_
ptestresult.util-linux.mount:_remount__mount/remount_
ptestresult.util-linux.blkid:_vmfs_volume__blkid/low-probe-vmfs_volume_
ptestresult.strace.setfsuid.gen.test
ptestresult.util-linux.misc:_fallocate__misc/fallocate_
ptestresult.util-linux.column:_multiple_files__ul_cd:_/usr/lib/util-linux/ptest/tests/output/column:_no_such_directory_
ptestresult.openssh.ust/overlap/test_overlap_11_-_Trace_match
ptestresult.strace.waitid-v.gen.test
ptestresult.strace.sched.gen.test
ptestresult.strace.times-fail.gen.test
ptestresult.strace.stat.gen.test
ptestresult.strace.trace_stat_like.gen.test
ptestresult.openssh.tools/save-load/test_autoload_2_-_Lttng-tool_list_command_with_option_load-42
ptestresult.strace.ioctl_dm.test
ptestresult.util-linux.mount:_by_devname__fstab_symlink___mount/fstab-symlink_
ptestresult.util-linux.ipcs:_mk-rm-shm__ipcs/mk-rm-shm_
ptestresult.openssh.ust/overlap/test_overlap_232_-_Trace_match
ptestresult.util-linux.hexdump:_4b_dec-7__hexdump/highlighting-4b_dec-7_
ptestresult.util-linux.blkid:_silicon-raid__blkid/low-probe-silicon-raid_
ptestresult.util-linux.blkid:_hfsplus__blkid/low-probe-hfsplus_
ptestresult.python.test_zipfile__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmpSVh0tw/test_zip.zip':
ptestresult.python.test_run_module_bug1764407__test.test_cmd_line.CmdLineTest_
ptestresult.util-linux.libfdisk:_mbr-err-nospace__libfdisk/mkpart-full-mbr-err-nospace_
ptestresult.util-linux.libsmartcols:_right-maxout__libsmartcols/fromfile-right-maxout_
ptestresult.util-linux.libsmartcols:_tree__libsmartcols/fromfile-tree_
ptestresult.strace.sendfile.gen.test
ptestresult.strace.truncate64.gen.test
ptestresult.util-linux.mount:_loop__fstab___mount/fstab-loop_
ptestresult.util-linux.libsmartcols:_column-separator__libsmartcols/fromfile-column-separator_
ptestresult.strace.signalfd4.gen.test
ptestresult.util-linux.blkid:_ext2__blkid/low-probe-ext2_
ptestresult.util-linux.libfdisk:_gpt-partno-gap__libfdisk/mkpart-full-gpt-partno-gap_
ptestresult.util-linux.cal:_3m__cal/month-3m_
ptestresult.util-linux.fdisk:_nested_BSD__fdisk/bsd_
ptestresult.strace.prctl-tid_address.test
ptestresult.util-linux.blkid:_udf-hdd-mkudffs-1.3-2__blkid/low-probe-udf-hdd-mkudffs-1.3-2_
ptestresult.strace.redirect-fds.test
ptestresult.util-linux.hexdump:_4b_dec-5__hexdump/highlighting-4b_dec-5_
ptestresult.util-linux.lsns:_NSFS_for_ip-netns-add__ul_cd:_/usr/lib/util-linux/ptest/tests/output/lsns:_no_such_directory_
ptestresult.openssh.ust/overlap/test_overlap_175_-_Trace_match
ptestresult.strace.net-y-unix.test
ptestresult.strace.readlinkat.gen.test
ptestresult.util-linux.hexdump:_1b_octal-4__hexdump/highlighting-1b_octal-4_
ptestresult.strace.sync_file_range.gen.test
ptestresult.openssh.ust/overlap/test_overlap_32_-_Trace_match
ptestresult.util-linux.cal:_1mj-week53-color__cal/weekarg-1mj-week53-color_
ptestresult.strace.prctl-seccomp-filter-v.test
ptestresult.util-linux.blkid:_iso__blkid/low-probe-iso_
ptestresult.python.test_package_compiled__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmpCOwSWo/launch.py':
ptestresult.strace.detach-sleeping.test
ptestresult.openssh.tools/save-load/test_autoload_5_-_Lttng-tool_list_command_with_option_load-42
ptestresult.util-linux.cal:_3mw-month__cal/sep1752-3mw-month_
ptestresult.strace.detach-running.test
ptestresult.strace.prctl-no-args.test
ptestresult.openssh.tools/save-load/test_autoload_8_-_Lttng-tool_list_command_with_option_load-42
ptestresult.strace.waitpid.gen.test
ptestresult.python.test_module_in_package__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmpbFx96e/launch.py':
ptestresult.util-linux.cal:_1mw-year__cal/sep1752-1mw-year_
ptestresult.util-linux.blkid:_drbdmanage-control-volume__blkid/low-probe-drbdmanage-control-volume_
ptestresult.strace.prctl-name.test
ptestresult.util-linux.getopt:_quiet_output_option_long__getopt/options-quiet_output_option_long_
ptestresult.util-linux.lsmem:_s390-zvm-6g__lsmemkdir:_cannot_create_directory_'/usr/lib/util-linux/ptest/tests/output':_No_space_left_on_device
ptestresult.util-linux.findmnt:_non-root__findmnt/target-non-root_
ptestresult.bash.run-vredir
ptestresult.strace.qual_fault-exit_group.test
ptestresult.openssh.tools/mi/test_mi_3_-_MI_test:_Version_xsd_validation
ptestresult.util-linux.findmnt:_options-nameval-neg__findmnt/filter-options-nameval-neg_
ptestresult.util-linux.cal:_3m-week40__cal/weekarg-3m-week40_
ptestresult.python.test_sizeof_without_buffer__test.test_array.UnsignedIntTest_
ptestresult.util-linux.hexdump:_1b_octal-5__hexdump/highlighting-1b_octal-5_
ptestresult.openssh.ust/overlap/test_overlap_19_-_Trace_match
ptestresult.strace.socketcall.gen.test
ptestresult.util-linux.cal:_1mj-week53__cal/weekarg-1mj-week53_
ptestresult.util-linux.cal:_3sjw-month__cal/bigyear-3sjw-month_
ptestresult.openssh.ust/overlap/test_overlap_198_-_Trace_match
ptestresult.util-linux.cal:_3w__cal/jan1753-3w_
ptestresult.util-linux.fdisk:_MBR_-_dos_mode__fdisk/mbr-dos-mode_
ptestresult.python.test_newempty__test.test_code.CodeTest_
ptestresult.util-linux.cal:_mj-week40__cal/weekarg-mj-week40_
ptestresult.lzo.tools/tracefile-limits/test_tracefile_count_81_-_Trace_validation_-_longfield
ptestresult.util-linux.libsmartcols:_hidden__libsmartcols/fromfile-hidden_
ptestresult.strace.signal_receive.gen.test
ptestresult.strace.symlinkat.gen.test
ptestresult.openssh.ust/overlap/test_overlap_9_-_Trace_match
ptestresult.util-linux.login:_islocal__login/islocal_
ptestresult.strace.umovestr-illptr.gen.test
ptestresult.strace.qual_inject-retval.test
ptestresult.strace.opipe.test
ptestresult.util-linux.dmesg:_decode__dmesg/decode_
ptestresult.strace.setrlimit.gen.test
ptestresult.e2fsprogs.j_recover_csum2_32bit:_recover_32-bit_journal_checksum_v2:_failed
ptestresult.util-linux.login:_defs__login/logindefs_
ptestresult.util-linux.hexdump:_2b_dec-2__hexdump/highlighting-2b_dec-2_
ptestresult.strace.caps.test
ptestresult.strace.personality.test
ptestresult.util-linux.hexdump:_1b_octal-2__hexdump/highlighting-1b_octal-2_
ptestresult.util-linux.misc:_mcookie__misc/mcookie_
ptestresult.strace.signal.gen.test
ptestresult.strace.sigprocmask.gen.test
ptestresult.util-linux.libsmartcols:_tree-end__libsmartcols/fromfile-tree-end_
ptestresult.util-linux.cal:_1sj__cal/month-1sj_
ptestresult.util-linux.blkid:_befs__blkid/low-probe-befs_
ptestresult.util-linux.getopt:_invalid_getopt_option__getopt/options-invalid_getopt_option_
ptestresult.util-linux.cal:_3s-month__cal/sep1752-3s-month_
ptestresult.strace.timer_xettime.gen.test
ptestresult.strace.seccomp-strict.test
ptestresult.util-linux.cal:_1sjw-year__cal/bigyear-1sjw-year_
ptestresult.strace.umoven-illptr.gen.test
ptestresult.util-linux.cal:_3sw-month__cal/bigyear-3sw-month_
ptestresult.util-linux.column:_invalid_multibyte__ul_cd:_/usr/lib/util-linux/ptest/tests/output/column:_no_such_directory_
ptestresult.util-linux.mount:_by_label__fstab_devname___mount/fstab-label2devname_
ptestresult.util-linux.libsmartcols:_raw__libsmartcols/fromfile-raw_
ptestresult.openssh.ust/overlap/test_overlap_220_-_Trace_match
ptestresult.util-linux.cal:_3mjw__cal/weeknum-3mjw_
ptestresult.util-linux.cal:_1s-month__cal/sep1752-1s-month_
ptestresult.util-linux.getopt:_compatible__getopt/options-compatible_
ptestresult.util-linux.mount:_by_devname__fstab_uuid___mount/fstab-devname2uuid_
ptestresult.util-linux.cal:_1s-month__cal/bigyear-1s-month_
ptestresult.openssh.ust/overlap/test_overlap_108_-_Trace_match
ptestresult.util-linux.blkid:_udf-hdd-win7__blkid/low-probe-udf-hdd-win7_
ptestresult.lzo.tools/tracefile-limits/test_tracefile_count_71_-_Trace_validation_-_longfield
ptestresult.strace.sched_get_priority_mxx.gen.test
ptestresult.strace.waitid.gen.test
ptestresult.util-linux.blkid:_udf-hdd-mkudffs-1.3-7__blkid/low-probe-udf-hdd-mkudffs-1.3-7_
ptestresult.strace.setreuid32.gen.test
ptestresult.util-linux.libsmartcols:_export__libsmartcols/fromfile-export_
ptestresult.strace.sync_file_range2.gen.test
ptestresult.util-linux.hexdump:_1b_octal-9__hexdump/highlighting-1b_octal-9_
ptestresult.strace.trace_stat.gen.test
ptestresult.util-linux.blkid:_dos+bsd__blkid/lowprobe-pt-dos+bsd_
ptestresult.util-linux.hexdump:_1b_octal-8__hexdump/highlighting-1b_octal-8_
ptestresult.strace.recvmmsg-timeout.gen.test
ptestresult.python.test_exception_with_doc__test.test_exceptions.TestSameStrAndUnicodeMsg_
ptestresult.strace.xattr-strings.gen.test
ptestresult.util-linux.cal:_1mw-month__cal/sep1752-1mw-month_
ptestresult.strace.stat64.gen.test
ptestresult.openssh.tools/save-load/test_load_2_-_Load_command_with_opts:_-i_./tools/save-load//load-42.lttng
ptestresult.util-linux.blkid:_promise-raid__blkid/low-probe-promise-raid_
ptestresult.strace.setgroups32.gen.test
ptestresult.strace.times.gen.test
ptestresult.strace.prctl-seccomp-strict.test
ptestresult.util-linux.blkid:_luks2__blkid/low-probe-luks2_
ptestresult.util-linux.cal:_ymjw__cal/weeknum-ymjw_
ptestresult.util-linux.column:_columnate__ul_cd:_/usr/lib/util-linux/ptest/tests/output/column:_no_such_directory_
ptestresult.busybox.wget--O-overrides--P
ptestresult.strace.sched_rr_get_interval.gen.test
ptestresult.util-linux.cal:_ys__cal/year-ys_
ptestresult.strace.trace_statfs_like.gen.test
ptestresult.util-linux.blkid:_jbd__blkid/low-probe-jbd_
ptestresult.util-linux.lsns:_NETNSID_compare_to_ip-link__ul_cd:_/usr/lib/util-linux/ptest/tests/output/lsns:_no_such_directory_
ptestresult.util-linux.blkid:_cramfs__blkid/low-probe-cramfs_
ptestresult.busybox.sed_embedded_NUL
ptestresult.strace.trace_lstat.gen.test
ptestresult.openssh.ust/overlap/test_overlap_110_-_Trace_match
ptestresult.util-linux.getopt:_posix_correctly__getopt/options-posix_correctly_
ptestresult.strace.clock.gen.test
ptestresult.strace.quotactmkdir:_cannot_create_directory_'quotactl-v.dir':_No_space_left_on_device
ptestresult.dbus-test.test/test-bus
ptestresult.strace.setuid32.gen.test
ptestresult.util-linux.libsmartcols:_tree-json__libsmartcols/fromfile-tree-json_
ptestresult.util-linux.getopt:_unquoted_option_tcsh__getopt/options-unquoted_option_tcsh_
ptestresult.util-linux.more:_squeeze__more/squeeze_
ptestresult.util-linux.getopt:_invocation_model_two_as_one__getopt/options-invocation_model_two_as_one_
ptestresult.util-linux.blkid:_exfat__blkid/low-probe-exfat_
ptestresult.util-linux.getopt:_unknown_options__getopt/options-unknown_options_
ptestresult.strace.prctl-arg2-intptr.test
ptestresult.strace.setns.gen.test
ptestresult.bash.run-new-exp
ptestresult.util-linux.libfdisk:_mbr-space-gap__libfdisk/mkpart-full-mbr-space-gap_
ptestresult.util-linux.cal:_s-week40__cal/weekarg-s-week40_
ptestresult.util-linux.blkid:_vmfs__blkid/low-probe-vmfs_
ptestresult.strace.umount2.test
ptestresult.python.test_run_module__test.test_cmd_line.CmdLineTest_
ptestresult.util-linux.cal:_1sj-month__cal/bigyear-1sj-month_
ptestresult.strace.poll.test
ptestresult.util-linux.libsmartcols:_right__libsmartcols/fromfile-right_
ptestresult.openssh.tools/crash/test_crash_53_-_Expect_10_recup_event_from_buffers_got_0
ptestresult.util-linux.cal:_1m-year__cal/bigyear-1m-year_
ptestresult.util-linux.cal:_3mjw-month__cal/sep1752-3mjw-month_
ptestresult.util-linux.misc:_ionice__misc/ionice_
ptestresult.util-linux.cal:_3s__cal/month-3s_
ptestresult.util-linux.cal:_ymj__cal/year-ymj_
ptestresult.strace.quotactl-v.gen.test
ptestresult.util-linux.getopt:_csh__getopt/options-csh_
ptestresult.openssh.ust/overlap/test_overlap_199_-_Trace_match
ptestresult.python.test_polar_errno__test.test_cmath.CMathTests_
ptestresult.util-linux.hexdump:_1b_char-3__hexdump/highlighting-1b_char-3_
ptestresult.util-linux.cal:_3m-week53-color__cal/weekarg-3m-week53-color_
ptestresult.util-linux.cal:_1sj-year__cal/bigyear-1sj-year_
ptestresult.strace.setuid.gen.test
ptestresult.openssh.ust/overlap/test_overlap_162_-_Trace_match
ptestresult.util-linux.cal:_3s-week54-color__cal/weekarg-3s-week54-color_
ptestresult.python.test_sizeof_with_buffer__test.test_array.UnsignedLongTest_
ptestresult.util-linux.blkdiscard:_offsets__blkdiscard/offsets_
ptestresult.strace.readv.test
ptestresult.strace.bexecve.test
ptestresult.util-linux.getopt:_invocation_without_parameters__getopt/options-invocation_without_parameters_
ptestresult.bash.run-read
ptestresult.strace.interactive_block.test
ptestresult.openssh.ust/overlap/test_overlap_10_-_Trace_match
ptestresult.strace.request_key.gen.test
ptestresult.util-linux.cal:_1m-month__cal/sep1752-1m-month_
ptestresult.util-linux.cal:_3mw__cal/weeknum-3mw_
ptestresult.util-linux.libfdisk:_mbr-nopartno__libfdisk/mkpart-mbr-nopartno_
ptestresult.openssh.ust/overlap/test_overlap_98_-_Trace_match
ptestresult.util-linux.libfdisk:_mbr-primary-nopartno__libfdisk/mkpart-full-mbr-primary-nopartno_
ptestresult.util-linux.findmnt:_options-nameval__findmnt/filter-options-nameval_
ptestresult.openssh.tools/crash/test_crash_77_-_Trace_match
ptestresult.util-linux.libfdisk:_mbr-nopartno__libfdisk/mkpart-full-mbr-nopartno_
ptestresult.util-linux.cal:_3mj__cal/month-3mj_
ptestresult.python.test_basic_script__test.test_cmd_line_script.CmdLineTest__..._Output_from_test_script_'/var/volatile/tmp/tmpddyv3r/script.py':
ptestresult.strace.regex.gen.test
ptestresult.util-linux.hexdump:_1b_octal-1__hexdump/highlighting-1b_octal-1_
ptestresult.util-linux.hexdump:_2b_octal__hexdump/format-strings-2b_octal_
ptestresult.util-linux.cal:_color_with_week_numbers__cal/colorw_
ptestresult.strace.ustat.gen.test
ptestresult.python.test_basics_capi__test.test_codecs.BasicUnicodeTest_
ptestresult.openssh.ust/overlap/test_overlap_121_-_Trace_match
ptestresult.util-linux.getopt:_quiet_option_long__getopt/options-quiet_option_long_
ptestresult.util-linux.getopt:_name_option_short__getopt/options-name_option_short_
ptestresult.util-linux.libfdisk:_mbr-logic__libfdisk/mkpart-mbr-logic_
ptestresult.strace.pipe.test
ptestresult.e2fsprogs.t_uninit_bg_rm:_remove_uninit_bg:_failed
ptestresult.util-linux.blkid:_xfs__blkid/low-probe-xfs_
ptestresult.util-linux.cal:_week-iso__cal/sep1752-week-iso_
ptestresult.util-linux.cal:_1mjw-month__cal/sep1752-1mjw-month_
ptestresult.util-linux.cal:_1sjw-month__cal/bigyear-1sjw-month_
ptestresult.openssh.ust/overlap/test_overlap_120_-_Trace_match
ptestresult.openssh.ust/overlap/test_overlap_96_-_Trace_match
ptestresult.util-linux.blkid:_swap1__blkid/low-probe-swap1_
ptestresult.strace.umovestr.test
ptestresult.openssh.ust/overlap/test_overlap_219_-_Trace_match
ptestresult.strace.sigpending.gen.test
ptestresult.util-linux.findmnt:_types-neg__findmnt/filter-types-neg_
ptestresult.strace.netlink_sock_diag.test
ptestresult.util-linux.blkid:_romfs__blkid/low-probe-romfs_
ptestresult.lzo.tools/tracefile-limits/test_tracefile_count_80_-_Trace_validation_-_netintfield
ptestresult.util-linux.blkid:_udf-hdd-mkudffs-1.3-4__blkid/low-probe-udf-hdd-mkudffs-1.3-4_
ptestresult.util-linux.fdisk:_gpt-resize__fdisk/gpt-resize_
ptestresult.strace.lseek.test
ptestresult.util-linux.blkid:_ubi__blkid/low-probe-ubi_
ptestresult.strace.setgid32.gen.test
ptestresult.util-linux.mount:_by_devname__mount/devname_
ptestresult.util-linux.hexdump:_empty-format__hexdump/format-strings-empty-format_
ptestresult.strace.eventfd.test
ptestresult.util-linux.blkid:_fat32_cp850_O_tilde__blkid/low-probe-fat32_cp850_O_tilde_
ptestresult.strace.get_regs.test
ptestresult.util-linux.mount:_regular_file__mount/regfile_
ptestresult.util-linux.cal:_1mjw-year__cal/sep1752-1mjw-year_
ptestresult.strace.attach-f-p.test
ptestresult.util-linux.misc:_strtosize__misc/strtosize_
ptestresult.strace.sched_yield.gen.test
ptestresult.strace.readdir.gen.test
ptestresult.openssh.tools/crash/test_crash_54_-_Expect_10_recup_event_from_symlink_buffers_got_0
ptestresult.strace.setregid.gen.test
ptestresult.e2fsprogs.j_recover_csum2_64bit:_recover_64-bit_journal_checksum_v2:_failed
ptestresult.strace.qual_inject-syntax.test
ptestresult.strace.xattr.gen.test
ptestresult.util-linux.mount:_by_uuid__fstab_devname___mount/fstab-uuid2devname_
ptestresult.strace.reboot.gen.test
ptestresult.util-linux.mount:_shared-subtree__mount/shared-subtree_
ptestresult.util-linux.misc:_setsid__misc/setsid_
ptestresult.util-linux.cal:_m-week40__cal/weekarg-m-week40_
ptestresult.util-linux.hexdump:_4b_dec-4__hexdump/highlighting-4b_dec-4_
ptestresult.util-linux.libfdisk:_gpt-space-gap__libfdisk/mkpart-full-gpt-space-gap_
ptestresult.openssh.ust/overlap/test_overlap_20_-_Trace_match
ptestresult.strace.prctl-securebits.test
ptestresult.openssh.ust/overlap/test_overlap_153_-_Trace_match
ptestresult.util-linux.getopt:_invocation_model_three_as_one__getopt/options-invocation_model_three_as_one_
ptestresult.strace.ugetrlimit.gen.test
ptestresult.util-linux.blkid:_udf-hdd-mkudffs-1.3-5__blkid/low-probe-udf-hdd-mkudffs-1.3-5_
ptestresult.util-linux.libfdisk:_gpt__libfdisk/mkpart-gpt_
ptestresult.strace.brk.test
ptestresult.strace.llseek.test
ptestresult.util-linux.blkid:_udf-hdd-mkudffs-1.3-8__blkid/low-probe-udf-hdd-mkudffs-1.3-8_
ptestresult.openssh.ust/overlap/test_overlap_99_-_Trace_match
ptestresult.util-linux.getopt:_same_long_short_options__getopt/options-same_long_short_options_
ptestresult.util-linux.blkid:_bcache-B__blkid/low-probe-bcache-B_
ptestresult.util-linux.libsmartcols:_wrap__libsmartcols/fromfile-wrap_
ptestresult.lzo.tools/tracefile-limits/test_tracefile_count_70_-_Trace_validation_-_netintfield
ptestresult.openssh.ust/overlap/test_overlap_207_-_Trace_match
ptestresult.util-linux.mount:_by_label__fstab_uuid___mount/fstab-label2uuid_
ptestresult.util-linux.cal:_1m__cal/month-1m_
ptestresult.openssh.ust/overlap/test_overlap_107_-_Trace_match
ptestresult.util-linux.more:_regexp__more/regexp_
ptestresult.util-linux.dmesg:_facilities__dmesg/facilities_
ptestresult.util-linux.cal:_3sj-week40__cal/weekarg-3sj-week40_
ptestresult.strace.printstrn-umoven-undumpable.gen.test
ptestresult.strace.qual_inject-error-signal.test
ptestresult.strace.so_peercred.gen.test
ptestresult.util-linux.cal:_3sjw__cal/month-3sjw_
ptestresult.util-linux.getopt:_basic__getopt/basic_
ptestresult.python.test_add_header__email.test.test_email_renamed.TestMIMEAudio_
ptestresult.python.test_sizeof_with_buffer__test.test_array.UnsignedIntTest_
ptestresult.util-linux.cal:_1sw__cal/month-1sw_
ptestresult.strace.ioctl_loop-nv.test
ptestresult.strace.rt_sigqueueinfo.gen.test
ptestresult.util-linux.cal:_3m-week52-color__cal/weekarg-3m-week52-color_
ptestresult.util-linux.blkid:_jmicron-raid__blkid/low-probe-jmicron-raid_
ptestresult.util-linux.blkid:_reiser3__blkid/low-probe-reiser3_
ptestresult.strace.net-yy-unix.test
ptestresult.util-linux.cal:_1s-year__cal/bigyear-1s-year_
ptestresult.e2fsprogs.j_recover_csum3_64bit:_recover_64-bit_journal_checksum_v3:_failed
ptestresult.util-linux.blkid:_gpt__blkid/lowprobe-pt-gpt_
ptestresult.openssh.ust/overlap/test_overlap_33_-_Trace_match
ptestresult.util-linux.mount:_by_uuid__fstab___mount/fstab-uuid_
ptestresult.openssh.ust/overlap/test_overlap_196_-_Trace_match
ptestresult.util-linux.fdisk:_MBR_-_non-dos_mode__fdisk/mbr-nondos-mode_
ptestresult.util-linux.blkid:_ufs__blkid/low-probe-ufs_
ptestresult.util-linux.cal:_1mj-year__cal/sep1752-1mj-year_
ptestresult.busybox.ls-l-works
ptestresult.python.test_add_header__email.test.test_email.TestMIMEAudio_
ptestresult.strace.qual_fault.test
ptestresult.strace.printstrn-umoven.gen.test
ptestresult.strace.process_vm_readv.gen.test
ptestresult.util-linux.cal:_ymw__cal/year-ymw_
ptestresult.util-linux.cal:_3sj-month__cal/bigyear-3sj-month_
ptestresult.strace.rt_sigsuspend.gen.test
ptestresult.util-linux.isosize:_sector_output__isosize/print-size-sector_output_
ptestresult.util-linux.hexdump:_2b_dec__hexdump/format-strings-2b_dec_
ptestresult.util-linux.mount:_btrfs__fstab___mount/fstab-btrfs_
ptestresult.strace.renameat2.gen.test
--------------------------------------------------------------------------------------------------------------
[-- Attachment #3: 2.5.3_RC3_results_merged.zip --]
[-- Type: application/x-zip-compressed, Size: 36499 bytes --]
[-- Attachment #4: 2.5.3_rc3_intel_wr_merged_report --]
[-- Type: application/octet-stream, Size: 7327 bytes --]
==============================================================================================================
Test Result Status Summary (Counts/Percentages sorted by testseries, ID)
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
Test Series | ID | Passed | Failed | Skipped
--------------------------------------------------------------------------------------------------------------
beaglebone | manual_bsps-hw_20190415101630 | 22 (48%) | 0 (0%) | 24 (52%)
beaglebone | manual_compliance-test_20190415102409 | 4 (67%) | 0 (0%) | 2 (33%)
beaglebone | runtime_core-image-sato-sdk_beaglebone-yocto_20190410043531 | 48 (79%) | 1 (2%) | 12 (20%)
bitbake | manual_oe-core_20190411155828 | 7 (100%) | 0 (0%) | 0 (0%)
build-appliance | manual_build-appliance_20190412091610 | 3 (75%) | 1 (25%) | 0 (0%)
crops | sdkext_core-image-sato-sdk_x86_64_qemux86_20190417070541 | 20 (100%) | 0 (0%) | 0 (0%)
eclipse-plugin | manual_eclipse-plugin_20190417104509 | 6 (100%) | 0 (0%) | 0 (0%)
eclipse-plugin | manual_eclipse-plugin_20190417132839 | 6 (100%) | 0 (0%) | 0 (0%)
edgerouter | manual_bsps-hw_20190415105842 | 11 (24%) | 0 (0%) | 35 (76%)
edgerouter | manual_compliance-test_20190415110648 | 2 (33%) | 0 (0%) | 4 (67%)
edgerouter | runtime_core-image-sato-sdk_edgerouter_20190411013527 | 47 (77%) | 2 (3%) | 12 (20%)
kernel-dev | manual_kernel-configuration_20190416165928 | 9 (100%) | 0 (0%) | 0 (0%)
minnowboard | manual_bsps-hw_20190416094344 | 43 (100%) | 0 (0%) | 0 (0%)
minnowboard | runtime_core-image-lsb-sdk_genericx86-64_20190416030314 | 48 (79%) | 0 (0%) | 13 (21%)
minnowboard | runtime_core-image-lsb-sdk_genericx86_20190416094619 | 48 (79%) | 0 (0%) | 13 (21%)
minnowboard | runtime_core-image-sato-sdk_genericx86-64_20190416012138 | 49 (80%) | 0 (0%) | 12 (20%)
minnowboard | runtime_core-image-sato-sdk_genericx86_20190416072759 | 49 (80%) | 0 (0%) | 12 (20%)
mpc8315e-rdb | manual_bsps-hw_20190415102615 | 15 (33%) | 0 (0%) | 31 (67%)
mpc8315e-rdb | manual_compliance-test_20190415103232 | 2 (33%) | 0 (0%) | 4 (67%)
mpc8315e-rdb | runtime_core-image-sato-sdk_mpc8315e-rdb_20190410070846 | 47 (77%) | 2 (3%) | 12 (20%)
nuc6 | manual_compliance-test_20190416170925 | 4 (67%) | 0 (0%) | 2 (33%)
nuc6 | runtime_core-image-lsb-sdk_genericx86-64_20190415062722 | 48 (79%) | 0 (0%) | 13 (21%)
nuc6 | runtime_core-image-sato-sdk_genericx86-64_20190415051153 | 49 (80%) | 0 (0%) | 12 (20%)
nuc7 | manual_bsps-hw_20190415085927 | 43 (100%) | 0 (0%) | 0 (0%)
nuc7 | runtime_core-image-lsb-sdk_genericx86-64_20190416070445 | 48 (79%) | 0 (0%) | 13 (21%)
nuc7 | runtime_core-image-sato-sdk_genericx86-64_20190412085847 | 49 (80%) | 0 (0%) | 12 (20%)
qemuarm64 | manual_bsps-qemu_20190416173043 | 5 (62%) | 0 (0%) | 3 (38%)
qemuarm | manual_bsps-qemu_20190416172910 | 5 (62%) | 0 (0%) | 3 (38%)
qemumips64 | manual_bsps-qemu_20190416172522 | 5 (62%) | 0 (0%) | 3 (38%)
qemumips | manual_bsps-qemu_20190416172259 | 5 (62%) | 0 (0%) | 3 (38%)
qemuppc | manual_bsps-qemu_20190416172725 | 5 (62%) | 0 (0%) | 3 (38%)
qemux86-64 | manual_bsps-qemu_20190416172006 | 5 (62%) | 0 (0%) | 3 (38%)
qemux86 | manual_bsps-qemu_20190416171331 | 8 (100%) | 0 (0%) | 0 (0%)
sdk-extras | manual_sdk_20190411165245 | 1 (100%) | 0 (0%) | 0 (0%)
sdk-extras | manual_sdk_20190416175031 | 1 (100%) | 0 (0%) | 0 (0%)
sdk-extras | oeselftest_ubuntu-16.04_qemuarm_20190411094931 | 3 (100%) | 0 (0%) | 0 (0%)
sdk-extras | oeselftest_ubuntu-16.04_qemumips_20190411101805 | 3 (100%) | 0 (0%) | 0 (0%)
sdk-extras | oeselftest_ubuntu-16.04_qemuppc_20190411105505 | 3 (100%) | 0 (0%) | 0 (0%)
sdk-extras | oeselftest_ubuntu-16.04_qemux86_20190410105457 | 3 (100%) | 0 (0%) | 0 (0%)
toaster | manual_toaster-managed-mode_20190417143733 | 62 (95%) | 3 (5%) | 0 (0%)
toaster | manual_toaster-unmanaged-mode_20190417110922 | 28 (100%) | 0 (0%) | 0 (0%)
--------------------------------------------------------------------------------------------------------------
There was no ptest data
==============================================================================================================
Failed test cases (sorted by testseries, ID)
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
testseries | result_id : beaglebone | runtime_core-image-sato-sdk_beaglebone-yocto_20190410043531
parselogs.ParseLogsTest.test_parselogs
##[QA comment]: WR team is investigating more on this issue. No bug filed as this is not a real Yocto bug.
testseries | result_id : build-appliance | manual_build-appliance_20190412091610
build-appliance.build-appliance.Bitbake_build-appliance-image
##[QA comment]: Bug 12991
testseries | result_id : edgerouter | runtime_core-image-sato-sdk_edgerouter_20190411013527
xorg.XorgTest.test_xorg_running
## [QA Comment]: This is an expected failure due to hw limitation.
parselogs.ParseLogsTest.test_parselogs
##[QA comment]: WR team is investigating more on this issue. No bug filed as this is not a real Yocto bug.
testseries | result_id : mpc8315e-rdb | runtime_core-image-sato-sdk_mpc8315e-rdb_20190410070846
xorg.XorgTest.test_xorg_running
## [QA Comment]: This is an expected failure due to hw limitation.
parselogs.ParseLogsTest.test_parselogs
##[QA comment]: WR team is investigating more on this issue. No bug filed as this is not a real Yocto bug.
testseries | result_id : toaster | manual_toaster-managed-mode_20190417143733
toaster-managed-mode.toaster-managed.Adding_packages_with_dependencies
toaster-managed-mode.toaster-managed.Removing_packages_without_and_with_dependencies__from_custom_images
toaster-managed-mode.toaster-managed.Adding_packages_without_dependencies_from_custom_images
## [QA Comment]: 3 failures - Bug 13191
--------------------------------------------------------------------------------------------------------------
[-- Attachment #5: 2.5.3_rc3_full_report --]
[-- Type: application/octet-stream, Size: 18007 bytes --]
==============================================================================================================
Test Result Status Summary (Counts/Percentages sorted by testseries, ID)
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
Test Series | ID | Passed | Failed | Skipped
--------------------------------------------------------------------------------------------------------------
beaglebone | manual_bsps-hw_20190415101630 | 22 (48%) | 0 (0%) | 24 (52%)
beaglebone | manual_compliance-test_20190415102409 | 4 (67%) | 0 (0%) | 2 (33%)
beaglebone | runtime_core-image-sato-sdk_beaglebone-yocto_20190410043531 | 48 (79%) | 1 (2%) | 12 (20%)
bitbake | manual_oe-core_20190411155828 | 7 (100%) | 0 (0%) | 0 (0%)
build-appliance | manual_build-appliance_20190412091610 | 3 (75%) | 1 (25%) | 0 (0%)
crops | sdkext_core-image-sato-sdk_x86_64_qemux86_20190417070541 | 20 (100%) | 0 (0%) | 0 (0%)
eclipse-plugin | manual_eclipse-plugin_20190417104509 | 6 (100%) | 0 (0%) | 0 (0%)
eclipse-plugin | manual_eclipse-plugin_20190417132839 | 6 (100%) | 0 (0%) | 0 (0%)
edgerouter | manual_bsps-hw_20190415105842 | 11 (24%) | 0 (0%) | 35 (76%)
edgerouter | manual_compliance-test_20190415110648 | 2 (33%) | 0 (0%) | 4 (67%)
edgerouter | runtime_core-image-sato-sdk_edgerouter_20190411013527 | 47 (77%) | 2 (3%) | 12 (20%)
kernel-dev | manual_kernel-configuration_20190416165928 | 9 (100%) | 0 (0%) | 0 (0%)
minnowboard | manual_bsps-hw_20190416094344 | 43 (100%) | 0 (0%) | 0 (0%)
minnowboard | runtime_core-image-lsb-sdk_genericx86-64_20190416030314 | 48 (79%) | 0 (0%) | 13 (21%)
minnowboard | runtime_core-image-lsb-sdk_genericx86_20190416094619 | 48 (79%) | 0 (0%) | 13 (21%)
minnowboard | runtime_core-image-sato-sdk_genericx86-64_20190416012138 | 49 (80%) | 0 (0%) | 12 (20%)
minnowboard | runtime_core-image-sato-sdk_genericx86_20190416072759 | 49 (80%) | 0 (0%) | 12 (20%)
mpc8315e-rdb | manual_bsps-hw_20190415102615 | 15 (33%) | 0 (0%) | 31 (67%)
mpc8315e-rdb | manual_compliance-test_20190415103232 | 2 (33%) | 0 (0%) | 4 (67%)
mpc8315e-rdb | runtime_core-image-sato-sdk_mpc8315e-rdb_20190410070846 | 47 (77%) | 2 (3%) | 12 (20%)
multilib | runtime_core-image-minimal_qemumips64_20190328012724 | 1 (2%) | 0 (0%) | 57 (98%)
multilib | runtime_core-image-sato_qemux86-64_20190328005537 | 36 (57%) | 0 (0%) | 27 (43%)
multilib | runtime_core-image-sato_qemux86-64_20190328010523 | 15 (24%) | 0 (0%) | 48 (76%)
multilib | runtime_lib32-core-image-minimal_qemux86-64_20190328004331 | 1 (100%) | 0 (0%) | 0 (0%)
multilib | runtime_lib32-core-image-minimal_qemux86-64_20190328004626 | 1 (100%) | 0 (0%) | 0 (0%)
multilib | sdk_core-image-minimal_x86_64_qemumips64_20190328012724 | 9 (82%) | 0 (0%) | 2 (18%)
musl-qemux86-64 | runtime_core-image-full-cmdline_qemux86-64_20190328005407 | 12 (20%) | 0 (0%) | 48 (80%)
musl-qemux86-64 | runtime_core-image-minimal_qemux86-64_20190328005407 | 1 (2%) | 0 (0%) | 59 (98%)
musl-qemux86-64 | runtime_core-image-sato-sdk_qemux86-64_20190328005407 | 48 (80%) | 0 (0%) | 12 (20%)
musl-qemux86 | runtime_core-image-full-cmdline_qemux86_20190328005325 | 12 (20%) | 0 (0%) | 48 (80%)
musl-qemux86 | runtime_core-image-minimal_qemux86_20190328005325 | 1 (2%) | 0 (0%) | 59 (98%)
musl-qemux86 | runtime_core-image-sato-sdk_qemux86_20190328005325 | 48 (80%) | 0 (0%) | 12 (20%)
no-x11 | runtime_core-image-full-cmdline_qemux86-64_20190328004745 | 12 (20%) | 0 (0%) | 49 (80%)
no-x11 | runtime_core-image-weston_qemux86-64_20190328004745 | 32 (52%) | 0 (0%) | 29 (48%)
nuc6 | manual_compliance-test_20190416170925 | 4 (67%) | 0 (0%) | 2 (33%)
nuc6 | runtime_core-image-lsb-sdk_genericx86-64_20190415062722 | 48 (79%) | 0 (0%) | 13 (21%)
nuc6 | runtime_core-image-sato-sdk_genericx86-64_20190415051153 | 49 (80%) | 0 (0%) | 12 (20%)
nuc7 | manual_bsps-hw_20190415085927 | 43 (100%) | 0 (0%) | 0 (0%)
nuc7 | runtime_core-image-lsb-sdk_genericx86-64_20190416070445 | 48 (79%) | 0 (0%) | 13 (21%)
nuc7 | runtime_core-image-sato-sdk_genericx86-64_20190412085847 | 49 (80%) | 0 (0%) | 12 (20%)
oe-selftest-centos | oeselftest_centos-7_qemux86-64_20190328005014 | 330 (100%) | 0 (0%) | 1 (0%)
oe-selftest-centos | runtime_core-image-full-cmdline_qemux86-64_20190328024414 | 4 (100%) | 0 (0%) | 0 (0%)
oe-selftest-centos | runtime_core-image-full-cmdline_qemux86-64_20190328024557 | 4 (100%) | 0 (0%) | 0 (0%)
oe-selftest-debian | oeselftest_debian-9_qemux86-64_20190328005049 | 330 (100%) | 0 (0%) | 1 (0%)
oe-selftest-debian | runtime_core-image-full-cmdline_qemux86-64_20190328022241 | 4 (100%) | 0 (0%) | 0 (0%)
oe-selftest-debian | runtime_core-image-full-cmdline_qemux86-64_20190328022404 | 4 (100%) | 0 (0%) | 0 (0%)
oe-selftest-fedora | oeselftest_fedora-28_qemux86-64_20190328010400 | 330 (100%) | 0 (0%) | 1 (0%)
oe-selftest-fedora | runtime_core-image-full-cmdline_qemux86-64_20190328024233 | 4 (100%) | 0 (0%) | 0 (0%)
oe-selftest-ubuntu | oeselftest_ubuntu-18.04_qemux86-64_20190328001839 | 330 (100%) | 0 (0%) | 1 (0%)
oe-selftest-ubuntu | runtime_core-image-full-cmdline_qemux86-64_20190328015546 | 4 (100%) | 0 (0%) | 0 (0%)
oe-selftest-ubuntu | runtime_core-image-full-cmdline_qemux86-64_20190328015706 | 4 (100%) | 0 (0%) | 0 (0%)
pkgman-deb-non-deb | runtime_core-image-minimal_qemux86_20190328003313 | 1 (2%) | 0 (0%) | 60 (98%)
pkgman-deb-non-deb | runtime_core-image-sato-sdk_qemux86_20190328003313 | 30 (49%) | 0 (0%) | 31 (51%)
pkgman-deb-non-deb | runtime_core-image-sato_qemux86_20190328003313 | 15 (25%) | 0 (0%) | 46 (75%)
pkgman-non-rpm | runtime_core-image-minimal_qemux86_20190328002823 | 1 (2%) | 0 (0%) | 60 (98%)
pkgman-non-rpm | runtime_core-image-minimal_qemux86_20190328004538 | 1 (2%) | 0 (0%) | 60 (98%)
pkgman-non-rpm | runtime_core-image-sato-sdk_qemux86_20190328002823 | 31 (51%) | 0 (0%) | 30 (49%)
pkgman-non-rpm | runtime_core-image-sato-sdk_qemux86_20190328004538 | 30 (49%) | 0 (0%) | 31 (51%)
pkgman-non-rpm | runtime_core-image-sato_qemux86_20190328002823 | 14 (23%) | 0 (0%) | 47 (77%)
pkgman-non-rpm | runtime_core-image-sato_qemux86_20190328004538 | 15 (25%) | 0 (0%) | 46 (75%)
pkgman-rpm-non-rpm | runtime_core-image-minimal_qemux86_20190328003915 | 1 (2%) | 0 (0%) | 60 (98%)
pkgman-rpm-non-rpm | runtime_core-image-sato-sdk_qemux86_20190328003915 | 49 (80%) | 0 (0%) | 12 (20%)
pkgman-rpm-non-rpm | runtime_core-image-sato_qemux86_20190328003915 | 35 (57%) | 0 (0%) | 26 (43%)
qa-extras2 | runtime_core-image-sato_qemux86-64_20190328003056 | 37 (61%) | 0 (0%) | 24 (39%)
qa-extras2 | runtime_core-image-sato_qemux86-64_20190328003507 | 36 (59%) | 0 (0%) | 25 (41%)
qa-extras2 | runtime_core-image-sato_qemux86-64_20190328003902 | 37 (59%) | 0 (0%) | 26 (41%)
qa-extras2 | runtime_core-image-sato_qemux86-64_20190328004342 | 42 (69%) | 0 (0%) | 19 (31%)
qa-extras2 | runtime_core-image-sato_qemux86-64_20190328004728 | 35 (57%) | 0 (0%) | 26 (43%)
qa-extras2 | runtime_core-image-sato_qemux86-64_20190328005231 | 42 (69%) | 0 (0%) | 19 (31%)
qemuarm-lsb | runtime_core-image-lsb-sdk_qemuarm_20190328010037 | 48 (79%) | 0 (0%) | 13 (21%)
qemuarm-lsb | runtime_core-image-lsb_qemuarm_20190328010037 | 34 (56%) | 0 (0%) | 27 (44%)
qemuarm64 | manual_bsps-qemu_20190416173043 | 5 (62%) | 0 (0%) | 3 (38%)
qemuarm64 | runtime_core-image-minimal_qemuarm64_20190328005246 | 1 (2%) | 0 (0%) | 59 (98%)
qemuarm64 | runtime_core-image-sato-sdk_qemuarm64_20190328005246 | 48 (80%) | 0 (0%) | 12 (20%)
qemuarm64 | runtime_core-image-sato_qemuarm64_20190328005246 | 34 (57%) | 0 (0%) | 26 (43%)
qemuarm64 | sdk_core-image-sato_i686_qemuarm64_20190328005246 | 11 (100%) | 0 (0%) | 0 (0%)
qemuarm64 | sdkext_core-image-minimal_x86_64_qemuarm64_20190328015152 | 15 (75%) | 0 (0%) | 5 (25%)
qemuarm64 | sdkext_core-image-sato_x86_64_qemuarm64_20190328015152 | 15 (75%) | 0 (0%) | 5 (25%)
qemuarm | manual_bsps-qemu_20190416172910 | 5 (62%) | 0 (0%) | 3 (38%)
qemuarm | runtime_core-image-minimal_qemuarm_20190328002950 | 1 (2%) | 0 (0%) | 60 (98%)
qemuarm | runtime_core-image-sato-sdk_qemuarm_20190328002950 | 49 (80%) | 0 (0%) | 12 (20%)
qemuarm | runtime_core-image-sato_qemuarm_20190328002950 | 35 (57%) | 0 (0%) | 26 (43%)
qemuarm | sdk_core-image-sato_i686_qemuarm_20190328002950 | 11 (100%) | 0 (0%) | 0 (0%)
qemuarm | sdkext_core-image-minimal_x86_64_qemuarm_20190328015353 | 15 (75%) | 0 (0%) | 5 (25%)
qemuarm | sdkext_core-image-sato_x86_64_qemuarm_20190328015353 | 15 (75%) | 0 (0%) | 5 (25%)
qemumips-lsb | runtime_core-image-lsb-sdk_qemumips_20190328010135 | 45 (78%) | 0 (0%) | 13 (22%)
qemumips-lsb | runtime_core-image-lsb_qemumips_20190328010135 | 34 (59%) | 0 (0%) | 24 (41%)
qemumips64 | manual_bsps-qemu_20190416172522 | 5 (62%) | 0 (0%) | 3 (38%)
qemumips64 | runtime_core-image-minimal_qemumips64_20190328010018 | 1 (2%) | 0 (0%) | 57 (98%)
qemumips64 | runtime_core-image-sato-sdk_qemumips64_20190328010018 | 46 (79%) | 0 (0%) | 12 (21%)
qemumips64 | runtime_core-image-sato_qemumips64_20190328010018 | 35 (60%) | 0 (0%) | 23 (40%)
qemumips64 | sdk_core-image-sato_i686_qemumips64_20190328010018 | 11 (100%) | 0 (0%) | 0 (0%)
qemumips64 | sdkext_core-image-minimal_x86_64_qemumips64_20190328023044 | 15 (75%) | 0 (0%) | 5 (25%)
qemumips64 | sdkext_core-image-sato_x86_64_qemumips64_20190328023044 | 15 (75%) | 0 (0%) | 5 (25%)
qemumips | manual_bsps-qemu_20190416172259 | 5 (62%) | 0 (0%) | 3 (38%)
qemumips | runtime_core-image-minimal_qemumips_20190328005803 | 1 (2%) | 0 (0%) | 57 (98%)
qemumips | runtime_core-image-sato-sdk_qemumips_20190328005803 | 46 (79%) | 0 (0%) | 12 (21%)
qemumips | runtime_core-image-sato_qemumips_20190328005803 | 35 (60%) | 0 (0%) | 23 (40%)
qemumips | sdk_core-image-sato_i686_qemumips_20190328005803 | 11 (100%) | 0 (0%) | 0 (0%)
qemumips | sdkext_core-image-minimal_x86_64_qemumips_20190328023934 | 15 (75%) | 0 (0%) | 5 (25%)
qemumips | sdkext_core-image-sato_x86_64_qemumips_20190328023934 | 15 (75%) | 0 (0%) | 5 (25%)
qemuppc-lsb | runtime_core-image-lsb-sdk_qemuppc_20190328011857 | 48 (79%) | 0 (0%) | 13 (21%)
qemuppc-lsb | runtime_core-image-lsb_qemuppc_20190328011857 | 34 (56%) | 0 (0%) | 27 (44%)
qemuppc | manual_bsps-qemu_20190416172725 | 5 (62%) | 0 (0%) | 3 (38%)
qemuppc | runtime_core-image-minimal_qemuppc_20190328010415 | 1 (2%) | 0 (0%) | 60 (98%)
qemuppc | runtime_core-image-sato-sdk_qemuppc_20190328010415 | 49 (80%) | 0 (0%) | 12 (20%)
qemuppc | runtime_core-image-sato_qemuppc_20190328010415 | 35 (57%) | 0 (0%) | 26 (43%)
qemuppc | sdk_core-image-sato_i686_qemuppc_20190328010415 | 11 (100%) | 0 (0%) | 0 (0%)
qemuppc | sdkext_core-image-minimal_x86_64_qemuppc_20190328020155 | 15 (75%) | 0 (0%) | 5 (25%)
qemuppc | sdkext_core-image-sato_x86_64_qemuppc_20190328020155 | 15 (75%) | 0 (0%) | 5 (25%)
qemux86-64-x32 | runtime_core-image-minimal_qemux86-64_20190328010011 | 1 (2%) | 0 (0%) | 60 (98%)
qemux86-64 | manual_bsps-qemu_20190416172006 | 5 (62%) | 0 (0%) | 3 (38%)
qemux86-64 | runtime_core-image-minimal_qemux86-64_20190328010727 | 1 (2%) | 0 (0%) | 60 (98%)
qemux86-64 | runtime_core-image-sato-sdk_qemux86-64_20190328010727 | 49 (80%) | 0 (0%) | 12 (20%)
qemux86-64 | runtime_core-image-sato_qemux86-64_20190328010727 | 35 (57%) | 0 (0%) | 26 (43%)
qemux86-64 | sdk_core-image-sato_i686_qemux86-64_20190328010727 | 11 (100%) | 0 (0%) | 0 (0%)
qemux86-64 | sdk_core-image-sato_x86_64_qemux86-64_20190328011455 | 11 (100%) | 0 (0%) | 0 (0%)
qemux86-64 | sdkext_core-image-minimal_x86_64_qemux86-64_20190328011455 | 15 (75%) | 0 (0%) | 5 (25%)
qemux86-64 | sdkext_core-image-sato_x86_64_qemux86-64_20190328011455 | 15 (75%) | 0 (0%) | 5 (25%)
qemux86-lsb | runtime_core-image-lsb-sdk_qemux86_20190328010015 | 48 (79%) | 0 (0%) | 13 (21%)
qemux86-lsb | runtime_core-image-lsb_qemux86_20190328010015 | 34 (56%) | 0 (0%) | 27 (44%)
qemux86 | manual_bsps-qemu_20190416171331 | 8 (100%) | 0 (0%) | 0 (0%)
qemux86 | runtime_core-image-minimal_qemux86_20190328010207 | 1 (2%) | 0 (0%) | 60 (98%)
qemux86 | runtime_core-image-sato-sdk_qemux86_20190328010207 | 49 (80%) | 0 (0%) | 12 (20%)
qemux86 | runtime_core-image-sato_qemux86_20190328010207 | 35 (57%) | 0 (0%) | 26 (43%)
qemux86 | sdk_core-image-sato_i686_qemux86_20190328010207 | 11 (100%) | 0 (0%) | 0 (0%)
qemux86 | sdk_core-image-sato_x86_64_qemux86_20190328011153 | 11 (100%) | 0 (0%) | 0 (0%)
qemux86 | sdkext_core-image-minimal_x86_64_qemux86_20190328011153 | 15 (75%) | 0 (0%) | 5 (25%)
qemux86 | sdkext_core-image-sato_x86_64_qemux86_20190328011153 | 15 (75%) | 0 (0%) | 5 (25%)
sdk-extras | manual_sdk_20190411165245 | 1 (100%) | 0 (0%) | 0 (0%)
sdk-extras | manual_sdk_20190416175031 | 1 (100%) | 0 (0%) | 0 (0%)
sdk-extras | oeselftest_ubuntu-16.04_qemuarm_20190411094931 | 3 (100%) | 0 (0%) | 0 (0%)
sdk-extras | oeselftest_ubuntu-16.04_qemumips_20190411101805 | 3 (100%) | 0 (0%) | 0 (0%)
sdk-extras | oeselftest_ubuntu-16.04_qemuppc_20190411105505 | 3 (100%) | 0 (0%) | 0 (0%)
sdk-extras | oeselftest_ubuntu-16.04_qemux86_20190410105457 | 3 (100%) | 0 (0%) | 0 (0%)
toaster | manual_toaster-managed-mode_20190417143733 | 62 (95%) | 3 (5%) | 0 (0%)
toaster | manual_toaster-unmanaged-mode_20190417110922 | 28 (100%) | 0 (0%) | 0 (0%)
--------------------------------------------------------------------------------------------------------------
There was no ptest data
==============================================================================================================
Failed test cases (sorted by testseries, ID)
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
testseries | result_id : beaglebone | runtime_core-image-sato-sdk_beaglebone-yocto_20190410043531
parselogs.ParseLogsTest.test_parselogs
testseries | result_id : build-appliance | manual_build-appliance_20190412091610
build-appliance.build-appliance.Bitbake_build-appliance-image
testseries | result_id : edgerouter | runtime_core-image-sato-sdk_edgerouter_20190411013527
parselogs.ParseLogsTest.test_parselogs
xorg.XorgTest.test_xorg_running
testseries | result_id : mpc8315e-rdb | runtime_core-image-sato-sdk_mpc8315e-rdb_20190410070846
parselogs.ParseLogsTest.test_parselogs
xorg.XorgTest.test_xorg_running
testseries | result_id : toaster | manual_toaster-managed-mode_20190417143733
toaster-managed-mode.toaster-managed.Removing_packages_without_and_with_dependencies__from_custom_images
toaster-managed-mode.toaster-managed.Adding_packages_without_dependencies_from_custom_images
toaster-managed-mode.toaster-managed.Adding_packages_with_dependencies
--------------------------------------------------------------------------------------------------------------
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: QA cycle report for 2.5.3 RC3
2019-04-18 4:58 QA cycle report for 2.5.3 RC3 Jain, Sangeeta
@ 2019-04-23 14:31 ` richard.purdie
2019-04-24 6:28 ` Jain, Sangeeta
2019-04-23 14:35 ` richard.purdie
1 sibling, 1 reply; 7+ messages in thread
From: richard.purdie @ 2019-04-23 14:31 UTC (permalink / raw)
To: Jain, Sangeeta, 'yocto@yoctoproject.org',
Eggleton, Paul, 'Michael Halstead',
Erway, Tracey M, 'sjolley.yp.pm@gmail.com',
'openembedded-core-request@lists.openembedded.org'
On Thu, 2019-04-18 at 04:58 +0000, Jain, Sangeeta wrote:
>
> QA cycle report for 2.5.3 RC3:
>
> No high milestone defects.
> Test results are available at following location:
> · For results of all automated tests, refer to results at
> public AB [1].
> · For other test results, refer to attachment [2].
> · For test report for test cases run by Intel and WR team,
> refer attachment [3]
> · For full test report, refer attachment [4]
> · For ptest results, please refer to results at public AB [5]
> · For ptest report, refer to attachment [6]
> No new defects are found in this cycle.
> Number of existing issues observed in this release is 2- toaster [7]
> and Build-appliance [8]
> For ptest, regression data is not available for this release. No
> timeout issues.
> Test result report on Public AB shows no failures.
I've been discussing with Tracy and Vineela how to best handle the test
results for the release for 2.5.3. In the end I:
a) Copied in the ptest results to the right place. This will happen
automagically in all future builds:
https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults/qemux86-64-ptest/
b) Copied in the intel test results to their own directory:
https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults-intel/
c) Manually generated an updated test report for the combined results
with the commands:
$ cd /srv/autobuilder/autobuilder.yoctoproject.org/pub/releases/yocto-2.5.3
$resulttool report . > testreport.txt
d) Added a header to the report which consisted of the report from QA
with details of the bugs etc.
This means we have a top level testreport file which contains all the
test information about the release.
I'd like to make this the standard procedure for release. The release
notes and announcement can refer to the test report included with the
release and the test results as its there all together.
Ultimately I'd like to improve the formatting of the report (separate
out ptest regressions from the other regressions, maybe html, maybe
graphs, better regression information) but that is something for the
future.
Does that work for everyone?
Cheers,
Richard
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: QA cycle report for 2.5.3 RC3
2019-04-18 4:58 QA cycle report for 2.5.3 RC3 Jain, Sangeeta
2019-04-23 14:31 ` richard.purdie
@ 2019-04-23 14:35 ` richard.purdie
2019-04-23 15:01 ` akuster808
1 sibling, 1 reply; 7+ messages in thread
From: richard.purdie @ 2019-04-23 14:35 UTC (permalink / raw)
To: Jain, Sangeeta, 'yocto@yoctoproject.org',
Eggleton, Paul, 'Michael Halstead',
Erway, Tracey M, 'sjolley.yp.pm@gmail.com',
'openembedded-core@lists.openembedded.org'
On Thu, 2019-04-18 at 04:58 +0000, Jain, Sangeeta wrote:
>
> QA cycle report for 2.5.3 RC3:
>
> No high milestone defects.
> Test results are available at following location:
> · For results of all automated tests, refer to results at
> public AB [1].
> · For other test results, refer to attachment [2].
> · For test report for test cases run by Intel and WR team,
> refer attachment [3]
> · For full test report, refer attachment [4]
> · For ptest results, please refer to results at public AB [5]
> · For ptest report, refer to attachment [6]
> No new defects are found in this cycle.
> Number of existing issues observed in this release is 2- toaster [7]
> and Build-appliance [8]
> For ptest, regression data is not available for this release. No
> timeout issues.
> Test result report on Public AB shows no failures.
I've been discussing with Tracy and Vineela how to best handle the test
results for the release for 2.5.3. In the end I:
a) Copied in the ptest results to the right place. This will happen
automagically in all future builds:
https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults/qemux86-64-ptest/
b) Copied in the intel test results to their own directory:
https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults-intel/
c) Manually generated an updated test report for the combined results
with the commands:
$ cd /srv/autobuilder/autobuilder.yoctoproject.org/pub/releases/yocto-
2.5.3
$resulttool report . > testreport.txt
d) Added a header to the report which consisted of the report from QA
with details of the bugs etc.
This means we have a top level testreport file which contains all the
test information about the release.
I'd like to make this the standard procedure for release. The release
notes and announcement can refer to the test report included with the
release and the test results as its there all together.
Ultimately I'd like to improve the formatting of the report (separate
out ptest regressions from the other regressions, maybe html, maybe
graphs, better regression information) but that is something for the
future.
Does that work for everyone?
Cheers,
Richard
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] QA cycle report for 2.5.3 RC3
2019-04-23 14:35 ` richard.purdie
@ 2019-04-23 15:01 ` akuster808
0 siblings, 0 replies; 7+ messages in thread
From: akuster808 @ 2019-04-23 15:01 UTC (permalink / raw)
To: richard.purdie, Jain, Sangeeta, 'yocto@yoctoproject.org',
Eggleton, Paul, 'Michael Halstead',
Erway, Tracey M, 'sjolley.yp.pm@gmail.com',
'openembedded-core@lists.openembedded.org'
On 4/23/19 7:35 AM, richard.purdie@linuxfoundation.org wrote:
> On Thu, 2019-04-18 at 04:58 +0000, Jain, Sangeeta wrote:
>>
>> QA cycle report for 2.5.3 RC3:
>>
>> No high milestone defects.
>> Test results are available at following location:
>> · For results of all automated tests, refer to results at
>> public AB [1].
>> · For other test results, refer to attachment [2].
>> · For test report for test cases run by Intel and WR team,
>> refer attachment [3]
>> · For full test report, refer attachment [4]
>> · For ptest results, please refer to results at public AB [5]
>> · For ptest report, refer to attachment [6]
>> No new defects are found in this cycle.
>> Number of existing issues observed in this release is 2- toaster [7]
>> and Build-appliance [8]
>> For ptest, regression data is not available for this release. No
>> timeout issues.
>> Test result report on Public AB shows no failures.
> I've been discussing with Tracy and Vineela how to best handle the test
> results for the release for 2.5.3. In the end I:
>
> a) Copied in the ptest results to the right place. This will happen
> automagically in all future builds:
>
> https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults/qemux86-64-ptest/
>
> b) Copied in the intel test results to their own directory:
>
> https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults-intel/
>
> c) Manually generated an updated test report for the combined results
> with the commands:
>
> $ cd /srv/autobuilder/autobuilder.yoctoproject.org/pub/releases/yocto-
> 2.5.3
> $resulttool report . > testreport.txt
>
> d) Added a header to the report which consisted of the report from QA
> with details of the bugs etc.
>
> This means we have a top level testreport file which contains all the
> test information about the release.
>
> I'd like to make this the standard procedure for release. The release
> notes and announcement can refer to the test report included with the
> release and the test results as its there all together.
>
> Ultimately I'd like to improve the formatting of the report (separate
> out ptest regressions from the other regressions, maybe html, maybe
> graphs, better regression information) but that is something for the
> future.
>
> Does that work for everyone?
This works for me.
- armin
>
> Cheers,
>
> Richard
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: QA cycle report for 2.5.3 RC3
@ 2019-04-23 15:01 ` akuster808
0 siblings, 0 replies; 7+ messages in thread
From: akuster808 @ 2019-04-23 15:01 UTC (permalink / raw)
To: richard.purdie, Jain, Sangeeta, 'yocto@yoctoproject.org',
Eggleton, Paul, 'Michael Halstead',
Erway, Tracey M, 'sjolley.yp.pm@gmail.com',
'openembedded-core@lists.openembedded.org'
On 4/23/19 7:35 AM, richard.purdie@linuxfoundation.org wrote:
> On Thu, 2019-04-18 at 04:58 +0000, Jain, Sangeeta wrote:
>>
>> QA cycle report for 2.5.3 RC3:
>>
>> No high milestone defects.
>> Test results are available at following location:
>> · For results of all automated tests, refer to results at
>> public AB [1].
>> · For other test results, refer to attachment [2].
>> · For test report for test cases run by Intel and WR team,
>> refer attachment [3]
>> · For full test report, refer attachment [4]
>> · For ptest results, please refer to results at public AB [5]
>> · For ptest report, refer to attachment [6]
>> No new defects are found in this cycle.
>> Number of existing issues observed in this release is 2- toaster [7]
>> and Build-appliance [8]
>> For ptest, regression data is not available for this release. No
>> timeout issues.
>> Test result report on Public AB shows no failures.
> I've been discussing with Tracy and Vineela how to best handle the test
> results for the release for 2.5.3. In the end I:
>
> a) Copied in the ptest results to the right place. This will happen
> automagically in all future builds:
>
> https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults/qemux86-64-ptest/
>
> b) Copied in the intel test results to their own directory:
>
> https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults-intel/
>
> c) Manually generated an updated test report for the combined results
> with the commands:
>
> $ cd /srv/autobuilder/autobuilder.yoctoproject.org/pub/releases/yocto-
> 2.5.3
> $resulttool report . > testreport.txt
>
> d) Added a header to the report which consisted of the report from QA
> with details of the bugs etc.
>
> This means we have a top level testreport file which contains all the
> test information about the release.
>
> I'd like to make this the standard procedure for release. The release
> notes and announcement can refer to the test report included with the
> release and the test results as its there all together.
>
> Ultimately I'd like to improve the formatting of the report (separate
> out ptest regressions from the other regressions, maybe html, maybe
> graphs, better regression information) but that is something for the
> future.
>
> Does that work for everyone?
This works for me.
- armin
>
> Cheers,
>
> Richard
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: QA cycle report for 2.5.3 RC3
2019-04-23 14:31 ` richard.purdie
@ 2019-04-24 6:28 ` Jain, Sangeeta
2019-04-25 9:55 ` richard.purdie
0 siblings, 1 reply; 7+ messages in thread
From: Jain, Sangeeta @ 2019-04-24 6:28 UTC (permalink / raw)
To: richard.purdie, 'yocto@yoctoproject.org',
Eggleton, Paul, 'Michael Halstead',
Erway, Tracey M, 'sjolley.yp.pm@gmail.com',
'openembedded-core-request@lists.openembedded.org'
>-----Original Message-----
>From: richard.purdie@linuxfoundation.org <richard.purdie@linuxfoundation.org>
>Sent: Tuesday, 23 April, 2019 10:31 PM
>To: Jain, Sangeeta <sangeeta.jain@intel.com>; 'yocto@yoctoproject.org'
><yocto@yoctoproject.org>; Eggleton, Paul <paul.eggleton@intel.com>; 'Michael
>Halstead' <mhalstead@linuxfoundation.org>; Erway, Tracey M
><tracey.m.erway@intel.com>; 'sjolley.yp.pm@gmail.com'
><sjolley.yp.pm@gmail.com>; 'openembedded-core-
>request@lists.openembedded.org' <openembedded-core-
>request@lists.openembedded.org>
>Cc: Sangal, Apoorv <apoorv.sangal@intel.com>; Yeoh, Ee Peng
><ee.peng.yeoh@intel.com>; Graydon, Tracy <tracy.graydon@intel.com>;
>Tummalapalli, Vineela <vineela.tummalapalli@intel.com>
>Subject: Re: QA cycle report for 2.5.3 RC3
>
>On Thu, 2019-04-18 at 04:58 +0000, Jain, Sangeeta wrote:
>>
>> QA cycle report for 2.5.3 RC3:
>>
>> No high milestone defects.
>> Test results are available at following location:
>> · For results of all automated tests, refer to results at
>> public AB [1].
>> · For other test results, refer to attachment [2].
>> · For test report for test cases run by Intel and WR team,
>> refer attachment [3]
>> · For full test report, refer attachment [4]
>> · For ptest results, please refer to results at public AB [5]
>> · For ptest report, refer to attachment [6]
>> No new defects are found in this cycle.
>> Number of existing issues observed in this release is 2- toaster [7]
>> and Build-appliance [8] For ptest, regression data is not available
>> for this release. No timeout issues.
>> Test result report on Public AB shows no failures.
>
>I've been discussing with Tracy and Vineela how to best handle the test results for
>the release for 2.5.3. In the end I:
>
>a) Copied in the ptest results to the right place. This will happen automagically in
>all future builds:
>
>https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults/qemux86-64-
>ptest/
>
>b) Copied in the intel test results to their own directory:
>
>https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults-intel/
>
>c) Manually generated an updated test report for the combined results with the
>commands:
>
>$ cd /srv/autobuilder/autobuilder.yoctoproject.org/pub/releases/yocto-2.5.3
>$resulttool report . > testreport.txt
>
>d) Added a header to the report which consisted of the report from QA with
>details of the bugs etc.
>
>This means we have a top level testreport file which contains all the test
>information about the release.
>
>I'd like to make this the standard procedure for release. The release notes and
>announcement can refer to the test report included with the release and the test
>results as its there all together.
>
>Ultimately I'd like to improve the formatting of the report (separate out ptest
>regressions from the other regressions, maybe html, maybe graphs, better
>regression information) but that is something for the future.
>
>Does that work for everyone?
That looks great to me.
As a further improvement for sharing of test results, I am working to create a wiki page for consolidated
test results. For 2.5.3, it can be viewed at:
https://wiki.yoctoproject.org/wiki/WW16_-_2019-04-18_-_Full_Test_Cycle_2.5.3_RC3
After you have added more results on AB, I will change some existing links on this web page with the new ones.
Another thought I have is that since for any release, all data will be removed from public AB after few days of release,
Should I include here a link to "yocto-testresults" git repo? It will make this wiki page more useful for reference in future as well.
>
>Cheers,
>
>Richard
>
Thanks & Regards,
Sangeeta Jain
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: QA cycle report for 2.5.3 RC3
2019-04-24 6:28 ` Jain, Sangeeta
@ 2019-04-25 9:55 ` richard.purdie
0 siblings, 0 replies; 7+ messages in thread
From: richard.purdie @ 2019-04-25 9:55 UTC (permalink / raw)
To: Jain, Sangeeta, 'yocto@yoctoproject.org',
Eggleton, Paul, 'Michael Halstead',
Erway, Tracey M, 'sjolley.yp.pm@gmail.com',
'openembedded-core-request@lists.openembedded.org'
On Wed, 2019-04-24 at 06:28 +0000, Jain, Sangeeta wrote:
> That looks great to me.
> As a further improvement for sharing of test results, I am working to
> create a wiki page for consolidated
> test results. For 2.5.3, it can be viewed at:
> https://wiki.yoctoproject.org/wiki/WW16_-_2019-04-18_-_Full_Test_Cycle_2.5.3_RC3
The reasons for a wiki page are historical, mainly from the testopia
plugin to the wiki. I think going forward we should focus on having a
test report included in the release artefacts and that we shouldn't
need the wiki page.
> After you have added more results on AB, I will change some existing
> links on this web page with the new ones.
>
> Another thought I have is that since for any release, all data will
> be removed from public AB after few days of release,
> Should I include here a link to "yocto-testresults" git repo? It
> will make this wiki page more useful for reference in future as well.
Release data is moved from the autobuilder to the main public release
location, e.g.:
http://downloads.yoctoproject.org/releases/yocto/yocto-2.6.2/testresults/
So if we put a test report as a release artefact, it moves with the
rest of the release artefacts and we're all good!
This avoids the need for any links as all the data is in one place.
Cheers,
Richard
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-04-25 9:55 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-04-18 4:58 QA cycle report for 2.5.3 RC3 Jain, Sangeeta
2019-04-23 14:31 ` richard.purdie
2019-04-24 6:28 ` Jain, Sangeeta
2019-04-25 9:55 ` richard.purdie
2019-04-23 14:35 ` richard.purdie
2019-04-23 15:01 ` [OE-core] " akuster808
2019-04-23 15:01 ` akuster808
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.