* ptest with BBCLASSEXTEND
@ 2019-05-16 10:54 Richard Purdie
2019-05-16 12:36 ` Richard Purdie
2019-05-17 16:22 ` Jonathan Rajotte-Julien
0 siblings, 2 replies; 11+ messages in thread
From: Richard Purdie @ 2019-05-16 10:54 UTC (permalink / raw)
To: openembedded-core
We have a need to check the dependencies of our ptest packages. The
best way to do that is to take a minimal image and run them under that
in isolation. That would be a bit tedious and boring.
It got me thinking about creative ways to do that and I thought I'd
share the result.
I created an image recipe:
$ cat p-t-image.bb
require recipes-core/images/core-image-minimal.bb
require conf/distro/include/ptest-packagelists.inc
PTESTS = "${PTESTS_SLOW} ${PTESTS_FAST}"
ALL_PTESTS = "${@' '.join(['p-t-image-'+x for x in d.getVar('PTESTS').split()])}"
IMAGE_INSTALL_append = " ${MCNAME} openssh"
BBCLASSEXTEND = "${@' '.join(['mcextend:'+x for x in d.getVar('PTESTS').split()])}"
IMAGE_OVERHEAD_FACTOR = "1.0"
IMAGE_ROOTFS_EXTRA_SPACE = "1324288"
# ptests need more memory than standard to avoid the OOM killer
QB_MEM = "-m 1024"
Its not fast to parse this creates about 59 variants of the image. Its
(ab)using the mcextend class to an image and add the ptest to that
image for each ptest in the PTESTS_SLOW and PTESTS_FAST variables. It
also adds openssh since we need an ssh client in the images for
testimage to work. I also had to add extra space and memory to the
image config so all the tests could work.
You can get a list of the target images with:
bitbake p-t-image-elfutils-ptest -e | grep ALL_PTESTS=
then run:
bitbake <targets>
and then
bitbake <targets> -c testimage
which batch build and then runs all the images in parallel.
The results also get aggregated automatically into the json file so if
you delete it before starting, the json file is equavalient from a
ptest perspective to if they'd all been run in the same image.
The results I got from this are:
--------------------------------------------------------------------------------------------------------------
Recipe | Passed | Failed | Skipped | Time(s)
--------------------------------------------------------------------------------------------------------------
acl | 1 | 1 | 0 | 0
apr | 0 | 0 | 0 | -
apr-util | 4 | 0 | 0 | -
attr | 0 | 0 | 0 | 0
bash | 36 | 8 | 0 | 107 T
bluez5 | 23 | 2 | 0 | 5
busybox | 528 | 4 | 76 | 4
bzip2 | 0 | 0 | 0 | 1
dbus-test | 0 | 0 | 0 | 91 T
diffstat | 439 | 0 | 0 | 4
diffutils | 19 | 0 | 2 | 2
e2fsprogs | 0 | 0 | 0 | 78
elfutils | 172 | 24 | 7 | 21
ethtool | 2 | 0 | 0 | 0
flex | 0 | 0 | 0 | 1
gawk | 342 | 0 | 0 | 2
gdbm | 24 | 0 | 0 | 1
gdk-pixbuf | 20 | 0 | 0 | 39
gettext | 0 | 0 | 0 | 1
glib-2.0 | 242 | 3 | 0 | 344
gstreamer1.0 | 103 | 1 | 0 | 105
gzip | 0 | 0 | 0 | 1
json-glib | 14 | 0 | 0 | 1
kbd | 5 | 2 | 0 | 0
libconvert-asn1-perl | 0 | 22 | 0 | 0
liberror-perl | 0 | 15 | 0 | 1
libevent | 1 | 0 | 0 | 86
libpcre | 0 | 0 | 0 | 0
libtest-needs-perl | 0 | 2 | 0 | 0
libtimedate-perl | 3 | 2 | 0 | 0
liburi-perl | 0 | 39 | 2 | 0
libusb1 | 4 | 0 | 0 | 6
libxml-namespacesupport-perl | 0 | 1 | 0 | 0
libxml-parser-perl | 0 | 15 | 0 | 0
libxml-perl | 0 | 6 | 0 | 0
libxml-sax-base-perl | 0 | 19 | 0 | 1
libxml-sax-perl | 0 | 14 | 0 | 0
libxml-simple-perl | 0 | 14 | 0 | 0
libxml2 | 134 | 0 | 0 | 16
lttng-tools | 3397 | 626 | 1 | 348 T
lzo | 5 | 0 | 0 | 2
m4 | 0 | 0 | 0 | 1
nettle | 97 | 0 | 2 | 10
openssh | 0 | 0 | 1 | 1
openssl | 146 | 1 | 8 | 78
opkg | 0 | 0 | 0 | 11
pango | 12 | 0 | 0 | 1
parted | 0 | 0 | 0 | 0
perl | 2423 | 19 | 238 | 703
python | 302 | 5 | 41 | 330
python3 | 30277 | 2 | 1043 | 1005
quilt | 0 | 57 | 0 | 1
sed | 86 | 0 | 0 | 4
slang | 0 | 0 | 0 | 1
strace | 466 | 18 | 147 | 179
tcl | 149 | 0 | 0 | 183
util-linux | 336 | 2 | 80 | 40
valgrind | 149 | 1 | 9 | 537
zlib | 1 | 0 | 0 | 1
which can be compared against:
https://autobuilder.yocto.io/pub/non-release/20190514-14/testresults/testresult-report.txt
which means we have a fair number of ptest recipes with dependency problems:
--------------------------------------------------------------------------------------------------------------
Recipe | Passed | Failed | Skipped | Time(s)
--------------------------------------------------------------------------------------------------------------
acl | 1 | 1 | 0 | 0
apr | 0 | 0 | 0 | -
apr-util | 4 | 0 | 0 | -
attr | 0 | 0 | 0 | 0
bash | 36 | 8 | 0 | 107 T
bluez5 | 23 | 2 | 0 | 5
bzip2 | 0 | 0 | 0 | 1
dbus-test | 0 | 0 | 0 | 91 T
diffutils | 19 | 0 | 2 | 2
e2fsprogs | 0 | 0 | 0 | 78
elfutils | 172 | 24 | 7 | 21
ethtool | 2 | 0 | 0 | 0
flex | 0 | 0 | 0 | 1
gettext | 0 | 0 | 0 | 1
glib-2.0 | 242 | 3 | 0 | 344
gzip | 0 | 0 | 0 | 1
kbd | 5 | 2 | 0 | 0
libconvert-asn1-perl | 0 | 22 | 0 | 0
liberror-perl | 0 | 15 | 0 | 1
libpcre | 0 | 0 | 0 | 0
libtest-needs-perl | 0 | 2 | 0 | 0
libtimedate-perl | 3 | 2 | 0 | 0
liburi-perl | 0 | 39 | 2 | 0
libxml-namespacesupport-perl | 0 | 1 | 0 | 0
libxml-parser-perl | 0 | 15 | 0 | 0
libxml-perl | 0 | 6 | 0 | 0
libxml-sax-base-perl | 0 | 19 | 0 | 1
libxml-sax-perl | 0 | 14 | 0 | 0
libxml-simple-perl | 0 | 14 | 0 | 0
lttng-tools | 3397 | 626 | 1 | 348 T
m4 | 0 | 0 | 0 | 1
openssh | 0 | 0 | 1 | 1
opkg | 0 | 0 | 0 | 11
parted | 0 | 0 | 0 | 0
python | 302 | 5 | 41 | 330
python3 | 30277 | 2 | 1043 | 1005
quilt | 0 | 57 | 0 | 1
slang | 0 | 0 | 0 | 1
util-linux | 336 | 2 | 80 | 40
The logs from this are available here:
https://www.rpsys.net/wp/rp/testresults.json.gz
(can be extracted with resulttool log)
Anyhow, I thought I'd share this slightly crazy but IMO fairly neat
trick :)
Cheers,
Richard
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: ptest with BBCLASSEXTEND
2019-05-16 10:54 ptest with BBCLASSEXTEND Richard Purdie
@ 2019-05-16 12:36 ` Richard Purdie
2019-05-16 21:13 ` Randy MacLeod
2019-05-17 16:22 ` Jonathan Rajotte-Julien
1 sibling, 1 reply; 11+ messages in thread
From: Richard Purdie @ 2019-05-16 12:36 UTC (permalink / raw)
To: openembedded-core
On Thu, 2019-05-16 at 11:54 +0100, Richard Purdie wrote:
> https://autobuilder.yocto.io/pub/non-release/20190514-14/testresults/testresult-report.txt
A couple of patches (on the list) gets us to the results below which is
better. The perl module failures and quilt are from missing perl module
dependencies.
--------------------------------------------------------------------------------------------------------------
Recipe | Passed | Failed | Skipped | Time(s)
--------------------------------------------------------------------------------------------------------------
acl | 3 | 0 | 0 | 0
apr | 44 | 0 | 0 | 58
apr-util | 19 | 0 | 0 | 8
attr | 0 | 0 | 0 | 0
bash | 36 | 8 | 0 | 108 T
bluez5 | 23 | 2 | 0 | 5
busybox | 528 | 4 | 76 | 4
bzip2 | 6 | 0 | 0 | 1
dbus-test | 0 | 0 | 0 | 91 T
diffstat | 439 | 0 | 0 | 3
diffutils | 19 | 0 | 2 | 2
e2fsprogs | 0 | 0 | 0 | 108 T
elfutils | 172 | 24 | 7 | 20
ethtool | 2 | 0 | 0 | 1
flex | 114 | 0 | 0 | 2
gawk | 342 | 0 | 0 | 2
gdbm | 24 | 0 | 0 | 2
gdk-pixbuf | 20 | 0 | 0 | 39
gettext | 354 | 21 | 37 | 33
glib-2.0 | 242 | 3 | 0 | 347
gstreamer1.0 | 103 | 1 | 0 | 104
gzip | 19 | 0 | 3 | 4
json-glib | 14 | 0 | 0 | 0
kbd | 5 | 2 | 0 | 1
libconvert-asn1-perl | 0 | 22 | 0 | 0
liberror-perl | 0 | 15 | 0 | 0
libevent | 1 | 0 | 0 | 86
libpcre | 0 | 0 | 0 | 1
libtest-needs-perl | 0 | 2 | 0 | 0
libtimedate-perl | 3 | 2 | 0 | 0
liburi-perl | 0 | 39 | 2 | 0
libusb1 | 4 | 0 | 0 | 6
libxml-namespacesupport-perl | 0 | 1 | 0 | 0
libxml-parser-perl | 0 | 15 | 0 | 0
libxml-perl | 0 | 6 | 0 | 0
libxml-sax-base-perl | 0 | 19 | 0 | 0
libxml-sax-perl | 0 | 14 | 0 | 0
libxml-simple-perl | 0 | 14 | 0 | 0
libxml2 | 134 | 0 | 0 | 17
lttng-tools | 3397 | 626 | 1 | 350 T
lzo | 5 | 0 | 0 | 2
m4 | 144 | 8 | 18 | 5
nettle | 97 | 0 | 2 | 9
openssh | 0 | 0 | 1 | 1
openssl | 146 | 1 | 8 | 91
opkg | 0 | 0 | 0 | 11
pango | 12 | 0 | 0 | 1
parted | 41 | 1 | 22 | 5
perl | 2423 | 19 | 238 | 713
python | 301 | 6 | 41 | 325
python3 | 30277 | 2 | 1043 | 990
quilt | 0 | 57 | 0 | 0
sed | 86 | 0 | 0 | 4
slang | 97 | 0 | 0 | 22
strace | 466 | 18 | 147 | 176
tcl | 149 | 0 | 0 | 179
util-linux | 336 | 2 | 80 | 41
valgrind | 149 | 1 | 9 | 544
zlib | 1 | 0 | 0 | 1
Cheers,
Richard
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: ptest with BBCLASSEXTEND
2019-05-16 12:36 ` Richard Purdie
@ 2019-05-16 21:13 ` Randy MacLeod
2019-05-16 22:28 ` richard.purdie
2019-05-17 1:58 ` Changqing Li
0 siblings, 2 replies; 11+ messages in thread
From: Randy MacLeod @ 2019-05-16 21:13 UTC (permalink / raw)
To: Richard Purdie, openembedded-core, Li, Changqing, Sakib Sajal
On 5/16/19 8:36 AM, Richard Purdie wrote:
> On Thu, 2019-05-16 at 11:54 +0100, Richard Purdie wrote:
>> https://autobuilder.yocto.io/pub/non-release/20190514-14/testresults/testresult-report.txt
>
> A couple of patches (on the list) gets us to the results below which is
> better. The perl module failures and quilt are from missing perl module
> dependencies.
>
> --------------------------------------------------------------------------------------------------------------
> Recipe | Passed | Failed | Skipped | Time(s)
> --------------------------------------------------------------------------------------------------------------
> acl | 3 | 0 | 0 | 0
> apr | 44 | 0 | 0 | 58
> apr-util | 19 | 0 | 0 | 8
> attr | 0 | 0 | 0 | 0
> bash | 36 | 8 | 0 | 108 T
> bluez5 | 23 | 2 | 0 | 5
> busybox | 528 | 4 | 76 | 4
> bzip2 | 6 | 0 | 0 | 1
> dbus-test | 0 | 0 | 0 | 91 T
> diffstat | 439 | 0 | 0 | 3
> diffutils | 19 | 0 | 2 | 2
> e2fsprogs | 0 | 0 | 0 | 108 T
> elfutils | 172 | 24 | 7 | 20
> ethtool | 2 | 0 | 0 | 1
> flex | 114 | 0 | 0 | 2
> gawk | 342 | 0 | 0 | 2
> gdbm | 24 | 0 | 0 | 2
> gdk-pixbuf | 20 | 0 | 0 | 39
> gettext | 354 | 21 | 37 | 33
> glib-2.0 | 242 | 3 | 0 | 347
> gstreamer1.0 | 103 | 1 | 0 | 104
> gzip | 19 | 0 | 3 | 4
> json-glib | 14 | 0 | 0 | 0
> kbd | 5 | 2 | 0 | 1
> libconvert-asn1-perl | 0 | 22 | 0 | 0
> liberror-perl | 0 | 15 | 0 | 0
> libevent | 1 | 0 | 0 | 86
> libpcre | 0 | 0 | 0 | 1
> libtest-needs-perl | 0 | 2 | 0 | 0
> libtimedate-perl | 3 | 2 | 0 | 0
> liburi-perl | 0 | 39 | 2 | 0
> libusb1 | 4 | 0 | 0 | 6
> libxml-namespacesupport-perl | 0 | 1 | 0 | 0
> libxml-parser-perl | 0 | 15 | 0 | 0
> libxml-perl | 0 | 6 | 0 | 0
> libxml-sax-base-perl | 0 | 19 | 0 | 0
> libxml-sax-perl | 0 | 14 | 0 | 0
> libxml-simple-perl | 0 | 14 | 0 | 0
> libxml2 | 134 | 0 | 0 | 17
> lttng-tools | 3397 | 626 | 1 | 350 T
> lzo | 5 | 0 | 0 | 2
> m4 | 144 | 8 | 18 | 5
> nettle | 97 | 0 | 2 | 9
> openssh | 0 | 0 | 1 | 1
> openssl | 146 | 1 | 8 | 91
> opkg | 0 | 0 | 0 | 11
> pango | 12 | 0 | 0 | 1
> parted | 41 | 1 | 22 | 5
> perl | 2423 | 19 | 238 | 713
> python | 301 | 6 | 41 | 325
> python3 | 30277 | 2 | 1043 | 990
> quilt | 0 | 57 | 0 | 0
> sed | 86 | 0 | 0 | 4
> slang | 97 | 0 | 0 | 22
> strace | 466 | 18 | 147 | 176
> tcl | 149 | 0 | 0 | 179
> util-linux | 336 | 2 | 80 | 41
> valgrind | 149 | 1 | 9 | 544
> zlib | 1 | 0 | 0 | 1
>
> Cheers,
>
> Richard
>
During the weekly YP bug review meeting, we discussed creating a
defect per package for each of these ptest failures and marking them
each as NEWCOMER bugs but we really need to keep working on ptest
improvements. I'd like Sandy (Changqing), Sajal and perhaps others
at Wind River (or elsewhere!) to split up the list above to fix
any missing dependencies.
Sajal is a new student working at working from Ottawa. He has some
experience with Yocto from a previous student placement. I've asked him
to work on the bash package ptest errors and I'll add quilt and strace
as well.
For the rest of the list, Sandy can pick a few and see if there's
a pattern to the failures or not. Once we have that info we can
decide how many people we need to deal with the rest of the
packages for fix most of the dependency issues over the coming
weeks. It would be nice to have most of them wrapped up for 2.8-M1.
--
# Randy MacLeod
# Wind River Linux
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: ptest with BBCLASSEXTEND
2019-05-16 21:13 ` Randy MacLeod
@ 2019-05-16 22:28 ` richard.purdie
2019-05-21 22:31 ` Richard Purdie
2019-05-22 22:28 ` Jonathan Rajotte-Julien
2019-05-17 1:58 ` Changqing Li
1 sibling, 2 replies; 11+ messages in thread
From: richard.purdie @ 2019-05-16 22:28 UTC (permalink / raw)
To: Randy MacLeod, openembedded-core, Li, Changqing, Sakib Sajal
On Thu, 2019-05-16 at 17:13 -0400, Randy MacLeod wrote:
> On 5/16/19 8:36 AM, Richard Purdie wrote:
> > On Thu, 2019-05-16 at 11:54 +0100, Richard Purdie wrote:
> > > https://autobuilder.yocto.io/pub/non-release/20190514-14/testresults/testresult-report.txt
> >
> > A couple of patches (on the list) gets us to the results below which is
> > better. The perl module failures and quilt are from missing perl module
> > dependencies.
> >
> > --------------------------------------------------------------------------------------------------------------
> > Recipe | Passed | Failed | Skipped | Time(s)
> > --------------------------------------------------------------------------------------------------------------
> > acl | 3 | 0 | 0 | 0
> > apr | 44 | 0 | 0 | 58
> > apr-util | 19 | 0 | 0 | 8
> > attr | 0 | 0 | 0 | 0
> > bash | 36 | 8 | 0 | 108 T
> > bluez5 | 23 | 2 | 0 | 5
> > busybox | 528 | 4 | 76 | 4
> > bzip2 | 6 | 0 | 0 | 1
> > dbus-test | 0 | 0 | 0 | 91 T
> > diffstat | 439 | 0 | 0 | 3
> > diffutils | 19 | 0 | 2 | 2
> > e2fsprogs | 0 | 0 | 0 | 108 T
> > elfutils | 172 | 24 | 7 | 20
> > ethtool | 2 | 0 | 0 | 1
> > flex | 114 | 0 | 0 | 2
> > gawk | 342 | 0 | 0 | 2
> > gdbm | 24 | 0 | 0 | 2
> > gdk-pixbuf | 20 | 0 | 0 | 39
> > gettext | 354 | 21 | 37 | 33
> > glib-2.0 | 242 | 3 | 0 | 347
> > gstreamer1.0 | 103 | 1 | 0 | 104
> > gzip | 19 | 0 | 3 | 4
> > json-glib | 14 | 0 | 0 | 0
> > kbd | 5 | 2 | 0 | 1
> > libconvert-asn1-perl | 0 | 22 | 0 | 0
> > liberror-perl | 0 | 15 | 0 | 0
> > libevent | 1 | 0 | 0 | 86
> > libpcre | 0 | 0 | 0 | 1
> > libtest-needs-perl | 0 | 2 | 0 | 0
> > libtimedate-perl | 3 | 2 | 0 | 0
> > liburi-perl | 0 | 39 | 2 | 0
> > libusb1 | 4 | 0 | 0 | 6
> > libxml-namespacesupport-perl | 0 | 1 | 0 | 0
> > libxml-parser-perl | 0 | 15 | 0 | 0
> > libxml-perl | 0 | 6 | 0 | 0
> > libxml-sax-base-perl | 0 | 19 | 0 | 0
> > libxml-sax-perl | 0 | 14 | 0 | 0
> > libxml-simple-perl | 0 | 14 | 0 | 0
> > libxml2 | 134 | 0 | 0 | 17
> > lttng-tools | 3397 | 626 | 1 | 350 T
> > lzo | 5 | 0 | 0 | 2
> > m4 | 144 | 8 | 18 | 5
> > nettle | 97 | 0 | 2 | 9
> > openssh | 0 | 0 | 1 | 1
> > openssl | 146 | 1 | 8 | 91
> > opkg | 0 | 0 | 0 | 11
> > pango | 12 | 0 | 0 | 1
> > parted | 41 | 1 | 22 | 5
> > perl | 2423 | 19 | 238 | 713
> > python | 301 | 6 | 41 | 325
> > python3 | 30277 | 2 | 1043 | 990
> > quilt | 0 | 57 | 0 | 0
> > sed | 86 | 0 | 0 | 4
> > slang | 97 | 0 | 0 | 22
> > strace | 466 | 18 | 147 | 176
> > tcl | 149 | 0 | 0 | 179
> > util-linux | 336 | 2 | 80 | 41
> > valgrind | 149 | 1 | 9 | 544
> > zlib | 1 | 0 | 0 | 1
> >
> > Cheers,
> >
> > Richard
> >
>
> During the weekly YP bug review meeting, we discussed creating a
> defect per package for each of these ptest failures and marking them
> each as NEWCOMER bugs but we really need to keep working on ptest
> improvements. I'd like Sandy (Changqing), Sajal and perhaps others
> at Wind River (or elsewhere!) to split up the list above to fix
> any missing dependencies.
>
> Sajal is a new student working at working from Ottawa. He has some
> experience with Yocto from a previous student placement. I've asked him
> to work on the bash package ptest errors and I'll add quilt and strace
> as well.
>
> For the rest of the list, Sandy can pick a few and see if there's
> a pattern to the failures or not. Once we have that info we can
> decide how many people we need to deal with the rest of the
> packages for fix most of the dependency issues over the coming
> weeks. It would be nice to have most of them wrapped up for 2.8-M1.
Thanks Randy!
Just to update from my perspective:
quilt and lib*-perl all look like missing perl module dependencies. The
place those dependencies are missing may be for the ptest or may be the
actual PN package itself, depending on where the module dependency is.
Ross looks like he's looking at the glib-2.0 issues.
I think Jonathan Rajotte might be able to help us with lttng-tools.
I have a local patch for acl which looks like a missing e2fsprogs-
mke2fs and make dependencies but I've not tested it yet.
The failures with timeouts (T next to them) are the ones which worry me
most of all.
Cheers,
Richard
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: ptest with BBCLASSEXTEND
2019-05-16 21:13 ` Randy MacLeod
2019-05-16 22:28 ` richard.purdie
@ 2019-05-17 1:58 ` Changqing Li
1 sibling, 0 replies; 11+ messages in thread
From: Changqing Li @ 2019-05-17 1:58 UTC (permalink / raw)
To: Randy MacLeod, Richard Purdie, openembedded-core, Sakib Sajal
On 5/17/19 5:13 AM, Randy MacLeod wrote:
> On 5/16/19 8:36 AM, Richard Purdie wrote:
>> On Thu, 2019-05-16 at 11:54 +0100, Richard Purdie wrote:
>>> https://autobuilder.yocto.io/pub/non-release/20190514-14/testresults/testresult-report.txt
>>>
>>
>> A couple of patches (on the list) gets us to the results below which is
>> better. The perl module failures and quilt are from missing perl module
>> dependencies.
>>
>> --------------------------------------------------------------------------------------------------------------
>>
>> Recipe | Passed | Failed | Skipped |
>> Time(s)
>> --------------------------------------------------------------------------------------------------------------
>>
>> acl | 3 | 0 | 0 | 0
>> apr | 44 | 0 | 0 | 58
>> apr-util | 19 | 0 | 0 | 8
>> attr | 0 | 0 | 0 | 0
>> bash | 36 | 8 | 0 |
>> 108 T
>> bluez5 | 23 | 2 | 0 | 5
>> busybox | 528 | 4 | 76 | 4
>> bzip2 | 6 | 0 | 0 | 1
>> dbus-test | 0 | 0 | 0 |
>> 91 T
>> diffstat | 439 | 0 | 0 | 3
>> diffutils | 19 | 0 | 2 | 2
>> e2fsprogs | 0 | 0 | 0 |
>> 108 T
>> elfutils | 172 | 24 | 7 | 20
>> ethtool | 2 | 0 | 0 | 1
>> flex | 114 | 0 | 0 | 2
>> gawk | 342 | 0 | 0 | 2
>> gdbm | 24 | 0 | 0 | 2
>> gdk-pixbuf | 20 | 0 | 0 | 39
>> gettext | 354 | 21 | 37 | 33
>> glib-2.0 | 242 | 3 | 0 | 347
>> gstreamer1.0 | 103 | 1 | 0 | 104
>> gzip | 19 | 0 | 3 | 4
>> json-glib | 14 | 0 | 0 | 0
>> kbd | 5 | 2 | 0 | 1
>> libconvert-asn1-perl | 0 | 22 | 0 | 0
>> liberror-perl | 0 | 15 | 0 | 0
>> libevent | 1 | 0 | 0 | 86
>> libpcre | 0 | 0 | 0 | 1
>> libtest-needs-perl | 0 | 2 | 0 | 0
>> libtimedate-perl | 3 | 2 | 0 | 0
>> liburi-perl | 0 | 39 | 2 | 0
>> libusb1 | 4 | 0 | 0 | 6
>> libxml-namespacesupport-perl | 0 | 1 | 0 | 0
>> libxml-parser-perl | 0 | 15 | 0 | 0
>> libxml-perl | 0 | 6 | 0 | 0
>> libxml-sax-base-perl | 0 | 19 | 0 | 0
>> libxml-sax-perl | 0 | 14 | 0 | 0
>> libxml-simple-perl | 0 | 14 | 0 | 0
>> libxml2 | 134 | 0 | 0 | 17
>> lttng-tools | 3397 | 626 | 1 |
>> 350 T
>> lzo | 5 | 0 | 0 | 2
>> m4 | 144 | 8 | 18 | 5
>> nettle | 97 | 0 | 2 | 9
>> openssh | 0 | 0 | 1 | 1
>> openssl | 146 | 1 | 8 | 91
>> opkg | 0 | 0 | 0 | 11
>> pango | 12 | 0 | 0 | 1
>> parted | 41 | 1 | 22 | 5
>> perl | 2423 | 19 | 238 | 713
>> python | 301 | 6 | 41 | 325
>> python3 | 30277 | 2 | 1043 | 990
>> quilt | 0 | 57 | 0 | 0
>> sed | 86 | 0 | 0 | 4
>> slang | 97 | 0 | 0 | 22
>> strace | 466 | 18 | 147 | 176
>> tcl | 149 | 0 | 0 | 179
>> util-linux | 336 | 2 | 80 | 41
>> valgrind | 149 | 1 | 9 | 544
>> zlib | 1 | 0 | 0 | 1
>>
>> Cheers,
>>
>> Richard
>>
>
> During the weekly YP bug review meeting, we discussed creating a
> defect per package for each of these ptest failures and marking them
> each as NEWCOMER bugs but we really need to keep working on ptest
> improvements. I'd like Sandy (Changqing), Sajal and perhaps others
> at Wind River (or elsewhere!) to split up the list above to fix
> any missing dependencies.
>
> Sajal is a new student working at working from Ottawa. He has some
> experience with Yocto from a previous student placement. I've asked him
> to work on the bash package ptest errors and I'll add quilt and strace
> as well.
>
> For the rest of the list, Sandy can pick a few and see if there's
> a pattern to the failures or not. Once we have that info we can
> decide how many people we need to deal with the rest of the
> packages for fix most of the dependency issues over the coming
> weeks. It would be nice to have most of them wrapped up for 2.8-M1.
>
OK, I will work on this after completing current one.
--
BRs
Sandy(Li Changqing)
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: ptest with BBCLASSEXTEND
2019-05-16 10:54 ptest with BBCLASSEXTEND Richard Purdie
2019-05-16 12:36 ` Richard Purdie
@ 2019-05-17 16:22 ` Jonathan Rajotte-Julien
1 sibling, 0 replies; 11+ messages in thread
From: Jonathan Rajotte-Julien @ 2019-05-17 16:22 UTC (permalink / raw)
To: Richard Purdie; +Cc: openembedded-core
> Its not fast to parse this creates about 59 variants of the image. Its
> (ab)using the mcextend class to an image and add the ptest to that
> image for each ptest in the PTESTS_SLOW and PTESTS_FAST variables. It
> also adds openssh since we need an ssh client in the images for
> testimage to work. I also had to add extra space and memory to the
> image config so all the tests could work.
>
> You can get a list of the target images with:
>
For the non-initiated, you will need to have the following in your local.conf
for this to work.
DISTRO_FEATURES_append = " ptest"
That said, I should be able to take a look at the problem related to
lttng-tools.
Thanks
--
Jonathan Rajotte-Julien
EfficiOS
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: ptest with BBCLASSEXTEND
2019-05-16 22:28 ` richard.purdie
@ 2019-05-21 22:31 ` Richard Purdie
2019-06-03 3:10 ` Changqing Li
2019-05-22 22:28 ` Jonathan Rajotte-Julien
1 sibling, 1 reply; 11+ messages in thread
From: Richard Purdie @ 2019-05-21 22:31 UTC (permalink / raw)
To: Randy MacLeod, openembedded-core, Li, Changqing, Sakib Sajal
On Thu, 2019-05-16 at 23:28 +0100, richard.purdie@linuxfoundation.org
wrote:
> Just to update from my perspective:
>
> quilt and lib*-perl all look like missing perl module dependencies.
> The place those dependencies are missing may be for the ptest or may
> be the actual PN package itself, depending on where the module
> dependency is.
I've sent out a further patch series which resolves some of the issues.
Some quilt tests now pass, some still fail. The perl modules fail in
different ways to before, there are clearly more missing dependencies
to be added.
A quick summary of roughly the recipes still with problems (not 100%
accurately filtered):
==============================================================================================================
PTest Result Summary
==============================================================================================================
--------------------------------------------------------------------------------------------------------------
Recipe | Passed | Failed | Skipped | Time(s)
--------------------------------------------------------------------------------------------------------------
bash | 41 | 3 | 0 | 107 T
bluez5 | 23 | 2 | 0 | 5
dbus-test | 0 | 0 | 0 | 91 T
diffutils | 19 | 0 | 2 | 1
e2fsprogs | 0 | 0 | 0 | 108 T
elfutils | 172 | 24 | 7 | 19
gettext | 354 | 21 | 37 | 33
glib-2.0 | 245 | 0 | 0 | 367
gstreamer1.0 | 103 | 1 | 0 | 103
gzip | 19 | 0 | 3 | 3
kbd | 5 | 2 | 0 | 0
libconvert-asn1-perl | 0 | 22 | 0 | 0
liberror-perl | 2 | 13 | 0 | 0
libpcre | 3 | 0 | 0 | 2
libtest-needs-perl | 0 | 2 | 0 | 0
libtimedate-perl | 3 | 2 | 0 | 0
liburi-perl | 0 | 39 | 2 | 0
libxml-namespacesupport-perl | 0 | 1 | 0 | 0
libxml-parser-perl | 0 | 15 | 0 | 0
libxml-perl | 0 | 6 | 0 | 0
libxml-sax-base-perl | 2 | 17 | 0 | 1
libxml-sax-perl | 0 | 14 | 0 | 0
libxml-simple-perl | 0 | 14 | 0 | 0
lttng-tools | 3397 | 626 | 1 | 350 T
m4 | 144 | 8 | 18 | 5
nettle | 97 | 0 | 2 | 8
openssh | 15 | 1 | 1 | 138
openssl | 146 | 1 | 8 | 87
opkg | 0 | 0 | 0 | 9
parted | 41 | 1 | 22 | 4
perl | 2423 | 19 | 238 | 727
python | 301 | 6 | 41 | 325
python3 | 30277 | 2 | 1043 | 1005
quilt | 10 | 47 | 0 | 24
tcl | 149 | 0 | 0 | 177
valgrind | 149 | 1 | 9 | 552
--------------------------------------------------------------------------------------------------------------
Cheers,
Richard
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: ptest with BBCLASSEXTEND
2019-05-16 22:28 ` richard.purdie
2019-05-21 22:31 ` Richard Purdie
@ 2019-05-22 22:28 ` Jonathan Rajotte-Julien
2019-05-22 22:33 ` richard.purdie
1 sibling, 1 reply; 11+ messages in thread
From: Jonathan Rajotte-Julien @ 2019-05-22 22:28 UTC (permalink / raw)
To: richard.purdie; +Cc: openembedded-core
Hi,
> I think Jonathan Rajotte might be able to help us with lttng-tools.
Finally had some time for this.
See [1][2].
[2] does bring the matter of forcing lttng-ust support of lttng-tools when ptest
is deployed. "lttng-ust" is already set in PACKAGECONFIG by default but is there any
way to "enforce" the lttng-ust config & dependency when lttng-tools-ptest is used?
[1] http://lists.openembedded.org/pipermail/openembedded-core/2019-May/282733.html
[2] http://lists.openembedded.org/pipermail/openembedded-core/2019-May/282734.html
Cheers
--
Jonathan Rajotte-Julien
EfficiOS
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: ptest with BBCLASSEXTEND
2019-05-22 22:28 ` Jonathan Rajotte-Julien
@ 2019-05-22 22:33 ` richard.purdie
0 siblings, 0 replies; 11+ messages in thread
From: richard.purdie @ 2019-05-22 22:33 UTC (permalink / raw)
To: Jonathan Rajotte-Julien; +Cc: openembedded-core
On Wed, 2019-05-22 at 18:28 -0400, Jonathan Rajotte-Julien wrote:
> Hi,
>
> > I think Jonathan Rajotte might be able to help us with lttng-tools.
>
> Finally had some time for this.
>
> See [1][2].
>
> [2] does bring the matter of forcing lttng-ust support of lttng-tools
> when ptest
> is deployed. "lttng-ust" is already set in PACKAGECONFIG by default
> but is there any
> way to "enforce" the lttng-ust config & dependency when lttng-tools-
> ptest is used?
>
> [1] http://lists.openembedded.org/pipermail/openembedded-core/2019-May/282733.html
> [2] http://lists.openembedded.org/pipermail/openembedded-core/2019-May/282734.html
Thanks for looking at this. It sounds like we need some runtime
dependencies as well as the PACKAGECONFIG option. I think the lttng-
tool ust option should be changed:
-PACKAGECONFIG[lttng-ust] = "--with-lttng-ust, --without-lttng-ust, lttng-ust"
+PACKAGECONFIG[lttng-ust] = "--with-lttng-ust, --without-lttng-ust, lttng-ust,lttng-ust"
i.e. force the dependency at runtime as well as buildtime.
Cheers,
Richard
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: ptest with BBCLASSEXTEND
2019-05-21 22:31 ` Richard Purdie
@ 2019-06-03 3:10 ` Changqing Li
2019-06-03 13:02 ` richard.purdie
0 siblings, 1 reply; 11+ messages in thread
From: Changqing Li @ 2019-06-03 3:10 UTC (permalink / raw)
To: Richard Purdie, Randy MacLeod, openembedded-core, Sakib Sajal
On 5/22/19 6:31 AM, Richard Purdie wrote:
> On Thu, 2019-05-16 at 23:28 +0100, richard.purdie@linuxfoundation.org
> wrote:
>> Just to update from my perspective:
>>
>> quilt and lib*-perl all look like missing perl module dependencies.
>> The place those dependencies are missing may be for the ptest or may
>> be the actual PN package itself, depending on where the module
>> dependency is.
> I've sent out a further patch series which resolves some of the issues.
> Some quilt tests now pass, some still fail. The perl modules fail in
> different ways to before, there are clearly more missing dependencies
> to be added.
>
> A quick summary of roughly the recipes still with problems (not 100%
> accurately filtered):
>
> ==============================================================================================================
> PTest Result Summary
> ==============================================================================================================
> --------------------------------------------------------------------------------------------------------------
> Recipe | Passed | Failed | Skipped | Time(s)
> --------------------------------------------------------------------------------------------------------------
> bash | 41 | 3 | 0 | 107 T
> bluez5 | 23 | 2 | 0 | 5
> dbus-test | 0 | 0 | 0 | 91 T
> diffutils | 19 | 0 | 2 | 1
> e2fsprogs | 0 | 0 | 0 | 108 T
> elfutils | 172 | 24 | 7 | 19
> gettext | 354 | 21 | 37 | 33
> glib-2.0 | 245 | 0 | 0 | 367
> gstreamer1.0 | 103 | 1 | 0 | 103
> gzip | 19 | 0 | 3 | 3
> kbd | 5 | 2 | 0 | 0
> libconvert-asn1-perl | 0 | 22 | 0 | 0
> liberror-perl | 2 | 13 | 0 | 0
> libpcre | 3 | 0 | 0 | 2
> libtest-needs-perl | 0 | 2 | 0 | 0
> libtimedate-perl | 3 | 2 | 0 | 0
> liburi-perl | 0 | 39 | 2 | 0
> libxml-namespacesupport-perl | 0 | 1 | 0 | 0
> libxml-parser-perl | 0 | 15 | 0 | 0
> libxml-perl | 0 | 6 | 0 | 0
> libxml-sax-base-perl | 2 | 17 | 0 | 1
> libxml-sax-perl | 0 | 14 | 0 | 0
> libxml-simple-perl | 0 | 14 | 0 | 0
> lttng-tools | 3397 | 626 | 1 | 350 T
> m4 | 144 | 8 | 18 | 5
> nettle | 97 | 0 | 2 | 8
> openssh | 15 | 1 | 1 | 138
> openssl | 146 | 1 | 8 | 87
> opkg | 0 | 0 | 0 | 9
> parted | 41 | 1 | 22 | 4
> perl | 2423 | 19 | 238 | 727
> python | 301 | 6 | 41 | 325
> python3 | 30277 | 2 | 1043 | 1005
> quilt | 10 | 47 | 0 | 24
> tcl | 149 | 0 | 0 | 177
> valgrind | 149 | 1 | 9 | 552
> --------------------------------------------------------------------------------------------------------------
>
> Cheers,
>
> Richard
>
>
Randy,
Based on this list, except lib*perl and packages that are dealing with
by others,
I picked some for your reference to evaluate the workload. If more info
is needed,
feel free to tell me.
diffutils depend on perl
elfutils: mingli have work on it and send patch to oe-core mail list
gettext: depend on certain locales for some tests
depend on autoconf for some test
gzip: have tests depend on diff, while diff in busybox
incompatible with POSIX
have tests depend on grep, while grep in busybox don't
support context options
have tests depend on perl
kbd: have tests depend on readlink, busybox's readlink don't
support -e option
m4: one test depend on perl, some test skip since depend on
certain locale/codeset, need to check one by one
--
BRs
Sandy(Li Changqing)
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: ptest with BBCLASSEXTEND
2019-06-03 3:10 ` Changqing Li
@ 2019-06-03 13:02 ` richard.purdie
0 siblings, 0 replies; 11+ messages in thread
From: richard.purdie @ 2019-06-03 13:02 UTC (permalink / raw)
To: Changqing Li, Randy MacLeod, openembedded-core, Sakib Sajal
On Mon, 2019-06-03 at 11:10 +0800, Changqing Li wrote:
> Based on this list, except lib*perl and packages that are dealing
> with by others,
>
> I picked some for your reference to evaluate the workload. If more
> info is needed,
>
> feel free to tell me.
If it helps, I reran the minimal image comparison and found the
following issues:
Recipes where we have problems (differences) in minimal images:
bash | 76 | 5 | 0 | 163
diffutils | 19 | 0 | 2 | 1
e2fsprogs | 0 | 0 | 0 | 111
elfutils | 194 | 0 | 9 | 32
gettext | 354 | 21 | 37 | 32
gzip | 19 | 0 | 3 | 3
kbd | 5 | 2 | 0 | 0
m4 | 145 | 7 | 18 | 5
openssh | 15 | 1 | 1 | 145
parted | 41 | 1 | 22 | 5
perl | 2413 | 18 | 245 | 861
python | 301 | 6 | 41 | 327
python3 | 30277 | 2 | 1043 | 1012
Others:
acl tests are timing out?
gawk shows failures in sdk image but minimal:
gawk | 407 | 24 | 0 | 20
gstreamer shows two fewer failures in minimal image:
gstreamer1.0 | 104 | 1 | 0 | 104
(I'm using
https://autobuilder.yocto.io/pub/non-release/20190601-2/testresults/testresult-report.txt
for comparison)
There are also some open ptest bugs in bugzilla which I didn't
duplicate here.
The above list is much smaller than the original particularly now we
have the perl module issues resolved.
Cheers,
Richard
> diffutils depend on perl
> elfutils: mingli have work on it and send patch to oe-core mail
> list
> gettext: depend on certain locales for some tests
>
> depend on autoconf for some test
> gzip: have tests depend on diff, while diff in busybox
> incompatible with POSIX
> have tests depend on grep, while grep in busybox
> don't
> support context options
> have tests depend on perl
> kbd: have tests depend on readlink, busybox's readlink don't
> support -e option
> m4: one test depend on perl, some test skip since depend on
> certain locale/codeset, need to check one by one
>
>
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2019-06-03 13:02 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-16 10:54 ptest with BBCLASSEXTEND Richard Purdie
2019-05-16 12:36 ` Richard Purdie
2019-05-16 21:13 ` Randy MacLeod
2019-05-16 22:28 ` richard.purdie
2019-05-21 22:31 ` Richard Purdie
2019-06-03 3:10 ` Changqing Li
2019-06-03 13:02 ` richard.purdie
2019-05-22 22:28 ` Jonathan Rajotte-Julien
2019-05-22 22:33 ` richard.purdie
2019-05-17 1:58 ` Changqing Li
2019-05-17 16:22 ` Jonathan Rajotte-Julien
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.