[PATCH 1/2] lttng-tools ptest: add missing dependencies

[PATCH 1/2] lttng-tools ptest: add missing dependencies

[Jonathan Rajotte]

Multiple tests are failing due to missing dependencies on a bare
core-image-minimal build with only lttng-tools ptest present.

"getconf LONG_BIT" is used to get the bitness of the host to run the
correct consumerd. Depend on glibc-utils.

perl-module-overloading is needed to validate trace. babelstats.pl

"head -c" is used to generate a random string. It is easier to depends
on coreutils to get a feature complete "head" than to enable to "fancy
head" feature of busybox. See utils.sh randstring.

"taskset" is used in a couple of scripts to ensure that events generated
finish in the same buffer. Depend on util-linux.

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
---

I also submitted the following patches [1] [2] to lttng upstream. It
should help get the ptest sucess ratio up when they are merged. They
should be released in the upcoming month.

[1] https://lists.lttng.org/pipermail/lttng-dev/2019-March/028573.html
[2] https://lists.lttng.org/pipermail/lttng-dev/2019-March/028574.html

---
 meta/recipes-kernel/lttng/lttng-tools_2.10.6.bb | 11 +----------
 1 file changed, 1 insertion(+), 10 deletions(-)

diff --git a/meta/recipes-kernel/lttng/lttng-tools_2.10.6.bb b/meta/recipes-kernel/lttng/lttng-tools_2.10.6.bb
index 9d21fb00bf..3324959cc3 100644
--- a/meta/recipes-kernel/lttng/lttng-tools_2.10.6.bb
+++ b/meta/recipes-kernel/lttng/lttng-tools_2.10.6.bb
@@ -11,7 +11,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=01d7fc4496aacf37d90df90b90b0cac1 \
 
 DEPENDS = "liburcu popt libxml2 util-linux"
 RDEPENDS_${PN} = "libgcc"
-RDEPENDS_${PN}-ptest += "make perl bash gawk ${PN} babeltrace procps"
+RDEPENDS_${PN}-ptest += "make perl bash gawk ${PN} babeltrace procps glibc-utils perl-module-overloading coreutils util-linux"
 # babelstats.pl wants getopt-long
 RDEPENDS_${PN}-ptest += "perl-module-getopt-long"
 
@@ -141,15 +141,6 @@ do_install_ptest () {
         -e 's#\(^test.*LDADD.=\)#disable\1#g' \
         -i ${D}${PTEST_PATH}/tests/unit/Makefile
 
-    #
-    # Disable notification tools tests as currently
-    # these hang and cause the rest of the ptests to timeout
-    #
-    sed -e 's#tools/notification/test_notification_ust##g' \
-        -e 's#tools/notification/test_notification_kernel##g' \
-        -e 's#tools/notification/test_notification_multi_app##g' \
-        -i ${D}${PTEST_PATH}/tests/regression/Makefile
-
     # Substitute links to installed binaries.
     for prog in lttng lttng-relayd lttng-sessiond lttng-consumerd lttng-crash; do
         exedir="${D}${PTEST_PATH}/src/bin/${prog}"
-- 
2.17.1

[PATCH 2/2] lttng-tools ptest: fix test_ust-dl

[Jonathan Rajotte]

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
---
 .../lttng/lttng-tools_2.10.6.bb               | 21 ++++++++++---------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/meta/recipes-kernel/lttng/lttng-tools_2.10.6.bb b/meta/recipes-kernel/lttng/lttng-tools_2.10.6.bb
index 3324959cc3..37b6161cda 100644
--- a/meta/recipes-kernel/lttng/lttng-tools_2.10.6.bb
+++ b/meta/recipes-kernel/lttng/lttng-tools_2.10.6.bb
@@ -87,6 +87,9 @@ do_install_ptest () {
         install -d "${D}${PTEST_PATH}/tests/$d"
         find "${B}/tests/$d" -maxdepth 1 -executable -type f \
             -exec install -t "${D}${PTEST_PATH}/tests/$d" {} +
+        # Take all .py scripts for tests using the python bindings.
+        find "${B}/tests/$d" -maxdepth 1 -type f -name "*.py" \
+            -exec install -t "${D}${PTEST_PATH}/tests/$d" {} +
         test -r "${B}/tests/$d/Makefile" && \
             install -t "${D}${PTEST_PATH}/tests/$d" "${B}/tests/$d/Makefile"
     done
@@ -98,11 +101,19 @@ do_install_ptest () {
                 *.so)
                     install -d ${D}${PTEST_PATH}/tests/$d/
                     ln -s  ../$f ${D}${PTEST_PATH}/tests/$d/$f
+                    # Remove any rpath/runpath to pass QA check.
+                    chrpath --delete ${D}${PTEST_PATH}/tests/$d/$f
                     ;;
             esac
         done
     done
 
+    #
+    # Use the versioned libs of liblttng-ust-dl.
+    #
+    sed -i -e 's!:liblttng-ust-dl.so!:liblttng-ust-dl.so.0!' \
+        "${D}${PTEST_PATH}/tests/regression/ust/ust-dl/test_ust-dl.py"
+
     install ${B}/tests/unit/ini_config/sample.ini ${D}${PTEST_PATH}/tests/unit/ini_config/
 
     # We shouldn't need to build anything in tests/regression/tools
@@ -119,16 +130,6 @@ do_install_ptest () {
         -e 's/^all-am:.*/all-am:/' \
         {} +
 
-    # These objects trigger [rpaths] QA checks; the test harness
-    # skips the associated tests if they're missing, so delete
-    # them.
-    objs=""
-    objs="$objs regression/ust/ust-dl/libbar.so"
-    objs="$objs regression/ust/ust-dl/libfoo.so"
-    for obj in $objs ; do
-        rm -f "${D}${PTEST_PATH}/tests/${obj}"
-    done
-
     find "${D}${PTEST_PATH}" -name Makefile -type f -exec \
         touch -r "${B}/Makefile" {} +
 
-- 
2.17.1

Re: [PATCH 1/2] lttng-tools ptest: add missing dependencies

[Richard Purdie]

On Tue, 2019-03-12 at 21:13 +0000, Jonathan Rajotte wrote:
> Multiple tests are failing due to missing dependencies on a bare
> core-image-minimal build with only lttng-tools ptest present.
> 
> "getconf LONG_BIT" is used to get the bitness of the host to run the
> correct consumerd. Depend on glibc-utils.
> 
> perl-module-overloading is needed to validate trace. babelstats.pl
> 
> "head -c" is used to generate a random string. It is easier to
> depends
> on coreutils to get a feature complete "head" than to enable to
> "fancy
> head" feature of busybox. See utils.sh randstring.
> 
> "taskset" is used in a couple of scripts to ensure that events
> generated
> finish in the same buffer. Depend on util-linux.
> 
> Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com
> >
> ---
> 
> I also submitted the following patches [1] [2] to lttng upstream. It
> should help get the ptest sucess ratio up when they are merged. They
> should be released in the upcoming month.
> 
> [1] 
> https://lists.lttng.org/pipermail/lttng-dev/2019-March/028573.html
> [2] 
> https://lists.lttng.org/pipermail/lttng-dev/2019-March/028574.html
> 
> ---
>  meta/recipes-kernel/lttng/lttng-tools_2.10.6.bb | 11 +----------
>  1 file changed, 1 insertion(+), 10 deletions(-)
> 
> diff --git a/meta/recipes-kernel/lttng/lttng-tools_2.10.6.bb b/meta/recipes-kernel/lttng/lttng-tools_2.10.6.bb
> index 9d21fb00bf..3324959cc3 100644
> --- a/meta/recipes-kernel/lttng/lttng-tools_2.10.6.bb
> +++ b/meta/recipes-kernel/lttng/lttng-tools_2.10.6.bb
> @@ -11,7 +11,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=01d7fc4496aacf37d90df90b90b0cac1 \
>  
>  DEPENDS = "liburcu popt libxml2 util-linux"
>  RDEPENDS_${PN} = "libgcc"
> -RDEPENDS_${PN}-ptest += "make perl bash gawk ${PN} babeltrace procps"
> +RDEPENDS_${PN}-ptest += "make perl bash gawk ${PN} babeltrace procps glibc-utils perl-module-overloading coreutils util-linux"
>  # babelstats.pl wants getopt-long
>  RDEPENDS_${PN}-ptest += "perl-module-getopt-long"

Firstly, I wanted to say thanks for looking at this, there are some
really great fixes in here.

It failed in automated testing as glibc-utils is glibc specific and
musl builds couldn't cope with that.

We can certainly do:

RDEPENDS_${PN}-ptest_append_libc-glibc = " glibc-utils"

but I'm not sure what that would mean for lttng on musl or if lttng
ptest on musl even makes sense.

I'm travelling so haven't been able to dig any further so sorry for
being terse but just wanted to give a heads up on this.

Autobuilder failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/64/builds/396

Cheers,

Richard

Re: [PATCH 1/2] lttng-tools ptest: add missing dependencies

[Jonathan Rajotte-Julien]

> Firstly, I wanted to say thanks for looking at this, there are some
> really great fixes in here.
> 
> It failed in automated testing as glibc-utils is glibc specific and
> musl builds couldn't cope with that.
> 
> We can certainly do:
> 
> RDEPENDS_${PN}-ptest_append_libc-glibc = " glibc-utils"

Sure, since we mostly need getconf from glibc-utils, we can use musl-utils
to provide it when building with musl.

RDEPENDS_${PN}-ptest_append_libc-musl = " musl-utils"

Also, I saw that lttng-ust is removed from PACKAGECONFIG when using musl, not
sure why the commit history is not particularly verbose regarding why it was a
problem. Still, it seems to build fine but we are seeing issue regarding musl.
We checked with Alpine since they also use musl and see similar problem. I'm
investigating on that front. I'll let you know what I find.

Side note, gdb simply segfault (at start) for me on a core-image-minirmnal build using musl.
How would someone open the corefile generated on the build system with the
proper libs and all?

Thanks!

-- 
Jonathan Rajotte-Julien
EfficiOS

Re: [PATCH 1/2] lttng-tools ptest: add missing dependencies

[Richard Purdie]

On Fri, 2019-03-15 at 10:40 -0400, Jonathan Rajotte-Julien wrote:
> > Firstly, I wanted to say thanks for looking at this, there are some
> > really great fixes in here.
> > 
> > It failed in automated testing as glibc-utils is glibc specific and
> > musl builds couldn't cope with that.
> > 
> > We can certainly do:
> > 
> > RDEPENDS_${PN}-ptest_append_libc-glibc = " glibc-utils"
> 
> Sure, since we mostly need getconf from glibc-utils, we can use musl-
> utils to provide it when building with musl.
> 
> RDEPENDS_${PN}-ptest_append_libc-musl = " musl-utils"

Thanks for confirmation of that.

Since the patches with a couple of fixes were passing the tests, I've
merged the patches with the glibc/muslc tweak and also a tweak to make
disabled ust work. I'd normally wait for v2 resubmission but I wanted
to get this fix into M3 and the M3 build is already behind schedule.

I was at a conference last week so a bit distracted, just catching up
now.

> Also, I saw that lttng-ust is removed from PACKAGECONFIG when using
> musl, not sure why the commit history is not particularly verbose
> regarding why it was a problem. Still, it seems to build fine but we
> are seeing issue regarding musl.

I think when we originally merged musl, lttng didn't build with it or
there were some kind of issues. I suspect lttng-ust still doesn't work
on all the architectures we build for so we likely need to be able to
optionally disable it. I'd be happy to see patches enabling it where it
does work though.


> We checked with Alpine since they also use musl and see similar
> problem. I'm investigating on that front. I'll let you know what I
> find.
> 
> Side note, gdb simply segfault (at start) for me on a core-image-
> minirmnal build using musl.
> How would someone open the corefile generated on the build system
> with the proper libs and all?

I saw other discussion about this so it looks like you're making
progress on this front. If not, Khem would be the person to ask about
this as our musl expert...

Cheers,

Richard

Re: [PATCH 1/2] lttng-tools ptest: add missing dependencies

[Jonathan Rajotte-Julien]

> Since the patches with a couple of fixes were passing the tests, I've
> merged the patches with the glibc/muslc tweak and also a tweak to make
> disabled ust work. I'd normally wait for v2 resubmission but I wanted
> to get this fix into M3 and the M3 build is already behind schedule.

Sounds good.

> 
> I was at a conference last week so a bit distracted, just catching up
> now.
> 
> > Also, I saw that lttng-ust is removed from PACKAGECONFIG when using
> > musl, not sure why the commit history is not particularly verbose
> > regarding why it was a problem. Still, it seems to build fine but we
> > are seeing issue regarding musl.
> 
> I think when we originally merged musl, lttng didn't build with it or
> there were some kind of issues. I suspect lttng-ust still doesn't work
> on all the architectures we build for so we likely need to be able to
> optionally disable it. I'd be happy to see patches enabling it where it
> does work though.

Well lttng-ust does work on musl as long as users do not fiddle with
sched_affinity... I cc'ed you on the musl thread about this [1].

[1] https://www.openwall.com/lists/musl/2019/03/15/5

For now we can leave the disable statement there. We will revisit when musl is
fixed and/or when lttng-ust have a fallback (most likely to happen) when using
musl.

> 
> 
> > We checked with Alpine since they also use musl and see similar
> > problem. I'm investigating on that front. I'll let you know what I
> > find.

See [1].

We also found a dynamic linker problem on our side for the notification test.
We will upstream it in the following weeks.

> > 
> > Side note, gdb simply segfault (at start) for me on a core-image-
> > minirmnal build using musl.
> > How would someone open the corefile generated on the build system
> > with the proper libs and all?
> 
> I saw other discussion about this so it looks like you're making
> progress on this front. If not, Khem would be the person to ask about
> this as our musl expert...

Do you have a link to said discussion? I ended up debugging using print
statement since even on Alpine gdb is "broken" (not working as expected). 
I'm not sure how much time I can use on this. Will let you know if any progress
is made to at least get a backtrace for the gdb coredump.

Thanks again!

-- 
Jonathan Rajotte-Julien
EfficiOS

Re: [PATCH 1/2] lttng-tools ptest: add missing dependencies

[Richard Purdie]

On Mon, 2019-03-18 at 12:47 -0400, Jonathan Rajotte-Julien wrote:
> > Since the patches with a couple of fixes were passing the tests,
> > I've
> > merged the patches with the glibc/muslc tweak and also a tweak to
> > make
> > disabled ust work. I'd normally wait for v2 resubmission but I
> > wanted
> > to get this fix into M3 and the M3 build is already behind
> > schedule.
> 
> Sounds good.

I should have done this before merging but for various reasons I
didn't. Your patches sounded good and fixed various problematic things
so wouldn't seem to make anything worse.

We have some ptest results numbers from a build and it shows before:

Recipe             | Passed      | Failed   | Skipped   | Time(s)   
lttng-tools        | 4431        | 7        | 391       | 549

After recent patches:

Recipe             | Passed      | Failed   | Skipped   | Time(s)   
lttng-tools        | 3994        | 5        | 0         | 684 T

So we're down on failures but its now not running all the tests and the
"T" means the ptest timed out and was killed.

I'm comparing:

https://autobuilder.yocto.io/pub/non-release/20190228-9/testresults/
testresult-report.txt 

with the report that will appear at:

https://autobuilder.yocto.io/pub/non-release/20190318-7/testresults/

Any idea what happened?

> Well lttng-ust does work on musl as long as users do not fiddle with
> sched_affinity... I cc'ed you on the musl thread about this [1].
> 
> [1] https://www.openwall.com/lists/musl/2019/03/15/5
> 
> For now we can leave the disable statement there. We will revisit
> when musl is fixed and/or when lttng-ust have a fallback (most likely
> to happen) when using musl.

Thanks, this is the discussion I was referring to before and I was
assuming it was leading to some of the problem you were seeing.

> > > Side note, gdb simply segfault (at start) for me on a core-image-
> > > minirmnal build using musl.
> > > How would someone open the corefile generated on the build system
> > > with the proper libs and all?
> > 
> > I saw other discussion about this so it looks like you're making
> > progress on this front. If not, Khem would be the person to ask
> > about
> > this as our musl expert...
> 
> Do you have a link to said discussion? I ended up debugging using
> print
> statement since even on Alpine gdb is "broken" (not working as
> expected). 
> I'm not sure how much time I can use on this. Will let you know if
> any progress
> is made to at least get a backtrace for the gdb coredump.

Khem is the one to talk to about this. I'm a little worried that gdb is
broken, that is something we need to investigate and clearly better
test too...

Cheers,

Richard