* [PATCH] systemtap: 3.3 -> 4.0 @ 2018-10-15 18:21 Victor Kamensky 2018-10-15 21:02 ` Randy MacLeod 2018-10-16 9:27 ` Burton, Ross 0 siblings, 2 replies; 13+ messages in thread From: Victor Kamensky @ 2018-10-15 18:21 UTC (permalink / raw) To: openembedded-core Upgrade systemtap from 3.3 to 4.0: Removed backported patch. Very short summary of major changes from SystemTap 4.0 announcement by Frank Ch. Eigler <fche@redhat.com>: > prometheus exporter network service; ebpf support extensions including > strings and implementation of traditional log(), sprintf() functions; > rebuilt rich tapset coverage for 4.17+ syscalls and for > tracepoint-based syscalls; script language tweaks for supporting > machine-generated scripts Fixes [YOCTO #12950] Signed-off-by: Victor Kamensky <kamensky@cisco.com> --- Hi Guys, I'll let you to decide whether you want to accept it for 2.6 at this point or push it for the next release. On one hand after move to 4.18 kernel SystemTap 3.3 fails to trace any system call on x86_64 - it is major functionality breakage. Look at: https://bugzilla.redhat.com/show_bug.cgi?id=1596456 On other hand it is quite late to do major version updates. I did go through limitted set of tests on all supported QEMU targets. As far as these tests concerned proposed 4.0 matches 3.3 results. Thanks, Victor ...entrypc-avoid-usage-of-uninitialized.patch | 46 ------------------- .../systemtap/systemtap_git.inc | 5 +- 2 files changed, 2 insertions(+), 49 deletions(-) delete mode 100644 meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch diff --git a/meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch b/meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch deleted file mode 100644 index d0082a1094..0000000000 --- a/meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch +++ /dev/null @@ -1,46 +0,0 @@ -From 8466fca2a074323a235ef38d425f994a2ff7e64f Mon Sep 17 00:00:00 2001 -From: Victor Kamensky <kamensky@cisco.com> -Date: Mon, 9 Jul 2018 09:31:19 -0700 -Subject: [PATCH] dwflpp::function_entrypc avoid usage of uninitialized memory - -Failure on 3.3 release was observed. Failure was elusive and -disappeared after seemingly random configure option change, or when -code was compiled with -O1 or -O0 (vs default -O2). Running failing -test case under valgrind memcheck pointed to couple places where -'Conditional jump or move depends on uninitialised value(s)' occured. - -After addressing these in two places in dwflpp::function_entrypc, -valgrind memcheck run is clean and original issue got fixed. - -Upstream-Status: Backport -Signed-off-by: Victor Kamensky <kamensky@cisco.com> ---- - dwflpp.cxx | 6 +++++- - 1 file changed, 5 insertions(+), 1 deletion(-) - -diff --git a/dwflpp.cxx b/dwflpp.cxx -index bfbb6b096..2172e705a 100644 ---- a/dwflpp.cxx -+++ b/dwflpp.cxx -@@ -2465,13 +2465,17 @@ bool - dwflpp::function_entrypc (Dwarf_Addr * addr) - { - assert (function); -+ -+ // assign default value -+ *addr = 0; -+ - // PR10574: reject 0, which tends to be eliminated COMDAT - if (dwarf_entrypc (function, addr) == 0 && *addr != 0) - return true; - - /* Assume the entry pc is the base address, or (if zero) - the first address of the ranges covering this DIE. */ -- Dwarf_Addr start, end; -+ Dwarf_Addr start = 0, end; - if (dwarf_ranges (function, 0, addr, &start, &end) >= 0) - { - if (*addr == 0) --- -2.17.1 - diff --git a/meta/recipes-kernel/systemtap/systemtap_git.inc b/meta/recipes-kernel/systemtap/systemtap_git.inc index 06924fc240..274fcde5c1 100644 --- a/meta/recipes-kernel/systemtap/systemtap_git.inc +++ b/meta/recipes-kernel/systemtap/systemtap_git.inc @@ -1,7 +1,7 @@ LICENSE = "GPLv2" LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263" -SRCREV = "48867d1cface9445d58e3c6e14497770b7eb77e1" -PV = "3.3" +SRCREV = "428f84e9e656bce71018e8902e4edb8aacafcc0e" +PV = "4.0" SRC_URI = "git://sourceware.org/git/systemtap.git \ file://configure-allow-to-disable-libvirt.patch \ @@ -11,7 +11,6 @@ SRC_URI = "git://sourceware.org/git/systemtap.git \ file://0001-Do-not-let-configure-write-a-python-location-into-th.patch \ file://0001-Install-python-modules-to-correct-library-dir.patch \ file://0001-staprun-stapbpf-don-t-support-installing-a-non-root.patch \ - file://0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch \ " COMPATIBLE_HOST = '(x86_64|i.86|powerpc|arm|aarch64|microblazeel|mips).*-linux' -- 2.17.1 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH] systemtap: 3.3 -> 4.0 2018-10-15 18:21 [PATCH] systemtap: 3.3 -> 4.0 Victor Kamensky @ 2018-10-15 21:02 ` Randy MacLeod 2018-10-16 3:54 ` Victor Kamensky 2018-10-16 9:27 ` Burton, Ross 1 sibling, 1 reply; 13+ messages in thread From: Randy MacLeod @ 2018-10-15 21:02 UTC (permalink / raw) To: Victor Kamensky, openembedded-core On 10/15/2018 02:21 PM, Victor Kamensky wrote: > Upgrade systemtap from 3.3 to 4.0: Removed backported patch. > > Very short summary of major changes from SystemTap 4.0 > announcement by Frank Ch. Eigler <fche@redhat.com>: > >> prometheus exporter network service; ebpf support extensions including >> strings and implementation of traditional log(), sprintf() functions; >> rebuilt rich tapset coverage for 4.17+ syscalls and for >> tracepoint-based syscalls; script language tweaks for supporting >> machine-generated scripts > > Fixes [YOCTO #12950] > > Signed-off-by: Victor Kamensky <kamensky@cisco.com> > --- > Hi Guys, > > I'll let you to decide whether you want to accept > it for 2.6 at this point or push it for the next release. > On one hand after move to 4.18 kernel SystemTap 3.3 > fails to trace any system call on x86_64 - it is major > functionality breakage. Look at: > > https://bugzilla.redhat.com/show_bug.cgi?id=1596456 Ouch! > > On other hand it is quite late to do major version > updates. It is but nothing depends on systemtap so I think we don't have much choice and we should do it unless there are autobuilder tests that regress. A quick read through the YP mega-manual suggests that we don't have to change any of the text. Victor, Could you take a look and confirm that please? I noticed that the link in section: 20.3.3. Documentation for: http://sourceware.org/systemtap/langref/ is dead. There is a pdf: http://sourceware.org/systemtap/langref.pdf Perhaps this is temporary due to the recent release. > > I did go through limited set of tests on all > supported QEMU targets. As far as these tests > concerned proposed 4.0 matches 3.3 results. > > Thanks, > Victor Thanks Victor. Talk about hot off the presses, 4.0 was released at: 13-Oct-2018 18:54 $ git log --oneline release-3.3..release-4.0 | wc -l 252 $ git diff release-3.3..release-4.0 | diffstat | tail -1 1574 files changed, 50756 insertions(+), 23604 deletions(-) Lots of changes to docs, testsuite but also quite a bit to core code and tapsets: $ git diff release-3.3..release-4.0 | diffstat | cut -d"/" -f-2 | uniq -c |grep -v " [1-5] " 770 b/doc 16 b/httpd 8 b/man 9 b/po 18 b/runtime 15 b/stap-exporter 6 b/staprun 440 b/tapset 223 b/testsuite ../Randy > > ...entrypc-avoid-usage-of-uninitialized.patch | 46 ------------------- > .../systemtap/systemtap_git.inc | 5 +- > 2 files changed, 2 insertions(+), 49 deletions(-) > delete mode 100644 meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch > > diff --git a/meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch b/meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch > deleted file mode 100644 > index d0082a1094..0000000000 > --- a/meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch > +++ /dev/null > @@ -1,46 +0,0 @@ > -From 8466fca2a074323a235ef38d425f994a2ff7e64f Mon Sep 17 00:00:00 2001 > -From: Victor Kamensky <kamensky@cisco.com> > -Date: Mon, 9 Jul 2018 09:31:19 -0700 > -Subject: [PATCH] dwflpp::function_entrypc avoid usage of uninitialized memory > - > -Failure on 3.3 release was observed. Failure was elusive and > -disappeared after seemingly random configure option change, or when > -code was compiled with -O1 or -O0 (vs default -O2). Running failing > -test case under valgrind memcheck pointed to couple places where > -'Conditional jump or move depends on uninitialised value(s)' occured. > - > -After addressing these in two places in dwflpp::function_entrypc, > -valgrind memcheck run is clean and original issue got fixed. > - > -Upstream-Status: Backport > -Signed-off-by: Victor Kamensky <kamensky@cisco.com> > ---- > - dwflpp.cxx | 6 +++++- > - 1 file changed, 5 insertions(+), 1 deletion(-) > - > -diff --git a/dwflpp.cxx b/dwflpp.cxx > -index bfbb6b096..2172e705a 100644 > ---- a/dwflpp.cxx > -+++ b/dwflpp.cxx > -@@ -2465,13 +2465,17 @@ bool > - dwflpp::function_entrypc (Dwarf_Addr * addr) > - { > - assert (function); > -+ > -+ // assign default value > -+ *addr = 0; > -+ > - // PR10574: reject 0, which tends to be eliminated COMDAT > - if (dwarf_entrypc (function, addr) == 0 && *addr != 0) > - return true; > - > - /* Assume the entry pc is the base address, or (if zero) > - the first address of the ranges covering this DIE. */ > -- Dwarf_Addr start, end; > -+ Dwarf_Addr start = 0, end; > - if (dwarf_ranges (function, 0, addr, &start, &end) >= 0) > - { > - if (*addr == 0) > --- > -2.17.1 > - > diff --git a/meta/recipes-kernel/systemtap/systemtap_git.inc b/meta/recipes-kernel/systemtap/systemtap_git.inc > index 06924fc240..274fcde5c1 100644 > --- a/meta/recipes-kernel/systemtap/systemtap_git.inc > +++ b/meta/recipes-kernel/systemtap/systemtap_git.inc > @@ -1,7 +1,7 @@ > LICENSE = "GPLv2" > LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263" > -SRCREV = "48867d1cface9445d58e3c6e14497770b7eb77e1" > -PV = "3.3" > +SRCREV = "428f84e9e656bce71018e8902e4edb8aacafcc0e" > +PV = "4.0" > > SRC_URI = "git://sourceware.org/git/systemtap.git \ > file://configure-allow-to-disable-libvirt.patch \ > @@ -11,7 +11,6 @@ SRC_URI = "git://sourceware.org/git/systemtap.git \ > file://0001-Do-not-let-configure-write-a-python-location-into-th.patch \ > file://0001-Install-python-modules-to-correct-library-dir.patch \ > file://0001-staprun-stapbpf-don-t-support-installing-a-non-root.patch \ > - file://0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch \ > " > > COMPATIBLE_HOST = '(x86_64|i.86|powerpc|arm|aarch64|microblazeel|mips).*-linux' > -- # Randy MacLeod # Wind River Linux ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] systemtap: 3.3 -> 4.0 2018-10-15 21:02 ` Randy MacLeod @ 2018-10-16 3:54 ` Victor Kamensky 0 siblings, 0 replies; 13+ messages in thread From: Victor Kamensky @ 2018-10-16 3:54 UTC (permalink / raw) To: Randy MacLeod; +Cc: openembedded-core Hi Randy, On Mon, 15 Oct 2018, Randy MacLeod wrote: > On 10/15/2018 02:21 PM, Victor Kamensky wrote: >> Upgrade systemtap from 3.3 to 4.0: Removed backported patch. >> >> Very short summary of major changes from SystemTap 4.0 >> announcement by Frank Ch. Eigler <fche@redhat.com>: >> >>> prometheus exporter network service; ebpf support extensions including >>> strings and implementation of traditional log(), sprintf() functions; >>> rebuilt rich tapset coverage for 4.17+ syscalls and for >>> tracepoint-based syscalls; script language tweaks for supporting >>> machine-generated scripts >> >> Fixes [YOCTO #12950] >> >> Signed-off-by: Victor Kamensky <kamensky@cisco.com> >> --- >> Hi Guys, >> >> I'll let you to decide whether you want to accept >> it for 2.6 at this point or push it for the next release. >> On one hand after move to 4.18 kernel SystemTap 3.3 >> fails to trace any system call on x86_64 - it is major >> functionality breakage. Look at: >> >> https://bugzilla.redhat.com/show_bug.cgi?id=1596456 > > Ouch! > >> >> On other hand it is quite late to do major version >> updates. > > It is but nothing depends on systemtap so I think we > don't have much choice and we should do it unless there > are autobuilder tests that regress. Personally, I agree. I had similar thoughts, since it is leaf project risk is quite contained. > > A quick read through the YP mega-manual suggests that > we don't have to change any of the text. > Victor, > Could you take a look and confirm that please? > > I noticed that the link in section: > 20.3.3. Documentation > for: > http://sourceware.org/systemtap/langref/ > is dead. There is a pdf: > http://sourceware.org/systemtap/langref.pdf > Perhaps this is temporary due to the recent release. Yes, I agree. It is better to change link to langref.pdf as you suggested. But I don't know how to do that. Is there somewhere git that holds YP mega-manual sources? >> >> I did go through limited set of tests on all >> supported QEMU targets. As far as these tests >> concerned proposed 4.0 matches 3.3 results. >> >> Thanks, >> Victor > > Thanks Victor. > > Talk about hot off the presses, 4.0 was released at: > 13-Oct-2018 18:54 Yes :), I was looking how to fix [YOCTO #12950] and was about to write email to SystemTap guys asking when the next release is planned and got email from Frank on SystemTap mailing list announcing 4.0 release. It was very good timing. My understanding is that FC28 and FC27 will get SystemTap 4.0 very soon since they moved to 4.18 kernel too. Thanks, Victor > $ git log --oneline release-3.3..release-4.0 | wc -l > 252 > > $ git diff release-3.3..release-4.0 | diffstat | tail -1 > 1574 files changed, 50756 insertions(+), 23604 deletions(-) > > Lots of changes to docs, testsuite but also quite a bit to core code > and tapsets: > > $ git diff release-3.3..release-4.0 | diffstat | cut -d"/" -f-2 | uniq -c > |grep -v " [1-5] " > 770 b/doc > 16 b/httpd > 8 b/man > 9 b/po > 18 b/runtime > 15 b/stap-exporter > 6 b/staprun > 440 b/tapset > 223 b/testsuite > > > ../Randy > >> >> ...entrypc-avoid-usage-of-uninitialized.patch | 46 ------------------- >> .../systemtap/systemtap_git.inc | 5 +- >> 2 files changed, 2 insertions(+), 49 deletions(-) >> delete mode 100644 >> meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch >> >> diff --git >> a/meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch >> b/meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch >> deleted file mode 100644 >> index d0082a1094..0000000000 >> --- >> a/meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch >> +++ /dev/null >> @@ -1,46 +0,0 @@ >> -From 8466fca2a074323a235ef38d425f994a2ff7e64f Mon Sep 17 00:00:00 2001 >> -From: Victor Kamensky <kamensky@cisco.com> >> -Date: Mon, 9 Jul 2018 09:31:19 -0700 >> -Subject: [PATCH] dwflpp::function_entrypc avoid usage of uninitialized >> memory >> - >> -Failure on 3.3 release was observed. Failure was elusive and >> -disappeared after seemingly random configure option change, or when >> -code was compiled with -O1 or -O0 (vs default -O2). Running failing >> -test case under valgrind memcheck pointed to couple places where >> -'Conditional jump or move depends on uninitialised value(s)' occured. >> - >> -After addressing these in two places in dwflpp::function_entrypc, >> -valgrind memcheck run is clean and original issue got fixed. >> - >> -Upstream-Status: Backport >> -Signed-off-by: Victor Kamensky <kamensky@cisco.com> >> ---- >> - dwflpp.cxx | 6 +++++- >> - 1 file changed, 5 insertions(+), 1 deletion(-) >> - >> -diff --git a/dwflpp.cxx b/dwflpp.cxx >> -index bfbb6b096..2172e705a 100644 >> ---- a/dwflpp.cxx >> -+++ b/dwflpp.cxx >> -@@ -2465,13 +2465,17 @@ bool >> - dwflpp::function_entrypc (Dwarf_Addr * addr) >> - { >> - assert (function); >> -+ >> -+ // assign default value >> -+ *addr = 0; >> -+ >> - // PR10574: reject 0, which tends to be eliminated COMDAT >> - if (dwarf_entrypc (function, addr) == 0 && *addr != 0) >> - return true; >> - >> - /* Assume the entry pc is the base address, or (if zero) >> - the first address of the ranges covering this DIE. */ >> -- Dwarf_Addr start, end; >> -+ Dwarf_Addr start = 0, end; >> - if (dwarf_ranges (function, 0, addr, &start, &end) >= 0) >> - { >> - if (*addr == 0) >> --- >> -2.17.1 >> - >> diff --git a/meta/recipes-kernel/systemtap/systemtap_git.inc >> b/meta/recipes-kernel/systemtap/systemtap_git.inc >> index 06924fc240..274fcde5c1 100644 >> --- a/meta/recipes-kernel/systemtap/systemtap_git.inc >> +++ b/meta/recipes-kernel/systemtap/systemtap_git.inc >> @@ -1,7 +1,7 @@ >> LICENSE = "GPLv2" >> LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263" >> -SRCREV = "48867d1cface9445d58e3c6e14497770b7eb77e1" >> -PV = "3.3" >> +SRCREV = "428f84e9e656bce71018e8902e4edb8aacafcc0e" >> +PV = "4.0" >> SRC_URI = "git://sourceware.org/git/systemtap.git \ >> file://configure-allow-to-disable-libvirt.patch \ >> @@ -11,7 +11,6 @@ SRC_URI = "git://sourceware.org/git/systemtap.git \ >> file://0001-Do-not-let-configure-write-a-python-location-into-th.patch >> \ >> file://0001-Install-python-modules-to-correct-library-dir.patch >> \ >> file://0001-staprun-stapbpf-don-t-support-installing-a-non-root.patch >> \ >> - >> file://0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch \ >> " >> COMPATIBLE_HOST = >> '(x86_64|i.86|powerpc|arm|aarch64|microblazeel|mips).*-linux' >> > > > -- > # Randy MacLeod > # Wind River Linux > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] systemtap: 3.3 -> 4.0 2018-10-15 18:21 [PATCH] systemtap: 3.3 -> 4.0 Victor Kamensky 2018-10-15 21:02 ` Randy MacLeod @ 2018-10-16 9:27 ` Burton, Ross 2018-10-16 9:54 ` Richard Purdie 1 sibling, 1 reply; 13+ messages in thread From: Burton, Ross @ 2018-10-16 9:27 UTC (permalink / raw) To: Victor Kamensky; +Cc: OE-core On Mon, 15 Oct 2018 at 19:22, Victor Kamensky <kamensky@cisco.com> wrote: > Upgrade systemtap from 3.3 to 4.0: Removed backported patch. WARNING: systemtap-4.0-r0 do_package_qa: QA Issue: systemtap: /systemtap/etc/stap-exporter/EXAMPLE is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination systemtap: /systemtap/etc/stap-exporter/EXAMPLE_NOERROR is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination systemtap: /systemtap/etc/stap-exporter/errno.stp is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination systemtap: /systemtap/etc/stap-exporter/syscalllatency.stp is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination systemtap: /systemtap/etc/stap-exporter/timeout.stp is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination systemtap: /systemtap/etc/stap-exporter/topsys.stp is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination [host-user-contaminated] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] systemtap: 3.3 -> 4.0 2018-10-16 9:27 ` Burton, Ross @ 2018-10-16 9:54 ` Richard Purdie 2018-10-17 5:56 ` Victor Kamensky 0 siblings, 1 reply; 13+ messages in thread From: Richard Purdie @ 2018-10-16 9:54 UTC (permalink / raw) To: Burton, Ross, Victor Kamensky; +Cc: OE-core On Tue, 2018-10-16 at 10:27 +0100, Burton, Ross wrote: > On Mon, 15 Oct 2018 at 19:22, Victor Kamensky <kamensky@cisco.com> > wrote: > > Upgrade systemtap from 3.3 to 4.0: Removed backported patch. > > WARNING: systemtap-4.0-r0 do_package_qa: QA Issue: systemtap: > /systemtap/etc/stap-exporter/EXAMPLE is owned by uid 1000, which is > the same as the user running bitbake. This may be due to host > contamination > systemtap: /systemtap/etc/stap-exporter/EXAMPLE_NOERROR is owned by > uid 1000, which is the same as the user running bitbake. This may be > due to host contamination > systemtap: /systemtap/etc/stap-exporter/errno.stp is owned by uid > 1000, which is the same as the user running bitbake. This may be due > to host contamination > systemtap: /systemtap/etc/stap-exporter/syscalllatency.stp is owned > by > uid 1000, which is the same as the user running bitbake. This may be > due to host contamination > systemtap: /systemtap/etc/stap-exporter/timeout.stp is owned by uid > 1000, which is the same as the user running bitbake. This may be due > to host contamination > systemtap: /systemtap/etc/stap-exporter/topsys.stp is owned by uid > 1000, which is the same as the user running bitbake. This may be due > to host contamination [host-user-contaminated] There are also other issues from the addition of systemd unit files in 4.0. I've a patch in -next which tweaks both issues which I'll put in for testing. Cheers, Richard ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] systemtap: 3.3 -> 4.0 2018-10-16 9:54 ` Richard Purdie @ 2018-10-17 5:56 ` Victor Kamensky 2018-10-17 11:59 ` richard.purdie 0 siblings, 1 reply; 13+ messages in thread From: Victor Kamensky @ 2018-10-17 5:56 UTC (permalink / raw) To: Richard Purdie; +Cc: OE-core On Tue, 16 Oct 2018, Richard Purdie wrote: > On Tue, 2018-10-16 at 10:27 +0100, Burton, Ross wrote: >> On Mon, 15 Oct 2018 at 19:22, Victor Kamensky <kamensky@cisco.com> >> wrote: >>> Upgrade systemtap from 3.3 to 4.0: Removed backported patch. >> >> WARNING: systemtap-4.0-r0 do_package_qa: QA Issue: systemtap: >> /systemtap/etc/stap-exporter/EXAMPLE is owned by uid 1000, which is >> the same as the user running bitbake. This may be due to host >> contamination >> systemtap: /systemtap/etc/stap-exporter/EXAMPLE_NOERROR is owned by >> uid 1000, which is the same as the user running bitbake. This may be >> due to host contamination >> systemtap: /systemtap/etc/stap-exporter/errno.stp is owned by uid >> 1000, which is the same as the user running bitbake. This may be due >> to host contamination >> systemtap: /systemtap/etc/stap-exporter/syscalllatency.stp is owned >> by >> uid 1000, which is the same as the user running bitbake. This may be >> due to host contamination >> systemtap: /systemtap/etc/stap-exporter/timeout.stp is owned by uid >> 1000, which is the same as the user running bitbake. This may be due >> to host contamination >> systemtap: /systemtap/etc/stap-exporter/topsys.stp is owned by uid >> 1000, which is the same as the user running bitbake. This may be due >> to host contamination [host-user-contaminated] > > There are also other issues from the addition of systemd unit files in > 4.0. I've a patch in -next which tweaks both issues which I'll put in > for testing. Richard, thank you very prompt help. In the long term it would be beneficial that those issues be fixed in SystemTap itself, correct? If so, I can look at creating proper patches and pushing them into SystemTap. Also on my todo list is to attempt to push few existing OE systemtap patches into SystemTap upstream source. I believe they look like quite legit issues and I hope they could be upstreamed. Thanks, Victor > Cheers, > > Richard > > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] systemtap: 3.3 -> 4.0 2018-10-17 5:56 ` Victor Kamensky @ 2018-10-17 11:59 ` richard.purdie 2018-10-17 14:50 ` Martin Jansa 0 siblings, 1 reply; 13+ messages in thread From: richard.purdie @ 2018-10-17 11:59 UTC (permalink / raw) To: Victor Kamensky; +Cc: OE-core On Tue, 2018-10-16 at 22:56 -0700, Victor Kamensky wrote: > In the long term it would be beneficial that those issues be > fixed in SystemTap itself, correct? If so, I can look at creating > proper patches and pushing them into SystemTap. Yes, fixing those upstream would be very helpful, probably through adding the ability to configure the systemd unit path and fixing the file installation method. > Also on my todo list is to attempt to push few existing > OE systemtap patches into SystemTap upstream source. I believe > they look like quite legit issues and I hope they could be > upstreamed. That kind of work is very much appreciated and makes everyone's life easier in the future! Cheers, Richard ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] systemtap: 3.3 -> 4.0 2018-10-17 11:59 ` richard.purdie @ 2018-10-17 14:50 ` Martin Jansa 2018-10-17 15:07 ` richard.purdie 0 siblings, 1 reply; 13+ messages in thread From: Martin Jansa @ 2018-10-17 14:50 UTC (permalink / raw) To: Richard Purdie; +Cc: Patches and discussions about the oe-core layer [-- Attachment #1: Type: text/plain, Size: 1604 bytes --] There seems to be one more issue in current master (for which I haven't seen a fix in master-next or ML): ERROR: systemtap-4.0-r0 do_package: QA Issue: systemtap: Files/directories were installed but not shipped in any package: /lib /lib/systemd /lib/systemd/system /lib/systemd/system/stap-exporter.service Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install. systemtap: 4 installed and not shipped files. [installed-vs-shipped] On Wed, Oct 17, 2018 at 1:59 PM <richard.purdie@linuxfoundation.org> wrote: > On Tue, 2018-10-16 at 22:56 -0700, Victor Kamensky wrote: > > In the long term it would be beneficial that those issues be > > fixed in SystemTap itself, correct? If so, I can look at creating > > proper patches and pushing them into SystemTap. > > Yes, fixing those upstream would be very helpful, probably through > adding the ability to configure the systemd unit path and fixing the > file installation method. > > > Also on my todo list is to attempt to push few existing > > OE systemtap patches into SystemTap upstream source. I believe > > they look like quite legit issues and I hope they could be > > upstreamed. > > That kind of work is very much appreciated and makes everyone's life > easier in the future! > > Cheers, > > Richard > > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core > [-- Attachment #2: Type: text/html, Size: 2409 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] systemtap: 3.3 -> 4.0 2018-10-17 14:50 ` Martin Jansa @ 2018-10-17 15:07 ` richard.purdie 2018-10-17 19:01 ` Martin Jansa 0 siblings, 1 reply; 13+ messages in thread From: richard.purdie @ 2018-10-17 15:07 UTC (permalink / raw) To: Martin Jansa; +Cc: Patches and discussions about the oe-core layer On Wed, 2018-10-17 at 16:50 +0200, Martin Jansa wrote: > There seems to be one more issue in current master (for which I > haven't seen a fix in master-next or ML): > > ERROR: systemtap-4.0-r0 do_package: QA Issue: systemtap: > Files/directories were installed but not shipped in any package: > /lib > /lib/systemd > /lib/systemd/system > /lib/systemd/system/stap-exporter.service > Please set FILES such that these items are packaged. Alternatively if > they are unneeded, avoid installing them or delete them within > do_install. > systemtap: 4 installed and not shipped files. [installed-vs-shipped] http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=3b77e7b7852549dcfbc426d4ce258e6e857c0acd was supposed to fix that :( Cheers, Richard ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] systemtap: 3.3 -> 4.0 2018-10-17 15:07 ` richard.purdie @ 2018-10-17 19:01 ` Martin Jansa 2018-10-17 19:03 ` [PATCH] systemtap: fix QA issue Martin Jansa 2018-10-17 20:21 ` [PATCH] systemtap: 3.3 -> 4.0 Victor Kamensky 0 siblings, 2 replies; 13+ messages in thread From: Martin Jansa @ 2018-10-17 19:01 UTC (permalink / raw) To: Richard Purdie; +Cc: Patches and discussions about the oe-core layer [-- Attachment #1: Type: text/plain, Size: 1049 bytes --] systemd.bbclass won't include that .service file in FILES_<pkg> without setting SYSTEMD_SERVICE_<pkg>. I'll send fix for that in few minutes. On Wed, Oct 17, 2018 at 5:07 PM <richard.purdie@linuxfoundation.org> wrote: > On Wed, 2018-10-17 at 16:50 +0200, Martin Jansa wrote: > > There seems to be one more issue in current master (for which I > > haven't seen a fix in master-next or ML): > > > > ERROR: systemtap-4.0-r0 do_package: QA Issue: systemtap: > > Files/directories were installed but not shipped in any package: > > /lib > > /lib/systemd > > /lib/systemd/system > > /lib/systemd/system/stap-exporter.service > > Please set FILES such that these items are packaged. Alternatively if > > they are unneeded, avoid installing them or delete them within > > do_install. > > systemtap: 4 installed and not shipped files. [installed-vs-shipped] > > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=3b77e7b7852549dcfbc426d4ce258e6e857c0acd > was supposed to fix that :( > > Cheers, > > Richard > > [-- Attachment #2: Type: text/html, Size: 1578 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH] systemtap: fix QA issue 2018-10-17 19:01 ` Martin Jansa @ 2018-10-17 19:03 ` Martin Jansa 2018-10-17 20:21 ` [PATCH] systemtap: 3.3 -> 4.0 Victor Kamensky 1 sibling, 0 replies; 13+ messages in thread From: Martin Jansa @ 2018-10-17 19:03 UTC (permalink / raw) To: openembedded-core ERROR: systemtap-4.0-r0 do_package: QA Issue: systemtap: Files/directories were installed but not shipped in any package: /lib /lib/systemd /lib/systemd/system /lib/systemd/system/stap-exporter.service Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install. systemtap: 4 installed and not shipped files. [installed-vs-shipped] Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> --- meta/recipes-kernel/systemtap/systemtap_git.bb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/meta/recipes-kernel/systemtap/systemtap_git.bb b/meta/recipes-kernel/systemtap/systemtap_git.bb index 3c45bfa937..88942a2362 100644 --- a/meta/recipes-kernel/systemtap/systemtap_git.bb +++ b/meta/recipes-kernel/systemtap/systemtap_git.bb @@ -27,6 +27,8 @@ PACKAGECONFIG[python3-probes] = "--with-python3-probes,--without-python3-probes, inherit autotools gettext pkgconfig distutils3-base systemd +SYSTEMD_SERVICE_${PN} = "stap-exporter.service" + do_configure_prepend () { # Improve reproducibility for c++ object files reltivepath="${@os.path.relpath(d.getVar('STAGING_INCDIR'), d.getVar('S'))}" -- 2.17.1 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH] systemtap: 3.3 -> 4.0 2018-10-17 19:01 ` Martin Jansa 2018-10-17 19:03 ` [PATCH] systemtap: fix QA issue Martin Jansa @ 2018-10-17 20:21 ` Victor Kamensky 2018-10-26 3:37 ` Victor Kamensky 1 sibling, 1 reply; 13+ messages in thread From: Victor Kamensky @ 2018-10-17 20:21 UTC (permalink / raw) To: Martin Jansa; +Cc: Patches and discussions about the oe-core layer [-- Attachment #1: Type: TEXT/PLAIN, Size: 1676 bytes --] On Wed, 17 Oct 2018, Martin Jansa wrote: > systemd.bbclass won't include that .service file in FILES_<pkg> without setting SYSTEMD_SERVICE_<pkg>. > I'll send fix for that in few minutes. Martin, Richard, thank you for the help. I've reproduced the issue reported by Martin in systemd enabled image, and verified that his fix solves the issue. The lesson I learnt: o Look for WARNINGs o Test systemd enabled image, along with default sysvinit stap-exporter service is a new feature in SystemTap 4.0, I've never used it before. I'll dig in and will try to make sure that it is functional in resulting OE image. Thanks, Victor > On Wed, Oct 17, 2018 at 5:07 PM <richard.purdie@linuxfoundation.org> wrote: > On Wed, 2018-10-17 at 16:50 +0200, Martin Jansa wrote: > > There seems to be one more issue in current master (for which I > > haven't seen a fix in master-next or ML): > > > > ERROR: systemtap-4.0-r0 do_package: QA Issue: systemtap: > > Files/directories were installed but not shipped in any package: > > /lib > > /lib/systemd > > /lib/systemd/system > > /lib/systemd/system/stap-exporter.service > > Please set FILES such that these items are packaged. Alternatively if > > they are unneeded, avoid installing them or delete them within > > do_install. > > systemtap: 4 installed and not shipped files. [installed-vs-shipped] > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=3b77e7b7852549dcfbc426d4ce258e6e857c0acd > was supposed to fix that :( > > Cheers, > > Richard > > > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH] systemtap: 3.3 -> 4.0 2018-10-17 20:21 ` [PATCH] systemtap: 3.3 -> 4.0 Victor Kamensky @ 2018-10-26 3:37 ` Victor Kamensky 0 siblings, 0 replies; 13+ messages in thread From: Victor Kamensky @ 2018-10-26 3:37 UTC (permalink / raw) To: Martin Jansa, Richard Purdie Cc: Patches and discussions about the oe-core layer Hi Guys, On Wed, 17 Oct 2018, Victor Kamensky wrote: > stap-exporter service is a new feature in SystemTap 4.0, I've never used > it before. I'll dig in and will try to make sure that it is functional > in resulting OE image. I've had a chance to look at stap-exporter and its service more closely. It looks like it is simple HTTP server that allows to run stap scripts on target and relay its information to Prometheus monitoring system. Please check. https://www.mankier.com/8/stap-exporter https://prometheus.io Since it is HTTP server, and its man page says that it is not supposed to run in untrusted environment I am quite hesitant to have it enabled by default. Moreover, it comes with some default example scripts that actually fail to compile in OE core-image-lsb-sdk because it does not include kernel-vmlinux package by default. stap-exporter is not prepared to handle script compilation failure and tries to compile it in the loop, so in core-image-lsb-sdk systemd image where stap-exporter.service would be enabled by default it would trash the system. Considering all that I suggest stap-exporter to be moved in separate systemtap-exporter package, so it can be installed explcitely. I posted patch for it at http://lists.openembedded.org/pipermail/openembedded-core/2018-October/157073.html BTW in FC it is separate systemtap-exporter package. I've tested systemtap-exporter and stap-exporter, if installed along with proper prerequisites (kernel-vmlinux) is functioning OK with given default example as far as my manual http testing with curl shows. I did not test it with prometheus. Thanks, Victor ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2018-10-26 3:37 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-10-15 18:21 [PATCH] systemtap: 3.3 -> 4.0 Victor Kamensky 2018-10-15 21:02 ` Randy MacLeod 2018-10-16 3:54 ` Victor Kamensky 2018-10-16 9:27 ` Burton, Ross 2018-10-16 9:54 ` Richard Purdie 2018-10-17 5:56 ` Victor Kamensky 2018-10-17 11:59 ` richard.purdie 2018-10-17 14:50 ` Martin Jansa 2018-10-17 15:07 ` richard.purdie 2018-10-17 19:01 ` Martin Jansa 2018-10-17 19:03 ` [PATCH] systemtap: fix QA issue Martin Jansa 2018-10-17 20:21 ` [PATCH] systemtap: 3.3 -> 4.0 Victor Kamensky 2018-10-26 3:37 ` Victor Kamensky
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.