[PATCH 0/8] systemtap improvements

[PATCH 0/8] systemtap improvements

[Yann Dirson]

From: Yann Dirson <yann@blade-group.com>

The goal of this series is to allow for packages to provide systemtap
probes and get them cross-compiled on the host, shipping only the
minimal runtime in the filesystem.

People willing to build the probes on target can continue to do so
with no changes for some many users, but usage of python probes now
requires systemtap-python.

This series also includes unrelated fixes to the systemtap packaging,
on which I stumbled while on the main subject.

Of note is the "fix finding proper System.map" patch, for which I
wonder if the real fix would not be to keep an unversionned System.map
file in kernel_build_tree, so stay compatible with upstream kernel.


Yann Dirson (8):
  systemtap: split examples and python scripts out of main package
  systemtap: remove extra dependencies
  systemtap: clarify the relation between exporter and python3-probes
    feature
  systemtap: fix install when python3-probes is disabled in
    PACKAGECONFIG
  systemtap: split runtime material in its own package
  systemtap-native: fix finding proper System.map
  kernel: consider vmlinux a build artifact, so systemtap can use it
  New systemtap.bb class, and systemtap-demo.bb to illustrate it.

 meta/classes/kernel.bbclass                   |  4 +
 meta/classes/systemtap.bbclass                | 74 +++++++++++++++++++
 .../systemtap/systemtap-demo_git.bb           | 71 ++++++++++++++++++
 .../systemtap/systemtap-native_git.bb         |  6 ++
 .../systemtap/system-map-location.patch       | 26 +++++++
 .../recipes-kernel/systemtap/systemtap_git.bb | 38 ++++++++--
 6 files changed, 213 insertions(+), 6 deletions(-)
 create mode 100644 meta/classes/systemtap.bbclass
 create mode 100644 meta/recipes-kernel/systemtap/systemtap-demo_git.bb
 create mode 100644 meta/recipes-kernel/systemtap/systemtap/system-map-location.patch

-- 
2.28.0

[PATCH 1/8] systemtap: split examples and python scripts out of main package

[Yann Dirson]

From: Yann Dirson <yann@blade-group.com>

---
 meta/recipes-kernel/systemtap/systemtap_git.bb | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/meta/recipes-kernel/systemtap/systemtap_git.bb b/meta/recipes-kernel/systemtap/systemtap_git.bb
index 1c9f2aed16..e98aff1851 100644
--- a/meta/recipes-kernel/systemtap/systemtap_git.bb
+++ b/meta/recipes-kernel/systemtap/systemtap_git.bb
@@ -38,6 +38,23 @@ RDEPENDS_${PN}-exporter = "${PN} python3-core python3-netclient"
 
 SYSTEMD_SERVICE_${PN}-exporter = "stap-exporter.service"
 
+PACKAGES =+ "${PN}-examples"
+FILES_${PN}-examples = "${datadir}/${BPN}/examples/"
+RDEPENDS_${PN}-examples += "${PN}"
+
+# don't complain that some examples involve bash, perl, php...
+INSANE_SKIP_${PN}-examples += "file-rdeps"
+
+PACKAGES =+ "${PN}-python"
+FILES_${PN}-python += "\
+ ${bindir}/dtrace \
+ ${libdir}/python*/ \
+ ${libexecdir}/${BPN}/python/ \
+"
+# python material requires sdt headers
+RDEPENDS_${PN}-python += "${PN}-dev python3-core"
+INSANE_SKIP_${PN}-python += "dev-deps"
+
 do_configure_prepend () {
     # Improve reproducibility for c++ object files
     reltivepath="${@os.path.relpath(d.getVar('STAGING_INCDIR'), d.getVar('S'))}"
-- 
2.28.0

[PATCH 2/8] systemtap: remove extra dependencies

[Yann Dirson]

From: Yann Dirson <yann@blade-group.com>

Most of the RDEPENDS part of PACKAGECONFIG[translator] appears to be related to
examples only.
---
 meta/recipes-kernel/systemtap/systemtap_git.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/systemtap/systemtap_git.bb b/meta/recipes-kernel/systemtap/systemtap_git.bb
index e98aff1851..375e570454 100644
--- a/meta/recipes-kernel/systemtap/systemtap_git.bb
+++ b/meta/recipes-kernel/systemtap/systemtap_git.bb
@@ -19,7 +19,7 @@ STAP_DOCS ?= "--disable-docs --disable-publican --disable-refdocs"
 EXTRA_OECONF += "${STAP_DOCS} "
 
 PACKAGECONFIG ??= "translator sqlite monitor python3-probes"
-PACKAGECONFIG[translator] = "--enable-translator,--disable-translator,boost,python3-core bash perl"
+PACKAGECONFIG[translator] = "--enable-translator,--disable-translator,boost,bash"
 PACKAGECONFIG[libvirt] = "--enable-libvirt,--disable-libvirt,libvirt"
 PACKAGECONFIG[sqlite] = "--enable-sqlite,--disable-sqlite,sqlite3"
 PACKAGECONFIG[monitor] = "--enable-monitor,--disable-monitor,ncurses json-c"
-- 
2.28.0

[PATCH 3/8] systemtap: clarify the relation between exporter and python3-probes feature

[Yann Dirson]

From: Yann Dirson <yann@blade-group.com>

---
 meta/recipes-kernel/systemtap/systemtap_git.bb | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/meta/recipes-kernel/systemtap/systemtap_git.bb b/meta/recipes-kernel/systemtap/systemtap_git.bb
index 375e570454..89f550c859 100644
--- a/meta/recipes-kernel/systemtap/systemtap_git.bb
+++ b/meta/recipes-kernel/systemtap/systemtap_git.bb
@@ -27,15 +27,13 @@ PACKAGECONFIG[python3-probes] = "--with-python3-probes,--without-python3-probes,
 
 inherit autotools gettext pkgconfig distutils3-base systemd
 
+# exporter comes with python3-probes
 PACKAGES =+ "${PN}-exporter"
-
 FILES_${PN}-exporter = "${sysconfdir}/stap-exporter/* \
                         ${sysconfdir}/sysconfig/stap-exporter \
                         ${systemd_unitdir}/system/stap-exporter.service \
                         ${sbindir}/stap-exporter"
-
 RDEPENDS_${PN}-exporter = "${PN} python3-core python3-netclient"
-
 SYSTEMD_SERVICE_${PN}-exporter = "stap-exporter.service"
 
 PACKAGES =+ "${PN}-examples"
-- 
2.28.0

[PATCH 4/8] systemtap: fix install when python3-probes is disabled in PACKAGECONFIG

[Yann Dirson]

From: Yann Dirson <yann@blade-group.com>

---
 meta/recipes-kernel/systemtap/systemtap_git.bb | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-kernel/systemtap/systemtap_git.bb b/meta/recipes-kernel/systemtap/systemtap_git.bb
index 89f550c859..74bf7cb35c 100644
--- a/meta/recipes-kernel/systemtap/systemtap_git.bb
+++ b/meta/recipes-kernel/systemtap/systemtap_git.bb
@@ -66,7 +66,7 @@ do_install_append () {
       rm ${D}${libexecdir}/${PN}/stap-env
    fi
 
-   if [ ${D}${prefix}/lib != `dirname ${D}${systemd_unitdir}` ]; then
+   if [ -d ${D}${prefix}/lib/systemd -a ${D}${prefix}/lib != `dirname ${D}${systemd_unitdir}` ]; then
       # Fix makefile hardcoded path assumptions for systemd (assumes $prefix)
       # without usrmerge distro feature enabled
       install -d `dirname ${D}${systemd_unitdir}`
@@ -75,7 +75,9 @@ do_install_append () {
    fi
 
    # Ensure correct ownership for files copied in
-   chown root:root ${D}${sysconfdir}/stap-exporter/* -R
+   if [ -d ${D}${sysconfdir}/stap-exporter ]; then
+       chown root:root ${D}${sysconfdir}/stap-exporter/* -R
+   fi
 }
 
 BBCLASSEXTEND = "nativesdk"
-- 
2.28.0

[PATCH 5/8] systemtap: split runtime material in its own package

[Yann Dirson]

From: Yann Dirson <yann@blade-group.com>

Note the _class-target qualifier, here to prevent a funky dependency of
systemtap-native on systemtap-native-runtime-native.  This possibly hints
to something deeper ?
---
 meta/recipes-kernel/systemtap/systemtap_git.bb | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/meta/recipes-kernel/systemtap/systemtap_git.bb b/meta/recipes-kernel/systemtap/systemtap_git.bb
index 74bf7cb35c..8dad5b15f0 100644
--- a/meta/recipes-kernel/systemtap/systemtap_git.bb
+++ b/meta/recipes-kernel/systemtap/systemtap_git.bb
@@ -36,6 +36,15 @@ FILES_${PN}-exporter = "${sysconfdir}/stap-exporter/* \
 RDEPENDS_${PN}-exporter = "${PN} python3-core python3-netclient"
 SYSTEMD_SERVICE_${PN}-exporter = "stap-exporter.service"
 
+PACKAGES =+ "${PN}-runtime"
+FILES_${PN}-runtime = "\
+ ${bindir}/staprun \
+ ${bindir}/stap-merge \
+ ${bindir}/stapsh \
+ ${libexecdir}/${BPN}/stapio \
+"
+RDEPENDS_${PN}_class-target += "${PN}-runtime"
+
 PACKAGES =+ "${PN}-examples"
 FILES_${PN}-examples = "${datadir}/${BPN}/examples/"
 RDEPENDS_${PN}-examples += "${PN}"
-- 
2.28.0

[PATCH 6/8] systemtap-native: fix finding proper System.map

[Yann Dirson]

From: Yann Dirson <yann@blade-group.com>

Upstream looks for unversionned System.map file in kernel_build_tree,
and if this fails goes for versionned filename in $sysroot/boot/.
OTOH we have a versionned filename in kernel_build_tree.
---
 .../systemtap/systemtap-native_git.bb         |  6 +++++
 .../systemtap/system-map-location.patch       | 26 +++++++++++++++++++
 2 files changed, 32 insertions(+)
 create mode 100644 meta/recipes-kernel/systemtap/systemtap/system-map-location.patch

diff --git a/meta/recipes-kernel/systemtap/systemtap-native_git.bb b/meta/recipes-kernel/systemtap/systemtap-native_git.bb
index 19cc1cf0f0..e24abcc2d6 100644
--- a/meta/recipes-kernel/systemtap/systemtap-native_git.bb
+++ b/meta/recipes-kernel/systemtap/systemtap-native_git.bb
@@ -3,4 +3,10 @@ require systemtap_git.bb
 
 inherit native
 
+# don't look in /boot, and use kernel version to find proper
+# System.map-xxx in kernel build tree
+SRC_URI += "\
+ file://system-map-location.patch \
+"
+
 addtask addto_recipe_sysroot after do_populate_sysroot before do_build
diff --git a/meta/recipes-kernel/systemtap/systemtap/system-map-location.patch b/meta/recipes-kernel/systemtap/systemtap/system-map-location.patch
new file mode 100644
index 0000000000..61e477bf69
--- /dev/null
+++ b/meta/recipes-kernel/systemtap/systemtap/system-map-location.patch
@@ -0,0 +1,26 @@
+Index: git/session.cxx
+===================================================================
+--- git.orig/session.cxx
++++ git/session.cxx
+@@ -2021,20 +2021,12 @@ systemtap_session::parse_kernel_exports
+ int
+ systemtap_session::parse_kernel_functions ()
+ {
+-  string system_map_path = kernel_build_tree + "/System.map";
++  string system_map_path = kernel_build_tree + "/System.map-" + kernel_release;
+   ifstream system_map;
+ 
+   system_map.open(system_map_path.c_str(), ifstream::in);
+   if (! system_map.is_open())
+     {
+-      if (verbose > 1)
+-	clog << _F("Kernel symbol table %s unavailable, (%s)",
+-		   system_map_path.c_str(), strerror(errno)) << endl;
+-
+-      system_map_path = sysroot + "/boot/System.map-" + kernel_release;
+-      system_map.clear();
+-      system_map.open(system_map_path.c_str(), ifstream::in);
+-      if (! system_map.is_open())
+         {
+ 	  if (verbose > 1)
+ 	    clog << _F("Kernel symbol table %s unavailable, (%s)",
-- 
2.28.0

[PATCH 7/8] kernel: consider vmlinux a build artifact, so systemtap can use it

[Yann Dirson]

From: Yann Dirson <yann@blade-group.com>

Cross-building a systemtap module requires access to debug information.
---
 meta/classes/kernel.bbclass | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 48135b3d41..fd2355bb68 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -495,6 +495,10 @@ do_shared_workdir () {
 		cp signing_key.* $kerneldir/
 	fi
 
+        # Copy vmlinux, needed by recipes depending on kernel debuginfo,
+        # like systemtap modules
+        cp vmlinux $kerneldir/
+
 	# We can also copy over all the generated files and avoid special cases
 	# like version.h, but we've opted to keep this small until file creep starts
 	# to happen
-- 
2.28.0

[PATCH 8/8] New systemtap.bb class, and systemtap-demo.bb to illustrate it.

[Yann Dirson]

From: Yann Dirson <yann@blade-group.com>

---
 meta/classes/systemtap.bbclass                | 74 +++++++++++++++++++
 .../systemtap/systemtap-demo_git.bb           | 71 ++++++++++++++++++
 2 files changed, 145 insertions(+)
 create mode 100644 meta/classes/systemtap.bbclass
 create mode 100644 meta/recipes-kernel/systemtap/systemtap-demo_git.bb

diff --git a/meta/classes/systemtap.bbclass b/meta/classes/systemtap.bbclass
new file mode 100644
index 0000000000..928202c1ff
--- /dev/null
+++ b/meta/classes/systemtap.bbclass
@@ -0,0 +1,74 @@
+# A bbclass to cross-build systemtap (.stp) scripts and install them
+# suitably for on-target usage with staprun.  All compiled .ko files
+# are installed directly in /lib/modules/${KERNEL_VERSION}/systemtap/
+# so their names must be unique.
+
+# FIXME:
+# - cannot handle building a probe several times with different args
+
+# verbosity flags easy to override and amend
+STAP_FLAGS = "-v"
+#STAP_FLAGS += "-B V=1"
+
+# set to "1" to avoid failing on first error
+STAP_WARNONLY = "0"
+
+# Should be absolute (use ${S}, ${WORKDIR}, etc)
+# .stp scripts to be compiled and installed.
+STAP_SCRIPTS ??= ""
+# subset of STAP_SCRIPTS not to compile (exclusion from glob pattern)
+STAP_EXCLUDE_SCRIPTS ??= ""
+
+STAP_OUT ??= "${B}/systemtap"
+
+inherit module-base
+
+DEPENDS += "systemtap-native"
+
+STAPCMD = "stap -p4 ${STAP_FLAGS}"
+STAPCMD += "-a '${ARCH}' -B CROSS_COMPILE='${CROSS_COMPILE}'"
+STAPCMD += "-r '${STAGING_KERNEL_BUILDDIR}'"
+
+# FIXME: most of those should be in the scope of systemtap-native
+STAPCMD += "--sysroot='${RECIPE_SYSROOT}'"
+STAPCMD += "-R '${RECIPE_SYSROOT_NATIVE}${datadir}/systemtap/runtime'"
+STAPCMD += "-I '${RECIPE_SYSROOT_NATIVE}${datadir}/systemtap/tapset'"
+
+## FIXME workaround LDFLAGS not being made for ld
+TARGET_LDFLAGS="-O1 --hash-style=gnu --as-needed"
+
+do_compile_append() {
+    rm -rf ${STAP_OUT}
+    mkdir -p ${STAP_OUT}
+    # expand glob patterns
+    stap_scripts=$(echo ${STAP_SCRIPTS})
+    stap_exclude_scripts=$(echo ${STAP_EXCLUDE_SCRIPTS})
+    for s in ${stap_scripts}; do
+        m=$(basename ${s%.stp})
+        m=$(printf $m | tr -c '[:alnum:]' _)
+        case " ${stap_exclude_scripts} " in
+          *\ $s\ *) continue ;;
+        esac
+        eval extraflags=\${STAP_EXTRAFLAGS_$m}
+        (
+            cd ${STAP_OUT}
+            [ ! -e "$m.ko" ] || bbwarn "compiling $s: $m.ko already exists"
+            if ! ${STAPCMD} ${extraflags} -m "$m" "$s"; then
+                if [ "${STAP_WARNONLY}" = 1 ]; then
+                    bbwarn "build failure for $s"
+                else
+                    bbfatal "build failure for $s"
+                fi
+            fi
+        )
+    done
+}
+
+do_install_append() {
+    DST=${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/systemtap
+    install -d -m755 ${DST}
+    install -m644 ${STAP_OUT}/*.ko ${DST}/
+}
+
+FILES_${PN} += "${nonarch_base_libdir}/modules"
+RRECOMMENDS_${PN} += "systemtap-runtime"
diff --git a/meta/recipes-kernel/systemtap/systemtap-demo_git.bb b/meta/recipes-kernel/systemtap/systemtap-demo_git.bb
new file mode 100644
index 0000000000..f6b6c40cc1
--- /dev/null
+++ b/meta/recipes-kernel/systemtap/systemtap-demo_git.bb
@@ -0,0 +1,71 @@
+require systemtap_git.inc
+
+# FIXME just to avoid looking for patches we don't care about
+SRC_URI = "git://sourceware.org/git/systemtap.git"
+
+inherit systemtap
+
+STAP_SCRIPTS = "\
+ ${S}/testsuite/systemtap.examples/*/*.stp \
+"
+
+# Many scripts have specific needs, so we don't really care if they don't build,
+# and there are too many of them to exclude them manually, so we just tell the build
+# to issue warnings on build failures:
+# - specific kernel symbols, depending on specific kernel config options
+# - files on disk for userspace instrumentation
+# - FIXME but also plain oversights of this packaging (kernel.trace ?)
+STAP_WARNONLY = "1"
+
+export STAP_EXTRAFLAGS_badname = "-g"
+export STAP_EXTRAFLAGS_keyhack = "-g"
+export STAP_EXTRAFLAGS_eatmydata = "-g"
+export STAP_EXTRAFLAGS_ttyspy = "-g"
+
+# those require parameters
+STAP_EXCLUDE_SCRIPTS = "\
+ ${S}/testsuite/systemtap.examples/general/callgraph.stp \
+ ${S}/testsuite/systemtap.examples/general/eventcount.stp \
+ ${S}/testsuite/systemtap.examples/general/func_time_stats.stp \
+ ${S}/testsuite/systemtap.examples/general/para-callgraph-verbose.stp \
+ ${S}/testsuite/systemtap.examples/general/para-callgraph.stp \
+ ${S}/testsuite/systemtap.examples/general/py2example.stp \
+ ${S}/testsuite/systemtap.examples/general/py3example.stp \
+ ${S}/testsuite/systemtap.examples/general/regex.stp \
+ ${S}/testsuite/systemtap.examples/general/sizeof.stp \
+ ${S}/testsuite/systemtap.examples/general/sizeof_interactive.stp \
+ ${S}/testsuite/systemtap.examples/general/varwatch.stp \
+ ${S}/testsuite/systemtap.examples/general/watchdog.stp \
+ ${S}/testsuite/systemtap.examples/general/whythefail.stp \
+ ${S}/testsuite/systemtap.examples/io/deviceseeks.stp \
+ ${S}/testsuite/systemtap.examples/io/inodewatch.stp \
+ ${S}/testsuite/systemtap.examples/io/inodewatch2.stp \
+ ${S}/testsuite/systemtap.examples/io/iostat-scsi.stp \
+ ${S}/testsuite/systemtap.examples/io/traceio2.stp \
+ ${S}/testsuite/systemtap.examples/locks/bkl.stp \
+ ${S}/testsuite/systemtap.examples/memory/hw_watch_addr.stp \
+ ${S}/testsuite/systemtap.examples/memory/hw_watch_sym.stp \
+ ${S}/testsuite/systemtap.examples/network/connect_stat.stp \
+ ${S}/testsuite/systemtap.examples/network/netfilter_drop.stp \
+ ${S}/testsuite/systemtap.examples/process/chng_cpu.stp \
+ ${S}/testsuite/systemtap.examples/process/errsnoop.stp \
+ ${S}/testsuite/systemtap.examples/process/migrate.stp \
+ ${S}/testsuite/systemtap.examples/process/pfiles.stp \
+ ${S}/testsuite/systemtap.examples/process/plimit.stp \
+ ${S}/testsuite/systemtap.examples/process/psig.stp \
+ ${S}/testsuite/systemtap.examples/process/pstrace_exec.stp \
+ ${S}/testsuite/systemtap.examples/process/sigmon.stp \
+ ${S}/testsuite/systemtap.examples/profiling/fntimes.stp \
+ ${S}/testsuite/systemtap.examples/profiling/functioncallcount.stp \
+ ${S}/testsuite/systemtap.examples/profiling/graphcall.stp \
+ ${S}/testsuite/systemtap.examples/profiling/linetimes.stp \
+ ${S}/testsuite/systemtap.examples/profiling/periodic.stp \
+ ${S}/testsuite/systemtap.examples/profiling/sched_switch.stp \
+"
+
+# FIXME what's the problem with those ?
+STAP_EXCLUDE_SCRIPTS += "\
+ ${S}/testsuite/systemtap.examples/interrupt/interrupts-by-dev.stp \
+"
+
+B = "${WORKDIR}/build"
-- 
2.28.0

Re: [OE-core] [PATCH 8/8] New systemtap.bb class, and systemtap-demo.bb to illustrate it.

[Richard Purdie]

On Mon, 2020-09-21 at 16:15 +0200, Yann Dirson wrote:
> From: Yann Dirson <yann@blade-group.com>
> 
> ---
>  meta/classes/systemtap.bbclass                | 74 +++++++++++++++++++
>  .../systemtap/systemtap-demo_git.bb           | 71 ++++++++++++++++++
>  2 files changed, 145 insertions(+)
>  create mode 100644 meta/classes/systemtap.bbclass
>  create mode 100644 meta/recipes-kernel/systemtap/systemtap-demo_git.bb

I'm afraid this patch is going to throw warnings all over the
autobuilder, e.g.:

https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/2488

(and there will be more).

We are trying to ensure all warnings are solved so that any that do
show are real issues and this patch would significantly distract from
that.

I'm also wary that we're now in feature freeze for release so perhaps
these changes should wait until 3.3?

Cheers,

Richard

Re: [OE-core] [PATCH 8/8] New systemtap.bb class, and systemtap-demo.bb to illustrate it.

[Yann Dirson]

[-- Attachment #1: Type: text/plain, Size: 1403 bytes --]

Le lun. 21 sept. 2020 à 21:46, Richard Purdie <
richard.purdie@linuxfoundation.org> a écrit :

> On Mon, 2020-09-21 at 16:15 +0200, Yann Dirson wrote:
> > From: Yann Dirson <yann@blade-group.com>
> >
> > ---
> >  meta/classes/systemtap.bbclass                | 74 +++++++++++++++++++
> >  .../systemtap/systemtap-demo_git.bb           | 71 ++++++++++++++++++
> >  2 files changed, 145 insertions(+)
> >  create mode 100644 meta/classes/systemtap.bbclass
> >  create mode 100644 meta/recipes-kernel/systemtap/systemtap-demo_git.bb
>
> I'm afraid this patch is going to throw warnings all over the
> autobuilder, e.g.:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/2488
>
> (and there will be more).
>
> We are trying to ensure all warnings are solved so that any that do
> show are real issues and this patch would significantly distract from
> that.
>

OK, we could default to built just the handful of probes that should always
build,
and give instructions for those willing to use the current behaviour.

I'm also wary that we're now in feature freeze for release so perhaps
> these changes should wait until 3.3?
>

OK, I can reroll a series with just the fixes.

For the rest should I just wait for the gatesgarth branch to be created and
resubmit then ?

-- 
Yann Dirson <yann@blade-group.com>
Blade / Shadow -- http://shadow.tech

[-- Attachment #2: Type: text/html, Size: 2602 bytes --]

Re: [OE-core] [PATCH 8/8] New systemtap.bb class, and systemtap-demo.bb to illustrate it.

[Richard Purdie]

On Mon, 2020-09-21 at 23:23 +0200, Yann Dirson wrote:
> 
> 
> Le lun. 21 sept. 2020 à 21:46, Richard Purdie <
> richard.purdie@linuxfoundation.org> a écrit :
> > On Mon, 2020-09-21 at 16:15 +0200, Yann Dirson wrote:
> > > From: Yann Dirson <yann@blade-group.com>
> > > 
> > > ---
> > >  meta/classes/systemtap.bbclass                | 74
> > +++++++++++++++++++
> > >  .../systemtap/systemtap-demo_git.bb           | 71
> > ++++++++++++++++++
> > >  2 files changed, 145 insertions(+)
> > >  create mode 100644 meta/classes/systemtap.bbclass
> > >  create mode 100644 meta/recipes-kernel/systemtap/systemtap-
> > demo_git.bb
> > 
> > I'm afraid this patch is going to throw warnings all over the
> > autobuilder, e.g.:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/2488
> > 
> > (and there will be more).
> > 
> > We are trying to ensure all warnings are solved so that any that do
> > show are real issues and this patch would significantly distract
> > from
> > that.
> 
> OK, we could default to built just the handful of probes that should
> always build, and give instructions for those willing to use the
> current behaviour.

I think that is going to be safer and we could grow into more examples
if/as/when we van build them.

> 
> > I'm also wary that we're now in feature freeze for release so
> > perhaps
> > these changes should wait until 3.3?
> 
> OK, I can reroll a series with just the fixes.

That would be great.

> For the rest should I just wait for the gatesgarth branch to be
> created and resubmit then ?

Once we start taking new work on master, yes.

Also, for a new recipe like this we need a maintainers entry, else we
see:

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1377

Cheers,

Richard

Re: [OE-core] [PATCH 8/8] New systemtap.bb class, and systemtap-demo.bb to illustrate it.

[Bruce Ashfield]

On Mon, Sep 21, 2020 at 6:51 PM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Mon, 2020-09-21 at 23:23 +0200, Yann Dirson wrote:
> >
> >
> > Le lun. 21 sept. 2020 à 21:46, Richard Purdie <
> > richard.purdie@linuxfoundation.org> a écrit :
> > > On Mon, 2020-09-21 at 16:15 +0200, Yann Dirson wrote:
> > > > From: Yann Dirson <yann@blade-group.com>
> > > >
> > > > ---
> > > >  meta/classes/systemtap.bbclass                | 74
> > > +++++++++++++++++++
> > > >  .../systemtap/systemtap-demo_git.bb           | 71
> > > ++++++++++++++++++
> > > >  2 files changed, 145 insertions(+)
> > > >  create mode 100644 meta/classes/systemtap.bbclass
> > > >  create mode 100644 meta/recipes-kernel/systemtap/systemtap-
> > > demo_git.bb
> > >
> > > I'm afraid this patch is going to throw warnings all over the
> > > autobuilder, e.g.:
> > >
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/2488
> > >
> > > (and there will be more).
> > >
> > > We are trying to ensure all warnings are solved so that any that do
> > > show are real issues and this patch would significantly distract
> > > from
> > > that.
> >
> > OK, we could default to built just the handful of probes that should
> > always build, and give instructions for those willing to use the
> > current behaviour.
>
> I think that is going to be safer and we could grow into more examples
> if/as/when we van build them.

Also, as the person that usually picks up systemtap breakage first
on our new kernel versions .. we need to make sure that the reworked
recipe and class pass the oe-selftest for stap.

It is doing on target compilation, so ideally, that test should be expanded
to test bot the cross and on-target variants.

Cheers,

Bruce

>
> >
> > > I'm also wary that we're now in feature freeze for release so
> > > perhaps
> > > these changes should wait until 3.3?
> >
> > OK, I can reroll a series with just the fixes.
>
> That would be great.
>
> > For the rest should I just wait for the gatesgarth branch to be
> > created and resubmit then ?
>
> Once we start taking new work on master, yes.
>
> Also, for a new recipe like this we need a maintainers entry, else we
> see:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1377
>
> Cheers,
>
> Richard
>
>
>
> 
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [PATCH 8/8] New systemtap.bb class, and systemtap-demo.bb to illustrate it.

[Yann Dirson]

[-- Attachment #1: Type: text/plain, Size: 3606 bytes --]

Le mar. 22 sept. 2020 à 02:41, Bruce Ashfield <bruce.ashfield@gmail.com> a
écrit :

> On Mon, Sep 21, 2020 at 6:51 PM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> >
> > On Mon, 2020-09-21 at 23:23 +0200, Yann Dirson wrote:
> > >
> > >
> > > Le lun. 21 sept. 2020 à 21:46, Richard Purdie <
> > > richard.purdie@linuxfoundation.org> a écrit :
> > > > On Mon, 2020-09-21 at 16:15 +0200, Yann Dirson wrote:
> > > > > From: Yann Dirson <yann@blade-group.com>
> > > > >
> > > > > ---
> > > > >  meta/classes/systemtap.bbclass                | 74
> > > > +++++++++++++++++++
> > > > >  .../systemtap/systemtap-demo_git.bb           | 71
> > > > ++++++++++++++++++
> > > > >  2 files changed, 145 insertions(+)
> > > > >  create mode 100644 meta/classes/systemtap.bbclass
> > > > >  create mode 100644 meta/recipes-kernel/systemtap/systemtap-
> > > > demo_git.bb
> > > >
> > > > I'm afraid this patch is going to throw warnings all over the
> > > > autobuilder, e.g.:
> > > >
> > > >
> https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/2488
> > > >
> > > > (and there will be more).
> > > >
> > > > We are trying to ensure all warnings are solved so that any that do
> > > > show are real issues and this patch would significantly distract
> > > > from
> > > > that.
> > >
> > > OK, we could default to built just the handful of probes that should
> > > always build, and give instructions for those willing to use the
> > > current behaviour.
> >
> > I think that is going to be safer and we could grow into more examples
> > if/as/when we van build them.
>
> Also, as the person that usually picks up systemtap breakage first
> on our new kernel versions .. we need to make sure that the reworked
> recipe and class pass the oe-selftest for stap.
>

AFAICT, if we exclude this last patch, the only impact on tests is that
those
testing modules with crosstap (which I only learnt about when looking for
those tests)
and pulling the full systemtap package should be changed to pull
systemtap-runtime
instead.

Now adding tests for the modules built with systemtap.class is a different
thing.

It is doing on target compilation, so ideally, that test should be expanded
> to test bot the cross and on-target variants.
>

The on-target compilation is already tested by
meta/lib/oeqa/runtime/cases/stap.py,
and there are crosstap tests in
meta/lib/oeqa/selftest/cases/runtime_test.py.

I have started to look at adding a test for modules cross-built by
systemtap.bbclass,
but as it takes ages to run those tests it seems it would make sense to
move this last
patch into a series of its own.



>
> Cheers,
>
> Bruce
>
> >
> > >
> > > > I'm also wary that we're now in feature freeze for release so
> > > > perhaps
> > > > these changes should wait until 3.3?
> > >
> > > OK, I can reroll a series with just the fixes.
> >
> > That would be great.
> >
> > > For the rest should I just wait for the gatesgarth branch to be
> > > created and resubmit then ?
> >
> > Once we start taking new work on master, yes.
> >
> > Also, for a new recipe like this we need a maintainers entry, else we
> > see:
> >
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1377
> >
> > Cheers,
> >
> > Richard
> >
> >
> >
> > 
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>


-- 
Yann Dirson <yann@blade-group.com>
Blade / Shadow -- http://shadow.tech

[-- Attachment #2: Type: text/html, Size: 5772 bytes --]

Re: [OE-core] [PATCH 8/8] New systemtap.bb class, and systemtap-demo.bb to illustrate it.

[Bruce Ashfield]

On Thu, Nov 12, 2020 at 12:02 PM Yann Dirson
<yann.dirson@blade-group.com> wrote:
>
>
>
> Le mar. 22 sept. 2020 à 02:41, Bruce Ashfield <bruce.ashfield@gmail.com> a écrit :
>>
>> On Mon, Sep 21, 2020 at 6:51 PM Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>> >
>> > On Mon, 2020-09-21 at 23:23 +0200, Yann Dirson wrote:
>> > >
>> > >
>> > > Le lun. 21 sept. 2020 à 21:46, Richard Purdie <
>> > > richard.purdie@linuxfoundation.org> a écrit :
>> > > > On Mon, 2020-09-21 at 16:15 +0200, Yann Dirson wrote:
>> > > > > From: Yann Dirson <yann@blade-group.com>
>> > > > >
>> > > > > ---
>> > > > >  meta/classes/systemtap.bbclass                | 74
>> > > > +++++++++++++++++++
>> > > > >  .../systemtap/systemtap-demo_git.bb           | 71
>> > > > ++++++++++++++++++
>> > > > >  2 files changed, 145 insertions(+)
>> > > > >  create mode 100644 meta/classes/systemtap.bbclass
>> > > > >  create mode 100644 meta/recipes-kernel/systemtap/systemtap-
>> > > > demo_git.bb
>> > > >
>> > > > I'm afraid this patch is going to throw warnings all over the
>> > > > autobuilder, e.g.:
>> > > >
>> > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/2488
>> > > >
>> > > > (and there will be more).
>> > > >
>> > > > We are trying to ensure all warnings are solved so that any that do
>> > > > show are real issues and this patch would significantly distract
>> > > > from
>> > > > that.
>> > >
>> > > OK, we could default to built just the handful of probes that should
>> > > always build, and give instructions for those willing to use the
>> > > current behaviour.
>> >
>> > I think that is going to be safer and we could grow into more examples
>> > if/as/when we van build them.
>>
>> Also, as the person that usually picks up systemtap breakage first
>> on our new kernel versions .. we need to make sure that the reworked
>> recipe and class pass the oe-selftest for stap.
>
>
> AFAICT, if we exclude this last patch, the only impact on tests is that those
> testing modules with crosstap (which I only learnt about when looking for those tests)
> and pulling the full systemtap package should be changed to pull systemtap-runtime
> instead.
>
> Now adding tests for the modules built with systemtap.class is a different thing.
>
>> It is doing on target compilation, so ideally, that test should be expanded
>> to test bot the cross and on-target variants.
>
>
> The on-target compilation is already tested by meta/lib/oeqa/runtime/cases/stap.py,
> and there are crosstap tests in meta/lib/oeqa/selftest/cases/runtime_test.py.

I'm aware of that ..... I'm the one that runs them against most new kernels :D

Bruce

>
> I have started to look at adding a test for modules cross-built by systemtap.bbclass,
> but as it takes ages to run those tests it seems it would make sense to move this last
> patch into a series of its own.
>
>
>>
>>
>> Cheers,
>>
>> Bruce
>>
>> >
>> > >
>> > > > I'm also wary that we're now in feature freeze for release so
>> > > > perhaps
>> > > > these changes should wait until 3.3?
>> > >
>> > > OK, I can reroll a series with just the fixes.
>> >
>> > That would be great.
>> >
>> > > For the rest should I just wait for the gatesgarth branch to be
>> > > created and resubmit then ?
>> >
>> > Once we start taking new work on master, yes.
>> >
>> > Also, for a new recipe like this we need a maintainers entry, else we
>> > see:
>> >
>> > https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/1377
>> >
>> > Cheers,
>> >
>> > Richard
>> >
>> >
>> >
>> > 
>> >
>>
>>
>> --
>> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> thee at its end
>> - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
> --
> Yann Dirson <yann@blade-group.com>
> Blade / Shadow -- http://shadow.tech
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II