[meta-zephyr][PATCH 1/1] zephyr-qemuboot.bbclass: Fix runqemu dependency on qemu native sysroot

[meta-zephyr][PATCH 1/1] zephyr-qemuboot.bbclass: Fix runqemu dependency on qemu native sysroot

[Andrei Gherzan]

From: Andrei Gherzan <andrei.gherzan@huawei.com>

The runqemu script depends on having the native sysroot populated for
the qemu recipes. Add the required dependency to the mix.

Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
---
 classes/zephyr-qemuboot.bbclass | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/classes/zephyr-qemuboot.bbclass b/classes/zephyr-qemuboot.bbclass
index 5ac1c86..f508b45 100644
--- a/classes/zephyr-qemuboot.bbclass
+++ b/classes/zephyr-qemuboot.bbclass
@@ -35,3 +35,22 @@ python do_bootconf_write() {
 }
 
 addtask do_bootconf_write before do_build after do_deploy
+
+# The runqemu script requires the native sysroot populated for the qemu
+# recipes. Usually, this is pulled in by a do_image dependency (see
+# baremetal-helloworld_git, for example), but in this case, there is no such
+# task, so we hook in the dependency to do_bootconf_write. This also ensures
+# that builds from sstate will also have this requirement satisfied.
+python () {
+    # do_addto_recipe_sysroot doesnt exist for all recipes, but we need it to have
+    # /usr/bin on recipe-sysroot (qemu) populated
+    def extraimage_getdepends(task):
+        deps = ""
+        for dep in (d.getVar('EXTRA_IMAGEDEPENDS') or "").split():
+        # Make sure we only add it for qemu
+            if 'qemu' in dep:
+                deps += " %s:%s" % (dep, task)
+        return deps
+    d.appendVarFlag('do_bootconf_write', 'depends', extraimage_getdepends('do_addto_recipe_sysroot'))
+    d.appendVarFlag('do_bootconf_write', 'depends', extraimage_getdepends('do_populate_sysroot'))
+}
-- 
2.31.1

Re: [yocto] [meta-zephyr][PATCH 1/1] zephyr-qemuboot.bbclass: Fix runqemu dependency on qemu native sysroot

[Ross Burton]

This is breaking our CI:

ERROR: qemu-system-native-5.2.0-r0 do_addto_recipe_sysroot: Error
executing a python function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:extend_recipe_sysroot(d)
     0003:
File: '/builds/yocto-poc/meta-arm/work/poky/meta/classes/staging.bbclass',
lineno: 502, function: extend_recipe_sysroot
     0498:                continue
     0499:
     0500:        msg_adding.append(c)
     0501:
 *** 0502:        os.symlink(c + "." + taskhash, depdir + "/" + c)
     0503:
     0504:        manifest, d2 = oe.sstatesig.find_sstate_manifest(c,
setscenedeps[dep][2], "populate_sysroot", d, multilibs)
     0505:        if d2 is not d:
     0506:            # If we don't do this, the recipe sysroot will
be placed in the wrong WORKDIR for multilibs
Exception: FileNotFoundError: [Errno 2] No such file or directory:
'zlib-native.0f0b3e4d16f9ad46dd8609d8c899a834104f5b572a4a5438ccccd1db88d67e97'
-> '/builds/yocto-poc/meta-arm/work/build/tmp/work/aarch64-linux/qemu-system-native/5.2.0-r0/recipe-sysroot-native/installeddeps/zlib-native'
ERROR: Logfile of failure stored in:
/builds/yocto-poc/meta-arm/work/build/tmp/work/aarch64-linux/qemu-system-native/5.2.0-r0/temp/log.do_addto_recipe_sysroot.7593
ERROR: Task (/builds/yocto-poc/meta-arm/work/poky/meta/recipes-devtools/qemu/qemu-system-native_5.2.0.bb:do_addto_recipe_sysroot)
failed with exit code '1'
ERROR: qemu-system-native-5.2.0-r0 do_rm_work: Execution of
'/builds/yocto-poc/meta-arm/work/build/tmp/work/aarch64-linux/qemu-system-native/5.2.0-r0/temp/run.do_rm_work.7583'
failed with exit code 1:
rm: cannot remove 'recipe-sysroot-native': Directory not empty

Can this be reverted whilst this is debugged?

Ross

On Fri, 7 May 2021 at 11:09, Andrei Gherzan <andrei@gherzan.com> wrote:
>
> From: Andrei Gherzan <andrei.gherzan@huawei.com>
>
> The runqemu script depends on having the native sysroot populated for
> the qemu recipes. Add the required dependency to the mix.
>
> Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
> ---
>  classes/zephyr-qemuboot.bbclass | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
>
> diff --git a/classes/zephyr-qemuboot.bbclass b/classes/zephyr-qemuboot.bbclass
> index 5ac1c86..f508b45 100644
> --- a/classes/zephyr-qemuboot.bbclass
> +++ b/classes/zephyr-qemuboot.bbclass
> @@ -35,3 +35,22 @@ python do_bootconf_write() {
>  }
>
>  addtask do_bootconf_write before do_build after do_deploy
> +
> +# The runqemu script requires the native sysroot populated for the qemu
> +# recipes. Usually, this is pulled in by a do_image dependency (see
> +# baremetal-helloworld_git, for example), but in this case, there is no such
> +# task, so we hook in the dependency to do_bootconf_write. This also ensures
> +# that builds from sstate will also have this requirement satisfied.
> +python () {
> +    # do_addto_recipe_sysroot doesnt exist for all recipes, but we need it to have
> +    # /usr/bin on recipe-sysroot (qemu) populated
> +    def extraimage_getdepends(task):
> +        deps = ""
> +        for dep in (d.getVar('EXTRA_IMAGEDEPENDS') or "").split():
> +        # Make sure we only add it for qemu
> +            if 'qemu' in dep:
> +                deps += " %s:%s" % (dep, task)
> +        return deps
> +    d.appendVarFlag('do_bootconf_write', 'depends', extraimage_getdepends('do_addto_recipe_sysroot'))
> +    d.appendVarFlag('do_bootconf_write', 'depends', extraimage_getdepends('do_populate_sysroot'))
> +}
> --
> 2.31.1
>
>
> 
>

Re: [yocto] [meta-zephyr][PATCH 1/1] zephyr-qemuboot.bbclass: Fix runqemu dependency on qemu native sysroot

[Ross Burton]

Debugged, the patch is broken.

Using a little tool I have:

Task qemu-system-native-5.2.0-r0:do_addto_recipe_sysroot failed
Active tasks are:
 virglrenderer-native-0.9.1-r0:do_rm_work
 binutils-cross-arm-2.36.1-r0:do_patch
 qemu-system-native-5.2.0-r0:do_addto_recipe_sysroot
 qemu-system-native-5.2.0-r0:do_rm_work
 libepoxy-native-1.5.5-r0:do_rm_work
 gcc-source-11.1.0-11.1.0-r0:do_unpack

Note how qemu is simultaneously adding itself to the sysroot, whilst
rm_work is running and deleting the sysroot.

This patch isn't compatible with rm_work and should be reverted.

Ross

On Thu, 13 May 2021 at 13:06, Ross Burton <ross@burtonini.com> wrote:
>
> This is breaking our CI:
>
> ERROR: qemu-system-native-5.2.0-r0 do_addto_recipe_sysroot: Error
> executing a python function in exec_python_func() autogenerated:
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
>      0001:
>  *** 0002:extend_recipe_sysroot(d)
>      0003:
> File: '/builds/yocto-poc/meta-arm/work/poky/meta/classes/staging.bbclass',
> lineno: 502, function: extend_recipe_sysroot
>      0498:                continue
>      0499:
>      0500:        msg_adding.append(c)
>      0501:
>  *** 0502:        os.symlink(c + "." + taskhash, depdir + "/" + c)
>      0503:
>      0504:        manifest, d2 = oe.sstatesig.find_sstate_manifest(c,
> setscenedeps[dep][2], "populate_sysroot", d, multilibs)
>      0505:        if d2 is not d:
>      0506:            # If we don't do this, the recipe sysroot will
> be placed in the wrong WORKDIR for multilibs
> Exception: FileNotFoundError: [Errno 2] No such file or directory:
> 'zlib-native.0f0b3e4d16f9ad46dd8609d8c899a834104f5b572a4a5438ccccd1db88d67e97'
> -> '/builds/yocto-poc/meta-arm/work/build/tmp/work/aarch64-linux/qemu-system-native/5.2.0-r0/recipe-sysroot-native/installeddeps/zlib-native'
> ERROR: Logfile of failure stored in:
> /builds/yocto-poc/meta-arm/work/build/tmp/work/aarch64-linux/qemu-system-native/5.2.0-r0/temp/log.do_addto_recipe_sysroot.7593
> ERROR: Task (/builds/yocto-poc/meta-arm/work/poky/meta/recipes-devtools/qemu/qemu-system-native_5.2.0.bb:do_addto_recipe_sysroot)
> failed with exit code '1'
> ERROR: qemu-system-native-5.2.0-r0 do_rm_work: Execution of
> '/builds/yocto-poc/meta-arm/work/build/tmp/work/aarch64-linux/qemu-system-native/5.2.0-r0/temp/run.do_rm_work.7583'
> failed with exit code 1:
> rm: cannot remove 'recipe-sysroot-native': Directory not empty
>
> Can this be reverted whilst this is debugged?
>
> Ross
>
> On Fri, 7 May 2021 at 11:09, Andrei Gherzan <andrei@gherzan.com> wrote:
> >
> > From: Andrei Gherzan <andrei.gherzan@huawei.com>
> >
> > The runqemu script depends on having the native sysroot populated for
> > the qemu recipes. Add the required dependency to the mix.
> >
> > Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
> > ---
> >  classes/zephyr-qemuboot.bbclass | 19 +++++++++++++++++++
> >  1 file changed, 19 insertions(+)
> >
> > diff --git a/classes/zephyr-qemuboot.bbclass b/classes/zephyr-qemuboot.bbclass
> > index 5ac1c86..f508b45 100644
> > --- a/classes/zephyr-qemuboot.bbclass
> > +++ b/classes/zephyr-qemuboot.bbclass
> > @@ -35,3 +35,22 @@ python do_bootconf_write() {
> >  }
> >
> >  addtask do_bootconf_write before do_build after do_deploy
> > +
> > +# The runqemu script requires the native sysroot populated for the qemu
> > +# recipes. Usually, this is pulled in by a do_image dependency (see
> > +# baremetal-helloworld_git, for example), but in this case, there is no such
> > +# task, so we hook in the dependency to do_bootconf_write. This also ensures
> > +# that builds from sstate will also have this requirement satisfied.
> > +python () {
> > +    # do_addto_recipe_sysroot doesnt exist for all recipes, but we need it to have
> > +    # /usr/bin on recipe-sysroot (qemu) populated
> > +    def extraimage_getdepends(task):
> > +        deps = ""
> > +        for dep in (d.getVar('EXTRA_IMAGEDEPENDS') or "").split():
> > +        # Make sure we only add it for qemu
> > +            if 'qemu' in dep:
> > +                deps += " %s:%s" % (dep, task)
> > +        return deps
> > +    d.appendVarFlag('do_bootconf_write', 'depends', extraimage_getdepends('do_addto_recipe_sysroot'))
> > +    d.appendVarFlag('do_bootconf_write', 'depends', extraimage_getdepends('do_populate_sysroot'))
> > +}
> > --
> > 2.31.1
> >
> >
> > 
> >

Re: [yocto] [meta-zephyr][PATCH 1/1] zephyr-qemuboot.bbclass: Fix runqemu dependency on qemu native sysroot

[Andrei Gherzan]

Hi,

On Thu, 13 May 2021, at 13:14, Ross Burton wrote:
> Debugged, the patch is broken.
> 
> Using a little tool I have:
> 
> Task qemu-system-native-5.2.0-r0:do_addto_recipe_sysroot failed
> Active tasks are:
>  virglrenderer-native-0.9.1-r0:do_rm_work
>  binutils-cross-arm-2.36.1-r0:do_patch
>  qemu-system-native-5.2.0-r0:do_addto_recipe_sysroot
>  qemu-system-native-5.2.0-r0:do_rm_work
>  libepoxy-native-1.5.5-r0:do_rm_work
>  gcc-source-11.1.0-11.1.0-r0:do_unpack
> 
> Note how qemu is simultaneously adding itself to the sysroot, whilst
> rm_work is running and deleting the sysroot.
> 
> This patch isn't compatible with rm_work and should be reverted.
> 

I'll look into it.

-- 
Andrei Gherzan 
gpg: rsa4096/D4D94F67AD0E9640

Re: [yocto] [meta-zephyr][PATCH 1/1] zephyr-qemuboot.bbclass: Fix runqemu dependency on qemu native sysroot

[Andrei Gherzan]

[-- Attachment #1: Type: text/plain, Size: 949 bytes --]

Hi Ross,

On Thu, 13 May 2021, at 13:37, Andrei Gherzan wrote:
> Hi,
> 
> On Thu, 13 May 2021, at 13:14, Ross Burton wrote:
> > Debugged, the patch is broken.
> > 
> > Using a little tool I have:
> > 
> > Task qemu-system-native-5.2.0-r0:do_addto_recipe_sysroot failed
> > Active tasks are:
> >  virglrenderer-native-0.9.1-r0:do_rm_work
> >  binutils-cross-arm-2.36.1-r0:do_patch
> >  qemu-system-native-5.2.0-r0:do_addto_recipe_sysroot
> >  qemu-system-native-5.2.0-r0:do_rm_work
> >  libepoxy-native-1.5.5-r0:do_rm_work
> >  gcc-source-11.1.0-11.1.0-r0:do_unpack
> > 
> > Note how qemu is simultaneously adding itself to the sysroot, whilst
> > rm_work is running and deleting the sysroot.
> > 
> > This patch isn't compatible with rm_work and should be reverted.
> > 
> 
> I'll look into it.


https://lists.yoctoproject.org/g/yocto/message/53508 should fix it.

-- 
Andrei Gherzan 
gpg: rsa4096/D4D94F67AD0E9640

[-- Attachment #2: Type: text/html, Size: 1819 bytes --]