Build break with patch for "qemuboot, xen-image-minimal: enable runqemu for qemuarm64 Xen images"

Build break with patch for "qemuboot, xen-image-minimal: enable runqemu for qemuarm64 Xen images"

[Jon Mason]

I'm building xen-image-minimal for gem5-arm64 machine (in the meta-arm
layer) and the recent patch to add qemuarm64 support is breaking my
builds.  The patch in question is:

19347a7c4e4c qemuboot, xen-image-minimal: enable runqemu for qemuarm64
Xen images

You can see an example of the breakage here:
https://gitlab.com/jonmason00/meta-arm/-/jobs/1468242014

The problematic line is
diff --git a/recipes-extended/images/xen-image-minimal.bb
b/recipes-extended/images/xen-image-minimal.bb
index 6733801cca5f..ca6d26836c42 100644
--- a/recipes-extended/images/xen-image-minimal.bb
+++ b/recipes-extended/images/xen-image-minimal.bb
@@ -31,7 +31,7 @@ XEN_PCIBACK_MODULE_x86-64 = "kernel-module-xen-pciback"

 LICENSE = "MIT"

-inherit core-image
+inherit core-image qemuboot-xen-defaults qemuboot-xen-dtb

Reverting this line "fixes" my problem.  It seems odd to add qemu
stuff for machines that don't need it.  Is it right to add qemu stuff
to an image.bb?


Thanks,
Jon

Re: [meta-virtualization] Build break with patch for "qemuboot, xen-image-minimal: enable runqemu for qemuarm64 Xen images"

[Christopher Clark]

On Sun, Aug 1, 2021 at 8:36 AM Jon Mason <jdmason@kudzu.us> wrote:
>
> I'm building xen-image-minimal for gem5-arm64 machine (in the meta-arm
> layer) and the recent patch to add qemuarm64 support is breaking my
> builds.  The patch in question is:
>
> 19347a7c4e4c qemuboot, xen-image-minimal: enable runqemu for qemuarm64
> Xen images
>
> You can see an example of the breakage here:
> https://gitlab.com/jonmason00/meta-arm/-/jobs/1468242014

Hi Jon

I'm sorry that my patch caused trouble and thanks for sending your
report. I now understand the issue.

> The problematic line is
> diff --git a/recipes-extended/images/xen-image-minimal.bb
> b/recipes-extended/images/xen-image-minimal.bb
> index 6733801cca5f..ca6d26836c42 100644
> --- a/recipes-extended/images/xen-image-minimal.bb
> +++ b/recipes-extended/images/xen-image-minimal.bb
> @@ -31,7 +31,7 @@ XEN_PCIBACK_MODULE_x86-64 = "kernel-module-xen-pciback"
>
>  LICENSE = "MIT"
>
> -inherit core-image
> +inherit core-image qemuboot-xen-defaults qemuboot-xen-dtb
>
> Reverting this line "fixes" my problem.

Would you be able to try applying this patch instead? I believe it
will resolve the issue.

diff --git a/classes/qemuboot-xen-dtb.bbclass b/classes/qemuboot-xen-dtb.bbclass
index 08f9b02..2d37e91 100644
--- a/classes/qemuboot-xen-dtb.bbclass
+++ b/classes/qemuboot-xen-dtb.bbclass
@@ -176,7 +176,7 @@ do_write_xen_qemuboot_dtb() {
     # Not all architectures qemuboot with a device tree binary, so check
     # to see if one is needed. This allows this bbclass file to be used
     # in the same image recipe for multiple architectures.
-    if [ -n "${QB_DTB}" ] ; then
+    if [ -n "${QB_DTB}" ] && [ -n "${QB_SYSTEM_NAME}" ] ; then
         generate_xen_qemuboot_dtb
     fi
 }


> It seems odd to add qemu
> stuff for machines that don't need it.  Is it right to add qemu stuff
> to an image.bb?

I believe that it is reasonable to do so as it enables support for
testing the image (or images derived from it) even if the intended
final target is non-qemu hardware. I have been working on using this
qemu support to enable the OpenEmbedded QA framework to launch a Xen
image and execute standard and Xen-specific OEQA test cases within it,
eg. via: bitbake -c testimage xen-image-minimal
It should be a valuable capability for images that are built even for
non-qemu machine targets. You can also see within this layer that the
xvisor image recipe similarly chooses to include qemu-specific items.

thanks,

Christopher

Re: [meta-virtualization] Build break with patch for "qemuboot, xen-image-minimal: enable runqemu for qemuarm64 Xen images"

[Jon Mason]

On Sun, Aug 1, 2021 at 9:03 PM Christopher Clark
<christopher.w.clark@gmail.com> wrote:
>
> On Sun, Aug 1, 2021 at 8:36 AM Jon Mason <jdmason@kudzu.us> wrote:
> >
> > I'm building xen-image-minimal for gem5-arm64 machine (in the meta-arm
> > layer) and the recent patch to add qemuarm64 support is breaking my
> > builds.  The patch in question is:
> >
> > 19347a7c4e4c qemuboot, xen-image-minimal: enable runqemu for qemuarm64
> > Xen images
> >
> > You can see an example of the breakage here:
> > https://gitlab.com/jonmason00/meta-arm/-/jobs/1468242014
>
> Hi Jon
>
> I'm sorry that my patch caused trouble and thanks for sending your
> report. I now understand the issue.

It happens, and I appreciate your rapid response (on a Sunday!).  This
is the reason why we have our nightly Gitlab CI runs :)

>
> > The problematic line is
> > diff --git a/recipes-extended/images/xen-image-minimal.bb
> > b/recipes-extended/images/xen-image-minimal.bb
> > index 6733801cca5f..ca6d26836c42 100644
> > --- a/recipes-extended/images/xen-image-minimal.bb
> > +++ b/recipes-extended/images/xen-image-minimal.bb
> > @@ -31,7 +31,7 @@ XEN_PCIBACK_MODULE_x86-64 = "kernel-module-xen-pciback"
> >
> >  LICENSE = "MIT"
> >
> > -inherit core-image
> > +inherit core-image qemuboot-xen-defaults qemuboot-xen-dtb
> >
> > Reverting this line "fixes" my problem.
>
> Would you be able to try applying this patch instead? I believe it
> will resolve the issue.
>
> diff --git a/classes/qemuboot-xen-dtb.bbclass b/classes/qemuboot-xen-dtb.bbclass
> index 08f9b02..2d37e91 100644
> --- a/classes/qemuboot-xen-dtb.bbclass
> +++ b/classes/qemuboot-xen-dtb.bbclass
> @@ -176,7 +176,7 @@ do_write_xen_qemuboot_dtb() {
>      # Not all architectures qemuboot with a device tree binary, so check
>      # to see if one is needed. This allows this bbclass file to be used
>      # in the same image recipe for multiple architectures.
> -    if [ -n "${QB_DTB}" ] ; then
> +    if [ -n "${QB_DTB}" ] && [ -n "${QB_SYSTEM_NAME}" ] ; then
>          generate_xen_qemuboot_dtb
>      fi
>  }

This does indeed allow it to compile without issue.  Please push this
as soon as possible.

>
> > It seems odd to add qemu
> > stuff for machines that don't need it.  Is it right to add qemu stuff
> > to an image.bb?
>
> I believe that it is reasonable to do so as it enables support for
> testing the image (or images derived from it) even if the intended
> final target is non-qemu hardware. I have been working on using this
> qemu support to enable the OpenEmbedded QA framework to launch a Xen
> image and execute standard and Xen-specific OEQA test cases within it,
> eg. via: bitbake -c testimage xen-image-minimal
> It should be a valuable capability for images that are built even for
> non-qemu machine targets. You can also see within this layer that the
> xvisor image recipe similarly chooses to include qemu-specific items.

Poor wording on my part.  I'm not saying QEMU has no value in testing,
I was trying to say that I don't see similar things in other layers
that use qemu (lazily looking at poky as my example).  This could very
well be the right way of doing it, and I'm just ignorant of it.

All of that being said, I am interested in what you are doing here.  I
think running CI on qemu xen arm64 as part of our development process
could be very beneficial.  In our current setup, we're running
testimage on qemu machines.  Do you have anything similar planned for
this machine?

Thanks,
Jon

>
> thanks,
>
> Christopher

Re: [meta-virtualization] Build break with patch for "qemuboot, xen-image-minimal: enable runqemu for qemuarm64 Xen images"

[Christopher Clark]

On Mon, Aug 2, 2021 at 6:54 AM Jon Mason <jdmason@kudzu.us> wrote:
>
> On Sun, Aug 1, 2021 at 9:03 PM Christopher Clark
> <christopher.w.clark@gmail.com> wrote:
> >
> > On Sun, Aug 1, 2021 at 8:36 AM Jon Mason <jdmason@kudzu.us> wrote:
> > >
> > > I'm building xen-image-minimal for gem5-arm64 machine (in the meta-arm
> > > layer) and the recent patch to add qemuarm64 support is breaking my
> > > builds.  The patch in question is:
> > >
> > > 19347a7c4e4c qemuboot, xen-image-minimal: enable runqemu for qemuarm64
> > > Xen images
> > >
> > > You can see an example of the breakage here:
> > > https://gitlab.com/jonmason00/meta-arm/-/jobs/1468242014
> >
> > Hi Jon
> >
> > I'm sorry that my patch caused trouble and thanks for sending your
> > report. I now understand the issue.
>
> It happens, and I appreciate your rapid response (on a Sunday!).  This
> is the reason why we have our nightly Gitlab CI runs :)
>
> >
> > > The problematic line is
> > > diff --git a/recipes-extended/images/xen-image-minimal.bb
> > > b/recipes-extended/images/xen-image-minimal.bb
> > > index 6733801cca5f..ca6d26836c42 100644
> > > --- a/recipes-extended/images/xen-image-minimal.bb
> > > +++ b/recipes-extended/images/xen-image-minimal.bb
> > > @@ -31,7 +31,7 @@ XEN_PCIBACK_MODULE_x86-64 = "kernel-module-xen-pciback"
> > >
> > >  LICENSE = "MIT"
> > >
> > > -inherit core-image
> > > +inherit core-image qemuboot-xen-defaults qemuboot-xen-dtb
> > >
> > > Reverting this line "fixes" my problem.
> >
> > Would you be able to try applying this patch instead? I believe it
> > will resolve the issue.
> >
> > diff --git a/classes/qemuboot-xen-dtb.bbclass b/classes/qemuboot-xen-dtb.bbclass
> > index 08f9b02..2d37e91 100644
> > --- a/classes/qemuboot-xen-dtb.bbclass
> > +++ b/classes/qemuboot-xen-dtb.bbclass
> > @@ -176,7 +176,7 @@ do_write_xen_qemuboot_dtb() {
> >      # Not all architectures qemuboot with a device tree binary, so check
> >      # to see if one is needed. This allows this bbclass file to be used
> >      # in the same image recipe for multiple architectures.
> > -    if [ -n "${QB_DTB}" ] ; then
> > +    if [ -n "${QB_DTB}" ] && [ -n "${QB_SYSTEM_NAME}" ] ; then
> >          generate_xen_qemuboot_dtb
> >      fi
> >  }
>
> This does indeed allow it to compile without issue.  Please push this
> as soon as possible.

Excellent, thanks!

>
> >
> > > It seems odd to add qemu
> > > stuff for machines that don't need it.  Is it right to add qemu stuff
> > > to an image.bb?
> >
> > I believe that it is reasonable to do so as it enables support for
> > testing the image (or images derived from it) even if the intended
> > final target is non-qemu hardware. I have been working on using this
> > qemu support to enable the OpenEmbedded QA framework to launch a Xen
> > image and execute standard and Xen-specific OEQA test cases within it,
> > eg. via: bitbake -c testimage xen-image-minimal
> > It should be a valuable capability for images that are built even for
> > non-qemu machine targets. You can also see within this layer that the
> > xvisor image recipe similarly chooses to include qemu-specific items.
>
> Poor wording on my part.  I'm not saying QEMU has no value in testing,
> I was trying to say that I don't see similar things in other layers
> that use qemu (lazily looking at poky as my example).  This could very
> well be the right way of doing it, and I'm just ignorant of it.

I think the default OE qemu settings are OK for most images so no
adjustments are needed, but those that include a hypervisor, or at
least a type-1 hypervisor, may require some tuning of the settings to
enable a successful boot. It gets even more complicated for Xen since
the x86 and Arm platform boot configurations are currently rather
different.

> All of that being said, I am interested in what you are doing here.  I
> think running CI on qemu xen arm64 as part of our development process
> could be very beneficial.  In our current setup, we're running
> testimage on qemu machines.  Do you have anything similar planned for
> this machine?

I've recently added recipes for the Xen Test Framework to
meta-virtualization as preparation towards enabling that to run in CI
with OEQA (ie. testimage). XTF is currently only for x86 platforms but
I am aware of development efforts to enable running XTF on Arm too, so
it should be useful to you when that lands. I have been working on the
integration needed for XTF to run as an OEQA test case and I do have a
working early version, but doing that work has surfaced some
challenges with the Xen toolstack that I need to resolve first, so
that's my current focus.

I'm very happy to hear that you're already running CI on the Xen
recipes and image - please do report any issues that you find.

best,

Christopher

>
> Thanks,
> Jon
>
> >
> > thanks,
> >
> > Christopher