linux-yocto virtualization requires virtio.scc from yocto-kernel-cache

linux-yocto virtualization requires virtio.scc from yocto-kernel-cache

[Kyle Russell]

[-- Attachment #1: Type: text/plain, Size: 1220 bytes --]

Hello,

meta-virtualization's linux-yocto_virtualization.inc includes
cfg/virtio.scc in KERNEL_FEATURES, which is a dependency from
yocto-kernel-cache.  However, we do not use yocto-kernel-cache with our
custom linux-* recipe, so we're hitting a metadata error because of a
missing virtio.scc.  It seems like the creation of linux-%.bbappend (in
f2f36a8061c600b35b5f0ce1599d59f1d144a3aa) was intentional to match kernel
recipes other than linux-yocto*, which we could benefit from, but the lack
of yocto-kernel-cache is problematic with virtio.scc.

Can meta-virtualization provide virtio.scc like it does for the other
kernel features to allow decoupling from yocto-kernel-cache?

meta-virtualization used to allow this decoupling by configuring the virtio
feature without the yocto-kernel-cache in
f1c03f390e95b63d5c2dfba9140d1b07c4d17acf, but that was restructured in
21d8bcdb791a1ea766a3e7e7663c7b6d49bc861a.  The dependency was not required
unless you were using linux-yocto, but as of f2f36a8061c60 is now required
for all linux-% recipes.  (We just now noticed because our linux recipe has
recently been updated to match a kernel version supported by
meta-virtualization.

Thanks,

Kyle

[-- Attachment #2: Type: text/html, Size: 1354 bytes --]

Re: linux-yocto virtualization requires virtio.scc from yocto-kernel-cache

[Diego Sueiro]

On Fri, Jul 23, 2021 at 05:03 AM, Kyle Russell wrote:

>
> Hello,
> 
> meta-virtualization's linux-yocto_virtualization.inc includes
> cfg/virtio.scc in KERNEL_FEATURES, which is a dependency from
> yocto-kernel-cache.  However, we do not use yocto-kernel-cache with our
> custom linux-* recipe, so we're hitting a metadata error because of a
> missing virtio.scc.  It seems like the creation of linux-%.bbappend (in
> f2f36a8061c600b35b5f0ce1599d59f1d144a3aa) was intentional to match kernel
> recipes other than linux-yocto*, which we could benefit from, but the lack
> of yocto-kernel-cache is problematic with virtio.scc.
> 
> Can meta-virtualization provide virtio.scc like it does for the other
> kernel features to allow decoupling from yocto-kernel-cache?
> 

Your question made me remember about the docker.scc. Which is being maintained
both on meta-virt and yocto-kernel-cache. And linux-yocto_virtualization.inc makes
usage of the one from meta-virt, which feels strange to me since it is a
**linux-yocto**_virtualization.inc file that supposedly extends linux-yocto*.bb from
oe-core.
IMHO, for scc/cfg files we should try to rely on a single canonical source, and in this
case, I believe is yocto-kernel-cache repo. Maintaining the same file for the same
purpose in two different places brings the risk of diversion and increases maintenance
effort to keep the instances aligned.
Also, the yocto-kernel-cache gives us the ability to easily manage .scc files
per kernel version.

> meta-virtualization used to allow this decoupling by configuring the virtio
> feature without the yocto-kernel-cache in
> f1c03f390e95b63d5c2dfba9140d1b07c4d17acf, but that was restructured in
> 21d8bcdb791a1ea766a3e7e7663c7b6d49bc861a.  The dependency was not required
> unless you were using linux-yocto, but as of f2f36a8061c60 is now required
> for all linux-% recipes.  (We just now noticed because our linux recipe has
> recently been updated to match a kernel version supported by
> meta-virtualization.
> 
> Thanks,
> 
> Kyle
>

Re: [meta-virtualization] linux-yocto virtualization requires virtio.scc from yocto-kernel-cache

[Bruce Ashfield]

On Fri, Jul 23, 2021 at 2:32 AM Diego Sueiro <diego.sueiro@arm.com> wrote:
>
> On Fri, Jul 23, 2021 at 05:03 AM, Kyle Russell wrote:
>
> >
> > Hello,
> >
> > meta-virtualization's linux-yocto_virtualization.inc includes
> > cfg/virtio.scc in KERNEL_FEATURES, which is a dependency from
> > yocto-kernel-cache.  However, we do not use yocto-kernel-cache with our
> > custom linux-* recipe, so we're hitting a metadata error because of a
> > missing virtio.scc.  It seems like the creation of linux-%.bbappend (in
> > f2f36a8061c600b35b5f0ce1599d59f1d144a3aa) was intentional to match kernel
> > recipes other than linux-yocto*, which we could benefit from, but the lack
> > of yocto-kernel-cache is problematic with virtio.scc.
> >
> > Can meta-virtualization provide virtio.scc like it does for the other
> > kernel features to allow decoupling from yocto-kernel-cache?
> >
>
> Your question made me remember about the docker.scc. Which is being maintained
> both on meta-virt and yocto-kernel-cache. And linux-yocto_virtualization.inc makes
> usage of the one from meta-virt, which feels strange to me since it is a
> **linux-yocto**_virtualization.inc file that supposedly extends linux-yocto*.bb from
> oe-core.
> IMHO, for scc/cfg files we should try to rely on a single canonical source, and in this
> case, I believe is yocto-kernel-cache repo. Maintaining the same file for the same
> purpose in two different places brings the risk of diversion and increases maintenance
> effort to keep the instances aligned.
> Also, the yocto-kernel-cache gives us the ability to easily manage .scc files
> per kernel version.

Yes .. precisely! I'll reply to both email here, so I'm mixing replies
to Diego and
Kyle throughout.

We had a similar discussion in #meta-virt about a month ago, when some AGL
builds were running into a similar issue. The layout of the includes
and requirement
for the fragment were intentional, as was the broad nature of the bbappend
matching.

Without those fragments, I end up debugging end users' subtle runtime errors for
any number of packages in meta-virt, so having issues pop up when the
fragments can't be found, while annoying, is a useful flag for potential issues.

I resisted making a copy of virtio.cfg available in meta-virt for exactly the
reason Diego is bringing up. Maintaining them in two places is not ideal, and
having them centralized is the goal.

To solve the short term issue as we were talking about in #meta-virt, I did
a couple of things:

A "hack": https://git.yoctoproject.org/cgit/cgit.cgi/meta-virtualization/commit/?h=master-wip&id=40cae93f631e77286de7fa79101abee2f6c04618

But if you use that, you'll need to ensure via other means that you get the
right configuration enabled.

And something that I had wanted to do for a while, which is the actual
solution I'm heading towards.

https://git.yoctoproject.org/cgit/cgit.cgi/meta-virtualization/log/?h=master-test

see commits: https://git.yoctoproject.org/cgit/cgit.cgi/meta-virtualization/commit/?h=master-test&id=44db8ee6ddf7c9eb1b5ffb22e9b83b3840997da9
and https://git.yoctoproject.org/cgit/cgit.cgi/meta-virtualization/commit/?h=master-test&id=8db75f2809318cfb4dbe1c6873c5a6bcfefc4d8c

I still haven't done extensive testing with a lot of different kernels, hence
why I haven't moved them to master yet.

But with those two commits, I can remove all of the fragments in the layer
and rely on the one true reference in the kernel-cache.

Bruce

>
> > meta-virtualization used to allow this decoupling by configuring the virtio
> > feature without the yocto-kernel-cache in
> > f1c03f390e95b63d5c2dfba9140d1b07c4d17acf, but that was restructured in
> > 21d8bcdb791a1ea766a3e7e7663c7b6d49bc861a.  The dependency was not required
> > unless you were using linux-yocto, but as of f2f36a8061c60 is now required
> > for all linux-% recipes.  (We just now noticed because our linux recipe has
> > recently been updated to match a kernel version supported by
> > meta-virtualization.
> >
> > Thanks,
> >
> > Kyle
> >
>
> 
>


--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] linux-yocto virtualization requires virtio.scc from yocto-kernel-cache

[Robert Berger]

Hi,

I am building with my own kernel meta data and that's my little patch:

https://github.com/RobertBerger/manifests/blob/master/meta-virtualization-master/0001-include-virtio.scc.patch

Regards,

Robert

Re: [meta-virtualization] linux-yocto virtualization requires virtio.scc from yocto-kernel-cache

[Kyle Russell]

[-- Attachment #1: Type: text/plain, Size: 4835 bytes --]

Thanks, Bruce!  That's very helpful.  I agree having two copies isn't
ideal, so it's nice to know an approach that allows decoupling has already
been considered.  We'll take a look at these and consider our options.

Kyle

On Fri, Jul 23, 2021 at 1:02 PM Bruce Ashfield <bruce.ashfield@gmail.com>
wrote:

> On Fri, Jul 23, 2021 at 2:32 AM Diego Sueiro <diego.sueiro@arm.com> wrote:
> >
> > On Fri, Jul 23, 2021 at 05:03 AM, Kyle Russell wrote:
> >
> > >
> > > Hello,
> > >
> > > meta-virtualization's linux-yocto_virtualization.inc includes
> > > cfg/virtio.scc in KERNEL_FEATURES, which is a dependency from
> > > yocto-kernel-cache.  However, we do not use yocto-kernel-cache with our
> > > custom linux-* recipe, so we're hitting a metadata error because of a
> > > missing virtio.scc.  It seems like the creation of linux-%.bbappend (in
> > > f2f36a8061c600b35b5f0ce1599d59f1d144a3aa) was intentional to match
> kernel
> > > recipes other than linux-yocto*, which we could benefit from, but the
> lack
> > > of yocto-kernel-cache is problematic with virtio.scc.
> > >
> > > Can meta-virtualization provide virtio.scc like it does for the other
> > > kernel features to allow decoupling from yocto-kernel-cache?
> > >
> >
> > Your question made me remember about the docker.scc. Which is being
> maintained
> > both on meta-virt and yocto-kernel-cache. And
> linux-yocto_virtualization.inc makes
> > usage of the one from meta-virt, which feels strange to me since it is a
> > **linux-yocto**_virtualization.inc file that supposedly extends
> linux-yocto*.bb from
> > oe-core.
> > IMHO, for scc/cfg files we should try to rely on a single canonical
> source, and in this
> > case, I believe is yocto-kernel-cache repo. Maintaining the same file
> for the same
> > purpose in two different places brings the risk of diversion and
> increases maintenance
> > effort to keep the instances aligned.
> > Also, the yocto-kernel-cache gives us the ability to easily manage .scc
> files
> > per kernel version.
>
> Yes .. precisely! I'll reply to both email here, so I'm mixing replies
> to Diego and
> Kyle throughout.
>
> We had a similar discussion in #meta-virt about a month ago, when some AGL
> builds were running into a similar issue. The layout of the includes
> and requirement
> for the fragment were intentional, as was the broad nature of the bbappend
> matching.
>
> Without those fragments, I end up debugging end users' subtle runtime
> errors for
> any number of packages in meta-virt, so having issues pop up when the
> fragments can't be found, while annoying, is a useful flag for potential
> issues.
>
> I resisted making a copy of virtio.cfg available in meta-virt for exactly
> the
> reason Diego is bringing up. Maintaining them in two places is not ideal,
> and
> having them centralized is the goal.
>
> To solve the short term issue as we were talking about in #meta-virt, I did
> a couple of things:
>
> A "hack":
> https://git.yoctoproject.org/cgit/cgit.cgi/meta-virtualization/commit/?h=master-wip&id=40cae93f631e77286de7fa79101abee2f6c04618
>
> But if you use that, you'll need to ensure via other means that you get the
> right configuration enabled.
>
> And something that I had wanted to do for a while, which is the actual
> solution I'm heading towards.
>
>
> https://git.yoctoproject.org/cgit/cgit.cgi/meta-virtualization/log/?h=master-test
>
> see commits:
> https://git.yoctoproject.org/cgit/cgit.cgi/meta-virtualization/commit/?h=master-test&id=44db8ee6ddf7c9eb1b5ffb22e9b83b3840997da9
> and
> https://git.yoctoproject.org/cgit/cgit.cgi/meta-virtualization/commit/?h=master-test&id=8db75f2809318cfb4dbe1c6873c5a6bcfefc4d8c
>
> I still haven't done extensive testing with a lot of different kernels,
> hence
> why I haven't moved them to master yet.
>
> But with those two commits, I can remove all of the fragments in the layer
> and rely on the one true reference in the kernel-cache.
>
> Bruce
>
> >
> > > meta-virtualization used to allow this decoupling by configuring the
> virtio
> > > feature without the yocto-kernel-cache in
> > > f1c03f390e95b63d5c2dfba9140d1b07c4d17acf, but that was restructured in
> > > 21d8bcdb791a1ea766a3e7e7663c7b6d49bc861a.  The dependency was not
> required
> > > unless you were using linux-yocto, but as of f2f36a8061c60 is now
> required
> > > for all linux-% recipes.  (We just now noticed because our linux
> recipe has
> > > recently been updated to match a kernel version supported by
> > > meta-virtualization.
> > >
> > > Thanks,
> > >
> > > Kyle
> > >
> >
> >
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 6336 bytes --]