[PATCH] qemu: Split the qemu package

[PATCH] qemu: Split the qemu package

[mingli.yu]

From: Mingli Yu <mingli.yu@windriver.com>

Currently all files packaged into one package such as qemu-7.2.0-*.rpm.
After the qemu package installed on the target, it will take up 464M
which includes not only the one matches the arch of the target but aslo
all available built qemu targets which set by QEMU_TARGETS.

Split the qemu package into qemu-7.2.0-*.rpm, qemu-aarch64-7.2.0*.rpm,
qemu-arm-7.2.0*.rpm, qemu-x86_64-7.2.0*.rpm and etc. And let user can only
choose the corresponding qemu arch package they want to install should ease
the concerns who cares much about the size in embedded device as it
decreases the qemu rpm(qemu-7.2.0*.rpm) size from about 65M to about 17M
and the size of the extracted qemu RPM decreased from about 464M to about
248M.

Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
---
 meta/recipes-devtools/qemu/qemu.inc | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
index a87dee5c99..7302d63747 100644
--- a/meta/recipes-devtools/qemu/qemu.inc
+++ b/meta/recipes-devtools/qemu/qemu.inc
@@ -230,6 +230,25 @@ INSANE_SKIP:${PN} = "arch"
 
 FILES:${PN} += "${datadir}/icons"
 
+python(){
+    allarchs = d.getVar('QEMU_TARGETS').split()
+    packages = d.getVar('PACKAGES').split()
+    pn = d.getVar('PN')
+
+    newpackages=[]
+    for arch in allarchs:
+        archpackage = 'qemu-' + arch
+        if archpackage not in packages:
+            newpackages.append(archpackage)
+        d.setVar('FILES:' + pn + "-" + arch, '${bindir}/qemu-' + arch)
+        d.appendVar('FILES:' + pn + "-" + arch, ' ' + '${bindir}/qemu-system-' + arch)
+        if arch == "mips":
+            d.appendVar('RDEPENDS:' + pn + '-' + arch, ' ' + 'bash')
+    packages = newpackages + packages
+    d.setVar('PACKAGES', ' '.join(packages))
+}
+
+
 # Put the guest agent in a separate package
 PACKAGES =+ "${PN}-guest-agent"
 SUMMARY:${PN}-guest-agent = "QEMU guest agent"
-- 
2.25.1

Re: [OE-core] [PATCH] qemu: Split the qemu package

[Richard Purdie]

On Tue, 2023-05-30 at 21:17 +0800, Yu, Mingli wrote:
> From: Mingli Yu <mingli.yu@windriver.com>
> 
> Currently all files packaged into one package such as qemu-7.2.0-*.rpm.
> After the qemu package installed on the target, it will take up 464M
> which includes not only the one matches the arch of the target but aslo
> all available built qemu targets which set by QEMU_TARGETS.
> 
> Split the qemu package into qemu-7.2.0-*.rpm, qemu-aarch64-7.2.0*.rpm,
> qemu-arm-7.2.0*.rpm, qemu-x86_64-7.2.0*.rpm and etc. And let user can only
> choose the corresponding qemu arch package they want to install should ease
> the concerns who cares much about the size in embedded device as it
> decreases the qemu rpm(qemu-7.2.0*.rpm) size from about 65M to about 17M
> and the size of the extracted qemu RPM decreased from about 464M to about
> 248M.
> 
> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> ---
>  meta/recipes-devtools/qemu/qemu.inc | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
> 
> diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
> index a87dee5c99..7302d63747 100644
> --- a/meta/recipes-devtools/qemu/qemu.inc
> +++ b/meta/recipes-devtools/qemu/qemu.inc
> @@ -230,6 +230,25 @@ INSANE_SKIP:${PN} = "arch"
>  
>  FILES:${PN} += "${datadir}/icons"
>  
> +python(){
> +    allarchs = d.getVar('QEMU_TARGETS').split()
> +    packages = d.getVar('PACKAGES').split()
> +    pn = d.getVar('PN')
> +
> +    newpackages=[]
> +    for arch in allarchs:
> +        archpackage = 'qemu-' + arch
> +        if archpackage not in packages:
> +            newpackages.append(archpackage)

Are any of the architectures already in packages? I'm a little curious
if there are and why?

> +        d.setVar('FILES:' + pn + "-" + arch, '${bindir}/qemu-' + arch)
> +        d.appendVar('FILES:' + pn + "-" + arch, ' ' + '${bindir}/qemu-system-' + arch)

Why set a variable, then append to it?

> +        if arch == "mips":
> +            d.appendVar('RDEPENDS:' + pn + '-' + arch, ' ' + 'bash')

May as well just do:

RDEPENDS:${PN}-mips += "bash"

?

> +    packages = newpackages + packages
> +    d.setVar('PACKAGES', ' '.join(packages))
> +}

Also, should the main qemu package pull in all the other subpackages?

Cheers,

Richard

Re: [OE-core] [PATCH] qemu: Split the qemu package

[Bruce Ashfield]

On Tue, May 30, 2023 at 9:17 AM Yu, Mingli <mingli.yu@eng.windriver.com> wrote:
>
> From: Mingli Yu <mingli.yu@windriver.com>
>
> Currently all files packaged into one package such as qemu-7.2.0-*.rpm.
> After the qemu package installed on the target, it will take up 464M
> which includes not only the one matches the arch of the target but aslo
> all available built qemu targets which set by QEMU_TARGETS.
>
> Split the qemu package into qemu-7.2.0-*.rpm, qemu-aarch64-7.2.0*.rpm,
> qemu-arm-7.2.0*.rpm, qemu-x86_64-7.2.0*.rpm and etc. And let user can only
> choose the corresponding qemu arch package they want to install should ease
> the concerns who cares much about the size in embedded device as it
> decreases the qemu rpm(qemu-7.2.0*.rpm) size from about 65M to about 17M
> and the size of the extracted qemu RPM decreased from about 464M to about
> 248M.

I've been splitting the qemu packages in meta-virtualization for years now.

I find the python more difficult to read than just using overrides, as
it is much more explicit.

Is there a reason why that approach didn't work in this scenario ?

As Richard mentioned, you should also pull in all the subpackages by
default with the main qemu package, to avoid regressions.

Also, in the experience of meta-virt, you only want this splitting on
target builds, not native/host.

Bruce

>
> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> ---
>  meta/recipes-devtools/qemu/qemu.inc | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
>
> diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
> index a87dee5c99..7302d63747 100644
> --- a/meta/recipes-devtools/qemu/qemu.inc
> +++ b/meta/recipes-devtools/qemu/qemu.inc
> @@ -230,6 +230,25 @@ INSANE_SKIP:${PN} = "arch"
>
>  FILES:${PN} += "${datadir}/icons"
>
> +python(){
> +    allarchs = d.getVar('QEMU_TARGETS').split()
> +    packages = d.getVar('PACKAGES').split()
> +    pn = d.getVar('PN')
> +
> +    newpackages=[]
> +    for arch in allarchs:
> +        archpackage = 'qemu-' + arch
> +        if archpackage not in packages:
> +            newpackages.append(archpackage)
> +        d.setVar('FILES:' + pn + "-" + arch, '${bindir}/qemu-' + arch)
> +        d.appendVar('FILES:' + pn + "-" + arch, ' ' + '${bindir}/qemu-system-' + arch)
> +        if arch == "mips":
> +            d.appendVar('RDEPENDS:' + pn + '-' + arch, ' ' + 'bash')
> +    packages = newpackages + packages
> +    d.setVar('PACKAGES', ' '.join(packages))
> +}
> +
> +
>  # Put the guest agent in a separate package
>  PACKAGES =+ "${PN}-guest-agent"
>  SUMMARY:${PN}-guest-agent = "QEMU guest agent"
> --
> 2.25.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#181922): https://lists.openembedded.org/g/openembedded-core/message/181922
> Mute This Topic: https://lists.openembedded.org/mt/99219254/1050810
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [PATCH] qemu: Split the qemu package

[Alexander Kanavin]

I might be missing something here, but can the free-form, anonymous
python code block be avoided? Don't we have PACKAGES_DYNAMIC for this
purpose?

Alex

On Tue, 30 May 2023 at 15:51, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> On Tue, May 30, 2023 at 9:17 AM Yu, Mingli <mingli.yu@eng.windriver.com> wrote:
> >
> > From: Mingli Yu <mingli.yu@windriver.com>
> >
> > Currently all files packaged into one package such as qemu-7.2.0-*.rpm.
> > After the qemu package installed on the target, it will take up 464M
> > which includes not only the one matches the arch of the target but aslo
> > all available built qemu targets which set by QEMU_TARGETS.
> >
> > Split the qemu package into qemu-7.2.0-*.rpm, qemu-aarch64-7.2.0*.rpm,
> > qemu-arm-7.2.0*.rpm, qemu-x86_64-7.2.0*.rpm and etc. And let user can only
> > choose the corresponding qemu arch package they want to install should ease
> > the concerns who cares much about the size in embedded device as it
> > decreases the qemu rpm(qemu-7.2.0*.rpm) size from about 65M to about 17M
> > and the size of the extracted qemu RPM decreased from about 464M to about
> > 248M.
>
> I've been splitting the qemu packages in meta-virtualization for years now.
>
> I find the python more difficult to read than just using overrides, as
> it is much more explicit.
>
> Is there a reason why that approach didn't work in this scenario ?
>
> As Richard mentioned, you should also pull in all the subpackages by
> default with the main qemu package, to avoid regressions.
>
> Also, in the experience of meta-virt, you only want this splitting on
> target builds, not native/host.
>
> Bruce
>
> >
> > Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> > ---
> >  meta/recipes-devtools/qemu/qemu.inc | 19 +++++++++++++++++++
> >  1 file changed, 19 insertions(+)
> >
> > diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
> > index a87dee5c99..7302d63747 100644
> > --- a/meta/recipes-devtools/qemu/qemu.inc
> > +++ b/meta/recipes-devtools/qemu/qemu.inc
> > @@ -230,6 +230,25 @@ INSANE_SKIP:${PN} = "arch"
> >
> >  FILES:${PN} += "${datadir}/icons"
> >
> > +python(){
> > +    allarchs = d.getVar('QEMU_TARGETS').split()
> > +    packages = d.getVar('PACKAGES').split()
> > +    pn = d.getVar('PN')
> > +
> > +    newpackages=[]
> > +    for arch in allarchs:
> > +        archpackage = 'qemu-' + arch
> > +        if archpackage not in packages:
> > +            newpackages.append(archpackage)
> > +        d.setVar('FILES:' + pn + "-" + arch, '${bindir}/qemu-' + arch)
> > +        d.appendVar('FILES:' + pn + "-" + arch, ' ' + '${bindir}/qemu-system-' + arch)
> > +        if arch == "mips":
> > +            d.appendVar('RDEPENDS:' + pn + '-' + arch, ' ' + 'bash')
> > +    packages = newpackages + packages
> > +    d.setVar('PACKAGES', ' '.join(packages))
> > +}
> > +
> > +
> >  # Put the guest agent in a separate package
> >  PACKAGES =+ "${PN}-guest-agent"
> >  SUMMARY:${PN}-guest-agent = "QEMU guest agent"
> > --
> > 2.25.1
> >
> >
> >
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#181924): https://lists.openembedded.org/g/openembedded-core/message/181924
> Mute This Topic: https://lists.openembedded.org/mt/99219254/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [PATCH] qemu: Split the qemu package

[Richard Purdie]

On Tue, 2023-05-30 at 16:33 +0200, Alexander Kanavin wrote:
> I might be missing something here, but can the free-form, anonymous
> python code block be avoided? Don't we have PACKAGES_DYNAMIC for this
> purpose?

PACKAGES_DYNAMIC is for when we can't predict the packages a recipe
might generate. A good example might be kernel modules.

You're right that we could add a do_split_packages() call to the qemu
recipe have have it generate these dynamically.

The downside would be the namespacing as dynamic packages need to have
specific namespaces (e.g. kernel-module-XXX). This means qemu-mips
wouldn't be an option (conflicts with non dynamic packages like qemu-
dbg).

We could use a more specific prefix like qemu-system-XXX and qemu-user-
XXX and use do_split_packages

I did also wonder about using more specific inline python for some of
this, things along the lines of:

PACKAGES += '${@" ".join("qemu-system-" + x for x in d.getVar('QEMU_TARGETS').split())}'

I'm also not a fan of the python code block.

We do use do_split_packages() in other recipes like gstreamer to handle
things like this.

Cheers,

Richard

Re: [OE-core] [PATCH] qemu: Split the qemu package

[Bruce Ashfield]

On Tue, May 30, 2023 at 10:54 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Tue, 2023-05-30 at 16:33 +0200, Alexander Kanavin wrote:
> > I might be missing something here, but can the free-form, anonymous
> > python code block be avoided? Don't we have PACKAGES_DYNAMIC for this
> > purpose?
>
> PACKAGES_DYNAMIC is for when we can't predict the packages a recipe
> might generate. A good example might be kernel modules.
>
> You're right that we could add a do_split_packages() call to the qemu
> recipe have have it generate these dynamically.
>
> The downside would be the namespacing as dynamic packages need to have
> specific namespaces (e.g. kernel-module-XXX). This means qemu-mips
> wouldn't be an option (conflicts with non dynamic packages like qemu-
> dbg).
>
> We could use a more specific prefix like qemu-system-XXX and qemu-user-
> XXX and use do_split_packages
>
> I did also wonder about using more specific inline python for some of
> this, things along the lines of:
>
> PACKAGES += '${@" ".join("qemu-system-" + x for x in d.getVar('QEMU_TARGETS').split())}'
>
> I'm also not a fan of the python code block.
>
> We do use do_split_packages() in other recipes like gstreamer to handle
> things like this.
>

And in case anyone hasn't looked it up, this is the meta-virt solution:

https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc

Which I'll have to re-work once (if) something lands in core.

It isn't suitable as-is, but it doesn't need any python code to suit
the on-target system emulation needs of meta-virt.

Bruce

> Cheers,
>
> Richard
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

[PATCH v2] qemu: Split the qemu package

[mingli.yu]

From: Mingli Yu <mingli.yu@windriver.com>

Currently all files as below packaged into one package such as
qemu-7.2.0-*.rpm. After the qemu package installed on the target,
it will take up about 464M which includes not only the one matches
the arch of the target but aslo all available built qemu targets
which set by QEMU_TARGETS.

 # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
 qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
 qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
 qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
 qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
 qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
 qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
 qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
 qemu-riscv32    qemu-system-arm      qemu-system-mips64el
 qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
 qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4

Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-aarch64-7.2.0*.rpm,
qemu-system-x86_64-7.2.0*.rpm and etc. And let user can only choose the
corresponding qemu arch package they want to install should ease the concerns
who cares much about the size in embedded device as it decreases the qemu rpm
(qemu-7.2.0*.rpm) size from about 65M to about 19M and the size of the
extracted qemu RPM decreased from about 464M to about 248M.

Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
---
 meta/recipes-devtools/qemu/qemu.inc | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
index a87dee5c99..e1ed3bbd4d 100644
--- a/meta/recipes-devtools/qemu/qemu.inc
+++ b/meta/recipes-devtools/qemu/qemu.inc
@@ -230,6 +230,11 @@ INSANE_SKIP:${PN} = "arch"
 
 FILES:${PN} += "${datadir}/icons"
 
+python populate_packages:prepend() {
+    archdir = d.expand('${bindir}/')
+    do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)', extra_depends='${PN}', prepend=True)
+}
+
 # Put the guest agent in a separate package
 PACKAGES =+ "${PN}-guest-agent"
 SUMMARY:${PN}-guest-agent = "QEMU guest agent"
-- 
2.25.1

Re: [OE-core] [PATCH] qemu: Split the qemu package

[Yu, Mingli]

On 5/30/23 23:09, Bruce Ashfield wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> On Tue, May 30, 2023 at 10:54 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>>
>> On Tue, 2023-05-30 at 16:33 +0200, Alexander Kanavin wrote:
>>> I might be missing something here, but can the free-form, anonymous
>>> python code block be avoided? Don't we have PACKAGES_DYNAMIC for this
>>> purpose?
>>
>> PACKAGES_DYNAMIC is for when we can't predict the packages a recipe
>> might generate. A good example might be kernel modules.
>>
>> You're right that we could add a do_split_packages() call to the qemu
>> recipe have have it generate these dynamically.
>>
>> The downside would be the namespacing as dynamic packages need to have
>> specific namespaces (e.g. kernel-module-XXX). This means qemu-mips
>> wouldn't be an option (conflicts with non dynamic packages like qemu-
>> dbg).
>>
>> We could use a more specific prefix like qemu-system-XXX and qemu-user-
>> XXX and use do_split_packages
>>
>> I did also wonder about using more specific inline python for some of
>> this, things along the lines of:
>>
>> PACKAGES += '${@" ".join("qemu-system-" + x for x in d.getVar('QEMU_TARGETS').split())}'
>>
>> I'm also not a fan of the python code block.
>>
>> We do use do_split_packages() in other recipes like gstreamer to handle
>> things like this.
>>
> 
> And in case anyone hasn't looked it up, this is the meta-virt solution:
> 
> https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc

Thanks! I did see this before I send out 
https://patchwork.yoctoproject.org/project/oe-core/patch/20230530131708.1916975-1-mingli.yu@eng.windriver.com. 


Considering to dynamically generate the sub-packages via QEMU_TARGETS, 
so I use a python block and don't need the change the code even 
QEMU_TARGETS has some change.

But I should let all sub-package to depend the qemu-7.2.0*.rpm.

Thanks,

> 
> Which I'll have to re-work once (if) something lands in core.
> 
> It isn't suitable as-is, but it doesn't need any python code to suit
> the on-target system emulation needs of meta-virt.
> 
> Bruce
> 
>> Cheers,
>>
>> Richard
>>
> 
> 
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [PATCH] qemu: Split the qemu package

[Bruce Ashfield]

On Thu, Jun 1, 2023 at 5:33 AM Yu, Mingli <mingli.yu@windriver.com> wrote:
>
>
>
> On 5/30/23 23:09, Bruce Ashfield wrote:
> > CAUTION: This email comes from a non Wind River email account!
> > Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> > On Tue, May 30, 2023 at 10:54 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> >>
> >> On Tue, 2023-05-30 at 16:33 +0200, Alexander Kanavin wrote:
> >>> I might be missing something here, but can the free-form, anonymous
> >>> python code block be avoided? Don't we have PACKAGES_DYNAMIC for this
> >>> purpose?
> >>
> >> PACKAGES_DYNAMIC is for when we can't predict the packages a recipe
> >> might generate. A good example might be kernel modules.
> >>
> >> You're right that we could add a do_split_packages() call to the qemu
> >> recipe have have it generate these dynamically.
> >>
> >> The downside would be the namespacing as dynamic packages need to have
> >> specific namespaces (e.g. kernel-module-XXX). This means qemu-mips
> >> wouldn't be an option (conflicts with non dynamic packages like qemu-
> >> dbg).
> >>
> >> We could use a more specific prefix like qemu-system-XXX and qemu-user-
> >> XXX and use do_split_packages
> >>
> >> I did also wonder about using more specific inline python for some of
> >> this, things along the lines of:
> >>
> >> PACKAGES += '${@" ".join("qemu-system-" + x for x in d.getVar('QEMU_TARGETS').split())}'
> >>
> >> I'm also not a fan of the python code block.
> >>
> >> We do use do_split_packages() in other recipes like gstreamer to handle
> >> things like this.
> >>
> >
> > And in case anyone hasn't looked it up, this is the meta-virt solution:
> >
> > https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc
>
> Thanks! I did see this before I send out
> https://patchwork.yoctoproject.org/project/oe-core/patch/20230530131708.1916975-1-mingli.yu@eng.windriver.com.
>
>
> Considering to dynamically generate the sub-packages via QEMU_TARGETS,
> so I use a python block and don't need the change the code even
> QEMU_TARGETS has some change.

QEMU_TARGETS is not something that changes very often.

I still think that there needs to be a way to opt-out of the split
packages, since there are some split requirements for virtualization,
etc, that don't follow the relatively simple pattern being introduced
here.

With the python code and the :prepend, I don't see how I'll be able to
opt out of it in meta-virtualization.

Also, the dependency I see in the v2 patch isn't what I'd expect to
keep existing use cases working.
i.e. For kernel modules we have the package that rdepends on all the
split packages, since there are expectations in places that all the
qemu-system packages are installed.

Bruce


>
> But I should let all sub-package to depend the qemu-7.2.0*.rpm.
>
> Thanks,
>
> >
> > Which I'll have to re-work once (if) something lands in core.
> >
> > It isn't suitable as-is, but it doesn't need any python code to suit
> > the on-target system emulation needs of meta-virt.
> >
> > Bruce
> >
> >> Cheers,
> >>
> >> Richard
> >>
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II



--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [PATCH v2] qemu: Split the qemu package

[Bruce Ashfield]

On Thu, Jun 1, 2023 at 5:28 AM Yu, Mingli <mingli.yu@eng.windriver.com> wrote:
>
> From: Mingli Yu <mingli.yu@windriver.com>
>
> Currently all files as below packaged into one package such as
> qemu-7.2.0-*.rpm. After the qemu package installed on the target,
> it will take up about 464M which includes not only the one matches
> the arch of the target but aslo all available built qemu targets
> which set by QEMU_TARGETS.
>
>  # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
>  qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
>  qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
>  qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
>  qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
>  qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
>  qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
>  qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
>  qemu-riscv32    qemu-system-arm      qemu-system-mips64el
>  qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
>  qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4
>
> Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-aarch64-7.2.0*.rpm,
> qemu-system-x86_64-7.2.0*.rpm and etc. And let user can only choose the
> corresponding qemu arch package they want to install should ease the concerns
> who cares much about the size in embedded device as it decreases the qemu rpm
> (qemu-7.2.0*.rpm) size from about 65M to about 19M and the size of the
> extracted qemu RPM decreased from about 464M to about 248M.
>
> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> ---
>  meta/recipes-devtools/qemu/qemu.inc | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
> index a87dee5c99..e1ed3bbd4d 100644
> --- a/meta/recipes-devtools/qemu/qemu.inc
> +++ b/meta/recipes-devtools/qemu/qemu.inc
> @@ -230,6 +230,11 @@ INSANE_SKIP:${PN} = "arch"
>
>  FILES:${PN} += "${datadir}/icons"
>
> +python populate_packages:prepend() {
> +    archdir = d.expand('${bindir}/')
> +    do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)', extra_depends='${PN}', prepend=True)

extra_depends='${PN}' is already the default IIRC.

Bruce

> +}
> +
>  # Put the guest agent in a separate package
>  PACKAGES =+ "${PN}-guest-agent"
>  SUMMARY:${PN}-guest-agent = "QEMU guest agent"
> --
> 2.25.1
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#182093): https://lists.openembedded.org/g/openembedded-core/message/182093
> Mute This Topic: https://lists.openembedded.org/mt/99260306/1050810
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

[PATCH v3] qemu: Split the qemu package

[mingli.yu]

From: Mingli Yu <mingli.yu@windriver.com>

Currently all files as below packaged into one package such as
qemu-7.2.0-*.rpm. After the qemu package installed on the target,
it will take up about 464M which includes not only the one matches
the arch of the target but aslo all available built qemu targets
which set by QEMU_TARGETS.

 # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
 qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
 qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
 qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
 qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
 qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
 qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
 qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
 qemu-riscv32    qemu-system-arm      qemu-system-mips64el
 qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
 qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4

Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-aarch64-7.2.0*.rpm,
qemu-system-x86_64-7.2.0*.rpm and etc. And let user can only choose the
corresponding qemu arch package they want to install should ease the concerns
who cares much about the size in embedded device as it decreases the qemu rpm
(qemu-7.2.0*.rpm) size from about 65M to about 19M and the size of the
extracted qemu RPM decreased from about 464M to about 248M.

Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
---
 meta/recipes-devtools/qemu/qemu.inc | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
index a87dee5c99..c6fd39aab6 100644
--- a/meta/recipes-devtools/qemu/qemu.inc
+++ b/meta/recipes-devtools/qemu/qemu.inc
@@ -230,6 +230,11 @@ INSANE_SKIP:${PN} = "arch"
 
 FILES:${PN} += "${datadir}/icons"
 
+python populate_packages:prepend() {
+    archdir = d.expand('${bindir}/')
+    do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)' , prepend=True)
+}
+
 # Put the guest agent in a separate package
 PACKAGES =+ "${PN}-guest-agent"
 SUMMARY:${PN}-guest-agent = "QEMU guest agent"
-- 
2.25.1

Re: [OE-core] [PATCH v3] qemu: Split the qemu package

[Bruce Ashfield]

On Thu, Jun 1, 2023 at 10:37 PM Yu, Mingli <mingli.yu@eng.windriver.com> wrote:
>
> From: Mingli Yu <mingli.yu@windriver.com>
>
> Currently all files as below packaged into one package such as
> qemu-7.2.0-*.rpm. After the qemu package installed on the target,
> it will take up about 464M which includes not only the one matches
> the arch of the target but aslo all available built qemu targets
> which set by QEMU_TARGETS.
>
>  # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
>  qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
>  qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
>  qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
>  qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
>  qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
>  qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
>  qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
>  qemu-riscv32    qemu-system-arm      qemu-system-mips64el
>  qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
>  qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4
>
> Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-aarch64-7.2.0*.rpm,
> qemu-system-x86_64-7.2.0*.rpm and etc. And let user can only choose the
> corresponding qemu arch package they want to install should ease the concerns
> who cares much about the size in embedded device as it decreases the qemu rpm
> (qemu-7.2.0*.rpm) size from about 65M to about 19M and the size of the
> extracted qemu RPM decreased from about 464M to about 248M.
>
> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> ---
>  meta/recipes-devtools/qemu/qemu.inc | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
> index a87dee5c99..c6fd39aab6 100644
> --- a/meta/recipes-devtools/qemu/qemu.inc
> +++ b/meta/recipes-devtools/qemu/qemu.inc
> @@ -230,6 +230,11 @@ INSANE_SKIP:${PN} = "arch"
>
>  FILES:${PN} += "${datadir}/icons"
>
> +python populate_packages:prepend() {
> +    archdir = d.expand('${bindir}/')
> +    do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)' , prepend=True)
> +}

I don't see a v3 changelog, but that does look like the unnecessary
default runtime dependency is gone, but what would be the method of
installing the packages if I have an image where I'd like the old
non-split functionality ?

Either a meta packages (qemu-system-all ?) or a way to override the
functionality (a variable around the do_split call ?) are options to
enable that sort of thing. As I mentioned before, there's a different
expected split of the packages in some scenarios, and it would be
ideal to not break those use cases.

Bruce

> +
>  # Put the guest agent in a separate package
>  PACKAGES =+ "${PN}-guest-agent"
>  SUMMARY:${PN}-guest-agent = "QEMU guest agent"
> --
> 2.25.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#182138): https://lists.openembedded.org/g/openembedded-core/message/182138
> Mute This Topic: https://lists.openembedded.org/mt/99279217/1050810
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [PATCH v3] qemu: Split the qemu package

[Richard Purdie]

On Fri, 2023-06-02 at 09:08 -0400, Bruce Ashfield wrote:
> On Thu, Jun 1, 2023 at 10:37 PM Yu, Mingli <mingli.yu@eng.windriver.com> wrote:
> > 
> > From: Mingli Yu <mingli.yu@windriver.com>
> > 
> > Currently all files as below packaged into one package such as
> > qemu-7.2.0-*.rpm. After the qemu package installed on the target,
> > it will take up about 464M which includes not only the one matches
> > the arch of the target but aslo all available built qemu targets
> > which set by QEMU_TARGETS.
> > 
> >  # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
> >  qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
> >  qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
> >  qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
> >  qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
> >  qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
> >  qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
> >  qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
> >  qemu-riscv32    qemu-system-arm      qemu-system-mips64el
> >  qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
> >  qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4
> > 
> > Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-aarch64-7.2.0*.rpm,
> > qemu-system-x86_64-7.2.0*.rpm and etc. And let user can only choose the
> > corresponding qemu arch package they want to install should ease the concerns
> > who cares much about the size in embedded device as it decreases the qemu rpm
> > (qemu-7.2.0*.rpm) size from about 65M to about 19M and the size of the
> > extracted qemu RPM decreased from about 464M to about 248M.
> > 
> > Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> > ---
> >  meta/recipes-devtools/qemu/qemu.inc | 5 +++++
> >  1 file changed, 5 insertions(+)
> > 
> > diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
> > index a87dee5c99..c6fd39aab6 100644
> > --- a/meta/recipes-devtools/qemu/qemu.inc
> > +++ b/meta/recipes-devtools/qemu/qemu.inc
> > @@ -230,6 +230,11 @@ INSANE_SKIP:${PN} = "arch"
> > 
> >  FILES:${PN} += "${datadir}/icons"
> > 
> > +python populate_packages:prepend() {
> > +    archdir = d.expand('${bindir}/')
> > +    do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)' , prepend=True)
> > +}
> 
> I don't see a v3 changelog, but that does look like the unnecessary
> default runtime dependency is gone, but what would be the method of
> installing the packages if I have an image where I'd like the old
> non-split functionality ?
> 
> Either a meta packages (qemu-system-all ?) or a way to override the
> functionality (a variable around the do_split call ?) are options to
> enable that sort of thing. As I mentioned before, there's a different
> expected split of the packages in some scenarios, and it would be
> ideal to not break those use cases.

This is quite a common need for things which call do_split_packages,
I'm starting to wonder if it should support some kind of "common"
package as a parameter which it would add dependencies to?

Cheers,

Richard

Re: [OE-core] [PATCH v3] qemu: Split the qemu package

[Yu, Mingli]

Hi Richard and Bruce,

On 6/2/23 21:19, Richard Purdie wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> On Fri, 2023-06-02 at 09:08 -0400, Bruce Ashfield wrote:
>> On Thu, Jun 1, 2023 at 10:37 PM Yu, Mingli <mingli.yu@eng.windriver.com> wrote:
>>>
>>> From: Mingli Yu <mingli.yu@windriver.com>
>>>
>>> Currently all files as below packaged into one package such as
>>> qemu-7.2.0-*.rpm. After the qemu package installed on the target,
>>> it will take up about 464M which includes not only the one matches
>>> the arch of the target but aslo all available built qemu targets
>>> which set by QEMU_TARGETS.
>>>
>>>   # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
>>>   qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
>>>   qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
>>>   qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
>>>   qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
>>>   qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
>>>   qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
>>>   qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
>>>   qemu-riscv32    qemu-system-arm      qemu-system-mips64el
>>>   qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
>>>   qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4
>>>
>>> Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-aarch64-7.2.0*.rpm,
>>> qemu-system-x86_64-7.2.0*.rpm and etc. And let user can only choose the
>>> corresponding qemu arch package they want to install should ease the concerns
>>> who cares much about the size in embedded device as it decreases the qemu rpm
>>> (qemu-7.2.0*.rpm) size from about 65M to about 19M and the size of the
>>> extracted qemu RPM decreased from about 464M to about 248M.
>>>
>>> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
>>> ---
>>>   meta/recipes-devtools/qemu/qemu.inc | 5 +++++
>>>   1 file changed, 5 insertions(+)
>>>
>>> diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
>>> index a87dee5c99..c6fd39aab6 100644
>>> --- a/meta/recipes-devtools/qemu/qemu.inc
>>> +++ b/meta/recipes-devtools/qemu/qemu.inc
>>> @@ -230,6 +230,11 @@ INSANE_SKIP:${PN} = "arch"
>>>
>>>   FILES:${PN} += "${datadir}/icons"
>>>
>>> +python populate_packages:prepend() {
>>> +    archdir = d.expand('${bindir}/')
>>> +    do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)' , prepend=True)
>>> +}
>>
>> I don't see a v3 changelog, but that does look like the unnecessary
>> default runtime dependency is gone, but what would be the method of
>> installing the packages if I have an image where I'd like the old
>> non-split functionality ?
>>
>> Either a meta packages (qemu-system-all ?) or a way to override the
>> functionality (a variable around the do_split call ?) are options to
>> enable that sort of thing. As I mentioned before, there's a different
>> expected split of the packages in some scenarios, and it would be
>> ideal to not break those use cases.
> 
> This is quite a common need for things which call do_split_packages,
> I'm starting to wonder if it should support some kind of "common"
> package as a parameter which it would add dependencies to?

I'm trying to improve the package method to try to meet the need for 
both the user who cares about the rpm size and the user who want the old 
non-split functionality.

And I just did a little search for the packages which use 
do_split_packages and didn't see there is any package which keep both 
split and no-split functionality and just only split.

So I don't quite understand "This is quite a common need for things 
which call do_split_packages".

BTW, if we still use the old non-split, is there any suitable method for 
the user who cares about the rpm size? Any suggestions?

Thanks,

> 
> Cheers,
> 
> Richard

Re: [OE-core] [PATCH v3] qemu: Split the qemu package

[Richard Purdie]

On Wed, 2023-06-07 at 11:09 +0800, Yu, Mingli wrote:
> Hi Richard and Bruce,
> 
> On 6/2/23 21:19, Richard Purdie wrote:
> > CAUTION: This email comes from a non Wind River email account!
> > Do not click links or open attachments unless you recognize the sender and know the content is safe.
> > 
> > On Fri, 2023-06-02 at 09:08 -0400, Bruce Ashfield wrote:
> > > On Thu, Jun 1, 2023 at 10:37 PM Yu, Mingli <mingli.yu@eng.windriver.com> wrote:
> > > > 
> > > > From: Mingli Yu <mingli.yu@windriver.com>
> > > > 
> > > > Currently all files as below packaged into one package such as
> > > > qemu-7.2.0-*.rpm. After the qemu package installed on the target,
> > > > it will take up about 464M which includes not only the one matches
> > > > the arch of the target but aslo all available built qemu targets
> > > > which set by QEMU_TARGETS.
> > > > 
> > > >   # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
> > > >   qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
> > > >   qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
> > > >   qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
> > > >   qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
> > > >   qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
> > > >   qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
> > > >   qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
> > > >   qemu-riscv32    qemu-system-arm      qemu-system-mips64el
> > > >   qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
> > > >   qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4
> > > > 
> > > > Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-aarch64-7.2.0*.rpm,
> > > > qemu-system-x86_64-7.2.0*.rpm and etc. And let user can only choose the
> > > > corresponding qemu arch package they want to install should ease the concerns
> > > > who cares much about the size in embedded device as it decreases the qemu rpm
> > > > (qemu-7.2.0*.rpm) size from about 65M to about 19M and the size of the
> > > > extracted qemu RPM decreased from about 464M to about 248M.
> > > > 
> > > > Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> > > > ---
> > > >   meta/recipes-devtools/qemu/qemu.inc | 5 +++++
> > > >   1 file changed, 5 insertions(+)
> > > > 
> > > > diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
> > > > index a87dee5c99..c6fd39aab6 100644
> > > > --- a/meta/recipes-devtools/qemu/qemu.inc
> > > > +++ b/meta/recipes-devtools/qemu/qemu.inc
> > > > @@ -230,6 +230,11 @@ INSANE_SKIP:${PN} = "arch"
> > > > 
> > > >   FILES:${PN} += "${datadir}/icons"
> > > > 
> > > > +python populate_packages:prepend() {
> > > > +    archdir = d.expand('${bindir}/')
> > > > +    do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)' , prepend=True)
> > > > +}
> > > 
> > > I don't see a v3 changelog, but that does look like the unnecessary
> > > default runtime dependency is gone, but what would be the method of
> > > installing the packages if I have an image where I'd like the old
> > > non-split functionality ?
> > > 
> > > Either a meta packages (qemu-system-all ?) or a way to override the
> > > functionality (a variable around the do_split call ?) are options to
> > > enable that sort of thing. As I mentioned before, there's a different
> > > expected split of the packages in some scenarios, and it would be
> > > ideal to not break those use cases.
> > 
> > This is quite a common need for things which call do_split_packages,
> > I'm starting to wonder if it should support some kind of "common"
> > package as a parameter which it would add dependencies to?
> 
> I'm trying to improve the package method to try to meet the need for 
> both the user who cares about the rpm size and the user who want the old 
> non-split functionality.
> 
> And I just did a little search for the packages which use 
> do_split_packages and didn't see there is any package which keep both 
> split and no-split functionality and just only split.
> 
> So I don't quite understand "This is quite a common need for things 
> which call do_split_packages".
> 
> BTW, if we still use the old non-split, is there any suitable method for 
> the user who cares about the rpm size? Any suggestions?

I said we should create a package which behaves the same as the old
situation, not that we should have the same binaries in both. By that I
mean a package you can install with dependencies such that all the qemu
binaries are pulled in together. This is something which many users of
do_split_packages already do.

Taking meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-
packaging.inc for example, it has some python which is commented as:

"""
# Go through all generated packages (excluding the main package and
# the -meta package itself) and add them to the -meta package as RDEPENDS.
"""

You can see alsa-plugins does:

    packages = " ".join(do_split_packages(d, plugindir, r'^libasound_module_(.*)\.so$', 'libasound-module-%s', 'Alsa plugin for %s', extra_depends=''))
    d.setVar("RDEPENDS:alsa-plugins", packages)

i.e. it sets the alsa-plugins package to have dependencies on all the
split packages by using the return value.

Similarly, wpa-supplicant:

split_packages = do_split_packages(d, libdir, r'^(.*)\.so', '${PN}-plugin-%s', 'wpa_supplicant %s plugin', prepend=True)
split_dbg_packages = do_split_packages(d, dbglibdir, r'^(.*)\.so', '${PN}-plugin-%s-dbg', 'wpa_supplicant %s plugin - Debugging files', prepend=True, extra_depends='${PN}-dbg')

if split_packages:
    pn = d.getVar('PN')
    d.setVar('RRECOMMENDS:' + pn + '-plugins', ' '.join(split_packages))
    d.appendVar('RRECOMMENDS:' + pn + '-dbg', ' ' + ' '.join(split_dbg_packages))


So we have two options. Either we decide the return value is good
enough and just use that to create this package, or we add a new
parameter to do_split_packages so you could call something like:

do_split_packages(d, libdir, [xxx], append_split_packages_var='RRECOMMENDS:' + pn + '-plugins')

directly. We might need a better name for the new parameter but you get
the idea.

Cheers,

Richard

[PATCH v4] qemu: Split the qemu package

[mingli.yu]

From: Mingli Yu <mingli.yu@windriver.com>

Currently all files as below packaged into one package such as
qemu-7.2.0-*.rpm. After the qemu package installed on the target,
it will take up about 464M which includes not only the one matches
the arch of the target but aslo all available built qemu targets
which set by QEMU_TARGETS.

 # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
 qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
 qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
 qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
 qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
 qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
 qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
 qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
 qemu-riscv32    qemu-system-arm      qemu-system-mips64el
 qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
 qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4

Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-*.rpm,
qemu-user-*.rpm and etc. And let user can only choose the corresponding
qemu arch package they want to install should ease the concerns who
cares much about the size in embedded device as it decreases the qemu rpm
(qemu-7.2.0*.rpm) size from about 65M to about 19M and the size of the
extracted qemu RPM decreased from about 464M to about 248M.

For the users who want to install all arch packages, they can install
qemu-system-all and qemu-user-all to meet their need.

Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
---
 meta/recipes-devtools/qemu/qemu.inc | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
index a87dee5c99..108dd0947a 100644
--- a/meta/recipes-devtools/qemu/qemu.inc
+++ b/meta/recipes-devtools/qemu/qemu.inc
@@ -230,6 +230,25 @@ INSANE_SKIP:${PN} = "arch"
 
 FILES:${PN} += "${datadir}/icons"
 
+# For user who want to install all arch packages
+PACKAGES =+ "${PN}-system-all ${PN}-user-all"
+
+ALLOW_EMPTY:${PN}-system-all = "1"
+ALLOW_EMPTY:${PN}-user-all = "1"
+
+python populate_packages:prepend() {
+    archdir = d.expand('${bindir}/')
+    syspackages = do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)' , prepend=True)
+    if syspackages:
+        d.setVar('RDEPENDS:' + d.getVar('PN') + '-system-all', ' '.join(syspackages))
+
+    userpackages = do_split_packages(d, archdir, r'^qemu-((?!system|edid|ga|img|io|nbd|pr-helper|storage-daemon).*)$', '${PN}-user-%s', 'QEMU full user emulation binaries(%s)' , prepend=True)
+    if "qemu-user-mips" in ' '.join(userpackages):
+        d.appendVar('RDEPENDS:qemu-user-mips', ' '+ 'bash')
+    if userpackages:
+        d.setVar('RDEPENDS:' + d.getVar('PN') + '-user-all', ' '.join(userpackages))
+}
+
 # Put the guest agent in a separate package
 PACKAGES =+ "${PN}-guest-agent"
 SUMMARY:${PN}-guest-agent = "QEMU guest agent"
-- 
2.25.1

Re: [OE-core] [PATCH v4] qemu: Split the qemu package

[Alexandre Belloni]

Hello,

I believe this causes:

https://autobuilder.yoctoproject.org/typhoon/#/builders/52/builds/7143/steps/12/logs/stdio

ERROR: lib32-qemu-8.0.0-r0 do_package_qa: QA Issue: /usr/bin/qemu-mips contained in package lib32-qemu-user-mips requires /bin/bash, but no providers found in RDEPENDS:lib32-qemu-user-mips? [file-rdeps]
ERROR: lib32-qemu-8.0.0-r0 do_package_qa: Fatal QA errors were found, failing task.
ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/qemux86-world/build/build/tmp/work/x86-pokymllib32-linux/lib32-qemu/8.0.0-r0/temp/log.do_package_qa.38833
NOTE: recipe lib32-qemu-8.0.0-r0: task do_package_qa: Failed
ERROR: Task (virtual:multilib:lib32:/home/pokybuild/yocto-worker/qemux86-world/build/meta/recipes-devtools/qemu/qemu_8.0.0.bb:do_package_qa) failed with exit code '1'
N

On 07/06/2023 17:08:13+0800, Yu, Mingli wrote:
> From: Mingli Yu <mingli.yu@windriver.com>
> 
> Currently all files as below packaged into one package such as
> qemu-7.2.0-*.rpm. After the qemu package installed on the target,
> it will take up about 464M which includes not only the one matches
> the arch of the target but aslo all available built qemu targets
> which set by QEMU_TARGETS.
> 
>  # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
>  qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
>  qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
>  qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
>  qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
>  qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
>  qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
>  qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
>  qemu-riscv32    qemu-system-arm      qemu-system-mips64el
>  qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
>  qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4
> 
> Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-*.rpm,
> qemu-user-*.rpm and etc. And let user can only choose the corresponding
> qemu arch package they want to install should ease the concerns who
> cares much about the size in embedded device as it decreases the qemu rpm
> (qemu-7.2.0*.rpm) size from about 65M to about 19M and the size of the
> extracted qemu RPM decreased from about 464M to about 248M.
> 
> For the users who want to install all arch packages, they can install
> qemu-system-all and qemu-user-all to meet their need.
> 
> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> ---
>  meta/recipes-devtools/qemu/qemu.inc | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
> 
> diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
> index a87dee5c99..108dd0947a 100644
> --- a/meta/recipes-devtools/qemu/qemu.inc
> +++ b/meta/recipes-devtools/qemu/qemu.inc
> @@ -230,6 +230,25 @@ INSANE_SKIP:${PN} = "arch"
>  
>  FILES:${PN} += "${datadir}/icons"
>  
> +# For user who want to install all arch packages
> +PACKAGES =+ "${PN}-system-all ${PN}-user-all"
> +
> +ALLOW_EMPTY:${PN}-system-all = "1"
> +ALLOW_EMPTY:${PN}-user-all = "1"
> +
> +python populate_packages:prepend() {
> +    archdir = d.expand('${bindir}/')
> +    syspackages = do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)' , prepend=True)
> +    if syspackages:
> +        d.setVar('RDEPENDS:' + d.getVar('PN') + '-system-all', ' '.join(syspackages))
> +
> +    userpackages = do_split_packages(d, archdir, r'^qemu-((?!system|edid|ga|img|io|nbd|pr-helper|storage-daemon).*)$', '${PN}-user-%s', 'QEMU full user emulation binaries(%s)' , prepend=True)
> +    if "qemu-user-mips" in ' '.join(userpackages):
> +        d.appendVar('RDEPENDS:qemu-user-mips', ' '+ 'bash')
> +    if userpackages:
> +        d.setVar('RDEPENDS:' + d.getVar('PN') + '-user-all', ' '.join(userpackages))
> +}
> +
>  # Put the guest agent in a separate package
>  PACKAGES =+ "${PN}-guest-agent"
>  SUMMARY:${PN}-guest-agent = "QEMU guest agent"
> -- 
> 2.25.1
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#182464): https://lists.openembedded.org/g/openembedded-core/message/182464
> Mute This Topic: https://lists.openembedded.org/mt/99380658/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH v5] qemu: Split the qemu package

[mingli.yu]

From: Mingli Yu <mingli.yu@windriver.com>

Currently all files as below packaged into one package such as
qemu-7.2.0-*.rpm. After the qemu package installed on the target,
it will take up about 464M which includes not only the one matches
the arch of the target but aslo all available built qemu targets
which set by QEMU_TARGETS.

 # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
 qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
 qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
 qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
 qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
 qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
 qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
 qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
 qemu-riscv32    qemu-system-arm      qemu-system-mips64el
 qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
 qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4

Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-*.rpm,
qemu-user-*.rpm and etc. And let user can only choose the corresponding
qemu arch package they want to install should ease the concerns who
cares much about the size in embedded device as it decreases the qemu rpm
(qemu-7.2.0*.rpm) size from about 65M to about 13M and the size of the
extracted qemu RPM decreased from about 464M to about 230M.

For the users who want to install all arch packages, they can install
qemu-system-all and qemu-user-all to meet their need.

Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
---
 meta/recipes-devtools/qemu/qemu.inc | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
index a87dee5c99..367b924f9c 100644
--- a/meta/recipes-devtools/qemu/qemu.inc
+++ b/meta/recipes-devtools/qemu/qemu.inc
@@ -230,6 +230,26 @@ INSANE_SKIP:${PN} = "arch"
 
 FILES:${PN} += "${datadir}/icons"
 
+# For user who want to install all arch packages
+PACKAGES =+ "${PN}-system-all ${PN}-user-all"
+
+ALLOW_EMPTY:${PN}-system-all = "1"
+ALLOW_EMPTY:${PN}-user-all = "1"
+
+python populate_packages:prepend() {
+    archdir = d.expand('${bindir}/')
+    syspackages = do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)' , prepend=True)
+    if syspackages:
+        d.setVar('RDEPENDS:' + d.getVar('PN') + '-system-all', ' '.join(syspackages))
+
+    userpackages = do_split_packages(d, archdir, r'^qemu-((?!system|edid|ga|img|io|nbd|pr-helper|storage-daemon).*)$', '${PN}-user-%s', 'QEMU full user emulation binaries(%s)' , prepend=True)
+    if userpackages:
+        d.setVar('RDEPENDS:' + d.getVar('PN') + '-user-all', ' '.join(userpackages))
+    mipspackage = d.getVar('PN') + "-user-mips"
+    if mipspackage in ' '.join(userpackages):
+        d.appendVar('RDEPENDS:' + mipspackage, ' ' + d.getVar("MLPREFIX") + 'bash')
+}
+
 # Put the guest agent in a separate package
 PACKAGES =+ "${PN}-guest-agent"
 SUMMARY:${PN}-guest-agent = "QEMU guest agent"
-- 
2.25.1

Re: [OE-core] [PATCH v5] qemu: Split the qemu package

[Bruce Ashfield]

On Thu, Jun 8, 2023 at 1:45 AM Yu, Mingli <mingli.yu@eng.windriver.com> wrote:
>
> From: Mingli Yu <mingli.yu@windriver.com>
>
> Currently all files as below packaged into one package such as
> qemu-7.2.0-*.rpm. After the qemu package installed on the target,
> it will take up about 464M which includes not only the one matches
> the arch of the target but aslo all available built qemu targets
> which set by QEMU_TARGETS.
>
>  # ls tmp-glibc/work/core2-64-wrs-linux/qemu/7.2.0-r0/image/usr/bin/
>  qemu-aarch64  qemu-img          qemu-mips64el   qemu-ppc64
>  qemu-sh4    qemu-system-loongarch64  qemu-system-ppc      qemu-system-x86_64
>  qemu-arm      qemu-io           qemu-mipsel     qemu-ppc64le
>  qemu-storage-daemon  qemu-system-mips         qemu-system-ppc64
>  qemu-x86_64 qemu-edid     qemu-loongarch64  qemu-mips.real
>  qemu-pr-helper  qemu-system-aarch64  qemu-system-mips64
>  qemu-system-riscv32 qemu-ga       qemu-mips         qemu-nbd
>  qemu-riscv32    qemu-system-arm      qemu-system-mips64el
>  qemu-system-riscv64 qemu-i386     qemu-mips64       qemu-ppc
>  qemu-riscv64    qemu-system-i386     qemu-system-mipsel qemu-system-sh4
>
> Split the qemu package into qemu-7.2.0-*.rpm, qemu-system-*.rpm,
> qemu-user-*.rpm and etc. And let user can only choose the corresponding
> qemu arch package they want to install should ease the concerns who
> cares much about the size in embedded device as it decreases the qemu rpm
> (qemu-7.2.0*.rpm) size from about 65M to about 13M and the size of the
> extracted qemu RPM decreased from about 464M to about 230M.
>
> For the users who want to install all arch packages, they can install
> qemu-system-all and qemu-user-all to meet their need.
>
> Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
> ---
>  meta/recipes-devtools/qemu/qemu.inc | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
>
> diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
> index a87dee5c99..367b924f9c 100644
> --- a/meta/recipes-devtools/qemu/qemu.inc
> +++ b/meta/recipes-devtools/qemu/qemu.inc
> @@ -230,6 +230,26 @@ INSANE_SKIP:${PN} = "arch"
>
>  FILES:${PN} += "${datadir}/icons"
>
> +# For user who want to install all arch packages
> +PACKAGES =+ "${PN}-system-all ${PN}-user-all"
> +
> +ALLOW_EMPTY:${PN}-system-all = "1"
> +ALLOW_EMPTY:${PN}-user-all = "1"
> +
> +python populate_packages:prepend() {
> +    archdir = d.expand('${bindir}/')
> +    syspackages = do_split_packages(d, archdir, r'^qemu-system-(.*)$', '${PN}-system-%s', 'QEMU full system emulation binaries(%s)' , prepend=True)
> +    if syspackages:
> +        d.setVar('RDEPENDS:' + d.getVar('PN') + '-system-all', ' '.join(syspackages))
> +
> +    userpackages = do_split_packages(d, archdir, r'^qemu-((?!system|edid|ga|img|io|nbd|pr-helper|storage-daemon).*)$', '${PN}-user-%s', 'QEMU full user emulation binaries(%s)' , prepend=True)
> +    if userpackages:
> +        d.setVar('RDEPENDS:' + d.getVar('PN') + '-user-all', ' '.join(userpackages))
> +    mipspackage = d.getVar('PN') + "-user-mips"
> +    if mipspackage in ' '.join(userpackages):
> +        d.appendVar('RDEPENDS:' + mipspackage, ' ' + d.getVar("MLPREFIX") + 'bash')
> +}

At this point, you can probably see why I ended up using the explicit
variables and overrides versus python when doing the
meta-virtualization splits. :)

I have a few more comments that I made in v1, that I haven't seen
directly handled or replied to.

My only remaining concern (and it may just be my own concern), is that
there's no way to change this packaging split. Either you take the
programatic split, or you take the -all packages.

Other packages (glibc, kernel-modules) have a variable that controls
whether the split happens or not, I'd like to see something similar
here .. but I do realize that it makes test complexity more, and that
Richard normally doesn't like conditionals like that.

Alternatively, did we rule out using PACKAGESPLITFUNCS to add the
splitting routine ? With that, we could remove the processing in
places we don't want it, in particular for the native/sdk builds, as I
found that we really don't want the splitting of qemu in those
scenarios.

Bruce

> +
>  # Put the guest agent in a separate package
>  PACKAGES =+ "${PN}-guest-agent"
>  SUMMARY:${PN}-guest-agent = "QEMU guest agent"
> --
> 2.25.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#182494): https://lists.openembedded.org/g/openembedded-core/message/182494
> Mute This Topic: https://lists.openembedded.org/mt/99401176/1050810
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [PATCH v5] qemu: Split the qemu package

[Richard Purdie]

On Thu, 2023-06-08 at 09:33 -0400, Bruce Ashfield wrote:
> At this point, you can probably see why I ended up using the explicit
> variables and overrides versus python when doing the
> meta-virtualization splits. :)
> 
> I have a few more comments that I made in v1, that I haven't seen
> directly handled or replied to.
> 
> My only remaining concern (and it may just be my own concern), is that
> there's no way to change this packaging split. Either you take the
> programatic split, or you take the -all packages.
> 
> Other packages (glibc, kernel-modules) have a variable that controls
> whether the split happens or not, I'd like to see something similar
> here .. but I do realize that it makes test complexity more, and that
> Richard normally doesn't like conditionals like that.

I only "like" conditionals if it is a code path that will be well
travelled, otherwise we tend to find one of the paths is broken. I
don't think we need to make this conditional.

> Alternatively, did we rule out using PACKAGESPLITFUNCS to add the
> splitting routine ? With that, we could remove the processing in
> places we don't want it, in particular for the native/sdk builds, as I
> found that we really don't want the splitting of qemu in those
> scenarios.

FWIW, native isn't packaged so that one at least isn't a problem!

I'm assuming the "all" package can be used in the SDK contexts easily
enough to get the needed behaviour there?

Cheers,

Richard

Re: [OE-core] [PATCH v5] qemu: Split the qemu package

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 2055 bytes --]

On Thu, Jun 8, 2023 at 9:55 AM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Thu, 2023-06-08 at 09:33 -0400, Bruce Ashfield wrote:
> > At this point, you can probably see why I ended up using the explicit
> > variables and overrides versus python when doing the
> > meta-virtualization splits. :)
> >
> > I have a few more comments that I made in v1, that I haven't seen
> > directly handled or replied to.
> >
> > My only remaining concern (and it may just be my own concern), is that
> > there's no way to change this packaging split. Either you take the
> > programatic split, or you take the -all packages.
> >
> > Other packages (glibc, kernel-modules) have a variable that controls
> > whether the split happens or not, I'd like to see something similar
> > here .. but I do realize that it makes test complexity more, and that
> > Richard normally doesn't like conditionals like that.
>
> I only "like" conditionals if it is a code path that will be well
> travelled, otherwise we tend to find one of the paths is broken. I
> don't think we need to make this conditional.
>
> > Alternatively, did we rule out using PACKAGESPLITFUNCS to add the
> > splitting routine ? With that, we could remove the processing in
> > places we don't want it, in particular for the native/sdk builds, as I
> > found that we really don't want the splitting of qemu in those
> > scenarios.
>

Can you think of a problem with the PACKAGESPLITFUNCS method for this
packaging split ?

At least that way I could inhibit it from my other layers, versus with this
prepend, I don't see any options to have my own package splitting.

Bruce



>
> FWIW, native isn't packaged so that one at least isn't a problem!
>
> I'm assuming the "all" package can be used in the SDK contexts easily
> enough to get the needed behaviour there?
>
> Cheers,
>
> Richard
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 3212 bytes --]

Re: [OE-core] [PATCH v5] qemu: Split the qemu package

[Richard Purdie]

On Thu, 2023-06-08 at 11:03 -0400, Bruce Ashfield wrote:
> 
> 
> On Thu, Jun 8, 2023 at 9:55 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Thu, 2023-06-08 at 09:33 -0400, Bruce Ashfield wrote:
> > > At this point, you can probably see why I ended up using the
> > > explicit
> > > variables and overrides versus python when doing the
> > > meta-virtualization splits. :)
> > > 
> > > I have a few more comments that I made in v1, that I haven't seen
> > > directly handled or replied to.
> > > 
> > > My only remaining concern (and it may just be my own concern), is
> > > that
> > > there's no way to change this packaging split. Either you take
> > > the
> > > programatic split, or you take the -all packages.
> > > 
> > > Other packages (glibc, kernel-modules) have a variable that
> > > controls
> > > whether the split happens or not, I'd like to see something
> > > similar
> > > here .. but I do realize that it makes test complexity more, and
> > > that
> > > Richard normally doesn't like conditionals like that.
> > 
> > I only "like" conditionals if it is a code path that will be well
> > travelled, otherwise we tend to find one of the paths is broken. I
> > don't think we need to make this conditional.
> > 
> > > Alternatively, did we rule out using PACKAGESPLITFUNCS to add the
> > > splitting routine ? With that, we could remove the processing in
> > > places we don't want it, in particular for the native/sdk builds,
> > > as I
> > > found that we really don't want the splitting of qemu in those
> > > scenarios.
> > 
> 
> 
> Can you think of a problem with the PACKAGESPLITFUNCS method for this
> packaging split ?
> 
> At least that way I could inhibit it from my other layers, versus
> with this prepend, I don't see any options to have my own package
> splitting.

We can do this in a PACKAGESPLITFUNCS function. I guess I'm just
nervous of having too many different ways of packaging qemu as I was
hoping we could get something which would work for all users.

Cheers,

Richard

Re: [OE-core] [PATCH v5] qemu: Split the qemu package

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 2971 bytes --]

On Thu, Jun 8, 2023 at 11:44 AM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Thu, 2023-06-08 at 11:03 -0400, Bruce Ashfield wrote:
> >
> >
> > On Thu, Jun 8, 2023 at 9:55 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > > On Thu, 2023-06-08 at 09:33 -0400, Bruce Ashfield wrote:
> > > > At this point, you can probably see why I ended up using the
> > > > explicit
> > > > variables and overrides versus python when doing the
> > > > meta-virtualization splits. :)
> > > >
> > > > I have a few more comments that I made in v1, that I haven't seen
> > > > directly handled or replied to.
> > > >
> > > > My only remaining concern (and it may just be my own concern), is
> > > > that
> > > > there's no way to change this packaging split. Either you take
> > > > the
> > > > programatic split, or you take the -all packages.
> > > >
> > > > Other packages (glibc, kernel-modules) have a variable that
> > > > controls
> > > > whether the split happens or not, I'd like to see something
> > > > similar
> > > > here .. but I do realize that it makes test complexity more, and
> > > > that
> > > > Richard normally doesn't like conditionals like that.
> > >
> > > I only "like" conditionals if it is a code path that will be well
> > > travelled, otherwise we tend to find one of the paths is broken. I
> > > don't think we need to make this conditional.
> > >
> > > > Alternatively, did we rule out using PACKAGESPLITFUNCS to add the
> > > > splitting routine ? With that, we could remove the processing in
> > > > places we don't want it, in particular for the native/sdk builds,
> > > > as I
> > > > found that we really don't want the splitting of qemu in those
> > > > scenarios.
> > >
> >
> >
> > Can you think of a problem with the PACKAGESPLITFUNCS method for this
> > packaging split ?
> >
> > At least that way I could inhibit it from my other layers, versus
> > with this prepend, I don't see any options to have my own package
> > splitting.
>
> We can do this in a PACKAGESPLITFUNCS function. I guess I'm just
> nervous of having too many different ways of packaging qemu as I was
> hoping we could get something which would work for all users.
>
>
There are some very specific use cases for virtualization, where some
separation models use different architectures for devices, even
architectures that don't match the target (host in the running system).

There are also some combinations of usermode and system, as well as support
and firmware, etc.

I can't see a clean/easy way to make the split being proposed here serve
all those existing (and admittedly odd) case. Having a way to override it
(even temporarily) is a better transition path for meta-virt.

Bruce



> Cheers,
>
> Richard
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 4615 bytes --]

Re: [OE-core] [PATCH v5] qemu: Split the qemu package

[Yu, Mingli]

[-- Attachment #1: Type: text/plain, Size: 4248 bytes --]

Hi Bruce,

I didn't reply the v1 patch directly and just include the comments and concerns when generate v2,v3,v4,v5 patch.

I'm sorry the overrides for qemu split in meta-vir as https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc doesn't work if we make the qemu split change.<https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc>

<https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc>
Is it okay for you to install the qemu arch rpms you need or qemu-all(if you want all qemu binary) after the qemu split change?
<https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc>

Thanks,
________________________________
From: Bruce Ashfield <bruce.ashfield@gmail.com>
Sent: Friday, June 9, 2023 00:16
To: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Yu, Mingli <Mingli.Yu@windriver.com>; openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
Subject: Re: [OE-core] [PATCH v5] qemu: Split the qemu package

CAUTION: This email comes from a non Wind River email account!
Do not click links or open attachments unless you recognize the sender and know the content is safe.


On Thu, Jun 8, 2023 at 11:44 AM Richard Purdie <richard.purdie@linuxfoundation.org<mailto:richard.purdie@linuxfoundation.org>> wrote:
On Thu, 2023-06-08 at 11:03 -0400, Bruce Ashfield wrote:
>
>
> On Thu, Jun 8, 2023 at 9:55 AM Richard Purdie
> <richard.purdie@linuxfoundation.org<mailto:richard.purdie@linuxfoundation.org>> wrote:
> > On Thu, 2023-06-08 at 09:33 -0400, Bruce Ashfield wrote:
> > > At this point, you can probably see why I ended up using the
> > > explicit
> > > variables and overrides versus python when doing the
> > > meta-virtualization splits. :)
> > >
> > > I have a few more comments that I made in v1, that I haven't seen
> > > directly handled or replied to.
> > >
> > > My only remaining concern (and it may just be my own concern), is
> > > that
> > > there's no way to change this packaging split. Either you take
> > > the
> > > programatic split, or you take the -all packages.
> > >
> > > Other packages (glibc, kernel-modules) have a variable that
> > > controls
> > > whether the split happens or not, I'd like to see something
> > > similar
> > > here .. but I do realize that it makes test complexity more, and
> > > that
> > > Richard normally doesn't like conditionals like that.
> >
> > I only "like" conditionals if it is a code path that will be well
> > travelled, otherwise we tend to find one of the paths is broken. I
> > don't think we need to make this conditional.
> >
> > > Alternatively, did we rule out using PACKAGESPLITFUNCS to add the
> > > splitting routine ? With that, we could remove the processing in
> > > places we don't want it, in particular for the native/sdk builds,
> > > as I
> > > found that we really don't want the splitting of qemu in those
> > > scenarios.
> >
>
>
> Can you think of a problem with the PACKAGESPLITFUNCS method for this
> packaging split ?
>
> At least that way I could inhibit it from my other layers, versus
> with this prepend, I don't see any options to have my own package
> splitting.

We can do this in a PACKAGESPLITFUNCS function. I guess I'm just
nervous of having too many different ways of packaging qemu as I was
hoping we could get something which would work for all users.


There are some very specific use cases for virtualization, where some separation models use different architectures for devices, even architectures that don't match the target (host in the running system).

There are also some combinations of usermode and system, as well as support and firmware, etc.

I can't see a clean/easy way to make the split being proposed here serve all those existing (and admittedly odd) case. Having a way to override it (even temporarily) is a better transition path for meta-virt.

Bruce


Cheers,

Richard


--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


[-- Attachment #2: Type: text/html, Size: 7669 bytes --]

Re: [OE-core] [PATCH v5] qemu: Split the qemu package

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 4966 bytes --]

On Thu, Jun 8, 2023 at 10:01 PM Yu, Mingli <Mingli.Yu@windriver.com> wrote:

> Hi Bruce,
>
> I didn't reply the v1 patch directly and just include the comments and
> concerns when generate v2,v3,v4,v5 patch.
>
> I'm sorry the overrides for qemu split in meta-vir as
> https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc
> doesn't work if we make the qemu split change.
> <https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc>
>
>
> <https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc>
> Is it okay for you to install the qemu arch rpms you need or qemu-all(if
> you want all qemu binary) after the qemu split change?
> <https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc>
>

It actually isn't that. I want a different split of the packages.

But I see your v6 is using the function variable, so I'll be able to remove
it from processing and restore my existing package splits.

Bruce



>
>
> <https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc>
>
> Thanks,
> ------------------------------
> *From:* Bruce Ashfield <bruce.ashfield@gmail.com>
> *Sent:* Friday, June 9, 2023 00:16
> *To:* Richard Purdie <richard.purdie@linuxfoundation.org>
> *Cc:* Yu, Mingli <Mingli.Yu@windriver.com>;
> openembedded-core@lists.openembedded.org <
> openembedded-core@lists.openembedded.org>
> *Subject:* Re: [OE-core] [PATCH v5] qemu: Split the qemu package
>
> * CAUTION: This email comes from a non Wind River email account!*
> Do not click links or open attachments unless you recognize the sender and
> know the content is safe.
>
>
> On Thu, Jun 8, 2023 at 11:44 AM Richard Purdie <
> richard.purdie@linuxfoundation.org> wrote:
>
> On Thu, 2023-06-08 at 11:03 -0400, Bruce Ashfield wrote:
> >
> >
> > On Thu, Jun 8, 2023 at 9:55 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > > On Thu, 2023-06-08 at 09:33 -0400, Bruce Ashfield wrote:
> > > > At this point, you can probably see why I ended up using the
> > > > explicit
> > > > variables and overrides versus python when doing the
> > > > meta-virtualization splits. :)
> > > >
> > > > I have a few more comments that I made in v1, that I haven't seen
> > > > directly handled or replied to.
> > > >
> > > > My only remaining concern (and it may just be my own concern), is
> > > > that
> > > > there's no way to change this packaging split. Either you take
> > > > the
> > > > programatic split, or you take the -all packages.
> > > >
> > > > Other packages (glibc, kernel-modules) have a variable that
> > > > controls
> > > > whether the split happens or not, I'd like to see something
> > > > similar
> > > > here .. but I do realize that it makes test complexity more, and
> > > > that
> > > > Richard normally doesn't like conditionals like that.
> > >
> > > I only "like" conditionals if it is a code path that will be well
> > > travelled, otherwise we tend to find one of the paths is broken. I
> > > don't think we need to make this conditional.
> > >
> > > > Alternatively, did we rule out using PACKAGESPLITFUNCS to add the
> > > > splitting routine ? With that, we could remove the processing in
> > > > places we don't want it, in particular for the native/sdk builds,
> > > > as I
> > > > found that we really don't want the splitting of qemu in those
> > > > scenarios.
> > >
> >
> >
> > Can you think of a problem with the PACKAGESPLITFUNCS method for this
> > packaging split ?
> >
> > At least that way I could inhibit it from my other layers, versus
> > with this prepend, I don't see any options to have my own package
> > splitting.
>
> We can do this in a PACKAGESPLITFUNCS function. I guess I'm just
> nervous of having too many different ways of packaging qemu as I was
> hoping we could get something which would work for all users.
>
>
> There are some very specific use cases for virtualization, where some
> separation models use different architectures for devices, even
> architectures that don't match the target (host in the running system).
>
> There are also some combinations of usermode and system, as well as
> support and firmware, etc.
>
> I can't see a clean/easy way to make the split being proposed here serve
> all those existing (and admittedly odd) case. Having a way to override it
> (even temporarily) is a better transition path for meta-virt.
>
> Bruce
>
>
>
> Cheers,
>
> Richard
>
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await thee
> at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
>

-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 9139 bytes --]

Re: [OE-core] [PATCH v5] qemu: Split the qemu package

[Yu, Mingli]

[-- Attachment #1: Type: text/plain, Size: 6172 bytes --]

Okay, let us use v6 to track the patch.

Thanks,
________________________________
From: Bruce Ashfield <bruce.ashfield@gmail.com>
Sent: Friday, June 9, 2023 11:25
To: Yu, Mingli <Mingli.Yu@windriver.com>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>; openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
Subject: Re: [OE-core] [PATCH v5] qemu: Split the qemu package

CAUTION: This email comes from a non Wind River email account!
Do not click links or open attachments unless you recognize the sender and know the content is safe.


On Thu, Jun 8, 2023 at 10:01 PM Yu, Mingli <Mingli.Yu@windriver.com<mailto:Mingli.Yu@windriver.com>> wrote:
Hi Bruce,

I didn't reply the v1 patch directly and just include the comments and concerns when generate v2,v3,v4,v5 patch.

I'm sorry the overrides for qemu split in meta-vir as https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc doesn't work if we make the qemu split change.<https://urldefense.com/v3/__https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc__;!!AjveYdw8EvQ!byty5y3CfmOOKUh8G_uYgU2QxFCl4KCFb-yjmzYuSrjQJ4NXaglf6vZ6IWtmyrHhPs4okTIp4TpusD8e4h8-YLvToqA$>

<https://urldefense.com/v3/__https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc__;!!AjveYdw8EvQ!byty5y3CfmOOKUh8G_uYgU2QxFCl4KCFb-yjmzYuSrjQJ4NXaglf6vZ6IWtmyrHhPs4okTIp4TpusD8e4h8-YLvToqA$>
Is it okay for you to install the qemu arch rpms you need or qemu-all(if you want all qemu binary) after the qemu split change?<https://urldefense.com/v3/__https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc__;!!AjveYdw8EvQ!byty5y3CfmOOKUh8G_uYgU2QxFCl4KCFb-yjmzYuSrjQJ4NXaglf6vZ6IWtmyrHhPs4okTIp4TpusD8e4h8-YLvToqA$>

It actually isn't that. I want a different split of the packages.

But I see your v6 is using the function variable, so I'll be able to remove it from processing and restore my existing package splits.

Bruce



<https://urldefense.com/v3/__https://git.yoctoproject.org/meta-virtualization/tree/recipes-devtools/qemu/qemu-package-split.inc__;!!AjveYdw8EvQ!byty5y3CfmOOKUh8G_uYgU2QxFCl4KCFb-yjmzYuSrjQJ4NXaglf6vZ6IWtmyrHhPs4okTIp4TpusD8e4h8-YLvToqA$>

Thanks,
________________________________
From: Bruce Ashfield <bruce.ashfield@gmail.com<mailto:bruce.ashfield@gmail.com>>
Sent: Friday, June 9, 2023 00:16
To: Richard Purdie <richard.purdie@linuxfoundation.org<mailto:richard.purdie@linuxfoundation.org>>
Cc: Yu, Mingli <Mingli.Yu@windriver.com<mailto:Mingli.Yu@windriver.com>>; openembedded-core@lists.openembedded.org<mailto:openembedded-core@lists.openembedded.org> <openembedded-core@lists.openembedded.org<mailto:openembedded-core@lists.openembedded.org>>
Subject: Re: [OE-core] [PATCH v5] qemu: Split the qemu package

CAUTION: This email comes from a non Wind River email account!

Do not click links or open attachments unless you recognize the sender and know the content is safe.


On Thu, Jun 8, 2023 at 11:44 AM Richard Purdie <richard.purdie@linuxfoundation.org<mailto:richard.purdie@linuxfoundation.org>> wrote:
On Thu, 2023-06-08 at 11:03 -0400, Bruce Ashfield wrote:
>
>
> On Thu, Jun 8, 2023 at 9:55 AM Richard Purdie
> <richard.purdie@linuxfoundation.org<mailto:richard.purdie@linuxfoundation.org>> wrote:
> > On Thu, 2023-06-08 at 09:33 -0400, Bruce Ashfield wrote:
> > > At this point, you can probably see why I ended up using the
> > > explicit
> > > variables and overrides versus python when doing the
> > > meta-virtualization splits. :)
> > >
> > > I have a few more comments that I made in v1, that I haven't seen
> > > directly handled or replied to.
> > >
> > > My only remaining concern (and it may just be my own concern), is
> > > that
> > > there's no way to change this packaging split. Either you take
> > > the
> > > programatic split, or you take the -all packages.
> > >
> > > Other packages (glibc, kernel-modules) have a variable that
> > > controls
> > > whether the split happens or not, I'd like to see something
> > > similar
> > > here .. but I do realize that it makes test complexity more, and
> > > that
> > > Richard normally doesn't like conditionals like that.
> >
> > I only "like" conditionals if it is a code path that will be well
> > travelled, otherwise we tend to find one of the paths is broken. I
> > don't think we need to make this conditional.
> >
> > > Alternatively, did we rule out using PACKAGESPLITFUNCS to add the
> > > splitting routine ? With that, we could remove the processing in
> > > places we don't want it, in particular for the native/sdk builds,
> > > as I
> > > found that we really don't want the splitting of qemu in those
> > > scenarios.
> >
>
>
> Can you think of a problem with the PACKAGESPLITFUNCS method for this
> packaging split ?
>
> At least that way I could inhibit it from my other layers, versus
> with this prepend, I don't see any options to have my own package
> splitting.

We can do this in a PACKAGESPLITFUNCS function. I guess I'm just
nervous of having too many different ways of packaging qemu as I was
hoping we could get something which would work for all users.


There are some very specific use cases for virtualization, where some separation models use different architectures for devices, even architectures that don't match the target (host in the running system).

There are also some combinations of usermode and system, as well as support and firmware, etc.

I can't see a clean/easy way to make the split being proposed here serve all those existing (and admittedly odd) case. Having a way to override it (even temporarily) is a better transition path for meta-virt.

Bruce


Cheers,

Richard


--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II



--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II


[-- Attachment #2: Type: text/html, Size: 11494 bytes --]