[Buildroot] [PATCH] package/pkg-generic: don't exclude virtual packages from packages list

[Buildroot] [PATCH] package/pkg-generic: don't exclude virtual packages from packages list

[Yann E. MORIN]

Currently, with a configuration with an internal toolchain, and no other
package is selected [0], especially when one wants to generate an SDK or
a pre-built, pre-installed toolchain, running 'make' will only build
glibc (and its dependencies), and not the full toolchain, as one would
have expected, so there would be no host-final-gcc.

The reason is that 'toolchain' is a virtual package, so it is excluded
from PACKAGES, the list of packages enabled in the configuration. so it
is not a dependency of target-finalize, and so nothing pulls it in the
build.

The reason for excluding virtual packages from that list is not obvious.

When virtual packages were introduced in 743982441201 (packages: add
infrastructure for virtual packages), there was no BR2_PACKAGE_FOO
symbol for virtual packages (but there was BR2_PACKAGE_HAS_FOO), so
there was no telling that the virtual package was enabled, like we had
for the other kinds of packages (normal, bootloader, toolchain, or linux
kernel).

That caused issues, so in f674c428c2ef (core/pkg-virtual: do not check
they are neabled [sic]), and then 3e1b33a5349b (pkg-generic: improve
incorrectly used package detection), we explicitly excluded the virtual
packages from causing a build failure when something depended on them,
as we could not yet now whether a virtual package was actually enabled
or not.

Then, in 842ba7eceffb (pkg-generic: fix rdepends and phony targets of
virtual packages), we eventually associated a virtual package to is
BR2_PACKAGE_HAS_FOO, which allows treating virtual packages like the
other kinds of packages. There, we explicitly kept virtual packages out
of the list, though (the reasoning was that virtual packages install
nothing in host/ or target/, so they do not directly contribute to the
final content, so we do not need to rsync them, so this was an
optimisation).

However, virtual packages are in fact actual generic packages, and it is
possible for virtual packages to actually provide content for the final
image. Even though we do not have any virtual package that has actual
_INSTALL_CMDS, we still have udev that provides a user for example;
virtual packages in br2-external trees may also very well provide
install commands (e.g. to install files common to their various
implementations).

So, there is currently no technical reason to exclude virtual packages
from PACKAGES, the list of packages enabled in the configuration.

Drop the excluding condition, and always add enabled package, whatever
their kind, to the list of enabled packages.

[0] defconfig to reproduce the issue:
    BR2_INIT_NONE=y
    BR2_SYSTEM_BIN_SH_NONE=y
    # BR2_PACKAGE_BUSYBOX is not set
    # BR2_PACKAGE_IFUPDOWN_SCRIPTS is not set
    # BR2_TARGET_ROOTFS_TAR is not set

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
---
 package/pkg-generic.mk | 2 --
 1 file changed, 2 deletions(-)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index b233b07548..f24e03a325 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -1207,9 +1207,7 @@ $(eval $(call check-deprecated-variable,$(2)_BUILD_OPT,$(2)_BUILD_OPTS))
 $(eval $(call check-deprecated-variable,$(2)_GETTEXTIZE_OPT,$(2)_GETTEXTIZE_OPTS))
 $(eval $(call check-deprecated-variable,$(2)_KCONFIG_OPT,$(2)_KCONFIG_OPTS))
 
-ifneq ($$($(2)_IS_VIRTUAL),YES)
 PACKAGES += $(1)
-endif
 
 ifneq ($$($(2)_PERMISSIONS),)
 PACKAGES_PERMISSIONS_TABLE += $$($(2)_PERMISSIONS)$$(sep)
-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] package/pkg-generic: don't exclude virtual packages from packages list

[Thomas Petazzoni]

On Sat, 13 Aug 2022 11:00:14 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> Currently, with a configuration with an internal toolchain, and no other
> package is selected [0], especially when one wants to generate an SDK or
> a pre-built, pre-installed toolchain, running 'make' will only build
> glibc (and its dependencies), and not the full toolchain, as one would
> have expected, so there would be no host-final-gcc.
> 
> The reason is that 'toolchain' is a virtual package, so it is excluded
> from PACKAGES, the list of packages enabled in the configuration. so it
> is not a dependency of target-finalize, and so nothing pulls it in the
> build.
> 
> The reason for excluding virtual packages from that list is not obvious.
> 
> When virtual packages were introduced in 743982441201 (packages: add
> infrastructure for virtual packages), there was no BR2_PACKAGE_FOO
> symbol for virtual packages (but there was BR2_PACKAGE_HAS_FOO), so
> there was no telling that the virtual package was enabled, like we had
> for the other kinds of packages (normal, bootloader, toolchain, or linux
> kernel).
> 
> That caused issues, so in f674c428c2ef (core/pkg-virtual: do not check
> they are neabled [sic]), and then 3e1b33a5349b (pkg-generic: improve
> incorrectly used package detection), we explicitly excluded the virtual
> packages from causing a build failure when something depended on them,
> as we could not yet now whether a virtual package was actually enabled
> or not.
> 
> Then, in 842ba7eceffb (pkg-generic: fix rdepends and phony targets of
> virtual packages), we eventually associated a virtual package to is
> BR2_PACKAGE_HAS_FOO, which allows treating virtual packages like the
> other kinds of packages. There, we explicitly kept virtual packages out
> of the list, though (the reasoning was that virtual packages install
> nothing in host/ or target/, so they do not directly contribute to the
> final content, so we do not need to rsync them, so this was an
> optimisation).
> 
> However, virtual packages are in fact actual generic packages, and it is
> possible for virtual packages to actually provide content for the final
> image. Even though we do not have any virtual package that has actual
> _INSTALL_CMDS, we still have udev that provides a user for example;
> virtual packages in br2-external trees may also very well provide
> install commands (e.g. to install files common to their various
> implementations).
> 
> So, there is currently no technical reason to exclude virtual packages
> from PACKAGES, the list of packages enabled in the configuration.
> 
> Drop the excluding condition, and always add enabled package, whatever
> their kind, to the list of enabled packages.
> 
> [0] defconfig to reproduce the issue:
>     BR2_INIT_NONE=y
>     BR2_SYSTEM_BIN_SH_NONE=y
>     # BR2_PACKAGE_BUSYBOX is not set
>     # BR2_PACKAGE_IFUPDOWN_SCRIPTS is not set
>     # BR2_TARGET_ROOTFS_TAR is not set
> 
> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
> ---
>  package/pkg-generic.mk | 2 --
>  1 file changed, 2 deletions(-)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] package/pkg-generic: don't exclude virtual packages from packages list

[Peter Korsgaard]

>>>>> "Yann" == Yann E MORIN <yann.morin.1998@free.fr> writes:

 > Currently, with a configuration with an internal toolchain, and no other
 > package is selected [0], especially when one wants to generate an SDK or
 > a pre-built, pre-installed toolchain, running 'make' will only build
 > glibc (and its dependencies), and not the full toolchain, as one would
 > have expected, so there would be no host-final-gcc.

 > The reason is that 'toolchain' is a virtual package, so it is excluded
 > from PACKAGES, the list of packages enabled in the configuration. so it
 > is not a dependency of target-finalize, and so nothing pulls it in the
 > build.

 > The reason for excluding virtual packages from that list is not obvious.

 > When virtual packages were introduced in 743982441201 (packages: add
 > infrastructure for virtual packages), there was no BR2_PACKAGE_FOO
 > symbol for virtual packages (but there was BR2_PACKAGE_HAS_FOO), so
 > there was no telling that the virtual package was enabled, like we had
 > for the other kinds of packages (normal, bootloader, toolchain, or linux
 > kernel).

 > That caused issues, so in f674c428c2ef (core/pkg-virtual: do not check
 > they are neabled [sic]), and then 3e1b33a5349b (pkg-generic: improve
 > incorrectly used package detection), we explicitly excluded the virtual
 > packages from causing a build failure when something depended on them,
 > as we could not yet now whether a virtual package was actually enabled
 > or not.

 > Then, in 842ba7eceffb (pkg-generic: fix rdepends and phony targets of
 > virtual packages), we eventually associated a virtual package to is
 > BR2_PACKAGE_HAS_FOO, which allows treating virtual packages like the
 > other kinds of packages. There, we explicitly kept virtual packages out
 > of the list, though (the reasoning was that virtual packages install
 > nothing in host/ or target/, so they do not directly contribute to the
 > final content, so we do not need to rsync them, so this was an
 > optimisation).

 > However, virtual packages are in fact actual generic packages, and it is
 > possible for virtual packages to actually provide content for the final
 > image. Even though we do not have any virtual package that has actual
 > _INSTALL_CMDS, we still have udev that provides a user for example;
 > virtual packages in br2-external trees may also very well provide
 > install commands (e.g. to install files common to their various
 > implementations).

 > So, there is currently no technical reason to exclude virtual packages
 > from PACKAGES, the list of packages enabled in the configuration.

 > Drop the excluding condition, and always add enabled package, whatever
 > their kind, to the list of enabled packages.

 > [0] defconfig to reproduce the issue:
 >     BR2_INIT_NONE=y
 >     BR2_SYSTEM_BIN_SH_NONE=y
 >     # BR2_PACKAGE_BUSYBOX is not set
 >     # BR2_PACKAGE_IFUPDOWN_SCRIPTS is not set
 >     # BR2_TARGET_ROOTFS_TAR is not set

 > Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>

Committed to 2022.02.x, 2022.05.x and 2022.08.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot