[Buildroot] [PATCH] Makefile: fix build when $(O) ends in _defconfig

[Buildroot] [PATCH] Makefile: fix build when $(O) ends in _defconfig

[Yann E. MORIN]

Commit e6195c53041f (Makefile: fix use of many br2-external trees) fixed
a slowdown with many br2-external trees. In doing so, it changed the
type of the %_defconfig rule: the stem is no longer present in the
prerequisites, so it changes from a pattern rule to an implicit pattern
rule [0].

It is not unusual to name the build directory after the defconfig that
is being built, so we may end up with a build directory named
meh_defconfig. Before e6195c53041f, the pattern rule would not match
[1], but now it does, which causes somewhat-cryptic build failures:

    Makefile:1015: *** "Can't find /some/path/meh_defconfig".  Stop.

The issue is that we have this set of rules and assignments (elided and
reordered for legibility):

    all: world
    world: target-post-image
    target-post-image: staging-finalize
    staging-finalize: $(STAGING_DIR_SYMLINK)
    $(STAGING_DIR_SYMLINK): | $(BASE_DIR)
    BASE_DIR := $(CANONICAL_O)
    CANONICAL_O := $(shell mkdir -p $(O) >/dev/null 2>&1)$(realpath $(O))

So, there is a rule that (eventually) has a dependency on $(O), but we
have no rule that provides it explicitly, so the %_defconfig rule kicks
in, with the stem as "/some/path/meh". When the loop searches all the
".../configs/" directories for a file named ".../configs/%_defconfig",
it actually lookes for a file named ".../configs//some/path/meh_defconfig"
and that indeed never matches anything.

The solution is to provide an actual rule for $(BASE_DIR), so that the
implicit rule does not kick in.

[0] Terminology and behaviour in make is hard, so the terms we used here
    may be wrong or incorrectly used, and/or the explanations for the
    behavious be wrong or incomplete... Still, the reasoning stands, and
    the root cause is the removal of the stem in the RHS of the rule
    (adding one back does fix the issue).

[1] not sure how the prerequisite was solved  before e6195c53041f,
    though...

Fixes: e6195c53041f

Reported-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Nevo Hed <nhed+buildroot@starry.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 8517d563d5..f39af5eae9 100644
--- a/Makefile
+++ b/Makefile
@@ -1042,7 +1042,7 @@ savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 
 # staging and target directories do NOT list these as
 # dependencies anywhere else
-$(BUILD_DIR) $(BASE_TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST) $(PER_PACKAGE_DIR):
+$(BASE_DIR) $(BUILD_DIR) $(BASE_TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST) $(PER_PACKAGE_DIR):
 	@mkdir -p $@
 
 # outputmakefile generates a Makefile in the output directory, if using a
-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] Makefile: fix build when $(O) ends in _defconfig

[Nevo Hed via buildroot]

Yann,

Though I'm a tad baffled by the concept of naming anything but a
defconfig (Kconf) file with the same pattern, the fix seems
reasonable.

--Nevo

On Sun, Jan 22, 2023 at 4:09 AM Yann E. MORIN <yann.morin.1998@free.fr> wrote:
>
> Commit e6195c53041f (Makefile: fix use of many br2-external trees) fixed
> a slowdown with many br2-external trees. In doing so, it changed the
> type of the %_defconfig rule: the stem is no longer present in the
> prerequisites, so it changes from a pattern rule to an implicit pattern
> rule [0].
>
> It is not unusual to name the build directory after the defconfig that
> is being built, so we may end up with a build directory named
> meh_defconfig. Before e6195c53041f, the pattern rule would not match
> [1], but now it does, which causes somewhat-cryptic build failures:
>
>     Makefile:1015: *** "Can't find /some/path/meh_defconfig".  Stop.
>
> The issue is that we have this set of rules and assignments (elided and
> reordered for legibility):
>
>     all: world
>     world: target-post-image
>     target-post-image: staging-finalize
>     staging-finalize: $(STAGING_DIR_SYMLINK)
>     $(STAGING_DIR_SYMLINK): | $(BASE_DIR)
>     BASE_DIR := $(CANONICAL_O)
>     CANONICAL_O := $(shell mkdir -p $(O) >/dev/null 2>&1)$(realpath $(O))
>
> So, there is a rule that (eventually) has a dependency on $(O), but we
> have no rule that provides it explicitly, so the %_defconfig rule kicks
> in, with the stem as "/some/path/meh". When the loop searches all the
> ".../configs/" directories for a file named ".../configs/%_defconfig",
> it actually lookes for a file named ".../configs//some/path/meh_defconfig"
> and that indeed never matches anything.
>
> The solution is to provide an actual rule for $(BASE_DIR), so that the
> implicit rule does not kick in.
>
> [0] Terminology and behaviour in make is hard, so the terms we used here
>     may be wrong or incorrectly used, and/or the explanations for the
>     behavious be wrong or incomplete... Still, the reasoning stands, and
>     the root cause is the removal of the stem in the RHS of the rule
>     (adding one back does fix the issue).
>
> [1] not sure how the prerequisite was solved  before e6195c53041f,
>     though...
>
> Fixes: e6195c53041f
>
> Reported-by: Romain Naour <romain.naour@gmail.com>
> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
> Cc: Nevo Hed <nhed+buildroot@starry.com>
> Cc: Peter Korsgaard <peter@korsgaard.com>
> ---
>  Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/Makefile b/Makefile
> index 8517d563d5..f39af5eae9 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1042,7 +1042,7 @@ savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>
>  # staging and target directories do NOT list these as
>  # dependencies anywhere else
> -$(BUILD_DIR) $(BASE_TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST) $(PER_PACKAGE_DIR):
> +$(BASE_DIR) $(BUILD_DIR) $(BASE_TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST) $(PER_PACKAGE_DIR):
>         @mkdir -p $@
>
>  # outputmakefile generates a Makefile in the output directory, if using a
> --
> 2.25.1
>
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] Makefile: fix build when $(O) ends in _defconfig

[Sebastian WEYER]

Hi all,

On 22/01/2023 10:09, Yann E. MORIN wrote:
> Commit e6195c53041f (Makefile: fix use of many br2-external trees) fixed
> a slowdown with many br2-external trees. In doing so, it changed the
> type of the %_defconfig rule: the stem is no longer present in the
> prerequisites, so it changes from a pattern rule to an implicit pattern
> rule [0].
>
> It is not unusual to name the build directory after the defconfig that
> is being built, so we may end up with a build directory named
> meh_defconfig. Before e6195c53041f, the pattern rule would not match
> [1], but now it does, which causes somewhat-cryptic build failures:
>
>     Makefile:1015: *** "Can't find /some/path/meh_defconfig".  Stop.
>
> The issue is that we have this set of rules and assignments (elided and
> reordered for legibility):
>
>     all: world
>     world: target-post-image
>     target-post-image: staging-finalize
>     staging-finalize: $(STAGING_DIR_SYMLINK)
>     $(STAGING_DIR_SYMLINK): | $(BASE_DIR)
>     BASE_DIR := $(CANONICAL_O)
>     CANONICAL_O := $(shell mkdir -p $(O) >/dev/null 2>&1)$(realpath $(O))
>
> So, there is a rule that (eventually) has a dependency on $(O), but we
> have no rule that provides it explicitly, so the %_defconfig rule kicks
> in, with the stem as "/some/path/meh". When the loop searches all the
> ".../configs/" directories for a file named ".../configs/%_defconfig",
> it actually lookes for a file named ".../configs//some/path/meh_defconfig"
> and that indeed never matches anything.
>
> The solution is to provide an actual rule for $(BASE_DIR), so that the
> implicit rule does not kick in.
>
> [0] Terminology and behaviour in make is hard, so the terms we used here
>     may be wrong or incorrectly used, and/or the explanations for the
>     behavious be wrong or incomplete... Still, the reasoning stands, and
>     the root cause is the removal of the stem in the RHS of the rule
>     (adding one back does fix the issue).
>
> [1] not sure how the prerequisite was solved  before e6195c53041f,
>     though...
>
> Fixes: e6195c53041f
>
> Reported-by: Romain Naour <romain.naour@gmail.com>
> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
> Cc: Nevo Hed <nhed+buildroot@starry.com>
> Cc: Peter Korsgaard <peter@korsgaard.com>
> ---
>  Makefile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/Makefile b/Makefile
> index 8517d563d5..f39af5eae9 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1042,7 +1042,7 @@ savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
>  
>  # staging and target directories do NOT list these as
>  # dependencies anywhere else
> -$(BUILD_DIR) $(BASE_TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST) $(PER_PACKAGE_DIR):
> +$(BASE_DIR) $(BUILD_DIR) $(BASE_TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST) $(PER_PACKAGE_DIR):
>  	@mkdir -p $@
>  
>  # outputmakefile generates a Makefile in the output directory, if using a

I first verified that the build fails when the output folder's name ends in
"_defconfig" with "make O=./test/second_defconfig qemu_aarch64_virt_defconfig"
followed by "make O=./test/second_defconfig".
I then applied the patch and redid "make O=./test/second_defconfig" and verified
that the build succeeded.

Tested-by: Sebastian Weyer <sebastian.weyer@smile.fr>

Best regards,
Sebastian

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] Makefile: fix build when $(O) ends in _defconfig

[Peter Korsgaard]

>>>>> "Sebastian" == Sebastian WEYER <sebastian.weyer@smile.fr> writes:

 > Hi all,
 > On 22/01/2023 10:09, Yann E. MORIN wrote:
 >> Commit e6195c53041f (Makefile: fix use of many br2-external trees) fixed
 >> a slowdown with many br2-external trees. In doing so, it changed the
 >> type of the %_defconfig rule: the stem is no longer present in the
 >> prerequisites, so it changes from a pattern rule to an implicit pattern
 >> rule [0].
 >> 
 >> It is not unusual to name the build directory after the defconfig that
 >> is being built, so we may end up with a build directory named
 >> meh_defconfig. Before e6195c53041f, the pattern rule would not match
 >> [1], but now it does, which causes somewhat-cryptic build failures:
 >> 
 >> Makefile:1015: *** "Can't find /some/path/meh_defconfig".  Stop.
 >> 
 >> The issue is that we have this set of rules and assignments (elided and
 >> reordered for legibility):
 >> 
 >> all: world
 >> world: target-post-image
 >> target-post-image: staging-finalize
 >> staging-finalize: $(STAGING_DIR_SYMLINK)
 >> $(STAGING_DIR_SYMLINK): | $(BASE_DIR)
 >> BASE_DIR := $(CANONICAL_O)
 >> CANONICAL_O := $(shell mkdir -p $(O) >/dev/null 2>&1)$(realpath $(O))
 >> 
 >> So, there is a rule that (eventually) has a dependency on $(O), but we
 >> have no rule that provides it explicitly, so the %_defconfig rule kicks
 >> in, with the stem as "/some/path/meh". When the loop searches all the
 >> ".../configs/" directories for a file named ".../configs/%_defconfig",
 >> it actually lookes for a file named ".../configs//some/path/meh_defconfig"
 >> and that indeed never matches anything.
 >> 
 >> The solution is to provide an actual rule for $(BASE_DIR), so that the
 >> implicit rule does not kick in.
 >> 
 >> [0] Terminology and behaviour in make is hard, so the terms we used here
 >> may be wrong or incorrectly used, and/or the explanations for the
 >> behavious be wrong or incomplete... Still, the reasoning stands, and
 >> the root cause is the removal of the stem in the RHS of the rule
 >> (adding one back does fix the issue).
 >> 
 >> [1] not sure how the prerequisite was solved  before e6195c53041f,
 >> though...
 >> 
 >> Fixes: e6195c53041f
 >> 
 >> Reported-by: Romain Naour <romain.naour@gmail.com>
 >> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
 >> Cc: Nevo Hed <nhed+buildroot@starry.com>
 >> Cc: Peter Korsgaard <peter@korsgaard.com>
 >> ---
 >> Makefile | 2 +-
 >> 1 file changed, 1 insertion(+), 1 deletion(-)
 >> 
 >> diff --git a/Makefile b/Makefile
 >> index 8517d563d5..f39af5eae9 100644
 >> --- a/Makefile
 >> +++ b/Makefile
 >> @@ -1042,7 +1042,7 @@ savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 >> 
 >> # staging and target directories do NOT list these as
 >> # dependencies anywhere else
 >> -$(BUILD_DIR) $(BASE_TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST) $(PER_PACKAGE_DIR):
 >> +$(BASE_DIR) $(BUILD_DIR) $(BASE_TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST) $(PER_PACKAGE_DIR):
 >> @mkdir -p $@
 >> 
 >> # outputmakefile generates a Makefile in the output directory, if using a

 > I first verified that the build fails when the output folder's name ends in
 > "_defconfig" with "make O=./test/second_defconfig qemu_aarch64_virt_defconfig"
 > followed by "make O=./test/second_defconfig".
 > I then applied the patch and redid "make O=./test/second_defconfig" and verified
 > that the build succeeded.

 > Tested-by: Sebastian Weyer <sebastian.weyer@smile.fr>

Committed to 2022.11.x and 2022.02.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot