* [Buildroot] Cannot rebuild autobuild @ 2017-11-13 10:27 Petr Vorel 2017-11-13 11:06 ` Baruch Siach 2017-11-13 14:42 ` Petr Vorel 0 siblings, 2 replies; 20+ messages in thread From: Petr Vorel @ 2017-11-13 10:27 UTC (permalink / raw) To: buildroot Hi Peter, I'm trying to fix failing feh on mips64el [1] (btw an easy fix, just to add -std=c99 or -std=c11 to TARGET_CFLAGS either globally or just for this toolchain/architecture). But I cannot rebuild it as it fails to pass configure: $ make distclean $ wget -c http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30/config -O .config.old $ make olddefconfig $ make make: *** No rule to make target 'toolchain-external-custom', needed by '/home/foo/install/src/buildroot.git/output/build/toolchain-external/.stamp_configured'. Stop. What can be wrong? I've done it this way in the past (with other autobuild items, which had different toolchains). Kind regards, Petr [1] http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30 ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-13 10:27 [Buildroot] Cannot rebuild autobuild Petr Vorel @ 2017-11-13 11:06 ` Baruch Siach 2017-11-13 14:42 ` Petr Vorel 1 sibling, 0 replies; 20+ messages in thread From: Baruch Siach @ 2017-11-13 11:06 UTC (permalink / raw) To: buildroot Hi Petr, On Mon, Nov 13, 2017 at 11:27:58AM +0100, Petr Vorel wrote: > I'm trying to fix failing feh on mips64el [1] (btw an easy fix, just to add > -std=c99 or > -std=c11 to TARGET_CFLAGS either globally or just for this toolchain/architecture). > > But I cannot rebuild it as it fails to pass configure: > > $ make distclean > $ wget -c http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30/config -O .config.old If you do '-O .config' here, does that work better? > $ make olddefconfig > $ make > make: *** No rule to make target 'toolchain-external-custom', needed by '/home/foo/install/src/buildroot.git/output/build/toolchain-external/.stamp_configured'. Stop. > > What can be wrong? I've done it this way in the past (with other autobuild items, which > had different toolchains). baruch -- http://baruch.siach.name/blog/ ~. .~ Tk Open Systems =}------------------------------------------------ooO--U--Ooo------------{= - baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il - ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-13 10:27 [Buildroot] Cannot rebuild autobuild Petr Vorel 2017-11-13 11:06 ` Baruch Siach @ 2017-11-13 14:42 ` Petr Vorel 2017-11-14 22:44 ` Peter Seiderer 1 sibling, 1 reply; 20+ messages in thread From: Petr Vorel @ 2017-11-13 14:42 UTC (permalink / raw) To: buildroot > But I cannot rebuild it as it fails to pass configure: > $ make distclean > $ wget -c http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30/config -O .config.old > $ make olddefconfig > $ make > make: *** No rule to make target 'toolchain-external-custom', needed by '/home/foo/install/src/buildroot.git/output/build/toolchain-external/.stamp_configured'. Stop. > What can be wrong? I've done it this way in the past (with other autobuild items, which > had different toolchains). > [1] http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30 And of course, I'm working on current master. Kind regards, Petr ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-13 14:42 ` Petr Vorel @ 2017-11-14 22:44 ` Peter Seiderer 2017-11-15 12:58 ` Petr Vorel 0 siblings, 1 reply; 20+ messages in thread From: Peter Seiderer @ 2017-11-14 22:44 UTC (permalink / raw) To: buildroot Hello Petr, On Mon, 13 Nov 2017 15:42:32 +0100, Petr Vorel <petr.vorel@gmail.com> wrote: > > But I cannot rebuild it as it fails to pass configure: > > > $ make distclean > > $ wget -c http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30/config -O .config.old Your should store the downloaded config file definitely as '.config' instead of '.config.old' (there is only the way .config --> .config.old and not the other way round in buildroot)... > > $ make olddefconfig > > $ make > > make: *** No rule to make target 'toolchain-external-custom', needed by '/home/foo/install/src/buildroot.git/output/build/toolchain-external/.stamp_configured'. Stop. > > > What can be wrong? I've done it this way in the past (with other autobuild items, which > > had different toolchains). Same here with a fresh installed openSUSE Tumbleweed x86_64 without 32-bit support packages installed. Until commit ec54092c916440501006bfa51a820eb953100510 ('toolchain-external-custom: new package'), or with 2016.11.3 the same config gives the following error instead: [...] Cannot execute cross-compiler '.../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc' make[1]: *** [package/pkg-generic.mk:209: .../build/toolchain-external/.stamp_configured] Error 1 make: *** [Makefile:16: _all] Error 2 Maybe the toolchain definition should select BR2_HOSTARCH_NEEDS_IA32_LIBS? Regards, Peter > > > [1] http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30 > > And of course, I'm working on current master. > > > Kind regards, > Petr > _______________________________________________ > buildroot mailing list > buildroot at busybox.net > http://lists.busybox.net/mailman/listinfo/buildroot ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-14 22:44 ` Peter Seiderer @ 2017-11-15 12:58 ` Petr Vorel 2017-11-15 23:03 ` Arnout Vandecappelle 0 siblings, 1 reply; 20+ messages in thread From: Petr Vorel @ 2017-11-15 12:58 UTC (permalink / raw) To: buildroot Hi Peter, ... > > > $ wget -c http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30/config -O .config.old > Your should store the downloaded config file definitely as '.config' instead of '.config.old' (there is only > the way .config --> .config.old and not the other way round in buildroot)... Thanks for clarification. I actually realised this as well. This is not an issue, as I tested it as being .config. > > > $ make olddefconfig > > > $ make > > > make: *** No rule to make target 'toolchain-external-custom', needed by '/home/foo/install/src/buildroot.git/output/build/toolchain-external/.stamp_configured'. Stop. > > > What can be wrong? I've done it this way in the past (with other autobuild items, which > > > had different toolchains). > Same here with a fresh installed openSUSE Tumbleweed x86_64 without 32-bit support packages installed. > Until commit ec54092c916440501006bfa51a820eb953100510 ('toolchain-external-custom: new package'), or with > 2016.11.3 the same config gives the following error instead: > [...] > Cannot execute cross-compiler '.../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc' > make[1]: *** [package/pkg-generic.mk:209: .../build/toolchain-external/.stamp_configured] Error 1 > make: *** [Makefile:16: _all] Error 2 > Maybe the toolchain definition should select BR2_HOSTARCH_NEEDS_IA32_LIBS? It must be some system library dependency as I was able to run the build on Debian unstable. Failing build is on openSUSE Tumbleweed x86_64 (the same as you), but I have glibc-32bit and many other 32 bit packages installed (as well on Debian machine). I tried to play with BR2_HOSTARCH_NEEDS_IA32_LIBS, but didn't found the solution. Thomas, any idea what can be wrong? Kind regards, Petr ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-15 12:58 ` Petr Vorel @ 2017-11-15 23:03 ` Arnout Vandecappelle 2017-11-15 23:19 ` Peter Seiderer 2017-11-16 7:14 ` [Buildroot] " Petr Vorel 0 siblings, 2 replies; 20+ messages in thread From: Arnout Vandecappelle @ 2017-11-15 23:03 UTC (permalink / raw) To: buildroot On 15-11-17 13:58, Petr Vorel wrote: >> Cannot execute cross-compiler '.../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc' >> make[1]: *** [package/pkg-generic.mk:209: .../build/toolchain-external/.stamp_configured] Error 1 >> make: *** [Makefile:16: _all] Error 2 >> Maybe the toolchain definition should select BR2_HOSTARCH_NEEDS_IA32_LIBS? > It must be some system library dependency as I was able to run the build on Debian unstable. > Failing build is on openSUSE Tumbleweed x86_64 (the same as you), but I have glibc-32bit > and many other 32 bit packages installed (as well on Debian machine). > I tried to play with BR2_HOSTARCH_NEEDS_IA32_LIBS, but didn't found the solution. > Thomas, any idea what can be wrong? What does .../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc -v report? When you run ldd on it? When you run it under strace? What about the other binaries bundled in that tarball? What about other external toolchains (e.g. sourcery)? Regards, Arnout -- Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-15 23:03 ` Arnout Vandecappelle @ 2017-11-15 23:19 ` Peter Seiderer 2017-11-15 23:40 ` Arnout Vandecappelle 2017-11-16 7:14 ` [Buildroot] " Petr Vorel 1 sibling, 1 reply; 20+ messages in thread From: Peter Seiderer @ 2017-11-15 23:19 UTC (permalink / raw) To: buildroot Hello Arnout, On Thu, 16 Nov 2017 00:03:45 +0100, Arnout Vandecappelle <arnout@mind.be> wrote: > On 15-11-17 13:58, Petr Vorel wrote: > >> Cannot execute cross-compiler '.../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc' > >> make[1]: *** [package/pkg-generic.mk:209: .../build/toolchain-external/.stamp_configured] Error 1 > >> make: *** [Makefile:16: _all] Error 2 > >> Maybe the toolchain definition should select BR2_HOSTARCH_NEEDS_IA32_LIBS? > > It must be some system library dependency as I was able to run the build on Debian unstable. > > Failing build is on openSUSE Tumbleweed x86_64 (the same as you), but I have glibc-32bit > > and many other 32 bit packages installed (as well on Debian machine). > > I tried to play with BR2_HOSTARCH_NEEDS_IA32_LIBS, but didn't found the solution. > > Thomas, any idea what can be wrong? > > > What does .../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc -v > report? When you run ldd on it? When you run it under strace? What about the > other binaries bundled in that tarball? What about other external toolchains > (e.g. sourcery)? The above error message is from old buildroot 2016.11.3 (the compiler tar file is downloaded and upacked and fails at runtime)... The original error is with git master (or all buildroot past the ec54092c916440501006bfa51a820eb953100510 ('toolchain-external-custom: new package') commit: /usr/bin/install -m 0644 support/misc/target-dir-warning.txt .../build-prebuild-toolchain-001/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM ^[[7m>>> skeleton-init-sysv Extracting^[[27m ^[[7m>>> skeleton-init-sysv Patching^[[27m ^[[7m>>> skeleton-init-sysv Configuring^[[27m ^[[7m>>> skeleton-init-sysv Building^[[27m ^[[7m>>> skeleton-init-sysv Installing to target^[[27m rsync -a --ignore-times --exclude .svn --exclude .git --exclude .hg --exclude .bzr --exclude CVS --chmod=u=rwX,go=rX --exclude .empty --exclude '*~' package/skeleton-init-sysv//skeleton/ .../build-prebuild-toolchain-001/target/ ^[[7m>>> skeleton Extracting^[[27m ^[[7m>>> skeleton Patching^[[27m ^[[7m>>> skeleton Configuring^[[27m ^[[7m>>> skeleton Building^[[27m ^[[7m>>> skeleton Installing to target^[[27m make[1]: *** No rule to make target 'toolchain-external-custom', needed by '.../build-prebuild-toolchain-001/build/toolchain-external/.stamp_configured'. Stop. make: *** [Makefile:16: _all] Error 2 The error occurs before the prebuild toolchain is downloaded/extracted... Regards, Peter > > Regards, > Arnout ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-15 23:19 ` Peter Seiderer @ 2017-11-15 23:40 ` Arnout Vandecappelle 2017-11-16 7:29 ` Petr Vorel 0 siblings, 1 reply; 20+ messages in thread From: Arnout Vandecappelle @ 2017-11-15 23:40 UTC (permalink / raw) To: buildroot On 16-11-17 00:19, Peter Seiderer wrote: > Hello Arnout, > > On Thu, 16 Nov 2017 00:03:45 +0100, Arnout Vandecappelle <arnout@mind.be> wrote: > >> On 15-11-17 13:58, Petr Vorel wrote: >>>> Cannot execute cross-compiler '.../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc' >>>> make[1]: *** [package/pkg-generic.mk:209: .../build/toolchain-external/.stamp_configured] Error 1 >>>> make: *** [Makefile:16: _all] Error 2 >>>> Maybe the toolchain definition should select BR2_HOSTARCH_NEEDS_IA32_LIBS? >>> It must be some system library dependency as I was able to run the build on Debian unstable. >>> Failing build is on openSUSE Tumbleweed x86_64 (the same as you), but I have glibc-32bit >>> and many other 32 bit packages installed (as well on Debian machine). >>> I tried to play with BR2_HOSTARCH_NEEDS_IA32_LIBS, but didn't found the solution. >>> Thomas, any idea what can be wrong? >> >> >> What does .../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc -v >> report? When you run ldd on it? When you run it under strace? What about the >> other binaries bundled in that tarball? What about other external toolchains >> (e.g. sourcery)? > > The above error message is from old buildroot 2016.11.3 (the compiler tar file is downloaded and > upacked and fails at runtime)... > > The original error is with git master (or all buildroot past the ec54092c916440501006bfa51a820eb953100510 > ('toolchain-external-custom: new package') commit: > > /usr/bin/install -m 0644 support/misc/target-dir-warning.txt .../build-prebuild-toolchain-001/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM > ^[[7m>>> skeleton-init-sysv Extracting^[[27m > ^[[7m>>> skeleton-init-sysv Patching^[[27m > ^[[7m>>> skeleton-init-sysv Configuring^[[27m > ^[[7m>>> skeleton-init-sysv Building^[[27m > ^[[7m>>> skeleton-init-sysv Installing to target^[[27m > rsync -a --ignore-times --exclude .svn --exclude .git --exclude .hg --exclude .bzr --exclude CVS --chmod=u=rwX,go=rX --exclude .empty --exclude '*~' package/skeleton-init-sysv//skeleton/ .../build-prebuild-toolchain-001/target/ > ^[[7m>>> skeleton Extracting^[[27m > ^[[7m>>> skeleton Patching^[[27m > ^[[7m>>> skeleton Configuring^[[27m > ^[[7m>>> skeleton Building^[[27m > ^[[7m>>> skeleton Installing to target^[[27m > make[1]: *** No rule to make target 'toolchain-external-custom', needed by '.../build-prebuild-toolchain-001/build/toolchain-external/.stamp_configured'. Stop. > make: *** [Makefile:16: _all] Error 2 > > The error occurs before the prebuild toolchain is downloaded/extracted... That is really weird, the toolchain-external-custom dependencies should be defined no matter what... Could you pastebin the output make 'make -qp' somewhere? I would think it would be somehow related to gcc not being executable, but since this occurs before the package is even extracted... Just to be sure, what is the contents of output/build when you get this error? There is no toolchain-external-custom directory yet, right? Regards, Arnout -- Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-15 23:40 ` Arnout Vandecappelle @ 2017-11-16 7:29 ` Petr Vorel 2017-11-16 13:43 ` Arnout Vandecappelle 0 siblings, 1 reply; 20+ messages in thread From: Petr Vorel @ 2017-11-16 7:29 UTC (permalink / raw) To: buildroot Hi, ... > > make[1]: *** No rule to make target 'toolchain-external-custom', needed by '.../build-prebuild-toolchain-001/build/toolchain-external/.stamp_configured'. Stop. > > make: *** [Makefile:16: _all] Error 2 > > The error occurs before the prebuild toolchain is downloaded/extracted... > That is really weird, the toolchain-external-custom dependencies should be > defined no matter what... Could you pastebin the output make 'make -qp' somewhere? Here it is: http://sipe.cz/buildroot/out.log (68M) http://sipe.cz/buildroot/out.log.tar.gz (9MB) Kind regards, Petr ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-16 7:29 ` Petr Vorel @ 2017-11-16 13:43 ` Arnout Vandecappelle 2017-11-18 21:22 ` Peter Seiderer 2017-11-18 21:32 ` Peter Seiderer 0 siblings, 2 replies; 20+ messages in thread From: Arnout Vandecappelle @ 2017-11-16 13:43 UTC (permalink / raw) To: buildroot On 16-11-17 08:29, Petr Vorel wrote: > Hi, > > ... >>> make[1]: *** No rule to make target 'toolchain-external-custom', needed by '.../build-prebuild-toolchain-001/build/toolchain-external/.stamp_configured'. Stop. >>> make: *** [Makefile:16: _all] Error 2 > >>> The error occurs before the prebuild toolchain is downloaded/extracted... > >> That is really weird, the toolchain-external-custom dependencies should be >> defined no matter what... Could you pastebin the output make 'make -qp' somewhere? > Here it is: > http://sipe.cz/buildroot/out.log (68M) > http://sipe.cz/buildroot/out.log.tar.gz (9MB) Oh boy... The problem is that on Tumbleweed the order of inclusion seems to be different, which causes toolchain-external-package not to be defined at the time that toolchain-external-custom.mk is included (same for all other external toolchains). Makefile:492:include toolchain/*/*.mk This causes inclusion of toolchain/toolchain-external/toolchain-external.mk:28:include toolchain/toolchain-external/*/*.mk and toolchain/toolchain-external/pkg-toolchain-external.mk:592:toolchain-external-package = ... If the *.mk is expanded in alphabetical order, pkg-toolchain-external.mk will always come before toolchain-external.mk. Otherwise it is possible (depending on the state of hash tables or inode numbers or whatever) that toolchain-external-custom.mk gets included first. The short-term solution is of course $(sort $(wildcard ...)) But for the long term, I think it's better to move pkg-*.mk to support/misc (or maybe to support/make) and organise the order of inclusion from there. Any takers? Yann? Regards, Arnout -- Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-16 13:43 ` Arnout Vandecappelle @ 2017-11-18 21:22 ` Peter Seiderer 2017-11-20 21:44 ` Yann E. MORIN 2017-11-18 21:32 ` Peter Seiderer 1 sibling, 1 reply; 20+ messages in thread From: Peter Seiderer @ 2017-11-18 21:22 UTC (permalink / raw) To: buildroot Hello Arnout, On Thu, 16 Nov 2017 14:43:48 +0100, Arnout Vandecappelle <arnout@mind.be> wrote: > On 16-11-17 08:29, Petr Vorel wrote: > > Hi, > > > > ... > >>> make[1]: *** No rule to make target 'toolchain-external-custom', needed by '.../build-prebuild-toolchain-001/build/toolchain-external/.stamp_configured'. Stop. > >>> make: *** [Makefile:16: _all] Error 2 > > > >>> The error occurs before the prebuild toolchain is downloaded/extracted... > > > >> That is really weird, the toolchain-external-custom dependencies should be > >> defined no matter what... Could you pastebin the output make 'make -qp' somewhere? > > Here it is: > > http://sipe.cz/buildroot/out.log (68M) > > http://sipe.cz/buildroot/out.log.tar.gz (9MB) > > Oh boy... > > The problem is that on Tumbleweed the order of inclusion seems to be different, > which causes toolchain-external-package not to be defined at the time that > toolchain-external-custom.mk is included (same for all other external toolchains). > > Makefile:492:include toolchain/*/*.mk > > This causes inclusion of > toolchain/toolchain-external/toolchain-external.mk:28:include > toolchain/toolchain-external/*/*.mk > and > toolchain/toolchain-external/pkg-toolchain-external.mk:592:toolchain-external-package > = ... > > If the *.mk is expanded in alphabetical order, pkg-toolchain-external.mk will > always come before toolchain-external.mk. Otherwise it is possible (depending on > the state of hash tables or inode numbers or whatever) that > toolchain-external-custom.mk gets included first. > > The short-term solution is of course $(sort $(wildcard ...)) Your investigation seems to be right, the following patch fixes it for me (no 32-bit support installed yet, but the cross compiler is extracted and I get the 'Cannot execute cross-compiler' error message again): diff --git a/Makefile b/Makefile index 55409b99af..9d9737332d 100644 --- a/Makefile +++ b/Makefile @@ -483,13 +483,13 @@ include system/system.mk include package/Makefile.in # arch/arch.mk.* must be after package/Makefile.in because it may need to # complement variables defined therein, like BR_NO_CHECK_HASH_FOR. --include $(wildcard arch/arch.mk.*) +-include $(sort $(wildcard arch/arch.mk.*)) include support/dependencies/dependencies.mk PACKAGES += $(DEPENDENCIES_HOST_PREREQ) -include toolchain/*.mk -include toolchain/*/*.mk +include $(sort $(wildcard toolchain/*.mk)) +include $(sort $(wildcard toolchain/*/*.mk)) # Include the package override file if one has been provided in the # configuration. diff --git a/toolchain/toolchain-external/toolchain-external.mk b/toolchain/toolchain-external/toolchain-external.mk index 48de1e71f2..dd07884c7e 100644 --- a/toolchain/toolchain-external/toolchain-external.mk +++ b/toolchain/toolchain-external/toolchain-external.mk @@ -25,4 +25,4 @@ $(error No prefix selected for external toolchain package $(BR2_PACKAGE_PROVIDES endif endif -include toolchain/toolchain-external/*/*.mk +include $(sort $(wildcard toolchain/toolchain-external/*/*.mk)) Regards, Peter > > But for the long term, I think it's better to move pkg-*.mk to support/misc (or > maybe to support/make) and organise the order of inclusion from there. > > Any takers? Yann? > > Regards, > Arnout > ^ permalink raw reply related [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-18 21:22 ` Peter Seiderer @ 2017-11-20 21:44 ` Yann E. MORIN 0 siblings, 0 replies; 20+ messages in thread From: Yann E. MORIN @ 2017-11-20 21:44 UTC (permalink / raw) To: buildroot Peter, Arnout, All, On 2017-11-18 22:22 +0100, Peter Seiderer spake thusly: > On Thu, 16 Nov 2017 14:43:48 +0100, Arnout Vandecappelle <arnout@mind.be> wrote: [--SNIP--] > > Oh boy... > > > > The problem is that on Tumbleweed the order of inclusion seems to be different, > > which causes toolchain-external-package not to be defined at the time that > > toolchain-external-custom.mk is included (same for all other external toolchains). > > > > Makefile:492:include toolchain/*/*.mk > > > > This causes inclusion of > > toolchain/toolchain-external/toolchain-external.mk:28:include > > toolchain/toolchain-external/*/*.mk > > and > > toolchain/toolchain-external/pkg-toolchain-external.mk:592:toolchain-external-package > > = ... > > > > If the *.mk is expanded in alphabetical order, pkg-toolchain-external.mk will > > always come before toolchain-external.mk. Otherwise it is possible (depending on > > the state of hash tables or inode numbers or whatever) that > > toolchain-external-custom.mk gets included first. > > > > The short-term solution is of course $(sort $(wildcard ...)) > > Your investigation seems to be right, the following patch fixes it for me (no > 32-bit support installed yet, but the cross compiler is extracted and I get > the 'Cannot execute cross-compiler' error message again): [--SNIP--] > -include toolchain/*.mk > -include toolchain/*/*.mk > +include $(sort $(wildcard toolchain/*.mk)) > +include $(sort $(wildcard toolchain/*/*.mk)) I think that using $(wildcard) explicitly is Good (TM), rather than using globs. Also, $(sort) is Good Too (TM) because it guarantees the build order. So far, $(sort) internally used strcmp() to compare strings [0] [1], so works on bytes, not characters, so does not account for the locale settings. However, that may change in the future, so we'd have to enforce the C locale to guarantee ordering. Can you submit this officially, please? Keep me in Cc when you do so, thanks! ;-) [0] https://git.savannah.gnu.org/cgit/make.git/tree/src/function.c#n1161 [1] https://git.savannah.gnu.org/cgit/make.git/tree/src/misc.c#n41 Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-16 13:43 ` Arnout Vandecappelle 2017-11-18 21:22 ` Peter Seiderer @ 2017-11-18 21:32 ` Peter Seiderer 2017-11-21 20:16 ` Petr Vorel 1 sibling, 1 reply; 20+ messages in thread From: Peter Seiderer @ 2017-11-18 21:32 UTC (permalink / raw) To: buildroot Hello Arnout, On Thu, 16 Nov 2017 14:43:48 +0100, Arnout Vandecappelle <arnout@mind.be> wrote: > On 16-11-17 08:29, Petr Vorel wrote: > > Hi, > > > > ... > >>> make[1]: *** No rule to make target 'toolchain-external-custom', needed by '.../build-prebuild-toolchain-001/build/toolchain-external/.stamp_configured'. Stop. > >>> make: *** [Makefile:16: _all] Error 2 > > > >>> The error occurs before the prebuild toolchain is downloaded/extracted... > > > >> That is really weird, the toolchain-external-custom dependencies should be > >> defined no matter what... Could you pastebin the output make 'make -qp' somewhere? > > Here it is: > > http://sipe.cz/buildroot/out.log (68M) > > http://sipe.cz/buildroot/out.log.tar.gz (9MB) > > Oh boy... > > The problem is that on Tumbleweed the order of inclusion seems to be different, > which causes toolchain-external-package not to be defined at the time that > toolchain-external-custom.mk is included (same for all other external toolchains). > > Makefile:492:include toolchain/*/*.mk > > This causes inclusion of > toolchain/toolchain-external/toolchain-external.mk:28:include > toolchain/toolchain-external/*/*.mk > and > toolchain/toolchain-external/pkg-toolchain-external.mk:592:toolchain-external-package > = ... > > If the *.mk is expanded in alphabetical order, pkg-toolchain-external.mk will > always come before toolchain-external.mk. Otherwise it is possible (depending on > the state of hash tables or inode numbers or whatever) that > toolchain-external-custom.mk gets included first. According to this [1] it is a change from the used gnu-make version (on Tumpbleweed GNU Make 4.2.1 is used)? Regards, Peter [1] https://stackoverflow.com/questions/40558385/gnu-make-wildcard-no-longer-gives-sorted-output-is-there-any-control-switch > > The short-term solution is of course $(sort $(wildcard ...)) > > But for the long term, I think it's better to move pkg-*.mk to support/misc (or > maybe to support/make) and organise the order of inclusion from there. > > Any takers? Yann? > > Regards, > Arnout > ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-18 21:32 ` Peter Seiderer @ 2017-11-21 20:16 ` Petr Vorel 2017-11-25 15:01 ` Yann E. MORIN 0 siblings, 1 reply; 20+ messages in thread From: Petr Vorel @ 2017-11-21 20:16 UTC (permalink / raw) To: buildroot Hi, > > If the *.mk is expanded in alphabetical order, pkg-toolchain-external.mk will > > always come before toolchain-external.mk. Otherwise it is possible (depending on > > the state of hash tables or inode numbers or whatever) that > > toolchain-external-custom.mk gets included first. > According to this [1] it is a change from the used gnu-make version (on Tumbleweed > GNU Make 4.2.1 is used)? I found by bisecting that our bug is caused by this change in make: 4fd5672 ("Use Jenkins hash.") https://git.savannah.gnu.org/cgit/make.git/commit/?id=4fd56724ad281498d3c8b27a4b25b4070f6e4e65 It's a version after last release (4.2.1). I wonder whether it's a bug in make (I guess so) and whether we should report it (probably). As 4.2.1 compiled from source works, I don't know what broke it for 4.2.1 from openSUSE. I expected this patch, but that is working. https://build.opensuse.org/package/view_file/Base:System/make/make-sorted-glob.patch?expand=1 I tested Peter's patch with broken make versions, fixes them. Kind regards, Petr > Regards, > Peter > [1] https://stackoverflow.com/questions/40558385/gnu-make-wildcard-no-longer-gives-sorted-output-is-there-any-control-switch ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-21 20:16 ` Petr Vorel @ 2017-11-25 15:01 ` Yann E. MORIN 2017-11-25 16:34 ` Arnout Vandecappelle 0 siblings, 1 reply; 20+ messages in thread From: Yann E. MORIN @ 2017-11-25 15:01 UTC (permalink / raw) To: buildroot Petr, All, On 2017-11-21 21:16 +0100, Petr Vorel spake thusly: > > > If the *.mk is expanded in alphabetical order, pkg-toolchain-external.mk will > > > always come before toolchain-external.mk. Otherwise it is possible (depending on > > > the state of hash tables or inode numbers or whatever) that > > > toolchain-external-custom.mk gets included first. [--SNIP--] > I found by bisecting that our bug is caused by this change in make: > 4fd5672 ("Use Jenkins hash.") > https://git.savannah.gnu.org/cgit/make.git/commit/?id=4fd56724ad281498d3c8b27a4b25b4070f6e4e65 Good work at bisecting it! :-) > It's a version after last release (4.2.1). > I wonder whether it's a bug in make (I guess so) and whether we should report it > (probably). This is indeed a change in behaviour. However, as make ever documented a guranteed/non-guaranteed ordering of wildcards to begin with? The manual for the current make version says nothing about the ordering, and neither do the manual for older versions (I've checked only a few of them). So, this is a change in behaviour, and should be reported upstream as a bug. Will you do so? Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-25 15:01 ` Yann E. MORIN @ 2017-11-25 16:34 ` Arnout Vandecappelle 2017-11-29 21:19 ` Petr Vorel 0 siblings, 1 reply; 20+ messages in thread From: Arnout Vandecappelle @ 2017-11-25 16:34 UTC (permalink / raw) To: buildroot On 25-11-17 16:01, Yann E. MORIN wrote: > Petr, All, > > On 2017-11-21 21:16 +0100, Petr Vorel spake thusly: >>>> If the *.mk is expanded in alphabetical order, pkg-toolchain-external.mk will >>>> always come before toolchain-external.mk. Otherwise it is possible (depending on >>>> the state of hash tables or inode numbers or whatever) that >>>> toolchain-external-custom.mk gets included first. > [--SNIP--] >> I found by bisecting that our bug is caused by this change in make: >> 4fd5672 ("Use Jenkins hash.") >> https://git.savannah.gnu.org/cgit/make.git/commit/?id=4fd56724ad281498d3c8b27a4b25b4070f6e4e65 > > Good work at bisecting it! :-) > >> It's a version after last release (4.2.1). >> I wonder whether it's a bug in make (I guess so) and whether we should report it >> (probably). > > This is indeed a change in behaviour. However, as make ever documented > a guranteed/non-guaranteed ordering of wildcards to begin with? > > The manual for the current make version says nothing about the ordering, > and neither do the manual for older versions (I've checked only a few of > them). > > So, this is a change in behaviour, and should be reported upstream as a > bug. Will you do so? As I explained in the extended commit message when I applied the patch fixing this (b9d2d4cb4eb: "Fix makefile include order by using sort/wildcard.") make actually has had this behaviour "forever" (since make 3.82). It calls glob with GLOB_NOSORT. Note that in most of the cases where we didn't already do $(sort $(wildcard ...)), the order actually doesn't (or shouldn't) matter. For example, package/*/*.mk hopefully does not depend on the order. In other words, it's very likely that nobody notices if things get included in the "wrong" order. As far as I could see, only the external-toolchain-package really could cause any problems. So my guess is that the bisected commit really has nothing to do with it, it just happens to trigger the random ordering issue. Now that I look at it again, however, it looks like we haven't been able to adequately explain why this issue suddenly occurred... First of all, external-toolchain-package has existed for a year now, so if it is truly random we'd expect someone to have noticed before. Also, the change in Tumbleweed that Petr pointed to [1] should actually *fix* the issue, because it removes the NOSORT. Debian and Ubuntu, conversely, have no such patch so the autobuilders should see this issue (they use external toolchains a lot)... /me is confused again... Regards, Arnout [1] https://build.opensuse.org/package/view_file/Base:System/make/make-sorted-glob.patch?expand=1 -- Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-25 16:34 ` Arnout Vandecappelle @ 2017-11-29 21:19 ` Petr Vorel 2017-12-15 20:22 ` Petr Vorel 0 siblings, 1 reply; 20+ messages in thread From: Petr Vorel @ 2017-11-29 21:19 UTC (permalink / raw) To: buildroot Hi, > > So, this is a change in behaviour, and should be reported upstream as a > > bug. Will you do so? > As I explained in the extended commit message when I applied the patch fixing > this (b9d2d4cb4eb: "Fix makefile include order by using sort/wildcard.") make > actually has had this behaviour "forever" (since make 3.82). It calls glob with > GLOB_NOSORT. Note that in most of the cases where we didn't already do $(sort > $(wildcard ...)), the order actually doesn't (or shouldn't) matter. For example, > package/*/*.mk hopefully does not depend on the order. In other words, it's very > likely that nobody notices if things get included in the "wrong" order. Even though I understand that the change in 3.82 you pointed out *should* change the behavior, it's different: 3.82 is working. > As far as I could see, only the external-toolchain-package really could cause > any problems. > So my guess is that the bisected commit really has nothing to do with it, it > just happens to trigger the random ordering issue. I don't understand changes in 4fd5672 ("Use Jenkins hash.") [2], but it does it permanently (I tested it several times). That's why I thought it would be good to notify upstream. Now I'm not sure (I haven't reported it yet). > Now that I look at it again, however, it looks like we haven't been able to > adequately explain why this issue suddenly occurred... First of all, > external-toolchain-package has existed for a year now, so if it is truly random > we'd expect someone to have noticed before. Also, the change in Tumbleweed that > Petr pointed to [1] should actually *fix* the issue, because it removes the > NOSORT. Debian and Ubuntu, conversely, have no such patch so the autobuilders > should see this issue (they use external toolchains a lot)... I've been testing buildroot in Debian unstable for few years and in openSUSE Tumbleweed for about one year. I never noticed this, until now. You can see from [3], that the only thing that changed in openSUSE's make is glob-lstat.patch [4]. It actually changes glob, so it can be relevant. I can test it with and without it this week. In Debian I've been testing it probably on 3.81, 4.0 and 4.1. > /me is confused again... > Regards, > Arnout Kind regards, Petr [1] https://build.opensuse.org/package/view_file/Base:System/make/make-sorted-glob.patch?expand=1 [2] https://git.savannah.gnu.org/cgit/make.git/commit/?id=4fd56724ad281498d3c8b27a4b25b4070f6e4e65 [3] https://build.opensuse.org/package/view_file/Base:System/make/make.changes?expand=1 [4] https://build.opensuse.org/package/view_file/Base:System/make/glob-lstat.patch?expand=1 ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-29 21:19 ` Petr Vorel @ 2017-12-15 20:22 ` Petr Vorel 2017-12-15 20:23 ` [Buildroot] FW: " Kees van Unen 0 siblings, 1 reply; 20+ messages in thread From: Petr Vorel @ 2017-12-15 20:22 UTC (permalink / raw) To: buildroot Hi, > > > So, this is a change in behaviour, and should be reported upstream as a > > > bug. Will you do so? TL;DR: No I don't, IMHO it's not a bug in make. 4fd5672 [1] which broke buildroot unified behaviour on different os, which is good. Since this revision it behaves nearly the same as 3.81. It's only coincidence that make from current openSUSE Tumbleweed with make-sorted-glob.patch patch [2] behaves the same (this patch is completely different from [1]). I did comparison of make of these git versions: 3.81, 3.82, 4.2.1, 0ca3108 (last working version on buildroot before fix b9d2d4cb4e), 4fd5672 (version which broke buildroot b9d2d4cb4e~), 48c8a11 (current master) with this simple "project": $ for i in a b c d; do touch $i.mk; sleep 1; done $ cat > Makefile << EOF $(info make: $(shell which make)) $(info make -v: $(shell make -v)) $(foreach i, $(sort $(wildcard *.mk)), $(info SORT $i)) $(foreach i, $(wildcard *.mk), $(info NOSORT $i)) include $(wildcard *.mk) EOF make version | 3.81 | 3.82 | 4.2.1 | 4.2.1+[2] | 0ca3108 | 4fd5672 ------------------------------------------------------------------------ wildcard the same | yes | no | no | no | no | no as sorted wildchard| | | | | ------------------------------------------------------------------------ wildcard the same | yes | no | no | yes | no | yes on different OS | | | | | | ------------------------------------------------------------------------ Breaks buildroot | no | no | no | yes | no | yes NOTES: wildcard changed in 3.82 and 4fd5672. Behaviour on 48c8a11 (current master) is the same as on 4fd5672. "4.2.1+[2]" is 4.2.1 + make-sorted-glob.patch patch [2]. [2] https://build.opensuse.org/package/view_file/Base:System/make/make-sorted-glob.patch?expand=1 [2] https://git.savannah.gnu.org/cgit/make.git/commit/?id=4fd56724ad281498d3c8b27a4b25b4070f6e4e65 Kind regards, Petr ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] FW: Cannot rebuild autobuild 2017-12-15 20:22 ` Petr Vorel @ 2017-12-15 20:23 ` Kees van Unen 0 siblings, 0 replies; 20+ messages in thread From: Kees van Unen @ 2017-12-15 20:23 UTC (permalink / raw) To: buildroot ________________________________________ Van: buildrootNamensPetr Vorel Verzonden: vrijdag 15 december 2017 21:22:46 (UTC+01:00) Amsterdam, Berlijn, Bern, Rome, Stockholm, Wenen Aan: Arnout Vandecappelle CC: Thomas Petazzoni; Peter Seiderer; Yann E. MORIN; buildroot at buildroot.org Onderwerp: Re: [Buildroot] Cannot rebuild autobuild Hi, > > > So, this is a change in behaviour, and should be reported upstream as a > > > bug. Will you do so? TL;DR: No I don't, IMHO it's not a bug in make. 4fd5672 [1] which broke buildroot unified behaviour on different os, which is good. Since this revision it behaves nearly the same as 3.81. It's only coincidence that make from current openSUSE Tumbleweed with make-sorted-glob.patch patch [2] behaves the same (this patch is completely different from [1]). I did comparison of make of these git versions: 3.81, 3.82, 4.2.1, 0ca3108 (last working version on buildroot before fix b9d2d4cb4e), 4fd5672 (version which broke buildroot b9d2d4cb4e~), 48c8a11 (current master) with this simple "project": $ for i in a b c d; do touch $i.mk; sleep 1; done $ cat > Makefile << EOF $(info make: $(shell which make)) $(info make -v: $(shell make -v)) $(foreach i, $(sort $(wildcard *.mk)), $(info SORT $i)) $(foreach i, $(wildcard *.mk), $(info NOSORT $i)) include $(wildcard *.mk) EOF make version | 3.81 | 3.82 | 4.2.1 | 4.2.1+[2] | 0ca3108 | 4fd5672 ------------------------------------------------------------------------ wildcard the same | yes | no | no | no | no | no as sorted wildchard| | | | | ------------------------------------------------------------------------ wildcard the same | yes | no | no | yes | no | yes on different OS | | | | | | ------------------------------------------------------------------------ Breaks buildroot | no | no | no | yes | no | yes NOTES: wildcard changed in 3.82 and 4fd5672. Behaviour on 48c8a11 (current master) is the same as on 4fd5672. "4.2.1+[2]" is 4.2.1 + make-sorted-glob.patch patch [2]. [2] https://build.opensuse.org/package/view_file/Base:System/make/make-sorted-glob.patch?expand=1 [2] https://git.savannah.gnu.org/cgit/make.git/commit/?id=4fd56724ad281498d3c8b27a4b25b4070f6e4e65 Kind regards, Petr _______________________________________________ buildroot mailing list buildroot@busybox.net http://lists.busybox.net/mailman/listinfo/buildroot ^ permalink raw reply [flat|nested] 20+ messages in thread
* [Buildroot] Cannot rebuild autobuild 2017-11-15 23:03 ` Arnout Vandecappelle 2017-11-15 23:19 ` Peter Seiderer @ 2017-11-16 7:14 ` Petr Vorel 1 sibling, 0 replies; 20+ messages in thread From: Petr Vorel @ 2017-11-16 7:14 UTC (permalink / raw) To: buildroot > On 15-11-17 13:58, Petr Vorel wrote: > >> Cannot execute cross-compiler '.../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc' > >> make[1]: *** [package/pkg-generic.mk:209: .../build/toolchain-external/.stamp_configured] Error 1 > >> make: *** [Makefile:16: _all] Error 2 > >> Maybe the toolchain definition should select BR2_HOSTARCH_NEEDS_IA32_LIBS? > > It must be some system library dependency as I was able to run the build on Debian unstable. > > Failing build is on openSUSE Tumbleweed x86_64 (the same as you), but I have glibc-32bit > > and many other 32 bit packages installed (as well on Debian machine). > > I tried to play with BR2_HOSTARCH_NEEDS_IA32_LIBS, but didn't found the solution. > > Thomas, any idea what can be wrong? > What does .../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc -v > report? When you run ldd on it? When you run it under strace? What about the > other binaries bundled in that tarball? What about other external toolchains > (e.g. sourcery)? NOTE: This is actually for Peter Seiderer. Mine error was: make: *** No rule to make target 'toolchain-external-custom', needed by '/home/foo/install/src/buildroot.git/output/build/toolchain-external/.stamp_configured'. Stop. > Regards, > Arnout Kind regards, Petr ^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2017-12-15 20:23 UTC | newest] Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-11-13 10:27 [Buildroot] Cannot rebuild autobuild Petr Vorel 2017-11-13 11:06 ` Baruch Siach 2017-11-13 14:42 ` Petr Vorel 2017-11-14 22:44 ` Peter Seiderer 2017-11-15 12:58 ` Petr Vorel 2017-11-15 23:03 ` Arnout Vandecappelle 2017-11-15 23:19 ` Peter Seiderer 2017-11-15 23:40 ` Arnout Vandecappelle 2017-11-16 7:29 ` Petr Vorel 2017-11-16 13:43 ` Arnout Vandecappelle 2017-11-18 21:22 ` Peter Seiderer 2017-11-20 21:44 ` Yann E. MORIN 2017-11-18 21:32 ` Peter Seiderer 2017-11-21 20:16 ` Petr Vorel 2017-11-25 15:01 ` Yann E. MORIN 2017-11-25 16:34 ` Arnout Vandecappelle 2017-11-29 21:19 ` Petr Vorel 2017-12-15 20:22 ` Petr Vorel 2017-12-15 20:23 ` [Buildroot] FW: " Kees van Unen 2017-11-16 7:14 ` [Buildroot] " Petr Vorel
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.