[Buildroot] Cannot rebuild autobuild

[Buildroot] Cannot rebuild autobuild

[Petr Vorel]

Hi Peter,

I'm trying to fix failing feh on mips64el [1] (btw an easy fix, just to add -std=c99 or
-std=c11 to TARGET_CFLAGS either globally or just for this toolchain/architecture).

But I cannot rebuild it as it fails to pass configure:

$ make distclean
$ wget -c http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30/config -O .config.old
$ make olddefconfig
$ make
make: *** No rule to make target 'toolchain-external-custom', needed by '/home/foo/install/src/buildroot.git/output/build/toolchain-external/.stamp_configured'.  Stop.

What can be wrong? I've done it this way in the past (with other autobuild items, which
had different toolchains).


Kind regards,
Petr


[1] http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30

[Buildroot] Cannot rebuild autobuild

[Baruch Siach]

Hi Petr,

On Mon, Nov 13, 2017 at 11:27:58AM +0100, Petr Vorel wrote:
> I'm trying to fix failing feh on mips64el [1] (btw an easy fix, just to add 
> -std=c99 or
> -std=c11 to TARGET_CFLAGS either globally or just for this toolchain/architecture).
> 
> But I cannot rebuild it as it fails to pass configure:
> 
> $ make distclean
> $ wget -c http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30/config -O .config.old

If you do '-O .config' here, does that work better?

> $ make olddefconfig
> $ make
> make: *** No rule to make target 'toolchain-external-custom', needed by '/home/foo/install/src/buildroot.git/output/build/toolchain-external/.stamp_configured'.  Stop.
> 
> What can be wrong? I've done it this way in the past (with other autobuild items, which
> had different toolchains).

baruch

-- 
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -

[Buildroot] Cannot rebuild autobuild

[Petr Vorel]

> But I cannot rebuild it as it fails to pass configure:

> $ make distclean
> $ wget -c http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30/config -O .config.old
> $ make olddefconfig
> $ make
> make: *** No rule to make target 'toolchain-external-custom', needed by '/home/foo/install/src/buildroot.git/output/build/toolchain-external/.stamp_configured'.  Stop.

> What can be wrong? I've done it this way in the past (with other autobuild items, which
> had different toolchains).

> [1] http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30

And of course, I'm working on current master.


Kind regards,
Petr

[Buildroot] Cannot rebuild autobuild

[Peter Seiderer]

Hello Petr,

On Mon, 13 Nov 2017 15:42:32 +0100, Petr Vorel <petr.vorel@gmail.com> wrote:

> > But I cannot rebuild it as it fails to pass configure:  
> 
> > $ make distclean
> > $ wget -c http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30/config -O .config.old

Your should store the downloaded config file definitely as '.config' instead of '.config.old' (there is only
the way .config --> .config.old and not the other way round in buildroot)...
 
> > $ make olddefconfig
> > $ make
> > make: *** No rule to make target 'toolchain-external-custom', needed by '/home/foo/install/src/buildroot.git/output/build/toolchain-external/.stamp_configured'.  Stop.  
> 
> > What can be wrong? I've done it this way in the past (with other autobuild items, which
> > had different toolchains).  

Same here with a fresh installed openSUSE Tumbleweed x86_64 without 32-bit support packages installed.

Until commit ec54092c916440501006bfa51a820eb953100510 ('toolchain-external-custom: new package'), or with
2016.11.3 the same config gives the following error instead:

[...]
Cannot execute cross-compiler '.../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc'
make[1]: *** [package/pkg-generic.mk:209: .../build/toolchain-external/.stamp_configured] Error 1
make: *** [Makefile:16: _all] Error 2

Maybe the toolchain definition should select BR2_HOSTARCH_NEEDS_IA32_LIBS?

Regards,
Peter

> 
> > [1] http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30  
> 
> And of course, I'm working on current master.
> 
> 
> Kind regards,
> Petr
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] Cannot rebuild autobuild

[Petr Vorel]

Hi Peter,

...
> > > $ wget -c http://autobuild.buildroot.net/results/bf31199cdaa18971a9b599c26fbefac73b2d6f30/config -O .config.old

> Your should store the downloaded config file definitely as '.config' instead of '.config.old' (there is only
> the way .config --> .config.old and not the other way round in buildroot)...
Thanks for clarification. I actually realised this as well. This is not an issue, as I
tested it as being .config.

> > > $ make olddefconfig
> > > $ make
> > > make: *** No rule to make target 'toolchain-external-custom', needed by '/home/foo/install/src/buildroot.git/output/build/toolchain-external/.stamp_configured'.  Stop.  

> > > What can be wrong? I've done it this way in the past (with other autobuild items, which
> > > had different toolchains).  

> Same here with a fresh installed openSUSE Tumbleweed x86_64 without 32-bit support packages installed.

> Until commit ec54092c916440501006bfa51a820eb953100510 ('toolchain-external-custom: new package'), or with
> 2016.11.3 the same config gives the following error instead:

> [...]
> Cannot execute cross-compiler '.../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc'
> make[1]: *** [package/pkg-generic.mk:209: .../build/toolchain-external/.stamp_configured] Error 1
> make: *** [Makefile:16: _all] Error 2

> Maybe the toolchain definition should select BR2_HOSTARCH_NEEDS_IA32_LIBS?
It must be some system library dependency as I was able to run the build on Debian unstable.
Failing build is on openSUSE Tumbleweed x86_64 (the same as you), but I have glibc-32bit
and many other 32 bit packages installed (as well on Debian machine).
I tried to play with BR2_HOSTARCH_NEEDS_IA32_LIBS, but didn't found the solution.
Thomas, any idea what can be wrong?


Kind regards,
Petr

[Buildroot] Cannot rebuild autobuild

[Arnout Vandecappelle]

On 15-11-17 13:58, Petr Vorel wrote:
>> Cannot execute cross-compiler '.../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc'
>> make[1]: *** [package/pkg-generic.mk:209: .../build/toolchain-external/.stamp_configured] Error 1
>> make: *** [Makefile:16: _all] Error 2
>> Maybe the toolchain definition should select BR2_HOSTARCH_NEEDS_IA32_LIBS?
> It must be some system library dependency as I was able to run the build on Debian unstable.
> Failing build is on openSUSE Tumbleweed x86_64 (the same as you), but I have glibc-32bit
> and many other 32 bit packages installed (as well on Debian machine).
> I tried to play with BR2_HOSTARCH_NEEDS_IA32_LIBS, but didn't found the solution.
> Thomas, any idea what can be wrong?


 What does .../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc -v
report? When you run ldd on it? When you run it under strace? What about the
other binaries bundled in that tarball? What about other external toolchains
(e.g. sourcery)?

 Regards,
 Arnout
-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] Cannot rebuild autobuild

[Peter Seiderer]

Hello Arnout,

On Thu, 16 Nov 2017 00:03:45 +0100, Arnout Vandecappelle <arnout@mind.be> wrote:

> On 15-11-17 13:58, Petr Vorel wrote:
> >> Cannot execute cross-compiler '.../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc'
> >> make[1]: *** [package/pkg-generic.mk:209: .../build/toolchain-external/.stamp_configured] Error 1
> >> make: *** [Makefile:16: _all] Error 2
> >> Maybe the toolchain definition should select BR2_HOSTARCH_NEEDS_IA32_LIBS?  
> > It must be some system library dependency as I was able to run the build on Debian unstable.
> > Failing build is on openSUSE Tumbleweed x86_64 (the same as you), but I have glibc-32bit
> > and many other 32 bit packages installed (as well on Debian machine).
> > I tried to play with BR2_HOSTARCH_NEEDS_IA32_LIBS, but didn't found the solution.
> > Thomas, any idea what can be wrong?  
> 
> 
>  What does .../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc -v
> report? When you run ldd on it? When you run it under strace? What about the
> other binaries bundled in that tarball? What about other external toolchains
> (e.g. sourcery)?

The above error message is from old buildroot 2016.11.3 (the compiler tar file is downloaded and
upacked and fails at runtime)...

The original error is with git master (or all buildroot past the ec54092c916440501006bfa51a820eb953100510
('toolchain-external-custom: new package') commit:

/usr/bin/install -m 0644 support/misc/target-dir-warning.txt .../build-prebuild-toolchain-001/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM
^[[7m>>> skeleton-init-sysv  Extracting^[[27m
^[[7m>>> skeleton-init-sysv  Patching^[[27m
^[[7m>>> skeleton-init-sysv  Configuring^[[27m
^[[7m>>> skeleton-init-sysv  Building^[[27m
^[[7m>>> skeleton-init-sysv  Installing to target^[[27m
rsync -a --ignore-times --exclude .svn --exclude .git --exclude .hg --exclude .bzr --exclude CVS --chmod=u=rwX,go=rX --exclude .empty --exclude '*~' package/skeleton-init-sysv//skeleton/ .../build-prebuild-toolchain-001/target/
^[[7m>>> skeleton  Extracting^[[27m
^[[7m>>> skeleton  Patching^[[27m
^[[7m>>> skeleton  Configuring^[[27m
^[[7m>>> skeleton  Building^[[27m
^[[7m>>> skeleton  Installing to target^[[27m
make[1]: *** No rule to make target 'toolchain-external-custom', needed by '.../build-prebuild-toolchain-001/build/toolchain-external/.stamp_configured'.  Stop.
make: *** [Makefile:16: _all] Error 2

The error occurs  before the prebuild toolchain is downloaded/extracted...

Regards,
Peter

> 
>  Regards,
>  Arnout

[Buildroot] Cannot rebuild autobuild

[Arnout Vandecappelle]

On 16-11-17 00:19, Peter Seiderer wrote:
> Hello Arnout,
> 
> On Thu, 16 Nov 2017 00:03:45 +0100, Arnout Vandecappelle <arnout@mind.be> wrote:
> 
>> On 15-11-17 13:58, Petr Vorel wrote:
>>>> Cannot execute cross-compiler '.../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc'
>>>> make[1]: *** [package/pkg-generic.mk:209: .../build/toolchain-external/.stamp_configured] Error 1
>>>> make: *** [Makefile:16: _all] Error 2
>>>> Maybe the toolchain definition should select BR2_HOSTARCH_NEEDS_IA32_LIBS?  
>>> It must be some system library dependency as I was able to run the build on Debian unstable.
>>> Failing build is on openSUSE Tumbleweed x86_64 (the same as you), but I have glibc-32bit
>>> and many other 32 bit packages installed (as well on Debian machine).
>>> I tried to play with BR2_HOSTARCH_NEEDS_IA32_LIBS, but didn't found the solution.
>>> Thomas, any idea what can be wrong?  
>>
>>
>>  What does .../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc -v
>> report? When you run ldd on it? When you run it under strace? What about the
>> other binaries bundled in that tarball? What about other external toolchains
>> (e.g. sourcery)?
> 
> The above error message is from old buildroot 2016.11.3 (the compiler tar file is downloaded and
> upacked and fails at runtime)...
> 
> The original error is with git master (or all buildroot past the ec54092c916440501006bfa51a820eb953100510
> ('toolchain-external-custom: new package') commit:
> 
> /usr/bin/install -m 0644 support/misc/target-dir-warning.txt .../build-prebuild-toolchain-001/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM
> ^[[7m>>> skeleton-init-sysv  Extracting^[[27m
> ^[[7m>>> skeleton-init-sysv  Patching^[[27m
> ^[[7m>>> skeleton-init-sysv  Configuring^[[27m
> ^[[7m>>> skeleton-init-sysv  Building^[[27m
> ^[[7m>>> skeleton-init-sysv  Installing to target^[[27m
> rsync -a --ignore-times --exclude .svn --exclude .git --exclude .hg --exclude .bzr --exclude CVS --chmod=u=rwX,go=rX --exclude .empty --exclude '*~' package/skeleton-init-sysv//skeleton/ .../build-prebuild-toolchain-001/target/
> ^[[7m>>> skeleton  Extracting^[[27m
> ^[[7m>>> skeleton  Patching^[[27m
> ^[[7m>>> skeleton  Configuring^[[27m
> ^[[7m>>> skeleton  Building^[[27m
> ^[[7m>>> skeleton  Installing to target^[[27m
> make[1]: *** No rule to make target 'toolchain-external-custom', needed by '.../build-prebuild-toolchain-001/build/toolchain-external/.stamp_configured'.  Stop.
> make: *** [Makefile:16: _all] Error 2
> 
> The error occurs  before the prebuild toolchain is downloaded/extracted...

 That is really weird, the toolchain-external-custom dependencies should be
defined no matter what... Could you pastebin the output make 'make -qp' somewhere?

 I would think it would be somehow related to gcc not being executable, but
since this occurs before the package is even extracted... Just to be sure, what
is the contents of output/build when you get this error? There is no
toolchain-external-custom directory yet, right?

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] Cannot rebuild autobuild

[Petr Vorel]

Hi,

...
> > make[1]: *** No rule to make target 'toolchain-external-custom', needed by '.../build-prebuild-toolchain-001/build/toolchain-external/.stamp_configured'.  Stop.
> > make: *** [Makefile:16: _all] Error 2

> > The error occurs  before the prebuild toolchain is downloaded/extracted...

>  That is really weird, the toolchain-external-custom dependencies should be
> defined no matter what... Could you pastebin the output make 'make -qp' somewhere?
Here it is:
http://sipe.cz/buildroot/out.log (68M)
http://sipe.cz/buildroot/out.log.tar.gz (9MB)


Kind regards,
Petr

[Buildroot] Cannot rebuild autobuild

[Arnout Vandecappelle]

On 16-11-17 08:29, Petr Vorel wrote:
> Hi,
> 
> ...
>>> make[1]: *** No rule to make target 'toolchain-external-custom', needed by '.../build-prebuild-toolchain-001/build/toolchain-external/.stamp_configured'.  Stop.
>>> make: *** [Makefile:16: _all] Error 2
> 
>>> The error occurs  before the prebuild toolchain is downloaded/extracted...
> 
>>  That is really weird, the toolchain-external-custom dependencies should be
>> defined no matter what... Could you pastebin the output make 'make -qp' somewhere?
> Here it is:
> http://sipe.cz/buildroot/out.log (68M)
> http://sipe.cz/buildroot/out.log.tar.gz (9MB)

 Oh boy...

 The problem is that on Tumbleweed the order of inclusion seems to be different,
which causes toolchain-external-package not to be defined at the time that
toolchain-external-custom.mk is included (same for all other external toolchains).

Makefile:492:include toolchain/*/*.mk

This causes inclusion of
toolchain/toolchain-external/toolchain-external.mk:28:include
toolchain/toolchain-external/*/*.mk
and
toolchain/toolchain-external/pkg-toolchain-external.mk:592:toolchain-external-package
= ...

 If the *.mk is expanded in alphabetical order, pkg-toolchain-external.mk will
always come before toolchain-external.mk. Otherwise it is possible (depending on
the state of hash tables or inode numbers or whatever) that
toolchain-external-custom.mk gets included first.

 The short-term solution is of course $(sort $(wildcard ...))

 But for the long term, I think it's better to move pkg-*.mk to support/misc (or
maybe to support/make) and organise the order of inclusion from there.

 Any takers? Yann?

 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] Cannot rebuild autobuild

[Peter Seiderer]

Hello Arnout,

On Thu, 16 Nov 2017 14:43:48 +0100, Arnout Vandecappelle <arnout@mind.be> wrote:

> On 16-11-17 08:29, Petr Vorel wrote:
> > Hi,
> > 
> > ...  
> >>> make[1]: *** No rule to make target 'toolchain-external-custom', needed by '.../build-prebuild-toolchain-001/build/toolchain-external/.stamp_configured'.  Stop.
> >>> make: *** [Makefile:16: _all] Error 2  
> >   
> >>> The error occurs  before the prebuild toolchain is downloaded/extracted...  
> >   
> >>  That is really weird, the toolchain-external-custom dependencies should be
> >> defined no matter what... Could you pastebin the output make 'make -qp' somewhere?  
> > Here it is:
> > http://sipe.cz/buildroot/out.log (68M)
> > http://sipe.cz/buildroot/out.log.tar.gz (9MB)  
> 
>  Oh boy...
> 
>  The problem is that on Tumbleweed the order of inclusion seems to be different,
> which causes toolchain-external-package not to be defined at the time that
> toolchain-external-custom.mk is included (same for all other external toolchains).
> 
> Makefile:492:include toolchain/*/*.mk
> 
> This causes inclusion of
> toolchain/toolchain-external/toolchain-external.mk:28:include
> toolchain/toolchain-external/*/*.mk
> and
> toolchain/toolchain-external/pkg-toolchain-external.mk:592:toolchain-external-package
> = ...
> 
>  If the *.mk is expanded in alphabetical order, pkg-toolchain-external.mk will
> always come before toolchain-external.mk. Otherwise it is possible (depending on
> the state of hash tables or inode numbers or whatever) that
> toolchain-external-custom.mk gets included first.
> 
>  The short-term solution is of course $(sort $(wildcard ...))

Your investigation seems to be right, the following patch fixes it for me (no
32-bit support installed yet, but the cross compiler is extracted and I get
the 'Cannot execute cross-compiler' error message again):

diff --git a/Makefile b/Makefile
index 55409b99af..9d9737332d 100644
--- a/Makefile
+++ b/Makefile
@@ -483,13 +483,13 @@ include system/system.mk
 include package/Makefile.in
 # arch/arch.mk.* must be after package/Makefile.in because it may need to
 # complement variables defined therein, like BR_NO_CHECK_HASH_FOR.
--include $(wildcard arch/arch.mk.*)
+-include $(sort $(wildcard arch/arch.mk.*))
 include support/dependencies/dependencies.mk
 
 PACKAGES += $(DEPENDENCIES_HOST_PREREQ)
 
-include toolchain/*.mk
-include toolchain/*/*.mk
+include $(sort $(wildcard toolchain/*.mk))
+include $(sort $(wildcard toolchain/*/*.mk))
 
 # Include the package override file if one has been provided in the
 # configuration.
diff --git a/toolchain/toolchain-external/toolchain-external.mk b/toolchain/toolchain-external/toolchain-external.mk
index 48de1e71f2..dd07884c7e 100644
--- a/toolchain/toolchain-external/toolchain-external.mk
+++ b/toolchain/toolchain-external/toolchain-external.mk
@@ -25,4 +25,4 @@ $(error No prefix selected for external toolchain package $(BR2_PACKAGE_PROVIDES
 endif
 endif
 
-include toolchain/toolchain-external/*/*.mk
+include $(sort $(wildcard toolchain/toolchain-external/*/*.mk))

Regards,
Peter

> 
>  But for the long term, I think it's better to move pkg-*.mk to support/misc (or
> maybe to support/make) and organise the order of inclusion from there.
> 
>  Any takers? Yann?
> 
>  Regards,
>  Arnout
> 

[Buildroot] Cannot rebuild autobuild

[Yann E. MORIN]

Peter, Arnout, All,

On 2017-11-18 22:22 +0100, Peter Seiderer spake thusly:
> On Thu, 16 Nov 2017 14:43:48 +0100, Arnout Vandecappelle <arnout@mind.be> wrote:
[--SNIP--]
> >  Oh boy...
> > 
> >  The problem is that on Tumbleweed the order of inclusion seems to be different,
> > which causes toolchain-external-package not to be defined at the time that
> > toolchain-external-custom.mk is included (same for all other external toolchains).
> > 
> > Makefile:492:include toolchain/*/*.mk
> > 
> > This causes inclusion of
> > toolchain/toolchain-external/toolchain-external.mk:28:include
> > toolchain/toolchain-external/*/*.mk
> > and
> > toolchain/toolchain-external/pkg-toolchain-external.mk:592:toolchain-external-package
> > = ...
> > 
> >  If the *.mk is expanded in alphabetical order, pkg-toolchain-external.mk will
> > always come before toolchain-external.mk. Otherwise it is possible (depending on
> > the state of hash tables or inode numbers or whatever) that
> > toolchain-external-custom.mk gets included first.
> > 
> >  The short-term solution is of course $(sort $(wildcard ...))
> 
> Your investigation seems to be right, the following patch fixes it for me (no
> 32-bit support installed yet, but the cross compiler is extracted and I get
> the 'Cannot execute cross-compiler' error message again):
[--SNIP--]
> -include toolchain/*.mk
> -include toolchain/*/*.mk
> +include $(sort $(wildcard toolchain/*.mk))
> +include $(sort $(wildcard toolchain/*/*.mk))

I think that using $(wildcard) explicitly is Good (TM), rather than using
globs.

Also, $(sort) is Good Too (TM) because it guarantees the build order. So
far, $(sort) internally used strcmp() to compare strings [0] [1], so
works on bytes, not characters, so does not account for the locale
settings. However, that may change in the future, so we'd have to enforce
the C locale to guarantee ordering.

Can you submit this officially, please? Keep me in Cc when you do so,
thanks! ;-)

[0] https://git.savannah.gnu.org/cgit/make.git/tree/src/function.c#n1161
[1] https://git.savannah.gnu.org/cgit/make.git/tree/src/misc.c#n41

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] Cannot rebuild autobuild

[Peter Seiderer]

Hello Arnout,
On Thu, 16 Nov 2017 14:43:48 +0100, Arnout Vandecappelle <arnout@mind.be> wrote:

> On 16-11-17 08:29, Petr Vorel wrote:
> > Hi,
> > 
> > ...  
> >>> make[1]: *** No rule to make target 'toolchain-external-custom', needed by '.../build-prebuild-toolchain-001/build/toolchain-external/.stamp_configured'.  Stop.
> >>> make: *** [Makefile:16: _all] Error 2  
> >   
> >>> The error occurs  before the prebuild toolchain is downloaded/extracted...  
> >   
> >>  That is really weird, the toolchain-external-custom dependencies should be
> >> defined no matter what... Could you pastebin the output make 'make -qp' somewhere?  
> > Here it is:
> > http://sipe.cz/buildroot/out.log (68M)
> > http://sipe.cz/buildroot/out.log.tar.gz (9MB)  
> 
>  Oh boy...
> 
>  The problem is that on Tumbleweed the order of inclusion seems to be different,
> which causes toolchain-external-package not to be defined at the time that
> toolchain-external-custom.mk is included (same for all other external toolchains).
> 
> Makefile:492:include toolchain/*/*.mk
> 
> This causes inclusion of
> toolchain/toolchain-external/toolchain-external.mk:28:include
> toolchain/toolchain-external/*/*.mk
> and
> toolchain/toolchain-external/pkg-toolchain-external.mk:592:toolchain-external-package
> = ...
> 
>  If the *.mk is expanded in alphabetical order, pkg-toolchain-external.mk will
> always come before toolchain-external.mk. Otherwise it is possible (depending on
> the state of hash tables or inode numbers or whatever) that
> toolchain-external-custom.mk gets included first.

According to this [1] it is a change from the used gnu-make version (on Tumpbleweed
GNU Make 4.2.1 is used)?

Regards,
Peter

[1] https://stackoverflow.com/questions/40558385/gnu-make-wildcard-no-longer-gives-sorted-output-is-there-any-control-switch

> 
>  The short-term solution is of course $(sort $(wildcard ...))
> 
>  But for the long term, I think it's better to move pkg-*.mk to support/misc (or
> maybe to support/make) and organise the order of inclusion from there.
> 
>  Any takers? Yann?
> 
>  Regards,
>  Arnout
> 

[Buildroot] Cannot rebuild autobuild

[Petr Vorel]

Hi,

> >  If the *.mk is expanded in alphabetical order, pkg-toolchain-external.mk will
> > always come before toolchain-external.mk. Otherwise it is possible (depending on
> > the state of hash tables or inode numbers or whatever) that
> > toolchain-external-custom.mk gets included first.

> According to this [1] it is a change from the used gnu-make version (on Tumbleweed
> GNU Make 4.2.1 is used)?
I found by bisecting that our bug is caused by this change in make:
4fd5672 ("Use Jenkins hash.")
https://git.savannah.gnu.org/cgit/make.git/commit/?id=4fd56724ad281498d3c8b27a4b25b4070f6e4e65
It's a version after last release (4.2.1).
I wonder whether it's a bug in make (I guess so) and whether we should report it
(probably).

As 4.2.1 compiled from source works, I don't know what broke it for 4.2.1 from openSUSE. I
expected this patch, but that is working.
https://build.opensuse.org/package/view_file/Base:System/make/make-sorted-glob.patch?expand=1

I tested Peter's patch with broken make versions, fixes them.

Kind regards,
Petr

> Regards,
> Peter

> [1] https://stackoverflow.com/questions/40558385/gnu-make-wildcard-no-longer-gives-sorted-output-is-there-any-control-switch

[Buildroot] Cannot rebuild autobuild

[Yann E. MORIN]

Petr, All,

On 2017-11-21 21:16 +0100, Petr Vorel spake thusly:
> > >  If the *.mk is expanded in alphabetical order, pkg-toolchain-external.mk will
> > > always come before toolchain-external.mk. Otherwise it is possible (depending on
> > > the state of hash tables or inode numbers or whatever) that
> > > toolchain-external-custom.mk gets included first.
[--SNIP--]
> I found by bisecting that our bug is caused by this change in make:
> 4fd5672 ("Use Jenkins hash.")
> https://git.savannah.gnu.org/cgit/make.git/commit/?id=4fd56724ad281498d3c8b27a4b25b4070f6e4e65

Good work at bisecting it! :-)

> It's a version after last release (4.2.1).
> I wonder whether it's a bug in make (I guess so) and whether we should report it
> (probably).

This is indeed a change in behaviour. However, as make ever documented
a guranteed/non-guaranteed ordering of wildcards to begin with?

The manual for the current make version says nothing about the ordering,
and neither do the manual for older versions (I've checked only a few of
them).

So, this is a change in behaviour, and should be reported upstream as a
bug. Will you do so?

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] Cannot rebuild autobuild

[Arnout Vandecappelle]

On 25-11-17 16:01, Yann E. MORIN wrote:
> Petr, All,
> 
> On 2017-11-21 21:16 +0100, Petr Vorel spake thusly:
>>>>  If the *.mk is expanded in alphabetical order, pkg-toolchain-external.mk will
>>>> always come before toolchain-external.mk. Otherwise it is possible (depending on
>>>> the state of hash tables or inode numbers or whatever) that
>>>> toolchain-external-custom.mk gets included first.
> [--SNIP--]
>> I found by bisecting that our bug is caused by this change in make:
>> 4fd5672 ("Use Jenkins hash.")
>> https://git.savannah.gnu.org/cgit/make.git/commit/?id=4fd56724ad281498d3c8b27a4b25b4070f6e4e65
> 
> Good work at bisecting it! :-)
> 
>> It's a version after last release (4.2.1).
>> I wonder whether it's a bug in make (I guess so) and whether we should report it
>> (probably).
> 
> This is indeed a change in behaviour. However, as make ever documented
> a guranteed/non-guaranteed ordering of wildcards to begin with?
> 
> The manual for the current make version says nothing about the ordering,
> and neither do the manual for older versions (I've checked only a few of
> them).
> 
> So, this is a change in behaviour, and should be reported upstream as a
> bug. Will you do so?

 As I explained in the extended commit message when I applied the patch fixing
this (b9d2d4cb4eb: "Fix makefile include order by using sort/wildcard.") make
actually has had this behaviour "forever" (since make 3.82). It calls glob with
GLOB_NOSORT. Note that in most of the cases where we didn't already do $(sort
$(wildcard ...)), the order actually doesn't (or shouldn't) matter. For example,
package/*/*.mk hopefully does not depend on the order. In other words, it's very
likely that nobody notices if things get included in the "wrong" order.

 As far as I could see, only the external-toolchain-package really could cause
any problems.

 So my guess is that the bisected commit really has nothing to do with it, it
just happens to trigger the random ordering issue.

 Now that I look at it again, however, it looks like we haven't been able to
adequately explain why this issue suddenly occurred... First of all,
external-toolchain-package has existed for a year now, so if it is truly random
we'd expect someone to have noticed before. Also, the change in Tumbleweed that
Petr pointed to [1] should actually *fix* the issue, because it removes the
NOSORT. Debian and Ubuntu, conversely, have no such patch so the autobuilders
should see this issue (they use external toolchains a lot)...

 /me is confused again...


 Regards,
 Arnout


[1]
https://build.opensuse.org/package/view_file/Base:System/make/make-sorted-glob.patch?expand=1


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] Cannot rebuild autobuild

[Petr Vorel]

Hi,

> > So, this is a change in behaviour, and should be reported upstream as a
> > bug. Will you do so?

>  As I explained in the extended commit message when I applied the patch fixing
> this (b9d2d4cb4eb: "Fix makefile include order by using sort/wildcard.") make
> actually has had this behaviour "forever" (since make 3.82). It calls glob with
> GLOB_NOSORT. Note that in most of the cases where we didn't already do $(sort
> $(wildcard ...)), the order actually doesn't (or shouldn't) matter. For example,
> package/*/*.mk hopefully does not depend on the order. In other words, it's very
> likely that nobody notices if things get included in the "wrong" order.
Even though I understand that the change in 3.82 you pointed out *should* change the
behavior, it's different: 3.82 is working.

>  As far as I could see, only the external-toolchain-package really could cause
> any problems.

>  So my guess is that the bisected commit really has nothing to do with it, it
> just happens to trigger the random ordering issue.
I don't understand changes in 4fd5672 ("Use Jenkins hash.") [2], but it does it
permanently (I tested it several times). That's why I thought it would be good to notify
upstream. Now I'm not sure (I haven't reported it yet).

>  Now that I look at it again, however, it looks like we haven't been able to
> adequately explain why this issue suddenly occurred... First of all,
> external-toolchain-package has existed for a year now, so if it is truly random
> we'd expect someone to have noticed before. Also, the change in Tumbleweed that
> Petr pointed to [1] should actually *fix* the issue, because it removes the
> NOSORT. Debian and Ubuntu, conversely, have no such patch so the autobuilders
> should see this issue (they use external toolchains a lot)...
I've been testing buildroot in Debian unstable for few years and in openSUSE Tumbleweed
for about one year. I never noticed this, until now. You can see from [3], that the only
thing that changed in openSUSE's make is glob-lstat.patch [4]. It actually changes glob,
so it can be relevant. I can test it with and without it this week.
In Debian I've been testing it probably on 3.81, 4.0 and 4.1.

>  /me is confused again...


>  Regards,
>  Arnout


Kind regards,
Petr


[1] https://build.opensuse.org/package/view_file/Base:System/make/make-sorted-glob.patch?expand=1
[2] https://git.savannah.gnu.org/cgit/make.git/commit/?id=4fd56724ad281498d3c8b27a4b25b4070f6e4e65
[3] https://build.opensuse.org/package/view_file/Base:System/make/make.changes?expand=1
[4] https://build.opensuse.org/package/view_file/Base:System/make/glob-lstat.patch?expand=1

[Buildroot] Cannot rebuild autobuild

[Petr Vorel]

Hi,

> > > So, this is a change in behaviour, and should be reported upstream as a
> > > bug. Will you do so?

TL;DR: No I don't, IMHO it's not a bug in make.

4fd5672 [1] which broke buildroot unified behaviour on different os, which is good. Since
this revision it behaves nearly the same as 3.81.  It's only coincidence that make from
current openSUSE Tumbleweed with make-sorted-glob.patch patch [2] behaves the same (this
patch is completely different from [1]).


I did comparison of make of these git versions: 3.81, 3.82, 4.2.1, 0ca3108 (last working
version on buildroot before fix b9d2d4cb4e), 4fd5672 (version which broke buildroot
b9d2d4cb4e~), 48c8a11 (current master) with this simple "project":

$ for i in a b c d; do touch $i.mk; sleep 1; done

$ cat > Makefile << EOF
$(info make: $(shell which make))
$(info make -v: $(shell make -v))

$(foreach i, $(sort $(wildcard *.mk)), $(info SORT $i))
$(foreach i, $(wildcard *.mk), $(info NOSORT $i))
include $(wildcard *.mk)
EOF


make version       | 3.81 | 3.82 | 4.2.1 | 4.2.1+[2] | 0ca3108 | 4fd5672
------------------------------------------------------------------------
wildcard the same  | yes  |  no  |  no   |    no     |   no    |   no
as sorted wildchard|      |      |       |           |
------------------------------------------------------------------------
wildcard the same  | yes  |  no  |  no   |    yes    |   no    |   yes
on different OS    |      |      |       |           |         |
------------------------------------------------------------------------
Breaks buildroot   |  no  |  no  |  no   |    yes    |   no    |   yes

NOTES:
wildcard changed in 3.82 and 4fd5672.
Behaviour on 48c8a11 (current master) is the same as on 4fd5672.
"4.2.1+[2]" is 4.2.1 + make-sorted-glob.patch patch [2].

[2] https://build.opensuse.org/package/view_file/Base:System/make/make-sorted-glob.patch?expand=1
[2] https://git.savannah.gnu.org/cgit/make.git/commit/?id=4fd56724ad281498d3c8b27a4b25b4070f6e4e65


Kind regards,
Petr

[Buildroot] FW: Cannot rebuild autobuild

[Kees van Unen]

________________________________________
Van: buildrootNamensPetr Vorel
Verzonden: vrijdag 15 december 2017 21:22:46 (UTC+01:00) Amsterdam, Berlijn, Bern, Rome, Stockholm, Wenen
Aan: Arnout Vandecappelle
CC: Thomas Petazzoni; Peter Seiderer; Yann E. MORIN; buildroot at buildroot.org
Onderwerp: Re: [Buildroot] Cannot rebuild autobuild

Hi,

> > > So, this is a change in behaviour, and should be reported upstream as a
> > > bug. Will you do so?

TL;DR: No I don't, IMHO it's not a bug in make.

4fd5672 [1] which broke buildroot unified behaviour on different os, which is good. Since
this revision it behaves nearly the same as 3.81.  It's only coincidence that make from
current openSUSE Tumbleweed with make-sorted-glob.patch patch [2] behaves the same (this
patch is completely different from [1]).


I did comparison of make of these git versions: 3.81, 3.82, 4.2.1, 0ca3108 (last working
version on buildroot before fix b9d2d4cb4e), 4fd5672 (version which broke buildroot
b9d2d4cb4e~), 48c8a11 (current master) with this simple "project":

$ for i in a b c d; do touch $i.mk; sleep 1; done

$ cat > Makefile << EOF
$(info make: $(shell which make))
$(info make -v: $(shell make -v))

$(foreach i, $(sort $(wildcard *.mk)), $(info SORT $i))
$(foreach i, $(wildcard *.mk), $(info NOSORT $i))
include $(wildcard *.mk)
EOF


make version       | 3.81 | 3.82 | 4.2.1 | 4.2.1+[2] | 0ca3108 | 4fd5672
------------------------------------------------------------------------
wildcard the same  | yes  |  no  |  no   |    no     |   no    |   no
as sorted wildchard|      |      |       |           |
------------------------------------------------------------------------
wildcard the same  | yes  |  no  |  no   |    yes    |   no    |   yes
on different OS    |      |      |       |           |         |
------------------------------------------------------------------------
Breaks buildroot   |  no  |  no  |  no   |    yes    |   no    |   yes

NOTES:
wildcard changed in 3.82 and 4fd5672.
Behaviour on 48c8a11 (current master) is the same as on 4fd5672.
"4.2.1+[2]" is 4.2.1 + make-sorted-glob.patch patch [2].

[2] https://build.opensuse.org/package/view_file/Base:System/make/make-sorted-glob.patch?expand=1
[2] https://git.savannah.gnu.org/cgit/make.git/commit/?id=4fd56724ad281498d3c8b27a4b25b4070f6e4e65


Kind regards,
Petr
_______________________________________________
buildroot mailing list
buildroot@busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] Cannot rebuild autobuild

[Petr Vorel]

> On 15-11-17 13:58, Petr Vorel wrote:
> >> Cannot execute cross-compiler '.../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc'
> >> make[1]: *** [package/pkg-generic.mk:209: .../build/toolchain-external/.stamp_configured] Error 1
> >> make: *** [Makefile:16: _all] Error 2
> >> Maybe the toolchain definition should select BR2_HOSTARCH_NEEDS_IA32_LIBS?
> > It must be some system library dependency as I was able to run the build on Debian unstable.
> > Failing build is on openSUSE Tumbleweed x86_64 (the same as you), but I have glibc-32bit
> > and many other 32 bit packages installed (as well on Debian machine).
> > I tried to play with BR2_HOSTARCH_NEEDS_IA32_LIBS, but didn't found the solution.
> > Thomas, any idea what can be wrong?


>  What does .../host/opt/ext-toolchain/bin/x86_64-ctng_locales-linux-gnu-gcc -v
> report? When you run ldd on it? When you run it under strace? What about the
> other binaries bundled in that tarball? What about other external toolchains
> (e.g. sourcery)?
NOTE: This is actually for Peter Seiderer. Mine error was:
make: *** No rule to make target 'toolchain-external-custom', needed by '/home/foo/install/src/buildroot.git/output/build/toolchain-external/.stamp_configured'.  Stop.

>  Regards,
>  Arnout


Kind regards,
Petr