[Buildroot] [for-next: 1/2] package/make: add host variant

[Buildroot] [for-next: 1/2] package/make: add host variant

[Romain Naour]

Add host variant for make package since glibc 2.28 now require
GNU Make >= 4.0 [1].

We have to ceate gnumake and gmake symlinks to make generated by
Buildroot for the host. Otherwise the one provided by the host
system can be used instead.

The glibc build system is looking for gnumake and gmake before
make.

[1] https://www.sourceware.org/ml/libc-alpha/2018-08/msg00003.html

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 package/make/make.mk | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/package/make/make.mk b/package/make/make.mk
index 01cd2eaa18..790391c76e 100644
--- a/package/make/make.mk
+++ b/package/make/make.mk
@@ -20,4 +20,18 @@ ifeq ($(BR2_STATIC_LIBS),y)
 MAKE_CONF_OPTS += --disable-load
 endif
 
+HOST_MAKE_DEPENDENCIES = host-pkgconf
+HOST_MAKE_CONF_OPTS = --without-guile
+
+# Some packages, like glibc, are looking for gnumake or gmake before
+# make program.
+# Create these symlink to use make binary generated by Buildroot for
+# the host.
+define HOST_MAKE_INSTALL_GNUMAKE
+	ln -fs make $(HOST_DIR)/bin/gnumake
+	ln -fs make $(HOST_DIR)/bin/gmake
+endef
+HOST_MAKE_POST_INSTALL_HOOKS += HOST_MAKE_INSTALL_GNUMAKE
+
 $(eval $(autotools-package))
+$(eval $(host-autotools-package))
-- 
2.14.4

[Buildroot] [for-next: 2/2] dependencies: host-make version check

[Romain Naour]

The host make program is already checked by dependencies.sh but we
want to check the version number even if Buildroot is able to use
GNU make >= 3.81 but some packages may require a more recent version.

For example, since version 2.28 [1], glibc requires GNU Make >= 4.0.

[1] https://www.sourceware.org/ml/libc-alpha/2018-08/msg00003.html

Fixes:
http://autobuild.buildroot.net/results/576/5760ea2635d9aecc9c789494a8b2cc73a1af1ceb
http://autobuild.buildroot.net/results/583/58347b94884eee2db28740486eda280e8c08e22f
http://autobuild.buildroot.net/results/dc7/dc7c8cd548409864ab0055e196c0280457a5fb5f

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Baruch Siach <baruch@tkos.co.il>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 package/glibc/glibc.mk                  |  3 ++-
 support/dependencies/check-host-make.mk | 17 +++++++++++++
 support/dependencies/check-host-make.sh | 42 +++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+), 1 deletion(-)
 create mode 100644 support/dependencies/check-host-make.mk
 create mode 100755 support/dependencies/check-host-make.sh

diff --git a/package/glibc/glibc.mk b/package/glibc/glibc.mk
index 399cf395ce..fd9e394f92 100644
--- a/package/glibc/glibc.mk
+++ b/package/glibc/glibc.mk
@@ -28,7 +28,8 @@ GLIBC_ADD_TOOLCHAIN_DEPENDENCY = NO
 
 # Before glibc is configured, we must have the first stage
 # cross-compiler and the kernel headers
-GLIBC_DEPENDENCIES = host-gcc-initial linux-headers host-bison host-gawk
+GLIBC_DEPENDENCIES = host-gcc-initial linux-headers host-bison host-gawk \
+	$(BR2_MAKE_HOST_DEPENDENCY)
 
 GLIBC_SUBDIR = build
 
diff --git a/support/dependencies/check-host-make.mk b/support/dependencies/check-host-make.mk
new file mode 100644
index 0000000000..fa45fac8e0
--- /dev/null
+++ b/support/dependencies/check-host-make.mk
@@ -0,0 +1,17 @@
+# Since version 2.28, glibc requires GNU Make >= 4.0
+# https://www.sourceware.org/ml/libc-alpha/2018-08/msg00003.html
+#
+# Set this to either 4.0 or higher, depending on the highest minimum
+# version required by any of the packages bundled in Buildroot. If a
+# package is bumped or a new one added, and it requires a higher
+# version, our package infra will catch it and whine.
+#
+BR2_MAKE_VERSION_MIN = 4.0
+
+BR2_MAKE ?= $(call suitable-host-package,make,\
+	$(BR2_MAKE_VERSION_MIN) $(MAKE))
+
+ifeq ($(BR2_MAKE),)
+BR2_MAKE = $(HOST_DIR)/bin/make
+BR2_MAKE_HOST_DEPENDENCY = host-make
+endif
diff --git a/support/dependencies/check-host-make.sh b/support/dependencies/check-host-make.sh
new file mode 100755
index 0000000000..9c31f4a415
--- /dev/null
+++ b/support/dependencies/check-host-make.sh
@@ -0,0 +1,42 @@
+#!/bin/sh
+
+# prevent shift error
+[ $# -lt 2 ] && exit 1
+
+major_min="${1%.*}"
+minor_min="${1#*.}"
+
+shift
+
+# The host make program is already checked by dependencies.sh but we
+# want to check the version number even if Buildroot is able to use
+# GNU make >= 3.81 but some packages may require a more recent version.
+make="$1"
+
+# Output of 'make --version' examples:
+# GNU Make 4.2.1
+# GNU Make 4.0
+# GNU Make 3.81
+version=`$make --version 2>&1 | sed -e 's/^.* \([0-9\.]\)/\1/g' -e 's/[-\
+].*//g' -e '1q'`
+
+major=`echo "$version" | cut -d. -f1`
+minor=`echo "$version" | cut -d. -f2`
+bugfix=`echo "$version" | cut -d. -f3`
+
+if [ -z "${bugfix}" ] ; then
+	bugfix=0
+fi
+
+if [ $major -lt $major_min ]; then
+	# echo nothing: no suitable make found
+	exit 1
+fi
+
+if [ $major -eq $major_min -a $minor -lt $minor_min ]; then
+	# echo nothing: no suitable make found
+	exit 1
+fi
+
+# valid
+echo $make
-- 
2.14.4

[Buildroot] [for-next: 1/2] package/make: add host variant

[Thomas Petazzoni]

Hello Romain,

Thanks for your work on this topic!

On Thu, 30 Aug 2018 22:30:53 +0200, Romain Naour wrote:

> We have to ceate gnumake and gmake symlinks to make generated by
> Buildroot for the host. Otherwise the one provided by the host
> system can be used instead.
> 
> The glibc build system is looking for gnumake and gmake before
> make.

But the PATH environment variable that we use has $(HOST_DIR)/bin
before the system /usr/bin, and therefore our make in
$(HOST_DIR)/bin/make should be found/used by the glibc build system
instead of the system /usr/bin/make.

Why isn't this working ?

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [for-next: 1/2] package/make: add host variant

[Romain Naour]

Hi Thomas,

Le 01/09/2018 ? 14:51, Thomas Petazzoni a ?crit?:
> Hello Romain,
> 
> Thanks for your work on this topic!
> 
> On Thu, 30 Aug 2018 22:30:53 +0200, Romain Naour wrote:
> 
>> We have to ceate gnumake and gmake symlinks to make generated by
>> Buildroot for the host. Otherwise the one provided by the host
>> system can be used instead.
>>
>> The glibc build system is looking for gnumake and gmake before
>> make.
> 
> But the PATH environment variable that we use has $(HOST_DIR)/bin
> before the system /usr/bin, and therefore our make in
> $(HOST_DIR)/bin/make should be found/used by the glibc build system
> instead of the system /usr/bin/make.
> 
> Why isn't this working ?

It is but glibc is looking at gnumake and gmake at first:

From [1]:
AC_CHECK_PROG_VER(MAKE, gnumake gmake make, --version,
  [GNU Make[^0-9]*\([0-9][0-9.]*\)],
  [3.79* | 3.[89]* | [4-9].* | [1-9][0-9]*], critic_missing="$critic_missing make")

On your host (Fedora) you have a symlink to make named gmake (gmake -> make)

You can verify that glibc is using gmake instead of make by looking at config.log.

configure:4672: checking for gnumake
configure:4702: result: no
configure:4672: checking for gmake
configure:4688: found /usr/bin/gmake
configure:4699: result: gmake
configure:4714: checking version of gmake
configure:4724: result: 4.2.1, ok

We can verify that glibc build-system is really using gmake:
ac_cv_prog_MAKE=gmake

So if we want glibc build system to use the make binary provided by Buildroot we
must create gnumake and gmake symlink in $(HOST_DIR)/bin when installing host-make.

Best regards,
Romain

[1]
https://sourceware.org/git/?p=glibc.git;a=blob;f=configure.ac;h=014e09a5d58d102d208ddfaa192d844b1792d94f;hb=refs/heads/release/2.28/master#l952

> 
> Thanks,
> 
> Thomas
> 

[Buildroot] [for-next: 1/2] package/make: add host variant

[Thomas Petazzoni]

Hello,

On Sat, 1 Sep 2018 21:20:36 +0200, Romain Naour wrote:

> From [1]:
> AC_CHECK_PROG_VER(MAKE, gnumake gmake make, --version,
>   [GNU Make[^0-9]*\([0-9][0-9.]*\)],
>   [3.79* | 3.[89]* | [4-9].* | [1-9][0-9]*], critic_missing="$critic_missing make")
> 
> On your host (Fedora) you have a symlink to make named gmake (gmake -> make)
> 
> You can verify that glibc is using gmake instead of make by looking at config.log.
> 
> configure:4672: checking for gnumake
> configure:4702: result: no
> configure:4672: checking for gmake
> configure:4688: found /usr/bin/gmake
> configure:4699: result: gmake
> configure:4714: checking version of gmake
> configure:4724: result: 4.2.1, ok

OK.

> We can verify that glibc build-system is really using gmake:
> ac_cv_prog_MAKE=gmake
> 
> So if we want glibc build system to use the make binary provided by Buildroot we
> must create gnumake and gmake symlink in $(HOST_DIR)/bin when installing host-make.

Or we can pass ac_cv_prog_MAKE=make. But OK, perhaps doing the symlinks
is better.

Thanks for the explanation!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [for-next: 1/2] package/make: add host variant

[Romain Naour]

Hi Thomas,

Le 02/09/2018 ? 09:39, Thomas Petazzoni a ?crit?:
> Hello,
> 
> On Sat, 1 Sep 2018 21:20:36 +0200, Romain Naour wrote:
> 
>> From [1]:
>> AC_CHECK_PROG_VER(MAKE, gnumake gmake make, --version,
>>   [GNU Make[^0-9]*\([0-9][0-9.]*\)],
>>   [3.79* | 3.[89]* | [4-9].* | [1-9][0-9]*], critic_missing="$critic_missing make")
>>
>> On your host (Fedora) you have a symlink to make named gmake (gmake -> make)
>>
>> You can verify that glibc is using gmake instead of make by looking at config.log.
>>
>> configure:4672: checking for gnumake
>> configure:4702: result: no
>> configure:4672: checking for gmake
>> configure:4688: found /usr/bin/gmake
>> configure:4699: result: gmake
>> configure:4714: checking version of gmake
>> configure:4724: result: 4.2.1, ok
> 
> OK.
> 
>> We can verify that glibc build-system is really using gmake:
>> ac_cv_prog_MAKE=gmake
>>
>> So if we want glibc build system to use the make binary provided by Buildroot we
>> must create gnumake and gmake symlink in $(HOST_DIR)/bin when installing host-make.
> 
> Or we can pass ac_cv_prog_MAKE=make. But OK, perhaps doing the symlinks
> is better.

Indeed (I tried with this solution) but I believe that creating gnumake and
gmake symlinks in HOST_DIR is more bulletproof. I don't know if another package
is trying to use gnumake or gmake.

> 
> Thanks for the explanation!

You're welcome!

Best regards,
Romain

> 
> Thomas
> 

[Buildroot] [for-next: 2/2] dependencies: host-make version check

[Thomas Petazzoni]

Hello,

On Thu, 30 Aug 2018 22:30:54 +0200, Romain Naour wrote:

> diff --git a/package/glibc/glibc.mk b/package/glibc/glibc.mk
> index 399cf395ce..fd9e394f92 100644
> --- a/package/glibc/glibc.mk
> +++ b/package/glibc/glibc.mk
> @@ -28,7 +28,8 @@ GLIBC_ADD_TOOLCHAIN_DEPENDENCY = NO
>  
>  # Before glibc is configured, we must have the first stage
>  # cross-compiler and the kernel headers
> -GLIBC_DEPENDENCIES = host-gcc-initial linux-headers host-bison host-gawk
> +GLIBC_DEPENDENCIES = host-gcc-initial linux-headers host-bison host-gawk \
> +	$(BR2_MAKE_HOST_DEPENDENCY)

As discussed on IRC, this is not sufficient, because you need to set
GLIBC_MAKE to the correct value, otherwise package/pkg-autotools.mk
uses $(MAKE), which points to /usr/bin/make.

Due to this, the make built by Buildroot is in fact not used.

Also, perhaps you could split this patch into to: one patch adding the
support/dependencies/ logic, and another tweaking the glibc package.

Could you look into this?

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com