[Buildroot] [PATCH 0/3] golang: fix build with oldisj codesourcery amd64 toolchain

[Buildroot] [PATCH 0/3] golang: fix build with oldisj codesourcery amd64 toolchain

[Yann E. MORIN]

Hello All!

That toolchain seems to suffer from bug #20006
    https://sourceware.org/bugzilla/show_bug.cgi?id=20006

So, like bug 19615, introduce a config option for 20006, and mask away
all golang packages.

Additionally, make golang packages abide by the number of CPUs to use.


Regards,
Yann E. MORIN.


The following changes since commit a554109af857b7b53598c5a9e0ee1d2729da73df

  package/usb_modeswitch: disable parallel build (2018-11-24 13:03:23 +0100)


are available in the git repository at:

  git://git.buildroot.org/~ymorin/git/buildroot.git

for you to fetch changes up to 3eaf225466a426cf89218531a5c9d685c2224a18

  package: hide golang packages for toolchains with binutils bug 20006 (2018-11-24 22:50:48 +0100)


----------------------------------------------------------------
Yann E. MORIN (3):
      infra/pkg-golang: enforce number of parallel jobs
      toolchain: CodeSourcery AMD64 affected by PR20006
      package: hide golang packages for toolchains with binutils bug 20006

 package/docker-containerd/Config.in                                 | 4 ++++
 package/docker-engine/Config.in                                     | 4 ++++
 package/docker-proxy/Config.in                                      | 4 ++++
 package/flannel/Config.in                                           | 4 ++++
 package/mender/Config.in                                            | 4 ++++
 package/pkg-golang.mk                                               | 6 ++++--
 package/runc/Config.in                                              | 4 ++++
 toolchain/Config.in                                                 | 5 +++++
 .../toolchain-external-codesourcery-amd64/Config.in                 | 1 +
 9 files changed, 34 insertions(+), 2 deletions(-)

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/3] infra/pkg-golang: enforce number of parallel jobs

[Yann E. MORIN]

By default, the go compiler will spawn as many jobs as there are CPUs
available, thus possibily over-shooting the limits set by the user.

Make it abide by the user's wish, and specify the number of jobs allowed
to run.

We can do so without fear of a package failing to build in parallel,
because they were already all building in parallel, as that is the
default for the go compiler.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
---
 package/pkg-golang.mk | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/package/pkg-golang.mk b/package/pkg-golang.mk
index 6eacd14180..4f2c7e77e1 100644
--- a/package/pkg-golang.mk
+++ b/package/pkg-golang.mk
@@ -55,8 +55,10 @@ ifeq ($(BR2_STATIC_LIBS),y)
 $(2)_LDFLAGS += -extldflags '-static'
 endif
 
-$(2)_BUILD_OPTS += -ldflags "$$($(2)_LDFLAGS)"
-$(2)_BUILD_OPTS += -tags "$$($(2)_TAGS)"
+$(2)_BUILD_OPTS += \
+	-ldflags "$$($(2)_LDFLAGS)" \
+	-tags "$$($(2)_TAGS)" \
+	-p $(PARALLEL_JOBS)
 
 # Target packages need the Go compiler on the host.
 $(2)_DEPENDENCIES += host-go
-- 
2.14.1

[Buildroot] [PATCH 2/3] toolchain: CodeSourcery AMD64 affected by PR20006

[Yann E. MORIN]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 toolchain/Config.in                                                  | 5 +++++
 .../toolchain-external-codesourcery-amd64/Config.in                  | 1 +
 2 files changed, 6 insertions(+)

diff --git a/toolchain/Config.in b/toolchain/Config.in
index c2192a52b1..196612b8b0 100644
--- a/toolchain/Config.in
+++ b/toolchain/Config.in
@@ -69,6 +69,11 @@ comment "Toolchain Generic Options"
 config BR2_TOOLCHAIN_HAS_BINUTILS_BUG_19615
 	bool
 
+# https://sourceware.org/bugzilla/show_bug.cgi?id=20006
+# Affect toolchains built with binutils 2.26 (fixed in binutils 2.26.1).
+config BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
+	bool
+
 # Atomic types can be:
 #  - never lock-free
 #  - sometimes lock-free
diff --git a/toolchain/toolchain-external/toolchain-external-codesourcery-amd64/Config.in b/toolchain/toolchain-external/toolchain-external-codesourcery-amd64/Config.in
index 50c5a20019..9187333b95 100644
--- a/toolchain/toolchain-external/toolchain-external-codesourcery-amd64/Config.in
+++ b/toolchain/toolchain-external/toolchain-external-codesourcery-amd64/Config.in
@@ -13,6 +13,7 @@ config BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_AMD64
 	select BR2_TOOLCHAIN_HEADERS_AT_LEAST_4_7
 	select BR2_TOOLCHAIN_GCC_AT_LEAST_6
 	select BR2_TOOLCHAIN_HAS_BINUTILS_BUG_19615 # based-on binutils-2.26
+	select BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006 # based-on binutils-2.26
 	help
 	  Sourcery CodeBench toolchain for the amd64 (x86_64)
 	  architectures, from Mentor Graphics. It uses gcc 6.2,
-- 
2.14.1

[Buildroot] [PATCH 3/3] package: hide golang packages for toolchains with binutils bug 20006

[Yann E. MORIN]

Fixes:
    http://autobuild.buildroot.org/results/020/02039969b16534d4020ecd4574bae71b91c1e6b8/ (flannel)
    http://autobuild.buildroot.org/results/e95/e9528b06b350ef84c1e2cb59fba87b4db77b4660/ (docker-engine)
    [...]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Romain Naour <romain.naour@gmail.com>
---
 package/docker-containerd/Config.in | 4 ++++
 package/docker-engine/Config.in     | 4 ++++
 package/docker-proxy/Config.in      | 4 ++++
 package/flannel/Config.in           | 4 ++++
 package/mender/Config.in            | 4 ++++
 package/runc/Config.in              | 4 ++++
 6 files changed, 24 insertions(+)

diff --git a/package/docker-containerd/Config.in b/package/docker-containerd/Config.in
index 851345f73e..39f6ea8ee8 100644
--- a/package/docker-containerd/Config.in
+++ b/package/docker-containerd/Config.in
@@ -4,6 +4,7 @@ config BR2_PACKAGE_DOCKER_CONTAINERD
 	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
 	depends on BR2_TOOLCHAIN_HAS_THREADS
 	depends on BR2_USE_MMU # util-linux
+	depends on !BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
 	select BR2_PACKAGE_RUNC # runtime dependency
 	select BR2_PACKAGE_UTIL_LINUX # runtime dependency
 	select BR2_PACKAGE_UTIL_LINUX_BINARIES
@@ -32,3 +33,6 @@ comment "docker-containerd needs a toolchain w/ threads"
 	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
 	depends on BR2_USE_MMU
 	depends on !BR2_TOOLCHAIN_HAS_THREADS
+
+comment "docker-containerd needs a toolchain not affected by binutils bug 20006"
+	depends on BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
diff --git a/package/docker-engine/Config.in b/package/docker-engine/Config.in
index 8feb11b48c..4393c93b90 100644
--- a/package/docker-engine/Config.in
+++ b/package/docker-engine/Config.in
@@ -3,6 +3,7 @@ config BR2_PACKAGE_DOCKER_ENGINE
 	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
 	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
 	depends on BR2_TOOLCHAIN_HAS_THREADS
+	depends on !BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
 	help
 	  Docker is a platform to build, ship,
 	  and run applications as lightweight containers.
@@ -70,3 +71,6 @@ comment "docker-engine needs a toolchain w/ threads"
 	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
 	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
 	depends on !BR2_TOOLCHAIN_HAS_THREADS
+
+comment "docker-engine needs a toolchain not affected by binutils bug 20006"
+	depends on BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
diff --git a/package/docker-proxy/Config.in b/package/docker-proxy/Config.in
index 596e18a3f3..6e244f6abf 100644
--- a/package/docker-proxy/Config.in
+++ b/package/docker-proxy/Config.in
@@ -3,6 +3,7 @@ config BR2_PACKAGE_DOCKER_PROXY
 	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
 	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
 	depends on BR2_TOOLCHAIN_HAS_THREADS
+	depends on !BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
 	help
 	  Libnetwork is a Container Network Model that provides a
 	  consistent programming interface and the required network
@@ -17,3 +18,6 @@ comment "docker-proxy needs a toolchain w/ threads"
 	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
 	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
 	depends on !BR2_TOOLCHAIN_HAS_THREADS
+
+comment "docker-proxy needs a toolchain not affected by binutils bug 20006"
+	depends on BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
diff --git a/package/flannel/Config.in b/package/flannel/Config.in
index 134111b1c0..4f96a6daa0 100644
--- a/package/flannel/Config.in
+++ b/package/flannel/Config.in
@@ -3,11 +3,15 @@ comment "flannel needs a toolchain w/ threads"
 		BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
 	depends on !BR2_TOOLCHAIN_HAS_THREADS
 
+comment "flannel needs a toolchain not affected by binutils bug 20006"
+	depends on BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
+
 config BR2_PACKAGE_FLANNEL
 	bool "flannel"
 	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
 	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
 	depends on BR2_TOOLCHAIN_HAS_THREADS
+	depends on !BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
 	help
 	  Flannel is a virtual network that gives a subnet to each
 	  host for use with container runtimes.
diff --git a/package/mender/Config.in b/package/mender/Config.in
index aeb0b2694f..086c9773c9 100644
--- a/package/mender/Config.in
+++ b/package/mender/Config.in
@@ -3,6 +3,7 @@ config BR2_PACKAGE_MENDER
 	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
 	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
 	depends on BR2_TOOLCHAIN_HAS_THREADS
+	depends on !BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
 	select BR2_PACKAGE_UBOOT_TOOLS # runtime
 	select BR2_PACKAGE_UBOOT_TOOLS_FWPRINTENV # runtime
 	help
@@ -17,3 +18,6 @@ comment "mender needs a toolchain w/ threads"
 	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
 	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
 	depends on !BR2_TOOLCHAIN_HAS_THREADS
+
+comment "mender needs a toolchain not affected by binutils bug 20006"
+	depends on BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
diff --git a/package/runc/Config.in b/package/runc/Config.in
index fd5dee7c5b..e867e2a230 100644
--- a/package/runc/Config.in
+++ b/package/runc/Config.in
@@ -3,6 +3,7 @@ config BR2_PACKAGE_RUNC
 	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
 	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
 	depends on BR2_TOOLCHAIN_HAS_THREADS
+	depends on !BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
 	help
 	  runC is a CLI tool for spawning and running containers
 	  according to the OCP specification.
@@ -13,3 +14,6 @@ comment "runc needs a toolchain w/ threads"
 	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS && \
 		BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
 	depends on !BR2_TOOLCHAIN_HAS_THREADS
+
+comment "runc needs a toolchain not affected by binutils bug 20006"
+	depends on BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
-- 
2.14.1

[Buildroot] [PATCH 3/3] package: hide golang packages for toolchains with binutils bug 20006

[Thomas Petazzoni]

Hello,

On Sat, 24 Nov 2018 22:52:01 +0100, Yann E. MORIN wrote:
> Fixes:
>     http://autobuild.buildroot.org/results/020/02039969b16534d4020ecd4574bae71b91c1e6b8/ (flannel)
>     http://autobuild.buildroot.org/results/e95/e9528b06b350ef84c1e2cb59fba87b4db77b4660/ (docker-engine)
>     [...]
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Cc: Romain Naour <romain.naour@gmail.com>
> ---
>  package/docker-containerd/Config.in | 4 ++++
>  package/docker-engine/Config.in     | 4 ++++
>  package/docker-proxy/Config.in      | 4 ++++
>  package/flannel/Config.in           | 4 ++++
>  package/mender/Config.in            | 4 ++++
>  package/runc/Config.in              | 4 ++++
>  6 files changed, 24 insertions(+)
> 
> diff --git a/package/docker-containerd/Config.in b/package/docker-containerd/Config.in
> index 851345f73e..39f6ea8ee8 100644
> --- a/package/docker-containerd/Config.in
> +++ b/package/docker-containerd/Config.in
> @@ -4,6 +4,7 @@ config BR2_PACKAGE_DOCKER_CONTAINERD
>  	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
>  	depends on BR2_TOOLCHAIN_HAS_THREADS
>  	depends on BR2_USE_MMU # util-linux
> +	depends on !BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
>  	select BR2_PACKAGE_RUNC # runtime dependency
>  	select BR2_PACKAGE_UTIL_LINUX # runtime dependency
>  	select BR2_PACKAGE_UTIL_LINUX_BINARIES
> @@ -32,3 +33,6 @@ comment "docker-containerd needs a toolchain w/ threads"
>  	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
>  	depends on BR2_USE_MMU
>  	depends on !BR2_TOOLCHAIN_HAS_THREADS
> +
> +comment "docker-containerd needs a toolchain not affected by binutils bug 20006"
> +	depends on BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006

This comment should not appear on architecture not supported by go, so
you should replicate at least:

  	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
  	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS

However, overall, I'm wondering if we shouldn't keep it simple and
stupid, and simply make BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS depend on
BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006. Yes, there won't be comments in
each and every package, but do we really need that for such a corner
case ?

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 3/3] package: hide golang packages for toolchains with binutils bug 20006

[Yann E. MORIN]

Thomas, All,

On 2018-11-24 22:56 +0100, Thomas Petazzoni spake thusly:
> On Sat, 24 Nov 2018 22:52:01 +0100, Yann E. MORIN wrote:
> > Fixes:
> >     http://autobuild.buildroot.org/results/020/02039969b16534d4020ecd4574bae71b91c1e6b8/ (flannel)
> >     http://autobuild.buildroot.org/results/e95/e9528b06b350ef84c1e2cb59fba87b4db77b4660/ (docker-engine)
> >     [...]
> > 
> > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> > Cc: Romain Naour <romain.naour@gmail.com>
> > ---
> >  package/docker-containerd/Config.in | 4 ++++
> >  package/docker-engine/Config.in     | 4 ++++
> >  package/docker-proxy/Config.in      | 4 ++++
> >  package/flannel/Config.in           | 4 ++++
> >  package/mender/Config.in            | 4 ++++
> >  package/runc/Config.in              | 4 ++++
> >  6 files changed, 24 insertions(+)
> > 
> > diff --git a/package/docker-containerd/Config.in b/package/docker-containerd/Config.in
> > index 851345f73e..39f6ea8ee8 100644
> > --- a/package/docker-containerd/Config.in
> > +++ b/package/docker-containerd/Config.in
> > @@ -4,6 +4,7 @@ config BR2_PACKAGE_DOCKER_CONTAINERD
> >  	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
> >  	depends on BR2_TOOLCHAIN_HAS_THREADS
> >  	depends on BR2_USE_MMU # util-linux
> > +	depends on !BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
> >  	select BR2_PACKAGE_RUNC # runtime dependency
> >  	select BR2_PACKAGE_UTIL_LINUX # runtime dependency
> >  	select BR2_PACKAGE_UTIL_LINUX_BINARIES
> > @@ -32,3 +33,6 @@ comment "docker-containerd needs a toolchain w/ threads"
> >  	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
> >  	depends on BR2_USE_MMU
> >  	depends on !BR2_TOOLCHAIN_HAS_THREADS
> > +
> > +comment "docker-containerd needs a toolchain not affected by binutils bug 20006"
> > +	depends on BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006
> 
> This comment should not appear on architecture not supported by go, so
> you should replicate at least:
> 
>   	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
>   	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
> 
> However, overall, I'm wondering if we shouldn't keep it simple and
> stupid, and simply make BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS depend on
> BR2_TOOLCHAIN_HAS_BINUTILS_BUG_20006.

ACK, good idea.

> Yes, there won't be comments in
> each and every package, but do we really need that for such a corner
> case ?

I don't care, at least.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'