[Buildroot] gmp.h error compiling Linux 5.11 with Buildroot 2021.02 default arm toolchain

[Buildroot] gmp.h error compiling Linux 5.11 with Buildroot 2021.02 default arm toolchain

[Michael Opdenacker]

Greetings,

I'm try to compile Linux 5.11 for BeagleBone Black with the default
toolchain generated by Buildroot 2021.02 (uClibc, gcc 9.3 for Cortex
A8). I'm on Ubuntu 20.04.

cd linux-5.11.8
export PATH=/home/tux/boot-time-labs/rootfs/buildroot/output/host/bin:$PATH
export ARCH=arm
export CROSS_COMPILE=arm-linux-
make omap2plus_defconfig
make

...

?HOSTLD scripts/dtc/dtc
HOSTCXX scripts/gcc-plugins/arm_ssp_per_task_plugin.so
In file included from
/home/tux/boot-time-labs/rootfs/buildroot/output/host/lib/gcc/arm-buildroot-linux-uclibcgnueabihf/9.3.0/plugin/include/gcc-plugin.h:28,
from scripts/gcc-plugins/gcc-common.h:7,
from scripts/gcc-plugins/arm_ssp_per_task_plugin.c:3:
/home/tux/boot-time-labs/rootfs/buildroot/output/host/lib/gcc/arm-buildroot-linux-uclibcgnueabihf/9.3.0/plugin/include/system.h:687:10:
fatal error: gmp.h: No such file or directory
687 | #include <gmp.h>
| ^~~~~~~
compilation terminated.
make[2]: *** [scripts/gcc-plugins/Makefile:47:
scripts/gcc-plugins/arm_ssp_per_task_plugin.so] Error 1
make[1]: *** [scripts/Makefile.build:496: scripts/gcc-plugins] Error 2

make: *** [Makefile:1185: scripts] Error 2


arm-linux-gcc --version
arm-linux-gcc.br_real (Buildroot 2021.02-378-g603b5d0fa9) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.? There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Does it ring a bell to some of you? I didn't find any similar error
reports on this list, so anyway I guess it's useful to report it.

Cheers,

Michael.


-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] gmp.h error compiling Linux 5.11 with Buildroot 2021.02 default arm toolchain

[Michael Opdenacker]

Hello again,

On 3/24/21 10:14 AM, Michael Opdenacker wrote:
> ?HOSTLD scripts/dtc/dtc
> HOSTCXX scripts/gcc-plugins/arm_ssp_per_task_plugin.so
> In file included from
> /home/tux/boot-time-labs/rootfs/buildroot/output/host/lib/gcc/arm-buildroot-linux-uclibcgnueabihf/9.3.0/plugin/include/gcc-plugin.h:28,
> from scripts/gcc-plugins/gcc-common.h:7,
> from scripts/gcc-plugins/arm_ssp_per_task_plugin.c:3:
> /home/tux/boot-time-labs/rootfs/buildroot/output/host/lib/gcc/arm-buildroot-linux-uclibcgnueabihf/9.3.0/plugin/include/system.h:687:10:
> fatal error: gmp.h: No such file or directory
> 687 | #include <gmp.h>
> | ^~~~~~~
> compilation terminated.
> make[2]: *** [scripts/gcc-plugins/Makefile:47:
> scripts/gcc-plugins/arm_ssp_per_task_plugin.so] Error 1
> make[1]: *** [scripts/Makefile.build:496: scripts/gcc-plugins] Error 2
>
> make: *** [Makefile:1185: scripts] Error 2

Good news, it turns out that it's not a Buildroot issue, but rather a
kernel one?

  * I also have it with the ARM GNU toolchain (gcc 10.2,
    https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-a/downloads)
  * I don't have it compiling the kernel with the sama5_defconfig
    configuration.

When I find a solution, I'll share it here.

Cheers,

Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] gmp.h error compiling Linux 5.11 with Buildroot 2021.02 default arm toolchain

[Michael Opdenacker]

Hello again,

On 3/24/21 10:35 AM, Michael Opdenacker wrote:
> Hello again,
>
> On 3/24/21 10:14 AM, Michael Opdenacker wrote:
>> ?HOSTLD scripts/dtc/dtc
>> HOSTCXX scripts/gcc-plugins/arm_ssp_per_task_plugin.so
>> In file included from
>> /home/tux/boot-time-labs/rootfs/buildroot/output/host/lib/gcc/arm-buildroot-linux-uclibcgnueabihf/9.3.0/plugin/include/gcc-plugin.h:28,
>> from scripts/gcc-plugins/gcc-common.h:7,
>> from scripts/gcc-plugins/arm_ssp_per_task_plugin.c:3:
>> /home/tux/boot-time-labs/rootfs/buildroot/output/host/lib/gcc/arm-buildroot-linux-uclibcgnueabihf/9.3.0/plugin/include/system.h:687:10:
>> fatal error: gmp.h: No such file or directory
>> 687 | #include <gmp.h>
>> | ^~~~~~~
>> compilation terminated.
>> make[2]: *** [scripts/gcc-plugins/Makefile:47:
>> scripts/gcc-plugins/arm_ssp_per_task_plugin.so] Error 1
>> make[1]: *** [scripts/Makefile.build:496: scripts/gcc-plugins] Error 2
>>
>> make: *** [Makefile:1185: scripts] Error 2
> Good news, it turns out that it's not a Buildroot issue, but rather a
> kernel one?
>
>   * I also have it with the ARM GNU toolchain (gcc 10.2,
>     https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-a/downloads)
>   * I don't have it compiling the kernel with the sama5_defconfig
>     configuration.
>
> When I find a solution, I'll share it here.

For Ubuntu 20.04 and the omap2plus_defconfig configuration, a working
solution is to use the ARM cross-compiler package (sudo apt install
gcc-arm-linux-gnueabi).

So, I'm not stuck, but I'd like to use the Buildroot toolchain anyway...

Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] gmp.h error compiling Linux 5.11 with Buildroot 2021.02 default arm toolchain

[Thomas Petazzoni]

Hello,

On Wed, 24 Mar 2021 10:14:26 +0100
Michael Opdenacker <michael.opdenacker@bootlin.com> wrote:

> I'm try to compile Linux 5.11 for BeagleBone Black with the default
> toolchain generated by Buildroot 2021.02 (uClibc, gcc 9.3 for Cortex
> A8). I'm on Ubuntu 20.04.
> 
> cd linux-5.11.8
> export PATH=/home/tux/boot-time-labs/rootfs/buildroot/output/host/bin:$PATH
> export ARCH=arm
> export CROSS_COMPILE=arm-linux-
> make omap2plus_defconfig
> make

Do you have a Buildroot configuration that uses an internal toolchain +
a Linux kernel built by Buildroot, that reproduces this? Or does it
only happens with an external toolchain and/or when you build the
kernel outside of Buildroot?

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] gmp.h error compiling Linux 5.11 with Buildroot 2021.02 default arm toolchain

[Michael Opdenacker]

Hi Thomas,

Thanks for your questions!

On 3/24/21 10:50 AM, Thomas Petazzoni wrote:
> Do you have a Buildroot configuration that uses an internal toolchain +
> a Linux kernel built by Buildroot, that reproduces this? Or does it
> only happens with an external toolchain and/or when you build the
> kernel outside of Buildroot?


Actually, the compile job succeeds with the "omap2plus" configuration
when the 5.11 kernel is built by Buildroot. Good guess.

What doesn't work is building the kernel outside Buildroot with (the
Buildroot toolchain from output/host/ or the latest ARM 32 GNU toolchain).

Does this help?

Cheers,

Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] gmp.h error compiling Linux 5.11 with Buildroot 2021.02 default arm toolchain

[Thomas Petazzoni]

On Wed, 24 Mar 2021 14:47:28 +0100
Michael Opdenacker <michael.opdenacker@bootlin.com> wrote:

> On 3/24/21 10:50 AM, Thomas Petazzoni wrote:
> > Do you have a Buildroot configuration that uses an internal toolchain +
> > a Linux kernel built by Buildroot, that reproduces this? Or does it
> > only happens with an external toolchain and/or when you build the
> > kernel outside of Buildroot?  
> 
> Actually, the compile job succeeds with the "omap2plus" configuration
> when the 5.11 kernel is built by Buildroot. Good guess.
> 
> What doesn't work is building the kernel outside Buildroot with (the
> Buildroot toolchain from output/host/ or the latest ARM 32 GNU toolchain).
> 
> Does this help?

It doesn't help per-se, but it's another useful data point. This whole
gmp.h is due to the gcc plugins that the kernel is building.

The gcc plugin causing issue there is built when
CONFIG_GCC_PLUGIN_ARM_SSP_PER_TASK=y in your kernel configuration.
Could you double check you have that enabled both when building the
kernel within Buildroot, and outside of Buildroot ?

To me, it seems like passing
HOSTCXXFLAGS=-I/path/to/buildroot/output/host/include when building
the kernel should get you passed the issue when building the kernel
externally.

And it probably works when Buildroot builds the kernel, because we're
passing:

        HOSTCC="$(HOSTCC) $(HOST_CFLAGS) $(HOST_LDFLAGS)" \

when building the kernel, so the kernel build system does invoke the
native gcc with the appropriate -I option pointing to
$(HOST_DIR)/include.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] gmp.h error compiling Linux 5.11 with Buildroot 2021.02 default arm toolchain

[Michael Opdenacker]

Hi Thomas

Thanks for your help!

On 3/24/21 3:12 PM, Thomas Petazzoni wrote:
> On Wed, 24 Mar 2021 14:47:28 +0100
> Michael Opdenacker <michael.opdenacker@bootlin.com> wrote:
>
>> On 3/24/21 10:50 AM, Thomas Petazzoni wrote:
>>> Do you have a Buildroot configuration that uses an internal toolchain +
>>> a Linux kernel built by Buildroot, that reproduces this? Or does it
>>> only happens with an external toolchain and/or when you build the
>>> kernel outside of Buildroot?  
>> Actually, the compile job succeeds with the "omap2plus" configuration
>> when the 5.11 kernel is built by Buildroot. Good guess.
>>
>> What doesn't work is building the kernel outside Buildroot with (the
>> Buildroot toolchain from output/host/ or the latest ARM 32 GNU toolchain).
>>
>> Does this help?
> It doesn't help per-se, but it's another useful data point. This whole
> gmp.h is due to the gcc plugins that the kernel is building.
>
> The gcc plugin causing issue there is built when
> CONFIG_GCC_PLUGIN_ARM_SSP_PER_TASK=y in your kernel configuration.
> Could you double check you have that enabled both when building the
> kernel within Buildroot, and outside of Buildroot ?


Indeed, when compiling the kernel from outside of Buildroot is breaking,
CONFIG_GCC_PLUGIN_ARM_SSP_PER_TASK is set,and if I disable this option,
the kernel builds fine. The kernel built by Buildroot internally didn't
have this flag.

What was unexpected for me is that this option is set by "make
omap2plus_defconfig" depending on your CROSS_COMPILE setting. With
arm-linux-gnueabi-gcc package by Ubuntu, you don't have it, but you do
have it with the Buildroot or ARM toolchain. It's because it depends on
a test for gcc plugins (see
https://elixir.bootlin.com/linux/latest/A/ident/CONFIG_GCC_PLUGIN_ARM_SSP_PER_TASK
and https://elixir.bootlin.com/linux/latest/K/ident/CONFIG_GCC_PLUGINS)

>
> To me, it seems like passing
> HOSTCXXFLAGS=-I/path/to/buildroot/output/host/include when building
> the kernel should get you passed the issue when building the kernel
> externally.


Right, I confirm this works. Thanks!

>
> And it probably works when Buildroot builds the kernel, because we're
> passing:
>
>         HOSTCC="$(HOSTCC) $(HOST_CFLAGS) $(HOST_LDFLAGS)" \
>
> when building the kernel, so the kernel build system does invoke the
> native gcc with the appropriate -I option pointing to
> $(HOST_DIR)/include.

Good to know, thanks! Is there anything that should be changed in the
Linux build system or in our toolchains ?

Anyway, thanks for documenting this behavior.

Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] gmp.h error compiling Linux 5.11 with Buildroot 2021.02 default arm toolchain

[Thomas Petazzoni]

On Wed, 24 Mar 2021 15:54:12 +0100
Michael Opdenacker <michael.opdenacker@bootlin.com> wrote:

> Indeed, when compiling the kernel from outside of Buildroot is breaking,
> CONFIG_GCC_PLUGIN_ARM_SSP_PER_TASK is set,and if I disable this option,
> the kernel builds fine. The kernel built by Buildroot internally didn't
> have this flag.

Ah, in Buildroot we have committed
40e0bf4c0c36970f8e94591fdb0a5a8910bf9859 which forcibly disables the
use of gcc plugins.

There's a lengthy explanation from Yann about that. Though admittedly,
long term, this is not really a very satisfying solution.

> What was unexpected for me is that this option is set by "make
> omap2plus_defconfig" depending on your CROSS_COMPILE setting. With
> arm-linux-gnueabi-gcc package by Ubuntu, you don't have it, but you do
> have it with the Buildroot or ARM toolchain. It's because it depends on
> a test for gcc plugins (see
> https://elixir.bootlin.com/linux/latest/A/ident/CONFIG_GCC_PLUGIN_ARM_SSP_PER_TASK
> and https://elixir.bootlin.com/linux/latest/K/ident/CONFIG_GCC_PLUGINS)
> 
> >
> > To me, it seems like passing
> > HOSTCXXFLAGS=-I/path/to/buildroot/output/host/include when building
> > the kernel should get you passed the issue when building the kernel
> > externally.  
> 
> 
> Right, I confirm this works. Thanks!

Good!

> > when building the kernel, so the kernel build system does invoke the
> > native gcc with the appropriate -I option pointing to
> > $(HOST_DIR)/include.  
> 
> Good to know, thanks! Is there anything that should be changed in the
> Linux build system or in our toolchains ?

No, I don't think there's anything that can be changed here. Or perhaps
pkg-config could be used to detect which compiler flags are needed to
build gcc plugins? Doesn't seem that easy though.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com