enabling COMPILE_TEST support for GCC plugins in v4.11

enabling COMPILE_TEST support for GCC plugins in v4.11

[Kees Cook]

Hi,

I'd like to get the GCC plugins building under
allyesconfig/allmodconfig for -next soon (with the intention of
landing the change in v4.11). Specifically, I intend to revert
a519167e753e ("gcc-plugins: disable under COMPILE_TEST").

Right now the plugins are only supported on x86, arm, and arm64,
though powerpc may happen in either v4.10 or v4.11 as well. This means
that the autobuilders for these architectures need to have the "gcc
plugin development" package installed which contains the GCC headers
needed for the plugins. For Debian/Ubuntu, this is gcc-$N-plugin-dev
(and for cross compilers: gcc-$N-plugin-dev-$arch-linux-$abi). For
Fedora, it is gcc-plugin-devel (though I'm not sure the naming for
cross compilers). Manual builds of compilers should already have these
headers installed.

The "noisy" plugin, cyc_complexity, is just an example, and I have
disabled it (which is pending[1] for v4.10). The remaining ones
(sancov and latent_entropy) are what I'm hoping to see tested
tree-wide (with the expectation that more are coming down the road:
initify, randstruct, structleak, constify, ...)

IIUC, the 0day builder already has the headers installed. I tried to
look through linux-next to find all the other folks that do
autobuilding on these architectures; apologies if I've missed anyone.

If you have a moment, applying 215e2aa6c024[1] and reverting
a519167e753e for an allyesconfig/allmodconfig build should let you
know if things are working correctly with headers installed. If anyone
sees any problems, please let me know and I can queue up fixes.

Thanks!

-Kees

[1] http://git.kernel.org/cgit/linux/kernel/git/kees/linux.git/commit/?h=for-next/gcc-plugins&id=215e2aa6c024d27cdbe88e2ea88cb59dcab588eb

-- 
Kees Cook
Nexus Security

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[Paul Gortmaker]

On Thu, Dec 8, 2016 at 2:00 PM, Kees Cook <keescook@google.com> wrote:
> Hi,
>
> I'd like to get the GCC plugins building under
> allyesconfig/allmodconfig for -next soon (with the intention of
> landing the change in v4.11). Specifically, I intend to revert
> a519167e753e ("gcc-plugins: disable under COMPILE_TEST").

If I recall correctly, I noted that the plugins broke the kernel.org
cross compiler toolchains which led to the above disable.

Has that changed?    People who have been doing tree wide changes
and have been compiling across a bunch of different arch to ensure
their changes don't cause breakage should not be left out in the cold.

If there are newer toolchains that I can download and use in "toaster"
mode, then great.   But I (and many others) don't want to waste a day
trying to make the latest gcc build some obsolete pa-risc architecture
just to support a Kconfig default setting change.

Can you elaborate on the motivation for this change?  At the moment
I see more downsides than advantages.

Thanks,
Paul.
--

>
> Right now the plugins are only supported on x86, arm, and arm64,
> though powerpc may happen in either v4.10 or v4.11 as well. This means
> that the autobuilders for these architectures need to have the "gcc
> plugin development" package installed which contains the GCC headers
> needed for the plugins. For Debian/Ubuntu, this is gcc-$N-plugin-dev
> (and for cross compilers: gcc-$N-plugin-dev-$arch-linux-$abi). For
> Fedora, it is gcc-plugin-devel (though I'm not sure the naming for
> cross compilers). Manual builds of compilers should already have these
> headers installed.
>
> The "noisy" plugin, cyc_complexity, is just an example, and I have
> disabled it (which is pending[1] for v4.10). The remaining ones
> (sancov and latent_entropy) are what I'm hoping to see tested
> tree-wide (with the expectation that more are coming down the road:
> initify, randstruct, structleak, constify, ...)
>
> IIUC, the 0day builder already has the headers installed. I tried to
> look through linux-next to find all the other folks that do
> autobuilding on these architectures; apologies if I've missed anyone.
>
> If you have a moment, applying 215e2aa6c024[1] and reverting
> a519167e753e for an allyesconfig/allmodconfig build should let you
> know if things are working correctly with headers installed. If anyone
> sees any problems, please let me know and I can queue up fixes.
>
> Thanks!
>
> -Kees
>
> [1] http://git.kernel.org/cgit/linux/kernel/git/kees/linux.git/commit/?h=for-next/gcc-plugins&id=215e2aa6c024d27cdbe88e2ea88cb59dcab588eb
>
> --
> Kees Cook
> Nexus Security
> --
> To unsubscribe from this list: send the line "unsubscribe linux-next" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[Kees Cook]

On Thu, Dec 8, 2016 at 5:52 PM, Paul Gortmaker
<paul.gortmaker@windriver.com> wrote:
> On Thu, Dec 8, 2016 at 2:00 PM, Kees Cook <keescook@google.com> wrote:
>> Hi,
>>
>> I'd like to get the GCC plugins building under
>> allyesconfig/allmodconfig for -next soon (with the intention of
>> landing the change in v4.11). Specifically, I intend to revert
>> a519167e753e ("gcc-plugins: disable under COMPILE_TEST").
>
> If I recall correctly, I noted that the plugins broke the kernel.org
> cross compiler toolchains which led to the above disable.

Do you mean these?
https://www.kernel.org/pub/tools/crosstool/

> Has that changed?    People who have been doing tree wide changes
> and have been compiling across a bunch of different arch to ensure
> their changes don't cause breakage should not be left out in the cold.

Totally agreed! :) That's why I want to get the ball rolling now, so
there's plenty of time.

> If there are newer toolchains that I can download and use in "toaster"
> mode, then great.   But I (and many others) don't want to waste a day
> trying to make the latest gcc build some obsolete pa-risc architecture
> just to support a Kconfig default setting change.

The good news is, pa-risc doesn't support the plugins yet. It's only
x86, arm, arm64, and soon powerpc. The point being, the less common
architectures won't be hit by this anyway.

Regardless, it would be nice to get a newer set of toolchains up on
kernel.org...

> Can you elaborate on the motivation for this change?  At the moment
> I see more downsides than advantages.

Since the plugins run during every object build, getting them running
in the widest possible coverage means we'll shake out bugs more
rapidly.

-Kees

>
> Thanks,
> Paul.
> --
>
>>
>> Right now the plugins are only supported on x86, arm, and arm64,
>> though powerpc may happen in either v4.10 or v4.11 as well. This means
>> that the autobuilders for these architectures need to have the "gcc
>> plugin development" package installed which contains the GCC headers
>> needed for the plugins. For Debian/Ubuntu, this is gcc-$N-plugin-dev
>> (and for cross compilers: gcc-$N-plugin-dev-$arch-linux-$abi). For
>> Fedora, it is gcc-plugin-devel (though I'm not sure the naming for
>> cross compilers). Manual builds of compilers should already have these
>> headers installed.
>>
>> The "noisy" plugin, cyc_complexity, is just an example, and I have
>> disabled it (which is pending[1] for v4.10). The remaining ones
>> (sancov and latent_entropy) are what I'm hoping to see tested
>> tree-wide (with the expectation that more are coming down the road:
>> initify, randstruct, structleak, constify, ...)
>>
>> IIUC, the 0day builder already has the headers installed. I tried to
>> look through linux-next to find all the other folks that do
>> autobuilding on these architectures; apologies if I've missed anyone.
>>
>> If you have a moment, applying 215e2aa6c024[1] and reverting
>> a519167e753e for an allyesconfig/allmodconfig build should let you
>> know if things are working correctly with headers installed. If anyone
>> sees any problems, please let me know and I can queue up fixes.
>>
>> Thanks!
>>
>> -Kees
>>
>> [1] http://git.kernel.org/cgit/linux/kernel/git/kees/linux.git/commit/?h=for-next/gcc-plugins&id=215e2aa6c024d27cdbe88e2ea88cb59dcab588eb
>>
>> --
>> Kees Cook
>> Nexus Security
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-next" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Kees Cook
Nexus Security

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[Arnd Bergmann]

On Thursday, December 8, 2016 11:00:42 AM CET Kees Cook wrote:

> If you have a moment, applying 215e2aa6c024[1] and reverting
> a519167e753e for an allyesconfig/allmodconfig build should let you
> know if things are working correctly with headers installed. If anyone
> sees any problems, please let me know and I can queue up fixes.

Using gcc-4.9.3 or gcc-5.3.1 for an ARM allmodconfig build, I get tons of
errors such as this one:

/git/arm-soc/init/initramfs.c: In function 'error':
/git/arm-soc/init/initramfs.c:50:1: error: unrecognizable insn:
 }
 ^
(insn 26 25 27 5 (set (reg:SI 111 [ local_entropy.243 ])
        (rotatert:SI (reg:SI 116 [ local_entropy.243 ])
            (const_int -30 [0xffffffffffffffe2]))) -1
     (nil))
*** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins.
Event                            | Plugins
PLUGIN_ATTRIBUTES                | latent_entropy_plugin
PLUGIN_START_UNIT                | latent_entropy_plugin
/git/arm-soc/init/initramfs.c:50:1: internal compiler error: in extract_insn, at recog.c:2202
/git/arm-soc/arch/arm/vfp/vfpmodule.c: In function 'vfp_init':
/git/arm-soc/arch/arm/vfp/vfpmodule.c:824:1: error: unrecognizable insn:
 }
 ^
(insn 138 137 139 17 (set (reg:SI 165 [ local_entropy.93 ])
        (rotatert:SI (reg:SI 150 [ local_entropy.93 ])
            (const_int -9 [0xfffffffffffffff7]))) -1
     (nil))
*** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins.

Using gcc-6.1.1 or gcc-7.0.0, everything works fine as far as I
can tell. With some older Ubuntu binary toolchains, I get this one:

In file included from <stdin>:1:0:
/git/arm-soc/scripts/gcc-plugins/gcc-common.h:4:22: fatal error: bversion.h: No such file or directory
compilation terminated.
Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?

and I can't find the packages for the headers (I still get them for
gcc-5, but not older versions). On very old toolchains (e.g. gcc-4.3),
I get this one:

Cannot use CONFIG_GCC_PLUGINS: your gcc version does not support plugins, you should upgrade it to at least gcc 4.5
scripts/Makefile.gcc-plugins:54: recipe for target 'gcc-plugins-check' failed

	Arnd

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[Arnd Bergmann]

On Thursday, December 8, 2016 11:00:42 AM CET Kees Cook wrote:
> Hi,
> 
> I'd like to get the GCC plugins building under
> allyesconfig/allmodconfig for -next soon (with the intention of
> landing the change in v4.11). Specifically, I intend to revert
> a519167e753e ("gcc-plugins: disable under COMPILE_TEST").
> 
> Right now the plugins are only supported on x86, arm, and arm64,
> though powerpc may happen in either v4.10 or v4.11 as well. This means
> that the autobuilders for these architectures need to have the "gcc
> plugin development" package installed which contains the GCC headers
> needed for the plugins. For Debian/Ubuntu, this is gcc-$N-plugin-dev
> (and for cross compilers: gcc-$N-plugin-dev-$arch-linux-$abi). For
> Fedora, it is gcc-plugin-devel (though I'm not sure the naming for
> cross compilers). Manual builds of compilers should already have these
> headers installed.
> 
> The "noisy" plugin, cyc_complexity, is just an example, and I have
> disabled it (which is pending[1] for v4.10). The remaining ones
> (sancov and latent_entropy) are what I'm hoping to see tested
> tree-wide (with the expectation that more are coming down the road:
> initify, randstruct, structleak, constify, ...)
> 
> IIUC, the 0day builder already has the headers installed. I tried to
> look through linux-next to find all the other folks that do
> autobuilding on these architectures; apologies if I've missed anyone.
> 
> If you have a moment, applying 215e2aa6c024[1] and reverting
> a519167e753e for an allyesconfig/allmodconfig build should let you
> know if things are working correctly with headers installed. If anyone
> sees any problems, please let me know and I can queue up fixes.
> 
> Thanks!
> 
> -Kees
> 
> [1] http://git.kernel.org/cgit/linux/kernel/git/kees/linux.git/commit/?h=for-next/gcc-plugins&id=215e2aa6c024d27cdbe88e2ea88cb59dcab588eb

This is what I got on x86-64 with a gcc-7.0.0 snapshot:

In file included from /git/arm-soc/scripts/gcc-plugins/gcc-common.h:42:0,
                 from <stdin>:1:
/home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/emit-rtl.h:371:41: error: use of enum ‘memmodel’ without previous declaration
 extern bool need_atomic_barrier_p (enum memmodel, bool);
                                         ^
In file included from /git/arm-soc/scripts/gcc-plugins/gcc-common.h:94:0,
                 from <stdin>:1:
/home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/tree-ssanames.h:70:40: error: use of enum ‘value_range_type’ without previous declaration
 extern void set_range_info (tree, enum value_range_type, const wide_int_ref &,
                                        ^
/home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/tree-ssanames.h:73:13: error: use of enum ‘value_range_type’ without previous declaration
 extern enum value_range_type get_range_info (const_tree, wide_int *,
             ^
/home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/tree-ssanames.h:98:55: error: use of enum ‘value_range_type’ without previous declaration
 extern void duplicate_ssa_name_range_info (tree, enum value_range_type,
                                                       ^
Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?
scripts/Makefile.gcc-plugins:51: recipe for target 'gcc-plugins-check' failed

I manually fixed up the gcc header files to include the ones with the
definition for now, to address those, but I don't know if that change is
correct.

	Arnd

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[Kees Cook]

On Fri, Dec 9, 2016 at 2:40 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Thursday, December 8, 2016 11:00:42 AM CET Kees Cook wrote:
>
>> If you have a moment, applying 215e2aa6c024[1] and reverting
>> a519167e753e for an allyesconfig/allmodconfig build should let you
>> know if things are working correctly with headers installed. If anyone
>> sees any problems, please let me know and I can queue up fixes.
>
> Using gcc-4.9.3 or gcc-5.3.1 for an ARM allmodconfig build, I get tons of
> errors such as this one:
>
> /git/arm-soc/init/initramfs.c: In function 'error':
> /git/arm-soc/init/initramfs.c:50:1: error: unrecognizable insn:
>  }
>  ^
> (insn 26 25 27 5 (set (reg:SI 111 [ local_entropy.243 ])
>         (rotatert:SI (reg:SI 116 [ local_entropy.243 ])
>             (const_int -30 [0xffffffffffffffe2]))) -1
>      (nil))
> *** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins.
> Event                            | Plugins
> PLUGIN_ATTRIBUTES                | latent_entropy_plugin
> PLUGIN_START_UNIT                | latent_entropy_plugin
> /git/arm-soc/init/initramfs.c:50:1: internal compiler error: in extract_insn, at recog.c:2202
> /git/arm-soc/arch/arm/vfp/vfpmodule.c: In function 'vfp_init':
> /git/arm-soc/arch/arm/vfp/vfpmodule.c:824:1: error: unrecognizable insn:
>  }
>  ^
> (insn 138 137 139 17 (set (reg:SI 165 [ local_entropy.93 ])
>         (rotatert:SI (reg:SI 150 [ local_entropy.93 ])
>             (const_int -9 [0xfffffffffffffff7]))) -1
>      (nil))
> *** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins.

Well that's exciting! :P

>
> Using gcc-6.1.1 or gcc-7.0.0, everything works fine as far as I
> can tell. With some older Ubuntu binary toolchains, I get this one:
>
> In file included from <stdin>:1:0:
> /git/arm-soc/scripts/gcc-plugins/gcc-common.h:4:22: fatal error: bversion.h: No such file or directory
> compilation terminated.
> Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?
>
> and I can't find the packages for the headers (I still get them for
> gcc-5, but not older versions). On very old toolchains (e.g. gcc-4.3),
> I get this one:
>
> Cannot use CONFIG_GCC_PLUGINS: your gcc version does not support plugins, you should upgrade it to at least gcc 4.5
> scripts/Makefile.gcc-plugins:54: recipe for target 'gcc-plugins-check' failed

I'm starting to wonder if we need to expose the compiler version to
Kconfig so that "all*config" builds for earlier compiler will
automatically leave things like plugins off. But I have no idea what
the right approach for that might be.

I'm not a fan of silently disabling stuff (like is done, for example,
for KCOV). If you've selected a Kconfig that your compiler can't
build, it should fail to build, but that's different from the
allyesconfig etc, since that's about compile tests not a sane kernel
image.

Perhaps if COMPILE_TEST is enabled, and there is something unsupported
by the compiler, only then would it warn and continue, instead of
killing build?

-Kees

-- 
Kees Cook
Nexus Security

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[Kees Cook]

On Fri, Dec 9, 2016 at 3:33 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Thursday, December 8, 2016 11:00:42 AM CET Kees Cook wrote:
>> Hi,
>>
>> I'd like to get the GCC plugins building under
>> allyesconfig/allmodconfig for -next soon (with the intention of
>> landing the change in v4.11). Specifically, I intend to revert
>> a519167e753e ("gcc-plugins: disable under COMPILE_TEST").
>>
>> Right now the plugins are only supported on x86, arm, and arm64,
>> though powerpc may happen in either v4.10 or v4.11 as well. This means
>> that the autobuilders for these architectures need to have the "gcc
>> plugin development" package installed which contains the GCC headers
>> needed for the plugins. For Debian/Ubuntu, this is gcc-$N-plugin-dev
>> (and for cross compilers: gcc-$N-plugin-dev-$arch-linux-$abi). For
>> Fedora, it is gcc-plugin-devel (though I'm not sure the naming for
>> cross compilers). Manual builds of compilers should already have these
>> headers installed.
>>
>> The "noisy" plugin, cyc_complexity, is just an example, and I have
>> disabled it (which is pending[1] for v4.10). The remaining ones
>> (sancov and latent_entropy) are what I'm hoping to see tested
>> tree-wide (with the expectation that more are coming down the road:
>> initify, randstruct, structleak, constify, ...)
>>
>> IIUC, the 0day builder already has the headers installed. I tried to
>> look through linux-next to find all the other folks that do
>> autobuilding on these architectures; apologies if I've missed anyone.
>>
>> If you have a moment, applying 215e2aa6c024[1] and reverting
>> a519167e753e for an allyesconfig/allmodconfig build should let you
>> know if things are working correctly with headers installed. If anyone
>> sees any problems, please let me know and I can queue up fixes.
>>
>> Thanks!
>>
>> -Kees
>>
>> [1] http://git.kernel.org/cgit/linux/kernel/git/kees/linux.git/commit/?h=for-next/gcc-plugins&id=215e2aa6c024d27cdbe88e2ea88cb59dcab588eb
>
> This is what I got on x86-64 with a gcc-7.0.0 snapshot:
>
> In file included from /git/arm-soc/scripts/gcc-plugins/gcc-common.h:42:0,
>                  from <stdin>:1:
> /home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/emit-rtl.h:371:41: error: use of enum ‘memmodel’ without previous declaration
>  extern bool need_atomic_barrier_p (enum memmodel, bool);
>                                          ^
> In file included from /git/arm-soc/scripts/gcc-plugins/gcc-common.h:94:0,
>                  from <stdin>:1:
> /home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/tree-ssanames.h:70:40: error: use of enum ‘value_range_type’ without previous declaration
>  extern void set_range_info (tree, enum value_range_type, const wide_int_ref &,
>                                         ^
> /home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/tree-ssanames.h:73:13: error: use of enum ‘value_range_type’ without previous declaration
>  extern enum value_range_type get_range_info (const_tree, wide_int *,
>              ^
> /home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/tree-ssanames.h:98:55: error: use of enum ‘value_range_type’ without previous declaration
>  extern void duplicate_ssa_name_range_info (tree, enum value_range_type,
>                                                        ^
> Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?
> scripts/Makefile.gcc-plugins:51: recipe for target 'gcc-plugins-check' failed
>
> I manually fixed up the gcc header files to include the ones with the
> definition for now, to address those, but I don't know if that change is
> correct.

What was needed?

-Kees

-- 
Kees Cook
Nexus Security

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[Linus Torvalds]

On Fri, Dec 9, 2016 at 11:12 AM, Kees Cook <keescook@google.com> wrote:
>
> I'm starting to wonder if we need to expose the compiler version to
> Kconfig so that "all*config" builds for earlier compiler will
> automatically leave things like plugins off. But I have no idea what
> the right approach for that might be.

We had some broken mock-up of a config option that did a "system()"
call to do some shell scripting for filling in config options (ie turn
a fail/pass into false/true boolean automatically with something like

   config COMPILER_SUPPORTS_XYZ
       bool
       option shell="gcc -XYZ"

The idea is really solid: move a lot of the nasty ad-hoc runtime
testing in the Makefiles to the configuration stage. I would seriously
like to see this, so that we could replace the stupid

  CFLAGS_KCOV     := $(call cc-option,-fsanitize-coverage=trace-pc,)

with just nicer thing (ok, that's a bad example, but some of the other
cc-option cases are really pretty fundamental to kernel configuration,
and it really matters whether the compiler supports something or not).

The main problem was actually that we don't have good infrastructure
for passing in the compiler environment etc to kbuild.

I forgot who did that mock-up (and what the exact syntax was), but it
was pretty simple.

                 Linus

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[Arnd Bergmann]

On Friday, December 9, 2016 11:13:20 AM CET Kees Cook wrote:
> On Fri, Dec 9, 2016 at 3:33 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> > On Thursday, December 8, 2016 11:00:42 AM CET Kees Cook wrote:
> >> If you have a moment, applying 215e2aa6c024[1] and reverting
> >> a519167e753e for an allyesconfig/allmodconfig build should let you
> >> know if things are working correctly with headers installed. If anyone
> >> sees any problems, please let me know and I can queue up fixes.
> >>
> > This is what I got on x86-64 with a gcc-7.0.0 snapshot:
> >
> > In file included from /git/arm-soc/scripts/gcc-plugins/gcc-common.h:42:0,
> >                  from <stdin>:1:
> > /home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/emit-rtl.h:371:41: error: use of enum ‘memmodel’ without previous declaration
> >  extern bool need_atomic_barrier_p (enum memmodel, bool);
> >                                          ^
> > In file included from /git/arm-soc/scripts/gcc-plugins/gcc-common.h:94:0,
> >                  from <stdin>:1:
> > /home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/tree-ssanames.h:70:40: error: use of enum ‘value_range_type’ without previous declaration
> >  extern void set_range_info (tree, enum value_range_type, const wide_int_ref &,
> >                                         ^
> > /home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/tree-ssanames.h:73:13: error: use of enum ‘value_range_type’ without previous declaration
> >  extern enum value_range_type get_range_info (const_tree, wide_int *,
> >              ^
> > /home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/tree-ssanames.h:98:55: error: use of enum ‘value_range_type’ without previous declaration
> >  extern void duplicate_ssa_name_range_info (tree, enum value_range_type,
> >                                                        ^
> > Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?
> > scripts/Makefile.gcc-plugins:51: recipe for target 'gcc-plugins-check' failed
> >
> > I manually fixed up the gcc header files to include the ones with the
> > definition for now, to address those, but I don't know if that change is
> > correct.
> 
> What was needed?

I added '#include "memmodel.h"' and '#include "tree-vrp.h"', respectively, to
the headers that failed to get compiled.

This might be the correct solution, or the headers might not be meant to
be used standalone and instead require being included in the right order.

Gcc commit svn+ssh://gcc.gnu.org/svn/gcc/trunk@239638 moved
value_range_type from tree-ssanames.h to tree-vrp.h, I've added
Kugan to Cc, maybe he can clarify what that means for plugins.

	Arnd

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[kugan]

On 10/12/16 07:35, Arnd Bergmann wrote:
> On Friday, December 9, 2016 11:13:20 AM CET Kees Cook wrote:
>> On Fri, Dec 9, 2016 at 3:33 AM, Arnd Bergmann <arnd@arndb.de> wrote:
>>> On Thursday, December 8, 2016 11:00:42 AM CET Kees Cook wrote:
>>>> If you have a moment, applying 215e2aa6c024[1] and reverting
>>>> a519167e753e for an allyesconfig/allmodconfig build should let you
>>>> know if things are working correctly with headers installed. If anyone
>>>> sees any problems, please let me know and I can queue up fixes.
>>>>
>>> This is what I got on x86-64 with a gcc-7.0.0 snapshot:
>>>
>>> In file included from /git/arm-soc/scripts/gcc-plugins/gcc-common.h:42:0,
>>>                  from <stdin>:1:
>>> /home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/emit-rtl.h:371:41: error: use of enum ‘memmodel’ without previous declaration
>>>  extern bool need_atomic_barrier_p (enum memmodel, bool);
>>>                                          ^
>>> In file included from /git/arm-soc/scripts/gcc-plugins/gcc-common.h:94:0,
>>>                  from <stdin>:1:
>>> /home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/tree-ssanames.h:70:40: error: use of enum ‘value_range_type’ without previous declaration
>>>  extern void set_range_info (tree, enum value_range_type, const wide_int_ref &,
>>>                                         ^
>>> /home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/tree-ssanames.h:73:13: error: use of enum ‘value_range_type’ without previous declaration
>>>  extern enum value_range_type get_range_info (const_tree, wide_int *,
>>>              ^
>>> /home/arnd/cross-gcc/lib/gcc/x86_64-linux/7.0.0/plugin/include/tree-ssanames.h:98:55: error: use of enum ‘value_range_type’ without previous declaration
>>>  extern void duplicate_ssa_name_range_info (tree, enum value_range_type,
>>>                                                        ^
>>> Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?
>>> scripts/Makefile.gcc-plugins:51: recipe for target 'gcc-plugins-check' failed
>>>
>>> I manually fixed up the gcc header files to include the ones with the
>>> definition for now, to address those, but I don't know if that change is
>>> correct.
>>
>> What was needed?
>
> I added '#include "memmodel.h"' and '#include "tree-vrp.h"', respectively, to
> the headers that failed to get compiled.
>
> This might be the correct solution, or the headers might not be meant to
> be used standalone and instead require being included in the right order.
>
> Gcc commit svn+ssh://gcc.gnu.org/svn/gcc/trunk@239638 moved
> value_range_type from tree-ssanames.h to tree-vrp.h, I've added
> Kugan to Cc, maybe he can clarify what that means for plugins.

Looks to me that in gcc-common.h  tree-vrp.h should be included before 
tree-ssanames.h as it is done in gcc now.

Thanks,
Kugan

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[Emese Revfy]

On Fri, 9 Dec 2016 11:12:18 -0800
Kees Cook <keescook@google.com> wrote:

> On Fri, Dec 9, 2016 at 2:40 AM, Arnd Bergmann <arnd@arndb.de> wrote:
> > On Thursday, December 8, 2016 11:00:42 AM CET Kees Cook wrote:
> >
> >> If you have a moment, applying 215e2aa6c024[1] and reverting
> >> a519167e753e for an allyesconfig/allmodconfig build should let you
> >> know if things are working correctly with headers installed. If anyone
> >> sees any problems, please let me know and I can queue up fixes.
> >
> > Using gcc-4.9.3 or gcc-5.3.1 for an ARM allmodconfig build, I get tons of
> > errors such as this one:
> >
> > /git/arm-soc/init/initramfs.c: In function 'error':
> > /git/arm-soc/init/initramfs.c:50:1: error: unrecognizable insn:
> >  }
> >  ^
> > (insn 26 25 27 5 (set (reg:SI 111 [ local_entropy.243 ])
> >         (rotatert:SI (reg:SI 116 [ local_entropy.243 ])
> >             (const_int -30 [0xffffffffffffffe2]))) -1
> >      (nil))
> > *** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins.
> > Event                            | Plugins
> > PLUGIN_ATTRIBUTES                | latent_entropy_plugin
> > PLUGIN_START_UNIT                | latent_entropy_plugin
> > /git/arm-soc/init/initramfs.c:50:1: internal compiler error: in extract_insn, at recog.c:2202
> > /git/arm-soc/arch/arm/vfp/vfpmodule.c: In function 'vfp_init':
> > /git/arm-soc/arch/arm/vfp/vfpmodule.c:824:1: error: unrecognizable insn:
> >  }
> >  ^
> > (insn 138 137 139 17 (set (reg:SI 165 [ local_entropy.93 ])
> >         (rotatert:SI (reg:SI 150 [ local_entropy.93 ])
> >             (const_int -9 [0xfffffffffffffff7]))) -1
> >      (nil))
> > *** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins.
> 
> Well that's exciting! :P

Hi,

You can find the fix here:
https://github.com/ephox-gcc-plugins/latent_entropy/commit/c91275a1bfcebbcfc0ca1af03396e06039f04db8

-- 
Emese

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[Kees Cook]

On Sat, Dec 10, 2016 at 8:45 AM, Emese Revfy <re.emese@gmail.com> wrote:
> On Fri, 9 Dec 2016 11:12:18 -0800
> Kees Cook <keescook@google.com> wrote:
>
>> On Fri, Dec 9, 2016 at 2:40 AM, Arnd Bergmann <arnd@arndb.de> wrote:
>> > On Thursday, December 8, 2016 11:00:42 AM CET Kees Cook wrote:
>> >
>> >> If you have a moment, applying 215e2aa6c024[1] and reverting
>> >> a519167e753e for an allyesconfig/allmodconfig build should let you
>> >> know if things are working correctly with headers installed. If anyone
>> >> sees any problems, please let me know and I can queue up fixes.
>> >
>> > Using gcc-4.9.3 or gcc-5.3.1 for an ARM allmodconfig build, I get tons of
>> > errors such as this one:
>> >
>> > /git/arm-soc/init/initramfs.c: In function 'error':
>> > /git/arm-soc/init/initramfs.c:50:1: error: unrecognizable insn:
>> >  }
>> >  ^
>> > (insn 26 25 27 5 (set (reg:SI 111 [ local_entropy.243 ])
>> >         (rotatert:SI (reg:SI 116 [ local_entropy.243 ])
>> >             (const_int -30 [0xffffffffffffffe2]))) -1
>> >      (nil))
>> > *** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins.
>> > Event                            | Plugins
>> > PLUGIN_ATTRIBUTES                | latent_entropy_plugin
>> > PLUGIN_START_UNIT                | latent_entropy_plugin
>> > /git/arm-soc/init/initramfs.c:50:1: internal compiler error: in extract_insn, at recog.c:2202
>> > /git/arm-soc/arch/arm/vfp/vfpmodule.c: In function 'vfp_init':
>> > /git/arm-soc/arch/arm/vfp/vfpmodule.c:824:1: error: unrecognizable insn:
>> >  }
>> >  ^
>> > (insn 138 137 139 17 (set (reg:SI 165 [ local_entropy.93 ])
>> >         (rotatert:SI (reg:SI 150 [ local_entropy.93 ])
>> >             (const_int -9 [0xfffffffffffffff7]))) -1
>> >      (nil))
>> > *** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins.
>>
>> Well that's exciting! :P
>
> Hi,
>
> You can find the fix here:
> https://github.com/ephox-gcc-plugins/latent_entropy/commit/c91275a1bfcebbcfc0ca1af03396e06039f04db8

Great, thanks!

-Kees

-- 
Kees Cook
Nexus Security

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[Kees Cook]

*thread necromancy*

On Fri, Dec 9, 2016 at 11:40 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Fri, Dec 9, 2016 at 11:12 AM, Kees Cook <keescook@google.com> wrote:
>> I'm starting to wonder if we need to expose the compiler version to
>> Kconfig so that "all*config" builds for earlier compiler will
>> automatically leave things like plugins off. But I have no idea what
>> the right approach for that might be.
>
> We had some broken mock-up of a config option that did a "system()"
> call to do some shell scripting for filling in config options (ie turn
> a fail/pass into false/true boolean automatically with something like
>
>    config COMPILER_SUPPORTS_XYZ
>        bool
>        option shell="gcc -XYZ"
>
> The idea is really solid: move a lot of the nasty ad-hoc runtime
> testing in the Makefiles to the configuration stage. I would seriously
> like to see this, so that we could replace the stupid
>
>   CFLAGS_KCOV     := $(call cc-option,-fsanitize-coverage=trace-pc,)
>
> with just nicer thing (ok, that's a bad example, but some of the other
> cc-option cases are really pretty fundamental to kernel configuration,
> and it really matters whether the compiler supports something or not).
>
> The main problem was actually that we don't have good infrastructure
> for passing in the compiler environment etc to kbuild.
>
> I forgot who did that mock-up (and what the exact syntax was), but it
> was pretty simple.

Yeah, agreed; I would love this for much more than just gcc plugins.
I'd rather that compiler-specific things were Kconfig-sane with
discoverable reasons for a Kconfig being unavailable.

Does anyone on the kbuild list remember where this mock-up was? I
haven't been able to find anything from searches (besides this thread
itself), and getting some ideas on how to approach it would be nice.
The way the .config file gets magically reloaded seems like solving
this could be very weird.

Thanks,

-Kees

-- 
Kees Cook
Pixel Security

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[Michael Ellerman]

[Old thread just popped up in my inbox]

Kees Cook <keescook@google.com> writes:
> On Thu, Dec 8, 2016 at 5:52 PM, Paul Gortmaker
> <paul.gortmaker@windriver.com> wrote:
>> On Thu, Dec 8, 2016 at 2:00 PM, Kees Cook <keescook@google.com> wrote:
>>> Hi,
>>>
>>> I'd like to get the GCC plugins building under
>>> allyesconfig/allmodconfig for -next soon (with the intention of
>>> landing the change in v4.11). Specifically, I intend to revert
>>> a519167e753e ("gcc-plugins: disable under COMPILE_TEST").
>>
>> If I recall correctly, I noted that the plugins broke the kernel.org
>> cross compiler toolchains which led to the above disable.
>
> Do you mean these?
> https://www.kernel.org/pub/tools/crosstool/

I think those are the ones Paul is talking about.

Those were mostly built by Tony, who is know, sadly, an Openstack
developer and for some reason doesn't seem to have time to build
toolchains for kernel hackers :P


As folks may have seen on LWN, there are now pre-built toolchains for
many architectures available from the Free Electrons guys, so that may
help to unblock this situation somewhat:

http://toolchains.free-electrons.com/


cheers

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[Kees Cook]

On Thu, Jun 22, 2017 at 10:43 PM, Michael Ellerman <mpe@ellerman.id.au> wrote:
> [Old thread just popped up in my inbox]
>
> Kees Cook <keescook@google.com> writes:
>> On Thu, Dec 8, 2016 at 5:52 PM, Paul Gortmaker
>> <paul.gortmaker@windriver.com> wrote:
>>> On Thu, Dec 8, 2016 at 2:00 PM, Kees Cook <keescook@google.com> wrote:
>>>> Hi,
>>>>
>>>> I'd like to get the GCC plugins building under
>>>> allyesconfig/allmodconfig for -next soon (with the intention of
>>>> landing the change in v4.11). Specifically, I intend to revert
>>>> a519167e753e ("gcc-plugins: disable under COMPILE_TEST").
>>>
>>> If I recall correctly, I noted that the plugins broke the kernel.org
>>> cross compiler toolchains which led to the above disable.
>>
>> Do you mean these?
>> https://www.kernel.org/pub/tools/crosstool/
>
> I think those are the ones Paul is talking about.
>
> Those were mostly built by Tony, who is know, sadly, an Openstack
> developer and for some reason doesn't seem to have time to build
> toolchains for kernel hackers :P
>
>
> As folks may have seen on LWN, there are now pre-built toolchains for
> many architectures available from the Free Electrons guys, so that may
> help to unblock this situation somewhat:
>
> http://toolchains.free-electrons.com/

Perhaps we should add a link in pub/tools/crosstool/ ? 4.6.3 is pretty
old at this point, but the Free Electrons toolchains don't include
some of the less used architectures.

-Kees

-- 
Kees Cook
Pixel Security

Re: enabling COMPILE_TEST support for GCC plugins in v4.11

[Tony Breeds]

[-- Attachment #1: Type: text/plain, Size: 2045 bytes --]

On Fri, Jun 23, 2017 at 03:43:58PM +1000, Michael Ellerman wrote:
> [Old thread just popped up in my inbox]
> 
> Kees Cook <keescook@google.com> writes:
> > On Thu, Dec 8, 2016 at 5:52 PM, Paul Gortmaker
> > <paul.gortmaker@windriver.com> wrote:
> >> On Thu, Dec 8, 2016 at 2:00 PM, Kees Cook <keescook@google.com> wrote:
> >>> Hi,
> >>>
> >>> I'd like to get the GCC plugins building under
> >>> allyesconfig/allmodconfig for -next soon (with the intention of
> >>> landing the change in v4.11). Specifically, I intend to revert
> >>> a519167e753e ("gcc-plugins: disable under COMPILE_TEST").
> >>
> >> If I recall correctly, I noted that the plugins broke the kernel.org
> >> cross compiler toolchains which led to the above disable.
> >
> > Do you mean these?
> > https://www.kernel.org/pub/tools/crosstool/
> 
> I think those are the ones Paul is talking about.
> 
> Those were mostly built by Tony, who is know, sadly, an Openstack
> developer and for some reason doesn't seem to have time to build
> toolchains for kernel hackers :P

Yeah sad but true.  Building a full suit took a couple of days and I'm
just not going to find that anytime soon.

I don't really want to build them one at a time but I could I suppose,
I'd play favotites x64_86, ppc64le and then the others if and when time
becomes an option.  My problem with that is it kinda defeats the aim on
why I put th etoolchains up there.  to enable testing accross the range
of architectures in a consistent way.

Another option would clearly be to find someone that is closer to the
kernel to do this thing.

Ho hum thoughts?

> As folks may have seen on LWN, there are now pre-built toolchains for
> many architectures available from the Free Electrons guys, so that may
> help to unblock this situation somewhat:
> 
> http://toolchains.free-electrons.com/

So ubuntu and Fedora each have cross toolschains in the distro and we
that those ^^.  Perhaps this no longer needs to be provided by the
kernel community?

Yours Tony.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]