Re: [PATCH v2 2/5] Makefile.compiler: Use KBUILD_AFLAGS for as-option

From: Nathan Chancellor <nathan@kernel.org>
To: Masahiro Yamada <masahiroy@kernel.org>
Cc: Nick Desaulniers <ndesaulniers@google.com>,
	Michal Marek <michal.lkml@markovi.net>, Tom Rix <trix@redhat.com>,
	Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	clang-built-linux <llvm@lists.linux.dev>, X86 ML <x86@kernel.org>,
	Dmitrii Bundin <dmitrii.bundin.a@gmail.com>,
	Fangrui Song <maskray@google.com>,
	Alexey Alexandrov <aalexand@google.com>,
	Bill Wendling <morbo@google.com>,
	Greg Thelen <gthelen@google.com>
Subject: Re: [PATCH v2 2/5] Makefile.compiler: Use KBUILD_AFLAGS for as-option
Date: Tue, 6 Sep 2022 09:08:12 -0700	[thread overview]
Message-ID: <YxdwbKA5ThYJcPBP@dev-arch.thelio-3990X> (raw)
In-Reply-To: <CAK7LNASQJ-B2kRGXea-dQt+1BgEsp_aLEPS_uJb1R6FSOj1Khg@mail.gmail.com>

On Mon, Sep 05, 2022 at 06:09:28PM +0900, Masahiro Yamada wrote:
> On Thu, Sep 1, 2022 at 4:53 AM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > On Wed, Aug 31, 2022 at 11:44:05AM -0700, Nick Desaulniers wrote:
> > > as-instr uses KBUILD_AFLAGS, but as-option uses KBUILD_CFLAGS.  This can
> > > cause as-option to fail unexpectedly because clang will emit
> > > -Werror,-Wunused-command-line-argument for various -m and -f flags for
> > > assembler sources.
> >
> > Now that I am looking closer at it, where does that '-Werror' come from?
> 
> The related commit is
> c3f0d0bc5b01ad90c45276952802455750444b4f
> 
> The previous discussion with Arnd is
> https://lore.kernel.org/linux-kbuild/20170314213724.3836900-1-arnd@arndb.de/

Right, although this is for cc-option, not as-option, no?

> > For cc-option, we add it to elevate clang's warnings about unused '-f',
> > '-m', and '-W' flags to errors so that we do not add those flags.
> > However, I do not see '-Werror' in as-option. I am going to assume it
> > came from CONFIG_WERROR, as I believe Android has that turned on by
> > default.
> 
> CONFIG_WERROR is added to CFLAGS.
> But, I guess it is more correct to do likewise for others.
> (https://patchwork.kernel.org/project/linux-kbuild/patch/20220905083619.672091-1-masahiroy@kernel.org/)

Ack.

> > I think that is the real problem: without '-Werror', the only
> > error that should come from as-option is when an option isn't supported
> > by the assembler, as clang will still warn but those will not be fatal
> > but with '-Werror', those warnings turn fatal, causing all subsequent
> > as-option calls to fail.
> 
> Presumably, it is correct to add -Werror to as-option as well.
> We have no reason to add it to cc-option, but not to as-option.
> 
> I also believe '-x assembler' should be changed to
> '-x assembler-with-cpp'.
> 
> As I mentioned somewhere before, our assembly code (*.S) is always
> preprocessed. There is no *.s file in the kernel source tree.
> 
> So, '-x assembler-with-cpp' matches the real situation.

All good points, I think that is a good fix as well.

> One interesting thing is, clang does not warn
> [-Wunused-command-line-argument] for *.S files.
> 
> $ clang -fomit-frame-pointer -c -x assembler /dev/null -o /dev/null
> clang: warning: argument unused during compilation:
> '-fomit-frame-pointer' [-Wunused-command-line-argument]
> 
> $ clang -fomit-frame-pointer -c -x assembler-with-cpp /dev/null -o /dev/null

Interesting... I suspect that is likely intentional, as some compiler
flags could be used during preprocessing (it's come up before:
https://github.com/llvm/llvm-project/issues/55656) but I was not able to
figure out exactly where in clang the flags were consumed but Driver.cpp
is quite large and I didn't look too hard :)

More importantly, it still allows us to catch invalid assembler
arguments:

$ clang -c -x assembler-with-cpp /dev/null -o /dev/null -Wa,-foo
clang-16: error: unsupported argument '-foo' to option '-Wa,'

$ clang -c -x assembler-with-cpp /dev/null -o /dev/null -Wa,--noexecstack

> The root cause is we are using '-x assembler', which
> never happens in the kernel tree.
> 
> To sum up, the code I think correct is:
> 
> as-option = $(call try-run,\
>    $(CC) -Werror $(KBUILD_AFLAGS) $(1) -c -x assembler-with-cpp
> /dev/null -o "$$TMP",$(1),$(2))
> 

Agreed. Thank you as always for your feedback!

Cheers,
Nathan