From: Arvind Sankar <nivedita@alum.mit.edu>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: "Arvind Sankar" <nivedita@alum.mit.edu>,
"Rasmus Villemoes" <linux@rasmusvillemoes.dk>,
"Nick Desaulniers" <ndesaulniers@google.com>,
"Dávid Bolvanský" <david.bolvansky@gmail.com>,
"Eli Friedman" <efriedma@quicinc.com>,
"H. Peter Anvin" <hpa@zytor.com>,
"Masahiro Yamada" <masahiroy@kernel.org>,
"Andrew Morton" <akpm@linux-foundation.org>,
"Thomas Gleixner" <tglx@linutronix.de>,
"Ingo Molnar" <mingo@redhat.com>,
"Borislav Petkov" <bp@alien8.de>,
"Michal Marek" <michal.lkml@markovi.net>,
"Linux Kbuild mailing list" <linux-kbuild@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
"Kees Cook" <keescook@chromium.org>,
"Tony Luck" <tony.luck@intel.com>,
"Dmitry Vyukov" <dvyukov@google.com>,
"Michael Ellerman" <mpe@ellerman.id.au>,
"Joe Perches" <joe@perches.com>,
"Joel Fernandes" <joel@joelfernandes.org>,
"Daniel Axtens" <dja@axtens.net>,
"Andy Shevchenko" <andriy.shevchenko@linux.intel.com>,
"Alexandru Ardelean" <alexandru.ardelean@analog.com>,
"Yury Norov" <yury.norov@gmail.com>,
"maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)"
<x86@kernel.org>, "Ard Biesheuvel" <ardb@kernel.org>,
"Paul E . McKenney" <paulmck@kernel.org>,
"Daniel Kiper" <daniel.kiper@oracle.com>,
"Bruce Ashfield" <bruce.ashfield@gmail.com>,
"Marco Elver" <elver@google.com>,
"Vamshi K Sthambamkadi" <vamshi.k.sthambamkadi@gmail.com>
Subject: Re: [PATCH 0/4] -ffreestanding/-fno-builtin-* patches
Date: Fri, 21 Aug 2020 13:29:35 -0400 [thread overview]
Message-ID: <20200821172935.GA1411923@rani.riverdale.lan> (raw)
In-Reply-To: <CAHk-=whn91ar+EbcBXQb9UXad00Q5WjU-TCB6UBzVba682a4ew@mail.gmail.com>
On Thu, Aug 20, 2020 at 04:33:03PM -0700, Linus Torvalds wrote:
> On Thu, Aug 20, 2020 at 10:56 AM Arvind Sankar <nivedita@alum.mit.edu> wrote:
> >
> > Clang's interpretation is more useful for embedded, since you can use
> > -fno-builtin-foo and avoid calling __builtin_foo directly, and be
> > guaranteed that there will be no calls to foo that you didn't write
> > explicitly (outside of memcpy/memset/memcmp). In this case you are free
> > to implement foo with non-standard semantics, or avoid implementing it
> > altogether, and be reasonably confident that it will all work.
>
> Honestly, I think concentrating on whether __builtin_foo() works or
> not misses the point entirely.
>
> That has never _ever_ been a problem for us, and I doubt it has been a
> problem for anybody else either.
>
> If you use __builtin_memcpy() in your source tree, then why would you
> possibly ever want to disable it? And if you _don't_ use it, then
> again - why would you ever want to disable it?
>
> No, the real (and only) problem has always been about the compilers
> magically and silently "recognizing" certain source patterns, and
> turning them into function calls behind the users back.
>
> And that has nothing to do with __builtin_foo(). At least it
> _shouldn't_ have anything to do with it.
>
There seems to be some confusion here. The recognition and
__builtin_foo() go hand-in-hand: memcpy() is special _because_ the
compiler defines it to be __builtin_memcpy(); and the compiler turns the
patterns into __builtin_foo() calls, which just end up as a call to
foo() if they can't be inlined. The no-builtin- options _don't_ disable
__builtin_ functions. They remove the default definition of foo() as
__builtin_foo().
Take the problem that instigated this thread. __builtin_stpcpy() doesn't
work in the kernel because the fallback, stpcpy(), isn't implemented.
The optimization is doing:
sprintf(buf,"%s",s)
-> __builtin_sprintf(buf,"%s",s)
-> __builtin_stpcpy(buf,s)-buf
-> stpcpy(buf,s)-buf
Now, further below, you basically say this is an example of the compiler
taking something non-stpcpy() and turning it into stpcpy(), and you ask
for a no-magic-stpcpy that would stop this optimization. That's what
clang's no-builtin-stpcpy already does. The only extra thing it does is
that the compiler will also not touch an explicit call to stpcpy(), but
you can still call __builtin_stpcpy() if you really want it.
This is what was going on in that LZ4 memcpy() issue: the compiler was
faithfully compiling code like memcpy(d,s,8) into a call to memcpy()
because we told it not to define memcpy() as __builtin_memcpy(), by
compiling for a freestanding environment.
This is why I'm saying clang's no-builtin-foo option is useful for
embedded: it doesn't prevent the programmer using __builtin_foo(), it
prevents the _compiler_ using __builtin_foo() on its own.
> So this is things like the compiler silently seeing "oh, you called
> your function 'free()', so we know that the stores you did to it are
> dead and we'll remove them".
>
> Or this is the compiler doing "Oh, you did four stores of zero in a
> row, and and you asked for size optimizations, so we'll turn those
> into a 'bzero()' call".
This one is slightly different from the previous one. The first case is
really a call to __builtin_free().
This one is turning something that wasn't a function call into
__builtin_bzero(), and I would hope that no-builtin-bzero would stop it
as well. OTOH, the compiler is free to turn it into memset(), just like
it could for structure/array initializers.
The memcpy/memset/memcmp family is a bit of an edge case: the compiler
requires them to be defined even for freestanding environments, so you
can't in general stop the compiler from turning something into memset().
(That -ffreestanding stops gcc from turning loops into memset() is a
pragmatic recognition that some people are going to try to implement
memset() in C.)
>
> Or the compiler doing completely broken things, and turning a
> "!memcmp()" expression into a "!bcmp()" because the compilier
> incorrectly assumes it's faster.
Stop it with the bcmp-shaming already. bcmp _can_ be implemented faster
than memcmp, and it's no great loss if it isn't, since then it'll just
be an alias to memcmp in any sensible libc.
>
> Notice? Not a single one of those had any __builtin_xyz() pattern in
> them. Quite the reverse. The compiler took something completely
> different, and assumed builtin semantics without us having told it to.
>
> So I think "-fno-builtin-xyz" is barking *completely* up the wrong
> tree. It's missing the problem. The problem is not "I have some
> builtin patterns, here, you can use them".
Nope: in a hosted environment, xyz() _is_ __builtin_xyz(), and that is
almost always a good thing for 99% of the code out there: you tell it to
use builtin semantics by choosing to compile for a hosted environment.
If you want something in between freestanding and hosted, you absolutely
need some way to tell the compiler exactly which xyz()'s can be treated
as __builtin_xyz() and which ones shouldn't. The no-builtin- flags allow
you to start from a hosted environment and turn off the specialness of
the functions that you don't want to be special. The proposed builtin-
flags would allow you to start from freestanding and turn on the
specialness of the functions that you do want to be special.
>
> It's the same as all the vector intrinsics. Those don't hurt anybody -
> as long as they only get used when people use the intrinsics. If the
> compiler starts to suddenly use vector intrinsics without being told
> to, *THAT* can be a problem. But there is never any reson to turn off
> any particular intrinsic otherwise.
>
> If you don't want it used, don't use it. And if you do use it, the
> compiler generates the vector code sequence. It's that simple.
>
> So to me, a compiler flag like "-fno-builtin-memcpy" is completely
> insane. The flag adds absolutely no value.
>
> The real value would be "-fno-magic-bcmp" which turns off stupid
> optimizations that magically turn non-bcmp things into bcmp. But it
> should not turn off *actual* __builtin_bcmp() if such a thing exists
> and people want to explicitly use it.
>
> Linus
next prev parent reply other threads:[~2020-08-21 17:30 UTC|newest]
Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-17 22:02 [PATCH 0/4] -ffreestanding/-fno-builtin-* patches Nick Desaulniers
2020-08-17 22:02 ` [PATCH 1/4] Makefile: add -fno-builtin-stpcpy Nick Desaulniers
2020-08-17 22:31 ` H. Peter Anvin
2020-08-17 23:36 ` Nick Desaulniers
2020-08-18 19:21 ` Kees Cook
2020-08-18 7:10 ` Ard Biesheuvel
2020-08-18 7:25 ` Greg KH
2020-08-18 7:29 ` Ard Biesheuvel
2020-08-18 7:34 ` Greg KH
2020-08-18 19:23 ` Kees Cook
2020-08-17 22:02 ` [PATCH 2/4] Revert "lib/string.c: implement a basic bcmp" Nick Desaulniers
2020-08-18 5:44 ` Nathan Chancellor
2020-08-18 18:00 ` Nick Desaulniers
2020-08-18 19:24 ` Kees Cook
2020-08-17 22:02 ` [PATCH 3/4] x86/boot: use -fno-builtin-bcmp Nick Desaulniers
2020-08-18 19:24 ` Kees Cook
2020-08-17 22:02 ` [PATCH 4/4] x86: don't build CONFIG_X86_32 as -ffreestanding Nick Desaulniers
2020-08-18 19:24 ` Kees Cook
2021-01-07 0:27 ` Fangrui Song
2022-04-07 15:34 ` [tip: x86/build] x86/build: Don't " tip-bot2 for Nick Desaulniers
2022-04-07 17:01 ` Nick Desaulniers
2022-04-07 22:28 ` Borislav Petkov
2020-08-17 22:44 ` [PATCH 0/4] -ffreestanding/-fno-builtin-* patches H. Peter Anvin
2020-08-18 17:56 ` Nick Desaulniers
2020-08-18 19:02 ` H. Peter Anvin
2020-08-18 19:13 ` Linus Torvalds
2020-08-18 19:25 ` Nick Desaulniers
2020-08-18 19:58 ` Nick Desaulniers
2020-08-19 12:19 ` Clement Courbet
2020-08-18 20:24 ` Arvind Sankar
2020-08-18 20:27 ` Nick Desaulniers
2020-08-18 20:58 ` Nick Desaulniers
2020-08-18 21:41 ` Arvind Sankar
2020-08-18 21:51 ` Dávid Bolvanský
2020-08-18 21:59 ` Nick Desaulniers
2020-08-18 22:05 ` Dávid Bolvanský
2020-08-18 23:22 ` Nick Desaulniers
2020-08-20 14:56 ` Rasmus Villemoes
2020-08-20 17:56 ` Arvind Sankar
2020-08-20 18:05 ` Dávid Bolvanský
2020-08-20 23:33 ` Linus Torvalds
2020-08-21 17:29 ` Arvind Sankar [this message]
2020-08-21 17:54 ` Linus Torvalds
2020-08-21 18:02 ` Linus Torvalds
2020-08-21 19:14 ` Arvind Sankar
2020-08-21 19:23 ` Linus Torvalds
2020-08-21 19:57 ` Arvind Sankar
2020-08-21 20:03 ` Peter Zijlstra
2020-08-21 21:39 ` Linus Torvalds
2020-08-22 0:12 ` Nick Desaulniers
2020-08-22 12:20 ` David Laight
2020-08-21 6:45 ` Rasmus Villemoes
2020-08-24 15:57 ` Masahiro Yamada
2020-08-24 17:34 ` Arvind Sankar
2020-08-25 7:10 ` Nick Desaulniers
2020-08-25 7:31 ` Nick Desaulniers
2020-08-25 12:28 ` Masahiro Yamada
2020-08-25 14:02 ` Nick Desaulniers
2020-08-26 13:28 ` Masahiro Yamada
2020-08-18 21:53 ` David Laight
2020-08-20 22:41 ` H. Peter Anvin
2020-08-20 23:17 ` Arvind Sankar
2020-08-18 19:35 ` Nick Desaulniers
2020-08-18 22:25 ` Arvind Sankar
2020-08-18 22:59 ` Nick Desaulniers
2020-08-18 23:51 ` Arvind Sankar
2020-08-19 0:20 ` Arvind Sankar
2020-08-19 8:26 ` David Laight
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200821172935.GA1411923@rani.riverdale.lan \
--to=nivedita@alum.mit.edu \
--cc=akpm@linux-foundation.org \
--cc=alexandru.ardelean@analog.com \
--cc=andriy.shevchenko@linux.intel.com \
--cc=ardb@kernel.org \
--cc=bp@alien8.de \
--cc=bruce.ashfield@gmail.com \
--cc=daniel.kiper@oracle.com \
--cc=david.bolvansky@gmail.com \
--cc=dja@axtens.net \
--cc=dvyukov@google.com \
--cc=efriedma@quicinc.com \
--cc=elver@google.com \
--cc=hpa@zytor.com \
--cc=joe@perches.com \
--cc=joel@joelfernandes.org \
--cc=keescook@chromium.org \
--cc=linux-kbuild@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@rasmusvillemoes.dk \
--cc=masahiroy@kernel.org \
--cc=michal.lkml@markovi.net \
--cc=mingo@redhat.com \
--cc=mpe@ellerman.id.au \
--cc=ndesaulniers@google.com \
--cc=paulmck@kernel.org \
--cc=tglx@linutronix.de \
--cc=tony.luck@intel.com \
--cc=torvalds@linux-foundation.org \
--cc=vamshi.k.sthambamkadi@gmail.com \
--cc=x86@kernel.org \
--cc=yury.norov@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).