clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Kees Cook]

On Tue, Feb 13, 2018 at 4:10 PM, David Woodhouse <dwmw2@infradead.org> wrote:
> We also need to resolve the asm-goto thing.

Yes, this is becoming much more urgent, assuming we'll be raising the
minimum GCC version soon and drop support for lacking asm-goto...

Do you happen to know who the right people are to include to move the
discussion forward? I know various kernel folks that are passionate
about it, but I'm still getting to know who to talk with from llvm.

I see an earlier thread here:
http://lists.llvm.org/pipermail/llvm-dev/2017-April/111748.html

It seems to end there? I'm still coming up to speed on it, so I'm
likely missing other context.

-Kees

-- 
Kees Cook
Pixel Security

RE: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Yatsina, Marina]

Hi Kees,

When I raised the question of whether we want to add support for "asm goto" in llvm I got some feedback from the community that "asm goto" might not be the best solution for the problem it was invented for (optimizing support for tracepoints), so I stopped perusing this issue.
I'm CC-ing the developers that participated in the original thread and a few developers that might be interested in adding support of "asm goto".
I'm also adding the llvm-dev mailing list, in case there are additional parties interested in voicing their opinion.

I hope this will give this issue a push forward and we will find a solution that will not prevent llvm from compiling linux kernel.

Thanks,
Marina

-----Original Message-----
From: Kees Cook [mailto:keescook@google.com] 
Sent: Wednesday, February 14, 2018 02:29
To: David Woodhouse <dwmw2@infradead.org>; Chandler Carruth <chandlerc@google.com>; Yatsina, Marina <marina.yatsina@intel.com>
Cc: Guenter Roeck <linux@roeck-us.net>; X86 ML <x86@kernel.org>; LKML <linux-kernel@vger.kernel.org>; Alan Cox <gnomes@lxorguk.ukuu.org.uk>; Rik van Riel <riel@redhat.com>; Andi Kleen <ak@linux.intel.com>; Josh Poimboeuf <jpoimboe@redhat.com>; Tom Lendacky <thomas.lendacky@amd.com>; Peter Zijlstra <peterz@infradead.org>; Linus Torvalds <torvalds@linux-foundation.org>; Jiri Kosina <jikos@kernel.org>; Andy Lutomirski <luto@amacapital.net>; Hansen, Dave <dave.hansen@intel.com>; Tim Chen <tim.c.chen@linux.intel.com>; Greg Kroah-Hartman <gregkh@linux-foundation.org>; Paul Turner <pjt@google.com>; Stephen Hines <srhines@google.com>; Nick Desaulniers <ndesaulniers@google.com>
Subject: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

On Tue, Feb 13, 2018 at 4:10 PM, David Woodhouse <dwmw2@infradead.org> wrote:
> We also need to resolve the asm-goto thing.

Yes, this is becoming much more urgent, assuming we'll be raising the minimum GCC version soon and drop support for lacking asm-goto...

Do you happen to know who the right people are to include to move the discussion forward? I know various kernel folks that are passionate about it, but I'm still getting to know who to talk with from llvm.

I see an earlier thread here:
http://lists.llvm.org/pipermail/llvm-dev/2017-April/111748.html

It seems to end there? I'm still coming up to speed on it, so I'm likely missing other context.

-Kees

--
Kees Cook
Pixel Security
---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Chandler Carruth]

[-- Attachment #1: Type: text/plain, Size: 3524 bytes --]

Marina, Kees:

I think the combination of LKML and the LLVM developer's list is probably
too large a set of folks to have a (somewhat) lengthy exploration of
options and how to go about this and find a good path forward.

Could I suggest a smaller group of interested folks from the Kernel
community and the LLVM community connect (over email, or however else
works) to talk about this instead? I'm happy to participate and help ensure
relevant folks from the LLVM community are involved. Perhaps you can find a
good group from the Linux Kernel community?

On Wed, Feb 14, 2018 at 12:33 AM Yatsina, Marina <marina.yatsina@intel.com>
wrote:

> Hi Kees,
>
> When I raised the question of whether we want to add support for "asm
> goto" in llvm I got some feedback from the community that "asm goto" might
> not be the best solution for the problem it was invented for (optimizing
> support for tracepoints), so I stopped perusing this issue.
> I'm CC-ing the developers that participated in the original thread and a
> few developers that might be interested in adding support of "asm goto".
> I'm also adding the llvm-dev mailing list, in case there are additional
> parties interested in voicing their opinion.
>
> I hope this will give this issue a push forward and we will find a
> solution that will not prevent llvm from compiling linux kernel.
>
> Thanks,
> Marina
>
> -----Original Message-----
> From: Kees Cook [mailto:keescook@google.com]
> Sent: Wednesday, February 14, 2018 02:29
> To: David Woodhouse <dwmw2@infradead.org>; Chandler Carruth <
> chandlerc@google.com>; Yatsina, Marina <marina.yatsina@intel.com>
> Cc: Guenter Roeck <linux@roeck-us.net>; X86 ML <x86@kernel.org>; LKML <
> linux-kernel@vger.kernel.org>; Alan Cox <gnomes@lxorguk.ukuu.org.uk>; Rik
> van Riel <riel@redhat.com>; Andi Kleen <ak@linux.intel.com>; Josh
> Poimboeuf <jpoimboe@redhat.com>; Tom Lendacky <thomas.lendacky@amd.com>;
> Peter Zijlstra <peterz@infradead.org>; Linus Torvalds <
> torvalds@linux-foundation.org>; Jiri Kosina <jikos@kernel.org>; Andy
> Lutomirski <luto@amacapital.net>; Hansen, Dave <dave.hansen@intel.com>;
> Tim Chen <tim.c.chen@linux.intel.com>; Greg Kroah-Hartman <
> gregkh@linux-foundation.org>; Paul Turner <pjt@google.com>; Stephen Hines
> <srhines@google.com>; Nick Desaulniers <ndesaulniers@google.com>
> Subject: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add
> clang support)
>
> On Tue, Feb 13, 2018 at 4:10 PM, David Woodhouse <dwmw2@infradead.org>
> wrote:
> > We also need to resolve the asm-goto thing.
>
> Yes, this is becoming much more urgent, assuming we'll be raising the
> minimum GCC version soon and drop support for lacking asm-goto...
>
> Do you happen to know who the right people are to include to move the
> discussion forward? I know various kernel folks that are passionate about
> it, but I'm still getting to know who to talk with from llvm.
>
> I see an earlier thread here:
> http://lists.llvm.org/pipermail/llvm-dev/2017-April/111748.html
>
> It seems to end there? I'm still coming up to speed on it, so I'm likely
> missing other context.
>
> -Kees
>
> --
> Kees Cook
> Pixel Security
> ---------------------------------------------------------------------
> Intel Israel (74) Limited
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>

[-- Attachment #2: Type: text/html, Size: 5563 bytes --]

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Peter Zijlstra]

On Wed, Feb 14, 2018 at 08:33:39AM +0000, Yatsina, Marina wrote:
> I hope this will give this issue a push forward and we will find a
> solution that will not prevent llvm from compiling linux kernel.

That solution is asm-goto. We (linux kernel) are not going to support
something like it. It's going to be asm-goto or bust. The feature is
over 7 years old by now, deal with it.

If you ever get to the point where it makes sense to build a kernel with
LLVM, then you can bike-shed with the GCC team on new features, until
then tough luck.

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Peter Zijlstra]

On Wed, Feb 14, 2018 at 08:41:43AM +0000, Chandler Carruth wrote:
> Marina, Kees:
> 
> I think the combination of LKML and the LLVM developer's list is probably
> too large a set of folks to have a (somewhat) lengthy exploration of
> options and how to go about this and find a good path forward.

As one of the original authors of the jump-label implementation I'm
saying there are no options. We're not going to do a second parallel
implementation. If you want to build the kernel, you get to implement
asm-goto, teh end.

RE: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Yatsina, Marina]

Hi Peter,

When I started the original thread last year I was in favor of adding "asm goto" and didn't understand why it wasn't done by that time.
The feedback I got is that this feature (optimizing tracepoints) is very useful and that we do want it in llvm, but perhaps there's a cleaner way of implementing than "asm goto". An alternative suggestion arose as well. 

I'm sure you can provide a lot of background for the decisions of why "asm goto" was chosen and which other alternatives were considered, as you were the one to implement this. This may shed a lot of light on the alternative suggestions that arose in my thread in the llvm community.
I think that including GCC folks in the "small team" discussion Chandler suggested might be a very good idea, so if you have some contacts, please send them to Chandler.

Anyway, I think we should consider the alternatives and not take "asm goto" as a given, even though this is the implementation of this feature for the past 7 years and it works well.
I'm not saying linux kernel needs to rush and adopt an alternative solution ASAP, but we cannot ignore that it exists and may (or may not) be the direction of the future (for both gcc and llvm).

Thanks,
Marina

-----Original Message-----
From: Peter Zijlstra [mailto:peterz@infradead.org] 
Sent: Wednesday, February 14, 2018 11:09
To: Yatsina, Marina <marina.yatsina@intel.com>
Cc: Kees Cook <keescook@google.com>; David Woodhouse <dwmw2@infradead.org>; Chandler Carruth <chandlerc@google.com>; Kreitzer, David L <david.l.kreitzer@intel.com>; Grischenko, Andrei L <andrei.l.grischenko@intel.com>; rnk@google.com; LLVM Developers <llvm-dev@lists.llvm.org>; ehsan@mozilla.com; Tayree, Coby <coby.tayree@intel.com>; Matthias Braun <matze@braunis.de>; Dean Michael Berris <dean.berris@gmail.com>; James Y Knight <jyknight@google.com>; Guenter Roeck <linux@roeck-us.net>; X86 ML <x86@kernel.org>; LKML <linux-kernel@vger.kernel.org>; Alan Cox <gnomes@lxorguk.ukuu.org.uk>; Rik van Riel <riel@redhat.com>; Andi Kleen <ak@linux.intel.com>; Josh Poimboeuf <jpoimboe@redhat.com>; Tom Lendacky <thomas.lendacky@amd.com>; Linus Torvalds <torvalds@linux-foundation.org>; Jiri Kosina <jikos@kernel.org>; Andy Lutomirski <luto@amacapital.net>; Hansen, Dave <dave.hansen@intel.com>; Tim Chen <tim.c.chen@linux.intel.com>; Greg Kroah-Hartman <gregkh@linux-foundation.org>; Paul Turner <pjt@google.com>; Stephen Hines <srhines@google.com>; Nick Desaulniers <ndesaulniers@google.com>
Subject: Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

On Wed, Feb 14, 2018 at 08:33:39AM +0000, Yatsina, Marina wrote:
> I hope this will give this issue a push forward and we will find a 
> solution that will not prevent llvm from compiling linux kernel.

That solution is asm-goto. We (linux kernel) are not going to support something like it. It's going to be asm-goto or bust. The feature is over 7 years old by now, deal with it.

If you ever get to the point where it makes sense to build a kernel with LLVM, then you can bike-shed with the GCC team on new features, until then tough luck.
---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Greg Kroah-Hartman]

On Wed, Feb 14, 2018 at 08:41:43AM +0000, Chandler Carruth wrote:
> Marina, Kees:
> 
> I think the combination of LKML and the LLVM developer's list is probably
> too large a set of folks to have a (somewhat) lengthy exploration of
> options and how to go about this and find a good path forward.

Why is there any need to discuss "options" here?  What is preventing the
implementation of "asm goto" in llvm?  Is it the lack of a working
patch, or something else?

thanks,

greg k-h

RE: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Yatsina, Marina]

I believe we do have the infrastructure in llvm to implement a working patch.
The "asm goto" was introduced as a solution to a problem (optimizing tracepoints/ nop patching).
When we started the discussion in llvm about adding this feature, questions arose if we can design a better solution for this problem than "asm goto".

You can find a more about the discussion in the llvm community here:
http://lists.llvm.org/pipermail/llvm-dev/2017-April/111748.html

* This is the middle of the thread, and there are more messages later on.

Thanks,
Marina

-----Original Message-----
From: Greg Kroah-Hartman [mailto:gregkh@linux-foundation.org] 
Sent: Wednesday, February 14, 2018 11:53
To: Chandler Carruth <chandlerc@google.com>
Cc: Yatsina, Marina <marina.yatsina@intel.com>; Kees Cook <keescook@google.com>; David Woodhouse <dwmw2@infradead.org>; Kreitzer, David L <david.l.kreitzer@intel.com>; Grischenko, Andrei L <andrei.l.grischenko@intel.com>; rnk@google.com; LLVM Developers <llvm-dev@lists.llvm.org>; ehsan@mozilla.com; Tayree, Coby <coby.tayree@intel.com>; Matthias Braun <matze@braunis.de>; Dean Michael Berris <dean.berris@gmail.com>; James Y Knight <jyknight@google.com>; Guenter Roeck <linux@roeck-us.net>; X86 ML <x86@kernel.org>; LKML <linux-kernel@vger.kernel.org>; Alan Cox <gnomes@lxorguk.ukuu.org.uk>; Rik van Riel <riel@redhat.com>; Andi Kleen <ak@linux.intel.com>; Josh Poimboeuf <jpoimboe@redhat.com>; Tom Lendacky <thomas.lendacky@amd.com>; Peter Zijlstra <peterz@infradead.org>; Linus Torvalds <torvalds@linux-foundation.org>; Jiri Kosina <jikos@kernel.org>; Andy Lutomirski <luto@amacapital.net>; Hansen, Dave <dave.hansen@intel.com>; Tim Chen <tim.c.chen@linux.intel.com>; Paul Turner <pjt@google.com>; Stephen Hines <srhines@google.com>; Nick Desaulniers <ndesaulniers@google.com>
Subject: Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

On Wed, Feb 14, 2018 at 08:41:43AM +0000, Chandler Carruth wrote:
> Marina, Kees:
> 
> I think the combination of LKML and the LLVM developer's list is 
> probably too large a set of folks to have a (somewhat) lengthy 
> exploration of options and how to go about this and find a good path forward.

Why is there any need to discuss "options" here?  What is preventing the implementation of "asm goto" in llvm?  Is it the lack of a working patch, or something else?

thanks,

greg k-h
---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Chandler Carruth]

[-- Attachment #1: Type: text/plain, Size: 3567 bytes --]

On Wed, Feb 14, 2018 at 2:11 AM Yatsina, Marina <marina.yatsina@intel.com>
wrote:

> I believe we do have the infrastructure in llvm to implement a working
> patch.
> The "asm goto" was introduced as a solution to a problem (optimizing
> tracepoints/ nop patching).
> When we started the discussion in llvm about adding this feature,
> questions arose if we can design a better solution for this problem than
> "asm goto".
>
> You can find a more about the discussion in the llvm community here:
> http://lists.llvm.org/pipermail/llvm-dev/2017-April/111748.html
>
> * This is the middle of the thread, and there are more messages later on.
>

FWIW, there is very strong interest in the LLVM community in working on a
feature along the lines of what John McCall suggested (as a significant
improvement to my initial idea) later in the thread:
http://lists.llvm.org/pipermail/llvm-dev/2017-April/111755.html

If this feature as-is or with additions / suggestions / improvements would
address the kernel's needs, we're all over it.

I think the concerns are already in that thread...

But as I said in my original email, if a different approach is needed I'm
happy to try and get folks to understand what would be needed and where a
reasonable compromise is...


>
> Thanks,
> Marina
>
> -----Original Message-----
> From: Greg Kroah-Hartman [mailto:gregkh@linux-foundation.org]
> Sent: Wednesday, February 14, 2018 11:53
> To: Chandler Carruth <chandlerc@google.com>
> Cc: Yatsina, Marina <marina.yatsina@intel.com>; Kees Cook <
> keescook@google.com>; David Woodhouse <dwmw2@infradead.org>; Kreitzer,
> David L <david.l.kreitzer@intel.com>; Grischenko, Andrei L <
> andrei.l.grischenko@intel.com>; rnk@google.com; LLVM Developers <
> llvm-dev@lists.llvm.org>; ehsan@mozilla.com; Tayree, Coby <
> coby.tayree@intel.com>; Matthias Braun <matze@braunis.de>; Dean Michael
> Berris <dean.berris@gmail.com>; James Y Knight <jyknight@google.com>;
> Guenter Roeck <linux@roeck-us.net>; X86 ML <x86@kernel.org>; LKML <
> linux-kernel@vger.kernel.org>; Alan Cox <gnomes@lxorguk.ukuu.org.uk>; Rik
> van Riel <riel@redhat.com>; Andi Kleen <ak@linux.intel.com>; Josh
> Poimboeuf <jpoimboe@redhat.com>; Tom Lendacky <thomas.lendacky@amd.com>;
> Peter Zijlstra <peterz@infradead.org>; Linus Torvalds <
> torvalds@linux-foundation.org>; Jiri Kosina <jikos@kernel.org>; Andy
> Lutomirski <luto@amacapital.net>; Hansen, Dave <dave.hansen@intel.com>;
> Tim Chen <tim.c.chen@linux.intel.com>; Paul Turner <pjt@google.com>;
> Stephen Hines <srhines@google.com>; Nick Desaulniers <
> ndesaulniers@google.com>
> Subject: Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add
> clang support)
>
> On Wed, Feb 14, 2018 at 08:41:43AM +0000, Chandler Carruth wrote:
> > Marina, Kees:
> >
> > I think the combination of LKML and the LLVM developer's list is
> > probably too large a set of folks to have a (somewhat) lengthy
> > exploration of options and how to go about this and find a good path
> forward.
>
> Why is there any need to discuss "options" here?  What is preventing the
> implementation of "asm goto" in llvm?  Is it the lack of a working patch,
> or something else?
>
> thanks,
>
> greg k-h
> ---------------------------------------------------------------------
> Intel Israel (74) Limited
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
>

[-- Attachment #2: Type: text/html, Size: 6367 bytes --]

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Peter Zijlstra]

On Wed, Feb 14, 2018 at 09:52:59AM +0000, Yatsina, Marina wrote:
> Hi Peter,
> 
> When I started the original thread last year I was in favor of adding
> "asm goto" and didn't understand why it wasn't done by that time.  The
> feedback I got is that this feature (optimizing tracepoints) is very
> useful and that we do want it in llvm, but perhaps there's a cleaner
> way of implementing than "asm goto". An alternative suggestion arose
> as well. 

So it's far more than just tracepoints. We use it all over the kernel to
do runtime branch patching.

One example is avoiding the scheduler preemption callbacks if we know
there are no users. This shaves a few % off a context switch
micro-bench.

But it is really _all_ over the place.

> I'm sure you can provide a lot of background for the decisions of why
> "asm goto" was chosen and which other alternatives were considered, as
> you were the one to implement this.

I have very little memories from back then, but it was mostly us asking
for label addresses in asm and them giving us asm-goto.

Using asm we can build our own primitives, and I realize the llvm
community doesn't like asm much, but then again, we treat C like a
glorified assembler and don't like our compilers too smart :-)

> Anyway, I think we should consider the alternatives and not take "asm
> goto" as a given.

Far too late for that, 7+ years ago when we did this was the time to
talk about alternatives, now we have this code base.

So we have the two jump_label things:

static __always_inline bool arch_static_branch(struct static_key *key, bool branch)
{
        asm_volatile_goto("1:"
                ".byte " __stringify(STATIC_KEY_INIT_NOP) "\n\t"
                ".pushsection __jump_table,  \"aw\" \n\t"
                _ASM_ALIGN "\n\t"
                _ASM_PTR "1b, %l[l_yes], %c0 + %c1 \n\t"
                ".popsection \n\t"
                : :  "i" (key), "i" (branch) : : l_yes);

        return false;
l_yes:
        return true;
}

static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch)
{
        asm_volatile_goto("1:"
                ".byte 0xe9\n\t .long %l[l_yes] - 2f\n\t"
                "2:\n\t"
                ".pushsection __jump_table,  \"aw\" \n\t"
                _ASM_ALIGN "\n\t"
                _ASM_PTR "1b, %l[l_yes], %c0 + %c1 \n\t"
                ".popsection \n\t"
                : :  "i" (key), "i" (branch) : : l_yes);

        return false;
l_yes:
        return true;
}

Where we emit either a 5 byte jump or a 5 byte nop and write a special
section with meta-data for the branch point.

You could possibly capture all that with a built-in, but would have to
exactly match our meta-data section and then we'd still be up some creek
without no paddle when we need to change it.

But we also have:

static __always_inline __pure bool _static_cpu_has(u16 bit)
{
        asm_volatile_goto("1: jmp 6f\n"
                 "2:\n"
                 ".skip -(((5f-4f) - (2b-1b)) > 0) * "
                         "((5f-4f) - (2b-1b)),0x90\n"
                 "3:\n"
                 ".section .altinstructions,\"a\"\n"
                 " .long 1b - .\n"              /* src offset */
                 " .long 4f - .\n"              /* repl offset */
                 " .word %P[always]\n"          /* always replace */
                 " .byte 3b - 1b\n"             /* src len */
                 " .byte 5f - 4f\n"             /* repl len */
                 " .byte 3b - 2b\n"             /* pad len */
                 ".previous\n"
                 ".section .altinstr_replacement,\"ax\"\n"
                 "4: jmp %l[t_no]\n"
                 "5:\n"
                 ".previous\n"
                 ".section .altinstructions,\"a\"\n"
                 " .long 1b - .\n"              /* src offset */
                 " .long 0\n"                   /* no replacement */
                 " .word %P[feature]\n"         /* feature bit */
                 " .byte 3b - 1b\n"             /* src len */
                 " .byte 0\n"                   /* repl len */
                 " .byte 0\n"                   /* pad len */
                 ".previous\n"
                 ".section .altinstr_aux,\"ax\"\n"
                 "6:\n"
                 " testb %[bitnum],%[cap_byte]\n"
                 " jnz %l[t_yes]\n"
                 " jmp %l[t_no]\n"
                 ".previous\n"
                 : : [feature]  "i" (bit),
                     [always]   "i" (X86_FEATURE_ALWAYS),
                     [bitnum]   "i" (1 << (bit & 7)),
                     [cap_byte] "m" (((const char *)boot_cpu_data.x86_capability)[bit >> 3])
                 : : t_yes, t_no);
t_yes:
        return true;
t_no:
        return false;
}

Which does something similar, but with a completely different meta-data
section and a different pre-patch fallback path.

But we also do things like:

#define __GEN_RMWcc(fullop, var, cc, clobbers, ...)                     \
do {                                                                    \
        asm_volatile_goto (fullop "; j" #cc " %l[cc_label]"             \
                        : : [counter] "m" (var), ## __VA_ARGS__         \
                        : clobbers : cc_label);                         \
        return 0;                                                       \
cc_label:                                                               \
        return 1;                                                       \
} while (0)

#define GEN_UNARY_RMWcc(op, var, arg0, cc)				\
	__GEN_RMWcc(op " " arg0, var, cc, __CLOBBERS_MEM)

static __always_inline bool atomic_dec_and_test(atomic_t *v)
{
	GEN_UNARY_RMWcc(LOCK_PREFIX "decl", v->counter, "%0", e);
}

In order to not generate crap asm with SETcc + TEST. Of course, the last
is superceded with asm-cc-output, which you _also_ don't support.

And I know you're going to tell me you guys would prefer it if we
switched to intrinsics for atomics, but then I'd have to tell you that
the C11 memory model doesn't match the Linux Kernel memory model [*],
another result of being late to the game. Also, we still support
compilers from before that.

So no, you're not going to give us something different.


[*]  http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0124r4.html

So ideally, the compilers would actually commit to also implementing the
linux-kernel memory model, otherwise we'll be fighting the compiler
(like we have been for a while now) for even longer.

Esp. with LTO we run a real risk of the compiler doing BAD things to our
code.

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Thomas Gleixner]

On Wed, 14 Feb 2018, Peter Zijlstra wrote:
> On Wed, Feb 14, 2018 at 08:41:43AM +0000, Chandler Carruth wrote:
> > Marina, Kees:
> > 
> > I think the combination of LKML and the LLVM developer's list is probably
> > too large a set of folks to have a (somewhat) lengthy exploration of
> > options and how to go about this and find a good path forward.
> 
> As one of the original authors of the jump-label implementation I'm
> saying there are no options. We're not going to do a second parallel
> implementation. If you want to build the kernel, you get to implement
> asm-goto, teh end.

Amen to that.

As x86 maintainer I'm really interested and willing to support llvm/clang
for x86 and the kernel in general, but asm-goto is not longer negotiable.

asm-goto is in the kernel since Oct 2010 and the relevant llvm/clang bug
#9295 is open since Feb 2011. All clang/llvm folks came up with in those 7
years is occasional handwaving about better solutions.

We really try to support old tools and tools which lack features for quite
a long time, but that's not a free pass for eternity.

That said, I'm not willing to wait another 7 years and I'm going to pull
the plug on compilers w/o asm-goto support soon whether llvm/clang gets
their act together or not.

Hint: GCC is not the only major compiler which supports asm-goto.

Thanks,

	tglx

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Ingo Molnar]

* Peter Zijlstra <peterz@infradead.org> wrote:

> On Wed, Feb 14, 2018 at 09:52:59AM +0000, Yatsina, Marina wrote:
> > Hi Peter,
> > 
> > When I started the original thread last year I was in favor of adding
> > "asm goto" and didn't understand why it wasn't done by that time.  The
> > feedback I got is that this feature (optimizing tracepoints) is very
> > useful and that we do want it in llvm, but perhaps there's a cleaner
> > way of implementing than "asm goto". An alternative suggestion arose
> > as well. 
> 
> So it's far more than just tracepoints. We use it all over the kernel to
> do runtime branch patching.
> 
> One example is avoiding the scheduler preemption callbacks if we know
> there are no users. This shaves a few % off a context switch
> micro-bench.
> 
> But it is really _all_ over the place.

To quantify it: I just performed a test build of a Linux distro kernel config 
(Fedora x86-64), and counted the number of callsites that use 'asm goto' 
functionality with the v4.15 kernel (including drivers).

The results:

                                                Linux distro | !CONFIG_TRACING
 -----------------------------------------------------------------------------
 total # of functions                         :      191,567 |         184,443
 total # of instructions                      :   14,251,355 |      13,526,112
 -----------------------------------------------------------------------------
 total # of spin_lock*() calls                :       25,246 |          25,177
 total # of mutex_lock*() calls               :       13,062 |          12,861
 total # of kmalloc*() calls                  :        5,148 |           5,118
 -----------------------------------------------------------------------------
 total # of 'asm goto' usage sites            :       34,851 |          31,059
 total # of 'asm goto' using functions        :       18,209 |          16,089
 -----------------------------------------------------------------------------
 percent of kernel functions using 'asm goto' :         9.5% |            8.7%
 -----------------------------------------------------------------------------


( Note: I added the 'tracing disabled' column only to demonstrate how widely
        'asm goto' is used beyond tracing - but since *all* major Linux 
        distributions have tracing enabled it's the first column that matters in 
        practice. )

So 'asm goto' is a major, major compiler feature the Linux kernel relies on:
in fact in the Linux kernel there's more 'asm goto' usage sites than there are
spin-lock critical sections!

Thanks,

	Ingo

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[James Y Knight]

I'd be definitely in favor having clang support asm goto. I wouldn't
want to exclude having other conversations about how to more directly
provide compiler features that the linux kernel could use, ALSO, but I
do not think that conversation should block implementing asm-goto.

IMO, inline asm is, generally, a valuable feature to provide in the
compiler as an escape hatch, and asm goto is a relatively sane
extension of it. Supporting outgoing edges from an inline asm block is
a reasonable thing for users to desire, and as far as anyone's said so
far, seems like it ought to be fairly easily implementable, without
causing bad side-effects in the compiler.

Of course, we generally do want to minimize the need for users to use
inline asm, by providing appropriate compiler support for the features
people would otherwise be forced to implement using asm. But I don't
see that as really any more important for asm-goto than any other
inline asm. There will always be a desire for escape hatches, to do
weird and unique things which aren't supported directly in the
compiler. (Also, the kernel is a pretty special case in terms of its
requirements, it seems exceedingly unlikely that we could ever provide
enough compiler support to let it eliminate inline asm.).

On Wed, Feb 14, 2018 at 3:33 AM, Yatsina, Marina
<marina.yatsina@intel.com> wrote:
>
> Hi Kees,
>
> When I raised the question of whether we want to add support for "asm goto" in llvm I got some feedback from the community that "asm goto" might not be the best solution for the problem it was invented for (optimizing support for tracepoints), so I stopped perusing this issue.
> I'm CC-ing the developers that participated in the original thread and a few developers that might be interested in adding support of "asm goto".
> I'm also adding the llvm-dev mailing list, in case there are additional parties interested in voicing their opinion.
>
> I hope this will give this issue a push forward and we will find a solution that will not prevent llvm from compiling linux kernel.
>
> Thanks,
> Marina
>
> -----Original Message-----
> From: Kees Cook [mailto:keescook@google.com]
> Sent: Wednesday, February 14, 2018 02:29
> To: David Woodhouse <dwmw2@infradead.org>; Chandler Carruth <chandlerc@google.com>; Yatsina, Marina <marina.yatsina@intel.com>
> Cc: Guenter Roeck <linux@roeck-us.net>; X86 ML <x86@kernel.org>; LKML <linux-kernel@vger.kernel.org>; Alan Cox <gnomes@lxorguk.ukuu.org.uk>; Rik van Riel <riel@redhat.com>; Andi Kleen <ak@linux.intel.com>; Josh Poimboeuf <jpoimboe@redhat.com>; Tom Lendacky <thomas.lendacky@amd.com>; Peter Zijlstra <peterz@infradead.org>; Linus Torvalds <torvalds@linux-foundation.org>; Jiri Kosina <jikos@kernel.org>; Andy Lutomirski <luto@amacapital.net>; Hansen, Dave <dave.hansen@intel.com>; Tim Chen <tim.c.chen@linux.intel.com>; Greg Kroah-Hartman <gregkh@linux-foundation.org>; Paul Turner <pjt@google.com>; Stephen Hines <srhines@google.com>; Nick Desaulniers <ndesaulniers@google.com>
> Subject: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
>
> On Tue, Feb 13, 2018 at 4:10 PM, David Woodhouse <dwmw2@infradead.org> wrote:
> > We also need to resolve the asm-goto thing.
>
> Yes, this is becoming much more urgent, assuming we'll be raising the minimum GCC version soon and drop support for lacking asm-goto...
>
> Do you happen to know who the right people are to include to move the discussion forward? I know various kernel folks that are passionate about it, but I'm still getting to know who to talk with from llvm.
>
> I see an earlier thread here:
> http://lists.llvm.org/pipermail/llvm-dev/2017-April/111748.html
>
> It seems to end there? I'm still coming up to speed on it, so I'm likely missing other context.
>
> -Kees
>
> --
> Kees Cook
> Pixel Security
> ---------------------------------------------------------------------
> Intel Israel (74) Limited
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Andy Lutomirski]

On Wed, Feb 14, 2018 at 3:31 PM, James Y Knight <jyknight@google.com> wrote:
> I'd be definitely in favor having clang support asm goto. I wouldn't
> want to exclude having other conversations about how to more directly
> provide compiler features that the linux kernel could use, ALSO, but I
> do not think that conversation should block implementing asm-goto.
>
> IMO, inline asm is, generally, a valuable feature to provide in the
> compiler as an escape hatch, and asm goto is a relatively sane
> extension of it. Supporting outgoing edges from an inline asm block is
> a reasonable thing for users to desire, and as far as anyone's said so
> far, seems like it ought to be fairly easily implementable, without
> causing bad side-effects in the compiler.
>
> Of course, we generally do want to minimize the need for users to use
> inline asm, by providing appropriate compiler support for the features
> people would otherwise be forced to implement using asm. But I don't
> see that as really any more important for asm-goto than any other
> inline asm. There will always be a desire for escape hatches, to do
> weird and unique things which aren't supported directly in the
> compiler. (Also, the kernel is a pretty special case in terms of its
> requirements, it seems exceedingly unlikely that we could ever provide
> enough compiler support to let it eliminate inline asm.).
>

If clang implements asm goto, could you also pretty please make it
better than GCC and support output constraints in the same asm
statements?  This would give us a very clean, short, and fast
implementation for __get_user() and __put_user(), and it would also
help put pressure on GCC to implement the same feature.

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Kees Cook]

On Wed, Feb 14, 2018 at 1:08 AM, Peter Zijlstra <peterz@infradead.org> wrote:
> If you ever get to the point where it makes sense to build a kernel with
> LLVM [...]

Just to speak to this point in particular: it makes sense to build
kernels with LLVM right now. It's already happened; millions of Pixel
2 phones are already running with Clang-built kernels. And even if
you're only interested in x86, that works too.

-Kees

-- 
Kees Cook
Pixel Security

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Linus Torvalds]

On Wed, Feb 14, 2018 at 2:34 AM, Peter Zijlstra <peterz@infradead.org> wrote:
>
> So it's far more than just tracepoints. We use it all over the kernel to
> do runtime branch patching.

Side note: I have a patch to the user access code to do "asm goto" for
the exception handling too.

In fact, the infrastructure is there already, but because we supported
older gcc versions that didn't have asm goto, I never applied the
final patch to actually make use of it.

See "unsafe_put_user()", which currently does

        if (unlikely(__pu_err)) goto err_label;

rather than just have the exception table itself directly reference
"err_label", and not needing any code at all in the regular path.

I suspect there are other things like that, where the lack of "asm
goto" means that it's not worth the pain of having two code paths.
Once we update to a newer gcc requirement, those suddenly will also
very naturally use asm goto.

The other asm use that the clang people might want to look at is the
"condition codes as return value" one. That one we currently handle
with CC_SET/CC_OUT, and it's somewhat more amenable to those kinds of
games, so missing it isn't as nasty. But it *is* a useful "modern"
extension of gcc asms.

Maybe clang already supports cflags output, I haven't checked.

              Linus

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Linus Torvalds]

On Wed, Feb 14, 2018 at 7:31 AM, James Y Knight <jyknight@google.com> wrote:
>
> IMO, inline asm is, generally, a valuable feature to provide in the
> compiler as an escape hatch, and asm goto is a relatively sane
> extension of it.

Side note: one thing that limits "asm goto" in gcc is the fact that
you can't have outputs.

If clang people are looking at "asm goto", the _syntax_ is actually
very straightforward, and there's really absolutely no point in trying
to make up some other interface.

But extending on what gcc does, and allowing outputs (possibly valid
in the fall-through case only, not in the cases where it jumps away to
a label) would be a big improvement on what gcc does.

At that point, we'd start to pester the gcc people to do the clang
thing, rather than the other way around. Because that's only fair ;)

                  Linus

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Peter Zijlstra]

On Wed, Feb 14, 2018 at 09:31:22AM -0800, Kees Cook wrote:
> On Wed, Feb 14, 2018 at 1:08 AM, Peter Zijlstra <peterz@infradead.org> wrote:
> > If you ever get to the point where it makes sense to build a kernel with
> > LLVM [...]
> 
> Just to speak to this point in particular: it makes sense to build
> kernels with LLVM right now. It's already happened; millions of Pixel
> 2 phones are already running with Clang-built kernels. And even if
> you're only interested in x86, that works too.

Only if you really don't care about performance. And as should be
evident from the rest of this thread, you soon won't be able to build
it anymore.

Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)

[Ingo Molnar]

* Ingo Molnar <mingo@kernel.org> wrote:

> To quantify it: I just performed a test build of a Linux distro kernel config 
> (Fedora x86-64), and counted the number of callsites that use 'asm goto' 
> functionality with the v4.15 kernel (including drivers).
> 
> The results:
> 
>                                                 Linux distro | !CONFIG_TRACING
>  -----------------------------------------------------------------------------
>  total # of functions                         :      191,567 |         184,443
>  total # of instructions                      :   14,251,355 |      13,526,112
>  -----------------------------------------------------------------------------
>  total # of spin_lock*() calls                :       25,246 |          25,177
>  total # of mutex_lock*() calls               :       13,062 |          12,861
>  total # of kmalloc*() calls                  :        5,148 |           5,118
>  -----------------------------------------------------------------------------
>  total # of 'asm goto' usage sites            :       34,851 |          31,059
>  total # of 'asm goto' using functions        :       18,209 |          16,089
>  -----------------------------------------------------------------------------
>  percent of kernel functions using 'asm goto' :         9.5% |            8.7%
>  -----------------------------------------------------------------------------

Here's the size stats of kernel/sched/built-in.o for the same distro config:

                                                     optimized |     no asm goto
   -----------------------------------------------------------------------------
   total # of functions                         :          765 |            764
   total # of instructions                      :       46,830 |         47,051

I.e. asm goto support reduces scheduler size by ~0.5%, which is a major generated 
code size reduction.

This doesn't count the live branch patching performance advantages: many of those 
asm goto usage sites are in hot paths, so the performance impact of it is much 
larger than that: easily a couple of percentage points in scheduler intensive 
benchmarks, as Peter mentioned.

For example here's a thread context switch benchmark comparison on a modern x86 
system running a v4.15 kernel:

  $ perf stat --repeat 20 --sync --null perf bench sched messaging -t -g 25

         no asm goto:     0.136778505 seconds time elapsed      ( stddev: +- 0.55% )
  asm goto optimized:     0.133773904 seconds time elapsed      ( stddev: +- 0.51% )

The asm goto enabled kernel is ~2.25% faster in this benchmark, and the 
performance penalty of not having asm goto support will only increase in the 
future.

i.e. it very much makes sense to implement asm goto support not just for 
compatibility reasons, but for performance reasons as well.

Thanks,

	Ingo