Plumbers CF MCs

Plumbers CF MCs

[Nick Desaulniers]

Hi all,
I saw plumbers opened call for microconferences:
https://www.linuxplumbersconf.org/blog/2021/index.php/2021/03/18/cfp-open-microconferences/

I was going to put together a submission; do we want to do a combined
toolchain MC, or have distinct ones this year?

I know in 2020 the GNU cauldron was co-located with Plumbers, as well
as a GNU Tools Track MC and LLVM MC.
-- 
Thanks,
~Nick Desaulniers

Re: Plumbers CF MCs

[Jose E. Marchesi]

Hi Nick.

> I saw plumbers opened call for microconferences:
> https://www.linuxplumbersconf.org/blog/2021/index.php/2021/03/18/cfp-open-microconferences/
>
> I was going to put together a submission; do we want to do a combined
> toolchain MC, or have distinct ones this year?
>
> I know in 2020 the GNU cauldron was co-located with Plumbers, as well
> as a GNU Tools Track MC and LLVM MC.

We are actually discussing in another thread about abusing LPC's
hospitality for another GNU Tools Track this year...

Regarding the micro-conferences, I would be ok with either combined or
separated.  I think both approaches have their advantages.

In either case I'm up for organizing the GNU part.

Re: Plumbers CF MCs

[Peter Zijlstra]

On Mon, Mar 22, 2021 at 01:23:03PM -0700, Nick Desaulniers wrote:
> Hi all,
> I saw plumbers opened call for microconferences:
> https://www.linuxplumbersconf.org/blog/2021/index.php/2021/03/18/cfp-open-microconferences/
> 
> I was going to put together a submission; do we want to do a combined
> toolchain MC, or have distinct ones this year?
> 
> I know in 2020 the GNU cauldron was co-located with Plumbers, as well
> as a GNU Tools Track MC and LLVM MC.

A combined MC focussed on kernel issues seems very interesting. We still
have the control dependency (volatile-if?) thing pending. We had a bit
of a discussion on that after last year, but I don't think anything
really came of that, can we pick that up? Ideally a compiler person does
an actual proposal for this year.

If we can sort that, there's the rest of the dependencies Will outlined
:-)

Then there seemed to be people that thought __always_inline was a
suggestion... I think we need to explore how that was ever possible.

There's the endless UB debate... can we define more to reduce the UB? I
mean, we're already bound by architecture ABI on the one hand, and
actual use on the other. It would be so very nice to be able to get more
-fwrapv and -fno-strict-aliasing knobs that define UBs away.

There also is talk about straight line speculation mitigations. for x86
we should probably emit an INT3 after every JMP and RET. Although this
might not be controversial and be sorted by the time Plumbers happens.

There was some talk about how compilers could help objtool make sense of
jump tables.

GCC's status on asm-goto with outputs?

Clang's getting asm-constraints wrong ("rm" and it always picks "m").


And I'm sure there was more..

Re: Plumbers CF MCs

[Peter Zijlstra]

On Tue, Mar 23, 2021 at 09:35:10AM +0100, Peter Zijlstra wrote:
> There's the endless UB debate... can we define more to reduce the UB? I
> mean, we're already bound by architecture ABI on the one hand, and
> actual use on the other. It would be so very nice to be able to get more
> -fwrapv and -fno-strict-aliasing knobs that define UBs away.

One that might be nice is -fcdecl that tells the compiler about the
calling convention and gives us leaway to abuse function pointers (like
we already do anyway).

Re: Plumbers CF MCs

[Andrew Cooper]

On 23/03/2021 08:35, Peter Zijlstra wrote:
> There also is talk about straight line speculation mitigations. for x86
> we should probably emit an INT3 after every JMP and RET. Although this
> might not be controversial and be sorted by the time Plumbers happens.

I guess this one is levied at me.

Hopefully it isn't controversial in any way.  I have just had sufficient
tuits to get around to making an x86 straight line speculation feature
request to the GCC and Clang communities yet.

For the compiler folk here, the tl;dr is that indirect CALL/JMP (i.e.
function pointers, jump tables) and RET instructions may speculatively
execute the next sequential instruction(s) before taking the control
flow change into account.  Having some toolchain support to deal with
this would be great, and preferable to some of the asm hackary currently
under suggestion.

~Andrew

Re: Plumbers CF MCs

[Peter Zijlstra]

On Tue, Mar 23, 2021 at 07:29:01PM +0000, Andrew Cooper wrote:
> On 23/03/2021 08:35, Peter Zijlstra wrote:
> > There also is talk about straight line speculation mitigations. for x86
> > we should probably emit an INT3 after every JMP and RET. Although this
> > might not be controversial and be sorted by the time Plumbers happens.
> 
> I guess this one is levied at me.
> 
> Hopefully it isn't controversial in any way.  I have just had sufficient
> tuits to get around to making an x86 straight line speculation feature
> request to the GCC and Clang communities yet.

Yep, just making sure it doesn't get lost.

> For the compiler folk here, the tl;dr is that indirect CALL/JMP (i.e.
> function pointers, jump tables) and RET instructions may speculatively
> execute the next sequential instruction(s) before taking the control
> flow change into account. 

So JMP and RET can trivially have INT3 following in such a way that that
instruction is never in the actual execution path. This is obviously not
possible for CALL.

For CALL we can follow with LFENCE (hurts), or if we can get a limit
on how many instructions deep the speculation gap is, a bunch of NOPs.
Or possibly nothing at all.

As such it might make sense to have these two cases separated out.

Another utterly disguisting option is to align all CALL instructions on
8 bytes and have it followed by 3 INT3s to make it 8 bytes long. Then
have every function prologue round up the return address. This should
work with tail-call optimizations because the round-up is NOP when
repeated. The obvious down-side is that it will utterly wreck the RSB

For unaffected chips we could then NOP both the trailing INT3s and the
alignment prologue at runtime.

> Having some toolchain support to deal with
> this would be great, and preferable to some of the asm hackary currently
> under suggestion.

Agreed, hence the inclusion in this list.

Re: Plumbers CF MCs

[Peter Zijlstra]

On Tue, Mar 23, 2021 at 08:53:58PM +0100, Peter Zijlstra wrote:

> Another utterly disguisting option is to align all CALL instructions on
> 8 bytes and have it followed by 3 INT3s to make it 8 bytes long. Then
> have every function prologue round up the return address. This should
> work with tail-call optimizations because the round-up is NOP when
> repeated. The obvious down-side is that it will utterly wreck the RSB

Slightly less horrible: "CALL func; INT3" + "INC (%RSP); RET; INT3".
It still completely wrecks RSB, but isn't nearly as wasteful.

Re: Plumbers CF MCs

[Andrew Cooper]

On 23/03/2021 19:53, Peter Zijlstra wrote:
> On Tue, Mar 23, 2021 at 07:29:01PM +0000, Andrew Cooper wrote:
>> For the compiler folk here, the tl;dr is that indirect CALL/JMP (i.e.
>> function pointers, jump tables) and RET instructions may speculatively
>> execute the next sequential instruction(s) before taking the control
>> flow change into account. 
> So JMP and RET can trivially have INT3 following in such a way that that
> instruction is never in the actual execution path. This is obviously not
> possible for CALL.
>
> For CALL we can follow with LFENCE (hurts), or if we can get a limit
> on how many instructions deep the speculation gap is, a bunch of NOPs.
> Or possibly nothing at all.
>
> As such it might make sense to have these two cases separated out.

Agreed.

There is architectural execution following an indirect CALL, so we will
only be speculating what we will execute non-speculatively later.  I
can't see any situation where this would be speculatively dangerous.

For indirect JMP, whether you care about stopping speculation depends on
how likely it is that you'll enter case 0 of a switch statement.  That
said, for profile optimised code, the chances are that the hotpath(s)
are in an if/else chain, so I don't think it is worth attempting to do
anything interesting here.

Both the Intel and AMD documents on the matter say to use LFENCE, but
that is a simplified set of instruction to cover all 3 cases.  Given
that we probably want to split CALL away from JMP/RET, then INT3 is the
shortest speculation block at a single byte, and thus has least .text
size impact.

~Andrew

Re: Plumbers CF MCs

[Nick Desaulniers]

On Tue, Mar 23, 2021 at 12:29 PM Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
>
> On 23/03/2021 08:35, Peter Zijlstra wrote:
> > There also is talk about straight line speculation mitigations. for x86
> > we should probably emit an INT3 after every JMP and RET. Although this
> > might not be controversial and be sorted by the time Plumbers happens.
>
> I guess this one is levied at me.
>
> Hopefully it isn't controversial in any way.  I have just had sufficient
> tuits to get around to making an x86 straight line speculation feature
> request to the GCC and Clang communities yet.
>
> For the compiler folk here, the tl;dr is that indirect CALL/JMP (i.e.
> function pointers, jump tables) and RET instructions may speculatively
> execute the next sequential instruction(s) before taking the control
> flow change into account.  Having some toolchain support to deal with
> this would be great, and preferable to some of the asm hackary currently
> under suggestion.

There is some prior art here, for ARM in -fharden-sls=*.
https://developer.arm.com/support/arm-security-updates/speculative-processor-vulnerability/downloads/straight-line-speculation

It's not clear from the kernel thread that the existing compiler
mitigations are amenable with live patching in/out mitigations for
hardware that is affected or not, respectively.
https://lore.kernel.org/lkml/20210305095256.GA22536@willie-the-truck/

Perhaps worth a discussion on the mailing list in
https://www.kernel.org/doc/html/latest/admin-guide/security-bugs.html ?
-- 
Thanks,
~Nick Desaulniers

Re: Plumbers CF MCs

[Steven Rostedt]

On Tue, 23 Mar 2021 23:12:46 +0100
Peter Zijlstra <peterz@infradead.org> wrote:

> On Tue, Mar 23, 2021 at 08:53:58PM +0100, Peter Zijlstra wrote:
> 
> > Another utterly disguisting option is to align all CALL instructions on
> > 8 bytes and have it followed by 3 INT3s to make it 8 bytes long. Then
> > have every function prologue round up the return address. This should
> > work with tail-call optimizations because the round-up is NOP when
> > repeated. The obvious down-side is that it will utterly wreck the RSB  
> 
> Slightly less horrible: "CALL func; INT3" + "INC (%RSP); RET; INT3".
> It still completely wrecks RSB, but isn't nearly as wasteful.

I'm confused. Why is speculation after a CALL dangerous? That code will
be executed on the return of the call anyway. That is, it's not
speculating something that wont execute in the future, whereas code
after RET and JMP wont be executed.

-- Steve

Re: Plumbers CF MCs

[Peter Zijlstra]

On Tue, Mar 23, 2021 at 03:26:29PM -0700, Nick Desaulniers wrote:
> On Tue, Mar 23, 2021 at 12:29 PM Andrew Cooper
> <andrew.cooper3@citrix.com> wrote:
> >
> > On 23/03/2021 08:35, Peter Zijlstra wrote:
> > > There also is talk about straight line speculation mitigations. for x86
> > > we should probably emit an INT3 after every JMP and RET. Although this
> > > might not be controversial and be sorted by the time Plumbers happens.
> >
> > I guess this one is levied at me.
> >
> > Hopefully it isn't controversial in any way.  I have just had sufficient
> > tuits to get around to making an x86 straight line speculation feature
> > request to the GCC and Clang communities yet.
> >
> > For the compiler folk here, the tl;dr is that indirect CALL/JMP (i.e.
> > function pointers, jump tables) and RET instructions may speculatively
> > execute the next sequential instruction(s) before taking the control
> > flow change into account.  Having some toolchain support to deal with
> > this would be great, and preferable to some of the asm hackary currently
> > under suggestion.
> 
> There is some prior art here, for ARM in -fharden-sls=*.
> https://developer.arm.com/support/arm-security-updates/speculative-processor-vulnerability/downloads/straight-line-speculation

Yes, I'm aware of that.

> It's not clear from the kernel thread that the existing compiler
> mitigations are amenable with live patching in/out mitigations for
> hardware that is affected or not, respectively.
> https://lore.kernel.org/lkml/20210305095256.GA22536@willie-the-truck/

ARM64 doesn't (yet) have objtool. Using objtool I could easily generate
a sites list to patch out things.

That said, the JMP,RET things don't need patching out (ARM's retbr
option) they're never on the execution path. It's the CALL thing (blr)
that would need runtime tinkering.

But I appreciate Will's stance; it would be very good to have a thread
analysis vs Spectre-v1.

> Perhaps worth a discussion on the mailing list in
> https://www.kernel.org/doc/html/latest/admin-guide/security-bugs.html ?

All of this is public by now, hence security@k.org isn't interested.

Re: Plumbers CF MCs

[Peter Zijlstra]

On Tue, Mar 23, 2021 at 10:23:10PM +0000, Andrew Cooper wrote:

> There is architectural execution following an indirect CALL, so we will
> only be speculating what we will execute non-speculatively later.  I
> can't see any situation where this would be speculatively dangerous.

Playing the devils advocate, suppose you have something like:

	x = array[foo(bar)];

Then the speculative flow potentially does an array dereference with
values prior to foo() having executed.

Now, on x86_64 we have the (fortunate in this case) situation that the
return register isn't the arg1 register, so exploiting this is going to
be more work, but I can see it being done in some cases.

(And, that's only the easy half of the spec leak story ofcourse.)

Anyway, CALL + LFENCE should be trivial enough to recognise and generate
a sites list for. Then all we need is a CPU list that is affected. If
only we had a shorter non-exception speculation fence... :/

Re: Plumbers CF MCs

[Peter Zijlstra]

On Tue, Mar 23, 2021 at 08:36:12PM -0400, Steven Rostedt wrote:
> On Tue, 23 Mar 2021 23:12:46 +0100
> Peter Zijlstra <peterz@infradead.org> wrote:
> 
> > On Tue, Mar 23, 2021 at 08:53:58PM +0100, Peter Zijlstra wrote:
> > 
> > > Another utterly disguisting option is to align all CALL instructions on
> > > 8 bytes and have it followed by 3 INT3s to make it 8 bytes long. Then
> > > have every function prologue round up the return address. This should
> > > work with tail-call optimizations because the round-up is NOP when
> > > repeated. The obvious down-side is that it will utterly wreck the RSB  
> > 
> > Slightly less horrible: "CALL func; INT3" + "INC (%RSP); RET; INT3".
> > It still completely wrecks RSB, but isn't nearly as wasteful.
> 
> I'm confused. Why is speculation after a CALL dangerous? That code will
> be executed on the return of the call anyway. That is, it's not
> speculating something that wont execute in the future, whereas code
> after RET and JMP wont be executed.

Like I just wrote to Andrew; it mostly isn't going to be. But there will
be cases where running the code after, with the register contents from
before, will get you a nice speculation gadget.

The big problem is, like with Spectre-v1 compiler mitigations,
recognising when it is and isn't a problem. Mostly compilers tend to
take the safe option (understandably) and we end up with an incredible
amount of LFENCE instructions and performance will suck.

For SLS at least the CALL+LFENCE pattern is trivial enough to recognise
and we can patch it out at runtime, with the only side effect being
code bloat and I$ fail. That same isn't true for Spectre-v1 mitigations,
where LFENCEs are emitted at less obviously recognised places.

Re: Plumbers CF MCs

[Christian Brauner]

On Tue, Mar 23, 2021 at 09:35:10AM +0100, Peter Zijlstra wrote:
> On Mon, Mar 22, 2021 at 01:23:03PM -0700, Nick Desaulniers wrote:
> > Hi all,
> > I saw plumbers opened call for microconferences:
> > https://www.linuxplumbersconf.org/blog/2021/index.php/2021/03/18/cfp-open-microconferences/
> > 
> > I was going to put together a submission; do we want to do a combined
> > toolchain MC, or have distinct ones this year?
> > 
> > I know in 2020 the GNU cauldron was co-located with Plumbers, as well
> > as a GNU Tools Track MC and LLVM MC.
> 
> A combined MC focussed on kernel issues seems very interesting. We still

Seeing the discussion here unfold already in such great detail it might
be worth considering either really running a separate MC focussed on
Compilers & Kernel as Peter suggests or have a full Track (combined or
not) and taking out a day just for this topic.
Both cases would guarantee that you have a "full" day available for
discussing these issues. But maybe these things are so fundamental that
having a Compilers & Kernel Track might be worth thinking about
(presupposing we'll have time in the schedule to accomodate it in
addition to e.g. a GNU Tools Track. But I think we would.).

Christian

Re: Plumbers CF MCs

[Andrew Cooper]

On 24/03/2021 08:08, Peter Zijlstra wrote:
> On Tue, Mar 23, 2021 at 10:23:10PM +0000, Andrew Cooper wrote:
>
>> There is architectural execution following an indirect CALL, so we will
>> only be speculating what we will execute non-speculatively later.  I
>> can't see any situation where this would be speculatively dangerous.

Sorry - I should have been more precise.  "... on its own".

> Playing the devils advocate, suppose you have something like:
>
> 	x = array[foo(bar)];
>
> Then the speculative flow potentially does an array dereference with
> values prior to foo() having executed.

Playing devils advocate, the issue here isn't caused by straight-line
speculation, but by the array access.

Consider foo() being a plain call rather than a function pointer call,
and having:

if ( cond )
    return bar; // or any junk really

at its head.  This is a split half Spectre gadget - the conditional in
the callee, and the (single) array access in the caller.

i.e. you require an array_index_nospec()/etc to fix this issue even when
there is no straight line speculation.

> Now, on x86_64 we have the (fortunate in this case) situation that the
> return register isn't the arg1 register, so exploiting this is going to
> be more work, but I can see it being done in some cases.
>
> (And, that's only the easy half of the spec leak story ofcourse.)

Don't underestimate half spectre gadgets.  With virt and hyperthreading,
it can be read directly out of the cache with L1TF.

Pulling it out of the load ports with MLPDS is in principle doable on
native and virt, but requires far more precise timing.

~Andrew

Re: Plumbers CF MCs

[Will Deacon]

On Tue, Mar 23, 2021 at 09:35:10AM +0100, Peter Zijlstra wrote:
> On Mon, Mar 22, 2021 at 01:23:03PM -0700, Nick Desaulniers wrote:
> > Hi all,
> > I saw plumbers opened call for microconferences:
> > https://www.linuxplumbersconf.org/blog/2021/index.php/2021/03/18/cfp-open-microconferences/
> > 
> > I was going to put together a submission; do we want to do a combined
> > toolchain MC, or have distinct ones this year?
> > 
> > I know in 2020 the GNU cauldron was co-located with Plumbers, as well
> > as a GNU Tools Track MC and LLVM MC.
> 
> A combined MC focussed on kernel issues seems very interesting. We still
> have the control dependency (volatile-if?) thing pending. We had a bit
> of a discussion on that after last year, but I don't think anything
> really came of that, can we pick that up? Ideally a compiler person does
> an actual proposal for this year.
> 
> If we can sort that, there's the rest of the dependencies Will outlined
> :-)
> 
> Then there seemed to be people that thought __always_inline was a
> suggestion... I think we need to explore how that was ever possible.
> 
> There's the endless UB debate... can we define more to reduce the UB? I
> mean, we're already bound by architecture ABI on the one hand, and
> actual use on the other. It would be so very nice to be able to get more
> -fwrapv and -fno-strict-aliasing knobs that define UBs away.
> 
> There also is talk about straight line speculation mitigations. for x86
> we should probably emit an INT3 after every JMP and RET. Although this
> might not be controversial and be sorted by the time Plumbers happens.
> 
> There was some talk about how compilers could help objtool make sense of
> jump tables.
> 
> GCC's status on asm-goto with outputs?
> 
> Clang's getting asm-constraints wrong ("rm" and it always picks "m").
> 
> 
> And I'm sure there was more..

One thing I'd like to add, and which I think is possibly relevant to the SLS
mitigation for arm64, is whether there is scope for allowing the compiler to
generate alternative instruction sequences (e.g. in a separate section),
which the kernel could then patch in during boot. We already do a tonne of
code patching on arm64 for things like CPU errata workarounds but also
for enabling support for optional architecture features, where the kernel
code would trap on CPUs without hardware support.

Another use of this would be to enable stack-taggging with MTE, where the
instrumentation is generated by the compiler but may use instructions which
are undefined if the CPU doesn't support MTE.

Will

Re: Plumbers CF MCs

[Elena Zannoni]

On 3/22/21 2:39 PM, Jose E. Marchesi wrote:
> Hi Nick.
>
>> I saw plumbers opened call for microconferences:
>> https://www.linuxplumbersconf.org/blog/2021/index.php/2021/03/18/cfp-open-microconferences/
>>
>> I was going to put together a submission; do we want to do a combined
>> toolchain MC, or have distinct ones this year?
>>
>> I know in 2020 the GNU cauldron was co-located with Plumbers, as well
>> as a GNU Tools Track MC and LLVM MC.
> We are actually discussing in another thread about abusing LPC's
> hospitality for another GNU Tools Track this year...
>
> Regarding the micro-conferences, I would be ok with either combined or
> separated.  I think both approaches have their advantages.
>
> In either case I'm up for organizing the GNU part.

Hi,
yes, so, it looks like a GNU toolchain track will happen again this year
at LPC (in lieu of the Cauldron conference).
For the toolchain related MC, we should pick a better name than last
year to avoid people getting confused between the MC and the track.
Something like "toolchains and kernel intersection" or similar.

I am neutral on whether it is with LLVM or separate. I guess it all
depends on how long we want the MC to be.
If there are enough topics for a double slot, you should request that
sooner rather than towards the end, when there might be no extra slots
left.

elena

Re: Plumbers CF MCs

[Nick Desaulniers]

On Wed, Mar 31, 2021 at 12:35 PM Elena Zannoni <elena.zannoni@oracle.com> wrote:
>
> On 3/22/21 2:39 PM, Jose E. Marchesi wrote:
> > Hi Nick.
> >
> >> I saw plumbers opened call for microconferences:
> >> https://www.linuxplumbersconf.org/blog/2021/index.php/2021/03/18/cfp-open-microconferences/
> >>
> >> I was going to put together a submission; do we want to do a combined
> >> toolchain MC, or have distinct ones this year?
> >>
> >> I know in 2020 the GNU cauldron was co-located with Plumbers, as well
> >> as a GNU Tools Track MC and LLVM MC.
> > We are actually discussing in another thread about abusing LPC's
> > hospitality for another GNU Tools Track this year...
> >
> > Regarding the micro-conferences, I would be ok with either combined or
> > separated.  I think both approaches have their advantages.
> >
> > In either case I'm up for organizing the GNU part.
>
> Hi,
> yes, so, it looks like a GNU toolchain track will happen again this year
> at LPC (in lieu of the Cauldron conference).

Cool; I generally find these talks of more interest to me personally
than many of the kernel topics. It's nice to be able to attend (as an
LLVM developer).

> For the toolchain related MC, we should pick a better name than last
> year to avoid people getting confused between the MC and the track.
> Something like "toolchains and kernel intersection" or similar.

Oh, yeah, that's a great idea.

> I am neutral on whether it is with LLVM or separate. I guess it all
> depends on how long we want the MC to be.
> If there are enough topics for a double slot, you should request that
> sooner rather than towards the end, when there might be no extra slots
> left.

Sure, that sounds good.  Maybe we can have a CFP for the proposed MC,
and if we get enough interesting proposals that it doesn't look like
we'll be able to accomodate what we'd like, we can split into two MCs
(if we even need to cross that bridge; maybe we can fit everything
into the 4hrs for an MC).  I was going to propose a timeline for us to
make such decisions by, but I just checked and it seems that the
deadline for MCs is not yet posted:
https://linuxplumbersconf.org/event/11/abstracts/#submit-abstract,
which makes working backwards from there tricky.

Steven, sorry if I missed it, but can the conf organizers pick and
post a date for the deadline for Call-for-MC submissions? This might
help MC organizers plan for putting out their own CFPs, and planning
deadlines.  In our case, it would be figuring out whether one day vs
more (if possible) or splitting the MC in two is necessary by the
deadline.  It might not even be an issue at all this year, but I think
that's the only concern folks have about doing a single
kernel+toolchain MC (vs split like last year).

Or if there's other ideas here, folks should feel empowered to speak
up. Thanks all for the feedback so far.
-- 
Thanks,
~Nick Desaulniers

Re: Plumbers CF MCs

[Kees Cook]

On Tue, Mar 30, 2021 at 03:13:12PM +0100, Will Deacon wrote:
> On Tue, Mar 23, 2021 at 09:35:10AM +0100, Peter Zijlstra wrote:
> > On Mon, Mar 22, 2021 at 01:23:03PM -0700, Nick Desaulniers wrote:
> > > Hi all,
> > > I saw plumbers opened call for microconferences:
> > > https://www.linuxplumbersconf.org/blog/2021/index.php/2021/03/18/cfp-open-microconferences/
> > > 
> > > I was going to put together a submission; do we want to do a combined
> > > toolchain MC, or have distinct ones this year?
> > > 
> > > I know in 2020 the GNU cauldron was co-located with Plumbers, as well
> > > as a GNU Tools Track MC and LLVM MC.
> > 
> > A combined MC focussed on kernel issues seems very interesting. We still
> > have the control dependency (volatile-if?) thing pending. We had a bit
> > of a discussion on that after last year, but I don't think anything
> > really came of that, can we pick that up? Ideally a compiler person does
> > an actual proposal for this year.
> > 
> > If we can sort that, there's the rest of the dependencies Will outlined
> > :-)
> > 
> > Then there seemed to be people that thought __always_inline was a
> > suggestion... I think we need to explore how that was ever possible.
> > 
> > There's the endless UB debate... can we define more to reduce the UB? I
> > mean, we're already bound by architecture ABI on the one hand, and
> > actual use on the other. It would be so very nice to be able to get more
> > -fwrapv and -fno-strict-aliasing knobs that define UBs away.
> > 
> > There also is talk about straight line speculation mitigations. for x86
> > we should probably emit an INT3 after every JMP and RET. Although this
> > might not be controversial and be sorted by the time Plumbers happens.
> > 
> > There was some talk about how compilers could help objtool make sense of
> > jump tables.
> > 
> > GCC's status on asm-goto with outputs?
> > 
> > Clang's getting asm-constraints wrong ("rm" and it always picks "m").
> > 
> > 
> > And I'm sure there was more..
> 
> One thing I'd like to add, and which I think is possibly relevant to the SLS
> mitigation for arm64, is whether there is scope for allowing the compiler to
> generate alternative instruction sequences (e.g. in a separate section),
> which the kernel could then patch in during boot. We already do a tonne of
> code patching on arm64 for things like CPU errata workarounds but also
> for enabling support for optional architecture features, where the kernel
> code would trap on CPUs without hardware support.
> 
> Another use of this would be to enable stack-taggging with MTE, where the
> instrumentation is generated by the compiler but may use instructions which
> are undefined if the CPU doesn't support MTE.

Or swapping out SCS and stack-protector for PAC when the hardware
supports it...

-- 
Kees Cook

Re: Plumbers CF MCs

[Jose E. Marchesi]

Hi Nick.

> On Wed, Mar 31, 2021 at 12:35 PM Elena Zannoni <elena.zannoni@oracle.com> wrote:
>>
>> On 3/22/21 2:39 PM, Jose E. Marchesi wrote:
>> > Hi Nick.
>> >
>> >> I saw plumbers opened call for microconferences:
>> >> https://www.linuxplumbersconf.org/blog/2021/index.php/2021/03/18/cfp-open-microconferences/
>> >>
>> >> I was going to put together a submission; do we want to do a combined
>> >> toolchain MC, or have distinct ones this year?
>> >>
>> >> I know in 2020 the GNU cauldron was co-located with Plumbers, as well
>> >> as a GNU Tools Track MC and LLVM MC.
>> > We are actually discussing in another thread about abusing LPC's
>> > hospitality for another GNU Tools Track this year...
>> >
>> > Regarding the micro-conferences, I would be ok with either combined or
>> > separated.  I think both approaches have their advantages.
>> >
>> > In either case I'm up for organizing the GNU part.
>>
>> Hi,
>> yes, so, it looks like a GNU toolchain track will happen again this year
>> at LPC (in lieu of the Cauldron conference).
>
> Cool; I generally find these talks of more interest to me personally
> than many of the kernel topics. It's nice to be able to attend (as an
> LLVM developer).
>
>> For the toolchain related MC, we should pick a better name than last
>> year to avoid people getting confused between the MC and the track.
>> Something like "toolchains and kernel intersection" or similar.
>
> Oh, yeah, that's a great idea.

What about "Toolchains and Kernel MC".

>> I am neutral on whether it is with LLVM or separate. I guess it all
>> depends on how long we want the MC to be.
>> If there are enough topics for a double slot, you should request that
>> sooner rather than towards the end, when there might be no extra slots
>> left.
>
> Sure, that sounds good.  Maybe we can have a CFP for the proposed MC,
> and if we get enough interesting proposals that it doesn't look like
> we'll be able to accomodate what we'd like, we can split into two MCs
> (if we even need to cross that bridge; maybe we can fit everything
> into the 4hrs for an MC).

I think that is a good plan.

So, how should we proceed?  Should I send a MC proposal, or you do it?
:)

Re: Plumbers CF MCs

[Steven Rostedt]

On Wed, 31 Mar 2021 13:34:47 -0600
Elena Zannoni <elena.zannoni@oracle.com> wrote:

> I am neutral on whether it is with LLVM or separate. I guess it all
> depends on how long we want the MC to be.
> If there are enough topics for a double slot, you should request that
> sooner rather than towards the end, when there might be no extra slots
> left.

I would only do double slots if there's room. I rather have more topics
than double sessions.

-- Steve

Re: Plumbers CF MCs

[Steven Rostedt]

On Wed, 31 Mar 2021 13:34:47 -0600
Elena Zannoni <elena.zannoni@oracle.com> wrote:

> I am neutral on whether it is with LLVM or separate. I guess it all
> depends on how long we want the MC to be.
> If there are enough topics for a double slot, you should request that
> sooner rather than towards the end, when there might be no extra slots
> left.

I would only do double slots if there's room. I rather have more MCs
than double sessions.

-- Steve

Re: Plumbers CF MCs

[Steven Rostedt]

On Wed, 31 Mar 2021 13:36:03 -0700
Nick Desaulniers <ndesaulniers@google.com> wrote:

> Steven, sorry if I missed it, but can the conf organizers pick and
> post a date for the deadline for Call-for-MC submissions? This might

You mean for MC topics, not the MC itself.

> help MC organizers plan for putting out their own CFPs, and planning
> deadlines.  In our case, it would be figuring out whether one day vs
> more (if possible) or splitting the MC in two is necessary by the
> deadline.  It might not even be an issue at all this year, but I think
> that's the only concern folks have about doing a single
> kernel+toolchain MC (vs split like last year).

As I replied to Elena. We don't give out double slots when an MC is ready.
It is only given out if they are slots available after all MCs have been
accepted.

Since it sounds like there will also be a toolchain track, I don't see why
you would need multiple MCs along with it.

-- Steve

Re: Plumbers CF MCs

[Steven Rostedt]

On Thu, 1 Apr 2021 09:29:35 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> On Wed, 31 Mar 2021 13:36:03 -0700
> Nick Desaulniers <ndesaulniers@google.com> wrote:
> 
> > Steven, sorry if I missed it, but can the conf organizers pick and
> > post a date for the deadline for Call-for-MC submissions? This might  
> 
> You mean for MC topics, not the MC itself.

Should have been a question. But if it is for MC topics, yes, the MC
organizers can choose their own deadlines for submissions. Which can be any
time until up to a week before the conference. All topics should be in by
a week before, and acceptance of those topics at least 3 days before, but
earlier is always better.

-- Steve

Re: Plumbers CF MCs

[Elena Zannoni]

On 4/1/21 7:29 AM, Steven Rostedt wrote:
> On Wed, 31 Mar 2021 13:36:03 -0700
> Nick Desaulniers <ndesaulniers@google.com> wrote:
>
>> Steven, sorry if I missed it, but can the conf organizers pick and
>> post a date for the deadline for Call-for-MC submissions? This might
> You mean for MC topics, not the MC itself.
>
>> help MC organizers plan for putting out their own CFPs, and planning
>> deadlines.  In our case, it would be figuring out whether one day vs
>> more (if possible) or splitting the MC in two is necessary by the
>> deadline.  It might not even be an issue at all this year, but I think
>> that's the only concern folks have about doing a single
>> kernel+toolchain MC (vs split like last year).
> As I replied to Elena. We don't give out double slots when an MC is ready.
> It is only given out if they are slots available after all MCs have been
> accepted.

We have been taking requests for double slots at any time though.
Putting them on the schedule is another thing.


> Since it sounds like there will also be a toolchain track, I don't see why
> you would need multiple MCs along with it.

Just like last year the toolchain track would be a substitute for
Cauldron and it would be a completely different thing 100% focused on
the toolchain. Here we are talking about discussing topics of interest
to the kernel as well. IIRC last year we had a llvm toolcahin MC as well.

elena


> -- Steve

Re: Plumbers CF MCs

[Miguel Ojeda]

On Thu, Apr 1, 2021 at 3:50 PM Elena Zannoni <elena.zannoni@oracle.com> wrote:
>
> Just like last year the toolchain track would be a substitute for
> Cauldron and it would be a completely different thing 100% focused on
> the toolchain. Here we are talking about discussing topics of interest
> to the kernel as well. IIRC last year we had a llvm toolcahin MC as well.

Not sure if it is too early to talk about it / request it, but
something on Rust support in the kernel ("Rust for Linux") can be
interesting, even if only on the toolchain side of it.

Cheers,
Miguel

Re: Plumbers CF MCs

[Nick Desaulniers]

On Thu, Apr 1, 2021 at 12:59 AM Jose E. Marchesi <jemarch@gnu.org> wrote:
>
>
> Hi Nick.
>
> > On Wed, Mar 31, 2021 at 12:35 PM Elena Zannoni <elena.zannoni@oracle.com> wrote:
> >>
> >> For the toolchain related MC, we should pick a better name than last
> >> year to avoid people getting confused between the MC and the track.
> >> Something like "toolchains and kernel intersection" or similar.
> >
> > Oh, yeah, that's a great idea.
>
> What about "Toolchains and Kernel MC".

SGTM

> So, how should we proceed?  Should I send a MC proposal, or you do it?
> :)

I shared a google doc with you; let's draft something up quickly, then
maybe next week (next Friday?) we can submit it to
https://linuxplumbersconf.org/event/11/abstracts/.  As long as we're
both listed as authors in that form, I think we can even still edit it
post-submision, IIRC from last year.

-- 
Thanks,
~Nick Desaulniers

Re: Plumbers CF MCs

[Nick Desaulniers]

On Thu, Apr 1, 2021 at 6:50 AM Elena Zannoni <elena.zannoni@oracle.com> wrote:
>
> > Since it sounds like there will also be a toolchain track, I don't see why
> > you would need multiple MCs along with it.
>
> Just like last year the toolchain track would be a substitute for
> Cauldron and it would be a completely different thing 100% focused on
> the toolchain. Here we are talking about discussing topics of interest
> to the kernel as well. IIRC last year we had a llvm toolcahin MC as well.

Precisely, recall last year there were 3 toolchain tracks.
https://linuxplumbersconf.org/event/7/timetable/#all

1. GNU Tools Track
(https://linuxplumbersconf.org/event/7/sessions/90/#all). This was a
substitute for the GNU Tools Cauldron conference that got cancelled
due to COVID. (https://gcc.gnu.org/wiki/cauldron2020)
2. GNU Toolchain MC
(https://linuxplumbersconf.org/event/7/sessions/95/#20200828).
3. LLVM MC (https://linuxplumbersconf.org/event/7/sessions/84/#20200827)

What we're discussing here is combining 2 and 3; it sounds like you're
suggesting combining 1, 2, and 3. I don't think it would be
appropriate for me to suggest LLVM specific topics (or kernel specific
topics for that matter) for what is effectively a replacement for the
GNU Tools Cauldron, and GNU Tools Cauldron isn't specific to the Linux
kernel (where as 2 and 3 very much are, or were).

While I suspect we can fit all relevant topics for GNU and LLVM
specific to the kernel in one MC, we would like to first get a sense
of CFP volume from potential speakers to decide whether to combine any
MCs at all this year, or repeat what we did last year.

Knowing when the deadline is for MC track proposals would help us help
potential speakers have a deadline to notify us.  But it's not clear
when MC proposals for the MC tracks themselves are due, which is what
I'm asking.  Not deadlines for an MC track's actual talks.
-- 
Thanks,
~Nick Desaulniers

Re: Plumbers CF MCs

[Segher Boessenkool]

On Tue, Mar 23, 2021 at 09:35:10AM +0100, Peter Zijlstra wrote:
> GCC's status on asm-goto with outputs?

http://gcc.gnu.org/g:e3b3b59683c1 (and later patches fixing issues).

asm goto with outputs is a jump instruction for GCC, like any other asm
goto is.


Segher

Re: Plumbers CF MCs

[Segher Boessenkool]

On Tue, Mar 23, 2021 at 08:53:58PM +0100, Peter Zijlstra wrote:
> On Tue, Mar 23, 2021 at 07:29:01PM +0000, Andrew Cooper wrote:
> > For the compiler folk here, the tl;dr is that indirect CALL/JMP (i.e.
> > function pointers, jump tables) and RET instructions may speculatively
> > execute the next sequential instruction(s) before taking the control
> > flow change into account. 
> 
> So JMP and RET can trivially have INT3 following in such a way that that
> instruction is never in the actual execution path. This is obviously not
> possible for CALL.

What about
  call ; jmp 0f ; int3 ; 0:
?  Only the jmp can be speculatively executed (if your (u)arch executes
jumps at all), but that cannot be a problem, no data is involved.

> > Having some toolchain support to deal with
> > this would be great, and preferable to some of the asm hackary currently
> > under suggestion.

I fail to see how it is anything more than some boring backend-specific
patches?  By nature you pretty much have to emit this as canned
instruction sequences, so all the rest of the compiler sees it is just a
call or jump or whathaveyou.


Segher