Re: [patch 51/60] x86/mm: Allow flushing for future ASID switches

From: Andy Lutomirski <luto@kernel.org>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Andy Lutomirski <luto@kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	LKML <linux-kernel@vger.kernel.org>, X86 ML <x86@kernel.org>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Dave Hansen <dave.hansen@intel.com>,
	Borislav Petkov <bpetkov@suse.de>,
	Greg KH <gregkh@linuxfoundation.org>,
	Kees Cook <keescook@google.com>, Hugh Dickins <hughd@google.com>,
	Brian Gerst <brgerst@gmail.com>,
	Josh Poimboeuf <jpoimboe@redhat.com>,
	Denys Vlasenko <dvlasenk@redhat.com>,
	Rik van Riel <riel@redhat.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>,
	David Laight <David.Laight@aculab.com>,
	Eduardo Valentin <eduval@amazon.com>,
	aliguori@amazon.com, Will Deacon <will.deacon@arm.com>,
	Daniel Gruss <daniel.gruss@iaik.tugraz.at>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Ingo Molnar <mingo@kernel.org>,
	michael.schwarz@iaik.tugraz.at, Borislav Petkov <bp@alien8.de>,
	moritz.lipp@iaik.tugraz.at, richard.fellner@student.tugraz.at,
	Andrew Banman <abanman@hpe.com>,
	mike.travis@hpe.com
Subject: Re: [patch 51/60] x86/mm: Allow flushing for future ASID switches
Date: Mon, 4 Dec 2017 14:54:46 -0800	[thread overview]
Message-ID: <CALCETrUHkwHLb7pwcsfUfaEfdZtSTVxd0=e2AXVmQ0zBm7srUA@mail.gmail.com> (raw)
In-Reply-To: <20171204224757.GC20227@worktop.programming.kicks-ass.net>

On Mon, Dec 4, 2017 at 2:47 PM, Peter Zijlstra <peterz@infradead.org> wrote:
> On Mon, Dec 04, 2017 at 02:22:54PM -0800, Andy Lutomirski wrote:
>
>> > +static inline void invalidate_pcid_other(void)
>> > +{
>> > +       /*
>> > +        * With global pages, all of the shared kenel page tables
>> > +        * are set as _PAGE_GLOBAL.  We have no shared nonglobals
>> > +        * and nothing to do here.
>> > +        */
>> > +       if (!static_cpu_has_bug(X86_BUG_CPU_SECURE_MODE_KPTI))
>> > +               return;
>>
>> I think I'd be more comfortable if this check were in the caller, not
>> here.  Shouldn't a function called invalidate_pcid_other() do what the
>> name says?
>
> Yeah, you're probably right. The thing is course that we only ever need
> that operation for kpti (as of now). But me renaming this stuff made
> this problem :/
>
>> > +       this_cpu_write(cpu_tlbstate.invalidate_other, true);
>>
>> Why do we need this extra variable instead of just looping over all
>> other ASIDs and invalidating them?  It would be something like:
>>
>>         for (i = 1; i < TLB_NR_DYN_ASIDS; i++) {
>>                 if (i != this_cpu_read(cpu_tlbstate.loaded_mm_asid))
>>                        this_cpu_write(cpu_tlbstate.ctxs[i].ctx_id, 0);
>>         }
>>
>> modulo epic whitespace damage and possible typos.
>
> I think the point is that we can do many invalidate_other's before we
> ever do a switch_mm(). The above would be more expensive.
>
> Not sure it would matter in practise though.
>
>> >  static inline void __flush_tlb_one(unsigned long addr)
>> >  {
>> >         count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ONE);
>> >         __flush_tlb_single(addr);
>> > +       /*
>> > +        * Invalidate other address spaces inaccessible to single-page
>> > +        * invalidation:
>> > +        */
>>
>> Ugh.  If I'm reading this right, __flush_tlb_single() means "flush one
>> user address" and __flush_tlb_one() means "flush one kernel address".
>
> That would make sense, woulnd't it? :-) But afaict the __flush_tlb_one()
> user in tlb_uv.c is in fact for userspace and should be
> __flush_tlb_single().
>
> Andrew, Mike, can either of you shed light on what exactly you need
> invalidated there?
>
>> That's, um, not exactly obvious.  Could this be at least commented
>> better?
>
> As is __flush_tlb_single() does user and __flush_tlb_one() does
> user+kernel.

Yep.  A one-liner above the function to that effect would make it
*way* clearer what's going on.