Re: [tip:x86/urgent] x86/mm/kmmio: Fix mmiotrace for page unaligned addresses

[Ilia Mirkin <imirkin@alum.mit.edu>]

On Tue, Dec 12, 2017 at 9:43 AM, Peter Zijlstra <peterz@infradead.org> wrote:
> On Tue, Dec 12, 2017 at 09:21:10AM -0500, Ilia Mirkin wrote:
>> The "thing" being mmiotrace, or the "thing" being page-unaligned addresses?
>
> mmiotrace
>
>> If the former, its primary use-case is for snooping on the NVIDIA
>> proprietary GPU driver in order to figure out how to drive the
>> underlying hardware. The driver does ioremap's to get at PCI space,
>> which mmiotrace "steals" and provides pages without a present bit set,
>> along with a fault handler. When the fault handler is hit, it
>> reinstates the faulting page, and single-steps the faulting
>> instruction
>
> At which point you have valid page-tables and another CPU can access
> that page too.
>
>> reading the before/after regs to determine what happened
>> (doesn't work universally, but enough for instructions used for PCI
>> MMIO accesses). See mmio-mod.c::pre and post (the latter is called
>> from the debug handler).
>
> And after that you only invalidate the TLBs for the CPU that took the
> initial fault, leaving possibly stale TLBs on other CPUs.
>
>
> So this 'thing' has huge gaping SMP holes in.

Sure does! Probably why the following happens when mmiotrace is enabled:

void enable_mmiotrace(void)
{
        mutex_lock(&mmiotrace_mutex);
        if (is_enabled())
                goto out;

        if (nommiotrace)
                pr_info("MMIO tracing disabled.\n");
        kmmio_init();
        enter_uniprocessor();
        spin_lock_irq(&trace_lock);
        atomic_inc(&mmiotrace_enabled);
        spin_unlock_irq(&trace_lock);
        pr_info("enabled.\n");
out:
        mutex_unlock(&mmiotrace_mutex);
}