Re: Memory regions and VMAs across architectures

[Sean Christopherson <sean.j.christopherson@intel.com>]

On Fri, Nov 08, 2019 at 12:19:20PM +0100, Christoffer Dall wrote:
> Hi,
> 
> I had a look at our relatively complicated logic in
> kvm_arch_prepare_memory_region(), and was wondering if there was room to
> unify some of this handling between architectures.
> 
> (If you haven't seen our implementation, you can find it in
> virt/kvm/arm/mmu.c, and it has lovely ASCII art!)
> 
> I then had a look at the x86 code, but that doesn't actually do anything
> when creating memory regions, which makes me wonder why the arhitectures
> differ in this aspect.
> 
> The reason we added the logic that we have for arm/arm64 is that we
> don't really want to take faults for I/O accesses.  I'm not actually
> sure if this is a corretness thing, or an optimization effort, and the
> original commit message doesn't really explain.  Ard, you wrote that
> code, do you recall the details?
> 
> In any case, what we do is to check for each VMA backing a memslot, we
> check if the memslot flags and vma flags are a reasonable match, and we
> try to detect I/O mappings by looking for the VM_PFNMAP flag on the VMA
> and pre-populate stage 2 page tables (our equivalent of EPT/NPT/...).
> However, there are some things which are not clear to me:
> 
> First, what prevents user space from messing around with the VMAs after
> kvm_arch_prepare_memory_region() completes?  If nothing, then what is
> the value of the cheks we perform wrt. to VMAs?

Arm's prepare_memory_region() holds mmap_sem and mmu_lock while processing
the VMAs and populating the stage 2 page tables.  Holding mmap_sem prevents
the VMAs from being invalidated while the stage 2 tables are populated,
e.g. prevents racing with the mmu notifier.  The VMAs could be modified
after prepare_memory_region(), but the mmu notifier will ensure they are
unmapped from stage2 prior the the host change taking effect.  So I think
you're safe (famous last words).

> Second, why would arm/arm64 need special handling for I/O mappings
> compared to other architectures, and how is this dealt with for
> x86/s390/power/... ?

As Ard mentioned, it looks like an optimization.  The "passthrough"
part from the changelog implies that VM_PFNMAP memory regions are exclusive
to the guest.  Mapping the entire thing would be a nice boot optimization
as it would save taking page faults on every page of the MMIO region.

As for how this is different from other archs... at least on x86, VM_PFNMAP
isn't guaranteed to be passthrough or even MMIO, e.g. prefaulting the
pages may actually trigger allocation, and remapping the addresses could be
flat out wrong.


  commit 8eef91239e57d2e932e7470879c9a504d5494ebb
  Author: Ard Biesheuvel <ard.biesheuvel@linaro.org>
  Date:   Fri Oct 10 17:00:32 2014 +0200

    arm/arm64: KVM: map MMIO regions at creation time

    There is really no point in faulting in memory regions page by page
    if they are not backed by demand paged system RAM but by a linear
    passthrough mapping of a host MMIO region. So instead, detect such
    regions at setup time and install the mappings for the backing all
    at once.


_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm