From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24193C432C3 for ; Thu, 21 Nov 2019 09:40:10 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id A571A20715 for ; Thu, 21 Nov 2019 09:40:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A571A20715 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 10F464ACF8; Thu, 21 Nov 2019 04:40:09 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id o-Jj3wkvmeFo; Thu, 21 Nov 2019 04:40:07 -0500 (EST) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id AF2A44AEA2; Thu, 21 Nov 2019 04:40:07 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id DFCF24ACF4 for ; Thu, 21 Nov 2019 04:40:06 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id umSuT-EGis5v for ; Thu, 21 Nov 2019 04:40:05 -0500 (EST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 6C6BF4A5A8 for ; Thu, 21 Nov 2019 04:40:05 -0500 (EST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E87E3DA7; Thu, 21 Nov 2019 01:40:04 -0800 (PST) Received: from localhost (e113682-lin.copenhagen.arm.com [10.32.145.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7C8603F52E; Thu, 21 Nov 2019 01:40:04 -0800 (PST) Date: Thu, 21 Nov 2019 10:40:03 +0100 From: Christoffer Dall To: Sean Christopherson Subject: Re: Memory regions and VMAs across architectures Message-ID: <20191121094003.GB22554@e113682-lin.lund.arm.com> References: <20191108111920.GD17608@e113682-lin.lund.arm.com> <20191120034448.GC25890@linux.intel.com> <20191120115216.GL8317@e113682-lin.lund.arm.com> <20191120152807.GA32572@linux.intel.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20191120152807.GA32572@linux.intel.com> User-Agent: Mutt/1.10.1 (2018-07-13) Cc: kvm@vger.kernel.org, Marc Zyngier , borntraeger@de.ibm.com, Ard Biesheuvel , Paolo Bonzini , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On Wed, Nov 20, 2019 at 07:28:07AM -0800, Sean Christopherson wrote: > On Wed, Nov 20, 2019 at 12:52:16PM +0100, Christoffer Dall wrote: > > On Tue, Nov 19, 2019 at 07:44:48PM -0800, Sean Christopherson wrote: > > > On Fri, Nov 08, 2019 at 12:19:20PM +0100, Christoffer Dall wrote: > > > > First, what prevents user space from messing around with the VMAs after > > > > kvm_arch_prepare_memory_region() completes? If nothing, then what is > > > > the value of the cheks we perform wrt. to VMAs? > > > > > > Arm's prepare_memory_region() holds mmap_sem and mmu_lock while processing > > > the VMAs and populating the stage 2 page tables. Holding mmap_sem prevents > > > the VMAs from being invalidated while the stage 2 tables are populated, > > > e.g. prevents racing with the mmu notifier. The VMAs could be modified > > > after prepare_memory_region(), but the mmu notifier will ensure they are > > > unmapped from stage2 prior the the host change taking effect. So I think > > > you're safe (famous last words). > > > > > > > So we for example check: > > > > writeable = !(memslot->falgs & KVM_MEM_READONLY); > > if (writeable && !(vma->vm_flags & VM_WRITE)) > > return -EPERM; > > > > And yes, user space can then unmap the VMAs and MMU notifiers will > > unmap the stage 2 entries, but user space can then create a new > > read-only VMA covering the area of the memslot and the fault-handling > > path will have to deal with this same check later.Only, the fault > > handling path, via gfn_to_pfn_prot(), returns an address based on an > > entirely different set of mechanics, than our prepare_memory_region, > > which I think indicates we are doing something wrong somewhere, and we > > should have a common path for faulting things in, for I/O, both if we do > > this up-front or if we do this at fault time. > > Unconditionally interpreting vm_pgoff as a physical address does not seem > correct. There are cases where that might be correct, e.g. if the backing > (virtual) file is a flat representation of the address space, which appears > to be the case on some architectures, e.g. for PCI handling. But even then > there should be some confirmation that the VMA is actually associated with > such a file, otherwise KVM is at the mercy of userspace to do the right > thing (unless there are other guarantees on arm I am unaware of). > > > > > Second, why would arm/arm64 need special handling for I/O mappings > > > > compared to other architectures, and how is this dealt with for > > > > x86/s390/power/... ? > > > > > > As Ard mentioned, it looks like an optimization. The "passthrough" > > > part from the changelog implies that VM_PFNMAP memory regions are exclusive > > > to the guest. Mapping the entire thing would be a nice boot optimization > > > as it would save taking page faults on every page of the MMIO region. > > > > > > As for how this is different from other archs... at least on x86, VM_PFNMAP > > > isn't guaranteed to be passthrough or even MMIO, e.g. prefaulting the > > > pages may actually trigger allocation, and remapping the addresses could be > > > flat out wrong. > > > > What does VM_PFNMAP mean on x86? I didn't think we were relying on > > anything architecture specific in their meaning in the arm code, and I > > thought the VM_PFNMAP was a generic mm flag with generic mm meaning, > > but I could be wrong here? > > No, you're correct, VM_PFNMAP is a generic flag that state the VMA doesn't > have an associated struct page and is being managed directly by something > other than the core mmu. > > But not having a struct page doesn't guarantee that the PFN is backed by > MMIO, or that it is exclusive to the guest (although in practice this is > probably the case 99.9999% of the time). E.g. x86 supports having guest > memory backed by regular ram that is hidden from the host kernel via > 'mem=', which will show up as VM_PFNMAP. > > > Is there any valid semantics for creating a memslot backed by a > > VM_PFNMAP on x86, and if so, what are those? > > > > Similarly, if you do map a device region straight to the guest on x86, > > how is that handled? (A pointer to the right place in the myriad of EPT > > and shadow code in x86 would be much appreciated.) > > There is no special handling in x86 for VM_PFNMAP memory, we rely on KVM's > generic __gfn_to_pfn_memslot() to retrieve the PFN on demand, and use > mmu_notifier_seq to ensure the stale PFNs (invalidated in the host) aren't > inserted into the guest page tables. Effectively the same thing arm does, > sans the prepare_memory_region() shenanigans. Thanks Sean, I'll have a look at reworking said shenanigans ;) Christoffer _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm