From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52057C433ED for ; Thu, 15 Apr 2021 10:23:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2BEE9613B0 for ; Thu, 15 Apr 2021 10:23:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231423AbhDOKYO (ORCPT ); Thu, 15 Apr 2021 06:24:14 -0400 Received: from mail.kernel.org ([198.145.29.99]:43920 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229943AbhDOKYM (ORCPT ); Thu, 15 Apr 2021 06:24:12 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 7C1D7610CB; Thu, 15 Apr 2021 10:23:47 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lWzA1-007cL6-Cd; Thu, 15 Apr 2021 11:23:45 +0100 Date: Thu, 15 Apr 2021 11:23:44 +0100 Message-ID: <87im4nn8wf.wl-maz@kernel.org> From: Marc Zyngier To: Keqian Zhu Cc: , , , , , Santosh Shukla Subject: Re: [PATCH v3 2/2] kvm/arm64: Try stage2 block mapping for host device MMIO In-Reply-To: References: <20210414065109.8616-1-zhukeqian1@huawei.com> <20210414065109.8616-3-zhukeqian1@huawei.com> <87pmyxme2m.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: zhukeqian1@huawei.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, wanghaibin.wang@huawei.com, sashukla@nvidia.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 15 Apr 2021 03:20:52 +0100, Keqian Zhu wrote: > > Hi Marc, > > On 2021/4/14 17:05, Marc Zyngier wrote: > > + Santosh, who found some interesting bugs in that area before. > > > > On Wed, 14 Apr 2021 07:51:09 +0100, > > Keqian Zhu wrote: > >> > >> The MMIO region of a device maybe huge (GB level), try to use > >> block mapping in stage2 to speedup both map and unmap. > >> > >> Compared to normal memory mapping, we should consider two more > >> points when try block mapping for MMIO region: > >> > >> 1. For normal memory mapping, the PA(host physical address) and > >> HVA have same alignment within PUD_SIZE or PMD_SIZE when we use > >> the HVA to request hugepage, so we don't need to consider PA > >> alignment when verifing block mapping. But for device memory > >> mapping, the PA and HVA may have different alignment. > >> > >> 2. For normal memory mapping, we are sure hugepage size properly > >> fit into vma, so we don't check whether the mapping size exceeds > >> the boundary of vma. But for device memory mapping, we should pay > >> attention to this. > >> > >> This adds device_rough_page_shift() to check these two points when > >> selecting block mapping size. > >> > >> Signed-off-by: Keqian Zhu > >> --- > >> arch/arm64/kvm/mmu.c | 37 +++++++++++++++++++++++++++++++++---- > >> 1 file changed, 33 insertions(+), 4 deletions(-) > >> > >> diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c > >> index c59af5ca01b0..1a6d96169d60 100644 > >> --- a/arch/arm64/kvm/mmu.c > >> +++ b/arch/arm64/kvm/mmu.c > >> @@ -624,6 +624,31 @@ static void kvm_send_hwpoison_signal(unsigned long address, short lsb) > >> send_sig_mceerr(BUS_MCEERR_AR, (void __user *)address, lsb, current); > >> } > >> > >> +/* > >> + * Find a max mapping size that properly insides the vma. And hva and pa must > >> + * have the same alignment to this mapping size. It's rough as there are still > >> + * other restrictions, will be checked by fault_supports_stage2_huge_mapping(). > >> + */ > >> +static short device_rough_page_shift(struct vm_area_struct *vma, > >> + unsigned long hva) > > > > My earlier question still stands. Under which circumstances would this > > function return something that is *not* the final mapping size? I > > really don't see a reason why this would not return the final mapping > > size. > > IIUC, all the restrictions are about alignment and area boundary. > > That's to say, HVA, IPA and PA must have same alignment within the > mapping size. And the areas are memslot and vma, which means the > mapping size must properly fit into the memslot and vma. > > In this function, we just checked the alignment of HVA and PA, and > the boundary of vma. So we still need to check the alignment of HVA > and IPA, and the boundary of memslot. These will be checked by > fault_supports_stage2_huge_mapping(). But that's no different from what we do with normal memory, is it? So it really feels like we should have *one* function that deals with establishing the basic mapping size from the VMA (see below for what I have in mind). > > > > >> +{ > >> + phys_addr_t pa = (vma->vm_pgoff << PAGE_SHIFT) + (hva - vma->vm_start); > >> + > >> +#ifndef __PAGETABLE_PMD_FOLDED > >> + if ((hva & (PUD_SIZE - 1)) == (pa & (PUD_SIZE - 1)) && > >> + ALIGN_DOWN(hva, PUD_SIZE) >= vma->vm_start && > >> + ALIGN(hva, PUD_SIZE) <= vma->vm_end) > >> + return PUD_SHIFT; > >> +#endif > >> + > >> + if ((hva & (PMD_SIZE - 1)) == (pa & (PMD_SIZE - 1)) && > >> + ALIGN_DOWN(hva, PMD_SIZE) >= vma->vm_start && > >> + ALIGN(hva, PMD_SIZE) <= vma->vm_end) > >> + return PMD_SHIFT; > >> + > >> + return PAGE_SHIFT; > >> +} > >> + > >> static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot, > >> unsigned long hva, > >> unsigned long map_size) > >> @@ -769,7 +794,10 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > >> return -EFAULT; > >> } > >> > >> - /* Let's check if we will get back a huge page backed by hugetlbfs */ > >> + /* > >> + * Let's check if we will get back a huge page backed by hugetlbfs, or > >> + * get block mapping for device MMIO region. > >> + */ > >> mmap_read_lock(current->mm); > >> vma = find_vma_intersection(current->mm, hva, hva + 1); > >> if (unlikely(!vma)) { > >> @@ -780,11 +808,12 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > >> > >> if (is_vm_hugetlb_page(vma)) > >> vma_shift = huge_page_shift(hstate_vma(vma)); > >> + else if (vma->vm_flags & VM_PFNMAP) > >> + vma_shift = device_rough_page_shift(vma, hva); > > > > What prevents a VMA from having both VM_HUGETLB and VM_PFNMAP? This is > > pretty unlikely, but I'd like to see this case catered for. > > > I'm not sure whether VM_HUGETLB and VM_PFNMAP are compatible, and I > failed to find a case. > > VM_PFNMAP is used for page-ranges managed without "struct page", > just pure PFN. IIUC, VM_HUGETLB is used for hugetlbfs, which always > has "struct page". So I think they should not be compatible, > otherwise it's a bug of driver. For now, maybe. But huge mappings of PFN could land at some point, and it'd be hard to catch. I think this case deserves a VM_BUG_ON(). > > >> else > >> vma_shift = PAGE_SHIFT; > >> > >> - if (logging_active || > >> - (vma->vm_flags & VM_PFNMAP)) { > >> + if (logging_active) { BTW, don't you introduce a bug here? Logging shouldn't affect device mappings. > >> force_pte = true; > >> vma_shift = PAGE_SHIFT; > >> } > >> @@ -855,7 +884,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > >> > >> if (kvm_is_device_pfn(pfn)) { > >> device = true; > >> - force_pte = true; > >> + force_pte = (vma_pagesize == PAGE_SIZE); > > > > Why do we need to set force_pte if we are already dealing with > > PAGE_SIZE? I guess you are doing this for the sake of avoiding the > > call to transparent_hugepage_adjust(), right? > Yes. > > > > > I'd rather you simply don't try to upgrade a device mapping by > > explicitly checking for this and keep force_pte for *memory* > > exclusively. > Agree, that's better. > > > > > Santosh, can you please take a look at this series and try to see if > > the problem you fixed in [1] (which ended up as commit 91a2c34b7d6f) > > is still OK with this series? > I searched the initial version[*], VM_PFNMAP is set when we call > gfn_to_pfn_prot()->vma_mmio_fault()->remap_pfn_range(). Then the > check of VM_PFNMAP in user_mem_abort() failed, so we will try to > call transparent_hugepage_adjust() for device pfn. > > In that case, our logic of trying block mapping for MMIO is not > used. And we still set force_pte for device pfn, so this bugfix is > not affected. Santosh, do you agree that? But isn't what we just agreed to get rid of just above? > > I still found that the reason vfio_pci does not have this > bug. vfio_pci set VM_PFNMAP for vma when userspace calls mmap(). I > will apply this logic for vfio_mdev too, let's see what vfio > maintainer think about it. I think that'd be good to see what Alex thinks about it... Here's the changes I propose. It is completely untested, of course. Thanks, M. diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 8711894db8c2..f32d956cc199 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -738,6 +738,35 @@ transparent_hugepage_adjust(struct kvm_memory_slot *memslot, return PAGE_SIZE; } +static int get_vma_page_shift(struct vm_area_struct *vma, unsigned long hva) +{ + unsigned long pa; + + if (is_vm_hugetlb_page(vma) && !(vma->vm_flags & VM_PFNMAP)) + return huge_page_shift(hstate_vma(vma)); + + if (!(vma->vm_flags & VM_PFNMAP)) + return PAGE_SHIFT; + + VM_BUG_ON(is_vm_hugetlb_page(vma)); + + pa = (vma->vm_pgoff << PAGE_SHIFT) + (hva - vma->vm_start); + +#ifndef __PAGETABLE_PMD_FOLDED + if ((hva & (PUD_SIZE - 1)) == (pa & (PUD_SIZE - 1)) && + ALIGN_DOWN(hva, PUD_SIZE) >= vma->vm_start && + ALIGN(hva, PUD_SIZE) <= vma->vm_end) + return PUD_SHIFT; +#endif + + if ((hva & (PMD_SIZE - 1)) == (pa & (PMD_SIZE - 1)) && + ALIGN_DOWN(hva, PMD_SIZE) >= vma->vm_start && + ALIGN(hva, PMD_SIZE) <= vma->vm_end) + return PMD_SHIFT; + + return PAGE_SHIFT; +} + static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, struct kvm_memory_slot *memslot, unsigned long hva, unsigned long fault_status) @@ -778,13 +807,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, return -EFAULT; } - if (is_vm_hugetlb_page(vma)) - vma_shift = huge_page_shift(hstate_vma(vma)); - else - vma_shift = PAGE_SHIFT; + vma_shift = get_vma_page_shift(vma, hva); - if (logging_active || - (vma->vm_flags & VM_PFNMAP)) { + if (logging_active && !(vma->vm_flags & VM_PFNMAP)) { force_pte = true; vma_shift = PAGE_SHIFT; } @@ -854,8 +879,17 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, return -EFAULT; if (kvm_is_device_pfn(pfn)) { + /* + * If the page was identified as device early by looking at + * the VMA flags, vma_pagesize is already representing the + * largest quantity we can map. If instead it was mapped + * via gfn_to_pfn_prot(), vma_pagesize is set to PAGE_SIZE + * and must not be upgraded. + * + * In both cases, we don't let transparent_hugepage_adjust() + * change things at the last minute. + */ device = true; - force_pte = true; } else if (logging_active && !write_fault) { /* * Only actually map the page as writable if this was a write @@ -876,7 +910,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, * If we are not forced to use page mapping, check if we are * backed by a THP and thus use block mapping if possible. */ - if (vma_pagesize == PAGE_SIZE && !force_pte) + if (vma_pagesize == PAGE_SIZE && !(force_pte || device)) vma_pagesize = transparent_hugepage_adjust(memslot, hva, &pfn, &fault_ipa); if (writable) -- Without deviation from the norm, progress is not possible.