From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F9F7C433ED for ; Wed, 7 Apr 2021 13:18:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 294F661242 for ; Wed, 7 Apr 2021 13:18:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352597AbhDGNSb (ORCPT ); Wed, 7 Apr 2021 09:18:31 -0400 Received: from mail.kernel.org ([198.145.29.99]:57818 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352577AbhDGNSQ (ORCPT ); Wed, 7 Apr 2021 09:18:16 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 4475E61369; Wed, 7 Apr 2021 13:18:07 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lU84L-0064x4-8f; Wed, 07 Apr 2021 14:18:05 +0100 Date: Wed, 07 Apr 2021 14:18:04 +0100 Message-ID: <878s5up71v.wl-maz@kernel.org> From: Marc Zyngier To: Keqian Zhu Cc: , , , , Will Deacon , Catalin Marinas , Mark Rutland , James Morse , Suzuki K Poulose , Julien Thierry , , , , Subject: Re: [RFC PATCH v2 2/2] kvm/arm64: Try stage2 block mapping for host device MMIO In-Reply-To: <20210316134338.18052-3-zhukeqian1@huawei.com> References: <20210316134338.18052-1-zhukeqian1@huawei.com> <20210316134338.18052-3-zhukeqian1@huawei.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: zhukeqian1@huawei.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, james.morse@arm.com, suzuki.poulose@arm.com, julien.thierry.kdev@gmail.com, wanghaibin.wang@huawei.com, jiangkunkun@huawei.com, yuzenghui@huawei.com, lushenming@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Tue, 16 Mar 2021 13:43:38 +0000, Keqian Zhu wrote: > > The MMIO region of a device maybe huge (GB level), try to use > block mapping in stage2 to speedup both map and unmap. > > Compared to normal memory mapping, we should consider two more > points when try block mapping for MMIO region: > > 1. For normal memory mapping, the PA(host physical address) and > HVA have same alignment within PUD_SIZE or PMD_SIZE when we use > the HVA to request hugepage, so we don't need to consider PA > alignment when verifing block mapping. But for device memory > mapping, the PA and HVA may have different alignment. > > 2. For normal memory mapping, we are sure hugepage size properly > fit into vma, so we don't check whether the mapping size exceeds > the boundary of vma. But for device memory mapping, we should pay > attention to this. > > This adds device_rough_page_shift() to check these two points when > selecting block mapping size. > > Signed-off-by: Keqian Zhu > --- > > Mainly for RFC, not fully tested. I will fully test it when the > code logic is well accepted. > > --- > arch/arm64/kvm/mmu.c | 42 ++++++++++++++++++++++++++++++++++++++---- > 1 file changed, 38 insertions(+), 4 deletions(-) > > diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c > index c59af5ca01b0..224aa15eb4d9 100644 > --- a/arch/arm64/kvm/mmu.c > +++ b/arch/arm64/kvm/mmu.c > @@ -624,6 +624,36 @@ static void kvm_send_hwpoison_signal(unsigned long address, short lsb) > send_sig_mceerr(BUS_MCEERR_AR, (void __user *)address, lsb, current); > } > > +/* > + * Find a mapping size that properly insides the intersection of vma and > + * memslot. And hva and pa have the same alignment to this mapping size. > + * It's rough because there are still other restrictions, which will be > + * checked by the following fault_supports_stage2_huge_mapping(). I don't think these restrictions make complete sense to me. If this is a PFNMAP VMA, we should use the biggest mapping size that covers the VMA, and not more than the VMA. > + */ > +static short device_rough_page_shift(struct kvm_memory_slot *memslot, > + struct vm_area_struct *vma, > + unsigned long hva) > +{ > + size_t size = memslot->npages * PAGE_SIZE; > + hva_t sec_start = max(memslot->userspace_addr, vma->vm_start); > + hva_t sec_end = min(memslot->userspace_addr + size, vma->vm_end); > + phys_addr_t pa = (vma->vm_pgoff << PAGE_SHIFT) + (hva - vma->vm_start); > + > +#ifndef __PAGETABLE_PMD_FOLDED > + if ((hva & (PUD_SIZE - 1)) == (pa & (PUD_SIZE - 1)) && > + ALIGN_DOWN(hva, PUD_SIZE) >= sec_start && > + ALIGN(hva, PUD_SIZE) <= sec_end) > + return PUD_SHIFT; > +#endif > + > + if ((hva & (PMD_SIZE - 1)) == (pa & (PMD_SIZE - 1)) && > + ALIGN_DOWN(hva, PMD_SIZE) >= sec_start && > + ALIGN(hva, PMD_SIZE) <= sec_end) > + return PMD_SHIFT; > + > + return PAGE_SHIFT; > +} > + > static bool fault_supports_stage2_huge_mapping(struct kvm_memory_slot *memslot, > unsigned long hva, > unsigned long map_size) > @@ -769,7 +799,10 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > return -EFAULT; > } > > - /* Let's check if we will get back a huge page backed by hugetlbfs */ > + /* > + * Let's check if we will get back a huge page backed by hugetlbfs, or > + * get block mapping for device MMIO region. > + */ > mmap_read_lock(current->mm); > vma = find_vma_intersection(current->mm, hva, hva + 1); > if (unlikely(!vma)) { > @@ -780,11 +813,12 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > > if (is_vm_hugetlb_page(vma)) > vma_shift = huge_page_shift(hstate_vma(vma)); > + else if (vma->vm_flags & VM_PFNMAP) > + vma_shift = device_rough_page_shift(memslot, vma, hva); > else > vma_shift = PAGE_SHIFT; > > - if (logging_active || > - (vma->vm_flags & VM_PFNMAP)) { > + if (logging_active) { > force_pte = true; > vma_shift = PAGE_SHIFT; But why should we downgrade to page-size mappings if logging? This is a device, and you aren't moving the device around, are you? Or is your device actually memory with a device mapping that you are trying to migrate? > } > @@ -855,7 +889,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > > if (kvm_is_device_pfn(pfn)) { > device = true; > - force_pte = true; > + force_pte = (vma_pagesize == PAGE_SIZE); > } else if (logging_active && !write_fault) { > /* > * Only actually map the page as writable if this was a write > -- > 2.19.1 > > Thanks, M. -- Without deviation from the norm, progress is not possible.