From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA892C433ED for ; Thu, 22 Apr 2021 06:50:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9E1AD6144A for ; Thu, 22 Apr 2021 06:50:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230341AbhDVGuk (ORCPT ); Thu, 22 Apr 2021 02:50:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:45152 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229962AbhDVGuk (ORCPT ); Thu, 22 Apr 2021 02:50:40 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 54CC561448; Thu, 22 Apr 2021 06:50:05 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lZTA3-008paW-5p; Thu, 22 Apr 2021 07:50:03 +0100 Date: Thu, 22 Apr 2021 07:50:02 +0100 Message-ID: <87v98eq0dh.wl-maz@kernel.org> From: Marc Zyngier To: Gavin Shan Cc: Keqian Zhu , kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, cjia@nvidia.com, linux-arm-kernel@lists.infradead.org, "Wanghaibin (D)" Subject: Re: [PATCH] KVM: arm64: Correctly handle the mmio faulting In-Reply-To: References: <1603297010-18787-1-git-send-email-sashukla@nvidia.com> <8b20dfc0-3b5e-c658-c47d-ebc50d20568d@huawei.com> <2e23aaa7-0c8d-13ba-2eae-9e6ab2adc587@redhat.com> <871rb3rgpl.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: gshan@redhat.com, zhukeqian1@huawei.com, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, cjia@nvidia.com, linux-arm-kernel@lists.infradead.org, wanghaibin.wang@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Thu, 22 Apr 2021 03:02:00 +0100, Gavin Shan wrote: > > Hi Marc, > > On 4/21/21 9:59 PM, Marc Zyngier wrote: > > On Wed, 21 Apr 2021 07:17:44 +0100, > > Keqian Zhu wrote: > >> On 2021/4/21 14:20, Gavin Shan wrote: > >>> On 4/21/21 12:59 PM, Keqian Zhu wrote: > >>>> On 2020/10/22 0:16, Santosh Shukla wrote: > >>>>> The Commit:6d674e28 introduces a notion to detect and handle the > >>>>> device mapping. The commit checks for the VM_PFNMAP flag is set > >>>>> in vma->flags and if set then marks force_pte to true such that > >>>>> if force_pte is true then ignore the THP function check > >>>>> (/transparent_hugepage_adjust()). > >>>>> > >>>>> There could be an issue with the VM_PFNMAP flag setting and checking. > >>>>> For example consider a case where the mdev vendor driver register's > >>>>> the vma_fault handler named vma_mmio_fault(), which maps the > >>>>> host MMIO region in-turn calls remap_pfn_range() and maps > >>>>> the MMIO's vma space. Where, remap_pfn_range implicitly sets > >>>>> the VM_PFNMAP flag into vma->flags. > >>>> Could you give the name of the mdev vendor driver that triggers this issue? > >>>> I failed to find one according to your description. Thanks. > >>>> > >>> > >>> I think it would be fixed in driver side to set VM_PFNMAP in > >>> its mmap() callback (call_mmap()), like vfio PCI driver does. > >>> It means it won't be delayed until page fault is issued and > >>> remap_pfn_range() is called. It's determined from the beginning > >>> that the vma associated the mdev vendor driver is serving as > >>> PFN remapping purpose. So the vma should be populated completely, > >>> including the VM_PFNMAP flag before it becomes visible to user > >>> space. > > > > Why should that be a requirement? Lazy populating of the VMA should be > > perfectly acceptable if the fault can only happen on the CPU side. > > > > It isn't a requirement and the drivers needn't follow strictly. I checked > several drivers before looking into the patch and found almost all the > drivers have VM_PFNMAP set at mmap() time. In drivers/vfio/vfio-pci.c, > there is a comment as below, but it doesn't reveal too much about why > we can't set VM_PFNMAP at fault time. > > static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma) > { > : > /* > * See remap_pfn_range(), called from vfio_pci_fault() but we can't > * change vm_flags within the fault handler. Set them now. > */ > vma->vm_flags |= VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP; > vma->vm_ops = &vfio_pci_mmap_ops; > > return 0; > } > > To set these flags in advance does have advantages. For example, > VM_DONTEXPAND prevents the vma to be merged with another > one. VM_DONTDUMP make this vma isn't eligible for > coredump. Otherwise, the address space, which is associated with the > vma is accessed and unnecessary page faults are triggered on > coredump. VM_IO and VM_PFNMAP avoids to walk the page frames > associated with the vma since we don't have valid PFN in the > mapping. But PCI clearly isn't the case we are dealing with here, and not everything is VFIO either. I can *today* create a driver that implements a mmap+fault handler, call mmap() on it, pass the result to a memslot, and get to the exact same result Santosh describes. No PCI, no VFIO, just a random driver. We are *required* to handle that. M. -- Without deviation from the norm, progress is not possible.