From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED, USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C87BC43381 for ; Mon, 4 Mar 2019 17:13:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5FFC8206B6 for ; Mon, 4 Mar 2019 17:13:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726800AbfCDRNa (ORCPT ); Mon, 4 Mar 2019 12:13:30 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:36242 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726182AbfCDRNa (ORCPT ); Mon, 4 Mar 2019 12:13:30 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8718CA78; Mon, 4 Mar 2019 09:13:29 -0800 (PST) Received: from en101 (en101.cambridge.arm.com [10.1.196.93]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 962F73F703; Mon, 4 Mar 2019 09:13:27 -0800 (PST) Date: Mon, 4 Mar 2019 17:13:21 +0000 From: Suzuki K Poulose To: Zenghui Yu Cc: christoffer.dall@arm.com, marc.zyngier@arm.com, Punit Agrawal , julien.thierry@arm.com, LKML , james.morse@arm.com, Zenghui Yu , wanghaibin.wang@huawei.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Subject: Re: [RFC PATCH] KVM: arm64: Force a PTE mapping when logging is enabled Message-ID: <20190304171320.GA3984@en101> References: <1551497728-12576-1-git-send-email-yuzenghui@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Zenghui, On Sun, Mar 03, 2019 at 11:14:38PM +0800, Zenghui Yu wrote: > I think there're still some problems in this patch... Details below. > > On Sat, Mar 2, 2019 at 11:39 AM Zenghui Yu wrote: > > > > The idea behind this is: we don't want to keep tracking of huge pages when > > logging_active is true, which will result in performance degradation. We > > still need to set vma_pagesize to PAGE_SIZE, so that we can make use of it > > to force a PTE mapping. Yes, you're right. We are indeed ignoring the force_pte flag. > > > > Cc: Suzuki K Poulose > > Cc: Punit Agrawal > > Signed-off-by: Zenghui Yu > > > > --- > > Atfer looking into https://patchwork.codeaurora.org/patch/647985/ , the > > "vma_pagesize = PAGE_SIZE" logic was not intended to be deleted. As far > > as I can tell, we used to have "hugetlb" to force the PTE mapping, but > > we have "vma_pagesize" currently instead. We should set it properly for > > performance reasons (e.g, in VM migration). Did I miss something important? > > > > --- > > virt/kvm/arm/mmu.c | 7 +++++++ > > 1 file changed, 7 insertions(+) > > > > diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c > > index 30251e2..7d41b16 100644 > > --- a/virt/kvm/arm/mmu.c > > +++ b/virt/kvm/arm/mmu.c > > @@ -1705,6 +1705,13 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > > (vma_pagesize == PUD_SIZE && kvm_stage2_has_pmd(kvm))) && > > !force_pte) { > > gfn = (fault_ipa & huge_page_mask(hstate_vma(vma))) >> PAGE_SHIFT; > > + } else { > > + /* > > + * Fallback to PTE if it's not one of the stage2 > > + * supported hugepage sizes or the corresponding level > > + * doesn't exist, or logging is enabled. > > First, Instead of "logging is enabled", it should be "force_pte is true", > since "force_pte" will be true when: > > 1) fault_supports_stage2_pmd_mappings() return false; or > 2) "logging is enabled" (e.g, in VM migration). > > Second, fallback some unsupported hugepage sizes (e.g, 64K hugepage with > 4K pages) to PTE is somewhat strange. And it will then _unexpectedly_ > reach transparent_hugepage_adjust(), though no real adjustment will happen > since commit fd2ef358282c ("KVM: arm/arm64: Ensure only THP is candidate > for adjustment"). Keeping "vma_pagesize" there as it is will be better, > right? > > So I'd just simplify the logic like: We could fix this right in the beginning. See patch below: > > } else if (force_pte) { > vma_pagesize = PAGE_SIZE; > } > > > Will send a V2 later and waiting for your comments :) diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c index 30251e2..529331e 100644 --- a/virt/kvm/arm/mmu.c +++ b/virt/kvm/arm/mmu.c @@ -1693,7 +1693,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, return -EFAULT; } - vma_pagesize = vma_kernel_pagesize(vma); + /* If we are forced to map at page granularity, force the pagesize here */ + vma_pagesize = force_pte ? PAGE_SIZE : vma_kernel_pagesize(vma); + /* * The stage2 has a minimum of 2 level table (For arm64 see * kvm_arm_setup_stage2()). Hence, we are guaranteed that we can @@ -1701,11 +1703,10 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, * As for PUD huge maps, we must make sure that we have at least * 3 levels, i.e, PMD is not folded. */ - if ((vma_pagesize == PMD_SIZE || - (vma_pagesize == PUD_SIZE && kvm_stage2_has_pmd(kvm))) && - !force_pte) { + if (vma_pagesize == PMD_SIZE || + (vma_pagesize == PUD_SIZE && kvm_stage2_has_pmd(kvm))) gfn = (fault_ipa & huge_page_mask(hstate_vma(vma))) >> PAGE_SHIFT; - } + up_read(¤t->mm->mmap_sem); /* We need minimum second+third level pages */ --- Suzuki From mboxrd@z Thu Jan 1 00:00:00 1970 From: Suzuki K Poulose Subject: Re: [RFC PATCH] KVM: arm64: Force a PTE mapping when logging is enabled Date: Mon, 4 Mar 2019 17:13:21 +0000 Message-ID: <20190304171320.GA3984@en101> References: <1551497728-12576-1-git-send-email-yuzenghui@huawei.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id C4E994A1DF for ; Mon, 4 Mar 2019 12:13:31 -0500 (EST) Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CsOxVg+mIIE2 for ; Mon, 4 Mar 2019 12:13:30 -0500 (EST) Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 299A049F94 for ; Mon, 4 Mar 2019 12:13:30 -0500 (EST) Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu To: Zenghui Yu Cc: marc.zyngier@arm.com, Punit Agrawal , LKML , Zenghui Yu , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org List-Id: kvmarm@lists.cs.columbia.edu Hi Zenghui, On Sun, Mar 03, 2019 at 11:14:38PM +0800, Zenghui Yu wrote: > I think there're still some problems in this patch... Details below. > > On Sat, Mar 2, 2019 at 11:39 AM Zenghui Yu wrote: > > > > The idea behind this is: we don't want to keep tracking of huge pages when > > logging_active is true, which will result in performance degradation. We > > still need to set vma_pagesize to PAGE_SIZE, so that we can make use of it > > to force a PTE mapping. Yes, you're right. We are indeed ignoring the force_pte flag. > > > > Cc: Suzuki K Poulose > > Cc: Punit Agrawal > > Signed-off-by: Zenghui Yu > > > > --- > > Atfer looking into https://patchwork.codeaurora.org/patch/647985/ , the > > "vma_pagesize = PAGE_SIZE" logic was not intended to be deleted. As far > > as I can tell, we used to have "hugetlb" to force the PTE mapping, but > > we have "vma_pagesize" currently instead. We should set it properly for > > performance reasons (e.g, in VM migration). Did I miss something important? > > > > --- > > virt/kvm/arm/mmu.c | 7 +++++++ > > 1 file changed, 7 insertions(+) > > > > diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c > > index 30251e2..7d41b16 100644 > > --- a/virt/kvm/arm/mmu.c > > +++ b/virt/kvm/arm/mmu.c > > @@ -1705,6 +1705,13 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > > (vma_pagesize == PUD_SIZE && kvm_stage2_has_pmd(kvm))) && > > !force_pte) { > > gfn = (fault_ipa & huge_page_mask(hstate_vma(vma))) >> PAGE_SHIFT; > > + } else { > > + /* > > + * Fallback to PTE if it's not one of the stage2 > > + * supported hugepage sizes or the corresponding level > > + * doesn't exist, or logging is enabled. > > First, Instead of "logging is enabled", it should be "force_pte is true", > since "force_pte" will be true when: > > 1) fault_supports_stage2_pmd_mappings() return false; or > 2) "logging is enabled" (e.g, in VM migration). > > Second, fallback some unsupported hugepage sizes (e.g, 64K hugepage with > 4K pages) to PTE is somewhat strange. And it will then _unexpectedly_ > reach transparent_hugepage_adjust(), though no real adjustment will happen > since commit fd2ef358282c ("KVM: arm/arm64: Ensure only THP is candidate > for adjustment"). Keeping "vma_pagesize" there as it is will be better, > right? > > So I'd just simplify the logic like: We could fix this right in the beginning. See patch below: > > } else if (force_pte) { > vma_pagesize = PAGE_SIZE; > } > > > Will send a V2 later and waiting for your comments :) diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c index 30251e2..529331e 100644 --- a/virt/kvm/arm/mmu.c +++ b/virt/kvm/arm/mmu.c @@ -1693,7 +1693,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, return -EFAULT; } - vma_pagesize = vma_kernel_pagesize(vma); + /* If we are forced to map at page granularity, force the pagesize here */ + vma_pagesize = force_pte ? PAGE_SIZE : vma_kernel_pagesize(vma); + /* * The stage2 has a minimum of 2 level table (For arm64 see * kvm_arm_setup_stage2()). Hence, we are guaranteed that we can @@ -1701,11 +1703,10 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, * As for PUD huge maps, we must make sure that we have at least * 3 levels, i.e, PMD is not folded. */ - if ((vma_pagesize == PMD_SIZE || - (vma_pagesize == PUD_SIZE && kvm_stage2_has_pmd(kvm))) && - !force_pte) { + if (vma_pagesize == PMD_SIZE || + (vma_pagesize == PUD_SIZE && kvm_stage2_has_pmd(kvm))) gfn = (fault_ipa & huge_page_mask(hstate_vma(vma))) >> PAGE_SHIFT; - } + up_read(¤t->mm->mmap_sem); /* We need minimum second+third level pages */ --- Suzuki From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.5 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED,USER_AGENT_MUTT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D3E6C43381 for ; Mon, 4 Mar 2019 17:13:39 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 46877206B6 for ; Mon, 4 Mar 2019 17:13:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="YU3LgGr/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 46877206B6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=etXRFV7ybvy7pbMQrmCvXoB1AbWSlh0XyemZWIpJKsE=; b=YU3LgGr//mA7tI A7qF0cJg45H3q5uPzrdISfrcyAclFgMA7xIrkWHG516MIb5upKWm9H/nJmTjz75aNNaTcT50V1EmT 86QXGpLTqBBl80zMRjYbsjOYfllyTajakCwUmAaY4ItyegGEgDkKtvcC8OY2o8gxrO/aqCXYumXzh UeBa3nVbDR6LgsVUSPr0cCefA9AV+Yldx5IFXPO+SbVqM2DPSlkKS2n3PpxtRTZFuYXr5MnYmcXGH 770XMt7E54QaouEL5yqcS5AUXElryBuR0mTuQnxIdF1f9ucWIKtTqH1EUr2x0++CEHO5LSL0EW3ah bGIi4sTDujcFE5Z0tblA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1h0r9i-0006R5-Hw; Mon, 04 Mar 2019 17:13:34 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1h0r9f-0006Qd-Ki for linux-arm-kernel@lists.infradead.org; Mon, 04 Mar 2019 17:13:33 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8718CA78; Mon, 4 Mar 2019 09:13:29 -0800 (PST) Received: from en101 (en101.cambridge.arm.com [10.1.196.93]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 962F73F703; Mon, 4 Mar 2019 09:13:27 -0800 (PST) Date: Mon, 4 Mar 2019 17:13:21 +0000 From: Suzuki K Poulose To: Zenghui Yu Subject: Re: [RFC PATCH] KVM: arm64: Force a PTE mapping when logging is enabled Message-ID: <20190304171320.GA3984@en101> References: <1551497728-12576-1-git-send-email-yuzenghui@huawei.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190304_091331_691066_C0309814 X-CRM114-Status: GOOD ( 28.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: julien.thierry@arm.com, marc.zyngier@arm.com, Punit Agrawal , LKML , christoffer.dall@arm.com, james.morse@arm.com, Zenghui Yu , wanghaibin.wang@huawei.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Zenghui, On Sun, Mar 03, 2019 at 11:14:38PM +0800, Zenghui Yu wrote: > I think there're still some problems in this patch... Details below. > > On Sat, Mar 2, 2019 at 11:39 AM Zenghui Yu wrote: > > > > The idea behind this is: we don't want to keep tracking of huge pages when > > logging_active is true, which will result in performance degradation. We > > still need to set vma_pagesize to PAGE_SIZE, so that we can make use of it > > to force a PTE mapping. Yes, you're right. We are indeed ignoring the force_pte flag. > > > > Cc: Suzuki K Poulose > > Cc: Punit Agrawal > > Signed-off-by: Zenghui Yu > > > > --- > > Atfer looking into https://patchwork.codeaurora.org/patch/647985/ , the > > "vma_pagesize = PAGE_SIZE" logic was not intended to be deleted. As far > > as I can tell, we used to have "hugetlb" to force the PTE mapping, but > > we have "vma_pagesize" currently instead. We should set it properly for > > performance reasons (e.g, in VM migration). Did I miss something important? > > > > --- > > virt/kvm/arm/mmu.c | 7 +++++++ > > 1 file changed, 7 insertions(+) > > > > diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c > > index 30251e2..7d41b16 100644 > > --- a/virt/kvm/arm/mmu.c > > +++ b/virt/kvm/arm/mmu.c > > @@ -1705,6 +1705,13 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, > > (vma_pagesize == PUD_SIZE && kvm_stage2_has_pmd(kvm))) && > > !force_pte) { > > gfn = (fault_ipa & huge_page_mask(hstate_vma(vma))) >> PAGE_SHIFT; > > + } else { > > + /* > > + * Fallback to PTE if it's not one of the stage2 > > + * supported hugepage sizes or the corresponding level > > + * doesn't exist, or logging is enabled. > > First, Instead of "logging is enabled", it should be "force_pte is true", > since "force_pte" will be true when: > > 1) fault_supports_stage2_pmd_mappings() return false; or > 2) "logging is enabled" (e.g, in VM migration). > > Second, fallback some unsupported hugepage sizes (e.g, 64K hugepage with > 4K pages) to PTE is somewhat strange. And it will then _unexpectedly_ > reach transparent_hugepage_adjust(), though no real adjustment will happen > since commit fd2ef358282c ("KVM: arm/arm64: Ensure only THP is candidate > for adjustment"). Keeping "vma_pagesize" there as it is will be better, > right? > > So I'd just simplify the logic like: We could fix this right in the beginning. See patch below: > > } else if (force_pte) { > vma_pagesize = PAGE_SIZE; > } > > > Will send a V2 later and waiting for your comments :) diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c index 30251e2..529331e 100644 --- a/virt/kvm/arm/mmu.c +++ b/virt/kvm/arm/mmu.c @@ -1693,7 +1693,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, return -EFAULT; } - vma_pagesize = vma_kernel_pagesize(vma); + /* If we are forced to map at page granularity, force the pagesize here */ + vma_pagesize = force_pte ? PAGE_SIZE : vma_kernel_pagesize(vma); + /* * The stage2 has a minimum of 2 level table (For arm64 see * kvm_arm_setup_stage2()). Hence, we are guaranteed that we can @@ -1701,11 +1703,10 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, * As for PUD huge maps, we must make sure that we have at least * 3 levels, i.e, PMD is not folded. */ - if ((vma_pagesize == PMD_SIZE || - (vma_pagesize == PUD_SIZE && kvm_stage2_has_pmd(kvm))) && - !force_pte) { + if (vma_pagesize == PMD_SIZE || + (vma_pagesize == PUD_SIZE && kvm_stage2_has_pmd(kvm))) gfn = (fault_ipa & huge_page_mask(hstate_vma(vma))) >> PAGE_SHIFT; - } + up_read(¤t->mm->mmap_sem); /* We need minimum second+third level pages */ --- Suzuki _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel