From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7638C433DB for ; Wed, 17 Mar 2021 14:42:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8B23964F40 for ; Wed, 17 Mar 2021 14:42:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232087AbhCQOmF (ORCPT ); Wed, 17 Mar 2021 10:42:05 -0400 Received: from mail.kernel.org ([198.145.29.99]:57986 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231932AbhCQOlf (ORCPT ); Wed, 17 Mar 2021 10:41:35 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 01BC864F70; Wed, 17 Mar 2021 14:41:35 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lMXMa-002EI2-PA; Wed, 17 Mar 2021 14:41:32 +0000 Date: Wed, 17 Mar 2021 14:41:31 +0000 Message-ID: <87a6r1j10k.wl-maz@kernel.org> From: Marc Zyngier To: Quentin Perret Cc: catalin.marinas@arm.com, will@kernel.org, james.morse@arm.com, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, android-kvm@google.com, seanjc@google.com, linux-kernel@vger.kernel.org, robh+dt@kernel.org, linux-arm-kernel@lists.infradead.org, kernel-team@android.com, kvmarm@lists.cs.columbia.edu, tabba@google.com, ardb@kernel.org, mark.rutland@arm.com, dbrazdil@google.com, mate.toth-pal@arm.com Subject: Re: [PATCH 1/2] KVM: arm64: Introduce KVM_PGTABLE_S2_NOFWB Stage-2 flag In-Reply-To: <20210317141714.383046-2-qperret@google.com> References: <20210315143536.214621-34-qperret@google.com> <20210317141714.383046-1-qperret@google.com> <20210317141714.383046-2-qperret@google.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: qperret@google.com, catalin.marinas@arm.com, will@kernel.org, james.morse@arm.com, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, android-kvm@google.com, seanjc@google.com, linux-kernel@vger.kernel.org, robh+dt@kernel.org, linux-arm-kernel@lists.infradead.org, kernel-team@android.com, kvmarm@lists.cs.columbia.edu, tabba@google.com, ardb@kernel.org, mark.rutland@arm.com, dbrazdil@google.com, mate.toth-pal@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Quentin, On Wed, 17 Mar 2021 14:17:13 +0000, Quentin Perret wrote: > > In order to further configure stage-2 page-tables, pass flags to the > init function using a new enum. > > The first of these flags allows to disable FWB even if the hardware > supports it as we will need to do so for the host stage-2. > > Signed-off-by: Quentin Perret > > --- > > One question is, do we want to use stage2_has_fwb() everywhere, including > guest-specific paths (e.g. kvm_arch_prepare_memory_region(), ...) ? > > That'd make this patch more intrusive, but would make the whole codebase > work with FWB enabled on a guest by guest basis. I don't see us use that > anytime soon (other than maybe debug of some sort?) but it'd be good to > have an agreement. I'm not sure how useful that would be. We fought long and hard to get FWB, and I can't see a good reason to disable it for guests unless the HW was buggy (but in which case that'd be for everyone). I'd rather keep the changes small for now (this whole series is invasive enough!). As for this patch, I only have a few cosmetic comments: > --- > arch/arm64/include/asm/kvm_pgtable.h | 19 +++++++++-- > arch/arm64/include/asm/pgtable-prot.h | 4 +-- > arch/arm64/kvm/hyp/pgtable.c | 49 +++++++++++++++++---------- > 3 files changed, 50 insertions(+), 22 deletions(-) > > diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h > index b93a2a3526ab..7382bdfb6284 100644 > --- a/arch/arm64/include/asm/kvm_pgtable.h > +++ b/arch/arm64/include/asm/kvm_pgtable.h > @@ -56,6 +56,15 @@ struct kvm_pgtable_mm_ops { > phys_addr_t (*virt_to_phys)(void *addr); > }; > > +/** > + * enum kvm_pgtable_stage2_flags - Stage-2 page-table flags. > + * @KVM_PGTABLE_S2_NOFWB: Don't enforce Normal-WB even if the CPUs have > + * ARM64_HAS_STAGE2_FWB. > + */ > +enum kvm_pgtable_stage2_flags { > + KVM_PGTABLE_S2_NOFWB = BIT(0), > +}; > + > /** > * struct kvm_pgtable - KVM page-table. > * @ia_bits: Maximum input address size, in bits. > @@ -72,6 +81,7 @@ struct kvm_pgtable { > > /* Stage-2 only */ > struct kvm_s2_mmu *mmu; > + enum kvm_pgtable_stage2_flags flags; > }; > > /** > @@ -201,11 +211,16 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift); > * @arch: Arch-specific KVM structure representing the guest virtual > * machine. > * @mm_ops: Memory management callbacks. > + * @flags: Stage-2 configuration flags. > * > * Return: 0 on success, negative error code on failure. > */ > -int kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, > - struct kvm_pgtable_mm_ops *mm_ops); > +int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch, > + struct kvm_pgtable_mm_ops *mm_ops, > + enum kvm_pgtable_stage2_flags flags); > + > +#define kvm_pgtable_stage2_init(pgt, arch, mm_ops) \ > + kvm_pgtable_stage2_init_flags(pgt, arch, mm_ops, 0) > > /** > * kvm_pgtable_stage2_destroy() - Destroy an unused guest stage-2 page-table. > diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h > index 046be789fbb4..beeb722a82d3 100644 > --- a/arch/arm64/include/asm/pgtable-prot.h > +++ b/arch/arm64/include/asm/pgtable-prot.h > @@ -72,10 +72,10 @@ extern bool arm64_use_ng_mappings; > #define PAGE_KERNEL_EXEC __pgprot(PROT_NORMAL & ~PTE_PXN) > #define PAGE_KERNEL_EXEC_CONT __pgprot((PROT_NORMAL & ~PTE_PXN) | PTE_CONT) > > -#define PAGE_S2_MEMATTR(attr) \ > +#define PAGE_S2_MEMATTR(attr, has_fwb) \ > ({ \ > u64 __val; \ > - if (cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) \ > + if (has_fwb) \ > __val = PTE_S2_MEMATTR(MT_S2_FWB_ ## attr); \ > else \ > __val = PTE_S2_MEMATTR(MT_S2_ ## attr); \ > diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c > index 3a971df278bd..dee8aaeaf13e 100644 > --- a/arch/arm64/kvm/hyp/pgtable.c > +++ b/arch/arm64/kvm/hyp/pgtable.c > @@ -507,12 +507,25 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift) > return vtcr; > } > > -static int stage2_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) > +static bool stage2_has_fwb(struct kvm_pgtable *pgt) > +{ > + if (!cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) > + return false; > + > + return !(pgt->flags & KVM_PGTABLE_S2_NOFWB); > +} > + > +static int stage2_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep, > + struct kvm_pgtable *pgt) nit: make pgt the first parameter, as it defines the context in which the rest applies. > { > bool device = prot & KVM_PGTABLE_PROT_DEVICE; > - kvm_pte_t attr = device ? PAGE_S2_MEMATTR(DEVICE_nGnRE) : > - PAGE_S2_MEMATTR(NORMAL); > u32 sh = KVM_PTE_LEAF_ATTR_LO_S2_SH_IS; > + kvm_pte_t attr; > + > + if (device) > + attr = PAGE_S2_MEMATTR(DEVICE_nGnRE, stage2_has_fwb(pgt)); > + else > + attr = PAGE_S2_MEMATTR(NORMAL, stage2_has_fwb(pgt)); Maybe define a new helper: #define KVM_S2_MEMATTR(pgt, attr) PAGE_S2_MEMATTR(attr, stage2_has_fwb(pgt)) to avoid the constant stage2_has_fwb() repetition. > > if (!(prot & KVM_PGTABLE_PROT_X)) > attr |= KVM_PTE_LEAF_ATTR_HI_S2_XN; > @@ -748,7 +761,7 @@ int kvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, > .arg = &map_data, > }; > > - ret = stage2_set_prot_attr(prot, &map_data.attr); > + ret = stage2_set_prot_attr(prot, &map_data.attr, pgt); > if (ret) > return ret; > > @@ -786,16 +799,13 @@ int kvm_pgtable_stage2_set_owner(struct kvm_pgtable *pgt, u64 addr, u64 size, > > static void stage2_flush_dcache(void *addr, u64 size) > { > - if (cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) > - return; > - > __flush_dcache_area(addr, size); > } Consider dropping the function altogether and use __flush_dcache_area directly (assuming the prototypes are identical). > > -static bool stage2_pte_cacheable(kvm_pte_t pte) > +static bool stage2_pte_cacheable(kvm_pte_t pte, struct kvm_pgtable *pgt) Same comment about pgt being the first argument. > { > u64 memattr = pte & KVM_PTE_LEAF_ATTR_LO_S2_MEMATTR; > - return memattr == PAGE_S2_MEMATTR(NORMAL); > + return memattr == PAGE_S2_MEMATTR(NORMAL, stage2_has_fwb(pgt)); > } > > static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, > @@ -821,8 +831,8 @@ static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, > > if (mm_ops->page_count(childp) != 1) > return 0; > - } else if (stage2_pte_cacheable(pte)) { > - need_flush = true; > + } else if (stage2_pte_cacheable(pte, pgt)) { > + need_flush = !stage2_has_fwb(pgt); > } > > /* > @@ -979,10 +989,11 @@ static int stage2_flush_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, > enum kvm_pgtable_walk_flags flag, > void * const arg) > { > - struct kvm_pgtable_mm_ops *mm_ops = arg; > + struct kvm_pgtable *pgt = arg; > + struct kvm_pgtable_mm_ops *mm_ops = pgt->mm_ops; > kvm_pte_t pte = *ptep; > > - if (!kvm_pte_valid(pte) || !stage2_pte_cacheable(pte)) > + if (!kvm_pte_valid(pte) || !stage2_pte_cacheable(pte, pgt)) > return 0; > > stage2_flush_dcache(kvm_pte_follow(pte, mm_ops), kvm_granule_size(level)); > @@ -994,17 +1005,18 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size) > struct kvm_pgtable_walker walker = { > .cb = stage2_flush_walker, > .flags = KVM_PGTABLE_WALK_LEAF, > - .arg = pgt->mm_ops, > + .arg = pgt, > }; > > - if (cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) > + if (stage2_has_fwb(pgt)) > return 0; > > return kvm_pgtable_walk(pgt, addr, size, &walker); > } > > -int kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, > - struct kvm_pgtable_mm_ops *mm_ops) > +int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch, > + struct kvm_pgtable_mm_ops *mm_ops, > + enum kvm_pgtable_stage2_flags flags) > { > size_t pgd_sz; > u64 vtcr = arch->vtcr; > @@ -1017,6 +1029,7 @@ int kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, > if (!pgt->pgd) > return -ENOMEM; > > + pgt->flags = flags; Try and keep the initialisation order similar to the definition of the structure if possible. > pgt->ia_bits = ia_bits; > pgt->start_level = start_level; > pgt->mm_ops = mm_ops; > @@ -1101,7 +1114,7 @@ int kvm_pgtable_stage2_find_range(struct kvm_pgtable *pgt, u64 addr, > u32 level; > int ret; > > - ret = stage2_set_prot_attr(prot, &attr); > + ret = stage2_set_prot_attr(prot, &attr, pgt); > if (ret) > return ret; > attr &= KVM_PTE_LEAF_S2_COMPAT_MASK; > -- > 2.31.0.rc2.261.g7f71774620-goog > > Thanks, M. -- Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24AACC433E0 for ; Wed, 17 Mar 2021 14:41:41 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 5AA6C64F77 for ; Wed, 17 Mar 2021 14:41:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5AA6C64F77 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id D3EA24B5F2; Wed, 17 Mar 2021 10:41:39 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iItzteDM8yr0; Wed, 17 Mar 2021 10:41:38 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 782184B63A; Wed, 17 Mar 2021 10:41:38 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id BB68A4B5F2 for ; Wed, 17 Mar 2021 10:41:37 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AMzUtWY+LOwp for ; Wed, 17 Mar 2021 10:41:36 -0400 (EDT) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 4CB504B581 for ; Wed, 17 Mar 2021 10:41:36 -0400 (EDT) Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 01BC864F70; Wed, 17 Mar 2021 14:41:35 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lMXMa-002EI2-PA; Wed, 17 Mar 2021 14:41:32 +0000 Date: Wed, 17 Mar 2021 14:41:31 +0000 Message-ID: <87a6r1j10k.wl-maz@kernel.org> From: Marc Zyngier To: Quentin Perret Subject: Re: [PATCH 1/2] KVM: arm64: Introduce KVM_PGTABLE_S2_NOFWB Stage-2 flag In-Reply-To: <20210317141714.383046-2-qperret@google.com> References: <20210315143536.214621-34-qperret@google.com> <20210317141714.383046-1-qperret@google.com> <20210317141714.383046-2-qperret@google.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: qperret@google.com, catalin.marinas@arm.com, will@kernel.org, james.morse@arm.com, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, android-kvm@google.com, seanjc@google.com, linux-kernel@vger.kernel.org, robh+dt@kernel.org, linux-arm-kernel@lists.infradead.org, kernel-team@android.com, kvmarm@lists.cs.columbia.edu, tabba@google.com, ardb@kernel.org, mark.rutland@arm.com, dbrazdil@google.com, mate.toth-pal@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Cc: kernel-team@android.com, android-kvm@google.com, catalin.marinas@arm.com, mate.toth-pal@arm.com, tabba@google.com, linux-kernel@vger.kernel.org, robh+dt@kernel.org, linux-arm-kernel@lists.infradead.org, seanjc@google.com, will@kernel.org, kvmarm@lists.cs.columbia.edu X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu Hi Quentin, On Wed, 17 Mar 2021 14:17:13 +0000, Quentin Perret wrote: > > In order to further configure stage-2 page-tables, pass flags to the > init function using a new enum. > > The first of these flags allows to disable FWB even if the hardware > supports it as we will need to do so for the host stage-2. > > Signed-off-by: Quentin Perret > > --- > > One question is, do we want to use stage2_has_fwb() everywhere, including > guest-specific paths (e.g. kvm_arch_prepare_memory_region(), ...) ? > > That'd make this patch more intrusive, but would make the whole codebase > work with FWB enabled on a guest by guest basis. I don't see us use that > anytime soon (other than maybe debug of some sort?) but it'd be good to > have an agreement. I'm not sure how useful that would be. We fought long and hard to get FWB, and I can't see a good reason to disable it for guests unless the HW was buggy (but in which case that'd be for everyone). I'd rather keep the changes small for now (this whole series is invasive enough!). As for this patch, I only have a few cosmetic comments: > --- > arch/arm64/include/asm/kvm_pgtable.h | 19 +++++++++-- > arch/arm64/include/asm/pgtable-prot.h | 4 +-- > arch/arm64/kvm/hyp/pgtable.c | 49 +++++++++++++++++---------- > 3 files changed, 50 insertions(+), 22 deletions(-) > > diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h > index b93a2a3526ab..7382bdfb6284 100644 > --- a/arch/arm64/include/asm/kvm_pgtable.h > +++ b/arch/arm64/include/asm/kvm_pgtable.h > @@ -56,6 +56,15 @@ struct kvm_pgtable_mm_ops { > phys_addr_t (*virt_to_phys)(void *addr); > }; > > +/** > + * enum kvm_pgtable_stage2_flags - Stage-2 page-table flags. > + * @KVM_PGTABLE_S2_NOFWB: Don't enforce Normal-WB even if the CPUs have > + * ARM64_HAS_STAGE2_FWB. > + */ > +enum kvm_pgtable_stage2_flags { > + KVM_PGTABLE_S2_NOFWB = BIT(0), > +}; > + > /** > * struct kvm_pgtable - KVM page-table. > * @ia_bits: Maximum input address size, in bits. > @@ -72,6 +81,7 @@ struct kvm_pgtable { > > /* Stage-2 only */ > struct kvm_s2_mmu *mmu; > + enum kvm_pgtable_stage2_flags flags; > }; > > /** > @@ -201,11 +211,16 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift); > * @arch: Arch-specific KVM structure representing the guest virtual > * machine. > * @mm_ops: Memory management callbacks. > + * @flags: Stage-2 configuration flags. > * > * Return: 0 on success, negative error code on failure. > */ > -int kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, > - struct kvm_pgtable_mm_ops *mm_ops); > +int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch, > + struct kvm_pgtable_mm_ops *mm_ops, > + enum kvm_pgtable_stage2_flags flags); > + > +#define kvm_pgtable_stage2_init(pgt, arch, mm_ops) \ > + kvm_pgtable_stage2_init_flags(pgt, arch, mm_ops, 0) > > /** > * kvm_pgtable_stage2_destroy() - Destroy an unused guest stage-2 page-table. > diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h > index 046be789fbb4..beeb722a82d3 100644 > --- a/arch/arm64/include/asm/pgtable-prot.h > +++ b/arch/arm64/include/asm/pgtable-prot.h > @@ -72,10 +72,10 @@ extern bool arm64_use_ng_mappings; > #define PAGE_KERNEL_EXEC __pgprot(PROT_NORMAL & ~PTE_PXN) > #define PAGE_KERNEL_EXEC_CONT __pgprot((PROT_NORMAL & ~PTE_PXN) | PTE_CONT) > > -#define PAGE_S2_MEMATTR(attr) \ > +#define PAGE_S2_MEMATTR(attr, has_fwb) \ > ({ \ > u64 __val; \ > - if (cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) \ > + if (has_fwb) \ > __val = PTE_S2_MEMATTR(MT_S2_FWB_ ## attr); \ > else \ > __val = PTE_S2_MEMATTR(MT_S2_ ## attr); \ > diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c > index 3a971df278bd..dee8aaeaf13e 100644 > --- a/arch/arm64/kvm/hyp/pgtable.c > +++ b/arch/arm64/kvm/hyp/pgtable.c > @@ -507,12 +507,25 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift) > return vtcr; > } > > -static int stage2_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) > +static bool stage2_has_fwb(struct kvm_pgtable *pgt) > +{ > + if (!cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) > + return false; > + > + return !(pgt->flags & KVM_PGTABLE_S2_NOFWB); > +} > + > +static int stage2_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep, > + struct kvm_pgtable *pgt) nit: make pgt the first parameter, as it defines the context in which the rest applies. > { > bool device = prot & KVM_PGTABLE_PROT_DEVICE; > - kvm_pte_t attr = device ? PAGE_S2_MEMATTR(DEVICE_nGnRE) : > - PAGE_S2_MEMATTR(NORMAL); > u32 sh = KVM_PTE_LEAF_ATTR_LO_S2_SH_IS; > + kvm_pte_t attr; > + > + if (device) > + attr = PAGE_S2_MEMATTR(DEVICE_nGnRE, stage2_has_fwb(pgt)); > + else > + attr = PAGE_S2_MEMATTR(NORMAL, stage2_has_fwb(pgt)); Maybe define a new helper: #define KVM_S2_MEMATTR(pgt, attr) PAGE_S2_MEMATTR(attr, stage2_has_fwb(pgt)) to avoid the constant stage2_has_fwb() repetition. > > if (!(prot & KVM_PGTABLE_PROT_X)) > attr |= KVM_PTE_LEAF_ATTR_HI_S2_XN; > @@ -748,7 +761,7 @@ int kvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, > .arg = &map_data, > }; > > - ret = stage2_set_prot_attr(prot, &map_data.attr); > + ret = stage2_set_prot_attr(prot, &map_data.attr, pgt); > if (ret) > return ret; > > @@ -786,16 +799,13 @@ int kvm_pgtable_stage2_set_owner(struct kvm_pgtable *pgt, u64 addr, u64 size, > > static void stage2_flush_dcache(void *addr, u64 size) > { > - if (cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) > - return; > - > __flush_dcache_area(addr, size); > } Consider dropping the function altogether and use __flush_dcache_area directly (assuming the prototypes are identical). > > -static bool stage2_pte_cacheable(kvm_pte_t pte) > +static bool stage2_pte_cacheable(kvm_pte_t pte, struct kvm_pgtable *pgt) Same comment about pgt being the first argument. > { > u64 memattr = pte & KVM_PTE_LEAF_ATTR_LO_S2_MEMATTR; > - return memattr == PAGE_S2_MEMATTR(NORMAL); > + return memattr == PAGE_S2_MEMATTR(NORMAL, stage2_has_fwb(pgt)); > } > > static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, > @@ -821,8 +831,8 @@ static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, > > if (mm_ops->page_count(childp) != 1) > return 0; > - } else if (stage2_pte_cacheable(pte)) { > - need_flush = true; > + } else if (stage2_pte_cacheable(pte, pgt)) { > + need_flush = !stage2_has_fwb(pgt); > } > > /* > @@ -979,10 +989,11 @@ static int stage2_flush_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, > enum kvm_pgtable_walk_flags flag, > void * const arg) > { > - struct kvm_pgtable_mm_ops *mm_ops = arg; > + struct kvm_pgtable *pgt = arg; > + struct kvm_pgtable_mm_ops *mm_ops = pgt->mm_ops; > kvm_pte_t pte = *ptep; > > - if (!kvm_pte_valid(pte) || !stage2_pte_cacheable(pte)) > + if (!kvm_pte_valid(pte) || !stage2_pte_cacheable(pte, pgt)) > return 0; > > stage2_flush_dcache(kvm_pte_follow(pte, mm_ops), kvm_granule_size(level)); > @@ -994,17 +1005,18 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size) > struct kvm_pgtable_walker walker = { > .cb = stage2_flush_walker, > .flags = KVM_PGTABLE_WALK_LEAF, > - .arg = pgt->mm_ops, > + .arg = pgt, > }; > > - if (cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) > + if (stage2_has_fwb(pgt)) > return 0; > > return kvm_pgtable_walk(pgt, addr, size, &walker); > } > > -int kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, > - struct kvm_pgtable_mm_ops *mm_ops) > +int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch, > + struct kvm_pgtable_mm_ops *mm_ops, > + enum kvm_pgtable_stage2_flags flags) > { > size_t pgd_sz; > u64 vtcr = arch->vtcr; > @@ -1017,6 +1029,7 @@ int kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, > if (!pgt->pgd) > return -ENOMEM; > > + pgt->flags = flags; Try and keep the initialisation order similar to the definition of the structure if possible. > pgt->ia_bits = ia_bits; > pgt->start_level = start_level; > pgt->mm_ops = mm_ops; > @@ -1101,7 +1114,7 @@ int kvm_pgtable_stage2_find_range(struct kvm_pgtable *pgt, u64 addr, > u32 level; > int ret; > > - ret = stage2_set_prot_attr(prot, &attr); > + ret = stage2_set_prot_attr(prot, &attr, pgt); > if (ret) > return ret; > attr &= KVM_PTE_LEAF_S2_COMPAT_MASK; > -- > 2.31.0.rc2.261.g7f71774620-goog > > Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BE6AC433DB for ; Wed, 17 Mar 2021 14:48:05 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B790064F3E for ; Wed, 17 Mar 2021 14:48:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B790064F3E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Subject:Cc:To: From:Message-ID:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=H+V8Z+YaoKfbb9ROOSL6XVrfa05oqNPjI1xnVFWC17w=; b=b7ghj6Q07z7byB0Moz0+yet9Y oDDIi3kDPYzFzD3Z5CAJwgb8UZLe/bnTczrU3qUWOG8dAW4ePMEdOEAzMttePyDGiBvxQf1hzJzxv W8NUA7QiRmHaKw2hslGOizpSrdyAlcCMIsvEfDHthKA78JoVgNbFZkSs3gSaMV4KeE6V13vcjnjmn tE+YKiXnJytjhB208I59W2XrOG3dWxMxnalVQiFa5hbYpIconu5MpAMnfWryvhL0stGF8Bb/sxiZ9 Wj0DHzJEC16J9ZPmY+REJZ1zy+2nawIq4m4ejQw3TmsxWrNVUy8tuM+kprcb8uOA0sPDLB91iuS6V q2hXGl29A==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lMXQp-003L3Z-Tj; Wed, 17 Mar 2021 14:45:56 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lMXMe-003JDn-KY for linux-arm-kernel@lists.infradead.org; Wed, 17 Mar 2021 14:41:48 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 01BC864F70; Wed, 17 Mar 2021 14:41:35 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lMXMa-002EI2-PA; Wed, 17 Mar 2021 14:41:32 +0000 Date: Wed, 17 Mar 2021 14:41:31 +0000 Message-ID: <87a6r1j10k.wl-maz@kernel.org> From: Marc Zyngier To: Quentin Perret Cc: catalin.marinas@arm.com, will@kernel.org, james.morse@arm.com, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, android-kvm@google.com, seanjc@google.com, linux-kernel@vger.kernel.org, robh+dt@kernel.org, linux-arm-kernel@lists.infradead.org, kernel-team@android.com, kvmarm@lists.cs.columbia.edu, tabba@google.com, ardb@kernel.org, mark.rutland@arm.com, dbrazdil@google.com, mate.toth-pal@arm.com Subject: Re: [PATCH 1/2] KVM: arm64: Introduce KVM_PGTABLE_S2_NOFWB Stage-2 flag In-Reply-To: <20210317141714.383046-2-qperret@google.com> References: <20210315143536.214621-34-qperret@google.com> <20210317141714.383046-1-qperret@google.com> <20210317141714.383046-2-qperret@google.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: qperret@google.com, catalin.marinas@arm.com, will@kernel.org, james.morse@arm.com, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, android-kvm@google.com, seanjc@google.com, linux-kernel@vger.kernel.org, robh+dt@kernel.org, linux-arm-kernel@lists.infradead.org, kernel-team@android.com, kvmarm@lists.cs.columbia.edu, tabba@google.com, ardb@kernel.org, mark.rutland@arm.com, dbrazdil@google.com, mate.toth-pal@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210317_144137_466949_23C0D833 X-CRM114-Status: GOOD ( 38.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Quentin, On Wed, 17 Mar 2021 14:17:13 +0000, Quentin Perret wrote: > > In order to further configure stage-2 page-tables, pass flags to the > init function using a new enum. > > The first of these flags allows to disable FWB even if the hardware > supports it as we will need to do so for the host stage-2. > > Signed-off-by: Quentin Perret > > --- > > One question is, do we want to use stage2_has_fwb() everywhere, including > guest-specific paths (e.g. kvm_arch_prepare_memory_region(), ...) ? > > That'd make this patch more intrusive, but would make the whole codebase > work with FWB enabled on a guest by guest basis. I don't see us use that > anytime soon (other than maybe debug of some sort?) but it'd be good to > have an agreement. I'm not sure how useful that would be. We fought long and hard to get FWB, and I can't see a good reason to disable it for guests unless the HW was buggy (but in which case that'd be for everyone). I'd rather keep the changes small for now (this whole series is invasive enough!). As for this patch, I only have a few cosmetic comments: > --- > arch/arm64/include/asm/kvm_pgtable.h | 19 +++++++++-- > arch/arm64/include/asm/pgtable-prot.h | 4 +-- > arch/arm64/kvm/hyp/pgtable.c | 49 +++++++++++++++++---------- > 3 files changed, 50 insertions(+), 22 deletions(-) > > diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h > index b93a2a3526ab..7382bdfb6284 100644 > --- a/arch/arm64/include/asm/kvm_pgtable.h > +++ b/arch/arm64/include/asm/kvm_pgtable.h > @@ -56,6 +56,15 @@ struct kvm_pgtable_mm_ops { > phys_addr_t (*virt_to_phys)(void *addr); > }; > > +/** > + * enum kvm_pgtable_stage2_flags - Stage-2 page-table flags. > + * @KVM_PGTABLE_S2_NOFWB: Don't enforce Normal-WB even if the CPUs have > + * ARM64_HAS_STAGE2_FWB. > + */ > +enum kvm_pgtable_stage2_flags { > + KVM_PGTABLE_S2_NOFWB = BIT(0), > +}; > + > /** > * struct kvm_pgtable - KVM page-table. > * @ia_bits: Maximum input address size, in bits. > @@ -72,6 +81,7 @@ struct kvm_pgtable { > > /* Stage-2 only */ > struct kvm_s2_mmu *mmu; > + enum kvm_pgtable_stage2_flags flags; > }; > > /** > @@ -201,11 +211,16 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift); > * @arch: Arch-specific KVM structure representing the guest virtual > * machine. > * @mm_ops: Memory management callbacks. > + * @flags: Stage-2 configuration flags. > * > * Return: 0 on success, negative error code on failure. > */ > -int kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, > - struct kvm_pgtable_mm_ops *mm_ops); > +int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch, > + struct kvm_pgtable_mm_ops *mm_ops, > + enum kvm_pgtable_stage2_flags flags); > + > +#define kvm_pgtable_stage2_init(pgt, arch, mm_ops) \ > + kvm_pgtable_stage2_init_flags(pgt, arch, mm_ops, 0) > > /** > * kvm_pgtable_stage2_destroy() - Destroy an unused guest stage-2 page-table. > diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h > index 046be789fbb4..beeb722a82d3 100644 > --- a/arch/arm64/include/asm/pgtable-prot.h > +++ b/arch/arm64/include/asm/pgtable-prot.h > @@ -72,10 +72,10 @@ extern bool arm64_use_ng_mappings; > #define PAGE_KERNEL_EXEC __pgprot(PROT_NORMAL & ~PTE_PXN) > #define PAGE_KERNEL_EXEC_CONT __pgprot((PROT_NORMAL & ~PTE_PXN) | PTE_CONT) > > -#define PAGE_S2_MEMATTR(attr) \ > +#define PAGE_S2_MEMATTR(attr, has_fwb) \ > ({ \ > u64 __val; \ > - if (cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) \ > + if (has_fwb) \ > __val = PTE_S2_MEMATTR(MT_S2_FWB_ ## attr); \ > else \ > __val = PTE_S2_MEMATTR(MT_S2_ ## attr); \ > diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c > index 3a971df278bd..dee8aaeaf13e 100644 > --- a/arch/arm64/kvm/hyp/pgtable.c > +++ b/arch/arm64/kvm/hyp/pgtable.c > @@ -507,12 +507,25 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift) > return vtcr; > } > > -static int stage2_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) > +static bool stage2_has_fwb(struct kvm_pgtable *pgt) > +{ > + if (!cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) > + return false; > + > + return !(pgt->flags & KVM_PGTABLE_S2_NOFWB); > +} > + > +static int stage2_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep, > + struct kvm_pgtable *pgt) nit: make pgt the first parameter, as it defines the context in which the rest applies. > { > bool device = prot & KVM_PGTABLE_PROT_DEVICE; > - kvm_pte_t attr = device ? PAGE_S2_MEMATTR(DEVICE_nGnRE) : > - PAGE_S2_MEMATTR(NORMAL); > u32 sh = KVM_PTE_LEAF_ATTR_LO_S2_SH_IS; > + kvm_pte_t attr; > + > + if (device) > + attr = PAGE_S2_MEMATTR(DEVICE_nGnRE, stage2_has_fwb(pgt)); > + else > + attr = PAGE_S2_MEMATTR(NORMAL, stage2_has_fwb(pgt)); Maybe define a new helper: #define KVM_S2_MEMATTR(pgt, attr) PAGE_S2_MEMATTR(attr, stage2_has_fwb(pgt)) to avoid the constant stage2_has_fwb() repetition. > > if (!(prot & KVM_PGTABLE_PROT_X)) > attr |= KVM_PTE_LEAF_ATTR_HI_S2_XN; > @@ -748,7 +761,7 @@ int kvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, > .arg = &map_data, > }; > > - ret = stage2_set_prot_attr(prot, &map_data.attr); > + ret = stage2_set_prot_attr(prot, &map_data.attr, pgt); > if (ret) > return ret; > > @@ -786,16 +799,13 @@ int kvm_pgtable_stage2_set_owner(struct kvm_pgtable *pgt, u64 addr, u64 size, > > static void stage2_flush_dcache(void *addr, u64 size) > { > - if (cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) > - return; > - > __flush_dcache_area(addr, size); > } Consider dropping the function altogether and use __flush_dcache_area directly (assuming the prototypes are identical). > > -static bool stage2_pte_cacheable(kvm_pte_t pte) > +static bool stage2_pte_cacheable(kvm_pte_t pte, struct kvm_pgtable *pgt) Same comment about pgt being the first argument. > { > u64 memattr = pte & KVM_PTE_LEAF_ATTR_LO_S2_MEMATTR; > - return memattr == PAGE_S2_MEMATTR(NORMAL); > + return memattr == PAGE_S2_MEMATTR(NORMAL, stage2_has_fwb(pgt)); > } > > static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, > @@ -821,8 +831,8 @@ static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, > > if (mm_ops->page_count(childp) != 1) > return 0; > - } else if (stage2_pte_cacheable(pte)) { > - need_flush = true; > + } else if (stage2_pte_cacheable(pte, pgt)) { > + need_flush = !stage2_has_fwb(pgt); > } > > /* > @@ -979,10 +989,11 @@ static int stage2_flush_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, > enum kvm_pgtable_walk_flags flag, > void * const arg) > { > - struct kvm_pgtable_mm_ops *mm_ops = arg; > + struct kvm_pgtable *pgt = arg; > + struct kvm_pgtable_mm_ops *mm_ops = pgt->mm_ops; > kvm_pte_t pte = *ptep; > > - if (!kvm_pte_valid(pte) || !stage2_pte_cacheable(pte)) > + if (!kvm_pte_valid(pte) || !stage2_pte_cacheable(pte, pgt)) > return 0; > > stage2_flush_dcache(kvm_pte_follow(pte, mm_ops), kvm_granule_size(level)); > @@ -994,17 +1005,18 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size) > struct kvm_pgtable_walker walker = { > .cb = stage2_flush_walker, > .flags = KVM_PGTABLE_WALK_LEAF, > - .arg = pgt->mm_ops, > + .arg = pgt, > }; > > - if (cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) > + if (stage2_has_fwb(pgt)) > return 0; > > return kvm_pgtable_walk(pgt, addr, size, &walker); > } > > -int kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, > - struct kvm_pgtable_mm_ops *mm_ops) > +int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch, > + struct kvm_pgtable_mm_ops *mm_ops, > + enum kvm_pgtable_stage2_flags flags) > { > size_t pgd_sz; > u64 vtcr = arch->vtcr; > @@ -1017,6 +1029,7 @@ int kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, > if (!pgt->pgd) > return -ENOMEM; > > + pgt->flags = flags; Try and keep the initialisation order similar to the definition of the structure if possible. > pgt->ia_bits = ia_bits; > pgt->start_level = start_level; > pgt->mm_ops = mm_ops; > @@ -1101,7 +1114,7 @@ int kvm_pgtable_stage2_find_range(struct kvm_pgtable *pgt, u64 addr, > u32 level; > int ret; > > - ret = stage2_set_prot_attr(prot, &attr); > + ret = stage2_set_prot_attr(prot, &attr, pgt); > if (ret) > return ret; > attr &= KVM_PTE_LEAF_S2_COMPAT_MASK; > -- > 2.31.0.rc2.261.g7f71774620-goog > > Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel