From: Quentin Perret <qperret@google.com>
To: catalin.marinas@arm.com, will@kernel.org, maz@kernel.org,
james.morse@arm.com, julien.thierry.kdev@gmail.com,
suzuki.poulose@arm.com
Cc: android-kvm@google.com, seanjc@google.com, mate.toth-pal@arm.com,
linux-kernel@vger.kernel.org, robh+dt@kernel.org,
linux-arm-kernel@lists.infradead.org, kernel-team@android.com,
kvmarm@lists.cs.columbia.edu, tabba@google.com, ardb@kernel.org,
mark.rutland@arm.com, dbrazdil@google.com, qperret@google.com
Subject: [PATCH v6 28/38] KVM: arm64: Always zero invalid PTEs
Date: Fri, 19 Mar 2021 10:01:36 +0000 [thread overview]
Message-ID: <20210319100146.1149909-29-qperret@google.com> (raw)
In-Reply-To: <20210319100146.1149909-1-qperret@google.com>
kvm_set_invalid_pte() currently only clears bit 0 from a PTE because
stage2_map_walk_table_post() needs to be able to follow the anchor. In
preparation for re-using bits 63-01 from invalid PTEs, make sure to zero
it entirely by ensuring to cache the anchor's child upfront.
Acked-by: Will Deacon <will@kernel.org>
Suggested-by: Will Deacon <will@kernel.org>
Signed-off-by: Quentin Perret <qperret@google.com>
---
arch/arm64/kvm/hyp/pgtable.c | 26 ++++++++++++++++----------
1 file changed, 16 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c
index 15de1708cfcd..0a674010afb6 100644
--- a/arch/arm64/kvm/hyp/pgtable.c
+++ b/arch/arm64/kvm/hyp/pgtable.c
@@ -156,10 +156,9 @@ static kvm_pte_t *kvm_pte_follow(kvm_pte_t pte, struct kvm_pgtable_mm_ops *mm_op
return mm_ops->phys_to_virt(kvm_pte_to_phys(pte));
}
-static void kvm_set_invalid_pte(kvm_pte_t *ptep)
+static void kvm_clear_pte(kvm_pte_t *ptep)
{
- kvm_pte_t pte = *ptep;
- WRITE_ONCE(*ptep, pte & ~KVM_PTE_VALID);
+ WRITE_ONCE(*ptep, 0);
}
static void kvm_set_table_pte(kvm_pte_t *ptep, kvm_pte_t *childp,
@@ -444,6 +443,7 @@ struct stage2_map_data {
kvm_pte_t attr;
kvm_pte_t *anchor;
+ kvm_pte_t *childp;
struct kvm_s2_mmu *mmu;
void *memcache;
@@ -533,7 +533,7 @@ static int stage2_map_walker_try_leaf(u64 addr, u64 end, u32 level,
* There's an existing different valid leaf entry, so perform
* break-before-make.
*/
- kvm_set_invalid_pte(ptep);
+ kvm_clear_pte(ptep);
kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, data->mmu, addr, level);
mm_ops->put_page(ptep);
}
@@ -554,7 +554,8 @@ static int stage2_map_walk_table_pre(u64 addr, u64 end, u32 level,
if (!kvm_block_mapping_supported(addr, end, data->phys, level))
return 0;
- kvm_set_invalid_pte(ptep);
+ data->childp = kvm_pte_follow(*ptep, data->mm_ops);
+ kvm_clear_pte(ptep);
/*
* Invalidate the whole stage-2, as we may have numerous leaf
@@ -600,7 +601,7 @@ static int stage2_map_walk_leaf(u64 addr, u64 end, u32 level, kvm_pte_t *ptep,
* will be mapped lazily.
*/
if (kvm_pte_valid(pte)) {
- kvm_set_invalid_pte(ptep);
+ kvm_clear_pte(ptep);
kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, data->mmu, addr, level);
mm_ops->put_page(ptep);
}
@@ -616,19 +617,24 @@ static int stage2_map_walk_table_post(u64 addr, u64 end, u32 level,
struct stage2_map_data *data)
{
struct kvm_pgtable_mm_ops *mm_ops = data->mm_ops;
+ kvm_pte_t *childp;
int ret = 0;
if (!data->anchor)
return 0;
- mm_ops->put_page(kvm_pte_follow(*ptep, mm_ops));
- mm_ops->put_page(ptep);
-
if (data->anchor == ptep) {
+ childp = data->childp;
data->anchor = NULL;
+ data->childp = NULL;
ret = stage2_map_walk_leaf(addr, end, level, ptep, data);
+ } else {
+ childp = kvm_pte_follow(*ptep, mm_ops);
}
+ mm_ops->put_page(childp);
+ mm_ops->put_page(ptep);
+
return ret;
}
@@ -737,7 +743,7 @@ static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep,
* block entry and rely on the remaining portions being faulted
* back lazily.
*/
- kvm_set_invalid_pte(ptep);
+ kvm_clear_pte(ptep);
kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, mmu, addr, level);
mm_ops->put_page(ptep);
--
2.31.0.rc2.261.g7f71774620-goog
next prev parent reply other threads:[~2021-03-19 10:03 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-19 10:01 [PATCH v6 00/38] KVM: arm64: Stage-2 for the host Quentin Perret
2021-03-19 10:01 ` [PATCH v6 01/38] arm64: lib: Annotate {clear,copy}_page() as position-independent Quentin Perret
2021-03-19 10:01 ` [PATCH v6 02/38] KVM: arm64: Link position-independent string routines into .hyp.text Quentin Perret
2021-03-19 10:01 ` [PATCH v6 03/38] arm64: kvm: Add standalone ticket spinlock implementation for use at hyp Quentin Perret
2021-03-19 10:01 ` [PATCH v6 04/38] KVM: arm64: Initialize kvm_nvhe_init_params early Quentin Perret
2021-03-19 10:01 ` [PATCH v6 05/38] KVM: arm64: Avoid free_page() in page-table allocator Quentin Perret
2021-03-19 10:01 ` [PATCH v6 06/38] KVM: arm64: Factor memory allocation out of pgtable.c Quentin Perret
2021-03-19 10:01 ` [PATCH v6 07/38] KVM: arm64: Introduce a BSS section for use at Hyp Quentin Perret
2021-03-19 10:01 ` [PATCH v6 08/38] KVM: arm64: Make kvm_call_hyp() a function call " Quentin Perret
2021-03-19 10:01 ` [PATCH v6 09/38] KVM: arm64: Allow using kvm_nvhe_sym() in hyp code Quentin Perret
2021-03-19 10:01 ` [PATCH v6 10/38] KVM: arm64: Introduce an early Hyp page allocator Quentin Perret
2021-03-19 10:01 ` [PATCH v6 11/38] KVM: arm64: Stub CONFIG_DEBUG_LIST at Hyp Quentin Perret
2021-03-19 10:01 ` [PATCH v6 12/38] KVM: arm64: Introduce a Hyp buddy page allocator Quentin Perret
2021-03-19 10:01 ` [PATCH v6 13/38] KVM: arm64: Enable access to sanitized CPU features at EL2 Quentin Perret
2021-03-22 11:24 ` Will Deacon
2021-03-22 13:44 ` Marc Zyngier
2021-03-22 14:19 ` Quentin Perret
2021-03-19 10:01 ` [PATCH v6 14/38] KVM: arm64: Provide __flush_dcache_area " Quentin Perret
2021-03-22 11:25 ` Will Deacon
2021-03-19 10:01 ` [PATCH v6 15/38] KVM: arm64: Factor out vector address calculation Quentin Perret
2021-03-19 10:01 ` [PATCH v6 16/38] arm64: asm: Provide set_sctlr_el2 macro Quentin Perret
2021-03-19 10:01 ` [PATCH v6 17/38] KVM: arm64: Prepare the creation of s1 mappings at EL2 Quentin Perret
2021-03-19 10:01 ` [PATCH v6 18/38] KVM: arm64: Elevate hypervisor mappings creation " Quentin Perret
2021-03-19 10:01 ` [PATCH v6 19/38] KVM: arm64: Use kvm_arch for stage 2 pgtable Quentin Perret
2021-03-19 10:01 ` [PATCH v6 20/38] KVM: arm64: Use kvm_arch in kvm_s2_mmu Quentin Perret
2021-03-19 10:01 ` [PATCH v6 21/38] KVM: arm64: Set host stage 2 using kvm_nvhe_init_params Quentin Perret
2021-03-19 10:01 ` [PATCH v6 22/38] KVM: arm64: Refactor kvm_arm_setup_stage2() Quentin Perret
2021-03-19 10:01 ` [PATCH v6 23/38] KVM: arm64: Refactor __load_guest_stage2() Quentin Perret
2021-03-19 10:01 ` [PATCH v6 24/38] KVM: arm64: Refactor __populate_fault_info() Quentin Perret
2021-03-19 10:01 ` [PATCH v6 25/38] KVM: arm64: Make memcache anonymous in pgtable allocator Quentin Perret
2021-03-19 10:01 ` [PATCH v6 26/38] KVM: arm64: Reserve memory for host stage 2 Quentin Perret
2021-03-19 10:01 ` [PATCH v6 27/38] KVM: arm64: Sort the hypervisor memblocks Quentin Perret
2021-03-19 10:01 ` Quentin Perret [this message]
2021-03-19 10:01 ` [PATCH v6 29/38] KVM: arm64: Use page-table to track page ownership Quentin Perret
2021-03-22 11:27 ` Will Deacon
2021-03-19 10:01 ` [PATCH v6 30/38] KVM: arm64: Refactor the *_map_set_prot_attr() helpers Quentin Perret
2021-03-19 10:01 ` [PATCH v6 31/38] KVM: arm64: Add kvm_pgtable_stage2_find_range() Quentin Perret
2021-03-19 10:01 ` [PATCH v6 32/38] KVM: arm64: Introduce KVM_PGTABLE_S2_NOFWB stage 2 flag Quentin Perret
2021-03-22 12:19 ` Will Deacon
2021-03-19 10:01 ` [PATCH v6 33/38] KVM: arm64: Introduce KVM_PGTABLE_S2_IDMAP " Quentin Perret
2021-03-22 11:33 ` Will Deacon
2021-03-19 10:01 ` [PATCH v6 34/38] KVM: arm64: Provide sanitized mmfr* registers at EL2 Quentin Perret
2021-03-19 10:01 ` [PATCH v6 35/38] KVM: arm64: Wrap the host with a stage 2 Quentin Perret
2021-03-19 10:01 ` [PATCH v6 36/38] KVM: arm64: Page-align the .hyp sections Quentin Perret
2021-03-19 10:01 ` [PATCH v6 37/38] KVM: arm64: Disable PMU support in protected mode Quentin Perret
2021-03-19 10:01 ` [PATCH v6 38/38] KVM: arm64: Protect the .hyp sections from the host Quentin Perret
2021-03-25 11:13 ` [PATCH v6 00/38] KVM: arm64: Stage-2 for " Marc Zyngier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210319100146.1149909-29-qperret@google.com \
--to=qperret@google.com \
--cc=android-kvm@google.com \
--cc=ardb@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=dbrazdil@google.com \
--cc=james.morse@arm.com \
--cc=julien.thierry.kdev@gmail.com \
--cc=kernel-team@android.com \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mate.toth-pal@arm.com \
--cc=maz@kernel.org \
--cc=robh+dt@kernel.org \
--cc=seanjc@google.com \
--cc=suzuki.poulose@arm.com \
--cc=tabba@google.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).