* [PATCH 5.15.y 0/2] fix softlockups in stage2_apply_range()
@ 2024-03-05 19:41 Krister Johansen
2024-03-05 19:41 ` [PATCH 5.15.y 1/2] KVM: arm64: Work out supported block level at compile time Krister Johansen
` (3 more replies)
0 siblings, 4 replies; 9+ messages in thread
From: Krister Johansen @ 2024-03-05 19:41 UTC (permalink / raw)
To: stable
Cc: Oliver Upton, Marc Zyngier, James Morse, Alexandru Elisei,
David Matlack, kvm
Hi Stable Team,
In 5.15, unmapping large kvm vms on arm64 can generate softlockups. My team has
been hitting this when tearing down VMs > 100Gb in size.
Oliver fixed this with the attached patches. They've been in mainline since
6.1.
I tested on 5.15.150 with these patches applied. When they're present,
both the dirty_log_perf_test detailed in the second patch, and
kvm_page_table_test no longer generate softlockups when unmapping VMs
with large memory configurations.
Would you please consider these patches for inclusion in an upcoming 5.15
release?
Thanks,
-K
Oliver Upton (2):
KVM: arm64: Work out supported block level at compile time
KVM: arm64: Limit stage2_apply_range() batch size to largest block
arch/arm64/include/asm/kvm_pgtable.h | 18 +++++++++++++-----
arch/arm64/include/asm/stage2_pgtable.h | 20 --------------------
arch/arm64/kvm/mmu.c | 9 ++++++++-
3 files changed, 21 insertions(+), 26 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 5.15.y 1/2] KVM: arm64: Work out supported block level at compile time
2024-03-05 19:41 [PATCH 5.15.y 0/2] fix softlockups in stage2_apply_range() Krister Johansen
@ 2024-03-05 19:41 ` Krister Johansen
2024-03-05 19:41 ` [PATCH 5.15.y 2/2] KVM: arm64: Limit stage2_apply_range() batch size to largest block Krister Johansen
` (2 subsequent siblings)
3 siblings, 0 replies; 9+ messages in thread
From: Krister Johansen @ 2024-03-05 19:41 UTC (permalink / raw)
To: stable
Cc: Oliver Upton, Marc Zyngier, James Morse, Alexandru Elisei,
David Matlack, kvm
commit 3b5c082bbfa20d9a57924edd655bbe63fe98ab06 upstream.
Work out the minimum page table level where KVM supports block mappings
at compile time. While at it, rewrite the comment around supported block
mappings to directly describe what KVM supports instead of phrasing in
terms of what it does not.
Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20221007234151.461779-2-oliver.upton@linux.dev
Signed-off-by: Krister Johansen <kjlx@templeofstupid.com>
---
arch/arm64/include/asm/kvm_pgtable.h | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h
index 027783829584..87e782eec925 100644
--- a/arch/arm64/include/asm/kvm_pgtable.h
+++ b/arch/arm64/include/asm/kvm_pgtable.h
@@ -13,6 +13,18 @@
#define KVM_PGTABLE_MAX_LEVELS 4U
+/*
+ * The largest supported block sizes for KVM (no 52-bit PA support):
+ * - 4K (level 1): 1GB
+ * - 16K (level 2): 32MB
+ * - 64K (level 2): 512MB
+ */
+#ifdef CONFIG_ARM64_4K_PAGES
+#define KVM_PGTABLE_MIN_BLOCK_LEVEL 1U
+#else
+#define KVM_PGTABLE_MIN_BLOCK_LEVEL 2U
+#endif
+
static inline u64 kvm_get_parange(u64 mmfr0)
{
u64 parange = cpuid_feature_extract_unsigned_field(mmfr0,
@@ -58,11 +70,7 @@ static inline u64 kvm_granule_size(u32 level)
static inline bool kvm_level_supports_block_mapping(u32 level)
{
- /*
- * Reject invalid block mappings and don't bother with 4TB mappings for
- * 52-bit PAs.
- */
- return !(level == 0 || (PAGE_SIZE != SZ_4K && level == 1));
+ return level >= KVM_PGTABLE_MIN_BLOCK_LEVEL;
}
/**
--
2.25.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 5.15.y 2/2] KVM: arm64: Limit stage2_apply_range() batch size to largest block
2024-03-05 19:41 [PATCH 5.15.y 0/2] fix softlockups in stage2_apply_range() Krister Johansen
2024-03-05 19:41 ` [PATCH 5.15.y 1/2] KVM: arm64: Work out supported block level at compile time Krister Johansen
@ 2024-03-05 19:41 ` Krister Johansen
2024-03-05 19:49 ` [PATCH 5.15.y 0/2] fix softlockups in stage2_apply_range() Oliver Upton
2024-03-06 0:49 ` [PATCH 5.15.y v2 " Krister Johansen
3 siblings, 0 replies; 9+ messages in thread
From: Krister Johansen @ 2024-03-05 19:41 UTC (permalink / raw)
To: stable
Cc: Oliver Upton, Marc Zyngier, James Morse, Alexandru Elisei,
David Matlack, kvm
commit 5994bc9e05c2f8811f233aa434e391cd2783f0f5 upstream.
Presently stage2_apply_range() works on a batch of memory addressed by a
stage 2 root table entry for the VM. Depending on the IPA limit of the
VM and PAGE_SIZE of the host, this could address a massive range of
memory. Some examples:
4 level, 4K paging -> 512 GB batch size
3 level, 64K paging -> 4TB batch size
Unsurprisingly, working on such a large range of memory can lead to soft
lockups. When running dirty_log_perf_test:
./dirty_log_perf_test -m -2 -s anonymous_thp -b 4G -v 48
watchdog: BUG: soft lockup - CPU#0 stuck for 45s! [dirty_log_perf_:16703]
Modules linked in: vfat fat cdc_ether usbnet mii xhci_pci xhci_hcd sha3_generic gq(O)
CPU: 0 PID: 16703 Comm: dirty_log_perf_ Tainted: G O 6.0.0-smp-DEV #1
pstate: 80400009 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : dcache_clean_inval_poc+0x24/0x38
lr : clean_dcache_guest_page+0x28/0x4c
sp : ffff800021763990
pmr_save: 000000e0
x29: ffff800021763990 x28: 0000000000000005 x27: 0000000000000de0
x26: 0000000000000001 x25: 00400830b13bc77f x24: ffffad4f91ead9c0
x23: 0000000000000000 x22: ffff8000082ad9c8 x21: 0000fffafa7bc000
x20: ffffad4f9066ce50 x19: 0000000000000003 x18: ffffad4f92402000
x17: 000000000000011b x16: 000000000000011b x15: 0000000000000124
x14: ffff07ff8301d280 x13: 0000000000000000 x12: 00000000ffffffff
x11: 0000000000010001 x10: fffffc0000000000 x9 : ffffad4f9069e580
x8 : 000000000000000c x7 : 0000000000000000 x6 : 000000000000003f
x5 : ffff07ffa2076980 x4 : 0000000000000001 x3 : 000000000000003f
x2 : 0000000000000040 x1 : ffff0830313bd000 x0 : ffff0830313bcc40
Call trace:
dcache_clean_inval_poc+0x24/0x38
stage2_unmap_walker+0x138/0x1ec
__kvm_pgtable_walk+0x130/0x1d4
__kvm_pgtable_walk+0x170/0x1d4
__kvm_pgtable_walk+0x170/0x1d4
__kvm_pgtable_walk+0x170/0x1d4
kvm_pgtable_stage2_unmap+0xc4/0xf8
kvm_arch_flush_shadow_memslot+0xa4/0x10c
kvm_set_memslot+0xb8/0x454
__kvm_set_memory_region+0x194/0x244
kvm_vm_ioctl_set_memory_region+0x58/0x7c
kvm_vm_ioctl+0x49c/0x560
__arm64_sys_ioctl+0x9c/0xd4
invoke_syscall+0x4c/0x124
el0_svc_common+0xc8/0x194
do_el0_svc+0x38/0xc0
el0_svc+0x2c/0xa4
el0t_64_sync_handler+0x84/0xf0
el0t_64_sync+0x1a0/0x1a4
Use the largest supported block mapping for the configured page size as
the batch granularity. In so doing the walker is guaranteed to visit a
leaf only once.
Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20221007234151.461779-3-oliver.upton@linux.dev
Signed-off-by: Krister Johansen <kjlx@templeofstupid.com>
---
arch/arm64/include/asm/stage2_pgtable.h | 20 --------------------
arch/arm64/kvm/mmu.c | 9 ++++++++-
2 files changed, 8 insertions(+), 21 deletions(-)
diff --git a/arch/arm64/include/asm/stage2_pgtable.h b/arch/arm64/include/asm/stage2_pgtable.h
index fe341a6578c3..c8dca8ae359c 100644
--- a/arch/arm64/include/asm/stage2_pgtable.h
+++ b/arch/arm64/include/asm/stage2_pgtable.h
@@ -10,13 +10,6 @@
#include <linux/pgtable.h>
-/*
- * PGDIR_SHIFT determines the size a top-level page table entry can map
- * and depends on the number of levels in the page table. Compute the
- * PGDIR_SHIFT for a given number of levels.
- */
-#define pt_levels_pgdir_shift(lvls) ARM64_HW_PGTABLE_LEVEL_SHIFT(4 - (lvls))
-
/*
* The hardware supports concatenation of up to 16 tables at stage2 entry
* level and we use the feature whenever possible, which means we resolve 4
@@ -30,11 +23,6 @@
#define stage2_pgtable_levels(ipa) ARM64_HW_PGTABLE_LEVELS((ipa) - 4)
#define kvm_stage2_levels(kvm) VTCR_EL2_LVLS(kvm->arch.vtcr)
-/* stage2_pgdir_shift() is the size mapped by top-level stage2 entry for the VM */
-#define stage2_pgdir_shift(kvm) pt_levels_pgdir_shift(kvm_stage2_levels(kvm))
-#define stage2_pgdir_size(kvm) (1ULL << stage2_pgdir_shift(kvm))
-#define stage2_pgdir_mask(kvm) ~(stage2_pgdir_size(kvm) - 1)
-
/*
* kvm_mmmu_cache_min_pages() is the number of pages required to install
* a stage-2 translation. We pre-allocate the entry level page table at
@@ -42,12 +30,4 @@
*/
#define kvm_mmu_cache_min_pages(kvm) (kvm_stage2_levels(kvm) - 1)
-static inline phys_addr_t
-stage2_pgd_addr_end(struct kvm *kvm, phys_addr_t addr, phys_addr_t end)
-{
- phys_addr_t boundary = (addr + stage2_pgdir_size(kvm)) & stage2_pgdir_mask(kvm);
-
- return (boundary - 1 < end - 1) ? boundary : end;
-}
-
#endif /* __ARM64_S2_PGTABLE_H_ */
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 38a8095744a0..db667b4ad103 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -31,6 +31,13 @@ static phys_addr_t hyp_idmap_vector;
static unsigned long io_map_base;
+static phys_addr_t stage2_range_addr_end(phys_addr_t addr, phys_addr_t end)
+{
+ phys_addr_t size = kvm_granule_size(KVM_PGTABLE_MIN_BLOCK_LEVEL);
+ phys_addr_t boundary = ALIGN_DOWN(addr + size, size);
+
+ return (boundary - 1 < end - 1) ? boundary : end;
+}
/*
* Release kvm_mmu_lock periodically if the memory region is large. Otherwise,
@@ -52,7 +59,7 @@ static int stage2_apply_range(struct kvm *kvm, phys_addr_t addr,
if (!pgt)
return -EINVAL;
- next = stage2_pgd_addr_end(kvm, addr, end);
+ next = stage2_range_addr_end(addr, end);
ret = fn(pgt, addr, next - addr);
if (ret)
break;
--
2.25.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 5.15.y 0/2] fix softlockups in stage2_apply_range()
2024-03-05 19:41 [PATCH 5.15.y 0/2] fix softlockups in stage2_apply_range() Krister Johansen
2024-03-05 19:41 ` [PATCH 5.15.y 1/2] KVM: arm64: Work out supported block level at compile time Krister Johansen
2024-03-05 19:41 ` [PATCH 5.15.y 2/2] KVM: arm64: Limit stage2_apply_range() batch size to largest block Krister Johansen
@ 2024-03-05 19:49 ` Oliver Upton
2024-03-06 0:49 ` [PATCH 5.15.y v2 " Krister Johansen
3 siblings, 0 replies; 9+ messages in thread
From: Oliver Upton @ 2024-03-05 19:49 UTC (permalink / raw)
To: Krister Johansen
Cc: stable, Marc Zyngier, James Morse, Alexandru Elisei, David Matlack, kvm
On Tue, Mar 05, 2024 at 11:41:38AM -0800, Krister Johansen wrote:
> Hi Stable Team,
> In 5.15, unmapping large kvm vms on arm64 can generate softlockups. My team has
> been hitting this when tearing down VMs > 100Gb in size.
>
> Oliver fixed this with the attached patches. They've been in mainline since
> 6.1.
>
> I tested on 5.15.150 with these patches applied. When they're present,
> both the dirty_log_perf_test detailed in the second patch, and
> kvm_page_table_test no longer generate softlockups when unmapping VMs
> with large memory configurations.
>
> Would you please consider these patches for inclusion in an upcoming 5.15
> release?
Backport looks fine, and I have no issues with this going to stable if
it helps folks.
Acked-by: Oliver Upton <oliver.upton@linux.dev>
--
Thanks,
Oliver
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 5.15.y v2 0/2] fix softlockups in stage2_apply_range()
2024-03-05 19:41 [PATCH 5.15.y 0/2] fix softlockups in stage2_apply_range() Krister Johansen
` (2 preceding siblings ...)
2024-03-05 19:49 ` [PATCH 5.15.y 0/2] fix softlockups in stage2_apply_range() Oliver Upton
@ 2024-03-06 0:49 ` Krister Johansen
2024-03-06 0:49 ` [PATCH 5.15.y v2 1/2] KVM: arm64: Work out supported block level at compile time Krister Johansen
` (2 more replies)
3 siblings, 3 replies; 9+ messages in thread
From: Krister Johansen @ 2024-03-06 0:49 UTC (permalink / raw)
To: stable
Cc: Oliver Upton, Marc Zyngier, James Morse, Alexandru Elisei,
David Matlack, kvm
Hi Stable Team,
In 5.15, unmapping large kvm vms on arm64 can generate softlockups. My team has
been hitting this when tearing down VMs > 100Gb in size.
Oliver fixed this with the attached patches. They've been in mainline since
6.1.
I tested on 5.15.150 with these patches applied. When they're present,
both the dirty_log_perf_test detailed in the second patch, and
kvm_page_table_test no longer generate softlockups when unmapping VMs
with large memory configurations.
Would you please consider these patches for inclusion in an upcoming 5.15
release?
Change in v2: I ran format-patch without the --from option which incorrectly
generated the first series without leaving Oliver in place as the author. The
v2 should retain the correct authorship. Apologies for the mistake.
-K
Oliver Upton (2):
KVM: arm64: Work out supported block level at compile time
KVM: arm64: Limit stage2_apply_range() batch size to largest block
arch/arm64/include/asm/kvm_pgtable.h | 18 +++++++++++++-----
arch/arm64/include/asm/stage2_pgtable.h | 20 --------------------
arch/arm64/kvm/mmu.c | 9 ++++++++-
3 files changed, 21 insertions(+), 26 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 5.15.y v2 1/2] KVM: arm64: Work out supported block level at compile time
2024-03-06 0:49 ` [PATCH 5.15.y v2 " Krister Johansen
@ 2024-03-06 0:49 ` Krister Johansen
2024-03-06 0:49 ` [PATCH 5.15.y v2 2/2] KVM: arm64: Limit stage2_apply_range() batch size to largest block Krister Johansen
2024-03-06 10:08 ` [PATCH 5.15.y v2 0/2] fix softlockups in stage2_apply_range() Marc Zyngier
2 siblings, 0 replies; 9+ messages in thread
From: Krister Johansen @ 2024-03-06 0:49 UTC (permalink / raw)
To: stable
Cc: Oliver Upton, Marc Zyngier, James Morse, Alexandru Elisei,
David Matlack, kvm
From: Oliver Upton <oliver.upton@linux.dev>
commit 3b5c082bbfa20d9a57924edd655bbe63fe98ab06 upstream.
Work out the minimum page table level where KVM supports block mappings
at compile time. While at it, rewrite the comment around supported block
mappings to directly describe what KVM supports instead of phrasing in
terms of what it does not.
Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20221007234151.461779-2-oliver.upton@linux.dev
Signed-off-by: Krister Johansen <kjlx@templeofstupid.com>
---
arch/arm64/include/asm/kvm_pgtable.h | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h
index 027783829584..87e782eec925 100644
--- a/arch/arm64/include/asm/kvm_pgtable.h
+++ b/arch/arm64/include/asm/kvm_pgtable.h
@@ -13,6 +13,18 @@
#define KVM_PGTABLE_MAX_LEVELS 4U
+/*
+ * The largest supported block sizes for KVM (no 52-bit PA support):
+ * - 4K (level 1): 1GB
+ * - 16K (level 2): 32MB
+ * - 64K (level 2): 512MB
+ */
+#ifdef CONFIG_ARM64_4K_PAGES
+#define KVM_PGTABLE_MIN_BLOCK_LEVEL 1U
+#else
+#define KVM_PGTABLE_MIN_BLOCK_LEVEL 2U
+#endif
+
static inline u64 kvm_get_parange(u64 mmfr0)
{
u64 parange = cpuid_feature_extract_unsigned_field(mmfr0,
@@ -58,11 +70,7 @@ static inline u64 kvm_granule_size(u32 level)
static inline bool kvm_level_supports_block_mapping(u32 level)
{
- /*
- * Reject invalid block mappings and don't bother with 4TB mappings for
- * 52-bit PAs.
- */
- return !(level == 0 || (PAGE_SIZE != SZ_4K && level == 1));
+ return level >= KVM_PGTABLE_MIN_BLOCK_LEVEL;
}
/**
--
2.25.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 5.15.y v2 2/2] KVM: arm64: Limit stage2_apply_range() batch size to largest block
2024-03-06 0:49 ` [PATCH 5.15.y v2 " Krister Johansen
2024-03-06 0:49 ` [PATCH 5.15.y v2 1/2] KVM: arm64: Work out supported block level at compile time Krister Johansen
@ 2024-03-06 0:49 ` Krister Johansen
2024-03-06 10:08 ` [PATCH 5.15.y v2 0/2] fix softlockups in stage2_apply_range() Marc Zyngier
2 siblings, 0 replies; 9+ messages in thread
From: Krister Johansen @ 2024-03-06 0:49 UTC (permalink / raw)
To: stable
Cc: Oliver Upton, Marc Zyngier, James Morse, Alexandru Elisei,
David Matlack, kvm
From: Oliver Upton <oliver.upton@linux.dev>
commit 5994bc9e05c2f8811f233aa434e391cd2783f0f5 upstream.
Presently stage2_apply_range() works on a batch of memory addressed by a
stage 2 root table entry for the VM. Depending on the IPA limit of the
VM and PAGE_SIZE of the host, this could address a massive range of
memory. Some examples:
4 level, 4K paging -> 512 GB batch size
3 level, 64K paging -> 4TB batch size
Unsurprisingly, working on such a large range of memory can lead to soft
lockups. When running dirty_log_perf_test:
./dirty_log_perf_test -m -2 -s anonymous_thp -b 4G -v 48
watchdog: BUG: soft lockup - CPU#0 stuck for 45s! [dirty_log_perf_:16703]
Modules linked in: vfat fat cdc_ether usbnet mii xhci_pci xhci_hcd sha3_generic gq(O)
CPU: 0 PID: 16703 Comm: dirty_log_perf_ Tainted: G O 6.0.0-smp-DEV #1
pstate: 80400009 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : dcache_clean_inval_poc+0x24/0x38
lr : clean_dcache_guest_page+0x28/0x4c
sp : ffff800021763990
pmr_save: 000000e0
x29: ffff800021763990 x28: 0000000000000005 x27: 0000000000000de0
x26: 0000000000000001 x25: 00400830b13bc77f x24: ffffad4f91ead9c0
x23: 0000000000000000 x22: ffff8000082ad9c8 x21: 0000fffafa7bc000
x20: ffffad4f9066ce50 x19: 0000000000000003 x18: ffffad4f92402000
x17: 000000000000011b x16: 000000000000011b x15: 0000000000000124
x14: ffff07ff8301d280 x13: 0000000000000000 x12: 00000000ffffffff
x11: 0000000000010001 x10: fffffc0000000000 x9 : ffffad4f9069e580
x8 : 000000000000000c x7 : 0000000000000000 x6 : 000000000000003f
x5 : ffff07ffa2076980 x4 : 0000000000000001 x3 : 000000000000003f
x2 : 0000000000000040 x1 : ffff0830313bd000 x0 : ffff0830313bcc40
Call trace:
dcache_clean_inval_poc+0x24/0x38
stage2_unmap_walker+0x138/0x1ec
__kvm_pgtable_walk+0x130/0x1d4
__kvm_pgtable_walk+0x170/0x1d4
__kvm_pgtable_walk+0x170/0x1d4
__kvm_pgtable_walk+0x170/0x1d4
kvm_pgtable_stage2_unmap+0xc4/0xf8
kvm_arch_flush_shadow_memslot+0xa4/0x10c
kvm_set_memslot+0xb8/0x454
__kvm_set_memory_region+0x194/0x244
kvm_vm_ioctl_set_memory_region+0x58/0x7c
kvm_vm_ioctl+0x49c/0x560
__arm64_sys_ioctl+0x9c/0xd4
invoke_syscall+0x4c/0x124
el0_svc_common+0xc8/0x194
do_el0_svc+0x38/0xc0
el0_svc+0x2c/0xa4
el0t_64_sync_handler+0x84/0xf0
el0t_64_sync+0x1a0/0x1a4
Use the largest supported block mapping for the configured page size as
the batch granularity. In so doing the walker is guaranteed to visit a
leaf only once.
Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20221007234151.461779-3-oliver.upton@linux.dev
Signed-off-by: Krister Johansen <kjlx@templeofstupid.com>
---
arch/arm64/include/asm/stage2_pgtable.h | 20 --------------------
arch/arm64/kvm/mmu.c | 9 ++++++++-
2 files changed, 8 insertions(+), 21 deletions(-)
diff --git a/arch/arm64/include/asm/stage2_pgtable.h b/arch/arm64/include/asm/stage2_pgtable.h
index fe341a6578c3..c8dca8ae359c 100644
--- a/arch/arm64/include/asm/stage2_pgtable.h
+++ b/arch/arm64/include/asm/stage2_pgtable.h
@@ -10,13 +10,6 @@
#include <linux/pgtable.h>
-/*
- * PGDIR_SHIFT determines the size a top-level page table entry can map
- * and depends on the number of levels in the page table. Compute the
- * PGDIR_SHIFT for a given number of levels.
- */
-#define pt_levels_pgdir_shift(lvls) ARM64_HW_PGTABLE_LEVEL_SHIFT(4 - (lvls))
-
/*
* The hardware supports concatenation of up to 16 tables at stage2 entry
* level and we use the feature whenever possible, which means we resolve 4
@@ -30,11 +23,6 @@
#define stage2_pgtable_levels(ipa) ARM64_HW_PGTABLE_LEVELS((ipa) - 4)
#define kvm_stage2_levels(kvm) VTCR_EL2_LVLS(kvm->arch.vtcr)
-/* stage2_pgdir_shift() is the size mapped by top-level stage2 entry for the VM */
-#define stage2_pgdir_shift(kvm) pt_levels_pgdir_shift(kvm_stage2_levels(kvm))
-#define stage2_pgdir_size(kvm) (1ULL << stage2_pgdir_shift(kvm))
-#define stage2_pgdir_mask(kvm) ~(stage2_pgdir_size(kvm) - 1)
-
/*
* kvm_mmmu_cache_min_pages() is the number of pages required to install
* a stage-2 translation. We pre-allocate the entry level page table at
@@ -42,12 +30,4 @@
*/
#define kvm_mmu_cache_min_pages(kvm) (kvm_stage2_levels(kvm) - 1)
-static inline phys_addr_t
-stage2_pgd_addr_end(struct kvm *kvm, phys_addr_t addr, phys_addr_t end)
-{
- phys_addr_t boundary = (addr + stage2_pgdir_size(kvm)) & stage2_pgdir_mask(kvm);
-
- return (boundary - 1 < end - 1) ? boundary : end;
-}
-
#endif /* __ARM64_S2_PGTABLE_H_ */
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 38a8095744a0..db667b4ad103 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -31,6 +31,13 @@ static phys_addr_t hyp_idmap_vector;
static unsigned long io_map_base;
+static phys_addr_t stage2_range_addr_end(phys_addr_t addr, phys_addr_t end)
+{
+ phys_addr_t size = kvm_granule_size(KVM_PGTABLE_MIN_BLOCK_LEVEL);
+ phys_addr_t boundary = ALIGN_DOWN(addr + size, size);
+
+ return (boundary - 1 < end - 1) ? boundary : end;
+}
/*
* Release kvm_mmu_lock periodically if the memory region is large. Otherwise,
@@ -52,7 +59,7 @@ static int stage2_apply_range(struct kvm *kvm, phys_addr_t addr,
if (!pgt)
return -EINVAL;
- next = stage2_pgd_addr_end(kvm, addr, end);
+ next = stage2_range_addr_end(addr, end);
ret = fn(pgt, addr, next - addr);
if (ret)
break;
--
2.25.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 5.15.y v2 0/2] fix softlockups in stage2_apply_range()
2024-03-06 0:49 ` [PATCH 5.15.y v2 " Krister Johansen
2024-03-06 0:49 ` [PATCH 5.15.y v2 1/2] KVM: arm64: Work out supported block level at compile time Krister Johansen
2024-03-06 0:49 ` [PATCH 5.15.y v2 2/2] KVM: arm64: Limit stage2_apply_range() batch size to largest block Krister Johansen
@ 2024-03-06 10:08 ` Marc Zyngier
2024-03-29 12:21 ` Greg KH
2 siblings, 1 reply; 9+ messages in thread
From: Marc Zyngier @ 2024-03-06 10:08 UTC (permalink / raw)
To: Krister Johansen
Cc: stable, Oliver Upton, James Morse, Alexandru Elisei, David Matlack, kvm
On Wed, 06 Mar 2024 00:49:34 +0000,
Krister Johansen <kjlx@templeofstupid.com> wrote:
>
> Hi Stable Team,
> In 5.15, unmapping large kvm vms on arm64 can generate softlockups. My team has
> been hitting this when tearing down VMs > 100Gb in size.
>
> Oliver fixed this with the attached patches. They've been in mainline since
> 6.1.
>
> I tested on 5.15.150 with these patches applied. When they're present,
> both the dirty_log_perf_test detailed in the second patch, and
> kvm_page_table_test no longer generate softlockups when unmapping VMs
> with large memory configurations.
>
> Would you please consider these patches for inclusion in an upcoming 5.15
> release?
>
> Change in v2: I ran format-patch without the --from option which incorrectly
> generated the first series without leaving Oliver in place as the author. The
> v2 should retain the correct authorship. Apologies for the mistake.
Thanks for this.
FWIW,
Acked-by: Marc Zyngier <maz@kernel.org>
M.
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 5.15.y v2 0/2] fix softlockups in stage2_apply_range()
2024-03-06 10:08 ` [PATCH 5.15.y v2 0/2] fix softlockups in stage2_apply_range() Marc Zyngier
@ 2024-03-29 12:21 ` Greg KH
0 siblings, 0 replies; 9+ messages in thread
From: Greg KH @ 2024-03-29 12:21 UTC (permalink / raw)
To: Marc Zyngier
Cc: Krister Johansen, stable, Oliver Upton, James Morse,
Alexandru Elisei, David Matlack, kvm
On Wed, Mar 06, 2024 at 10:08:40AM +0000, Marc Zyngier wrote:
> On Wed, 06 Mar 2024 00:49:34 +0000,
> Krister Johansen <kjlx@templeofstupid.com> wrote:
> >
> > Hi Stable Team,
> > In 5.15, unmapping large kvm vms on arm64 can generate softlockups. My team has
> > been hitting this when tearing down VMs > 100Gb in size.
> >
> > Oliver fixed this with the attached patches. They've been in mainline since
> > 6.1.
> >
> > I tested on 5.15.150 with these patches applied. When they're present,
> > both the dirty_log_perf_test detailed in the second patch, and
> > kvm_page_table_test no longer generate softlockups when unmapping VMs
> > with large memory configurations.
> >
> > Would you please consider these patches for inclusion in an upcoming 5.15
> > release?
> >
> > Change in v2: I ran format-patch without the --from option which incorrectly
> > generated the first series without leaving Oliver in place as the author. The
> > v2 should retain the correct authorship. Apologies for the mistake.
>
> Thanks for this.
>
> FWIW,
>
> Acked-by: Marc Zyngier <maz@kernel.org>
Now queued up,t hanks.
greg k-h
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2024-03-29 12:21 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-03-05 19:41 [PATCH 5.15.y 0/2] fix softlockups in stage2_apply_range() Krister Johansen
2024-03-05 19:41 ` [PATCH 5.15.y 1/2] KVM: arm64: Work out supported block level at compile time Krister Johansen
2024-03-05 19:41 ` [PATCH 5.15.y 2/2] KVM: arm64: Limit stage2_apply_range() batch size to largest block Krister Johansen
2024-03-05 19:49 ` [PATCH 5.15.y 0/2] fix softlockups in stage2_apply_range() Oliver Upton
2024-03-06 0:49 ` [PATCH 5.15.y v2 " Krister Johansen
2024-03-06 0:49 ` [PATCH 5.15.y v2 1/2] KVM: arm64: Work out supported block level at compile time Krister Johansen
2024-03-06 0:49 ` [PATCH 5.15.y v2 2/2] KVM: arm64: Limit stage2_apply_range() batch size to largest block Krister Johansen
2024-03-06 10:08 ` [PATCH 5.15.y v2 0/2] fix softlockups in stage2_apply_range() Marc Zyngier
2024-03-29 12:21 ` Greg KH
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.