* [PATCH] KVM: PPC: Book3S HV: increase KVMPPC_NR_LPIDS on POWER8 and POWER9
@ 2020-06-08 11:57 Cédric Le Goater
2020-06-23 15:08 ` Cédric Le Goater
2020-07-23 6:20 ` Paul Mackerras
0 siblings, 2 replies; 4+ messages in thread
From: Cédric Le Goater @ 2020-06-08 11:57 UTC (permalink / raw)
To: Michael Ellerman
Cc: Paul Mackerras, Nicholas Piggin, linuxppc-dev, kvm-ppc, kvm,
Cédric Le Goater
POWER8 and POWER9 have 12-bit LPIDs. Change LPID_RSVD to support up to
(4096 - 2) guests on these processors. POWER7 is kept the same with a
limitation of (1024 - 2), but it might be time to drop KVM support for
POWER7.
Tested with 2048 guests * 4 vCPUs on a witherspoon system with 512G
RAM and a bit of swap.
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
arch/powerpc/include/asm/reg.h | 3 ++-
arch/powerpc/kvm/book3s_64_mmu_hv.c | 8 ++++++--
2 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
index 88e6c78100d9..b70bbfb0ea3c 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -473,7 +473,8 @@
#ifndef SPRN_LPID
#define SPRN_LPID 0x13F /* Logical Partition Identifier */
#endif
-#define LPID_RSVD 0x3ff /* Reserved LPID for partn switching */
+#define LPID_RSVD_POWER7 0x3ff /* Reserved LPID for partn switching */
+#define LPID_RSVD 0xfff /* Reserved LPID for partn switching */
#define SPRN_HMER 0x150 /* Hypervisor maintenance exception reg */
#define HMER_DEBUG_TRIG (1ul << (63 - 17)) /* Debug trigger */
#define SPRN_HMEER 0x151 /* Hyp maintenance exception enable reg */
diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
index 18aed9775a3c..23035ab2ec50 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
@@ -260,11 +260,15 @@ int kvmppc_mmu_hv_init(void)
if (!mmu_has_feature(MMU_FTR_LOCKLESS_TLBIE))
return -EINVAL;
- /* POWER7 has 10-bit LPIDs (12-bit in POWER8) */
host_lpid = 0;
if (cpu_has_feature(CPU_FTR_HVMODE))
host_lpid = mfspr(SPRN_LPID);
- rsvd_lpid = LPID_RSVD;
+
+ /* POWER8 and above have 12-bit LPIDs (10-bit in POWER7) */
+ if (cpu_has_feature(CPU_FTR_ARCH_207S))
+ rsvd_lpid = LPID_RSVD;
+ else
+ rsvd_lpid = LPID_RSVD_POWER7;
kvmppc_init_lpid(rsvd_lpid + 1);
--
2.25.4
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] KVM: PPC: Book3S HV: increase KVMPPC_NR_LPIDS on POWER8 and POWER9
2020-06-08 11:57 [PATCH] KVM: PPC: Book3S HV: increase KVMPPC_NR_LPIDS on POWER8 and POWER9 Cédric Le Goater
@ 2020-06-23 15:08 ` Cédric Le Goater
2020-07-23 6:20 ` Paul Mackerras
1 sibling, 0 replies; 4+ messages in thread
From: Cédric Le Goater @ 2020-06-23 15:08 UTC (permalink / raw)
To: Michael Ellerman
Cc: Paul Mackerras, Nicholas Piggin, linuxppc-dev, kvm-ppc, kvm
On 6/8/20 1:57 PM, Cédric Le Goater wrote:
> POWER8 and POWER9 have 12-bit LPIDs. Change LPID_RSVD to support up to
> (4096 - 2) guests on these processors. POWER7 is kept the same with a
> limitation of (1024 - 2), but it might be time to drop KVM support for
> POWER7.
>
> Tested with 2048 guests * 4 vCPUs on a witherspoon system with 512G
> RAM and a bit of swap.
For the record, it is possible to run 4094 guests * 4 vCPUs on a POWER9
system with 1TB. It takes ~5m to boot them all.
CONFIG_NR_IRQS needs to be increased to support 4094 * 4 escalation
interrupts.
Cheers,
C.
>
> Signed-off-by: Cédric Le Goater <clg@kaod.org>
> ---
> arch/powerpc/include/asm/reg.h | 3 ++-
> arch/powerpc/kvm/book3s_64_mmu_hv.c | 8 ++++++--
> 2 files changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
> index 88e6c78100d9..b70bbfb0ea3c 100644
> --- a/arch/powerpc/include/asm/reg.h
> +++ b/arch/powerpc/include/asm/reg.h
> @@ -473,7 +473,8 @@
> #ifndef SPRN_LPID
> #define SPRN_LPID 0x13F /* Logical Partition Identifier */
> #endif
> -#define LPID_RSVD 0x3ff /* Reserved LPID for partn switching */
> +#define LPID_RSVD_POWER7 0x3ff /* Reserved LPID for partn switching */
> +#define LPID_RSVD 0xfff /* Reserved LPID for partn switching */
> #define SPRN_HMER 0x150 /* Hypervisor maintenance exception reg */
> #define HMER_DEBUG_TRIG (1ul << (63 - 17)) /* Debug trigger */
> #define SPRN_HMEER 0x151 /* Hyp maintenance exception enable reg */
> diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> index 18aed9775a3c..23035ab2ec50 100644
> --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
> +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
> @@ -260,11 +260,15 @@ int kvmppc_mmu_hv_init(void)
> if (!mmu_has_feature(MMU_FTR_LOCKLESS_TLBIE))
> return -EINVAL;
>
> - /* POWER7 has 10-bit LPIDs (12-bit in POWER8) */
> host_lpid = 0;
> if (cpu_has_feature(CPU_FTR_HVMODE))
> host_lpid = mfspr(SPRN_LPID);
> - rsvd_lpid = LPID_RSVD;
> +
> + /* POWER8 and above have 12-bit LPIDs (10-bit in POWER7) */
> + if (cpu_has_feature(CPU_FTR_ARCH_207S))
> + rsvd_lpid = LPID_RSVD;
> + else
> + rsvd_lpid = LPID_RSVD_POWER7;
>
> kvmppc_init_lpid(rsvd_lpid + 1);
>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] KVM: PPC: Book3S HV: increase KVMPPC_NR_LPIDS on POWER8 and POWER9
2020-06-08 11:57 [PATCH] KVM: PPC: Book3S HV: increase KVMPPC_NR_LPIDS on POWER8 and POWER9 Cédric Le Goater
2020-06-23 15:08 ` Cédric Le Goater
@ 2020-07-23 6:20 ` Paul Mackerras
2020-07-27 9:38 ` Cédric Le Goater
1 sibling, 1 reply; 4+ messages in thread
From: Paul Mackerras @ 2020-07-23 6:20 UTC (permalink / raw)
To: Cédric Le Goater
Cc: Michael Ellerman, kvm, Nicholas Piggin, kvm-ppc, linuxppc-dev
On Mon, Jun 08, 2020 at 01:57:14PM +0200, Cédric Le Goater wrote:
> POWER8 and POWER9 have 12-bit LPIDs. Change LPID_RSVD to support up to
> (4096 - 2) guests on these processors. POWER7 is kept the same with a
> limitation of (1024 - 2), but it might be time to drop KVM support for
> POWER7.
>
> Tested with 2048 guests * 4 vCPUs on a witherspoon system with 512G
> RAM and a bit of swap.
>
> Signed-off-by: Cédric Le Goater <clg@kaod.org>
Thanks, patch applied to my kvm-ppc-next branch.
Paul.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] KVM: PPC: Book3S HV: increase KVMPPC_NR_LPIDS on POWER8 and POWER9
2020-07-23 6:20 ` Paul Mackerras
@ 2020-07-27 9:38 ` Cédric Le Goater
0 siblings, 0 replies; 4+ messages in thread
From: Cédric Le Goater @ 2020-07-27 9:38 UTC (permalink / raw)
To: Paul Mackerras
Cc: Michael Ellerman, kvm, Nicholas Piggin, kvm-ppc, linuxppc-dev
On 7/23/20 8:20 AM, Paul Mackerras wrote:
> On Mon, Jun 08, 2020 at 01:57:14PM +0200, Cédric Le Goater wrote:
>> POWER8 and POWER9 have 12-bit LPIDs. Change LPID_RSVD to support up to
>> (4096 - 2) guests on these processors. POWER7 is kept the same with a
>> limitation of (1024 - 2), but it might be time to drop KVM support for
>> POWER7.
>>
>> Tested with 2048 guests * 4 vCPUs on a witherspoon system with 512G
>> RAM and a bit of swap.
>>
>> Signed-off-by: Cédric Le Goater <clg@kaod.org>
>
> Thanks, patch applied to my kvm-ppc-next branch.
We have pushed the limits further on a 1TB system and reached the limit
of 4094 guests with 16 vCPUs.
With more vCPUs, the system starts to check-stop. We believe that the
pages used by the interrupt controller for the backing store of the
XIVE internal tables (END and NVT) allocated with GFP_KERNEL are
reclaimable.
I am thinking of changing the allocation flags with :
__GFP_NORETRY | __GFP_NOWARN | __GFP_NOMEMALLOC
because XIVE should be able to fail gracefully if the system is
low on mem. Is that correct ?
Thanks,
C.
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-07-27 13:16 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-08 11:57 [PATCH] KVM: PPC: Book3S HV: increase KVMPPC_NR_LPIDS on POWER8 and POWER9 Cédric Le Goater
2020-06-23 15:08 ` Cédric Le Goater
2020-07-23 6:20 ` Paul Mackerras
2020-07-27 9:38 ` Cédric Le Goater
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).