[PATCH v2 0/2] powerpc/xive: fix CPU hot unplug

[PATCH v2 0/2] powerpc/xive: fix CPU hot unplug

[Cédric Le Goater]

Hi,

Recent commit eac1e731b59e ("powerpc/xive: guest exploitation of the
XIVE interrupt controller") introduced support for the XIVE
exploitation mode of the P9 interrupt controller on the pseries
platform.

At the time the patchset was sent, support for CPU removal was not
complete on PowerVM and only CPU hotplug was tested. To also support
CPU hot unplug, some cleanups of the XIVE internal structures are
needed. These two patches include the missing bits without which the
kernel crashes in a rtas call when a CPU is released.

There are still some corner cases to address when stressing the lpar
with plug/unplug loops. These are not necessarily Linux issues. Under
QEMU, it showed to be the decrementer waking the dying CPU. Under
phyp, it is much more rare and investigation is under progress.

Tested under a phyp and a XIVE QEMU model for pseries.

These changes should only impact the pseries platform. 

Thanks,

C.

Changes since v2 :

 - clarified commit log. 

Cédric Le Goater (2):
  powerpc/xive: fix IPI reset
  powerpc/xive: fix cpu removal

 arch/powerpc/sysdev/xive/common.c | 8 ++++++++
 arch/powerpc/sysdev/xive/spapr.c  | 4 ++++
 2 files changed, 12 insertions(+)

-- 
2.13.5

[PATCH v2 1/2] powerpc/xive: fix IPI reset

[Cédric Le Goater]

When resetting an IPI, hw_ipi should also be set to zero.

Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 arch/powerpc/sysdev/xive/spapr.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/powerpc/sysdev/xive/spapr.c b/arch/powerpc/sysdev/xive/spapr.c
index f24a70bc6855..d9c4c9366049 100644
--- a/arch/powerpc/sysdev/xive/spapr.c
+++ b/arch/powerpc/sysdev/xive/spapr.c
@@ -431,7 +431,11 @@ static int xive_spapr_get_ipi(unsigned int cpu, struct xive_cpu *xc)
 
 static void xive_spapr_put_ipi(unsigned int cpu, struct xive_cpu *xc)
 {
+	if (!xc->hw_ipi)
+		return;
+
 	xive_irq_bitmap_free(xc->hw_ipi);
+	xc->hw_ipi = 0;
 }
 #endif /* CONFIG_SMP */
 
-- 
2.13.5

Re: [v2,1/2] powerpc/xive: fix IPI reset

[Michael Ellerman]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 305 bytes --]

On Wed, 2017-10-04 at 09:15:04 UTC, =?utf-8?q?C=C3=A9dric_Le_Goater?= wrote:
> When resetting an IPI, hw_ipi should also be set to zero.
> 
> Signed-off-by: Cédric Le Goater <clg@kaod.org>

Series applied to powerpc fixes, thanks.

https://git.kernel.org/powerpc/c/74f1282114acc7d67e25745efe200f

cheers

[PATCH v2 2/2] powerpc/xive: clear XIVE internal structures when a CPU is removed

[Cédric Le Goater]

Commit eac1e731b59e ("powerpc/xive: guest exploitation of the XIVE
interrupt controller") introduced support for the XIVE exploitation
mode of the P9 interrupt controller on the pseries platform. 

At that time, support for CPU removal was not complete on PowerVM and
CPU hot unplug remained untested. It appears that some cleanups of the
XIVE internal structures are required before releasing the CPU,
without which the kernel crashes in a rtas call doing the CPU
isolation.

These changes fix the crash by deconfiguring the IPI interrupt source
and clearing the event queues of the CPU when it is removed.

Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
 arch/powerpc/sysdev/xive/common.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/powerpc/sysdev/xive/common.c b/arch/powerpc/sysdev/xive/common.c
index 1c087ed7427f..b9440ac7a3c1 100644
--- a/arch/powerpc/sysdev/xive/common.c
+++ b/arch/powerpc/sysdev/xive/common.c
@@ -1398,6 +1398,14 @@ void xive_teardown_cpu(void)
 
 	if (xive_ops->teardown_cpu)
 		xive_ops->teardown_cpu(cpu, xc);
+
+#ifdef CONFIG_SMP
+	/* Get rid of IPI */
+	xive_cleanup_cpu_ipi(cpu, xc);
+#endif
+
+	/* Disable and free the queues */
+	xive_cleanup_cpu_queues(cpu, xc);
 }
 
 void xive_kexec_teardown_cpu(int secondary)
-- 
2.13.5