[PATCH v3 0/1] Make RAS IRQ explicitly dependent on DLPAR workqueue

[PATCH v3 0/1] Make RAS IRQ explicitly dependent on DLPAR workqueue

[Jose Ricardo Ziviani]

This patch replaces "[PATCH v2 0/1] Increase dlpar initcall priority"

1. Testing without this patch, using QEMU:

[    0.022800] UDP-Lite hash table entries: 2048 (order: 0, 65536 bytes)
[    0.022865] NET: Registered protocol family 1
[    0.023435] Unpacking initramfs...
(qemu) object_add memory-backend-ram,id=mem1,size=10G                                   
(qemu) device_add pc-dimm,id=dimm1,memdev=mem1                                          
[    0.095354] Unable to handle kernel paging request for data at address 0xf94d03007c421378
[    0.095409] Faulting instruction address: 0xc00000000012d744
[    0.095450] Oops: Kernel access of bad area, sig: 11 [#1]
[    0.095482] LE SMP NR_CPUS=2048 NUMA pSeries
[    0.095516] Modules linked in:
[    0.095542] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.15.0-rc2-ziviani+ #26
[    0.095592] task:         (ptrval) task.stack:         (ptrval)
[    0.095634] NIP:  c00000000012d744 LR: c00000000012d744 CTR: 0000000000000000
[    0.095683] REGS:         (ptrval) TRAP: 0380   Not tainted  (4.15.0-rc2-ziviani+)
[    0.095729] MSR:  8000000000009033 <SF,EE,ME,IR,DR,RI,LE>  CR: 28088042  XER: 20040000
[    0.095780] CFAR: c00000000012d3c4 SOFTE: 0 
[    0.095780] GPR00: c00000000012d744 c0000000fffefb90 c000000001290c00 0000000000000000 
[    0.095780] GPR04: 000000007fffffff c0000000fe20bb40 00000000000003d7 c0000000012cdd78 
[    0.095780] GPR08: 0000000000000001 00000000ffffffff 00000000fed10000 0000000000001001 
[    0.095780] GPR12: c0000000000c6800 c00000000fac0000 c00000000000d1a8 0000000000008000 
[    0.095780] GPR16: 0000000000000000 c0000000fea0a007 c000000000c44d78 0000000000000001 
[    0.095780] GPR20: 0000000000000000 c000000000e2c8a8 c000000001452af8 0000000000000000 
[    0.095780] GPR24: c000000001452af0 c0000000fffefe70 0000000000000000 c000000000e2c8a8 
[    0.095780] GPR28: 0000000000000800 0000000000000000 c0000000fe20bb40 f94d03007c421378 
[    0.096174] NIP [c00000000012d744] __queue_work+0xd4/0x5c0
[    0.096207] LR [c00000000012d744] __queue_work+0xd4/0x5c0
[    0.096241] Call Trace:
[    0.096258] [c0000000fffefb90] [c00000000012d744] __queue_work+0xd4/0x5c0 (unreliable)
[    0.096308] [c0000000fffefc70] [c00000000012dce4] queue_work_on+0xb4/0xf0
snip
[    0.097459] ---[ end trace b527625ce4033fad ]---
[    0.099233] 
[    1.099254] Kernel panic - not syncing: Fatal exception in interrupt
[    1.103312] Rebooting in 10 seconds..

2. Testing with this patch, with QEMU again:

[    0.023086] NET: Registered protocol family 1
[    0.023664] Unpacking initramfs...
(qemu) object_add memory-backend-ram,id=mem1,size=10G                                   
(qemu) device_add pc-dimm,id=dimm1,memdev=mem1                                          
[    0.163178] pseries-hotplug-mem: Attempting to hot-add 40 LMB(s) at index 80000010
[    0.163416] radix-mmu: Mapped 0xc000000100000000-0xc000000110000000 with 2.00 MiB pages
[    0.163830] radix-mmu: Mapped 0xc000000110000000-0xc000000120000000 with 2.00 MiB pages
snip
[    0.189706] pseries-hotplug-mem: Memory at 100000000 (drc index 80000010) was hot-added
[    0.189753] pseries-hotplug-mem: Memory at 110000000 (drc index 80000011) was hot-added
snip
...normal initialization

(qemu)  info memory-devices 
Memory device [dimm]: "dimm1"
  addr: 0x100000000
  slot: 0
  node: 0
  size: 10737418240
  memdev: /objects/mem1
  hotplugged: true
  hotpluggable: true

Jose Ricardo Ziviani (1):
  powerpc/pseries: Make RAS IRQ explicitly dependent on DLPAR WQ

 arch/powerpc/platforms/pseries/dlpar.c   | 21 ++++++++++++++++++---
 arch/powerpc/platforms/pseries/pseries.h |  2 ++
 arch/powerpc/platforms/pseries/ras.c     |  3 ++-
 3 files changed, 22 insertions(+), 4 deletions(-)

-- 
2.14.1

[PATCH v3 1/1] powerpc/pseries: Make RAS IRQ explicitly dependent on DLPAR WQ

[Jose Ricardo Ziviani]

The hotplug engine uses its own workqueue to handle IRQ requests, the
problem is that such workqueue is initialized after init_ras_IRQ, which
will cause a kernel panic if any hotplug interruption is issued in that
period of time.

This patch makes RAS IRQ registration explicitly dependent on DLPAR
workqueue and adds a check to make sure that the workqueue is not
initialized more than once.

Reported-by: Min Deng <mdeng@redhat.com>
Reported-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com>
Tested-by: Jose Ricardo Ziviani <joserz@linux.vnet.ibm.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
 arch/powerpc/platforms/pseries/dlpar.c   | 21 ++++++++++++++++++---
 arch/powerpc/platforms/pseries/pseries.h |  2 ++
 arch/powerpc/platforms/pseries/ras.c     |  3 ++-
 3 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/arch/powerpc/platforms/pseries/dlpar.c b/arch/powerpc/platforms/pseries/dlpar.c
index 6e35780c5962..a0b20c03f078 100644
--- a/arch/powerpc/platforms/pseries/dlpar.c
+++ b/arch/powerpc/platforms/pseries/dlpar.c
@@ -574,11 +574,26 @@ static ssize_t dlpar_show(struct class *class, struct class_attribute *attr,
 
 static CLASS_ATTR_RW(dlpar);
 
-static int __init pseries_dlpar_init(void)
+int __init dlpar_workqueue_init(void)
 {
+	if (pseries_hp_wq)
+		return 0;
+
 	pseries_hp_wq = alloc_workqueue("pseries hotplug workqueue",
-					WQ_UNBOUND, 1);
+			WQ_UNBOUND, 1);
+
+	return pseries_hp_wq ? 0 : -ENOMEM;
+}
+
+static int __init dlpar_sysfs_init(void)
+{
+	int rc;
+
+	rc = dlpar_workqueue_init();
+	if (rc)
+		return rc;
+
 	return sysfs_create_file(kernel_kobj, &class_attr_dlpar.attr);
 }
-machine_device_initcall(pseries, pseries_dlpar_init);
+machine_device_initcall(pseries, dlpar_sysfs_init);
 
diff --git a/arch/powerpc/platforms/pseries/pseries.h b/arch/powerpc/platforms/pseries/pseries.h
index 4470a3194311..1ae1d9f4dbe9 100644
--- a/arch/powerpc/platforms/pseries/pseries.h
+++ b/arch/powerpc/platforms/pseries/pseries.h
@@ -98,4 +98,6 @@ static inline unsigned long cmo_get_page_size(void)
 	return CMO_PageSize;
 }
 
+int dlpar_workqueue_init(void);
+
 #endif /* _PSERIES_PSERIES_H */
diff --git a/arch/powerpc/platforms/pseries/ras.c b/arch/powerpc/platforms/pseries/ras.c
index 4923ffe230cf..81d8614e7379 100644
--- a/arch/powerpc/platforms/pseries/ras.c
+++ b/arch/powerpc/platforms/pseries/ras.c
@@ -69,7 +69,8 @@ static int __init init_ras_IRQ(void)
 	/* Hotplug Events */
 	np = of_find_node_by_path("/event-sources/hot-plug-events");
 	if (np != NULL) {
-		request_event_sources_irqs(np, ras_hotplug_interrupt,
+		if (dlpar_workqueue_init() == 0)
+			request_event_sources_irqs(np, ras_hotplug_interrupt,
 					   "RAS_HOTPLUG");
 		of_node_put(np);
 	}
-- 
2.14.1

Re: [PATCH v3 1/1] powerpc/pseries: Make RAS IRQ explicitly dependent on DLPAR WQ

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1099 bytes --]

On Wed, Jan 03, 2018 at 07:12:52PM -0200, Jose Ricardo Ziviani wrote:
> The hotplug engine uses its own workqueue to handle IRQ requests, the
> problem is that such workqueue is initialized after init_ras_IRQ, which
> will cause a kernel panic if any hotplug interruption is issued in that
> period of time.
> 
> This patch makes RAS IRQ registration explicitly dependent on DLPAR
> workqueue and adds a check to make sure that the workqueue is not
> initialized more than once.
> 
> Reported-by: Min Deng <mdeng@redhat.com>
> Reported-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com>
> Tested-by: Jose Ricardo Ziviani <joserz@linux.vnet.ibm.com>
> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>

Reviewed-by: David Gibson <david@gibson.dropbear.id.au>

Michael, If this can be merged to a non-rebasing tree sooner rather
than later, that would be quite helpful to me.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [v3, 1/1] powerpc/pseries: Make RAS IRQ explicitly dependent on DLPAR WQ

[Michael Ellerman]

On Wed, 2018-01-03 at 21:12:52 UTC, Jose Ricardo Ziviani wrote:
> The hotplug engine uses its own workqueue to handle IRQ requests, the
> problem is that such workqueue is initialized after init_ras_IRQ, which
> will cause a kernel panic if any hotplug interruption is issued in that
> period of time.
> 
> This patch makes RAS IRQ registration explicitly dependent on DLPAR
> workqueue and adds a check to make sure that the workqueue is not
> initialized more than once.
> 
> Reported-by: Min Deng <mdeng@redhat.com>
> Reported-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com>
> Tested-by: Jose Ricardo Ziviani <joserz@linux.vnet.ibm.com>
> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>

Applied to powerpc fixes, thanks.

https://git.kernel.org/powerpc/c/e2d5915293ffdff977ddcfc12b817b

cheers