* [PATCH] powerpc/pci: Fix get_phb_number() locking
@ 2022-08-15 6:55 Michael Ellerman
2022-08-15 20:01 ` Guenter Roeck
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: Michael Ellerman @ 2022-08-15 6:55 UTC (permalink / raw)
To: linuxppc-dev; +Cc: pali, linux
The recent change to get_phb_number() causes a DEBUG_ATOMIC_SLEEP
warning on some systems:
BUG: sleeping function called from invalid context at kernel/locking/mutex.c:580
in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1, name: swapper
preempt_count: 1, expected: 0
RCU nest depth: 0, expected: 0
1 lock held by swapper/1:
#0: c157efb0 (hose_spinlock){+.+.}-{2:2}, at: pcibios_alloc_controller+0x64/0x220
Preemption disabled at:
[<00000000>] 0x0
CPU: 0 PID: 1 Comm: swapper Not tainted 5.19.0-yocto-standard+ #1
Call Trace:
[d101dc90] [c073b264] dump_stack_lvl+0x50/0x8c (unreliable)
[d101dcb0] [c0093b70] __might_resched+0x258/0x2a8
[d101dcd0] [c0d3e634] __mutex_lock+0x6c/0x6ec
[d101dd50] [c0a84174] of_alias_get_id+0x50/0xf4
[d101dd80] [c002ec78] pcibios_alloc_controller+0x1b8/0x220
[d101ddd0] [c140c9dc] pmac_pci_init+0x198/0x784
[d101de50] [c140852c] discover_phbs+0x30/0x4c
[d101de60] [c0007fd4] do_one_initcall+0x94/0x344
[d101ded0] [c1403b40] kernel_init_freeable+0x1a8/0x22c
[d101df10] [c00086e0] kernel_init+0x34/0x160
[d101df30] [c001b334] ret_from_kernel_thread+0x5c/0x64
This is because pcibios_alloc_controller() holds hose_spinlock but
of_alias_get_id() takes of_mutex which can sleep.
The hose_spinlock protects the phb_bitmap, and also the hose_list, but
it doesn't need to be held while get_phb_number() calls the OF routines,
because those are only looking up information in the device tree.
So fix it by having get_phb_number() take the hose_spinlock itself, only
where required, and then dropping the lock before returning.
pcibios_alloc_controller() then needs to take the lock again before the
list_add() but that's safe, the order of the list is not important.
Fixes: 0fe1e96fef0a ("powerpc/pci: Prefer PCI domain assignment via DT 'linux,pci-domain' and alias")
Reported-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
---
arch/powerpc/kernel/pci-common.c | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index bdd3332200c5..31de91c8359c 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -68,10 +68,6 @@ void __init set_pci_dma_ops(const struct dma_map_ops *dma_ops)
pci_dma_ops = dma_ops;
}
-/*
- * This function should run under locking protection, specifically
- * hose_spinlock.
- */
static int get_phb_number(struct device_node *dn)
{
int ret, phb_id = -1;
@@ -108,15 +104,20 @@ static int get_phb_number(struct device_node *dn)
if (!ret)
phb_id = (int)(prop & (MAX_PHBS - 1));
+ spin_lock(&hose_spinlock);
+
/* We need to be sure to not use the same PHB number twice. */
if ((phb_id >= 0) && !test_and_set_bit(phb_id, phb_bitmap))
- return phb_id;
+ goto out_unlock;
/* If everything fails then fallback to dynamic PHB numbering. */
phb_id = find_first_zero_bit(phb_bitmap, MAX_PHBS);
BUG_ON(phb_id >= MAX_PHBS);
set_bit(phb_id, phb_bitmap);
+out_unlock:
+ spin_unlock(&hose_spinlock);
+
return phb_id;
}
@@ -127,10 +128,13 @@ struct pci_controller *pcibios_alloc_controller(struct device_node *dev)
phb = zalloc_maybe_bootmem(sizeof(struct pci_controller), GFP_KERNEL);
if (phb == NULL)
return NULL;
- spin_lock(&hose_spinlock);
+
phb->global_number = get_phb_number(dev);
+
+ spin_lock(&hose_spinlock);
list_add_tail(&phb->list_node, &hose_list);
spin_unlock(&hose_spinlock);
+
phb->dn = dev;
phb->is_dynamic = slab_is_available();
#ifdef CONFIG_PPC64
--
2.37.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] powerpc/pci: Fix get_phb_number() locking
2022-08-15 6:55 [PATCH] powerpc/pci: Fix get_phb_number() locking Michael Ellerman
@ 2022-08-15 20:01 ` Guenter Roeck
2022-08-17 16:37 ` Pali Rohár
2022-08-19 11:29 ` Michael Ellerman
2 siblings, 0 replies; 4+ messages in thread
From: Guenter Roeck @ 2022-08-15 20:01 UTC (permalink / raw)
To: Michael Ellerman; +Cc: pali, linuxppc-dev
On Mon, Aug 15, 2022 at 04:55:50PM +1000, Michael Ellerman wrote:
> The recent change to get_phb_number() causes a DEBUG_ATOMIC_SLEEP
> warning on some systems:
>
> BUG: sleeping function called from invalid context at kernel/locking/mutex.c:580
> in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1, name: swapper
> preempt_count: 1, expected: 0
> RCU nest depth: 0, expected: 0
> 1 lock held by swapper/1:
> #0: c157efb0 (hose_spinlock){+.+.}-{2:2}, at: pcibios_alloc_controller+0x64/0x220
> Preemption disabled at:
> [<00000000>] 0x0
> CPU: 0 PID: 1 Comm: swapper Not tainted 5.19.0-yocto-standard+ #1
> Call Trace:
> [d101dc90] [c073b264] dump_stack_lvl+0x50/0x8c (unreliable)
> [d101dcb0] [c0093b70] __might_resched+0x258/0x2a8
> [d101dcd0] [c0d3e634] __mutex_lock+0x6c/0x6ec
> [d101dd50] [c0a84174] of_alias_get_id+0x50/0xf4
> [d101dd80] [c002ec78] pcibios_alloc_controller+0x1b8/0x220
> [d101ddd0] [c140c9dc] pmac_pci_init+0x198/0x784
> [d101de50] [c140852c] discover_phbs+0x30/0x4c
> [d101de60] [c0007fd4] do_one_initcall+0x94/0x344
> [d101ded0] [c1403b40] kernel_init_freeable+0x1a8/0x22c
> [d101df10] [c00086e0] kernel_init+0x34/0x160
> [d101df30] [c001b334] ret_from_kernel_thread+0x5c/0x64
>
> This is because pcibios_alloc_controller() holds hose_spinlock but
> of_alias_get_id() takes of_mutex which can sleep.
>
> The hose_spinlock protects the phb_bitmap, and also the hose_list, but
> it doesn't need to be held while get_phb_number() calls the OF routines,
> because those are only looking up information in the device tree.
>
> So fix it by having get_phb_number() take the hose_spinlock itself, only
> where required, and then dropping the lock before returning.
> pcibios_alloc_controller() then needs to take the lock again before the
> list_add() but that's safe, the order of the list is not important.
>
> Fixes: 0fe1e96fef0a ("powerpc/pci: Prefer PCI domain assignment via DT 'linux,pci-domain' and alias")
> Reported-by: Guenter Roeck <linux@roeck-us.net>
> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
The problem is no longer seen with this patch applied.
Tested-by: Guenter Roeck <linux@roeck-us.net>
Guenter
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] powerpc/pci: Fix get_phb_number() locking
2022-08-15 6:55 [PATCH] powerpc/pci: Fix get_phb_number() locking Michael Ellerman
2022-08-15 20:01 ` Guenter Roeck
@ 2022-08-17 16:37 ` Pali Rohár
2022-08-19 11:29 ` Michael Ellerman
2 siblings, 0 replies; 4+ messages in thread
From: Pali Rohár @ 2022-08-17 16:37 UTC (permalink / raw)
To: Michael Ellerman; +Cc: linuxppc-dev, linux
On Monday 15 August 2022 16:55:50 Michael Ellerman wrote:
> The recent change to get_phb_number() causes a DEBUG_ATOMIC_SLEEP
> warning on some systems:
>
> BUG: sleeping function called from invalid context at kernel/locking/mutex.c:580
> in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1, name: swapper
> preempt_count: 1, expected: 0
> RCU nest depth: 0, expected: 0
> 1 lock held by swapper/1:
> #0: c157efb0 (hose_spinlock){+.+.}-{2:2}, at: pcibios_alloc_controller+0x64/0x220
> Preemption disabled at:
> [<00000000>] 0x0
> CPU: 0 PID: 1 Comm: swapper Not tainted 5.19.0-yocto-standard+ #1
> Call Trace:
> [d101dc90] [c073b264] dump_stack_lvl+0x50/0x8c (unreliable)
> [d101dcb0] [c0093b70] __might_resched+0x258/0x2a8
> [d101dcd0] [c0d3e634] __mutex_lock+0x6c/0x6ec
> [d101dd50] [c0a84174] of_alias_get_id+0x50/0xf4
> [d101dd80] [c002ec78] pcibios_alloc_controller+0x1b8/0x220
> [d101ddd0] [c140c9dc] pmac_pci_init+0x198/0x784
> [d101de50] [c140852c] discover_phbs+0x30/0x4c
> [d101de60] [c0007fd4] do_one_initcall+0x94/0x344
> [d101ded0] [c1403b40] kernel_init_freeable+0x1a8/0x22c
> [d101df10] [c00086e0] kernel_init+0x34/0x160
> [d101df30] [c001b334] ret_from_kernel_thread+0x5c/0x64
>
> This is because pcibios_alloc_controller() holds hose_spinlock but
> of_alias_get_id() takes of_mutex which can sleep.
>
> The hose_spinlock protects the phb_bitmap, and also the hose_list, but
> it doesn't need to be held while get_phb_number() calls the OF routines,
> because those are only looking up information in the device tree.
>
> So fix it by having get_phb_number() take the hose_spinlock itself, only
> where required, and then dropping the lock before returning.
> pcibios_alloc_controller() then needs to take the lock again before the
> list_add() but that's safe, the order of the list is not important.
>
> Fixes: 0fe1e96fef0a ("powerpc/pci: Prefer PCI domain assignment via DT 'linux,pci-domain' and alias")
> Reported-by: Guenter Roeck <linux@roeck-us.net>
> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Thanks for fixing it!
Acked-by: Pali Rohár <pali@kernel.org>
> ---
> arch/powerpc/kernel/pci-common.c | 16 ++++++++++------
> 1 file changed, 10 insertions(+), 6 deletions(-)
>
> diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
> index bdd3332200c5..31de91c8359c 100644
> --- a/arch/powerpc/kernel/pci-common.c
> +++ b/arch/powerpc/kernel/pci-common.c
> @@ -68,10 +68,6 @@ void __init set_pci_dma_ops(const struct dma_map_ops *dma_ops)
> pci_dma_ops = dma_ops;
> }
>
> -/*
> - * This function should run under locking protection, specifically
> - * hose_spinlock.
> - */
> static int get_phb_number(struct device_node *dn)
> {
> int ret, phb_id = -1;
> @@ -108,15 +104,20 @@ static int get_phb_number(struct device_node *dn)
> if (!ret)
> phb_id = (int)(prop & (MAX_PHBS - 1));
>
> + spin_lock(&hose_spinlock);
> +
> /* We need to be sure to not use the same PHB number twice. */
> if ((phb_id >= 0) && !test_and_set_bit(phb_id, phb_bitmap))
> - return phb_id;
> + goto out_unlock;
>
> /* If everything fails then fallback to dynamic PHB numbering. */
> phb_id = find_first_zero_bit(phb_bitmap, MAX_PHBS);
> BUG_ON(phb_id >= MAX_PHBS);
> set_bit(phb_id, phb_bitmap);
>
> +out_unlock:
> + spin_unlock(&hose_spinlock);
> +
> return phb_id;
> }
>
> @@ -127,10 +128,13 @@ struct pci_controller *pcibios_alloc_controller(struct device_node *dev)
> phb = zalloc_maybe_bootmem(sizeof(struct pci_controller), GFP_KERNEL);
> if (phb == NULL)
> return NULL;
> - spin_lock(&hose_spinlock);
> +
> phb->global_number = get_phb_number(dev);
> +
> + spin_lock(&hose_spinlock);
> list_add_tail(&phb->list_node, &hose_list);
> spin_unlock(&hose_spinlock);
> +
> phb->dn = dev;
> phb->is_dynamic = slab_is_available();
> #ifdef CONFIG_PPC64
> --
> 2.37.1
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] powerpc/pci: Fix get_phb_number() locking
2022-08-15 6:55 [PATCH] powerpc/pci: Fix get_phb_number() locking Michael Ellerman
2022-08-15 20:01 ` Guenter Roeck
2022-08-17 16:37 ` Pali Rohár
@ 2022-08-19 11:29 ` Michael Ellerman
2 siblings, 0 replies; 4+ messages in thread
From: Michael Ellerman @ 2022-08-19 11:29 UTC (permalink / raw)
To: Michael Ellerman, linuxppc-dev; +Cc: pali, linux
On Mon, 15 Aug 2022 16:55:50 +1000, Michael Ellerman wrote:
> The recent change to get_phb_number() causes a DEBUG_ATOMIC_SLEEP
> warning on some systems:
>
> BUG: sleeping function called from invalid context at kernel/locking/mutex.c:580
> in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1, name: swapper
> preempt_count: 1, expected: 0
> RCU nest depth: 0, expected: 0
> 1 lock held by swapper/1:
> #0: c157efb0 (hose_spinlock){+.+.}-{2:2}, at: pcibios_alloc_controller+0x64/0x220
> Preemption disabled at:
> [<00000000>] 0x0
> CPU: 0 PID: 1 Comm: swapper Not tainted 5.19.0-yocto-standard+ #1
> Call Trace:
> [d101dc90] [c073b264] dump_stack_lvl+0x50/0x8c (unreliable)
> [d101dcb0] [c0093b70] __might_resched+0x258/0x2a8
> [d101dcd0] [c0d3e634] __mutex_lock+0x6c/0x6ec
> [d101dd50] [c0a84174] of_alias_get_id+0x50/0xf4
> [d101dd80] [c002ec78] pcibios_alloc_controller+0x1b8/0x220
> [d101ddd0] [c140c9dc] pmac_pci_init+0x198/0x784
> [d101de50] [c140852c] discover_phbs+0x30/0x4c
> [d101de60] [c0007fd4] do_one_initcall+0x94/0x344
> [d101ded0] [c1403b40] kernel_init_freeable+0x1a8/0x22c
> [d101df10] [c00086e0] kernel_init+0x34/0x160
> [d101df30] [c001b334] ret_from_kernel_thread+0x5c/0x64
>
> [...]
Applied to powerpc/fixes.
[1/1] powerpc/pci: Fix get_phb_number() locking
https://git.kernel.org/powerpc/c/8d48562a2729742f767b0fdd994d6b2a56a49c63
cheers
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2022-08-19 11:29 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-15 6:55 [PATCH] powerpc/pci: Fix get_phb_number() locking Michael Ellerman
2022-08-15 20:01 ` Guenter Roeck
2022-08-17 16:37 ` Pali Rohár
2022-08-19 11:29 ` Michael Ellerman
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.