Re: [PATCH] powerpc/powernv/npu: Fix oops in pnv_try_setup_npu_table_group()

From: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
To: Frederic Barrat <fbarrat@linux.ibm.com>,
	linuxppc-dev@lists.ozlabs.org, aik@ozlabs.ru
Subject: Re: [PATCH] powerpc/powernv/npu: Fix oops in pnv_try_setup_npu_table_group()
Date: Thu, 10 Jan 2019 11:40:26 +1100	[thread overview]
Message-ID: <c5f6b3e0-f926-ec81-e7e2-292c8c03941d@au1.ibm.com> (raw)
In-Reply-To: <20190109151342.19953-1-fbarrat@linux.ibm.com>

On 10/1/19 2:13 am, Frederic Barrat wrote:
> With a recent change around IOMMU group, a system with an opencapi
> adapter is no longer booting and we get a kernel oops:
> 
> BUG: Kernel NULL pointer dereference at 0x00000028
> Faulting instruction address: 0xc0000000000aa38c
> Oops: Kernel access of bad area, sig: 7 [#1]
> LE SMP NR_CPUS=2048 NUMA PowerNV
> Modules linked in:
> CPU: 5 PID: 1 Comm: swapper/4 Not tainted 5.0.0-rc1-fxb-00001-g3bd6e94bec12
> NIP:  c0000000000aa38c LR: c0000000000a6608 CTR: c000000000097480
> REGS: c000000005783700 TRAP: 0300   Not tainted  (5.0.0-rc1-fxb-00001-g3bd6
> MSR:  9000000002009033 <SF,HV,VEC,EE,ME,IR,DR,RI,LE>  CR: 28000228  XER: 20
> CFAR: c0000000000a6604 DAR: 0000000000000028 DSISR: 00080000 IRQMASK: 0
> GPR00: c0000000000a6608 c000000005783990 c000000001036100 c0000007bf761860
> GPR04: 0000000000000000 c000000005783834 0000000000000000 0000000000000000
> GPR08: 69626d2c6e707500 0000000000000000 0000000000000000 9000000002001003
> GPR12: 0000000000000000 c0000007bfff8300 c000000000010450 0000000000000000
> GPR16: c000000000ced938 0000000000000100 c000000000ced948 00000000000a0000
> GPR20: 00000000000bfffe c000000000ced9a8 0000000000000200 c000000000ced978
> GPR24: 00000000006080c0 c000000716d09828 c00000002e6fd000 0000000000000000
> GPR28: c0000007bf4aff68 c0000007bf8d0080 c000000000f23938 c0000007bf761860
> NIP [c0000000000aa38c] pnv_try_setup_npu_table_group+0x1c/0x1a0
> LR [c0000000000a6608] pnv_pci_ioda_fixup+0x1f8/0x660
> Call Trace:
> [c000000005783990] [c0000000000aa3d0] pnv_try_setup_npu_table_group+0x60/0x
> [c0000000057839d0] [c0000000000a661c] pnv_pci_ioda_fixup+0x20c/0x660
> [c000000005783ab0] [c000000000e1d4c0] pcibios_resource_survey+0x2c8/0x31c
> [c000000005783b90] [c000000000e1caf4] pcibios_init+0xb0/0xe4
> [c000000005783c10] [c000000000010054] do_one_initcall+0x64/0x264
> [c000000005783ce0] [c000000000e1132c] kernel_init_freeable+0x36c/0x468
> [c000000005783db0] [c000000000010474] kernel_init+0x2c/0x148
> [c000000005783e20] [c00000000000b794] ret_from_kernel_thread+0x5c/0x68
> 
> An opencapi device is using a device PE, so the current code breaks
> because pe->pbus is not defined.
> 
> More generally, there's no need to define an IOMMU group for opencapi,
> as the device sends real addresses directly (admittedly, the
> virtualization story is yet to be written). So let's fix it by
> skipping the IOMMU group setup for opencapi PHBs.
> 
> Fixes: 0bd971676e68 ("powerpc/powernv/npu: Add compound IOMMU groups")
> Signed-off-by: Frederic Barrat <fbarrat@linux.ibm.com>

Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>

> ---
>   arch/powerpc/platforms/powernv/pci-ioda.c | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
> index 1d6406a051f1..7db3119f8a5b 100644
> --- a/arch/powerpc/platforms/powernv/pci-ioda.c
> +++ b/arch/powerpc/platforms/powernv/pci-ioda.c
> @@ -2681,7 +2681,8 @@ static void pnv_pci_ioda_setup_iommu_api(void)
>   	list_for_each_entry(hose, &hose_list, list_node) {
>   		phb = hose->private_data;
>   
> -		if (phb->type == PNV_PHB_NPU_NVLINK)
> +		if (phb->type == PNV_PHB_NPU_NVLINK ||
> +		    phb->type == PNV_PHB_NPU_OCAPI)
>   			continue;
>   
>   		list_for_each_entry(pe, &phb->ioda.pe_list, list) {
> 

-- 
Andrew Donnellan              OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com  IBM Australia Limited