[PATCH kernel] powerpc/powernv/sriov: Register IOMMU groups for VFs

[PATCH kernel] powerpc/powernv/sriov: Register IOMMU groups for VFs

[Alexey Kardashevskiy]

The compound IOMMU group rework moved iommu_register_group() together in
pnv_pci_ioda_setup_iommu_api() (which is a part of ppc_md.pcibios_fixup).
As the result, pnv_ioda_setup_bus_iommu_group() does not create groups
any more, it only adds devices to groups.

This works fine for boot time devices. However IOMMU groups for SRIOV's
VFs were added by pnv_ioda_setup_bus_iommu_group() so this got broken:
pnv_tce_iommu_bus_notifier() expects a group to be registered for VF
and it is not.

This adds missing group registration and adds a NULL pointer check into
the bus notifier so we won't stop in xmon if there is no group, although
it is not expected to happen now.

Fixes: 0bd971676e68 "powerpc/powernv/npu: Add compound IOMMU groups"
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---

Fixes https://bugzilla.linux.ibm.com/show_bug.cgi?id=175550
"Kernel Oops while creating SRIOV VF's."


---
 arch/powerpc/platforms/powernv/pci-ioda.c | 2 ++
 arch/powerpc/platforms/powernv/pci.c      | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 7db3119..145373f 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -1593,6 +1593,8 @@ static void pnv_ioda_setup_vf_PE(struct pci_dev *pdev, u16 num_vfs)
 
 		pnv_pci_ioda2_setup_dma_pe(phb, pe);
 #ifdef CONFIG_IOMMU_API
+		iommu_register_group(&pe->table_group,
+				pe->phb->hose->global_number, pe->pe_number);
 		pnv_ioda_setup_bus_iommu_group(pe, &pe->table_group, NULL);
 #endif
 	}
diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
index 45fb70b..ef9448a 100644
--- a/arch/powerpc/platforms/powernv/pci.c
+++ b/arch/powerpc/platforms/powernv/pci.c
@@ -1147,6 +1147,8 @@ static int pnv_tce_iommu_bus_notifier(struct notifier_block *nb,
 			return 0;
 
 		pe = &phb->ioda.pe_array[pdn->pe_number];
+		if (!pe->table_group.group)
+			return 0;
 		iommu_add_device(&pe->table_group, dev);
 		return 0;
 	case BUS_NOTIFY_DEL_DEVICE:
-- 
2.17.1

Re: [PATCH kernel] powerpc/powernv/sriov: Register IOMMU groups for VFs

[Alexey Kardashevskiy]

On 18/02/2019 16:58, Alexey Kardashevskiy wrote:
> The compound IOMMU group rework moved iommu_register_group() together in
> pnv_pci_ioda_setup_iommu_api() (which is a part of ppc_md.pcibios_fixup).
> As the result, pnv_ioda_setup_bus_iommu_group() does not create groups
> any more, it only adds devices to groups.
> 
> This works fine for boot time devices. However IOMMU groups for SRIOV's
> VFs were added by pnv_ioda_setup_bus_iommu_group() so this got broken:
> pnv_tce_iommu_bus_notifier() expects a group to be registered for VF
> and it is not.
> 
> This adds missing group registration and adds a NULL pointer check into
> the bus notifier so we won't stop in xmon if there is no group, although
> it is not expected to happen now.
> 
> Fixes: 0bd971676e68 "powerpc/powernv/npu: Add compound IOMMU groups"
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>

Forgot this:

Reported-by: Santwana Samantray<santwana.samantray@in.ibm.com>


> ---
> 
> Fixes https://bugzilla.linux.ibm.com/show_bug.cgi?id=175550
> "Kernel Oops while creating SRIOV VF's."
> 
> 
> ---
>  arch/powerpc/platforms/powernv/pci-ioda.c | 2 ++
>  arch/powerpc/platforms/powernv/pci.c      | 2 ++
>  2 files changed, 4 insertions(+)
> 
> diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
> index 7db3119..145373f 100644
> --- a/arch/powerpc/platforms/powernv/pci-ioda.c
> +++ b/arch/powerpc/platforms/powernv/pci-ioda.c
> @@ -1593,6 +1593,8 @@ static void pnv_ioda_setup_vf_PE(struct pci_dev *pdev, u16 num_vfs)
>  
>  		pnv_pci_ioda2_setup_dma_pe(phb, pe);
>  #ifdef CONFIG_IOMMU_API
> +		iommu_register_group(&pe->table_group,
> +				pe->phb->hose->global_number, pe->pe_number);
>  		pnv_ioda_setup_bus_iommu_group(pe, &pe->table_group, NULL);
>  #endif
>  	}
> diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
> index 45fb70b..ef9448a 100644
> --- a/arch/powerpc/platforms/powernv/pci.c
> +++ b/arch/powerpc/platforms/powernv/pci.c
> @@ -1147,6 +1147,8 @@ static int pnv_tce_iommu_bus_notifier(struct notifier_block *nb,
>  			return 0;
>  
>  		pe = &phb->ioda.pe_array[pdn->pe_number];
> +		if (!pe->table_group.group)
> +			return 0;
>  		iommu_add_device(&pe->table_group, dev);
>  		return 0;
>  	case BUS_NOTIFY_DEL_DEVICE:
> 

-- 
Alexey

Re: [PATCH kernel] powerpc/powernv/sriov: Register IOMMU groups for VFs

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2559 bytes --]

On Mon, Feb 18, 2019 at 04:58:01PM +1100, Alexey Kardashevskiy wrote:
> The compound IOMMU group rework moved iommu_register_group() together in
> pnv_pci_ioda_setup_iommu_api() (which is a part of ppc_md.pcibios_fixup).
> As the result, pnv_ioda_setup_bus_iommu_group() does not create groups
> any more, it only adds devices to groups.
> 
> This works fine for boot time devices. However IOMMU groups for SRIOV's
> VFs were added by pnv_ioda_setup_bus_iommu_group() so this got broken:
> pnv_tce_iommu_bus_notifier() expects a group to be registered for VF
> and it is not.
> 
> This adds missing group registration and adds a NULL pointer check into
> the bus notifier so we won't stop in xmon if there is no group, although
> it is not expected to happen now.
> 
> Fixes: 0bd971676e68 "powerpc/powernv/npu: Add compound IOMMU groups"
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>

Reviewed-by: David Gibson <david@gibson.dropbear.id.au>

> ---
> 
> Fixes https://bugzilla.linux.ibm.com/show_bug.cgi?id=175550
> "Kernel Oops while creating SRIOV VF's."
> 
> 
> ---
>  arch/powerpc/platforms/powernv/pci-ioda.c | 2 ++
>  arch/powerpc/platforms/powernv/pci.c      | 2 ++
>  2 files changed, 4 insertions(+)
> 
> diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
> index 7db3119..145373f 100644
> --- a/arch/powerpc/platforms/powernv/pci-ioda.c
> +++ b/arch/powerpc/platforms/powernv/pci-ioda.c
> @@ -1593,6 +1593,8 @@ static void pnv_ioda_setup_vf_PE(struct pci_dev *pdev, u16 num_vfs)
>  
>  		pnv_pci_ioda2_setup_dma_pe(phb, pe);
>  #ifdef CONFIG_IOMMU_API
> +		iommu_register_group(&pe->table_group,
> +				pe->phb->hose->global_number, pe->pe_number);
>  		pnv_ioda_setup_bus_iommu_group(pe, &pe->table_group, NULL);
>  #endif
>  	}
> diff --git a/arch/powerpc/platforms/powernv/pci.c b/arch/powerpc/platforms/powernv/pci.c
> index 45fb70b..ef9448a 100644
> --- a/arch/powerpc/platforms/powernv/pci.c
> +++ b/arch/powerpc/platforms/powernv/pci.c
> @@ -1147,6 +1147,8 @@ static int pnv_tce_iommu_bus_notifier(struct notifier_block *nb,
>  			return 0;
>  
>  		pe = &phb->ioda.pe_array[pdn->pe_number];
> +		if (!pe->table_group.group)
> +			return 0;
>  		iommu_add_device(&pe->table_group, dev);
>  		return 0;
>  	case BUS_NOTIFY_DEL_DEVICE:

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [kernel] powerpc/powernv/sriov: Register IOMMU groups for VFs

[Michael Ellerman]

On Mon, 2019-02-18 at 05:58:01 UTC, Alexey Kardashevskiy wrote:
> The compound IOMMU group rework moved iommu_register_group() together in
> pnv_pci_ioda_setup_iommu_api() (which is a part of ppc_md.pcibios_fixup).
> As the result, pnv_ioda_setup_bus_iommu_group() does not create groups
> any more, it only adds devices to groups.
> 
> This works fine for boot time devices. However IOMMU groups for SRIOV's
> VFs were added by pnv_ioda_setup_bus_iommu_group() so this got broken:
> pnv_tce_iommu_bus_notifier() expects a group to be registered for VF
> and it is not.
> 
> This adds missing group registration and adds a NULL pointer check into
> the bus notifier so we won't stop in xmon if there is no group, although
> it is not expected to happen now.
> 
> Fixes: 0bd971676e68 "powerpc/powernv/npu: Add compound IOMMU groups"
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> Reported-by: Santwana Samantray<santwana.samantray@in.ibm.com>
> Reviewed-by: David Gibson <david@gibson.dropbear.id.au>

Applied to powerpc fixes, thanks.

https://git.kernel.org/powerpc/c/8f5b27347e88b171c755562f0090ce40

cheers