[PATCH kernel 0/4] Fixes for powerpc/pseries/iommu: Make use of DDW for indirect mapping

[PATCH kernel 0/4] Fixes for powerpc/pseries/iommu: Make use of DDW for indirect mapping

[Alexey Kardashevskiy]

Found some issues on SRIOV enabled PHYP.
It probably should be one patch, or not?

Please comment. Thanks.



Alexey Kardashevskiy (4):
  powerpc/pseries/iommu: Fix indentations
  powerpc/pseries/iommu: Use correct vfree for it_map
  powerpc/pseries/iommu: Check if the default window in use before
    removing it
  powerpc/pseries/iommu: Create huge DMA window if no MMIO32 is present

 arch/powerpc/platforms/pseries/iommu.c | 33 +++++++++++++-------------
 1 file changed, 17 insertions(+), 16 deletions(-)

-- 
2.30.2

[PATCH kernel 1/4] powerpc/pseries/iommu: Fix indentations

[Alexey Kardashevskiy]

This fixes broken indentations. The first hunk might suggest that
the introducing patch was applied incorrectly but it is correct.

Fixes: 381ceda88c4c ("powerpc/pseries/iommu: Make use of DDW for indirect mapping")
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
 arch/powerpc/platforms/pseries/iommu.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index 269f61d519c2..09d59088864a 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -1404,8 +1404,8 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn)
 			dev_info(&dev->dev, "failed to map DMA window for %pOF: %d\n",
 				 dn, ret);
 
-		/* Make sure to clean DDW if any TCE was set*/
-		clean_dma_window(pdn, win64->value);
+			/* Make sure to clean DDW if any TCE was set*/
+			clean_dma_window(pdn, win64->value);
 			goto out_del_list;
 		}
 	} else {
@@ -1490,7 +1490,7 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn)
 	if (pmem_present && ddw_enabled && direct_mapping && len == max_ram_len)
 		dev->dev.bus_dma_limit = dev->dev.archdata.dma_offset + (1ULL << len);
 
-    return ddw_enabled && direct_mapping;
+	return ddw_enabled && direct_mapping;
 }
 
 static void pci_dma_dev_setup_pSeriesLP(struct pci_dev *dev)
-- 
2.30.2

[PATCH kernel 2/4] powerpc/pseries/iommu: Use correct vfree for it_map

[Alexey Kardashevskiy]

The it_map array is vzalloc'ed so use vfree() for it when creating
a huge DMA window failed for whatever reason.

While at this, write zero to it_map.

Fixes: 381ceda88c4c ("powerpc/pseries/iommu: Make use of DDW for indirect mapping")
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
 arch/powerpc/platforms/pseries/iommu.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index 09d59088864a..45564547cd80 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -1440,7 +1440,8 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn)
 		/* Keep default DMA window stuct if removed */
 		if (default_win_removed) {
 			tbl->it_size = 0;
-			kfree(tbl->it_map);
+			vfree(tbl->it_map);
+			tbl->it_map = NULL;
 		}
 
 		set_iommu_table_base(&dev->dev, newtbl);
-- 
2.30.2

[PATCH kernel 3/4] powerpc/pseries/iommu: Check if the default window in use before removing it

[Alexey Kardashevskiy]

At the moment this check is performed after we remove the default window
which is late and disallows to revert whatever changes enable_ddw()
has made to DMA windows.

This moves the check and error exit before removing the window.

This raised the message severity from "debug" to "warning" as this
should not happen in practice and cannot be triggered by the userspace.

Fixes: 381ceda88c4c ("powerpc/pseries/iommu: Make use of DDW for indirect mapping")
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
 arch/powerpc/platforms/pseries/iommu.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index 45564547cd80..9bbcfdece9e3 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -1302,6 +1302,12 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn)
 		struct property *default_win;
 		int reset_win_ext;
 
+		/* DDW + IOMMU on single window may fail if there is any allocation */
+		if (iommu_table_in_use(tbl)) {
+			dev_warn(&dev->dev, "current IOMMU table in use, can't be replaced.\n");
+			goto out_failed;
+		}
+
 		default_win = of_find_property(pdn, "ibm,dma-window", NULL);
 		if (!default_win)
 			goto out_failed;
@@ -1356,12 +1362,6 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn)
 			query.largest_available_block,
 			1ULL << page_shift);
 
-		/* DDW + IOMMU on single window may fail if there is any allocation */
-		if (default_win_removed && iommu_table_in_use(tbl)) {
-			dev_dbg(&dev->dev, "current IOMMU table in use, can't be replaced.\n");
-			goto out_failed;
-		}
-
 		len = order_base_2(query.largest_available_block << page_shift);
 		win_name = DMA64_PROPNAME;
 	} else {
-- 
2.30.2

[PATCH kernel 4/4] powerpc/pseries/iommu: Create huge DMA window if no MMIO32 is present

[Alexey Kardashevskiy]

The iommu_init_table() helper takes an address range to reserve in
the IOMMU table being initialized to exclude MMIO addresses, this is
useful if the window stretches far beyond 4GB (although wastes some TCEs).
At the moment the code searches for such MMIO32 range and fails if none
found which is considered a problem while it really is not: it is actually
better as this says there is no MMIO32 to reserve and we can use
usually wasted TCEs. Furthermore PHYP never actually allows creating
windows starting at busaddress=0 so this MMIO32 range is never useful.

This removes error exit and initializes the table with zero range if
no MMIO32 is detected.

Fixes: 381ceda88c4c ("powerpc/pseries/iommu: Make use of DDW for indirect mapping")
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
 arch/powerpc/platforms/pseries/iommu.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c
index 9bbcfdece9e3..258e98c10095 100644
--- a/arch/powerpc/platforms/pseries/iommu.c
+++ b/arch/powerpc/platforms/pseries/iommu.c
@@ -1411,18 +1411,19 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn)
 	} else {
 		struct iommu_table *newtbl;
 		int i;
+		unsigned long start = 0, end = 0;
 
 		for (i = 0; i < ARRAY_SIZE(pci->phb->mem_resources); i++) {
 			const unsigned long mask = IORESOURCE_MEM_64 | IORESOURCE_MEM;
 
 			/* Look for MMIO32 */
-			if ((pci->phb->mem_resources[i].flags & mask) == IORESOURCE_MEM)
+			if ((pci->phb->mem_resources[i].flags & mask) == IORESOURCE_MEM) {
+				start = pci->phb->mem_resources[i].start;
+				end = pci->phb->mem_resources[i].end;
 				break;
+			}
 		}
 
-		if (i == ARRAY_SIZE(pci->phb->mem_resources))
-			goto out_del_list;
-
 		/* New table for using DDW instead of the default DMA window */
 		newtbl = iommu_pseries_alloc_table(pci->phb->node);
 		if (!newtbl) {
@@ -1432,8 +1433,7 @@ static bool enable_ddw(struct pci_dev *dev, struct device_node *pdn)
 
 		iommu_table_setparms_common(newtbl, pci->phb->bus->number, create.liobn, win_addr,
 					    1UL << len, page_shift, NULL, &iommu_table_lpar_multi_ops);
-		iommu_init_table(newtbl, pci->phb->node, pci->phb->mem_resources[i].start,
-				 pci->phb->mem_resources[i].end);
+		iommu_init_table(newtbl, pci->phb->node, start, end);
 
 		pci->table_group->tables[1] = newtbl;
 
-- 
2.30.2

Re: [PATCH kernel 0/4] Fixes for powerpc/pseries/iommu: Make use of DDW for indirect mapping

[Michael Ellerman]

On Thu, 21 Oct 2021 00:23:11 +1100, Alexey Kardashevskiy wrote:
> Found some issues on SRIOV enabled PHYP.
> It probably should be one patch, or not?
> 
> Please comment. Thanks.
> 
> 
> 
> [...]

Patches 2-4 applied to powerpc/fixes.

[2/4] powerpc/pseries/iommu: Use correct vfree for it_map
      https://git.kernel.org/powerpc/c/41ee7232fa5f3c034f22baa52bc287e494e2129a
[3/4] powerpc/pseries/iommu: Check if the default window in use before removing it
      https://git.kernel.org/powerpc/c/92fe01b7c655b9767724e7d62bdded0761d232ff
[4/4] powerpc/pseries/iommu: Create huge DMA window if no MMIO32 is present
      https://git.kernel.org/powerpc/c/d853adc7adf601d7d6823afe3ed396065a3e08d1

cheers

Re: [PATCH kernel 0/4] Fixes for powerpc/pseries/iommu: Make use of DDW for indirect mapping

[Frederic Barrat]

On 02/11/2021 03:43, Michael Ellerman wrote:
> On Thu, 21 Oct 2021 00:23:11 +1100, Alexey Kardashevskiy wrote:
>> Found some issues on SRIOV enabled PHYP.
>> It probably should be one patch, or not?
>>
>> Please comment. Thanks.
>>
>>
>>
>> [...]
> 
> Patches 2-4 applied to powerpc/fixes.


Any reason why patch 1 was not applied? The indents are still wrong in 
5.16-rc1

   Fred



> [2/4] powerpc/pseries/iommu: Use correct vfree for it_map
>        https://git.kernel.org/powerpc/c/41ee7232fa5f3c034f22baa52bc287e494e2129a
> [3/4] powerpc/pseries/iommu: Check if the default window in use before removing it
>        https://git.kernel.org/powerpc/c/92fe01b7c655b9767724e7d62bdded0761d232ff
> [4/4] powerpc/pseries/iommu: Create huge DMA window if no MMIO32 is present
>        https://git.kernel.org/powerpc/c/d853adc7adf601d7d6823afe3ed396065a3e08d1
> 
> cheers
> 

Re: [PATCH kernel 0/4] Fixes for powerpc/pseries/iommu: Make use of DDW for indirect mapping

[Frederic Barrat]

On 17/11/2021 11:19, Frederic Barrat wrote:
> 
> 
> On 02/11/2021 03:43, Michael Ellerman wrote:
>> On Thu, 21 Oct 2021 00:23:11 +1100, Alexey Kardashevskiy wrote:
>>> Found some issues on SRIOV enabled PHYP.
>>> It probably should be one patch, or not?
>>>
>>> Please comment. Thanks.
>>>
>>>
>>>
>>> [...]
>>
>> Patches 2-4 applied to powerpc/fixes.
> 
> 
> Any reason why patch 1 was not applied? The indents are still wrong in 
> 5.16-rc1


Ah, Alexey has reposted it later:
https://patchwork.ozlabs.org/project/linuxppc-dev/patch/20211108040320.3857636-3-aik@ozlabs.ru/

Apologies for the noise...

   Fred



> 
>> [2/4] powerpc/pseries/iommu: Use correct vfree for it_map
>>        
>> https://git.kernel.org/powerpc/c/41ee7232fa5f3c034f22baa52bc287e494e2129a
>> [3/4] powerpc/pseries/iommu: Check if the default window in use before 
>> removing it
>>        
>> https://git.kernel.org/powerpc/c/92fe01b7c655b9767724e7d62bdded0761d232ff
>> [4/4] powerpc/pseries/iommu: Create huge DMA window if no MMIO32 is 
>> present
>>        
>> https://git.kernel.org/powerpc/c/d853adc7adf601d7d6823afe3ed396065a3e08d1
>>
>> cheers
>>

Re: [PATCH kernel 0/4] Fixes for powerpc/pseries/iommu: Make use of DDW for indirect mapping

[Michael Ellerman]

Frederic Barrat <fbarrat@linux.ibm.com> writes:
> On 02/11/2021 03:43, Michael Ellerman wrote:
>> On Thu, 21 Oct 2021 00:23:11 +1100, Alexey Kardashevskiy wrote:
>>> Found some issues on SRIOV enabled PHYP.
>>> It probably should be one patch, or not?
>>>
>>> Please comment. Thanks.
>>>
>>> [...]
>> 
>> Patches 2-4 applied to powerpc/fixes.
>
>
> Any reason why patch 1 was not applied? The indents are still wrong in 
> 5.16-rc1

Just because it was late in the rc series, and it wasn't crucial that
the whitespace fix go into 5.15.

And yeah Alexey has now reposted it and I have picked that series up.

cheers