Re: [PATCH v2 7/7] iommu/arm-smmu: Support non-PCI devices with SMMUv3

[Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>]

On 17/06/16 02:54, Leizhen (ThunderTown) wrote:
> Hi,
> I only applied these patch series on lastest 4.7-rc3, is there any patches I missed?
> According to my test, it seems can not work. The problem is:

Thanks, that's helpful (if irritating) confirmation.

Would you mind trying this patch on top to see if that helps: 
http://article.gmane.org/gmane.linux.kernel.iommu/13991

Failing that, we could go back to the forced-device-creation workaround 
(although I was hoping we'd be able to avoid it):

-----8<-----
diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index e5acd601ee96..556a48f836b3 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -2683,9 +2683,15 @@ module_exit(arm_smmu_exit);

  static int __init arm_smmu_of_init(struct device_node *np)
  {
+	static bool registered;
+
  	of_iommu_set_ops(np, &arm_smmu_ops);

-	return 0;
+	if (!registered)
+		registered = !arm_smmu_init();
+
+	return PTR_ERR_OR_ZERO(of_platform_device_create(np, NULL,
+					platform_bus_type.dev_root));
  }
  IOMMU_OF_DECLARE(arm_smmuv3, "arm,smmu-v3", arm_smmu_of_init);

-----8<-----

Robin.

>
> [    0.916536] Call trace:
> [    0.918967] [<ffff00000808882c>] dump_backtrace+0x0/0x1ac
> [    0.924361] [<ffff0000080889ec>] show_stack+0x14/0x1c
> [    0.929384] [<ffff00000833c290>] dump_stack+0x8c/0xac
> [    0.934425] [<ffff000008093550>] do_iommu_attach.isra.11+0x24/0x7c				//called iommu_get_domain_for_dev, but currently dev->iommu_group is NULL
> [    0.940589] [<ffff000008093604>] __iommu_attach_notifier.part.13+0x5c/0xc8
> [    0.947443] [<ffff000008093690>] __iommu_attach_notifier+0x20/0x2c
> [    0.953608] [<ffff0000080d4c88>] notifier_call_chain+0x4c/0x88
> [    0.959408] [<ffff0000080d5010>] __blocking_notifier_call_chain+0x44/0x74
> [    0.966159] [<ffff0000080d5054>] blocking_notifier_call_chain+0x14/0x1c
> [    0.972754] [<ffff000008489600>] device_add+0x3ac/0x53c
> [    0.977959] [<ffff000008671810>] of_device_add+0x58/0x64
> [    0.983240] [<ffff0000086720c8>] of_platform_device_create_pdata+0x90/0xe8
> [    0.990077] [<ffff00000867220c>] of_platform_bus_create+0xd0/0x31c
> [    0.996265] [<ffff000008672574>] of_platform_populate+0x48/0xac
> [    1.002169] [<ffff000008af2654>] arm64_device_init+0x30/0x58
> [    1.007813] [<ffff000008081a14>] do_one_initcall+0x38/0x128
> [    1.013354] [<ffff000008af0cc4>] kernel_init_freeable+0x14c/0x1ec
> [    1.019439] [<ffff0000087a92d0>] kernel_init+0x10/0xfc
> [    1.024556] [<ffff000008084e10>] ret_from_fork+0x10/0x40
>
> [    0.906207] Failed to set up IOMMU for device tst_smmu; retaining platform DMA ops		//error info
>
> [    0.963782] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> [    0.970442] ACPI: Interpreter disabled.
> [    0.974429] arm-smmu-v3 d0040000.smmu_alg: ias 44-bit, oas 44-bit (features 0x00000f0d)
>
> [    0.996948] Call trace:
> [    0.999400] [<ffff00000808882c>] dump_backtrace+0x0/0x1ac
> [    1.004787] [<ffff0000080889ec>] show_stack+0x14/0x1c
> [    1.009846] [<ffff00000833c290>] dump_stack+0x8c/0xac
> [    1.014905] [<ffff0000084829d4>] arm_smmu_add_device+0x44/0x160
> [    1.020801] [<ffff00000847a218>] add_iommu_group+0x20/0x44
> [    1.026291] [<ffff00000848a43c>] bus_for_each_dev+0x58/0x98
> [    1.031866] [<ffff00000847b220>] bus_set_iommu+0x9c/0xf8
> [    1.037150] [<ffff000008b1b420>] arm_smmu_init+0x78/0x84
> [    1.042475] [<ffff000008081a14>] do_one_initcall+0x38/0x128
> [    1.048043] [<ffff000008af0cc4>] kernel_init_freeable+0x14c/0x1ec
> [    1.054138] [<ffff0000087a931c>] kernel_init+0x10/0xfc
> [    1.059264] [<ffff000008084e10>] ret_from_fork+0x10/0x40
>
> [    1.171026] iommu: Adding device tst_smmu to group 0						//It's already late.
>
> On 2016/6/15 9:22, Leizhen (ThunderTown) wrote:
>>
>>
>> On 2016/6/14 23:16, Will Deacon wrote:
>>> [adding ThunderTown, since he might be able to test this for us]
>>
>> OK. I'm so glad to do it.
>>
>>>
>>> On Fri, Jun 03, 2016 at 06:15:42PM +0100, Robin Murphy wrote:
>>>> With the device <-> stream ID relationship suitably abstracted and
>>>> of_xlate() hooked up, we no longer have any PCI-specifics in play,
>>>> so adding support for the simpler kinds of platform device (a single
>>>> unique stream ID each) becomes trivial; let's do it!
>>>>
>>>> Signed-off-by: Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>
>>>> ---
>>>>
>>>> v2: New. Consider this one "extra bonus material" as I'm not sure there
>>>>      are even any suitable devices on our model to test it with (it
>>>>      _should_ be OK, given that I know the basic infrastructure on either
>>>>      side works...)
>>>>
>>>>   drivers/iommu/Kconfig       |  2 +-
>>>>   drivers/iommu/arm-smmu-v3.c | 36 ++++++++++++++++++++++++++++++------
>>>>   2 files changed, 31 insertions(+), 7 deletions(-)
>>>>
>>>> diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
>>>> index ad0860383cb3..d1c66afefeed 100644
>>>> --- a/drivers/iommu/Kconfig
>>>> +++ b/drivers/iommu/Kconfig
>>>> @@ -308,7 +308,7 @@ config ARM_SMMU
>>>>
>>>>   config ARM_SMMU_V3
>>>>   	bool "ARM Ltd. System MMU Version 3 (SMMUv3) Support"
>>>> -	depends on ARM64 && PCI
>>>> +	depends on ARM64
>>>>   	select IOMMU_API
>>>>   	select IOMMU_IO_PGTABLE_LPAE
>>>>   	select GENERIC_MSI_IRQ_DOMAIN
>>>> diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
>>>> index 28dcc5ca237e..6379f0ab24fc 100644
>>>> --- a/drivers/iommu/arm-smmu-v3.c
>>>> +++ b/drivers/iommu/arm-smmu-v3.c
>>>> @@ -35,6 +35,8 @@
>>>>   #include <linux/pci.h>
>>>>   #include <linux/platform_device.h>
>>>>
>>>> +#include <linux/amba/bus.h>
>>>> +
>>>>   #include "io-pgtable.h"
>>>>
>>>>   /* MMIO registers */
>>>> @@ -1773,6 +1775,22 @@ static void arm_smmu_remove_device(struct device *dev)
>>>>   	iommu_group_remove_device(dev);
>>>>   }
>>>>
>>>> +static struct iommu_group *arm_smmu_device_group(struct device *dev)
>>>> +{
>>>> +	struct iommu_group *group;
>>>> +
>>>> +	/*
>>>> +	 * We've currently no means of grouping non-PCI masters, so
>>>> +	 * there'd better not be any non-unique stream IDs in the DT...
>>>> +	 */
>>>
>>> Worse: what if a SID in the DT aliases with a PCI master? It might be
>>> nice to have some basic snity checking, at least.
>>>
>>> Will
>>>
>>> .
>>>
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>