* [PATCH v2 0/7] Add PCI ATS support to Arm SMMUv3 @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon-5wv7dgnIgG8 Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, sudeep.holla-5wv7dgnIgG8, robin.murphy-5wv7dgnIgG8, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A This series enables PCI ATS in SMMUv3. Changes since v1 [1]: * Simplify the SMMU structures (patches 2-4 are new). * Don't enable ATS for devices that are attached to a bypass domain, because in that case a translation request would cause F_BAD_ATS_TREQ. Translation requests in that case cause F_BAD_ATS_TREQ. Enable ATS in attach_dev() rather than add_device(). * Enable ATS for stage-1 and stage-2 STEs alike. There is no benefit to disabling ATS for stage-2 domains. [1] https://www.spinics.net/lists/arm-kernel/msg714628.html Jean-Philippe Brucker (7): ACPI/IORT: Check ATS capability in root complex nodes iommu/arm-smmu-v3: Rename arm_smmu_master_data to arm_smmu_master iommu/arm-smmu-v3: Store SteamIDs in master iommu/arm-smmu-v3: Add a master->domain pointer iommu/arm-smmu-v3: Link domains and devices iommu/arm-smmu-v3: Add support for PCI ATS iommu/arm-smmu-v3: Disable tagged pointers drivers/acpi/arm64/iort.c | 11 ++ drivers/iommu/arm-smmu-v3.c | 340 ++++++++++++++++++++++++++++-------- include/linux/iommu.h | 4 + 3 files changed, 286 insertions(+), 69 deletions(-) -- 2.21.0 ^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v2 0/7] Add PCI ATS support to Arm SMMUv3 @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: lorenzo.pieralisi, eric.auger, zhongmiao, okaya, joro, rjw, linux-acpi, iommu, hanjun.guo, sudeep.holla, robin.murphy, linux-arm-kernel, lenb This series enables PCI ATS in SMMUv3. Changes since v1 [1]: * Simplify the SMMU structures (patches 2-4 are new). * Don't enable ATS for devices that are attached to a bypass domain, because in that case a translation request would cause F_BAD_ATS_TREQ. Translation requests in that case cause F_BAD_ATS_TREQ. Enable ATS in attach_dev() rather than add_device(). * Enable ATS for stage-1 and stage-2 STEs alike. There is no benefit to disabling ATS for stage-2 domains. [1] https://www.spinics.net/lists/arm-kernel/msg714628.html Jean-Philippe Brucker (7): ACPI/IORT: Check ATS capability in root complex nodes iommu/arm-smmu-v3: Rename arm_smmu_master_data to arm_smmu_master iommu/arm-smmu-v3: Store SteamIDs in master iommu/arm-smmu-v3: Add a master->domain pointer iommu/arm-smmu-v3: Link domains and devices iommu/arm-smmu-v3: Add support for PCI ATS iommu/arm-smmu-v3: Disable tagged pointers drivers/acpi/arm64/iort.c | 11 ++ drivers/iommu/arm-smmu-v3.c | 340 ++++++++++++++++++++++++++++-------- include/linux/iommu.h | 4 + 3 files changed, 286 insertions(+), 69 deletions(-) -- 2.21.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v2 0/7] Add PCI ATS support to Arm SMMUv3 @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb This series enables PCI ATS in SMMUv3. Changes since v1 [1]: * Simplify the SMMU structures (patches 2-4 are new). * Don't enable ATS for devices that are attached to a bypass domain, because in that case a translation request would cause F_BAD_ATS_TREQ. Translation requests in that case cause F_BAD_ATS_TREQ. Enable ATS in attach_dev() rather than add_device(). * Enable ATS for stage-1 and stage-2 STEs alike. There is no benefit to disabling ATS for stage-2 domains. [1] https://www.spinics.net/lists/arm-kernel/msg714628.html Jean-Philippe Brucker (7): ACPI/IORT: Check ATS capability in root complex nodes iommu/arm-smmu-v3: Rename arm_smmu_master_data to arm_smmu_master iommu/arm-smmu-v3: Store SteamIDs in master iommu/arm-smmu-v3: Add a master->domain pointer iommu/arm-smmu-v3: Link domains and devices iommu/arm-smmu-v3: Add support for PCI ATS iommu/arm-smmu-v3: Disable tagged pointers drivers/acpi/arm64/iort.c | 11 ++ drivers/iommu/arm-smmu-v3.c | 340 ++++++++++++++++++++++++++++-------- include/linux/iommu.h | 4 + 3 files changed, 286 insertions(+), 69 deletions(-) -- 2.21.0 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v2 0/7] Add PCI ATS support to Arm SMMUv3 @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: iommu, linux-arm-kernel, linux-acpi, robin.murphy, joro, hanjun.guo, lorenzo.pieralisi, sudeep.holla, rjw, lenb, okaya, zhongmiao, eric.auger This series enables PCI ATS in SMMUv3. Changes since v1 [1]: * Simplify the SMMU structures (patches 2-4 are new). * Don't enable ATS for devices that are attached to a bypass domain, because in that case a translation request would cause F_BAD_ATS_TREQ. Translation requests in that case cause F_BAD_ATS_TREQ. Enable ATS in attach_dev() rather than add_device(). * Enable ATS for stage-1 and stage-2 STEs alike. There is no benefit to disabling ATS for stage-2 domains. [1] https://www.spinics.net/lists/arm-kernel/msg714628.html Jean-Philippe Brucker (7): ACPI/IORT: Check ATS capability in root complex nodes iommu/arm-smmu-v3: Rename arm_smmu_master_data to arm_smmu_master iommu/arm-smmu-v3: Store SteamIDs in master iommu/arm-smmu-v3: Add a master->domain pointer iommu/arm-smmu-v3: Link domains and devices iommu/arm-smmu-v3: Add support for PCI ATS iommu/arm-smmu-v3: Disable tagged pointers drivers/acpi/arm64/iort.c | 11 ++ drivers/iommu/arm-smmu-v3.c | 340 ++++++++++++++++++++++++++++-------- include/linux/iommu.h | 4 + 3 files changed, 286 insertions(+), 69 deletions(-) -- 2.21.0 ^ permalink raw reply [flat|nested] 64+ messages in thread
[parent not found: <20190409165245.26500-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>]
* [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon-5wv7dgnIgG8 Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, sudeep.holla-5wv7dgnIgG8, robin.murphy-5wv7dgnIgG8, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A Root complex node in IORT has a bit telling whether it supports ATS or not. Store this bit in the IOMMU fwspec when setting up a device, so it can be accessed later by an IOMMU driver. Use the negative version (NO_ATS) at the moment because it's not clear if/how the bit needs to be integrated in other firmware descriptions. The SMMU has a feature bit telling if it supports ATS, which might be sufficient in most systems for deciding whether or not we should enable the ATS capability in endpoints. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> --- drivers/acpi/arm64/iort.c | 11 +++++++++++ include/linux/iommu.h | 4 ++++ 2 files changed, 15 insertions(+) diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c index e48894e002ba..7f2c1c9c6b38 100644 --- a/drivers/acpi/arm64/iort.c +++ b/drivers/acpi/arm64/iort.c @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); } +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) +{ + struct acpi_iort_root_complex *pci_rc; + + pci_rc = (struct acpi_iort_root_complex *)node->node_data; + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; +} + /** * iort_iommu_configure - Set-up IOMMU configuration for a device. * @@ -1063,6 +1071,9 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev) info.node = node; err = pci_for_each_dma_alias(to_pci_dev(dev), iort_pci_iommu_init, &info); + + if (!err && !iort_pci_rc_supports_ats(node)) + dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_NO_ATS; } else { int i = 0; diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 3dbeb457fb16..ed6738c358ca 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -509,10 +509,14 @@ struct iommu_fwspec { const struct iommu_ops *ops; struct fwnode_handle *iommu_fwnode; void *iommu_priv; + u32 flags; unsigned int num_ids; u32 ids[1]; }; +/* Firmware disabled ATS in the root complex */ +#define IOMMU_FWSPEC_PCI_NO_ATS (1 << 0) + int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwnode, const struct iommu_ops *ops); void iommu_fwspec_free(struct device *dev); -- 2.21.0 ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: lorenzo.pieralisi, eric.auger, zhongmiao, okaya, joro, rjw, linux-acpi, iommu, hanjun.guo, sudeep.holla, robin.murphy, linux-arm-kernel, lenb Root complex node in IORT has a bit telling whether it supports ATS or not. Store this bit in the IOMMU fwspec when setting up a device, so it can be accessed later by an IOMMU driver. Use the negative version (NO_ATS) at the moment because it's not clear if/how the bit needs to be integrated in other firmware descriptions. The SMMU has a feature bit telling if it supports ATS, which might be sufficient in most systems for deciding whether or not we should enable the ATS capability in endpoints. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/acpi/arm64/iort.c | 11 +++++++++++ include/linux/iommu.h | 4 ++++ 2 files changed, 15 insertions(+) diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c index e48894e002ba..7f2c1c9c6b38 100644 --- a/drivers/acpi/arm64/iort.c +++ b/drivers/acpi/arm64/iort.c @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); } +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) +{ + struct acpi_iort_root_complex *pci_rc; + + pci_rc = (struct acpi_iort_root_complex *)node->node_data; + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; +} + /** * iort_iommu_configure - Set-up IOMMU configuration for a device. * @@ -1063,6 +1071,9 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev) info.node = node; err = pci_for_each_dma_alias(to_pci_dev(dev), iort_pci_iommu_init, &info); + + if (!err && !iort_pci_rc_supports_ats(node)) + dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_NO_ATS; } else { int i = 0; diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 3dbeb457fb16..ed6738c358ca 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -509,10 +509,14 @@ struct iommu_fwspec { const struct iommu_ops *ops; struct fwnode_handle *iommu_fwnode; void *iommu_priv; + u32 flags; unsigned int num_ids; u32 ids[1]; }; +/* Firmware disabled ATS in the root complex */ +#define IOMMU_FWSPEC_PCI_NO_ATS (1 << 0) + int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwnode, const struct iommu_ops *ops); void iommu_fwspec_free(struct device *dev); -- 2.21.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb Root complex node in IORT has a bit telling whether it supports ATS or not. Store this bit in the IOMMU fwspec when setting up a device, so it can be accessed later by an IOMMU driver. Use the negative version (NO_ATS) at the moment because it's not clear if/how the bit needs to be integrated in other firmware descriptions. The SMMU has a feature bit telling if it supports ATS, which might be sufficient in most systems for deciding whether or not we should enable the ATS capability in endpoints. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/acpi/arm64/iort.c | 11 +++++++++++ include/linux/iommu.h | 4 ++++ 2 files changed, 15 insertions(+) diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c index e48894e002ba..7f2c1c9c6b38 100644 --- a/drivers/acpi/arm64/iort.c +++ b/drivers/acpi/arm64/iort.c @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); } +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) +{ + struct acpi_iort_root_complex *pci_rc; + + pci_rc = (struct acpi_iort_root_complex *)node->node_data; + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; +} + /** * iort_iommu_configure - Set-up IOMMU configuration for a device. * @@ -1063,6 +1071,9 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev) info.node = node; err = pci_for_each_dma_alias(to_pci_dev(dev), iort_pci_iommu_init, &info); + + if (!err && !iort_pci_rc_supports_ats(node)) + dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_NO_ATS; } else { int i = 0; diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 3dbeb457fb16..ed6738c358ca 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -509,10 +509,14 @@ struct iommu_fwspec { const struct iommu_ops *ops; struct fwnode_handle *iommu_fwnode; void *iommu_priv; + u32 flags; unsigned int num_ids; u32 ids[1]; }; +/* Firmware disabled ATS in the root complex */ +#define IOMMU_FWSPEC_PCI_NO_ATS (1 << 0) + int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwnode, const struct iommu_ops *ops); void iommu_fwspec_free(struct device *dev); -- 2.21.0 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: iommu, linux-arm-kernel, linux-acpi, robin.murphy, joro, hanjun.guo, lorenzo.pieralisi, sudeep.holla, rjw, lenb, okaya, zhongmiao, eric.auger Root complex node in IORT has a bit telling whether it supports ATS or not. Store this bit in the IOMMU fwspec when setting up a device, so it can be accessed later by an IOMMU driver. Use the negative version (NO_ATS) at the moment because it's not clear if/how the bit needs to be integrated in other firmware descriptions. The SMMU has a feature bit telling if it supports ATS, which might be sufficient in most systems for deciding whether or not we should enable the ATS capability in endpoints. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/acpi/arm64/iort.c | 11 +++++++++++ include/linux/iommu.h | 4 ++++ 2 files changed, 15 insertions(+) diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c index e48894e002ba..7f2c1c9c6b38 100644 --- a/drivers/acpi/arm64/iort.c +++ b/drivers/acpi/arm64/iort.c @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); } +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) +{ + struct acpi_iort_root_complex *pci_rc; + + pci_rc = (struct acpi_iort_root_complex *)node->node_data; + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; +} + /** * iort_iommu_configure - Set-up IOMMU configuration for a device. * @@ -1063,6 +1071,9 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev) info.node = node; err = pci_for_each_dma_alias(to_pci_dev(dev), iort_pci_iommu_init, &info); + + if (!err && !iort_pci_rc_supports_ats(node)) + dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_NO_ATS; } else { int i = 0; diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 3dbeb457fb16..ed6738c358ca 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -509,10 +509,14 @@ struct iommu_fwspec { const struct iommu_ops *ops; struct fwnode_handle *iommu_fwnode; void *iommu_priv; + u32 flags; unsigned int num_ids; u32 ids[1]; }; +/* Firmware disabled ATS in the root complex */ +#define IOMMU_FWSPEC_PCI_NO_ATS (1 << 0) + int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwnode, const struct iommu_ops *ops); void iommu_fwspec_free(struct device *dev); -- 2.21.0 ^ permalink raw reply related [flat|nested] 64+ messages in thread
[parent not found: <20190409165245.26500-2-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>]
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-15 13:21 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-15 13:21 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, sudeep.holla-5wv7dgnIgG8, robin.murphy-5wv7dgnIgG8, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A On Tue, Apr 09, 2019 at 05:52:39PM +0100, Jean-Philippe Brucker wrote: > Root complex node in IORT has a bit telling whether it supports ATS or > not. Store this bit in the IOMMU fwspec when setting up a device, so it > can be accessed later by an IOMMU driver. > > Use the negative version (NO_ATS) at the moment because it's not clear > if/how the bit needs to be integrated in other firmware descriptions. The > SMMU has a feature bit telling if it supports ATS, which might be > sufficient in most systems for deciding whether or not we should enable > the ATS capability in endpoints. Hmm, the SMMUv3 architecture manual is pretty explicit about this: | It [SMMU_IDR0.ATS] does not guarantee that client devices and intermediate | components also support ATS and this must be determined separately. so we may need to extend the PCI bindings to describe this. I think the negative logic is likely to get in the way if that's the case. > Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> > --- > drivers/acpi/arm64/iort.c | 11 +++++++++++ > include/linux/iommu.h | 4 ++++ > 2 files changed, 15 insertions(+) > > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c > index e48894e002ba..7f2c1c9c6b38 100644 > --- a/drivers/acpi/arm64/iort.c > +++ b/drivers/acpi/arm64/iort.c > @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) > dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); > } > > +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) > +{ > + struct acpi_iort_root_complex *pci_rc; > + > + pci_rc = (struct acpi_iort_root_complex *)node->node_data; > + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; > +} Do we need to worry about the "noats" command-line option here? It feels like we should be checking with the PCI subsystem before telling the SMMU we're good to go. I'll need Lorenzo's ack on this. Will ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-15 13:21 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-15 13:21 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: lorenzo.pieralisi, eric.auger, zhongmiao, okaya, joro, rjw, linux-acpi, iommu, hanjun.guo, sudeep.holla, robin.murphy, linux-arm-kernel, lenb On Tue, Apr 09, 2019 at 05:52:39PM +0100, Jean-Philippe Brucker wrote: > Root complex node in IORT has a bit telling whether it supports ATS or > not. Store this bit in the IOMMU fwspec when setting up a device, so it > can be accessed later by an IOMMU driver. > > Use the negative version (NO_ATS) at the moment because it's not clear > if/how the bit needs to be integrated in other firmware descriptions. The > SMMU has a feature bit telling if it supports ATS, which might be > sufficient in most systems for deciding whether or not we should enable > the ATS capability in endpoints. Hmm, the SMMUv3 architecture manual is pretty explicit about this: | It [SMMU_IDR0.ATS] does not guarantee that client devices and intermediate | components also support ATS and this must be determined separately. so we may need to extend the PCI bindings to describe this. I think the negative logic is likely to get in the way if that's the case. > Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> > --- > drivers/acpi/arm64/iort.c | 11 +++++++++++ > include/linux/iommu.h | 4 ++++ > 2 files changed, 15 insertions(+) > > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c > index e48894e002ba..7f2c1c9c6b38 100644 > --- a/drivers/acpi/arm64/iort.c > +++ b/drivers/acpi/arm64/iort.c > @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) > dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); > } > > +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) > +{ > + struct acpi_iort_root_complex *pci_rc; > + > + pci_rc = (struct acpi_iort_root_complex *)node->node_data; > + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; > +} Do we need to worry about the "noats" command-line option here? It feels like we should be checking with the PCI subsystem before telling the SMMU we're good to go. I'll need Lorenzo's ack on this. Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-15 13:21 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-15 13:21 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb On Tue, Apr 09, 2019 at 05:52:39PM +0100, Jean-Philippe Brucker wrote: > Root complex node in IORT has a bit telling whether it supports ATS or > not. Store this bit in the IOMMU fwspec when setting up a device, so it > can be accessed later by an IOMMU driver. > > Use the negative version (NO_ATS) at the moment because it's not clear > if/how the bit needs to be integrated in other firmware descriptions. The > SMMU has a feature bit telling if it supports ATS, which might be > sufficient in most systems for deciding whether or not we should enable > the ATS capability in endpoints. Hmm, the SMMUv3 architecture manual is pretty explicit about this: | It [SMMU_IDR0.ATS] does not guarantee that client devices and intermediate | components also support ATS and this must be determined separately. so we may need to extend the PCI bindings to describe this. I think the negative logic is likely to get in the way if that's the case. > Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> > --- > drivers/acpi/arm64/iort.c | 11 +++++++++++ > include/linux/iommu.h | 4 ++++ > 2 files changed, 15 insertions(+) > > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c > index e48894e002ba..7f2c1c9c6b38 100644 > --- a/drivers/acpi/arm64/iort.c > +++ b/drivers/acpi/arm64/iort.c > @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) > dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); > } > > +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) > +{ > + struct acpi_iort_root_complex *pci_rc; > + > + pci_rc = (struct acpi_iort_root_complex *)node->node_data; > + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; > +} Do we need to worry about the "noats" command-line option here? It feels like we should be checking with the PCI subsystem before telling the SMMU we're good to go. I'll need Lorenzo's ack on this. Will _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-15 13:21 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-15 13:21 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: iommu, linux-arm-kernel, linux-acpi, robin.murphy, joro, hanjun.guo, lorenzo.pieralisi, sudeep.holla, rjw, lenb, okaya, zhongmiao, eric.auger On Tue, Apr 09, 2019 at 05:52:39PM +0100, Jean-Philippe Brucker wrote: > Root complex node in IORT has a bit telling whether it supports ATS or > not. Store this bit in the IOMMU fwspec when setting up a device, so it > can be accessed later by an IOMMU driver. > > Use the negative version (NO_ATS) at the moment because it's not clear > if/how the bit needs to be integrated in other firmware descriptions. The > SMMU has a feature bit telling if it supports ATS, which might be > sufficient in most systems for deciding whether or not we should enable > the ATS capability in endpoints. Hmm, the SMMUv3 architecture manual is pretty explicit about this: | It [SMMU_IDR0.ATS] does not guarantee that client devices and intermediate | components also support ATS and this must be determined separately. so we may need to extend the PCI bindings to describe this. I think the negative logic is likely to get in the way if that's the case. > Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> > --- > drivers/acpi/arm64/iort.c | 11 +++++++++++ > include/linux/iommu.h | 4 ++++ > 2 files changed, 15 insertions(+) > > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c > index e48894e002ba..7f2c1c9c6b38 100644 > --- a/drivers/acpi/arm64/iort.c > +++ b/drivers/acpi/arm64/iort.c > @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) > dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); > } > > +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) > +{ > + struct acpi_iort_root_complex *pci_rc; > + > + pci_rc = (struct acpi_iort_root_complex *)node->node_data; > + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; > +} Do we need to worry about the "noats" command-line option here? It feels like we should be checking with the PCI subsystem before telling the SMMU we're good to go. I'll need Lorenzo's ack on this. Will ^ permalink raw reply [flat|nested] 64+ messages in thread
[parent not found: <20190415132108.GB15023-UDVVEH7NWB15pKCnmE3YQBJ8xKzm50AiAL8bYrjMMd8@public.gmane.org>]
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-15 18:00 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-15 18:00 UTC (permalink / raw) To: Will Deacon Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, sudeep.holla-5wv7dgnIgG8, robin.murphy-5wv7dgnIgG8, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A On 15/04/2019 14:21, Will Deacon wrote: > On Tue, Apr 09, 2019 at 05:52:39PM +0100, Jean-Philippe Brucker wrote: >> Root complex node in IORT has a bit telling whether it supports ATS or >> not. Store this bit in the IOMMU fwspec when setting up a device, so it >> can be accessed later by an IOMMU driver. >> >> Use the negative version (NO_ATS) at the moment because it's not clear >> if/how the bit needs to be integrated in other firmware descriptions. The >> SMMU has a feature bit telling if it supports ATS, which might be >> sufficient in most systems for deciding whether or not we should enable >> the ATS capability in endpoints. > > Hmm, the SMMUv3 architecture manual is pretty explicit about this: > > | It [SMMU_IDR0.ATS] does not guarantee that client devices and intermediate > | components also support ATS and this must be determined separately. > > so we may need to extend the PCI bindings to describe this. I think the > negative logic is likely to get in the way if that's the case. Right. For devicetree I can resurrect the patch I proposed a while ago [1]. Rob wasn't keen on adding an "ats-supported" property to PCI host bridge nodes. Instead the host controller drivers should deduce whether ATS is supported from the compatible string. But I'll resend that patch adding the property only to pci-host-ecam-generic. [1] https://lists.linuxfoundation.org/pipermail/iommu/2017-May/022043.html >> Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> >> --- >> drivers/acpi/arm64/iort.c | 11 +++++++++++ >> include/linux/iommu.h | 4 ++++ >> 2 files changed, 15 insertions(+) >> >> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c >> index e48894e002ba..7f2c1c9c6b38 100644 >> --- a/drivers/acpi/arm64/iort.c >> +++ b/drivers/acpi/arm64/iort.c >> @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) >> dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); >> } >> >> +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) >> +{ >> + struct acpi_iort_root_complex *pci_rc; >> + >> + pci_rc = (struct acpi_iort_root_complex *)node->node_data; >> + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; >> +} > > Do we need to worry about the "noats" command-line option here? It feels > like we should be checking with the PCI subsystem before telling the SMMU > we're good to go. I'm checking the noats option in arm_smmu_enable_ats() at the moment, using the pci_ats_disabled() helper. Thanks, Jean ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-15 18:00 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-15 18:00 UTC (permalink / raw) To: Will Deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb On 15/04/2019 14:21, Will Deacon wrote: > On Tue, Apr 09, 2019 at 05:52:39PM +0100, Jean-Philippe Brucker wrote: >> Root complex node in IORT has a bit telling whether it supports ATS or >> not. Store this bit in the IOMMU fwspec when setting up a device, so it >> can be accessed later by an IOMMU driver. >> >> Use the negative version (NO_ATS) at the moment because it's not clear >> if/how the bit needs to be integrated in other firmware descriptions. The >> SMMU has a feature bit telling if it supports ATS, which might be >> sufficient in most systems for deciding whether or not we should enable >> the ATS capability in endpoints. > > Hmm, the SMMUv3 architecture manual is pretty explicit about this: > > | It [SMMU_IDR0.ATS] does not guarantee that client devices and intermediate > | components also support ATS and this must be determined separately. > > so we may need to extend the PCI bindings to describe this. I think the > negative logic is likely to get in the way if that's the case. Right. For devicetree I can resurrect the patch I proposed a while ago [1]. Rob wasn't keen on adding an "ats-supported" property to PCI host bridge nodes. Instead the host controller drivers should deduce whether ATS is supported from the compatible string. But I'll resend that patch adding the property only to pci-host-ecam-generic. [1] https://lists.linuxfoundation.org/pipermail/iommu/2017-May/022043.html >> Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> >> --- >> drivers/acpi/arm64/iort.c | 11 +++++++++++ >> include/linux/iommu.h | 4 ++++ >> 2 files changed, 15 insertions(+) >> >> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c >> index e48894e002ba..7f2c1c9c6b38 100644 >> --- a/drivers/acpi/arm64/iort.c >> +++ b/drivers/acpi/arm64/iort.c >> @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) >> dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); >> } >> >> +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) >> +{ >> + struct acpi_iort_root_complex *pci_rc; >> + >> + pci_rc = (struct acpi_iort_root_complex *)node->node_data; >> + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; >> +} > > Do we need to worry about the "noats" command-line option here? It feels > like we should be checking with the PCI subsystem before telling the SMMU > we're good to go. I'm checking the noats option in arm_smmu_enable_ats() at the moment, using the pci_ats_disabled() helper. Thanks, Jean _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-15 18:00 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-15 18:00 UTC (permalink / raw) To: Will Deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb On 15/04/2019 14:21, Will Deacon wrote: > On Tue, Apr 09, 2019 at 05:52:39PM +0100, Jean-Philippe Brucker wrote: >> Root complex node in IORT has a bit telling whether it supports ATS or >> not. Store this bit in the IOMMU fwspec when setting up a device, so it >> can be accessed later by an IOMMU driver. >> >> Use the negative version (NO_ATS) at the moment because it's not clear >> if/how the bit needs to be integrated in other firmware descriptions. The >> SMMU has a feature bit telling if it supports ATS, which might be >> sufficient in most systems for deciding whether or not we should enable >> the ATS capability in endpoints. > > Hmm, the SMMUv3 architecture manual is pretty explicit about this: > > | It [SMMU_IDR0.ATS] does not guarantee that client devices and intermediate > | components also support ATS and this must be determined separately. > > so we may need to extend the PCI bindings to describe this. I think the > negative logic is likely to get in the way if that's the case. Right. For devicetree I can resurrect the patch I proposed a while ago [1]. Rob wasn't keen on adding an "ats-supported" property to PCI host bridge nodes. Instead the host controller drivers should deduce whether ATS is supported from the compatible string. But I'll resend that patch adding the property only to pci-host-ecam-generic. [1] https://lists.linuxfoundation.org/pipermail/iommu/2017-May/022043.html >> Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> >> --- >> drivers/acpi/arm64/iort.c | 11 +++++++++++ >> include/linux/iommu.h | 4 ++++ >> 2 files changed, 15 insertions(+) >> >> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c >> index e48894e002ba..7f2c1c9c6b38 100644 >> --- a/drivers/acpi/arm64/iort.c >> +++ b/drivers/acpi/arm64/iort.c >> @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) >> dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); >> } >> >> +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) >> +{ >> + struct acpi_iort_root_complex *pci_rc; >> + >> + pci_rc = (struct acpi_iort_root_complex *)node->node_data; >> + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; >> +} > > Do we need to worry about the "noats" command-line option here? It feels > like we should be checking with the PCI subsystem before telling the SMMU > we're good to go. I'm checking the noats option in arm_smmu_enable_ats() at the moment, using the pci_ats_disabled() helper. Thanks, Jean _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-15 18:00 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-15 18:00 UTC (permalink / raw) To: Will Deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb On 15/04/2019 14:21, Will Deacon wrote: > On Tue, Apr 09, 2019 at 05:52:39PM +0100, Jean-Philippe Brucker wrote: >> Root complex node in IORT has a bit telling whether it supports ATS or >> not. Store this bit in the IOMMU fwspec when setting up a device, so it >> can be accessed later by an IOMMU driver. >> >> Use the negative version (NO_ATS) at the moment because it's not clear >> if/how the bit needs to be integrated in other firmware descriptions. The >> SMMU has a feature bit telling if it supports ATS, which might be >> sufficient in most systems for deciding whether or not we should enable >> the ATS capability in endpoints. > > Hmm, the SMMUv3 architecture manual is pretty explicit about this: > > | It [SMMU_IDR0.ATS] does not guarantee that client devices and intermediate > | components also support ATS and this must be determined separately. > > so we may need to extend the PCI bindings to describe this. I think the > negative logic is likely to get in the way if that's the case. Right. For devicetree I can resurrect the patch I proposed a while ago [1]. Rob wasn't keen on adding an "ats-supported" property to PCI host bridge nodes. Instead the host controller drivers should deduce whether ATS is supported from the compatible string. But I'll resend that patch adding the property only to pci-host-ecam-generic. [1] https://lists.linuxfoundation.org/pipermail/iommu/2017-May/022043.html >> Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> >> --- >> drivers/acpi/arm64/iort.c | 11 +++++++++++ >> include/linux/iommu.h | 4 ++++ >> 2 files changed, 15 insertions(+) >> >> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c >> index e48894e002ba..7f2c1c9c6b38 100644 >> --- a/drivers/acpi/arm64/iort.c >> +++ b/drivers/acpi/arm64/iort.c >> @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) >> dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); >> } >> >> +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) >> +{ >> + struct acpi_iort_root_complex *pci_rc; >> + >> + pci_rc = (struct acpi_iort_root_complex *)node->node_data; >> + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; >> +} > > Do we need to worry about the "noats" command-line option here? It feels > like we should be checking with the PCI subsystem before telling the SMMU > we're good to go. I'm checking the noats option in arm_smmu_enable_ats() at the moment, using the pci_ats_disabled() helper. Thanks, Jean ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-15 18:31 ` Robin Murphy 0 siblings, 0 replies; 64+ messages in thread From: Robin Murphy @ 2019-04-15 18:31 UTC (permalink / raw) To: Jean-Philippe Brucker, will.deacon Cc: lorenzo.pieralisi, eric.auger, zhongmiao, okaya, joro, rjw, linux-acpi, iommu, hanjun.guo, sudeep.holla, linux-arm-kernel, lenb On 09/04/2019 17:52, Jean-Philippe Brucker wrote: > Root complex node in IORT has a bit telling whether it supports ATS or > not. Store this bit in the IOMMU fwspec when setting up a device, so it > can be accessed later by an IOMMU driver. Hmm, it doesn't feel quite right to store a copy of the same RC/SMMU integration property for every endpoint... It seems like it might be more logical to track this at the SMMU end, i.e. only allow ARM_SMMU_FEAT_ATS to be set if all RCs targeting that SMMU also support ATS. For the moment that seems sufficiently realistic, and unless some wacky topology ever actually shows up in silicon to complain, I'm inclined not to care too much about it being potentially overly restrictive. Furthermore, I'm now wondering whether it would make sense to push this into the PCI layer as well (or instead), i.e. hook into pci_init_ats() or pci_enable_ats() and it the device is untrusted or the topology doesn't support ATS, prevent the capability from ever being enabled at all, rather than trying to mitigate it later at the SMMU end. What do you reckon? > Use the negative version (NO_ATS) at the moment because it's not clear > if/how the bit needs to be integrated in other firmware descriptions. The > SMMU has a feature bit telling if it supports ATS, which might be > sufficient in most systems for deciding whether or not we should enable > the ATS capability in endpoints. I can fairly confidently guarantee that it won't. For instance, MMU-600 reports IDR0.ATS==1 because MMU-600 implements the SMMUv3 architectural ATS support. Actually making use of that support, though, still requires an RC capable of generating the appropriate DTI-ATS messages, and that a DTI interface is wired up correctly between the two. > Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> > --- > drivers/acpi/arm64/iort.c | 11 +++++++++++ > include/linux/iommu.h | 4 ++++ > 2 files changed, 15 insertions(+) > > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c > index e48894e002ba..7f2c1c9c6b38 100644 > --- a/drivers/acpi/arm64/iort.c > +++ b/drivers/acpi/arm64/iort.c > @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) > dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); > } > > +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) > +{ > + struct acpi_iort_root_complex *pci_rc; > + > + pci_rc = (struct acpi_iort_root_complex *)node->node_data; > + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; > +} > + > /** > * iort_iommu_configure - Set-up IOMMU configuration for a device. > * > @@ -1063,6 +1071,9 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev) > info.node = node; > err = pci_for_each_dma_alias(to_pci_dev(dev), > iort_pci_iommu_init, &info); > + > + if (!err && !iort_pci_rc_supports_ats(node)) > + dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_NO_ATS; > } else { > int i = 0; > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > index 3dbeb457fb16..ed6738c358ca 100644 > --- a/include/linux/iommu.h > +++ b/include/linux/iommu.h > @@ -509,10 +509,14 @@ struct iommu_fwspec { > const struct iommu_ops *ops; > struct fwnode_handle *iommu_fwnode; > void *iommu_priv; > + u32 flags; > unsigned int num_ids; > u32 ids[1]; > }; > > +/* Firmware disabled ATS in the root complex */ More likely firmware is just describing how the hardware was built ;) Robin. > +#define IOMMU_FWSPEC_PCI_NO_ATS (1 << 0) > + > int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwnode, > const struct iommu_ops *ops); > void iommu_fwspec_free(struct device *dev); > ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-15 18:31 ` Robin Murphy 0 siblings, 0 replies; 64+ messages in thread From: Robin Murphy @ 2019-04-15 18:31 UTC (permalink / raw) To: Jean-Philippe Brucker, will.deacon Cc: lorenzo.pieralisi, eric.auger, zhongmiao, okaya, joro, rjw, linux-acpi, iommu, hanjun.guo, sudeep.holla, linux-arm-kernel, lenb On 09/04/2019 17:52, Jean-Philippe Brucker wrote: > Root complex node in IORT has a bit telling whether it supports ATS or > not. Store this bit in the IOMMU fwspec when setting up a device, so it > can be accessed later by an IOMMU driver. Hmm, it doesn't feel quite right to store a copy of the same RC/SMMU integration property for every endpoint... It seems like it might be more logical to track this at the SMMU end, i.e. only allow ARM_SMMU_FEAT_ATS to be set if all RCs targeting that SMMU also support ATS. For the moment that seems sufficiently realistic, and unless some wacky topology ever actually shows up in silicon to complain, I'm inclined not to care too much about it being potentially overly restrictive. Furthermore, I'm now wondering whether it would make sense to push this into the PCI layer as well (or instead), i.e. hook into pci_init_ats() or pci_enable_ats() and it the device is untrusted or the topology doesn't support ATS, prevent the capability from ever being enabled at all, rather than trying to mitigate it later at the SMMU end. What do you reckon? > Use the negative version (NO_ATS) at the moment because it's not clear > if/how the bit needs to be integrated in other firmware descriptions. The > SMMU has a feature bit telling if it supports ATS, which might be > sufficient in most systems for deciding whether or not we should enable > the ATS capability in endpoints. I can fairly confidently guarantee that it won't. For instance, MMU-600 reports IDR0.ATS==1 because MMU-600 implements the SMMUv3 architectural ATS support. Actually making use of that support, though, still requires an RC capable of generating the appropriate DTI-ATS messages, and that a DTI interface is wired up correctly between the two. > Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> > --- > drivers/acpi/arm64/iort.c | 11 +++++++++++ > include/linux/iommu.h | 4 ++++ > 2 files changed, 15 insertions(+) > > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c > index e48894e002ba..7f2c1c9c6b38 100644 > --- a/drivers/acpi/arm64/iort.c > +++ b/drivers/acpi/arm64/iort.c > @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) > dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); > } > > +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) > +{ > + struct acpi_iort_root_complex *pci_rc; > + > + pci_rc = (struct acpi_iort_root_complex *)node->node_data; > + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; > +} > + > /** > * iort_iommu_configure - Set-up IOMMU configuration for a device. > * > @@ -1063,6 +1071,9 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev) > info.node = node; > err = pci_for_each_dma_alias(to_pci_dev(dev), > iort_pci_iommu_init, &info); > + > + if (!err && !iort_pci_rc_supports_ats(node)) > + dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_NO_ATS; > } else { > int i = 0; > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > index 3dbeb457fb16..ed6738c358ca 100644 > --- a/include/linux/iommu.h > +++ b/include/linux/iommu.h > @@ -509,10 +509,14 @@ struct iommu_fwspec { > const struct iommu_ops *ops; > struct fwnode_handle *iommu_fwnode; > void *iommu_priv; > + u32 flags; > unsigned int num_ids; > u32 ids[1]; > }; > > +/* Firmware disabled ATS in the root complex */ More likely firmware is just describing how the hardware was built ;) Robin. > +#define IOMMU_FWSPEC_PCI_NO_ATS (1 << 0) > + > int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwnode, > const struct iommu_ops *ops); > void iommu_fwspec_free(struct device *dev); > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-15 18:31 ` Robin Murphy 0 siblings, 0 replies; 64+ messages in thread From: Robin Murphy @ 2019-04-15 18:31 UTC (permalink / raw) To: Jean-Philippe Brucker, will.deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, linux-arm-kernel, lenb On 09/04/2019 17:52, Jean-Philippe Brucker wrote: > Root complex node in IORT has a bit telling whether it supports ATS or > not. Store this bit in the IOMMU fwspec when setting up a device, so it > can be accessed later by an IOMMU driver. Hmm, it doesn't feel quite right to store a copy of the same RC/SMMU integration property for every endpoint... It seems like it might be more logical to track this at the SMMU end, i.e. only allow ARM_SMMU_FEAT_ATS to be set if all RCs targeting that SMMU also support ATS. For the moment that seems sufficiently realistic, and unless some wacky topology ever actually shows up in silicon to complain, I'm inclined not to care too much about it being potentially overly restrictive. Furthermore, I'm now wondering whether it would make sense to push this into the PCI layer as well (or instead), i.e. hook into pci_init_ats() or pci_enable_ats() and it the device is untrusted or the topology doesn't support ATS, prevent the capability from ever being enabled at all, rather than trying to mitigate it later at the SMMU end. What do you reckon? > Use the negative version (NO_ATS) at the moment because it's not clear > if/how the bit needs to be integrated in other firmware descriptions. The > SMMU has a feature bit telling if it supports ATS, which might be > sufficient in most systems for deciding whether or not we should enable > the ATS capability in endpoints. I can fairly confidently guarantee that it won't. For instance, MMU-600 reports IDR0.ATS==1 because MMU-600 implements the SMMUv3 architectural ATS support. Actually making use of that support, though, still requires an RC capable of generating the appropriate DTI-ATS messages, and that a DTI interface is wired up correctly between the two. > Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> > --- > drivers/acpi/arm64/iort.c | 11 +++++++++++ > include/linux/iommu.h | 4 ++++ > 2 files changed, 15 insertions(+) > > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c > index e48894e002ba..7f2c1c9c6b38 100644 > --- a/drivers/acpi/arm64/iort.c > +++ b/drivers/acpi/arm64/iort.c > @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) > dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); > } > > +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) > +{ > + struct acpi_iort_root_complex *pci_rc; > + > + pci_rc = (struct acpi_iort_root_complex *)node->node_data; > + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; > +} > + > /** > * iort_iommu_configure - Set-up IOMMU configuration for a device. > * > @@ -1063,6 +1071,9 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev) > info.node = node; > err = pci_for_each_dma_alias(to_pci_dev(dev), > iort_pci_iommu_init, &info); > + > + if (!err && !iort_pci_rc_supports_ats(node)) > + dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_NO_ATS; > } else { > int i = 0; > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > index 3dbeb457fb16..ed6738c358ca 100644 > --- a/include/linux/iommu.h > +++ b/include/linux/iommu.h > @@ -509,10 +509,14 @@ struct iommu_fwspec { > const struct iommu_ops *ops; > struct fwnode_handle *iommu_fwnode; > void *iommu_priv; > + u32 flags; > unsigned int num_ids; > u32 ids[1]; > }; > > +/* Firmware disabled ATS in the root complex */ More likely firmware is just describing how the hardware was built ;) Robin. > +#define IOMMU_FWSPEC_PCI_NO_ATS (1 << 0) > + > int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwnode, > const struct iommu_ops *ops); > void iommu_fwspec_free(struct device *dev); > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-15 18:31 ` Robin Murphy 0 siblings, 0 replies; 64+ messages in thread From: Robin Murphy @ 2019-04-15 18:31 UTC (permalink / raw) To: Jean-Philippe Brucker, will.deacon Cc: iommu, linux-arm-kernel, linux-acpi, joro, hanjun.guo, lorenzo.pieralisi, sudeep.holla, rjw, lenb, okaya, zhongmiao, eric.auger On 09/04/2019 17:52, Jean-Philippe Brucker wrote: > Root complex node in IORT has a bit telling whether it supports ATS or > not. Store this bit in the IOMMU fwspec when setting up a device, so it > can be accessed later by an IOMMU driver. Hmm, it doesn't feel quite right to store a copy of the same RC/SMMU integration property for every endpoint... It seems like it might be more logical to track this at the SMMU end, i.e. only allow ARM_SMMU_FEAT_ATS to be set if all RCs targeting that SMMU also support ATS. For the moment that seems sufficiently realistic, and unless some wacky topology ever actually shows up in silicon to complain, I'm inclined not to care too much about it being potentially overly restrictive. Furthermore, I'm now wondering whether it would make sense to push this into the PCI layer as well (or instead), i.e. hook into pci_init_ats() or pci_enable_ats() and it the device is untrusted or the topology doesn't support ATS, prevent the capability from ever being enabled at all, rather than trying to mitigate it later at the SMMU end. What do you reckon? > Use the negative version (NO_ATS) at the moment because it's not clear > if/how the bit needs to be integrated in other firmware descriptions. The > SMMU has a feature bit telling if it supports ATS, which might be > sufficient in most systems for deciding whether or not we should enable > the ATS capability in endpoints. I can fairly confidently guarantee that it won't. For instance, MMU-600 reports IDR0.ATS==1 because MMU-600 implements the SMMUv3 architectural ATS support. Actually making use of that support, though, still requires an RC capable of generating the appropriate DTI-ATS messages, and that a DTI interface is wired up correctly between the two. > Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> > --- > drivers/acpi/arm64/iort.c | 11 +++++++++++ > include/linux/iommu.h | 4 ++++ > 2 files changed, 15 insertions(+) > > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c > index e48894e002ba..7f2c1c9c6b38 100644 > --- a/drivers/acpi/arm64/iort.c > +++ b/drivers/acpi/arm64/iort.c > @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) > dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); > } > > +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) > +{ > + struct acpi_iort_root_complex *pci_rc; > + > + pci_rc = (struct acpi_iort_root_complex *)node->node_data; > + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; > +} > + > /** > * iort_iommu_configure - Set-up IOMMU configuration for a device. > * > @@ -1063,6 +1071,9 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev) > info.node = node; > err = pci_for_each_dma_alias(to_pci_dev(dev), > iort_pci_iommu_init, &info); > + > + if (!err && !iort_pci_rc_supports_ats(node)) > + dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_NO_ATS; > } else { > int i = 0; > > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > index 3dbeb457fb16..ed6738c358ca 100644 > --- a/include/linux/iommu.h > +++ b/include/linux/iommu.h > @@ -509,10 +509,14 @@ struct iommu_fwspec { > const struct iommu_ops *ops; > struct fwnode_handle *iommu_fwnode; > void *iommu_priv; > + u32 flags; > unsigned int num_ids; > u32 ids[1]; > }; > > +/* Firmware disabled ATS in the root complex */ More likely firmware is just describing how the hardware was built ;) Robin. > +#define IOMMU_FWSPEC_PCI_NO_ATS (1 << 0) > + > int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwnode, > const struct iommu_ops *ops); > void iommu_fwspec_free(struct device *dev); > ^ permalink raw reply [flat|nested] 64+ messages in thread
[parent not found: <c10c7adb-c7f6-f8c6-05cc-f4f143427a2d-5wv7dgnIgG8@public.gmane.org>]
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-16 16:27 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-16 16:27 UTC (permalink / raw) To: Robin Murphy, Will Deacon Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, Sudeep Holla, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A On 15/04/2019 19:31, Robin Murphy wrote: > On 09/04/2019 17:52, Jean-Philippe Brucker wrote: >> Root complex node in IORT has a bit telling whether it supports ATS or >> not. Store this bit in the IOMMU fwspec when setting up a device, so it >> can be accessed later by an IOMMU driver. > > Hmm, it doesn't feel quite right to store a copy of the same RC/SMMU > integration property for every endpoint... > > It seems like it might be more logical to track this at the SMMU end, > i.e. only allow ARM_SMMU_FEAT_ATS to be set if all RCs targeting that > SMMU also support ATS. For the moment that seems sufficiently realistic, > and unless some wacky topology ever actually shows up in silicon to > complain, I'm inclined not to care too much about it being potentially > overly restrictive. Doesn't that require us to implement a reverse lookup of smmu-node -> RC-node using the iommu maps, in both DT and IORT? Seems like a lot of work for little gain. Putting the bit in fwspec isn't ideal, but I don't find aggregating properties of RCs into ARM_SMMU_FEAT_ATS much nicer. > Furthermore, I'm now wondering whether it would make sense to push this > into the PCI layer as well (or instead), i.e. hook into pci_init_ats() > or pci_enable_ats() and it the device is untrusted or the topology > doesn't support ATS, prevent the capability from ever being enabled at > all, rather than trying to mitigate it later at the SMMU end. What do > you reckon? For the RC property it's a bit difficult because everyone does it differently. All of DMAR, IVRS, IORT and DT-based implementations would have to call into the PCI core to declare whether a device is allowed to do ATS or not. Which is essentially what the IOMMU drivers do now by calling pci_enable_ats() themselves. Having pci_enable_ats() check the untrusted bit seems like a good cleanup. I'm not sure yet but given that it has side effects (AMD IOMMU doesn't check that bit at the moment) it would probably fit better in a separate series. >> Use the negative version (NO_ATS) at the moment because it's not clear >> if/how the bit needs to be integrated in other firmware descriptions. The >> SMMU has a feature bit telling if it supports ATS, which might be >> sufficient in most systems for deciding whether or not we should enable >> the ATS capability in endpoints. > > I can fairly confidently guarantee that it won't. For instance, MMU-600 > reports IDR0.ATS==1 because MMU-600 implements the SMMUv3 architectural > ATS support. Actually making use of that support, though, still requires > an RC capable of generating the appropriate DTI-ATS messages, and that a > DTI interface is wired up correctly between the two. Right, we need to explicitly validate that the RC understands ATS. If we enable ATS but the root complex doesn't support it, in addition to being unusable it might be possible for the EP to corrupt memory, because to an RC that ignores the AT field, a Translation Request looks like a Memory Read Request. The endpoint could end up storing the content of memory in its ATC instead of a translated address, and later use that as address of a write. >> Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> >> --- >> drivers/acpi/arm64/iort.c | 11 +++++++++++ >> include/linux/iommu.h | 4 ++++ >> 2 files changed, 15 insertions(+) >> >> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c >> index e48894e002ba..7f2c1c9c6b38 100644 >> --- a/drivers/acpi/arm64/iort.c >> +++ b/drivers/acpi/arm64/iort.c >> @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) >> dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); >> } >> >> +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) >> +{ >> + struct acpi_iort_root_complex *pci_rc; >> + >> + pci_rc = (struct acpi_iort_root_complex *)node->node_data; >> + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; >> +} >> + >> /** >> * iort_iommu_configure - Set-up IOMMU configuration for a device. >> * >> @@ -1063,6 +1071,9 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev) >> info.node = node; >> err = pci_for_each_dma_alias(to_pci_dev(dev), >> iort_pci_iommu_init, &info); >> + >> + if (!err && !iort_pci_rc_supports_ats(node)) >> + dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_NO_ATS; >> } else { >> int i = 0; >> >> diff --git a/include/linux/iommu.h b/include/linux/iommu.h >> index 3dbeb457fb16..ed6738c358ca 100644 >> --- a/include/linux/iommu.h >> +++ b/include/linux/iommu.h >> @@ -509,10 +509,14 @@ struct iommu_fwspec { >> const struct iommu_ops *ops; >> struct fwnode_handle *iommu_fwnode; >> void *iommu_priv; >> + u32 flags; >> unsigned int num_ids; >> u32 ids[1]; >> }; >> >> +/* Firmware disabled ATS in the root complex */ > > More likely firmware is just describing how the hardware was built ;) Right, I'll fix this Thanks, Jean ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-16 16:27 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-16 16:27 UTC (permalink / raw) To: Robin Murphy, Will Deacon Cc: Lorenzo Pieralisi, eric.auger, zhongmiao, okaya, joro, rjw, linux-acpi, iommu, hanjun.guo, Sudeep Holla, linux-arm-kernel, lenb On 15/04/2019 19:31, Robin Murphy wrote: > On 09/04/2019 17:52, Jean-Philippe Brucker wrote: >> Root complex node in IORT has a bit telling whether it supports ATS or >> not. Store this bit in the IOMMU fwspec when setting up a device, so it >> can be accessed later by an IOMMU driver. > > Hmm, it doesn't feel quite right to store a copy of the same RC/SMMU > integration property for every endpoint... > > It seems like it might be more logical to track this at the SMMU end, > i.e. only allow ARM_SMMU_FEAT_ATS to be set if all RCs targeting that > SMMU also support ATS. For the moment that seems sufficiently realistic, > and unless some wacky topology ever actually shows up in silicon to > complain, I'm inclined not to care too much about it being potentially > overly restrictive. Doesn't that require us to implement a reverse lookup of smmu-node -> RC-node using the iommu maps, in both DT and IORT? Seems like a lot of work for little gain. Putting the bit in fwspec isn't ideal, but I don't find aggregating properties of RCs into ARM_SMMU_FEAT_ATS much nicer. > Furthermore, I'm now wondering whether it would make sense to push this > into the PCI layer as well (or instead), i.e. hook into pci_init_ats() > or pci_enable_ats() and it the device is untrusted or the topology > doesn't support ATS, prevent the capability from ever being enabled at > all, rather than trying to mitigate it later at the SMMU end. What do > you reckon? For the RC property it's a bit difficult because everyone does it differently. All of DMAR, IVRS, IORT and DT-based implementations would have to call into the PCI core to declare whether a device is allowed to do ATS or not. Which is essentially what the IOMMU drivers do now by calling pci_enable_ats() themselves. Having pci_enable_ats() check the untrusted bit seems like a good cleanup. I'm not sure yet but given that it has side effects (AMD IOMMU doesn't check that bit at the moment) it would probably fit better in a separate series. >> Use the negative version (NO_ATS) at the moment because it's not clear >> if/how the bit needs to be integrated in other firmware descriptions. The >> SMMU has a feature bit telling if it supports ATS, which might be >> sufficient in most systems for deciding whether or not we should enable >> the ATS capability in endpoints. > > I can fairly confidently guarantee that it won't. For instance, MMU-600 > reports IDR0.ATS==1 because MMU-600 implements the SMMUv3 architectural > ATS support. Actually making use of that support, though, still requires > an RC capable of generating the appropriate DTI-ATS messages, and that a > DTI interface is wired up correctly between the two. Right, we need to explicitly validate that the RC understands ATS. If we enable ATS but the root complex doesn't support it, in addition to being unusable it might be possible for the EP to corrupt memory, because to an RC that ignores the AT field, a Translation Request looks like a Memory Read Request. The endpoint could end up storing the content of memory in its ATC instead of a translated address, and later use that as address of a write. >> Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> >> --- >> drivers/acpi/arm64/iort.c | 11 +++++++++++ >> include/linux/iommu.h | 4 ++++ >> 2 files changed, 15 insertions(+) >> >> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c >> index e48894e002ba..7f2c1c9c6b38 100644 >> --- a/drivers/acpi/arm64/iort.c >> +++ b/drivers/acpi/arm64/iort.c >> @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) >> dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); >> } >> >> +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) >> +{ >> + struct acpi_iort_root_complex *pci_rc; >> + >> + pci_rc = (struct acpi_iort_root_complex *)node->node_data; >> + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; >> +} >> + >> /** >> * iort_iommu_configure - Set-up IOMMU configuration for a device. >> * >> @@ -1063,6 +1071,9 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev) >> info.node = node; >> err = pci_for_each_dma_alias(to_pci_dev(dev), >> iort_pci_iommu_init, &info); >> + >> + if (!err && !iort_pci_rc_supports_ats(node)) >> + dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_NO_ATS; >> } else { >> int i = 0; >> >> diff --git a/include/linux/iommu.h b/include/linux/iommu.h >> index 3dbeb457fb16..ed6738c358ca 100644 >> --- a/include/linux/iommu.h >> +++ b/include/linux/iommu.h >> @@ -509,10 +509,14 @@ struct iommu_fwspec { >> const struct iommu_ops *ops; >> struct fwnode_handle *iommu_fwnode; >> void *iommu_priv; >> + u32 flags; >> unsigned int num_ids; >> u32 ids[1]; >> }; >> >> +/* Firmware disabled ATS in the root complex */ > > More likely firmware is just describing how the hardware was built ;) Right, I'll fix this Thanks, Jean _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-16 16:27 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-16 16:27 UTC (permalink / raw) To: Robin Murphy, Will Deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, Sudeep Holla, linux-arm-kernel, lenb On 15/04/2019 19:31, Robin Murphy wrote: > On 09/04/2019 17:52, Jean-Philippe Brucker wrote: >> Root complex node in IORT has a bit telling whether it supports ATS or >> not. Store this bit in the IOMMU fwspec when setting up a device, so it >> can be accessed later by an IOMMU driver. > > Hmm, it doesn't feel quite right to store a copy of the same RC/SMMU > integration property for every endpoint... > > It seems like it might be more logical to track this at the SMMU end, > i.e. only allow ARM_SMMU_FEAT_ATS to be set if all RCs targeting that > SMMU also support ATS. For the moment that seems sufficiently realistic, > and unless some wacky topology ever actually shows up in silicon to > complain, I'm inclined not to care too much about it being potentially > overly restrictive. Doesn't that require us to implement a reverse lookup of smmu-node -> RC-node using the iommu maps, in both DT and IORT? Seems like a lot of work for little gain. Putting the bit in fwspec isn't ideal, but I don't find aggregating properties of RCs into ARM_SMMU_FEAT_ATS much nicer. > Furthermore, I'm now wondering whether it would make sense to push this > into the PCI layer as well (or instead), i.e. hook into pci_init_ats() > or pci_enable_ats() and it the device is untrusted or the topology > doesn't support ATS, prevent the capability from ever being enabled at > all, rather than trying to mitigate it later at the SMMU end. What do > you reckon? For the RC property it's a bit difficult because everyone does it differently. All of DMAR, IVRS, IORT and DT-based implementations would have to call into the PCI core to declare whether a device is allowed to do ATS or not. Which is essentially what the IOMMU drivers do now by calling pci_enable_ats() themselves. Having pci_enable_ats() check the untrusted bit seems like a good cleanup. I'm not sure yet but given that it has side effects (AMD IOMMU doesn't check that bit at the moment) it would probably fit better in a separate series. >> Use the negative version (NO_ATS) at the moment because it's not clear >> if/how the bit needs to be integrated in other firmware descriptions. The >> SMMU has a feature bit telling if it supports ATS, which might be >> sufficient in most systems for deciding whether or not we should enable >> the ATS capability in endpoints. > > I can fairly confidently guarantee that it won't. For instance, MMU-600 > reports IDR0.ATS==1 because MMU-600 implements the SMMUv3 architectural > ATS support. Actually making use of that support, though, still requires > an RC capable of generating the appropriate DTI-ATS messages, and that a > DTI interface is wired up correctly between the two. Right, we need to explicitly validate that the RC understands ATS. If we enable ATS but the root complex doesn't support it, in addition to being unusable it might be possible for the EP to corrupt memory, because to an RC that ignores the AT field, a Translation Request looks like a Memory Read Request. The endpoint could end up storing the content of memory in its ATC instead of a translated address, and later use that as address of a write. >> Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> >> --- >> drivers/acpi/arm64/iort.c | 11 +++++++++++ >> include/linux/iommu.h | 4 ++++ >> 2 files changed, 15 insertions(+) >> >> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c >> index e48894e002ba..7f2c1c9c6b38 100644 >> --- a/drivers/acpi/arm64/iort.c >> +++ b/drivers/acpi/arm64/iort.c >> @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) >> dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); >> } >> >> +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) >> +{ >> + struct acpi_iort_root_complex *pci_rc; >> + >> + pci_rc = (struct acpi_iort_root_complex *)node->node_data; >> + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; >> +} >> + >> /** >> * iort_iommu_configure - Set-up IOMMU configuration for a device. >> * >> @@ -1063,6 +1071,9 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev) >> info.node = node; >> err = pci_for_each_dma_alias(to_pci_dev(dev), >> iort_pci_iommu_init, &info); >> + >> + if (!err && !iort_pci_rc_supports_ats(node)) >> + dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_NO_ATS; >> } else { >> int i = 0; >> >> diff --git a/include/linux/iommu.h b/include/linux/iommu.h >> index 3dbeb457fb16..ed6738c358ca 100644 >> --- a/include/linux/iommu.h >> +++ b/include/linux/iommu.h >> @@ -509,10 +509,14 @@ struct iommu_fwspec { >> const struct iommu_ops *ops; >> struct fwnode_handle *iommu_fwnode; >> void *iommu_priv; >> + u32 flags; >> unsigned int num_ids; >> u32 ids[1]; >> }; >> >> +/* Firmware disabled ATS in the root complex */ > > More likely firmware is just describing how the hardware was built ;) Right, I'll fix this Thanks, Jean _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes @ 2019-04-16 16:27 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-16 16:27 UTC (permalink / raw) To: Robin Murphy, Will Deacon Cc: iommu, linux-arm-kernel, linux-acpi, joro, hanjun.guo, Lorenzo Pieralisi, Sudeep Holla, rjw, lenb, okaya, zhongmiao, eric.auger On 15/04/2019 19:31, Robin Murphy wrote: > On 09/04/2019 17:52, Jean-Philippe Brucker wrote: >> Root complex node in IORT has a bit telling whether it supports ATS or >> not. Store this bit in the IOMMU fwspec when setting up a device, so it >> can be accessed later by an IOMMU driver. > > Hmm, it doesn't feel quite right to store a copy of the same RC/SMMU > integration property for every endpoint... > > It seems like it might be more logical to track this at the SMMU end, > i.e. only allow ARM_SMMU_FEAT_ATS to be set if all RCs targeting that > SMMU also support ATS. For the moment that seems sufficiently realistic, > and unless some wacky topology ever actually shows up in silicon to > complain, I'm inclined not to care too much about it being potentially > overly restrictive. Doesn't that require us to implement a reverse lookup of smmu-node -> RC-node using the iommu maps, in both DT and IORT? Seems like a lot of work for little gain. Putting the bit in fwspec isn't ideal, but I don't find aggregating properties of RCs into ARM_SMMU_FEAT_ATS much nicer. > Furthermore, I'm now wondering whether it would make sense to push this > into the PCI layer as well (or instead), i.e. hook into pci_init_ats() > or pci_enable_ats() and it the device is untrusted or the topology > doesn't support ATS, prevent the capability from ever being enabled at > all, rather than trying to mitigate it later at the SMMU end. What do > you reckon? For the RC property it's a bit difficult because everyone does it differently. All of DMAR, IVRS, IORT and DT-based implementations would have to call into the PCI core to declare whether a device is allowed to do ATS or not. Which is essentially what the IOMMU drivers do now by calling pci_enable_ats() themselves. Having pci_enable_ats() check the untrusted bit seems like a good cleanup. I'm not sure yet but given that it has side effects (AMD IOMMU doesn't check that bit at the moment) it would probably fit better in a separate series. >> Use the negative version (NO_ATS) at the moment because it's not clear >> if/how the bit needs to be integrated in other firmware descriptions. The >> SMMU has a feature bit telling if it supports ATS, which might be >> sufficient in most systems for deciding whether or not we should enable >> the ATS capability in endpoints. > > I can fairly confidently guarantee that it won't. For instance, MMU-600 > reports IDR0.ATS==1 because MMU-600 implements the SMMUv3 architectural > ATS support. Actually making use of that support, though, still requires > an RC capable of generating the appropriate DTI-ATS messages, and that a > DTI interface is wired up correctly between the two. Right, we need to explicitly validate that the RC understands ATS. If we enable ATS but the root complex doesn't support it, in addition to being unusable it might be possible for the EP to corrupt memory, because to an RC that ignores the AT field, a Translation Request looks like a Memory Read Request. The endpoint could end up storing the content of memory in its ATC instead of a translated address, and later use that as address of a write. >> Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> >> --- >> drivers/acpi/arm64/iort.c | 11 +++++++++++ >> include/linux/iommu.h | 4 ++++ >> 2 files changed, 15 insertions(+) >> >> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c >> index e48894e002ba..7f2c1c9c6b38 100644 >> --- a/drivers/acpi/arm64/iort.c >> +++ b/drivers/acpi/arm64/iort.c >> @@ -1028,6 +1028,14 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size) >> dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset); >> } >> >> +static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node) >> +{ >> + struct acpi_iort_root_complex *pci_rc; >> + >> + pci_rc = (struct acpi_iort_root_complex *)node->node_data; >> + return pci_rc->ats_attribute & ACPI_IORT_ATS_SUPPORTED; >> +} >> + >> /** >> * iort_iommu_configure - Set-up IOMMU configuration for a device. >> * >> @@ -1063,6 +1071,9 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev) >> info.node = node; >> err = pci_for_each_dma_alias(to_pci_dev(dev), >> iort_pci_iommu_init, &info); >> + >> + if (!err && !iort_pci_rc_supports_ats(node)) >> + dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_NO_ATS; >> } else { >> int i = 0; >> >> diff --git a/include/linux/iommu.h b/include/linux/iommu.h >> index 3dbeb457fb16..ed6738c358ca 100644 >> --- a/include/linux/iommu.h >> +++ b/include/linux/iommu.h >> @@ -509,10 +509,14 @@ struct iommu_fwspec { >> const struct iommu_ops *ops; >> struct fwnode_handle *iommu_fwnode; >> void *iommu_priv; >> + u32 flags; >> unsigned int num_ids; >> u32 ids[1]; >> }; >> >> +/* Firmware disabled ATS in the root complex */ > > More likely firmware is just describing how the hardware was built ;) Right, I'll fix this Thanks, Jean ^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v2 2/7] iommu/arm-smmu-v3: Rename arm_smmu_master_data to arm_smmu_master @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon-5wv7dgnIgG8 Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, sudeep.holla-5wv7dgnIgG8, robin.murphy-5wv7dgnIgG8, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A The arm_smmu_master_data structure already represents more than just the firmware data associated to a master, and will be used extensively to represent a device's state when implementing more SMMU features. Rename the structure to arm_smmu_master. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> --- drivers/iommu/arm-smmu-v3.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index d3880010c6cf..50cb037f3d8a 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -591,7 +591,7 @@ struct arm_smmu_device { }; /* SMMU private data for each master */ -struct arm_smmu_master_data { +struct arm_smmu_master { struct arm_smmu_device *smmu; struct arm_smmu_strtab_ent ste; }; @@ -1691,7 +1691,7 @@ static __le64 *arm_smmu_get_step_for_sid(struct arm_smmu_device *smmu, u32 sid) static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) { int i, j; - struct arm_smmu_master_data *master = fwspec->iommu_priv; + struct arm_smmu_master *master = fwspec->iommu_priv; struct arm_smmu_device *smmu = master->smmu; for (i = 0; i < fwspec->num_ids; ++i) { @@ -1712,7 +1712,7 @@ static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) static void arm_smmu_detach_dev(struct device *dev) { struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - struct arm_smmu_master_data *master = fwspec->iommu_priv; + struct arm_smmu_master *master = fwspec->iommu_priv; master->ste.assigned = false; arm_smmu_install_ste_for_dev(fwspec); @@ -1724,7 +1724,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); - struct arm_smmu_master_data *master; + struct arm_smmu_master *master; struct arm_smmu_strtab_ent *ste; if (!fwspec) @@ -1860,7 +1860,7 @@ static int arm_smmu_add_device(struct device *dev) { int i, ret; struct arm_smmu_device *smmu; - struct arm_smmu_master_data *master; + struct arm_smmu_master *master; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct iommu_group *group; @@ -1913,7 +1913,7 @@ static int arm_smmu_add_device(struct device *dev) static void arm_smmu_remove_device(struct device *dev) { struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - struct arm_smmu_master_data *master; + struct arm_smmu_master *master; struct arm_smmu_device *smmu; if (!fwspec || fwspec->ops != &arm_smmu_ops) -- 2.21.0 ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 2/7] iommu/arm-smmu-v3: Rename arm_smmu_master_data to arm_smmu_master @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: lorenzo.pieralisi, eric.auger, zhongmiao, okaya, joro, rjw, linux-acpi, iommu, hanjun.guo, sudeep.holla, robin.murphy, linux-arm-kernel, lenb The arm_smmu_master_data structure already represents more than just the firmware data associated to a master, and will be used extensively to represent a device's state when implementing more SMMU features. Rename the structure to arm_smmu_master. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index d3880010c6cf..50cb037f3d8a 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -591,7 +591,7 @@ struct arm_smmu_device { }; /* SMMU private data for each master */ -struct arm_smmu_master_data { +struct arm_smmu_master { struct arm_smmu_device *smmu; struct arm_smmu_strtab_ent ste; }; @@ -1691,7 +1691,7 @@ static __le64 *arm_smmu_get_step_for_sid(struct arm_smmu_device *smmu, u32 sid) static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) { int i, j; - struct arm_smmu_master_data *master = fwspec->iommu_priv; + struct arm_smmu_master *master = fwspec->iommu_priv; struct arm_smmu_device *smmu = master->smmu; for (i = 0; i < fwspec->num_ids; ++i) { @@ -1712,7 +1712,7 @@ static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) static void arm_smmu_detach_dev(struct device *dev) { struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - struct arm_smmu_master_data *master = fwspec->iommu_priv; + struct arm_smmu_master *master = fwspec->iommu_priv; master->ste.assigned = false; arm_smmu_install_ste_for_dev(fwspec); @@ -1724,7 +1724,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); - struct arm_smmu_master_data *master; + struct arm_smmu_master *master; struct arm_smmu_strtab_ent *ste; if (!fwspec) @@ -1860,7 +1860,7 @@ static int arm_smmu_add_device(struct device *dev) { int i, ret; struct arm_smmu_device *smmu; - struct arm_smmu_master_data *master; + struct arm_smmu_master *master; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct iommu_group *group; @@ -1913,7 +1913,7 @@ static int arm_smmu_add_device(struct device *dev) static void arm_smmu_remove_device(struct device *dev) { struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - struct arm_smmu_master_data *master; + struct arm_smmu_master *master; struct arm_smmu_device *smmu; if (!fwspec || fwspec->ops != &arm_smmu_ops) -- 2.21.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 2/7] iommu/arm-smmu-v3: Rename arm_smmu_master_data to arm_smmu_master @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb The arm_smmu_master_data structure already represents more than just the firmware data associated to a master, and will be used extensively to represent a device's state when implementing more SMMU features. Rename the structure to arm_smmu_master. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index d3880010c6cf..50cb037f3d8a 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -591,7 +591,7 @@ struct arm_smmu_device { }; /* SMMU private data for each master */ -struct arm_smmu_master_data { +struct arm_smmu_master { struct arm_smmu_device *smmu; struct arm_smmu_strtab_ent ste; }; @@ -1691,7 +1691,7 @@ static __le64 *arm_smmu_get_step_for_sid(struct arm_smmu_device *smmu, u32 sid) static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) { int i, j; - struct arm_smmu_master_data *master = fwspec->iommu_priv; + struct arm_smmu_master *master = fwspec->iommu_priv; struct arm_smmu_device *smmu = master->smmu; for (i = 0; i < fwspec->num_ids; ++i) { @@ -1712,7 +1712,7 @@ static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) static void arm_smmu_detach_dev(struct device *dev) { struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - struct arm_smmu_master_data *master = fwspec->iommu_priv; + struct arm_smmu_master *master = fwspec->iommu_priv; master->ste.assigned = false; arm_smmu_install_ste_for_dev(fwspec); @@ -1724,7 +1724,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); - struct arm_smmu_master_data *master; + struct arm_smmu_master *master; struct arm_smmu_strtab_ent *ste; if (!fwspec) @@ -1860,7 +1860,7 @@ static int arm_smmu_add_device(struct device *dev) { int i, ret; struct arm_smmu_device *smmu; - struct arm_smmu_master_data *master; + struct arm_smmu_master *master; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct iommu_group *group; @@ -1913,7 +1913,7 @@ static int arm_smmu_add_device(struct device *dev) static void arm_smmu_remove_device(struct device *dev) { struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - struct arm_smmu_master_data *master; + struct arm_smmu_master *master; struct arm_smmu_device *smmu; if (!fwspec || fwspec->ops != &arm_smmu_ops) -- 2.21.0 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 2/7] iommu/arm-smmu-v3: Rename arm_smmu_master_data to arm_smmu_master @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: iommu, linux-arm-kernel, linux-acpi, robin.murphy, joro, hanjun.guo, lorenzo.pieralisi, sudeep.holla, rjw, lenb, okaya, zhongmiao, eric.auger The arm_smmu_master_data structure already represents more than just the firmware data associated to a master, and will be used extensively to represent a device's state when implementing more SMMU features. Rename the structure to arm_smmu_master. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index d3880010c6cf..50cb037f3d8a 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -591,7 +591,7 @@ struct arm_smmu_device { }; /* SMMU private data for each master */ -struct arm_smmu_master_data { +struct arm_smmu_master { struct arm_smmu_device *smmu; struct arm_smmu_strtab_ent ste; }; @@ -1691,7 +1691,7 @@ static __le64 *arm_smmu_get_step_for_sid(struct arm_smmu_device *smmu, u32 sid) static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) { int i, j; - struct arm_smmu_master_data *master = fwspec->iommu_priv; + struct arm_smmu_master *master = fwspec->iommu_priv; struct arm_smmu_device *smmu = master->smmu; for (i = 0; i < fwspec->num_ids; ++i) { @@ -1712,7 +1712,7 @@ static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) static void arm_smmu_detach_dev(struct device *dev) { struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - struct arm_smmu_master_data *master = fwspec->iommu_priv; + struct arm_smmu_master *master = fwspec->iommu_priv; master->ste.assigned = false; arm_smmu_install_ste_for_dev(fwspec); @@ -1724,7 +1724,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); - struct arm_smmu_master_data *master; + struct arm_smmu_master *master; struct arm_smmu_strtab_ent *ste; if (!fwspec) @@ -1860,7 +1860,7 @@ static int arm_smmu_add_device(struct device *dev) { int i, ret; struct arm_smmu_device *smmu; - struct arm_smmu_master_data *master; + struct arm_smmu_master *master; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct iommu_group *group; @@ -1913,7 +1913,7 @@ static int arm_smmu_add_device(struct device *dev) static void arm_smmu_remove_device(struct device *dev) { struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - struct arm_smmu_master_data *master; + struct arm_smmu_master *master; struct arm_smmu_device *smmu; if (!fwspec || fwspec->ops != &arm_smmu_ops) -- 2.21.0 ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 3/7] iommu/arm-smmu-v3: Store SteamIDs in master @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon-5wv7dgnIgG8 Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, sudeep.holla-5wv7dgnIgG8, robin.murphy-5wv7dgnIgG8, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A Simplify the attach/detach code a bit by keeping a pointer to the stream IDs in the master structure. Although not completely obvious here, it does make the subsequent support for ATS, PRI and PASID a bit simpler. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> --- drivers/iommu/arm-smmu-v3.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 50cb037f3d8a..25ba546cae7f 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -594,6 +594,8 @@ struct arm_smmu_device { struct arm_smmu_master { struct arm_smmu_device *smmu; struct arm_smmu_strtab_ent ste; + u32 *sids; + unsigned int num_sids; }; /* SMMU private data for an IOMMU domain */ @@ -1688,19 +1690,18 @@ static __le64 *arm_smmu_get_step_for_sid(struct arm_smmu_device *smmu, u32 sid) return step; } -static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) +static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) { int i, j; - struct arm_smmu_master *master = fwspec->iommu_priv; struct arm_smmu_device *smmu = master->smmu; - for (i = 0; i < fwspec->num_ids; ++i) { - u32 sid = fwspec->ids[i]; + for (i = 0; i < master->num_sids; ++i) { + u32 sid = master->sids[i]; __le64 *step = arm_smmu_get_step_for_sid(smmu, sid); /* Bridged PCI devices may end up with duplicated IDs */ for (j = 0; j < i; j++) - if (fwspec->ids[j] == sid) + if (master->sids[j] == sid) break; if (j < i) continue; @@ -1709,13 +1710,10 @@ static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) } } -static void arm_smmu_detach_dev(struct device *dev) +static void arm_smmu_detach_dev(struct arm_smmu_master *master) { - struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - struct arm_smmu_master *master = fwspec->iommu_priv; - master->ste.assigned = false; - arm_smmu_install_ste_for_dev(fwspec); + arm_smmu_install_ste_for_dev(master); } static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) @@ -1736,7 +1734,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) /* Already attached to a different domain? */ if (ste->assigned) - arm_smmu_detach_dev(dev); + arm_smmu_detach_dev(master); mutex_lock(&smmu_domain->init_mutex); @@ -1770,7 +1768,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) ste->s2_cfg = &smmu_domain->s2_cfg; } - arm_smmu_install_ste_for_dev(fwspec); + arm_smmu_install_ste_for_dev(master); out_unlock: mutex_unlock(&smmu_domain->init_mutex); return ret; @@ -1883,12 +1881,14 @@ static int arm_smmu_add_device(struct device *dev) return -ENOMEM; master->smmu = smmu; + master->sids = fwspec->ids; + master->num_sids = fwspec->num_ids; fwspec->iommu_priv = master; } /* Check the SIDs are in range of the SMMU and our stream table */ - for (i = 0; i < fwspec->num_ids; i++) { - u32 sid = fwspec->ids[i]; + for (i = 0; i < master->num_sids; i++) { + u32 sid = master->sids[i]; if (!arm_smmu_sid_in_range(smmu, sid)) return -ERANGE; @@ -1922,7 +1922,7 @@ static void arm_smmu_remove_device(struct device *dev) master = fwspec->iommu_priv; smmu = master->smmu; if (master && master->ste.assigned) - arm_smmu_detach_dev(dev); + arm_smmu_detach_dev(master); iommu_group_remove_device(dev); iommu_device_unlink(&smmu->iommu, dev); kfree(master); -- 2.21.0 ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 3/7] iommu/arm-smmu-v3: Store SteamIDs in master @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: lorenzo.pieralisi, eric.auger, zhongmiao, okaya, joro, rjw, linux-acpi, iommu, hanjun.guo, sudeep.holla, robin.murphy, linux-arm-kernel, lenb Simplify the attach/detach code a bit by keeping a pointer to the stream IDs in the master structure. Although not completely obvious here, it does make the subsequent support for ATS, PRI and PASID a bit simpler. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 50cb037f3d8a..25ba546cae7f 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -594,6 +594,8 @@ struct arm_smmu_device { struct arm_smmu_master { struct arm_smmu_device *smmu; struct arm_smmu_strtab_ent ste; + u32 *sids; + unsigned int num_sids; }; /* SMMU private data for an IOMMU domain */ @@ -1688,19 +1690,18 @@ static __le64 *arm_smmu_get_step_for_sid(struct arm_smmu_device *smmu, u32 sid) return step; } -static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) +static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) { int i, j; - struct arm_smmu_master *master = fwspec->iommu_priv; struct arm_smmu_device *smmu = master->smmu; - for (i = 0; i < fwspec->num_ids; ++i) { - u32 sid = fwspec->ids[i]; + for (i = 0; i < master->num_sids; ++i) { + u32 sid = master->sids[i]; __le64 *step = arm_smmu_get_step_for_sid(smmu, sid); /* Bridged PCI devices may end up with duplicated IDs */ for (j = 0; j < i; j++) - if (fwspec->ids[j] == sid) + if (master->sids[j] == sid) break; if (j < i) continue; @@ -1709,13 +1710,10 @@ static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) } } -static void arm_smmu_detach_dev(struct device *dev) +static void arm_smmu_detach_dev(struct arm_smmu_master *master) { - struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - struct arm_smmu_master *master = fwspec->iommu_priv; - master->ste.assigned = false; - arm_smmu_install_ste_for_dev(fwspec); + arm_smmu_install_ste_for_dev(master); } static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) @@ -1736,7 +1734,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) /* Already attached to a different domain? */ if (ste->assigned) - arm_smmu_detach_dev(dev); + arm_smmu_detach_dev(master); mutex_lock(&smmu_domain->init_mutex); @@ -1770,7 +1768,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) ste->s2_cfg = &smmu_domain->s2_cfg; } - arm_smmu_install_ste_for_dev(fwspec); + arm_smmu_install_ste_for_dev(master); out_unlock: mutex_unlock(&smmu_domain->init_mutex); return ret; @@ -1883,12 +1881,14 @@ static int arm_smmu_add_device(struct device *dev) return -ENOMEM; master->smmu = smmu; + master->sids = fwspec->ids; + master->num_sids = fwspec->num_ids; fwspec->iommu_priv = master; } /* Check the SIDs are in range of the SMMU and our stream table */ - for (i = 0; i < fwspec->num_ids; i++) { - u32 sid = fwspec->ids[i]; + for (i = 0; i < master->num_sids; i++) { + u32 sid = master->sids[i]; if (!arm_smmu_sid_in_range(smmu, sid)) return -ERANGE; @@ -1922,7 +1922,7 @@ static void arm_smmu_remove_device(struct device *dev) master = fwspec->iommu_priv; smmu = master->smmu; if (master && master->ste.assigned) - arm_smmu_detach_dev(dev); + arm_smmu_detach_dev(master); iommu_group_remove_device(dev); iommu_device_unlink(&smmu->iommu, dev); kfree(master); -- 2.21.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 3/7] iommu/arm-smmu-v3: Store SteamIDs in master @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb Simplify the attach/detach code a bit by keeping a pointer to the stream IDs in the master structure. Although not completely obvious here, it does make the subsequent support for ATS, PRI and PASID a bit simpler. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 50cb037f3d8a..25ba546cae7f 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -594,6 +594,8 @@ struct arm_smmu_device { struct arm_smmu_master { struct arm_smmu_device *smmu; struct arm_smmu_strtab_ent ste; + u32 *sids; + unsigned int num_sids; }; /* SMMU private data for an IOMMU domain */ @@ -1688,19 +1690,18 @@ static __le64 *arm_smmu_get_step_for_sid(struct arm_smmu_device *smmu, u32 sid) return step; } -static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) +static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) { int i, j; - struct arm_smmu_master *master = fwspec->iommu_priv; struct arm_smmu_device *smmu = master->smmu; - for (i = 0; i < fwspec->num_ids; ++i) { - u32 sid = fwspec->ids[i]; + for (i = 0; i < master->num_sids; ++i) { + u32 sid = master->sids[i]; __le64 *step = arm_smmu_get_step_for_sid(smmu, sid); /* Bridged PCI devices may end up with duplicated IDs */ for (j = 0; j < i; j++) - if (fwspec->ids[j] == sid) + if (master->sids[j] == sid) break; if (j < i) continue; @@ -1709,13 +1710,10 @@ static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) } } -static void arm_smmu_detach_dev(struct device *dev) +static void arm_smmu_detach_dev(struct arm_smmu_master *master) { - struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - struct arm_smmu_master *master = fwspec->iommu_priv; - master->ste.assigned = false; - arm_smmu_install_ste_for_dev(fwspec); + arm_smmu_install_ste_for_dev(master); } static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) @@ -1736,7 +1734,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) /* Already attached to a different domain? */ if (ste->assigned) - arm_smmu_detach_dev(dev); + arm_smmu_detach_dev(master); mutex_lock(&smmu_domain->init_mutex); @@ -1770,7 +1768,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) ste->s2_cfg = &smmu_domain->s2_cfg; } - arm_smmu_install_ste_for_dev(fwspec); + arm_smmu_install_ste_for_dev(master); out_unlock: mutex_unlock(&smmu_domain->init_mutex); return ret; @@ -1883,12 +1881,14 @@ static int arm_smmu_add_device(struct device *dev) return -ENOMEM; master->smmu = smmu; + master->sids = fwspec->ids; + master->num_sids = fwspec->num_ids; fwspec->iommu_priv = master; } /* Check the SIDs are in range of the SMMU and our stream table */ - for (i = 0; i < fwspec->num_ids; i++) { - u32 sid = fwspec->ids[i]; + for (i = 0; i < master->num_sids; i++) { + u32 sid = master->sids[i]; if (!arm_smmu_sid_in_range(smmu, sid)) return -ERANGE; @@ -1922,7 +1922,7 @@ static void arm_smmu_remove_device(struct device *dev) master = fwspec->iommu_priv; smmu = master->smmu; if (master && master->ste.assigned) - arm_smmu_detach_dev(dev); + arm_smmu_detach_dev(master); iommu_group_remove_device(dev); iommu_device_unlink(&smmu->iommu, dev); kfree(master); -- 2.21.0 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 3/7] iommu/arm-smmu-v3: Store SteamIDs in master @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: iommu, linux-arm-kernel, linux-acpi, robin.murphy, joro, hanjun.guo, lorenzo.pieralisi, sudeep.holla, rjw, lenb, okaya, zhongmiao, eric.auger Simplify the attach/detach code a bit by keeping a pointer to the stream IDs in the master structure. Although not completely obvious here, it does make the subsequent support for ATS, PRI and PASID a bit simpler. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 50cb037f3d8a..25ba546cae7f 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -594,6 +594,8 @@ struct arm_smmu_device { struct arm_smmu_master { struct arm_smmu_device *smmu; struct arm_smmu_strtab_ent ste; + u32 *sids; + unsigned int num_sids; }; /* SMMU private data for an IOMMU domain */ @@ -1688,19 +1690,18 @@ static __le64 *arm_smmu_get_step_for_sid(struct arm_smmu_device *smmu, u32 sid) return step; } -static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) +static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) { int i, j; - struct arm_smmu_master *master = fwspec->iommu_priv; struct arm_smmu_device *smmu = master->smmu; - for (i = 0; i < fwspec->num_ids; ++i) { - u32 sid = fwspec->ids[i]; + for (i = 0; i < master->num_sids; ++i) { + u32 sid = master->sids[i]; __le64 *step = arm_smmu_get_step_for_sid(smmu, sid); /* Bridged PCI devices may end up with duplicated IDs */ for (j = 0; j < i; j++) - if (fwspec->ids[j] == sid) + if (master->sids[j] == sid) break; if (j < i) continue; @@ -1709,13 +1710,10 @@ static void arm_smmu_install_ste_for_dev(struct iommu_fwspec *fwspec) } } -static void arm_smmu_detach_dev(struct device *dev) +static void arm_smmu_detach_dev(struct arm_smmu_master *master) { - struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); - struct arm_smmu_master *master = fwspec->iommu_priv; - master->ste.assigned = false; - arm_smmu_install_ste_for_dev(fwspec); + arm_smmu_install_ste_for_dev(master); } static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) @@ -1736,7 +1734,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) /* Already attached to a different domain? */ if (ste->assigned) - arm_smmu_detach_dev(dev); + arm_smmu_detach_dev(master); mutex_lock(&smmu_domain->init_mutex); @@ -1770,7 +1768,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) ste->s2_cfg = &smmu_domain->s2_cfg; } - arm_smmu_install_ste_for_dev(fwspec); + arm_smmu_install_ste_for_dev(master); out_unlock: mutex_unlock(&smmu_domain->init_mutex); return ret; @@ -1883,12 +1881,14 @@ static int arm_smmu_add_device(struct device *dev) return -ENOMEM; master->smmu = smmu; + master->sids = fwspec->ids; + master->num_sids = fwspec->num_ids; fwspec->iommu_priv = master; } /* Check the SIDs are in range of the SMMU and our stream table */ - for (i = 0; i < fwspec->num_ids; i++) { - u32 sid = fwspec->ids[i]; + for (i = 0; i < master->num_sids; i++) { + u32 sid = master->sids[i]; if (!arm_smmu_sid_in_range(smmu, sid)) return -ERANGE; @@ -1922,7 +1922,7 @@ static void arm_smmu_remove_device(struct device *dev) master = fwspec->iommu_priv; smmu = master->smmu; if (master && master->ste.assigned) - arm_smmu_detach_dev(dev); + arm_smmu_detach_dev(master); iommu_group_remove_device(dev); iommu_device_unlink(&smmu->iommu, dev); kfree(master); -- 2.21.0 ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 4/7] iommu/arm-smmu-v3: Add a master->domain pointer @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon-5wv7dgnIgG8 Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, sudeep.holla-5wv7dgnIgG8, robin.murphy-5wv7dgnIgG8, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A As we're going to track domain-master links more closely for ATS and CD invalidation, add pointer to the attached domain in struct arm_smmu_master. As a result, arm_smmu_strtab_ent is redundant and can be removed. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> --- drivers/iommu/arm-smmu-v3.c | 92 ++++++++++++++++++------------------- 1 file changed, 45 insertions(+), 47 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 25ba546cae7f..7b425483f4b6 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -505,19 +505,6 @@ struct arm_smmu_s2_cfg { u64 vtcr; }; -struct arm_smmu_strtab_ent { - /* - * An STE is "assigned" if the master emitting the corresponding SID - * is attached to a domain. The behaviour of an unassigned STE is - * determined by the disable_bypass parameter, whereas an assigned - * STE behaves according to s1_cfg/s2_cfg, which themselves are - * configured according to the domain type. - */ - bool assigned; - struct arm_smmu_s1_cfg *s1_cfg; - struct arm_smmu_s2_cfg *s2_cfg; -}; - struct arm_smmu_strtab_cfg { __le64 *strtab; dma_addr_t strtab_dma; @@ -593,7 +580,7 @@ struct arm_smmu_device { /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; - struct arm_smmu_strtab_ent ste; + struct arm_smmu_domain *domain; u32 *sids; unsigned int num_sids; }; @@ -1087,8 +1074,8 @@ static void arm_smmu_sync_ste_for_sid(struct arm_smmu_device *smmu, u32 sid) arm_smmu_cmdq_issue_sync(smmu); } -static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, - __le64 *dst, struct arm_smmu_strtab_ent *ste) +static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, + __le64 *dst) { /* * This is hideously complicated, but we only really care about @@ -1108,6 +1095,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, */ u64 val = le64_to_cpu(dst[0]); bool ste_live = false; + struct arm_smmu_device *smmu = NULL; + struct arm_smmu_s1_cfg *s1_cfg = NULL; + struct arm_smmu_s2_cfg *s2_cfg = NULL; + struct arm_smmu_domain *smmu_domain = NULL; struct arm_smmu_cmdq_ent prefetch_cmd = { .opcode = CMDQ_OP_PREFETCH_CFG, .prefetch = { @@ -1115,6 +1106,25 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, }, }; + if (master) { + smmu_domain = master->domain; + smmu = master->smmu; + } + + if (smmu_domain) { + switch (smmu_domain->stage) { + case ARM_SMMU_DOMAIN_S1: + s1_cfg = &smmu_domain->s1_cfg; + break; + case ARM_SMMU_DOMAIN_S2: + case ARM_SMMU_DOMAIN_NESTED: + s2_cfg = &smmu_domain->s2_cfg; + break; + default: + break; + } + } + if (val & STRTAB_STE_0_V) { switch (FIELD_GET(STRTAB_STE_0_CFG, val)) { case STRTAB_STE_0_CFG_BYPASS: @@ -1135,8 +1145,8 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, val = STRTAB_STE_0_V; /* Bypass/fault */ - if (!ste->assigned || !(ste->s1_cfg || ste->s2_cfg)) { - if (!ste->assigned && disable_bypass) + if (!smmu_domain || !(s1_cfg || s2_cfg)) { + if (!smmu_domain && disable_bypass) val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT); else val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS); @@ -1154,7 +1164,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, return; } - if (ste->s1_cfg) { + if (s1_cfg) { BUG_ON(ste_live); dst[1] = cpu_to_le64( FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | @@ -1169,22 +1179,22 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE)) dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD); - val |= (ste->s1_cfg->cdptr_dma & STRTAB_STE_0_S1CTXPTR_MASK) | + val |= (s1_cfg->cdptr_dma & STRTAB_STE_0_S1CTXPTR_MASK) | FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS); } - if (ste->s2_cfg) { + if (s2_cfg) { BUG_ON(ste_live); dst[2] = cpu_to_le64( - FIELD_PREP(STRTAB_STE_2_S2VMID, ste->s2_cfg->vmid) | - FIELD_PREP(STRTAB_STE_2_VTCR, ste->s2_cfg->vtcr) | + FIELD_PREP(STRTAB_STE_2_S2VMID, s2_cfg->vmid) | + FIELD_PREP(STRTAB_STE_2_VTCR, s2_cfg->vtcr) | #ifdef __BIG_ENDIAN STRTAB_STE_2_S2ENDI | #endif STRTAB_STE_2_S2PTW | STRTAB_STE_2_S2AA64 | STRTAB_STE_2_S2R); - dst[3] = cpu_to_le64(ste->s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK); + dst[3] = cpu_to_le64(s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK); val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS); } @@ -1201,10 +1211,9 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, static void arm_smmu_init_bypass_stes(u64 *strtab, unsigned int nent) { unsigned int i; - struct arm_smmu_strtab_ent ste = { .assigned = false }; for (i = 0; i < nent; ++i) { - arm_smmu_write_strtab_ent(NULL, -1, strtab, &ste); + arm_smmu_write_strtab_ent(NULL, -1, strtab); strtab += STRTAB_STE_DWORDS; } } @@ -1706,13 +1715,16 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) if (j < i) continue; - arm_smmu_write_strtab_ent(smmu, sid, step, &master->ste); + arm_smmu_write_strtab_ent(master, sid, step); } } static void arm_smmu_detach_dev(struct arm_smmu_master *master) { - master->ste.assigned = false; + if (!master->domain) + return; + + master->domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -1723,18 +1735,14 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct arm_smmu_master *master; - struct arm_smmu_strtab_ent *ste; if (!fwspec) return -ENOENT; master = fwspec->iommu_priv; smmu = master->smmu; - ste = &master->ste; - /* Already attached to a different domain? */ - if (ste->assigned) - arm_smmu_detach_dev(master); + arm_smmu_detach_dev(master); mutex_lock(&smmu_domain->init_mutex); @@ -1754,19 +1762,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) goto out_unlock; } - ste->assigned = true; + master->domain = smmu_domain; - if (smmu_domain->stage == ARM_SMMU_DOMAIN_BYPASS) { - ste->s1_cfg = NULL; - ste->s2_cfg = NULL; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - ste->s1_cfg = &smmu_domain->s1_cfg; - ste->s2_cfg = NULL; - arm_smmu_write_ctx_desc(smmu, ste->s1_cfg); - } else { - ste->s1_cfg = NULL; - ste->s2_cfg = &smmu_domain->s2_cfg; - } + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) + arm_smmu_write_ctx_desc(smmu, &smmu_domain->s1_cfg); arm_smmu_install_ste_for_dev(master); out_unlock: @@ -1921,8 +1920,7 @@ static void arm_smmu_remove_device(struct device *dev) master = fwspec->iommu_priv; smmu = master->smmu; - if (master && master->ste.assigned) - arm_smmu_detach_dev(master); + arm_smmu_detach_dev(master); iommu_group_remove_device(dev); iommu_device_unlink(&smmu->iommu, dev); kfree(master); -- 2.21.0 ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 4/7] iommu/arm-smmu-v3: Add a master->domain pointer @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: lorenzo.pieralisi, eric.auger, zhongmiao, okaya, joro, rjw, linux-acpi, iommu, hanjun.guo, sudeep.holla, robin.murphy, linux-arm-kernel, lenb As we're going to track domain-master links more closely for ATS and CD invalidation, add pointer to the attached domain in struct arm_smmu_master. As a result, arm_smmu_strtab_ent is redundant and can be removed. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 92 ++++++++++++++++++------------------- 1 file changed, 45 insertions(+), 47 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 25ba546cae7f..7b425483f4b6 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -505,19 +505,6 @@ struct arm_smmu_s2_cfg { u64 vtcr; }; -struct arm_smmu_strtab_ent { - /* - * An STE is "assigned" if the master emitting the corresponding SID - * is attached to a domain. The behaviour of an unassigned STE is - * determined by the disable_bypass parameter, whereas an assigned - * STE behaves according to s1_cfg/s2_cfg, which themselves are - * configured according to the domain type. - */ - bool assigned; - struct arm_smmu_s1_cfg *s1_cfg; - struct arm_smmu_s2_cfg *s2_cfg; -}; - struct arm_smmu_strtab_cfg { __le64 *strtab; dma_addr_t strtab_dma; @@ -593,7 +580,7 @@ struct arm_smmu_device { /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; - struct arm_smmu_strtab_ent ste; + struct arm_smmu_domain *domain; u32 *sids; unsigned int num_sids; }; @@ -1087,8 +1074,8 @@ static void arm_smmu_sync_ste_for_sid(struct arm_smmu_device *smmu, u32 sid) arm_smmu_cmdq_issue_sync(smmu); } -static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, - __le64 *dst, struct arm_smmu_strtab_ent *ste) +static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, + __le64 *dst) { /* * This is hideously complicated, but we only really care about @@ -1108,6 +1095,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, */ u64 val = le64_to_cpu(dst[0]); bool ste_live = false; + struct arm_smmu_device *smmu = NULL; + struct arm_smmu_s1_cfg *s1_cfg = NULL; + struct arm_smmu_s2_cfg *s2_cfg = NULL; + struct arm_smmu_domain *smmu_domain = NULL; struct arm_smmu_cmdq_ent prefetch_cmd = { .opcode = CMDQ_OP_PREFETCH_CFG, .prefetch = { @@ -1115,6 +1106,25 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, }, }; + if (master) { + smmu_domain = master->domain; + smmu = master->smmu; + } + + if (smmu_domain) { + switch (smmu_domain->stage) { + case ARM_SMMU_DOMAIN_S1: + s1_cfg = &smmu_domain->s1_cfg; + break; + case ARM_SMMU_DOMAIN_S2: + case ARM_SMMU_DOMAIN_NESTED: + s2_cfg = &smmu_domain->s2_cfg; + break; + default: + break; + } + } + if (val & STRTAB_STE_0_V) { switch (FIELD_GET(STRTAB_STE_0_CFG, val)) { case STRTAB_STE_0_CFG_BYPASS: @@ -1135,8 +1145,8 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, val = STRTAB_STE_0_V; /* Bypass/fault */ - if (!ste->assigned || !(ste->s1_cfg || ste->s2_cfg)) { - if (!ste->assigned && disable_bypass) + if (!smmu_domain || !(s1_cfg || s2_cfg)) { + if (!smmu_domain && disable_bypass) val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT); else val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS); @@ -1154,7 +1164,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, return; } - if (ste->s1_cfg) { + if (s1_cfg) { BUG_ON(ste_live); dst[1] = cpu_to_le64( FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | @@ -1169,22 +1179,22 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE)) dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD); - val |= (ste->s1_cfg->cdptr_dma & STRTAB_STE_0_S1CTXPTR_MASK) | + val |= (s1_cfg->cdptr_dma & STRTAB_STE_0_S1CTXPTR_MASK) | FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS); } - if (ste->s2_cfg) { + if (s2_cfg) { BUG_ON(ste_live); dst[2] = cpu_to_le64( - FIELD_PREP(STRTAB_STE_2_S2VMID, ste->s2_cfg->vmid) | - FIELD_PREP(STRTAB_STE_2_VTCR, ste->s2_cfg->vtcr) | + FIELD_PREP(STRTAB_STE_2_S2VMID, s2_cfg->vmid) | + FIELD_PREP(STRTAB_STE_2_VTCR, s2_cfg->vtcr) | #ifdef __BIG_ENDIAN STRTAB_STE_2_S2ENDI | #endif STRTAB_STE_2_S2PTW | STRTAB_STE_2_S2AA64 | STRTAB_STE_2_S2R); - dst[3] = cpu_to_le64(ste->s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK); + dst[3] = cpu_to_le64(s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK); val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS); } @@ -1201,10 +1211,9 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, static void arm_smmu_init_bypass_stes(u64 *strtab, unsigned int nent) { unsigned int i; - struct arm_smmu_strtab_ent ste = { .assigned = false }; for (i = 0; i < nent; ++i) { - arm_smmu_write_strtab_ent(NULL, -1, strtab, &ste); + arm_smmu_write_strtab_ent(NULL, -1, strtab); strtab += STRTAB_STE_DWORDS; } } @@ -1706,13 +1715,16 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) if (j < i) continue; - arm_smmu_write_strtab_ent(smmu, sid, step, &master->ste); + arm_smmu_write_strtab_ent(master, sid, step); } } static void arm_smmu_detach_dev(struct arm_smmu_master *master) { - master->ste.assigned = false; + if (!master->domain) + return; + + master->domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -1723,18 +1735,14 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct arm_smmu_master *master; - struct arm_smmu_strtab_ent *ste; if (!fwspec) return -ENOENT; master = fwspec->iommu_priv; smmu = master->smmu; - ste = &master->ste; - /* Already attached to a different domain? */ - if (ste->assigned) - arm_smmu_detach_dev(master); + arm_smmu_detach_dev(master); mutex_lock(&smmu_domain->init_mutex); @@ -1754,19 +1762,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) goto out_unlock; } - ste->assigned = true; + master->domain = smmu_domain; - if (smmu_domain->stage == ARM_SMMU_DOMAIN_BYPASS) { - ste->s1_cfg = NULL; - ste->s2_cfg = NULL; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - ste->s1_cfg = &smmu_domain->s1_cfg; - ste->s2_cfg = NULL; - arm_smmu_write_ctx_desc(smmu, ste->s1_cfg); - } else { - ste->s1_cfg = NULL; - ste->s2_cfg = &smmu_domain->s2_cfg; - } + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) + arm_smmu_write_ctx_desc(smmu, &smmu_domain->s1_cfg); arm_smmu_install_ste_for_dev(master); out_unlock: @@ -1921,8 +1920,7 @@ static void arm_smmu_remove_device(struct device *dev) master = fwspec->iommu_priv; smmu = master->smmu; - if (master && master->ste.assigned) - arm_smmu_detach_dev(master); + arm_smmu_detach_dev(master); iommu_group_remove_device(dev); iommu_device_unlink(&smmu->iommu, dev); kfree(master); -- 2.21.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 4/7] iommu/arm-smmu-v3: Add a master->domain pointer @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb As we're going to track domain-master links more closely for ATS and CD invalidation, add pointer to the attached domain in struct arm_smmu_master. As a result, arm_smmu_strtab_ent is redundant and can be removed. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 92 ++++++++++++++++++------------------- 1 file changed, 45 insertions(+), 47 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 25ba546cae7f..7b425483f4b6 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -505,19 +505,6 @@ struct arm_smmu_s2_cfg { u64 vtcr; }; -struct arm_smmu_strtab_ent { - /* - * An STE is "assigned" if the master emitting the corresponding SID - * is attached to a domain. The behaviour of an unassigned STE is - * determined by the disable_bypass parameter, whereas an assigned - * STE behaves according to s1_cfg/s2_cfg, which themselves are - * configured according to the domain type. - */ - bool assigned; - struct arm_smmu_s1_cfg *s1_cfg; - struct arm_smmu_s2_cfg *s2_cfg; -}; - struct arm_smmu_strtab_cfg { __le64 *strtab; dma_addr_t strtab_dma; @@ -593,7 +580,7 @@ struct arm_smmu_device { /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; - struct arm_smmu_strtab_ent ste; + struct arm_smmu_domain *domain; u32 *sids; unsigned int num_sids; }; @@ -1087,8 +1074,8 @@ static void arm_smmu_sync_ste_for_sid(struct arm_smmu_device *smmu, u32 sid) arm_smmu_cmdq_issue_sync(smmu); } -static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, - __le64 *dst, struct arm_smmu_strtab_ent *ste) +static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, + __le64 *dst) { /* * This is hideously complicated, but we only really care about @@ -1108,6 +1095,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, */ u64 val = le64_to_cpu(dst[0]); bool ste_live = false; + struct arm_smmu_device *smmu = NULL; + struct arm_smmu_s1_cfg *s1_cfg = NULL; + struct arm_smmu_s2_cfg *s2_cfg = NULL; + struct arm_smmu_domain *smmu_domain = NULL; struct arm_smmu_cmdq_ent prefetch_cmd = { .opcode = CMDQ_OP_PREFETCH_CFG, .prefetch = { @@ -1115,6 +1106,25 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, }, }; + if (master) { + smmu_domain = master->domain; + smmu = master->smmu; + } + + if (smmu_domain) { + switch (smmu_domain->stage) { + case ARM_SMMU_DOMAIN_S1: + s1_cfg = &smmu_domain->s1_cfg; + break; + case ARM_SMMU_DOMAIN_S2: + case ARM_SMMU_DOMAIN_NESTED: + s2_cfg = &smmu_domain->s2_cfg; + break; + default: + break; + } + } + if (val & STRTAB_STE_0_V) { switch (FIELD_GET(STRTAB_STE_0_CFG, val)) { case STRTAB_STE_0_CFG_BYPASS: @@ -1135,8 +1145,8 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, val = STRTAB_STE_0_V; /* Bypass/fault */ - if (!ste->assigned || !(ste->s1_cfg || ste->s2_cfg)) { - if (!ste->assigned && disable_bypass) + if (!smmu_domain || !(s1_cfg || s2_cfg)) { + if (!smmu_domain && disable_bypass) val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT); else val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS); @@ -1154,7 +1164,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, return; } - if (ste->s1_cfg) { + if (s1_cfg) { BUG_ON(ste_live); dst[1] = cpu_to_le64( FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | @@ -1169,22 +1179,22 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE)) dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD); - val |= (ste->s1_cfg->cdptr_dma & STRTAB_STE_0_S1CTXPTR_MASK) | + val |= (s1_cfg->cdptr_dma & STRTAB_STE_0_S1CTXPTR_MASK) | FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS); } - if (ste->s2_cfg) { + if (s2_cfg) { BUG_ON(ste_live); dst[2] = cpu_to_le64( - FIELD_PREP(STRTAB_STE_2_S2VMID, ste->s2_cfg->vmid) | - FIELD_PREP(STRTAB_STE_2_VTCR, ste->s2_cfg->vtcr) | + FIELD_PREP(STRTAB_STE_2_S2VMID, s2_cfg->vmid) | + FIELD_PREP(STRTAB_STE_2_VTCR, s2_cfg->vtcr) | #ifdef __BIG_ENDIAN STRTAB_STE_2_S2ENDI | #endif STRTAB_STE_2_S2PTW | STRTAB_STE_2_S2AA64 | STRTAB_STE_2_S2R); - dst[3] = cpu_to_le64(ste->s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK); + dst[3] = cpu_to_le64(s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK); val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS); } @@ -1201,10 +1211,9 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, static void arm_smmu_init_bypass_stes(u64 *strtab, unsigned int nent) { unsigned int i; - struct arm_smmu_strtab_ent ste = { .assigned = false }; for (i = 0; i < nent; ++i) { - arm_smmu_write_strtab_ent(NULL, -1, strtab, &ste); + arm_smmu_write_strtab_ent(NULL, -1, strtab); strtab += STRTAB_STE_DWORDS; } } @@ -1706,13 +1715,16 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) if (j < i) continue; - arm_smmu_write_strtab_ent(smmu, sid, step, &master->ste); + arm_smmu_write_strtab_ent(master, sid, step); } } static void arm_smmu_detach_dev(struct arm_smmu_master *master) { - master->ste.assigned = false; + if (!master->domain) + return; + + master->domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -1723,18 +1735,14 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct arm_smmu_master *master; - struct arm_smmu_strtab_ent *ste; if (!fwspec) return -ENOENT; master = fwspec->iommu_priv; smmu = master->smmu; - ste = &master->ste; - /* Already attached to a different domain? */ - if (ste->assigned) - arm_smmu_detach_dev(master); + arm_smmu_detach_dev(master); mutex_lock(&smmu_domain->init_mutex); @@ -1754,19 +1762,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) goto out_unlock; } - ste->assigned = true; + master->domain = smmu_domain; - if (smmu_domain->stage == ARM_SMMU_DOMAIN_BYPASS) { - ste->s1_cfg = NULL; - ste->s2_cfg = NULL; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - ste->s1_cfg = &smmu_domain->s1_cfg; - ste->s2_cfg = NULL; - arm_smmu_write_ctx_desc(smmu, ste->s1_cfg); - } else { - ste->s1_cfg = NULL; - ste->s2_cfg = &smmu_domain->s2_cfg; - } + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) + arm_smmu_write_ctx_desc(smmu, &smmu_domain->s1_cfg); arm_smmu_install_ste_for_dev(master); out_unlock: @@ -1921,8 +1920,7 @@ static void arm_smmu_remove_device(struct device *dev) master = fwspec->iommu_priv; smmu = master->smmu; - if (master && master->ste.assigned) - arm_smmu_detach_dev(master); + arm_smmu_detach_dev(master); iommu_group_remove_device(dev); iommu_device_unlink(&smmu->iommu, dev); kfree(master); -- 2.21.0 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 4/7] iommu/arm-smmu-v3: Add a master->domain pointer @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: iommu, linux-arm-kernel, linux-acpi, robin.murphy, joro, hanjun.guo, lorenzo.pieralisi, sudeep.holla, rjw, lenb, okaya, zhongmiao, eric.auger As we're going to track domain-master links more closely for ATS and CD invalidation, add pointer to the attached domain in struct arm_smmu_master. As a result, arm_smmu_strtab_ent is redundant and can be removed. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 92 ++++++++++++++++++------------------- 1 file changed, 45 insertions(+), 47 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 25ba546cae7f..7b425483f4b6 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -505,19 +505,6 @@ struct arm_smmu_s2_cfg { u64 vtcr; }; -struct arm_smmu_strtab_ent { - /* - * An STE is "assigned" if the master emitting the corresponding SID - * is attached to a domain. The behaviour of an unassigned STE is - * determined by the disable_bypass parameter, whereas an assigned - * STE behaves according to s1_cfg/s2_cfg, which themselves are - * configured according to the domain type. - */ - bool assigned; - struct arm_smmu_s1_cfg *s1_cfg; - struct arm_smmu_s2_cfg *s2_cfg; -}; - struct arm_smmu_strtab_cfg { __le64 *strtab; dma_addr_t strtab_dma; @@ -593,7 +580,7 @@ struct arm_smmu_device { /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; - struct arm_smmu_strtab_ent ste; + struct arm_smmu_domain *domain; u32 *sids; unsigned int num_sids; }; @@ -1087,8 +1074,8 @@ static void arm_smmu_sync_ste_for_sid(struct arm_smmu_device *smmu, u32 sid) arm_smmu_cmdq_issue_sync(smmu); } -static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, - __le64 *dst, struct arm_smmu_strtab_ent *ste) +static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, + __le64 *dst) { /* * This is hideously complicated, but we only really care about @@ -1108,6 +1095,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, */ u64 val = le64_to_cpu(dst[0]); bool ste_live = false; + struct arm_smmu_device *smmu = NULL; + struct arm_smmu_s1_cfg *s1_cfg = NULL; + struct arm_smmu_s2_cfg *s2_cfg = NULL; + struct arm_smmu_domain *smmu_domain = NULL; struct arm_smmu_cmdq_ent prefetch_cmd = { .opcode = CMDQ_OP_PREFETCH_CFG, .prefetch = { @@ -1115,6 +1106,25 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, }, }; + if (master) { + smmu_domain = master->domain; + smmu = master->smmu; + } + + if (smmu_domain) { + switch (smmu_domain->stage) { + case ARM_SMMU_DOMAIN_S1: + s1_cfg = &smmu_domain->s1_cfg; + break; + case ARM_SMMU_DOMAIN_S2: + case ARM_SMMU_DOMAIN_NESTED: + s2_cfg = &smmu_domain->s2_cfg; + break; + default: + break; + } + } + if (val & STRTAB_STE_0_V) { switch (FIELD_GET(STRTAB_STE_0_CFG, val)) { case STRTAB_STE_0_CFG_BYPASS: @@ -1135,8 +1145,8 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, val = STRTAB_STE_0_V; /* Bypass/fault */ - if (!ste->assigned || !(ste->s1_cfg || ste->s2_cfg)) { - if (!ste->assigned && disable_bypass) + if (!smmu_domain || !(s1_cfg || s2_cfg)) { + if (!smmu_domain && disable_bypass) val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT); else val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS); @@ -1154,7 +1164,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, return; } - if (ste->s1_cfg) { + if (s1_cfg) { BUG_ON(ste_live); dst[1] = cpu_to_le64( FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | @@ -1169,22 +1179,22 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE)) dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD); - val |= (ste->s1_cfg->cdptr_dma & STRTAB_STE_0_S1CTXPTR_MASK) | + val |= (s1_cfg->cdptr_dma & STRTAB_STE_0_S1CTXPTR_MASK) | FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS); } - if (ste->s2_cfg) { + if (s2_cfg) { BUG_ON(ste_live); dst[2] = cpu_to_le64( - FIELD_PREP(STRTAB_STE_2_S2VMID, ste->s2_cfg->vmid) | - FIELD_PREP(STRTAB_STE_2_VTCR, ste->s2_cfg->vtcr) | + FIELD_PREP(STRTAB_STE_2_S2VMID, s2_cfg->vmid) | + FIELD_PREP(STRTAB_STE_2_VTCR, s2_cfg->vtcr) | #ifdef __BIG_ENDIAN STRTAB_STE_2_S2ENDI | #endif STRTAB_STE_2_S2PTW | STRTAB_STE_2_S2AA64 | STRTAB_STE_2_S2R); - dst[3] = cpu_to_le64(ste->s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK); + dst[3] = cpu_to_le64(s2_cfg->vttbr & STRTAB_STE_3_S2TTB_MASK); val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS); } @@ -1201,10 +1211,9 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, static void arm_smmu_init_bypass_stes(u64 *strtab, unsigned int nent) { unsigned int i; - struct arm_smmu_strtab_ent ste = { .assigned = false }; for (i = 0; i < nent; ++i) { - arm_smmu_write_strtab_ent(NULL, -1, strtab, &ste); + arm_smmu_write_strtab_ent(NULL, -1, strtab); strtab += STRTAB_STE_DWORDS; } } @@ -1706,13 +1715,16 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) if (j < i) continue; - arm_smmu_write_strtab_ent(smmu, sid, step, &master->ste); + arm_smmu_write_strtab_ent(master, sid, step); } } static void arm_smmu_detach_dev(struct arm_smmu_master *master) { - master->ste.assigned = false; + if (!master->domain) + return; + + master->domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -1723,18 +1735,14 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct arm_smmu_master *master; - struct arm_smmu_strtab_ent *ste; if (!fwspec) return -ENOENT; master = fwspec->iommu_priv; smmu = master->smmu; - ste = &master->ste; - /* Already attached to a different domain? */ - if (ste->assigned) - arm_smmu_detach_dev(master); + arm_smmu_detach_dev(master); mutex_lock(&smmu_domain->init_mutex); @@ -1754,19 +1762,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) goto out_unlock; } - ste->assigned = true; + master->domain = smmu_domain; - if (smmu_domain->stage == ARM_SMMU_DOMAIN_BYPASS) { - ste->s1_cfg = NULL; - ste->s2_cfg = NULL; - } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { - ste->s1_cfg = &smmu_domain->s1_cfg; - ste->s2_cfg = NULL; - arm_smmu_write_ctx_desc(smmu, ste->s1_cfg); - } else { - ste->s1_cfg = NULL; - ste->s2_cfg = &smmu_domain->s2_cfg; - } + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) + arm_smmu_write_ctx_desc(smmu, &smmu_domain->s1_cfg); arm_smmu_install_ste_for_dev(master); out_unlock: @@ -1921,8 +1920,7 @@ static void arm_smmu_remove_device(struct device *dev) master = fwspec->iommu_priv; smmu = master->smmu; - if (master && master->ste.assigned) - arm_smmu_detach_dev(master); + arm_smmu_detach_dev(master); iommu_group_remove_device(dev); iommu_device_unlink(&smmu->iommu, dev); kfree(master); -- 2.21.0 ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 5/7] iommu/arm-smmu-v3: Link domains and devices @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon-5wv7dgnIgG8 Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, sudeep.holla-5wv7dgnIgG8, robin.murphy-5wv7dgnIgG8, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A When removing a mapping from a domain, we need to send an invalidation to all devices that might have stored it in their Address Translation Cache (ATC). In addition when updating the context descriptor of a live domain, we'll need to send invalidations for all devices attached to it. Maintain a list of devices in each domain, protected by a spinlock. It is updated every time we attach or detach devices to and from domains. It needs to be a spinlock because we'll invalidate ATC entries from within hardirq-safe contexts, but it may be possible to relax the read side with RCU later. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> --- drivers/iommu/arm-smmu-v3.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 7b425483f4b6..3e7198ee9530 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -581,6 +581,7 @@ struct arm_smmu_device { struct arm_smmu_master { struct arm_smmu_device *smmu; struct arm_smmu_domain *domain; + struct list_head domain_head; u32 *sids; unsigned int num_sids; }; @@ -607,6 +608,9 @@ struct arm_smmu_domain { }; struct iommu_domain domain; + + struct list_head devices; + spinlock_t devices_lock; }; struct arm_smmu_option_prop { @@ -1504,6 +1508,9 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) } mutex_init(&smmu_domain->init_mutex); + INIT_LIST_HEAD(&smmu_domain->devices); + spin_lock_init(&smmu_domain->devices_lock); + return &smmu_domain->domain; } @@ -1721,9 +1728,16 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) static void arm_smmu_detach_dev(struct arm_smmu_master *master) { - if (!master->domain) + unsigned long flags; + struct arm_smmu_domain *smmu_domain = master->domain; + + if (!smmu_domain) return; + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_del(&master->domain_head); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + master->domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -1731,6 +1745,7 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; + unsigned long flags; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); @@ -1764,6 +1779,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master->domain = smmu_domain; + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_add(&master->domain_head, &smmu_domain->devices); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) arm_smmu_write_ctx_desc(smmu, &smmu_domain->s1_cfg); -- 2.21.0 ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 5/7] iommu/arm-smmu-v3: Link domains and devices @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: lorenzo.pieralisi, eric.auger, zhongmiao, okaya, joro, rjw, linux-acpi, iommu, hanjun.guo, sudeep.holla, robin.murphy, linux-arm-kernel, lenb When removing a mapping from a domain, we need to send an invalidation to all devices that might have stored it in their Address Translation Cache (ATC). In addition when updating the context descriptor of a live domain, we'll need to send invalidations for all devices attached to it. Maintain a list of devices in each domain, protected by a spinlock. It is updated every time we attach or detach devices to and from domains. It needs to be a spinlock because we'll invalidate ATC entries from within hardirq-safe contexts, but it may be possible to relax the read side with RCU later. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 7b425483f4b6..3e7198ee9530 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -581,6 +581,7 @@ struct arm_smmu_device { struct arm_smmu_master { struct arm_smmu_device *smmu; struct arm_smmu_domain *domain; + struct list_head domain_head; u32 *sids; unsigned int num_sids; }; @@ -607,6 +608,9 @@ struct arm_smmu_domain { }; struct iommu_domain domain; + + struct list_head devices; + spinlock_t devices_lock; }; struct arm_smmu_option_prop { @@ -1504,6 +1508,9 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) } mutex_init(&smmu_domain->init_mutex); + INIT_LIST_HEAD(&smmu_domain->devices); + spin_lock_init(&smmu_domain->devices_lock); + return &smmu_domain->domain; } @@ -1721,9 +1728,16 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) static void arm_smmu_detach_dev(struct arm_smmu_master *master) { - if (!master->domain) + unsigned long flags; + struct arm_smmu_domain *smmu_domain = master->domain; + + if (!smmu_domain) return; + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_del(&master->domain_head); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + master->domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -1731,6 +1745,7 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; + unsigned long flags; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); @@ -1764,6 +1779,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master->domain = smmu_domain; + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_add(&master->domain_head, &smmu_domain->devices); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) arm_smmu_write_ctx_desc(smmu, &smmu_domain->s1_cfg); -- 2.21.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 5/7] iommu/arm-smmu-v3: Link domains and devices @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb When removing a mapping from a domain, we need to send an invalidation to all devices that might have stored it in their Address Translation Cache (ATC). In addition when updating the context descriptor of a live domain, we'll need to send invalidations for all devices attached to it. Maintain a list of devices in each domain, protected by a spinlock. It is updated every time we attach or detach devices to and from domains. It needs to be a spinlock because we'll invalidate ATC entries from within hardirq-safe contexts, but it may be possible to relax the read side with RCU later. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 7b425483f4b6..3e7198ee9530 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -581,6 +581,7 @@ struct arm_smmu_device { struct arm_smmu_master { struct arm_smmu_device *smmu; struct arm_smmu_domain *domain; + struct list_head domain_head; u32 *sids; unsigned int num_sids; }; @@ -607,6 +608,9 @@ struct arm_smmu_domain { }; struct iommu_domain domain; + + struct list_head devices; + spinlock_t devices_lock; }; struct arm_smmu_option_prop { @@ -1504,6 +1508,9 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) } mutex_init(&smmu_domain->init_mutex); + INIT_LIST_HEAD(&smmu_domain->devices); + spin_lock_init(&smmu_domain->devices_lock); + return &smmu_domain->domain; } @@ -1721,9 +1728,16 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) static void arm_smmu_detach_dev(struct arm_smmu_master *master) { - if (!master->domain) + unsigned long flags; + struct arm_smmu_domain *smmu_domain = master->domain; + + if (!smmu_domain) return; + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_del(&master->domain_head); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + master->domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -1731,6 +1745,7 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; + unsigned long flags; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); @@ -1764,6 +1779,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master->domain = smmu_domain; + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_add(&master->domain_head, &smmu_domain->devices); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) arm_smmu_write_ctx_desc(smmu, &smmu_domain->s1_cfg); -- 2.21.0 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 5/7] iommu/arm-smmu-v3: Link domains and devices @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: iommu, linux-arm-kernel, linux-acpi, robin.murphy, joro, hanjun.guo, lorenzo.pieralisi, sudeep.holla, rjw, lenb, okaya, zhongmiao, eric.auger When removing a mapping from a domain, we need to send an invalidation to all devices that might have stored it in their Address Translation Cache (ATC). In addition when updating the context descriptor of a live domain, we'll need to send invalidations for all devices attached to it. Maintain a list of devices in each domain, protected by a spinlock. It is updated every time we attach or detach devices to and from domains. It needs to be a spinlock because we'll invalidate ATC entries from within hardirq-safe contexts, but it may be possible to relax the read side with RCU later. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 7b425483f4b6..3e7198ee9530 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -581,6 +581,7 @@ struct arm_smmu_device { struct arm_smmu_master { struct arm_smmu_device *smmu; struct arm_smmu_domain *domain; + struct list_head domain_head; u32 *sids; unsigned int num_sids; }; @@ -607,6 +608,9 @@ struct arm_smmu_domain { }; struct iommu_domain domain; + + struct list_head devices; + spinlock_t devices_lock; }; struct arm_smmu_option_prop { @@ -1504,6 +1508,9 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) } mutex_init(&smmu_domain->init_mutex); + INIT_LIST_HEAD(&smmu_domain->devices); + spin_lock_init(&smmu_domain->devices_lock); + return &smmu_domain->domain; } @@ -1721,9 +1728,16 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) static void arm_smmu_detach_dev(struct arm_smmu_master *master) { - if (!master->domain) + unsigned long flags; + struct arm_smmu_domain *smmu_domain = master->domain; + + if (!smmu_domain) return; + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_del(&master->domain_head); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + master->domain = NULL; arm_smmu_install_ste_for_dev(master); } @@ -1731,6 +1745,7 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; + unsigned long flags; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); @@ -1764,6 +1779,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master->domain = smmu_domain; + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_add(&master->domain_head, &smmu_domain->devices); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) arm_smmu_write_ctx_desc(smmu, &smmu_domain->s1_cfg); -- 2.21.0 ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon-5wv7dgnIgG8 Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, sudeep.holla-5wv7dgnIgG8, robin.murphy-5wv7dgnIgG8, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A PCIe devices can implement their own TLB, named Address Translation Cache (ATC). Enable Address Translation Service (ATS) for devices that support it and send them invalidation requests whenever we invalidate the IOTLBs. ATC invalidation is allowed to take up to 90 seconds, according to the PCIe spec, so it is possible to get a SMMU command queue timeout during normal operations. However we expect implementations to complete invalidation in reasonable time. We only enable ATS for "trusted" devices, and currently rely on the pci_dev->untrusted bit. For ATS we have to trust that: (a) The device doesn't issue "translated" memory requests for addresses that weren't returned by the SMMU in a Translation Completion. In particular, if we give control of a device or device partition to a VM or userspace, software cannot program the device to access arbitrary "translated" addresses. (b) The device follows permissions granted by the SMMU in a Translation Completion. If the device requested read+write permission and only got read, then it doesn't write. (c) The device doesn't send Translated transactions for an address that was invalidated by an ATC invalidation. Note that the PCIe specification explicitly requires all of these, so we can assume that implementations will cleanly shield ATCs from software. All ATS translated requests still go through the SMMU, to walk the stream table and check that the device is actually allowed to send translated requests. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> --- drivers/iommu/arm-smmu-v3.c | 196 +++++++++++++++++++++++++++++++++++- 1 file changed, 191 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 3e7198ee9530..7819cd60d08b 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -29,6 +29,7 @@ #include <linux/of_iommu.h> #include <linux/of_platform.h> #include <linux/pci.h> +#include <linux/pci-ats.h> #include <linux/platform_device.h> #include <linux/amba/bus.h> @@ -86,6 +87,7 @@ #define IDR5_VAX_52_BIT 1 #define ARM_SMMU_CR0 0x20 +#define CR0_ATSCHK (1 << 4) #define CR0_CMDQEN (1 << 3) #define CR0_EVTQEN (1 << 2) #define CR0_PRIQEN (1 << 1) @@ -294,6 +296,7 @@ #define CMDQ_ERR_CERROR_NONE_IDX 0 #define CMDQ_ERR_CERROR_ILL_IDX 1 #define CMDQ_ERR_CERROR_ABT_IDX 2 +#define CMDQ_ERR_CERROR_ATC_INV_IDX 3 #define CMDQ_0_OP GENMASK_ULL(7, 0) #define CMDQ_0_SSV (1UL << 11) @@ -312,6 +315,12 @@ #define CMDQ_TLBI_1_VA_MASK GENMASK_ULL(63, 12) #define CMDQ_TLBI_1_IPA_MASK GENMASK_ULL(51, 12) +#define CMDQ_ATC_0_SSID GENMASK_ULL(31, 12) +#define CMDQ_ATC_0_SID GENMASK_ULL(63, 32) +#define CMDQ_ATC_0_GLOBAL (1UL << 9) +#define CMDQ_ATC_1_SIZE GENMASK_ULL(5, 0) +#define CMDQ_ATC_1_ADDR_MASK GENMASK_ULL(63, 12) + #define CMDQ_PRI_0_SSID GENMASK_ULL(31, 12) #define CMDQ_PRI_0_SID GENMASK_ULL(63, 32) #define CMDQ_PRI_1_GRPID GENMASK_ULL(8, 0) @@ -365,6 +374,11 @@ module_param_named(disable_bypass, disable_bypass, bool, S_IRUGO); MODULE_PARM_DESC(disable_bypass, "Disable bypass streams such that incoming transactions from devices that are not attached to an iommu domain will report an abort back to the device and will not be allowed to pass through the SMMU."); +static bool disable_ats_check; +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); +MODULE_PARM_DESC(disable_ats_check, + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); + enum pri_resp { PRI_RESP_DENY = 0, PRI_RESP_FAIL = 1, @@ -433,6 +447,16 @@ struct arm_smmu_cmdq_ent { u64 addr; } tlbi; + #define CMDQ_OP_ATC_INV 0x40 + #define ATC_INV_SIZE_ALL 52 + struct { + u32 sid; + u32 ssid; + u64 addr; + u8 size; + bool global; + } atc; + #define CMDQ_OP_PRI_RESP 0x41 struct { u32 sid; @@ -580,10 +604,12 @@ struct arm_smmu_device { /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; + struct device *dev; struct arm_smmu_domain *domain; struct list_head domain_head; u32 *sids; unsigned int num_sids; + bool ats_enabled :1; }; /* SMMU private data for an IOMMU domain */ @@ -813,6 +839,14 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) case CMDQ_OP_TLBI_S12_VMALL: cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_VMID, ent->tlbi.vmid); break; + case CMDQ_OP_ATC_INV: + cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_GLOBAL, ent->atc.global); + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SSID, ent->atc.ssid); + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SID, ent->atc.sid); + cmd[1] |= FIELD_PREP(CMDQ_ATC_1_SIZE, ent->atc.size); + cmd[1] |= ent->atc.addr & CMDQ_ATC_1_ADDR_MASK; + break; case CMDQ_OP_PRI_RESP: cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); cmd[0] |= FIELD_PREP(CMDQ_PRI_0_SSID, ent->pri.ssid); @@ -857,6 +891,7 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) [CMDQ_ERR_CERROR_NONE_IDX] = "No error", [CMDQ_ERR_CERROR_ILL_IDX] = "Illegal command", [CMDQ_ERR_CERROR_ABT_IDX] = "Abort on command fetch", + [CMDQ_ERR_CERROR_ATC_INV_IDX] = "ATC invalidate timeout", }; int i; @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) dev_err(smmu->dev, "retrying command fetch\n"); case CMDQ_ERR_CERROR_NONE_IDX: return; + case CMDQ_ERR_CERROR_ATC_INV_IDX: + /* + * ATC Invalidation Completion timeout. CONS is still pointing + * at the CMD_SYNC. Attempt to complete other pending commands + * by repeating the CMD_SYNC, though we might well end up back + * here since the ATC invalidation may still be pending. + */ + return; case CMDQ_ERR_CERROR_ILL_IDX: /* Fallthrough */ default: @@ -1174,9 +1217,6 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) | FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) | -#ifdef CONFIG_PCI_ATS - FIELD_PREP(STRTAB_STE_1_EATS, STRTAB_STE_1_EATS_TRANS) | -#endif FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1)); if (smmu->features & ARM_SMMU_FEAT_STALLS && @@ -1203,6 +1243,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS); } + if (master->ats_enabled) + dst[1] |= cpu_to_le64(FIELD_PREP(STRTAB_STE_1_EATS, + STRTAB_STE_1_EATS_TRANS)); + arm_smmu_sync_ste_for_sid(smmu, sid); dst[0] = cpu_to_le64(val); arm_smmu_sync_ste_for_sid(smmu, sid); @@ -1405,6 +1449,86 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) return IRQ_WAKE_THREAD; } +static void +arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, + struct arm_smmu_cmdq_ent *cmd) +{ + size_t log2_span; + size_t span_mask; + /* ATC invalidates are always on 4096 bytes pages */ + size_t inval_grain_shift = 12; + unsigned long page_start, page_end; + + *cmd = (struct arm_smmu_cmdq_ent) { + .opcode = CMDQ_OP_ATC_INV, + .substream_valid = !!ssid, + .atc.ssid = ssid, + }; + + if (!size) { + cmd->atc.size = ATC_INV_SIZE_ALL; + return; + } + + page_start = iova >> inval_grain_shift; + page_end = (iova + size - 1) >> inval_grain_shift; + + /* + * Find the smallest power of two that covers the range. Most + * significant differing bit between start and end address indicates the + * required span, ie. fls(start ^ end). For example: + * + * We want to invalidate pages [8; 11]. This is already the ideal range: + * x = 0b1000 ^ 0b1011 = 0b11 + * span = 1 << fls(x) = 4 + * + * To invalidate pages [7; 10], we need to invalidate [0; 15]: + * x = 0b0111 ^ 0b1010 = 0b1101 + * span = 1 << fls(x) = 16 + */ + log2_span = fls_long(page_start ^ page_end); + span_mask = (1ULL << log2_span) - 1; + + page_start &= ~span_mask; + + cmd->atc.addr = page_start << inval_grain_shift; + cmd->atc.size = log2_span; +} + +static void arm_smmu_atc_inv_master(struct arm_smmu_master *master, + struct arm_smmu_cmdq_ent *cmd) +{ + int i; + + if (!master->ats_enabled) + return; + + for (i = 0; i < master->num_sids; i++) { + cmd->atc.sid = master->sids[i]; + arm_smmu_cmdq_issue_cmd(master->smmu, cmd); + } + + arm_smmu_cmdq_issue_sync(master->smmu); +} + +static void arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + int ssid, unsigned long iova, size_t size) +{ + unsigned long flags; + struct arm_smmu_cmdq_ent cmd; + struct arm_smmu_master *master; + + if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) + return; + + arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); + + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) + arm_smmu_atc_inv_master(master, &cmd); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); +} + /* IO_PGTABLE API */ static void arm_smmu_tlb_sync(void *cookie) { @@ -1726,6 +1850,45 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) } } +static int arm_smmu_enable_ats(struct arm_smmu_master *master) +{ + int ret; + size_t stu; + struct pci_dev *pdev; + struct arm_smmu_device *smmu = master->smmu; + struct iommu_fwspec *fwspec = master->dev->iommu_fwspec; + + if (!(smmu->features & ARM_SMMU_FEAT_ATS) || !dev_is_pci(master->dev) || + (fwspec->flags & IOMMU_FWSPEC_PCI_NO_ATS) || pci_ats_disabled()) + return -ENOSYS; + + pdev = to_pci_dev(master->dev); + if (pdev->untrusted) + return -EPERM; + + /* Smallest Translation Unit: log2 of the smallest supported granule */ + stu = __ffs(smmu->pgsize_bitmap); + + ret = pci_enable_ats(pdev, stu); + if (ret) + return ret; + + master->ats_enabled = true; + dev_dbg(&pdev->dev, "enabled ATS (STU=%zu, QDEP=%d)\n", stu, + pci_ats_queue_depth(pdev)); + + return 0; +} + +static void arm_smmu_disable_ats(struct arm_smmu_master *master) +{ + if (!master->ats_enabled || !dev_is_pci(master->dev)) + return; + + pci_disable_ats(to_pci_dev(master->dev)); + master->ats_enabled = false; +} + static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; @@ -1740,6 +1903,9 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; arm_smmu_install_ste_for_dev(master); + + /* Disabling ATS invalidates all ATC entries */ + arm_smmu_disable_ats(master); } static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) @@ -1783,6 +1949,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) list_add(&master->domain_head, &smmu_domain->devices); spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) + arm_smmu_enable_ats(master); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) arm_smmu_write_ctx_desc(smmu, &smmu_domain->s1_cfg); @@ -1806,12 +1975,18 @@ static int arm_smmu_map(struct iommu_domain *domain, unsigned long iova, static size_t arm_smmu_unmap(struct iommu_domain *domain, unsigned long iova, size_t size) { - struct io_pgtable_ops *ops = to_smmu_domain(domain)->pgtbl_ops; + int ret; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops; if (!ops) return 0; - return ops->unmap(ops, iova, size); + ret = ops->unmap(ops, iova, size); + if (ret) + arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size); + + return ret; } static void arm_smmu_flush_iotlb_all(struct iommu_domain *domain) @@ -1898,6 +2073,7 @@ static int arm_smmu_add_device(struct device *dev) if (!master) return -ENOMEM; + master->dev = dev; master->smmu = smmu; master->sids = fwspec->ids; master->num_sids = fwspec->num_ids; @@ -2564,6 +2740,16 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass) } } + if (smmu->features & ARM_SMMU_FEAT_ATS && !disable_ats_check) { + enables |= CR0_ATSCHK; + ret = arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0, + ARM_SMMU_CR0ACK); + if (ret) { + dev_err(smmu->dev, "failed to enable ATS check\n"); + return ret; + } + } + ret = arm_smmu_setup_irqs(smmu); if (ret) { dev_err(smmu->dev, "failed to setup irqs\n"); -- 2.21.0 ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: lorenzo.pieralisi, eric.auger, zhongmiao, okaya, joro, rjw, linux-acpi, iommu, hanjun.guo, sudeep.holla, robin.murphy, linux-arm-kernel, lenb PCIe devices can implement their own TLB, named Address Translation Cache (ATC). Enable Address Translation Service (ATS) for devices that support it and send them invalidation requests whenever we invalidate the IOTLBs. ATC invalidation is allowed to take up to 90 seconds, according to the PCIe spec, so it is possible to get a SMMU command queue timeout during normal operations. However we expect implementations to complete invalidation in reasonable time. We only enable ATS for "trusted" devices, and currently rely on the pci_dev->untrusted bit. For ATS we have to trust that: (a) The device doesn't issue "translated" memory requests for addresses that weren't returned by the SMMU in a Translation Completion. In particular, if we give control of a device or device partition to a VM or userspace, software cannot program the device to access arbitrary "translated" addresses. (b) The device follows permissions granted by the SMMU in a Translation Completion. If the device requested read+write permission and only got read, then it doesn't write. (c) The device doesn't send Translated transactions for an address that was invalidated by an ATC invalidation. Note that the PCIe specification explicitly requires all of these, so we can assume that implementations will cleanly shield ATCs from software. All ATS translated requests still go through the SMMU, to walk the stream table and check that the device is actually allowed to send translated requests. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 196 +++++++++++++++++++++++++++++++++++- 1 file changed, 191 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 3e7198ee9530..7819cd60d08b 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -29,6 +29,7 @@ #include <linux/of_iommu.h> #include <linux/of_platform.h> #include <linux/pci.h> +#include <linux/pci-ats.h> #include <linux/platform_device.h> #include <linux/amba/bus.h> @@ -86,6 +87,7 @@ #define IDR5_VAX_52_BIT 1 #define ARM_SMMU_CR0 0x20 +#define CR0_ATSCHK (1 << 4) #define CR0_CMDQEN (1 << 3) #define CR0_EVTQEN (1 << 2) #define CR0_PRIQEN (1 << 1) @@ -294,6 +296,7 @@ #define CMDQ_ERR_CERROR_NONE_IDX 0 #define CMDQ_ERR_CERROR_ILL_IDX 1 #define CMDQ_ERR_CERROR_ABT_IDX 2 +#define CMDQ_ERR_CERROR_ATC_INV_IDX 3 #define CMDQ_0_OP GENMASK_ULL(7, 0) #define CMDQ_0_SSV (1UL << 11) @@ -312,6 +315,12 @@ #define CMDQ_TLBI_1_VA_MASK GENMASK_ULL(63, 12) #define CMDQ_TLBI_1_IPA_MASK GENMASK_ULL(51, 12) +#define CMDQ_ATC_0_SSID GENMASK_ULL(31, 12) +#define CMDQ_ATC_0_SID GENMASK_ULL(63, 32) +#define CMDQ_ATC_0_GLOBAL (1UL << 9) +#define CMDQ_ATC_1_SIZE GENMASK_ULL(5, 0) +#define CMDQ_ATC_1_ADDR_MASK GENMASK_ULL(63, 12) + #define CMDQ_PRI_0_SSID GENMASK_ULL(31, 12) #define CMDQ_PRI_0_SID GENMASK_ULL(63, 32) #define CMDQ_PRI_1_GRPID GENMASK_ULL(8, 0) @@ -365,6 +374,11 @@ module_param_named(disable_bypass, disable_bypass, bool, S_IRUGO); MODULE_PARM_DESC(disable_bypass, "Disable bypass streams such that incoming transactions from devices that are not attached to an iommu domain will report an abort back to the device and will not be allowed to pass through the SMMU."); +static bool disable_ats_check; +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); +MODULE_PARM_DESC(disable_ats_check, + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); + enum pri_resp { PRI_RESP_DENY = 0, PRI_RESP_FAIL = 1, @@ -433,6 +447,16 @@ struct arm_smmu_cmdq_ent { u64 addr; } tlbi; + #define CMDQ_OP_ATC_INV 0x40 + #define ATC_INV_SIZE_ALL 52 + struct { + u32 sid; + u32 ssid; + u64 addr; + u8 size; + bool global; + } atc; + #define CMDQ_OP_PRI_RESP 0x41 struct { u32 sid; @@ -580,10 +604,12 @@ struct arm_smmu_device { /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; + struct device *dev; struct arm_smmu_domain *domain; struct list_head domain_head; u32 *sids; unsigned int num_sids; + bool ats_enabled :1; }; /* SMMU private data for an IOMMU domain */ @@ -813,6 +839,14 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) case CMDQ_OP_TLBI_S12_VMALL: cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_VMID, ent->tlbi.vmid); break; + case CMDQ_OP_ATC_INV: + cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_GLOBAL, ent->atc.global); + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SSID, ent->atc.ssid); + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SID, ent->atc.sid); + cmd[1] |= FIELD_PREP(CMDQ_ATC_1_SIZE, ent->atc.size); + cmd[1] |= ent->atc.addr & CMDQ_ATC_1_ADDR_MASK; + break; case CMDQ_OP_PRI_RESP: cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); cmd[0] |= FIELD_PREP(CMDQ_PRI_0_SSID, ent->pri.ssid); @@ -857,6 +891,7 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) [CMDQ_ERR_CERROR_NONE_IDX] = "No error", [CMDQ_ERR_CERROR_ILL_IDX] = "Illegal command", [CMDQ_ERR_CERROR_ABT_IDX] = "Abort on command fetch", + [CMDQ_ERR_CERROR_ATC_INV_IDX] = "ATC invalidate timeout", }; int i; @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) dev_err(smmu->dev, "retrying command fetch\n"); case CMDQ_ERR_CERROR_NONE_IDX: return; + case CMDQ_ERR_CERROR_ATC_INV_IDX: + /* + * ATC Invalidation Completion timeout. CONS is still pointing + * at the CMD_SYNC. Attempt to complete other pending commands + * by repeating the CMD_SYNC, though we might well end up back + * here since the ATC invalidation may still be pending. + */ + return; case CMDQ_ERR_CERROR_ILL_IDX: /* Fallthrough */ default: @@ -1174,9 +1217,6 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) | FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) | -#ifdef CONFIG_PCI_ATS - FIELD_PREP(STRTAB_STE_1_EATS, STRTAB_STE_1_EATS_TRANS) | -#endif FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1)); if (smmu->features & ARM_SMMU_FEAT_STALLS && @@ -1203,6 +1243,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS); } + if (master->ats_enabled) + dst[1] |= cpu_to_le64(FIELD_PREP(STRTAB_STE_1_EATS, + STRTAB_STE_1_EATS_TRANS)); + arm_smmu_sync_ste_for_sid(smmu, sid); dst[0] = cpu_to_le64(val); arm_smmu_sync_ste_for_sid(smmu, sid); @@ -1405,6 +1449,86 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) return IRQ_WAKE_THREAD; } +static void +arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, + struct arm_smmu_cmdq_ent *cmd) +{ + size_t log2_span; + size_t span_mask; + /* ATC invalidates are always on 4096 bytes pages */ + size_t inval_grain_shift = 12; + unsigned long page_start, page_end; + + *cmd = (struct arm_smmu_cmdq_ent) { + .opcode = CMDQ_OP_ATC_INV, + .substream_valid = !!ssid, + .atc.ssid = ssid, + }; + + if (!size) { + cmd->atc.size = ATC_INV_SIZE_ALL; + return; + } + + page_start = iova >> inval_grain_shift; + page_end = (iova + size - 1) >> inval_grain_shift; + + /* + * Find the smallest power of two that covers the range. Most + * significant differing bit between start and end address indicates the + * required span, ie. fls(start ^ end). For example: + * + * We want to invalidate pages [8; 11]. This is already the ideal range: + * x = 0b1000 ^ 0b1011 = 0b11 + * span = 1 << fls(x) = 4 + * + * To invalidate pages [7; 10], we need to invalidate [0; 15]: + * x = 0b0111 ^ 0b1010 = 0b1101 + * span = 1 << fls(x) = 16 + */ + log2_span = fls_long(page_start ^ page_end); + span_mask = (1ULL << log2_span) - 1; + + page_start &= ~span_mask; + + cmd->atc.addr = page_start << inval_grain_shift; + cmd->atc.size = log2_span; +} + +static void arm_smmu_atc_inv_master(struct arm_smmu_master *master, + struct arm_smmu_cmdq_ent *cmd) +{ + int i; + + if (!master->ats_enabled) + return; + + for (i = 0; i < master->num_sids; i++) { + cmd->atc.sid = master->sids[i]; + arm_smmu_cmdq_issue_cmd(master->smmu, cmd); + } + + arm_smmu_cmdq_issue_sync(master->smmu); +} + +static void arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + int ssid, unsigned long iova, size_t size) +{ + unsigned long flags; + struct arm_smmu_cmdq_ent cmd; + struct arm_smmu_master *master; + + if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) + return; + + arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); + + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) + arm_smmu_atc_inv_master(master, &cmd); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); +} + /* IO_PGTABLE API */ static void arm_smmu_tlb_sync(void *cookie) { @@ -1726,6 +1850,45 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) } } +static int arm_smmu_enable_ats(struct arm_smmu_master *master) +{ + int ret; + size_t stu; + struct pci_dev *pdev; + struct arm_smmu_device *smmu = master->smmu; + struct iommu_fwspec *fwspec = master->dev->iommu_fwspec; + + if (!(smmu->features & ARM_SMMU_FEAT_ATS) || !dev_is_pci(master->dev) || + (fwspec->flags & IOMMU_FWSPEC_PCI_NO_ATS) || pci_ats_disabled()) + return -ENOSYS; + + pdev = to_pci_dev(master->dev); + if (pdev->untrusted) + return -EPERM; + + /* Smallest Translation Unit: log2 of the smallest supported granule */ + stu = __ffs(smmu->pgsize_bitmap); + + ret = pci_enable_ats(pdev, stu); + if (ret) + return ret; + + master->ats_enabled = true; + dev_dbg(&pdev->dev, "enabled ATS (STU=%zu, QDEP=%d)\n", stu, + pci_ats_queue_depth(pdev)); + + return 0; +} + +static void arm_smmu_disable_ats(struct arm_smmu_master *master) +{ + if (!master->ats_enabled || !dev_is_pci(master->dev)) + return; + + pci_disable_ats(to_pci_dev(master->dev)); + master->ats_enabled = false; +} + static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; @@ -1740,6 +1903,9 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; arm_smmu_install_ste_for_dev(master); + + /* Disabling ATS invalidates all ATC entries */ + arm_smmu_disable_ats(master); } static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) @@ -1783,6 +1949,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) list_add(&master->domain_head, &smmu_domain->devices); spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) + arm_smmu_enable_ats(master); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) arm_smmu_write_ctx_desc(smmu, &smmu_domain->s1_cfg); @@ -1806,12 +1975,18 @@ static int arm_smmu_map(struct iommu_domain *domain, unsigned long iova, static size_t arm_smmu_unmap(struct iommu_domain *domain, unsigned long iova, size_t size) { - struct io_pgtable_ops *ops = to_smmu_domain(domain)->pgtbl_ops; + int ret; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops; if (!ops) return 0; - return ops->unmap(ops, iova, size); + ret = ops->unmap(ops, iova, size); + if (ret) + arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size); + + return ret; } static void arm_smmu_flush_iotlb_all(struct iommu_domain *domain) @@ -1898,6 +2073,7 @@ static int arm_smmu_add_device(struct device *dev) if (!master) return -ENOMEM; + master->dev = dev; master->smmu = smmu; master->sids = fwspec->ids; master->num_sids = fwspec->num_ids; @@ -2564,6 +2740,16 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass) } } + if (smmu->features & ARM_SMMU_FEAT_ATS && !disable_ats_check) { + enables |= CR0_ATSCHK; + ret = arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0, + ARM_SMMU_CR0ACK); + if (ret) { + dev_err(smmu->dev, "failed to enable ATS check\n"); + return ret; + } + } + ret = arm_smmu_setup_irqs(smmu); if (ret) { dev_err(smmu->dev, "failed to setup irqs\n"); -- 2.21.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb PCIe devices can implement their own TLB, named Address Translation Cache (ATC). Enable Address Translation Service (ATS) for devices that support it and send them invalidation requests whenever we invalidate the IOTLBs. ATC invalidation is allowed to take up to 90 seconds, according to the PCIe spec, so it is possible to get a SMMU command queue timeout during normal operations. However we expect implementations to complete invalidation in reasonable time. We only enable ATS for "trusted" devices, and currently rely on the pci_dev->untrusted bit. For ATS we have to trust that: (a) The device doesn't issue "translated" memory requests for addresses that weren't returned by the SMMU in a Translation Completion. In particular, if we give control of a device or device partition to a VM or userspace, software cannot program the device to access arbitrary "translated" addresses. (b) The device follows permissions granted by the SMMU in a Translation Completion. If the device requested read+write permission and only got read, then it doesn't write. (c) The device doesn't send Translated transactions for an address that was invalidated by an ATC invalidation. Note that the PCIe specification explicitly requires all of these, so we can assume that implementations will cleanly shield ATCs from software. All ATS translated requests still go through the SMMU, to walk the stream table and check that the device is actually allowed to send translated requests. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 196 +++++++++++++++++++++++++++++++++++- 1 file changed, 191 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 3e7198ee9530..7819cd60d08b 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -29,6 +29,7 @@ #include <linux/of_iommu.h> #include <linux/of_platform.h> #include <linux/pci.h> +#include <linux/pci-ats.h> #include <linux/platform_device.h> #include <linux/amba/bus.h> @@ -86,6 +87,7 @@ #define IDR5_VAX_52_BIT 1 #define ARM_SMMU_CR0 0x20 +#define CR0_ATSCHK (1 << 4) #define CR0_CMDQEN (1 << 3) #define CR0_EVTQEN (1 << 2) #define CR0_PRIQEN (1 << 1) @@ -294,6 +296,7 @@ #define CMDQ_ERR_CERROR_NONE_IDX 0 #define CMDQ_ERR_CERROR_ILL_IDX 1 #define CMDQ_ERR_CERROR_ABT_IDX 2 +#define CMDQ_ERR_CERROR_ATC_INV_IDX 3 #define CMDQ_0_OP GENMASK_ULL(7, 0) #define CMDQ_0_SSV (1UL << 11) @@ -312,6 +315,12 @@ #define CMDQ_TLBI_1_VA_MASK GENMASK_ULL(63, 12) #define CMDQ_TLBI_1_IPA_MASK GENMASK_ULL(51, 12) +#define CMDQ_ATC_0_SSID GENMASK_ULL(31, 12) +#define CMDQ_ATC_0_SID GENMASK_ULL(63, 32) +#define CMDQ_ATC_0_GLOBAL (1UL << 9) +#define CMDQ_ATC_1_SIZE GENMASK_ULL(5, 0) +#define CMDQ_ATC_1_ADDR_MASK GENMASK_ULL(63, 12) + #define CMDQ_PRI_0_SSID GENMASK_ULL(31, 12) #define CMDQ_PRI_0_SID GENMASK_ULL(63, 32) #define CMDQ_PRI_1_GRPID GENMASK_ULL(8, 0) @@ -365,6 +374,11 @@ module_param_named(disable_bypass, disable_bypass, bool, S_IRUGO); MODULE_PARM_DESC(disable_bypass, "Disable bypass streams such that incoming transactions from devices that are not attached to an iommu domain will report an abort back to the device and will not be allowed to pass through the SMMU."); +static bool disable_ats_check; +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); +MODULE_PARM_DESC(disable_ats_check, + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); + enum pri_resp { PRI_RESP_DENY = 0, PRI_RESP_FAIL = 1, @@ -433,6 +447,16 @@ struct arm_smmu_cmdq_ent { u64 addr; } tlbi; + #define CMDQ_OP_ATC_INV 0x40 + #define ATC_INV_SIZE_ALL 52 + struct { + u32 sid; + u32 ssid; + u64 addr; + u8 size; + bool global; + } atc; + #define CMDQ_OP_PRI_RESP 0x41 struct { u32 sid; @@ -580,10 +604,12 @@ struct arm_smmu_device { /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; + struct device *dev; struct arm_smmu_domain *domain; struct list_head domain_head; u32 *sids; unsigned int num_sids; + bool ats_enabled :1; }; /* SMMU private data for an IOMMU domain */ @@ -813,6 +839,14 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) case CMDQ_OP_TLBI_S12_VMALL: cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_VMID, ent->tlbi.vmid); break; + case CMDQ_OP_ATC_INV: + cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_GLOBAL, ent->atc.global); + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SSID, ent->atc.ssid); + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SID, ent->atc.sid); + cmd[1] |= FIELD_PREP(CMDQ_ATC_1_SIZE, ent->atc.size); + cmd[1] |= ent->atc.addr & CMDQ_ATC_1_ADDR_MASK; + break; case CMDQ_OP_PRI_RESP: cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); cmd[0] |= FIELD_PREP(CMDQ_PRI_0_SSID, ent->pri.ssid); @@ -857,6 +891,7 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) [CMDQ_ERR_CERROR_NONE_IDX] = "No error", [CMDQ_ERR_CERROR_ILL_IDX] = "Illegal command", [CMDQ_ERR_CERROR_ABT_IDX] = "Abort on command fetch", + [CMDQ_ERR_CERROR_ATC_INV_IDX] = "ATC invalidate timeout", }; int i; @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) dev_err(smmu->dev, "retrying command fetch\n"); case CMDQ_ERR_CERROR_NONE_IDX: return; + case CMDQ_ERR_CERROR_ATC_INV_IDX: + /* + * ATC Invalidation Completion timeout. CONS is still pointing + * at the CMD_SYNC. Attempt to complete other pending commands + * by repeating the CMD_SYNC, though we might well end up back + * here since the ATC invalidation may still be pending. + */ + return; case CMDQ_ERR_CERROR_ILL_IDX: /* Fallthrough */ default: @@ -1174,9 +1217,6 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) | FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) | -#ifdef CONFIG_PCI_ATS - FIELD_PREP(STRTAB_STE_1_EATS, STRTAB_STE_1_EATS_TRANS) | -#endif FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1)); if (smmu->features & ARM_SMMU_FEAT_STALLS && @@ -1203,6 +1243,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS); } + if (master->ats_enabled) + dst[1] |= cpu_to_le64(FIELD_PREP(STRTAB_STE_1_EATS, + STRTAB_STE_1_EATS_TRANS)); + arm_smmu_sync_ste_for_sid(smmu, sid); dst[0] = cpu_to_le64(val); arm_smmu_sync_ste_for_sid(smmu, sid); @@ -1405,6 +1449,86 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) return IRQ_WAKE_THREAD; } +static void +arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, + struct arm_smmu_cmdq_ent *cmd) +{ + size_t log2_span; + size_t span_mask; + /* ATC invalidates are always on 4096 bytes pages */ + size_t inval_grain_shift = 12; + unsigned long page_start, page_end; + + *cmd = (struct arm_smmu_cmdq_ent) { + .opcode = CMDQ_OP_ATC_INV, + .substream_valid = !!ssid, + .atc.ssid = ssid, + }; + + if (!size) { + cmd->atc.size = ATC_INV_SIZE_ALL; + return; + } + + page_start = iova >> inval_grain_shift; + page_end = (iova + size - 1) >> inval_grain_shift; + + /* + * Find the smallest power of two that covers the range. Most + * significant differing bit between start and end address indicates the + * required span, ie. fls(start ^ end). For example: + * + * We want to invalidate pages [8; 11]. This is already the ideal range: + * x = 0b1000 ^ 0b1011 = 0b11 + * span = 1 << fls(x) = 4 + * + * To invalidate pages [7; 10], we need to invalidate [0; 15]: + * x = 0b0111 ^ 0b1010 = 0b1101 + * span = 1 << fls(x) = 16 + */ + log2_span = fls_long(page_start ^ page_end); + span_mask = (1ULL << log2_span) - 1; + + page_start &= ~span_mask; + + cmd->atc.addr = page_start << inval_grain_shift; + cmd->atc.size = log2_span; +} + +static void arm_smmu_atc_inv_master(struct arm_smmu_master *master, + struct arm_smmu_cmdq_ent *cmd) +{ + int i; + + if (!master->ats_enabled) + return; + + for (i = 0; i < master->num_sids; i++) { + cmd->atc.sid = master->sids[i]; + arm_smmu_cmdq_issue_cmd(master->smmu, cmd); + } + + arm_smmu_cmdq_issue_sync(master->smmu); +} + +static void arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + int ssid, unsigned long iova, size_t size) +{ + unsigned long flags; + struct arm_smmu_cmdq_ent cmd; + struct arm_smmu_master *master; + + if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) + return; + + arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); + + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) + arm_smmu_atc_inv_master(master, &cmd); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); +} + /* IO_PGTABLE API */ static void arm_smmu_tlb_sync(void *cookie) { @@ -1726,6 +1850,45 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) } } +static int arm_smmu_enable_ats(struct arm_smmu_master *master) +{ + int ret; + size_t stu; + struct pci_dev *pdev; + struct arm_smmu_device *smmu = master->smmu; + struct iommu_fwspec *fwspec = master->dev->iommu_fwspec; + + if (!(smmu->features & ARM_SMMU_FEAT_ATS) || !dev_is_pci(master->dev) || + (fwspec->flags & IOMMU_FWSPEC_PCI_NO_ATS) || pci_ats_disabled()) + return -ENOSYS; + + pdev = to_pci_dev(master->dev); + if (pdev->untrusted) + return -EPERM; + + /* Smallest Translation Unit: log2 of the smallest supported granule */ + stu = __ffs(smmu->pgsize_bitmap); + + ret = pci_enable_ats(pdev, stu); + if (ret) + return ret; + + master->ats_enabled = true; + dev_dbg(&pdev->dev, "enabled ATS (STU=%zu, QDEP=%d)\n", stu, + pci_ats_queue_depth(pdev)); + + return 0; +} + +static void arm_smmu_disable_ats(struct arm_smmu_master *master) +{ + if (!master->ats_enabled || !dev_is_pci(master->dev)) + return; + + pci_disable_ats(to_pci_dev(master->dev)); + master->ats_enabled = false; +} + static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; @@ -1740,6 +1903,9 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; arm_smmu_install_ste_for_dev(master); + + /* Disabling ATS invalidates all ATC entries */ + arm_smmu_disable_ats(master); } static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) @@ -1783,6 +1949,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) list_add(&master->domain_head, &smmu_domain->devices); spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) + arm_smmu_enable_ats(master); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) arm_smmu_write_ctx_desc(smmu, &smmu_domain->s1_cfg); @@ -1806,12 +1975,18 @@ static int arm_smmu_map(struct iommu_domain *domain, unsigned long iova, static size_t arm_smmu_unmap(struct iommu_domain *domain, unsigned long iova, size_t size) { - struct io_pgtable_ops *ops = to_smmu_domain(domain)->pgtbl_ops; + int ret; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops; if (!ops) return 0; - return ops->unmap(ops, iova, size); + ret = ops->unmap(ops, iova, size); + if (ret) + arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size); + + return ret; } static void arm_smmu_flush_iotlb_all(struct iommu_domain *domain) @@ -1898,6 +2073,7 @@ static int arm_smmu_add_device(struct device *dev) if (!master) return -ENOMEM; + master->dev = dev; master->smmu = smmu; master->sids = fwspec->ids; master->num_sids = fwspec->num_ids; @@ -2564,6 +2740,16 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass) } } + if (smmu->features & ARM_SMMU_FEAT_ATS && !disable_ats_check) { + enables |= CR0_ATSCHK; + ret = arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0, + ARM_SMMU_CR0ACK); + if (ret) { + dev_err(smmu->dev, "failed to enable ATS check\n"); + return ret; + } + } + ret = arm_smmu_setup_irqs(smmu); if (ret) { dev_err(smmu->dev, "failed to setup irqs\n"); -- 2.21.0 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: iommu, linux-arm-kernel, linux-acpi, robin.murphy, joro, hanjun.guo, lorenzo.pieralisi, sudeep.holla, rjw, lenb, okaya, zhongmiao, eric.auger PCIe devices can implement their own TLB, named Address Translation Cache (ATC). Enable Address Translation Service (ATS) for devices that support it and send them invalidation requests whenever we invalidate the IOTLBs. ATC invalidation is allowed to take up to 90 seconds, according to the PCIe spec, so it is possible to get a SMMU command queue timeout during normal operations. However we expect implementations to complete invalidation in reasonable time. We only enable ATS for "trusted" devices, and currently rely on the pci_dev->untrusted bit. For ATS we have to trust that: (a) The device doesn't issue "translated" memory requests for addresses that weren't returned by the SMMU in a Translation Completion. In particular, if we give control of a device or device partition to a VM or userspace, software cannot program the device to access arbitrary "translated" addresses. (b) The device follows permissions granted by the SMMU in a Translation Completion. If the device requested read+write permission and only got read, then it doesn't write. (c) The device doesn't send Translated transactions for an address that was invalidated by an ATC invalidation. Note that the PCIe specification explicitly requires all of these, so we can assume that implementations will cleanly shield ATCs from software. All ATS translated requests still go through the SMMU, to walk the stream table and check that the device is actually allowed to send translated requests. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 196 +++++++++++++++++++++++++++++++++++- 1 file changed, 191 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 3e7198ee9530..7819cd60d08b 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -29,6 +29,7 @@ #include <linux/of_iommu.h> #include <linux/of_platform.h> #include <linux/pci.h> +#include <linux/pci-ats.h> #include <linux/platform_device.h> #include <linux/amba/bus.h> @@ -86,6 +87,7 @@ #define IDR5_VAX_52_BIT 1 #define ARM_SMMU_CR0 0x20 +#define CR0_ATSCHK (1 << 4) #define CR0_CMDQEN (1 << 3) #define CR0_EVTQEN (1 << 2) #define CR0_PRIQEN (1 << 1) @@ -294,6 +296,7 @@ #define CMDQ_ERR_CERROR_NONE_IDX 0 #define CMDQ_ERR_CERROR_ILL_IDX 1 #define CMDQ_ERR_CERROR_ABT_IDX 2 +#define CMDQ_ERR_CERROR_ATC_INV_IDX 3 #define CMDQ_0_OP GENMASK_ULL(7, 0) #define CMDQ_0_SSV (1UL << 11) @@ -312,6 +315,12 @@ #define CMDQ_TLBI_1_VA_MASK GENMASK_ULL(63, 12) #define CMDQ_TLBI_1_IPA_MASK GENMASK_ULL(51, 12) +#define CMDQ_ATC_0_SSID GENMASK_ULL(31, 12) +#define CMDQ_ATC_0_SID GENMASK_ULL(63, 32) +#define CMDQ_ATC_0_GLOBAL (1UL << 9) +#define CMDQ_ATC_1_SIZE GENMASK_ULL(5, 0) +#define CMDQ_ATC_1_ADDR_MASK GENMASK_ULL(63, 12) + #define CMDQ_PRI_0_SSID GENMASK_ULL(31, 12) #define CMDQ_PRI_0_SID GENMASK_ULL(63, 32) #define CMDQ_PRI_1_GRPID GENMASK_ULL(8, 0) @@ -365,6 +374,11 @@ module_param_named(disable_bypass, disable_bypass, bool, S_IRUGO); MODULE_PARM_DESC(disable_bypass, "Disable bypass streams such that incoming transactions from devices that are not attached to an iommu domain will report an abort back to the device and will not be allowed to pass through the SMMU."); +static bool disable_ats_check; +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); +MODULE_PARM_DESC(disable_ats_check, + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); + enum pri_resp { PRI_RESP_DENY = 0, PRI_RESP_FAIL = 1, @@ -433,6 +447,16 @@ struct arm_smmu_cmdq_ent { u64 addr; } tlbi; + #define CMDQ_OP_ATC_INV 0x40 + #define ATC_INV_SIZE_ALL 52 + struct { + u32 sid; + u32 ssid; + u64 addr; + u8 size; + bool global; + } atc; + #define CMDQ_OP_PRI_RESP 0x41 struct { u32 sid; @@ -580,10 +604,12 @@ struct arm_smmu_device { /* SMMU private data for each master */ struct arm_smmu_master { struct arm_smmu_device *smmu; + struct device *dev; struct arm_smmu_domain *domain; struct list_head domain_head; u32 *sids; unsigned int num_sids; + bool ats_enabled :1; }; /* SMMU private data for an IOMMU domain */ @@ -813,6 +839,14 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) case CMDQ_OP_TLBI_S12_VMALL: cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_VMID, ent->tlbi.vmid); break; + case CMDQ_OP_ATC_INV: + cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_GLOBAL, ent->atc.global); + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SSID, ent->atc.ssid); + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SID, ent->atc.sid); + cmd[1] |= FIELD_PREP(CMDQ_ATC_1_SIZE, ent->atc.size); + cmd[1] |= ent->atc.addr & CMDQ_ATC_1_ADDR_MASK; + break; case CMDQ_OP_PRI_RESP: cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); cmd[0] |= FIELD_PREP(CMDQ_PRI_0_SSID, ent->pri.ssid); @@ -857,6 +891,7 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) [CMDQ_ERR_CERROR_NONE_IDX] = "No error", [CMDQ_ERR_CERROR_ILL_IDX] = "Illegal command", [CMDQ_ERR_CERROR_ABT_IDX] = "Abort on command fetch", + [CMDQ_ERR_CERROR_ATC_INV_IDX] = "ATC invalidate timeout", }; int i; @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) dev_err(smmu->dev, "retrying command fetch\n"); case CMDQ_ERR_CERROR_NONE_IDX: return; + case CMDQ_ERR_CERROR_ATC_INV_IDX: + /* + * ATC Invalidation Completion timeout. CONS is still pointing + * at the CMD_SYNC. Attempt to complete other pending commands + * by repeating the CMD_SYNC, though we might well end up back + * here since the ATC invalidation may still be pending. + */ + return; case CMDQ_ERR_CERROR_ILL_IDX: /* Fallthrough */ default: @@ -1174,9 +1217,6 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) | FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) | -#ifdef CONFIG_PCI_ATS - FIELD_PREP(STRTAB_STE_1_EATS, STRTAB_STE_1_EATS_TRANS) | -#endif FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1)); if (smmu->features & ARM_SMMU_FEAT_STALLS && @@ -1203,6 +1243,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS); } + if (master->ats_enabled) + dst[1] |= cpu_to_le64(FIELD_PREP(STRTAB_STE_1_EATS, + STRTAB_STE_1_EATS_TRANS)); + arm_smmu_sync_ste_for_sid(smmu, sid); dst[0] = cpu_to_le64(val); arm_smmu_sync_ste_for_sid(smmu, sid); @@ -1405,6 +1449,86 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) return IRQ_WAKE_THREAD; } +static void +arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, + struct arm_smmu_cmdq_ent *cmd) +{ + size_t log2_span; + size_t span_mask; + /* ATC invalidates are always on 4096 bytes pages */ + size_t inval_grain_shift = 12; + unsigned long page_start, page_end; + + *cmd = (struct arm_smmu_cmdq_ent) { + .opcode = CMDQ_OP_ATC_INV, + .substream_valid = !!ssid, + .atc.ssid = ssid, + }; + + if (!size) { + cmd->atc.size = ATC_INV_SIZE_ALL; + return; + } + + page_start = iova >> inval_grain_shift; + page_end = (iova + size - 1) >> inval_grain_shift; + + /* + * Find the smallest power of two that covers the range. Most + * significant differing bit between start and end address indicates the + * required span, ie. fls(start ^ end). For example: + * + * We want to invalidate pages [8; 11]. This is already the ideal range: + * x = 0b1000 ^ 0b1011 = 0b11 + * span = 1 << fls(x) = 4 + * + * To invalidate pages [7; 10], we need to invalidate [0; 15]: + * x = 0b0111 ^ 0b1010 = 0b1101 + * span = 1 << fls(x) = 16 + */ + log2_span = fls_long(page_start ^ page_end); + span_mask = (1ULL << log2_span) - 1; + + page_start &= ~span_mask; + + cmd->atc.addr = page_start << inval_grain_shift; + cmd->atc.size = log2_span; +} + +static void arm_smmu_atc_inv_master(struct arm_smmu_master *master, + struct arm_smmu_cmdq_ent *cmd) +{ + int i; + + if (!master->ats_enabled) + return; + + for (i = 0; i < master->num_sids; i++) { + cmd->atc.sid = master->sids[i]; + arm_smmu_cmdq_issue_cmd(master->smmu, cmd); + } + + arm_smmu_cmdq_issue_sync(master->smmu); +} + +static void arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, + int ssid, unsigned long iova, size_t size) +{ + unsigned long flags; + struct arm_smmu_cmdq_ent cmd; + struct arm_smmu_master *master; + + if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) + return; + + arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); + + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) + arm_smmu_atc_inv_master(master, &cmd); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); +} + /* IO_PGTABLE API */ static void arm_smmu_tlb_sync(void *cookie) { @@ -1726,6 +1850,45 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) } } +static int arm_smmu_enable_ats(struct arm_smmu_master *master) +{ + int ret; + size_t stu; + struct pci_dev *pdev; + struct arm_smmu_device *smmu = master->smmu; + struct iommu_fwspec *fwspec = master->dev->iommu_fwspec; + + if (!(smmu->features & ARM_SMMU_FEAT_ATS) || !dev_is_pci(master->dev) || + (fwspec->flags & IOMMU_FWSPEC_PCI_NO_ATS) || pci_ats_disabled()) + return -ENOSYS; + + pdev = to_pci_dev(master->dev); + if (pdev->untrusted) + return -EPERM; + + /* Smallest Translation Unit: log2 of the smallest supported granule */ + stu = __ffs(smmu->pgsize_bitmap); + + ret = pci_enable_ats(pdev, stu); + if (ret) + return ret; + + master->ats_enabled = true; + dev_dbg(&pdev->dev, "enabled ATS (STU=%zu, QDEP=%d)\n", stu, + pci_ats_queue_depth(pdev)); + + return 0; +} + +static void arm_smmu_disable_ats(struct arm_smmu_master *master) +{ + if (!master->ats_enabled || !dev_is_pci(master->dev)) + return; + + pci_disable_ats(to_pci_dev(master->dev)); + master->ats_enabled = false; +} + static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; @@ -1740,6 +1903,9 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) master->domain = NULL; arm_smmu_install_ste_for_dev(master); + + /* Disabling ATS invalidates all ATC entries */ + arm_smmu_disable_ats(master); } static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) @@ -1783,6 +1949,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) list_add(&master->domain_head, &smmu_domain->devices); spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) + arm_smmu_enable_ats(master); + if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) arm_smmu_write_ctx_desc(smmu, &smmu_domain->s1_cfg); @@ -1806,12 +1975,18 @@ static int arm_smmu_map(struct iommu_domain *domain, unsigned long iova, static size_t arm_smmu_unmap(struct iommu_domain *domain, unsigned long iova, size_t size) { - struct io_pgtable_ops *ops = to_smmu_domain(domain)->pgtbl_ops; + int ret; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops; if (!ops) return 0; - return ops->unmap(ops, iova, size); + ret = ops->unmap(ops, iova, size); + if (ret) + arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size); + + return ret; } static void arm_smmu_flush_iotlb_all(struct iommu_domain *domain) @@ -1898,6 +2073,7 @@ static int arm_smmu_add_device(struct device *dev) if (!master) return -ENOMEM; + master->dev = dev; master->smmu = smmu; master->sids = fwspec->ids; master->num_sids = fwspec->num_ids; @@ -2564,6 +2740,16 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass) } } + if (smmu->features & ARM_SMMU_FEAT_ATS && !disable_ats_check) { + enables |= CR0_ATSCHK; + ret = arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0, + ARM_SMMU_CR0ACK); + if (ret) { + dev_err(smmu->dev, "failed to enable ATS check\n"); + return ret; + } + } + ret = arm_smmu_setup_irqs(smmu); if (ret) { dev_err(smmu->dev, "failed to setup irqs\n"); -- 2.21.0 ^ permalink raw reply related [flat|nested] 64+ messages in thread
[parent not found: <20190409165245.26500-7-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>]
* Re: [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-15 13:21 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-15 13:21 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, sudeep.holla-5wv7dgnIgG8, robin.murphy-5wv7dgnIgG8, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A On Tue, Apr 09, 2019 at 05:52:44PM +0100, Jean-Philippe Brucker wrote: > PCIe devices can implement their own TLB, named Address Translation Cache > (ATC). Enable Address Translation Service (ATS) for devices that support > it and send them invalidation requests whenever we invalidate the IOTLBs. > > ATC invalidation is allowed to take up to 90 seconds, according to the > PCIe spec, so it is possible to get a SMMU command queue timeout during > normal operations. However we expect implementations to complete > invalidation in reasonable time. > > We only enable ATS for "trusted" devices, and currently rely on the > pci_dev->untrusted bit. For ATS we have to trust that: AFAICT, devicetree has no way to describe a device as untrusted, so everything will be trusted by default on those systems. Is that right? > (a) The device doesn't issue "translated" memory requests for addresses > that weren't returned by the SMMU in a Translation Completion. In > particular, if we give control of a device or device partition to a VM > or userspace, software cannot program the device to access arbitrary > "translated" addresses. Any plans to look at split-stage ATS later on? I think that would allow us to pass untrusted devices through to a VM behind S1 ATS. > (b) The device follows permissions granted by the SMMU in a Translation > Completion. If the device requested read+write permission and only > got read, then it doesn't write. Guessing we just ignore execute permissions, or do we need read implies exec? > (c) The device doesn't send Translated transactions for an address that > was invalidated by an ATC invalidation. > > Note that the PCIe specification explicitly requires all of these, so we > can assume that implementations will cleanly shield ATCs from software. > > All ATS translated requests still go through the SMMU, to walk the stream > table and check that the device is actually allowed to send translated > requests. > > Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> > --- > drivers/iommu/arm-smmu-v3.c | 196 +++++++++++++++++++++++++++++++++++- > 1 file changed, 191 insertions(+), 5 deletions(-) > > diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c > index 3e7198ee9530..7819cd60d08b 100644 > --- a/drivers/iommu/arm-smmu-v3.c > +++ b/drivers/iommu/arm-smmu-v3.c > @@ -29,6 +29,7 @@ > #include <linux/of_iommu.h> > #include <linux/of_platform.h> > #include <linux/pci.h> > +#include <linux/pci-ats.h> > #include <linux/platform_device.h> > > #include <linux/amba/bus.h> > @@ -86,6 +87,7 @@ > #define IDR5_VAX_52_BIT 1 > > #define ARM_SMMU_CR0 0x20 > +#define CR0_ATSCHK (1 << 4) > #define CR0_CMDQEN (1 << 3) > #define CR0_EVTQEN (1 << 2) > #define CR0_PRIQEN (1 << 1) > @@ -294,6 +296,7 @@ > #define CMDQ_ERR_CERROR_NONE_IDX 0 > #define CMDQ_ERR_CERROR_ILL_IDX 1 > #define CMDQ_ERR_CERROR_ABT_IDX 2 > +#define CMDQ_ERR_CERROR_ATC_INV_IDX 3 > > #define CMDQ_0_OP GENMASK_ULL(7, 0) > #define CMDQ_0_SSV (1UL << 11) > @@ -312,6 +315,12 @@ > #define CMDQ_TLBI_1_VA_MASK GENMASK_ULL(63, 12) > #define CMDQ_TLBI_1_IPA_MASK GENMASK_ULL(51, 12) > > +#define CMDQ_ATC_0_SSID GENMASK_ULL(31, 12) > +#define CMDQ_ATC_0_SID GENMASK_ULL(63, 32) > +#define CMDQ_ATC_0_GLOBAL (1UL << 9) > +#define CMDQ_ATC_1_SIZE GENMASK_ULL(5, 0) > +#define CMDQ_ATC_1_ADDR_MASK GENMASK_ULL(63, 12) > + > #define CMDQ_PRI_0_SSID GENMASK_ULL(31, 12) > #define CMDQ_PRI_0_SID GENMASK_ULL(63, 32) > #define CMDQ_PRI_1_GRPID GENMASK_ULL(8, 0) > @@ -365,6 +374,11 @@ module_param_named(disable_bypass, disable_bypass, bool, S_IRUGO); > MODULE_PARM_DESC(disable_bypass, > "Disable bypass streams such that incoming transactions from devices that are not attached to an iommu domain will report an abort back to the device and will not be allowed to pass through the SMMU."); > > +static bool disable_ats_check; > +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); > +MODULE_PARM_DESC(disable_ats_check, > + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); Yikes, do we really want this option, or is it just a leftover from debugging? > enum pri_resp { > PRI_RESP_DENY = 0, > PRI_RESP_FAIL = 1, > @@ -433,6 +447,16 @@ struct arm_smmu_cmdq_ent { > u64 addr; > } tlbi; > > + #define CMDQ_OP_ATC_INV 0x40 > + #define ATC_INV_SIZE_ALL 52 > + struct { > + u32 sid; > + u32 ssid; > + u64 addr; > + u8 size; > + bool global; > + } atc; > + > #define CMDQ_OP_PRI_RESP 0x41 > struct { > u32 sid; > @@ -580,10 +604,12 @@ struct arm_smmu_device { > /* SMMU private data for each master */ > struct arm_smmu_master { > struct arm_smmu_device *smmu; > + struct device *dev; > struct arm_smmu_domain *domain; > struct list_head domain_head; > u32 *sids; > unsigned int num_sids; > + bool ats_enabled :1; > }; > > /* SMMU private data for an IOMMU domain */ > @@ -813,6 +839,14 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) > case CMDQ_OP_TLBI_S12_VMALL: > cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_VMID, ent->tlbi.vmid); > break; > + case CMDQ_OP_ATC_INV: > + cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); > + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_GLOBAL, ent->atc.global); > + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SSID, ent->atc.ssid); > + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SID, ent->atc.sid); > + cmd[1] |= FIELD_PREP(CMDQ_ATC_1_SIZE, ent->atc.size); > + cmd[1] |= ent->atc.addr & CMDQ_ATC_1_ADDR_MASK; > + break; > case CMDQ_OP_PRI_RESP: > cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); > cmd[0] |= FIELD_PREP(CMDQ_PRI_0_SSID, ent->pri.ssid); > @@ -857,6 +891,7 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) > [CMDQ_ERR_CERROR_NONE_IDX] = "No error", > [CMDQ_ERR_CERROR_ILL_IDX] = "Illegal command", > [CMDQ_ERR_CERROR_ABT_IDX] = "Abort on command fetch", > + [CMDQ_ERR_CERROR_ATC_INV_IDX] = "ATC invalidate timeout", > }; > > int i; > @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) > dev_err(smmu->dev, "retrying command fetch\n"); > case CMDQ_ERR_CERROR_NONE_IDX: > return; > + case CMDQ_ERR_CERROR_ATC_INV_IDX: > + /* > + * ATC Invalidation Completion timeout. CONS is still pointing > + * at the CMD_SYNC. Attempt to complete other pending commands > + * by repeating the CMD_SYNC, though we might well end up back > + * here since the ATC invalidation may still be pending. > + */ > + return; This is pretty bad, as it means we're unable to unmap a page safely from a misbehaving device. Ideally, we'd block further transactions from the problematic endpoint, but I suppose we can't easily know which one it was, or inject a fault back into the unmap() path. Having said that, won't we get a CMD_SYNC timeout long before the ATC timeout? Perhaps we could propagate that out of arm_smmu_cmdq_issue_sync() and fail the unmap? Not sure, what do you think? > case CMDQ_ERR_CERROR_ILL_IDX: > /* Fallthrough */ > default: > @@ -1174,9 +1217,6 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, > FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | > FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) | > FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) | > -#ifdef CONFIG_PCI_ATS > - FIELD_PREP(STRTAB_STE_1_EATS, STRTAB_STE_1_EATS_TRANS) | > -#endif > FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1)); > > if (smmu->features & ARM_SMMU_FEAT_STALLS && > @@ -1203,6 +1243,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, > val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS); > } > > + if (master->ats_enabled) > + dst[1] |= cpu_to_le64(FIELD_PREP(STRTAB_STE_1_EATS, > + STRTAB_STE_1_EATS_TRANS)); > + > arm_smmu_sync_ste_for_sid(smmu, sid); > dst[0] = cpu_to_le64(val); > arm_smmu_sync_ste_for_sid(smmu, sid); > @@ -1405,6 +1449,86 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) > return IRQ_WAKE_THREAD; > } > > +static void > +arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, > + struct arm_smmu_cmdq_ent *cmd) > +{ > + size_t log2_span; > + size_t span_mask; > + /* ATC invalidates are always on 4096 bytes pages */ > + size_t inval_grain_shift = 12; > + unsigned long page_start, page_end; > + > + *cmd = (struct arm_smmu_cmdq_ent) { > + .opcode = CMDQ_OP_ATC_INV, > + .substream_valid = !!ssid, > + .atc.ssid = ssid, > + }; > + > + if (!size) { > + cmd->atc.size = ATC_INV_SIZE_ALL; > + return; > + } > + > + page_start = iova >> inval_grain_shift; > + page_end = (iova + size - 1) >> inval_grain_shift; > + > + /* > + * Find the smallest power of two that covers the range. Most > + * significant differing bit between start and end address indicates the > + * required span, ie. fls(start ^ end). For example: > + * > + * We want to invalidate pages [8; 11]. This is already the ideal range: > + * x = 0b1000 ^ 0b1011 = 0b11 > + * span = 1 << fls(x) = 4 > + * > + * To invalidate pages [7; 10], we need to invalidate [0; 15]: > + * x = 0b0111 ^ 0b1010 = 0b1101 > + * span = 1 << fls(x) = 16 > + */ Urgh, "The Address span is aligned to its size by the SMMU" is what makes this horrible. Please can you add that to the comment? An alternative would be to emit multiple ATC_INV commands. Do you have a feeling for what would be more efficient? > + log2_span = fls_long(page_start ^ page_end); > + span_mask = (1ULL << log2_span) - 1; > + > + page_start &= ~span_mask; > + > + cmd->atc.addr = page_start << inval_grain_shift; > + cmd->atc.size = log2_span; > +} > + > +static void arm_smmu_atc_inv_master(struct arm_smmu_master *master, > + struct arm_smmu_cmdq_ent *cmd) > +{ > + int i; > + > + if (!master->ats_enabled) > + return; > + > + for (i = 0; i < master->num_sids; i++) { > + cmd->atc.sid = master->sids[i]; > + arm_smmu_cmdq_issue_cmd(master->smmu, cmd); > + } > + > + arm_smmu_cmdq_issue_sync(master->smmu); > +} > + > +static void arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, > + int ssid, unsigned long iova, size_t size) > +{ > + unsigned long flags; > + struct arm_smmu_cmdq_ent cmd; > + struct arm_smmu_master *master; > + > + if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) > + return; > + > + arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); > + > + spin_lock_irqsave(&smmu_domain->devices_lock, flags); > + list_for_each_entry(master, &smmu_domain->devices, domain_head) > + arm_smmu_atc_inv_master(master, &cmd); > + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); > +} > + > /* IO_PGTABLE API */ > static void arm_smmu_tlb_sync(void *cookie) > { > @@ -1726,6 +1850,45 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) > } > } > > +static int arm_smmu_enable_ats(struct arm_smmu_master *master) > +{ > + int ret; > + size_t stu; > + struct pci_dev *pdev; > + struct arm_smmu_device *smmu = master->smmu; > + struct iommu_fwspec *fwspec = master->dev->iommu_fwspec; > + > + if (!(smmu->features & ARM_SMMU_FEAT_ATS) || !dev_is_pci(master->dev) || > + (fwspec->flags & IOMMU_FWSPEC_PCI_NO_ATS) || pci_ats_disabled()) > + return -ENOSYS; I'd probably make this -ENXIO. > + > + pdev = to_pci_dev(master->dev); > + if (pdev->untrusted) > + return -EPERM; > + > + /* Smallest Translation Unit: log2 of the smallest supported granule */ > + stu = __ffs(smmu->pgsize_bitmap); > + > + ret = pci_enable_ats(pdev, stu); > + if (ret) > + return ret; > + > + master->ats_enabled = true; > + dev_dbg(&pdev->dev, "enabled ATS (STU=%zu, QDEP=%d)\n", stu, > + pci_ats_queue_depth(pdev)); > + > + return 0; > +} > + > +static void arm_smmu_disable_ats(struct arm_smmu_master *master) > +{ > + if (!master->ats_enabled || !dev_is_pci(master->dev)) > + return; > + > + pci_disable_ats(to_pci_dev(master->dev)); > + master->ats_enabled = false; > +} > + > static void arm_smmu_detach_dev(struct arm_smmu_master *master) > { > unsigned long flags; > @@ -1740,6 +1903,9 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) > > master->domain = NULL; > arm_smmu_install_ste_for_dev(master); > + > + /* Disabling ATS invalidates all ATC entries */ > + arm_smmu_disable_ats(master); > } > > static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) > @@ -1783,6 +1949,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) > list_add(&master->domain_head, &smmu_domain->devices); > spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); > > + if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) > + arm_smmu_enable_ats(master); Do we care about the return value? Will ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-15 13:21 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-15 13:21 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: lorenzo.pieralisi, eric.auger, zhongmiao, okaya, joro, rjw, linux-acpi, iommu, hanjun.guo, sudeep.holla, robin.murphy, linux-arm-kernel, lenb On Tue, Apr 09, 2019 at 05:52:44PM +0100, Jean-Philippe Brucker wrote: > PCIe devices can implement their own TLB, named Address Translation Cache > (ATC). Enable Address Translation Service (ATS) for devices that support > it and send them invalidation requests whenever we invalidate the IOTLBs. > > ATC invalidation is allowed to take up to 90 seconds, according to the > PCIe spec, so it is possible to get a SMMU command queue timeout during > normal operations. However we expect implementations to complete > invalidation in reasonable time. > > We only enable ATS for "trusted" devices, and currently rely on the > pci_dev->untrusted bit. For ATS we have to trust that: AFAICT, devicetree has no way to describe a device as untrusted, so everything will be trusted by default on those systems. Is that right? > (a) The device doesn't issue "translated" memory requests for addresses > that weren't returned by the SMMU in a Translation Completion. In > particular, if we give control of a device or device partition to a VM > or userspace, software cannot program the device to access arbitrary > "translated" addresses. Any plans to look at split-stage ATS later on? I think that would allow us to pass untrusted devices through to a VM behind S1 ATS. > (b) The device follows permissions granted by the SMMU in a Translation > Completion. If the device requested read+write permission and only > got read, then it doesn't write. Guessing we just ignore execute permissions, or do we need read implies exec? > (c) The device doesn't send Translated transactions for an address that > was invalidated by an ATC invalidation. > > Note that the PCIe specification explicitly requires all of these, so we > can assume that implementations will cleanly shield ATCs from software. > > All ATS translated requests still go through the SMMU, to walk the stream > table and check that the device is actually allowed to send translated > requests. > > Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> > --- > drivers/iommu/arm-smmu-v3.c | 196 +++++++++++++++++++++++++++++++++++- > 1 file changed, 191 insertions(+), 5 deletions(-) > > diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c > index 3e7198ee9530..7819cd60d08b 100644 > --- a/drivers/iommu/arm-smmu-v3.c > +++ b/drivers/iommu/arm-smmu-v3.c > @@ -29,6 +29,7 @@ > #include <linux/of_iommu.h> > #include <linux/of_platform.h> > #include <linux/pci.h> > +#include <linux/pci-ats.h> > #include <linux/platform_device.h> > > #include <linux/amba/bus.h> > @@ -86,6 +87,7 @@ > #define IDR5_VAX_52_BIT 1 > > #define ARM_SMMU_CR0 0x20 > +#define CR0_ATSCHK (1 << 4) > #define CR0_CMDQEN (1 << 3) > #define CR0_EVTQEN (1 << 2) > #define CR0_PRIQEN (1 << 1) > @@ -294,6 +296,7 @@ > #define CMDQ_ERR_CERROR_NONE_IDX 0 > #define CMDQ_ERR_CERROR_ILL_IDX 1 > #define CMDQ_ERR_CERROR_ABT_IDX 2 > +#define CMDQ_ERR_CERROR_ATC_INV_IDX 3 > > #define CMDQ_0_OP GENMASK_ULL(7, 0) > #define CMDQ_0_SSV (1UL << 11) > @@ -312,6 +315,12 @@ > #define CMDQ_TLBI_1_VA_MASK GENMASK_ULL(63, 12) > #define CMDQ_TLBI_1_IPA_MASK GENMASK_ULL(51, 12) > > +#define CMDQ_ATC_0_SSID GENMASK_ULL(31, 12) > +#define CMDQ_ATC_0_SID GENMASK_ULL(63, 32) > +#define CMDQ_ATC_0_GLOBAL (1UL << 9) > +#define CMDQ_ATC_1_SIZE GENMASK_ULL(5, 0) > +#define CMDQ_ATC_1_ADDR_MASK GENMASK_ULL(63, 12) > + > #define CMDQ_PRI_0_SSID GENMASK_ULL(31, 12) > #define CMDQ_PRI_0_SID GENMASK_ULL(63, 32) > #define CMDQ_PRI_1_GRPID GENMASK_ULL(8, 0) > @@ -365,6 +374,11 @@ module_param_named(disable_bypass, disable_bypass, bool, S_IRUGO); > MODULE_PARM_DESC(disable_bypass, > "Disable bypass streams such that incoming transactions from devices that are not attached to an iommu domain will report an abort back to the device and will not be allowed to pass through the SMMU."); > > +static bool disable_ats_check; > +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); > +MODULE_PARM_DESC(disable_ats_check, > + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); Yikes, do we really want this option, or is it just a leftover from debugging? > enum pri_resp { > PRI_RESP_DENY = 0, > PRI_RESP_FAIL = 1, > @@ -433,6 +447,16 @@ struct arm_smmu_cmdq_ent { > u64 addr; > } tlbi; > > + #define CMDQ_OP_ATC_INV 0x40 > + #define ATC_INV_SIZE_ALL 52 > + struct { > + u32 sid; > + u32 ssid; > + u64 addr; > + u8 size; > + bool global; > + } atc; > + > #define CMDQ_OP_PRI_RESP 0x41 > struct { > u32 sid; > @@ -580,10 +604,12 @@ struct arm_smmu_device { > /* SMMU private data for each master */ > struct arm_smmu_master { > struct arm_smmu_device *smmu; > + struct device *dev; > struct arm_smmu_domain *domain; > struct list_head domain_head; > u32 *sids; > unsigned int num_sids; > + bool ats_enabled :1; > }; > > /* SMMU private data for an IOMMU domain */ > @@ -813,6 +839,14 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) > case CMDQ_OP_TLBI_S12_VMALL: > cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_VMID, ent->tlbi.vmid); > break; > + case CMDQ_OP_ATC_INV: > + cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); > + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_GLOBAL, ent->atc.global); > + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SSID, ent->atc.ssid); > + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SID, ent->atc.sid); > + cmd[1] |= FIELD_PREP(CMDQ_ATC_1_SIZE, ent->atc.size); > + cmd[1] |= ent->atc.addr & CMDQ_ATC_1_ADDR_MASK; > + break; > case CMDQ_OP_PRI_RESP: > cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); > cmd[0] |= FIELD_PREP(CMDQ_PRI_0_SSID, ent->pri.ssid); > @@ -857,6 +891,7 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) > [CMDQ_ERR_CERROR_NONE_IDX] = "No error", > [CMDQ_ERR_CERROR_ILL_IDX] = "Illegal command", > [CMDQ_ERR_CERROR_ABT_IDX] = "Abort on command fetch", > + [CMDQ_ERR_CERROR_ATC_INV_IDX] = "ATC invalidate timeout", > }; > > int i; > @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) > dev_err(smmu->dev, "retrying command fetch\n"); > case CMDQ_ERR_CERROR_NONE_IDX: > return; > + case CMDQ_ERR_CERROR_ATC_INV_IDX: > + /* > + * ATC Invalidation Completion timeout. CONS is still pointing > + * at the CMD_SYNC. Attempt to complete other pending commands > + * by repeating the CMD_SYNC, though we might well end up back > + * here since the ATC invalidation may still be pending. > + */ > + return; This is pretty bad, as it means we're unable to unmap a page safely from a misbehaving device. Ideally, we'd block further transactions from the problematic endpoint, but I suppose we can't easily know which one it was, or inject a fault back into the unmap() path. Having said that, won't we get a CMD_SYNC timeout long before the ATC timeout? Perhaps we could propagate that out of arm_smmu_cmdq_issue_sync() and fail the unmap? Not sure, what do you think? > case CMDQ_ERR_CERROR_ILL_IDX: > /* Fallthrough */ > default: > @@ -1174,9 +1217,6 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, > FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | > FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) | > FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) | > -#ifdef CONFIG_PCI_ATS > - FIELD_PREP(STRTAB_STE_1_EATS, STRTAB_STE_1_EATS_TRANS) | > -#endif > FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1)); > > if (smmu->features & ARM_SMMU_FEAT_STALLS && > @@ -1203,6 +1243,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, > val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS); > } > > + if (master->ats_enabled) > + dst[1] |= cpu_to_le64(FIELD_PREP(STRTAB_STE_1_EATS, > + STRTAB_STE_1_EATS_TRANS)); > + > arm_smmu_sync_ste_for_sid(smmu, sid); > dst[0] = cpu_to_le64(val); > arm_smmu_sync_ste_for_sid(smmu, sid); > @@ -1405,6 +1449,86 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) > return IRQ_WAKE_THREAD; > } > > +static void > +arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, > + struct arm_smmu_cmdq_ent *cmd) > +{ > + size_t log2_span; > + size_t span_mask; > + /* ATC invalidates are always on 4096 bytes pages */ > + size_t inval_grain_shift = 12; > + unsigned long page_start, page_end; > + > + *cmd = (struct arm_smmu_cmdq_ent) { > + .opcode = CMDQ_OP_ATC_INV, > + .substream_valid = !!ssid, > + .atc.ssid = ssid, > + }; > + > + if (!size) { > + cmd->atc.size = ATC_INV_SIZE_ALL; > + return; > + } > + > + page_start = iova >> inval_grain_shift; > + page_end = (iova + size - 1) >> inval_grain_shift; > + > + /* > + * Find the smallest power of two that covers the range. Most > + * significant differing bit between start and end address indicates the > + * required span, ie. fls(start ^ end). For example: > + * > + * We want to invalidate pages [8; 11]. This is already the ideal range: > + * x = 0b1000 ^ 0b1011 = 0b11 > + * span = 1 << fls(x) = 4 > + * > + * To invalidate pages [7; 10], we need to invalidate [0; 15]: > + * x = 0b0111 ^ 0b1010 = 0b1101 > + * span = 1 << fls(x) = 16 > + */ Urgh, "The Address span is aligned to its size by the SMMU" is what makes this horrible. Please can you add that to the comment? An alternative would be to emit multiple ATC_INV commands. Do you have a feeling for what would be more efficient? > + log2_span = fls_long(page_start ^ page_end); > + span_mask = (1ULL << log2_span) - 1; > + > + page_start &= ~span_mask; > + > + cmd->atc.addr = page_start << inval_grain_shift; > + cmd->atc.size = log2_span; > +} > + > +static void arm_smmu_atc_inv_master(struct arm_smmu_master *master, > + struct arm_smmu_cmdq_ent *cmd) > +{ > + int i; > + > + if (!master->ats_enabled) > + return; > + > + for (i = 0; i < master->num_sids; i++) { > + cmd->atc.sid = master->sids[i]; > + arm_smmu_cmdq_issue_cmd(master->smmu, cmd); > + } > + > + arm_smmu_cmdq_issue_sync(master->smmu); > +} > + > +static void arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, > + int ssid, unsigned long iova, size_t size) > +{ > + unsigned long flags; > + struct arm_smmu_cmdq_ent cmd; > + struct arm_smmu_master *master; > + > + if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) > + return; > + > + arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); > + > + spin_lock_irqsave(&smmu_domain->devices_lock, flags); > + list_for_each_entry(master, &smmu_domain->devices, domain_head) > + arm_smmu_atc_inv_master(master, &cmd); > + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); > +} > + > /* IO_PGTABLE API */ > static void arm_smmu_tlb_sync(void *cookie) > { > @@ -1726,6 +1850,45 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) > } > } > > +static int arm_smmu_enable_ats(struct arm_smmu_master *master) > +{ > + int ret; > + size_t stu; > + struct pci_dev *pdev; > + struct arm_smmu_device *smmu = master->smmu; > + struct iommu_fwspec *fwspec = master->dev->iommu_fwspec; > + > + if (!(smmu->features & ARM_SMMU_FEAT_ATS) || !dev_is_pci(master->dev) || > + (fwspec->flags & IOMMU_FWSPEC_PCI_NO_ATS) || pci_ats_disabled()) > + return -ENOSYS; I'd probably make this -ENXIO. > + > + pdev = to_pci_dev(master->dev); > + if (pdev->untrusted) > + return -EPERM; > + > + /* Smallest Translation Unit: log2 of the smallest supported granule */ > + stu = __ffs(smmu->pgsize_bitmap); > + > + ret = pci_enable_ats(pdev, stu); > + if (ret) > + return ret; > + > + master->ats_enabled = true; > + dev_dbg(&pdev->dev, "enabled ATS (STU=%zu, QDEP=%d)\n", stu, > + pci_ats_queue_depth(pdev)); > + > + return 0; > +} > + > +static void arm_smmu_disable_ats(struct arm_smmu_master *master) > +{ > + if (!master->ats_enabled || !dev_is_pci(master->dev)) > + return; > + > + pci_disable_ats(to_pci_dev(master->dev)); > + master->ats_enabled = false; > +} > + > static void arm_smmu_detach_dev(struct arm_smmu_master *master) > { > unsigned long flags; > @@ -1740,6 +1903,9 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) > > master->domain = NULL; > arm_smmu_install_ste_for_dev(master); > + > + /* Disabling ATS invalidates all ATC entries */ > + arm_smmu_disable_ats(master); > } > > static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) > @@ -1783,6 +1949,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) > list_add(&master->domain_head, &smmu_domain->devices); > spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); > > + if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) > + arm_smmu_enable_ats(master); Do we care about the return value? Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-15 13:21 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-15 13:21 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb On Tue, Apr 09, 2019 at 05:52:44PM +0100, Jean-Philippe Brucker wrote: > PCIe devices can implement their own TLB, named Address Translation Cache > (ATC). Enable Address Translation Service (ATS) for devices that support > it and send them invalidation requests whenever we invalidate the IOTLBs. > > ATC invalidation is allowed to take up to 90 seconds, according to the > PCIe spec, so it is possible to get a SMMU command queue timeout during > normal operations. However we expect implementations to complete > invalidation in reasonable time. > > We only enable ATS for "trusted" devices, and currently rely on the > pci_dev->untrusted bit. For ATS we have to trust that: AFAICT, devicetree has no way to describe a device as untrusted, so everything will be trusted by default on those systems. Is that right? > (a) The device doesn't issue "translated" memory requests for addresses > that weren't returned by the SMMU in a Translation Completion. In > particular, if we give control of a device or device partition to a VM > or userspace, software cannot program the device to access arbitrary > "translated" addresses. Any plans to look at split-stage ATS later on? I think that would allow us to pass untrusted devices through to a VM behind S1 ATS. > (b) The device follows permissions granted by the SMMU in a Translation > Completion. If the device requested read+write permission and only > got read, then it doesn't write. Guessing we just ignore execute permissions, or do we need read implies exec? > (c) The device doesn't send Translated transactions for an address that > was invalidated by an ATC invalidation. > > Note that the PCIe specification explicitly requires all of these, so we > can assume that implementations will cleanly shield ATCs from software. > > All ATS translated requests still go through the SMMU, to walk the stream > table and check that the device is actually allowed to send translated > requests. > > Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> > --- > drivers/iommu/arm-smmu-v3.c | 196 +++++++++++++++++++++++++++++++++++- > 1 file changed, 191 insertions(+), 5 deletions(-) > > diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c > index 3e7198ee9530..7819cd60d08b 100644 > --- a/drivers/iommu/arm-smmu-v3.c > +++ b/drivers/iommu/arm-smmu-v3.c > @@ -29,6 +29,7 @@ > #include <linux/of_iommu.h> > #include <linux/of_platform.h> > #include <linux/pci.h> > +#include <linux/pci-ats.h> > #include <linux/platform_device.h> > > #include <linux/amba/bus.h> > @@ -86,6 +87,7 @@ > #define IDR5_VAX_52_BIT 1 > > #define ARM_SMMU_CR0 0x20 > +#define CR0_ATSCHK (1 << 4) > #define CR0_CMDQEN (1 << 3) > #define CR0_EVTQEN (1 << 2) > #define CR0_PRIQEN (1 << 1) > @@ -294,6 +296,7 @@ > #define CMDQ_ERR_CERROR_NONE_IDX 0 > #define CMDQ_ERR_CERROR_ILL_IDX 1 > #define CMDQ_ERR_CERROR_ABT_IDX 2 > +#define CMDQ_ERR_CERROR_ATC_INV_IDX 3 > > #define CMDQ_0_OP GENMASK_ULL(7, 0) > #define CMDQ_0_SSV (1UL << 11) > @@ -312,6 +315,12 @@ > #define CMDQ_TLBI_1_VA_MASK GENMASK_ULL(63, 12) > #define CMDQ_TLBI_1_IPA_MASK GENMASK_ULL(51, 12) > > +#define CMDQ_ATC_0_SSID GENMASK_ULL(31, 12) > +#define CMDQ_ATC_0_SID GENMASK_ULL(63, 32) > +#define CMDQ_ATC_0_GLOBAL (1UL << 9) > +#define CMDQ_ATC_1_SIZE GENMASK_ULL(5, 0) > +#define CMDQ_ATC_1_ADDR_MASK GENMASK_ULL(63, 12) > + > #define CMDQ_PRI_0_SSID GENMASK_ULL(31, 12) > #define CMDQ_PRI_0_SID GENMASK_ULL(63, 32) > #define CMDQ_PRI_1_GRPID GENMASK_ULL(8, 0) > @@ -365,6 +374,11 @@ module_param_named(disable_bypass, disable_bypass, bool, S_IRUGO); > MODULE_PARM_DESC(disable_bypass, > "Disable bypass streams such that incoming transactions from devices that are not attached to an iommu domain will report an abort back to the device and will not be allowed to pass through the SMMU."); > > +static bool disable_ats_check; > +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); > +MODULE_PARM_DESC(disable_ats_check, > + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); Yikes, do we really want this option, or is it just a leftover from debugging? > enum pri_resp { > PRI_RESP_DENY = 0, > PRI_RESP_FAIL = 1, > @@ -433,6 +447,16 @@ struct arm_smmu_cmdq_ent { > u64 addr; > } tlbi; > > + #define CMDQ_OP_ATC_INV 0x40 > + #define ATC_INV_SIZE_ALL 52 > + struct { > + u32 sid; > + u32 ssid; > + u64 addr; > + u8 size; > + bool global; > + } atc; > + > #define CMDQ_OP_PRI_RESP 0x41 > struct { > u32 sid; > @@ -580,10 +604,12 @@ struct arm_smmu_device { > /* SMMU private data for each master */ > struct arm_smmu_master { > struct arm_smmu_device *smmu; > + struct device *dev; > struct arm_smmu_domain *domain; > struct list_head domain_head; > u32 *sids; > unsigned int num_sids; > + bool ats_enabled :1; > }; > > /* SMMU private data for an IOMMU domain */ > @@ -813,6 +839,14 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) > case CMDQ_OP_TLBI_S12_VMALL: > cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_VMID, ent->tlbi.vmid); > break; > + case CMDQ_OP_ATC_INV: > + cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); > + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_GLOBAL, ent->atc.global); > + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SSID, ent->atc.ssid); > + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SID, ent->atc.sid); > + cmd[1] |= FIELD_PREP(CMDQ_ATC_1_SIZE, ent->atc.size); > + cmd[1] |= ent->atc.addr & CMDQ_ATC_1_ADDR_MASK; > + break; > case CMDQ_OP_PRI_RESP: > cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); > cmd[0] |= FIELD_PREP(CMDQ_PRI_0_SSID, ent->pri.ssid); > @@ -857,6 +891,7 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) > [CMDQ_ERR_CERROR_NONE_IDX] = "No error", > [CMDQ_ERR_CERROR_ILL_IDX] = "Illegal command", > [CMDQ_ERR_CERROR_ABT_IDX] = "Abort on command fetch", > + [CMDQ_ERR_CERROR_ATC_INV_IDX] = "ATC invalidate timeout", > }; > > int i; > @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) > dev_err(smmu->dev, "retrying command fetch\n"); > case CMDQ_ERR_CERROR_NONE_IDX: > return; > + case CMDQ_ERR_CERROR_ATC_INV_IDX: > + /* > + * ATC Invalidation Completion timeout. CONS is still pointing > + * at the CMD_SYNC. Attempt to complete other pending commands > + * by repeating the CMD_SYNC, though we might well end up back > + * here since the ATC invalidation may still be pending. > + */ > + return; This is pretty bad, as it means we're unable to unmap a page safely from a misbehaving device. Ideally, we'd block further transactions from the problematic endpoint, but I suppose we can't easily know which one it was, or inject a fault back into the unmap() path. Having said that, won't we get a CMD_SYNC timeout long before the ATC timeout? Perhaps we could propagate that out of arm_smmu_cmdq_issue_sync() and fail the unmap? Not sure, what do you think? > case CMDQ_ERR_CERROR_ILL_IDX: > /* Fallthrough */ > default: > @@ -1174,9 +1217,6 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, > FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | > FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) | > FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) | > -#ifdef CONFIG_PCI_ATS > - FIELD_PREP(STRTAB_STE_1_EATS, STRTAB_STE_1_EATS_TRANS) | > -#endif > FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1)); > > if (smmu->features & ARM_SMMU_FEAT_STALLS && > @@ -1203,6 +1243,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, > val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS); > } > > + if (master->ats_enabled) > + dst[1] |= cpu_to_le64(FIELD_PREP(STRTAB_STE_1_EATS, > + STRTAB_STE_1_EATS_TRANS)); > + > arm_smmu_sync_ste_for_sid(smmu, sid); > dst[0] = cpu_to_le64(val); > arm_smmu_sync_ste_for_sid(smmu, sid); > @@ -1405,6 +1449,86 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) > return IRQ_WAKE_THREAD; > } > > +static void > +arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, > + struct arm_smmu_cmdq_ent *cmd) > +{ > + size_t log2_span; > + size_t span_mask; > + /* ATC invalidates are always on 4096 bytes pages */ > + size_t inval_grain_shift = 12; > + unsigned long page_start, page_end; > + > + *cmd = (struct arm_smmu_cmdq_ent) { > + .opcode = CMDQ_OP_ATC_INV, > + .substream_valid = !!ssid, > + .atc.ssid = ssid, > + }; > + > + if (!size) { > + cmd->atc.size = ATC_INV_SIZE_ALL; > + return; > + } > + > + page_start = iova >> inval_grain_shift; > + page_end = (iova + size - 1) >> inval_grain_shift; > + > + /* > + * Find the smallest power of two that covers the range. Most > + * significant differing bit between start and end address indicates the > + * required span, ie. fls(start ^ end). For example: > + * > + * We want to invalidate pages [8; 11]. This is already the ideal range: > + * x = 0b1000 ^ 0b1011 = 0b11 > + * span = 1 << fls(x) = 4 > + * > + * To invalidate pages [7; 10], we need to invalidate [0; 15]: > + * x = 0b0111 ^ 0b1010 = 0b1101 > + * span = 1 << fls(x) = 16 > + */ Urgh, "The Address span is aligned to its size by the SMMU" is what makes this horrible. Please can you add that to the comment? An alternative would be to emit multiple ATC_INV commands. Do you have a feeling for what would be more efficient? > + log2_span = fls_long(page_start ^ page_end); > + span_mask = (1ULL << log2_span) - 1; > + > + page_start &= ~span_mask; > + > + cmd->atc.addr = page_start << inval_grain_shift; > + cmd->atc.size = log2_span; > +} > + > +static void arm_smmu_atc_inv_master(struct arm_smmu_master *master, > + struct arm_smmu_cmdq_ent *cmd) > +{ > + int i; > + > + if (!master->ats_enabled) > + return; > + > + for (i = 0; i < master->num_sids; i++) { > + cmd->atc.sid = master->sids[i]; > + arm_smmu_cmdq_issue_cmd(master->smmu, cmd); > + } > + > + arm_smmu_cmdq_issue_sync(master->smmu); > +} > + > +static void arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, > + int ssid, unsigned long iova, size_t size) > +{ > + unsigned long flags; > + struct arm_smmu_cmdq_ent cmd; > + struct arm_smmu_master *master; > + > + if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) > + return; > + > + arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); > + > + spin_lock_irqsave(&smmu_domain->devices_lock, flags); > + list_for_each_entry(master, &smmu_domain->devices, domain_head) > + arm_smmu_atc_inv_master(master, &cmd); > + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); > +} > + > /* IO_PGTABLE API */ > static void arm_smmu_tlb_sync(void *cookie) > { > @@ -1726,6 +1850,45 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) > } > } > > +static int arm_smmu_enable_ats(struct arm_smmu_master *master) > +{ > + int ret; > + size_t stu; > + struct pci_dev *pdev; > + struct arm_smmu_device *smmu = master->smmu; > + struct iommu_fwspec *fwspec = master->dev->iommu_fwspec; > + > + if (!(smmu->features & ARM_SMMU_FEAT_ATS) || !dev_is_pci(master->dev) || > + (fwspec->flags & IOMMU_FWSPEC_PCI_NO_ATS) || pci_ats_disabled()) > + return -ENOSYS; I'd probably make this -ENXIO. > + > + pdev = to_pci_dev(master->dev); > + if (pdev->untrusted) > + return -EPERM; > + > + /* Smallest Translation Unit: log2 of the smallest supported granule */ > + stu = __ffs(smmu->pgsize_bitmap); > + > + ret = pci_enable_ats(pdev, stu); > + if (ret) > + return ret; > + > + master->ats_enabled = true; > + dev_dbg(&pdev->dev, "enabled ATS (STU=%zu, QDEP=%d)\n", stu, > + pci_ats_queue_depth(pdev)); > + > + return 0; > +} > + > +static void arm_smmu_disable_ats(struct arm_smmu_master *master) > +{ > + if (!master->ats_enabled || !dev_is_pci(master->dev)) > + return; > + > + pci_disable_ats(to_pci_dev(master->dev)); > + master->ats_enabled = false; > +} > + > static void arm_smmu_detach_dev(struct arm_smmu_master *master) > { > unsigned long flags; > @@ -1740,6 +1903,9 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) > > master->domain = NULL; > arm_smmu_install_ste_for_dev(master); > + > + /* Disabling ATS invalidates all ATC entries */ > + arm_smmu_disable_ats(master); > } > > static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) > @@ -1783,6 +1949,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) > list_add(&master->domain_head, &smmu_domain->devices); > spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); > > + if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) > + arm_smmu_enable_ats(master); Do we care about the return value? Will _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-15 13:21 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-15 13:21 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: iommu, linux-arm-kernel, linux-acpi, robin.murphy, joro, hanjun.guo, lorenzo.pieralisi, sudeep.holla, rjw, lenb, okaya, zhongmiao, eric.auger On Tue, Apr 09, 2019 at 05:52:44PM +0100, Jean-Philippe Brucker wrote: > PCIe devices can implement their own TLB, named Address Translation Cache > (ATC). Enable Address Translation Service (ATS) for devices that support > it and send them invalidation requests whenever we invalidate the IOTLBs. > > ATC invalidation is allowed to take up to 90 seconds, according to the > PCIe spec, so it is possible to get a SMMU command queue timeout during > normal operations. However we expect implementations to complete > invalidation in reasonable time. > > We only enable ATS for "trusted" devices, and currently rely on the > pci_dev->untrusted bit. For ATS we have to trust that: AFAICT, devicetree has no way to describe a device as untrusted, so everything will be trusted by default on those systems. Is that right? > (a) The device doesn't issue "translated" memory requests for addresses > that weren't returned by the SMMU in a Translation Completion. In > particular, if we give control of a device or device partition to a VM > or userspace, software cannot program the device to access arbitrary > "translated" addresses. Any plans to look at split-stage ATS later on? I think that would allow us to pass untrusted devices through to a VM behind S1 ATS. > (b) The device follows permissions granted by the SMMU in a Translation > Completion. If the device requested read+write permission and only > got read, then it doesn't write. Guessing we just ignore execute permissions, or do we need read implies exec? > (c) The device doesn't send Translated transactions for an address that > was invalidated by an ATC invalidation. > > Note that the PCIe specification explicitly requires all of these, so we > can assume that implementations will cleanly shield ATCs from software. > > All ATS translated requests still go through the SMMU, to walk the stream > table and check that the device is actually allowed to send translated > requests. > > Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> > --- > drivers/iommu/arm-smmu-v3.c | 196 +++++++++++++++++++++++++++++++++++- > 1 file changed, 191 insertions(+), 5 deletions(-) > > diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c > index 3e7198ee9530..7819cd60d08b 100644 > --- a/drivers/iommu/arm-smmu-v3.c > +++ b/drivers/iommu/arm-smmu-v3.c > @@ -29,6 +29,7 @@ > #include <linux/of_iommu.h> > #include <linux/of_platform.h> > #include <linux/pci.h> > +#include <linux/pci-ats.h> > #include <linux/platform_device.h> > > #include <linux/amba/bus.h> > @@ -86,6 +87,7 @@ > #define IDR5_VAX_52_BIT 1 > > #define ARM_SMMU_CR0 0x20 > +#define CR0_ATSCHK (1 << 4) > #define CR0_CMDQEN (1 << 3) > #define CR0_EVTQEN (1 << 2) > #define CR0_PRIQEN (1 << 1) > @@ -294,6 +296,7 @@ > #define CMDQ_ERR_CERROR_NONE_IDX 0 > #define CMDQ_ERR_CERROR_ILL_IDX 1 > #define CMDQ_ERR_CERROR_ABT_IDX 2 > +#define CMDQ_ERR_CERROR_ATC_INV_IDX 3 > > #define CMDQ_0_OP GENMASK_ULL(7, 0) > #define CMDQ_0_SSV (1UL << 11) > @@ -312,6 +315,12 @@ > #define CMDQ_TLBI_1_VA_MASK GENMASK_ULL(63, 12) > #define CMDQ_TLBI_1_IPA_MASK GENMASK_ULL(51, 12) > > +#define CMDQ_ATC_0_SSID GENMASK_ULL(31, 12) > +#define CMDQ_ATC_0_SID GENMASK_ULL(63, 32) > +#define CMDQ_ATC_0_GLOBAL (1UL << 9) > +#define CMDQ_ATC_1_SIZE GENMASK_ULL(5, 0) > +#define CMDQ_ATC_1_ADDR_MASK GENMASK_ULL(63, 12) > + > #define CMDQ_PRI_0_SSID GENMASK_ULL(31, 12) > #define CMDQ_PRI_0_SID GENMASK_ULL(63, 32) > #define CMDQ_PRI_1_GRPID GENMASK_ULL(8, 0) > @@ -365,6 +374,11 @@ module_param_named(disable_bypass, disable_bypass, bool, S_IRUGO); > MODULE_PARM_DESC(disable_bypass, > "Disable bypass streams such that incoming transactions from devices that are not attached to an iommu domain will report an abort back to the device and will not be allowed to pass through the SMMU."); > > +static bool disable_ats_check; > +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); > +MODULE_PARM_DESC(disable_ats_check, > + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); Yikes, do we really want this option, or is it just a leftover from debugging? > enum pri_resp { > PRI_RESP_DENY = 0, > PRI_RESP_FAIL = 1, > @@ -433,6 +447,16 @@ struct arm_smmu_cmdq_ent { > u64 addr; > } tlbi; > > + #define CMDQ_OP_ATC_INV 0x40 > + #define ATC_INV_SIZE_ALL 52 > + struct { > + u32 sid; > + u32 ssid; > + u64 addr; > + u8 size; > + bool global; > + } atc; > + > #define CMDQ_OP_PRI_RESP 0x41 > struct { > u32 sid; > @@ -580,10 +604,12 @@ struct arm_smmu_device { > /* SMMU private data for each master */ > struct arm_smmu_master { > struct arm_smmu_device *smmu; > + struct device *dev; > struct arm_smmu_domain *domain; > struct list_head domain_head; > u32 *sids; > unsigned int num_sids; > + bool ats_enabled :1; > }; > > /* SMMU private data for an IOMMU domain */ > @@ -813,6 +839,14 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) > case CMDQ_OP_TLBI_S12_VMALL: > cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_VMID, ent->tlbi.vmid); > break; > + case CMDQ_OP_ATC_INV: > + cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); > + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_GLOBAL, ent->atc.global); > + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SSID, ent->atc.ssid); > + cmd[0] |= FIELD_PREP(CMDQ_ATC_0_SID, ent->atc.sid); > + cmd[1] |= FIELD_PREP(CMDQ_ATC_1_SIZE, ent->atc.size); > + cmd[1] |= ent->atc.addr & CMDQ_ATC_1_ADDR_MASK; > + break; > case CMDQ_OP_PRI_RESP: > cmd[0] |= FIELD_PREP(CMDQ_0_SSV, ent->substream_valid); > cmd[0] |= FIELD_PREP(CMDQ_PRI_0_SSID, ent->pri.ssid); > @@ -857,6 +891,7 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) > [CMDQ_ERR_CERROR_NONE_IDX] = "No error", > [CMDQ_ERR_CERROR_ILL_IDX] = "Illegal command", > [CMDQ_ERR_CERROR_ABT_IDX] = "Abort on command fetch", > + [CMDQ_ERR_CERROR_ATC_INV_IDX] = "ATC invalidate timeout", > }; > > int i; > @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) > dev_err(smmu->dev, "retrying command fetch\n"); > case CMDQ_ERR_CERROR_NONE_IDX: > return; > + case CMDQ_ERR_CERROR_ATC_INV_IDX: > + /* > + * ATC Invalidation Completion timeout. CONS is still pointing > + * at the CMD_SYNC. Attempt to complete other pending commands > + * by repeating the CMD_SYNC, though we might well end up back > + * here since the ATC invalidation may still be pending. > + */ > + return; This is pretty bad, as it means we're unable to unmap a page safely from a misbehaving device. Ideally, we'd block further transactions from the problematic endpoint, but I suppose we can't easily know which one it was, or inject a fault back into the unmap() path. Having said that, won't we get a CMD_SYNC timeout long before the ATC timeout? Perhaps we could propagate that out of arm_smmu_cmdq_issue_sync() and fail the unmap? Not sure, what do you think? > case CMDQ_ERR_CERROR_ILL_IDX: > /* Fallthrough */ > default: > @@ -1174,9 +1217,6 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, > FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | > FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) | > FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) | > -#ifdef CONFIG_PCI_ATS > - FIELD_PREP(STRTAB_STE_1_EATS, STRTAB_STE_1_EATS_TRANS) | > -#endif > FIELD_PREP(STRTAB_STE_1_STRW, STRTAB_STE_1_STRW_NSEL1)); > > if (smmu->features & ARM_SMMU_FEAT_STALLS && > @@ -1203,6 +1243,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, > val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S2_TRANS); > } > > + if (master->ats_enabled) > + dst[1] |= cpu_to_le64(FIELD_PREP(STRTAB_STE_1_EATS, > + STRTAB_STE_1_EATS_TRANS)); > + > arm_smmu_sync_ste_for_sid(smmu, sid); > dst[0] = cpu_to_le64(val); > arm_smmu_sync_ste_for_sid(smmu, sid); > @@ -1405,6 +1449,86 @@ static irqreturn_t arm_smmu_combined_irq_handler(int irq, void *dev) > return IRQ_WAKE_THREAD; > } > > +static void > +arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size, > + struct arm_smmu_cmdq_ent *cmd) > +{ > + size_t log2_span; > + size_t span_mask; > + /* ATC invalidates are always on 4096 bytes pages */ > + size_t inval_grain_shift = 12; > + unsigned long page_start, page_end; > + > + *cmd = (struct arm_smmu_cmdq_ent) { > + .opcode = CMDQ_OP_ATC_INV, > + .substream_valid = !!ssid, > + .atc.ssid = ssid, > + }; > + > + if (!size) { > + cmd->atc.size = ATC_INV_SIZE_ALL; > + return; > + } > + > + page_start = iova >> inval_grain_shift; > + page_end = (iova + size - 1) >> inval_grain_shift; > + > + /* > + * Find the smallest power of two that covers the range. Most > + * significant differing bit between start and end address indicates the > + * required span, ie. fls(start ^ end). For example: > + * > + * We want to invalidate pages [8; 11]. This is already the ideal range: > + * x = 0b1000 ^ 0b1011 = 0b11 > + * span = 1 << fls(x) = 4 > + * > + * To invalidate pages [7; 10], we need to invalidate [0; 15]: > + * x = 0b0111 ^ 0b1010 = 0b1101 > + * span = 1 << fls(x) = 16 > + */ Urgh, "The Address span is aligned to its size by the SMMU" is what makes this horrible. Please can you add that to the comment? An alternative would be to emit multiple ATC_INV commands. Do you have a feeling for what would be more efficient? > + log2_span = fls_long(page_start ^ page_end); > + span_mask = (1ULL << log2_span) - 1; > + > + page_start &= ~span_mask; > + > + cmd->atc.addr = page_start << inval_grain_shift; > + cmd->atc.size = log2_span; > +} > + > +static void arm_smmu_atc_inv_master(struct arm_smmu_master *master, > + struct arm_smmu_cmdq_ent *cmd) > +{ > + int i; > + > + if (!master->ats_enabled) > + return; > + > + for (i = 0; i < master->num_sids; i++) { > + cmd->atc.sid = master->sids[i]; > + arm_smmu_cmdq_issue_cmd(master->smmu, cmd); > + } > + > + arm_smmu_cmdq_issue_sync(master->smmu); > +} > + > +static void arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, > + int ssid, unsigned long iova, size_t size) > +{ > + unsigned long flags; > + struct arm_smmu_cmdq_ent cmd; > + struct arm_smmu_master *master; > + > + if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) > + return; > + > + arm_smmu_atc_inv_to_cmd(ssid, iova, size, &cmd); > + > + spin_lock_irqsave(&smmu_domain->devices_lock, flags); > + list_for_each_entry(master, &smmu_domain->devices, domain_head) > + arm_smmu_atc_inv_master(master, &cmd); > + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); > +} > + > /* IO_PGTABLE API */ > static void arm_smmu_tlb_sync(void *cookie) > { > @@ -1726,6 +1850,45 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master) > } > } > > +static int arm_smmu_enable_ats(struct arm_smmu_master *master) > +{ > + int ret; > + size_t stu; > + struct pci_dev *pdev; > + struct arm_smmu_device *smmu = master->smmu; > + struct iommu_fwspec *fwspec = master->dev->iommu_fwspec; > + > + if (!(smmu->features & ARM_SMMU_FEAT_ATS) || !dev_is_pci(master->dev) || > + (fwspec->flags & IOMMU_FWSPEC_PCI_NO_ATS) || pci_ats_disabled()) > + return -ENOSYS; I'd probably make this -ENXIO. > + > + pdev = to_pci_dev(master->dev); > + if (pdev->untrusted) > + return -EPERM; > + > + /* Smallest Translation Unit: log2 of the smallest supported granule */ > + stu = __ffs(smmu->pgsize_bitmap); > + > + ret = pci_enable_ats(pdev, stu); > + if (ret) > + return ret; > + > + master->ats_enabled = true; > + dev_dbg(&pdev->dev, "enabled ATS (STU=%zu, QDEP=%d)\n", stu, > + pci_ats_queue_depth(pdev)); > + > + return 0; > +} > + > +static void arm_smmu_disable_ats(struct arm_smmu_master *master) > +{ > + if (!master->ats_enabled || !dev_is_pci(master->dev)) > + return; > + > + pci_disable_ats(to_pci_dev(master->dev)); > + master->ats_enabled = false; > +} > + > static void arm_smmu_detach_dev(struct arm_smmu_master *master) > { > unsigned long flags; > @@ -1740,6 +1903,9 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) > > master->domain = NULL; > arm_smmu_install_ste_for_dev(master); > + > + /* Disabling ATS invalidates all ATC entries */ > + arm_smmu_disable_ats(master); > } > > static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) > @@ -1783,6 +1949,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) > list_add(&master->domain_head, &smmu_domain->devices); > spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); > > + if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) > + arm_smmu_enable_ats(master); Do we care about the return value? Will ^ permalink raw reply [flat|nested] 64+ messages in thread
[parent not found: <20190415132121.GC15023-UDVVEH7NWB15pKCnmE3YQBJ8xKzm50AiAL8bYrjMMd8@public.gmane.org>]
* Re: [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-15 18:00 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-15 18:00 UTC (permalink / raw) To: Will Deacon Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, sudeep.holla-5wv7dgnIgG8, robin.murphy-5wv7dgnIgG8, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A On 15/04/2019 14:21, Will Deacon wrote: > On Tue, Apr 09, 2019 at 05:52:44PM +0100, Jean-Philippe Brucker wrote: >> PCIe devices can implement their own TLB, named Address Translation Cache >> (ATC). Enable Address Translation Service (ATS) for devices that support >> it and send them invalidation requests whenever we invalidate the IOTLBs. >> >> ATC invalidation is allowed to take up to 90 seconds, according to the >> PCIe spec, so it is possible to get a SMMU command queue timeout during >> normal operations. However we expect implementations to complete >> invalidation in reasonable time. >> >> We only enable ATS for "trusted" devices, and currently rely on the >> pci_dev->untrusted bit. For ATS we have to trust that: > > AFAICT, devicetree has no way to describe a device as untrusted, so > everything will be trusted by default on those systems. Is that right? Yes, although I'm adding a devicetree property for PCI in v5.2: https://lore.kernel.org/linux-pci/20190411211823.GU256045-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org/T/#m9f2c036748bab6e262233280b22c1e6fab4e1607 >> (a) The device doesn't issue "translated" memory requests for addresses >> that weren't returned by the SMMU in a Translation Completion. In >> particular, if we give control of a device or device partition to a VM >> or userspace, software cannot program the device to access arbitrary >> "translated" addresses. > > Any plans to look at split-stage ATS later on? I think that would allow > us to pass untrusted devices through to a VM behind S1 ATS. I haven't tested it so far, we can look at that after adding support for nested translation >> (b) The device follows permissions granted by the SMMU in a Translation >> Completion. If the device requested read+write permission and only >> got read, then it doesn't write. > > Guessing we just ignore execute permissions, or do we need read implies > exec? Without PASID, a function cannot ask for execute permission, only RO and RW. In this case execution by the endpoint is never permitted (because the Exe bit in an ATS completion is always zero). With PASID, the endpoint must explicitly ask for execute permission (and interestingly, can't obtain it if the page is mapped exec-only, because in ATS exec implies read.) [...] >> +static bool disable_ats_check; >> +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); >> +MODULE_PARM_DESC(disable_ats_check, >> + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); > > Yikes, do we really want this option, or is it just a leftover from > debugging? I wasn't sure what to do with it, I'll drop it in next version [...] >> @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) >> dev_err(smmu->dev, "retrying command fetch\n"); >> case CMDQ_ERR_CERROR_NONE_IDX: >> return; >> + case CMDQ_ERR_CERROR_ATC_INV_IDX: >> + /* >> + * ATC Invalidation Completion timeout. CONS is still pointing >> + * at the CMD_SYNC. Attempt to complete other pending commands >> + * by repeating the CMD_SYNC, though we might well end up back >> + * here since the ATC invalidation may still be pending. >> + */ >> + return; > > This is pretty bad, as it means we're unable to unmap a page safely from a > misbehaving device. Ideally, we'd block further transactions from the > problematic endpoint, but I suppose we can't easily know which one it was, > or inject a fault back into the unmap() path. > > Having said that, won't we get a CMD_SYNC timeout long before the ATC timeout? Yes, CMD_SYNC timeout is 1s, ATC timeout is 90s... > Perhaps we could propagate that out of arm_smmu_cmdq_issue_sync() and fail the > unmap? > > Not sure, what do you think? The callers of iommu_unmap() will free the page regardless of the return value, even though the device could still be accessing it. But I'll look at returning 0 if the CMD_SYNC times out, it's a good start for consolidating this. With dma-iommu.c it will trigger a WARN(). It gets a worse with PRI, when the invalidation comes from an MMU notifier and we can't even return an error. Ideally we'd hold a reference to these pages until invalidation completes. [...] >> + /* >> + * Find the smallest power of two that covers the range. Most >> + * significant differing bit between start and end address indicates the >> + * required span, ie. fls(start ^ end). For example: >> + * >> + * We want to invalidate pages [8; 11]. This is already the ideal range: >> + * x = 0b1000 ^ 0b1011 = 0b11 >> + * span = 1 << fls(x) = 4 >> + * >> + * To invalidate pages [7; 10], we need to invalidate [0; 15]: >> + * x = 0b0111 ^ 0b1010 = 0b1101 >> + * span = 1 << fls(x) = 16 >> + */ > > Urgh, "The Address span is aligned to its size by the SMMU" is what makes > this horrible. Please can you add that to the comment? Sure (but the culprit is really the PCIe spec, with its "naturally aligned" ranges.) > An alternative would be to emit multiple ATC_INV commands. Do you have a > feeling for what would be more efficient? With the current code, we might be removing cached entries of long-term mappings (tables and ring buffers) every time we unmap a buffer, in which case enabling ATS would certainly make the device slower. Multiple ATC_INV commands may be more efficient but I'm not too comfortable implementing it until I have some hardware to test it. I suspect we might need to optimize it a bit to avoid sending too many invalidations for large mappings. [...] >> +static int arm_smmu_enable_ats(struct arm_smmu_master *master) >> +{ >> + int ret; >> + size_t stu; >> + struct pci_dev *pdev; >> + struct arm_smmu_device *smmu = master->smmu; >> + struct iommu_fwspec *fwspec = master->dev->iommu_fwspec; >> + >> + if (!(smmu->features & ARM_SMMU_FEAT_ATS) || !dev_is_pci(master->dev) || >> + (fwspec->flags & IOMMU_FWSPEC_PCI_NO_ATS) || pci_ats_disabled()) >> + return -ENOSYS; > > I'd probably make this -ENXIO. Ok > >> + >> + pdev = to_pci_dev(master->dev); >> + if (pdev->untrusted) >> + return -EPERM; >> + >> + /* Smallest Translation Unit: log2 of the smallest supported granule */ >> + stu = __ffs(smmu->pgsize_bitmap); >> + >> + ret = pci_enable_ats(pdev, stu); >> + if (ret) >> + return ret; >> + >> + master->ats_enabled = true; >> + dev_dbg(&pdev->dev, "enabled ATS (STU=%zu, QDEP=%d)\n", stu, >> + pci_ats_queue_depth(pdev)); I'll also remove this. It's completely pointless since lspci gives us everything. >> + >> + return 0; >> +} >> + >> +static void arm_smmu_disable_ats(struct arm_smmu_master *master) >> +{ >> + if (!master->ats_enabled || !dev_is_pci(master->dev)) >> + return; >> + >> + pci_disable_ats(to_pci_dev(master->dev)); >> + master->ats_enabled = false; >> +} >> + >> static void arm_smmu_detach_dev(struct arm_smmu_master *master) >> { >> unsigned long flags; >> @@ -1740,6 +1903,9 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) >> >> master->domain = NULL; >> arm_smmu_install_ste_for_dev(master); >> + >> + /* Disabling ATS invalidates all ATC entries */ >> + arm_smmu_disable_ats(master); >> } >> >> static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) >> @@ -1783,6 +1949,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) >> list_add(&master->domain_head, &smmu_domain->devices); >> spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); >> >> + if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) >> + arm_smmu_enable_ats(master); > > Do we care about the return value? Not at the moment, I think. attach_dev() should succeed even if we can't enable ATS, since it's only an optimization. Thanks, Jean ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-15 18:00 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-15 18:00 UTC (permalink / raw) To: Will Deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb On 15/04/2019 14:21, Will Deacon wrote: > On Tue, Apr 09, 2019 at 05:52:44PM +0100, Jean-Philippe Brucker wrote: >> PCIe devices can implement their own TLB, named Address Translation Cache >> (ATC). Enable Address Translation Service (ATS) for devices that support >> it and send them invalidation requests whenever we invalidate the IOTLBs. >> >> ATC invalidation is allowed to take up to 90 seconds, according to the >> PCIe spec, so it is possible to get a SMMU command queue timeout during >> normal operations. However we expect implementations to complete >> invalidation in reasonable time. >> >> We only enable ATS for "trusted" devices, and currently rely on the >> pci_dev->untrusted bit. For ATS we have to trust that: > > AFAICT, devicetree has no way to describe a device as untrusted, so > everything will be trusted by default on those systems. Is that right? Yes, although I'm adding a devicetree property for PCI in v5.2: https://lore.kernel.org/linux-pci/20190411211823.GU256045@google.com/T/#m9f2c036748bab6e262233280b22c1e6fab4e1607 >> (a) The device doesn't issue "translated" memory requests for addresses >> that weren't returned by the SMMU in a Translation Completion. In >> particular, if we give control of a device or device partition to a VM >> or userspace, software cannot program the device to access arbitrary >> "translated" addresses. > > Any plans to look at split-stage ATS later on? I think that would allow > us to pass untrusted devices through to a VM behind S1 ATS. I haven't tested it so far, we can look at that after adding support for nested translation >> (b) The device follows permissions granted by the SMMU in a Translation >> Completion. If the device requested read+write permission and only >> got read, then it doesn't write. > > Guessing we just ignore execute permissions, or do we need read implies > exec? Without PASID, a function cannot ask for execute permission, only RO and RW. In this case execution by the endpoint is never permitted (because the Exe bit in an ATS completion is always zero). With PASID, the endpoint must explicitly ask for execute permission (and interestingly, can't obtain it if the page is mapped exec-only, because in ATS exec implies read.) [...] >> +static bool disable_ats_check; >> +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); >> +MODULE_PARM_DESC(disable_ats_check, >> + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); > > Yikes, do we really want this option, or is it just a leftover from > debugging? I wasn't sure what to do with it, I'll drop it in next version [...] >> @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) >> dev_err(smmu->dev, "retrying command fetch\n"); >> case CMDQ_ERR_CERROR_NONE_IDX: >> return; >> + case CMDQ_ERR_CERROR_ATC_INV_IDX: >> + /* >> + * ATC Invalidation Completion timeout. CONS is still pointing >> + * at the CMD_SYNC. Attempt to complete other pending commands >> + * by repeating the CMD_SYNC, though we might well end up back >> + * here since the ATC invalidation may still be pending. >> + */ >> + return; > > This is pretty bad, as it means we're unable to unmap a page safely from a > misbehaving device. Ideally, we'd block further transactions from the > problematic endpoint, but I suppose we can't easily know which one it was, > or inject a fault back into the unmap() path. > > Having said that, won't we get a CMD_SYNC timeout long before the ATC timeout? Yes, CMD_SYNC timeout is 1s, ATC timeout is 90s... > Perhaps we could propagate that out of arm_smmu_cmdq_issue_sync() and fail the > unmap? > > Not sure, what do you think? The callers of iommu_unmap() will free the page regardless of the return value, even though the device could still be accessing it. But I'll look at returning 0 if the CMD_SYNC times out, it's a good start for consolidating this. With dma-iommu.c it will trigger a WARN(). It gets a worse with PRI, when the invalidation comes from an MMU notifier and we can't even return an error. Ideally we'd hold a reference to these pages until invalidation completes. [...] >> + /* >> + * Find the smallest power of two that covers the range. Most >> + * significant differing bit between start and end address indicates the >> + * required span, ie. fls(start ^ end). For example: >> + * >> + * We want to invalidate pages [8; 11]. This is already the ideal range: >> + * x = 0b1000 ^ 0b1011 = 0b11 >> + * span = 1 << fls(x) = 4 >> + * >> + * To invalidate pages [7; 10], we need to invalidate [0; 15]: >> + * x = 0b0111 ^ 0b1010 = 0b1101 >> + * span = 1 << fls(x) = 16 >> + */ > > Urgh, "The Address span is aligned to its size by the SMMU" is what makes > this horrible. Please can you add that to the comment? Sure (but the culprit is really the PCIe spec, with its "naturally aligned" ranges.) > An alternative would be to emit multiple ATC_INV commands. Do you have a > feeling for what would be more efficient? With the current code, we might be removing cached entries of long-term mappings (tables and ring buffers) every time we unmap a buffer, in which case enabling ATS would certainly make the device slower. Multiple ATC_INV commands may be more efficient but I'm not too comfortable implementing it until I have some hardware to test it. I suspect we might need to optimize it a bit to avoid sending too many invalidations for large mappings. [...] >> +static int arm_smmu_enable_ats(struct arm_smmu_master *master) >> +{ >> + int ret; >> + size_t stu; >> + struct pci_dev *pdev; >> + struct arm_smmu_device *smmu = master->smmu; >> + struct iommu_fwspec *fwspec = master->dev->iommu_fwspec; >> + >> + if (!(smmu->features & ARM_SMMU_FEAT_ATS) || !dev_is_pci(master->dev) || >> + (fwspec->flags & IOMMU_FWSPEC_PCI_NO_ATS) || pci_ats_disabled()) >> + return -ENOSYS; > > I'd probably make this -ENXIO. Ok > >> + >> + pdev = to_pci_dev(master->dev); >> + if (pdev->untrusted) >> + return -EPERM; >> + >> + /* Smallest Translation Unit: log2 of the smallest supported granule */ >> + stu = __ffs(smmu->pgsize_bitmap); >> + >> + ret = pci_enable_ats(pdev, stu); >> + if (ret) >> + return ret; >> + >> + master->ats_enabled = true; >> + dev_dbg(&pdev->dev, "enabled ATS (STU=%zu, QDEP=%d)\n", stu, >> + pci_ats_queue_depth(pdev)); I'll also remove this. It's completely pointless since lspci gives us everything. >> + >> + return 0; >> +} >> + >> +static void arm_smmu_disable_ats(struct arm_smmu_master *master) >> +{ >> + if (!master->ats_enabled || !dev_is_pci(master->dev)) >> + return; >> + >> + pci_disable_ats(to_pci_dev(master->dev)); >> + master->ats_enabled = false; >> +} >> + >> static void arm_smmu_detach_dev(struct arm_smmu_master *master) >> { >> unsigned long flags; >> @@ -1740,6 +1903,9 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) >> >> master->domain = NULL; >> arm_smmu_install_ste_for_dev(master); >> + >> + /* Disabling ATS invalidates all ATC entries */ >> + arm_smmu_disable_ats(master); >> } >> >> static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) >> @@ -1783,6 +1949,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) >> list_add(&master->domain_head, &smmu_domain->devices); >> spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); >> >> + if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) >> + arm_smmu_enable_ats(master); > > Do we care about the return value? Not at the moment, I think. attach_dev() should succeed even if we can't enable ATS, since it's only an optimization. Thanks, Jean _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-15 18:00 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-15 18:00 UTC (permalink / raw) To: Will Deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb On 15/04/2019 14:21, Will Deacon wrote: > On Tue, Apr 09, 2019 at 05:52:44PM +0100, Jean-Philippe Brucker wrote: >> PCIe devices can implement their own TLB, named Address Translation Cache >> (ATC). Enable Address Translation Service (ATS) for devices that support >> it and send them invalidation requests whenever we invalidate the IOTLBs. >> >> ATC invalidation is allowed to take up to 90 seconds, according to the >> PCIe spec, so it is possible to get a SMMU command queue timeout during >> normal operations. However we expect implementations to complete >> invalidation in reasonable time. >> >> We only enable ATS for "trusted" devices, and currently rely on the >> pci_dev->untrusted bit. For ATS we have to trust that: > > AFAICT, devicetree has no way to describe a device as untrusted, so > everything will be trusted by default on those systems. Is that right? Yes, although I'm adding a devicetree property for PCI in v5.2: https://lore.kernel.org/linux-pci/20190411211823.GU256045@google.com/T/#m9f2c036748bab6e262233280b22c1e6fab4e1607 >> (a) The device doesn't issue "translated" memory requests for addresses >> that weren't returned by the SMMU in a Translation Completion. In >> particular, if we give control of a device or device partition to a VM >> or userspace, software cannot program the device to access arbitrary >> "translated" addresses. > > Any plans to look at split-stage ATS later on? I think that would allow > us to pass untrusted devices through to a VM behind S1 ATS. I haven't tested it so far, we can look at that after adding support for nested translation >> (b) The device follows permissions granted by the SMMU in a Translation >> Completion. If the device requested read+write permission and only >> got read, then it doesn't write. > > Guessing we just ignore execute permissions, or do we need read implies > exec? Without PASID, a function cannot ask for execute permission, only RO and RW. In this case execution by the endpoint is never permitted (because the Exe bit in an ATS completion is always zero). With PASID, the endpoint must explicitly ask for execute permission (and interestingly, can't obtain it if the page is mapped exec-only, because in ATS exec implies read.) [...] >> +static bool disable_ats_check; >> +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); >> +MODULE_PARM_DESC(disable_ats_check, >> + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); > > Yikes, do we really want this option, or is it just a leftover from > debugging? I wasn't sure what to do with it, I'll drop it in next version [...] >> @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) >> dev_err(smmu->dev, "retrying command fetch\n"); >> case CMDQ_ERR_CERROR_NONE_IDX: >> return; >> + case CMDQ_ERR_CERROR_ATC_INV_IDX: >> + /* >> + * ATC Invalidation Completion timeout. CONS is still pointing >> + * at the CMD_SYNC. Attempt to complete other pending commands >> + * by repeating the CMD_SYNC, though we might well end up back >> + * here since the ATC invalidation may still be pending. >> + */ >> + return; > > This is pretty bad, as it means we're unable to unmap a page safely from a > misbehaving device. Ideally, we'd block further transactions from the > problematic endpoint, but I suppose we can't easily know which one it was, > or inject a fault back into the unmap() path. > > Having said that, won't we get a CMD_SYNC timeout long before the ATC timeout? Yes, CMD_SYNC timeout is 1s, ATC timeout is 90s... > Perhaps we could propagate that out of arm_smmu_cmdq_issue_sync() and fail the > unmap? > > Not sure, what do you think? The callers of iommu_unmap() will free the page regardless of the return value, even though the device could still be accessing it. But I'll look at returning 0 if the CMD_SYNC times out, it's a good start for consolidating this. With dma-iommu.c it will trigger a WARN(). It gets a worse with PRI, when the invalidation comes from an MMU notifier and we can't even return an error. Ideally we'd hold a reference to these pages until invalidation completes. [...] >> + /* >> + * Find the smallest power of two that covers the range. Most >> + * significant differing bit between start and end address indicates the >> + * required span, ie. fls(start ^ end). For example: >> + * >> + * We want to invalidate pages [8; 11]. This is already the ideal range: >> + * x = 0b1000 ^ 0b1011 = 0b11 >> + * span = 1 << fls(x) = 4 >> + * >> + * To invalidate pages [7; 10], we need to invalidate [0; 15]: >> + * x = 0b0111 ^ 0b1010 = 0b1101 >> + * span = 1 << fls(x) = 16 >> + */ > > Urgh, "The Address span is aligned to its size by the SMMU" is what makes > this horrible. Please can you add that to the comment? Sure (but the culprit is really the PCIe spec, with its "naturally aligned" ranges.) > An alternative would be to emit multiple ATC_INV commands. Do you have a > feeling for what would be more efficient? With the current code, we might be removing cached entries of long-term mappings (tables and ring buffers) every time we unmap a buffer, in which case enabling ATS would certainly make the device slower. Multiple ATC_INV commands may be more efficient but I'm not too comfortable implementing it until I have some hardware to test it. I suspect we might need to optimize it a bit to avoid sending too many invalidations for large mappings. [...] >> +static int arm_smmu_enable_ats(struct arm_smmu_master *master) >> +{ >> + int ret; >> + size_t stu; >> + struct pci_dev *pdev; >> + struct arm_smmu_device *smmu = master->smmu; >> + struct iommu_fwspec *fwspec = master->dev->iommu_fwspec; >> + >> + if (!(smmu->features & ARM_SMMU_FEAT_ATS) || !dev_is_pci(master->dev) || >> + (fwspec->flags & IOMMU_FWSPEC_PCI_NO_ATS) || pci_ats_disabled()) >> + return -ENOSYS; > > I'd probably make this -ENXIO. Ok > >> + >> + pdev = to_pci_dev(master->dev); >> + if (pdev->untrusted) >> + return -EPERM; >> + >> + /* Smallest Translation Unit: log2 of the smallest supported granule */ >> + stu = __ffs(smmu->pgsize_bitmap); >> + >> + ret = pci_enable_ats(pdev, stu); >> + if (ret) >> + return ret; >> + >> + master->ats_enabled = true; >> + dev_dbg(&pdev->dev, "enabled ATS (STU=%zu, QDEP=%d)\n", stu, >> + pci_ats_queue_depth(pdev)); I'll also remove this. It's completely pointless since lspci gives us everything. >> + >> + return 0; >> +} >> + >> +static void arm_smmu_disable_ats(struct arm_smmu_master *master) >> +{ >> + if (!master->ats_enabled || !dev_is_pci(master->dev)) >> + return; >> + >> + pci_disable_ats(to_pci_dev(master->dev)); >> + master->ats_enabled = false; >> +} >> + >> static void arm_smmu_detach_dev(struct arm_smmu_master *master) >> { >> unsigned long flags; >> @@ -1740,6 +1903,9 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) >> >> master->domain = NULL; >> arm_smmu_install_ste_for_dev(master); >> + >> + /* Disabling ATS invalidates all ATC entries */ >> + arm_smmu_disable_ats(master); >> } >> >> static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) >> @@ -1783,6 +1949,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) >> list_add(&master->domain_head, &smmu_domain->devices); >> spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); >> >> + if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) >> + arm_smmu_enable_ats(master); > > Do we care about the return value? Not at the moment, I think. attach_dev() should succeed even if we can't enable ATS, since it's only an optimization. Thanks, Jean _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-15 18:00 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-15 18:00 UTC (permalink / raw) To: Will Deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb On 15/04/2019 14:21, Will Deacon wrote: > On Tue, Apr 09, 2019 at 05:52:44PM +0100, Jean-Philippe Brucker wrote: >> PCIe devices can implement their own TLB, named Address Translation Cache >> (ATC). Enable Address Translation Service (ATS) for devices that support >> it and send them invalidation requests whenever we invalidate the IOTLBs. >> >> ATC invalidation is allowed to take up to 90 seconds, according to the >> PCIe spec, so it is possible to get a SMMU command queue timeout during >> normal operations. However we expect implementations to complete >> invalidation in reasonable time. >> >> We only enable ATS for "trusted" devices, and currently rely on the >> pci_dev->untrusted bit. For ATS we have to trust that: > > AFAICT, devicetree has no way to describe a device as untrusted, so > everything will be trusted by default on those systems. Is that right? Yes, although I'm adding a devicetree property for PCI in v5.2: https://lore.kernel.org/linux-pci/20190411211823.GU256045@google.com/T/#m9f2c036748bab6e262233280b22c1e6fab4e1607 >> (a) The device doesn't issue "translated" memory requests for addresses >> that weren't returned by the SMMU in a Translation Completion. In >> particular, if we give control of a device or device partition to a VM >> or userspace, software cannot program the device to access arbitrary >> "translated" addresses. > > Any plans to look at split-stage ATS later on? I think that would allow > us to pass untrusted devices through to a VM behind S1 ATS. I haven't tested it so far, we can look at that after adding support for nested translation >> (b) The device follows permissions granted by the SMMU in a Translation >> Completion. If the device requested read+write permission and only >> got read, then it doesn't write. > > Guessing we just ignore execute permissions, or do we need read implies > exec? Without PASID, a function cannot ask for execute permission, only RO and RW. In this case execution by the endpoint is never permitted (because the Exe bit in an ATS completion is always zero). With PASID, the endpoint must explicitly ask for execute permission (and interestingly, can't obtain it if the page is mapped exec-only, because in ATS exec implies read.) [...] >> +static bool disable_ats_check; >> +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); >> +MODULE_PARM_DESC(disable_ats_check, >> + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); > > Yikes, do we really want this option, or is it just a leftover from > debugging? I wasn't sure what to do with it, I'll drop it in next version [...] >> @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) >> dev_err(smmu->dev, "retrying command fetch\n"); >> case CMDQ_ERR_CERROR_NONE_IDX: >> return; >> + case CMDQ_ERR_CERROR_ATC_INV_IDX: >> + /* >> + * ATC Invalidation Completion timeout. CONS is still pointing >> + * at the CMD_SYNC. Attempt to complete other pending commands >> + * by repeating the CMD_SYNC, though we might well end up back >> + * here since the ATC invalidation may still be pending. >> + */ >> + return; > > This is pretty bad, as it means we're unable to unmap a page safely from a > misbehaving device. Ideally, we'd block further transactions from the > problematic endpoint, but I suppose we can't easily know which one it was, > or inject a fault back into the unmap() path. > > Having said that, won't we get a CMD_SYNC timeout long before the ATC timeout? Yes, CMD_SYNC timeout is 1s, ATC timeout is 90s... > Perhaps we could propagate that out of arm_smmu_cmdq_issue_sync() and fail the > unmap? > > Not sure, what do you think? The callers of iommu_unmap() will free the page regardless of the return value, even though the device could still be accessing it. But I'll look at returning 0 if the CMD_SYNC times out, it's a good start for consolidating this. With dma-iommu.c it will trigger a WARN(). It gets a worse with PRI, when the invalidation comes from an MMU notifier and we can't even return an error. Ideally we'd hold a reference to these pages until invalidation completes. [...] >> + /* >> + * Find the smallest power of two that covers the range. Most >> + * significant differing bit between start and end address indicates the >> + * required span, ie. fls(start ^ end). For example: >> + * >> + * We want to invalidate pages [8; 11]. This is already the ideal range: >> + * x = 0b1000 ^ 0b1011 = 0b11 >> + * span = 1 << fls(x) = 4 >> + * >> + * To invalidate pages [7; 10], we need to invalidate [0; 15]: >> + * x = 0b0111 ^ 0b1010 = 0b1101 >> + * span = 1 << fls(x) = 16 >> + */ > > Urgh, "The Address span is aligned to its size by the SMMU" is what makes > this horrible. Please can you add that to the comment? Sure (but the culprit is really the PCIe spec, with its "naturally aligned" ranges.) > An alternative would be to emit multiple ATC_INV commands. Do you have a > feeling for what would be more efficient? With the current code, we might be removing cached entries of long-term mappings (tables and ring buffers) every time we unmap a buffer, in which case enabling ATS would certainly make the device slower. Multiple ATC_INV commands may be more efficient but I'm not too comfortable implementing it until I have some hardware to test it. I suspect we might need to optimize it a bit to avoid sending too many invalidations for large mappings. [...] >> +static int arm_smmu_enable_ats(struct arm_smmu_master *master) >> +{ >> + int ret; >> + size_t stu; >> + struct pci_dev *pdev; >> + struct arm_smmu_device *smmu = master->smmu; >> + struct iommu_fwspec *fwspec = master->dev->iommu_fwspec; >> + >> + if (!(smmu->features & ARM_SMMU_FEAT_ATS) || !dev_is_pci(master->dev) || >> + (fwspec->flags & IOMMU_FWSPEC_PCI_NO_ATS) || pci_ats_disabled()) >> + return -ENOSYS; > > I'd probably make this -ENXIO. Ok > >> + >> + pdev = to_pci_dev(master->dev); >> + if (pdev->untrusted) >> + return -EPERM; >> + >> + /* Smallest Translation Unit: log2 of the smallest supported granule */ >> + stu = __ffs(smmu->pgsize_bitmap); >> + >> + ret = pci_enable_ats(pdev, stu); >> + if (ret) >> + return ret; >> + >> + master->ats_enabled = true; >> + dev_dbg(&pdev->dev, "enabled ATS (STU=%zu, QDEP=%d)\n", stu, >> + pci_ats_queue_depth(pdev)); I'll also remove this. It's completely pointless since lspci gives us everything. >> + >> + return 0; >> +} >> + >> +static void arm_smmu_disable_ats(struct arm_smmu_master *master) >> +{ >> + if (!master->ats_enabled || !dev_is_pci(master->dev)) >> + return; >> + >> + pci_disable_ats(to_pci_dev(master->dev)); >> + master->ats_enabled = false; >> +} >> + >> static void arm_smmu_detach_dev(struct arm_smmu_master *master) >> { >> unsigned long flags; >> @@ -1740,6 +1903,9 @@ static void arm_smmu_detach_dev(struct arm_smmu_master *master) >> >> master->domain = NULL; >> arm_smmu_install_ste_for_dev(master); >> + >> + /* Disabling ATS invalidates all ATC entries */ >> + arm_smmu_disable_ats(master); >> } >> >> static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) >> @@ -1783,6 +1949,9 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) >> list_add(&master->domain_head, &smmu_domain->devices); >> spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); >> >> + if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) >> + arm_smmu_enable_ats(master); > > Do we care about the return value? Not at the moment, I think. attach_dev() should succeed even if we can't enable ATS, since it's only an optimization. Thanks, Jean ^ permalink raw reply [flat|nested] 64+ messages in thread
[parent not found: <0b9b600f-60e0-0740-e1db-6b684bf5a195-5wv7dgnIgG8@public.gmane.org>]
* Re: [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-16 10:00 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-16 10:00 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, sudeep.holla-5wv7dgnIgG8, robin.murphy-5wv7dgnIgG8, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A On Mon, Apr 15, 2019 at 07:00:27PM +0100, Jean-Philippe Brucker wrote: > On 15/04/2019 14:21, Will Deacon wrote: > > On Tue, Apr 09, 2019 at 05:52:44PM +0100, Jean-Philippe Brucker wrote: > >> PCIe devices can implement their own TLB, named Address Translation Cache > >> (ATC). Enable Address Translation Service (ATS) for devices that support > >> it and send them invalidation requests whenever we invalidate the IOTLBs. > >> > >> ATC invalidation is allowed to take up to 90 seconds, according to the > >> PCIe spec, so it is possible to get a SMMU command queue timeout during > >> normal operations. However we expect implementations to complete > >> invalidation in reasonable time. > >> > >> We only enable ATS for "trusted" devices, and currently rely on the > >> pci_dev->untrusted bit. For ATS we have to trust that: > > > > AFAICT, devicetree has no way to describe a device as untrusted, so > > everything will be trusted by default on those systems. Is that right? > > Yes, although I'm adding a devicetree property for PCI in v5.2: > https://lore.kernel.org/linux-pci/20190411211823.GU256045-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org/T/#m9f2c036748bab6e262233280b22c1e6fab4e1607 Perfect :) > >> (a) The device doesn't issue "translated" memory requests for addresses > >> that weren't returned by the SMMU in a Translation Completion. In > >> particular, if we give control of a device or device partition to a VM > >> or userspace, software cannot program the device to access arbitrary > >> "translated" addresses. > > > > Any plans to look at split-stage ATS later on? I think that would allow > > us to pass untrusted devices through to a VM behind S1 ATS. > > I haven't tested it so far, we can look at that after adding support for > nested translation Sure, just curious. Thanks. > > >> (b) The device follows permissions granted by the SMMU in a Translation > >> Completion. If the device requested read+write permission and only > >> got read, then it doesn't write. > > > > Guessing we just ignore execute permissions, or do we need read implies > > exec? > > Without PASID, a function cannot ask for execute permission, only RO and > RW. In this case execution by the endpoint is never permitted (because > the Exe bit in an ATS completion is always zero). > > With PASID, the endpoint must explicitly ask for execute permission (and > interestingly, can't obtain it if the page is mapped exec-only, because > in ATS exec implies read.) Got it, thanks again. > [...] > >> +static bool disable_ats_check; > >> +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); > >> +MODULE_PARM_DESC(disable_ats_check, > >> + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); > > > > Yikes, do we really want this option, or is it just a leftover from > > debugging? > > I wasn't sure what to do with it, I'll drop it in next version Cheers. > [...] > >> @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) > >> dev_err(smmu->dev, "retrying command fetch\n"); > >> case CMDQ_ERR_CERROR_NONE_IDX: > >> return; > >> + case CMDQ_ERR_CERROR_ATC_INV_IDX: > >> + /* > >> + * ATC Invalidation Completion timeout. CONS is still pointing > >> + * at the CMD_SYNC. Attempt to complete other pending commands > >> + * by repeating the CMD_SYNC, though we might well end up back > >> + * here since the ATC invalidation may still be pending. > >> + */ > >> + return; > > > > This is pretty bad, as it means we're unable to unmap a page safely from a > > misbehaving device. Ideally, we'd block further transactions from the > > problematic endpoint, but I suppose we can't easily know which one it was, > > or inject a fault back into the unmap() path. > > > > Having said that, won't we get a CMD_SYNC timeout long before the ATC timeout? > > Yes, CMD_SYNC timeout is 1s, ATC timeout is 90s... > > > Perhaps we could propagate that out of arm_smmu_cmdq_issue_sync() and fail the > > unmap? > > > > Not sure, what do you think? > > The callers of iommu_unmap() will free the page regardless of the return > value, even though the device could still be accessing it. But I'll look > at returning 0 if the CMD_SYNC times out, it's a good start for > consolidating this. With dma-iommu.c it will trigger a WARN(). If it's not too tricky, that would be good. > It gets a worse with PRI, when the invalidation comes from an MMU > notifier and we can't even return an error. Ideally we'd hold a > reference to these pages until invalidation completes. Agreed. > [...] > >> + /* > >> + * Find the smallest power of two that covers the range. Most > >> + * significant differing bit between start and end address indicates the > >> + * required span, ie. fls(start ^ end). For example: > >> + * > >> + * We want to invalidate pages [8; 11]. This is already the ideal range: > >> + * x = 0b1000 ^ 0b1011 = 0b11 > >> + * span = 1 << fls(x) = 4 > >> + * > >> + * To invalidate pages [7; 10], we need to invalidate [0; 15]: > >> + * x = 0b0111 ^ 0b1010 = 0b1101 > >> + * span = 1 << fls(x) = 16 > >> + */ > > > > Urgh, "The Address span is aligned to its size by the SMMU" is what makes > > this horrible. Please can you add that to the comment? > > Sure (but the culprit is really the PCIe spec, with its "naturally > aligned" ranges.) Ok, blame them then :) > > An alternative would be to emit multiple ATC_INV commands. Do you have a > > feeling for what would be more efficient? > > With the current code, we might be removing cached entries of long-term > mappings (tables and ring buffers) every time we unmap a buffer, in > which case enabling ATS would certainly make the device slower. > > Multiple ATC_INV commands may be more efficient but I'm not too > comfortable implementing it until I have some hardware to test it. I > suspect we might need to optimize it a bit to avoid sending too many > invalidations for large mappings. Ok, just stick that in the comment as well then, please. Will ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-16 10:00 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-16 10:00 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb On Mon, Apr 15, 2019 at 07:00:27PM +0100, Jean-Philippe Brucker wrote: > On 15/04/2019 14:21, Will Deacon wrote: > > On Tue, Apr 09, 2019 at 05:52:44PM +0100, Jean-Philippe Brucker wrote: > >> PCIe devices can implement their own TLB, named Address Translation Cache > >> (ATC). Enable Address Translation Service (ATS) for devices that support > >> it and send them invalidation requests whenever we invalidate the IOTLBs. > >> > >> ATC invalidation is allowed to take up to 90 seconds, according to the > >> PCIe spec, so it is possible to get a SMMU command queue timeout during > >> normal operations. However we expect implementations to complete > >> invalidation in reasonable time. > >> > >> We only enable ATS for "trusted" devices, and currently rely on the > >> pci_dev->untrusted bit. For ATS we have to trust that: > > > > AFAICT, devicetree has no way to describe a device as untrusted, so > > everything will be trusted by default on those systems. Is that right? > > Yes, although I'm adding a devicetree property for PCI in v5.2: > https://lore.kernel.org/linux-pci/20190411211823.GU256045@google.com/T/#m9f2c036748bab6e262233280b22c1e6fab4e1607 Perfect :) > >> (a) The device doesn't issue "translated" memory requests for addresses > >> that weren't returned by the SMMU in a Translation Completion. In > >> particular, if we give control of a device or device partition to a VM > >> or userspace, software cannot program the device to access arbitrary > >> "translated" addresses. > > > > Any plans to look at split-stage ATS later on? I think that would allow > > us to pass untrusted devices through to a VM behind S1 ATS. > > I haven't tested it so far, we can look at that after adding support for > nested translation Sure, just curious. Thanks. > > >> (b) The device follows permissions granted by the SMMU in a Translation > >> Completion. If the device requested read+write permission and only > >> got read, then it doesn't write. > > > > Guessing we just ignore execute permissions, or do we need read implies > > exec? > > Without PASID, a function cannot ask for execute permission, only RO and > RW. In this case execution by the endpoint is never permitted (because > the Exe bit in an ATS completion is always zero). > > With PASID, the endpoint must explicitly ask for execute permission (and > interestingly, can't obtain it if the page is mapped exec-only, because > in ATS exec implies read.) Got it, thanks again. > [...] > >> +static bool disable_ats_check; > >> +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); > >> +MODULE_PARM_DESC(disable_ats_check, > >> + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); > > > > Yikes, do we really want this option, or is it just a leftover from > > debugging? > > I wasn't sure what to do with it, I'll drop it in next version Cheers. > [...] > >> @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) > >> dev_err(smmu->dev, "retrying command fetch\n"); > >> case CMDQ_ERR_CERROR_NONE_IDX: > >> return; > >> + case CMDQ_ERR_CERROR_ATC_INV_IDX: > >> + /* > >> + * ATC Invalidation Completion timeout. CONS is still pointing > >> + * at the CMD_SYNC. Attempt to complete other pending commands > >> + * by repeating the CMD_SYNC, though we might well end up back > >> + * here since the ATC invalidation may still be pending. > >> + */ > >> + return; > > > > This is pretty bad, as it means we're unable to unmap a page safely from a > > misbehaving device. Ideally, we'd block further transactions from the > > problematic endpoint, but I suppose we can't easily know which one it was, > > or inject a fault back into the unmap() path. > > > > Having said that, won't we get a CMD_SYNC timeout long before the ATC timeout? > > Yes, CMD_SYNC timeout is 1s, ATC timeout is 90s... > > > Perhaps we could propagate that out of arm_smmu_cmdq_issue_sync() and fail the > > unmap? > > > > Not sure, what do you think? > > The callers of iommu_unmap() will free the page regardless of the return > value, even though the device could still be accessing it. But I'll look > at returning 0 if the CMD_SYNC times out, it's a good start for > consolidating this. With dma-iommu.c it will trigger a WARN(). If it's not too tricky, that would be good. > It gets a worse with PRI, when the invalidation comes from an MMU > notifier and we can't even return an error. Ideally we'd hold a > reference to these pages until invalidation completes. Agreed. > [...] > >> + /* > >> + * Find the smallest power of two that covers the range. Most > >> + * significant differing bit between start and end address indicates the > >> + * required span, ie. fls(start ^ end). For example: > >> + * > >> + * We want to invalidate pages [8; 11]. This is already the ideal range: > >> + * x = 0b1000 ^ 0b1011 = 0b11 > >> + * span = 1 << fls(x) = 4 > >> + * > >> + * To invalidate pages [7; 10], we need to invalidate [0; 15]: > >> + * x = 0b0111 ^ 0b1010 = 0b1101 > >> + * span = 1 << fls(x) = 16 > >> + */ > > > > Urgh, "The Address span is aligned to its size by the SMMU" is what makes > > this horrible. Please can you add that to the comment? > > Sure (but the culprit is really the PCIe spec, with its "naturally > aligned" ranges.) Ok, blame them then :) > > An alternative would be to emit multiple ATC_INV commands. Do you have a > > feeling for what would be more efficient? > > With the current code, we might be removing cached entries of long-term > mappings (tables and ring buffers) every time we unmap a buffer, in > which case enabling ATS would certainly make the device slower. > > Multiple ATC_INV commands may be more efficient but I'm not too > comfortable implementing it until I have some hardware to test it. I > suspect we might need to optimize it a bit to avoid sending too many > invalidations for large mappings. Ok, just stick that in the comment as well then, please. Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-16 10:00 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-16 10:00 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb On Mon, Apr 15, 2019 at 07:00:27PM +0100, Jean-Philippe Brucker wrote: > On 15/04/2019 14:21, Will Deacon wrote: > > On Tue, Apr 09, 2019 at 05:52:44PM +0100, Jean-Philippe Brucker wrote: > >> PCIe devices can implement their own TLB, named Address Translation Cache > >> (ATC). Enable Address Translation Service (ATS) for devices that support > >> it and send them invalidation requests whenever we invalidate the IOTLBs. > >> > >> ATC invalidation is allowed to take up to 90 seconds, according to the > >> PCIe spec, so it is possible to get a SMMU command queue timeout during > >> normal operations. However we expect implementations to complete > >> invalidation in reasonable time. > >> > >> We only enable ATS for "trusted" devices, and currently rely on the > >> pci_dev->untrusted bit. For ATS we have to trust that: > > > > AFAICT, devicetree has no way to describe a device as untrusted, so > > everything will be trusted by default on those systems. Is that right? > > Yes, although I'm adding a devicetree property for PCI in v5.2: > https://lore.kernel.org/linux-pci/20190411211823.GU256045@google.com/T/#m9f2c036748bab6e262233280b22c1e6fab4e1607 Perfect :) > >> (a) The device doesn't issue "translated" memory requests for addresses > >> that weren't returned by the SMMU in a Translation Completion. In > >> particular, if we give control of a device or device partition to a VM > >> or userspace, software cannot program the device to access arbitrary > >> "translated" addresses. > > > > Any plans to look at split-stage ATS later on? I think that would allow > > us to pass untrusted devices through to a VM behind S1 ATS. > > I haven't tested it so far, we can look at that after adding support for > nested translation Sure, just curious. Thanks. > > >> (b) The device follows permissions granted by the SMMU in a Translation > >> Completion. If the device requested read+write permission and only > >> got read, then it doesn't write. > > > > Guessing we just ignore execute permissions, or do we need read implies > > exec? > > Without PASID, a function cannot ask for execute permission, only RO and > RW. In this case execution by the endpoint is never permitted (because > the Exe bit in an ATS completion is always zero). > > With PASID, the endpoint must explicitly ask for execute permission (and > interestingly, can't obtain it if the page is mapped exec-only, because > in ATS exec implies read.) Got it, thanks again. > [...] > >> +static bool disable_ats_check; > >> +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); > >> +MODULE_PARM_DESC(disable_ats_check, > >> + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); > > > > Yikes, do we really want this option, or is it just a leftover from > > debugging? > > I wasn't sure what to do with it, I'll drop it in next version Cheers. > [...] > >> @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) > >> dev_err(smmu->dev, "retrying command fetch\n"); > >> case CMDQ_ERR_CERROR_NONE_IDX: > >> return; > >> + case CMDQ_ERR_CERROR_ATC_INV_IDX: > >> + /* > >> + * ATC Invalidation Completion timeout. CONS is still pointing > >> + * at the CMD_SYNC. Attempt to complete other pending commands > >> + * by repeating the CMD_SYNC, though we might well end up back > >> + * here since the ATC invalidation may still be pending. > >> + */ > >> + return; > > > > This is pretty bad, as it means we're unable to unmap a page safely from a > > misbehaving device. Ideally, we'd block further transactions from the > > problematic endpoint, but I suppose we can't easily know which one it was, > > or inject a fault back into the unmap() path. > > > > Having said that, won't we get a CMD_SYNC timeout long before the ATC timeout? > > Yes, CMD_SYNC timeout is 1s, ATC timeout is 90s... > > > Perhaps we could propagate that out of arm_smmu_cmdq_issue_sync() and fail the > > unmap? > > > > Not sure, what do you think? > > The callers of iommu_unmap() will free the page regardless of the return > value, even though the device could still be accessing it. But I'll look > at returning 0 if the CMD_SYNC times out, it's a good start for > consolidating this. With dma-iommu.c it will trigger a WARN(). If it's not too tricky, that would be good. > It gets a worse with PRI, when the invalidation comes from an MMU > notifier and we can't even return an error. Ideally we'd hold a > reference to these pages until invalidation completes. Agreed. > [...] > >> + /* > >> + * Find the smallest power of two that covers the range. Most > >> + * significant differing bit between start and end address indicates the > >> + * required span, ie. fls(start ^ end). For example: > >> + * > >> + * We want to invalidate pages [8; 11]. This is already the ideal range: > >> + * x = 0b1000 ^ 0b1011 = 0b11 > >> + * span = 1 << fls(x) = 4 > >> + * > >> + * To invalidate pages [7; 10], we need to invalidate [0; 15]: > >> + * x = 0b0111 ^ 0b1010 = 0b1101 > >> + * span = 1 << fls(x) = 16 > >> + */ > > > > Urgh, "The Address span is aligned to its size by the SMMU" is what makes > > this horrible. Please can you add that to the comment? > > Sure (but the culprit is really the PCIe spec, with its "naturally > aligned" ranges.) Ok, blame them then :) > > An alternative would be to emit multiple ATC_INV commands. Do you have a > > feeling for what would be more efficient? > > With the current code, we might be removing cached entries of long-term > mappings (tables and ring buffers) every time we unmap a buffer, in > which case enabling ATS would certainly make the device slower. > > Multiple ATC_INV commands may be more efficient but I'm not too > comfortable implementing it until I have some hardware to test it. I > suspect we might need to optimize it a bit to avoid sending too many > invalidations for large mappings. Ok, just stick that in the comment as well then, please. Will _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS @ 2019-04-16 10:00 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-16 10:00 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb On Mon, Apr 15, 2019 at 07:00:27PM +0100, Jean-Philippe Brucker wrote: > On 15/04/2019 14:21, Will Deacon wrote: > > On Tue, Apr 09, 2019 at 05:52:44PM +0100, Jean-Philippe Brucker wrote: > >> PCIe devices can implement their own TLB, named Address Translation Cache > >> (ATC). Enable Address Translation Service (ATS) for devices that support > >> it and send them invalidation requests whenever we invalidate the IOTLBs. > >> > >> ATC invalidation is allowed to take up to 90 seconds, according to the > >> PCIe spec, so it is possible to get a SMMU command queue timeout during > >> normal operations. However we expect implementations to complete > >> invalidation in reasonable time. > >> > >> We only enable ATS for "trusted" devices, and currently rely on the > >> pci_dev->untrusted bit. For ATS we have to trust that: > > > > AFAICT, devicetree has no way to describe a device as untrusted, so > > everything will be trusted by default on those systems. Is that right? > > Yes, although I'm adding a devicetree property for PCI in v5.2: > https://lore.kernel.org/linux-pci/20190411211823.GU256045@google.com/T/#m9f2c036748bab6e262233280b22c1e6fab4e1607 Perfect :) > >> (a) The device doesn't issue "translated" memory requests for addresses > >> that weren't returned by the SMMU in a Translation Completion. In > >> particular, if we give control of a device or device partition to a VM > >> or userspace, software cannot program the device to access arbitrary > >> "translated" addresses. > > > > Any plans to look at split-stage ATS later on? I think that would allow > > us to pass untrusted devices through to a VM behind S1 ATS. > > I haven't tested it so far, we can look at that after adding support for > nested translation Sure, just curious. Thanks. > > >> (b) The device follows permissions granted by the SMMU in a Translation > >> Completion. If the device requested read+write permission and only > >> got read, then it doesn't write. > > > > Guessing we just ignore execute permissions, or do we need read implies > > exec? > > Without PASID, a function cannot ask for execute permission, only RO and > RW. In this case execution by the endpoint is never permitted (because > the Exe bit in an ATS completion is always zero). > > With PASID, the endpoint must explicitly ask for execute permission (and > interestingly, can't obtain it if the page is mapped exec-only, because > in ATS exec implies read.) Got it, thanks again. > [...] > >> +static bool disable_ats_check; > >> +module_param_named(disable_ats_check, disable_ats_check, bool, S_IRUGO); > >> +MODULE_PARM_DESC(disable_ats_check, > >> + "By default, the SMMU checks whether each incoming transaction marked as translated is allowed by the stream configuration. This option disables the check."); > > > > Yikes, do we really want this option, or is it just a leftover from > > debugging? > > I wasn't sure what to do with it, I'll drop it in next version Cheers. > [...] > >> @@ -876,6 +911,14 @@ static void arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu) > >> dev_err(smmu->dev, "retrying command fetch\n"); > >> case CMDQ_ERR_CERROR_NONE_IDX: > >> return; > >> + case CMDQ_ERR_CERROR_ATC_INV_IDX: > >> + /* > >> + * ATC Invalidation Completion timeout. CONS is still pointing > >> + * at the CMD_SYNC. Attempt to complete other pending commands > >> + * by repeating the CMD_SYNC, though we might well end up back > >> + * here since the ATC invalidation may still be pending. > >> + */ > >> + return; > > > > This is pretty bad, as it means we're unable to unmap a page safely from a > > misbehaving device. Ideally, we'd block further transactions from the > > problematic endpoint, but I suppose we can't easily know which one it was, > > or inject a fault back into the unmap() path. > > > > Having said that, won't we get a CMD_SYNC timeout long before the ATC timeout? > > Yes, CMD_SYNC timeout is 1s, ATC timeout is 90s... > > > Perhaps we could propagate that out of arm_smmu_cmdq_issue_sync() and fail the > > unmap? > > > > Not sure, what do you think? > > The callers of iommu_unmap() will free the page regardless of the return > value, even though the device could still be accessing it. But I'll look > at returning 0 if the CMD_SYNC times out, it's a good start for > consolidating this. With dma-iommu.c it will trigger a WARN(). If it's not too tricky, that would be good. > It gets a worse with PRI, when the invalidation comes from an MMU > notifier and we can't even return an error. Ideally we'd hold a > reference to these pages until invalidation completes. Agreed. > [...] > >> + /* > >> + * Find the smallest power of two that covers the range. Most > >> + * significant differing bit between start and end address indicates the > >> + * required span, ie. fls(start ^ end). For example: > >> + * > >> + * We want to invalidate pages [8; 11]. This is already the ideal range: > >> + * x = 0b1000 ^ 0b1011 = 0b11 > >> + * span = 1 << fls(x) = 4 > >> + * > >> + * To invalidate pages [7; 10], we need to invalidate [0; 15]: > >> + * x = 0b0111 ^ 0b1010 = 0b1101 > >> + * span = 1 << fls(x) = 16 > >> + */ > > > > Urgh, "The Address span is aligned to its size by the SMMU" is what makes > > this horrible. Please can you add that to the comment? > > Sure (but the culprit is really the PCIe spec, with its "naturally > aligned" ranges.) Ok, blame them then :) > > An alternative would be to emit multiple ATC_INV commands. Do you have a > > feeling for what would be more efficient? > > With the current code, we might be removing cached entries of long-term > mappings (tables and ring buffers) every time we unmap a buffer, in > which case enabling ATS would certainly make the device slower. > > Multiple ATC_INV commands may be more efficient but I'm not too > comfortable implementing it until I have some hardware to test it. I > suspect we might need to optimize it a bit to avoid sending too many > invalidations for large mappings. Ok, just stick that in the comment as well then, please. Will ^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v2 7/7] iommu/arm-smmu-v3: Disable tagged pointers @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon-5wv7dgnIgG8 Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, sudeep.holla-5wv7dgnIgG8, robin.murphy-5wv7dgnIgG8, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A The ARM architecture has a "Top Byte Ignore" (TBI) option that makes the MMU mask out bits [63:56] of an address, allowing a userspace application to store data in its pointers. This option is incompatible with PCI ATS. If TBI is enabled in the SMMU and userspace triggers DMA transactions on tagged pointers, the endpoint might create ATC entries for addresses that include a tag. Software would then have to send ATC invalidation packets for each 255 possible alias of an address, or just wipe the whole address space. This is not a viable option, so disable TBI. The impact of this change is unclear, since there are very few users of tagged pointers, much less SVA. But the requirement introduced by this patch doesn't seem excessive: a userspace application using both tagged pointers and SVA should now sanitize addresses (clear the tag) before using them for device DMA. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> --- drivers/iommu/arm-smmu-v3.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 7819cd60d08b..811dd7d83bf0 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -1061,7 +1061,6 @@ static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr) val |= ARM_SMMU_TCR2CD(tcr, EPD0); val |= ARM_SMMU_TCR2CD(tcr, EPD1); val |= ARM_SMMU_TCR2CD(tcr, IPS); - val |= ARM_SMMU_TCR2CD(tcr, TBI0); return val; } -- 2.21.0 ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 7/7] iommu/arm-smmu-v3: Disable tagged pointers @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: lorenzo.pieralisi, eric.auger, zhongmiao, okaya, joro, rjw, linux-acpi, iommu, hanjun.guo, sudeep.holla, robin.murphy, linux-arm-kernel, lenb The ARM architecture has a "Top Byte Ignore" (TBI) option that makes the MMU mask out bits [63:56] of an address, allowing a userspace application to store data in its pointers. This option is incompatible with PCI ATS. If TBI is enabled in the SMMU and userspace triggers DMA transactions on tagged pointers, the endpoint might create ATC entries for addresses that include a tag. Software would then have to send ATC invalidation packets for each 255 possible alias of an address, or just wipe the whole address space. This is not a viable option, so disable TBI. The impact of this change is unclear, since there are very few users of tagged pointers, much less SVA. But the requirement introduced by this patch doesn't seem excessive: a userspace application using both tagged pointers and SVA should now sanitize addresses (clear the tag) before using them for device DMA. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 7819cd60d08b..811dd7d83bf0 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -1061,7 +1061,6 @@ static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr) val |= ARM_SMMU_TCR2CD(tcr, EPD0); val |= ARM_SMMU_TCR2CD(tcr, EPD1); val |= ARM_SMMU_TCR2CD(tcr, IPS); - val |= ARM_SMMU_TCR2CD(tcr, TBI0); return val; } -- 2.21.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 7/7] iommu/arm-smmu-v3: Disable tagged pointers @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb The ARM architecture has a "Top Byte Ignore" (TBI) option that makes the MMU mask out bits [63:56] of an address, allowing a userspace application to store data in its pointers. This option is incompatible with PCI ATS. If TBI is enabled in the SMMU and userspace triggers DMA transactions on tagged pointers, the endpoint might create ATC entries for addresses that include a tag. Software would then have to send ATC invalidation packets for each 255 possible alias of an address, or just wipe the whole address space. This is not a viable option, so disable TBI. The impact of this change is unclear, since there are very few users of tagged pointers, much less SVA. But the requirement introduced by this patch doesn't seem excessive: a userspace application using both tagged pointers and SVA should now sanitize addresses (clear the tag) before using them for device DMA. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 7819cd60d08b..811dd7d83bf0 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -1061,7 +1061,6 @@ static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr) val |= ARM_SMMU_TCR2CD(tcr, EPD0); val |= ARM_SMMU_TCR2CD(tcr, EPD1); val |= ARM_SMMU_TCR2CD(tcr, IPS); - val |= ARM_SMMU_TCR2CD(tcr, TBI0); return val; } -- 2.21.0 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 7/7] iommu/arm-smmu-v3: Disable tagged pointers @ 2019-04-09 16:52 ` Jean-Philippe Brucker 0 siblings, 0 replies; 64+ messages in thread From: Jean-Philippe Brucker @ 2019-04-09 16:52 UTC (permalink / raw) To: will.deacon Cc: iommu, linux-arm-kernel, linux-acpi, robin.murphy, joro, hanjun.guo, lorenzo.pieralisi, sudeep.holla, rjw, lenb, okaya, zhongmiao, eric.auger The ARM architecture has a "Top Byte Ignore" (TBI) option that makes the MMU mask out bits [63:56] of an address, allowing a userspace application to store data in its pointers. This option is incompatible with PCI ATS. If TBI is enabled in the SMMU and userspace triggers DMA transactions on tagged pointers, the endpoint might create ATC entries for addresses that include a tag. Software would then have to send ATC invalidation packets for each 255 possible alias of an address, or just wipe the whole address space. This is not a viable option, so disable TBI. The impact of this change is unclear, since there are very few users of tagged pointers, much less SVA. But the requirement introduced by this patch doesn't seem excessive: a userspace application using both tagged pointers and SVA should now sanitize addresses (clear the tag) before using them for device DMA. Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> --- drivers/iommu/arm-smmu-v3.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 7819cd60d08b..811dd7d83bf0 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -1061,7 +1061,6 @@ static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr) val |= ARM_SMMU_TCR2CD(tcr, EPD0); val |= ARM_SMMU_TCR2CD(tcr, EPD1); val |= ARM_SMMU_TCR2CD(tcr, IPS); - val |= ARM_SMMU_TCR2CD(tcr, TBI0); return val; } -- 2.21.0 ^ permalink raw reply related [flat|nested] 64+ messages in thread
* Re: [PATCH v2 0/7] Add PCI ATS support to Arm SMMUv3 @ 2019-04-15 13:20 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-15 13:20 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: zhongmiao-C8/M+/jPZTeaMJb+Lgu22Q, okaya-DgEjT+Ai2ygdnm+yROfE0A, rjw-LthD3rsA81gm4RdzfppkhA, linux-acpi-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, sudeep.holla-5wv7dgnIgG8, robin.murphy-5wv7dgnIgG8, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, lenb-DgEjT+Ai2ygdnm+yROfE0A Hi Jean-Philippe, On Tue, Apr 09, 2019 at 05:52:38PM +0100, Jean-Philippe Brucker wrote: > This series enables PCI ATS in SMMUv3. Changes since v1 [1]: > > * Simplify the SMMU structures (patches 2-4 are new). > > * Don't enable ATS for devices that are attached to a bypass domain, > because in that case a translation request would cause F_BAD_ATS_TREQ. > Translation requests in that case cause F_BAD_ATS_TREQ. Enable ATS in > attach_dev() rather than add_device(). > > * Enable ATS for stage-1 and stage-2 STEs alike. There is no benefit to > disabling ATS for stage-2 domains. > > [1] https://www.spinics.net/lists/arm-kernel/msg714628.html > > Jean-Philippe Brucker (7): > ACPI/IORT: Check ATS capability in root complex nodes > iommu/arm-smmu-v3: Rename arm_smmu_master_data to arm_smmu_master > iommu/arm-smmu-v3: Store SteamIDs in master > iommu/arm-smmu-v3: Add a master->domain pointer > iommu/arm-smmu-v3: Link domains and devices > iommu/arm-smmu-v3: Add support for PCI ATS > iommu/arm-smmu-v3: Disable tagged pointers Patches 2, 3, 4, 5 and 7 look fine to me. I've left some comments on the other two. Cheers, Will ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 0/7] Add PCI ATS support to Arm SMMUv3 @ 2019-04-15 13:20 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-15 13:20 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: lorenzo.pieralisi, eric.auger, zhongmiao, okaya, joro, rjw, linux-acpi, iommu, hanjun.guo, sudeep.holla, robin.murphy, linux-arm-kernel, lenb Hi Jean-Philippe, On Tue, Apr 09, 2019 at 05:52:38PM +0100, Jean-Philippe Brucker wrote: > This series enables PCI ATS in SMMUv3. Changes since v1 [1]: > > * Simplify the SMMU structures (patches 2-4 are new). > > * Don't enable ATS for devices that are attached to a bypass domain, > because in that case a translation request would cause F_BAD_ATS_TREQ. > Translation requests in that case cause F_BAD_ATS_TREQ. Enable ATS in > attach_dev() rather than add_device(). > > * Enable ATS for stage-1 and stage-2 STEs alike. There is no benefit to > disabling ATS for stage-2 domains. > > [1] https://www.spinics.net/lists/arm-kernel/msg714628.html > > Jean-Philippe Brucker (7): > ACPI/IORT: Check ATS capability in root complex nodes > iommu/arm-smmu-v3: Rename arm_smmu_master_data to arm_smmu_master > iommu/arm-smmu-v3: Store SteamIDs in master > iommu/arm-smmu-v3: Add a master->domain pointer > iommu/arm-smmu-v3: Link domains and devices > iommu/arm-smmu-v3: Add support for PCI ATS > iommu/arm-smmu-v3: Disable tagged pointers Patches 2, 3, 4, 5 and 7 look fine to me. I've left some comments on the other two. Cheers, Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 0/7] Add PCI ATS support to Arm SMMUv3 @ 2019-04-15 13:20 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-15 13:20 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: zhongmiao, okaya, rjw, linux-acpi, iommu, sudeep.holla, robin.murphy, linux-arm-kernel, lenb Hi Jean-Philippe, On Tue, Apr 09, 2019 at 05:52:38PM +0100, Jean-Philippe Brucker wrote: > This series enables PCI ATS in SMMUv3. Changes since v1 [1]: > > * Simplify the SMMU structures (patches 2-4 are new). > > * Don't enable ATS for devices that are attached to a bypass domain, > because in that case a translation request would cause F_BAD_ATS_TREQ. > Translation requests in that case cause F_BAD_ATS_TREQ. Enable ATS in > attach_dev() rather than add_device(). > > * Enable ATS for stage-1 and stage-2 STEs alike. There is no benefit to > disabling ATS for stage-2 domains. > > [1] https://www.spinics.net/lists/arm-kernel/msg714628.html > > Jean-Philippe Brucker (7): > ACPI/IORT: Check ATS capability in root complex nodes > iommu/arm-smmu-v3: Rename arm_smmu_master_data to arm_smmu_master > iommu/arm-smmu-v3: Store SteamIDs in master > iommu/arm-smmu-v3: Add a master->domain pointer > iommu/arm-smmu-v3: Link domains and devices > iommu/arm-smmu-v3: Add support for PCI ATS > iommu/arm-smmu-v3: Disable tagged pointers Patches 2, 3, 4, 5 and 7 look fine to me. I've left some comments on the other two. Cheers, Will _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 0/7] Add PCI ATS support to Arm SMMUv3 @ 2019-04-15 13:20 ` Will Deacon 0 siblings, 0 replies; 64+ messages in thread From: Will Deacon @ 2019-04-15 13:20 UTC (permalink / raw) To: Jean-Philippe Brucker Cc: iommu, linux-arm-kernel, linux-acpi, robin.murphy, joro, hanjun.guo, lorenzo.pieralisi, sudeep.holla, rjw, lenb, okaya, zhongmiao, eric.auger Hi Jean-Philippe, On Tue, Apr 09, 2019 at 05:52:38PM +0100, Jean-Philippe Brucker wrote: > This series enables PCI ATS in SMMUv3. Changes since v1 [1]: > > * Simplify the SMMU structures (patches 2-4 are new). > > * Don't enable ATS for devices that are attached to a bypass domain, > because in that case a translation request would cause F_BAD_ATS_TREQ. > Translation requests in that case cause F_BAD_ATS_TREQ. Enable ATS in > attach_dev() rather than add_device(). > > * Enable ATS for stage-1 and stage-2 STEs alike. There is no benefit to > disabling ATS for stage-2 domains. > > [1] https://www.spinics.net/lists/arm-kernel/msg714628.html > > Jean-Philippe Brucker (7): > ACPI/IORT: Check ATS capability in root complex nodes > iommu/arm-smmu-v3: Rename arm_smmu_master_data to arm_smmu_master > iommu/arm-smmu-v3: Store SteamIDs in master > iommu/arm-smmu-v3: Add a master->domain pointer > iommu/arm-smmu-v3: Link domains and devices > iommu/arm-smmu-v3: Add support for PCI ATS > iommu/arm-smmu-v3: Disable tagged pointers Patches 2, 3, 4, 5 and 7 look fine to me. I've left some comments on the other two. Cheers, Will ^ permalink raw reply [flat|nested] 64+ messages in thread
end of thread, other threads:[~2019-04-16 16:28 UTC | newest] Thread overview: 64+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-04-09 16:52 [PATCH v2 0/7] Add PCI ATS support to Arm SMMUv3 Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker [not found] ` <20190409165245.26500-1-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> 2019-04-09 16:52 ` [PATCH v2 1/7] ACPI/IORT: Check ATS capability in root complex nodes Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker [not found] ` <20190409165245.26500-2-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> 2019-04-15 13:21 ` Will Deacon 2019-04-15 13:21 ` Will Deacon 2019-04-15 13:21 ` Will Deacon 2019-04-15 13:21 ` Will Deacon [not found] ` <20190415132108.GB15023-UDVVEH7NWB15pKCnmE3YQBJ8xKzm50AiAL8bYrjMMd8@public.gmane.org> 2019-04-15 18:00 ` Jean-Philippe Brucker 2019-04-15 18:00 ` Jean-Philippe Brucker 2019-04-15 18:00 ` Jean-Philippe Brucker 2019-04-15 18:00 ` Jean-Philippe Brucker 2019-04-15 18:31 ` Robin Murphy 2019-04-15 18:31 ` Robin Murphy 2019-04-15 18:31 ` Robin Murphy 2019-04-15 18:31 ` Robin Murphy [not found] ` <c10c7adb-c7f6-f8c6-05cc-f4f143427a2d-5wv7dgnIgG8@public.gmane.org> 2019-04-16 16:27 ` Jean-Philippe Brucker 2019-04-16 16:27 ` Jean-Philippe Brucker 2019-04-16 16:27 ` Jean-Philippe Brucker 2019-04-16 16:27 ` Jean-Philippe Brucker 2019-04-09 16:52 ` [PATCH v2 2/7] iommu/arm-smmu-v3: Rename arm_smmu_master_data to arm_smmu_master Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` [PATCH v2 3/7] iommu/arm-smmu-v3: Store SteamIDs in master Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` [PATCH v2 4/7] iommu/arm-smmu-v3: Add a master->domain pointer Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` [PATCH v2 5/7] iommu/arm-smmu-v3: Link domains and devices Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` [PATCH v2 6/7] iommu/arm-smmu-v3: Add support for PCI ATS Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker [not found] ` <20190409165245.26500-7-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> 2019-04-15 13:21 ` Will Deacon 2019-04-15 13:21 ` Will Deacon 2019-04-15 13:21 ` Will Deacon 2019-04-15 13:21 ` Will Deacon [not found] ` <20190415132121.GC15023-UDVVEH7NWB15pKCnmE3YQBJ8xKzm50AiAL8bYrjMMd8@public.gmane.org> 2019-04-15 18:00 ` Jean-Philippe Brucker 2019-04-15 18:00 ` Jean-Philippe Brucker 2019-04-15 18:00 ` Jean-Philippe Brucker 2019-04-15 18:00 ` Jean-Philippe Brucker [not found] ` <0b9b600f-60e0-0740-e1db-6b684bf5a195-5wv7dgnIgG8@public.gmane.org> 2019-04-16 10:00 ` Will Deacon 2019-04-16 10:00 ` Will Deacon 2019-04-16 10:00 ` Will Deacon 2019-04-16 10:00 ` Will Deacon 2019-04-09 16:52 ` [PATCH v2 7/7] iommu/arm-smmu-v3: Disable tagged pointers Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-09 16:52 ` Jean-Philippe Brucker 2019-04-15 13:20 ` [PATCH v2 0/7] Add PCI ATS support to Arm SMMUv3 Will Deacon 2019-04-15 13:20 ` Will Deacon 2019-04-15 13:20 ` Will Deacon 2019-04-15 13:20 ` Will Deacon
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.