From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Tian, Kevin" Subject: RE: [PATCH 01/37] iommu: Introduce Shared Virtual Addressing API Date: Tue, 13 Feb 2018 07:31:51 +0000 Message-ID: References: <20180212183352.22730-1-jean-philippe.brucker@arm.com> <20180212183352.22730-2-jean-philippe.brucker@arm.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20180212183352.22730-2-jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Jean-Philippe Brucker , "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" , "linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "linux-acpi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "devicetree-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org" , "kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" Cc: "mark.rutland-5wv7dgnIgG8@public.gmane.org" , "bharatku-gjFFaj9aHVfQT0dZR+AlfA@public.gmane.org" , "Raj, Ashok" , "shunyong.yang-PT9Dzx9SjPiXmMXjJBpWqg@public.gmane.org" , "rjw-LthD3rsA81gm4RdzfppkhA@public.gmane.org" , "catalin.marinas-5wv7dgnIgG8@public.gmane.org" , "xuzaibo-hv44wF8Li93QT0dZR+AlfA@public.gmane.org" , "ilias.apalodimas-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org" , "will.deacon-5wv7dgnIgG8@public.gmane.org" , "okaya-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org" , "bhelgaas-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org" , "robh+dt-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org" , "sudeep.holla-5wv7dgnIgG8@public.gmane.org" , "rfranz-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org" , "dwmw2-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org" , "christian.koenig-5C7GfCeVMHo@public.gmane.org" , "lenb-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org" List-Id: linux-acpi@vger.kernel.org > From: Jean-Philippe Brucker > Sent: Tuesday, February 13, 2018 2:33 AM > > Shared Virtual Addressing (SVA) provides a way for device drivers to bind > process address spaces to devices. This requires the IOMMU to support the > same page table format as CPUs, and requires the system to support I/O "same" is a bit restrictive. "compatible" is better as you used in coverletter. :-) > Page Faults (IOPF) and Process Address Space ID (PASID). When all of these > are available, DMA can access virtual addresses of a process. A PASID is > allocated for each process, and the device driver programs it into the > device in an implementation-specific way. > > Add a new API for sharing process page tables with devices. Introduce two > IOMMU operations, sva_device_init() and sva_device_shutdown(), that > prepare the IOMMU driver for SVA. For example allocate PASID tables and > fault queues. Subsequent patches will implement the bind() and unbind() > operations. > > Signed-off-by: Jean-Philippe Brucker > --- > drivers/iommu/Kconfig | 10 ++++++ > drivers/iommu/Makefile | 1 + > drivers/iommu/iommu-sva.c | 90 > +++++++++++++++++++++++++++++++++++++++++++++++ > include/linux/iommu.h | 32 +++++++++++++++++ > 4 files changed, 133 insertions(+) > create mode 100644 drivers/iommu/iommu-sva.c > > diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig > index f3a21343e636..555147a61f7c 100644 > --- a/drivers/iommu/Kconfig > +++ b/drivers/iommu/Kconfig > @@ -74,6 +74,16 @@ config IOMMU_DMA > select IOMMU_IOVA > select NEED_SG_DMA_LENGTH > > +config IOMMU_SVA > + bool "Shared Virtual Addressing API for the IOMMU" > + select IOMMU_API > + help > + Enable process address space management for the IOMMU API. In > systems > + that support it, device drivers can bind process address spaces to > + devices and share their page tables using this API. "their page table" is a bit confusing here. > + > + If unsure, say N here. > + > config FSL_PAMU > bool "Freescale IOMMU support" > depends on PCI > diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile > index 1fb695854809..1dbcc89ebe4c 100644 > --- a/drivers/iommu/Makefile > +++ b/drivers/iommu/Makefile > @@ -3,6 +3,7 @@ obj-$(CONFIG_IOMMU_API) += iommu.o > obj-$(CONFIG_IOMMU_API) += iommu-traces.o > obj-$(CONFIG_IOMMU_API) += iommu-sysfs.o > obj-$(CONFIG_IOMMU_DMA) += dma-iommu.o > +obj-$(CONFIG_IOMMU_SVA) += iommu-sva.o > obj-$(CONFIG_IOMMU_IO_PGTABLE) += io-pgtable.o > obj-$(CONFIG_IOMMU_IO_PGTABLE_ARMV7S) += io-pgtable-arm-v7s.o > obj-$(CONFIG_IOMMU_IO_PGTABLE_LPAE) += io-pgtable-arm.o > diff --git a/drivers/iommu/iommu-sva.c b/drivers/iommu/iommu-sva.c > new file mode 100644 > index 000000000000..cab5d723520f > --- /dev/null > +++ b/drivers/iommu/iommu-sva.c > @@ -0,0 +1,90 @@ > +/* > + * Track processes address spaces bound to devices and allocate PASIDs. > + * > + * Copyright (C) 2018 ARM Ltd. > + * Author: Jean-Philippe Brucker > + * > + * SPDX-License-Identifier: GPL-2.0 > + */ > + > +#include > + > +/** > + * iommu_sva_device_init() - Initialize Shared Virtual Addressing for a > device > + * @dev: the device > + * @features: bitmask of features that need to be initialized > + * @max_pasid: max PASID value supported by the device > + * > + * Users of the bind()/unbind() API must call this function to initialize all > + * features required for SVA. > + * > + * - If the device should support multiple address spaces (e.g. PCI PASID), > + * IOMMU_SVA_FEAT_PASID must be requested. I think it is by default assumed when using this API, based on definition of SVA. Can you elaborate the situation where this flag can be cleared? > + * > + * By default the PASID allocated during bind() is limited by the IOMMU > + * capacity, and by the device PASID width defined in the PCI capability or > in > + * the firmware description. Setting @max_pasid to a non-zero value > smaller > + * than this limit overrides it. > + * > + * - If the device should support I/O Page Faults (e.g. PCI PRI), > + * IOMMU_SVA_FEAT_IOPF must be requested. > + * > + * The device should not be be performing any DMA while this function is remove double "be" > + * running. "otherwise the behavior is undefined" > + * > + * Return 0 if initialization succeeded, or an error. > + */ > +int iommu_sva_device_init(struct device *dev, unsigned long features, > + unsigned int max_pasid) > +{ > + int ret; > + unsigned int min_pasid = 0; > + struct iommu_param *dev_param = dev->iommu_param; > + struct iommu_domain *domain = > iommu_get_domain_for_dev(dev); > + > + if (!domain || !dev_param || !domain->ops->sva_device_init) > + return -ENODEV; > + > + /* > + * IOMMU driver updates the limits depending on the IOMMU and > device > + * capabilities. > + */ > + ret = domain->ops->sva_device_init(dev, features, &min_pasid, > + &max_pasid); > + if (ret) > + return ret; > + > + /* FIXME: racy. Next version should have a mutex (same as fault > handler) */ > + dev_param->sva_features = features; > + dev_param->min_pasid = min_pasid; > + dev_param->max_pasid = max_pasid; what's the point of min_pasid here? > + > + return 0; > +} > +EXPORT_SYMBOL_GPL(iommu_sva_device_init); > + > +/** > + * iommu_sva_device_shutdown() - Shutdown Shared Virtual Addressing > for a device > + * @dev: the device > + * > + * Disable SVA. The device should not be performing any DMA while this > function > + * is running. > + */ > +int iommu_sva_device_shutdown(struct device *dev) > +{ > + struct iommu_param *dev_param = dev->iommu_param; > + struct iommu_domain *domain = > iommu_get_domain_for_dev(dev); > + > + if (!domain) > + return -ENODEV; > + > + if (domain->ops->sva_device_shutdown) > + domain->ops->sva_device_shutdown(dev); > + > + dev_param->sva_features = 0; > + dev_param->min_pasid = 0; > + dev_param->max_pasid = 0; > + > + return 0; > +} > +EXPORT_SYMBOL_GPL(iommu_sva_device_shutdown); > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > index 66ef406396e9..e9e09eecdece 100644 > --- a/include/linux/iommu.h > +++ b/include/linux/iommu.h > @@ -60,6 +60,11 @@ typedef int (*iommu_fault_handler_t)(struct > iommu_domain *, > struct device *, unsigned long, int, void *); > typedef int (*iommu_dev_fault_handler_t)(struct iommu_fault_event *, > void *); > > +/* Request PASID support */ > +#define IOMMU_SVA_FEAT_PASID (1 << 0) > +/* Request I/O page fault support */ > +#define IOMMU_SVA_FEAT_IOPF (1 << 1) > + > struct iommu_domain_geometry { > dma_addr_t aperture_start; /* First address that can be mapped > */ > dma_addr_t aperture_end; /* Last address that can be mapped > */ > @@ -197,6 +202,8 @@ struct page_response_msg { > * @domain_free: free iommu domain > * @attach_dev: attach device to an iommu domain > * @detach_dev: detach device from an iommu domain > + * @sva_device_init: initialize Shared Virtual Adressing for a device > + * @sva_device_shutdown: shutdown Shared Virtual Adressing for a > device > * @map: map a physically contiguous memory region to an iommu > domain > * @unmap: unmap a physically contiguous memory region from an > iommu domain > * @map_sg: map a scatter-gather list of physically contiguous memory > chunks > @@ -230,6 +237,10 @@ struct iommu_ops { > > int (*attach_dev)(struct iommu_domain *domain, struct device > *dev); > void (*detach_dev)(struct iommu_domain *domain, struct device > *dev); > + int (*sva_device_init)(struct device *dev, unsigned long features, > + unsigned int *min_pasid, > + unsigned int *max_pasid); > + void (*sva_device_shutdown)(struct device *dev); > int (*map)(struct iommu_domain *domain, unsigned long iova, > phys_addr_t paddr, size_t size, int prot); > size_t (*unmap)(struct iommu_domain *domain, unsigned long > iova, > @@ -385,6 +396,9 @@ struct iommu_fault_param { > */ > struct iommu_param { > struct iommu_fault_param *fault_param; > + unsigned long sva_features; > + unsigned int min_pasid; > + unsigned int max_pasid; > }; > > int iommu_device_register(struct iommu_device *iommu); > @@ -878,4 +892,22 @@ const struct iommu_ops > *iommu_ops_from_fwnode(struct fwnode_handle *fwnode) > > #endif /* CONFIG_IOMMU_API */ > > +#ifdef CONFIG_IOMMU_SVA > +extern int iommu_sva_device_init(struct device *dev, unsigned long > features, > + unsigned int max_pasid); > +extern int iommu_sva_device_shutdown(struct device *dev); > +#else /* CONFIG_IOMMU_SVA */ > +static inline int iommu_sva_device_init(struct device *dev, > + unsigned long features, > + unsigned int max_pasid) > +{ > + return -ENODEV; > +} > + > +static inline int iommu_sva_device_shutdown(struct device *dev) > +{ > + return -ENODEV; > +} > +#endif /* CONFIG_IOMMU_SVA */ > + > #endif /* __LINUX_IOMMU_H */ > -- > 2.15.1 > > _______________________________________________ > iommu mailing list > iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Return-Path: From: "Tian, Kevin" To: Jean-Philippe Brucker , "linux-arm-kernel@lists.infradead.org" , "linux-pci@vger.kernel.org" , "linux-acpi@vger.kernel.org" , "devicetree@vger.kernel.org" , "iommu@lists.linux-foundation.org" , "kvm@vger.kernel.org" Subject: RE: [PATCH 01/37] iommu: Introduce Shared Virtual Addressing API Date: Tue, 13 Feb 2018 07:31:51 +0000 Message-ID: References: <20180212183352.22730-1-jean-philippe.brucker@arm.com> <20180212183352.22730-2-jean-philippe.brucker@arm.com> In-Reply-To: <20180212183352.22730-2-jean-philippe.brucker@arm.com> MIME-Version: 1.0 List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "mark.rutland@arm.com" , "bharatku@xilinx.com" , "Raj, Ashok" , "shunyong.yang@hxt-semitech.com" , "rjw@rjwysocki.net" , "catalin.marinas@arm.com" , "xuzaibo@huawei.com" , "ilias.apalodimas@linaro.org" , "will.deacon@arm.com" , "okaya@codeaurora.org" , "bhelgaas@google.com" , "robh+dt@kernel.org" , "sudeep.holla@arm.com" , "rfranz@cavium.com" , "dwmw2@infradead.org" , "christian.koenig@amd.com" , "lenb@kernel.org" Content-Type: text/plain; charset="us-ascii" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+bjorn=helgaas.com@lists.infradead.org List-ID: > From: Jean-Philippe Brucker > Sent: Tuesday, February 13, 2018 2:33 AM > > Shared Virtual Addressing (SVA) provides a way for device drivers to bind > process address spaces to devices. This requires the IOMMU to support the > same page table format as CPUs, and requires the system to support I/O "same" is a bit restrictive. "compatible" is better as you used in coverletter. :-) > Page Faults (IOPF) and Process Address Space ID (PASID). When all of these > are available, DMA can access virtual addresses of a process. A PASID is > allocated for each process, and the device driver programs it into the > device in an implementation-specific way. > > Add a new API for sharing process page tables with devices. Introduce two > IOMMU operations, sva_device_init() and sva_device_shutdown(), that > prepare the IOMMU driver for SVA. For example allocate PASID tables and > fault queues. Subsequent patches will implement the bind() and unbind() > operations. > > Signed-off-by: Jean-Philippe Brucker > --- > drivers/iommu/Kconfig | 10 ++++++ > drivers/iommu/Makefile | 1 + > drivers/iommu/iommu-sva.c | 90 > +++++++++++++++++++++++++++++++++++++++++++++++ > include/linux/iommu.h | 32 +++++++++++++++++ > 4 files changed, 133 insertions(+) > create mode 100644 drivers/iommu/iommu-sva.c > > diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig > index f3a21343e636..555147a61f7c 100644 > --- a/drivers/iommu/Kconfig > +++ b/drivers/iommu/Kconfig > @@ -74,6 +74,16 @@ config IOMMU_DMA > select IOMMU_IOVA > select NEED_SG_DMA_LENGTH > > +config IOMMU_SVA > + bool "Shared Virtual Addressing API for the IOMMU" > + select IOMMU_API > + help > + Enable process address space management for the IOMMU API. In > systems > + that support it, device drivers can bind process address spaces to > + devices and share their page tables using this API. "their page table" is a bit confusing here. > + > + If unsure, say N here. > + > config FSL_PAMU > bool "Freescale IOMMU support" > depends on PCI > diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile > index 1fb695854809..1dbcc89ebe4c 100644 > --- a/drivers/iommu/Makefile > +++ b/drivers/iommu/Makefile > @@ -3,6 +3,7 @@ obj-$(CONFIG_IOMMU_API) += iommu.o > obj-$(CONFIG_IOMMU_API) += iommu-traces.o > obj-$(CONFIG_IOMMU_API) += iommu-sysfs.o > obj-$(CONFIG_IOMMU_DMA) += dma-iommu.o > +obj-$(CONFIG_IOMMU_SVA) += iommu-sva.o > obj-$(CONFIG_IOMMU_IO_PGTABLE) += io-pgtable.o > obj-$(CONFIG_IOMMU_IO_PGTABLE_ARMV7S) += io-pgtable-arm-v7s.o > obj-$(CONFIG_IOMMU_IO_PGTABLE_LPAE) += io-pgtable-arm.o > diff --git a/drivers/iommu/iommu-sva.c b/drivers/iommu/iommu-sva.c > new file mode 100644 > index 000000000000..cab5d723520f > --- /dev/null > +++ b/drivers/iommu/iommu-sva.c > @@ -0,0 +1,90 @@ > +/* > + * Track processes address spaces bound to devices and allocate PASIDs. > + * > + * Copyright (C) 2018 ARM Ltd. > + * Author: Jean-Philippe Brucker > + * > + * SPDX-License-Identifier: GPL-2.0 > + */ > + > +#include > + > +/** > + * iommu_sva_device_init() - Initialize Shared Virtual Addressing for a > device > + * @dev: the device > + * @features: bitmask of features that need to be initialized > + * @max_pasid: max PASID value supported by the device > + * > + * Users of the bind()/unbind() API must call this function to initialize all > + * features required for SVA. > + * > + * - If the device should support multiple address spaces (e.g. PCI PASID), > + * IOMMU_SVA_FEAT_PASID must be requested. I think it is by default assumed when using this API, based on definition of SVA. Can you elaborate the situation where this flag can be cleared? > + * > + * By default the PASID allocated during bind() is limited by the IOMMU > + * capacity, and by the device PASID width defined in the PCI capability or > in > + * the firmware description. Setting @max_pasid to a non-zero value > smaller > + * than this limit overrides it. > + * > + * - If the device should support I/O Page Faults (e.g. PCI PRI), > + * IOMMU_SVA_FEAT_IOPF must be requested. > + * > + * The device should not be be performing any DMA while this function is remove double "be" > + * running. "otherwise the behavior is undefined" > + * > + * Return 0 if initialization succeeded, or an error. > + */ > +int iommu_sva_device_init(struct device *dev, unsigned long features, > + unsigned int max_pasid) > +{ > + int ret; > + unsigned int min_pasid = 0; > + struct iommu_param *dev_param = dev->iommu_param; > + struct iommu_domain *domain = > iommu_get_domain_for_dev(dev); > + > + if (!domain || !dev_param || !domain->ops->sva_device_init) > + return -ENODEV; > + > + /* > + * IOMMU driver updates the limits depending on the IOMMU and > device > + * capabilities. > + */ > + ret = domain->ops->sva_device_init(dev, features, &min_pasid, > + &max_pasid); > + if (ret) > + return ret; > + > + /* FIXME: racy. Next version should have a mutex (same as fault > handler) */ > + dev_param->sva_features = features; > + dev_param->min_pasid = min_pasid; > + dev_param->max_pasid = max_pasid; what's the point of min_pasid here? > + > + return 0; > +} > +EXPORT_SYMBOL_GPL(iommu_sva_device_init); > + > +/** > + * iommu_sva_device_shutdown() - Shutdown Shared Virtual Addressing > for a device > + * @dev: the device > + * > + * Disable SVA. The device should not be performing any DMA while this > function > + * is running. > + */ > +int iommu_sva_device_shutdown(struct device *dev) > +{ > + struct iommu_param *dev_param = dev->iommu_param; > + struct iommu_domain *domain = > iommu_get_domain_for_dev(dev); > + > + if (!domain) > + return -ENODEV; > + > + if (domain->ops->sva_device_shutdown) > + domain->ops->sva_device_shutdown(dev); > + > + dev_param->sva_features = 0; > + dev_param->min_pasid = 0; > + dev_param->max_pasid = 0; > + > + return 0; > +} > +EXPORT_SYMBOL_GPL(iommu_sva_device_shutdown); > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > index 66ef406396e9..e9e09eecdece 100644 > --- a/include/linux/iommu.h > +++ b/include/linux/iommu.h > @@ -60,6 +60,11 @@ typedef int (*iommu_fault_handler_t)(struct > iommu_domain *, > struct device *, unsigned long, int, void *); > typedef int (*iommu_dev_fault_handler_t)(struct iommu_fault_event *, > void *); > > +/* Request PASID support */ > +#define IOMMU_SVA_FEAT_PASID (1 << 0) > +/* Request I/O page fault support */ > +#define IOMMU_SVA_FEAT_IOPF (1 << 1) > + > struct iommu_domain_geometry { > dma_addr_t aperture_start; /* First address that can be mapped > */ > dma_addr_t aperture_end; /* Last address that can be mapped > */ > @@ -197,6 +202,8 @@ struct page_response_msg { > * @domain_free: free iommu domain > * @attach_dev: attach device to an iommu domain > * @detach_dev: detach device from an iommu domain > + * @sva_device_init: initialize Shared Virtual Adressing for a device > + * @sva_device_shutdown: shutdown Shared Virtual Adressing for a > device > * @map: map a physically contiguous memory region to an iommu > domain > * @unmap: unmap a physically contiguous memory region from an > iommu domain > * @map_sg: map a scatter-gather list of physically contiguous memory > chunks > @@ -230,6 +237,10 @@ struct iommu_ops { > > int (*attach_dev)(struct iommu_domain *domain, struct device > *dev); > void (*detach_dev)(struct iommu_domain *domain, struct device > *dev); > + int (*sva_device_init)(struct device *dev, unsigned long features, > + unsigned int *min_pasid, > + unsigned int *max_pasid); > + void (*sva_device_shutdown)(struct device *dev); > int (*map)(struct iommu_domain *domain, unsigned long iova, > phys_addr_t paddr, size_t size, int prot); > size_t (*unmap)(struct iommu_domain *domain, unsigned long > iova, > @@ -385,6 +396,9 @@ struct iommu_fault_param { > */ > struct iommu_param { > struct iommu_fault_param *fault_param; > + unsigned long sva_features; > + unsigned int min_pasid; > + unsigned int max_pasid; > }; > > int iommu_device_register(struct iommu_device *iommu); > @@ -878,4 +892,22 @@ const struct iommu_ops > *iommu_ops_from_fwnode(struct fwnode_handle *fwnode) > > #endif /* CONFIG_IOMMU_API */ > > +#ifdef CONFIG_IOMMU_SVA > +extern int iommu_sva_device_init(struct device *dev, unsigned long > features, > + unsigned int max_pasid); > +extern int iommu_sva_device_shutdown(struct device *dev); > +#else /* CONFIG_IOMMU_SVA */ > +static inline int iommu_sva_device_init(struct device *dev, > + unsigned long features, > + unsigned int max_pasid) > +{ > + return -ENODEV; > +} > + > +static inline int iommu_sva_device_shutdown(struct device *dev) > +{ > + return -ENODEV; > +} > +#endif /* CONFIG_IOMMU_SVA */ > + > #endif /* __LINUX_IOMMU_H */ > -- > 2.15.1 > > _______________________________________________ > iommu mailing list > iommu@lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 From: kevin.tian@intel.com (Tian, Kevin) Date: Tue, 13 Feb 2018 07:31:51 +0000 Subject: [PATCH 01/37] iommu: Introduce Shared Virtual Addressing API In-Reply-To: <20180212183352.22730-2-jean-philippe.brucker@arm.com> References: <20180212183352.22730-1-jean-philippe.brucker@arm.com> <20180212183352.22730-2-jean-philippe.brucker@arm.com> Message-ID: To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org > From: Jean-Philippe Brucker > Sent: Tuesday, February 13, 2018 2:33 AM > > Shared Virtual Addressing (SVA) provides a way for device drivers to bind > process address spaces to devices. This requires the IOMMU to support the > same page table format as CPUs, and requires the system to support I/O "same" is a bit restrictive. "compatible" is better as you used in coverletter. :-) > Page Faults (IOPF) and Process Address Space ID (PASID). When all of these > are available, DMA can access virtual addresses of a process. A PASID is > allocated for each process, and the device driver programs it into the > device in an implementation-specific way. > > Add a new API for sharing process page tables with devices. Introduce two > IOMMU operations, sva_device_init() and sva_device_shutdown(), that > prepare the IOMMU driver for SVA. For example allocate PASID tables and > fault queues. Subsequent patches will implement the bind() and unbind() > operations. > > Signed-off-by: Jean-Philippe Brucker > --- > drivers/iommu/Kconfig | 10 ++++++ > drivers/iommu/Makefile | 1 + > drivers/iommu/iommu-sva.c | 90 > +++++++++++++++++++++++++++++++++++++++++++++++ > include/linux/iommu.h | 32 +++++++++++++++++ > 4 files changed, 133 insertions(+) > create mode 100644 drivers/iommu/iommu-sva.c > > diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig > index f3a21343e636..555147a61f7c 100644 > --- a/drivers/iommu/Kconfig > +++ b/drivers/iommu/Kconfig > @@ -74,6 +74,16 @@ config IOMMU_DMA > select IOMMU_IOVA > select NEED_SG_DMA_LENGTH > > +config IOMMU_SVA > + bool "Shared Virtual Addressing API for the IOMMU" > + select IOMMU_API > + help > + Enable process address space management for the IOMMU API. In > systems > + that support it, device drivers can bind process address spaces to > + devices and share their page tables using this API. "their page table" is a bit confusing here. > + > + If unsure, say N here. > + > config FSL_PAMU > bool "Freescale IOMMU support" > depends on PCI > diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile > index 1fb695854809..1dbcc89ebe4c 100644 > --- a/drivers/iommu/Makefile > +++ b/drivers/iommu/Makefile > @@ -3,6 +3,7 @@ obj-$(CONFIG_IOMMU_API) += iommu.o > obj-$(CONFIG_IOMMU_API) += iommu-traces.o > obj-$(CONFIG_IOMMU_API) += iommu-sysfs.o > obj-$(CONFIG_IOMMU_DMA) += dma-iommu.o > +obj-$(CONFIG_IOMMU_SVA) += iommu-sva.o > obj-$(CONFIG_IOMMU_IO_PGTABLE) += io-pgtable.o > obj-$(CONFIG_IOMMU_IO_PGTABLE_ARMV7S) += io-pgtable-arm-v7s.o > obj-$(CONFIG_IOMMU_IO_PGTABLE_LPAE) += io-pgtable-arm.o > diff --git a/drivers/iommu/iommu-sva.c b/drivers/iommu/iommu-sva.c > new file mode 100644 > index 000000000000..cab5d723520f > --- /dev/null > +++ b/drivers/iommu/iommu-sva.c > @@ -0,0 +1,90 @@ > +/* > + * Track processes address spaces bound to devices and allocate PASIDs. > + * > + * Copyright (C) 2018 ARM Ltd. > + * Author: Jean-Philippe Brucker > + * > + * SPDX-License-Identifier: GPL-2.0 > + */ > + > +#include > + > +/** > + * iommu_sva_device_init() - Initialize Shared Virtual Addressing for a > device > + * @dev: the device > + * @features: bitmask of features that need to be initialized > + * @max_pasid: max PASID value supported by the device > + * > + * Users of the bind()/unbind() API must call this function to initialize all > + * features required for SVA. > + * > + * - If the device should support multiple address spaces (e.g. PCI PASID), > + * IOMMU_SVA_FEAT_PASID must be requested. I think it is by default assumed when using this API, based on definition of SVA. Can you elaborate the situation where this flag can be cleared? > + * > + * By default the PASID allocated during bind() is limited by the IOMMU > + * capacity, and by the device PASID width defined in the PCI capability or > in > + * the firmware description. Setting @max_pasid to a non-zero value > smaller > + * than this limit overrides it. > + * > + * - If the device should support I/O Page Faults (e.g. PCI PRI), > + * IOMMU_SVA_FEAT_IOPF must be requested. > + * > + * The device should not be be performing any DMA while this function is remove double "be" > + * running. "otherwise the behavior is undefined" > + * > + * Return 0 if initialization succeeded, or an error. > + */ > +int iommu_sva_device_init(struct device *dev, unsigned long features, > + unsigned int max_pasid) > +{ > + int ret; > + unsigned int min_pasid = 0; > + struct iommu_param *dev_param = dev->iommu_param; > + struct iommu_domain *domain = > iommu_get_domain_for_dev(dev); > + > + if (!domain || !dev_param || !domain->ops->sva_device_init) > + return -ENODEV; > + > + /* > + * IOMMU driver updates the limits depending on the IOMMU and > device > + * capabilities. > + */ > + ret = domain->ops->sva_device_init(dev, features, &min_pasid, > + &max_pasid); > + if (ret) > + return ret; > + > + /* FIXME: racy. Next version should have a mutex (same as fault > handler) */ > + dev_param->sva_features = features; > + dev_param->min_pasid = min_pasid; > + dev_param->max_pasid = max_pasid; what's the point of min_pasid here? > + > + return 0; > +} > +EXPORT_SYMBOL_GPL(iommu_sva_device_init); > + > +/** > + * iommu_sva_device_shutdown() - Shutdown Shared Virtual Addressing > for a device > + * @dev: the device > + * > + * Disable SVA. The device should not be performing any DMA while this > function > + * is running. > + */ > +int iommu_sva_device_shutdown(struct device *dev) > +{ > + struct iommu_param *dev_param = dev->iommu_param; > + struct iommu_domain *domain = > iommu_get_domain_for_dev(dev); > + > + if (!domain) > + return -ENODEV; > + > + if (domain->ops->sva_device_shutdown) > + domain->ops->sva_device_shutdown(dev); > + > + dev_param->sva_features = 0; > + dev_param->min_pasid = 0; > + dev_param->max_pasid = 0; > + > + return 0; > +} > +EXPORT_SYMBOL_GPL(iommu_sva_device_shutdown); > diff --git a/include/linux/iommu.h b/include/linux/iommu.h > index 66ef406396e9..e9e09eecdece 100644 > --- a/include/linux/iommu.h > +++ b/include/linux/iommu.h > @@ -60,6 +60,11 @@ typedef int (*iommu_fault_handler_t)(struct > iommu_domain *, > struct device *, unsigned long, int, void *); > typedef int (*iommu_dev_fault_handler_t)(struct iommu_fault_event *, > void *); > > +/* Request PASID support */ > +#define IOMMU_SVA_FEAT_PASID (1 << 0) > +/* Request I/O page fault support */ > +#define IOMMU_SVA_FEAT_IOPF (1 << 1) > + > struct iommu_domain_geometry { > dma_addr_t aperture_start; /* First address that can be mapped > */ > dma_addr_t aperture_end; /* Last address that can be mapped > */ > @@ -197,6 +202,8 @@ struct page_response_msg { > * @domain_free: free iommu domain > * @attach_dev: attach device to an iommu domain > * @detach_dev: detach device from an iommu domain > + * @sva_device_init: initialize Shared Virtual Adressing for a device > + * @sva_device_shutdown: shutdown Shared Virtual Adressing for a > device > * @map: map a physically contiguous memory region to an iommu > domain > * @unmap: unmap a physically contiguous memory region from an > iommu domain > * @map_sg: map a scatter-gather list of physically contiguous memory > chunks > @@ -230,6 +237,10 @@ struct iommu_ops { > > int (*attach_dev)(struct iommu_domain *domain, struct device > *dev); > void (*detach_dev)(struct iommu_domain *domain, struct device > *dev); > + int (*sva_device_init)(struct device *dev, unsigned long features, > + unsigned int *min_pasid, > + unsigned int *max_pasid); > + void (*sva_device_shutdown)(struct device *dev); > int (*map)(struct iommu_domain *domain, unsigned long iova, > phys_addr_t paddr, size_t size, int prot); > size_t (*unmap)(struct iommu_domain *domain, unsigned long > iova, > @@ -385,6 +396,9 @@ struct iommu_fault_param { > */ > struct iommu_param { > struct iommu_fault_param *fault_param; > + unsigned long sva_features; > + unsigned int min_pasid; > + unsigned int max_pasid; > }; > > int iommu_device_register(struct iommu_device *iommu); > @@ -878,4 +892,22 @@ const struct iommu_ops > *iommu_ops_from_fwnode(struct fwnode_handle *fwnode) > > #endif /* CONFIG_IOMMU_API */ > > +#ifdef CONFIG_IOMMU_SVA > +extern int iommu_sva_device_init(struct device *dev, unsigned long > features, > + unsigned int max_pasid); > +extern int iommu_sva_device_shutdown(struct device *dev); > +#else /* CONFIG_IOMMU_SVA */ > +static inline int iommu_sva_device_init(struct device *dev, > + unsigned long features, > + unsigned int max_pasid) > +{ > + return -ENODEV; > +} > + > +static inline int iommu_sva_device_shutdown(struct device *dev) > +{ > + return -ENODEV; > +} > +#endif /* CONFIG_IOMMU_SVA */ > + > #endif /* __LINUX_IOMMU_H */ > -- > 2.15.1 > > _______________________________________________ > iommu mailing list > iommu at lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu