All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jacob Pan <jacob.jun.pan@linux.intel.com>
To: iommu@lists.linux-foundation.org,
	LKML <linux-kernel@vger.kernel.org>,
	Joerg Roedel <joro@8bytes.org>, Jason Gunthorpe <jgg@nvidia.com>,
	"Christoph Hellwig" <hch@infradead.org>,
	"Lu Baolu" <baolu.lu@linux.intel.com>,
	Jean-Philippe Brucker <jean-philippe@linaro.com>
Cc: "Tian, Kevin" <kevin.tian@intel.com>,
	Tony Luck <tony.luck@intel.com>,
	Dave Jiang <dave.jiang@intel.com>,
	Raj Ashok <ashok.raj@intel.com>,
	"Zanussi, Tom" <tom.zanussi@intel.com>,
	"Kumar, Sanjay K" <sanjay.k.kumar@intel.com>,
	Jacob Pan <jacob.jun.pan@intel.com>,
	Dan Williams <dan.j.williams@intel.com>
Subject: [PATCH v2 6/8] dmaengine: idxd: Use DMA API for in-kernel DMA with PASID
Date: Mon, 14 Mar 2022 22:07:10 -0700	[thread overview]
Message-ID: <20220315050713.2000518-7-jacob.jun.pan@linux.intel.com> (raw)
In-Reply-To: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com>

The current in-kernel supervisor PASID support is based on the SVM/SVA
machinery in SVA lib. The binding between a kernel PASID and kernel
mapping has many flaws. See discussions in the link below.

This patch enables in-kernel DMA by switching from SVA lib to the
standard DMA mapping APIs. Since both DMA requests with and without
PASIDs are mapped identically, there is no change to how DMA APIs are
used after the kernel PASID is enabled.

Link: https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.com/
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
---
 drivers/dma/idxd/idxd.h  |  1 -
 drivers/dma/idxd/init.c  | 34 +++++++++-------------------------
 drivers/dma/idxd/sysfs.c |  7 -------
 3 files changed, 9 insertions(+), 33 deletions(-)

diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h
index da72eb15f610..a09ab4a6e1c1 100644
--- a/drivers/dma/idxd/idxd.h
+++ b/drivers/dma/idxd/idxd.h
@@ -276,7 +276,6 @@ struct idxd_device {
 	struct idxd_wq **wqs;
 	struct idxd_engine **engines;
 
-	struct iommu_sva *sva;
 	unsigned int pasid;
 
 	int num_groups;
diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c
index 08a5f4310188..5d1f8dd4abf6 100644
--- a/drivers/dma/idxd/init.c
+++ b/drivers/dma/idxd/init.c
@@ -16,6 +16,7 @@
 #include <linux/idr.h>
 #include <linux/intel-svm.h>
 #include <linux/iommu.h>
+#include <linux/dma-iommu.h>
 #include <uapi/linux/idxd.h>
 #include <linux/dmaengine.h>
 #include "../dmaengine.h"
@@ -466,36 +467,22 @@ static struct idxd_device *idxd_alloc(struct pci_dev *pdev, struct idxd_driver_d
 
 static int idxd_enable_system_pasid(struct idxd_device *idxd)
 {
-	int flags;
-	unsigned int pasid;
-	struct iommu_sva *sva;
+	u32 pasid;
+	int ret;
 
-	flags = SVM_FLAG_SUPERVISOR_MODE;
-
-	sva = iommu_sva_bind_device(&idxd->pdev->dev, NULL, &flags);
-	if (IS_ERR(sva)) {
-		dev_warn(&idxd->pdev->dev,
-			 "iommu sva bind failed: %ld\n", PTR_ERR(sva));
-		return PTR_ERR(sva);
-	}
-
-	pasid = iommu_sva_get_pasid(sva);
-	if (pasid == IOMMU_PASID_INVALID) {
-		iommu_sva_unbind_device(sva);
-		return -ENODEV;
+	ret = iommu_enable_pasid_dma(&idxd->pdev->dev, &pasid);
+	if (ret) {
+		dev_err(&idxd->pdev->dev, "No DMA PASID %d\n", ret);
+		return ret;
 	}
-
-	idxd->sva = sva;
 	idxd->pasid = pasid;
-	dev_dbg(&idxd->pdev->dev, "system pasid: %u\n", pasid);
+
 	return 0;
 }
 
 static void idxd_disable_system_pasid(struct idxd_device *idxd)
 {
-
-	iommu_sva_unbind_device(idxd->sva);
-	idxd->sva = NULL;
+	iommu_disable_pasid_dma(&idxd->pdev->dev, idxd->pasid);
 }
 
 static int idxd_probe(struct idxd_device *idxd)
@@ -524,10 +511,7 @@ static int idxd_probe(struct idxd_device *idxd)
 		} else {
 			dev_warn(dev, "Unable to turn on SVA feature.\n");
 		}
-	} else if (!sva) {
-		dev_warn(dev, "User forced SVA off via module param.\n");
 	}
-
 	idxd_read_caps(idxd);
 	idxd_read_table_offsets(idxd);
 
diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c
index 7e19ab92b61a..fde6656695ba 100644
--- a/drivers/dma/idxd/sysfs.c
+++ b/drivers/dma/idxd/sysfs.c
@@ -839,13 +839,6 @@ static ssize_t wq_name_store(struct device *dev,
 	if (strlen(buf) > WQ_NAME_SIZE || strlen(buf) == 0)
 		return -EINVAL;
 
-	/*
-	 * This is temporarily placed here until we have SVM support for
-	 * dmaengine.
-	 */
-	if (wq->type == IDXD_WQT_KERNEL && device_pasid_enabled(wq->idxd))
-		return -EOPNOTSUPP;
-
 	memset(wq->name, 0, WQ_NAME_SIZE + 1);
 	strncpy(wq->name, buf, WQ_NAME_SIZE);
 	strreplace(wq->name, '\n', '\0');
-- 
2.25.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

WARNING: multiple messages have this Message-ID (diff)
From: Jacob Pan <jacob.jun.pan@linux.intel.com>
To: iommu@lists.linux-foundation.org,
	LKML <linux-kernel@vger.kernel.org>,
	Joerg Roedel <joro@8bytes.org>, Jason Gunthorpe <jgg@nvidia.com>,
	"Christoph Hellwig" <hch@infradead.org>,
	"Lu Baolu" <baolu.lu@linux.intel.com>,
	Jean-Philippe Brucker <jean-philippe@linaro.com>
Cc: Jacob Pan <jacob.jun.pan@intel.com>,
	Raj Ashok <ashok.raj@intel.com>,
	"Kumar, Sanjay K" <sanjay.k.kumar@intel.com>,
	Dave Jiang <dave.jiang@intel.com>,
	Tony Luck <tony.luck@intel.com>,
	"Zanussi, Tom" <tom.zanussi@intel.com>,
	Dan Williams <dan.j.williams@intel.com>,
	"Tian, Kevin" <kevin.tian@intel.com>, Yi Liu <yi.l.liu@intel.com>
Subject: [PATCH v2 6/8] dmaengine: idxd: Use DMA API for in-kernel DMA with PASID
Date: Mon, 14 Mar 2022 22:07:10 -0700	[thread overview]
Message-ID: <20220315050713.2000518-7-jacob.jun.pan@linux.intel.com> (raw)
In-Reply-To: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com>

The current in-kernel supervisor PASID support is based on the SVM/SVA
machinery in SVA lib. The binding between a kernel PASID and kernel
mapping has many flaws. See discussions in the link below.

This patch enables in-kernel DMA by switching from SVA lib to the
standard DMA mapping APIs. Since both DMA requests with and without
PASIDs are mapped identically, there is no change to how DMA APIs are
used after the kernel PASID is enabled.

Link: https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.com/
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
---
 drivers/dma/idxd/idxd.h  |  1 -
 drivers/dma/idxd/init.c  | 34 +++++++++-------------------------
 drivers/dma/idxd/sysfs.c |  7 -------
 3 files changed, 9 insertions(+), 33 deletions(-)

diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h
index da72eb15f610..a09ab4a6e1c1 100644
--- a/drivers/dma/idxd/idxd.h
+++ b/drivers/dma/idxd/idxd.h
@@ -276,7 +276,6 @@ struct idxd_device {
 	struct idxd_wq **wqs;
 	struct idxd_engine **engines;
 
-	struct iommu_sva *sva;
 	unsigned int pasid;
 
 	int num_groups;
diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c
index 08a5f4310188..5d1f8dd4abf6 100644
--- a/drivers/dma/idxd/init.c
+++ b/drivers/dma/idxd/init.c
@@ -16,6 +16,7 @@
 #include <linux/idr.h>
 #include <linux/intel-svm.h>
 #include <linux/iommu.h>
+#include <linux/dma-iommu.h>
 #include <uapi/linux/idxd.h>
 #include <linux/dmaengine.h>
 #include "../dmaengine.h"
@@ -466,36 +467,22 @@ static struct idxd_device *idxd_alloc(struct pci_dev *pdev, struct idxd_driver_d
 
 static int idxd_enable_system_pasid(struct idxd_device *idxd)
 {
-	int flags;
-	unsigned int pasid;
-	struct iommu_sva *sva;
+	u32 pasid;
+	int ret;
 
-	flags = SVM_FLAG_SUPERVISOR_MODE;
-
-	sva = iommu_sva_bind_device(&idxd->pdev->dev, NULL, &flags);
-	if (IS_ERR(sva)) {
-		dev_warn(&idxd->pdev->dev,
-			 "iommu sva bind failed: %ld\n", PTR_ERR(sva));
-		return PTR_ERR(sva);
-	}
-
-	pasid = iommu_sva_get_pasid(sva);
-	if (pasid == IOMMU_PASID_INVALID) {
-		iommu_sva_unbind_device(sva);
-		return -ENODEV;
+	ret = iommu_enable_pasid_dma(&idxd->pdev->dev, &pasid);
+	if (ret) {
+		dev_err(&idxd->pdev->dev, "No DMA PASID %d\n", ret);
+		return ret;
 	}
-
-	idxd->sva = sva;
 	idxd->pasid = pasid;
-	dev_dbg(&idxd->pdev->dev, "system pasid: %u\n", pasid);
+
 	return 0;
 }
 
 static void idxd_disable_system_pasid(struct idxd_device *idxd)
 {
-
-	iommu_sva_unbind_device(idxd->sva);
-	idxd->sva = NULL;
+	iommu_disable_pasid_dma(&idxd->pdev->dev, idxd->pasid);
 }
 
 static int idxd_probe(struct idxd_device *idxd)
@@ -524,10 +511,7 @@ static int idxd_probe(struct idxd_device *idxd)
 		} else {
 			dev_warn(dev, "Unable to turn on SVA feature.\n");
 		}
-	} else if (!sva) {
-		dev_warn(dev, "User forced SVA off via module param.\n");
 	}
-
 	idxd_read_caps(idxd);
 	idxd_read_table_offsets(idxd);
 
diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c
index 7e19ab92b61a..fde6656695ba 100644
--- a/drivers/dma/idxd/sysfs.c
+++ b/drivers/dma/idxd/sysfs.c
@@ -839,13 +839,6 @@ static ssize_t wq_name_store(struct device *dev,
 	if (strlen(buf) > WQ_NAME_SIZE || strlen(buf) == 0)
 		return -EINVAL;
 
-	/*
-	 * This is temporarily placed here until we have SVM support for
-	 * dmaengine.
-	 */
-	if (wq->type == IDXD_WQT_KERNEL && device_pasid_enabled(wq->idxd))
-		return -EOPNOTSUPP;
-
 	memset(wq->name, 0, WQ_NAME_SIZE + 1);
 	strncpy(wq->name, buf, WQ_NAME_SIZE);
 	strreplace(wq->name, '\n', '\0');
-- 
2.25.1


  parent reply	other threads:[~2022-03-15  5:04 UTC|newest]

Thread overview: 122+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-15  5:07 [PATCH v2 0/8] Enable PASID for DMA API users Jacob Pan
2022-03-15  5:07 ` Jacob Pan
2022-03-15  5:07 ` [PATCH v2 1/8] iommu: Assign per device max PASID Jacob Pan
2022-03-15  5:07   ` Jacob Pan
2022-03-15  5:07 ` [PATCH v2 2/8] iommu: Add attach/detach_dev_pasid domain ops Jacob Pan
2022-03-15  5:07   ` Jacob Pan
2022-03-15 10:24   ` Tian, Kevin
2022-03-15 10:24     ` Tian, Kevin
2022-03-15 11:26   ` Jean-Philippe Brucker
2022-03-15 11:26     ` Jean-Philippe Brucker
2022-03-15 11:49     ` Tian, Kevin
2022-03-15 11:49       ` Tian, Kevin
2022-03-15 16:11       ` Jacob Pan
2022-03-15 16:11         ` Jacob Pan
2022-03-18 12:01       ` Lu Baolu
2022-03-18 12:01         ` Lu Baolu
2022-03-18 13:50         ` Jason Gunthorpe
2022-03-18 13:50           ` Jason Gunthorpe via iommu
2022-03-18 11:52   ` Lu Baolu
2022-03-18 11:52     ` Lu Baolu
2022-03-18 13:48     ` Jason Gunthorpe
2022-03-18 13:48       ` Jason Gunthorpe via iommu
2022-03-15  5:07 ` [PATCH v2 3/8] iommu/vt-d: Implement device_pasid domain attach ops Jacob Pan
2022-03-15  5:07   ` Jacob Pan
2022-03-15 10:33   ` Tian, Kevin
2022-03-15 10:33     ` Tian, Kevin
2022-03-15 22:23     ` Jacob Pan
2022-03-15 22:23       ` Jacob Pan
2022-03-15 14:33   ` Jason Gunthorpe
2022-03-15 14:33     ` Jason Gunthorpe via iommu
2022-03-15 22:36     ` Jacob Pan
2022-03-15 22:36       ` Jacob Pan
2022-03-15 23:04       ` Jason Gunthorpe
2022-03-15 23:04         ` Jason Gunthorpe via iommu
2022-03-16 20:50         ` Jacob Pan
2022-03-16 20:50           ` Jacob Pan
2022-03-16 22:15           ` Jason Gunthorpe
2022-03-16 22:15             ` Jason Gunthorpe via iommu
2022-03-16 22:23             ` Luck, Tony
2022-03-16 22:23               ` Luck, Tony
2022-03-17  0:04               ` Jason Gunthorpe
2022-03-17  0:04                 ` Jason Gunthorpe via iommu
2022-03-18  5:47                 ` Tian, Kevin
2022-03-18  5:47                   ` Tian, Kevin
2022-03-18 13:47                   ` Jason Gunthorpe
2022-03-18 13:47                     ` Jason Gunthorpe via iommu
2022-03-17  0:49             ` Jacob Pan
2022-03-17  0:49               ` Jacob Pan
2022-03-17 13:23               ` Jason Gunthorpe
2022-03-17 13:23                 ` Jason Gunthorpe via iommu
2022-03-17 18:23                 ` Jacob Pan
2022-03-17 18:23                   ` Jacob Pan
2022-03-16  7:41     ` Tian, Kevin
2022-03-16  7:41       ` Tian, Kevin
2022-03-16 21:01       ` Jacob Pan
2022-03-16 21:01         ` Jacob Pan
2022-03-18  5:33         ` Tian, Kevin
2022-03-18  5:33           ` Tian, Kevin
2022-03-28 21:41           ` Jacob Pan
2022-03-28 21:41             ` Jacob Pan
2022-03-16  7:39   ` Tian, Kevin
2022-03-16  7:39     ` Tian, Kevin
2022-03-16 20:51     ` Jacob Pan
2022-03-16 20:51       ` Jacob Pan
2022-03-15  5:07 ` [PATCH v2 4/8] iommu/vt-d: Use device_pasid attach op for RID2PASID Jacob Pan
2022-03-15  5:07   ` Jacob Pan
2022-03-16  7:54   ` Tian, Kevin
2022-03-16  7:54     ` Tian, Kevin
2022-03-17 20:45     ` Jacob Pan
2022-03-17 20:45       ` Jacob Pan
2022-03-15  5:07 ` [PATCH v2 5/8] iommu: Add PASID support for DMA mapping API users Jacob Pan
2022-03-15  5:07   ` Jacob Pan
2022-03-15 11:16   ` Robin Murphy
2022-03-15 11:16     ` Robin Murphy
2022-03-15 14:22     ` Jason Gunthorpe
2022-03-15 14:22       ` Jason Gunthorpe via iommu
2022-03-15 16:31       ` Jacob Pan
2022-03-15 16:31         ` Jacob Pan
2022-03-15 17:05         ` Jason Gunthorpe
2022-03-15 17:05           ` Jason Gunthorpe via iommu
2022-03-15 21:24           ` Jacob Pan
2022-03-15 21:24             ` Jacob Pan
2022-03-16 10:32             ` Tian, Kevin
2022-03-16 10:32               ` Tian, Kevin
2022-03-16  8:41       ` Tian, Kevin
2022-03-16  8:41         ` Tian, Kevin
2022-03-16 14:07         ` Jason Gunthorpe
2022-03-16 14:07           ` Jason Gunthorpe via iommu
2022-03-15 14:35   ` Jason Gunthorpe
2022-03-15 14:35     ` Jason Gunthorpe via iommu
2022-03-15 16:38     ` Jacob Pan
2022-03-15 16:38       ` Jacob Pan
2022-03-15 23:05       ` Jason Gunthorpe
2022-03-15 23:05         ` Jason Gunthorpe via iommu
2022-03-18 12:43   ` Lu Baolu
2022-03-18 12:43     ` Lu Baolu
2022-03-28 21:44     ` Jacob Pan
2022-03-28 21:44       ` Jacob Pan
2022-03-15  5:07 ` Jacob Pan [this message]
2022-03-15  5:07   ` [PATCH v2 6/8] dmaengine: idxd: Use DMA API for in-kernel DMA with PASID Jacob Pan
2022-03-18  6:10   ` Tian, Kevin
2022-03-18  6:10     ` Tian, Kevin
2022-03-29 17:39     ` Jacob Pan
2022-03-29 17:39       ` Jacob Pan
2022-03-15  5:07 ` [PATCH v2 7/8] iommu/vt-d: Delete supervisor/kernel SVA Jacob Pan
2022-03-15  5:07   ` Jacob Pan
2022-03-18  6:16   ` Tian, Kevin
2022-03-18  6:16     ` Tian, Kevin
2022-03-29 17:42     ` Jacob Pan
2022-03-29 17:42       ` Jacob Pan
2022-03-15  5:07 ` [PATCH v2 8/8] iommu: Remove unused driver data in sva_bind_device Jacob Pan
2022-03-15  5:07   ` Jacob Pan
2022-03-15 11:37   ` Jean-Philippe Brucker
2022-03-15 11:37     ` Jean-Philippe Brucker
2022-03-15  5:07 ` [PATCH v2 9/9] dmaengine: idxd: separate user and kernel pasid enabling Jacob Pan
2022-03-15  5:07   ` Jacob Pan
2022-03-18  6:28   ` Tian, Kevin
2022-03-18  6:28     ` Tian, Kevin
2022-03-15  8:16 ` [PATCH v2 0/8] Enable PASID for DMA API users Tian, Kevin
2022-03-15  8:16   ` Tian, Kevin
2022-03-15 15:49   ` Jacob Pan
2022-03-15 15:49     ` Jacob Pan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220315050713.2000518-7-jacob.jun.pan@linux.intel.com \
    --to=jacob.jun.pan@linux.intel.com \
    --cc=ashok.raj@intel.com \
    --cc=baolu.lu@linux.intel.com \
    --cc=dan.j.williams@intel.com \
    --cc=dave.jiang@intel.com \
    --cc=hch@infradead.org \
    --cc=iommu@lists.linux-foundation.org \
    --cc=jacob.jun.pan@intel.com \
    --cc=jean-philippe@linaro.com \
    --cc=jgg@nvidia.com \
    --cc=joro@8bytes.org \
    --cc=kevin.tian@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=sanjay.k.kumar@intel.com \
    --cc=tom.zanussi@intel.com \
    --cc=tony.luck@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.