From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DFD5C4167B for ; Wed, 18 May 2022 18:17:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241349AbiERSRk (ORCPT ); Wed, 18 May 2022 14:17:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50410 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241344AbiERSRj (ORCPT ); Wed, 18 May 2022 14:17:39 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1894E185412; Wed, 18 May 2022 11:17:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652897858; x=1684433858; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Bzje02YdvCwfEdXd4VLSz7VQ2pQZoJC0EJfVFm1lN80=; b=K9oLTWBYi/4hg6aJ8MivRJbc99JN3Pk/MYYPtsKpk3a7cROmqLtq2ITm mBW0E8arwCbNKI/Cdvkoe8YNRg9FmAVVoNg+2v0RxHYef8woiWS5KIT4f qRM21Wj1XsIy+hDYFUhuv9JpoGBAHEY9pbL7NtDljPc+8X8FFQxYU64gZ XRgXdjfnErXG7GxZnS0p0F2VyXsOBHBV7VNWufFsiKKySSfrYPSasfzpp SyjZlJqtTKkB/v7coLLyySl5lU/dxkGJu+Yn3Q65cdU8mxcc5uVlHJcLe R6CGGBHxq/Y/eklu8/dViscFOO7lYg2i+338H1/lX4csP5NJYjaPjXsHt A==; X-IronPort-AV: E=McAfee;i="6400,9594,10351"; a="270648744" X-IronPort-AV: E=Sophos;i="5.91,235,1647327600"; d="scan'208";a="270648744" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2022 11:17:33 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,235,1647327600"; d="scan'208";a="639405505" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by fmsmga004.fm.intel.com with ESMTP; 18 May 2022 11:17:33 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , dmaengine@vger.kernel.org, Joerg Roedel , David Woodhouse , Jean-Philippe Brucker , "Lu Baolu" , Jason Gunthorpe , "Christoph Hellwig" , vkoul@kernel.org, robin.murphy@arm.com, will@kernel.org Cc: Yi Liu , Dave Jiang , "Tian, Kevin" , Raj Ashok , Eric Auger , Jacob Pan Subject: [PATCH v4 5/6] dmaengine: idxd: Use DMA API for in-kernel DMA with PASID Date: Wed, 18 May 2022 11:21:19 -0700 Message-Id: <20220518182120.1136715-6-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220518182120.1136715-1-jacob.jun.pan@linux.intel.com> References: <20220518182120.1136715-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: dmaengine@vger.kernel.org The current in-kernel supervisor PASID support is based on the SVM/SVA machinery in SVA lib. The binding between a kernel PASID and kernel mapping has many flaws. See discussions in the link below. This patch enables in-kernel DMA by switching from SVA lib to the standard DMA mapping APIs. Since both DMA requests with and without PASIDs are mapped identically, there is no change to how DMA APIs are used after the kernel PASID is enabled. Link: https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.com/ Signed-off-by: Jacob Pan --- drivers/dma/idxd/idxd.h | 1 - drivers/dma/idxd/init.c | 34 +++++++++------------------------- drivers/dma/idxd/sysfs.c | 7 ------- 3 files changed, 9 insertions(+), 33 deletions(-) diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index ccbefd0be617..190b08bd7c08 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -277,7 +277,6 @@ struct idxd_device { struct idxd_wq **wqs; struct idxd_engine **engines; - struct iommu_sva *sva; unsigned int pasid; int num_groups; diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index e1b5d1e4a949..e2e1c0eae6d6 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include "../dmaengine.h" @@ -466,36 +467,22 @@ static struct idxd_device *idxd_alloc(struct pci_dev *pdev, struct idxd_driver_d static int idxd_enable_system_pasid(struct idxd_device *idxd) { - int flags; - unsigned int pasid; - struct iommu_sva *sva; + u32 pasid; + int ret; - flags = SVM_FLAG_SUPERVISOR_MODE; - - sva = iommu_sva_bind_device(&idxd->pdev->dev, NULL, &flags); - if (IS_ERR(sva)) { - dev_warn(&idxd->pdev->dev, - "iommu sva bind failed: %ld\n", PTR_ERR(sva)); - return PTR_ERR(sva); - } - - pasid = iommu_sva_get_pasid(sva); - if (pasid == IOMMU_PASID_INVALID) { - iommu_sva_unbind_device(sva); - return -ENODEV; + ret = iommu_attach_dma_pasid(&idxd->pdev->dev, &pasid); + if (ret) { + dev_err(&idxd->pdev->dev, "No DMA PASID %d\n", ret); + return ret; } - - idxd->sva = sva; idxd->pasid = pasid; - dev_dbg(&idxd->pdev->dev, "system pasid: %u\n", pasid); + return 0; } static void idxd_disable_system_pasid(struct idxd_device *idxd) { - - iommu_sva_unbind_device(idxd->sva); - idxd->sva = NULL; + iommu_detach_dma_pasid(&idxd->pdev->dev); } static int idxd_probe(struct idxd_device *idxd) @@ -527,10 +514,7 @@ static int idxd_probe(struct idxd_device *idxd) else set_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags); } - } else if (!sva) { - dev_warn(dev, "User forced SVA off via module param.\n"); } - idxd_read_caps(idxd); idxd_read_table_offsets(idxd); diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c index dfd549685c46..a48928973bd4 100644 --- a/drivers/dma/idxd/sysfs.c +++ b/drivers/dma/idxd/sysfs.c @@ -839,13 +839,6 @@ static ssize_t wq_name_store(struct device *dev, if (strlen(buf) > WQ_NAME_SIZE || strlen(buf) == 0) return -EINVAL; - /* - * This is temporarily placed here until we have SVM support for - * dmaengine. - */ - if (wq->type == IDXD_WQT_KERNEL && device_pasid_enabled(wq->idxd)) - return -EOPNOTSUPP; - memset(wq->name, 0, WQ_NAME_SIZE + 1); strncpy(wq->name, buf, WQ_NAME_SIZE); strreplace(wq->name, '\n', '\0'); -- 2.25.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 99186C4332F for ; Wed, 18 May 2022 18:17:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 45A3B4168B; Wed, 18 May 2022 18:17:43 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TN7oMLW1pvxS; Wed, 18 May 2022 18:17:42 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id CE86C416AC; Wed, 18 May 2022 18:17:41 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 55CF8C002D; Wed, 18 May 2022 18:17:41 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0B7D0C007C for ; Wed, 18 May 2022 18:17:40 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 70B9D61245 for ; Wed, 18 May 2022 18:17:36 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TCc2Zuo1Gzpl for ; Wed, 18 May 2022 18:17:35 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by smtp3.osuosl.org (Postfix) with ESMTPS id 852EA6124E for ; Wed, 18 May 2022 18:17:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652897855; x=1684433855; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Bzje02YdvCwfEdXd4VLSz7VQ2pQZoJC0EJfVFm1lN80=; b=gLrAxFVxl7c3bcCnfATO1EPD5h8oWsJQStZRFDzBJzAsjIqdxFJTqwIB 26rAv5eSroI5vtt3goAGvYdLo/Vf7SkfRFcWEvfKAuFPavdKkOPFrtHPa rKEZR3bkB3TyPFlRgNc084LWfnR4Ww2RxQOXHYB3SGx54HJ4ftJ02IG9W VmH2Ocfy1izuoPX44q4ox6eZms8m5oT0RkQNMDiVZQYFDvhtlNTZZGBUT Z4xal7GR59p0pEfpaHmutw8mRThmPolfgYyjpT3ZGEwb5oXgtiD9X2pLE svkTKBN04gIhjTq4Y3z9/MxgEvvBvL9f9/hXrMuH760bYFsPydUmqgA6e g==; X-IronPort-AV: E=McAfee;i="6400,9594,10351"; a="259392427" X-IronPort-AV: E=Sophos;i="5.91,235,1647327600"; d="scan'208";a="259392427" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2022 11:17:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,235,1647327600"; d="scan'208";a="639405505" Received: from otc-wp-03.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.79]) by fmsmga004.fm.intel.com with ESMTP; 18 May 2022 11:17:33 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , dmaengine@vger.kernel.org, Joerg Roedel , David Woodhouse , Jean-Philippe Brucker , "Lu Baolu" , Jason Gunthorpe , "Christoph Hellwig" , vkoul@kernel.org, robin.murphy@arm.com, will@kernel.org Subject: [PATCH v4 5/6] dmaengine: idxd: Use DMA API for in-kernel DMA with PASID Date: Wed, 18 May 2022 11:21:19 -0700 Message-Id: <20220518182120.1136715-6-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220518182120.1136715-1-jacob.jun.pan@linux.intel.com> References: <20220518182120.1136715-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Cc: "Tian, Kevin" , Dave Jiang , Raj Ashok X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" The current in-kernel supervisor PASID support is based on the SVM/SVA machinery in SVA lib. The binding between a kernel PASID and kernel mapping has many flaws. See discussions in the link below. This patch enables in-kernel DMA by switching from SVA lib to the standard DMA mapping APIs. Since both DMA requests with and without PASIDs are mapped identically, there is no change to how DMA APIs are used after the kernel PASID is enabled. Link: https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.com/ Signed-off-by: Jacob Pan --- drivers/dma/idxd/idxd.h | 1 - drivers/dma/idxd/init.c | 34 +++++++++------------------------- drivers/dma/idxd/sysfs.c | 7 ------- 3 files changed, 9 insertions(+), 33 deletions(-) diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h index ccbefd0be617..190b08bd7c08 100644 --- a/drivers/dma/idxd/idxd.h +++ b/drivers/dma/idxd/idxd.h @@ -277,7 +277,6 @@ struct idxd_device { struct idxd_wq **wqs; struct idxd_engine **engines; - struct iommu_sva *sva; unsigned int pasid; int num_groups; diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index e1b5d1e4a949..e2e1c0eae6d6 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include "../dmaengine.h" @@ -466,36 +467,22 @@ static struct idxd_device *idxd_alloc(struct pci_dev *pdev, struct idxd_driver_d static int idxd_enable_system_pasid(struct idxd_device *idxd) { - int flags; - unsigned int pasid; - struct iommu_sva *sva; + u32 pasid; + int ret; - flags = SVM_FLAG_SUPERVISOR_MODE; - - sva = iommu_sva_bind_device(&idxd->pdev->dev, NULL, &flags); - if (IS_ERR(sva)) { - dev_warn(&idxd->pdev->dev, - "iommu sva bind failed: %ld\n", PTR_ERR(sva)); - return PTR_ERR(sva); - } - - pasid = iommu_sva_get_pasid(sva); - if (pasid == IOMMU_PASID_INVALID) { - iommu_sva_unbind_device(sva); - return -ENODEV; + ret = iommu_attach_dma_pasid(&idxd->pdev->dev, &pasid); + if (ret) { + dev_err(&idxd->pdev->dev, "No DMA PASID %d\n", ret); + return ret; } - - idxd->sva = sva; idxd->pasid = pasid; - dev_dbg(&idxd->pdev->dev, "system pasid: %u\n", pasid); + return 0; } static void idxd_disable_system_pasid(struct idxd_device *idxd) { - - iommu_sva_unbind_device(idxd->sva); - idxd->sva = NULL; + iommu_detach_dma_pasid(&idxd->pdev->dev); } static int idxd_probe(struct idxd_device *idxd) @@ -527,10 +514,7 @@ static int idxd_probe(struct idxd_device *idxd) else set_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags); } - } else if (!sva) { - dev_warn(dev, "User forced SVA off via module param.\n"); } - idxd_read_caps(idxd); idxd_read_table_offsets(idxd); diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c index dfd549685c46..a48928973bd4 100644 --- a/drivers/dma/idxd/sysfs.c +++ b/drivers/dma/idxd/sysfs.c @@ -839,13 +839,6 @@ static ssize_t wq_name_store(struct device *dev, if (strlen(buf) > WQ_NAME_SIZE || strlen(buf) == 0) return -EINVAL; - /* - * This is temporarily placed here until we have SVM support for - * dmaengine. - */ - if (wq->type == IDXD_WQT_KERNEL && device_pasid_enabled(wq->idxd)) - return -EOPNOTSUPP; - memset(wq->name, 0, WQ_NAME_SIZE + 1); strncpy(wq->name, buf, WQ_NAME_SIZE); strreplace(wq->name, '\n', '\0'); -- 2.25.1 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu