From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, DATE_IN_FUTURE_03_06,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C099C433E0 for ; Tue, 2 Mar 2021 16:10:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0488064F14 for ; Tue, 2 Mar 2021 16:10:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1839102AbhCBQFe (ORCPT ); Tue, 2 Mar 2021 11:05:34 -0500 Received: from mga01.intel.com ([192.55.52.88]:37927 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343520AbhCBMpB (ORCPT ); Tue, 2 Mar 2021 07:45:01 -0500 IronPort-SDR: xCCDjtu+zN6K6c9FdWgFGpy9prsGbkzl8s/VDEDJFQmo1kpu6CwhMWqjL/+XkKnKSNaJ930MUJ ZcpQAVFiO9dw== X-IronPort-AV: E=McAfee;i="6000,8403,9910"; a="206431237" X-IronPort-AV: E=Sophos;i="5.81,216,1610438400"; d="scan'208";a="206431237" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Mar 2021 04:37:21 -0800 IronPort-SDR: b01cnbmL4VNWHjMTVTn0VrFg5n+zP3jzVtQDTNf6cGCvNZJCyLEWmN92u+AztyGIMvAM/xls97 FCFXND7pZ2FQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,216,1610438400"; d="scan'208";a="427472133" Received: from yiliu-dev.bj.intel.com (HELO dual-ub.bj.intel.com) ([10.238.156.135]) by fmsmga004.fm.intel.com with ESMTP; 02 Mar 2021 04:37:16 -0800 From: Liu Yi L To: alex.williamson@redhat.com, eric.auger@redhat.com, baolu.lu@linux.intel.com, joro@8bytes.org Cc: kevin.tian@intel.com, jacob.jun.pan@linux.intel.com, ashok.raj@intel.com, yi.l.liu@intel.com, jun.j.tian@intel.com, yi.y.sun@intel.com, jean-philippe@linaro.org, peterx@redhat.com, jasowang@redhat.com, hao.wu@intel.com, stefanha@gmail.com, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, jgg@nvidia.com, Lingshan.Zhu@intel.com, vivek.gautam@arm.com Subject: [Patch v8 07/10] vfio/type1: Add vSVA support for IOMMU-backed mdevs Date: Wed, 3 Mar 2021 04:35:42 +0800 Message-Id: <20210302203545.436623-8-yi.l.liu@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210302203545.436623-1-yi.l.liu@intel.com> References: <20210302203545.436623-1-yi.l.liu@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Recent years, mediated device pass-through framework (e.g. vfio-mdev) is used to achieve flexible device sharing across domains (e.g. VMs). Also there are hardware assisted mediated pass-through solutions from platform vendors. e.g. Intel VT-d scalable mode which supports Intel Scalable I/O Virtualization technology. Such mdevs are called IOMMU- backed mdevs as there are IOMMU enforced DMA isolation for such mdevs. In kernel, IOMMU-backed mdevs are exposed to IOMMU layer by aux-domain concept, which means mdevs are protected by an iommu domain which is auxiliary to the domain that the kernel driver primarily uses for DMA API. Details can be found in the KVM presentation as below: https://events19.linuxfoundation.org/wp-content/uploads/2017/12/Hardware-Assisted-Mediated-Pass-Through-with-VFIO-Kevin-Tian-Intel.pdf This patch extends NESTING_IOMMU ops to IOMMU-backed mdev devices. The main requirement is to use the auxiliary domain associated with mdev. Cc: Kevin Tian CC: Jacob Pan CC: Jun Tian Cc: Alex Williamson Cc: Eric Auger Cc: Jean-Philippe Brucker Cc: Joerg Roedel Cc: Lu Baolu Signed-off-by: Liu Yi L Reviewed-by: Eric Auger --- v5 -> v6: *) add review-by from Eric Auger. v1 -> v2: *) check the iommu_device to ensure the handling mdev is IOMMU-backed --- drivers/vfio/vfio_iommu_type1.c | 35 ++++++++++++++++++++++++++++----- 1 file changed, 30 insertions(+), 5 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 86b6d8f9789a..883a79f36c46 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -2635,18 +2635,37 @@ static int vfio_iommu_resv_refresh(struct vfio_iommu *iommu, return ret; } +static struct device *vfio_get_iommu_device(struct vfio_group *group, + struct device *dev) +{ + if (group->mdev_group) + return vfio_mdev_get_iommu_device(dev); + else + return dev; +} + static int vfio_dev_bind_gpasid_fn(struct device *dev, void *data) { struct domain_capsule *dc = (struct domain_capsule *)data; unsigned long arg = *(unsigned long *)dc->data; + struct device *iommu_device; + + iommu_device = vfio_get_iommu_device(dc->group, dev); + if (!iommu_device) + return -EINVAL; - return iommu_uapi_sva_bind_gpasid(dc->domain, dev, + return iommu_uapi_sva_bind_gpasid(dc->domain, iommu_device, (void __user *)arg); } static int vfio_dev_unbind_gpasid_fn(struct device *dev, void *data) { struct domain_capsule *dc = (struct domain_capsule *)data; + struct device *iommu_device; + + iommu_device = vfio_get_iommu_device(dc->group, dev); + if (!iommu_device) + return -EINVAL; /* * dc->user is a toggle for the unbind operation. When user @@ -2659,12 +2678,12 @@ static int vfio_dev_unbind_gpasid_fn(struct device *dev, void *data) if (dc->user) { unsigned long arg = *(unsigned long *)dc->data; - iommu_uapi_sva_unbind_gpasid(dc->domain, - dev, (void __user *)arg); + iommu_uapi_sva_unbind_gpasid(dc->domain, iommu_device, + (void __user *)arg); } else { ioasid_t pasid = *(ioasid_t *)dc->data; - iommu_sva_unbind_gpasid(dc->domain, dev, pasid); + iommu_sva_unbind_gpasid(dc->domain, iommu_device, pasid); } return 0; } @@ -3295,8 +3314,14 @@ static int vfio_dev_cache_invalidate_fn(struct device *dev, void *data) { struct domain_capsule *dc = (struct domain_capsule *)data; unsigned long arg = *(unsigned long *)dc->data; + struct device *iommu_device; + + iommu_device = vfio_get_iommu_device(dc->group, dev); + if (!iommu_device) + return -EINVAL; - iommu_uapi_cache_invalidate(dc->domain, dev, (void __user *)arg); + iommu_uapi_cache_invalidate(dc->domain, iommu_device, + (void __user *)arg); return 0; } -- 2.25.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.8 required=3.0 tests=BAYES_00, DATE_IN_FUTURE_06_12,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07937C433DB for ; Tue, 2 Mar 2021 12:37:26 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A4169614A7 for ; Tue, 2 Mar 2021 12:37:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A4169614A7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 6921A4EFD8; Tue, 2 Mar 2021 12:37:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xQ8FuSLPA9F8; Tue, 2 Mar 2021 12:37:24 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTP id F23664EFCF; Tue, 2 Mar 2021 12:37:23 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id DED8FC000A; Tue, 2 Mar 2021 12:37:23 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id D8E39C0001 for ; Tue, 2 Mar 2021 12:37:22 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id D506583F3F for ; Tue, 2 Mar 2021 12:37:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bqH7EwPMntQk for ; Tue, 2 Mar 2021 12:37:21 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by smtp1.osuosl.org (Postfix) with ESMTPS id C6C5583F3C for ; Tue, 2 Mar 2021 12:37:21 +0000 (UTC) IronPort-SDR: U5R/C78ZSB/htthrGueBTDGK1SWMUm1yqKUEoPnp7cOcP18NikmNxJbIh9qGnxUOPk7MMexZD0 iDOKiuQAFKQA== X-IronPort-AV: E=McAfee;i="6000,8403,9910"; a="166675158" X-IronPort-AV: E=Sophos;i="5.81,216,1610438400"; d="scan'208";a="166675158" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Mar 2021 04:37:21 -0800 IronPort-SDR: b01cnbmL4VNWHjMTVTn0VrFg5n+zP3jzVtQDTNf6cGCvNZJCyLEWmN92u+AztyGIMvAM/xls97 FCFXND7pZ2FQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,216,1610438400"; d="scan'208";a="427472133" Received: from yiliu-dev.bj.intel.com (HELO dual-ub.bj.intel.com) ([10.238.156.135]) by fmsmga004.fm.intel.com with ESMTP; 02 Mar 2021 04:37:16 -0800 From: Liu Yi L To: alex.williamson@redhat.com, eric.auger@redhat.com, baolu.lu@linux.intel.com, joro@8bytes.org Subject: [Patch v8 07/10] vfio/type1: Add vSVA support for IOMMU-backed mdevs Date: Wed, 3 Mar 2021 04:35:42 +0800 Message-Id: <20210302203545.436623-8-yi.l.liu@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210302203545.436623-1-yi.l.liu@intel.com> References: <20210302203545.436623-1-yi.l.liu@intel.com> MIME-Version: 1.0 Cc: jean-philippe@linaro.org, kevin.tian@intel.com, ashok.raj@intel.com, kvm@vger.kernel.org, stefanha@gmail.com, jun.j.tian@intel.com, iommu@lists.linux-foundation.org, vivek.gautam@arm.com, yi.y.sun@intel.com, jgg@nvidia.com, Lingshan.Zhu@intel.com, jasowang@redhat.com, hao.wu@intel.com X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" Recent years, mediated device pass-through framework (e.g. vfio-mdev) is used to achieve flexible device sharing across domains (e.g. VMs). Also there are hardware assisted mediated pass-through solutions from platform vendors. e.g. Intel VT-d scalable mode which supports Intel Scalable I/O Virtualization technology. Such mdevs are called IOMMU- backed mdevs as there are IOMMU enforced DMA isolation for such mdevs. In kernel, IOMMU-backed mdevs are exposed to IOMMU layer by aux-domain concept, which means mdevs are protected by an iommu domain which is auxiliary to the domain that the kernel driver primarily uses for DMA API. Details can be found in the KVM presentation as below: https://events19.linuxfoundation.org/wp-content/uploads/2017/12/Hardware-Assisted-Mediated-Pass-Through-with-VFIO-Kevin-Tian-Intel.pdf This patch extends NESTING_IOMMU ops to IOMMU-backed mdev devices. The main requirement is to use the auxiliary domain associated with mdev. Cc: Kevin Tian CC: Jacob Pan CC: Jun Tian Cc: Alex Williamson Cc: Eric Auger Cc: Jean-Philippe Brucker Cc: Joerg Roedel Cc: Lu Baolu Signed-off-by: Liu Yi L Reviewed-by: Eric Auger --- v5 -> v6: *) add review-by from Eric Auger. v1 -> v2: *) check the iommu_device to ensure the handling mdev is IOMMU-backed --- drivers/vfio/vfio_iommu_type1.c | 35 ++++++++++++++++++++++++++++----- 1 file changed, 30 insertions(+), 5 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 86b6d8f9789a..883a79f36c46 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -2635,18 +2635,37 @@ static int vfio_iommu_resv_refresh(struct vfio_iommu *iommu, return ret; } +static struct device *vfio_get_iommu_device(struct vfio_group *group, + struct device *dev) +{ + if (group->mdev_group) + return vfio_mdev_get_iommu_device(dev); + else + return dev; +} + static int vfio_dev_bind_gpasid_fn(struct device *dev, void *data) { struct domain_capsule *dc = (struct domain_capsule *)data; unsigned long arg = *(unsigned long *)dc->data; + struct device *iommu_device; + + iommu_device = vfio_get_iommu_device(dc->group, dev); + if (!iommu_device) + return -EINVAL; - return iommu_uapi_sva_bind_gpasid(dc->domain, dev, + return iommu_uapi_sva_bind_gpasid(dc->domain, iommu_device, (void __user *)arg); } static int vfio_dev_unbind_gpasid_fn(struct device *dev, void *data) { struct domain_capsule *dc = (struct domain_capsule *)data; + struct device *iommu_device; + + iommu_device = vfio_get_iommu_device(dc->group, dev); + if (!iommu_device) + return -EINVAL; /* * dc->user is a toggle for the unbind operation. When user @@ -2659,12 +2678,12 @@ static int vfio_dev_unbind_gpasid_fn(struct device *dev, void *data) if (dc->user) { unsigned long arg = *(unsigned long *)dc->data; - iommu_uapi_sva_unbind_gpasid(dc->domain, - dev, (void __user *)arg); + iommu_uapi_sva_unbind_gpasid(dc->domain, iommu_device, + (void __user *)arg); } else { ioasid_t pasid = *(ioasid_t *)dc->data; - iommu_sva_unbind_gpasid(dc->domain, dev, pasid); + iommu_sva_unbind_gpasid(dc->domain, iommu_device, pasid); } return 0; } @@ -3295,8 +3314,14 @@ static int vfio_dev_cache_invalidate_fn(struct device *dev, void *data) { struct domain_capsule *dc = (struct domain_capsule *)data; unsigned long arg = *(unsigned long *)dc->data; + struct device *iommu_device; + + iommu_device = vfio_get_iommu_device(dc->group, dev); + if (!iommu_device) + return -EINVAL; - iommu_uapi_cache_invalidate(dc->domain, dev, (void __user *)arg); + iommu_uapi_cache_invalidate(dc->domain, iommu_device, + (void __user *)arg); return 0; } -- 2.25.1 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu