From: Xu Zaibo <xuzaibo-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
To: Lu Baolu <baolu.lu-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>,
Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org>,
"David Woodhouse" <dwmw2-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>,
Alex Williamson
<alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>,
Kirti Wankhede
<kwankhede-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org>
Cc: kevin.tian-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
ashok.raj-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
tiwei.bie-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
Jean-Philippe Brucker
<jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org>,
sanjay.k.kumar-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org,
jacob.jun.pan-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org,
kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
yi.y.sun-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
Subject: Re: [PATCH v4 0/8] vfio/mdev: IOMMU aware mediated device
Date: Tue, 4 Dec 2018 11:46:05 +0800 [thread overview]
Message-ID: <5C05F87D.4050206@huawei.com> (raw)
In-Reply-To: <20181105073408.21815-1-baolu.lu-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
Hi,
Is this solution trying to support general user space processes who are
directly working on devices?
Thanks,
Zaibo
.
On 2018/11/5 15:34, Lu Baolu wrote:
> Hi,
>
> The Mediate Device is a framework for fine-grained physical device
> sharing across the isolated domains. Currently the mdev framework
> is designed to be independent of the platform IOMMU support. As the
> result, the DMA isolation relies on the mdev parent device in a
> vendor specific way.
>
> There are several cases where a mediated device could be protected
> and isolated by the platform IOMMU. For example, Intel vt-d rev3.0
> [1] introduces a new translation mode called 'scalable mode', which
> enables PASID-granular translations. The vt-d scalable mode is the
> key ingredient for Scalable I/O Virtualization [2] [3] which allows
> sharing a device in minimal possible granularity (ADI - Assignable
> Device Interface).
>
> A mediated device backed by an ADI could be protected and isolated
> by the IOMMU since 1) the parent device supports tagging an unique
> PASID to all DMA traffic out of the mediated device; and 2) the DMA
> translation unit (IOMMU) supports the PASID granular translation.
> We can apply IOMMU protection and isolation to this kind of devices
> just as what we are doing with an assignable PCI device.
>
> In order to distinguish the IOMMU-capable mediated devices from those
> which still need to rely on parent devices, this patch set adds two
> new members in struct mdev_device.
>
> * iommu_device
> - This, if set, indicates that the mediated device could
> be fully isolated and protected by IOMMU via attaching
> an iommu domain to this device. If empty, it indicates
> using vendor defined isolation.
>
> * iommu_domain
> - This is a place holder for an iommu domain. A domain
> could be store here for later use once it has been
> attached to the iommu_device of this mdev.
>
> Below helpers are added to set and get above iommu device
> and iommu domain pointers in mdev core implementation.
>
> * mdev_set/get_iommu_device(dev, iommu_device)
> - Set or get the iommu device which represents this mdev
> in IOMMU's device scope. Drivers don't need to set the
> iommu device if it uses vendor defined isolation.
>
> * mdev_set/get_iommu_domain(domain)
> - A iommu domain which has been attached to the iommu
> device in order to protect and isolate the mediated
> device will be kept in the mdev data structure and
> could be retrieved later.
>
> The mdev parent device driver could opt-in that the mdev could be
> fully isolated and protected by the IOMMU when the mdev is being
> created by invoking mdev_set_iommu_device() in its @create().
>
> In the vfio_iommu_type1_attach_group(), a domain allocated through
> iommu_domain_alloc() will be attached to the mdev iommu device if
> an iommu device has been set. Otherwise, the dummy external domain
> will be used and all the DMA isolation and protection are routed to
> parent driver as the result.
>
> On IOMMU side, a basic requirement is allowing to attach multiple
> domains to a PCI device if the device advertises the capability
> and the IOMMU hardware supports finer granularity translations than
> the normal PCI Source ID based translation.
>
> As the result, a PCI device could work in two modes: normal mode
> and auxiliary mode. In the normal mode, a pci device could be
> isolated in the Source ID granularity; the pci device itself could
> be assigned to a user application by attaching a single domain
> to it. In the auxiliary mode, a pci device could be isolated in
> finer granularity, hence subsets of the device could be assigned
> to different user level application by attaching a different domain
> to each subset.
>
> The device driver is able to switch between above two modes with
> below interfaces:
>
> * iommu_get_dev_attr(dev, IOMMU_DEV_ATTR_AUXD_CAPABILITY)
> - Represents the ability of supporting multiple domains
> per device.
>
> * iommu_set_dev_attr(dev, IOMMU_DEV_ATTR_AUXD_ENABLE)
> - Enable the multiple domains capability for the device
> referenced by @dev.
>
> * iommu_set_dev_attr(dev, IOMMU_DEV_ATTR_AUXD_DISABLE)
> - Disable the multiple domains capability for the device
> referenced by @dev.
>
> * iommu_domain_get_attr(domain, DOMAIN_ATTR_AUXD_ID)
> - Return ID used for finer-granularity DMA translation.
>
> * iommu_attach_device_aux(domain, dev)
> - Attach a domain to the device in the auxiliary mode.
>
> * iommu_detach_device_aux(domain, dev)
> - Detach the aux domain from device.
>
> In order for the ease of discussion, sometimes we call "a domain in
> auxiliary mode' or simply 'an auxiliary domain' when a domain is
> attached to a device for finer granularity translations. But we need
> to keep in mind that this doesn't mean there is a differnt domain
> type. A same domain could be bound to a device for Source ID based
> translation, and bound to another device for finer granularity
> translation at the same time.
>
> This patch series extends both IOMMU and vfio components to support
> mdev device passing through when it could be isolated and protected
> by the IOMMU units. The first part of this series (PATCH 1/08~5/08)
> adds the interfaces and implementation of the multiple domains per
> device. The second part (PATCH 6/08~8/08) adds the iommu device
> attribute to each mdev, determines isolation type according to the
> existence of an iommu device when attaching group in vfio type1 iommu
> module, and attaches the domain to iommu aware mediated devices.
>
> This patch series depends on a patch set posted here [4] for discussion
> which added scalable mode support in Intel IOMMU driver.
>
> References:
> [1] https://software.intel.com/en-us/download/intel-virtualization-technology-for-directed-io-architecture-specification
> [2] https://software.intel.com/en-us/download/intel-scalable-io-virtualization-technical-specification
> [3] https://schd.ws/hosted_files/lc32018/00/LC3-SIOV-final.pdf
> [4] https://lkml.org/lkml/2018/11/5/136
>
> Best regards,
> Lu Baolu
>
> Change log:
> v3->v4:
> - Use aux domain specific interfaces for domain attach and detach.
> - Rebase all patches to 4.20-rc1.
>
> v2->v3:
> - Remove domain type enum and use a pointer on mdev_device instead.
> - Add a generic interface for getting/setting per device iommu
> attributions. And use it for query aux domain capability, enable
> aux domain and disable aux domain purpose.
> - Reuse iommu_domain_get_attr() to retrieve the id in a aux domain.
> - We discussed the impact of the default domain implementation
> on reusing iommu_at(de)tach_device() interfaces. We agreed
> that reusing iommu_at(de)tach_device() interfaces is the right
> direction and we could tweak the code to remove the impact.
> https://www.spinics.net/lists/kvm/msg175285.html
> - Removed the RFC tag since no objections received.
> - This patch has been submitted separately.
> https://www.spinics.net/lists/kvm/msg173936.html
>
> v1->v2:
> - Rewrite the patches with the concept of auxiliary domains.
>
> Lu Baolu (8):
> iommu: Add APIs for multiple domains per device
> iommu/vt-d: Add multiple domains per device query
> iommu/vt-d: Enable/disable multiple domains per device
> iommu/vt-d: Attach/detach domains in auxiliary mode
> iommu/vt-d: Return ID associated with an auxiliary domain
> vfio/mdev: Add iommu place holders in mdev_device
> vfio/type1: Add domain at(de)taching group helpers
> vfio/type1: Handle different mdev isolation type
>
> drivers/iommu/intel-iommu.c | 315 ++++++++++++++++++++++++++++---
> drivers/iommu/iommu.c | 52 +++++
> drivers/vfio/mdev/mdev_core.c | 36 ++++
> drivers/vfio/mdev/mdev_private.h | 2 +
> drivers/vfio/vfio_iommu_type1.c | 162 ++++++++++++++--
> include/linux/intel-iommu.h | 11 ++
> include/linux/iommu.h | 52 +++++
> include/linux/mdev.h | 23 +++
> 8 files changed, 618 insertions(+), 35 deletions(-)
>
next prev parent reply other threads:[~2018-12-04 3:46 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-05 7:34 [PATCH v4 0/8] vfio/mdev: IOMMU aware mediated device Lu Baolu
[not found] ` <20181105073408.21815-1-baolu.lu-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2018-11-05 7:34 ` [PATCH v4 1/8] iommu: Add APIs for multiple domains per device Lu Baolu
2018-11-23 10:50 ` Auger Eric
[not found] ` <51871885-05d2-febc-1dba-4d74108d0f46-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2018-11-26 2:04 ` Lu Baolu
2018-12-04 3:46 ` Xu Zaibo [this message]
[not found] ` <5C05F87D.4050206-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2018-12-04 6:20 ` [PATCH v4 0/8] vfio/mdev: IOMMU aware mediated device Lu Baolu
2018-12-04 6:50 ` Xu Zaibo
2018-11-05 7:34 ` [PATCH v4 2/8] iommu/vt-d: Add multiple domains per device query Lu Baolu
[not found] ` <20181105073408.21815-3-baolu.lu-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2018-11-23 10:49 ` Auger Eric
2018-11-26 2:10 ` Lu Baolu
2018-11-05 7:34 ` [PATCH v4 3/8] iommu/vt-d: Enable/disable multiple domains per device Lu Baolu
2018-11-05 7:34 ` [PATCH v4 4/8] iommu/vt-d: Attach/detach domains in auxiliary mode Lu Baolu
2018-11-23 10:49 ` Auger Eric
[not found] ` <d714fb14-6217-f9c7-a5b2-2fc67e4bcd4c-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2018-11-26 2:37 ` Lu Baolu
2018-11-05 7:34 ` [PATCH v4 5/8] iommu/vt-d: Return ID associated with an auxiliary domain Lu Baolu
2018-11-05 7:34 ` [PATCH v4 6/8] vfio/mdev: Add iommu place holders in mdev_device Lu Baolu
[not found] ` <20181105073408.21815-7-baolu.lu-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2018-11-05 14:51 ` Christoph Hellwig
2018-11-05 23:33 ` Lu Baolu
2018-11-06 23:53 ` Alex Williamson
2018-11-07 1:48 ` Lu Baolu
2018-11-15 21:31 ` Kirti Wankhede
2018-11-16 1:20 ` Lu Baolu
2018-11-16 8:57 ` Christoph Hellwig
2018-11-17 2:37 ` Lu Baolu
2018-11-20 20:52 ` Kirti Wankhede
2018-11-21 8:45 ` Christoph Hellwig
2018-12-03 16:27 ` Kirti Wankhede
2018-11-05 7:34 ` [PATCH v4 7/8] vfio/type1: Add domain at(de)taching group helpers Lu Baolu
[not found] ` <20181105073408.21815-8-baolu.lu-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2018-11-23 14:13 ` Auger Eric
[not found] ` <b5f5eda1-2dbf-dd7e-3b68-bfd9e5099c06-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2018-11-26 3:05 ` Lu Baolu
2018-11-05 7:34 ` [PATCH v4 8/8] vfio/type1: Handle different mdev isolation type Lu Baolu
[not found] ` <20181105073408.21815-9-baolu.lu-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2018-11-23 14:23 ` Auger Eric
2018-11-26 3:09 ` Lu Baolu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5C05F87D.4050206@huawei.com \
--to=xuzaibo-hv44wf8li93qt0dzr+alfa@public.gmane.org \
--cc=alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org \
--cc=ashok.raj-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
--cc=baolu.lu-VuQAYsv1563Yd54FQh9/CA@public.gmane.org \
--cc=dwmw2-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org \
--cc=iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org \
--cc=jacob.jun.pan-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
--cc=jean-philippe.brucker-5wv7dgnIgG8@public.gmane.org \
--cc=joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org \
--cc=kevin.tian-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
--cc=kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=kwankhede-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org \
--cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=sanjay.k.kumar-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
--cc=tiwei.bie-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
--cc=yi.y.sun-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).