From: David Gibson <david@gibson.dropbear.id.au>
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: Jason Gunthorpe <jgg@nvidia.com>,
Alex Williamson <alex.williamson@redhat.com>,
"Liu, Yi L" <yi.l.liu@intel.com>,
Jacob Pan <jacob.jun.pan@linux.intel.com>,
Auger Eric <eric.auger@redhat.com>,
Jean-Philippe Brucker <jean-philippe@linaro.org>,
LKML <linux-kernel@vger.kernel.org>,
Joerg Roedel <joro@8bytes.org>,
Lu Baolu <baolu.lu@linux.intel.com>,
David Woodhouse <dwmw2@infradead.org>,
"iommu@lists.linux-foundation.org"
<iommu@lists.linux-foundation.org>,
"cgroups@vger.kernel.org" <cgroups@vger.kernel.org>,
Tejun Heo <tj@kernel.org>, Li Zefan <lizefan@huawei.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Jean-Philippe Brucker <jean-philippe@linaro.com>,
Jonathan Corbet <corbet@lwn.net>,
"Raj, Ashok" <ashok.raj@intel.com>, "Wu, Hao" <hao.wu@intel.com>,
"Jiang, Dave" <dave.jiang@intel.com>,
Alexey Kardashevskiy <aik@ozlabs.ru>
Subject: Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs
Date: Tue, 27 Apr 2021 15:11:25 +1000 [thread overview]
Message-ID: <YIec/Rt7OxvfFw7W@yekko.fritz.box> (raw)
In-Reply-To: <MWHPR11MB1886A98D9176B5571530EF1D8C459@MWHPR11MB1886.namprd11.prod.outlook.com>
[-- Attachment #1: Type: text/plain, Size: 6472 bytes --]
On Fri, Apr 23, 2021 at 10:31:46AM +0000, Tian, Kevin wrote:
> > From: Jason Gunthorpe <jgg@nvidia.com>
> > Sent: Friday, April 23, 2021 7:40 AM
> >
> > On Thu, Apr 22, 2021 at 04:38:08PM -0600, Alex Williamson wrote:
> >
> > > Because it's fundamental to the isolation of the device? What you're
> > > proposing doesn't get around the group issue, it just makes it implicit
> > > rather than explicit in the uapi.
> >
> > I'm not even sure it makes it explicit or implicit, it just takes away
> > the FD.
> >
> > There are four group IOCTLs, I see them mapping to /dev/ioasid follows:
> > VFIO_GROUP_GET_STATUS -
> > + VFIO_GROUP_FLAGS_CONTAINER_SET is fairly redundant
> > + VFIO_GROUP_FLAGS_VIABLE could be in a new sysfs under
> > kernel/iomm_groups, or could be an IOCTL on /dev/ioasid
> > IOASID_ALL_DEVICES_VIABLE
> >
> > VFIO_GROUP_SET_CONTAINER -
> > + This happens implicitly when the device joins the IOASID
> > so it gets moved to the vfio_device FD:
> > ioctl(vifo_device_fd, JOIN_IOASID_FD, ioasifd)
> >
> > VFIO_GROUP_UNSET_CONTAINER -
> > + Also moved to the vfio_device FD, opposite of JOIN_IOASID_FD
> >
> > VFIO_GROUP_GET_DEVICE_FD -
> > + Replaced by opening /dev/vfio/deviceX
> > Learn the deviceX which will be the cdev sysfs shows as:
> > /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/vfio/deviceX/dev
> > Open /dev/vfio/deviceX
> >
> > > > How do we model the VFIO group security concept to something like
> > > > VDPA?
> > >
> > > Is it really a "VFIO group security concept"? We're reflecting the
> > > reality of the hardware, not all devices are fully isolated.
> >
> > Well, exactly.
> >
> > /dev/ioasid should understand the group concept somehow, otherwise it
> > is incomplete and maybe even security broken.
> >
> > So, how do I add groups to, say, VDPA in a way that makes sense? The
> > only answer I come to is broadly what I outlined here - make
> > /dev/ioasid do all the group operations, and do them when we enjoin
> > the VDPA device to the ioasid.
> >
> > Once I have solved all the groups problems with the non-VFIO users,
> > then where does that leave VFIO? Why does VFIO need a group FD if
> > everyone else doesn't?
> >
> > > IOMMU group. This is the reality that any userspace driver needs to
> > > play in, it doesn't magically go away because we drop the group file
> > > descriptor.
> >
> > I'm not saying it does, I'm saying it makes the uAPI more regular and
> > easier to fit into /dev/ioasid without the group FD.
> >
> > > It only makes the uapi more difficult to use correctly because
> > > userspace drivers need to go outside of the uapi to have any idea
> > > that this restriction exists.
> >
> > I don't think it makes any substantive difference one way or the
> > other.
> >
> > With the group FD: the userspace has to read sysfs, find the list of
> > devices in the group, open the group fd, create device FDs for each
> > device using the name from sysfs.
> >
> > Starting from a BDF the general pseudo code is
> > group_path = readlink("/sys/bus/pci/devices/BDF/iommu_group")
> > group_name = basename(group_path)
> > group_fd = open("/dev/vfio/"+group_name)
> > device_fd = ioctl(VFIO_GROUP_GET_DEVICE_FD, BDF);
> >
> > Without the group FD: the userspace has to read sysfs, find the list
> > of devices in the group and then open the device-specific cdev (found
> > via sysfs) and link them to a /dev/ioasid FD.
> >
> > Starting from a BDF the general pseudo code is:
> > device_name = first_directory_of("/sys/bus/pci/devices/BDF/vfio/")
> > device_fd = open("/dev/vfio/"+device_name)
> > ioasidfd = open("/dev/ioasid")
> > ioctl(device_fd, JOIN_IOASID_FD, ioasidfd)
> >
> > These two routes can have identical outcomes and identical security
> > checks.
> >
> > In both cases if userspace wants a list of BDFs in the same group as
> > the BDF it is interested in:
> > readdir("/sys/bus/pci/devices/BDF/iommu_group/devices")
> >
> > It seems like a very small difference to me.
> >
> > I still don't see how the group restriction gets surfaced to the
> > application through the group FD. The applications I looked through
> > just treat the group FD as a step on their way to get the device_fd.
> >
>
> So your proposal sort of moves the entire container/group/domain
> managment into /dev/ioasid and then leaves vfio only provide device
> specific uAPI. An ioasid represents a page table (address space), thus
> is equivalent to the scope of VFIO container.
Right. I don't really know how /dev/iosasid is supposed to work, and
so far I don't see how it conceptually differs from a container. What
is it adding?
> Having the device join
> an ioasid is equivalent to attaching a device to VFIO container, and
> here the group integrity must be enforced. Then /dev/ioasid anyway
> needs to manage group objects and their association with ioasid and
> underlying iommu domain thus it's pointless to keep same logic within
> VFIO. Is this understanding correct?
>
> btw one remaining open is whether you expect /dev/ioasid to be
> associated with a single iommu domain, or multiple. If only a single
> domain is allowed, the ioasid_fd is equivalent to the scope of VFIO
> container. It is supposed to have only one gpa_ioasid_id since one
> iommu domain can only have a single 2nd level pgtable. Then all other
> ioasids, once allocated, must be nested on this gpa_ioasid_id to fit
> in the same domain. if a legacy vIOMMU is exposed (which disallows
> nesting), the userspace has to open an ioasid_fd for every group.
> This is basically the VFIO way. On the other hand if multiple domains
> is allowed, there could be multiple ioasid_ids each holding a 2nd level
> pgtable and an iommu domain (or a list of pgtables and domains due to
> incompatibility issue as discussed in another thread), and can be
> nested by other ioasids respectively. The application only needs
> to open /dev/ioasid once regardless of whether vIOMMU allows
> nesting, and has a single interface for ioasid allocation. Which way
> do you prefer to?
>
> Thanks
> Kevin
>
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2021-04-27 5:15 UTC|newest]
Thread overview: 269+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-27 22:01 [PATCH V4 00/18] IOASID extensions for guest SVA Jacob Pan
2021-02-27 22:01 ` [PATCH V4 01/18] docs: Document IO Address Space ID (IOASID) APIs Jacob Pan
2021-02-27 22:01 ` [PATCH V4 02/18] iommu/ioasid: Rename ioasid_set_data() Jacob Pan
2021-02-27 22:01 ` [PATCH V4 03/18] iommu/ioasid: Add a separate function for detach data Jacob Pan
2021-02-27 22:01 ` [PATCH V4 04/18] iommu/ioasid: Support setting system-wide capacity Jacob Pan
2021-02-27 22:01 ` [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs Jacob Pan
2021-03-19 0:22 ` Jacob Pan
2021-03-19 9:58 ` Jean-Philippe Brucker
2021-03-19 12:46 ` Jason Gunthorpe
2021-03-19 13:41 ` Jean-Philippe Brucker
2021-03-19 13:54 ` Jason Gunthorpe
2021-03-19 18:22 ` Jacob Pan
2021-03-22 9:24 ` Jean-Philippe Brucker
2021-03-24 17:02 ` Jacob Pan
2021-03-24 17:03 ` Jason Gunthorpe
2021-03-24 22:12 ` Jacob Pan
2021-03-25 10:21 ` Jean-Philippe Brucker
2021-03-25 17:02 ` Jacob Pan
2021-03-25 17:16 ` Jason Gunthorpe
2021-03-25 18:23 ` Jacob Pan
2021-03-26 8:06 ` Jean-Philippe Brucker
2021-03-30 13:07 ` Jason Gunthorpe
2021-03-30 13:42 ` Jean-Philippe Brucker
2021-03-30 13:46 ` Jason Gunthorpe
2021-03-25 10:26 ` Jean-Philippe Brucker
2021-03-22 12:03 ` Jason Gunthorpe
2021-03-24 19:05 ` Jacob Pan
2021-03-29 16:31 ` Jason Gunthorpe
2021-03-29 22:55 ` Jacob Pan
2021-03-30 13:43 ` Jason Gunthorpe
2021-03-31 0:10 ` Jacob Pan
2021-03-31 12:28 ` Jason Gunthorpe
2021-03-31 16:34 ` Jacob Pan
2021-03-31 17:31 ` Jason Gunthorpe
2021-03-31 18:20 ` Jacob Pan
2021-03-31 18:33 ` Jason Gunthorpe
2021-03-31 21:50 ` Jacob Pan
2021-03-31 8:38 ` Liu, Yi L
2021-03-30 1:37 ` Tian, Kevin
2021-03-30 13:28 ` Jason Gunthorpe
2021-03-31 7:38 ` Liu, Yi L
2021-03-31 12:40 ` Jason Gunthorpe
2021-04-01 4:38 ` Liu, Yi L
2021-04-01 7:04 ` Liu, Yi L
2021-04-01 11:54 ` Jason Gunthorpe
2021-04-02 12:46 ` Liu, Yi L
2021-04-01 12:05 ` Jean-Philippe Brucker
2021-04-01 12:12 ` Jason Gunthorpe
2021-04-01 13:38 ` Liu, Yi L
2021-04-01 13:42 ` Jason Gunthorpe
2021-04-01 14:08 ` Liu, Yi L
2021-04-01 16:03 ` Jason Gunthorpe
2021-04-02 7:30 ` Tian, Kevin
2021-04-05 23:35 ` Jason Gunthorpe
2021-04-06 0:37 ` Tian, Kevin
2021-04-06 12:15 ` Jason Gunthorpe
2021-04-15 13:11 ` Auger Eric
2021-04-15 23:07 ` Jason Gunthorpe
2021-04-16 13:12 ` Jacob Pan
2021-04-16 15:45 ` Alex Williamson
2021-04-16 17:23 ` Jacob Pan
2021-04-16 17:54 ` Jason Gunthorpe
2021-04-21 13:18 ` Liu, Yi L
2021-04-21 16:23 ` Jason Gunthorpe
2021-04-21 16:54 ` Alex Williamson
2021-04-21 17:52 ` Jason Gunthorpe
2021-04-21 19:33 ` Alex Williamson
2021-04-21 23:03 ` Jason Gunthorpe
2021-04-22 8:34 ` Tian, Kevin
2021-04-22 12:10 ` Jason Gunthorpe
2021-04-23 9:06 ` Tian, Kevin
2021-04-23 11:49 ` Jason Gunthorpe
2021-04-25 9:24 ` Tian, Kevin
2021-04-26 12:38 ` Jason Gunthorpe
2021-04-28 6:34 ` Tian, Kevin
2021-04-28 15:06 ` Alex Williamson
2021-05-07 7:36 ` Tian, Kevin
2021-05-07 11:56 ` Jason Gunthorpe
2021-05-07 17:06 ` Alex Williamson
2021-05-07 17:10 ` Jason Gunthorpe
2021-05-08 6:08 ` Tian, Kevin
2021-05-08 7:31 ` Tian, Kevin
2021-05-10 2:56 ` Lu Baolu
2021-04-28 20:46 ` Jason Gunthorpe
2021-05-04 16:22 ` Jacob Pan
2021-05-04 16:31 ` Jason Gunthorpe
2021-05-08 5:46 ` Tian, Kevin
2021-05-04 15:41 ` Jacob Pan
2021-05-04 18:00 ` Jason Gunthorpe
2021-05-04 22:11 ` Jacob Pan
2021-05-04 23:15 ` Jason Gunthorpe
2021-05-05 17:22 ` Jacob Pan
2021-05-05 18:00 ` Jason Gunthorpe
2021-05-05 20:04 ` Jacob Pan
2021-05-05 22:21 ` Jason Gunthorpe
2021-05-05 23:23 ` Raj, Ashok
2021-05-06 12:22 ` Jason Gunthorpe
2021-05-08 7:06 ` Liu Yi L
2021-05-06 7:23 ` Jean-Philippe Brucker
2021-05-06 12:27 ` Jason Gunthorpe
2021-05-06 16:32 ` Raj, Ashok
2021-05-07 17:20 ` Jason Gunthorpe
2021-05-07 18:14 ` Raj, Ashok
2021-05-07 18:20 ` Jason Gunthorpe
2021-05-07 19:23 ` Raj, Ashok
2021-05-07 19:28 ` Jason Gunthorpe
2021-05-07 22:15 ` Jacob Pan
2021-05-08 9:56 ` Tian, Kevin
2021-05-10 12:37 ` Jason Gunthorpe
2021-05-10 15:25 ` Raj, Ashok
2021-05-10 15:31 ` Jason Gunthorpe
2021-05-10 16:22 ` Raj, Ashok
2021-05-10 16:39 ` Jason Gunthorpe
2021-05-10 22:28 ` Jacob Pan
2021-05-10 23:45 ` Jason Gunthorpe
2021-05-11 3:56 ` Jacob Pan
2021-05-11 9:10 ` Tian, Kevin
2021-05-11 13:24 ` Liu Yi L
2021-05-11 22:52 ` Tian, Kevin
2021-05-11 14:38 ` Jason Gunthorpe
2021-05-11 22:51 ` Tian, Kevin
2021-05-11 23:39 ` Jason Gunthorpe
2021-05-12 0:21 ` Tian, Kevin
2021-05-12 0:25 ` Jason Gunthorpe
2021-05-12 0:40 ` Tian, Kevin
2021-04-29 8:54 ` Auger Eric
2021-04-29 8:55 ` Auger Eric
2021-04-29 13:26 ` Auger Eric
2021-04-29 20:04 ` Jason Gunthorpe
2021-05-05 9:10 ` Auger Eric
2021-04-22 17:13 ` Alex Williamson
2021-04-22 17:57 ` Jason Gunthorpe
2021-04-22 19:37 ` Alex Williamson
2021-04-22 20:00 ` Jason Gunthorpe
2021-04-22 22:38 ` Alex Williamson
2021-04-22 23:39 ` Jason Gunthorpe
2021-04-23 10:31 ` Tian, Kevin
2021-04-23 11:57 ` Jason Gunthorpe
2021-04-27 5:11 ` David Gibson [this message]
2021-04-27 16:39 ` Jason Gunthorpe
2021-04-28 0:49 ` David Gibson
2021-04-23 16:38 ` Alex Williamson
2021-04-23 22:28 ` Jason Gunthorpe
2021-04-27 5:15 ` David Gibson
2021-04-27 5:08 ` David Gibson
2021-04-27 17:12 ` Jason Gunthorpe
2021-04-28 0:58 ` David Gibson
2021-04-28 14:56 ` Jason Gunthorpe
2021-04-29 3:04 ` David Gibson
2021-05-03 16:15 ` Jason Gunthorpe
2021-05-13 5:48 ` David Gibson
2021-05-13 13:59 ` Jason Gunthorpe
2021-05-24 7:52 ` David Gibson
2021-05-24 23:37 ` Jason Gunthorpe
2021-05-25 19:26 ` Kirti Wankhede
2021-05-25 19:52 ` Jason Gunthorpe
2021-05-25 21:18 ` Kirti Wankhede
2021-05-27 5:00 ` David Gibson
2021-05-27 18:25 ` Kirti Wankhede
2021-06-01 3:45 ` David Gibson
2021-05-27 4:58 ` David Gibson
2021-05-27 18:48 ` Jason Gunthorpe
2021-06-01 4:03 ` David Gibson
2021-06-01 12:57 ` Jason Gunthorpe
2021-06-08 0:44 ` David Gibson
2021-06-08 18:34 ` Jason Gunthorpe
2021-05-25 22:52 ` Alex Williamson
2021-05-26 18:10 ` Kirti Wankhede
2021-05-26 18:59 ` Alex Williamson
2021-05-26 19:13 ` Jason Gunthorpe
2021-05-27 4:53 ` David Gibson
2021-05-27 19:06 ` Jason Gunthorpe
2021-06-01 4:27 ` David Gibson
2021-04-28 6:58 ` Tian, Kevin
2021-05-04 17:12 ` Jason Gunthorpe
2021-05-07 8:09 ` Tian, Kevin
2021-04-28 7:47 ` Tian, Kevin
2021-04-28 18:41 ` Jason Gunthorpe
2021-04-27 4:50 ` David Gibson
2021-04-27 17:24 ` Jason Gunthorpe
2021-04-28 1:23 ` David Gibson
2021-04-29 0:21 ` Jason Gunthorpe
2021-04-29 3:20 ` David Gibson
2021-05-03 16:05 ` Jason Gunthorpe
2021-05-04 3:54 ` David Gibson
2021-05-04 18:15 ` Jason Gunthorpe
2021-05-05 4:28 ` Alexey Kardashevskiy
2021-05-05 16:39 ` Jason Gunthorpe
2021-05-13 6:07 ` David Gibson
2021-05-13 13:50 ` Jason Gunthorpe
2021-05-24 7:56 ` David Gibson
2021-05-13 6:01 ` David Gibson
2021-05-13 6:52 ` Tian, Kevin
2021-05-13 13:47 ` Jason Gunthorpe
2021-04-22 12:55 ` Liu Yi L
2021-04-16 13:38 ` Auger Eric
2021-04-16 14:05 ` Jason Gunthorpe
2021-04-16 14:26 ` Auger Eric
2021-04-16 14:34 ` Jason Gunthorpe
2021-04-16 15:00 ` Auger Eric
2021-04-01 11:46 ` Jason Gunthorpe
2021-04-01 13:10 ` Liu, Yi L
2021-04-01 13:15 ` Jason Gunthorpe
2021-04-01 13:43 ` Liu, Yi L
2021-04-01 13:46 ` Jason Gunthorpe
2021-04-02 7:58 ` Tian, Kevin
2021-04-05 23:39 ` Jason Gunthorpe
2021-04-06 1:02 ` Tian, Kevin
2021-04-06 12:21 ` Jason Gunthorpe
2021-04-07 2:23 ` Tian, Kevin
[not found] ` <MWHPR11MB188628BDB37A4EE36F3D99338C769@MWHPR11MB1886.namprd11.prod.outlook.com>
2021-04-06 2:08 ` Tian, Kevin
2021-04-02 10:01 ` Tian, Kevin
2021-04-02 8:22 ` Tian, Kevin
2021-04-05 23:42 ` Jason Gunthorpe
2021-04-06 1:27 ` Tian, Kevin
2021-04-06 12:34 ` Jason Gunthorpe
2021-04-07 2:08 ` Tian, Kevin
2021-04-07 12:20 ` Jason Gunthorpe
2021-04-07 23:50 ` Tian, Kevin
2021-04-08 11:41 ` Jason Gunthorpe
2021-04-06 1:35 ` Jason Wang
2021-04-06 12:42 ` Jason Gunthorpe
2021-04-07 2:06 ` Jason Wang
2021-04-07 8:17 ` Tian, Kevin
2021-04-07 11:58 ` Jason Gunthorpe
2021-04-07 18:43 ` Jean-Philippe Brucker
2021-04-07 19:36 ` Jason Gunthorpe
2021-04-08 9:37 ` Jean-Philippe Brucker
2021-03-30 2:24 ` Tian, Kevin
2021-03-30 13:24 ` Jason Gunthorpe
2021-03-30 4:14 ` Tian, Kevin
2021-03-30 13:27 ` Jason Gunthorpe
2021-03-31 7:41 ` Liu, Yi L
2021-03-31 12:38 ` Jason Gunthorpe
2021-03-31 23:46 ` Jacob Pan
2021-04-01 0:37 ` Jason Gunthorpe
2021-04-01 17:23 ` Jacob Pan
2021-04-01 17:26 ` Jason Gunthorpe
2021-03-19 17:14 ` Jacob Pan
2021-02-27 22:01 ` [PATCH V4 06/18] iommu/ioasid: Add free function and states Jacob Pan
2021-02-27 22:01 ` [PATCH V4 07/18] iommu/ioasid: Add ioasid_set iterator helper functions Jacob Pan
2021-02-27 22:01 ` [PATCH V4 08/18] iommu/ioasid: Introduce ioasid_set private ID Jacob Pan
2021-02-27 22:01 ` [PATCH V4 09/18] iommu/ioasid: Introduce notification APIs Jacob Pan
2021-02-27 22:01 ` [PATCH V4 10/18] iommu/ioasid: Support mm token type ioasid_set notifications Jacob Pan
2021-02-27 22:01 ` [PATCH V4 11/18] iommu/ioasid: Add ownership check in guest bind Jacob Pan
2021-02-27 22:01 ` [PATCH V4 12/18] iommu/vt-d: Remove mm reference for guest SVA Jacob Pan
2021-02-27 22:01 ` [PATCH V4 13/18] iommu/ioasid: Add a workqueue for cleanup work Jacob Pan
2021-02-27 22:01 ` [PATCH V4 14/18] iommu/vt-d: Listen to IOASID notifications Jacob Pan
2021-02-27 22:01 ` [RFC PATCH 15/18] cgroup: Introduce ioasids controller Jacob Pan
2021-03-03 15:44 ` Tejun Heo
2021-03-03 21:17 ` Jacob Pan
2021-03-04 0:02 ` Jacob Pan
2021-03-04 0:23 ` Jason Gunthorpe
2021-03-04 9:49 ` Jean-Philippe Brucker
2021-03-04 17:46 ` Jacob Pan
2021-03-04 17:54 ` Jason Gunthorpe
2021-03-04 19:01 ` Jacob Pan
2021-03-04 19:02 ` Jason Gunthorpe
2021-03-04 21:28 ` Jacob Pan
2021-03-05 8:30 ` Jean-Philippe Brucker
2021-03-05 17:16 ` Jean-Philippe Brucker
2021-03-05 18:20 ` Jacob Pan
2021-02-27 22:01 ` [RFC PATCH 16/18] iommu/ioasid: Consult IOASIDs cgroup for allocation Jacob Pan
2021-02-27 22:01 ` [RFC PATCH 17/18] docs: cgroup-v1: Add IOASIDs controller Jacob Pan
2021-02-27 22:01 ` [RFC PATCH 18/18] ioasid: Add /dev/ioasid for userspace Jacob Pan
2021-03-10 19:23 ` Jason Gunthorpe
2021-03-11 22:55 ` Jacob Pan
2021-03-12 14:54 ` Jason Gunthorpe
2021-03-02 12:58 ` [PATCH V4 00/18] IOASID extensions for guest SVA Liu, Yi L
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YIec/Rt7OxvfFw7W@yekko.fritz.box \
--to=david@gibson.dropbear.id.au \
--cc=aik@ozlabs.ru \
--cc=alex.williamson@redhat.com \
--cc=ashok.raj@intel.com \
--cc=baolu.lu@linux.intel.com \
--cc=cgroups@vger.kernel.org \
--cc=corbet@lwn.net \
--cc=dave.jiang@intel.com \
--cc=dwmw2@infradead.org \
--cc=eric.auger@redhat.com \
--cc=hannes@cmpxchg.org \
--cc=hao.wu@intel.com \
--cc=iommu@lists.linux-foundation.org \
--cc=jacob.jun.pan@linux.intel.com \
--cc=jean-philippe@linaro.com \
--cc=jean-philippe@linaro.org \
--cc=jgg@nvidia.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan@huawei.com \
--cc=tj@kernel.org \
--cc=yi.l.liu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).