From: Alex Williamson <alex.williamson@redhat.com> To: Jason Gunthorpe <jgg@nvidia.com> Cc: "Tian, Kevin" <kevin.tian@intel.com>, Jean-Philippe Brucker <jean-philippe@linaro.org>, "Jiang, Dave" <dave.jiang@intel.com>, "Raj, Ashok" <ashok.raj@intel.com>, "kvm@vger.kernel.org" <kvm@vger.kernel.org>, Jonathan Corbet <corbet@lwn.net>, Robin Murphy <robin.murphy@arm.com>, LKML <linux-kernel@vger.kernel.org>, "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>, David Gibson <david@gibson.dropbear.id.au>, Kirti Wankhede <kwankhede@nvidia.com>, David Woodhouse <dwmw2@infradead.org>, Jason Wang <jasowang@redhat.com> Subject: Re: [RFC] /dev/ioasid uAPI proposal Date: Wed, 2 Jun 2021 14:37:34 -0600 [thread overview] Message-ID: <20210602143734.72fb4fa4.alex.williamson@redhat.com> (raw) In-Reply-To: <20210602195404.GI1002214@nvidia.com> On Wed, 2 Jun 2021 16:54:04 -0300 Jason Gunthorpe <jgg@nvidia.com> wrote: > On Wed, Jun 02, 2021 at 01:00:53PM -0600, Alex Williamson wrote: > > > > Right, the device can generate the no-snoop transactions, but it's the > > IOMMU that essentially determines whether those transactions are > > actually still cache coherent, AIUI. > > Wow, this is really confusing stuff in the code. > > At the PCI level there is a TLP bit called no-snoop that is platform > specific. The general intention is to allow devices to selectively > bypass the CPU caching for DMAs. GPUs like to use this feature for > performance. Yes > I assume there is some exciting security issues here. Looks like > allowing cache bypass does something bad inside VMs? Looks like > allowing the VM to use the cache clear instruction that is mandatory > with cache bypass DMA causes some QOS issues? OK. IIRC, largely a DoS issue if userspace gets to choose when to emulate wbinvd rather than it being demanded for correct operation. > So how does it work? > > What I see in the intel/iommu.c is that some domains support "snoop > control" or not, based on some HW flag. This indicates if the > DMA_PTE_SNP bit is supported on a page by page basis or not. > > Since x86 always leans toward "DMA cache coherent" I'm reading some > tea leaves here: > > IOMMU_CAP_CACHE_COHERENCY, /* IOMMU can enforce cache coherent DMA > transactions */ > > And guessing that IOMMUs that implement DMA_PTE_SNP will ignore the > snoop bit in TLPs for IOVA's that have DMA_PTE_SNP set? That's my understanding as well. > Further, I guess IOMMUs that don't support PTE_SNP, or have > DMA_PTE_SNP clear will always honour the snoop bit. (backwards compat > and all) Yes. > So, IOMMU_CAP_CACHE_COHERENCY does not mean the IOMMU is DMA > incoherent with the CPU caches, it just means that that snoop bit in > the TLP cannot be enforced. ie the device *could* do no-shoop DMA > if it wants. Devices that never do no-snoop remain DMA coherent on > x86, as they always have been. Yes, IOMMU_CAP_CACHE_COHERENCY=false means we cannot force the device DMA to be coherent via the IOMMU. > IOMMU_CACHE does not mean the IOMMU is DMA cache coherent, it means > the PCI device is blocked from using no-snoop in its TLPs. > > I wonder if ARM implemented this consistently? I see VDPA is > confused.. I was confused. What a terrible set of names. > > In VFIO generic code I see it always sets IOMMU_CACHE: > > if (iommu_capable(bus, IOMMU_CAP_CACHE_COHERENCY)) > domain->prot |= IOMMU_CACHE; > > And thus also always provides IOMMU_CACHE to iommu_map: > > ret = iommu_map(d->domain, iova, (phys_addr_t)pfn << PAGE_SHIFT, > npage << PAGE_SHIFT, prot | d->prot); > > So when the IOMMU supports the no-snoop blocking security feature VFIO > turns it on and blocks no-snoop to all pages? Ok.. Yep, I'd forgotten this nuance that we need to enable it via the mapping flags. > But I must be missing something big because *something* in the IOVA > map should work with no-snoopable DMA, right? Otherwise what is the > point of exposing the invalidate instruction to the guest? > > I would think userspace should be relaying the DMA_PTE_SNP bit from > the guest's page tables up to here?? > > The KVM hookup is driven by IOMMU_CACHE which is driven by > IOMMU_CAP_CACHE_COHERENCY. So we turn on the special KVM support only > if the IOMMU can block the SNP bit? And then we map all the pages to > block the snoop bit? Huh? Right. I don't follow where you're jumping to relaying DMA_PTE_SNP from the guest page table... what page table? We don't necessarily have a vIOMMU to expose such things, I don't think it even existed when this we added. Essentially if we can ignore no-snoop at the IOMMU, then KVM doesn't need to worry about emulating wbinvd because of an assigned device, whether that device uses it or not. Win-win. > Your explanation makes perfect sense: Block guests from using the > dangerous cache invalidate instruction unless a device that uses > no-snoop is plugged in. Block devices from using no-snoop because > something about it is insecure. Ok. No-snoop itself is not insecure, but to support no-snoop in a VM KVM can't ignore wbinvd, which has overhead and abuse implications. > But the conditions I'm looking for "device that uses no-snoop" is: > - The device will issue no-snoop TLPs at all We can't really know this generically. We can try to set the enable bit to see if the device is capable of no-snoop, but that doesn't mean it will use no-snoop. > - The IOMMU will let no-snoop through > - The platform will honor no-snoop > > Only if all three are met we should allow the dangerous instruction in > KVM, right? We test at the IOMMU and assume that the IOMMU knowledge encompasses whether the platform honors no-snoop (note for example how amd and arm report true for IOMMU_CAP_CACHE_COHERENCY but seem to ignore the IOMMU_CACHE flag). We could probably use an iommu_group_for_each_dev to test if any devices within the group are capable of no-snoop if the IOMMU can't protect us, but at the time it didn't seem worthwhile. I'm still not sure if it is. > Which brings me back to my original point - this is at least partially > a device specific behavior. It depends on the content of the IOMMU > page table, it depends if the device even supports no-snoop at all. > > My guess is this works correctly for the mdev Intel kvmgt which > probably somehow allows no-snoop DMA throught the mdev SW iommu > mappings. (assuming I didn't miss a tricky iommu_map without > IOMMU_CACHe set in the type1 code?) This support existed before mdev, IIRC we needed it for direct assignment of NVIDIA GPUs. > But why is vfio-pci using it? Hmm? Use the IOMMU to reduce hypervisor overhead, let the hypervisor learn about it, ignore the subtleties of whether the device actually uses no-snoop as imprecise and poor ROI given the apparent direction of hardware. ¯\_(ツ)_/¯, Alex
WARNING: multiple messages have this Message-ID (diff)
From: Alex Williamson <alex.williamson@redhat.com> To: Jason Gunthorpe <jgg@nvidia.com> Cc: Jean-Philippe Brucker <jean-philippe@linaro.org>, "Tian, Kevin" <kevin.tian@intel.com>, "Jiang, Dave" <dave.jiang@intel.com>, "Raj, Ashok" <ashok.raj@intel.com>, "kvm@vger.kernel.org" <kvm@vger.kernel.org>, Jonathan Corbet <corbet@lwn.net>, David Woodhouse <dwmw2@infradead.org>, Jason Wang <jasowang@redhat.com>, LKML <linux-kernel@vger.kernel.org>, Kirti Wankhede <kwankhede@nvidia.com>, "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>, Robin Murphy <robin.murphy@arm.com>, David Gibson <david@gibson.dropbear.id.au> Subject: Re: [RFC] /dev/ioasid uAPI proposal Date: Wed, 2 Jun 2021 14:37:34 -0600 [thread overview] Message-ID: <20210602143734.72fb4fa4.alex.williamson@redhat.com> (raw) In-Reply-To: <20210602195404.GI1002214@nvidia.com> On Wed, 2 Jun 2021 16:54:04 -0300 Jason Gunthorpe <jgg@nvidia.com> wrote: > On Wed, Jun 02, 2021 at 01:00:53PM -0600, Alex Williamson wrote: > > > > Right, the device can generate the no-snoop transactions, but it's the > > IOMMU that essentially determines whether those transactions are > > actually still cache coherent, AIUI. > > Wow, this is really confusing stuff in the code. > > At the PCI level there is a TLP bit called no-snoop that is platform > specific. The general intention is to allow devices to selectively > bypass the CPU caching for DMAs. GPUs like to use this feature for > performance. Yes > I assume there is some exciting security issues here. Looks like > allowing cache bypass does something bad inside VMs? Looks like > allowing the VM to use the cache clear instruction that is mandatory > with cache bypass DMA causes some QOS issues? OK. IIRC, largely a DoS issue if userspace gets to choose when to emulate wbinvd rather than it being demanded for correct operation. > So how does it work? > > What I see in the intel/iommu.c is that some domains support "snoop > control" or not, based on some HW flag. This indicates if the > DMA_PTE_SNP bit is supported on a page by page basis or not. > > Since x86 always leans toward "DMA cache coherent" I'm reading some > tea leaves here: > > IOMMU_CAP_CACHE_COHERENCY, /* IOMMU can enforce cache coherent DMA > transactions */ > > And guessing that IOMMUs that implement DMA_PTE_SNP will ignore the > snoop bit in TLPs for IOVA's that have DMA_PTE_SNP set? That's my understanding as well. > Further, I guess IOMMUs that don't support PTE_SNP, or have > DMA_PTE_SNP clear will always honour the snoop bit. (backwards compat > and all) Yes. > So, IOMMU_CAP_CACHE_COHERENCY does not mean the IOMMU is DMA > incoherent with the CPU caches, it just means that that snoop bit in > the TLP cannot be enforced. ie the device *could* do no-shoop DMA > if it wants. Devices that never do no-snoop remain DMA coherent on > x86, as they always have been. Yes, IOMMU_CAP_CACHE_COHERENCY=false means we cannot force the device DMA to be coherent via the IOMMU. > IOMMU_CACHE does not mean the IOMMU is DMA cache coherent, it means > the PCI device is blocked from using no-snoop in its TLPs. > > I wonder if ARM implemented this consistently? I see VDPA is > confused.. I was confused. What a terrible set of names. > > In VFIO generic code I see it always sets IOMMU_CACHE: > > if (iommu_capable(bus, IOMMU_CAP_CACHE_COHERENCY)) > domain->prot |= IOMMU_CACHE; > > And thus also always provides IOMMU_CACHE to iommu_map: > > ret = iommu_map(d->domain, iova, (phys_addr_t)pfn << PAGE_SHIFT, > npage << PAGE_SHIFT, prot | d->prot); > > So when the IOMMU supports the no-snoop blocking security feature VFIO > turns it on and blocks no-snoop to all pages? Ok.. Yep, I'd forgotten this nuance that we need to enable it via the mapping flags. > But I must be missing something big because *something* in the IOVA > map should work with no-snoopable DMA, right? Otherwise what is the > point of exposing the invalidate instruction to the guest? > > I would think userspace should be relaying the DMA_PTE_SNP bit from > the guest's page tables up to here?? > > The KVM hookup is driven by IOMMU_CACHE which is driven by > IOMMU_CAP_CACHE_COHERENCY. So we turn on the special KVM support only > if the IOMMU can block the SNP bit? And then we map all the pages to > block the snoop bit? Huh? Right. I don't follow where you're jumping to relaying DMA_PTE_SNP from the guest page table... what page table? We don't necessarily have a vIOMMU to expose such things, I don't think it even existed when this we added. Essentially if we can ignore no-snoop at the IOMMU, then KVM doesn't need to worry about emulating wbinvd because of an assigned device, whether that device uses it or not. Win-win. > Your explanation makes perfect sense: Block guests from using the > dangerous cache invalidate instruction unless a device that uses > no-snoop is plugged in. Block devices from using no-snoop because > something about it is insecure. Ok. No-snoop itself is not insecure, but to support no-snoop in a VM KVM can't ignore wbinvd, which has overhead and abuse implications. > But the conditions I'm looking for "device that uses no-snoop" is: > - The device will issue no-snoop TLPs at all We can't really know this generically. We can try to set the enable bit to see if the device is capable of no-snoop, but that doesn't mean it will use no-snoop. > - The IOMMU will let no-snoop through > - The platform will honor no-snoop > > Only if all three are met we should allow the dangerous instruction in > KVM, right? We test at the IOMMU and assume that the IOMMU knowledge encompasses whether the platform honors no-snoop (note for example how amd and arm report true for IOMMU_CAP_CACHE_COHERENCY but seem to ignore the IOMMU_CACHE flag). We could probably use an iommu_group_for_each_dev to test if any devices within the group are capable of no-snoop if the IOMMU can't protect us, but at the time it didn't seem worthwhile. I'm still not sure if it is. > Which brings me back to my original point - this is at least partially > a device specific behavior. It depends on the content of the IOMMU > page table, it depends if the device even supports no-snoop at all. > > My guess is this works correctly for the mdev Intel kvmgt which > probably somehow allows no-snoop DMA throught the mdev SW iommu > mappings. (assuming I didn't miss a tricky iommu_map without > IOMMU_CACHe set in the type1 code?) This support existed before mdev, IIRC we needed it for direct assignment of NVIDIA GPUs. > But why is vfio-pci using it? Hmm? Use the IOMMU to reduce hypervisor overhead, let the hypervisor learn about it, ignore the subtleties of whether the device actually uses no-snoop as imprecise and poor ROI given the apparent direction of hardware. ¯\_(ツ)_/¯, Alex _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2021-06-02 20:37 UTC|newest] Thread overview: 518+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-05-27 7:58 [RFC] /dev/ioasid uAPI proposal Tian, Kevin 2021-05-27 7:58 ` Tian, Kevin 2021-05-28 2:24 ` Jason Wang 2021-05-28 2:24 ` Jason Wang 2021-05-28 20:25 ` Jason Gunthorpe 2021-05-28 20:25 ` Jason Gunthorpe 2021-05-31 8:41 ` Liu Yi L 2021-06-01 2:36 ` Jason Wang 2021-06-01 2:36 ` Jason Wang 2021-06-01 3:31 ` Liu Yi L 2021-06-01 5:08 ` Jason Wang 2021-06-01 5:08 ` Jason Wang 2021-06-01 5:23 ` Lu Baolu 2021-06-01 5:23 ` Lu Baolu 2021-06-01 5:29 ` Jason Wang 2021-06-01 5:29 ` Jason Wang 2021-06-01 5:42 ` Tian, Kevin 2021-06-01 5:42 ` Tian, Kevin 2021-06-01 6:07 ` Jason Wang 2021-06-01 6:07 ` Jason Wang 2021-06-01 6:16 ` Tian, Kevin 2021-06-01 6:16 ` Tian, Kevin 2021-06-01 8:47 ` Jason Wang 2021-06-01 8:47 ` Jason Wang 2021-06-01 17:31 ` Jason Gunthorpe 2021-06-01 17:31 ` Jason Gunthorpe 2021-06-02 8:54 ` Jason Wang 2021-06-02 8:54 ` Jason Wang 2021-06-02 17:21 ` Jason Gunthorpe 2021-06-02 17:21 ` Jason Gunthorpe 2021-06-07 13:30 ` Enrico Weigelt, metux IT consult 2021-06-07 13:30 ` Enrico Weigelt, metux IT consult 2021-06-07 18:01 ` Jason Gunthorpe 2021-06-07 18:01 ` Jason Gunthorpe 2021-06-08 10:45 ` Enrico Weigelt, metux IT consult 2021-06-08 10:45 ` Enrico Weigelt, metux IT consult 2021-06-10 2:16 ` Jason Wang 2021-06-10 2:16 ` Jason Wang 2021-06-08 1:10 ` Jason Wang 2021-06-08 1:10 ` Jason Wang 2021-06-08 13:20 ` Jason Gunthorpe 2021-06-08 13:20 ` Jason Gunthorpe 2021-06-10 2:00 ` Jason Wang 2021-06-10 2:00 ` Jason Wang 2021-06-10 4:03 ` Jason Wang 2021-06-10 4:03 ` Jason Wang 2021-06-10 11:47 ` Jason Gunthorpe 2021-06-10 11:47 ` Jason Gunthorpe 2021-06-11 5:43 ` Jason Wang 2021-06-11 5:43 ` Jason Wang 2021-06-01 17:29 ` Jason Gunthorpe 2021-06-01 17:29 ` Jason Gunthorpe 2021-06-02 8:58 ` Jason Wang 2021-06-02 8:58 ` Jason Wang 2021-06-01 4:27 ` Shenming Lu 2021-06-01 4:27 ` Shenming Lu 2021-06-01 5:10 ` Jason Wang 2021-06-01 5:10 ` Jason Wang 2021-05-28 16:23 ` Jean-Philippe Brucker 2021-05-28 16:23 ` Jean-Philippe Brucker 2021-05-28 20:16 ` Jason Gunthorpe 2021-05-28 20:16 ` Jason Gunthorpe 2021-06-01 7:50 ` Tian, Kevin 2021-06-01 7:50 ` Tian, Kevin 2021-05-28 17:35 ` Jason Gunthorpe 2021-05-28 17:35 ` Jason Gunthorpe 2021-06-01 8:10 ` Tian, Kevin 2021-06-01 8:10 ` Tian, Kevin 2021-06-01 17:42 ` Jason Gunthorpe 2021-06-01 17:42 ` Jason Gunthorpe 2021-06-02 1:33 ` Tian, Kevin 2021-06-02 1:33 ` Tian, Kevin 2021-06-02 16:09 ` Jason Gunthorpe 2021-06-02 16:09 ` Jason Gunthorpe 2021-06-03 1:29 ` Tian, Kevin 2021-06-03 1:29 ` Tian, Kevin 2021-06-03 5:09 ` David Gibson 2021-06-03 5:09 ` David Gibson 2021-06-03 6:49 ` Tian, Kevin 2021-06-03 6:49 ` Tian, Kevin 2021-06-03 11:47 ` Jason Gunthorpe 2021-06-03 11:47 ` Jason Gunthorpe 2021-06-04 2:15 ` Tian, Kevin 2021-06-04 2:15 ` Tian, Kevin 2021-06-08 0:49 ` David Gibson 2021-06-08 0:49 ` David Gibson 2021-06-09 2:52 ` Tian, Kevin 2021-06-09 2:52 ` Tian, Kevin 2021-06-02 6:32 ` David Gibson 2021-06-02 6:32 ` David Gibson 2021-06-02 16:16 ` Jason Gunthorpe 2021-06-02 16:16 ` Jason Gunthorpe 2021-06-03 2:11 ` Tian, Kevin 2021-06-03 2:11 ` Tian, Kevin 2021-06-03 5:13 ` David Gibson 2021-06-03 5:13 ` David Gibson 2021-06-03 11:52 ` Jason Gunthorpe 2021-06-03 11:52 ` Jason Gunthorpe 2021-06-08 0:53 ` David Gibson 2021-06-08 0:53 ` David Gibson 2021-06-08 19:04 ` Jason Gunthorpe 2021-06-08 19:04 ` Jason Gunthorpe 2021-06-17 2:42 ` David Gibson 2021-06-17 2:42 ` David Gibson 2021-05-28 19:58 ` Jason Gunthorpe 2021-05-28 19:58 ` Jason Gunthorpe 2021-06-01 8:38 ` Tian, Kevin 2021-06-01 8:38 ` Tian, Kevin 2021-06-01 17:56 ` Jason Gunthorpe 2021-06-01 17:56 ` Jason Gunthorpe 2021-06-02 2:00 ` Tian, Kevin 2021-06-02 2:00 ` Tian, Kevin 2021-06-02 6:57 ` David Gibson 2021-06-02 6:57 ` David Gibson 2021-06-02 16:37 ` Jason Gunthorpe 2021-06-02 16:37 ` Jason Gunthorpe 2021-06-03 5:23 ` David Gibson 2021-06-03 5:23 ` David Gibson 2021-06-03 12:28 ` Jason Gunthorpe 2021-06-03 12:28 ` Jason Gunthorpe 2021-06-08 6:04 ` David Gibson 2021-06-08 6:04 ` David Gibson 2021-06-08 19:23 ` Jason Gunthorpe 2021-06-08 19:23 ` Jason Gunthorpe 2021-06-02 6:48 ` David Gibson 2021-06-02 6:48 ` David Gibson 2021-06-02 16:58 ` Jason Gunthorpe 2021-06-02 16:58 ` Jason Gunthorpe 2021-06-03 2:49 ` Tian, Kevin 2021-06-03 2:49 ` Tian, Kevin 2021-06-03 5:48 ` David Gibson 2021-06-03 5:48 ` David Gibson 2021-06-03 5:45 ` David Gibson 2021-06-03 5:45 ` David Gibson 2021-06-03 12:11 ` Jason Gunthorpe 2021-06-03 12:11 ` Jason Gunthorpe 2021-06-04 6:08 ` Tian, Kevin 2021-06-04 6:08 ` Tian, Kevin 2021-06-04 12:33 ` Jason Gunthorpe 2021-06-04 12:33 ` Jason Gunthorpe 2021-06-04 23:20 ` Tian, Kevin 2021-06-04 23:20 ` Tian, Kevin 2021-06-08 6:13 ` David Gibson 2021-06-08 6:13 ` David Gibson 2021-06-04 10:24 ` Jean-Philippe Brucker 2021-06-04 10:24 ` Jean-Philippe Brucker 2021-06-04 12:05 ` Jason Gunthorpe 2021-06-04 12:05 ` Jason Gunthorpe 2021-06-04 17:27 ` Jacob Pan 2021-06-04 17:27 ` Jacob Pan 2021-06-04 17:40 ` Jason Gunthorpe 2021-06-04 17:40 ` Jason Gunthorpe 2021-06-08 6:31 ` David Gibson 2021-06-08 6:31 ` David Gibson 2021-06-10 16:37 ` Jean-Philippe Brucker 2021-06-10 16:37 ` Jean-Philippe Brucker 2021-06-17 3:00 ` David Gibson 2021-06-17 3:00 ` David Gibson 2021-06-18 17:03 ` Jean-Philippe Brucker 2021-06-18 17:03 ` Jean-Philippe Brucker 2021-06-18 18:30 ` Jason Gunthorpe 2021-06-18 18:30 ` Jason Gunthorpe 2021-06-23 8:19 ` Tian, Kevin 2021-06-23 8:19 ` Tian, Kevin 2021-06-23 7:57 ` Tian, Kevin 2021-06-23 7:57 ` Tian, Kevin 2021-06-24 3:49 ` David Gibson 2021-06-24 3:49 ` David Gibson 2021-05-28 20:03 ` Jason Gunthorpe 2021-05-28 20:03 ` Jason Gunthorpe 2021-06-01 7:01 ` Tian, Kevin 2021-06-01 7:01 ` Tian, Kevin 2021-06-01 20:28 ` Jason Gunthorpe 2021-06-01 20:28 ` Jason Gunthorpe 2021-06-02 1:25 ` Tian, Kevin 2021-06-02 1:25 ` Tian, Kevin 2021-06-02 23:27 ` Jason Gunthorpe 2021-06-02 23:27 ` Jason Gunthorpe 2021-06-04 8:17 ` Jean-Philippe Brucker 2021-06-04 8:17 ` Jean-Philippe Brucker 2021-06-04 8:43 ` Tian, Kevin 2021-06-04 8:43 ` Tian, Kevin 2021-06-02 8:52 ` Jason Wang 2021-06-02 8:52 ` Jason Wang 2021-06-02 16:07 ` Jason Gunthorpe 2021-06-02 16:07 ` Jason Gunthorpe 2021-06-01 22:22 ` Alex Williamson 2021-06-01 22:22 ` Alex Williamson 2021-06-02 2:20 ` Tian, Kevin 2021-06-02 2:20 ` Tian, Kevin 2021-06-02 16:01 ` Jason Gunthorpe 2021-06-02 16:01 ` Jason Gunthorpe 2021-06-02 17:11 ` Alex Williamson 2021-06-02 17:11 ` Alex Williamson 2021-06-02 17:35 ` Jason Gunthorpe 2021-06-02 17:35 ` Jason Gunthorpe 2021-06-02 18:01 ` Alex Williamson 2021-06-02 18:01 ` Alex Williamson 2021-06-02 18:09 ` Jason Gunthorpe 2021-06-02 18:09 ` Jason Gunthorpe 2021-06-02 19:00 ` Alex Williamson 2021-06-02 19:00 ` Alex Williamson 2021-06-02 19:54 ` Jason Gunthorpe 2021-06-02 19:54 ` Jason Gunthorpe 2021-06-02 20:37 ` Alex Williamson [this message] 2021-06-02 20:37 ` Alex Williamson 2021-06-02 22:45 ` Jason Gunthorpe 2021-06-02 22:45 ` Jason Gunthorpe 2021-06-03 2:50 ` Alex Williamson 2021-06-03 2:50 ` Alex Williamson 2021-06-03 3:22 ` Tian, Kevin 2021-06-03 3:22 ` Tian, Kevin 2021-06-03 4:14 ` Alex Williamson 2021-06-03 4:14 ` Alex Williamson 2021-06-03 5:18 ` Tian, Kevin 2021-06-03 5:18 ` Tian, Kevin 2021-06-03 12:40 ` Jason Gunthorpe 2021-06-03 12:40 ` Jason Gunthorpe 2021-06-03 20:41 ` Alex Williamson 2021-06-03 20:41 ` Alex Williamson 2021-06-04 9:19 ` Tian, Kevin 2021-06-04 9:19 ` Tian, Kevin 2021-06-04 15:37 ` Alex Williamson 2021-06-04 15:37 ` Alex Williamson 2021-06-04 12:13 ` Jason Gunthorpe 2021-06-04 12:13 ` Jason Gunthorpe 2021-06-04 21:45 ` Alex Williamson 2021-06-04 21:45 ` Alex Williamson 2021-06-04 7:33 ` Tian, Kevin 2021-06-04 7:33 ` Tian, Kevin 2021-06-03 12:34 ` Jason Gunthorpe 2021-06-03 12:34 ` Jason Gunthorpe 2021-06-03 20:01 ` Alex Williamson 2021-06-03 20:01 ` Alex Williamson 2021-06-03 20:10 ` Jason Gunthorpe 2021-06-03 20:10 ` Jason Gunthorpe 2021-06-03 21:44 ` Alex Williamson 2021-06-03 21:44 ` Alex Williamson 2021-06-04 8:38 ` Tian, Kevin 2021-06-04 8:38 ` Tian, Kevin 2021-06-04 12:28 ` Jason Gunthorpe 2021-06-04 12:28 ` Jason Gunthorpe 2021-06-04 15:26 ` Alex Williamson 2021-06-04 15:26 ` Alex Williamson 2021-06-04 15:40 ` Paolo Bonzini 2021-06-04 15:40 ` Paolo Bonzini 2021-06-04 15:50 ` Jason Gunthorpe 2021-06-04 15:50 ` Jason Gunthorpe 2021-06-04 15:57 ` Paolo Bonzini 2021-06-04 15:57 ` Paolo Bonzini 2021-06-04 16:03 ` Jason Gunthorpe 2021-06-04 16:03 ` Jason Gunthorpe 2021-06-04 16:10 ` Paolo Bonzini 2021-06-04 16:10 ` Paolo Bonzini 2021-06-04 17:22 ` Jason Gunthorpe 2021-06-04 17:22 ` Jason Gunthorpe 2021-06-04 21:29 ` Alex Williamson 2021-06-04 21:29 ` Alex Williamson 2021-06-04 23:01 ` Jason Gunthorpe 2021-06-04 23:01 ` Jason Gunthorpe 2021-06-07 15:41 ` Alex Williamson 2021-06-07 15:41 ` Alex Williamson 2021-06-07 18:18 ` Jason Gunthorpe 2021-06-07 18:18 ` Jason Gunthorpe 2021-06-07 18:59 ` Alex Williamson 2021-06-07 18:59 ` Alex Williamson 2021-06-07 19:08 ` Jason Gunthorpe 2021-06-07 19:08 ` Jason Gunthorpe 2021-06-07 19:41 ` Alex Williamson 2021-06-07 19:41 ` Alex Williamson 2021-06-07 23:03 ` Jason Gunthorpe 2021-06-07 23:03 ` Jason Gunthorpe 2021-06-08 0:30 ` Alex Williamson 2021-06-08 0:30 ` Alex Williamson 2021-06-08 1:20 ` Jason Wang 2021-06-08 1:20 ` Jason Wang 2021-06-30 6:53 ` Christoph Hellwig 2021-06-30 6:53 ` Christoph Hellwig 2021-06-30 6:49 ` Christoph Hellwig 2021-06-30 6:49 ` Christoph Hellwig 2021-06-07 3:25 ` Tian, Kevin 2021-06-07 3:25 ` Tian, Kevin 2021-06-07 6:51 ` Paolo Bonzini 2021-06-07 6:51 ` Paolo Bonzini 2021-06-07 18:01 ` Jason Gunthorpe 2021-06-07 18:01 ` Jason Gunthorpe 2021-06-30 6:56 ` Christoph Hellwig 2021-06-30 6:56 ` Christoph Hellwig 2021-06-05 6:22 ` Paolo Bonzini 2021-06-05 6:22 ` Paolo Bonzini 2021-06-07 3:50 ` Tian, Kevin 2021-06-07 3:50 ` Tian, Kevin 2021-06-07 17:59 ` Jason Gunthorpe 2021-06-07 17:59 ` Jason Gunthorpe 2021-06-08 7:56 ` Paolo Bonzini 2021-06-08 7:56 ` Paolo Bonzini 2021-06-08 13:15 ` Jason Gunthorpe 2021-06-08 13:15 ` Jason Gunthorpe 2021-06-08 13:44 ` Paolo Bonzini 2021-06-08 13:44 ` Paolo Bonzini 2021-06-08 18:47 ` Alex Williamson 2021-06-08 18:47 ` Alex Williamson 2021-06-08 19:00 ` Jason Gunthorpe 2021-06-08 19:00 ` Jason Gunthorpe 2021-06-09 8:51 ` Enrico Weigelt, metux IT consult 2021-06-09 8:51 ` Enrico Weigelt, metux IT consult 2021-06-09 9:11 ` Paolo Bonzini 2021-06-09 9:11 ` Paolo Bonzini 2021-06-09 11:54 ` Jason Gunthorpe 2021-06-09 11:54 ` Jason Gunthorpe 2021-06-09 14:31 ` Alex Williamson 2021-06-09 14:31 ` Alex Williamson 2021-06-09 14:45 ` Jason Gunthorpe 2021-06-09 14:45 ` Jason Gunthorpe 2021-06-09 15:20 ` Paolo Bonzini 2021-06-09 15:20 ` Paolo Bonzini 2021-10-27 6:18 ` Tian, Kevin 2021-10-27 6:18 ` Tian, Kevin 2021-10-27 10:32 ` Paolo Bonzini 2021-10-27 10:32 ` Paolo Bonzini 2021-10-28 1:50 ` Tian, Kevin 2021-10-28 1:50 ` Tian, Kevin 2021-06-09 2:49 ` Tian, Kevin 2021-06-09 2:49 ` Tian, Kevin 2021-06-09 11:57 ` Jason Gunthorpe 2021-06-09 11:57 ` Jason Gunthorpe 2021-06-09 12:46 ` Paolo Bonzini 2021-06-09 12:46 ` Paolo Bonzini 2021-06-09 12:47 ` Jason Gunthorpe 2021-06-09 12:47 ` Jason Gunthorpe 2021-06-09 13:24 ` Paolo Bonzini 2021-06-09 13:24 ` Paolo Bonzini 2021-06-09 14:32 ` Jason Gunthorpe 2021-06-09 14:32 ` Jason Gunthorpe 2021-06-30 7:01 ` Christoph Hellwig 2021-06-30 7:01 ` Christoph Hellwig 2021-06-09 18:09 ` Alex Williamson 2021-06-09 18:09 ` Alex Williamson 2021-06-03 2:52 ` Jason Wang 2021-06-03 2:52 ` Jason Wang 2021-06-03 13:09 ` Jason Gunthorpe 2021-06-03 13:09 ` Jason Gunthorpe 2021-06-04 1:11 ` Jason Wang 2021-06-04 1:11 ` Jason Wang 2021-06-04 11:58 ` Jason Gunthorpe 2021-06-04 11:58 ` Jason Gunthorpe 2021-06-07 3:18 ` Jason Wang 2021-06-07 3:18 ` Jason Wang 2021-06-07 14:14 ` Jason Gunthorpe 2021-06-07 14:14 ` Jason Gunthorpe 2021-06-08 1:00 ` Jason Wang 2021-06-08 1:00 ` Jason Wang 2021-06-08 8:54 ` Enrico Weigelt, metux IT consult 2021-06-08 8:54 ` Enrico Weigelt, metux IT consult 2021-06-08 12:52 ` Jason Gunthorpe 2021-06-08 12:52 ` Jason Gunthorpe 2021-06-30 7:07 ` Christoph Hellwig 2021-06-30 7:07 ` Christoph Hellwig 2021-06-30 7:05 ` Christoph Hellwig 2021-06-30 7:05 ` Christoph Hellwig 2021-06-08 2:37 ` David Gibson 2021-06-08 2:37 ` David Gibson 2021-06-08 13:17 ` Jason Gunthorpe 2021-06-08 13:17 ` Jason Gunthorpe 2021-06-17 3:47 ` David Gibson 2021-06-17 3:47 ` David Gibson 2021-06-23 7:59 ` Tian, Kevin 2021-06-23 7:59 ` Tian, Kevin 2021-06-24 3:53 ` David Gibson 2021-06-24 3:53 ` David Gibson 2021-05-28 23:36 ` Jason Gunthorpe 2021-05-28 23:36 ` Jason Gunthorpe 2021-05-31 11:31 ` Liu Yi L 2021-05-31 11:31 ` Liu Yi L 2021-05-31 18:09 ` Jason Gunthorpe 2021-05-31 18:09 ` Jason Gunthorpe 2021-06-01 3:08 ` Lu Baolu 2021-06-01 3:08 ` Lu Baolu 2021-06-01 17:24 ` Jason Gunthorpe 2021-06-01 17:24 ` Jason Gunthorpe 2021-06-01 1:25 ` Lu Baolu 2021-06-01 1:25 ` Lu Baolu 2021-06-01 11:09 ` Lu Baolu 2021-06-01 11:09 ` Lu Baolu 2021-06-01 17:26 ` Jason Gunthorpe 2021-06-01 17:26 ` Jason Gunthorpe 2021-06-02 4:01 ` Lu Baolu 2021-06-02 4:01 ` Lu Baolu 2021-06-02 23:23 ` Jason Gunthorpe 2021-06-02 23:23 ` Jason Gunthorpe 2021-06-03 5:49 ` Lu Baolu 2021-06-03 5:49 ` Lu Baolu 2021-06-03 5:54 ` David Gibson 2021-06-03 5:54 ` David Gibson 2021-06-03 6:50 ` Lu Baolu 2021-06-03 6:50 ` Lu Baolu 2021-06-03 12:56 ` Jason Gunthorpe 2021-06-03 12:56 ` Jason Gunthorpe 2021-06-02 7:22 ` David Gibson 2021-06-02 7:22 ` David Gibson 2021-06-03 6:39 ` Tian, Kevin 2021-06-03 6:39 ` Tian, Kevin 2021-06-03 13:05 ` Jason Gunthorpe 2021-06-03 13:05 ` Jason Gunthorpe 2021-06-04 6:37 ` Tian, Kevin 2021-06-04 6:37 ` Tian, Kevin 2021-06-04 12:09 ` Jason Gunthorpe 2021-06-04 12:09 ` Jason Gunthorpe 2021-06-04 23:10 ` Tian, Kevin 2021-06-04 23:10 ` Tian, Kevin 2021-06-07 17:54 ` Jason Gunthorpe 2021-06-07 17:54 ` Jason Gunthorpe 2021-06-15 8:59 ` Tian, Kevin 2021-06-15 8:59 ` Tian, Kevin 2021-06-15 15:06 ` Jason Gunthorpe 2021-06-15 15:06 ` Jason Gunthorpe 2021-06-15 22:59 ` Tian, Kevin 2021-06-15 22:59 ` Tian, Kevin 2021-06-15 23:02 ` Jason Gunthorpe 2021-06-15 23:02 ` Jason Gunthorpe 2021-06-15 23:09 ` Tian, Kevin 2021-06-15 23:09 ` Tian, Kevin 2021-06-15 23:40 ` Jason Gunthorpe 2021-06-15 23:40 ` Jason Gunthorpe 2021-06-15 23:56 ` Tian, Kevin 2021-06-15 23:56 ` Tian, Kevin 2021-06-15 23:59 ` Jason Gunthorpe 2021-06-15 23:59 ` Jason Gunthorpe 2021-06-16 0:02 ` Tian, Kevin 2021-06-16 0:02 ` Tian, Kevin 2021-05-31 17:37 ` Parav Pandit 2021-05-31 17:37 ` Parav Pandit 2021-05-31 18:12 ` Jason Gunthorpe 2021-05-31 18:12 ` Jason Gunthorpe 2021-06-01 12:04 ` Parav Pandit 2021-06-01 12:04 ` Parav Pandit 2021-06-01 17:36 ` Jason Gunthorpe 2021-06-01 17:36 ` Jason Gunthorpe 2021-06-02 8:38 ` Enrico Weigelt, metux IT consult 2021-06-02 8:38 ` Enrico Weigelt, metux IT consult 2021-06-02 12:41 ` Parav Pandit 2021-06-02 12:41 ` Parav Pandit 2021-06-01 4:31 ` Shenming Lu 2021-06-01 4:31 ` Shenming Lu 2021-06-01 5:10 ` Lu Baolu 2021-06-01 5:10 ` Lu Baolu 2021-06-01 7:15 ` Shenming Lu 2021-06-01 7:15 ` Shenming Lu 2021-06-01 12:30 ` Lu Baolu 2021-06-01 12:30 ` Lu Baolu 2021-06-01 13:10 ` Shenming Lu 2021-06-01 13:10 ` Shenming Lu 2021-06-01 17:33 ` Jason Gunthorpe 2021-06-01 17:33 ` Jason Gunthorpe 2021-06-02 4:50 ` Shenming Lu 2021-06-02 4:50 ` Shenming Lu 2021-06-03 18:19 ` Jacob Pan 2021-06-03 18:19 ` Jacob Pan 2021-06-04 1:30 ` Jason Wang 2021-06-04 1:30 ` Jason Wang 2021-06-04 16:22 ` Jacob Pan 2021-06-04 16:22 ` Jacob Pan 2021-06-04 16:22 ` Jason Gunthorpe 2021-06-04 16:22 ` Jason Gunthorpe 2021-06-04 18:05 ` Jacob Pan 2021-06-04 18:05 ` Jacob Pan 2021-06-04 2:03 ` Shenming Lu 2021-06-04 2:03 ` Shenming Lu 2021-06-07 12:19 ` Liu, Yi L 2021-06-07 12:19 ` Liu, Yi L 2021-06-08 1:09 ` Shenming Lu 2021-06-08 1:09 ` Shenming Lu 2021-06-01 17:30 ` Parav Pandit 2021-06-01 17:30 ` Parav Pandit 2021-06-03 20:58 ` Jacob Pan 2021-06-03 20:58 ` Jacob Pan 2021-06-08 6:30 ` Parav Pandit 2021-06-08 6:30 ` Parav Pandit 2021-06-02 6:15 ` David Gibson 2021-06-02 6:15 ` David Gibson 2021-06-02 17:19 ` Jason Gunthorpe 2021-06-02 17:19 ` Jason Gunthorpe 2021-06-03 3:02 ` Tian, Kevin 2021-06-03 3:02 ` Tian, Kevin 2021-06-03 6:26 ` David Gibson 2021-06-03 6:26 ` David Gibson 2021-06-03 12:46 ` Jason Gunthorpe 2021-06-03 12:46 ` Jason Gunthorpe 2021-06-04 6:27 ` Tian, Kevin 2021-06-04 6:27 ` Tian, Kevin 2021-06-03 7:17 ` Tian, Kevin 2021-06-03 7:17 ` Tian, Kevin 2021-06-03 12:49 ` Jason Gunthorpe 2021-06-03 12:49 ` Jason Gunthorpe 2021-06-08 5:49 ` David Gibson 2021-06-08 5:49 ` David Gibson 2021-06-03 8:12 ` Tian, Kevin 2021-06-03 8:12 ` Tian, Kevin 2021-06-17 4:07 ` David Gibson 2021-06-17 4:07 ` David Gibson 2021-06-23 8:00 ` Tian, Kevin 2021-06-23 8:00 ` Tian, Kevin 2021-06-24 3:55 ` David Gibson 2021-06-24 3:55 ` David Gibson 2021-06-02 8:56 ` Enrico Weigelt, metux IT consult 2021-06-02 8:56 ` Enrico Weigelt, metux IT consult 2021-06-02 17:24 ` Jason Gunthorpe 2021-06-02 17:24 ` Jason Gunthorpe 2021-06-04 10:44 ` Enrico Weigelt, metux IT consult 2021-06-04 10:44 ` Enrico Weigelt, metux IT consult 2021-06-04 12:30 ` Jason Gunthorpe 2021-06-04 12:30 ` Jason Gunthorpe 2021-06-08 1:15 ` David Gibson 2021-06-08 1:15 ` David Gibson 2021-06-08 10:43 ` Enrico Weigelt, metux IT consult 2021-06-08 10:43 ` Enrico Weigelt, metux IT consult 2021-06-08 13:11 ` Jason Gunthorpe 2021-06-08 13:11 ` Jason Gunthorpe
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20210602143734.72fb4fa4.alex.williamson@redhat.com \ --to=alex.williamson@redhat.com \ --cc=ashok.raj@intel.com \ --cc=corbet@lwn.net \ --cc=dave.jiang@intel.com \ --cc=david@gibson.dropbear.id.au \ --cc=dwmw2@infradead.org \ --cc=iommu@lists.linux-foundation.org \ --cc=jasowang@redhat.com \ --cc=jean-philippe@linaro.org \ --cc=jgg@nvidia.com \ --cc=kevin.tian@intel.com \ --cc=kvm@vger.kernel.org \ --cc=kwankhede@nvidia.com \ --cc=linux-kernel@vger.kernel.org \ --cc=robin.murphy@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.