iommu.lists.linux-foundation.org archive mirror
 help / color / mirror / Atom feed
From: Alex Williamson <alex.williamson@redhat.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Jean-Philippe Brucker <jean-philippe@linaro.org>,
	"Tian, Kevin" <kevin.tian@intel.com>,
	"Jiang, Dave" <dave.jiang@intel.com>,
	"Raj,  Ashok" <ashok.raj@intel.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	Jonathan Corbet <corbet@lwn.net>,
	David Woodhouse <dwmw2@infradead.org>,
	Jason Wang <jasowang@redhat.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Kirti Wankhede <kwankhede@nvidia.com>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>,
	Robin Murphy <robin.murphy@arm.com>,
	David Gibson <david@gibson.dropbear.id.au>
Subject: Re: [RFC] /dev/ioasid uAPI proposal
Date: Wed, 2 Jun 2021 13:00:53 -0600	[thread overview]
Message-ID: <20210602130053.615db578.alex.williamson@redhat.com> (raw)
In-Reply-To: <20210602180925.GH1002214@nvidia.com>

On Wed, 2 Jun 2021 15:09:25 -0300
Jason Gunthorpe <jgg@nvidia.com> wrote:

> On Wed, Jun 02, 2021 at 12:01:11PM -0600, Alex Williamson wrote:
> > On Wed, 2 Jun 2021 14:35:10 -0300
> > Jason Gunthorpe <jgg@nvidia.com> wrote:
> >   
> > > On Wed, Jun 02, 2021 at 11:11:17AM -0600, Alex Williamson wrote:
> > >   
> > > > > > > present and be able to test if DMA for that device is cache
> > > > > > > coherent.      
> > > > > 
> > > > > Why is this such a strong linkage to VFIO and not just a 'hey kvm
> > > > > emulate wbinvd' flag from qemu?    
> > > > 
> > > > IIRC, wbinvd has host implications, a malicious user could tell KVM to
> > > > emulate wbinvd then run the op in a loop and induce a disproportionate
> > > > load on the system.  We therefore wanted a way that it would only be
> > > > enabled when required.    
> > > 
> > > I think the non-coherentness is vfio_device specific? eg a specific
> > > device will decide if it is coherent or not?  
> > 
> > No, this is specifically whether DMA is cache coherent to the
> > processor, ie. in the case of wbinvd whether the processor needs to
> > invalidate its cache in order to see data from DMA.  
> 
> I'm confused. This is x86, all DMA is cache coherent unless the device
> is doing something special.
> 
> > > If yes I'd recast this to call kvm_arch_register_noncoherent_dma()
> > > from the VFIO_GROUP_NOTIFY_SET_KVM in the struct vfio_device
> > > implementation and not link it through the IOMMU.  
> > 
> > The IOMMU tells us if DMA is cache coherent, VFIO_DMA_CC_IOMMU maps to
> > IOMMU_CAP_CACHE_COHERENCY for all domains within a container.  
> 
> And this special IOMMU mode is basically requested by the device
> driver, right? Because if you use this mode you have to also use
> special programming techniques.
> 
> This smells like all the "snoop bypass" stuff from PCIE (for GPUs
> even) in a different guise - it is device triggered, not platform
> triggered behavior.

Right, the device can generate the no-snoop transactions, but it's the
IOMMU that essentially determines whether those transactions are
actually still cache coherent, AIUI.

I did experiment with virtually hardwiring the Enable No-Snoop bit in
the Device Control Register to zero, which would be generically allowed
by the PCIe spec, but then we get into subtle dependencies in the device
drivers and clearing the bit again after any sort of reset and the
backdoor accesses to config space which exist mostly in the class of
devices that might use no-snoop transactions (yes, GPUs suck).

It was much easier and more robust to ignore the device setting and rely
on the IOMMU behavior.  Yes, maybe we sometimes emulate wbinvd for VMs
where the device doesn't support no-snoop, but it seemed like platforms
were headed in this direction where no-snoop was ignored anyway.
Thanks,

Alex

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

  reply	other threads:[~2021-06-02 19:01 UTC|newest]

Thread overview: 260+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-27  7:58 [RFC] /dev/ioasid uAPI proposal Tian, Kevin
2021-05-28  2:24 ` Jason Wang
2021-05-28 20:25   ` Jason Gunthorpe
2021-05-31  8:41   ` Liu Yi L
2021-06-01  2:36     ` Jason Wang
2021-06-01  3:31       ` Liu Yi L
2021-06-01  5:08         ` Jason Wang
2021-06-01  5:23           ` Lu Baolu
2021-06-01  5:29             ` Jason Wang
2021-06-01  5:42               ` Tian, Kevin
2021-06-01  6:07                 ` Jason Wang
2021-06-01  6:16                   ` Tian, Kevin
2021-06-01  8:47                     ` Jason Wang
2021-06-01 17:31                       ` Jason Gunthorpe
2021-06-02  8:54                         ` Jason Wang
2021-06-02 17:21                           ` Jason Gunthorpe
2021-06-07 13:30                             ` Enrico Weigelt, metux IT consult
2021-06-07 18:01                               ` Jason Gunthorpe
2021-06-08 10:45                                 ` Enrico Weigelt, metux IT consult
2021-06-10  2:16                                   ` Jason Wang
2021-06-08  1:10                             ` Jason Wang
2021-06-08 13:20                               ` Jason Gunthorpe
2021-06-10  2:00                                 ` Jason Wang
2021-06-10  4:03                                   ` Jason Wang
2021-06-10 11:47                                   ` Jason Gunthorpe
2021-06-11  5:43                                     ` Jason Wang
2021-06-01 17:29                   ` Jason Gunthorpe
2021-06-02  8:58                     ` Jason Wang
2021-06-01  4:27       ` Shenming Lu
2021-06-01  5:10         ` Jason Wang
2021-05-28 16:23 ` Jean-Philippe Brucker
2021-05-28 20:16   ` Jason Gunthorpe
2021-06-01  7:50   ` Tian, Kevin
2021-05-28 17:35 ` Jason Gunthorpe
2021-06-01  8:10   ` Tian, Kevin
2021-06-01 17:42     ` Jason Gunthorpe
2021-06-02  1:33       ` Tian, Kevin
2021-06-02 16:09         ` Jason Gunthorpe
2021-06-03  1:29           ` Tian, Kevin
2021-06-03  5:09             ` David Gibson
2021-06-03  6:49               ` Tian, Kevin
2021-06-03 11:47                 ` Jason Gunthorpe
2021-06-04  2:15                   ` Tian, Kevin
2021-06-08  0:49                 ` David Gibson
2021-06-09  2:52                   ` Tian, Kevin
2021-06-02  6:32   ` David Gibson
2021-06-02 16:16     ` Jason Gunthorpe
2021-06-03  2:11       ` Tian, Kevin
2021-06-03  5:13       ` David Gibson
2021-06-03 11:52         ` Jason Gunthorpe
2021-06-08  0:53           ` David Gibson
2021-06-08 19:04             ` Jason Gunthorpe
2021-06-17  2:42               ` David Gibson
2021-05-28 19:58 ` Jason Gunthorpe
2021-06-01  8:38   ` Tian, Kevin
2021-06-01 17:56     ` Jason Gunthorpe
2021-06-02  2:00       ` Tian, Kevin
2021-06-02  6:57       ` David Gibson
2021-06-02 16:37         ` Jason Gunthorpe
2021-06-03  5:23           ` David Gibson
2021-06-03 12:28             ` Jason Gunthorpe
2021-06-08  6:04               ` David Gibson
2021-06-08 19:23                 ` Jason Gunthorpe
2021-06-02  6:48   ` David Gibson
2021-06-02 16:58     ` Jason Gunthorpe
2021-06-03  2:49       ` Tian, Kevin
2021-06-03  5:48         ` David Gibson
2021-06-03  5:45       ` David Gibson
2021-06-03 12:11         ` Jason Gunthorpe
2021-06-04  6:08           ` Tian, Kevin
2021-06-04 12:33             ` Jason Gunthorpe
2021-06-04 23:20               ` Tian, Kevin
2021-06-08  6:13           ` David Gibson
2021-06-04 10:24         ` Jean-Philippe Brucker
2021-06-04 12:05           ` Jason Gunthorpe
2021-06-04 17:27             ` Jacob Pan
2021-06-04 17:40               ` Jason Gunthorpe
2021-06-08  6:31           ` David Gibson
2021-06-10 16:37             ` Jean-Philippe Brucker
2021-06-17  3:00               ` David Gibson
2021-06-18 17:03                 ` Jean-Philippe Brucker
2021-06-18 18:30                   ` Jason Gunthorpe
2021-06-23  8:19                     ` Tian, Kevin
2021-06-23  7:57                   ` Tian, Kevin
2021-06-24  3:49                   ` David Gibson
2021-05-28 20:03 ` Jason Gunthorpe
2021-06-01  7:01   ` Tian, Kevin
2021-06-01 20:28     ` Jason Gunthorpe
2021-06-02  1:25       ` Tian, Kevin
2021-06-02 23:27         ` Jason Gunthorpe
2021-06-04  8:17         ` Jean-Philippe Brucker
2021-06-04  8:43           ` Tian, Kevin
2021-06-02  8:52       ` Jason Wang
2021-06-02 16:07         ` Jason Gunthorpe
2021-06-01 22:22     ` Alex Williamson
2021-06-02  2:20       ` Tian, Kevin
2021-06-02 16:01         ` Jason Gunthorpe
2021-06-02 17:11           ` Alex Williamson
2021-06-02 17:35             ` Jason Gunthorpe
2021-06-02 18:01               ` Alex Williamson
2021-06-02 18:09                 ` Jason Gunthorpe
2021-06-02 19:00                   ` Alex Williamson [this message]
2021-06-02 19:54                     ` Jason Gunthorpe
2021-06-02 20:37                       ` Alex Williamson
2021-06-02 22:45                         ` Jason Gunthorpe
2021-06-03  2:50                           ` Alex Williamson
2021-06-03  3:22                             ` Tian, Kevin
2021-06-03  4:14                               ` Alex Williamson
2021-06-03  5:18                                 ` Tian, Kevin
2021-06-03 12:40                               ` Jason Gunthorpe
2021-06-03 20:41                                 ` Alex Williamson
2021-06-04  9:19                                   ` Tian, Kevin
2021-06-04 15:37                                     ` Alex Williamson
2021-06-04 12:13                                   ` Jason Gunthorpe
2021-06-04 21:45                                     ` Alex Williamson
2021-06-04  7:33                                 ` Tian, Kevin
2021-06-03 12:34                             ` Jason Gunthorpe
2021-06-03 20:01                               ` Alex Williamson
2021-06-03 20:10                                 ` Jason Gunthorpe
2021-06-03 21:44                                   ` Alex Williamson
2021-06-04  8:38                                     ` Tian, Kevin
2021-06-04 12:28                                       ` Jason Gunthorpe
2021-06-04 15:26                                         ` Alex Williamson
2021-06-04 15:40                                           ` Paolo Bonzini
2021-06-04 15:50                                             ` Jason Gunthorpe
2021-06-04 15:57                                               ` Paolo Bonzini
2021-06-04 16:03                                                 ` Jason Gunthorpe
2021-06-04 16:10                                                   ` Paolo Bonzini
2021-06-04 17:22                                                     ` Jason Gunthorpe
2021-06-04 21:29                                                       ` Alex Williamson
2021-06-04 23:01                                                         ` Jason Gunthorpe
2021-06-07 15:41                                                           ` Alex Williamson
2021-06-07 18:18                                                             ` Jason Gunthorpe
2021-06-07 18:59                                                               ` Alex Williamson
2021-06-07 19:08                                                                 ` Jason Gunthorpe
2021-06-07 19:41                                                                   ` Alex Williamson
2021-06-07 23:03                                                                     ` Jason Gunthorpe
2021-06-08  0:30                                                                       ` Alex Williamson
2021-06-08  1:20                                                                     ` Jason Wang
2021-06-30  6:53                                                                       ` Christoph Hellwig
2021-06-30  6:49                                                                   ` Christoph Hellwig
2021-06-07  3:25                                                         ` Tian, Kevin
2021-06-07  6:51                                                           ` Paolo Bonzini
2021-06-07 18:01                                                             ` Jason Gunthorpe
2021-06-30  6:56                                                           ` Christoph Hellwig
2021-06-05  6:22                                                       ` Paolo Bonzini
2021-06-07  3:50                                                         ` Tian, Kevin
2021-06-07 17:59                                                         ` Jason Gunthorpe
2021-06-08  7:56                                                           ` Paolo Bonzini
2021-06-08 13:15                                                             ` Jason Gunthorpe
2021-06-08 13:44                                                               ` Paolo Bonzini
2021-06-08 18:47                                                                 ` Alex Williamson
2021-06-08 19:00                                                                   ` Jason Gunthorpe
2021-06-09  8:51                                                                     ` Enrico Weigelt, metux IT consult
2021-06-09  9:11                                                                       ` Paolo Bonzini
2021-06-09 11:54                                                                         ` Jason Gunthorpe
2021-06-09 14:31                                                                           ` Alex Williamson
2021-06-09 14:45                                                                             ` Jason Gunthorpe
2021-06-09 15:20                                                                               ` Paolo Bonzini
2021-10-27  6:18                                                                                 ` Tian, Kevin
2021-10-27 10:32                                                                                   ` Paolo Bonzini
2021-10-28  1:50                                                                                     ` Tian, Kevin
2021-06-09  2:49                                                                   ` Tian, Kevin
2021-06-09 11:57                                                                     ` Jason Gunthorpe
2021-06-09 12:46                                                                       ` Paolo Bonzini
2021-06-09 12:47                                                                         ` Jason Gunthorpe
2021-06-09 13:24                                                                           ` Paolo Bonzini
2021-06-09 14:32                                                                             ` Jason Gunthorpe
2021-06-30  7:01                                                                           ` Christoph Hellwig
2021-06-09 18:09                                                                     ` Alex Williamson
2021-06-03  2:52                         ` Jason Wang
2021-06-03 13:09                           ` Jason Gunthorpe
2021-06-04  1:11                             ` Jason Wang
2021-06-04 11:58                               ` Jason Gunthorpe
2021-06-07  3:18                                 ` Jason Wang
2021-06-07 14:14                                   ` Jason Gunthorpe
2021-06-08  1:00                                     ` Jason Wang
2021-06-08  8:54                                       ` Enrico Weigelt, metux IT consult
2021-06-08 12:52                                         ` Jason Gunthorpe
2021-06-30  7:07                                     ` Christoph Hellwig
2021-06-30  7:05                                 ` Christoph Hellwig
2021-06-08  2:37       ` David Gibson
2021-06-08 13:17         ` Jason Gunthorpe
2021-06-17  3:47           ` David Gibson
2021-06-23  7:59             ` Tian, Kevin
2021-06-24  3:53               ` David Gibson
2021-05-28 23:36 ` Jason Gunthorpe
2021-05-31 11:31   ` Liu Yi L
2021-05-31 18:09     ` Jason Gunthorpe
2021-06-01  3:08       ` Lu Baolu
2021-06-01 17:24         ` Jason Gunthorpe
2021-06-01  1:25     ` Lu Baolu
2021-06-01 11:09   ` Lu Baolu
2021-06-01 17:26     ` Jason Gunthorpe
2021-06-02  4:01       ` Lu Baolu
2021-06-02 23:23         ` Jason Gunthorpe
2021-06-03  5:49           ` Lu Baolu
2021-06-03  5:54     ` David Gibson
2021-06-03  6:50       ` Lu Baolu
2021-06-03 12:56         ` Jason Gunthorpe
2021-06-02  7:22   ` David Gibson
2021-06-03  6:39   ` Tian, Kevin
2021-06-03 13:05     ` Jason Gunthorpe
2021-06-04  6:37       ` Tian, Kevin
2021-06-04 12:09         ` Jason Gunthorpe
2021-06-04 23:10           ` Tian, Kevin
2021-06-07 17:54             ` Jason Gunthorpe
2021-06-15  8:59       ` Tian, Kevin
2021-06-15 15:06         ` Jason Gunthorpe
2021-06-15 22:59           ` Tian, Kevin
2021-06-15 23:02             ` Jason Gunthorpe
2021-06-15 23:09               ` Tian, Kevin
2021-06-15 23:40                 ` Jason Gunthorpe
2021-06-15 23:56                   ` Tian, Kevin
2021-06-15 23:59                     ` Jason Gunthorpe
2021-06-16  0:02                       ` Tian, Kevin
2021-05-31 17:37 ` Parav Pandit
2021-05-31 18:12   ` Jason Gunthorpe
2021-06-01 12:04     ` Parav Pandit
2021-06-01 17:36       ` Jason Gunthorpe
2021-06-02  8:38   ` Enrico Weigelt, metux IT consult
2021-06-02 12:41     ` Parav Pandit
2021-06-01  4:31 ` Shenming Lu
2021-06-01  5:10   ` Lu Baolu
2021-06-01  7:15     ` Shenming Lu
2021-06-01 12:30       ` Lu Baolu
2021-06-01 13:10         ` Shenming Lu
2021-06-01 17:33         ` Jason Gunthorpe
2021-06-02  4:50           ` Shenming Lu
2021-06-03 18:19             ` Jacob Pan
2021-06-04  1:30               ` Jason Wang
2021-06-04 16:22                 ` Jacob Pan
2021-06-04 16:22                   ` Jason Gunthorpe
2021-06-04 18:05                     ` Jacob Pan
2021-06-04  2:03               ` Shenming Lu
2021-06-07 12:19                 ` Liu, Yi L
2021-06-08  1:09                   ` Shenming Lu
2021-06-01 17:30 ` Parav Pandit
2021-06-03 20:58   ` Jacob Pan
2021-06-08  6:30     ` Parav Pandit
2021-06-02  6:15 ` David Gibson
2021-06-02 17:19   ` Jason Gunthorpe
2021-06-03  3:02     ` Tian, Kevin
2021-06-03  6:26     ` David Gibson
2021-06-03 12:46       ` Jason Gunthorpe
2021-06-04  6:27         ` Tian, Kevin
2021-06-03  7:17   ` Tian, Kevin
2021-06-03 12:49     ` Jason Gunthorpe
2021-06-08  5:49     ` David Gibson
2021-06-03  8:12   ` Tian, Kevin
2021-06-17  4:07     ` David Gibson
2021-06-23  8:00       ` Tian, Kevin
2021-06-24  3:55         ` David Gibson
2021-06-02  8:56 ` Enrico Weigelt, metux IT consult
2021-06-02 17:24   ` Jason Gunthorpe
2021-06-04 10:44     ` Enrico Weigelt, metux IT consult
2021-06-04 12:30       ` Jason Gunthorpe
2021-06-08  1:15         ` David Gibson
2021-06-08 10:43         ` Enrico Weigelt, metux IT consult
2021-06-08 13:11           ` Jason Gunthorpe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210602130053.615db578.alex.williamson@redhat.com \
    --to=alex.williamson@redhat.com \
    --cc=ashok.raj@intel.com \
    --cc=corbet@lwn.net \
    --cc=dave.jiang@intel.com \
    --cc=david@gibson.dropbear.id.au \
    --cc=dwmw2@infradead.org \
    --cc=iommu@lists.linux-foundation.org \
    --cc=jasowang@redhat.com \
    --cc=jean-philippe@linaro.org \
    --cc=jgg@nvidia.com \
    --cc=kevin.tian@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=kwankhede@nvidia.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=robin.murphy@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).