From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-x234.google.com (mail-oi0-x234.google.com [IPv6:2607:f8b0:4003:c06::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 08F6821FC747A for ; Fri, 6 Oct 2017 16:11:44 -0700 (PDT) Received: by mail-oi0-x234.google.com with SMTP id v132so12552841oie.1 for ; Fri, 06 Oct 2017 16:15:09 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <1507331434.29211.439.camel@infradead.org> References: <150732931273.22363.8436792888326501071.stgit@dwillia2-desk3.amr.corp.intel.com> <150732935473.22363.1853399637339625023.stgit@dwillia2-desk3.amr.corp.intel.com> <1507329939.29211.434.camel@infradead.org> <1507331434.29211.439.camel@infradead.org> From: Dan Williams Date: Fri, 6 Oct 2017 16:15:08 -0700 Message-ID: Subject: Re: [PATCH v7 07/12] dma-mapping: introduce dma_has_iommu() List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: David Woodhouse Cc: Jan Kara , Ashok Raj , "Darrick J. Wong" , linux-rdma@vger.kernel.org, Greg Kroah-Hartman , Joerg Roedel , "linux-nvdimm@lists.01.org" , Dave Chinner , linux-xfs@vger.kernel.org, Linux MM , Linux API , linux-fsdevel , Robin Murphy , Christoph Hellwig , Marek Szyprowski List-ID: T24gRnJpLCBPY3QgNiwgMjAxNyBhdCA0OjEwIFBNLCBEYXZpZCBXb29kaG91c2UgPGR3bXcyQGlu ZnJhZGVhZC5vcmc+IHdyb3RlOgo+IE9uIEZyaSwgMjAxNy0xMC0wNiBhdCAxNTo1MiAtMDcwMCwg RGFuIFdpbGxpYW1zIHdyb3RlOgo+PiBPbiBGcmksIE9jdCA2LCAyMDE3IGF0IDM6NDUgUE0sIERh dmlkIFdvb2Rob3VzZSA8ZHdtdzJAaW5mcmFkZWFkLm9yZz4gd3JvdGU6Cj4+ID4KPj4gPiBPbiBG cmksIDIwMTctMTAtMDYgYXQgMTU6MzUgLTA3MDAsIERhbiBXaWxsaWFtcyB3cm90ZToKPj4gPiA+ Cj4+ID4gPiBBZGQgYSBoZWxwZXIgdG8gZGV0ZXJtaW5lIGlmIHRoZSBkbWEgbWFwcGluZ3Mgc2V0 IHVwIGZvciBhIGdpdmVuIGRldmljZQo+PiA+ID4gYXJlIGJhY2tlZCBieSBhbiBpb21tdS4gSW4g cGFydGljdWxhciwgdGhpcyBsZXRzIGNvZGUgcGF0aHMga25vdyB0aGF0IGEKPj4gPiA+IGRtYV91 bm1hcCBvcGVyYXRpb24gd2lsbCByZXZva2UgYWNjZXNzIHRvIG1lbW9yeSBpZiB0aGUgZGV2aWNl IGNhbiBub3QKPj4gPiA+IG90aGVyd2lzZSBiZSBxdWllc2NlZC4gVGhlIG5lZWQgZm9yIHRoaXMg a25vd2xlZGdlIGlzIGRyaXZlbiBieSBhIG5lZWQKPj4gPiA+IHRvIG1ha2UgUkRNQSB0cmFuc2Zl cnMgdG8gREFYIG1hcHBpbmdzIHNhZmUuIElmIHRoZSBEQVggZmlsZSdzIGJsb2NrIG1hcAo+PiA+ ID4gY2hhbmdlcyB3ZSBuZWVkIHRvIGJlIHRvIHJlbGlhYmx5IHN0b3AgYWNjZXNzZXMgdG8gYmxv Y2tzIHRoYXQgaGF2ZSBiZWVuCj4+ID4gPiBmcmVlZCBvciByZS1hc3NpZ25lZCB0byBhIG5ldyBm aWxlLgo+PiA+ICJhIGRtYV91bm1hcCBvcGVyYXRpb24gcmV2b2tlIGFjY2VzcyB0byBtZW1vcnki Li4uIGJ1dCBpdCdzIE9LIHRoYXQgdGhlCj4+ID4gbmV4dCAqbWFwKiB3aWxsIGdpdmUgdGhlIHNh bWUgRE1BIGFkZHJlc3MgdG8gc29tZW9uZSBlbHNlLCByaWdodD8KPj4KPj4gSSdtIGFzc3VtaW5n IHRoZSBuZXh0IG1hcCB3aWxsIGJlIHRvIG90aGVyIHBoeXNpY2FsIGFkZHJlc3NlcyBhbmQgYQo+ PiBkaWZmZXJlbnQgcmVxdWVzdGVyIGRldmljZSBzaW5jZSB0aGUgbWVtb3J5IGlzIHN0aWxsIHJl Z2lzdGVyZWQKPj4gZXhjbHVzaXZlbHkuCj4KPiBJIG1lYW50IHRoZSBuZXh0IG1hcCBmb3IgdGhp cyBkZXZpY2UvZ3JvdXAuCj4KPiBJdCBtYXkgd2VsbCB1c2UgdGhlIHNhbWUgdmlydHVhbCBETUEg YWRkcmVzcyBhcyB0aGUgb25lIHlvdSBqdXN0Cj4gdW5tYXBwZWQsIHlldCBhY3R1YWxseSBtYXAg dG8gYSBkaWZmZXJlbnQgcGh5c2ljYWwgYWRkcmVzcy4gU28gaWYgdGhlCj4gRE1BIHN0aWxsIG9j Y3VycyB0byB0aGUgIm9sZCIgYWRkcmVzcywgdGhhdCBpc24ndCByZXZva2VkIGF0IGFsbCDigJQg aXQncwo+IGp1c3QgZ29pbmcgdG8gdGhlIHdyb25nIHBoeXNpY2FsIGxvY2F0aW9uLgo+Cj4gQW5k IGlmIHlvdSBhcmUgc3VyZSB0aGF0IHRoZSBETUEgd2lsbCBuZXZlciBoYXBwZW4sIHdoeSBkbyB5 b3UgbmVlZCB0bwo+IHJldm9rZSB0aGUgbWFwcGluZyBpbiB0aGUgZmlyc3QgcGxhY2U/CgpSaWdo dCwgY3Jvc3NlZCBtYWlscy4gVGhlIHNlbWFudGljIEkgd2FudCBpcyB0aGF0IHRoZSBJT1ZBIGlz CmludmFsaWRhdGVkIC8gc3RhcnRzIHRocm93aW5nIGVycm9ycyB0byB0aGUgZGV2aWNlIGJlY2F1 c2UgdGhlIGFkZHJlc3MKaXQgdGhvdWdodCBpdCB3YXMgdGFsa2luZyB0byBoYXMgYmVlbiByZW1h cHBlZCBpbiB0aGUgZmlsZS4gT25jZQp1c2Vyc3BhY2Ugd2FrZXMgdXAgYW5kIHJlc3BvbmRzIHRv IHRoaXMgaW52YWxpZGF0aW9uIGV2ZW50IGl0IGNhbiBkbwp0aGUgYWN0dWFsIHVubWFwIHRvIG1h a2UgdGhlIElPVkEgcmV1c2FibGUgYWdhaW4uCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fCkxpbnV4LW52ZGltbSBtYWlsaW5nIGxpc3QKTGludXgtbnZkaW1t QGxpc3RzLjAxLm9yZwpodHRwczovL2xpc3RzLjAxLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4 LW52ZGltbQo= From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dan Williams Subject: Re: [PATCH v7 07/12] dma-mapping: introduce dma_has_iommu() Date: Fri, 6 Oct 2017 16:15:08 -0700 Message-ID: References: <150732931273.22363.8436792888326501071.stgit@dwillia2-desk3.amr.corp.intel.com> <150732935473.22363.1853399637339625023.stgit@dwillia2-desk3.amr.corp.intel.com> <1507329939.29211.434.camel@infradead.org> <1507331434.29211.439.camel@infradead.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <1507331434.29211.439.camel-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linux-nvdimm-bounces-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org Sender: "Linux-nvdimm" To: David Woodhouse Cc: Jan Kara , Ashok Raj , "Darrick J. Wong" , linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Greg Kroah-Hartman , Joerg Roedel , "linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org" , Dave Chinner , linux-xfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Linux MM , Linux API , linux-fsdevel , Robin Murphy , Christoph Hellwig , Marek Szyprowski List-Id: linux-rdma@vger.kernel.org T24gRnJpLCBPY3QgNiwgMjAxNyBhdCA0OjEwIFBNLCBEYXZpZCBXb29kaG91c2UgPGR3bXcyQGlu ZnJhZGVhZC5vcmc+IHdyb3RlOgo+IE9uIEZyaSwgMjAxNy0xMC0wNiBhdCAxNTo1MiAtMDcwMCwg RGFuIFdpbGxpYW1zIHdyb3RlOgo+PiBPbiBGcmksIE9jdCA2LCAyMDE3IGF0IDM6NDUgUE0sIERh dmlkIFdvb2Rob3VzZSA8ZHdtdzJAaW5mcmFkZWFkLm9yZz4gd3JvdGU6Cj4+ID4KPj4gPiBPbiBG cmksIDIwMTctMTAtMDYgYXQgMTU6MzUgLTA3MDAsIERhbiBXaWxsaWFtcyB3cm90ZToKPj4gPiA+ Cj4+ID4gPiBBZGQgYSBoZWxwZXIgdG8gZGV0ZXJtaW5lIGlmIHRoZSBkbWEgbWFwcGluZ3Mgc2V0 IHVwIGZvciBhIGdpdmVuIGRldmljZQo+PiA+ID4gYXJlIGJhY2tlZCBieSBhbiBpb21tdS4gSW4g cGFydGljdWxhciwgdGhpcyBsZXRzIGNvZGUgcGF0aHMga25vdyB0aGF0IGEKPj4gPiA+IGRtYV91 bm1hcCBvcGVyYXRpb24gd2lsbCByZXZva2UgYWNjZXNzIHRvIG1lbW9yeSBpZiB0aGUgZGV2aWNl IGNhbiBub3QKPj4gPiA+IG90aGVyd2lzZSBiZSBxdWllc2NlZC4gVGhlIG5lZWQgZm9yIHRoaXMg a25vd2xlZGdlIGlzIGRyaXZlbiBieSBhIG5lZWQKPj4gPiA+IHRvIG1ha2UgUkRNQSB0cmFuc2Zl cnMgdG8gREFYIG1hcHBpbmdzIHNhZmUuIElmIHRoZSBEQVggZmlsZSdzIGJsb2NrIG1hcAo+PiA+ ID4gY2hhbmdlcyB3ZSBuZWVkIHRvIGJlIHRvIHJlbGlhYmx5IHN0b3AgYWNjZXNzZXMgdG8gYmxv Y2tzIHRoYXQgaGF2ZSBiZWVuCj4+ID4gPiBmcmVlZCBvciByZS1hc3NpZ25lZCB0byBhIG5ldyBm aWxlLgo+PiA+ICJhIGRtYV91bm1hcCBvcGVyYXRpb24gcmV2b2tlIGFjY2VzcyB0byBtZW1vcnki Li4uIGJ1dCBpdCdzIE9LIHRoYXQgdGhlCj4+ID4gbmV4dCAqbWFwKiB3aWxsIGdpdmUgdGhlIHNh bWUgRE1BIGFkZHJlc3MgdG8gc29tZW9uZSBlbHNlLCByaWdodD8KPj4KPj4gSSdtIGFzc3VtaW5n IHRoZSBuZXh0IG1hcCB3aWxsIGJlIHRvIG90aGVyIHBoeXNpY2FsIGFkZHJlc3NlcyBhbmQgYQo+ PiBkaWZmZXJlbnQgcmVxdWVzdGVyIGRldmljZSBzaW5jZSB0aGUgbWVtb3J5IGlzIHN0aWxsIHJl Z2lzdGVyZWQKPj4gZXhjbHVzaXZlbHkuCj4KPiBJIG1lYW50IHRoZSBuZXh0IG1hcCBmb3IgdGhp cyBkZXZpY2UvZ3JvdXAuCj4KPiBJdCBtYXkgd2VsbCB1c2UgdGhlIHNhbWUgdmlydHVhbCBETUEg YWRkcmVzcyBhcyB0aGUgb25lIHlvdSBqdXN0Cj4gdW5tYXBwZWQsIHlldCBhY3R1YWxseSBtYXAg dG8gYSBkaWZmZXJlbnQgcGh5c2ljYWwgYWRkcmVzcy4gU28gaWYgdGhlCj4gRE1BIHN0aWxsIG9j Y3VycyB0byB0aGUgIm9sZCIgYWRkcmVzcywgdGhhdCBpc24ndCByZXZva2VkIGF0IGFsbCDigJQg aXQncwo+IGp1c3QgZ29pbmcgdG8gdGhlIHdyb25nIHBoeXNpY2FsIGxvY2F0aW9uLgo+Cj4gQW5k IGlmIHlvdSBhcmUgc3VyZSB0aGF0IHRoZSBETUEgd2lsbCBuZXZlciBoYXBwZW4sIHdoeSBkbyB5 b3UgbmVlZCB0bwo+IHJldm9rZSB0aGUgbWFwcGluZyBpbiB0aGUgZmlyc3QgcGxhY2U/CgpSaWdo dCwgY3Jvc3NlZCBtYWlscy4gVGhlIHNlbWFudGljIEkgd2FudCBpcyB0aGF0IHRoZSBJT1ZBIGlz CmludmFsaWRhdGVkIC8gc3RhcnRzIHRocm93aW5nIGVycm9ycyB0byB0aGUgZGV2aWNlIGJlY2F1 c2UgdGhlIGFkZHJlc3MKaXQgdGhvdWdodCBpdCB3YXMgdGFsa2luZyB0byBoYXMgYmVlbiByZW1h cHBlZCBpbiB0aGUgZmlsZS4gT25jZQp1c2Vyc3BhY2Ugd2FrZXMgdXAgYW5kIHJlc3BvbmRzIHRv IHRoaXMgaW52YWxpZGF0aW9uIGV2ZW50IGl0IGNhbiBkbwp0aGUgYWN0dWFsIHVubWFwIHRvIG1h a2UgdGhlIElPVkEgcmV1c2FibGUgYWdhaW4uCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fCkxpbnV4LW52ZGltbSBtYWlsaW5nIGxpc3QKTGludXgtbnZkaW1t QGxpc3RzLjAxLm9yZwpodHRwczovL2xpc3RzLjAxLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4 LW52ZGltbQo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: MIME-Version: 1.0 In-Reply-To: <1507331434.29211.439.camel@infradead.org> References: <150732931273.22363.8436792888326501071.stgit@dwillia2-desk3.amr.corp.intel.com> <150732935473.22363.1853399637339625023.stgit@dwillia2-desk3.amr.corp.intel.com> <1507329939.29211.434.camel@infradead.org> <1507331434.29211.439.camel@infradead.org> From: Dan Williams Date: Fri, 6 Oct 2017 16:15:08 -0700 Message-ID: Subject: Re: [PATCH v7 07/12] dma-mapping: introduce dma_has_iommu() To: David Woodhouse Cc: "linux-nvdimm@lists.01.org" , Jan Kara , Ashok Raj , "Darrick J. Wong" , linux-rdma@vger.kernel.org, Greg Kroah-Hartman , Joerg Roedel , Dave Chinner , linux-xfs@vger.kernel.org, Linux MM , Jeff Moyer , Linux API , linux-fsdevel , Ross Zwisler , Robin Murphy , Christoph Hellwig , Marek Szyprowski Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sender: owner-linux-mm@kvack.org List-ID: On Fri, Oct 6, 2017 at 4:10 PM, David Woodhouse wrote= : > On Fri, 2017-10-06 at 15:52 -0700, Dan Williams wrote: >> On Fri, Oct 6, 2017 at 3:45 PM, David Woodhouse wr= ote: >> > >> > On Fri, 2017-10-06 at 15:35 -0700, Dan Williams wrote: >> > > >> > > Add a helper to determine if the dma mappings set up for a given dev= ice >> > > are backed by an iommu. In particular, this lets code paths know tha= t a >> > > dma_unmap operation will revoke access to memory if the device can n= ot >> > > otherwise be quiesced. The need for this knowledge is driven by a ne= ed >> > > to make RDMA transfers to DAX mappings safe. If the DAX file's block= map >> > > changes we need to be to reliably stop accesses to blocks that have = been >> > > freed or re-assigned to a new file. >> > "a dma_unmap operation revoke access to memory"... but it's OK that th= e >> > next *map* will give the same DMA address to someone else, right? >> >> I'm assuming the next map will be to other physical addresses and a >> different requester device since the memory is still registered >> exclusively. > > I meant the next map for this device/group. > > It may well use the same virtual DMA address as the one you just > unmapped, yet actually map to a different physical address. So if the > DMA still occurs to the "old" address, that isn't revoked at all =E2=80= =94 it's > just going to the wrong physical location. > > And if you are sure that the DMA will never happen, why do you need to > revoke the mapping in the first place? Right, crossed mails. The semantic I want is that the IOVA is invalidated / starts throwing errors to the device because the address it thought it was talking to has been remapped in the file. Once userspace wakes up and responds to this invalidation event it can do the actual unmap to make the IOVA reusable again. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f53.google.com ([209.85.218.53]:55569 "EHLO mail-oi0-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751485AbdJFXPJ (ORCPT ); Fri, 6 Oct 2017 19:15:09 -0400 Received: by mail-oi0-f53.google.com with SMTP id g125so14432389oib.12 for ; Fri, 06 Oct 2017 16:15:08 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <1507331434.29211.439.camel@infradead.org> References: <150732931273.22363.8436792888326501071.stgit@dwillia2-desk3.amr.corp.intel.com> <150732935473.22363.1853399637339625023.stgit@dwillia2-desk3.amr.corp.intel.com> <1507329939.29211.434.camel@infradead.org> <1507331434.29211.439.camel@infradead.org> From: Dan Williams Date: Fri, 6 Oct 2017 16:15:08 -0700 Message-ID: Subject: Re: [PATCH v7 07/12] dma-mapping: introduce dma_has_iommu() Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sender: linux-xfs-owner@vger.kernel.org List-ID: List-Id: xfs To: David Woodhouse Cc: "linux-nvdimm@lists.01.org" , Jan Kara , Ashok Raj , "Darrick J. Wong" , linux-rdma@vger.kernel.org, Greg Kroah-Hartman , Joerg Roedel , Dave Chinner , linux-xfs@vger.kernel.org, Linux MM , Jeff Moyer , Linux API , linux-fsdevel , Ross Zwisler , Robin Murphy , Christoph Hellwig , Marek Szyprowski On Fri, Oct 6, 2017 at 4:10 PM, David Woodhouse wrote= : > On Fri, 2017-10-06 at 15:52 -0700, Dan Williams wrote: >> On Fri, Oct 6, 2017 at 3:45 PM, David Woodhouse wr= ote: >> > >> > On Fri, 2017-10-06 at 15:35 -0700, Dan Williams wrote: >> > > >> > > Add a helper to determine if the dma mappings set up for a given dev= ice >> > > are backed by an iommu. In particular, this lets code paths know tha= t a >> > > dma_unmap operation will revoke access to memory if the device can n= ot >> > > otherwise be quiesced. The need for this knowledge is driven by a ne= ed >> > > to make RDMA transfers to DAX mappings safe. If the DAX file's block= map >> > > changes we need to be to reliably stop accesses to blocks that have = been >> > > freed or re-assigned to a new file. >> > "a dma_unmap operation revoke access to memory"... but it's OK that th= e >> > next *map* will give the same DMA address to someone else, right? >> >> I'm assuming the next map will be to other physical addresses and a >> different requester device since the memory is still registered >> exclusively. > > I meant the next map for this device/group. > > It may well use the same virtual DMA address as the one you just > unmapped, yet actually map to a different physical address. So if the > DMA still occurs to the "old" address, that isn't revoked at all =E2=80= =94 it's > just going to the wrong physical location. > > And if you are sure that the DMA will never happen, why do you need to > revoke the mapping in the first place? Right, crossed mails. The semantic I want is that the IOVA is invalidated / starts throwing errors to the device because the address it thought it was talking to has been remapped in the file. Once userspace wakes up and responds to this invalidation event it can do the actual unmap to make the IOVA reusable again.