From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jordan Crouse Subject: Re: [RFC PATCH v1 0/6] Resolve unwanted DMA backing with IOMMU Date: Fri, 3 Aug 2018 11:00:26 -0600 Message-ID: <20180803170026.GD21283@jcrouse-lnx.qualcomm.com> References: <20180726231624.21084-1-digetx@gmail.com> <20180727170326.GA21283@jcrouse-lnx.qualcomm.com> <2402373.RyBdR7VSnO@dimapc> <2887450.sPhIOOMKZK@dimapc> <2e7fab6e-0640-8f48-07b8-2d475538b8ae@arm.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: <2e7fab6e-0640-8f48-07b8-2d475538b8ae@arm.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Robin Murphy Cc: devicetree@vger.kernel.org, Mikko Perttunen , Frank Rowand , "Rafael J. Wysocki" , Nicolas Chauvet , nouveau@lists.freedesktop.org, Will Deacon , Russell King , dri-devel@lists.freedesktop.org, Jonathan Hunter , Rob Herring , iommu@lists.linux-foundation.org, Thierry Reding , Ben Skeggs , Greg Kroah-Hartman , linux-tegra@vger.kernel.org, Dmitry Osipenko , Catalin Marinas , linux-kernel@vger.kernel.org List-Id: linux-tegra@vger.kernel.org T24gRnJpLCBBdWcgMDMsIDIwMTggYXQgMDQ6NDM6NDFQTSArMDEwMCwgUm9iaW4gTXVycGh5IHdy b3RlOgo+IE9uIDAyLzA4LzE4IDE5OjI0LCBEbWl0cnkgT3NpcGVua28gd3JvdGU6Cj4gPk9uIEZy aWRheSwgMjcgSnVseSAyMDE4IDIwOjE2OjUzIE1TSyBEbWl0cnkgT3NpcGVua28gd3JvdGU6Cj4g Pj5PbiBGcmlkYXksIDI3IEp1bHkgMjAxOCAyMDowMzoyNiBNU0sgSm9yZGFuIENyb3VzZSB3cm90 ZToKPiA+Pj5PbiBGcmksIEp1bCAyNywgMjAxOCBhdCAwNTowMjozN1BNICswMTAwLCBSb2JpbiBN dXJwaHkgd3JvdGU6Cj4gPj4+Pk9uIDI3LzA3LzE4IDE1OjEwLCBEbWl0cnkgT3NpcGVua28gd3Jv dGU6Cj4gPj4+Pj5PbiBGcmlkYXksIDI3IEp1bHkgMjAxOCAxMjowMzoyOCBNU0sgV2lsbCBEZWFj b24gd3JvdGU6Cj4gPj4+Pj4+T24gRnJpLCBKdWwgMjcsIDIwMTggYXQgMTA6MjU6MTNBTSArMDIw MCwgSm9lcmcgUm9lZGVsIHdyb3RlOgo+ID4+Pj4+Pj5PbiBGcmksIEp1bCAyNywgMjAxOCBhdCAw MjoxNjoxOEFNICswMzAwLCBEbWl0cnkgT3NpcGVua28gd3JvdGU6Cj4gPj4+Pj4+Pj5UaGUgcHJv cG9zZWQgc29sdXRpb24gYWRkcyBhIG5ldyBvcHRpb24gdG8gdGhlIGJhc2UgZGV2aWNlIGRyaXZl cgo+ID4+Pj4+Pj4+c3RydWN0dXJlIHRoYXQgYWxsb3dzIGRldmljZSBkcml2ZXJzIHRvIGV4cGxp Y2l0bHkgY29udmV5IHRvIHRoZQo+ID4+Pj4+Pj4+ZHJpdmVycwo+ID4+Pj4+Pj4+Y29yZSB0aGF0 IHRoZSBpbXBsaWNpdCBJT01NVSBiYWNraW5nIGZvciBkZXZpY2VzIG11c3Qgbm90IGhhcHBlbi4K PiA+Pj4+Pj4+Cj4gPj4+Pj4+PldoeSBpcyBJT01NVSBtYXBwaW5nIGEgcHJvYmxlbSBmb3IgdGhl IFRlZ3JhIEdQVSBkcml2ZXI/Cj4gPj4+Pj4+Pgo+ID4+Pj4+Pj5JZiB3ZSBhZGQgc29tZXRoaW5n IGxpa2UgdGhpcyB0aGVuIGl0IHNob3VsZCBub3QgYmUgdGhlIGNob2ljZSBvZiB0aGUKPiA+Pj4+ Pj4+ZGV2aWNlIGRyaXZlciwgYnV0IG9mIHRoZSB1c2VyIGFuZC9vciB0aGUgZmlybXdhcmUuCj4g Pj4+Pj4+Cj4gPj4+Pj4+QWdyZWVkLCBhbmQgaXQgd291bGQgc3RpbGwgbmVlZCBzb21lYm9keSB0 byBjb25maWd1cmUgYW4gaWRlbnRpdHkKPiA+Pj4+Pj5kb21haW4KPiA+Pj4+Pj5zbwo+ID4+Pj4+ PnRoYXQgdHJhbnNhY3Rpb25zIGFyZW4ndCBhYm9ydGVkIGltbWVkaWF0ZWx5LiBXZSBjdXJyZW50 bHkgYWxsb3cgdGhlCj4gPj4+Pj4+aWRlbnRpdHkgZG9tYWluIHRvIGJlIHVzZWQgYnkgZGVmYXVs dCB2aWEgYSBjb21tYW5kLWxpbmUgb3B0aW9uLCBzbyBJCj4gPj4+Pj4+Z3Vlc3MKPiA+Pj4+Pj53 ZSdkIG5lZWQgYSB3YXkgZm9yIGZpcm13YXJlIHRvIHJlcXVlc3QgdGhhdCBvbiBhIHBlci1kZXZp Y2UgYmFzaXMuCj4gPj4+Pj4KPiA+Pj4+PlRoZSBJT01NVSBtYXBwaW5nIGl0c2VsZiBpcyBub3Qg YSBwcm9ibGVtLCB0aGUgcHJvYmxlbSBpcyB0aGUKPiA+Pj4+Pm1hbmFnZW1lbnQKPiA+Pj4+Pm9m Cj4gPj4+Pj50aGUgSU9NTVUuIEZvciBUZWdyYSB3ZSBkb24ndCB3YW50IGFueXRoaW5nIHRvIGlu dHJ1ZGUgaW50byB0aGUgSU9NTVUKPiA+Pj4+PmFjdGl2aXRpZXMgYmVjYXVzZToKPiA+Pj4+Pgo+ ID4+Pj4+MSkgR1BVIEhXIHJlcXVpcmUgYWRkaXRpb25hbCBjb25maWd1cmF0aW9uIGZvciB0aGUg SU9NTVUgdXNhZ2UgYW5kIGR1bWIKPiA+Pj4+Pm1hcHBpbmcgb2YgdGhlIGFsbG9jYXRpb25zIHNp bXBseSBkb2Vzbid0IHdvcmsuCj4gPj4+Pgo+ID4+Pj5HZW5lcmFsbHksIHRoYXQncyBhbHJlYWR5 IGhhbmRsZWQgYnkgdGhlIERSTSBkcml2ZXJzIGFsbG9jYXRpbmcKPiA+Pj4+dGhlaXIgb3duIHVu bWFuYWdlZCBkb21haW5zLiBUaGUgb25seSBwcm9ibGVtIHdlIHJlYWxseSBuZWVkIHRvCj4gPj4+ PnNvbHZlIGluIHRoYXQgcmVnYXJkIGlzIHRoYXQgY3VycmVudGx5IHRoZSBkZXZpY2UgRE1BIG9w cyBkb24ndCBnZXQKPiA+Pj4+dXBkYXRlZCB3aGVuIG1vdmluZyBhd2F5IGZyb20gdGhlIG1hbmFn ZWQgZG9tYWluLiBUaGF0J3MgYmVlbiBPSyBmb3IKPiA+Pj4+dGhlIFZGSU8gY2FzZSB3aGVyZSB0 aGUgZGV2aWNlIGlzIGJvdW5kIHRvIGEgZGlmZmVyZW50IGRyaXZlciB3aGljaAo+ID4+Pj53ZSBr bm93IHdvbid0IG1ha2UgYW55IGV4cGxpY2l0IERNQSBBUEkgY2FsbHMsIGJ1dCBmb3IgdGhlIG1v cmUKPiA+Pj4+Z2VuZXJhbCBjYXNlIG9mIElPTU1VLWF3YXJlIGRyaXZlcnMgd2UgY291bGQgY2Vy dGFpbmx5IGRvIHdpdGggYSBiaXQKPiA+Pj4+b2YgY29vcGVyYXRpb24gYmV0d2VlbiB0aGUgSU9N TVUgQVBJLCBETUEgQVBJLCBhbmQgYXJjaCBjb2RlIHRvCj4gPj4+PnVwZGF0ZSB0aGUgRE1BIG9w cyBkeW5hbWljYWxseSB0byBjb3BlIHdpdGggaW50ZXJtZWRpYXRlIHN1YnN5c3RlbXMKPiA+Pj4+ bWFraW5nIERNQSBBUEkgY2FsbHMgb24gYmVoYWxmIG9mIGRldmljZXMgdGhleSBkb24ndCBrbm93 IHRoZQo+ID4+Pj5pbnRpbWF0ZSBkZXRhaWxzIG9mLgo+ID4+Pj4KPiA+Pj4+PjIpIE9sZGVyIFRl Z3JhIGdlbmVyYXRpb25zIGhhdmUgYSBsaW1pdGVkIHJlc291cmNlIGFuZCBjYXBhYmlsaXRpZXMg aW4KPiA+Pj4+PnJlZ2FyZHMgdG8gSU9NTVUgdXNhZ2UsIGFsbG9jYXRpbmcgSU9NTVUgZG9tYWlu IHBlci1kZXZpY2UgaXMganVzdAo+ID4+Pj4+aW1wb3NzaWJsZSBmb3IgZXhhbXBsZS4KPiA+Pj4+ Pgo+ID4+Pj4+MykgSFcgcGVyZm9ybXMgY29udGV4dCBzd2l0Y2hlcyBhbmQgc28gcGFydGljdWxh ciBhbGxvY2F0aW9ucyBoYXZlIHRvCj4gPj4+Pj5iZQo+ID4+Pj4+YXNzaWduZWQgdG8gYSBwYXJ0 aWN1bGFyIGNvbnRleHRzIElPTU1VIGRvbWFpbi4KPiA+Pj4+Cj4gPj4+PkkgdW5kZXJzdGFuZCBR dWFsY29tbSBTb0NzIGhhdmUgYSBzaW1pbGFyIHRoaW5nIHRvbywgYW5kIEFGQUlDUyB0aGF0Cj4g Pj4+PmNhc2UganVzdCBkb2Vzbid0IGZpdCBpbnRvIHRoZSBjdXJyZW50IEFQSSBtb2RlbCBhdCBh bGwuIFdlIG5lZWQgdGhlCj4gPj4+PklPTU1VIGRyaXZlciB0byBzb21laG93IGtub3cgYWJvdXQg dGhlIHNwZWNpZmljIGRldGFpbHMgb2Ygd2hpY2gKPiA+Pj4+ZGV2aWNlcyBoYXZlIG1hZ2ljIGFz c29jaWF0aW9ucyB3aXRoIHNwZWNpZmljIGNvbnRleHRzLCBhbmQgd2UKPiA+Pj4+YWxtb3N0IGNl cnRhaW5seSBuZWVkIGEgbW9yZSBleHByZXNzaXZlIGludGVyZmFjZSB0aGFuCj4gPj4+PmlvbW11 X2RvbWFpbl9hbGxvYygpIHRvIGhhdmUgYW55IGhvcGUgb2YgcmVsaWFibGUgcmVzdWx0cy4KPiA+ Pj4KPiA+Pj5UaGlzIGlzIGNvcnJlY3QgZm9yIFF1YWxjb21tIEdQVXMgLSBUaGUgR1BVIGhhcmR3 YXJlIGNvbnRleHQgc3dpdGNoaW5nCj4gPj4+cmVxdWlyZXMgYSBzcGVjaWZpYyBjb250ZXh0IGFu ZCB0aGVyZSBhcmUgc29tZSByZXN0cmljdGlvbnMgYXJvdW5kCj4gPj4+c2VjdXJlIGNvbnRleHRz IGFzIHdlbGwuCj4gPj4+Cj4gPj4+V2UgZG9uJ3QgcmVhbGx5IGNhcmUgaWYgdGhlIERNQSBhdHRh Y2hlcyB0byBhIGNvbnRleHQganVzdCBhcyBsb25nIGFzIGl0Cj4gPj4+ZG9lc24ndCBhdHRhY2gg dG8gdGhlIG9uZShzKSB3ZSBjYXJlIGFib3V0LiBQZXJoYXBzIGEgInZhbGlkIGNvbnRleHQiIG1h c2sKPiA+Pj53b3VsZCB3b3JrIGluIGZyb20gdGhlIERUIG9yIHRoZSBkZXZpY2Ugc3RydWN0IHRv IGdpdmUgdGhlIHN1YnN5c3RlbXMgYQo+ID4+PmNsdWUgYXMgdG8gd2hpY2ggZG9tYWlucyB0aGV5 IHdlcmUgYWxsb3dlZCB0byB1c2UuIEkgcmVjb2duaXplIHRoYXQgdGhlcmUKPiA+Pj5pc24ndCBh IG9uZS1zaXplLWZpdHMtYWxsIHNvbHV0aW9uIHRvIHRoaXMgcHJvYmxlbSBzbyBJJ20gb3BlbiB0 bwo+ID4+PmRpZmZlcmVudAo+ID4+PmlkZWFzLgo+ID4+Cj4gPj5EZXNpZ25hdGluZyB3aGV0aGVy IGltcGxpY2l0IElPTU1VIGJhY2tpbmcgaXMgYXBwcm9wcmlhdGUgZm9yIGEgZGV2aWNlIHZpYQo+ ID4+ZGV2aWNlLXRyZWUgcHJvcGVydHkgc291bmRzIGEgYml0IGF3a3dhcmQgYmVjYXVzZSB0aGF0 IHdpbGwgYmUgYSBraW5kYQo+ID4+c29mdHdhcmUgZGVzY3JpcHRpb24gKG9mIGEgY3VzdG9tIExp bnV4IGRyaXZlciBtb2RlbCksIHdoaWxlIGRldmljZS10cmVlIGlzCj4gPj5zdXBwb3NlZCB0byBk ZXNjcmliZSBIVy4KPiA+Pgo+ID4+V2hhdCBhYm91dCB0byBncmFudCBJT01NVSBkcml2ZXJzIHdp dGggYWJpbGl0eSB0byBkZWNpZGUgd2hldGhlciB0aGUKPiA+PmltcGxpY2l0IGJhY2tpbmcgZm9y IGEgZGV2aWNlIGlzIGFwcHJvcHJpYXRlPyBMaWtlIHRoaXM6Cj4gPj4KPiA+PmJvb2wgaW1wbGlj aXRfaW9tbXVfZm9yX2RtYV9pc19hbGxvd2VkKHN0cnVjdCBkZXZpY2UgKmRldikKPiA+PnsKPiA+ Pgljb25zdCBzdHJ1Y3QgaW9tbXVfb3BzICpvcHMgPSBkZXYtPmJ1cy0+aW9tbXVfb3BzOwo+ID4+ CXN0cnVjdCBpb21tdV9ncm91cCAqZ3JvdXA7Cj4gPj4KPiA+Pglncm91cCA9IGlvbW11X2dyb3Vw X2dldChkZXYpOwo+ID4+CWlmICghZ3JvdXApCj4gPj4JCXJldHVybiBOVUxMOwo+ID4+Cj4gPj4J aW9tbXVfZ3JvdXBfcHV0KGdyb3VwKTsKPiA+Pgo+ID4+CWlmICghb3BzLT5pbXBsaWNpdF9pb21t dV9mb3JfZG1hX2lzX2FsbG93ZWQpCj4gPj4JCXJldHVybiB0cnVlOwo+ID4+Cj4gPj4JcmV0dXJu IG9wcy0+aW1wbGljaXRfaW9tbXVfZm9yX2RtYV9pc19hbGxvd2VkKGRldik7Cj4gPj59Cj4gPj4K PiA+PlRoZW4gYXJjaF9zZXR1cF9kbWFfb3BzKCkgY291bGQgaGF2ZSBhIGNsdWUgd2hldGhlciBp bXBsaWNpdCBJT01NVSBiYWNraW5nCj4gPj5mb3IgYSBkZXZpY2UgaXMgYXBwcm9wcmlhdGUuCj4g Pgo+ID5HdXlzLCBkb2VzIGl0IHNvdW5kIGdvb2QgdG8geW91IG9yIG1heWJlIHlvdSBoYXZlIHNv bWV0aGluZyBlbHNlIG9uIHlvdXIgbWluZD8KPiA+RXZlbiBpZiBpdCdzIG5vdCBhbiBpZGVhbCBz b2x1dGlvbiwgaXQgZml4ZXMgdGhlIGltbWVkaWF0ZSBwcm9ibGVtIGFuZCBzaG91bGQKPiA+YmUg Z29vZCBlbm91Z2ggZm9yIHRoZSBzdGFydGVyLgo+IAo+IFRvIG1lIHRoYXQgbG9va3MgbGlrZSBh IHN0ZXAgaW9uIHRoZSB3cm9uZyBkaXJlY3Rpb24gdGhhdCB3b24ndCBoZWxwCj4gYXQgYWxsIGlu IGFjdHVhbGx5IGFkZHJlc3NpbmcgdGhlIHVuZGVybHlpbmcgaXNzdWVzLgo+IAo+IElmIHRoZSBH UFUgZHJpdmVyIHdhbnRzIHRvIGV4cGxpY2l0bHkgY29udHJvbCBJT01NVSBtYXBwaW5ncyBpbnN0 ZWFkCj4gb2YgcmVseWluZyBvbiB0aGUgSU9NTVVfRE9NQUlOX0RNQSBhYnN0cmFjdGlvbiwgdGhl biBpdCBzaG91bGQgdXNlCj4gaXRzIG93biB1bm1hbmFnZWQgZG9tYWluLiBBdCB0aGF0IHBvaW50 IGl0IHNob3VsZG4ndCBtYXR0ZXIgaWYgYSBETUEKPiBvcHMgZG9tYWluIHdhcyBhbGxvY2F0ZWQs IHNpbmNlIHRoZSBHUFUgZGV2aWNlIHdpbGwgbm8gbG9uZ2VyIGJlCj4gYXR0YWNoZWQgdG8gaXQu IFllcywgdGhlcmUgbWF5IGJlIHNvbWUgaW1wcm92ZW1lbnRzIHRvIG1ha2UgbGlrZQo+IGhhdmlu ZyB1bnVzZWQgZG9tYWlucyBub3QgY29uc3VtZSBoYXJkd2FyZSBjb250ZXh0cywgYnV0IHRoYXQn cwo+IGludGVybmFsIHRvIHRoZSByZWxldmFudCBJT01NVSBkcml2ZXJzLiBJZiBtb3ZpbmcgaW4g YW5kIG91dCBvZiBETUEKPiBvcHMgZG9tYWlucyBsZWF2ZXMgdGhlIGFjdHVhbCBkbWFfb3BzIGJy b2tlbiwgdGhhdCdzIGFscmVhZHkgYQo+IHByb2JsZW0gYmV0d2VlbiB0aGUgSU9NTVUgQVBJIGFu ZCB0aGUgYXJjaCBETUEgY29kZSBhcyBJJ3ZlCj4gbWVudGlvbmVkIGJlZm9yZS4KPiAKPiBGdXJ0 aGVybW9yZSwgZ2l2ZW4gd2hhdCB0aGUgZXhhbXBsZSBhYm92ZSBpcyB0cnlpbmcgdG8gZG8sCj4g YXJjaF9zZXR1cF9kbWFfb3BzKCkgaXMgd2F5IHRvbyBsYXRlIHRvIGRvIGl0IC0gdGhlIGRlZmF1 bHQgZG9tYWluCj4gd2FzIGFscmVhZHkgc2V0IHVwIGluIGlvbW11X2dyb3VwX2dldF9mb3JfZGV2 KCkgd2hlbiB0aGUgSU9NTVUKPiBkcml2ZXIgZmlyc3Qgc2F3IHRoYXQgZGV2aWNlLiBBbiAib3B0 LW91dCIgbWVjaGFuaXNtIHRoYXQgZG9lc24ndAo+IGFjdHVhbGx5IG9wdCBvdXQgYW5kIGp1c3Qg Ym9kZ2VzIGFyb3VuZCBiZWluZyBvcHRlZC1pbiBhZnRlciB0aGUKPiBmYWN0IGRvZXNuJ3Qgc3Ry aWtlIG1lIGFzIHNvbWV0aGluZyB3aGljaCBjYW4gZ3JvdyB0byBiZSByb2J1c3QgYW5kCj4gbWFp bnRhaW5hYmxlLgo+IAo+IEZvciB0aGUgY2FzZSB3aGVyZSAgYSBkZXZpY2UgaGFzIHNvbWUgc3Bl Y2lhbCBoYXJkd2FyZSByZWxhdGlvbnNoaXAKPiB3aXRoIGEgcGFydGljdWxhciBJT01NVSBjb250 ZXh0LCB0aGUgSU9NTVUgZHJpdmVyICpoYXMqIHRvIGJlCj4gY29tcGxldGVseSBhd2FyZSBvZiB0 aGF0LCBpLmUuIGl0IG5lZWRzIHRvIGJlIGRlc2NyaWJlZCBpbiBEVC9BQ1BJLAo+IGVpdGhlciB2 aWEgc29tZSBleHBsaWNpdCBiaW5kaW5nIG9yIGF0IGxlYXN0IGluZmVycmVkIGZyb20gc29tZQo+ IFNvQy9pbnN0YW5jZS1zcGVjaWZpYyBJT01NVSBjb21wYXRpYmxlLiBUaGVuIHRoZSBJT01NVSBk cml2ZXIgbmVlZHMKPiB0byBrbm93IHdoZW4gdGhlIGRyaXZlciBmb3IgdGhhdCBkZXZpY2UgaXMg cmVxdWVzdGluZyBpdHMgc3BlY2lhbAo+IGRvbWFpbiBzbyB0aGF0IGl0IHByb3ZpZGUgdGhlIGNv cnJlY3QgY29udGV4dCAoYW5kICpub3QqIGFsbG9jYXRlCj4gdGhhdCBjb250ZXh0IGZvciBvdGhl ciB1c2VzKS4gQW55dGhpbmcgd2hpY2gganVzdCByZWxpZXMgb24gdGhlCj4gb3JkZXIgaW4gd2hp Y2ggdGhpbmdzIGN1cnJlbnRseSBoYXBwZW4gdG8gYmUgYWxsb2NhdGVkIGlzIGZhciB0b28KPiBm cmFnaWxlIGxvbmctdGVybS4KClNwZWN1bGF0aW5nIHdoYXQgdGhpcyB3b3VsZCBsb29rIGxpa2Ug Zm9yIGFybS1zbW11LXYyLiBUaGlua2luZyB3ZQp3b3VsZCBhZGQgZWl0aGVyIGEgbWFzayBvciBh IGJpdCB0byB0aGUgcGVyLWRldmljZSBzbW11IGRhdGEgdG8gaWRlbnRpZnkKdGhlICJhdmFpbGFi bGUiIGNvbnRleHRzIGZvciBkbWEgKG9yIGFsdGVybmF0aXZlbHksIHRoZSAicmVzZXJ2ZWQiIGRv bWFpbnMKZm9yICB0aGUgZGV2aWNlKSBhbmQgdGhlbiBjaGFuZ2UgdXAgX2FybV9zbW11X2FsbG9j X2JpdG1hcCgpIHRvIGhhbmQgb3V0CnRoZSByaWdodCBudW1iZXJzIGFjY29yZGluZ2x5LgoKVGhl biB0aGF0IGxlYXZlcyB0aGUgaW50ZXJmYWNlIGZvciB0aGUgZGV2aWNlIHRvIHJlcXVlc3QgYSBz cGVjaWZpYyBjb250ZXh0Ci0gSSBndWVzcyBhIERPTUFJTl9BVFRSIHdvdWxkIHdvcmsgZm9yIHRo YXQgaW4gbGlldSBvZiBjaGFuZ2luZwppb21tdV9hbGxvY19kb21haW4oKSBhbmQgZnJpZW5kcy4K CldvdWxkIHRoaXMgbWF0Y2ggdXAgd2l0aCB5b3VyIHRoaW5raW5nPwoKSm9yZGFuCgotLSAKVGhl IFF1YWxjb21tIElubm92YXRpb24gQ2VudGVyLCBJbmMuIGlzIGEgbWVtYmVyIG9mIENvZGUgQXVy b3JhIEZvcnVtLAphIExpbnV4IEZvdW5kYXRpb24gQ29sbGFib3JhdGl2ZSBQcm9qZWN0Cl9fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmRyaS1kZXZlbCBtYWls aW5nIGxpc3QKZHJpLWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZwpodHRwczovL2xpc3RzLmZy ZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2RyaS1kZXZlbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS,T_DKIM_INVALID, USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48627C28CF6 for ; Fri, 3 Aug 2018 17:00:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BECDD2177A for ; Fri, 3 Aug 2018 17:00:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="key not found in DNS" (0-bit key) header.d=codeaurora.org header.i=@codeaurora.org header.b="huCXoHha"; dkim=fail reason="key not found in DNS" (0-bit key) header.d=codeaurora.org header.i=@codeaurora.org header.b="W63g19ri" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BECDD2177A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727843AbeHCS5l (ORCPT ); Fri, 3 Aug 2018 14:57:41 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:35174 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727171AbeHCS5l (ORCPT ); Fri, 3 Aug 2018 14:57:41 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 9858C602B7; Fri, 3 Aug 2018 17:00:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1533315632; bh=6RJFM3v39Wb2v7D5Smk+rPyw23m0556pRLdxNAyjMOk=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=huCXoHhamEjpqDDI8EIu/17Ut8eOpxntZW2mX0ijdUTIooQ1kMJrsAoSuuZR43AYW Gv4M/hSUUWC7f5d/dbgp7mfy428FFdKcljlM6/3zofhoXdBBbRX/IGUcTxL3PSYJN2 QdSTKWVfP8uRKxrkmwGanxwbPHv/ybcEXCOJZifs= Received: from jcrouse-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 1834F602B7; Fri, 3 Aug 2018 17:00:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1533315630; bh=6RJFM3v39Wb2v7D5Smk+rPyw23m0556pRLdxNAyjMOk=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=W63g19riGJWaLMisWL64cZQ6QEQnftWbcytvtXEimkfT7tDXXBfeosTmwAnVCE5pu K0pzJyO/6g2+pE2tZia0xO5Xt6pI5MNm6oNizIeQNvyZXWXlLcK6XHmQ6bIDELWO37 sQWl2r5qtzrgSq9kFOMxbfYs1U9vXDxpPCRhOe0k= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 1834F602B7 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org Date: Fri, 3 Aug 2018 11:00:26 -0600 From: Jordan Crouse To: Robin Murphy Cc: Dmitry Osipenko , Will Deacon , Joerg Roedel , Mikko Perttunen , Thierry Reding , devicetree@vger.kernel.org, nouveau@lists.freedesktop.org, "Rafael J. Wysocki" , Nicolas Chauvet , Greg Kroah-Hartman , Russell King , dri-devel@lists.freedesktop.org, Jonathan Hunter , iommu@lists.linux-foundation.org, Rob Herring , Ben Skeggs , Catalin Marinas , linux-tegra@vger.kernel.org, Frank Rowand , linux-kernel@vger.kernel.org Subject: Re: [RFC PATCH v1 0/6] Resolve unwanted DMA backing with IOMMU Message-ID: <20180803170026.GD21283@jcrouse-lnx.qualcomm.com> Mail-Followup-To: Robin Murphy , Dmitry Osipenko , Will Deacon , Joerg Roedel , Mikko Perttunen , Thierry Reding , devicetree@vger.kernel.org, nouveau@lists.freedesktop.org, "Rafael J. Wysocki" , Nicolas Chauvet , Greg Kroah-Hartman , Russell King , dri-devel@lists.freedesktop.org, Jonathan Hunter , iommu@lists.linux-foundation.org, Rob Herring , Ben Skeggs , Catalin Marinas , linux-tegra@vger.kernel.org, Frank Rowand , linux-kernel@vger.kernel.org References: <20180726231624.21084-1-digetx@gmail.com> <20180727170326.GA21283@jcrouse-lnx.qualcomm.com> <2402373.RyBdR7VSnO@dimapc> <2887450.sPhIOOMKZK@dimapc> <2e7fab6e-0640-8f48-07b8-2d475538b8ae@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2e7fab6e-0640-8f48-07b8-2d475538b8ae@arm.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Aug 03, 2018 at 04:43:41PM +0100, Robin Murphy wrote: > On 02/08/18 19:24, Dmitry Osipenko wrote: > >On Friday, 27 July 2018 20:16:53 MSK Dmitry Osipenko wrote: > >>On Friday, 27 July 2018 20:03:26 MSK Jordan Crouse wrote: > >>>On Fri, Jul 27, 2018 at 05:02:37PM +0100, Robin Murphy wrote: > >>>>On 27/07/18 15:10, Dmitry Osipenko wrote: > >>>>>On Friday, 27 July 2018 12:03:28 MSK Will Deacon wrote: > >>>>>>On Fri, Jul 27, 2018 at 10:25:13AM +0200, Joerg Roedel wrote: > >>>>>>>On Fri, Jul 27, 2018 at 02:16:18AM +0300, Dmitry Osipenko wrote: > >>>>>>>>The proposed solution adds a new option to the base device driver > >>>>>>>>structure that allows device drivers to explicitly convey to the > >>>>>>>>drivers > >>>>>>>>core that the implicit IOMMU backing for devices must not happen. > >>>>>>> > >>>>>>>Why is IOMMU mapping a problem for the Tegra GPU driver? > >>>>>>> > >>>>>>>If we add something like this then it should not be the choice of the > >>>>>>>device driver, but of the user and/or the firmware. > >>>>>> > >>>>>>Agreed, and it would still need somebody to configure an identity > >>>>>>domain > >>>>>>so > >>>>>>that transactions aren't aborted immediately. We currently allow the > >>>>>>identity domain to be used by default via a command-line option, so I > >>>>>>guess > >>>>>>we'd need a way for firmware to request that on a per-device basis. > >>>>> > >>>>>The IOMMU mapping itself is not a problem, the problem is the > >>>>>management > >>>>>of > >>>>>the IOMMU. For Tegra we don't want anything to intrude into the IOMMU > >>>>>activities because: > >>>>> > >>>>>1) GPU HW require additional configuration for the IOMMU usage and dumb > >>>>>mapping of the allocations simply doesn't work. > >>>> > >>>>Generally, that's already handled by the DRM drivers allocating > >>>>their own unmanaged domains. The only problem we really need to > >>>>solve in that regard is that currently the device DMA ops don't get > >>>>updated when moving away from the managed domain. That's been OK for > >>>>the VFIO case where the device is bound to a different driver which > >>>>we know won't make any explicit DMA API calls, but for the more > >>>>general case of IOMMU-aware drivers we could certainly do with a bit > >>>>of cooperation between the IOMMU API, DMA API, and arch code to > >>>>update the DMA ops dynamically to cope with intermediate subsystems > >>>>making DMA API calls on behalf of devices they don't know the > >>>>intimate details of. > >>>> > >>>>>2) Older Tegra generations have a limited resource and capabilities in > >>>>>regards to IOMMU usage, allocating IOMMU domain per-device is just > >>>>>impossible for example. > >>>>> > >>>>>3) HW performs context switches and so particular allocations have to > >>>>>be > >>>>>assigned to a particular contexts IOMMU domain. > >>>> > >>>>I understand Qualcomm SoCs have a similar thing too, and AFAICS that > >>>>case just doesn't fit into the current API model at all. We need the > >>>>IOMMU driver to somehow know about the specific details of which > >>>>devices have magic associations with specific contexts, and we > >>>>almost certainly need a more expressive interface than > >>>>iommu_domain_alloc() to have any hope of reliable results. > >>> > >>>This is correct for Qualcomm GPUs - The GPU hardware context switching > >>>requires a specific context and there are some restrictions around > >>>secure contexts as well. > >>> > >>>We don't really care if the DMA attaches to a context just as long as it > >>>doesn't attach to the one(s) we care about. Perhaps a "valid context" mask > >>>would work in from the DT or the device struct to give the subsystems a > >>>clue as to which domains they were allowed to use. I recognize that there > >>>isn't a one-size-fits-all solution to this problem so I'm open to > >>>different > >>>ideas. > >> > >>Designating whether implicit IOMMU backing is appropriate for a device via > >>device-tree property sounds a bit awkward because that will be a kinda > >>software description (of a custom Linux driver model), while device-tree is > >>supposed to describe HW. > >> > >>What about to grant IOMMU drivers with ability to decide whether the > >>implicit backing for a device is appropriate? Like this: > >> > >>bool implicit_iommu_for_dma_is_allowed(struct device *dev) > >>{ > >> const struct iommu_ops *ops = dev->bus->iommu_ops; > >> struct iommu_group *group; > >> > >> group = iommu_group_get(dev); > >> if (!group) > >> return NULL; > >> > >> iommu_group_put(group); > >> > >> if (!ops->implicit_iommu_for_dma_is_allowed) > >> return true; > >> > >> return ops->implicit_iommu_for_dma_is_allowed(dev); > >>} > >> > >>Then arch_setup_dma_ops() could have a clue whether implicit IOMMU backing > >>for a device is appropriate. > > > >Guys, does it sound good to you or maybe you have something else on your mind? > >Even if it's not an ideal solution, it fixes the immediate problem and should > >be good enough for the starter. > > To me that looks like a step ion the wrong direction that won't help > at all in actually addressing the underlying issues. > > If the GPU driver wants to explicitly control IOMMU mappings instead > of relying on the IOMMU_DOMAIN_DMA abstraction, then it should use > its own unmanaged domain. At that point it shouldn't matter if a DMA > ops domain was allocated, since the GPU device will no longer be > attached to it. Yes, there may be some improvements to make like > having unused domains not consume hardware contexts, but that's > internal to the relevant IOMMU drivers. If moving in and out of DMA > ops domains leaves the actual dma_ops broken, that's already a > problem between the IOMMU API and the arch DMA code as I've > mentioned before. > > Furthermore, given what the example above is trying to do, > arch_setup_dma_ops() is way too late to do it - the default domain > was already set up in iommu_group_get_for_dev() when the IOMMU > driver first saw that device. An "opt-out" mechanism that doesn't > actually opt out and just bodges around being opted-in after the > fact doesn't strike me as something which can grow to be robust and > maintainable. > > For the case where a device has some special hardware relationship > with a particular IOMMU context, the IOMMU driver *has* to be > completely aware of that, i.e. it needs to be described in DT/ACPI, > either via some explicit binding or at least inferred from some > SoC/instance-specific IOMMU compatible. Then the IOMMU driver needs > to know when the driver for that device is requesting its special > domain so that it provide the correct context (and *not* allocate > that context for other uses). Anything which just relies on the > order in which things currently happen to be allocated is far too > fragile long-term. Speculating what this would look like for arm-smmu-v2. Thinking we would add either a mask or a bit to the per-device smmu data to identify the "available" contexts for dma (or alternatively, the "reserved" domains for the device) and then change up _arm_smmu_alloc_bitmap() to hand out the right numbers accordingly. Then that leaves the interface for the device to request a specific context - I guess a DOMAIN_ATTR would work for that in lieu of changing iommu_alloc_domain() and friends. Would this match up with your thinking? Jordan -- The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project