From mboxrd@z Thu Jan 1 00:00:00 1970 From: Will Deacon Subject: Re: [PATCH 2/3] iommu/arm-smmu: Add initial driver support for ARM SMMUv3 devices Date: Thu, 21 May 2015 12:25:55 +0100 Message-ID: <20150521112555.GH21920@arm.com> References: <1431108046-9675-1-git-send-email-will.deacon@arm.com> <1431108046-9675-3-git-send-email-will.deacon@arm.com> <5551AE56.6050906@huawei.com> <20150512165500.GE2062@arm.com> <55530C4F.5000605@huawei.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Content-Disposition: inline In-Reply-To: <55530C4F.5000605-hv44wF8Li93QT0dZR+AlfA@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: leizhen Cc: "huxinwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org" , "iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org" , Sanil kumar , Gaojianbo , Dingtianhong , "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" List-Id: iommu@lists.linux-foundation.org SGkgYWdhaW4sCgpTb3JyeSBmb3IgdGhlIGRlbGF5IGluIHJlcGx5aW5nLCBJJ3ZlIGJlZW4gdGll ZCB1cCB3aXRoIG90aGVyIHN0dWZmLgoKT24gV2VkLCBNYXkgMTMsIDIwMTUgYXQgMDk6MzM6MTlB TSArMDEwMCwgbGVpemhlbiB3cm90ZToKPiBPbiAyMDE1LzUvMTMgMDo1NSwgV2lsbCBEZWFjb24g d3JvdGU6Cj4gPiBUaGUgcHVycG9zZSBvZiB0aGUgdHdvIGxldmVsIGFwcHJvYWNoIGlzbid0IHRv IHNhdmUgbWVtb3J5OyBpdCdzIHRvIHJlbW92ZQo+ID4gdGhlIG5lZWQgZm9yIGEgc2luZ2xlIChs YXJnZSkgY29udGlndW91cyBibG9jay4gRnVydGhlcm1vcmUsIHdlIG5lZWQgYWxsCj4gPiBtYXN0 ZXIgZGV2aWNlcyBpbiB0aGUgc3lzdGVtIHRvIGNvbWUgdXAgaW4gYSBzdGF0ZSB3aGVyZSB0aGVp ciB0cmFuc2FjdGlvbnMKPiA+IGJ5cGFzcyB0aGUgU01NVSAoaS5lLiB3ZSBkb24ndCByZXF1aXJl IHRoZW0gdG8gdXNlIHRoZSBTTU1VIGZvcgo+ID4gdHJhbnNsYXRpb24pLiBBY2hpZXZpbmcgdGhp cyBuZWNlc3NpdGF0ZXMgYSBmdWxseS1wb3B1bGF0ZWQgc3RyZWFtLXRhYmxlCj4gPiBiZWZvcmUg YW55IERNQSBvY2N1cnMuCj4gCj4gT0ssIGJ1dCBmb3Igbm9uLXBjaSBkZXZpY2VzKG1heWJlIHBj aSBkZXZpY2VzIGFsc28pLCBpbml0aWFsaXplIGFsbAo+IGRldmljZXMoU3RyZWFtSURzKSB0byBi eXBhc3MgbW9kZSBtYXliZSBpbmNvcnJlY3QuIFNvbWUgZGV2aWNlcyBtYXliZQo+IGNhcGFibGUg YnJpbmcgYXR0cmlidXRlcyBieSBpdHNlbGYsIGFuZCBhY2Nlc3MgZGlmZmVyZW50IG1lbW9yeSB3 aXRoCj4gZGlmZmVyZW50IGF0dHJpYnV0ZXMoZGV2aWNlIGF0dHJpYnV0ZSBvciBjYWNoZWFibGUg YXR0cmlidXRlKTsgc29tZQo+IGRldmljZXMgbWF5YmUgbm90IGNhcGFibGUgYnJpbmcgYXR0cmli dXRlcyBieSBpdHNlbGYsIGJ1dCBuZWVkIGFjY2Vzcwo+IG1lbW9yeSB3aXRoIGNhY2hlYWJsZSBh dHRyaWJ1dGUgb25seSwgc28gd2Ugc2hvdWxkIHNldCBTVEUuTVRDRkcgPSAxLgo+IFByb3ZpZGUg ZHRzIGNvbmZpZ3VyYXRpb24gd2lsbCBiZSBiZXR0ZXIuCgpJZiB3ZSB3YW50IHRvIG1ha2UgdXNl IG9mIG1lbW9yeSBvdmVycmlkZXMsIHRoZW4gSSB0aGluayB3ZSBzaG91bGQgYWRkCnRoYXQgYXMg YSBzZXBhcmF0ZSBwYXRjaCBiZWNhdXNlIGl0IHdvdWxkIGhhdmUgaW1wbGljYXRpb25zIG9uIHRo ZSBJT01NVQpBUEkuIEkgZG9uJ3QgcGFydGljdWxhcmx5IGxpa2UgcHV0dGluZyB0aGlzIHBvbGlj eSBpbmZvcm1hdGlvbiBpbiB0aGUKZGV2aWNlLXRyZWUgYmluZGluZyBvbiBhIHBlci1kcml2ZXIg YmFzaXMuCgo+IEZ1cnRoZXJtb3JlLCBJIGRvbid0IGFncmVlIGluaXRpYWxpemUgYWxsIGRldmlj ZXMgdG8gYnlwYXNzIGJ5IGRlZmF1bHQuCj4gU3VwcG9zZSBhIG5vbi1wY2kgZGV2aWNlJ3MgU3Ry ZWFtSUQgY2FuIGJlIGR5bmFtaWMgY29uZmlndXJlZC4gV2UgcmVxdWlyZQo+IFN0cmVhbUlELUEs IGJ1dCB0aGUgZGV2aWNlIGFjdHVhbGx5IGlzc3VlIFN0cmVhbUlELUIgYmVjYXVzZSBvZiBzb2Z0 d2FyZQo+IGNvbmZpZ3VyYXRpb24gZmF1bHQuIFdlIHJlYWxseSBob3BlIFNNTVUgY2FuIHJlcG9y dCBCYWQgU3RyZWFtSUQgZmF1bHQuCgpJIHRoaW5rIHRoZSBkZWZhdWx0IGJlaGF2aW91ciBoYXMg dG8gYmUgdG8gYnlwYXNzIGJ5IGRlZmF1bHQsIG90aGVyd2lzZQp3ZSBjYW4gYnJlYWsgRE1BIGZv ciBhbnkgZGV2aWNlcyBiZWhpbmQgYW4gU01NVSBidXQgbm90IGF0dGFjaGVkIHRvIGFuCklPTU1V IGRvbWFpbi4gSG93IGFib3V0IEkgYWRkIGEgY21kbGluZSBwYXJhbWV0ZXIgdG8gY2hhbmdlIHRo ZSBkZWZhdWx0CmJlaGF2aW91cj8KCj4gPiBJIHN1cHBvc2Ugd2UgY291bGQgbG9vayBpbnRvIHBv cHVsYXRpbmcgaXQgYmFzZWQgb24gdGhlIC0+YWRkX2RldmljZQo+ID4gY2FsbGJhY2ssIHdoaWNo IGN1cnJlbnRseSBkb2VzIGhhdmUgc29tZSByYW5nZSBjaGVja3MKPiA+IChhcm1fc21tdV9zaWRf aW5fcmFuZ2UpLiBJcyB0aGF0IHdoYXQgeW91IGhhZCBpbiBtaW5kPwo+IAo+IFllcywgbWF5YmUg YXR0YWNoX2Rldi4gQnV0IHdlIHNob3VsZCBhbGxvY2F0ZWQgdGhlIHdob2xlIEx2MSB0YWJsZSBi YXNlIG9uCj4gU01NVV9JRFIxLlNJRFNJWkUKCkkgdGhpbmsgaXQgbmVlZHMgdG8gYmUgaW4gYWRk X2RldmljZSBzbyB0aGF0IHdlIGNhbiBpbXBsZW1lbnQgdGhlIGRlZmF1bHQKYnlwYXNzIGNhc2Uu IFdlIGNvdWxkIGV2ZW4gaGF2ZSBvbmUgTDIgdGFibGUgcGVyIFBDSSBidXMsIGJ1dCBJIG5lZWQg dG8KdGhpbmsgYWJvdXQgaG93IGJpZyB0aGUgTDEgdGFibGUgc2hvdWxkIGJlLgoKPiA+Pj4gKyAg ICAgICAgICAgICBpZiAoIWRlc2MtPmwycHRyKSB7Cj4gPj4+ICsgICAgICAgICAgICAgICAgICAg ICBkZXZfZXJyKHNtbXUtPmRldiwKPiA+Pj4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ImZhaWxlZCB0byBhbGxvY2F0ZSBsMiBzdHJlYW0gdGFibGUgJXVcbiIsIGkpOwo+ID4+PiArICAg ICAgICAgICAgICAgICAgICAgcmV0ID0gLUVOT01FTTsKPiA+Pj4gKyAgICAgICAgICAgICAgICAg ICAgIGdvdG8gb3V0X2ZyZWVfbDI7Cj4gPj4+ICsgICAgICAgICAgICAgfQo+ID4+PiArCj4gPj4+ ICsgICAgICAgICAgICAgYXJtX3NtbXVfaW5pdF9ieXBhc3Nfc3RlcyhkZXNjLT5sMnB0ciwgMSA8 PCBTVFJUQUJfU1BMSVQpOwo+ID4+PiArICAgICAgICAgICAgIGFybV9zbW11X3dyaXRlX3N0cnRh Yl9sMV9kZXNjKHN0cnRhYiwgZGVzYyk7Cj4gPj4+ICsgICAgICAgICAgICAgc3RydGFiICs9IFNU UlRBQl9TVEVfRFdPUkRTOwo+ID4+PiArICAgICB9Cj4gPj4+ICsKPiA+Pj4gKyAgICAgcmV0dXJu IDA7Cj4gPj4+ICsKPiA+Pj4gK291dF9mcmVlX2wyOgo+ID4+PiArICAgICBhcm1fc21tdV9mcmVl X2wyX3N0cnRhYihzbW11KTsKPiA+Pj4gKyAgICAgcmV0dXJuIHJldDsKPiA+Pj4gK30KPiA+Pj4g Kwo+ID4+PiArc3RhdGljIGludCBhcm1fc21tdV9pbml0X3N0cnRhYihzdHJ1Y3QgYXJtX3NtbXVf ZGV2aWNlICpzbW11KQo+ID4+PiArewo+ID4+PiArICAgICB2b2lkICpzdHJ0YWI7Cj4gPj4+ICsg ICAgIHU2NCByZWc7Cj4gPj4+ICsgICAgIHUzMiBzaXplOwo+ID4+PiArICAgICBpbnQgcmV0ID0g MDsKPiA+Pj4gKwo+ID4+PiArICAgICBzdHJ0YWIgPSBkbWFfemFsbG9jX2NvaGVyZW50KHNtbXUt PmRldiwgMSA8PCBTVFJUQUJfTDFfU1pfU0hJRlQsCj4gPj4+ICsgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgJnNtbXUtPnN0cnRhYl9jZmcuc3RydGFiX2RtYSwgR0ZQX0tFUk5FTCk7 Cj4gPj4KPiA+PiBBcyBhYm92ZSwgd2hlbiBMdjIgdGFibGVzIGFyZSBkeW5hbWljIGFsbG9jYXRp b24sIHdlIGNhbiBjcmVhdGUgTHYxIHRhYmxlCj4gPj4gYmFzZSBvbiBTTU1VX0lEUjEuU0lEU0la RSBhbmQgc3VwcG9ydCBub24tcGljIGRldmljZXMuIE9oLCBpZiBTSURTSVpFIGlzCj4gPj4gdG9v IGxhcmdlLCBsaWtlIDMyLiBNYXliZSB3ZSBzaG91bGQgdXNlIDY0SyBzaXplIEx2MiB0YWJsZS4g IEJ1dCB3ZSBjYW4KPiA+PiBvbmx5IHVzZSBQQUdFX1NJWkUgZmlyc3QsIGZvciBsYXp5Lgo+ID4K PiA+IFJpZ2h0IG5vdywgdGhlIGwyIHRhYmxlcyBhcmUgNGsgYW5kIGwxIHRhYmxlIGlzIDhrLiBU aGF0J3MgKGEpIG5pY2UgZm9yCj4gPiBhbGxvY2F0aW9uIG9uIGEgc3lzdGVtIHdpdGggNGsgcGFn ZXMgYW5kIChiKSBlbm91Z2ggdG8gY292ZXIgYSBQQ0kgaG9zdAo+ID4gY29udHJvbGxlci4gVGhl IHByb2JsZW0gd2l0aCBzdXBwb3J0aW5nIDMyLWJpdCBTSURzIGlzIHRoYXQgdGhlIGwxIHdpbGwK PiA+IGJlY29tZSBhIHNpemVhYmxlIGNvbnRpZ3VvdXMgY2h1bmsgd2hpY2ggd2UnbGwgaGF2ZSB0 byBhbGxvY2F0ZQo+ID4gcmVnYXJkbGVzcy4KPiA+Cj4gPiBTbywgaWYgd2Uga2VlcCB0aGUgbDIg c2l6ZSBhdCA0ayBmb3IgdGhlIG1vbWVudCwgaG93IGJpZyB3b3VsZCB5b3UgbGlrZQo+ID4gdGhl IGwxPyBGb3IgYSAzMi1iaXQgbnVtYmVyc3BhY2UsIGl0IHdvdWxkIGJlIDRNQiwgd2hpY2ggSSBk b24ndCB0aGluayBpcwo+ID4gcHJhY3RpY2FsLgo+IAo+IFllcywgYmVjYXVzZSBJIGRvbid0IHRo aW5rIFNNTVVfSURSMS5TSURTSVpFID0gMzIgcmVhbGx5IGV4aXN0LCBzbyBJIHNhaWQKPiB1c2Ug UEFHRV9TSVpFIGZpcnN0LgoKV2VsbCwgSSBjZXJ0YWlubHkgd291bGRuJ3QgcnVsZSBhbnl0aGlu ZyBvdXQuCgo+IEJ1dCBub3csIHRoaXMgcGF0Y2ggb25seSBzdXBwb3J0IFNNTVVfSURSMS5TSURT SVpFIDw9IDE2LiBTdXBwb3NlCj4gU01NVV9JRFIxLlNJRFNJWkUgPSAyNe+8jEx2MiB0YWJsZSBz aXplIGF0IDRLLCB0aGVuIEx2MSB0YWJsZSBuZWVkIDJeKDI1IC0KPiA2ICsgMykgPSAyXjIyID0g NE0uIElmIHdlIHByZS1hbGxvY2F0ZWQgYWxsIEx2MiB0YWJsZSwgd2UgbmVlZCAyXjI1ICogNjQg PQo+IDJeMzEgPSAyRywgdGhhdCdzIHRvbyBiaWcuIFNvIHdlIG11c3Qgb25seSBhbGxvY2F0ZSBM djIgdGFibGUgd2hlbiB3ZQo+IG5lZWRlZC4KCkkgYWdyZWUgdGhhdCdzIHdheSB0b28gYmlnLCBi dXQgSSBoYXZlIGxpbWl0ZWQgdGhpbmdzIHRvIDE2LWJpdCBmb3IgYSByZWFzb24KOykKCj4gSWYg U01NVV9JRFIxLlNJRFNJWkUgPSAzMiByZWFsbHkgZXhpc3Qob3IgdG9vIGJpZyksIHdlIG5lZWQg ZHluYW1pYyBjaG9vc2UKPiBMdjIgdGFibGUgc2l6ZSg0SywxNkssNjRLKS4gIEJlY2F1c2UgTHYx IHRhYmxlIG1heWJlIHRvbyBiaWcsIGFuZCBjYW4gbm90Cj4gYmUgYWxsb2NhdGVkIGJ5IGN1cnJl bnQgQVBJLCBhIGR0cyBjb25maWd1cmF0aW9uIHNob3VsZCBiZSBhZGRlZCwgbGlrZQo+IGx2MS10 YWJsZS1iYXNlID0gPDB4MCAweDA+LCBhbmQgd2UgdXNlIGlvcmVtYXBfY2FjaGUgZ2V0IFZBKG1h eWJlIGlvcmVtYXAsCj4gZm9yIG5vbi1jb2hlcmVudCBTTU1VKS4KPiAKPiBPaCwgSSBjYW4gZG8g aXQgYWZ0ZXIgeW91ciBwYXRjaGVzIHVwc3RyZWFtZWQsIGJlY2F1c2UgdGhpcyBwcm9ibGVtIG1h eWJlCj4gb25seSBJIG1ldC4KCkknbGwgaGF2ZSBhIHRoaW5rIGFib3V0IHRoaXMgYW5kIHNlZSB3 aGF0IEkgY2FuIGNvbWUgdXAgd2l0aCBmb3IgdmVyc2lvbgoyIG9mIHRoZSBwYXRjaC4gSSdkIGxp a2UgdG8gYXZvaWQgYWRkaW5nIGFkZGl0aW9uYWwgcHJvcGVydGllcyB0byB0aGUgRFQKdW50aWwg dGhleSdyZSBhY3R1YWxseSBuZWVkZWQsIHRob3VnaC4KCj4gPj4+ICsgICAgIC8qIEludmFsaWRh dGUgYW55IGNhY2hlZCBjb25maWd1cmF0aW9uICovCj4gPj4+ICsgICAgIGNtZC5vcGNvZGUgPSBD TURRX09QX0NGR0lfQUxMOwo+ID4+PiArICAgICBhcm1fc21tdV9jbWRxX2lzc3VlX2NtZChzbW11 LCAmY21kKTsKPiA+Pj4gKyAgICAgY21kLm9wY29kZSA9IENNRFFfT1BfQ01EX1NZTkM7Cj4gPj4+ ICsgICAgIGFybV9zbW11X2NtZHFfaXNzdWVfY21kKHNtbXUsICZjbWQpOwo+ID4+PiArCj4gPj4+ ICsgICAgIC8qIEludmFsaWRhdGUgYW55IHN0YWxlIFRMQiBlbnRyaWVzICovCj4gPj4+ICsgICAg IGNtZC5vcGNvZGUgPSBDTURRX09QX1RMQklfRUwyX0FMTDsKPiA+Pgo+ID4+IERvIHdlIG5lZWQg dG8gZXhlY3V0ZSBDTURRX09QX1RMQklfRUwyX0FMTD8gTGludXggb25seSBydW4gYXQgRUwxLiBJ dCBhdAo+ID4+IGxlYXN0IHJlbHkgb24gU01NVV9JRFIwLkh5cAo+ID4KPiA+IFRoZSBTTU1VIGlz bid0IGd1YXJhbnRlZWQgdG8gY29tZSB1cCBjbGVhbiBvdXQgb2YgcmVzZXQsIHNvIGl0J3MgYSBn b29kCj4gPiBpZGVhIHRvIHBlcmZvcm0gdGhpcyBpbnZhbGlkYXRpb24gaW4gY2FzZSBvZiBqdW5r IGluIHRoZSBUTEIuIEdpdmVuIHRoYXQKPiA+IExpbnV4IG93bnMgdGhlIHN0YWdlLTIgdHJhbnNs YXRpb24sIHRoZW4gdGhpcyBpcyB0aGUgcmlnaHQgcGxhY2UgdG8gZG8gaXQuCj4gCj4gT0suIEJ1 dCBpdCBzaG91bGQgYmUgY29udHJvbGVkIGJ5IFNNTVVfSURSMC5IeXAuIEkgbWVhbnM6Cj4gaWYg KFNNTVVfSURSMC5IeXApCj4gICAgICAgICBleGVjdXRlIGNvbW1hbmQgQ01EUV9PUF9UTEJJX0VM Ml9BTEwKPiAKPiBXaGVuIFNNTVVfSURSMC5IeXA94oCZMOKAmSwgdGhpcyBjb21tYW5kIGNhdXNl cyBDRVJST1JfSUxMCgpXZWxsIHNwb3R0ZWQsIHRoYW5rcyEKCldpbGwKX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KaW9tbXUgbWFpbGluZyBsaXN0CmlvbW11 QGxpc3RzLmxpbnV4LWZvdW5kYXRpb24ub3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9u Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2lvbW11 From mboxrd@z Thu Jan 1 00:00:00 1970 From: will.deacon@arm.com (Will Deacon) Date: Thu, 21 May 2015 12:25:55 +0100 Subject: [PATCH 2/3] iommu/arm-smmu: Add initial driver support for ARM SMMUv3 devices In-Reply-To: <55530C4F.5000605@huawei.com> References: <1431108046-9675-1-git-send-email-will.deacon@arm.com> <1431108046-9675-3-git-send-email-will.deacon@arm.com> <5551AE56.6050906@huawei.com> <20150512165500.GE2062@arm.com> <55530C4F.5000605@huawei.com> Message-ID: <20150521112555.GH21920@arm.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org Hi again, Sorry for the delay in replying, I've been tied up with other stuff. On Wed, May 13, 2015 at 09:33:19AM +0100, leizhen wrote: > On 2015/5/13 0:55, Will Deacon wrote: > > The purpose of the two level approach isn't to save memory; it's to remove > > the need for a single (large) contiguous block. Furthermore, we need all > > master devices in the system to come up in a state where their transactions > > bypass the SMMU (i.e. we don't require them to use the SMMU for > > translation). Achieving this necessitates a fully-populated stream-table > > before any DMA occurs. > > OK, but for non-pci devices(maybe pci devices also), initialize all > devices(StreamIDs) to bypass mode maybe incorrect. Some devices maybe > capable bring attributes by itself, and access different memory with > different attributes(device attribute or cacheable attribute); some > devices maybe not capable bring attributes by itself, but need access > memory with cacheable attribute only, so we should set STE.MTCFG = 1. > Provide dts configuration will be better. If we want to make use of memory overrides, then I think we should add that as a separate patch because it would have implications on the IOMMU API. I don't particularly like putting this policy information in the device-tree binding on a per-driver basis. > Furthermore, I don't agree initialize all devices to bypass by default. > Suppose a non-pci device's StreamID can be dynamic configured. We require > StreamID-A, but the device actually issue StreamID-B because of software > configuration fault. We really hope SMMU can report Bad StreamID fault. I think the default behaviour has to be to bypass by default, otherwise we can break DMA for any devices behind an SMMU but not attached to an IOMMU domain. How about I add a cmdline parameter to change the default behaviour? > > I suppose we could look into populating it based on the ->add_device > > callback, which currently does have some range checks > > (arm_smmu_sid_in_range). Is that what you had in mind? > > Yes, maybe attach_dev. But we should allocated the whole Lv1 table base on > SMMU_IDR1.SIDSIZE I think it needs to be in add_device so that we can implement the default bypass case. We could even have one L2 table per PCI bus, but I need to think about how big the L1 table should be. > >>> + if (!desc->l2ptr) { > >>> + dev_err(smmu->dev, > >>> + "failed to allocate l2 stream table %u\n", i); > >>> + ret = -ENOMEM; > >>> + goto out_free_l2; > >>> + } > >>> + > >>> + arm_smmu_init_bypass_stes(desc->l2ptr, 1 << STRTAB_SPLIT); > >>> + arm_smmu_write_strtab_l1_desc(strtab, desc); > >>> + strtab += STRTAB_STE_DWORDS; > >>> + } > >>> + > >>> + return 0; > >>> + > >>> +out_free_l2: > >>> + arm_smmu_free_l2_strtab(smmu); > >>> + return ret; > >>> +} > >>> + > >>> +static int arm_smmu_init_strtab(struct arm_smmu_device *smmu) > >>> +{ > >>> + void *strtab; > >>> + u64 reg; > >>> + u32 size; > >>> + int ret = 0; > >>> + > >>> + strtab = dma_zalloc_coherent(smmu->dev, 1 << STRTAB_L1_SZ_SHIFT, > >>> + &smmu->strtab_cfg.strtab_dma, GFP_KERNEL); > >> > >> As above, when Lv2 tables are dynamic allocation, we can create Lv1 table > >> base on SMMU_IDR1.SIDSIZE and support non-pic devices. Oh, if SIDSIZE is > >> too large, like 32. Maybe we should use 64K size Lv2 table. But we can > >> only use PAGE_SIZE first, for lazy. > > > > Right now, the l2 tables are 4k and l1 table is 8k. That's (a) nice for > > allocation on a system with 4k pages and (b) enough to cover a PCI host > > controller. The problem with supporting 32-bit SIDs is that the l1 will > > become a sizeable contiguous chunk which we'll have to allocate > > regardless. > > > > So, if we keep the l2 size at 4k for the moment, how big would you like > > the l1? For a 32-bit numberspace, it would be 4MB, which I don't think is > > practical. > > Yes, because I don't think SMMU_IDR1.SIDSIZE = 32 really exist, so I said > use PAGE_SIZE first. Well, I certainly wouldn't rule anything out. > But now, this patch only support SMMU_IDR1.SIDSIZE <= 16. Suppose > SMMU_IDR1.SIDSIZE = 25?Lv2 table size at 4K, then Lv1 table need 2^(25 - > 6 + 3) = 2^22 = 4M. If we pre-allocated all Lv2 table, we need 2^25 * 64 = > 2^31 = 2G, that's too big. So we must only allocate Lv2 table when we > needed. I agree that's way too big, but I have limited things to 16-bit for a reason ;) > If SMMU_IDR1.SIDSIZE = 32 really exist(or too big), we need dynamic choose > Lv2 table size(4K,16K,64K). Because Lv1 table maybe too big, and can not > be allocated by current API, a dts configuration should be added, like > lv1-table-base = <0x0 0x0>, and we use ioremap_cache get VA(maybe ioremap, > for non-coherent SMMU). > > Oh, I can do it after your patches upstreamed, because this problem maybe > only I met. I'll have a think about this and see what I can come up with for version 2 of the patch. I'd like to avoid adding additional properties to the DT until they're actually needed, though. > >>> + /* Invalidate any cached configuration */ > >>> + cmd.opcode = CMDQ_OP_CFGI_ALL; > >>> + arm_smmu_cmdq_issue_cmd(smmu, &cmd); > >>> + cmd.opcode = CMDQ_OP_CMD_SYNC; > >>> + arm_smmu_cmdq_issue_cmd(smmu, &cmd); > >>> + > >>> + /* Invalidate any stale TLB entries */ > >>> + cmd.opcode = CMDQ_OP_TLBI_EL2_ALL; > >> > >> Do we need to execute CMDQ_OP_TLBI_EL2_ALL? Linux only run at EL1. It at > >> least rely on SMMU_IDR0.Hyp > > > > The SMMU isn't guaranteed to come up clean out of reset, so it's a good > > idea to perform this invalidation in case of junk in the TLB. Given that > > Linux owns the stage-2 translation, then this is the right place to do it. > > OK. But it should be controled by SMMU_IDR0.Hyp. I means: > if (SMMU_IDR0.Hyp) > execute command CMDQ_OP_TLBI_EL2_ALL > > When SMMU_IDR0.Hyp=?0?, this command causes CERROR_ILL Well spotted, thanks! Will