From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751795AbbDNGb2 (ORCPT ); Tue, 14 Apr 2015 02:31:28 -0400 Received: from mailgw01.mediatek.com ([218.249.47.110]:41403 "EHLO mailgw01.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1751381AbbDNGbU (ORCPT ); Tue, 14 Apr 2015 02:31:20 -0400 X-Listener-Flag: 11101 Message-ID: <1428993067.13552.14.camel@mhfsdcap03> Subject: Re: [PATCH 2/5] iommu/mediatek: Add mt8173 IOMMU driver From: Yong Wu To: Tomasz Figa CC: Robin Murphy , Mark Rutland , , , Catalin Marinas , Joerg Roedel , Will Deacon , "linux-kernel@vger.kernel.org" , , Rob Herring , "Daniel Kurtz" , Sasha Hauer , "Matthias Brugger" , , "linux-arm-kernel@lists.infradead.org" , Lucas Stach Date: Tue, 14 Apr 2015 14:31:07 +0800 In-Reply-To: References: <1425638900-24989-1-git-send-email-yong.wu@mediatek.com> <1425638900-24989-3-git-send-email-yong.wu@mediatek.com> <1426677749.22581.38.camel@mhfsdcap03> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.3-0ubuntu6 Content-Transfer-Encoding: 8bit MIME-Version: 1.0 X-MTK: N Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Tomasz, Thanks very much for you suggestion and explain so detail. please help check below. On Fri, 2015-03-27 at 18:41 +0900, Tomasz Figa wrote: > Hi Yong Wu, > > Sorry for long delay, I had to figure out some time to look at this again. > > On Wed, Mar 18, 2015 at 8:22 PM, Yong Wu wrote: > >> > >> > + imudev = piommu->dev; > >> > + > >> > + spin_lock_irqsave(&priv->portlock, flags); > >> > >> What is protected by this spinlock? > > We will write a register of the local arbiter while config port. If > > some modules are in the same local arbiter, it may be overwrite. so I > > add it here. > >> > > OK. Maybe it could be called larb_lock then? It would be good to have > structures or code that should be running under this spinlock > annotated with proper comments. And purpose of the lock documented in > a comment as well (probably in a kerneldoc-style documentation of > priv). Thanks. I have move the spinlock into the smi driver, it will lock for writing the local arbiter regsiter only. > > >> > +static void mtk_iommu_detach_device(struct iommu_domain *domain, > >> > + struct device *dev) > >> > +{ > >> > >> No hardware (de)configuration or clean-up necessary? > > I will add it. Actually we design like this:If a device have attached to > > iommu domain, it won't detach from it. > > Isn't proper clean-up required for module removal? Some drivers might > be required to be loadable modules, which should be unloadable. > > >> > >> > + > >> > + piommu->protect_va = devm_kmalloc(piommu->dev, MTK_PROTECT_PA_ALIGN*2, > >> > >> style: Operators like * should have space on both sides. > >> > >> > + GFP_KERNEL); > >> > >> Shouldn't dma_alloc_coherent() be used for this? > > We don't care the data in it. I think they are the same. Could you > > help tell me why dma_alloc_coherent may be better. > > Can you guarantee that at the time you allocate the memory using > devm_kmalloc() the memory is not dirty (i.e. some write back data are > stored in CPU cache) and is not going to be written back in some time, > overwriting data put there by IOMMU hardware? > As I noted in the function "mtk_iommu_hw_init": /* protect memory,HW will write here while translation fault */ protectpa = __virt_to_phys(piommu->protect_va); We don’t care the content of this buffer, It is ok even though its data is dirty. It seem to be a the protect memory. While a translation fault happened, The iommu HW will overwrite here instead of writing to the fault physical address which may be 0 or some random address. > >> > + > >> > + iommu_set_fault_handler(domain, mtk_iommu_fault_handler, piommu); > >> > >> I don't see any other drivers doing this. Isn't this for upper layers, > >> so that they can set their own generic fault handlers? > > I think that this function is related with the iommu domain, we > > have only one multimedia iommu domain. so I add it after the iommu > > domain are created. > > No, this function is for drivers of IOMMU clients (i.e. master IP > blocks) which want to subscribe to page fault to do things like paging > on demand and so on. It shouldn't be called by IOMMU driver. Please > see other IOMMU drivers, for example rockchip-iommmu.c. Thanks. I have read it. I will delete it and print the error info in the ISR. Also call the report_iommu_fault in the ISR. > Best regards, > Tomasz From mboxrd@z Thu Jan 1 00:00:00 1970 From: Yong Wu Subject: Re: [PATCH 2/5] iommu/mediatek: Add mt8173 IOMMU driver Date: Tue, 14 Apr 2015 14:31:07 +0800 Message-ID: <1428993067.13552.14.camel@mhfsdcap03> References: <1425638900-24989-1-git-send-email-yong.wu@mediatek.com> <1425638900-24989-3-git-send-email-yong.wu@mediatek.com> <1426677749.22581.38.camel@mhfsdcap03> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Tomasz Figa Cc: Mark Rutland , devicetree-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, srv_heupstream-NuS5LvNUpcJWk0Htik3J/w@public.gmane.org, Catalin Marinas , Will Deacon , "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, Rob Herring , Daniel Kurtz , Sasha Hauer , Matthias Brugger , linux-mediatek-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" , Lucas Stach List-Id: devicetree@vger.kernel.org SGkgVG9tYXN6LAoKICAgICBUaGFua3MgdmVyeSBtdWNoIGZvciB5b3Ugc3VnZ2VzdGlvbiBhbmQg ZXhwbGFpbiBzbyBkZXRhaWwuCiAgICAgcGxlYXNlIGhlbHAgY2hlY2sgYmVsb3cuCgpPbiBGcmks IDIwMTUtMDMtMjcgYXQgMTg6NDEgKzA5MDAsIFRvbWFzeiBGaWdhIHdyb3RlOgo+IEhpIFlvbmcg V3UsCj4gCj4gU29ycnkgZm9yIGxvbmcgZGVsYXksIEkgaGFkIHRvIGZpZ3VyZSBvdXQgc29tZSB0 aW1lIHRvIGxvb2sgYXQgdGhpcyBhZ2Fpbi4KPiAKPiBPbiBXZWQsIE1hciAxOCwgMjAxNSBhdCA4 OjIyIFBNLCBZb25nIFd1IDx5b25nLnd1QG1lZGlhdGVrLmNvbT4gd3JvdGU6Cj4gPj4KPiA+PiA+ ICsgICAgICAgICAgICAgICBpbXVkZXYgPSBwaW9tbXUtPmRldjsKPiA+PiA+ICsKPiA+PiA+ICsg ICAgICAgc3Bpbl9sb2NrX2lycXNhdmUoJnByaXYtPnBvcnRsb2NrLCBmbGFncyk7Cj4gPj4KPiA+ PiBXaGF0IGlzIHByb3RlY3RlZCBieSB0aGlzIHNwaW5sb2NrPwo+ID4gICAgICAgICBXZSB3aWxs IHdyaXRlIGEgcmVnaXN0ZXIgb2YgdGhlIGxvY2FsIGFyYml0ZXIgd2hpbGUgY29uZmlnIHBvcnQu IElmCj4gPiBzb21lIG1vZHVsZXMgYXJlIGluIHRoZSBzYW1lIGxvY2FsIGFyYml0ZXIsIGl0IG1h eSBiZSBvdmVyd3JpdGUuIHNvIEkKPiA+IGFkZCBpdCBoZXJlLgo+ID4+Cj4gCj4gT0suIE1heWJl IGl0IGNvdWxkIGJlIGNhbGxlZCBsYXJiX2xvY2sgdGhlbj8gSXQgd291bGQgYmUgZ29vZCB0byBo YXZlCj4gc3RydWN0dXJlcyBvciBjb2RlIHRoYXQgc2hvdWxkIGJlIHJ1bm5pbmcgdW5kZXIgdGhp cyBzcGlubG9jawo+IGFubm90YXRlZCB3aXRoIHByb3BlciBjb21tZW50cy4gQW5kIHB1cnBvc2Ug b2YgdGhlIGxvY2sgZG9jdW1lbnRlZCBpbgo+IGEgY29tbWVudCBhcyB3ZWxsIChwcm9iYWJseSBp biBhIGtlcm5lbGRvYy1zdHlsZSBkb2N1bWVudGF0aW9uIG9mCj4gcHJpdikuCiAgIFRoYW5rcy4g SSBoYXZlIG1vdmUgdGhlIHNwaW5sb2NrIGludG8gdGhlIHNtaSBkcml2ZXIsIGl0IHdpbGwgbG9j awpmb3Igd3JpdGluZyB0aGUgbG9jYWwgYXJiaXRlciByZWdzaXRlciBvbmx5Lgo+IAo+ID4+ID4g K3N0YXRpYyB2b2lkIG10a19pb21tdV9kZXRhY2hfZGV2aWNlKHN0cnVjdCBpb21tdV9kb21haW4g KmRvbWFpbiwKPiA+PiA+ICsgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0cnVj dCBkZXZpY2UgKmRldikKPiA+PiA+ICt7Cj4gPj4KPiA+PiBObyBoYXJkd2FyZSAoZGUpY29uZmln dXJhdGlvbiBvciBjbGVhbi11cCBuZWNlc3Nhcnk/Cj4gPiBJIHdpbGwgYWRkIGl0LiBBY3R1YWxs eSB3ZSBkZXNpZ24gbGlrZSB0aGlzOklmIGEgZGV2aWNlIGhhdmUgYXR0YWNoZWQgdG8KPiA+IGlv bW11IGRvbWFpbiwgaXQgd29uJ3QgZGV0YWNoIGZyb20gaXQuCj4gCj4gSXNuJ3QgcHJvcGVyIGNs ZWFuLXVwIHJlcXVpcmVkIGZvciBtb2R1bGUgcmVtb3ZhbD8gU29tZSBkcml2ZXJzIG1pZ2h0Cj4g YmUgcmVxdWlyZWQgdG8gYmUgbG9hZGFibGUgbW9kdWxlcywgd2hpY2ggc2hvdWxkIGJlIHVubG9h ZGFibGUuCj4gCj4gPj4KPiA+PiA+ICsKPiA+PiA+ICsgICAgICAgcGlvbW11LT5wcm90ZWN0X3Zh ID0gZGV2bV9rbWFsbG9jKHBpb21tdS0+ZGV2LCBNVEtfUFJPVEVDVF9QQV9BTElHTioyLAo+ID4+ Cj4gPj4gc3R5bGU6IE9wZXJhdG9ycyBsaWtlICogc2hvdWxkIGhhdmUgc3BhY2Ugb24gYm90aCBz aWRlcy4KPiA+Pgo+ID4+ID4gKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgR0ZQX0tFUk5FTCk7Cj4gPj4KPiA+PiBTaG91bGRuJ3QgZG1hX2FsbG9jX2NvaGVyZW50KCkg YmUgdXNlZCBmb3IgdGhpcz8KPiA+ICAgICAgV2UgZG9uJ3QgY2FyZSB0aGUgZGF0YSBpbiBpdC4g SSB0aGluayB0aGV5IGFyZSB0aGUgc2FtZS4gQ291bGQgeW91Cj4gPiBoZWxwIHRlbGwgbWUgd2h5 IGRtYV9hbGxvY19jb2hlcmVudCBtYXkgYmUgYmV0dGVyLgo+IAo+IENhbiB5b3UgZ3VhcmFudGVl IHRoYXQgYXQgdGhlIHRpbWUgeW91IGFsbG9jYXRlIHRoZSBtZW1vcnkgdXNpbmcKPiBkZXZtX2tt YWxsb2MoKSB0aGUgbWVtb3J5IGlzIG5vdCBkaXJ0eSAoaS5lLiBzb21lIHdyaXRlIGJhY2sgZGF0 YSBhcmUKPiBzdG9yZWQgaW4gQ1BVIGNhY2hlKSBhbmQgaXMgbm90IGdvaW5nIHRvIGJlIHdyaXR0 ZW4gYmFjayBpbiBzb21lIHRpbWUsCj4gb3ZlcndyaXRpbmcgZGF0YSBwdXQgdGhlcmUgYnkgSU9N TVUgaGFyZHdhcmU/Cj4gCkFzIEkgbm90ZWQgaW4gdGhlIGZ1bmN0aW9uICJtdGtfaW9tbXVfaHdf aW5pdCI6CgogICAgICAgLyogcHJvdGVjdCBtZW1vcnksSFcgd2lsbCB3cml0ZSBoZXJlIHdoaWxl IHRyYW5zbGF0aW9uIGZhdWx0ICovCiAgICAgICBwcm90ZWN0cGEgPSBfX3ZpcnRfdG9fcGh5cyhw aW9tbXUtPnByb3RlY3RfdmEpOwoKICAgICBXZSBkb27igJl0IGNhcmUgdGhlIGNvbnRlbnQgb2Yg dGhpcyBidWZmZXIsIEl0IGlzIG9rIGV2ZW4gdGhvdWdoIGl0cwpkYXRhIGlzIGRpcnR5LgogICAg SXQgc2VlbSB0byBiZSBhIHRoZSBwcm90ZWN0IG1lbW9yeS4gV2hpbGUgYSB0cmFuc2xhdGlvbiBm YXVsdApoYXBwZW5lZCwgVGhlIGlvbW11IEhXIHdpbGwgb3ZlcndyaXRlIGhlcmUgaW5zdGVhZCBv ZiB3cml0aW5nIHRvIHRoZQpmYXVsdCBwaHlzaWNhbCBhZGRyZXNzIHdoaWNoIG1heSBiZSAwIG9y IHNvbWUgcmFuZG9tIGFkZHJlc3MuCgo+ID4+ID4gKwo+ID4+ID4gKyAgICAgICBpb21tdV9zZXRf ZmF1bHRfaGFuZGxlcihkb21haW4sIG10a19pb21tdV9mYXVsdF9oYW5kbGVyLCBwaW9tbXUpOwo+ ID4+Cj4gPj4gSSBkb24ndCBzZWUgYW55IG90aGVyIGRyaXZlcnMgZG9pbmcgdGhpcy4gSXNuJ3Qg dGhpcyBmb3IgdXBwZXIgbGF5ZXJzLAo+ID4+IHNvIHRoYXQgdGhleSBjYW4gc2V0IHRoZWlyIG93 biBnZW5lcmljIGZhdWx0IGhhbmRsZXJzPwo+ID4gICAgICBJIHRoaW5rIHRoYXQgdGhpcyBmdW5j dGlvbiBpcyByZWxhdGVkIHdpdGggdGhlIGlvbW11IGRvbWFpbiwgd2UKPiA+IGhhdmUgb25seSBv bmUgbXVsdGltZWRpYSBpb21tdSBkb21haW4uIHNvIEkgYWRkIGl0IGFmdGVyIHRoZSBpb21tdQo+ ID4gZG9tYWluIGFyZSBjcmVhdGVkLgo+IAo+IE5vLCB0aGlzIGZ1bmN0aW9uIGlzIGZvciBkcml2 ZXJzIG9mIElPTU1VIGNsaWVudHMgKGkuZS4gbWFzdGVyIElQCj4gYmxvY2tzKSB3aGljaCB3YW50 IHRvIHN1YnNjcmliZSB0byBwYWdlIGZhdWx0IHRvIGRvIHRoaW5ncyBsaWtlIHBhZ2luZwo+IG9u IGRlbWFuZCBhbmQgc28gb24uIEl0IHNob3VsZG4ndCBiZSBjYWxsZWQgYnkgSU9NTVUgZHJpdmVy LiBQbGVhc2UKPiBzZWUgb3RoZXIgSU9NTVUgZHJpdmVycywgZm9yIGV4YW1wbGUgcm9ja2NoaXAt aW9tbW11LmMuCiAgICAgVGhhbmtzLiBJIGhhdmUgcmVhZCBpdC4gSSB3aWxsIGRlbGV0ZSBpdCBh bmQgcHJpbnQgdGhlIGVycm9yIGluZm8KaW4gdGhlIElTUi4gQWxzbyBjYWxsIHRoZSByZXBvcnRf aW9tbXVfZmF1bHQgaW4gdGhlIElTUi4KCj4gQmVzdCByZWdhcmRzLAo+IFRvbWFzegoKCl9fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmlvbW11IG1haWxpbmcg bGlzdAppb21tdUBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4 Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0aW5mby9pb21tdQ== From mboxrd@z Thu Jan 1 00:00:00 1970 From: yong.wu@mediatek.com (Yong Wu) Date: Tue, 14 Apr 2015 14:31:07 +0800 Subject: [PATCH 2/5] iommu/mediatek: Add mt8173 IOMMU driver In-Reply-To: References: <1425638900-24989-1-git-send-email-yong.wu@mediatek.com> <1425638900-24989-3-git-send-email-yong.wu@mediatek.com> <1426677749.22581.38.camel@mhfsdcap03> Message-ID: <1428993067.13552.14.camel@mhfsdcap03> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org Hi Tomasz, Thanks very much for you suggestion and explain so detail. please help check below. On Fri, 2015-03-27 at 18:41 +0900, Tomasz Figa wrote: > Hi Yong Wu, > > Sorry for long delay, I had to figure out some time to look at this again. > > On Wed, Mar 18, 2015 at 8:22 PM, Yong Wu wrote: > >> > >> > + imudev = piommu->dev; > >> > + > >> > + spin_lock_irqsave(&priv->portlock, flags); > >> > >> What is protected by this spinlock? > > We will write a register of the local arbiter while config port. If > > some modules are in the same local arbiter, it may be overwrite. so I > > add it here. > >> > > OK. Maybe it could be called larb_lock then? It would be good to have > structures or code that should be running under this spinlock > annotated with proper comments. And purpose of the lock documented in > a comment as well (probably in a kerneldoc-style documentation of > priv). Thanks. I have move the spinlock into the smi driver, it will lock for writing the local arbiter regsiter only. > > >> > +static void mtk_iommu_detach_device(struct iommu_domain *domain, > >> > + struct device *dev) > >> > +{ > >> > >> No hardware (de)configuration or clean-up necessary? > > I will add it. Actually we design like this:If a device have attached to > > iommu domain, it won't detach from it. > > Isn't proper clean-up required for module removal? Some drivers might > be required to be loadable modules, which should be unloadable. > > >> > >> > + > >> > + piommu->protect_va = devm_kmalloc(piommu->dev, MTK_PROTECT_PA_ALIGN*2, > >> > >> style: Operators like * should have space on both sides. > >> > >> > + GFP_KERNEL); > >> > >> Shouldn't dma_alloc_coherent() be used for this? > > We don't care the data in it. I think they are the same. Could you > > help tell me why dma_alloc_coherent may be better. > > Can you guarantee that at the time you allocate the memory using > devm_kmalloc() the memory is not dirty (i.e. some write back data are > stored in CPU cache) and is not going to be written back in some time, > overwriting data put there by IOMMU hardware? > As I noted in the function "mtk_iommu_hw_init": /* protect memory,HW will write here while translation fault */ protectpa = __virt_to_phys(piommu->protect_va); We don?t care the content of this buffer, It is ok even though its data is dirty. It seem to be a the protect memory. While a translation fault happened, The iommu HW will overwrite here instead of writing to the fault physical address which may be 0 or some random address. > >> > + > >> > + iommu_set_fault_handler(domain, mtk_iommu_fault_handler, piommu); > >> > >> I don't see any other drivers doing this. Isn't this for upper layers, > >> so that they can set their own generic fault handlers? > > I think that this function is related with the iommu domain, we > > have only one multimedia iommu domain. so I add it after the iommu > > domain are created. > > No, this function is for drivers of IOMMU clients (i.e. master IP > blocks) which want to subscribe to page fault to do things like paging > on demand and so on. It shouldn't be called by IOMMU driver. Please > see other IOMMU drivers, for example rockchip-iommmu.c. Thanks. I have read it. I will delete it and print the error info in the ISR. Also call the report_iommu_fault in the ISR. > Best regards, > Tomasz