From mboxrd@z Thu Jan 1 00:00:00 1970 From: Robin Murphy Subject: Re: [PATCH v16 2/5] iommu/arm-smmu: Invoke pm_runtime during probe, add/remove device Date: Tue, 25 Sep 2018 19:55:33 +0100 Message-ID: References: <20180830144541.17740-1-vivek.gautam@codeaurora.org> <20180830144541.17740-3-vivek.gautam@codeaurora.org> <3ccc3690-dc9d-56e7-e2d1-62e73a189bff@codeaurora.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: Content-Language: en-GB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: freedreno-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "Freedreno" To: Vivek Gautam , Will Deacon Cc: Mark Rutland , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , Linux PM , sboyd-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, linux-arm-msm , Joerg Roedel , "Rafael J. Wysocki" , open list , iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, Tomasz Figa , alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, robh+dt , freedreno List-Id: linux-arm-msm@vger.kernel.org SGkgVml2ZWssCgpPbiAyMDE4LTA5LTI1IDY6NTYgQU0sIFZpdmVrIEdhdXRhbSB3cm90ZToKPiBI aSBSb2JpbiwgV2lsbCwKPiAKPiBPbiBUdWUsIFNlcCAxOCwgMjAxOCBhdCA4OjQxIEFNIFZpdmVr IEdhdXRhbQo+IDx2aXZlay5nYXV0YW1AY29kZWF1cm9yYS5vcmc+IHdyb3RlOgo+Pgo+PiBIaSBS b2JpbiwKPj4KPj4gT24gRnJpLCBTZXAgNywgMjAxOCBhdCAzOjUyIFBNIFZpdmVrIEdhdXRhbSA8 dml2ZWsuZ2F1dGFtQGNvZGVhdXJvcmEub3JnPiB3cm90ZToKPj4+Cj4+PiBPbiBGcmksIFNlcCA3 LCAyMDE4IGF0IDM6MjIgUE0gVG9tYXN6IEZpZ2EgPHRmaWdhQGNocm9taXVtLm9yZz4gd3JvdGU6 Cj4+Pj4KPj4+PiBPbiBGcmksIFNlcCA3LCAyMDE4IGF0IDY6MzggUE0gVml2ZWsgR2F1dGFtIDx2 aXZlay5nYXV0YW1AY29kZWF1cm9yYS5vcmc+IHdyb3RlOgo+Pj4+Pgo+Pj4+PiBIaSBUb21hc3os Cj4+Pj4+Cj4+Pj4+Cj4+Pj4+IE9uIDkvNy8yMDE4IDI6NDYgUE0sIFRvbWFzeiBGaWdhIHdyb3Rl Ogo+Pj4+Pj4gSGkgVml2ZWssCj4+Pj4+Pgo+Pj4+Pj4gT24gVGh1LCBBdWcgMzAsIDIwMTggYXQg MTE6NDYgUE0gVml2ZWsgR2F1dGFtCj4+Pj4+PiA8dml2ZWsuZ2F1dGFtQGNvZGVhdXJvcmEub3Jn PiB3cm90ZToKPj4+Pj4+PiBGcm9tOiBTcmljaGFyYW4gUiA8c3JpY2hhcmFuQGNvZGVhdXJvcmEu b3JnPgo+Pj4+Pj4+Cj4+Pj4+Pj4gVGhlIHNtbXUgZGV2aWNlIHByb2JlL3JlbW92ZSBhbmQgYWRk L3JlbW92ZSBtYXN0ZXIgZGV2aWNlIGNhbGxiYWNrcwo+Pj4+Pj4+IGdldHMgY2FsbGVkIHdoZW4g dGhlIHNtbXUgaXMgbm90IGxpbmtlZCB0byBpdHMgbWFzdGVyLCB0aGF0IGlzIHdpdGhvdXQKPj4+ Pj4+PiB0aGUgY29udGV4dCBvZiB0aGUgbWFzdGVyIGRldmljZS4gU28gY2FsbGluZyBydW50aW1l IGFwaXMgaW4gdGhvc2UgcGxhY2VzCj4+Pj4+Pj4gc2VwYXJhdGVseS4KPj4+Pj4+PiBHbG9iYWwg bG9ja3MgYXJlIGFsc28gaW5pdGlhbGl6ZWQgYmVmb3JlIGVuYWJsaW5nIHJ1bnRpbWUgcG0gYXMg dGhlCj4+Pj4+Pj4gcnVudGltZV9yZXN1bWUoKSBjYWxscyBkZXZpY2VfcmVzZXQoKSB3aGljaCBk b2VzIHRsYl9zeW5jX2dsb2JhbCgpCj4+Pj4+Pj4gdGhhdCB1bHRpbWF0ZWx5IHJlcXVpcmVzIGxv Y2tzIHRvIGJlIGluaXRpYWxpemVkLgo+Pj4+Pj4+Cj4+Pj4+Pj4gU2lnbmVkLW9mZi1ieTogU3Jp Y2hhcmFuIFIgPHNyaWNoYXJhbkBjb2RlYXVyb3JhLm9yZz4KPj4+Pj4+PiBbdml2ZWs6IENsZWFu dXAgcG0gcnVudGltZSBjYWxsc10KPj4+Pj4+PiBTaWduZWQtb2ZmLWJ5OiBWaXZlayBHYXV0YW0g PHZpdmVrLmdhdXRhbUBjb2RlYXVyb3JhLm9yZz4KPj4+Pj4+PiBSZXZpZXdlZC1ieTogVG9tYXN6 IEZpZ2EgPHRmaWdhQGNocm9taXVtLm9yZz4KPj4+Pj4+PiBUZXN0ZWQtYnk6IFNyaW5pdmFzIEth bmRhZ2F0bGEgPHNyaW5pdmFzLmthbmRhZ2F0bGFAbGluYXJvLm9yZz4KPj4+Pj4+PiAtLS0KPj4+ Pj4+PiAgICBkcml2ZXJzL2lvbW11L2FybS1zbW11LmMgfCA4OSArKysrKysrKysrKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrLS0tLS0KPj4+Pj4+PiAgICAxIGZpbGUgY2hhbmdlZCwg ODEgaW5zZXJ0aW9ucygrKSwgOCBkZWxldGlvbnMoLSkKPj4+Pj4+IFtzbmlwXQo+Pj4+Pj4+IEBA IC0yMjE1LDEwICsyMjgxLDE3IEBAIHN0YXRpYyBpbnQgYXJtX3NtbXVfZGV2aWNlX3JlbW92ZShz dHJ1Y3QgcGxhdGZvcm1fZGV2aWNlICpwZGV2KQo+Pj4+Pj4+ICAgICAgICAgICBpZiAoIWJpdG1h cF9lbXB0eShzbW11LT5jb250ZXh0X21hcCwgQVJNX1NNTVVfTUFYX0NCUykpCj4+Pj4+Pj4gICAg ICAgICAgICAgICAgICAgZGV2X2VycigmcGRldi0+ZGV2LCAicmVtb3ZpbmcgZGV2aWNlIHdpdGgg YWN0aXZlIGRvbWFpbnMhXG4iKTsKPj4+Pj4+Pgo+Pj4+Pj4+ICsgICAgICAgYXJtX3NtbXVfcnBt X2dldChzbW11KTsKPj4+Pj4+PiAgICAgICAgICAgLyogVHVybiB0aGUgdGhpbmcgb2ZmICovCj4+ Pj4+Pj4gICAgICAgICAgIHdyaXRlbChzQ1IwX0NMSUVOVFBELCBBUk1fU01NVV9HUjBfTlMoc21t dSkgKyBBUk1fU01NVV9HUjBfc0NSMCk7Cj4+Pj4+Pj4gKyAgICAgICBhcm1fc21tdV9ycG1fcHV0 KHNtbXUpOwo+Pj4+Pj4+ICsKPj4+Pj4+PiArICAgICAgIGlmIChwbV9ydW50aW1lX2VuYWJsZWQo c21tdS0+ZGV2KSkKPj4+Pj4+PiArICAgICAgICAgICAgICAgcG1fcnVudGltZV9mb3JjZV9zdXNw ZW5kKHNtbXUtPmRldik7Cj4+Pj4+Pj4gKyAgICAgICBlbHNlCj4+Pj4+Pj4gKyAgICAgICAgICAg ICAgIGNsa19idWxrX2Rpc2FibGUoc21tdS0+bnVtX2Nsa3MsIHNtbXUtPmNsa3MpOwo+Pj4+Pj4+ Cj4+Pj4+Pj4gLSAgICAgICBjbGtfYnVsa19kaXNhYmxlX3VucHJlcGFyZShzbW11LT5udW1fY2xr cywgc21tdS0+Y2xrcyk7Cj4+Pj4+Pj4gKyAgICAgICBjbGtfYnVsa191bnByZXBhcmUoc21tdS0+ bnVtX2Nsa3MsIHNtbXUtPmNsa3MpOwo+Pj4+Pj4gQXJlbid0IHdlIG1pc3NpbmcgcG1fcnVudGlt ZV9kaXNhYmxlKCkgaGVyZT8gV2UnbGwgaGF2ZSB0aGUgZW5hYmxlCj4+Pj4+PiBjb3VudCB1bmJh bGFuY2VkIGlmIHRoZSBkcml2ZXIgaXMgcmVtb3ZlZCBhbmQgcHJvYmVkIGFnYWluLgo+Pj4+Pgo+ Pj4+PiBwbV9ydW50aW1lX2ZvcmNlX3N1c3BlbmQoKSBkb2VzIGEgcG1fcnVudGltZV9kaXNhYmxl KCkgYWxzbyBpZiBpIGFtIG5vdAo+Pj4+PiB3cm9uZy4KPj4+Pj4gQW5kLCBhcyBtZW50aW9uZWQg aW4gYSBwcmV2aW91cyB0aHJlYWQgWzFdLCB3ZSB3ZXJlIHNlZWluZyBhIHdhcm5pbmcKPj4+Pj4g d2hpY2ggd2UgYXZvaWRlZAo+Pj4+PiBieSBrZWVwaW5nIGZvcmNlX3N1c3BlbmQoKS4KPj4+Pj4K Pj4+Pj4gWzFdIGh0dHBzOi8vbGttbC5vcmcvbGttbC8yMDE4LzcvOC8xMjQKPj4+Pgo+Pj4+IEkg c2VlLCB0aGFua3MuIEkgZGlkbid0IHJlYWxpemUgdGhhdCBwbV9ydW50aW1lX2ZvcmNlX3N1c3Bl bmQoKQo+Pj4+IGFscmVhZHkgZGlzYWJsZXMgcnVudGltZSBQTSBpbmRlZWQuIFNvcnJ5IGZvciB0 aGUgbm9pc2UuCj4+Pgo+Pj4gSGkgVG9tYXN6LAo+Pj4gTm8gcHJvYmxlbS4gVGhhbmtzIGZvciBs b29raW5nIGJhY2sgYXQgaXQuCj4+Pgo+Pj4gSGkgUm9iaW4sCj4+PiBJZiB5b3UgYXJlIGZpbmUg d2l0aCB0aGlzIHNlcmllcywgdGhlbiBjYW4geW91IHBsZWFzZSBjb25zaWRlciBnaXZpbmcKPj4+ IFJldmlld2VkLWJ5LCBzbyB0aGF0IHdlIGFyZSBjZXJ0YWluIHRoYXQgdGhpcyBzZXJpZXMgd2ls bCBnbyBpbiB0aGUgbmV4dCBtZXJnZQo+Pj4gd2luZG93Lgo+Pj4gVGhhbmtzCj4+Cj4+IEdlbnRs ZSBwaW5nLgo+PiBZb3UgYWNrIHdpbGwgYmUgdmVyeSBoZWxwZnVsIGluIGxldHRpbmcgV2lsbCBw dWxsIHRoaXMgc2VyaWVzIGZvciA0LjIwLgo+PiBUaGFua3MuCj4gCj4gSSB3b3VsZCByZWFsbHkg YXBwcmVjaWF0ZSBpZiB5b3UgY291bGQgcHJvdmlkZSB5b3VyIGFjayBmb3IgdGhpcyBzZXJpZXMu Cj4gT3IgaWYgdGhlcmUgYXJlIGFueSBjb25jZXJucywgSSBhbSB3aWxsaW5nIHRvIGFkZHJlc3Mg dGhlbS4KCkFwb2xvZ2llcywgSSB0aG91Z2h0IEknZCByZXBsaWVkIHRvIHNheSBJJ2QgYmUgZ2V0 dGluZyB0byB0aGlzIHNob3J0bHksIApidXQgYXBwYXJlbnRseSBub3QgOigKCkZXSVcsICJzaG9y dGx5IiBpcyBub3cgdG9tb3Jyb3cgLSBJIGRvbid0ICp0aGluayogdGhlcmUncyBhbnl0aGluZyAK b3V0c3RhbmRpbmcsIGJ1dCBnaXZlbiB0aGUgbnVtYmVyIG9mIHN1YnRsZXRpZXMgd2UndmUgdHVy bmVkIHVwIHNvIGZhciBJIApkbyBqdXN0IHdhbnQgb25lIGxhc3QgdGhvcm91Z2ggZG91YmxlLWNo ZWNrIHRvIG1ha2Ugc3VyZS4KClRoYW5rcywKUm9iaW4uCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fCkZyZWVkcmVubyBtYWlsaW5nIGxpc3QKRnJlZWRyZW5v QGxpc3RzLmZyZWVkZXNrdG9wLm9yZwpodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWls bWFuL2xpc3RpbmZvL2ZyZWVkcmVubwo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03D8BC43382 for ; Tue, 25 Sep 2018 18:55:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AF12F2086B for ; Tue, 25 Sep 2018 18:55:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AF12F2086B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727700AbeIZBEr (ORCPT ); Tue, 25 Sep 2018 21:04:47 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:57252 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727053AbeIZBEr (ORCPT ); Tue, 25 Sep 2018 21:04:47 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DCE4CED1; Tue, 25 Sep 2018 11:55:51 -0700 (PDT) Received: from [192.168.1.123] (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EE7163F5D3; Tue, 25 Sep 2018 11:55:47 -0700 (PDT) Subject: Re: [PATCH v16 2/5] iommu/arm-smmu: Invoke pm_runtime during probe, add/remove device To: Vivek Gautam , Will Deacon Cc: Mark Rutland , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , alex.williamson@redhat.com, Linux PM , sboyd@kernel.org, "Rafael J. Wysocki" , open list , "list@263.net:IOMMU DRIVERS" , Joerg Roedel , iommu@lists.linux-foundation.org, robh+dt , linux-arm-msm , freedreno , Tomasz Figa References: <20180830144541.17740-1-vivek.gautam@codeaurora.org> <20180830144541.17740-3-vivek.gautam@codeaurora.org> <3ccc3690-dc9d-56e7-e2d1-62e73a189bff@codeaurora.org> From: Robin Murphy Message-ID: Date: Tue, 25 Sep 2018 19:55:33 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Vivek, On 2018-09-25 6:56 AM, Vivek Gautam wrote: > Hi Robin, Will, > > On Tue, Sep 18, 2018 at 8:41 AM Vivek Gautam > wrote: >> >> Hi Robin, >> >> On Fri, Sep 7, 2018 at 3:52 PM Vivek Gautam wrote: >>> >>> On Fri, Sep 7, 2018 at 3:22 PM Tomasz Figa wrote: >>>> >>>> On Fri, Sep 7, 2018 at 6:38 PM Vivek Gautam wrote: >>>>> >>>>> Hi Tomasz, >>>>> >>>>> >>>>> On 9/7/2018 2:46 PM, Tomasz Figa wrote: >>>>>> Hi Vivek, >>>>>> >>>>>> On Thu, Aug 30, 2018 at 11:46 PM Vivek Gautam >>>>>> wrote: >>>>>>> From: Sricharan R >>>>>>> >>>>>>> The smmu device probe/remove and add/remove master device callbacks >>>>>>> gets called when the smmu is not linked to its master, that is without >>>>>>> the context of the master device. So calling runtime apis in those places >>>>>>> separately. >>>>>>> Global locks are also initialized before enabling runtime pm as the >>>>>>> runtime_resume() calls device_reset() which does tlb_sync_global() >>>>>>> that ultimately requires locks to be initialized. >>>>>>> >>>>>>> Signed-off-by: Sricharan R >>>>>>> [vivek: Cleanup pm runtime calls] >>>>>>> Signed-off-by: Vivek Gautam >>>>>>> Reviewed-by: Tomasz Figa >>>>>>> Tested-by: Srinivas Kandagatla >>>>>>> --- >>>>>>> drivers/iommu/arm-smmu.c | 89 +++++++++++++++++++++++++++++++++++++++++++----- >>>>>>> 1 file changed, 81 insertions(+), 8 deletions(-) >>>>>> [snip] >>>>>>> @@ -2215,10 +2281,17 @@ static int arm_smmu_device_remove(struct platform_device *pdev) >>>>>>> if (!bitmap_empty(smmu->context_map, ARM_SMMU_MAX_CBS)) >>>>>>> dev_err(&pdev->dev, "removing device with active domains!\n"); >>>>>>> >>>>>>> + arm_smmu_rpm_get(smmu); >>>>>>> /* Turn the thing off */ >>>>>>> writel(sCR0_CLIENTPD, ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sCR0); >>>>>>> + arm_smmu_rpm_put(smmu); >>>>>>> + >>>>>>> + if (pm_runtime_enabled(smmu->dev)) >>>>>>> + pm_runtime_force_suspend(smmu->dev); >>>>>>> + else >>>>>>> + clk_bulk_disable(smmu->num_clks, smmu->clks); >>>>>>> >>>>>>> - clk_bulk_disable_unprepare(smmu->num_clks, smmu->clks); >>>>>>> + clk_bulk_unprepare(smmu->num_clks, smmu->clks); >>>>>> Aren't we missing pm_runtime_disable() here? We'll have the enable >>>>>> count unbalanced if the driver is removed and probed again. >>>>> >>>>> pm_runtime_force_suspend() does a pm_runtime_disable() also if i am not >>>>> wrong. >>>>> And, as mentioned in a previous thread [1], we were seeing a warning >>>>> which we avoided >>>>> by keeping force_suspend(). >>>>> >>>>> [1] https://lkml.org/lkml/2018/7/8/124 >>>> >>>> I see, thanks. I didn't realize that pm_runtime_force_suspend() >>>> already disables runtime PM indeed. Sorry for the noise. >>> >>> Hi Tomasz, >>> No problem. Thanks for looking back at it. >>> >>> Hi Robin, >>> If you are fine with this series, then can you please consider giving >>> Reviewed-by, so that we are certain that this series will go in the next merge >>> window. >>> Thanks >> >> Gentle ping. >> You ack will be very helpful in letting Will pull this series for 4.20. >> Thanks. > > I would really appreciate if you could provide your ack for this series. > Or if there are any concerns, I am willing to address them. Apologies, I thought I'd replied to say I'd be getting to this shortly, but apparently not :( FWIW, "shortly" is now tomorrow - I don't *think* there's anything outstanding, but given the number of subtleties we've turned up so far I do just want one last thorough double-check to make sure. Thanks, Robin.