From mboxrd@z Thu Jan 1 00:00:00 1970 From: Vivek Gautam Subject: Re: [PATCH v16 2/5] iommu/arm-smmu: Invoke pm_runtime during probe, add/remove device Date: Fri, 7 Sep 2018 15:52:06 +0530 Message-ID: References: <20180830144541.17740-1-vivek.gautam@codeaurora.org> <20180830144541.17740-3-vivek.gautam@codeaurora.org> <3ccc3690-dc9d-56e7-e2d1-62e73a189bff@codeaurora.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: freedreno-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "Freedreno" To: Tomasz Figa , Robin Murphy Cc: Mark Rutland , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , Linux PM , sboyd-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, Will Deacon , "Rafael J. Wysocki" , open list , "list-Y9sIeH5OGRo@public.gmane.org:IOMMU DRIVERS , Joerg Roedel , " , alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, robh+dt , linux-arm-msm , freedreno List-Id: linux-arm-msm@vger.kernel.org T24gRnJpLCBTZXAgNywgMjAxOCBhdCAzOjIyIFBNIFRvbWFzeiBGaWdhIDx0ZmlnYUBjaHJvbWl1 bS5vcmc+IHdyb3RlOgo+Cj4gT24gRnJpLCBTZXAgNywgMjAxOCBhdCA2OjM4IFBNIFZpdmVrIEdh dXRhbSA8dml2ZWsuZ2F1dGFtQGNvZGVhdXJvcmEub3JnPiB3cm90ZToKPiA+Cj4gPiBIaSBUb21h c3osCj4gPgo+ID4KPiA+IE9uIDkvNy8yMDE4IDI6NDYgUE0sIFRvbWFzeiBGaWdhIHdyb3RlOgo+ ID4gPiBIaSBWaXZlaywKPiA+ID4KPiA+ID4gT24gVGh1LCBBdWcgMzAsIDIwMTggYXQgMTE6NDYg UE0gVml2ZWsgR2F1dGFtCj4gPiA+IDx2aXZlay5nYXV0YW1AY29kZWF1cm9yYS5vcmc+IHdyb3Rl Ogo+ID4gPj4gRnJvbTogU3JpY2hhcmFuIFIgPHNyaWNoYXJhbkBjb2RlYXVyb3JhLm9yZz4KPiA+ ID4+Cj4gPiA+PiBUaGUgc21tdSBkZXZpY2UgcHJvYmUvcmVtb3ZlIGFuZCBhZGQvcmVtb3ZlIG1h c3RlciBkZXZpY2UgY2FsbGJhY2tzCj4gPiA+PiBnZXRzIGNhbGxlZCB3aGVuIHRoZSBzbW11IGlz IG5vdCBsaW5rZWQgdG8gaXRzIG1hc3RlciwgdGhhdCBpcyB3aXRob3V0Cj4gPiA+PiB0aGUgY29u dGV4dCBvZiB0aGUgbWFzdGVyIGRldmljZS4gU28gY2FsbGluZyBydW50aW1lIGFwaXMgaW4gdGhv c2UgcGxhY2VzCj4gPiA+PiBzZXBhcmF0ZWx5Lgo+ID4gPj4gR2xvYmFsIGxvY2tzIGFyZSBhbHNv IGluaXRpYWxpemVkIGJlZm9yZSBlbmFibGluZyBydW50aW1lIHBtIGFzIHRoZQo+ID4gPj4gcnVu dGltZV9yZXN1bWUoKSBjYWxscyBkZXZpY2VfcmVzZXQoKSB3aGljaCBkb2VzIHRsYl9zeW5jX2ds b2JhbCgpCj4gPiA+PiB0aGF0IHVsdGltYXRlbHkgcmVxdWlyZXMgbG9ja3MgdG8gYmUgaW5pdGlh bGl6ZWQuCj4gPiA+Pgo+ID4gPj4gU2lnbmVkLW9mZi1ieTogU3JpY2hhcmFuIFIgPHNyaWNoYXJh bkBjb2RlYXVyb3JhLm9yZz4KPiA+ID4+IFt2aXZlazogQ2xlYW51cCBwbSBydW50aW1lIGNhbGxz XQo+ID4gPj4gU2lnbmVkLW9mZi1ieTogVml2ZWsgR2F1dGFtIDx2aXZlay5nYXV0YW1AY29kZWF1 cm9yYS5vcmc+Cj4gPiA+PiBSZXZpZXdlZC1ieTogVG9tYXN6IEZpZ2EgPHRmaWdhQGNocm9taXVt Lm9yZz4KPiA+ID4+IFRlc3RlZC1ieTogU3Jpbml2YXMgS2FuZGFnYXRsYSA8c3Jpbml2YXMua2Fu ZGFnYXRsYUBsaW5hcm8ub3JnPgo+ID4gPj4gLS0tCj4gPiA+PiAgIGRyaXZlcnMvaW9tbXUvYXJt LXNtbXUuYyB8IDg5ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKyst LS0tLQo+ID4gPj4gICAxIGZpbGUgY2hhbmdlZCwgODEgaW5zZXJ0aW9ucygrKSwgOCBkZWxldGlv bnMoLSkKPiA+ID4gW3NuaXBdCj4gPiA+PiBAQCAtMjIxNSwxMCArMjI4MSwxNyBAQCBzdGF0aWMg aW50IGFybV9zbW11X2RldmljZV9yZW1vdmUoc3RydWN0IHBsYXRmb3JtX2RldmljZSAqcGRldikK PiA+ID4+ICAgICAgICAgIGlmICghYml0bWFwX2VtcHR5KHNtbXUtPmNvbnRleHRfbWFwLCBBUk1f U01NVV9NQVhfQ0JTKSkKPiA+ID4+ICAgICAgICAgICAgICAgICAgZGV2X2VycigmcGRldi0+ZGV2 LCAicmVtb3ZpbmcgZGV2aWNlIHdpdGggYWN0aXZlIGRvbWFpbnMhXG4iKTsKPiA+ID4+Cj4gPiA+ PiArICAgICAgIGFybV9zbW11X3JwbV9nZXQoc21tdSk7Cj4gPiA+PiAgICAgICAgICAvKiBUdXJu IHRoZSB0aGluZyBvZmYgKi8KPiA+ID4+ICAgICAgICAgIHdyaXRlbChzQ1IwX0NMSUVOVFBELCBB Uk1fU01NVV9HUjBfTlMoc21tdSkgKyBBUk1fU01NVV9HUjBfc0NSMCk7Cj4gPiA+PiArICAgICAg IGFybV9zbW11X3JwbV9wdXQoc21tdSk7Cj4gPiA+PiArCj4gPiA+PiArICAgICAgIGlmIChwbV9y dW50aW1lX2VuYWJsZWQoc21tdS0+ZGV2KSkKPiA+ID4+ICsgICAgICAgICAgICAgICBwbV9ydW50 aW1lX2ZvcmNlX3N1c3BlbmQoc21tdS0+ZGV2KTsKPiA+ID4+ICsgICAgICAgZWxzZQo+ID4gPj4g KyAgICAgICAgICAgICAgIGNsa19idWxrX2Rpc2FibGUoc21tdS0+bnVtX2Nsa3MsIHNtbXUtPmNs a3MpOwo+ID4gPj4KPiA+ID4+IC0gICAgICAgY2xrX2J1bGtfZGlzYWJsZV91bnByZXBhcmUoc21t dS0+bnVtX2Nsa3MsIHNtbXUtPmNsa3MpOwo+ID4gPj4gKyAgICAgICBjbGtfYnVsa191bnByZXBh cmUoc21tdS0+bnVtX2Nsa3MsIHNtbXUtPmNsa3MpOwo+ID4gPiBBcmVuJ3Qgd2UgbWlzc2luZyBw bV9ydW50aW1lX2Rpc2FibGUoKSBoZXJlPyBXZSdsbCBoYXZlIHRoZSBlbmFibGUKPiA+ID4gY291 bnQgdW5iYWxhbmNlZCBpZiB0aGUgZHJpdmVyIGlzIHJlbW92ZWQgYW5kIHByb2JlZCBhZ2Fpbi4K PiA+Cj4gPiBwbV9ydW50aW1lX2ZvcmNlX3N1c3BlbmQoKSBkb2VzIGEgcG1fcnVudGltZV9kaXNh YmxlKCkgYWxzbyBpZiBpIGFtIG5vdAo+ID4gd3JvbmcuCj4gPiBBbmQsIGFzIG1lbnRpb25lZCBp biBhIHByZXZpb3VzIHRocmVhZCBbMV0sIHdlIHdlcmUgc2VlaW5nIGEgd2FybmluZwo+ID4gd2hp Y2ggd2UgYXZvaWRlZAo+ID4gYnkga2VlcGluZyBmb3JjZV9zdXNwZW5kKCkuCj4gPgo+ID4gWzFd IGh0dHBzOi8vbGttbC5vcmcvbGttbC8yMDE4LzcvOC8xMjQKPgo+IEkgc2VlLCB0aGFua3MuIEkg ZGlkbid0IHJlYWxpemUgdGhhdCBwbV9ydW50aW1lX2ZvcmNlX3N1c3BlbmQoKQo+IGFscmVhZHkg ZGlzYWJsZXMgcnVudGltZSBQTSBpbmRlZWQuIFNvcnJ5IGZvciB0aGUgbm9pc2UuCgpIaSBUb21h c3osCk5vIHByb2JsZW0uIFRoYW5rcyBmb3IgbG9va2luZyBiYWNrIGF0IGl0LgoKSGkgUm9iaW4s CklmIHlvdSBhcmUgZmluZSB3aXRoIHRoaXMgc2VyaWVzLCB0aGVuIGNhbiB5b3UgcGxlYXNlIGNv bnNpZGVyIGdpdmluZwpSZXZpZXdlZC1ieSwgc28gdGhhdCB3ZSBhcmUgY2VydGFpbiB0aGF0IHRo aXMgc2VyaWVzIHdpbGwgZ28gaW4gdGhlIG5leHQgbWVyZ2UKd2luZG93LgpUaGFua3MKCkJlc3Qg cmVnYXJkcwpWaXZlawoKCgoKLS0gClFVQUxDT01NIElORElBLCBvbiBiZWhhbGYgb2YgUXVhbGNv bW0gSW5ub3ZhdGlvbiBDZW50ZXIsIEluYy4gaXMgYSBtZW1iZXIKb2YgQ29kZSBBdXJvcmEgRm9y dW0sIGhvc3RlZCBieSBUaGUgTGludXggRm91bmRhdGlvbgpfX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fXwpGcmVlZHJlbm8gbWFpbGluZyBsaXN0CkZyZWVkcmVu b0BsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFp bG1hbi9saXN0aW5mby9mcmVlZHJlbm8K From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS,T_DKIM_INVALID, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 102B2C4321E for ; Fri, 7 Sep 2018 10:22:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AE8D52083D for ; Fri, 7 Sep 2018 10:22:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="key not found in DNS" (0-bit key) header.d=codeaurora.org header.i=@codeaurora.org header.b="I1t++FzR"; dkim=fail reason="key not found in DNS" (0-bit key) header.d=codeaurora.org header.i=@codeaurora.org header.b="iv7+yLSj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AE8D52083D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728470AbeIGPCm (ORCPT ); Fri, 7 Sep 2018 11:02:42 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:35818 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727651AbeIGPCm (ORCPT ); Fri, 7 Sep 2018 11:02:42 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id D96F86085F; Fri, 7 Sep 2018 10:22:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1536315744; bh=QasiAzmxXre/E6Fwdx+gr7PwnYd8+a16b7uv7i3BwN4=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=I1t++FzRFB3FcnbS4GUNMs9qG3I1LCZevlTmq3jE2xatom1Ec0zs4IYETCb5IbJCT DvweAyItF7b7rM48PVMhgcd7qw1iZo6BhFpb0hivhjcb2RsAiHBacb4qtmPo71GbGS kRr85MjBcpYIV8dIHWrBm7F6S4Lef/rsoKgSHzhA= Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com [209.85.222.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: vivek.gautam@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id A577A608BF; Fri, 7 Sep 2018 10:22:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1536315740; bh=QasiAzmxXre/E6Fwdx+gr7PwnYd8+a16b7uv7i3BwN4=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=iv7+yLSjMtg9X9XonkVWmjJhOnL8Lh7AexTieZB7Zhgfu4UzFNdviAKSktgc3J110 G0P/T53RlsSRdjmAMZXIyGRDjp4D1Ey4uM9Oc8Jro10WPSllHTpFqeD/nUtyuEVGfJ t+p9/Z7nJufrEpTDGTH0TqLOngJ3NLhNWFIwUhKY= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org A577A608BF Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=vivek.gautam@codeaurora.org Received: by mail-qk1-f178.google.com with SMTP id g197-v6so9359812qke.5; Fri, 07 Sep 2018 03:22:20 -0700 (PDT) X-Gm-Message-State: APzg51DzIJbHIZEqZQa5HUPZHCAGxO1mTAwvuexCQ22WMDhY9/0AWHS8 zk9h+U/uXgiKUFaBDWblFy0XOcBRwqZIRs8HQmY= X-Google-Smtp-Source: ANB0VdZ6ul2fKt6rH1jjnDlsAzrkZbuYT7iyjoamhEF4G7WxyNZ6fJxa6u/oTP1rUtnBMXsYNFfdAAvj2Wl8ETMDJU8= X-Received: by 2002:a37:9904:: with SMTP id b4-v6mr5340996qke.102.1536315739645; Fri, 07 Sep 2018 03:22:19 -0700 (PDT) MIME-Version: 1.0 References: <20180830144541.17740-1-vivek.gautam@codeaurora.org> <20180830144541.17740-3-vivek.gautam@codeaurora.org> <3ccc3690-dc9d-56e7-e2d1-62e73a189bff@codeaurora.org> In-Reply-To: From: Vivek Gautam Date: Fri, 7 Sep 2018 15:52:06 +0530 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v16 2/5] iommu/arm-smmu: Invoke pm_runtime during probe, add/remove device To: Tomasz Figa , Robin Murphy Cc: Mark Rutland , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , alex.williamson@redhat.com, Linux PM , sboyd@kernel.org, "Rafael J. Wysocki" , Will Deacon , open list , "list@263.net:IOMMU DRIVERS , Joerg Roedel ," , "robh+dt" , linux-arm-msm , freedreno Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Sep 7, 2018 at 3:22 PM Tomasz Figa wrote: > > On Fri, Sep 7, 2018 at 6:38 PM Vivek Gautam wrote: > > > > Hi Tomasz, > > > > > > On 9/7/2018 2:46 PM, Tomasz Figa wrote: > > > Hi Vivek, > > > > > > On Thu, Aug 30, 2018 at 11:46 PM Vivek Gautam > > > wrote: > > >> From: Sricharan R > > >> > > >> The smmu device probe/remove and add/remove master device callbacks > > >> gets called when the smmu is not linked to its master, that is without > > >> the context of the master device. So calling runtime apis in those places > > >> separately. > > >> Global locks are also initialized before enabling runtime pm as the > > >> runtime_resume() calls device_reset() which does tlb_sync_global() > > >> that ultimately requires locks to be initialized. > > >> > > >> Signed-off-by: Sricharan R > > >> [vivek: Cleanup pm runtime calls] > > >> Signed-off-by: Vivek Gautam > > >> Reviewed-by: Tomasz Figa > > >> Tested-by: Srinivas Kandagatla > > >> --- > > >> drivers/iommu/arm-smmu.c | 89 +++++++++++++++++++++++++++++++++++++++++++----- > > >> 1 file changed, 81 insertions(+), 8 deletions(-) > > > [snip] > > >> @@ -2215,10 +2281,17 @@ static int arm_smmu_device_remove(struct platform_device *pdev) > > >> if (!bitmap_empty(smmu->context_map, ARM_SMMU_MAX_CBS)) > > >> dev_err(&pdev->dev, "removing device with active domains!\n"); > > >> > > >> + arm_smmu_rpm_get(smmu); > > >> /* Turn the thing off */ > > >> writel(sCR0_CLIENTPD, ARM_SMMU_GR0_NS(smmu) + ARM_SMMU_GR0_sCR0); > > >> + arm_smmu_rpm_put(smmu); > > >> + > > >> + if (pm_runtime_enabled(smmu->dev)) > > >> + pm_runtime_force_suspend(smmu->dev); > > >> + else > > >> + clk_bulk_disable(smmu->num_clks, smmu->clks); > > >> > > >> - clk_bulk_disable_unprepare(smmu->num_clks, smmu->clks); > > >> + clk_bulk_unprepare(smmu->num_clks, smmu->clks); > > > Aren't we missing pm_runtime_disable() here? We'll have the enable > > > count unbalanced if the driver is removed and probed again. > > > > pm_runtime_force_suspend() does a pm_runtime_disable() also if i am not > > wrong. > > And, as mentioned in a previous thread [1], we were seeing a warning > > which we avoided > > by keeping force_suspend(). > > > > [1] https://lkml.org/lkml/2018/7/8/124 > > I see, thanks. I didn't realize that pm_runtime_force_suspend() > already disables runtime PM indeed. Sorry for the noise. Hi Tomasz, No problem. Thanks for looking back at it. Hi Robin, If you are fine with this series, then can you please consider giving Reviewed-by, so that we are certain that this series will go in the next merge window. Thanks Best regards Vivek -- QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation