From: Rob Clark <robdclark@gmail.com>
To: Vivek Gautam <vivek.gautam@codeaurora.org>
Cc: Stephen Boyd <sboyd@codeaurora.org>,
Robin Murphy <robin.murphy@arm.com>,
Will Deacon <will.deacon@arm.com>,
"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
Sricharan R <sricharan@codeaurora.org>,
Joerg Roedel <joro@8bytes.org>, Rob Herring <robh+dt@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Marek Szyprowski <m.szyprowski@samsung.com>,
"iommu@lists.linux-foundation.org"
<iommu@lists.linux-foundation.org>,
"devicetree@vger.kernel.org" <devicetree@vger.kernel.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
linux-clk <linux-clk@vger.kernel.org>,
linux-arm-msm <linux-arm-msm@vger.kernel.org>,
Stanimir Varbanov <stanimir.varbanov@linaro.org>,
Archit Taneja <architt@codeaurora.org>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>
Subject: Re: [PATCH V4 3/6] iommu/arm-smmu: Invoke pm_runtime during probe, add/remove device
Date: Tue, 28 Nov 2017 15:05:14 -0500 [thread overview]
Message-ID: <CAF6AEGu3nZiyNN+KjRmKm57fPOvT+qHUPNJzs=84EWEU+CDcCA@mail.gmail.com> (raw)
In-Reply-To: <3a2f74e9-90cf-d843-d801-15eb614d7abe@codeaurora.org>
On Tue, Nov 28, 2017 at 8:43 AM, Vivek Gautam
<vivek.gautam@codeaurora.org> wrote:
>
>
> On 11/28/2017 05:13 AM, Rob Clark wrote:
>>
>> On Mon, Nov 27, 2017 at 5:22 PM, Stephen Boyd<sboyd@codeaurora.org>
>> wrote:
>>>
>>> On 11/15, Vivek Gautam wrote:
>>>>
>>>> Hi,
>>>>
>>>>
>>>> On Mon, Aug 7, 2017 at 5:59 PM, Rob Clark<robdclark@gmail.com> wrote:
>>>>>
>>>>> On Mon, Aug 7, 2017 at 4:27 AM, Vivek Gautam
>>>>> <vivek.gautam@codeaurora.org> wrote:
>>>>>>
>>>>>> On Thu, Jul 13, 2017 at 5:20 PM, Rob Clark<robdclark@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> On Thu, Jul 13, 2017 at 1:35 AM, Sricharan
>>>>>>> R<sricharan@codeaurora.org> wrote:
>>>>>>>>
>>>>>>>> Hi Vivek,
>>>>>>>>
>>>>>>>> On 7/13/2017 10:43 AM, Vivek Gautam wrote:
>>>>>>>>>
>>>>>>>>> Hi Stephen,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 07/13/2017 04:24 AM, Stephen Boyd wrote:
>>>>>>>>>>
>>>>>>>>>> On 07/06, Vivek Gautam wrote:
>>>>>>>>>>>
>>>>>>>>>>> @@ -1231,12 +1237,18 @@ static int arm_smmu_map(struct
>>>>>>>>>>> iommu_domain *domain, unsigned long iova,
>>>>>>>>>>> static size_t arm_smmu_unmap(struct iommu_domain *domain,
>>>>>>>>>>> unsigned long iova,
>>>>>>>>>>> size_t size)
>>>>>>>>>>> {
>>>>>>>>>>> - struct io_pgtable_ops *ops =
>>>>>>>>>>> to_smmu_domain(domain)->pgtbl_ops;
>>>>>>>>>>> + struct arm_smmu_domain *smmu_domain =
>>>>>>>>>>> to_smmu_domain(domain);
>>>>>>>>>>> + struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
>>>>>>>>>>> + size_t ret;
>>>>>>>>>>> if (!ops)
>>>>>>>>>>> return 0;
>>>>>>>>>>> - return ops->unmap(ops, iova, size);
>>>>>>>>>>> + pm_runtime_get_sync(smmu_domain->smmu->dev);
>>>>>>>>>>
>>>>>>>>>> Can these map/unmap ops be called from an atomic context? I seem
>>>>>>>>>> to recall that being a problem before.
>>>>>>>>>
>>>>>>>>> That's something which was dropped in the following patch merged in
>>>>>>>>> master:
>>>>>>>>> 523d7423e21b iommu/arm-smmu: Remove io-pgtable spinlock
>>>>>>>>>
>>>>>>>>> Looks like we don't need locks here anymore?
>>>>>>>>
>>>>>>>> Apart from the locking, wonder why a explicit pm_runtime is needed
>>>>>>>> from unmap. Somehow looks like some path in the master using that
>>>>>>>> should have enabled the pm ?
>>>>>>>>
>>>>>>> Yes, there are a bunch of scenarios where unmap can happen with
>>>>>>> disabled master (but not in atomic context).
>>>>>>
>>>>>> I would like to understand whether there is a situation where an unmap
>>>>>> is
>>>>>> called in atomic context without an enabled master?
>>>>>>
>>>>>> Let's say we have the case where all the unmap calls in atomic context
>>>>>> happen
>>>>>> only from the master's context (in which case the device link should
>>>>>> take care of
>>>>>> the pm state of smmu), and the only unmap that happen in non-atomic
>>>>>> context
>>>>>> is the one with master disabled. In such a case doesn it make sense to
>>>>>> distinguish
>>>>>> the atomic/non-atomic context and add pm_runtime_get_sync()/put_sync()
>>>>>> only
>>>>>> for the non-atomic context since that would be the one with master
>>>>>> disabled.
>>>>>>
>>>>> At least drm/msm needs to hold obj->lock (a mutex) in unmap, so it
>>>>> won't unmap anything in atomic ctx (but it can unmap w/ master
>>>>> disabled). I can't really comment about other non-gpu drivers. It
>>>>> seems like a reasonable constraint that either master is enabled or
>>>>> not in atomic ctx.
>>>>>
>>>>> Currently we actually wrap unmap w/ pm_runtime_get/put_sync(), but I'd
>>>>> like to drop that to avoid powering up the gpu.
>>>>
>>>> Since the deferring the TLB maintenance doesn't look like the best
>>>> approach [1],
>>>> how about if we try to power-up only the smmu from different client
>>>> devices such as,
>>>> GPU in the unmap path. Then we won't need to add pm_runtime_get/put()
>>>> calls in
>>>> arm_smmu_unmap().
>>>>
>>>> The client device can use something like - pm_runtime_get_supplier()
>>>> since
>>>> we already have the device link in place with this patch series. This
>>>> should
>>>> power-on the supplier (which is smmu) without turning on the consumer
>>>> (such as GPU).
>>>>
>>>> pm_runtime_get_supplier() however is not exported at this moment.
>>>> Will it be useful to export this API and use it in the drivers.
>>>>
>>> I'm not sure pm_runtime_get_supplier() is correct either. That
>>> feels like we're relying on the GPU driver knowing the internal
>>> details of how the device links are configured.
>>>
>> what does pm_runtime_get_supplier() do if IOMMU driver hasn't setup
>> device-link?
>
>
> It will be a no-op.
>
>> If it is a no-op, then I guess the GPU driver calling
>> pm_runtime_get_supplier() seems reasonable, and less annoying than
>> having special cases in pm_resume path.. I don't feel too bad about
>> having "just in case" get/put_supplier() calls in the unmap path.
>>
>> Also, presumably we still want to avoid powering up GPU even if we
>> short circuit the firmware loading and rest of "booting up the GPU"..
>> since presumably the GPU draws somewhat more power than the IOMMU..
>> having the pm_resume/suspend path know about the diff between waking
>> up / suspending the iommu and itself doesn't really feel less-bad than
>> just doing "just in case" get/put_supplier() calls.
>
>
> If it sounds okay, then i can send a patch that exports the
> pm_runtime_get/put_suppliers() APIs.
>
sounds good to me
BR,
-R
>
> Best regards
> Vivek
>
>> BR,
>> -R
>>
>>> Is there some way to have the GPU driver know in its runtime PM
>>> resume hook that it doesn't need to be powered on because it
>>> isn't actively drawing anything or processing commands? I'm
>>> thinking of the code calling pm_runtime_get() as proposed around
>>> the IOMMU unmap path in the GPU driver and then having the
>>> runtime PM resume hook in the GPU driver return some special
>>> value to indicate that it didn't really resume because it didn't
>>> need to and to treat the device as runtime suspended but not
>>> return an error. Then the runtime PM core can keep track of that
>>> and try to power the GPU on again when another pm_runtime_get()
>>> is called on the GPU device.
>>>
>>> This keeps the consumer API the same, always pm_runtime_get(),
>>> but leaves the device driver logic of what to do when the GPU
>>> doesn't need to power on to the runtime PM hook where the driver
>>> has all the information.
>>>
>>> --
>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
>>> a Linux Foundation Collaborative Project
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm"
>> in
>> the body of a message tomajordomo@vger.kernel.org
>> More majordomo info athttp://vger.kernel.org/majordomo-info.html
>
>
> --
> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
>
> a Linux Foundation Collaborative Project
>
next prev parent reply other threads:[~2017-11-28 20:05 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-06 9:36 [PATCH V4 0/6] iommu/arm-smmu: Add runtime pm/sleep support Vivek Gautam
2017-07-06 9:37 ` [PATCH V4 1/6] iommu/arm-smmu: Fix the error path in arm_smmu_add_device Vivek Gautam
2017-07-06 9:37 ` [PATCH V4 2/6] iommu/arm-smmu: Add pm_runtime/sleep ops Vivek Gautam
2017-07-12 22:58 ` Stephen Boyd
2017-07-12 23:01 ` Stephen Boyd
2017-07-13 3:57 ` Vivek Gautam
2017-07-06 9:37 ` [PATCH V4 3/6] iommu/arm-smmu: Invoke pm_runtime during probe, add/remove device Vivek Gautam
2017-07-12 22:54 ` Stephen Boyd
2017-07-13 5:13 ` Vivek Gautam
2017-07-13 5:35 ` Sricharan R
2017-07-13 11:50 ` Rob Clark
2017-07-13 12:02 ` Marek Szyprowski
2017-07-13 12:10 ` Rob Clark
2017-07-13 12:23 ` Marek Szyprowski
2017-07-13 13:53 ` Sricharan R
2017-07-13 14:55 ` Rob Clark
2017-07-14 17:07 ` Will Deacon
2017-07-14 17:42 ` Rob Clark
2017-07-14 18:06 ` Will Deacon
2017-07-14 18:25 ` Rob Clark
2017-07-14 19:01 ` Will Deacon
2017-07-14 19:34 ` Rob Clark
2017-07-14 19:36 ` Will Deacon
2017-07-14 19:39 ` Rob Clark
2017-07-17 11:46 ` Sricharan R
2017-07-17 12:28 ` Sricharan R
2017-07-24 15:31 ` Vivek Gautam
2017-08-02 9:53 ` [PATCH] iommu/arm-smmu: Defer TLB flush in case of unmap op Vivek Gautam
2017-08-02 12:17 ` Robin Murphy
2017-08-03 5:35 ` Vivek Gautam
2017-08-04 17:04 ` Robin Murphy
2017-08-07 7:44 ` Vivek Gautam
2017-08-07 8:27 ` [PATCH V4 3/6] iommu/arm-smmu: Invoke pm_runtime during probe, add/remove device Vivek Gautam
2017-08-07 12:29 ` Rob Clark
2017-11-14 18:30 ` Vivek Gautam
2017-11-27 22:22 ` Stephen Boyd
2017-11-27 23:43 ` Rob Clark
2017-11-28 13:43 ` Vivek Gautam
2017-11-28 20:05 ` Rob Clark [this message]
2017-07-13 13:57 ` Vivek Gautam
2017-07-13 14:01 ` Vivek Gautam
2017-07-13 6:48 ` Stephen Boyd
2017-07-13 9:50 ` Robin Murphy
2017-07-13 11:53 ` Rob Clark
2017-07-06 9:37 ` [PATCH V4 4/6] iommu/arm-smmu: Add the device_link between masters and smmu Vivek Gautam
2017-07-12 22:55 ` Stephen Boyd
2017-07-13 3:59 ` Vivek Gautam
2017-07-06 9:37 ` [PATCH V4 5/6] iommu/arm-smmu: Add support for MMU40x/500 clocks Vivek Gautam
2017-07-10 3:37 ` Rob Herring
2017-07-11 5:18 ` Vivek Gautam
2017-07-06 9:37 ` [PATCH V4 6/6] iommu/arm-smmu: Add support for qcom,msm8996-smmu-v2 clocks Vivek Gautam
2017-07-10 3:40 ` Rob Herring
2017-07-10 6:42 ` Vivek Gautam
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAF6AEGu3nZiyNN+KjRmKm57fPOvT+qHUPNJzs=84EWEU+CDcCA@mail.gmail.com' \
--to=robdclark@gmail.com \
--cc=architt@codeaurora.org \
--cc=devicetree@vger.kernel.org \
--cc=iommu@lists.linux-foundation.org \
--cc=joro@8bytes.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-clk@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=m.szyprowski@samsung.com \
--cc=mark.rutland@arm.com \
--cc=rafael.j.wysocki@intel.com \
--cc=robh+dt@kernel.org \
--cc=robin.murphy@arm.com \
--cc=sboyd@codeaurora.org \
--cc=sricharan@codeaurora.org \
--cc=stanimir.varbanov@linaro.org \
--cc=vivek.gautam@codeaurora.org \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).