From: Doug Anderson <dianders@chromium.org>
To: Brian Norris <briannorris@chromium.org>
Cc: MyungJoo Ham <myungjoo.ham@samsung.com>,
Kyungmin Park <kyungmin.park@samsung.com>,
Chanwoo Choi <cw00.choi@samsung.com>,
Heiko Stuebner <heiko@sntech.de>,
LKML <linux-kernel@vger.kernel.org>,
Elaine Zhang <zhangqing@rock-chips.com>,
Linux PM <linux-pm@vger.kernel.org>,
Linux ARM <linux-arm-kernel@lists.infradead.org>,
"open list:ARM/Rockchip SoC..."
<linux-rockchip@lists.infradead.org>
Subject: Re: [RFC PATCH 1/2] soc: rockchip: power-domain: Manage resource conflicts with firmware
Date: Tue, 5 Apr 2022 19:26:22 -0700 [thread overview]
Message-ID: <CAD=FV=Xz+KLySeUBHe1CXEbMrjeT97HvCOdiRRG7wGcwbjQX8A@mail.gmail.com> (raw)
In-Reply-To: <20220405184816.RFC.1.Ib865f199d15221eab4ff77f70bd7e9e2eb04d32f@changeid>
Hi,
On Tue, Apr 5, 2022 at 6:49 PM Brian Norris <briannorris@chromium.org> wrote:
>
> On RK3399 platforms, power domains are managed mostly by the kernel
> (drivers/soc/rockchip/pm_domains.c), but there are a few exceptions
> where ARM Trusted Firmware has to be involved:
>
> (1) system suspend/resume
> (2) DRAM DVFS (a.k.a., "ddrfreq")
>
> Exception (1) does not cause much conflict, since the kernel has
> quiesced itself by the time we make the relevant PSCI call.
>
> Exception (2) can cause conflict, because of two actions:
>
> (a) ARM Trusted Firmware needs to read/modify/write the PMU_BUS_IDLE_REQ
> register to idle the memory controller domain; the kernel driver
> also has to touch this register for other domains.
> (b) ARM Trusted Firmware needs to manage the clocks associated with
> these domains.
>
> To elaborate on (b): idling a power domain has always required ungating
> an array of clocks; see this old explanation from Rockchip:
> https://lore.kernel.org/linux-arm-kernel/54503C19.9060607@rock-chips.com/
>
> Historically, ARM Trusted Firmware has avoided this issue by using a
> special PMU_CRU_GATEDIS_CON0 register -- this register ungates all the
> necessary clocks -- when idling the memory controller. Unfortunately,
> we've found that this register is not 100% sufficient; it does not turn
> the relevant PLLs on [0].
>
> So it's possible to trigger issues with something like the following:
>
> 1. enable a power domain (e.g., RK3399_PD_VDU) -- kernel will
> temporarily enable relevant clocks/PLLs, then turn them back off
> 2. a PLL (e.g., PLL_NPLL) is part of the clock tree for
> RK3399_PD_VDU's clocks but otherwise unused; NPLL is disabled
> 3. perform a ddrfreq transition (rk3399_dmcfreq_target() -> ...
> drivers/clk/rockchip/clk-ddr.c / ROCKCHIP_SIP_DRAM_FREQ)
> 4. ARM Trusted Firmware unagates VDU clocks (via PMU_CRU_GATEDIS_CON0)
> 5. ARM Trusted firmware idles the memory controller domain
> 6. Step 5 waits on the VDU domain/clocks, but NPLL is still off
>
> i.e., we hang the system.
>
> So for (b), we need to at a minimum manage the relevant PLLs on behalf
> of firmware. It's easier to simply manage the whole clock tree, in a
> similar way we do in rockchip_pd_power().
>
> For (a), we need to provide mutual exclusion betwen rockchip_pd_power()
> and firmware. To resolve that, we simply grab the PMU mutex and release
> it when ddrfreq is done.
>
> The Chromium OS kernel has been carrying versions of part of this hack
> for a while, based on some new custom notifiers [1]. I've rewritten as a
> simple function call between the drivers, which is OK because:
>
> * the PMU driver isn't enabled, and we don't have this problem at all
> (the firmware should have left us in an OK state, and there are no
> runtime conflicts); or
> * the PMU driver is present, and is a single instance.
>
> And the power-domain driver cannot be removed, so there's no lifetime
> management to worry about.
>
> For completeness, there's a 'dmc_pmu_mutex' to guard (likely
> theoretical?) probe()-time races. It's OK for the memory controller
> driver to start running before the PMU, because the PMU will avoid any
> critical actions during the block() sequence.
>
> [0] The RK3399 TRM for PMU_CRU_GATEDIS_CON0 only talks about ungating
> clocks. Based on experimentation, we've found that it does not power
> up the necessary PLLs.
>
> [1] CHROMIUM: soc: rockchip: power-domain: Add notifier to dmc driver
> https://chromium-review.googlesource.com/q/I242dbd706d352f74ff706f5cbf42ebb92f9bcc60
> Notably, the Chromium solution only handled conflict (a), not (b).
> In practice, item (b) wasn't a problem in many cases because we
> never managed to fully power off PLLs. Now that the (upstream) video
> decoder driver performs runtime clock management, we often power off
> NPLL.
>
> Signed-off-by: Brian Norris <briannorris@chromium.org>
> ---
>
> drivers/soc/rockchip/pm_domains.c | 118 ++++++++++++++++++++++++++++++
> include/soc/rockchip/pm_domains.h | 25 +++++++
> 2 files changed, 143 insertions(+)
I've already done several pre-review of a few versions of this, so at
this point I'm pretty happy with where things are. Feel free to add:
Reviewed-by: Douglas Anderson <dianders@chromium.org>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
next prev parent reply other threads:[~2022-04-06 2:26 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-06 1:48 [RFC PATCH 0/2] rockchip / devfreq: Coordinate DRAM controller resources between ATF and kernel Brian Norris
2022-04-06 1:48 ` [RFC PATCH 1/2] soc: rockchip: power-domain: Manage resource conflicts with firmware Brian Norris
2022-04-06 2:26 ` Doug Anderson [this message]
2022-04-07 5:04 ` Chanwoo Choi
2022-04-09 3:34 ` Brian Norris
2022-04-12 16:01 ` Peter Geis
2022-04-13 22:14 ` Chanwoo Choi
2022-05-08 15:05 ` Heiko Stuebner
2022-04-06 1:48 ` [RFC PATCH 2/2] PM / devfreq: rk3399_dmc: Block PMU during transitions Brian Norris
2022-04-06 2:26 ` Doug Anderson
2022-04-13 22:14 ` Chanwoo Choi
2022-04-13 22:45 ` Heiko Stübner
2022-04-13 23:13 ` Chanwoo Choi
2022-05-07 14:21 ` Chanwoo Choi
2022-05-08 15:07 ` Heiko Stuebner
2022-05-08 18:42 ` Chanwoo Choi
2022-05-08 18:40 ` [RFC PATCH 0/2] rockchip / devfreq: Coordinate DRAM controller resources between ATF and kernel Chanwoo Choi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAD=FV=Xz+KLySeUBHe1CXEbMrjeT97HvCOdiRRG7wGcwbjQX8A@mail.gmail.com' \
--to=dianders@chromium.org \
--cc=briannorris@chromium.org \
--cc=cw00.choi@samsung.com \
--cc=heiko@sntech.de \
--cc=kyungmin.park@samsung.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=linux-rockchip@lists.infradead.org \
--cc=myungjoo.ham@samsung.com \
--cc=zhangqing@rock-chips.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).