Re: [PATCH v7 0/5] PM / devfreq: Add dynamic scaling for imx8m ddr controller

[Leonard Crestez <leonard.crestez@nxp.com>]

On 18.12.2019 15:35, Adam Ford wrote:
> On Fri, Nov 22, 2019 at 3:45 PM Leonard Crestez <leonard.crestez@nxp.com> wrote:
>>
>> This adds support for dynamic scaling of the DDR Controller (ddrc)
>> present on i.MX8M series chips. Actual frequency switching is
>> implemented inside TF-A, this driver wraps the SMC calls and
>> synchronizes the clk tree.
>>
>> DRAM frequency switching requires clock manipulation but during this operation
>> DRAM itself is briefly inaccessible so this operation is performed a SMC call
>> to by TF-A which runs from a SRAM area. Upon returning to linux the clock tree
>> is updated to correspond to hardware configuration.
>>
>> This is handled via CLK_GET_RATE_NO_CACHE for dividers but muxes are handled
>> manually: the driver will prepare/enable the new parents ahead of switching (so
>> that the expected roots are enabled) and afterwards it will call clk_set_parent
>> to ensure the parents in clock framework are up-to-date.
>>
>> This series is useful standalone and roughly similar to devfreq drivers for
>> tegra and rockchip.
>>
>> Running at lower dram rates saves power but can affect the functionality of
>> other blocks in the chip (display, vpu etc). Support for in-kernel constraints
>> will some separately.
>>
>> This series has no dependencies outside linux-next. The driver depends
>> on features from the NXP branch of TF-A and will cleanly fail to probe
>> on mainline. There are also plans to upstream dram dvfs in TF-A.
>>
>> Changes since v6:
>> * Replace ARCH_MXC || COMPILE_TEST with ARCH_MXC && HAVE_ARM_SMCCC
>> * Collect reviews
>> Link to v6: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.kernel.org%2Fcover%2F11244283%2F&amp;data=02%7C01%7Cleonard.crestez%40nxp.com%7Cb7adb366b79f4c564c7908d783bf23ae%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637122729275120308&amp;sdata=ZEZnG6pijjj4MObC99c6%2BvC8BFEfT1KLVxbJCNocoWw%3D&amp;reserved=0
>>
>> I'd rather not fix COMPILE_TEST with ifdefs for this driver, if anything
>> that should be fixed in smccc header. ARCH_MXC doesn't imply SMCCC, it
>> also covers some very old chips which don't have it.
>>
>> Resending full series because that's the standard method.
>>
>> Changes since v5:
>> * Fix a dram_apb/dram_alt mixup in imx8m_ddrc_set_freq
>> * Make clk_get_parent_by_index static (kbuild robot)
>> * Adjust messages in imx8m_ddrc_set_freq
>> * Use a for loop inside imx8m_ddrc_check_opps instead of while
>> * More elaborate description in dt-bindings file.
>> Link to v5: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.kernel.org%2Fcover%2F11240289%2F&amp;data=02%7C01%7Cleonard.crestez%40nxp.com%7Cb7adb366b79f4c564c7908d783bf23ae%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637122729275130306&amp;sdata=mMqglPQign%2B6NHgmDYoZ74%2FZeThGI6%2FgNkajo1VaHTI%3D&amp;reserved=0
>>
>> Changes since v4:
>> * Restore empty _get_dev_status: testing shows this is *NOT* optional. If
>> absent then switching to simple_ondemand governor will trigger an Oops.
>> * Keep clk registration on single-line in clk-imx8m* for consistency with rest
>> of the file.
>> * Drop explicit "select PM_OPP"
>> * Check for NULL new_dram_core_parent
>> * Rename "out_dis_" labels to out_disable_*
>> * Use dev_warn on imx8m_ddrc_set_freq error paths after SMC call (where
>> operation is not abandoned).
>> * More elaborate error messages in imx8m_ddrc_target
>> * More elaborate checks when fetching clks in imx8m_ddrc_set_freq
>> * Rename ddrc nodes to memory-controller@* as per devicetree.org "Generic Names
>> Recommendation"
>> * Defer perf support, it requires perf changes to fetch PMU by DT
>> Link to v4: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.kernel.org%2Fcover%2F11235685%2F&amp;data=02%7C01%7Cleonard.crestez%40nxp.com%7Cb7adb366b79f4c564c7908d783bf23ae%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637122729275130306&amp;sdata=LXG4bo0l7FOttZIolJE73CK67AAAW72xfx8yq3Vld7o%3D&amp;reserved=0
>>
>> Changes since v3:
>> * Rename to imx8m-ddrc. Similar blocks are present on imx7d and imx8qxp/imx8qm
>> but soc integration is different.
>> * Move dt bindings to /memory-controllers/fsl/
>> * Fix dt validation issues
>> * Fix imx8mm.dtsi ddrc referencing ddrc_opp_table which is only defined in evk
>> * Move opps to child of ddrc device node
>> * Only add imx_ddrc_get_dev_status in perf patch.
>> * Adjust print messages
>> Link to v3: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.kernel.org%2Fcover%2F11221935%2F&amp;data=02%7C01%7Cleonard.crestez%40nxp.com%7Cb7adb366b79f4c564c7908d783bf23ae%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637122729275130306&amp;sdata=%2FFGddgm7jq87j7tz6gNd%2B7V%2BGX4KF5RaPsnXK2kITdQ%3D&amp;reserved=0
>>
>> Changes since v2:
>> * Add support for entire imx8m family including imx8mq B0.
>> * Also mark dram PLLs as CLK_GET_RATE_NO_CACHE (required for imx8mq b0 low OPP)
>> * Explicitly update dram pll rate at the end of imx_ddrc_set_freq.
>> * Use do_div in imx-ddrc (kbuild robot)
>> * Improve explanations around adding CLK_GET_RATE_NO_CACHE to dram clks.
>> (Stephen Boyd)
>> * Handle ddrc devfreq-events earlier for fewer probe defers.
>> * Validate DDRC opp tables versus firmware: supported OPPs depend on board and
>> SOC revision.
>> * Move DDRC opp tables to board dts because they can vary based on ram type on
>> board.
>> * Verify DDRC rate is changed in clk tree and otherwise report an error.
>> * Change imx_ddrc_freq.rate to be measure in MT/s and round down from HZ in
>> imx_ddrc_find_freq instead.
>> * Split away from NOC scaling and interconnect support.
>> Link to v2: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.kernel.org%2Fcover%2F11104113%2F&amp;data=02%7C01%7Cleonard.crestez%40nxp.com%7Cb7adb366b79f4c564c7908d783bf23ae%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637122729275130306&amp;sdata=3hh0dR1h4Esc6qo79QQKH%2FkUQjrOUgLANR0PcIz1Pss%3D&amp;reserved=0
>>
>> Changes since v1:
>> * bindings: Stop using "contains" for "compatible"
>> * bindings: Set "additionalProperties: false" and document missing stuff.
>> * Remove (c) from NXP copyright notice
>> * Fix various checkpatch issues
>> * Remove unused dram_alt_root clk from imx-ddrc
>> Link to v1: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.kernel.org%2Fcover%2F11090649%2F&amp;data=02%7C01%7Cleonard.crestez%40nxp.com%7Cb7adb366b79f4c564c7908d783bf23ae%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637122729275130306&amp;sdata=Hj6NEmoaRHoR%2BQpFoUDdTlybO%2FSTatO2fFo20UGLJf0%3D&amp;reserved=0
>>
>> Leonard Crestez (5):
>>    clk: imx8m: Set CLK_GET_RATE_NOCACHE on dram clocks
>>    clk: imx: Mark dram pll on 8mm and 8mn with CLK_GET_RATE_NOCACHE
>>    dt-bindings: memory: Add bindings for imx8m ddr controller
>>    PM / devfreq: Add dynamic scaling for imx8m ddr controller
>>    arm64: dts: imx8m: Add ddr controller nodes
>>
>>   .../memory-controllers/fsl/imx8m-ddrc.yaml    |  72 +++
>>   arch/arm64/boot/dts/freescale/imx8mm-evk.dts  |  18 +
>>   arch/arm64/boot/dts/freescale/imx8mm.dtsi     |  10 +
>>   .../boot/dts/freescale/imx8mn-ddr4-evk.dts    |  18 +
>>   arch/arm64/boot/dts/freescale/imx8mn.dtsi     |  10 +
>>   arch/arm64/boot/dts/freescale/imx8mq-evk.dts  |  24 +
>>   arch/arm64/boot/dts/freescale/imx8mq.dtsi     |  10 +
>>   drivers/clk/imx/clk-imx8mm.c                  |  11 +-
>>   drivers/clk/imx/clk-imx8mn.c                  |  12 +-
>>   drivers/clk/imx/clk-imx8mq.c                  |  12 +-
>>   drivers/clk/imx/clk-pll14xx.c                 |   7 +
>>   drivers/clk/imx/clk.h                         |   1 +
>>   drivers/devfreq/Kconfig                       |   9 +
> 
> Since there is a Kconfig change, should there me a defconfig change?

Yes, you need to enable CONFIG_ARM_IMX8M_DDRC_DEVFREQ in order to test 
this. Enabling as "m" should work.

>>   drivers/devfreq/Makefile                      |   1 +
>>   drivers/devfreq/imx8m-ddrc.c                  | 465 ++++++++++++++++++
>>   15 files changed, 670 insertions(+), 10 deletions(-)
>>   create mode 100644 Documentation/devicetree/bindings/memory-controllers/fsl/imx8m-ddrc.yaml
>>   create mode 100644 drivers/devfreq/imx8m-ddrc.c
> 
> I applied the whole series against 5.5-rc1 and I am trying to test it.
> I know the 4.14 kernel NXP posted on Code Aurora is capable to
> lowering the DDRC controller to 25MHz on the 8MM when the video is
> off.  Since there is no video support yet for the 8MM, I was expecting
> to see the DDRC clock to be at or around 25MHz.
> 
> Using debug FS, I can see the dram core clock is still running at
> 750MHz, and measuring power, it shows something consistent with what I
> see on the Code Aurora kernel with video turned on and the clock at
> 750MHz.
> 
> Is there some way to get the dram_core_clk to drop to 25MHz to see
> some power reduction?  The same commands used in the Yocto build don't
> apply here since we don't have video.

Current upstream driver just keeps current frequency by default. Try the 
following:

cd /sys/class/devfreq/devices/devfreq0
echo userspace > governor
echo 25000000 > userspace/set_freq
cat /sys/kernel/debug/clk/dram_core_clk/clk_rate
echo 750000000 > userspace/set_freq
cat /sys/kernel/debug/clk/dram_core_clk/clk_rate

--
Regards,
Leonard