linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH 0/9] Apple SoC CPU P-state switching
@ 2021-10-11 16:56 Hector Martin
  2021-10-11 16:56 ` [RFC PATCH 1/9] MAINTAINERS: apple: Add apple-mcc and clk-apple-cluster paths Hector Martin
                   ` (8 more replies)
  0 siblings, 9 replies; 40+ messages in thread
From: Hector Martin @ 2021-10-11 16:56 UTC (permalink / raw)
  To: linux-arm-kernel
  Cc: Hector Martin, Alyssa Rosenzweig, Sven Peter, Marc Zyngier,
	Mark Kettenis, Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

Hi folks, here's a first attempt at cpufreq support for the Apple M1.
I'd appreciate any feedback on this approach.

The hardware in these SoCs is very simple: you just poke a single
register to change the performance state of a CPU cluster. There's
some init required on older firmware versions, but we're going to
say that's the bootloader's responsibility. This keeps the driver
nice and simple and generic and likely to work on future SoCs too.

However, there's a catch: the memory controller config should also be
modified when higher clock states are engaged on the P-cores, to
get optimal performance.

This series implements this using two drivers, on top of the existing
cpufreq-dt infrastructure. The cpu clock driver represents the p-state
switching as if it were a standard clock driver, so it can work with
cpufreq-dt. To do this, it also needs access to the OPP table, so it can
map the incoming clock frequences back to P-State index numbers, so that
is present in the binding. This might be a bit strange, since the same
OPP table is referenced by the CPUs themselves, and by the clocks driver
that provides the actual switching for them...

The memory controller stuff is implemented as a genpd provider that
exposes two performance states that the CPU OPP tables can depend on.
Unfortunately, the cpufreq path doesn't work properly for this, since
the CPUs aren't typical devices participating in runtime-pm. So instead
I opted to put that logic in the clock driver, which means it gets a
power-domains reference to the memory controller. This required a hack
to the OPP core so that it wouldn't complain about the missing parent
domain when evaluating the OPPs in the context of the CPUs...

The actual memory controller config is two magic numbers per performance
state. I'd love to find out what they do, but this seems unlikely
without docs or a deep memory performance analysis expedition... so I
think we're going to have to settle for this way, at least for now. If
things become better understood in the future, we can modify the binding
and keep the driver backwards-compatible with old DTs at least.

I did benchmark the CPU p-state switching, so the latency numbers there
have been experimentally measured. The CPU capacity numbers are also
based on real benchmarks (oddly enough, Dhrystone was a big outlier
here that was not representative of everything else, so we didn't use
it).

Patches:
 #1:   MAINTAINERS updates, split out so this can go through the SoC
       tree so we can spare all the subsystem maintainers the merge
       conflicts, since we have a bunch of other changes here going on
       in parallel.
 #2-3: DT bindings
 #4:   The aforementioned hack for the OPP core
 #5:   Add of_genpd_add_provider_simple_noclk()
 #6:   Memory controller driver
 #7:   CPU p-state clock driver
 #8:   Add some deps for ARCH_APPLE
 #9:   DT updates (for asahi-soc tree)

Hector Martin (9):
  MAINTAINERS: apple: Add apple-mcc and clk-apple-cluster paths
  dt-bindings: memory-controller: Add apple,mcc binding
  dt-bindings: clock: Add apple,cluster-clk binding
  opp: core: Don't warn if required OPP device does not exist
  PM: domains: Add of_genpd_add_provider_simple_noclk()
  memory: apple: Add apple-mcc driver to manage MCC perf in Apple SoCs
  clk: apple: Add clk-apple-cluster driver to manage CPU p-states
  arm64: apple: Select MEMORY and APPLE_MCC
  arm64: apple: Add CPU frequency scaling support for t8103

 .../bindings/clock/apple,cluster-clk.yaml     | 115 ++++++++
 .../memory-controllers/apple,mcc.yaml         |  80 ++++++
 .../opp/apple,mcc-operating-points.yaml       |  62 +++++
 MAINTAINERS                                   |   5 +
 arch/arm64/Kconfig.platforms                  |   2 +
 arch/arm64/boot/dts/apple/t8103.dtsi          | 255 +++++++++++++++++-
 drivers/base/power/domain.c                   |  39 ++-
 drivers/clk/Kconfig                           |   9 +
 drivers/clk/Makefile                          |   1 +
 drivers/clk/clk-apple-cluster.c               | 184 +++++++++++++
 drivers/memory/Kconfig                        |   9 +
 drivers/memory/Makefile                       |   1 +
 drivers/memory/apple-mcc.c                    | 130 +++++++++
 drivers/opp/core.c                            |   5 +-
 include/linux/pm_domain.h                     |   8 +
 15 files changed, 887 insertions(+), 18 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/clock/apple,cluster-clk.yaml
 create mode 100644 Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml
 create mode 100644 Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml
 create mode 100644 drivers/clk/clk-apple-cluster.c
 create mode 100644 drivers/memory/apple-mcc.c

-- 
2.33.0


^ permalink raw reply	[flat|nested] 40+ messages in thread

* [RFC PATCH 1/9] MAINTAINERS: apple: Add apple-mcc and clk-apple-cluster paths
  2021-10-11 16:56 [RFC PATCH 0/9] Apple SoC CPU P-state switching Hector Martin
@ 2021-10-11 16:56 ` Hector Martin
  2021-10-11 16:57 ` [RFC PATCH 2/9] dt-bindings: memory-controller: Add apple,mcc binding Hector Martin
                   ` (7 subsequent siblings)
  8 siblings, 0 replies; 40+ messages in thread
From: Hector Martin @ 2021-10-11 16:56 UTC (permalink / raw)
  To: linux-arm-kernel
  Cc: Hector Martin, Alyssa Rosenzweig, Sven Peter, Marc Zyngier,
	Mark Kettenis, Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

Splitting this out from the drivers, since we have enough things going
on in parallel that MAINTAINERS merges are going to cause pain for
subsystems. This will go through the SoC tree instead.

Signed-off-by: Hector Martin <marcan@marcan.st>
---
 MAINTAINERS | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index abdcbcfef73d..81ced53e291b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1719,10 +1719,15 @@ B:	https://github.com/AsahiLinux/linux/issues
 C:	irc://irc.oftc.net/asahi-dev
 T:	git https://github.com/AsahiLinux/linux.git
 F:	Documentation/devicetree/bindings/arm/apple.yaml
+F:	Documentation/devicetree/bindings/clock/apple,*
 F:	Documentation/devicetree/bindings/interrupt-controller/apple,aic.yaml
+F:	Documentation/devicetree/bindings/memory-controllers/apple,*
+F:	Documentation/devicetree/bindings/opp/apple,*
 F:	Documentation/devicetree/bindings/pinctrl/apple,pinctrl.yaml
 F:	arch/arm64/boot/dts/apple/
+F:	drivers/clk/clk-apple-cluster.c
 F:	drivers/irqchip/irq-apple-aic.c
+F:	drivers/memory/apple-mcc.c
 F:	include/dt-bindings/interrupt-controller/apple-aic.h
 F:	include/dt-bindings/pinctrl/apple.h
 
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 40+ messages in thread

* [RFC PATCH 2/9] dt-bindings: memory-controller: Add apple,mcc binding
  2021-10-11 16:56 [RFC PATCH 0/9] Apple SoC CPU P-state switching Hector Martin
  2021-10-11 16:56 ` [RFC PATCH 1/9] MAINTAINERS: apple: Add apple-mcc and clk-apple-cluster paths Hector Martin
@ 2021-10-11 16:57 ` Hector Martin
  2021-10-12  8:48   ` Krzysztof Kozlowski
  2021-10-11 16:57 ` [RFC PATCH 3/9] dt-bindings: clock: Add apple,cluster-clk binding Hector Martin
                   ` (6 subsequent siblings)
  8 siblings, 1 reply; 40+ messages in thread
From: Hector Martin @ 2021-10-11 16:57 UTC (permalink / raw)
  To: linux-arm-kernel
  Cc: Hector Martin, Alyssa Rosenzweig, Sven Peter, Marc Zyngier,
	Mark Kettenis, Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

This device represents the memory controller in Apple SoCs, and is
chiefly in charge of adjusting performance characteristics according to
system demand.

Signed-off-by: Hector Martin <marcan@marcan.st>
---
 .../memory-controllers/apple,mcc.yaml         | 80 +++++++++++++++++++
 .../opp/apple,mcc-operating-points.yaml       | 62 ++++++++++++++
 2 files changed, 142 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml
 create mode 100644 Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml

diff --git a/Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml b/Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml
new file mode 100644
index 000000000000..0774f10e65ed
--- /dev/null
+++ b/Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml
@@ -0,0 +1,80 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/memory-controllers/apple,mcc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Apple SoC MCC memory controller performance controls
+
+maintainers:
+  - Hector Martin <marcan@marcan.st>
+
+description: |
+  Apple SoCs contain a multichannel memory controller that can have its
+  configuration changed to adjust to changing performance requirements from
+  the rest of the SoC. This node represents the controller and provides a
+  power domain provider that downstream devices can use to adjust the memory
+  controller performance level.
+
+properties:
+  compatible:
+    items:
+      - enum:
+          - apple,t8103-mcc
+      - const: apple,mcc
+
+  reg:
+    maxItems: 1
+
+  "#power-domain-cells":
+    const: 0
+
+  operating-points-v2:
+    $ref: /schemas/types.yaml#/definitions/phandle-array
+    description:
+      A reference to the OPP table describing the memory controller performance
+      levels. Each OPP node should contain an `apple,memory-perf-config`
+      property that contains the configuration values for that performance
+      level.
+
+  apple,num-channels:
+    $ref: /schemas/types.yaml#/definitions/uint32
+    description:
+      The number of memory channels in use.
+
+required:
+  - compatible
+  - reg
+  - "#power-domain-cells"
+  - operating-points-v2
+  - apple,num-channels
+
+additionalProperties: false
+
+examples:
+  # See clock/apple,cluster-clock.yaml for an example of downstream usage.
+  - |
+    mcc_opp: opp-table-2 {
+        compatible = "operating-points-v2";
+
+        mcc_lowperf: opp0 {
+            opp-level = <0>;
+            apple,memory-perf-config = <0x813057f 0x1800180>;
+        };
+        mcc_highperf: opp1 {
+            opp-level = <1>;
+            apple,memory-perf-config = <0x133 0x55555340>;
+        };
+    };
+    soc {
+        #address-cells = <2>;
+        #size-cells = <2>;
+
+        mcc: memory-controller@200200000 {
+            compatible = "apple,t8103-mcc", "apple,mcc";
+            #power-domain-cells = <0>;
+            reg = <0x2 0x200000 0x0 0x200000>;
+            operating-points-v2 = <&mcc_opp>;
+            apple,num-channels = <8>;
+        };
+    };
diff --git a/Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml b/Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml
new file mode 100644
index 000000000000..babf27841bb7
--- /dev/null
+++ b/Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml
@@ -0,0 +1,62 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/opp/apple,mcc-operating-points.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Apple SoC memory controller OPP bindings
+
+maintainers:
+  - Hector Martin <marcan@marcan.st>
+
+description: |
+  Apple SoCs can have their memory controller performance adjusted depending on
+  system requirements. These performance states are represented by specific
+  memory controller register values. The apple-mcc driver uses these values
+  to change the MCC performance.
+
+allOf:
+  - $ref: opp-v2-base.yaml#
+
+properties:
+  compatible:
+    const: apple,mcc-operating-points
+
+required:
+  - compatible
+
+patternProperties:
+  "opp[0-9]+":
+    type: object
+
+    properties:
+      opp-level: true
+      apple,memory-perf-config:
+        $ref: /schemas/types.yaml#/definitions/uint32-array
+        description: |
+          A pair of register values used to configure this performance state.
+        minItems: 2
+        maxItems: 2
+
+    required:
+      - opp-level
+      - apple,memory-perf-config
+
+    unevaluatedProperties: false
+
+additionalProperties: false
+
+examples:
+  - |
+    mcc_opp: opp-table-2 {
+        compatible = "operating-points-v2";
+
+        mcc_lowperf: opp0 {
+            opp-level = <0>;
+            apple,memory-perf-config = <0x813057f 0x1800180>;
+        };
+        mcc_highperf: opp1 {
+            opp-level = <1>;
+            apple,memory-perf-config = <0x133 0x55555340>;
+        };
+    };
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 40+ messages in thread

* [RFC PATCH 3/9] dt-bindings: clock: Add apple,cluster-clk binding
  2021-10-11 16:56 [RFC PATCH 0/9] Apple SoC CPU P-state switching Hector Martin
  2021-10-11 16:56 ` [RFC PATCH 1/9] MAINTAINERS: apple: Add apple-mcc and clk-apple-cluster paths Hector Martin
  2021-10-11 16:57 ` [RFC PATCH 2/9] dt-bindings: memory-controller: Add apple,mcc binding Hector Martin
@ 2021-10-11 16:57 ` Hector Martin
  2021-10-12  8:51   ` Krzysztof Kozlowski
  2021-10-11 16:57 ` [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist Hector Martin
                   ` (5 subsequent siblings)
  8 siblings, 1 reply; 40+ messages in thread
From: Hector Martin @ 2021-10-11 16:57 UTC (permalink / raw)
  To: linux-arm-kernel
  Cc: Hector Martin, Alyssa Rosenzweig, Sven Peter, Marc Zyngier,
	Mark Kettenis, Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

This device represents the CPU performance state switching mechanism as
a clock controller, to be used with the standard cpufreq-dt
infrastructure.

Signed-off-by: Hector Martin <marcan@marcan.st>
---
 .../bindings/clock/apple,cluster-clk.yaml     | 115 ++++++++++++++++++
 1 file changed, 115 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/clock/apple,cluster-clk.yaml

diff --git a/Documentation/devicetree/bindings/clock/apple,cluster-clk.yaml b/Documentation/devicetree/bindings/clock/apple,cluster-clk.yaml
new file mode 100644
index 000000000000..9a8b863dadc0
--- /dev/null
+++ b/Documentation/devicetree/bindings/clock/apple,cluster-clk.yaml
@@ -0,0 +1,115 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/clock/apple,cluster-clk.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: CPU cluster frequency scaling for Apple SoCs
+
+maintainers:
+  - Hector Martin <marcan@marcan.st>
+
+description: |
+  Apple SoCs control CPU cluster frequencies by using a performance state
+  index. This node represents the feature as a clock controller, and uses
+  a reference to the CPU OPP table to translate clock frequencies into
+  performance states. This allows the CPUs to use the standard cpufreq-dt
+  mechanism for frequency scaling.
+
+properties:
+  compatible:
+    items:
+      - enum:
+          - apple,t8103-cluster-clk
+      - const: apple,cluster-clk
+
+  reg:
+    maxItems: 1
+
+  '#clock-cells':
+    const: 0
+
+  operating-points-v2:
+    $ref: /schemas/types.yaml#/definitions/phandle-array
+    description:
+      A reference to the OPP table used for the CPU cluster controlled by this
+      device instance. The table should contain an `opp-level` property for
+      every OPP, which represents the p-state index used by the hardware to
+      represent this performance level.
+
+      OPPs may also have a `required-opps` property (see power-domains).
+
+  power-domains:
+    maxItems: 1
+    description:
+      An optional reference to a power domain provider that links its
+      performance state to the CPU cluster performance state. This is typically
+      a memory controller. If set, the `required-opps` property in the CPU
+      frequency OPP nodes will be used to change the performance state of this
+      provider state in tandem with CPU frequency changes.
+
+required:
+  - compatible
+  - reg
+  - '#clock-cells'
+  - operating-points-v2
+
+additionalProperties: false
+
+
+examples:
+  - |
+    pcluster_opp: opp-table-1 {
+        compatible = "operating-points-v2";
+        opp-shared;
+
+        opp01 {
+            opp-hz = /bits/ 64 <600000000>;
+            opp-microvolt = <781000>;
+            opp-level = <1>;
+            clock-latency-ns = <8000>;
+            required-opps = <&mcc_lowperf>;
+        };
+        /* intermediate p-states omitted */
+        opp15 {
+            opp-hz = /bits/ 64 <3204000000>;
+            opp-microvolt = <1081000>;
+            opp-level = <15>;
+            clock-latency-ns = <56000>;
+            required-opps = <&mcc_highperf>;
+        };
+    };
+
+    mcc_opp: opp-table-2 {
+        compatible = "operating-points-v2";
+
+        mcc_lowperf: opp0 {
+            opp-level = <0>;
+            apple,memory-perf-config = <0x813057f 0x1800180>;
+        };
+        mcc_highperf: opp1 {
+            opp-level = <1>;
+            apple,memory-perf-config = <0x133 0x55555340>;
+        };
+    };
+
+    soc {
+        #address-cells = <2>;
+        #size-cells = <2>;
+
+        mcc: memory-controller@200200000 {
+            compatible = "apple,t8103-mcc", "apple,mcc";
+            #power-domain-cells = <0>;
+            reg = <0x2 0x200000 0x0 0x200000>;
+            operating-points-v2 = <&mcc_opp>;
+            apple,num-channels = <8>;
+        };
+
+        clk_pcluster: clock-controller@211e20000 {
+            compatible = "apple,t8103-cluster-clk", "apple,cluster-clk";
+            #clock-cells = <0>;
+            reg = <0x2 0x11e20000 0x0 0x4000>;
+            operating-points-v2 = <&pcluster_opp>;
+            power-domains = <&mcc>;
+        };
+    };
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 40+ messages in thread

* [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-11 16:56 [RFC PATCH 0/9] Apple SoC CPU P-state switching Hector Martin
                   ` (2 preceding siblings ...)
  2021-10-11 16:57 ` [RFC PATCH 3/9] dt-bindings: clock: Add apple,cluster-clk binding Hector Martin
@ 2021-10-11 16:57 ` Hector Martin
  2021-10-12  3:21   ` Viresh Kumar
  2021-10-11 16:57 ` [RFC PATCH 5/9] PM: domains: Add of_genpd_add_provider_simple_noclk() Hector Martin
                   ` (4 subsequent siblings)
  8 siblings, 1 reply; 40+ messages in thread
From: Hector Martin @ 2021-10-11 16:57 UTC (permalink / raw)
  To: linux-arm-kernel
  Cc: Hector Martin, Alyssa Rosenzweig, Sven Peter, Marc Zyngier,
	Mark Kettenis, Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

When required-opps is used in CPU OPP tables, there is no parent power
domain to drive it. Squelch this error, to allow a clock driver to
handle this directly instead.

Signed-off-by: Hector Martin <marcan@marcan.st>
---
 drivers/opp/core.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/opp/core.c b/drivers/opp/core.c
index 04b4691a8aac..89e616721f70 100644
--- a/drivers/opp/core.c
+++ b/drivers/opp/core.c
@@ -873,12 +873,13 @@ static int _set_required_opp(struct device *dev, struct device *pd_dev,
 		return 0;
 
 	ret = dev_pm_genpd_set_performance_state(pd_dev, pstate);
-	if (ret) {
+	if (ret && ret != -ENODEV) {
 		dev_err(dev, "Failed to set performance rate of %s: %d (%d)\n",
 			dev_name(pd_dev), pstate, ret);
+		return ret;
 	}
 
-	return ret;
+	return 0;
 }
 
 /* This is only called for PM domain for now */
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 40+ messages in thread

* [RFC PATCH 5/9] PM: domains: Add of_genpd_add_provider_simple_noclk()
  2021-10-11 16:56 [RFC PATCH 0/9] Apple SoC CPU P-state switching Hector Martin
                   ` (3 preceding siblings ...)
  2021-10-11 16:57 ` [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist Hector Martin
@ 2021-10-11 16:57 ` Hector Martin
  2021-10-11 16:57 ` [RFC PATCH 6/9] memory: apple: Add apple-mcc driver to manage MCC perf in Apple SoCs Hector Martin
                   ` (3 subsequent siblings)
  8 siblings, 0 replies; 40+ messages in thread
From: Hector Martin @ 2021-10-11 16:57 UTC (permalink / raw)
  To: linux-arm-kernel
  Cc: Hector Martin, Alyssa Rosenzweig, Sven Peter, Marc Zyngier,
	Mark Kettenis, Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

This is analogous to of_genpd_add_provider_simple(), but calls
dev_pm_opp_of_add_table_noclk() instead of dev_pm_opp_of_add_table().

Signed-off-by: Hector Martin <marcan@marcan.st>
---
 drivers/base/power/domain.c | 39 +++++++++++++++++++++++++++++--------
 include/linux/pm_domain.h   |  8 ++++++++
 2 files changed, 39 insertions(+), 8 deletions(-)

diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c
index 5db704f02e71..3377b9dd514c 100644
--- a/drivers/base/power/domain.c
+++ b/drivers/base/power/domain.c
@@ -2227,13 +2227,9 @@ static bool genpd_present(const struct generic_pm_domain *genpd)
 	return ret;
 }
 
-/**
- * of_genpd_add_provider_simple() - Register a simple PM domain provider
- * @np: Device node pointer associated with the PM domain provider.
- * @genpd: Pointer to PM domain associated with the PM domain provider.
- */
-int of_genpd_add_provider_simple(struct device_node *np,
-				 struct generic_pm_domain *genpd)
+static int _of_genpd_add_provider_simple(struct device_node *np,
+					 struct generic_pm_domain *genpd,
+					 bool getclk)
 {
 	int ret;
 
@@ -2247,7 +2243,10 @@ int of_genpd_add_provider_simple(struct device_node *np,
 
 	/* Parse genpd OPP table */
 	if (genpd->set_performance_state) {
-		ret = dev_pm_opp_of_add_table(&genpd->dev);
+		if (getclk)
+			ret = dev_pm_opp_of_add_table(&genpd->dev);
+		else
+			ret = dev_pm_opp_of_add_table_noclk(&genpd->dev, 0);
 		if (ret) {
 			if (ret != -EPROBE_DEFER)
 				dev_err(&genpd->dev, "Failed to add OPP table: %d\n",
@@ -2278,8 +2277,32 @@ int of_genpd_add_provider_simple(struct device_node *np,
 
 	return 0;
 }
+
+/**
+ * of_genpd_add_provider_simple() - Register a simple PM domain provider
+ * @np: Device node pointer associated with the PM domain provider.
+ * @genpd: Pointer to PM domain associated with the PM domain provider.
+ */
+int of_genpd_add_provider_simple(struct device_node *np,
+				 struct generic_pm_domain *genpd)
+{
+	return _of_genpd_add_provider_simple(np, genpd, true);
+}
 EXPORT_SYMBOL_GPL(of_genpd_add_provider_simple);
 
+/**
+ * of_genpd_add_provider_simple_noclk() - Register a simple clockless
+ *   PM domain provider
+ * @np: Device node pointer associated with the PM domain provider.
+ * @genpd: Pointer to PM domain associated with the PM domain provider.
+ */
+int of_genpd_add_provider_simple_noclk(struct device_node *np,
+				       struct generic_pm_domain *genpd)
+{
+	return _of_genpd_add_provider_simple(np, genpd, false);
+}
+EXPORT_SYMBOL_GPL(of_genpd_add_provider_simple_noclk);
+
 /**
  * of_genpd_add_provider_onecell() - Register a onecell PM domain provider
  * @np: Device node pointer associated with the PM domain provider.
diff --git a/include/linux/pm_domain.h b/include/linux/pm_domain.h
index 67017c9390c8..db3f75612512 100644
--- a/include/linux/pm_domain.h
+++ b/include/linux/pm_domain.h
@@ -316,6 +316,8 @@ struct genpd_onecell_data {
 #ifdef CONFIG_PM_GENERIC_DOMAINS_OF
 int of_genpd_add_provider_simple(struct device_node *np,
 				 struct generic_pm_domain *genpd);
+int of_genpd_add_provider_simple_noclk(struct device_node *np,
+				       struct generic_pm_domain *genpd);
 int of_genpd_add_provider_onecell(struct device_node *np,
 				  struct genpd_onecell_data *data);
 void of_genpd_del_provider(struct device_node *np);
@@ -342,6 +344,12 @@ static inline int of_genpd_add_provider_simple(struct device_node *np,
 	return -EOPNOTSUPP;
 }
 
+static inline int of_genpd_add_provider_simple_noclk(struct device_node *np,
+					struct generic_pm_domain *genpd)
+{
+	return -EOPNOTSUPP;
+}
+
 static inline int of_genpd_add_provider_onecell(struct device_node *np,
 					struct genpd_onecell_data *data)
 {
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 40+ messages in thread

* [RFC PATCH 6/9] memory: apple: Add apple-mcc driver to manage MCC perf in Apple SoCs
  2021-10-11 16:56 [RFC PATCH 0/9] Apple SoC CPU P-state switching Hector Martin
                   ` (4 preceding siblings ...)
  2021-10-11 16:57 ` [RFC PATCH 5/9] PM: domains: Add of_genpd_add_provider_simple_noclk() Hector Martin
@ 2021-10-11 16:57 ` Hector Martin
  2021-10-12  9:19   ` Krzysztof Kozlowski
  2021-10-11 16:57 ` [RFC PATCH 7/9] clk: apple: Add clk-apple-cluster driver to manage CPU p-states Hector Martin
                   ` (2 subsequent siblings)
  8 siblings, 1 reply; 40+ messages in thread
From: Hector Martin @ 2021-10-11 16:57 UTC (permalink / raw)
  To: linux-arm-kernel
  Cc: Hector Martin, Alyssa Rosenzweig, Sven Peter, Marc Zyngier,
	Mark Kettenis, Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

This driver binds to the memory controller hardware in Apple SoCs such
as the Apple M1, and provides a power domain that downstream devices can
use to change the performance state of the memory controller.

Signed-off-by: Hector Martin <marcan@marcan.st>
---
 drivers/memory/Kconfig     |   9 +++
 drivers/memory/Makefile    |   1 +
 drivers/memory/apple-mcc.c | 130 +++++++++++++++++++++++++++++++++++++
 3 files changed, 140 insertions(+)
 create mode 100644 drivers/memory/apple-mcc.c

diff --git a/drivers/memory/Kconfig b/drivers/memory/Kconfig
index 72c0df129d5c..48ef3d563a1c 100644
--- a/drivers/memory/Kconfig
+++ b/drivers/memory/Kconfig
@@ -30,6 +30,15 @@ config ARM_PL172_MPMC
 	  If you have an embedded system with an AMBA bus and a PL172
 	  controller, say Y or M here.
 
+config APPLE_MCC
+	tristate "Apple SoC MCC driver"
+	default y if ARCH_APPLE
+	select PM_GENERIC_DOMAINS
+	depends on ARCH_APPLE || COMPILE_TEST
+	help
+	  This driver manages performance tuning for the memory controller in
+	  Apple SoCs, such as the Apple M1.
+
 config ATMEL_SDRAMC
 	bool "Atmel (Multi-port DDR-)SDRAM Controller"
 	default y if ARCH_AT91
diff --git a/drivers/memory/Makefile b/drivers/memory/Makefile
index bc7663ed1c25..947840cbd2d4 100644
--- a/drivers/memory/Makefile
+++ b/drivers/memory/Makefile
@@ -8,6 +8,7 @@ ifeq ($(CONFIG_DDR),y)
 obj-$(CONFIG_OF)		+= of_memory.o
 endif
 obj-$(CONFIG_ARM_PL172_MPMC)	+= pl172.o
+obj-$(CONFIG_APPLE_MCC)		+= apple-mcc.o
 obj-$(CONFIG_ATMEL_SDRAMC)	+= atmel-sdramc.o
 obj-$(CONFIG_ATMEL_EBI)		+= atmel-ebi.o
 obj-$(CONFIG_BRCMSTB_DPFE)	+= brcmstb_dpfe.o
diff --git a/drivers/memory/apple-mcc.c b/drivers/memory/apple-mcc.c
new file mode 100644
index 000000000000..55959f034b9a
--- /dev/null
+++ b/drivers/memory/apple-mcc.c
@@ -0,0 +1,130 @@
+// SPDX-License-Identifier: GPL-2.0-only OR MIT
+/*
+ * Apple SoC MCC memory controller performance control driver
+ *
+ * Copyright The Asahi Linux Contributors
+ */
+
+#include <linux/delay.h>
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/pm_domain.h>
+#include <linux/pm_opp.h>
+
+#define APPLE_MCC_PERF_CONFIG1  0xdc4
+#define APPLE_MCC_PERF_CONFIG2  0xdbc
+#define APPLE_MCC_CHANNEL(x)	((x) * 0x40000)
+
+struct apple_mcc {
+	struct device *dev;
+	struct generic_pm_domain genpd;
+	void __iomem *reg_base;
+	u32 num_channels;
+};
+
+#define to_apple_mcc(_genpd) container_of(_genpd, struct apple_mcc, genpd)
+
+static int apple_mcc_set_performance_state(struct generic_pm_domain *genpd, unsigned int state)
+{
+	struct apple_mcc *mcc = to_apple_mcc(genpd);
+	struct dev_pm_opp *opp;
+	struct device_node *np;
+	u32 perf_config[2];
+	unsigned int i;
+
+	dev_dbg(mcc->dev, "switching to perf state %d\n", state);
+
+	opp = dev_pm_opp_find_level_exact(&mcc->genpd.dev, state);
+	if (IS_ERR(opp))
+		return PTR_ERR(opp);
+
+	np = dev_pm_opp_get_of_node(opp);
+	if (of_property_read_u32_array(np, "apple,memory-perf-config",
+		perf_config, ARRAY_SIZE(perf_config))) {
+		dev_err(mcc->dev, "missing apple,memory-perf-config property");
+		of_node_put(np);
+		return -EINVAL;
+	}
+	of_node_put(np);
+
+	for (i = 0; i < mcc->num_channels; i++) {
+		writel_relaxed(perf_config[0],
+			       mcc->reg_base + APPLE_MCC_CHANNEL(i) + APPLE_MCC_PERF_CONFIG1);
+		writel_relaxed(perf_config[1],
+			       mcc->reg_base + APPLE_MCC_CHANNEL(i) + APPLE_MCC_PERF_CONFIG2);
+	}
+
+	return 0;
+}
+
+static unsigned int apple_mcc_opp_to_performance_state(struct generic_pm_domain *genpd,
+						       struct dev_pm_opp *opp)
+{
+	return dev_pm_opp_get_level(opp);
+}
+
+static int apple_mcc_probe(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct device_node *node = dev->of_node;
+	struct apple_mcc *mcc;
+	int ret;
+
+	mcc = devm_kzalloc(dev, sizeof(*mcc), GFP_KERNEL);
+	if (!mcc)
+		return -ENOMEM;
+
+	mcc->reg_base = devm_platform_ioremap_resource(pdev, 0);
+	if (IS_ERR(mcc->reg_base))
+		return PTR_ERR(mcc->reg_base);
+
+	if (of_property_read_u32(node, "apple,num-channels", &mcc->num_channels)) {
+		dev_err(dev, "missing apple,num-channels property\n");
+		return -ENOENT;
+	}
+
+	mcc->dev = dev;
+	mcc->genpd.name = "apple-mcc-perf";
+	mcc->genpd.opp_to_performance_state = apple_mcc_opp_to_performance_state;
+	mcc->genpd.set_performance_state = apple_mcc_set_performance_state;
+
+	ret = pm_genpd_init(&mcc->genpd, NULL, false);
+	if (ret < 0) {
+		dev_err(dev, "pm_genpd_init failed\n");
+		return ret;
+	}
+
+	ret = of_genpd_add_provider_simple_noclk(node, &mcc->genpd);
+	if (ret < 0) {
+		dev_err(dev, "of_genpd_add_provider_simple failed\n");
+		return ret;
+	}
+
+	dev_info(dev, "Apple MCC performance driver initialized\n");
+
+	return 0;
+}
+
+static const struct of_device_id apple_mcc_of_match[] = {
+	{ .compatible = "apple,mcc" },
+	{}
+};
+
+MODULE_DEVICE_TABLE(of, apple_mcc_of_match);
+
+static struct platform_driver apple_mcc_driver = {
+	.probe = apple_mcc_probe,
+	.driver = {
+		.name = "apple-mcc",
+		.of_match_table = apple_mcc_of_match,
+	},
+};
+
+MODULE_AUTHOR("Hector Martin <marcan@marcan.st>");
+MODULE_DESCRIPTION("MCC memory controller performance tuning driver for Apple SoCs");
+MODULE_LICENSE("GPL v2");
+
+module_platform_driver(apple_mcc_driver);
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 40+ messages in thread

* [RFC PATCH 7/9] clk: apple: Add clk-apple-cluster driver to manage CPU p-states
  2021-10-11 16:56 [RFC PATCH 0/9] Apple SoC CPU P-state switching Hector Martin
                   ` (5 preceding siblings ...)
  2021-10-11 16:57 ` [RFC PATCH 6/9] memory: apple: Add apple-mcc driver to manage MCC perf in Apple SoCs Hector Martin
@ 2021-10-11 16:57 ` Hector Martin
       [not found]   ` <163424925931.1688384.9647104000291025081@swboyd.mtv.corp.google.com>
  2021-10-11 16:57 ` [RFC PATCH 8/9] arm64: apple: Select MEMORY and APPLE_MCC Hector Martin
  2021-10-11 16:57 ` [RFC PATCH 9/9] arm64: apple: Add CPU frequency scaling support for t8103 Hector Martin
  8 siblings, 1 reply; 40+ messages in thread
From: Hector Martin @ 2021-10-11 16:57 UTC (permalink / raw)
  To: linux-arm-kernel
  Cc: Hector Martin, Alyssa Rosenzweig, Sven Peter, Marc Zyngier,
	Mark Kettenis, Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

This driver exposes the CPU performance state switching hardware in
Apple SoCs as a clock controller that can be used together with the
generic cpufreq-dt mechanism to implement cpufreq support. It also
supports binding to an apple-mcc instance, to increase memory controller
performance when the CPUs are in the highest performance states.

Signed-off-by: Hector Martin <marcan@marcan.st>
---
 drivers/clk/Kconfig             |   9 ++
 drivers/clk/Makefile            |   1 +
 drivers/clk/clk-apple-cluster.c | 184 ++++++++++++++++++++++++++++++++
 3 files changed, 194 insertions(+)
 create mode 100644 drivers/clk/clk-apple-cluster.c

diff --git a/drivers/clk/Kconfig b/drivers/clk/Kconfig
index c5b3dc97396a..f3c8ad041f91 100644
--- a/drivers/clk/Kconfig
+++ b/drivers/clk/Kconfig
@@ -390,6 +390,15 @@ config COMMON_CLK_K210
 	help
 	  Support for the Canaan Kendryte K210 RISC-V SoC clocks.
 
+config COMMON_CLK_APPLE_CLUSTER
+	bool "Clock driver for Apple SoC CPU clusters"
+	depends on ARCH_APPLE || COMPILE_TEST
+	select CPUFREQ_DT
+	default ARCH_APPLE
+	help
+	  This driver supports CPU cluster frequency switching on Apple SoC
+	  platforms.
+
 source "drivers/clk/actions/Kconfig"
 source "drivers/clk/analogbits/Kconfig"
 source "drivers/clk/baikal-t1/Kconfig"
diff --git a/drivers/clk/Makefile b/drivers/clk/Makefile
index e42312121e51..6dba8c2052c7 100644
--- a/drivers/clk/Makefile
+++ b/drivers/clk/Makefile
@@ -31,6 +31,7 @@ obj-$(CONFIG_COMMON_CLK_FIXED_MMIO)	+= clk-fixed-mmio.o
 obj-$(CONFIG_COMMON_CLK_FSL_FLEXSPI)	+= clk-fsl-flexspi.o
 obj-$(CONFIG_COMMON_CLK_FSL_SAI)	+= clk-fsl-sai.o
 obj-$(CONFIG_COMMON_CLK_GEMINI)		+= clk-gemini.o
+obj-$(CONFIG_COMMON_CLK_APPLE_CLUSTER)	+= clk-apple-cluster.o
 obj-$(CONFIG_COMMON_CLK_ASPEED)		+= clk-aspeed.o
 obj-$(CONFIG_MACH_ASPEED_G6)		+= clk-ast2600.o
 obj-$(CONFIG_ARCH_HIGHBANK)		+= clk-highbank.o
diff --git a/drivers/clk/clk-apple-cluster.c b/drivers/clk/clk-apple-cluster.c
new file mode 100644
index 000000000000..9e9be38f13b2
--- /dev/null
+++ b/drivers/clk/clk-apple-cluster.c
@@ -0,0 +1,184 @@
+// SPDX-License-Identifier: GPL-2.0-only OR MIT
+/*
+ * Apple SoC CPU cluster performance state driver
+ *
+ * Copyright The Asahi Linux Contributors
+ */
+
+#include <linux/bitfield.h>
+#include <linux/bitops.h>
+#include <linux/clk-provider.h>
+#include <linux/delay.h>
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/iopoll.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/pm_domain.h>
+#include <linux/pm_opp.h>
+
+#define APPLE_CLUSTER_PSTATE    0x20
+#define APPLE_CLUSTER_PSTATE_BUSY	BIT(31)
+#define APPLE_CLUSTER_PSTATE_SET	BIT(25)
+#define APPLE_CLUSTER_PSTATE_DESIRED2	GENMASK(15, 12)
+#define APPLE_CLUSTER_PSTATE_DESIRED1	GENMASK(3, 0)
+
+struct apple_cluster_clk {
+	struct clk_hw hw;
+	struct device *dev;
+	void __iomem *reg_base;
+	bool has_pd;
+};
+
+#define to_apple_cluster_clk(_hw) container_of(_hw, struct apple_cluster_clk, hw)
+
+#define APPLE_CLUSTER_SWITCH_TIMEOUT 100
+
+static int apple_cluster_clk_set_rate(struct clk_hw *hw, unsigned long rate,
+					 unsigned long parent_rate)
+{
+	struct apple_cluster_clk *cluster = to_apple_cluster_clk(hw);
+	struct dev_pm_opp *opp;
+	unsigned int level;
+	u64 reg;
+
+	opp = dev_pm_opp_find_freq_floor(cluster->dev, &rate);
+
+	if (IS_ERR(opp))
+		return PTR_ERR(opp);
+
+	level = dev_pm_opp_get_level(opp);
+
+	dev_dbg(cluster->dev, "set_rate: %ld -> %d\n", rate, level);
+
+	if (readq_poll_timeout(cluster->reg_base + APPLE_CLUSTER_PSTATE, reg,
+			       !(reg & APPLE_CLUSTER_PSTATE_BUSY), 2,
+			       APPLE_CLUSTER_SWITCH_TIMEOUT)) {
+		dev_err(cluster->dev, "timed out waiting for busy flag\n");
+		return -EIO;
+	}
+
+	reg &= ~(APPLE_CLUSTER_PSTATE_DESIRED1 | APPLE_CLUSTER_PSTATE_DESIRED2);
+	reg |= FIELD_PREP(APPLE_CLUSTER_PSTATE_DESIRED1, level);
+	reg |= FIELD_PREP(APPLE_CLUSTER_PSTATE_DESIRED2, level);
+	reg |= APPLE_CLUSTER_PSTATE_SET;
+
+	writeq_relaxed(reg, cluster->reg_base + APPLE_CLUSTER_PSTATE);
+
+	if (cluster->has_pd)
+		dev_pm_genpd_set_performance_state(cluster->dev,
+						   dev_pm_opp_get_required_pstate(opp, 0));
+
+	return 0;
+}
+
+static unsigned long apple_cluster_clk_recalc_rate(struct clk_hw *hw, unsigned long parent_rate)
+{
+	struct apple_cluster_clk *cluster = to_apple_cluster_clk(hw);
+	struct dev_pm_opp *opp;
+	u64 reg;
+
+	reg = readq_relaxed(cluster->reg_base + APPLE_CLUSTER_PSTATE);
+
+	opp = dev_pm_opp_find_level_exact(cluster->dev,
+					  FIELD_GET(APPLE_CLUSTER_PSTATE_DESIRED1, reg));
+
+	if (IS_ERR(opp)) {
+		dev_err(cluster->dev, "failed to find level: 0x%llx (%ld)\n", reg, PTR_ERR(opp));
+		return 0;
+	}
+
+	return dev_pm_opp_get_freq(opp);
+}
+
+static long apple_cluster_clk_round_rate(struct clk_hw *hw, unsigned long rate,
+						  unsigned long *parent_rate)
+{
+	struct apple_cluster_clk *cluster = to_apple_cluster_clk(hw);
+	struct dev_pm_opp *opp;
+
+	opp = dev_pm_opp_find_freq_floor(cluster->dev, &rate);
+
+	if (IS_ERR(opp)) {
+		dev_err(cluster->dev, "failed to find rate: %ld (%ld)\n", rate, PTR_ERR(opp));
+		return PTR_ERR(opp);
+	}
+
+	return rate;
+}
+
+static const struct clk_ops apple_cluster_clk_ops = {
+	.set_rate = apple_cluster_clk_set_rate,
+	.recalc_rate = apple_cluster_clk_recalc_rate,
+	.round_rate = apple_cluster_clk_round_rate,
+};
+
+static int apple_cluster_clk_probe(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct device_node *node = dev->of_node;
+	struct apple_cluster_clk *cluster;
+	struct clk_hw *hw;
+	struct clk_init_data init;
+	int ret;
+
+	memset(&init, 0, sizeof(init));
+	cluster = devm_kzalloc(dev, sizeof(*cluster), GFP_KERNEL);
+	if (!cluster)
+		return -ENOMEM;
+
+	cluster->dev = dev;
+	cluster->reg_base = devm_platform_ioremap_resource(pdev, 0);
+	if (IS_ERR(cluster->reg_base))
+		return PTR_ERR(cluster->reg_base);
+
+	hw = &cluster->hw;
+	hw->init = &init;
+
+	init.name = pdev->name;
+	init.num_parents = 0;
+	init.ops = &apple_cluster_clk_ops;
+	init.flags = 0;
+
+	ret = dev_pm_opp_of_add_table_noclk(dev, 0);
+	if (ret < 0) {
+		dev_err(dev, "failed to get opp table\n");
+		return ret;
+	}
+
+	cluster->has_pd = of_property_read_bool(node, "power-domains");
+
+	ret = devm_of_clk_add_hw_provider(dev, of_clk_hw_simple_get, hw);
+	if (ret < 0)
+		return ret;
+
+	ret = devm_clk_hw_register(dev, hw);
+	if (ret) {
+		dev_err(dev, "failed to register clock\n");
+		return ret;
+	}
+
+	return 0;
+}
+
+static const struct of_device_id apple_cluster_clk_of_match[] = {
+	{ .compatible = "apple,cluster-clk" },
+	{}
+};
+
+MODULE_DEVICE_TABLE(of, apple_cluster_clk_of_match);
+
+static struct platform_driver apple_cluster_clk_driver = {
+	.probe = apple_cluster_clk_probe,
+	.driver = {
+		.name = "apple-cluster-clk",
+		.of_match_table = apple_cluster_clk_of_match,
+	},
+};
+
+MODULE_AUTHOR("Hector Martin <marcan@marcan.st>");
+MODULE_DESCRIPTION("CPU cluster performance state driver for Apple SoCs");
+MODULE_LICENSE("GPL v2");
+
+module_platform_driver(apple_cluster_clk_driver);
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 40+ messages in thread

* [RFC PATCH 8/9] arm64: apple: Select MEMORY and APPLE_MCC
  2021-10-11 16:56 [RFC PATCH 0/9] Apple SoC CPU P-state switching Hector Martin
                   ` (6 preceding siblings ...)
  2021-10-11 16:57 ` [RFC PATCH 7/9] clk: apple: Add clk-apple-cluster driver to manage CPU p-states Hector Martin
@ 2021-10-11 16:57 ` Hector Martin
  2021-10-11 16:57 ` [RFC PATCH 9/9] arm64: apple: Add CPU frequency scaling support for t8103 Hector Martin
  8 siblings, 0 replies; 40+ messages in thread
From: Hector Martin @ 2021-10-11 16:57 UTC (permalink / raw)
  To: linux-arm-kernel
  Cc: Hector Martin, Alyssa Rosenzweig, Sven Peter, Marc Zyngier,
	Mark Kettenis, Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

This driver is for the memory controller and users should always have
it; it is also a requirement with our devicetree configs for cpufreq to
work properly if that driver is available.

Signed-off-by: Hector Martin <marcan@marcan.st>
---
 arch/arm64/Kconfig.platforms | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm64/Kconfig.platforms b/arch/arm64/Kconfig.platforms
index b0ce18d4cc98..b60ed872dae1 100644
--- a/arch/arm64/Kconfig.platforms
+++ b/arch/arm64/Kconfig.platforms
@@ -30,6 +30,8 @@ config ARCH_ALPINE
 config ARCH_APPLE
 	bool "Apple Silicon SoC family"
 	select APPLE_AIC
+	select MEMORY
+	select APPLE_MCC
 	help
 	  This enables support for Apple's in-house ARM SoC family, starting
 	  with the Apple M1.
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 40+ messages in thread

* [RFC PATCH 9/9] arm64: apple: Add CPU frequency scaling support for t8103
  2021-10-11 16:56 [RFC PATCH 0/9] Apple SoC CPU P-state switching Hector Martin
                   ` (7 preceding siblings ...)
  2021-10-11 16:57 ` [RFC PATCH 8/9] arm64: apple: Select MEMORY and APPLE_MCC Hector Martin
@ 2021-10-11 16:57 ` Hector Martin
  8 siblings, 0 replies; 40+ messages in thread
From: Hector Martin @ 2021-10-11 16:57 UTC (permalink / raw)
  To: linux-arm-kernel
  Cc: Hector Martin, Alyssa Rosenzweig, Sven Peter, Marc Zyngier,
	Mark Kettenis, Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

- Proper CPU topology
- CPU capacities
  * The 714 value is based on the CoreMark benchmark [1]. This is
    roughly in line with other real-world test cases, like gzip. For
    some reason, Dhrystone gives a wildly different value of 326, but
    this doesn't seem representative of real-world workloads. This might
    be adjusted in the future.
- MCC instance to control memory controller performance
- MCC OPP for t8103 using config values from hardware/ADT
- E-Cluster and P-Cluster clock controllers for CPU frequency switching
- Cluster OPP tables, including latency values determined
  experimentally.

This relies on the generic cpufreq-dt driver to implement the cpufreq
side. It also assumes the bootloader did any required init (iBoot does
everything on firmware 12.0 and later; for 11.x we will have m1n1 do
what's missing).

[1] https://github.com/kdrag0n/linux-m1/commit/05c296604a42189cb61a0f7e8665566de192cbe9

Signed-off-by: Hector Martin <marcan@marcan.st>
---
 arch/arm64/boot/dts/apple/t8103.dtsi | 255 ++++++++++++++++++++++++++-
 1 file changed, 247 insertions(+), 8 deletions(-)

diff --git a/arch/arm64/boot/dts/apple/t8103.dtsi b/arch/arm64/boot/dts/apple/t8103.dtsi
index a1e22a2ea2e5..5eec86917d72 100644
--- a/arch/arm64/boot/dts/apple/t8103.dtsi
+++ b/arch/arm64/boot/dts/apple/t8103.dtsi
@@ -20,68 +20,284 @@ cpus {
 		#address-cells = <2>;
 		#size-cells = <0>;
 
-		cpu0: cpu@0 {
+		cpu-map {
+			cluster0 {
+				core0 {
+					cpu = <&cpu_e0>;
+				};
+				core1 {
+					cpu = <&cpu_e1>;
+				};
+				core2 {
+					cpu = <&cpu_e2>;
+				};
+				core3 {
+					cpu = <&cpu_e3>;
+				};
+			};
+
+			cluster1 {
+				core0 {
+					cpu = <&cpu_p0>;
+				};
+				core1 {
+					cpu = <&cpu_p1>;
+				};
+				core2 {
+					cpu = <&cpu_p2>;
+				};
+				core3 {
+					cpu = <&cpu_p3>;
+				};
+			};
+		};
+
+		cpu_e0: cpu@0 {
 			compatible = "apple,icestorm";
 			device_type = "cpu";
 			reg = <0x0 0x0>;
 			enable-method = "spin-table";
 			cpu-release-addr = <0 0>; /* To be filled by loader */
+			clocks = <&clk_ecluster>;
+			operating-points-v2 = <&ecluster_opp>;
+			capacity-dmips-mhz = <714>;
 		};
 
-		cpu1: cpu@1 {
+		cpu_e1: cpu@1 {
 			compatible = "apple,icestorm";
 			device_type = "cpu";
 			reg = <0x0 0x1>;
 			enable-method = "spin-table";
 			cpu-release-addr = <0 0>; /* To be filled by loader */
+			clocks = <&clk_ecluster>;
+			operating-points-v2 = <&ecluster_opp>;
+			capacity-dmips-mhz = <714>;
 		};
 
-		cpu2: cpu@2 {
+		cpu_e2: cpu@2 {
 			compatible = "apple,icestorm";
 			device_type = "cpu";
 			reg = <0x0 0x2>;
 			enable-method = "spin-table";
 			cpu-release-addr = <0 0>; /* To be filled by loader */
+			clocks = <&clk_ecluster>;
+			operating-points-v2 = <&ecluster_opp>;
+			capacity-dmips-mhz = <714>;
 		};
 
-		cpu3: cpu@3 {
+		cpu_e3: cpu@3 {
 			compatible = "apple,icestorm";
 			device_type = "cpu";
 			reg = <0x0 0x3>;
 			enable-method = "spin-table";
 			cpu-release-addr = <0 0>; /* To be filled by loader */
+			clocks = <&clk_ecluster>;
+			operating-points-v2 = <&ecluster_opp>;
+			capacity-dmips-mhz = <714>;
 		};
 
-		cpu4: cpu@10100 {
+		cpu_p0: cpu@10100 {
 			compatible = "apple,firestorm";
 			device_type = "cpu";
 			reg = <0x0 0x10100>;
 			enable-method = "spin-table";
 			cpu-release-addr = <0 0>; /* To be filled by loader */
+			clocks = <&clk_pcluster>;
+			operating-points-v2 = <&pcluster_opp>;
+			capacity-dmips-mhz = <1024>;
 		};
 
-		cpu5: cpu@10101 {
+		cpu_p1: cpu@10101 {
 			compatible = "apple,firestorm";
 			device_type = "cpu";
 			reg = <0x0 0x10101>;
 			enable-method = "spin-table";
 			cpu-release-addr = <0 0>; /* To be filled by loader */
+			clocks = <&clk_pcluster>;
+			operating-points-v2 = <&pcluster_opp>;
+			capacity-dmips-mhz = <1024>;
 		};
 
-		cpu6: cpu@10102 {
+		cpu_p2: cpu@10102 {
 			compatible = "apple,firestorm";
 			device_type = "cpu";
 			reg = <0x0 0x10102>;
 			enable-method = "spin-table";
 			cpu-release-addr = <0 0>; /* To be filled by loader */
+			clocks = <&clk_pcluster>;
+			operating-points-v2 = <&pcluster_opp>;
+			capacity-dmips-mhz = <1024>;
 		};
 
-		cpu7: cpu@10103 {
+		cpu_p3: cpu@10103 {
 			compatible = "apple,firestorm";
 			device_type = "cpu";
 			reg = <0x0 0x10103>;
 			enable-method = "spin-table";
 			cpu-release-addr = <0 0>; /* To be filled by loader */
+			clocks = <&clk_pcluster>;
+			operating-points-v2 = <&pcluster_opp>;
+			capacity-dmips-mhz = <1024>;
+		};
+	};
+
+	ecluster_opp: opp-table-0 {
+		compatible = "operating-points-v2";
+		opp-shared;
+
+		opp01 {
+			opp-hz = /bits/ 64 <600000000>;
+			opp-microvolt = <559000>;
+			opp-level = <1>;
+			clock-latency-ns = <7500>;
+		};
+		opp02 {
+			opp-hz = /bits/ 64 <972000000>;
+			opp-microvolt = <628000>;
+			opp-level = <2>;
+			clock-latency-ns = <22000>;
+		};
+		opp03 {
+			opp-hz = /bits/ 64 <1332000000>;
+			opp-microvolt = <684000>;
+			opp-level = <3>;
+			clock-latency-ns = <27000>;
+		};
+		opp04 {
+			opp-hz = /bits/ 64 <1704000000>;
+			opp-microvolt = <765000>;
+			opp-level = <4>;
+			clock-latency-ns = <33000>;
+		};
+		opp05 {
+			opp-hz = /bits/ 64 <2064000000>;
+			opp-microvolt = <868000>;
+			opp-level = <5>;
+			clock-latency-ns = <50000>;
+		};
+	};
+
+	pcluster_opp: opp-table-1 {
+		compatible = "operating-points-v2";
+		opp-shared;
+
+		opp01 {
+			opp-hz = /bits/ 64 <600000000>;
+			opp-microvolt = <781000>;
+			opp-level = <1>;
+			clock-latency-ns = <8000>;
+			required-opps = <&mcc_lowperf>;
+		};
+		opp02 {
+			opp-hz = /bits/ 64 <828000000>;
+			opp-microvolt = <781000>;
+			opp-level = <2>;
+			clock-latency-ns = <19000>;
+			required-opps = <&mcc_lowperf>;
+		};
+		opp03 {
+			opp-hz = /bits/ 64 <1056000000>;
+			opp-microvolt = <781000>;
+			opp-level = <3>;
+			clock-latency-ns = <21000>;
+			required-opps = <&mcc_lowperf>;
+		};
+		opp04 {
+			opp-hz = /bits/ 64 <1284000000>;
+			opp-microvolt = <800000>;
+			opp-level = <4>;
+			clock-latency-ns = <23000>;
+			required-opps = <&mcc_lowperf>;
+		};
+		opp05 {
+			opp-hz = /bits/ 64 <1500000000>;
+			opp-microvolt = <821000>;
+			opp-level = <5>;
+			clock-latency-ns = <24000>;
+			required-opps = <&mcc_lowperf>;
+		};
+		opp06 {
+			opp-hz = /bits/ 64 <1728000000>;
+			opp-microvolt = <831000>;
+			opp-level = <6>;
+			clock-latency-ns = <29000>;
+			required-opps = <&mcc_lowperf>;
+		};
+		opp07 {
+			opp-hz = /bits/ 64 <1956000000>;
+			opp-microvolt = <865000>;
+			opp-level = <7>;
+			clock-latency-ns = <31000>;
+			required-opps = <&mcc_lowperf>;
+		};
+		opp08 {
+			opp-hz = /bits/ 64 <2184000000>;
+			opp-microvolt = <909000>;
+			opp-level = <8>;
+			clock-latency-ns = <34000>;
+			required-opps = <&mcc_highperf>;
+		};
+		opp09 {
+			opp-hz = /bits/ 64 <2388000000>;
+			opp-microvolt = <953000>;
+			opp-level = <9>;
+			clock-latency-ns = <36000>;
+			required-opps = <&mcc_highperf>;
+		};
+		opp10 {
+			opp-hz = /bits/ 64 <2592000000>;
+			opp-microvolt = <1003000>;
+			opp-level = <10>;
+			clock-latency-ns = <51000>;
+			required-opps = <&mcc_highperf>;
+		};
+		opp11 {
+			opp-hz = /bits/ 64 <2772000000>;
+			opp-microvolt = <1053000>;
+			opp-level = <11>;
+			clock-latency-ns = <54000>;
+			required-opps = <&mcc_highperf>;
+		};
+		opp12 {
+			opp-hz = /bits/ 64 <2988000000>;
+			opp-microvolt = <1081000>;
+			opp-level = <12>;
+			clock-latency-ns = <55000>;
+			required-opps = <&mcc_highperf>;
+		};
+		opp13 {
+			opp-hz = /bits/ 64 <3096000000>;
+			opp-microvolt = <1081000>;
+			opp-level = <13>;
+			clock-latency-ns = <55000>;
+			required-opps = <&mcc_highperf>;
+		};
+		opp14 {
+			opp-hz = /bits/ 64 <3144000000>;
+			opp-microvolt = <1081000>;
+			opp-level = <14>;
+			clock-latency-ns = <56000>;
+			required-opps = <&mcc_highperf>;
+		};
+		opp15 {
+			opp-hz = /bits/ 64 <3204000000>;
+			opp-microvolt = <1081000>;
+			opp-level = <15>;
+			clock-latency-ns = <56000>;
+			required-opps = <&mcc_highperf>;
+		};
+	};
+
+	mcc_opp: opp-table-2 {
+		compatible = "apple,mcc-operating-points";
+
+		mcc_lowperf: opp0 {
+			opp-level = <0>;
+			apple,memory-perf-config = <0x813057f 0x1800180>;
+		};
+		mcc_highperf: opp1 {
+			opp-level = <1>;
+			apple,memory-perf-config = <0x133 0x55555340>;
 		};
 	};
 
@@ -110,6 +326,29 @@ soc {
 		ranges;
 		nonposted-mmio;
 
+		mcc: memory-controller@200200000 {
+			compatible = "apple,t8103-mcc", "apple,mcc";
+			#power-domain-cells = <0>;
+			reg = <0x2 0x200000 0x0 0x200000>;
+			operating-points-v2 = <&mcc_opp>;
+			apple,num-channels = <8>;
+		};
+
+		clk_ecluster: clock-controller@210e20000 {
+			compatible = "apple,t8103-cluster-clk", "apple,cluster-clk";
+			#clock-cells = <0>;
+			reg = <0x2 0x10e20000 0x0 0x4000>;
+			operating-points-v2 = <&ecluster_opp>;
+		};
+
+		clk_pcluster: clock-controller@211e20000 {
+			compatible = "apple,t8103-cluster-clk", "apple,cluster-clk";
+			#clock-cells = <0>;
+			reg = <0x2 0x11e20000 0x0 0x4000>;
+			operating-points-v2 = <&pcluster_opp>;
+			power-domains = <&mcc>;
+		};
+
 		serial0: serial@235200000 {
 			compatible = "apple,s5l-uart";
 			reg = <0x2 0x35200000 0x0 0x1000>;
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-11 16:57 ` [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist Hector Martin
@ 2021-10-12  3:21   ` Viresh Kumar
  2021-10-12  5:34     ` Hector Martin
  0 siblings, 1 reply; 40+ messages in thread
From: Viresh Kumar @ 2021-10-12  3:21 UTC (permalink / raw)
  To: Hector Martin
  Cc: linux-arm-kernel, Alyssa Rosenzweig, Sven Peter, Marc Zyngier,
	Mark Kettenis, Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

On 12-10-21, 01:57, Hector Martin wrote:
> When required-opps is used in CPU OPP tables, there is no parent power
> domain to drive it. Squelch this error, to allow a clock driver to
> handle this directly instead.
> 
> Signed-off-by: Hector Martin <marcan@marcan.st>
> ---
>  drivers/opp/core.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/opp/core.c b/drivers/opp/core.c
> index 04b4691a8aac..89e616721f70 100644
> --- a/drivers/opp/core.c
> +++ b/drivers/opp/core.c
> @@ -873,12 +873,13 @@ static int _set_required_opp(struct device *dev, struct device *pd_dev,
>  		return 0;
>  
>  	ret = dev_pm_genpd_set_performance_state(pd_dev, pstate);
> -	if (ret) {
> +	if (ret && ret != -ENODEV) {
>  		dev_err(dev, "Failed to set performance rate of %s: %d (%d)\n",
>  			dev_name(pd_dev), pstate, ret);
> +		return ret;
>  	}
>  
> -	return ret;
> +	return 0;
>  }
>  
>  /* This is only called for PM domain for now */

I am not sure why you need this, since _set_required_opps() has this check:

	if (unlikely(!required_opp_tables[0]->is_genpd)) {
		dev_err(dev, "required-opps don't belong to a genpd\n");
		return -ENOENT;
	}

-- 
viresh

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-12  3:21   ` Viresh Kumar
@ 2021-10-12  5:34     ` Hector Martin
  2021-10-12  5:51       ` Viresh Kumar
  0 siblings, 1 reply; 40+ messages in thread
From: Hector Martin @ 2021-10-12  5:34 UTC (permalink / raw)
  To: Viresh Kumar
  Cc: linux-arm-kernel, Alyssa Rosenzweig, Sven Peter, Marc Zyngier,
	Mark Kettenis, Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

On 12/10/2021 12.21, Viresh Kumar wrote:
> I am not sure why you need this, since _set_required_opps() has this check:
> 
> 	if (unlikely(!required_opp_tables[0]->is_genpd)) {
> 		dev_err(dev, "required-opps don't belong to a genpd\n");
> 		return -ENOENT;
> 	}
> 

The table *is* assigned to a genpd (the memory controller), it's just 
that that genpd isn't actually a parent of the CPU device. Without the 
patch you end up with:

[    3.040060] cpu cpu4: Failed to set performance rate of cpu4: 0 (-19)
[    3.042881] cpu cpu4: Failed to set required opps: -19
[    3.045508] cpufreq: __target_index: Failed to change cpu frequency: -19


-- 
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-12  5:34     ` Hector Martin
@ 2021-10-12  5:51       ` Viresh Kumar
  2021-10-12  5:57         ` Hector Martin
  0 siblings, 1 reply; 40+ messages in thread
From: Viresh Kumar @ 2021-10-12  5:51 UTC (permalink / raw)
  To: Hector Martin, Sibi Sankar, Saravana Kannan
  Cc: linux-arm-kernel, Alyssa Rosenzweig, Sven Peter, Marc Zyngier,
	Mark Kettenis, Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

On 12-10-21, 14:34, Hector Martin wrote:
> The table *is* assigned to a genpd (the memory controller), it's just that
> that genpd isn't actually a parent of the CPU device. Without the patch you
> end up with:
> 
> [    3.040060] cpu cpu4: Failed to set performance rate of cpu4: 0 (-19)
> [    3.042881] cpu cpu4: Failed to set required opps: -19
> [    3.045508] cpufreq: __target_index: Failed to change cpu frequency: -19

Hmm, Saravana and Sibi were working on a similar problem earlier and decided to
solve this using devfreq instead. Don't remember the exact series which got
merged for this, Sibi ?

If this part fails, how do you actually set the performance state of the memory
controller's genpd ?

-- 
viresh

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-12  5:51       ` Viresh Kumar
@ 2021-10-12  5:57         ` Hector Martin
  2021-10-12  9:26           ` Viresh Kumar
  2021-10-14  9:55           ` Ulf Hansson
  0 siblings, 2 replies; 40+ messages in thread
From: Hector Martin @ 2021-10-12  5:57 UTC (permalink / raw)
  To: Viresh Kumar, Sibi Sankar, Saravana Kannan
  Cc: linux-arm-kernel, Alyssa Rosenzweig, Sven Peter, Marc Zyngier,
	Mark Kettenis, Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

On 12/10/2021 14.51, Viresh Kumar wrote:
> On 12-10-21, 14:34, Hector Martin wrote:
>> The table *is* assigned to a genpd (the memory controller), it's just that
>> that genpd isn't actually a parent of the CPU device. Without the patch you
>> end up with:
>>
>> [    3.040060] cpu cpu4: Failed to set performance rate of cpu4: 0 (-19)
>> [    3.042881] cpu cpu4: Failed to set required opps: -19
>> [    3.045508] cpufreq: __target_index: Failed to change cpu frequency: -19
> 
> Hmm, Saravana and Sibi were working on a similar problem earlier and decided to
> solve this using devfreq instead. Don't remember the exact series which got
> merged for this, Sibi ?
> 
> If this part fails, how do you actually set the performance state of the memory
> controller's genpd ?

The clock controller has the genpd as an actual power-domain parent, so 
it does it instead. From patch #7:

> +	if (cluster->has_pd)
> +		dev_pm_genpd_set_performance_state(cluster->dev,
> +						   dev_pm_opp_get_required_pstate(opp, 0));
> +

This is arguably not entirely representative of how the hardware works, 
since technically the cluster switching couldn't care less what the 
memory controller is doing; it's a soft dependency, states that should 
be switched together but are not interdependent (in fact, the clock code 
does this unconditionally after the CPU p-state change, regardless of 
whether we're shifting up or down; this is, FWIW, the same order macOS 
uses, and it clearly doesn't matter which way you do it).

-- 
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 2/9] dt-bindings: memory-controller: Add apple,mcc binding
  2021-10-11 16:57 ` [RFC PATCH 2/9] dt-bindings: memory-controller: Add apple,mcc binding Hector Martin
@ 2021-10-12  8:48   ` Krzysztof Kozlowski
  2021-10-19 22:43     ` Rob Herring
  0 siblings, 1 reply; 40+ messages in thread
From: Krzysztof Kozlowski @ 2021-10-12  8:48 UTC (permalink / raw)
  To: Hector Martin, linux-arm-kernel
  Cc: Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring, Viresh Kumar,
	Nishanth Menon, Catalin Marinas, Rafael J. Wysocki, Kevin Hilman,
	Ulf Hansson, linux-clk, devicetree, linux-pm, linux-kernel

On 11/10/2021 18:57, Hector Martin wrote:
> This device represents the memory controller in Apple SoCs, and is
> chiefly in charge of adjusting performance characteristics according to
> system demand.
> 
> Signed-off-by: Hector Martin <marcan@marcan.st>
> ---
>  .../memory-controllers/apple,mcc.yaml         | 80 +++++++++++++++++++
>  .../opp/apple,mcc-operating-points.yaml       | 62 ++++++++++++++
>  2 files changed, 142 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml
>  create mode 100644 Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml
> 
> diff --git a/Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml b/Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml
> new file mode 100644
> index 000000000000..0774f10e65ed
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml
> @@ -0,0 +1,80 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/memory-controllers/apple,mcc.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Apple SoC MCC memory controller performance controls
> +
> +maintainers:
> +  - Hector Martin <marcan@marcan.st>
> +
> +description: |
> +  Apple SoCs contain a multichannel memory controller that can have its
> +  configuration changed to adjust to changing performance requirements from
> +  the rest of the SoC. This node represents the controller and provides a
> +  power domain provider that downstream devices can use to adjust the memory
> +  controller performance level.
> +
> +properties:
> +  compatible:
> +    items:
> +      - enum:
> +          - apple,t8103-mcc
> +      - const: apple,mcc
> +
> +  reg:
> +    maxItems: 1
> +
> +  "#power-domain-cells":
> +    const: 0
> +
> +  operating-points-v2:
> +    $ref: /schemas/types.yaml#/definitions/phandle-array
> +    description:
> +      A reference to the OPP table describing the memory controller performance
> +      levels. Each OPP node should contain an `apple,memory-perf-config`
> +      property that contains the configuration values for that performance
> +      level.
> +
> +  apple,num-channels:
> +    $ref: /schemas/types.yaml#/definitions/uint32
> +    description:
> +      The number of memory channels in use.
> +
> +required:
> +  - compatible
> +  - reg
> +  - "#power-domain-cells"
> +  - operating-points-v2
> +  - apple,num-channels
> +
> +additionalProperties: false
> +
> +examples:
> +  # See clock/apple,cluster-clock.yaml for an example of downstream usage.
> +  - |
> +    mcc_opp: opp-table-2 {
> +        compatible = "operating-points-v2";

apple,mcc-operating-points?

> +
> +        mcc_lowperf: opp0 {
> +            opp-level = <0>;
> +            apple,memory-perf-config = <0x813057f 0x1800180>;
> +        };
> +        mcc_highperf: opp1 {
> +            opp-level = <1>;
> +            apple,memory-perf-config = <0x133 0x55555340>;
> +        };
> +    };
> +    soc {
> +        #address-cells = <2>;
> +        #size-cells = <2>;
> +
> +        mcc: memory-controller@200200000 {
> +            compatible = "apple,t8103-mcc", "apple,mcc";
> +            #power-domain-cells = <0>;
> +            reg = <0x2 0x200000 0x0 0x200000>;
> +            operating-points-v2 = <&mcc_opp>;
> +            apple,num-channels = <8>;
> +        };
> +    };
> diff --git a/Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml b/Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml
> new file mode 100644
> index 000000000000..babf27841bb7
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml
> @@ -0,0 +1,62 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/opp/apple,mcc-operating-points.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Apple SoC memory controller OPP bindings
> +
> +maintainers:
> +  - Hector Martin <marcan@marcan.st>
> +
> +description: |
> +  Apple SoCs can have their memory controller performance adjusted depending on
> +  system requirements. These performance states are represented by specific
> +  memory controller register values. The apple-mcc driver uses these values
> +  to change the MCC performance.
> +
> +allOf:
> +  - $ref: opp-v2-base.yaml#
> +
> +properties:
> +  compatible:
> +    const: apple,mcc-operating-points
> +
> +required:
> +  - compatible
> +
> +patternProperties:
> +  "opp[0-9]+":
> +    type: object
> +
> +    properties:
> +      opp-level: true

You don't need to mention it.

> +      apple,memory-perf-config:
> +        $ref: /schemas/types.yaml#/definitions/uint32-array
> +        description: |
> +          A pair of register values used to configure this performance state.
> +        minItems: 2
> +        maxItems: 2
> +
> +    required:
> +      - opp-level
> +      - apple,memory-perf-config
> +
> +    unevaluatedProperties: false
> +
> +additionalProperties: false
> +
> +examples:
> +  - |
> +    mcc_opp: opp-table-2 {
> +        compatible = "operating-points-v2";

Different compatible.

> +
> +        mcc_lowperf: opp0 {
> +            opp-level = <0>;
> +            apple,memory-perf-config = <0x813057f 0x1800180>;
> +        };
> +        mcc_highperf: opp1 {
> +            opp-level = <1>;
> +            apple,memory-perf-config = <0x133 0x55555340>;
> +        };
> +    };
> 


Best regards,
Krzysztof

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 3/9] dt-bindings: clock: Add apple,cluster-clk binding
  2021-10-11 16:57 ` [RFC PATCH 3/9] dt-bindings: clock: Add apple,cluster-clk binding Hector Martin
@ 2021-10-12  8:51   ` Krzysztof Kozlowski
  2021-10-12  9:35     ` Viresh Kumar
  0 siblings, 1 reply; 40+ messages in thread
From: Krzysztof Kozlowski @ 2021-10-12  8:51 UTC (permalink / raw)
  To: Hector Martin, linux-arm-kernel
  Cc: Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring, Viresh Kumar,
	Nishanth Menon, Catalin Marinas, Rafael J. Wysocki, Kevin Hilman,
	Ulf Hansson, linux-clk, devicetree, linux-pm, linux-kernel

On 11/10/2021 18:57, Hector Martin wrote:
> This device represents the CPU performance state switching mechanism as
> a clock controller, to be used with the standard cpufreq-dt
> infrastructure.
> 
> Signed-off-by: Hector Martin <marcan@marcan.st>
> ---
>  .../bindings/clock/apple,cluster-clk.yaml     | 115 ++++++++++++++++++
>  1 file changed, 115 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/clock/apple,cluster-clk.yaml
> 
> diff --git a/Documentation/devicetree/bindings/clock/apple,cluster-clk.yaml b/Documentation/devicetree/bindings/clock/apple,cluster-clk.yaml
> new file mode 100644
> index 000000000000..9a8b863dadc0
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/clock/apple,cluster-clk.yaml
> @@ -0,0 +1,115 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/clock/apple,cluster-clk.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: CPU cluster frequency scaling for Apple SoCs
> +
> +maintainers:
> +  - Hector Martin <marcan@marcan.st>
> +
> +description: |
> +  Apple SoCs control CPU cluster frequencies by using a performance state
> +  index. This node represents the feature as a clock controller, and uses
> +  a reference to the CPU OPP table to translate clock frequencies into
> +  performance states. This allows the CPUs to use the standard cpufreq-dt
> +  mechanism for frequency scaling.
> +
> +properties:
> +  compatible:
> +    items:
> +      - enum:
> +          - apple,t8103-cluster-clk
> +      - const: apple,cluster-clk
> +
> +  reg:
> +    maxItems: 1
> +
> +  '#clock-cells':
> +    const: 0
> +
> +  operating-points-v2:
> +    $ref: /schemas/types.yaml#/definitions/phandle-array
> +    description:
> +      A reference to the OPP table used for the CPU cluster controlled by this
> +      device instance. The table should contain an `opp-level` property for
> +      every OPP, which represents the p-state index used by the hardware to
> +      represent this performance level.
> +
> +      OPPs may also have a `required-opps` property (see power-domains).
> +
> +  power-domains:
> +    maxItems: 1
> +    description:
> +      An optional reference to a power domain provider that links its
> +      performance state to the CPU cluster performance state. This is typically
> +      a memory controller. If set, the `required-opps` property in the CPU
> +      frequency OPP nodes will be used to change the performance state of this
> +      provider state in tandem with CPU frequency changes.
> +
> +required:
> +  - compatible
> +  - reg
> +  - '#clock-cells'
> +  - operating-points-v2
> +
> +additionalProperties: false
> +
> +

One line break.

> +examples:
> +  - |
> +    pcluster_opp: opp-table-1 {
> +        compatible = "operating-points-v2";
> +        opp-shared;
> +
> +        opp01 {
> +            opp-hz = /bits/ 64 <600000000>;
> +            opp-microvolt = <781000>;
> +            opp-level = <1>;
> +            clock-latency-ns = <8000>;
> +            required-opps = <&mcc_lowperf>;
> +        };
> +        /* intermediate p-states omitted */
> +        opp15 {
> +            opp-hz = /bits/ 64 <3204000000>;
> +            opp-microvolt = <1081000>;
> +            opp-level = <15>;
> +            clock-latency-ns = <56000>;
> +            required-opps = <&mcc_highperf>;
> +        };
> +    };
> +
> +    mcc_opp: opp-table-2 {
> +        compatible = "operating-points-v2";

Wrong compatible.

> +
> +        mcc_lowperf: opp0 {
> +            opp-level = <0>;
> +            apple,memory-perf-config = <0x813057f 0x1800180>;
> +        };
> +        mcc_highperf: opp1 {
> +            opp-level = <1>;
> +            apple,memory-perf-config = <0x133 0x55555340>;
> +        };
> +    };
> +
> +    soc {
> +        #address-cells = <2>;
> +        #size-cells = <2>;
> +
> +        mcc: memory-controller@200200000 {
> +            compatible = "apple,t8103-mcc", "apple,mcc";
> +            #power-domain-cells = <0>;
> +            reg = <0x2 0x200000 0x0 0x200000>;
> +            operating-points-v2 = <&mcc_opp>;
> +            apple,num-channels = <8>;
> +        };
> +
> +        clk_pcluster: clock-controller@211e20000 {
> +            compatible = "apple,t8103-cluster-clk", "apple,cluster-clk";
> +            #clock-cells = <0>;
> +            reg = <0x2 0x11e20000 0x0 0x4000>;
> +            operating-points-v2 = <&pcluster_opp>;
> +            power-domains = <&mcc>;
> +        };
> +    };
> 


Best regards,
Krzysztof

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 6/9] memory: apple: Add apple-mcc driver to manage MCC perf in Apple SoCs
  2021-10-11 16:57 ` [RFC PATCH 6/9] memory: apple: Add apple-mcc driver to manage MCC perf in Apple SoCs Hector Martin
@ 2021-10-12  9:19   ` Krzysztof Kozlowski
  2021-10-14  6:59     ` Hector Martin
  0 siblings, 1 reply; 40+ messages in thread
From: Krzysztof Kozlowski @ 2021-10-12  9:19 UTC (permalink / raw)
  To: Hector Martin, linux-arm-kernel
  Cc: Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring, Viresh Kumar,
	Nishanth Menon, Catalin Marinas, Rafael J. Wysocki, Kevin Hilman,
	Ulf Hansson, linux-clk, devicetree, linux-pm, linux-kernel

On 11/10/2021 18:57, Hector Martin wrote:
> This driver binds to the memory controller hardware in Apple SoCs such
> as the Apple M1, and provides a power domain that downstream devices can
> use to change the performance state of the memory controller.
> 
> Signed-off-by: Hector Martin <marcan@marcan.st>
> ---
>  drivers/memory/Kconfig     |   9 +++
>  drivers/memory/Makefile    |   1 +
>  drivers/memory/apple-mcc.c | 130 +++++++++++++++++++++++++++++++++++++
>  3 files changed, 140 insertions(+)
>  create mode 100644 drivers/memory/apple-mcc.c
> 
> diff --git a/drivers/memory/Kconfig b/drivers/memory/Kconfig
> index 72c0df129d5c..48ef3d563a1c 100644
> --- a/drivers/memory/Kconfig
> +++ b/drivers/memory/Kconfig
> @@ -30,6 +30,15 @@ config ARM_PL172_MPMC
>  	  If you have an embedded system with an AMBA bus and a PL172
>  	  controller, say Y or M here.
>  
> +config APPLE_MCC
> +	tristate "Apple SoC MCC driver"
> +	default y if ARCH_APPLE
> +	select PM_GENERIC_DOMAINS
> +	depends on ARCH_APPLE || COMPILE_TEST
> +	help
> +	  This driver manages performance tuning for the memory controller in
> +	  Apple SoCs, such as the Apple M1.
> +
>  config ATMEL_SDRAMC
>  	bool "Atmel (Multi-port DDR-)SDRAM Controller"
>  	default y if ARCH_AT91
> diff --git a/drivers/memory/Makefile b/drivers/memory/Makefile
> index bc7663ed1c25..947840cbd2d4 100644
> --- a/drivers/memory/Makefile
> +++ b/drivers/memory/Makefile
> @@ -8,6 +8,7 @@ ifeq ($(CONFIG_DDR),y)
>  obj-$(CONFIG_OF)		+= of_memory.o
>  endif
>  obj-$(CONFIG_ARM_PL172_MPMC)	+= pl172.o
> +obj-$(CONFIG_APPLE_MCC)		+= apple-mcc.o
>  obj-$(CONFIG_ATMEL_SDRAMC)	+= atmel-sdramc.o
>  obj-$(CONFIG_ATMEL_EBI)		+= atmel-ebi.o
>  obj-$(CONFIG_BRCMSTB_DPFE)	+= brcmstb_dpfe.o
> diff --git a/drivers/memory/apple-mcc.c b/drivers/memory/apple-mcc.c
> new file mode 100644
> index 000000000000..55959f034b9a
> --- /dev/null
> +++ b/drivers/memory/apple-mcc.c
> @@ -0,0 +1,130 @@
> +// SPDX-License-Identifier: GPL-2.0-only OR MIT
> +/*
> + * Apple SoC MCC memory controller performance control driver
> + *
> + * Copyright The Asahi Linux Contributors

Copyright date?

> + */
> +
> +#include <linux/delay.h>
> +#include <linux/err.h>
> +#include <linux/io.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_domain.h>
> +#include <linux/pm_opp.h>
> +
> +#define APPLE_MCC_PERF_CONFIG1  0xdc4
> +#define APPLE_MCC_PERF_CONFIG2  0xdbc
> +#define APPLE_MCC_CHANNEL(x)	((x) * 0x40000)
> +
> +struct apple_mcc {
> +	struct device *dev;
> +	struct generic_pm_domain genpd;
> +	void __iomem *reg_base;
> +	u32 num_channels;
> +};
> +
> +#define to_apple_mcc(_genpd) container_of(_genpd, struct apple_mcc, genpd)
> +
> +static int apple_mcc_set_performance_state(struct generic_pm_domain *genpd, unsigned int state)
> +{
> +	struct apple_mcc *mcc = to_apple_mcc(genpd);
> +	struct dev_pm_opp *opp;
> +	struct device_node *np;
> +	u32 perf_config[2];
> +	unsigned int i;
> +
> +	dev_dbg(mcc->dev, "switching to perf state %d\n", state);
> +
> +	opp = dev_pm_opp_find_level_exact(&mcc->genpd.dev, state);
> +	if (IS_ERR(opp))
> +		return PTR_ERR(opp);
> +
> +	np = dev_pm_opp_get_of_node(opp);
> +	if (of_property_read_u32_array(np, "apple,memory-perf-config",
> +		perf_config, ARRAY_SIZE(perf_config))) {
> +		dev_err(mcc->dev, "missing apple,memory-perf-config property");
> +		of_node_put(np);
> +		return -EINVAL;
> +	}
> +	of_node_put(np);
> +
> +	for (i = 0; i < mcc->num_channels; i++) {
> +		writel_relaxed(perf_config[0],
> +			       mcc->reg_base + APPLE_MCC_CHANNEL(i) + APPLE_MCC_PERF_CONFIG1);
> +		writel_relaxed(perf_config[1],
> +			       mcc->reg_base + APPLE_MCC_CHANNEL(i) + APPLE_MCC_PERF_CONFIG2);
> +	}
> +
> +	return 0;
> +}
> +
> +static unsigned int apple_mcc_opp_to_performance_state(struct generic_pm_domain *genpd,
> +						       struct dev_pm_opp *opp)
> +{
> +	return dev_pm_opp_get_level(opp);
> +}
> +
> +static int apple_mcc_probe(struct platform_device *pdev)
> +{
> +	struct device *dev = &pdev->dev;
> +	struct device_node *node = dev->of_node;

By convention mostly we call the variable "np".

> +	struct apple_mcc *mcc;
> +	int ret;
> +
> +	mcc = devm_kzalloc(dev, sizeof(*mcc), GFP_KERNEL);
> +	if (!mcc)
> +		return -ENOMEM;
> +
> +	mcc->reg_base = devm_platform_ioremap_resource(pdev, 0);
> +	if (IS_ERR(mcc->reg_base))
> +		return PTR_ERR(mcc->reg_base);
> +
> +	if (of_property_read_u32(node, "apple,num-channels", &mcc->num_channels)) {

Don't you have a limit of supported channels? It cannot be any uint32...

> +		dev_err(dev, "missing apple,num-channels property\n");

Use almost everywhere dev_err_probe - less code and you get error msg
printed.

> +		return -ENOENT;
> +	}
> +
> +	mcc->dev = dev;
> +	mcc->genpd.name = "apple-mcc-perf";
> +	mcc->genpd.opp_to_performance_state = apple_mcc_opp_to_performance_state;
> +	mcc->genpd.set_performance_state = apple_mcc_set_performance_state;
> +
> +	ret = pm_genpd_init(&mcc->genpd, NULL, false);
> +	if (ret < 0) {
> +		dev_err(dev, "pm_genpd_init failed\n");
> +		return ret;
> +	}
> +
> +	ret = of_genpd_add_provider_simple_noclk(node, &mcc->genpd);
> +	if (ret < 0) {
> +		dev_err(dev, "of_genpd_add_provider_simple failed\n");
> +		return ret;
> +	}
> +
> +	dev_info(dev, "Apple MCC performance driver initialized\n");

Please skip it, or at least make it a dev_dbg, you don't print any
valuable information here.

> +
> +	return 0;
> +}
> +
> +static const struct of_device_id apple_mcc_of_match[] = {
> +	{ .compatible = "apple,mcc" },
> +	{}
> +};
> +
> +MODULE_DEVICE_TABLE(of, apple_mcc_of_match);
> +
> +static struct platform_driver apple_mcc_driver = {
> +	.probe = apple_mcc_probe,
> +	.driver = {
> +		.name = "apple-mcc",
> +		.of_match_table = apple_mcc_of_match,
> +	},
> +};

module_platform_driver() goes here.

> +
> +MODULE_AUTHOR("Hector Martin <marcan@marcan.st>");
> +MODULE_DESCRIPTION("MCC memory controller performance tuning driver for Apple SoCs");
> +MODULE_LICENSE("GPL v2");

I think this will be "Dual MIT/GPL", based on your SPDX.

> +
> +module_platform_driver(apple_mcc_driver);
> 


Best regards,
Krzysztof

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-12  5:57         ` Hector Martin
@ 2021-10-12  9:26           ` Viresh Kumar
  2021-10-12  9:31             ` Hector Martin "marcan"
  2021-10-14  9:55           ` Ulf Hansson
  1 sibling, 1 reply; 40+ messages in thread
From: Viresh Kumar @ 2021-10-12  9:26 UTC (permalink / raw)
  To: Hector Martin
  Cc: Sibi Sankar, Saravana Kannan, linux-arm-kernel,
	Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

On 12-10-21, 14:57, Hector Martin wrote:
> On 12/10/2021 14.51, Viresh Kumar wrote:
> > On 12-10-21, 14:34, Hector Martin wrote:
> > > The table *is* assigned to a genpd (the memory controller), it's just that
> > > that genpd isn't actually a parent of the CPU device. Without the patch you
> > > end up with:
> > > 
> > > [    3.040060] cpu cpu4: Failed to set performance rate of cpu4: 0 (-19)
> > > [    3.042881] cpu cpu4: Failed to set required opps: -19
> > > [    3.045508] cpufreq: __target_index: Failed to change cpu frequency: -19
> > 
> > Hmm, Saravana and Sibi were working on a similar problem earlier and decided to
> > solve this using devfreq instead. Don't remember the exact series which got
> > merged for this, Sibi ?
> > 
> > If this part fails, how do you actually set the performance state of the memory
> > controller's genpd ?
> 
> The clock controller has the genpd as an actual power-domain parent, so it
> does it instead. From patch #7:
> 
> > +	if (cluster->has_pd)
> > +		dev_pm_genpd_set_performance_state(cluster->dev,
> > +						   dev_pm_opp_get_required_pstate(opp, 0));
> > +
> 
> This is arguably not entirely representative of how the hardware works,
> since technically the cluster switching couldn't care less what the memory
> controller is doing; it's a soft dependency, states that should be switched
> together but are not interdependent (in fact, the clock code does this
> unconditionally after the CPU p-state change, regardless of whether we're
> shifting up or down; this is, FWIW, the same order macOS uses, and it
> clearly doesn't matter which way you do it).

Yeah, I understand what you are doing. But the current patch is
incorrect in the sense that it can cause a bug on other platforms. To
make this work, you should rather set this genpd as parent of CPU
devices (which are doing anyway since you are updating them with CPU's
DVFS). With that the clk driver won't be required to do the magic
behind the scene.

-- 
viresh

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-12  9:26           ` Viresh Kumar
@ 2021-10-12  9:31             ` Hector Martin "marcan"
  2021-10-12  9:32               ` Viresh Kumar
  0 siblings, 1 reply; 40+ messages in thread
From: Hector Martin "marcan" @ 2021-10-12  9:31 UTC (permalink / raw)
  To: Viresh Kumar
  Cc: Sibi Sankar, Saravana Kannan, linux-arm-kernel,
	Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel



On 2021年10月12日 18:26:03 JST, Viresh Kumar <viresh.kumar@linaro.org> wrote:
>On 12-10-21, 14:57, Hector Martin wrote:
>> 
>> This is arguably not entirely representative of how the hardware works,
>> since technically the cluster switching couldn't care less what the memory
>> controller is doing; it's a soft dependency, states that should be switched
>> together but are not interdependent (in fact, the clock code does this
>> unconditionally after the CPU p-state change, regardless of whether we're
>> shifting up or down; this is, FWIW, the same order macOS uses, and it
>> clearly doesn't matter which way you do it).
>
>Yeah, I understand what you are doing. But the current patch is
>incorrect in the sense that it can cause a bug on other platforms. To
>make this work, you should rather set this genpd as parent of CPU
>devices (which are doing anyway since you are updating them with CPU's
>DVFS). With that the clk driver won't be required to do the magic
>behind the scene.
>

That doesn't work, though, because the CPUs aren't normal devices with runtime-pm. That was the first thing I tried :).

If you think this *should* be made to work instead then I can try that.


-- 
Hector Martin "marcan" (marcan@marcan.st)
Public key: https://mrcn.st/pub

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-12  9:31             ` Hector Martin "marcan"
@ 2021-10-12  9:32               ` Viresh Kumar
  2021-10-14  6:52                 ` Hector Martin
  0 siblings, 1 reply; 40+ messages in thread
From: Viresh Kumar @ 2021-10-12  9:32 UTC (permalink / raw)
  To: Hector Martin "marcan"
  Cc: Sibi Sankar, Saravana Kannan, linux-arm-kernel,
	Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

On 12-10-21, 18:31, Hector Martin "marcan" wrote:
> That doesn't work, though, because the CPUs aren't normal devices
> with runtime-pm. That was the first thing I tried :).

What's the exact problem with runtime PM here ?

> If you think this *should* be made to work instead then I can try that.

-- 
viresh

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 3/9] dt-bindings: clock: Add apple,cluster-clk binding
  2021-10-12  8:51   ` Krzysztof Kozlowski
@ 2021-10-12  9:35     ` Viresh Kumar
       [not found]       ` <D0DE08FE-562E-4A48-BCA0-9094DAFCA564@marcan.st>
  0 siblings, 1 reply; 40+ messages in thread
From: Viresh Kumar @ 2021-10-12  9:35 UTC (permalink / raw)
  To: Krzysztof Kozlowski
  Cc: Hector Martin, linux-arm-kernel, Alyssa Rosenzweig, Sven Peter,
	Marc Zyngier, Mark Kettenis, Michael Turquette, Stephen Boyd,
	Rob Herring, Viresh Kumar, Nishanth Menon, Catalin Marinas,
	Rafael J. Wysocki, Kevin Hilman, Ulf Hansson, linux-clk,
	devicetree, linux-pm, linux-kernel

Apart from what Krzysztof already said:

On 12-10-21, 10:51, Krzysztof Kozlowski wrote:
> On 11/10/2021 18:57, Hector Martin wrote:
> > +    pcluster_opp: opp-table-1 {
> > +        compatible = "operating-points-v2";
> > +        opp-shared;
> > +
> > +        opp01 {
> > +            opp-hz = /bits/ 64 <600000000>;
> > +            opp-microvolt = <781000>;
> > +            opp-level = <1>;

The opp-level thing wasn't designed to work this way, though it may
work just fine. It was designed as a unique key for power-domains,
which don't have opp-hz. The OPP core currently looks at 3 different
values, which can act as a unique key to identify the OPP. clk-rate,
bandwidth and level.

I think this is the first platform which has both hz and level in the
CPUs OPP table. What exactly is level in this case ?

Again, it may work fine, I just don't know where it may end up
breaking :)

-- 
viresh

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 3/9] dt-bindings: clock: Add apple,cluster-clk binding
       [not found]             ` <20211012095735.mhh2lzu52ohtotl6@vireshk-i7>
@ 2021-10-12 13:48               ` Hector Martin
  0 siblings, 0 replies; 40+ messages in thread
From: Hector Martin @ 2021-10-12 13:48 UTC (permalink / raw)
  To: Viresh Kumar
  Cc: Hector Martin, linux-arm-kernel, Alyssa Rosenzweig, Sven Peter,
	Marc Zyngier, Mark Kettenis, Michael Turquette, Stephen Boyd,
	Rob Herring, Nishanth Menon, Catalin Marinas, Rafael J. Wysocki,
	Kevin Hilman, Ulf Hansson, linux-clk, devicetree, linux-pm,
	linux-kernel, Krzysztof Kozlowski

On 12/10/2021 18.57, Viresh Kumar wrote:
> I didn't realize earlier that we have moved out of lists :)

Whoops, sorry, I was on mobile and must've hit the wrong reply button! 
My apologies.

> On 12-10-21, 18:54, Hector Martin "marcan" wrote:
>> Typically cpufreq-dt is used with clock drivers that directly take
>> the clock frequency and do whatever voodoo is necessary to set it
>> for the CPU. But here, the hardware just wants to know the index,
>> and does everything itself. So we need to encode that somewhere, to
>> avoid hardcoding it in the clock driver.
>>
>> In general, based on how these SoCs are designed, we're trying to
>> avoid having tables of volatile information in the drivers, and
>> instead keep everything in the DT. This means we have a good chance
>> that these drivers will continue to work with future SoC
>> generations, since Apple doesn't change register definitions
>> randomly most of the time.
> 
> Yeah I get that and it is actually better this way. I just wanted to
> point out that we didn't think of it this way earlier :)

Yeah, makes sense. Seems to work fine :)


-- 
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-12  9:32               ` Viresh Kumar
@ 2021-10-14  6:52                 ` Hector Martin
  2021-10-14  6:56                   ` Viresh Kumar
  0 siblings, 1 reply; 40+ messages in thread
From: Hector Martin @ 2021-10-14  6:52 UTC (permalink / raw)
  To: Viresh Kumar
  Cc: Sibi Sankar, Saravana Kannan, linux-arm-kernel,
	Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
	linux-clk, devicetree, linux-pm, linux-kernel

On 12/10/2021 18.32, Viresh Kumar wrote:
> On 12-10-21, 18:31, Hector Martin "marcan" wrote:
>> That doesn't work, though, because the CPUs aren't normal devices
>> with runtime-pm. That was the first thing I tried :).
> 
> What's the exact problem with runtime PM here ?

The CPU devices aren't attached to their genpd, so the required OPP
transition fails with the same error.

However, this was easier to fix than I expected. With this patch to
cpufreq-dt, it all works properly, and I can drop the parent genpd
from the clock node and related handling. Thoughts?

commit c4f88743374c1f4678ee7f17fb6cae30ded9ed59
Author: Hector Martin <marcan@marcan.st>
Date:   Thu Oct 14 15:47:45 2021 +0900

     cpufreq: dt: Attach CPU devices to power domains
     
     This allows the required-opps mechanism to work for CPU OPP tables,
     triggering specific OPP levels in a parent power domain.
     
     Signed-off-by: Hector Martin <marcan@marcan.st>

diff --git a/drivers/cpufreq/cpufreq-dt.c b/drivers/cpufreq/cpufreq-dt.c
index 8fcaba541539..5b22846b557d 100644
--- a/drivers/cpufreq/cpufreq-dt.c
+++ b/drivers/cpufreq/cpufreq-dt.c
@@ -16,6 +16,7 @@
  #include <linux/list.h>
  #include <linux/module.h>
  #include <linux/of.h>
+#include <linux/pm_domain.h>
  #include <linux/pm_opp.h>
  #include <linux/platform_device.h>
  #include <linux/regulator/consumer.h>
@@ -264,6 +265,16 @@ static int dt_cpufreq_early_init(struct device *dev, int cpu)
  		goto out;
  	}
  
+	/*
+	 * Attach the CPU device to its genpd domain (if any), to allow OPP
+	 * dependencies to be satisfied.
+	 */
+	ret = genpd_dev_pm_attach(cpu_dev);
+	if (ret <= 0) {
+		dev_err(cpu_dev, "Failed to attach CPU device to genpd\n");
+		goto out;
+	}
+
  	/*
  	 * The OPP table must be initialized, statically or dynamically, by this
  	 * point.

-- 
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub

^ permalink raw reply related	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-14  6:52                 ` Hector Martin
@ 2021-10-14  6:56                   ` Viresh Kumar
  2021-10-14  7:03                     ` Hector Martin
  0 siblings, 1 reply; 40+ messages in thread
From: Viresh Kumar @ 2021-10-14  6:56 UTC (permalink / raw)
  To: Ulf Hansson, Hector Martin
  Cc: Sibi Sankar, Saravana Kannan, linux-arm-kernel,
	Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, linux-clk,
	devicetree, linux-pm, linux-kernel

On 14-10-21, 15:52, Hector Martin wrote:
> The CPU devices aren't attached to their genpd, so the required OPP
> transition fails with the same error.
> 
> However, this was easier to fix than I expected. With this patch to
> cpufreq-dt, it all works properly, and I can drop the parent genpd
> from the clock node and related handling. Thoughts?
> 
> commit c4f88743374c1f4678ee7f17fb6cae30ded9ed59
> Author: Hector Martin <marcan@marcan.st>
> Date:   Thu Oct 14 15:47:45 2021 +0900
> 
>     cpufreq: dt: Attach CPU devices to power domains
>     This allows the required-opps mechanism to work for CPU OPP tables,
>     triggering specific OPP levels in a parent power domain.
>     Signed-off-by: Hector Martin <marcan@marcan.st>
> 
> diff --git a/drivers/cpufreq/cpufreq-dt.c b/drivers/cpufreq/cpufreq-dt.c
> index 8fcaba541539..5b22846b557d 100644
> --- a/drivers/cpufreq/cpufreq-dt.c
> +++ b/drivers/cpufreq/cpufreq-dt.c
> @@ -16,6 +16,7 @@
>  #include <linux/list.h>
>  #include <linux/module.h>
>  #include <linux/of.h>
> +#include <linux/pm_domain.h>
>  #include <linux/pm_opp.h>
>  #include <linux/platform_device.h>
>  #include <linux/regulator/consumer.h>
> @@ -264,6 +265,16 @@ static int dt_cpufreq_early_init(struct device *dev, int cpu)
>  		goto out;
>  	}
> +	/*
> +	 * Attach the CPU device to its genpd domain (if any), to allow OPP
> +	 * dependencies to be satisfied.
> +	 */
> +	ret = genpd_dev_pm_attach(cpu_dev);
> +	if (ret <= 0) {
> +		dev_err(cpu_dev, "Failed to attach CPU device to genpd\n");
> +		goto out;
> +	}
> +

Other platform do this from some other place I think.

Ulf, where should this code be moved ? cpu-clk driver ?

-- 
viresh

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 6/9] memory: apple: Add apple-mcc driver to manage MCC perf in Apple SoCs
  2021-10-12  9:19   ` Krzysztof Kozlowski
@ 2021-10-14  6:59     ` Hector Martin
  2021-10-14  7:36       ` Krzysztof Kozlowski
  0 siblings, 1 reply; 40+ messages in thread
From: Hector Martin @ 2021-10-14  6:59 UTC (permalink / raw)
  To: Krzysztof Kozlowski, linux-arm-kernel
  Cc: Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring, Viresh Kumar,
	Nishanth Menon, Catalin Marinas, Rafael J. Wysocki, Kevin Hilman,
	Ulf Hansson, linux-clk, devicetree, linux-pm, linux-kernel

On 12/10/2021 18.19, Krzysztof Kozlowski wrote:
>> +// SPDX-License-Identifier: GPL-2.0-only OR MIT
>> +/*
>> + * Apple SoC MCC memory controller performance control driver
>> + *
>> + * Copyright The Asahi Linux Contributors
> 
> Copyright date?

We've gone over this one a few times already; most copyright dates 
quickly become outdated and meaningless :)

See: 
https://www.linuxfoundation.org/blog/copyright-notices-in-open-source-software-projects/

>> +static int apple_mcc_probe(struct platform_device *pdev)
>> +{
>> +	struct device *dev = &pdev->dev;
>> +	struct device_node *node = dev->of_node;
> 
> By convention mostly we call the variable "np".

Ack, I'll change it for v2.

>> +	mcc->reg_base = devm_platform_ioremap_resource(pdev, 0);
>> +	if (IS_ERR(mcc->reg_base))
>> +		return PTR_ERR(mcc->reg_base);
>> +
>> +	if (of_property_read_u32(node, "apple,num-channels", &mcc->num_channels)) {
> 
> Don't you have a limit of supported channels? It cannot be any uint32...

Today, it's max 8. But if come Monday we find out Apple's new chips have 
16 channels and otherwise the same register layout, I'd much rather not 
have to change the driver...

>> +		dev_err(dev, "missing apple,num-channels property\n");
> 
> Use almost everywhere dev_err_probe - less code and you get error msg
> printed.

Heh, I didn't know about that one. Thanks!

>> +
>> +	dev_info(dev, "Apple MCC performance driver initialized\n");
> 
> Please skip it, or at least make it a dev_dbg, you don't print any
> valuable information here.

Ack, I'll remove this.

>> +static struct platform_driver apple_mcc_driver = {
>> +	.probe = apple_mcc_probe,
>> +	.driver = {
>> +		.name = "apple-mcc",
>> +		.of_match_table = apple_mcc_of_match,
>> +	},
>> +};
> 
> module_platform_driver() goes here.

Ack, will fix for v2.

> 
>> +
>> +MODULE_AUTHOR("Hector Martin <marcan@marcan.st>");
>> +MODULE_DESCRIPTION("MCC memory controller performance tuning driver for Apple SoCs");
>> +MODULE_LICENSE("GPL v2");
> 
> I think this will be "Dual MIT/GPL", based on your SPDX.

Ah, I didn't realize that was a valid option for MODULE_LICENSE. I guess 
anything containing "GPL" works with EXPORT_SYMBOL_GPL?

Thanks for the review!

-- 
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-14  6:56                   ` Viresh Kumar
@ 2021-10-14  7:03                     ` Hector Martin
  2021-10-14  7:22                       ` Viresh Kumar
  2021-10-14  7:23                       ` Hector Martin
  0 siblings, 2 replies; 40+ messages in thread
From: Hector Martin @ 2021-10-14  7:03 UTC (permalink / raw)
  To: Viresh Kumar, Ulf Hansson
  Cc: Sibi Sankar, Saravana Kannan, linux-arm-kernel,
	Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, linux-clk,
	devicetree, linux-pm, linux-kernel

On 14/10/2021 15.56, Viresh Kumar wrote:
>> +	/*
>> +	 * Attach the CPU device to its genpd domain (if any), to allow OPP
>> +	 * dependencies to be satisfied.
>> +	 */
>> +	ret = genpd_dev_pm_attach(cpu_dev);
>> +	if (ret <= 0) {
>> +		dev_err(cpu_dev, "Failed to attach CPU device to genpd\n");
>> +		goto out;
>> +	}
>> +
> 
> Other platform do this from some other place I think.
> 
> Ulf, where should this code be moved ? cpu-clk driver ?
> 

I see one driver that does this is drivers/clk/qcom/apcs-sdx55.c (via 
dev_pm_domain_attach). Though it only does it for CPU#0; we need to do 
it for all CPUs.

-- 
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-14  7:03                     ` Hector Martin
@ 2021-10-14  7:22                       ` Viresh Kumar
  2021-10-14  7:23                       ` Hector Martin
  1 sibling, 0 replies; 40+ messages in thread
From: Viresh Kumar @ 2021-10-14  7:22 UTC (permalink / raw)
  To: Hector Martin
  Cc: Ulf Hansson, Sibi Sankar, Saravana Kannan, linux-arm-kernel,
	Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, linux-clk,
	devicetree, linux-pm, linux-kernel

On 14-10-21, 16:03, Hector Martin wrote:
> On 14/10/2021 15.56, Viresh Kumar wrote:
> > > +	/*
> > > +	 * Attach the CPU device to its genpd domain (if any), to allow OPP
> > > +	 * dependencies to be satisfied.
> > > +	 */
> > > +	ret = genpd_dev_pm_attach(cpu_dev);
> > > +	if (ret <= 0) {
> > > +		dev_err(cpu_dev, "Failed to attach CPU device to genpd\n");
> > > +		goto out;
> > > +	}
> > > +
> > 
> > Other platform do this from some other place I think.
> > 
> > Ulf, where should this code be moved ? cpu-clk driver ?
> > 
> 
> I see one driver that does this is drivers/clk/qcom/apcs-sdx55.c (via
> dev_pm_domain_attach).

That may be a good place since you are already adding it and it is related to
CPU clk.

> Though it only does it for CPU#0; we need to do it
> for all CPUs.

Sure.

-- 
viresh

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-14  7:03                     ` Hector Martin
  2021-10-14  7:22                       ` Viresh Kumar
@ 2021-10-14  7:23                       ` Hector Martin
  2021-10-14 11:08                         ` Ulf Hansson
  1 sibling, 1 reply; 40+ messages in thread
From: Hector Martin @ 2021-10-14  7:23 UTC (permalink / raw)
  To: Viresh Kumar, Ulf Hansson
  Cc: Sibi Sankar, Saravana Kannan, linux-arm-kernel,
	Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, linux-clk,
	devicetree, linux-pm, linux-kernel

On 14/10/2021 16.03, Hector Martin wrote:
> On 14/10/2021 15.56, Viresh Kumar wrote:
>>> +	/*
>>> +	 * Attach the CPU device to its genpd domain (if any), to allow OPP
>>> +	 * dependencies to be satisfied.
>>> +	 */
>>> +	ret = genpd_dev_pm_attach(cpu_dev);
>>> +	if (ret <= 0) {
>>> +		dev_err(cpu_dev, "Failed to attach CPU device to genpd\n");
>>> +		goto out;
>>> +	}
>>> +
>>
>> Other platform do this from some other place I think.
>>
>> Ulf, where should this code be moved ? cpu-clk driver ?
>>
> 
> I see one driver that does this is drivers/clk/qcom/apcs-sdx55.c (via
> dev_pm_domain_attach). Though it only does it for CPU#0; we need to do
> it for all CPUs.

Looking into this further, I'm not sure I like the idea of doing this in 
the clocks driver. There might be locking issues since it gets 
instantiated twice and yet doesn't really itself know what subset of 
CPUs it applies to.

There's another driver that does this: 
drivers/cpuidle/cpuidle-psci-domain.c. That one specifically looks for a 
power domain called "psci". Perhaps it would make sense to make this 
generic in cpufreq-dt as per my prior patch, but explicitly request a 
"cpufreq" domain? That way only devicetrees that opt in to having this 
handled by cpufreq by naming it that way would get this behavior.

-- 
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 6/9] memory: apple: Add apple-mcc driver to manage MCC perf in Apple SoCs
  2021-10-14  6:59     ` Hector Martin
@ 2021-10-14  7:36       ` Krzysztof Kozlowski
  2021-10-14  7:52         ` Hector Martin
  0 siblings, 1 reply; 40+ messages in thread
From: Krzysztof Kozlowski @ 2021-10-14  7:36 UTC (permalink / raw)
  To: Hector Martin, linux-arm-kernel
  Cc: Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring, Viresh Kumar,
	Nishanth Menon, Catalin Marinas, Rafael J. Wysocki, Kevin Hilman,
	Ulf Hansson, linux-clk, devicetree, linux-pm, linux-kernel

On 14/10/2021 08:59, Hector Martin wrote:
> On 12/10/2021 18.19, Krzysztof Kozlowski wrote:
>>> +// SPDX-License-Identifier: GPL-2.0-only OR MIT
>>> +/*
>>> + * Apple SoC MCC memory controller performance control driver
>>> + *
>>> + * Copyright The Asahi Linux Contributors
>>
>> Copyright date?
> 
> We've gone over this one a few times already; most copyright dates 
> quickly become outdated and meaningless :)
> 
> See: 
> https://www.linuxfoundation.org/blog/copyright-notices-in-open-source-software-projects/
> 
>>> +static int apple_mcc_probe(struct platform_device *pdev)
>>> +{
>>> +	struct device *dev = &pdev->dev;
>>> +	struct device_node *node = dev->of_node;
>>
>> By convention mostly we call the variable "np".
> 
> Ack, I'll change it for v2.
> 
>>> +	mcc->reg_base = devm_platform_ioremap_resource(pdev, 0);
>>> +	if (IS_ERR(mcc->reg_base))
>>> +		return PTR_ERR(mcc->reg_base);
>>> +
>>> +	if (of_property_read_u32(node, "apple,num-channels", &mcc->num_channels)) {
>>
>> Don't you have a limit of supported channels? It cannot be any uint32...
> 
> Today, it's max 8. But if come Monday we find out Apple's new chips have 
> 16 channels and otherwise the same register layout, I'd much rather not 
> have to change the driver...

OK, however if the driver ever receives different DT with a different
value, it will accept it unconditionally and go via address space. I am
just saying that being conservative on received values is safer, but I
am fine with skipping this problem. At the end we trust DT that it will
always match the kernel, don't we? Oh wait, someone can use DT from
other kernel in this one...

> 
>>> +		dev_err(dev, "missing apple,num-channels property\n");
>>
>> Use almost everywhere dev_err_probe - less code and you get error msg
>> printed.
> 
> Heh, I didn't know about that one. Thanks!
> 
>>> +
>>> +	dev_info(dev, "Apple MCC performance driver initialized\n");
>>
>> Please skip it, or at least make it a dev_dbg, you don't print any
>> valuable information here.
> 
> Ack, I'll remove this.
> 
>>> +static struct platform_driver apple_mcc_driver = {
>>> +	.probe = apple_mcc_probe,
>>> +	.driver = {
>>> +		.name = "apple-mcc",
>>> +		.of_match_table = apple_mcc_of_match,
>>> +	},
>>> +};
>>
>> module_platform_driver() goes here.
> 
> Ack, will fix for v2.
> 
>>
>>> +
>>> +MODULE_AUTHOR("Hector Martin <marcan@marcan.st>");
>>> +MODULE_DESCRIPTION("MCC memory controller performance tuning driver for Apple SoCs");
>>> +MODULE_LICENSE("GPL v2");
>>
>> I think this will be "Dual MIT/GPL", based on your SPDX.
> 
> Ah, I didn't realize that was a valid option for MODULE_LICENSE. I guess 
> anything containing "GPL" works with EXPORT_SYMBOL_GPL?

I don't think exporting symbols is related to how you license your code.


Best regards,
Krzysztof

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 6/9] memory: apple: Add apple-mcc driver to manage MCC perf in Apple SoCs
  2021-10-14  7:36       ` Krzysztof Kozlowski
@ 2021-10-14  7:52         ` Hector Martin
  2021-10-14  8:04           ` Krzysztof Kozlowski
  0 siblings, 1 reply; 40+ messages in thread
From: Hector Martin @ 2021-10-14  7:52 UTC (permalink / raw)
  To: Krzysztof Kozlowski, linux-arm-kernel
  Cc: Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring, Viresh Kumar,
	Nishanth Menon, Catalin Marinas, Rafael J. Wysocki, Kevin Hilman,
	Ulf Hansson, linux-clk, devicetree, linux-pm, linux-kernel

On 14/10/2021 16.36, Krzysztof Kozlowski wrote:
> On 14/10/2021 08:59, Hector Martin wrote:
>>> Don't you have a limit of supported channels? It cannot be any uint32...
>>
>> Today, it's max 8. But if come Monday we find out Apple's new chips have
>> 16 channels and otherwise the same register layout, I'd much rather not
>> have to change the driver...
> 
> OK, however if the driver ever receives different DT with a different
> value, it will accept it unconditionally and go via address space. I am
> just saying that being conservative on received values is safer, but I
> am fine with skipping this problem. At the end we trust DT that it will
> always match the kernel, don't we? Oh wait, someone can use DT from
> other kernel in this one...

DTs using these compatibles should have the same register layout, and 
should work with this driver; if a new chip comes out that has a 
different register layout we will change the compatibles (both) and 
therefore older kernels won't bind at all. If it has the same layout 
we'll keep the base compatible, `reg` will grow as needed to accomodate 
the extra channels, and e.g. num-channels=16 will then just work on 
older kernels with no changes.

Obviously a broken DT with an insane value here would crash the driver, 
but so would any other number of crazy DT things; however, I don't 
expect that to ever happen.

There's also the case where we end up with multiple memory controllers 
at discrete offsets (e.g. rumored multi-die configurations); in that 
case we'll end up with multiple genpd parents and have to add code to 
support that, and in the meantime older kernels will just have broken 
cpufreq on the p-cores. But I think that is ~acceptable as long as the 
system boots; we don't expect to be able to *fully* support newer SoCs 
on older kernels with no code changes. What I'm aiming for is just 
making the system work, hopefully with NVMe and USB and a dumb 
framebuffer, so that distro installers can run and then users can later 
install a proper up to date kernel will full support for the new SoC.

>> Ah, I didn't realize that was a valid option for MODULE_LICENSE. I guess
>> anything containing "GPL" works with EXPORT_SYMBOL_GPL?
> 
> I don't think exporting symbols is related to how you license your code.

It is; only modules with a GPL-compatible MODULE_LICENSE get to use 
symbols exported via EXPORT_SYMBOL_GPL.

See kernel/module.c for the symbol lookup logic and 
include/linux/license.h for the logic to check the string (seems like 
"Dual MIT/GPL" is explicitly whitelisted there).

Of course, this is a futile effort, as ~every time I see a proprietary 
module in some embedded device, it either falsely declares itself to be 
GPL, or they have a shim module that re-exports GPL symbols as non-GPL.

-- 
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 6/9] memory: apple: Add apple-mcc driver to manage MCC perf in Apple SoCs
  2021-10-14  7:52         ` Hector Martin
@ 2021-10-14  8:04           ` Krzysztof Kozlowski
  2021-10-14  8:31             ` Hector Martin
  0 siblings, 1 reply; 40+ messages in thread
From: Krzysztof Kozlowski @ 2021-10-14  8:04 UTC (permalink / raw)
  To: Hector Martin, linux-arm-kernel
  Cc: Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring, Viresh Kumar,
	Nishanth Menon, Catalin Marinas, Rafael J. Wysocki, Kevin Hilman,
	Ulf Hansson, linux-clk, devicetree, linux-pm, linux-kernel

On 14/10/2021 09:52, Hector Martin wrote:
> On 14/10/2021 16.36, Krzysztof Kozlowski wrote:

(...)

> 
>>> Ah, I didn't realize that was a valid option for MODULE_LICENSE. I guess
>>> anything containing "GPL" works with EXPORT_SYMBOL_GPL?
>>
>> I don't think exporting symbols is related to how you license your code.
> 
> It is; only modules with a GPL-compatible MODULE_LICENSE get to use 
> symbols exported via EXPORT_SYMBOL_GPL.

Although there might be such correlation but it's not a rule. You can
have a GPL module exporting symbols without GPL requirement
(EXPORT_SYMBOLS). You can have a GPL+MIT module exporting symbols as
GPL. Obviously you cannot have a non-GPL module, as we do not accept
these and there is no such choice.

So answering your question that "GPL" works with EXPORT_SYMBOL_GPL -
everything is GPL but it works with both EXPORT_SYMBOL and
EXPORT_SYMBOL_GPL.

> 
> See kernel/module.c for the symbol lookup logic and 
> include/linux/license.h for the logic to check the string (seems like 
> "Dual MIT/GPL" is explicitly whitelisted there).

Not related to export symbol. It is used for determining the tainted
kernel via other licenses.

> 
> Of course, this is a futile effort, as ~every time I see a proprietary 
> module in some embedded device, it either falsely declares itself to be 
> GPL, or they have a shim module that re-exports GPL symbols as non-GPL.
> 

This is being removed soon (or already).


Best regards,
Krzysztof

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 6/9] memory: apple: Add apple-mcc driver to manage MCC perf in Apple SoCs
  2021-10-14  8:04           ` Krzysztof Kozlowski
@ 2021-10-14  8:31             ` Hector Martin
  0 siblings, 0 replies; 40+ messages in thread
From: Hector Martin @ 2021-10-14  8:31 UTC (permalink / raw)
  To: Krzysztof Kozlowski, linux-arm-kernel
  Cc: Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring, Viresh Kumar,
	Nishanth Menon, Catalin Marinas, Rafael J. Wysocki, Kevin Hilman,
	Ulf Hansson, linux-clk, devicetree, linux-pm, linux-kernel

On 14/10/2021 17.04, Krzysztof Kozlowski wrote:
>> It is; only modules with a GPL-compatible MODULE_LICENSE get to use
>> symbols exported via EXPORT_SYMBOL_GPL.
> 
> Although there might be such correlation but it's not a rule. You can
> have a GPL module exporting symbols without GPL requirement
> (EXPORT_SYMBOLS). You can have a GPL+MIT module exporting symbols as
> GPL. Obviously you cannot have a non-GPL module, as we do not accept
> these and there is no such choice.

What I mean is that modules can only import GPL symbols if they 
themselves are GPL compatible. What I didn't know is that "Dual MIT/GPL" 
is a valid string for MODULE_LICENSE to qualify as such.

>> See kernel/module.c for the symbol lookup logic and
>> include/linux/license.h for the logic to check the string (seems like
>> "Dual MIT/GPL" is explicitly whitelisted there).
> 
> Not related to export symbol. It is used for determining the tainted
> kernel via other licenses.
> 

Not just that; that module taint is used as a filter so that 
non-GPL-compatible modules are technically prevented from resolving 
EXPORT_SYMBOL_GPL symbols.

>> Of course, this is a futile effort, as ~every time I see a proprietary
>> module in some embedded device, it either falsely declares itself to be
>> GPL, or they have a shim module that re-exports GPL symbols as non-GPL.
>>
> 
> This is being removed soon (or already).

? Good luck getting proprietary embedded vendors to start following 
licenses... :)

-- 
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-12  5:57         ` Hector Martin
  2021-10-12  9:26           ` Viresh Kumar
@ 2021-10-14  9:55           ` Ulf Hansson
  2021-10-14 11:43             ` Hector Martin
  1 sibling, 1 reply; 40+ messages in thread
From: Ulf Hansson @ 2021-10-14  9:55 UTC (permalink / raw)
  To: Hector Martin
  Cc: Viresh Kumar, Sibi Sankar, Saravana Kannan, Linux ARM,
	Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, linux-clk,
	DTML, Linux PM, Linux Kernel Mailing List

On Tue, 12 Oct 2021 at 07:57, Hector Martin <marcan@marcan.st> wrote:
>
> On 12/10/2021 14.51, Viresh Kumar wrote:
> > On 12-10-21, 14:34, Hector Martin wrote:
> >> The table *is* assigned to a genpd (the memory controller), it's just that
> >> that genpd isn't actually a parent of the CPU device. Without the patch you
> >> end up with:
> >>
> >> [    3.040060] cpu cpu4: Failed to set performance rate of cpu4: 0 (-19)
> >> [    3.042881] cpu cpu4: Failed to set required opps: -19
> >> [    3.045508] cpufreq: __target_index: Failed to change cpu frequency: -19
> >
> > Hmm, Saravana and Sibi were working on a similar problem earlier and decided to
> > solve this using devfreq instead. Don't remember the exact series which got
> > merged for this, Sibi ?
> >
> > If this part fails, how do you actually set the performance state of the memory
> > controller's genpd ?
>
> The clock controller has the genpd as an actual power-domain parent, so
> it does it instead. From patch #7:
>
> > +     if (cluster->has_pd)
> > +             dev_pm_genpd_set_performance_state(cluster->dev,
> > +                                                dev_pm_opp_get_required_pstate(opp, 0));
> > +
>
> This is arguably not entirely representative of how the hardware works,
> since technically the cluster switching couldn't care less what the
> memory controller is doing; it's a soft dependency, states that should
> be switched together but are not interdependent (in fact, the clock code
> does this unconditionally after the CPU p-state change, regardless of
> whether we're shifting up or down; this is, FWIW, the same order macOS
> uses, and it clearly doesn't matter which way you do it).

Yes, this sounds like you should move away from modeling the memory
part as a parent genpd for the CPUs' genpd.

As Viresh pointed out, a devfreq driver seems like a better way to do
this. As a matter of fact, there are already devfreq drivers that do
this, unless I am mistaken.

It looks like devfreq providers are listening to opp/cpufreq
notifiers, as to get an indication of when it could make sense to
change a performance state.

In some cases the devfreq provider is also modeled as an interconnect
provider, allowing consumers to specify memory bandwidth constraints,
which may trigger a new performance state to be set for the memory
controller.

In the tegra case, the memory controller is modelled as an
interconnect provider and the devfreq node is modelled as an
interconnect-consumer of the memory controller. Perhaps this can work
for apple SoCs too?

That said, perhaps as an option to move forward, we can try to get the
cpufreq pieces solved first. Then as a step on top, add the
performance scaling for the memory controller?

>
> --
> Hector Martin (marcan@marcan.st)
> Public Key: https://mrcn.st/pub

Kind regards
Uffe

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-14  7:23                       ` Hector Martin
@ 2021-10-14 11:08                         ` Ulf Hansson
  0 siblings, 0 replies; 40+ messages in thread
From: Ulf Hansson @ 2021-10-14 11:08 UTC (permalink / raw)
  To: Hector Martin
  Cc: Viresh Kumar, Sibi Sankar, Saravana Kannan, Linux ARM,
	Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, linux-clk,
	DTML, Linux PM, Linux Kernel Mailing List

On Thu, 14 Oct 2021 at 09:23, Hector Martin <marcan@marcan.st> wrote:
>
> On 14/10/2021 16.03, Hector Martin wrote:
> > On 14/10/2021 15.56, Viresh Kumar wrote:
> >>> +   /*
> >>> +    * Attach the CPU device to its genpd domain (if any), to allow OPP
> >>> +    * dependencies to be satisfied.
> >>> +    */
> >>> +   ret = genpd_dev_pm_attach(cpu_dev);
> >>> +   if (ret <= 0) {
> >>> +           dev_err(cpu_dev, "Failed to attach CPU device to genpd\n");
> >>> +           goto out;
> >>> +   }
> >>> +
> >>
> >> Other platform do this from some other place I think.
> >>
> >> Ulf, where should this code be moved ? cpu-clk driver ?
> >>
> >
> > I see one driver that does this is drivers/clk/qcom/apcs-sdx55.c (via
> > dev_pm_domain_attach). Though it only does it for CPU#0; we need to do
> > it for all CPUs.
>
> Looking into this further, I'm not sure I like the idea of doing this in
> the clocks driver. There might be locking issues since it gets
> instantiated twice and yet doesn't really itself know what subset of
> CPUs it applies to.

I agree. I suggest you look into using a genpd provider and hook up
all CPU's devices to it. I think that is what Viresh also suggested
earlier - and this makes most sense to me.

As a reference you may have a look at some Qcom platforms that already use this:

arch/arm64/boot/dts/qcom/qcs404.dtsi

drivers/cpufreq/qcom-cpufreq-nvmem.c:
To hook up CPU devices to their PM domains (genpds) - it calls
dev_pm_opp_attach_genpd(), which is a kind of wrapper for
dev_pm_domain_attach_by_name().

drivers/soc/qcom/cpr.c
Registers the genpd provider that is capable of dealing with
performance states/OPPs for CPUs.

>
> There's another driver that does this:
> drivers/cpuidle/cpuidle-psci-domain.c. That one specifically looks for a
> power domain called "psci". Perhaps it would make sense to make this
> generic in cpufreq-dt as per my prior patch, but explicitly request a
> "cpufreq" domain? That way only devicetrees that opt in to having this
> handled by cpufreq by naming it that way would get this behavior.

That sounds like an idea that is worth exploring. In this way, the
only thing that needs to be implemented for new cases would be the
genpd provider driver.

BTW, as you will figure out by looking at the above references, for
the qcom case we are using "cpr" as the domain name for cpufreq. Of
course, that doesn't mean we can use "cpufreq" (or whatever name that
makes sense) going forward for new cases.

>
> --
> Hector Martin (marcan@marcan.st)
> Public Key: https://mrcn.st/pub

Kind regards
Uffe

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-14  9:55           ` Ulf Hansson
@ 2021-10-14 11:43             ` Hector Martin
  2021-10-14 12:55               ` Ulf Hansson
  0 siblings, 1 reply; 40+ messages in thread
From: Hector Martin @ 2021-10-14 11:43 UTC (permalink / raw)
  To: Ulf Hansson
  Cc: Viresh Kumar, Sibi Sankar, Saravana Kannan, Linux ARM,
	Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, linux-clk,
	DTML, Linux PM, Linux Kernel Mailing List

On 14/10/2021 18.55, Ulf Hansson wrote:
> Yes, this sounds like you should move away from modeling the memory
> part as a parent genpd for the CPUs' genpd.
> 
> As Viresh pointed out, a devfreq driver seems like a better way to do
> this. As a matter of fact, there are already devfreq drivers that do
> this, unless I am mistaken.
> 
> It looks like devfreq providers are listening to opp/cpufreq
> notifiers, as to get an indication of when it could make sense to
> change a performance state.
> 
> In some cases the devfreq provider is also modeled as an interconnect
> provider, allowing consumers to specify memory bandwidth constraints,
> which may trigger a new performance state to be set for the memory
> controller.
> 
> In the tegra case, the memory controller is modelled as an
> interconnect provider and the devfreq node is modelled as an
> interconnect-consumer of the memory controller. Perhaps this can work
> for apple SoCs too?

I was poking around and noticed the OPP core can already integrate with 
interconnect requirements, so perhaps the memory controller can be an 
interconnect provider, and the CPU nodes can directly reference it as a 
consumer? This seems like a more accurate model of what the hardware 
does, and I think I saw some devices doing this already.

(only problem is I have no idea of the actual bandwidth numbers involved 
here... I'll have to run some benchmarks to make sure this isn't just 
completely dummy data)

> 
> That said, perhaps as an option to move forward, we can try to get the
> cpufreq pieces solved first. Then as a step on top, add the
> performance scaling for the memory controller?

Sure; that's a pretty much independent part of this patchset, though I'm 
thinking I might as well try some things out for v2 anyway; if it looks 
like it'll take longer we can split it out and do just the cpufreq side.

-- 
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-14 11:43             ` Hector Martin
@ 2021-10-14 12:55               ` Ulf Hansson
  2021-10-14 17:02                 ` Hector Martin
  0 siblings, 1 reply; 40+ messages in thread
From: Ulf Hansson @ 2021-10-14 12:55 UTC (permalink / raw)
  To: Hector Martin
  Cc: Viresh Kumar, Sibi Sankar, Saravana Kannan, Linux ARM,
	Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, linux-clk,
	DTML, Linux PM, Linux Kernel Mailing List

On Thu, 14 Oct 2021 at 13:43, Hector Martin <marcan@marcan.st> wrote:
>
> On 14/10/2021 18.55, Ulf Hansson wrote:
> > Yes, this sounds like you should move away from modeling the memory
> > part as a parent genpd for the CPUs' genpd.
> >
> > As Viresh pointed out, a devfreq driver seems like a better way to do
> > this. As a matter of fact, there are already devfreq drivers that do
> > this, unless I am mistaken.
> >
> > It looks like devfreq providers are listening to opp/cpufreq
> > notifiers, as to get an indication of when it could make sense to
> > change a performance state.
> >
> > In some cases the devfreq provider is also modeled as an interconnect
> > provider, allowing consumers to specify memory bandwidth constraints,
> > which may trigger a new performance state to be set for the memory
> > controller.
> >
> > In the tegra case, the memory controller is modelled as an
> > interconnect provider and the devfreq node is modelled as an
> > interconnect-consumer of the memory controller. Perhaps this can work
> > for apple SoCs too?
>
> I was poking around and noticed the OPP core can already integrate with
> interconnect requirements, so perhaps the memory controller can be an
> interconnect provider, and the CPU nodes can directly reference it as a
> consumer? This seems like a more accurate model of what the hardware
> does, and I think I saw some devices doing this already.

Yeah, that could work too. And, yes, I agree, it may be a better
description of the HW.

>
> (only problem is I have no idea of the actual bandwidth numbers involved
> here... I'll have to run some benchmarks to make sure this isn't just
> completely dummy data)
>
> >
> > That said, perhaps as an option to move forward, we can try to get the
> > cpufreq pieces solved first. Then as a step on top, add the
> > performance scaling for the memory controller?
>
> Sure; that's a pretty much independent part of this patchset, though I'm
> thinking I might as well try some things out for v2 anyway; if it looks
> like it'll take longer we can split it out and do just the cpufreq side.

In any case, I do my best to help with review.

Kind regards
Uffe

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-14 12:55               ` Ulf Hansson
@ 2021-10-14 17:02                 ` Hector Martin
  2021-10-15 11:26                   ` Ulf Hansson
  0 siblings, 1 reply; 40+ messages in thread
From: Hector Martin @ 2021-10-14 17:02 UTC (permalink / raw)
  To: Ulf Hansson
  Cc: Viresh Kumar, Sibi Sankar, Saravana Kannan, Linux ARM,
	Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, linux-clk,
	DTML, Linux PM, Linux Kernel Mailing List

On 14/10/2021 21.55, Ulf Hansson wrote:
> On Thu, 14 Oct 2021 at 13:43, Hector Martin <marcan@marcan.st> wrote:
>> I was poking around and noticed the OPP core can already integrate with
>> interconnect requirements, so perhaps the memory controller can be an
>> interconnect provider, and the CPU nodes can directly reference it as a
>> consumer? This seems like a more accurate model of what the hardware
>> does, and I think I saw some devices doing this already.
> 
> Yeah, that could work too. And, yes, I agree, it may be a better
> description of the HW.
> 
>>
>> (only problem is I have no idea of the actual bandwidth numbers involved
>> here... I'll have to run some benchmarks to make sure this isn't just
>> completely dummy data)
>>

So... I tried getting bandwidth numbers and failed. It seems these 
registers don't actually affect peak performance in any measurable way. 
I'm also getting almost the same GeekBench scores on macOS with and 
without this mechanism enabled, although there is one subtest that seems 
to show a measurable difference.

My current guess is this is something more subtle (latencies? idle 
timers and such?) than a performance state. If that is the case, do you 
have any ideas as to the best way to model it in Linux? Should we even 
bother if it mostly has a minimal performance gain for typical workloads?

I'll try to do some latency tests, see if I can make sense of what it's 
actually doing.

-- 
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist
  2021-10-14 17:02                 ` Hector Martin
@ 2021-10-15 11:26                   ` Ulf Hansson
  0 siblings, 0 replies; 40+ messages in thread
From: Ulf Hansson @ 2021-10-15 11:26 UTC (permalink / raw)
  To: Hector Martin
  Cc: Viresh Kumar, Sibi Sankar, Saravana Kannan, Linux ARM,
	Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Stephen Boyd, Rob Herring,
	Krzysztof Kozlowski, Viresh Kumar, Nishanth Menon,
	Catalin Marinas, Rafael J. Wysocki, Kevin Hilman, linux-clk,
	DTML, Linux PM, Linux Kernel Mailing List

On Thu, 14 Oct 2021 at 19:02, Hector Martin <marcan@marcan.st> wrote:
>
> On 14/10/2021 21.55, Ulf Hansson wrote:
> > On Thu, 14 Oct 2021 at 13:43, Hector Martin <marcan@marcan.st> wrote:
> >> I was poking around and noticed the OPP core can already integrate with
> >> interconnect requirements, so perhaps the memory controller can be an
> >> interconnect provider, and the CPU nodes can directly reference it as a
> >> consumer? This seems like a more accurate model of what the hardware
> >> does, and I think I saw some devices doing this already.
> >
> > Yeah, that could work too. And, yes, I agree, it may be a better
> > description of the HW.
> >
> >>
> >> (only problem is I have no idea of the actual bandwidth numbers involved
> >> here... I'll have to run some benchmarks to make sure this isn't just
> >> completely dummy data)
> >>
>
> So... I tried getting bandwidth numbers and failed. It seems these
> registers don't actually affect peak performance in any measurable way.
> I'm also getting almost the same GeekBench scores on macOS with and
> without this mechanism enabled, although there is one subtest that seems
> to show a measurable difference.
>
> My current guess is this is something more subtle (latencies? idle
> timers and such?) than a performance state. If that is the case, do you
> have any ideas as to the best way to model it in Linux? Should we even
> bother if it mostly has a minimal performance gain for typical workloads?

For latency constraints, we have dev_pm_qos. This will make the genpd
governor, to prevent deeper idle states for the device and its
corresponding PM domain (genpd). But that doesn't sound like a good
fit here.

If you are right, it rather sounds like there is some kind of
quiescence mode of the memory controller that can be prevented. But I
have no clue, of course. :-)

>
> I'll try to do some latency tests, see if I can make sense of what it's
> actually doing.
>

Kind regards
Uffe

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 7/9] clk: apple: Add clk-apple-cluster driver to manage CPU p-states
       [not found]   ` <163424925931.1688384.9647104000291025081@swboyd.mtv.corp.google.com>
@ 2021-10-17  9:16     ` Hector Martin
  0 siblings, 0 replies; 40+ messages in thread
From: Hector Martin @ 2021-10-17  9:16 UTC (permalink / raw)
  To: Stephen Boyd, linux-arm-kernel
  Cc: Alyssa Rosenzweig, Sven Peter, Marc Zyngier, Mark Kettenis,
	Michael Turquette, Rob Herring, Krzysztof Kozlowski,
	Viresh Kumar, Nishanth Menon, Catalin Marinas, Rafael J. Wysocki,
	Kevin Hilman, Ulf Hansson, linux-clk, devicetree, linux-pm,
	linux-kernel

On 15/10/2021 07.07, Stephen Boyd wrote:
> This looks bad from a locking perspective. How is lockdep holding up
> with this driver? We're underneath the prepare lock here and we're
> setting a couple level registers which is all good but now we're calling
> into genpd code and who knows what's going to happen locking wise.

It seems this is all going away given the other discussion threads point 
towards handling this directly via OPP in the cpufreq-dt driver. I'll 
run whatever I end up with for v2 through lockdep though, good call!

> I don't actually see anything in here that indicates this is supposed to
> be a clk provider. Is it being modeled as a clk so that it can use
> cpufreq-dt? If it was a clk provider I'd expect it to be looking at
> parent clk rates, and reading hardware to calculate frequencies based on
> dividers and multipliers, etc. None of that is happening here.
> 
> Why not write a cpufreq driver, similar to qcom-cpufreq-hw.c that looks
> through the OPP table and then writes the value into the pstate
> registers? The registers in here look awfully similar to the qcom
> hardware. I don't know what the DESIRED1 and DESIRED2 registers are for
> though. Maybe they're so that one or the other frequency can be used if
> available? Like a min/max?
> 
> Either way, writing this as a cpufreq driver avoids the clk framework
> entirely which is super great for me :) It also avoids locking headaches
> from the clk prepare lock, and it also lets you support lockless cpufreq
> transitions by implementing the fast_switch function. I don't see any
> downsides to the cpufreq driver approach.

I wasn't too sure about this approach; I thought using a clk provider 
would end up simplifying things since I could use the cpufreq-dt 
machinery to take care of all the OPP stuff, and a lot of SoCs seemed to 
be going that way, but it seems cpufreq might be a better approach for 
this SoC?

There can only be one cpufreq driver instance, while I used two clock 
controllers to model the two clusters. So in the cpufreq case, the 
driver itself would have to deal with all potential CPU cluster 
instances/combinations itself. Not sure how much more code that will be, 
hopefully not too much...

I see qcom-cpufreq-hw uses a qcom,freq-domain prop to link CPUs to the 
cpufreq domains. cpufreq-dt and vexpress-spc-cpufreq instead use 
dev_pm_opp_get_sharing_cpus to look for shared OPP tables. Is there a 
reason not to do it that way and avoid the vendor prop? I guess the prop 
is more explicit while the sharing approach would have an implicit order 
dependency (i.e. CPUs are always grouped by cluster and clusters are 
listed in /cpus in the same order as in the cpufreq node)...

(Ack on the other comments, but if this becomes a cpufreq driver most of 
it is going to end up rewritten... :))

For the cpufreq case, do you have any suggestions as to how to relate it 
to the memory controller configuration tweaks? Ideally this would go 
through the OPP tables so it can be customized for future SoCs without 
stuff hardcoded in the driver... it seems the configuration affects 
power saving behavior / latencies, so it doesn't quite match the 
interconnect framework bandwidth request stuff. I'm also not sure how 
this would affect fast_switch, since going through those frameworks 
might imply locks... we might even find ourselves with a situation in 
the near future where multiple cpufreq policies can request memory 
controller latency reduction independently; I can come up with how to do 
this locklessly using atomics, but I can't imagine that being workable 
with higher-level frameworks, it would have to be a vendor-specific 
mechanism at that point...

-- 
Hector Martin (marcan@marcan.st)
Public Key: https://mrcn.st/pub

^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: [RFC PATCH 2/9] dt-bindings: memory-controller: Add apple,mcc binding
  2021-10-12  8:48   ` Krzysztof Kozlowski
@ 2021-10-19 22:43     ` Rob Herring
  0 siblings, 0 replies; 40+ messages in thread
From: Rob Herring @ 2021-10-19 22:43 UTC (permalink / raw)
  To: Krzysztof Kozlowski
  Cc: Hector Martin, linux-arm-kernel, Alyssa Rosenzweig, Sven Peter,
	Marc Zyngier, Mark Kettenis, Michael Turquette, Stephen Boyd,
	Viresh Kumar, Nishanth Menon, Catalin Marinas, Rafael J. Wysocki,
	Kevin Hilman, Ulf Hansson, linux-clk, devicetree, linux-pm,
	linux-kernel

On Tue, Oct 12, 2021 at 10:48:12AM +0200, Krzysztof Kozlowski wrote:
> On 11/10/2021 18:57, Hector Martin wrote:
> > This device represents the memory controller in Apple SoCs, and is
> > chiefly in charge of adjusting performance characteristics according to
> > system demand.
> > 
> > Signed-off-by: Hector Martin <marcan@marcan.st>
> > ---
> >  .../memory-controllers/apple,mcc.yaml         | 80 +++++++++++++++++++
> >  .../opp/apple,mcc-operating-points.yaml       | 62 ++++++++++++++
> >  2 files changed, 142 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml
> >  create mode 100644 Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml
> > 
> > diff --git a/Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml b/Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml
> > new file mode 100644
> > index 000000000000..0774f10e65ed
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/memory-controllers/apple,mcc.yaml
> > @@ -0,0 +1,80 @@
> > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> > +%YAML 1.2
> > +---
> > +$id: http://devicetree.org/schemas/memory-controllers/apple,mcc.yaml#
> > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > +
> > +title: Apple SoC MCC memory controller performance controls
> > +
> > +maintainers:
> > +  - Hector Martin <marcan@marcan.st>
> > +
> > +description: |
> > +  Apple SoCs contain a multichannel memory controller that can have its
> > +  configuration changed to adjust to changing performance requirements from
> > +  the rest of the SoC. This node represents the controller and provides a
> > +  power domain provider that downstream devices can use to adjust the memory
> > +  controller performance level.
> > +
> > +properties:
> > +  compatible:
> > +    items:
> > +      - enum:
> > +          - apple,t8103-mcc
> > +      - const: apple,mcc
> > +
> > +  reg:
> > +    maxItems: 1
> > +
> > +  "#power-domain-cells":
> > +    const: 0
> > +
> > +  operating-points-v2:
> > +    $ref: /schemas/types.yaml#/definitions/phandle-array
> > +    description:
> > +      A reference to the OPP table describing the memory controller performance
> > +      levels. Each OPP node should contain an `apple,memory-perf-config`
> > +      property that contains the configuration values for that performance
> > +      level.
> > +
> > +  apple,num-channels:
> > +    $ref: /schemas/types.yaml#/definitions/uint32
> > +    description:
> > +      The number of memory channels in use.
> > +
> > +required:
> > +  - compatible
> > +  - reg
> > +  - "#power-domain-cells"
> > +  - operating-points-v2
> > +  - apple,num-channels
> > +
> > +additionalProperties: false
> > +
> > +examples:
> > +  # See clock/apple,cluster-clock.yaml for an example of downstream usage.
> > +  - |
> > +    mcc_opp: opp-table-2 {
> > +        compatible = "operating-points-v2";
> 
> apple,mcc-operating-points?

+1


> > +
> > +        mcc_lowperf: opp0 {
> > +            opp-level = <0>;
> > +            apple,memory-perf-config = <0x813057f 0x1800180>;
> > +        };
> > +        mcc_highperf: opp1 {
> > +            opp-level = <1>;
> > +            apple,memory-perf-config = <0x133 0x55555340>;
> > +        };
> > +    };
> > +    soc {
> > +        #address-cells = <2>;
> > +        #size-cells = <2>;
> > +
> > +        mcc: memory-controller@200200000 {
> > +            compatible = "apple,t8103-mcc", "apple,mcc";
> > +            #power-domain-cells = <0>;
> > +            reg = <0x2 0x200000 0x0 0x200000>;
> > +            operating-points-v2 = <&mcc_opp>;
> > +            apple,num-channels = <8>;
> > +        };
> > +    };
> > diff --git a/Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml b/Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml
> > new file mode 100644
> > index 000000000000..babf27841bb7
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/opp/apple,mcc-operating-points.yaml
> > @@ -0,0 +1,62 @@
> > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> > +%YAML 1.2
> > +---
> > +$id: http://devicetree.org/schemas/opp/apple,mcc-operating-points.yaml#
> > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > +
> > +title: Apple SoC memory controller OPP bindings
> > +
> > +maintainers:
> > +  - Hector Martin <marcan@marcan.st>
> > +
> > +description: |
> > +  Apple SoCs can have their memory controller performance adjusted depending on
> > +  system requirements. These performance states are represented by specific
> > +  memory controller register values. The apple-mcc driver uses these values
> > +  to change the MCC performance.
> > +
> > +allOf:
> > +  - $ref: opp-v2-base.yaml#
> > +
> > +properties:
> > +  compatible:
> > +    const: apple,mcc-operating-points
> > +
> > +required:
> > +  - compatible
> > +
> > +patternProperties:
> > +  "opp[0-9]+":
> > +    type: object
> > +
> > +    properties:
> > +      opp-level: true
> 
> You don't need to mention it.

Actually, you do.

You are thinking unevaluatedProperties takes care of it, but it doesn't 
here. The problem is if you have 2 schemas (this one and 
opp-v2-base.yaml) with child nodes, the child nodes in each schema are 
evaluated separately.

So anywhere we have child nodes, we need the child node schema to be a 
separate file or able to be directly referenced (i.e. under $defs). I 
only realized this when testing out unevaluatedProperties support.

Rob

^ permalink raw reply	[flat|nested] 40+ messages in thread

end of thread, other threads:[~2021-10-19 22:44 UTC | newest]

Thread overview: 40+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-11 16:56 [RFC PATCH 0/9] Apple SoC CPU P-state switching Hector Martin
2021-10-11 16:56 ` [RFC PATCH 1/9] MAINTAINERS: apple: Add apple-mcc and clk-apple-cluster paths Hector Martin
2021-10-11 16:57 ` [RFC PATCH 2/9] dt-bindings: memory-controller: Add apple,mcc binding Hector Martin
2021-10-12  8:48   ` Krzysztof Kozlowski
2021-10-19 22:43     ` Rob Herring
2021-10-11 16:57 ` [RFC PATCH 3/9] dt-bindings: clock: Add apple,cluster-clk binding Hector Martin
2021-10-12  8:51   ` Krzysztof Kozlowski
2021-10-12  9:35     ` Viresh Kumar
     [not found]       ` <D0DE08FE-562E-4A48-BCA0-9094DAFCA564@marcan.st>
     [not found]         ` <20211012094302.3cownyzr4phxwifs@vireshk-i7>
     [not found]           ` <64584F8C-D49F-41B5-9658-CF8A25186E67@marcan.st>
     [not found]             ` <20211012095735.mhh2lzu52ohtotl6@vireshk-i7>
2021-10-12 13:48               ` Hector Martin
2021-10-11 16:57 ` [RFC PATCH 4/9] opp: core: Don't warn if required OPP device does not exist Hector Martin
2021-10-12  3:21   ` Viresh Kumar
2021-10-12  5:34     ` Hector Martin
2021-10-12  5:51       ` Viresh Kumar
2021-10-12  5:57         ` Hector Martin
2021-10-12  9:26           ` Viresh Kumar
2021-10-12  9:31             ` Hector Martin "marcan"
2021-10-12  9:32               ` Viresh Kumar
2021-10-14  6:52                 ` Hector Martin
2021-10-14  6:56                   ` Viresh Kumar
2021-10-14  7:03                     ` Hector Martin
2021-10-14  7:22                       ` Viresh Kumar
2021-10-14  7:23                       ` Hector Martin
2021-10-14 11:08                         ` Ulf Hansson
2021-10-14  9:55           ` Ulf Hansson
2021-10-14 11:43             ` Hector Martin
2021-10-14 12:55               ` Ulf Hansson
2021-10-14 17:02                 ` Hector Martin
2021-10-15 11:26                   ` Ulf Hansson
2021-10-11 16:57 ` [RFC PATCH 5/9] PM: domains: Add of_genpd_add_provider_simple_noclk() Hector Martin
2021-10-11 16:57 ` [RFC PATCH 6/9] memory: apple: Add apple-mcc driver to manage MCC perf in Apple SoCs Hector Martin
2021-10-12  9:19   ` Krzysztof Kozlowski
2021-10-14  6:59     ` Hector Martin
2021-10-14  7:36       ` Krzysztof Kozlowski
2021-10-14  7:52         ` Hector Martin
2021-10-14  8:04           ` Krzysztof Kozlowski
2021-10-14  8:31             ` Hector Martin
2021-10-11 16:57 ` [RFC PATCH 7/9] clk: apple: Add clk-apple-cluster driver to manage CPU p-states Hector Martin
     [not found]   ` <163424925931.1688384.9647104000291025081@swboyd.mtv.corp.google.com>
2021-10-17  9:16     ` Hector Martin
2021-10-11 16:57 ` [RFC PATCH 8/9] arm64: apple: Select MEMORY and APPLE_MCC Hector Martin
2021-10-11 16:57 ` [RFC PATCH 9/9] arm64: apple: Add CPU frequency scaling support for t8103 Hector Martin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).