Rob, Lorenzo, Sudeep - apologize for sending a new version in the middle of the merge window, but I thought I might as well get this out, especially since there are only a very small amount of updates. Please review, whenever you find some time for it. Kind regards Ulf Hansson Changes in v3: - Take one step further to completely avoid executing any OSI specific code from the ->enter() callback, while operating in the default PSCI Platform Coordinated mode. - Update example for the PSCI DT bindings to make it compile with "make dt_binding_check" Changes in v2: - Avoid to affect the non-OSI path with specific changes for OSI. This forced me to re-order the series and a caused more or less minor changes to most of the patches. - Updated the DT bindings for PSCI to clarify and to include the "psci" name of the PM domain to attach to. - Replaced patch1 with another patch from Sudeep, solving the same problem, but in a different way. This series enables initial support for hierarchical CPU arrangement, managed by PSCI and its corresponding cpuidle driver. It's based on using the generic PM domain (genpd), which nowadays also supports devices belonging to CPUs. The last DTS patch enables the hierarchical topology to be used for the Qcom 410c Dragonboard, which supports the PSCI OS-initiated mode. More detailed background can be found from previous submissions [1]. The series is also available at: git.linaro.org/people/ulf.hansson/linux-pm.git next Kind regards Ulf Hansson [1] https://lwn.net/Articles/788306/ Lina Iyer (1): cpuidle: dt: Support hierarchical CPU idle states Sudeep Holla (1): cpuidle: psci: Align psci_power_state count with idle state count Ulf Hansson (11): dt: psci: Update DT bindings to support hierarchical PSCI states firmware: psci: Export functions to manage the OSI mode of: base: Add of_get_cpu_state_node() to get idle states for a CPU node cpuidle: psci: Simplify OF parsing of CPU idle state nodes cpuidle: psci: Support hierarchical CPU idle states cpuidle: psci: Add a helper to attach a CPU to its PM domain cpuidle: psci: Attach CPU devices to their PM domains cpuidle: psci: Prepare to use OS initiated suspend mode via PM domains cpuidle: psci: Manage runtime PM in the idle path cpuidle: psci: Add support for PM domains by using genpd arm64: dts: Convert to the hierarchical CPU topology layout for MSM8916 .../devicetree/bindings/arm/cpus.yaml | 15 + .../devicetree/bindings/arm/psci.yaml | 104 ++++++ arch/arm64/boot/dts/qcom/msm8916.dtsi | 57 +++- drivers/cpuidle/Makefile | 4 +- drivers/cpuidle/cpuidle-psci-domain.c | 302 ++++++++++++++++++ drivers/cpuidle/cpuidle-psci.c | 114 +++++-- drivers/cpuidle/cpuidle-psci.h | 17 + drivers/cpuidle/dt_idle_states.c | 5 +- drivers/firmware/psci/psci.c | 18 +- drivers/of/base.c | 36 +++ include/linux/of.h | 8 + include/linux/psci.h | 2 + 12 files changed, 647 insertions(+), 35 deletions(-) create mode 100644 drivers/cpuidle/cpuidle-psci-domain.c create mode 100644 drivers/cpuidle/cpuidle-psci.h -- 2.17.1
From: Sudeep Holla <sudeep.holla@arm.com> Instead of allocating 'n-1' states in psci_power_state to manage 'n' idle states which include "ARM WFI" state, it would be simpler to have 1:1 mapping between psci_power_state and cpuidle driver states. ARM WFI state(i.e. idx == 0) is handled specially in the generic macro CPU_PM_CPU_IDLE_ENTER_PARAM and hence state[-1] is not possible. However for sake of code readability, it is better to have 1:1 mapping and not use [idx - 1] to access psci_power_state corresponding to driver cpuidle state for idx. psci_power_state[0] is default initialised to 0 and is never accessed while entering WFI state. Reported-by: Ulf Hansson <ulf.hansson@linaro.org> Signed-off-by: Sudeep Holla <sudeep.holla@arm.com> Reviewed-by: Ulf Hansson <ulf.hansson@linaro.org> --- Changes in v3: - None. --- drivers/cpuidle/cpuidle-psci.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/cpuidle/cpuidle-psci.c b/drivers/cpuidle/cpuidle-psci.c index f3c1a2396f98..361985f52ddd 100644 --- a/drivers/cpuidle/cpuidle-psci.c +++ b/drivers/cpuidle/cpuidle-psci.c @@ -30,7 +30,7 @@ static int psci_enter_idle_state(struct cpuidle_device *dev, u32 *state = __this_cpu_read(psci_power_state); return CPU_PM_CPU_IDLE_ENTER_PARAM(psci_cpu_suspend_enter, - idx, state[idx - 1]); + idx, state[idx]); } static struct cpuidle_driver psci_idle_driver __initdata = { @@ -89,12 +89,14 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, int cpu) if (!count) return -ENODEV; + count++; /* Add WFI state too */ psci_states = kcalloc(count, sizeof(*psci_states), GFP_KERNEL); if (!psci_states) return -ENOMEM; - for (i = 0; i < count; i++) { - state_node = of_parse_phandle(cpu_node, "cpu-idle-states", i); + for (i = 1; i < count; i++) { + state_node = of_parse_phandle(cpu_node, "cpu-idle-states", + i - 1); ret = psci_dt_parse_state_node(state_node, &psci_states[i]); of_node_put(state_node); -- 2.17.1
Update PSCI DT bindings to allow to represent idle states for CPUs and the CPU topology, by using a hierarchical layout. Primarily this is done by re-using the existing DT bindings for PM domains [1] and for PM domain idle states [2]. Let's also add an example into the document for the PSCI DT bindings, to clearly show the new hierarchical based layout. The currently supported flattened layout, is already described in the ARM idle states bindings [3], so let's leave that as is. [1] Documentation/devicetree/bindings/power/power_domain.txt [2] Documentation/devicetree/bindings/power/domain-idle-state.txt [3] Documentation/devicetree/bindings/arm/idle-states.txt Co-developed-by: Lina Iyer <lina.iyer@linaro.org> Signed-off-by: Lina Iyer <lina.iyer@linaro.org> Reviewed-by: Sudeep Holla <sudeep.holla@arm.com> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> --- Changes in v3: - Updated example to make it compile with "make dt_binding_check". --- .../devicetree/bindings/arm/cpus.yaml | 15 +++ .../devicetree/bindings/arm/psci.yaml | 104 ++++++++++++++++++ 2 files changed, 119 insertions(+) diff --git a/Documentation/devicetree/bindings/arm/cpus.yaml b/Documentation/devicetree/bindings/arm/cpus.yaml index cb30895e3b67..92a775d6fc0e 100644 --- a/Documentation/devicetree/bindings/arm/cpus.yaml +++ b/Documentation/devicetree/bindings/arm/cpus.yaml @@ -241,6 +241,21 @@ properties: where voltage is in V, frequency is in MHz. + power-domains: + $ref: '/schemas/types.yaml#/definitions/phandle-array' + description: + List of phandles and PM domain specifiers, as defined by bindings of the + PM domain provider (see also ../power_domain.txt). + + power-domain-names: + $ref: '/schemas/types.yaml#/definitions/string-array' + description: + A list of power domain name strings sorted in the same order as the + power-domains property. + + For PSCI based platforms, the name corresponding to the index of the PSCI + PM domain provider, must be "psci". + qcom,saw: $ref: '/schemas/types.yaml#/definitions/phandle' description: | diff --git a/Documentation/devicetree/bindings/arm/psci.yaml b/Documentation/devicetree/bindings/arm/psci.yaml index 7abdf58b335e..8ef85420b2ab 100644 --- a/Documentation/devicetree/bindings/arm/psci.yaml +++ b/Documentation/devicetree/bindings/arm/psci.yaml @@ -102,6 +102,34 @@ properties: [1] Kernel documentation - ARM idle states bindings Documentation/devicetree/bindings/arm/idle-states.txt + "#power-domain-cells": + description: + The number of cells in a PM domain specifier as per binding in [3]. + Must be 0 as to represent a single PM domain. + + ARM systems can have multiple cores, sometimes in an hierarchical + arrangement. This often, but not always, maps directly to the processor + power topology of the system. Individual nodes in a topology have their + own specific power states and can be better represented hierarchically. + + For these cases, the definitions of the idle states for the CPUs and the + CPU topology, must conform to the binding in [3]. The idle states + themselves must conform to the binding in [4] and must specify the + arm,psci-suspend-param property. + + It should also be noted that, in PSCI firmware v1.0 the OS-Initiated + (OSI) CPU suspend mode is introduced. Using a hierarchical representation + helps to implement support for OSI mode and OS implementations may choose + to mandate it. + + [3] Documentation/devicetree/bindings/power/power_domain.txt + [4] Documentation/devicetree/bindings/power/domain-idle-state.txt + + power-domains: + $ref: '/schemas/types.yaml#/definitions/phandle-array' + description: + List of phandles and PM domain specifiers, as defined by bindings of the + PM domain provider. required: - compatible @@ -160,4 +188,80 @@ examples: cpu_on = <0x95c10002>; cpu_off = <0x95c10001>; }; + + - |+ + + // Case 4: CPUs and CPU idle states described using the hierarchical model. + + cpus { + #size-cells = <0>; + #address-cells = <1>; + + CPU0: cpu@0 { + device_type = "cpu"; + compatible = "arm,cortex-a53", "arm,armv8"; + reg = <0x0>; + enable-method = "psci"; + power-domains = <&CPU_PD0>; + power-domain-names = "psci"; + }; + + CPU1: cpu@1 { + device_type = "cpu"; + compatible = "arm,cortex-a57", "arm,armv8"; + reg = <0x100>; + enable-method = "psci"; + power-domains = <&CPU_PD1>; + power-domain-names = "psci"; + }; + + idle-states { + + CPU_PWRDN: cpu-power-down { + compatible = "arm,idle-state"; + arm,psci-suspend-param = <0x0000001>; + entry-latency-us = <10>; + exit-latency-us = <10>; + min-residency-us = <100>; + }; + + CLUSTER_RET: cluster-retention { + compatible = "domain-idle-state"; + arm,psci-suspend-param = <0x1000011>; + entry-latency-us = <500>; + exit-latency-us = <500>; + min-residency-us = <2000>; + }; + + CLUSTER_PWRDN: cluster-power-down { + compatible = "domain-idle-state"; + arm,psci-suspend-param = <0x1000031>; + entry-latency-us = <2000>; + exit-latency-us = <2000>; + min-residency-us = <6000>; + }; + }; + }; + + psci { + compatible = "arm,psci-1.0"; + method = "smc"; + + CPU_PD0: cpu-pd0 { + #power-domain-cells = <0>; + domain-idle-states = <&CPU_PWRDN>; + power-domains = <&CLUSTER_PD>; + }; + + CPU_PD1: cpu-pd1 { + #power-domain-cells = <0>; + domain-idle-states = <&CPU_PWRDN>; + power-domains = <&CLUSTER_PD>; + }; + + CLUSTER_PD: cluster-pd { + #power-domain-cells = <0>; + domain-idle-states = <&CLUSTER_RET>, <&CLUSTER_PWRDN>; + }; + }; ... -- 2.17.1
To allow subsequent changes to implement support for OSI mode through the cpuidle-psci driver, export the existing psci_has_osi_support(). Export also a new function, psci_set_osi_mode(), that allows its caller to enable the OS-initiated CPU-suspend mode in the PSCI FW. To deal with backwards compatibility for a kernel started through a kexec call, default to set the CPU-suspend mode to the Platform Coordinated mode during boot. Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> Reviewed-by: Sudeep Holla <sudeep.holla@arm.com> --- Changes in v3: - None. --- drivers/firmware/psci/psci.c | 18 ++++++++++++++++-- include/linux/psci.h | 2 ++ 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/drivers/firmware/psci/psci.c b/drivers/firmware/psci/psci.c index b3b6c15e7b36..2937d44b5df4 100644 --- a/drivers/firmware/psci/psci.c +++ b/drivers/firmware/psci/psci.c @@ -97,7 +97,7 @@ static inline bool psci_has_ext_power_state(void) PSCI_1_0_FEATURES_CPU_SUSPEND_PF_MASK; } -static inline bool psci_has_osi_support(void) +bool psci_has_osi_support(void) { return psci_cpu_suspend_feature & PSCI_1_0_OS_INITIATED; } @@ -162,6 +162,15 @@ static u32 psci_get_version(void) return invoke_psci_fn(PSCI_0_2_FN_PSCI_VERSION, 0, 0, 0); } +int psci_set_osi_mode(void) +{ + int err; + + err = invoke_psci_fn(PSCI_1_0_FN_SET_SUSPEND_MODE, + PSCI_1_0_SUSPEND_MODE_OSI, 0, 0); + return psci_to_linux_errno(err); +} + static int psci_cpu_suspend(u32 state, unsigned long entry_point) { int err; @@ -544,9 +553,14 @@ static int __init psci_1_0_init(struct device_node *np) if (err) return err; - if (psci_has_osi_support()) + if (psci_has_osi_support()) { pr_info("OSI mode supported.\n"); + /* Default to PC mode. */ + invoke_psci_fn(PSCI_1_0_FN_SET_SUSPEND_MODE, + PSCI_1_0_SUSPEND_MODE_PC, 0, 0); + } + return 0; } diff --git a/include/linux/psci.h b/include/linux/psci.h index ebe0a881d13d..a67712b73b6c 100644 --- a/include/linux/psci.h +++ b/include/linux/psci.h @@ -18,6 +18,8 @@ bool psci_tos_resident_on(int cpu); int psci_cpu_suspend_enter(u32 state); bool psci_power_state_is_valid(u32 state); +int psci_set_osi_mode(void); +bool psci_has_osi_support(void); enum smccc_version { SMCCC_VERSION_1_0, -- 2.17.1
The CPU's idle state nodes are currently parsed at the common cpuidle DT library, but also when initializing data for specific CPU idle operations, as in the PSCI cpuidle driver case and qcom-spm cpuidle case. To avoid open-coding, let's introduce of_get_cpu_state_node(), which takes the device node for the CPU and the index to the requested idle state node, as in-parameters. In case a corresponding idle state node is found, it returns the node with the refcount incremented for it, else it returns NULL. Moreover, for PSCI there are two options to describe the CPU's idle states [1], either via a flattened description or a hierarchical layout. Hence, let's take both options into account. [1] Documentation/devicetree/bindings/arm/psci.yaml Suggested-by: Sudeep Holla <sudeep.holla@arm.com> Co-developed-by: Lina Iyer <lina.iyer@linaro.org> Signed-off-by: Lina Iyer <lina.iyer@linaro.org> Reviewed-by: Rob Herring <robh@kernel.org> Reviewed-by: Daniel Lezcano <daniel.lezcano@linaro.org> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> Reviewed-by: Sudeep Holla <sudeep.holla@arm.com> --- Changes in v3: - None. --- drivers/of/base.c | 36 ++++++++++++++++++++++++++++++++++++ include/linux/of.h | 8 ++++++++ 2 files changed, 44 insertions(+) diff --git a/drivers/of/base.c b/drivers/of/base.c index 1d667eb730e1..0e4cdf0f3864 100644 --- a/drivers/of/base.c +++ b/drivers/of/base.c @@ -477,6 +477,42 @@ int of_cpu_node_to_id(struct device_node *cpu_node) } EXPORT_SYMBOL(of_cpu_node_to_id); +/** + * of_get_cpu_state_node - Get CPU's idle state node at the given index + * + * @cpu_node: The device node for the CPU + * @index: The index in the list of the idle states + * + * Two generic methods can be used to describe a CPU's idle states, either via + * a flattened description through the "cpu-idle-states" binding or via the + * hierarchical layout, using the "power-domains" and the "domain-idle-states" + * bindings. This function check for both and returns the idle state node for + * the requested index. + * + * In case an idle state node is found at @index, the refcount is incremented + * for it, so call of_node_put() on it when done. Returns NULL if not found. + */ +struct device_node *of_get_cpu_state_node(struct device_node *cpu_node, + int index) +{ + struct of_phandle_args args; + int err; + + err = of_parse_phandle_with_args(cpu_node, "power-domains", + "#power-domain-cells", 0, &args); + if (!err) { + struct device_node *state_node = + of_parse_phandle(args.np, "domain-idle-states", index); + + of_node_put(args.np); + if (state_node) + return state_node; + } + + return of_parse_phandle(cpu_node, "cpu-idle-states", index); +} +EXPORT_SYMBOL(of_get_cpu_state_node); + /** * __of_device_is_compatible() - Check if the node matches given constraints * @device: pointer to node diff --git a/include/linux/of.h b/include/linux/of.h index 844f89e1b039..c669c0a4732f 100644 --- a/include/linux/of.h +++ b/include/linux/of.h @@ -351,6 +351,8 @@ extern const void *of_get_property(const struct device_node *node, int *lenp); extern struct device_node *of_get_cpu_node(int cpu, unsigned int *thread); extern struct device_node *of_get_next_cpu_node(struct device_node *prev); +extern struct device_node *of_get_cpu_state_node(struct device_node *cpu_node, + int index); #define for_each_property_of_node(dn, pp) \ for (pp = dn->properties; pp != NULL; pp = pp->next) @@ -765,6 +767,12 @@ static inline struct device_node *of_get_next_cpu_node(struct device_node *prev) return NULL; } +static inline struct device_node *of_get_cpu_state_node(struct device_node *cpu_node, + int index) +{ + return NULL; +} + static inline int of_n_addr_cells(struct device_node *np) { return 0; -- 2.17.1
From: Lina Iyer <lina.iyer@linaro.org> Currently CPU's idle states are represented using the flattened model. Let's add support for the hierarchical layout, via converting to use of_get_cpu_state_node(). Suggested-by: Sudeep Holla <sudeep.holla@arm.com> Signed-off-by: Lina Iyer <lina.iyer@linaro.org> Reviewed-by: Daniel Lezcano <daniel.lezcano@linaro.org> Co-developed-by: Ulf Hansson <ulf.hansson@linaro.org> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> Reviewed-by: Sudeep Holla <sudeep.holla@arm.com> --- Changes in v3: - None. --- drivers/cpuidle/dt_idle_states.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/drivers/cpuidle/dt_idle_states.c b/drivers/cpuidle/dt_idle_states.c index d06d21a9525d..252f2a9686a6 100644 --- a/drivers/cpuidle/dt_idle_states.c +++ b/drivers/cpuidle/dt_idle_states.c @@ -111,8 +111,7 @@ static bool idle_state_valid(struct device_node *state_node, unsigned int idx, for (cpu = cpumask_next(cpumask_first(cpumask), cpumask); cpu < nr_cpu_ids; cpu = cpumask_next(cpu, cpumask)) { cpu_node = of_cpu_device_node_get(cpu); - curr_state_node = of_parse_phandle(cpu_node, "cpu-idle-states", - idx); + curr_state_node = of_get_cpu_state_node(cpu_node, idx); if (state_node != curr_state_node) valid = false; @@ -170,7 +169,7 @@ int dt_init_idle_driver(struct cpuidle_driver *drv, cpu_node = of_cpu_device_node_get(cpumask_first(cpumask)); for (i = 0; ; i++) { - state_node = of_parse_phandle(cpu_node, "cpu-idle-states", i); + state_node = of_get_cpu_state_node(cpu_node, i); if (!state_node) break; -- 2.17.1
Iterating through the idle state nodes in DT, to find out the number of states that needs to be allocated is unnecessary, as it has already been done from dt_init_idle_driver(). Therefore, drop the iteration and use the number we already have at hand. Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> Reviewed-by: Sudeep Holla <sudeep.holla@arm.com> --- Changes in v3: - None. --- drivers/cpuidle/cpuidle-psci.c | 35 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 18 deletions(-) diff --git a/drivers/cpuidle/cpuidle-psci.c b/drivers/cpuidle/cpuidle-psci.c index 361985f52ddd..761359be50f2 100644 --- a/drivers/cpuidle/cpuidle-psci.c +++ b/drivers/cpuidle/cpuidle-psci.c @@ -73,30 +73,24 @@ static int __init psci_dt_parse_state_node(struct device_node *np, u32 *state) return 0; } -static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, int cpu) +static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, + unsigned int state_count, int cpu) { - int i, ret = 0, count = 0; + int i, ret = 0; u32 *psci_states; struct device_node *state_node; - /* Count idle states */ - while ((state_node = of_parse_phandle(cpu_node, "cpu-idle-states", - count))) { - count++; - of_node_put(state_node); - } - - if (!count) - return -ENODEV; - - count++; /* Add WFI state too */ - psci_states = kcalloc(count, sizeof(*psci_states), GFP_KERNEL); + state_count++; /* Add WFI state too */ + psci_states = kcalloc(state_count, sizeof(*psci_states), GFP_KERNEL); if (!psci_states) return -ENOMEM; - for (i = 1; i < count; i++) { + for (i = 1; i < state_count; i++) { state_node = of_parse_phandle(cpu_node, "cpu-idle-states", i - 1); + if (!state_node) + break; + ret = psci_dt_parse_state_node(state_node, &psci_states[i]); of_node_put(state_node); @@ -106,6 +100,11 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, int cpu) pr_debug("psci-power-state %#x index %d\n", psci_states[i], i); } + if (i != state_count) { + ret = -ENODEV; + goto free_mem; + } + /* Idle states parsed correctly, initialize per-cpu pointer */ per_cpu(psci_power_state, cpu) = psci_states; return 0; @@ -115,7 +114,7 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, int cpu) return ret; } -static __init int psci_cpu_init_idle(unsigned int cpu) +static __init int psci_cpu_init_idle(unsigned int cpu, unsigned int state_count) { struct device_node *cpu_node; int ret; @@ -131,7 +130,7 @@ static __init int psci_cpu_init_idle(unsigned int cpu) if (!cpu_node) return -ENODEV; - ret = psci_dt_cpu_init_idle(cpu_node, cpu); + ret = psci_dt_cpu_init_idle(cpu_node, state_count, cpu); of_node_put(cpu_node); @@ -187,7 +186,7 @@ static int __init psci_idle_init_cpu(int cpu) /* * Initialize PSCI idle states. */ - ret = psci_cpu_init_idle(cpu); + ret = psci_cpu_init_idle(cpu, ret); if (ret) { pr_err("CPU %d failed to PSCI idle\n", cpu); goto out_kfree_drv; -- 2.17.1
Currently CPU's idle states are represented using the flattened model. Let's add support for the hierarchical layout, via converting to use of_get_cpu_state_node(). Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> Reviewed-by: Sudeep Holla <sudeep.holla@arm.com> --- Changes in v3: - None. --- drivers/cpuidle/cpuidle-psci.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/cpuidle/cpuidle-psci.c b/drivers/cpuidle/cpuidle-psci.c index 761359be50f2..830995b8a56f 100644 --- a/drivers/cpuidle/cpuidle-psci.c +++ b/drivers/cpuidle/cpuidle-psci.c @@ -86,8 +86,7 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, return -ENOMEM; for (i = 1; i < state_count; i++) { - state_node = of_parse_phandle(cpu_node, "cpu-idle-states", - i - 1); + state_node = of_get_cpu_state_node(cpu_node, i - 1); if (!state_node) break; -- 2.17.1
Introduce a PSCI DT helper function, psci_dt_attach_cpu(), which takes a CPU number as an in-parameter and tries to attach the CPU's struct device to its corresponding PM domain. Let's makes use of dev_pm_domain_attach_by_name(), as it allows us to specify "psci" as the "name" of the PM domain to attach to. Additionally, let's also prepare the attached device to be power managed via runtime PM. Note that, the implementation of the new helper function is in a new separate c-file, which may seems a bit too much at this point. However, subsequent changes that implements the remaining part of the PM domain support for cpuidle-psci, helps to justify this split. Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> --- Changes in v3: - None. --- drivers/cpuidle/Makefile | 4 ++- drivers/cpuidle/cpuidle-psci-domain.c | 36 +++++++++++++++++++++++++++ drivers/cpuidle/cpuidle-psci.h | 12 +++++++++ 3 files changed, 51 insertions(+), 1 deletion(-) create mode 100644 drivers/cpuidle/cpuidle-psci-domain.c create mode 100644 drivers/cpuidle/cpuidle-psci.h diff --git a/drivers/cpuidle/Makefile b/drivers/cpuidle/Makefile index ee70d5cc5b99..cc8c769d7fa9 100644 --- a/drivers/cpuidle/Makefile +++ b/drivers/cpuidle/Makefile @@ -21,7 +21,9 @@ obj-$(CONFIG_ARM_U8500_CPUIDLE) += cpuidle-ux500.o obj-$(CONFIG_ARM_AT91_CPUIDLE) += cpuidle-at91.o obj-$(CONFIG_ARM_EXYNOS_CPUIDLE) += cpuidle-exynos.o obj-$(CONFIG_ARM_CPUIDLE) += cpuidle-arm.o -obj-$(CONFIG_ARM_PSCI_CPUIDLE) += cpuidle-psci.o +obj-$(CONFIG_ARM_PSCI_CPUIDLE) += cpuidle_psci.o +cpuidle_psci-y := cpuidle-psci.o +cpuidle_psci-$(CONFIG_PM_GENERIC_DOMAINS_OF) += cpuidle-psci-domain.o ############################################################################### # MIPS drivers diff --git a/drivers/cpuidle/cpuidle-psci-domain.c b/drivers/cpuidle/cpuidle-psci-domain.c new file mode 100644 index 000000000000..bc7df4dc0686 --- /dev/null +++ b/drivers/cpuidle/cpuidle-psci-domain.c @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * PM domains for CPUs via genpd - managed by cpuidle-psci. + * + * Copyright (C) 2019 Linaro Ltd. + * Author: Ulf Hansson <ulf.hansson@linaro.org> + * + */ + +#include <linux/cpu.h> +#include <linux/device.h> +#include <linux/kernel.h> +#include <linux/pm_domain.h> +#include <linux/pm_runtime.h> +#include <linux/psci.h> + +#include "cpuidle-psci.h" + +struct device *psci_dt_attach_cpu(int cpu) +{ + struct device *dev; + + /* Currently limit the hierarchical topology to be used in OSI mode. */ + if (!psci_has_osi_support()) + return NULL; + + dev = dev_pm_domain_attach_by_name(get_cpu_device(cpu), "psci"); + if (IS_ERR_OR_NULL(dev)) + return dev; + + pm_runtime_irq_safe(dev); + if (cpu_online(cpu)) + pm_runtime_get_sync(dev); + + return dev; +} diff --git a/drivers/cpuidle/cpuidle-psci.h b/drivers/cpuidle/cpuidle-psci.h new file mode 100644 index 000000000000..0cadbb71dc55 --- /dev/null +++ b/drivers/cpuidle/cpuidle-psci.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __CPUIDLE_PSCI_H +#define __CPUIDLE_PSCI_H + +#ifdef CONFIG_PM_GENERIC_DOMAINS_OF +struct device *psci_dt_attach_cpu(int cpu); +#else +static inline struct device *psci_dt_attach_cpu(int cpu) { return NULL; } +#endif + +#endif /* __CPUIDLE_PSCI_H */ -- 2.17.1
In order to enable a CPU to be power managed through its PM domain, let's try to attach it by calling psci_dt_attach_cpu() during the cpuidle initialization. psci_dt_attach_cpu() returns a pointer to the attached struct device, which later should be used for runtime PM, hence we need to store it somewhere. Rather than adding yet another per CPU variable, let's create a per CPU struct to collect the relevant per CPU variables. Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> --- Changes in v3: - None. --- drivers/cpuidle/cpuidle-psci.c | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/drivers/cpuidle/cpuidle-psci.c b/drivers/cpuidle/cpuidle-psci.c index 830995b8a56f..167249d0493f 100644 --- a/drivers/cpuidle/cpuidle-psci.c +++ b/drivers/cpuidle/cpuidle-psci.c @@ -20,14 +20,20 @@ #include <asm/cpuidle.h> +#include "cpuidle-psci.h" #include "dt_idle_states.h" -static DEFINE_PER_CPU_READ_MOSTLY(u32 *, psci_power_state); +struct psci_cpuidle_data { + u32 *psci_states; + struct device *dev; +}; + +static DEFINE_PER_CPU_READ_MOSTLY(struct psci_cpuidle_data, psci_cpuidle_data); static int psci_enter_idle_state(struct cpuidle_device *dev, struct cpuidle_driver *drv, int idx) { - u32 *state = __this_cpu_read(psci_power_state); + u32 *state = __this_cpu_read(psci_cpuidle_data.psci_states); return CPU_PM_CPU_IDLE_ENTER_PARAM(psci_cpu_suspend_enter, idx, state[idx]); @@ -78,7 +84,9 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, { int i, ret = 0; u32 *psci_states; + struct device *dev; struct device_node *state_node; + struct psci_cpuidle_data *data = per_cpu_ptr(&psci_cpuidle_data, cpu); state_count++; /* Add WFI state too */ psci_states = kcalloc(state_count, sizeof(*psci_states), GFP_KERNEL); @@ -104,8 +112,16 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, goto free_mem; } - /* Idle states parsed correctly, initialize per-cpu pointer */ - per_cpu(psci_power_state, cpu) = psci_states; + dev = psci_dt_attach_cpu(cpu); + if (IS_ERR(dev)) { + ret = PTR_ERR(dev); + goto free_mem; + } + + data->dev = dev; + + /* Idle states parsed correctly, store them in the per-cpu struct. */ + data->psci_states = psci_states; return 0; free_mem: -- 2.17.1
The per CPU variable psci_power_state, contains an array of fixed values, which reflects the corresponding arm,psci-suspend-param parsed from DT, for each of the available CPU idle states. This isn't sufficient when using the hierarchical CPU topology in DT, in combination with having PSCI OS initiated (OSI) mode enabled. More precisely, in OSI mode, Linux is responsible of telling the PSCI FW what idle state the cluster (a group of CPUs) should enter, while in PSCI Platform Coordinated (PC) mode, each CPU independently votes for an idle state of the cluster. For this reason, introduce a per CPU variable called domain_state and implement two helper functions to read/write its value. Then let the domain_state take precedence over the regular selected state, when entering and idle state. To avoid executing the above OSI specific code in the ->enter() callback, while operating in the default PSCI Platform Coordinated mode, let's also add a new enter-function and use it for OSI. Co-developed-by: Lina Iyer <lina.iyer@linaro.org> Signed-off-by: Lina Iyer <lina.iyer@linaro.org> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> --- Changes in v3: - Avoid executing any OSI specific code in psci_enter_idle_state(), while operating in the default PSCI Platform Coordinated mode. --- drivers/cpuidle/cpuidle-psci.c | 52 ++++++++++++++++++++++++++++++---- 1 file changed, 46 insertions(+), 6 deletions(-) diff --git a/drivers/cpuidle/cpuidle-psci.c b/drivers/cpuidle/cpuidle-psci.c index 167249d0493f..fd664e134c3f 100644 --- a/drivers/cpuidle/cpuidle-psci.c +++ b/drivers/cpuidle/cpuidle-psci.c @@ -29,14 +29,47 @@ struct psci_cpuidle_data { }; static DEFINE_PER_CPU_READ_MOSTLY(struct psci_cpuidle_data, psci_cpuidle_data); +static DEFINE_PER_CPU(u32, domain_state); + +static inline void psci_set_domain_state(u32 state) +{ + __this_cpu_write(domain_state, state); +} + +static inline u32 psci_get_domain_state(void) +{ + return __this_cpu_read(domain_state); +} + +static inline int psci_enter_state(int idx, u32 state) +{ + return CPU_PM_CPU_IDLE_ENTER_PARAM(psci_cpu_suspend_enter, idx, state); +} + +static int psci_enter_domain_idle_state(struct cpuidle_device *dev, + struct cpuidle_driver *drv, int idx) +{ + struct psci_cpuidle_data *data = this_cpu_ptr(&psci_cpuidle_data); + u32 *states = data->psci_states; + u32 state = psci_get_domain_state(); + int ret; + + if (!state) + state = states[idx]; + + ret = psci_enter_state(idx, state); + + /* Clear the domain state to start fresh when back from idle. */ + psci_set_domain_state(0); + return ret; +} static int psci_enter_idle_state(struct cpuidle_device *dev, struct cpuidle_driver *drv, int idx) { u32 *state = __this_cpu_read(psci_cpuidle_data.psci_states); - return CPU_PM_CPU_IDLE_ENTER_PARAM(psci_cpu_suspend_enter, - idx, state[idx]); + return psci_enter_state(idx, state[idx]); } static struct cpuidle_driver psci_idle_driver __initdata = { @@ -79,7 +112,8 @@ static int __init psci_dt_parse_state_node(struct device_node *np, u32 *state) return 0; } -static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, +static int __init psci_dt_cpu_init_idle(struct cpuidle_driver *drv, + struct device_node *cpu_node, unsigned int state_count, int cpu) { int i, ret = 0; @@ -118,6 +152,11 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, goto free_mem; } + /* Manage the deepest state via a dedicated enter-function. */ + if (dev) + drv->states[state_count - 1].enter = + psci_enter_domain_idle_state; + data->dev = dev; /* Idle states parsed correctly, store them in the per-cpu struct. */ @@ -129,7 +168,8 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, return ret; } -static __init int psci_cpu_init_idle(unsigned int cpu, unsigned int state_count) +static __init int psci_cpu_init_idle(struct cpuidle_driver *drv, + unsigned int cpu, unsigned int state_count) { struct device_node *cpu_node; int ret; @@ -145,7 +185,7 @@ static __init int psci_cpu_init_idle(unsigned int cpu, unsigned int state_count) if (!cpu_node) return -ENODEV; - ret = psci_dt_cpu_init_idle(cpu_node, state_count, cpu); + ret = psci_dt_cpu_init_idle(drv, cpu_node, state_count, cpu); of_node_put(cpu_node); @@ -201,7 +241,7 @@ static int __init psci_idle_init_cpu(int cpu) /* * Initialize PSCI idle states. */ - ret = psci_cpu_init_idle(cpu, ret); + ret = psci_cpu_init_idle(drv, cpu, ret); if (ret) { pr_err("CPU %d failed to PSCI idle\n", cpu); goto out_kfree_drv; -- 2.17.1
In case we have succeeded to attach a CPU to its PM domain, let's deploy runtime PM support for the corresponding attached device, to allow the CPU to be powered-managed accordingly. The triggering point for when runtime PM reference counting should be done, has been selected to the deepest idle state for the CPU. However, from the hierarchical point view, there may be good reasons to do runtime PM reference counting even on shallower idle states, but at this point this isn't supported, mainly due to limitations set by the generic PM domain. Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> --- Changes in v3: - Rebased. --- drivers/cpuidle/cpuidle-psci.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/cpuidle/cpuidle-psci.c b/drivers/cpuidle/cpuidle-psci.c index fd664e134c3f..0707222a40bd 100644 --- a/drivers/cpuidle/cpuidle-psci.c +++ b/drivers/cpuidle/cpuidle-psci.c @@ -16,6 +16,7 @@ #include <linux/of.h> #include <linux/of_device.h> #include <linux/psci.h> +#include <linux/pm_runtime.h> #include <linux/slab.h> #include <asm/cpuidle.h> @@ -51,14 +52,21 @@ static int psci_enter_domain_idle_state(struct cpuidle_device *dev, { struct psci_cpuidle_data *data = this_cpu_ptr(&psci_cpuidle_data); u32 *states = data->psci_states; - u32 state = psci_get_domain_state(); + struct device *pd_dev = data->dev; + u32 state; int ret; + /* Do runtime PM to manage a hierarchical CPU toplogy. */ + pm_runtime_put_sync_suspend(pd_dev); + + state = psci_get_domain_state(); if (!state) state = states[idx]; ret = psci_enter_state(idx, state); + pm_runtime_get_sync(pd_dev); + /* Clear the domain state to start fresh when back from idle. */ psci_set_domain_state(0); return ret; -- 2.17.1
When the hierarchical CPU topology layout is used in DT and the PSCI OSI mode is supported by the PSCI FW, let's initialize a corresponding PM domain topology by using genpd. This enables a CPU and a group of CPUs, when attached to the topology, to be power-managed accordingly. To trigger the attempt to initialize the genpd data structures let's use a subsys_initcall, which should be early enough to allow CPUs, but also other devices to be attached. The initialization consists of parsing the PSCI OF node for the topology and the "domain idle states" DT bindings. In case the idle states are compatible with "domain-idle-state", the initialized genpd becomes responsible of selecting an idle state for the PM domain, via assigning it a genpd governor. Note that, a successful initialization of the genpd data structures, is followed by a call to psci_set_osi_mode(), as to try to enable the OSI mode in the PSCI FW. In case this fails, we fall back into a degraded mode rather than bailing out and returning an error code. Co-developed-by: Lina Iyer <lina.iyer@linaro.org> Signed-off-by: Lina Iyer <lina.iyer@linaro.org> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> --- Changes in v3: - None. --- drivers/cpuidle/cpuidle-psci-domain.c | 266 ++++++++++++++++++++++++++ drivers/cpuidle/cpuidle-psci.c | 4 +- drivers/cpuidle/cpuidle-psci.h | 5 + 3 files changed, 273 insertions(+), 2 deletions(-) diff --git a/drivers/cpuidle/cpuidle-psci-domain.c b/drivers/cpuidle/cpuidle-psci-domain.c index bc7df4dc0686..7429fd7626a1 100644 --- a/drivers/cpuidle/cpuidle-psci-domain.c +++ b/drivers/cpuidle/cpuidle-psci-domain.c @@ -7,15 +7,281 @@ * */ +#define pr_fmt(fmt) "CPUidle PSCI: " fmt + #include <linux/cpu.h> #include <linux/device.h> #include <linux/kernel.h> #include <linux/pm_domain.h> #include <linux/pm_runtime.h> #include <linux/psci.h> +#include <linux/slab.h> +#include <linux/string.h> #include "cpuidle-psci.h" +struct psci_pd_provider { + struct list_head link; + struct device_node *node; +}; + +static LIST_HEAD(psci_pd_providers); +static bool osi_mode_enabled; + +static int psci_pd_power_off(struct generic_pm_domain *pd) +{ + struct genpd_power_state *state = &pd->states[pd->state_idx]; + u32 *pd_state; + + /* If we have failed to enable OSI mode, then abort power off. */ + if (!osi_mode_enabled) + return -EBUSY; + + if (!state->data) + return 0; + + /* OSI mode is enabled, set the corresponding domain state. */ + pd_state = state->data; + psci_set_domain_state(*pd_state); + + return 0; +} + +static int __init psci_pd_parse_state_nodes(struct genpd_power_state *states, + int state_count) +{ + int i, ret; + u32 psci_state, *psci_state_buf; + + for (i = 0; i < state_count; i++) { + ret = psci_dt_parse_state_node(to_of_node(states[i].fwnode), + &psci_state); + if (ret) + goto free_state; + + psci_state_buf = kmalloc(sizeof(u32), GFP_KERNEL); + if (!psci_state_buf) { + ret = -ENOMEM; + goto free_state; + } + *psci_state_buf = psci_state; + states[i].data = psci_state_buf; + } + + return 0; + +free_state: + i--; + for (; i >= 0; i--) + kfree(states[i].data); + return ret; +} + +static int __init psci_pd_parse_states(struct device_node *np, + struct genpd_power_state **states, int *state_count) +{ + int ret; + + /* Parse the domain idle states. */ + ret = of_genpd_parse_idle_states(np, states, state_count); + if (ret) + return ret; + + /* Fill out the PSCI specifics for each found state. */ + ret = psci_pd_parse_state_nodes(*states, *state_count); + if (ret) + kfree(*states); + + return ret; +} + +static void psci_pd_free_states(struct genpd_power_state *states, + unsigned int state_count) +{ + int i; + + for (i = 0; i < state_count; i++) + kfree(states[i].data); + kfree(states); +} + +static int __init psci_pd_init(struct device_node *np) +{ + struct generic_pm_domain *pd; + struct psci_pd_provider *pd_provider; + struct dev_power_governor *pd_gov; + struct genpd_power_state *states = NULL; + int ret = -ENOMEM, state_count = 0; + + pd = kzalloc(sizeof(*pd), GFP_KERNEL); + if (!pd) + goto out; + + pd_provider = kzalloc(sizeof(*pd_provider), GFP_KERNEL); + if (!pd_provider) + goto free_pd; + + pd->name = kasprintf(GFP_KERNEL, "%pOF", np); + if (!pd->name) + goto free_pd_prov; + + /* + * Parse the domain idle states and let genpd manage the state selection + * for those being compatible with "domain-idle-state". + */ + ret = psci_pd_parse_states(np, &states, &state_count); + if (ret) + goto free_name; + + pd->free_states = psci_pd_free_states; + pd->name = kbasename(pd->name); + pd->power_off = psci_pd_power_off; + pd->states = states; + pd->state_count = state_count; + pd->flags |= GENPD_FLAG_IRQ_SAFE | GENPD_FLAG_CPU_DOMAIN; + + /* Use governor for CPU PM domains if it has some states to manage. */ + pd_gov = state_count > 0 ? &pm_domain_cpu_gov : NULL; + + ret = pm_genpd_init(pd, pd_gov, false); + if (ret) { + psci_pd_free_states(states, state_count); + goto free_name; + } + + ret = of_genpd_add_provider_simple(np, pd); + if (ret) + goto remove_pd; + + pd_provider->node = of_node_get(np); + list_add(&pd_provider->link, &psci_pd_providers); + + pr_debug("init PM domain %s\n", pd->name); + return 0; + +remove_pd: + pm_genpd_remove(pd); +free_name: + kfree(pd->name); +free_pd_prov: + kfree(pd_provider); +free_pd: + kfree(pd); +out: + pr_err("failed to init PM domain ret=%d %pOF\n", ret, np); + return ret; +} + +static void __init psci_pd_remove(void) +{ + struct psci_pd_provider *pd_provider, *it; + struct generic_pm_domain *genpd; + + list_for_each_entry_safe(pd_provider, it, &psci_pd_providers, link) { + of_genpd_del_provider(pd_provider->node); + + genpd = of_genpd_remove_last(pd_provider->node); + if (!IS_ERR(genpd)) + kfree(genpd); + + of_node_put(pd_provider->node); + list_del(&pd_provider->link); + kfree(pd_provider); + } +} + +static int __init psci_pd_init_topology(struct device_node *np) +{ + struct device_node *node; + struct of_phandle_args child, parent; + int ret; + + for_each_child_of_node(np, node) { + if (of_parse_phandle_with_args(node, "power-domains", + "#power-domain-cells", 0, &parent)) + continue; + + child.np = node; + child.args_count = 0; + + ret = of_genpd_add_subdomain(&parent, &child); + of_node_put(parent.np); + if (ret) { + of_node_put(node); + return ret; + } + } + + return 0; +} + +static const struct of_device_id psci_of_match[] __initconst = { + { .compatible = "arm,psci" }, + { .compatible = "arm,psci-0.2" }, + { .compatible = "arm,psci-1.0" }, + {} +}; + +static int __init psci_idle_init_domains(void) +{ + struct device_node *np = of_find_matching_node(NULL, psci_of_match); + struct device_node *node; + int ret = 0, pd_count = 0; + + if (!np) + return -ENODEV; + + /* Currently limit the hierarchical topology to be used in OSI mode. */ + if (!psci_has_osi_support()) + goto out; + + /* + * Parse child nodes for the "#power-domain-cells" property and + * initialize a genpd/genpd-of-provider pair when it's found. + */ + for_each_child_of_node(np, node) { + if (!of_find_property(node, "#power-domain-cells", NULL)) + continue; + + ret = psci_pd_init(node); + if (ret) + goto put_node; + + pd_count++; + } + + /* Bail out if not using the hierarchical CPU topology. */ + if (!pd_count) + goto out; + + /* Link genpd masters/subdomains to model the CPU topology. */ + ret = psci_pd_init_topology(np); + if (ret) + goto remove_pd; + + /* Try to enable OSI mode. */ + ret = psci_set_osi_mode(); + if (ret) + pr_warn("failed to enable OSI mode: %d\n", ret); + else + osi_mode_enabled = true; + + of_node_put(np); + pr_info("Initialized CPU PM domain topology\n"); + return pd_count; + +put_node: + of_node_put(node); +remove_pd: + if (pd_count) + psci_pd_remove(); + pr_err("failed to create CPU PM domains ret=%d\n", ret); +out: + of_node_put(np); + return ret; +} +subsys_initcall(psci_idle_init_domains); + struct device *psci_dt_attach_cpu(int cpu) { struct device *dev; diff --git a/drivers/cpuidle/cpuidle-psci.c b/drivers/cpuidle/cpuidle-psci.c index 0707222a40bd..835c7c3aa118 100644 --- a/drivers/cpuidle/cpuidle-psci.c +++ b/drivers/cpuidle/cpuidle-psci.c @@ -32,7 +32,7 @@ struct psci_cpuidle_data { static DEFINE_PER_CPU_READ_MOSTLY(struct psci_cpuidle_data, psci_cpuidle_data); static DEFINE_PER_CPU(u32, domain_state); -static inline void psci_set_domain_state(u32 state) +void psci_set_domain_state(u32 state) { __this_cpu_write(domain_state, state); } @@ -103,7 +103,7 @@ static const struct of_device_id psci_idle_state_match[] __initconst = { { }, }; -static int __init psci_dt_parse_state_node(struct device_node *np, u32 *state) +int __init psci_dt_parse_state_node(struct device_node *np, u32 *state) { int err = of_property_read_u32(np, "arm,psci-suspend-param", state); diff --git a/drivers/cpuidle/cpuidle-psci.h b/drivers/cpuidle/cpuidle-psci.h index 0cadbb71dc55..d2e55cad9ac6 100644 --- a/drivers/cpuidle/cpuidle-psci.h +++ b/drivers/cpuidle/cpuidle-psci.h @@ -3,6 +3,11 @@ #ifndef __CPUIDLE_PSCI_H #define __CPUIDLE_PSCI_H +struct device_node; + +void psci_set_domain_state(u32 state); +int __init psci_dt_parse_state_node(struct device_node *np, u32 *state); + #ifdef CONFIG_PM_GENERIC_DOMAINS_OF struct device *psci_dt_attach_cpu(int cpu); #else -- 2.17.1
To enable the OS to better support PSCI OS initiated CPU suspend mode, let's convert from the flattened layout to the hierarchical layout. In the hierarchical layout, let's create a power domain provider per CPU and describe the idle states for each CPU inside the power domain provider node. To group the CPUs into a cluster, let's add another power domain provider and make it act as the master domain. Note that, the CPU's idle states remains compatible with "arm,idle-state", while the cluster's idle state becomes compatible with "domain-idle-state". Co-developed-by: Lina Iyer <lina.iyer@linaro.org> Signed-off-by: Lina Iyer <lina.iyer@linaro.org> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> --- Changes in v3: - None. --- arch/arm64/boot/dts/qcom/msm8916.dtsi | 57 +++++++++++++++++++++++++-- 1 file changed, 53 insertions(+), 4 deletions(-) diff --git a/arch/arm64/boot/dts/qcom/msm8916.dtsi b/arch/arm64/boot/dts/qcom/msm8916.dtsi index 5ea9fb8f2f87..1ece0c763592 100644 --- a/arch/arm64/boot/dts/qcom/msm8916.dtsi +++ b/arch/arm64/boot/dts/qcom/msm8916.dtsi @@ -102,10 +102,11 @@ reg = <0x0>; next-level-cache = <&L2_0>; enable-method = "psci"; - cpu-idle-states = <&CPU_SLEEP_0>; clocks = <&apcs>; operating-points-v2 = <&cpu_opp_table>; #cooling-cells = <2>; + power-domains = <&CPU_PD0>; + power-domain-names = "psci"; }; CPU1: cpu@1 { @@ -114,10 +115,11 @@ reg = <0x1>; next-level-cache = <&L2_0>; enable-method = "psci"; - cpu-idle-states = <&CPU_SLEEP_0>; clocks = <&apcs>; operating-points-v2 = <&cpu_opp_table>; #cooling-cells = <2>; + power-domains = <&CPU_PD1>; + power-domain-names = "psci"; }; CPU2: cpu@2 { @@ -126,10 +128,11 @@ reg = <0x2>; next-level-cache = <&L2_0>; enable-method = "psci"; - cpu-idle-states = <&CPU_SLEEP_0>; clocks = <&apcs>; operating-points-v2 = <&cpu_opp_table>; #cooling-cells = <2>; + power-domains = <&CPU_PD2>; + power-domain-names = "psci"; }; CPU3: cpu@3 { @@ -138,10 +141,11 @@ reg = <0x3>; next-level-cache = <&L2_0>; enable-method = "psci"; - cpu-idle-states = <&CPU_SLEEP_0>; clocks = <&apcs>; operating-points-v2 = <&cpu_opp_table>; #cooling-cells = <2>; + power-domains = <&CPU_PD3>; + power-domain-names = "psci"; }; L2_0: l2-cache { @@ -161,12 +165,57 @@ min-residency-us = <2000>; local-timer-stop; }; + + CLUSTER_RET: cluster-retention { + compatible = "domain-idle-state"; + arm,psci-suspend-param = <0x41000012>; + entry-latency-us = <500>; + exit-latency-us = <500>; + min-residency-us = <2000>; + }; + + CLUSTER_PWRDN: cluster-gdhs { + compatible = "domain-idle-state"; + arm,psci-suspend-param = <0x41000032>; + entry-latency-us = <2000>; + exit-latency-us = <2000>; + min-residency-us = <6000>; + }; }; }; psci { compatible = "arm,psci-1.0"; method = "smc"; + + CPU_PD0: cpu-pd0 { + #power-domain-cells = <0>; + power-domains = <&CLUSTER_PD>; + domain-idle-states = <&CPU_SLEEP_0>; + }; + + CPU_PD1: cpu-pd1 { + #power-domain-cells = <0>; + power-domains = <&CLUSTER_PD>; + domain-idle-states = <&CPU_SLEEP_0>; + }; + + CPU_PD2: cpu-pd2 { + #power-domain-cells = <0>; + power-domains = <&CLUSTER_PD>; + domain-idle-states = <&CPU_SLEEP_0>; + }; + + CPU_PD3: cpu-pd3 { + #power-domain-cells = <0>; + power-domains = <&CLUSTER_PD>; + domain-idle-states = <&CPU_SLEEP_0>; + }; + + CLUSTER_PD: cluster-pd { + #power-domain-cells = <0>; + domain-idle-states = <&CLUSTER_RET>, <&CLUSTER_PWRDN>; + }; }; pmu { -- 2.17.1
On Wed, Nov 27, 2019 at 11:29:09AM +0100, Ulf Hansson wrote:
[...]
> +struct device *psci_dt_attach_cpu(int cpu)
> +{
> + struct device *dev;
> +
> + /* Currently limit the hierarchical topology to be used in OSI mode. */
> + if (!psci_has_osi_support())
> + return NULL;
> +
> + dev = dev_pm_domain_attach_by_name(get_cpu_device(cpu), "psci");
> + if (IS_ERR_OR_NULL(dev))
> + return dev;
> +
> + pm_runtime_irq_safe(dev);
> + if (cpu_online(cpu))
It is unclear to me how we handle (or rather we don't) CPU hotplug
with this series - it does not look OK unless genpd code manages
that automatically.
Lorenzo
On Thu, 28 Nov 2019 at 15:15, Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Wed, Nov 27, 2019 at 11:29:09AM +0100, Ulf Hansson wrote:
>
> [...]
>
> > +struct device *psci_dt_attach_cpu(int cpu)
> > +{
> > + struct device *dev;
> > +
> > + /* Currently limit the hierarchical topology to be used in OSI mode. */
> > + if (!psci_has_osi_support())
> > + return NULL;
> > +
> > + dev = dev_pm_domain_attach_by_name(get_cpu_device(cpu), "psci");
> > + if (IS_ERR_OR_NULL(dev))
> > + return dev;
> > +
> > + pm_runtime_irq_safe(dev);
> > + if (cpu_online(cpu))
>
> It is unclear to me how we handle (or rather we don't) CPU hotplug
> with this series - it does not look OK unless genpd code manages
> that automatically.
The series doesn't handle CPU hotplug at the moment, simply because I
am targeting to get the basic support, upstream first.
For a functionality point of view, this isn't a problem in my opinion.
Simply because the consequence is only that the idle states for the
"cluster" will not be reached if there is a CPU brought offline.
As we talked about at LPC and as also told Sudeep for the v2 series,
CPU hotplug is going to be implemented by using a CPU HP notifier.
That should be fine, right?
Kind regards
Uffe
On Thu, Nov 28, 2019 at 06:21:03PM +0100, Ulf Hansson wrote: > On Thu, 28 Nov 2019 at 15:15, Lorenzo Pieralisi > <lorenzo.pieralisi@arm.com> wrote: > > > > On Wed, Nov 27, 2019 at 11:29:09AM +0100, Ulf Hansson wrote: > > > > [...] > > > > > +struct device *psci_dt_attach_cpu(int cpu) > > > +{ > > > + struct device *dev; > > > + > > > + /* Currently limit the hierarchical topology to be used in OSI mode. */ > > > + if (!psci_has_osi_support()) > > > + return NULL; > > > + > > > + dev = dev_pm_domain_attach_by_name(get_cpu_device(cpu), "psci"); > > > + if (IS_ERR_OR_NULL(dev)) > > > + return dev; > > > + > > > + pm_runtime_irq_safe(dev); > > > + if (cpu_online(cpu)) > > > > It is unclear to me how we handle (or rather we don't) CPU hotplug > > with this series - it does not look OK unless genpd code manages > > that automatically. > > The series doesn't handle CPU hotplug at the moment, simply because I > am targeting to get the basic support, upstream first. Basic support must work and that includes CPU hotplug - I don't want to merge code that work with assumptions that aren't valid. > For a functionality point of view, this isn't a problem in my opinion. > Simply because the consequence is only that the idle states for the > "cluster" will not be reached if there is a CPU brought offline. > > As we talked about at LPC and as also told Sudeep for the v2 series, > CPU hotplug is going to be implemented by using a CPU HP notifier. Yes, it should be part of the series. > That should be fine, right? Yes, hopefully but it has to be part of the series. Thanks, Lorenzo
On Thu, 28 Nov 2019 at 19:31, Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Thu, Nov 28, 2019 at 06:21:03PM +0100, Ulf Hansson wrote:
> > On Thu, 28 Nov 2019 at 15:15, Lorenzo Pieralisi
> > <lorenzo.pieralisi@arm.com> wrote:
> > >
> > > On Wed, Nov 27, 2019 at 11:29:09AM +0100, Ulf Hansson wrote:
> > >
> > > [...]
> > >
> > > > +struct device *psci_dt_attach_cpu(int cpu)
> > > > +{
> > > > + struct device *dev;
> > > > +
> > > > + /* Currently limit the hierarchical topology to be used in OSI mode. */
> > > > + if (!psci_has_osi_support())
> > > > + return NULL;
> > > > +
> > > > + dev = dev_pm_domain_attach_by_name(get_cpu_device(cpu), "psci");
> > > > + if (IS_ERR_OR_NULL(dev))
> > > > + return dev;
> > > > +
> > > > + pm_runtime_irq_safe(dev);
> > > > + if (cpu_online(cpu))
> > >
> > > It is unclear to me how we handle (or rather we don't) CPU hotplug
> > > with this series - it does not look OK unless genpd code manages
> > > that automatically.
> >
> > The series doesn't handle CPU hotplug at the moment, simply because I
> > am targeting to get the basic support, upstream first.
>
> Basic support must work and that includes CPU hotplug - I don't want
> to merge code that work with assumptions that aren't valid.
>
> > For a functionality point of view, this isn't a problem in my opinion.
> > Simply because the consequence is only that the idle states for the
> > "cluster" will not be reached if there is a CPU brought offline.
> >
> > As we talked about at LPC and as also told Sudeep for the v2 series,
> > CPU hotplug is going to be implemented by using a CPU HP notifier.
>
> Yes, it should be part of the series.
>
> > That should be fine, right?
>
> Yes, hopefully but it has to be part of the series.
Alright, I send a patch on top, asap.
Please review the rest and if it needs a respin, I can fold it into the series.
Kind regards
Uffe
On Wed, Nov 27, 2019 at 11:29:11AM +0100, Ulf Hansson wrote: [...] > -static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, > +static int __init psci_dt_cpu_init_idle(struct cpuidle_driver *drv, > + struct device_node *cpu_node, > unsigned int state_count, int cpu) > { > int i, ret = 0; > @@ -118,6 +152,11 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, > goto free_mem; > } > > + /* Manage the deepest state via a dedicated enter-function. */ > + if (dev) > + drv->states[state_count - 1].enter = > + psci_enter_domain_idle_state; It is unfortunate to make this arbitrary choice, it would be best if you could detect which states are "domain" states aka are governed by multiple cpus. This inizialization though does not belong in here, it is done at driver level, it should not be done in this per-cpu path. IIUC the logic the enter pointer should only be overridden if and only if all cpus managed by the driver have a corresponding device associated. To be frank I would even move the psci_has_osi_support() check from psci_dt_attach_cpu() to this path and prevent calling psci_dt_attach_cpu() and the tail of the function if (!psci_has_osi_support()). > data->dev = dev; I think Sudeep already mentioned that, by using psci_has_osi_support() as above you can prevent running this code, there is really no point, the data->dev NULL sentinel is already initialized. Lorenzo > /* Idle states parsed correctly, store them in the per-cpu struct. */ > @@ -129,7 +168,8 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, > return ret; > } > > -static __init int psci_cpu_init_idle(unsigned int cpu, unsigned int state_count) > +static __init int psci_cpu_init_idle(struct cpuidle_driver *drv, > + unsigned int cpu, unsigned int state_count) > { > struct device_node *cpu_node; > int ret; > @@ -145,7 +185,7 @@ static __init int psci_cpu_init_idle(unsigned int cpu, unsigned int state_count) > if (!cpu_node) > return -ENODEV; > > - ret = psci_dt_cpu_init_idle(cpu_node, state_count, cpu); > + ret = psci_dt_cpu_init_idle(drv, cpu_node, state_count, cpu); > > of_node_put(cpu_node); > > @@ -201,7 +241,7 @@ static int __init psci_idle_init_cpu(int cpu) > /* > * Initialize PSCI idle states. > */ > - ret = psci_cpu_init_idle(cpu, ret); > + ret = psci_cpu_init_idle(drv, cpu, ret); > if (ret) { > pr_err("CPU %d failed to PSCI idle\n", cpu); > goto out_kfree_drv; > -- > 2.17.1 >
On Thu, 5 Dec 2019 at 19:35, Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> wrote: > > On Wed, Nov 27, 2019 at 11:29:11AM +0100, Ulf Hansson wrote: > > [...] > > > -static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, > > +static int __init psci_dt_cpu_init_idle(struct cpuidle_driver *drv, > > + struct device_node *cpu_node, > > unsigned int state_count, int cpu) > > { > > int i, ret = 0; > > @@ -118,6 +152,11 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, > > goto free_mem; > > } > > > > + /* Manage the deepest state via a dedicated enter-function. */ > > + if (dev) > > + drv->states[state_count - 1].enter = > > + psci_enter_domain_idle_state; > > > It is unfortunate to make this arbitrary choice, it would be best > if you could detect which states are "domain" states aka are governed > by multiple cpus. The domain states are managed and selected by the genpd providers, via using runtime PM reference counting. Please have a closer look at the code in cpuidle-psci-domain.c and in the generic PM domain, that should give you the needed details. I am overriding the enter callback for the *deepest* known idle state of the CPU, which is according to what you requested [1]. So, unless I am missing your point, I think the above code does exactly what you want, no? In regards to the "arbitrary choice" of what cpuidle state to use, there are more details about why that is, in the changelog. > > This inizialization though does not belong in here, it is done at driver > level, it should not be done in this per-cpu path. IIUC the logic the > enter pointer should only be overridden if and only if all cpus managed > by the driver have a corresponding device associated. I think you have overlooked the fact that there are one cpuidle driver registered per CPU. The above doesn't make sense to me, sorry. > > To be frank I would even move the psci_has_osi_support() check from > psci_dt_attach_cpu() to this path and prevent calling > psci_dt_attach_cpu() and the tail of the function if > (!psci_has_osi_support()). > > > data->dev = dev; > > I think Sudeep already mentioned that, by using psci_has_osi_support() > as above you can prevent running this code, there is really no point, > the data->dev NULL sentinel is already initialized. Yes, I discussed this with Sudeep, but we didn't reach a consensus. Let me explain the reasons behind the selected approach, once more. The data->dev is a pointer within a static declared struct. Are you sure it's assigned NULL by initialization? Don't we explicitly need to set it to NULL, else it will be undefined, no? Of course, I can move the check for psci_has_osi_support() into here and avoid calling psci_dt_attach_cpu(). Just wondering what that actually gain us, especially if we need to explicitly set the pointer to NULL anyway. That said, can you please confirm your thoughts around this, I will change to whatever you think is best. [...] Kind regards Uffe [1] https://www.spinics.net/lists/arm-kernel/msg770558.html
On Thu, 5 Dec 2019 at 21:25, Ulf Hansson <ulf.hansson@linaro.org> wrote:
>
> On Thu, 5 Dec 2019 at 19:35, Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
> >
> > On Wed, Nov 27, 2019 at 11:29:11AM +0100, Ulf Hansson wrote:
> >
> > [...]
> >
> > > -static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node,
> > > +static int __init psci_dt_cpu_init_idle(struct cpuidle_driver *drv,
> > > + struct device_node *cpu_node,
> > > unsigned int state_count, int cpu)
> > > {
> > > int i, ret = 0;
> > > @@ -118,6 +152,11 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node,
> > > goto free_mem;
> > > }
> > >
> > > + /* Manage the deepest state via a dedicated enter-function. */
> > > + if (dev)
> > > + drv->states[state_count - 1].enter =
> > > + psci_enter_domain_idle_state;
> >
> >
> > It is unfortunate to make this arbitrary choice, it would be best
> > if you could detect which states are "domain" states aka are governed
> > by multiple cpus.
>
> The domain states are managed and selected by the genpd providers, via
> using runtime PM reference counting. Please have a closer look at the
> code in cpuidle-psci-domain.c and in the generic PM domain, that
> should give you the needed details.
>
> I am overriding the enter callback for the *deepest* known idle state
> of the CPU, which is according to what you requested [1].
>
> So, unless I am missing your point, I think the above code does
> exactly what you want, no?
>
> In regards to the "arbitrary choice" of what cpuidle state to use,
> there are more details about why that is, in the changelog.
Correction: Since I have moved patches around, I realized that the
explanation is actually put in the changelog of patch11.
For clarity, let me cut and paste it here as well:
"The triggering point for when runtime PM reference counting should be done,
has been selected to the deepest idle state for the CPU. However, from the
hierarchical point view, there may be good reasons to do runtime PM
reference counting even on shallower idle states, but at this point this
isn't supported, mainly due to limitations set by the generic PM domain."
Is that good enough or you want some of this information also in the
changelog of $subject patch? Or if you have any other idea for how to
make this more clear?
[...]
Kind regards
Uffe
On Thu, Dec 05, 2019 at 09:25:54PM +0100, Ulf Hansson wrote: > On Thu, 5 Dec 2019 at 19:35, Lorenzo Pieralisi > <lorenzo.pieralisi@arm.com> wrote: > > > > On Wed, Nov 27, 2019 at 11:29:11AM +0100, Ulf Hansson wrote: > > > > [...] > > > > > -static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, > > > +static int __init psci_dt_cpu_init_idle(struct cpuidle_driver *drv, > > > + struct device_node *cpu_node, > > > unsigned int state_count, int cpu) > > > { > > > int i, ret = 0; > > > @@ -118,6 +152,11 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, > > > goto free_mem; > > > } > > > > > > + /* Manage the deepest state via a dedicated enter-function. */ > > > + if (dev) > > > + drv->states[state_count - 1].enter = > > > + psci_enter_domain_idle_state; > > > > > > It is unfortunate to make this arbitrary choice, it would be best > > if you could detect which states are "domain" states aka are governed > > by multiple cpus. > > The domain states are managed and selected by the genpd providers, via > using runtime PM reference counting. Please have a closer look at the > code in cpuidle-psci-domain.c and in the generic PM domain, that > should give you the needed details. > > I am overriding the enter callback for the *deepest* known idle state > of the CPU, which is according to what you requested [1]. Overriding it yes but I have not requested to do it only for the deepest idle state that, I repeat, in my opinion is an arbitrary choice that works for the platform you are testing on but is not generic as it should. You can merge it as it is but that's how things stand and adding a comment to the *code* would help understand its logic. > So, unless I am missing your point, I think the above code does > exactly what you want, no? > > In regards to the "arbitrary choice" of what cpuidle state to use, > there are more details about why that is, in the changelog. > > > > > This inizialization though does not belong in here, it is done at driver > > level, it should not be done in this per-cpu path. IIUC the logic the > > enter pointer should only be overridden if and only if all cpus managed > > by the driver have a corresponding device associated. > > I think you have overlooked the fact that there are one cpuidle driver > registered per CPU. The above doesn't make sense to me, sorry. You are calling psci_dt_cpu_init_idle() for every possibile cpu. Every time psci_dt_attach_cpu() is called, we check dev and override the idle driver enter method. There is one driver, what I am saying is that it is not correct to check dev and override the enter pointer for *every* cpu that we try to attach to a power domain. This must be done once for all by checking that *all* devices could be attached to a power domain. > > To be frank I would even move the psci_has_osi_support() check from > > psci_dt_attach_cpu() to this path and prevent calling > > psci_dt_attach_cpu() and the tail of the function if > > (!psci_has_osi_support()). > > > > > data->dev = dev; > > > > I think Sudeep already mentioned that, by using psci_has_osi_support() > > as above you can prevent running this code, there is really no point, > > the data->dev NULL sentinel is already initialized. > > Yes, I discussed this with Sudeep, but we didn't reach a consensus. Consensus was already reached. http://www.open-std.org/jtc1/sc22/WG14/www/docs/n1570.pdf > Let me explain the reasons behind the selected approach, once more. > > The data->dev is a pointer within a static declared struct. Are you > sure it's assigned NULL by initialization? Don't we explicitly need to > set it to NULL, else it will be undefined, no? http://www.open-std.org/jtc1/sc22/WG14/www/docs/n1570.pdf 6.7.9 (10) page 140 > Of course, I can move the check for psci_has_osi_support() into here > and avoid calling psci_dt_attach_cpu(). Just wondering what that > actually gain us, especially if we need to explicitly set the pointer > to NULL anyway. See above. Thanks, Lorenzo
On Fri, 6 Dec 2019 at 12:25, Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> wrote: > > On Thu, Dec 05, 2019 at 09:25:54PM +0100, Ulf Hansson wrote: > > On Thu, 5 Dec 2019 at 19:35, Lorenzo Pieralisi > > <lorenzo.pieralisi@arm.com> wrote: > > > > > > On Wed, Nov 27, 2019 at 11:29:11AM +0100, Ulf Hansson wrote: > > > > > > [...] > > > > > > > -static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, > > > > +static int __init psci_dt_cpu_init_idle(struct cpuidle_driver *drv, > > > > + struct device_node *cpu_node, > > > > unsigned int state_count, int cpu) > > > > { > > > > int i, ret = 0; > > > > @@ -118,6 +152,11 @@ static int __init psci_dt_cpu_init_idle(struct device_node *cpu_node, > > > > goto free_mem; > > > > } > > > > > > > > + /* Manage the deepest state via a dedicated enter-function. */ > > > > + if (dev) > > > > + drv->states[state_count - 1].enter = > > > > + psci_enter_domain_idle_state; > > > > > > > > > It is unfortunate to make this arbitrary choice, it would be best > > > if you could detect which states are "domain" states aka are governed > > > by multiple cpus. > > > > The domain states are managed and selected by the genpd providers, via > > using runtime PM reference counting. Please have a closer look at the > > code in cpuidle-psci-domain.c and in the generic PM domain, that > > should give you the needed details. > > > > I am overriding the enter callback for the *deepest* known idle state > > of the CPU, which is according to what you requested [1]. > > Overriding it yes but I have not requested to do it only for the > deepest idle state that, I repeat, in my opinion is an arbitrary > choice that works for the platform you are testing on but is > not generic as it should. Right, I agree. I recall we have discussed this already. > > You can merge it as it is but that's how things stand and adding > a comment to the *code* would help understand its logic. Okay, how about adding a comment along the lines of this: "Using the deepest state for the CPU to trigger a potential selection of a shared state for the domain, assumes the domain states are all deeper states. This assumption may not be true for all platforms, thus we may consider to revisit this, if it turns out that optimizations can be made." > > > So, unless I am missing your point, I think the above code does > > exactly what you want, no? > > > > In regards to the "arbitrary choice" of what cpuidle state to use, > > there are more details about why that is, in the changelog. > > > > > > > > This inizialization though does not belong in here, it is done at driver > > > level, it should not be done in this per-cpu path. IIUC the logic the > > > enter pointer should only be overridden if and only if all cpus managed > > > by the driver have a corresponding device associated. > > > > I think you have overlooked the fact that there are one cpuidle driver > > registered per CPU. The above doesn't make sense to me, sorry. > > You are calling psci_dt_cpu_init_idle() for every possibile cpu. > > Every time psci_dt_attach_cpu() is called, we check dev and override > the idle driver enter method. There is one driver, what I am saying > is that it is not correct to check dev and override the enter pointer > for *every* cpu that we try to attach to a power domain. This must > be done once for all by checking that *all* devices could be attached > to a power domain. Ah, now I think get your point. You want me to re-iterate through all the registered cpuidle drivers, which means one per CPU - and then override the enter callback for each of them, but only if all devices was successfully attached to a PM domain. Is that correct? My only worries with this, is that we have already registered the cpuidle drivers and I don't think it's a good idea to update the enter callbacks, beyond that point. Perhaps another option is to track whether the first CPU gets attached (and then update the enter callback), but after that require all the remaining CPUs to be attached as well - else bail out with an error code, failing to register all the driver instances. What do you think about that? > > > > To be frank I would even move the psci_has_osi_support() check from > > > psci_dt_attach_cpu() to this path and prevent calling > > > psci_dt_attach_cpu() and the tail of the function if > > > (!psci_has_osi_support()). > > > > > > > data->dev = dev; > > > > > > I think Sudeep already mentioned that, by using psci_has_osi_support() > > > as above you can prevent running this code, there is really no point, > > > the data->dev NULL sentinel is already initialized. > > > > Yes, I discussed this with Sudeep, but we didn't reach a consensus. > > Consensus was already reached. > > http://www.open-std.org/jtc1/sc22/WG14/www/docs/n1570.pdf > > > Let me explain the reasons behind the selected approach, once more. > > > > The data->dev is a pointer within a static declared struct. Are you > > sure it's assigned NULL by initialization? Don't we explicitly need to > > set it to NULL, else it will be undefined, no? > > http://www.open-std.org/jtc1/sc22/WG14/www/docs/n1570.pdf > > 6.7.9 (10) page 140 Thanks for sharing, didn't know we could rely on this behaviour! Alright! > > > Of course, I can move the check for psci_has_osi_support() into here > > and avoid calling psci_dt_attach_cpu(). Just wondering what that > > actually gain us, especially if we need to explicitly set the pointer > > to NULL anyway. > > See above. Yes, makes more sense now. I will adopt your suggestions! Kind regards Uffe
On Fri, Dec 06, 2019 at 03:26:16PM +0100, Ulf Hansson wrote: [...] > > You can merge it as it is but that's how things stand and adding > > a comment to the *code* would help understand its logic. > > Okay, how about adding a comment along the lines of this: > > "Using the deepest state for the CPU to trigger a potential selection > of a shared state for the domain, assumes the domain states are all > deeper states". Just this it should be fine (I trimmed it a bit). > > > So, unless I am missing your point, I think the above code does > > > exactly what you want, no? > > > > > > In regards to the "arbitrary choice" of what cpuidle state to use, > > > there are more details about why that is, in the changelog. > > > > > > > > > > > This inizialization though does not belong in here, it is done at driver > > > > level, it should not be done in this per-cpu path. IIUC the logic the > > > > enter pointer should only be overridden if and only if all cpus managed > > > > by the driver have a corresponding device associated. > > > > > > I think you have overlooked the fact that there are one cpuidle driver > > > registered per CPU. The above doesn't make sense to me, sorry. > > > > You are calling psci_dt_cpu_init_idle() for every possibile cpu. > > > > Every time psci_dt_attach_cpu() is called, we check dev and override > > the idle driver enter method. There is one driver, what I am saying > > is that it is not correct to check dev and override the enter pointer > > for *every* cpu that we try to attach to a power domain. This must > > be done once for all by checking that *all* devices could be attached > > to a power domain. > > Ah, now I think get your point. > > You want me to re-iterate through all the registered cpuidle drivers, > which means one per CPU - and then override the enter callback for > each of them, but only if all devices was successfully attached to a > PM domain. Is that correct? > > My only worries with this, is that we have already registered the > cpuidle drivers and I don't think it's a good idea to update the enter > callbacks, beyond that point. > > Perhaps another option is to track whether the first CPU gets attached > (and then update the enter callback), but after that require all the > remaining CPUs to be attached as well - else bail out with an error > code, failing to register all the driver instances. > > What do you think about that? I was confused - now we have one cpuidle driver per cpu so this comment was bogus from this perspective (I was still reasoning wit a *single* cpuidle driver across cpus. Apologies). Sudeep will follow up on this but please forget this specific comment - I was wrong. Thanks, Lorenzo
On Fri, 6 Dec 2019 at 16:14, Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> wrote: > > On Fri, Dec 06, 2019 at 03:26:16PM +0100, Ulf Hansson wrote: > > [...] > > > > You can merge it as it is but that's how things stand and adding > > > a comment to the *code* would help understand its logic. > > > > Okay, how about adding a comment along the lines of this: > > > > "Using the deepest state for the CPU to trigger a potential selection > > of a shared state for the domain, assumes the domain states are all > > deeper states". > > Just this it should be fine (I trimmed it a bit). Great, I add that! > > > > > So, unless I am missing your point, I think the above code does > > > > exactly what you want, no? > > > > > > > > In regards to the "arbitrary choice" of what cpuidle state to use, > > > > there are more details about why that is, in the changelog. > > > > > > > > > > > > > > This inizialization though does not belong in here, it is done at driver > > > > > level, it should not be done in this per-cpu path. IIUC the logic the > > > > > enter pointer should only be overridden if and only if all cpus managed > > > > > by the driver have a corresponding device associated. > > > > > > > > I think you have overlooked the fact that there are one cpuidle driver > > > > registered per CPU. The above doesn't make sense to me, sorry. > > > > > > You are calling psci_dt_cpu_init_idle() for every possibile cpu. > > > > > > Every time psci_dt_attach_cpu() is called, we check dev and override > > > the idle driver enter method. There is one driver, what I am saying > > > is that it is not correct to check dev and override the enter pointer > > > for *every* cpu that we try to attach to a power domain. This must > > > be done once for all by checking that *all* devices could be attached > > > to a power domain. > > > > Ah, now I think get your point. > > > > You want me to re-iterate through all the registered cpuidle drivers, > > which means one per CPU - and then override the enter callback for > > each of them, but only if all devices was successfully attached to a > > PM domain. Is that correct? > > > > My only worries with this, is that we have already registered the > > cpuidle drivers and I don't think it's a good idea to update the enter > > callbacks, beyond that point. > > > > Perhaps another option is to track whether the first CPU gets attached > > (and then update the enter callback), but after that require all the > > remaining CPUs to be attached as well - else bail out with an error > > code, failing to register all the driver instances. > > > > What do you think about that? > > I was confused - now we have one cpuidle driver per cpu so this > comment was bogus from this perspective (I was still reasoning > wit a *single* cpuidle driver across cpus. Apologies). No worries! We agreed on the way forward, so I am happy. :-) > > Sudeep will follow up on this but please forget this specific > comment - I was wrong. Alright, thanks! Does that also mean I can add your ack for the rest of the patches in the series (besides the last hotplug patch) or is there any other issues you want to raise? Have a nice weekend! Kind regards Uffe