All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2 0/7] The power allocator thermal governor
@ 2015-02-26 19:00 Javi Merino
  2015-02-26 19:00 ` [PATCH v2 1/7] thermal: let governors have private data for each thermal zone Javi Merino
                   ` (7 more replies)
  0 siblings, 8 replies; 22+ messages in thread
From: Javi Merino @ 2015-02-26 19:00 UTC (permalink / raw)
  To: edubezval, rui.zhang
  Cc: linux-pm, linux-kernel, punit.agrawal, broonie, tixy, Javi Merino

Hi linux-pm,

The power allocator governor allocates device power to control
temperature.  This requires transforming performance requests into
requested power, which we do with an extended cooling device API
introduced in patch 2 (thermal: extend the cooling device API to
include power information).  Patch 3 (thermal: cpu_cooling: implement
the power cooling device API) extends the cpu cooling device using a
simple power model.

This series are based on branch "linus" of Eduardo's linux-soc-thermal
tree:

git://git.kernel.org/pub/scm/linux/kernel/git/evalenti/linux-soc-thermal.git

Changes since v1:
  - Removed a memory leak pointed out by Steve Rostedt
  - Relax the constraint of the trip points to be the first and last
    passive trip point as Lina Iyer suggested.
  - Update how the governor treats weights to mimic the behavior of
    fair share in v3 of the weight fixes series.
  - Don't cache the cpufreq_cooling_device and scan for it whenever
    we need it.
  - Consider the rate of changes in the derivative term of the PID
    controller

Changes since RFC v6:
  - Addressed Eduardo's review
    + Pass the interval to the static power function as suggested by
      Eduardo
    + Make the cooling device ops return 0 or -E* and put the
      calculation in a parameter, like the rest of the cooling device
      ops
    + Documentation improvements
  - Use thermal_cdev_update() to change cooling device states
  - Add a patch to export the power allocator governor's tzp
    parameters to sysfs

Changes since RFC v5:
  - Addressed Stephen's review of the trace patches.
  - Removed power actors and extended the cooling device interface
    instead.
  - Let platforms override the power allocator governor parameters in
    their thermal zone parameters

Changes since RFC v4:
  - Add more tracing
  - Document some of the limitations of the power allocator governor
  - Export the power_actor API and move power_actor.h to include/linux

Changes since RFC v3:
  - Use tz->passive to poll faster when the first trip point is hit.
  - Don't make a special directory for power_actors
  - Add a DT property for sustainable-power
  - Simplify the static power interface and pass the current thermal
    zone in every power_actor_ops to remove the controversial
    enum power_actor_types
  - Use locks with the actor_list list
  - Use cpufreq_get() to get the frequency of the cpu instead of
    using the notifiers.
  - Remove the prompt for THERMAL_POWER_ACTOR_CPU when configuring
    the kernel

Changes since RFC v2:
  - Changed the PI controller into a PID controller
  - Added static power to the cpu power model
  - tz parameter max_dissipatable_power renamed to sustainable_power
  - Register the cpufreq cooling device as part of the
    power_cpu_actor registration.

Changes since RFC v1:
  - Fixed finding cpufreq cooling devices in cpufreq_frequency_change()
  - Replaced the cooling device interface with a separate power actor
    API
  - Addressed most of Eduardo's comments
  - Incorporated ftrace support for bitmask to trace cpumasks

Cheers,
Javi & Punit

Javi Merino (6):
  thermal: let governors have private data for each thermal zone
  thermal: extend the cooling device API to include power information
  thermal: cpu_cooling: implement the power cooling device API
  thermal: introduce the Power Allocator governor
  thermal: add trace events to the power allocator governor
  thermal: export thermal_zone_parameters to sysfs

Punit Agrawal (1):
  of: thermal: Introduce sustainable power for a thermal zone

 .../devicetree/bindings/thermal/thermal.txt        |   9 +
 Documentation/thermal/cpu-cooling-api.txt          | 156 +++++-
 Documentation/thermal/power_allocator.txt          | 247 +++++++++
 Documentation/thermal/sysfs-api.txt                |  52 ++
 drivers/thermal/Kconfig                            |  15 +
 drivers/thermal/Makefile                           |   1 +
 drivers/thermal/cpu_cooling.c                      | 610 ++++++++++++++++++++-
 drivers/thermal/of-thermal.c                       |   4 +
 drivers/thermal/power_allocator.c                  | 533 ++++++++++++++++++
 drivers/thermal/thermal_core.c                     | 254 ++++++++-
 drivers/thermal/thermal_core.h                     |   8 +
 include/linux/cpu_cooling.h                        |  39 ++
 include/linux/thermal.h                            |  64 ++-
 include/trace/events/thermal.h                     |  58 ++
 include/trace/events/thermal_power_allocator.h     |  87 +++
 15 files changed, 2104 insertions(+), 33 deletions(-)
 create mode 100644 Documentation/thermal/power_allocator.txt
 create mode 100644 drivers/thermal/power_allocator.c
 create mode 100644 include/trace/events/thermal_power_allocator.h

-- 
1.9.1


^ permalink raw reply	[flat|nested] 22+ messages in thread

* [PATCH v2 1/7] thermal: let governors have private data for each thermal zone
  2015-02-26 19:00 [PATCH v2 0/7] The power allocator thermal governor Javi Merino
@ 2015-02-26 19:00 ` Javi Merino
  2015-02-26 19:00 ` [PATCH v2 2/7] thermal: extend the cooling device API to include power information Javi Merino
                   ` (6 subsequent siblings)
  7 siblings, 0 replies; 22+ messages in thread
From: Javi Merino @ 2015-02-26 19:00 UTC (permalink / raw)
  To: edubezval, rui.zhang
  Cc: linux-pm, linux-kernel, punit.agrawal, broonie, tixy, Javi Merino

A governor may need to store its current state between calls to
throttle().  That state depends on the thermal zone, so store it as
private data in struct thermal_zone_device.

The governors may have two new ops: bind_to_tz() and unbind_from_tz().
When provided, these functions let governors do some initialization
and teardown when they are bound/unbound to a tz and possibly store that
information in the governor_data field of the struct
thermal_zone_device.

Cc: Zhang Rui <rui.zhang@intel.com>
Cc: Eduardo Valentin <edubezval@gmail.com>
Signed-off-by: Javi Merino <javi.merino@arm.com>
---
 drivers/thermal/thermal_core.c | 83 ++++++++++++++++++++++++++++++++++++++----
 include/linux/thermal.h        |  9 +++++
 2 files changed, 84 insertions(+), 8 deletions(-)

diff --git a/drivers/thermal/thermal_core.c b/drivers/thermal/thermal_core.c
index c51ccc82580c..3bbda8979049 100644
--- a/drivers/thermal/thermal_core.c
+++ b/drivers/thermal/thermal_core.c
@@ -75,6 +75,58 @@ static struct thermal_governor *__find_governor(const char *name)
 	return NULL;
 }
 
+/**
+ * bind_previous_governor() - bind the previous governor of the thermal zone
+ * @tz:		a valid pointer to a struct thermal_zone_device
+ * @failed_gov_name:	the name of the governor that failed to register
+ *
+ * Register the previous governor of the thermal zone after a new
+ * governor has failed to be bound.
+ */
+static void bind_previous_governor(struct thermal_zone_device *tz,
+				   const char *failed_gov_name)
+{
+	if (tz->governor && tz->governor->bind_to_tz) {
+		if (tz->governor->bind_to_tz(tz)) {
+			dev_err(&tz->device,
+				"governor %s failed to bind and the previous one (%s) failed to bind again, thermal zone %s has no governor\n",
+				failed_gov_name, tz->governor->name, tz->type);
+			tz->governor = NULL;
+		}
+	}
+}
+
+/**
+ * thermal_set_governor() - Switch to another governor
+ * @tz:		a valid pointer to a struct thermal_zone_device
+ * @new_gov:	pointer to the new governor
+ *
+ * Change the governor of thermal zone @tz.
+ *
+ * Return: 0 on success, an error if the new governor's bind_to_tz() failed.
+ */
+static int thermal_set_governor(struct thermal_zone_device *tz,
+				struct thermal_governor *new_gov)
+{
+	int ret = 0;
+
+	if (tz->governor && tz->governor->unbind_from_tz)
+		tz->governor->unbind_from_tz(tz);
+
+	if (new_gov && new_gov->bind_to_tz) {
+		ret = new_gov->bind_to_tz(tz);
+		if (ret) {
+			bind_previous_governor(tz, new_gov->name);
+
+			return ret;
+		}
+	}
+
+	tz->governor = new_gov;
+
+	return ret;
+}
+
 int thermal_register_governor(struct thermal_governor *governor)
 {
 	int err;
@@ -107,8 +159,15 @@ int thermal_register_governor(struct thermal_governor *governor)
 
 		name = pos->tzp->governor_name;
 
-		if (!strncasecmp(name, governor->name, THERMAL_NAME_LENGTH))
-			pos->governor = governor;
+		if (!strncasecmp(name, governor->name, THERMAL_NAME_LENGTH)) {
+			int ret;
+
+			ret = thermal_set_governor(pos, governor);
+			if (ret)
+				dev_err(&pos->device,
+					"Failed to set governor %s for thermal zone %s: %d\n",
+					governor->name, pos->type, ret);
+		}
 	}
 
 	mutex_unlock(&thermal_list_lock);
@@ -134,7 +193,7 @@ void thermal_unregister_governor(struct thermal_governor *governor)
 	list_for_each_entry(pos, &thermal_tz_list, node) {
 		if (!strncasecmp(pos->governor->name, governor->name,
 						THERMAL_NAME_LENGTH))
-			pos->governor = NULL;
+			thermal_set_governor(pos, NULL);
 	}
 
 	mutex_unlock(&thermal_list_lock);
@@ -768,8 +827,9 @@ policy_store(struct device *dev, struct device_attribute *attr,
 	if (!gov)
 		goto exit;
 
-	tz->governor = gov;
-	ret = count;
+	ret = thermal_set_governor(tz, gov);
+	if (!ret)
+		ret = count;
 
 exit:
 	mutex_unlock(&tz->lock);
@@ -1513,6 +1573,7 @@ struct thermal_zone_device *thermal_zone_device_register(const char *type,
 	int result;
 	int count;
 	int passive = 0;
+	struct thermal_governor *governor;
 
 	if (type && strlen(type) >= THERMAL_NAME_LENGTH)
 		return ERR_PTR(-EINVAL);
@@ -1603,9 +1664,15 @@ struct thermal_zone_device *thermal_zone_device_register(const char *type,
 	mutex_lock(&thermal_governor_lock);
 
 	if (tz->tzp)
-		tz->governor = __find_governor(tz->tzp->governor_name);
+		governor = __find_governor(tz->tzp->governor_name);
 	else
-		tz->governor = def_governor;
+		governor = def_governor;
+
+	result = thermal_set_governor(tz, governor);
+	if (result) {
+		mutex_unlock(&thermal_governor_lock);
+		goto unregister;
+	}
 
 	mutex_unlock(&thermal_governor_lock);
 
@@ -1694,7 +1761,7 @@ void thermal_zone_device_unregister(struct thermal_zone_device *tz)
 		device_remove_file(&tz->device, &dev_attr_mode);
 	device_remove_file(&tz->device, &dev_attr_policy);
 	remove_trip_attrs(tz);
-	tz->governor = NULL;
+	thermal_set_governor(tz, NULL);
 
 	thermal_remove_hwmon_sysfs(tz);
 	release_idr(&thermal_tz_idr, &thermal_idr_lock, tz->id);
diff --git a/include/linux/thermal.h b/include/linux/thermal.h
index 89873f93b40d..358c17990711 100644
--- a/include/linux/thermal.h
+++ b/include/linux/thermal.h
@@ -165,6 +165,7 @@ struct thermal_attr {
  * @ops:	operations this &thermal_zone_device supports
  * @tzp:	thermal zone parameters
  * @governor:	pointer to the governor for this thermal zone
+ * @governor_data:	private pointer for governor data
  * @thermal_instances:	list of &struct thermal_instance of this thermal zone
  * @idr:	&struct idr to generate unique id for this zone's cooling
  *		devices
@@ -191,6 +192,7 @@ struct thermal_zone_device {
 	struct thermal_zone_device_ops *ops;
 	const struct thermal_zone_params *tzp;
 	struct thermal_governor *governor;
+	void *governor_data;
 	struct list_head thermal_instances;
 	struct idr idr;
 	struct mutex lock;
@@ -201,12 +203,19 @@ struct thermal_zone_device {
 /**
  * struct thermal_governor - structure that holds thermal governor information
  * @name:	name of the governor
+ * @bind_to_tz: callback called when binding to a thermal zone.  If it
+ *		returns 0, the governor is bound to the thermal zone,
+ *		otherwise it fails.
+ * @unbind_from_tz:	callback called when a governor is unbound from a
+ *			thermal zone.
  * @throttle:	callback called for every trip point even if temperature is
  *		below the trip point temperature
  * @governor_list:	node in thermal_governor_list (in thermal_core.c)
  */
 struct thermal_governor {
 	char name[THERMAL_NAME_LENGTH];
+	int (*bind_to_tz)(struct thermal_zone_device *tz);
+	void (*unbind_from_tz)(struct thermal_zone_device *tz);
 	int (*throttle)(struct thermal_zone_device *tz, int trip);
 	struct list_head	governor_list;
 };
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v2 2/7] thermal: extend the cooling device API to include power information
  2015-02-26 19:00 [PATCH v2 0/7] The power allocator thermal governor Javi Merino
  2015-02-26 19:00 ` [PATCH v2 1/7] thermal: let governors have private data for each thermal zone Javi Merino
@ 2015-02-26 19:00 ` Javi Merino
  2015-02-26 19:00 ` [PATCH v2 3/7] thermal: cpu_cooling: implement the power cooling device API Javi Merino
                   ` (5 subsequent siblings)
  7 siblings, 0 replies; 22+ messages in thread
From: Javi Merino @ 2015-02-26 19:00 UTC (permalink / raw)
  To: edubezval, rui.zhang
  Cc: linux-pm, linux-kernel, punit.agrawal, broonie, tixy, Javi Merino

Add three optional callbacks to the cooling device interface to allow
them to express power.  In addition to the callbacks, add helpers to
identify cooling devices that implement the power cooling device API.

Cc: Zhang Rui <rui.zhang@intel.com>
Cc: Eduardo Valentin <edubezval@gmail.com>
Signed-off-by: Javi Merino <javi.merino@arm.com>
---
 drivers/thermal/thermal_core.c | 52 ++++++++++++++++++++++++++++++++++++++++++
 include/linux/thermal.h        | 18 +++++++++++++++
 2 files changed, 70 insertions(+)

diff --git a/drivers/thermal/thermal_core.c b/drivers/thermal/thermal_core.c
index 3bbda8979049..82f1cca9c8ce 100644
--- a/drivers/thermal/thermal_core.c
+++ b/drivers/thermal/thermal_core.c
@@ -873,6 +873,58 @@ emul_temp_store(struct device *dev, struct device_attribute *attr,
 static DEVICE_ATTR(emul_temp, S_IWUSR, NULL, emul_temp_store);
 #endif/*CONFIG_THERMAL_EMULATION*/
 
+/**
+ * power_actor_get_max_power() - get the maximum power that a cdev can consume
+ * @cdev:	pointer to &thermal_cooling_device
+ * @tz:		a valid thermal zone device pointer
+ * @max_power:	pointer in which to store the maximum power
+ *
+ * Calculate the maximum power consumption in milliwats that the
+ * cooling device can currently consume and store it in @max_power.
+ *
+ * Return: 0 on success, -EINVAL if @cdev doesn't support the
+ * power_actor API or -E* on other error.
+ */
+int power_actor_get_max_power(struct thermal_cooling_device *cdev,
+			      struct thermal_zone_device *tz, u32 *max_power)
+{
+	if (!cdev_is_power_actor(cdev))
+		return -EINVAL;
+
+	return cdev->ops->state2power(cdev, tz, 0, max_power);
+}
+
+/**
+ * power_actor_set_power() - limit the maximum power that a cooling device can consume
+ * @cdev:	pointer to &thermal_cooling_device
+ * @instance:	thermal instance to update
+ * @power:	the power in milliwatts
+ *
+ * Set the cooling device to consume at most @power milliwatts.
+ *
+ * Return: 0 on success, -EINVAL if the cooling device does not
+ * implement the power actor API or -E* for other failures.
+ */
+int power_actor_set_power(struct thermal_cooling_device *cdev,
+			  struct thermal_instance *instance, u32 power)
+{
+	unsigned long state;
+	int ret;
+
+	if (!cdev_is_power_actor(cdev))
+		return -EINVAL;
+
+	ret = cdev->ops->power2state(cdev, instance->tz, power, &state);
+	if (ret)
+		return ret;
+
+	instance->target = state;
+	cdev->updated = false;
+	thermal_cdev_update(cdev);
+
+	return 0;
+}
+
 static DEVICE_ATTR(type, 0444, type_show, NULL);
 static DEVICE_ATTR(temp, 0444, temp_show, NULL);
 static DEVICE_ATTR(mode, 0644, mode_show, mode_store);
diff --git a/include/linux/thermal.h b/include/linux/thermal.h
index 358c17990711..d69b1806f430 100644
--- a/include/linux/thermal.h
+++ b/include/linux/thermal.h
@@ -63,6 +63,7 @@
 
 struct thermal_zone_device;
 struct thermal_cooling_device;
+struct thermal_instance;
 
 enum thermal_device_mode {
 	THERMAL_DEVICE_DISABLED = 0,
@@ -116,6 +117,12 @@ struct thermal_cooling_device_ops {
 	int (*get_max_state) (struct thermal_cooling_device *, unsigned long *);
 	int (*get_cur_state) (struct thermal_cooling_device *, unsigned long *);
 	int (*set_cur_state) (struct thermal_cooling_device *, unsigned long);
+	int (*get_requested_power)(struct thermal_cooling_device *,
+				   struct thermal_zone_device *, u32 *);
+	int (*state2power)(struct thermal_cooling_device *,
+			   struct thermal_zone_device *, unsigned long, u32 *);
+	int (*power2state)(struct thermal_cooling_device *,
+			   struct thermal_zone_device *, u32, unsigned long *);
 };
 
 struct thermal_cooling_device {
@@ -329,6 +336,17 @@ void thermal_zone_of_sensor_unregister(struct device *dev,
 }
 
 #endif
+
+static inline bool cdev_is_power_actor(struct thermal_cooling_device *cdev)
+{
+	return cdev->ops->get_requested_power && cdev->ops->state2power &&
+		cdev->ops->power2state;
+}
+
+int power_actor_get_max_power(struct thermal_cooling_device *,
+			      struct thermal_zone_device *tz, u32 *max_power);
+int power_actor_set_power(struct thermal_cooling_device *,
+			  struct thermal_instance *, u32);
 struct thermal_zone_device *thermal_zone_device_register(const char *, int, int,
 		void *, struct thermal_zone_device_ops *,
 		const struct thermal_zone_params *, int, int);
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v2 3/7] thermal: cpu_cooling: implement the power cooling device API
  2015-02-26 19:00 [PATCH v2 0/7] The power allocator thermal governor Javi Merino
  2015-02-26 19:00 ` [PATCH v2 1/7] thermal: let governors have private data for each thermal zone Javi Merino
  2015-02-26 19:00 ` [PATCH v2 2/7] thermal: extend the cooling device API to include power information Javi Merino
@ 2015-02-26 19:00 ` Javi Merino
  2015-02-26 19:00 ` [PATCH v2 4/7] thermal: introduce the Power Allocator governor Javi Merino
                   ` (4 subsequent siblings)
  7 siblings, 0 replies; 22+ messages in thread
From: Javi Merino @ 2015-02-26 19:00 UTC (permalink / raw)
  To: edubezval, rui.zhang
  Cc: linux-pm, linux-kernel, punit.agrawal, broonie, tixy,
	Javi Merino, Kapileshwar Singh

Add a basic power model to the cpu cooling device to implement the
power cooling device API.  The power model uses the current frequency,
current load and OPPs for the power calculations.  The cpus must have
registered their OPPs using the OPP library.

Cc: Zhang Rui <rui.zhang@intel.com>
Cc: Eduardo Valentin <edubezval@gmail.com>
Signed-off-by: Kapileshwar Singh <kapileshwar.singh@arm.com>
Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
Signed-off-by: Javi Merino <javi.merino@arm.com>
---
 Documentation/thermal/cpu-cooling-api.txt | 156 +++++++-
 drivers/thermal/cpu_cooling.c             | 583 +++++++++++++++++++++++++++++-
 include/linux/cpu_cooling.h               |  39 ++
 3 files changed, 760 insertions(+), 18 deletions(-)

diff --git a/Documentation/thermal/cpu-cooling-api.txt b/Documentation/thermal/cpu-cooling-api.txt
index 753e47cc2e20..71653584cd03 100644
--- a/Documentation/thermal/cpu-cooling-api.txt
+++ b/Documentation/thermal/cpu-cooling-api.txt
@@ -36,8 +36,162 @@ the user. The registration APIs returns the cooling device pointer.
     np: pointer to the cooling device device tree node
     clip_cpus: cpumask of cpus where the frequency constraints will happen.
 
-1.1.3 void cpufreq_cooling_unregister(struct thermal_cooling_device *cdev)
+1.1.3 struct thermal_cooling_device *cpufreq_power_cooling_register(
+    const struct cpumask *clip_cpus, u32 capacitance,
+    get_static_t plat_static_func)
+
+Similar to cpufreq_cooling_register, this function registers a cpufreq
+cooling device.  Using this function, the cooling device will
+implement the power extensions by using a simple cpu power model.  The
+cpus must have registered their OPPs using the OPP library.
+
+The additional parameters are needed for the power model (See 2. Power
+models).  "capacitance" is the dynamic power coefficient (See 2.1
+Dynamic power).  "plat_static_func" is a function to calculate the
+static power consumed by these cpus (See 2.2 Static power).
+
+1.1.4 struct thermal_cooling_device *of_cpufreq_power_cooling_register(
+    struct device_node *np, const struct cpumask *clip_cpus, u32 capacitance,
+    get_static_t plat_static_func)
+
+Similar to cpufreq_power_cooling_register, this function register a
+cpufreq cooling device with power extensions using the device tree
+information supplied by the np parameter.
+
+1.1.5 void cpufreq_cooling_unregister(struct thermal_cooling_device *cdev)
 
     This interface function unregisters the "thermal-cpufreq-%x" cooling device.
 
     cdev: Cooling device pointer which has to be unregistered.
+
+2. Power models
+
+The power API registration functions provide a simple power model for
+CPUs.  The current power is calculated as dynamic + (optionally)
+static power.  This power model requires that the operating-points of
+the CPUs are registered using the kernel's opp library and the
+`cpufreq_frequency_table` is assigned to the `struct device` of the
+cpu.  If you are using CONFIG_CPUFREQ_DT then the
+`cpufreq_frequency_table` should already be assigned to the cpu
+device.
+
+The `plat_static_func` parameter of `cpufreq_power_cooling_register()`
+and `of_cpufreq_power_cooling_register()` is optional.  If you don't
+provide it, only dynamic power will be considered.
+
+2.1 Dynamic power
+
+The dynamic power consumption of a processor depends on many factors.
+For a given processor implementation the primary factors are:
+
+- The time the processor spends running, consuming dynamic power, as
+  compared to the time in idle states where dynamic consumption is
+  negligible.  Herein we refer to this as 'utilisation'.
+- The voltage and frequency levels as a result of DVFS.  The DVFS
+  level is a dominant factor governing power consumption.
+- In running time the 'execution' behaviour (instruction types, memory
+  access patterns and so forth) causes, in most cases, a second order
+  variation.  In pathological cases this variation can be significant,
+  but typically it is of a much lesser impact than the factors above.
+
+A high level dynamic power consumption model may then be represented as:
+
+Pdyn = f(run) * Voltage^2 * Frequency * Utilisation
+
+f(run) here represents the described execution behaviour and its
+result has a units of Watts/Hz/Volt^2 (this often expressed in
+mW/MHz/uVolt^2)
+
+The detailed behaviour for f(run) could be modelled on-line.  However,
+in practice, such an on-line model has dependencies on a number of
+implementation specific processor support and characterisation
+factors.  Therefore, in initial implementation that contribution is
+represented as a constant coefficient.  This is a simplification
+consistent with the relative contribution to overall power variation.
+
+In this simplified representation our model becomes:
+
+Pdyn = Capacitance * Voltage^2 * Frequency * Utilisation
+
+Where `capacitance` is a constant that represents an indicative
+running time dynamic power coefficient in fundamental units of
+mW/MHz/uVolt^2.  Typical values for mobile CPUs might lie in range
+from 100 to 500.  For reference, the approximate values for the SoC in
+ARM's Juno Development Platform are 530 for the Cortex-A57 cluster and
+140 for the Cortex-A53 cluster.
+
+
+2.2 Static power
+
+Static leakage power consumption depends on a number of factors.  For a
+given circuit implementation the primary factors are:
+
+- Time the circuit spends in each 'power state'
+- Temperature
+- Operating voltage
+- Process grade
+
+The time the circuit spends in each 'power state' for a given
+evaluation period at first order means OFF or ON.  However,
+'retention' states can also be supported that reduce power during
+inactive periods without loss of context.
+
+Note: The visibility of state entries to the OS can vary, according to
+platform specifics, and this can then impact the accuracy of a model
+based on OS state information alone.  It might be possible in some
+cases to extract more accurate information from system resources.
+
+The temperature, operating voltage and process 'grade' (slow to fast)
+of the circuit are all significant factors in static leakage power
+consumption.  All of these have complex relationships to static power.
+
+Circuit implementation specific factors include the chosen silicon
+process as well as the type, number and size of transistors in both
+the logic gates and any RAM elements included.
+
+The static power consumption modelling must take into account the
+power managed regions that are implemented.  Taking the example of an
+ARM processor cluster, the modelling would take into account whether
+each CPU can be powered OFF separately or if only a single power
+region is implemented for the complete cluster.
+
+In one view, there are others, a static power consumption model can
+then start from a set of reference values for each power managed
+region (e.g. CPU, Cluster/L2) in each state (e.g. ON, OFF) at an
+arbitrary process grade, voltage and temperature point.  These values
+are then scaled for all of the following: the time in each state, the
+process grade, the current temperature and the operating voltage.
+However, since both implementation specific and complex relationships
+dominate the estimate, the appropriate interface to the model from the
+cpu cooling device is to provide a function callback that calculates
+the static power in this platform.  When registering the cpu cooling
+device pass a function pointer that follows the `get_static_t`
+prototype:
+
+    int plat_get_static(cpumask_t *cpumask, int interval,
+                        unsigned long voltage, u32 &power);
+
+`cpumask` is the cpumask of the cpus involved in the calculation.
+`voltage` is the voltage at which they are operating.  The function
+should calculate the average static power for the last `interval`
+milliseconds.  It returns 0 on success, -E* on error.  If it
+succeeds, it should store the static power in `power`.  Reading the
+temperature of the cpus described by `cpumask` is left for
+plat_get_static() to do as the platform knows best which thermal
+sensor is closest to the cpu.
+
+If `plat_static_func` is NULL, static power is considered to be
+negligible for this platform and only dynamic power is considered.
+
+The platform specific callback can then use any combination of tables
+and/or equations to permute the estimated value.  Process grade
+information is not passed to the model since access to such data, from
+on-chip measurement capability or manufacture time data, is platform
+specific.
+
+Note: the significance of static power for CPUs in comparison to
+dynamic power is highly dependent on implementation.  Given the
+potential complexity in implementation, the importance and accuracy of
+its inclusion when using cpu cooling devices should be assessed on a
+case by case basis.
+
diff --git a/drivers/thermal/cpu_cooling.c b/drivers/thermal/cpu_cooling.c
index f65f0d109fc8..ba23150c7bde 100644
--- a/drivers/thermal/cpu_cooling.c
+++ b/drivers/thermal/cpu_cooling.c
@@ -26,6 +26,7 @@
 #include <linux/thermal.h>
 #include <linux/cpufreq.h>
 #include <linux/err.h>
+#include <linux/pm_opp.h>
 #include <linux/slab.h>
 #include <linux/cpu.h>
 #include <linux/cpu_cooling.h>
@@ -45,6 +46,19 @@
  */
 
 /**
+ * struct power_table - frequency to power conversion
+ * @frequency:	frequency in KHz
+ * @power:	power in mW
+ *
+ * This structure is built when the cooling device registers and helps
+ * in translating frequency to power and viceversa.
+ */
+struct power_table {
+	u32 frequency;
+	u32 power;
+};
+
+/**
  * struct cpufreq_cooling_device - data for cooling device with cpufreq
  * @id: unique integer value corresponding to each cpufreq_cooling_device
  *	registered.
@@ -58,6 +72,15 @@
  *	cpufreq frequencies.
  * @allowed_cpus: all the cpus involved for this cpufreq_cooling_device.
  * @node: list_head to link all cpufreq_cooling_device together.
+ * @last_load: load measured by the latest call to cpufreq_get_actual_power()
+ * @time_in_idle: previous reading of the absolute time that this cpu was idle
+ * @time_in_idle_timestamp: wall time of the last invocation of
+ *	get_cpu_idle_time_us()
+ * @dyn_power_table: array of struct power_table for frequency to power
+ *	conversion, sorted in ascending order.
+ * @dyn_power_table_entries: number of entries in the @dyn_power_table array
+ * @cpu_dev: the first cpu_device from @allowed_cpus that has OPPs registered
+ * @plat_get_static_power: callback to calculate the static power
  *
  * This structure is required for keeping information of each registered
  * cpufreq_cooling_device.
@@ -71,6 +94,13 @@ struct cpufreq_cooling_device {
 	unsigned int *freq_table;	/* In descending order */
 	struct cpumask allowed_cpus;
 	struct list_head node;
+	u32 last_load;
+	u64 *time_in_idle;
+	u64 *time_in_idle_timestamp;
+	struct power_table *dyn_power_table;
+	int dyn_power_table_entries;
+	struct device *cpu_dev;
+	get_static_t plat_get_static_power;
 };
 static DEFINE_IDR(cpufreq_idr);
 static DEFINE_MUTEX(cooling_cpufreq_lock);
@@ -167,6 +197,39 @@ unsigned long cpufreq_cooling_get_level(unsigned int cpu, unsigned int freq)
 }
 EXPORT_SYMBOL_GPL(cpufreq_cooling_get_level);
 
+static void update_cpu_device(int cpu)
+{
+	struct cpufreq_cooling_device *cpufreq_dev;
+
+	mutex_lock(&cooling_cpufreq_lock);
+	list_for_each_entry(cpufreq_dev, &cpufreq_dev_list, node) {
+		if (cpumask_test_cpu(cpu, &cpufreq_dev->allowed_cpus)) {
+			cpufreq_dev->cpu_dev = get_cpu_device(cpu);
+			if (!cpufreq_dev->cpu_dev) {
+				dev_warn(&cpufreq_dev->cool_dev->device,
+					"No cpu device for new policy cpu %d\n",
+					 cpu);
+			}
+			break;
+		}
+	}
+	mutex_unlock(&cooling_cpufreq_lock);
+}
+
+static void remove_cpu_device(int cpu)
+{
+	struct cpufreq_cooling_device *cpufreq_dev;
+
+	mutex_lock(&cooling_cpufreq_lock);
+	list_for_each_entry(cpufreq_dev, &cpufreq_dev_list, node) {
+		if (cpumask_test_cpu(cpu, &cpufreq_dev->allowed_cpus)) {
+			cpufreq_dev->cpu_dev = NULL;
+			break;
+		}
+	}
+	mutex_unlock(&cooling_cpufreq_lock);
+}
+
 /**
  * cpufreq_thermal_notifier - notifier callback for cpufreq policy change.
  * @nb:	struct notifier_block * with callback info.
@@ -186,23 +249,240 @@ static int cpufreq_thermal_notifier(struct notifier_block *nb,
 	unsigned long max_freq = 0;
 	struct cpufreq_cooling_device *cpufreq_dev;
 
-	if (event != CPUFREQ_ADJUST)
-		return 0;
+	switch (event) {
 
-	mutex_lock(&cooling_cpufreq_lock);
-	list_for_each_entry(cpufreq_dev, &cpufreq_dev_list, node) {
-		if (!cpumask_test_cpu(policy->cpu,
-					&cpufreq_dev->allowed_cpus))
+	case CPUFREQ_ADJUST:
+		mutex_lock(&cooling_cpufreq_lock);
+		list_for_each_entry(cpufreq_dev, &cpufreq_dev_list, node) {
+			if (!cpumask_test_cpu(policy->cpu,
+					      &cpufreq_dev->allowed_cpus))
+				continue;
+
+			max_freq = cpufreq_dev->cpufreq_val;
+
+			if (policy->max != max_freq)
+				cpufreq_verify_within_limits(policy, 0,
+							     max_freq);
+		}
+		mutex_unlock(&cooling_cpufreq_lock);
+		break;
+
+	case CPUFREQ_CREATE_POLICY:
+		update_cpu_device(policy->cpu);
+		break;
+	case CPUFREQ_REMOVE_POLICY:
+		remove_cpu_device(policy->cpu);
+		break;
+	default:
+		return NOTIFY_DONE;
+	}
+
+	return NOTIFY_OK;
+}
+
+/**
+ * build_dyn_power_table() - create a dynamic power to frequency table
+ * @cpufreq_device:	the cpufreq cooling device in which to store the table
+ * @capacitance: dynamic power coefficient for these cpus
+ *
+ * Build a dynamic power to frequency table for this cpu and store it
+ * in @cpufreq_device.  This table will be used in cpu_power_to_freq() and
+ * cpu_freq_to_power() to convert between power and frequency
+ * efficiently.  Power is stored in mW, frequency in KHz.  The
+ * resulting table is in ascending order.
+ *
+ * Return: 0 on success, -E* on error.
+ */
+static int build_dyn_power_table(struct cpufreq_cooling_device *cpufreq_device,
+				 u32 capacitance)
+{
+	struct power_table *power_table;
+	struct dev_pm_opp *opp;
+	struct device *dev = NULL;
+	int num_opps = 0, cpu, i, ret = 0;
+	unsigned long freq;
+
+	rcu_read_lock();
+
+	for_each_cpu(cpu, &cpufreq_device->allowed_cpus) {
+		dev = get_cpu_device(cpu);
+		if (!dev) {
+			dev_warn(&cpufreq_device->cool_dev->device,
+				 "No cpu device for cpu %d\n", cpu);
 			continue;
+		}
 
-		max_freq = cpufreq_dev->cpufreq_val;
+		num_opps = dev_pm_opp_get_opp_count(dev);
+		if (num_opps > 0) {
+			break;
+		} else if (num_opps < 0) {
+			ret = num_opps;
+			goto unlock;
+		}
+	}
 
-		if (policy->max != max_freq)
-			cpufreq_verify_within_limits(policy, 0, max_freq);
+	if (num_opps == 0) {
+		ret = -EINVAL;
+		goto unlock;
 	}
-	mutex_unlock(&cooling_cpufreq_lock);
 
-	return 0;
+	power_table = kcalloc(num_opps, sizeof(*power_table), GFP_KERNEL);
+
+	for (freq = 0, i = 0;
+	     opp = dev_pm_opp_find_freq_ceil(dev, &freq), !IS_ERR(opp);
+	     freq++, i++) {
+		u32 freq_mhz, voltage_mv;
+		u64 power;
+
+		freq_mhz = freq / 1000000;
+		voltage_mv = dev_pm_opp_get_voltage(opp) / 1000;
+
+		/*
+		 * Do the multiplication with MHz and millivolt so as
+		 * to not overflow.
+		 */
+		power = (u64)capacitance * freq_mhz * voltage_mv * voltage_mv;
+		do_div(power, 1000000000);
+
+		/* frequency is stored in power_table in KHz */
+		power_table[i].frequency = freq / 1000;
+
+		/* power is stored in mW */
+		power_table[i].power = power;
+	}
+
+	if (i == 0) {
+		ret = PTR_ERR(opp);
+		goto unlock;
+	}
+
+	cpufreq_device->cpu_dev = dev;
+	cpufreq_device->dyn_power_table = power_table;
+	cpufreq_device->dyn_power_table_entries = i;
+
+unlock:
+	rcu_read_unlock();
+	return ret;
+}
+
+static u32 cpu_freq_to_power(struct cpufreq_cooling_device *cpufreq_device,
+			     u32 freq)
+{
+	int i;
+	struct power_table *pt = cpufreq_device->dyn_power_table;
+
+	for (i = 1; i < cpufreq_device->dyn_power_table_entries; i++)
+		if (freq < pt[i].frequency)
+			break;
+
+	return pt[i - 1].power;
+}
+
+static u32 cpu_power_to_freq(struct cpufreq_cooling_device *cpufreq_device,
+			     u32 power)
+{
+	int i;
+	struct power_table *pt = cpufreq_device->dyn_power_table;
+
+	for (i = 1; i < cpufreq_device->dyn_power_table_entries; i++)
+		if (power < pt[i].power)
+			break;
+
+	return pt[i - 1].frequency;
+}
+
+/**
+ * get_load() - get load for a cpu since last updated
+ * @cpufreq_device:	&struct cpufreq_cooling_device for this cpu
+ * @cpu:	cpu number
+ *
+ * Return: The average load of cpu @cpu in percentage since this
+ * function was last called.
+ */
+static u32 get_load(struct cpufreq_cooling_device *cpufreq_device, int cpu)
+{
+	u32 load;
+	u64 now, now_idle, delta_time, delta_idle;
+
+	now_idle = get_cpu_idle_time(cpu, &now, 0);
+	delta_idle = now_idle - cpufreq_device->time_in_idle[cpu];
+	delta_time = now - cpufreq_device->time_in_idle_timestamp[cpu];
+
+	if (delta_time <= delta_idle)
+		load = 0;
+	else
+		load = div64_u64(100 * (delta_time - delta_idle), delta_time);
+
+	cpufreq_device->time_in_idle[cpu] = now_idle;
+	cpufreq_device->time_in_idle_timestamp[cpu] = now;
+
+	return load;
+}
+
+/**
+ * get_static_power() - calculate the static power consumed by the cpus
+ * @cpufreq_device:	struct &cpufreq_cooling_device for this cpu cdev
+ * @tz:		thermal zone device in which we're operating
+ * @freq:	frequency in KHz
+ * @power:	pointer in which to store the calculated static power
+ *
+ * Calculate the static power consumed by the cpus described by
+ * @cpu_actor running at frequency @freq.  This function relies on a
+ * platform specific function that should have been provided when the
+ * actor was registered.  If it wasn't, the static power is assumed to
+ * be negligible.  The calculated static power is stored in @power.
+ *
+ * Return: 0 on success, -E* on failure.
+ */
+static int get_static_power(struct cpufreq_cooling_device *cpufreq_device,
+			    struct thermal_zone_device *tz, unsigned long freq,
+			    u32 *power)
+{
+	struct dev_pm_opp *opp;
+	unsigned long voltage;
+	struct cpumask *cpumask = &cpufreq_device->allowed_cpus;
+	unsigned long freq_hz = freq * 1000;
+
+	if (!cpufreq_device->plat_get_static_power ||
+	    !cpufreq_device->cpu_dev) {
+		*power = 0;
+		return 0;
+	}
+
+	rcu_read_lock();
+
+	opp = dev_pm_opp_find_freq_exact(cpufreq_device->cpu_dev, freq_hz,
+					 true);
+	voltage = dev_pm_opp_get_voltage(opp);
+
+	rcu_read_unlock();
+
+	if (voltage == 0) {
+		dev_warn_ratelimited(cpufreq_device->cpu_dev,
+				     "Failed to get voltage for frequency %lu: %ld\n",
+				     freq_hz, IS_ERR(opp) ? PTR_ERR(opp) : 0);
+		return -EINVAL;
+	}
+
+	return cpufreq_device->plat_get_static_power(cpumask, tz->passive_delay,
+						     voltage, power);
+}
+
+/**
+ * get_dynamic_power() - calculate the dynamic power
+ * @cpufreq_device:	&cpufreq_cooling_device for this cdev
+ * @freq:	current frequency
+ *
+ * Return: the dynamic power consumed by the cpus described by
+ * @cpufreq_device.
+ */
+static u32 get_dynamic_power(struct cpufreq_cooling_device *cpufreq_device,
+			     unsigned long freq)
+{
+	u32 raw_cpu_power;
+
+	raw_cpu_power = cpu_freq_to_power(cpufreq_device, freq);
+	return (raw_cpu_power * cpufreq_device->last_load) / 100;
 }
 
 /* cpufreq cooling device callback functions are defined below */
@@ -280,8 +560,169 @@ static int cpufreq_set_cur_state(struct thermal_cooling_device *cdev,
 	return 0;
 }
 
+/**
+ * cpufreq_get_requested_power() - get the current power
+ * @cdev:	&thermal_cooling_device pointer
+ * @tz:		a valid thermal zone device pointer
+ * @power:	pointer in which to store the resulting power
+ *
+ * Calculate the current power consumption of the cpus in milliwatts
+ * and store it in @power.  This function should actually calculate
+ * the requested power, but it's hard to get the frequency that
+ * cpufreq would have assigned if there were no thermal limits.
+ * Instead, we calculate the current power on the assumption that the
+ * immediate future will look like the immediate past.
+ *
+ * We use the current frequency and the average load since this
+ * function was last called.  In reality, there could have been
+ * multiple opps since this function was last called and that affects
+ * the load calculation.  While it's not perfectly accurate, this
+ * simplification is good enough and works.  REVISIT this, as more
+ * complex code may be needed if experiments show that it's not
+ * accurate enough.
+ *
+ * Return: 0 on success, -E* if getting the static power failed.
+ */
+static int cpufreq_get_requested_power(struct thermal_cooling_device *cdev,
+				       struct thermal_zone_device *tz,
+				       u32 *power)
+{
+	unsigned long freq;
+	int cpu, ret;
+	u32 static_power, dynamic_power, total_load = 0;
+	struct cpufreq_cooling_device *cpufreq_device = cdev->devdata;
+
+	freq = cpufreq_quick_get(cpumask_any(&cpufreq_device->allowed_cpus));
+
+	for_each_cpu(cpu, &cpufreq_device->allowed_cpus) {
+		u32 load;
+
+		if (cpu_online(cpu))
+			load = get_load(cpufreq_device, cpu);
+		else
+			load = 0;
+
+		total_load += load;
+	}
+
+	cpufreq_device->last_load = total_load;
+
+	dynamic_power = get_dynamic_power(cpufreq_device, freq);
+	ret = get_static_power(cpufreq_device, tz, freq, &static_power);
+	if (ret)
+		return ret;
+
+	*power = static_power + dynamic_power;
+	return 0;
+}
+
+/**
+ * cpufreq_state2power() - convert a cpu cdev state to power consumed
+ * @cdev:	&thermal_cooling_device pointer
+ * @tz:		a valid thermal zone device pointer
+ * @state:	cooling device state to be converted
+ * @power:	pointer in which to store the resulting power
+ *
+ * Convert cooling device state @state into power consumption in
+ * milliwatts assuming 100% load.  Store the calculated power in
+ * @power.
+ *
+ * Return: 0 on success, -EINVAL if the cooling device state could not
+ * be converted into a frequency or other -E* if there was an error
+ * when calculating the static power.
+ */
+static int cpufreq_state2power(struct thermal_cooling_device *cdev,
+			       struct thermal_zone_device *tz,
+			       unsigned long state, u32 *power)
+{
+	unsigned int freq, num_cpus;
+	cpumask_t cpumask;
+	u32 static_power, dynamic_power;
+	int ret;
+	struct cpufreq_cooling_device *cpufreq_device = cdev->devdata;
+
+	cpumask_and(&cpumask, &cpufreq_device->allowed_cpus, cpu_online_mask);
+	num_cpus = cpumask_weight(&cpumask);
+
+	/* None of our cpus are online, so no power */
+	if (num_cpus == 0) {
+		*power = 0;
+		return 0;
+	}
+
+	freq = cpufreq_device->freq_table[state];
+	if (!freq)
+		return -EINVAL;
+
+	dynamic_power = cpu_freq_to_power(cpufreq_device, freq) * num_cpus;
+	ret = get_static_power(cpufreq_device, tz, freq, &static_power);
+	if (ret)
+		return ret;
+
+	*power = static_power + dynamic_power;
+	return 0;
+}
+
+/**
+ * cpufreq_power2state() - convert power to a cooling device state
+ * @cdev:	&thermal_cooling_device pointer
+ * @tz:		a valid thermal zone device pointer
+ * @power:	power in milliwatts to be converted
+ * @state:	pointer in which to store the resulting state
+ *
+ * Calculate a cooling device state for the cpus described by @cdev
+ * that would allow them to consume at most @power mW and store it in
+ * @state.  Note that this calculation depends on external factors
+ * such as the cpu load or the current static power.  Calling this
+ * function with the same power as input can yield different cooling
+ * device states depending on those external factors.
+ *
+ * Return: 0 on success, -ENODEV if no cpus are online or -EINVAL if
+ * the calculated frequency could not be converted to a valid state.
+ * The latter should not happen unless the frequencies available to
+ * cpufreq have changed since the initialization of the cpu cooling
+ * device.
+ */
+static int cpufreq_power2state(struct thermal_cooling_device *cdev,
+			       struct thermal_zone_device *tz, u32 power,
+			       unsigned long *state)
+{
+	unsigned int cpu, cur_freq, target_freq;
+	int ret;
+	s32 dyn_power;
+	u32 last_load, normalised_power, static_power;
+	struct cpufreq_cooling_device *cpufreq_device = cdev->devdata;
+
+	cpu = cpumask_any_and(&cpufreq_device->allowed_cpus, cpu_online_mask);
+
+	/* None of our cpus are online */
+	if (cpu >= nr_cpu_ids)
+		return -ENODEV;
+
+	cur_freq = cpufreq_quick_get(cpu);
+	ret = get_static_power(cpufreq_device, tz, cur_freq, &static_power);
+	if (ret)
+		return ret;
+
+	dyn_power = power - static_power;
+	dyn_power = dyn_power > 0 ? dyn_power : 0;
+	last_load = cpufreq_device->last_load ?: 1;
+	normalised_power = (dyn_power * 100) / last_load;
+	target_freq = cpu_power_to_freq(cpufreq_device, normalised_power);
+
+	*state = cpufreq_cooling_get_level(cpu, target_freq);
+	if (*state == THERMAL_CSTATE_INVALID) {
+		dev_warn_ratelimited(&cdev->device,
+				     "Failed to convert %dKHz for cpu %d into a cdev state\n",
+				     target_freq, cpu);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 /* Bind cpufreq callbacks to thermal cooling device ops */
-static struct thermal_cooling_device_ops const cpufreq_cooling_ops = {
+static struct thermal_cooling_device_ops cpufreq_cooling_ops = {
 	.get_max_state = cpufreq_get_max_state,
 	.get_cur_state = cpufreq_get_cur_state,
 	.set_cur_state = cpufreq_set_cur_state,
@@ -311,6 +752,9 @@ static unsigned int find_next_max(struct cpufreq_frequency_table *table,
  * @np: a valid struct device_node to the cooling device device tree node
  * @clip_cpus: cpumask of cpus where the frequency constraints will happen.
  * Normally this should be same as cpufreq policy->related_cpus.
+ * @capacitance: dynamic power coefficient for these cpus
+ * @plat_static_func: function to calculate the static power consumed by these
+ *                    cpus (optional)
  *
  * This interface function registers the cpufreq cooling device with the name
  * "thermal-cpufreq-%x". This api can support multiple instances of cpufreq
@@ -322,13 +766,14 @@ static unsigned int find_next_max(struct cpufreq_frequency_table *table,
  */
 static struct thermal_cooling_device *
 __cpufreq_cooling_register(struct device_node *np,
-			   const struct cpumask *clip_cpus)
+			const struct cpumask *clip_cpus, u32 capacitance,
+			get_static_t plat_static_func)
 {
 	struct thermal_cooling_device *cool_dev;
 	struct cpufreq_cooling_device *cpufreq_dev;
 	char dev_name[THERMAL_NAME_LENGTH];
 	struct cpufreq_frequency_table *pos, *table;
-	unsigned int freq, i;
+	unsigned int freq, i, num_cpus;
 	int ret;
 
 	table = cpufreq_frequency_get_table(cpumask_first(clip_cpus));
@@ -341,6 +786,23 @@ __cpufreq_cooling_register(struct device_node *np,
 	if (!cpufreq_dev)
 		return ERR_PTR(-ENOMEM);
 
+	num_cpus = cpumask_weight(clip_cpus);
+	cpufreq_dev->time_in_idle = kcalloc(num_cpus,
+					    sizeof(*cpufreq_dev->time_in_idle),
+					    GFP_KERNEL);
+	if (!cpufreq_dev->time_in_idle) {
+		cool_dev = ERR_PTR(-ENOMEM);
+		goto free_cdev;
+	}
+
+	cpufreq_dev->time_in_idle_timestamp =
+		kcalloc(num_cpus, sizeof(*cpufreq_dev->time_in_idle_timestamp),
+			GFP_KERNEL);
+	if (!cpufreq_dev->time_in_idle_timestamp) {
+		cool_dev = ERR_PTR(-ENOMEM);
+		goto free_time_in_idle;
+	}
+
 	/* Find max levels */
 	cpufreq_for_each_valid_entry(pos, table)
 		cpufreq_dev->max_level++;
@@ -349,7 +811,7 @@ __cpufreq_cooling_register(struct device_node *np,
 					  cpufreq_dev->max_level, GFP_KERNEL);
 	if (!cpufreq_dev->freq_table) {
 		cool_dev = ERR_PTR(-ENOMEM);
-		goto free_cdev;
+		goto free_time_in_idle_timestamp;
 	}
 
 	/* max_level is an index, not a counter */
@@ -357,6 +819,20 @@ __cpufreq_cooling_register(struct device_node *np,
 
 	cpumask_copy(&cpufreq_dev->allowed_cpus, clip_cpus);
 
+	if (capacitance) {
+		cpufreq_cooling_ops.get_requested_power =
+			cpufreq_get_requested_power;
+		cpufreq_cooling_ops.state2power = cpufreq_state2power;
+		cpufreq_cooling_ops.power2state = cpufreq_power2state;
+		cpufreq_dev->plat_get_static_power = plat_static_func;
+
+		ret = build_dyn_power_table(cpufreq_dev, capacitance);
+		if (ret) {
+			cool_dev = ERR_PTR(ret);
+			goto free_table;
+		}
+	}
+
 	ret = get_idr(&cpufreq_idr, &cpufreq_dev->id);
 	if (ret) {
 		cool_dev = ERR_PTR(ret);
@@ -402,6 +878,10 @@ remove_idr:
 	release_idr(&cpufreq_idr, cpufreq_dev->id);
 free_table:
 	kfree(cpufreq_dev->freq_table);
+free_time_in_idle_timestamp:
+	kfree(cpufreq_dev->time_in_idle_timestamp);
+free_time_in_idle:
+	kfree(cpufreq_dev->time_in_idle);
 free_cdev:
 	kfree(cpufreq_dev);
 
@@ -422,7 +902,7 @@ free_cdev:
 struct thermal_cooling_device *
 cpufreq_cooling_register(const struct cpumask *clip_cpus)
 {
-	return __cpufreq_cooling_register(NULL, clip_cpus);
+	return __cpufreq_cooling_register(NULL, clip_cpus, 0, NULL);
 }
 EXPORT_SYMBOL_GPL(cpufreq_cooling_register);
 
@@ -446,11 +926,78 @@ of_cpufreq_cooling_register(struct device_node *np,
 	if (!np)
 		return ERR_PTR(-EINVAL);
 
-	return __cpufreq_cooling_register(np, clip_cpus);
+	return __cpufreq_cooling_register(np, clip_cpus, 0, NULL);
 }
 EXPORT_SYMBOL_GPL(of_cpufreq_cooling_register);
 
 /**
+ * cpufreq_power_cooling_register() - create cpufreq cooling device with power extensions
+ * @clip_cpus:	cpumask of cpus where the frequency constraints will happen
+ * @capacitance:	dynamic power coefficient for these cpus
+ * @plat_static_func:	function to calculate the static power consumed by these
+ *			cpus (optional)
+ *
+ * This interface function registers the cpufreq cooling device with
+ * the name "thermal-cpufreq-%x".  This api can support multiple
+ * instances of cpufreq cooling devices.  Using this function, the
+ * cooling device will implement the power extensions by using a
+ * simple cpu power model.  The cpus must have registered their OPPs
+ * using the OPP library.
+ *
+ * An optional @plat_static_func may be provided to calculate the
+ * static power consumed by these cpus.  If the platform's static
+ * power consumption is unknown or negligible, make it NULL.
+ *
+ * Return: a valid struct thermal_cooling_device pointer on success,
+ * on failure, it returns a corresponding ERR_PTR().
+ */
+struct thermal_cooling_device *
+cpufreq_power_cooling_register(const struct cpumask *clip_cpus, u32 capacitance,
+			       get_static_t plat_static_func)
+{
+	return __cpufreq_cooling_register(NULL, clip_cpus, capacitance,
+				plat_static_func);
+}
+EXPORT_SYMBOL(cpufreq_power_cooling_register);
+
+/**
+ * of_cpufreq_power_cooling_register() - create cpufreq cooling device with power extensions
+ * @np:	a valid struct device_node to the cooling device device tree node
+ * @clip_cpus:	cpumask of cpus where the frequency constraints will happen
+ * @capacitance:	dynamic power coefficient for these cpus
+ * @plat_static_func:	function to calculate the static power consumed by these
+ *			cpus (optional)
+ *
+ * This interface function registers the cpufreq cooling device with
+ * the name "thermal-cpufreq-%x".  This api can support multiple
+ * instances of cpufreq cooling devices.  Using this API, the cpufreq
+ * cooling device will be linked to the device tree node provided.
+ * Using this function, the cooling device will implement the power
+ * extensions by using a simple cpu power model.  The cpus must have
+ * registered their OPPs using the OPP library.
+ *
+ * An optional @plat_static_func may be provided to calculate the
+ * static power consumed by these cpus.  If the platform's static
+ * power consumption is unknown or negligible, make it NULL.
+ *
+ * Return: a valid struct thermal_cooling_device pointer on success,
+ * on failure, it returns a corresponding ERR_PTR().
+ */
+struct thermal_cooling_device *
+of_cpufreq_power_cooling_register(struct device_node *np,
+				  const struct cpumask *clip_cpus,
+				  u32 capacitance,
+				  get_static_t plat_static_func)
+{
+	if (!np)
+		return ERR_PTR(-EINVAL);
+
+	return __cpufreq_cooling_register(np, clip_cpus, capacitance,
+				plat_static_func);
+}
+EXPORT_SYMBOL(of_cpufreq_power_cooling_register);
+
+/**
  * cpufreq_cooling_unregister - function to remove cpufreq cooling device.
  * @cdev: thermal cooling device pointer.
  *
@@ -475,6 +1022,8 @@ void cpufreq_cooling_unregister(struct thermal_cooling_device *cdev)
 
 	thermal_cooling_device_unregister(cpufreq_dev->cool_dev);
 	release_idr(&cpufreq_idr, cpufreq_dev->id);
+	kfree(cpufreq_dev->time_in_idle_timestamp);
+	kfree(cpufreq_dev->time_in_idle);
 	kfree(cpufreq_dev->freq_table);
 	kfree(cpufreq_dev);
 }
diff --git a/include/linux/cpu_cooling.h b/include/linux/cpu_cooling.h
index bd955270d5aa..c156f5082758 100644
--- a/include/linux/cpu_cooling.h
+++ b/include/linux/cpu_cooling.h
@@ -28,6 +28,9 @@
 #include <linux/thermal.h>
 #include <linux/cpumask.h>
 
+typedef int (*get_static_t)(cpumask_t *cpumask, int interval,
+			    unsigned long voltage, u32 *power);
+
 #ifdef CONFIG_CPU_THERMAL
 /**
  * cpufreq_cooling_register - function to create cpufreq cooling device.
@@ -36,6 +39,10 @@
 struct thermal_cooling_device *
 cpufreq_cooling_register(const struct cpumask *clip_cpus);
 
+struct thermal_cooling_device *
+cpufreq_power_cooling_register(const struct cpumask *clip_cpus,
+			       u32 capacitance, get_static_t plat_static_func);
+
 /**
  * of_cpufreq_cooling_register - create cpufreq cooling device based on DT.
  * @np: a valid struct device_node to the cooling device device tree node.
@@ -45,6 +52,12 @@ cpufreq_cooling_register(const struct cpumask *clip_cpus);
 struct thermal_cooling_device *
 of_cpufreq_cooling_register(struct device_node *np,
 			    const struct cpumask *clip_cpus);
+
+struct thermal_cooling_device *
+of_cpufreq_power_cooling_register(struct device_node *np,
+				  const struct cpumask *clip_cpus,
+				  u32 capacitance,
+				  get_static_t plat_static_func);
 #else
 static inline struct thermal_cooling_device *
 of_cpufreq_cooling_register(struct device_node *np,
@@ -52,6 +65,15 @@ of_cpufreq_cooling_register(struct device_node *np,
 {
 	return ERR_PTR(-ENOSYS);
 }
+
+static inline struct thermal_cooling_device *
+of_cpufreq_power_cooling_register(struct device_node *np,
+				  const struct cpumask *clip_cpus,
+				  u32 capacitance,
+				  get_static_t plat_static_func)
+{
+	return NULL;
+}
 #endif
 
 /**
@@ -68,11 +90,28 @@ cpufreq_cooling_register(const struct cpumask *clip_cpus)
 	return ERR_PTR(-ENOSYS);
 }
 static inline struct thermal_cooling_device *
+cpufreq_power_cooling_register(const struct cpumask *clip_cpus,
+			       u32 capacitance, get_static_t plat_static_func)
+{
+	return NULL;
+}
+
+static inline struct thermal_cooling_device *
 of_cpufreq_cooling_register(struct device_node *np,
 			    const struct cpumask *clip_cpus)
 {
 	return ERR_PTR(-ENOSYS);
 }
+
+static inline struct thermal_cooling_device *
+of_cpufreq_power_cooling_register(struct device_node *np,
+				  const struct cpumask *clip_cpus,
+				  u32 capacitance,
+				  get_static_t plat_static_func)
+{
+	return NULL;
+}
+
 static inline
 void cpufreq_cooling_unregister(struct thermal_cooling_device *cdev)
 {
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v2 4/7] thermal: introduce the Power Allocator governor
  2015-02-26 19:00 [PATCH v2 0/7] The power allocator thermal governor Javi Merino
                   ` (2 preceding siblings ...)
  2015-02-26 19:00 ` [PATCH v2 3/7] thermal: cpu_cooling: implement the power cooling device API Javi Merino
@ 2015-02-26 19:00 ` Javi Merino
  2015-02-26 20:36   ` Eduardo Valentin
  2015-02-26 19:00 ` [PATCH v2 5/7] thermal: add trace events to the power allocator governor Javi Merino
                   ` (3 subsequent siblings)
  7 siblings, 1 reply; 22+ messages in thread
From: Javi Merino @ 2015-02-26 19:00 UTC (permalink / raw)
  To: edubezval, rui.zhang
  Cc: linux-pm, linux-kernel, punit.agrawal, broonie, tixy, Javi Merino

The power allocator governor is a thermal governor that controls system
and device power allocation to control temperature.  Conceptually, the
implementation divides the sustainable power of a thermal zone among
all the heat sources in that zone.

This governor relies on "power actors", entities that represent heat
sources.  They can report current and maximum power consumption and
can set a given maximum power consumption, usually via a cooling
device.

The governor uses a Proportional Integral Derivative (PID) controller
driven by the temperature of the thermal zone.  The output of the
controller is a power budget that is then allocated to each power
actor that can have bearing on the temperature we are trying to
control.  It decides how much power to give each cooling device based
on the performance they are requesting.  The PID controller ensures
that the total power budget does not exceed the control temperature.

Cc: Zhang Rui <rui.zhang@intel.com>
Cc: Eduardo Valentin <edubezval@gmail.com>
Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
Signed-off-by: Javi Merino <javi.merino@arm.com>
---
 Documentation/thermal/power_allocator.txt | 247 ++++++++++++++
 drivers/thermal/Kconfig                   |  15 +
 drivers/thermal/Makefile                  |   1 +
 drivers/thermal/power_allocator.c         | 515 ++++++++++++++++++++++++++++++
 drivers/thermal/thermal_core.c            |   9 +-
 drivers/thermal/thermal_core.h            |   8 +
 include/linux/thermal.h                   |  37 ++-
 7 files changed, 825 insertions(+), 7 deletions(-)
 create mode 100644 Documentation/thermal/power_allocator.txt
 create mode 100644 drivers/thermal/power_allocator.c

diff --git a/Documentation/thermal/power_allocator.txt b/Documentation/thermal/power_allocator.txt
new file mode 100644
index 000000000000..c3797b529991
--- /dev/null
+++ b/Documentation/thermal/power_allocator.txt
@@ -0,0 +1,247 @@
+Power allocator governor tunables
+=================================
+
+Trip points
+-----------
+
+The governor requires the following two passive trip points:
+
+1.  "switch on" trip point: temperature above which the governor
+    control loop starts operating.  This is the first passive trip
+    point of the thermal zone.
+
+2.  "desired temperature" trip point: it should be higher than the
+    "switch on" trip point.  This the target temperature the governor
+    is controlling for.  This is the last passive trip point of the
+    thermal zone.
+
+PID Controller
+--------------
+
+The power allocator governor implements a
+Proportional-Integral-Derivative controller (PID controller) with
+temperature as the control input and power as the controlled output:
+
+    P_max = k_p * e + k_i * err_integral + k_d * diff_err + sustainable_power
+
+where
+    e = desired_temperature - current_temperature
+    err_integral is the sum of previous errors
+    diff_err = e - previous_error
+
+It is similar to the one depicted below:
+
+                                      k_d
+                                       |
+current_temp                           |
+     |                                 v
+     |                +----------+   +---+
+     |         +----->| diff_err |-->| X |------+
+     |         |      +----------+   +---+      |
+     |         |                                |      tdp        actor
+     |         |                      k_i       |       |  get_requested_power()
+     |         |                       |        |       |        |     |
+     |         |                       |        |       |        |     | ...
+     v         |                       v        v       v        v     v
+   +---+       |      +-------+      +---+    +---+   +---+   +----------+
+   | S |-------+----->| sum e |----->| X |--->| S |-->| S |-->|power     |
+   +---+       |      +-------+      +---+    +---+   +---+   |allocation|
+     ^         |                                ^             +----------+
+     |         |                                |                |     |
+     |         |        +---+                   |                |     |
+     |         +------->| X |-------------------+                v     v
+     |                  +---+                               granted performance
+desired_temperature       ^
+                          |
+                          |
+                      k_po/k_pu
+
+Sustainable power
+-----------------
+
+An estimate of the sustainable dissipatable power (in mW) should be
+provided while registering the thermal zone.  This estimates the
+sustained power that can be dissipated at the desired control
+temperature.  This is the maximum sustained power for allocation at
+the desired maximum temperature.  The actual sustained power can vary
+for a number of reasons.  The closed loop controller will take care of
+variations such as environmental conditions, and some factors related
+to the speed-grade of the silicon.  `sustainable_power` is therefore
+simply an estimate, and may be tuned to affect the aggressiveness of
+the thermal ramp. For reference, the sustainable power of a 4" phone
+is typically 2000mW, while on a 10" tablet is around 4500mW (may vary
+depending on screen size).
+
+If you are using device tree, do add it as a property of the
+thermal-zone.  For example:
+
+	thermal-zones {
+		soc_thermal {
+			polling-delay = <1000>;
+			polling-delay-passive = <100>;
+			sustainable-power = <2500>;
+			...
+
+Instead, if the thermal zone is registered from the platform code, pass a
+`thermal_zone_params` that has a `sustainable_power`.  If no
+`thermal_zone_params` were being passed, then something like below
+will suffice:
+
+	static const struct thermal_zone_params tz_params = {
+		.sustainable_power = 3500,
+	};
+
+and then pass `tz_params` as the 5th parameter to
+`thermal_zone_device_register()`
+
+k_po and k_pu
+-------------
+
+The implementation of the PID controller in the power allocator
+thermal governor allows the configuration of two proportional term
+constants: `k_po` and `k_pu`.  `k_po` is the proportional term
+constant during temperature overshoot periods (current temperature is
+above "desired temperature" trip point).  Conversely, `k_pu` is the
+proportional term constant during temperature undershoot periods
+(current temperature below "desired temperature" trip point).
+
+These controls are intended as the primary mechanism for configuring
+the permitted thermal "ramp" of the system.  For instance, a lower
+`k_pu` value will provide a slower ramp, at the cost of capping
+available capacity at a low temperature.  On the other hand, a high
+value of `k_pu` will result in the governor granting very high power
+whilst temperature is low, and may lead to temperature overshooting.
+
+The default value for `k_pu` is:
+
+    2 * sustainable_power / (desired_temperature - switch_on_temp)
+
+This means that at `switch_on_temp` the output of the controller's
+proportional term will be 2 * `sustainable_power`.  The default value
+for `k_po` is:
+
+    sustainable_power / (desired_temperature - switch_on_temp)
+
+Focusing on the proportional and feed forward values of the PID
+controller equation we have:
+
+    P_max = k_p * e + sustainable_power
+
+The proportional term is proportional to the difference between the
+desired temperature and the current one.  When the current temperature
+is the desired one, then the proportional component is zero and
+`P_max` = `sustainable_power`.  That is, the system should operate in
+thermal equilibrium under constant load.  `sustainable_power` is only
+an estimate, which is the reason for closed-loop control such as this.
+
+Expanding `k_pu` we get:
+    P_max = 2 * sustainable_power * (T_set - T) / (T_set - T_on) +
+        sustainable_power
+
+where
+    T_set is the desired temperature
+    T is the current temperature
+    T_on is the switch on temperature
+
+When the current temperature is the switch_on temperature, the above
+formula becomes:
+
+    P_max = 2 * sustainable_power * (T_set - T_on) / (T_set - T_on) +
+        sustainable_power = 2 * sustainable_power + sustainable_power =
+        3 * sustainable_power
+
+Therefore, the proportional term alone linearly decreases power from
+3 * `sustainable_power` to `sustainable_power` as the temperature
+rises from the switch on temperature to the desired temperature.
+
+k_i and integral_cutoff
+-----------------------
+
+`k_i` configures the PID loop's integral term constant.  This term
+allows the PID controller to compensate for long term drift and for
+the quantized nature of the output control: cooling devices can't set
+the exact power that the governor requests.  When the temperature
+error is below `integral_cutoff`, errors are accumulated in the
+integral term.  This term is then multiplied by `k_i` and the result
+added to the output of the controller.  Typically `k_i` is set low (1
+or 2) and `integral_cutoff` is 0.
+
+k_d
+---
+
+`k_d` configures the PID loop's derivative term constant.  It's
+recommended to leave it as the default: 0.
+
+Cooling device power API
+========================
+
+Cooling devices controlled by this governor must supply the additional
+"power" API in their `cooling_device_ops`.  It consists on three ops:
+
+1. int get_requested_power(struct thermal_cooling_device *cdev,
+	struct thermal_zone_device *tz, u32 *power);
+@cdev: The `struct thermal_cooling_device` pointer
+@tz: thermal zone in which we are currently operating
+@power: pointer in which to store the calculated power
+
+`get_requested_power()` calculates the power requested by the device
+in milliwatts and stores it in @power .  It should return 0 on
+success, -E* on failure.  This is currently used by the power
+allocator governor to calculate how much power to give to each cooling
+device.
+
+2. int state2power(struct thermal_cooling_device *cdev, struct
+        thermal_zone_device *tz, unsigned long state, u32 *power);
+@cdev: The `struct thermal_cooling_device` pointer
+@tz: thermal zone in which we are currently operating
+@state: A cooling device state
+@power: pointer in which to store the equivalent power
+
+Convert cooling device state @state into power consumption in
+milliwatts and store it in @power.  It should return 0 on success, -E*
+on failure.  This is currently used by thermal core to calculate the
+maximum power that an actor can consume.
+
+3. int power2state(struct thermal_cooling_device *cdev, u32 power,
+	unsigned long *state);
+@cdev: The `struct thermal_cooling_device` pointer
+@power: power in milliwatts
+@state: pointer in which to store the resulting state
+
+Calculate a cooling device state that would make the device consume at
+most @power mW and store it in @state.  It should return 0 on success,
+-E* on failure.  This is currently used by the thermal core to convert
+a given power set by the power allocator governor to a state that the
+cooling device can set.  It is a function because this conversion may
+depend on external factors that may change so this function should the
+best conversion given "current circumstances".
+
+Cooling device weights
+----------------------
+
+Weights are a mechanism to bias the allocation among cooling
+devices.  They express the relative power efficiency of different
+cooling devices.  Higher weight can be used to express higher power
+efficiency.  Weighting is relative such that if each cooling device
+has a weight of one they are considered equal.  This is particularly
+useful in heterogeneous systems where two cooling devices may perform
+the same kind of compute, but with different efficiency.  For example,
+a system with two different types of processors.
+
+If the thermal zone is registered using
+`thermal_zone_device_register()` (i.e., platform code), then weights
+are passed as part of the thermal zone's `thermal_bind_parameters`.
+If the platform is registered using device tree, then they are passed
+as the `contribution` property of each map in the `cooling-maps` node.
+
+Limitations of the power allocator governor
+===========================================
+
+The power allocator governor's PID controller works best if there is a
+periodic tick.  If you have a driver that calls
+`thermal_zone_device_update()` (or anything that ends up calling the
+governor's `throttle()` function) repetitively, the governor response
+won't be very good.  Note that this is not particular to this
+governor, step-wise will also misbehave if you call its throttle()
+faster than the normal thermal framework tick (due to interrupts for
+example) as it will overreact.
diff --git a/drivers/thermal/Kconfig b/drivers/thermal/Kconfig
index 30aee81e9f5b..a1b43eab0a70 100644
--- a/drivers/thermal/Kconfig
+++ b/drivers/thermal/Kconfig
@@ -71,6 +71,14 @@ config THERMAL_DEFAULT_GOV_USER_SPACE
 	  Select this if you want to let the user space manage the
 	  platform thermals.
 
+config THERMAL_DEFAULT_GOV_POWER_ALLOCATOR
+	bool "power_allocator"
+	select THERMAL_GOV_POWER_ALLOCATOR
+	help
+	  Select this if you want to control temperature based on
+	  system and device power allocation. This governor can only
+	  operate on cooling devices that implement the power API.
+
 endchoice
 
 config THERMAL_GOV_FAIR_SHARE
@@ -99,6 +107,13 @@ config THERMAL_GOV_USER_SPACE
 	help
 	  Enable this to let the user space manage the platform thermals.
 
+config THERMAL_GOV_POWER_ALLOCATOR
+	bool "Power allocator thermal governor"
+	select THERMAL_POWER_ACTOR
+	help
+	  Enable this to manage platform thermals by dynamically
+	  allocating and limiting power to devices.
+
 config CPU_THERMAL
 	bool "generic cpu cooling support"
 	depends on CPU_FREQ
diff --git a/drivers/thermal/Makefile b/drivers/thermal/Makefile
index 1fe86652cfb6..b1783cf37ed2 100644
--- a/drivers/thermal/Makefile
+++ b/drivers/thermal/Makefile
@@ -14,6 +14,7 @@ thermal_sys-$(CONFIG_THERMAL_GOV_FAIR_SHARE)	+= fair_share.o
 thermal_sys-$(CONFIG_THERMAL_GOV_BANG_BANG)	+= gov_bang_bang.o
 thermal_sys-$(CONFIG_THERMAL_GOV_STEP_WISE)	+= step_wise.o
 thermal_sys-$(CONFIG_THERMAL_GOV_USER_SPACE)	+= user_space.o
+thermal_sys-$(CONFIG_THERMAL_GOV_POWER_ALLOCATOR)	+= power_allocator.o
 
 # cpufreq cooling
 thermal_sys-$(CONFIG_CPU_THERMAL)	+= cpu_cooling.o
diff --git a/drivers/thermal/power_allocator.c b/drivers/thermal/power_allocator.c
new file mode 100644
index 000000000000..bfea8d35c0a2
--- /dev/null
+++ b/drivers/thermal/power_allocator.c
@@ -0,0 +1,515 @@
+/*
+ * A power allocator to manage temperature
+ *
+ * Copyright (C) 2014 ARM Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#define pr_fmt(fmt) "Power allocator: " fmt
+
+#include <linux/rculist.h>
+#include <linux/slab.h>
+#include <linux/thermal.h>
+
+#include "thermal_core.h"
+
+#define FRAC_BITS 10
+#define int_to_frac(x) ((x) << FRAC_BITS)
+#define frac_to_int(x) ((x) >> FRAC_BITS)
+
+/**
+ * mul_frac() - multiply two fixed-point numbers
+ * @x:	first multiplicand
+ * @y:	second multiplicand
+ *
+ * Return: the result of multiplying two fixed-point numbers.  The
+ * result is also a fixed-point number.
+ */
+static inline s64 mul_frac(s64 x, s64 y)
+{
+	return (x * y) >> FRAC_BITS;
+}
+
+/**
+ * div_frac() - divide two fixed-point numbers
+ * @x:	the dividend
+ * @y:	the divisor
+ *
+ * Return: the result of dividing two fixed-point numbers.  The
+ * result is also a fixed-point number.
+ */
+static inline s64 div_frac(s64 x, s64 y)
+{
+	return div_s64(x << FRAC_BITS, y);
+}
+
+/**
+ * struct power_allocator_params - parameters for the power allocator governor
+ * @err_integral:	accumulated error in the PID controller.
+ * @prev_err:	error in the previous iteration of the PID controller.
+ *		Used to calculate the derivative term.
+ * @trip_switch_on:	first passive trip point of the thermal zone.  The
+ *			governor switches on when this trip point is crossed.
+ * @trip_max_desired_temperature:	last passive trip point of the thermal
+ *					zone.  The temperature we are
+ *					controlling for.
+ */
+struct power_allocator_params {
+	s64 err_integral;
+	s32 prev_err;
+	int trip_switch_on;
+	int trip_max_desired_temperature;
+};
+
+/**
+ * pid_controller() - PID controller
+ * @tz:	thermal zone we are operating in
+ * @current_temp:	the current temperature in millicelsius
+ * @control_temp:	the target temperature in millicelsius
+ * @max_allocatable_power:	maximum allocatable power for this thermal zone
+ *
+ * This PID controller increases the available power budget so that the
+ * temperature of the thermal zone gets as close as possible to
+ * @control_temp and limits the power if it exceeds it.  k_po is the
+ * proportional term when we are overshooting, k_pu is the
+ * proportional term when we are undershooting.  integral_cutoff is a
+ * threshold below which we stop accumulating the error.  The
+ * accumulated error is only valid if the requested power will make
+ * the system warmer.  If the system is mostly idle, there's no point
+ * in accumulating positive error.
+ *
+ * Return: The power budget for the next period.
+ */
+static u32 pid_controller(struct thermal_zone_device *tz,
+			  unsigned long current_temp,
+			  unsigned long control_temp,
+			  u32 max_allocatable_power)
+{
+	s64 p, i, d, power_range;
+	s32 err, max_power_frac;
+	struct power_allocator_params *params = tz->governor_data;
+
+	max_power_frac = int_to_frac(max_allocatable_power);
+
+	err = ((s32)control_temp - (s32)current_temp);
+	err = int_to_frac(err);
+
+	/* Calculate the proportional term */
+	p = mul_frac(err < 0 ? tz->tzp->k_po : tz->tzp->k_pu, err);
+
+	/*
+	 * Calculate the integral term
+	 *
+	 * if the error is less than cut off allow integration (but
+	 * the integral is limited to max power)
+	 */
+	i = mul_frac(tz->tzp->k_i, params->err_integral);
+
+	if (err < int_to_frac(tz->tzp->integral_cutoff)) {
+		s64 i_next = i + mul_frac(tz->tzp->k_i, err);
+
+		if (abs64(i_next) < max_power_frac) {
+			i = i_next;
+			params->err_integral += err;
+		}
+	}
+
+	/*
+	 * Calculate the derivative term
+	 *
+	 * We do err - prev_err, so with a positive k_d, a decreasing
+	 * error (i.e. driving closer to the line) results in less
+	 * power being applied, slowing down the controller)
+	 */
+	d = mul_frac(tz->tzp->k_d, err - params->prev_err);
+	d = div_frac(d, tz->passive_delay);
+	params->prev_err = err;
+
+	power_range = p + i + d;
+
+	/* feed-forward the known sustainable dissipatable power */
+	power_range = tz->tzp->sustainable_power + frac_to_int(power_range);
+
+	return clamp(power_range, (s64)0, (s64)max_allocatable_power);
+}
+
+/**
+ * divvy_up_power() - divvy the allocated power between the actors
+ * @req_power:	each actor's requested power
+ * @max_power:	each actor's maximum available power
+ * @num_actors:	size of the @req_power, @max_power and @granted_power's array
+ * @total_req_power: sum of @req_power
+ * @power_range:	total allocated power
+ * @granted_power:	output array: each actor's granted power
+ *
+ * This function divides the total allocated power (@power_range)
+ * fairly between the actors.  It first tries to give each actor a
+ * share of the @power_range according to how much power it requested
+ * compared to the rest of the actors.  For example, if only one actor
+ * requests power, then it receives all the @power_range.  If
+ * three actors each requests 1mW, each receives a third of the
+ * @power_range.
+ *
+ * If any actor received more than their maximum power, then that
+ * surplus is re-divvied among the actors based on how far they are
+ * from their respective maximums.
+ *
+ * Granted power for each actor is written to @granted_power, which
+ * should've been allocated by the calling function.
+ */
+static void divvy_up_power(u32 *req_power, u32 *max_power, int num_actors,
+			   u32 total_req_power, u32 power_range,
+			   u32 *granted_power)
+{
+	u32 extra_power, capped_extra_power, extra_actor_power[num_actors];
+	int i;
+
+	/*
+	 * Prevent division by 0 if none of the actors request power.
+	 */
+	if (!total_req_power)
+		total_req_power = 1;
+
+	capped_extra_power = 0;
+	extra_power = 0;
+	for (i = 0; i < num_actors; i++) {
+		u64 req_range = req_power[i] * power_range;
+
+		granted_power[i] = div_u64(req_range, total_req_power);
+
+		if (granted_power[i] > max_power[i]) {
+			extra_power += granted_power[i] - max_power[i];
+			granted_power[i] = max_power[i];
+		}
+
+		extra_actor_power[i] = max_power[i] - granted_power[i];
+		capped_extra_power += extra_actor_power[i];
+	}
+
+	if (!extra_power)
+		return;
+
+	/*
+	 * Re-divvy the reclaimed extra among actors based on
+	 * how far they are from the max
+	 */
+	extra_power = min(extra_power, capped_extra_power);
+	if (capped_extra_power > 0)
+		for (i = 0; i < num_actors; i++)
+			granted_power[i] += (extra_actor_power[i] *
+					extra_power) / capped_extra_power;
+}
+
+static int allocate_power(struct thermal_zone_device *tz,
+			  unsigned long current_temp,
+			  unsigned long control_temp)
+{
+	struct thermal_instance *instance;
+	struct power_allocator_params *params = tz->governor_data;
+	u32 *req_power, *max_power, *granted_power;
+	u32 total_req_power, max_allocatable_power;
+	u32 power_range;
+	int i, num_actors, total_weight, ret = 0;
+	int trip_max_desired_temperature = params->trip_max_desired_temperature;
+
+	mutex_lock(&tz->lock);
+
+	num_actors = 0;
+	total_weight = 0;
+	list_for_each_entry(instance, &tz->thermal_instances, tz_node) {
+		if ((instance->trip == trip_max_desired_temperature) &&
+		    cdev_is_power_actor(instance->cdev)) {
+			num_actors++;
+			total_weight += instance->weight;
+		}
+	}
+
+	/*
+	 * We need to allocate three arrays of the same size:
+	 * req_power, max_power and granted_power.  They are going to
+	 * be needed until this function returns.  Allocate them all
+	 * in one go to simplify the allocation and deallocation
+	 * logic.
+	 */
+	BUILD_BUG_ON(sizeof(*req_power) != sizeof(*max_power));
+	BUILD_BUG_ON(sizeof(*req_power) != sizeof(*granted_power));
+	req_power = devm_kcalloc(&tz->device, num_actors * 3,
+				 sizeof(*req_power), GFP_KERNEL);
+	if (!req_power) {
+		ret = -ENOMEM;
+		goto unlock;
+	}
+
+	max_power = &req_power[num_actors];
+	granted_power = &req_power[2 * num_actors];
+
+	i = 0;
+	total_req_power = 0;
+	max_allocatable_power = 0;
+
+	list_for_each_entry(instance, &tz->thermal_instances, tz_node) {
+		int weight;
+		struct thermal_cooling_device *cdev = instance->cdev;
+
+		if (instance->trip != trip_max_desired_temperature)
+			continue;
+
+		if (!cdev_is_power_actor(cdev))
+			continue;
+
+		if (cdev->ops->get_requested_power(cdev, tz, &req_power[i]))
+			continue;
+
+		if (!total_weight)
+			weight = 1 << FRAC_BITS;
+		else
+			weight = instance->weight;
+
+		req_power[i] = frac_to_int(weight * req_power[i]);
+
+		if (power_actor_get_max_power(cdev, tz, &max_power[i]))
+			continue;
+
+		total_req_power += req_power[i];
+		max_allocatable_power += max_power[i];
+
+		i++;
+	}
+
+	power_range = pid_controller(tz, current_temp, control_temp,
+				     max_allocatable_power);
+
+	divvy_up_power(req_power, max_power, num_actors, total_req_power,
+		       power_range, granted_power);
+
+	i = 0;
+	list_for_each_entry(instance, &tz->thermal_instances, tz_node) {
+		if (instance->trip != trip_max_desired_temperature)
+			continue;
+
+		if (!cdev_is_power_actor(instance->cdev))
+			continue;
+
+		power_actor_set_power(instance->cdev, instance,
+				      granted_power[i]);
+
+		i++;
+	}
+
+	devm_kfree(&tz->device, req_power);
+unlock:
+	mutex_unlock(&tz->lock);
+
+	return ret;
+}
+
+static int get_governor_trips(struct thermal_zone_device *tz,
+			      struct power_allocator_params *params)
+{
+	int i, ret, last_passive;
+	bool found_first_passive;
+
+	found_first_passive = false;
+	last_passive = -1;
+	ret = -EINVAL;
+
+	for (i = 0; i < tz->trips; i++) {
+		enum thermal_trip_type type;
+
+		ret = tz->ops->get_trip_type(tz, i, &type);
+		if (ret)
+			return ret;
+
+		if (!found_first_passive) {
+			if (type == THERMAL_TRIP_PASSIVE) {
+				params->trip_switch_on = i;
+				found_first_passive = true;
+			}
+		} else if (type == THERMAL_TRIP_PASSIVE) {
+			last_passive = i;
+		} else {
+			break;
+		}
+	}
+
+	if (last_passive != -1) {
+		params->trip_max_desired_temperature = last_passive;
+		ret = 0;
+	} else {
+		ret = -EINVAL;
+	}
+
+	return ret;
+}
+
+static void reset_pid_controller(struct power_allocator_params *params)
+{
+	params->err_integral = 0;
+	params->prev_err = 0;
+}
+
+static void allow_maximum_power(struct thermal_zone_device *tz)
+{
+	struct thermal_instance *instance;
+	struct power_allocator_params *params = tz->governor_data;
+
+	list_for_each_entry(instance, &tz->thermal_instances, tz_node) {
+		if ((instance->trip != params->trip_max_desired_temperature) ||
+		    (!cdev_is_power_actor(instance->cdev)))
+			continue;
+
+		instance->target = 0;
+		instance->cdev->updated = false;
+		thermal_cdev_update(instance->cdev);
+	}
+}
+
+/**
+ * power_allocator_bind() - bind the power_allocator governor to a thermal zone
+ * @tz:	thermal zone to bind it to
+ *
+ * Check that the thermal zone is valid for this governor, that is, it
+ * has two thermal trips.  If so, initialize the PID controller
+ * parameters and bind it to the thermal zone.
+ *
+ * Return: 0 on success, -EINVAL if the trips were invalid or -ENOMEM
+ * if we ran out of memory.
+ */
+static int power_allocator_bind(struct thermal_zone_device *tz)
+{
+	int ret;
+	struct power_allocator_params *params;
+	unsigned long switch_on_temp, control_temp;
+	u32 temperature_threshold;
+
+	if (!tz->tzp || !tz->tzp->sustainable_power) {
+		dev_err(&tz->device,
+			"power_allocator: missing sustainable_power\n");
+		return -EINVAL;
+	}
+
+	params = devm_kzalloc(&tz->device, sizeof(*params), GFP_KERNEL);
+	if (!params)
+		return -ENOMEM;
+
+	ret = get_governor_trips(tz, params);
+	if (ret) {
+		dev_err(&tz->device,
+			"thermal zone %s has wrong trip setup for power allocator\n",
+			tz->type);
+		goto free;
+	}
+
+	ret = tz->ops->get_trip_temp(tz, params->trip_switch_on,
+				     &switch_on_temp);
+	if (ret)
+		goto free;
+
+	ret = tz->ops->get_trip_temp(tz, params->trip_max_desired_temperature,
+				     &control_temp);
+	if (ret)
+		goto free;
+
+	temperature_threshold = control_temp - switch_on_temp;
+
+	tz->tzp->k_po = tz->tzp->k_po ?:
+		int_to_frac(tz->tzp->sustainable_power) / temperature_threshold;
+	tz->tzp->k_pu = tz->tzp->k_pu ?:
+		int_to_frac(2 * tz->tzp->sustainable_power) /
+		temperature_threshold;
+	tz->tzp->k_i = tz->tzp->k_i ?: int_to_frac(10) / 1000;
+	/*
+	 * The default for k_d and integral_cutoff is 0, so we can
+	 * leave them as they are.
+	 */
+
+	reset_pid_controller(params);
+
+	tz->governor_data = params;
+
+	return 0;
+
+free:
+	devm_kfree(&tz->device, params);
+	return ret;
+}
+
+static void power_allocator_unbind(struct thermal_zone_device *tz)
+{
+	dev_dbg(&tz->device, "Unbinding from thermal zone %d\n", tz->id);
+	devm_kfree(&tz->device, tz->governor_data);
+	tz->governor_data = NULL;
+}
+
+static int power_allocator_throttle(struct thermal_zone_device *tz, int trip)
+{
+	int ret;
+	unsigned long switch_on_temp, control_temp, current_temp;
+	struct power_allocator_params *params = tz->governor_data;
+
+	/*
+	 * We get called for every trip point but we only need to do
+	 * our calculations once
+	 */
+	if (trip != params->trip_max_desired_temperature)
+		return 0;
+
+	ret = thermal_zone_get_temp(tz, &current_temp);
+	if (ret) {
+		dev_warn(&tz->device, "Failed to get temperature: %d\n", ret);
+		return ret;
+	}
+
+	ret = tz->ops->get_trip_temp(tz, params->trip_switch_on,
+				     &switch_on_temp);
+	if (ret) {
+		dev_warn(&tz->device,
+			 "Failed to get switch on temperature: %d\n", ret);
+		return ret;
+	}
+
+	if (current_temp < switch_on_temp) {
+		tz->passive = 0;
+		reset_pid_controller(params);
+		allow_maximum_power(tz);
+		return 0;
+	}
+
+	tz->passive = 1;
+
+	ret = tz->ops->get_trip_temp(tz, params->trip_max_desired_temperature,
+				&control_temp);
+	if (ret) {
+		dev_warn(&tz->device,
+			 "Failed to get the maximum desired temperature: %d\n",
+			 ret);
+		return ret;
+	}
+
+	return allocate_power(tz, current_temp, control_temp);
+}
+
+static struct thermal_governor thermal_gov_power_allocator = {
+	.name		= "power_allocator",
+	.bind_to_tz	= power_allocator_bind,
+	.unbind_from_tz	= power_allocator_unbind,
+	.throttle	= power_allocator_throttle,
+};
+
+int thermal_gov_power_allocator_register(void)
+{
+	return thermal_register_governor(&thermal_gov_power_allocator);
+}
+
+void thermal_gov_power_allocator_unregister(void)
+{
+	thermal_unregister_governor(&thermal_gov_power_allocator);
+}
diff --git a/drivers/thermal/thermal_core.c b/drivers/thermal/thermal_core.c
index 82f1cca9c8ce..df9ba3bf55dc 100644
--- a/drivers/thermal/thermal_core.c
+++ b/drivers/thermal/thermal_core.c
@@ -1617,7 +1617,7 @@ static void remove_trip_attrs(struct thermal_zone_device *tz)
 struct thermal_zone_device *thermal_zone_device_register(const char *type,
 	int trips, int mask, void *devdata,
 	struct thermal_zone_device_ops *ops,
-	const struct thermal_zone_params *tzp,
+	struct thermal_zone_params *tzp,
 	int passive_delay, int polling_delay)
 {
 	struct thermal_zone_device *tz;
@@ -1969,7 +1969,11 @@ static int __init thermal_register_governors(void)
 	if (result)
 		return result;
 
-	return thermal_gov_user_space_register();
+	result = thermal_gov_user_space_register();
+	if (result)
+		return result;
+
+	return thermal_gov_power_allocator_register();
 }
 
 static void thermal_unregister_governors(void)
@@ -1978,6 +1982,7 @@ static void thermal_unregister_governors(void)
 	thermal_gov_fair_share_unregister();
 	thermal_gov_bang_bang_unregister();
 	thermal_gov_user_space_unregister();
+	thermal_gov_power_allocator_unregister();
 }
 
 static int __init thermal_init(void)
diff --git a/drivers/thermal/thermal_core.h b/drivers/thermal/thermal_core.h
index faebe881f062..8a6624488cc5 100644
--- a/drivers/thermal/thermal_core.h
+++ b/drivers/thermal/thermal_core.h
@@ -88,6 +88,14 @@ static inline int thermal_gov_user_space_register(void) { return 0; }
 static inline void thermal_gov_user_space_unregister(void) {}
 #endif /* CONFIG_THERMAL_GOV_USER_SPACE */
 
+#ifdef CONFIG_THERMAL_GOV_POWER_ALLOCATOR
+int thermal_gov_power_allocator_register(void);
+void thermal_gov_power_allocator_unregister(void);
+#else
+static inline int thermal_gov_power_allocator_register(void) { return 0; }
+static inline void thermal_gov_power_allocator_unregister(void) {}
+#endif /* CONFIG_THERMAL_GOV_POWER_ALLOCATOR */
+
 /* device tree support */
 #ifdef CONFIG_THERMAL_OF
 int of_parse_thermal_zones(void);
diff --git a/include/linux/thermal.h b/include/linux/thermal.h
index d69b1806f430..f8f503cd50a6 100644
--- a/include/linux/thermal.h
+++ b/include/linux/thermal.h
@@ -59,6 +59,8 @@
 #define DEFAULT_THERMAL_GOVERNOR       "fair_share"
 #elif defined(CONFIG_THERMAL_DEFAULT_GOV_USER_SPACE)
 #define DEFAULT_THERMAL_GOVERNOR       "user_space"
+#elif defined(CONFIG_THERMAL_DEFAULT_GOV_POWER_ALLOCATOR)
+#define DEFAULT_THERMAL_GOVERNOR       "power_allocator"
 #endif
 
 struct thermal_zone_device;
@@ -154,8 +156,7 @@ struct thermal_attr {
  * @devdata:	private pointer for device private data
  * @trips:	number of trip points the thermal zone supports
  * @passive_delay:	number of milliseconds to wait between polls when
- *			performing passive cooling.  Currenty only used by the
- *			step-wise governor
+ *			performing passive cooling.
  * @polling_delay:	number of milliseconds to wait between polls when
  *			checking whether trip points have been crossed (0 for
  *			interrupt driven systems)
@@ -165,7 +166,6 @@ struct thermal_attr {
  * @last_temperature:	previous temperature read
  * @emul_temperature:	emulated temperature when using CONFIG_THERMAL_EMULATION
  * @passive:		1 if you've crossed a passive trip point, 0 otherwise.
- *			Currenty only used by the step-wise governor.
  * @forced_passive:	If > 0, temperature at which to switch on all ACPI
  *			processor cooling devices.  Currently only used by the
  *			step-wise governor.
@@ -197,7 +197,7 @@ struct thermal_zone_device {
 	int passive;
 	unsigned int forced_passive;
 	struct thermal_zone_device_ops *ops;
-	const struct thermal_zone_params *tzp;
+	struct thermal_zone_params *tzp;
 	struct thermal_governor *governor;
 	void *governor_data;
 	struct list_head thermal_instances;
@@ -275,6 +275,33 @@ struct thermal_zone_params {
 
 	int num_tbps;	/* Number of tbp entries */
 	struct thermal_bind_params *tbp;
+
+	/*
+	 * Sustainable power (heat) that this thermal zone can dissipate in
+	 * mW
+	 */
+	u32 sustainable_power;
+
+	/*
+	 * Proportional parameter of the PID controller when
+	 * overshooting (i.e., when temperature is below the target)
+	 */
+	s32 k_po;
+
+	/*
+	 * Proportional parameter of the PID controller when
+	 * undershooting
+	 */
+	s32 k_pu;
+
+	/* Integral parameter of the PID controller */
+	s32 k_i;
+
+	/* Derivative parameter of the PID controller */
+	s32 k_d;
+
+	/* threshold below which the error is no longer accumulated */
+	s32 integral_cutoff;
 };
 
 struct thermal_genl_event {
@@ -349,7 +376,7 @@ int power_actor_set_power(struct thermal_cooling_device *,
 			  struct thermal_instance *, u32);
 struct thermal_zone_device *thermal_zone_device_register(const char *, int, int,
 		void *, struct thermal_zone_device_ops *,
-		const struct thermal_zone_params *, int, int);
+		struct thermal_zone_params *, int, int);
 void thermal_zone_device_unregister(struct thermal_zone_device *);
 
 int thermal_zone_bind_cooling_device(struct thermal_zone_device *, int,
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v2 5/7] thermal: add trace events to the power allocator governor
  2015-02-26 19:00 [PATCH v2 0/7] The power allocator thermal governor Javi Merino
                   ` (3 preceding siblings ...)
  2015-02-26 19:00 ` [PATCH v2 4/7] thermal: introduce the Power Allocator governor Javi Merino
@ 2015-02-26 19:00 ` Javi Merino
  2015-02-26 19:38   ` Steven Rostedt
  2015-02-26 19:00 ` [PATCH v2 6/7] of: thermal: Introduce sustainable power for a thermal zone Javi Merino
                   ` (2 subsequent siblings)
  7 siblings, 1 reply; 22+ messages in thread
From: Javi Merino @ 2015-02-26 19:00 UTC (permalink / raw)
  To: edubezval, rui.zhang
  Cc: linux-pm, linux-kernel, punit.agrawal, broonie, tixy,
	Javi Merino, Steven Rostedt, Frederic Weisbecker, Ingo Molnar

Add trace events for the power allocator governor and the power actor
interface of the cpu cooling device.

Cc: Zhang Rui <rui.zhang@intel.com>
Cc: Eduardo Valentin <edubezval@gmail.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@redhat.com>
Signed-off-by: Javi Merino <javi.merino@arm.com>
---
 drivers/thermal/cpu_cooling.c                  | 31 ++++++++-
 drivers/thermal/power_allocator.c              | 22 ++++++-
 include/trace/events/thermal.h                 | 58 +++++++++++++++++
 include/trace/events/thermal_power_allocator.h | 87 ++++++++++++++++++++++++++
 4 files changed, 194 insertions(+), 4 deletions(-)
 create mode 100644 include/trace/events/thermal_power_allocator.h

diff --git a/drivers/thermal/cpu_cooling.c b/drivers/thermal/cpu_cooling.c
index ba23150c7bde..c4974144c787 100644
--- a/drivers/thermal/cpu_cooling.c
+++ b/drivers/thermal/cpu_cooling.c
@@ -31,6 +31,8 @@
 #include <linux/cpu.h>
 #include <linux/cpu_cooling.h>
 
+#include <trace/events/thermal.h>
+
 /*
  * Cooling state <-> CPUFreq frequency
  *
@@ -588,12 +590,20 @@ static int cpufreq_get_requested_power(struct thermal_cooling_device *cdev,
 				       u32 *power)
 {
 	unsigned long freq;
-	int cpu, ret;
+	int i = 0, cpu, ret;
 	u32 static_power, dynamic_power, total_load = 0;
 	struct cpufreq_cooling_device *cpufreq_device = cdev->devdata;
+	u32 *load_cpu = NULL;
 
 	freq = cpufreq_quick_get(cpumask_any(&cpufreq_device->allowed_cpus));
 
+	if (trace_thermal_power_cpu_get_power_enabled()) {
+		u32 ncpus = cpumask_weight(&cpufreq_device->allowed_cpus);
+
+		load_cpu = devm_kcalloc(&cdev->device, ncpus, sizeof(*load_cpu),
+					GFP_KERNEL);
+	}
+
 	for_each_cpu(cpu, &cpufreq_device->allowed_cpus) {
 		u32 load;
 
@@ -603,14 +613,29 @@ static int cpufreq_get_requested_power(struct thermal_cooling_device *cdev,
 			load = 0;
 
 		total_load += load;
+		if (trace_thermal_power_cpu_limit_enabled() && load_cpu)
+			load_cpu[i] = load;
+
+		i++;
 	}
 
 	cpufreq_device->last_load = total_load;
 
 	dynamic_power = get_dynamic_power(cpufreq_device, freq);
 	ret = get_static_power(cpufreq_device, tz, freq, &static_power);
-	if (ret)
+	if (ret) {
+		if (load_cpu)
+			devm_kfree(&cdev->device, load_cpu);
 		return ret;
+	}
+
+	if (load_cpu) {
+		trace_thermal_power_cpu_get_power(
+			&cpufreq_device->allowed_cpus,
+			freq, load_cpu, i, dynamic_power, static_power);
+
+		devm_kfree(&cdev->device, load_cpu);
+	}
 
 	*power = static_power + dynamic_power;
 	return 0;
@@ -718,6 +743,8 @@ static int cpufreq_power2state(struct thermal_cooling_device *cdev,
 		return -EINVAL;
 	}
 
+	trace_thermal_power_cpu_limit(&cpufreq_device->allowed_cpus,
+				      target_freq, *state, power);
 	return 0;
 }
 
diff --git a/drivers/thermal/power_allocator.c b/drivers/thermal/power_allocator.c
index bfea8d35c0a2..9b32ccb9ea82 100644
--- a/drivers/thermal/power_allocator.c
+++ b/drivers/thermal/power_allocator.c
@@ -19,6 +19,9 @@
 #include <linux/slab.h>
 #include <linux/thermal.h>
 
+#define CREATE_TRACE_POINTS
+#include <trace/events/thermal_power_allocator.h>
+
 #include "thermal_core.h"
 
 #define FRAC_BITS 10
@@ -138,7 +141,14 @@ static u32 pid_controller(struct thermal_zone_device *tz,
 	/* feed-forward the known sustainable dissipatable power */
 	power_range = tz->tzp->sustainable_power + frac_to_int(power_range);
 
-	return clamp(power_range, (s64)0, (s64)max_allocatable_power);
+	power_range = clamp(power_range, (s64)0, (s64)max_allocatable_power);
+
+	trace_thermal_power_allocator_pid(tz, frac_to_int(err),
+					  frac_to_int(params->err_integral),
+					  frac_to_int(p), frac_to_int(i),
+					  frac_to_int(d), power_range);
+
+	return power_range;
 }
 
 /**
@@ -216,7 +226,7 @@ static int allocate_power(struct thermal_zone_device *tz,
 	struct power_allocator_params *params = tz->governor_data;
 	u32 *req_power, *max_power, *granted_power;
 	u32 total_req_power, max_allocatable_power;
-	u32 power_range;
+	u32 total_granted_power, power_range;
 	int i, num_actors, total_weight, ret = 0;
 	int trip_max_desired_temperature = params->trip_max_desired_temperature;
 
@@ -290,6 +300,7 @@ static int allocate_power(struct thermal_zone_device *tz,
 	divvy_up_power(req_power, max_power, num_actors, total_req_power,
 		       power_range, granted_power);
 
+	total_granted_power = 0;
 	i = 0;
 	list_for_each_entry(instance, &tz->thermal_instances, tz_node) {
 		if (instance->trip != trip_max_desired_temperature)
@@ -300,10 +311,17 @@ static int allocate_power(struct thermal_zone_device *tz,
 
 		power_actor_set_power(instance->cdev, instance,
 				      granted_power[i]);
+		total_granted_power += granted_power[i];
 
 		i++;
 	}
 
+	trace_thermal_power_allocator(tz, req_power, total_req_power,
+				      granted_power, total_granted_power,
+				      num_actors, power_range,
+				      max_allocatable_power, current_temp,
+				      (s32)control_temp - (s32)current_temp);
+
 	devm_kfree(&tz->device, req_power);
 unlock:
 	mutex_unlock(&tz->lock);
diff --git a/include/trace/events/thermal.h b/include/trace/events/thermal.h
index 0f4f95d63c03..8b1f80682b80 100644
--- a/include/trace/events/thermal.h
+++ b/include/trace/events/thermal.h
@@ -77,6 +77,64 @@ TRACE_EVENT(thermal_zone_trip,
 		__entry->trip_type)
 );
 
+TRACE_EVENT(thermal_power_cpu_get_power,
+	TP_PROTO(const struct cpumask *cpus, unsigned long freq, u32 *load,
+		size_t load_len, u32 dynamic_power, u32 static_power),
+
+	TP_ARGS(cpus, freq, load, load_len, dynamic_power, static_power),
+
+	TP_STRUCT__entry(
+		__bitmask(cpumask, num_possible_cpus())
+		__field(unsigned long, freq          )
+		__dynamic_array(u32,   load, load_len)
+		__field(size_t,        load_len      )
+		__field(u32,           dynamic_power )
+		__field(u32,           static_power  )
+	),
+
+	TP_fast_assign(
+		__assign_bitmask(cpumask, cpumask_bits(cpus),
+				num_possible_cpus());
+		__entry->freq = freq;
+		memcpy(__get_dynamic_array(load), load,
+			load_len * sizeof(*load));
+		__entry->load_len = load_len;
+		__entry->dynamic_power = dynamic_power;
+		__entry->static_power = static_power;
+	),
+
+	TP_printk("cpus=%s freq=%lu load={%s} dynamic_power=%d static_power=%d",
+		__get_bitmask(cpumask), __entry->freq,
+		__print_array(__get_dynamic_array(load), __entry->load_len, 4),
+		__entry->dynamic_power, __entry->static_power)
+);
+
+TRACE_EVENT(thermal_power_cpu_limit,
+	TP_PROTO(const struct cpumask *cpus, unsigned int freq,
+		unsigned long cdev_state, u32 power),
+
+	TP_ARGS(cpus, freq, cdev_state, power),
+
+	TP_STRUCT__entry(
+		__bitmask(cpumask, num_possible_cpus())
+		__field(unsigned int,  freq      )
+		__field(unsigned long, cdev_state)
+		__field(u32,           power     )
+	),
+
+	TP_fast_assign(
+		__assign_bitmask(cpumask, cpumask_bits(cpus),
+				num_possible_cpus());
+		__entry->freq = freq;
+		__entry->cdev_state = cdev_state;
+		__entry->power = power;
+	),
+
+	TP_printk("cpus=%s freq=%u cdev_state=%lu power=%u",
+		__get_bitmask(cpumask), __entry->freq, __entry->cdev_state,
+		__entry->power)
+);
+
 #endif /* _TRACE_THERMAL_H */
 
 /* This part must be outside protection */
diff --git a/include/trace/events/thermal_power_allocator.h b/include/trace/events/thermal_power_allocator.h
new file mode 100644
index 000000000000..12e1321c4e0c
--- /dev/null
+++ b/include/trace/events/thermal_power_allocator.h
@@ -0,0 +1,87 @@
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM thermal_power_allocator
+
+#if !defined(_TRACE_THERMAL_POWER_ALLOCATOR_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_THERMAL_POWER_ALLOCATOR_H
+
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(thermal_power_allocator,
+	TP_PROTO(struct thermal_zone_device *tz, u32 *req_power,
+		 u32 total_req_power, u32 *granted_power,
+		 u32 total_granted_power, size_t num_actors,
+		 u32 power_range, u32 max_allocatable_power,
+		 unsigned long current_temp, s32 delta_temp),
+	TP_ARGS(tz, req_power, total_req_power, granted_power,
+		total_granted_power, num_actors, power_range,
+		max_allocatable_power, current_temp, delta_temp),
+	TP_STRUCT__entry(
+		__field(int,           tz_id          )
+		__dynamic_array(u32,   req_power, num_actors    )
+		__field(u32,           total_req_power          )
+		__dynamic_array(u32,   granted_power, num_actors)
+		__field(u32,           total_granted_power      )
+		__field(size_t,        num_actors               )
+		__field(u32,           power_range              )
+		__field(u32,           max_allocatable_power    )
+		__field(unsigned long, current_temp             )
+		__field(s32,           delta_temp               )
+	),
+	TP_fast_assign(
+		__entry->tz_id = tz->id;
+		memcpy(__get_dynamic_array(req_power), req_power,
+			num_actors * sizeof(*req_power));
+		__entry->total_req_power = total_req_power;
+		memcpy(__get_dynamic_array(granted_power), granted_power,
+			num_actors * sizeof(*granted_power));
+		__entry->total_granted_power = total_granted_power;
+		__entry->num_actors = num_actors;
+		__entry->power_range = power_range;
+		__entry->max_allocatable_power = max_allocatable_power;
+		__entry->current_temp = current_temp;
+		__entry->delta_temp = delta_temp;
+	),
+
+	TP_printk("thermal_zone_id=%d req_power={%s} total_req_power=%u granted_power={%s} total_granted_power=%u power_range=%u max_allocatable_power=%u current_temperature=%lu delta_temperature=%d",
+		__entry->tz_id,
+		__print_array(__get_dynamic_array(req_power),
+                              __entry->num_actors, 4),
+		__entry->total_req_power,
+		__print_array(__get_dynamic_array(granted_power),
+                              __entry->num_actors, 4),
+		__entry->total_granted_power, __entry->power_range,
+		__entry->max_allocatable_power, __entry->current_temp,
+		__entry->delta_temp)
+);
+
+TRACE_EVENT(thermal_power_allocator_pid,
+	TP_PROTO(struct thermal_zone_device *tz, s32 err, s32 err_integral,
+		 s64 p, s64 i, s64 d, s32 output),
+	TP_ARGS(tz, err, err_integral, p, i, d, output),
+	TP_STRUCT__entry(
+		__field(int, tz_id       )
+		__field(s32, err         )
+		__field(s32, err_integral)
+		__field(s64, p           )
+		__field(s64, i           )
+		__field(s64, d           )
+		__field(s32, output      )
+	),
+	TP_fast_assign(
+		__entry->tz_id = tz->id;
+		__entry->err = err;
+		__entry->err_integral = err_integral;
+		__entry->p = p;
+		__entry->i = i;
+		__entry->d = d;
+		__entry->output = output;
+	),
+
+	TP_printk("thermal_zone_id=%d err=%d err_integral=%d p=%lld i=%lld d=%lld output=%d",
+		  __entry->tz_id, __entry->err, __entry->err_integral,
+		  __entry->p, __entry->i, __entry->d, __entry->output)
+);
+#endif /* _TRACE_THERMAL_POWER_ALLOCATOR_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v2 6/7] of: thermal: Introduce sustainable power for a thermal zone
  2015-02-26 19:00 [PATCH v2 0/7] The power allocator thermal governor Javi Merino
                   ` (4 preceding siblings ...)
  2015-02-26 19:00 ` [PATCH v2 5/7] thermal: add trace events to the power allocator governor Javi Merino
@ 2015-02-26 19:00 ` Javi Merino
  2015-02-26 19:00 ` [PATCH v2 7/7] thermal: export thermal_zone_parameters to sysfs Javi Merino
  2015-02-26 22:16 ` [PATCH v2 0/7] The power allocator thermal governor Eduardo Valentin
  7 siblings, 0 replies; 22+ messages in thread
From: Javi Merino @ 2015-02-26 19:00 UTC (permalink / raw)
  To: edubezval, rui.zhang; +Cc: linux-pm, linux-kernel, punit.agrawal, broonie, tixy

From: Punit Agrawal <punit.agrawal@arm.com>

Introduce an optional property called, sustainable-power, which
represents the power (in mW) which the thermal zone can safely
dissipate.

If provided the property is parsed and associated with the thermal
zone via the thermal zone parameters.

Cc: Zhang Rui <rui.zhang@intel.com>
Cc: Eduardo Valentin <edubezval@gmail.com>
Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
---
 Documentation/devicetree/bindings/thermal/thermal.txt | 9 +++++++++
 drivers/thermal/of-thermal.c                          | 4 ++++
 2 files changed, 13 insertions(+)

diff --git a/Documentation/devicetree/bindings/thermal/thermal.txt b/Documentation/devicetree/bindings/thermal/thermal.txt
index 29fe0bfae38e..8a49362dea6e 100644
--- a/Documentation/devicetree/bindings/thermal/thermal.txt
+++ b/Documentation/devicetree/bindings/thermal/thermal.txt
@@ -167,6 +167,13 @@ Optional property:
 			by means of sensor ID. Additional coefficients are
 			interpreted as constant offset.
 
+- sustainable-power:	An estimate of the sustainable power (in mW) that the
+  Type: unsigned	thermal zone can dissipate at the desired
+  Size: one cell	control temperature.  For reference, the
+			sustainable power of a 4'' phone is typically
+			2000mW, while on a 10'' tablet is around
+			4500mW.
+
 Note: The delay properties are bound to the maximum dT/dt (temperature
 derivative over time) in two situations for a thermal zone:
 (i)  - when passive cooling is activated (polling-delay-passive); and
@@ -546,6 +553,8 @@ thermal-zones {
 		 */
 		coefficients =		<1200	-345	890>;
 
+		sustainable-power = <2500>;
+
 		trips {
 			/* Trips are based on resulting linear equation */
 			cpu_trip: cpu-trip {
diff --git a/drivers/thermal/of-thermal.c b/drivers/thermal/of-thermal.c
index c606b85ea9f4..705b21d01f1c 100644
--- a/drivers/thermal/of-thermal.c
+++ b/drivers/thermal/of-thermal.c
@@ -866,6 +866,7 @@ int __init of_parse_thermal_zones(void)
 	for_each_child_of_node(np, child) {
 		struct thermal_zone_device *zone;
 		struct thermal_zone_params *tzp;
+		u32 prop;
 
 		/* Check whether child is enabled or not */
 		if (!of_device_is_available(child))
@@ -892,6 +893,9 @@ int __init of_parse_thermal_zones(void)
 		/* No hwmon because there might be hwmon drivers registering */
 		tzp->no_hwmon = true;
 
+		if (!of_property_read_u32(child, "sustainable-power", &prop))
+			tzp->sustainable_power = prop;
+
 		zone = thermal_zone_device_register(child->name, tz->ntrips,
 						    0, tz,
 						    ops, tzp,
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v2 7/7] thermal: export thermal_zone_parameters to sysfs
  2015-02-26 19:00 [PATCH v2 0/7] The power allocator thermal governor Javi Merino
                   ` (5 preceding siblings ...)
  2015-02-26 19:00 ` [PATCH v2 6/7] of: thermal: Introduce sustainable power for a thermal zone Javi Merino
@ 2015-02-26 19:00 ` Javi Merino
  2015-02-26 21:30   ` Eduardo Valentin
  2015-02-26 22:16 ` [PATCH v2 0/7] The power allocator thermal governor Eduardo Valentin
  7 siblings, 1 reply; 22+ messages in thread
From: Javi Merino @ 2015-02-26 19:00 UTC (permalink / raw)
  To: edubezval, rui.zhang
  Cc: linux-pm, linux-kernel, punit.agrawal, broonie, tixy, Javi Merino

It's useful for tuning to be able to edit thermal_zone_parameters from
userspace.  Export them to the thermal_zone sysfs so that they can be
easily changed.

Cc: Zhang Rui <rui.zhang@intel.com>
Cc: Eduardo Valentin <edubezval@gmail.com>
Signed-off-by: Javi Merino <javi.merino@arm.com>
---
 Documentation/thermal/sysfs-api.txt |  52 +++++++++++++++++
 drivers/thermal/thermal_core.c      | 110 ++++++++++++++++++++++++++++++++++++
 2 files changed, 162 insertions(+)

diff --git a/Documentation/thermal/sysfs-api.txt b/Documentation/thermal/sysfs-api.txt
index fc7dfe10778b..7d44d7f1a71b 100644
--- a/Documentation/thermal/sysfs-api.txt
+++ b/Documentation/thermal/sysfs-api.txt
@@ -184,6 +184,12 @@ Thermal zone device sys I/F, created once it's registered:
     |---trip_point_[0-*]_type:	Trip point type
     |---trip_point_[0-*]_hyst:	Hysteresis value for this trip point
     |---emul_temp:		Emulated temperature set node
+    |---sustainable_power:      Sustainable dissipatable power
+    |---k_po:                   Proportional term during temperature overshoot
+    |---k_pu:                   Proportional term during temperature undershoot
+    |---k_i:                    PID's integral term in the power allocator gov
+    |---k_d:                    PID's derivative term in the power allocator
+    |---integral_cutoff:        Offset above which errors are accumulated
 
 Thermal cooling device sys I/F, created once it's registered:
 /sys/class/thermal/cooling_device[0-*]:
@@ -307,6 +313,52 @@ emul_temp
 	  because userland can easily disable the thermal policy by simply
 	  flooding this sysfs node with low temperature values.
 
+sustainable_power
+	An estimate of the sustained power that can be dissipated by
+	the thermal zone. Used by the power allocator governor. For
+	more information see Documentation/thermal/power_allocator.txt
+	Unit: milliwatts
+	RW, Optional
+
+k_po
+	The proportional term of the power allocator governor's PID
+	controller during temperature overshoot. Temperature overshoot
+	is when the current temperature is above the "desired
+	temperature" trip point. For more information see
+	Documentation/thermal/power_allocator.txt
+	RW, Optional
+
+k_pu
+	The proportional term of the power allocator governor's PID
+	controller during temperature undershoot. Temperature undershoot
+	is when the current temperature is below the "desired
+	temperature" trip point. For more information see
+	Documentation/thermal/power_allocator.txt
+	RW, Optional
+
+k_i
+	The integral term of the power allocator governor's PID
+	controller. This term allows the PID controller to compensate
+	for long term drift. For more information see
+	Documentation/thermal/power_allocator.txt
+	RW, Optional
+
+k_d
+	The derivative term of the power allocator governor's PID
+	controller. For more information see
+	Documentation/thermal/power_allocator.txt
+	RW, Optional
+
+integral_cutoff
+	Temperature offset from the desired temperature trip point
+	above which the integral term of the power allocator
+	governor's PID controller starts accumulating errors. For
+	example, if integral_cutoff is 0, then the integral term only
+	accumulates error when temperature is above the desired
+	temperature trip point. For more information see
+	Documentation/thermal/power_allocator.txt
+	RW, Optional
+
 *****************************
 * Cooling device attributes *
 *****************************
diff --git a/drivers/thermal/thermal_core.c b/drivers/thermal/thermal_core.c
index df9ba3bf55dc..228e93f8a146 100644
--- a/drivers/thermal/thermal_core.c
+++ b/drivers/thermal/thermal_core.c
@@ -873,6 +873,111 @@ emul_temp_store(struct device *dev, struct device_attribute *attr,
 static DEVICE_ATTR(emul_temp, S_IWUSR, NULL, emul_temp_store);
 #endif/*CONFIG_THERMAL_EMULATION*/
 
+#ifdef CONFIG_THERMAL_GOV_POWER_ALLOCATOR
+
+static ssize_t
+sustainable_power_show(struct device *dev, struct device_attribute *devattr,
+		       char *buf)
+{
+	struct thermal_zone_device *tz = to_thermal_zone(dev);
+
+	if (tz->tzp)
+		return sprintf(buf, "%u\n", tz->tzp->sustainable_power);
+	else
+		return -EIO;
+}
+
+static ssize_t
+sustainable_power_store(struct device *dev, struct device_attribute *devattr,
+			const char *buf, size_t count)
+{
+	struct thermal_zone_device *tz = to_thermal_zone(dev);
+	u32 sustainable_power;
+
+	if (!tz->tzp)
+		return -EIO;
+
+	if (kstrtou32(buf, 10, &sustainable_power))
+		return -EINVAL;
+
+	tz->tzp->sustainable_power = sustainable_power;
+
+	return count;
+}
+static DEVICE_ATTR(sustainable_power, S_IWUSR | S_IRUGO, sustainable_power_show,
+		sustainable_power_store);
+
+#define create_s32_tzp_attr(name)					\
+	static ssize_t							\
+	name##_show(struct device *dev, struct device_attribute *devattr, \
+		char *buf)						\
+	{								\
+	struct thermal_zone_device *tz = to_thermal_zone(dev);		\
+									\
+	if (tz->tzp)							\
+		return sprintf(buf, "%u\n", tz->tzp->name);		\
+	else								\
+		return -EIO;						\
+	}								\
+									\
+	static ssize_t							\
+	name##_store(struct device *dev, struct device_attribute *devattr, \
+		const char *buf, size_t count)				\
+	{								\
+		struct thermal_zone_device *tz = to_thermal_zone(dev);	\
+		s32 value;						\
+									\
+		if (!tz->tzp)						\
+			return -EIO;					\
+									\
+		if (kstrtos32(buf, 10, &value))				\
+			return -EINVAL;					\
+									\
+		tz->tzp->name = value;					\
+									\
+		return count;						\
+	}								\
+	static DEVICE_ATTR(name, S_IWUSR | S_IRUGO, name##_show, name##_store)
+
+create_s32_tzp_attr(k_po);
+create_s32_tzp_attr(k_pu);
+create_s32_tzp_attr(k_i);
+create_s32_tzp_attr(k_d);
+create_s32_tzp_attr(integral_cutoff);
+#undef create_s32_tzp_attr
+
+static struct device_attribute *dev_tzp_attrs[] = {
+	&dev_attr_sustainable_power,
+	&dev_attr_k_po,
+	&dev_attr_k_pu,
+	&dev_attr_k_i,
+	&dev_attr_k_d,
+	&dev_attr_integral_cutoff,
+};
+
+static int create_power_allocator_tzp_attrs(struct device *dev)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(dev_tzp_attrs); i++) {
+		int ret;
+		struct device_attribute *dev_attr = dev_tzp_attrs[i];
+
+		ret = device_create_file(dev, dev_attr);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+#else  /* !CONFIG_THERMAL_GOV_POWER_ALLOCATOR */
+static int create_power_allocator_tzp_attrs(struct device *dev)
+{
+	return 0;
+}
+#endif
+
 /**
  * power_actor_get_max_power() - get the maximum power that a cdev can consume
  * @cdev:	pointer to &thermal_cooling_device
@@ -1712,6 +1817,11 @@ struct thermal_zone_device *thermal_zone_device_register(const char *type,
 	if (result)
 		goto unregister;
 
+	/* Add power_allocator specific thermal zone params */
+	result = create_power_allocator_tzp_attrs(&tz->device);
+	if (result)
+		goto unregister;
+
 	/* Update 'this' zone's governor information */
 	mutex_lock(&thermal_governor_lock);
 
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* Re: [PATCH v2 5/7] thermal: add trace events to the power allocator governor
  2015-02-26 19:00 ` [PATCH v2 5/7] thermal: add trace events to the power allocator governor Javi Merino
@ 2015-02-26 19:38   ` Steven Rostedt
  2015-02-26 19:46     ` Eduardo Valentin
  0 siblings, 1 reply; 22+ messages in thread
From: Steven Rostedt @ 2015-02-26 19:38 UTC (permalink / raw)
  To: Javi Merino
  Cc: edubezval, rui.zhang, linux-pm, linux-kernel, punit.agrawal,
	broonie, tixy, Frederic Weisbecker, Ingo Molnar

On Thu, 26 Feb 2015 19:00:31 +0000
Javi Merino <javi.merino@arm.com> wrote:

> Add trace events for the power allocator governor and the power actor
> interface of the cpu cooling device.

For the tracing parts...

Acked-by: Steven Rostedt <rostedt@goodmis.org>

-- Steve

> 
> Cc: Zhang Rui <rui.zhang@intel.com>
> Cc: Eduardo Valentin <edubezval@gmail.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Frederic Weisbecker <fweisbec@gmail.com>
> Cc: Ingo Molnar <mingo@redhat.com>
> Signed-off-by: Javi Merino <javi.merino@arm.com>
> ---

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v2 5/7] thermal: add trace events to the power allocator governor
  2015-02-26 19:38   ` Steven Rostedt
@ 2015-02-26 19:46     ` Eduardo Valentin
  2015-02-26 22:52       ` Steven Rostedt
  0 siblings, 1 reply; 22+ messages in thread
From: Eduardo Valentin @ 2015-02-26 19:46 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: Javi Merino, rui.zhang, linux-pm, linux-kernel, punit.agrawal,
	broonie, tixy, Frederic Weisbecker, Ingo Molnar

[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

On Thu, Feb 26, 2015 at 02:38:45PM -0500, Steven Rostedt wrote:
> On Thu, 26 Feb 2015 19:00:31 +0000
> Javi Merino <javi.merino@arm.com> wrote:
> 
> > Add trace events for the power allocator governor and the power actor
> > interface of the cpu cooling device.
> 
> For the tracing parts...
> 
> Acked-by: Steven Rostedt <rostedt@goodmis.org>

Thanks Steve.

I am planing to apply this series to my thermal soc tree. Do you have
any concerns if I send this patch, which touches the tracing files, via
my tree?

BR,


Eduardo Valentin

> 
> -- Steve
> 
> > 
> > Cc: Zhang Rui <rui.zhang@intel.com>
> > Cc: Eduardo Valentin <edubezval@gmail.com>
> > Cc: Steven Rostedt <rostedt@goodmis.org>
> > Cc: Frederic Weisbecker <fweisbec@gmail.com>
> > Cc: Ingo Molnar <mingo@redhat.com>
> > Signed-off-by: Javi Merino <javi.merino@arm.com>
> > ---

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v2 4/7] thermal: introduce the Power Allocator governor
  2015-02-26 19:00 ` [PATCH v2 4/7] thermal: introduce the Power Allocator governor Javi Merino
@ 2015-02-26 20:36   ` Eduardo Valentin
  2015-02-27 16:12     ` Javi Merino
  0 siblings, 1 reply; 22+ messages in thread
From: Eduardo Valentin @ 2015-02-26 20:36 UTC (permalink / raw)
  To: Javi Merino
  Cc: rui.zhang, linux-pm, linux-kernel, punit.agrawal, broonie, tixy

[-- Attachment #1: Type: text/plain, Size: 1898 bytes --]

Javi,

Replying in v2.

On Thu, Feb 26, 2015 at 07:00:30PM +0000, Javi Merino wrote:

<big cut>

> +
> +/**
> + * divvy_up_power() - divvy the allocated power between the actors
> + * @req_power:	each actor's requested power
> + * @max_power:	each actor's maximum available power
> + * @num_actors:	size of the @req_power, @max_power and @granted_power's array
> + * @total_req_power: sum of @req_power
> + * @power_range:	total allocated power
> + * @granted_power:	output array: each actor's granted power
> + *
> + * This function divides the total allocated power (@power_range)
> + * fairly between the actors.  It first tries to give each actor a
> + * share of the @power_range according to how much power it requested
> + * compared to the rest of the actors.  For example, if only one actor
> + * requests power, then it receives all the @power_range.  If
> + * three actors each requests 1mW, each receives a third of the
> + * @power_range.
> + *
> + * If any actor received more than their maximum power, then that
> + * surplus is re-divvied among the actors based on how far they are
> + * from their respective maximums.
> + *
> + * Granted power for each actor is written to @granted_power, which
> + * should've been allocated by the calling function.
> + */
> +static void divvy_up_power(u32 *req_power, u32 *max_power, int num_actors,
> +			   u32 total_req_power, u32 power_range,
> +			   u32 *granted_power)
> +{
> +	u32 extra_power, capped_extra_power, extra_actor_power[num_actors];

Just now I noticed this variable-size array. I know the number of actors
are constant, and in average it should be less than 10 in magnitude, but
I don't believe variable-sized arrays are a good practice in kernel.

Even though I don't see this one as a treat, can we avoid it?
Sparse error:

drivers/thermal/power_allocator.c:172:64: error: bad constant expression


BR,

Eduardo Valentin

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v2 5/7] thermal: add trace events to the power allocator governor
  2015-02-26 22:52       ` Steven Rostedt
@ 2015-02-26 21:18         ` Eduardo Valentin
  2015-02-27  0:09           ` Steven Rostedt
  0 siblings, 1 reply; 22+ messages in thread
From: Eduardo Valentin @ 2015-02-26 21:18 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: Javi Merino, rui.zhang, linux-pm, linux-kernel, punit.agrawal,
	broonie, tixy, Frederic Weisbecker, Ingo Molnar

[-- Attachment #1: Type: text/plain, Size: 695 bytes --]

Steve,

On Thu, Feb 26, 2015 at 05:52:59PM -0500, Steven Rostedt wrote:
> On Thu, 26 Feb 2015 15:46:34 -0400
> Eduardo Valentin <edubezval@gmail.com> wrote:
> 
>  
> > I am planing to apply this series to my thermal soc tree. Do you have
> > any concerns if I send this patch, which touches the tracing files, via
> > my tree?
> > 
> 
> Which tracing files?

I am referring to:
include/trace/events/thermal.h                 | 58 +++++++++++++++++
include/trace/events/thermal_power_allocator.h | 87
++++++++++++++++++++++++++

I know they are new files anyway, but just checking if those can go
altogether via the thermal tree.

BR,

Eduardo Valentin

> 
> -- Steve

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v2 7/7] thermal: export thermal_zone_parameters to sysfs
  2015-02-26 19:00 ` [PATCH v2 7/7] thermal: export thermal_zone_parameters to sysfs Javi Merino
@ 2015-02-26 21:30   ` Eduardo Valentin
  2015-02-26 22:04     ` Eduardo Valentin
  0 siblings, 1 reply; 22+ messages in thread
From: Eduardo Valentin @ 2015-02-26 21:30 UTC (permalink / raw)
  To: Javi Merino
  Cc: rui.zhang, linux-pm, linux-kernel, punit.agrawal, broonie, tixy

[-- Attachment #1: Type: text/plain, Size: 8111 bytes --]

On Thu, Feb 26, 2015 at 07:00:33PM +0000, Javi Merino wrote:
> It's useful for tuning to be able to edit thermal_zone_parameters from
> userspace.  Export them to the thermal_zone sysfs so that they can be
> easily changed.
> 
> Cc: Zhang Rui <rui.zhang@intel.com>
> Cc: Eduardo Valentin <edubezval@gmail.com>
> Signed-off-by: Javi Merino <javi.merino@arm.com>
> ---
>  Documentation/thermal/sysfs-api.txt |  52 +++++++++++++++++
>  drivers/thermal/thermal_core.c      | 110 ++++++++++++++++++++++++++++++++++++
>  2 files changed, 162 insertions(+)
> 
> diff --git a/Documentation/thermal/sysfs-api.txt b/Documentation/thermal/sysfs-api.txt
> index fc7dfe10778b..7d44d7f1a71b 100644
> --- a/Documentation/thermal/sysfs-api.txt
> +++ b/Documentation/thermal/sysfs-api.txt
> @@ -184,6 +184,12 @@ Thermal zone device sys I/F, created once it's registered:
>      |---trip_point_[0-*]_type:	Trip point type
>      |---trip_point_[0-*]_hyst:	Hysteresis value for this trip point
>      |---emul_temp:		Emulated temperature set node
> +    |---sustainable_power:      Sustainable dissipatable power
> +    |---k_po:                   Proportional term during temperature overshoot
> +    |---k_pu:                   Proportional term during temperature undershoot
> +    |---k_i:                    PID's integral term in the power allocator gov
> +    |---k_d:                    PID's derivative term in the power allocator
> +    |---integral_cutoff:        Offset above which errors are accumulated

Can this be under a specific directory?

I thought of something like
/sys/class/thermal/thermal_zoneX/governor_params/<gov spec files>

The above node can be handled by the governor code.


>  
>  Thermal cooling device sys I/F, created once it's registered:
>  /sys/class/thermal/cooling_device[0-*]:
> @@ -307,6 +313,52 @@ emul_temp
>  	  because userland can easily disable the thermal policy by simply
>  	  flooding this sysfs node with low temperature values.
>  
> +sustainable_power
> +	An estimate of the sustained power that can be dissipated by
> +	the thermal zone. Used by the power allocator governor. For
> +	more information see Documentation/thermal/power_allocator.txt
> +	Unit: milliwatts
> +	RW, Optional
> +
> +k_po
> +	The proportional term of the power allocator governor's PID
> +	controller during temperature overshoot. Temperature overshoot
> +	is when the current temperature is above the "desired
> +	temperature" trip point. For more information see
> +	Documentation/thermal/power_allocator.txt
> +	RW, Optional
> +
> +k_pu
> +	The proportional term of the power allocator governor's PID
> +	controller during temperature undershoot. Temperature undershoot
> +	is when the current temperature is below the "desired
> +	temperature" trip point. For more information see
> +	Documentation/thermal/power_allocator.txt
> +	RW, Optional
> +
> +k_i
> +	The integral term of the power allocator governor's PID
> +	controller. This term allows the PID controller to compensate
> +	for long term drift. For more information see
> +	Documentation/thermal/power_allocator.txt
> +	RW, Optional
> +
> +k_d
> +	The derivative term of the power allocator governor's PID
> +	controller. For more information see
> +	Documentation/thermal/power_allocator.txt
> +	RW, Optional
> +
> +integral_cutoff
> +	Temperature offset from the desired temperature trip point
> +	above which the integral term of the power allocator
> +	governor's PID controller starts accumulating errors. For
> +	example, if integral_cutoff is 0, then the integral term only
> +	accumulates error when temperature is above the desired
> +	temperature trip point. For more information see
> +	Documentation/thermal/power_allocator.txt
> +	RW, Optional
> +
>  *****************************
>  * Cooling device attributes *
>  *****************************
> diff --git a/drivers/thermal/thermal_core.c b/drivers/thermal/thermal_core.c
> index df9ba3bf55dc..228e93f8a146 100644
> --- a/drivers/thermal/thermal_core.c
> +++ b/drivers/thermal/thermal_core.c
> @@ -873,6 +873,111 @@ emul_temp_store(struct device *dev, struct device_attribute *attr,
>  static DEVICE_ATTR(emul_temp, S_IWUSR, NULL, emul_temp_store);
>  #endif/*CONFIG_THERMAL_EMULATION*/
>  
> +#ifdef CONFIG_THERMAL_GOV_POWER_ALLOCATOR
> +
> +static ssize_t
> +sustainable_power_show(struct device *dev, struct device_attribute *devattr,
> +		       char *buf)
> +{
> +	struct thermal_zone_device *tz = to_thermal_zone(dev);
> +
> +	if (tz->tzp)
> +		return sprintf(buf, "%u\n", tz->tzp->sustainable_power);
> +	else
> +		return -EIO;
> +}
> +
> +static ssize_t
> +sustainable_power_store(struct device *dev, struct device_attribute *devattr,
> +			const char *buf, size_t count)
> +{
> +	struct thermal_zone_device *tz = to_thermal_zone(dev);
> +	u32 sustainable_power;
> +
> +	if (!tz->tzp)
> +		return -EIO;
> +
> +	if (kstrtou32(buf, 10, &sustainable_power))
> +		return -EINVAL;
> +
> +	tz->tzp->sustainable_power = sustainable_power;
> +
> +	return count;
> +}
> +static DEVICE_ATTR(sustainable_power, S_IWUSR | S_IRUGO, sustainable_power_show,
> +		sustainable_power_store);
> +
> +#define create_s32_tzp_attr(name)					\
> +	static ssize_t							\
> +	name##_show(struct device *dev, struct device_attribute *devattr, \
> +		char *buf)						\
> +	{								\
> +	struct thermal_zone_device *tz = to_thermal_zone(dev);		\
> +									\
> +	if (tz->tzp)							\
> +		return sprintf(buf, "%u\n", tz->tzp->name);		\
> +	else								\
> +		return -EIO;						\
> +	}								\
> +									\
> +	static ssize_t							\
> +	name##_store(struct device *dev, struct device_attribute *devattr, \
> +		const char *buf, size_t count)				\
> +	{								\
> +		struct thermal_zone_device *tz = to_thermal_zone(dev);	\
> +		s32 value;						\
> +									\
> +		if (!tz->tzp)						\
> +			return -EIO;					\
> +									\
> +		if (kstrtos32(buf, 10, &value))				\
> +			return -EINVAL;					\
> +									\
> +		tz->tzp->name = value;					\
> +									\
> +		return count;						\
> +	}								\
> +	static DEVICE_ATTR(name, S_IWUSR | S_IRUGO, name##_show, name##_store)
> +
> +create_s32_tzp_attr(k_po);
> +create_s32_tzp_attr(k_pu);
> +create_s32_tzp_attr(k_i);
> +create_s32_tzp_attr(k_d);
> +create_s32_tzp_attr(integral_cutoff);
> +#undef create_s32_tzp_attr
> +
> +static struct device_attribute *dev_tzp_attrs[] = {
> +	&dev_attr_sustainable_power,
> +	&dev_attr_k_po,
> +	&dev_attr_k_pu,
> +	&dev_attr_k_i,
> +	&dev_attr_k_d,
> +	&dev_attr_integral_cutoff,
> +};
> +
> +static int create_power_allocator_tzp_attrs(struct device *dev)
> +{
> +	int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(dev_tzp_attrs); i++) {
> +		int ret;
> +		struct device_attribute *dev_attr = dev_tzp_attrs[i];
> +
> +		ret = device_create_file(dev, dev_attr);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +#else  /* !CONFIG_THERMAL_GOV_POWER_ALLOCATOR */
> +static int create_power_allocator_tzp_attrs(struct device *dev)
> +{
> +	return 0;
> +}
> +#endif
> +

It is better if this code is not part of thermal_core. The governor
specific code must be in the governor code. Durga has done a pretty good
job splitting governor code out of thermal_core.c. I don't think we want
to get them back.


>  /**
>   * power_actor_get_max_power() - get the maximum power that a cdev can consume
>   * @cdev:	pointer to &thermal_cooling_device
> @@ -1712,6 +1817,11 @@ struct thermal_zone_device *thermal_zone_device_register(const char *type,
>  	if (result)
>  		goto unregister;
>  
> +	/* Add power_allocator specific thermal zone params */
> +	result = create_power_allocator_tzp_attrs(&tz->device);
> +	if (result)
> +		goto unregister;

Here you could create the governor_params and have a callback from
governor to populate it.

> +
>  	/* Update 'this' zone's governor information */
>  	mutex_lock(&thermal_governor_lock);
>  
> -- 
> 1.9.1
> 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v2 7/7] thermal: export thermal_zone_parameters to sysfs
  2015-02-26 21:30   ` Eduardo Valentin
@ 2015-02-26 22:04     ` Eduardo Valentin
  2015-02-27 17:19       ` Javi Merino
  0 siblings, 1 reply; 22+ messages in thread
From: Eduardo Valentin @ 2015-02-26 22:04 UTC (permalink / raw)
  To: Javi Merino
  Cc: rui.zhang, linux-pm, linux-kernel, punit.agrawal, broonie, tixy

[-- Attachment #1: Type: text/plain, Size: 9323 bytes --]

On Thu, Feb 26, 2015 at 05:30:00PM -0400, Eduardo Valentin wrote:
> On Thu, Feb 26, 2015 at 07:00:33PM +0000, Javi Merino wrote:
> > It's useful for tuning to be able to edit thermal_zone_parameters from
> > userspace.  Export them to the thermal_zone sysfs so that they can be
> > easily changed.
> > 
> > Cc: Zhang Rui <rui.zhang@intel.com>
> > Cc: Eduardo Valentin <edubezval@gmail.com>
> > Signed-off-by: Javi Merino <javi.merino@arm.com>
> > ---
> >  Documentation/thermal/sysfs-api.txt |  52 +++++++++++++++++
> >  drivers/thermal/thermal_core.c      | 110 ++++++++++++++++++++++++++++++++++++
> >  2 files changed, 162 insertions(+)
> > 
> > diff --git a/Documentation/thermal/sysfs-api.txt b/Documentation/thermal/sysfs-api.txt
> > index fc7dfe10778b..7d44d7f1a71b 100644
> > --- a/Documentation/thermal/sysfs-api.txt
> > +++ b/Documentation/thermal/sysfs-api.txt
> > @@ -184,6 +184,12 @@ Thermal zone device sys I/F, created once it's registered:
> >      |---trip_point_[0-*]_type:	Trip point type
> >      |---trip_point_[0-*]_hyst:	Hysteresis value for this trip point
> >      |---emul_temp:		Emulated temperature set node
> > +    |---sustainable_power:      Sustainable dissipatable power
> > +    |---k_po:                   Proportional term during temperature overshoot
> > +    |---k_pu:                   Proportional term during temperature undershoot
> > +    |---k_i:                    PID's integral term in the power allocator gov
> > +    |---k_d:                    PID's derivative term in the power allocator
> > +    |---integral_cutoff:        Offset above which errors are accumulated
> 
> Can this be under a specific directory?
> 
> I thought of something like
> /sys/class/thermal/thermal_zoneX/governor_params/<gov spec files>
> 
> The above node can be handled by the governor code.
> 
> 
> >  
> >  Thermal cooling device sys I/F, created once it's registered:
> >  /sys/class/thermal/cooling_device[0-*]:
> > @@ -307,6 +313,52 @@ emul_temp
> >  	  because userland can easily disable the thermal policy by simply
> >  	  flooding this sysfs node with low temperature values.
> >  
> > +sustainable_power
> > +	An estimate of the sustained power that can be dissipated by
> > +	the thermal zone. Used by the power allocator governor. For
> > +	more information see Documentation/thermal/power_allocator.txt
> > +	Unit: milliwatts
> > +	RW, Optional
> > +
> > +k_po
> > +	The proportional term of the power allocator governor's PID
> > +	controller during temperature overshoot. Temperature overshoot
> > +	is when the current temperature is above the "desired
> > +	temperature" trip point. For more information see
> > +	Documentation/thermal/power_allocator.txt
> > +	RW, Optional
> > +
> > +k_pu
> > +	The proportional term of the power allocator governor's PID
> > +	controller during temperature undershoot. Temperature undershoot
> > +	is when the current temperature is below the "desired
> > +	temperature" trip point. For more information see
> > +	Documentation/thermal/power_allocator.txt
> > +	RW, Optional
> > +
> > +k_i
> > +	The integral term of the power allocator governor's PID
> > +	controller. This term allows the PID controller to compensate
> > +	for long term drift. For more information see
> > +	Documentation/thermal/power_allocator.txt
> > +	RW, Optional
> > +
> > +k_d
> > +	The derivative term of the power allocator governor's PID
> > +	controller. For more information see
> > +	Documentation/thermal/power_allocator.txt
> > +	RW, Optional
> > +
> > +integral_cutoff
> > +	Temperature offset from the desired temperature trip point
> > +	above which the integral term of the power allocator
> > +	governor's PID controller starts accumulating errors. For
> > +	example, if integral_cutoff is 0, then the integral term only
> > +	accumulates error when temperature is above the desired
> > +	temperature trip point. For more information see
> > +	Documentation/thermal/power_allocator.txt
> > +	RW, Optional
> > +
> >  *****************************
> >  * Cooling device attributes *
> >  *****************************
> > diff --git a/drivers/thermal/thermal_core.c b/drivers/thermal/thermal_core.c
> > index df9ba3bf55dc..228e93f8a146 100644
> > --- a/drivers/thermal/thermal_core.c
> > +++ b/drivers/thermal/thermal_core.c
> > @@ -873,6 +873,111 @@ emul_temp_store(struct device *dev, struct device_attribute *attr,
> >  static DEVICE_ATTR(emul_temp, S_IWUSR, NULL, emul_temp_store);
> >  #endif/*CONFIG_THERMAL_EMULATION*/
> >  
> > +#ifdef CONFIG_THERMAL_GOV_POWER_ALLOCATOR


What bugged me in the first place was the fact that we are doing this
#ifdef here. But in fact, it is not really necessary, as the parameters
are stored in tzp, and they will be there regardless of the config
status.

> > +
> > +static ssize_t
> > +sustainable_power_show(struct device *dev, struct device_attribute *devattr,
> > +		       char *buf)
> > +{
> > +	struct thermal_zone_device *tz = to_thermal_zone(dev);
> > +
> > +	if (tz->tzp)
> > +		return sprintf(buf, "%u\n", tz->tzp->sustainable_power);
> > +	else
> > +		return -EIO;
> > +}
> > +
> > +static ssize_t
> > +sustainable_power_store(struct device *dev, struct device_attribute *devattr,
> > +			const char *buf, size_t count)
> > +{
> > +	struct thermal_zone_device *tz = to_thermal_zone(dev);
> > +	u32 sustainable_power;
> > +
> > +	if (!tz->tzp)
> > +		return -EIO;
> > +
> > +	if (kstrtou32(buf, 10, &sustainable_power))
> > +		return -EINVAL;
> > +
> > +	tz->tzp->sustainable_power = sustainable_power;
> > +
> > +	return count;
> > +}
> > +static DEVICE_ATTR(sustainable_power, S_IWUSR | S_IRUGO, sustainable_power_show,
> > +		sustainable_power_store);
> > +
> > +#define create_s32_tzp_attr(name)					\
> > +	static ssize_t							\
> > +	name##_show(struct device *dev, struct device_attribute *devattr, \
> > +		char *buf)						\
> > +	{								\
> > +	struct thermal_zone_device *tz = to_thermal_zone(dev);		\
> > +									\
> > +	if (tz->tzp)							\
> > +		return sprintf(buf, "%u\n", tz->tzp->name);		\
> > +	else								\
> > +		return -EIO;						\
> > +	}								\
> > +									\
> > +	static ssize_t							\
> > +	name##_store(struct device *dev, struct device_attribute *devattr, \
> > +		const char *buf, size_t count)				\
> > +	{								\
> > +		struct thermal_zone_device *tz = to_thermal_zone(dev);	\
> > +		s32 value;						\
> > +									\
> > +		if (!tz->tzp)						\
> > +			return -EIO;					\
> > +									\
> > +		if (kstrtos32(buf, 10, &value))				\
> > +			return -EINVAL;					\
> > +									\
> > +		tz->tzp->name = value;					\
> > +									\
> > +		return count;						\
> > +	}								\
> > +	static DEVICE_ATTR(name, S_IWUSR | S_IRUGO, name##_show, name##_store)
> > +
> > +create_s32_tzp_attr(k_po);
> > +create_s32_tzp_attr(k_pu);
> > +create_s32_tzp_attr(k_i);
> > +create_s32_tzp_attr(k_d);
> > +create_s32_tzp_attr(integral_cutoff);
> > +#undef create_s32_tzp_attr
> > +
> > +static struct device_attribute *dev_tzp_attrs[] = {
> > +	&dev_attr_sustainable_power,
> > +	&dev_attr_k_po,
> > +	&dev_attr_k_pu,
> > +	&dev_attr_k_i,
> > +	&dev_attr_k_d,
> > +	&dev_attr_integral_cutoff,
> > +};
> > +
> > +static int create_power_allocator_tzp_attrs(struct device *dev)

I would rename this to thermal_create_zone_params_attrs and remove the
ifdefiry. If you are not exposing the complete info under tzp, then make
a comment about it.

> > +{
> > +	int i;
> > +
> > +	for (i = 0; i < ARRAY_SIZE(dev_tzp_attrs); i++) {
> > +		int ret;
> > +		struct device_attribute *dev_attr = dev_tzp_attrs[i];
> > +
> > +		ret = device_create_file(dev, dev_attr);
> > +		if (ret)
> > +			return ret;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +#else  /* !CONFIG_THERMAL_GOV_POWER_ALLOCATOR */
> > +static int create_power_allocator_tzp_attrs(struct device *dev)
> > +{
> > +	return 0;
> > +}
> > +#endif
> > +
> 
> It is better if this code is not part of thermal_core. The governor
> specific code must be in the governor code. Durga has done a pretty good
> job splitting governor code out of thermal_core.c. I don't think we want
> to get them back.

Unless you really want to expose something that is only inside the
governor data struture, you can ignore the above comment.

> 
> 
> >  /**
> >   * power_actor_get_max_power() - get the maximum power that a cdev can consume
> >   * @cdev:	pointer to &thermal_cooling_device
> > @@ -1712,6 +1817,11 @@ struct thermal_zone_device *thermal_zone_device_register(const char *type,
> >  	if (result)
> >  		goto unregister;
> >  
> > +	/* Add power_allocator specific thermal zone params */
> > +	result = create_power_allocator_tzp_attrs(&tz->device);
> > +	if (result)
> > +		goto unregister;
> 
> Here you could create the governor_params and have a callback from
> governor to populate it.

ditto.

Of course, assuming all we are doing is exposing what is in
thermal_zone_params, which is generic enough to be in thermal_core.c.
> 
> > +
> >  	/* Update 'this' zone's governor information */
> >  	mutex_lock(&thermal_governor_lock);
> >  
> > -- 
> > 1.9.1
> > 



[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v2 7/7] thermal: export thermal_zone_parameters to sysfs
  2015-02-27 17:19       ` Javi Merino
@ 2015-02-26 22:10         ` Eduardo Valentin
  2015-03-02 14:17           ` Javi Merino
  0 siblings, 1 reply; 22+ messages in thread
From: Eduardo Valentin @ 2015-02-26 22:10 UTC (permalink / raw)
  To: Javi Merino
  Cc: rui.zhang, linux-pm, linux-kernel, Punit Agrawal, broonie, tixy

[-- Attachment #1: Type: text/plain, Size: 11684 bytes --]

On Fri, Feb 27, 2015 at 05:19:05PM +0000, Javi Merino wrote:
> On Thu, Feb 26, 2015 at 10:04:24PM +0000, Eduardo Valentin wrote:
> > On Thu, Feb 26, 2015 at 05:30:00PM -0400, Eduardo Valentin wrote:
> > > On Thu, Feb 26, 2015 at 07:00:33PM +0000, Javi Merino wrote:
> > > > It's useful for tuning to be able to edit thermal_zone_parameters from
> > > > userspace.  Export them to the thermal_zone sysfs so that they can be
> > > > easily changed.
> > > > 
> > > > Cc: Zhang Rui <rui.zhang@intel.com>
> > > > Cc: Eduardo Valentin <edubezval@gmail.com>
> > > > Signed-off-by: Javi Merino <javi.merino@arm.com>
> > > > ---
> > > >  Documentation/thermal/sysfs-api.txt |  52 +++++++++++++++++
> > > >  drivers/thermal/thermal_core.c      | 110 ++++++++++++++++++++++++++++++++++++
> > > >  2 files changed, 162 insertions(+)
> > > > 
> > > > diff --git a/Documentation/thermal/sysfs-api.txt b/Documentation/thermal/sysfs-api.txt
> > > > index fc7dfe10778b..7d44d7f1a71b 100644
> > > > --- a/Documentation/thermal/sysfs-api.txt
> > > > +++ b/Documentation/thermal/sysfs-api.txt
> > > > @@ -184,6 +184,12 @@ Thermal zone device sys I/F, created once it's registered:
> > > >      |---trip_point_[0-*]_type:	Trip point type
> > > >      |---trip_point_[0-*]_hyst:	Hysteresis value for this trip point
> > > >      |---emul_temp:		Emulated temperature set node
> > > > +    |---sustainable_power:      Sustainable dissipatable power
> > > > +    |---k_po:                   Proportional term during temperature overshoot
> > > > +    |---k_pu:                   Proportional term during temperature undershoot
> > > > +    |---k_i:                    PID's integral term in the power allocator gov
> > > > +    |---k_d:                    PID's derivative term in the power allocator
> > > > +    |---integral_cutoff:        Offset above which errors are accumulated
> > > 
> > > Can this be under a specific directory?
> > > 
> > > I thought of something like
> > > /sys/class/thermal/thermal_zoneX/governor_params/<gov spec files>
> > > 
> > > The above node can be handled by the governor code.
> 
> I thought about doing that, but creating a sysfs directory was a lot
> of boilerplate code that I thought didn't bring us anything.  That's
> why I avoided it.

OK. I see.

> 
> > > >  Thermal cooling device sys I/F, created once it's registered:
> > > >  /sys/class/thermal/cooling_device[0-*]:
> > > > @@ -307,6 +313,52 @@ emul_temp
> > > >  	  because userland can easily disable the thermal policy by simply
> > > >  	  flooding this sysfs node with low temperature values.
> > > >  
> > > > +sustainable_power
> > > > +	An estimate of the sustained power that can be dissipated by
> > > > +	the thermal zone. Used by the power allocator governor. For
> > > > +	more information see Documentation/thermal/power_allocator.txt
> > > > +	Unit: milliwatts
> > > > +	RW, Optional
> > > > +
> > > > +k_po
> > > > +	The proportional term of the power allocator governor's PID
> > > > +	controller during temperature overshoot. Temperature overshoot
> > > > +	is when the current temperature is above the "desired
> > > > +	temperature" trip point. For more information see
> > > > +	Documentation/thermal/power_allocator.txt
> > > > +	RW, Optional
> > > > +
> > > > +k_pu
> > > > +	The proportional term of the power allocator governor's PID
> > > > +	controller during temperature undershoot. Temperature undershoot
> > > > +	is when the current temperature is below the "desired
> > > > +	temperature" trip point. For more information see
> > > > +	Documentation/thermal/power_allocator.txt
> > > > +	RW, Optional
> > > > +
> > > > +k_i
> > > > +	The integral term of the power allocator governor's PID
> > > > +	controller. This term allows the PID controller to compensate
> > > > +	for long term drift. For more information see
> > > > +	Documentation/thermal/power_allocator.txt
> > > > +	RW, Optional
> > > > +
> > > > +k_d
> > > > +	The derivative term of the power allocator governor's PID
> > > > +	controller. For more information see
> > > > +	Documentation/thermal/power_allocator.txt
> > > > +	RW, Optional
> > > > +
> > > > +integral_cutoff
> > > > +	Temperature offset from the desired temperature trip point
> > > > +	above which the integral term of the power allocator
> > > > +	governor's PID controller starts accumulating errors. For
> > > > +	example, if integral_cutoff is 0, then the integral term only
> > > > +	accumulates error when temperature is above the desired
> > > > +	temperature trip point. For more information see
> > > > +	Documentation/thermal/power_allocator.txt
> > > > +	RW, Optional
> > > > +
> > > >  *****************************
> > > >  * Cooling device attributes *
> > > >  *****************************
> > > > diff --git a/drivers/thermal/thermal_core.c b/drivers/thermal/thermal_core.c
> > > > index df9ba3bf55dc..228e93f8a146 100644
> > > > --- a/drivers/thermal/thermal_core.c
> > > > +++ b/drivers/thermal/thermal_core.c
> > > > @@ -873,6 +873,111 @@ emul_temp_store(struct device *dev, struct device_attribute *attr,
> > > >  static DEVICE_ATTR(emul_temp, S_IWUSR, NULL, emul_temp_store);
> > > >  #endif/*CONFIG_THERMAL_EMULATION*/
> > > >  
> > > > +#ifdef CONFIG_THERMAL_GOV_POWER_ALLOCATOR
> > 
> > 
> > What bugged me in the first place was the fact that we are doing this
> > #ifdef here. But in fact, it is not really necessary, as the parameters
> > are stored in tzp, and they will be there regardless of the config
> > status.
> 
> My reasoning behind the ifdefs was because it seems we are putting a
> number of files in sysfs that are not interesting if this was not
> compiled in.  It's true that technically the ifdefs are not needed.
> I'll remove them.

OK. Good.

> 
> > > > +
> > > > +static ssize_t
> > > > +sustainable_power_show(struct device *dev, struct device_attribute *devattr,
> > > > +		       char *buf)
> > > > +{
> > > > +	struct thermal_zone_device *tz = to_thermal_zone(dev);
> > > > +
> > > > +	if (tz->tzp)
> > > > +		return sprintf(buf, "%u\n", tz->tzp->sustainable_power);
> > > > +	else
> > > > +		return -EIO;
> > > > +}
> > > > +
> > > > +static ssize_t
> > > > +sustainable_power_store(struct device *dev, struct device_attribute *devattr,
> > > > +			const char *buf, size_t count)
> > > > +{
> > > > +	struct thermal_zone_device *tz = to_thermal_zone(dev);
> > > > +	u32 sustainable_power;
> > > > +
> > > > +	if (!tz->tzp)
> > > > +		return -EIO;
> > > > +
> > > > +	if (kstrtou32(buf, 10, &sustainable_power))
> > > > +		return -EINVAL;
> > > > +
> > > > +	tz->tzp->sustainable_power = sustainable_power;
> > > > +
> > > > +	return count;
> > > > +}
> > > > +static DEVICE_ATTR(sustainable_power, S_IWUSR | S_IRUGO, sustainable_power_show,
> > > > +		sustainable_power_store);
> > > > +
> > > > +#define create_s32_tzp_attr(name)					\
> > > > +	static ssize_t							\
> > > > +	name##_show(struct device *dev, struct device_attribute *devattr, \
> > > > +		char *buf)						\
> > > > +	{								\
> > > > +	struct thermal_zone_device *tz = to_thermal_zone(dev);		\
> > > > +									\
> > > > +	if (tz->tzp)							\
> > > > +		return sprintf(buf, "%u\n", tz->tzp->name);		\
> > > > +	else								\
> > > > +		return -EIO;						\
> > > > +	}								\
> > > > +									\
> > > > +	static ssize_t							\
> > > > +	name##_store(struct device *dev, struct device_attribute *devattr, \
> > > > +		const char *buf, size_t count)				\
> > > > +	{								\
> > > > +		struct thermal_zone_device *tz = to_thermal_zone(dev);	\
> > > > +		s32 value;						\
> > > > +									\
> > > > +		if (!tz->tzp)						\
> > > > +			return -EIO;					\
> > > > +									\
> > > > +		if (kstrtos32(buf, 10, &value))				\
> > > > +			return -EINVAL;					\
> > > > +									\
> > > > +		tz->tzp->name = value;					\
> > > > +									\
> > > > +		return count;						\
> > > > +	}								\
> > > > +	static DEVICE_ATTR(name, S_IWUSR | S_IRUGO, name##_show, name##_store)
> > > > +
> > > > +create_s32_tzp_attr(k_po);
> > > > +create_s32_tzp_attr(k_pu);
> > > > +create_s32_tzp_attr(k_i);
> > > > +create_s32_tzp_attr(k_d);
> > > > +create_s32_tzp_attr(integral_cutoff);
> > > > +#undef create_s32_tzp_attr
> > > > +
> > > > +static struct device_attribute *dev_tzp_attrs[] = {
> > > > +	&dev_attr_sustainable_power,
> > > > +	&dev_attr_k_po,
> > > > +	&dev_attr_k_pu,
> > > > +	&dev_attr_k_i,
> > > > +	&dev_attr_k_d,
> > > > +	&dev_attr_integral_cutoff,
> > > > +};
> > > > +
> > > > +static int create_power_allocator_tzp_attrs(struct device *dev)
> > 
> > I would rename this to thermal_create_zone_params_attrs and remove the
> > ifdefiry.
> 
> Ok, I will remove them.
> 
> >           If you are not exposing the complete info under tzp, then make
> > a comment about it.
> 
> There's nothing else worth populating.  The governor_name is not
> interesting, you have the thermal zone policy for that.  Similar for
> no_hwmon.  For tbps, weight and trip_mask are already exposed in the
> instance.  The binding_limits is currently not available in sysfs, but
> if we were to populate it, I would expose the ones that end up in the
> instance, as we do with trips and weights.
> 
> Is it worth putting this in a comment in the code?


Well, I was more interested in governor specific / internal data,
accessible only in the governor file (e.g.: power_allocator.c). Do you
think we need to expose something that is accessible only from the
governor code or do you think the current info under tzp is enough?


If the former is the case, then we need restructure with callbacks. If
not, meaning, if all you need is in tzp, then we can keep the code in
thermal core.

> 
> > > > +{
> > > > +	int i;
> > > > +
> > > > +	for (i = 0; i < ARRAY_SIZE(dev_tzp_attrs); i++) {
> > > > +		int ret;
> > > > +		struct device_attribute *dev_attr = dev_tzp_attrs[i];
> > > > +
> > > > +		ret = device_create_file(dev, dev_attr);
> > > > +		if (ret)
> > > > +			return ret;
> > > > +	}
> > > > +
> > > > +	return 0;
> > > > +}
> > > > +
> > > > +#else  /* !CONFIG_THERMAL_GOV_POWER_ALLOCATOR */
> > > > +static int create_power_allocator_tzp_attrs(struct device *dev)
> > > > +{
> > > > +	return 0;
> > > > +}
> > > > +#endif
> > > > +
> > > 
> > > It is better if this code is not part of thermal_core. The governor
> > > specific code must be in the governor code. Durga has done a pretty good
> > > job splitting governor code out of thermal_core.c. I don't think we want
> > > to get them back.
> > 
> > Unless you really want to expose something that is only inside the
> > governor data struture, you can ignore the above comment.
> 
> Ok
> 
> > > >  /**
> > > >   * power_actor_get_max_power() - get the maximum power that a cdev can consume
> > > >   * @cdev:	pointer to &thermal_cooling_device
> > > > @@ -1712,6 +1817,11 @@ struct thermal_zone_device *thermal_zone_device_register(const char *type,
> > > >  	if (result)
> > > >  		goto unregister;
> > > >  
> > > > +	/* Add power_allocator specific thermal zone params */
> > > > +	result = create_power_allocator_tzp_attrs(&tz->device);
> > > > +	if (result)
> > > > +		goto unregister;
> > > 
> > > Here you could create the governor_params and have a callback from
> > > governor to populate it.
> > 
> > ditto.
> > 
> > Of course, assuming all we are doing is exposing what is in
> > thermal_zone_params, which is generic enough to be in thermal_core.c.
> 
> That's right.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v2 0/7] The power allocator thermal governor
  2015-02-26 19:00 [PATCH v2 0/7] The power allocator thermal governor Javi Merino
                   ` (6 preceding siblings ...)
  2015-02-26 19:00 ` [PATCH v2 7/7] thermal: export thermal_zone_parameters to sysfs Javi Merino
@ 2015-02-26 22:16 ` Eduardo Valentin
  2015-03-02 16:28   ` Javi Merino
  7 siblings, 1 reply; 22+ messages in thread
From: Eduardo Valentin @ 2015-02-26 22:16 UTC (permalink / raw)
  To: Javi Merino
  Cc: rui.zhang, linux-pm, linux-kernel, punit.agrawal, broonie, tixy

[-- Attachment #1: Type: text/plain, Size: 5324 bytes --]

On Thu, Feb 26, 2015 at 07:00:26PM +0000, Javi Merino wrote:
> Hi linux-pm,
> 
> The power allocator governor allocates device power to control
> temperature.  This requires transforming performance requests into
> requested power, which we do with an extended cooling device API
> introduced in patch 2 (thermal: extend the cooling device API to
> include power information).  Patch 3 (thermal: cpu_cooling: implement
> the power cooling device API) extends the cpu cooling device using a
> simple power model.
> 
> This series are based on branch "linus" of Eduardo's linux-soc-thermal
> tree:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/evalenti/linux-soc-thermal.git
> 
> Changes since v1:
>   - Removed a memory leak pointed out by Steve Rostedt
>   - Relax the constraint of the trip points to be the first and last
>     passive trip point as Lina Iyer suggested.
>   - Update how the governor treats weights to mimic the behavior of
>     fair share in v3 of the weight fixes series.
>   - Don't cache the cpufreq_cooling_device and scan for it whenever
>     we need it.
>   - Consider the rate of changes in the derivative term of the PID
>     controller
> 
> Changes since RFC v6:
>   - Addressed Eduardo's review
>     + Pass the interval to the static power function as suggested by
>       Eduardo
>     + Make the cooling device ops return 0 or -E* and put the
>       calculation in a parameter, like the rest of the cooling device
>       ops
>     + Documentation improvements
>   - Use thermal_cdev_update() to change cooling device states
>   - Add a patch to export the power allocator governor's tzp
>     parameters to sysfs
> 
> Changes since RFC v5:
>   - Addressed Stephen's review of the trace patches.
>   - Removed power actors and extended the cooling device interface
>     instead.
>   - Let platforms override the power allocator governor parameters in
>     their thermal zone parameters
> 
> Changes since RFC v4:
>   - Add more tracing
>   - Document some of the limitations of the power allocator governor
>   - Export the power_actor API and move power_actor.h to include/linux
> 
> Changes since RFC v3:
>   - Use tz->passive to poll faster when the first trip point is hit.
>   - Don't make a special directory for power_actors
>   - Add a DT property for sustainable-power
>   - Simplify the static power interface and pass the current thermal
>     zone in every power_actor_ops to remove the controversial
>     enum power_actor_types
>   - Use locks with the actor_list list
>   - Use cpufreq_get() to get the frequency of the cpu instead of
>     using the notifiers.
>   - Remove the prompt for THERMAL_POWER_ACTOR_CPU when configuring
>     the kernel
> 
> Changes since RFC v2:
>   - Changed the PI controller into a PID controller
>   - Added static power to the cpu power model
>   - tz parameter max_dissipatable_power renamed to sustainable_power
>   - Register the cpufreq cooling device as part of the
>     power_cpu_actor registration.
> 
> Changes since RFC v1:
>   - Fixed finding cpufreq cooling devices in cpufreq_frequency_change()
>   - Replaced the cooling device interface with a separate power actor
>     API
>   - Addressed most of Eduardo's comments
>   - Incorporated ftrace support for bitmask to trace cpumasks
> 
> Cheers,
> Javi & Punit
> 
> Javi Merino (6):
>   thermal: let governors have private data for each thermal zone
>   thermal: extend the cooling device API to include power information
>   thermal: cpu_cooling: implement the power cooling device API
>   thermal: introduce the Power Allocator governor
>   thermal: add trace events to the power allocator governor
>   thermal: export thermal_zone_parameters to sysfs

I am applying patches 1-3 and 6. Patches 4 and 7 need extra effort, but are
minor changes. Patch 5 depends on 4, so, cannot be applied now.

> 
> Punit Agrawal (1):
>   of: thermal: Introduce sustainable power for a thermal zone
> 
>  .../devicetree/bindings/thermal/thermal.txt        |   9 +
>  Documentation/thermal/cpu-cooling-api.txt          | 156 +++++-
>  Documentation/thermal/power_allocator.txt          | 247 +++++++++
>  Documentation/thermal/sysfs-api.txt                |  52 ++
>  drivers/thermal/Kconfig                            |  15 +
>  drivers/thermal/Makefile                           |   1 +
>  drivers/thermal/cpu_cooling.c                      | 610 ++++++++++++++++++++-
>  drivers/thermal/of-thermal.c                       |   4 +
>  drivers/thermal/power_allocator.c                  | 533 ++++++++++++++++++
>  drivers/thermal/thermal_core.c                     | 254 ++++++++-
>  drivers/thermal/thermal_core.h                     |   8 +
>  include/linux/cpu_cooling.h                        |  39 ++
>  include/linux/thermal.h                            |  64 ++-
>  include/trace/events/thermal.h                     |  58 ++
>  include/trace/events/thermal_power_allocator.h     |  87 +++
>  15 files changed, 2104 insertions(+), 33 deletions(-)
>  create mode 100644 Documentation/thermal/power_allocator.txt
>  create mode 100644 drivers/thermal/power_allocator.c
>  create mode 100644 include/trace/events/thermal_power_allocator.h
> 
> -- 
> 1.9.1
> 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v2 5/7] thermal: add trace events to the power allocator governor
  2015-02-26 19:46     ` Eduardo Valentin
@ 2015-02-26 22:52       ` Steven Rostedt
  2015-02-26 21:18         ` Eduardo Valentin
  0 siblings, 1 reply; 22+ messages in thread
From: Steven Rostedt @ 2015-02-26 22:52 UTC (permalink / raw)
  To: Eduardo Valentin
  Cc: Javi Merino, rui.zhang, linux-pm, linux-kernel, punit.agrawal,
	broonie, tixy, Frederic Weisbecker, Ingo Molnar

On Thu, 26 Feb 2015 15:46:34 -0400
Eduardo Valentin <edubezval@gmail.com> wrote:

 
> I am planing to apply this series to my thermal soc tree. Do you have
> any concerns if I send this patch, which touches the tracing files, via
> my tree?
> 

Which tracing files?

-- Steve

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v2 5/7] thermal: add trace events to the power allocator governor
  2015-02-26 21:18         ` Eduardo Valentin
@ 2015-02-27  0:09           ` Steven Rostedt
  0 siblings, 0 replies; 22+ messages in thread
From: Steven Rostedt @ 2015-02-27  0:09 UTC (permalink / raw)
  To: Eduardo Valentin
  Cc: Javi Merino, rui.zhang, linux-pm, linux-kernel, punit.agrawal,
	broonie, tixy, Frederic Weisbecker, Ingo Molnar

On Thu, 26 Feb 2015 17:18:25 -0400
Eduardo Valentin <edubezval@gmail.com> wrote:

> Steve,
> 
> On Thu, Feb 26, 2015 at 05:52:59PM -0500, Steven Rostedt wrote:
> > On Thu, 26 Feb 2015 15:46:34 -0400
> > Eduardo Valentin <edubezval@gmail.com> wrote:
> > 
> >  
> > > I am planing to apply this series to my thermal soc tree. Do you have
> > > any concerns if I send this patch, which touches the tracing files, via
> > > my tree?
> > > 
> > 
> > Which tracing files?
> 
> I am referring to:
> include/trace/events/thermal.h                 | 58 +++++++++++++++++
> include/trace/events/thermal_power_allocator.h | 87
> ++++++++++++++++++++++++++
> 
> I know they are new files anyway, but just checking if those can go
> altogether via the thermal tree.

The individual files in include/trace/events belong more to the
maintainers of where the tracepoints are located. I just monitor them
to make sure they are not too whacky ;-)

Feel free to incorporate them in your tree.

If you want...

Acked-by: Steven Rostedt <rostedt@goodmis.org>

-- Steve

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v2 4/7] thermal: introduce the Power Allocator governor
  2015-02-26 20:36   ` Eduardo Valentin
@ 2015-02-27 16:12     ` Javi Merino
  0 siblings, 0 replies; 22+ messages in thread
From: Javi Merino @ 2015-02-27 16:12 UTC (permalink / raw)
  To: Eduardo Valentin
  Cc: rui.zhang, linux-pm, linux-kernel, Punit Agrawal, broonie, tixy

On Thu, Feb 26, 2015 at 08:36:16PM +0000, Eduardo Valentin wrote:
> Javi,
> 
> Replying in v2.
> 
> On Thu, Feb 26, 2015 at 07:00:30PM +0000, Javi Merino wrote:
> 
> <big cut>
> 
> > +
> > +/**
> > + * divvy_up_power() - divvy the allocated power between the actors
> > + * @req_power:	each actor's requested power
> > + * @max_power:	each actor's maximum available power
> > + * @num_actors:	size of the @req_power, @max_power and @granted_power's array
> > + * @total_req_power: sum of @req_power
> > + * @power_range:	total allocated power
> > + * @granted_power:	output array: each actor's granted power
> > + *
> > + * This function divides the total allocated power (@power_range)
> > + * fairly between the actors.  It first tries to give each actor a
> > + * share of the @power_range according to how much power it requested
> > + * compared to the rest of the actors.  For example, if only one actor
> > + * requests power, then it receives all the @power_range.  If
> > + * three actors each requests 1mW, each receives a third of the
> > + * @power_range.
> > + *
> > + * If any actor received more than their maximum power, then that
> > + * surplus is re-divvied among the actors based on how far they are
> > + * from their respective maximums.
> > + *
> > + * Granted power for each actor is written to @granted_power, which
> > + * should've been allocated by the calling function.
> > + */
> > +static void divvy_up_power(u32 *req_power, u32 *max_power, int num_actors,
> > +			   u32 total_req_power, u32 power_range,
> > +			   u32 *granted_power)
> > +{
> > +	u32 extra_power, capped_extra_power, extra_actor_power[num_actors];
> 
> Just now I noticed this variable-size array. I know the number of actors
> are constant, and in average it should be less than 10 in magnitude, but
> I don't believe variable-sized arrays are a good practice in kernel.
> 
> Even though I don't see this one as a treat, can we avoid it?

I can turn it into a devm_kcalloc() and merge it with the other
three allocations that happen earlier.  I'll do that for v3.

Cheers,
Javi

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v2 7/7] thermal: export thermal_zone_parameters to sysfs
  2015-02-26 22:04     ` Eduardo Valentin
@ 2015-02-27 17:19       ` Javi Merino
  2015-02-26 22:10         ` Eduardo Valentin
  0 siblings, 1 reply; 22+ messages in thread
From: Javi Merino @ 2015-02-27 17:19 UTC (permalink / raw)
  To: Eduardo Valentin
  Cc: rui.zhang, linux-pm, linux-kernel, Punit Agrawal, broonie, tixy

On Thu, Feb 26, 2015 at 10:04:24PM +0000, Eduardo Valentin wrote:
> On Thu, Feb 26, 2015 at 05:30:00PM -0400, Eduardo Valentin wrote:
> > On Thu, Feb 26, 2015 at 07:00:33PM +0000, Javi Merino wrote:
> > > It's useful for tuning to be able to edit thermal_zone_parameters from
> > > userspace.  Export them to the thermal_zone sysfs so that they can be
> > > easily changed.
> > > 
> > > Cc: Zhang Rui <rui.zhang@intel.com>
> > > Cc: Eduardo Valentin <edubezval@gmail.com>
> > > Signed-off-by: Javi Merino <javi.merino@arm.com>
> > > ---
> > >  Documentation/thermal/sysfs-api.txt |  52 +++++++++++++++++
> > >  drivers/thermal/thermal_core.c      | 110 ++++++++++++++++++++++++++++++++++++
> > >  2 files changed, 162 insertions(+)
> > > 
> > > diff --git a/Documentation/thermal/sysfs-api.txt b/Documentation/thermal/sysfs-api.txt
> > > index fc7dfe10778b..7d44d7f1a71b 100644
> > > --- a/Documentation/thermal/sysfs-api.txt
> > > +++ b/Documentation/thermal/sysfs-api.txt
> > > @@ -184,6 +184,12 @@ Thermal zone device sys I/F, created once it's registered:
> > >      |---trip_point_[0-*]_type:	Trip point type
> > >      |---trip_point_[0-*]_hyst:	Hysteresis value for this trip point
> > >      |---emul_temp:		Emulated temperature set node
> > > +    |---sustainable_power:      Sustainable dissipatable power
> > > +    |---k_po:                   Proportional term during temperature overshoot
> > > +    |---k_pu:                   Proportional term during temperature undershoot
> > > +    |---k_i:                    PID's integral term in the power allocator gov
> > > +    |---k_d:                    PID's derivative term in the power allocator
> > > +    |---integral_cutoff:        Offset above which errors are accumulated
> > 
> > Can this be under a specific directory?
> > 
> > I thought of something like
> > /sys/class/thermal/thermal_zoneX/governor_params/<gov spec files>
> > 
> > The above node can be handled by the governor code.

I thought about doing that, but creating a sysfs directory was a lot
of boilerplate code that I thought didn't bring us anything.  That's
why I avoided it.

> > >  Thermal cooling device sys I/F, created once it's registered:
> > >  /sys/class/thermal/cooling_device[0-*]:
> > > @@ -307,6 +313,52 @@ emul_temp
> > >  	  because userland can easily disable the thermal policy by simply
> > >  	  flooding this sysfs node with low temperature values.
> > >  
> > > +sustainable_power
> > > +	An estimate of the sustained power that can be dissipated by
> > > +	the thermal zone. Used by the power allocator governor. For
> > > +	more information see Documentation/thermal/power_allocator.txt
> > > +	Unit: milliwatts
> > > +	RW, Optional
> > > +
> > > +k_po
> > > +	The proportional term of the power allocator governor's PID
> > > +	controller during temperature overshoot. Temperature overshoot
> > > +	is when the current temperature is above the "desired
> > > +	temperature" trip point. For more information see
> > > +	Documentation/thermal/power_allocator.txt
> > > +	RW, Optional
> > > +
> > > +k_pu
> > > +	The proportional term of the power allocator governor's PID
> > > +	controller during temperature undershoot. Temperature undershoot
> > > +	is when the current temperature is below the "desired
> > > +	temperature" trip point. For more information see
> > > +	Documentation/thermal/power_allocator.txt
> > > +	RW, Optional
> > > +
> > > +k_i
> > > +	The integral term of the power allocator governor's PID
> > > +	controller. This term allows the PID controller to compensate
> > > +	for long term drift. For more information see
> > > +	Documentation/thermal/power_allocator.txt
> > > +	RW, Optional
> > > +
> > > +k_d
> > > +	The derivative term of the power allocator governor's PID
> > > +	controller. For more information see
> > > +	Documentation/thermal/power_allocator.txt
> > > +	RW, Optional
> > > +
> > > +integral_cutoff
> > > +	Temperature offset from the desired temperature trip point
> > > +	above which the integral term of the power allocator
> > > +	governor's PID controller starts accumulating errors. For
> > > +	example, if integral_cutoff is 0, then the integral term only
> > > +	accumulates error when temperature is above the desired
> > > +	temperature trip point. For more information see
> > > +	Documentation/thermal/power_allocator.txt
> > > +	RW, Optional
> > > +
> > >  *****************************
> > >  * Cooling device attributes *
> > >  *****************************
> > > diff --git a/drivers/thermal/thermal_core.c b/drivers/thermal/thermal_core.c
> > > index df9ba3bf55dc..228e93f8a146 100644
> > > --- a/drivers/thermal/thermal_core.c
> > > +++ b/drivers/thermal/thermal_core.c
> > > @@ -873,6 +873,111 @@ emul_temp_store(struct device *dev, struct device_attribute *attr,
> > >  static DEVICE_ATTR(emul_temp, S_IWUSR, NULL, emul_temp_store);
> > >  #endif/*CONFIG_THERMAL_EMULATION*/
> > >  
> > > +#ifdef CONFIG_THERMAL_GOV_POWER_ALLOCATOR
> 
> 
> What bugged me in the first place was the fact that we are doing this
> #ifdef here. But in fact, it is not really necessary, as the parameters
> are stored in tzp, and they will be there regardless of the config
> status.

My reasoning behind the ifdefs was because it seems we are putting a
number of files in sysfs that are not interesting if this was not
compiled in.  It's true that technically the ifdefs are not needed.
I'll remove them.

> > > +
> > > +static ssize_t
> > > +sustainable_power_show(struct device *dev, struct device_attribute *devattr,
> > > +		       char *buf)
> > > +{
> > > +	struct thermal_zone_device *tz = to_thermal_zone(dev);
> > > +
> > > +	if (tz->tzp)
> > > +		return sprintf(buf, "%u\n", tz->tzp->sustainable_power);
> > > +	else
> > > +		return -EIO;
> > > +}
> > > +
> > > +static ssize_t
> > > +sustainable_power_store(struct device *dev, struct device_attribute *devattr,
> > > +			const char *buf, size_t count)
> > > +{
> > > +	struct thermal_zone_device *tz = to_thermal_zone(dev);
> > > +	u32 sustainable_power;
> > > +
> > > +	if (!tz->tzp)
> > > +		return -EIO;
> > > +
> > > +	if (kstrtou32(buf, 10, &sustainable_power))
> > > +		return -EINVAL;
> > > +
> > > +	tz->tzp->sustainable_power = sustainable_power;
> > > +
> > > +	return count;
> > > +}
> > > +static DEVICE_ATTR(sustainable_power, S_IWUSR | S_IRUGO, sustainable_power_show,
> > > +		sustainable_power_store);
> > > +
> > > +#define create_s32_tzp_attr(name)					\
> > > +	static ssize_t							\
> > > +	name##_show(struct device *dev, struct device_attribute *devattr, \
> > > +		char *buf)						\
> > > +	{								\
> > > +	struct thermal_zone_device *tz = to_thermal_zone(dev);		\
> > > +									\
> > > +	if (tz->tzp)							\
> > > +		return sprintf(buf, "%u\n", tz->tzp->name);		\
> > > +	else								\
> > > +		return -EIO;						\
> > > +	}								\
> > > +									\
> > > +	static ssize_t							\
> > > +	name##_store(struct device *dev, struct device_attribute *devattr, \
> > > +		const char *buf, size_t count)				\
> > > +	{								\
> > > +		struct thermal_zone_device *tz = to_thermal_zone(dev);	\
> > > +		s32 value;						\
> > > +									\
> > > +		if (!tz->tzp)						\
> > > +			return -EIO;					\
> > > +									\
> > > +		if (kstrtos32(buf, 10, &value))				\
> > > +			return -EINVAL;					\
> > > +									\
> > > +		tz->tzp->name = value;					\
> > > +									\
> > > +		return count;						\
> > > +	}								\
> > > +	static DEVICE_ATTR(name, S_IWUSR | S_IRUGO, name##_show, name##_store)
> > > +
> > > +create_s32_tzp_attr(k_po);
> > > +create_s32_tzp_attr(k_pu);
> > > +create_s32_tzp_attr(k_i);
> > > +create_s32_tzp_attr(k_d);
> > > +create_s32_tzp_attr(integral_cutoff);
> > > +#undef create_s32_tzp_attr
> > > +
> > > +static struct device_attribute *dev_tzp_attrs[] = {
> > > +	&dev_attr_sustainable_power,
> > > +	&dev_attr_k_po,
> > > +	&dev_attr_k_pu,
> > > +	&dev_attr_k_i,
> > > +	&dev_attr_k_d,
> > > +	&dev_attr_integral_cutoff,
> > > +};
> > > +
> > > +static int create_power_allocator_tzp_attrs(struct device *dev)
> 
> I would rename this to thermal_create_zone_params_attrs and remove the
> ifdefiry.

Ok, I will remove them.

>           If you are not exposing the complete info under tzp, then make
> a comment about it.

There's nothing else worth populating.  The governor_name is not
interesting, you have the thermal zone policy for that.  Similar for
no_hwmon.  For tbps, weight and trip_mask are already exposed in the
instance.  The binding_limits is currently not available in sysfs, but
if we were to populate it, I would expose the ones that end up in the
instance, as we do with trips and weights.

Is it worth putting this in a comment in the code?

> > > +{
> > > +	int i;
> > > +
> > > +	for (i = 0; i < ARRAY_SIZE(dev_tzp_attrs); i++) {
> > > +		int ret;
> > > +		struct device_attribute *dev_attr = dev_tzp_attrs[i];
> > > +
> > > +		ret = device_create_file(dev, dev_attr);
> > > +		if (ret)
> > > +			return ret;
> > > +	}
> > > +
> > > +	return 0;
> > > +}
> > > +
> > > +#else  /* !CONFIG_THERMAL_GOV_POWER_ALLOCATOR */
> > > +static int create_power_allocator_tzp_attrs(struct device *dev)
> > > +{
> > > +	return 0;
> > > +}
> > > +#endif
> > > +
> > 
> > It is better if this code is not part of thermal_core. The governor
> > specific code must be in the governor code. Durga has done a pretty good
> > job splitting governor code out of thermal_core.c. I don't think we want
> > to get them back.
> 
> Unless you really want to expose something that is only inside the
> governor data struture, you can ignore the above comment.

Ok

> > >  /**
> > >   * power_actor_get_max_power() - get the maximum power that a cdev can consume
> > >   * @cdev:	pointer to &thermal_cooling_device
> > > @@ -1712,6 +1817,11 @@ struct thermal_zone_device *thermal_zone_device_register(const char *type,
> > >  	if (result)
> > >  		goto unregister;
> > >  
> > > +	/* Add power_allocator specific thermal zone params */
> > > +	result = create_power_allocator_tzp_attrs(&tz->device);
> > > +	if (result)
> > > +		goto unregister;
> > 
> > Here you could create the governor_params and have a callback from
> > governor to populate it.
> 
> ditto.
> 
> Of course, assuming all we are doing is exposing what is in
> thermal_zone_params, which is generic enough to be in thermal_core.c.

That's right.

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v2 7/7] thermal: export thermal_zone_parameters to sysfs
  2015-02-26 22:10         ` Eduardo Valentin
@ 2015-03-02 14:17           ` Javi Merino
  0 siblings, 0 replies; 22+ messages in thread
From: Javi Merino @ 2015-03-02 14:17 UTC (permalink / raw)
  To: Eduardo Valentin
  Cc: rui.zhang, linux-pm, linux-kernel, Punit Agrawal, broonie, tixy

On Thu, Feb 26, 2015 at 10:10:07PM +0000, Eduardo Valentin wrote:
> On Fri, Feb 27, 2015 at 05:19:05PM +0000, Javi Merino wrote:
> > On Thu, Feb 26, 2015 at 10:04:24PM +0000, Eduardo Valentin wrote:
> > > On Thu, Feb 26, 2015 at 05:30:00PM -0400, Eduardo Valentin wrote:
> > > > On Thu, Feb 26, 2015 at 07:00:33PM +0000, Javi Merino wrote:

[snip]

> > > > > +
> > > > > +static ssize_t
> > > > > +sustainable_power_show(struct device *dev, struct device_attribute *devattr,
> > > > > +		       char *buf)
> > > > > +{
> > > > > +	struct thermal_zone_device *tz = to_thermal_zone(dev);
> > > > > +
> > > > > +	if (tz->tzp)
> > > > > +		return sprintf(buf, "%u\n", tz->tzp->sustainable_power);
> > > > > +	else
> > > > > +		return -EIO;
> > > > > +}
> > > > > +
> > > > > +static ssize_t
> > > > > +sustainable_power_store(struct device *dev, struct device_attribute *devattr,
> > > > > +			const char *buf, size_t count)
> > > > > +{
> > > > > +	struct thermal_zone_device *tz = to_thermal_zone(dev);
> > > > > +	u32 sustainable_power;
> > > > > +
> > > > > +	if (!tz->tzp)
> > > > > +		return -EIO;
> > > > > +
> > > > > +	if (kstrtou32(buf, 10, &sustainable_power))
> > > > > +		return -EINVAL;
> > > > > +
> > > > > +	tz->tzp->sustainable_power = sustainable_power;
> > > > > +
> > > > > +	return count;
> > > > > +}
> > > > > +static DEVICE_ATTR(sustainable_power, S_IWUSR | S_IRUGO, sustainable_power_show,
> > > > > +		sustainable_power_store);
> > > > > +
> > > > > +#define create_s32_tzp_attr(name)					\
> > > > > +	static ssize_t							\
> > > > > +	name##_show(struct device *dev, struct device_attribute *devattr, \
> > > > > +		char *buf)						\
> > > > > +	{								\
> > > > > +	struct thermal_zone_device *tz = to_thermal_zone(dev);		\
> > > > > +									\
> > > > > +	if (tz->tzp)							\
> > > > > +		return sprintf(buf, "%u\n", tz->tzp->name);		\
> > > > > +	else								\
> > > > > +		return -EIO;						\
> > > > > +	}								\
> > > > > +									\
> > > > > +	static ssize_t							\
> > > > > +	name##_store(struct device *dev, struct device_attribute *devattr, \
> > > > > +		const char *buf, size_t count)				\
> > > > > +	{								\
> > > > > +		struct thermal_zone_device *tz = to_thermal_zone(dev);	\
> > > > > +		s32 value;						\
> > > > > +									\
> > > > > +		if (!tz->tzp)						\
> > > > > +			return -EIO;					\
> > > > > +									\
> > > > > +		if (kstrtos32(buf, 10, &value))				\
> > > > > +			return -EINVAL;					\
> > > > > +									\
> > > > > +		tz->tzp->name = value;					\
> > > > > +									\
> > > > > +		return count;						\
> > > > > +	}								\
> > > > > +	static DEVICE_ATTR(name, S_IWUSR | S_IRUGO, name##_show, name##_store)
> > > > > +
> > > > > +create_s32_tzp_attr(k_po);
> > > > > +create_s32_tzp_attr(k_pu);
> > > > > +create_s32_tzp_attr(k_i);
> > > > > +create_s32_tzp_attr(k_d);
> > > > > +create_s32_tzp_attr(integral_cutoff);
> > > > > +#undef create_s32_tzp_attr
> > > > > +
> > > > > +static struct device_attribute *dev_tzp_attrs[] = {
> > > > > +	&dev_attr_sustainable_power,
> > > > > +	&dev_attr_k_po,
> > > > > +	&dev_attr_k_pu,
> > > > > +	&dev_attr_k_i,
> > > > > +	&dev_attr_k_d,
> > > > > +	&dev_attr_integral_cutoff,
> > > > > +};
> > > > > +
> > > > > +static int create_power_allocator_tzp_attrs(struct device *dev)
> > > 
> > > I would rename this to thermal_create_zone_params_attrs and remove the
> > > ifdefiry.
> > 
> > Ok, I will remove them.
> > 
> > >           If you are not exposing the complete info under tzp, then make
> > > a comment about it.
> > 
> > There's nothing else worth populating.  The governor_name is not
> > interesting, you have the thermal zone policy for that.  Similar for
> > no_hwmon.  For tbps, weight and trip_mask are already exposed in the
> > instance.  The binding_limits is currently not available in sysfs, but
> > if we were to populate it, I would expose the ones that end up in the
> > instance, as we do with trips and weights.
> > 
> > Is it worth putting this in a comment in the code?
> 
> 
> Well, I was more interested in governor specific / internal data,
> accessible only in the governor file (e.g.: power_allocator.c). Do you
> think we need to expose something that is accessible only from the
> governor code or do you think the current info under tzp is enough?
> 
> 
> If the former is the case, then we need restructure with callbacks. If
> not, meaning, if all you need is in tzp, then we can keep the code in
> thermal core.

I don't want to expose anything from the governor file, I think the
current info under tzp is enough.

Cheers,
Javi

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v2 0/7] The power allocator thermal governor
  2015-02-26 22:16 ` [PATCH v2 0/7] The power allocator thermal governor Eduardo Valentin
@ 2015-03-02 16:28   ` Javi Merino
  0 siblings, 0 replies; 22+ messages in thread
From: Javi Merino @ 2015-03-02 16:28 UTC (permalink / raw)
  To: Eduardo Valentin
  Cc: rui.zhang, linux-pm, linux-kernel, Punit Agrawal, broonie, tixy

On Thu, Feb 26, 2015 at 10:16:09PM +0000, Eduardo Valentin wrote:
> On Thu, Feb 26, 2015 at 07:00:26PM +0000, Javi Merino wrote:
> > Hi linux-pm,
> > 
> > The power allocator governor allocates device power to control
> > temperature.  This requires transforming performance requests into
> > requested power, which we do with an extended cooling device API
> > introduced in patch 2 (thermal: extend the cooling device API to
> > include power information).  Patch 3 (thermal: cpu_cooling: implement
> > the power cooling device API) extends the cpu cooling device using a
> > simple power model.
> > 
> > This series are based on branch "linus" of Eduardo's linux-soc-thermal
> > tree:
> > 
> > git://git.kernel.org/pub/scm/linux/kernel/git/evalenti/linux-soc-thermal.git
> > 
> > Changes since v1:
> >   - Removed a memory leak pointed out by Steve Rostedt
> >   - Relax the constraint of the trip points to be the first and last
> >     passive trip point as Lina Iyer suggested.
> >   - Update how the governor treats weights to mimic the behavior of
> >     fair share in v3 of the weight fixes series.
> >   - Don't cache the cpufreq_cooling_device and scan for it whenever
> >     we need it.
> >   - Consider the rate of changes in the derivative term of the PID
> >     controller
> > 
> > Changes since RFC v6:
> >   - Addressed Eduardo's review
> >     + Pass the interval to the static power function as suggested by
> >       Eduardo
> >     + Make the cooling device ops return 0 or -E* and put the
> >       calculation in a parameter, like the rest of the cooling device
> >       ops
> >     + Documentation improvements
> >   - Use thermal_cdev_update() to change cooling device states
> >   - Add a patch to export the power allocator governor's tzp
> >     parameters to sysfs
> > 
> > Changes since RFC v5:
> >   - Addressed Stephen's review of the trace patches.
> >   - Removed power actors and extended the cooling device interface
> >     instead.
> >   - Let platforms override the power allocator governor parameters in
> >     their thermal zone parameters
> > 
> > Changes since RFC v4:
> >   - Add more tracing
> >   - Document some of the limitations of the power allocator governor
> >   - Export the power_actor API and move power_actor.h to include/linux
> > 
> > Changes since RFC v3:
> >   - Use tz->passive to poll faster when the first trip point is hit.
> >   - Don't make a special directory for power_actors
> >   - Add a DT property for sustainable-power
> >   - Simplify the static power interface and pass the current thermal
> >     zone in every power_actor_ops to remove the controversial
> >     enum power_actor_types
> >   - Use locks with the actor_list list
> >   - Use cpufreq_get() to get the frequency of the cpu instead of
> >     using the notifiers.
> >   - Remove the prompt for THERMAL_POWER_ACTOR_CPU when configuring
> >     the kernel
> > 
> > Changes since RFC v2:
> >   - Changed the PI controller into a PID controller
> >   - Added static power to the cpu power model
> >   - tz parameter max_dissipatable_power renamed to sustainable_power
> >   - Register the cpufreq cooling device as part of the
> >     power_cpu_actor registration.
> > 
> > Changes since RFC v1:
> >   - Fixed finding cpufreq cooling devices in cpufreq_frequency_change()
> >   - Replaced the cooling device interface with a separate power actor
> >     API
> >   - Addressed most of Eduardo's comments
> >   - Incorporated ftrace support for bitmask to trace cpumasks
> > 
> > Cheers,
> > Javi & Punit
> > 
> > Javi Merino (6):
> >   thermal: let governors have private data for each thermal zone
> >   thermal: extend the cooling device API to include power information
> >   thermal: cpu_cooling: implement the power cooling device API
> >   thermal: introduce the Power Allocator governor
> >   thermal: add trace events to the power allocator governor
> >   thermal: export thermal_zone_parameters to sysfs
> 
> I am applying patches 1-3 and 6.

Patch 6 depends on patch 4, as it accesses tzp->sustainable_power
which is introduced in patch 4.  Can you unapply it and keep the
order?

> Patches 4 and 7 need extra effort, but are
> minor changes. Patch 5 depends on 4, so, cannot be applied now.

I'll send a v3 with the minor changes based on your linus branch later
today.

Cheers,
Javi

^ permalink raw reply	[flat|nested] 22+ messages in thread

end of thread, other threads:[~2015-03-02 16:28 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-02-26 19:00 [PATCH v2 0/7] The power allocator thermal governor Javi Merino
2015-02-26 19:00 ` [PATCH v2 1/7] thermal: let governors have private data for each thermal zone Javi Merino
2015-02-26 19:00 ` [PATCH v2 2/7] thermal: extend the cooling device API to include power information Javi Merino
2015-02-26 19:00 ` [PATCH v2 3/7] thermal: cpu_cooling: implement the power cooling device API Javi Merino
2015-02-26 19:00 ` [PATCH v2 4/7] thermal: introduce the Power Allocator governor Javi Merino
2015-02-26 20:36   ` Eduardo Valentin
2015-02-27 16:12     ` Javi Merino
2015-02-26 19:00 ` [PATCH v2 5/7] thermal: add trace events to the power allocator governor Javi Merino
2015-02-26 19:38   ` Steven Rostedt
2015-02-26 19:46     ` Eduardo Valentin
2015-02-26 22:52       ` Steven Rostedt
2015-02-26 21:18         ` Eduardo Valentin
2015-02-27  0:09           ` Steven Rostedt
2015-02-26 19:00 ` [PATCH v2 6/7] of: thermal: Introduce sustainable power for a thermal zone Javi Merino
2015-02-26 19:00 ` [PATCH v2 7/7] thermal: export thermal_zone_parameters to sysfs Javi Merino
2015-02-26 21:30   ` Eduardo Valentin
2015-02-26 22:04     ` Eduardo Valentin
2015-02-27 17:19       ` Javi Merino
2015-02-26 22:10         ` Eduardo Valentin
2015-03-02 14:17           ` Javi Merino
2015-02-26 22:16 ` [PATCH v2 0/7] The power allocator thermal governor Eduardo Valentin
2015-03-02 16:28   ` Javi Merino

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.