[PATCH v5 0/4] nvme: add thermal zone devices

[PATCH v5 0/4] nvme: add thermal zone devices

[Akinobu Mita]

The NVMe controller reports up to nine temperature values in the SMART /
Health log page (the composite temperature and temperature sensor 1 through
temperature sensor 8).
The temperature threshold feature (Feature Identifier 04h) configures the
asynchronous event request command to complete when the temperature is
crossed its corresponding temperature threshold.

This provides these temperatures and thresholds via thermal zone devices.

The main purpose of this is to turn on a fan when overheated without
polling the device for the smart log that could prevent the lower power
state transitions.

In addition to the standard thermal zone device, this adds support for
registering the DT thermal zone device.

* v5
- remove 'Export get and set features' patch as it has been merged in -next
- split the DT thermal zone support into separate patch
- round up temperature when writing trip point
- add DT binding documentation
- don't register both standard and DT thermal zone
- use thermal_zone_device_update() instead of thermal_notify_framework()

* v4
- add thermal.c to hold thermal zone related code
- add 'use_thermal_zone' module parameter
- add CONFIG_THERMAL_WRITABLE_TRIPS dependency
- add comment about the return value of nvme_thermal_zones_register()
- support DT thermal zone device.
- use bitmap to iterate over implemented sensors

* v3
- Change the type name of thermal zone devices from 'nvme_temp<sensor>' to
  'nvme<instance>_temp<sensor>'
- Pass a NULL to the status argument of nvme_set_feature()
- Change the name of symbolic link from 'nvme_temp<sensor>' to 'temp<sensor>'
- Don't make it fatal error if the device provides a response
- Don't register thermal zone for composite temperature if smart log reports
  zero value
- Move the thermal zones registration and unregistration into the core module.

* v2
- s/correspoinding/corresponding/ typo in commit log
- Borrowed nvme_get_features() from Keith's patch
- Temperature threshold notification is splitted into another patch
- Change the data type of 'sensor' to unsigned
- Add BUILD_BUG_ON for the array size of tzdev member in nvme_ctrl
- Add WARN_ON_ONCE for paranoid checks
- Fix off-by-one error in nvme_get_temp
- Validate 'sensor' where the value is actually used
- Define and utilize two enums related to the temperature threshold feature
- Remove hysteresis value for this trip point and don't utilize the under
  temperature threshold
- Print error message for thermal_zone_device_register() failure
- Add function comments for nvme_thermal_zones_{,un}register
- Suppress non-fatal errors from nvme_thermal_zones_register()
- Add comment about implemented temperature sensors 
- Instead of creating a new 'thermal_work', append async smart event's
  action to the existing async_event_work
- Add comment for tzdev member in nvme_ctrl
- Call nvme_thermal_zones_unregister() earlier than the last reference
  release

Akinobu Mita (4):
  nvme: add thermal zone devices
  dt-bindings: thermal: nvme: Add binding documentation
  nvme: support DT thermal zone device
  nvme: notify thermal framework when temperature threshold events occur

 Documentation/devicetree/bindings/thermal/nvme.txt |  56 ++++
 drivers/nvme/host/Kconfig                          |   1 +
 drivers/nvme/host/Makefile                         |   1 +
 drivers/nvme/host/core.c                           |  19 ++
 drivers/nvme/host/nvme.h                           |  40 +++
 drivers/nvme/host/thermal.c                        | 343 +++++++++++++++++++++
 include/linux/nvme.h                               |  12 +
 7 files changed, 472 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/thermal/nvme.txt
 create mode 100644 drivers/nvme/host/thermal.c

Cc: Rob Herring <robh@kernel.org>
Cc: Zhang Rui <rui.zhang@intel.com>
Cc: Eduardo Valentin <edubezval@gmail.com>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Keith Busch <kbusch@kernel.org>
Cc: Jens Axboe <axboe@fb.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagi@grimberg.me>
Cc: Minwoo Im <minwoo.im.dev@gmail.com>
Cc: Kenneth Heitke <kenneth.heitke@intel.com>
Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni@wdc.com>
-- 
2.7.4

[PATCH v5 0/4] nvme: add thermal zone devices

[Akinobu Mita]

The NVMe controller reports up to nine temperature values in the SMART /
Health log page (the composite temperature and temperature sensor 1 through
temperature sensor 8).
The temperature threshold feature (Feature Identifier 04h) configures the
asynchronous event request command to complete when the temperature is
crossed its corresponding temperature threshold.

This provides these temperatures and thresholds via thermal zone devices.

The main purpose of this is to turn on a fan when overheated without
polling the device for the smart log that could prevent the lower power
state transitions.

In addition to the standard thermal zone device, this adds support for
registering the DT thermal zone device.

* v5
- remove 'Export get and set features' patch as it has been merged in -next
- split the DT thermal zone support into separate patch
- round up temperature when writing trip point
- add DT binding documentation
- don't register both standard and DT thermal zone
- use thermal_zone_device_update() instead of thermal_notify_framework()

* v4
- add thermal.c to hold thermal zone related code
- add 'use_thermal_zone' module parameter
- add CONFIG_THERMAL_WRITABLE_TRIPS dependency
- add comment about the return value of nvme_thermal_zones_register()
- support DT thermal zone device.
- use bitmap to iterate over implemented sensors

* v3
- Change the type name of thermal zone devices from 'nvme_temp<sensor>' to
  'nvme<instance>_temp<sensor>'
- Pass a NULL to the status argument of nvme_set_feature()
- Change the name of symbolic link from 'nvme_temp<sensor>' to 'temp<sensor>'
- Don't make it fatal error if the device provides a response
- Don't register thermal zone for composite temperature if smart log reports
  zero value
- Move the thermal zones registration and unregistration into the core module.

* v2
- s/correspoinding/corresponding/ typo in commit log
- Borrowed nvme_get_features() from Keith's patch
- Temperature threshold notification is splitted into another patch
- Change the data type of 'sensor' to unsigned
- Add BUILD_BUG_ON for the array size of tzdev member in nvme_ctrl
- Add WARN_ON_ONCE for paranoid checks
- Fix off-by-one error in nvme_get_temp
- Validate 'sensor' where the value is actually used
- Define and utilize two enums related to the temperature threshold feature
- Remove hysteresis value for this trip point and don't utilize the under
  temperature threshold
- Print error message for thermal_zone_device_register() failure
- Add function comments for nvme_thermal_zones_{,un}register
- Suppress non-fatal errors from nvme_thermal_zones_register()
- Add comment about implemented temperature sensors 
- Instead of creating a new 'thermal_work', append async smart event's
  action to the existing async_event_work
- Add comment for tzdev member in nvme_ctrl
- Call nvme_thermal_zones_unregister() earlier than the last reference
  release

Akinobu Mita (4):
  nvme: add thermal zone devices
  dt-bindings: thermal: nvme: Add binding documentation
  nvme: support DT thermal zone device
  nvme: notify thermal framework when temperature threshold events occur

 Documentation/devicetree/bindings/thermal/nvme.txt |  56 ++++
 drivers/nvme/host/Kconfig                          |   1 +
 drivers/nvme/host/Makefile                         |   1 +
 drivers/nvme/host/core.c                           |  19 ++
 drivers/nvme/host/nvme.h                           |  40 +++
 drivers/nvme/host/thermal.c                        | 343 +++++++++++++++++++++
 include/linux/nvme.h                               |  12 +
 7 files changed, 472 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/thermal/nvme.txt
 create mode 100644 drivers/nvme/host/thermal.c

Cc: Rob Herring <robh at kernel.org>
Cc: Zhang Rui <rui.zhang at intel.com>
Cc: Eduardo Valentin <edubezval at gmail.com>
Cc: Daniel Lezcano <daniel.lezcano at linaro.org>
Cc: Keith Busch <kbusch at kernel.org>
Cc: Jens Axboe <axboe at fb.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Minwoo Im <minwoo.im.dev at gmail.com>
Cc: Kenneth Heitke <kenneth.heitke at intel.com>
Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni at wdc.com>
-- 
2.7.4

[PATCH v5 1/4] nvme: add thermal zone devices

[Akinobu Mita]

The NVMe controller reports up to nine temperature values in the SMART /
Health log page (the composite temperature and temperature sensor 1 through
temperature sensor 8).

This provides these temperatures via thermal zone devices.

Once the controller is identified, the thermal zone devices is created for
each implemented temperature sensors including the composite temperature.

/sys/class/thermal/thermal_zone[0-*]:
    |---type: 'nvme<instance>-temp<sensor>'
    |---temp: Temperature
    |---trip_point_0_temp: Over temperature threshold

The thermal_zone[0-*] contains a 'device' symlink to the corresponding nvme
device.

On the other hand, the following symlinks to the thermal zone devices are
created in the nvme device sysfs directory.

- temp0: Composite temperature
- temp1: Temperature sensor 1
...
- temp8: Temperature sensor 8

Cc: Rob Herring <robh@kernel.org>
Cc: Zhang Rui <rui.zhang@intel.com>
Cc: Eduardo Valentin <edubezval@gmail.com>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Keith Busch <kbusch@kernel.org>
Cc: Jens Axboe <axboe@fb.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagi@grimberg.me>
Cc: Minwoo Im <minwoo.im.dev@gmail.com>
Cc: Kenneth Heitke <kenneth.heitke@intel.com>
Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni@wdc.com>
Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
---
* v5
- split the DT thermal zone support into separate patch
- round up temperature when writing trip point

 drivers/nvme/host/Kconfig   |   1 +
 drivers/nvme/host/Makefile  |   1 +
 drivers/nvme/host/core.c    |   5 +
 drivers/nvme/host/nvme.h    |  34 ++++++
 drivers/nvme/host/thermal.c | 291 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/nvme.h        |   5 +
 6 files changed, 337 insertions(+)
 create mode 100644 drivers/nvme/host/thermal.c

diff --git a/drivers/nvme/host/Kconfig b/drivers/nvme/host/Kconfig
index ec43ac9..90429f8 100644
--- a/drivers/nvme/host/Kconfig
+++ b/drivers/nvme/host/Kconfig
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0-only
 config NVME_CORE
 	tristate
+	select THERMAL_WRITABLE_TRIPS if THERMAL
 
 config BLK_DEV_NVME
 	tristate "NVM Express block device"
diff --git a/drivers/nvme/host/Makefile b/drivers/nvme/host/Makefile
index 8a4b671..6018668 100644
--- a/drivers/nvme/host/Makefile
+++ b/drivers/nvme/host/Makefile
@@ -14,6 +14,7 @@ nvme-core-$(CONFIG_TRACING)		+= trace.o
 nvme-core-$(CONFIG_NVME_MULTIPATH)	+= multipath.o
 nvme-core-$(CONFIG_NVM)			+= lightnvm.o
 nvme-core-$(CONFIG_FAULT_INJECTION_DEBUG_FS)	+= fault_inject.o
+nvme-core-$(CONFIG_THERMAL)		+= thermal.o
 
 nvme-y					+= pci.o
 
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 179f7cf..1d84685 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2696,6 +2696,10 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 	if (ret < 0)
 		return ret;
 
+	ret = nvme_thermal_zones_register(ctrl);
+	if (ret)
+		return ret;
+
 	ctrl->identified = true;
 
 	return 0;
@@ -3699,6 +3703,7 @@ void nvme_stop_ctrl(struct nvme_ctrl *ctrl)
 {
 	nvme_mpath_stop(ctrl);
 	nvme_stop_keep_alive(ctrl);
+	nvme_thermal_zones_unregister(ctrl);
 	flush_work(&ctrl->async_event_work);
 	cancel_work_sync(&ctrl->fw_act_work);
 }
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index ea45d7d..49dd59ec 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -15,6 +15,7 @@
 #include <linux/sed-opal.h>
 #include <linux/fault-inject.h>
 #include <linux/rcupdate.h>
+#include <linux/thermal.h>
 
 extern unsigned int nvme_io_timeout;
 #define NVME_IO_TIMEOUT	(nvme_io_timeout * HZ)
@@ -155,6 +156,12 @@ struct nvme_fault_inject {
 #endif
 };
 
+struct nvme_tz {
+	struct thermal_zone_params params;
+	struct thermal_zone_device *dev;
+	unsigned int sensor;
+};
+
 struct nvme_ctrl {
 	bool comp_seen;
 	enum nvme_ctrl_state state;
@@ -258,6 +265,15 @@ struct nvme_ctrl {
 	unsigned long discard_page_busy;
 
 	struct nvme_fault_inject fault_inject;
+
+#ifdef CONFIG_THERMAL
+	/*
+	 * tz[0]: composite temperature
+	 * tz[1-8]: temperature sensor 1 through 8
+	 */
+	struct nvme_tz tz[9];
+	DECLARE_BITMAP(tz_enabled, 9);
+#endif
 };
 
 enum nvme_iopolicy {
@@ -590,4 +606,22 @@ static inline struct nvme_ns *nvme_get_ns_from_dev(struct device *dev)
 	return dev_to_disk(dev)->private_data;
 }
 
+#ifdef CONFIG_THERMAL
+
+int nvme_thermal_zones_register(struct nvme_ctrl *ctrl);
+void nvme_thermal_zones_unregister(struct nvme_ctrl *ctrl);
+
+#else
+
+static inline int nvme_thermal_zones_register(struct nvme_ctrl *ctrl)
+{
+	return 0;
+}
+
+static inline void nvme_thermal_zones_unregister(struct nvme_ctrl *ctrl)
+{
+}
+
+#endif /* CONFIG_THERMAL */
+
 #endif /* _NVME_H */
diff --git a/drivers/nvme/host/thermal.c b/drivers/nvme/host/thermal.c
new file mode 100644
index 0000000..c3608f6
--- /dev/null
+++ b/drivers/nvme/host/thermal.c
@@ -0,0 +1,291 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/moduleparam.h>
+#include <asm/unaligned.h>
+
+#include "nvme.h"
+
+static bool use_thermal_zone = true;
+module_param(use_thermal_zone, bool, 0444);
+MODULE_PARM_DESC(use_thermal_zone,
+	"Export nvme temperature via generic thermal interface");
+
+static int nvme_get_temp(struct nvme_ctrl *ctrl, unsigned int sensor, int *temp)
+{
+	struct nvme_smart_log *log;
+	int ret;
+
+	BUILD_BUG_ON(ARRAY_SIZE(log->temp_sensor) + 1 != ARRAY_SIZE(ctrl->tz));
+
+	if (WARN_ON_ONCE(sensor > ARRAY_SIZE(log->temp_sensor)))
+		return -EINVAL;
+
+	log = kzalloc(sizeof(*log), GFP_KERNEL);
+	if (!log)
+		return -ENOMEM;
+
+	ret = nvme_get_log(ctrl, NVME_NSID_ALL, NVME_LOG_SMART, 0, log,
+			   sizeof(*log), 0);
+	if (ret) {
+		ret = ret > 0 ? -EINVAL : ret;
+		goto free_log;
+	}
+
+	if (sensor)
+		*temp = le16_to_cpu(log->temp_sensor[sensor - 1]);
+	else
+		*temp = get_unaligned_le16(log->temperature);
+
+free_log:
+	kfree(log);
+
+	return ret;
+}
+
+#define KELVIN_TO_MILLICELSIUS(t) (((t) * 1000) - 273150)
+#define MILLICELSIUS_TO_KELVIN(t) (((t) + 273150) / 1000)
+#define MILLICELSIUS_TO_KELVIN_ROUNDUP(t) MILLICELSIUS_TO_KELVIN((t) + 999)
+
+static int nvme_tz_of_get_temp(void *data, int *temp)
+{
+	struct nvme_tz *tz = data;
+	unsigned int sensor = tz->sensor;
+	struct nvme_ctrl *ctrl = container_of(tz, struct nvme_ctrl, tz[sensor]);
+	int ret;
+
+	ret = nvme_get_temp(ctrl, sensor, temp);
+	if (!ret)
+		*temp = KELVIN_TO_MILLICELSIUS(*temp);
+
+	return ret;
+}
+
+static int nvme_tz_get_temp(struct thermal_zone_device *tzdev, int *temp)
+{
+	return nvme_tz_of_get_temp(tzdev->devdata, temp);
+}
+
+static int nvme_tz_get_trip_type(struct thermal_zone_device *tzdev,
+				 int trip, enum thermal_trip_type *type)
+{
+	*type = THERMAL_TRIP_ACTIVE;
+
+	return 0;
+}
+
+static int nvme_get_over_temp_thresh(struct nvme_ctrl *ctrl,
+				     unsigned int sensor, int *temp)
+{
+	unsigned int threshold = sensor << NVME_TEMP_THRESH_SELECT_SHIFT;
+	int status;
+	int ret;
+
+	if (WARN_ON_ONCE(sensor >= ARRAY_SIZE(ctrl->tz)))
+		return -EINVAL;
+
+	ret = nvme_get_features(ctrl, NVME_FEAT_TEMP_THRESH, threshold, NULL, 0,
+				&status);
+	if (!ret)
+		*temp = status & NVME_TEMP_THRESH_MASK;
+
+	return ret > 0 ? -EINVAL : ret;
+}
+
+static int nvme_set_over_temp_thresh(struct nvme_ctrl *ctrl,
+				     unsigned int sensor, int temp)
+{
+	unsigned int threshold = sensor << NVME_TEMP_THRESH_SELECT_SHIFT;
+	int ret;
+
+	if (WARN_ON_ONCE(sensor >= ARRAY_SIZE(ctrl->tz)))
+		return -EINVAL;
+
+	if (temp > NVME_TEMP_THRESH_MASK)
+		return -EINVAL;
+
+	threshold |= temp & NVME_TEMP_THRESH_MASK;
+
+	ret = nvme_set_features(ctrl, NVME_FEAT_TEMP_THRESH, threshold, NULL, 0,
+				NULL);
+
+	return ret > 0 ? -EINVAL : ret;
+}
+
+static int nvme_tz_get_trip_temp(struct thermal_zone_device *tzdev,
+				 int trip, int *temp)
+{
+	struct nvme_tz *tz = tzdev->devdata;
+	unsigned int sensor = tz->sensor;
+	struct nvme_ctrl *ctrl = container_of(tz, struct nvme_ctrl, tz[sensor]);
+	int ret;
+
+	ret = nvme_get_over_temp_thresh(ctrl, sensor, temp);
+	if (!ret)
+		*temp = KELVIN_TO_MILLICELSIUS(*temp);
+
+	return ret;
+}
+
+static int nvme_tz_of_set_trip_temp(void *data, int trip, int temp)
+{
+	struct nvme_tz *tz = data;
+	unsigned int sensor = tz->sensor;
+	struct nvme_ctrl *ctrl = container_of(tz, struct nvme_ctrl, tz[sensor]);
+
+	temp = MILLICELSIUS_TO_KELVIN_ROUNDUP(temp);
+
+	return nvme_set_over_temp_thresh(ctrl, sensor, temp);
+}
+
+static int nvme_tz_set_trip_temp(struct thermal_zone_device *tzdev,
+				 int trip, int temp)
+{
+	return nvme_tz_of_set_trip_temp(tzdev->devdata, trip, temp);
+}
+
+static struct thermal_zone_device_ops nvme_tz_ops = {
+	.get_temp = nvme_tz_get_temp,
+	.get_trip_type = nvme_tz_get_trip_type,
+	.get_trip_temp = nvme_tz_get_trip_temp,
+	.set_trip_temp = nvme_tz_set_trip_temp,
+};
+
+static const struct thermal_zone_params nvme_tz_params = {
+	.governor_name = "user_space",
+	.no_hwmon = true,
+};
+
+static int nvme_thermal_zone_register(struct nvme_ctrl *ctrl,
+				      unsigned int sensor)
+{
+	struct thermal_zone_device *tzdev;
+	struct nvme_tz *tz = &ctrl->tz[sensor];
+	char name[THERMAL_NAME_LENGTH];
+	int ret;
+
+	tz->sensor = sensor;
+	tz->params = nvme_tz_params;
+	snprintf(name, sizeof(name), "nvme%d_temp%u", ctrl->instance, sensor);
+
+	tzdev = thermal_zone_device_register(name, 1, 1, tz, &nvme_tz_ops,
+					     &tz->params, 0, 0);
+	if (IS_ERR(tzdev)) {
+		dev_err(ctrl->device,
+			"Failed to register thermal zone device: %ld\n",
+			PTR_ERR(tzdev));
+		return PTR_ERR(tzdev);
+	}
+
+	snprintf(name, sizeof(name), "temp%d", sensor);
+	ret = sysfs_create_link(&ctrl->ctrl_device.kobj, &tzdev->device.kobj,
+				name);
+	if (ret)
+		goto device_unregister;
+
+	ret = sysfs_create_link(&tzdev->device.kobj,
+				&ctrl->ctrl_device.kobj, "device");
+	if (ret)
+		goto remove_sensor_link;
+
+	tz->dev = tzdev;
+
+	return 0;
+
+remove_sensor_link:
+	sysfs_remove_link(&ctrl->ctrl_device.kobj, name);
+device_unregister:
+	thermal_zone_device_unregister(tzdev);
+
+	return ret;
+}
+
+/**
+ * nvme_thermal_zones_register() - register nvme thermal zone devices
+ * @ctrl: controller instance
+ *
+ * This function creates up to nine thermal zone devices for all implemented
+ * temperature sensors including the composite temperature.
+ * Each thermal zone device provides a single trip point temperature that is
+ * associated with an over temperature threshold.
+ * A negative value is returned when the device doesn't respond to the host
+ * and won't be able to continue initialization.
+ */
+int nvme_thermal_zones_register(struct nvme_ctrl *ctrl)
+{
+	struct nvme_smart_log *log;
+	int ret = 0;
+	int err;
+	int i;
+
+	if (!use_thermal_zone)
+		return 0;
+
+	log = kzalloc(sizeof(*log), GFP_KERNEL);
+	if (!log)
+		return 0;
+
+	err = nvme_get_log(ctrl, NVME_NSID_ALL, NVME_LOG_SMART, 0, log,
+			   sizeof(*log), 0);
+	if (err) {
+		dev_err(ctrl->device, "Failed to get SMART log: %d\n", err);
+		/* If the device provided a response, then it's non-fatal */
+		ret = (err > 0) ? 0 : err;
+		goto out;
+	}
+
+	for_each_clear_bit(i, ctrl->tz_enabled, ARRAY_SIZE(ctrl->tz)) {
+		int temp;
+
+		if (i)
+			temp = le16_to_cpu(log->temp_sensor[i - 1]);
+		else
+			temp = get_unaligned_le16(log->temperature);
+
+		/*
+		 * All implemented temperature sensors report a non-zero value
+		 * in temperature sensor fields in the smart log page.
+		 */
+		if (!temp)
+			continue;
+
+		err = nvme_thermal_zone_register(ctrl, i);
+		if (err)
+			goto out;
+
+		__set_bit(i, ctrl->tz_enabled);
+	}
+out:
+	if (err)
+		nvme_thermal_zones_unregister(ctrl);
+
+	kfree(log);
+
+	return ret;
+}
+
+/**
+ * nvme_thermal_zones_unregister() - unregister nvme thermal zone devices
+ * @ctrl: controller instance
+ *
+ * This function removes the registered thermal zone devices and symlinks.
+ */
+void nvme_thermal_zones_unregister(struct nvme_ctrl *ctrl)
+{
+	int i;
+
+	for_each_set_bit(i, ctrl->tz_enabled, ARRAY_SIZE(ctrl->tz)) {
+		struct nvme_tz *tz = &ctrl->tz[i];
+
+		if (tz->dev) {
+			char name[20];
+
+			sysfs_remove_link(&tz->dev->device.kobj, "device");
+			snprintf(name, sizeof(name), "temp%d", i);
+			sysfs_remove_link(&ctrl->ctrl_device.kobj, name);
+			thermal_zone_device_unregister(tz->dev);
+			tz->dev = NULL;
+		}
+
+		__clear_bit(i, ctrl->tz_enabled);
+	}
+}
diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index 8028ada..f29728b 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -749,6 +749,11 @@ struct nvme_write_zeroes_cmd {
 
 /* Features */
 
+enum {
+	NVME_TEMP_THRESH_MASK		= 0xffff,
+	NVME_TEMP_THRESH_SELECT_SHIFT	= 16,
+};
+
 struct nvme_feat_auto_pst {
 	__le64 entries[32];
 };
-- 
2.7.4

[PATCH v5 1/4] nvme: add thermal zone devices

[Akinobu Mita]

The NVMe controller reports up to nine temperature values in the SMART /
Health log page (the composite temperature and temperature sensor 1 through
temperature sensor 8).

This provides these temperatures via thermal zone devices.

Once the controller is identified, the thermal zone devices is created for
each implemented temperature sensors including the composite temperature.

/sys/class/thermal/thermal_zone[0-*]:
    |---type: 'nvme<instance>-temp<sensor>'
    |---temp: Temperature
    |---trip_point_0_temp: Over temperature threshold

The thermal_zone[0-*] contains a 'device' symlink to the corresponding nvme
device.

On the other hand, the following symlinks to the thermal zone devices are
created in the nvme device sysfs directory.

- temp0: Composite temperature
- temp1: Temperature sensor 1
...
- temp8: Temperature sensor 8

Cc: Rob Herring <robh at kernel.org>
Cc: Zhang Rui <rui.zhang at intel.com>
Cc: Eduardo Valentin <edubezval at gmail.com>
Cc: Daniel Lezcano <daniel.lezcano at linaro.org>
Cc: Keith Busch <kbusch at kernel.org>
Cc: Jens Axboe <axboe at fb.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Minwoo Im <minwoo.im.dev at gmail.com>
Cc: Kenneth Heitke <kenneth.heitke at intel.com>
Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni at wdc.com>
Signed-off-by: Akinobu Mita <akinobu.mita at gmail.com>
---
* v5
- split the DT thermal zone support into separate patch
- round up temperature when writing trip point

 drivers/nvme/host/Kconfig   |   1 +
 drivers/nvme/host/Makefile  |   1 +
 drivers/nvme/host/core.c    |   5 +
 drivers/nvme/host/nvme.h    |  34 ++++++
 drivers/nvme/host/thermal.c | 291 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/nvme.h        |   5 +
 6 files changed, 337 insertions(+)
 create mode 100644 drivers/nvme/host/thermal.c

diff --git a/drivers/nvme/host/Kconfig b/drivers/nvme/host/Kconfig
index ec43ac9..90429f8 100644
--- a/drivers/nvme/host/Kconfig
+++ b/drivers/nvme/host/Kconfig
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0-only
 config NVME_CORE
 	tristate
+	select THERMAL_WRITABLE_TRIPS if THERMAL
 
 config BLK_DEV_NVME
 	tristate "NVM Express block device"
diff --git a/drivers/nvme/host/Makefile b/drivers/nvme/host/Makefile
index 8a4b671..6018668 100644
--- a/drivers/nvme/host/Makefile
+++ b/drivers/nvme/host/Makefile
@@ -14,6 +14,7 @@ nvme-core-$(CONFIG_TRACING)		+= trace.o
 nvme-core-$(CONFIG_NVME_MULTIPATH)	+= multipath.o
 nvme-core-$(CONFIG_NVM)			+= lightnvm.o
 nvme-core-$(CONFIG_FAULT_INJECTION_DEBUG_FS)	+= fault_inject.o
+nvme-core-$(CONFIG_THERMAL)		+= thermal.o
 
 nvme-y					+= pci.o
 
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 179f7cf..1d84685 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2696,6 +2696,10 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 	if (ret < 0)
 		return ret;
 
+	ret = nvme_thermal_zones_register(ctrl);
+	if (ret)
+		return ret;
+
 	ctrl->identified = true;
 
 	return 0;
@@ -3699,6 +3703,7 @@ void nvme_stop_ctrl(struct nvme_ctrl *ctrl)
 {
 	nvme_mpath_stop(ctrl);
 	nvme_stop_keep_alive(ctrl);
+	nvme_thermal_zones_unregister(ctrl);
 	flush_work(&ctrl->async_event_work);
 	cancel_work_sync(&ctrl->fw_act_work);
 }
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index ea45d7d..49dd59ec 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -15,6 +15,7 @@
 #include <linux/sed-opal.h>
 #include <linux/fault-inject.h>
 #include <linux/rcupdate.h>
+#include <linux/thermal.h>
 
 extern unsigned int nvme_io_timeout;
 #define NVME_IO_TIMEOUT	(nvme_io_timeout * HZ)
@@ -155,6 +156,12 @@ struct nvme_fault_inject {
 #endif
 };
 
+struct nvme_tz {
+	struct thermal_zone_params params;
+	struct thermal_zone_device *dev;
+	unsigned int sensor;
+};
+
 struct nvme_ctrl {
 	bool comp_seen;
 	enum nvme_ctrl_state state;
@@ -258,6 +265,15 @@ struct nvme_ctrl {
 	unsigned long discard_page_busy;
 
 	struct nvme_fault_inject fault_inject;
+
+#ifdef CONFIG_THERMAL
+	/*
+	 * tz[0]: composite temperature
+	 * tz[1-8]: temperature sensor 1 through 8
+	 */
+	struct nvme_tz tz[9];
+	DECLARE_BITMAP(tz_enabled, 9);
+#endif
 };
 
 enum nvme_iopolicy {
@@ -590,4 +606,22 @@ static inline struct nvme_ns *nvme_get_ns_from_dev(struct device *dev)
 	return dev_to_disk(dev)->private_data;
 }
 
+#ifdef CONFIG_THERMAL
+
+int nvme_thermal_zones_register(struct nvme_ctrl *ctrl);
+void nvme_thermal_zones_unregister(struct nvme_ctrl *ctrl);
+
+#else
+
+static inline int nvme_thermal_zones_register(struct nvme_ctrl *ctrl)
+{
+	return 0;
+}
+
+static inline void nvme_thermal_zones_unregister(struct nvme_ctrl *ctrl)
+{
+}
+
+#endif /* CONFIG_THERMAL */
+
 #endif /* _NVME_H */
diff --git a/drivers/nvme/host/thermal.c b/drivers/nvme/host/thermal.c
new file mode 100644
index 0000000..c3608f6
--- /dev/null
+++ b/drivers/nvme/host/thermal.c
@@ -0,0 +1,291 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <linux/moduleparam.h>
+#include <asm/unaligned.h>
+
+#include "nvme.h"
+
+static bool use_thermal_zone = true;
+module_param(use_thermal_zone, bool, 0444);
+MODULE_PARM_DESC(use_thermal_zone,
+	"Export nvme temperature via generic thermal interface");
+
+static int nvme_get_temp(struct nvme_ctrl *ctrl, unsigned int sensor, int *temp)
+{
+	struct nvme_smart_log *log;
+	int ret;
+
+	BUILD_BUG_ON(ARRAY_SIZE(log->temp_sensor) + 1 != ARRAY_SIZE(ctrl->tz));
+
+	if (WARN_ON_ONCE(sensor > ARRAY_SIZE(log->temp_sensor)))
+		return -EINVAL;
+
+	log = kzalloc(sizeof(*log), GFP_KERNEL);
+	if (!log)
+		return -ENOMEM;
+
+	ret = nvme_get_log(ctrl, NVME_NSID_ALL, NVME_LOG_SMART, 0, log,
+			   sizeof(*log), 0);
+	if (ret) {
+		ret = ret > 0 ? -EINVAL : ret;
+		goto free_log;
+	}
+
+	if (sensor)
+		*temp = le16_to_cpu(log->temp_sensor[sensor - 1]);
+	else
+		*temp = get_unaligned_le16(log->temperature);
+
+free_log:
+	kfree(log);
+
+	return ret;
+}
+
+#define KELVIN_TO_MILLICELSIUS(t) (((t) * 1000) - 273150)
+#define MILLICELSIUS_TO_KELVIN(t) (((t) + 273150) / 1000)
+#define MILLICELSIUS_TO_KELVIN_ROUNDUP(t) MILLICELSIUS_TO_KELVIN((t) + 999)
+
+static int nvme_tz_of_get_temp(void *data, int *temp)
+{
+	struct nvme_tz *tz = data;
+	unsigned int sensor = tz->sensor;
+	struct nvme_ctrl *ctrl = container_of(tz, struct nvme_ctrl, tz[sensor]);
+	int ret;
+
+	ret = nvme_get_temp(ctrl, sensor, temp);
+	if (!ret)
+		*temp = KELVIN_TO_MILLICELSIUS(*temp);
+
+	return ret;
+}
+
+static int nvme_tz_get_temp(struct thermal_zone_device *tzdev, int *temp)
+{
+	return nvme_tz_of_get_temp(tzdev->devdata, temp);
+}
+
+static int nvme_tz_get_trip_type(struct thermal_zone_device *tzdev,
+				 int trip, enum thermal_trip_type *type)
+{
+	*type = THERMAL_TRIP_ACTIVE;
+
+	return 0;
+}
+
+static int nvme_get_over_temp_thresh(struct nvme_ctrl *ctrl,
+				     unsigned int sensor, int *temp)
+{
+	unsigned int threshold = sensor << NVME_TEMP_THRESH_SELECT_SHIFT;
+	int status;
+	int ret;
+
+	if (WARN_ON_ONCE(sensor >= ARRAY_SIZE(ctrl->tz)))
+		return -EINVAL;
+
+	ret = nvme_get_features(ctrl, NVME_FEAT_TEMP_THRESH, threshold, NULL, 0,
+				&status);
+	if (!ret)
+		*temp = status & NVME_TEMP_THRESH_MASK;
+
+	return ret > 0 ? -EINVAL : ret;
+}
+
+static int nvme_set_over_temp_thresh(struct nvme_ctrl *ctrl,
+				     unsigned int sensor, int temp)
+{
+	unsigned int threshold = sensor << NVME_TEMP_THRESH_SELECT_SHIFT;
+	int ret;
+
+	if (WARN_ON_ONCE(sensor >= ARRAY_SIZE(ctrl->tz)))
+		return -EINVAL;
+
+	if (temp > NVME_TEMP_THRESH_MASK)
+		return -EINVAL;
+
+	threshold |= temp & NVME_TEMP_THRESH_MASK;
+
+	ret = nvme_set_features(ctrl, NVME_FEAT_TEMP_THRESH, threshold, NULL, 0,
+				NULL);
+
+	return ret > 0 ? -EINVAL : ret;
+}
+
+static int nvme_tz_get_trip_temp(struct thermal_zone_device *tzdev,
+				 int trip, int *temp)
+{
+	struct nvme_tz *tz = tzdev->devdata;
+	unsigned int sensor = tz->sensor;
+	struct nvme_ctrl *ctrl = container_of(tz, struct nvme_ctrl, tz[sensor]);
+	int ret;
+
+	ret = nvme_get_over_temp_thresh(ctrl, sensor, temp);
+	if (!ret)
+		*temp = KELVIN_TO_MILLICELSIUS(*temp);
+
+	return ret;
+}
+
+static int nvme_tz_of_set_trip_temp(void *data, int trip, int temp)
+{
+	struct nvme_tz *tz = data;
+	unsigned int sensor = tz->sensor;
+	struct nvme_ctrl *ctrl = container_of(tz, struct nvme_ctrl, tz[sensor]);
+
+	temp = MILLICELSIUS_TO_KELVIN_ROUNDUP(temp);
+
+	return nvme_set_over_temp_thresh(ctrl, sensor, temp);
+}
+
+static int nvme_tz_set_trip_temp(struct thermal_zone_device *tzdev,
+				 int trip, int temp)
+{
+	return nvme_tz_of_set_trip_temp(tzdev->devdata, trip, temp);
+}
+
+static struct thermal_zone_device_ops nvme_tz_ops = {
+	.get_temp = nvme_tz_get_temp,
+	.get_trip_type = nvme_tz_get_trip_type,
+	.get_trip_temp = nvme_tz_get_trip_temp,
+	.set_trip_temp = nvme_tz_set_trip_temp,
+};
+
+static const struct thermal_zone_params nvme_tz_params = {
+	.governor_name = "user_space",
+	.no_hwmon = true,
+};
+
+static int nvme_thermal_zone_register(struct nvme_ctrl *ctrl,
+				      unsigned int sensor)
+{
+	struct thermal_zone_device *tzdev;
+	struct nvme_tz *tz = &ctrl->tz[sensor];
+	char name[THERMAL_NAME_LENGTH];
+	int ret;
+
+	tz->sensor = sensor;
+	tz->params = nvme_tz_params;
+	snprintf(name, sizeof(name), "nvme%d_temp%u", ctrl->instance, sensor);
+
+	tzdev = thermal_zone_device_register(name, 1, 1, tz, &nvme_tz_ops,
+					     &tz->params, 0, 0);
+	if (IS_ERR(tzdev)) {
+		dev_err(ctrl->device,
+			"Failed to register thermal zone device: %ld\n",
+			PTR_ERR(tzdev));
+		return PTR_ERR(tzdev);
+	}
+
+	snprintf(name, sizeof(name), "temp%d", sensor);
+	ret = sysfs_create_link(&ctrl->ctrl_device.kobj, &tzdev->device.kobj,
+				name);
+	if (ret)
+		goto device_unregister;
+
+	ret = sysfs_create_link(&tzdev->device.kobj,
+				&ctrl->ctrl_device.kobj, "device");
+	if (ret)
+		goto remove_sensor_link;
+
+	tz->dev = tzdev;
+
+	return 0;
+
+remove_sensor_link:
+	sysfs_remove_link(&ctrl->ctrl_device.kobj, name);
+device_unregister:
+	thermal_zone_device_unregister(tzdev);
+
+	return ret;
+}
+
+/**
+ * nvme_thermal_zones_register() - register nvme thermal zone devices
+ * @ctrl: controller instance
+ *
+ * This function creates up to nine thermal zone devices for all implemented
+ * temperature sensors including the composite temperature.
+ * Each thermal zone device provides a single trip point temperature that is
+ * associated with an over temperature threshold.
+ * A negative value is returned when the device doesn't respond to the host
+ * and won't be able to continue initialization.
+ */
+int nvme_thermal_zones_register(struct nvme_ctrl *ctrl)
+{
+	struct nvme_smart_log *log;
+	int ret = 0;
+	int err;
+	int i;
+
+	if (!use_thermal_zone)
+		return 0;
+
+	log = kzalloc(sizeof(*log), GFP_KERNEL);
+	if (!log)
+		return 0;
+
+	err = nvme_get_log(ctrl, NVME_NSID_ALL, NVME_LOG_SMART, 0, log,
+			   sizeof(*log), 0);
+	if (err) {
+		dev_err(ctrl->device, "Failed to get SMART log: %d\n", err);
+		/* If the device provided a response, then it's non-fatal */
+		ret = (err > 0) ? 0 : err;
+		goto out;
+	}
+
+	for_each_clear_bit(i, ctrl->tz_enabled, ARRAY_SIZE(ctrl->tz)) {
+		int temp;
+
+		if (i)
+			temp = le16_to_cpu(log->temp_sensor[i - 1]);
+		else
+			temp = get_unaligned_le16(log->temperature);
+
+		/*
+		 * All implemented temperature sensors report a non-zero value
+		 * in temperature sensor fields in the smart log page.
+		 */
+		if (!temp)
+			continue;
+
+		err = nvme_thermal_zone_register(ctrl, i);
+		if (err)
+			goto out;
+
+		__set_bit(i, ctrl->tz_enabled);
+	}
+out:
+	if (err)
+		nvme_thermal_zones_unregister(ctrl);
+
+	kfree(log);
+
+	return ret;
+}
+
+/**
+ * nvme_thermal_zones_unregister() - unregister nvme thermal zone devices
+ * @ctrl: controller instance
+ *
+ * This function removes the registered thermal zone devices and symlinks.
+ */
+void nvme_thermal_zones_unregister(struct nvme_ctrl *ctrl)
+{
+	int i;
+
+	for_each_set_bit(i, ctrl->tz_enabled, ARRAY_SIZE(ctrl->tz)) {
+		struct nvme_tz *tz = &ctrl->tz[i];
+
+		if (tz->dev) {
+			char name[20];
+
+			sysfs_remove_link(&tz->dev->device.kobj, "device");
+			snprintf(name, sizeof(name), "temp%d", i);
+			sysfs_remove_link(&ctrl->ctrl_device.kobj, name);
+			thermal_zone_device_unregister(tz->dev);
+			tz->dev = NULL;
+		}
+
+		__clear_bit(i, ctrl->tz_enabled);
+	}
+}
diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index 8028ada..f29728b 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -749,6 +749,11 @@ struct nvme_write_zeroes_cmd {
 
 /* Features */
 
+enum {
+	NVME_TEMP_THRESH_MASK		= 0xffff,
+	NVME_TEMP_THRESH_SELECT_SHIFT	= 16,
+};
+
 struct nvme_feat_auto_pst {
 	__le64 entries[32];
 };
-- 
2.7.4

[PATCH v5 2/4] dt-bindings: thermal: nvme: Add binding documentation

[Akinobu Mita]

Add thermal binding documentation for NVMe temperature sensor.

Cc: Rob Herring <robh@kernel.org>
Cc: Zhang Rui <rui.zhang@intel.com>
Cc: Eduardo Valentin <edubezval@gmail.com>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Keith Busch <kbusch@kernel.org>
Cc: Jens Axboe <axboe@fb.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagi@grimberg.me>
Cc: Minwoo Im <minwoo.im.dev@gmail.com>
Cc: Kenneth Heitke <kenneth.heitke@intel.com>
Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni@wdc.com>
Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
---
* v5
- New patch

 Documentation/devicetree/bindings/thermal/nvme.txt | 56 ++++++++++++++++++++++
 1 file changed, 56 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/thermal/nvme.txt

diff --git a/Documentation/devicetree/bindings/thermal/nvme.txt b/Documentation/devicetree/bindings/thermal/nvme.txt
new file mode 100644
index 0000000..60b90de
--- /dev/null
+++ b/Documentation/devicetree/bindings/thermal/nvme.txt
@@ -0,0 +1,56 @@
+Binding for NVMe temperature sensor
+
+An NVMe controller reports up to nine temperature values in the SMART / Health
+log.
+
+Required properties:
+- reg: A five-cell address encoded as (phys.hi phys.mid phys.lo size.hi
+  size.lo). phys.hi should contain the device's BDF (Bus/Device/Function)
+  as 0b00000000 bbbbbbbb dddddfff 00000000. The other cells should be zero.
+  See also Documentation/devicetree/bindings/pci/pci.txt
+
+- #thermal-sensor-cells: Must be 1. See ./thermal.txt for a description.
+  In the thermal-sensors property, the sensor ID 0 for composite temperature,
+  1 through 8 for NVMe temperature sensor N.
+
+Example:
+
+&pcie0 {
+	...
+	nvme: nvme@0,0 {
+		reg = <0x0000 0 0 0 0>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+
+		nvmetemp: nvmetemp {
+			reg = <0x0000 0 0 0 0>; /* DEVFN = 0x00 (0:0) */
+			#thermal-sensor-cells = <1>;
+		};
+	};
+};
+
+&thermal_zones {
+	nvme_thermal: nvme {
+		polling-delay-passive = <2000>; /* milliseconds */
+		polling-delay = <0>; /* asynchronous event driven */
+
+				/* sensor	ID */
+		thermal-sensors = <&nvmetemp	0>;
+
+		trips {
+			nvme_alert0: nvme_alert0 {
+				temperature = <80000>; /* millicelsius */
+				hysteresis = <2000>; /* millicelsius */
+				type = "passive";
+			};
+		};
+
+		cooling-maps {
+			map0 {
+				trip = <&nvme_alert0>;
+				cooling-device =
+				<&fan0 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+			};
+		};
+	};
+};
-- 
2.7.4

[PATCH v5 2/4] dt-bindings: thermal: nvme: Add binding documentation

[Akinobu Mita]

Add thermal binding documentation for NVMe temperature sensor.

Cc: Rob Herring <robh at kernel.org>
Cc: Zhang Rui <rui.zhang at intel.com>
Cc: Eduardo Valentin <edubezval at gmail.com>
Cc: Daniel Lezcano <daniel.lezcano at linaro.org>
Cc: Keith Busch <kbusch at kernel.org>
Cc: Jens Axboe <axboe at fb.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Minwoo Im <minwoo.im.dev at gmail.com>
Cc: Kenneth Heitke <kenneth.heitke at intel.com>
Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni at wdc.com>
Signed-off-by: Akinobu Mita <akinobu.mita at gmail.com>
---
* v5
- New patch

 Documentation/devicetree/bindings/thermal/nvme.txt | 56 ++++++++++++++++++++++
 1 file changed, 56 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/thermal/nvme.txt

diff --git a/Documentation/devicetree/bindings/thermal/nvme.txt b/Documentation/devicetree/bindings/thermal/nvme.txt
new file mode 100644
index 0000000..60b90de
--- /dev/null
+++ b/Documentation/devicetree/bindings/thermal/nvme.txt
@@ -0,0 +1,56 @@
+Binding for NVMe temperature sensor
+
+An NVMe controller reports up to nine temperature values in the SMART / Health
+log.
+
+Required properties:
+- reg: A five-cell address encoded as (phys.hi phys.mid phys.lo size.hi
+  size.lo). phys.hi should contain the device's BDF (Bus/Device/Function)
+  as 0b00000000 bbbbbbbb dddddfff 00000000. The other cells should be zero.
+  See also Documentation/devicetree/bindings/pci/pci.txt
+
+- #thermal-sensor-cells: Must be 1. See ./thermal.txt for a description.
+  In the thermal-sensors property, the sensor ID 0 for composite temperature,
+  1 through 8 for NVMe temperature sensor N.
+
+Example:
+
+&pcie0 {
+	...
+	nvme: nvme at 0,0 {
+		reg = <0x0000 0 0 0 0>;
+		#address-cells = <3>;
+		#size-cells = <2>;
+
+		nvmetemp: nvmetemp {
+			reg = <0x0000 0 0 0 0>; /* DEVFN = 0x00 (0:0) */
+			#thermal-sensor-cells = <1>;
+		};
+	};
+};
+
+&thermal_zones {
+	nvme_thermal: nvme {
+		polling-delay-passive = <2000>; /* milliseconds */
+		polling-delay = <0>; /* asynchronous event driven */
+
+				/* sensor	ID */
+		thermal-sensors = <&nvmetemp	0>;
+
+		trips {
+			nvme_alert0: nvme_alert0 {
+				temperature = <80000>; /* millicelsius */
+				hysteresis = <2000>; /* millicelsius */
+				type = "passive";
+			};
+		};
+
+		cooling-maps {
+			map0 {
+				trip = <&nvme_alert0>;
+				cooling-device =
+				<&fan0 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
+			};
+		};
+	};
+};
-- 
2.7.4

[PATCH v5 3/4] nvme: support DT thermal zone device

[Akinobu Mita]

In addition to the standard thermal zone device, this adds support for
registering the DT thermal zone device.

If there is a device tree thermal zone node with the nvme temperature
sensor, the standard thermal zone device is not created.
Because we don't need two thermal zone devices for the same sensor.

Cc: Rob Herring <robh@kernel.org>
Cc: Zhang Rui <rui.zhang@intel.com>
Cc: Eduardo Valentin <edubezval@gmail.com>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Keith Busch <kbusch@kernel.org>
Cc: Jens Axboe <axboe@fb.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagi@grimberg.me>
Cc: Minwoo Im <minwoo.im.dev@gmail.com>
Cc: Kenneth Heitke <kenneth.heitke@intel.com>
Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni@wdc.com>
Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
---
* v5
- split the DT thermal zone support into separate patch
- don't register both standard and DT thermal zone

 drivers/nvme/host/nvme.h    |  1 +
 drivers/nvme/host/thermal.c | 38 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 39 insertions(+)

diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 49dd59ec..d501567 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -159,6 +159,7 @@ struct nvme_fault_inject {
 struct nvme_tz {
 	struct thermal_zone_params params;
 	struct thermal_zone_device *dev;
+	struct thermal_zone_device *of_dev;
 	unsigned int sensor;
 };
 
diff --git a/drivers/nvme/host/thermal.c b/drivers/nvme/host/thermal.c
index c3608f6..431aeb4 100644
--- a/drivers/nvme/host/thermal.c
+++ b/drivers/nvme/host/thermal.c
@@ -150,6 +150,11 @@ static struct thermal_zone_device_ops nvme_tz_ops = {
 	.set_trip_temp = nvme_tz_set_trip_temp,
 };
 
+static struct thermal_zone_of_device_ops nvme_tz_of_ops = {
+	.get_temp = nvme_tz_of_get_temp,
+	.set_trip_temp = nvme_tz_of_set_trip_temp,
+};
+
 static const struct thermal_zone_params nvme_tz_params = {
 	.governor_name = "user_space",
 	.no_hwmon = true,
@@ -164,6 +169,36 @@ static int nvme_thermal_zone_register(struct nvme_ctrl *ctrl,
 	int ret;
 
 	tz->sensor = sensor;
+
+	tzdev = thermal_zone_of_sensor_register(ctrl->dev, sensor, tz,
+						&nvme_tz_of_ops);
+	if (!IS_ERR(tzdev)) {
+		int trip_temp;
+
+		ret = tzdev->ops->get_trip_temp(tzdev, 0, &trip_temp);
+		if (ret) {
+			dev_err(ctrl->device,
+				"Failed to get trip temp: %d\n", ret);
+			return ret;
+		}
+
+		ret = tzdev->ops->set_trip_temp(tzdev, 0, trip_temp);
+		if (ret) {
+			dev_err(ctrl->device,
+				"Failed to set trip temp: %d\n", ret);
+			return ret;
+		}
+
+		tz->of_dev = tzdev;
+
+		return 0;
+	}
+
+	if (PTR_ERR(tzdev) != -ENODEV)
+		dev_warn(ctrl->device,
+			 "Failed to register thermal zone of sensor %d: %ld\n",
+			 sensor, PTR_ERR(tzdev));
+
 	tz->params = nvme_tz_params;
 	snprintf(name, sizeof(name), "nvme%d_temp%u", ctrl->instance, sensor);
 
@@ -286,6 +321,9 @@ void nvme_thermal_zones_unregister(struct nvme_ctrl *ctrl)
 			tz->dev = NULL;
 		}
 
+		thermal_zone_of_sensor_unregister(ctrl->dev, tz->of_dev);
+		tz->of_dev = NULL;
+
 		__clear_bit(i, ctrl->tz_enabled);
 	}
 }
-- 
2.7.4

[PATCH v5 3/4] nvme: support DT thermal zone device

[Akinobu Mita]

In addition to the standard thermal zone device, this adds support for
registering the DT thermal zone device.

If there is a device tree thermal zone node with the nvme temperature
sensor, the standard thermal zone device is not created.
Because we don't need two thermal zone devices for the same sensor.

Cc: Rob Herring <robh at kernel.org>
Cc: Zhang Rui <rui.zhang at intel.com>
Cc: Eduardo Valentin <edubezval at gmail.com>
Cc: Daniel Lezcano <daniel.lezcano at linaro.org>
Cc: Keith Busch <kbusch at kernel.org>
Cc: Jens Axboe <axboe at fb.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Minwoo Im <minwoo.im.dev at gmail.com>
Cc: Kenneth Heitke <kenneth.heitke at intel.com>
Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni at wdc.com>
Signed-off-by: Akinobu Mita <akinobu.mita at gmail.com>
---
* v5
- split the DT thermal zone support into separate patch
- don't register both standard and DT thermal zone

 drivers/nvme/host/nvme.h    |  1 +
 drivers/nvme/host/thermal.c | 38 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 39 insertions(+)

diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 49dd59ec..d501567 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -159,6 +159,7 @@ struct nvme_fault_inject {
 struct nvme_tz {
 	struct thermal_zone_params params;
 	struct thermal_zone_device *dev;
+	struct thermal_zone_device *of_dev;
 	unsigned int sensor;
 };
 
diff --git a/drivers/nvme/host/thermal.c b/drivers/nvme/host/thermal.c
index c3608f6..431aeb4 100644
--- a/drivers/nvme/host/thermal.c
+++ b/drivers/nvme/host/thermal.c
@@ -150,6 +150,11 @@ static struct thermal_zone_device_ops nvme_tz_ops = {
 	.set_trip_temp = nvme_tz_set_trip_temp,
 };
 
+static struct thermal_zone_of_device_ops nvme_tz_of_ops = {
+	.get_temp = nvme_tz_of_get_temp,
+	.set_trip_temp = nvme_tz_of_set_trip_temp,
+};
+
 static const struct thermal_zone_params nvme_tz_params = {
 	.governor_name = "user_space",
 	.no_hwmon = true,
@@ -164,6 +169,36 @@ static int nvme_thermal_zone_register(struct nvme_ctrl *ctrl,
 	int ret;
 
 	tz->sensor = sensor;
+
+	tzdev = thermal_zone_of_sensor_register(ctrl->dev, sensor, tz,
+						&nvme_tz_of_ops);
+	if (!IS_ERR(tzdev)) {
+		int trip_temp;
+
+		ret = tzdev->ops->get_trip_temp(tzdev, 0, &trip_temp);
+		if (ret) {
+			dev_err(ctrl->device,
+				"Failed to get trip temp: %d\n", ret);
+			return ret;
+		}
+
+		ret = tzdev->ops->set_trip_temp(tzdev, 0, trip_temp);
+		if (ret) {
+			dev_err(ctrl->device,
+				"Failed to set trip temp: %d\n", ret);
+			return ret;
+		}
+
+		tz->of_dev = tzdev;
+
+		return 0;
+	}
+
+	if (PTR_ERR(tzdev) != -ENODEV)
+		dev_warn(ctrl->device,
+			 "Failed to register thermal zone of sensor %d: %ld\n",
+			 sensor, PTR_ERR(tzdev));
+
 	tz->params = nvme_tz_params;
 	snprintf(name, sizeof(name), "nvme%d_temp%u", ctrl->instance, sensor);
 
@@ -286,6 +321,9 @@ void nvme_thermal_zones_unregister(struct nvme_ctrl *ctrl)
 			tz->dev = NULL;
 		}
 
+		thermal_zone_of_sensor_unregister(ctrl->dev, tz->of_dev);
+		tz->of_dev = NULL;
+
 		__clear_bit(i, ctrl->tz_enabled);
 	}
 }
-- 
2.7.4

[PATCH v5 4/4] nvme: notify thermal framework when temperature threshold events occur

[Akinobu Mita]

The NVMe controller supports the temperature threshold feature (Feature
Identifier 04h) that enables to configure the asynchronous event request
command to complete when the temperature is crossed its corresponding
temperature threshold.

This enables the reporting of asynchronous events from the controller when
the temperature reached or exceeded a temperature threshold.
In the case of the temperature threshold conditions, this notifies the
thermal framework.

The main purpose of this is to turn on a fan when overheated without
polling the device for the smart log that could prevent the lower power
state transitions.

Cc: Rob Herring <robh@kernel.org>
Cc: Zhang Rui <rui.zhang@intel.com>
Cc: Eduardo Valentin <edubezval@gmail.com>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Keith Busch <kbusch@kernel.org>
Cc: Jens Axboe <axboe@fb.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagi@grimberg.me>
Cc: Minwoo Im <minwoo.im.dev@gmail.com>
Cc: Kenneth Heitke <kenneth.heitke@intel.com>
Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni@wdc.com>
Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
---
* v5
- use thermal_zone_device_update() instead of thermal_notify_framework()

 drivers/nvme/host/core.c    | 14 ++++++++++++++
 drivers/nvme/host/nvme.h    |  5 +++++
 drivers/nvme/host/thermal.c | 14 ++++++++++++++
 include/linux/nvme.h        |  7 +++++++
 4 files changed, 40 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 1d84685..80403c3 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1186,6 +1186,9 @@ static void nvme_enable_aen(struct nvme_ctrl *ctrl)
 	u32 result, supported_aens = ctrl->oaes & NVME_AEN_SUPPORTED;
 	int status;
 
+	if (IS_ENABLED(CONFIG_THERMAL))
+		supported_aens |= NVME_SMART_CRIT_TEMPERATURE;
+
 	if (!supported_aens)
 		return;
 
@@ -3561,6 +3564,16 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 }
 EXPORT_SYMBOL_GPL(nvme_remove_namespaces);
 
+static void nvme_handle_aen_smart(struct nvme_ctrl *ctrl, u32 result)
+{
+	u32 aer_type = result & NVME_AER_TYPE_MASK;
+	u32 aer_info = (result >> NVME_AER_INFO_SHIFT) & NVME_AER_INFO_MASK;
+
+	if (aer_type == NVME_AER_SMART &&
+	    aer_info == NVME_AER_SMART_TEMP_THRESH)
+		nvme_thermal_zones_update(ctrl);
+}
+
 static void nvme_aen_uevent(struct nvme_ctrl *ctrl)
 {
 	char *envp[2] = { NULL, NULL };
@@ -3582,6 +3595,7 @@ static void nvme_async_event_work(struct work_struct *work)
 	struct nvme_ctrl *ctrl =
 		container_of(work, struct nvme_ctrl, async_event_work);
 
+	nvme_handle_aen_smart(ctrl, ctrl->aen_result);
 	nvme_aen_uevent(ctrl);
 	ctrl->ops->submit_async_event(ctrl);
 }
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index d501567..f29bc77 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -611,6 +611,7 @@ static inline struct nvme_ns *nvme_get_ns_from_dev(struct device *dev)
 
 int nvme_thermal_zones_register(struct nvme_ctrl *ctrl);
 void nvme_thermal_zones_unregister(struct nvme_ctrl *ctrl);
+void nvme_thermal_zones_update(struct nvme_ctrl *ctrl);
 
 #else
 
@@ -623,6 +624,10 @@ static inline void nvme_thermal_zones_unregister(struct nvme_ctrl *ctrl)
 {
 }
 
+static inline void nvme_thermal_zones_update(struct nvme_ctrl *ctrl)
+{
+}
+
 #endif /* CONFIG_THERMAL */
 
 #endif /* _NVME_H */
diff --git a/drivers/nvme/host/thermal.c b/drivers/nvme/host/thermal.c
index 431aeb4..9dcc1dc 100644
--- a/drivers/nvme/host/thermal.c
+++ b/drivers/nvme/host/thermal.c
@@ -327,3 +327,17 @@ void nvme_thermal_zones_unregister(struct nvme_ctrl *ctrl)
 		__clear_bit(i, ctrl->tz_enabled);
 	}
 }
+
+void nvme_thermal_zones_update(struct nvme_ctrl *ctrl)
+{
+	int i;
+
+	for_each_set_bit(i, ctrl->tz_enabled, ARRAY_SIZE(ctrl->tz)) {
+		if (ctrl->tz[i].dev)
+			thermal_zone_device_update(ctrl->tz[i].dev,
+						   THERMAL_EVENT_UNSPECIFIED);
+		if (ctrl->tz[i].of_dev)
+			thermal_zone_device_update(ctrl->tz[i].of_dev,
+						   THERMAL_EVENT_UNSPECIFIED);
+	}
+}
diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index f29728b..069b962 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -476,6 +476,7 @@ enum {
 };
 
 enum {
+	NVME_AER_TYPE_MASK		= 0x7,
 	NVME_AER_ERROR			= 0,
 	NVME_AER_SMART			= 1,
 	NVME_AER_NOTICE			= 2,
@@ -484,6 +485,12 @@ enum {
 };
 
 enum {
+	NVME_AER_INFO_SHIFT		= 8,
+	NVME_AER_INFO_MASK		= 0xff,
+	NVME_AER_SMART_TEMP_THRESH	= 0x01,
+};
+
+enum {
 	NVME_AER_NOTICE_NS_CHANGED	= 0x00,
 	NVME_AER_NOTICE_FW_ACT_STARTING = 0x01,
 	NVME_AER_NOTICE_ANA		= 0x03,
-- 
2.7.4

[PATCH v5 4/4] nvme: notify thermal framework when temperature threshold events occur

[Akinobu Mita]

The NVMe controller supports the temperature threshold feature (Feature
Identifier 04h) that enables to configure the asynchronous event request
command to complete when the temperature is crossed its corresponding
temperature threshold.

This enables the reporting of asynchronous events from the controller when
the temperature reached or exceeded a temperature threshold.
In the case of the temperature threshold conditions, this notifies the
thermal framework.

The main purpose of this is to turn on a fan when overheated without
polling the device for the smart log that could prevent the lower power
state transitions.

Cc: Rob Herring <robh at kernel.org>
Cc: Zhang Rui <rui.zhang at intel.com>
Cc: Eduardo Valentin <edubezval at gmail.com>
Cc: Daniel Lezcano <daniel.lezcano at linaro.org>
Cc: Keith Busch <kbusch at kernel.org>
Cc: Jens Axboe <axboe at fb.com>
Cc: Christoph Hellwig <hch at lst.de>
Cc: Sagi Grimberg <sagi at grimberg.me>
Cc: Minwoo Im <minwoo.im.dev at gmail.com>
Cc: Kenneth Heitke <kenneth.heitke at intel.com>
Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni at wdc.com>
Signed-off-by: Akinobu Mita <akinobu.mita at gmail.com>
---
* v5
- use thermal_zone_device_update() instead of thermal_notify_framework()

 drivers/nvme/host/core.c    | 14 ++++++++++++++
 drivers/nvme/host/nvme.h    |  5 +++++
 drivers/nvme/host/thermal.c | 14 ++++++++++++++
 include/linux/nvme.h        |  7 +++++++
 4 files changed, 40 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 1d84685..80403c3 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -1186,6 +1186,9 @@ static void nvme_enable_aen(struct nvme_ctrl *ctrl)
 	u32 result, supported_aens = ctrl->oaes & NVME_AEN_SUPPORTED;
 	int status;
 
+	if (IS_ENABLED(CONFIG_THERMAL))
+		supported_aens |= NVME_SMART_CRIT_TEMPERATURE;
+
 	if (!supported_aens)
 		return;
 
@@ -3561,6 +3564,16 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
 }
 EXPORT_SYMBOL_GPL(nvme_remove_namespaces);
 
+static void nvme_handle_aen_smart(struct nvme_ctrl *ctrl, u32 result)
+{
+	u32 aer_type = result & NVME_AER_TYPE_MASK;
+	u32 aer_info = (result >> NVME_AER_INFO_SHIFT) & NVME_AER_INFO_MASK;
+
+	if (aer_type == NVME_AER_SMART &&
+	    aer_info == NVME_AER_SMART_TEMP_THRESH)
+		nvme_thermal_zones_update(ctrl);
+}
+
 static void nvme_aen_uevent(struct nvme_ctrl *ctrl)
 {
 	char *envp[2] = { NULL, NULL };
@@ -3582,6 +3595,7 @@ static void nvme_async_event_work(struct work_struct *work)
 	struct nvme_ctrl *ctrl =
 		container_of(work, struct nvme_ctrl, async_event_work);
 
+	nvme_handle_aen_smart(ctrl, ctrl->aen_result);
 	nvme_aen_uevent(ctrl);
 	ctrl->ops->submit_async_event(ctrl);
 }
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index d501567..f29bc77 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -611,6 +611,7 @@ static inline struct nvme_ns *nvme_get_ns_from_dev(struct device *dev)
 
 int nvme_thermal_zones_register(struct nvme_ctrl *ctrl);
 void nvme_thermal_zones_unregister(struct nvme_ctrl *ctrl);
+void nvme_thermal_zones_update(struct nvme_ctrl *ctrl);
 
 #else
 
@@ -623,6 +624,10 @@ static inline void nvme_thermal_zones_unregister(struct nvme_ctrl *ctrl)
 {
 }
 
+static inline void nvme_thermal_zones_update(struct nvme_ctrl *ctrl)
+{
+}
+
 #endif /* CONFIG_THERMAL */
 
 #endif /* _NVME_H */
diff --git a/drivers/nvme/host/thermal.c b/drivers/nvme/host/thermal.c
index 431aeb4..9dcc1dc 100644
--- a/drivers/nvme/host/thermal.c
+++ b/drivers/nvme/host/thermal.c
@@ -327,3 +327,17 @@ void nvme_thermal_zones_unregister(struct nvme_ctrl *ctrl)
 		__clear_bit(i, ctrl->tz_enabled);
 	}
 }
+
+void nvme_thermal_zones_update(struct nvme_ctrl *ctrl)
+{
+	int i;
+
+	for_each_set_bit(i, ctrl->tz_enabled, ARRAY_SIZE(ctrl->tz)) {
+		if (ctrl->tz[i].dev)
+			thermal_zone_device_update(ctrl->tz[i].dev,
+						   THERMAL_EVENT_UNSPECIFIED);
+		if (ctrl->tz[i].of_dev)
+			thermal_zone_device_update(ctrl->tz[i].of_dev,
+						   THERMAL_EVENT_UNSPECIFIED);
+	}
+}
diff --git a/include/linux/nvme.h b/include/linux/nvme.h
index f29728b..069b962 100644
--- a/include/linux/nvme.h
+++ b/include/linux/nvme.h
@@ -476,6 +476,7 @@ enum {
 };
 
 enum {
+	NVME_AER_TYPE_MASK		= 0x7,
 	NVME_AER_ERROR			= 0,
 	NVME_AER_SMART			= 1,
 	NVME_AER_NOTICE			= 2,
@@ -484,6 +485,12 @@ enum {
 };
 
 enum {
+	NVME_AER_INFO_SHIFT		= 8,
+	NVME_AER_INFO_MASK		= 0xff,
+	NVME_AER_SMART_TEMP_THRESH	= 0x01,
+};
+
+enum {
 	NVME_AER_NOTICE_NS_CHANGED	= 0x00,
 	NVME_AER_NOTICE_FW_ACT_STARTING = 0x01,
 	NVME_AER_NOTICE_ANA		= 0x03,
-- 
2.7.4

Re: [PATCH v5 2/4] dt-bindings: thermal: nvme: Add binding documentation

[Rob Herring]

On Mon, Jul 01, 2019 at 11:12:32PM +0900, Akinobu Mita wrote:
> Add thermal binding documentation for NVMe temperature sensor.
> 
> Cc: Rob Herring <robh@kernel.org>
> Cc: Zhang Rui <rui.zhang@intel.com>
> Cc: Eduardo Valentin <edubezval@gmail.com>
> Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
> Cc: Keith Busch <kbusch@kernel.org>
> Cc: Jens Axboe <axboe@fb.com>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Sagi Grimberg <sagi@grimberg.me>
> Cc: Minwoo Im <minwoo.im.dev@gmail.com>
> Cc: Kenneth Heitke <kenneth.heitke@intel.com>
> Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni@wdc.com>
> Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
> ---
> * v5
> - New patch
> 
>  Documentation/devicetree/bindings/thermal/nvme.txt | 56 ++++++++++++++++++++++
>  1 file changed, 56 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/thermal/nvme.txt
> 
> diff --git a/Documentation/devicetree/bindings/thermal/nvme.txt b/Documentation/devicetree/bindings/thermal/nvme.txt
> new file mode 100644
> index 0000000..60b90de
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/thermal/nvme.txt
> @@ -0,0 +1,56 @@
> +Binding for NVMe temperature sensor
> +
> +An NVMe controller reports up to nine temperature values in the SMART / Health
> +log.
> +
> +Required properties:
> +- reg: A five-cell address encoded as (phys.hi phys.mid phys.lo size.hi
> +  size.lo). phys.hi should contain the device's BDF (Bus/Device/Function)
> +  as 0b00000000 bbbbbbbb dddddfff 00000000. The other cells should be zero.
> +  See also Documentation/devicetree/bindings/pci/pci.txt
> +
> +- #thermal-sensor-cells: Must be 1. See ./thermal.txt for a description.
> +  In the thermal-sensors property, the sensor ID 0 for composite temperature,
> +  1 through 8 for NVMe temperature sensor N.
> +
> +Example:
> +
> +&pcie0 {
> +	...
> +	nvme: nvme@0,0 {
> +		reg = <0x0000 0 0 0 0>;
> +		#address-cells = <3>;
> +		#size-cells = <2>;
> +
> +		nvmetemp: nvmetemp {
> +			reg = <0x0000 0 0 0 0>; /* DEVFN = 0x00 (0:0) */

I'm not sure this is really valid PCI addressing as the parent has the 
same address.

> +			#thermal-sensor-cells = <1>;

Can't you just put this in the parent? Is this really a separate 
addressable device from the parent?

> +		};
> +	};
> +};
> +
> +&thermal_zones {
> +	nvme_thermal: nvme {
> +		polling-delay-passive = <2000>; /* milliseconds */
> +		polling-delay = <0>; /* asynchronous event driven */
> +
> +				/* sensor	ID */
> +		thermal-sensors = <&nvmetemp	0>;
> +
> +		trips {
> +			nvme_alert0: nvme_alert0 {
> +				temperature = <80000>; /* millicelsius */
> +				hysteresis = <2000>; /* millicelsius */
> +				type = "passive";
> +			};
> +		};
> +
> +		cooling-maps {
> +			map0 {
> +				trip = <&nvme_alert0>;
> +				cooling-device =
> +				<&fan0 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
> +			};
> +		};
> +	};
> +};
> -- 
> 2.7.4
> 

[PATCH v5 2/4] dt-bindings: thermal: nvme: Add binding documentation

[Rob Herring]

On Mon, Jul 01, 2019@11:12:32PM +0900, Akinobu Mita wrote:
> Add thermal binding documentation for NVMe temperature sensor.
> 
> Cc: Rob Herring <robh at kernel.org>
> Cc: Zhang Rui <rui.zhang at intel.com>
> Cc: Eduardo Valentin <edubezval at gmail.com>
> Cc: Daniel Lezcano <daniel.lezcano at linaro.org>
> Cc: Keith Busch <kbusch at kernel.org>
> Cc: Jens Axboe <axboe at fb.com>
> Cc: Christoph Hellwig <hch at lst.de>
> Cc: Sagi Grimberg <sagi at grimberg.me>
> Cc: Minwoo Im <minwoo.im.dev at gmail.com>
> Cc: Kenneth Heitke <kenneth.heitke at intel.com>
> Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni at wdc.com>
> Signed-off-by: Akinobu Mita <akinobu.mita at gmail.com>
> ---
> * v5
> - New patch
> 
>  Documentation/devicetree/bindings/thermal/nvme.txt | 56 ++++++++++++++++++++++
>  1 file changed, 56 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/thermal/nvme.txt
> 
> diff --git a/Documentation/devicetree/bindings/thermal/nvme.txt b/Documentation/devicetree/bindings/thermal/nvme.txt
> new file mode 100644
> index 0000000..60b90de
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/thermal/nvme.txt
> @@ -0,0 +1,56 @@
> +Binding for NVMe temperature sensor
> +
> +An NVMe controller reports up to nine temperature values in the SMART / Health
> +log.
> +
> +Required properties:
> +- reg: A five-cell address encoded as (phys.hi phys.mid phys.lo size.hi
> +  size.lo). phys.hi should contain the device's BDF (Bus/Device/Function)
> +  as 0b00000000 bbbbbbbb dddddfff 00000000. The other cells should be zero.
> +  See also Documentation/devicetree/bindings/pci/pci.txt
> +
> +- #thermal-sensor-cells: Must be 1. See ./thermal.txt for a description.
> +  In the thermal-sensors property, the sensor ID 0 for composite temperature,
> +  1 through 8 for NVMe temperature sensor N.
> +
> +Example:
> +
> +&pcie0 {
> +	...
> +	nvme: nvme at 0,0 {
> +		reg = <0x0000 0 0 0 0>;
> +		#address-cells = <3>;
> +		#size-cells = <2>;
> +
> +		nvmetemp: nvmetemp {
> +			reg = <0x0000 0 0 0 0>; /* DEVFN = 0x00 (0:0) */

I'm not sure this is really valid PCI addressing as the parent has the 
same address.

> +			#thermal-sensor-cells = <1>;

Can't you just put this in the parent? Is this really a separate 
addressable device from the parent?

> +		};
> +	};
> +};
> +
> +&thermal_zones {
> +	nvme_thermal: nvme {
> +		polling-delay-passive = <2000>; /* milliseconds */
> +		polling-delay = <0>; /* asynchronous event driven */
> +
> +				/* sensor	ID */
> +		thermal-sensors = <&nvmetemp	0>;
> +
> +		trips {
> +			nvme_alert0: nvme_alert0 {
> +				temperature = <80000>; /* millicelsius */
> +				hysteresis = <2000>; /* millicelsius */
> +				type = "passive";
> +			};
> +		};
> +
> +		cooling-maps {
> +			map0 {
> +				trip = <&nvme_alert0>;
> +				cooling-device =
> +				<&fan0 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>;
> +			};
> +		};
> +	};
> +};
> -- 
> 2.7.4
> 

Re: [PATCH v5 2/4] dt-bindings: thermal: nvme: Add binding documentation

[Akinobu Mita]

2019年7月23日(火) 7:16 Rob Herring <robh@kernel.org>:
>
> On Mon, Jul 01, 2019 at 11:12:32PM +0900, Akinobu Mita wrote:
> > Add thermal binding documentation for NVMe temperature sensor.
> >
> > Cc: Rob Herring <robh@kernel.org>
> > Cc: Zhang Rui <rui.zhang@intel.com>
> > Cc: Eduardo Valentin <edubezval@gmail.com>
> > Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
> > Cc: Keith Busch <kbusch@kernel.org>
> > Cc: Jens Axboe <axboe@fb.com>
> > Cc: Christoph Hellwig <hch@lst.de>
> > Cc: Sagi Grimberg <sagi@grimberg.me>
> > Cc: Minwoo Im <minwoo.im.dev@gmail.com>
> > Cc: Kenneth Heitke <kenneth.heitke@intel.com>
> > Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni@wdc.com>
> > Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
> > ---
> > * v5
> > - New patch
> >
> >  Documentation/devicetree/bindings/thermal/nvme.txt | 56 ++++++++++++++++++++++
> >  1 file changed, 56 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/thermal/nvme.txt
> >
> > diff --git a/Documentation/devicetree/bindings/thermal/nvme.txt b/Documentation/devicetree/bindings/thermal/nvme.txt
> > new file mode 100644
> > index 0000000..60b90de
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/thermal/nvme.txt
> > @@ -0,0 +1,56 @@
> > +Binding for NVMe temperature sensor
> > +
> > +An NVMe controller reports up to nine temperature values in the SMART / Health
> > +log.
> > +
> > +Required properties:
> > +- reg: A five-cell address encoded as (phys.hi phys.mid phys.lo size.hi
> > +  size.lo). phys.hi should contain the device's BDF (Bus/Device/Function)
> > +  as 0b00000000 bbbbbbbb dddddfff 00000000. The other cells should be zero.
> > +  See also Documentation/devicetree/bindings/pci/pci.txt
> > +
> > +- #thermal-sensor-cells: Must be 1. See ./thermal.txt for a description.
> > +  In the thermal-sensors property, the sensor ID 0 for composite temperature,
> > +  1 through 8 for NVMe temperature sensor N.
> > +
> > +Example:
> > +
> > +&pcie0 {
> > +     ...
> > +     nvme: nvme@0,0 {
> > +             reg = <0x0000 0 0 0 0>;
> > +             #address-cells = <3>;
> > +             #size-cells = <2>;
> > +
> > +             nvmetemp: nvmetemp {
> > +                     reg = <0x0000 0 0 0 0>; /* DEVFN = 0x00 (0:0) */
>
> I'm not sure this is really valid PCI addressing as the parent has the
> same address.
>
> > +                     #thermal-sensor-cells = <1>;
>
> Can't you just put this in the parent? Is this really a separate
> addressable device from the parent?

How about this?

&pcie0 {
...
        pci-bridge@0 {
                reg = <0x00000 0 0 0 0>;
                #address-cells = <3>;
                #size-cells = <2>;

                nvme: nvme@0,0 {
                        reg = <0x10000 0 0 0 0>;
                        #thermal-sensor-cells = <1>;
                };
        };
};

and

&thermal_zones {
...
thermal-sensors = <&nvme 0>;
};

I tested this with the RockPro64 and edited
arch/arm64/boot/dts/rockchip/rk3399-rockpro64.dts.

$ lspci
00:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 0100
01:00.0 Non-Volatile memory controller: Micron/Crucial Technology
Device 2263 (rev 03)

$ lspci -tv
-[0000:00]---00.0-[01]----00.0  Micron/Crucial Technology Device 2263

[PATCH v5 2/4] dt-bindings: thermal: nvme: Add binding documentation

[Akinobu Mita]

2019?7?23?(?) 7:16 Rob Herring <robh at kernel.org>:
>
> On Mon, Jul 01, 2019@11:12:32PM +0900, Akinobu Mita wrote:
> > Add thermal binding documentation for NVMe temperature sensor.
> >
> > Cc: Rob Herring <robh at kernel.org>
> > Cc: Zhang Rui <rui.zhang at intel.com>
> > Cc: Eduardo Valentin <edubezval at gmail.com>
> > Cc: Daniel Lezcano <daniel.lezcano at linaro.org>
> > Cc: Keith Busch <kbusch at kernel.org>
> > Cc: Jens Axboe <axboe at fb.com>
> > Cc: Christoph Hellwig <hch at lst.de>
> > Cc: Sagi Grimberg <sagi at grimberg.me>
> > Cc: Minwoo Im <minwoo.im.dev at gmail.com>
> > Cc: Kenneth Heitke <kenneth.heitke at intel.com>
> > Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni at wdc.com>
> > Signed-off-by: Akinobu Mita <akinobu.mita at gmail.com>
> > ---
> > * v5
> > - New patch
> >
> >  Documentation/devicetree/bindings/thermal/nvme.txt | 56 ++++++++++++++++++++++
> >  1 file changed, 56 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/thermal/nvme.txt
> >
> > diff --git a/Documentation/devicetree/bindings/thermal/nvme.txt b/Documentation/devicetree/bindings/thermal/nvme.txt
> > new file mode 100644
> > index 0000000..60b90de
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/thermal/nvme.txt
> > @@ -0,0 +1,56 @@
> > +Binding for NVMe temperature sensor
> > +
> > +An NVMe controller reports up to nine temperature values in the SMART / Health
> > +log.
> > +
> > +Required properties:
> > +- reg: A five-cell address encoded as (phys.hi phys.mid phys.lo size.hi
> > +  size.lo). phys.hi should contain the device's BDF (Bus/Device/Function)
> > +  as 0b00000000 bbbbbbbb dddddfff 00000000. The other cells should be zero.
> > +  See also Documentation/devicetree/bindings/pci/pci.txt
> > +
> > +- #thermal-sensor-cells: Must be 1. See ./thermal.txt for a description.
> > +  In the thermal-sensors property, the sensor ID 0 for composite temperature,
> > +  1 through 8 for NVMe temperature sensor N.
> > +
> > +Example:
> > +
> > +&pcie0 {
> > +     ...
> > +     nvme: nvme at 0,0 {
> > +             reg = <0x0000 0 0 0 0>;
> > +             #address-cells = <3>;
> > +             #size-cells = <2>;
> > +
> > +             nvmetemp: nvmetemp {
> > +                     reg = <0x0000 0 0 0 0>; /* DEVFN = 0x00 (0:0) */
>
> I'm not sure this is really valid PCI addressing as the parent has the
> same address.
>
> > +                     #thermal-sensor-cells = <1>;
>
> Can't you just put this in the parent? Is this really a separate
> addressable device from the parent?

How about this?

&pcie0 {
...
        pci-bridge at 0 {
                reg = <0x00000 0 0 0 0>;
                #address-cells = <3>;
                #size-cells = <2>;

                nvme: nvme at 0,0 {
                        reg = <0x10000 0 0 0 0>;
                        #thermal-sensor-cells = <1>;
                };
        };
};

and

&thermal_zones {
...
thermal-sensors = <&nvme 0>;
};

I tested this with the RockPro64 and edited
arch/arm64/boot/dts/rockchip/rk3399-rockpro64.dts.

$ lspci
00:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 0100
01:00.0 Non-Volatile memory controller: Micron/Crucial Technology
Device 2263 (rev 03)

$ lspci -tv
-[0000:00]---00.0-[01]----00.0  Micron/Crucial Technology Device 2263

Re: [PATCH v5 2/4] dt-bindings: thermal: nvme: Add binding documentation

[Rob Herring]

On Thu, Jul 25, 2019 at 8:24 AM Akinobu Mita <akinobu.mita@gmail.com> wrote:
>
> 2019年7月23日(火) 7:16 Rob Herring <robh@kernel.org>:
> >
> > On Mon, Jul 01, 2019 at 11:12:32PM +0900, Akinobu Mita wrote:
> > > Add thermal binding documentation for NVMe temperature sensor.
> > >
> > > Cc: Rob Herring <robh@kernel.org>
> > > Cc: Zhang Rui <rui.zhang@intel.com>
> > > Cc: Eduardo Valentin <edubezval@gmail.com>
> > > Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
> > > Cc: Keith Busch <kbusch@kernel.org>
> > > Cc: Jens Axboe <axboe@fb.com>
> > > Cc: Christoph Hellwig <hch@lst.de>
> > > Cc: Sagi Grimberg <sagi@grimberg.me>
> > > Cc: Minwoo Im <minwoo.im.dev@gmail.com>
> > > Cc: Kenneth Heitke <kenneth.heitke@intel.com>
> > > Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni@wdc.com>
> > > Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
> > > ---
> > > * v5
> > > - New patch
> > >
> > >  Documentation/devicetree/bindings/thermal/nvme.txt | 56 ++++++++++++++++++++++
> > >  1 file changed, 56 insertions(+)
> > >  create mode 100644 Documentation/devicetree/bindings/thermal/nvme.txt
> > >
> > > diff --git a/Documentation/devicetree/bindings/thermal/nvme.txt b/Documentation/devicetree/bindings/thermal/nvme.txt
> > > new file mode 100644
> > > index 0000000..60b90de
> > > --- /dev/null
> > > +++ b/Documentation/devicetree/bindings/thermal/nvme.txt
> > > @@ -0,0 +1,56 @@
> > > +Binding for NVMe temperature sensor
> > > +
> > > +An NVMe controller reports up to nine temperature values in the SMART / Health
> > > +log.
> > > +
> > > +Required properties:
> > > +- reg: A five-cell address encoded as (phys.hi phys.mid phys.lo size.hi
> > > +  size.lo). phys.hi should contain the device's BDF (Bus/Device/Function)
> > > +  as 0b00000000 bbbbbbbb dddddfff 00000000. The other cells should be zero.
> > > +  See also Documentation/devicetree/bindings/pci/pci.txt
> > > +
> > > +- #thermal-sensor-cells: Must be 1. See ./thermal.txt for a description.
> > > +  In the thermal-sensors property, the sensor ID 0 for composite temperature,
> > > +  1 through 8 for NVMe temperature sensor N.
> > > +
> > > +Example:
> > > +
> > > +&pcie0 {
> > > +     ...
> > > +     nvme: nvme@0,0 {
> > > +             reg = <0x0000 0 0 0 0>;
> > > +             #address-cells = <3>;
> > > +             #size-cells = <2>;
> > > +
> > > +             nvmetemp: nvmetemp {
> > > +                     reg = <0x0000 0 0 0 0>; /* DEVFN = 0x00 (0:0) */
> >
> > I'm not sure this is really valid PCI addressing as the parent has the
> > same address.
> >
> > > +                     #thermal-sensor-cells = <1>;
> >
> > Can't you just put this in the parent? Is this really a separate
> > addressable device from the parent?
>
> How about this?
>
> &pcie0 {
> ...
>         pci-bridge@0 {
>                 reg = <0x00000 0 0 0 0>;
>                 #address-cells = <3>;
>                 #size-cells = <2>;
>
>                 nvme: nvme@0,0 {
>                         reg = <0x10000 0 0 0 0>;
>                         #thermal-sensor-cells = <1>;
>                 };
>         };
> };
>
> and
>
> &thermal_zones {
> ...
> thermal-sensors = <&nvme 0>;
> };
>
> I tested this with the RockPro64 and edited
> arch/arm64/boot/dts/rockchip/rk3399-rockpro64.dts.
>
> $ lspci
> 00:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 0100
> 01:00.0 Non-Volatile memory controller: Micron/Crucial Technology
> Device 2263 (rev 03)
>
> $ lspci -tv
> -[0000:00]---00.0-[01]----00.0  Micron/Crucial Technology Device 2263

Looks better to me.

Rob

[PATCH v5 2/4] dt-bindings: thermal: nvme: Add binding documentation

[Rob Herring]

On Thu, Jul 25, 2019@8:24 AM Akinobu Mita <akinobu.mita@gmail.com> wrote:
>
> 2019?7?23?(?) 7:16 Rob Herring <robh at kernel.org>:
> >
> > On Mon, Jul 01, 2019@11:12:32PM +0900, Akinobu Mita wrote:
> > > Add thermal binding documentation for NVMe temperature sensor.
> > >
> > > Cc: Rob Herring <robh at kernel.org>
> > > Cc: Zhang Rui <rui.zhang at intel.com>
> > > Cc: Eduardo Valentin <edubezval at gmail.com>
> > > Cc: Daniel Lezcano <daniel.lezcano at linaro.org>
> > > Cc: Keith Busch <kbusch at kernel.org>
> > > Cc: Jens Axboe <axboe at fb.com>
> > > Cc: Christoph Hellwig <hch at lst.de>
> > > Cc: Sagi Grimberg <sagi at grimberg.me>
> > > Cc: Minwoo Im <minwoo.im.dev at gmail.com>
> > > Cc: Kenneth Heitke <kenneth.heitke at intel.com>
> > > Cc: Chaitanya Kulkarni <Chaitanya.Kulkarni at wdc.com>
> > > Signed-off-by: Akinobu Mita <akinobu.mita at gmail.com>
> > > ---
> > > * v5
> > > - New patch
> > >
> > >  Documentation/devicetree/bindings/thermal/nvme.txt | 56 ++++++++++++++++++++++
> > >  1 file changed, 56 insertions(+)
> > >  create mode 100644 Documentation/devicetree/bindings/thermal/nvme.txt
> > >
> > > diff --git a/Documentation/devicetree/bindings/thermal/nvme.txt b/Documentation/devicetree/bindings/thermal/nvme.txt
> > > new file mode 100644
> > > index 0000000..60b90de
> > > --- /dev/null
> > > +++ b/Documentation/devicetree/bindings/thermal/nvme.txt
> > > @@ -0,0 +1,56 @@
> > > +Binding for NVMe temperature sensor
> > > +
> > > +An NVMe controller reports up to nine temperature values in the SMART / Health
> > > +log.
> > > +
> > > +Required properties:
> > > +- reg: A five-cell address encoded as (phys.hi phys.mid phys.lo size.hi
> > > +  size.lo). phys.hi should contain the device's BDF (Bus/Device/Function)
> > > +  as 0b00000000 bbbbbbbb dddddfff 00000000. The other cells should be zero.
> > > +  See also Documentation/devicetree/bindings/pci/pci.txt
> > > +
> > > +- #thermal-sensor-cells: Must be 1. See ./thermal.txt for a description.
> > > +  In the thermal-sensors property, the sensor ID 0 for composite temperature,
> > > +  1 through 8 for NVMe temperature sensor N.
> > > +
> > > +Example:
> > > +
> > > +&pcie0 {
> > > +     ...
> > > +     nvme: nvme at 0,0 {
> > > +             reg = <0x0000 0 0 0 0>;
> > > +             #address-cells = <3>;
> > > +             #size-cells = <2>;
> > > +
> > > +             nvmetemp: nvmetemp {
> > > +                     reg = <0x0000 0 0 0 0>; /* DEVFN = 0x00 (0:0) */
> >
> > I'm not sure this is really valid PCI addressing as the parent has the
> > same address.
> >
> > > +                     #thermal-sensor-cells = <1>;
> >
> > Can't you just put this in the parent? Is this really a separate
> > addressable device from the parent?
>
> How about this?
>
> &pcie0 {
> ...
>         pci-bridge at 0 {
>                 reg = <0x00000 0 0 0 0>;
>                 #address-cells = <3>;
>                 #size-cells = <2>;
>
>                 nvme: nvme at 0,0 {
>                         reg = <0x10000 0 0 0 0>;
>                         #thermal-sensor-cells = <1>;
>                 };
>         };
> };
>
> and
>
> &thermal_zones {
> ...
> thermal-sensors = <&nvme 0>;
> };
>
> I tested this with the RockPro64 and edited
> arch/arm64/boot/dts/rockchip/rk3399-rockpro64.dts.
>
> $ lspci
> 00:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 0100
> 01:00.0 Non-Volatile memory controller: Micron/Crucial Technology
> Device 2263 (rev 03)
>
> $ lspci -tv
> -[0000:00]---00.0-[01]----00.0  Micron/Crucial Technology Device 2263

Looks better to me.

Rob