[PATCH v12 0/5] Introduce SoC sleep stats driver

[PATCH v12 0/5] Introduce SoC sleep stats driver

[Maulik Shah]

Changes in v12:
- Address Stephan's comments from v11
- Rename driver and compatible to qcom,rpm(h)-stats
- Skip reading SMEM for RPM targets
- Make driver register in late_init to avoid -EPROBE_DEFER from smem.
- Change size to 0x10000 for RPM targets since area contains many others stats.

Changes in v11:
- Address Bjorn's comments from v10
- Add a case for RPM based targets dynamic offset in driver
- Update commit messages to use qcom sleep stats instead of soc sleep stats
- Drop individual target dtsi changes for sc7180 and sc7280
- Add single change to enable sleep stats for RPMh based targets
- Add single change to enable sleep stats for RPM based targets

Changes in v10:
- Updated device node name to use memory instead of aop_msgram
- Remove Lina's email from maintainers 
- Rename driver to qcom_sleep_stats. Update makefile, Kconfig accordingly
- Address Bjorn's comments from v9

Changes in v9:
- Remove soft dependency on smem module
- Return -EIO to userspace in case of error
- Make struct sleep_stats *stat a const pointer
- Remove the driver from soc_sleep_stats_driver name
- Remove offset address and directly mention the msgram address in dtsi
- Use devm_platform_get_and_ioremap_resource() to ioremap dtsi address
- Update device node name to mention aop_msgram instead rpmh-sleep-stats
- Update dtsi and documentation accordingly but retain the reviews

Changes in v8:
- Addressed bjorn's comments in driver from v7
- Update aoss_qmp device node reg size for sc7280

Changes in v7:
- Fix example in bindings documentation as per #address/size-cells = <1>.
- Add comment in driver from where 'ddr' subsystems name is read.
- Update comment in driver to s/beside/besides and others from v6.
- Rename debugfs_create_entries() from v6.
- Drop use of memcpy_fromio() to find the name.
- Use sizeof(*prv_data) in devm_kzalloc().
- Add change to define readq() if its not yet defined for compile support.
- Add wpss subsystem in the list of subsystems.
- Add module soft dependency on smem module.
- Add new change to add device node for sc7280.

Changes in v6:
- Address stephen's comments from v5 which includes below
- Pad 0 in documentation example to make address 8 digit
- define macro to calculate offset in driver
- Add appended_stats_avail to prv_data instead of using entire stats_config
- make array subsystems[] as const
- Add comment for SSR case
- Use memcpy_fromio() and devm_kcalloc() during probe
- Change file permission mode from 444 to 400 

- Address guenter's comments to add depends on QCOM_SMEM

- Add adsp_island and cdsp_island subsystems
- Use strim() to remove whitespace in stat name

Changes in v5:
- Remove underscore from node name in Documentation and DTSI change
- Remove global config from driver change

Changes in v4:
- Address bjorn's comments from v3 on change 2.
- Add bjorn's Reviewed-by on change 3 and 4.

Changes in v3:
- Address stephen's comments from v2 in change 1 and 2.
- Address bjorn's comments from v2 in change 3 and 4.
- Add Rob and bjorn's Reviewed-by on YAML change.

Changes in v2:
- Convert Documentation to YAML.
- Address stephen's comments from v1.
- Use debugfs instead of sysfs.
- Add sc7180 dts changes for sleep stats
- Add defconfig changes to enable driver
- Include subsystem stats from [1] in this single stats driver.
- Address stephen's comments from [1]
- Update cover letter inline to mention [1]

Qualcomm Technologies, Inc. (QTI)'s chipsets support SoC level low power
modes. SoCs Always On Processor/Resource Power Manager produces statistics
of the SoC sleep modes involving lowering or powering down of the rails and
the oscillator clock.

Additionally multiple subsystems present on SoC like modem, spss, adsp,
cdsp maintains their low power mode statistics in shared memory (SMEM).

Statistics includes SoC sleep mode type, number of times LPM entered, time
of last entry, exit, and accumulated sleep duration in seconds.

This series adds a driver to read the stats and export to debugfs.

[1] https://lore.kernel.org/patchwork/patch/1149381/

Mahesh Sivasubramanian (2):
  dt-bindings: Introduce QCOM Sleep stats bindings
  soc: qcom: Add Sleep stats driver

Maulik Shah (3):
  arm64: defconfig: Enable Sleep stats driver
  arm64: dts: qcom: Enable RPMh Sleep stats
  arm64: dts: qcom: Enable RPM Sleep stats

 .../devicetree/bindings/soc/qcom/qcom-stats.yaml   |  47 ++++
 arch/arm64/boot/dts/qcom/msm8996.dtsi              |   5 +
 arch/arm64/boot/dts/qcom/msm8998.dtsi              |   5 +
 arch/arm64/boot/dts/qcom/qcs404.dtsi               |   5 +
 arch/arm64/boot/dts/qcom/sc7180.dtsi               |   7 +-
 arch/arm64/boot/dts/qcom/sc7280.dtsi               |   7 +-
 arch/arm64/boot/dts/qcom/sdm630.dtsi               |   5 +
 arch/arm64/boot/dts/qcom/sm6125.dtsi               |   5 +
 arch/arm64/boot/dts/qcom/sm8150.dtsi               |   7 +-
 arch/arm64/boot/dts/qcom/sm8250.dtsi               |   7 +-
 arch/arm64/boot/dts/qcom/sm8350.dtsi               |   7 +-
 arch/arm64/configs/defconfig                       |   1 +
 drivers/soc/qcom/Kconfig                           |  10 +
 drivers/soc/qcom/Makefile                          |   1 +
 drivers/soc/qcom/qcom_stats.c                      | 277 +++++++++++++++++++++
 15 files changed, 391 insertions(+), 5 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml
 create mode 100644 drivers/soc/qcom/qcom_stats.c

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation

[PATCH v12 1/5] dt-bindings: Introduce QCOM Sleep stats bindings

[Maulik Shah]

From: Mahesh Sivasubramanian <msivasub@codeaurora.org>

Add device binding documentation for Qualcomm Technologies, Inc. (QTI)
Sleep stats driver. The driver is used for displaying Sleep statistic maintained
by Always On Processor or Resource Power Manager.

Cc: devicetree@vger.kernel.org
Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
Signed-off-by: Maulik Shah <mkshah@codeaurora.org>
Reviewed-by: Rob Herring <robh@kernel.org>
Reviewed-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Reviewed-by: Stephen Boyd <swboyd@chromium.org>
---
 .../devicetree/bindings/soc/qcom/qcom-stats.yaml   | 47 ++++++++++++++++++++++
 1 file changed, 47 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml

diff --git a/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml b/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml
new file mode 100644
index 0000000..99dff7d
--- /dev/null
+++ b/Documentation/devicetree/bindings/soc/qcom/qcom-stats.yaml
@@ -0,0 +1,47 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/soc/qcom/qcom-stats.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Technologies, Inc. (QTI) Stats bindings
+
+maintainers:
+  - Maulik Shah <mkshah@codeaurora.org>
+
+description:
+  Always On Processor/Resource Power Manager maintains statistics of the SoC
+  sleep modes involving powering down of the rails and oscillator clock.
+
+  Statistics includes SoC sleep mode type, number of times low power mode were
+  entered, time of last entry, time of last exit and accumulated sleep duration.
+
+properties:
+  compatible:
+    enum:
+      - qcom,rpmh-stats
+      - qcom,rpm-stats
+
+  reg:
+    maxItems: 1
+
+required:
+  - compatible
+  - reg
+
+additionalProperties: false
+
+examples:
+  # Example of rpmh sleep stats
+  - |
+    sram@c3f0000 {
+      compatible = "qcom,rpmh-stats";
+      reg = <0x0c3f0000 0x400>;
+    };
+  # Example of rpm sleep stats
+  - |
+    sram@4690000 {
+      compatible = "qcom,rpm-stats";
+      reg = <0x04690000 0x10000>;
+    };
+...
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation

[PATCH v12 2/5] soc: qcom: Add Sleep stats driver

[Maulik Shah]

From: Mahesh Sivasubramanian <msivasub@codeaurora.org>

Let's add a driver to read the stats from remote processor and
export to debugfs.

The driver creates "qcom_sleep_stats" directory in debugfs and
adds files for various low power mode available. Below is sample
output with command

cat /sys/kernel/debug/qcom_sleep_stats/ddr
count = 0
Last Entered At = 0
Last Exited At = 0
Accumulated Duration = 0

Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
[mkshah: add subsystem sleep stats, create one file for each stat]
Signed-off-by: Maulik Shah <mkshah@codeaurora.org>
---
 drivers/soc/qcom/Kconfig      |  10 ++
 drivers/soc/qcom/Makefile     |   1 +
 drivers/soc/qcom/qcom_stats.c | 277 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 288 insertions(+)
 create mode 100644 drivers/soc/qcom/qcom_stats.c

diff --git a/drivers/soc/qcom/Kconfig b/drivers/soc/qcom/Kconfig
index abfef20..3cf3a6b 100644
--- a/drivers/soc/qcom/Kconfig
+++ b/drivers/soc/qcom/Kconfig
@@ -199,6 +199,16 @@ config QCOM_SPM
 	  to manage cores, L2 low power modes and to configure the internal
 	  Adaptive Voltage Scaler parameters, where supported.
 
+config QCOM_STATS
+	tristate "Qualcomm Technologies, Inc. (QTI) Sleep stats driver"
+	depends on (ARCH_QCOM && DEBUG_FS) || COMPILE_TEST
+	depends on QCOM_SMEM
+	help
+	  Qualcomm Technologies, Inc. (QTI) Sleep stats driver to read
+	  the shared memory exported by the remote processor related to
+	  various SoC level low power modes statistics and export to debugfs
+	  interface.
+
 config QCOM_WCNSS_CTRL
 	tristate "Qualcomm WCNSS control driver"
 	depends on ARCH_QCOM || COMPILE_TEST
diff --git a/drivers/soc/qcom/Makefile b/drivers/soc/qcom/Makefile
index 24514c7..70d5de6 100644
--- a/drivers/soc/qcom/Makefile
+++ b/drivers/soc/qcom/Makefile
@@ -21,6 +21,7 @@ obj-$(CONFIG_QCOM_SMP2P)	+= smp2p.o
 obj-$(CONFIG_QCOM_SMSM)	+= smsm.o
 obj-$(CONFIG_QCOM_SOCINFO)	+= socinfo.o
 obj-$(CONFIG_QCOM_SPM)		+= spm.o
+obj-$(CONFIG_QCOM_STATS)	+= qcom_stats.o
 obj-$(CONFIG_QCOM_WCNSS_CTRL) += wcnss_ctrl.o
 obj-$(CONFIG_QCOM_APR) += apr.o
 obj-$(CONFIG_QCOM_LLCC) += llcc-qcom.o
diff --git a/drivers/soc/qcom/qcom_stats.c b/drivers/soc/qcom/qcom_stats.c
new file mode 100644
index 0000000..817505b
--- /dev/null
+++ b/drivers/soc/qcom/qcom_stats.c
@@ -0,0 +1,277 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2011-2021, The Linux Foundation. All rights reserved.
+ */
+
+#include <linux/debugfs.h>
+#include <linux/device.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/seq_file.h>
+
+#include <linux/soc/qcom/smem.h>
+#include <clocksource/arm_arch_timer.h>
+
+#define RPM_DYNAMIC_ADDR	0x14
+#define RPM_DYNAMIC_ADDR_MASK	0xFFFF
+
+#define STAT_TYPE_OFFSET	0x0
+#define COUNT_OFFSET		0x4
+#define LAST_ENTERED_AT_OFFSET	0x8
+#define LAST_EXITED_AT_OFFSET	0x10
+#define ACCUMULATED_OFFSET	0x18
+#define CLIENT_VOTES_OFFSET	0x1c
+
+struct subsystem_data {
+	const char *name;
+	u32 smem_item;
+	u32 pid;
+};
+
+static const struct subsystem_data subsystems[] = {
+	{ "modem", 605, 1 },
+	{ "wpss", 605, 13 },
+	{ "adsp", 606, 2 },
+	{ "cdsp", 607, 5 },
+	{ "slpi", 608, 3 },
+	{ "gpu", 609, 0 },
+	{ "display", 610, 0 },
+	{ "adsp_island", 613, 2 },
+	{ "slpi_island", 613, 3 },
+};
+
+struct stats_config {
+	size_t stats_offset;
+	size_t num_records;
+	bool appended_stats_avail;
+	bool dynamic_offset;
+	bool subsystem_stats_in_smem;
+};
+
+struct stats_data {
+	bool appended_stats_avail;
+	void __iomem *base;
+};
+
+struct sleep_stats {
+	u32 stat_type;
+	u32 count;
+	u64 last_entered_at;
+	u64 last_exited_at;
+	u64 accumulated;
+};
+
+struct appended_stats {
+	u32 client_votes;
+	u32 reserved[3];
+};
+
+static void qcom_print_stats(struct seq_file *s, const struct sleep_stats *stat)
+{
+	u64 accumulated = stat->accumulated;
+	/*
+	 * If a subsystem is in sleep when reading the sleep stats adjust
+	 * the accumulated sleep duration to show actual sleep time.
+	 */
+	if (stat->last_entered_at > stat->last_exited_at)
+		accumulated += arch_timer_read_counter() - stat->last_entered_at;
+
+	seq_printf(s, "Count: %u\n", stat->count);
+	seq_printf(s, "Last Entered At: %llu\n", stat->last_entered_at);
+	seq_printf(s, "Last Exited At: %llu\n", stat->last_exited_at);
+	seq_printf(s, "Accumulated Duration: %llu\n", accumulated);
+}
+
+static int qcom_subsystem_sleep_stats_show(struct seq_file *s, void *unused)
+{
+	struct subsystem_data *subsystem = s->private;
+	struct sleep_stats *stat;
+
+	/* Items are allocated lazily, so lookup pointer each time */
+	stat = qcom_smem_get(subsystem->pid, subsystem->smem_item, NULL);
+	if (IS_ERR(stat))
+		return -EIO;
+
+	qcom_print_stats(s, stat);
+
+	return 0;
+}
+
+static int qcom_soc_sleep_stats_show(struct seq_file *s, void *unused)
+{
+	struct stats_data *d = s->private;
+	void __iomem *reg = d->base;
+	struct sleep_stats stat;
+
+	memcpy_fromio(&stat, reg, sizeof(stat));
+	qcom_print_stats(s, &stat);
+
+	if (d->appended_stats_avail) {
+		struct appended_stats votes;
+
+		memcpy_fromio(&votes, reg + CLIENT_VOTES_OFFSET, sizeof(votes));
+		seq_printf(s, "Client Votes: %#x\n", votes.client_votes);
+	}
+
+	return 0;
+}
+
+DEFINE_SHOW_ATTRIBUTE(qcom_soc_sleep_stats);
+DEFINE_SHOW_ATTRIBUTE(qcom_subsystem_sleep_stats);
+
+static void qcom_create_soc_sleep_stat_files(struct dentry *root, void __iomem *reg,
+					     struct stats_data *d,
+					     const struct stats_config *config)
+{
+	char stat_type[sizeof(u32) + 1] = {0};
+	size_t stats_offset = config->stats_offset;
+	u32 offset = 0, type;
+	int i, j;
+
+	/*
+	 * On RPM targets, stats offset location is dynamic and changes from target
+	 * to target and sometimes from build to build for same target.
+	 *
+	 * In such cases the dynamic address is present at 0x14 offset from base
+	 * address in devicetree. The last 16bits indicates the stats_offset.
+	 */
+	if (config->dynamic_offset) {
+		stats_offset = readl(reg + RPM_DYNAMIC_ADDR);
+		stats_offset &= RPM_DYNAMIC_ADDR_MASK;
+	}
+
+	for (i = 0; i < config->num_records; i++) {
+		d[i].base = reg + offset + stats_offset;
+
+		/*
+		 * Read the low power mode name and create debugfs file for it.
+		 * The names read could be of below,
+		 * (may change depending on low power mode supported).
+		 * For rpmh-sleep-stats: "aosd", "cxsd" and "ddr".
+		 * For rpm-sleep-stats: "vmin" and "vlow".
+		 */
+		type = readl(d[i].base);
+		for (j = 0; j < sizeof(u32); j++) {
+			stat_type[j] = type & 0xff;
+			type = type >> 8;
+		}
+		strim(stat_type);
+		debugfs_create_file(stat_type, 0400, root, &d[i],
+				    &qcom_soc_sleep_stats_fops);
+
+		offset += sizeof(struct sleep_stats);
+		if (d[i].appended_stats_avail)
+			offset += sizeof(struct appended_stats);
+	}
+}
+
+static void qcom_create_subsystem_stat_files(struct dentry *root,
+					     const struct stats_config *config)
+{
+	const struct sleep_stats *stat;
+	int i;
+
+	if (!config->subsystem_stats_in_smem)
+		return;
+
+	for (i = 0; i < ARRAY_SIZE(subsystems); i++) {
+		stat = qcom_smem_get(subsystems[i].pid, subsystems[i].smem_item, NULL);
+		if (IS_ERR(stat))
+			continue;
+
+		debugfs_create_file(subsystems[i].name, 0400, root, (void *)&subsystems[i],
+				    &qcom_subsystem_sleep_stats_fops);
+	}
+}
+
+static int qcom_stats_probe(struct platform_device *pdev)
+{
+	void __iomem *reg;
+	struct dentry *root;
+	const struct stats_config *config;
+	struct stats_data *d;
+	int i;
+
+	config = device_get_match_data(&pdev->dev);
+	if (!config)
+		return -ENODEV;
+
+	reg = devm_platform_get_and_ioremap_resource(pdev, 0, NULL);
+	if (IS_ERR(reg))
+		return -ENOMEM;
+
+	d = devm_kcalloc(&pdev->dev, config->num_records,
+			 sizeof(*d), GFP_KERNEL);
+	if (!d)
+		return -ENOMEM;
+
+	for (i = 0; i < config->num_records; i++)
+		d[i].appended_stats_avail = config->appended_stats_avail;
+
+	root = debugfs_create_dir("qcom_stats", NULL);
+
+	qcom_create_subsystem_stat_files(root, config);
+	qcom_create_soc_sleep_stat_files(root, reg, d, config);
+
+	platform_set_drvdata(pdev, root);
+
+	return 0;
+}
+
+static int qcom_stats_remove(struct platform_device *pdev)
+{
+	struct dentry *root = platform_get_drvdata(pdev);
+
+	debugfs_remove_recursive(root);
+
+	return 0;
+}
+
+static const struct stats_config rpm_data = {
+	.stats_offset = 0,
+	.num_records = 2,
+	.appended_stats_avail = true,
+	.dynamic_offset = true,
+	.subsystem_stats_in_smem = false,
+};
+
+static const struct stats_config rpmh_data = {
+	.stats_offset = 0x48,
+	.num_records = 3,
+	.appended_stats_avail = false,
+	.dynamic_offset = false,
+	.subsystem_stats_in_smem = true,
+};
+
+static const struct of_device_id qcom_stats_table[] = {
+	{ .compatible = "qcom,rpm-stats", .data = &rpm_data },
+	{ .compatible = "qcom,rpmh-stats", .data = &rpmh_data },
+	{ }
+};
+MODULE_DEVICE_TABLE(of, qcom_stats_table);
+
+static struct platform_driver qcom_stats = {
+	.probe = qcom_stats_probe,
+	.remove = qcom_stats_remove,
+	.driver = {
+		.name = "qcom_stats",
+		.of_match_table = qcom_stats_table,
+	},
+};
+
+static int __init qcom_stats_init(void)
+{
+	return platform_driver_register(&qcom_stats);
+}
+late_initcall(qcom_stats_init);
+
+static void __exit qcom_stats_exit(void)
+{
+	platform_driver_unregister(&qcom_stats);
+}
+module_exit(qcom_stats_exit)
+
+MODULE_DESCRIPTION("Qualcomm Technologies, Inc. (QTI) Stats driver");
+MODULE_LICENSE("GPL v2");
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation

[PATCH v12 3/5] arm64: defconfig: Enable Sleep stats driver

[Maulik Shah]

Enable Sleep stats driver. The driver gives statistics for various
low power modes on Qualcomm Technologies, Inc. (QTI) SoCs.

Signed-off-by: Maulik Shah <mkshah@codeaurora.org>
Reviewed-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Reviewed-by: Stephen Boyd <swboyd@chromium.org>
---
 arch/arm64/configs/defconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index fc40ae9..ea1f1f8 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -1048,6 +1048,7 @@ CONFIG_QCOM_SMD_RPM=y
 CONFIG_QCOM_SMP2P=y
 CONFIG_QCOM_SMSM=y
 CONFIG_QCOM_SOCINFO=m
+CONFIG_QCOM_STATS=m
 CONFIG_QCOM_APR=m
 CONFIG_ARCH_R8A774A1=y
 CONFIG_ARCH_R8A774B1=y
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation

[PATCH v12 4/5] arm64: dts: qcom: Enable RPMh Sleep stats

[Maulik Shah]

Add device node for Sleep stats driver which provides various
low power mode stats on sc7180, sc7280, sm8150, sm8250 and sm8350.

Also update the reg size of aoss_qmp device to 0x400.

Cc: devicetree@vger.kernel.org
Signed-off-by: Maulik Shah <mkshah@codeaurora.org>
---
 arch/arm64/boot/dts/qcom/sc7180.dtsi | 7 ++++++-
 arch/arm64/boot/dts/qcom/sc7280.dtsi | 7 ++++++-
 arch/arm64/boot/dts/qcom/sm8150.dtsi | 7 ++++++-
 arch/arm64/boot/dts/qcom/sm8250.dtsi | 7 ++++++-
 arch/arm64/boot/dts/qcom/sm8350.dtsi | 7 ++++++-
 5 files changed, 30 insertions(+), 5 deletions(-)

diff --git a/arch/arm64/boot/dts/qcom/sc7180.dtsi b/arch/arm64/boot/dts/qcom/sc7180.dtsi
index f10217c..30efae7 100644
--- a/arch/arm64/boot/dts/qcom/sc7180.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc7180.dtsi
@@ -3219,13 +3219,18 @@
 
 		aoss_qmp: power-controller@c300000 {
 			compatible = "qcom,sc7180-aoss-qmp";
-			reg = <0 0x0c300000 0 0x100000>;
+			reg = <0 0x0c300000 0 0x400>;
 			interrupts = <GIC_SPI 389 IRQ_TYPE_EDGE_RISING>;
 			mboxes = <&apss_shared 0>;
 
 			#clock-cells = <0>;
 		};
 
+		sram@c3f0000 {
+			compatible = "qcom,rpmh-stats";
+			reg = <0 0x0c3f0000 0 0x400>;
+		};
+
 		spmi_bus: spmi@c440000 {
 			compatible = "qcom,spmi-pmic-arb";
 			reg = <0 0x0c440000 0 0x1100>,
diff --git a/arch/arm64/boot/dts/qcom/sc7280.dtsi b/arch/arm64/boot/dts/qcom/sc7280.dtsi
index 39635da..d74a4c8 100644
--- a/arch/arm64/boot/dts/qcom/sc7280.dtsi
+++ b/arch/arm64/boot/dts/qcom/sc7280.dtsi
@@ -2637,7 +2637,7 @@
 
 		aoss_qmp: power-controller@c300000 {
 			compatible = "qcom,sc7280-aoss-qmp";
-			reg = <0 0x0c300000 0 0x100000>;
+			reg = <0 0x0c300000 0 0x400>;
 			interrupts-extended = <&ipcc IPCC_CLIENT_AOP
 						     IPCC_MPROC_SIGNAL_GLINK_QMP
 						     IRQ_TYPE_EDGE_RISING>;
@@ -2647,6 +2647,11 @@
 			#clock-cells = <0>;
 		};
 
+		sram@c3f0000 {
+			compatible = "qcom,rpmh-stats";
+			reg = <0 0x0c3f0000 0 0x400>;
+		};
+
 		spmi_bus: spmi@c440000 {
 			compatible = "qcom,spmi-pmic-arb";
 			reg = <0 0x0c440000 0 0x1100>,
diff --git a/arch/arm64/boot/dts/qcom/sm8150.dtsi b/arch/arm64/boot/dts/qcom/sm8150.dtsi
index 8a03569..123d641 100644
--- a/arch/arm64/boot/dts/qcom/sm8150.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8150.dtsi
@@ -3170,13 +3170,18 @@
 
 		aoss_qmp: power-controller@c300000 {
 			compatible = "qcom,sm8150-aoss-qmp";
-			reg = <0x0 0x0c300000 0x0 0x100000>;
+			reg = <0x0 0x0c300000 0x0 0x400>;
 			interrupts = <GIC_SPI 389 IRQ_TYPE_EDGE_RISING>;
 			mboxes = <&apss_shared 0>;
 
 			#clock-cells = <0>;
 		};
 
+		sram@c3f0000 {
+			compatible = "qcom,rpmh-stats";
+			reg = <0 0x0c3f0000 0 0x400>;
+		};
+
 		tsens0: thermal-sensor@c263000 {
 			compatible = "qcom,sm8150-tsens", "qcom,tsens-v2";
 			reg = <0 0x0c263000 0 0x1ff>, /* TM */
diff --git a/arch/arm64/boot/dts/qcom/sm8250.dtsi b/arch/arm64/boot/dts/qcom/sm8250.dtsi
index 2796b27..e4bba03 100644
--- a/arch/arm64/boot/dts/qcom/sm8250.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8250.dtsi
@@ -2899,7 +2899,7 @@
 
 		aoss_qmp: power-controller@c300000 {
 			compatible = "qcom,sm8250-aoss-qmp";
-			reg = <0 0x0c300000 0 0x100000>;
+			reg = <0 0x0c300000 0 0x400>;
 			interrupts-extended = <&ipcc IPCC_CLIENT_AOP
 						     IPCC_MPROC_SIGNAL_GLINK_QMP
 						     IRQ_TYPE_EDGE_RISING>;
@@ -2909,6 +2909,11 @@
 			#clock-cells = <0>;
 		};
 
+		sram@c3f0000 {
+			compatible = "qcom,rpmh-stats";
+			reg = <0 0x0c3f0000 0 0x400>;
+		};
+
 		spmi_bus: spmi@c440000 {
 			compatible = "qcom,spmi-pmic-arb";
 			reg = <0x0 0x0c440000 0x0 0x0001100>,
diff --git a/arch/arm64/boot/dts/qcom/sm8350.dtsi b/arch/arm64/boot/dts/qcom/sm8350.dtsi
index 6c83cd5..90a0e19 100644
--- a/arch/arm64/boot/dts/qcom/sm8350.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8350.dtsi
@@ -788,7 +788,7 @@
 
 		aoss_qmp: power-controller@c300000 {
 			compatible = "qcom,sm8350-aoss-qmp";
-			reg = <0 0x0c300000 0 0x100000>;
+			reg = <0 0x0c300000 0 0x400>;
 			interrupts-extended = <&ipcc IPCC_CLIENT_AOP IPCC_MPROC_SIGNAL_GLINK_QMP
 						     IRQ_TYPE_EDGE_RISING>;
 			mboxes = <&ipcc IPCC_CLIENT_AOP IPCC_MPROC_SIGNAL_GLINK_QMP>;
@@ -796,6 +796,11 @@
 			#clock-cells = <0>;
 		};
 
+		sram@c3f0000 {
+			compatible = "qcom,rpmh-stats";
+			reg = <0 0x0c3f0000 0 0x400>;
+		};
+
 		spmi_bus: spmi@c440000 {
 			compatible = "qcom,spmi-pmic-arb";
 			reg = <0x0 0xc440000 0x0 0x1100>,
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation

[PATCH v12 5/5] arm64: dts: qcom: Enable RPM Sleep stats

[Maulik Shah]

Add device node for Sleep stats driver which provides various
low power mode stats on msm8996, msm8998, qcs404, sdm630 and
sm6125.

Cc: devicetree@vger.kernel.org
Signed-off-by: Maulik Shah <mkshah@codeaurora.org>
---
 arch/arm64/boot/dts/qcom/msm8996.dtsi | 5 +++++
 arch/arm64/boot/dts/qcom/msm8998.dtsi | 5 +++++
 arch/arm64/boot/dts/qcom/qcs404.dtsi  | 5 +++++
 arch/arm64/boot/dts/qcom/sdm630.dtsi  | 5 +++++
 arch/arm64/boot/dts/qcom/sm6125.dtsi  | 5 +++++
 5 files changed, 25 insertions(+)

diff --git a/arch/arm64/boot/dts/qcom/msm8996.dtsi b/arch/arm64/boot/dts/qcom/msm8996.dtsi
index eb3ec5f..465cd19 100644
--- a/arch/arm64/boot/dts/qcom/msm8996.dtsi
+++ b/arch/arm64/boot/dts/qcom/msm8996.dtsi
@@ -1523,6 +1523,11 @@
 			};
 		};
 
+		sram@290000 {
+			compatible = "qcom,rpm-stats";
+			reg = <0x00290000 0x10000>;
+		};
+
 		spmi_bus: qcom,spmi@400f000 {
 			compatible = "qcom,spmi-pmic-arb";
 			reg = <0x0400f000 0x1000>,
diff --git a/arch/arm64/boot/dts/qcom/msm8998.dtsi b/arch/arm64/boot/dts/qcom/msm8998.dtsi
index 3c1f133..887ba7c 100644
--- a/arch/arm64/boot/dts/qcom/msm8998.dtsi
+++ b/arch/arm64/boot/dts/qcom/msm8998.dtsi
@@ -2022,6 +2022,11 @@
 			};
 		};
 
+		sram@290000 {
+			compatible = "qcom,rpm-stats";
+			reg = <0x00290000 0x10000>;
+		};
+
 		spmi_bus: spmi@800f000 {
 			compatible = "qcom,spmi-pmic-arb";
 			reg =	<0x0800f000 0x1000>,
diff --git a/arch/arm64/boot/dts/qcom/qcs404.dtsi b/arch/arm64/boot/dts/qcom/qcs404.dtsi
index 339790b..8abede3 100644
--- a/arch/arm64/boot/dts/qcom/qcs404.dtsi
+++ b/arch/arm64/boot/dts/qcom/qcs404.dtsi
@@ -719,6 +719,11 @@
 			reg = <0x01937000 0x25000>;
 		};
 
+		sram@290000 {
+			compatible = "qcom,rpm-stats";
+			reg = <0x00290000 0x10000>;
+		};
+
 		spmi_bus: spmi@200f000 {
 			compatible = "qcom,spmi-pmic-arb";
 			reg = <0x0200f000 0x001000>,
diff --git a/arch/arm64/boot/dts/qcom/sdm630.dtsi b/arch/arm64/boot/dts/qcom/sdm630.dtsi
index 9c7f87e..4b66305 100644
--- a/arch/arm64/boot/dts/qcom/sdm630.dtsi
+++ b/arch/arm64/boot/dts/qcom/sdm630.dtsi
@@ -1176,6 +1176,11 @@
 			status = "disabled";
 		};
 
+		sram@290000 {
+			compatible = "qcom,rpm-stats";
+			reg = <0x00290000 0x10000>;
+		};
+
 		spmi_bus: spmi@800f000 {
 			compatible = "qcom,spmi-pmic-arb";
 			reg =	<0x0800f000 0x1000>,
diff --git a/arch/arm64/boot/dts/qcom/sm6125.dtsi b/arch/arm64/boot/dts/qcom/sm6125.dtsi
index c2317dd..6b03beb 100644
--- a/arch/arm64/boot/dts/qcom/sm6125.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm6125.dtsi
@@ -460,6 +460,11 @@
 			};
 		};
 
+		sram@4690000 {
+			compatible = "qcom,rpm-stats";
+			reg = <0x04690000 0x10000>;
+		};
+
 		spmi_bus: spmi@1c40000 {
 			compatible = "qcom,spmi-pmic-arb";
 			reg = <0x01c40000 0x1100>,
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH v12 0/5] Introduce SoC sleep stats driver

[Shawn Guo]

On Wed, Oct 13, 2021 at 12:08:19PM +0530, Maulik Shah wrote:
> Changes in v12:
> - Address Stephan's comments from v11
> - Rename driver and compatible to qcom,rpm(h)-stats
> - Skip reading SMEM for RPM targets
> - Make driver register in late_init to avoid -EPROBE_DEFER from smem.
> - Change size to 0x10000 for RPM targets since area contains many others stats.
> 
> Changes in v11:
> - Address Bjorn's comments from v10
> - Add a case for RPM based targets dynamic offset in driver
> - Update commit messages to use qcom sleep stats instead of soc sleep stats
> - Drop individual target dtsi changes for sc7180 and sc7280
> - Add single change to enable sleep stats for RPMh based targets
> - Add single change to enable sleep stats for RPM based targets
> 
> Changes in v10:
> - Updated device node name to use memory instead of aop_msgram
> - Remove Lina's email from maintainers 
> - Rename driver to qcom_sleep_stats. Update makefile, Kconfig accordingly
> - Address Bjorn's comments from v9
> 
> Changes in v9:
> - Remove soft dependency on smem module
> - Return -EIO to userspace in case of error
> - Make struct sleep_stats *stat a const pointer
> - Remove the driver from soc_sleep_stats_driver name
> - Remove offset address and directly mention the msgram address in dtsi
> - Use devm_platform_get_and_ioremap_resource() to ioremap dtsi address
> - Update device node name to mention aop_msgram instead rpmh-sleep-stats
> - Update dtsi and documentation accordingly but retain the reviews
> 
> Changes in v8:
> - Addressed bjorn's comments in driver from v7
> - Update aoss_qmp device node reg size for sc7280
> 
> Changes in v7:
> - Fix example in bindings documentation as per #address/size-cells = <1>.
> - Add comment in driver from where 'ddr' subsystems name is read.
> - Update comment in driver to s/beside/besides and others from v6.
> - Rename debugfs_create_entries() from v6.
> - Drop use of memcpy_fromio() to find the name.
> - Use sizeof(*prv_data) in devm_kzalloc().
> - Add change to define readq() if its not yet defined for compile support.
> - Add wpss subsystem in the list of subsystems.
> - Add module soft dependency on smem module.
> - Add new change to add device node for sc7280.
> 
> Changes in v6:
> - Address stephen's comments from v5 which includes below
> - Pad 0 in documentation example to make address 8 digit
> - define macro to calculate offset in driver
> - Add appended_stats_avail to prv_data instead of using entire stats_config
> - make array subsystems[] as const
> - Add comment for SSR case
> - Use memcpy_fromio() and devm_kcalloc() during probe
> - Change file permission mode from 444 to 400 
> 
> - Address guenter's comments to add depends on QCOM_SMEM
> 
> - Add adsp_island and cdsp_island subsystems
> - Use strim() to remove whitespace in stat name
> 
> Changes in v5:
> - Remove underscore from node name in Documentation and DTSI change
> - Remove global config from driver change
> 
> Changes in v4:
> - Address bjorn's comments from v3 on change 2.
> - Add bjorn's Reviewed-by on change 3 and 4.
> 
> Changes in v3:
> - Address stephen's comments from v2 in change 1 and 2.
> - Address bjorn's comments from v2 in change 3 and 4.
> - Add Rob and bjorn's Reviewed-by on YAML change.
> 
> Changes in v2:
> - Convert Documentation to YAML.
> - Address stephen's comments from v1.
> - Use debugfs instead of sysfs.
> - Add sc7180 dts changes for sleep stats
> - Add defconfig changes to enable driver
> - Include subsystem stats from [1] in this single stats driver.
> - Address stephen's comments from [1]
> - Update cover letter inline to mention [1]
> 
> Qualcomm Technologies, Inc. (QTI)'s chipsets support SoC level low power
> modes. SoCs Always On Processor/Resource Power Manager produces statistics
> of the SoC sleep modes involving lowering or powering down of the rails and
> the oscillator clock.
> 
> Additionally multiple subsystems present on SoC like modem, spss, adsp,
> cdsp maintains their low power mode statistics in shared memory (SMEM).
> 
> Statistics includes SoC sleep mode type, number of times LPM entered, time
> of last entry, exit, and accumulated sleep duration in seconds.
> 
> This series adds a driver to read the stats and export to debugfs.
> 
> [1] https://lore.kernel.org/patchwork/patch/1149381/
> 
> Mahesh Sivasubramanian (2):
>   dt-bindings: Introduce QCOM Sleep stats bindings
>   soc: qcom: Add Sleep stats driver
> 
> Maulik Shah (3):
>   arm64: defconfig: Enable Sleep stats driver
>   arm64: dts: qcom: Enable RPMh Sleep stats
>   arm64: dts: qcom: Enable RPM Sleep stats

On SDM660:

Tested-by: Shawn Guo <shawn.guo@linaro.org>

Re: [PATCH v12 2/5] soc: qcom: Add Sleep stats driver

[Shawn Guo]

Hi Mahesh,

Thanks for the work!

While I tested it good on SDM660 and MSM8939, I have a question
regarding 'vlow' mode below. 

On Wed, Oct 13, 2021 at 12:08:21PM +0530, Maulik Shah wrote:
> From: Mahesh Sivasubramanian <msivasub@codeaurora.org>
> 
> Let's add a driver to read the stats from remote processor and
> export to debugfs.
> 
> The driver creates "qcom_sleep_stats" directory in debugfs and
> adds files for various low power mode available. Below is sample
> output with command
> 
> cat /sys/kernel/debug/qcom_sleep_stats/ddr
> count = 0
> Last Entered At = 0
> Last Exited At = 0
> Accumulated Duration = 0
> 
> Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
> Signed-off-by: Lina Iyer <ilina@codeaurora.org>
> [mkshah: add subsystem sleep stats, create one file for each stat]
> Signed-off-by: Maulik Shah <mkshah@codeaurora.org>
> ---
>  drivers/soc/qcom/Kconfig      |  10 ++
>  drivers/soc/qcom/Makefile     |   1 +
>  drivers/soc/qcom/qcom_stats.c | 277 ++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 288 insertions(+)
>  create mode 100644 drivers/soc/qcom/qcom_stats.c
> 
> diff --git a/drivers/soc/qcom/Kconfig b/drivers/soc/qcom/Kconfig
> index abfef20..3cf3a6b 100644
> --- a/drivers/soc/qcom/Kconfig
> +++ b/drivers/soc/qcom/Kconfig
> @@ -199,6 +199,16 @@ config QCOM_SPM
>  	  to manage cores, L2 low power modes and to configure the internal
>  	  Adaptive Voltage Scaler parameters, where supported.
>  
> +config QCOM_STATS
> +	tristate "Qualcomm Technologies, Inc. (QTI) Sleep stats driver"
> +	depends on (ARCH_QCOM && DEBUG_FS) || COMPILE_TEST
> +	depends on QCOM_SMEM
> +	help
> +	  Qualcomm Technologies, Inc. (QTI) Sleep stats driver to read
> +	  the shared memory exported by the remote processor related to
> +	  various SoC level low power modes statistics and export to debugfs
> +	  interface.
> +
>  config QCOM_WCNSS_CTRL
>  	tristate "Qualcomm WCNSS control driver"
>  	depends on ARCH_QCOM || COMPILE_TEST
> diff --git a/drivers/soc/qcom/Makefile b/drivers/soc/qcom/Makefile
> index 24514c7..70d5de6 100644
> --- a/drivers/soc/qcom/Makefile
> +++ b/drivers/soc/qcom/Makefile
> @@ -21,6 +21,7 @@ obj-$(CONFIG_QCOM_SMP2P)	+= smp2p.o
>  obj-$(CONFIG_QCOM_SMSM)	+= smsm.o
>  obj-$(CONFIG_QCOM_SOCINFO)	+= socinfo.o
>  obj-$(CONFIG_QCOM_SPM)		+= spm.o
> +obj-$(CONFIG_QCOM_STATS)	+= qcom_stats.o
>  obj-$(CONFIG_QCOM_WCNSS_CTRL) += wcnss_ctrl.o
>  obj-$(CONFIG_QCOM_APR) += apr.o
>  obj-$(CONFIG_QCOM_LLCC) += llcc-qcom.o
> diff --git a/drivers/soc/qcom/qcom_stats.c b/drivers/soc/qcom/qcom_stats.c
> new file mode 100644
> index 0000000..817505b
> --- /dev/null
> +++ b/drivers/soc/qcom/qcom_stats.c
> @@ -0,0 +1,277 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c) 2011-2021, The Linux Foundation. All rights reserved.
> + */
> +
> +#include <linux/debugfs.h>
> +#include <linux/device.h>
> +#include <linux/io.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/platform_device.h>
> +#include <linux/seq_file.h>
> +
> +#include <linux/soc/qcom/smem.h>
> +#include <clocksource/arm_arch_timer.h>
> +
> +#define RPM_DYNAMIC_ADDR	0x14
> +#define RPM_DYNAMIC_ADDR_MASK	0xFFFF
> +
> +#define STAT_TYPE_OFFSET	0x0
> +#define COUNT_OFFSET		0x4
> +#define LAST_ENTERED_AT_OFFSET	0x8
> +#define LAST_EXITED_AT_OFFSET	0x10
> +#define ACCUMULATED_OFFSET	0x18
> +#define CLIENT_VOTES_OFFSET	0x1c
> +
> +struct subsystem_data {
> +	const char *name;
> +	u32 smem_item;
> +	u32 pid;
> +};
> +
> +static const struct subsystem_data subsystems[] = {
> +	{ "modem", 605, 1 },
> +	{ "wpss", 605, 13 },
> +	{ "adsp", 606, 2 },
> +	{ "cdsp", 607, 5 },
> +	{ "slpi", 608, 3 },
> +	{ "gpu", 609, 0 },
> +	{ "display", 610, 0 },
> +	{ "adsp_island", 613, 2 },
> +	{ "slpi_island", 613, 3 },
> +};
> +
> +struct stats_config {
> +	size_t stats_offset;
> +	size_t num_records;
> +	bool appended_stats_avail;
> +	bool dynamic_offset;
> +	bool subsystem_stats_in_smem;
> +};
> +
> +struct stats_data {
> +	bool appended_stats_avail;
> +	void __iomem *base;
> +};
> +
> +struct sleep_stats {
> +	u32 stat_type;
> +	u32 count;
> +	u64 last_entered_at;
> +	u64 last_exited_at;
> +	u64 accumulated;
> +};
> +
> +struct appended_stats {
> +	u32 client_votes;
> +	u32 reserved[3];
> +};
> +
> +static void qcom_print_stats(struct seq_file *s, const struct sleep_stats *stat)
> +{
> +	u64 accumulated = stat->accumulated;
> +	/*
> +	 * If a subsystem is in sleep when reading the sleep stats adjust
> +	 * the accumulated sleep duration to show actual sleep time.
> +	 */
> +	if (stat->last_entered_at > stat->last_exited_at)
> +		accumulated += arch_timer_read_counter() - stat->last_entered_at;
> +
> +	seq_printf(s, "Count: %u\n", stat->count);
> +	seq_printf(s, "Last Entered At: %llu\n", stat->last_entered_at);
> +	seq_printf(s, "Last Exited At: %llu\n", stat->last_exited_at);
> +	seq_printf(s, "Accumulated Duration: %llu\n", accumulated);
> +}
> +
> +static int qcom_subsystem_sleep_stats_show(struct seq_file *s, void *unused)
> +{
> +	struct subsystem_data *subsystem = s->private;
> +	struct sleep_stats *stat;
> +
> +	/* Items are allocated lazily, so lookup pointer each time */
> +	stat = qcom_smem_get(subsystem->pid, subsystem->smem_item, NULL);
> +	if (IS_ERR(stat))
> +		return -EIO;
> +
> +	qcom_print_stats(s, stat);
> +
> +	return 0;
> +}
> +
> +static int qcom_soc_sleep_stats_show(struct seq_file *s, void *unused)
> +{
> +	struct stats_data *d = s->private;
> +	void __iomem *reg = d->base;
> +	struct sleep_stats stat;
> +
> +	memcpy_fromio(&stat, reg, sizeof(stat));
> +	qcom_print_stats(s, &stat);
> +
> +	if (d->appended_stats_avail) {
> +		struct appended_stats votes;
> +
> +		memcpy_fromio(&votes, reg + CLIENT_VOTES_OFFSET, sizeof(votes));
> +		seq_printf(s, "Client Votes: %#x\n", votes.client_votes);
> +	}
> +
> +	return 0;
> +}
> +
> +DEFINE_SHOW_ATTRIBUTE(qcom_soc_sleep_stats);
> +DEFINE_SHOW_ATTRIBUTE(qcom_subsystem_sleep_stats);
> +
> +static void qcom_create_soc_sleep_stat_files(struct dentry *root, void __iomem *reg,
> +					     struct stats_data *d,
> +					     const struct stats_config *config)
> +{
> +	char stat_type[sizeof(u32) + 1] = {0};
> +	size_t stats_offset = config->stats_offset;
> +	u32 offset = 0, type;
> +	int i, j;
> +
> +	/*
> +	 * On RPM targets, stats offset location is dynamic and changes from target
> +	 * to target and sometimes from build to build for same target.
> +	 *
> +	 * In such cases the dynamic address is present at 0x14 offset from base
> +	 * address in devicetree. The last 16bits indicates the stats_offset.
> +	 */
> +	if (config->dynamic_offset) {
> +		stats_offset = readl(reg + RPM_DYNAMIC_ADDR);
> +		stats_offset &= RPM_DYNAMIC_ADDR_MASK;
> +	}
> +
> +	for (i = 0; i < config->num_records; i++) {
> +		d[i].base = reg + offset + stats_offset;
> +
> +		/*
> +		 * Read the low power mode name and create debugfs file for it.
> +		 * The names read could be of below,
> +		 * (may change depending on low power mode supported).
> +		 * For rpmh-sleep-stats: "aosd", "cxsd" and "ddr".
> +		 * For rpm-sleep-stats: "vmin" and "vlow".

It reports 'vmin' and 'xosd' on MSM8939, 'vmin' and 'vlow' on SDM660.
I know that 'vmin' is VDD Minimization mode, and 'xosd' is XO Shutdown
mode.  But I'm not sure about 'vlow' mode.  Could you share some
information regarding what this low power mode is, and how it differs
from 'vmin' and 'xosd'?

Also I guess XO Shutdown mode is still available on SoCs that report
'vlow' instead of 'xosd', just like SDM660.  Is there any way we can
still check the stats of XO Shutdown mode on these SoCs?  Thanks!

Shawn

> +		 */
> +		type = readl(d[i].base);
> +		for (j = 0; j < sizeof(u32); j++) {
> +			stat_type[j] = type & 0xff;
> +			type = type >> 8;
> +		}
> +		strim(stat_type);
> +		debugfs_create_file(stat_type, 0400, root, &d[i],
> +				    &qcom_soc_sleep_stats_fops);
> +
> +		offset += sizeof(struct sleep_stats);
> +		if (d[i].appended_stats_avail)
> +			offset += sizeof(struct appended_stats);
> +	}
> +}
> +
> +static void qcom_create_subsystem_stat_files(struct dentry *root,
> +					     const struct stats_config *config)
> +{
> +	const struct sleep_stats *stat;
> +	int i;
> +
> +	if (!config->subsystem_stats_in_smem)
> +		return;
> +
> +	for (i = 0; i < ARRAY_SIZE(subsystems); i++) {
> +		stat = qcom_smem_get(subsystems[i].pid, subsystems[i].smem_item, NULL);
> +		if (IS_ERR(stat))
> +			continue;
> +
> +		debugfs_create_file(subsystems[i].name, 0400, root, (void *)&subsystems[i],
> +				    &qcom_subsystem_sleep_stats_fops);
> +	}
> +}
> +
> +static int qcom_stats_probe(struct platform_device *pdev)
> +{
> +	void __iomem *reg;
> +	struct dentry *root;
> +	const struct stats_config *config;
> +	struct stats_data *d;
> +	int i;
> +
> +	config = device_get_match_data(&pdev->dev);
> +	if (!config)
> +		return -ENODEV;
> +
> +	reg = devm_platform_get_and_ioremap_resource(pdev, 0, NULL);
> +	if (IS_ERR(reg))
> +		return -ENOMEM;
> +
> +	d = devm_kcalloc(&pdev->dev, config->num_records,
> +			 sizeof(*d), GFP_KERNEL);
> +	if (!d)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < config->num_records; i++)
> +		d[i].appended_stats_avail = config->appended_stats_avail;
> +
> +	root = debugfs_create_dir("qcom_stats", NULL);
> +
> +	qcom_create_subsystem_stat_files(root, config);
> +	qcom_create_soc_sleep_stat_files(root, reg, d, config);
> +
> +	platform_set_drvdata(pdev, root);
> +
> +	return 0;
> +}
> +
> +static int qcom_stats_remove(struct platform_device *pdev)
> +{
> +	struct dentry *root = platform_get_drvdata(pdev);
> +
> +	debugfs_remove_recursive(root);
> +
> +	return 0;
> +}
> +
> +static const struct stats_config rpm_data = {
> +	.stats_offset = 0,
> +	.num_records = 2,
> +	.appended_stats_avail = true,
> +	.dynamic_offset = true,
> +	.subsystem_stats_in_smem = false,
> +};
> +
> +static const struct stats_config rpmh_data = {
> +	.stats_offset = 0x48,
> +	.num_records = 3,
> +	.appended_stats_avail = false,
> +	.dynamic_offset = false,
> +	.subsystem_stats_in_smem = true,
> +};
> +
> +static const struct of_device_id qcom_stats_table[] = {
> +	{ .compatible = "qcom,rpm-stats", .data = &rpm_data },
> +	{ .compatible = "qcom,rpmh-stats", .data = &rpmh_data },
> +	{ }
> +};
> +MODULE_DEVICE_TABLE(of, qcom_stats_table);
> +
> +static struct platform_driver qcom_stats = {
> +	.probe = qcom_stats_probe,
> +	.remove = qcom_stats_remove,
> +	.driver = {
> +		.name = "qcom_stats",
> +		.of_match_table = qcom_stats_table,
> +	},
> +};
> +
> +static int __init qcom_stats_init(void)
> +{
> +	return platform_driver_register(&qcom_stats);
> +}
> +late_initcall(qcom_stats_init);
> +
> +static void __exit qcom_stats_exit(void)
> +{
> +	platform_driver_unregister(&qcom_stats);
> +}
> +module_exit(qcom_stats_exit)
> +
> +MODULE_DESCRIPTION("Qualcomm Technologies, Inc. (QTI) Stats driver");
> +MODULE_LICENSE("GPL v2");
> -- 
> QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
> of Code Aurora Forum, hosted by The Linux Foundation
> 

Re: (subset) [PATCH v12 0/5] Introduce SoC sleep stats driver

[Bjorn Andersson]

On Wed, 13 Oct 2021 12:08:19 +0530, Maulik Shah wrote:
> Changes in v12:
> - Address Stephan's comments from v11
> - Rename driver and compatible to qcom,rpm(h)-stats
> - Skip reading SMEM for RPM targets
> - Make driver register in late_init to avoid -EPROBE_DEFER from smem.
> - Change size to 0x10000 for RPM targets since area contains many others stats.
> 
> [...]

Applied, thanks!

[3/5] arm64: defconfig: Enable Sleep stats driver
      commit: 551ed64388fd9a0a648eee81b534cd4765ff484a

Best regards,
-- 
Bjorn Andersson <bjorn.andersson@linaro.org>

Re: (subset) [PATCH v12 3/5] arm64: defconfig: Enable Sleep stats driver

[Bjorn Andersson]

On Wed, 13 Oct 2021 12:08:22 +0530, Maulik Shah wrote:
> Enable Sleep stats driver. The driver gives statistics for various
> low power modes on Qualcomm Technologies, Inc. (QTI) SoCs.
> 
> 

Applied, thanks!

[3/5] arm64: defconfig: Enable Sleep stats driver
      commit: 551ed64388fd9a0a648eee81b534cd4765ff484a

Best regards,
-- 
Bjorn Andersson <bjorn.andersson@linaro.org>

Re: [PATCH v12 0/5] Introduce SoC sleep stats driver

[patchwork-bot+linux-arm-msm]

Hello:

This series was applied to qcom/linux.git (for-next)
by Bjorn Andersson <bjorn.andersson@linaro.org>:

On Wed, 13 Oct 2021 12:08:19 +0530 you wrote:
> Changes in v12:
> - Address Stephan's comments from v11
> - Rename driver and compatible to qcom,rpm(h)-stats
> - Skip reading SMEM for RPM targets
> - Make driver register in late_init to avoid -EPROBE_DEFER from smem.
> - Change size to 0x10000 for RPM targets since area contains many others stats.
> 
> [...]

Here is the summary with links:
  - [v12,1/5] dt-bindings: Introduce QCOM Sleep stats bindings
    https://git.kernel.org/qcom/c/ac3f1ee77cbe
  - [v12,2/5] soc: qcom: Add Sleep stats driver
    https://git.kernel.org/qcom/c/1d7724690344
  - [v12,3/5] arm64: defconfig: Enable Sleep stats driver
    https://git.kernel.org/qcom/c/551ed64388fd
  - [v12,4/5] arm64: dts: qcom: Enable RPMh Sleep stats
    https://git.kernel.org/qcom/c/47cb6a068409
  - [v12,5/5] arm64: dts: qcom: Enable RPM Sleep stats
    https://git.kernel.org/qcom/c/290bc6846547

You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

Re: [PATCH v12 2/5] soc: qcom: Add Sleep stats driver

[Maulik Shah]

Hi Shawn,

On 10/15/2021 7:28 PM, Shawn Guo wrote:
> Hi Mahesh,
> 
> Thanks for the work!
> 
> While I tested it good on SDM660 and MSM8939, I have a question
> regarding 'vlow' mode below.
> 
> On Wed, Oct 13, 2021 at 12:08:21PM +0530, Maulik Shah wrote:
>> From: Mahesh Sivasubramanian <msivasub@codeaurora.org>
>>
>> Let's add a driver to read the stats from remote processor and
>> export to debugfs.
>>
>> The driver creates "qcom_sleep_stats" directory in debugfs and
>> adds files for various low power mode available. Below is sample
>> output with command
>>
>> cat /sys/kernel/debug/qcom_sleep_stats/ddr
>> count = 0
>> Last Entered At = 0
>> Last Exited At = 0
>> Accumulated Duration = 0
>>
>> Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
>> Signed-off-by: Lina Iyer <ilina@codeaurora.org>
>> [mkshah: add subsystem sleep stats, create one file for each stat]
>> Signed-off-by: Maulik Shah <mkshah@codeaurora.org>
>> ---
>>   drivers/soc/qcom/Kconfig      |  10 ++
>>   drivers/soc/qcom/Makefile     |   1 +
>>   drivers/soc/qcom/qcom_stats.c | 277 ++++++++++++++++++++++++++++++++++++++++++
>>   3 files changed, 288 insertions(+)
>>   create mode 100644 drivers/soc/qcom/qcom_stats.c
>>
>> diff --git a/drivers/soc/qcom/Kconfig b/drivers/soc/qcom/Kconfig
>> index abfef20..3cf3a6b 100644
>> --- a/drivers/soc/qcom/Kconfig
>> +++ b/drivers/soc/qcom/Kconfig
>> @@ -199,6 +199,16 @@ config QCOM_SPM
>>   	  to manage cores, L2 low power modes and to configure the internal
>>   	  Adaptive Voltage Scaler parameters, where supported.
>>   
>> +config QCOM_STATS
>> +	tristate "Qualcomm Technologies, Inc. (QTI) Sleep stats driver"
>> +	depends on (ARCH_QCOM && DEBUG_FS) || COMPILE_TEST
>> +	depends on QCOM_SMEM
>> +	help
>> +	  Qualcomm Technologies, Inc. (QTI) Sleep stats driver to read
>> +	  the shared memory exported by the remote processor related to
>> +	  various SoC level low power modes statistics and export to debugfs
>> +	  interface.
>> +
>>   config QCOM_WCNSS_CTRL
>>   	tristate "Qualcomm WCNSS control driver"
>>   	depends on ARCH_QCOM || COMPILE_TEST
>> diff --git a/drivers/soc/qcom/Makefile b/drivers/soc/qcom/Makefile
>> index 24514c7..70d5de6 100644
>> --- a/drivers/soc/qcom/Makefile
>> +++ b/drivers/soc/qcom/Makefile
>> @@ -21,6 +21,7 @@ obj-$(CONFIG_QCOM_SMP2P)	+= smp2p.o
>>   obj-$(CONFIG_QCOM_SMSM)	+= smsm.o
>>   obj-$(CONFIG_QCOM_SOCINFO)	+= socinfo.o
>>   obj-$(CONFIG_QCOM_SPM)		+= spm.o
>> +obj-$(CONFIG_QCOM_STATS)	+= qcom_stats.o
>>   obj-$(CONFIG_QCOM_WCNSS_CTRL) += wcnss_ctrl.o
>>   obj-$(CONFIG_QCOM_APR) += apr.o
>>   obj-$(CONFIG_QCOM_LLCC) += llcc-qcom.o
>> diff --git a/drivers/soc/qcom/qcom_stats.c b/drivers/soc/qcom/qcom_stats.c
>> new file mode 100644
>> index 0000000..817505b
>> --- /dev/null
>> +++ b/drivers/soc/qcom/qcom_stats.c
>> @@ -0,0 +1,277 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * Copyright (c) 2011-2021, The Linux Foundation. All rights reserved.
>> + */
>> +
>> +#include <linux/debugfs.h>
>> +#include <linux/device.h>
>> +#include <linux/io.h>
>> +#include <linux/module.h>
>> +#include <linux/of.h>
>> +#include <linux/platform_device.h>
>> +#include <linux/seq_file.h>
>> +
>> +#include <linux/soc/qcom/smem.h>
>> +#include <clocksource/arm_arch_timer.h>
>> +
>> +#define RPM_DYNAMIC_ADDR	0x14
>> +#define RPM_DYNAMIC_ADDR_MASK	0xFFFF
>> +
>> +#define STAT_TYPE_OFFSET	0x0
>> +#define COUNT_OFFSET		0x4
>> +#define LAST_ENTERED_AT_OFFSET	0x8
>> +#define LAST_EXITED_AT_OFFSET	0x10
>> +#define ACCUMULATED_OFFSET	0x18
>> +#define CLIENT_VOTES_OFFSET	0x1c
>> +
>> +struct subsystem_data {
>> +	const char *name;
>> +	u32 smem_item;
>> +	u32 pid;
>> +};
>> +
>> +static const struct subsystem_data subsystems[] = {
>> +	{ "modem", 605, 1 },
>> +	{ "wpss", 605, 13 },
>> +	{ "adsp", 606, 2 },
>> +	{ "cdsp", 607, 5 },
>> +	{ "slpi", 608, 3 },
>> +	{ "gpu", 609, 0 },
>> +	{ "display", 610, 0 },
>> +	{ "adsp_island", 613, 2 },
>> +	{ "slpi_island", 613, 3 },
>> +};
>> +
>> +struct stats_config {
>> +	size_t stats_offset;
>> +	size_t num_records;
>> +	bool appended_stats_avail;
>> +	bool dynamic_offset;
>> +	bool subsystem_stats_in_smem;
>> +};
>> +
>> +struct stats_data {
>> +	bool appended_stats_avail;
>> +	void __iomem *base;
>> +};
>> +
>> +struct sleep_stats {
>> +	u32 stat_type;
>> +	u32 count;
>> +	u64 last_entered_at;
>> +	u64 last_exited_at;
>> +	u64 accumulated;
>> +};
>> +
>> +struct appended_stats {
>> +	u32 client_votes;
>> +	u32 reserved[3];
>> +};
>> +
>> +static void qcom_print_stats(struct seq_file *s, const struct sleep_stats *stat)
>> +{
>> +	u64 accumulated = stat->accumulated;
>> +	/*
>> +	 * If a subsystem is in sleep when reading the sleep stats adjust
>> +	 * the accumulated sleep duration to show actual sleep time.
>> +	 */
>> +	if (stat->last_entered_at > stat->last_exited_at)
>> +		accumulated += arch_timer_read_counter() - stat->last_entered_at;
>> +
>> +	seq_printf(s, "Count: %u\n", stat->count);
>> +	seq_printf(s, "Last Entered At: %llu\n", stat->last_entered_at);
>> +	seq_printf(s, "Last Exited At: %llu\n", stat->last_exited_at);
>> +	seq_printf(s, "Accumulated Duration: %llu\n", accumulated);
>> +}
>> +
>> +static int qcom_subsystem_sleep_stats_show(struct seq_file *s, void *unused)
>> +{
>> +	struct subsystem_data *subsystem = s->private;
>> +	struct sleep_stats *stat;
>> +
>> +	/* Items are allocated lazily, so lookup pointer each time */
>> +	stat = qcom_smem_get(subsystem->pid, subsystem->smem_item, NULL);
>> +	if (IS_ERR(stat))
>> +		return -EIO;
>> +
>> +	qcom_print_stats(s, stat);
>> +
>> +	return 0;
>> +}
>> +
>> +static int qcom_soc_sleep_stats_show(struct seq_file *s, void *unused)
>> +{
>> +	struct stats_data *d = s->private;
>> +	void __iomem *reg = d->base;
>> +	struct sleep_stats stat;
>> +
>> +	memcpy_fromio(&stat, reg, sizeof(stat));
>> +	qcom_print_stats(s, &stat);
>> +
>> +	if (d->appended_stats_avail) {
>> +		struct appended_stats votes;
>> +
>> +		memcpy_fromio(&votes, reg + CLIENT_VOTES_OFFSET, sizeof(votes));
>> +		seq_printf(s, "Client Votes: %#x\n", votes.client_votes);
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +DEFINE_SHOW_ATTRIBUTE(qcom_soc_sleep_stats);
>> +DEFINE_SHOW_ATTRIBUTE(qcom_subsystem_sleep_stats);
>> +
>> +static void qcom_create_soc_sleep_stat_files(struct dentry *root, void __iomem *reg,
>> +					     struct stats_data *d,
>> +					     const struct stats_config *config)
>> +{
>> +	char stat_type[sizeof(u32) + 1] = {0};
>> +	size_t stats_offset = config->stats_offset;
>> +	u32 offset = 0, type;
>> +	int i, j;
>> +
>> +	/*
>> +	 * On RPM targets, stats offset location is dynamic and changes from target
>> +	 * to target and sometimes from build to build for same target.
>> +	 *
>> +	 * In such cases the dynamic address is present at 0x14 offset from base
>> +	 * address in devicetree. The last 16bits indicates the stats_offset.
>> +	 */
>> +	if (config->dynamic_offset) {
>> +		stats_offset = readl(reg + RPM_DYNAMIC_ADDR);
>> +		stats_offset &= RPM_DYNAMIC_ADDR_MASK;
>> +	}
>> +
>> +	for (i = 0; i < config->num_records; i++) {
>> +		d[i].base = reg + offset + stats_offset;
>> +
>> +		/*
>> +		 * Read the low power mode name and create debugfs file for it.
>> +		 * The names read could be of below,
>> +		 * (may change depending on low power mode supported).
>> +		 * For rpmh-sleep-stats: "aosd", "cxsd" and "ddr".
>> +		 * For rpm-sleep-stats: "vmin" and "vlow".
> 
> It reports 'vmin' and 'xosd' on MSM8939, 'vmin' and 'vlow' on SDM660.
> I know that 'vmin' is VDD Minimization mode, and 'xosd' is XO Shutdown
> mode.  But I'm not sure about 'vlow' mode.  Could you share some
> information regarding what this low power mode is, and how it differs
> from 'vmin' and 'xosd'?

vlow and xosd are same.
vmin is xosd plus voltage minimization of chip, memory rails.

Thanks,
Maulik

> 
> Also I guess XO Shutdown mode is still available on SoCs that report
> 'vlow' instead of 'xosd', just like SDM660.  Is there any way we can
> still check the stats of XO Shutdown mode on these SoCs?  Thanks!
> 
> Shawn
> 
>> +		 */
>> +		type = readl(d[i].base);
>> +		for (j = 0; j < sizeof(u32); j++) {
>> +			stat_type[j] = type & 0xff;
>> +			type = type >> 8;
>> +		}
>> +		strim(stat_type);
>> +		debugfs_create_file(stat_type, 0400, root, &d[i],
>> +				    &qcom_soc_sleep_stats_fops);
>> +
>> +		offset += sizeof(struct sleep_stats);
>> +		if (d[i].appended_stats_avail)
>> +			offset += sizeof(struct appended_stats);
>> +	}
>> +}
>> +
>> +static void qcom_create_subsystem_stat_files(struct dentry *root,
>> +					     const struct stats_config *config)
>> +{
>> +	const struct sleep_stats *stat;
>> +	int i;
>> +
>> +	if (!config->subsystem_stats_in_smem)
>> +		return;
>> +
>> +	for (i = 0; i < ARRAY_SIZE(subsystems); i++) {
>> +		stat = qcom_smem_get(subsystems[i].pid, subsystems[i].smem_item, NULL);
>> +		if (IS_ERR(stat))
>> +			continue;
>> +
>> +		debugfs_create_file(subsystems[i].name, 0400, root, (void *)&subsystems[i],
>> +				    &qcom_subsystem_sleep_stats_fops);
>> +	}
>> +}
>> +
>> +static int qcom_stats_probe(struct platform_device *pdev)
>> +{
>> +	void __iomem *reg;
>> +	struct dentry *root;
>> +	const struct stats_config *config;
>> +	struct stats_data *d;
>> +	int i;
>> +
>> +	config = device_get_match_data(&pdev->dev);
>> +	if (!config)
>> +		return -ENODEV;
>> +
>> +	reg = devm_platform_get_and_ioremap_resource(pdev, 0, NULL);
>> +	if (IS_ERR(reg))
>> +		return -ENOMEM;
>> +
>> +	d = devm_kcalloc(&pdev->dev, config->num_records,
>> +			 sizeof(*d), GFP_KERNEL);
>> +	if (!d)
>> +		return -ENOMEM;
>> +
>> +	for (i = 0; i < config->num_records; i++)
>> +		d[i].appended_stats_avail = config->appended_stats_avail;
>> +
>> +	root = debugfs_create_dir("qcom_stats", NULL);
>> +
>> +	qcom_create_subsystem_stat_files(root, config);
>> +	qcom_create_soc_sleep_stat_files(root, reg, d, config);
>> +
>> +	platform_set_drvdata(pdev, root);
>> +
>> +	return 0;
>> +}
>> +
>> +static int qcom_stats_remove(struct platform_device *pdev)
>> +{
>> +	struct dentry *root = platform_get_drvdata(pdev);
>> +
>> +	debugfs_remove_recursive(root);
>> +
>> +	return 0;
>> +}
>> +
>> +static const struct stats_config rpm_data = {
>> +	.stats_offset = 0,
>> +	.num_records = 2,
>> +	.appended_stats_avail = true,
>> +	.dynamic_offset = true,
>> +	.subsystem_stats_in_smem = false,
>> +};
>> +
>> +static const struct stats_config rpmh_data = {
>> +	.stats_offset = 0x48,
>> +	.num_records = 3,
>> +	.appended_stats_avail = false,
>> +	.dynamic_offset = false,
>> +	.subsystem_stats_in_smem = true,
>> +};
>> +
>> +static const struct of_device_id qcom_stats_table[] = {
>> +	{ .compatible = "qcom,rpm-stats", .data = &rpm_data },
>> +	{ .compatible = "qcom,rpmh-stats", .data = &rpmh_data },
>> +	{ }
>> +};
>> +MODULE_DEVICE_TABLE(of, qcom_stats_table);
>> +
>> +static struct platform_driver qcom_stats = {
>> +	.probe = qcom_stats_probe,
>> +	.remove = qcom_stats_remove,
>> +	.driver = {
>> +		.name = "qcom_stats",
>> +		.of_match_table = qcom_stats_table,
>> +	},
>> +};
>> +
>> +static int __init qcom_stats_init(void)
>> +{
>> +	return platform_driver_register(&qcom_stats);
>> +}
>> +late_initcall(qcom_stats_init);
>> +
>> +static void __exit qcom_stats_exit(void)
>> +{
>> +	platform_driver_unregister(&qcom_stats);
>> +}
>> +module_exit(qcom_stats_exit)
>> +
>> +MODULE_DESCRIPTION("Qualcomm Technologies, Inc. (QTI) Stats driver");
>> +MODULE_LICENSE("GPL v2");
>> -- 
>> QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
>> of Code Aurora Forum, hosted by The Linux Foundation
>>

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH v12 2/5] soc: qcom: Add Sleep stats driver

[Shawn Guo]

Hi Maulik,

On Wed, Oct 13, 2021 at 12:08:21PM +0530, Maulik Shah wrote:
> From: Mahesh Sivasubramanian <msivasub@codeaurora.org>
> 
> Let's add a driver to read the stats from remote processor and
> export to debugfs.
> 
> The driver creates "qcom_sleep_stats" directory in debugfs and
> adds files for various low power mode available. Below is sample
> output with command
> 
> cat /sys/kernel/debug/qcom_sleep_stats/ddr
> count = 0
> Last Entered At = 0
> Last Exited At = 0
> Accumulated Duration = 0
> 
> Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
> Signed-off-by: Lina Iyer <ilina@codeaurora.org>
> [mkshah: add subsystem sleep stats, create one file for each stat]
> Signed-off-by: Maulik Shah <mkshah@codeaurora.org>
> ---
>  drivers/soc/qcom/Kconfig      |  10 ++
>  drivers/soc/qcom/Makefile     |   1 +
>  drivers/soc/qcom/qcom_stats.c | 277 ++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 288 insertions(+)
>  create mode 100644 drivers/soc/qcom/qcom_stats.c
> 
> diff --git a/drivers/soc/qcom/Kconfig b/drivers/soc/qcom/Kconfig
> index abfef20..3cf3a6b 100644
> --- a/drivers/soc/qcom/Kconfig
> +++ b/drivers/soc/qcom/Kconfig
> @@ -199,6 +199,16 @@ config QCOM_SPM
>  	  to manage cores, L2 low power modes and to configure the internal
>  	  Adaptive Voltage Scaler parameters, where supported.
>  
> +config QCOM_STATS
> +	tristate "Qualcomm Technologies, Inc. (QTI) Sleep stats driver"
> +	depends on (ARCH_QCOM && DEBUG_FS) || COMPILE_TEST
> +	depends on QCOM_SMEM
> +	help
> +	  Qualcomm Technologies, Inc. (QTI) Sleep stats driver to read
> +	  the shared memory exported by the remote processor related to
> +	  various SoC level low power modes statistics and export to debugfs
> +	  interface.
> +
>  config QCOM_WCNSS_CTRL
>  	tristate "Qualcomm WCNSS control driver"
>  	depends on ARCH_QCOM || COMPILE_TEST
> diff --git a/drivers/soc/qcom/Makefile b/drivers/soc/qcom/Makefile
> index 24514c7..70d5de6 100644
> --- a/drivers/soc/qcom/Makefile
> +++ b/drivers/soc/qcom/Makefile
> @@ -21,6 +21,7 @@ obj-$(CONFIG_QCOM_SMP2P)	+= smp2p.o
>  obj-$(CONFIG_QCOM_SMSM)	+= smsm.o
>  obj-$(CONFIG_QCOM_SOCINFO)	+= socinfo.o
>  obj-$(CONFIG_QCOM_SPM)		+= spm.o
> +obj-$(CONFIG_QCOM_STATS)	+= qcom_stats.o
>  obj-$(CONFIG_QCOM_WCNSS_CTRL) += wcnss_ctrl.o
>  obj-$(CONFIG_QCOM_APR) += apr.o
>  obj-$(CONFIG_QCOM_LLCC) += llcc-qcom.o
> diff --git a/drivers/soc/qcom/qcom_stats.c b/drivers/soc/qcom/qcom_stats.c
> new file mode 100644
> index 0000000..817505b
> --- /dev/null
> +++ b/drivers/soc/qcom/qcom_stats.c
> @@ -0,0 +1,277 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Copyright (c) 2011-2021, The Linux Foundation. All rights reserved.
> + */
> +
> +#include <linux/debugfs.h>
> +#include <linux/device.h>
> +#include <linux/io.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/platform_device.h>
> +#include <linux/seq_file.h>
> +
> +#include <linux/soc/qcom/smem.h>
> +#include <clocksource/arm_arch_timer.h>
> +
> +#define RPM_DYNAMIC_ADDR	0x14
> +#define RPM_DYNAMIC_ADDR_MASK	0xFFFF
> +
> +#define STAT_TYPE_OFFSET	0x0
> +#define COUNT_OFFSET		0x4
> +#define LAST_ENTERED_AT_OFFSET	0x8
> +#define LAST_EXITED_AT_OFFSET	0x10
> +#define ACCUMULATED_OFFSET	0x18
> +#define CLIENT_VOTES_OFFSET	0x1c

From downstream 'struct msm_rpm_stats_data' [1], it looks that
CLIENT_VOTES_OFFSET should be 0x20?

Shawn

[1] https://source.codeaurora.org/quic/la/kernel/msm-4.19/tree/drivers/soc/qcom/rpm_stats.c?h=kernel.lnx.4.19.r17-rel#n46

> +
> +struct subsystem_data {
> +	const char *name;
> +	u32 smem_item;
> +	u32 pid;
> +};
> +
> +static const struct subsystem_data subsystems[] = {
> +	{ "modem", 605, 1 },
> +	{ "wpss", 605, 13 },
> +	{ "adsp", 606, 2 },
> +	{ "cdsp", 607, 5 },
> +	{ "slpi", 608, 3 },
> +	{ "gpu", 609, 0 },
> +	{ "display", 610, 0 },
> +	{ "adsp_island", 613, 2 },
> +	{ "slpi_island", 613, 3 },
> +};
> +
> +struct stats_config {
> +	size_t stats_offset;
> +	size_t num_records;
> +	bool appended_stats_avail;
> +	bool dynamic_offset;
> +	bool subsystem_stats_in_smem;
> +};
> +
> +struct stats_data {
> +	bool appended_stats_avail;
> +	void __iomem *base;
> +};
> +
> +struct sleep_stats {
> +	u32 stat_type;
> +	u32 count;
> +	u64 last_entered_at;
> +	u64 last_exited_at;
> +	u64 accumulated;
> +};
> +
> +struct appended_stats {
> +	u32 client_votes;
> +	u32 reserved[3];
> +};
> +
> +static void qcom_print_stats(struct seq_file *s, const struct sleep_stats *stat)
> +{
> +	u64 accumulated = stat->accumulated;
> +	/*
> +	 * If a subsystem is in sleep when reading the sleep stats adjust
> +	 * the accumulated sleep duration to show actual sleep time.
> +	 */
> +	if (stat->last_entered_at > stat->last_exited_at)
> +		accumulated += arch_timer_read_counter() - stat->last_entered_at;
> +
> +	seq_printf(s, "Count: %u\n", stat->count);
> +	seq_printf(s, "Last Entered At: %llu\n", stat->last_entered_at);
> +	seq_printf(s, "Last Exited At: %llu\n", stat->last_exited_at);
> +	seq_printf(s, "Accumulated Duration: %llu\n", accumulated);
> +}
> +
> +static int qcom_subsystem_sleep_stats_show(struct seq_file *s, void *unused)
> +{
> +	struct subsystem_data *subsystem = s->private;
> +	struct sleep_stats *stat;
> +
> +	/* Items are allocated lazily, so lookup pointer each time */
> +	stat = qcom_smem_get(subsystem->pid, subsystem->smem_item, NULL);
> +	if (IS_ERR(stat))
> +		return -EIO;
> +
> +	qcom_print_stats(s, stat);
> +
> +	return 0;
> +}
> +
> +static int qcom_soc_sleep_stats_show(struct seq_file *s, void *unused)
> +{
> +	struct stats_data *d = s->private;
> +	void __iomem *reg = d->base;
> +	struct sleep_stats stat;
> +
> +	memcpy_fromio(&stat, reg, sizeof(stat));
> +	qcom_print_stats(s, &stat);
> +
> +	if (d->appended_stats_avail) {
> +		struct appended_stats votes;
> +
> +		memcpy_fromio(&votes, reg + CLIENT_VOTES_OFFSET, sizeof(votes));
> +		seq_printf(s, "Client Votes: %#x\n", votes.client_votes);
> +	}
> +
> +	return 0;
> +}
> +
> +DEFINE_SHOW_ATTRIBUTE(qcom_soc_sleep_stats);
> +DEFINE_SHOW_ATTRIBUTE(qcom_subsystem_sleep_stats);
> +
> +static void qcom_create_soc_sleep_stat_files(struct dentry *root, void __iomem *reg,
> +					     struct stats_data *d,
> +					     const struct stats_config *config)
> +{
> +	char stat_type[sizeof(u32) + 1] = {0};
> +	size_t stats_offset = config->stats_offset;
> +	u32 offset = 0, type;
> +	int i, j;
> +
> +	/*
> +	 * On RPM targets, stats offset location is dynamic and changes from target
> +	 * to target and sometimes from build to build for same target.
> +	 *
> +	 * In such cases the dynamic address is present at 0x14 offset from base
> +	 * address in devicetree. The last 16bits indicates the stats_offset.
> +	 */
> +	if (config->dynamic_offset) {
> +		stats_offset = readl(reg + RPM_DYNAMIC_ADDR);
> +		stats_offset &= RPM_DYNAMIC_ADDR_MASK;
> +	}
> +
> +	for (i = 0; i < config->num_records; i++) {
> +		d[i].base = reg + offset + stats_offset;
> +
> +		/*
> +		 * Read the low power mode name and create debugfs file for it.
> +		 * The names read could be of below,
> +		 * (may change depending on low power mode supported).
> +		 * For rpmh-sleep-stats: "aosd", "cxsd" and "ddr".
> +		 * For rpm-sleep-stats: "vmin" and "vlow".
> +		 */
> +		type = readl(d[i].base);
> +		for (j = 0; j < sizeof(u32); j++) {
> +			stat_type[j] = type & 0xff;
> +			type = type >> 8;
> +		}
> +		strim(stat_type);
> +		debugfs_create_file(stat_type, 0400, root, &d[i],
> +				    &qcom_soc_sleep_stats_fops);
> +
> +		offset += sizeof(struct sleep_stats);
> +		if (d[i].appended_stats_avail)
> +			offset += sizeof(struct appended_stats);
> +	}
> +}
> +
> +static void qcom_create_subsystem_stat_files(struct dentry *root,
> +					     const struct stats_config *config)
> +{
> +	const struct sleep_stats *stat;
> +	int i;
> +
> +	if (!config->subsystem_stats_in_smem)
> +		return;
> +
> +	for (i = 0; i < ARRAY_SIZE(subsystems); i++) {
> +		stat = qcom_smem_get(subsystems[i].pid, subsystems[i].smem_item, NULL);
> +		if (IS_ERR(stat))
> +			continue;
> +
> +		debugfs_create_file(subsystems[i].name, 0400, root, (void *)&subsystems[i],
> +				    &qcom_subsystem_sleep_stats_fops);
> +	}
> +}
> +
> +static int qcom_stats_probe(struct platform_device *pdev)
> +{
> +	void __iomem *reg;
> +	struct dentry *root;
> +	const struct stats_config *config;
> +	struct stats_data *d;
> +	int i;
> +
> +	config = device_get_match_data(&pdev->dev);
> +	if (!config)
> +		return -ENODEV;
> +
> +	reg = devm_platform_get_and_ioremap_resource(pdev, 0, NULL);
> +	if (IS_ERR(reg))
> +		return -ENOMEM;
> +
> +	d = devm_kcalloc(&pdev->dev, config->num_records,
> +			 sizeof(*d), GFP_KERNEL);
> +	if (!d)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < config->num_records; i++)
> +		d[i].appended_stats_avail = config->appended_stats_avail;
> +
> +	root = debugfs_create_dir("qcom_stats", NULL);
> +
> +	qcom_create_subsystem_stat_files(root, config);
> +	qcom_create_soc_sleep_stat_files(root, reg, d, config);
> +
> +	platform_set_drvdata(pdev, root);
> +
> +	return 0;
> +}
> +
> +static int qcom_stats_remove(struct platform_device *pdev)
> +{
> +	struct dentry *root = platform_get_drvdata(pdev);
> +
> +	debugfs_remove_recursive(root);
> +
> +	return 0;
> +}
> +
> +static const struct stats_config rpm_data = {
> +	.stats_offset = 0,
> +	.num_records = 2,
> +	.appended_stats_avail = true,
> +	.dynamic_offset = true,
> +	.subsystem_stats_in_smem = false,
> +};
> +
> +static const struct stats_config rpmh_data = {
> +	.stats_offset = 0x48,
> +	.num_records = 3,
> +	.appended_stats_avail = false,
> +	.dynamic_offset = false,
> +	.subsystem_stats_in_smem = true,
> +};
> +
> +static const struct of_device_id qcom_stats_table[] = {
> +	{ .compatible = "qcom,rpm-stats", .data = &rpm_data },
> +	{ .compatible = "qcom,rpmh-stats", .data = &rpmh_data },
> +	{ }
> +};
> +MODULE_DEVICE_TABLE(of, qcom_stats_table);
> +
> +static struct platform_driver qcom_stats = {
> +	.probe = qcom_stats_probe,
> +	.remove = qcom_stats_remove,
> +	.driver = {
> +		.name = "qcom_stats",
> +		.of_match_table = qcom_stats_table,
> +	},
> +};
> +
> +static int __init qcom_stats_init(void)
> +{
> +	return platform_driver_register(&qcom_stats);
> +}
> +late_initcall(qcom_stats_init);
> +
> +static void __exit qcom_stats_exit(void)
> +{
> +	platform_driver_unregister(&qcom_stats);
> +}
> +module_exit(qcom_stats_exit)
> +
> +MODULE_DESCRIPTION("Qualcomm Technologies, Inc. (QTI) Stats driver");
> +MODULE_LICENSE("GPL v2");
> -- 
> QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
> of Code Aurora Forum, hosted by The Linux Foundation
> 

Re: [PATCH v12 2/5] soc: qcom: Add Sleep stats driver

[Shawn Guo]

On Mon, Oct 18, 2021 at 07:45:30PM +0530, Maulik Shah wrote:
> > > +static void qcom_create_soc_sleep_stat_files(struct dentry *root, void __iomem *reg,
> > > +					     struct stats_data *d,
> > > +					     const struct stats_config *config)
> > > +{
> > > +	char stat_type[sizeof(u32) + 1] = {0};
> > > +	size_t stats_offset = config->stats_offset;
> > > +	u32 offset = 0, type;
> > > +	int i, j;
> > > +
> > > +	/*
> > > +	 * On RPM targets, stats offset location is dynamic and changes from target
> > > +	 * to target and sometimes from build to build for same target.
> > > +	 *
> > > +	 * In such cases the dynamic address is present at 0x14 offset from base
> > > +	 * address in devicetree. The last 16bits indicates the stats_offset.
> > > +	 */
> > > +	if (config->dynamic_offset) {
> > > +		stats_offset = readl(reg + RPM_DYNAMIC_ADDR);
> > > +		stats_offset &= RPM_DYNAMIC_ADDR_MASK;
> > > +	}
> > > +
> > > +	for (i = 0; i < config->num_records; i++) {
> > > +		d[i].base = reg + offset + stats_offset;
> > > +
> > > +		/*
> > > +		 * Read the low power mode name and create debugfs file for it.
> > > +		 * The names read could be of below,
> > > +		 * (may change depending on low power mode supported).
> > > +		 * For rpmh-sleep-stats: "aosd", "cxsd" and "ddr".
> > > +		 * For rpm-sleep-stats: "vmin" and "vlow".
> > 
> > It reports 'vmin' and 'xosd' on MSM8939, 'vmin' and 'vlow' on SDM660.
> > I know that 'vmin' is VDD Minimization mode, and 'xosd' is XO Shutdown
> > mode.  But I'm not sure about 'vlow' mode.  Could you share some
> > information regarding what this low power mode is, and how it differs
> > from 'vmin' and 'xosd'?
> 
> vlow and xosd are same.
> vmin is xosd plus voltage minimization of chip, memory rails.

Thanks much for the info, Maulik!

I'm running your driver on qcm2290 and trying to reach vlow mode.

# cat /sys/kernel/debug/qcom_sleep_stats/vlow
Count: 0
Last Entered At: 0
Last Exited At: 0
Accumulated Duration: 0
Client Votes: 0x81
# echo mem > /sys/power/state 
[  551.446603] PM: suspend entry (s2idle)
[  551.450948] Filesystems sync: 0.000 seconds
[  551.462828] Freezing user space processes ... (elapsed 0.002 seconds) done.
[  551.472276] OOM killer disabled.
[  551.475556] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
[  551.484461] printk: Suspending console(s) (use no_console_suspend to debug)
[  551.561280] OOM killer enabled.
[  551.564461] Restarting tasks ... done.
[  551.569652] PM: suspend exit
# cat /sys/kernel/debug/qcom_sleep_stats/vlow
Count: 0
Last Entered At: 0
Last Exited At: 0
Accumulated Duration: 0
Client Votes: 0x818081

The count doesn't increases along with suspend/resume cycle at the
moment.  But as you can see, 'Client Votes' field changes.  If possible,
could you shed some light on what this means?

As the comparison, I'm also running the downstream 'rpm_master_stats'
driver in the same kernel, and the 'xo_count' field of APSS does
increase along with suspend/resume cycle.  May I ask some advices what
I'm possibly missing and thus getting different result between 'vlow'
and 'rpm_master_stats' report?

# cat /sys/kernel/debug/rpm_master_stats 
APSS
        shutdown_req:0x37EA3CC74
        wakeup_ind:0x0
        bringup_req:0x37F041958
        bringup_ack:0x37F042D54
        xo_last_entered_at:0x286FF36AC
        xo_last_exited_at:0x28AF94178
        xo_accumulated_duration:0x3EDD55B
        last_sleep_transition_duration:0x122f
        last_wake_transition_duration:0x11f8
        xo_count:0x1
        wakeup_reason:0x0
        numshutdowns:0x641
        active_cores:0x1
                core0
MPSS
        shutdown_req:0x0
        wakeup_ind:0x0
        bringup_req:0x0
        bringup_ack:0x0
        xo_last_entered_at:0x0
        xo_last_exited_at:0x0
        xo_accumulated_duration:0x0
        last_sleep_transition_duration:0x0
        last_wake_transition_duration:0x0
        xo_count:0x0
        wakeup_reason:0x0
        numshutdowns:0x0
        active_cores:0x1
                core0
ADSP
        shutdown_req:0x0
        wakeup_ind:0x0
        bringup_req:0x0
        bringup_ack:0x0
        xo_last_entered_at:0x0
        xo_last_exited_at:0x0
        xo_accumulated_duration:0x0
        last_sleep_transition_duration:0x0
        last_wake_transition_duration:0x0
        xo_count:0x0
        wakeup_reason:0x0
        numshutdowns:0x0
        active_cores:0x1
                core0
CDSP
        shutdown_req:0x0
        wakeup_ind:0x0
        bringup_req:0x0
        bringup_ack:0x0
        xo_last_entered_at:0x0
        xo_last_exited_at:0x0
        xo_accumulated_duration:0x0
        last_sleep_transition_duration:0x0
        last_wake_transition_duration:0x0
        xo_count:0x0
        wakeup_reason:0x0
        numshutdowns:0x0
        active_cores:0x0
TZ
        shutdown_req:0x0
        wakeup_ind:0x0
        bringup_req:0x0
        bringup_ack:0x0
        xo_last_entered_at:0x0
        xo_last_exited_at:0x0
        xo_accumulated_duration:0x0
        last_sleep_transition_duration:0x0
        last_wake_transition_duration:0x0
        xo_count:0x0
        wakeup_reason:0x0
        numshutdowns:0x0
        active_cores:0x0

Any comment or suggestion would be much appreciated!

Shawn

Re: [PATCH v12 2/5] soc: qcom: Add Sleep stats driver

[Maulik Shah]

Hi Shawn,

On 10/19/2021 2:54 PM, Shawn Guo wrote:
> Hi Maulik,
> 
> On Wed, Oct 13, 2021 at 12:08:21PM +0530, Maulik Shah wrote:
>> From: Mahesh Sivasubramanian <msivasub@codeaurora.org>
>>
>> Let's add a driver to read the stats from remote processor and
>> export to debugfs.
>>
>> The driver creates "qcom_sleep_stats" directory in debugfs and
>> adds files for various low power mode available. Below is sample
>> output with command
>>
>> cat /sys/kernel/debug/qcom_sleep_stats/ddr
>> count = 0
>> Last Entered At = 0
>> Last Exited At = 0
>> Accumulated Duration = 0
>>
>> Signed-off-by: Mahesh Sivasubramanian <msivasub@codeaurora.org>
>> Signed-off-by: Lina Iyer <ilina@codeaurora.org>
>> [mkshah: add subsystem sleep stats, create one file for each stat]
>> Signed-off-by: Maulik Shah <mkshah@codeaurora.org>
>> ---
>>   drivers/soc/qcom/Kconfig      |  10 ++
>>   drivers/soc/qcom/Makefile     |   1 +
>>   drivers/soc/qcom/qcom_stats.c | 277 ++++++++++++++++++++++++++++++++++++++++++
>>   3 files changed, 288 insertions(+)
>>   create mode 100644 drivers/soc/qcom/qcom_stats.c
>>
>> diff --git a/drivers/soc/qcom/Kconfig b/drivers/soc/qcom/Kconfig
>> index abfef20..3cf3a6b 100644
>> --- a/drivers/soc/qcom/Kconfig
>> +++ b/drivers/soc/qcom/Kconfig
>> @@ -199,6 +199,16 @@ config QCOM_SPM
>>   	  to manage cores, L2 low power modes and to configure the internal
>>   	  Adaptive Voltage Scaler parameters, where supported.
>>   
>> +config QCOM_STATS
>> +	tristate "Qualcomm Technologies, Inc. (QTI) Sleep stats driver"
>> +	depends on (ARCH_QCOM && DEBUG_FS) || COMPILE_TEST
>> +	depends on QCOM_SMEM
>> +	help
>> +	  Qualcomm Technologies, Inc. (QTI) Sleep stats driver to read
>> +	  the shared memory exported by the remote processor related to
>> +	  various SoC level low power modes statistics and export to debugfs
>> +	  interface.
>> +
>>   config QCOM_WCNSS_CTRL
>>   	tristate "Qualcomm WCNSS control driver"
>>   	depends on ARCH_QCOM || COMPILE_TEST
>> diff --git a/drivers/soc/qcom/Makefile b/drivers/soc/qcom/Makefile
>> index 24514c7..70d5de6 100644
>> --- a/drivers/soc/qcom/Makefile
>> +++ b/drivers/soc/qcom/Makefile
>> @@ -21,6 +21,7 @@ obj-$(CONFIG_QCOM_SMP2P)	+= smp2p.o
>>   obj-$(CONFIG_QCOM_SMSM)	+= smsm.o
>>   obj-$(CONFIG_QCOM_SOCINFO)	+= socinfo.o
>>   obj-$(CONFIG_QCOM_SPM)		+= spm.o
>> +obj-$(CONFIG_QCOM_STATS)	+= qcom_stats.o
>>   obj-$(CONFIG_QCOM_WCNSS_CTRL) += wcnss_ctrl.o
>>   obj-$(CONFIG_QCOM_APR) += apr.o
>>   obj-$(CONFIG_QCOM_LLCC) += llcc-qcom.o
>> diff --git a/drivers/soc/qcom/qcom_stats.c b/drivers/soc/qcom/qcom_stats.c
>> new file mode 100644
>> index 0000000..817505b
>> --- /dev/null
>> +++ b/drivers/soc/qcom/qcom_stats.c
>> @@ -0,0 +1,277 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * Copyright (c) 2011-2021, The Linux Foundation. All rights reserved.
>> + */
>> +
>> +#include <linux/debugfs.h>
>> +#include <linux/device.h>
>> +#include <linux/io.h>
>> +#include <linux/module.h>
>> +#include <linux/of.h>
>> +#include <linux/platform_device.h>
>> +#include <linux/seq_file.h>
>> +
>> +#include <linux/soc/qcom/smem.h>
>> +#include <clocksource/arm_arch_timer.h>
>> +
>> +#define RPM_DYNAMIC_ADDR	0x14
>> +#define RPM_DYNAMIC_ADDR_MASK	0xFFFF
>> +
>> +#define STAT_TYPE_OFFSET	0x0
>> +#define COUNT_OFFSET		0x4
>> +#define LAST_ENTERED_AT_OFFSET	0x8
>> +#define LAST_EXITED_AT_OFFSET	0x10
>> +#define ACCUMULATED_OFFSET	0x18
>> +#define CLIENT_VOTES_OFFSET	0x1c
> 
>  From downstream 'struct msm_rpm_stats_data' [1], it looks that
> CLIENT_VOTES_OFFSET should be 0x20?

Thanks for catching this. yeah this should be 0x20 offset.
I can submit a new patch to fix this as the current series is merged in 
linux-next.

Thanks,
Maulik

> 
> Shawn
> 
> [1] https://source.codeaurora.org/quic/la/kernel/msm-4.19/tree/drivers/soc/qcom/rpm_stats.c?h=kernel.lnx.4.19.r17-rel#n46
> 
>> +
>> +struct subsystem_data {
>> +	const char *name;
>> +	u32 smem_item;
>> +	u32 pid;
>> +};
>> +
>> +static const struct subsystem_data subsystems[] = {
>> +	{ "modem", 605, 1 },
>> +	{ "wpss", 605, 13 },
>> +	{ "adsp", 606, 2 },
>> +	{ "cdsp", 607, 5 },
>> +	{ "slpi", 608, 3 },
>> +	{ "gpu", 609, 0 },
>> +	{ "display", 610, 0 },
>> +	{ "adsp_island", 613, 2 },
>> +	{ "slpi_island", 613, 3 },
>> +};
>> +
>> +struct stats_config {
>> +	size_t stats_offset;
>> +	size_t num_records;
>> +	bool appended_stats_avail;
>> +	bool dynamic_offset;
>> +	bool subsystem_stats_in_smem;
>> +};
>> +
>> +struct stats_data {
>> +	bool appended_stats_avail;
>> +	void __iomem *base;
>> +};
>> +
>> +struct sleep_stats {
>> +	u32 stat_type;
>> +	u32 count;
>> +	u64 last_entered_at;
>> +	u64 last_exited_at;
>> +	u64 accumulated;
>> +};
>> +
>> +struct appended_stats {
>> +	u32 client_votes;
>> +	u32 reserved[3];
>> +};
>> +
>> +static void qcom_print_stats(struct seq_file *s, const struct sleep_stats *stat)
>> +{
>> +	u64 accumulated = stat->accumulated;
>> +	/*
>> +	 * If a subsystem is in sleep when reading the sleep stats adjust
>> +	 * the accumulated sleep duration to show actual sleep time.
>> +	 */
>> +	if (stat->last_entered_at > stat->last_exited_at)
>> +		accumulated += arch_timer_read_counter() - stat->last_entered_at;
>> +
>> +	seq_printf(s, "Count: %u\n", stat->count);
>> +	seq_printf(s, "Last Entered At: %llu\n", stat->last_entered_at);
>> +	seq_printf(s, "Last Exited At: %llu\n", stat->last_exited_at);
>> +	seq_printf(s, "Accumulated Duration: %llu\n", accumulated);
>> +}
>> +
>> +static int qcom_subsystem_sleep_stats_show(struct seq_file *s, void *unused)
>> +{
>> +	struct subsystem_data *subsystem = s->private;
>> +	struct sleep_stats *stat;
>> +
>> +	/* Items are allocated lazily, so lookup pointer each time */
>> +	stat = qcom_smem_get(subsystem->pid, subsystem->smem_item, NULL);
>> +	if (IS_ERR(stat))
>> +		return -EIO;
>> +
>> +	qcom_print_stats(s, stat);
>> +
>> +	return 0;
>> +}
>> +
>> +static int qcom_soc_sleep_stats_show(struct seq_file *s, void *unused)
>> +{
>> +	struct stats_data *d = s->private;
>> +	void __iomem *reg = d->base;
>> +	struct sleep_stats stat;
>> +
>> +	memcpy_fromio(&stat, reg, sizeof(stat));
>> +	qcom_print_stats(s, &stat);
>> +
>> +	if (d->appended_stats_avail) {
>> +		struct appended_stats votes;
>> +
>> +		memcpy_fromio(&votes, reg + CLIENT_VOTES_OFFSET, sizeof(votes));
>> +		seq_printf(s, "Client Votes: %#x\n", votes.client_votes);
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +DEFINE_SHOW_ATTRIBUTE(qcom_soc_sleep_stats);
>> +DEFINE_SHOW_ATTRIBUTE(qcom_subsystem_sleep_stats);
>> +
>> +static void qcom_create_soc_sleep_stat_files(struct dentry *root, void __iomem *reg,
>> +					     struct stats_data *d,
>> +					     const struct stats_config *config)
>> +{
>> +	char stat_type[sizeof(u32) + 1] = {0};
>> +	size_t stats_offset = config->stats_offset;
>> +	u32 offset = 0, type;
>> +	int i, j;
>> +
>> +	/*
>> +	 * On RPM targets, stats offset location is dynamic and changes from target
>> +	 * to target and sometimes from build to build for same target.
>> +	 *
>> +	 * In such cases the dynamic address is present at 0x14 offset from base
>> +	 * address in devicetree. The last 16bits indicates the stats_offset.
>> +	 */
>> +	if (config->dynamic_offset) {
>> +		stats_offset = readl(reg + RPM_DYNAMIC_ADDR);
>> +		stats_offset &= RPM_DYNAMIC_ADDR_MASK;
>> +	}
>> +
>> +	for (i = 0; i < config->num_records; i++) {
>> +		d[i].base = reg + offset + stats_offset;
>> +
>> +		/*
>> +		 * Read the low power mode name and create debugfs file for it.
>> +		 * The names read could be of below,
>> +		 * (may change depending on low power mode supported).
>> +		 * For rpmh-sleep-stats: "aosd", "cxsd" and "ddr".
>> +		 * For rpm-sleep-stats: "vmin" and "vlow".
>> +		 */
>> +		type = readl(d[i].base);
>> +		for (j = 0; j < sizeof(u32); j++) {
>> +			stat_type[j] = type & 0xff;
>> +			type = type >> 8;
>> +		}
>> +		strim(stat_type);
>> +		debugfs_create_file(stat_type, 0400, root, &d[i],
>> +				    &qcom_soc_sleep_stats_fops);
>> +
>> +		offset += sizeof(struct sleep_stats);
>> +		if (d[i].appended_stats_avail)
>> +			offset += sizeof(struct appended_stats);
>> +	}
>> +}
>> +
>> +static void qcom_create_subsystem_stat_files(struct dentry *root,
>> +					     const struct stats_config *config)
>> +{
>> +	const struct sleep_stats *stat;
>> +	int i;
>> +
>> +	if (!config->subsystem_stats_in_smem)
>> +		return;
>> +
>> +	for (i = 0; i < ARRAY_SIZE(subsystems); i++) {
>> +		stat = qcom_smem_get(subsystems[i].pid, subsystems[i].smem_item, NULL);
>> +		if (IS_ERR(stat))
>> +			continue;
>> +
>> +		debugfs_create_file(subsystems[i].name, 0400, root, (void *)&subsystems[i],
>> +				    &qcom_subsystem_sleep_stats_fops);
>> +	}
>> +}
>> +
>> +static int qcom_stats_probe(struct platform_device *pdev)
>> +{
>> +	void __iomem *reg;
>> +	struct dentry *root;
>> +	const struct stats_config *config;
>> +	struct stats_data *d;
>> +	int i;
>> +
>> +	config = device_get_match_data(&pdev->dev);
>> +	if (!config)
>> +		return -ENODEV;
>> +
>> +	reg = devm_platform_get_and_ioremap_resource(pdev, 0, NULL);
>> +	if (IS_ERR(reg))
>> +		return -ENOMEM;
>> +
>> +	d = devm_kcalloc(&pdev->dev, config->num_records,
>> +			 sizeof(*d), GFP_KERNEL);
>> +	if (!d)
>> +		return -ENOMEM;
>> +
>> +	for (i = 0; i < config->num_records; i++)
>> +		d[i].appended_stats_avail = config->appended_stats_avail;
>> +
>> +	root = debugfs_create_dir("qcom_stats", NULL);
>> +
>> +	qcom_create_subsystem_stat_files(root, config);
>> +	qcom_create_soc_sleep_stat_files(root, reg, d, config);
>> +
>> +	platform_set_drvdata(pdev, root);
>> +
>> +	return 0;
>> +}
>> +
>> +static int qcom_stats_remove(struct platform_device *pdev)
>> +{
>> +	struct dentry *root = platform_get_drvdata(pdev);
>> +
>> +	debugfs_remove_recursive(root);
>> +
>> +	return 0;
>> +}
>> +
>> +static const struct stats_config rpm_data = {
>> +	.stats_offset = 0,
>> +	.num_records = 2,
>> +	.appended_stats_avail = true,
>> +	.dynamic_offset = true,
>> +	.subsystem_stats_in_smem = false,
>> +};
>> +
>> +static const struct stats_config rpmh_data = {
>> +	.stats_offset = 0x48,
>> +	.num_records = 3,
>> +	.appended_stats_avail = false,
>> +	.dynamic_offset = false,
>> +	.subsystem_stats_in_smem = true,
>> +};
>> +
>> +static const struct of_device_id qcom_stats_table[] = {
>> +	{ .compatible = "qcom,rpm-stats", .data = &rpm_data },
>> +	{ .compatible = "qcom,rpmh-stats", .data = &rpmh_data },
>> +	{ }
>> +};
>> +MODULE_DEVICE_TABLE(of, qcom_stats_table);
>> +
>> +static struct platform_driver qcom_stats = {
>> +	.probe = qcom_stats_probe,
>> +	.remove = qcom_stats_remove,
>> +	.driver = {
>> +		.name = "qcom_stats",
>> +		.of_match_table = qcom_stats_table,
>> +	},
>> +};
>> +
>> +static int __init qcom_stats_init(void)
>> +{
>> +	return platform_driver_register(&qcom_stats);
>> +}
>> +late_initcall(qcom_stats_init);
>> +
>> +static void __exit qcom_stats_exit(void)
>> +{
>> +	platform_driver_unregister(&qcom_stats);
>> +}
>> +module_exit(qcom_stats_exit)
>> +
>> +MODULE_DESCRIPTION("Qualcomm Technologies, Inc. (QTI) Stats driver");
>> +MODULE_LICENSE("GPL v2");
>> -- 
>> QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member
>> of Code Aurora Forum, hosted by The Linux Foundation
>>

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH v12 2/5] soc: qcom: Add Sleep stats driver

[Maulik Shah]

Hi Shawn,

On 10/19/2021 3:17 PM, Shawn Guo wrote:
> On Mon, Oct 18, 2021 at 07:45:30PM +0530, Maulik Shah wrote:
>>>> +static void qcom_create_soc_sleep_stat_files(struct dentry *root, void __iomem *reg,
>>>> +					     struct stats_data *d,
>>>> +					     const struct stats_config *config)
>>>> +{
>>>> +	char stat_type[sizeof(u32) + 1] = {0};
>>>> +	size_t stats_offset = config->stats_offset;
>>>> +	u32 offset = 0, type;
>>>> +	int i, j;
>>>> +
>>>> +	/*
>>>> +	 * On RPM targets, stats offset location is dynamic and changes from target
>>>> +	 * to target and sometimes from build to build for same target.
>>>> +	 *
>>>> +	 * In such cases the dynamic address is present at 0x14 offset from base
>>>> +	 * address in devicetree. The last 16bits indicates the stats_offset.
>>>> +	 */
>>>> +	if (config->dynamic_offset) {
>>>> +		stats_offset = readl(reg + RPM_DYNAMIC_ADDR);
>>>> +		stats_offset &= RPM_DYNAMIC_ADDR_MASK;
>>>> +	}
>>>> +
>>>> +	for (i = 0; i < config->num_records; i++) {
>>>> +		d[i].base = reg + offset + stats_offset;
>>>> +
>>>> +		/*
>>>> +		 * Read the low power mode name and create debugfs file for it.
>>>> +		 * The names read could be of below,
>>>> +		 * (may change depending on low power mode supported).
>>>> +		 * For rpmh-sleep-stats: "aosd", "cxsd" and "ddr".
>>>> +		 * For rpm-sleep-stats: "vmin" and "vlow".
>>>
>>> It reports 'vmin' and 'xosd' on MSM8939, 'vmin' and 'vlow' on SDM660.
>>> I know that 'vmin' is VDD Minimization mode, and 'xosd' is XO Shutdown
>>> mode.  But I'm not sure about 'vlow' mode.  Could you share some
>>> information regarding what this low power mode is, and how it differs
>>> from 'vmin' and 'xosd'?
>>
>> vlow and xosd are same.
>> vmin is xosd plus voltage minimization of chip, memory rails.
> 
> Thanks much for the info, Maulik!
> 
> I'm running your driver on qcm2290 and trying to reach vlow mode.
> 
> # cat /sys/kernel/debug/qcom_sleep_stats/vlow
> Count: 0
> Last Entered At: 0
> Last Exited At: 0
> Accumulated Duration: 0
> Client Votes: 0x81
> # echo mem > /sys/power/state
> [  551.446603] PM: suspend entry (s2idle)
> [  551.450948] Filesystems sync: 0.000 seconds
> [  551.462828] Freezing user space processes ... (elapsed 0.002 seconds) done.
> [  551.472276] OOM killer disabled.
> [  551.475556] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
> [  551.484461] printk: Suspending console(s) (use no_console_suspend to debug)
> [  551.561280] OOM killer enabled.
> [  551.564461] Restarting tasks ... done.
> [  551.569652] PM: suspend exit
> # cat /sys/kernel/debug/qcom_sleep_stats/vlow
> Count: 0
> Last Entered At: 0
> Last Exited At: 0
> Accumulated Duration: 0
> Client Votes: 0x818081
> 
> The count doesn't increases along with suspend/resume cycle at the
> moment.  But as you can see, 'Client Votes' field changes.  If possible,
> could you shed some light on what this means?

The count will increase only when all the subsystems (APSS/Modem,etc) 
are in power down mode and finally RPM decides to turn off xo clock.

> 
> As the comparison, I'm also running the downstream 'rpm_master_stats'
> driver in the same kernel, and the 'xo_count' field of APSS does
> increase along with suspend/resume cycle.  May I ask some advices what
> I'm possibly missing and thus getting different result between 'vlow'
> and 'rpm_master_stats' report?

The vlow is a SoC level state whereas the rpm master stats indicate
individual subsystem state. Since you are running suspend-resume the 
APSS is going to sleep so you see xo_count incremented for it but for 
MPSS i see it does not increase (modem is not entering to low power 
mode). similarly for ADSP/CDSP it does not increment. if all of these 
subsystems goes to power down and then there is sufficient sleep time 
for the SoC then you may see vlow/vmin incrementing.

Hope this clarifies.

Thanks,
Maulik

> 
> # cat /sys/kernel/debug/rpm_master_stats
> APSS
>          shutdown_req:0x37EA3CC74
>          wakeup_ind:0x0
>          bringup_req:0x37F041958
>          bringup_ack:0x37F042D54
>          xo_last_entered_at:0x286FF36AC
>          xo_last_exited_at:0x28AF94178
>          xo_accumulated_duration:0x3EDD55B
>          last_sleep_transition_duration:0x122f
>          last_wake_transition_duration:0x11f8
>          xo_count:0x1
>          wakeup_reason:0x0
>          numshutdowns:0x641
>          active_cores:0x1
>                  core0
> MPSS
>          shutdown_req:0x0
>          wakeup_ind:0x0
>          bringup_req:0x0
>          bringup_ack:0x0
>          xo_last_entered_at:0x0
>          xo_last_exited_at:0x0
>          xo_accumulated_duration:0x0
>          last_sleep_transition_duration:0x0
>          last_wake_transition_duration:0x0
>          xo_count:0x0
>          wakeup_reason:0x0
>          numshutdowns:0x0
>          active_cores:0x1
>                  core0
> ADSP
>          shutdown_req:0x0
>          wakeup_ind:0x0
>          bringup_req:0x0
>          bringup_ack:0x0
>          xo_last_entered_at:0x0
>          xo_last_exited_at:0x0
>          xo_accumulated_duration:0x0
>          last_sleep_transition_duration:0x0
>          last_wake_transition_duration:0x0
>          xo_count:0x0
>          wakeup_reason:0x0
>          numshutdowns:0x0
>          active_cores:0x1
>                  core0
> CDSP
>          shutdown_req:0x0
>          wakeup_ind:0x0
>          bringup_req:0x0
>          bringup_ack:0x0
>          xo_last_entered_at:0x0
>          xo_last_exited_at:0x0
>          xo_accumulated_duration:0x0
>          last_sleep_transition_duration:0x0
>          last_wake_transition_duration:0x0
>          xo_count:0x0
>          wakeup_reason:0x0
>          numshutdowns:0x0
>          active_cores:0x0
> TZ
>          shutdown_req:0x0
>          wakeup_ind:0x0
>          bringup_req:0x0
>          bringup_ack:0x0
>          xo_last_entered_at:0x0
>          xo_last_exited_at:0x0
>          xo_accumulated_duration:0x0
>          last_sleep_transition_duration:0x0
>          last_wake_transition_duration:0x0
>          xo_count:0x0
>          wakeup_reason:0x0
>          numshutdowns:0x0
>          active_cores:0x0
> 
> Any comment or suggestion would be much appreciated!
> 
> Shawn
> 

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH v12 2/5] soc: qcom: Add Sleep stats driver

[Shawn Guo]

On Tue, Oct 19, 2021 at 06:16:57PM +0530, Maulik Shah wrote:
> Hi Shawn,
> 
> On 10/19/2021 3:17 PM, Shawn Guo wrote:
> > On Mon, Oct 18, 2021 at 07:45:30PM +0530, Maulik Shah wrote:
> > > > > +static void qcom_create_soc_sleep_stat_files(struct dentry *root, void __iomem *reg,
> > > > > +					     struct stats_data *d,
> > > > > +					     const struct stats_config *config)
> > > > > +{
> > > > > +	char stat_type[sizeof(u32) + 1] = {0};
> > > > > +	size_t stats_offset = config->stats_offset;
> > > > > +	u32 offset = 0, type;
> > > > > +	int i, j;
> > > > > +
> > > > > +	/*
> > > > > +	 * On RPM targets, stats offset location is dynamic and changes from target
> > > > > +	 * to target and sometimes from build to build for same target.
> > > > > +	 *
> > > > > +	 * In such cases the dynamic address is present at 0x14 offset from base
> > > > > +	 * address in devicetree. The last 16bits indicates the stats_offset.
> > > > > +	 */
> > > > > +	if (config->dynamic_offset) {
> > > > > +		stats_offset = readl(reg + RPM_DYNAMIC_ADDR);
> > > > > +		stats_offset &= RPM_DYNAMIC_ADDR_MASK;
> > > > > +	}
> > > > > +
> > > > > +	for (i = 0; i < config->num_records; i++) {
> > > > > +		d[i].base = reg + offset + stats_offset;
> > > > > +
> > > > > +		/*
> > > > > +		 * Read the low power mode name and create debugfs file for it.
> > > > > +		 * The names read could be of below,
> > > > > +		 * (may change depending on low power mode supported).
> > > > > +		 * For rpmh-sleep-stats: "aosd", "cxsd" and "ddr".
> > > > > +		 * For rpm-sleep-stats: "vmin" and "vlow".
> > > > 
> > > > It reports 'vmin' and 'xosd' on MSM8939, 'vmin' and 'vlow' on SDM660.
> > > > I know that 'vmin' is VDD Minimization mode, and 'xosd' is XO Shutdown
> > > > mode.  But I'm not sure about 'vlow' mode.  Could you share some
> > > > information regarding what this low power mode is, and how it differs
> > > > from 'vmin' and 'xosd'?
> > > 
> > > vlow and xosd are same.
> > > vmin is xosd plus voltage minimization of chip, memory rails.
> > 
> > Thanks much for the info, Maulik!
> > 
> > I'm running your driver on qcm2290 and trying to reach vlow mode.
> > 
> > # cat /sys/kernel/debug/qcom_sleep_stats/vlow
> > Count: 0
> > Last Entered At: 0
> > Last Exited At: 0
> > Accumulated Duration: 0
> > Client Votes: 0x81
> > # echo mem > /sys/power/state
> > [  551.446603] PM: suspend entry (s2idle)
> > [  551.450948] Filesystems sync: 0.000 seconds
> > [  551.462828] Freezing user space processes ... (elapsed 0.002 seconds) done.
> > [  551.472276] OOM killer disabled.
> > [  551.475556] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
> > [  551.484461] printk: Suspending console(s) (use no_console_suspend to debug)
> > [  551.561280] OOM killer enabled.
> > [  551.564461] Restarting tasks ... done.
> > [  551.569652] PM: suspend exit
> > # cat /sys/kernel/debug/qcom_sleep_stats/vlow
> > Count: 0
> > Last Entered At: 0
> > Last Exited At: 0
> > Accumulated Duration: 0
> > Client Votes: 0x818081
> > 
> > The count doesn't increases along with suspend/resume cycle at the
> > moment.  But as you can see, 'Client Votes' field changes.  If possible,
> > could you shed some light on what this means?
> 
> The count will increase only when all the subsystems (APSS/Modem,etc) are in
> power down mode and finally RPM decides to turn off xo clock.
> 
> > 
> > As the comparison, I'm also running the downstream 'rpm_master_stats'
> > driver in the same kernel, and the 'xo_count' field of APSS does
> > increase along with suspend/resume cycle.  May I ask some advices what
> > I'm possibly missing and thus getting different result between 'vlow'
> > and 'rpm_master_stats' report?
> 
> The vlow is a SoC level state whereas the rpm master stats indicate
> individual subsystem state. Since you are running suspend-resume the APSS is
> going to sleep so you see xo_count incremented for it but for MPSS i see it
> does not increase (modem is not entering to low power mode). similarly for
> ADSP/CDSP it does not increment. if all of these subsystems goes to power
> down and then there is sufficient sleep time for the SoC then you may see
> vlow/vmin incrementing.
> 
> Hope this clarifies.

Thanks Maulik!  It's very helpful.  I have a couple of further
questions, if you do not mind.

1. We can understand most of vlow/vmin output.  But could you help
   decode 'Client Votes'?  It looks like the bits are shifting along
   with suspend/resume cycle.

2. In the rpm_master_stats output below, I know masters (processors)
   APSS, MPSS, ADSP and CDSP, but not really sure what TZ is.  If it's
   TrustZone, shouldn't it covered by APSS?

Thanks for sharing your insights!

Shawn

> > # cat /sys/kernel/debug/rpm_master_stats
> > APSS
> >          shutdown_req:0x37EA3CC74
> >          wakeup_ind:0x0
> >          bringup_req:0x37F041958
> >          bringup_ack:0x37F042D54
> >          xo_last_entered_at:0x286FF36AC
> >          xo_last_exited_at:0x28AF94178
> >          xo_accumulated_duration:0x3EDD55B
> >          last_sleep_transition_duration:0x122f
> >          last_wake_transition_duration:0x11f8
> >          xo_count:0x1
> >          wakeup_reason:0x0
> >          numshutdowns:0x641
> >          active_cores:0x1
> >                  core0
> > MPSS
> >          shutdown_req:0x0
> >          wakeup_ind:0x0
> >          bringup_req:0x0
> >          bringup_ack:0x0
> >          xo_last_entered_at:0x0
> >          xo_last_exited_at:0x0
> >          xo_accumulated_duration:0x0
> >          last_sleep_transition_duration:0x0
> >          last_wake_transition_duration:0x0
> >          xo_count:0x0
> >          wakeup_reason:0x0
> >          numshutdowns:0x0
> >          active_cores:0x1
> >                  core0
> > ADSP
> >          shutdown_req:0x0
> >          wakeup_ind:0x0
> >          bringup_req:0x0
> >          bringup_ack:0x0
> >          xo_last_entered_at:0x0
> >          xo_last_exited_at:0x0
> >          xo_accumulated_duration:0x0
> >          last_sleep_transition_duration:0x0
> >          last_wake_transition_duration:0x0
> >          xo_count:0x0
> >          wakeup_reason:0x0
> >          numshutdowns:0x0
> >          active_cores:0x1
> >                  core0
> > CDSP
> >          shutdown_req:0x0
> >          wakeup_ind:0x0
> >          bringup_req:0x0
> >          bringup_ack:0x0
> >          xo_last_entered_at:0x0
> >          xo_last_exited_at:0x0
> >          xo_accumulated_duration:0x0
> >          last_sleep_transition_duration:0x0
> >          last_wake_transition_duration:0x0
> >          xo_count:0x0
> >          wakeup_reason:0x0
> >          numshutdowns:0x0
> >          active_cores:0x0
> > TZ
> >          shutdown_req:0x0
> >          wakeup_ind:0x0
> >          bringup_req:0x0
> >          bringup_ack:0x0
> >          xo_last_entered_at:0x0
> >          xo_last_exited_at:0x0
> >          xo_accumulated_duration:0x0
> >          last_sleep_transition_duration:0x0
> >          last_wake_transition_duration:0x0
> >          xo_count:0x0
> >          wakeup_reason:0x0
> >          numshutdowns:0x0
> >          active_cores:0x0

Re: [PATCH v12 2/5] soc: qcom: Add Sleep stats driver

[Maulik Shah]

Hi Shawn,

On 10/21/2021 7:27 AM, Shawn Guo wrote:
> On Tue, Oct 19, 2021 at 06:16:57PM +0530, Maulik Shah wrote:
>> Hi Shawn,
>>
>> On 10/19/2021 3:17 PM, Shawn Guo wrote:
>>> On Mon, Oct 18, 2021 at 07:45:30PM +0530, Maulik Shah wrote:
>>>>>> +static void qcom_create_soc_sleep_stat_files(struct dentry *root, void __iomem *reg,
>>>>>> +					     struct stats_data *d,
>>>>>> +					     const struct stats_config *config)
>>>>>> +{
>>>>>> +	char stat_type[sizeof(u32) + 1] = {0};
>>>>>> +	size_t stats_offset = config->stats_offset;
>>>>>> +	u32 offset = 0, type;
>>>>>> +	int i, j;
>>>>>> +
>>>>>> +	/*
>>>>>> +	 * On RPM targets, stats offset location is dynamic and changes from target
>>>>>> +	 * to target and sometimes from build to build for same target.
>>>>>> +	 *
>>>>>> +	 * In such cases the dynamic address is present at 0x14 offset from base
>>>>>> +	 * address in devicetree. The last 16bits indicates the stats_offset.
>>>>>> +	 */
>>>>>> +	if (config->dynamic_offset) {
>>>>>> +		stats_offset = readl(reg + RPM_DYNAMIC_ADDR);
>>>>>> +		stats_offset &= RPM_DYNAMIC_ADDR_MASK;
>>>>>> +	}
>>>>>> +
>>>>>> +	for (i = 0; i < config->num_records; i++) {
>>>>>> +		d[i].base = reg + offset + stats_offset;
>>>>>> +
>>>>>> +		/*
>>>>>> +		 * Read the low power mode name and create debugfs file for it.
>>>>>> +		 * The names read could be of below,
>>>>>> +		 * (may change depending on low power mode supported).
>>>>>> +		 * For rpmh-sleep-stats: "aosd", "cxsd" and "ddr".
>>>>>> +		 * For rpm-sleep-stats: "vmin" and "vlow".
>>>>>
>>>>> It reports 'vmin' and 'xosd' on MSM8939, 'vmin' and 'vlow' on SDM660.
>>>>> I know that 'vmin' is VDD Minimization mode, and 'xosd' is XO Shutdown
>>>>> mode.  But I'm not sure about 'vlow' mode.  Could you share some
>>>>> information regarding what this low power mode is, and how it differs
>>>>> from 'vmin' and 'xosd'?
>>>>
>>>> vlow and xosd are same.
>>>> vmin is xosd plus voltage minimization of chip, memory rails.
>>>
>>> Thanks much for the info, Maulik!
>>>
>>> I'm running your driver on qcm2290 and trying to reach vlow mode.
>>>
>>> # cat /sys/kernel/debug/qcom_sleep_stats/vlow
>>> Count: 0
>>> Last Entered At: 0
>>> Last Exited At: 0
>>> Accumulated Duration: 0
>>> Client Votes: 0x81
>>> # echo mem > /sys/power/state
>>> [  551.446603] PM: suspend entry (s2idle)
>>> [  551.450948] Filesystems sync: 0.000 seconds
>>> [  551.462828] Freezing user space processes ... (elapsed 0.002 seconds) done.
>>> [  551.472276] OOM killer disabled.
>>> [  551.475556] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
>>> [  551.484461] printk: Suspending console(s) (use no_console_suspend to debug)
>>> [  551.561280] OOM killer enabled.
>>> [  551.564461] Restarting tasks ... done.
>>> [  551.569652] PM: suspend exit
>>> # cat /sys/kernel/debug/qcom_sleep_stats/vlow
>>> Count: 0
>>> Last Entered At: 0
>>> Last Exited At: 0
>>> Accumulated Duration: 0
>>> Client Votes: 0x818081
>>>
>>> The count doesn't increases along with suspend/resume cycle at the
>>> moment.  But as you can see, 'Client Votes' field changes.  If possible,
>>> could you shed some light on what this means?
>>
>> The count will increase only when all the subsystems (APSS/Modem,etc) are in
>> power down mode and finally RPM decides to turn off xo clock.
>>
>>>
>>> As the comparison, I'm also running the downstream 'rpm_master_stats'
>>> driver in the same kernel, and the 'xo_count' field of APSS does
>>> increase along with suspend/resume cycle.  May I ask some advices what
>>> I'm possibly missing and thus getting different result between 'vlow'
>>> and 'rpm_master_stats' report?
>>
>> The vlow is a SoC level state whereas the rpm master stats indicate
>> individual subsystem state. Since you are running suspend-resume the APSS is
>> going to sleep so you see xo_count incremented for it but for MPSS i see it
>> does not increase (modem is not entering to low power mode). similarly for
>> ADSP/CDSP it does not increment. if all of these subsystems goes to power
>> down and then there is sufficient sleep time for the SoC then you may see
>> vlow/vmin incrementing.
>>
>> Hope this clarifies.
> 
> Thanks Maulik!  It's very helpful.  I have a couple of further
> questions, if you do not mind.
> 
> 1. We can understand most of vlow/vmin output.  But could you help
>     decode 'Client Votes'?  It looks like the bits are shifting along
>     with suspend/resume cycle.
> 

Client votes are history of client votes on xo. Each byte represents xo 
clock votes from subsystems.


> 2. In the rpm_master_stats output below, I know masters (processors)
>     APSS, MPSS, ADSP and CDSP, but not really sure what TZ is.  If it's
>     TrustZone, shouldn't it covered by APSS?

Yes it indicates for Trustzone but should be covered by APSS.

Thanks,
Maulik

> 
> Thanks for sharing your insights!
> 
> Shawn
> 
>>> # cat /sys/kernel/debug/rpm_master_stats
>>> APSS
>>>           shutdown_req:0x37EA3CC74
>>>           wakeup_ind:0x0
>>>           bringup_req:0x37F041958
>>>           bringup_ack:0x37F042D54
>>>           xo_last_entered_at:0x286FF36AC
>>>           xo_last_exited_at:0x28AF94178
>>>           xo_accumulated_duration:0x3EDD55B
>>>           last_sleep_transition_duration:0x122f
>>>           last_wake_transition_duration:0x11f8
>>>           xo_count:0x1
>>>           wakeup_reason:0x0
>>>           numshutdowns:0x641
>>>           active_cores:0x1
>>>                   core0
>>> MPSS
>>>           shutdown_req:0x0
>>>           wakeup_ind:0x0
>>>           bringup_req:0x0
>>>           bringup_ack:0x0
>>>           xo_last_entered_at:0x0
>>>           xo_last_exited_at:0x0
>>>           xo_accumulated_duration:0x0
>>>           last_sleep_transition_duration:0x0
>>>           last_wake_transition_duration:0x0
>>>           xo_count:0x0
>>>           wakeup_reason:0x0
>>>           numshutdowns:0x0
>>>           active_cores:0x1
>>>                   core0
>>> ADSP
>>>           shutdown_req:0x0
>>>           wakeup_ind:0x0
>>>           bringup_req:0x0
>>>           bringup_ack:0x0
>>>           xo_last_entered_at:0x0
>>>           xo_last_exited_at:0x0
>>>           xo_accumulated_duration:0x0
>>>           last_sleep_transition_duration:0x0
>>>           last_wake_transition_duration:0x0
>>>           xo_count:0x0
>>>           wakeup_reason:0x0
>>>           numshutdowns:0x0
>>>           active_cores:0x1
>>>                   core0
>>> CDSP
>>>           shutdown_req:0x0
>>>           wakeup_ind:0x0
>>>           bringup_req:0x0
>>>           bringup_ack:0x0
>>>           xo_last_entered_at:0x0
>>>           xo_last_exited_at:0x0
>>>           xo_accumulated_duration:0x0
>>>           last_sleep_transition_duration:0x0
>>>           last_wake_transition_duration:0x0
>>>           xo_count:0x0
>>>           wakeup_reason:0x0
>>>           numshutdowns:0x0
>>>           active_cores:0x0
>>> TZ
>>>           shutdown_req:0x0
>>>           wakeup_ind:0x0
>>>           bringup_req:0x0
>>>           bringup_ack:0x0
>>>           xo_last_entered_at:0x0
>>>           xo_last_exited_at:0x0
>>>           xo_accumulated_duration:0x0
>>>           last_sleep_transition_duration:0x0
>>>           last_wake_transition_duration:0x0
>>>           xo_count:0x0
>>>           wakeup_reason:0x0
>>>           numshutdowns:0x0
>>>           active_cores:0x0

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a 
member of Code Aurora Forum, hosted by The Linux Foundation

Re: [PATCH v12 2/5] soc: qcom: Add Sleep stats driver

[Shawn Guo]

On Thu, Oct 21, 2021 at 03:11:26PM +0530, Maulik Shah wrote:
> Hi Shawn,
> 
> On 10/21/2021 7:27 AM, Shawn Guo wrote:
> > On Tue, Oct 19, 2021 at 06:16:57PM +0530, Maulik Shah wrote:
> > > Hi Shawn,
> > > 
> > > On 10/19/2021 3:17 PM, Shawn Guo wrote:
> > > > On Mon, Oct 18, 2021 at 07:45:30PM +0530, Maulik Shah wrote:
> > > > > > > +static void qcom_create_soc_sleep_stat_files(struct dentry *root, void __iomem *reg,
> > > > > > > +					     struct stats_data *d,
> > > > > > > +					     const struct stats_config *config)
> > > > > > > +{
> > > > > > > +	char stat_type[sizeof(u32) + 1] = {0};
> > > > > > > +	size_t stats_offset = config->stats_offset;
> > > > > > > +	u32 offset = 0, type;
> > > > > > > +	int i, j;
> > > > > > > +
> > > > > > > +	/*
> > > > > > > +	 * On RPM targets, stats offset location is dynamic and changes from target
> > > > > > > +	 * to target and sometimes from build to build for same target.
> > > > > > > +	 *
> > > > > > > +	 * In such cases the dynamic address is present at 0x14 offset from base
> > > > > > > +	 * address in devicetree. The last 16bits indicates the stats_offset.
> > > > > > > +	 */
> > > > > > > +	if (config->dynamic_offset) {
> > > > > > > +		stats_offset = readl(reg + RPM_DYNAMIC_ADDR);
> > > > > > > +		stats_offset &= RPM_DYNAMIC_ADDR_MASK;
> > > > > > > +	}
> > > > > > > +
> > > > > > > +	for (i = 0; i < config->num_records; i++) {
> > > > > > > +		d[i].base = reg + offset + stats_offset;
> > > > > > > +
> > > > > > > +		/*
> > > > > > > +		 * Read the low power mode name and create debugfs file for it.
> > > > > > > +		 * The names read could be of below,
> > > > > > > +		 * (may change depending on low power mode supported).
> > > > > > > +		 * For rpmh-sleep-stats: "aosd", "cxsd" and "ddr".
> > > > > > > +		 * For rpm-sleep-stats: "vmin" and "vlow".
> > > > > > 
> > > > > > It reports 'vmin' and 'xosd' on MSM8939, 'vmin' and 'vlow' on SDM660.
> > > > > > I know that 'vmin' is VDD Minimization mode, and 'xosd' is XO Shutdown
> > > > > > mode.  But I'm not sure about 'vlow' mode.  Could you share some
> > > > > > information regarding what this low power mode is, and how it differs
> > > > > > from 'vmin' and 'xosd'?
> > > > > 
> > > > > vlow and xosd are same.
> > > > > vmin is xosd plus voltage minimization of chip, memory rails.
> > > > 
> > > > Thanks much for the info, Maulik!
> > > > 
> > > > I'm running your driver on qcm2290 and trying to reach vlow mode.
> > > > 
> > > > # cat /sys/kernel/debug/qcom_sleep_stats/vlow
> > > > Count: 0
> > > > Last Entered At: 0
> > > > Last Exited At: 0
> > > > Accumulated Duration: 0
> > > > Client Votes: 0x81
> > > > # echo mem > /sys/power/state
> > > > [  551.446603] PM: suspend entry (s2idle)
> > > > [  551.450948] Filesystems sync: 0.000 seconds
> > > > [  551.462828] Freezing user space processes ... (elapsed 0.002 seconds) done.
> > > > [  551.472276] OOM killer disabled.
> > > > [  551.475556] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done.
> > > > [  551.484461] printk: Suspending console(s) (use no_console_suspend to debug)
> > > > [  551.561280] OOM killer enabled.
> > > > [  551.564461] Restarting tasks ... done.
> > > > [  551.569652] PM: suspend exit
> > > > # cat /sys/kernel/debug/qcom_sleep_stats/vlow
> > > > Count: 0
> > > > Last Entered At: 0
> > > > Last Exited At: 0
> > > > Accumulated Duration: 0
> > > > Client Votes: 0x818081
> > > > 
> > > > The count doesn't increases along with suspend/resume cycle at the
> > > > moment.  But as you can see, 'Client Votes' field changes.  If possible,
> > > > could you shed some light on what this means?
> > > 
> > > The count will increase only when all the subsystems (APSS/Modem,etc) are in
> > > power down mode and finally RPM decides to turn off xo clock.
> > > 
> > > > 
> > > > As the comparison, I'm also running the downstream 'rpm_master_stats'
> > > > driver in the same kernel, and the 'xo_count' field of APSS does
> > > > increase along with suspend/resume cycle.  May I ask some advices what
> > > > I'm possibly missing and thus getting different result between 'vlow'
> > > > and 'rpm_master_stats' report?
> > > 
> > > The vlow is a SoC level state whereas the rpm master stats indicate
> > > individual subsystem state. Since you are running suspend-resume the APSS is
> > > going to sleep so you see xo_count incremented for it but for MPSS i see it
> > > does not increase (modem is not entering to low power mode). similarly for
> > > ADSP/CDSP it does not increment. if all of these subsystems goes to power
> > > down and then there is sufficient sleep time for the SoC then you may see
> > > vlow/vmin incrementing.
> > > 
> > > Hope this clarifies.
> > 
> > Thanks Maulik!  It's very helpful.  I have a couple of further
> > questions, if you do not mind.
> > 
> > 1. We can understand most of vlow/vmin output.  But could you help
> >     decode 'Client Votes'?  It looks like the bits are shifting along
> >     with suspend/resume cycle.
> > 
> 
> Client votes are history of client votes on xo. Each byte represents xo
> clock votes from subsystems.
> 
> 
> > 2. In the rpm_master_stats output below, I know masters (processors)
> >     APSS, MPSS, ADSP and CDSP, but not really sure what TZ is.  If it's
> >     TrustZone, shouldn't it covered by APSS?
> 
> Yes it indicates for Trustzone but should be covered by APSS.

Thanks for confirming, Maulik!

Interestingly, while I'm seeing xo_count of APSS and TZ increase together
along with suspend on downstream kernel, with my upstream based kernel,
only APSS xo_count increases and TZ always stay at 0.  I guess that's
why I cannot reach vlow mode.  Do you have any suggestion why this
happens?  I must have something missing from my kernel.  Thanks!

Shawn

> > > > # cat /sys/kernel/debug/rpm_master_stats
> > > > APSS
> > > >           shutdown_req:0x37EA3CC74
> > > >           wakeup_ind:0x0
> > > >           bringup_req:0x37F041958
> > > >           bringup_ack:0x37F042D54
> > > >           xo_last_entered_at:0x286FF36AC
> > > >           xo_last_exited_at:0x28AF94178
> > > >           xo_accumulated_duration:0x3EDD55B
> > > >           last_sleep_transition_duration:0x122f
> > > >           last_wake_transition_duration:0x11f8
> > > >           xo_count:0x1
> > > >           wakeup_reason:0x0
> > > >           numshutdowns:0x641
> > > >           active_cores:0x1
> > > >                   core0
> > > > MPSS
> > > >           shutdown_req:0x0
> > > >           wakeup_ind:0x0
> > > >           bringup_req:0x0
> > > >           bringup_ack:0x0
> > > >           xo_last_entered_at:0x0
> > > >           xo_last_exited_at:0x0
> > > >           xo_accumulated_duration:0x0
> > > >           last_sleep_transition_duration:0x0
> > > >           last_wake_transition_duration:0x0
> > > >           xo_count:0x0
> > > >           wakeup_reason:0x0
> > > >           numshutdowns:0x0
> > > >           active_cores:0x1
> > > >                   core0
> > > > ADSP
> > > >           shutdown_req:0x0
> > > >           wakeup_ind:0x0
> > > >           bringup_req:0x0
> > > >           bringup_ack:0x0
> > > >           xo_last_entered_at:0x0
> > > >           xo_last_exited_at:0x0
> > > >           xo_accumulated_duration:0x0
> > > >           last_sleep_transition_duration:0x0
> > > >           last_wake_transition_duration:0x0
> > > >           xo_count:0x0
> > > >           wakeup_reason:0x0
> > > >           numshutdowns:0x0
> > > >           active_cores:0x1
> > > >                   core0
> > > > CDSP
> > > >           shutdown_req:0x0
> > > >           wakeup_ind:0x0
> > > >           bringup_req:0x0
> > > >           bringup_ack:0x0
> > > >           xo_last_entered_at:0x0
> > > >           xo_last_exited_at:0x0
> > > >           xo_accumulated_duration:0x0
> > > >           last_sleep_transition_duration:0x0
> > > >           last_wake_transition_duration:0x0
> > > >           xo_count:0x0
> > > >           wakeup_reason:0x0
> > > >           numshutdowns:0x0
> > > >           active_cores:0x0
> > > > TZ
> > > >           shutdown_req:0x0
> > > >           wakeup_ind:0x0
> > > >           bringup_req:0x0
> > > >           bringup_ack:0x0
> > > >           xo_last_entered_at:0x0
> > > >           xo_last_exited_at:0x0
> > > >           xo_accumulated_duration:0x0
> > > >           last_sleep_transition_duration:0x0
> > > >           last_wake_transition_duration:0x0
> > > >           xo_count:0x0
> > > >           wakeup_reason:0x0
> > > >           numshutdowns:0x0
> > > >           active_cores:0x0
> 
> -- 
> QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of
> Code Aurora Forum, hosted by The Linux Foundation