Linux-PM Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH v4 00/24] More improvements for Tegra30 devfreq driver
@ 2019-07-07 22:32 Dmitry Osipenko
  2019-07-07 22:32 ` [PATCH v4 01/24] PM / devfreq: tegra30: Change irq type to unsigned int Dmitry Osipenko
                   ` (23 more replies)
  0 siblings, 24 replies; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Hello,

This series addresses some additional review comments that were made by
Thierry Reding to [1] and makes several important changes to the driver,
fixing excessive interrupts activity. In the end I'm proposing myself as
a maintainer for the Tegra devfreq drivers.

[1] https://lore.kernel.org/lkml/0fb50eb1-a173-1756-6889-2526a10ac707@gmail.com/T/

Changelog:

v4:  Added two new patches to the series:

       PM / devfreq: tegra30: Synchronize average count on target's update
       PM / devfreq: tegra30: Increase sampling period to 16ms

     The first patch addresses problem where governor could get stuck due
     to outdated "average count" value which is snapshoted by ISR and there
     are cases where manual update of the value is required.

     The second patch is just a minor optimization.

v3:  Added support for tracepoints, replacing the debug messages.
     Fixed few more bugs with the help of tracepoints.

     New patches in this version:

       PM / devfreq: tegra30: Use tracepoints for debugging
       PM / devfreq: tegra30: Optimize CPUFreq notifier
       PM / devfreq: tegra30: Optimize upper consecutive watermark selection
       PM / devfreq: tegra30: Optimize upper average watermark selection
       PM / devfreq: tegra30: Include appropriate header

     Some of older patches of this series also got some extra minor polish.

v2:  Added more patches that are cleaning driver's code further and
     squashing another kHz conversion bug.

     The patch "Rework frequency management logic" of the v1 series is now
     converted to "Set up watermarks properly" because I found some problems
     in the original patch and then realized that there is no need to change
     the logic much. So the logic mostly preserved and only got improvements.

     The series is based on the today's linux-next (25 Jun) and takes into
     account minor changes that MyungJoo Ham made to the already queued
     patches from the first batch [1].

Dmitry Osipenko (24):
  PM / devfreq: tegra30: Change irq type to unsigned int
  PM / devfreq: tegra30: Keep interrupt disabled while governor is
    stopped
  PM / devfreq: tegra30: Handle possible round-rate error
  PM / devfreq: tegra30: Drop write-barrier
  PM / devfreq: tegra30: Set up watermarks properly
  PM / devfreq: tegra30: Tune up boosting thresholds
  PM / devfreq: tegra30: Use CPUFreq notifier
  PM / devfreq: tegra30: Move clk-notifier's registration to governor's
    start
  PM / devfreq: tegra30: Reset boosting on startup
  PM / devfreq: tegra30: Don't enable consecutive-down interrupt on
    startup
  PM / devfreq: tegra30: Add debug messages
  PM / devfreq: tegra30: Inline all one-line functions
  PM / devfreq: tegra30: Constify structs
  PM / devfreq: tegra30: Ensure that target freq won't overflow
  PM / devfreq: tegra30: Fix integer overflow on CPU's freq max out
  PM / devfreq: tegra30: Use kHz units uniformly in the code
  PM / devfreq: tegra30: Use tracepoints for debugging
  PM / devfreq: tegra30: Optimize CPUFreq notifier
  PM / devfreq: tegra30: Optimize upper consecutive watermark selection
  PM / devfreq: tegra30: Optimize upper average watermark selection
  PM / devfreq: tegra30: Synchronize average count on target's update
  PM / devfreq: tegra30: Include appropriate header
  PM / devfreq: tegra30: Increase sampling period to 16ms
  PM / devfreq: tegra20/30: Add Dmitry as a maintainer

 MAINTAINERS                            |   9 +
 drivers/devfreq/tegra30-devfreq.c      | 648 +++++++++++++++++++------
 include/trace/events/tegra30_devfreq.h | 105 ++++
 3 files changed, 607 insertions(+), 155 deletions(-)
 create mode 100644 include/trace/events/tegra30_devfreq.h

-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 01/24] PM / devfreq: tegra30: Change irq type to unsigned int
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-16 11:35   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 02/24] PM / devfreq: tegra30: Keep interrupt disabled while governor is stopped Dmitry Osipenko
                   ` (22 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

IRQ numbers are always positive, hence the corresponding variable should
be unsigned to keep types consistent. This is a minor change that cleans
up code a tad more.

Suggested-by: Thierry Reding <thierry.reding@gmail.com>
Acked-by: MyungJoo Ham <myungjoo.ham@samsung.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index a6ba75f4106d..a27300f40b0b 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -160,7 +160,7 @@ struct tegra_devfreq {
 
 	struct tegra_devfreq_device devices[ARRAY_SIZE(actmon_device_configs)];
 
-	int irq;
+	unsigned int		irq;
 };
 
 struct tegra_actmon_emc_ratio {
@@ -618,12 +618,12 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
 		return PTR_ERR(tegra->emc_clock);
 	}
 
-	tegra->irq = platform_get_irq(pdev, 0);
-	if (tegra->irq < 0) {
-		err = tegra->irq;
+	err = platform_get_irq(pdev, 0);
+	if (err < 0) {
 		dev_err(&pdev->dev, "Failed to get IRQ: %d\n", err);
 		return err;
 	}
+	tegra->irq = err;
 
 	reset_control_assert(tegra->reset);
 
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 02/24] PM / devfreq: tegra30: Keep interrupt disabled while governor is stopped
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
  2019-07-07 22:32 ` [PATCH v4 01/24] PM / devfreq: tegra30: Change irq type to unsigned int Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-16 11:47   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 03/24] PM / devfreq: tegra30: Handle possible round-rate error Dmitry Osipenko
                   ` (21 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

There is no real need to keep interrupt always-enabled, will be nicer
to keep it disabled while governor is inactive.

Suggested-by: Thierry Reding <thierry.reding@gmail.com>
Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 43 ++++++++++++++++---------------
 1 file changed, 22 insertions(+), 21 deletions(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index a27300f40b0b..5e2b133babdd 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -11,6 +11,7 @@
 #include <linux/devfreq.h>
 #include <linux/interrupt.h>
 #include <linux/io.h>
+#include <linux/irq.h>
 #include <linux/module.h>
 #include <linux/mod_devicetable.h>
 #include <linux/platform_device.h>
@@ -416,8 +417,6 @@ static void tegra_actmon_start(struct tegra_devfreq *tegra)
 {
 	unsigned int i;
 
-	disable_irq(tegra->irq);
-
 	actmon_writel(tegra, ACTMON_SAMPLING_PERIOD - 1,
 		      ACTMON_GLB_PERIOD_CTRL);
 
@@ -442,8 +441,6 @@ static void tegra_actmon_stop(struct tegra_devfreq *tegra)
 	}
 
 	actmon_write_barrier(tegra);
-
-	enable_irq(tegra->irq);
 }
 
 static int tegra_devfreq_target(struct device *dev, unsigned long *freq,
@@ -552,6 +549,12 @@ static int tegra_governor_event_handler(struct devfreq *devfreq,
 {
 	struct tegra_devfreq *tegra = dev_get_drvdata(devfreq->dev.parent);
 
+	/*
+	 * Couple device with the governor early as it is needed at
+	 * the moment of governor's start (used by ISR).
+	 */
+	tegra->devfreq = devfreq;
+
 	switch (event) {
 	case DEVFREQ_GOV_START:
 		devfreq_monitor_start(devfreq);
@@ -586,10 +589,11 @@ static struct devfreq_governor tegra_devfreq_governor = {
 
 static int tegra_devfreq_probe(struct platform_device *pdev)
 {
-	struct tegra_devfreq *tegra;
 	struct tegra_devfreq_device *dev;
-	unsigned int i;
+	struct tegra_devfreq *tegra;
+	struct devfreq *devfreq;
 	unsigned long rate;
+	unsigned int i;
 	int err;
 
 	tegra = devm_kzalloc(&pdev->dev, sizeof(*tegra), GFP_KERNEL);
@@ -625,6 +629,16 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
 	}
 	tegra->irq = err;
 
+	irq_set_status_flags(tegra->irq, IRQ_NOAUTOEN);
+
+	err = devm_request_threaded_irq(&pdev->dev, tegra->irq, NULL,
+					actmon_thread_isr, IRQF_ONESHOT,
+					"tegra-devfreq", tegra);
+	if (err) {
+		dev_err(&pdev->dev, "Interrupt request failed: %d\n", err);
+		return err;
+	}
+
 	reset_control_assert(tegra->reset);
 
 	err = clk_prepare_enable(tegra->clock);
@@ -672,28 +686,15 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
 	}
 
 	tegra_devfreq_profile.initial_freq = clk_get_rate(tegra->emc_clock);
-	tegra->devfreq = devfreq_add_device(&pdev->dev,
-					    &tegra_devfreq_profile,
-					    "tegra_actmon",
-					    NULL);
+	devfreq = devfreq_add_device(&pdev->dev, &tegra_devfreq_profile,
+				     "tegra_actmon", NULL);
 	if (IS_ERR(tegra->devfreq)) {
 		err = PTR_ERR(tegra->devfreq);
 		goto remove_governor;
 	}
 
-	err = devm_request_threaded_irq(&pdev->dev, tegra->irq, NULL,
-					actmon_thread_isr, IRQF_ONESHOT,
-					"tegra-devfreq", tegra);
-	if (err) {
-		dev_err(&pdev->dev, "Interrupt request failed: %d\n", err);
-		goto remove_devfreq;
-	}
-
 	return 0;
 
-remove_devfreq:
-	devfreq_remove_device(tegra->devfreq);
-
 remove_governor:
 	devfreq_remove_governor(&tegra_devfreq_governor);
 
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 03/24] PM / devfreq: tegra30: Handle possible round-rate error
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
  2019-07-07 22:32 ` [PATCH v4 01/24] PM / devfreq: tegra30: Change irq type to unsigned int Dmitry Osipenko
  2019-07-07 22:32 ` [PATCH v4 02/24] PM / devfreq: tegra30: Keep interrupt disabled while governor is stopped Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-16 11:50   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 04/24] PM / devfreq: tegra30: Drop write-barrier Dmitry Osipenko
                   ` (20 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

The EMC clock rate rounding technically could fail, hence let's handle
the error cases properly.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 5e2b133babdd..5e606ae3f238 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -592,8 +592,8 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
 	struct tegra_devfreq_device *dev;
 	struct tegra_devfreq *tegra;
 	struct devfreq *devfreq;
-	unsigned long rate;
 	unsigned int i;
+	long rate;
 	int err;
 
 	tegra = devm_kzalloc(&pdev->dev, sizeof(*tegra), GFP_KERNEL);
@@ -650,8 +650,14 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
 
 	reset_control_deassert(tegra->reset);
 
-	tegra->max_freq = clk_round_rate(tegra->emc_clock, ULONG_MAX) / KHZ;
+	rate = clk_round_rate(tegra->emc_clock, ULONG_MAX);
+	if (rate < 0) {
+		dev_err(&pdev->dev, "Failed to round clock rate: %ld\n", rate);
+		return rate;
+	}
+
 	tegra->cur_freq = clk_get_rate(tegra->emc_clock) / KHZ;
+	tegra->max_freq = rate / KHZ;
 
 	for (i = 0; i < ARRAY_SIZE(actmon_device_configs); i++) {
 		dev = tegra->devices + i;
@@ -662,6 +668,13 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
 	for (rate = 0; rate <= tegra->max_freq * KHZ; rate++) {
 		rate = clk_round_rate(tegra->emc_clock, rate);
 
+		if (rate < 0) {
+			dev_err(&pdev->dev,
+				"Failed to round clock rate: %ld\n", rate);
+			err = rate;
+			goto remove_opps;
+		}
+
 		err = dev_pm_opp_add(&pdev->dev, rate, 0);
 		if (err) {
 			dev_err(&pdev->dev, "Failed to add OPP: %d\n", err);
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 04/24] PM / devfreq: tegra30: Drop write-barrier
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (2 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 03/24] PM / devfreq: tegra30: Handle possible round-rate error Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-16 11:51   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 05/24] PM / devfreq: tegra30: Set up watermarks properly Dmitry Osipenko
                   ` (19 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

There is no need in a write-barrier now, given that interrupt masking is
handled by CPU's GIC now. Hence we know exactly that interrupt won't fire
after stopping the devfreq's governor. In other cases we don't care about
potential buffering of the writes to hardware and thus there is no need to
stall CPU.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 14 --------------
 1 file changed, 14 deletions(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 5e606ae3f238..4be7858c33bc 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -230,12 +230,6 @@ static void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
 		      ACTMON_DEV_LOWER_WMARK);
 }
 
-static void actmon_write_barrier(struct tegra_devfreq *tegra)
-{
-	/* ensure the update has reached the ACTMON */
-	readl(tegra->regs + ACTMON_GLB_STATUS);
-}
-
 static void actmon_isr_device(struct tegra_devfreq *tegra,
 			      struct tegra_devfreq_device *dev)
 {
@@ -287,8 +281,6 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
 	device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
 
 	device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
-
-	actmon_write_barrier(tegra);
 }
 
 static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq *tegra,
@@ -376,8 +368,6 @@ static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
 		tegra_devfreq_update_wmark(tegra, dev);
 	}
 
-	actmon_write_barrier(tegra);
-
 	return NOTIFY_OK;
 }
 
@@ -423,8 +413,6 @@ static void tegra_actmon_start(struct tegra_devfreq *tegra)
 	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++)
 		tegra_actmon_configure_device(tegra, &tegra->devices[i]);
 
-	actmon_write_barrier(tegra);
-
 	enable_irq(tegra->irq);
 }
 
@@ -439,8 +427,6 @@ static void tegra_actmon_stop(struct tegra_devfreq *tegra)
 		device_writel(&tegra->devices[i], ACTMON_INTR_STATUS_CLEAR,
 			      ACTMON_DEV_INTR_STATUS);
 	}
-
-	actmon_write_barrier(tegra);
 }
 
 static int tegra_devfreq_target(struct device *dev, unsigned long *freq,
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 05/24] PM / devfreq: tegra30: Set up watermarks properly
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (3 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 04/24] PM / devfreq: tegra30: Drop write-barrier Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-18 10:17   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 06/24] PM / devfreq: tegra30: Tune up boosting thresholds Dmitry Osipenko
                   ` (18 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

The current implementation is inaccurate and results in very intensive
interrupt activity, which neglects the whole idea of polling offload to
hardware. The reason of the shortcoming is that watermarks are not set
up correctly and this results in ACTMON constantly asking to change freq
and then these requests are ignored. The end result of this patch is that
there are few hundreds of ACTMON's interrupts instead of tens thousands
after few minutes of a working devfreq, meanwhile the transitions activity
stays about the same and governor becomes more reactive.

Since watermarks are set precisely correct now, the boosting logic is
changed a tad to accommodate the change. The "average sustain coefficient"
multiplier is gone now since there is no need to compensate the improper
watermarks and EMC frequency-bump happens once boosting hits the upper
watermark enough times, depending on the per-device boosting threshold.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 293 +++++++++++++++++++++---------
 1 file changed, 209 insertions(+), 84 deletions(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 4be7858c33bc..16f7e6cf3b99 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -47,6 +47,8 @@
 
 #define ACTMON_DEV_INTR_CONSECUTIVE_UPPER			BIT(31)
 #define ACTMON_DEV_INTR_CONSECUTIVE_LOWER			BIT(30)
+#define ACTMON_DEV_INTR_AVG_BELOW_WMARK				BIT(25)
+#define ACTMON_DEV_INTR_AVG_ABOVE_WMARK				BIT(24)
 
 #define ACTMON_ABOVE_WMARK_WINDOW				1
 #define ACTMON_BELOW_WMARK_WINDOW				3
@@ -63,9 +65,8 @@
  * ACTMON_AVERAGE_WINDOW_LOG2: default value for @DEV_CTRL_K_VAL, which
  * translates to 2 ^ (K_VAL + 1). ex: 2 ^ (6 + 1) = 128
  */
-#define ACTMON_AVERAGE_WINDOW_LOG2			6
-#define ACTMON_SAMPLING_PERIOD				12 /* ms */
-#define ACTMON_DEFAULT_AVG_BAND				6  /* 1/10 of % */
+#define ACTMON_AVERAGE_WINDOW_LOG2				6
+#define ACTMON_SAMPLING_PERIOD					12 /* ms */
 
 #define KHZ							1000
 
@@ -142,9 +143,6 @@ struct tegra_devfreq_device {
 	 * watermark breaches.
 	 */
 	unsigned long boost_freq;
-
-	/* Optimal frequency calculated from the stats for this device */
-	unsigned long target_freq;
 };
 
 struct tegra_devfreq {
@@ -156,7 +154,6 @@ struct tegra_devfreq {
 
 	struct clk		*emc_clock;
 	unsigned long		max_freq;
-	unsigned long		cur_freq;
 	struct notifier_block	rate_change_nb;
 
 	struct tegra_devfreq_device devices[ARRAY_SIZE(actmon_device_configs)];
@@ -205,42 +202,182 @@ static unsigned long do_percent(unsigned long val, unsigned int pct)
 	return val * pct / 100;
 }
 
+static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq *tegra)
+{
+	struct tegra_actmon_emc_ratio *ratio = actmon_emc_ratios;
+	unsigned int cpu_freq = cpufreq_get(0);
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(actmon_emc_ratios); i++, ratio++) {
+		if (cpu_freq >= ratio->cpu_freq) {
+			if (ratio->emc_freq >= tegra->max_freq)
+				return tegra->max_freq;
+			else
+				return ratio->emc_freq;
+		}
+	}
+
+	return 0;
+}
+
+static unsigned long
+tegra_actmon_account_cpu_freq(struct tegra_devfreq *tegra,
+			      struct tegra_devfreq_device *dev,
+			      unsigned long target_freq)
+{
+	unsigned long static_cpu_emc_freq;
+
+	if (dev->config->avg_dependency_threshold &&
+	    dev->config->avg_dependency_threshold < dev->avg_count) {
+		static_cpu_emc_freq = actmon_cpu_to_emc_rate(tegra);
+		target_freq = max(target_freq, static_cpu_emc_freq);
+	}
+
+	return target_freq;
+}
+
+static unsigned long tegra_actmon_lower_freq(struct tegra_devfreq *tegra,
+					     unsigned long target_freq)
+{
+	unsigned long lower = target_freq;
+	struct dev_pm_opp *opp;
+
+	opp = dev_pm_opp_find_freq_floor(tegra->devfreq->dev.parent, &lower);
+	if (IS_ERR(opp))
+		lower = 0;
+	else
+		dev_pm_opp_put(opp);
+
+	return lower;
+}
+
+static unsigned long tegra_actmon_upper_freq(struct tegra_devfreq *tegra,
+					     unsigned long target_freq)
+{
+	unsigned long upper = target_freq + 1;
+	struct dev_pm_opp *opp;
+
+	opp = dev_pm_opp_find_freq_ceil(tegra->devfreq->dev.parent, &upper);
+	if (IS_ERR(opp))
+		upper = ULONG_MAX;
+	else
+		dev_pm_opp_put(opp);
+
+	return upper;
+}
+
+static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
+					 struct tegra_devfreq_device *dev,
+					 unsigned long target_freq,
+					 unsigned long *lower,
+					 unsigned long *upper)
+{
+	/*
+	 * Memory frequencies are guaranteed to have 1MHz granularity
+	 * and thus we need this rounding down to get a proper watermarks
+	 * range in a case where target_freq falls into a range of
+	 * next_possible_opp_freq - 1MHz.
+	 */
+	target_freq = round_down(target_freq, 1000000);
+
+	/* watermarks are set at the borders of the corresponding OPPs */
+	*lower = tegra_actmon_lower_freq(tegra, target_freq);
+	*upper = tegra_actmon_upper_freq(tegra, target_freq);
+
+	*lower /= KHZ;
+	*upper /= KHZ;
+
+	/*
+	 * The upper watermark should take into account CPU's frequency
+	 * because cpu_to_emc_rate() may override the target_freq with
+	 * a higher value and thus upper watermark need to be set up
+	 * accordingly to avoid parasitic upper-events.
+	 */
+	*upper = tegra_actmon_account_cpu_freq(tegra, dev, *upper);
+
+	*lower *= ACTMON_SAMPLING_PERIOD;
+	*upper *= ACTMON_SAMPLING_PERIOD;
+}
+
 static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
 					   struct tegra_devfreq_device *dev)
 {
-	u32 avg = dev->avg_count;
-	u32 avg_band_freq = tegra->max_freq * ACTMON_DEFAULT_AVG_BAND / KHZ;
-	u32 band = avg_band_freq * ACTMON_SAMPLING_PERIOD;
+	unsigned long lower, upper, freq;
 
-	device_writel(dev, avg + band, ACTMON_DEV_AVG_UPPER_WMARK);
+	freq = dev->avg_count / ACTMON_SAMPLING_PERIOD * KHZ;
+	tegra_actmon_get_lower_upper(tegra, dev, freq, &lower, &upper);
 
-	avg = max(dev->avg_count, band);
-	device_writel(dev, avg - band, ACTMON_DEV_AVG_LOWER_WMARK);
+	/*
+	 * We want to get interrupts when MCCPU client crosses the
+	 * dependency threshold in order to take into / out of account
+	 * the CPU's freq.
+	 */
+	if (lower < dev->config->avg_dependency_threshold &&
+	    upper > dev->config->avg_dependency_threshold) {
+		if (dev->avg_count < dev->config->avg_dependency_threshold)
+			upper = dev->config->avg_dependency_threshold;
+		else
+			lower = dev->config->avg_dependency_threshold;
+	}
+
+	device_writel(dev, lower, ACTMON_DEV_AVG_LOWER_WMARK);
+	device_writel(dev, upper, ACTMON_DEV_AVG_UPPER_WMARK);
 }
 
 static void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
-				       struct tegra_devfreq_device *dev)
+				       struct tegra_devfreq_device *dev,
+				       unsigned long freq)
 {
-	u32 val = tegra->cur_freq * ACTMON_SAMPLING_PERIOD;
+	unsigned long lower, upper, delta;
+
+	/*
+	 * Boosting logic kicks-in once lower / upper watermark is hit.
+	 * The watermarks are based on the updated EMC rate and the
+	 * average activity.
+	 *
+	 * The higher watermark is set in accordance to the EMC rate
+	 * because we want to set it to the highest mark here and EMC rate
+	 * represents that mark. The consecutive-upper interrupts are
+	 * always enabled and we don't want to receive them if they won't
+	 * do anything useful, hence the upper watermark is capped to maximum.
+	 * Note that the EMC rate is changed once boosting pushed the rate
+	 * too high, in that case boosting-up will be stopped because
+	 * upper watermark is much higher now and it is *important* to
+	 * stop the unwanted interrupts.
+	 */
+	tegra_actmon_get_lower_upper(tegra, dev, freq - 1, &lower, &upper);
+
+	delta = do_percent(upper - lower, dev->config->boost_up_threshold);
+	device_writel(dev, lower + delta, ACTMON_DEV_UPPER_WMARK);
 
-	device_writel(dev, do_percent(val, dev->config->boost_up_threshold),
-		      ACTMON_DEV_UPPER_WMARK);
+	/*
+	 * Meanwhile the lower mark is based on the average value
+	 * because it is the lowest possible consecutive-mark for this
+	 * device. Once that mark is hit and boosting is stopped, the
+	 * interrupt is disabled by ISR.
+	 */
+	freq = dev->avg_count / ACTMON_SAMPLING_PERIOD * KHZ;
+	tegra_actmon_get_lower_upper(tegra, dev, freq, &lower, &upper);
 
-	device_writel(dev, do_percent(val, dev->config->boost_down_threshold),
-		      ACTMON_DEV_LOWER_WMARK);
+	delta = do_percent(upper - lower, dev->config->boost_down_threshold);
+	device_writel(dev, lower + delta, ACTMON_DEV_LOWER_WMARK);
 }
 
 static void actmon_isr_device(struct tegra_devfreq *tegra,
 			      struct tegra_devfreq_device *dev)
 {
-	u32 intr_status, dev_ctrl;
+	u32 intr_status, dev_ctrl, avg_intr_mask;
 
 	dev->avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
-	tegra_devfreq_update_avg_wmark(tegra, dev);
-
 	intr_status = device_readl(dev, ACTMON_DEV_INTR_STATUS);
 	dev_ctrl = device_readl(dev, ACTMON_DEV_CTRL);
 
+	avg_intr_mask = ACTMON_DEV_INTR_AVG_BELOW_WMARK |
+			ACTMON_DEV_INTR_AVG_ABOVE_WMARK;
+
+	if (intr_status & avg_intr_mask)
+		tegra_devfreq_update_avg_wmark(tegra, dev);
+
 	if (intr_status & ACTMON_DEV_INTR_CONSECUTIVE_UPPER) {
 		/*
 		 * new_boost = min(old_boost * up_coef + step, max_freq)
@@ -253,8 +390,6 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
 
 		if (dev->boost_freq >= tegra->max_freq)
 			dev->boost_freq = tegra->max_freq;
-		else
-			dev_ctrl |= ACTMON_DEV_CTRL_CONSECUTIVE_ABOVE_WMARK_EN;
 	} else if (intr_status & ACTMON_DEV_INTR_CONSECUTIVE_LOWER) {
 		/*
 		 * new_boost = old_boost * down_coef
@@ -263,63 +398,37 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
 		dev->boost_freq = do_percent(dev->boost_freq,
 					     dev->config->boost_down_coeff);
 
-		dev_ctrl |= ACTMON_DEV_CTRL_CONSECUTIVE_ABOVE_WMARK_EN;
-
 		if (dev->boost_freq < (ACTMON_BOOST_FREQ_STEP >> 1))
 			dev->boost_freq = 0;
-		else
-			dev_ctrl |= ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
 	}
 
-	if (dev->config->avg_dependency_threshold) {
-		if (dev->avg_count >= dev->config->avg_dependency_threshold)
-			dev_ctrl |= ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
-		else if (dev->boost_freq == 0)
-			dev_ctrl &= ~ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
+	if (intr_status & avg_intr_mask) {
+		/*
+		 * Once average watermark is hit, it means that the memory
+		 * activity changed significantly and thus boosting-up shall
+		 * be reset because EMC clock rate will be changed and
+		 * boosting will restart in this case.
+		 */
+		dev->boost_freq = 0;
 	}
 
-	device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
+	/* no boosting => no need for consecutive-down interrupt */
+	if (dev->boost_freq == 0)
+		dev_ctrl &= ~ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
 
+	device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
 	device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
 }
 
-static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq *tegra,
-					    unsigned long cpu_freq)
-{
-	unsigned int i;
-	struct tegra_actmon_emc_ratio *ratio = actmon_emc_ratios;
-
-	for (i = 0; i < ARRAY_SIZE(actmon_emc_ratios); i++, ratio++) {
-		if (cpu_freq >= ratio->cpu_freq) {
-			if (ratio->emc_freq >= tegra->max_freq)
-				return tegra->max_freq;
-			else
-				return ratio->emc_freq;
-		}
-	}
-
-	return 0;
-}
-
-static void actmon_update_target(struct tegra_devfreq *tegra,
-				 struct tegra_devfreq_device *dev)
+static unsigned long actmon_update_target(struct tegra_devfreq *tegra,
+					  struct tegra_devfreq_device *dev)
 {
-	unsigned long cpu_freq = 0;
-	unsigned long static_cpu_emc_freq = 0;
-	unsigned int avg_sustain_coef;
-
-	if (dev->config->avg_dependency_threshold) {
-		cpu_freq = cpufreq_get(0);
-		static_cpu_emc_freq = actmon_cpu_to_emc_rate(tegra, cpu_freq);
-	}
+	unsigned long target_freq;
 
-	dev->target_freq = dev->avg_count / ACTMON_SAMPLING_PERIOD;
-	avg_sustain_coef = 100 * 100 / dev->config->boost_up_threshold;
-	dev->target_freq = do_percent(dev->target_freq, avg_sustain_coef);
-	dev->target_freq += dev->boost_freq;
+	target_freq = dev->avg_count / ACTMON_SAMPLING_PERIOD + dev->boost_freq;
+	target_freq = tegra_actmon_account_cpu_freq(tegra, dev, target_freq);
 
-	if (dev->avg_count >= dev->config->avg_dependency_threshold)
-		dev->target_freq = max(dev->target_freq, static_cpu_emc_freq);
+	return target_freq;
 }
 
 static irqreturn_t actmon_thread_isr(int irq, void *data)
@@ -351,8 +460,8 @@ static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
 				       unsigned long action, void *ptr)
 {
 	struct clk_notifier_data *data = ptr;
-	struct tegra_devfreq *tegra;
 	struct tegra_devfreq_device *dev;
+	struct tegra_devfreq *tegra;
 	unsigned int i;
 
 	if (action != POST_RATE_CHANGE)
@@ -360,12 +469,28 @@ static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
 
 	tegra = container_of(nb, struct tegra_devfreq, rate_change_nb);
 
-	tegra->cur_freq = data->new_rate / KHZ;
-
+	/*
+	 * EMC rate could change due to three reasons:
+	 *
+	 *    1. Average watermark hit
+	 *    2. Boosting overflow
+	 *    3. CPU freq change
+	 *
+	 * Once rate is changed, the consecutive watermarks need to be
+	 * updated in order for boosting to work properly and to avoid
+	 * unnecessary interrupts. Note that the consecutive range is set for
+	 * all of devices using the same rate, hence if CPU is doing much
+	 * less than the other memory clients, then its upper watermark will
+	 * be very high in comparison to the actual activity (lower watermark)
+	 * and thus unnecessary upper-interrupts will be suppressed.
+	 *
+	 * The average watermarks also should be updated because of 3.
+	 */
 	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++) {
 		dev = &tegra->devices[i];
 
-		tegra_devfreq_update_wmark(tegra, dev);
+		tegra_devfreq_update_avg_wmark(tegra, dev);
+		tegra_devfreq_update_wmark(tegra, dev, data->new_rate);
 	}
 
 	return NOTIFY_OK;
@@ -374,15 +499,14 @@ static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
 static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
 					  struct tegra_devfreq_device *dev)
 {
-	u32 val = 0;
-
-	dev->target_freq = tegra->cur_freq;
+	u32 val = 0, target_freq;
 
-	dev->avg_count = tegra->cur_freq * ACTMON_SAMPLING_PERIOD;
+	target_freq = clk_get_rate(tegra->emc_clock) / KHZ;
+	dev->avg_count = target_freq * ACTMON_SAMPLING_PERIOD;
 	device_writel(dev, dev->avg_count, ACTMON_DEV_INIT_AVG);
 
 	tegra_devfreq_update_avg_wmark(tegra, dev);
-	tegra_devfreq_update_wmark(tegra, dev);
+	tegra_devfreq_update_wmark(tegra, dev, target_freq);
 
 	device_writel(dev, ACTMON_COUNT_WEIGHT, ACTMON_DEV_COUNT_WEIGHT);
 	device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
@@ -469,13 +593,13 @@ static int tegra_devfreq_get_dev_status(struct device *dev,
 	struct tegra_devfreq_device *actmon_dev;
 	unsigned long cur_freq;
 
-	cur_freq = READ_ONCE(tegra->cur_freq);
+	cur_freq = clk_get_rate(tegra->emc_clock);
 
 	/* To be used by the tegra governor */
 	stat->private_data = tegra;
 
 	/* The below are to be used by the other governors */
-	stat->current_frequency = cur_freq * KHZ;
+	stat->current_frequency = cur_freq;
 
 	actmon_dev = &tegra->devices[MCALL];
 
@@ -486,7 +610,7 @@ static int tegra_devfreq_get_dev_status(struct device *dev,
 	stat->busy_time *= 100 / BUS_SATURATION_RATIO;
 
 	/* Number of cycles in a sampling period */
-	stat->total_time = ACTMON_SAMPLING_PERIOD * cur_freq;
+	stat->total_time = cur_freq / KHZ * ACTMON_SAMPLING_PERIOD;
 
 	stat->busy_time = min(stat->busy_time, stat->total_time);
 
@@ -505,6 +629,7 @@ static int tegra_governor_get_target(struct devfreq *devfreq,
 	struct devfreq_dev_status *stat;
 	struct tegra_devfreq *tegra;
 	struct tegra_devfreq_device *dev;
+	unsigned long dev_target_freq;
 	unsigned long target_freq = 0;
 	unsigned int i;
 	int err;
@@ -520,9 +645,9 @@ static int tegra_governor_get_target(struct devfreq *devfreq,
 	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++) {
 		dev = &tegra->devices[i];
 
-		actmon_update_target(tegra, dev);
+		dev_target_freq = actmon_update_target(tegra, dev);
 
-		target_freq = max(target_freq, dev->target_freq);
+		target_freq = max(target_freq, dev_target_freq);
 	}
 
 	*freq = target_freq * KHZ;
@@ -642,7 +767,6 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
 		return rate;
 	}
 
-	tegra->cur_freq = clk_get_rate(tegra->emc_clock) / KHZ;
 	tegra->max_freq = rate / KHZ;
 
 	for (i = 0; i < ARRAY_SIZE(actmon_device_configs); i++) {
@@ -671,7 +795,8 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
 	platform_set_drvdata(pdev, tegra);
 
 	tegra->rate_change_nb.notifier_call = tegra_actmon_rate_notify_cb;
-	err = clk_notifier_register(tegra->emc_clock, &tegra->rate_change_nb);
+	err = clk_notifier_register(tegra->emc_clock,
+				    &tegra->rate_change_nb);
 	if (err) {
 		dev_err(&pdev->dev,
 			"Failed to register rate change notifier\n");
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 06/24] PM / devfreq: tegra30: Tune up boosting thresholds
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (4 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 05/24] PM / devfreq: tegra30: Set up watermarks properly Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-16 11:55   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 07/24] PM / devfreq: tegra30: Use CPUFreq notifier Dmitry Osipenko
                   ` (17 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Now that average-sustain coefficient / multiplier is gone, it won't hurt
to re-tune the boosting thresholds to get a bit harder boosting for MCALL
clients, resulting in a more reactive governing in a case of multimedia
applications usage like 3d / video.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 16f7e6cf3b99..2bf65409ddd8 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -111,8 +111,8 @@ static struct tegra_devfreq_device_config actmon_device_configs[] = {
 		.irq_mask = 1 << 26,
 		.boost_up_coeff = 200,
 		.boost_down_coeff = 50,
-		.boost_up_threshold = 60,
-		.boost_down_threshold = 40,
+		.boost_up_threshold = 50,
+		.boost_down_threshold = 25,
 	},
 	{
 		/* MCCPU: memory accesses from the CPUs */
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 07/24] PM / devfreq: tegra30: Use CPUFreq notifier
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (5 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 06/24] PM / devfreq: tegra30: Tune up boosting thresholds Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-16 12:08   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 08/24] PM / devfreq: tegra30: Move clk-notifier's registration to governor's start Dmitry Osipenko
                   ` (16 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

The CPU's client need to take into account that CPUFreq may change
while memory activity not, staying high. Thus an appropriate frequency
notifier should be used in addition to the clk-notifier.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 105 +++++++++++++++++++++++++-----
 1 file changed, 88 insertions(+), 17 deletions(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 2bf65409ddd8..48a799fa5f63 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -17,6 +17,7 @@
 #include <linux/platform_device.h>
 #include <linux/pm_opp.h>
 #include <linux/reset.h>
+#include <linux/workqueue.h>
 
 #include "governor.h"
 
@@ -154,7 +155,10 @@ struct tegra_devfreq {
 
 	struct clk		*emc_clock;
 	unsigned long		max_freq;
-	struct notifier_block	rate_change_nb;
+	struct notifier_block	clk_rate_change_nb;
+
+	struct work_struct	update_work;
+	struct notifier_block	cpu_rate_change_nb;
 
 	struct tegra_devfreq_device devices[ARRAY_SIZE(actmon_device_configs)];
 
@@ -456,8 +460,8 @@ static irqreturn_t actmon_thread_isr(int irq, void *data)
 	return handled ? IRQ_HANDLED : IRQ_NONE;
 }
 
-static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
-				       unsigned long action, void *ptr)
+static int tegra_actmon_clk_notify_cb(struct notifier_block *nb,
+				      unsigned long action, void *ptr)
 {
 	struct clk_notifier_data *data = ptr;
 	struct tegra_devfreq_device *dev;
@@ -467,7 +471,7 @@ static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
 	if (action != POST_RATE_CHANGE)
 		return NOTIFY_OK;
 
-	tegra = container_of(nb, struct tegra_devfreq, rate_change_nb);
+	tegra = container_of(nb, struct tegra_devfreq, clk_rate_change_nb);
 
 	/*
 	 * EMC rate could change due to three reasons:
@@ -496,6 +500,37 @@ static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
 	return NOTIFY_OK;
 }
 
+static void tegra_actmon_delayed_update(struct work_struct *work)
+{
+	struct tegra_devfreq *tegra = container_of(work, struct tegra_devfreq,
+						   update_work);
+
+	mutex_lock(&tegra->devfreq->lock);
+	update_devfreq(tegra->devfreq);
+	mutex_unlock(&tegra->devfreq->lock);
+}
+
+static int tegra_actmon_cpu_notify_cb(struct notifier_block *nb,
+				      unsigned long action, void *ptr)
+{
+	struct tegra_devfreq *tegra;
+
+	if (action != CPUFREQ_POSTCHANGE)
+		return NOTIFY_OK;
+
+	tegra = container_of(nb, struct tegra_devfreq, cpu_rate_change_nb);
+
+	/*
+	 * CPUFreq driver should support CPUFREQ_ASYNC_NOTIFICATION in order
+	 * to allow asynchronous notifications. This means we can't block
+	 * here for too long, otherwise CPUFreq's core will complain with a
+	 * warning splat.
+	 */
+	schedule_work(&tegra->update_work);
+
+	return NOTIFY_OK;
+}
+
 static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
 					  struct tegra_devfreq_device *dev)
 {
@@ -527,9 +562,16 @@ static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
 	device_writel(dev, val, ACTMON_DEV_CTRL);
 }
 
-static void tegra_actmon_start(struct tegra_devfreq *tegra)
+static void tegra_actmon_stop_device(struct tegra_devfreq_device *dev)
+{
+	device_writel(dev, 0x00000000, ACTMON_DEV_CTRL);
+	device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
+}
+
+static int tegra_actmon_start(struct tegra_devfreq *tegra)
 {
 	unsigned int i;
+	int err;
 
 	actmon_writel(tegra, ACTMON_SAMPLING_PERIOD - 1,
 		      ACTMON_GLB_PERIOD_CTRL);
@@ -537,7 +579,30 @@ static void tegra_actmon_start(struct tegra_devfreq *tegra)
 	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++)
 		tegra_actmon_configure_device(tegra, &tegra->devices[i]);
 
+	/*
+	 * We are estimating CPU's memory bandwidth requirement based on
+	 * amount of memory accesses and system's load, judging by CPU's
+	 * frequency. We also don't want to receive events about CPU's
+	 * frequency transaction when governor is stopped, hence notifier
+	 * is registered dynamically.
+	 */
+	err = cpufreq_register_notifier(&tegra->cpu_rate_change_nb,
+					CPUFREQ_TRANSITION_NOTIFIER);
+	if (err) {
+		dev_err(tegra->devfreq->dev.parent,
+			"Failed to register rate change notifier: %d\n", err);
+		goto err_stop;
+	}
+
 	enable_irq(tegra->irq);
+
+	return 0;
+
+err_stop:
+	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++)
+		tegra_actmon_stop_device(&tegra->devices[i]);
+
+	return err;
 }
 
 static void tegra_actmon_stop(struct tegra_devfreq *tegra)
@@ -546,11 +611,13 @@ static void tegra_actmon_stop(struct tegra_devfreq *tegra)
 
 	disable_irq(tegra->irq);
 
-	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++) {
-		device_writel(&tegra->devices[i], 0x00000000, ACTMON_DEV_CTRL);
-		device_writel(&tegra->devices[i], ACTMON_INTR_STATUS_CLEAR,
-			      ACTMON_DEV_INTR_STATUS);
-	}
+	cpufreq_unregister_notifier(&tegra->cpu_rate_change_nb,
+				    CPUFREQ_TRANSITION_NOTIFIER);
+
+	cancel_work_sync(&tegra->update_work);
+
+	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++)
+		tegra_actmon_stop_device(&tegra->devices[i]);
 }
 
 static int tegra_devfreq_target(struct device *dev, unsigned long *freq,
@@ -659,6 +726,7 @@ static int tegra_governor_event_handler(struct devfreq *devfreq,
 					unsigned int event, void *data)
 {
 	struct tegra_devfreq *tegra = dev_get_drvdata(devfreq->dev.parent);
+	int ret = 0;
 
 	/*
 	 * Couple device with the governor early as it is needed at
@@ -669,7 +737,7 @@ static int tegra_governor_event_handler(struct devfreq *devfreq,
 	switch (event) {
 	case DEVFREQ_GOV_START:
 		devfreq_monitor_start(devfreq);
-		tegra_actmon_start(tegra);
+		ret = tegra_actmon_start(tegra);
 		break;
 
 	case DEVFREQ_GOV_STOP:
@@ -684,11 +752,11 @@ static int tegra_governor_event_handler(struct devfreq *devfreq,
 
 	case DEVFREQ_GOV_RESUME:
 		devfreq_monitor_resume(devfreq);
-		tegra_actmon_start(tegra);
+		ret = tegra_actmon_start(tegra);
 		break;
 	}
 
-	return 0;
+	return ret;
 }
 
 static struct devfreq_governor tegra_devfreq_governor = {
@@ -794,9 +862,12 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
 
 	platform_set_drvdata(pdev, tegra);
 
-	tegra->rate_change_nb.notifier_call = tegra_actmon_rate_notify_cb;
+	tegra->cpu_rate_change_nb.notifier_call = tegra_actmon_cpu_notify_cb;
+	INIT_WORK(&tegra->update_work, tegra_actmon_delayed_update);
+
+	tegra->clk_rate_change_nb.notifier_call = tegra_actmon_clk_notify_cb;
 	err = clk_notifier_register(tegra->emc_clock,
-				    &tegra->rate_change_nb);
+				    &tegra->clk_rate_change_nb);
 	if (err) {
 		dev_err(&pdev->dev,
 			"Failed to register rate change notifier\n");
@@ -823,7 +894,7 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
 	devfreq_remove_governor(&tegra_devfreq_governor);
 
 unreg_notifier:
-	clk_notifier_unregister(tegra->emc_clock, &tegra->rate_change_nb);
+	clk_notifier_unregister(tegra->emc_clock, &tegra->clk_rate_change_nb);
 
 remove_opps:
 	dev_pm_opp_remove_all_dynamic(&pdev->dev);
@@ -841,7 +912,7 @@ static int tegra_devfreq_remove(struct platform_device *pdev)
 	devfreq_remove_device(tegra->devfreq);
 	devfreq_remove_governor(&tegra_devfreq_governor);
 
-	clk_notifier_unregister(tegra->emc_clock, &tegra->rate_change_nb);
+	clk_notifier_unregister(tegra->emc_clock, &tegra->clk_rate_change_nb);
 	dev_pm_opp_remove_all_dynamic(&pdev->dev);
 
 	reset_control_reset(tegra->reset);
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 08/24] PM / devfreq: tegra30: Move clk-notifier's registration to governor's start
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (6 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 07/24] PM / devfreq: tegra30: Use CPUFreq notifier Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-16 12:11   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 09/24] PM / devfreq: tegra30: Reset boosting on startup Dmitry Osipenko
                   ` (15 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

There is no point in receiving of the notifications while governor is
stopped, let's keep them disabled like we do for the CPU freq-change
notifications. This also fixes a potential use-after-free bug if
notification happens after device's removal.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 33 ++++++++++++++++++-------------
 1 file changed, 19 insertions(+), 14 deletions(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 48a799fa5f63..d5d04c25023b 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -576,6 +576,19 @@ static int tegra_actmon_start(struct tegra_devfreq *tegra)
 	actmon_writel(tegra, ACTMON_SAMPLING_PERIOD - 1,
 		      ACTMON_GLB_PERIOD_CTRL);
 
+	/*
+	 * CLK notifications are needed in order to reconfigure the upper
+	 * consecutive watermark in accordance to the actual clock rate
+	 * to avoid unnecessary upper interrupts.
+	 */
+	err = clk_notifier_register(tegra->emc_clock,
+				    &tegra->clk_rate_change_nb);
+	if (err) {
+		dev_err(tegra->devfreq->dev.parent,
+			"Failed to register rate change notifier\n");
+		return err;
+	}
+
 	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++)
 		tegra_actmon_configure_device(tegra, &tegra->devices[i]);
 
@@ -602,6 +615,8 @@ static int tegra_actmon_start(struct tegra_devfreq *tegra)
 	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++)
 		tegra_actmon_stop_device(&tegra->devices[i]);
 
+	clk_notifier_unregister(tegra->emc_clock, &tegra->clk_rate_change_nb);
+
 	return err;
 }
 
@@ -618,6 +633,8 @@ static void tegra_actmon_stop(struct tegra_devfreq *tegra)
 
 	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++)
 		tegra_actmon_stop_device(&tegra->devices[i]);
+
+	clk_notifier_unregister(tegra->emc_clock, &tegra->clk_rate_change_nb);
 }
 
 static int tegra_devfreq_target(struct device *dev, unsigned long *freq,
@@ -862,22 +879,14 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
 
 	platform_set_drvdata(pdev, tegra);
 
+	tegra->clk_rate_change_nb.notifier_call = tegra_actmon_clk_notify_cb;
 	tegra->cpu_rate_change_nb.notifier_call = tegra_actmon_cpu_notify_cb;
 	INIT_WORK(&tegra->update_work, tegra_actmon_delayed_update);
 
-	tegra->clk_rate_change_nb.notifier_call = tegra_actmon_clk_notify_cb;
-	err = clk_notifier_register(tegra->emc_clock,
-				    &tegra->clk_rate_change_nb);
-	if (err) {
-		dev_err(&pdev->dev,
-			"Failed to register rate change notifier\n");
-		goto remove_opps;
-	}
-
 	err = devfreq_add_governor(&tegra_devfreq_governor);
 	if (err) {
 		dev_err(&pdev->dev, "Failed to add governor: %d\n", err);
-		goto unreg_notifier;
+		goto remove_opps;
 	}
 
 	tegra_devfreq_profile.initial_freq = clk_get_rate(tegra->emc_clock);
@@ -893,9 +902,6 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
 remove_governor:
 	devfreq_remove_governor(&tegra_devfreq_governor);
 
-unreg_notifier:
-	clk_notifier_unregister(tegra->emc_clock, &tegra->clk_rate_change_nb);
-
 remove_opps:
 	dev_pm_opp_remove_all_dynamic(&pdev->dev);
 
@@ -912,7 +918,6 @@ static int tegra_devfreq_remove(struct platform_device *pdev)
 	devfreq_remove_device(tegra->devfreq);
 	devfreq_remove_governor(&tegra_devfreq_governor);
 
-	clk_notifier_unregister(tegra->emc_clock, &tegra->clk_rate_change_nb);
 	dev_pm_opp_remove_all_dynamic(&pdev->dev);
 
 	reset_control_reset(tegra->reset);
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 09/24] PM / devfreq: tegra30: Reset boosting on startup
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (7 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 08/24] PM / devfreq: tegra30: Move clk-notifier's registration to governor's start Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-16 12:13   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 10/24] PM / devfreq: tegra30: Don't enable consecutive-down interrupt " Dmitry Osipenko
                   ` (14 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Governor could be stopped while boosting is active. We have assumption
that everything is reset on governor's restart, including the boosting
value, which was missed.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index d5d04c25023b..32fe95458ee7 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -536,6 +536,9 @@ static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
 {
 	u32 val = 0, target_freq;
 
+	/* we don't want boosting on restart */
+	dev->boost_freq = 0;
+
 	target_freq = clk_get_rate(tegra->emc_clock) / KHZ;
 	dev->avg_count = target_freq * ACTMON_SAMPLING_PERIOD;
 	device_writel(dev, dev->avg_count, ACTMON_DEV_INIT_AVG);
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 10/24] PM / devfreq: tegra30: Don't enable consecutive-down interrupt on startup
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (8 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 09/24] PM / devfreq: tegra30: Reset boosting on startup Dmitry Osipenko
@ 2019-07-07 22:32 ` " Dmitry Osipenko
  2019-07-16 12:17   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 11/24] PM / devfreq: tegra30: Add debug messages Dmitry Osipenko
                   ` (13 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

The consecutive-down event tells that we should perform frequency
de-boosting, but boosting is in a reset state on start and hence the
event won't do anything useful for us and it will be just a dummy
interrupt request.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 32fe95458ee7..878c9396bb8c 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -558,7 +558,6 @@ static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
 		<< ACTMON_DEV_CTRL_CONSECUTIVE_ABOVE_WMARK_NUM_SHIFT;
 	val |= ACTMON_DEV_CTRL_AVG_ABOVE_WMARK_EN;
 	val |= ACTMON_DEV_CTRL_AVG_BELOW_WMARK_EN;
-	val |= ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
 	val |= ACTMON_DEV_CTRL_CONSECUTIVE_ABOVE_WMARK_EN;
 	val |= ACTMON_DEV_CTRL_ENB;
 
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 11/24] PM / devfreq: tegra30: Add debug messages
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (9 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 10/24] PM / devfreq: tegra30: Don't enable consecutive-down interrupt " Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-16 12:23   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 12/24] PM / devfreq: tegra30: Inline all one-line functions Dmitry Osipenko
                   ` (12 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Add debug messages to know about what's happening in hardware and how
driver reacts.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 35 +++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 878c9396bb8c..c6c4a07d3e07 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -41,6 +41,7 @@
 #define ACTMON_DEV_AVG_UPPER_WMARK				0x10
 #define ACTMON_DEV_AVG_LOWER_WMARK				0x14
 #define ACTMON_DEV_COUNT_WEIGHT					0x18
+#define ACTMON_DEV_COUNT					0x1c
 #define ACTMON_DEV_AVG_COUNT					0x20
 #define ACTMON_DEV_INTR_STATUS					0x24
 
@@ -276,6 +277,9 @@ static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
 					 unsigned long *lower,
 					 unsigned long *upper)
 {
+	struct device *ddev = tegra->devfreq->dev.parent;
+	u32 offset = dev->config->offset;
+
 	/*
 	 * Memory frequencies are guaranteed to have 1MHz granularity
 	 * and thus we need this rounding down to get a proper watermarks
@@ -288,6 +292,9 @@ static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
 	*lower = tegra_actmon_lower_freq(tegra, target_freq);
 	*upper = tegra_actmon_upper_freq(tegra, target_freq);
 
+	dev_dbg(ddev, "%03x: target_freq %lu lower freq %lu upper freq %lu\n",
+		offset, target_freq, *lower, *upper);
+
 	*lower /= KHZ;
 	*upper /= KHZ;
 
@@ -367,11 +374,31 @@ static void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
 	device_writel(dev, lower + delta, ACTMON_DEV_LOWER_WMARK);
 }
 
+static void actmon_device_debug(struct tegra_devfreq *tegra,
+				struct tegra_devfreq_device *dev,
+				const char *prefix)
+{
+	dev_dbg(tegra->devfreq->dev.parent,
+		"%03x: %s: 0x%08x 0x%08x a %u %u %u c %u %u %u b %lu cpu %u\n",
+		dev->config->offset, prefix,
+		device_readl(dev, ACTMON_DEV_INTR_STATUS),
+		device_readl(dev, ACTMON_DEV_CTRL),
+		device_readl(dev, ACTMON_DEV_AVG_COUNT),
+		device_readl(dev, ACTMON_DEV_AVG_LOWER_WMARK),
+		device_readl(dev, ACTMON_DEV_AVG_UPPER_WMARK),
+		device_readl(dev, ACTMON_DEV_COUNT),
+		device_readl(dev, ACTMON_DEV_LOWER_WMARK),
+		device_readl(dev, ACTMON_DEV_UPPER_WMARK),
+		dev->boost_freq, cpufreq_get(0));
+}
+
 static void actmon_isr_device(struct tegra_devfreq *tegra,
 			      struct tegra_devfreq_device *dev)
 {
 	u32 intr_status, dev_ctrl, avg_intr_mask;
 
+	actmon_device_debug(tegra, dev, "isr+");
+
 	dev->avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
 	intr_status = device_readl(dev, ACTMON_DEV_INTR_STATUS);
 	dev_ctrl = device_readl(dev, ACTMON_DEV_CTRL);
@@ -422,6 +449,8 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
 
 	device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
 	device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
+
+	actmon_device_debug(tegra, dev, "isr-");
 }
 
 static unsigned long actmon_update_target(struct tegra_devfreq *tegra,
@@ -712,6 +741,7 @@ static struct devfreq_dev_profile tegra_devfreq_profile = {
 static int tegra_governor_get_target(struct devfreq *devfreq,
 				     unsigned long *freq)
 {
+	struct device *ddev = devfreq->dev.parent;
 	struct devfreq_dev_status *stat;
 	struct tegra_devfreq *tegra;
 	struct tegra_devfreq_device *dev;
@@ -734,6 +764,11 @@ static int tegra_governor_get_target(struct devfreq *devfreq,
 		dev_target_freq = actmon_update_target(tegra, dev);
 
 		target_freq = max(target_freq, dev_target_freq);
+
+		dev_dbg(ddev, "%03x: upd: dev_target_freq %lu\n",
+			dev->config->offset, dev_target_freq);
+
+		actmon_device_debug(tegra, dev, "upd");
 	}
 
 	*freq = target_freq * KHZ;
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 12/24] PM / devfreq: tegra30: Inline all one-line functions
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (10 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 11/24] PM / devfreq: tegra30: Add debug messages Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-16 12:26   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 13/24] PM / devfreq: tegra30: Constify structs Dmitry Osipenko
                   ` (11 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Depending on a kernel's configuration, a single line functions may not be
inlined by compiler (like enabled ftracing for example). Let's inline such
functions explicitly for consistency.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index c6c4a07d3e07..1a10df5dbbed 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -181,28 +181,29 @@ static struct tegra_actmon_emc_ratio actmon_emc_ratios[] = {
 	{  250000,    100000 },
 };
 
-static u32 actmon_readl(struct tegra_devfreq *tegra, u32 offset)
+static inline u32 actmon_readl(struct tegra_devfreq *tegra, u32 offset)
 {
 	return readl_relaxed(tegra->regs + offset);
 }
 
-static void actmon_writel(struct tegra_devfreq *tegra, u32 val, u32 offset)
+static inline void actmon_writel(struct tegra_devfreq *tegra,
+				 u32 val, u32 offset)
 {
 	writel_relaxed(val, tegra->regs + offset);
 }
 
-static u32 device_readl(struct tegra_devfreq_device *dev, u32 offset)
+static inline u32 device_readl(struct tegra_devfreq_device *dev, u32 offset)
 {
 	return readl_relaxed(dev->regs + offset);
 }
 
-static void device_writel(struct tegra_devfreq_device *dev, u32 val,
-			  u32 offset)
+static inline void device_writel(struct tegra_devfreq_device *dev,
+				 u32 val, u32 offset)
 {
 	writel_relaxed(val, dev->regs + offset);
 }
 
-static unsigned long do_percent(unsigned long val, unsigned int pct)
+static inline unsigned long do_percent(unsigned long val, unsigned int pct)
 {
 	return val * pct / 100;
 }
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 13/24] PM / devfreq: tegra30: Constify structs
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (11 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 12/24] PM / devfreq: tegra30: Inline all one-line functions Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-16 12:26   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 14/24] PM / devfreq: tegra30: Ensure that target freq won't overflow Dmitry Osipenko
                   ` (10 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Constify unmodifiable structs for consistency.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 1a10df5dbbed..2f59c78930bd 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -106,7 +106,7 @@ enum tegra_actmon_device {
 	MCCPU,
 };
 
-static struct tegra_devfreq_device_config actmon_device_configs[] = {
+static const struct tegra_devfreq_device_config actmon_device_configs[] = {
 	{
 		/* MCALL: All memory accesses (including from the CPUs) */
 		.offset = 0x1c0,
@@ -171,7 +171,7 @@ struct tegra_actmon_emc_ratio {
 	unsigned long emc_freq;
 };
 
-static struct tegra_actmon_emc_ratio actmon_emc_ratios[] = {
+static const struct tegra_actmon_emc_ratio actmon_emc_ratios[] = {
 	{ 1400000, ULONG_MAX },
 	{ 1200000,    750000 },
 	{ 1100000,    600000 },
@@ -210,7 +210,7 @@ static inline unsigned long do_percent(unsigned long val, unsigned int pct)
 
 static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq *tegra)
 {
-	struct tegra_actmon_emc_ratio *ratio = actmon_emc_ratios;
+	const struct tegra_actmon_emc_ratio *ratio = actmon_emc_ratios;
 	unsigned int cpu_freq = cpufreq_get(0);
 	unsigned int i;
 
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 14/24] PM / devfreq: tegra30: Ensure that target freq won't overflow
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (12 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 13/24] PM / devfreq: tegra30: Constify structs Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-16 12:30   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 15/24] PM / devfreq: tegra30: Fix integer overflow on CPU's freq max out Dmitry Osipenko
                   ` (9 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Potentially very high boosting could cause an integer overflow for a
highly clocked memory after conversion to MHz.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 2f59c78930bd..0de1efdaabf4 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -460,6 +460,7 @@ static unsigned long actmon_update_target(struct tegra_devfreq *tegra,
 	unsigned long target_freq;
 
 	target_freq = dev->avg_count / ACTMON_SAMPLING_PERIOD + dev->boost_freq;
+	target_freq = min(target_freq, ULONG_MAX / KHZ);
 	target_freq = tegra_actmon_account_cpu_freq(tegra, dev, target_freq);
 
 	return target_freq;
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 15/24] PM / devfreq: tegra30: Fix integer overflow on CPU's freq max out
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (13 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 14/24] PM / devfreq: tegra30: Ensure that target freq won't overflow Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-16 12:32   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 16/24] PM / devfreq: tegra30: Use kHz units uniformly in the code Dmitry Osipenko
                   ` (8 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

There is another kHz-conversion bug in the code, resulting in integer
overflow. Although, this time the resulting value is 4294966296 and it's
close to ULONG_MAX, which is okay in this case.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 0de1efdaabf4..60ebf9c9cd86 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -172,7 +172,7 @@ struct tegra_actmon_emc_ratio {
 };
 
 static const struct tegra_actmon_emc_ratio actmon_emc_ratios[] = {
-	{ 1400000, ULONG_MAX },
+	{ 1400000, ULONG_MAX / KHZ },
 	{ 1200000,    750000 },
 	{ 1100000,    600000 },
 	{ 1000000,    500000 },
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 16/24] PM / devfreq: tegra30: Use kHz units uniformly in the code
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (14 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 15/24] PM / devfreq: tegra30: Fix integer overflow on CPU's freq max out Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-07 22:32 ` [PATCH v4 17/24] PM / devfreq: tegra30: Use tracepoints for debugging Dmitry Osipenko
                   ` (7 subsequent siblings)
  23 siblings, 0 replies; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Now that all kHz-conversion related bugs are fixed, we can use the kHz
uniformly. This makes code cleaner and avoids integer divisions in the
code, which is useful in a case of Tegra30 that has Cortex A9 CPU that
doesn't support integer division instructions, hence all divisions are
actually made in software mode. Another small benefit from this change
is that now powertop utility correctly displays devfreq's stats, for
some reason it expects them to be in kHz.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 78 +++++++++++++++++--------------
 1 file changed, 43 insertions(+), 35 deletions(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 60ebf9c9cd86..6ebf0f505767 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -72,6 +72,8 @@
 
 #define KHZ							1000
 
+#define KHZ_MAX						(ULONG_MAX / KHZ)
+
 /* Assume that the bus is saturated if the utilization is 25% */
 #define BUS_SATURATION_RATIO					25
 
@@ -124,7 +126,7 @@ static const struct tegra_devfreq_device_config actmon_device_configs[] = {
 		.boost_down_coeff = 90,
 		.boost_up_threshold = 27,
 		.boost_down_threshold = 10,
-		.avg_dependency_threshold = 50000,
+		.avg_dependency_threshold = 50000 / ACTMON_SAMPLING_PERIOD,
 	},
 };
 
@@ -137,8 +139,11 @@ struct tegra_devfreq_device {
 	const struct tegra_devfreq_device_config *config;
 	void __iomem *regs;
 
-	/* Average event count sampled in the last interrupt */
-	u32 avg_count;
+	/*
+	 * Average event count sampled in the last interrupt and converted
+	 * to frequency value.
+	 */
+	u32 avg_freq;
 
 	/*
 	 * Extra frequency to increase the target by due to consecutive
@@ -172,7 +177,7 @@ struct tegra_actmon_emc_ratio {
 };
 
 static const struct tegra_actmon_emc_ratio actmon_emc_ratios[] = {
-	{ 1400000, ULONG_MAX / KHZ },
+	{ 1400000,    KHZ_MAX },
 	{ 1200000,    750000 },
 	{ 1100000,    600000 },
 	{ 1000000,    500000 },
@@ -234,7 +239,7 @@ tegra_actmon_account_cpu_freq(struct tegra_devfreq *tegra,
 	unsigned long static_cpu_emc_freq;
 
 	if (dev->config->avg_dependency_threshold &&
-	    dev->config->avg_dependency_threshold < dev->avg_count) {
+	    dev->config->avg_dependency_threshold < dev->avg_freq) {
 		static_cpu_emc_freq = actmon_cpu_to_emc_rate(tegra);
 		target_freq = max(target_freq, static_cpu_emc_freq);
 	}
@@ -265,7 +270,7 @@ static unsigned long tegra_actmon_upper_freq(struct tegra_devfreq *tegra,
 
 	opp = dev_pm_opp_find_freq_ceil(tegra->devfreq->dev.parent, &upper);
 	if (IS_ERR(opp))
-		upper = ULONG_MAX;
+		upper = KHZ_MAX;
 	else
 		dev_pm_opp_put(opp);
 
@@ -287,7 +292,7 @@ static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
 	 * range in a case where target_freq falls into a range of
 	 * next_possible_opp_freq - 1MHz.
 	 */
-	target_freq = round_down(target_freq, 1000000);
+	target_freq = round_down(target_freq, 1000);
 
 	/* watermarks are set at the borders of the corresponding OPPs */
 	*lower = tegra_actmon_lower_freq(tegra, target_freq);
@@ -296,9 +301,6 @@ static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
 	dev_dbg(ddev, "%03x: target_freq %lu lower freq %lu upper freq %lu\n",
 		offset, target_freq, *lower, *upper);
 
-	*lower /= KHZ;
-	*upper /= KHZ;
-
 	/*
 	 * The upper watermark should take into account CPU's frequency
 	 * because cpu_to_emc_rate() may override the target_freq with
@@ -314,22 +316,23 @@ static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
 static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
 					   struct tegra_devfreq_device *dev)
 {
-	unsigned long lower, upper, freq;
+	unsigned long avg_threshold, lower, upper;
 
-	freq = dev->avg_count / ACTMON_SAMPLING_PERIOD * KHZ;
-	tegra_actmon_get_lower_upper(tegra, dev, freq, &lower, &upper);
+	tegra_actmon_get_lower_upper(tegra, dev, dev->avg_freq, &lower, &upper);
+
+	avg_threshold = dev->config->avg_dependency_threshold;
+	avg_threshold = avg_threshold * ACTMON_SAMPLING_PERIOD;
 
 	/*
 	 * We want to get interrupts when MCCPU client crosses the
 	 * dependency threshold in order to take into / out of account
 	 * the CPU's freq.
 	 */
-	if (lower < dev->config->avg_dependency_threshold &&
-	    upper > dev->config->avg_dependency_threshold) {
-		if (dev->avg_count < dev->config->avg_dependency_threshold)
-			upper = dev->config->avg_dependency_threshold;
+	if (lower < avg_threshold && upper > avg_threshold) {
+		if (dev->avg_freq < dev->config->avg_dependency_threshold)
+			upper = avg_threshold;
 		else
-			lower = dev->config->avg_dependency_threshold;
+			lower = avg_threshold;
 	}
 
 	device_writel(dev, lower, ACTMON_DEV_AVG_LOWER_WMARK);
@@ -368,8 +371,7 @@ static void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
 	 * device. Once that mark is hit and boosting is stopped, the
 	 * interrupt is disabled by ISR.
 	 */
-	freq = dev->avg_count / ACTMON_SAMPLING_PERIOD * KHZ;
-	tegra_actmon_get_lower_upper(tegra, dev, freq, &lower, &upper);
+	tegra_actmon_get_lower_upper(tegra, dev, dev->avg_freq, &lower, &upper);
 
 	delta = do_percent(upper - lower, dev->config->boost_down_threshold);
 	device_writel(dev, lower + delta, ACTMON_DEV_LOWER_WMARK);
@@ -396,14 +398,16 @@ static void actmon_device_debug(struct tegra_devfreq *tegra,
 static void actmon_isr_device(struct tegra_devfreq *tegra,
 			      struct tegra_devfreq_device *dev)
 {
-	u32 intr_status, dev_ctrl, avg_intr_mask;
+	u32 intr_status, dev_ctrl, avg_intr_mask, avg_count;
 
 	actmon_device_debug(tegra, dev, "isr+");
 
-	dev->avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
 	intr_status = device_readl(dev, ACTMON_DEV_INTR_STATUS);
+	avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
 	dev_ctrl = device_readl(dev, ACTMON_DEV_CTRL);
 
+	dev->avg_freq = avg_count / ACTMON_SAMPLING_PERIOD;
+
 	avg_intr_mask = ACTMON_DEV_INTR_AVG_BELOW_WMARK |
 			ACTMON_DEV_INTR_AVG_ABOVE_WMARK;
 
@@ -459,8 +463,7 @@ static unsigned long actmon_update_target(struct tegra_devfreq *tegra,
 {
 	unsigned long target_freq;
 
-	target_freq = dev->avg_count / ACTMON_SAMPLING_PERIOD + dev->boost_freq;
-	target_freq = min(target_freq, ULONG_MAX / KHZ);
+	target_freq = min(dev->avg_freq + dev->boost_freq, KHZ_MAX);
 	target_freq = tegra_actmon_account_cpu_freq(tegra, dev, target_freq);
 
 	return target_freq;
@@ -497,6 +500,7 @@ static int tegra_actmon_clk_notify_cb(struct notifier_block *nb,
 	struct clk_notifier_data *data = ptr;
 	struct tegra_devfreq_device *dev;
 	struct tegra_devfreq *tegra;
+	unsigned long freq;
 	unsigned int i;
 
 	if (action != POST_RATE_CHANGE)
@@ -504,6 +508,8 @@ static int tegra_actmon_clk_notify_cb(struct notifier_block *nb,
 
 	tegra = container_of(nb, struct tegra_devfreq, clk_rate_change_nb);
 
+	freq = data->new_rate / KHZ;
+
 	/*
 	 * EMC rate could change due to three reasons:
 	 *
@@ -525,7 +531,7 @@ static int tegra_actmon_clk_notify_cb(struct notifier_block *nb,
 		dev = &tegra->devices[i];
 
 		tegra_devfreq_update_avg_wmark(tegra, dev);
-		tegra_devfreq_update_wmark(tegra, dev, data->new_rate);
+		tegra_devfreq_update_wmark(tegra, dev, freq);
 	}
 
 	return NOTIFY_OK;
@@ -565,17 +571,17 @@ static int tegra_actmon_cpu_notify_cb(struct notifier_block *nb,
 static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
 					  struct tegra_devfreq_device *dev)
 {
-	u32 val = 0, target_freq;
+	u32 val = 0;
 
 	/* we don't want boosting on restart */
 	dev->boost_freq = 0;
 
-	target_freq = clk_get_rate(tegra->emc_clock) / KHZ;
-	dev->avg_count = target_freq * ACTMON_SAMPLING_PERIOD;
-	device_writel(dev, dev->avg_count, ACTMON_DEV_INIT_AVG);
+	dev->avg_freq = clk_get_rate(tegra->emc_clock) / KHZ;
+	device_writel(dev, dev->avg_freq * ACTMON_SAMPLING_PERIOD,
+		      ACTMON_DEV_INIT_AVG);
 
 	tegra_devfreq_update_avg_wmark(tegra, dev);
-	tegra_devfreq_update_wmark(tegra, dev, target_freq);
+	tegra_devfreq_update_wmark(tegra, dev, dev->avg_freq);
 
 	device_writel(dev, ACTMON_COUNT_WEIGHT, ACTMON_DEV_COUNT_WEIGHT);
 	device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
@@ -687,7 +693,7 @@ static int tegra_devfreq_target(struct device *dev, unsigned long *freq,
 	rate = dev_pm_opp_get_freq(opp);
 	dev_pm_opp_put(opp);
 
-	err = clk_set_min_rate(tegra->emc_clock, rate);
+	err = clk_set_min_rate(tegra->emc_clock, rate * KHZ);
 	if (err)
 		return err;
 
@@ -710,7 +716,7 @@ static int tegra_devfreq_get_dev_status(struct device *dev,
 	struct tegra_devfreq_device *actmon_dev;
 	unsigned long cur_freq;
 
-	cur_freq = clk_get_rate(tegra->emc_clock);
+	cur_freq = clk_get_rate(tegra->emc_clock) / KHZ;
 
 	/* To be used by the tegra governor */
 	stat->private_data = tegra;
@@ -727,7 +733,7 @@ static int tegra_devfreq_get_dev_status(struct device *dev,
 	stat->busy_time *= 100 / BUS_SATURATION_RATIO;
 
 	/* Number of cycles in a sampling period */
-	stat->total_time = cur_freq / KHZ * ACTMON_SAMPLING_PERIOD;
+	stat->total_time = cur_freq * ACTMON_SAMPLING_PERIOD;
 
 	stat->busy_time = min(stat->busy_time, stat->total_time);
 
@@ -773,7 +779,7 @@ static int tegra_governor_get_target(struct devfreq *devfreq,
 		actmon_device_debug(tegra, dev, "upd");
 	}
 
-	*freq = target_freq * KHZ;
+	*freq = target_freq;
 
 	return 0;
 }
@@ -909,7 +915,7 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
 			goto remove_opps;
 		}
 
-		err = dev_pm_opp_add(&pdev->dev, rate, 0);
+		err = dev_pm_opp_add(&pdev->dev, rate / KHZ, 0);
 		if (err) {
 			dev_err(&pdev->dev, "Failed to add OPP: %d\n", err);
 			goto remove_opps;
@@ -929,6 +935,8 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
 	}
 
 	tegra_devfreq_profile.initial_freq = clk_get_rate(tegra->emc_clock);
+	tegra_devfreq_profile.initial_freq /= KHZ;
+
 	devfreq = devfreq_add_device(&pdev->dev, &tegra_devfreq_profile,
 				     "tegra_actmon", NULL);
 	if (IS_ERR(tegra->devfreq)) {
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 17/24] PM / devfreq: tegra30: Use tracepoints for debugging
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (15 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 16/24] PM / devfreq: tegra30: Use kHz units uniformly in the code Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-18  9:47   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 18/24] PM / devfreq: tegra30: Optimize CPUFreq notifier Dmitry Osipenko
                   ` (6 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Debug messages create too much CPU and memory activity by themselves,
so it's difficult to debug lower rates and catch unwanted interrupts
that happen rarely. Tracepoints are ideal in that regards because they
do not contribute to the sampled date at all. This allowed me to catch
few problems which are fixed by the followup patches, without tracepoints
it would be much harder to do.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c      |  43 +++-------
 include/trace/events/tegra30_devfreq.h | 105 +++++++++++++++++++++++++
 2 files changed, 117 insertions(+), 31 deletions(-)
 create mode 100644 include/trace/events/tegra30_devfreq.h

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 6ebf0f505767..43c9c5fbfe91 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -19,6 +19,9 @@
 #include <linux/reset.h>
 #include <linux/workqueue.h>
 
+#define CREATE_TRACE_POINTS
+#include <trace/events/tegra30_devfreq.h>
+
 #include "governor.h"
 
 #define ACTMON_GLB_STATUS					0x0
@@ -283,9 +286,6 @@ static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
 					 unsigned long *lower,
 					 unsigned long *upper)
 {
-	struct device *ddev = tegra->devfreq->dev.parent;
-	u32 offset = dev->config->offset;
-
 	/*
 	 * Memory frequencies are guaranteed to have 1MHz granularity
 	 * and thus we need this rounding down to get a proper watermarks
@@ -298,8 +298,8 @@ static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
 	*lower = tegra_actmon_lower_freq(tegra, target_freq);
 	*upper = tegra_actmon_upper_freq(tegra, target_freq);
 
-	dev_dbg(ddev, "%03x: target_freq %lu lower freq %lu upper freq %lu\n",
-		offset, target_freq, *lower, *upper);
+	trace_device_lower_upper(dev->config->offset, target_freq,
+				 *lower, *upper);
 
 	/*
 	 * The upper watermark should take into account CPU's frequency
@@ -377,30 +377,13 @@ static void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
 	device_writel(dev, lower + delta, ACTMON_DEV_LOWER_WMARK);
 }
 
-static void actmon_device_debug(struct tegra_devfreq *tegra,
-				struct tegra_devfreq_device *dev,
-				const char *prefix)
-{
-	dev_dbg(tegra->devfreq->dev.parent,
-		"%03x: %s: 0x%08x 0x%08x a %u %u %u c %u %u %u b %lu cpu %u\n",
-		dev->config->offset, prefix,
-		device_readl(dev, ACTMON_DEV_INTR_STATUS),
-		device_readl(dev, ACTMON_DEV_CTRL),
-		device_readl(dev, ACTMON_DEV_AVG_COUNT),
-		device_readl(dev, ACTMON_DEV_AVG_LOWER_WMARK),
-		device_readl(dev, ACTMON_DEV_AVG_UPPER_WMARK),
-		device_readl(dev, ACTMON_DEV_COUNT),
-		device_readl(dev, ACTMON_DEV_LOWER_WMARK),
-		device_readl(dev, ACTMON_DEV_UPPER_WMARK),
-		dev->boost_freq, cpufreq_get(0));
-}
-
 static void actmon_isr_device(struct tegra_devfreq *tegra,
 			      struct tegra_devfreq_device *dev)
 {
 	u32 intr_status, dev_ctrl, avg_intr_mask, avg_count;
 
-	actmon_device_debug(tegra, dev, "isr+");
+	trace_device_isr_enter(tegra->regs, dev->config->offset,
+			       dev->boost_freq, cpufreq_get(0));
 
 	intr_status = device_readl(dev, ACTMON_DEV_INTR_STATUS);
 	avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
@@ -455,7 +438,8 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
 	device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
 	device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
 
-	actmon_device_debug(tegra, dev, "isr-");
+	trace_device_isr_exit(tegra->regs, dev->config->offset,
+			      dev->boost_freq, cpufreq_get(0));
 }
 
 static unsigned long actmon_update_target(struct tegra_devfreq *tegra,
@@ -749,7 +733,6 @@ static struct devfreq_dev_profile tegra_devfreq_profile = {
 static int tegra_governor_get_target(struct devfreq *devfreq,
 				     unsigned long *freq)
 {
-	struct device *ddev = devfreq->dev.parent;
 	struct devfreq_dev_status *stat;
 	struct tegra_devfreq *tegra;
 	struct tegra_devfreq_device *dev;
@@ -770,13 +753,11 @@ static int tegra_governor_get_target(struct devfreq *devfreq,
 		dev = &tegra->devices[i];
 
 		dev_target_freq = actmon_update_target(tegra, dev);
-
 		target_freq = max(target_freq, dev_target_freq);
 
-		dev_dbg(ddev, "%03x: upd: dev_target_freq %lu\n",
-			dev->config->offset, dev_target_freq);
-
-		actmon_device_debug(tegra, dev, "upd");
+		trace_device_target_freq(dev->config->offset, dev_target_freq);
+		trace_device_target_update(tegra->regs, dev->config->offset,
+					   dev->boost_freq, cpufreq_get(0));
 	}
 
 	*freq = target_freq;
diff --git a/include/trace/events/tegra30_devfreq.h b/include/trace/events/tegra30_devfreq.h
new file mode 100644
index 000000000000..8f264a489daf
--- /dev/null
+++ b/include/trace/events/tegra30_devfreq.h
@@ -0,0 +1,105 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM tegra30_devfreq
+
+#if !defined(_TRACE_TEGRA30_DEVFREQ_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_TEGRA30_DEVFREQ_H
+
+#include <linux/io.h>
+#include <linux/tracepoint.h>
+#include <linux/types.h>
+
+DECLARE_EVENT_CLASS(device_state,
+	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32 cpufreq),
+	TP_ARGS(base, offset, boost, cpufreq),
+	TP_STRUCT__entry(
+		__field(u32, offset)
+		__field(u32, intr_status)
+		__field(u32, ctrl)
+		__field(u32, avg_count)
+		__field(u32, avg_lower)
+		__field(u32, avg_upper)
+		__field(u32, count)
+		__field(u32, lower)
+		__field(u32, upper)
+		__field(u32, boost_freq)
+		__field(u32, cpu_freq)
+	),
+	TP_fast_assign(
+		__entry->offset		= offset;
+		__entry->intr_status	= readl_relaxed(base + offset + 0x24);
+		__entry->ctrl		= readl_relaxed(base + offset + 0x0);
+		__entry->avg_count	= readl_relaxed(base + offset + 0x20);
+		__entry->avg_lower	= readl_relaxed(base + offset + 0x14);
+		__entry->avg_upper	= readl_relaxed(base + offset + 0x10);
+		__entry->count		= readl_relaxed(base + offset + 0x1c);
+		__entry->lower		= readl_relaxed(base + offset + 0x8);
+		__entry->upper		= readl_relaxed(base + offset + 0x4);
+		__entry->boost_freq	= boost;
+		__entry->cpu_freq	= cpufreq;
+	),
+	TP_printk("%03x: intr 0x%08x ctrl 0x%08x avg %010u %010u %010u cnt %010u %010u %010u boost %010u cpu %u",
+		__entry->offset,
+		__entry->intr_status,
+		__entry->ctrl,
+		__entry->avg_count,
+		__entry->avg_lower,
+		__entry->avg_upper,
+		__entry->count,
+		__entry->lower,
+		__entry->upper,
+		__entry->boost_freq,
+		__entry->cpu_freq)
+);
+
+DEFINE_EVENT(device_state, device_isr_enter,
+	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32 cpufreq),
+	TP_ARGS(base, offset, boost, cpufreq));
+
+DEFINE_EVENT(device_state, device_isr_exit,
+	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32 cpufreq),
+	TP_ARGS(base, offset, boost, cpufreq));
+
+DEFINE_EVENT(device_state, device_target_update,
+	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32 cpufreq),
+	TP_ARGS(base, offset, boost, cpufreq));
+
+TRACE_EVENT(device_lower_upper,
+	TP_PROTO(u32 offset, u32 target, u32 lower, u32 upper),
+	TP_ARGS(offset, target, lower, upper),
+	TP_STRUCT__entry(
+		__field(u32, offset)
+		__field(u32, target)
+		__field(u32, lower)
+		__field(u32, upper)
+	),
+	TP_fast_assign(
+		__entry->offset = offset;
+		__entry->target = target;
+		__entry->lower = lower;
+		__entry->upper = upper;
+	),
+	TP_printk("%03x: freq %010u lower freq %010u upper freq %010u",
+		__entry->offset,
+		__entry->target,
+		__entry->lower,
+		__entry->upper)
+);
+
+TRACE_EVENT(device_target_freq,
+	TP_PROTO(u32 offset, u32 target),
+	TP_ARGS(offset, target),
+	TP_STRUCT__entry(
+		__field(u32, offset)
+		__field(u32, target)
+	),
+	TP_fast_assign(
+		__entry->offset = offset;
+		__entry->target = target;
+	),
+	TP_printk("%03x: freq %010u", __entry->offset, __entry->target)
+);
+#endif /* _TRACE_TEGRA30_DEVFREQ_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 18/24] PM / devfreq: tegra30: Optimize CPUFreq notifier
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (16 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 17/24] PM / devfreq: tegra30: Use tracepoints for debugging Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-18  9:48   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 19/24] PM / devfreq: tegra30: Optimize upper consecutive watermark selection Dmitry Osipenko
                   ` (5 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

When CPU's memory activity is low or memory activity is high such that
CPU's frequency contribution to the boosting is not taken into account,
then there is no need to schedule devfreq's update. This eliminates
unnecessary CPU activity during of idling caused by the scheduled work.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 73 +++++++++++++++++++++++++++----
 1 file changed, 64 insertions(+), 9 deletions(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 43c9c5fbfe91..8d6bf6e9f1ae 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -216,10 +216,10 @@ static inline unsigned long do_percent(unsigned long val, unsigned int pct)
 	return val * pct / 100;
 }
 
-static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq *tegra)
+static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq *tegra,
+					    unsigned int cpu_freq)
 {
 	const struct tegra_actmon_emc_ratio *ratio = actmon_emc_ratios;
-	unsigned int cpu_freq = cpufreq_get(0);
 	unsigned int i;
 
 	for (i = 0; i < ARRAY_SIZE(actmon_emc_ratios); i++, ratio++) {
@@ -239,15 +239,15 @@ tegra_actmon_account_cpu_freq(struct tegra_devfreq *tegra,
 			      struct tegra_devfreq_device *dev,
 			      unsigned long target_freq)
 {
-	unsigned long static_cpu_emc_freq;
+	unsigned long cpu_emc_freq = 0;
 
-	if (dev->config->avg_dependency_threshold &&
-	    dev->config->avg_dependency_threshold < dev->avg_freq) {
-		static_cpu_emc_freq = actmon_cpu_to_emc_rate(tegra);
-		target_freq = max(target_freq, static_cpu_emc_freq);
-	}
+	if (!dev->config->avg_dependency_threshold)
+		return target_freq;
 
-	return target_freq;
+	if (dev->avg_freq > dev->config->avg_dependency_threshold)
+		cpu_emc_freq = actmon_cpu_to_emc_rate(tegra, cpufreq_get(0));
+
+	return max(target_freq, cpu_emc_freq);
 }
 
 static unsigned long tegra_actmon_lower_freq(struct tegra_devfreq *tegra,
@@ -531,16 +531,71 @@ static void tegra_actmon_delayed_update(struct work_struct *work)
 	mutex_unlock(&tegra->devfreq->lock);
 }
 
+static unsigned long
+tegra_actmon_cpufreq_contribution(struct tegra_devfreq *tegra,
+				  unsigned int cpu_freq)
+{
+	unsigned long freq, static_cpu_emc_freq;
+
+	/* check whether CPU's freq is taken into account at all */
+	if (tegra->devices[MCCPU].avg_freq <=
+	    tegra->devices[MCCPU].config->avg_dependency_threshold)
+		return 0;
+
+	static_cpu_emc_freq = actmon_cpu_to_emc_rate(tegra, cpu_freq);
+
+	/* compare static CPU-EMC freq with MCALL */
+	freq = tegra->devices[MCALL].avg_freq +
+	       tegra->devices[MCALL].boost_freq;
+
+	freq = tegra_actmon_upper_freq(tegra, freq);
+
+	if (freq == tegra->max_freq || freq >= static_cpu_emc_freq)
+		return 0;
+
+	/* compare static CPU-EMC freq with MCCPU */
+	freq = tegra->devices[MCCPU].avg_freq +
+	       tegra->devices[MCCPU].boost_freq;
+
+	freq = tegra_actmon_upper_freq(tegra, freq);
+
+	if (freq == tegra->max_freq || freq >= static_cpu_emc_freq)
+		return 0;
+
+	return static_cpu_emc_freq;
+}
+
 static int tegra_actmon_cpu_notify_cb(struct notifier_block *nb,
 				      unsigned long action, void *ptr)
 {
+	struct cpufreq_freqs *freqs = ptr;
 	struct tegra_devfreq *tegra;
+	unsigned long old, new;
 
 	if (action != CPUFREQ_POSTCHANGE)
 		return NOTIFY_OK;
 
 	tegra = container_of(nb, struct tegra_devfreq, cpu_rate_change_nb);
 
+	/*
+	 * Quickly check whether CPU frequency should be taken into account
+	 * at all, without blocking CPUFreq's core.
+	 */
+	if (mutex_trylock(&tegra->devfreq->lock)) {
+		old = tegra_actmon_cpufreq_contribution(tegra, freqs->old);
+		new = tegra_actmon_cpufreq_contribution(tegra, freqs->new);
+		mutex_unlock(&tegra->devfreq->lock);
+
+		/*
+		 * If CPU's frequency shouldn't be taken into account at
+		 * the moment, then there is no need to update the devfreq's
+		 * state because ISR will re-check CPU's frequency on the
+		 * next interrupt.
+		 */
+		if (old == new)
+			return NOTIFY_OK;
+	}
+
 	/*
 	 * CPUFreq driver should support CPUFREQ_ASYNC_NOTIFICATION in order
 	 * to allow asynchronous notifications. This means we can't block
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 19/24] PM / devfreq: tegra30: Optimize upper consecutive watermark selection
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (17 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 18/24] PM / devfreq: tegra30: Optimize CPUFreq notifier Dmitry Osipenko
@ 2019-07-07 22:32 ` Dmitry Osipenko
  2019-07-18  9:51   ` Chanwoo Choi
  2019-07-07 22:32 ` [PATCH v4 20/24] PM / devfreq: tegra30: Optimize upper average " Dmitry Osipenko
                   ` (4 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

The memory activity counter may get a bit higher than a watermark which
is selected based on OPP that corresponds to a highest EMC rate, in this
case watermark is lower than the actual memory activity is and thus
results in unwanted "upper" interrupts.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 8d6bf6e9f1ae..c3cf87231d25 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -363,7 +363,18 @@ static void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
 	tegra_actmon_get_lower_upper(tegra, dev, freq - 1, &lower, &upper);
 
 	delta = do_percent(upper - lower, dev->config->boost_up_threshold);
-	device_writel(dev, lower + delta, ACTMON_DEV_UPPER_WMARK);
+
+	/*
+	 * The memory events count could go a bit higher than the maximum
+	 * defined by the OPPs, hence make the upper watermark infinitely
+	 * high to avoid unnecessary upper interrupts in that case.
+	 */
+	if (freq == tegra->max_freq)
+		upper = ULONG_MAX;
+	else
+		upper = lower + delta;
+
+	device_writel(dev, upper, ACTMON_DEV_UPPER_WMARK);
 
 	/*
 	 * Meanwhile the lower mark is based on the average value
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 20/24] PM / devfreq: tegra30: Optimize upper average watermark selection
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (18 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 19/24] PM / devfreq: tegra30: Optimize upper consecutive watermark selection Dmitry Osipenko
@ 2019-07-07 22:32 ` " Dmitry Osipenko
  2019-07-19  1:36   ` Chanwoo Choi
  2019-07-07 22:33 ` [PATCH v4 21/24] PM / devfreq: tegra30: Synchronize average count on target's update Dmitry Osipenko
                   ` (3 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:32 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

I noticed that CPU may be crossing the dependency threshold very
frequently for some workloads and this results in a lot of interrupts
which could be avoided if MCALL client is keeping actual EMC frequency
at a higher rate.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 23 ++++++++++++++++++-----
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index c3cf87231d25..4d582809acb6 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -314,7 +314,8 @@ static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
 }
 
 static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
-					   struct tegra_devfreq_device *dev)
+					   struct tegra_devfreq_device *dev,
+					   unsigned long freq)
 {
 	unsigned long avg_threshold, lower, upper;
 
@@ -323,6 +324,15 @@ static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
 	avg_threshold = dev->config->avg_dependency_threshold;
 	avg_threshold = avg_threshold * ACTMON_SAMPLING_PERIOD;
 
+	/*
+	 * If cumulative EMC frequency selection is higher than the
+	 * device's, then there is no need to set upper watermark to
+	 * a lower value because it will result in unnecessary upper
+	 * interrupts.
+	 */
+	if (freq * ACTMON_SAMPLING_PERIOD > upper)
+		upper = freq * ACTMON_SAMPLING_PERIOD;
+
 	/*
 	 * We want to get interrupts when MCCPU client crosses the
 	 * dependency threshold in order to take into / out of account
@@ -392,6 +402,7 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
 			      struct tegra_devfreq_device *dev)
 {
 	u32 intr_status, dev_ctrl, avg_intr_mask, avg_count;
+	unsigned long freq;
 
 	trace_device_isr_enter(tegra->regs, dev->config->offset,
 			       dev->boost_freq, cpufreq_get(0));
@@ -405,8 +416,10 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
 	avg_intr_mask = ACTMON_DEV_INTR_AVG_BELOW_WMARK |
 			ACTMON_DEV_INTR_AVG_ABOVE_WMARK;
 
-	if (intr_status & avg_intr_mask)
-		tegra_devfreq_update_avg_wmark(tegra, dev);
+	if (intr_status & avg_intr_mask) {
+		freq = clk_get_rate(tegra->emc_clock) / KHZ;
+		tegra_devfreq_update_avg_wmark(tegra, dev, freq);
+	}
 
 	if (intr_status & ACTMON_DEV_INTR_CONSECUTIVE_UPPER) {
 		/*
@@ -525,7 +538,7 @@ static int tegra_actmon_clk_notify_cb(struct notifier_block *nb,
 	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++) {
 		dev = &tegra->devices[i];
 
-		tegra_devfreq_update_avg_wmark(tegra, dev);
+		tegra_devfreq_update_avg_wmark(tegra, dev, freq);
 		tegra_devfreq_update_wmark(tegra, dev, freq);
 	}
 
@@ -630,7 +643,7 @@ static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
 	device_writel(dev, dev->avg_freq * ACTMON_SAMPLING_PERIOD,
 		      ACTMON_DEV_INIT_AVG);
 
-	tegra_devfreq_update_avg_wmark(tegra, dev);
+	tegra_devfreq_update_avg_wmark(tegra, dev, dev->avg_freq);
 	tegra_devfreq_update_wmark(tegra, dev, dev->avg_freq);
 
 	device_writel(dev, ACTMON_COUNT_WEIGHT, ACTMON_DEV_COUNT_WEIGHT);
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 21/24] PM / devfreq: tegra30: Synchronize average count on target's update
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (19 preceding siblings ...)
  2019-07-07 22:32 ` [PATCH v4 20/24] PM / devfreq: tegra30: Optimize upper average " Dmitry Osipenko
@ 2019-07-07 22:33 ` Dmitry Osipenko
  2019-07-18 10:15   ` Chanwoo Choi
  2019-07-07 22:33 ` [PATCH v4 22/24] PM / devfreq: tegra30: Include appropriate header Dmitry Osipenko
                   ` (2 subsequent siblings)
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:33 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

The average count may get out of sync if interrupt was disabled / avoided
for a long time due to upper watermark optimization, hence it should be
re-synced on each target's update to ensure that watermarks are set up
correctly on EMC rate-change notification and that a correct frequency
is selected for device.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 4d582809acb6..8a674fad26be 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -466,11 +466,41 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
 			      dev->boost_freq, cpufreq_get(0));
 }
 
+static void tegra_devfreq_sync_avg_count(struct tegra_devfreq *tegra,
+					 struct tegra_devfreq_device *dev)
+{
+	u32 avg_count, avg_freq, old_upper, new_upper;
+
+	avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
+	avg_freq = avg_count / ACTMON_SAMPLING_PERIOD;
+
+	old_upper = tegra_actmon_upper_freq(tegra, dev->avg_freq);
+	new_upper = tegra_actmon_upper_freq(tegra, avg_freq);
+
+	/* similar to ISR, see comments in actmon_isr_device() */
+	if (old_upper != new_upper) {
+		dev->avg_freq = avg_freq;
+		dev->boost_freq = 0;
+	}
+}
+
 static unsigned long actmon_update_target(struct tegra_devfreq *tegra,
 					  struct tegra_devfreq_device *dev)
 {
 	unsigned long target_freq;
 
+	/*
+	 * The avg_count / avg_freq is getting snapshoted on device's
+	 * interrupt, but there are cases where actual value need to
+	 * be utilized on target's update, like CPUFreq boosting and
+	 * overriding the min freq via /sys/class/devfreq/devfreq0/min_freq
+	 * because we're optimizing the upper watermark based on the
+	 * actual EMC frequency. This means that interrupt may be
+	 * inactive for a long time and thus making snapshoted value
+	 * outdated.
+	 */
+	tegra_devfreq_sync_avg_count(tegra, dev);
+
 	target_freq = min(dev->avg_freq + dev->boost_freq, KHZ_MAX);
 	target_freq = tegra_actmon_account_cpu_freq(tegra, dev, target_freq);
 
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 22/24] PM / devfreq: tegra30: Include appropriate header
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (20 preceding siblings ...)
  2019-07-07 22:33 ` [PATCH v4 21/24] PM / devfreq: tegra30: Synchronize average count on target's update Dmitry Osipenko
@ 2019-07-07 22:33 ` Dmitry Osipenko
  2019-07-18  9:58   ` Chanwoo Choi
  2019-07-07 22:33 ` [PATCH v4 23/24] PM / devfreq: tegra30: Increase sampling period to 16ms Dmitry Osipenko
  2019-07-07 22:33 ` [PATCH v4 24/24] PM / devfreq: tegra20/30: Add Dmitry as a maintainer Dmitry Osipenko
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:33 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

It's not very correct to include mod_devicetable.h for the OF device
drivers and of_device.h should be included instead.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 8a674fad26be..19e872a64148 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -13,7 +13,7 @@
 #include <linux/io.h>
 #include <linux/irq.h>
 #include <linux/module.h>
-#include <linux/mod_devicetable.h>
+#include <linux/of_device.h>
 #include <linux/platform_device.h>
 #include <linux/pm_opp.h>
 #include <linux/reset.h>
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 23/24] PM / devfreq: tegra30: Increase sampling period to 16ms
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (21 preceding siblings ...)
  2019-07-07 22:33 ` [PATCH v4 22/24] PM / devfreq: tegra30: Include appropriate header Dmitry Osipenko
@ 2019-07-07 22:33 ` Dmitry Osipenko
  2019-07-18 10:00   ` Chanwoo Choi
  2019-07-07 22:33 ` [PATCH v4 24/24] PM / devfreq: tegra20/30: Add Dmitry as a maintainer Dmitry Osipenko
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:33 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Increase sampling period by 4ms to get a nicer pow2 value, converting
diving into shifts in the code. That's more preferable for Tegra30 that
doesn't have hardware divider instructions because of older Cortex-A9 CPU.
In a result boosting events are delayed by 4ms, which is not sensible in
practice at all.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 drivers/devfreq/tegra30-devfreq.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
index 19e872a64148..bde19b758643 100644
--- a/drivers/devfreq/tegra30-devfreq.c
+++ b/drivers/devfreq/tegra30-devfreq.c
@@ -71,7 +71,7 @@
  * translates to 2 ^ (K_VAL + 1). ex: 2 ^ (6 + 1) = 128
  */
 #define ACTMON_AVERAGE_WINDOW_LOG2				6
-#define ACTMON_SAMPLING_PERIOD					12 /* ms */
+#define ACTMON_SAMPLING_PERIOD					16 /* ms */
 
 #define KHZ							1000
 
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* [PATCH v4 24/24] PM / devfreq: tegra20/30: Add Dmitry as a maintainer
  2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
                   ` (22 preceding siblings ...)
  2019-07-07 22:33 ` [PATCH v4 23/24] PM / devfreq: tegra30: Increase sampling period to 16ms Dmitry Osipenko
@ 2019-07-07 22:33 ` Dmitry Osipenko
  2019-07-18  9:56   ` Chanwoo Choi
  23 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-07 22:33 UTC (permalink / raw)
  To: Thierry Reding, MyungJoo Ham, Kyungmin Park, Chanwoo Choi,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

I was contributing to the NVIDIA Tegra20+ devfreq drivers recently and
want to help keep them working and evolving in the future.

Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
---
 MAINTAINERS | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 95d4bd85df44..4e47ce737376 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -10356,6 +10356,15 @@ F:	include/linux/memblock.h
 F:	mm/memblock.c
 F:	Documentation/core-api/boot-time-mm.rst
 
+MEMORY FREQUENCY SCALING DRIVERS FOR NVIDIA TEGRA
+M:	Dmitry Osipenko <digetx@gmail.com>
+L:	linux-pm@vger.kernel.org
+L:	linux-tegra@vger.kernel.org
+T:	git git://git.kernel.org/pub/scm/linux/kernel/git/mzx/devfreq.git
+S:	Maintained
+F:	drivers/devfreq/tegra20-devfreq.c
+F:	drivers/devfreq/tegra30-devfreq.c
+
 MEMORY MANAGEMENT
 L:	linux-mm@kvack.org
 W:	http://www.linux-mm.org
-- 
2.22.0


^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 01/24] PM / devfreq: tegra30: Change irq type to unsigned int
  2019-07-07 22:32 ` [PATCH v4 01/24] PM / devfreq: tegra30: Change irq type to unsigned int Dmitry Osipenko
@ 2019-07-16 11:35   ` Chanwoo Choi
  0 siblings, 0 replies; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-16 11:35 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Hi Dmitry,

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> IRQ numbers are always positive, hence the corresponding variable should
> be unsigned to keep types consistent. This is a minor change that cleans
> up code a tad more.
> 
> Suggested-by: Thierry Reding <thierry.reding@gmail.com>
> Acked-by: MyungJoo Ham <myungjoo.ham@samsung.com>
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index a6ba75f4106d..a27300f40b0b 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -160,7 +160,7 @@ struct tegra_devfreq {
>  
>  	struct tegra_devfreq_device devices[ARRAY_SIZE(actmon_device_configs)];
>  
> -	int irq;
> +	unsigned int		irq;
>  };
>  
>  struct tegra_actmon_emc_ratio {
> @@ -618,12 +618,12 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>  		return PTR_ERR(tegra->emc_clock);
>  	}
>  
> -	tegra->irq = platform_get_irq(pdev, 0);
> -	if (tegra->irq < 0) {
> -		err = tegra->irq;
> +	err = platform_get_irq(pdev, 0);
> +	if (err < 0) {
>  		dev_err(&pdev->dev, "Failed to get IRQ: %d\n", err);
>  		return err;
>  	}
> +	tegra->irq = err;
>  
>  	reset_control_assert(tegra->reset);
>  
> 

Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>

-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 02/24] PM / devfreq: tegra30: Keep interrupt disabled while governor is stopped
  2019-07-07 22:32 ` [PATCH v4 02/24] PM / devfreq: tegra30: Keep interrupt disabled while governor is stopped Dmitry Osipenko
@ 2019-07-16 11:47   ` Chanwoo Choi
  2019-07-16 13:03     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-16 11:47 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> There is no real need to keep interrupt always-enabled, will be nicer
> to keep it disabled while governor is inactive.
> 
> Suggested-by: Thierry Reding <thierry.reding@gmail.com>
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 43 ++++++++++++++++---------------
>  1 file changed, 22 insertions(+), 21 deletions(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index a27300f40b0b..5e2b133babdd 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -11,6 +11,7 @@
>  #include <linux/devfreq.h>
>  #include <linux/interrupt.h>
>  #include <linux/io.h>
> +#include <linux/irq.h>
>  #include <linux/module.h>
>  #include <linux/mod_devicetable.h>
>  #include <linux/platform_device.h>
> @@ -416,8 +417,6 @@ static void tegra_actmon_start(struct tegra_devfreq *tegra)
>  {
>  	unsigned int i;
>  
> -	disable_irq(tegra->irq);
> -
>  	actmon_writel(tegra, ACTMON_SAMPLING_PERIOD - 1,
>  		      ACTMON_GLB_PERIOD_CTRL);
>  
> @@ -442,8 +441,6 @@ static void tegra_actmon_stop(struct tegra_devfreq *tegra)
>  	}
>  
>  	actmon_write_barrier(tegra);
> -
> -	enable_irq(tegra->irq);
>  }
>  
>  static int tegra_devfreq_target(struct device *dev, unsigned long *freq,
> @@ -552,6 +549,12 @@ static int tegra_governor_event_handler(struct devfreq *devfreq,
>  {
>  	struct tegra_devfreq *tegra = dev_get_drvdata(devfreq->dev.parent);
>  
> +	/*
> +	 * Couple device with the governor early as it is needed at
> +	 * the moment of governor's start (used by ISR).
> +	 */
> +	tegra->devfreq = devfreq;

I'm not sure it is necessary. Almost devfreq device get
the devfreq instance on probe timing through devfreq_add_device directly.


> +
>  	switch (event) {
>  	case DEVFREQ_GOV_START:
>  		devfreq_monitor_start(devfreq);
> @@ -586,10 +589,11 @@ static struct devfreq_governor tegra_devfreq_governor = {
>  
>  static int tegra_devfreq_probe(struct platform_device *pdev)
>  {
> -	struct tegra_devfreq *tegra;
>  	struct tegra_devfreq_device *dev;
> -	unsigned int i;
> +	struct tegra_devfreq *tegra;
> +	struct devfreq *devfreq;
>  	unsigned long rate;
> +	unsigned int i;
>  	int err;
>  
>  	tegra = devm_kzalloc(&pdev->dev, sizeof(*tegra), GFP_KERNEL);
> @@ -625,6 +629,16 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>  	}
>  	tegra->irq = err;
>  
> +	irq_set_status_flags(tegra->irq, IRQ_NOAUTOEN);
> +
> +	err = devm_request_threaded_irq(&pdev->dev, tegra->irq, NULL,
> +					actmon_thread_isr, IRQF_ONESHOT,
> +					"tegra-devfreq", tegra);
> +	if (err) {
> +		dev_err(&pdev->dev, "Interrupt request failed: %d\n", err);
> +		return err;
> +	}
> +
>  	reset_control_assert(tegra->reset);
>  
>  	err = clk_prepare_enable(tegra->clock);
> @@ -672,28 +686,15 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>  	}
>  
>  	tegra_devfreq_profile.initial_freq = clk_get_rate(tegra->emc_clock);
> -	tegra->devfreq = devfreq_add_device(&pdev->dev,
> -					    &tegra_devfreq_profile,
> -					    "tegra_actmon",
> -					    NULL);
> +	devfreq = devfreq_add_device(&pdev->dev, &tegra_devfreq_profile,
> +				     "tegra_actmon", NULL);
>  	if (IS_ERR(tegra->devfreq)) {

Have to check 'devfreq' instead of 'tegra->devfreq'.
Did you test it? It might be failed because 'tegra->devfreq is NULL.

>  		err = PTR_ERR(tegra->devfreq);

ditto.

>  		goto remove_governor;
>  	}
>  
> -	err = devm_request_threaded_irq(&pdev->dev, tegra->irq, NULL,
> -					actmon_thread_isr, IRQF_ONESHOT,
> -					"tegra-devfreq", tegra);
> -	if (err) {
> -		dev_err(&pdev->dev, "Interrupt request failed: %d\n", err);
> -		goto remove_devfreq;
> -	}
> -
>  	return 0;
>  
> -remove_devfreq:
> -	devfreq_remove_device(tegra->devfreq);
> -
>  remove_governor:
>  	devfreq_remove_governor(&tegra_devfreq_governor);
>  
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 03/24] PM / devfreq: tegra30: Handle possible round-rate error
  2019-07-07 22:32 ` [PATCH v4 03/24] PM / devfreq: tegra30: Handle possible round-rate error Dmitry Osipenko
@ 2019-07-16 11:50   ` Chanwoo Choi
  2019-07-16 13:09     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-16 11:50 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> The EMC clock rate rounding technically could fail, hence let's handle
> the error cases properly.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 17 +++++++++++++++--
>  1 file changed, 15 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 5e2b133babdd..5e606ae3f238 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -592,8 +592,8 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>  	struct tegra_devfreq_device *dev;
>  	struct tegra_devfreq *tegra;
>  	struct devfreq *devfreq;
> -	unsigned long rate;
>  	unsigned int i;
> +	long rate;
>  	int err;
>  
>  	tegra = devm_kzalloc(&pdev->dev, sizeof(*tegra), GFP_KERNEL);
> @@ -650,8 +650,14 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>  
>  	reset_control_deassert(tegra->reset);
>  
> -	tegra->max_freq = clk_round_rate(tegra->emc_clock, ULONG_MAX) / KHZ;
> +	rate = clk_round_rate(tegra->emc_clock, ULONG_MAX);
> +	if (rate < 0) {
> +		dev_err(&pdev->dev, "Failed to round clock rate: %ld\n", rate);
> +		return rate;
> +	}
> +
>  	tegra->cur_freq = clk_get_rate(tegra->emc_clock) / KHZ;
> +	tegra->max_freq = rate / KHZ;
>  
>  	for (i = 0; i < ARRAY_SIZE(actmon_device_configs); i++) {
>  		dev = tegra->devices + i;
> @@ -662,6 +668,13 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>  	for (rate = 0; rate <= tegra->max_freq * KHZ; rate++) {
>  		rate = clk_round_rate(tegra->emc_clock, rate);
>  

Please remove unneeded blank line.

> +		if (rate < 0) {
> +			dev_err(&pdev->dev,
> +				"Failed to round clock rate: %ld\n", rate);
> +			err = rate;
> +			goto remove_opps;
> +		}

Also, this patch doesn't contain code which restore the previous
tegra->cur_freq/max_freq when error happen.

> +
>  		err = dev_pm_opp_add(&pdev->dev, rate, 0);
>  		if (err) {
>  			dev_err(&pdev->dev, "Failed to add OPP: %d\n", err);
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 04/24] PM / devfreq: tegra30: Drop write-barrier
  2019-07-07 22:32 ` [PATCH v4 04/24] PM / devfreq: tegra30: Drop write-barrier Dmitry Osipenko
@ 2019-07-16 11:51   ` Chanwoo Choi
  0 siblings, 0 replies; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-16 11:51 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Hi Dmitry,

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> There is no need in a write-barrier now, given that interrupt masking is
> handled by CPU's GIC now. Hence we know exactly that interrupt won't fire
> after stopping the devfreq's governor. In other cases we don't care about
> potential buffering of the writes to hardware and thus there is no need to
> stall CPU.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 14 --------------
>  1 file changed, 14 deletions(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 5e606ae3f238..4be7858c33bc 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -230,12 +230,6 @@ static void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
>  		      ACTMON_DEV_LOWER_WMARK);
>  }
>  
> -static void actmon_write_barrier(struct tegra_devfreq *tegra)
> -{
> -	/* ensure the update has reached the ACTMON */
> -	readl(tegra->regs + ACTMON_GLB_STATUS);
> -}
> -
>  static void actmon_isr_device(struct tegra_devfreq *tegra,
>  			      struct tegra_devfreq_device *dev)
>  {
> @@ -287,8 +281,6 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
>  	device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
>  
>  	device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
> -
> -	actmon_write_barrier(tegra);
>  }
>  
>  static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq *tegra,
> @@ -376,8 +368,6 @@ static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
>  		tegra_devfreq_update_wmark(tegra, dev);
>  	}
>  
> -	actmon_write_barrier(tegra);
> -
>  	return NOTIFY_OK;
>  }
>  
> @@ -423,8 +413,6 @@ static void tegra_actmon_start(struct tegra_devfreq *tegra)
>  	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++)
>  		tegra_actmon_configure_device(tegra, &tegra->devices[i]);
>  
> -	actmon_write_barrier(tegra);
> -
>  	enable_irq(tegra->irq);
>  }
>  
> @@ -439,8 +427,6 @@ static void tegra_actmon_stop(struct tegra_devfreq *tegra)
>  		device_writel(&tegra->devices[i], ACTMON_INTR_STATUS_CLEAR,
>  			      ACTMON_DEV_INTR_STATUS);
>  	}
> -
> -	actmon_write_barrier(tegra);
>  }
>  
>  static int tegra_devfreq_target(struct device *dev, unsigned long *freq,
> 

Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>

-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 06/24] PM / devfreq: tegra30: Tune up boosting thresholds
  2019-07-07 22:32 ` [PATCH v4 06/24] PM / devfreq: tegra30: Tune up boosting thresholds Dmitry Osipenko
@ 2019-07-16 11:55   ` Chanwoo Choi
  0 siblings, 0 replies; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-16 11:55 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> Now that average-sustain coefficient / multiplier is gone, it won't hurt
> to re-tune the boosting thresholds to get a bit harder boosting for MCALL
> clients, resulting in a more reactive governing in a case of multimedia
> applications usage like 3d / video.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 16f7e6cf3b99..2bf65409ddd8 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -111,8 +111,8 @@ static struct tegra_devfreq_device_config actmon_device_configs[] = {
>  		.irq_mask = 1 << 26,
>  		.boost_up_coeff = 200,
>  		.boost_down_coeff = 50,
> -		.boost_up_threshold = 60,
> -		.boost_down_threshold = 40,
> +		.boost_up_threshold = 50,
> +		.boost_down_threshold = 25,
>  	},
>  	{
>  		/* MCCPU: memory accesses from the CPUs */
> 

It just adjusts the tunable point.
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>

-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 07/24] PM / devfreq: tegra30: Use CPUFreq notifier
  2019-07-07 22:32 ` [PATCH v4 07/24] PM / devfreq: tegra30: Use CPUFreq notifier Dmitry Osipenko
@ 2019-07-16 12:08   ` Chanwoo Choi
  2019-07-16 13:18     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-16 12:08 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> The CPU's client need to take into account that CPUFreq may change
> while memory activity not, staying high. Thus an appropriate frequency
> notifier should be used in addition to the clk-notifier.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 105 +++++++++++++++++++++++++-----
>  1 file changed, 88 insertions(+), 17 deletions(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 2bf65409ddd8..48a799fa5f63 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -17,6 +17,7 @@
>  #include <linux/platform_device.h>
>  #include <linux/pm_opp.h>
>  #include <linux/reset.h>
> +#include <linux/workqueue.h>
>  
>  #include "governor.h"
>  
> @@ -154,7 +155,10 @@ struct tegra_devfreq {
>  
>  	struct clk		*emc_clock;
>  	unsigned long		max_freq;
> -	struct notifier_block	rate_change_nb;
> +	struct notifier_block	clk_rate_change_nb;
> +
> +	struct work_struct	update_work;

nitpick.
I think 'update_work' is not clear to indicate the work
for changing the clock according to the cpu clock change.
You better to change the name for more clearly.

> +	struct notifier_block	cpu_rate_change_nb;
>  
>  	struct tegra_devfreq_device devices[ARRAY_SIZE(actmon_device_configs)];
>  
> @@ -456,8 +460,8 @@ static irqreturn_t actmon_thread_isr(int irq, void *data)
>  	return handled ? IRQ_HANDLED : IRQ_NONE;
>  }
>  
> -static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
> -				       unsigned long action, void *ptr)
> +static int tegra_actmon_clk_notify_cb(struct notifier_block *nb,
> +				      unsigned long action, void *ptr)
>  {
>  	struct clk_notifier_data *data = ptr;
>  	struct tegra_devfreq_device *dev;
> @@ -467,7 +471,7 @@ static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
>  	if (action != POST_RATE_CHANGE)
>  		return NOTIFY_OK;
>  
> -	tegra = container_of(nb, struct tegra_devfreq, rate_change_nb);
> +	tegra = container_of(nb, struct tegra_devfreq, clk_rate_change_nb);
>  
>  	/*
>  	 * EMC rate could change due to three reasons:
> @@ -496,6 +500,37 @@ static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
>  	return NOTIFY_OK;
>  }
>  
> +static void tegra_actmon_delayed_update(struct work_struct *work)
> +{
> +	struct tegra_devfreq *tegra = container_of(work, struct tegra_devfreq,
> +						   update_work);
> +
> +	mutex_lock(&tegra->devfreq->lock);
> +	update_devfreq(tegra->devfreq);
> +	mutex_unlock(&tegra->devfreq->lock);
> +}
> +
> +static int tegra_actmon_cpu_notify_cb(struct notifier_block *nb,
> +				      unsigned long action, void *ptr)
> +{
> +	struct tegra_devfreq *tegra;
> +
> +	if (action != CPUFREQ_POSTCHANGE)
> +		return NOTIFY_OK;
> +
> +	tegra = container_of(nb, struct tegra_devfreq, cpu_rate_change_nb);

nitpick. Better to check whether 'tegra' is NULL or not.

> +
> +	/*
> +	 * CPUFreq driver should support CPUFREQ_ASYNC_NOTIFICATION in order
> +	 * to allow asynchronous notifications. This means we can't block
> +	 * here for too long, otherwise CPUFreq's core will complain with a
> +	 * warning splat.
> +	 */
> +	schedule_work(&tegra->update_work);
> +
> +	return NOTIFY_OK;
> +}
> +
>  static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
>  					  struct tegra_devfreq_device *dev)
>  {
> @@ -527,9 +562,16 @@ static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
>  	device_writel(dev, val, ACTMON_DEV_CTRL);
>  }
>  
> -static void tegra_actmon_start(struct tegra_devfreq *tegra)
> +static void tegra_actmon_stop_device(struct tegra_devfreq_device *dev)
> +{
> +	device_writel(dev, 0x00000000, ACTMON_DEV_CTRL);

Better to define the constant definition of 0x00000000
in order to explain the correct meaning.

For example,
#define ACTMON_DEV_RESET	0x00000000

> +	device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
> +}
> +
> +static int tegra_actmon_start(struct tegra_devfreq *tegra)
>  {
>  	unsigned int i;
> +	int err;
>  
>  	actmon_writel(tegra, ACTMON_SAMPLING_PERIOD - 1,
>  		      ACTMON_GLB_PERIOD_CTRL);
> @@ -537,7 +579,30 @@ static void tegra_actmon_start(struct tegra_devfreq *tegra)
>  	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++)
>  		tegra_actmon_configure_device(tegra, &tegra->devices[i]);
>  
> +	/*
> +	 * We are estimating CPU's memory bandwidth requirement based on
> +	 * amount of memory accesses and system's load, judging by CPU's
> +	 * frequency. We also don't want to receive events about CPU's
> +	 * frequency transaction when governor is stopped, hence notifier
> +	 * is registered dynamically.
> +	 */
> +	err = cpufreq_register_notifier(&tegra->cpu_rate_change_nb,
> +					CPUFREQ_TRANSITION_NOTIFIER);
> +	if (err) {
> +		dev_err(tegra->devfreq->dev.parent,
> +			"Failed to register rate change notifier: %d\n", err);
> +		goto err_stop;
> +	}> +
>  	enable_irq(tegra->irq);
> +
> +	return 0;
> +
> +err_stop:
> +	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++)
> +		tegra_actmon_stop_device(&tegra->devices[i]);
> +
> +	return err;
>  }
>  
>  static void tegra_actmon_stop(struct tegra_devfreq *tegra)
> @@ -546,11 +611,13 @@ static void tegra_actmon_stop(struct tegra_devfreq *tegra)
>  
>  	disable_irq(tegra->irq);
>  
> -	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++) {
> -		device_writel(&tegra->devices[i], 0x00000000, ACTMON_DEV_CTRL);
> -		device_writel(&tegra->devices[i], ACTMON_INTR_STATUS_CLEAR,
> -			      ACTMON_DEV_INTR_STATUS);
> -	}
> +	cpufreq_unregister_notifier(&tegra->cpu_rate_change_nb,
> +				    CPUFREQ_TRANSITION_NOTIFIER);
> +
> +	cancel_work_sync(&tegra->update_work);
> +
> +	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++)
> +		tegra_actmon_stop_device(&tegra->devices[i]);
>  }
>  
>  static int tegra_devfreq_target(struct device *dev, unsigned long *freq,
> @@ -659,6 +726,7 @@ static int tegra_governor_event_handler(struct devfreq *devfreq,
>  					unsigned int event, void *data)
>  {
>  	struct tegra_devfreq *tegra = dev_get_drvdata(devfreq->dev.parent);
> +	int ret = 0;
>  
>  	/*
>  	 * Couple device with the governor early as it is needed at
> @@ -669,7 +737,7 @@ static int tegra_governor_event_handler(struct devfreq *devfreq,
>  	switch (event) {
>  	case DEVFREQ_GOV_START:
>  		devfreq_monitor_start(devfreq);
> -		tegra_actmon_start(tegra);
> +		ret = tegra_actmon_start(tegra);
>  		break;
>  
>  	case DEVFREQ_GOV_STOP:
> @@ -684,11 +752,11 @@ static int tegra_governor_event_handler(struct devfreq *devfreq,
>  
>  	case DEVFREQ_GOV_RESUME:
>  		devfreq_monitor_resume(devfreq);
> -		tegra_actmon_start(tegra);
> +		ret = tegra_actmon_start(tegra);
>  		break;
>  	}
>  
> -	return 0;
> +	return ret;
>  }
>  
>  static struct devfreq_governor tegra_devfreq_governor = {
> @@ -794,9 +862,12 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>  
>  	platform_set_drvdata(pdev, tegra);
>  
> -	tegra->	.notifier_call = tegra_actmon_rate_notify_cb;
> +	tegra->cpu_rate_change_nb.notifier_call = tegra_actmon_cpu_notify_cb;
> +	INIT_WORK(&tegra->update_work, tegra_actmon_delayed_update);
> +
> +	tegra->clk_rate_change_nb.notifier_call = tegra_actmon_clk_notify_cb;
>  	err = clk_notifier_register(tegra->emc_clock,
> -				    &tegra->rate_change_nb);
> +				    &tegra->clk_rate_change_nb);
>  	if (err) {
>  		dev_err(&pdev->dev,
>  			"Failed to register rate change notifier\n");
> @@ -823,7 +894,7 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>  	devfreq_remove_governor(&tegra_devfreq_governor);
>  
>  unreg_notifier:
> -	clk_notifier_unregister(tegra->emc_clock, &tegra->rate_change_nb);
> +	clk_notifier_unregister(tegra->emc_clock, &tegra->clk_rate_change_nb);
>  
>  remove_opps:
>  	dev_pm_opp_remove_all_dynamic(&pdev->dev);
> @@ -841,7 +912,7 @@ static int tegra_devfreq_remove(struct platform_device *pdev)
>  	devfreq_remove_device(tegra->devfreq);
>  	devfreq_remove_governor(&tegra_devfreq_governor);
>  
> -	clk_notifier_unregister(tegra->emc_clock, &tegra->rate_change_nb);
> +	clk_notifier_unregister(tegra->emc_clock, &tegra->clk_rate_change_nb);
>  	dev_pm_opp_remove_all_dynamic(&pdev->dev);
>  
>  	reset_control_reset(tegra->reset);
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 08/24] PM / devfreq: tegra30: Move clk-notifier's registration to governor's start
  2019-07-07 22:32 ` [PATCH v4 08/24] PM / devfreq: tegra30: Move clk-notifier's registration to governor's start Dmitry Osipenko
@ 2019-07-16 12:11   ` Chanwoo Choi
  0 siblings, 0 replies; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-16 12:11 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Hi Dmitiry,

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> There is no point in receiving of the notifications while governor is
> stopped, let's keep them disabled like we do for the CPU freq-change
> notifications. This also fixes a potential use-after-free bug if
> notification happens after device's removal.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 33 ++++++++++++++++++-------------
>  1 file changed, 19 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 48a799fa5f63..d5d04c25023b 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -576,6 +576,19 @@ static int tegra_actmon_start(struct tegra_devfreq *tegra)
>  	actmon_writel(tegra, ACTMON_SAMPLING_PERIOD - 1,
>  		      ACTMON_GLB_PERIOD_CTRL);
>  
> +	/*
> +	 * CLK notifications are needed in order to reconfigure the upper
> +	 * consecutive watermark in accordance to the actual clock rate
> +	 * to avoid unnecessary upper interrupts.
> +	 */
> +	err = clk_notifier_register(tegra->emc_clock,
> +				    &tegra->clk_rate_change_nb);
> +	if (err) {
> +		dev_err(tegra->devfreq->dev.parent,
> +			"Failed to register rate change notifier\n");
> +		return err;
> +	}
> +
>  	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++)
>  		tegra_actmon_configure_device(tegra, &tegra->devices[i]);
>  
> @@ -602,6 +615,8 @@ static int tegra_actmon_start(struct tegra_devfreq *tegra)
>  	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++)
>  		tegra_actmon_stop_device(&tegra->devices[i]);
>  
> +	clk_notifier_unregister(tegra->emc_clock, &tegra->clk_rate_change_nb);
> +
>  	return err;
>  }
>  
> @@ -618,6 +633,8 @@ static void tegra_actmon_stop(struct tegra_devfreq *tegra)
>  
>  	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++)
>  		tegra_actmon_stop_device(&tegra->devices[i]);
> +
> +	clk_notifier_unregister(tegra->emc_clock, &tegra->clk_rate_change_nb);
>  }
>  
>  static int tegra_devfreq_target(struct device *dev, unsigned long *freq,
> @@ -862,22 +879,14 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>  
>  	platform_set_drvdata(pdev, tegra);
>  
> +	tegra->clk_rate_change_nb.notifier_call = tegra_actmon_clk_notify_cb;
>  	tegra->cpu_rate_change_nb.notifier_call = tegra_actmon_cpu_notify_cb;
>  	INIT_WORK(&tegra->update_work, tegra_actmon_delayed_update);
>  
> -	tegra->clk_rate_change_nb.notifier_call = tegra_actmon_clk_notify_cb;
> -	err = clk_notifier_register(tegra->emc_clock,
> -				    &tegra->clk_rate_change_nb);
> -	if (err) {
> -		dev_err(&pdev->dev,
> -			"Failed to register rate change notifier\n");
> -		goto remove_opps;
> -	}
> -
>  	err = devfreq_add_governor(&tegra_devfreq_governor);
>  	if (err) {
>  		dev_err(&pdev->dev, "Failed to add governor: %d\n", err);
> -		goto unreg_notifier;
> +		goto remove_opps;
>  	}
>  
>  	tegra_devfreq_profile.initial_freq = clk_get_rate(tegra->emc_clock);
> @@ -893,9 +902,6 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>  remove_governor:
>  	devfreq_remove_governor(&tegra_devfreq_governor);
>  
> -unreg_notifier:
> -	clk_notifier_unregister(tegra->emc_clock, &tegra->clk_rate_change_nb);
> -
>  remove_opps:
>  	dev_pm_opp_remove_all_dynamic(&pdev->dev);
>  
> @@ -912,7 +918,6 @@ static int tegra_devfreq_remove(struct platform_device *pdev)
>  	devfreq_remove_device(tegra->devfreq);
>  	devfreq_remove_governor(&tegra_devfreq_governor);
>  
> -	clk_notifier_unregister(tegra->emc_clock, &tegra->clk_rate_change_nb);
>  	dev_pm_opp_remove_all_dynamic(&pdev->dev);
>  
>  	reset_control_reset(tegra->reset);
> 

Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>

-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 09/24] PM / devfreq: tegra30: Reset boosting on startup
  2019-07-07 22:32 ` [PATCH v4 09/24] PM / devfreq: tegra30: Reset boosting on startup Dmitry Osipenko
@ 2019-07-16 12:13   ` Chanwoo Choi
  2019-07-16 13:19     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-16 12:13 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> Governor could be stopped while boosting is active. We have assumption
> that everything is reset on governor's restart, including the boosting
> value, which was missed.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index d5d04c25023b..32fe95458ee7 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -536,6 +536,9 @@ static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
>  {
>  	u32 val = 0, target_freq;
>  
> +	/* we don't want boosting on restart */

nitpick.
I think that following comment is proper in my case.
In my case, I think 'we' expression is not good

	/* Reset the boost frequency on restart */

> +	dev->boost_freq = 0;
> +
>  	target_freq = clk_get_rate(tegra->emc_clock) / KHZ;
>  	dev->avg_count = target_freq * ACTMON_SAMPLING_PERIOD;
>  	device_writel(dev, dev->avg_count, ACTMON_DEV_INIT_AVG);
> 

Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 10/24] PM / devfreq: tegra30: Don't enable consecutive-down interrupt on startup
  2019-07-07 22:32 ` [PATCH v4 10/24] PM / devfreq: tegra30: Don't enable consecutive-down interrupt " Dmitry Osipenko
@ 2019-07-16 12:17   ` Chanwoo Choi
  2019-07-16 15:17     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-16 12:17 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Hi Dmitry,

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> The consecutive-down event tells that we should perform frequency
> de-boosting, but boosting is in a reset state on start and hence the
> event won't do anything useful for us and it will be just a dummy
> interrupt request.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 32fe95458ee7..878c9396bb8c 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -558,7 +558,6 @@ static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
>  		<< ACTMON_DEV_CTRL_CONSECUTIVE_ABOVE_WMARK_NUM_SHIFT;
>  	val |= ACTMON_DEV_CTRL_AVG_ABOVE_WMARK_EN;
>  	val |= ACTMON_DEV_CTRL_AVG_BELOW_WMARK_EN;
> -	val |= ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
>  	val |= ACTMON_DEV_CTRL_CONSECUTIVE_ABOVE_WMARK_EN;
>  	val |= ACTMON_DEV_CTRL_ENB;
>  
> 

Maybe, I think that better to review it by Thierry.
I'm not sure it is right or not because it depend on h/w.

-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 11/24] PM / devfreq: tegra30: Add debug messages
  2019-07-07 22:32 ` [PATCH v4 11/24] PM / devfreq: tegra30: Add debug messages Dmitry Osipenko
@ 2019-07-16 12:23   ` Chanwoo Choi
  2019-07-16 13:26     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-16 12:23 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Hi Dmitry,

Usually, the kernel log print for all users
such as changing the frequency, fail or success.

But, if the log just show the register dump,
it is not useful for all users. It is just used
for only specific developer.

I recommend that you better to add more exception handling
code on many points instead of just showing the register dump.

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> Add debug messages to know about what's happening in hardware and how
> driver reacts.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 35 +++++++++++++++++++++++++++++++
>  1 file changed, 35 insertions(+)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 878c9396bb8c..c6c4a07d3e07 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -41,6 +41,7 @@
>  #define ACTMON_DEV_AVG_UPPER_WMARK				0x10
>  #define ACTMON_DEV_AVG_LOWER_WMARK				0x14
>  #define ACTMON_DEV_COUNT_WEIGHT					0x18
> +#define ACTMON_DEV_COUNT					0x1c
>  #define ACTMON_DEV_AVG_COUNT					0x20
>  #define ACTMON_DEV_INTR_STATUS					0x24
>  
> @@ -276,6 +277,9 @@ static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
>  					 unsigned long *lower,
>  					 unsigned long *upper)
>  {
> +	struct device *ddev = tegra->devfreq->dev.parent;
> +	u32 offset = dev->config->offset;
> +
>  	/*
>  	 * Memory frequencies are guaranteed to have 1MHz granularity
>  	 * and thus we need this rounding down to get a proper watermarks
> @@ -288,6 +292,9 @@ static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
>  	*lower = tegra_actmon_lower_freq(tegra, target_freq);
>  	*upper = tegra_actmon_upper_freq(tegra, target_freq);
>  
> +	dev_dbg(ddev, "%03x: target_freq %lu lower freq %lu upper freq %lu\n",
> +		offset, target_freq, *lower, *upper);
> +
>  	*lower /= KHZ;
>  	*upper /= KHZ;
>  
> @@ -367,11 +374,31 @@ static void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
>  	device_writel(dev, lower + delta, ACTMON_DEV_LOWER_WMARK);
>  }
>  
> +static void actmon_device_debug(struct tegra_devfreq *tegra,
> +				struct tegra_devfreq_device *dev,
> +				const char *prefix)
> +{
> +	dev_dbg(tegra->devfreq->dev.parent,
> +		"%03x: %s: 0x%08x 0x%08x a %u %u %u c %u %u %u b %lu cpu %u\n",
> +		dev->config->offset, prefix,
> +		device_readl(dev, ACTMON_DEV_INTR_STATUS),
> +		device_readl(dev, ACTMON_DEV_CTRL),
> +		device_readl(dev, ACTMON_DEV_AVG_COUNT),
> +		device_readl(dev, ACTMON_DEV_AVG_LOWER_WMARK),
> +		device_readl(dev, ACTMON_DEV_AVG_UPPER_WMARK),
> +		device_readl(dev, ACTMON_DEV_COUNT),
> +		device_readl(dev, ACTMON_DEV_LOWER_WMARK),
> +		device_readl(dev, ACTMON_DEV_UPPER_WMARK),
> +		dev->boost_freq, cpufreq_get(0));
> +}
> +
>  static void actmon_isr_device(struct tegra_devfreq *tegra,
>  			      struct tegra_devfreq_device *dev)
>  {
>  	u32 intr_status, dev_ctrl, avg_intr_mask;
>  
> +	actmon_device_debug(tegra, dev, "isr+");
> +
>  	dev->avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
>  	intr_status = device_readl(dev, ACTMON_DEV_INTR_STATUS);
>  	dev_ctrl = device_readl(dev, ACTMON_DEV_CTRL);
> @@ -422,6 +449,8 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
>  
>  	device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
>  	device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
> +
> +	actmon_device_debug(tegra, dev, "isr-");
>  }
>  
>  static unsigned long actmon_update_target(struct tegra_devfreq *tegra,
> @@ -712,6 +741,7 @@ static struct devfreq_dev_profile tegra_devfreq_profile = {
>  static int tegra_governor_get_target(struct devfreq *devfreq,
>  				     unsigned long *freq)
>  {
> +	struct device *ddev = devfreq->dev.parent;
>  	struct devfreq_dev_status *stat;
>  	struct tegra_devfreq *tegra;
>  	struct tegra_devfreq_device *dev;
> @@ -734,6 +764,11 @@ static int tegra_governor_get_target(struct devfreq *devfreq,
>  		dev_target_freq = actmon_update_target(tegra, dev);
>  
>  		target_freq = max(target_freq, dev_target_freq);
> +
> +		dev_dbg(ddev, "%03x: upd: dev_target_freq %lu\n",
> +			dev->config->offset, dev_target_freq);
> +
> +		actmon_device_debug(tegra, dev, "upd");
>  	}
>  
>  	*freq = target_freq * KHZ;
> 

-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 12/24] PM / devfreq: tegra30: Inline all one-line functions
  2019-07-07 22:32 ` [PATCH v4 12/24] PM / devfreq: tegra30: Inline all one-line functions Dmitry Osipenko
@ 2019-07-16 12:26   ` Chanwoo Choi
  2019-07-16 13:35     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-16 12:26 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

Hi Dmitry,

I'm not sure that it is necessary.
As I knew, usally, the 'inline' is used on header file
to define the empty functions.

Do we have to change it with 'inline' keyword?

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> Depending on a kernel's configuration, a single line functions may not be
> inlined by compiler (like enabled ftracing for example). Let's inline such
> functions explicitly for consistency.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 13 +++++++------
>  1 file changed, 7 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index c6c4a07d3e07..1a10df5dbbed 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -181,28 +181,29 @@ static struct tegra_actmon_emc_ratio actmon_emc_ratios[] = {
>  	{  250000,    100000 },
>  };
>  
> -static u32 actmon_readl(struct tegra_devfreq *tegra, u32 offset)
> +static inline u32 actmon_readl(struct tegra_devfreq *tegra, u32 offset)
>  {
>  	return readl_relaxed(tegra->regs + offset);
>  }
>  
> -static void actmon_writel(struct tegra_devfreq *tegra, u32 val, u32 offset)
> +static inline void actmon_writel(struct tegra_devfreq *tegra,
> +				 u32 val, u32 offset)
>  {
>  	writel_relaxed(val, tegra->regs + offset);
>  }
>  
> -static u32 device_readl(struct tegra_devfreq_device *dev, u32 offset)
> +static inline u32 device_readl(struct tegra_devfreq_device *dev, u32 offset)
>  {
>  	return readl_relaxed(dev->regs + offset);
>  }
>  
> -static void device_writel(struct tegra_devfreq_device *dev, u32 val,
> -			  u32 offset)
> +static inline void device_writel(struct tegra_devfreq_device *dev,
> +				 u32 val, u32 offset)
>  {
>  	writel_relaxed(val, dev->regs + offset);
>  }
>  
> -static unsigned long do_percent(unsigned long val, unsigned int pct)
> +static inline unsigned long do_percent(unsigned long val, unsigned int pct)
>  {
>  	return val * pct / 100;
>  }
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 13/24] PM / devfreq: tegra30: Constify structs
  2019-07-07 22:32 ` [PATCH v4 13/24] PM / devfreq: tegra30: Constify structs Dmitry Osipenko
@ 2019-07-16 12:26   ` Chanwoo Choi
  0 siblings, 0 replies; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-16 12:26 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> Constify unmodifiable structs for consistency.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 1a10df5dbbed..2f59c78930bd 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -106,7 +106,7 @@ enum tegra_actmon_device {
>  	MCCPU,
>  };
>  
> -static struct tegra_devfreq_device_config actmon_device_configs[] = {
> +static const struct tegra_devfreq_device_config actmon_device_configs[] = {
>  	{
>  		/* MCALL: All memory accesses (including from the CPUs) */
>  		.offset = 0x1c0,
> @@ -171,7 +171,7 @@ struct tegra_actmon_emc_ratio {
>  	unsigned long emc_freq;
>  };
>  
> -static struct tegra_actmon_emc_ratio actmon_emc_ratios[] = {
> +static const struct tegra_actmon_emc_ratio actmon_emc_ratios[] = {
>  	{ 1400000, ULONG_MAX },
>  	{ 1200000,    750000 },
>  	{ 1100000,    600000 },
> @@ -210,7 +210,7 @@ static inline unsigned long do_percent(unsigned long val, unsigned int pct)
>  
>  static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq *tegra)
>  {
> -	struct tegra_actmon_emc_ratio *ratio = actmon_emc_ratios;
> +	const struct tegra_actmon_emc_ratio *ratio = actmon_emc_ratios;
>  	unsigned int cpu_freq = cpufreq_get(0);
>  	unsigned int i;
>  
> 

Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>

-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 14/24] PM / devfreq: tegra30: Ensure that target freq won't overflow
  2019-07-07 22:32 ` [PATCH v4 14/24] PM / devfreq: tegra30: Ensure that target freq won't overflow Dmitry Osipenko
@ 2019-07-16 12:30   ` Chanwoo Choi
  2019-07-16 13:59     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-16 12:30 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> Potentially very high boosting could cause an integer overflow for a
> highly clocked memory after conversion to MHz.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 2f59c78930bd..0de1efdaabf4 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -460,6 +460,7 @@ static unsigned long actmon_update_target(struct tegra_devfreq *tegra,
>  	unsigned long target_freq;
>  
>  	target_freq = dev->avg_count / ACTMON_SAMPLING_PERIOD + dev->boost_freq;
> +	target_freq = min(target_freq, ULONG_MAX / KHZ);

Did you meet this corner case?
If have to change it, you better to use 'tegra->max_freq' as following:
	min(target_freq, tegra->max_freq);

>  	target_freq = tegra_actmon_account_cpu_freq(tegra, dev, target_freq);
>  
>  	return target_freq;
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 15/24] PM / devfreq: tegra30: Fix integer overflow on CPU's freq max out
  2019-07-07 22:32 ` [PATCH v4 15/24] PM / devfreq: tegra30: Fix integer overflow on CPU's freq max out Dmitry Osipenko
@ 2019-07-16 12:32   ` Chanwoo Choi
  0 siblings, 0 replies; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-16 12:32 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> There is another kHz-conversion bug in the code, resulting in integer
> overflow. Although, this time the resulting value is 4294966296 and it's
> close to ULONG_MAX, which is okay in this case.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 0de1efdaabf4..60ebf9c9cd86 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -172,7 +172,7 @@ struct tegra_actmon_emc_ratio {
>  };
>  
>  static const struct tegra_actmon_emc_ratio actmon_emc_ratios[] = {
> -	{ 1400000, ULONG_MAX },
> +	{ 1400000, ULONG_MAX / KHZ },
>  	{ 1200000,    750000 },
>  	{ 1100000,    600000 },
>  	{ 1000000,    500000 },
> 

Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>

-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 02/24] PM / devfreq: tegra30: Keep interrupt disabled while governor is stopped
  2019-07-16 11:47   ` Chanwoo Choi
@ 2019-07-16 13:03     ` Dmitry Osipenko
  2019-07-17  6:37       ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-16 13:03 UTC (permalink / raw)
  To: Chanwoo Choi, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

16.07.2019 14:47, Chanwoo Choi пишет:
> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>> There is no real need to keep interrupt always-enabled, will be nicer
>> to keep it disabled while governor is inactive.
>>
>> Suggested-by: Thierry Reding <thierry.reding@gmail.com>
>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>> ---
>>  drivers/devfreq/tegra30-devfreq.c | 43 ++++++++++++++++---------------
>>  1 file changed, 22 insertions(+), 21 deletions(-)
>>
>> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
>> index a27300f40b0b..5e2b133babdd 100644
>> --- a/drivers/devfreq/tegra30-devfreq.c
>> +++ b/drivers/devfreq/tegra30-devfreq.c
>> @@ -11,6 +11,7 @@
>>  #include <linux/devfreq.h>
>>  #include <linux/interrupt.h>
>>  #include <linux/io.h>
>> +#include <linux/irq.h>
>>  #include <linux/module.h>
>>  #include <linux/mod_devicetable.h>
>>  #include <linux/platform_device.h>
>> @@ -416,8 +417,6 @@ static void tegra_actmon_start(struct tegra_devfreq *tegra)
>>  {
>>  	unsigned int i;
>>  
>> -	disable_irq(tegra->irq);
>> -
>>  	actmon_writel(tegra, ACTMON_SAMPLING_PERIOD - 1,
>>  		      ACTMON_GLB_PERIOD_CTRL);
>>  
>> @@ -442,8 +441,6 @@ static void tegra_actmon_stop(struct tegra_devfreq *tegra)
>>  	}
>>  
>>  	actmon_write_barrier(tegra);
>> -
>> -	enable_irq(tegra->irq);
>>  }
>>  
>>  static int tegra_devfreq_target(struct device *dev, unsigned long *freq,
>> @@ -552,6 +549,12 @@ static int tegra_governor_event_handler(struct devfreq *devfreq,
>>  {
>>  	struct tegra_devfreq *tegra = dev_get_drvdata(devfreq->dev.parent);
>>  
>> +	/*
>> +	 * Couple device with the governor early as it is needed at
>> +	 * the moment of governor's start (used by ISR).
>> +	 */
>> +	tegra->devfreq = devfreq;
> 
> I'm not sure it is necessary. Almost devfreq device get
> the devfreq instance on probe timing through devfreq_add_device directly.

This is necessary because this assignment is for the "governor" and not
the "device". Governor is started during of devfreq_add_device(), hence
there is no better way to assign device to the driver's governor.

>> +
>>  	switch (event) {
>>  	case DEVFREQ_GOV_START:
>>  		devfreq_monitor_start(devfreq);
>> @@ -586,10 +589,11 @@ static struct devfreq_governor tegra_devfreq_governor = {
>>  
>>  static int tegra_devfreq_probe(struct platform_device *pdev)
>>  {
>> -	struct tegra_devfreq *tegra;
>>  	struct tegra_devfreq_device *dev;
>> -	unsigned int i;
>> +	struct tegra_devfreq *tegra;
>> +	struct devfreq *devfreq;
>>  	unsigned long rate;
>> +	unsigned int i;
>>  	int err;
>>  
>>  	tegra = devm_kzalloc(&pdev->dev, sizeof(*tegra), GFP_KERNEL);
>> @@ -625,6 +629,16 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>>  	}
>>  	tegra->irq = err;
>>  
>> +	irq_set_status_flags(tegra->irq, IRQ_NOAUTOEN);
>> +
>> +	err = devm_request_threaded_irq(&pdev->dev, tegra->irq, NULL,
>> +					actmon_thread_isr, IRQF_ONESHOT,
>> +					"tegra-devfreq", tegra);
>> +	if (err) {
>> +		dev_err(&pdev->dev, "Interrupt request failed: %d\n", err);
>> +		return err;
>> +	}
>> +
>>  	reset_control_assert(tegra->reset);
>>  
>>  	err = clk_prepare_enable(tegra->clock);
>> @@ -672,28 +686,15 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>>  	}
>>  
>>  	tegra_devfreq_profile.initial_freq = clk_get_rate(tegra->emc_clock);
>> -	tegra->devfreq = devfreq_add_device(&pdev->dev,
>> -					    &tegra_devfreq_profile,
>> -					    "tegra_actmon",
>> -					    NULL);
>> +	devfreq = devfreq_add_device(&pdev->dev, &tegra_devfreq_profile,
>> +				     "tegra_actmon", NULL);
>>  	if (IS_ERR(tegra->devfreq)) {
> 
> Have to check 'devfreq' instead of 'tegra->devfreq'.
> Did you test it? It might be failed because 'tegra->devfreq is NULL.

That's a good catch! Thank you very much.

>>  		err = PTR_ERR(tegra->devfreq);
> 
> ditto.

Ok

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 03/24] PM / devfreq: tegra30: Handle possible round-rate error
  2019-07-16 11:50   ` Chanwoo Choi
@ 2019-07-16 13:09     ` Dmitry Osipenko
  2019-07-17  6:38       ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-16 13:09 UTC (permalink / raw)
  To: Chanwoo Choi, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

16.07.2019 14:50, Chanwoo Choi пишет:
> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>> The EMC clock rate rounding technically could fail, hence let's handle
>> the error cases properly.
>>
>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>> ---
>>  drivers/devfreq/tegra30-devfreq.c | 17 +++++++++++++++--
>>  1 file changed, 15 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
>> index 5e2b133babdd..5e606ae3f238 100644
>> --- a/drivers/devfreq/tegra30-devfreq.c
>> +++ b/drivers/devfreq/tegra30-devfreq.c
>> @@ -592,8 +592,8 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>>  	struct tegra_devfreq_device *dev;
>>  	struct tegra_devfreq *tegra;
>>  	struct devfreq *devfreq;
>> -	unsigned long rate;
>>  	unsigned int i;
>> +	long rate;
>>  	int err;
>>  
>>  	tegra = devm_kzalloc(&pdev->dev, sizeof(*tegra), GFP_KERNEL);
>> @@ -650,8 +650,14 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>>  
>>  	reset_control_deassert(tegra->reset);
>>  
>> -	tegra->max_freq = clk_round_rate(tegra->emc_clock, ULONG_MAX) / KHZ;
>> +	rate = clk_round_rate(tegra->emc_clock, ULONG_MAX);
>> +	if (rate < 0) {
>> +		dev_err(&pdev->dev, "Failed to round clock rate: %ld\n", rate);
>> +		return rate;
>> +	}
>> +
>>  	tegra->cur_freq = clk_get_rate(tegra->emc_clock) / KHZ;
>> +	tegra->max_freq = rate / KHZ;
>>  
>>  	for (i = 0; i < ARRAY_SIZE(actmon_device_configs); i++) {
>>  		dev = tegra->devices + i;
>> @@ -662,6 +668,13 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>>  	for (rate = 0; rate <= tegra->max_freq * KHZ; rate++) {
>>  		rate = clk_round_rate(tegra->emc_clock, rate);
>>  
> 
> Please remove unneeded blank line.

I can remove it, but it was added specifically to ease reading of the code.

>> +		if (rate < 0) {
>> +			dev_err(&pdev->dev,
>> +				"Failed to round clock rate: %ld\n", rate);
>> +			err = rate;
>> +			goto remove_opps;
>> +		}
> 
> Also, this patch doesn't contain code which restore the previous
> tegra->cur_freq/max_freq when error happen.

The error here results in abortion of the driver's probing, hence
nothing need to be restored in that case because nothing was changed at
this point yet.

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 07/24] PM / devfreq: tegra30: Use CPUFreq notifier
  2019-07-16 12:08   ` Chanwoo Choi
@ 2019-07-16 13:18     ` Dmitry Osipenko
  0 siblings, 0 replies; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-16 13:18 UTC (permalink / raw)
  To: Chanwoo Choi, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

16.07.2019 15:08, Chanwoo Choi пишет:
> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>> The CPU's client need to take into account that CPUFreq may change
>> while memory activity not, staying high. Thus an appropriate frequency
>> notifier should be used in addition to the clk-notifier.
>>
>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>> ---
>>  drivers/devfreq/tegra30-devfreq.c | 105 +++++++++++++++++++++++++-----
>>  1 file changed, 88 insertions(+), 17 deletions(-)
>>
>> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
>> index 2bf65409ddd8..48a799fa5f63 100644
>> --- a/drivers/devfreq/tegra30-devfreq.c
>> +++ b/drivers/devfreq/tegra30-devfreq.c
>> @@ -17,6 +17,7 @@
>>  #include <linux/platform_device.h>
>>  #include <linux/pm_opp.h>
>>  #include <linux/reset.h>
>> +#include <linux/workqueue.h>
>>  
>>  #include "governor.h"
>>  
>> @@ -154,7 +155,10 @@ struct tegra_devfreq {
>>  
>>  	struct clk		*emc_clock;
>>  	unsigned long		max_freq;
>> -	struct notifier_block	rate_change_nb;
>> +	struct notifier_block	clk_rate_change_nb;
>> +
>> +	struct work_struct	update_work;
> 
> nitpick.
> I think 'update_work' is not clear to indicate the work
> for changing the clock according to the cpu clock change.
> You better to change the name for more clearly.

Okay, I'll rename it to 'cpufreq_update_work' or something like that.

>> +	struct notifier_block	cpu_rate_change_nb;
>>  
>>  	struct tegra_devfreq_device devices[ARRAY_SIZE(actmon_device_configs)];
>>  
>> @@ -456,8 +460,8 @@ static irqreturn_t actmon_thread_isr(int irq, void *data)
>>  	return handled ? IRQ_HANDLED : IRQ_NONE;
>>  }
>>  
>> -static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
>> -				       unsigned long action, void *ptr)
>> +static int tegra_actmon_clk_notify_cb(struct notifier_block *nb,
>> +				      unsigned long action, void *ptr)
>>  {
>>  	struct clk_notifier_data *data = ptr;
>>  	struct tegra_devfreq_device *dev;
>> @@ -467,7 +471,7 @@ static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
>>  	if (action != POST_RATE_CHANGE)
>>  		return NOTIFY_OK;
>>  
>> -	tegra = container_of(nb, struct tegra_devfreq, rate_change_nb);
>> +	tegra = container_of(nb, struct tegra_devfreq, clk_rate_change_nb);
>>  
>>  	/*
>>  	 * EMC rate could change due to three reasons:
>> @@ -496,6 +500,37 @@ static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
>>  	return NOTIFY_OK;
>>  }
>>  
>> +static void tegra_actmon_delayed_update(struct work_struct *work)
>> +{
>> +	struct tegra_devfreq *tegra = container_of(work, struct tegra_devfreq,
>> +						   update_work);
>> +
>> +	mutex_lock(&tegra->devfreq->lock);
>> +	update_devfreq(tegra->devfreq);
>> +	mutex_unlock(&tegra->devfreq->lock);
>> +}
>> +
>> +static int tegra_actmon_cpu_notify_cb(struct notifier_block *nb,
>> +				      unsigned long action, void *ptr)
>> +{
>> +	struct tegra_devfreq *tegra;
>> +
>> +	if (action != CPUFREQ_POSTCHANGE)
>> +		return NOTIFY_OK;
>> +
>> +	tegra = container_of(nb, struct tegra_devfreq, cpu_rate_change_nb);
> 
> nitpick. Better to check whether 'tegra' is NULL or not.

It can't be NULL, unless 'nb' is NULL and even in that case
container_of() will cause kernel crash. Hence it's absolutely
unnecessary to check for NULL here.

>> +
>> +	/*
>> +	 * CPUFreq driver should support CPUFREQ_ASYNC_NOTIFICATION in order
>> +	 * to allow asynchronous notifications. This means we can't block
>> +	 * here for too long, otherwise CPUFreq's core will complain with a
>> +	 * warning splat.
>> +	 */
>> +	schedule_work(&tegra->update_work);
>> +
>> +	return NOTIFY_OK;
>> +}
>> +
>>  static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
>>  					  struct tegra_devfreq_device *dev)
>>  {
>> @@ -527,9 +562,16 @@ static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
>>  	device_writel(dev, val, ACTMON_DEV_CTRL);
>>  }
>>  
>> -static void tegra_actmon_start(struct tegra_devfreq *tegra)
>> +static void tegra_actmon_stop_device(struct tegra_devfreq_device *dev)
>> +{
>> +	device_writel(dev, 0x00000000, ACTMON_DEV_CTRL);
> 
> Better to define the constant definition of 0x00000000
> in order to explain the correct meaning.
> 
> For example,
> #define ACTMON_DEV_RESET	0x00000000

Okay.

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 09/24] PM / devfreq: tegra30: Reset boosting on startup
  2019-07-16 12:13   ` Chanwoo Choi
@ 2019-07-16 13:19     ` Dmitry Osipenko
  0 siblings, 0 replies; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-16 13:19 UTC (permalink / raw)
  To: Chanwoo Choi, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

16.07.2019 15:13, Chanwoo Choi пишет:
> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>> Governor could be stopped while boosting is active. We have assumption
>> that everything is reset on governor's restart, including the boosting
>> value, which was missed.
>>
>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>> ---
>>  drivers/devfreq/tegra30-devfreq.c | 3 +++
>>  1 file changed, 3 insertions(+)
>>
>> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
>> index d5d04c25023b..32fe95458ee7 100644
>> --- a/drivers/devfreq/tegra30-devfreq.c
>> +++ b/drivers/devfreq/tegra30-devfreq.c
>> @@ -536,6 +536,9 @@ static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
>>  {
>>  	u32 val = 0, target_freq;
>>  
>> +	/* we don't want boosting on restart */
> 
> nitpick.
> I think that following comment is proper in my case.
> In my case, I think 'we' expression is not good
> 
> 	/* Reset the boost frequency on restart */

Okay, I'll change the wording.

>> +	dev->boost_freq = 0;
>> +
>>  	target_freq = clk_get_rate(tegra->emc_clock) / KHZ;
>>  	dev->avg_count = target_freq * ACTMON_SAMPLING_PERIOD;
>>  	device_writel(dev, dev->avg_count, ACTMON_DEV_INIT_AVG);
>>
> 
> Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
> 
> 


^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 11/24] PM / devfreq: tegra30: Add debug messages
  2019-07-16 12:23   ` Chanwoo Choi
@ 2019-07-16 13:26     ` Dmitry Osipenko
  2019-07-17  6:45       ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-16 13:26 UTC (permalink / raw)
  To: Chanwoo Choi, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

16.07.2019 15:23, Chanwoo Choi пишет:
> Hi Dmitry,
> 
> Usually, the kernel log print for all users
> such as changing the frequency, fail or success.
> 
> But, if the log just show the register dump,
> it is not useful for all users. It is just used
> for only specific developer.
> 
> I recommend that you better to add more exception handling
> code on many points instead of just showing the register dump.

The debug messages are not users, but for developers. Yes, I primarily
made the debugging to be useful for myself and will be happy to change
the way debugging is done if there will be any other active developer
for this driver. The registers dump is more than enough in order to
understand what's going on, I don't see any real need to change anything
here for now.

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 12/24] PM / devfreq: tegra30: Inline all one-line functions
  2019-07-16 12:26   ` Chanwoo Choi
@ 2019-07-16 13:35     ` Dmitry Osipenko
  2019-07-18  9:09       ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-16 13:35 UTC (permalink / raw)
  To: Chanwoo Choi, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

16.07.2019 15:26, Chanwoo Choi пишет:
> Hi Dmitry,
> 
> I'm not sure that it is necessary.
> As I knew, usally, the 'inline' is used on header file
> to define the empty functions.
> 
> Do we have to change it with 'inline' keyword?

The 'inline' attribute tells compiler that instead of jumping into the
function, it should take the function's code and replace the function's
invocation with that code. This is done in order to help compiler
optimize code properly, please see [1]. There is absolutely no need to
create a function call into a function that consists of a single
instruction.

[1] https://gcc.gnu.org/onlinedocs/gcc-9.1.0/gcc/Inline.html

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 14/24] PM / devfreq: tegra30: Ensure that target freq won't overflow
  2019-07-16 12:30   ` Chanwoo Choi
@ 2019-07-16 13:59     ` Dmitry Osipenko
  0 siblings, 0 replies; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-16 13:59 UTC (permalink / raw)
  To: Chanwoo Choi, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

16.07.2019 15:30, Chanwoo Choi пишет:
> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>> Potentially very high boosting could cause an integer overflow for a
>> highly clocked memory after conversion to MHz.
>>
>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>> ---
>>  drivers/devfreq/tegra30-devfreq.c | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
>> index 2f59c78930bd..0de1efdaabf4 100644
>> --- a/drivers/devfreq/tegra30-devfreq.c
>> +++ b/drivers/devfreq/tegra30-devfreq.c
>> @@ -460,6 +460,7 @@ static unsigned long actmon_update_target(struct tegra_devfreq *tegra,
>>  	unsigned long target_freq;
>>  
>>  	target_freq = dev->avg_count / ACTMON_SAMPLING_PERIOD + dev->boost_freq;
>> +	target_freq = min(target_freq, ULONG_MAX / KHZ);
> 
> Did you meet this corner case?

I can't recall that, technically it could happen and I don't feel
comfortable by having potential integer overflows anyway.

> If have to change it, you better to use 'tegra->max_freq' as following:
> 	min(target_freq, tegra->max_freq);

The 'tegra->max_freq' will work here, but 'ULONG_MAX / KHZ' is a
constant value which is also correct and doesn't case any harm.

Probably will be even more expressive to write this as:

	target_freq = dev->avg_freq + dev->boost_freq;
	target_freq = tegra_actmon_account_cpu_freq(tegra, dev, target_freq);

	return min(target_freq, tegra->max_freq);


Thank you very much for the reviews!

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 10/24] PM / devfreq: tegra30: Don't enable consecutive-down interrupt on startup
  2019-07-16 12:17   ` Chanwoo Choi
@ 2019-07-16 15:17     ` Dmitry Osipenko
  0 siblings, 0 replies; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-16 15:17 UTC (permalink / raw)
  To: Chanwoo Choi, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

16.07.2019 15:17, Chanwoo Choi пишет:
> Hi Dmitry,
> 
> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>> The consecutive-down event tells that we should perform frequency
>> de-boosting, but boosting is in a reset state on start and hence the
>> event won't do anything useful for us and it will be just a dummy
>> interrupt request.
>>
>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>> ---
>>  drivers/devfreq/tegra30-devfreq.c | 1 -
>>  1 file changed, 1 deletion(-)
>>
>> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
>> index 32fe95458ee7..878c9396bb8c 100644
>> --- a/drivers/devfreq/tegra30-devfreq.c
>> +++ b/drivers/devfreq/tegra30-devfreq.c
>> @@ -558,7 +558,6 @@ static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
>>  		<< ACTMON_DEV_CTRL_CONSECUTIVE_ABOVE_WMARK_NUM_SHIFT;
>>  	val |= ACTMON_DEV_CTRL_AVG_ABOVE_WMARK_EN;
>>  	val |= ACTMON_DEV_CTRL_AVG_BELOW_WMARK_EN;
>> -	val |= ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
>>  	val |= ACTMON_DEV_CTRL_CONSECUTIVE_ABOVE_WMARK_EN;
>>  	val |= ACTMON_DEV_CTRL_ENB;
>>  
>>
> 
> Maybe, I think that better to review it by Thierry.
> I'm not sure it is right or not because it depend on h/w.
> 

The CONSECUTIVE_LOWER interrupts are generated if
ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN is enabled and ISR enables
the BELOW_WMARK_EN bit once CONSECUTIVE_UPPER event is received. The
CONSECUTIVE events are used in for the frequency boosting logic and
there is no boosting on start, hence the CONSECUTIVE_LOWER event isn't
needed on start as well. Hope this helps.

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 02/24] PM / devfreq: tegra30: Keep interrupt disabled while governor is stopped
  2019-07-16 13:03     ` Dmitry Osipenko
@ 2019-07-17  6:37       ` Chanwoo Choi
  2019-07-17 16:44         ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-17  6:37 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 16. 오후 10:03, Dmitry Osipenko wrote:
> 16.07.2019 14:47, Chanwoo Choi пишет:
>> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>>> There is no real need to keep interrupt always-enabled, will be nicer
>>> to keep it disabled while governor is inactive.
>>>
>>> Suggested-by: Thierry Reding <thierry.reding@gmail.com>
>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>> ---
>>>  drivers/devfreq/tegra30-devfreq.c | 43 ++++++++++++++++---------------
>>>  1 file changed, 22 insertions(+), 21 deletions(-)
>>>
>>> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
>>> index a27300f40b0b..5e2b133babdd 100644
>>> --- a/drivers/devfreq/tegra30-devfreq.c
>>> +++ b/drivers/devfreq/tegra30-devfreq.c
>>> @@ -11,6 +11,7 @@
>>>  #include <linux/devfreq.h>
>>>  #include <linux/interrupt.h>
>>>  #include <linux/io.h>
>>> +#include <linux/irq.h>
>>>  #include <linux/module.h>
>>>  #include <linux/mod_devicetable.h>
>>>  #include <linux/platform_device.h>
>>> @@ -416,8 +417,6 @@ static void tegra_actmon_start(struct tegra_devfreq *tegra)
>>>  {
>>>  	unsigned int i;
>>>  
>>> -	disable_irq(tegra->irq);
>>> -
>>>  	actmon_writel(tegra, ACTMON_SAMPLING_PERIOD - 1,
>>>  		      ACTMON_GLB_PERIOD_CTRL);
>>>  
>>> @@ -442,8 +441,6 @@ static void tegra_actmon_stop(struct tegra_devfreq *tegra)
>>>  	}
>>>  
>>>  	actmon_write_barrier(tegra);
>>> -
>>> -	enable_irq(tegra->irq);
>>>  }
>>>  
>>>  static int tegra_devfreq_target(struct device *dev, unsigned long *freq,
>>> @@ -552,6 +549,12 @@ static int tegra_governor_event_handler(struct devfreq *devfreq,
>>>  {
>>>  	struct tegra_devfreq *tegra = dev_get_drvdata(devfreq->dev.parent);
>>>  
>>> +	/*
>>> +	 * Couple device with the governor early as it is needed at
>>> +	 * the moment of governor's start (used by ISR).
>>> +	 */
>>> +	tegra->devfreq = devfreq;
>>
>> I'm not sure it is necessary. Almost devfreq device get
>> the devfreq instance on probe timing through devfreq_add_device directly.
> 
> This is necessary because this assignment is for the "governor" and not
> the "device". Governor is started during of devfreq_add_device(), hence
> there is no better way to assign device to the driver's governor.

OK. I understand.

But, I have a question. Is it working before this patch?
How can you test it on that tegra->devfreq is NULL?

> 
>>> +
>>>  	switch (event) {
>>>  	case DEVFREQ_GOV_START:
>>>  		devfreq_monitor_start(devfreq);
>>> @@ -586,10 +589,11 @@ static struct devfreq_governor tegra_devfreq_governor = {
>>>  
>>>  static int tegra_devfreq_probe(struct platform_device *pdev)
>>>  {
>>> -	struct tegra_devfreq *tegra;
>>>  	struct tegra_devfreq_device *dev;
>>> -	unsigned int i;
>>> +	struct tegra_devfreq *tegra;
>>> +	struct devfreq *devfreq;
>>>  	unsigned long rate;
>>> +	unsigned int i;
>>>  	int err;
>>>  
>>>  	tegra = devm_kzalloc(&pdev->dev, sizeof(*tegra), GFP_KERNEL);
>>> @@ -625,6 +629,16 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>>>  	}
>>>  	tegra->irq = err;
>>>  
>>> +	irq_set_status_flags(tegra->irq, IRQ_NOAUTOEN);
>>> +
>>> +	err = devm_request_threaded_irq(&pdev->dev, tegra->irq, NULL,
>>> +					actmon_thread_isr, IRQF_ONESHOT,
>>> +					"tegra-devfreq", tegra);
>>> +	if (err) {
>>> +		dev_err(&pdev->dev, "Interrupt request failed: %d\n", err);
>>> +		return err;
>>> +	}
>>> +
>>>  	reset_control_assert(tegra->reset);
>>>  
>>>  	err = clk_prepare_enable(tegra->clock);
>>> @@ -672,28 +686,15 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>>>  	}
>>>  
>>>  	tegra_devfreq_profile.initial_freq = clk_get_rate(tegra->emc_clock);
>>> -	tegra->devfreq = devfreq_add_device(&pdev->dev,
>>> -					    &tegra_devfreq_profile,
>>> -					    "tegra_actmon",
>>> -					    NULL);
>>> +	devfreq = devfreq_add_device(&pdev->dev, &tegra_devfreq_profile,
>>> +				     "tegra_actmon", NULL);
>>>  	if (IS_ERR(tegra->devfreq)) {
>>
>> Have to check 'devfreq' instead of 'tegra->devfreq'.
>> Did you test it? It might be failed because 'tegra->devfreq is NULL.
> 
> That's a good catch! Thank you very much.
> 
>>>  		err = PTR_ERR(tegra->devfreq);
>>
>> ditto.
> 
> Ok
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 03/24] PM / devfreq: tegra30: Handle possible round-rate error
  2019-07-16 13:09     ` Dmitry Osipenko
@ 2019-07-17  6:38       ` Chanwoo Choi
  0 siblings, 0 replies; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-17  6:38 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 16. 오후 10:09, Dmitry Osipenko wrote:
> 16.07.2019 14:50, Chanwoo Choi пишет:
>> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>>> The EMC clock rate rounding technically could fail, hence let's handle
>>> the error cases properly.
>>>
>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>> ---
>>>  drivers/devfreq/tegra30-devfreq.c | 17 +++++++++++++++--
>>>  1 file changed, 15 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
>>> index 5e2b133babdd..5e606ae3f238 100644
>>> --- a/drivers/devfreq/tegra30-devfreq.c
>>> +++ b/drivers/devfreq/tegra30-devfreq.c
>>> @@ -592,8 +592,8 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>>>  	struct tegra_devfreq_device *dev;
>>>  	struct tegra_devfreq *tegra;
>>>  	struct devfreq *devfreq;
>>> -	unsigned long rate;
>>>  	unsigned int i;
>>> +	long rate;
>>>  	int err;
>>>  
>>>  	tegra = devm_kzalloc(&pdev->dev, sizeof(*tegra), GFP_KERNEL);
>>> @@ -650,8 +650,14 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>>>  
>>>  	reset_control_deassert(tegra->reset);
>>>  
>>> -	tegra->max_freq = clk_round_rate(tegra->emc_clock, ULONG_MAX) / KHZ;
>>> +	rate = clk_round_rate(tegra->emc_clock, ULONG_MAX);
>>> +	if (rate < 0) {
>>> +		dev_err(&pdev->dev, "Failed to round clock rate: %ld\n", rate);
>>> +		return rate;
>>> +	}
>>> +
>>>  	tegra->cur_freq = clk_get_rate(tegra->emc_clock) / KHZ;
>>> +	tegra->max_freq = rate / KHZ;
>>>  
>>>  	for (i = 0; i < ARRAY_SIZE(actmon_device_configs); i++) {
>>>  		dev = tegra->devices + i;
>>> @@ -662,6 +668,13 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>>>  	for (rate = 0; rate <= tegra->max_freq * KHZ; rate++) {
>>>  		rate = clk_round_rate(tegra->emc_clock, rate);
>>>  
>>
>> Please remove unneeded blank line.
> 
> I can remove it, but it was added specifically to ease reading of the code.
> 
>>> +		if (rate < 0) {
>>> +			dev_err(&pdev->dev,
>>> +				"Failed to round clock rate: %ld\n", rate);
>>> +			err = rate;
>>> +			goto remove_opps;
>>> +		}
>>
>> Also, this patch doesn't contain code which restore the previous
>> tegra->cur_freq/max_freq when error happen.
> 
> The error here results in abortion of the driver's probing, hence
> nothing need to be restored in that case because nothing was changed at
> this point yet.
> 
> 

OK.


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 11/24] PM / devfreq: tegra30: Add debug messages
  2019-07-16 13:26     ` Dmitry Osipenko
@ 2019-07-17  6:45       ` Chanwoo Choi
  2019-07-17 15:46         ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-17  6:45 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 16. 오후 10:26, Dmitry Osipenko wrote:
> 16.07.2019 15:23, Chanwoo Choi пишет:
>> Hi Dmitry,
>>
>> Usually, the kernel log print for all users
>> such as changing the frequency, fail or success.
>>
>> But, if the log just show the register dump,
>> it is not useful for all users. It is just used
>> for only specific developer.
>>
>> I recommend that you better to add more exception handling
>> code on many points instead of just showing the register dump.
> 
> The debug messages are not users, but for developers. Yes, I primarily
> made the debugging to be useful for myself and will be happy to change
> the way debugging is done if there will be any other active developer
> for this driver. The registers dump is more than enough in order to
> understand what's going on, I don't see any real need to change anything
> here for now.

Basically, we have to develop code and add the log for anyone.
As you commented, even if there are no other developer, we never
guarantee this assumption forever. And also, if added debug message
for only you, you can add them when testing it temporarily.

If you want to add the just register dump log for you,
I can't agree. Once again, I hope that anyone understand
the meaning of debug message as much possible as.

-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 11/24] PM / devfreq: tegra30: Add debug messages
  2019-07-17  6:45       ` Chanwoo Choi
@ 2019-07-17 15:46         ` Dmitry Osipenko
  2019-07-18  9:07           ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-17 15:46 UTC (permalink / raw)
  To: Chanwoo Choi, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

17.07.2019 9:45, Chanwoo Choi пишет:
> On 19. 7. 16. 오후 10:26, Dmitry Osipenko wrote:
>> 16.07.2019 15:23, Chanwoo Choi пишет:
>>> Hi Dmitry,
>>>
>>> Usually, the kernel log print for all users
>>> such as changing the frequency, fail or success.
>>>
>>> But, if the log just show the register dump,
>>> it is not useful for all users. It is just used
>>> for only specific developer.
>>>
>>> I recommend that you better to add more exception handling
>>> code on many points instead of just showing the register dump.
>>
>> The debug messages are not users, but for developers. Yes, I primarily
>> made the debugging to be useful for myself and will be happy to change
>> the way debugging is done if there will be any other active developer
>> for this driver. The registers dump is more than enough in order to
>> understand what's going on, I don't see any real need to change anything
>> here for now.
> 
> Basically, we have to develop code and add the log for anyone.
> As you commented, even if there are no other developer, we never
> guarantee this assumption forever. And also, if added debug message
> for only you, you can add them when testing it temporarily.
> 
> If you want to add the just register dump log for you,
> I can't agree. Once again, I hope that anyone understand
> the meaning of debug message as much possible as.
> 

The registers dump should be good for everyone because it's a
self-explanatory information for anyone who is familiar with the
hardware. I don't think there is a need for anything else than what is
proposed in this patch, at least for now. I also simply don't see any
other better way to debug the state of this particular hardware, again
this logging is for the driver developers and not for users.

Initially, I was temporarily adding the debug messages. Now they are
pretty much mandatory for verifying that driver is working properly. And
of course the debugging messages got into the shape of this patch after
several iterations of refinements. So again, I suppose that this should
be good enough for everyone who is familiar with the hardware. And of
course I'm open to the constructive suggestions, the debugging aid is
not an ABI and could be changed/improved at any time.

You're suggesting to break down the debugging into several smaller
pieces, but I'm finding that as not a constructive suggestion because
the information about the full hardware state is actually necessary for
the productive debugging.

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 02/24] PM / devfreq: tegra30: Keep interrupt disabled while governor is stopped
  2019-07-17  6:37       ` Chanwoo Choi
@ 2019-07-17 16:44         ` Dmitry Osipenko
  0 siblings, 0 replies; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-17 16:44 UTC (permalink / raw)
  To: Chanwoo Choi, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

17.07.2019 9:37, Chanwoo Choi пишет:
> On 19. 7. 16. 오후 10:03, Dmitry Osipenko wrote:
>> 16.07.2019 14:47, Chanwoo Choi пишет:
>>> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>>>> There is no real need to keep interrupt always-enabled, will be nicer
>>>> to keep it disabled while governor is inactive.
>>>>
>>>> Suggested-by: Thierry Reding <thierry.reding@gmail.com>
>>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>>> ---
>>>>  drivers/devfreq/tegra30-devfreq.c | 43 ++++++++++++++++---------------
>>>>  1 file changed, 22 insertions(+), 21 deletions(-)
>>>>
>>>> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
>>>> index a27300f40b0b..5e2b133babdd 100644
>>>> --- a/drivers/devfreq/tegra30-devfreq.c
>>>> +++ b/drivers/devfreq/tegra30-devfreq.c
>>>> @@ -11,6 +11,7 @@
>>>>  #include <linux/devfreq.h>
>>>>  #include <linux/interrupt.h>
>>>>  #include <linux/io.h>
>>>> +#include <linux/irq.h>
>>>>  #include <linux/module.h>
>>>>  #include <linux/mod_devicetable.h>
>>>>  #include <linux/platform_device.h>
>>>> @@ -416,8 +417,6 @@ static void tegra_actmon_start(struct tegra_devfreq *tegra)
>>>>  {
>>>>  	unsigned int i;
>>>>  
>>>> -	disable_irq(tegra->irq);
>>>> -
>>>>  	actmon_writel(tegra, ACTMON_SAMPLING_PERIOD - 1,
>>>>  		      ACTMON_GLB_PERIOD_CTRL);
>>>>  
>>>> @@ -442,8 +441,6 @@ static void tegra_actmon_stop(struct tegra_devfreq *tegra)
>>>>  	}
>>>>  
>>>>  	actmon_write_barrier(tegra);
>>>> -
>>>> -	enable_irq(tegra->irq);
>>>>  }
>>>>  
>>>>  static int tegra_devfreq_target(struct device *dev, unsigned long *freq,
>>>> @@ -552,6 +549,12 @@ static int tegra_governor_event_handler(struct devfreq *devfreq,
>>>>  {
>>>>  	struct tegra_devfreq *tegra = dev_get_drvdata(devfreq->dev.parent);
>>>>  
>>>> +	/*
>>>> +	 * Couple device with the governor early as it is needed at
>>>> +	 * the moment of governor's start (used by ISR).
>>>> +	 */
>>>> +	tegra->devfreq = devfreq;
>>>
>>> I'm not sure it is necessary. Almost devfreq device get
>>> the devfreq instance on probe timing through devfreq_add_device directly.
>>
>> This is necessary because this assignment is for the "governor" and not
>> the "device". Governor is started during of devfreq_add_device(), hence
>> there is no better way to assign device to the driver's governor.
> 
> OK. I understand.
> 
> But, I have a question. Is it working before this patch?
> How can you test it on that tegra->devfreq is NULL?

It was working before this patch because previously interrupt was
requested *after* devfreq_add_device(), now the IRQ requesting happens
*before* devfreq_add_device() and enabling *during of*. If interrupt
fires before the assignment happened, then ISR gets a NULL deference and
this is easily reproducible.

Please note that 'tegra->devfreq' is used only by the ISR and onward, in
the further patches of this series the usage is extended by the cpufreq
notifier.

>>
>>>> +
>>>>  	switch (event) {
>>>>  	case DEVFREQ_GOV_START:
>>>>  		devfreq_monitor_start(devfreq);
>>>> @@ -586,10 +589,11 @@ static struct devfreq_governor tegra_devfreq_governor = {
>>>>  
>>>>  static int tegra_devfreq_probe(struct platform_device *pdev)
>>>>  {
>>>> -	struct tegra_devfreq *tegra;
>>>>  	struct tegra_devfreq_device *dev;
>>>> -	unsigned int i;
>>>> +	struct tegra_devfreq *tegra;
>>>> +	struct devfreq *devfreq;
>>>>  	unsigned long rate;
>>>> +	unsigned int i;
>>>>  	int err;
>>>>  
>>>>  	tegra = devm_kzalloc(&pdev->dev, sizeof(*tegra), GFP_KERNEL);
>>>> @@ -625,6 +629,16 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>>>>  	}
>>>>  	tegra->irq = err;
>>>>  
>>>> +	irq_set_status_flags(tegra->irq, IRQ_NOAUTOEN);
>>>> +
>>>> +	err = devm_request_threaded_irq(&pdev->dev, tegra->irq, NULL,
>>>> +					actmon_thread_isr, IRQF_ONESHOT,
>>>> +					"tegra-devfreq", tegra);
>>>> +	if (err) {
>>>> +		dev_err(&pdev->dev, "Interrupt request failed: %d\n", err);
>>>> +		return err;
>>>> +	}
>>>> +
>>>>  	reset_control_assert(tegra->reset);
>>>>  
>>>>  	err = clk_prepare_enable(tegra->clock);
>>>> @@ -672,28 +686,15 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>>>>  	}
>>>>  
>>>>  	tegra_devfreq_profile.initial_freq = clk_get_rate(tegra->emc_clock);
>>>> -	tegra->devfreq = devfreq_add_device(&pdev->dev,
>>>> -					    &tegra_devfreq_profile,
>>>> -					    "tegra_actmon",
>>>> -					    NULL);
>>>> +	devfreq = devfreq_add_device(&pdev->dev, &tegra_devfreq_profile,
>>>> +				     "tegra_actmon", NULL);
>>>>  	if (IS_ERR(tegra->devfreq)) {
>>>
>>> Have to check 'devfreq' instead of 'tegra->devfreq'.
>>> Did you test it? It might be failed because 'tegra->devfreq is NULL.
>>
>> That's a good catch! Thank you very much.
>>
>>>>  		err = PTR_ERR(tegra->devfreq);
>>>
>>> ditto.
>>
>> Ok

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 11/24] PM / devfreq: tegra30: Add debug messages
  2019-07-17 15:46         ` Dmitry Osipenko
@ 2019-07-18  9:07           ` Chanwoo Choi
  2019-07-19  1:13             ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-18  9:07 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 18. 오전 12:46, Dmitry Osipenko wrote:
> 17.07.2019 9:45, Chanwoo Choi пишет:
>> On 19. 7. 16. 오후 10:26, Dmitry Osipenko wrote:
>>> 16.07.2019 15:23, Chanwoo Choi пишет:
>>>> Hi Dmitry,
>>>>
>>>> Usually, the kernel log print for all users
>>>> such as changing the frequency, fail or success.
>>>>
>>>> But, if the log just show the register dump,
>>>> it is not useful for all users. It is just used
>>>> for only specific developer.
>>>>
>>>> I recommend that you better to add more exception handling
>>>> code on many points instead of just showing the register dump.
>>>
>>> The debug messages are not users, but for developers. Yes, I primarily
>>> made the debugging to be useful for myself and will be happy to change
>>> the way debugging is done if there will be any other active developer
>>> for this driver. The registers dump is more than enough in order to
>>> understand what's going on, I don't see any real need to change anything
>>> here for now.
>>
>> Basically, we have to develop code and add the log for anyone.
>> As you commented, even if there are no other developer, we never
>> guarantee this assumption forever. And also, if added debug message
>> for only you, you can add them when testing it temporarily.
>>
>> If you want to add the just register dump log for you,
>> I can't agree. Once again, I hope that anyone understand
>> the meaning of debug message as much possible as.
>>
> 
> The registers dump should be good for everyone because it's a
> self-explanatory information for anyone who is familiar with the
> hardware. I don't think there is a need for anything else than what is
> proposed in this patch, at least for now. I also simply don't see any
> other better way to debug the state of this particular hardware, again
> this logging is for the driver developers and not for users.
> 
> Initially, I was temporarily adding the debug messages. Now they are
> pretty much mandatory for verifying that driver is working properly. And
> of course the debugging messages got into the shape of this patch after
> several iterations of refinements. So again, I suppose that this should
> be good enough for everyone who is familiar with the hardware. And of
> course I'm open to the constructive suggestions, the debugging aid is
> not an ABI and could be changed/improved at any time.
> 
> You're suggesting to break down the debugging into several smaller
> pieces, but I'm finding that as not a constructive suggestion because
> the information about the full hardware state is actually necessary for
> the productive debugging.
> 
> 

Sorry for that as I saie, I cannot agree this patch. In my case,
I don't understand what is meaning of register dump of this patch.
I knew that just register dump are useful for real developer.

If you want to show the register dump, you better to add some feature
with debugfs for devfreq framework in order to read the register dump.
As I knew, sound framework (alsa) has the similar feature for checking
the register dump.

-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 12/24] PM / devfreq: tegra30: Inline all one-line functions
  2019-07-16 13:35     ` Dmitry Osipenko
@ 2019-07-18  9:09       ` Chanwoo Choi
  2019-07-19  1:22         ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-18  9:09 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 16. 오후 10:35, Dmitry Osipenko wrote:
> 16.07.2019 15:26, Chanwoo Choi пишет:
>> Hi Dmitry,
>>
>> I'm not sure that it is necessary.
>> As I knew, usally, the 'inline' is used on header file
>> to define the empty functions.
>>
>> Do we have to change it with 'inline' keyword?
> 
> The 'inline' attribute tells compiler that instead of jumping into the
> function, it should take the function's code and replace the function's
> invocation with that code. This is done in order to help compiler
> optimize code properly, please see [1]. There is absolutely no need to
> create a function call into a function that consists of a single
> instruction.
> 
> [1] https://gcc.gnu.org/onlinedocs/gcc-9.1.0/gcc/Inline.html
> 

If you want to add 'inline' keyword, I recommend that 
you better to remove the modified function in this patch
and then just call the 'write_relaxed or read_relaxed' function
directly. It is same result when using inline keyword.


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 17/24] PM / devfreq: tegra30: Use tracepoints for debugging
  2019-07-07 22:32 ` [PATCH v4 17/24] PM / devfreq: tegra30: Use tracepoints for debugging Dmitry Osipenko
@ 2019-07-18  9:47   ` Chanwoo Choi
  2019-07-19  0:49     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-18  9:47 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> Debug messages create too much CPU and memory activity by themselves,
> so it's difficult to debug lower rates and catch unwanted interrupts
> that happen rarely. Tracepoints are ideal in that regards because they
> do not contribute to the sampled date at all. This allowed me to catch
> few problems which are fixed by the followup patches, without tracepoints
> it would be much harder to do.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c      |  43 +++-------
>  include/trace/events/tegra30_devfreq.h | 105 +++++++++++++++++++++++++
>  2 files changed, 117 insertions(+), 31 deletions(-)
>  create mode 100644 include/trace/events/tegra30_devfreq.h

As I knew, 'include/trace/events' don't include the header file
for only one device driver. Usually, the trace event is provided
by framework instead of each devic driver.

> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 6ebf0f505767..43c9c5fbfe91 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -19,6 +19,9 @@
>  #include <linux/reset.h>
>  #include <linux/workqueue.h>
>  
> +#define CREATE_TRACE_POINTS
> +#include <trace/events/tegra30_devfreq.h>
> +
>  #include "governor.h"
>  
>  #define ACTMON_GLB_STATUS					0x0
> @@ -283,9 +286,6 @@ static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
>  					 unsigned long *lower,
>  					 unsigned long *upper)
>  {
> -	struct device *ddev = tegra->devfreq->dev.parent;
> -	u32 offset = dev->config->offset;
> -
>  	/*
>  	 * Memory frequencies are guaranteed to have 1MHz granularity
>  	 * and thus we need this rounding down to get a proper watermarks
> @@ -298,8 +298,8 @@ static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
>  	*lower = tegra_actmon_lower_freq(tegra, target_freq);
>  	*upper = tegra_actmon_upper_freq(tegra, target_freq);
>  
> -	dev_dbg(ddev, "%03x: target_freq %lu lower freq %lu upper freq %lu\n",
> -		offset, target_freq, *lower, *upper);
> +	trace_device_lower_upper(dev->config->offset, target_freq,
> +				 *lower, *upper);
>  
>  	/*
>  	 * The upper watermark should take into account CPU's frequency
> @@ -377,30 +377,13 @@ static void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
>  	device_writel(dev, lower + delta, ACTMON_DEV_LOWER_WMARK);
>  }
>  
> -static void actmon_device_debug(struct tegra_devfreq *tegra,
> -				struct tegra_devfreq_device *dev,
> -				const char *prefix)
> -{
> -	dev_dbg(tegra->devfreq->dev.parent,
> -		"%03x: %s: 0x%08x 0x%08x a %u %u %u c %u %u %u b %lu cpu %u\n",
> -		dev->config->offset, prefix,
> -		device_readl(dev, ACTMON_DEV_INTR_STATUS),
> -		device_readl(dev, ACTMON_DEV_CTRL),
> -		device_readl(dev, ACTMON_DEV_AVG_COUNT),
> -		device_readl(dev, ACTMON_DEV_AVG_LOWER_WMARK),
> -		device_readl(dev, ACTMON_DEV_AVG_UPPER_WMARK),
> -		device_readl(dev, ACTMON_DEV_COUNT),
> -		device_readl(dev, ACTMON_DEV_LOWER_WMARK),
> -		device_readl(dev, ACTMON_DEV_UPPER_WMARK),
> -		dev->boost_freq, cpufreq_get(0));
> -}
> -
>  static void actmon_isr_device(struct tegra_devfreq *tegra,
>  			      struct tegra_devfreq_device *dev)
>  {
>  	u32 intr_status, dev_ctrl, avg_intr_mask, avg_count;
>  
> -	actmon_device_debug(tegra, dev, "isr+");
> +	trace_device_isr_enter(tegra->regs, dev->config->offset,
> +			       dev->boost_freq, cpufreq_get(0));
>  
>  	intr_status = device_readl(dev, ACTMON_DEV_INTR_STATUS);
>  	avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
> @@ -455,7 +438,8 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
>  	device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
>  	device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
>  
> -	actmon_device_debug(tegra, dev, "isr-");
> +	trace_device_isr_exit(tegra->regs, dev->config->offset,
> +			      dev->boost_freq, cpufreq_get(0));
>  }
>  
>  static unsigned long actmon_update_target(struct tegra_devfreq *tegra,
> @@ -749,7 +733,6 @@ static struct devfreq_dev_profile tegra_devfreq_profile = {
>  static int tegra_governor_get_target(struct devfreq *devfreq,
>  				     unsigned long *freq)
>  {
> -	struct device *ddev = devfreq->dev.parent;
>  	struct devfreq_dev_status *stat;
>  	struct tegra_devfreq *tegra;
>  	struct tegra_devfreq_device *dev;
> @@ -770,13 +753,11 @@ static int tegra_governor_get_target(struct devfreq *devfreq,
>  		dev = &tegra->devices[i];
>  
>  		dev_target_freq = actmon_update_target(tegra, dev);
> -
>  		target_freq = max(target_freq, dev_target_freq);
>  
> -		dev_dbg(ddev, "%03x: upd: dev_target_freq %lu\n",
> -			dev->config->offset, dev_target_freq);
> -
> -		actmon_device_debug(tegra, dev, "upd");
> +		trace_device_target_freq(dev->config->offset, dev_target_freq);
> +		trace_device_target_update(tegra->regs, dev->config->offset,
> +					   dev->boost_freq, cpufreq_get(0));
>  	}
>  
>  	*freq = target_freq;
> diff --git a/include/trace/events/tegra30_devfreq.h b/include/trace/events/tegra30_devfreq.h
> new file mode 100644
> index 000000000000..8f264a489daf
> --- /dev/null
> +++ b/include/trace/events/tegra30_devfreq.h
> @@ -0,0 +1,105 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#undef TRACE_SYSTEM
> +#define TRACE_SYSTEM tegra30_devfreq
> +
> +#if !defined(_TRACE_TEGRA30_DEVFREQ_H) || defined(TRACE_HEADER_MULTI_READ)
> +#define _TRACE_TEGRA30_DEVFREQ_H
> +
> +#include <linux/io.h>
> +#include <linux/tracepoint.h>
> +#include <linux/types.h>
> +
> +DECLARE_EVENT_CLASS(device_state,
> +	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32 cpufreq),
> +	TP_ARGS(base, offset, boost, cpufreq),
> +	TP_STRUCT__entry(
> +		__field(u32, offset)
> +		__field(u32, intr_status)
> +		__field(u32, ctrl)
> +		__field(u32, avg_count)
> +		__field(u32, avg_lower)
> +		__field(u32, avg_upper)
> +		__field(u32, count)
> +		__field(u32, lower)
> +		__field(u32, upper)
> +		__field(u32, boost_freq)
> +		__field(u32, cpu_freq)
> +	),
> +	TP_fast_assign(
> +		__entry->offset		= offset;
> +		__entry->intr_status	= readl_relaxed(base + offset + 0x24);
> +		__entry->ctrl		= readl_relaxed(base + offset + 0x0);
> +		__entry->avg_count	= readl_relaxed(base + offset + 0x20);
> +		__entry->avg_lower	= readl_relaxed(base + offset + 0x14);
> +		__entry->avg_upper	= readl_relaxed(base + offset + 0x10);
> +		__entry->count		= readl_relaxed(base + offset + 0x1c);
> +		__entry->lower		= readl_relaxed(base + offset + 0x8);
> +		__entry->upper		= readl_relaxed(base + offset + 0x4);
> +		__entry->boost_freq	= boost;
> +		__entry->cpu_freq	= cpufreq;
> +	),
> +	TP_printk("%03x: intr 0x%08x ctrl 0x%08x avg %010u %010u %010u cnt %010u %010u %010u boost %010u cpu %u",
> +		__entry->offset,
> +		__entry->intr_status,
> +		__entry->ctrl,
> +		__entry->avg_count,
> +		__entry->avg_lower,
> +		__entry->avg_upper,
> +		__entry->count,
> +		__entry->lower,
> +		__entry->upper,
> +		__entry->boost_freq,
> +		__entry->cpu_freq)
> +);
> +
> +DEFINE_EVENT(device_state, device_isr_enter,
> +	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32 cpufreq),
> +	TP_ARGS(base, offset, boost, cpufreq));
> +
> +DEFINE_EVENT(device_state, device_isr_exit,
> +	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32 cpufreq),
> +	TP_ARGS(base, offset, boost, cpufreq));
> +
> +DEFINE_EVENT(device_state, device_target_update,
> +	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32 cpufreq),
> +	TP_ARGS(base, offset, boost, cpufreq));
> +
> +TRACE_EVENT(device_lower_upper,
> +	TP_PROTO(u32 offset, u32 target, u32 lower, u32 upper),
> +	TP_ARGS(offset, target, lower, upper),
> +	TP_STRUCT__entry(
> +		__field(u32, offset)
> +		__field(u32, target)
> +		__field(u32, lower)
> +		__field(u32, upper)
> +	),
> +	TP_fast_assign(
> +		__entry->offset = offset;
> +		__entry->target = target;
> +		__entry->lower = lower;
> +		__entry->upper = upper;
> +	),
> +	TP_printk("%03x: freq %010u lower freq %010u upper freq %010u",
> +		__entry->offset,
> +		__entry->target,
> +		__entry->lower,
> +		__entry->upper)
> +);
> +
> +TRACE_EVENT(device_target_freq,
> +	TP_PROTO(u32 offset, u32 target),
> +	TP_ARGS(offset, target),
> +	TP_STRUCT__entry(
> +		__field(u32, offset)
> +		__field(u32, target)
> +	),
> +	TP_fast_assign(
> +		__entry->offset = offset;
> +		__entry->target = target;
> +	),
> +	TP_printk("%03x: freq %010u", __entry->offset, __entry->target)
> +);
> +#endif /* _TRACE_TEGRA30_DEVFREQ_H */
> +
> +/* This part must be outside protection */
> +#include <trace/define_trace.h>
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 18/24] PM / devfreq: tegra30: Optimize CPUFreq notifier
  2019-07-07 22:32 ` [PATCH v4 18/24] PM / devfreq: tegra30: Optimize CPUFreq notifier Dmitry Osipenko
@ 2019-07-18  9:48   ` Chanwoo Choi
  2019-07-19  0:42     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-18  9:48 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> When CPU's memory activity is low or memory activity is high such that
> CPU's frequency contribution to the boosting is not taken into account,
> then there is no need to schedule devfreq's update. This eliminates
> unnecessary CPU activity during of idling caused by the scheduled work.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 73 +++++++++++++++++++++++++++----
>  1 file changed, 64 insertions(+), 9 deletions(-)

Patch4 add the 'cpufreq notifier' and this patch optimize the cpufreq notifier.
I think t hat you can combine two patches.

> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 43c9c5fbfe91..8d6bf6e9f1ae 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -216,10 +216,10 @@ static inline unsigned long do_percent(unsigned long val, unsigned int pct)
>  	return val * pct / 100;
>  }
>  
> -static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq *tegra)
> +static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq *tegra,
> +					    unsigned int cpu_freq)
>  {
>  	const struct tegra_actmon_emc_ratio *ratio = actmon_emc_ratios;
> -	unsigned int cpu_freq = cpufreq_get(0);
>  	unsigned int i;
>  
>  	for (i = 0; i < ARRAY_SIZE(actmon_emc_ratios); i++, ratio++) {
> @@ -239,15 +239,15 @@ tegra_actmon_account_cpu_freq(struct tegra_devfreq *tegra,
>  			      struct tegra_devfreq_device *dev,
>  			      unsigned long target_freq)
>  {
> -	unsigned long static_cpu_emc_freq;
> +	unsigned long cpu_emc_freq = 0;
>  
> -	if (dev->config->avg_dependency_threshold &&
> -	    dev->config->avg_dependency_threshold < dev->avg_freq) {
> -		static_cpu_emc_freq = actmon_cpu_to_emc_rate(tegra);
> -		target_freq = max(target_freq, static_cpu_emc_freq);
> -	}
> +	if (!dev->config->avg_dependency_threshold)
> +		return target_freq;
>  
> -	return target_freq;
> +	if (dev->avg_freq > dev->config->avg_dependency_threshold)
> +		cpu_emc_freq = actmon_cpu_to_emc_rate(tegra, cpufreq_get(0));
> +
> +	return max(target_freq, cpu_emc_freq);
>  }
>  
>  static unsigned long tegra_actmon_lower_freq(struct tegra_devfreq *tegra,
> @@ -531,16 +531,71 @@ static void tegra_actmon_delayed_update(struct work_struct *work)
>  	mutex_unlock(&tegra->devfreq->lock);
>  }
>  
> +static unsigned long
> +tegra_actmon_cpufreq_contribution(struct tegra_devfreq *tegra,
> +				  unsigned int cpu_freq)
> +{
> +	unsigned long freq, static_cpu_emc_freq;
> +
> +	/* check whether CPU's freq is taken into account at all */
> +	if (tegra->devices[MCCPU].avg_freq <=
> +	    tegra->devices[MCCPU].config->avg_dependency_threshold)
> +		return 0;
> +
> +	static_cpu_emc_freq = actmon_cpu_to_emc_rate(tegra, cpu_freq);
> +
> +	/* compare static CPU-EMC freq with MCALL */
> +	freq = tegra->devices[MCALL].avg_freq +
> +	       tegra->devices[MCALL].boost_freq;
> +
> +	freq = tegra_actmon_upper_freq(tegra, freq);
> +
> +	if (freq == tegra->max_freq || freq >= static_cpu_emc_freq)
> +		return 0;
> +
> +	/* compare static CPU-EMC freq with MCCPU */
> +	freq = tegra->devices[MCCPU].avg_freq +
> +	       tegra->devices[MCCPU].boost_freq;
> +
> +	freq = tegra_actmon_upper_freq(tegra, freq);
> +
> +	if (freq == tegra->max_freq || freq >= static_cpu_emc_freq)
> +		return 0;
> +
> +	return static_cpu_emc_freq;
> +}
> +
>  static int tegra_actmon_cpu_notify_cb(struct notifier_block *nb,
>  				      unsigned long action, void *ptr)
>  {
> +	struct cpufreq_freqs *freqs = ptr;
>  	struct tegra_devfreq *tegra;
> +	unsigned long old, new;
>  
>  	if (action != CPUFREQ_POSTCHANGE)
>  		return NOTIFY_OK;
>  
>  	tegra = container_of(nb, struct tegra_devfreq, cpu_rate_change_nb);
>  
> +	/*
> +	 * Quickly check whether CPU frequency should be taken into account
> +	 * at all, without blocking CPUFreq's core.
> +	 */
> +	if (mutex_trylock(&tegra->devfreq->lock)) {
> +		old = tegra_actmon_cpufreq_contribution(tegra, freqs->old);
> +		new = tegra_actmon_cpufreq_contribution(tegra, freqs->new);
> +		mutex_unlock(&tegra->devfreq->lock);
> +
> +		/*
> +		 * If CPU's frequency shouldn't be taken into account at
> +		 * the moment, then there is no need to update the devfreq's
> +		 * state because ISR will re-check CPU's frequency on the
> +		 * next interrupt.
> +		 */
> +		if (old == new)
> +			return NOTIFY_OK;
> +	}
> +
>  	/*
>  	 * CPUFreq driver should support CPUFREQ_ASYNC_NOTIFICATION in order
>  	 * to allow asynchronous notifications. This means we can't block
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 19/24] PM / devfreq: tegra30: Optimize upper consecutive watermark selection
  2019-07-07 22:32 ` [PATCH v4 19/24] PM / devfreq: tegra30: Optimize upper consecutive watermark selection Dmitry Osipenko
@ 2019-07-18  9:51   ` Chanwoo Choi
  2019-07-19  0:40     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-18  9:51 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> The memory activity counter may get a bit higher than a watermark which
> is selected based on OPP that corresponds to a highest EMC rate, in this
> case watermark is lower than the actual memory activity is and thus
> results in unwanted "upper" interrupts.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 13 ++++++++++++-
>  1 file changed, 12 insertions(+), 1 deletion(-)

It seems that you can combine patch19 with patch20.

> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 8d6bf6e9f1ae..c3cf87231d25 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -363,7 +363,18 @@ static void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
>  	tegra_actmon_get_lower_upper(tegra, dev, freq - 1, &lower, &upper);
>  
>  	delta = do_percent(upper - lower, dev->config->boost_up_threshold);
> -	device_writel(dev, lower + delta, ACTMON_DEV_UPPER_WMARK);
> +
> +	/*
> +	 * The memory events count could go a bit higher than the maximum
> +	 * defined by the OPPs, hence make the upper watermark infinitely
> +	 * high to avoid unnecessary upper interrupts in that case.
> +	 */
> +	if (freq == tegra->max_freq)
> +		upper = ULONG_MAX;
> +	else
> +		upper = lower + delta;
> +
> +	device_writel(dev, upper, ACTMON_DEV_UPPER_WMARK);
>  
>  	/*
>  	 * Meanwhile the lower mark is based on the average value
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 24/24] PM / devfreq: tegra20/30: Add Dmitry as a maintainer
  2019-07-07 22:33 ` [PATCH v4 24/24] PM / devfreq: tegra20/30: Add Dmitry as a maintainer Dmitry Osipenko
@ 2019-07-18  9:56   ` Chanwoo Choi
  0 siblings, 0 replies; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-18  9:56 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:33, Dmitry Osipenko wrote:
> I was contributing to the NVIDIA Tegra20+ devfreq drivers recently and
> want to help keep them working and evolving in the future.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  MAINTAINERS | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 95d4bd85df44..4e47ce737376 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -10356,6 +10356,15 @@ F:	include/linux/memblock.h
>  F:	mm/memblock.c
>  F:	Documentation/core-api/boot-time-mm.rst
>  
> +MEMORY FREQUENCY SCALING DRIVERS FOR NVIDIA TEGRA
> +M:	Dmitry Osipenko <digetx@gmail.com>
> +L:	linux-pm@vger.kernel.org
> +L:	linux-tegra@vger.kernel.org
> +T:	git git://git.kernel.org/pub/scm/linux/kernel/git/mzx/devfreq.git
> +S:	Maintained
> +F:	drivers/devfreq/tegra20-devfreq.c
> +F:	drivers/devfreq/tegra30-devfreq.c
> +
>  MEMORY MANAGEMENT
>  L:	linux-mm@kvack.org
>  W:	http://www.linux-mm.org
> 

Acked-by: Chanwoo Choi <cw00.choi@samsung.com>


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 22/24] PM / devfreq: tegra30: Include appropriate header
  2019-07-07 22:33 ` [PATCH v4 22/24] PM / devfreq: tegra30: Include appropriate header Dmitry Osipenko
@ 2019-07-18  9:58   ` Chanwoo Choi
  2019-07-19  0:34     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-18  9:58 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:33, Dmitry Osipenko wrote:
> It's not very correct to include mod_devicetable.h for the OF device
> drivers and of_device.h should be included instead.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 8a674fad26be..19e872a64148 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -13,7 +13,7 @@
>  #include <linux/io.h>
>  #include <linux/irq.h>
>  #include <linux/module.h>
> -#include <linux/mod_devicetable.h>
> +#include <linux/of_device.h>
>  #include <linux/platform_device.h>
>  #include <linux/pm_opp.h>
>  #include <linux/reset.h>
> 

Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>

Is not there other unused header file anymore?

-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 23/24] PM / devfreq: tegra30: Increase sampling period to 16ms
  2019-07-07 22:33 ` [PATCH v4 23/24] PM / devfreq: tegra30: Increase sampling period to 16ms Dmitry Osipenko
@ 2019-07-18 10:00   ` Chanwoo Choi
  0 siblings, 0 replies; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-18 10:00 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:33, Dmitry Osipenko wrote:
> Increase sampling period by 4ms to get a nicer pow2 value, converting
> diving into shifts in the code. That's more preferable for Tegra30 that
> doesn't have hardware divider instructions because of older Cortex-A9 CPU.
> In a result boosting events are delayed by 4ms, which is not sensible in
> practice at all.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 19e872a64148..bde19b758643 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -71,7 +71,7 @@
>   * translates to 2 ^ (K_VAL + 1). ex: 2 ^ (6 + 1) = 128
>   */
>  #define ACTMON_AVERAGE_WINDOW_LOG2				6
> -#define ACTMON_SAMPLING_PERIOD					12 /* ms */
> +#define ACTMON_SAMPLING_PERIOD					16 /* ms */
>  
>  #define KHZ							1000
>  
> 

Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>

-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 21/24] PM / devfreq: tegra30: Synchronize average count on target's update
  2019-07-07 22:33 ` [PATCH v4 21/24] PM / devfreq: tegra30: Synchronize average count on target's update Dmitry Osipenko
@ 2019-07-18 10:15   ` Chanwoo Choi
  2019-07-19  0:31     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-18 10:15 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:33, Dmitry Osipenko wrote:
> The average count may get out of sync if interrupt was disabled / avoided
> for a long time due to upper watermark optimization, hence it should be
> re-synced on each target's update to ensure that watermarks are set up
> correctly on EMC rate-change notification and that a correct frequency
> is selected for device.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 30 ++++++++++++++++++++++++++++++
>  1 file changed, 30 insertions(+)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 4d582809acb6..8a674fad26be 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -466,11 +466,41 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
>  			      dev->boost_freq, cpufreq_get(0));
>  }
>  
> +static void tegra_devfreq_sync_avg_count(struct tegra_devfreq *tegra,
> +					 struct tegra_devfreq_device *dev)
> +{
> +	u32 avg_count, avg_freq, old_upper, new_upper;
> +
> +	avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
> +	avg_freq = avg_count / ACTMON_SAMPLING_PERIOD;
> +
> +	old_upper = tegra_actmon_upper_freq(tegra, dev->avg_freq);
> +	new_upper = tegra_actmon_upper_freq(tegra, avg_freq);
> +
> +	/* similar to ISR, see comments in actmon_isr_device() */
> +	if (old_upper != new_upper) {
> +		dev->avg_freq = avg_freq;
> +		dev->boost_freq = 0;
> +	}
> +}
> +
>  static unsigned long actmon_update_target(struct tegra_devfreq *tegra,
>  					  struct tegra_devfreq_device *dev)
>  {
>  	unsigned long target_freq;
>  
> +	/*
> +	 * The avg_count / avg_freq is getting snapshoted on device's
> +	 * interrupt, but there are cases where actual value need to
> +	 * be utilized on target's update, like CPUFreq boosting and
> +	 * overriding the min freq via /sys/class/devfreq/devfreq0/min_freq
> +	 * because we're optimizing the upper watermark based on the
> +	 * actual EMC frequency. This means that interrupt may be
> +	 * inactive for a long time and thus making snapshoted value
> +	 * outdated.
> +	 */
> +	tegra_devfreq_sync_avg_count(tegra, dev);

I think that you don't need to add the separate function to calculate
the 'dev->avg_freq'. It is enough with your detailed comment to add
this code in this function.

> +
>  	target_freq = min(dev->avg_freq + dev->boost_freq, KHZ_MAX);
>  	target_freq = tegra_actmon_account_cpu_freq(tegra, dev, target_freq);
>  
> 

And also, is it impossible to squash this patch with patch19/patch20?

-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 05/24] PM / devfreq: tegra30: Set up watermarks properly
  2019-07-07 22:32 ` [PATCH v4 05/24] PM / devfreq: tegra30: Set up watermarks properly Dmitry Osipenko
@ 2019-07-18 10:17   ` Chanwoo Choi
  2019-07-19  0:00     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-18 10:17 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> The current implementation is inaccurate and results in very intensive
> interrupt activity, which neglects the whole idea of polling offload to
> hardware. The reason of the shortcoming is that watermarks are not set
> up correctly and this results in ACTMON constantly asking to change freq
> and then these requests are ignored. The end result of this patch is that
> there are few hundreds of ACTMON's interrupts instead of tens thousands
> after few minutes of a working devfreq, meanwhile the transitions activity
> stays about the same and governor becomes more reactive.
> 
> Since watermarks are set precisely correct now, the boosting logic is
> changed a tad to accommodate the change. The "average sustain coefficient"
> multiplier is gone now since there is no need to compensate the improper
> watermarks and EMC frequency-bump happens once boosting hits the upper
> watermark enough times, depending on the per-device boosting threshold.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 293 +++++++++++++++++++++---------
>  1 file changed, 209 insertions(+), 84 deletions(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index 4be7858c33bc..16f7e6cf3b99 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -47,6 +47,8 @@
>  
>  #define ACTMON_DEV_INTR_CONSECUTIVE_UPPER			BIT(31)
>  #define ACTMON_DEV_INTR_CONSECUTIVE_LOWER			BIT(30)
> +#define ACTMON_DEV_INTR_AVG_BELOW_WMARK				BIT(25)
> +#define ACTMON_DEV_INTR_AVG_ABOVE_WMARK				BIT(24)
>  
>  #define ACTMON_ABOVE_WMARK_WINDOW				1
>  #define ACTMON_BELOW_WMARK_WINDOW				3
> @@ -63,9 +65,8 @@
>   * ACTMON_AVERAGE_WINDOW_LOG2: default value for @DEV_CTRL_K_VAL, which
>   * translates to 2 ^ (K_VAL + 1). ex: 2 ^ (6 + 1) = 128
>   */
> -#define ACTMON_AVERAGE_WINDOW_LOG2			6
> -#define ACTMON_SAMPLING_PERIOD				12 /* ms */
> -#define ACTMON_DEFAULT_AVG_BAND				6  /* 1/10 of % */
> +#define ACTMON_AVERAGE_WINDOW_LOG2				6
> +#define ACTMON_SAMPLING_PERIOD					12 /* ms */
>  
>  #define KHZ							1000
>  
> @@ -142,9 +143,6 @@ struct tegra_devfreq_device {
>  	 * watermark breaches.
>  	 */
>  	unsigned long boost_freq;
> -
> -	/* Optimal frequency calculated from the stats for this device */
> -	unsigned long target_freq;
>  };
>  
>  struct tegra_devfreq {
> @@ -156,7 +154,6 @@ struct tegra_devfreq {
>  
>  	struct clk		*emc_clock;
>  	unsigned long		max_freq;
> -	unsigned long		cur_freq;
>  	struct notifier_block	rate_change_nb;
>  
>  	struct tegra_devfreq_device devices[ARRAY_SIZE(actmon_device_configs)];
> @@ -205,42 +202,182 @@ static unsigned long do_percent(unsigned long val, unsigned int pct)
>  	return val * pct / 100;
>  }
>  
> +static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq *tegra)
> +{
> +	struct tegra_actmon_emc_ratio *ratio = actmon_emc_ratios;
> +	unsigned int cpu_freq = cpufreq_get(0);
> +	unsigned int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(actmon_emc_ratios); i++, ratio++) {
> +		if (cpu_freq >= ratio->cpu_freq) {
> +			if (ratio->emc_freq >= tegra->max_freq)
> +				return tegra->max_freq;
> +			else
> +				return ratio->emc_freq;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static unsigned long
> +tegra_actmon_account_cpu_freq(struct tegra_devfreq *tegra,
> +			      struct tegra_devfreq_device *dev,
> +			      unsigned long target_freq)
> +{
> +	unsigned long static_cpu_emc_freq;
> +
> +	if (dev->config->avg_dependency_threshold &&
> +	    dev->config->avg_dependency_threshold < dev->avg_count) {
> +		static_cpu_emc_freq = actmon_cpu_to_emc_rate(tegra);
> +		target_freq = max(target_freq, static_cpu_emc_freq);
> +	}
> +
> +	return target_freq;
> +}
> +
> +static unsigned long tegra_actmon_lower_freq(struct tegra_devfreq *tegra,
> +					     unsigned long target_freq)
> +{
> +	unsigned long lower = target_freq;
> +	struct dev_pm_opp *opp;
> +
> +	opp = dev_pm_opp_find_freq_floor(tegra->devfreq->dev.parent, &lower);
> +	if (IS_ERR(opp))
> +		lower = 0;
> +	else
> +		dev_pm_opp_put(opp);
> +
> +	return lower;
> +}
> +
> +static unsigned long tegra_actmon_upper_freq(struct tegra_devfreq *tegra,
> +					     unsigned long target_freq)
> +{
> +	unsigned long upper = target_freq + 1;
> +	struct dev_pm_opp *opp;
> +
> +	opp = dev_pm_opp_find_freq_ceil(tegra->devfreq->dev.parent, &upper);
> +	if (IS_ERR(opp))
> +		upper = ULONG_MAX;
> +	else
> +		dev_pm_opp_put(opp);
> +
> +	return upper;
> +}
> +
> +static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
> +					 struct tegra_devfreq_device *dev,
> +					 unsigned long target_freq,
> +					 unsigned long *lower,
> +					 unsigned long *upper)
> +{
> +	/*
> +	 * Memory frequencies are guaranteed to have 1MHz granularity
> +	 * and thus we need this rounding down to get a proper watermarks
> +	 * range in a case where target_freq falls into a range of
> +	 * next_possible_opp_freq - 1MHz.
> +	 */
> +	target_freq = round_down(target_freq, 1000000);
> +
> +	/* watermarks are set at the borders of the corresponding OPPs */
> +	*lower = tegra_actmon_lower_freq(tegra, target_freq);
> +	*upper = tegra_actmon_upper_freq(tegra, target_freq);
> +
> +	*lower /= KHZ;
> +	*upper /= KHZ;
> +
> +	/*
> +	 * The upper watermark should take into account CPU's frequency
> +	 * because cpu_to_emc_rate() may override the target_freq with
> +	 * a higher value and thus upper watermark need to be set up
> +	 * accordingly to avoid parasitic upper-events.
> +	 */
> +	*upper = tegra_actmon_account_cpu_freq(tegra, dev, *upper);
> +
> +	*lower *= ACTMON_SAMPLING_PERIOD;
> +	*upper *= ACTMON_SAMPLING_PERIOD;
> +}
> +
>  static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
>  					   struct tegra_devfreq_device *dev)
>  {
> -	u32 avg = dev->avg_count;
> -	u32 avg_band_freq = tegra->max_freq * ACTMON_DEFAULT_AVG_BAND / KHZ;
> -	u32 band = avg_band_freq * ACTMON_SAMPLING_PERIOD;
> +	unsigned long lower, upper, freq;
>  
> -	device_writel(dev, avg + band, ACTMON_DEV_AVG_UPPER_WMARK);
> +	freq = dev->avg_count / ACTMON_SAMPLING_PERIOD * KHZ;
> +	tegra_actmon_get_lower_upper(tegra, dev, freq, &lower, &upper);
>  
> -	avg = max(dev->avg_count, band);
> -	device_writel(dev, avg - band, ACTMON_DEV_AVG_LOWER_WMARK);
> +	/*
> +	 * We want to get interrupts when MCCPU client crosses the
> +	 * dependency threshold in order to take into / out of account
> +	 * the CPU's freq.
> +	 */
> +	if (lower < dev->config->avg_dependency_threshold &&
> +	    upper > dev->config->avg_dependency_threshold) {
> +		if (dev->avg_count < dev->config->avg_dependency_threshold)
> +			upper = dev->config->avg_dependency_threshold;
> +		else
> +			lower = dev->config->avg_dependency_threshold;
> +	}
> +
> +	device_writel(dev, lower, ACTMON_DEV_AVG_LOWER_WMARK);
> +	device_writel(dev, upper, ACTMON_DEV_AVG_UPPER_WMARK);
>  }
>  
>  static void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
> -				       struct tegra_devfreq_device *dev)
> +				       struct tegra_devfreq_device *dev,
> +				       unsigned long freq)
>  {
> -	u32 val = tegra->cur_freq * ACTMON_SAMPLING_PERIOD;
> +	unsigned long lower, upper, delta;
> +
> +	/*
> +	 * Boosting logic kicks-in once lower / upper watermark is hit.
> +	 * The watermarks are based on the updated EMC rate and the
> +	 * average activity.
> +	 *
> +	 * The higher watermark is set in accordance to the EMC rate
> +	 * because we want to set it to the highest mark here and EMC rate
> +	 * represents that mark. The consecutive-upper interrupts are
> +	 * always enabled and we don't want to receive them if they won't
> +	 * do anything useful, hence the upper watermark is capped to maximum.
> +	 * Note that the EMC rate is changed once boosting pushed the rate
> +	 * too high, in that case boosting-up will be stopped because
> +	 * upper watermark is much higher now and it is *important* to
> +	 * stop the unwanted interrupts.
> +	 */
> +	tegra_actmon_get_lower_upper(tegra, dev, freq - 1, &lower, &upper);
> +
> +	delta = do_percent(upper - lower, dev->config->boost_up_threshold);
> +	device_writel(dev, lower + delta, ACTMON_DEV_UPPER_WMARK);
>  
> -	device_writel(dev, do_percent(val, dev->config->boost_up_threshold),
> -		      ACTMON_DEV_UPPER_WMARK);
> +	/*
> +	 * Meanwhile the lower mark is based on the average value
> +	 * because it is the lowest possible consecutive-mark for this
> +	 * device. Once that mark is hit and boosting is stopped, the
> +	 * interrupt is disabled by ISR.
> +	 */
> +	freq = dev->avg_count / ACTMON_SAMPLING_PERIOD * KHZ;
> +	tegra_actmon_get_lower_upper(tegra, dev, freq, &lower, &upper);
>  
> -	device_writel(dev, do_percent(val, dev->config->boost_down_threshold),
> -		      ACTMON_DEV_LOWER_WMARK);
> +	delta = do_percent(upper - lower, dev->config->boost_down_threshold);
> +	device_writel(dev, lower + delta, ACTMON_DEV_LOWER_WMARK);
>  }
>  
>  static void actmon_isr_device(struct tegra_devfreq *tegra,
>  			      struct tegra_devfreq_device *dev)
>  {
> -	u32 intr_status, dev_ctrl;
> +	u32 intr_status, dev_ctrl, avg_intr_mask;
>  
>  	dev->avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
> -	tegra_devfreq_update_avg_wmark(tegra, dev);
> -
>  	intr_status = device_readl(dev, ACTMON_DEV_INTR_STATUS);
>  	dev_ctrl = device_readl(dev, ACTMON_DEV_CTRL);
>  
> +	avg_intr_mask = ACTMON_DEV_INTR_AVG_BELOW_WMARK |
> +			ACTMON_DEV_INTR_AVG_ABOVE_WMARK;
> +
> +	if (intr_status & avg_intr_mask)
> +		tegra_devfreq_update_avg_wmark(tegra, dev);
> +
>  	if (intr_status & ACTMON_DEV_INTR_CONSECUTIVE_UPPER) {
>  		/*
>  		 * new_boost = min(old_boost * up_coef + step, max_freq)
> @@ -253,8 +390,6 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
>  
>  		if (dev->boost_freq >= tegra->max_freq)
>  			dev->boost_freq = tegra->max_freq;
> -		else
> -			dev_ctrl |= ACTMON_DEV_CTRL_CONSECUTIVE_ABOVE_WMARK_EN;
>  	} else if (intr_status & ACTMON_DEV_INTR_CONSECUTIVE_LOWER) {
>  		/*
>  		 * new_boost = old_boost * down_coef
> @@ -263,63 +398,37 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
>  		dev->boost_freq = do_percent(dev->boost_freq,
>  					     dev->config->boost_down_coeff);
>  
> -		dev_ctrl |= ACTMON_DEV_CTRL_CONSECUTIVE_ABOVE_WMARK_EN;
> -
>  		if (dev->boost_freq < (ACTMON_BOOST_FREQ_STEP >> 1))
>  			dev->boost_freq = 0;
> -		else
> -			dev_ctrl |= ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
>  	}
>  
> -	if (dev->config->avg_dependency_threshold) {
> -		if (dev->avg_count >= dev->config->avg_dependency_threshold)
> -			dev_ctrl |= ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
> -		else if (dev->boost_freq == 0)
> -			dev_ctrl &= ~ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
> +	if (intr_status & avg_intr_mask) {
> +		/*
> +		 * Once average watermark is hit, it means that the memory
> +		 * activity changed significantly and thus boosting-up shall
> +		 * be reset because EMC clock rate will be changed and
> +		 * boosting will restart in this case.
> +		 */
> +		dev->boost_freq = 0;
>  	}
>  
> -	device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
> +	/* no boosting => no need for consecutive-down interrupt */
> +	if (dev->boost_freq == 0)
> +		dev_ctrl &= ~ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
>  
> +	device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
>  	device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
>  }
>  
> -static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq *tegra,
> -					    unsigned long cpu_freq)
> -{
> -	unsigned int i;
> -	struct tegra_actmon_emc_ratio *ratio = actmon_emc_ratios;
> -
> -	for (i = 0; i < ARRAY_SIZE(actmon_emc_ratios); i++, ratio++) {
> -		if (cpu_freq >= ratio->cpu_freq) {
> -			if (ratio->emc_freq >= tegra->max_freq)
> -				return tegra->max_freq;
> -			else
> -				return ratio->emc_freq;
> -		}
> -	}
> -
> -	return 0;
> -}
> -
> -static void actmon_update_target(struct tegra_devfreq *tegra,
> -				 struct tegra_devfreq_device *dev)
> +static unsigned long actmon_update_target(struct tegra_devfreq *tegra,
> +					  struct tegra_devfreq_device *dev)
>  {
> -	unsigned long cpu_freq = 0;
> -	unsigned long static_cpu_emc_freq = 0;
> -	unsigned int avg_sustain_coef;
> -
> -	if (dev->config->avg_dependency_threshold) {
> -		cpu_freq = cpufreq_get(0);
> -		static_cpu_emc_freq = actmon_cpu_to_emc_rate(tegra, cpu_freq);
> -	}
> +	unsigned long target_freq;
>  
> -	dev->target_freq = dev->avg_count / ACTMON_SAMPLING_PERIOD;
> -	avg_sustain_coef = 100 * 100 / dev->config->boost_up_threshold;
> -	dev->target_freq = do_percent(dev->target_freq, avg_sustain_coef);
> -	dev->target_freq += dev->boost_freq;
> +	target_freq = dev->avg_count / ACTMON_SAMPLING_PERIOD + dev->boost_freq;
> +	target_freq = tegra_actmon_account_cpu_freq(tegra, dev, target_freq);
>  
> -	if (dev->avg_count >= dev->config->avg_dependency_threshold)
> -		dev->target_freq = max(dev->target_freq, static_cpu_emc_freq);
> +	return target_freq;
>  }
>  
>  static irqreturn_t actmon_thread_isr(int irq, void *data)
> @@ -351,8 +460,8 @@ static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
>  				       unsigned long action, void *ptr)
>  {
>  	struct clk_notifier_data *data = ptr;
> -	struct tegra_devfreq *tegra;
>  	struct tegra_devfreq_device *dev;
> +	struct tegra_devfreq *tegra;
>  	unsigned int i;
>  
>  	if (action != POST_RATE_CHANGE)
> @@ -360,12 +469,28 @@ static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
>  
>  	tegra = container_of(nb, struct tegra_devfreq, rate_change_nb);
>  
> -	tegra->cur_freq = data->new_rate / KHZ;
> -
> +	/*
> +	 * EMC rate could change due to three reasons:
> +	 *
> +	 *    1. Average watermark hit
> +	 *    2. Boosting overflow
> +	 *    3. CPU freq change
> +	 *
> +	 * Once rate is changed, the consecutive watermarks need to be
> +	 * updated in order for boosting to work properly and to avoid
> +	 * unnecessary interrupts. Note that the consecutive range is set for
> +	 * all of devices using the same rate, hence if CPU is doing much
> +	 * less than the other memory clients, then its upper watermark will
> +	 * be very high in comparison to the actual activity (lower watermark)
> +	 * and thus unnecessary upper-interrupts will be suppressed.
> +	 *
> +	 * The average watermarks also should be updated because of 3.
> +	 */
>  	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++) {
>  		dev = &tegra->devices[i];
>  
> -		tegra_devfreq_update_wmark(tegra, dev);
> +		tegra_devfreq_update_avg_wmark(tegra, dev);
> +		tegra_devfreq_update_wmark(tegra, dev, data->new_rate);
>  	}
>  
>  	return NOTIFY_OK;
> @@ -374,15 +499,14 @@ static int tegra_actmon_rate_notify_cb(struct notifier_block *nb,
>  static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
>  					  struct tegra_devfreq_device *dev)
>  {
> -	u32 val = 0;
> -
> -	dev->target_freq = tegra->cur_freq;
> +	u32 val = 0, target_freq;
>  
> -	dev->avg_count = tegra->cur_freq * ACTMON_SAMPLING_PERIOD;
> +	target_freq = clk_get_rate(tegra->emc_clock) / KHZ;
> +	dev->avg_count = target_freq * ACTMON_SAMPLING_PERIOD;
>  	device_writel(dev, dev->avg_count, ACTMON_DEV_INIT_AVG);
>  
>  	tegra_devfreq_update_avg_wmark(tegra, dev);
> -	tegra_devfreq_update_wmark(tegra, dev);
> +	tegra_devfreq_update_wmark(tegra, dev, target_freq);
>  
>  	device_writel(dev, ACTMON_COUNT_WEIGHT, ACTMON_DEV_COUNT_WEIGHT);
>  	device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
> @@ -469,13 +593,13 @@ static int tegra_devfreq_get_dev_status(struct device *dev,
>  	struct tegra_devfreq_device *actmon_dev;
>  	unsigned long cur_freq;
>  
> -	cur_freq = READ_ONCE(tegra->cur_freq);
> +	cur_freq = clk_get_rate(tegra->emc_clock);
>  
>  	/* To be used by the tegra governor */
>  	stat->private_data = tegra;
>  
>  	/* The below are to be used by the other governors */
> -	stat->current_frequency = cur_freq * KHZ;
> +	stat->current_frequency = cur_freq;
>  
>  	actmon_dev = &tegra->devices[MCALL];
>  
> @@ -486,7 +610,7 @@ static int tegra_devfreq_get_dev_status(struct device *dev,
>  	stat->busy_time *= 100 / BUS_SATURATION_RATIO;
>  
>  	/* Number of cycles in a sampling period */
> -	stat->total_time = ACTMON_SAMPLING_PERIOD * cur_freq;
> +	stat->total_time = cur_freq / KHZ * ACTMON_SAMPLING_PERIOD;
>  
>  	stat->busy_time = min(stat->busy_time, stat->total_time);
>  
> @@ -505,6 +629,7 @@ static int tegra_governor_get_target(struct devfreq *devfreq,
>  	struct devfreq_dev_status *stat;
>  	struct tegra_devfreq *tegra;
>  	struct tegra_devfreq_device *dev;
> +	unsigned long dev_target_freq;
>  	unsigned long target_freq = 0;
>  	unsigned int i;
>  	int err;
> @@ -520,9 +645,9 @@ static int tegra_governor_get_target(struct devfreq *devfreq,
>  	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++) {
>  		dev = &tegra->devices[i];
>  
> -		actmon_update_target(tegra, dev);
> +		dev_target_freq = actmon_update_target(tegra, dev);
>  
> -		target_freq = max(target_freq, dev->target_freq);
> +		target_freq = max(target_freq, dev_target_freq);
>  	}
>  
>  	*freq = target_freq * KHZ;
> @@ -642,7 +767,6 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>  		return rate;
>  	}
>  
> -	tegra->cur_freq = clk_get_rate(tegra->emc_clock) / KHZ;
>  	tegra->max_freq = rate / KHZ;
>  
>  	for (i = 0; i < ARRAY_SIZE(actmon_device_configs); i++) {
> @@ -671,7 +795,8 @@ static int tegra_devfreq_probe(struct platform_device *pdev)
>  	platform_set_drvdata(pdev, tegra);
>  
>  	tegra->rate_change_nb.notifier_call = tegra_actmon_rate_notify_cb;
> -	err = clk_notifier_register(tegra->emc_clock, &tegra->rate_change_nb);
> +	err = clk_notifier_register(tegra->emc_clock,
> +				    &tegra->rate_change_nb);
>  	if (err) {
>  		dev_err(&pdev->dev,
>  			"Failed to register rate change notifier\n");
> 


Maybe, it is possible to merge patch4/patch19/patch20 to one patch.


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 05/24] PM / devfreq: tegra30: Set up watermarks properly
  2019-07-18 10:17   ` Chanwoo Choi
@ 2019-07-19  0:00     ` Dmitry Osipenko
  2019-07-19  1:31       ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19  0:00 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

В Thu, 18 Jul 2019 19:17:17 +0900
Chanwoo Choi <cw00.choi@samsung.com> пишет:

> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> > The current implementation is inaccurate and results in very
> > intensive interrupt activity, which neglects the whole idea of
> > polling offload to hardware. The reason of the shortcoming is that
> > watermarks are not set up correctly and this results in ACTMON
> > constantly asking to change freq and then these requests are
> > ignored. The end result of this patch is that there are few
> > hundreds of ACTMON's interrupts instead of tens thousands after few
> > minutes of a working devfreq, meanwhile the transitions activity
> > stays about the same and governor becomes more reactive.
> > 
> > Since watermarks are set precisely correct now, the boosting logic
> > is changed a tad to accommodate the change. The "average sustain
> > coefficient" multiplier is gone now since there is no need to
> > compensate the improper watermarks and EMC frequency-bump happens
> > once boosting hits the upper watermark enough times, depending on
> > the per-device boosting threshold.
> > 
> > Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> > ---
> >  drivers/devfreq/tegra30-devfreq.c | 293
> > +++++++++++++++++++++--------- 1 file changed, 209 insertions(+),
> > 84 deletions(-)
> > 
> > diff --git a/drivers/devfreq/tegra30-devfreq.c
> > b/drivers/devfreq/tegra30-devfreq.c index
> > 4be7858c33bc..16f7e6cf3b99 100644 ---
> > a/drivers/devfreq/tegra30-devfreq.c +++
> > b/drivers/devfreq/tegra30-devfreq.c @@ -47,6 +47,8 @@
> >  
> >  #define ACTMON_DEV_INTR_CONSECUTIVE_UPPER
> > BIT(31) #define
> > ACTMON_DEV_INTR_CONSECUTIVE_LOWER			BIT(30)
> > +#define
> > ACTMON_DEV_INTR_AVG_BELOW_WMARK
> > BIT(25) +#define
> > ACTMON_DEV_INTR_AVG_ABOVE_WMARK
> > BIT(24) #define
> > ACTMON_ABOVE_WMARK_WINDOW				1 #define
> > ACTMON_BELOW_WMARK_WINDOW				3 @@ -63,9
> > +65,8 @@
> >   * ACTMON_AVERAGE_WINDOW_LOG2: default value for @DEV_CTRL_K_VAL,
> > which
> >   * translates to 2 ^ (K_VAL + 1). ex: 2 ^ (6 + 1) = 128
> >   */
> > -#define ACTMON_AVERAGE_WINDOW_LOG2			6
> > -#define ACTMON_SAMPLING_PERIOD
> > 12 /* ms */ -#define
> > ACTMON_DEFAULT_AVG_BAND				6  /* 1/10
> > of % */ +#define
> > ACTMON_AVERAGE_WINDOW_LOG2				6
> > +#define
> > ACTMON_SAMPLING_PERIOD					12 /*
> > ms */ #define
> > KHZ							1000 @@
> > -142,9 +143,6 @@ struct tegra_devfreq_device {
> >  	 * watermark breaches.
> >  	 */
> >  	unsigned long boost_freq;
> > -
> > -	/* Optimal frequency calculated from the stats for this
> > device */
> > -	unsigned long target_freq;
> >  };
> >  
> >  struct tegra_devfreq {
> > @@ -156,7 +154,6 @@ struct tegra_devfreq {
> >  
> >  	struct clk		*emc_clock;
> >  	unsigned long		max_freq;
> > -	unsigned long		cur_freq;
> >  	struct notifier_block	rate_change_nb;
> >  
> >  	struct tegra_devfreq_device
> > devices[ARRAY_SIZE(actmon_device_configs)]; @@ -205,42 +202,182 @@
> > static unsigned long do_percent(unsigned long val, unsigned int
> > pct) return val * pct / 100; }
> >  
> > +static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq
> > *tegra) +{
> > +	struct tegra_actmon_emc_ratio *ratio = actmon_emc_ratios;
> > +	unsigned int cpu_freq = cpufreq_get(0);
> > +	unsigned int i;
> > +
> > +	for (i = 0; i < ARRAY_SIZE(actmon_emc_ratios); i++,
> > ratio++) {
> > +		if (cpu_freq >= ratio->cpu_freq) {
> > +			if (ratio->emc_freq >= tegra->max_freq)
> > +				return tegra->max_freq;
> > +			else
> > +				return ratio->emc_freq;
> > +		}
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static unsigned long
> > +tegra_actmon_account_cpu_freq(struct tegra_devfreq *tegra,
> > +			      struct tegra_devfreq_device *dev,
> > +			      unsigned long target_freq)
> > +{
> > +	unsigned long static_cpu_emc_freq;
> > +
> > +	if (dev->config->avg_dependency_threshold &&
> > +	    dev->config->avg_dependency_threshold <
> > dev->avg_count) {
> > +		static_cpu_emc_freq =
> > actmon_cpu_to_emc_rate(tegra);
> > +		target_freq = max(target_freq,
> > static_cpu_emc_freq);
> > +	}
> > +
> > +	return target_freq;
> > +}
> > +
> > +static unsigned long tegra_actmon_lower_freq(struct tegra_devfreq
> > *tegra,
> > +					     unsigned long
> > target_freq) +{
> > +	unsigned long lower = target_freq;
> > +	struct dev_pm_opp *opp;
> > +
> > +	opp =
> > dev_pm_opp_find_freq_floor(tegra->devfreq->dev.parent, &lower);
> > +	if (IS_ERR(opp))
> > +		lower = 0;
> > +	else
> > +		dev_pm_opp_put(opp);
> > +
> > +	return lower;
> > +}
> > +
> > +static unsigned long tegra_actmon_upper_freq(struct tegra_devfreq
> > *tegra,
> > +					     unsigned long
> > target_freq) +{
> > +	unsigned long upper = target_freq + 1;
> > +	struct dev_pm_opp *opp;
> > +
> > +	opp =
> > dev_pm_opp_find_freq_ceil(tegra->devfreq->dev.parent, &upper);
> > +	if (IS_ERR(opp))
> > +		upper = ULONG_MAX;
> > +	else
> > +		dev_pm_opp_put(opp);
> > +
> > +	return upper;
> > +}
> > +
> > +static void tegra_actmon_get_lower_upper(struct tegra_devfreq
> > *tegra,
> > +					 struct
> > tegra_devfreq_device *dev,
> > +					 unsigned long target_freq,
> > +					 unsigned long *lower,
> > +					 unsigned long *upper)
> > +{
> > +	/*
> > +	 * Memory frequencies are guaranteed to have 1MHz
> > granularity
> > +	 * and thus we need this rounding down to get a proper
> > watermarks
> > +	 * range in a case where target_freq falls into a range of
> > +	 * next_possible_opp_freq - 1MHz.
> > +	 */
> > +	target_freq = round_down(target_freq, 1000000);
> > +
> > +	/* watermarks are set at the borders of the corresponding
> > OPPs */
> > +	*lower = tegra_actmon_lower_freq(tegra, target_freq);
> > +	*upper = tegra_actmon_upper_freq(tegra, target_freq);
> > +
> > +	*lower /= KHZ;
> > +	*upper /= KHZ;
> > +
> > +	/*
> > +	 * The upper watermark should take into account CPU's
> > frequency
> > +	 * because cpu_to_emc_rate() may override the target_freq
> > with
> > +	 * a higher value and thus upper watermark need to be set
> > up
> > +	 * accordingly to avoid parasitic upper-events.
> > +	 */
> > +	*upper = tegra_actmon_account_cpu_freq(tegra, dev, *upper);
> > +
> > +	*lower *= ACTMON_SAMPLING_PERIOD;
> > +	*upper *= ACTMON_SAMPLING_PERIOD;
> > +}
> > +
> >  static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq
> > *tegra, struct tegra_devfreq_device *dev)
> >  {
> > -	u32 avg = dev->avg_count;
> > -	u32 avg_band_freq = tegra->max_freq *
> > ACTMON_DEFAULT_AVG_BAND / KHZ;
> > -	u32 band = avg_band_freq * ACTMON_SAMPLING_PERIOD;
> > +	unsigned long lower, upper, freq;
> >  
> > -	device_writel(dev, avg + band, ACTMON_DEV_AVG_UPPER_WMARK);
> > +	freq = dev->avg_count / ACTMON_SAMPLING_PERIOD * KHZ;
> > +	tegra_actmon_get_lower_upper(tegra, dev, freq, &lower,
> > &upper); 
> > -	avg = max(dev->avg_count, band);
> > -	device_writel(dev, avg - band, ACTMON_DEV_AVG_LOWER_WMARK);
> > +	/*
> > +	 * We want to get interrupts when MCCPU client crosses the
> > +	 * dependency threshold in order to take into / out of
> > account
> > +	 * the CPU's freq.
> > +	 */
> > +	if (lower < dev->config->avg_dependency_threshold &&
> > +	    upper > dev->config->avg_dependency_threshold) {
> > +		if (dev->avg_count <
> > dev->config->avg_dependency_threshold)
> > +			upper =
> > dev->config->avg_dependency_threshold;
> > +		else
> > +			lower =
> > dev->config->avg_dependency_threshold;
> > +	}
> > +
> > +	device_writel(dev, lower, ACTMON_DEV_AVG_LOWER_WMARK);
> > +	device_writel(dev, upper, ACTMON_DEV_AVG_UPPER_WMARK);
> >  }
> >  
> >  static void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
> > -				       struct tegra_devfreq_device
> > *dev)
> > +				       struct tegra_devfreq_device
> > *dev,
> > +				       unsigned long freq)
> >  {
> > -	u32 val = tegra->cur_freq * ACTMON_SAMPLING_PERIOD;
> > +	unsigned long lower, upper, delta;
> > +
> > +	/*
> > +	 * Boosting logic kicks-in once lower / upper watermark is
> > hit.
> > +	 * The watermarks are based on the updated EMC rate and the
> > +	 * average activity.
> > +	 *
> > +	 * The higher watermark is set in accordance to the EMC
> > rate
> > +	 * because we want to set it to the highest mark here and
> > EMC rate
> > +	 * represents that mark. The consecutive-upper interrupts
> > are
> > +	 * always enabled and we don't want to receive them if
> > they won't
> > +	 * do anything useful, hence the upper watermark is capped
> > to maximum.
> > +	 * Note that the EMC rate is changed once boosting pushed
> > the rate
> > +	 * too high, in that case boosting-up will be stopped
> > because
> > +	 * upper watermark is much higher now and it is
> > *important* to
> > +	 * stop the unwanted interrupts.
> > +	 */
> > +	tegra_actmon_get_lower_upper(tegra, dev, freq - 1, &lower,
> > &upper); +
> > +	delta = do_percent(upper - lower,
> > dev->config->boost_up_threshold);
> > +	device_writel(dev, lower + delta, ACTMON_DEV_UPPER_WMARK);
> >  
> > -	device_writel(dev, do_percent(val,
> > dev->config->boost_up_threshold),
> > -		      ACTMON_DEV_UPPER_WMARK);
> > +	/*
> > +	 * Meanwhile the lower mark is based on the average value
> > +	 * because it is the lowest possible consecutive-mark for
> > this
> > +	 * device. Once that mark is hit and boosting is stopped,
> > the
> > +	 * interrupt is disabled by ISR.
> > +	 */
> > +	freq = dev->avg_count / ACTMON_SAMPLING_PERIOD * KHZ;
> > +	tegra_actmon_get_lower_upper(tegra, dev, freq, &lower,
> > &upper); 
> > -	device_writel(dev, do_percent(val,
> > dev->config->boost_down_threshold),
> > -		      ACTMON_DEV_LOWER_WMARK);
> > +	delta = do_percent(upper - lower,
> > dev->config->boost_down_threshold);
> > +	device_writel(dev, lower + delta, ACTMON_DEV_LOWER_WMARK);
> >  }
> >  
> >  static void actmon_isr_device(struct tegra_devfreq *tegra,
> >  			      struct tegra_devfreq_device *dev)
> >  {
> > -	u32 intr_status, dev_ctrl;
> > +	u32 intr_status, dev_ctrl, avg_intr_mask;
> >  
> >  	dev->avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
> > -	tegra_devfreq_update_avg_wmark(tegra, dev);
> > -
> >  	intr_status = device_readl(dev, ACTMON_DEV_INTR_STATUS);
> >  	dev_ctrl = device_readl(dev, ACTMON_DEV_CTRL);
> >  
> > +	avg_intr_mask = ACTMON_DEV_INTR_AVG_BELOW_WMARK |
> > +			ACTMON_DEV_INTR_AVG_ABOVE_WMARK;
> > +
> > +	if (intr_status & avg_intr_mask)
> > +		tegra_devfreq_update_avg_wmark(tegra, dev);
> > +
> >  	if (intr_status & ACTMON_DEV_INTR_CONSECUTIVE_UPPER) {
> >  		/*
> >  		 * new_boost = min(old_boost * up_coef + step,
> > max_freq) @@ -253,8 +390,6 @@ static void actmon_isr_device(struct
> > tegra_devfreq *tegra, 
> >  		if (dev->boost_freq >= tegra->max_freq)
> >  			dev->boost_freq = tegra->max_freq;
> > -		else
> > -			dev_ctrl |=
> > ACTMON_DEV_CTRL_CONSECUTIVE_ABOVE_WMARK_EN; } else if (intr_status
> > & ACTMON_DEV_INTR_CONSECUTIVE_LOWER) { /*
> >  		 * new_boost = old_boost * down_coef
> > @@ -263,63 +398,37 @@ static void actmon_isr_device(struct
> > tegra_devfreq *tegra, dev->boost_freq = do_percent(dev->boost_freq,
> >  					     dev->config->boost_down_coeff);
> >  
> > -		dev_ctrl |=
> > ACTMON_DEV_CTRL_CONSECUTIVE_ABOVE_WMARK_EN; -
> >  		if (dev->boost_freq < (ACTMON_BOOST_FREQ_STEP >>
> > 1)) dev->boost_freq = 0;
> > -		else
> > -			dev_ctrl |=
> > ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN; }
> >  
> > -	if (dev->config->avg_dependency_threshold) {
> > -		if (dev->avg_count >=
> > dev->config->avg_dependency_threshold)
> > -			dev_ctrl |=
> > ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
> > -		else if (dev->boost_freq == 0)
> > -			dev_ctrl &=
> > ~ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
> > +	if (intr_status & avg_intr_mask) {
> > +		/*
> > +		 * Once average watermark is hit, it means that
> > the memory
> > +		 * activity changed significantly and thus
> > boosting-up shall
> > +		 * be reset because EMC clock rate will be changed
> > and
> > +		 * boosting will restart in this case.
> > +		 */
> > +		dev->boost_freq = 0;
> >  	}
> >  
> > -	device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
> > +	/* no boosting => no need for consecutive-down interrupt */
> > +	if (dev->boost_freq == 0)
> > +		dev_ctrl &=
> > ~ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN; 
> > +	device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
> >  	device_writel(dev, ACTMON_INTR_STATUS_CLEAR,
> > ACTMON_DEV_INTR_STATUS); }
> >  
> > -static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq
> > *tegra,
> > -					    unsigned long cpu_freq)
> > -{
> > -	unsigned int i;
> > -	struct tegra_actmon_emc_ratio *ratio = actmon_emc_ratios;
> > -
> > -	for (i = 0; i < ARRAY_SIZE(actmon_emc_ratios); i++,
> > ratio++) {
> > -		if (cpu_freq >= ratio->cpu_freq) {
> > -			if (ratio->emc_freq >= tegra->max_freq)
> > -				return tegra->max_freq;
> > -			else
> > -				return ratio->emc_freq;
> > -		}
> > -	}
> > -
> > -	return 0;
> > -}
> > -
> > -static void actmon_update_target(struct tegra_devfreq *tegra,
> > -				 struct tegra_devfreq_device *dev)
> > +static unsigned long actmon_update_target(struct tegra_devfreq
> > *tegra,
> > +					  struct
> > tegra_devfreq_device *dev) {
> > -	unsigned long cpu_freq = 0;
> > -	unsigned long static_cpu_emc_freq = 0;
> > -	unsigned int avg_sustain_coef;
> > -
> > -	if (dev->config->avg_dependency_threshold) {
> > -		cpu_freq = cpufreq_get(0);
> > -		static_cpu_emc_freq =
> > actmon_cpu_to_emc_rate(tegra, cpu_freq);
> > -	}
> > +	unsigned long target_freq;
> >  
> > -	dev->target_freq = dev->avg_count / ACTMON_SAMPLING_PERIOD;
> > -	avg_sustain_coef = 100 * 100 /
> > dev->config->boost_up_threshold;
> > -	dev->target_freq = do_percent(dev->target_freq,
> > avg_sustain_coef);
> > -	dev->target_freq += dev->boost_freq;
> > +	target_freq = dev->avg_count / ACTMON_SAMPLING_PERIOD +
> > dev->boost_freq;
> > +	target_freq = tegra_actmon_account_cpu_freq(tegra, dev,
> > target_freq); 
> > -	if (dev->avg_count >=
> > dev->config->avg_dependency_threshold)
> > -		dev->target_freq = max(dev->target_freq,
> > static_cpu_emc_freq);
> > +	return target_freq;
> >  }
> >  
> >  static irqreturn_t actmon_thread_isr(int irq, void *data)
> > @@ -351,8 +460,8 @@ static int tegra_actmon_rate_notify_cb(struct
> > notifier_block *nb, unsigned long action, void *ptr)
> >  {
> >  	struct clk_notifier_data *data = ptr;
> > -	struct tegra_devfreq *tegra;
> >  	struct tegra_devfreq_device *dev;
> > +	struct tegra_devfreq *tegra;
> >  	unsigned int i;
> >  
> >  	if (action != POST_RATE_CHANGE)
> > @@ -360,12 +469,28 @@ static int tegra_actmon_rate_notify_cb(struct
> > notifier_block *nb, 
> >  	tegra = container_of(nb, struct tegra_devfreq,
> > rate_change_nb); 
> > -	tegra->cur_freq = data->new_rate / KHZ;
> > -
> > +	/*
> > +	 * EMC rate could change due to three reasons:
> > +	 *
> > +	 *    1. Average watermark hit
> > +	 *    2. Boosting overflow
> > +	 *    3. CPU freq change
> > +	 *
> > +	 * Once rate is changed, the consecutive watermarks need
> > to be
> > +	 * updated in order for boosting to work properly and to
> > avoid
> > +	 * unnecessary interrupts. Note that the consecutive range
> > is set for
> > +	 * all of devices using the same rate, hence if CPU is
> > doing much
> > +	 * less than the other memory clients, then its upper
> > watermark will
> > +	 * be very high in comparison to the actual activity
> > (lower watermark)
> > +	 * and thus unnecessary upper-interrupts will be
> > suppressed.
> > +	 *
> > +	 * The average watermarks also should be updated because
> > of 3.
> > +	 */
> >  	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++) {
> >  		dev = &tegra->devices[i];
> >  
> > -		tegra_devfreq_update_wmark(tegra, dev);
> > +		tegra_devfreq_update_avg_wmark(tegra, dev);
> > +		tegra_devfreq_update_wmark(tegra, dev,
> > data->new_rate); }
> >  
> >  	return NOTIFY_OK;
> > @@ -374,15 +499,14 @@ static int tegra_actmon_rate_notify_cb(struct
> > notifier_block *nb, static void
> > tegra_actmon_configure_device(struct tegra_devfreq *tegra, struct
> > tegra_devfreq_device *dev) {
> > -	u32 val = 0;
> > -
> > -	dev->target_freq = tegra->cur_freq;
> > +	u32 val = 0, target_freq;
> >  
> > -	dev->avg_count = tegra->cur_freq * ACTMON_SAMPLING_PERIOD;
> > +	target_freq = clk_get_rate(tegra->emc_clock) / KHZ;
> > +	dev->avg_count = target_freq * ACTMON_SAMPLING_PERIOD;
> >  	device_writel(dev, dev->avg_count, ACTMON_DEV_INIT_AVG);
> >  
> >  	tegra_devfreq_update_avg_wmark(tegra, dev);
> > -	tegra_devfreq_update_wmark(tegra, dev);
> > +	tegra_devfreq_update_wmark(tegra, dev, target_freq);
> >  
> >  	device_writel(dev, ACTMON_COUNT_WEIGHT,
> > ACTMON_DEV_COUNT_WEIGHT); device_writel(dev,
> > ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS); @@ -469,13
> > +593,13 @@ static int tegra_devfreq_get_dev_status(struct device
> > *dev, struct tegra_devfreq_device *actmon_dev; unsigned long
> > cur_freq; 
> > -	cur_freq = READ_ONCE(tegra->cur_freq);
> > +	cur_freq = clk_get_rate(tegra->emc_clock);
> >  
> >  	/* To be used by the tegra governor */
> >  	stat->private_data = tegra;
> >  
> >  	/* The below are to be used by the other governors */
> > -	stat->current_frequency = cur_freq * KHZ;
> > +	stat->current_frequency = cur_freq;
> >  
> >  	actmon_dev = &tegra->devices[MCALL];
> >  
> > @@ -486,7 +610,7 @@ static int tegra_devfreq_get_dev_status(struct
> > device *dev, stat->busy_time *= 100 / BUS_SATURATION_RATIO;
> >  
> >  	/* Number of cycles in a sampling period */
> > -	stat->total_time = ACTMON_SAMPLING_PERIOD * cur_freq;
> > +	stat->total_time = cur_freq / KHZ * ACTMON_SAMPLING_PERIOD;
> >  
> >  	stat->busy_time = min(stat->busy_time, stat->total_time);
> >  
> > @@ -505,6 +629,7 @@ static int tegra_governor_get_target(struct
> > devfreq *devfreq, struct devfreq_dev_status *stat;
> >  	struct tegra_devfreq *tegra;
> >  	struct tegra_devfreq_device *dev;
> > +	unsigned long dev_target_freq;
> >  	unsigned long target_freq = 0;
> >  	unsigned int i;
> >  	int err;
> > @@ -520,9 +645,9 @@ static int tegra_governor_get_target(struct
> > devfreq *devfreq, for (i = 0; i < ARRAY_SIZE(tegra->devices); i++) {
> >  		dev = &tegra->devices[i];
> >  
> > -		actmon_update_target(tegra, dev);
> > +		dev_target_freq = actmon_update_target(tegra, dev);
> >  
> > -		target_freq = max(target_freq, dev->target_freq);
> > +		target_freq = max(target_freq, dev_target_freq);
> >  	}
> >  
> >  	*freq = target_freq * KHZ;
> > @@ -642,7 +767,6 @@ static int tegra_devfreq_probe(struct
> > platform_device *pdev) return rate;
> >  	}
> >  
> > -	tegra->cur_freq = clk_get_rate(tegra->emc_clock) / KHZ;
> >  	tegra->max_freq = rate / KHZ;
> >  
> >  	for (i = 0; i < ARRAY_SIZE(actmon_device_configs); i++) {
> > @@ -671,7 +795,8 @@ static int tegra_devfreq_probe(struct
> > platform_device *pdev) platform_set_drvdata(pdev, tegra);
> >  
> >  	tegra->rate_change_nb.notifier_call =
> > tegra_actmon_rate_notify_cb;
> > -	err = clk_notifier_register(tegra->emc_clock,
> > &tegra->rate_change_nb);
> > +	err = clk_notifier_register(tegra->emc_clock,
> > +				    &tegra->rate_change_nb);
> >  	if (err) {
> >  		dev_err(&pdev->dev,
> >  			"Failed to register rate change
> > notifier\n"); 
> 
> 
> Maybe, it is possible to merge patch4/patch19/patch20 to one patch.

All these three patches are completely separate changes, thus they
should be kept separate.

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 21/24] PM / devfreq: tegra30: Synchronize average count on target's update
  2019-07-18 10:15   ` Chanwoo Choi
@ 2019-07-19  0:31     ` Dmitry Osipenko
  2019-07-19  1:40       ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19  0:31 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

В Thu, 18 Jul 2019 19:15:54 +0900
Chanwoo Choi <cw00.choi@samsung.com> пишет:

> On 19. 7. 8. 오전 7:33, Dmitry Osipenko wrote:
> > The average count may get out of sync if interrupt was disabled /
> > avoided for a long time due to upper watermark optimization, hence
> > it should be re-synced on each target's update to ensure that
> > watermarks are set up correctly on EMC rate-change notification and
> > that a correct frequency is selected for device.
> > 
> > Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> > ---
> >  drivers/devfreq/tegra30-devfreq.c | 30
> > ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
> > 
> > diff --git a/drivers/devfreq/tegra30-devfreq.c
> > b/drivers/devfreq/tegra30-devfreq.c index
> > 4d582809acb6..8a674fad26be 100644 ---
> > a/drivers/devfreq/tegra30-devfreq.c +++
> > b/drivers/devfreq/tegra30-devfreq.c @@ -466,11 +466,41 @@ static
> > void actmon_isr_device(struct tegra_devfreq *tegra,
> > dev->boost_freq, cpufreq_get(0)); }
> >  
> > +static void tegra_devfreq_sync_avg_count(struct tegra_devfreq
> > *tegra,
> > +					 struct
> > tegra_devfreq_device *dev) +{
> > +	u32 avg_count, avg_freq, old_upper, new_upper;
> > +
> > +	avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
> > +	avg_freq = avg_count / ACTMON_SAMPLING_PERIOD;
> > +
> > +	old_upper = tegra_actmon_upper_freq(tegra, dev->avg_freq);
> > +	new_upper = tegra_actmon_upper_freq(tegra, avg_freq);
> > +
> > +	/* similar to ISR, see comments in actmon_isr_device() */
> > +	if (old_upper != new_upper) {
> > +		dev->avg_freq = avg_freq;
> > +		dev->boost_freq = 0;
> > +	}
> > +}
> > +
> >  static unsigned long actmon_update_target(struct tegra_devfreq
> > *tegra, struct tegra_devfreq_device *dev)
> >  {
> >  	unsigned long target_freq;
> >  
> > +	/*
> > +	 * The avg_count / avg_freq is getting snapshoted on
> > device's
> > +	 * interrupt, but there are cases where actual value need
> > to
> > +	 * be utilized on target's update, like CPUFreq boosting
> > and
> > +	 * overriding the min freq
> > via /sys/class/devfreq/devfreq0/min_freq
> > +	 * because we're optimizing the upper watermark based on
> > the
> > +	 * actual EMC frequency. This means that interrupt may be
> > +	 * inactive for a long time and thus making snapshoted
> > value
> > +	 * outdated.
> > +	 */
> > +	tegra_devfreq_sync_avg_count(tegra, dev);  
> 
> I think that you don't need to add the separate function to calculate
> the 'dev->avg_freq'. It is enough with your detailed comment to add
> this code in this function.

The separate function is indeed not mandatory here, but I'm finding that
it usually makes easier to read and follow the code when it is properly
split up into logical blocks. Don't you agree?

> > +
> >  	target_freq = min(dev->avg_freq + dev->boost_freq,
> > KHZ_MAX); target_freq = tegra_actmon_account_cpu_freq(tegra, dev,
> > target_freq); 
> >   
> 
> And also, is it impossible to squash this patch with patch19/patch20?
> 

It should be possible to squash this patch with #20, but wouldn't
be better to keep changes in the chronological order? It's also better
to keep changes separate simply to aid bisection in case of a problem.


^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 22/24] PM / devfreq: tegra30: Include appropriate header
  2019-07-18  9:58   ` Chanwoo Choi
@ 2019-07-19  0:34     ` Dmitry Osipenko
  0 siblings, 0 replies; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19  0:34 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

В Thu, 18 Jul 2019 18:58:50 +0900
Chanwoo Choi <cw00.choi@samsung.com> пишет:

> On 19. 7. 8. 오전 7:33, Dmitry Osipenko wrote:
> > It's not very correct to include mod_devicetable.h for the OF device
> > drivers and of_device.h should be included instead.
> > 
> > Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> > ---
> >  drivers/devfreq/tegra30-devfreq.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/drivers/devfreq/tegra30-devfreq.c
> > b/drivers/devfreq/tegra30-devfreq.c index
> > 8a674fad26be..19e872a64148 100644 ---
> > a/drivers/devfreq/tegra30-devfreq.c +++
> > b/drivers/devfreq/tegra30-devfreq.c @@ -13,7 +13,7 @@
> >  #include <linux/io.h>
> >  #include <linux/irq.h>
> >  #include <linux/module.h>
> > -#include <linux/mod_devicetable.h>
> > +#include <linux/of_device.h>
> >  #include <linux/platform_device.h>
> >  #include <linux/pm_opp.h>
> >  #include <linux/reset.h>
> >   
> 
> Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
> 
> Is not there other unused header file anymore?
> 

The rest looks good to me.

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 19/24] PM / devfreq: tegra30: Optimize upper consecutive watermark selection
  2019-07-18  9:51   ` Chanwoo Choi
@ 2019-07-19  0:40     ` Dmitry Osipenko
  2019-07-19  1:15       ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19  0:40 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

В Thu, 18 Jul 2019 18:51:02 +0900
Chanwoo Choi <cw00.choi@samsung.com> пишет:

> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> > The memory activity counter may get a bit higher than a watermark
> > which is selected based on OPP that corresponds to a highest EMC
> > rate, in this case watermark is lower than the actual memory
> > activity is and thus results in unwanted "upper" interrupts.
> > 
> > Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> > ---
> >  drivers/devfreq/tegra30-devfreq.c | 13 ++++++++++++-
> >  1 file changed, 12 insertions(+), 1 deletion(-)  
> 
> It seems that you can combine patch19 with patch20.

No, consecutive and average watermarks are different things that have
different purposes. Consecutive are used for boosting, while average
are for significant memory bandwidth changes.

> > 
> > diff --git a/drivers/devfreq/tegra30-devfreq.c
> > b/drivers/devfreq/tegra30-devfreq.c index
> > 8d6bf6e9f1ae..c3cf87231d25 100644 ---
> > a/drivers/devfreq/tegra30-devfreq.c +++
> > b/drivers/devfreq/tegra30-devfreq.c @@ -363,7 +363,18 @@ static
> > void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
> > tegra_actmon_get_lower_upper(tegra, dev, freq - 1, &lower, &upper); 
> >  	delta = do_percent(upper - lower,
> > dev->config->boost_up_threshold);
> > -	device_writel(dev, lower + delta, ACTMON_DEV_UPPER_WMARK);
> > +
> > +	/*
> > +	 * The memory events count could go a bit higher than the
> > maximum
> > +	 * defined by the OPPs, hence make the upper watermark
> > infinitely
> > +	 * high to avoid unnecessary upper interrupts in that case.
> > +	 */
> > +	if (freq == tegra->max_freq)
> > +		upper = ULONG_MAX;
> > +	else
> > +		upper = lower + delta;
> > +
> > +	device_writel(dev, upper, ACTMON_DEV_UPPER_WMARK);
> >  
> >  	/*
> >  	 * Meanwhile the lower mark is based on the average value
> >   
> 
> 


^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 18/24] PM / devfreq: tegra30: Optimize CPUFreq notifier
  2019-07-18  9:48   ` Chanwoo Choi
@ 2019-07-19  0:42     ` Dmitry Osipenko
  2019-07-19  1:09       ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19  0:42 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

В Thu, 18 Jul 2019 18:48:42 +0900
Chanwoo Choi <cw00.choi@samsung.com> пишет:

> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> > When CPU's memory activity is low or memory activity is high such
> > that CPU's frequency contribution to the boosting is not taken into
> > account, then there is no need to schedule devfreq's update. This
> > eliminates unnecessary CPU activity during of idling caused by the
> > scheduled work.
> > 
> > Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> > ---
> >  drivers/devfreq/tegra30-devfreq.c | 73
> > +++++++++++++++++++++++++++---- 1 file changed, 64 insertions(+), 9
> > deletions(-)  
> 
> Patch4 add the 'cpufreq notifier' and this patch optimize the cpufreq
> notifier. I think t hat you can combine two patches.

I'd prefer to keep them separate for a sake of git bisection.

> > 
> > diff --git a/drivers/devfreq/tegra30-devfreq.c
> > b/drivers/devfreq/tegra30-devfreq.c index
> > 43c9c5fbfe91..8d6bf6e9f1ae 100644 ---
> > a/drivers/devfreq/tegra30-devfreq.c +++
> > b/drivers/devfreq/tegra30-devfreq.c @@ -216,10 +216,10 @@ static
> > inline unsigned long do_percent(unsigned long val, unsigned int
> > pct) return val * pct / 100; }
> >  
> > -static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq
> > *tegra) +static unsigned long actmon_cpu_to_emc_rate(struct
> > tegra_devfreq *tegra,
> > +					    unsigned int cpu_freq)
> >  {
> >  	const struct tegra_actmon_emc_ratio *ratio =
> > actmon_emc_ratios;
> > -	unsigned int cpu_freq = cpufreq_get(0);
> >  	unsigned int i;
> >  
> >  	for (i = 0; i < ARRAY_SIZE(actmon_emc_ratios); i++,
> > ratio++) { @@ -239,15 +239,15 @@
> > tegra_actmon_account_cpu_freq(struct tegra_devfreq *tegra, struct
> > tegra_devfreq_device *dev, unsigned long target_freq)
> >  {
> > -	unsigned long static_cpu_emc_freq;
> > +	unsigned long cpu_emc_freq = 0;
> >  
> > -	if (dev->config->avg_dependency_threshold &&
> > -	    dev->config->avg_dependency_threshold < dev->avg_freq)
> > {
> > -		static_cpu_emc_freq =
> > actmon_cpu_to_emc_rate(tegra);
> > -		target_freq = max(target_freq,
> > static_cpu_emc_freq);
> > -	}
> > +	if (!dev->config->avg_dependency_threshold)
> > +		return target_freq;
> >  
> > -	return target_freq;
> > +	if (dev->avg_freq > dev->config->avg_dependency_threshold)
> > +		cpu_emc_freq = actmon_cpu_to_emc_rate(tegra,
> > cpufreq_get(0)); +
> > +	return max(target_freq, cpu_emc_freq);
> >  }
> >  
> >  static unsigned long tegra_actmon_lower_freq(struct tegra_devfreq
> > *tegra, @@ -531,16 +531,71 @@ static void
> > tegra_actmon_delayed_update(struct work_struct *work)
> > mutex_unlock(&tegra->devfreq->lock); }
> >  
> > +static unsigned long
> > +tegra_actmon_cpufreq_contribution(struct tegra_devfreq *tegra,
> > +				  unsigned int cpu_freq)
> > +{
> > +	unsigned long freq, static_cpu_emc_freq;
> > +
> > +	/* check whether CPU's freq is taken into account at all */
> > +	if (tegra->devices[MCCPU].avg_freq <=
> > +	    tegra->devices[MCCPU].config->avg_dependency_threshold)
> > +		return 0;
> > +
> > +	static_cpu_emc_freq = actmon_cpu_to_emc_rate(tegra,
> > cpu_freq); +
> > +	/* compare static CPU-EMC freq with MCALL */
> > +	freq = tegra->devices[MCALL].avg_freq +
> > +	       tegra->devices[MCALL].boost_freq;
> > +
> > +	freq = tegra_actmon_upper_freq(tegra, freq);
> > +
> > +	if (freq == tegra->max_freq || freq >= static_cpu_emc_freq)
> > +		return 0;
> > +
> > +	/* compare static CPU-EMC freq with MCCPU */
> > +	freq = tegra->devices[MCCPU].avg_freq +
> > +	       tegra->devices[MCCPU].boost_freq;
> > +
> > +	freq = tegra_actmon_upper_freq(tegra, freq);
> > +
> > +	if (freq == tegra->max_freq || freq >= static_cpu_emc_freq)
> > +		return 0;
> > +
> > +	return static_cpu_emc_freq;
> > +}
> > +
> >  static int tegra_actmon_cpu_notify_cb(struct notifier_block *nb,
> >  				      unsigned long action, void
> > *ptr) {
> > +	struct cpufreq_freqs *freqs = ptr;
> >  	struct tegra_devfreq *tegra;
> > +	unsigned long old, new;
> >  
> >  	if (action != CPUFREQ_POSTCHANGE)
> >  		return NOTIFY_OK;
> >  
> >  	tegra = container_of(nb, struct tegra_devfreq,
> > cpu_rate_change_nb); 
> > +	/*
> > +	 * Quickly check whether CPU frequency should be taken
> > into account
> > +	 * at all, without blocking CPUFreq's core.
> > +	 */
> > +	if (mutex_trylock(&tegra->devfreq->lock)) {
> > +		old = tegra_actmon_cpufreq_contribution(tegra,
> > freqs->old);
> > +		new = tegra_actmon_cpufreq_contribution(tegra,
> > freqs->new);
> > +		mutex_unlock(&tegra->devfreq->lock);
> > +
> > +		/*
> > +		 * If CPU's frequency shouldn't be taken into
> > account at
> > +		 * the moment, then there is no need to update the
> > devfreq's
> > +		 * state because ISR will re-check CPU's frequency
> > on the
> > +		 * next interrupt.
> > +		 */
> > +		if (old == new)
> > +			return NOTIFY_OK;
> > +	}
> > +
> >  	/*
> >  	 * CPUFreq driver should support
> > CPUFREQ_ASYNC_NOTIFICATION in order
> >  	 * to allow asynchronous notifications. This means we
> > can't block 
> 
> 


^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 17/24] PM / devfreq: tegra30: Use tracepoints for debugging
  2019-07-18  9:47   ` Chanwoo Choi
@ 2019-07-19  0:49     ` Dmitry Osipenko
  2019-07-19  1:01       ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19  0:49 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

В Thu, 18 Jul 2019 18:47:09 +0900
Chanwoo Choi <cw00.choi@samsung.com> пишет:

> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> > Debug messages create too much CPU and memory activity by
> > themselves, so it's difficult to debug lower rates and catch
> > unwanted interrupts that happen rarely. Tracepoints are ideal in
> > that regards because they do not contribute to the sampled date at
> > all. This allowed me to catch few problems which are fixed by the
> > followup patches, without tracepoints it would be much harder to do.
> > 
> > Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> > ---
> >  drivers/devfreq/tegra30-devfreq.c      |  43 +++-------
> >  include/trace/events/tegra30_devfreq.h | 105
> > +++++++++++++++++++++++++ 2 files changed, 117 insertions(+), 31
> > deletions(-) create mode 100644
> > include/trace/events/tegra30_devfreq.h  
> 
> As I knew, 'include/trace/events' don't include the header file
> for only one device driver. Usually, the trace event is provided
> by framework instead of each devic driver.

There are at least trace headers there for the tegra-apbdma,
tegra-host1x, intel-sst and intel-ish devices. I don't think that there
is a strict rule for the trace headers placement.

> > 
> > diff --git a/drivers/devfreq/tegra30-devfreq.c
> > b/drivers/devfreq/tegra30-devfreq.c index
> > 6ebf0f505767..43c9c5fbfe91 100644 ---
> > a/drivers/devfreq/tegra30-devfreq.c +++
> > b/drivers/devfreq/tegra30-devfreq.c @@ -19,6 +19,9 @@
> >  #include <linux/reset.h>
> >  #include <linux/workqueue.h>
> >  
> > +#define CREATE_TRACE_POINTS
> > +#include <trace/events/tegra30_devfreq.h>
> > +
> >  #include "governor.h"
> >  
> >  #define ACTMON_GLB_STATUS
> > 0x0 @@ -283,9 +286,6 @@ static void
> > tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra, unsigned
> > long *lower, unsigned long *upper)
> >  {
> > -	struct device *ddev = tegra->devfreq->dev.parent;
> > -	u32 offset = dev->config->offset;
> > -
> >  	/*
> >  	 * Memory frequencies are guaranteed to have 1MHz
> > granularity
> >  	 * and thus we need this rounding down to get a proper
> > watermarks @@ -298,8 +298,8 @@ static void
> > tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra, *lower =
> > tegra_actmon_lower_freq(tegra, target_freq); *upper =
> > tegra_actmon_upper_freq(tegra, target_freq); 
> > -	dev_dbg(ddev, "%03x: target_freq %lu lower freq %lu upper
> > freq %lu\n",
> > -		offset, target_freq, *lower, *upper);
> > +	trace_device_lower_upper(dev->config->offset, target_freq,
> > +				 *lower, *upper);
> >  
> >  	/*
> >  	 * The upper watermark should take into account CPU's
> > frequency @@ -377,30 +377,13 @@ static void
> > tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
> > device_writel(dev, lower + delta, ACTMON_DEV_LOWER_WMARK); }
> >  
> > -static void actmon_device_debug(struct tegra_devfreq *tegra,
> > -				struct tegra_devfreq_device *dev,
> > -				const char *prefix)
> > -{
> > -	dev_dbg(tegra->devfreq->dev.parent,
> > -		"%03x: %s: 0x%08x 0x%08x a %u %u %u c %u %u %u b
> > %lu cpu %u\n",
> > -		dev->config->offset, prefix,
> > -		device_readl(dev, ACTMON_DEV_INTR_STATUS),
> > -		device_readl(dev, ACTMON_DEV_CTRL),
> > -		device_readl(dev, ACTMON_DEV_AVG_COUNT),
> > -		device_readl(dev, ACTMON_DEV_AVG_LOWER_WMARK),
> > -		device_readl(dev, ACTMON_DEV_AVG_UPPER_WMARK),
> > -		device_readl(dev, ACTMON_DEV_COUNT),
> > -		device_readl(dev, ACTMON_DEV_LOWER_WMARK),
> > -		device_readl(dev, ACTMON_DEV_UPPER_WMARK),
> > -		dev->boost_freq, cpufreq_get(0));
> > -}
> > -
> >  static void actmon_isr_device(struct tegra_devfreq *tegra,
> >  			      struct tegra_devfreq_device *dev)
> >  {
> >  	u32 intr_status, dev_ctrl, avg_intr_mask, avg_count;
> >  
> > -	actmon_device_debug(tegra, dev, "isr+");
> > +	trace_device_isr_enter(tegra->regs, dev->config->offset,
> > +			       dev->boost_freq, cpufreq_get(0));
> >  
> >  	intr_status = device_readl(dev, ACTMON_DEV_INTR_STATUS);
> >  	avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
> > @@ -455,7 +438,8 @@ static void actmon_isr_device(struct
> > tegra_devfreq *tegra, device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
> >  	device_writel(dev, ACTMON_INTR_STATUS_CLEAR,
> > ACTMON_DEV_INTR_STATUS); 
> > -	actmon_device_debug(tegra, dev, "isr-");
> > +	trace_device_isr_exit(tegra->regs, dev->config->offset,
> > +			      dev->boost_freq, cpufreq_get(0));
> >  }
> >  
> >  static unsigned long actmon_update_target(struct tegra_devfreq
> > *tegra, @@ -749,7 +733,6 @@ static struct devfreq_dev_profile
> > tegra_devfreq_profile = { static int
> > tegra_governor_get_target(struct devfreq *devfreq, unsigned long
> > *freq) {
> > -	struct device *ddev = devfreq->dev.parent;
> >  	struct devfreq_dev_status *stat;
> >  	struct tegra_devfreq *tegra;
> >  	struct tegra_devfreq_device *dev;
> > @@ -770,13 +753,11 @@ static int tegra_governor_get_target(struct
> > devfreq *devfreq, dev = &tegra->devices[i];
> >  
> >  		dev_target_freq = actmon_update_target(tegra, dev);
> > -
> >  		target_freq = max(target_freq, dev_target_freq);
> >  
> > -		dev_dbg(ddev, "%03x: upd: dev_target_freq %lu\n",
> > -			dev->config->offset, dev_target_freq);
> > -
> > -		actmon_device_debug(tegra, dev, "upd");
> > +		trace_device_target_freq(dev->config->offset,
> > dev_target_freq);
> > +		trace_device_target_update(tegra->regs,
> > dev->config->offset,
> > +					   dev->boost_freq,
> > cpufreq_get(0)); }
> >  
> >  	*freq = target_freq;
> > diff --git a/include/trace/events/tegra30_devfreq.h
> > b/include/trace/events/tegra30_devfreq.h new file mode 100644
> > index 000000000000..8f264a489daf
> > --- /dev/null
> > +++ b/include/trace/events/tegra30_devfreq.h
> > @@ -0,0 +1,105 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +#undef TRACE_SYSTEM
> > +#define TRACE_SYSTEM tegra30_devfreq
> > +
> > +#if !defined(_TRACE_TEGRA30_DEVFREQ_H) ||
> > defined(TRACE_HEADER_MULTI_READ) +#define _TRACE_TEGRA30_DEVFREQ_H
> > +
> > +#include <linux/io.h>
> > +#include <linux/tracepoint.h>
> > +#include <linux/types.h>
> > +
> > +DECLARE_EVENT_CLASS(device_state,
> > +	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32
> > cpufreq),
> > +	TP_ARGS(base, offset, boost, cpufreq),
> > +	TP_STRUCT__entry(
> > +		__field(u32, offset)
> > +		__field(u32, intr_status)
> > +		__field(u32, ctrl)
> > +		__field(u32, avg_count)
> > +		__field(u32, avg_lower)
> > +		__field(u32, avg_upper)
> > +		__field(u32, count)
> > +		__field(u32, lower)
> > +		__field(u32, upper)
> > +		__field(u32, boost_freq)
> > +		__field(u32, cpu_freq)
> > +	),
> > +	TP_fast_assign(
> > +		__entry->offset		= offset;
> > +		__entry->intr_status	= readl_relaxed(base +
> > offset + 0x24);
> > +		__entry->ctrl		= readl_relaxed(base
> > + offset + 0x0);
> > +		__entry->avg_count	= readl_relaxed(base +
> > offset + 0x20);
> > +		__entry->avg_lower	= readl_relaxed(base +
> > offset + 0x14);
> > +		__entry->avg_upper	= readl_relaxed(base +
> > offset + 0x10);
> > +		__entry->count		= readl_relaxed(base
> > + offset + 0x1c);
> > +		__entry->lower		= readl_relaxed(base
> > + offset + 0x8);
> > +		__entry->upper		= readl_relaxed(base
> > + offset + 0x4);
> > +		__entry->boost_freq	= boost;
> > +		__entry->cpu_freq	= cpufreq;
> > +	),
> > +	TP_printk("%03x: intr 0x%08x ctrl 0x%08x avg %010u %010u
> > %010u cnt %010u %010u %010u boost %010u cpu %u",
> > +		__entry->offset,
> > +		__entry->intr_status,
> > +		__entry->ctrl,
> > +		__entry->avg_count,
> > +		__entry->avg_lower,
> > +		__entry->avg_upper,
> > +		__entry->count,
> > +		__entry->lower,
> > +		__entry->upper,
> > +		__entry->boost_freq,
> > +		__entry->cpu_freq)
> > +);
> > +
> > +DEFINE_EVENT(device_state, device_isr_enter,
> > +	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32
> > cpufreq),
> > +	TP_ARGS(base, offset, boost, cpufreq));
> > +
> > +DEFINE_EVENT(device_state, device_isr_exit,
> > +	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32
> > cpufreq),
> > +	TP_ARGS(base, offset, boost, cpufreq));
> > +
> > +DEFINE_EVENT(device_state, device_target_update,
> > +	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32
> > cpufreq),
> > +	TP_ARGS(base, offset, boost, cpufreq));
> > +
> > +TRACE_EVENT(device_lower_upper,
> > +	TP_PROTO(u32 offset, u32 target, u32 lower, u32 upper),
> > +	TP_ARGS(offset, target, lower, upper),
> > +	TP_STRUCT__entry(
> > +		__field(u32, offset)
> > +		__field(u32, target)
> > +		__field(u32, lower)
> > +		__field(u32, upper)
> > +	),
> > +	TP_fast_assign(
> > +		__entry->offset = offset;
> > +		__entry->target = target;
> > +		__entry->lower = lower;
> > +		__entry->upper = upper;
> > +	),
> > +	TP_printk("%03x: freq %010u lower freq %010u upper freq
> > %010u",
> > +		__entry->offset,
> > +		__entry->target,
> > +		__entry->lower,
> > +		__entry->upper)
> > +);
> > +
> > +TRACE_EVENT(device_target_freq,
> > +	TP_PROTO(u32 offset, u32 target),
> > +	TP_ARGS(offset, target),
> > +	TP_STRUCT__entry(
> > +		__field(u32, offset)
> > +		__field(u32, target)
> > +	),
> > +	TP_fast_assign(
> > +		__entry->offset = offset;
> > +		__entry->target = target;
> > +	),
> > +	TP_printk("%03x: freq %010u", __entry->offset,
> > __entry->target) +);
> > +#endif /* _TRACE_TEGRA30_DEVFREQ_H */
> > +
> > +/* This part must be outside protection */
> > +#include <trace/define_trace.h>
> >   
> 
> 


^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 17/24] PM / devfreq: tegra30: Use tracepoints for debugging
  2019-07-19  0:49     ` Dmitry Osipenko
@ 2019-07-19  1:01       ` Chanwoo Choi
  2019-07-19  1:50         ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-19  1:01 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 19. 오전 9:49, Dmitry Osipenko wrote:
> В Thu, 18 Jul 2019 18:47:09 +0900
> Chanwoo Choi <cw00.choi@samsung.com> пишет:
> 
>> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>>> Debug messages create too much CPU and memory activity by
>>> themselves, so it's difficult to debug lower rates and catch
>>> unwanted interrupts that happen rarely. Tracepoints are ideal in
>>> that regards because they do not contribute to the sampled date at
>>> all. This allowed me to catch few problems which are fixed by the
>>> followup patches, without tracepoints it would be much harder to do.
>>>
>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>> ---
>>>  drivers/devfreq/tegra30-devfreq.c      |  43 +++-------
>>>  include/trace/events/tegra30_devfreq.h | 105
>>> +++++++++++++++++++++++++ 2 files changed, 117 insertions(+), 31
>>> deletions(-) create mode 100644
>>> include/trace/events/tegra30_devfreq.h  
>>
>> As I knew, 'include/trace/events' don't include the header file
>> for only one device driver. Usually, the trace event is provided
>> by framework instead of each devic driver.
> 
> There are at least trace headers there for the tegra-apbdma,
> tegra-host1x, intel-sst and intel-ish devices. I don't think that there
> is a strict rule for the trace headers placement.

OK.

But, As I already replied on patch4, if you want to show the register dump,
you better to add the debugfs feature to devfreq framework for showing
the register dump instead of printing the register dump with debug level
and this trace event. 

As I said, just register dump is not useful for all developers.
Almost developer cannot understand the meaning of debug log for register dump.

Also, it is not proper way that front patch adds the some code
and then later patch removes the additional code in the same series.
Before sending the patches, you can renew them.

> 
>>>
>>> diff --git a/drivers/devfreq/tegra30-devfreq.c
>>> b/drivers/devfreq/tegra30-devfreq.c index
>>> 6ebf0f505767..43c9c5fbfe91 100644 ---
>>> a/drivers/devfreq/tegra30-devfreq.c +++
>>> b/drivers/devfreq/tegra30-devfreq.c @@ -19,6 +19,9 @@
>>>  #include <linux/reset.h>
>>>  #include <linux/workqueue.h>
>>>  
>>> +#define CREATE_TRACE_POINTS
>>> +#include <trace/events/tegra30_devfreq.h>
>>> +
>>>  #include "governor.h"
>>>  
>>>  #define ACTMON_GLB_STATUS
>>> 0x0 @@ -283,9 +286,6 @@ static void
>>> tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra, unsigned
>>> long *lower, unsigned long *upper)
>>>  {
>>> -	struct device *ddev = tegra->devfreq->dev.parent;
>>> -	u32 offset = dev->config->offset;
>>> -
>>>  	/*
>>>  	 * Memory frequencies are guaranteed to have 1MHz
>>> granularity
>>>  	 * and thus we need this rounding down to get a proper
>>> watermarks @@ -298,8 +298,8 @@ static void
>>> tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra, *lower =
>>> tegra_actmon_lower_freq(tegra, target_freq); *upper =
>>> tegra_actmon_upper_freq(tegra, target_freq); 
>>> -	dev_dbg(ddev, "%03x: target_freq %lu lower freq %lu upper
>>> freq %lu\n",
>>> -		offset, target_freq, *lower, *upper);
>>> +	trace_device_lower_upper(dev->config->offset, target_freq,
>>> +				 *lower, *upper);
>>>  
>>>  	/*
>>>  	 * The upper watermark should take into account CPU's
>>> frequency @@ -377,30 +377,13 @@ static void
>>> tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
>>> device_writel(dev, lower + delta, ACTMON_DEV_LOWER_WMARK); }
>>>  
>>> -static void actmon_device_debug(struct tegra_devfreq *tegra,
>>> -				struct tegra_devfreq_device *dev,
>>> -				const char *prefix)
>>> -{
>>> -	dev_dbg(tegra->devfreq->dev.parent,
>>> -		"%03x: %s: 0x%08x 0x%08x a %u %u %u c %u %u %u b
>>> %lu cpu %u\n",
>>> -		dev->config->offset, prefix,
>>> -		device_readl(dev, ACTMON_DEV_INTR_STATUS),
>>> -		device_readl(dev, ACTMON_DEV_CTRL),
>>> -		device_readl(dev, ACTMON_DEV_AVG_COUNT),
>>> -		device_readl(dev, ACTMON_DEV_AVG_LOWER_WMARK),
>>> -		device_readl(dev, ACTMON_DEV_AVG_UPPER_WMARK),
>>> -		device_readl(dev, ACTMON_DEV_COUNT),
>>> -		device_readl(dev, ACTMON_DEV_LOWER_WMARK),
>>> -		device_readl(dev, ACTMON_DEV_UPPER_WMARK),
>>> -		dev->boost_freq, cpufreq_get(0));
>>> -}
>>> -
>>>  static void actmon_isr_device(struct tegra_devfreq *tegra,
>>>  			      struct tegra_devfreq_device *dev)
>>>  {
>>>  	u32 intr_status, dev_ctrl, avg_intr_mask, avg_count;
>>>  
>>> -	actmon_device_debug(tegra, dev, "isr+");
>>> +	trace_device_isr_enter(tegra->regs, dev->config->offset,
>>> +			       dev->boost_freq, cpufreq_get(0));
>>>  
>>>  	intr_status = device_readl(dev, ACTMON_DEV_INTR_STATUS);
>>>  	avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
>>> @@ -455,7 +438,8 @@ static void actmon_isr_device(struct
>>> tegra_devfreq *tegra, device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
>>>  	device_writel(dev, ACTMON_INTR_STATUS_CLEAR,
>>> ACTMON_DEV_INTR_STATUS); 
>>> -	actmon_device_debug(tegra, dev, "isr-");
>>> +	trace_device_isr_exit(tegra->regs, dev->config->offset,
>>> +			      dev->boost_freq, cpufreq_get(0));
>>>  }
>>>  
>>>  static unsigned long actmon_update_target(struct tegra_devfreq
>>> *tegra, @@ -749,7 +733,6 @@ static struct devfreq_dev_profile
>>> tegra_devfreq_profile = { static int
>>> tegra_governor_get_target(struct devfreq *devfreq, unsigned long
>>> *freq) {
>>> -	struct device *ddev = devfreq->dev.parent;
>>>  	struct devfreq_dev_status *stat;
>>>  	struct tegra_devfreq *tegra;
>>>  	struct tegra_devfreq_device *dev;
>>> @@ -770,13 +753,11 @@ static int tegra_governor_get_target(struct
>>> devfreq *devfreq, dev = &tegra->devices[i];
>>>  
>>>  		dev_target_freq = actmon_update_target(tegra, dev);
>>> -
>>>  		target_freq = max(target_freq, dev_target_freq);
>>>  
>>> -		dev_dbg(ddev, "%03x: upd: dev_target_freq %lu\n",
>>> -			dev->config->offset, dev_target_freq);
>>> -
>>> -		actmon_device_debug(tegra, dev, "upd");
>>> +		trace_device_target_freq(dev->config->offset,
>>> dev_target_freq);
>>> +		trace_device_target_update(tegra->regs,
>>> dev->config->offset,
>>> +					   dev->boost_freq,
>>> cpufreq_get(0)); }
>>>  
>>>  	*freq = target_freq;
>>> diff --git a/include/trace/events/tegra30_devfreq.h
>>> b/include/trace/events/tegra30_devfreq.h new file mode 100644
>>> index 000000000000..8f264a489daf
>>> --- /dev/null
>>> +++ b/include/trace/events/tegra30_devfreq.h
>>> @@ -0,0 +1,105 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +#undef TRACE_SYSTEM
>>> +#define TRACE_SYSTEM tegra30_devfreq
>>> +
>>> +#if !defined(_TRACE_TEGRA30_DEVFREQ_H) ||
>>> defined(TRACE_HEADER_MULTI_READ) +#define _TRACE_TEGRA30_DEVFREQ_H
>>> +
>>> +#include <linux/io.h>
>>> +#include <linux/tracepoint.h>
>>> +#include <linux/types.h>
>>> +
>>> +DECLARE_EVENT_CLASS(device_state,
>>> +	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32
>>> cpufreq),
>>> +	TP_ARGS(base, offset, boost, cpufreq),
>>> +	TP_STRUCT__entry(
>>> +		__field(u32, offset)
>>> +		__field(u32, intr_status)
>>> +		__field(u32, ctrl)
>>> +		__field(u32, avg_count)
>>> +		__field(u32, avg_lower)
>>> +		__field(u32, avg_upper)
>>> +		__field(u32, count)
>>> +		__field(u32, lower)
>>> +		__field(u32, upper)
>>> +		__field(u32, boost_freq)
>>> +		__field(u32, cpu_freq)
>>> +	),
>>> +	TP_fast_assign(
>>> +		__entry->offset		= offset;
>>> +		__entry->intr_status	= readl_relaxed(base +
>>> offset + 0x24);
>>> +		__entry->ctrl		= readl_relaxed(base
>>> + offset + 0x0);
>>> +		__entry->avg_count	= readl_relaxed(base +
>>> offset + 0x20);
>>> +		__entry->avg_lower	= readl_relaxed(base +
>>> offset + 0x14);
>>> +		__entry->avg_upper	= readl_relaxed(base +
>>> offset + 0x10);
>>> +		__entry->count		= readl_relaxed(base
>>> + offset + 0x1c);
>>> +		__entry->lower		= readl_relaxed(base
>>> + offset + 0x8);
>>> +		__entry->upper		= readl_relaxed(base
>>> + offset + 0x4);
>>> +		__entry->boost_freq	= boost;
>>> +		__entry->cpu_freq	= cpufreq;
>>> +	),
>>> +	TP_printk("%03x: intr 0x%08x ctrl 0x%08x avg %010u %010u
>>> %010u cnt %010u %010u %010u boost %010u cpu %u",
>>> +		__entry->offset,
>>> +		__entry->intr_status,
>>> +		__entry->ctrl,
>>> +		__entry->avg_count,
>>> +		__entry->avg_lower,
>>> +		__entry->avg_upper,
>>> +		__entry->count,
>>> +		__entry->lower,
>>> +		__entry->upper,
>>> +		__entry->boost_freq,
>>> +		__entry->cpu_freq)
>>> +);
>>> +
>>> +DEFINE_EVENT(device_state, device_isr_enter,
>>> +	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32
>>> cpufreq),
>>> +	TP_ARGS(base, offset, boost, cpufreq));
>>> +
>>> +DEFINE_EVENT(device_state, device_isr_exit,
>>> +	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32
>>> cpufreq),
>>> +	TP_ARGS(base, offset, boost, cpufreq));
>>> +
>>> +DEFINE_EVENT(device_state, device_target_update,
>>> +	TP_PROTO(void __iomem *base, u32 offset, u32 boost, u32
>>> cpufreq),
>>> +	TP_ARGS(base, offset, boost, cpufreq));
>>> +
>>> +TRACE_EVENT(device_lower_upper,
>>> +	TP_PROTO(u32 offset, u32 target, u32 lower, u32 upper),
>>> +	TP_ARGS(offset, target, lower, upper),
>>> +	TP_STRUCT__entry(
>>> +		__field(u32, offset)
>>> +		__field(u32, target)
>>> +		__field(u32, lower)
>>> +		__field(u32, upper)
>>> +	),
>>> +	TP_fast_assign(
>>> +		__entry->offset = offset;
>>> +		__entry->target = target;
>>> +		__entry->lower = lower;
>>> +		__entry->upper = upper;
>>> +	),
>>> +	TP_printk("%03x: freq %010u lower freq %010u upper freq
>>> %010u",
>>> +		__entry->offset,
>>> +		__entry->target,
>>> +		__entry->lower,
>>> +		__entry->upper)
>>> +);
>>> +
>>> +TRACE_EVENT(device_target_freq,
>>> +	TP_PROTO(u32 offset, u32 target),
>>> +	TP_ARGS(offset, target),
>>> +	TP_STRUCT__entry(
>>> +		__field(u32, offset)
>>> +		__field(u32, target)
>>> +	),
>>> +	TP_fast_assign(
>>> +		__entry->offset = offset;
>>> +		__entry->target = target;
>>> +	),
>>> +	TP_printk("%03x: freq %010u", __entry->offset,
>>> __entry->target) +);
>>> +#endif /* _TRACE_TEGRA30_DEVFREQ_H */
>>> +
>>> +/* This part must be outside protection */
>>> +#include <trace/define_trace.h>
>>>   
>>
>>
> 
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 18/24] PM / devfreq: tegra30: Optimize CPUFreq notifier
  2019-07-19  0:42     ` Dmitry Osipenko
@ 2019-07-19  1:09       ` Chanwoo Choi
  0 siblings, 0 replies; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-19  1:09 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 19. 오전 9:42, Dmitry Osipenko wrote:
> В Thu, 18 Jul 2019 18:48:42 +0900
> Chanwoo Choi <cw00.choi@samsung.com> пишет:
> 
>> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>>> When CPU's memory activity is low or memory activity is high such
>>> that CPU's frequency contribution to the boosting is not taken into
>>> account, then there is no need to schedule devfreq's update. This
>>> eliminates unnecessary CPU activity during of idling caused by the
>>> scheduled work.
>>>
>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>> ---
>>>  drivers/devfreq/tegra30-devfreq.c | 73
>>> +++++++++++++++++++++++++++---- 1 file changed, 64 insertions(+), 9
>>> deletions(-)  
>>
>> Patch4 add the 'cpufreq notifier' and this patch optimize the cpufreq
>> notifier. I think t hat you can combine two patches.
> 
> I'd prefer to keep them separate for a sake of git bisection.

Sorry, patch7 instead of patch4.

Patch7 made the 'tegra_actmon_cpu_notify_cb()' function
and this patch makes 'tegra_actmon_cpufreq_contribution' function
which is only called in the 'tegra_actmon_cpu_notify_cb()' function.

It is enough to make them as the only one patch related to
the cpu notifier. As I replied on patch17, you can merge
the patch if the patches has some relationship in this patchset.

> 
>>>
>>> diff --git a/drivers/devfreq/tegra30-devfreq.c
>>> b/drivers/devfreq/tegra30-devfreq.c index
>>> 43c9c5fbfe91..8d6bf6e9f1ae 100644 ---
>>> a/drivers/devfreq/tegra30-devfreq.c +++
>>> b/drivers/devfreq/tegra30-devfreq.c @@ -216,10 +216,10 @@ static
>>> inline unsigned long do_percent(unsigned long val, unsigned int
>>> pct) return val * pct / 100; }
>>>  
>>> -static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq
>>> *tegra) +static unsigned long actmon_cpu_to_emc_rate(struct
>>> tegra_devfreq *tegra,
>>> +					    unsigned int cpu_freq)
>>>  {
>>>  	const struct tegra_actmon_emc_ratio *ratio =
>>> actmon_emc_ratios;
>>> -	unsigned int cpu_freq = cpufreq_get(0);
>>>  	unsigned int i;
>>>  
>>>  	for (i = 0; i < ARRAY_SIZE(actmon_emc_ratios); i++,
>>> ratio++) { @@ -239,15 +239,15 @@
>>> tegra_actmon_account_cpu_freq(struct tegra_devfreq *tegra, struct
>>> tegra_devfreq_device *dev, unsigned long target_freq)
>>>  {
>>> -	unsigned long static_cpu_emc_freq;
>>> +	unsigned long cpu_emc_freq = 0;
>>>  
>>> -	if (dev->config->avg_dependency_threshold &&
>>> -	    dev->config->avg_dependency_threshold < dev->avg_freq)
>>> {
>>> -		static_cpu_emc_freq =
>>> actmon_cpu_to_emc_rate(tegra);
>>> -		target_freq = max(target_freq,
>>> static_cpu_emc_freq);
>>> -	}
>>> +	if (!dev->config->avg_dependency_threshold)
>>> +		return target_freq;
>>>  
>>> -	return target_freq;
>>> +	if (dev->avg_freq > dev->config->avg_dependency_threshold)
>>> +		cpu_emc_freq = actmon_cpu_to_emc_rate(tegra,
>>> cpufreq_get(0)); +
>>> +	return max(target_freq, cpu_emc_freq);
>>>  }
>>>  
>>>  static unsigned long tegra_actmon_lower_freq(struct tegra_devfreq
>>> *tegra, @@ -531,16 +531,71 @@ static void
>>> tegra_actmon_delayed_update(struct work_struct *work)
>>> mutex_unlock(&tegra->devfreq->lock); }
>>>  
>>> +static unsigned long
>>> +tegra_actmon_cpufreq_contribution(struct tegra_devfreq *tegra,
>>> +				  unsigned int cpu_freq)
>>> +{
>>> +	unsigned long freq, static_cpu_emc_freq;
>>> +
>>> +	/* check whether CPU's freq is taken into account at all */
>>> +	if (tegra->devices[MCCPU].avg_freq <=
>>> +	    tegra->devices[MCCPU].config->avg_dependency_threshold)
>>> +		return 0;
>>> +
>>> +	static_cpu_emc_freq = actmon_cpu_to_emc_rate(tegra,
>>> cpu_freq); +
>>> +	/* compare static CPU-EMC freq with MCALL */
>>> +	freq = tegra->devices[MCALL].avg_freq +
>>> +	       tegra->devices[MCALL].boost_freq;
>>> +
>>> +	freq = tegra_actmon_upper_freq(tegra, freq);
>>> +
>>> +	if (freq == tegra->max_freq || freq >= static_cpu_emc_freq)
>>> +		return 0;
>>> +
>>> +	/* compare static CPU-EMC freq with MCCPU */
>>> +	freq = tegra->devices[MCCPU].avg_freq +
>>> +	       tegra->devices[MCCPU].boost_freq;
>>> +
>>> +	freq = tegra_actmon_upper_freq(tegra, freq);
>>> +
>>> +	if (freq == tegra->max_freq || freq >= static_cpu_emc_freq)
>>> +		return 0;
>>> +
>>> +	return static_cpu_emc_freq;
>>> +}
>>> +
>>>  static int tegra_actmon_cpu_notify_cb(struct notifier_block *nb,
>>>  				      unsigned long action, void
>>> *ptr) {
>>> +	struct cpufreq_freqs *freqs = ptr;
>>>  	struct tegra_devfreq *tegra;
>>> +	unsigned long old, new;
>>>  
>>>  	if (action != CPUFREQ_POSTCHANGE)
>>>  		return NOTIFY_OK;
>>>  
>>>  	tegra = container_of(nb, struct tegra_devfreq,
>>> cpu_rate_change_nb); 
>>> +	/*
>>> +	 * Quickly check whether CPU frequency should be taken
>>> into account
>>> +	 * at all, without blocking CPUFreq's core.
>>> +	 */
>>> +	if (mutex_trylock(&tegra->devfreq->lock)) {
>>> +		old = tegra_actmon_cpufreq_contribution(tegra,
>>> freqs->old);
>>> +		new = tegra_actmon_cpufreq_contribution(tegra,
>>> freqs->new);
>>> +		mutex_unlock(&tegra->devfreq->lock);
>>> +
>>> +		/*
>>> +		 * If CPU's frequency shouldn't be taken into
>>> account at
>>> +		 * the moment, then there is no need to update the
>>> devfreq's
>>> +		 * state because ISR will re-check CPU's frequency
>>> on the
>>> +		 * next interrupt.
>>> +		 */
>>> +		if (old == new)
>>> +			return NOTIFY_OK;
>>> +	}
>>> +
>>>  	/*
>>>  	 * CPUFreq driver should support
>>> CPUFREQ_ASYNC_NOTIFICATION in order
>>>  	 * to allow asynchronous notifications. This means we
>>> can't block 
>>
>>
> 
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 11/24] PM / devfreq: tegra30: Add debug messages
  2019-07-18  9:07           ` Chanwoo Choi
@ 2019-07-19  1:13             ` Dmitry Osipenko
  2019-07-19  1:22               ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19  1:13 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

В Thu, 18 Jul 2019 18:07:05 +0900
Chanwoo Choi <cw00.choi@samsung.com> пишет:

> On 19. 7. 18. 오전 12:46, Dmitry Osipenko wrote:
> > 17.07.2019 9:45, Chanwoo Choi пишет:  
> >> On 19. 7. 16. 오후 10:26, Dmitry Osipenko wrote:  
> >>> 16.07.2019 15:23, Chanwoo Choi пишет:  
> >>>> Hi Dmitry,
> >>>>
> >>>> Usually, the kernel log print for all users
> >>>> such as changing the frequency, fail or success.
> >>>>
> >>>> But, if the log just show the register dump,
> >>>> it is not useful for all users. It is just used
> >>>> for only specific developer.
> >>>>
> >>>> I recommend that you better to add more exception handling
> >>>> code on many points instead of just showing the register dump.  
> >>>
> >>> The debug messages are not users, but for developers. Yes, I
> >>> primarily made the debugging to be useful for myself and will be
> >>> happy to change the way debugging is done if there will be any
> >>> other active developer for this driver. The registers dump is
> >>> more than enough in order to understand what's going on, I don't
> >>> see any real need to change anything here for now.  
> >>
> >> Basically, we have to develop code and add the log for anyone.
> >> As you commented, even if there are no other developer, we never
> >> guarantee this assumption forever. And also, if added debug message
> >> for only you, you can add them when testing it temporarily.
> >>
> >> If you want to add the just register dump log for you,
> >> I can't agree. Once again, I hope that anyone understand
> >> the meaning of debug message as much possible as.
> >>  
> > 
> > The registers dump should be good for everyone because it's a
> > self-explanatory information for anyone who is familiar with the
> > hardware. I don't think there is a need for anything else than what
> > is proposed in this patch, at least for now. I also simply don't
> > see any other better way to debug the state of this particular
> > hardware, again this logging is for the driver developers and not
> > for users.
> > 
> > Initially, I was temporarily adding the debug messages. Now they are
> > pretty much mandatory for verifying that driver is working
> > properly. And of course the debugging messages got into the shape
> > of this patch after several iterations of refinements. So again, I
> > suppose that this should be good enough for everyone who is
> > familiar with the hardware. And of course I'm open to the
> > constructive suggestions, the debugging aid is not an ABI and could
> > be changed/improved at any time.
> > 
> > You're suggesting to break down the debugging into several smaller
> > pieces, but I'm finding that as not a constructive suggestion
> > because the information about the full hardware state is actually
> > necessary for the productive debugging.
> > 
> >   
> 
> Sorry for that as I saie, I cannot agree this patch. In my case,
> I don't understand what is meaning of register dump of this patch.
> I knew that just register dump are useful for real developer.

It's not only a registers dump, as you may see there is also a dump of
other properties like boosting value, OPPs selection and etc.

It looks to me that you're also missing important detail that debug
messages are compiled out unless DEBUG is defined for the drivers
build. So in order to get the debug message a user shall explicitly add
#define DEBUG macro to the code or enable debug messages globally in
the kernel's config. There is also an option for dynamic debug messages
in the kernel, but it doesn't matter now because all these messages are
turned into tracepoints later in the patch #17.

> If you want to show the register dump, you better to add some feature
> with debugfs for devfreq framework in order to read the register dump.
> As I knew, sound framework (alsa) has the similar feature for checking
> the register dump.
> 

The intent was to have an option for dynamic debugging of the driver and
initially debug messages were good enough, but then it became not enough
and hence the debug messages were turned into tracepoints in the patch
#17. Would it be acceptable to squash this patch and #17?


^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 19/24] PM / devfreq: tegra30: Optimize upper consecutive watermark selection
  2019-07-19  0:40     ` Dmitry Osipenko
@ 2019-07-19  1:15       ` Chanwoo Choi
  2019-07-19  1:17         ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-19  1:15 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 19. 오전 9:40, Dmitry Osipenko wrote:
> В Thu, 18 Jul 2019 18:51:02 +0900
> Chanwoo Choi <cw00.choi@samsung.com> пишет:
> 
>> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>>> The memory activity counter may get a bit higher than a watermark
>>> which is selected based on OPP that corresponds to a highest EMC
>>> rate, in this case watermark is lower than the actual memory
>>> activity is and thus results in unwanted "upper" interrupts.
>>>
>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>> ---
>>>  drivers/devfreq/tegra30-devfreq.c | 13 ++++++++++++-
>>>  1 file changed, 12 insertions(+), 1 deletion(-)  
>>
>> It seems that you can combine patch19 with patch20.
> 
> No, consecutive and average watermarks are different things that have
> different purposes. Consecutive are used for boosting, while average
> are for significant memory bandwidth changes.
> 
>>>
>>> diff --git a/drivers/devfreq/tegra30-devfreq.c
>>> b/drivers/devfreq/tegra30-devfreq.c index
>>> 8d6bf6e9f1ae..c3cf87231d25 100644 ---
>>> a/drivers/devfreq/tegra30-devfreq.c +++
>>> b/drivers/devfreq/tegra30-devfreq.c @@ -363,7 +363,18 @@ static
>>> void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
>>> tegra_actmon_get_lower_upper(tegra, dev, freq - 1, &lower, &upper); 
>>>  	delta = do_percent(upper - lower,
>>> dev->config->boost_up_threshold);
>>> -	device_writel(dev, lower + delta, ACTMON_DEV_UPPER_WMARK);

This line was added on patch5 and then this line is removed on this patch.
It is wrong method to make the patch in the same patchset.

It is enough to reduce the inefficient add/remove code in the same patchset.

Have to merge this patch to patch5.

>>> +
>>> +	/*
>>> +	 * The memory events count could go a bit higher than the
>>> maximum
>>> +	 * defined by the OPPs, hence make the upper watermark
>>> infinitely
>>> +	 * high to avoid unnecessary upper interrupts in that case.
>>> +	 */
>>> +	if (freq == tegra->max_freq)
>>> +		upper = ULONG_MAX;
>>> +	else
>>> +		upper = lower + delta;
>>> +
>>> +	device_writel(dev, upper, ACTMON_DEV_UPPER_WMARK);
>>>  
>>>  	/*
>>>  	 * Meanwhile the lower mark is based on the average value
>>>   
>>
>>
> 
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 19/24] PM / devfreq: tegra30: Optimize upper consecutive watermark selection
  2019-07-19  1:15       ` Chanwoo Choi
@ 2019-07-19  1:17         ` Chanwoo Choi
  0 siblings, 0 replies; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-19  1:17 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 19. 오전 10:15, Chanwoo Choi wrote:
> On 19. 7. 19. 오전 9:40, Dmitry Osipenko wrote:
>> В Thu, 18 Jul 2019 18:51:02 +0900
>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>
>>> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>>>> The memory activity counter may get a bit higher than a watermark
>>>> which is selected based on OPP that corresponds to a highest EMC
>>>> rate, in this case watermark is lower than the actual memory
>>>> activity is and thus results in unwanted "upper" interrupts.
>>>>
>>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>>> ---
>>>>  drivers/devfreq/tegra30-devfreq.c | 13 ++++++++++++-
>>>>  1 file changed, 12 insertions(+), 1 deletion(-)  
>>>
>>> It seems that you can combine patch19 with patch20.
>>
>> No, consecutive and average watermarks are different things that have
>> different purposes. Consecutive are used for boosting, while average
>> are for significant memory bandwidth changes.
>>
>>>>
>>>> diff --git a/drivers/devfreq/tegra30-devfreq.c
>>>> b/drivers/devfreq/tegra30-devfreq.c index
>>>> 8d6bf6e9f1ae..c3cf87231d25 100644 ---
>>>> a/drivers/devfreq/tegra30-devfreq.c +++
>>>> b/drivers/devfreq/tegra30-devfreq.c @@ -363,7 +363,18 @@ static
>>>> void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
>>>> tegra_actmon_get_lower_upper(tegra, dev, freq - 1, &lower, &upper); 
>>>>  	delta = do_percent(upper - lower,
>>>> dev->config->boost_up_threshold);
>>>> -	device_writel(dev, lower + delta, ACTMON_DEV_UPPER_WMARK);
> 
> This line was added on patch5 and then this line is removed on this patch.
> It is wrong method to make the patch in the same patchset.
> 
> It is enough to reduce the inefficient add/remove code in the same patchset.

I'm missing some expression. So, I update my comment as following:
It is possible to reduce the inefficient add/remove code in the same patchset
if you combine the patches.

> 
> Have to merge this patch to patch5.
> 
>>>> +
>>>> +	/*
>>>> +	 * The memory events count could go a bit higher than the
>>>> maximum
>>>> +	 * defined by the OPPs, hence make the upper watermark
>>>> infinitely
>>>> +	 * high to avoid unnecessary upper interrupts in that case.
>>>> +	 */
>>>> +	if (freq == tegra->max_freq)
>>>> +		upper = ULONG_MAX;
>>>> +	else
>>>> +		upper = lower + delta;
>>>> +
>>>> +	device_writel(dev, upper, ACTMON_DEV_UPPER_WMARK);
>>>>  
>>>>  	/*
>>>>  	 * Meanwhile the lower mark is based on the average value
>>>>   
>>>
>>>
>>
>>
>>
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 11/24] PM / devfreq: tegra30: Add debug messages
  2019-07-19  1:13             ` Dmitry Osipenko
@ 2019-07-19  1:22               ` Chanwoo Choi
  2019-07-19 17:10                 ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-19  1:22 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 19. 오전 10:13, Dmitry Osipenko wrote:
> В Thu, 18 Jul 2019 18:07:05 +0900
> Chanwoo Choi <cw00.choi@samsung.com> пишет:
> 
>> On 19. 7. 18. 오전 12:46, Dmitry Osipenko wrote:
>>> 17.07.2019 9:45, Chanwoo Choi пишет:  
>>>> On 19. 7. 16. 오후 10:26, Dmitry Osipenko wrote:  
>>>>> 16.07.2019 15:23, Chanwoo Choi пишет:  
>>>>>> Hi Dmitry,
>>>>>>
>>>>>> Usually, the kernel log print for all users
>>>>>> such as changing the frequency, fail or success.
>>>>>>
>>>>>> But, if the log just show the register dump,
>>>>>> it is not useful for all users. It is just used
>>>>>> for only specific developer.
>>>>>>
>>>>>> I recommend that you better to add more exception handling
>>>>>> code on many points instead of just showing the register dump.  
>>>>>
>>>>> The debug messages are not users, but for developers. Yes, I
>>>>> primarily made the debugging to be useful for myself and will be
>>>>> happy to change the way debugging is done if there will be any
>>>>> other active developer for this driver. The registers dump is
>>>>> more than enough in order to understand what's going on, I don't
>>>>> see any real need to change anything here for now.  
>>>>
>>>> Basically, we have to develop code and add the log for anyone.
>>>> As you commented, even if there are no other developer, we never
>>>> guarantee this assumption forever. And also, if added debug message
>>>> for only you, you can add them when testing it temporarily.
>>>>
>>>> If you want to add the just register dump log for you,
>>>> I can't agree. Once again, I hope that anyone understand
>>>> the meaning of debug message as much possible as.
>>>>  
>>>
>>> The registers dump should be good for everyone because it's a
>>> self-explanatory information for anyone who is familiar with the
>>> hardware. I don't think there is a need for anything else than what
>>> is proposed in this patch, at least for now. I also simply don't
>>> see any other better way to debug the state of this particular
>>> hardware, again this logging is for the driver developers and not
>>> for users.
>>>
>>> Initially, I was temporarily adding the debug messages. Now they are
>>> pretty much mandatory for verifying that driver is working
>>> properly. And of course the debugging messages got into the shape
>>> of this patch after several iterations of refinements. So again, I
>>> suppose that this should be good enough for everyone who is
>>> familiar with the hardware. And of course I'm open to the
>>> constructive suggestions, the debugging aid is not an ABI and could
>>> be changed/improved at any time.
>>>
>>> You're suggesting to break down the debugging into several smaller
>>> pieces, but I'm finding that as not a constructive suggestion
>>> because the information about the full hardware state is actually
>>> necessary for the productive debugging.
>>>
>>>   
>>
>> Sorry for that as I saie, I cannot agree this patch. In my case,
>> I don't understand what is meaning of register dump of this patch.
>> I knew that just register dump are useful for real developer.
> 
> It's not only a registers dump, as you may see there is also a dump of
> other properties like boosting value, OPPs selection and etc.
> 
> It looks to me that you're also missing important detail that debug
> messages are compiled out unless DEBUG is defined for the drivers
> build. So in order to get the debug message a user shall explicitly add
> #define DEBUG macro to the code or enable debug messages globally in
> the kernel's config. There is also an option for dynamic debug messages
> in the kernel, but it doesn't matter now because all these messages are
> turned into tracepoints later in the patch #17.


Right. But, this patch could not the split up between register dump and others.
As I said repeatly, I hope to add the log that anyone can understand. 


> 
>> If you want to show the register dump, you better to add some feature
>> with debugfs for devfreq framework in order to read the register dump.
>> As I knew, sound framework (alsa) has the similar feature for checking
>> the register dump.
>>
> 
> The intent was to have an option for dynamic debugging of the driver and
> initially debug messages were good enough, but then it became not enough
> and hence the debug messages were turned into tracepoints in the patch
> #17. Would it be acceptable to squash this patch and #17?
> 
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 12/24] PM / devfreq: tegra30: Inline all one-line functions
  2019-07-18  9:09       ` Chanwoo Choi
@ 2019-07-19  1:22         ` Dmitry Osipenko
  2019-07-19  1:24           ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19  1:22 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

В Thu, 18 Jul 2019 18:09:05 +0900
Chanwoo Choi <cw00.choi@samsung.com> пишет:

> On 19. 7. 16. 오후 10:35, Dmitry Osipenko wrote:
> > 16.07.2019 15:26, Chanwoo Choi пишет:  
> >> Hi Dmitry,
> >>
> >> I'm not sure that it is necessary.
> >> As I knew, usally, the 'inline' is used on header file
> >> to define the empty functions.
> >>
> >> Do we have to change it with 'inline' keyword?  
> > 
> > The 'inline' attribute tells compiler that instead of jumping into
> > the function, it should take the function's code and replace the
> > function's invocation with that code. This is done in order to help
> > compiler optimize code properly, please see [1]. There is
> > absolutely no need to create a function call into a function that
> > consists of a single instruction.
> > 
> > [1] https://gcc.gnu.org/onlinedocs/gcc-9.1.0/gcc/Inline.html
> >   
> 
> If you want to add 'inline' keyword, I recommend that 
> you better to remove the modified function in this patch
> and then just call the 'write_relaxed or read_relaxed' function
> directly. It is same result when using inline keyword.

That could be done, but it makes code less readable.

See the difference:

device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);

writel_relaxed(ACTMON_INTR_STATUS_CLEAR,
	       dev->regs + ACTMON_DEV_INTR_STATUS);


^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 12/24] PM / devfreq: tegra30: Inline all one-line functions
  2019-07-19  1:22         ` Dmitry Osipenko
@ 2019-07-19  1:24           ` Chanwoo Choi
  2019-07-19  1:27             ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-19  1:24 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 19. 오전 10:22, Dmitry Osipenko wrote:
> В Thu, 18 Jul 2019 18:09:05 +0900
> Chanwoo Choi <cw00.choi@samsung.com> пишет:
> 
>> On 19. 7. 16. 오후 10:35, Dmitry Osipenko wrote:
>>> 16.07.2019 15:26, Chanwoo Choi пишет:  
>>>> Hi Dmitry,
>>>>
>>>> I'm not sure that it is necessary.
>>>> As I knew, usally, the 'inline' is used on header file
>>>> to define the empty functions.
>>>>
>>>> Do we have to change it with 'inline' keyword?  
>>>
>>> The 'inline' attribute tells compiler that instead of jumping into
>>> the function, it should take the function's code and replace the
>>> function's invocation with that code. This is done in order to help
>>> compiler optimize code properly, please see [1]. There is
>>> absolutely no need to create a function call into a function that
>>> consists of a single instruction.
>>>
>>> [1] https://gcc.gnu.org/onlinedocs/gcc-9.1.0/gcc/Inline.html
>>>   
>>
>> If you want to add 'inline' keyword, I recommend that 
>> you better to remove the modified function in this patch
>> and then just call the 'write_relaxed or read_relaxed' function
>> directly. It is same result when using inline keyword.
> 
> That could be done, but it makes code less readable.
> 
> See the difference:
> 
> device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
> 
> writel_relaxed(ACTMON_INTR_STATUS_CLEAR,
> 	       dev->regs + ACTMON_DEV_INTR_STATUS);

No problem if you add the detailed comment and you want to use
the 'inline' keyword.

> 
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 12/24] PM / devfreq: tegra30: Inline all one-line functions
  2019-07-19  1:24           ` Chanwoo Choi
@ 2019-07-19  1:27             ` Chanwoo Choi
  2019-07-19  2:14               ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-19  1:27 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 19. 오전 10:24, Chanwoo Choi wrote:
> On 19. 7. 19. 오전 10:22, Dmitry Osipenko wrote:
>> В Thu, 18 Jul 2019 18:09:05 +0900
>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>
>>> On 19. 7. 16. 오후 10:35, Dmitry Osipenko wrote:
>>>> 16.07.2019 15:26, Chanwoo Choi пишет:  
>>>>> Hi Dmitry,
>>>>>
>>>>> I'm not sure that it is necessary.
>>>>> As I knew, usally, the 'inline' is used on header file
>>>>> to define the empty functions.
>>>>>
>>>>> Do we have to change it with 'inline' keyword?  
>>>>
>>>> The 'inline' attribute tells compiler that instead of jumping into
>>>> the function, it should take the function's code and replace the
>>>> function's invocation with that code. This is done in order to help
>>>> compiler optimize code properly, please see [1]. There is
>>>> absolutely no need to create a function call into a function that
>>>> consists of a single instruction.
>>>>
>>>> [1] https://gcc.gnu.org/onlinedocs/gcc-9.1.0/gcc/Inline.html
>>>>   
>>>
>>> If you want to add 'inline' keyword, I recommend that 
>>> you better to remove the modified function in this patch
>>> and then just call the 'write_relaxed or read_relaxed' function
>>> directly. It is same result when using inline keyword.
>>
>> That could be done, but it makes code less readable.
>>
>> See the difference:
>>
>> device_writel(dev, ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS);
>>
>> writel_relaxed(ACTMON_INTR_STATUS_CLEAR,
>> 	       dev->regs + ACTMON_DEV_INTR_STATUS);
> 
> No problem if you add the detailed comment and you want to use
> the 'inline' keyword.

Basically, I think that 'inline' keyword is not necessary.
But if you want to use 'inline' keyword, I recommend
that call the 'write_relaxed or read_relaxed' function directly
with detailed description. 

> 
>>
>>
>>
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 05/24] PM / devfreq: tegra30: Set up watermarks properly
  2019-07-19  0:00     ` Dmitry Osipenko
@ 2019-07-19  1:31       ` Chanwoo Choi
  0 siblings, 0 replies; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-19  1:31 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 19. 오전 9:00, Dmitry Osipenko wrote:
> В Thu, 18 Jul 2019 19:17:17 +0900
> Chanwoo Choi <cw00.choi@samsung.com> пишет:
> 
>> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>>> The current implementation is inaccurate and results in very
>>> intensive interrupt activity, which neglects the whole idea of
>>> polling offload to hardware. The reason of the shortcoming is that
>>> watermarks are not set up correctly and this results in ACTMON
>>> constantly asking to change freq and then these requests are
>>> ignored. The end result of this patch is that there are few
>>> hundreds of ACTMON's interrupts instead of tens thousands after few
>>> minutes of a working devfreq, meanwhile the transitions activity
>>> stays about the same and governor becomes more reactive.
>>>
>>> Since watermarks are set precisely correct now, the boosting logic
>>> is changed a tad to accommodate the change. The "average sustain
>>> coefficient" multiplier is gone now since there is no need to
>>> compensate the improper watermarks and EMC frequency-bump happens
>>> once boosting hits the upper watermark enough times, depending on
>>> the per-device boosting threshold.
>>>
>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>> ---
>>>  drivers/devfreq/tegra30-devfreq.c | 293
>>> +++++++++++++++++++++--------- 1 file changed, 209 insertions(+),
>>> 84 deletions(-)
>>>
>>> diff --git a/drivers/devfreq/tegra30-devfreq.c
>>> b/drivers/devfreq/tegra30-devfreq.c index
>>> 4be7858c33bc..16f7e6cf3b99 100644 ---
>>> a/drivers/devfreq/tegra30-devfreq.c +++
>>> b/drivers/devfreq/tegra30-devfreq.c @@ -47,6 +47,8 @@
>>>  
>>>  #define ACTMON_DEV_INTR_CONSECUTIVE_UPPER
>>> BIT(31) #define
>>> ACTMON_DEV_INTR_CONSECUTIVE_LOWER			BIT(30)
>>> +#define
>>> ACTMON_DEV_INTR_AVG_BELOW_WMARK
>>> BIT(25) +#define
>>> ACTMON_DEV_INTR_AVG_ABOVE_WMARK
>>> BIT(24) #define
>>> ACTMON_ABOVE_WMARK_WINDOW				1 #define
>>> ACTMON_BELOW_WMARK_WINDOW				3 @@ -63,9
>>> +65,8 @@
>>>   * ACTMON_AVERAGE_WINDOW_LOG2: default value for @DEV_CTRL_K_VAL,
>>> which
>>>   * translates to 2 ^ (K_VAL + 1). ex: 2 ^ (6 + 1) = 128
>>>   */
>>> -#define ACTMON_AVERAGE_WINDOW_LOG2			6
>>> -#define ACTMON_SAMPLING_PERIOD
>>> 12 /* ms */ -#define
>>> ACTMON_DEFAULT_AVG_BAND				6  /* 1/10
>>> of % */ +#define
>>> ACTMON_AVERAGE_WINDOW_LOG2				6
>>> +#define
>>> ACTMON_SAMPLING_PERIOD					12 /*
>>> ms */ #define
>>> KHZ							1000 @@
>>> -142,9 +143,6 @@ struct tegra_devfreq_device {
>>>  	 * watermark breaches.
>>>  	 */
>>>  	unsigned long boost_freq;
>>> -
>>> -	/* Optimal frequency calculated from the stats for this
>>> device */
>>> -	unsigned long target_freq;
>>>  };
>>>  
>>>  struct tegra_devfreq {
>>> @@ -156,7 +154,6 @@ struct tegra_devfreq {
>>>  
>>>  	struct clk		*emc_clock;
>>>  	unsigned long		max_freq;
>>> -	unsigned long		cur_freq;
>>>  	struct notifier_block	rate_change_nb;
>>>  
>>>  	struct tegra_devfreq_device
>>> devices[ARRAY_SIZE(actmon_device_configs)]; @@ -205,42 +202,182 @@
>>> static unsigned long do_percent(unsigned long val, unsigned int
>>> pct) return val * pct / 100; }
>>>  
>>> +static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq
>>> *tegra) +{
>>> +	struct tegra_actmon_emc_ratio *ratio = actmon_emc_ratios;
>>> +	unsigned int cpu_freq = cpufreq_get(0);
>>> +	unsigned int i;
>>> +
>>> +	for (i = 0; i < ARRAY_SIZE(actmon_emc_ratios); i++,
>>> ratio++) {
>>> +		if (cpu_freq >= ratio->cpu_freq) {
>>> +			if (ratio->emc_freq >= tegra->max_freq)
>>> +				return tegra->max_freq;
>>> +			else
>>> +				return ratio->emc_freq;
>>> +		}
>>> +	}
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static unsigned long
>>> +tegra_actmon_account_cpu_freq(struct tegra_devfreq *tegra,
>>> +			      struct tegra_devfreq_device *dev,
>>> +			      unsigned long target_freq)
>>> +{
>>> +	unsigned long static_cpu_emc_freq;
>>> +
>>> +	if (dev->config->avg_dependency_threshold &&
>>> +	    dev->config->avg_dependency_threshold <
>>> dev->avg_count) {
>>> +		static_cpu_emc_freq =
>>> actmon_cpu_to_emc_rate(tegra);
>>> +		target_freq = max(target_freq,
>>> static_cpu_emc_freq);
>>> +	}
>>> +
>>> +	return target_freq;
>>> +}
>>> +
>>> +static unsigned long tegra_actmon_lower_freq(struct tegra_devfreq
>>> *tegra,
>>> +					     unsigned long
>>> target_freq) +{
>>> +	unsigned long lower = target_freq;
>>> +	struct dev_pm_opp *opp;
>>> +
>>> +	opp =
>>> dev_pm_opp_find_freq_floor(tegra->devfreq->dev.parent, &lower);
>>> +	if (IS_ERR(opp))
>>> +		lower = 0;
>>> +	else
>>> +		dev_pm_opp_put(opp);
>>> +
>>> +	return lower;
>>> +}
>>> +
>>> +static unsigned long tegra_actmon_upper_freq(struct tegra_devfreq
>>> *tegra,
>>> +					     unsigned long
>>> target_freq) +{
>>> +	unsigned long upper = target_freq + 1;
>>> +	struct dev_pm_opp *opp;
>>> +
>>> +	opp =
>>> dev_pm_opp_find_freq_ceil(tegra->devfreq->dev.parent, &upper);
>>> +	if (IS_ERR(opp))
>>> +		upper = ULONG_MAX;
>>> +	else
>>> +		dev_pm_opp_put(opp);
>>> +
>>> +	return upper;
>>> +}
>>> +
>>> +static void tegra_actmon_get_lower_upper(struct tegra_devfreq
>>> *tegra,
>>> +					 struct
>>> tegra_devfreq_device *dev,
>>> +					 unsigned long target_freq,
>>> +					 unsigned long *lower,
>>> +					 unsigned long *upper)
>>> +{
>>> +	/*
>>> +	 * Memory frequencies are guaranteed to have 1MHz
>>> granularity
>>> +	 * and thus we need this rounding down to get a proper
>>> watermarks
>>> +	 * range in a case where target_freq falls into a range of
>>> +	 * next_possible_opp_freq - 1MHz.
>>> +	 */
>>> +	target_freq = round_down(target_freq, 1000000);
>>> +
>>> +	/* watermarks are set at the borders of the corresponding
>>> OPPs */
>>> +	*lower = tegra_actmon_lower_freq(tegra, target_freq);
>>> +	*upper = tegra_actmon_upper_freq(tegra, target_freq);
>>> +
>>> +	*lower /= KHZ;
>>> +	*upper /= KHZ;
>>> +
>>> +	/*
>>> +	 * The upper watermark should take into account CPU's
>>> frequency
>>> +	 * because cpu_to_emc_rate() may override the target_freq
>>> with
>>> +	 * a higher value and thus upper watermark need to be set
>>> up
>>> +	 * accordingly to avoid parasitic upper-events.
>>> +	 */
>>> +	*upper = tegra_actmon_account_cpu_freq(tegra, dev, *upper);
>>> +
>>> +	*lower *= ACTMON_SAMPLING_PERIOD;
>>> +	*upper *= ACTMON_SAMPLING_PERIOD;
>>> +}
>>> +
>>>  static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq
>>> *tegra, struct tegra_devfreq_device *dev)
>>>  {
>>> -	u32 avg = dev->avg_count;
>>> -	u32 avg_band_freq = tegra->max_freq *
>>> ACTMON_DEFAULT_AVG_BAND / KHZ;
>>> -	u32 band = avg_band_freq * ACTMON_SAMPLING_PERIOD;
>>> +	unsigned long lower, upper, freq;
>>>  
>>> -	device_writel(dev, avg + band, ACTMON_DEV_AVG_UPPER_WMARK);
>>> +	freq = dev->avg_count / ACTMON_SAMPLING_PERIOD * KHZ;
>>> +	tegra_actmon_get_lower_upper(tegra, dev, freq, &lower,
>>> &upper); 
>>> -	avg = max(dev->avg_count, band);
>>> -	device_writel(dev, avg - band, ACTMON_DEV_AVG_LOWER_WMARK);
>>> +	/*
>>> +	 * We want to get interrupts when MCCPU client crosses the
>>> +	 * dependency threshold in order to take into / out of
>>> account
>>> +	 * the CPU's freq.
>>> +	 */
>>> +	if (lower < dev->config->avg_dependency_threshold &&
>>> +	    upper > dev->config->avg_dependency_threshold) {
>>> +		if (dev->avg_count <
>>> dev->config->avg_dependency_threshold)
>>> +			upper =
>>> dev->config->avg_dependency_threshold;
>>> +		else
>>> +			lower =
>>> dev->config->avg_dependency_threshold;
>>> +	}
>>> +
>>> +	device_writel(dev, lower, ACTMON_DEV_AVG_LOWER_WMARK);
>>> +	device_writel(dev, upper, ACTMON_DEV_AVG_UPPER_WMARK);
>>>  }
>>>  
>>>  static void tegra_devfreq_update_wmark(struct tegra_devfreq *tegra,
>>> -				       struct tegra_devfreq_device
>>> *dev)
>>> +				       struct tegra_devfreq_device
>>> *dev,
>>> +				       unsigned long freq)
>>>  {
>>> -	u32 val = tegra->cur_freq * ACTMON_SAMPLING_PERIOD;
>>> +	unsigned long lower, upper, delta;
>>> +
>>> +	/*
>>> +	 * Boosting logic kicks-in once lower / upper watermark is
>>> hit.
>>> +	 * The watermarks are based on the updated EMC rate and the
>>> +	 * average activity.
>>> +	 *
>>> +	 * The higher watermark is set in accordance to the EMC
>>> rate
>>> +	 * because we want to set it to the highest mark here and
>>> EMC rate
>>> +	 * represents that mark. The consecutive-upper interrupts
>>> are
>>> +	 * always enabled and we don't want to receive them if
>>> they won't
>>> +	 * do anything useful, hence the upper watermark is capped
>>> to maximum.
>>> +	 * Note that the EMC rate is changed once boosting pushed
>>> the rate
>>> +	 * too high, in that case boosting-up will be stopped
>>> because
>>> +	 * upper watermark is much higher now and it is
>>> *important* to
>>> +	 * stop the unwanted interrupts.
>>> +	 */
>>> +	tegra_actmon_get_lower_upper(tegra, dev, freq - 1, &lower,
>>> &upper); +
>>> +	delta = do_percent(upper - lower,
>>> dev->config->boost_up_threshold);
>>> +	device_writel(dev, lower + delta, ACTMON_DEV_UPPER_WMARK);
>>>  
>>> -	device_writel(dev, do_percent(val,
>>> dev->config->boost_up_threshold),
>>> -		      ACTMON_DEV_UPPER_WMARK);
>>> +	/*
>>> +	 * Meanwhile the lower mark is based on the average value
>>> +	 * because it is the lowest possible consecutive-mark for
>>> this
>>> +	 * device. Once that mark is hit and boosting is stopped,
>>> the
>>> +	 * interrupt is disabled by ISR.
>>> +	 */
>>> +	freq = dev->avg_count / ACTMON_SAMPLING_PERIOD * KHZ;
>>> +	tegra_actmon_get_lower_upper(tegra, dev, freq, &lower,
>>> &upper); 
>>> -	device_writel(dev, do_percent(val,
>>> dev->config->boost_down_threshold),
>>> -		      ACTMON_DEV_LOWER_WMARK);
>>> +	delta = do_percent(upper - lower,
>>> dev->config->boost_down_threshold);
>>> +	device_writel(dev, lower + delta, ACTMON_DEV_LOWER_WMARK);
>>>  }
>>>  
>>>  static void actmon_isr_device(struct tegra_devfreq *tegra,
>>>  			      struct tegra_devfreq_device *dev)
>>>  {
>>> -	u32 intr_status, dev_ctrl;
>>> +	u32 intr_status, dev_ctrl, avg_intr_mask;
>>>  
>>>  	dev->avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
>>> -	tegra_devfreq_update_avg_wmark(tegra, dev);
>>> -
>>>  	intr_status = device_readl(dev, ACTMON_DEV_INTR_STATUS);
>>>  	dev_ctrl = device_readl(dev, ACTMON_DEV_CTRL);
>>>  
>>> +	avg_intr_mask = ACTMON_DEV_INTR_AVG_BELOW_WMARK |
>>> +			ACTMON_DEV_INTR_AVG_ABOVE_WMARK;
>>> +
>>> +	if (intr_status & avg_intr_mask)
>>> +		tegra_devfreq_update_avg_wmark(tegra, dev);
>>> +
>>>  	if (intr_status & ACTMON_DEV_INTR_CONSECUTIVE_UPPER) {
>>>  		/*
>>>  		 * new_boost = min(old_boost * up_coef + step,
>>> max_freq) @@ -253,8 +390,6 @@ static void actmon_isr_device(struct
>>> tegra_devfreq *tegra, 
>>>  		if (dev->boost_freq >= tegra->max_freq)
>>>  			dev->boost_freq = tegra->max_freq;
>>> -		else
>>> -			dev_ctrl |=
>>> ACTMON_DEV_CTRL_CONSECUTIVE_ABOVE_WMARK_EN; } else if (intr_status
>>> & ACTMON_DEV_INTR_CONSECUTIVE_LOWER) { /*
>>>  		 * new_boost = old_boost * down_coef
>>> @@ -263,63 +398,37 @@ static void actmon_isr_device(struct
>>> tegra_devfreq *tegra, dev->boost_freq = do_percent(dev->boost_freq,
>>>  					     dev->config->boost_down_coeff);
>>>  
>>> -		dev_ctrl |=
>>> ACTMON_DEV_CTRL_CONSECUTIVE_ABOVE_WMARK_EN; -
>>>  		if (dev->boost_freq < (ACTMON_BOOST_FREQ_STEP >>
>>> 1)) dev->boost_freq = 0;
>>> -		else
>>> -			dev_ctrl |=
>>> ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN; }
>>>  
>>> -	if (dev->config->avg_dependency_threshold) {
>>> -		if (dev->avg_count >=
>>> dev->config->avg_dependency_threshold)
>>> -			dev_ctrl |=
>>> ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
>>> -		else if (dev->boost_freq == 0)
>>> -			dev_ctrl &=
>>> ~ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN;
>>> +	if (intr_status & avg_intr_mask) {
>>> +		/*
>>> +		 * Once average watermark is hit, it means that
>>> the memory
>>> +		 * activity changed significantly and thus
>>> boosting-up shall
>>> +		 * be reset because EMC clock rate will be changed
>>> and
>>> +		 * boosting will restart in this case.
>>> +		 */
>>> +		dev->boost_freq = 0;
>>>  	}
>>>  
>>> -	device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
>>> +	/* no boosting => no need for consecutive-down interrupt */
>>> +	if (dev->boost_freq == 0)
>>> +		dev_ctrl &=
>>> ~ACTMON_DEV_CTRL_CONSECUTIVE_BELOW_WMARK_EN; 
>>> +	device_writel(dev, dev_ctrl, ACTMON_DEV_CTRL);
>>>  	device_writel(dev, ACTMON_INTR_STATUS_CLEAR,
>>> ACTMON_DEV_INTR_STATUS); }
>>>  
>>> -static unsigned long actmon_cpu_to_emc_rate(struct tegra_devfreq
>>> *tegra,
>>> -					    unsigned long cpu_freq)
>>> -{
>>> -	unsigned int i;
>>> -	struct tegra_actmon_emc_ratio *ratio = actmon_emc_ratios;
>>> -
>>> -	for (i = 0; i < ARRAY_SIZE(actmon_emc_ratios); i++,
>>> ratio++) {
>>> -		if (cpu_freq >= ratio->cpu_freq) {
>>> -			if (ratio->emc_freq >= tegra->max_freq)
>>> -				return tegra->max_freq;
>>> -			else
>>> -				return ratio->emc_freq;
>>> -		}
>>> -	}
>>> -
>>> -	return 0;
>>> -}
>>> -
>>> -static void actmon_update_target(struct tegra_devfreq *tegra,
>>> -				 struct tegra_devfreq_device *dev)
>>> +static unsigned long actmon_update_target(struct tegra_devfreq
>>> *tegra,
>>> +					  struct
>>> tegra_devfreq_device *dev) {
>>> -	unsigned long cpu_freq = 0;
>>> -	unsigned long static_cpu_emc_freq = 0;
>>> -	unsigned int avg_sustain_coef;
>>> -
>>> -	if (dev->config->avg_dependency_threshold) {
>>> -		cpu_freq = cpufreq_get(0);
>>> -		static_cpu_emc_freq =
>>> actmon_cpu_to_emc_rate(tegra, cpu_freq);
>>> -	}
>>> +	unsigned long target_freq;
>>>  
>>> -	dev->target_freq = dev->avg_count / ACTMON_SAMPLING_PERIOD;
>>> -	avg_sustain_coef = 100 * 100 /
>>> dev->config->boost_up_threshold;
>>> -	dev->target_freq = do_percent(dev->target_freq,
>>> avg_sustain_coef);
>>> -	dev->target_freq += dev->boost_freq;
>>> +	target_freq = dev->avg_count / ACTMON_SAMPLING_PERIOD +
>>> dev->boost_freq;
>>> +	target_freq = tegra_actmon_account_cpu_freq(tegra, dev,
>>> target_freq); 
>>> -	if (dev->avg_count >=
>>> dev->config->avg_dependency_threshold)
>>> -		dev->target_freq = max(dev->target_freq,
>>> static_cpu_emc_freq);
>>> +	return target_freq;
>>>  }
>>>  
>>>  static irqreturn_t actmon_thread_isr(int irq, void *data)
>>> @@ -351,8 +460,8 @@ static int tegra_actmon_rate_notify_cb(struct
>>> notifier_block *nb, unsigned long action, void *ptr)
>>>  {
>>>  	struct clk_notifier_data *data = ptr;
>>> -	struct tegra_devfreq *tegra;
>>>  	struct tegra_devfreq_device *dev;
>>> +	struct tegra_devfreq *tegra;
>>>  	unsigned int i;
>>>  
>>>  	if (action != POST_RATE_CHANGE)
>>> @@ -360,12 +469,28 @@ static int tegra_actmon_rate_notify_cb(struct
>>> notifier_block *nb, 
>>>  	tegra = container_of(nb, struct tegra_devfreq,
>>> rate_change_nb); 
>>> -	tegra->cur_freq = data->new_rate / KHZ;
>>> -
>>> +	/*
>>> +	 * EMC rate could change due to three reasons:
>>> +	 *
>>> +	 *    1. Average watermark hit
>>> +	 *    2. Boosting overflow
>>> +	 *    3. CPU freq change
>>> +	 *
>>> +	 * Once rate is changed, the consecutive watermarks need
>>> to be
>>> +	 * updated in order for boosting to work properly and to
>>> avoid
>>> +	 * unnecessary interrupts. Note that the consecutive range
>>> is set for
>>> +	 * all of devices using the same rate, hence if CPU is
>>> doing much
>>> +	 * less than the other memory clients, then its upper
>>> watermark will
>>> +	 * be very high in comparison to the actual activity
>>> (lower watermark)
>>> +	 * and thus unnecessary upper-interrupts will be
>>> suppressed.
>>> +	 *
>>> +	 * The average watermarks also should be updated because
>>> of 3.
>>> +	 */
>>>  	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++) {
>>>  		dev = &tegra->devices[i];
>>>  
>>> -		tegra_devfreq_update_wmark(tegra, dev);
>>> +		tegra_devfreq_update_avg_wmark(tegra, dev);
>>> +		tegra_devfreq_update_wmark(tegra, dev,
>>> data->new_rate); }
>>>  
>>>  	return NOTIFY_OK;
>>> @@ -374,15 +499,14 @@ static int tegra_actmon_rate_notify_cb(struct
>>> notifier_block *nb, static void
>>> tegra_actmon_configure_device(struct tegra_devfreq *tegra, struct
>>> tegra_devfreq_device *dev) {
>>> -	u32 val = 0;
>>> -
>>> -	dev->target_freq = tegra->cur_freq;
>>> +	u32 val = 0, target_freq;
>>>  
>>> -	dev->avg_count = tegra->cur_freq * ACTMON_SAMPLING_PERIOD;
>>> +	target_freq = clk_get_rate(tegra->emc_clock) / KHZ;
>>> +	dev->avg_count = target_freq * ACTMON_SAMPLING_PERIOD;
>>>  	device_writel(dev, dev->avg_count, ACTMON_DEV_INIT_AVG);
>>>  
>>>  	tegra_devfreq_update_avg_wmark(tegra, dev);
>>> -	tegra_devfreq_update_wmark(tegra, dev);
>>> +	tegra_devfreq_update_wmark(tegra, dev, target_freq);
>>>  
>>>  	device_writel(dev, ACTMON_COUNT_WEIGHT,
>>> ACTMON_DEV_COUNT_WEIGHT); device_writel(dev,
>>> ACTMON_INTR_STATUS_CLEAR, ACTMON_DEV_INTR_STATUS); @@ -469,13
>>> +593,13 @@ static int tegra_devfreq_get_dev_status(struct device
>>> *dev, struct tegra_devfreq_device *actmon_dev; unsigned long
>>> cur_freq; 
>>> -	cur_freq = READ_ONCE(tegra->cur_freq);
>>> +	cur_freq = clk_get_rate(tegra->emc_clock);
>>>  
>>>  	/* To be used by the tegra governor */
>>>  	stat->private_data = tegra;
>>>  
>>>  	/* The below are to be used by the other governors */
>>> -	stat->current_frequency = cur_freq * KHZ;
>>> +	stat->current_frequency = cur_freq;
>>>  
>>>  	actmon_dev = &tegra->devices[MCALL];
>>>  
>>> @@ -486,7 +610,7 @@ static int tegra_devfreq_get_dev_status(struct
>>> device *dev, stat->busy_time *= 100 / BUS_SATURATION_RATIO;
>>>  
>>>  	/* Number of cycles in a sampling period */
>>> -	stat->total_time = ACTMON_SAMPLING_PERIOD * cur_freq;
>>> +	stat->total_time = cur_freq / KHZ * ACTMON_SAMPLING_PERIOD;
>>>  
>>>  	stat->busy_time = min(stat->busy_time, stat->total_time);
>>>  
>>> @@ -505,6 +629,7 @@ static int tegra_governor_get_target(struct
>>> devfreq *devfreq, struct devfreq_dev_status *stat;
>>>  	struct tegra_devfreq *tegra;
>>>  	struct tegra_devfreq_device *dev;
>>> +	unsigned long dev_target_freq;
>>>  	unsigned long target_freq = 0;
>>>  	unsigned int i;
>>>  	int err;
>>> @@ -520,9 +645,9 @@ static int tegra_governor_get_target(struct
>>> devfreq *devfreq, for (i = 0; i < ARRAY_SIZE(tegra->devices); i++) {
>>>  		dev = &tegra->devices[i];
>>>  
>>> -		actmon_update_target(tegra, dev);
>>> +		dev_target_freq = actmon_update_target(tegra, dev);
>>>  
>>> -		target_freq = max(target_freq, dev->target_freq);
>>> +		target_freq = max(target_freq, dev_target_freq);
>>>  	}
>>>  
>>>  	*freq = target_freq * KHZ;
>>> @@ -642,7 +767,6 @@ static int tegra_devfreq_probe(struct
>>> platform_device *pdev) return rate;
>>>  	}
>>>  
>>> -	tegra->cur_freq = clk_get_rate(tegra->emc_clock) / KHZ;
>>>  	tegra->max_freq = rate / KHZ;
>>>  
>>>  	for (i = 0; i < ARRAY_SIZE(actmon_device_configs); i++) {
>>> @@ -671,7 +795,8 @@ static int tegra_devfreq_probe(struct
>>> platform_device *pdev) platform_set_drvdata(pdev, tegra);
>>>  
>>>  	tegra->rate_change_nb.notifier_call =
>>> tegra_actmon_rate_notify_cb;
>>> -	err = clk_notifier_register(tegra->emc_clock,
>>> &tegra->rate_change_nb);
>>> +	err = clk_notifier_register(tegra->emc_clock,
>>> +				    &tegra->rate_change_nb);
>>>  	if (err) {
>>>  		dev_err(&pdev->dev,
>>>  			"Failed to register rate change
>>> notifier\n"); 
>>
>>
>> Maybe, it is possible to merge patch4/patch19/patch20 to one patch.
> 
> All these three patches are completely separate changes, thus they
> should be kept separate.
> 

I replied on patch19 why it is possible to merge patch5 and patch19.
Please check my comment.


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 20/24] PM / devfreq: tegra30: Optimize upper average watermark selection
  2019-07-07 22:32 ` [PATCH v4 20/24] PM / devfreq: tegra30: Optimize upper average " Dmitry Osipenko
@ 2019-07-19  1:36   ` Chanwoo Choi
  2019-07-19  1:59     ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-19  1:36 UTC (permalink / raw)
  To: Dmitry Osipenko, Thierry Reding, MyungJoo Ham, Kyungmin Park,
	Jonathan Hunter, Tomeu Vizoso
  Cc: linux-pm, linux-tegra, linux-kernel

On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> I noticed that CPU may be crossing the dependency threshold very
> frequently for some workloads and this results in a lot of interrupts
> which could be avoided if MCALL client is keeping actual EMC frequency
> at a higher rate.
> 
> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> ---
>  drivers/devfreq/tegra30-devfreq.c | 23 ++++++++++++++++++-----
>  1 file changed, 18 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/devfreq/tegra30-devfreq.c b/drivers/devfreq/tegra30-devfreq.c
> index c3cf87231d25..4d582809acb6 100644
> --- a/drivers/devfreq/tegra30-devfreq.c
> +++ b/drivers/devfreq/tegra30-devfreq.c
> @@ -314,7 +314,8 @@ static void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra,
>  }
>  
>  static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
> -					   struct tegra_devfreq_device *dev)
> +					   struct tegra_devfreq_device *dev,
> +					   unsigned long freq)
>  {
>  	unsigned long avg_threshold, lower, upper;
>  
> @@ -323,6 +324,15 @@ static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
>  	avg_threshold = dev->config->avg_dependency_threshold;
>  	avg_threshold = avg_threshold * ACTMON_SAMPLING_PERIOD;
>  
> +	/*
> +	 * If cumulative EMC frequency selection is higher than the
> +	 * device's, then there is no need to set upper watermark to
> +	 * a lower value because it will result in unnecessary upper
> +	 * interrupts.
> +	 */
> +	if (freq * ACTMON_SAMPLING_PERIOD > upper)
> +		upper = freq * ACTMON_SAMPLING_PERIOD;

Also, 'upper value is used on the patch5. You can combine this code to patch5
or if this patch depends on the cpu notifier, you can combine it to the patch
of adding cpu notifier without separate patch.

> +
>  	/*
>  	 * We want to get interrupts when MCCPU client crosses the
>  	 * dependency threshold in order to take into / out of account
> @@ -392,6 +402,7 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
>  			      struct tegra_devfreq_device *dev)
>  {
>  	u32 intr_status, dev_ctrl, avg_intr_mask, avg_count;
> +	unsigned long freq;
>  
>  	trace_device_isr_enter(tegra->regs, dev->config->offset,
>  			       dev->boost_freq, cpufreq_get(0));
> @@ -405,8 +416,10 @@ static void actmon_isr_device(struct tegra_devfreq *tegra,
>  	avg_intr_mask = ACTMON_DEV_INTR_AVG_BELOW_WMARK |
>  			ACTMON_DEV_INTR_AVG_ABOVE_WMARK;
>  
> -	if (intr_status & avg_intr_mask)
> -		tegra_devfreq_update_avg_wmark(tegra, dev);
> +	if (intr_status & avg_intr_mask) {
> +		freq = clk_get_rate(tegra->emc_clock) / KHZ;
> +		tegra_devfreq_update_avg_wmark(tegra, dev, freq);
> +	}
>  
>  	if (intr_status & ACTMON_DEV_INTR_CONSECUTIVE_UPPER) {
>  		/*
> @@ -525,7 +538,7 @@ static int tegra_actmon_clk_notify_cb(struct notifier_block *nb,
>  	for (i = 0; i < ARRAY_SIZE(tegra->devices); i++) {
>  		dev = &tegra->devices[i];
>  
> -		tegra_devfreq_update_avg_wmark(tegra, dev);
> +		tegra_devfreq_update_avg_wmark(tegra, dev, freq);
>  		tegra_devfreq_update_wmark(tegra, dev, freq);
>  	}
>  
> @@ -630,7 +643,7 @@ static void tegra_actmon_configure_device(struct tegra_devfreq *tegra,
>  	device_writel(dev, dev->avg_freq * ACTMON_SAMPLING_PERIOD,
>  		      ACTMON_DEV_INIT_AVG);
>  
> -	tegra_devfreq_update_avg_wmark(tegra, dev);
> +	tegra_devfreq_update_avg_wmark(tegra, dev, dev->avg_freq);
>  	tegra_devfreq_update_wmark(tegra, dev, dev->avg_freq);
>  
>  	device_writel(dev, ACTMON_COUNT_WEIGHT, ACTMON_DEV_COUNT_WEIGHT);
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 21/24] PM / devfreq: tegra30: Synchronize average count on target's update
  2019-07-19  0:31     ` Dmitry Osipenko
@ 2019-07-19  1:40       ` Chanwoo Choi
  2019-07-19 16:46         ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-19  1:40 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 19. 오전 9:31, Dmitry Osipenko wrote:
> В Thu, 18 Jul 2019 19:15:54 +0900
> Chanwoo Choi <cw00.choi@samsung.com> пишет:
> 
>> On 19. 7. 8. 오전 7:33, Dmitry Osipenko wrote:
>>> The average count may get out of sync if interrupt was disabled /
>>> avoided for a long time due to upper watermark optimization, hence
>>> it should be re-synced on each target's update to ensure that
>>> watermarks are set up correctly on EMC rate-change notification and
>>> that a correct frequency is selected for device.
>>>
>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>> ---
>>>  drivers/devfreq/tegra30-devfreq.c | 30
>>> ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
>>>
>>> diff --git a/drivers/devfreq/tegra30-devfreq.c
>>> b/drivers/devfreq/tegra30-devfreq.c index
>>> 4d582809acb6..8a674fad26be 100644 ---
>>> a/drivers/devfreq/tegra30-devfreq.c +++
>>> b/drivers/devfreq/tegra30-devfreq.c @@ -466,11 +466,41 @@ static
>>> void actmon_isr_device(struct tegra_devfreq *tegra,
>>> dev->boost_freq, cpufreq_get(0)); }
>>>  
>>> +static void tegra_devfreq_sync_avg_count(struct tegra_devfreq
>>> *tegra,
>>> +					 struct
>>> tegra_devfreq_device *dev) +{
>>> +	u32 avg_count, avg_freq, old_upper, new_upper;
>>> +
>>> +	avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
>>> +	avg_freq = avg_count / ACTMON_SAMPLING_PERIOD;
>>> +
>>> +	old_upper = tegra_actmon_upper_freq(tegra, dev->avg_freq);
>>> +	new_upper = tegra_actmon_upper_freq(tegra, avg_freq);
>>> +
>>> +	/* similar to ISR, see comments in actmon_isr_device() */
>>> +	if (old_upper != new_upper) {
>>> +		dev->avg_freq = avg_freq;
>>> +		dev->boost_freq = 0;
>>> +	}
>>> +}
>>> +
>>>  static unsigned long actmon_update_target(struct tegra_devfreq
>>> *tegra, struct tegra_devfreq_device *dev)
>>>  {
>>>  	unsigned long target_freq;
>>>  
>>> +	/*
>>> +	 * The avg_count / avg_freq is getting snapshoted on
>>> device's
>>> +	 * interrupt, but there are cases where actual value need
>>> to
>>> +	 * be utilized on target's update, like CPUFreq boosting
>>> and
>>> +	 * overriding the min freq
>>> via /sys/class/devfreq/devfreq0/min_freq
>>> +	 * because we're optimizing the upper watermark based on
>>> the
>>> +	 * actual EMC frequency. This means that interrupt may be
>>> +	 * inactive for a long time and thus making snapshoted
>>> value
>>> +	 * outdated.
>>> +	 */
>>> +	tegra_devfreq_sync_avg_count(tegra, dev);  
>>
>> I think that you don't need to add the separate function to calculate
>> the 'dev->avg_freq'. It is enough with your detailed comment to add
>> this code in this function.
> 
> The separate function is indeed not mandatory here, but I'm finding that
> it usually makes easier to read and follow the code when it is properly
> split up into logical blocks. Don't you agree?

It is right to make the separate function if function is too long or 
function is called on the multiple points.

But, in this case, I think that it is enough to add this code
to the actmon_update_target() because you already added the detailed comment. 

It is enough to understand the role of this code with your comment.

> 
>>> +
>>>  	target_freq = min(dev->avg_freq + dev->boost_freq,
>>> KHZ_MAX); target_freq = tegra_actmon_account_cpu_freq(tegra, dev,
>>> target_freq); 
>>>   
>>
>> And also, is it impossible to squash this patch with patch19/patch20?
>>
> 
> It should be possible to squash this patch with #20, but wouldn't
> be better to keep changes in the chronological order? It's also better
> to keep changes separate simply to aid bisection in case of a problem.
> 
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 17/24] PM / devfreq: tegra30: Use tracepoints for debugging
  2019-07-19  1:01       ` Chanwoo Choi
@ 2019-07-19  1:50         ` Dmitry Osipenko
  0 siblings, 0 replies; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19  1:50 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

В Fri, 19 Jul 2019 10:01:55 +0900
Chanwoo Choi <cw00.choi@samsung.com> пишет:

> On 19. 7. 19. 오전 9:49, Dmitry Osipenko wrote:
> > В Thu, 18 Jul 2019 18:47:09 +0900
> > Chanwoo Choi <cw00.choi@samsung.com> пишет:
> >   
> >> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:  
> >>> Debug messages create too much CPU and memory activity by
> >>> themselves, so it's difficult to debug lower rates and catch
> >>> unwanted interrupts that happen rarely. Tracepoints are ideal in
> >>> that regards because they do not contribute to the sampled date at
> >>> all. This allowed me to catch few problems which are fixed by the
> >>> followup patches, without tracepoints it would be much harder to
> >>> do.
> >>>
> >>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> >>> ---
> >>>  drivers/devfreq/tegra30-devfreq.c      |  43 +++-------
> >>>  include/trace/events/tegra30_devfreq.h | 105
> >>> +++++++++++++++++++++++++ 2 files changed, 117 insertions(+), 31
> >>> deletions(-) create mode 100644
> >>> include/trace/events/tegra30_devfreq.h    
> >>
> >> As I knew, 'include/trace/events' don't include the header file
> >> for only one device driver. Usually, the trace event is provided
> >> by framework instead of each devic driver.  
> > 
> > There are at least trace headers there for the tegra-apbdma,
> > tegra-host1x, intel-sst and intel-ish devices. I don't think that
> > there is a strict rule for the trace headers placement.  
> 
> OK.
> 
> But, As I already replied on patch4, if you want to show the register
> dump, you better to add the debugfs feature to devfreq framework for
> showing the register dump instead of printing the register dump with
> debug level and this trace event. 
> 
> As I said, just register dump is not useful for all developers.
> Almost developer cannot understand the meaning of debug log for
> register dump.

I think there is some disconnect here. I'm finding that the raw
register values are essential for debugging of this driver. The
registers tracing is very trivial and self-explanatory, just can't see
any better variant.

The registers documentation is available for everyone, you can go to
NVIDIA website and download it (after registration).

We have registers tracing in other Tegra drivers, please see for the
quick example:

https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/staging/media/tegra-vde/trace.h
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/gpu/drm/tegra/trace.h

It's the first time I'm seeing complains about debug tracing and
currently having hard time trying to understand yours point.

> Also, it is not proper way that front patch adds the some code
> and then later patch removes the additional code in the same series.
> Before sending the patches, you can renew them.

Okay.

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 20/24] PM / devfreq: tegra30: Optimize upper average watermark selection
  2019-07-19  1:36   ` Chanwoo Choi
@ 2019-07-19  1:59     ` Dmitry Osipenko
  2019-07-19  2:06       ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19  1:59 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

В Fri, 19 Jul 2019 10:36:30 +0900
Chanwoo Choi <cw00.choi@samsung.com> пишет:

> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
> > I noticed that CPU may be crossing the dependency threshold very
> > frequently for some workloads and this results in a lot of
> > interrupts which could be avoided if MCALL client is keeping actual
> > EMC frequency at a higher rate.
> > 
> > Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> > ---
> >  drivers/devfreq/tegra30-devfreq.c | 23 ++++++++++++++++++-----
> >  1 file changed, 18 insertions(+), 5 deletions(-)
> > 
> > diff --git a/drivers/devfreq/tegra30-devfreq.c
> > b/drivers/devfreq/tegra30-devfreq.c index
> > c3cf87231d25..4d582809acb6 100644 ---
> > a/drivers/devfreq/tegra30-devfreq.c +++
> > b/drivers/devfreq/tegra30-devfreq.c @@ -314,7 +314,8 @@ static void
> > tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra, }
> >  
> >  static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq
> > *tegra,
> > -					   struct
> > tegra_devfreq_device *dev)
> > +					   struct
> > tegra_devfreq_device *dev,
> > +					   unsigned long freq)
> >  {
> >  	unsigned long avg_threshold, lower, upper;
> >  
> > @@ -323,6 +324,15 @@ static void
> > tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
> > avg_threshold = dev->config->avg_dependency_threshold;
> > avg_threshold = avg_threshold * ACTMON_SAMPLING_PERIOD; 
> > +	/*
> > +	 * If cumulative EMC frequency selection is higher than the
> > +	 * device's, then there is no need to set upper watermark
> > to
> > +	 * a lower value because it will result in unnecessary
> > upper
> > +	 * interrupts.
> > +	 */
> > +	if (freq * ACTMON_SAMPLING_PERIOD > upper)
> > +		upper = freq * ACTMON_SAMPLING_PERIOD;  
> 
> Also, 'upper value is used on the patch5. You can combine this code
> to patch5 or if this patch depends on the cpu notifier, you can
> combine it to the patch of adding cpu notifier without separate patch.

Well okay, I'll try to squash some of the patches in the next revision.
Usually I'm receiving comments in the other direction, asking to
separate patches into smaller changes ;) So that's more a personal
preference of each maintainer, I'd say.

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 20/24] PM / devfreq: tegra30: Optimize upper average watermark selection
  2019-07-19  1:59     ` Dmitry Osipenko
@ 2019-07-19  2:06       ` Chanwoo Choi
  2019-07-19  2:21         ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-19  2:06 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 19. 오전 10:59, Dmitry Osipenko wrote:
> В Fri, 19 Jul 2019 10:36:30 +0900
> Chanwoo Choi <cw00.choi@samsung.com> пишет:
> 
>> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:
>>> I noticed that CPU may be crossing the dependency threshold very
>>> frequently for some workloads and this results in a lot of
>>> interrupts which could be avoided if MCALL client is keeping actual
>>> EMC frequency at a higher rate.
>>>
>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>> ---
>>>  drivers/devfreq/tegra30-devfreq.c | 23 ++++++++++++++++++-----
>>>  1 file changed, 18 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/drivers/devfreq/tegra30-devfreq.c
>>> b/drivers/devfreq/tegra30-devfreq.c index
>>> c3cf87231d25..4d582809acb6 100644 ---
>>> a/drivers/devfreq/tegra30-devfreq.c +++
>>> b/drivers/devfreq/tegra30-devfreq.c @@ -314,7 +314,8 @@ static void
>>> tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra, }
>>>  
>>>  static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq
>>> *tegra,
>>> -					   struct
>>> tegra_devfreq_device *dev)
>>> +					   struct
>>> tegra_devfreq_device *dev,
>>> +					   unsigned long freq)
>>>  {
>>>  	unsigned long avg_threshold, lower, upper;
>>>  
>>> @@ -323,6 +324,15 @@ static void
>>> tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
>>> avg_threshold = dev->config->avg_dependency_threshold;
>>> avg_threshold = avg_threshold * ACTMON_SAMPLING_PERIOD; 
>>> +	/*
>>> +	 * If cumulative EMC frequency selection is higher than the
>>> +	 * device's, then there is no need to set upper watermark
>>> to
>>> +	 * a lower value because it will result in unnecessary
>>> upper
>>> +	 * interrupts.
>>> +	 */
>>> +	if (freq * ACTMON_SAMPLING_PERIOD > upper)
>>> +		upper = freq * ACTMON_SAMPLING_PERIOD;  
>>
>> Also, 'upper value is used on the patch5. You can combine this code
>> to patch5 or if this patch depends on the cpu notifier, you can
>> combine it to the patch of adding cpu notifier without separate patch.
> 
> Well okay, I'll try to squash some of the patches in the next revision.
> Usually I'm receiving comments in the other direction, asking to
> separate patches into smaller changes ;) So that's more a personal
> preference of each maintainer, I'd say.
> 

Right. We have to make the patch with atomic attribute.
But, if there are patches which touch the same code
in the same patchset. We can squash or do refactorig
of this code.

And also, if possible, I'd like you to make the patch
list according to the role of patch. For example,
the patches related to the 'watermark' could be sequentially
listed. But, it is not forced opinion. If just possible.



-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 12/24] PM / devfreq: tegra30: Inline all one-line functions
  2019-07-19  1:27             ` Chanwoo Choi
@ 2019-07-19  2:14               ` Dmitry Osipenko
  2019-07-19  6:01                 ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19  2:14 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

В Fri, 19 Jul 2019 10:27:16 +0900
Chanwoo Choi <cw00.choi@samsung.com> пишет:

> On 19. 7. 19. 오전 10:24, Chanwoo Choi wrote:
> > On 19. 7. 19. 오전 10:22, Dmitry Osipenko wrote:  
> >> В Thu, 18 Jul 2019 18:09:05 +0900
> >> Chanwoo Choi <cw00.choi@samsung.com> пишет:
> >>  
> >>> On 19. 7. 16. 오후 10:35, Dmitry Osipenko wrote:  
> >>>> 16.07.2019 15:26, Chanwoo Choi пишет:    
> >>>>> Hi Dmitry,
> >>>>>
> >>>>> I'm not sure that it is necessary.
> >>>>> As I knew, usally, the 'inline' is used on header file
> >>>>> to define the empty functions.
> >>>>>
> >>>>> Do we have to change it with 'inline' keyword?    
> >>>>
> >>>> The 'inline' attribute tells compiler that instead of jumping
> >>>> into the function, it should take the function's code and
> >>>> replace the function's invocation with that code. This is done
> >>>> in order to help compiler optimize code properly, please see
> >>>> [1]. There is absolutely no need to create a function call into
> >>>> a function that consists of a single instruction.
> >>>>
> >>>> [1] https://gcc.gnu.org/onlinedocs/gcc-9.1.0/gcc/Inline.html
> >>>>     
> >>>
> >>> If you want to add 'inline' keyword, I recommend that 
> >>> you better to remove the modified function in this patch
> >>> and then just call the 'write_relaxed or read_relaxed' function
> >>> directly. It is same result when using inline keyword.  
> >>
> >> That could be done, but it makes code less readable.
> >>
> >> See the difference:
> >>
> >> device_writel(dev, ACTMON_INTR_STATUS_CLEAR,
> >> ACTMON_DEV_INTR_STATUS);
> >>
> >> writel_relaxed(ACTMON_INTR_STATUS_CLEAR,
> >> 	       dev->regs + ACTMON_DEV_INTR_STATUS);  
> > 
> > No problem if you add the detailed comment and you want to use
> > the 'inline' keyword.  
> 
> Basically, I think that 'inline' keyword is not necessary.

Sure, but I'm finding that it's always nicer to explicitly inline a very
simple functions because compiler may not do it properly itself in some
cases.

> But if you want to use 'inline' keyword, I recommend
> that call the 'write_relaxed or read_relaxed' function directly
> with detailed description. 

Could you please reword this sentence? Not sure that I'm understanding
it correctly.

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 20/24] PM / devfreq: tegra30: Optimize upper average watermark selection
  2019-07-19  2:06       ` Chanwoo Choi
@ 2019-07-19  2:21         ` Dmitry Osipenko
  2019-07-19  6:09           ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19  2:21 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

В Fri, 19 Jul 2019 11:06:05 +0900
Chanwoo Choi <cw00.choi@samsung.com> пишет:

> On 19. 7. 19. 오전 10:59, Dmitry Osipenko wrote:
> > В Fri, 19 Jul 2019 10:36:30 +0900
> > Chanwoo Choi <cw00.choi@samsung.com> пишет:
> >   
> >> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:  
> >>> I noticed that CPU may be crossing the dependency threshold very
> >>> frequently for some workloads and this results in a lot of
> >>> interrupts which could be avoided if MCALL client is keeping
> >>> actual EMC frequency at a higher rate.
> >>>
> >>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
> >>> ---
> >>>  drivers/devfreq/tegra30-devfreq.c | 23 ++++++++++++++++++-----
> >>>  1 file changed, 18 insertions(+), 5 deletions(-)
> >>>
> >>> diff --git a/drivers/devfreq/tegra30-devfreq.c
> >>> b/drivers/devfreq/tegra30-devfreq.c index
> >>> c3cf87231d25..4d582809acb6 100644 ---
> >>> a/drivers/devfreq/tegra30-devfreq.c +++
> >>> b/drivers/devfreq/tegra30-devfreq.c @@ -314,7 +314,8 @@ static
> >>> void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra, }
> >>>  
> >>>  static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq
> >>> *tegra,
> >>> -					   struct
> >>> tegra_devfreq_device *dev)
> >>> +					   struct
> >>> tegra_devfreq_device *dev,
> >>> +					   unsigned long freq)
> >>>  {
> >>>  	unsigned long avg_threshold, lower, upper;
> >>>  
> >>> @@ -323,6 +324,15 @@ static void
> >>> tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
> >>> avg_threshold = dev->config->avg_dependency_threshold;
> >>> avg_threshold = avg_threshold * ACTMON_SAMPLING_PERIOD; 
> >>> +	/*
> >>> +	 * If cumulative EMC frequency selection is higher than
> >>> the
> >>> +	 * device's, then there is no need to set upper watermark
> >>> to
> >>> +	 * a lower value because it will result in unnecessary
> >>> upper
> >>> +	 * interrupts.
> >>> +	 */
> >>> +	if (freq * ACTMON_SAMPLING_PERIOD > upper)
> >>> +		upper = freq * ACTMON_SAMPLING_PERIOD;    
> >>
> >> Also, 'upper value is used on the patch5. You can combine this code
> >> to patch5 or if this patch depends on the cpu notifier, you can
> >> combine it to the patch of adding cpu notifier without separate
> >> patch.  
> > 
> > Well okay, I'll try to squash some of the patches in the next
> > revision. Usually I'm receiving comments in the other direction,
> > asking to separate patches into smaller changes ;) So that's more a
> > personal preference of each maintainer, I'd say.
> >   
> 
> Right. We have to make the patch with atomic attribute.
> But, if there are patches which touch the same code
> in the same patchset. We can squash or do refactorig
> of this code.

The main benefit of having smaller logical changes is that when there is
a bug, it's easier to narrow down the offending change using bisection.
And it's just easier to review smaller patches, of course.

> And also, if possible, I'd like you to make the patch
> list according to the role of patch. For example,
> the patches related to the 'watermark' could be sequentially
> listed. But, it is not forced opinion. If just possible.

Okay, will take this into account.


^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 12/24] PM / devfreq: tegra30: Inline all one-line functions
  2019-07-19  2:14               ` Dmitry Osipenko
@ 2019-07-19  6:01                 ` Chanwoo Choi
  2019-07-19 16:52                   ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-19  6:01 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 19. 오전 11:14, Dmitry Osipenko wrote:
> В Fri, 19 Jul 2019 10:27:16 +0900
> Chanwoo Choi <cw00.choi@samsung.com> пишет:
> 
>> On 19. 7. 19. 오전 10:24, Chanwoo Choi wrote:
>>> On 19. 7. 19. 오전 10:22, Dmitry Osipenko wrote:  
>>>> В Thu, 18 Jul 2019 18:09:05 +0900
>>>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>>>  
>>>>> On 19. 7. 16. 오후 10:35, Dmitry Osipenko wrote:  
>>>>>> 16.07.2019 15:26, Chanwoo Choi пишет:    
>>>>>>> Hi Dmitry,
>>>>>>>
>>>>>>> I'm not sure that it is necessary.
>>>>>>> As I knew, usally, the 'inline' is used on header file
>>>>>>> to define the empty functions.
>>>>>>>
>>>>>>> Do we have to change it with 'inline' keyword?    
>>>>>>
>>>>>> The 'inline' attribute tells compiler that instead of jumping
>>>>>> into the function, it should take the function's code and
>>>>>> replace the function's invocation with that code. This is done
>>>>>> in order to help compiler optimize code properly, please see
>>>>>> [1]. There is absolutely no need to create a function call into
>>>>>> a function that consists of a single instruction.
>>>>>>
>>>>>> [1] https://gcc.gnu.org/onlinedocs/gcc-9.1.0/gcc/Inline.html
>>>>>>     
>>>>>
>>>>> If you want to add 'inline' keyword, I recommend that 
>>>>> you better to remove the modified function in this patch
>>>>> and then just call the 'write_relaxed or read_relaxed' function
>>>>> directly. It is same result when using inline keyword.  
>>>>
>>>> That could be done, but it makes code less readable.
>>>>
>>>> See the difference:
>>>>
>>>> device_writel(dev, ACTMON_INTR_STATUS_CLEAR,
>>>> ACTMON_DEV_INTR_STATUS);
>>>>
>>>> writel_relaxed(ACTMON_INTR_STATUS_CLEAR,
>>>> 	       dev->regs + ACTMON_DEV_INTR_STATUS);  
>>>
>>> No problem if you add the detailed comment and you want to use
>>> the 'inline' keyword.  
>>
>> Basically, I think that 'inline' keyword is not necessary.
> 
> Sure, but I'm finding that it's always nicer to explicitly inline a very
> simple functions because compiler may not do it properly itself in some
> cases.
> 
>> But if you want to use 'inline' keyword, I recommend
>> that call the 'write_relaxed or read_relaxed' function directly
>> with detailed description. 
> 
> Could you please reword this sentence? Not sure that I'm understanding
> it correctly.
> 

If you want to used 'inline' keyword,
Instead, I recommend that remove 'actmon_readl/writel' wrapper functions
and then you calls 'write_relaxed or read_relaxed' function directly
with detailed description.

-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 20/24] PM / devfreq: tegra30: Optimize upper average watermark selection
  2019-07-19  2:21         ` Dmitry Osipenko
@ 2019-07-19  6:09           ` Chanwoo Choi
  2019-07-19  6:11             ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-19  6:09 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 19. 오전 11:21, Dmitry Osipenko wrote:
> В Fri, 19 Jul 2019 11:06:05 +0900
> Chanwoo Choi <cw00.choi@samsung.com> пишет:
> 
>> On 19. 7. 19. 오전 10:59, Dmitry Osipenko wrote:
>>> В Fri, 19 Jul 2019 10:36:30 +0900
>>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>>   
>>>> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:  
>>>>> I noticed that CPU may be crossing the dependency threshold very
>>>>> frequently for some workloads and this results in a lot of
>>>>> interrupts which could be avoided if MCALL client is keeping
>>>>> actual EMC frequency at a higher rate.
>>>>>
>>>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>>>> ---
>>>>>  drivers/devfreq/tegra30-devfreq.c | 23 ++++++++++++++++++-----
>>>>>  1 file changed, 18 insertions(+), 5 deletions(-)
>>>>>
>>>>> diff --git a/drivers/devfreq/tegra30-devfreq.c
>>>>> b/drivers/devfreq/tegra30-devfreq.c index
>>>>> c3cf87231d25..4d582809acb6 100644 ---
>>>>> a/drivers/devfreq/tegra30-devfreq.c +++
>>>>> b/drivers/devfreq/tegra30-devfreq.c @@ -314,7 +314,8 @@ static
>>>>> void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra, }
>>>>>  
>>>>>  static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq
>>>>> *tegra,
>>>>> -					   struct
>>>>> tegra_devfreq_device *dev)
>>>>> +					   struct
>>>>> tegra_devfreq_device *dev,
>>>>> +					   unsigned long freq)
>>>>>  {
>>>>>  	unsigned long avg_threshold, lower, upper;
>>>>>  
>>>>> @@ -323,6 +324,15 @@ static void
>>>>> tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
>>>>> avg_threshold = dev->config->avg_dependency_threshold;
>>>>> avg_threshold = avg_threshold * ACTMON_SAMPLING_PERIOD; 
>>>>> +	/*
>>>>> +	 * If cumulative EMC frequency selection is higher than
>>>>> the
>>>>> +	 * device's, then there is no need to set upper watermark
>>>>> to
>>>>> +	 * a lower value because it will result in unnecessary
>>>>> upper
>>>>> +	 * interrupts.
>>>>> +	 */
>>>>> +	if (freq * ACTMON_SAMPLING_PERIOD > upper)
>>>>> +		upper = freq * ACTMON_SAMPLING_PERIOD;    
>>>>
>>>> Also, 'upper value is used on the patch5. You can combine this code
>>>> to patch5 or if this patch depends on the cpu notifier, you can
>>>> combine it to the patch of adding cpu notifier without separate
>>>> patch.  
>>>
>>> Well okay, I'll try to squash some of the patches in the next
>>> revision. Usually I'm receiving comments in the other direction,
>>> asking to separate patches into smaller changes ;) So that's more a
>>> personal preference of each maintainer, I'd say.
>>>   
>>
>> Right. We have to make the patch with atomic attribute.
>> But, if there are patches which touch the same code
>> in the same patchset. We can squash or do refactorig
>> of this code.
> 
> The main benefit of having smaller logical changes is that when there is
> a bug, it's easier to narrow down the offending change using bisection.
> And it's just easier to review smaller patches, of course.

I agree that the patch should contain the atomic feature.
To remove the some communication confusion between us,
I don't mean that you have to merge patches to only one patch.

It is important to remove the following two cases on the same patchset.

1. the front patch adds the code and then later patch remove the added code.
2. the front patch changes the code and the later patch again modified
   the changed code of the front patch


> 
>> And also, if possible, I'd like you to make the patch
>> list according to the role of patch. For example,
>> the patches related to the 'watermark' could be sequentially
>> listed. But, it is not forced opinion. If just possible.
> 
> Okay, will take this into account.
> 
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 20/24] PM / devfreq: tegra30: Optimize upper average watermark selection
  2019-07-19  6:09           ` Chanwoo Choi
@ 2019-07-19  6:11             ` Chanwoo Choi
  2019-07-19 17:52               ` Dmitry Osipenko
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-19  6:11 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 19. 오후 3:09, Chanwoo Choi wrote:
> On 19. 7. 19. 오전 11:21, Dmitry Osipenko wrote:
>> В Fri, 19 Jul 2019 11:06:05 +0900
>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>
>>> On 19. 7. 19. 오전 10:59, Dmitry Osipenko wrote:
>>>> В Fri, 19 Jul 2019 10:36:30 +0900
>>>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>>>   
>>>>> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:  
>>>>>> I noticed that CPU may be crossing the dependency threshold very
>>>>>> frequently for some workloads and this results in a lot of
>>>>>> interrupts which could be avoided if MCALL client is keeping
>>>>>> actual EMC frequency at a higher rate.
>>>>>>
>>>>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>>>>> ---
>>>>>>  drivers/devfreq/tegra30-devfreq.c | 23 ++++++++++++++++++-----
>>>>>>  1 file changed, 18 insertions(+), 5 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/devfreq/tegra30-devfreq.c
>>>>>> b/drivers/devfreq/tegra30-devfreq.c index
>>>>>> c3cf87231d25..4d582809acb6 100644 ---
>>>>>> a/drivers/devfreq/tegra30-devfreq.c +++
>>>>>> b/drivers/devfreq/tegra30-devfreq.c @@ -314,7 +314,8 @@ static
>>>>>> void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra, }
>>>>>>  
>>>>>>  static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq
>>>>>> *tegra,
>>>>>> -					   struct
>>>>>> tegra_devfreq_device *dev)
>>>>>> +					   struct
>>>>>> tegra_devfreq_device *dev,
>>>>>> +					   unsigned long freq)
>>>>>>  {
>>>>>>  	unsigned long avg_threshold, lower, upper;
>>>>>>  
>>>>>> @@ -323,6 +324,15 @@ static void
>>>>>> tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
>>>>>> avg_threshold = dev->config->avg_dependency_threshold;
>>>>>> avg_threshold = avg_threshold * ACTMON_SAMPLING_PERIOD; 
>>>>>> +	/*
>>>>>> +	 * If cumulative EMC frequency selection is higher than
>>>>>> the
>>>>>> +	 * device's, then there is no need to set upper watermark
>>>>>> to
>>>>>> +	 * a lower value because it will result in unnecessary
>>>>>> upper
>>>>>> +	 * interrupts.
>>>>>> +	 */
>>>>>> +	if (freq * ACTMON_SAMPLING_PERIOD > upper)
>>>>>> +		upper = freq * ACTMON_SAMPLING_PERIOD;    
>>>>>
>>>>> Also, 'upper value is used on the patch5. You can combine this code
>>>>> to patch5 or if this patch depends on the cpu notifier, you can
>>>>> combine it to the patch of adding cpu notifier without separate
>>>>> patch.  
>>>>
>>>> Well okay, I'll try to squash some of the patches in the next
>>>> revision. Usually I'm receiving comments in the other direction,
>>>> asking to separate patches into smaller changes ;) So that's more a
>>>> personal preference of each maintainer, I'd say.
>>>>   
>>>
>>> Right. We have to make the patch with atomic attribute.
>>> But, if there are patches which touch the same code
>>> in the same patchset. We can squash or do refactorig
>>> of this code.
>>
>> The main benefit of having smaller logical changes is that when there is
>> a bug, it's easier to narrow down the offending change using bisection.
>> And it's just easier to review smaller patches, of course.
> 
> I agree that the patch should contain the atomic feature.
> To remove the some communication confusion between us,
> I don't mean that you have to merge patches to only one patch.

If each patch has the atomic attribute, it have to be made as the separate patch.
But, if some patches are included in the the following two case,
can combine patches to one patch.

> 
> It is important to remove the following two cases on the same patchset.
> 
> 1. the front patch adds the code and then later patch remove the added code.
> 2. the front patch changes the code and the later patch again modified
>    the changed code of the front patch
> 
> 
>>
>>> And also, if possible, I'd like you to make the patch
>>> list according to the role of patch. For example,
>>> the patches related to the 'watermark' could be sequentially
>>> listed. But, it is not forced opinion. If just possible.
>>
>> Okay, will take this into account.
>>
>>
>>
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 21/24] PM / devfreq: tegra30: Synchronize average count on target's update
  2019-07-19  1:40       ` Chanwoo Choi
@ 2019-07-19 16:46         ` Dmitry Osipenko
  0 siblings, 0 replies; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19 16:46 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

19.07.2019 4:40, Chanwoo Choi пишет:
> On 19. 7. 19. 오전 9:31, Dmitry Osipenko wrote:
>> В Thu, 18 Jul 2019 19:15:54 +0900
>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>
>>> On 19. 7. 8. 오전 7:33, Dmitry Osipenko wrote:
>>>> The average count may get out of sync if interrupt was disabled /
>>>> avoided for a long time due to upper watermark optimization, hence
>>>> it should be re-synced on each target's update to ensure that
>>>> watermarks are set up correctly on EMC rate-change notification and
>>>> that a correct frequency is selected for device.
>>>>
>>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>>> ---
>>>>  drivers/devfreq/tegra30-devfreq.c | 30
>>>> ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
>>>>
>>>> diff --git a/drivers/devfreq/tegra30-devfreq.c
>>>> b/drivers/devfreq/tegra30-devfreq.c index
>>>> 4d582809acb6..8a674fad26be 100644 ---
>>>> a/drivers/devfreq/tegra30-devfreq.c +++
>>>> b/drivers/devfreq/tegra30-devfreq.c @@ -466,11 +466,41 @@ static
>>>> void actmon_isr_device(struct tegra_devfreq *tegra,
>>>> dev->boost_freq, cpufreq_get(0)); }
>>>>  
>>>> +static void tegra_devfreq_sync_avg_count(struct tegra_devfreq
>>>> *tegra,
>>>> +					 struct
>>>> tegra_devfreq_device *dev) +{
>>>> +	u32 avg_count, avg_freq, old_upper, new_upper;
>>>> +
>>>> +	avg_count = device_readl(dev, ACTMON_DEV_AVG_COUNT);
>>>> +	avg_freq = avg_count / ACTMON_SAMPLING_PERIOD;
>>>> +
>>>> +	old_upper = tegra_actmon_upper_freq(tegra, dev->avg_freq);
>>>> +	new_upper = tegra_actmon_upper_freq(tegra, avg_freq);
>>>> +
>>>> +	/* similar to ISR, see comments in actmon_isr_device() */
>>>> +	if (old_upper != new_upper) {
>>>> +		dev->avg_freq = avg_freq;
>>>> +		dev->boost_freq = 0;
>>>> +	}
>>>> +}
>>>> +
>>>>  static unsigned long actmon_update_target(struct tegra_devfreq
>>>> *tegra, struct tegra_devfreq_device *dev)
>>>>  {
>>>>  	unsigned long target_freq;
>>>>  
>>>> +	/*
>>>> +	 * The avg_count / avg_freq is getting snapshoted on
>>>> device's
>>>> +	 * interrupt, but there are cases where actual value need
>>>> to
>>>> +	 * be utilized on target's update, like CPUFreq boosting
>>>> and
>>>> +	 * overriding the min freq
>>>> via /sys/class/devfreq/devfreq0/min_freq
>>>> +	 * because we're optimizing the upper watermark based on
>>>> the
>>>> +	 * actual EMC frequency. This means that interrupt may be
>>>> +	 * inactive for a long time and thus making snapshoted
>>>> value
>>>> +	 * outdated.
>>>> +	 */
>>>> +	tegra_devfreq_sync_avg_count(tegra, dev);  
>>>
>>> I think that you don't need to add the separate function to calculate
>>> the 'dev->avg_freq'. It is enough with your detailed comment to add
>>> this code in this function.
>>
>> The separate function is indeed not mandatory here, but I'm finding that
>> it usually makes easier to read and follow the code when it is properly
>> split up into logical blocks. Don't you agree?
> 
> It is right to make the separate function if function is too long or 
> function is called on the multiple points.
> 
> But, in this case, I think that it is enough to add this code
> to the actmon_update_target() because you already added the detailed comment. 
> 
> It is enough to understand the role of this code with your comment.

That's another personal preference, but I'm fine with yours variant as
well. Will change this in the next version.

>>
>>>> +
>>>>  	target_freq = min(dev->avg_freq + dev->boost_freq,
>>>> KHZ_MAX); target_freq = tegra_actmon_account_cpu_freq(tegra, dev,
>>>> target_freq); 
>>>>   
>>>
>>> And also, is it impossible to squash this patch with patch19/patch20?
>>>
>>
>> It should be possible to squash this patch with #20, but wouldn't
>> be better to keep changes in the chronological order? It's also better
>> to keep changes separate simply to aid bisection in case of a problem.
>>
>>
>>
> 
> 


^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 12/24] PM / devfreq: tegra30: Inline all one-line functions
  2019-07-19  6:01                 ` Chanwoo Choi
@ 2019-07-19 16:52                   ` Dmitry Osipenko
  0 siblings, 0 replies; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19 16:52 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

19.07.2019 9:01, Chanwoo Choi пишет:
> On 19. 7. 19. 오전 11:14, Dmitry Osipenko wrote:
>> В Fri, 19 Jul 2019 10:27:16 +0900
>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>
>>> On 19. 7. 19. 오전 10:24, Chanwoo Choi wrote:
>>>> On 19. 7. 19. 오전 10:22, Dmitry Osipenko wrote:  
>>>>> В Thu, 18 Jul 2019 18:09:05 +0900
>>>>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>>>>  
>>>>>> On 19. 7. 16. 오후 10:35, Dmitry Osipenko wrote:  
>>>>>>> 16.07.2019 15:26, Chanwoo Choi пишет:    
>>>>>>>> Hi Dmitry,
>>>>>>>>
>>>>>>>> I'm not sure that it is necessary.
>>>>>>>> As I knew, usally, the 'inline' is used on header file
>>>>>>>> to define the empty functions.
>>>>>>>>
>>>>>>>> Do we have to change it with 'inline' keyword?    
>>>>>>>
>>>>>>> The 'inline' attribute tells compiler that instead of jumping
>>>>>>> into the function, it should take the function's code and
>>>>>>> replace the function's invocation with that code. This is done
>>>>>>> in order to help compiler optimize code properly, please see
>>>>>>> [1]. There is absolutely no need to create a function call into
>>>>>>> a function that consists of a single instruction.
>>>>>>>
>>>>>>> [1] https://gcc.gnu.org/onlinedocs/gcc-9.1.0/gcc/Inline.html
>>>>>>>     
>>>>>>
>>>>>> If you want to add 'inline' keyword, I recommend that 
>>>>>> you better to remove the modified function in this patch
>>>>>> and then just call the 'write_relaxed or read_relaxed' function
>>>>>> directly. It is same result when using inline keyword.  
>>>>>
>>>>> That could be done, but it makes code less readable.
>>>>>
>>>>> See the difference:
>>>>>
>>>>> device_writel(dev, ACTMON_INTR_STATUS_CLEAR,
>>>>> ACTMON_DEV_INTR_STATUS);
>>>>>
>>>>> writel_relaxed(ACTMON_INTR_STATUS_CLEAR,
>>>>> 	       dev->regs + ACTMON_DEV_INTR_STATUS);  
>>>>
>>>> No problem if you add the detailed comment and you want to use
>>>> the 'inline' keyword.  
>>>
>>> Basically, I think that 'inline' keyword is not necessary.
>>
>> Sure, but I'm finding that it's always nicer to explicitly inline a very
>> simple functions because compiler may not do it properly itself in some
>> cases.
>>
>>> But if you want to use 'inline' keyword, I recommend
>>> that call the 'write_relaxed or read_relaxed' function directly
>>> with detailed description. 
>>
>> Could you please reword this sentence? Not sure that I'm understanding
>> it correctly.
>>
> 
> If you want to used 'inline' keyword,
> Instead, I recommend that remove 'actmon_readl/writel' wrapper functions
> and then you calls 'write_relaxed or read_relaxed' function directly
> with detailed description.
> 

This is a step into a wrong direction. Look, there is no need for extra
comments and the code is clean with the variant I'm proposing, while you
are asking to make code less readable and then paper that over with
comments.

I'll probably just drop this, #11 and #17 for now. Since these patches
and not essential for the functionality of the driver and they are
raising more questions than should be. Maybe we could get back to them
at some point later.

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 11/24] PM / devfreq: tegra30: Add debug messages
  2019-07-19  1:22               ` Chanwoo Choi
@ 2019-07-19 17:10                 ` Dmitry Osipenko
  0 siblings, 0 replies; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19 17:10 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

19.07.2019 4:22, Chanwoo Choi пишет:
> On 19. 7. 19. 오전 10:13, Dmitry Osipenko wrote:
>> В Thu, 18 Jul 2019 18:07:05 +0900
>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>
>>> On 19. 7. 18. 오전 12:46, Dmitry Osipenko wrote:
>>>> 17.07.2019 9:45, Chanwoo Choi пишет:  
>>>>> On 19. 7. 16. 오후 10:26, Dmitry Osipenko wrote:  
>>>>>> 16.07.2019 15:23, Chanwoo Choi пишет:  
>>>>>>> Hi Dmitry,
>>>>>>>
>>>>>>> Usually, the kernel log print for all users
>>>>>>> such as changing the frequency, fail or success.
>>>>>>>
>>>>>>> But, if the log just show the register dump,
>>>>>>> it is not useful for all users. It is just used
>>>>>>> for only specific developer.
>>>>>>>
>>>>>>> I recommend that you better to add more exception handling
>>>>>>> code on many points instead of just showing the register dump.  
>>>>>>
>>>>>> The debug messages are not users, but for developers. Yes, I
>>>>>> primarily made the debugging to be useful for myself and will be
>>>>>> happy to change the way debugging is done if there will be any
>>>>>> other active developer for this driver. The registers dump is
>>>>>> more than enough in order to understand what's going on, I don't
>>>>>> see any real need to change anything here for now.  
>>>>>
>>>>> Basically, we have to develop code and add the log for anyone.
>>>>> As you commented, even if there are no other developer, we never
>>>>> guarantee this assumption forever. And also, if added debug message
>>>>> for only you, you can add them when testing it temporarily.
>>>>>
>>>>> If you want to add the just register dump log for you,
>>>>> I can't agree. Once again, I hope that anyone understand
>>>>> the meaning of debug message as much possible as.
>>>>>  
>>>>
>>>> The registers dump should be good for everyone because it's a
>>>> self-explanatory information for anyone who is familiar with the
>>>> hardware. I don't think there is a need for anything else than what
>>>> is proposed in this patch, at least for now. I also simply don't
>>>> see any other better way to debug the state of this particular
>>>> hardware, again this logging is for the driver developers and not
>>>> for users.
>>>>
>>>> Initially, I was temporarily adding the debug messages. Now they are
>>>> pretty much mandatory for verifying that driver is working
>>>> properly. And of course the debugging messages got into the shape
>>>> of this patch after several iterations of refinements. So again, I
>>>> suppose that this should be good enough for everyone who is
>>>> familiar with the hardware. And of course I'm open to the
>>>> constructive suggestions, the debugging aid is not an ABI and could
>>>> be changed/improved at any time.
>>>>
>>>> You're suggesting to break down the debugging into several smaller
>>>> pieces, but I'm finding that as not a constructive suggestion
>>>> because the information about the full hardware state is actually
>>>> necessary for the productive debugging.
>>>>
>>>>   
>>>
>>> Sorry for that as I saie, I cannot agree this patch. In my case,
>>> I don't understand what is meaning of register dump of this patch.
>>> I knew that just register dump are useful for real developer.
>>
>> It's not only a registers dump, as you may see there is also a dump of
>> other properties like boosting value, OPPs selection and etc.
>>
>> It looks to me that you're also missing important detail that debug
>> messages are compiled out unless DEBUG is defined for the drivers
>> build. So in order to get the debug message a user shall explicitly add
>> #define DEBUG macro to the code or enable debug messages globally in
>> the kernel's config. There is also an option for dynamic debug messages
>> in the kernel, but it doesn't matter now because all these messages are
>> turned into tracepoints later in the patch #17.
> 
> 
> Right. But, this patch could not the split up between register dump and others.
> As I said repeatly, I hope to add the log that anyone can understand. 

I'm afraid that's a way too big request to make it universal for anyone
or I'm just failing to understand what you're asking for. In my opinion
it's already understandable by everybody who is really interested in
debugging of this driver. I really don't see how to make it better, in
my opinion it's already ideal.

I'll drop the debug patches from the series in the next revision and
keep them locally for now. Maybe we could get back to this later sometime.

>>
>>> If you want to show the register dump, you better to add some feature
>>> with debugfs for devfreq framework in order to read the register dump.
>>> As I knew, sound framework (alsa) has the similar feature for checking
>>> the register dump.
>>>
>>
>> The intent was to have an option for dynamic debugging of the driver and
>> initially debug messages were good enough, but then it became not enough
>> and hence the debug messages were turned into tracepoints in the patch
>> #17. Would it be acceptable to squash this patch and #17?


^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 20/24] PM / devfreq: tegra30: Optimize upper average watermark selection
  2019-07-19  6:11             ` Chanwoo Choi
@ 2019-07-19 17:52               ` Dmitry Osipenko
  2019-07-24 11:17                 ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Dmitry Osipenko @ 2019-07-19 17:52 UTC (permalink / raw)
  To: Chanwoo Choi
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

19.07.2019 9:11, Chanwoo Choi пишет:
> On 19. 7. 19. 오후 3:09, Chanwoo Choi wrote:
>> On 19. 7. 19. 오전 11:21, Dmitry Osipenko wrote:
>>> В Fri, 19 Jul 2019 11:06:05 +0900
>>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>>
>>>> On 19. 7. 19. 오전 10:59, Dmitry Osipenko wrote:
>>>>> В Fri, 19 Jul 2019 10:36:30 +0900
>>>>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>>>>   
>>>>>> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:  
>>>>>>> I noticed that CPU may be crossing the dependency threshold very
>>>>>>> frequently for some workloads and this results in a lot of
>>>>>>> interrupts which could be avoided if MCALL client is keeping
>>>>>>> actual EMC frequency at a higher rate.
>>>>>>>
>>>>>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>>>>>> ---
>>>>>>>  drivers/devfreq/tegra30-devfreq.c | 23 ++++++++++++++++++-----
>>>>>>>  1 file changed, 18 insertions(+), 5 deletions(-)
>>>>>>>
>>>>>>> diff --git a/drivers/devfreq/tegra30-devfreq.c
>>>>>>> b/drivers/devfreq/tegra30-devfreq.c index
>>>>>>> c3cf87231d25..4d582809acb6 100644 ---
>>>>>>> a/drivers/devfreq/tegra30-devfreq.c +++
>>>>>>> b/drivers/devfreq/tegra30-devfreq.c @@ -314,7 +314,8 @@ static
>>>>>>> void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra, }
>>>>>>>  
>>>>>>>  static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq
>>>>>>> *tegra,
>>>>>>> -					   struct
>>>>>>> tegra_devfreq_device *dev)
>>>>>>> +					   struct
>>>>>>> tegra_devfreq_device *dev,
>>>>>>> +					   unsigned long freq)
>>>>>>>  {
>>>>>>>  	unsigned long avg_threshold, lower, upper;
>>>>>>>  
>>>>>>> @@ -323,6 +324,15 @@ static void
>>>>>>> tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
>>>>>>> avg_threshold = dev->config->avg_dependency_threshold;
>>>>>>> avg_threshold = avg_threshold * ACTMON_SAMPLING_PERIOD; 
>>>>>>> +	/*
>>>>>>> +	 * If cumulative EMC frequency selection is higher than
>>>>>>> the
>>>>>>> +	 * device's, then there is no need to set upper watermark
>>>>>>> to
>>>>>>> +	 * a lower value because it will result in unnecessary
>>>>>>> upper
>>>>>>> +	 * interrupts.
>>>>>>> +	 */
>>>>>>> +	if (freq * ACTMON_SAMPLING_PERIOD > upper)
>>>>>>> +		upper = freq * ACTMON_SAMPLING_PERIOD;    
>>>>>>
>>>>>> Also, 'upper value is used on the patch5. You can combine this code
>>>>>> to patch5 or if this patch depends on the cpu notifier, you can
>>>>>> combine it to the patch of adding cpu notifier without separate
>>>>>> patch.  
>>>>>
>>>>> Well okay, I'll try to squash some of the patches in the next
>>>>> revision. Usually I'm receiving comments in the other direction,
>>>>> asking to separate patches into smaller changes ;) So that's more a
>>>>> personal preference of each maintainer, I'd say.
>>>>>   
>>>>
>>>> Right. We have to make the patch with atomic attribute.
>>>> But, if there are patches which touch the same code
>>>> in the same patchset. We can squash or do refactorig
>>>> of this code.
>>>
>>> The main benefit of having smaller logical changes is that when there is
>>> a bug, it's easier to narrow down the offending change using bisection.
>>> And it's just easier to review smaller patches, of course.
>>
>> I agree that the patch should contain the atomic feature.
>> To remove the some communication confusion between us,
>> I don't mean that you have to merge patches to only one patch.
> 
> If each patch has the atomic attribute, it have to be made as the separate patch.
> But, if some patches are included in the the following two case,
> can combine patches to one patch.
> 
>>
>> It is important to remove the following two cases on the same patchset.
>>
>> 1. the front patch adds the code and then later patch remove the added code.

Okay, I agree that this is applicable to patch #11.

>> 2. the front patch changes the code and the later patch again modified
>>    the changed code of the front patch

If patch A adds a new feature and then patch B adds another new feature
on top of A, do you consider each of these patches as atomic?

[snip]

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 20/24] PM / devfreq: tegra30: Optimize upper average watermark selection
  2019-07-19 17:52               ` Dmitry Osipenko
@ 2019-07-24 11:17                 ` Chanwoo Choi
  2019-07-24 11:19                   ` Chanwoo Choi
  0 siblings, 1 reply; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-24 11:17 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 20. 오전 2:52, Dmitry Osipenko wrote:
> 19.07.2019 9:11, Chanwoo Choi пишет:
>> On 19. 7. 19. 오후 3:09, Chanwoo Choi wrote:
>>> On 19. 7. 19. 오전 11:21, Dmitry Osipenko wrote:
>>>> В Fri, 19 Jul 2019 11:06:05 +0900
>>>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>>>
>>>>> On 19. 7. 19. 오전 10:59, Dmitry Osipenko wrote:
>>>>>> В Fri, 19 Jul 2019 10:36:30 +0900
>>>>>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>>>>>   
>>>>>>> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:  
>>>>>>>> I noticed that CPU may be crossing the dependency threshold very
>>>>>>>> frequently for some workloads and this results in a lot of
>>>>>>>> interrupts which could be avoided if MCALL client is keeping
>>>>>>>> actual EMC frequency at a higher rate.
>>>>>>>>
>>>>>>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>>>>>>> ---
>>>>>>>>  drivers/devfreq/tegra30-devfreq.c | 23 ++++++++++++++++++-----
>>>>>>>>  1 file changed, 18 insertions(+), 5 deletions(-)
>>>>>>>>
>>>>>>>> diff --git a/drivers/devfreq/tegra30-devfreq.c
>>>>>>>> b/drivers/devfreq/tegra30-devfreq.c index
>>>>>>>> c3cf87231d25..4d582809acb6 100644 ---
>>>>>>>> a/drivers/devfreq/tegra30-devfreq.c +++
>>>>>>>> b/drivers/devfreq/tegra30-devfreq.c @@ -314,7 +314,8 @@ static
>>>>>>>> void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra, }
>>>>>>>>  
>>>>>>>>  static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq
>>>>>>>> *tegra,
>>>>>>>> -					   struct
>>>>>>>> tegra_devfreq_device *dev)
>>>>>>>> +					   struct
>>>>>>>> tegra_devfreq_device *dev,
>>>>>>>> +					   unsigned long freq)
>>>>>>>>  {
>>>>>>>>  	unsigned long avg_threshold, lower, upper;
>>>>>>>>  
>>>>>>>> @@ -323,6 +324,15 @@ static void
>>>>>>>> tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
>>>>>>>> avg_threshold = dev->config->avg_dependency_threshold;
>>>>>>>> avg_threshold = avg_threshold * ACTMON_SAMPLING_PERIOD; 
>>>>>>>> +	/*
>>>>>>>> +	 * If cumulative EMC frequency selection is higher than
>>>>>>>> the
>>>>>>>> +	 * device's, then there is no need to set upper watermark
>>>>>>>> to
>>>>>>>> +	 * a lower value because it will result in unnecessary
>>>>>>>> upper
>>>>>>>> +	 * interrupts.
>>>>>>>> +	 */
>>>>>>>> +	if (freq * ACTMON_SAMPLING_PERIOD > upper)
>>>>>>>> +		upper = freq * ACTMON_SAMPLING_PERIOD;    
>>>>>>>
>>>>>>> Also, 'upper value is used on the patch5. You can combine this code
>>>>>>> to patch5 or if this patch depends on the cpu notifier, you can
>>>>>>> combine it to the patch of adding cpu notifier without separate
>>>>>>> patch.  
>>>>>>
>>>>>> Well okay, I'll try to squash some of the patches in the next
>>>>>> revision. Usually I'm receiving comments in the other direction,
>>>>>> asking to separate patches into smaller changes ;) So that's more a
>>>>>> personal preference of each maintainer, I'd say.
>>>>>>   
>>>>>
>>>>> Right. We have to make the patch with atomic attribute.
>>>>> But, if there are patches which touch the same code
>>>>> in the same patchset. We can squash or do refactorig
>>>>> of this code.
>>>>
>>>> The main benefit of having smaller logical changes is that when there is
>>>> a bug, it's easier to narrow down the offending change using bisection.
>>>> And it's just easier to review smaller patches, of course.
>>>
>>> I agree that the patch should contain the atomic feature.
>>> To remove the some communication confusion between us,
>>> I don't mean that you have to merge patches to only one patch.
>>
>> If each patch has the atomic attribute, it have to be made as the separate patch.
>> But, if some patches are included in the the following two case,
>> can combine patches to one patch.
>>
>>>
>>> It is important to remove the following two cases on the same patchset.
>>>
>>> 1. the front patch adds the code and then later patch remove the added code.
> 
> Okay, I agree that this is applicable to patch #11.
> 
>>> 2. the front patch changes the code and the later patch again modified
>>>    the changed code of the front patch
> 
> If patch A adds a new feature and then patch B adds another new feature
> on top of A, do you consider each of these patches as atomic?

Yes, if patch A and patch A have the different role
for the same device driver, it is possible to make them
as the separate patches.


> 
> [snip]
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

* Re: [PATCH v4 20/24] PM / devfreq: tegra30: Optimize upper average watermark selection
  2019-07-24 11:17                 ` Chanwoo Choi
@ 2019-07-24 11:19                   ` Chanwoo Choi
  0 siblings, 0 replies; 94+ messages in thread
From: Chanwoo Choi @ 2019-07-24 11:19 UTC (permalink / raw)
  To: Dmitry Osipenko
  Cc: Thierry Reding, MyungJoo Ham, Kyungmin Park, Jonathan Hunter,
	Tomeu Vizoso, linux-pm, linux-tegra, linux-kernel

On 19. 7. 24. 오후 8:17, Chanwoo Choi wrote:
> On 19. 7. 20. 오전 2:52, Dmitry Osipenko wrote:
>> 19.07.2019 9:11, Chanwoo Choi пишет:
>>> On 19. 7. 19. 오후 3:09, Chanwoo Choi wrote:
>>>> On 19. 7. 19. 오전 11:21, Dmitry Osipenko wrote:
>>>>> В Fri, 19 Jul 2019 11:06:05 +0900
>>>>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>>>>
>>>>>> On 19. 7. 19. 오전 10:59, Dmitry Osipenko wrote:
>>>>>>> В Fri, 19 Jul 2019 10:36:30 +0900
>>>>>>> Chanwoo Choi <cw00.choi@samsung.com> пишет:
>>>>>>>   
>>>>>>>> On 19. 7. 8. 오전 7:32, Dmitry Osipenko wrote:  
>>>>>>>>> I noticed that CPU may be crossing the dependency threshold very
>>>>>>>>> frequently for some workloads and this results in a lot of
>>>>>>>>> interrupts which could be avoided if MCALL client is keeping
>>>>>>>>> actual EMC frequency at a higher rate.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Dmitry Osipenko <digetx@gmail.com>
>>>>>>>>> ---
>>>>>>>>>  drivers/devfreq/tegra30-devfreq.c | 23 ++++++++++++++++++-----
>>>>>>>>>  1 file changed, 18 insertions(+), 5 deletions(-)
>>>>>>>>>
>>>>>>>>> diff --git a/drivers/devfreq/tegra30-devfreq.c
>>>>>>>>> b/drivers/devfreq/tegra30-devfreq.c index
>>>>>>>>> c3cf87231d25..4d582809acb6 100644 ---
>>>>>>>>> a/drivers/devfreq/tegra30-devfreq.c +++
>>>>>>>>> b/drivers/devfreq/tegra30-devfreq.c @@ -314,7 +314,8 @@ static
>>>>>>>>> void tegra_actmon_get_lower_upper(struct tegra_devfreq *tegra, }
>>>>>>>>>  
>>>>>>>>>  static void tegra_devfreq_update_avg_wmark(struct tegra_devfreq
>>>>>>>>> *tegra,
>>>>>>>>> -					   struct
>>>>>>>>> tegra_devfreq_device *dev)
>>>>>>>>> +					   struct
>>>>>>>>> tegra_devfreq_device *dev,
>>>>>>>>> +					   unsigned long freq)
>>>>>>>>>  {
>>>>>>>>>  	unsigned long avg_threshold, lower, upper;
>>>>>>>>>  
>>>>>>>>> @@ -323,6 +324,15 @@ static void
>>>>>>>>> tegra_devfreq_update_avg_wmark(struct tegra_devfreq *tegra,
>>>>>>>>> avg_threshold = dev->config->avg_dependency_threshold;
>>>>>>>>> avg_threshold = avg_threshold * ACTMON_SAMPLING_PERIOD; 
>>>>>>>>> +	/*
>>>>>>>>> +	 * If cumulative EMC frequency selection is higher than
>>>>>>>>> the
>>>>>>>>> +	 * device's, then there is no need to set upper watermark
>>>>>>>>> to
>>>>>>>>> +	 * a lower value because it will result in unnecessary
>>>>>>>>> upper
>>>>>>>>> +	 * interrupts.
>>>>>>>>> +	 */
>>>>>>>>> +	if (freq * ACTMON_SAMPLING_PERIOD > upper)
>>>>>>>>> +		upper = freq * ACTMON_SAMPLING_PERIOD;    
>>>>>>>>
>>>>>>>> Also, 'upper value is used on the patch5. You can combine this code
>>>>>>>> to patch5 or if this patch depends on the cpu notifier, you can
>>>>>>>> combine it to the patch of adding cpu notifier without separate
>>>>>>>> patch.  
>>>>>>>
>>>>>>> Well okay, I'll try to squash some of the patches in the next
>>>>>>> revision. Usually I'm receiving comments in the other direction,
>>>>>>> asking to separate patches into smaller changes ;) So that's more a
>>>>>>> personal preference of each maintainer, I'd say.
>>>>>>>   
>>>>>>
>>>>>> Right. We have to make the patch with atomic attribute.
>>>>>> But, if there are patches which touch the same code
>>>>>> in the same patchset. We can squash or do refactorig
>>>>>> of this code.
>>>>>
>>>>> The main benefit of having smaller logical changes is that when there is
>>>>> a bug, it's easier to narrow down the offending change using bisection.
>>>>> And it's just easier to review smaller patches, of course.
>>>>
>>>> I agree that the patch should contain the atomic feature.
>>>> To remove the some communication confusion between us,
>>>> I don't mean that you have to merge patches to only one patch.
>>>
>>> If each patch has the atomic attribute, it have to be made as the separate patch.
>>> But, if some patches are included in the the following two case,
>>> can combine patches to one patch.
>>>
>>>>
>>>> It is important to remove the following two cases on the same patchset.
>>>>
>>>> 1. the front patch adds the code and then later patch remove the added code.
>>
>> Okay, I agree that this is applicable to patch #11.
>>
>>>> 2. the front patch changes the code and the later patch again modified
>>>>    the changed code of the front patch
>>
>> If patch A adds a new feature and then patch B adds another new feature
>> on top of A, do you consider each of these patches as atomic?
> 
> Yes, if patch A and patch A have the different role

Sorry for my mistake. Modify the sentence as following:

Yes, if patch A and patch B have the different role
for the same device driver, it is possible to make them
as the separate patches.


> 
> 
>>
>> [snip]
>>
>>
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

^ permalink raw reply	[flat|nested] 94+ messages in thread

end of thread, back to index

Thread overview: 94+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-07 22:32 [PATCH v4 00/24] More improvements for Tegra30 devfreq driver Dmitry Osipenko
2019-07-07 22:32 ` [PATCH v4 01/24] PM / devfreq: tegra30: Change irq type to unsigned int Dmitry Osipenko
2019-07-16 11:35   ` Chanwoo Choi
2019-07-07 22:32 ` [PATCH v4 02/24] PM / devfreq: tegra30: Keep interrupt disabled while governor is stopped Dmitry Osipenko
2019-07-16 11:47   ` Chanwoo Choi
2019-07-16 13:03     ` Dmitry Osipenko
2019-07-17  6:37       ` Chanwoo Choi
2019-07-17 16:44         ` Dmitry Osipenko
2019-07-07 22:32 ` [PATCH v4 03/24] PM / devfreq: tegra30: Handle possible round-rate error Dmitry Osipenko
2019-07-16 11:50   ` Chanwoo Choi
2019-07-16 13:09     ` Dmitry Osipenko
2019-07-17  6:38       ` Chanwoo Choi
2019-07-07 22:32 ` [PATCH v4 04/24] PM / devfreq: tegra30: Drop write-barrier Dmitry Osipenko
2019-07-16 11:51   ` Chanwoo Choi
2019-07-07 22:32 ` [PATCH v4 05/24] PM / devfreq: tegra30: Set up watermarks properly Dmitry Osipenko
2019-07-18 10:17   ` Chanwoo Choi
2019-07-19  0:00     ` Dmitry Osipenko
2019-07-19  1:31       ` Chanwoo Choi
2019-07-07 22:32 ` [PATCH v4 06/24] PM / devfreq: tegra30: Tune up boosting thresholds Dmitry Osipenko
2019-07-16 11:55   ` Chanwoo Choi
2019-07-07 22:32 ` [PATCH v4 07/24] PM / devfreq: tegra30: Use CPUFreq notifier Dmitry Osipenko
2019-07-16 12:08   ` Chanwoo Choi
2019-07-16 13:18     ` Dmitry Osipenko
2019-07-07 22:32 ` [PATCH v4 08/24] PM / devfreq: tegra30: Move clk-notifier's registration to governor's start Dmitry Osipenko
2019-07-16 12:11   ` Chanwoo Choi
2019-07-07 22:32 ` [PATCH v4 09/24] PM / devfreq: tegra30: Reset boosting on startup Dmitry Osipenko
2019-07-16 12:13   ` Chanwoo Choi
2019-07-16 13:19     ` Dmitry Osipenko
2019-07-07 22:32 ` [PATCH v4 10/24] PM / devfreq: tegra30: Don't enable consecutive-down interrupt " Dmitry Osipenko
2019-07-16 12:17   ` Chanwoo Choi
2019-07-16 15:17     ` Dmitry Osipenko
2019-07-07 22:32 ` [PATCH v4 11/24] PM / devfreq: tegra30: Add debug messages Dmitry Osipenko
2019-07-16 12:23   ` Chanwoo Choi
2019-07-16 13:26     ` Dmitry Osipenko
2019-07-17  6:45       ` Chanwoo Choi
2019-07-17 15:46         ` Dmitry Osipenko
2019-07-18  9:07           ` Chanwoo Choi
2019-07-19  1:13             ` Dmitry Osipenko
2019-07-19  1:22               ` Chanwoo Choi
2019-07-19 17:10                 ` Dmitry Osipenko
2019-07-07 22:32 ` [PATCH v4 12/24] PM / devfreq: tegra30: Inline all one-line functions Dmitry Osipenko
2019-07-16 12:26   ` Chanwoo Choi
2019-07-16 13:35     ` Dmitry Osipenko
2019-07-18  9:09       ` Chanwoo Choi
2019-07-19  1:22         ` Dmitry Osipenko
2019-07-19  1:24           ` Chanwoo Choi
2019-07-19  1:27             ` Chanwoo Choi
2019-07-19  2:14               ` Dmitry Osipenko
2019-07-19  6:01                 ` Chanwoo Choi
2019-07-19 16:52                   ` Dmitry Osipenko
2019-07-07 22:32 ` [PATCH v4 13/24] PM / devfreq: tegra30: Constify structs Dmitry Osipenko
2019-07-16 12:26   ` Chanwoo Choi
2019-07-07 22:32 ` [PATCH v4 14/24] PM / devfreq: tegra30: Ensure that target freq won't overflow Dmitry Osipenko
2019-07-16 12:30   ` Chanwoo Choi
2019-07-16 13:59     ` Dmitry Osipenko
2019-07-07 22:32 ` [PATCH v4 15/24] PM / devfreq: tegra30: Fix integer overflow on CPU's freq max out Dmitry Osipenko
2019-07-16 12:32   ` Chanwoo Choi
2019-07-07 22:32 ` [PATCH v4 16/24] PM / devfreq: tegra30: Use kHz units uniformly in the code Dmitry Osipenko
2019-07-07 22:32 ` [PATCH v4 17/24] PM / devfreq: tegra30: Use tracepoints for debugging Dmitry Osipenko
2019-07-18  9:47   ` Chanwoo Choi
2019-07-19  0:49     ` Dmitry Osipenko
2019-07-19  1:01       ` Chanwoo Choi
2019-07-19  1:50         ` Dmitry Osipenko
2019-07-07 22:32 ` [PATCH v4 18/24] PM / devfreq: tegra30: Optimize CPUFreq notifier Dmitry Osipenko
2019-07-18  9:48   ` Chanwoo Choi
2019-07-19  0:42     ` Dmitry Osipenko
2019-07-19  1:09       ` Chanwoo Choi
2019-07-07 22:32 ` [PATCH v4 19/24] PM / devfreq: tegra30: Optimize upper consecutive watermark selection Dmitry Osipenko
2019-07-18  9:51   ` Chanwoo Choi
2019-07-19  0:40     ` Dmitry Osipenko
2019-07-19  1:15       ` Chanwoo Choi
2019-07-19  1:17         ` Chanwoo Choi
2019-07-07 22:32 ` [PATCH v4 20/24] PM / devfreq: tegra30: Optimize upper average " Dmitry Osipenko
2019-07-19  1:36   ` Chanwoo Choi
2019-07-19  1:59     ` Dmitry Osipenko
2019-07-19  2:06       ` Chanwoo Choi
2019-07-19  2:21         ` Dmitry Osipenko
2019-07-19  6:09           ` Chanwoo Choi
2019-07-19  6:11             ` Chanwoo Choi
2019-07-19 17:52               ` Dmitry Osipenko
2019-07-24 11:17                 ` Chanwoo Choi
2019-07-24 11:19                   ` Chanwoo Choi
2019-07-07 22:33 ` [PATCH v4 21/24] PM / devfreq: tegra30: Synchronize average count on target's update Dmitry Osipenko
2019-07-18 10:15   ` Chanwoo Choi
2019-07-19  0:31     ` Dmitry Osipenko
2019-07-19  1:40       ` Chanwoo Choi
2019-07-19 16:46         ` Dmitry Osipenko
2019-07-07 22:33 ` [PATCH v4 22/24] PM / devfreq: tegra30: Include appropriate header Dmitry Osipenko
2019-07-18  9:58   ` Chanwoo Choi
2019-07-19  0:34     ` Dmitry Osipenko
2019-07-07 22:33 ` [PATCH v4 23/24] PM / devfreq: tegra30: Increase sampling period to 16ms Dmitry Osipenko
2019-07-18 10:00   ` Chanwoo Choi
2019-07-07 22:33 ` [PATCH v4 24/24] PM / devfreq: tegra20/30: Add Dmitry as a maintainer Dmitry Osipenko
2019-07-18  9:56   ` Chanwoo Choi

Linux-PM Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-pm/0 linux-pm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-pm linux-pm/ https://lore.kernel.org/linux-pm \
		linux-pm@vger.kernel.org
	public-inbox-index linux-pm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-pm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git