[PATCH v3 0/5] devfreq: handle suspend/resume

[PATCH v3 0/5] devfreq: handle suspend/resume

[Lukasz Luba]

Hi all,

This v3 patch set aims to address the issue with devfreq devices' frequency
during suspend/resume. It extends suspend/resume by calls to Devfreq
framework. In the devfreq framework there is a small refactoring to avoid
code duplication in changging frequency (patch 1) and there are extensions
for suspending devices. The suspending device has now chance to set proper
state when the system is going for suspend. This phase is the right place
to set needed frequences for the next resume process.

It has been tested on Odroid u3 with Exynos 4412.

The patch set draws on Tobias Jakobi's work posted ~2 years ago, who tried
to solve issue with devfreq device's frequency during suspend/resume.
During the discussion on LKML some corner cases and comments appeared
related to the design. This patch set address them keeping in mind
suggestions from Chanwoo Choi.
Tobias's paches:
https://www.spinics.net/lists/linux-samsung-soc/msg56602.html 

Changes:
v3:
- cleaned commit messages according to Chanwoo's comments
- moved atomic operation to beggining of function in
  devfreq_{suspend|resume}_device()
- changed dev_warn() to dev_err() with also new error message,
  when devfreq_{suspend|resume}() failed
v2:
- refactored patchset and merget patch 1 and 3 as suggested by Chanwoo Choi,
- changed devfreq_{susped|resume}_device functions,
- added doxygen information for new entres in 'struct devfreq',
- devfreq_set_target skipped one argument, now resume_freq is set inside,
- minor changes addresing comments from maintainers regarding the style,

Regards,
Lukasz Luba

Lukasz Luba (5):
  devfreq: refactor set_target frequency function
  devfreq: add support for suspend/resume of a devfreq device
  devfreq: add devfreq_suspend/resume() functions
  drivers: power: suspend: call devfreq suspend/resume
  arm: dts: exynos4: opp-suspend in DMC and leftbus

 arch/arm/boot/dts/exynos4210.dtsi |   2 +
 arch/arm/boot/dts/exynos4412.dtsi |   2 +
 drivers/base/power/main.c         |   3 +
 drivers/devfreq/devfreq.c         | 153 ++++++++++++++++++++++++++++++--------
 include/linux/devfreq.h           |  13 ++++
 5 files changed, 141 insertions(+), 32 deletions(-)

-- 
2.7.4

[PATCH v3 1/5] devfreq: refactor set_target frequency function

[Lukasz Luba]

The refactoring is needed for the new client in devfreq: suspend.
To avoid code duplication, move it to the new local function
devfreq_set_target.

Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
---
 drivers/devfreq/devfreq.c | 62 +++++++++++++++++++++++++++--------------------
 1 file changed, 36 insertions(+), 26 deletions(-)

diff --git a/drivers/devfreq/devfreq.c b/drivers/devfreq/devfreq.c
index 1414130..a9fd61b 100644
--- a/drivers/devfreq/devfreq.c
+++ b/drivers/devfreq/devfreq.c
@@ -285,6 +285,40 @@ static int devfreq_notify_transition(struct devfreq *devfreq,
 	return 0;
 }
 
+static int devfreq_set_target(struct devfreq *devfreq, unsigned long new_freq,
+			      u32 flags)
+{
+	struct devfreq_freqs freqs;
+	unsigned long cur_freq;
+	int err = 0;
+
+	if (devfreq->profile->get_cur_freq)
+		devfreq->profile->get_cur_freq(devfreq->dev.parent, &cur_freq);
+	else
+		cur_freq = devfreq->previous_freq;
+
+	freqs.old = cur_freq;
+	freqs.new = new_freq;
+	devfreq_notify_transition(devfreq, &freqs, DEVFREQ_PRECHANGE);
+
+	err = devfreq->profile->target(devfreq->dev.parent, &new_freq, flags);
+	if (err) {
+		freqs.new = cur_freq;
+		devfreq_notify_transition(devfreq, &freqs, DEVFREQ_POSTCHANGE);
+		return err;
+	}
+
+	freqs.new = new_freq;
+	devfreq_notify_transition(devfreq, &freqs, DEVFREQ_POSTCHANGE);
+
+	if (devfreq_update_status(devfreq, new_freq))
+		dev_err(&devfreq->dev,
+			"Couldn't update frequency transition information.\n");
+
+	devfreq->previous_freq = new_freq;
+	return err;
+}
+
 /* Load monitoring helper functions for governors use */
 
 /**
@@ -296,8 +330,7 @@ static int devfreq_notify_transition(struct devfreq *devfreq,
  */
 int update_devfreq(struct devfreq *devfreq)
 {
-	struct devfreq_freqs freqs;
-	unsigned long freq, cur_freq, min_freq, max_freq;
+	unsigned long freq, min_freq, max_freq;
 	int err = 0;
 	u32 flags = 0;
 
@@ -333,31 +366,8 @@ int update_devfreq(struct devfreq *devfreq)
 		flags |= DEVFREQ_FLAG_LEAST_UPPER_BOUND; /* Use LUB */
 	}
 
-	if (devfreq->profile->get_cur_freq)
-		devfreq->profile->get_cur_freq(devfreq->dev.parent, &cur_freq);
-	else
-		cur_freq = devfreq->previous_freq;
-
-	freqs.old = cur_freq;
-	freqs.new = freq;
-	devfreq_notify_transition(devfreq, &freqs, DEVFREQ_PRECHANGE);
+	return devfreq_set_target(devfreq, freq, flags);
 
-	err = devfreq->profile->target(devfreq->dev.parent, &freq, flags);
-	if (err) {
-		freqs.new = cur_freq;
-		devfreq_notify_transition(devfreq, &freqs, DEVFREQ_POSTCHANGE);
-		return err;
-	}
-
-	freqs.new = freq;
-	devfreq_notify_transition(devfreq, &freqs, DEVFREQ_POSTCHANGE);
-
-	if (devfreq_update_status(devfreq, freq))
-		dev_err(&devfreq->dev,
-			"Couldn't update frequency transition information.\n");
-
-	devfreq->previous_freq = freq;
-	return err;
 }
 EXPORT_SYMBOL(update_devfreq);
 
-- 
2.7.4

[PATCH v3 2/5] devfreq: add support for suspend/resume of a devfreq device

[Lukasz Luba]

The patch prepares devfreq device for handling suspend/resume
functionality. The new fields will store needed information during this
process. Devfreq framework handles opp-suspend DT entry and there is no
need of modyfications in the drivers code. It uses atomic variables to
make sure no race condition affects the process.

Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
---
 drivers/devfreq/devfreq.c | 47 +++++++++++++++++++++++++++++++++++++++++------
 include/linux/devfreq.h   |  7 +++++++
 2 files changed, 48 insertions(+), 6 deletions(-)

diff --git a/drivers/devfreq/devfreq.c b/drivers/devfreq/devfreq.c
index a9fd61b..46517b6 100644
--- a/drivers/devfreq/devfreq.c
+++ b/drivers/devfreq/devfreq.c
@@ -316,6 +316,10 @@ static int devfreq_set_target(struct devfreq *devfreq, unsigned long new_freq,
 			"Couldn't update frequency transition information.\n");
 
 	devfreq->previous_freq = new_freq;
+
+	if (devfreq->suspend_freq)
+		devfreq->resume_freq = cur_freq;
+
 	return err;
 }
 
@@ -667,6 +671,9 @@ struct devfreq *devfreq_add_device(struct device *dev,
 	}
 	devfreq->max_freq = devfreq->scaling_max_freq;
 
+	devfreq->suspend_freq = dev_pm_opp_get_suspend_opp_freq(dev);
+	atomic_set(&devfreq->suspend_count, 0);
+
 	dev_set_name(&devfreq->dev, "devfreq%d",
 				atomic_inc_return(&devfreq_no));
 	err = device_register(&devfreq->dev);
@@ -867,14 +874,28 @@ EXPORT_SYMBOL(devm_devfreq_remove_device);
  */
 int devfreq_suspend_device(struct devfreq *devfreq)
 {
+	int ret;
+
 	if (!devfreq)
 		return -EINVAL;
 
-	if (!devfreq->governor)
+	if (atomic_inc_return(&devfreq->suspend_count) > 1)
 		return 0;
 
-	return devfreq->governor->event_handler(devfreq,
-				DEVFREQ_GOV_SUSPEND, NULL);
+	if (devfreq->governor) {
+		ret = devfreq->governor->event_handler(devfreq,
+					DEVFREQ_GOV_SUSPEND, NULL);
+		if (ret)
+			return ret;
+	}
+
+	if (devfreq->suspend_freq) {
+		ret = devfreq_set_target(devfreq, devfreq->suspend_freq, 0);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
 }
 EXPORT_SYMBOL(devfreq_suspend_device);
 
@@ -888,14 +909,28 @@ EXPORT_SYMBOL(devfreq_suspend_device);
  */
 int devfreq_resume_device(struct devfreq *devfreq)
 {
+	int ret;
+
 	if (!devfreq)
 		return -EINVAL;
 
-	if (!devfreq->governor)
+	if (atomic_dec_return(&devfreq->suspend_count) >= 1)
 		return 0;
 
-	return devfreq->governor->event_handler(devfreq,
-				DEVFREQ_GOV_RESUME, NULL);
+	if (devfreq->resume_freq) {
+		ret = devfreq_set_target(devfreq, devfreq->resume_freq, 0);
+		if (ret)
+			return ret;
+	}
+
+	if (devfreq->governor) {
+		ret = devfreq->governor->event_handler(devfreq,
+					DEVFREQ_GOV_RESUME, NULL);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
 }
 EXPORT_SYMBOL(devfreq_resume_device);
 
diff --git a/include/linux/devfreq.h b/include/linux/devfreq.h
index e4963b0..d985199 100644
--- a/include/linux/devfreq.h
+++ b/include/linux/devfreq.h
@@ -131,6 +131,9 @@ struct devfreq_dev_profile {
  * @scaling_min_freq:	Limit minimum frequency requested by OPP interface
  * @scaling_max_freq:	Limit maximum frequency requested by OPP interface
  * @stop_polling:	 devfreq polling status of a device.
+ * @suspend_freq:	 frequency of a device set during suspend phase.
+ * @resume_freq:	 frequency of a device set in resume phase.
+ * @suspend_count:	 suspend requests counter for a device.
  * @total_trans:	Number of devfreq transitions
  * @trans_table:	Statistics of devfreq transitions
  * @time_in_state:	Statistics of devfreq states
@@ -167,6 +170,10 @@ struct devfreq {
 	unsigned long scaling_max_freq;
 	bool stop_polling;
 
+	unsigned long suspend_freq;
+	unsigned long resume_freq;
+	atomic_t suspend_count;
+
 	/* information for device frequency transition */
 	unsigned int total_trans;
 	unsigned int *trans_table;
-- 
2.7.4

[PATCH v3 3/5] devfreq: add devfreq_suspend/resume() functions

[Lukasz Luba]

This patch adds implementation for global suspend/resume for
devfreq framework. System suspend will next use these functions.

Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
---
 drivers/devfreq/devfreq.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/devfreq.h   |  6 ++++++
 2 files changed, 50 insertions(+)

diff --git a/drivers/devfreq/devfreq.c b/drivers/devfreq/devfreq.c
index 46517b6..0ae3de7 100644
--- a/drivers/devfreq/devfreq.c
+++ b/drivers/devfreq/devfreq.c
@@ -935,6 +935,50 @@ int devfreq_resume_device(struct devfreq *devfreq)
 EXPORT_SYMBOL(devfreq_resume_device);
 
 /**
+ * devfreq_suspend() - Suspend devfreq governors and devices
+ *
+ * Called during system wide Suspend/Hibernate cycles for suspending governors
+ * and devices preserving the state for resume. On some platforms the devfreq
+ * device must have precise state (frequency) after resume in order to provide
+ * fully operating setup.
+ */
+void devfreq_suspend(void)
+{
+	struct devfreq *devfreq;
+	int ret;
+
+	mutex_lock(&devfreq_list_lock);
+	list_for_each_entry(devfreq, &devfreq_list, node) {
+		ret = devfreq_suspend_device(devfreq);
+		if (ret)
+			dev_err(&devfreq->dev,
+				"failed to suspend devfreq device\n");
+	}
+	mutex_unlock(&devfreq_list_lock);
+}
+
+/**
+ * devfreq_resume() - Resume devfreq governors and devices
+ *
+ * Called during system wide Suspend/Hibernate cycle for resuming governors and
+ * devices that are suspended with devfreq_suspend().
+ */
+void devfreq_resume(void)
+{
+	struct devfreq *devfreq;
+	int ret;
+
+	mutex_lock(&devfreq_list_lock);
+	list_for_each_entry(devfreq, &devfreq_list, node) {
+		ret = devfreq_resume_device(devfreq);
+		if (ret)
+			dev_warn(&devfreq->dev,
+				 "failed to resume devfreq device\n");
+	}
+	mutex_unlock(&devfreq_list_lock);
+}
+
+/**
  * devfreq_add_governor() - Add devfreq governor
  * @governor:	the devfreq governor to be added
  */
diff --git a/include/linux/devfreq.h b/include/linux/devfreq.h
index d985199..fbffa74 100644
--- a/include/linux/devfreq.h
+++ b/include/linux/devfreq.h
@@ -205,6 +205,9 @@ extern void devm_devfreq_remove_device(struct device *dev,
 extern int devfreq_suspend_device(struct devfreq *devfreq);
 extern int devfreq_resume_device(struct devfreq *devfreq);
 
+extern void devfreq_suspend(void);
+extern void devfreq_resume(void);
+
 /**
  * update_devfreq() - Reevaluate the device and configure frequency
  * @devfreq:	the devfreq device
@@ -331,6 +334,9 @@ static inline int devfreq_resume_device(struct devfreq *devfreq)
 	return 0;
 }
 
+static inline void devfreq_suspend(void) {}
+static inline void devfreq_resume(void) {}
+
 static inline struct dev_pm_opp *devfreq_recommended_opp(struct device *dev,
 					   unsigned long *freq, u32 flags)
 {
-- 
2.7.4

[PATCH v3 4/5] drivers: power: suspend: call devfreq suspend/resume

[Lukasz Luba]

Devfreq framework supports suspend of its devices.
Call the the devfreq interface and allow devfreq devices preserve/restore
their states during suspend/resume.

Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
---
 drivers/base/power/main.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/base/power/main.c b/drivers/base/power/main.c
index a690fd4..0992e67 100644
--- a/drivers/base/power/main.c
+++ b/drivers/base/power/main.c
@@ -32,6 +32,7 @@
 #include <trace/events/power.h>
 #include <linux/cpufreq.h>
 #include <linux/cpuidle.h>
+#include <linux/devfreq.h>
 #include <linux/timer.h>
 
 #include "../base.h"
@@ -1078,6 +1079,7 @@ void dpm_resume(pm_message_t state)
 	dpm_show_time(starttime, state, 0, NULL);
 
 	cpufreq_resume();
+	devfreq_resume();
 	trace_suspend_resume(TPS("dpm_resume"), state.event, false);
 }
 
@@ -1852,6 +1854,7 @@ int dpm_suspend(pm_message_t state)
 	trace_suspend_resume(TPS("dpm_suspend"), state.event, true);
 	might_sleep();
 
+	devfreq_suspend();
 	cpufreq_suspend();
 
 	mutex_lock(&dpm_list_mtx);
-- 
2.7.4

[PATCH v3 5/5] arm: dts: exynos4: opp-suspend in DMC and leftbus

[Lukasz Luba]

Mark the state for devfreq device while entring suspend/resume process.

Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
---
 arch/arm/boot/dts/exynos4210.dtsi | 2 ++
 arch/arm/boot/dts/exynos4412.dtsi | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/arch/arm/boot/dts/exynos4210.dtsi b/arch/arm/boot/dts/exynos4210.dtsi
index b6091c2..4429b72 100644
--- a/arch/arm/boot/dts/exynos4210.dtsi
+++ b/arch/arm/boot/dts/exynos4210.dtsi
@@ -298,6 +298,7 @@
 			opp-400000000 {
 				opp-hz = /bits/ 64 <400000000>;
 				opp-microvolt = <1150000>;
+				opp-suspend;
 			};
 		};
 
@@ -367,6 +368,7 @@
 			};
 			opp-200000000 {
 				opp-hz = /bits/ 64 <200000000>;
+				opp-suspend;
 			};
 		};
 	};
diff --git a/arch/arm/boot/dts/exynos4412.dtsi b/arch/arm/boot/dts/exynos4412.dtsi
index 51f72f0..908c0c4 100644
--- a/arch/arm/boot/dts/exynos4412.dtsi
+++ b/arch/arm/boot/dts/exynos4412.dtsi
@@ -432,6 +432,7 @@
 			opp-400000000 {
 				opp-hz = /bits/ 64 <400000000>;
 				opp-microvolt = <1050000>;
+				opp-suspend;
 			};
 		};
 
@@ -520,6 +521,7 @@
 			opp-200000000 {
 				opp-hz = /bits/ 64 <200000000>;
 				opp-microvolt = <1000000>;
+				opp-suspend;
 			};
 		};
 
-- 
2.7.4

Re: [PATCH v3 5/5] arm: dts: exynos4: opp-suspend in DMC and leftbus

[Krzysztof Kozlowski]

On Wed, 5 Dec 2018 at 12:06, Lukasz Luba <l.luba@partner.samsung.com> wrote:
>
> Mark the state for devfreq device while entring suspend/resume process.
>
> Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
> Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
> Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
> Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>

I already applied v2:
https://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux.git/commit/?h=next/dt

Best regards,
Krzysztof

> ---
>  arch/arm/boot/dts/exynos4210.dtsi | 2 ++
>  arch/arm/boot/dts/exynos4412.dtsi | 2 ++
>  2 files changed, 4 insertions(+)
>
> diff --git a/arch/arm/boot/dts/exynos4210.dtsi b/arch/arm/boot/dts/exynos4210.dtsi
> index b6091c2..4429b72 100644
> --- a/arch/arm/boot/dts/exynos4210.dtsi
> +++ b/arch/arm/boot/dts/exynos4210.dtsi
> @@ -298,6 +298,7 @@
>                         opp-400000000 {
>                                 opp-hz = /bits/ 64 <400000000>;
>                                 opp-microvolt = <1150000>;
> +                               opp-suspend;
>                         };
>                 };
>
> @@ -367,6 +368,7 @@
>                         };
>                         opp-200000000 {
>                                 opp-hz = /bits/ 64 <200000000>;
> +                               opp-suspend;
>                         };
>                 };
>         };
> diff --git a/arch/arm/boot/dts/exynos4412.dtsi b/arch/arm/boot/dts/exynos4412.dtsi
> index 51f72f0..908c0c4 100644
> --- a/arch/arm/boot/dts/exynos4412.dtsi
> +++ b/arch/arm/boot/dts/exynos4412.dtsi
> @@ -432,6 +432,7 @@
>                         opp-400000000 {
>                                 opp-hz = /bits/ 64 <400000000>;
>                                 opp-microvolt = <1050000>;
> +                               opp-suspend;
>                         };
>                 };
>
> @@ -520,6 +521,7 @@
>                         opp-200000000 {
>                                 opp-hz = /bits/ 64 <200000000>;
>                                 opp-microvolt = <1000000>;
> +                               opp-suspend;
>                         };
>                 };
>
> --
> 2.7.4
>

Re: [PATCH v3 5/5] arm: dts: exynos4: opp-suspend in DMC and leftbus

[Lukasz Luba]

On 12/5/18 12:12 PM, Krzysztof Kozlowski wrote:
> On Wed, 5 Dec 2018 at 12:06, Lukasz Luba <l.luba@partner.samsung.com> wrote:
>>
>> Mark the state for devfreq device while entring suspend/resume process.
>>
>> Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
>> Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
>> Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
>> Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
> 
> I already applied v2:
> https://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux.git/commit/?h=next/dt

Thank you!

Regards,
Lukasz
> 
> Best regards,
> Krzysztof
> 
>> ---
>>   arch/arm/boot/dts/exynos4210.dtsi | 2 ++
>>   arch/arm/boot/dts/exynos4412.dtsi | 2 ++
>>   2 files changed, 4 insertions(+)
>>
>> diff --git a/arch/arm/boot/dts/exynos4210.dtsi b/arch/arm/boot/dts/exynos4210.dtsi
>> index b6091c2..4429b72 100644
>> --- a/arch/arm/boot/dts/exynos4210.dtsi
>> +++ b/arch/arm/boot/dts/exynos4210.dtsi
>> @@ -298,6 +298,7 @@
>>                          opp-400000000 {
>>                                  opp-hz = /bits/ 64 <400000000>;
>>                                  opp-microvolt = <1150000>;
>> +                               opp-suspend;
>>                          };
>>                  };
>>
>> @@ -367,6 +368,7 @@
>>                          };
>>                          opp-200000000 {
>>                                  opp-hz = /bits/ 64 <200000000>;
>> +                               opp-suspend;
>>                          };
>>                  };
>>          };
>> diff --git a/arch/arm/boot/dts/exynos4412.dtsi b/arch/arm/boot/dts/exynos4412.dtsi
>> index 51f72f0..908c0c4 100644
>> --- a/arch/arm/boot/dts/exynos4412.dtsi
>> +++ b/arch/arm/boot/dts/exynos4412.dtsi
>> @@ -432,6 +432,7 @@
>>                          opp-400000000 {
>>                                  opp-hz = /bits/ 64 <400000000>;
>>                                  opp-microvolt = <1050000>;
>> +                               opp-suspend;
>>                          };
>>                  };
>>
>> @@ -520,6 +521,7 @@
>>                          opp-200000000 {
>>                                  opp-hz = /bits/ 64 <200000000>;
>>                                  opp-microvolt = <1000000>;
>> +                               opp-suspend;
>>                          };
>>                  };
>>
>> --
>> 2.7.4
>>
> 
> 

Re: [PATCH v3 2/5] devfreq: add support for suspend/resume of a devfreq device

[Chanwoo Choi]

Hi Lukasz,

On 2018년 12월 05일 20:05, Lukasz Luba wrote:
> The patch prepares devfreq device for handling suspend/resume
> functionality. The new fields will store needed information during this
> process. Devfreq framework handles opp-suspend DT entry and there is no
> need of modyfications in the drivers code. It uses atomic variables to
> make sure no race condition affects the process.
> 
> Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
> Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
> Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
> ---
>  drivers/devfreq/devfreq.c | 47 +++++++++++++++++++++++++++++++++++++++++------
>  include/linux/devfreq.h   |  7 +++++++
>  2 files changed, 48 insertions(+), 6 deletions(-)

Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>

> 
> diff --git a/drivers/devfreq/devfreq.c b/drivers/devfreq/devfreq.c
> index a9fd61b..46517b6 100644
> --- a/drivers/devfreq/devfreq.c
> +++ b/drivers/devfreq/devfreq.c
> @@ -316,6 +316,10 @@ static int devfreq_set_target(struct devfreq *devfreq, unsigned long new_freq,
>  			"Couldn't update frequency transition information.\n");
>  
>  	devfreq->previous_freq = new_freq;
> +
> +	if (devfreq->suspend_freq)
> +		devfreq->resume_freq = cur_freq;
> +
>  	return err;
>  }
>  
> @@ -667,6 +671,9 @@ struct devfreq *devfreq_add_device(struct device *dev,
>  	}
>  	devfreq->max_freq = devfreq->scaling_max_freq;
>  
> +	devfreq->suspend_freq = dev_pm_opp_get_suspend_opp_freq(dev);
> +	atomic_set(&devfreq->suspend_count, 0);
> +
>  	dev_set_name(&devfreq->dev, "devfreq%d",
>  				atomic_inc_return(&devfreq_no));
>  	err = device_register(&devfreq->dev);
> @@ -867,14 +874,28 @@ EXPORT_SYMBOL(devm_devfreq_remove_device);
>   */
>  int devfreq_suspend_device(struct devfreq *devfreq)
>  {
> +	int ret;
> +
>  	if (!devfreq)
>  		return -EINVAL;
>  
> -	if (!devfreq->governor)
> +	if (atomic_inc_return(&devfreq->suspend_count) > 1)
>  		return 0;
>  
> -	return devfreq->governor->event_handler(devfreq,
> -				DEVFREQ_GOV_SUSPEND, NULL);
> +	if (devfreq->governor) {
> +		ret = devfreq->governor->event_handler(devfreq,
> +					DEVFREQ_GOV_SUSPEND, NULL);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	if (devfreq->suspend_freq) {
> +		ret = devfreq_set_target(devfreq, devfreq->suspend_freq, 0);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	return 0;
>  }
>  EXPORT_SYMBOL(devfreq_suspend_device);
>  
> @@ -888,14 +909,28 @@ EXPORT_SYMBOL(devfreq_suspend_device);
>   */
>  int devfreq_resume_device(struct devfreq *devfreq)
>  {
> +	int ret;
> +
>  	if (!devfreq)
>  		return -EINVAL;
>  
> -	if (!devfreq->governor)
> +	if (atomic_dec_return(&devfreq->suspend_count) >= 1)
>  		return 0;
>  
> -	return devfreq->governor->event_handler(devfreq,
> -				DEVFREQ_GOV_RESUME, NULL);
> +	if (devfreq->resume_freq) {
> +		ret = devfreq_set_target(devfreq, devfreq->resume_freq, 0);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	if (devfreq->governor) {
> +		ret = devfreq->governor->event_handler(devfreq,
> +					DEVFREQ_GOV_RESUME, NULL);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	return 0;
>  }
>  EXPORT_SYMBOL(devfreq_resume_device);
>  
> diff --git a/include/linux/devfreq.h b/include/linux/devfreq.h
> index e4963b0..d985199 100644
> --- a/include/linux/devfreq.h
> +++ b/include/linux/devfreq.h
> @@ -131,6 +131,9 @@ struct devfreq_dev_profile {
>   * @scaling_min_freq:	Limit minimum frequency requested by OPP interface
>   * @scaling_max_freq:	Limit maximum frequency requested by OPP interface
>   * @stop_polling:	 devfreq polling status of a device.
> + * @suspend_freq:	 frequency of a device set during suspend phase.
> + * @resume_freq:	 frequency of a device set in resume phase.
> + * @suspend_count:	 suspend requests counter for a device.
>   * @total_trans:	Number of devfreq transitions
>   * @trans_table:	Statistics of devfreq transitions
>   * @time_in_state:	Statistics of devfreq states
> @@ -167,6 +170,10 @@ struct devfreq {
>  	unsigned long scaling_max_freq;
>  	bool stop_polling;
>  
> +	unsigned long suspend_freq;
> +	unsigned long resume_freq;
> +	atomic_t suspend_count;
> +
>  	/* information for device frequency transition */
>  	unsigned int total_trans;
>  	unsigned int *trans_table;
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

Re: [PATCH v3 3/5] devfreq: add devfreq_suspend/resume() functions

[Chanwoo Choi]

Hi Lukasz,

On 2018년 12월 05일 20:05, Lukasz Luba wrote:
> This patch adds implementation for global suspend/resume for
> devfreq framework. System suspend will next use these functions.
> 
> Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
> Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
> Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
> ---
>  drivers/devfreq/devfreq.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
>  include/linux/devfreq.h   |  6 ++++++
>  2 files changed, 50 insertions(+)

Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>

> 
> diff --git a/drivers/devfreq/devfreq.c b/drivers/devfreq/devfreq.c
> index 46517b6..0ae3de7 100644
> --- a/drivers/devfreq/devfreq.c
> +++ b/drivers/devfreq/devfreq.c
> @@ -935,6 +935,50 @@ int devfreq_resume_device(struct devfreq *devfreq)
>  EXPORT_SYMBOL(devfreq_resume_device);
>  
>  /**
> + * devfreq_suspend() - Suspend devfreq governors and devices
> + *
> + * Called during system wide Suspend/Hibernate cycles for suspending governors
> + * and devices preserving the state for resume. On some platforms the devfreq
> + * device must have precise state (frequency) after resume in order to provide
> + * fully operating setup.
> + */
> +void devfreq_suspend(void)
> +{
> +	struct devfreq *devfreq;
> +	int ret;
> +
> +	mutex_lock(&devfreq_list_lock);
> +	list_for_each_entry(devfreq, &devfreq_list, node) {
> +		ret = devfreq_suspend_device(devfreq);
> +		if (ret)
> +			dev_err(&devfreq->dev,
> +				"failed to suspend devfreq device\n");
> +	}
> +	mutex_unlock(&devfreq_list_lock);
> +}
> +
> +/**
> + * devfreq_resume() - Resume devfreq governors and devices
> + *
> + * Called during system wide Suspend/Hibernate cycle for resuming governors and
> + * devices that are suspended with devfreq_suspend().
> + */
> +void devfreq_resume(void)
> +{
> +	struct devfreq *devfreq;
> +	int ret;
> +
> +	mutex_lock(&devfreq_list_lock);
> +	list_for_each_entry(devfreq, &devfreq_list, node) {
> +		ret = devfreq_resume_device(devfreq);
> +		if (ret)
> +			dev_warn(&devfreq->dev,
> +				 "failed to resume devfreq device\n");
> +	}
> +	mutex_unlock(&devfreq_list_lock);
> +}
> +
> +/**
>   * devfreq_add_governor() - Add devfreq governor
>   * @governor:	the devfreq governor to be added
>   */
> diff --git a/include/linux/devfreq.h b/include/linux/devfreq.h
> index d985199..fbffa74 100644
> --- a/include/linux/devfreq.h
> +++ b/include/linux/devfreq.h
> @@ -205,6 +205,9 @@ extern void devm_devfreq_remove_device(struct device *dev,
>  extern int devfreq_suspend_device(struct devfreq *devfreq);
>  extern int devfreq_resume_device(struct devfreq *devfreq);
>  
> +extern void devfreq_suspend(void);
> +extern void devfreq_resume(void);
> +
>  /**
>   * update_devfreq() - Reevaluate the device and configure frequency
>   * @devfreq:	the devfreq device
> @@ -331,6 +334,9 @@ static inline int devfreq_resume_device(struct devfreq *devfreq)
>  	return 0;
>  }
>  
> +static inline void devfreq_suspend(void) {}
> +static inline void devfreq_resume(void) {}
> +
>  static inline struct dev_pm_opp *devfreq_recommended_opp(struct device *dev,
>  					   unsigned long *freq, u32 flags)
>  {
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics

Re: [PATCH v3 2/5] devfreq: add support for suspend/resume of a devfreq device

[Lukasz Luba]

Hi Chanwoo,

On 12/6/18 2:17 AM, Chanwoo Choi wrote:
> Hi Lukasz,
> 
> On 2018년 12월 05일 20:05, Lukasz Luba wrote:
>> The patch prepares devfreq device for handling suspend/resume
>> functionality. The new fields will store needed information during this
>> process. Devfreq framework handles opp-suspend DT entry and there is no
>> need of modyfications in the drivers code. It uses atomic variables to
>> make sure no race condition affects the process.
>>
>> Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
>> Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
>> Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
>> ---
>>   drivers/devfreq/devfreq.c | 47 +++++++++++++++++++++++++++++++++++++++++------
>>   include/linux/devfreq.h   |  7 +++++++
>>   2 files changed, 48 insertions(+), 6 deletions(-)
> 
> Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
Thank you for the review and comments for the whole patch series.

Regards,
Lukasz

> 
>>
>> diff --git a/drivers/devfreq/devfreq.c b/drivers/devfreq/devfreq.c
>> index a9fd61b..46517b6 100644
>> --- a/drivers/devfreq/devfreq.c
>> +++ b/drivers/devfreq/devfreq.c
>> @@ -316,6 +316,10 @@ static int devfreq_set_target(struct devfreq *devfreq, unsigned long new_freq,
>>   			"Couldn't update frequency transition information.\n");
>>   
>>   	devfreq->previous_freq = new_freq;
>> +
>> +	if (devfreq->suspend_freq)
>> +		devfreq->resume_freq = cur_freq;
>> +
>>   	return err;
>>   }
>>   
>> @@ -667,6 +671,9 @@ struct devfreq *devfreq_add_device(struct device *dev,
>>   	}
>>   	devfreq->max_freq = devfreq->scaling_max_freq;
>>   
>> +	devfreq->suspend_freq = dev_pm_opp_get_suspend_opp_freq(dev);
>> +	atomic_set(&devfreq->suspend_count, 0);
>> +
>>   	dev_set_name(&devfreq->dev, "devfreq%d",
>>   				atomic_inc_return(&devfreq_no));
>>   	err = device_register(&devfreq->dev);
>> @@ -867,14 +874,28 @@ EXPORT_SYMBOL(devm_devfreq_remove_device);
>>    */
>>   int devfreq_suspend_device(struct devfreq *devfreq)
>>   {
>> +	int ret;
>> +
>>   	if (!devfreq)
>>   		return -EINVAL;
>>   
>> -	if (!devfreq->governor)
>> +	if (atomic_inc_return(&devfreq->suspend_count) > 1)
>>   		return 0;
>>   
>> -	return devfreq->governor->event_handler(devfreq,
>> -				DEVFREQ_GOV_SUSPEND, NULL);
>> +	if (devfreq->governor) {
>> +		ret = devfreq->governor->event_handler(devfreq,
>> +					DEVFREQ_GOV_SUSPEND, NULL);
>> +		if (ret)
>> +			return ret;
>> +	}
>> +
>> +	if (devfreq->suspend_freq) {
>> +		ret = devfreq_set_target(devfreq, devfreq->suspend_freq, 0);
>> +		if (ret)
>> +			return ret;
>> +	}
>> +
>> +	return 0;
>>   }
>>   EXPORT_SYMBOL(devfreq_suspend_device);
>>   
>> @@ -888,14 +909,28 @@ EXPORT_SYMBOL(devfreq_suspend_device);
>>    */
>>   int devfreq_resume_device(struct devfreq *devfreq)
>>   {
>> +	int ret;
>> +
>>   	if (!devfreq)
>>   		return -EINVAL;
>>   
>> -	if (!devfreq->governor)
>> +	if (atomic_dec_return(&devfreq->suspend_count) >= 1)
>>   		return 0;
>>   
>> -	return devfreq->governor->event_handler(devfreq,
>> -				DEVFREQ_GOV_RESUME, NULL);
>> +	if (devfreq->resume_freq) {
>> +		ret = devfreq_set_target(devfreq, devfreq->resume_freq, 0);
>> +		if (ret)
>> +			return ret;
>> +	}
>> +
>> +	if (devfreq->governor) {
>> +		ret = devfreq->governor->event_handler(devfreq,
>> +					DEVFREQ_GOV_RESUME, NULL);
>> +		if (ret)
>> +			return ret;
>> +	}
>> +
>> +	return 0;
>>   }
>>   EXPORT_SYMBOL(devfreq_resume_device);
>>   
>> diff --git a/include/linux/devfreq.h b/include/linux/devfreq.h
>> index e4963b0..d985199 100644
>> --- a/include/linux/devfreq.h
>> +++ b/include/linux/devfreq.h
>> @@ -131,6 +131,9 @@ struct devfreq_dev_profile {
>>    * @scaling_min_freq:	Limit minimum frequency requested by OPP interface
>>    * @scaling_max_freq:	Limit maximum frequency requested by OPP interface
>>    * @stop_polling:	 devfreq polling status of a device.
>> + * @suspend_freq:	 frequency of a device set during suspend phase.
>> + * @resume_freq:	 frequency of a device set in resume phase.
>> + * @suspend_count:	 suspend requests counter for a device.
>>    * @total_trans:	Number of devfreq transitions
>>    * @trans_table:	Statistics of devfreq transitions
>>    * @time_in_state:	Statistics of devfreq states
>> @@ -167,6 +170,10 @@ struct devfreq {
>>   	unsigned long scaling_max_freq;
>>   	bool stop_polling;
>>   
>> +	unsigned long suspend_freq;
>> +	unsigned long resume_freq;
>> +	atomic_t suspend_count;
>> +
>>   	/* information for device frequency transition */
>>   	unsigned int total_trans;
>>   	unsigned int *trans_table;
>>
> 
> 

RE: Re: [PATCH v3 2/5] devfreq: add support for suspend/resume of a devfreq device

[MyungJoo Ham]

> Hi Lukasz,
> 
> On 2018년 12월 05일 20:05, Lukasz Luba wrote:
> > The patch prepares devfreq device for handling suspend/resume
> > functionality. The new fields will store needed information during this
> > process. Devfreq framework handles opp-suspend DT entry and there is no
> > need of modyfications in the drivers code. It uses atomic variables to
> > make sure no race condition affects the process.
> > 
> > Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
> > Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
> > Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
> > ---
> >  drivers/devfreq/devfreq.c | 47 +++++++++++++++++++++++++++++++++++++++++------
> >  include/linux/devfreq.h   |  7 +++++++
> >  2 files changed, 48 insertions(+), 6 deletions(-)
> 
> Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
> 

Looks goot do me as well.

Acked-by: MyungJoo Ham <myungjoo.ham@samsung.com>


Anyway, for the sake of curiosity...

Having suspend-frequency is usually required when
the frequency configuration is reset with suspend-resume
as older Exynos's CPU did (I don't know whether it still does).

Does GPU do this as well?
(memory-bus won't do this because they are kept turned on during suspend)


Cheers,
MyungJoo

RE: [PATCH v3 1/5] devfreq: refactor set_target frequency function

[MyungJoo Ham]

> The refactoring is needed for the new client in devfreq: suspend.
> To avoid code duplication, move it to the new local function
> devfreq_set_target.
> 
> Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
> Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
> Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
> Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
> ---

Acked-by: MyungJoo Ham <myungjoo.ham@samsung.com>

RE: Re: [PATCH v3 3/5] devfreq: add devfreq_suspend/resume() functions

[MyungJoo Ham]

> Hi Lukasz,
> 
> On 2018년 12월 05일 20:05, Lukasz Luba wrote:
> > This patch adds implementation for global suspend/resume for
> > devfreq framework. System suspend will next use these functions.
> > 
> > Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
> > Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
> > Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
> > ---
> >  drivers/devfreq/devfreq.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
> >  include/linux/devfreq.h   |  6 ++++++
> >  2 files changed, 50 insertions(+)
> 
> Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>

Acked-by: MyungJoo Ham <myungjoo.ham@samsung.com>

The patches from 1 to 3 are being applied and tested.
I'll probably send pull-request to Rafael today after some testing targetting 4.21+

Cheers,
MyungJoo

Re: [PATCH v3 2/5] devfreq: add support for suspend/resume of a devfreq device

[Lukasz Luba]

Hi MyungJoo,

Thank you for taking the patch set.
Please check my response bellow.

On 12/11/18 2:43 AM, MyungJoo Ham wrote:
>> Hi Lukasz,
>>
>> On 2018년 12월 05일 20:05, Lukasz Luba wrote:
>>> The patch prepares devfreq device for handling suspend/resume
>>> functionality. The new fields will store needed information during this
>>> process. Devfreq framework handles opp-suspend DT entry and there is no
>>> need of modyfications in the drivers code. It uses atomic variables to
>>> make sure no race condition affects the process.
>>>
>>> Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
>>> Suggested-by: Chanwoo Choi <cw00.choi@samsung.com>
>>> Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
>>> ---
>>>   drivers/devfreq/devfreq.c | 47 +++++++++++++++++++++++++++++++++++++++++------
>>>   include/linux/devfreq.h   |  7 +++++++
>>>   2 files changed, 48 insertions(+), 6 deletions(-)
>>
>> Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
>>
> 
> Looks goot do me as well.
> 
> Acked-by: MyungJoo Ham <myungjoo.ham@samsung.com>
> 
> 
> Anyway, for the sake of curiosity...
> 
> Having suspend-frequency is usually required when
> the frequency configuration is reset with suspend-resume
> as older Exynos's CPU did (I don't know whether it still does).
You are right, some Exynos SoCs like 4412 needs the proper configuration.
> 
> Does GPU do this as well?
> (memory-bus won't do this because they are kept turned on during suspend)
I haven't seen GPU failures.
The CPU resume process can fail when some buses ('leftbus' (data 
bus/peripheral bus), 'rightbus', 'memory-bus')
are not operating with needed frequency.

Here is the log showing CPU1,2,3 booting failure:
-------------------------------------------------------------------
root@target:~# echo mem > /sys/power/state
[   43.432066] PM: suspend entry (deep)
[   43.481742] PM: Syncing filesystems ... done.
[   43.553433] Freezing user space processes ... (elapsed 0.002 seconds) 
done.
[   43.556687] OOM killer disabled.
[   43.556694] Freezing remaining freezable tasks ... (elapsed 0.001 
seconds) done.
[   43.558305] printk: Suspending console(s) (use no_console_suspend to 
debug)

[   43.718007] dwc2 12480000.hsotg: suspending usb gadget g_ether
[   43.718229] dwc2 12480000.hsotg: dwc2_hsotg_ep_disable: called for ep0
[   43.718260] dwc2 12480000.hsotg: dwc2_hsotg_ep_disable: called for ep0
[   43.722021] dwc2 12480000.hsotg: dwc2_hsotg_ep_disable: called for ep0
[   43.722038] dwc2 12480000.hsotg: dwc2_hsotg_ep_disable: called for ep0
[   43.781714] wake enabled for irq 134
[   43.781822] CAM_ISP_CORE_1.2V: No configuration
[   43.781918] VMEM_VDDF_3.0V: No configuration
[   43.781952] VCC_SUB_2.0V: No configuration
[   43.781986] VCC_SUB_1.35V: No configuration
[   43.782018] VMEM_1.2V_AP: No configuration
[   43.787139] MOTOR_VCC_3.0V: No configuration
[   43.787176] LCD_VCC_3.3V: No configuration
[   43.787209] TSP_VDD_1.8V: No configuration
[   43.787242] TSP_AVDD_3.3V: No configuration
[   43.787273] VMEM_VDD_2.8V: No configuration
[   43.787305] VTF_2.8V: No configuration
[   43.787422] VDDQ_PRE_1.8V: No configuration
[   43.787456] VT_CAM_1.8V: No configuration
[   43.787488] CAM_ISP_SEN_IO_1.8V: No configuration
[   43.787521] CAM_SENSOR_CORE_1.2V: No configuration
[   43.789160] NFC_AVDD_1.8V: No configuration
[   43.792571] CAM_ISP_MIPI_1.2V: No configuration
[   43.794209] VCC_1.8V_IO: No configuration
[   43.794244] VCC_2.8V_AP: No configuration
[   43.794279] VCC_1.8V_AP: No configuration
[   43.794331] VALIVE_1.0V_AP: No configuration
[   43.805821] wake enabled for irq 138
[   43.806368] wake enabled for irq 160
[   43.806382] wake enabled for irq 161
[   43.824045] samsung-pinctrl 11000000.pinctrl: Setting external wakeup 
interrupt mask: 0xff77ff7d
[   43.841162] Disabling non-boot CPUs ...
[   43.863016] Enabling non-boot CPUs ...
[   43.864749] ------------[ cut here ]------------
[   43.864779] WARNING: CPU: 1 PID: 0 at 
./arch/arm/include/asm/proc-fns.h:124 secondary_start_kernel+0x20c/0x268
[   43.864788] Modules linked in:
[   43.864805] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 
4.20.0-rc2-next-20181114-00007-g5506a9cd6bd7 #1042
[   43.864813] Hardware name: SAMSUNG EXYNOS (Flattened Device Tree)
[   43.864835] [<c011261c>] (unwind_backtrace) from [<c010e150>] 
(show_stack+0x10/0x14)
[   43.864852] [<c010e150>] (show_stack) from [<c0a4b0a0>] 
(dump_stack+0x98/0xc4)
[   43.864870] [<c0a4b0a0>] (dump_stack) from [<c01270a4>] 
(__warn+0x10c/0x124)
[   43.864884] [<c01270a4>] (__warn) from [<c01271d0>] 
(warn_slowpath_null+0x40/0x48)
[   43.864899] [<c01271d0>] (warn_slowpath_null) from [<c0110ee0>] 
(secondary_start_kernel+0x20c/0x268)
[   43.864913] [<c0110ee0>] (secondary_start_kernel) from [<401027ac>] 
(0x401027ac)
[   43.864923] irq event stamp: 982984
[   43.864942] hardirqs last  enabled at (982983): [<c0a6cc6c>] 
_raw_spin_unlock_irqrestore+0x6c/0x74
[   43.864954] hardirqs last disabled at (982984): [<c0110c68>] 
arch_cpu_idle_dead+0x18/0x84
[   43.864969] softirqs last  enabled at (982964): [<c012f0e0>] 
irq_enter+0x78/0x80
[   43.864980] softirqs last disabled at (982963): [<c012f0cc>] 
irq_enter+0x64/0x80
[   43.864989] ---[ end trace 6401b9331547e26f ]---
[   43.864996] ------------[ cut here ]------------
[   43.865009] WARNING: CPU: 1 PID: 0 at 
./arch/arm/include/asm/proc-fns.h:126 secondary_start_kernel+0x244/0x268
[   43.865016] Modules linked in:
[   43.865030] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G        W 
4.20.0-rc2-next-20181114-00007-g5506a9cd6bd7 #1042
[   43.865039] Hardware name: SAMSUNG EXYNOS (Flattened Device Tree)
[   43.865054] [<c011261c>] (unwind_backtrace) from [<c010e150>] 
(show_stack+0x10/0x14)
[   43.865068] [<c010e150>] (show_stack) from [<c0a4b0a0>] 
(dump_stack+0x98/0xc4)
[   43.865081] [<c0a4b0a0>] (dump_stack) from [<c01270a4>] 
(__warn+0x10c/0x124)
[   43.865095] [<c01270a4>] (__warn) from [<c01271d0>] 
(warn_slowpath_null+0x40/0x48)
[   43.865109] [<c01271d0>] (warn_slowpath_null) from [<c0110f18>] 
(secondary_start_kernel+0x244/0x268)
[   43.865122] [<c0110f18>] (secondary_start_kernel) from [<401027ac>] 
(0x401027ac)
[   43.865131] irq event stamp: 982984
[   43.865144] hardirqs last  enabled at (982983): [<c0a6cc6c>] 
_raw_spin_unlock_irqrestore+0x6c/0x74
[   43.865156] hardirqs last disabled at (982984): [<c0110c68>] 
arch_cpu_idle_dead+0x18/0x84
[   43.865168] softirqs last  enabled at (982964): [<c012f0e0>] 
irq_enter+0x78/0x80
[   43.865179] softirqs last disabled at (982963): [<c012f0cc>] 
irq_enter+0x64/0x80
[   43.865187] ---[ end trace 6401b9331547e270 ]---
[   44.856140] CPU1: failed to boot: -110
[   44.856956] Error taking CPU1 up: -110
[   45.856655] CPU2: failed to boot: -110
[   45.857770] Error taking CPU2 up: -110
[   46.856059] CPU3: failed to boot: -110
[   46.856721] Error taking CPU3 up: -110
-------------------------------------------------------------------

With the patch set, the CPUs are up and running.
-------------------------------------------------------------------
==========================================================
root@target:~# echo mem > /sys/power/state
[ 1135.879168] PM: suspend entry (deep)
[ 1135.906378] PM: Syncing filesystems ...
[ 1135.919299] mmc_host mmc2: Bus speed (slot 0) = 50000000Hz (slot req 
400000Hz, actual 396825HZ div = 63)
[ 1136.137522] mmc_host mmc2: Bus speed (slot 0) = 50000000Hz (slot req 
52000000Hz, actual 50000000HZ div = 0)
[ 1136.184722] done.
[ 1136.196373] Freezing user space processes ... (elapsed 0.001 seconds) 
done.
[ 1136.197689] OOM killer disabled.
[ 1136.197692] Freezing remaining freezable tasks ... (elapsed 0.001 
seconds) done.
[ 1136.198923] printk: Suspending console(s) (use no_console_suspend to 
debug)
[ 1136.364056] devfreq8, suspend_freq=200000000
[ 1136.364114] devfreq4, suspend_freq=200000000
[ 1136.364174] devfreq3, suspend_freq=200000000
[ 1136.364217] devfreq2, suspend_freq=400000000
[ 1136.364311] devfreq0, suspend_freq=400000000
[ 1136.425416] wake enabled for irq 134
[ 1136.425439] CAM_ISP_CORE_1.2V: No configuration
[ 1136.425457] VMEM_VDDF_3.0V: No configuration
[ 1136.425476] VCC_SUB_2.0V: No configuration
[ 1136.425493] VCC_SUB_1.35V: No configuration
[ 1136.425510] VMEM_1.2V_AP: No configuration
[ 1136.426939] MOTOR_VCC_3.0V: No configuration
[ 1136.426958] LCD_VCC_3.3V: No configuration
[ 1136.426976] TSP_VDD_1.8V: No configuration
[ 1136.426994] TSP_AVDD_3.3V: No configuration
[ 1136.427012] VMEM_VDD_2.8V: No configuration
[ 1136.427029] VTF_2.8V: No configuration
[ 1136.427047] VDDQ_PRE_1.8V: No configuration
[ 1136.427064] VT_CAM_1.8V: No configuration
[ 1136.427082] CAM_ISP_SEN_IO_1.8V: No configuration
[ 1136.427100] CAM_SENSOR_CORE_1.2V: No configuration
[ 1136.427605] NFC_AVDD_1.8V: No configuration
[ 1136.429020] CAM_ISP_MIPI_1.2V: No configuration
[ 1136.429859] VCC_1.8V_IO: No configuration
[ 1136.429879] VCC_2.8V_AP: No configuration
[ 1136.429898] VCC_1.8V_AP: No configuration
[ 1136.429926] VALIVE_1.0V_AP: No configuration
[ 1136.435110] wake enabled for irq 138
[ 1136.435381] wake enabled for irq 160
[ 1136.435391] wake enabled for irq 161
[ 1136.443372] samsung-pinctrl 11000000.pinctrl: Setting external wakeup 
interrupt mask: 0xff77ff7d
[ 1136.450942] Disabling non-boot CPUs ...
[ 1136.575256] Enabling non-boot CPUs ...
[ 1136.577278] CPU1 is up
[ 1136.579226] CPU2 is up
[ 1136.581283] CPU3 is up
[ 1136.581773] s3c-i2c 13860000.i2c: slave address 0x10
[ 1136.581788] s3c-i2c 13860000.i2c: bus frequency set to 390 KHz
[ 1136.581809] s3c-i2c 13890000.i2c: slave address 0x10
[ 1136.581821] s3c-i2c 13890000.i2c: bus frequency set to 390 KHz
[ 1136.581841] s3c-i2c 138a0000.i2c: slave address 0x10
[ 1136.581866] s3c-i2c 138a0000.i2c: bus frequency set to 97 KHz
[ 1136.581887] s3c-i2c 138b0000.i2c: slave address 0x00
[ 1136.581899] s3c-i2c 138b0000.i2c: bus frequency set to 97 KHz
[ 1136.581919] s3c-i2c 138d0000.i2c: slave address 0x10
[ 1136.581930] s3c-i2c 138d0000.i2c: bus frequency set to 97 KHz
[ 1136.581949] s3c-i2c 138e0000.i2c: slave address 0x00
[ 1136.581961] s3c-i2c 138e0000.i2c: bus frequency set to 97 KHz
[ 1136.590858] s3c-rtc 10070000.rtc: rtc disabled, re-enabling
[ 1136.592539] s3c2410-wdt 10060000.watchdog: watchdog disabled
[ 1136.592636] wake disabled for irq 160
[ 1136.592646] wake disabled for irq 161
[ 1136.592768] wake disabled for irq 138
[ 1136.596830] wake disabled for irq 134
[ 1136.648824] mmc1: queuing unknown CIS tuple 0x80 (7 bytes)
[ 1136.651430] mmc1: queuing unknown CIS tuple 0x80 (6 bytes)
[ 1136.721638] devfreq8 resume_freq=200000000
[ 1136.721664] devfreq4 resume_freq=200000000
[ 1136.721681] devfreq3 resume_freq=200000000
[ 1136.721704] devfreq2 resume_freq=267000000
[ 1136.721738] devfreq0 resume_freq=267000000
[ 1136.944032] panel-samsung-s6e8aa0 11c80000.dsi.0: ID: 0xa2, 0x20, 0x8c
[ 1137.754939] OOM killer enabled.
[ 1137.760515] Restarting tasks ... done.
root@target:~# [ 1137.768132] PM: suspend exit
-------------------------------------------------------------------

Regards,
Lukasz
> 
> 
> Cheers,
> MyungJoo
> 
> 
> 

Re: [PATCH v3 4/5] drivers: power: suspend: call devfreq suspend/resume

[Lukasz Luba]

Hi Rafael,

I see that you have pulled 3 patches.
Please take also this patch, which is actually enabling the feature.

Regards,
Lukasz

On 12/5/18 12:05 PM, Lukasz Luba wrote:
> Devfreq framework supports suspend of its devices.
> Call the the devfreq interface and allow devfreq devices preserve/restore
> their states during suspend/resume.
> 
> Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
> Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
> Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
> ---
>   drivers/base/power/main.c | 3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/base/power/main.c b/drivers/base/power/main.c
> index a690fd4..0992e67 100644
> --- a/drivers/base/power/main.c
> +++ b/drivers/base/power/main.c
> @@ -32,6 +32,7 @@
>   #include <trace/events/power.h>
>   #include <linux/cpufreq.h>
>   #include <linux/cpuidle.h>
> +#include <linux/devfreq.h>
>   #include <linux/timer.h>
>   
>   #include "../base.h"
> @@ -1078,6 +1079,7 @@ void dpm_resume(pm_message_t state)
>   	dpm_show_time(starttime, state, 0, NULL);
>   
>   	cpufreq_resume();
> +	devfreq_resume();
>   	trace_suspend_resume(TPS("dpm_resume"), state.event, false);
>   }
>   
> @@ -1852,6 +1854,7 @@ int dpm_suspend(pm_message_t state)
>   	trace_suspend_resume(TPS("dpm_suspend"), state.event, true);
>   	might_sleep();
>   
> +	devfreq_suspend();
>   	cpufreq_suspend();
>   
>   	mutex_lock(&dpm_list_mtx);
> 

RE: Re: [PATCH v3 4/5] drivers: power: suspend: call devfreq suspend/resume

[MyungJoo Ham]

>On 2018-12-05 12:05, Lukasz Luba wrote:
>> Devfreq framework supports suspend of its devices.
>> Call the the devfreq interface and allow devfreq devices preserve/restore
>> their states during suspend/resume.
>>
>> Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
>> Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
>> Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>

This looks all good to me.

Acked-by: MyungJoo Ham <myungjoo.ham@samsung.com>

>> ---
>>  drivers/base/power/main.c | 3 +++
>>  1 file changed, 3 insertions(+)
>>
>> diff --git a/drivers/base/power/main.c b/drivers/base/power/main.c
>> index a690fd4..0992e67 100644
>> --- a/drivers/base/power/main.c
>> +++ b/drivers/base/power/main.c
>> @@ -32,6 +32,7 @@
>>  #include <trace/events/power.h>
>>  #include <linux/cpufreq.h>
>>  #include <linux/cpuidle.h>
>> +#include <linux/devfreq.h>
>>  #include <linux/timer.h>
>>  
>>  #include "../base.h"
>> @@ -1078,6 +1079,7 @@ void dpm_resume(pm_message_t state)
>>  	dpm_show_time(starttime, state, 0, NULL);
>>  
>>  	cpufreq_resume();
>> +	devfreq_resume();
>>  	trace_suspend_resume(TPS("dpm_resume"), state.event, false);
>>  }
>>  
>> @@ -1852,6 +1854,7 @@ int dpm_suspend(pm_message_t state)
>>  	trace_suspend_resume(TPS("dpm_suspend"), state.event, true);
>>  	might_sleep();
>>  
>> +	devfreq_suspend();
>>  	cpufreq_suspend();
>>  
>>  	mutex_lock(&dpm_list_mtx);

Re: [PATCH v3 4/5] drivers: power: suspend: call devfreq suspend/resume

[Lukasz Luba]

Hi Rafael,

Gentle ping...
Please take this patch, it got ACK.
The other patches have been taken through MyungJoo's
tree, but this one is actually enabling the feature.

Regards,
Lukasz


On 12/21/18 1:33 AM, MyungJoo Ham wrote:
>> On 2018-12-05 12:05, Lukasz Luba wrote:
>>> Devfreq framework supports suspend of its devices.
>>> Call the the devfreq interface and allow devfreq devices preserve/restore
>>> their states during suspend/resume.
>>>
>>> Suggested-by: Tobias Jakobi <tjakobi@math.uni-bielefeld.de>
>>> Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com>
>>> Signed-off-by: Lukasz Luba <l.luba@partner.samsung.com>
> 
> This looks all good to me.
> 
> Acked-by: MyungJoo Ham <myungjoo.ham@samsung.com>
> 
>>> ---
>>>   drivers/base/power/main.c | 3 +++
>>>   1 file changed, 3 insertions(+)
>>>
>>> diff --git a/drivers/base/power/main.c b/drivers/base/power/main.c
>>> index a690fd4..0992e67 100644
>>> --- a/drivers/base/power/main.c
>>> +++ b/drivers/base/power/main.c
>>> @@ -32,6 +32,7 @@
>>>   #include <trace/events/power.h>
>>>   #include <linux/cpufreq.h>
>>>   #include <linux/cpuidle.h>
>>> +#include <linux/devfreq.h>
>>>   #include <linux/timer.h>
>>>   
>>>   #include "../base.h"
>>> @@ -1078,6 +1079,7 @@ void dpm_resume(pm_message_t state)
>>>   	dpm_show_time(starttime, state, 0, NULL);
>>>   
>>>   	cpufreq_resume();
>>> +	devfreq_resume();
>>>   	trace_suspend_resume(TPS("dpm_resume"), state.event, false);
>>>   }
>>>   
>>> @@ -1852,6 +1854,7 @@ int dpm_suspend(pm_message_t state)
>>>   	trace_suspend_resume(TPS("dpm_suspend"), state.event, true);
>>>   	might_sleep();
>>>   
>>> +	devfreq_suspend();
>>>   	cpufreq_suspend();
>>>   
>>>   	mutex_lock(&dpm_list_mtx);
> 
> 

Re: [PATCH v3 4/5] drivers: power: suspend: call devfreq suspend/resume

[Rafael J. Wysocki]

On Wed, Jan 2, 2019 at 4:08 PM Lukasz Luba <l.luba@partner.samsung.com> wrote:
>
> Hi Rafael,
>
> Gentle ping...
> Please take this patch, it got ACK.
> The other patches have been taken through MyungJoo's
> tree, but this one is actually enabling the feature.

Applied now and sorry for missing it.

Thanks!