[RFC 0/3] Introduce cpufreq minimum load QoS

[RFC 0/3] Introduce cpufreq minimum load QoS

[Benjamin Gaignard]

When start streaming from the sensor the CPU load could remain very low 
because almost all the capture pipeline is done in hardware (i.e. without 
using the CPU) and let believe to cpufreq governor that it could use lower 
frequencies. If the governor decides to use a too low frequency that 
becomes a problem when we need to acknowledge the interrupt during the 
blanking time.
The delay to ack the interrupt and perform all the other actions before
the next frame is very short and doesn't allow to the cpufreq governor to
provide the required burst of power. That led to drop the half of the frames.

To avoid this problem, DCMI driver informs the cpufreq governors by adding
a cpufreq minimum load QoS resquest.

Benjamin Gaignard (3):
  PM: QoS: Introduce cpufreq minimum load QoS
  cpufreq: governor: Use minimum load QoS
  media: stm32-dcmi: Inform cpufreq governors about cpu load needs

 drivers/cpufreq/cpufreq_governor.c        |   5 +
 drivers/media/platform/stm32/stm32-dcmi.c |   8 ++
 include/linux/pm_qos.h                    |  12 ++
 kernel/power/qos.c                        | 213 ++++++++++++++++++++++++++++++
 4 files changed, 238 insertions(+)

-- 
2.15.0

[RFC 1/3] PM: QoS: Introduce cpufreq minimum load QoS

[Benjamin Gaignard]

Introduce cpufreq minimum load QoS, based on the "raw" low-level
PM QoS, to represent the minimum expected cpu load by various devices.

The cpufreq_minload_qos_limit() helper is defined to retrieve the
aggregated constraints.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@st.com>
---
 include/linux/pm_qos.h |  12 +++
 kernel/power/qos.c     | 213 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 225 insertions(+)

diff --git a/include/linux/pm_qos.h b/include/linux/pm_qos.h
index 4a69d4af3ff8..e2cc099322e3 100644
--- a/include/linux/pm_qos.h
+++ b/include/linux/pm_qos.h
@@ -316,4 +316,16 @@ int freq_qos_remove_notifier(struct freq_constraints *qos,
 			     enum freq_qos_req_type type,
 			     struct notifier_block *notifier);
 
+/* Definitions related to the cpufreq minimum load QoS. */
+
+#define CPUFREQ_GOV_QOS_MIN_LOAD_DEFAULT_VALUE	0
+#define CPUFREQ_GOV_QOS_MIN_LOAD_MAX_VALUE	10
+
+s32 cpufreq_minload_qos_limit(void);
+bool cpufreq_minload_qos_request_active(struct pm_qos_request *req);
+void cpufreq_minload_qos_add_request(struct pm_qos_request *req, s32 value);
+void cpufreq_minload_qos_update_request(struct pm_qos_request *req,
+					s32 new_value);
+void cpufreq_minload_qos_remove_request(struct pm_qos_request *req);
+
 #endif
diff --git a/kernel/power/qos.c b/kernel/power/qos.c
index db0bed2cae26..df2fdd962f35 100644
--- a/kernel/power/qos.c
+++ b/kernel/power/qos.c
@@ -671,3 +671,216 @@ int freq_qos_remove_notifier(struct freq_constraints *qos,
 	return ret;
 }
 EXPORT_SYMBOL_GPL(freq_qos_remove_notifier);
+
+/* Definitions related to the cpufreq minimum load QoS. */
+
+static struct pm_qos_constraints cpufreq_minload_constraints = {
+	.list = PLIST_HEAD_INIT(cpufreq_minload_constraints.list),
+	.target_value = CPUFREQ_GOV_QOS_MIN_LOAD_DEFAULT_VALUE,
+	.default_value = CPUFREQ_GOV_QOS_MIN_LOAD_DEFAULT_VALUE,
+	.no_constraint_value = CPUFREQ_GOV_QOS_MIN_LOAD_DEFAULT_VALUE,
+	.type = PM_QOS_MAX,
+};
+
+/**
+ * cpufreq_minload_qos_limit - Return current system-wide cpufreq
+ * minimum load QoS limit.
+ */
+s32 cpufreq_minload_qos_limit(void)
+{
+	return pm_qos_read_value(&cpufreq_minload_constraints);
+}
+EXPORT_SYMBOL_GPL(cpufreq_minload_qos_limit);
+
+/**
+ * cpufreq_minload_qos_request_active - Check the given PM QoS request.
+ * @req: PM QoS request to check.
+ *
+ * Return: 'true' if @req has been added to the cpufreq minimum load
+ * QoS list, 'false' otherwise.
+ */
+bool cpufreq_minload_qos_request_active(struct pm_qos_request *req)
+{
+	return req->qos == &cpufreq_minload_constraints;
+}
+EXPORT_SYMBOL_GPL(cpufreq_minload_qos_request_active);
+
+/**
+ * cpufreq_minload_qos_add_request - Add new cpufreq minimum load QoS request.
+ * @req: Pointer to a preallocated handle.
+ * @value: Requested constraint value.
+ *
+ * Use @value to initialize the request handle pointed to by @req, insert it as
+ * a new entry to the cpufreq minimum load QoS list and recompute the effective
+ * QoS constraint for that list.
+ *
+ * Callers need to save the handle for later use in updates and removal of the
+ * QoS request represented by it.
+ */
+void cpufreq_minload_qos_add_request(struct pm_qos_request *req, s32 value)
+{
+	if (!req)
+		return;
+
+	if (cpufreq_minload_qos_request_active(req)) {
+		WARN(1, KERN_ERR "%s called for already added request\n", __func__);
+		return;
+	}
+
+	trace_pm_qos_add_request(value);
+
+	req->qos = &cpufreq_minload_constraints;
+	pm_qos_update_target(req->qos, &req->node, PM_QOS_ADD_REQ, value);
+}
+EXPORT_SYMBOL_GPL(cpufreq_minload_qos_add_request);
+
+/**
+ * cpufreq_minload_qos_update_request - Modify existing cpufreq minimum load
+ * QoS request.
+ * @req : QoS request to update.
+ * @new_value: New requested constraint value.
+ *
+ * Use @new_value to update the QoS request represented by @req in the cpufreq
+ * minimum load QoS list along with updating the effective constraint value for
+ * that list.
+ */
+void cpufreq_minload_qos_update_request(struct pm_qos_request *req,
+					s32 new_value)
+{
+	if (!req)
+		return;
+
+	if (!cpufreq_minload_qos_request_active(req)) {
+		WARN(1, KERN_ERR "%s called for unknown object\n", __func__);
+		return;
+	}
+
+	trace_pm_qos_update_request(new_value);
+
+	if (new_value == req->node.prio)
+		return;
+
+	pm_qos_update_target(req->qos, &req->node, PM_QOS_UPDATE_REQ, new_value);
+}
+EXPORT_SYMBOL_GPL(cpufreq_minload_qos_update_request);
+
+/**
+ * cpufreq_minload_qos_remove_request - Remove existing cpufreq minimum load QoS
+ * request.
+ * @req: QoS request to remove.
+ *
+ * Remove the cpufreq minimum load QoS request represented by @req from the
+ * cpufreq minimum load QoS list along with updating the effective constraint
+ * value for that list.
+ */
+void cpufreq_minload_qos_remove_request(struct pm_qos_request *req)
+{
+	if (!req)
+		return;
+
+	if (!cpufreq_minload_qos_request_active(req)) {
+		WARN(1, KERN_ERR "%s called for unknown object\n", __func__);
+		return;
+	}
+
+	trace_pm_qos_remove_request(PM_QOS_DEFAULT_VALUE);
+
+	pm_qos_update_target(req->qos, &req->node,
+			     PM_QOS_REMOVE_REQ, PM_QOS_DEFAULT_VALUE);
+	memset(req, 0, sizeof(*req));
+}
+EXPORT_SYMBOL_GPL(cpufreq_minload_qos_remove_request);
+
+/* User space interface to the cpufreq minimum load QoS via misc device. */
+
+static int cpufreq_minload_qos_open(struct inode *inode, struct file *filp)
+{
+	struct pm_qos_request *req;
+
+	req = kzalloc(sizeof(*req), GFP_KERNEL);
+	if (!req)
+		return -ENOMEM;
+
+	cpufreq_minload_qos_add_request(req, PM_QOS_DEFAULT_VALUE);
+	filp->private_data = req;
+
+	return 0;
+}
+
+static int cpufreq_minload_qos_release(struct inode *inode, struct file *filp)
+{
+	struct pm_qos_request *req = filp->private_data;
+
+	filp->private_data = NULL;
+
+	cpufreq_minload_qos_remove_request(req);
+	kfree(req);
+
+	return 0;
+}
+
+static ssize_t cpufreq_minload_qos_read(struct file *filp, char __user *buf,
+					size_t count, loff_t *f_pos)
+{
+	struct pm_qos_request *req = filp->private_data;
+	unsigned long flags;
+	s32 value;
+
+	if (!req || !cpufreq_minload_qos_request_active(req))
+		return -EINVAL;
+
+	spin_lock_irqsave(&pm_qos_lock, flags);
+	value = pm_qos_get_value(&cpufreq_minload_constraints);
+	spin_unlock_irqrestore(&pm_qos_lock, flags);
+
+	return simple_read_from_buffer(buf, count, f_pos, &value, sizeof(s32));
+}
+
+static ssize_t cpufreq_minload_qos_write(struct file *filp,
+					 const char __user *buf,
+					 size_t count, loff_t *f_pos)
+{
+	s32 value;
+
+	if (count == sizeof(s32)) {
+		if (copy_from_user(&value, buf, sizeof(s32)))
+			return -EFAULT;
+	} else {
+		int ret;
+
+		ret = kstrtos32_from_user(buf, count, 16, &value);
+		if (ret)
+			return ret;
+	}
+
+	cpufreq_minload_qos_update_request(filp->private_data, value);
+
+	return count;
+}
+
+static const struct file_operations cpufreq_minload_qos_fops = {
+	.write = cpufreq_minload_qos_write,
+	.read = cpufreq_minload_qos_read,
+	.open = cpufreq_minload_qos_open,
+	.release = cpufreq_minload_qos_release,
+	.llseek = noop_llseek,
+};
+
+static struct miscdevice cpufreq_minload_qos_miscdev = {
+	.minor = MISC_DYNAMIC_MINOR,
+	.name = "cpufreq_minimum_load",
+	.fops = &cpufreq_minload_qos_fops,
+};
+
+static int __init cpufreq_minload_qos_init(void)
+{
+	int ret;
+
+	ret = misc_register(&cpufreq_minload_qos_miscdev);
+	if (ret < 0)
+		pr_err("%s: %s setup failed\n", __func__,
+		       cpufreq_minload_qos_miscdev.name);
+
+	return ret;
+}
+late_initcall(cpufreq_minload_qos_init);
-- 
2.15.0

[RFC 2/3] cpufreq: governor: Use minimum load QoS

[Benjamin Gaignard]

Make sure that the returned load is above the system-wide minimum
load QoS.
Devices could set this specific QoS to inform governors about their
need in terms of CPU load when computing it from idle time isn't accurate.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@st.com>
---
 drivers/cpufreq/cpufreq_governor.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/cpufreq/cpufreq_governor.c b/drivers/cpufreq/cpufreq_governor.c
index f99ae45efaea..1494e5e4c788 100644
--- a/drivers/cpufreq/cpufreq_governor.c
+++ b/drivers/cpufreq/cpufreq_governor.c
@@ -118,6 +118,7 @@ unsigned int dbs_update(struct cpufreq_policy *policy)
 	unsigned int ignore_nice = dbs_data->ignore_nice_load;
 	unsigned int max_load = 0, idle_periods = UINT_MAX;
 	unsigned int sampling_rate, io_busy, j;
+	unsigned int qos_min_load;
 
 	/*
 	 * Sometimes governors may use an additional multiplier to increase
@@ -225,6 +226,10 @@ unsigned int dbs_update(struct cpufreq_policy *policy)
 
 	policy_dbs->idle_periods = idle_periods;
 
+	qos_min_load = cpufreq_minload_qos_limit();
+	if (qos_min_load > max_load)
+		max_load = qos_min_load;
+
 	return max_load;
 }
 EXPORT_SYMBOL_GPL(dbs_update);
-- 
2.15.0

[RFC 3/3] media: stm32-dcmi: Inform cpufreq governors about cpu load needs

[Benjamin Gaignard]

When start streaming the CPU load could remain very low because almost
all the capture pipeline is done in hardware (i.e. without using the CPU)
and let believe to cpufreq governor that it could use lower frequencies.
If the governor decides to use a too low frequency that becomes a problem
when we need to acknowledge the interrupt during the blanking time.

To avoid this problem, DCMI driver informs the cpufreq governors by adding
a cpufreq minimum load QoS resquest.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@st.com>
---
 drivers/media/platform/stm32/stm32-dcmi.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/media/platform/stm32/stm32-dcmi.c b/drivers/media/platform/stm32/stm32-dcmi.c
index b8931490b83b..774f2506b2f1 100644
--- a/drivers/media/platform/stm32/stm32-dcmi.c
+++ b/drivers/media/platform/stm32/stm32-dcmi.c
@@ -24,6 +24,7 @@
 #include <linux/of_graph.h>
 #include <linux/pinctrl/consumer.h>
 #include <linux/platform_device.h>
+#include <linux/pm_qos.h>
 #include <linux/pm_runtime.h>
 #include <linux/reset.h>
 #include <linux/videodev2.h>
@@ -173,6 +174,8 @@ struct stm32_dcmi {
 	struct media_device		mdev;
 	struct media_pad		vid_cap_pad;
 	struct media_pipeline		pipeline;
+
+	struct pm_qos_request		qos_request;
 };
 
 static inline struct stm32_dcmi *notifier_to_dcmi(struct v4l2_async_notifier *n)
@@ -827,6 +830,9 @@ static int dcmi_start_streaming(struct vb2_queue *vq, unsigned int count)
 	else
 		reg_set(dcmi->regs, DCMI_IER, IT_OVR | IT_ERR);
 
+	cpufreq_minload_qos_add_request(&dcmi->qos_request,
+					CPUFREQ_GOV_QOS_MIN_LOAD_MAX_VALUE);
+
 	return 0;
 
 err_pipeline_stop:
@@ -859,6 +865,8 @@ static void dcmi_stop_streaming(struct vb2_queue *vq)
 	struct stm32_dcmi *dcmi = vb2_get_drv_priv(vq);
 	struct dcmi_buf *buf, *node;
 
+	cpufreq_minload_qos_remove_request(&dcmi->qos_request);
+
 	dcmi_pipeline_stop(dcmi);
 
 	media_pipeline_stop(&dcmi->vdev->entity);
-- 
2.15.0

Re: [RFC 0/3] Introduce cpufreq minimum load QoS

[Rafael J. Wysocki]

On Friday, April 24, 2020 1:40:55 PM CEST Benjamin Gaignard wrote:
> When start streaming from the sensor the CPU load could remain very low 
> because almost all the capture pipeline is done in hardware (i.e. without 
> using the CPU) and let believe to cpufreq governor that it could use lower 
> frequencies. If the governor decides to use a too low frequency that 
> becomes a problem when we need to acknowledge the interrupt during the 
> blanking time.
> The delay to ack the interrupt and perform all the other actions before
> the next frame is very short and doesn't allow to the cpufreq governor to
> provide the required burst of power. That led to drop the half of the frames.
> 
> To avoid this problem, DCMI driver informs the cpufreq governors by adding
> a cpufreq minimum load QoS resquest.

This seems to be addressing a use case that can be addressed with the help of
utilization clamps with less power overhead.

Thanks!

Re: [RFC 0/3] Introduce cpufreq minimum load QoS

[Benjamin GAIGNARD]

On 4/29/20 5:50 PM, Rafael J. Wysocki wrote:
> On Friday, April 24, 2020 1:40:55 PM CEST Benjamin Gaignard wrote:
>> When start streaming from the sensor the CPU load could remain very low
>> because almost all the capture pipeline is done in hardware (i.e. without
>> using the CPU) and let believe to cpufreq governor that it could use lower
>> frequencies. If the governor decides to use a too low frequency that
>> becomes a problem when we need to acknowledge the interrupt during the
>> blanking time.
>> The delay to ack the interrupt and perform all the other actions before
>> the next frame is very short and doesn't allow to the cpufreq governor to
>> provide the required burst of power. That led to drop the half of the frames.
>>
>> To avoid this problem, DCMI driver informs the cpufreq governors by adding
>> a cpufreq minimum load QoS resquest.
> This seems to be addressing a use case that can be addressed with the help of
> utilization clamps with less power overhead.
Do mean clamping the policy frequencies ? I may have miss the API to do 
that...
>
> Thanks!
>
>
>

Re: [RFC 0/3] Introduce cpufreq minimum load QoS

[Valentin Schneider]

On 29/04/2020 16:57, Benjamin GAIGNARD wrote:
> 
> 
> On 4/29/20 5:50 PM, Rafael J. Wysocki wrote:
>> On Friday, April 24, 2020 1:40:55 PM CEST Benjamin Gaignard wrote:
>>> When start streaming from the sensor the CPU load could remain very low
>>> because almost all the capture pipeline is done in hardware (i.e. without
>>> using the CPU) and let believe to cpufreq governor that it could use lower
>>> frequencies. If the governor decides to use a too low frequency that
>>> becomes a problem when we need to acknowledge the interrupt during the
>>> blanking time.
>>> The delay to ack the interrupt and perform all the other actions before
>>> the next frame is very short and doesn't allow to the cpufreq governor to
>>> provide the required burst of power. That led to drop the half of the frames.
>>>
>>> To avoid this problem, DCMI driver informs the cpufreq governors by adding
>>> a cpufreq minimum load QoS resquest.
>> This seems to be addressing a use case that can be addressed with the help of
>> utilization clamps with less power overhead.
> Do mean clamping the policy frequencies ? I may have miss the API to do 
> that...
>>

IIUC Rafael is referring to uclamp, i.e. scheduler utilization clamping, see:

  https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html#cpu

The above describes the cgroup interface, note that you can also set clamps
per task (via sched_setattr()).

One thing that comes to mind however is that schedutil only "sees" the clamps
of runnable tasks, and from reading your changelog you may not have moments
without any (i.e. gears are grinding in HW). You'd have to try boosting
(setting a high uclamp.min) whatever tasks you have on the software side and
see how it all behaves.

>> Thanks!
>>
>>
>>

Re: [RFC 0/3] Introduce cpufreq minimum load QoS

[Vincent Guittot]

On Wed, 29 Apr 2020 at 17:50, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
>
> On Friday, April 24, 2020 1:40:55 PM CEST Benjamin Gaignard wrote:
> > When start streaming from the sensor the CPU load could remain very low
> > because almost all the capture pipeline is done in hardware (i.e. without
> > using the CPU) and let believe to cpufreq governor that it could use lower
> > frequencies. If the governor decides to use a too low frequency that
> > becomes a problem when we need to acknowledge the interrupt during the
> > blanking time.
> > The delay to ack the interrupt and perform all the other actions before
> > the next frame is very short and doesn't allow to the cpufreq governor to
> > provide the required burst of power. That led to drop the half of the frames.
> >
> > To avoid this problem, DCMI driver informs the cpufreq governors by adding
> > a cpufreq minimum load QoS resquest.
>
> This seems to be addressing a use case that can be addressed with the help of
> utilization clamps with less power overhead.

Can't freq_qos_update_request() be also used if you don't have cgroup
enabled on your system ?

>
> Thanks!
>
>
>

Re: [RFC 0/3] Introduce cpufreq minimum load QoS

[Benjamin GAIGNARD]

On 4/29/20 6:12 PM, Valentin Schneider wrote:
> On 29/04/2020 16:57, Benjamin GAIGNARD wrote:
>>
>> On 4/29/20 5:50 PM, Rafael J. Wysocki wrote:
>>> On Friday, April 24, 2020 1:40:55 PM CEST Benjamin Gaignard wrote:
>>>> When start streaming from the sensor the CPU load could remain very low
>>>> because almost all the capture pipeline is done in hardware (i.e. without
>>>> using the CPU) and let believe to cpufreq governor that it could use lower
>>>> frequencies. If the governor decides to use a too low frequency that
>>>> becomes a problem when we need to acknowledge the interrupt during the
>>>> blanking time.
>>>> The delay to ack the interrupt and perform all the other actions before
>>>> the next frame is very short and doesn't allow to the cpufreq governor to
>>>> provide the required burst of power. That led to drop the half of the frames.
>>>>
>>>> To avoid this problem, DCMI driver informs the cpufreq governors by adding
>>>> a cpufreq minimum load QoS resquest.
>>> This seems to be addressing a use case that can be addressed with the help of
>>> utilization clamps with less power overhead.
>> Do mean clamping the policy frequencies ? I may have miss the API to do
>> that...
> IIUC Rafael is referring to uclamp, i.e. scheduler utilization clamping, see:
>
>    https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html#cpu
>
> The above describes the cgroup interface, note that you can also set clamps
> per task (via sched_setattr()).
>
> One thing that comes to mind however is that schedutil only "sees" the clamps
> of runnable tasks, and from reading your changelog you may not have moments
> without any (i.e. gears are grinding in HW). You'd have to try boosting
> (setting a high uclamp.min) whatever tasks you have on the software side and
> see how it all behaves.
Relying on userland side means that various applications need to be aware
of this specific hardware case and fix it. I was hoping to find a 
solution in side the kernel
to not impact the software side.

>
>>> Thanks!
>>>
>>>
>>>

Re: [RFC 0/3] Introduce cpufreq minimum load QoS

[Rafael J. Wysocki]

On Wed, Apr 29, 2020 at 7:08 PM Vincent Guittot
<vincent.guittot@linaro.org> wrote:
>
> On Wed, 29 Apr 2020 at 17:50, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> >
> > On Friday, April 24, 2020 1:40:55 PM CEST Benjamin Gaignard wrote:
> > > When start streaming from the sensor the CPU load could remain very low
> > > because almost all the capture pipeline is done in hardware (i.e. without
> > > using the CPU) and let believe to cpufreq governor that it could use lower
> > > frequencies. If the governor decides to use a too low frequency that
> > > becomes a problem when we need to acknowledge the interrupt during the
> > > blanking time.
> > > The delay to ack the interrupt and perform all the other actions before
> > > the next frame is very short and doesn't allow to the cpufreq governor to
> > > provide the required burst of power. That led to drop the half of the frames.
> > >
> > > To avoid this problem, DCMI driver informs the cpufreq governors by adding
> > > a cpufreq minimum load QoS resquest.
> >
> > This seems to be addressing a use case that can be addressed with the help of
> > utilization clamps with less power overhead.
>
> Can't freq_qos_update_request() be also used if you don't have cgroup
> enabled on your system ?

It can.

The problem here is that imposing a global minimum frequency limit
generally causes the power draw of the system to increase regardless
of what is going on, including the CPUs that are not involved in the
handling of the interrupt in question.

That seems a bit excessive ...

Re: [RFC 0/3] Introduce cpufreq minimum load QoS

[Rafael J. Wysocki]

On Thu, Apr 30, 2020 at 9:53 AM Benjamin GAIGNARD
<benjamin.gaignard@st.com> wrote:
>
>
>
> On 4/29/20 6:12 PM, Valentin Schneider wrote:
> > On 29/04/2020 16:57, Benjamin GAIGNARD wrote:
> >>
> >> On 4/29/20 5:50 PM, Rafael J. Wysocki wrote:
> >>> On Friday, April 24, 2020 1:40:55 PM CEST Benjamin Gaignard wrote:
> >>>> When start streaming from the sensor the CPU load could remain very low
> >>>> because almost all the capture pipeline is done in hardware (i.e. without
> >>>> using the CPU) and let believe to cpufreq governor that it could use lower
> >>>> frequencies. If the governor decides to use a too low frequency that
> >>>> becomes a problem when we need to acknowledge the interrupt during the
> >>>> blanking time.
> >>>> The delay to ack the interrupt and perform all the other actions before
> >>>> the next frame is very short and doesn't allow to the cpufreq governor to
> >>>> provide the required burst of power. That led to drop the half of the frames.
> >>>>
> >>>> To avoid this problem, DCMI driver informs the cpufreq governors by adding
> >>>> a cpufreq minimum load QoS resquest.
> >>> This seems to be addressing a use case that can be addressed with the help of
> >>> utilization clamps with less power overhead.
> >> Do mean clamping the policy frequencies ? I may have miss the API to do
> >> that...
> > IIUC Rafael is referring to uclamp, i.e. scheduler utilization clamping, see:
> >
> >    https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html#cpu
> >
> > The above describes the cgroup interface, note that you can also set clamps
> > per task (via sched_setattr()).
> >
> > One thing that comes to mind however is that schedutil only "sees" the clamps
> > of runnable tasks, and from reading your changelog you may not have moments
> > without any (i.e. gears are grinding in HW). You'd have to try boosting
> > (setting a high uclamp.min) whatever tasks you have on the software side and
> > see how it all behaves.
>
> Relying on userland side means that various applications need to be aware
> of this specific hardware case and fix it. I was hoping to find a
> solution in side the kernel
> to not impact the software side.

That's not what I meant.

I suppose that the interrupt processing in question takes place in
process context and so you may set the lower clamp on the utilization
of the task carrying that out.

Alternatively, that task may be a deadline one.

Re: [RFC 0/3] Introduce cpufreq minimum load QoS

[Benjamin GAIGNARD]

On 4/30/20 11:03 AM, Rafael J. Wysocki wrote:
> On Thu, Apr 30, 2020 at 9:53 AM Benjamin GAIGNARD
> <benjamin.gaignard@st.com> wrote:
>>
>>
>> On 4/29/20 6:12 PM, Valentin Schneider wrote:
>>> On 29/04/2020 16:57, Benjamin GAIGNARD wrote:
>>>> On 4/29/20 5:50 PM, Rafael J. Wysocki wrote:
>>>>> On Friday, April 24, 2020 1:40:55 PM CEST Benjamin Gaignard wrote:
>>>>>> When start streaming from the sensor the CPU load could remain very low
>>>>>> because almost all the capture pipeline is done in hardware (i.e. without
>>>>>> using the CPU) and let believe to cpufreq governor that it could use lower
>>>>>> frequencies. If the governor decides to use a too low frequency that
>>>>>> becomes a problem when we need to acknowledge the interrupt during the
>>>>>> blanking time.
>>>>>> The delay to ack the interrupt and perform all the other actions before
>>>>>> the next frame is very short and doesn't allow to the cpufreq governor to
>>>>>> provide the required burst of power. That led to drop the half of the frames.
>>>>>>
>>>>>> To avoid this problem, DCMI driver informs the cpufreq governors by adding
>>>>>> a cpufreq minimum load QoS resquest.
>>>>> This seems to be addressing a use case that can be addressed with the help of
>>>>> utilization clamps with less power overhead.
>>>> Do mean clamping the policy frequencies ? I may have miss the API to do
>>>> that...
>>> IIUC Rafael is referring to uclamp, i.e. scheduler utilization clamping, see:
>>>
>>>     https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html#cpu
>>>
>>> The above describes the cgroup interface, note that you can also set clamps
>>> per task (via sched_setattr()).
>>>
>>> One thing that comes to mind however is that schedutil only "sees" the clamps
>>> of runnable tasks, and from reading your changelog you may not have moments
>>> without any (i.e. gears are grinding in HW). You'd have to try boosting
>>> (setting a high uclamp.min) whatever tasks you have on the software side and
>>> see how it all behaves.
>> Relying on userland side means that various applications need to be aware
>> of this specific hardware case and fix it. I was hoping to find a
>> solution in side the kernel
>> to not impact the software side.
> That's not what I meant.
>
> I suppose that the interrupt processing in question takes place in
> process context and so you may set the lower clamp on the utilization
> of the task carrying that out.
I have try to add this code when starting streaming (before the first 
interrupt) the frames from the sensor:
const struct sched_attr sched_attr = {
         .sched_util_min = 10000, /* 100% of usage */
         .sched_flags = SCHED_FLAG_UTIL_CLAMP_MIN,
     };

sched_setattr(current, &sched_attr);

I don't see any benefices maybe there is some configuration flags to set.

How changing sched_util_min could impact cpufreq ondemand governor ?
Does it change the value returned when the governor check the idle time ?

>
> Alternatively, that task may be a deadline one.

Re: [RFC 0/3] Introduce cpufreq minimum load QoS

[Valentin Schneider]

On 30/04/20 14:46, Benjamin GAIGNARD wrote:
>> That's not what I meant.
>>
>> I suppose that the interrupt processing in question takes place in
>> process context and so you may set the lower clamp on the utilization
>> of the task carrying that out.
>
> I have try to add this code when starting streaming (before the first
> interrupt) the frames from the sensor:
> const struct sched_attr sched_attr = {
>    .sched_util_min = 10000, /* 100% of usage */

Unless you play with SCHED_CAPACITY_SHIFT, the max should be 1024 -
i.e. SCHED_CAPACITY_SCALE. That's a really big boost, but that's for you to
benchmark.

>    .sched_flags = SCHED_FLAG_UTIL_CLAMP_MIN,
>   };
>
> sched_setattr(current, &sched_attr);
>
> I don't see any benefices maybe there is some configuration flags to set.
>
> How changing sched_util_min could impact cpufreq ondemand governor ?
> Does it change the value returned when the governor check the idle time ?
>

You'll have to use the schedutil governor for uclamp to have an effect. And
arguably that's what you should be using, unless something explicitly
prevents you from doing that.

>>
>> Alternatively, that task may be a deadline one.

Re: [RFC 0/3] Introduce cpufreq minimum load QoS

[Benjamin GAIGNARD]

On 4/30/20 4:33 PM, Valentin Schneider wrote:
> On 30/04/20 14:46, Benjamin GAIGNARD wrote:
>>> That's not what I meant.
>>>
>>> I suppose that the interrupt processing in question takes place in
>>> process context and so you may set the lower clamp on the utilization
>>> of the task carrying that out.
>> I have try to add this code when starting streaming (before the first
>> interrupt) the frames from the sensor:
>> const struct sched_attr sched_attr = {
>>     .sched_util_min = 10000, /* 100% of usage */
> Unless you play with SCHED_CAPACITY_SHIFT, the max should be 1024 -
> i.e. SCHED_CAPACITY_SCALE. That's a really big boost, but that's for you to
> benchmark.
>
>>     .sched_flags = SCHED_FLAG_UTIL_CLAMP_MIN,
>>    };
>>
>> sched_setattr(current, &sched_attr);
>>
>> I don't see any benefices maybe there is some configuration flags to set.
>>
>> How changing sched_util_min could impact cpufreq ondemand governor ?
>> Does it change the value returned when the governor check the idle time ?
>>
> You'll have to use the schedutil governor for uclamp to have an effect. And
> arguably that's what you should be using, unless something explicitly
> prevents you from doing that.
Even with schedutil and SCHED_CAPACITY_SCALE that it doesn't work.
cpufreq/cpuinfo_cur_freq values are always on the max value even if the 
stats show transitions between the available frequencies.

I see two possibles reasons to explain that:
- sched_setattr() is called in userland process context, but the 
threaded irq handler is running in another process.
- because this use case is almost running all in hardware the process 
isn't doing anything so the scheduler doesn't take care of it.

>
>>> Alternatively, that task may be a deadline one.

Re: [RFC 0/3] Introduce cpufreq minimum load QoS

[Valentin Schneider]

On 30/04/20 16:37, Benjamin GAIGNARD wrote:
> On 4/30/20 4:33 PM, Valentin Schneider wrote:
>> On 30/04/20 14:46, Benjamin GAIGNARD wrote:
>>>> That's not what I meant.
>>>>
>>>> I suppose that the interrupt processing in question takes place in
>>>> process context and so you may set the lower clamp on the utilization
>>>> of the task carrying that out.
>>> I have try to add this code when starting streaming (before the first
>>> interrupt) the frames from the sensor:
>>> const struct sched_attr sched_attr = {
>>>     .sched_util_min = 10000, /* 100% of usage */
>> Unless you play with SCHED_CAPACITY_SHIFT, the max should be 1024 -
>> i.e. SCHED_CAPACITY_SCALE. That's a really big boost, but that's for you to
>> benchmark.
>>
>>>     .sched_flags = SCHED_FLAG_UTIL_CLAMP_MIN,
>>>    };
>>>
>>> sched_setattr(current, &sched_attr);
>>>
>>> I don't see any benefices maybe there is some configuration flags to set.
>>>
>>> How changing sched_util_min could impact cpufreq ondemand governor ?
>>> Does it change the value returned when the governor check the idle time ?
>>>
>> You'll have to use the schedutil governor for uclamp to have an effect. And
>> arguably that's what you should be using, unless something explicitly
>> prevents you from doing that.
> Even with schedutil and SCHED_CAPACITY_SCALE that it doesn't work.
> cpufreq/cpuinfo_cur_freq values are always on the max value even if the
> stats show transitions between the available frequencies.
>
> I see two possibles reasons to explain that:
> - sched_setattr() is called in userland process context, but the
> threaded irq handler is running in another process.

Ah yes, this only works if the task you boost is the one that will handle
whatever work you care about (in this case handling the irq). That said, if
you do use threaded IRQs, that should give you a SCHED_FIFO thread, which
should drive the frequency to its max when using schedutil (unrelated to
uclamp).

> - because this use case is almost running all in hardware the process
> isn't doing anything so the scheduler doesn't take care of it.
>
>>
>>>> Alternatively, that task may be a deadline one.

Re: [RFC 0/3] Introduce cpufreq minimum load QoS

[Benjamin GAIGNARD]

On 4/30/20 5:50 PM, Valentin Schneider wrote:
> On 30/04/20 16:37, Benjamin GAIGNARD wrote:
>> On 4/30/20 4:33 PM, Valentin Schneider wrote:
>>> On 30/04/20 14:46, Benjamin GAIGNARD wrote:
>>>>> That's not what I meant.
>>>>>
>>>>> I suppose that the interrupt processing in question takes place in
>>>>> process context and so you may set the lower clamp on the utilization
>>>>> of the task carrying that out.
>>>> I have try to add this code when starting streaming (before the first
>>>> interrupt) the frames from the sensor:
>>>> const struct sched_attr sched_attr = {
>>>>      .sched_util_min = 10000, /* 100% of usage */
>>> Unless you play with SCHED_CAPACITY_SHIFT, the max should be 1024 -
>>> i.e. SCHED_CAPACITY_SCALE. That's a really big boost, but that's for you to
>>> benchmark.
>>>
>>>>      .sched_flags = SCHED_FLAG_UTIL_CLAMP_MIN,
>>>>     };
>>>>
>>>> sched_setattr(current, &sched_attr);
>>>>
>>>> I don't see any benefices maybe there is some configuration flags to set.
>>>>
>>>> How changing sched_util_min could impact cpufreq ondemand governor ?
>>>> Does it change the value returned when the governor check the idle time ?
>>>>
>>> You'll have to use the schedutil governor for uclamp to have an effect. And
>>> arguably that's what you should be using, unless something explicitly
>>> prevents you from doing that.
>> Even with schedutil and SCHED_CAPACITY_SCALE that it doesn't work.
>> cpufreq/cpuinfo_cur_freq values are always on the max value even if the
>> stats show transitions between the available frequencies.
>>
>> I see two possibles reasons to explain that:
>> - sched_setattr() is called in userland process context, but the
>> threaded irq handler is running in another process.
> Ah yes, this only works if the task you boost is the one that will handle
> whatever work you care about (in this case handling the irq). That said, if
> you do use threaded IRQs, that should give you a SCHED_FIFO thread, which
> should drive the frequency to its max when using schedutil (unrelated to
> uclamp).
Can I conclude that sched_setattr() isn't the good way to solve this 
problem ?
Does my patches make sense in this case ?

>> - because this use case is almost running all in hardware the process
>> isn't doing anything so the scheduler doesn't take care of it.
>>
>>>>> Alternatively, that task may be a deadline one.

Re: [Linux-stm32] [RFC 0/3] Introduce cpufreq minimum load QoS

[Benjamin GAIGNARD]

On 5/4/20 11:17 AM, Benjamin GAIGNARD wrote:
>
> On 4/30/20 5:50 PM, Valentin Schneider wrote:
>> On 30/04/20 16:37, Benjamin GAIGNARD wrote:
>>> On 4/30/20 4:33 PM, Valentin Schneider wrote:
>>>> On 30/04/20 14:46, Benjamin GAIGNARD wrote:
>>>>>> That's not what I meant.
>>>>>>
>>>>>> I suppose that the interrupt processing in question takes place in
>>>>>> process context and so you may set the lower clamp on the utilization
>>>>>> of the task carrying that out.
>>>>> I have try to add this code when starting streaming (before the first
>>>>> interrupt) the frames from the sensor:
>>>>> const struct sched_attr sched_attr = {
>>>>>       .sched_util_min = 10000, /* 100% of usage */
>>>> Unless you play with SCHED_CAPACITY_SHIFT, the max should be 1024 -
>>>> i.e. SCHED_CAPACITY_SCALE. That's a really big boost, but that's for you to
>>>> benchmark.
>>>>
>>>>>       .sched_flags = SCHED_FLAG_UTIL_CLAMP_MIN,
>>>>>      };
>>>>>
>>>>> sched_setattr(current, &sched_attr);
>>>>>
>>>>> I don't see any benefices maybe there is some configuration flags to set.
>>>>>
>>>>> How changing sched_util_min could impact cpufreq ondemand governor ?
>>>>> Does it change the value returned when the governor check the idle time ?
>>>>>
>>>> You'll have to use the schedutil governor for uclamp to have an effect. And
>>>> arguably that's what you should be using, unless something explicitly
>>>> prevents you from doing that.
>>> Even with schedutil and SCHED_CAPACITY_SCALE that it doesn't work.
>>> cpufreq/cpuinfo_cur_freq values are always on the max value even if the
>>> stats show transitions between the available frequencies.
>>>
>>> I see two possibles reasons to explain that:
>>> - sched_setattr() is called in userland process context, but the
>>> threaded irq handler is running in another process.
>> Ah yes, this only works if the task you boost is the one that will handle
>> whatever work you care about (in this case handling the irq). That said, if
>> you do use threaded IRQs, that should give you a SCHED_FIFO thread, which
>> should drive the frequency to its max when using schedutil (unrelated to
>> uclamp).
> Can I conclude that sched_setattr() isn't the good way to solve this
> problem ?
> Does my patches make sense in this case ?
Gentle up on this series beacause I haven't found any other way to solve
this problem.

Thanks,
Benjamin
>
>>> - because this use case is almost running all in hardware the process
>>> isn't doing anything so the scheduler doesn't take care of it.
>>>
>>>>>> Alternatively, that task may be a deadline one.
> _______________________________________________
> Linux-stm32 mailing list
> Linux-stm32@st-md-mailman.stormreply.com
> https://st-md-mailman.stormreply.com/mailman/listinfo/linux-stm32