Re: [PATCH v3 0/3] Introduce per-task latency_nice for scheduler hints

From: chris hyser <chris.hyser@oracle.com>
To: David Laight <David.Laight@ACULAB.COM>,
	Parth Shah <parth@linux.ibm.com>,
	"vincent.guittot@linaro.org" <vincent.guittot@linaro.org>,
	"patrick.bellasi@matbug.net" <patrick.bellasi@matbug.net>,
	"valentin.schneider@arm.com" <valentin.schneider@arm.com>,
	"dhaval.giani@oracle.com" <dhaval.giani@oracle.com>,
	"dietmar.eggemann@arm.com" <dietmar.eggemann@arm.com>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"peterz@infradead.org" <peterz@infradead.org>,
	"mingo@redhat.com" <mingo@redhat.com>,
	"qais.yousef@arm.com" <qais.yousef@arm.com>,
	"pavel@ucw.cz" <pavel@ucw.cz>,
	"qperret@qperret.net" <qperret@qperret.net>,
	"pjt@google.com" <pjt@google.com>,
	"tj@kernel.org" <tj@kernel.org>
Subject: Re: [PATCH v3 0/3] Introduce per-task latency_nice for scheduler hints
Date: Thu, 20 Feb 2020 10:55:58 -0500	[thread overview]
Message-ID: <7b50c0ee-5e2b-5cec-a7f6-514abdc5ee18@oracle.com> (raw)
In-Reply-To: <2870e44f41414fd58b58f7831d7386fe@AcuMS.aculab.com>

On 2/20/20 9:39 AM, David Laight wrote:
> From: chris hyser <chris.hyser@oracle.com>
>> Sent: 19 February 2020 17:17
>>
>> On 2/19/20 6:18 AM, David Laight wrote:
>>> From: chris hyser
>>>> Sent: 18 February 2020 23:00
>>> ...
>>>> All, I was asked to take a look at the original latency_nice patchset.
>>>> First, to clarify objectives, Oracle is not
>>>> interested in trading throughput for latency.
>>>> What we found is that the DB has specific tasks which do very little but
>>>> need to do this as absolutely quickly as possible, ie extreme latency
>>>> sensitivity. Second, the key to latency reduction
>>>> in the task wakeup path seems to be limiting variations of "idle cpu" search.
>>>> The latter particularly interests me as an example of "platform size
>>>> based latency" which I believe to be important given all the varying size
>>>> VMs and containers.
>>>
>>>   From my experiments there are a few things that seem to affect latency
>>> of waking up real time (sched fifo) tasks on a normal kernel:
>>
>> Sorry. I was only ever talking about sched_other as per the original patchset. I realize the term
>> extreme latency
>> sensitivity may have caused confusion. What that means to DB people is no doubt different than audio
>> people. :-)
> 
> Shorter lines.....
> 
> ISTM you are making some already complicated code even more complex.
> Better to make it simpler instead.

The code already exists to set a limit to bail out of what is sometimes a needlessly excessive search. Setting that 
based on an integer doesn't seem particularly complex. Now whether that is actually useful is what I'm currently looking at.

> 
> If you need a thread to run as soon as possible after it is woken
> why not use the RT scheduler (eg SCHED_FIFO) that is what it is for.

Overkill and doesn't play well with cpu cgroup controller.

> 
> If there are delays finding an idle cpu to migrate a process to
> (especially on systems with large numbers of cpu) then that is a
> general problem that can be addressed without extra knobs.

There is no if. It is a brute force search. There are delays proportional to the search domain size. You can optimize 
the hell of out the brute force, or you use obtained knowledge to bail out early. Getting that knowledge from the user 
is a time honored tradition. :-)

-chrish