Re: [RFC PATCH 0/3] sched: delayed thread migration

From: Redha <redha.gouicem@gmail.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: julien.sopena@lip6.fr, julia.lawall@inria.fr,
	gilles.muller@inria.fr, carverdamien@gmail.com,
	jean-pierre.lozi@oracle.com, baptiste.lepers@sydney.edu.au,
	nicolas.palix@univ-grenoble-alpes.fr,
	willy.zwaenepoel@sydney.edu.au,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
	"Rafael J. Wysocki" <rjw@rjwysocki.net>,
	Viresh Kumar <viresh.kumar@linaro.org>,
	Juri Lelli <juri.lelli@redhat.com>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
	Daniel Bristot de Oliveira <bristot@redhat.com>,
	Luis Chamberlain <mcgrof@kernel.org>,
	Kees Cook <keescook@chromium.org>,
	Iurii Zaikin <yzaikin@google.com>,
	Qais Yousef <qais.yousef@arm.com>,
	Al Viro <viro@zeniv.linux.org.uk>, Andrey Ignatov <rdna@fb.com>,
	"Guilherme G. Piccoli" <gpiccoli@canonical.com>,
	linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org,
	linux-fsdevel@vger.kernel.org
Subject: Re: [RFC PATCH 0/3] sched: delayed thread migration
Date: Wed, 21 Oct 2020 12:40:44 +0200	[thread overview]
Message-ID: <ad9b8a29-7f14-d8bf-0c6d-5aeb8c6c7912@gmail.com> (raw)
In-Reply-To: <20201021072612.GV2628@hirez.programming.kicks-ass.net>

On 21/10/2020 09:26, Peter Zijlstra wrote:
> On Tue, Oct 20, 2020 at 05:44:38PM +0200, Redha Gouicem wrote:
>> The first patch of the series is not specific to scheduling. It allows us
>> (or anyone else) to use the cpufreq infrastructure at a different sampling
>> rate without compromising the cpufreq subsystem and applications that
>> depend on it.
> It's also completely redudant as the scheduler already reads aperf/mperf
> on every tick. Clearly you didn't do your homework ;-)
My bad. I worked on this a year ago, just never got time to submit to the
lkml and I should have re-done my homework more thoroughly before
submitting. The paper was submitted approximately at the same time as the
patch introducing support frequency invariance and frequency reading at
every tick (1 week apart!)
Again, my bad.

>
>> The main idea behind this patch series is to bring to light the frequency
>> inversion problem that will become more and more prominent with new CPUs
>> that feature per-core DVFS. The solution proposed is a first idea for
>> solving this problem that still needs to be tested across more CPUs and
>> with more applications.
> Which is why schedutil (the only cpufreq gov anybody should be using) is
> integrated with the scheduler and closes the loop and tells the CPU
> about the expected load.
>
While I agree that schedutil is probably a good option, I'm not sure we
treat exactly the same problem. schedutil aims at mapping the frequency of
the CPU to the actual load. What I'm saying is that since it takes some
time for the frequency to match the load, why not account for the frequency
when making placement/migration decisions. I know that with the frequency
invariance code, capacity accounts for frequency, which means that thread
placement decisions do account for frequency indirectly. However, we still
have performance improvements with our patch for the workloads with
fork/wait patterns. I really believe that we can still gain performance if
we make decisions while accounting for the frequency more directly.