Re: [patch V4 4/8] sched: Make migrate_disable/enable() independent of RT

From: Thomas Gleixner <tglx@linutronix.de>
To: Peter Zijlstra <peterz@infradead.org>,
	Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mel Gorman <mgorman@suse.de>, LKML <linux-kernel@vger.kernel.org>,
	the arch/x86 maintainers <x86@kernel.org>,
	Christoph Hellwig <hch@lst.de>,
	Matthew Wilcox <willy@infradead.org>,
	Daniel Vetter <daniel@ffwll.ch>,
	Andrew Morton <akpm@linux-foundation.org>,
	Linux-MM <linux-mm@kvack.org>, Ingo Molnar <mingo@kernel.org>,
	Juri Lelli <juri.lelli@redhat.com>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Ben Segall <bsegall@google.com>,
	Daniel Bristot de Oliveira <bristot@redhat.com>
Subject: Re: [patch V4 4/8] sched: Make migrate_disable/enable() independent of RT
Date: Fri, 20 Nov 2020 02:33:58 +0100	[thread overview]
Message-ID: <87tutkolq1.fsf@nanos.tec.linutronix.de> (raw)
In-Reply-To: <20201119182843.GA2414@hirez.programming.kicks-ass.net>

On Thu, Nov 19 2020 at 19:28, Peter Zijlstra wrote:
> On Thu, Nov 19, 2020 at 09:23:47AM -0800, Linus Torvalds wrote:
>> Because this is certainly not the only time migration limiting has
>> come up, and no, it has absolutely nothing to do with per-cpu page
>> tables being completely unacceptable.
>
> It is for this instance; but sure, it's come up before in other
> contexts.

Indeed. And one of the really bad outcomes of this is that people are
forced to use preempt_disable() to prevent migration which entails a
slew of consequences:

     - Using spinlocks where it wouldn't be needed otherwise
     - Spinwaiting instead of sleeping
     - The whole crazyness of doing copy_to/from_user_in_atomic() along
       with the necessary out of line error handling.
     - ....

The introduction of per-cpu storage happened almost 20 years ago (2002)
and still the only answer we have is preempt_disable().

I know the scheduling theory folks still try to wrap their heads around
the introduction of SMP which dates back to 1962 IIRC...

>> The scheduler people need to get used to this. Really. Because ASMP is
>> just going to be a fact.
>
> ASMP is different in that it is a hardware constraint, you're just not
> going to be able to run more of X than there's X capable hardware units
> on (be if FPUs, Vector units, 32bit or whatever)

ASMP is as old as SMP. The first SMP systems were in fact ASMP.  The
reasons for ASMP 60 years ago were not that different from the reasons
for ASMP today. Just the scale and the effectivness are different.

>> There are few things more futile than railing against reality, Peter.
>
> But, but, my windmills! :-)

At least you have windmills where you live so you can pull off the real
Don Quixote while other people have to find substitutes :)

Thanks,

        tglx