Re: [patch V4 4/8] sched: Make migrate_disable/enable() independent of RT

From: Mel Gorman <mgorman@suse.de>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: LKML <linux-kernel@vger.kernel.org>,
	x86@kernel.org, Linus Torvalds <torvalds@linuxfoundation.org>,
	Christoph Hellwig <hch@lst.de>,
	Matthew Wilcox <willy@infradead.org>,
	Daniel Vetter <daniel@ffwll.ch>,
	Andrew Morton <akpm@linux-foundation.org>,
	linux-mm@kvack.org, Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@kernel.org>,
	Juri Lelli <juri.lelli@redhat.com>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Ben Segall <bsegall@google.com>,
	Daniel Bristot de Oliveira <bristot@redhat.com>,
	Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: [patch V4 4/8] sched: Make migrate_disable/enable() independent of RT
Date: Thu, 19 Nov 2020 09:38:34 +0000	[thread overview]
Message-ID: <20201119093834.GH3306@suse.de> (raw)
In-Reply-To: <20201118204007.269943012@linutronix.de>

[-- Attachment #1: Type: text/plain, Size: 2467 bytes --]

On Wed, Nov 18, 2020 at 08:48:42PM +0100, Thomas Gleixner wrote:
> From: Thomas Gleixner <tglx@linutronix.de>
> 
> Now that the scheduler can deal with migrate disable properly, there is no
> real compelling reason to make it only available for RT.
> 
> There are quite some code pathes which needlessly disable preemption in
> order to prevent migration and some constructs like kmap_atomic() enforce
> it implicitly.
> 
> Making it available independent of RT allows to provide a preemptible
> variant of kmap_atomic() and makes the code more consistent in general.
> 
> FIXME: Rework the comment in preempt.h - Peter?
> 

I didn't keep up to date and there is clearly a dependency on patches in
tip for migrate_enable/migrate_disable . It's not 100% clear to me what
reworking you're asking for but then again, I'm not Peter!

From tip;

/**
 * migrate_disable - Prevent migration of the current task
 *
 * Maps to preempt_disable() which also disables preemption. Use
 * migrate_disable() to annotate that the intent is to prevent migration,
 * but not necessarily preemption.
 *
 * Can be invoked nested like preempt_disable() and needs the corresponding
 * number of migrate_enable() invocations.
 */

I assume that the rework is to document the distinction between
migrate_disable and preempt_disable() because it may not be clear to some
people why one should be used over another and the risk of cut&paste
cargo cult programming.

So I assume the rework is for the middle paragraph

 * Maps to preempt_disable() which also disables preemption. Use
 * migrate_disable() to annotate that the intent is to prevent migration,
 * but not necessarily preemption. The distinction is that preemption
 * disabling will protect a per-cpu structure from concurrent
 * modifications due to preemption. migrate_disable partially protects
 * the tasks address space and potentially preserves the TLB entries
 * even if preempted such as an needed for a local IO mapping or a
 * kmap_atomic() referenced by on-stack pointers to avoid interference
 * between user threads or kernel threads sharing the same address space.

I know it can have other examples that are rt-specific and some tricks on
percpu page alloc draining that relies on a combination of migrate_disable
and interrupt disabling to protect the structures but the above example
might be understandable to a non-RT audience.

-- 
Mel Gorman
SUSE Labs

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]