Re: Use case for TASKS_RCU

From: Steven Rostedt <rostedt@goodmis.org>
To: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: mingo@kernel.org, linux-kernel@vger.kernel.org
Subject: Re: Use case for TASKS_RCU
Date: Mon, 15 May 2017 14:48:10 -0400	[thread overview]
Message-ID: <20170515144810.563a4d9b@gandalf.local.home> (raw)
In-Reply-To: <20170515182354.GA25440@linux.vnet.ibm.com>

On Mon, 15 May 2017 11:23:54 -0700
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:

> Hello!
> 
> The question of the use case for TASKS_RCU came up, and here is my
> understanding.  Steve will not be shy about correcting any misconceptions
> I might have.  ;-)
> 
> The use case is to support freeing of trampolines used in tracing/probing
> in CONFIG_PREEMPT=y kernels.  It is necessary to wait until any task
> executing in the trampoline in question has left it, taking into account
> that the trampoline's code might be interrupted and preempted.  However,
> the code in the trampolines is guaranteed never to context switch.

nit, "never to voluntarily context switch" as it can still be
preempted. It should never call schedule nor a mutex. And really it
shouldn't even call any spinlocks. Although, trace_stack does, but it
does so after checking if in_nmi(), which it bails if that is true.

> 
> Note that in CONFIG_PREEMPT=n kernels, synchronize_sched() suffices.
> It is therefore tempting to think in terms of disabling preemption across
> the trampolines, but there is apparently not enough room to accommodate
> the needed preempt_disable() and preempt_enable() in the code invoking
> the trampoline, and putting the preempt_disable() and preempt_enable()
> in the trampoline itself fails because of the possibility of preemption
> just before the preempt_disable() and just after the preempt_enable().
> Similar reasoning rules out use of rcu_read_lock() and rcu_read_unlock().

Correct, as the jump to the trampoline may be preempted. And preemption
happens just before the first instruction on the trampoline is being
executed.

> 
> Another possibility would be to place the trampolines in a known region
> of memory, and check for the task's PC being in that region.  This fails
> because trampolines can be interrupted, and I vaguely recall something
> about them calling function as well.  Stack tracing could be added,
> but stack tracing is not as reliable as it would need to be.

Correct.

> 
> The solution chosen relies on the fact that code in trampolines
> (and code invoked from trampolines) is not permitted to do voluntary
> context switches.  Thus, if a trampoline is removed, and a given task
> later does a voluntary context switch (or has been seen in usermode),
> that task will never again reference that trampoline.  Once all tasks
> are accounted for, the trampoline may safely be removed.

Correct.

> 
> TASKS_RCU implements a flavor of RCU that does exactly this.  It has
> only a single use at the moment, but avoiding memory leaks on
> production machines being instrumented seems to me to be quite valuable.

Optimized kprobes can also benefit from this, as it currently is
disabled on CONFIG_PREEMPT due to exactly the same issue. I'll poke
Masami about this again. I should be seeing him in a couple of weeks at
the Open Source Summit in Tokyo.

> 
> So, Steve, please correct any misconceptions!

Nope, all looks good.

-- Steve