Re: Instrumentation and RCU

From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
To: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>,
	rostedt <rostedt@goodmis.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Alexei Starovoitov <ast@kernel.org>, paulmck <paulmck@kernel.org>,
	"Joel Fernandes, Google" <joel@joelfernandes.org>,
	Frederic Weisbecker <frederic@kernel.org>,
	Jason Wessel <jason.wessel@windriver.com>
Subject: Re: Instrumentation and RCU
Date: Tue, 10 Mar 2020 11:24:24 -0400 (EDT)	[thread overview]
Message-ID: <1659630114.23432.1583853864272.JavaMail.zimbra@efficios.com> (raw)
In-Reply-To: <20200310170951.87c29e9c1cfbddd93ccd92b3@kernel.org>

----- On Mar 10, 2020, at 4:09 AM, Masami Hiramatsu mhiramat@kernel.org wrote:

> Hi,
> 
> On Mon, 09 Mar 2020 19:59:18 +0100
> Thomas Gleixner <tglx@linutronix.de> wrote:
> 

[...]

>> which is also in section "text" then the analysis tool will find the
>> missing offlimit_safecall() - or what ever method we chose to annotate
>> that stuff. Surely not an annotation on the called function itself
>> because that might be safe to call in one context but not in another.
> 
> Hmm, what the offlimit_safecall() does? and what happen if the
> do_fragile_stuff_on_enter() invokes a library code? I think we also need
> to tweak kbuild to duplicate some library code to the off-limit text area.
> 
>> These annotations are halfways easy to monitor for abuse and they should
>> be prominent enough in the code that at least for the people dealing
>> with that kind of code they act as a warning flag.
> 
> This off-limit text will be good for entries, but I think we still not
> able to remove all NOKPROBE_SYMBOLS with this.
> 
> For example __die() is marked a NOKPROBE because if we hit a recursive
> int3, it calls BUG() to dump stacks etc for debug. So that function
> must NOT probed. (I think we also should mark all backtrace functions
> in this case, but not yet) Would we move those backtrace related
> functions (including printk, and console drivers?) into the offlimit
> text too?
> 
> Hmm, if there is a bust_kprobes(), that can be easy to fix this issue.

In order to solve the recursion issue without losing instrumentation
coverage of traps and significant system events, I really think we should
look into adding some kind of "in_tracing" flag near each stack. We could
name this "struct recursion_context" or something similar. We could then
add this structure alongside each thread and ISR stack, but make sure traps
(including breakpoints) still point to the recursion context of whatever
caused the trap.

This should allow us to detect and prevent recursion from tracers directly
at the instrumentation level.

Thoughts ?

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com