LKML Archive on lore.kernel.org
 help / color / Atom feed
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Subject: Re: [BUG] stack tracing causes: kernel/module.c:271 module_assert_mutex_or_preempt
Date: Thu, 6 Apr 2017 07:59:43 -0700
Message-ID: <20170406145943.GA1600@linux.vnet.ibm.com> (raw)
In-Reply-To: <20170406101425.61f661b7@gandalf.local.home>

On Thu, Apr 06, 2017 at 10:14:25AM -0400, Steven Rostedt wrote:
> On Wed, 5 Apr 2017 21:15:15 -0700
> "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:
> \> > diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> > > index 8efd9fe..28e3019 100644
> > > --- a/kernel/trace/ftrace.c
> > > +++ b/kernel/trace/ftrace.c
> > > @@ -2808,18 +2808,28 @@ static int ftrace_shutdown(struct ftrace_ops *ops, int command)
> > >  	 * callers are done before leaving this function.
> > >  	 * The same goes for freeing the per_cpu data of the per_cpu
> > >  	 * ops.
> > > -	 *
> > > -	 * Again, normal synchronize_sched() is not good enough.
> > > -	 * We need to do a hard force of sched synchronization.
> > > -	 * This is because we use preempt_disable() to do RCU, but
> > > -	 * the function tracers can be called where RCU is not watching
> > > -	 * (like before user_exit()). We can not rely on the RCU
> > > -	 * infrastructure to do the synchronization, thus we must do it
> > > -	 * ourselves.
> > >  	 */
> > >  	if (ops->flags & (FTRACE_OPS_FL_DYNAMIC | FTRACE_OPS_FL_PER_CPU)) {
> > > +		/*
> > > +		 * We need to do a hard force of sched synchronization.
> > > +		 * This is because we use preempt_disable() to do RCU, but
> > > +		 * the function tracers can be called where RCU is not watching
> > > +		 * (like before user_exit()). We can not rely on the RCU
> > > +		 * infrastructure to do the synchronization, thus we must do it
> > > +		 * ourselves.
> > > +		 */
> > >  		schedule_on_each_cpu(ftrace_sync);  
> > 
> > Great header comment on ftrace_sync(): "Yes, function tracing is rude."
> > And schedule_on_each_cpu() looks like a great workqueue gatling gun!  ;-)
> > 
> > > +#ifdef CONFIG_PREEMPT
> > > +		/*
> > > +		 * When the kernel is preeptive, tasks can be preempted
> > > +		 * while on a ftrace trampoline. Just scheduling a task on
> > > +		 * a CPU is not good enough to flush them. Calling
> > > +		 * synchronize_rcu_tasks() will wait for those tasks to
> > > +		 * execute and either schedule voluntarily or enter user space.
> > > +		 */
> > > +		synchronize_rcu_tasks();
> > > +#endif  
> > 
> > How about this to save a line?
> > 
> > 		if (IS_ENABLED(CONFIG_PREEMPT))
> > 			synchronize_rcu_tasks();
> 
> Ah, this works as gcc optimizes it out. Otherwise I received a compile
> error with synchronize_rcu_tasks() not defined. But that's because I
> never enabled CONFIG_TASKS_RCU.

Should I define a synchronize_rcu_tasks() that does BUG() for this case?

> > One thing that might speed this up a bit (or might not) would be to
> > doe the schedule_on_each_cpu() from a delayed workqueue.  That way,
> > if any of the activity from schedule_on_each_cpu() involved a voluntary
> > context switch (from a cond_resched() or some such), then
> > synchronize_rcu_tasks() would get the benefit of that context switch.
> > 
> > You would need a flush_work() to wait for that delayed workqueue
> > as well, of course.
> 
> This is a very slow path, I'm not too interested in making it complex
> to speed it up.

Makes sense to me!

> > Not sure whether it is worth it, but figured I should pass it along.
> > 
> > >  		arch_ftrace_trampoline_free(ops);
> > > 
> > >  		if (ops->flags & FTRACE_OPS_FL_PER_CPU)
> > > @@ -5366,22 +5376,6 @@ void __weak arch_ftrace_update_trampoline(struct ftrace_ops *ops)
> > > 
> > >  static void ftrace_update_trampoline(struct ftrace_ops *ops)
> > >  {
> > > -
> > > -/*
> > > - * Currently there's no safe way to free a trampoline when the kernel
> > > - * is configured with PREEMPT. That is because a task could be preempted
> > > - * when it jumped to the trampoline, it may be preempted for a long time
> > > - * depending on the system load, and currently there's no way to know
> > > - * when it will be off the trampoline. If the trampoline is freed
> > > - * too early, when the task runs again, it will be executing on freed
> > > - * memory and crash.
> > > - */
> > > -#ifdef CONFIG_PREEMPT
> > > -	/* Currently, only non dynamic ops can have a trampoline */
> > > -	if (ops->flags & FTRACE_OPS_FL_DYNAMIC)
> > > -		return;
> > > -#endif
> > > -
> > >  	arch_ftrace_update_trampoline(ops);
> > >  }  
> > 
> > Agreed, straightforward patch!
> 
> Great, I'll start making it official then.

Sounds very good!

							Thanx, Paul

      reply index

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-04-05 13:32 Steven Rostedt
2017-04-05 16:25 ` Paul E. McKenney
2017-04-05 16:45   ` Steven Rostedt
2017-04-05 17:59     ` Paul E. McKenney
2017-04-05 18:54       ` Steven Rostedt
2017-04-05 19:08         ` Paul E. McKenney
2017-04-05 19:21           ` Steven Rostedt
2017-04-05 19:39             ` Paul E. McKenney
2017-04-05 19:52               ` Steven Rostedt
2017-04-05 20:42                 ` Paul E. McKenney
2017-04-06  1:31                   ` Steven Rostedt
2017-04-06  4:07                     ` Paul E. McKenney
2017-04-06  2:12       ` Steven Rostedt
2017-04-06  4:15         ` Paul E. McKenney
2017-04-06 14:14           ` Steven Rostedt
2017-04-06 14:59             ` Paul E. McKenney [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170406145943.GA1600@linux.vnet.ibm.com \
    --to=paulmck@linux.vnet.ibm.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=rostedt@goodmis.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git