All of lore.kernel.org
 help / color / mirror / Atom feed
From: Masami Hiramatsu <mhiramat@redhat.com>
To: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>, LKML <linux-kernel@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Arnaldo Carvalho de Melo <acme@redhat.com>,
	Mike Galbraith <efault@gmx.de>, Paul Mackerras <paulus@samba.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Li Zefan <lizf@cn.fujitsu.com>, Jason Baron <jbaron@redhat.com>
Subject: Re: [PATCH] tracing: Protect the buffer from recursion in perf
Date: Wed, 04 Nov 2009 22:06:44 -0500	[thread overview]
Message-ID: <4AF24144.30003@redhat.com> (raw)
In-Reply-To: <1257373437-5332-1-git-send-email-fweisbec@gmail.com>

Frederic Weisbecker wrote:
> While tracing using events with perf, if one enables the
> lockdep:lock_acquire event, it will infect every other perf trace
> events.
> 
> Basically, you can enable whatever set of trace events through perf
> but if this event is part of the set, the only result we can get is a
> long list of lock_acquire events of rcu read lock, and only that.
> 
> This is because of a recursion inside perf.
> 
> 1) When a trace event is triggered, it will fill a per cpu buffer and
>     submit it to perf.
> 2) Perf will commit this event but will also protect some data using
>     rcu_read_lock
> 3) A recursion appears: rcu_read_lock triggers a lock_acquire event that
>     will fill the per cpu event and then submit the buffer to perf.
> 4) Perf detects a recursion and ignore it
> 5) Perf continue its work on the previous event, but its buffer has been
>     overwritten by the lock_acquire event, it has then been turned into
>     a lock_acquire event of rcu read lock
> 
> Such scenario also happens with lock_release with rcu_read_unlock().
> 
> We could turn the rcu_read_lock() into __rcu_read_lock() to drop the
> lock debugging from perf fast path, but that would make us lose
> the rcu debugging and that doesn't prevent from other possible kind of
> recursion from perf in the future.
> 
> This patch adds a recursion protection on the perf trace per cpu
> buffers to solve the problem.

OK, so you added a counter to find recursion.
I found a style issue, see below.

> diff --git a/include/trace/ftrace.h b/include/trace/ftrace.h
> index a7f9460..d80928d 100644
> --- a/include/trace/ftrace.h
> +++ b/include/trace/ftrace.h
> @@ -649,6 +649,7 @@ __attribute__((section("_ftrace_events"))) event_##call = {		\
>    *	struct ftrace_event_call *event_call =&event_<call>;
>    *	extern void perf_tp_event(int, u64, u64, void *, int);
>    *	struct ftrace_raw_##call *entry;
> +* 	struct perf_trace_buf *trace_buf;
  ^^ Here, we need a space :-)

Others looks good to me, Thanks!

Reviewed-by: Masami Hiramatsu <mhiramat@redhat.com>


-- 
Masami Hiramatsu

Software Engineer
Hitachi Computer Products (America), Inc.
Software Solutions Division

e-mail: mhiramat@redhat.com


  reply	other threads:[~2009-11-05  3:07 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-11-04 22:23 [PATCH] tracing: Protect the buffer from recursion in perf Frederic Weisbecker
2009-11-05  3:06 ` Masami Hiramatsu [this message]
2009-11-06  3:13   ` [PATCH v2] " Frederic Weisbecker
2009-11-08  9:40     ` [tip:perf/probes] tracing, perf_events: " tip-bot for Frederic Weisbecker
2009-11-10 10:27     ` [PATCH v2] tracing: " Peter Zijlstra
2009-11-10 10:40       ` Frederic Weisbecker

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4AF24144.30003@redhat.com \
    --to=mhiramat@redhat.com \
    --cc=acme@redhat.com \
    --cc=efault@gmx.de \
    --cc=fweisbec@gmail.com \
    --cc=jbaron@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lizf@cn.fujitsu.com \
    --cc=mingo@elte.hu \
    --cc=paulus@samba.org \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.