All of lore.kernel.org
 help / color / mirror / Atom feed
* suspicious RCU usage (perf)
@ 2013-08-26 14:58 Dave Jones
  2013-08-26 16:29 ` Paul E. McKenney
  0 siblings, 1 reply; 17+ messages in thread
From: Dave Jones @ 2013-08-26 14:58 UTC (permalink / raw)
  To: Linux Kernel; +Cc: Paul McKenney

Another day, another rcu backtrace..
This says rc6, but it's pretty darn close to rc7, I think it was running
a build from Friday.

[260431.854524] ===============================
[260431.855485] [ INFO: suspicious RCU usage. ]
[260431.856437] 3.11.0-rc6+ #12 Not tainted
[260431.857377] -------------------------------
[260431.858318] include/linux/rcupdate.h:779 rcu_read_lock() used illegally while idle!
[260431.859278] other info that might help us debug this:
[260431.862144] RCU used illegally from idle CPU!
rcu_scheduler_active = 1, debug_locks = 0
[260431.864990] RCU used illegally from extended quiescent state!
[260431.865949] 1 lock held by trinity-kid3/15917:
[260431.866901]  #0:  (rcu_read_lock){.+.+..}, at: [<ffffffff81146d50>] __perf_event_overflow+0x100/0x320
[260431.867896] stack backtrace:
[260431.869815] CPU: 0 PID: 15917 Comm: trinity-kid3 Not tainted 3.11.0-rc6+ #12
[260431.871885]  0000000000000000 ffff8801095d3ad0 ffffffff816f9c6f ffff88023ce86950
[260431.872912]  ffff8801095d3b00 ffffffff810be6a7 ffff880079695cd0 0000000000000000
[260431.873910]  ffff8801095d3c00 0000000000000000 ffff8801095d3b80 ffffffff81146ef4
[260431.874909] Call Trace:
[260431.875883]  [<ffffffff816f9c6f>] dump_stack+0x54/0x74
[260431.876869]  [<ffffffff810be6a7>] lockdep_rcu_suspicious+0xe7/0x120
[260431.877852]  [<ffffffff81146ef4>] __perf_event_overflow+0x2a4/0x320
[260431.878834]  [<ffffffff81146d50>] ? __perf_event_overflow+0x100/0x320
[260431.879811]  [<ffffffff81146e0c>] ? __perf_event_overflow+0x1bc/0x320
[260431.880784]  [<ffffffff8170ce00>] ? ftrace_call+0x5/0x2f
[260431.881758]  [<ffffffff81147101>] perf_swevent_overflow+0x51/0xe0
[260431.882728]  [<ffffffff811471ef>] perf_swevent_event+0x5f/0x90
[260431.883685]  [<ffffffff81147329>] perf_tp_event+0x109/0x4f0
[260431.884637]  [<ffffffff81147542>] ? perf_tp_event+0x322/0x4f0
[260431.885586]  [<ffffffff811079fb>] ? rcu_eqs_enter_common+0x5b/0x420
[260431.886539]  [<ffffffff8107a320>] ? task_work_run+0xe0/0xe0
[260431.887486]  [<ffffffff81135910>] perf_ftrace_function_call+0xc0/0xd0
[260431.888417]  [<ffffffff81114d37>] ? ftrace_ops_control_func+0xe7/0x110
[260431.889353]  [<ffffffff8107a320>] ? task_work_run+0xe0/0xe0
[260431.890281]  [<ffffffff811079fb>] ? rcu_eqs_enter_common+0x5b/0x420
[260431.891191]  [<ffffffff81114d37>] ftrace_ops_control_func+0xe7/0x110
[260431.892088]  [<ffffffff8170ce00>] ftrace_call+0x5/0x2f
[260431.892975]  [<ffffffff81114cbb>] ? ftrace_ops_control_func+0x6b/0x110
[260431.893862]  [<ffffffff8170ce00>] ? ftrace_call+0x5/0x2f
[260431.894745]  [<ffffffff81097def>] ? local_clock+0x3f/0x50
[260431.895631]  [<ffffffff8107a325>] ? debug_lockdep_rcu_enabled+0x5/0x40
[260431.896516]  [<ffffffff8107a325>] ? debug_lockdep_rcu_enabled+0x5/0x40
[260431.897387]  [<ffffffff811079fb>] ? rcu_eqs_enter_common+0x5b/0x420
[260431.898264]  [<ffffffff81107e24>] rcu_eqs_enter+0x64/0xa0
[260431.899133]  [<ffffffff8110c673>] rcu_user_enter+0x13/0x20
[260431.899999]  [<ffffffff8114e67a>] user_enter+0x6a/0xd0
[260431.900859]  [<ffffffff81010208>] syscall_trace_leave+0x78/0x150
[260431.901712]  [<ffffffff8170d4af>] int_check_syscall_exit_work+0x34/0x3d

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-26 14:58 suspicious RCU usage (perf) Dave Jones
@ 2013-08-26 16:29 ` Paul E. McKenney
  2013-08-26 17:30   ` Steven Rostedt
  0 siblings, 1 reply; 17+ messages in thread
From: Paul E. McKenney @ 2013-08-26 16:29 UTC (permalink / raw)
  To: Dave Jones, Linux Kernel; +Cc: rostedt

On Mon, Aug 26, 2013 at 10:58:38AM -0400, Dave Jones wrote:
> Another day, another rcu backtrace..
> This says rc6, but it's pretty darn close to rc7, I think it was running
> a build from Friday.

Could you please send your .config?  Also, were you running ftrace,
perf, RCU event tracing, or what?

Looks like you are running ftrace, but I though Steven had set that
up so that it could be called from an extended quiescent state.

							Thanx, Paul

> [260431.854524] ===============================
> [260431.855485] [ INFO: suspicious RCU usage. ]
> [260431.856437] 3.11.0-rc6+ #12 Not tainted
> [260431.857377] -------------------------------
> [260431.858318] include/linux/rcupdate.h:779 rcu_read_lock() used illegally while idle!
> [260431.859278] other info that might help us debug this:
> [260431.862144] RCU used illegally from idle CPU!
> rcu_scheduler_active = 1, debug_locks = 0
> [260431.864990] RCU used illegally from extended quiescent state!
> [260431.865949] 1 lock held by trinity-kid3/15917:
> [260431.866901]  #0:  (rcu_read_lock){.+.+..}, at: [<ffffffff81146d50>] __perf_event_overflow+0x100/0x320
> [260431.867896] stack backtrace:
> [260431.869815] CPU: 0 PID: 15917 Comm: trinity-kid3 Not tainted 3.11.0-rc6+ #12
> [260431.871885]  0000000000000000 ffff8801095d3ad0 ffffffff816f9c6f ffff88023ce86950
> [260431.872912]  ffff8801095d3b00 ffffffff810be6a7 ffff880079695cd0 0000000000000000
> [260431.873910]  ffff8801095d3c00 0000000000000000 ffff8801095d3b80 ffffffff81146ef4
> [260431.874909] Call Trace:
> [260431.875883]  [<ffffffff816f9c6f>] dump_stack+0x54/0x74
> [260431.876869]  [<ffffffff810be6a7>] lockdep_rcu_suspicious+0xe7/0x120
> [260431.877852]  [<ffffffff81146ef4>] __perf_event_overflow+0x2a4/0x320
> [260431.878834]  [<ffffffff81146d50>] ? __perf_event_overflow+0x100/0x320
> [260431.879811]  [<ffffffff81146e0c>] ? __perf_event_overflow+0x1bc/0x320
> [260431.880784]  [<ffffffff8170ce00>] ? ftrace_call+0x5/0x2f
> [260431.881758]  [<ffffffff81147101>] perf_swevent_overflow+0x51/0xe0
> [260431.882728]  [<ffffffff811471ef>] perf_swevent_event+0x5f/0x90
> [260431.883685]  [<ffffffff81147329>] perf_tp_event+0x109/0x4f0
> [260431.884637]  [<ffffffff81147542>] ? perf_tp_event+0x322/0x4f0
> [260431.885586]  [<ffffffff811079fb>] ? rcu_eqs_enter_common+0x5b/0x420
> [260431.886539]  [<ffffffff8107a320>] ? task_work_run+0xe0/0xe0
> [260431.887486]  [<ffffffff81135910>] perf_ftrace_function_call+0xc0/0xd0
> [260431.888417]  [<ffffffff81114d37>] ? ftrace_ops_control_func+0xe7/0x110
> [260431.889353]  [<ffffffff8107a320>] ? task_work_run+0xe0/0xe0
> [260431.890281]  [<ffffffff811079fb>] ? rcu_eqs_enter_common+0x5b/0x420
> [260431.891191]  [<ffffffff81114d37>] ftrace_ops_control_func+0xe7/0x110
> [260431.892088]  [<ffffffff8170ce00>] ftrace_call+0x5/0x2f
> [260431.892975]  [<ffffffff81114cbb>] ? ftrace_ops_control_func+0x6b/0x110
> [260431.893862]  [<ffffffff8170ce00>] ? ftrace_call+0x5/0x2f
> [260431.894745]  [<ffffffff81097def>] ? local_clock+0x3f/0x50
> [260431.895631]  [<ffffffff8107a325>] ? debug_lockdep_rcu_enabled+0x5/0x40
> [260431.896516]  [<ffffffff8107a325>] ? debug_lockdep_rcu_enabled+0x5/0x40
> [260431.897387]  [<ffffffff811079fb>] ? rcu_eqs_enter_common+0x5b/0x420
> [260431.898264]  [<ffffffff81107e24>] rcu_eqs_enter+0x64/0xa0
> [260431.899133]  [<ffffffff8110c673>] rcu_user_enter+0x13/0x20
> [260431.899999]  [<ffffffff8114e67a>] user_enter+0x6a/0xd0
> [260431.900859]  [<ffffffff81010208>] syscall_trace_leave+0x78/0x150
> [260431.901712]  [<ffffffff8170d4af>] int_check_syscall_exit_work+0x34/0x3d
> 


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-26 16:29 ` Paul E. McKenney
@ 2013-08-26 17:30   ` Steven Rostedt
  2013-08-26 17:50     ` Dave Jones
  0 siblings, 1 reply; 17+ messages in thread
From: Steven Rostedt @ 2013-08-26 17:30 UTC (permalink / raw)
  To: paulmck; +Cc: Dave Jones, Linux Kernel

On Mon, 26 Aug 2013 09:29:28 -0700
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:

> On Mon, Aug 26, 2013 at 10:58:38AM -0400, Dave Jones wrote:
> > Another day, another rcu backtrace..
> > This says rc6, but it's pretty darn close to rc7, I think it was running
> > a build from Friday.
> 
> Could you please send your .config?  Also, were you running ftrace,
> perf, RCU event tracing, or what?
> 
> Looks like you are running ftrace, but I though Steven had set that
> up so that it could be called from an extended quiescent state.
> 

I know exactly what the issue is. Yes ftrace is safe to call even from
these extended quiescent states, the problem is that ftrace is also the
infrastructure of other users, where some of those users are not safe.
Namely, perf.

Right now perf is not safe to trace all functions, as some of those
functions have this issue. I was going to add something like:

FTRACE_NON_SAFE(rcu_eqs_enter);

where it will record locations that are not safe for all users, such
that unless a function registers to ftrace with a flag of
"FTRACE_FL_NON_SAFE_OK", anything that is on the non safe list (from
the macro) will not be traced.

Now, how urgent is this fix? perf can only trace functions as root, and
there's no reason for perf to be tracing all functions at the moment.
But yes, a root user could run that and get this warning. Because I was
going to implement this for 3.12 and not for this release.

-- Steve

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-26 17:30   ` Steven Rostedt
@ 2013-08-26 17:50     ` Dave Jones
  2013-08-26 18:18       ` Steven Rostedt
  0 siblings, 1 reply; 17+ messages in thread
From: Dave Jones @ 2013-08-26 17:50 UTC (permalink / raw)
  To: Steven Rostedt; +Cc: paulmck, Linux Kernel

On Mon, Aug 26, 2013 at 01:30:41PM -0400, Steven Rostedt wrote:

 > > On Mon, Aug 26, 2013 at 10:58:38AM -0400, Dave Jones wrote:
 > > > Another day, another rcu backtrace..
 > > > This says rc6, but it's pretty darn close to rc7, I think it was running
 > > > a build from Friday.
 > > 
 > > Could you please send your .config?  Also, were you running ftrace,
 > > perf, RCU event tracing, or what?
 > > 
 > > Looks like you are running ftrace, but I though Steven had set that
 > > up so that it could be called from an extended quiescent state.
 > > 
 > 
 > I know exactly what the issue is. Yes ftrace is safe to call even from
 > these extended quiescent states, the problem is that ftrace is also the
 > infrastructure of other users, where some of those users are not safe.
 > Namely, perf.
 > 
 > Right now perf is not safe to trace all functions, as some of those
 > functions have this issue. I was going to add something like:
 > 
 > FTRACE_NON_SAFE(rcu_eqs_enter);
 > 
 > where it will record locations that are not safe for all users, such
 > that unless a function registers to ftrace with a flag of
 > "FTRACE_FL_NON_SAFE_OK", anything that is on the non safe list (from
 > the macro) will not be traced.
 > 
 > Now, how urgent is this fix? perf can only trace functions as root, and
 > there's no reason for perf to be tracing all functions at the moment.
 > But yes, a root user could run that and get this warning. Because I was
 > going to implement this for 3.12 and not for this release.

This was triggered as a regular user fwiw.
I had not been running perf, or any other tracing. It was just left
fuzzing over the weekend with no interaction at all.

	Dave


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-26 17:50     ` Dave Jones
@ 2013-08-26 18:18       ` Steven Rostedt
  2013-08-26 18:29         ` Dave Jones
  0 siblings, 1 reply; 17+ messages in thread
From: Steven Rostedt @ 2013-08-26 18:18 UTC (permalink / raw)
  To: Dave Jones; +Cc: paulmck, Linux Kernel

On Mon, 26 Aug 2013 13:50:12 -0400
Dave Jones <davej@redhat.com> wrote:
> 
> This was triggered as a regular user fwiw.
> I had not been running perf, or any other tracing. It was just left
> fuzzing over the weekend with no interaction at all.

So you are telling me that ftrace was enabled by a regular user? If so,
that's a huge issue.

> [260431.875883]  [<ffffffff816f9c6f>] dump_stack+0x54/0x74
> [260431.876869]  [<ffffffff810be6a7>] lockdep_rcu_suspicious+0xe7/0x120
> [260431.877852]  [<ffffffff81146ef4>] __perf_event_overflow+0x2a4/0x320
> [260431.878834]  [<ffffffff81146d50>] ? __perf_event_overflow+0x100/0x320
> [260431.879811]  [<ffffffff81146e0c>] ? __perf_event_overflow+0x1bc/0x320
> [260431.880784]  [<ffffffff8170ce00>] ? ftrace_call+0x5/0x2f
> [260431.881758]  [<ffffffff81147101>] perf_swevent_overflow+0x51/0xe0
> [260431.882728]  [<ffffffff811471ef>] perf_swevent_event+0x5f/0x90
> [260431.883685]  [<ffffffff81147329>] perf_tp_event+0x109/0x4f0
> [260431.884637]  [<ffffffff81147542>] ? perf_tp_event+0x322/0x4f0
> [260431.885586]  [<ffffffff811079fb>] ? rcu_eqs_enter_common+0x5b/0x420
> [260431.886539]  [<ffffffff8107a320>] ? task_work_run+0xe0/0xe0
> [260431.887486]  [<ffffffff81135910>] perf_ftrace_function_call+0xc0/0xd0

This is the perf function tracing call.

> [260431.888417]  [<ffffffff81114d37>] ? ftrace_ops_control_func+0xe7/0x110
> [260431.889353]  [<ffffffff8107a320>] ? task_work_run+0xe0/0xe0
> [260431.890281]  [<ffffffff811079fb>] ? rcu_eqs_enter_common+0x5b/0x420
> [260431.891191]  [<ffffffff81114d37>] ftrace_ops_control_func+0xe7/0x110
> [260431.892088]  [<ffffffff8170ce00>] ftrace_call+0x5/0x2f

ftrace_call is the mcount trampoline. The only way to get there is via
function tracing, and function tracing should only be enabled by root.

> [260431.892975]  [<ffffffff81114cbb>] ? ftrace_ops_control_func+0x6b/0x110
> [260431.893862]  [<ffffffff8170ce00>] ? ftrace_call+0x5/0x2f
> [260431.894745]  [<ffffffff81097def>] ? local_clock+0x3f/0x50
> [260431.895631]  [<ffffffff8107a325>] ? debug_lockdep_rcu_enabled+0x5/0x40
> [260431.896516]  [<ffffffff8107a325>] ? debug_lockdep_rcu_enabled+0x5/0x40
> [260431.897387]  [<ffffffff811079fb>] ? rcu_eqs_enter_common+0x5b/0x420
> [260431.898264]  [<ffffffff81107e24>] rcu_eqs_enter+0x64/0xa0
> [260431.899133]  [<ffffffff8110c673>] rcu_user_enter+0x13/0x20
> [260431.899999]  [<ffffffff8114e67a>] user_enter+0x6a/0xd0
> [260431.900859]  [<ffffffff81010208>] syscall_trace_leave+0x78/0x150
> [260431.901712]  [<ffffffff8170d4af>] int_check_syscall_exit_work+0x34/0x3d

So my question to you. If you were not running perf or any other
tracing, and this is all just non-root user. How the hell did perf
function tracing get started on your box????

-- Steve

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-26 18:18       ` Steven Rostedt
@ 2013-08-26 18:29         ` Dave Jones
  2013-08-26 19:03           ` Steven Rostedt
  2013-08-26 19:43           ` David Ahern
  0 siblings, 2 replies; 17+ messages in thread
From: Dave Jones @ 2013-08-26 18:29 UTC (permalink / raw)
  To: Steven Rostedt; +Cc: paulmck, Linux Kernel

On Mon, Aug 26, 2013 at 02:18:14PM -0400, Steven Rostedt wrote:
 > On Mon, 26 Aug 2013 13:50:12 -0400
 > Dave Jones <davej@redhat.com> wrote:
 > > 
 > > This was triggered as a regular user fwiw.
 > > I had not been running perf, or any other tracing. It was just left
 > > fuzzing over the weekend with no interaction at all.
 > 
 > So you are telling me that ftrace was enabled by a regular user? If so,
 > that's a huge issue.
 
quite.

 > So my question to you. If you were not running perf or any other
 > tracing, and this is all just non-root user. How the hell did perf
 > function tracing get started on your box????

What mechanisms are available that would trigger it being enabled ?

Is there some path through sys_perf_open_event that might be
missing a capability check perhaps ?

	Dave


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-26 18:29         ` Dave Jones
@ 2013-08-26 19:03           ` Steven Rostedt
  2013-08-27 12:16             ` Peter Zijlstra
  2013-08-26 19:43           ` David Ahern
  1 sibling, 1 reply; 17+ messages in thread
From: Steven Rostedt @ 2013-08-26 19:03 UTC (permalink / raw)
  To: Dave Jones; +Cc: paulmck, Linux Kernel, Ingo Molnar, Peter Zijlstra, Jiri Olsa

On Mon, 26 Aug 2013 14:29:07 -0400
Dave Jones <davej@redhat.com> wrote:

> On Mon, Aug 26, 2013 at 02:18:14PM -0400, Steven Rostedt wrote:
>  > On Mon, 26 Aug 2013 13:50:12 -0400
>  > Dave Jones <davej@redhat.com> wrote:
>  > > 
>  > > This was triggered as a regular user fwiw.
>  > > I had not been running perf, or any other tracing. It was just left
>  > > fuzzing over the weekend with no interaction at all.
>  > 
>  > So you are telling me that ftrace was enabled by a regular user? If so,
>  > that's a huge issue.
>  
> quite.
> 
>  > So my question to you. If you were not running perf or any other
>  > tracing, and this is all just non-root user. How the hell did perf
>  > function tracing get started on your box????
> 
> What mechanisms are available that would trigger it being enabled ?
> 
> Is there some path through sys_perf_open_event that might be
> missing a capability check perhaps ?
> 

That's a question for Ingo, Peter or Jiri.

-- Steve

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-26 18:29         ` Dave Jones
  2013-08-26 19:03           ` Steven Rostedt
@ 2013-08-26 19:43           ` David Ahern
  2013-08-26 19:49             ` Dave Jones
  1 sibling, 1 reply; 17+ messages in thread
From: David Ahern @ 2013-08-26 19:43 UTC (permalink / raw)
  To: Dave Jones, Steven Rostedt, paulmck, Linux Kernel

On 8/26/13 12:29 PM, Dave Jones wrote:
> On Mon, Aug 26, 2013 at 02:18:14PM -0400, Steven Rostedt wrote:
>   > On Mon, 26 Aug 2013 13:50:12 -0400
>   > Dave Jones <davej@redhat.com> wrote:
>   > >
>   > > This was triggered as a regular user fwiw.
>   > > I had not been running perf, or any other tracing. It was just left
>   > > fuzzing over the weekend with no interaction at all.
>   >
>   > So you are telling me that ftrace was enabled by a regular user? If so,
>   > that's a huge issue.
>
> quite.
>
>   > So my question to you. If you were not running perf or any other
>   > tracing, and this is all just non-root user. How the hell did perf
>   > function tracing get started on your box????
>
> What mechanisms are available that would trigger it being enabled ?
>
> Is there some path through sys_perf_open_event that might be
> missing a capability check perhaps ?

Do you have /sys/kernel/debug with access permissions?

David


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-26 19:43           ` David Ahern
@ 2013-08-26 19:49             ` Dave Jones
  2013-08-27 13:10               ` Steven Rostedt
  0 siblings, 1 reply; 17+ messages in thread
From: Dave Jones @ 2013-08-26 19:49 UTC (permalink / raw)
  To: David Ahern; +Cc: Steven Rostedt, paulmck, Linux Kernel

On Mon, Aug 26, 2013 at 01:43:15PM -0600, David Ahern wrote:
 > On 8/26/13 12:29 PM, Dave Jones wrote:
 > > On Mon, Aug 26, 2013 at 02:18:14PM -0400, Steven Rostedt wrote:
 > >   > On Mon, 26 Aug 2013 13:50:12 -0400
 > >   > Dave Jones <davej@redhat.com> wrote:
 > >   > >
 > >   > > This was triggered as a regular user fwiw.
 > >   > > I had not been running perf, or any other tracing. It was just left
 > >   > > fuzzing over the weekend with no interaction at all.
 > >   >
 > >   > So you are telling me that ftrace was enabled by a regular user? If so,
 > >   > that's a huge issue.
 > >
 > > quite.
 > >
 > >   > So my question to you. If you were not running perf or any other
 > >   > tracing, and this is all just non-root user. How the hell did perf
 > >   > function tracing get started on your box????
 > >
 > > What mechanisms are available that would trigger it being enabled ?
 > >
 > > Is there some path through sys_perf_open_event that might be
 > > missing a capability check perhaps ?
 > 
 > Do you have /sys/kernel/debug with access permissions?

Ah, yeah, that'll be it. Good catch.

Sorry for the false alarm Steve ;)

	Dave


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-26 19:03           ` Steven Rostedt
@ 2013-08-27 12:16             ` Peter Zijlstra
  2013-08-30 15:49               ` Frederic Weisbecker
  2013-08-30 15:52               ` Frederic Weisbecker
  0 siblings, 2 replies; 17+ messages in thread
From: Peter Zijlstra @ 2013-08-27 12:16 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: Dave Jones, paulmck, Linux Kernel, Ingo Molnar, Jiri Olsa,
	Frederic Weisbecker

On Mon, Aug 26, 2013 at 03:03:04PM -0400, Steven Rostedt wrote:
> > Is there some path through sys_perf_open_event that might be
> > missing a capability check perhaps ?
> > 
> 
> That's a question for Ingo, Peter or Jiri.

Its not something I've looked at recently, git blames Jiri and fweisbec
for most of that code.

Permission checks appear to live in
kernel/trace/trace_event_perf.c:perf_trace_event_perm().

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-26 19:49             ` Dave Jones
@ 2013-08-27 13:10               ` Steven Rostedt
  2013-08-27 13:58                 ` David Ahern
  0 siblings, 1 reply; 17+ messages in thread
From: Steven Rostedt @ 2013-08-27 13:10 UTC (permalink / raw)
  To: Dave Jones; +Cc: David Ahern, paulmck, Linux Kernel

On Mon, 26 Aug 2013 15:49:24 -0400
Dave Jones <davej@redhat.com> wrote:


>  > Do you have /sys/kernel/debug with access permissions?
> 
> Ah, yeah, that'll be it. Good catch.
> 
> Sorry for the false alarm Steve ;)

Yeah, but it still does not explain how perf got started. Perf requires
a sys_perf_event_open() call to run.

-- Steve

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-27 13:10               ` Steven Rostedt
@ 2013-08-27 13:58                 ` David Ahern
  2013-08-27 14:13                   ` Dave Jones
  0 siblings, 1 reply; 17+ messages in thread
From: David Ahern @ 2013-08-27 13:58 UTC (permalink / raw)
  To: Steven Rostedt; +Cc: Dave Jones, paulmck, Linux Kernel

On 8/27/13 7:10 AM, Steven Rostedt wrote:
> On Mon, 26 Aug 2013 15:49:24 -0400
> Dave Jones <davej@redhat.com> wrote:
>
>
>>   > Do you have /sys/kernel/debug with access permissions?
>>
>> Ah, yeah, that'll be it. Good catch.
>>
>> Sorry for the false alarm Steve ;)
>
> Yeah, but it still does not explain how perf got started. Perf requires
> a sys_perf_event_open() call to run.

Per Peter's response another option is the paranoia level:

# perf event paranoia level:
# -1 - not paranoid at all
#  0 - disallow raw tracepoint access for unpriv
#  1 - disallow cpu events for unpriv
#  2 - disallow kernel profiling for unpriv
kernel.perf_event_paranoid = -1

I keep my dev box at -1 and set /sys/kernel/debug to 755 to run perf as 
non-root.

David

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-27 13:58                 ` David Ahern
@ 2013-08-27 14:13                   ` Dave Jones
  0 siblings, 0 replies; 17+ messages in thread
From: Dave Jones @ 2013-08-27 14:13 UTC (permalink / raw)
  To: David Ahern; +Cc: Steven Rostedt, paulmck, Linux Kernel

On Tue, Aug 27, 2013 at 07:58:12AM -0600, David Ahern wrote:
 > On 8/27/13 7:10 AM, Steven Rostedt wrote:
 > > On Mon, 26 Aug 2013 15:49:24 -0400
 > > Dave Jones <davej@redhat.com> wrote:
 > >
 > >
 > >>   > Do you have /sys/kernel/debug with access permissions?
 > >>
 > >> Ah, yeah, that'll be it. Good catch.
 > >>
 > >> Sorry for the false alarm Steve ;)
 > >
 > > Yeah, but it still does not explain how perf got started. Perf requires
 > > a sys_perf_event_open() call to run.
 > 
 > Per Peter's response another option is the paranoia level:
 > 
 > # perf event paranoia level:
 > # -1 - not paranoid at all
 > #  0 - disallow raw tracepoint access for unpriv
 > #  1 - disallow cpu events for unpriv
 > #  2 - disallow kernel profiling for unpriv
 > kernel.perf_event_paranoid = -1

This I have set to 1 on that box. (The default)

	Dave


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-27 12:16             ` Peter Zijlstra
@ 2013-08-30 15:49               ` Frederic Weisbecker
  2013-08-30 15:52               ` Frederic Weisbecker
  1 sibling, 0 replies; 17+ messages in thread
From: Frederic Weisbecker @ 2013-08-30 15:49 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Steven Rostedt, Dave Jones, paulmck, Linux Kernel, Ingo Molnar,
	Jiri Olsa

On Tue, Aug 27, 2013 at 02:16:29PM +0200, Peter Zijlstra wrote:
> On Mon, Aug 26, 2013 at 03:03:04PM -0400, Steven Rostedt wrote:
> > > Is there some path through sys_perf_open_event that might be
> > > missing a capability check perhaps ?
> > > 
> > 
> > That's a question for Ingo, Peter or Jiri.
> 
> Its not something I've looked at recently, git blames Jiri and fweisbec
> for most of that code.
> 
> Permission checks appear to live in
> kernel/trace/trace_event_perf.c:perf_trace_event_perm().

Well, it also depends on sysctl kernel.perf_event_paranoid settings.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-27 12:16             ` Peter Zijlstra
  2013-08-30 15:49               ` Frederic Weisbecker
@ 2013-08-30 15:52               ` Frederic Weisbecker
  2013-08-30 15:59                 ` Peter Zijlstra
  1 sibling, 1 reply; 17+ messages in thread
From: Frederic Weisbecker @ 2013-08-30 15:52 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Steven Rostedt, Dave Jones, paulmck, Linux Kernel, Ingo Molnar,
	Jiri Olsa

On Tue, Aug 27, 2013 at 02:16:29PM +0200, Peter Zijlstra wrote:
> On Mon, Aug 26, 2013 at 03:03:04PM -0400, Steven Rostedt wrote:
> > > Is there some path through sys_perf_open_event that might be
> > > missing a capability check perhaps ?
> > > 
> > 
> > That's a question for Ingo, Peter or Jiri.
> 
> Its not something I've looked at recently, git blames Jiri and fweisbec
> for most of that code.
> 
> Permission checks appear to live in
> kernel/trace/trace_event_perf.c:perf_trace_event_perm().

Actually the following condition is weird:

	 /* The ftrace function trace is allowed only for root. */
	 if (ftrace_event_is_function(tp_event) &&
	     perf_paranoid_kernel() && !capable(CAP_SYS_ADMIN))
     			    return -EPERM;

We probably intended to do:

   /* The ftrace function trace is allowed only for root. */
   if (ftrace_event_is_function(tp_event) ||
       perf_paranoid_kernel() && !capable(CAP_SYS_ADMIN))
       			      return -EPERM;

Can somebody confirm?

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-30 15:52               ` Frederic Weisbecker
@ 2013-08-30 15:59                 ` Peter Zijlstra
  2013-08-30 16:06                   ` Frederic Weisbecker
  0 siblings, 1 reply; 17+ messages in thread
From: Peter Zijlstra @ 2013-08-30 15:59 UTC (permalink / raw)
  To: Frederic Weisbecker
  Cc: Steven Rostedt, Dave Jones, paulmck, Linux Kernel, Ingo Molnar,
	Jiri Olsa

On Fri, Aug 30, 2013 at 05:52:43PM +0200, Frederic Weisbecker wrote:
> On Tue, Aug 27, 2013 at 02:16:29PM +0200, Peter Zijlstra wrote:
> > On Mon, Aug 26, 2013 at 03:03:04PM -0400, Steven Rostedt wrote:
> > > > Is there some path through sys_perf_open_event that might be
> > > > missing a capability check perhaps ?
> > > > 
> > > 
> > > That's a question for Ingo, Peter or Jiri.
> > 
> > Its not something I've looked at recently, git blames Jiri and fweisbec
> > for most of that code.
> > 
> > Permission checks appear to live in
> > kernel/trace/trace_event_perf.c:perf_trace_event_perm().
> 
> Actually the following condition is weird:
> 
> 	 /* The ftrace function trace is allowed only for root. */
> 	 if (ftrace_event_is_function(tp_event) &&
> 	     perf_paranoid_kernel() && !capable(CAP_SYS_ADMIN))
>      			    return -EPERM;
> 

That says: If its its a function-event and we're paranoid but we don't
have root, bail.

> We probably intended to do:
> 
>    /* The ftrace function trace is allowed only for root. */
>    if (ftrace_event_is_function(tp_event) ||
>        perf_paranoid_kernel() && !capable(CAP_SYS_ADMIN))
>        			      return -EPERM;
> 
> Can somebody confirm?

That would always disallow function-events, no?

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: suspicious RCU usage (perf)
  2013-08-30 15:59                 ` Peter Zijlstra
@ 2013-08-30 16:06                   ` Frederic Weisbecker
  0 siblings, 0 replies; 17+ messages in thread
From: Frederic Weisbecker @ 2013-08-30 16:06 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Steven Rostedt, Dave Jones, paulmck, Linux Kernel, Ingo Molnar,
	Jiri Olsa

On Fri, Aug 30, 2013 at 05:59:36PM +0200, Peter Zijlstra wrote:
> On Fri, Aug 30, 2013 at 05:52:43PM +0200, Frederic Weisbecker wrote:
> > On Tue, Aug 27, 2013 at 02:16:29PM +0200, Peter Zijlstra wrote:
> > > On Mon, Aug 26, 2013 at 03:03:04PM -0400, Steven Rostedt wrote:
> > > > > Is there some path through sys_perf_open_event that might be
> > > > > missing a capability check perhaps ?
> > > > > 
> > > > 
> > > > That's a question for Ingo, Peter or Jiri.
> > > 
> > > Its not something I've looked at recently, git blames Jiri and fweisbec
> > > for most of that code.
> > > 
> > > Permission checks appear to live in
> > > kernel/trace/trace_event_perf.c:perf_trace_event_perm().
> > 
> > Actually the following condition is weird:
> > 
> > 	 /* The ftrace function trace is allowed only for root. */
> > 	 if (ftrace_event_is_function(tp_event) &&
> > 	     perf_paranoid_kernel() && !capable(CAP_SYS_ADMIN))
> >      			    return -EPERM;
> > 
> 
> That says: If its its a function-event and we're paranoid but we don't
> have root, bail.

Right, I misunderstood and thought we messed up general tracepoint permissions
with function events.

> 
> > We probably intended to do:
> > 
> >    /* The ftrace function trace is allowed only for root. */
> >    if (ftrace_event_is_function(tp_event) ||
> >        perf_paranoid_kernel() && !capable(CAP_SYS_ADMIN))
> >        			      return -EPERM;
> > 
> > Can somebody confirm?
> 
> That would always disallow function-events, no?

Yeah, sorry, returning from holidays require more dense coffee.

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2013-08-30 16:06 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-08-26 14:58 suspicious RCU usage (perf) Dave Jones
2013-08-26 16:29 ` Paul E. McKenney
2013-08-26 17:30   ` Steven Rostedt
2013-08-26 17:50     ` Dave Jones
2013-08-26 18:18       ` Steven Rostedt
2013-08-26 18:29         ` Dave Jones
2013-08-26 19:03           ` Steven Rostedt
2013-08-27 12:16             ` Peter Zijlstra
2013-08-30 15:49               ` Frederic Weisbecker
2013-08-30 15:52               ` Frederic Weisbecker
2013-08-30 15:59                 ` Peter Zijlstra
2013-08-30 16:06                   ` Frederic Weisbecker
2013-08-26 19:43           ` David Ahern
2013-08-26 19:49             ` Dave Jones
2013-08-27 13:10               ` Steven Rostedt
2013-08-27 13:58                 ` David Ahern
2013-08-27 14:13                   ` Dave Jones

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.