Re: [RFC][PATCH 2/3] perf: Add a bit of paranoia

[Peter Zijlstra <peterz@infradead.org>]

On Fri, Jan 23, 2015 at 01:52:01PM +0100, Peter Zijlstra wrote:
> @@ -1442,6 +1450,10 @@ event_sched_out(struct perf_event *event
>  {
>  	u64 tstamp = perf_event_time(event);
>  	u64 delta;
> +
> +	WARN_ON_ONCE(event->ctx != ctx);
> +	lockdep_assert_held(&ctx->lock);
> +
>  	/*
>  	 * An event which could not be activated because of
>  	 * filter mismatch still needs to have its timings

Jiri reported triggering that WARN_ON_ONCE over the weekend:

 event_sched_out.isra.79+0x2b9/0x2d0
 group_sched_out+0x69/0xc0
 ctx_sched_out+0x106/0x130
 task_ctx_sched_out+0x37/0x70
 __perf_install_in_context+0x70/0x1a0
 remote_function+0x48/0x60
 generic_exec_single+0x15b/0x1d0
 smp_call_function_single+0x67/0xa0
 task_function_call+0x53/0x80
 perf_install_in_context+0x8b/0x110


I think the below should cure this; if we install a group leader it will
iterate the (still intact) group list and find its siblings and try and
install those too -- even though those still have the old event->ctx --
in the new ctx.

Upon installing the first group sibling we'd try and schedule out the
group and trigger the above warn.

Fix this by installing the group leader last, installing siblings would
have no effect, they're not reachable through the group lists and
therefore we don't schedule them.

Also delay resetting the state until we're absolutely sure the events
are quiescent -- which raises the question; should perf_pmu_migrate()
not also have perf_event__state_init() calls in?

---
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -7645,16 +7645,9 @@ SYSCALL_DEFINE5(perf_event_open,
 
 		perf_remove_from_context(group_leader, false);
 
-		/*
-		 * Removing from the context ends up with disabled
-		 * event. What we want here is event in the initial
-		 * startup state, ready to be add into new context.
-		 */
-		perf_event__state_init(group_leader);
 		list_for_each_entry(sibling, &group_leader->sibling_list,
 				    group_entry) {
 			perf_remove_from_context(sibling, false);
-			perf_event__state_init(sibling);
 			put_ctx(gctx);
 		}
 	} else {
@@ -7670,13 +7663,31 @@ SYSCALL_DEFINE5(perf_event_open,
 		 */
 		synchronize_rcu();
 
-		perf_install_in_context(ctx, group_leader, group_leader->cpu);
-		get_ctx(ctx);
+		/*
+		 * Install the group siblings before the group leader.
+		 *
+		 * Because a group leader will try and install the entire group
+		 * (through the sibling list, which is still in-tact), we can
+		 * end up with siblings installed in the wrong context.
+		 *
+		 * By installing siblings first we NO-OP because they're not
+		 * reachable through the group lists.
+		 */
 		list_for_each_entry(sibling, &group_leader->sibling_list,
 				    group_entry) {
+			perf_event__state_init(sibling);
 			perf_install_in_context(ctx, sibling, sibling->cpu);
 			get_ctx(ctx);
 		}
+
+		/*
+		 * Removing from the context ends up with disabled
+		 * event. What we want here is event in the initial
+		 * startup state, ready to be add into new context.
+		 */
+		perf_event__state_init(group_leader);
+		perf_install_in_context(ctx, group_leader, group_leader->cpu);
+		get_ctx(ctx);
 	}
 
 	perf_install_in_context(ctx, event, event->cpu);