All of lore.kernel.org
 help / color / mirror / Atom feed
From: Vince Weaver <vincent.weaver@maine.edu>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Vince Weaver <vincent.weaver@maine.edu>,
	Ingo Molnar <mingo@kernel.org>,
	linux-kernel@vger.kernel.org,
	Thomas Gleixner <tglx@linutronix.de>,
	Steven Rostedt <rostedt@goodmis.org>
Subject: Re: [perf] more perf_fuzzer memory corruption
Date: Tue, 29 Apr 2014 14:21:56 -0400 (EDT)	[thread overview]
Message-ID: <alpine.DEB.2.10.1404291411220.22490@vincent-weaver-1.umelst.maine.edu> (raw)
In-Reply-To: <20140429094632.GP27561@twins.programming.kicks-ass.net>

On Tue, 29 Apr 2014, Peter Zijlstra wrote:

> > Event #16 is a SW event created and running in the parent on CPU0.
> 
> A regular software one, right? Not a timer one.

Maybe.  From traces I have it looks like it's a regular one (i.e. calls 
 perf_swevent_add() ) but who knows at this point.

When I actually got a trace with perf_event_open() instrumented to print 
some attr values it looked like things were being caused by
PERF_COUNT_SW_TASK_CLOCK which makes no sense.

> > CPU6 (child) shutting down.
> >    last user of event #16
> >    perf_release() called on event
> >    which eventually calls event_sched_out()
> >    which calls pmu->del which removes event from swevent_htable
> >    *but only on CPU6*
> 
> So on fork() we'll also clone the counter; after which there's two. One
> will run on each task.

even if inherit isn't set?

> Because of a context switch optimization they can actually flip around
> (the below patch disables that).

ENOPATCH?

> quite the puzzle this one

yes.

I'm tediously working on trying to get a good trace of this happening.

I have a random seed that will trigger the bug in the fuzzer around 1 time 
in 10.

Unfortunately many of the times it crashes so hard/quickly there's no 
chance of getting the trace data (dump trace on oops never holds enough 
state, and often the fuzzing triggers its own random trace events that 
clutter those logs).

Also trace-cmd is a pain to use.  Any suggested events I should trace 
beyond the obvious?

Part of the problem is that despite what the documentation says it doesn't 
look like you can combine the "-P pid" and "-c" children option, which 
makes debugging a forking problem like this a lot harder to trace.

It's sort of possible to get around that with a really complicated -F ""
command line that does sudo back to me (don't want to fuzz as root) and 
such, but still awkward.

Vince

  reply	other threads:[~2014-04-29 18:18 UTC|newest]

Thread overview: 81+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-04-15 21:37 [perf] more perf_fuzzer memory corruption Vince Weaver
2014-04-15 21:49 ` Thomas Gleixner
2014-04-16  3:21   ` Vince Weaver
2014-04-16  4:18     ` Vince Weaver
2014-04-16 14:15 ` Peter Zijlstra
2014-04-16 17:30   ` Vince Weaver
2014-04-16 17:43     ` Vince Weaver
2014-04-16 17:47       ` Peter Zijlstra
2014-04-17  9:48       ` Ingo Molnar
2014-04-17 11:45         ` Peter Zijlstra
2014-04-17 14:22           ` Ingo Molnar
2014-04-17 14:42             ` Vince Weaver
2014-04-17 14:54               ` Peter Zijlstra
2014-04-17 15:35                 ` Vince Weaver
2014-04-18 14:45                 ` Vince Weaver
2014-04-18 14:51                   ` Vince Weaver
2014-04-18 15:23                   ` Peter Zijlstra
2014-04-18 16:59                     ` Peter Zijlstra
2014-04-18 17:15                       ` Peter Zijlstra
2014-04-23 20:58                         ` Vince Weaver
2014-04-25  2:51                           ` Vince Weaver
2014-04-28 14:21                             ` Vince Weaver
2014-04-28 19:38                               ` Vince Weaver
2014-04-29  9:46                                 ` Peter Zijlstra
2014-04-29 18:21                                   ` Vince Weaver [this message]
2014-04-29 19:01                                     ` Peter Zijlstra
2014-04-29 20:59                                       ` Vince Weaver
2014-04-30 18:44                                         ` Peter Zijlstra
2014-04-30 21:08                                           ` Vince Weaver
2014-04-30 22:51                                             ` Thomas Gleixner
2014-05-01 10:26                                               ` Peter Zijlstra
2014-05-01 11:50                                                 ` Peter Zijlstra
2014-05-01 12:35                                                   ` Thomas Gleixner
2014-05-01 13:12                                                     ` Peter Zijlstra
2014-05-01 13:29                                                     ` Thomas Gleixner
2014-05-01 13:22                                                 ` Vince Weaver
2014-05-01 14:07                                           ` Vince Weaver
2014-05-01 14:27                                             ` Vince Weaver
2014-05-01 15:09                                               ` Peter Zijlstra
2014-05-01 15:50                                                 ` Vince Weaver
2014-05-01 16:31                                                   ` Thomas Gleixner
2014-05-01 17:18                                                     ` Vince Weaver
2014-05-01 18:49                                                       ` Vince Weaver
2014-05-01 21:32                                                         ` Vince Weaver
2014-05-02 11:15                                                         ` Peter Zijlstra
2014-05-02 15:42                                                         ` Peter Zijlstra
2014-05-02 16:22                                                           ` Vince Weaver
2014-05-02 16:22                                                             ` Peter Zijlstra
2014-05-02 16:43                                                               ` Vince Weaver
2014-05-02 17:27                                                                 ` Peter Zijlstra
2014-05-02 17:46                                                                   ` Vince Weaver
2014-05-02 19:12                                                                     ` Thomas Gleixner
2014-05-02 20:15                                                                       ` Vince Weaver
2014-05-02 20:45                                                                         ` Thomas Gleixner
2014-05-03  2:32                                                                           ` Vince Weaver
2014-05-03  3:02                                                                             ` Vince Weaver
2014-05-03  7:33                                                                               ` Peter Zijlstra
2014-05-05  9:31                                                                               ` Peter Zijlstra
2014-05-05 16:00                                                                                 ` Vince Weaver
2014-05-05 17:10                                                                                   ` Vince Weaver
2014-05-05 17:14                                                                                     ` Peter Zijlstra
2014-05-05 18:47                                                                                       ` Vince Weaver
2014-05-05 19:36                                                                                         ` Peter Zijlstra
2014-05-05 19:51                                                                                           ` Vince Weaver
2014-05-06  1:06                                                                                         ` Vince Weaver
2014-05-06 16:57                                                                                           ` Vince Weaver
2014-05-07 16:45                                                                                             ` Peter Zijlstra
2014-05-08 10:40                                                                                       ` [tip:perf/core] perf: Fix perf_event_init_context() tip-bot for Peter Zijlstra
2014-05-05 17:29                                                                                   ` [perf] more perf_fuzzer memory corruption Ingo Molnar
2014-05-06  4:51                                                                                     ` Vince Weaver
2014-05-06 17:06                                                                                       ` Vince Weaver
2014-05-07 19:12                                                                                         ` Ingo Molnar
2014-05-07 19:11                                                                                       ` Ingo Molnar
2014-05-08 10:40                                                                                 ` [tip:perf/core] perf: Fix race in removing an event tip-bot for Peter Zijlstra
2014-05-02 17:06                                                           ` [perf] more perf_fuzzer memory corruption Vince Weaver
2014-05-02 17:04                                                             ` Peter Zijlstra
2014-04-29 19:26                                     ` Steven Rostedt
2014-04-29  8:52                               ` Peter Zijlstra
2014-04-29 18:11                                 ` Vince Weaver
2014-04-29 19:21                                   ` Steven Rostedt
2014-04-28 17:48                             ` Thomas Gleixner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=alpine.DEB.2.10.1404291411220.22490@vincent-weaver-1.umelst.maine.edu \
    --to=vincent.weaver@maine.edu \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.