All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andi Kleen <ak@linux.intel.com>
To: Jiri Olsa <jolsa@redhat.com>
Cc: Andi Kleen <andi@firstfloor.org>,
	acme@kernel.org, jolsa@kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 10/10] tools, perf, script: Implement --graph-function
Date: Mon, 3 Sep 2018 08:44:08 -0700	[thread overview]
Message-ID: <20180903154408.GG27886@tassilo.jf.intel.com> (raw)
In-Reply-To: <20180903075128.GE24575@krava>

On Mon, Sep 03, 2018 at 09:51:28AM +0200, Jiri Olsa wrote:
> On Fri, Aug 31, 2018 at 03:02:06PM -0700, Andi Kleen wrote:
> > From: Andi Kleen <ak@linux.intel.com>
> > 
> > Add a ftrace style --graph-function argument to perf script that allows
> > to print itrace function calls only below a given function. This
> > makes it easier to find the code of interest in a large trace.
> > 
> > % perf record -e intel_pt//k -a sleep 1
> > % perf script --graph-function group_sched_in --call-trace
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])          group_sched_in
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])              __x86_indirect_thunk_rax
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])              event_sched_in.isra.107
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])                  perf_event_set_state.part.71
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])                      perf_event_update_time
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])                  perf_pmu_disable
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])                  perf_log_itrace_start
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])                  __x86_indirect_thunk_rax
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])                      perf_event_update_userpage
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])                          calc_timer_values
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])                              sched_clock_cpu
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])                          __x86_indirect_thunk_rax
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])                          arch_perf_update_userpage
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])                              __fentry__
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])                              using_native_sched_clock
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])                              sched_clock_stable
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])                  perf_pmu_enable
> >             perf   900 [000] 194167.205652203: ([kernel.kallsyms])              __x86_indirect_thunk_rax
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])          group_sched_in
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])              __x86_indirect_thunk_rax
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])              event_sched_in.isra.107
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])                  perf_event_set_state.part.71
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])                      perf_event_update_time
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])                  perf_pmu_disable
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])                  perf_log_itrace_start
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])                  __x86_indirect_thunk_rax
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])                      perf_event_update_userpage
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])                          calc_timer_values
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])                              sched_clock_cpu
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])                          __x86_indirect_thunk_rax
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])                          arch_perf_update_userpage
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])                              __fentry__
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])                              using_native_sched_clock
> >          swapper     0 [001] 194167.205660693: ([kernel.kallsyms])                              sched_clock_stable
> 
> nice, but I'm also getting a lot of these lines:
> 
>  instruction trace error type 1 cpu 0 pid 0 tid 0 ip 0xffffffffb3a017a0 code 6: Trace doesn't match instruction
>  instruction trace error type 1 cpu 0 pid 0 tid 0 ip 0xffffffffb3a017a0 code 6: Trace doesn't match instruction
>  instruction trace error type 1 cpu 0 pid 0 tid 0 ip 0xffffffffb3a017a0 code 6: Trace doesn't match instruction
> 
> any idea where are those comming from?

PT decoder didn't see the same instructions as during the trace collection.

There are many reasons which can cause this, usually (but not always) some form of
self modifying code.  The Linux kernel does a lot of self modification these days,
which is a challenge for PT.

If you run the decoder as root it will decode from /proc/kcore, which helps, but
only if no jump label has changed state since the collection. 

The perf PT decoder tends to resync relatively quickly, so usually not too much
is lost.

-Andi


  reply	other threads:[~2018-09-03 15:44 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-31 22:01 Make perf script easier to use for itrace Andi Kleen
2018-08-31 22:01 ` [PATCH v2 01/10] perf tools: Report itrace options in help Andi Kleen
2018-08-31 22:01 ` [PATCH v2 02/10] tools, pager: Support overwriting the pager Andi Kleen
2018-08-31 22:01 ` [PATCH v2 03/10] tools, perf, script: Add --insn-trace for instruction decoding Andi Kleen
2018-08-31 22:02 ` [PATCH v2 04/10] perf, tools, script: Allow sym and dso without ip, addr Andi Kleen
2018-09-03  7:15   ` Jiri Olsa
2018-09-03 16:12     ` Andi Kleen
2018-08-31 22:02 ` [PATCH v2 05/10] tools, perf, script: Set user_set/wildcard_set for +- fields in -F Andi Kleen
2018-08-31 22:02 ` [PATCH v2 06/10] perf, tools, script: Implement - for typed fields Andi Kleen
2018-09-03  7:17   ` Jiri Olsa
2018-08-31 22:02 ` [PATCH v2 07/10] perf, tools, script: Print DSO for callindent Andi Kleen
2018-08-31 22:02 ` [PATCH v2 08/10] perf, tools, script: Make itrace script default to all calls Andi Kleen
2018-08-31 22:02 ` [PATCH v2 09/10] tools, perf, script: Add --call-trace and --call-ret-trace Andi Kleen
2018-08-31 22:02 ` [PATCH v2 10/10] tools, perf, script: Implement --graph-function Andi Kleen
2018-09-03  7:51   ` Jiri Olsa
2018-09-03 15:44     ` Andi Kleen [this message]
2018-08-31 22:54 ` Make perf script easier to use for itrace Kim Phillips
2018-09-03  7:31   ` Jiri Olsa
2018-09-03 15:50   ` Andi Kleen
2018-09-03  7:52 ` Jiri Olsa

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180903154408.GG27886@tassilo.jf.intel.com \
    --to=ak@linux.intel.com \
    --cc=acme@kernel.org \
    --cc=andi@firstfloor.org \
    --cc=jolsa@kernel.org \
    --cc=jolsa@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.