[GIT PULL 0/4] perf/urgent fixes

[GIT PULL 0/4] perf/urgent fixes

[Arnaldo Carvalho de Melo]

Hi Ingo,

	Please consider pulling,

- Arnaldo

The following changes since commit 5b5e76218fbdbb71a01d5480f289ead624232876:

  Merge tag 'perf-urgent-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/urgent (2015-01-01 22:24:36 +0100)

are available in the git repository at:


  git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-urgent-for-mingo

for you to fetch changes up to e7024fc3783317608b8e07048116a72a7d1cd26d:

  perf diff: Fix to sort by baseline field by default (2015-01-02 23:27:18 -0300)

----------------------------------------------------------------
perf/urgent fixes:

- 'perf probe' should fall back to find probe point in symbols when failing
  to do so in a debuginfo file (Masami Hiramatsu)

- Fix 'perf probe' crash in dwarf_getcfi_elf (Namhyung Kim)

- Fix shell completion with 'perf list' --raw-dump option (Taesoo Kim)

- Fix 'perf diff' to sort by baseline field by default (Namhyung Kim)

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

----------------------------------------------------------------
Masami Hiramatsu (1):
      perf probe: Fix to fall back to find probe point in symbols

Namhyung Kim (2):
      perf probe: Fix crash in dwarf_getcfi_elf
      perf diff: Fix to sort by baseline field by default

Taesoo Kim (1):
      perf list: Fix --raw-dump option

 tools/perf/builtin-diff.c      | 44 ++++++++++++++++++++++++++++++++++++++++++
 tools/perf/builtin-list.c      | 13 ++++++++++---
 tools/perf/util/probe-event.c  |  6 ++++--
 tools/perf/util/probe-finder.c | 18 ++++++++++++++++-
 4 files changed, 75 insertions(+), 6 deletions(-)

[PATCH 1/4] perf probe: Fix to fall back to find probe point in symbols

[Arnaldo Carvalho de Melo]

From: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>

Fix to fall back to find a probe point in symbols if perf fails to find
it in debuginfo.

This can happen when the target function is an alias of another
function. Such alias doesn't have an entry in debuginfo but in symbols.

David Ahern reported this problem in https://lkml.org/lkml/2014/12/29/355

I ensured the problem and deeper investigation discovers it.
 -----
 eu-readelf --debug-dump=info /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.19.so | grep \"malloc\" -A6
             name                 (strp) "malloc"
             decl_file            (data1) 25
             decl_line            (data2) 466
             prototyped           (flag_present)
             type                 (ref4) [  81b5]
             declaration          (flag_present)
 [  8f58]      formal_parameter
 --
             name                 (strp) "malloc"
             decl_file            (data1) 23
             decl_line            (data2) 466
             prototyped           (flag_present)
             type                 (ref4) [  9f4a]
             declaration          (flag_present)
             sibling              (ref4) [  bb29]
 ...
 -----
All these entires have no instances (all of them are declarations)
This is why the perf probe failed to find it in debuginfo.

However, there are some malloc instances in symbols.
 -----
 eu-readelf --symbols /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.19.so | grep malloc$
  1181: 0000000000080700   5332 FUNC    LOCAL  DEFAULT       12 _int_malloc
  4537: 00000000000831d0    339 FUNC    LOCAL  DEFAULT       12 __GI___libc_malloc
  5545: 00000000000831d0    339 FUNC    LOCAL  DEFAULT       12 __malloc
  6063: 00000000000831d0    339 FUNC    GLOBAL DEFAULT       12 malloc
  7302: 00000000000831d0    339 FUNC    GLOBAL DEFAULT       12 __libc_malloc
 -----
As you an see, malloc and __libc_malloc have same address, and actually
__libc_malloc has an entry in debuginfo. So you can set up a probe on
__libc_malloc.

To fix this problem shortly, perf probe simply falls back to find probe
point(malloc) in symbols if it is not found in debuginfo.

Reported-by: David Ahern <dsahern@gmail.com>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: yrl.pp-manager.tt@hitachi.com
Link: http://lkml.kernel.org/r/20141231062747.2087.80961.stgit@localhost.localdomain
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/probe-event.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/probe-event.c b/tools/perf/util/probe-event.c
index 28eb1417cb2a..7f9b8632e433 100644
--- a/tools/perf/util/probe-event.c
+++ b/tools/perf/util/probe-event.c
@@ -495,9 +495,11 @@ static int try_to_find_probe_trace_events(struct perf_probe_event *pev,
 	}
 
 	if (ntevs == 0)	{	/* No error but failed to find probe point. */
-		pr_warning("Probe point '%s' not found.\n",
+		pr_warning("Probe point '%s' not found in debuginfo.\n",
 			   synthesize_perf_probe_point(&pev->point));
-		return -ENOENT;
+		if (need_dwarf)
+			return -ENOENT;
+		return 0;
 	}
 	/* Error path : ntevs < 0 */
 	pr_debug("An error occurred in debuginfo analysis (%d).\n", ntevs);
-- 
1.9.3

[PATCH 2/4] perf probe: Fix crash in dwarf_getcfi_elf

[Arnaldo Carvalho de Melo]

From: Namhyung Kim <namhyung@kernel.org>

David reported that perf can segfault when adding an uprobe event like
this:

  $ perf probe -x /lib64/libc-2.14.90.so -a 'malloc  size=%di'

  (gdb) bt
  #0  parse_eh_frame_hdr (hdr=0x0, hdr_size=2596, hdr_vaddr=71788,
      ehdr=0x7fffffffd390, eh_frame_vaddr=
      0x7fffffffd378, table_entries=0x8808d8, table_encoding=0x8808e0 "") at
      dwarf_getcfi_elf.c:79
  #1  0x000000385f81615a in getcfi_scn_eh_frame (hdr_vaddr=71788,
      hdr_scn=0x8839b0, shdr=0x7fffffffd2f0, scn=<optimized out>,
      ehdr=0x7fffffffd390, elf=0x882b30) at dwarf_getcfi_elf.c:231
  #2  getcfi_shdr (ehdr=0x7fffffffd390, elf=0x882b30) at dwarf_getcfi_elf.c:283
  #3  dwarf_getcfi_elf (elf=0x882b30) at dwarf_getcfi_elf.c:309
  #4  0x00000000004d5bac in debuginfo__find_probes (pf=0x7fffffffd4f0,
      dbg=Unhandled dwarf expression opcode 0xfa) at util/probe-finder.c:993
  #5  0x00000000004d634a in debuginfo__find_trace_events (dbg=0x880840,
      pev=<optimized out>, tevs=0x880f88, max_tevs=<optimized out>) at
      util/probe-finder.c:1200
  #6  0x00000000004aed6b in try_to_find_probe_trace_events (target=0x881b20
      "/lib64/libpthread-2.14.90.so",
      max_tevs=128, tevs=0x880f88, pev=0x859b30) at util/probe-event.c:482
  #7  convert_to_probe_trace_events (target=0x881b20
      "/lib64/libpthread-2.14.90.so", max_tevs=128, tevs=0x880f88,
      pev=0x859b30) at util/probe-event.c:2356
  #8  add_perf_probe_events (pevs=<optimized out>, npevs=1, max_tevs=128,
      target=0x881b20 "/lib64/libpthread-2.14.90.so", force_add=false) at
      util/probe-event.c:2391
  #9  0x000000000044014f in __cmd_probe (argc=<optimized out>,
      argv=0x7fffffffe2f0, prefix=Unhandled dwarf expression opcode 0xfa) at
      at builtin-probe.c:488
  #10 0x0000000000440313 in cmd_probe (argc=5, argv=0x7fffffffe2f0,
      prefix=<optimized out>) at builtin-probe.c:506
  #11 0x000000000041d133 in run_builtin (p=0x805680, argc=5,
      argv=0x7fffffffe2f0) at perf.c:341
  #12 0x000000000041c8b2 in handle_internal_command (argv=<optimized out>,
      argc=<optimized out>) at perf.c:400
  #13 run_argv (argv=<optimized out>, argcp=<optimized out>) at perf.c:444
  #14 main (argc=5, argv=0x7fffffffe2f0) at perf.c:559

And I found a related commit (5704c8c4fa71 "getcfi_scn_eh_frame: Don't
crash and burn when .eh_frame bits aren't there.") in elfutils that can
lead to a unexpected crash like this.  To safely use the function, it
needs to check the .eh_frame section is a PROGBITS type.

Reported-by: David Ahern <dsahern@gmail.com>
Tested-by: David Ahern <dsahern@gmail.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Acked-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Mark Wielaard <mjw@redhat.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Link: http://lkml.kernel.org/r/20141230090533.GH6081@sejong
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/probe-finder.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
index c7918f83b300..b5247d777f0e 100644
--- a/tools/perf/util/probe-finder.c
+++ b/tools/perf/util/probe-finder.c
@@ -989,8 +989,24 @@ static int debuginfo__find_probes(struct debuginfo *dbg,
 	int ret = 0;
 
 #if _ELFUTILS_PREREQ(0, 142)
+	Elf *elf;
+	GElf_Ehdr ehdr;
+	GElf_Shdr shdr;
+
 	/* Get the call frame information from this dwarf */
-	pf->cfi = dwarf_getcfi_elf(dwarf_getelf(dbg->dbg));
+	elf = dwarf_getelf(dbg->dbg);
+	if (elf == NULL)
+		return -EINVAL;
+
+	if (gelf_getehdr(elf, &ehdr) == NULL)
+		return -EINVAL;
+
+	if (elf_section_by_name(elf, &ehdr, &shdr, ".eh_frame", NULL) &&
+	    shdr.sh_type == SHT_PROGBITS) {
+		pf->cfi = dwarf_getcfi_elf(elf);
+	} else {
+		pf->cfi = dwarf_getcfi(dbg->dbg);
+	}
 #endif
 
 	off = 0;
-- 
1.9.3

[PATCH 3/4] perf list: Fix --raw-dump option

[Arnaldo Carvalho de Melo]

From: Taesoo Kim <tsgatesv@gmail.com>

Currently, 'perf list --raw-dump' requires extra arguments
(e.g., hw) to invoke, which breaks bash/zsh completion
(perf-completion.sh).

  $ perf list --raw-dump
    Error: unknown option `raw-dump'

     usage: perf list [hw|sw|cache|tracepoint|pmu|event_glob]

After,

  $ perf list --raw-dump
  cpu-cycles instructions cache-references cache-misses ...

Signed-off-by: Taesoo Kim <tsgatesv@gmail.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Taesoo kim <taesoo@gatech.edu>
Link: http://lkml.kernel.org/r/1419997015-11071-1-git-send-email-tsgatesv@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/builtin-list.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/tools/perf/builtin-list.c b/tools/perf/builtin-list.c
index 011195e38f21..198f3c3aff95 100644
--- a/tools/perf/builtin-list.c
+++ b/tools/perf/builtin-list.c
@@ -19,7 +19,9 @@
 int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
 {
 	int i;
-	const struct option list_options[] = {
+	bool raw_dump = false;
+	struct option list_options[] = {
+		OPT_BOOLEAN(0, "raw-dump", &raw_dump, "Dump raw events"),
 		OPT_END()
 	};
 	const char * const list_usage[] = {
@@ -27,11 +29,18 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
 		NULL
 	};
 
+	set_option_flag(list_options, 0, "raw-dump", PARSE_OPT_HIDDEN);
+
 	argc = parse_options(argc, argv, list_options, list_usage,
 			     PARSE_OPT_STOP_AT_NON_OPTION);
 
 	setup_pager();
 
+	if (raw_dump) {
+		print_events(NULL, true);
+		return 0;
+	}
+
 	if (argc == 0) {
 		print_events(NULL, false);
 		return 0;
@@ -53,8 +62,6 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)
 			print_hwcache_events(NULL, false);
 		else if (strcmp(argv[i], "pmu") == 0)
 			print_pmu_events(NULL, false);
-		else if (strcmp(argv[i], "--raw-dump") == 0)
-			print_events(NULL, true);
 		else {
 			char *sep = strchr(argv[i], ':'), *s;
 			int sep_idx;
-- 
1.9.3

[PATCH 4/4] perf diff: Fix to sort by baseline field by default

[Arnaldo Carvalho de Melo]

From: Namhyung Kim <namhyung@kernel.org>

The currently perf diff didn't add the baseline and delta (or other
compute) fields to the sort list so output will be sorted by other
fields like alphabetical order of DSO or symbol as below example.

Fix it by adding hpp formats for the fields and provides default compare
functions.

Before:

  $ perf diff
  # Event 'cycles'
  #
  # Baseline    Delta  Shared Object       Symbol
  # ........  .......  ..................  ...............................
  #
                       [bridge]            [k] ip_sabotage_in
                       [btrfs]             [k] __etree_search.constprop.47
       0.01%           [btrfs]             [k] btrfs_file_mmap
       0.01%   -0.01%  [btrfs]             [k] btrfs_getattr
                       [e1000e]            [k] e1000_watchdog
       0.00%           [kernel.vmlinux]    [k] PageHuge
       0.00%           [kernel.vmlinux]    [k] __acct_update_integrals
       0.00%           [kernel.vmlinux]    [k] __activate_page
                       [kernel.vmlinux]    [k] __alloc_fd
       0.02%   +0.02%  [kernel.vmlinux]    [k] __alloc_pages_nodemask
       ...

After:

  # Baseline    Delta  Shared Object       Symbol
  # ........  .......  ..................  ................................
  #
      24.73%   -4.62%  perf                [.] append_chain_children
       7.96%   -1.29%  perf                [.] dso__find_symbol
       6.97%   -2.07%  libc-2.20.so        [.] vfprintf
       4.61%   +0.88%  libc-2.20.so        [.] __fprintf_chk
       4.41%   +2.43%  perf                [.] sort__comm_cmp
       4.10%   -0.16%  perf                [.] comm__str
       4.03%   -0.93%  perf                [.] machine__findnew_thread_time
       3.82%   +3.09%  perf                [.] __hists__add_entry
       2.95%   -0.18%  perf                [.] sort__dso_cmp
       ...

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1419656793-32756-1-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/builtin-diff.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/tools/perf/builtin-diff.c b/tools/perf/builtin-diff.c
index 303c1e151dcf..1fd96c13f199 100644
--- a/tools/perf/builtin-diff.c
+++ b/tools/perf/builtin-diff.c
@@ -545,6 +545,42 @@ hist_entry__cmp_compute(struct hist_entry *left, struct hist_entry *right,
 	return __hist_entry__cmp_compute(p_left, p_right, c);
 }
 
+static int64_t
+hist_entry__cmp_nop(struct hist_entry *left __maybe_unused,
+		    struct hist_entry *right __maybe_unused)
+{
+	return 0;
+}
+
+static int64_t
+hist_entry__cmp_baseline(struct hist_entry *left, struct hist_entry *right)
+{
+	if (sort_compute)
+		return 0;
+
+	if (left->stat.period == right->stat.period)
+		return 0;
+	return left->stat.period > right->stat.period ? 1 : -1;
+}
+
+static int64_t
+hist_entry__cmp_delta(struct hist_entry *left, struct hist_entry *right)
+{
+	return hist_entry__cmp_compute(right, left, COMPUTE_DELTA);
+}
+
+static int64_t
+hist_entry__cmp_ratio(struct hist_entry *left, struct hist_entry *right)
+{
+	return hist_entry__cmp_compute(right, left, COMPUTE_RATIO);
+}
+
+static int64_t
+hist_entry__cmp_wdiff(struct hist_entry *left, struct hist_entry *right)
+{
+	return hist_entry__cmp_compute(right, left, COMPUTE_WEIGHTED_DIFF);
+}
+
 static void insert_hist_entry_by_compute(struct rb_root *root,
 					 struct hist_entry *he,
 					 int c)
@@ -1038,27 +1074,35 @@ static void data__hpp_register(struct data__file *d, int idx)
 	fmt->header = hpp__header;
 	fmt->width  = hpp__width;
 	fmt->entry  = hpp__entry_global;
+	fmt->cmp    = hist_entry__cmp_nop;
+	fmt->collapse = hist_entry__cmp_nop;
 
 	/* TODO more colors */
 	switch (idx) {
 	case PERF_HPP_DIFF__BASELINE:
 		fmt->color = hpp__color_baseline;
+		fmt->sort  = hist_entry__cmp_baseline;
 		break;
 	case PERF_HPP_DIFF__DELTA:
 		fmt->color = hpp__color_delta;
+		fmt->sort  = hist_entry__cmp_delta;
 		break;
 	case PERF_HPP_DIFF__RATIO:
 		fmt->color = hpp__color_ratio;
+		fmt->sort  = hist_entry__cmp_ratio;
 		break;
 	case PERF_HPP_DIFF__WEIGHTED_DIFF:
 		fmt->color = hpp__color_wdiff;
+		fmt->sort  = hist_entry__cmp_wdiff;
 		break;
 	default:
+		fmt->sort  = hist_entry__cmp_nop;
 		break;
 	}
 
 	init_header(d, dfmt);
 	perf_hpp__column_register(fmt);
+	perf_hpp__register_sort_field(fmt);
 }
 
 static void ui_init(void)
-- 
1.9.3

Re: [GIT PULL 0/4] perf/urgent fixes

[Ingo Molnar]

* Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> Hi Ingo,
> 
> 	Please consider pulling,
> 
> - Arnaldo
> 
> The following changes since commit 5b5e76218fbdbb71a01d5480f289ead624232876:
> 
>   Merge tag 'perf-urgent-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/urgent (2015-01-01 22:24:36 +0100)
> 
> are available in the git repository at:
> 
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-urgent-for-mingo
> 
> for you to fetch changes up to e7024fc3783317608b8e07048116a72a7d1cd26d:
> 
>   perf diff: Fix to sort by baseline field by default (2015-01-02 23:27:18 -0300)
> 
> ----------------------------------------------------------------
> perf/urgent fixes:
> 
> - 'perf probe' should fall back to find probe point in symbols when failing
>   to do so in a debuginfo file (Masami Hiramatsu)
> 
> - Fix 'perf probe' crash in dwarf_getcfi_elf (Namhyung Kim)
> 
> - Fix shell completion with 'perf list' --raw-dump option (Taesoo Kim)
> 
> - Fix 'perf diff' to sort by baseline field by default (Namhyung Kim)
> 
> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> 
> ----------------------------------------------------------------
> Masami Hiramatsu (1):
>       perf probe: Fix to fall back to find probe point in symbols
> 
> Namhyung Kim (2):
>       perf probe: Fix crash in dwarf_getcfi_elf
>       perf diff: Fix to sort by baseline field by default
> 
> Taesoo Kim (1):
>       perf list: Fix --raw-dump option
> 
>  tools/perf/builtin-diff.c      | 44 ++++++++++++++++++++++++++++++++++++++++++
>  tools/perf/builtin-list.c      | 13 ++++++++++---
>  tools/perf/util/probe-event.c  |  6 ++++--
>  tools/perf/util/probe-finder.c | 18 ++++++++++++++++-
>  4 files changed, 75 insertions(+), 6 deletions(-)

Pulled, thanks a lot Arnaldo!

	Ingo