[PATCHv2 00/14] perf tools: Adding xz decompression support

[PATCHv2 00/14] perf tools: Adding xz decompression support

[Jiri Olsa]

hi,
adding support to decompress kernel modules on Fedora 21
(lzma compression) vaguely continuing in previous RFC:
  http://marc.info/?l=linux-kernel&m=142261340327588&w=2

I'm basically extending he decompression framework Namhyung
introduced earlier plus other fixes for kernel modules dso
objects handling.

Last patch adds decompression support to annotate command, so
it now displays annotation for compressed kernel modules.

Current work available at:
  git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
  perf/lzma

NOTE It's rebased on 'perf/build' branch.

v2 changes:
  - added warning message for undetected lzma support (Arnaldo)
  - simplyfied code in decompress_kmodule (Namhyung)
  - renamed dsos__new to dsos__addnew (Arnaldo)
  - moved kmod_path__parse to dso.c object because of the python's
    perf.so compilation issue
  - added extra m.comp check and comment into map_groups__set_module_path

thanks,
jirka


Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
Jiri Olsa (14):
      perf tools: Remove superfluous thread->comm_set setting
      tools build: Add feature check for lzma library
      perf tools: Add lzma decompression support for kernel module
      perf tools: Add kmod_path__parse function
      perf tools: Add dsos__addnew function
      perf tools: Add machine__module_dso function
      perf tools: Use kmod_path__parse for machine__new_dso
      perf tools: Use kmod_path__parse in map_groups__set_modules_path_dir
      perf tools: Use kmod_path__parse in decompress_kmodule
      perf tools: Use kmod_path__parse in is_kernel_module
      perf tools: Remove compressed argument from is_kernel_module
      perf tools: Remove is_kmodule_extension function
      perf tools: Try to lookup kernel module map before creating one
      perf annotate: Allow annotation for decompressed kernel modules

 tools/build/Makefile.feature    |   6 +-
 tools/build/feature/Makefile    |   8 ++-
 tools/build/feature/test-all.c  |   5 ++
 tools/build/feature/test-lzma.c |  10 +++
 tools/perf/Makefile.perf        |   2 +
 tools/perf/config/Makefile      |  11 +++
 tools/perf/tests/Build          |   1 +
 tools/perf/tests/builtin-test.c |   4 ++
 tools/perf/tests/kmod-path.c    |  73 +++++++++++++++++++
 tools/perf/tests/tests.h        |   1 +
 tools/perf/util/Build           |   1 +
 tools/perf/util/annotate.c      |  32 ++++++++-
 tools/perf/util/dso.c           | 118 +++++++++++++++++++++---------
 tools/perf/util/dso.h           |  18 ++++-
 tools/perf/util/header.c        |   2 +-
 tools/perf/util/lzma.c          |  95 ++++++++++++++++++++++++
 tools/perf/util/machine.c       | 155 ++++++++++++++++++++++------------------
 tools/perf/util/symbol-elf.c    |  20 +++---
 tools/perf/util/thread.c        |   1 -
 tools/perf/util/util.h          |   4 ++
 20 files changed, 449 insertions(+), 118 deletions(-)
 create mode 100644 tools/build/feature/test-lzma.c
 create mode 100644 tools/perf/tests/kmod-path.c
 create mode 100644 tools/perf/util/lzma.c

[PATCH 01/14] perf tools: Remove superfluous thread->comm_set setting

[Jiri Olsa]

It is set by calling thread__set_comm right before
the removed line.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/util/thread.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/tools/perf/util/thread.c b/tools/perf/util/thread.c
index a5dbba95107f..1c8fbc9588c5 100644
--- a/tools/perf/util/thread.c
+++ b/tools/perf/util/thread.c
@@ -206,7 +206,6 @@ int thread__fork(struct thread *thread, struct thread *parent, u64 timestamp)
 		err = thread__set_comm(thread, comm, timestamp);
 		if (err)
 			return err;
-		thread->comm_set = true;
 	}
 
 	thread->ppid = parent->tid;
-- 
1.9.3

[PATCH 02/14] tools build: Add feature check for lzma library

[Jiri Olsa]

Will be used to decompress 'xz' objects. The check detects
the liblzma.so devel library normally delivered by xz package.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/build/Makefile.feature    |  6 ++++--
 tools/build/feature/Makefile    |  8 ++++++--
 tools/build/feature/test-all.c  |  5 +++++
 tools/build/feature/test-lzma.c | 10 ++++++++++
 4 files changed, 25 insertions(+), 4 deletions(-)
 create mode 100644 tools/build/feature/test-lzma.c

diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
index 12e58703a364..2975632d51e2 100644
--- a/tools/build/Makefile.feature
+++ b/tools/build/Makefile.feature
@@ -50,7 +50,8 @@ FEATURE_TESTS ?=			\
 	stackprotector-all		\
 	timerfd				\
 	libdw-dwarf-unwind		\
-	zlib
+	zlib				\
+	lzma
 
 FEATURE_DISPLAY ?=			\
 	dwarf				\
@@ -65,7 +66,8 @@ FEATURE_DISPLAY ?=			\
 	libslang			\
 	libunwind			\
 	libdw-dwarf-unwind		\
-	zlib
+	zlib				\
+	lzma
 
 # Set FEATURE_CHECK_(C|LD)FLAGS-all for all FEATURE_TESTS features.
 # If in the future we need per-feature checks/flags for features not
diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
index b6e2e312ba54..a52ff3ce527f 100644
--- a/tools/build/feature/Makefile
+++ b/tools/build/feature/Makefile
@@ -32,7 +32,8 @@ FILES=					\
 	test-libbabeltrace.bin		\
 	test-compile-32.bin		\
 	test-compile-x32.bin		\
-	test-zlib.bin
+	test-zlib.bin			\
+	test-lzma.bin
 
 CC := $(CROSS_COMPILE)gcc -MD
 PKG_CONFIG := $(CROSS_COMPILE)pkg-config
@@ -44,7 +45,7 @@ BUILD = $(CC) $(CFLAGS) -Wall -Werror -o $(OUTPUT)$@ $(patsubst %.bin,%.c,$@) $(
 ###############################
 
 test-all.bin:
-	$(BUILD) -fstack-protector-all -O2 -D_FORTIFY_SOURCE=2 -ldw -lelf -lnuma -lelf -laudit -I/usr/include/slang -lslang $(shell $(PKG_CONFIG) --libs --cflags gtk+-2.0 2>/dev/null) $(FLAGS_PERL_EMBED) $(FLAGS_PYTHON_EMBED) -DPACKAGE='"perf"' -lbfd -ldl -lz
+	$(BUILD) -fstack-protector-all -O2 -D_FORTIFY_SOURCE=2 -ldw -lelf -lnuma -lelf -laudit -I/usr/include/slang -lslang $(shell $(PKG_CONFIG) --libs --cflags gtk+-2.0 2>/dev/null) $(FLAGS_PERL_EMBED) $(FLAGS_PYTHON_EMBED) -DPACKAGE='"perf"' -lbfd -ldl -lz -llzma
 
 test-hello.bin:
 	$(BUILD)
@@ -151,6 +152,9 @@ test-compile-x32.bin:
 test-zlib.bin:
 	$(BUILD) -lz
 
+test-lzma.bin:
+	$(BUILD) -llzma
+
 -include *.d
 
 ###############################
diff --git a/tools/build/feature/test-all.c b/tools/build/feature/test-all.c
index 06f5c8a67cb2..577dfb430416 100644
--- a/tools/build/feature/test-all.c
+++ b/tools/build/feature/test-all.c
@@ -113,6 +113,10 @@
 #undef main
 #endif
 
+#define main main_test_lzma
+# include "test-lzma.c"
+#undef main
+
 int main(int argc, char *argv[])
 {
 	main_test_libpython();
@@ -138,6 +142,7 @@ int main(int argc, char *argv[])
 	main_test_sync_compare_and_swap(argc, argv);
 	main_test_zlib();
 	main_test_pthread_attr_setaffinity_np();
+	main_test_lzma();
 
 	return 0;
 }
diff --git a/tools/build/feature/test-lzma.c b/tools/build/feature/test-lzma.c
new file mode 100644
index 000000000000..95adc8ced3dd
--- /dev/null
+++ b/tools/build/feature/test-lzma.c
@@ -0,0 +1,10 @@
+#include <lzma.h>
+
+int main(void)
+{
+	lzma_stream strm = LZMA_STREAM_INIT;
+	int ret;
+
+	ret = lzma_stream_decoder(&strm, UINT64_MAX, LZMA_CONCATENATED);
+	return ret ? -1 : 0;
+}
-- 
1.9.3

[PATCH 03/14] perf tools: Add lzma decompression support for kernel module

[Jiri Olsa]

In short, Fedora compresses kernel modules now (since version 21)
with lzma compression.

Adding lzma decompress support into the dso.c:compressions array
introduced by Nmahyung earlier.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/Makefile.perf   |  2 +
 tools/perf/config/Makefile | 11 ++++++
 tools/perf/util/Build      |  1 +
 tools/perf/util/dso.c      |  3 ++
 tools/perf/util/lzma.c     | 95 ++++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/util.h     |  4 ++
 6 files changed, 116 insertions(+)
 create mode 100644 tools/perf/util/lzma.c

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index d5b9e0dae334..e9925e6ad1d0 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -71,6 +71,8 @@ include config/utilities.mak
 #
 # Define NO_LIBBABELTRACE if you do not want libbabeltrace support
 # for CTF data format.
+#
+# Define NO_LZMA if you do not want to support compressed (xz) kernel modules
 
 ifeq ($(srctree),)
 srctree := $(patsubst %/,%,$(dir $(shell pwd)))
diff --git a/tools/perf/config/Makefile b/tools/perf/config/Makefile
index 480631eccf50..f14ce6ddf887 100644
--- a/tools/perf/config/Makefile
+++ b/tools/perf/config/Makefile
@@ -541,6 +541,17 @@ ifndef NO_ZLIB
   endif
 endif
 
+ifndef NO_LZMA
+  ifeq ($(feature-lzma), 1)
+    CFLAGS += -DHAVE_LZMA_SUPPORT
+    EXTLIBS += -llzma
+    $(call detected,CONFIG_LZMA)
+  else
+    msg := $(warning No liblzma found, disables xz kernel module decompression, please install xz-devel/liblzma-dev);
+    NO_LZMA := 1
+  endif
+endif
+
 ifndef NO_BACKTRACE
   ifeq ($(feature-backtrace), 1)
     CFLAGS += -DHAVE_BACKTRACE_SUPPORT
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 972a6e0da7ad..797490a40075 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -94,6 +94,7 @@ libperf-y += scripting-engines/
 
 libperf-$(CONFIG_PERF_REGS) += perf_regs.o
 libperf-$(CONFIG_ZLIB) += zlib.o
+libperf-$(CONFIG_LZMA) += lzma.o
 
 CFLAGS_config.o   += -DETC_PERFCONFIG="BUILD_STR($(ETC_PERFCONFIG_SQ))"
 CFLAGS_exec_cmd.o += -DPERF_EXEC_PATH="BUILD_STR($(perfexecdir_SQ))" -DPREFIX="BUILD_STR($(prefix_SQ))"
diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index 814554d1b857..be368414036c 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -148,6 +148,9 @@ static const struct {
 #ifdef HAVE_ZLIB_SUPPORT
 	{ "gz", gzip_decompress_to_file },
 #endif
+#ifdef HAVE_LZMA_SUPPORT
+	{ "xz", lzma_decompress_to_file },
+#endif
 	{ NULL, NULL },
 };
 
diff --git a/tools/perf/util/lzma.c b/tools/perf/util/lzma.c
new file mode 100644
index 000000000000..95a1acb61245
--- /dev/null
+++ b/tools/perf/util/lzma.c
@@ -0,0 +1,95 @@
+#include <lzma.h>
+#include <stdio.h>
+#include <linux/compiler.h>
+#include "util.h"
+#include "debug.h"
+
+#define BUFSIZE 8192
+
+static const char *lzma_strerror(lzma_ret ret)
+{
+	switch ((int) ret) {
+	case LZMA_MEM_ERROR:
+		return "Memory allocation failed";
+	case LZMA_OPTIONS_ERROR:
+		return "Unsupported decompressor flags";
+	case LZMA_FORMAT_ERROR:
+		return "The input is not in the .xz format";
+	case LZMA_DATA_ERROR:
+		return "Compressed file is corrupt";
+	case LZMA_BUF_ERROR:
+		return "Compressed file is truncated or otherwise corrupt";
+	default:
+		return "Unknown error, possibly a bug";
+	}
+}
+
+int lzma_decompress_to_file(const char *input, int output_fd)
+{
+	lzma_action action = LZMA_RUN;
+	lzma_stream strm   = LZMA_STREAM_INIT;
+	lzma_ret ret;
+
+	u8 buf_in[BUFSIZE];
+	u8 buf_out[BUFSIZE];
+	FILE *infile;
+
+	infile = fopen(input, "rb");
+	if (!infile) {
+		pr_err("lzma: fopen failed on %s: '%s'\n",
+		       input, strerror(errno));
+		return -1;
+	}
+
+	ret = lzma_stream_decoder(&strm, UINT64_MAX, LZMA_CONCATENATED);
+	if (ret != LZMA_OK) {
+		pr_err("lzma: lzma_stream_decoder failed %s (%d)\n",
+			lzma_strerror(ret), ret);
+		return -1;
+	}
+
+	strm.next_in   = NULL;
+	strm.avail_in  = 0;
+	strm.next_out  = buf_out;
+	strm.avail_out = sizeof(buf_out);
+
+	while (1) {
+		if (strm.avail_in == 0 && !feof(infile)) {
+			strm.next_in  = buf_in;
+			strm.avail_in = fread(buf_in, 1, sizeof(buf_in), infile);
+
+			if (ferror(infile)) {
+				pr_err("lzma: read error: %s\n", strerror(errno));
+				return -1;
+			}
+
+			if (feof(infile))
+				action = LZMA_FINISH;
+		}
+
+		ret = lzma_code(&strm, action);
+
+		if (strm.avail_out == 0 || ret == LZMA_STREAM_END) {
+			ssize_t write_size = sizeof(buf_out) - strm.avail_out;
+
+			if (writen(output_fd, buf_out, write_size) != write_size) {
+				pr_err("lzma: write error: %s\n", strerror(errno));
+				return -1;
+			}
+
+			strm.next_out  = buf_out;
+			strm.avail_out = sizeof(buf_out);
+		}
+
+		if (ret != LZMA_OK) {
+			if (ret == LZMA_STREAM_END)
+				return 0;
+
+			pr_err("lzma: failed %s\n", lzma_strerror(ret));
+			return -1;
+		}
+	}
+
+	fclose(infile);
+	return 0;
+}
diff --git a/tools/perf/util/util.h b/tools/perf/util/util.h
index fbd598afc606..1ff23e04ad27 100644
--- a/tools/perf/util/util.h
+++ b/tools/perf/util/util.h
@@ -329,4 +329,8 @@ bool find_process(const char *name);
 int gzip_decompress_to_file(const char *input, int output_fd);
 #endif
 
+#ifdef HAVE_LZMA_SUPPORT
+int lzma_decompress_to_file(const char *input, int output_fd);
+#endif
+
 #endif /* GIT_COMPAT_UTIL_H */
-- 
1.9.3

[PATCH 04/14] perf tools: Add kmod_path__parse function

[Jiri Olsa]

Provides united way of parsing kernel module path
into several components.

The new kmod_path__parse function and few defines:

  int __kmod_path__parse(struct kmod_path *m, const char *path,
                         bool alloc_name, bool alloc_ext);

  #define kmod_path__parse(__m, __p)      __kmod_path__parse(__m, __p, false, false)
  #define kmod_path__parse_name(__m, __p) __kmod_path__parse(__m, __p, true , false)
  #define kmod_path__parse_ext(__m, __p)  __kmod_path__parse(__m, __p, false, true)

parse kernel module @path and updates @m argument like:

  @comp - true if @path contains supported compression suffix,
          false otherwise
  @kmod - true if @path contains '.ko' suffix in right position,
          false otherwise
  @name - if (@alloc_name && @kmod) is true, it contains strdup-ed base name
          of the kernel module without suffixes, otherwise strudup-ed
          base name of @path
  @ext  - if (@alloc_ext && @comp) is true, it contains strdup-ed string
          the compression suffix

It returns 0 if there's no strdup error, -ENOMEM otherwise.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/tests/Build          |  1 +
 tools/perf/tests/builtin-test.c |  4 +++
 tools/perf/tests/kmod-path.c    | 73 +++++++++++++++++++++++++++++++++++++++++
 tools/perf/tests/tests.h        |  1 +
 tools/perf/util/dso.c           | 66 +++++++++++++++++++++++++++++++++++++
 tools/perf/util/dso.h           | 14 ++++++++
 6 files changed, 159 insertions(+)
 create mode 100644 tools/perf/tests/kmod-path.c

diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index 2de01a4b4084..6a8801b32017 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -30,6 +30,7 @@ perf-y += keep-tracking.o
 perf-y += code-reading.o
 perf-y += sample-parsing.o
 perf-y += parse-no-sample-id-all.o
+perf-y += kmod-path.o
 
 perf-$(CONFIG_X86) += perf-time-to-tsc.o
 
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index 4b7d9ab0f049..31f9959c04c8 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -167,6 +167,10 @@ static struct test {
 		.func = test__fdarray__add,
 	},
 	{
+		.desc = "Test kmod_path__parse function",
+		.func = test__kmod_path__parse,
+	},
+	{
 		.func = NULL,
 	},
 };
diff --git a/tools/perf/tests/kmod-path.c b/tools/perf/tests/kmod-path.c
new file mode 100644
index 000000000000..e8d7cbb9320c
--- /dev/null
+++ b/tools/perf/tests/kmod-path.c
@@ -0,0 +1,73 @@
+#include <stdbool.h>
+#include "tests.h"
+#include "dso.h"
+#include "debug.h"
+
+static int test(const char *path, bool alloc_name, bool alloc_ext,
+		bool kmod, bool comp, const char *name, const char *ext)
+{
+	struct kmod_path m;
+
+	memset(&m, 0x0, sizeof(m));
+
+	TEST_ASSERT_VAL("kmod_path__parse",
+			!__kmod_path__parse(&m, path, alloc_name, alloc_ext));
+
+	pr_debug("%s - alloc name %d, alloc ext %d, kmod %d, comp %d, name '%s', ext '%s'\n",
+		 path, alloc_name, alloc_ext, m.kmod, m.comp, m.name, m.ext);
+
+	TEST_ASSERT_VAL("wrong kmod", m.kmod == kmod);
+	TEST_ASSERT_VAL("wrong comp", m.comp == comp);
+
+	if (ext)
+		TEST_ASSERT_VAL("wrong ext", m.ext && !strcmp(ext, m.ext));
+	else
+		TEST_ASSERT_VAL("wrong ext", !m.ext);
+
+	if (name)
+		TEST_ASSERT_VAL("wrong name", m.name && !strcmp(name, m.name));
+	else
+		TEST_ASSERT_VAL("wrong name", !m.name);
+
+	free(m.name);
+	free(m.ext);
+	return 0;
+}
+
+#define T(path, an, ae, k, c, n, e) \
+	TEST_ASSERT_VAL("failed", !test(path, an, ae, k, c, n, e))
+
+int test__kmod_path__parse(void)
+{
+	/* path                alloc_name  alloc_ext   kmod  comp   name     ext */
+	T("/xxxx/xxxx/x-x.ko", true      , true      , true, false, "[x_x]", NULL);
+	T("/xxxx/xxxx/x-x.ko", false     , true      , true, false, NULL   , NULL);
+	T("/xxxx/xxxx/x-x.ko", true      , false     , true, false, "[x_x]", NULL);
+	T("/xxxx/xxxx/x-x.ko", false     , false     , true, false, NULL   , NULL);
+
+	/* path                alloc_name  alloc_ext   kmod  comp  name   ext */
+	T("/xxxx/xxxx/x.ko.gz", true     , true      , true, true, "[x]", "gz");
+	T("/xxxx/xxxx/x.ko.gz", false    , true      , true, true, NULL , "gz");
+	T("/xxxx/xxxx/x.ko.gz", true     , false     , true, true, "[x]", NULL);
+	T("/xxxx/xxxx/x.ko.gz", false    , false     , true, true, NULL , NULL);
+
+	/* path              alloc_name  alloc_ext  kmod   comp  name    ext */
+	T("/xxxx/xxxx/x.gz", true      , true     , false, true, "x.gz" ,"gz");
+	T("/xxxx/xxxx/x.gz", false     , true     , false, true, NULL   ,"gz");
+	T("/xxxx/xxxx/x.gz", true      , false    , false, true, "x.gz" , NULL);
+	T("/xxxx/xxxx/x.gz", false     , false    , false, true, NULL   , NULL);
+
+	/* path   alloc_name  alloc_ext  kmod   comp  name     ext */
+	T("x.gz", true      , true     , false, true, "x.gz", "gz");
+	T("x.gz", false     , true     , false, true, NULL  , "gz");
+	T("x.gz", true      , false    , false, true, "x.gz", NULL);
+	T("x.gz", false     , false    , false, true, NULL  , NULL);
+
+	/* path      alloc_name  alloc_ext  kmod  comp  name  ext */
+	T("x.ko.gz", true      , true     , true, true, "[x]", "gz");
+	T("x.ko.gz", false     , true     , true, true, NULL , "gz");
+	T("x.ko.gz", true      , false    , true, true, "[x]", NULL);
+	T("x.ko.gz", false     , false    , true, true, NULL , NULL);
+
+	return 0;
+}
diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
index 00e776a87a9c..52758a33f64c 100644
--- a/tools/perf/tests/tests.h
+++ b/tools/perf/tests/tests.h
@@ -51,6 +51,7 @@ int test__hists_cumulate(void);
 int test__switch_tracking(void);
 int test__fdarray__filter(void);
 int test__fdarray__add(void);
+int test__kmod_path__parse(void);
 
 #if defined(__x86_64__) || defined(__i386__) || defined(__arm__)
 #ifdef HAVE_DWARF_UNWIND_SUPPORT
diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index be368414036c..3e6863feb066 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -212,6 +212,72 @@ bool dso__needs_decompress(struct dso *dso)
 }
 
 /*
+ * Parses kernel module specified in @path and updates
+ * @m argument like:
+ *
+ *    @comp - true if @path contains supported compression suffix,
+ *            false otherwise
+ *    @kmod - true if @path contains '.ko' suffix in right position,
+ *            false otherwise
+ *    @name - if (@alloc_name && @kmod) is true, it contains strdup-ed base name
+ *            of the kernel module without suffixes, otherwise strudup-ed
+ *            base name of @path
+ *    @ext  - if (@alloc_ext && @comp) is true, it contains strdup-ed string
+ *            the compression suffix
+ *
+ * Returns 0 if there's no strdup error, -ENOMEM otherwise.
+ */
+int __kmod_path__parse(struct kmod_path *m, const char *path,
+		       bool alloc_name, bool alloc_ext)
+{
+	const char *name = strrchr(path, '/');
+	const char *ext  = strrchr(path, '.');
+
+	memset(m, 0x0, sizeof(*m));
+	name = name ? name + 1 : path;
+
+	/* No extension, just return name. */
+	if (ext == NULL) {
+		if (alloc_name) {
+			m->name = strdup(name);
+			return m->name ? 0 : -ENOMEM;
+		}
+		return 0;
+	}
+
+	if (is_supported_compression(ext + 1)) {
+		m->comp = true;
+		ext -= 3;
+	}
+
+	/* Check .ko extension only if there's enough name left. */
+	if (ext > name)
+		m->kmod = !strncmp(ext, ".ko", 3);
+
+	if (alloc_name) {
+		if (m->kmod) {
+			if (asprintf(&m->name, "[%.*s]", (int) (ext - name), name) == -1)
+				return -ENOMEM;
+		} else {
+			if (asprintf(&m->name, "%s", name) == -1)
+				return -ENOMEM;
+		}
+
+		strxfrchar(m->name, '-', '_');
+	}
+
+	if (alloc_ext && m->comp) {
+		m->ext = strdup(ext + 4);
+		if (!m->ext) {
+			free((void *) m->name);
+			return -ENOMEM;
+		}
+	}
+
+	return 0;
+}
+
+/*
  * Global list of open DSOs and the counter.
  */
 static LIST_HEAD(dso__data_open);
diff --git a/tools/perf/util/dso.h b/tools/perf/util/dso.h
index ced92841ff97..6c0255572e04 100644
--- a/tools/perf/util/dso.h
+++ b/tools/perf/util/dso.h
@@ -194,6 +194,20 @@ bool is_kernel_module(const char *pathname, bool *compressed);
 bool decompress_to_file(const char *ext, const char *filename, int output_fd);
 bool dso__needs_decompress(struct dso *dso);
 
+struct kmod_path {
+	char *name;
+	char *ext;
+	bool  comp;
+	bool  kmod;
+};
+
+int __kmod_path__parse(struct kmod_path *m, const char *path,
+		     bool alloc_name, bool alloc_ext);
+
+#define kmod_path__parse(__m, __p)      __kmod_path__parse(__m, __p, false, false)
+#define kmod_path__parse_name(__m, __p) __kmod_path__parse(__m, __p, true , false)
+#define kmod_path__parse_ext(__m, __p)  __kmod_path__parse(__m, __p, false, true)
+
 /*
  * The dso__data_* external interface provides following functions:
  *   dso__data_fd
-- 
1.9.3

[PATCH 05/14] perf tools: Add dsos__addnew function

[Jiri Olsa]

Separate the creation of new dso object and its addition
to the dsos list. It will be used in following patch.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/util/dso.c | 21 ++++++++++++---------
 tools/perf/util/dso.h |  1 +
 2 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index 3e6863feb066..7a7c54b42b41 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -1071,21 +1071,24 @@ struct dso *dsos__find(const struct dsos *dsos, const char *name,
 	return dso__find_by_longname(&dsos->root, name);
 }
 
-struct dso *__dsos__findnew(struct dsos *dsos, const char *name)
+struct dso *dsos__addnew(struct dsos *dsos, const char *name)
 {
-	struct dso *dso = dsos__find(dsos, name, false);
+	struct dso *dso = dso__new(name);
 
-	if (!dso) {
-		dso = dso__new(name);
-		if (dso != NULL) {
-			dsos__add(dsos, dso);
-			dso__set_basename(dso);
-		}
+	if (dso != NULL) {
+		dsos__add(dsos, dso);
+		dso__set_basename(dso);
 	}
-
 	return dso;
 }
 
+struct dso *__dsos__findnew(struct dsos *dsos, const char *name)
+{
+	struct dso *dso = dsos__find(dsos, name, false);
+
+	return dso ? dso : dsos__addnew(dsos, name);
+}
+
 size_t __dsos__fprintf_buildid(struct list_head *head, FILE *fp,
 			       bool (skip)(struct dso *dso, int parm), int parm)
 {
diff --git a/tools/perf/util/dso.h b/tools/perf/util/dso.h
index 6c0255572e04..dd1ce562419b 100644
--- a/tools/perf/util/dso.h
+++ b/tools/perf/util/dso.h
@@ -263,6 +263,7 @@ struct dso *dso__kernel_findnew(struct machine *machine, const char *name,
 				const char *short_name, int dso_type);
 
 void dsos__add(struct dsos *dsos, struct dso *dso);
+struct dso *dsos__addnew(struct dsos *dsos, const char *name);
 struct dso *dsos__find(const struct dsos *dsos, const char *name,
 		       bool cmp_short);
 struct dso *__dsos__findnew(struct dsos *dsos, const char *name);
-- 
1.9.3

[PATCH 06/14] perf tools: Add machine__module_dso function

[Jiri Olsa]

Separate the dso object addition and update when adding
new kernel module.

Currently we update dso's symtab_type any time we find
it in the list, because we can't distinguish between
new and found dso from __dsos__findnew function.

Adding machine__module_dso that separates finding and adding
new dso objects, so there's no superfluous update of dso.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/util/machine.c | 36 +++++++++++++++++++++++++-----------
 1 file changed, 25 insertions(+), 11 deletions(-)

diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
index 24f8c978cfd4..dbb5f0367005 100644
--- a/tools/perf/util/machine.c
+++ b/tools/perf/util/machine.c
@@ -460,12 +460,35 @@ int machine__process_lost_event(struct machine *machine __maybe_unused,
 	return 0;
 }
 
+static struct dso *machine__module_dso(struct machine *machine, const char *filename)
+{
+	struct dso *dso;
+	bool compressed;
+
+	dso = dsos__find(&machine->kernel_dsos, filename, false);
+	if (!dso) {
+		dso = dsos__addnew(&machine->kernel_dsos, filename);
+		if (dso == NULL)
+			return NULL;
+
+		if (machine__is_host(machine))
+			dso->symtab_type = DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE;
+		else
+			dso->symtab_type = DSO_BINARY_TYPE__GUEST_KMODULE;
+
+		/* _KMODULE_COMP should be next to _KMODULE */
+		if (is_kernel_module(filename, &compressed) && compressed)
+			dso->symtab_type++;
+	}
+
+	return dso;
+}
+
 struct map *machine__new_module(struct machine *machine, u64 start,
 				const char *filename)
 {
 	struct map *map;
-	struct dso *dso = __dsos__findnew(&machine->kernel_dsos, filename);
-	bool compressed;
+	struct dso *dso = machine__module_dso(machine, filename);
 
 	if (dso == NULL)
 		return NULL;
@@ -474,15 +497,6 @@ struct map *machine__new_module(struct machine *machine, u64 start,
 	if (map == NULL)
 		return NULL;
 
-	if (machine__is_host(machine))
-		dso->symtab_type = DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE;
-	else
-		dso->symtab_type = DSO_BINARY_TYPE__GUEST_KMODULE;
-
-	/* _KMODULE_COMP should be next to _KMODULE */
-	if (is_kernel_module(filename, &compressed) && compressed)
-		dso->symtab_type++;
-
 	map_groups__insert(&machine->kmaps, map);
 	return map;
 }
-- 
1.9.3

[PATCH 07/14] perf tools: Use kmod_path__parse for machine__new_dso

[Jiri Olsa]

Using kmod_path__parse to get the module name and update
the dso short name within machine__new_dso function.

This way it's done only first time when dso is created,
unlike the current way when we update it all the time
we process memory map of the kernel module.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/util/machine.c | 59 +++++++++++++++++------------------------------
 1 file changed, 21 insertions(+), 38 deletions(-)

diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
index dbb5f0367005..eb95b883fb44 100644
--- a/tools/perf/util/machine.c
+++ b/tools/perf/util/machine.c
@@ -460,14 +460,15 @@ int machine__process_lost_event(struct machine *machine __maybe_unused,
 	return 0;
 }
 
-static struct dso *machine__module_dso(struct machine *machine, const char *filename)
+static struct dso*
+machine__module_dso(struct machine *machine, struct kmod_path *m,
+		    const char *filename)
 {
 	struct dso *dso;
-	bool compressed;
 
-	dso = dsos__find(&machine->kernel_dsos, filename, false);
+	dso = dsos__find(&machine->kernel_dsos, m->name, true);
 	if (!dso) {
-		dso = dsos__addnew(&machine->kernel_dsos, filename);
+		dso = dsos__addnew(&machine->kernel_dsos, m->name);
 		if (dso == NULL)
 			return NULL;
 
@@ -477,8 +478,11 @@ static struct dso *machine__module_dso(struct machine *machine, const char *file
 			dso->symtab_type = DSO_BINARY_TYPE__GUEST_KMODULE;
 
 		/* _KMODULE_COMP should be next to _KMODULE */
-		if (is_kernel_module(filename, &compressed) && compressed)
+		if (m->kmod && m->comp)
 			dso->symtab_type++;
+
+		dso__set_short_name(dso, strdup(m->name), true);
+		dso__set_long_name(dso, strdup(filename), true);
 	}
 
 	return dso;
@@ -487,17 +491,25 @@ static struct dso *machine__module_dso(struct machine *machine, const char *file
 struct map *machine__new_module(struct machine *machine, u64 start,
 				const char *filename)
 {
-	struct map *map;
-	struct dso *dso = machine__module_dso(machine, filename);
+	struct map *map = NULL;
+	struct dso *dso;
+	struct kmod_path m;
 
-	if (dso == NULL)
+	if (kmod_path__parse_name(&m, filename))
 		return NULL;
 
+	dso = machine__module_dso(machine, &m, filename);
+	if (dso == NULL)
+		goto out;
+
 	map = map__new2(start, dso, MAP__FUNCTION);
 	if (map == NULL)
-		return NULL;
+		goto out;
 
 	map_groups__insert(&machine->kmaps, map);
+
+out:
+	free(m.name);
 	return map;
 }
 
@@ -1058,40 +1070,11 @@ static int machine__process_kernel_mmap_event(struct machine *machine,
 				strlen(kmmap_prefix) - 1) == 0;
 	if (event->mmap.filename[0] == '/' ||
 	    (!is_kernel_mmap && event->mmap.filename[0] == '[')) {
-
-		char short_module_name[1024];
-		char *name, *dot;
-
-		if (event->mmap.filename[0] == '/') {
-			name = strrchr(event->mmap.filename, '/');
-			if (name == NULL)
-				goto out_problem;
-
-			++name; /* skip / */
-			dot = strrchr(name, '.');
-			if (dot == NULL)
-				goto out_problem;
-			/* On some system, modules are compressed like .ko.gz */
-			if (is_supported_compression(dot + 1))
-				dot -= 3;
-			if (!is_kmodule_extension(dot + 1))
-				goto out_problem;
-			snprintf(short_module_name, sizeof(short_module_name),
-					"[%.*s]", (int)(dot - name), name);
-			strxfrchar(short_module_name, '-', '_');
-		} else
-			strcpy(short_module_name, event->mmap.filename);
-
 		map = machine__new_module(machine, event->mmap.start,
 					  event->mmap.filename);
 		if (map == NULL)
 			goto out_problem;
 
-		name = strdup(short_module_name);
-		if (name == NULL)
-			goto out_problem;
-
-		dso__set_short_name(map->dso, name, true);
 		map->end = map->start + event->mmap.len;
 	} else if (is_kernel_mmap) {
 		const char *symbol_name = (event->mmap.filename +
-- 
1.9.3

[PATCH 08/14] perf tools: Use kmod_path__parse in map_groups__set_modules_path_dir

[Jiri Olsa]

Replacing the file name parsing with kmod_path__parse
and moving the dso update into new separate function.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/util/machine.c | 65 ++++++++++++++++++++++++++++++-----------------
 1 file changed, 41 insertions(+), 24 deletions(-)

diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
index eb95b883fb44..6ca61a3427a6 100644
--- a/tools/perf/util/machine.c
+++ b/tools/perf/util/machine.c
@@ -851,6 +851,39 @@ static char *get_kernel_version(const char *root_dir)
 	return strdup(name);
 }
 
+static bool is_kmod_dso(struct dso *dso)
+{
+	return dso->symtab_type == DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE ||
+	       dso->symtab_type == DSO_BINARY_TYPE__GUEST_KMODULE;
+}
+
+static int map_groups__set_module_path(struct map_groups *mg, const char *path,
+				       struct kmod_path *m)
+{
+	struct map *map;
+	char *long_name;
+
+	map = map_groups__find_by_name(mg, MAP__FUNCTION, m->name);
+	if (map == NULL)
+		return 0;
+
+	long_name = strdup(path);
+	if (long_name == NULL)
+		return -ENOMEM;
+
+	dso__set_long_name(map->dso, long_name, true);
+	dso__kernel_module_get_build_id(map->dso, "");
+
+	/*
+	 * Full name could reveal us kmod compression, so
+	 * we need to update the symtab_type if needed.
+	 */
+	if (m->comp && is_kmod_dso(map->dso))
+		map->dso->symtab_type++;
+
+	return 0;
+}
+
 static int map_groups__set_modules_path_dir(struct map_groups *mg,
 				const char *dir_name, int depth)
 {
@@ -889,35 +922,19 @@ static int map_groups__set_modules_path_dir(struct map_groups *mg,
 			if (ret < 0)
 				goto out;
 		} else {
-			char *dot = strrchr(dent->d_name, '.'),
-			     dso_name[PATH_MAX];
-			struct map *map;
-			char *long_name;
+			struct kmod_path m;
 
-			if (dot == NULL)
-				continue;
+			ret = kmod_path__parse_name(&m, dent->d_name);
+			if (ret)
+				goto out;
 
-			/* On some system, modules are compressed like .ko.gz */
-			if (is_supported_compression(dot + 1) &&
-			    is_kmodule_extension(dot - 2))
-				dot -= 3;
+			if (m.kmod)
+				ret = map_groups__set_module_path(mg, path, &m);
 
-			snprintf(dso_name, sizeof(dso_name), "[%.*s]",
-				 (int)(dot - dent->d_name), dent->d_name);
+			free(m.name);
 
-			strxfrchar(dso_name, '-', '_');
-			map = map_groups__find_by_name(mg, MAP__FUNCTION,
-						       dso_name);
-			if (map == NULL)
-				continue;
-
-			long_name = strdup(path);
-			if (long_name == NULL) {
-				ret = -1;
+			if (ret)
 				goto out;
-			}
-			dso__set_long_name(map->dso, long_name, true);
-			dso__kernel_module_get_build_id(map->dso, "");
 		}
 	}
 
-- 
1.9.3

[PATCH 09/14] perf tools: Use kmod_path__parse in decompress_kmodule

[Jiri Olsa]

Replacing the file name parsing with kmod_path__parse.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/util/symbol-elf.c | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
index ada16762fac2..f060a7f98c37 100644
--- a/tools/perf/util/symbol-elf.c
+++ b/tools/perf/util/symbol-elf.c
@@ -579,32 +579,34 @@ static int dso__swap_init(struct dso *dso, unsigned char eidata)
 static int decompress_kmodule(struct dso *dso, const char *name,
 			      enum dso_binary_type type)
 {
-	int fd;
-	const char *ext = strrchr(name, '.');
+	int fd = -1;
 	char tmpbuf[] = "/tmp/perf-kmod-XXXXXX";
+	struct kmod_path m;
 
 	if (type != DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP &&
 	    type != DSO_BINARY_TYPE__GUEST_KMODULE_COMP &&
 	    type != DSO_BINARY_TYPE__BUILD_ID_CACHE)
 		return -1;
 
-	if (!ext || !is_supported_compression(ext + 1)) {
-		ext = strrchr(dso->name, '.');
-		if (!ext || !is_supported_compression(ext + 1))
-			return -1;
-	}
+	if (type == DSO_BINARY_TYPE__BUILD_ID_CACHE)
+		name = dso->long_name;
+
+	if (kmod_path__parse_ext(&m, name) || !m.comp)
+		return -1;
 
 	fd = mkstemp(tmpbuf);
 	if (fd < 0)
-		return -1;
+		goto out;
 
-	if (!decompress_to_file(ext + 1, name, fd)) {
+	if (!decompress_to_file(m.ext, name, fd)) {
 		close(fd);
 		fd = -1;
 	}
 
 	unlink(tmpbuf);
 
+out:
+	free(m.ext);
 	return fd;
 }
 
-- 
1.9.3

[PATCH 10/14] perf tools: Use kmod_path__parse in is_kernel_module

[Jiri Olsa]

Replacing the current parsing code with kmod_path__parse
function call.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/util/dso.c | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)

diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index 7a7c54b42b41..f37548322c9d 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -178,19 +178,15 @@ bool is_kmodule_extension(const char *ext)
 
 bool is_kernel_module(const char *pathname, bool *compressed)
 {
-	const char *ext = strrchr(pathname, '.');
+	struct kmod_path m;
 
-	if (ext == NULL)
-		return false;
+	if (kmod_path__parse(&m, pathname))
+		return NULL;
 
-	if (is_supported_compression(ext + 1)) {
-		if (compressed)
-			*compressed = true;
-		ext -= 3;
-	} else if (compressed)
-		*compressed = false;
+	if (compressed)
+		*compressed = m.comp;
 
-	return is_kmodule_extension(ext + 1);
+	return m.kmod;
 }
 
 bool decompress_to_file(const char *ext, const char *filename, int output_fd)
-- 
1.9.3

[PATCH 11/14] perf tools: Remove compressed argument from is_kernel_module

[Jiri Olsa]

We no longer need the 'compressed' argument, because all
current users use 'NULL' for it.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/util/dso.c     | 5 +----
 tools/perf/util/dso.h     | 2 +-
 tools/perf/util/header.c  | 2 +-
 tools/perf/util/machine.c | 2 +-
 4 files changed, 4 insertions(+), 7 deletions(-)

diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index f37548322c9d..8803a69fc260 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -176,16 +176,13 @@ bool is_kmodule_extension(const char *ext)
 	return false;
 }
 
-bool is_kernel_module(const char *pathname, bool *compressed)
+bool is_kernel_module(const char *pathname)
 {
 	struct kmod_path m;
 
 	if (kmod_path__parse(&m, pathname))
 		return NULL;
 
-	if (compressed)
-		*compressed = m.comp;
-
 	return m.kmod;
 }
 
diff --git a/tools/perf/util/dso.h b/tools/perf/util/dso.h
index dd1ce562419b..bdfca4d58356 100644
--- a/tools/perf/util/dso.h
+++ b/tools/perf/util/dso.h
@@ -190,7 +190,7 @@ int dso__read_binary_type_filename(const struct dso *dso, enum dso_binary_type t
 				   char *root_dir, char *filename, size_t size);
 bool is_supported_compression(const char *ext);
 bool is_kmodule_extension(const char *ext);
-bool is_kernel_module(const char *pathname, bool *compressed);
+bool is_kernel_module(const char *pathname);
 bool decompress_to_file(const char *ext, const char *filename, int output_fd);
 bool dso__needs_decompress(struct dso *dso);
 
diff --git a/tools/perf/util/header.c b/tools/perf/util/header.c
index 1f407f7352a7..fb432153e2aa 100644
--- a/tools/perf/util/header.c
+++ b/tools/perf/util/header.c
@@ -1266,7 +1266,7 @@ static int __event_process_build_id(struct build_id_event *bev,
 
 		dso__set_build_id(dso, &bev->build_id);
 
-		if (!is_kernel_module(filename, NULL))
+		if (!is_kernel_module(filename))
 			dso->kernel = dso_type;
 
 		build_id__sprintf(dso->build_id, sizeof(dso->build_id),
diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
index 6ca61a3427a6..1de5438ad070 100644
--- a/tools/perf/util/machine.c
+++ b/tools/perf/util/machine.c
@@ -1104,7 +1104,7 @@ static int machine__process_kernel_mmap_event(struct machine *machine,
 		struct dso *dso;
 
 		list_for_each_entry(dso, &machine->kernel_dsos.head, node) {
-			if (is_kernel_module(dso->long_name, NULL))
+			if (is_kernel_module(dso->long_name))
 				continue;
 
 			kernel = dso;
-- 
1.9.3

[PATCH 12/14] perf tools: Remove is_kmodule_extension function

[Jiri Olsa]

Because it's no longer needed.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/util/dso.c | 11 -----------
 tools/perf/util/dso.h |  1 -
 2 files changed, 12 deletions(-)

diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index 8803a69fc260..0d3667f92023 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -165,17 +165,6 @@ bool is_supported_compression(const char *ext)
 	return false;
 }
 
-bool is_kmodule_extension(const char *ext)
-{
-	if (strncmp(ext, "ko", 2))
-		return false;
-
-	if (ext[2] == '\0' || (ext[2] == '.' && is_supported_compression(ext+3)))
-		return true;
-
-	return false;
-}
-
 bool is_kernel_module(const char *pathname)
 {
 	struct kmod_path m;
diff --git a/tools/perf/util/dso.h b/tools/perf/util/dso.h
index bdfca4d58356..c95a3be41f69 100644
--- a/tools/perf/util/dso.h
+++ b/tools/perf/util/dso.h
@@ -189,7 +189,6 @@ char dso__symtab_origin(const struct dso *dso);
 int dso__read_binary_type_filename(const struct dso *dso, enum dso_binary_type type,
 				   char *root_dir, char *filename, size_t size);
 bool is_supported_compression(const char *ext);
-bool is_kmodule_extension(const char *ext);
 bool is_kernel_module(const char *pathname);
 bool decompress_to_file(const char *ext, const char *filename, int output_fd);
 bool dso__needs_decompress(struct dso *dso);
-- 
1.9.3

[PATCH 13/14] perf tools: Try to lookup kernel module map before creating one

[Jiri Olsa]

Currently we assume machine__new_module is called only once
for each module so we create its map&dso unconditionally.

However it's possible that it's called multiple times for
same module. Like for perf record:
  1) via machine__create_module during machine init
  2) via kernel MMAP event processing

Trying to lookup kernel module map before creating one.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/util/machine.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
index 1de5438ad070..e3353307330c 100644
--- a/tools/perf/util/machine.c
+++ b/tools/perf/util/machine.c
@@ -498,6 +498,11 @@ struct map *machine__new_module(struct machine *machine, u64 start,
 	if (kmod_path__parse_name(&m, filename))
 		return NULL;
 
+	map = map_groups__find_by_name(&machine->kmaps, MAP__FUNCTION,
+				       m.name);
+	if (map)
+		goto out;
+
 	dso = machine__module_dso(machine, &m, filename);
 	if (dso == NULL)
 		goto out;
-- 
1.9.3

[PATCH 14/14] perf annotate: Allow annotation for decompressed kernel modules

[Jiri Olsa]

Decompressing kernel module file for objdump command
if needed. Annotation commands now display annotation
for compressed kernel modules.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Stephane Eranian <eranian@google.com>
---
 tools/perf/util/annotate.c | 32 +++++++++++++++++++++++++++++++-
 1 file changed, 31 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/annotate.c b/tools/perf/util/annotate.c
index 61bf9128e1f2..b72086eca943 100644
--- a/tools/perf/util/annotate.c
+++ b/tools/perf/util/annotate.c
@@ -1008,6 +1008,32 @@ fallback:
 			}
 			filename = symfs_filename;
 		}
+	} else if (dso__needs_decompress(dso)) {
+		char tmp[PATH_MAX];
+		struct kmod_path m;
+		int fd;
+		bool ret;
+
+		if (kmod_path__parse_ext(&m, symfs_filename))
+			goto out_free_filename;
+
+		snprintf(tmp, PATH_MAX, "/tmp/perf-kmod-XXXXXX");
+
+		fd = mkstemp(tmp);
+		if (fd < 0) {
+			free(m.ext);
+			goto out_free_filename;
+		}
+
+		ret = decompress_to_file(m.ext, symfs_filename, fd);
+
+		free(m.ext);
+		close(fd);
+
+		if (!ret)
+			goto out_free_filename;
+
+		strcpy(symfs_filename, tmp);
 	}
 
 	snprintf(command, sizeof(command),
@@ -1027,7 +1053,7 @@ fallback:
 
 	file = popen(command, "r");
 	if (!file)
-		goto out_free_filename;
+		goto out_remove_tmp;
 
 	while (!feof(file))
 		if (symbol__parse_objdump_line(sym, map, file, privsize,
@@ -1042,6 +1068,10 @@ fallback:
 		delete_last_nop(sym);
 
 	pclose(file);
+
+out_remove_tmp:
+	if (dso__needs_decompress(dso))
+		unlink(symfs_filename);
 out_free_filename:
 	if (delete_extract)
 		kcore_extract__delete(&kce);
-- 
1.9.3

Re: [PATCHv2 00/14] perf tools: Adding xz decompression support

[Jiri Olsa]

On Fri, Mar 06, 2015 at 12:13:48PM +0100, Jiri Olsa wrote:
> hi,
> adding support to decompress kernel modules on Fedora 21
> (lzma compression) vaguely continuing in previous RFC:
>   http://marc.info/?l=linux-kernel&m=142261340327588&w=2
> 
> I'm basically extending he decompression framework Namhyung
> introduced earlier plus other fixes for kernel modules dso
> objects handling.
> 
> Last patch adds decompression support to annotate command, so
> it now displays annotation for compressed kernel modules.
> 
> Current work available at:
>   git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
>   perf/lzma
> 
> NOTE It's rebased on 'perf/build' branch.
> 
> v2 changes:
>   - added warning message for undetected lzma support (Arnaldo)
>   - simplyfied code in decompress_kmodule (Namhyung)
>   - renamed dsos__new to dsos__addnew (Arnaldo)
>   - moved kmod_path__parse to dso.c object because of the python's
>     perf.so compilation issue
>   - added extra m.comp check and comment into map_groups__set_module_path

oops, forgot to add Namhyug's ACK.. :-\

jirka

Re: [PATCHv2 00/14] perf tools: Adding xz decompression support

[Jiri Olsa]

On Fri, Mar 06, 2015 at 01:54:02PM +0100, Jiri Olsa wrote:
> On Fri, Mar 06, 2015 at 12:13:48PM +0100, Jiri Olsa wrote:
> > hi,
> > adding support to decompress kernel modules on Fedora 21
> > (lzma compression) vaguely continuing in previous RFC:
> >   http://marc.info/?l=linux-kernel&m=142261340327588&w=2
> > 
> > I'm basically extending he decompression framework Namhyung
> > introduced earlier plus other fixes for kernel modules dso
> > objects handling.
> > 
> > Last patch adds decompression support to annotate command, so
> > it now displays annotation for compressed kernel modules.
> > 
> > Current work available at:
> >   git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
> >   perf/lzma
> > 
> > NOTE It's rebased on 'perf/build' branch.

updated perf/lzma branch to latest perf/build

jirka

Re: [PATCHv2 00/14] perf tools: Adding xz decompression support

[Jiri Olsa]

On Wed, Mar 11, 2015 at 10:08:11AM +0100, Jiri Olsa wrote:
> On Fri, Mar 06, 2015 at 01:54:02PM +0100, Jiri Olsa wrote:
> > On Fri, Mar 06, 2015 at 12:13:48PM +0100, Jiri Olsa wrote:
> > > hi,
> > > adding support to decompress kernel modules on Fedora 21
> > > (lzma compression) vaguely continuing in previous RFC:
> > >   http://marc.info/?l=linux-kernel&m=142261340327588&w=2
> > > 
> > > I'm basically extending he decompression framework Namhyung
> > > introduced earlier plus other fixes for kernel modules dso
> > > objects handling.
> > > 
> > > Last patch adds decompression support to annotate command, so
> > > it now displays annotation for compressed kernel modules.
> > > 
> > > Current work available at:
> > >   git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
> > >   perf/lzma
> > > 
> > > NOTE It's rebased on 'perf/build' branch.
> 
> updated perf/lzma branch to latest perf/build

updated perf/lzma branch to latest perf/build (which is updated to acme's perf/core)

jirka

Re: [PATCHv2 00/14] perf tools: Adding xz decompression support

[Jiri Olsa]

On Fri, Mar 13, 2015 at 12:51:35PM +0100, Jiri Olsa wrote:
> On Wed, Mar 11, 2015 at 10:08:11AM +0100, Jiri Olsa wrote:
> > On Fri, Mar 06, 2015 at 01:54:02PM +0100, Jiri Olsa wrote:
> > > On Fri, Mar 06, 2015 at 12:13:48PM +0100, Jiri Olsa wrote:
> > > > hi,
> > > > adding support to decompress kernel modules on Fedora 21
> > > > (lzma compression) vaguely continuing in previous RFC:
> > > >   http://marc.info/?l=linux-kernel&m=142261340327588&w=2
> > > > 
> > > > I'm basically extending he decompression framework Namhyung
> > > > introduced earlier plus other fixes for kernel modules dso
> > > > objects handling.
> > > > 
> > > > Last patch adds decompression support to annotate command, so
> > > > it now displays annotation for compressed kernel modules.
> > > > 
> > > > Current work available at:
> > > >   git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
> > > >   perf/lzma
> > > > 
> > > > NOTE It's rebased on 'perf/build' branch.
> > 
> > updated perf/lzma branch to latest perf/build
> 
> updated perf/lzma branch to latest perf/build (which is updated to acme's perf/core)

updated perf/lzma branch to latest perf/build (which is updated to acme's perf/core)

;-)

jirka

Re: [PATCH 08/14] perf tools: Use kmod_path__parse in map_groups__set_modules_path_dir

[Arnaldo Carvalho de Melo]

Em Fri, Mar 06, 2015 at 12:13:56PM +0100, Jiri Olsa escreveu:
> Replacing the file name parsing with kmod_path__parse
> and moving the dso update into new separate function.
> 
> ---
>  tools/perf/util/machine.c | 65 ++++++++++++++++++++++++++++++-----------------
>  1 file changed, 41 insertions(+), 24 deletions(-)
> 
> diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c

> +static int map_groups__set_module_path(struct map_groups *mg, const char *path,
> +				       struct kmod_path *m)
> +{
> +	struct map *map;
> +	char *long_name;
> +
> +	map = map_groups__find_by_name(mg, MAP__FUNCTION, m->name);
> +	if (map == NULL)
> +		return 0;
> +
> +	long_name = strdup(path);
> +	if (long_name == NULL)
> +		return -ENOMEM;
> +
> +	dso__set_long_name(map->dso, long_name, true);
> +	dso__kernel_module_get_build_id(map->dso, "");
> +
> +	/*
> +	 * Full name could reveal us kmod compression, so
> +	 * we need to update the symtab_type if needed.
> +	 */
> +	if (m->comp && is_kmod_dso(map->dso))
> +		map->dso->symtab_type++;

Ouch, does this assumes this is a:

DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE

And that, because in the dso_binary_type enum
DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP comes right after it, and thus
this cryptic increment is equivalent to:

		map->dso->symtab_type = DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP;

?

Also, this part:

----
    Replacing the file name parsing with kmod_path__parse and moving the
dso update into new separate function.
----

Indicates that this probably would be best done in two separate patches,
one that introduces the new function, but is equivalent to the existing
code, and the next one adding the new logic?

Processed all the patches up to this one, stopping now ;-)

- Arnaldo

Re: [PATCH 08/14] perf tools: Use kmod_path__parse in map_groups__set_modules_path_dir

[Namhyung Kim]

Hi Arnaldo,

On Sat, Mar 21, 2015 at 03:20:43PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Fri, Mar 06, 2015 at 12:13:56PM +0100, Jiri Olsa escreveu:
> > Replacing the file name parsing with kmod_path__parse
> > and moving the dso update into new separate function.
> > 
> > ---
> >  tools/perf/util/machine.c | 65 ++++++++++++++++++++++++++++++-----------------
> >  1 file changed, 41 insertions(+), 24 deletions(-)
> > 
> > diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
> 
> > +static int map_groups__set_module_path(struct map_groups *mg, const char *path,
> > +				       struct kmod_path *m)
> > +{
> > +	struct map *map;
> > +	char *long_name;
> > +
> > +	map = map_groups__find_by_name(mg, MAP__FUNCTION, m->name);
> > +	if (map == NULL)
> > +		return 0;
> > +
> > +	long_name = strdup(path);
> > +	if (long_name == NULL)
> > +		return -ENOMEM;
> > +
> > +	dso__set_long_name(map->dso, long_name, true);
> > +	dso__kernel_module_get_build_id(map->dso, "");
> > +
> > +	/*
> > +	 * Full name could reveal us kmod compression, so
> > +	 * we need to update the symtab_type if needed.
> > +	 */
> > +	if (m->comp && is_kmod_dso(map->dso))
> > +		map->dso->symtab_type++;
> 
> Ouch, does this assumes this is a:
> 
> DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE

Nope, it's one of _SYSTEM_PATH_KMODULE or _GUEST_KMODULE.


> 
> And that, because in the dso_binary_type enum
> DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP comes right after it, and thus
> this cryptic increment is equivalent to:
> 
> 		map->dso->symtab_type = DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP;

Again, _SYSTEM_PATH_KMODULE_COMP or _GUEST_KMODULE_COMP.

Thanks,
Namhyung


> 
> ?
> 
> Also, this part:
> 
> ----
>     Replacing the file name parsing with kmod_path__parse and moving the
> dso update into new separate function.
> ----
> 
> Indicates that this probably would be best done in two separate patches,
> one that introduces the new function, but is equivalent to the existing
> code, and the next one adding the new logic?
> 
> Processed all the patches up to this one, stopping now ;-)
> 
> - Arnaldo
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 08/14] perf tools: Use kmod_path__parse in map_groups__set_modules_path_dir

[Jiri Olsa]

On Mon, Mar 23, 2015 at 08:55:09AM +0900, Namhyung Kim wrote:

SNIP

> > > +	 */
> > > +	if (m->comp && is_kmod_dso(map->dso))
> > > +		map->dso->symtab_type++;
> > 
> > Ouch, does this assumes this is a:
> > 
> > DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE
> 
> Nope, it's one of _SYSTEM_PATH_KMODULE or _GUEST_KMODULE.
> 
> 
> > 
> > And that, because in the dso_binary_type enum
> > DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP comes right after it, and thus
> > this cryptic increment is equivalent to:
> > 
> > 		map->dso->symtab_type = DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP;
> 
> Again, _SYSTEM_PATH_KMODULE_COMP or _GUEST_KMODULE_COMP.
> 

yep, the code assumes (correctly) that symtab_type is
either _SYSTEM_PATH_KMODULE or _GUEST_KMODULE.

Namhyung placed the _COMP enum right behind 'no comp' values:

enum dso_binary_type {
	...
        DSO_BINARY_TYPE__GUEST_KMODULE,
        DSO_BINARY_TYPE__GUEST_KMODULE_COMP,
        DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE,
        DSO_BINARY_TYPE__SYSTEM_PATH_KMODULE_COMP,
	...

so we could safely just increase the symtab_type in case
we detect it's compressed kernel module.

Perhaps the code could be more verbose on this, but since such
comment wasn't part of the previous code I did not feel the need
to add it now ;-)

jirka