[GIT PULL 00/18] perf/core improvements and fixes

[GIT PULL 00/18] perf/core improvements and fixes

[Arnaldo Carvalho de Melo]

Hi Ingo,

	Please consider pulling,

- Arnaldo

The following changes since commit d4cf1949f9689314aef962eea95df84a8288d097:

  perf/x86/intel: Add {rd,wr}lbr_{to,from} wrappers (2016-06-27 11:34:21 +0200)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-20160627

for you to fetch changes up to 6d9c675078e697309c1c06a1051f01de8151c476:

  perf data ctf: Generate fork and exit events to CTF output (2016-06-27 15:50:23 -0300)

----------------------------------------------------------------
perf/core improvements and fixes:

New features:

- Generate comm, fork and exit events when converting perf.data files to CTF (Wang Nan)

Documentation:

- Document perf.data on disk format (Andi Kleen)

Infrastructure:

- Add libbabeltrace to build-test (Wang Nan)

- 'perf record' prep work to support multiple evlists (Wang Nan)

- Remove unused hist_entry__annotate function (Ravi Bangoria)

- Add more toolchain triplets (Ravi Bangoria)

- Update message for slang devel packages on Ubuntu (Neeraj Badlani)

- Generalize handling of 'ret' instructions in the annotate TUI (Naveen N. Rao)

- Use proper dso name for is_regular_file, fixing device file handling (Jiri Olsa)

Build Fixes:

- Add missing config.h include, fixing the build with libabeltrace (Jiri Olsa)

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

----------------------------------------------------------------
Andi Kleen (1):
      perf tools: Add documentation for perf.data on disk format

Jiri Olsa (2):
      perf data convert: Include config.h header
      perf symbols: Use proper dso name for is_regular_file

Naveen N. Rao (1):
      perf annotate: Generalize handling of 'ret' instructions

Neeraj Badlani (1):
      perf tools: Update makefile message for installing slang devel package

Ravi Bangoria (2):
      perf annotate: Remove unused hist_entry__annotate function
      perf tools: Add more toolchain triplets

Wang Nan (11):
      perf build: Add libbabeltrace to build-test
      perf record: Move mmap setup block to separate function
      perf record: Prepare reading from multiple evlists in record__mmap_read_all()
      perf record: Prepare picking perf_event_mmap_page from multiple evlists
      perf data ctf: Add value_set_string() helper
      perf data ctf: Pass convert options through opts structure
      perf data ctf: Add 'all' option
      perf data ctf: Prepare collect non-sample events
      perf data ctf: Generate comm event to CTF output
      perf data ctf: Add '--all' option for 'perf data convert'
      perf data ctf: Generate fork and exit events to CTF output

 tools/perf/Documentation/perf-data.txt        |   4 +
 tools/perf/Documentation/perf-file-format.txt | 442 ++++++++++++++++++++++++++
 tools/perf/arch/common.c                      |  17 +
 tools/perf/builtin-data.c                     |  11 +-
 tools/perf/builtin-record.c                   | 105 ++++--
 tools/perf/config/Makefile                    |   2 +-
 tools/perf/tests/make                         |   2 +
 tools/perf/ui/browsers/annotate.c             |  20 +-
 tools/perf/util/annotate.c                    |  15 +-
 tools/perf/util/annotate.h                    |   3 +-
 tools/perf/util/data-convert-bt.c             | 196 +++++++++++-
 tools/perf/util/data-convert-bt.h             |   4 +-
 tools/perf/util/data-convert.h                |   9 +
 tools/perf/util/symbol.c                      |   2 +-
 14 files changed, 773 insertions(+), 59 deletions(-)
 create mode 100644 tools/perf/Documentation/perf-file-format.txt
 create mode 100644 tools/perf/util/data-convert.h

[PATCH 01/18] perf annotate: Remove unused hist_entry__annotate function

[Arnaldo Carvalho de Melo]

From: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>

hist_entry__annotate looks part of API but I don't find any caller
of this function. Removing it.

Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Daniel Axtens <dja@axtens.net>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Link: http://lkml.kernel.org/r/1466769240-12376-2-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/annotate.c | 5 -----
 tools/perf/util/annotate.h | 2 --
 2 files changed, 7 deletions(-)

diff --git a/tools/perf/util/annotate.c b/tools/perf/util/annotate.c
index 7e5a1e8874ce..b2c7ae465465 100644
--- a/tools/perf/util/annotate.c
+++ b/tools/perf/util/annotate.c
@@ -1676,11 +1676,6 @@ int symbol__tty_annotate(struct symbol *sym, struct map *map,
 	return 0;
 }
 
-int hist_entry__annotate(struct hist_entry *he, size_t privsize)
-{
-	return symbol__annotate(he->ms.sym, he->ms.map, privsize);
-}
-
 bool ui__has_annotation(void)
 {
 	return use_browser == 1 && perf_hpp_list.sym;
diff --git a/tools/perf/util/annotate.h b/tools/perf/util/annotate.h
index 9241f8c2b7e1..82f3781138f9 100644
--- a/tools/perf/util/annotate.h
+++ b/tools/perf/util/annotate.h
@@ -156,8 +156,6 @@ void symbol__annotate_zero_histograms(struct symbol *sym);
 
 int symbol__annotate(struct symbol *sym, struct map *map, size_t privsize);
 
-int hist_entry__annotate(struct hist_entry *he, size_t privsize);
-
 int symbol__annotate_init(struct map *map, struct symbol *sym);
 int symbol__annotate_printf(struct symbol *sym, struct map *map,
 			    struct perf_evsel *evsel, bool full_paths,
-- 
2.7.4

[PATCH 02/18] perf tools: Update makefile message for installing slang devel package

[Arnaldo Carvalho de Melo]

From: Neeraj Badlani <neerajbadlani@gmail.com>

In case of missing library (libslang), give hint to install library
(libslang2-dev), since libslang-dev is not provided by Ubuntu.

Signed-off-by: Neeraj Badlani <neerajbadlani@gmail.com>
Link: http://lkml.kernel.org/r/1467035997-9100-1-git-send-email-neerajbadlani@gmail.com
[ removed excessive 'or' usage ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/config/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/config/Makefile b/tools/perf/config/Makefile
index 534c81176f6c..bf1a0a0dd0ad 100644
--- a/tools/perf/config/Makefile
+++ b/tools/perf/config/Makefile
@@ -482,7 +482,7 @@ endif
 
 ifndef NO_SLANG
   ifneq ($(feature-libslang), 1)
-    msg := $(warning slang not found, disables TUI support. Please install slang-devel or libslang-dev);
+    msg := $(warning slang not found, disables TUI support. Please install slang-devel, libslang-dev or libslang2-dev);
     NO_SLANG := 1
   else
     # Fedora has /usr/include/slang/slang.h, but ubuntu /usr/include/slang.h
-- 
2.7.4

[PATCH 03/18] perf annotate: Generalize handling of 'ret' instructions

[Arnaldo Carvalho de Melo]

From: "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com>

Introduce helper to detect 'ret' instructions and use the same in the TUI.
A helper is needed since some architectures such as powerpc have more
than one return instruction.

Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Daniel Axtens <dja@axtens.net>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Link: http://lkml.kernel.org/r/1466769240-12376-5-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/ui/browsers/annotate.c | 20 +++++++++-----------
 tools/perf/util/annotate.c        | 10 ++++++++++
 tools/perf/util/annotate.h        |  1 +
 3 files changed, 20 insertions(+), 11 deletions(-)

diff --git a/tools/perf/ui/browsers/annotate.c b/tools/perf/ui/browsers/annotate.c
index 0e106bb97525..29dc6d20364e 100644
--- a/tools/perf/ui/browsers/annotate.c
+++ b/tools/perf/ui/browsers/annotate.c
@@ -223,16 +223,14 @@ static void annotate_browser__write(struct ui_browser *browser, void *entry, int
 			} else if (ins__is_call(dl->ins)) {
 				ui_browser__write_graph(browser, SLSMG_RARROW_CHAR);
 				SLsmg_write_char(' ');
+			} else if (ins__is_ret(dl->ins)) {
+				ui_browser__write_graph(browser, SLSMG_LARROW_CHAR);
+				SLsmg_write_char(' ');
 			} else {
 				ui_browser__write_nstring(browser, " ", 2);
 			}
 		} else {
-			if (strcmp(dl->name, "retq")) {
-				ui_browser__write_nstring(browser, " ", 2);
-			} else {
-				ui_browser__write_graph(browser, SLSMG_LARROW_CHAR);
-				SLsmg_write_char(' ');
-			}
+			ui_browser__write_nstring(browser, " ", 2);
 		}
 
 		disasm_line__scnprintf(dl, bf, sizeof(bf), !annotate_browser__opts.use_offset);
@@ -843,14 +841,14 @@ show_help:
 				ui_helpline__puts("Huh? No selection. Report to linux-kernel@vger.kernel.org");
 			else if (browser->selection->offset == -1)
 				ui_helpline__puts("Actions are only available for assembly lines.");
-			else if (!browser->selection->ins) {
-				if (strcmp(browser->selection->name, "retq"))
-					goto show_sup_ins;
+			else if (!browser->selection->ins)
+				goto show_sup_ins;
+			else if (ins__is_ret(browser->selection->ins))
 				goto out;
-			} else if (!(annotate_browser__jump(browser) ||
+			else if (!(annotate_browser__jump(browser) ||
 				     annotate_browser__callq(browser, evsel, hbt))) {
 show_sup_ins:
-				ui_helpline__puts("Actions are only available for 'callq', 'retq' & jump instructions.");
+				ui_helpline__puts("Actions are only available for function call/return & jump/branch instructions.");
 			}
 			continue;
 		case 't':
diff --git a/tools/perf/util/annotate.c b/tools/perf/util/annotate.c
index b2c7ae465465..c385fecb9d32 100644
--- a/tools/perf/util/annotate.c
+++ b/tools/perf/util/annotate.c
@@ -354,6 +354,15 @@ static struct ins_ops nop_ops = {
 	.scnprintf = nop__scnprintf,
 };
 
+static struct ins_ops ret_ops = {
+	.scnprintf = ins__raw_scnprintf,
+};
+
+bool ins__is_ret(const struct ins *ins)
+{
+	return ins->ops == &ret_ops;
+}
+
 static struct ins instructions[] = {
 	{ .name = "add",   .ops  = &mov_ops, },
 	{ .name = "addl",  .ops  = &mov_ops, },
@@ -444,6 +453,7 @@ static struct ins instructions[] = {
 	{ .name = "xadd",  .ops  = &mov_ops, },
 	{ .name = "xbeginl", .ops  = &jump_ops, },
 	{ .name = "xbeginq", .ops  = &jump_ops, },
+	{ .name = "retq",  .ops  = &ret_ops, },
 };
 
 static int ins__key_cmp(const void *name, const void *insp)
diff --git a/tools/perf/util/annotate.h b/tools/perf/util/annotate.h
index 82f3781138f9..a23084f54128 100644
--- a/tools/perf/util/annotate.h
+++ b/tools/perf/util/annotate.h
@@ -48,6 +48,7 @@ struct ins {
 
 bool ins__is_jump(const struct ins *ins);
 bool ins__is_call(const struct ins *ins);
+bool ins__is_ret(const struct ins *ins);
 int ins__scnprintf(struct ins *ins, char *bf, size_t size, struct ins_operands *ops);
 
 struct annotation;
-- 
2.7.4

[PATCH 04/18] perf tools: Add more toolchain triplets

[Arnaldo Carvalho de Melo]

From: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>

Add few more triplets based on Fedora and Ubuntu binutils (cross tools).

Before applying patch on x86:

  ( Install binutils-powerpc64-linux-gnu.x86_64 )
  $ perf report -i perf.data.powerpc --vmlinux vmlinux.powerpc \
      --objdump powerpc64-linux-gnu-objdump

After applying patch on x86:

  $ perf report -i perf.data.powerpc --vmlinux vmlinux.powerpc

I.e. it will find the right objdump from the environment data recorded
in the perf.data file + these triplets.

Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Cc: Anton Blanchard <anton@ozlabs.org>
Cc: Daniel Axtens <dja@axtens.net>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Link: http://lkml.kernel.org/r/1466769240-12376-7-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/arch/common.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/tools/perf/arch/common.c b/tools/perf/arch/common.c
index fa090a9eaa38..ee6966812a5a 100644
--- a/tools/perf/arch/common.c
+++ b/tools/perf/arch/common.c
@@ -9,34 +9,44 @@ const char *const arm_triplets[] = {
 	"arm-unknown-linux-",
 	"arm-unknown-linux-gnu-",
 	"arm-unknown-linux-gnueabi-",
+	"arm-linux-gnu-",
+	"arm-linux-gnueabihf-",
+	"arm-none-eabi-",
 	NULL
 };
 
 const char *const arm64_triplets[] = {
 	"aarch64-linux-android-",
+	"aarch64-linux-gnu-",
 	NULL
 };
 
 const char *const powerpc_triplets[] = {
 	"powerpc-unknown-linux-gnu-",
 	"powerpc64-unknown-linux-gnu-",
+	"powerpc64-linux-gnu-",
+	"powerpc64le-linux-gnu-",
 	NULL
 };
 
 const char *const s390_triplets[] = {
 	"s390-ibm-linux-",
+	"s390x-linux-gnu-",
 	NULL
 };
 
 const char *const sh_triplets[] = {
 	"sh-unknown-linux-gnu-",
 	"sh64-unknown-linux-gnu-",
+	"sh-linux-gnu-",
+	"sh64-linux-gnu-",
 	NULL
 };
 
 const char *const sparc_triplets[] = {
 	"sparc-unknown-linux-gnu-",
 	"sparc64-unknown-linux-gnu-",
+	"sparc64-linux-gnu-",
 	NULL
 };
 
@@ -49,12 +59,19 @@ const char *const x86_triplets[] = {
 	"i386-pc-linux-gnu-",
 	"i686-linux-android-",
 	"i686-android-linux-",
+	"x86_64-linux-gnu-",
+	"i586-linux-gnu-",
 	NULL
 };
 
 const char *const mips_triplets[] = {
 	"mips-unknown-linux-gnu-",
 	"mipsel-linux-android-",
+	"mips-linux-gnu-",
+	"mips64-linux-gnu-",
+	"mips64el-linux-gnuabi64-",
+	"mips64-linux-gnuabi64-",
+	"mipsel-linux-gnu-",
 	NULL
 };
 
-- 
2.7.4

[PATCH 05/18] perf tools: Add documentation for perf.data on disk format

[Arnaldo Carvalho de Melo]

From: Andi Kleen <ak@linux.intel.com>

Add some documentation for the on disk format of perf.data. This is not
documenting the actual perf events -- which are documented in
perf_event.h -- but just the additional headers that perf record adds
around them when writing the data to disk.

Signed-off-by: Andi Kleen <ak@linux.intel.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Link: http://lkml.kernel.org/r/1466800885-12974-1-git-send-email-andi@firstfloor.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/Documentation/perf-file-format.txt | 442 ++++++++++++++++++++++++++
 1 file changed, 442 insertions(+)
 create mode 100644 tools/perf/Documentation/perf-file-format.txt

diff --git a/tools/perf/Documentation/perf-file-format.txt b/tools/perf/Documentation/perf-file-format.txt
new file mode 100644
index 000000000000..fdc99fe6bbc3
--- /dev/null
+++ b/tools/perf/Documentation/perf-file-format.txt
@@ -0,0 +1,442 @@
+perf.data format
+
+Uptodate as of v4.7
+
+This document describes the on-disk perf.data format, generated by perf record
+or perf inject and consumed by the other perf tools.
+
+On a high level perf.data contains the events generated by the PMUs, plus metadata.
+
+All fields are in native-endian of the machine that generated the perf.data.
+
+When perf is writing to a pipe it uses a special version of the file
+format that does not rely on seeking to adjust data offsets.  This
+format is not described here. The pipe version can be converted to
+normal perf.data with perf inject.
+
+The file starts with a perf_header:
+
+struct perf_header {
+	char magic[8];		/* PERFILE2 */
+	uint64_t size;		/* size of the header */
+	uint64_t attr_size;	/* size of an attribute in attrs */
+	struct perf_file_section attrs;
+	struct perf_file_section data;
+	struct perf_file_section event_types;
+	uint64_t flags;
+	uint64_t flags1[3];
+};
+
+The magic number identifies the perf file and the version. Current perf versions
+use PERFILE2. Old perf versions generated a version 1 format (PERFFILE). Version 1
+is not described here. The magic number also identifies the endian. When the
+magic value is 64bit byte swapped compared the file is in non-native
+endian.
+
+A perf_file_section contains a pointer to another section of the perf file.
+The header contains three such pointers: for attributes, data and event types.
+
+struct perf_file_section {
+	uint64_t offset;	/* offset from start of file */
+	uint64_t size;		/* size of the section */
+};
+
+Flags section:
+
+The header is followed by different optional headers, described by the bits set
+in flags. Only headers for which the bit is set are included. Each header
+consists of a perf_file_section located after the initial header.
+The respective perf_file_section points to the data of the additional
+header and defines its size.
+
+Some headers consist of strings, which are defined like this:
+
+struct perf_header_string {
+       uint32_t len;
+       char string[len]; /* zero terminated */
+};
+
+Some headers consist of a sequence of strings, which start with a
+
+struct perf_header_string_list {
+     uint32_t nr;
+     struct perf_header_string strings[nr]; /* variable length records */
+};
+
+The bits are the flags bits in a 256 bit bitmap starting with
+flags. These define the valid bits:
+
+	HEADER_RESERVED		= 0,	/* always cleared */
+	HEADER_FIRST_FEATURE	= 1,
+	HEADER_TRACING_DATA	= 1,
+
+Describe me.
+
+	HEADER_BUILD_ID = 2,
+
+The header consists of an sequence of build_id_event. The size of each record
+is defined by header.size (see perf_event.h). Each event defines a ELF build id
+for a executable file name for a pid. An ELF build id is a unique identifier
+assigned by the linker to an executable.
+
+struct build_id_event {
+	struct perf_event_header header;
+	pid_t			 pid;
+	uint8_t			 build_id[24];
+	char			 filename[header.size - offsetof(struct build_id_event, filename)];
+};
+
+	HEADER_HOSTNAME = 3,
+
+A perf_header_string with the hostname where the data was collected
+(uname -n)
+
+	HEADER_OSRELEASE = 4,
+
+A perf_header_string with the os release where the data was collected
+(uname -r)
+
+	HEADER_VERSION = 5,
+
+A perf_header_string with the perf user tool version where the
+data was collected. This is the same as the version of the source tree
+the perf tool was built from.
+
+	HEADER_ARCH = 6,
+
+A perf_header_string with the CPU architecture (uname -m)
+
+	HEADER_NRCPUS = 7,
+
+A structure defining the number of CPUs.
+
+struct nr_cpus {
+       uint32_t nr_cpus_online;
+       uint32_t nr_cpus_available; /* CPUs not yet onlined */
+};
+
+	HEADER_CPUDESC = 8,
+
+A perf_header_string with description of the CPU. On x86 this is the model name
+in /proc/cpuinfo
+
+	HEADER_CPUID = 9,
+
+A perf_header_string with the exact CPU type. On x86 this is
+vendor,family,model,stepping. For example: GenuineIntel,6,69,1
+
+	HEADER_TOTAL_MEM = 10,
+
+An uint64_t with the total memory in bytes.
+
+	HEADER_CMDLINE = 11,
+
+A perf_header_string with the perf command line used to collect the data.
+
+	HEADER_EVENT_DESC = 12,
+
+Another description of the perf_event_attrs, more detailed than header.attrs
+including IDs and names. See perf_event.h or the man page for a description
+of a struct perf_event_attr.
+
+struct {
+       uint32_t nr; /* number of events */
+       uint32_t attr_size; /* size of each perf_event_attr */
+       struct {
+	      struct perf_event_attr attr;  /* size of attr_size */
+	      uint32_t nr_ids;
+	      struct perf_header_string event_string;
+	      uint64_t ids[nr_ids];
+       } events[nr]; /* Variable length records */
+};
+
+	HEADER_CPU_TOPOLOGY = 13,
+
+String lists defining the core and CPU threads topology.
+
+struct {
+       struct perf_header_string_list cores; /* Variable length */
+       struct perf_header_string_list threads; /* Variable length */
+};
+
+Example:
+	sibling cores   : 0-3
+	sibling threads : 0-1
+	sibling threads : 2-3
+
+	HEADER_NUMA_TOPOLOGY = 14,
+
+	A list of NUMA node descriptions
+
+struct {
+       uint32_t nr;
+       struct {
+	      uint32_t nodenr;
+	      uint64_t mem_total;
+	      uint64_t mem_free;
+	      struct perf_header_string cpus;
+       } nodes[nr]; /* Variable length records */
+};
+
+	HEADER_BRANCH_STACK = 15,
+
+Not implemented in perf.
+
+	HEADER_PMU_MAPPINGS = 16,
+
+	A list of PMU structures, defining the different PMUs supported by perf.
+
+struct {
+       uint32_t nr;
+       struct pmu {
+	      uint32_t pmu_type;
+	      struct perf_header_string pmu_name;
+       } [nr]; /* Variable length records */
+};
+
+	HEADER_GROUP_DESC = 17,
+
+	Description of counter groups ({...} in perf syntax)
+
+struct {
+         uint32_t nr;
+         struct {
+		struct perf_header_string string;
+		uint32_t leader_idx;
+		uint32_t nr_members;
+	 } [nr]; /* Variable length records */
+};
+
+	HEADER_AUXTRACE = 18,
+
+Define additional auxtrace areas in the perf.data. auxtrace is used to store
+undecoded hardware tracing information, such as Intel Processor Trace data.
+
+/**
+ * struct auxtrace_index_entry - indexes a AUX area tracing event within a
+ *                               perf.data file.
+ * @file_offset: offset within the perf.data file
+ * @sz: size of the event
+ */
+struct auxtrace_index_entry {
+	u64			file_offset;
+	u64			sz;
+};
+
+#define PERF_AUXTRACE_INDEX_ENTRY_COUNT 256
+
+/**
+ * struct auxtrace_index - index of AUX area tracing events within a perf.data
+ *                         file.
+ * @list: linking a number of arrays of entries
+ * @nr: number of entries
+ * @entries: array of entries
+ */
+struct auxtrace_index {
+	struct list_head	list;
+	size_t			nr;
+	struct auxtrace_index_entry entries[PERF_AUXTRACE_INDEX_ENTRY_COUNT];
+};
+
+	other bits are reserved and should ignored for now
+	HEADER_FEAT_BITS	= 256,
+
+Attributes
+
+This is an array of perf_event_attrs, each attr_size bytes long, which defines
+each event collected. See perf_event.h or the man page for a detailed
+description.
+
+Data
+
+This section is the bulk of the file. It consist of a stream of perf_events
+describing events. This matches the format generated by the kernel.
+See perf_event.h or the manpage for a detailed description.
+
+Some notes on parsing:
+
+Ordering
+
+The events are not necessarily in time stamp order, as they can be
+collected in parallel on different CPUs. If the events should be
+processed in time order they need to be sorted first. It is possible
+to only do a partial sort using the FINISHED_ROUND event header (see
+below). perf record guarantees that there is no reordering over a
+FINISHED_ROUND.
+
+ID vs IDENTIFIER
+
+When the event stream contains multiple events each event is identified
+by an ID. This can be either through the PERF_SAMPLE_ID or the
+PERF_SAMPLE_IDENTIFIER header. The PERF_SAMPLE_IDENTIFIER header is
+at a fixed offset from the event header, which allows reliable
+parsing of the header. Relying on ID may be ambigious.
+IDENTIFIER is only supported by newer Linux kernels.
+
+Perf record specific events:
+
+In addition to the kernel generated event types perf record adds its
+own event types (in addition it also synthesizes some kernel events,
+for example MMAP events)
+
+	PERF_RECORD_USER_TYPE_START		= 64,
+	PERF_RECORD_HEADER_ATTR			= 64,
+
+struct attr_event {
+	struct perf_event_header header;
+	struct perf_event_attr attr;
+	uint64_t id[];
+};
+
+	PERF_RECORD_HEADER_EVENT_TYPE		= 65, /* depreceated */
+
+#define MAX_EVENT_NAME 64
+
+struct perf_trace_event_type {
+	uint64_t	event_id;
+	char	name[MAX_EVENT_NAME];
+};
+
+struct event_type_event {
+	struct perf_event_header header;
+	struct perf_trace_event_type event_type;
+};
+
+
+	PERF_RECORD_HEADER_TRACING_DATA		= 66,
+
+Describe me
+
+struct tracing_data_event {
+	struct perf_event_header header;
+	uint32_t size;
+};
+
+	PERF_RECORD_HEADER_BUILD_ID		= 67,
+
+Define a ELF build ID for a referenced executable.
+
+       struct build_id_event;   /* See above */
+
+	PERF_RECORD_FINISHED_ROUND		= 68,
+
+No event reordering over this header. No payload.
+
+	PERF_RECORD_ID_INDEX			= 69,
+
+Map event ids to CPUs and TIDs.
+
+struct id_index_entry {
+	uint64_t id;
+	uint64_t idx;
+	uint64_t cpu;
+	uint64_t tid;
+};
+
+struct id_index_event {
+	struct perf_event_header header;
+	uint64_t nr;
+	struct id_index_entry entries[nr];
+};
+
+	PERF_RECORD_AUXTRACE_INFO		= 70,
+
+Auxtrace type specific information. Describe me
+
+struct auxtrace_info_event {
+	struct perf_event_header header;
+	uint32_t type;
+	uint32_t reserved__; /* For alignment */
+	uint64_t priv[];
+};
+
+	PERF_RECORD_AUXTRACE			= 71,
+
+Defines auxtrace data. Followed by the actual data. The contents of
+the auxtrace data is dependent on the event and the CPU. For example
+for Intel Processor Trace it contains Processor Trace data generated
+by the CPU.
+
+struct auxtrace_event {
+	struct perf_event_header header;
+	uint64_t size;
+	uint64_t offset;
+	uint64_t reference;
+	uint32_t idx;
+	uint32_t tid;
+	uint32_t cpu;
+	uint32_t reserved__; /* For alignment */
+};
+
+struct aux_event {
+	struct perf_event_header header;
+	uint64_t	aux_offset;
+	uint64_t	aux_size;
+	uint64_t	flags;
+};
+
+	PERF_RECORD_AUXTRACE_ERROR		= 72,
+
+Describes an error in hardware tracing
+
+enum auxtrace_error_type {
+	PERF_AUXTRACE_ERROR_ITRACE  = 1,
+	PERF_AUXTRACE_ERROR_MAX
+};
+
+#define MAX_AUXTRACE_ERROR_MSG 64
+
+struct auxtrace_error_event {
+	struct perf_event_header header;
+	uint32_t type;
+	uint32_t code;
+	uint32_t cpu;
+	uint32_t pid;
+	uint32_t tid;
+	uint32_t reserved__; /* For alignment */
+	uint64_t ip;
+	char msg[MAX_AUXTRACE_ERROR_MSG];
+};
+
+Event types
+
+Define the event attributes with their IDs.
+
+An array bound by the perf_file_section size.
+
+	struct {
+		struct perf_event_attr attr;   /* Size defined by header.attr_size */
+		struct perf_file_section ids;
+	}
+
+ids points to a array of uint64_t defining the ids for event attr attr.
+
+References:
+
+include/uapi/linux/perf_event.h
+
+This is the canonical description of the kernel generated perf_events
+and the perf_event_attrs.
+
+perf_events manpage
+
+A manpage describing perf_event and perf_event_attr is here:
+http://web.eece.maine.edu/~vweaver/projects/perf_events/programming.html
+This tends to be slightly behind the kernel include, but has better
+descriptions.  An (typically older) version of the man page may be
+included with the standard Linux man pages, available with "man
+perf_events"
+
+pmu-tools
+
+https://github.com/andikleen/pmu-tools/tree/master/parser
+
+A definition of the perf.data format in python "construct" format is available
+in pmu-tools parser. This allows to read perf.data from python and dump it.
+
+quipper
+
+The quipper C++ parser is available at
+https://chromium.googlesource.com/chromiumos/platform/chromiumos-wide-profiling/
+Unfortunately this parser tends to be many versions behind and may not be able
+to parse data files generated by recent perf.
-- 
2.7.4

[PATCH 06/18] perf build: Add libbabeltrace to build-test

[Arnaldo Carvalho de Melo]

From: Wang Nan <wangnan0@huawei.com>

'make build-test' doesn't test LIBBABELTRACE=1. It misses a building
failure caused by commit 41840d211c51 ("perf config: Move config
declarations from util/cache.h to util/config.h"), breaks bisect.

Add LIBBABELTRACE=1 to build-test.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1466818918-131281-1-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/tests/make | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index cac15d93aea6..51966d92fc82 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -81,6 +81,7 @@ make_no_libbionic   := NO_LIBBIONIC=1
 make_no_auxtrace    := NO_AUXTRACE=1
 make_no_libbpf	    := NO_LIBBPF=1
 make_no_libcrypto   := NO_LIBCRYPTO=1
+make_with_babeltrace:= LIBBABELTRACE=1
 make_tags           := tags
 make_cscope         := cscope
 make_help           := help
@@ -136,6 +137,7 @@ run += make_no_libaudit
 run += make_no_libbionic
 run += make_no_auxtrace
 run += make_no_libbpf
+run += make_with_babeltrace
 run += make_help
 run += make_doc
 run += make_perf_o
-- 
2.7.4

[PATCH 07/18] perf data convert: Include config.h header

[Arnaldo Carvalho de Melo]

From: Jiri Olsa <jolsa@kernel.org>

Otherwise some compiler might scream:

  $ make LIBBABELTRACE_DIR=/opt/libbabeltrace/ LIBBABELTRACE=1
    BUILD:   Doing 'make -j4' parallel build
    CC       util/data-convert-bt.o
  util/data-convert-bt.c: In function ‘convert__config’:
  util/data-convert-bt.c:1299:19: error: implicit declaration of function ‘perf_config_u64’ [-Werror=implicit-function-declaration]
     c->queue_size = perf_config_u64(var, value);
  ...

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Marc Kleine-Budde <mkl@pengutronix.de>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Taeung Song <treeze.taeung@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
Fixes: 41840d211c51 ("perf config: Move config declarations from util/cache.h to util/config.h")
Link: http://lkml.kernel.org/r/1466772025-17471-1-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/data-convert-bt.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index 4b59879391c0..7b1bc24c382e 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -26,6 +26,7 @@
 #include "evlist.h"
 #include "evsel.h"
 #include "machine.h"
+#include "config.h"
 
 #define pr_N(n, fmt, ...) \
 	eprintf(n, debug_data_convert, fmt, ##__VA_ARGS__)
-- 
2.7.4

[PATCH 08/18] perf record: Move mmap setup block to separate function

[Arnaldo Carvalho de Melo]

From: Wang Nan <wangnan0@huawei.com>

Following commits introduce multiple evlists to record. This patch
extracts perf_evlist__mmap_ex() processing to a new function, creates
record__mmap() and record__mmap_evlist() to wrap perf_evlist__mmap_ex()
and its error processing. They will be improvemented to create mmap for
all evlists.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Nilay Vaish <nilayvaish@gmail.com>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1467023052-146749-2-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/builtin-record.c | 56 +++++++++++++++++++++++++++++----------------
 1 file changed, 36 insertions(+), 20 deletions(-)

diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 81411b14df4c..7eb8d7d29fbc 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -342,6 +342,40 @@ int auxtrace_record__snapshot_start(struct auxtrace_record *itr __maybe_unused)
 
 #endif
 
+static int record__mmap_evlist(struct record *rec,
+			       struct perf_evlist *evlist)
+{
+	struct record_opts *opts = &rec->opts;
+	char msg[512];
+
+	if (perf_evlist__mmap_ex(evlist, opts->mmap_pages, false,
+				 opts->auxtrace_mmap_pages,
+				 opts->auxtrace_snapshot_mode) < 0) {
+		if (errno == EPERM) {
+			pr_err("Permission error mapping pages.\n"
+			       "Consider increasing "
+			       "/proc/sys/kernel/perf_event_mlock_kb,\n"
+			       "or try again with a smaller value of -m/--mmap_pages.\n"
+			       "(current value: %u,%u)\n",
+			       opts->mmap_pages, opts->auxtrace_mmap_pages);
+			return -errno;
+		} else {
+			pr_err("failed to mmap with %d (%s)\n", errno,
+				strerror_r(errno, msg, sizeof(msg)));
+			if (errno)
+				return -errno;
+			else
+				return -EINVAL;
+		}
+	}
+	return 0;
+}
+
+static int record__mmap(struct record *rec)
+{
+	return record__mmap_evlist(rec, rec->evlist);
+}
+
 static int record__open(struct record *rec)
 {
 	char msg[512];
@@ -378,27 +412,9 @@ try_again:
 		goto out;
 	}
 
-	if (perf_evlist__mmap_ex(evlist, opts->mmap_pages, false,
-				 opts->auxtrace_mmap_pages,
-				 opts->auxtrace_snapshot_mode) < 0) {
-		if (errno == EPERM) {
-			pr_err("Permission error mapping pages.\n"
-			       "Consider increasing "
-			       "/proc/sys/kernel/perf_event_mlock_kb,\n"
-			       "or try again with a smaller value of -m/--mmap_pages.\n"
-			       "(current value: %u,%u)\n",
-			       opts->mmap_pages, opts->auxtrace_mmap_pages);
-			rc = -errno;
-		} else {
-			pr_err("failed to mmap with %d (%s)\n", errno,
-				strerror_r(errno, msg, sizeof(msg)));
-			if (errno)
-				rc = -errno;
-			else
-				rc = -EINVAL;
-		}
+	rc = record__mmap(rec);
+	if (rc)
 		goto out;
-	}
 
 	session->evlist = evlist;
 	perf_session__set_id_hdr_size(session);
-- 
2.7.4

[PATCH 09/18] perf record: Prepare reading from multiple evlists in record__mmap_read_all()

[Arnaldo Carvalho de Melo]

From: Wang Nan <wangnan0@huawei.com>

Following commits introduce new evlists to record. This patch adjusts
record__mmap_read_all() and record__mmap_read(): converting original
record__mmap_read_all() to record__mmap_read_evlist(), read from one
evlist; makes record__mmap_read() reading from specific evlist.
record__mmap_read_all() will be improved to read from multiple evlists.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Nilay Vaish <nilayvaish@gmail.com>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1467023052-146749-3-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/builtin-record.c | 34 ++++++++++++++++++++++++----------
 1 file changed, 24 insertions(+), 10 deletions(-)

diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 7eb8d7d29fbc..18e9abc90759 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -132,9 +132,9 @@ rb_find_range(struct perf_evlist *evlist,
 	return backward_rb_find_range(data, mask, head, start, end);
 }
 
-static int record__mmap_read(struct record *rec, int idx)
+static int record__mmap_read(struct record *rec, struct perf_evlist *evlist, int idx)
 {
-	struct perf_mmap *md = &rec->evlist->mmap[idx];
+	struct perf_mmap *md = &evlist->mmap[idx];
 	u64 head = perf_mmap__read_head(md);
 	u64 old = md->prev;
 	u64 end = head, start = old;
@@ -143,7 +143,7 @@ static int record__mmap_read(struct record *rec, int idx)
 	void *buf;
 	int rc = 0;
 
-	if (rb_find_range(rec->evlist, data, md->mask, head,
+	if (rb_find_range(evlist, data, md->mask, head,
 			  old, &start, &end))
 		return -1;
 
@@ -157,7 +157,7 @@ static int record__mmap_read(struct record *rec, int idx)
 		WARN_ONCE(1, "failed to keep up with mmap data. (warn only once)\n");
 
 		md->prev = head;
-		perf_evlist__mmap_consume(rec->evlist, idx);
+		perf_evlist__mmap_consume(evlist, idx);
 		return 0;
 	}
 
@@ -182,7 +182,7 @@ static int record__mmap_read(struct record *rec, int idx)
 	}
 
 	md->prev = head;
-	perf_evlist__mmap_consume(rec->evlist, idx);
+	perf_evlist__mmap_consume(evlist, idx);
 out:
 	return rc;
 }
@@ -498,17 +498,20 @@ static struct perf_event_header finished_round_event = {
 	.type = PERF_RECORD_FINISHED_ROUND,
 };
 
-static int record__mmap_read_all(struct record *rec)
+static int record__mmap_read_evlist(struct record *rec, struct perf_evlist *evlist)
 {
 	u64 bytes_written = rec->bytes_written;
 	int i;
 	int rc = 0;
 
-	for (i = 0; i < rec->evlist->nr_mmaps; i++) {
-		struct auxtrace_mmap *mm = &rec->evlist->mmap[i].auxtrace_mmap;
+	if (!evlist)
+		return 0;
+
+	for (i = 0; i < evlist->nr_mmaps; i++) {
+		struct auxtrace_mmap *mm = &evlist->mmap[i].auxtrace_mmap;
 
-		if (rec->evlist->mmap[i].base) {
-			if (record__mmap_read(rec, i) != 0) {
+		if (evlist->mmap[i].base) {
+			if (record__mmap_read(rec, evlist, i) != 0) {
 				rc = -1;
 				goto out;
 			}
@@ -532,6 +535,17 @@ out:
 	return rc;
 }
 
+static int record__mmap_read_all(struct record *rec)
+{
+	int err;
+
+	err = record__mmap_read_evlist(rec, rec->evlist);
+	if (err)
+		return err;
+
+	return err;
+}
+
 static void record__init_features(struct record *rec)
 {
 	struct perf_session *session = rec->session;
-- 
2.7.4

[PATCH 10/18] perf record: Prepare picking perf_event_mmap_page from multiple evlists

[Arnaldo Carvalho de Melo]

From: Wang Nan <wangnan0@huawei.com>

Following commits introduce new evlists to record. This patch adjusts
record__pick_pc() and introduces perf_evlist__pick_pc() to read control
page from one specific evlist. record__pick_pc() will be improved to
search control page from multiple evlists.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Nilay Vaish <nilayvaish@gmail.com>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1467023052-146749-4-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/builtin-record.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 18e9abc90759..b2b3b600adf5 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -686,10 +686,21 @@ perf_event__synth_time_conv(const struct perf_event_mmap_page *pc __maybe_unused
 	return 0;
 }
 
+static const struct perf_event_mmap_page *
+perf_evlist__pick_pc(struct perf_evlist *evlist)
+{
+	if (evlist && evlist->mmap && evlist->mmap[0].base)
+		return evlist->mmap[0].base;
+	return NULL;
+}
+
 static const struct perf_event_mmap_page *record__pick_pc(struct record *rec)
 {
-	if (rec->evlist && rec->evlist->mmap && rec->evlist->mmap[0].base)
-		return rec->evlist->mmap[0].base;
+	const struct perf_event_mmap_page *pc;
+
+	pc = perf_evlist__pick_pc(rec->evlist);
+	if (pc)
+		return pc;
 	return NULL;
 }
 
-- 
2.7.4

[PATCH 11/18] perf symbols: Use proper dso name for is_regular_file

[Arnaldo Carvalho de Melo]

From: Jiri Olsa <jolsa@redhat.com>

Marc reported use of uninitialized memory:

> In commit "403567217d3f perf symbols: Do not read symbols/data from
> device files" a check to uninitialzied memory was added. This leads to
> the following valgrind output:
>
>  ==24515== Syscall param stat(file_name) points to uninitialised byte(s)
>  ==24515==    at 0x75B26D5: _xstat (in /lib/x86_64-linux-gnu/libc-2.22.so)
>  ==24515==    by 0x4E548D: stat (stat.h:454)
>  ==24515==    by 0x4E548D: is_regular_file (util.c:687)
>  ==24515==    by 0x4A5BEE: dso__load (symbol.c:1435)
>  ==24515==    by 0x4BB1AE: map__load (map.c:289)
>  ==24515==    by 0x4BB1AE: map__find_symbol (map.c:333)
>  ==24515==    by 0x4835B3: thread__find_addr_location (event.c:1300)
>  ==24515==    by 0x4B5342: add_callchain_ip (machine.c:1652)
>  ==24515==    by 0x4B5342: thread__resolve_callchain_sample (machine.c:1906)
>  ==24515==    by 0x4B9E7D: thread__resolve_callchain (machine.c:1958)
>  ==24515==    by 0x441B3E: process_event (builtin-script.c:795)
>  ==24515==    by 0x441B3E: process_sample_event (builtin-script.c:920)
>  ==24515==    by 0x4BEE29: perf_evlist__deliver_sample (session.c:1192)
>  ==24515==    by 0x4BEE29: machines__deliver_event (session.c:1229)
>  ==24515==    by 0x4BF770: perf_session__deliver_event (session.c:1286)
>  ==24515==    by 0x4BF770: ordered_events__deliver_event (session.c:114)
>  ==24515==    by 0x4C1D17: __ordered_events__flush (ordered-events.c:207)
>  ==24515==    by 0x4C1D17: ordered_events__flush.part.3 (ordered-events.c:274)
>  ==24515==    by 0x4BF44C: perf_session__process_user_event (session.c:1325)
>  ==24515==    by 0x4BF44C: perf_session__process_event (session.c:1451)
>  ==24515==  Address 0x807c6a0 is 0 bytes inside a block of size 4,096 alloc'd
>  ==24515==    at 0x4C29C0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
>  ==24515==    by 0x4A5BCB: dso__load (symbol.c:1421)
>  ==24515==    by 0x4BB1AE: map__load (map.c:289)
>  ==24515==    by 0x4BB1AE: map__find_symbol (map.c:333)
>  ==24515==    by 0x4835B3: thread__find_addr_location (event.c:1300)
>  ==24515==    by 0x4B5342: add_callchain_ip (machine.c:1652)
>  ==24515==    by 0x4B5342: thread__resolve_callchain_sample (machine.c:1906)
>  ==24515==    by 0x4B9E7D: thread__resolve_callchain (machine.c:1958)
>  ==24515==    by 0x441B3E: process_event (builtin-script.c:795)
>  ==24515==    by 0x441B3E: process_sample_event (builtin-script.c:920)
>  ==24515==    by 0x4BEE29: perf_evlist__deliver_sample (session.c:1192)
>  ==24515==    by 0x4BEE29: machines__deliver_event (session.c:1229)
>  ==24515==    by 0x4BF770: perf_session__deliver_event (session.c:1286)
>  ==24515==    by 0x4BF770: ordered_events__deliver_event (session.c:114)
>  ==24515==    by 0x4C1D17: __ordered_events__flush (ordered-events.c:207)
>  ==24515==    by 0x4C1D17: ordered_events__flush.part.3 (ordered-events.c:274)
>  ==24515==    by 0x4BF44C: perf_session__process_user_event (session.c:1325)
>  ==24515==    by 0x4BF44C: perf_session__process_event (session.c:1451)
>  ==24515==    by 0x4C0EAC: __perf_session__process_events (session.c:1804)
>  ==24515==    by 0x4C0EAC: perf_session__process_events (session.c:1858)

The reason was a typo that passed global 'name' variable as the
is_regular_file argument instead dso->long_name.

Reported-by: Marc Kleine-Budde <mkl@pengutronix.de>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Marc Kleine-Budde <mkl@pengutronix.de>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Fixes: 403567217d3f ("perf symbols: Do not read symbols/data from device files")
Link: http://lkml.kernel.org/r/1466772025-17471-2-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/symbol.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index b044f1a32d16..37e8d20ae03e 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -1430,7 +1430,7 @@ int dso__load(struct dso *dso, struct map *map, symbol_filter_t filter)
 	 * Read the build id if possible. This is required for
 	 * DSO_BINARY_TYPE__BUILDID_DEBUGINFO to work
 	 */
-	if (is_regular_file(name) &&
+	if (is_regular_file(dso->long_name) &&
 	    filename__read_build_id(dso->long_name, build_id, BUILD_ID_SIZE) > 0)
 		dso__set_build_id(dso, build_id);
 
-- 
2.7.4

[PATCH 12/18] perf data ctf: Add value_set_string() helper

[Arnaldo Carvalho de Melo]

From: Wang Nan <wangnan0@huawei.com>

There are many value_set_##x helper for integer, but only for integer.
This patch adds value_set_string() helper to help following commits
create string fields.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1466767332-114472-2-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/data-convert-bt.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index 7b1bc24c382e..4b68e7b9ee0c 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -141,6 +141,36 @@ FUNC_VALUE_SET(s64)
 FUNC_VALUE_SET(u64)
 __FUNC_VALUE_SET(u64_hex, u64)
 
+static int string_set_value(struct bt_ctf_field *field, const char *string);
+static __maybe_unused int
+value_set_string(struct ctf_writer *cw, struct bt_ctf_event *event,
+		 const char *name, const char *string)
+{
+	struct bt_ctf_field_type *type = cw->data.string;
+	struct bt_ctf_field *field;
+	int ret = 0;
+
+	field = bt_ctf_field_create(type);
+	if (!field) {
+		pr_err("failed to create a field %s\n", name);
+		return -1;
+	}
+
+	ret = string_set_value(field, string);
+	if (ret) {
+		pr_err("failed to set value %s\n", name);
+		goto err_put_field;
+	}
+
+	ret = bt_ctf_event_set_payload(event, name, field);
+	if (ret)
+		pr_err("failed to set payload %s\n", name);
+
+err_put_field:
+	bt_ctf_field_put(field);
+	return ret;
+}
+
 static struct bt_ctf_field_type*
 get_tracepoint_field_type(struct ctf_writer *cw, struct format_field *field)
 {
-- 
2.7.4

[PATCH 13/18] perf data ctf: Pass convert options through opts structure

[Arnaldo Carvalho de Melo]

From: Wang Nan <wangnan0@huawei.com>

Following commits will add new option to 'perf data convert'. All options
should be grouped into a structure and passed to low level converter
(currently there's only one converter).

Introduce data-convert.h and define 'struct perf_data_convert_opts' in
it. Pass 'force' through opts.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1466767332-114472-3-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/builtin-data.c         | 9 ++++++---
 tools/perf/util/data-convert-bt.c | 5 +++--
 tools/perf/util/data-convert-bt.h | 4 +++-
 tools/perf/util/data-convert.h    | 8 ++++++++
 4 files changed, 20 insertions(+), 6 deletions(-)
 create mode 100644 tools/perf/util/data-convert.h

diff --git a/tools/perf/builtin-data.c b/tools/perf/builtin-data.c
index b97bc1518b44..38111a97d900 100644
--- a/tools/perf/builtin-data.c
+++ b/tools/perf/builtin-data.c
@@ -3,6 +3,7 @@
 #include "perf.h"
 #include "debug.h"
 #include <subcmd/parse-options.h>
+#include "data-convert.h"
 #include "data-convert-bt.h"
 
 typedef int (*data_cmd_fn_t)(int argc, const char **argv, const char *prefix);
@@ -53,14 +54,16 @@ static int cmd_data_convert(int argc, const char **argv,
 			    const char *prefix __maybe_unused)
 {
 	const char *to_ctf     = NULL;
-	bool force = false;
+	struct perf_data_convert_opts opts = {
+		.force = false,
+	};
 	const struct option options[] = {
 		OPT_INCR('v', "verbose", &verbose, "be more verbose"),
 		OPT_STRING('i', "input", &input_name, "file", "input file name"),
 #ifdef HAVE_LIBBABELTRACE_SUPPORT
 		OPT_STRING(0, "to-ctf", &to_ctf, NULL, "Convert to CTF format"),
 #endif
-		OPT_BOOLEAN('f', "force", &force, "don't complain, do it"),
+		OPT_BOOLEAN('f', "force", &opts.force, "don't complain, do it"),
 		OPT_END()
 	};
 
@@ -78,7 +81,7 @@ static int cmd_data_convert(int argc, const char **argv,
 
 	if (to_ctf) {
 #ifdef HAVE_LIBBABELTRACE_SUPPORT
-		return bt_convert__perf2ctf(input_name, to_ctf, force);
+		return bt_convert__perf2ctf(input_name, to_ctf, &opts);
 #else
 		pr_err("The libbabeltrace support is not compiled in.\n");
 		return -1;
diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index 4b68e7b9ee0c..09571b39f58a 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -1304,13 +1304,14 @@ static int convert__config(const char *var, const char *value, void *cb)
 	return 0;
 }
 
-int bt_convert__perf2ctf(const char *input, const char *path, bool force)
+int bt_convert__perf2ctf(const char *input, const char *path,
+			 struct perf_data_convert_opts *opts)
 {
 	struct perf_session *session;
 	struct perf_data_file file = {
 		.path = input,
 		.mode = PERF_DATA_MODE_READ,
-		.force = force,
+		.force = opts->force,
 	};
 	struct convert c = {
 		.tool = {
diff --git a/tools/perf/util/data-convert-bt.h b/tools/perf/util/data-convert-bt.h
index 4c204342a9d8..9a3b587f76c1 100644
--- a/tools/perf/util/data-convert-bt.h
+++ b/tools/perf/util/data-convert-bt.h
@@ -1,8 +1,10 @@
 #ifndef __DATA_CONVERT_BT_H
 #define __DATA_CONVERT_BT_H
+#include "data-convert.h"
 #ifdef HAVE_LIBBABELTRACE_SUPPORT
 
-int bt_convert__perf2ctf(const char *input_name, const char *to_ctf, bool force);
+int bt_convert__perf2ctf(const char *input_name, const char *to_ctf,
+			 struct perf_data_convert_opts *opts);
 
 #endif /* HAVE_LIBBABELTRACE_SUPPORT */
 #endif /* __DATA_CONVERT_BT_H */
diff --git a/tools/perf/util/data-convert.h b/tools/perf/util/data-convert.h
new file mode 100644
index 000000000000..97cfd36aab6f
--- /dev/null
+++ b/tools/perf/util/data-convert.h
@@ -0,0 +1,8 @@
+#ifndef __DATA_CONVERT_H
+#define __DATA_CONVERT_H
+
+struct perf_data_convert_opts {
+	bool force;
+};
+
+#endif /* __DATA_CONVERT_H */
-- 
2.7.4

[PATCH 14/18] perf data ctf: Add 'all' option

[Arnaldo Carvalho de Melo]

From: Wang Nan <wangnan0@huawei.com>

If 'all' option is selected, 'perf data convert' should convert not only
samples, but non-sample events such as comm and fork. Add this option in
perf_data_convert_opts. Following commits will add cmdline option to
select it.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1466767332-114472-4-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/builtin-data.c      | 1 +
 tools/perf/util/data-convert.h | 1 +
 2 files changed, 2 insertions(+)

diff --git a/tools/perf/builtin-data.c b/tools/perf/builtin-data.c
index 38111a97d900..ddfe3ac4c7f7 100644
--- a/tools/perf/builtin-data.c
+++ b/tools/perf/builtin-data.c
@@ -56,6 +56,7 @@ static int cmd_data_convert(int argc, const char **argv,
 	const char *to_ctf     = NULL;
 	struct perf_data_convert_opts opts = {
 		.force = false,
+		.all = false,
 	};
 	const struct option options[] = {
 		OPT_INCR('v', "verbose", &verbose, "be more verbose"),
diff --git a/tools/perf/util/data-convert.h b/tools/perf/util/data-convert.h
index 97cfd36aab6f..5314962fe95b 100644
--- a/tools/perf/util/data-convert.h
+++ b/tools/perf/util/data-convert.h
@@ -3,6 +3,7 @@
 
 struct perf_data_convert_opts {
 	bool force;
+	bool all;
 };
 
 #endif /* __DATA_CONVERT_H */
-- 
2.7.4

[PATCH 15/18] perf data ctf: Prepare collect non-sample events

[Arnaldo Carvalho de Melo]

From: Wang Nan <wangnan0@huawei.com>

Following commits are going to allow 'perf data convert' to collect not
only samples, but also non-sample events like comm and fork. In this
patch we count non-sample events using c.non_sample_count, and prepare
to print number of both type of events like:

  # ~/perf data convert --all --to-ctf ./out.ctf
  [ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ]
  [ perf data convert: Converted and wrote 0.846 MB (6508 samples, 686 non-samples) ]

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1466767332-114472-5-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/data-convert-bt.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index 09571b39f58a..3b3ac7c143e1 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -77,6 +77,7 @@ struct convert {
 
 	u64			events_size;
 	u64			events_count;
+	u64			non_sample_count;
 
 	/* Ordered events configured queue size. */
 	u64			queue_size;
@@ -1369,10 +1370,15 @@ int bt_convert__perf2ctf(const char *input, const char *path,
 		file.path, path);
 
 	fprintf(stderr,
-		"[ perf data convert: Converted and wrote %.3f MB (%" PRIu64 " samples) ]\n",
+		"[ perf data convert: Converted and wrote %.3f MB (%" PRIu64 " samples",
 		(double) c.events_size / 1024.0 / 1024.0,
 		c.events_count);
 
+	if (!c.non_sample_count)
+		fprintf(stderr, ") ]\n");
+	else
+		fprintf(stderr, ", %" PRIu64 " non-samples) ]\n", c.non_sample_count);
+
 	cleanup_events(session);
 	perf_session__delete(session);
 	ctf_writer__cleanup(cw);
-- 
2.7.4

[PATCH 16/18] perf data ctf: Generate comm event to CTF output

[Arnaldo Carvalho de Melo]

From: Wang Nan <wangnan0@huawei.com>

If 'all' is selected, convert comm event to output CTF stream.

setup_non_sample_events() is called if non_sample is selected. It
creates a comm_class for comm event.

Use macros to generate and process_comm_event and add_comm_event. These
macros can be reused for other non-sample events.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1466767332-114472-6-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/data-convert-bt.c | 110 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 110 insertions(+)

diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index 3b3ac7c143e1..5dd62ba07438 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -69,6 +69,7 @@ struct ctf_writer {
 		};
 		struct bt_ctf_field_type *array[6];
 	} data;
+	struct bt_ctf_event_class	*comm_class;
 };
 
 struct convert {
@@ -763,6 +764,57 @@ static int process_sample_event(struct perf_tool *tool,
 	return cs ? 0 : -1;
 }
 
+#define __NON_SAMPLE_SET_FIELD(_name, _type, _field) 	\
+do {							\
+	ret = value_set_##_type(cw, event, #_field, _event->_name._field);\
+	if (ret)					\
+		return -1;				\
+} while(0)
+
+#define __FUNC_PROCESS_NON_SAMPLE(_name, body) 	\
+static int process_##_name##_event(struct perf_tool *tool,	\
+				   union perf_event *_event,	\
+				   struct perf_sample *sample,	\
+				   struct machine *machine)	\
+{								\
+	struct convert *c = container_of(tool, struct convert, tool);\
+	struct ctf_writer *cw = &c->writer;			\
+	struct bt_ctf_event_class *event_class = cw->_name##_class;\
+	struct bt_ctf_event *event;				\
+	struct ctf_stream *cs;					\
+	int ret;						\
+								\
+	c->non_sample_count++;					\
+	c->events_size += _event->header.size;			\
+	event = bt_ctf_event_create(event_class);		\
+	if (!event) {						\
+		pr_err("Failed to create an CTF event\n");	\
+		return -1;					\
+	}							\
+								\
+	bt_ctf_clock_set_time(cw->clock, sample->time);		\
+	body							\
+	cs = ctf_stream(cw, 0);					\
+	if (cs) {						\
+		if (is_flush_needed(cs))			\
+			ctf_stream__flush(cs);			\
+								\
+		cs->count++;					\
+		bt_ctf_stream_append_event(cs->stream, event);	\
+	}							\
+	bt_ctf_event_put(event);				\
+								\
+	return perf_event__process_##_name(tool, _event, sample, machine);\
+}
+
+__FUNC_PROCESS_NON_SAMPLE(comm,
+	__NON_SAMPLE_SET_FIELD(comm, u32, pid);
+	__NON_SAMPLE_SET_FIELD(comm, u32, tid);
+	__NON_SAMPLE_SET_FIELD(comm, string, comm);
+)
+#undef __NON_SAMPLE_SET_FIELD
+#undef __FUNC_PROCESS_NON_SAMPLE
+
 /* If dup < 0, add a prefix. Else, add _dupl_X suffix. */
 static char *change_name(char *name, char *orig_name, int dup)
 {
@@ -1037,6 +1089,58 @@ static int setup_events(struct ctf_writer *cw, struct perf_session *session)
 	return 0;
 }
 
+#define __NON_SAMPLE_ADD_FIELD(t, n)						\
+	do {							\
+		pr2("  field '%s'\n", #n);			\
+		if (bt_ctf_event_class_add_field(event_class, cw->data.t, #n)) {\
+			pr_err("Failed to add field '%s';\n", #n);\
+			return -1;				\
+		}						\
+	} while(0)
+
+#define __FUNC_ADD_NON_SAMPLE_EVENT_CLASS(_name, body) 		\
+static int add_##_name##_event(struct ctf_writer *cw)		\
+{								\
+	struct bt_ctf_event_class *event_class;			\
+	int ret;						\
+								\
+	pr("Adding "#_name" event\n");				\
+	event_class = bt_ctf_event_class_create("perf_" #_name);\
+	if (!event_class)					\
+		return -1;					\
+	body							\
+								\
+	ret = bt_ctf_stream_class_add_event_class(cw->stream_class, event_class);\
+	if (ret) {						\
+		pr("Failed to add event class '"#_name"' into stream.\n");\
+		return ret;					\
+	}							\
+								\
+	cw->_name##_class = event_class;			\
+	bt_ctf_event_class_put(event_class);			\
+	return 0;						\
+}
+
+__FUNC_ADD_NON_SAMPLE_EVENT_CLASS(comm,
+	__NON_SAMPLE_ADD_FIELD(u32, pid);
+	__NON_SAMPLE_ADD_FIELD(u32, tid);
+	__NON_SAMPLE_ADD_FIELD(string, comm);
+)
+
+#undef __NON_SAMPLE_ADD_FIELD
+#undef __FUNC_ADD_NON_SAMPLE_EVENT_CLASS
+
+static int setup_non_sample_events(struct ctf_writer *cw,
+				   struct perf_session *session __maybe_unused)
+{
+	int ret;
+
+	ret = add_comm_event(cw);
+	if (ret)
+		return ret;
+	return 0;
+}
+
 static void cleanup_events(struct perf_session *session)
 {
 	struct perf_evlist *evlist = session->evlist;
@@ -1332,6 +1436,9 @@ int bt_convert__perf2ctf(const char *input, const char *path,
 	struct ctf_writer *cw = &c.writer;
 	int err = -1;
 
+	if (opts->all)
+		c.tool.comm = process_comm_event;
+
 	perf_config(convert__config, &c);
 
 	/* CTF writer */
@@ -1356,6 +1463,9 @@ int bt_convert__perf2ctf(const char *input, const char *path,
 	if (setup_events(cw, session))
 		goto free_session;
 
+	if (opts->all && setup_non_sample_events(cw, session))
+		goto free_session;
+
 	if (setup_streams(cw, session))
 		goto free_session;
 
-- 
2.7.4

[PATCH 17/18] perf data ctf: Add '--all' option for 'perf data convert'

[Arnaldo Carvalho de Melo]

From: Wang Nan <wangnan0@huawei.com>

After this patch, 'perf data convert' convert comm events to output CTF
stream.

Result:

  # perf record -a sleep 1
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.378 MB perf.data (73 samples)  ]

  # perf data convert --to-ctf ./out.ctf
  [ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ]
  [ perf data convert: Converted and wrote 0.003 MB (73 samples) ]

  # babeltrace --clock-seconds ./out.ctf/
  [10627.402515791] (+?.?????????) cycles:ppp: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF81065AF4, perf_tid = 0, perf_pid = 0, perf_period = 1 }
  [10627.402518972] (+0.000003181) cycles:ppp: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF81065AF4, perf_tid = 0, perf_pid = 0, perf_period = 1 }
  ...    // only sample event is converted

  # perf data convert --all --to-ctf ./out.ctf
  [ perf data convert: Converted 'perf.data' into CTF data './out.ctf' ]
  [ perf data convert: Converted and wrote 0.023 MB (73 samples, 384 non-samples) ]

  # babeltrace --clock-seconds ./out.ctf/
  [  0.000000000] (+?.?????????) perf_comm: { cpu_id = 0 }, { pid = 1, tid = 1, comm = "init" }
  [  0.000000000] (+0.000000000) perf_comm: { cpu_id = 0 }, { pid = 2, tid = 2, comm = "kthreadd" }
  [  0.000000000] (+0.000000000) perf_comm: { cpu_id = 0 }, { pid = 3, tid = 3, comm = "ksoftirqd/0" }
  ...    // comm events are converted
  [10627.402515791] (+10627.402515791) cycles:ppp: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF81065AF4, perf_tid = 0, perf_pid = 0, perf_period = 1 }
  [10627.402518972] (+0.000003181) cycles:ppp: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF81065AF4, perf_tid = 0, perf_pid = 0, perf_period = 1 }
  ...    // samples are also converted

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1466767332-114472-7-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/Documentation/perf-data.txt | 4 ++++
 tools/perf/builtin-data.c              | 1 +
 2 files changed, 5 insertions(+)

diff --git a/tools/perf/Documentation/perf-data.txt b/tools/perf/Documentation/perf-data.txt
index be8fa1a0a97e..f0796a47dfa3 100644
--- a/tools/perf/Documentation/perf-data.txt
+++ b/tools/perf/Documentation/perf-data.txt
@@ -34,6 +34,10 @@ OPTIONS for 'convert'
 --verbose::
         Be more verbose (show counter open errors, etc).
 
+--all::
+	Convert all events, including non-sample events (comm, fork, ...), to output.
+	Default is off, only convert samples.
+
 SEE ALSO
 --------
 linkperf:perf[1]
diff --git a/tools/perf/builtin-data.c b/tools/perf/builtin-data.c
index ddfe3ac4c7f7..7ad6e17ac6b3 100644
--- a/tools/perf/builtin-data.c
+++ b/tools/perf/builtin-data.c
@@ -65,6 +65,7 @@ static int cmd_data_convert(int argc, const char **argv,
 		OPT_STRING(0, "to-ctf", &to_ctf, NULL, "Convert to CTF format"),
 #endif
 		OPT_BOOLEAN('f', "force", &opts.force, "don't complain, do it"),
+		OPT_BOOLEAN(0, "all", &opts.all, "Convert all events"),
 		OPT_END()
 	};
 
-- 
2.7.4

[PATCH 18/18] perf data ctf: Generate fork and exit events to CTF output

[Arnaldo Carvalho de Melo]

From: Wang Nan <wangnan0@huawei.com>

If 'all' is selected, convert fork and exit events to output CTF stream.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1466767332-114472-8-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/data-convert-bt.c | 44 ++++++++++++++++++++++++++++++++++++++-
 1 file changed, 43 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index 5dd62ba07438..4f979bb27b6c 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -70,6 +70,8 @@ struct ctf_writer {
 		struct bt_ctf_field_type *array[6];
 	} data;
 	struct bt_ctf_event_class	*comm_class;
+	struct bt_ctf_event_class	*exit_class;
+	struct bt_ctf_event_class	*fork_class;
 };
 
 struct convert {
@@ -812,6 +814,21 @@ __FUNC_PROCESS_NON_SAMPLE(comm,
 	__NON_SAMPLE_SET_FIELD(comm, u32, tid);
 	__NON_SAMPLE_SET_FIELD(comm, string, comm);
 )
+__FUNC_PROCESS_NON_SAMPLE(fork,
+	__NON_SAMPLE_SET_FIELD(fork, u32, pid);
+	__NON_SAMPLE_SET_FIELD(fork, u32, ppid);
+	__NON_SAMPLE_SET_FIELD(fork, u32, tid);
+	__NON_SAMPLE_SET_FIELD(fork, u32, ptid);
+	__NON_SAMPLE_SET_FIELD(fork, u64, time);
+)
+
+__FUNC_PROCESS_NON_SAMPLE(exit,
+	__NON_SAMPLE_SET_FIELD(fork, u32, pid);
+	__NON_SAMPLE_SET_FIELD(fork, u32, ppid);
+	__NON_SAMPLE_SET_FIELD(fork, u32, tid);
+	__NON_SAMPLE_SET_FIELD(fork, u32, ptid);
+	__NON_SAMPLE_SET_FIELD(fork, u64, time);
+)
 #undef __NON_SAMPLE_SET_FIELD
 #undef __FUNC_PROCESS_NON_SAMPLE
 
@@ -1127,6 +1144,22 @@ __FUNC_ADD_NON_SAMPLE_EVENT_CLASS(comm,
 	__NON_SAMPLE_ADD_FIELD(string, comm);
 )
 
+__FUNC_ADD_NON_SAMPLE_EVENT_CLASS(fork,
+	__NON_SAMPLE_ADD_FIELD(u32, pid);
+	__NON_SAMPLE_ADD_FIELD(u32, ppid);
+	__NON_SAMPLE_ADD_FIELD(u32, tid);
+	__NON_SAMPLE_ADD_FIELD(u32, ptid);
+	__NON_SAMPLE_ADD_FIELD(u64, time);
+)
+
+__FUNC_ADD_NON_SAMPLE_EVENT_CLASS(exit,
+	__NON_SAMPLE_ADD_FIELD(u32, pid);
+	__NON_SAMPLE_ADD_FIELD(u32, ppid);
+	__NON_SAMPLE_ADD_FIELD(u32, tid);
+	__NON_SAMPLE_ADD_FIELD(u32, ptid);
+	__NON_SAMPLE_ADD_FIELD(u64, time);
+)
+
 #undef __NON_SAMPLE_ADD_FIELD
 #undef __FUNC_ADD_NON_SAMPLE_EVENT_CLASS
 
@@ -1138,6 +1171,12 @@ static int setup_non_sample_events(struct ctf_writer *cw,
 	ret = add_comm_event(cw);
 	if (ret)
 		return ret;
+	ret = add_exit_event(cw);
+	if (ret)
+		return ret;
+	ret = add_fork_event(cw);
+	if (ret)
+		return ret;
 	return 0;
 }
 
@@ -1436,8 +1475,11 @@ int bt_convert__perf2ctf(const char *input, const char *path,
 	struct ctf_writer *cw = &c.writer;
 	int err = -1;
 
-	if (opts->all)
+	if (opts->all) {
 		c.tool.comm = process_comm_event;
+		c.tool.exit = process_exit_event;
+		c.tool.fork = process_fork_event;
+	}
 
 	perf_config(convert__config, &c);
 
-- 
2.7.4

Re: [GIT PULL 00/18] perf/core improvements and fixes

[Ingo Molnar]

* Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> Hi Ingo,
> 
> 	Please consider pulling,
> 
> - Arnaldo
> 
> The following changes since commit d4cf1949f9689314aef962eea95df84a8288d097:
> 
>   perf/x86/intel: Add {rd,wr}lbr_{to,from} wrappers (2016-06-27 11:34:21 +0200)
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-20160627
> 
> for you to fetch changes up to 6d9c675078e697309c1c06a1051f01de8151c476:
> 
>   perf data ctf: Generate fork and exit events to CTF output (2016-06-27 15:50:23 -0300)
> 
> ----------------------------------------------------------------
> perf/core improvements and fixes:
> 
> New features:
> 
> - Generate comm, fork and exit events when converting perf.data files to CTF (Wang Nan)
> 
> Documentation:
> 
> - Document perf.data on disk format (Andi Kleen)
> 
> Infrastructure:
> 
> - Add libbabeltrace to build-test (Wang Nan)
> 
> - 'perf record' prep work to support multiple evlists (Wang Nan)
> 
> - Remove unused hist_entry__annotate function (Ravi Bangoria)
> 
> - Add more toolchain triplets (Ravi Bangoria)
> 
> - Update message for slang devel packages on Ubuntu (Neeraj Badlani)
> 
> - Generalize handling of 'ret' instructions in the annotate TUI (Naveen N. Rao)
> 
> - Use proper dso name for is_regular_file, fixing device file handling (Jiri Olsa)
> 
> Build Fixes:
> 
> - Add missing config.h include, fixing the build with libabeltrace (Jiri Olsa)
> 
> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> 
> ----------------------------------------------------------------
> Andi Kleen (1):
>       perf tools: Add documentation for perf.data on disk format

The doc build fails with:

asciidoc: FAILED: manpage document title is mandatory
Makefile:276: recipe for target 'perf-file-format.xml' failed
make[2]: *** [perf-file-format.xml] Error 1
make[2]: *** Waiting for unfinished jobs....

Thanks,

	Ingo

Re: [GIT PULL 00/18] perf/core improvements and fixes

[Jiri Olsa]

On Tue, Jun 28, 2016 at 09:02:57AM +0200, Ingo Molnar wrote:

SNIP

> > - Add missing config.h include, fixing the build with libabeltrace (Jiri Olsa)
> > 
> > Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> > 
> > ----------------------------------------------------------------
> > Andi Kleen (1):
> >       perf tools: Add documentation for perf.data on disk format
> 
> The doc build fails with:
> 
> asciidoc: FAILED: manpage document title is mandatory
> Makefile:276: recipe for target 'perf-file-format.xml' failed
> make[2]: *** [perf-file-format.xml] Error 1
> make[2]: *** Waiting for unfinished jobs....
> 

hum, my asciidoc does not complain

jirka

Re: [GIT PULL 00/18] perf/core improvements and fixes

[Arnaldo Carvalho de Melo]

Em Tue, Jun 28, 2016 at 09:02:57AM +0200, Ingo Molnar escreveu:
> 
> * Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> 
> > Hi Ingo,
> > 
> > 	Please consider pulling,
> > 
> > - Arnaldo
> > 
> > The following changes since commit d4cf1949f9689314aef962eea95df84a8288d097:
> > 
> >   perf/x86/intel: Add {rd,wr}lbr_{to,from} wrappers (2016-06-27 11:34:21 +0200)
> > 
> > are available in the git repository at:
> > 
> >   git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-20160627
> > 
> > for you to fetch changes up to 6d9c675078e697309c1c06a1051f01de8151c476:
> > 
> >   perf data ctf: Generate fork and exit events to CTF output (2016-06-27 15:50:23 -0300)
> > 
> > ----------------------------------------------------------------
> > perf/core improvements and fixes:
> > 
> > New features:
> > 
> > - Generate comm, fork and exit events when converting perf.data files to CTF (Wang Nan)
> > 
> > Documentation:
> > 
> > - Document perf.data on disk format (Andi Kleen)
> > 
> > Infrastructure:
> > 
> > - Add libbabeltrace to build-test (Wang Nan)
> > 
> > - 'perf record' prep work to support multiple evlists (Wang Nan)
> > 
> > - Remove unused hist_entry__annotate function (Ravi Bangoria)
> > 
> > - Add more toolchain triplets (Ravi Bangoria)
> > 
> > - Update message for slang devel packages on Ubuntu (Neeraj Badlani)
> > 
> > - Generalize handling of 'ret' instructions in the annotate TUI (Naveen N. Rao)
> > 
> > - Use proper dso name for is_regular_file, fixing device file handling (Jiri Olsa)
> > 
> > Build Fixes:
> > 
> > - Add missing config.h include, fixing the build with libabeltrace (Jiri Olsa)
> > 
> > Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> > 
> > ----------------------------------------------------------------
> > Andi Kleen (1):
> >       perf tools: Add documentation for perf.data on disk format
> 
> The doc build fails with:
> 
> asciidoc: FAILED: manpage document title is mandatory
> Makefile:276: recipe for target 'perf-file-format.xml' failed
> make[2]: *** [perf-file-format.xml] Error 1
> make[2]: *** Waiting for unfinished jobs....

Ok, I've created a perf-core-for-mingo-20160628 with that patch removed
while we investigate this, please consider pulling that one instead.

build-test should've caught this... Reproduced, trying to fix...

- Arnaldo

Re: [GIT PULL 00/18] perf/core improvements and fixes

[Arnaldo Carvalho de Melo]

Em Tue, Jun 28, 2016 at 11:01:55AM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Tue, Jun 28, 2016 at 09:02:57AM +0200, Ingo Molnar escreveu:
> > * Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > > Andi Kleen (1):
> > >       perf tools: Add documentation for perf.data on disk format

> > The doc build fails with:

> > asciidoc: FAILED: manpage document title is mandatory
> > Makefile:276: recipe for target 'perf-file-format.xml' failed
> > make[2]: *** [perf-file-format.xml] Error 1
> > make[2]: *** Waiting for unfinished jobs....
 
> Ok, I've created a perf-core-for-mingo-20160628 with that patch removed
> while we investigate this, please consider pulling that one instead.
 
> build-test should've caught this... Reproduced, trying to fix...

So, when doing the docs install it expects files starting with perf-*
will be fed to asciidoc to generate man pages, etc, so the quick fix
here is to rename perf-file-format.txt to perf.data-file-format.txt,
which I did, so this will go in the next pull request.

Volunteers accepted to turn this into asciidoc format so that we can do
a man perf-data-file-format or somesuch.

- Arnaldo

Re: [GIT PULL 00/18] perf/core improvements and fixes

[Ingo Molnar]

* Arnaldo Carvalho de Melo <acme@kernel.org> wrote:

> Em Tue, Jun 28, 2016 at 09:02:57AM +0200, Ingo Molnar escreveu:
> > 
> > * Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > 
> > > Hi Ingo,
> > > 
> > > 	Please consider pulling,
> > > 
> > > - Arnaldo
> > > 
> > > The following changes since commit d4cf1949f9689314aef962eea95df84a8288d097:
> > > 
> > >   perf/x86/intel: Add {rd,wr}lbr_{to,from} wrappers (2016-06-27 11:34:21 +0200)
> > > 
> > > are available in the git repository at:
> > > 
> > >   git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-20160627
> > > 
> > > for you to fetch changes up to 6d9c675078e697309c1c06a1051f01de8151c476:
> > > 
> > >   perf data ctf: Generate fork and exit events to CTF output (2016-06-27 15:50:23 -0300)
> > > 
> > > ----------------------------------------------------------------
> > > perf/core improvements and fixes:
> > > 
> > > New features:
> > > 
> > > - Generate comm, fork and exit events when converting perf.data files to CTF (Wang Nan)
> > > 
> > > Documentation:
> > > 
> > > - Document perf.data on disk format (Andi Kleen)
> > > 
> > > Infrastructure:
> > > 
> > > - Add libbabeltrace to build-test (Wang Nan)
> > > 
> > > - 'perf record' prep work to support multiple evlists (Wang Nan)
> > > 
> > > - Remove unused hist_entry__annotate function (Ravi Bangoria)
> > > 
> > > - Add more toolchain triplets (Ravi Bangoria)
> > > 
> > > - Update message for slang devel packages on Ubuntu (Neeraj Badlani)
> > > 
> > > - Generalize handling of 'ret' instructions in the annotate TUI (Naveen N. Rao)
> > > 
> > > - Use proper dso name for is_regular_file, fixing device file handling (Jiri Olsa)
> > > 
> > > Build Fixes:
> > > 
> > > - Add missing config.h include, fixing the build with libabeltrace (Jiri Olsa)
> > > 
> > > Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> > > 
> > > ----------------------------------------------------------------
> > > Andi Kleen (1):
> > >       perf tools: Add documentation for perf.data on disk format
> > 
> > The doc build fails with:
> > 
> > asciidoc: FAILED: manpage document title is mandatory
> > Makefile:276: recipe for target 'perf-file-format.xml' failed
> > make[2]: *** [perf-file-format.xml] Error 1
> > make[2]: *** Waiting for unfinished jobs....
> 
> Ok, I've created a perf-core-for-mingo-20160628 with that patch removed
> while we investigate this, please consider pulling that one instead.

Pulled, thanks a lot Arnaldo!

	Ingo