All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/5] perf record: Put a copy of kcore into the perf.data directory
@ 2019-10-04  8:31 Adrian Hunter
  2019-10-04  8:31 ` [PATCH 1/5] perf data: Correctly identify directory data files Adrian Hunter
                   ` (6 more replies)
  0 siblings, 7 replies; 24+ messages in thread
From: Adrian Hunter @ 2019-10-04  8:31 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo, Jiri Olsa; +Cc: linux-kernel

Hi

Here are patches to add a new 'perf record' option '--kcore' which will put
a copy of /proc/kcore, kallsyms and modules into a perf.data directory.
Note, that without the --kcore option, output goes to a file as previously.
The tools' -o and -i options work with either a file name or directory
name.

Example:

 $ sudo perf record --kcore uname

 $ sudo tree perf.data
 perf.data
 ├── kcore_dir
 │   ├── kallsyms
 │   ├── kcore
 │   └── modules
 └── data

 $ sudo perf script -v
 build id event received for vmlinux: 1eaa285996affce2d74d8e66dcea09a80c9941de
 build id event received for [vdso]: 8bbaf5dc62a9b644b4d4e4539737e104e4a84541
 Samples for 'cycles' event do not have CPU attribute set. Skipping 'cpu' field.
 Using CPUID GenuineIntel-6-8E-A
 Using perf.data/kcore_dir/kcore for kernel data
 Using perf.data/kcore_dir/kallsyms for symbols
             perf 19058 506778.423729:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423733:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423734:          7 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423736:        117 cycles:  ffffffffa2caa54a native_write_msr+0xa (vmlinux)
             perf 19058 506778.423738:       2092 cycles:  ffffffffa2c9b7b0 native_apic_msr_write+0x0 (vmlinux)
             perf 19058 506778.423740:      37380 cycles:  ffffffffa2f121d0 perf_event_addr_filters_exec+0x0 (vmlinux)
            uname 19058 506778.423751:     582673 cycles:  ffffffffa303a407 propagate_protected_usage+0x147 (vmlinux)
            uname 19058 506778.423892:    2241841 cycles:  ffffffffa2cae0c9 unwind_next_frame.part.5+0x79 (vmlinux)
            uname 19058 506778.424430:    2457397 cycles:  ffffffffa3019232 check_memory_region+0x52 (vmlinux)


Adrian Hunter (5):
      perf data: Correctly identify directory data files
      perf data: Move perf_dir_version into data.h
      perf data: Rename directory "header" file to "data"
      perf tools: Support single perf.data file directory
      perf record: Put a copy of kcore into the perf.data directory

 tools/perf/Documentation/perf-record.txt           |  3 ++
 .../Documentation/perf.data-directory-format.txt   | 63 ++++++++++++++++++++++
 tools/perf/builtin-record.c                        | 54 ++++++++++++++++++-
 tools/perf/util/data.c                             | 46 ++++++++++++++--
 tools/perf/util/data.h                             | 12 +++++
 tools/perf/util/header.h                           |  4 --
 tools/perf/util/record.h                           |  1 +
 tools/perf/util/session.c                          |  4 ++
 tools/perf/util/util.c                             | 19 ++++++-
 9 files changed, 197 insertions(+), 9 deletions(-)
 create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt


Regards
Adrian

^ permalink raw reply	[flat|nested] 24+ messages in thread

* [PATCH 1/5] perf data: Correctly identify directory data files
  2019-10-04  8:31 [PATCH 0/5] perf record: Put a copy of kcore into the perf.data directory Adrian Hunter
@ 2019-10-04  8:31 ` Adrian Hunter
  2019-11-12 11:18   ` [tip: perf/core] " tip-bot2 for Adrian Hunter
  2019-10-04  8:31 ` [PATCH 2/5] perf data: Move perf_dir_version into data.h Adrian Hunter
                   ` (5 subsequent siblings)
  6 siblings, 1 reply; 24+ messages in thread
From: Adrian Hunter @ 2019-10-04  8:31 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo, Jiri Olsa; +Cc: linux-kernel

In order to rename the "header" file to "data" without conflicting,
correctly identify the non-header files as starting with "data."

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
---
 tools/perf/util/data.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/util/data.c b/tools/perf/util/data.c
index 88fba2ba549f..8993253c5564 100644
--- a/tools/perf/util/data.c
+++ b/tools/perf/util/data.c
@@ -96,7 +96,7 @@ int perf_data__open_dir(struct perf_data *data)
 		if (stat(path, &st))
 			continue;
 
-		if (!S_ISREG(st.st_mode) || strncmp(dent->d_name, "data", 4))
+		if (!S_ISREG(st.st_mode) || strncmp(dent->d_name, "data.", 5))
 			continue;
 
 		ret = -ENOMEM;
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 2/5] perf data: Move perf_dir_version into data.h
  2019-10-04  8:31 [PATCH 0/5] perf record: Put a copy of kcore into the perf.data directory Adrian Hunter
  2019-10-04  8:31 ` [PATCH 1/5] perf data: Correctly identify directory data files Adrian Hunter
@ 2019-10-04  8:31 ` Adrian Hunter
  2019-11-12 11:18   ` [tip: perf/core] " tip-bot2 for Adrian Hunter
  2019-10-04  8:31 ` [PATCH 3/5] perf data: Rename directory "header" file to "data" Adrian Hunter
                   ` (4 subsequent siblings)
  6 siblings, 1 reply; 24+ messages in thread
From: Adrian Hunter @ 2019-10-04  8:31 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo, Jiri Olsa; +Cc: linux-kernel

perf_dir_version belongs to struct perf_data which is declared in data.h.
To allow its use in inline perf_data functions, move perf_dir_version to
data.h

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
---
 tools/perf/util/data.h   | 4 ++++
 tools/perf/util/header.h | 4 ----
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/perf/util/data.h b/tools/perf/util/data.h
index 259868a39019..218fe9a16801 100644
--- a/tools/perf/util/data.h
+++ b/tools/perf/util/data.h
@@ -9,6 +9,10 @@ enum perf_data_mode {
 	PERF_DATA_MODE_READ,
 };
 
+enum perf_dir_version {
+	PERF_DIR_VERSION	= 1,
+};
+
 struct perf_data_file {
 	char		*path;
 	int		 fd;
diff --git a/tools/perf/util/header.h b/tools/perf/util/header.h
index ca53a929e9fd..840f95cee349 100644
--- a/tools/perf/util/header.h
+++ b/tools/perf/util/header.h
@@ -52,10 +52,6 @@ enum perf_header_version {
 	PERF_HEADER_VERSION_2,
 };
 
-enum perf_dir_version {
-	PERF_DIR_VERSION	= 1,
-};
-
 struct perf_file_section {
 	u64 offset;
 	u64 size;
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 3/5] perf data: Rename directory "header" file to "data"
  2019-10-04  8:31 [PATCH 0/5] perf record: Put a copy of kcore into the perf.data directory Adrian Hunter
  2019-10-04  8:31 ` [PATCH 1/5] perf data: Correctly identify directory data files Adrian Hunter
  2019-10-04  8:31 ` [PATCH 2/5] perf data: Move perf_dir_version into data.h Adrian Hunter
@ 2019-10-04  8:31 ` Adrian Hunter
  2019-11-12 11:18   ` [tip: perf/core] " tip-bot2 for Adrian Hunter
  2019-10-04  8:31 ` [PATCH 4/5] perf tools: Support single perf.data file directory Adrian Hunter
                   ` (3 subsequent siblings)
  6 siblings, 1 reply; 24+ messages in thread
From: Adrian Hunter @ 2019-10-04  8:31 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo, Jiri Olsa; +Cc: linux-kernel

In preparation to support a single file directory format, rename "header"
to "data" because "header" is a mis-leading name when there is only 1 file.
Note, in the multi-file case, the "header" file also contains data.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
---
 tools/perf/util/data.c | 2 +-
 tools/perf/util/util.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/data.c b/tools/perf/util/data.c
index 8993253c5564..df173f0bf654 100644
--- a/tools/perf/util/data.c
+++ b/tools/perf/util/data.c
@@ -306,7 +306,7 @@ static int open_dir(struct perf_data *data)
 	 * So far we open only the header, so we can read the data version and
 	 * layout.
 	 */
-	if (asprintf(&data->file.path, "%s/header", data->path) < 0)
+	if (asprintf(&data->file.path, "%s/data", data->path) < 0)
 		return -1;
 
 	if (perf_data__is_write(data) &&
diff --git a/tools/perf/util/util.c b/tools/perf/util/util.c
index 5eda6e19c947..56f30ad29ca7 100644
--- a/tools/perf/util/util.c
+++ b/tools/perf/util/util.c
@@ -183,7 +183,7 @@ static int rm_rf_depth_pat(const char *path, int depth, const char **pat)
 int rm_rf_perf_data(const char *path)
 {
 	const char *pat[] = {
-		"header",
+		"data",
 		"data.*",
 		NULL,
 	};
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 4/5] perf tools: Support single perf.data file directory
  2019-10-04  8:31 [PATCH 0/5] perf record: Put a copy of kcore into the perf.data directory Adrian Hunter
                   ` (2 preceding siblings ...)
  2019-10-04  8:31 ` [PATCH 3/5] perf data: Rename directory "header" file to "data" Adrian Hunter
@ 2019-10-04  8:31 ` Adrian Hunter
  2019-10-07 11:20   ` Jiri Olsa
  2019-11-12 11:18   ` [tip: perf/core] perf data: Support single perf.data file directory tip-bot2 for Adrian Hunter
  2019-10-04  8:31 ` [PATCH 5/5] perf record: Put a copy of kcore into the perf.data directory Adrian Hunter
                   ` (2 subsequent siblings)
  6 siblings, 2 replies; 24+ messages in thread
From: Adrian Hunter @ 2019-10-04  8:31 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo, Jiri Olsa; +Cc: linux-kernel

Support directory output that contains a regular perf.data file, named
"data". By default the directory is named perf.data i.e.
	perf.data
	└── data

Most of the infrastucture to support a directory is already there. This
patch makes the changes needed to support the format above.

Presently there is no 'perf record' option to output a directory.

This is preparation for adding support for putting a copy of /proc/kcore in
the directory.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
---
 .../perf.data-directory-format.txt            | 28 +++++++++++++++++++
 tools/perf/builtin-record.c                   |  2 +-
 tools/perf/util/data.c                        |  9 +++++-
 tools/perf/util/data.h                        |  6 ++++
 4 files changed, 43 insertions(+), 2 deletions(-)
 create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt

diff --git a/tools/perf/Documentation/perf.data-directory-format.txt b/tools/perf/Documentation/perf.data-directory-format.txt
new file mode 100644
index 000000000000..4bf08908178d
--- /dev/null
+++ b/tools/perf/Documentation/perf.data-directory-format.txt
@@ -0,0 +1,28 @@
+perf.data directory format
+
+DISCLAIMER This is not ABI yet and is subject to possible change
+           in following versions of perf. We will remove this
+           disclaimer once the directory format soaks in.
+
+
+This document describes the on-disk perf.data directory format.
+
+The layout is described by HEADER_DIR_FORMAT feature.
+Currently it holds only version number (0):
+
+  HEADER_DIR_FORMAT = 24
+
+  struct {
+     uint64_t version;
+  }
+
+The current only version value 0 means that:
+  - there is a single perf.data file named 'data' within the directory.
+  e.g.
+
+    $ tree -ps perf.data
+    perf.data
+    └── [-rw-------       25912]  data
+
+Future versions are expected to describe different data files
+layout according to special needs.
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 23332861de6e..fceac9d42b4e 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -537,7 +537,7 @@ static int record__process_auxtrace(struct perf_tool *tool,
 	size_t padding;
 	u8 pad[8] = {0};
 
-	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
+	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
 		off_t file_offset;
 		int fd = perf_data__fd(data);
 		int err;
diff --git a/tools/perf/util/data.c b/tools/perf/util/data.c
index df173f0bf654..964ea101dba6 100644
--- a/tools/perf/util/data.c
+++ b/tools/perf/util/data.c
@@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
 	DIR *dir;
 	int nr = 0;
 
+	/*
+	 * Directory containing a single regular perf data file which is already
+	 * open, means there is nothing more to do here.
+	 */
+	if (perf_data__is_single_file(data))
+		return 0;
+
 	if (WARN_ON(!data->is_dir))
 		return -EINVAL;
 
@@ -406,7 +413,7 @@ unsigned long perf_data__size(struct perf_data *data)
 	u64 size = data->file.size;
 	int i;
 
-	if (!data->is_dir)
+	if (perf_data__is_single_file(data))
 		return size;
 
 	for (i = 0; i < data->dir.nr; i++) {
diff --git a/tools/perf/util/data.h b/tools/perf/util/data.h
index 218fe9a16801..f68815f7e428 100644
--- a/tools/perf/util/data.h
+++ b/tools/perf/util/data.h
@@ -10,6 +10,7 @@ enum perf_data_mode {
 };
 
 enum perf_dir_version {
+	PERF_DIR_SINGLE_FILE	= 0,
 	PERF_DIR_VERSION	= 1,
 };
 
@@ -54,6 +55,11 @@ static inline bool perf_data__is_dir(struct perf_data *data)
 	return data->is_dir;
 }
 
+static inline bool perf_data__is_single_file(struct perf_data *data)
+{
+	return data->dir.version == PERF_DIR_SINGLE_FILE;
+}
+
 static inline int perf_data__fd(struct perf_data *data)
 {
 	return data->file.fd;
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [PATCH 5/5] perf record: Put a copy of kcore into the perf.data directory
  2019-10-04  8:31 [PATCH 0/5] perf record: Put a copy of kcore into the perf.data directory Adrian Hunter
                   ` (3 preceding siblings ...)
  2019-10-04  8:31 ` [PATCH 4/5] perf tools: Support single perf.data file directory Adrian Hunter
@ 2019-10-04  8:31 ` Adrian Hunter
  2019-10-07 11:20   ` Jiri Olsa
                     ` (2 more replies)
  2019-10-07 12:55 ` [PATCH 0/5] " Jiri Olsa
  2019-10-21 14:36 ` Arnaldo Carvalho de Melo
  6 siblings, 3 replies; 24+ messages in thread
From: Adrian Hunter @ 2019-10-04  8:31 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo, Jiri Olsa; +Cc: linux-kernel

Add a new 'perf record' option '--kcore' which will put a copy of
/proc/kcore, kallsyms and modules into a perf.data directory. Note, that
without the --kcore option, output goes to a file as previously.
The tools' -o and -i options work with either a file name or directory
name.

Example:

 $ sudo perf record --kcore uname

 $ sudo tree perf.data
 perf.data
 ├── kcore_dir
 │   ├── kallsyms
 │   ├── kcore
 │   └── modules
 └── data

 $ sudo perf script -v
 build id event received for vmlinux: 1eaa285996affce2d74d8e66dcea09a80c9941de
 build id event received for [vdso]: 8bbaf5dc62a9b644b4d4e4539737e104e4a84541
 Samples for 'cycles' event do not have CPU attribute set. Skipping 'cpu' field.
 Using CPUID GenuineIntel-6-8E-A
 Using perf.data/kcore_dir/kcore for kernel data
 Using perf.data/kcore_dir/kallsyms for symbols
             perf 19058 506778.423729:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423733:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423734:          7 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423736:        117 cycles:  ffffffffa2caa54a native_write_msr+0xa (vmlinux)
             perf 19058 506778.423738:       2092 cycles:  ffffffffa2c9b7b0 native_apic_msr_write+0x0 (vmlinux)
             perf 19058 506778.423740:      37380 cycles:  ffffffffa2f121d0 perf_event_addr_filters_exec+0x0 (vmlinux)
            uname 19058 506778.423751:     582673 cycles:  ffffffffa303a407 propagate_protected_usage+0x147 (vmlinux)
            uname 19058 506778.423892:    2241841 cycles:  ffffffffa2cae0c9 unwind_next_frame.part.5+0x79 (vmlinux)
            uname 19058 506778.424430:    2457397 cycles:  ffffffffa3019232 check_memory_region+0x52 (vmlinux)

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
---
 tools/perf/Documentation/perf-record.txt      |  3 ++
 .../perf.data-directory-format.txt            | 35 +++++++++++++
 tools/perf/builtin-record.c                   | 52 +++++++++++++++++++
 tools/perf/util/data.c                        | 33 ++++++++++++
 tools/perf/util/data.h                        |  2 +
 tools/perf/util/record.h                      |  1 +
 tools/perf/util/session.c                     |  4 ++
 tools/perf/util/util.c                        | 17 ++++++
 8 files changed, 147 insertions(+)

diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt
index c6f9f31b6039..8a4506113d9f 100644
--- a/tools/perf/Documentation/perf-record.txt
+++ b/tools/perf/Documentation/perf-record.txt
@@ -571,6 +571,9 @@ config terms. For example: 'cycles/overwrite/' and 'instructions/no-overwrite/'.
 
 Implies --tail-synthesize.
 
+--kcore::
+Make a copy of /proc/kcore and place it into a directory with the perf data file.
+
 SEE ALSO
 --------
 linkperf:perf-stat[1], linkperf:perf-list[1]
diff --git a/tools/perf/Documentation/perf.data-directory-format.txt b/tools/perf/Documentation/perf.data-directory-format.txt
index 4bf08908178d..f37fbd29112e 100644
--- a/tools/perf/Documentation/perf.data-directory-format.txt
+++ b/tools/perf/Documentation/perf.data-directory-format.txt
@@ -26,3 +26,38 @@ The current only version value 0 means that:
 
 Future versions are expected to describe different data files
 layout according to special needs.
+
+Currently the only 'perf record' option to output to a directory is
+the --kcore option which puts a copy of /proc/kcore into the directory.
+e.g.
+
+  $ sudo perf record --kcore uname
+  Linux
+  [ perf record: Woken up 1 times to write data ]
+  [ perf record: Captured and wrote 0.015 MB perf.data (9 samples) ]
+  $ sudo tree -ps perf.data
+  perf.data
+  ├── [-rw-------       23744]  data
+  └── [drwx------        4096]  kcore_dir
+      ├── [-r--------     6731125]  kallsyms
+      ├── [-r--------    40230912]  kcore
+      └── [-r--------        5419]  modules
+
+  1 directory, 4 files
+  $ sudo perf script -v
+  build id event received for vmlinux: 1eaa285996affce2d74d8e66dcea09a80c9941de
+  build id event received for [vdso]: 8bbaf5dc62a9b644b4d4e4539737e104e4a84541
+  build id event received for /lib/x86_64-linux-gnu/libc-2.28.so: 5b157f49586a3ca84d55837f97ff466767dd3445
+  Samples for 'cycles' event do not have CPU attribute set. Skipping 'cpu' field.
+  Using CPUID GenuineIntel-6-8E-A
+  Using perf.data/kcore_dir/kcore for kernel data
+  Using perf.data/kcore_dir/kallsyms for symbols
+              perf 15316 2060795.480902:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
+              perf 15316 2060795.480906:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
+              perf 15316 2060795.480908:          7 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
+              perf 15316 2060795.480910:        119 cycles:  ffffffffa2caa54a native_write_msr+0xa (vmlinux)
+              perf 15316 2060795.480912:       2109 cycles:  ffffffffa2c9b7b0 native_apic_msr_write+0x0 (vmlinux)
+              perf 15316 2060795.480914:      37606 cycles:  ffffffffa2f121fe perf_event_addr_filters_exec+0x2e (vmlinux)
+             uname 15316 2060795.480924:     588287 cycles:  ffffffffa303a56d page_counter_try_charge+0x6d (vmlinux)
+             uname 15316 2060795.481067:    2261945 cycles:  ffffffffa301438f kmem_cache_free+0x4f (vmlinux)
+             uname 15316 2060795.481643:    2172167 cycles:      7f1a48c393c0 _IO_un_link+0x0 (/lib/x86_64-linux-gnu/libc-2.28.so)
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index fceac9d42b4e..ca9bf249cfb2 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -55,6 +55,9 @@
 #include <signal.h>
 #include <sys/mman.h>
 #include <sys/wait.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
 #include <linux/err.h>
 #include <linux/string.h>
 #include <linux/time64.h>
@@ -699,6 +702,37 @@ static int record__auxtrace_init(struct record *rec __maybe_unused)
 
 #endif
 
+static bool record__kcore_readable(struct machine *machine)
+{
+	char kcore[PATH_MAX];
+	int fd;
+
+	scnprintf(kcore, sizeof(kcore), "%s/proc/kcore", machine->root_dir);
+
+	fd = open(kcore, O_RDONLY);
+	if (fd < 0)
+		return false;
+
+	close(fd);
+
+	return true;
+}
+
+static int record__kcore_copy(struct machine *machine, struct perf_data *data)
+{
+	char from_dir[PATH_MAX];
+	char kcore_dir[PATH_MAX];
+	int ret;
+
+	snprintf(from_dir, sizeof(from_dir), "%s/proc", machine->root_dir);
+
+	ret = perf_data__make_kcore_dir(data, kcore_dir, sizeof(kcore_dir));
+	if (ret)
+		return ret;
+
+	return kcore_copy(from_dir, kcore_dir);
+}
+
 static int record__mmap_evlist(struct record *rec,
 			       struct evlist *evlist)
 {
@@ -1383,6 +1417,12 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 	session->header.env.comp_type  = PERF_COMP_ZSTD;
 	session->header.env.comp_level = rec->opts.comp_level;
 
+	if (rec->opts.kcore &&
+	    !record__kcore_readable(&session->machines.host)) {
+		pr_err("ERROR: kcore is not readable.\n");
+		return -1;
+	}
+
 	record__init_features(rec);
 
 	if (rec->opts.use_clockid && rec->opts.clockid_res_ns)
@@ -1414,6 +1454,14 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 	}
 	session->header.env.comp_mmap_len = session->evlist->core.mmap_len;
 
+	if (rec->opts.kcore) {
+		err = record__kcore_copy(&session->machines.host, data);
+		if (err) {
+			pr_err("ERROR: Failed to copy kcore\n");
+			goto out_child;
+		}
+	}
+
 	err = bpf__apply_obj_config();
 	if (err) {
 		char errbuf[BUFSIZ];
@@ -2184,6 +2232,7 @@ static struct option __record_options[] = {
 		     parse_cgroups),
 	OPT_UINTEGER('D', "delay", &record.opts.initial_delay,
 		  "ms to wait before starting measurement after program start"),
+	OPT_BOOLEAN(0, "kcore", &record.opts.kcore, "copy /proc/kcore"),
 	OPT_STRING('u', "uid", &record.opts.target.uid_str, "user",
 		   "user to profile"),
 
@@ -2322,6 +2371,9 @@ int cmd_record(int argc, const char **argv)
 
 	}
 
+	if (rec->opts.kcore)
+		rec->data.is_dir = true;
+
 	if (rec->opts.comp_level != 0) {
 		pr_debug("Compression enabled, disabling build id collection at the end of the session.\n");
 		rec->no_buildid = true;
diff --git a/tools/perf/util/data.c b/tools/perf/util/data.c
index 964ea101dba6..c47aa34fdc0a 100644
--- a/tools/perf/util/data.c
+++ b/tools/perf/util/data.c
@@ -424,3 +424,36 @@ unsigned long perf_data__size(struct perf_data *data)
 
 	return size;
 }
+
+int perf_data__make_kcore_dir(struct perf_data *data, char *buf, size_t buf_sz)
+{
+	int ret;
+
+	if (!data->is_dir)
+		return -1;
+
+	ret = snprintf(buf, buf_sz, "%s/kcore_dir", data->path);
+	if (ret < 0 || (size_t)ret >= buf_sz)
+		return -1;
+
+	return mkdir(buf, S_IRWXU);
+}
+
+char *perf_data__kallsyms_name(struct perf_data *data)
+{
+	char *kallsyms_name;
+	struct stat st;
+
+	if (!data->is_dir)
+		return NULL;
+
+	if (asprintf(&kallsyms_name, "%s/kcore_dir/kallsyms", data->path) < 0)
+		return NULL;
+
+	if (stat(kallsyms_name, &st)) {
+		free(kallsyms_name);
+		return NULL;
+	}
+
+	return kallsyms_name;
+}
diff --git a/tools/perf/util/data.h b/tools/perf/util/data.h
index f68815f7e428..75947ef6bc17 100644
--- a/tools/perf/util/data.h
+++ b/tools/perf/util/data.h
@@ -87,4 +87,6 @@ int perf_data__open_dir(struct perf_data *data);
 void perf_data__close_dir(struct perf_data *data);
 int perf_data__update_dir(struct perf_data *data);
 unsigned long perf_data__size(struct perf_data *data);
+int perf_data__make_kcore_dir(struct perf_data *data, char *buf, size_t buf_sz);
+char *perf_data__kallsyms_name(struct perf_data *data);
 #endif /* __PERF_DATA_H */
diff --git a/tools/perf/util/record.h b/tools/perf/util/record.h
index 00275afc524d..948bbcf9aef3 100644
--- a/tools/perf/util/record.h
+++ b/tools/perf/util/record.h
@@ -44,6 +44,7 @@ struct record_opts {
 	bool	      strict_freq;
 	bool	      sample_id;
 	bool	      no_bpf_event;
+	bool	      kcore;
 	unsigned int  freq;
 	unsigned int  mmap_pages;
 	unsigned int  auxtrace_mmap_pages;
diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
index 061bb4d6a3f5..bfa80fe8d369 100644
--- a/tools/perf/util/session.c
+++ b/tools/perf/util/session.c
@@ -230,6 +230,10 @@ struct perf_session *perf_session__new(struct perf_data *data,
 			if (ret)
 				goto out_delete;
 			}
+
+			if (!symbol_conf.kallsyms_name &&
+			    !symbol_conf.vmlinux_name)
+				symbol_conf.kallsyms_name = perf_data__kallsyms_name(data);
 		}
 	} else  {
 		session->machines.host.env = &perf_env;
diff --git a/tools/perf/util/util.c b/tools/perf/util/util.c
index 56f30ad29ca7..16f81a0a9208 100644
--- a/tools/perf/util/util.c
+++ b/tools/perf/util/util.c
@@ -180,6 +180,21 @@ static int rm_rf_depth_pat(const char *path, int depth, const char **pat)
 	return rmdir(path);
 }
 
+static int rm_rf_kcore_dir(const char *path)
+{
+	char kcore_dir_path[PATH_MAX];
+	const char *pat[] = {
+		"kcore",
+		"kallsyms",
+		"modules",
+		NULL,
+	};
+
+	snprintf(kcore_dir_path, sizeof(kcore_dir_path), "%s/kcore_dir", path);
+
+	return rm_rf_depth_pat(kcore_dir_path, 0, pat);
+}
+
 int rm_rf_perf_data(const char *path)
 {
 	const char *pat[] = {
@@ -188,6 +203,8 @@ int rm_rf_perf_data(const char *path)
 		NULL,
 	};
 
+	rm_rf_kcore_dir(path);
+
 	return rm_rf_depth_pat(path, 0, pat);
 }
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 24+ messages in thread

* Re: [PATCH 4/5] perf tools: Support single perf.data file directory
  2019-10-04  8:31 ` [PATCH 4/5] perf tools: Support single perf.data file directory Adrian Hunter
@ 2019-10-07 11:20   ` Jiri Olsa
  2019-10-07 12:06     ` Adrian Hunter
  2019-11-12 11:18     ` [tip: perf/core] perf session: Fix indent in perf_session__new()" tip-bot2 for Jiri Olsa
  2019-11-12 11:18   ` [tip: perf/core] perf data: Support single perf.data file directory tip-bot2 for Adrian Hunter
  1 sibling, 2 replies; 24+ messages in thread
From: Jiri Olsa @ 2019-10-07 11:20 UTC (permalink / raw)
  To: Adrian Hunter; +Cc: Arnaldo Carvalho de Melo, linux-kernel

On Fri, Oct 04, 2019 at 11:31:20AM +0300, Adrian Hunter wrote:

SNIP

>  	u8 pad[8] = {0};
>  
> -	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
> +	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
>  		off_t file_offset;
>  		int fd = perf_data__fd(data);
>  		int err;
> diff --git a/tools/perf/util/data.c b/tools/perf/util/data.c
> index df173f0bf654..964ea101dba6 100644
> --- a/tools/perf/util/data.c
> +++ b/tools/perf/util/data.c
> @@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
>  	DIR *dir;
>  	int nr = 0;
>  
> +	/*
> +	 * Directory containing a single regular perf data file which is already
> +	 * open, means there is nothing more to do here.
> +	 */
> +	if (perf_data__is_single_file(data))
> +		return 0;
> +

cool, I like this approach much more than the previous flag

any change you (if there's repost) or Arnaldo
could squeeze in indent change below?

thanks,
jirka


---
diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
index bfa80fe8d369..7f567a521cea 100644
--- a/tools/perf/util/session.c
+++ b/tools/perf/util/session.c
@@ -227,8 +227,8 @@ struct perf_session *perf_session__new(struct perf_data *data,
 			/* Open the directory data. */
 			if (data->is_dir) {
 				ret = perf_data__open_dir(data);
-			if (ret)
-				goto out_delete;
+				if (ret)
+					goto out_delete;
 			}
 
 			if (!symbol_conf.kallsyms_name &&

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* Re: [PATCH 5/5] perf record: Put a copy of kcore into the perf.data directory
  2019-10-04  8:31 ` [PATCH 5/5] perf record: Put a copy of kcore into the perf.data directory Adrian Hunter
@ 2019-10-07 11:20   ` Jiri Olsa
  2019-10-07 12:06     ` Adrian Hunter
  2019-10-07 11:20   ` Jiri Olsa
  2019-11-12 11:18   ` [tip: perf/core] " tip-bot2 for Adrian Hunter
  2 siblings, 1 reply; 24+ messages in thread
From: Jiri Olsa @ 2019-10-07 11:20 UTC (permalink / raw)
  To: Adrian Hunter; +Cc: Arnaldo Carvalho de Melo, linux-kernel

On Fri, Oct 04, 2019 at 11:31:21AM +0300, Adrian Hunter wrote:

SNIP

> +}
> +
>  static int record__mmap_evlist(struct record *rec,
>  			       struct evlist *evlist)
>  {
> @@ -1383,6 +1417,12 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
>  	session->header.env.comp_type  = PERF_COMP_ZSTD;
>  	session->header.env.comp_level = rec->opts.comp_level;
>  
> +	if (rec->opts.kcore &&
> +	    !record__kcore_readable(&session->machines.host)) {
> +		pr_err("ERROR: kcore is not readable.\n");
> +		return -1;
> +	}
> +

is there any reason why this change is not merged with the change below?


>  	record__init_features(rec);
>  
>  	if (rec->opts.use_clockid && rec->opts.clockid_res_ns)
> @@ -1414,6 +1454,14 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
>  	}
>  	session->header.env.comp_mmap_len = session->evlist->core.mmap_len;
>  
> +	if (rec->opts.kcore) {
> +		err = record__kcore_copy(&session->machines.host, data);
> +		if (err) {
> +			pr_err("ERROR: Failed to copy kcore\n");
> +			goto out_child;
> +		}
> +	}
> +

thanks,
jirka

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 5/5] perf record: Put a copy of kcore into the perf.data directory
  2019-10-04  8:31 ` [PATCH 5/5] perf record: Put a copy of kcore into the perf.data directory Adrian Hunter
  2019-10-07 11:20   ` Jiri Olsa
@ 2019-10-07 11:20   ` Jiri Olsa
  2019-10-07 12:14     ` Adrian Hunter
  2019-11-12 11:18   ` [tip: perf/core] " tip-bot2 for Adrian Hunter
  2 siblings, 1 reply; 24+ messages in thread
From: Jiri Olsa @ 2019-10-07 11:20 UTC (permalink / raw)
  To: Adrian Hunter; +Cc: Arnaldo Carvalho de Melo, linux-kernel

On Fri, Oct 04, 2019 at 11:31:21AM +0300, Adrian Hunter wrote:

SNIP

>  	bool	      strict_freq;
>  	bool	      sample_id;
>  	bool	      no_bpf_event;
> +	bool	      kcore;
>  	unsigned int  freq;
>  	unsigned int  mmap_pages;
>  	unsigned int  auxtrace_mmap_pages;
> diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
> index 061bb4d6a3f5..bfa80fe8d369 100644
> --- a/tools/perf/util/session.c
> +++ b/tools/perf/util/session.c
> @@ -230,6 +230,10 @@ struct perf_session *perf_session__new(struct perf_data *data,
>  			if (ret)
>  				goto out_delete;
>  			}
> +
> +			if (!symbol_conf.kallsyms_name &&
> +			    !symbol_conf.vmlinux_name)
> +				symbol_conf.kallsyms_name = perf_data__kallsyms_name(data);

hum, should this also depend on rec->opts.kcore ?

jirka

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 4/5] perf tools: Support single perf.data file directory
  2019-10-07 11:20   ` Jiri Olsa
@ 2019-10-07 12:06     ` Adrian Hunter
  2019-10-21 12:42       ` Adrian Hunter
  2019-11-12 11:18     ` [tip: perf/core] perf session: Fix indent in perf_session__new()" tip-bot2 for Jiri Olsa
  1 sibling, 1 reply; 24+ messages in thread
From: Adrian Hunter @ 2019-10-07 12:06 UTC (permalink / raw)
  To: Jiri Olsa; +Cc: Arnaldo Carvalho de Melo, linux-kernel

On 7/10/19 2:20 PM, Jiri Olsa wrote:
> On Fri, Oct 04, 2019 at 11:31:20AM +0300, Adrian Hunter wrote:
> 
> SNIP
> 
>>  	u8 pad[8] = {0};
>>  
>> -	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
>> +	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
>>  		off_t file_offset;
>>  		int fd = perf_data__fd(data);
>>  		int err;
>> diff --git a/tools/perf/util/data.c b/tools/perf/util/data.c
>> index df173f0bf654..964ea101dba6 100644
>> --- a/tools/perf/util/data.c
>> +++ b/tools/perf/util/data.c
>> @@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
>>  	DIR *dir;
>>  	int nr = 0;
>>  
>> +	/*
>> +	 * Directory containing a single regular perf data file which is already
>> +	 * open, means there is nothing more to do here.
>> +	 */
>> +	if (perf_data__is_single_file(data))
>> +		return 0;
>> +
> 
> cool, I like this approach much more than the previous flag

Yes it is much nicer.  Thanks for your direction on that.

> 
> any change you (if there's repost) or Arnaldo
> could squeeze in indent change below?

Sent a patch, to be applied before these.

> 
> thanks,
> jirka
> 
> 
> ---
> diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
> index bfa80fe8d369..7f567a521cea 100644
> --- a/tools/perf/util/session.c
> +++ b/tools/perf/util/session.c
> @@ -227,8 +227,8 @@ struct perf_session *perf_session__new(struct perf_data *data,
>  			/* Open the directory data. */
>  			if (data->is_dir) {
>  				ret = perf_data__open_dir(data);
> -			if (ret)
> -				goto out_delete;
> +				if (ret)
> +					goto out_delete;
>  			}
>  
>  			if (!symbol_conf.kallsyms_name &&
> 


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 5/5] perf record: Put a copy of kcore into the perf.data directory
  2019-10-07 11:20   ` Jiri Olsa
@ 2019-10-07 12:06     ` Adrian Hunter
  0 siblings, 0 replies; 24+ messages in thread
From: Adrian Hunter @ 2019-10-07 12:06 UTC (permalink / raw)
  To: Jiri Olsa; +Cc: Arnaldo Carvalho de Melo, linux-kernel

On 7/10/19 2:20 PM, Jiri Olsa wrote:
> On Fri, Oct 04, 2019 at 11:31:21AM +0300, Adrian Hunter wrote:
> 
> SNIP
> 
>> +}
>> +
>>  static int record__mmap_evlist(struct record *rec,
>>  			       struct evlist *evlist)
>>  {
>> @@ -1383,6 +1417,12 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
>>  	session->header.env.comp_type  = PERF_COMP_ZSTD;
>>  	session->header.env.comp_level = rec->opts.comp_level;
>>  
>> +	if (rec->opts.kcore &&
>> +	    !record__kcore_readable(&session->machines.host)) {
>> +		pr_err("ERROR: kcore is not readable.\n");
>> +		return -1;
>> +	}
>> +
> 
> is there any reason why this change is not merged with the change below?

It seemed better to do the validation before opening all the events.

> 
> 
>>  	record__init_features(rec);
>>  
>>  	if (rec->opts.use_clockid && rec->opts.clockid_res_ns)
>> @@ -1414,6 +1454,14 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
>>  	}
>>  	session->header.env.comp_mmap_len = session->evlist->core.mmap_len;
>>  
>> +	if (rec->opts.kcore) {
>> +		err = record__kcore_copy(&session->machines.host, data);
>> +		if (err) {
>> +			pr_err("ERROR: Failed to copy kcore\n");
>> +			goto out_child;
>> +		}
>> +	}
>> +
> 
> thanks,
> jirka
> 


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 5/5] perf record: Put a copy of kcore into the perf.data directory
  2019-10-07 11:20   ` Jiri Olsa
@ 2019-10-07 12:14     ` Adrian Hunter
  0 siblings, 0 replies; 24+ messages in thread
From: Adrian Hunter @ 2019-10-07 12:14 UTC (permalink / raw)
  To: Jiri Olsa; +Cc: Arnaldo Carvalho de Melo, linux-kernel

On 7/10/19 2:20 PM, Jiri Olsa wrote:
> On Fri, Oct 04, 2019 at 11:31:21AM +0300, Adrian Hunter wrote:
> 
> SNIP
> 
>>  	bool	      strict_freq;
>>  	bool	      sample_id;
>>  	bool	      no_bpf_event;
>> +	bool	      kcore;
>>  	unsigned int  freq;
>>  	unsigned int  mmap_pages;
>>  	unsigned int  auxtrace_mmap_pages;
>> diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
>> index 061bb4d6a3f5..bfa80fe8d369 100644
>> --- a/tools/perf/util/session.c
>> +++ b/tools/perf/util/session.c
>> @@ -230,6 +230,10 @@ struct perf_session *perf_session__new(struct perf_data *data,
>>  			if (ret)
>>  				goto out_delete;
>>  			}
>> +
>> +			if (!symbol_conf.kallsyms_name &&
>> +			    !symbol_conf.vmlinux_name)
>> +				symbol_conf.kallsyms_name = perf_data__kallsyms_name(data);
> 
> hum, should this also depend on rec->opts.kcore ?

This is the bit that makes 'perf script' work.  It has the same affect as
though the --kallsyms option was used. i.e. it makes 'perf script' behave
the same as:
	
	perf script --kallsyms perf.data/kcore_dir/kallsyms

Which works because perf looks for kcore in the same directory as kallsyms.

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 0/5] perf record: Put a copy of kcore into the perf.data directory
  2019-10-04  8:31 [PATCH 0/5] perf record: Put a copy of kcore into the perf.data directory Adrian Hunter
                   ` (4 preceding siblings ...)
  2019-10-04  8:31 ` [PATCH 5/5] perf record: Put a copy of kcore into the perf.data directory Adrian Hunter
@ 2019-10-07 12:55 ` Jiri Olsa
  2019-10-21 14:36 ` Arnaldo Carvalho de Melo
  6 siblings, 0 replies; 24+ messages in thread
From: Jiri Olsa @ 2019-10-07 12:55 UTC (permalink / raw)
  To: Adrian Hunter; +Cc: Arnaldo Carvalho de Melo, linux-kernel

On Fri, Oct 04, 2019 at 11:31:16AM +0300, Adrian Hunter wrote:
> Hi
> 
> Here are patches to add a new 'perf record' option '--kcore' which will put
> a copy of /proc/kcore, kallsyms and modules into a perf.data directory.
> Note, that without the --kcore option, output goes to a file as previously.
> The tools' -o and -i options work with either a file name or directory
> name.
> 
> Example:
> 
>  $ sudo perf record --kcore uname
> 
>  $ sudo tree perf.data
>  perf.data
>  ├── kcore_dir
>  │   ├── kallsyms
>  │   ├── kcore
>  │   └── modules
>  └── data
> 
>  $ sudo perf script -v
>  build id event received for vmlinux: 1eaa285996affce2d74d8e66dcea09a80c9941de
>  build id event received for [vdso]: 8bbaf5dc62a9b644b4d4e4539737e104e4a84541
>  Samples for 'cycles' event do not have CPU attribute set. Skipping 'cpu' field.
>  Using CPUID GenuineIntel-6-8E-A
>  Using perf.data/kcore_dir/kcore for kernel data
>  Using perf.data/kcore_dir/kallsyms for symbols
>              perf 19058 506778.423729:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
>              perf 19058 506778.423733:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
>              perf 19058 506778.423734:          7 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
>              perf 19058 506778.423736:        117 cycles:  ffffffffa2caa54a native_write_msr+0xa (vmlinux)
>              perf 19058 506778.423738:       2092 cycles:  ffffffffa2c9b7b0 native_apic_msr_write+0x0 (vmlinux)
>              perf 19058 506778.423740:      37380 cycles:  ffffffffa2f121d0 perf_event_addr_filters_exec+0x0 (vmlinux)
>             uname 19058 506778.423751:     582673 cycles:  ffffffffa303a407 propagate_protected_usage+0x147 (vmlinux)
>             uname 19058 506778.423892:    2241841 cycles:  ffffffffa2cae0c9 unwind_next_frame.part.5+0x79 (vmlinux)
>             uname 19058 506778.424430:    2457397 cycles:  ffffffffa3019232 check_memory_region+0x52 (vmlinux)
> 
> 
> Adrian Hunter (5):
>       perf data: Correctly identify directory data files
>       perf data: Move perf_dir_version into data.h
>       perf data: Rename directory "header" file to "data"
>       perf tools: Support single perf.data file directory
>       perf record: Put a copy of kcore into the perf.data directory

Reviewed-by: Jiri Olsa <jolsa@kernel.org>

thanks,
jirka

> 
>  tools/perf/Documentation/perf-record.txt           |  3 ++
>  .../Documentation/perf.data-directory-format.txt   | 63 ++++++++++++++++++++++
>  tools/perf/builtin-record.c                        | 54 ++++++++++++++++++-
>  tools/perf/util/data.c                             | 46 ++++++++++++++--
>  tools/perf/util/data.h                             | 12 +++++
>  tools/perf/util/header.h                           |  4 --
>  tools/perf/util/record.h                           |  1 +
>  tools/perf/util/session.c                          |  4 ++
>  tools/perf/util/util.c                             | 19 ++++++-
>  9 files changed, 197 insertions(+), 9 deletions(-)
>  create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt
> 
> 
> Regards
> Adrian

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 4/5] perf tools: Support single perf.data file directory
  2019-10-07 12:06     ` Adrian Hunter
@ 2019-10-21 12:42       ` Adrian Hunter
  2019-10-21 13:46         ` Arnaldo Carvalho de Melo
  2019-10-21 14:03         ` Arnaldo Carvalho de Melo
  0 siblings, 2 replies; 24+ messages in thread
From: Adrian Hunter @ 2019-10-21 12:42 UTC (permalink / raw)
  To: Jiri Olsa; +Cc: Arnaldo Carvalho de Melo, linux-kernel

On 7/10/19 3:06 PM, Adrian Hunter wrote:
> On 7/10/19 2:20 PM, Jiri Olsa wrote:
>> On Fri, Oct 04, 2019 at 11:31:20AM +0300, Adrian Hunter wrote:
>>
>> SNIP
>>
>>>  	u8 pad[8] = {0};
>>>  
>>> -	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
>>> +	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
>>>  		off_t file_offset;
>>>  		int fd = perf_data__fd(data);
>>>  		int err;
>>> diff --git a/tools/perf/util/data.c b/tools/perf/util/data.c
>>> index df173f0bf654..964ea101dba6 100644
>>> --- a/tools/perf/util/data.c
>>> +++ b/tools/perf/util/data.c
>>> @@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
>>>  	DIR *dir;
>>>  	int nr = 0;
>>>  
>>> +	/*
>>> +	 * Directory containing a single regular perf data file which is already
>>> +	 * open, means there is nothing more to do here.
>>> +	 */
>>> +	if (perf_data__is_single_file(data))
>>> +		return 0;
>>> +
>>
>> cool, I like this approach much more than the previous flag
> 
> Yes it is much nicer.  Thanks for your direction on that.
> 
>>
>> any change you (if there's repost) or Arnaldo
>> could squeeze in indent change below?
> 
> Sent a patch, to be applied before these.
> 

That is:

"[PATCH] perf session: Fix indent in perf_session__new()"

Any comments on this patch set Arnaldo?

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 4/5] perf tools: Support single perf.data file directory
  2019-10-21 12:42       ` Adrian Hunter
@ 2019-10-21 13:46         ` Arnaldo Carvalho de Melo
  2019-10-21 13:52           ` Jiri Olsa
  2019-10-21 14:03         ` Arnaldo Carvalho de Melo
  1 sibling, 1 reply; 24+ messages in thread
From: Arnaldo Carvalho de Melo @ 2019-10-21 13:46 UTC (permalink / raw)
  To: Adrian Hunter; +Cc: Jiri Olsa, linux-kernel

Em Mon, Oct 21, 2019 at 03:42:39PM +0300, Adrian Hunter escreveu:
> On 7/10/19 3:06 PM, Adrian Hunter wrote:
> > On 7/10/19 2:20 PM, Jiri Olsa wrote:
> >> On Fri, Oct 04, 2019 at 11:31:20AM +0300, Adrian Hunter wrote:
> >>
> >> SNIP
> >>
> >>>  	u8 pad[8] = {0};
> >>>  
> >>> -	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
> >>> +	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
> >>>  		off_t file_offset;
> >>>  		int fd = perf_data__fd(data);
> >>>  		int err;
> >>> diff --git a/tools/perf/util/data.c b/tools/perf/util/data.c
> >>> index df173f0bf654..964ea101dba6 100644
> >>> --- a/tools/perf/util/data.c
> >>> +++ b/tools/perf/util/data.c
> >>> @@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
> >>>  	DIR *dir;
> >>>  	int nr = 0;
> >>>  
> >>> +	/*
> >>> +	 * Directory containing a single regular perf data file which is already
> >>> +	 * open, means there is nothing more to do here.
> >>> +	 */
> >>> +	if (perf_data__is_single_file(data))
> >>> +		return 0;
> >>> +
> >>
> >> cool, I like this approach much more than the previous flag
> > 
> > Yes it is much nicer.  Thanks for your direction on that.
> > 
> >>
> >> any change you (if there's repost) or Arnaldo
> >> could squeeze in indent change below?
> > 
> > Sent a patch, to be applied before these.
> > 
> 
> That is:
> 
> "[PATCH] perf session: Fix indent in perf_session__new()"
> 
> Any comments on this patch set Arnaldo?

I'll take a look at it later, I think Jiri is ok with it already?

- Arnaldo

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 4/5] perf tools: Support single perf.data file directory
  2019-10-21 13:46         ` Arnaldo Carvalho de Melo
@ 2019-10-21 13:52           ` Jiri Olsa
  0 siblings, 0 replies; 24+ messages in thread
From: Jiri Olsa @ 2019-10-21 13:52 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo; +Cc: Adrian Hunter, linux-kernel

On Mon, Oct 21, 2019 at 10:46:11AM -0300, Arnaldo Carvalho de Melo wrote:
> Em Mon, Oct 21, 2019 at 03:42:39PM +0300, Adrian Hunter escreveu:
> > On 7/10/19 3:06 PM, Adrian Hunter wrote:
> > > On 7/10/19 2:20 PM, Jiri Olsa wrote:
> > >> On Fri, Oct 04, 2019 at 11:31:20AM +0300, Adrian Hunter wrote:
> > >>
> > >> SNIP
> > >>
> > >>>  	u8 pad[8] = {0};
> > >>>  
> > >>> -	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
> > >>> +	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
> > >>>  		off_t file_offset;
> > >>>  		int fd = perf_data__fd(data);
> > >>>  		int err;
> > >>> diff --git a/tools/perf/util/data.c b/tools/perf/util/data.c
> > >>> index df173f0bf654..964ea101dba6 100644
> > >>> --- a/tools/perf/util/data.c
> > >>> +++ b/tools/perf/util/data.c
> > >>> @@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
> > >>>  	DIR *dir;
> > >>>  	int nr = 0;
> > >>>  
> > >>> +	/*
> > >>> +	 * Directory containing a single regular perf data file which is already
> > >>> +	 * open, means there is nothing more to do here.
> > >>> +	 */
> > >>> +	if (perf_data__is_single_file(data))
> > >>> +		return 0;
> > >>> +
> > >>
> > >> cool, I like this approach much more than the previous flag
> > > 
> > > Yes it is much nicer.  Thanks for your direction on that.
> > > 
> > >>
> > >> any change you (if there's repost) or Arnaldo
> > >> could squeeze in indent change below?
> > > 
> > > Sent a patch, to be applied before these.
> > > 
> > 
> > That is:
> > 
> > "[PATCH] perf session: Fix indent in perf_session__new()"
> > 
> > Any comments on this patch set Arnaldo?
> 
> I'll take a look at it later, I think Jiri is ok with it already?

yes, it's just indent fix

jirka


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 4/5] perf tools: Support single perf.data file directory
  2019-10-21 12:42       ` Adrian Hunter
  2019-10-21 13:46         ` Arnaldo Carvalho de Melo
@ 2019-10-21 14:03         ` Arnaldo Carvalho de Melo
  1 sibling, 0 replies; 24+ messages in thread
From: Arnaldo Carvalho de Melo @ 2019-10-21 14:03 UTC (permalink / raw)
  To: Adrian Hunter; +Cc: Jiri Olsa, linux-kernel

Em Mon, Oct 21, 2019 at 03:42:39PM +0300, Adrian Hunter escreveu:
> On 7/10/19 3:06 PM, Adrian Hunter wrote:
> > On 7/10/19 2:20 PM, Jiri Olsa wrote:
> >> On Fri, Oct 04, 2019 at 11:31:20AM +0300, Adrian Hunter wrote:
> >>
> >> SNIP
> >>
> >>>  	u8 pad[8] = {0};
> >>>  
> >>> -	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
> >>> +	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
> >>>  		off_t file_offset;
> >>>  		int fd = perf_data__fd(data);
> >>>  		int err;
> >>> diff --git a/tools/perf/util/data.c b/tools/perf/util/data.c
> >>> index df173f0bf654..964ea101dba6 100644
> >>> --- a/tools/perf/util/data.c
> >>> +++ b/tools/perf/util/data.c
> >>> @@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
> >>>  	DIR *dir;
> >>>  	int nr = 0;
> >>>  
> >>> +	/*
> >>> +	 * Directory containing a single regular perf data file which is already
> >>> +	 * open, means there is nothing more to do here.
> >>> +	 */
> >>> +	if (perf_data__is_single_file(data))
> >>> +		return 0;
> >>> +
> >>
> >> cool, I like this approach much more than the previous flag
> > 
> > Yes it is much nicer.  Thanks for your direction on that.
> > 
> >>
> >> any change you (if there's repost) or Arnaldo
> >> could squeeze in indent change below?
> > 
> > Sent a patch, to be applied before these.
> > 
> 
> That is:
> 
> "[PATCH] perf session: Fix indent in perf_session__new()"

Done it as a separate patch, etc.
 
> Any comments on this patch set Arnaldo?

-- 

- Arnaldo

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: [PATCH 0/5] perf record: Put a copy of kcore into the perf.data directory
  2019-10-04  8:31 [PATCH 0/5] perf record: Put a copy of kcore into the perf.data directory Adrian Hunter
                   ` (5 preceding siblings ...)
  2019-10-07 12:55 ` [PATCH 0/5] " Jiri Olsa
@ 2019-10-21 14:36 ` Arnaldo Carvalho de Melo
  6 siblings, 0 replies; 24+ messages in thread
From: Arnaldo Carvalho de Melo @ 2019-10-21 14:36 UTC (permalink / raw)
  To: Adrian Hunter
  Cc: Jiri Olsa, Namhyung Kim, Ingo Molnar, Thomas Gleixner, linux-kernel

Em Fri, Oct 04, 2019 at 11:31:16AM +0300, Adrian Hunter escreveu:
> Hi
> 
> Here are patches to add a new 'perf record' option '--kcore' which will put
> a copy of /proc/kcore, kallsyms and modules into a perf.data directory.
> Note, that without the --kcore option, output goes to a file as previously.
> The tools' -o and -i options work with either a file name or directory
> name.

Its great that you two got in agreement about adding the new feature
using the infrastructure that is being put in place for perf.data as a
directory, great team work.

Applied, thanks a lot,

- Arnaldo
 
> Example:
> 
>  $ sudo perf record --kcore uname
> 
>  $ sudo tree perf.data
>  perf.data
>  ├── kcore_dir
>  │   ├── kallsyms
>  │   ├── kcore
>  │   └── modules
>  └── data
> 
>  $ sudo perf script -v
>  build id event received for vmlinux: 1eaa285996affce2d74d8e66dcea09a80c9941de
>  build id event received for [vdso]: 8bbaf5dc62a9b644b4d4e4539737e104e4a84541
>  Samples for 'cycles' event do not have CPU attribute set. Skipping 'cpu' field.
>  Using CPUID GenuineIntel-6-8E-A
>  Using perf.data/kcore_dir/kcore for kernel data
>  Using perf.data/kcore_dir/kallsyms for symbols
>              perf 19058 506778.423729:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
>              perf 19058 506778.423733:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
>              perf 19058 506778.423734:          7 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
>              perf 19058 506778.423736:        117 cycles:  ffffffffa2caa54a native_write_msr+0xa (vmlinux)
>              perf 19058 506778.423738:       2092 cycles:  ffffffffa2c9b7b0 native_apic_msr_write+0x0 (vmlinux)
>              perf 19058 506778.423740:      37380 cycles:  ffffffffa2f121d0 perf_event_addr_filters_exec+0x0 (vmlinux)
>             uname 19058 506778.423751:     582673 cycles:  ffffffffa303a407 propagate_protected_usage+0x147 (vmlinux)
>             uname 19058 506778.423892:    2241841 cycles:  ffffffffa2cae0c9 unwind_next_frame.part.5+0x79 (vmlinux)
>             uname 19058 506778.424430:    2457397 cycles:  ffffffffa3019232 check_memory_region+0x52 (vmlinux)
> 
> 
> Adrian Hunter (5):
>       perf data: Correctly identify directory data files
>       perf data: Move perf_dir_version into data.h
>       perf data: Rename directory "header" file to "data"
>       perf tools: Support single perf.data file directory
>       perf record: Put a copy of kcore into the perf.data directory
> 
>  tools/perf/Documentation/perf-record.txt           |  3 ++
>  .../Documentation/perf.data-directory-format.txt   | 63 ++++++++++++++++++++++
>  tools/perf/builtin-record.c                        | 54 ++++++++++++++++++-
>  tools/perf/util/data.c                             | 46 ++++++++++++++--
>  tools/perf/util/data.h                             | 12 +++++
>  tools/perf/util/header.h                           |  4 --
>  tools/perf/util/record.h                           |  1 +
>  tools/perf/util/session.c                          |  4 ++
>  tools/perf/util/util.c                             | 19 ++++++-
>  9 files changed, 197 insertions(+), 9 deletions(-)
>  create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt
> 
> 
> Regards
> Adrian

-- 

- Arnaldo

^ permalink raw reply	[flat|nested] 24+ messages in thread

* [tip: perf/core] perf record: Put a copy of kcore into the perf.data directory
  2019-10-04  8:31 ` [PATCH 5/5] perf record: Put a copy of kcore into the perf.data directory Adrian Hunter
  2019-10-07 11:20   ` Jiri Olsa
  2019-10-07 11:20   ` Jiri Olsa
@ 2019-11-12 11:18   ` tip-bot2 for Adrian Hunter
  2 siblings, 0 replies; 24+ messages in thread
From: tip-bot2 for Adrian Hunter @ 2019-11-12 11:18 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: Adrian Hunter, Jiri Olsa, Arnaldo Carvalho de Melo, Ingo Molnar,
	Borislav Petkov, linux-kernel

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     eeb399b531a1576e36016f8a7f0c50d10194e190
Gitweb:        https://git.kernel.org/tip/eeb399b531a1576e36016f8a7f0c50d10194e190
Author:        Adrian Hunter <adrian.hunter@intel.com>
AuthorDate:    Fri, 04 Oct 2019 11:31:21 +03:00
Committer:     Arnaldo Carvalho de Melo <acme@redhat.com>
CommitterDate: Wed, 06 Nov 2019 15:43:05 -03:00

perf record: Put a copy of kcore into the perf.data directory

Add a new 'perf record' option '--kcore' which will put a copy of
/proc/kcore, kallsyms and modules into a perf.data directory. Note, that
without the --kcore option, output goes to a file as previously.  The
tools' -o and -i options work with either a file name or directory name.

Example:

  $ sudo perf record --kcore uname

  $ sudo tree perf.data
  perf.data
  ├── kcore_dir
  │   ├── kallsyms
  │   ├── kcore
  │   └── modules
  └── data

  $ sudo perf script -v
  build id event received for vmlinux: 1eaa285996affce2d74d8e66dcea09a80c9941de
  build id event received for [vdso]: 8bbaf5dc62a9b644b4d4e4539737e104e4a84541
  Samples for 'cycles' event do not have CPU attribute set. Skipping 'cpu' field.
  Using CPUID GenuineIntel-6-8E-A
  Using perf.data/kcore_dir/kcore for kernel data
  Using perf.data/kcore_dir/kallsyms for symbols
             perf 19058 506778.423729:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423733:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423734:          7 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
             perf 19058 506778.423736:        117 cycles:  ffffffffa2caa54a native_write_msr+0xa (vmlinux)
             perf 19058 506778.423738:       2092 cycles:  ffffffffa2c9b7b0 native_apic_msr_write+0x0 (vmlinux)
             perf 19058 506778.423740:      37380 cycles:  ffffffffa2f121d0 perf_event_addr_filters_exec+0x0 (vmlinux)
            uname 19058 506778.423751:     582673 cycles:  ffffffffa303a407 propagate_protected_usage+0x147 (vmlinux)
            uname 19058 506778.423892:    2241841 cycles:  ffffffffa2cae0c9 unwind_next_frame.part.5+0x79 (vmlinux)
            uname 19058 506778.424430:    2457397 cycles:  ffffffffa3019232 check_memory_region+0x52 (vmlinux)

Committer testing:

  # rm -rf perf.data*
  # perf record sleep 1
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.024 MB perf.data (7 samples) ]
  # ls -l perf.data
  -rw-------. 1 root root 34772 Oct 21 11:08 perf.data
  # perf record --kcore uname
  Linux
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.024 MB perf.data (7 samples) ]
  ls[root@quaco ~]# ls -lad perf.data*
  drwx------. 3 root root  4096 Oct 21 11:08 perf.data
  -rw-------. 1 root root 34772 Oct 21 11:08 perf.data.old
  # perf evlist -v
  cycles: size: 112, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|PERIOD, read_format: ID, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, precise_ip: 3, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1, ksymbol: 1, bpf_event: 1
  # perf evlist -v -i perf.data/data
  cycles: size: 112, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|PERIOD, read_format: ID, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, precise_ip: 3, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1, ksymbol: 1, bpf_event: 1
  #

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Reviewed-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lore.kernel.org/lkml/20191004083121.12182-6-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/Documentation/perf-record.txt                |  3 +-
 tools/perf/Documentation/perf.data-directory-format.txt | 35 +++++-
 tools/perf/builtin-record.c                             | 52 ++++++++-
 tools/perf/util/data.c                                  | 33 +++++-
 tools/perf/util/data.h                                  |  2 +-
 tools/perf/util/record.h                                |  1 +-
 tools/perf/util/session.c                               |  4 +-
 tools/perf/util/util.c                                  | 17 +++-
 8 files changed, 147 insertions(+)

diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt
index c6f9f31..8a45061 100644
--- a/tools/perf/Documentation/perf-record.txt
+++ b/tools/perf/Documentation/perf-record.txt
@@ -571,6 +571,9 @@ config terms. For example: 'cycles/overwrite/' and 'instructions/no-overwrite/'.
 
 Implies --tail-synthesize.
 
+--kcore::
+Make a copy of /proc/kcore and place it into a directory with the perf data file.
+
 SEE ALSO
 --------
 linkperf:perf-stat[1], linkperf:perf-list[1]
diff --git a/tools/perf/Documentation/perf.data-directory-format.txt b/tools/perf/Documentation/perf.data-directory-format.txt
index 4bf0890..f37fbd2 100644
--- a/tools/perf/Documentation/perf.data-directory-format.txt
+++ b/tools/perf/Documentation/perf.data-directory-format.txt
@@ -26,3 +26,38 @@ The current only version value 0 means that:
 
 Future versions are expected to describe different data files
 layout according to special needs.
+
+Currently the only 'perf record' option to output to a directory is
+the --kcore option which puts a copy of /proc/kcore into the directory.
+e.g.
+
+  $ sudo perf record --kcore uname
+  Linux
+  [ perf record: Woken up 1 times to write data ]
+  [ perf record: Captured and wrote 0.015 MB perf.data (9 samples) ]
+  $ sudo tree -ps perf.data
+  perf.data
+  ├── [-rw-------       23744]  data
+  └── [drwx------        4096]  kcore_dir
+      ├── [-r--------     6731125]  kallsyms
+      ├── [-r--------    40230912]  kcore
+      └── [-r--------        5419]  modules
+
+  1 directory, 4 files
+  $ sudo perf script -v
+  build id event received for vmlinux: 1eaa285996affce2d74d8e66dcea09a80c9941de
+  build id event received for [vdso]: 8bbaf5dc62a9b644b4d4e4539737e104e4a84541
+  build id event received for /lib/x86_64-linux-gnu/libc-2.28.so: 5b157f49586a3ca84d55837f97ff466767dd3445
+  Samples for 'cycles' event do not have CPU attribute set. Skipping 'cpu' field.
+  Using CPUID GenuineIntel-6-8E-A
+  Using perf.data/kcore_dir/kcore for kernel data
+  Using perf.data/kcore_dir/kallsyms for symbols
+              perf 15316 2060795.480902:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
+              perf 15316 2060795.480906:          1 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
+              perf 15316 2060795.480908:          7 cycles:  ffffffffa2caa548 native_write_msr+0x8 (vmlinux)
+              perf 15316 2060795.480910:        119 cycles:  ffffffffa2caa54a native_write_msr+0xa (vmlinux)
+              perf 15316 2060795.480912:       2109 cycles:  ffffffffa2c9b7b0 native_apic_msr_write+0x0 (vmlinux)
+              perf 15316 2060795.480914:      37606 cycles:  ffffffffa2f121fe perf_event_addr_filters_exec+0x2e (vmlinux)
+             uname 15316 2060795.480924:     588287 cycles:  ffffffffa303a56d page_counter_try_charge+0x6d (vmlinux)
+             uname 15316 2060795.481067:    2261945 cycles:  ffffffffa301438f kmem_cache_free+0x4f (vmlinux)
+             uname 15316 2060795.481643:    2172167 cycles:      7f1a48c393c0 _IO_un_link+0x0 (/lib/x86_64-linux-gnu/libc-2.28.so)
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index e402459..f6664bb 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -55,6 +55,9 @@
 #include <signal.h>
 #include <sys/mman.h>
 #include <sys/wait.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
 #include <linux/err.h>
 #include <linux/string.h>
 #include <linux/time64.h>
@@ -699,6 +702,37 @@ static int record__auxtrace_init(struct record *rec __maybe_unused)
 
 #endif
 
+static bool record__kcore_readable(struct machine *machine)
+{
+	char kcore[PATH_MAX];
+	int fd;
+
+	scnprintf(kcore, sizeof(kcore), "%s/proc/kcore", machine->root_dir);
+
+	fd = open(kcore, O_RDONLY);
+	if (fd < 0)
+		return false;
+
+	close(fd);
+
+	return true;
+}
+
+static int record__kcore_copy(struct machine *machine, struct perf_data *data)
+{
+	char from_dir[PATH_MAX];
+	char kcore_dir[PATH_MAX];
+	int ret;
+
+	snprintf(from_dir, sizeof(from_dir), "%s/proc", machine->root_dir);
+
+	ret = perf_data__make_kcore_dir(data, kcore_dir, sizeof(kcore_dir));
+	if (ret)
+		return ret;
+
+	return kcore_copy(from_dir, kcore_dir);
+}
+
 static int record__mmap_evlist(struct record *rec,
 			       struct evlist *evlist)
 {
@@ -1383,6 +1417,12 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 	session->header.env.comp_type  = PERF_COMP_ZSTD;
 	session->header.env.comp_level = rec->opts.comp_level;
 
+	if (rec->opts.kcore &&
+	    !record__kcore_readable(&session->machines.host)) {
+		pr_err("ERROR: kcore is not readable.\n");
+		return -1;
+	}
+
 	record__init_features(rec);
 
 	if (rec->opts.use_clockid && rec->opts.clockid_res_ns)
@@ -1414,6 +1454,14 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 	}
 	session->header.env.comp_mmap_len = session->evlist->core.mmap_len;
 
+	if (rec->opts.kcore) {
+		err = record__kcore_copy(&session->machines.host, data);
+		if (err) {
+			pr_err("ERROR: Failed to copy kcore\n");
+			goto out_child;
+		}
+	}
+
 	err = bpf__apply_obj_config();
 	if (err) {
 		char errbuf[BUFSIZ];
@@ -2184,6 +2232,7 @@ static struct option __record_options[] = {
 		     parse_cgroups),
 	OPT_UINTEGER('D', "delay", &record.opts.initial_delay,
 		  "ms to wait before starting measurement after program start"),
+	OPT_BOOLEAN(0, "kcore", &record.opts.kcore, "copy /proc/kcore"),
 	OPT_STRING('u', "uid", &record.opts.target.uid_str, "user",
 		   "user to profile"),
 
@@ -2322,6 +2371,9 @@ int cmd_record(int argc, const char **argv)
 
 	}
 
+	if (rec->opts.kcore)
+		rec->data.is_dir = true;
+
 	if (rec->opts.comp_level != 0) {
 		pr_debug("Compression enabled, disabling build id collection at the end of the session.\n");
 		rec->no_buildid = true;
diff --git a/tools/perf/util/data.c b/tools/perf/util/data.c
index 964ea10..c47aa34 100644
--- a/tools/perf/util/data.c
+++ b/tools/perf/util/data.c
@@ -424,3 +424,36 @@ unsigned long perf_data__size(struct perf_data *data)
 
 	return size;
 }
+
+int perf_data__make_kcore_dir(struct perf_data *data, char *buf, size_t buf_sz)
+{
+	int ret;
+
+	if (!data->is_dir)
+		return -1;
+
+	ret = snprintf(buf, buf_sz, "%s/kcore_dir", data->path);
+	if (ret < 0 || (size_t)ret >= buf_sz)
+		return -1;
+
+	return mkdir(buf, S_IRWXU);
+}
+
+char *perf_data__kallsyms_name(struct perf_data *data)
+{
+	char *kallsyms_name;
+	struct stat st;
+
+	if (!data->is_dir)
+		return NULL;
+
+	if (asprintf(&kallsyms_name, "%s/kcore_dir/kallsyms", data->path) < 0)
+		return NULL;
+
+	if (stat(kallsyms_name, &st)) {
+		free(kallsyms_name);
+		return NULL;
+	}
+
+	return kallsyms_name;
+}
diff --git a/tools/perf/util/data.h b/tools/perf/util/data.h
index f68815f..75947ef 100644
--- a/tools/perf/util/data.h
+++ b/tools/perf/util/data.h
@@ -87,4 +87,6 @@ int perf_data__open_dir(struct perf_data *data);
 void perf_data__close_dir(struct perf_data *data);
 int perf_data__update_dir(struct perf_data *data);
 unsigned long perf_data__size(struct perf_data *data);
+int perf_data__make_kcore_dir(struct perf_data *data, char *buf, size_t buf_sz);
+char *perf_data__kallsyms_name(struct perf_data *data);
 #endif /* __PERF_DATA_H */
diff --git a/tools/perf/util/record.h b/tools/perf/util/record.h
index 00275af..948bbcf 100644
--- a/tools/perf/util/record.h
+++ b/tools/perf/util/record.h
@@ -44,6 +44,7 @@ struct record_opts {
 	bool	      strict_freq;
 	bool	      sample_id;
 	bool	      no_bpf_event;
+	bool	      kcore;
 	unsigned int  freq;
 	unsigned int  mmap_pages;
 	unsigned int  auxtrace_mmap_pages;
diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
index 0266604..f07b8ec 100644
--- a/tools/perf/util/session.c
+++ b/tools/perf/util/session.c
@@ -230,6 +230,10 @@ struct perf_session *perf_session__new(struct perf_data *data,
 				if (ret)
 					goto out_delete;
 			}
+
+			if (!symbol_conf.kallsyms_name &&
+			    !symbol_conf.vmlinux_name)
+				symbol_conf.kallsyms_name = perf_data__kallsyms_name(data);
 		}
 	} else  {
 		session->machines.host.env = &perf_env;
diff --git a/tools/perf/util/util.c b/tools/perf/util/util.c
index 3096654..969ae56 100644
--- a/tools/perf/util/util.c
+++ b/tools/perf/util/util.c
@@ -182,6 +182,21 @@ static int rm_rf_depth_pat(const char *path, int depth, const char **pat)
 	return rmdir(path);
 }
 
+static int rm_rf_kcore_dir(const char *path)
+{
+	char kcore_dir_path[PATH_MAX];
+	const char *pat[] = {
+		"kcore",
+		"kallsyms",
+		"modules",
+		NULL,
+	};
+
+	snprintf(kcore_dir_path, sizeof(kcore_dir_path), "%s/kcore_dir", path);
+
+	return rm_rf_depth_pat(kcore_dir_path, 0, pat);
+}
+
 int rm_rf_perf_data(const char *path)
 {
 	const char *pat[] = {
@@ -190,6 +205,8 @@ int rm_rf_perf_data(const char *path)
 		NULL,
 	};
 
+	rm_rf_kcore_dir(path);
+
 	return rm_rf_depth_pat(path, 0, pat);
 }
 

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [tip: perf/core] perf data: Support single perf.data file directory
  2019-10-04  8:31 ` [PATCH 4/5] perf tools: Support single perf.data file directory Adrian Hunter
  2019-10-07 11:20   ` Jiri Olsa
@ 2019-11-12 11:18   ` tip-bot2 for Adrian Hunter
  1 sibling, 0 replies; 24+ messages in thread
From: tip-bot2 for Adrian Hunter @ 2019-11-12 11:18 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: Adrian Hunter, Jiri Olsa, Arnaldo Carvalho de Melo, Ingo Molnar,
	Borislav Petkov, linux-kernel

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     46e201efa15b70ec39df5237116fddebb4f5057c
Gitweb:        https://git.kernel.org/tip/46e201efa15b70ec39df5237116fddebb4f5057c
Author:        Adrian Hunter <adrian.hunter@intel.com>
AuthorDate:    Fri, 04 Oct 2019 11:31:20 +03:00
Committer:     Arnaldo Carvalho de Melo <acme@redhat.com>
CommitterDate: Wed, 06 Nov 2019 15:43:05 -03:00

perf data: Support single perf.data file directory

Support directory output that contains a regular perf.data file, named
"data". By default the directory is named perf.data i.e.
	perf.data
	└── data

Most of the infrastructure to support a directory is already there. This
patch makes the changes needed to support the format above.

Presently there is no 'perf record' option to output a directory.

This is preparation for adding support for putting a copy of /proc/kcore in
the directory.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Reviewed-by: Jiri Olsa <jolsa@kernel.org>
Link: http://lore.kernel.org/lkml/20191004083121.12182-5-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/Documentation/perf.data-directory-format.txt | 28 ++++++++-
 tools/perf/builtin-record.c                             |  2 +-
 tools/perf/util/data.c                                  |  9 ++-
 tools/perf/util/data.h                                  |  6 ++-
 4 files changed, 43 insertions(+), 2 deletions(-)
 create mode 100644 tools/perf/Documentation/perf.data-directory-format.txt

diff --git a/tools/perf/Documentation/perf.data-directory-format.txt b/tools/perf/Documentation/perf.data-directory-format.txt
new file mode 100644
index 0000000..4bf0890
--- /dev/null
+++ b/tools/perf/Documentation/perf.data-directory-format.txt
@@ -0,0 +1,28 @@
+perf.data directory format
+
+DISCLAIMER This is not ABI yet and is subject to possible change
+           in following versions of perf. We will remove this
+           disclaimer once the directory format soaks in.
+
+
+This document describes the on-disk perf.data directory format.
+
+The layout is described by HEADER_DIR_FORMAT feature.
+Currently it holds only version number (0):
+
+  HEADER_DIR_FORMAT = 24
+
+  struct {
+     uint64_t version;
+  }
+
+The current only version value 0 means that:
+  - there is a single perf.data file named 'data' within the directory.
+  e.g.
+
+    $ tree -ps perf.data
+    perf.data
+    └── [-rw-------       25912]  data
+
+Future versions are expected to describe different data files
+layout according to special needs.
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 2fb83aa..e402459 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -537,7 +537,7 @@ static int record__process_auxtrace(struct perf_tool *tool,
 	size_t padding;
 	u8 pad[8] = {0};
 
-	if (!perf_data__is_pipe(data) && !perf_data__is_dir(data)) {
+	if (!perf_data__is_pipe(data) && perf_data__is_single_file(data)) {
 		off_t file_offset;
 		int fd = perf_data__fd(data);
 		int err;
diff --git a/tools/perf/util/data.c b/tools/perf/util/data.c
index df173f0..964ea10 100644
--- a/tools/perf/util/data.c
+++ b/tools/perf/util/data.c
@@ -76,6 +76,13 @@ int perf_data__open_dir(struct perf_data *data)
 	DIR *dir;
 	int nr = 0;
 
+	/*
+	 * Directory containing a single regular perf data file which is already
+	 * open, means there is nothing more to do here.
+	 */
+	if (perf_data__is_single_file(data))
+		return 0;
+
 	if (WARN_ON(!data->is_dir))
 		return -EINVAL;
 
@@ -406,7 +413,7 @@ unsigned long perf_data__size(struct perf_data *data)
 	u64 size = data->file.size;
 	int i;
 
-	if (!data->is_dir)
+	if (perf_data__is_single_file(data))
 		return size;
 
 	for (i = 0; i < data->dir.nr; i++) {
diff --git a/tools/perf/util/data.h b/tools/perf/util/data.h
index 218fe9a..f68815f 100644
--- a/tools/perf/util/data.h
+++ b/tools/perf/util/data.h
@@ -10,6 +10,7 @@ enum perf_data_mode {
 };
 
 enum perf_dir_version {
+	PERF_DIR_SINGLE_FILE	= 0,
 	PERF_DIR_VERSION	= 1,
 };
 
@@ -54,6 +55,11 @@ static inline bool perf_data__is_dir(struct perf_data *data)
 	return data->is_dir;
 }
 
+static inline bool perf_data__is_single_file(struct perf_data *data)
+{
+	return data->dir.version == PERF_DIR_SINGLE_FILE;
+}
+
 static inline int perf_data__fd(struct perf_data *data)
 {
 	return data->file.fd;

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [tip: perf/core] perf data: Rename directory "header" file to "data"
  2019-10-04  8:31 ` [PATCH 3/5] perf data: Rename directory "header" file to "data" Adrian Hunter
@ 2019-11-12 11:18   ` tip-bot2 for Adrian Hunter
  0 siblings, 0 replies; 24+ messages in thread
From: tip-bot2 for Adrian Hunter @ 2019-11-12 11:18 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: Adrian Hunter, Jiri Olsa, Arnaldo Carvalho de Melo, Ingo Molnar,
	Borislav Petkov, linux-kernel

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     9b70b9db4e0cc03d224795a18088fdb916dec823
Gitweb:        https://git.kernel.org/tip/9b70b9db4e0cc03d224795a18088fdb916dec823
Author:        Adrian Hunter <adrian.hunter@intel.com>
AuthorDate:    Fri, 04 Oct 2019 11:31:19 +03:00
Committer:     Arnaldo Carvalho de Melo <acme@redhat.com>
CommitterDate: Wed, 06 Nov 2019 15:43:05 -03:00

perf data: Rename directory "header" file to "data"

In preparation to support a single file directory format, rename "header"
to "data" because "header" is a mis-leading name when there is only 1 file.
Note, in the multi-file case, the "header" file also contains data.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Reviewed-by: Jiri Olsa <jolsa@kernel.org>
Link: http://lore.kernel.org/lkml/20191004083121.12182-4-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/data.c | 2 +-
 tools/perf/util/util.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/data.c b/tools/perf/util/data.c
index 8993253..df173f0 100644
--- a/tools/perf/util/data.c
+++ b/tools/perf/util/data.c
@@ -306,7 +306,7 @@ static int open_dir(struct perf_data *data)
 	 * So far we open only the header, so we can read the data version and
 	 * layout.
 	 */
-	if (asprintf(&data->file.path, "%s/header", data->path) < 0)
+	if (asprintf(&data->file.path, "%s/data", data->path) < 0)
 		return -1;
 
 	if (perf_data__is_write(data) &&
diff --git a/tools/perf/util/util.c b/tools/perf/util/util.c
index ae56c76..3096654 100644
--- a/tools/perf/util/util.c
+++ b/tools/perf/util/util.c
@@ -185,7 +185,7 @@ static int rm_rf_depth_pat(const char *path, int depth, const char **pat)
 int rm_rf_perf_data(const char *path)
 {
 	const char *pat[] = {
-		"header",
+		"data",
 		"data.*",
 		NULL,
 	};

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [tip: perf/core] perf session: Fix indent in perf_session__new()"
  2019-10-07 11:20   ` Jiri Olsa
  2019-10-07 12:06     ` Adrian Hunter
@ 2019-11-12 11:18     ` tip-bot2 for Jiri Olsa
  1 sibling, 0 replies; 24+ messages in thread
From: tip-bot2 for Jiri Olsa @ 2019-11-12 11:18 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: Jiri Olsa, Adrian Hunter, Arnaldo Carvalho de Melo, Ingo Molnar,
	Borislav Petkov, linux-kernel

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     01e97a59ea3e071e5c60ad46c40b4db748bfe928
Gitweb:        https://git.kernel.org/tip/01e97a59ea3e071e5c60ad46c40b4db748bfe928
Author:        Jiri Olsa <jolsa@redhat.com>
AuthorDate:    Mon, 07 Oct 2019 13:20:27 +02:00
Committer:     Arnaldo Carvalho de Melo <acme@redhat.com>
CommitterDate: Wed, 06 Nov 2019 15:43:05 -03:00

perf session: Fix indent in perf_session__new()"

Fix up indentation.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Link: http://lore.kernel.org/lkml/20191007112027.GD6919@krava
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/session.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
index 6cc32f5..0266604 100644
--- a/tools/perf/util/session.c
+++ b/tools/perf/util/session.c
@@ -227,8 +227,8 @@ struct perf_session *perf_session__new(struct perf_data *data,
 			/* Open the directory data. */
 			if (data->is_dir) {
 				ret = perf_data__open_dir(data);
-			if (ret)
-				goto out_delete;
+				if (ret)
+					goto out_delete;
 			}
 		}
 	} else  {

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [tip: perf/core] perf data: Correctly identify directory data files
  2019-10-04  8:31 ` [PATCH 1/5] perf data: Correctly identify directory data files Adrian Hunter
@ 2019-11-12 11:18   ` tip-bot2 for Adrian Hunter
  0 siblings, 0 replies; 24+ messages in thread
From: tip-bot2 for Adrian Hunter @ 2019-11-12 11:18 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: Adrian Hunter, Jiri Olsa, Arnaldo Carvalho de Melo, Ingo Molnar,
	Borislav Petkov, linux-kernel

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     490e6db09a9035d7b40a328a2a004ef70b6bdee6
Gitweb:        https://git.kernel.org/tip/490e6db09a9035d7b40a328a2a004ef70b6bdee6
Author:        Adrian Hunter <adrian.hunter@intel.com>
AuthorDate:    Fri, 04 Oct 2019 11:31:17 +03:00
Committer:     Arnaldo Carvalho de Melo <acme@redhat.com>
CommitterDate: Wed, 06 Nov 2019 15:43:05 -03:00

perf data: Correctly identify directory data files

In order to rename the "header" file to "data" without conflicting,
correctly identify the non-header files as starting with "data."

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Reviewed-by: Jiri Olsa <jolsa@kernel.org>
Link: http://lore.kernel.org/lkml/20191004083121.12182-2-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/data.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/util/data.c b/tools/perf/util/data.c
index 88fba2b..8993253 100644
--- a/tools/perf/util/data.c
+++ b/tools/perf/util/data.c
@@ -96,7 +96,7 @@ int perf_data__open_dir(struct perf_data *data)
 		if (stat(path, &st))
 			continue;
 
-		if (!S_ISREG(st.st_mode) || strncmp(dent->d_name, "data", 4))
+		if (!S_ISREG(st.st_mode) || strncmp(dent->d_name, "data.", 5))
 			continue;
 
 		ret = -ENOMEM;

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* [tip: perf/core] perf data: Move perf_dir_version into data.h
  2019-10-04  8:31 ` [PATCH 2/5] perf data: Move perf_dir_version into data.h Adrian Hunter
@ 2019-11-12 11:18   ` tip-bot2 for Adrian Hunter
  0 siblings, 0 replies; 24+ messages in thread
From: tip-bot2 for Adrian Hunter @ 2019-11-12 11:18 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: Adrian Hunter, Jiri Olsa, Arnaldo Carvalho de Melo, Ingo Molnar,
	Borislav Petkov, linux-kernel

The following commit has been merged into the perf/core branch of tip:

Commit-ID:     3dedec4f5ccc8048b9a2cfe89838c3b3275b6b2b
Gitweb:        https://git.kernel.org/tip/3dedec4f5ccc8048b9a2cfe89838c3b3275b6b2b
Author:        Adrian Hunter <adrian.hunter@intel.com>
AuthorDate:    Fri, 04 Oct 2019 11:31:18 +03:00
Committer:     Arnaldo Carvalho de Melo <acme@redhat.com>
CommitterDate: Wed, 06 Nov 2019 15:43:05 -03:00

perf data: Move perf_dir_version into data.h

perf_dir_version belongs to struct perf_data which is declared in data.h.
To allow its use in inline perf_data functions, move perf_dir_version to
data.h

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Reviewed-by: Jiri Olsa <jolsa@kernel.org>
Link: http://lore.kernel.org/lkml/20191004083121.12182-3-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/util/data.h   | 4 ++++
 tools/perf/util/header.h | 4 ----
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/perf/util/data.h b/tools/perf/util/data.h
index 259868a..218fe9a 100644
--- a/tools/perf/util/data.h
+++ b/tools/perf/util/data.h
@@ -9,6 +9,10 @@ enum perf_data_mode {
 	PERF_DATA_MODE_READ,
 };
 
+enum perf_dir_version {
+	PERF_DIR_VERSION	= 1,
+};
+
 struct perf_data_file {
 	char		*path;
 	int		 fd;
diff --git a/tools/perf/util/header.h b/tools/perf/util/header.h
index ca53a92..840f95c 100644
--- a/tools/perf/util/header.h
+++ b/tools/perf/util/header.h
@@ -52,10 +52,6 @@ enum perf_header_version {
 	PERF_HEADER_VERSION_2,
 };
 
-enum perf_dir_version {
-	PERF_DIR_VERSION	= 1,
-};
-
 struct perf_file_section {
 	u64 offset;
 	u64 size;

^ permalink raw reply related	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2019-11-12 11:20 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-04  8:31 [PATCH 0/5] perf record: Put a copy of kcore into the perf.data directory Adrian Hunter
2019-10-04  8:31 ` [PATCH 1/5] perf data: Correctly identify directory data files Adrian Hunter
2019-11-12 11:18   ` [tip: perf/core] " tip-bot2 for Adrian Hunter
2019-10-04  8:31 ` [PATCH 2/5] perf data: Move perf_dir_version into data.h Adrian Hunter
2019-11-12 11:18   ` [tip: perf/core] " tip-bot2 for Adrian Hunter
2019-10-04  8:31 ` [PATCH 3/5] perf data: Rename directory "header" file to "data" Adrian Hunter
2019-11-12 11:18   ` [tip: perf/core] " tip-bot2 for Adrian Hunter
2019-10-04  8:31 ` [PATCH 4/5] perf tools: Support single perf.data file directory Adrian Hunter
2019-10-07 11:20   ` Jiri Olsa
2019-10-07 12:06     ` Adrian Hunter
2019-10-21 12:42       ` Adrian Hunter
2019-10-21 13:46         ` Arnaldo Carvalho de Melo
2019-10-21 13:52           ` Jiri Olsa
2019-10-21 14:03         ` Arnaldo Carvalho de Melo
2019-11-12 11:18     ` [tip: perf/core] perf session: Fix indent in perf_session__new()" tip-bot2 for Jiri Olsa
2019-11-12 11:18   ` [tip: perf/core] perf data: Support single perf.data file directory tip-bot2 for Adrian Hunter
2019-10-04  8:31 ` [PATCH 5/5] perf record: Put a copy of kcore into the perf.data directory Adrian Hunter
2019-10-07 11:20   ` Jiri Olsa
2019-10-07 12:06     ` Adrian Hunter
2019-10-07 11:20   ` Jiri Olsa
2019-10-07 12:14     ` Adrian Hunter
2019-11-12 11:18   ` [tip: perf/core] " tip-bot2 for Adrian Hunter
2019-10-07 12:55 ` [PATCH 0/5] " Jiri Olsa
2019-10-21 14:36 ` Arnaldo Carvalho de Melo

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.