* [PATCH v6 0/4] perf: add support for profiling jitted code
@ 2015-03-30 22:19 Stephane Eranian
2015-03-30 22:19 ` [PATCH v6 1/4] perf,record: Add clockid parameter Stephane Eranian
` (4 more replies)
0 siblings, 5 replies; 18+ messages in thread
From: Stephane Eranian @ 2015-03-30 22:19 UTC (permalink / raw)
To: linux-kernel
Cc: acme, peterz, mingo, ak, jolsa, namhyung, cel, sukadev, sonnyrao,
johnmccutchan, dsahern, adrian.hunter, pawel.moll
This patch series extends perf record/report/annotate to enable
profiling of jitted (just-in-time compiled) code. The current
perf tool provides very limited support for profiling jitted
code for some runtime environments. But the support is experimental
and cannot be used in complex environments. It relies on files
in /tmp, for instance. It does not support annotate mode or
rejitted code.
This patch series adds a better way of profiling jitted code
with the following advantages:
- support any jitted code environment (some with modifications)
- support Java runtime with JVMTI interface with no modifications
- provides a portable JVMTI agent library
- known to support V8 runtime
- known to support DART runtime
- supports code rejitting and movements
- no files in /tmp
- meta-data file is unique to each run
- no changes to perf report/annotate
- support per-thread and system-wide profiling
- support monitoring of multiple simultaneous Jit runtimes
The support is based on cooperation with the runtime. For Java runtimes,
supporting the JVMTI interface, there is no change necessary. For other
runtimes, modifications are necessary to emit the meta-data necessary
to support symbolization and annotation of the samples. Those modifications
are fairly straighforward and already tested on V8 and DART.
The jit environment emits a binary dump file which contains the jitted
code (in raw format) and meta-data describing the mapping of functions.
The binary format is documented in the jitdump.h header file. It is
adapted from the OProfile jitdump format.
To enable synchronization of the runtime MMAPs with those recorded by
the kernel on behalf of the perf tool, the runtime needs to timestamp
any record in the dump file using the same time source. This third
version is using Pawel Moll's CLOCK_MONOTONIC patch posted
https://lkml.org/lkml/2015/1/21/711
The patch is packaged in this series for convenience. Without this patch
installed, records emitted by the runtime cannot be properly synchronized
(inserted in the flow of MMAPS) and symbolization is not possible.
The current support only works when the runtime is monitored from
start to finish: perf record java --agentpath:libpfmjvmti.so my_class.
Once the run is completed, the jitdump file needs to be injected into
the perf.data file. This is accomplished by using the perf inject command.
This will also generate an ELF image for each jitted function. The
inject MMAP records will point to those ELF images. The reasoning
behind using ELF images is that it makes processing for perf report
and annotate automatic and transparent. It also makes it easier to
package and analyze on a remote machine.
The reporting is unchanged, simply invoke perf report or perf annotate
on the modified perf.data file. The jitted code will appear symbolized
and the assembly view will display the instruction level profile!
As an added bonus, the series includes support for demangling function
signature from OpenJDK.
The current patch series does not include support for source line
information. Therefore, the source code cannot yet be displayed
in perf annotate. This will come in a later update.
Furthermore, we believe there is a way to skip the perf inject phase
and have perf report/annotate directly inject the MMAP records
on the fly during processing of the perf.data file. Perf report would
also generate the ELF files if necessary. Such optimization, would
make using this extension seamless in system-wide mode and larger
environments. This will be added in a later update as well.
In V2, we have switched to Pawell Moll and David Ahern posix
clock kernel module instead. We have dropped the patch which
modified the arguments to map_init() because the change was
not used. We are not printing the return type of Java methods
anymore and have made the Java demangler a separate module.
We also rebased to 3.19.0+ from tip.git.
In V3, we switched to Pawel Moll's CLOCK_MONOTONIC perf
clock patches. This patch switch perf_events from sched_clock
to CLOCK_MONOTONIC, a clock source which is available to users.
In V4, we rebased to 4.0-rc5. We also simplified the process by
getting rid of the requirement to pass the jitdump file name to
perf inject. Now, perf injects automtically detects if jitdumps
were generated and it merges the relevant meta-data. This is
accomplished by having the jit runtime mmap the jitdump file
for the purpose of creating a MMAP record in the perf.data file.
That MMAP contains all the info to locate the jitdump file and
generate the ELF images for jitted functions.
In V5, we rebase to acme's perf/core branch (instead of tip.git).
We fixed some bswap issues, switched to using scnprintf() and fixed
formatting issues. Also made sure all the files were included in the
patches. We also fix one error message in the JVMTI agent.
In V6, we switched back to using tip.git to leverage PeterZ's clockid
patch for perf_events in 4.0.0-rc6. Clock source can now be specified
per event and they are connected with the MONOTONIC Posix clock. We
leverage this extension to timestamp samples in the jit runtime and
correlate them with perf samples. Notice the -k mono option in perf
record example below.
To use the new feature:
- need to run with 4.0.0-rc6
- compile perf
- cd tools/perf/jvmti; make; install wherever is appropriate
Example using openJDK:
$ perf record -k mono java -agentpath:libjvmti.so my_class
$ perf inject -i perf.data -jit -o perf.data.jitted
$ perf report -i perf.data.jitted
Thanks to all the contributors and testers. Special thanks
to PeterZ for adding the clock source to perf_events and solving
the problem of common timesource for user and kernel level samples.
Enjoy,
Peter Zijlstra (1):
perf,record: Add clockid parameter
Stephane Eranian (3):
perf tools: add Java demangling support
perf inject: add jitdump mmap injection support
perf tools: add JVMTI agent library
tools/build/Makefile.feature | 2 +
tools/build/feature/Makefile | 4 +
tools/build/feature/test-libcrypto.c | 9 +
tools/perf/Documentation/perf-inject.txt | 6 +
tools/perf/Documentation/perf-record.txt | 7 +
tools/perf/builtin-inject.c | 60 +++-
tools/perf/builtin-record.c | 78 ++++
tools/perf/config/Makefile | 11 +
tools/perf/jvmti/Makefile | 70 ++++
tools/perf/jvmti/jvmti_agent.c | 367 +++++++++++++++++++
tools/perf/jvmti/jvmti_agent.h | 23 ++
tools/perf/jvmti/libjvmti.c | 149 ++++++++
tools/perf/perf.h | 1 +
tools/perf/util/Build | 3 +
tools/perf/util/demangle-java.c | 199 +++++++++++
tools/perf/util/demangle-java.h | 10 +
tools/perf/util/evsel.c | 31 +-
tools/perf/util/genelf.c | 479 +++++++++++++++++++++++++
tools/perf/util/genelf.h | 6 +
tools/perf/util/header.c | 2 +
tools/perf/util/jit.h | 15 +
tools/perf/util/jitdump.c | 588 +++++++++++++++++++++++++++++++
tools/perf/util/jitdump.h | 92 +++++
tools/perf/util/symbol-elf.c | 3 +
24 files changed, 2201 insertions(+), 14 deletions(-)
create mode 100644 tools/build/feature/test-libcrypto.c
create mode 100644 tools/perf/jvmti/Makefile
create mode 100644 tools/perf/jvmti/jvmti_agent.c
create mode 100644 tools/perf/jvmti/jvmti_agent.h
create mode 100644 tools/perf/jvmti/libjvmti.c
create mode 100644 tools/perf/util/demangle-java.c
create mode 100644 tools/perf/util/demangle-java.h
create mode 100644 tools/perf/util/genelf.c
create mode 100644 tools/perf/util/genelf.h
create mode 100644 tools/perf/util/jit.h
create mode 100644 tools/perf/util/jitdump.c
create mode 100644 tools/perf/util/jitdump.h
--
1.9.1
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v6 1/4] perf,record: Add clockid parameter
2015-03-30 22:19 [PATCH v6 0/4] perf: add support for profiling jitted code Stephane Eranian
@ 2015-03-30 22:19 ` Stephane Eranian
2015-03-30 22:24 ` David Ahern
` (2 more replies)
2015-03-30 22:19 ` [PATCH v6 2/4] perf tools: add Java demangling support Stephane Eranian
` (3 subsequent siblings)
4 siblings, 3 replies; 18+ messages in thread
From: Stephane Eranian @ 2015-03-30 22:19 UTC (permalink / raw)
To: linux-kernel
Cc: acme, peterz, mingo, ak, jolsa, namhyung, cel, sukadev, sonnyrao,
johnmccutchan, dsahern, adrian.hunter, pawel.moll
From: Peter Zijlstra <peterz@infradead.org>
Teach perf-record about the new perf_event_attr:{use_+clockid, clockid}
fields. Add a simple parameter to set the clock (if any) to be used for
the events to be recorded into the dat file.
Incorporated improvements suggested by from David Ahern <dsahern@gmail.com>
on LKML.
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
---
tools/perf/Documentation/perf-record.txt | 7 +++
tools/perf/builtin-record.c | 78 ++++++++++++++++++++++++++++++++
tools/perf/perf.h | 1 +
tools/perf/util/evsel.c | 31 ++++++++++++-
| 2 +
5 files changed, 118 insertions(+), 1 deletion(-)
diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt
index 355c4f5..4847a79 100644
--- a/tools/perf/Documentation/perf-record.txt
+++ b/tools/perf/Documentation/perf-record.txt
@@ -250,6 +250,13 @@ is off by default.
--running-time::
Record running and enabled time for read events (:S)
+-k::
+--clockid::
+Sets the clock id to use for the various time fields in the perf_event_type
+records. See clock_gettime(). In particular CLOCK_MONOTONIC and
+CLOCK_MONOTONIC_RAW are supported, some events might also allow
+CLOCK_BOOTTIME, CLOCK_REALTIME and CLOCK_TAI.
+
SEE ALSO
--------
linkperf:perf-stat[1], linkperf:perf-list[1]
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 18aad239..fc42a9a 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -711,6 +711,80 @@ static int perf_record_config(const char *var, const char *value, void *cb)
return perf_default_config(var, value, cb);
}
+struct clockid_map {
+ const char *name;
+ int clockid;
+};
+
+#define CLOCKID_MAP(n, c) \
+ { .name = n, .clockid = (c), }
+
+#define CLOCKID_END { .name = NULL, }
+
+/*
+ * Doesn't appear to have made it into userspace so define here if missing.
+ */
+#ifndef CLOCK_TAI
+#define CLOCK_TAI 11
+#endif
+
+static const struct clockid_map clockids[] = {
+ /* available for all events, NMI safe */
+ CLOCKID_MAP("monotonic", CLOCK_MONOTONIC),
+ CLOCKID_MAP("monotonic_raw", CLOCK_MONOTONIC_RAW),
+
+ /* available for some events */
+ CLOCKID_MAP("realtime", CLOCK_REALTIME),
+ CLOCKID_MAP("boottime", CLOCK_BOOTTIME),
+ CLOCKID_MAP("tai", CLOCK_TAI),
+
+ /* available for the lazy */
+ CLOCKID_MAP("mono", CLOCK_MONOTONIC),
+ CLOCKID_MAP("raw", CLOCK_MONOTONIC_RAW),
+ CLOCKID_MAP("real", CLOCK_REALTIME),
+
+
+ CLOCKID_END,
+};
+
+static int parse_clockid(const struct option *opt, const char *str, int unset)
+{
+ clockid_t *clk = (clockid_t *)opt->value;
+ const struct clockid_map *cm;
+ const char *ostr = str;
+
+ if (unset) {
+ *clk = -1;
+ return 0;
+ }
+
+ /* no arg passed */
+ if (!str)
+ return 0;
+
+ /* no setting it twice */
+ if (*clk != -1)
+ return -1;
+
+ /* if its a number, we're done */
+ if (sscanf(str, "%d", clk) == 1)
+ return 0;
+
+ /* allow a "CLOCK_" prefix to the name */
+ if (!strncasecmp(str, "CLOCK_", 6))
+ str += 6;
+
+ for (cm = clockids; cm->name; cm++) {
+ if (!strcasecmp(str, cm->name)) {
+ *clk = cm->clockid;
+ return 0;
+ }
+ }
+
+ ui__warning("unknown clockid %s, check man page\n", ostr);
+ return -1;
+}
+
static const char * const __record_usage[] = {
"perf record [<options>] [<command>]",
"perf record [<options>] -- <command> [<options>]",
@@ -739,6 +813,7 @@ static struct record record = {
.uses_mmap = true,
.default_per_cpu = true,
},
+ .clockid = -1,
},
.tool = {
.sample = process_sample_event,
@@ -842,6 +917,9 @@ struct option __record_options[] = {
"Sample machine registers on interrupt"),
OPT_BOOLEAN(0, "running-time", &record.opts.running_time,
"Record running/enabled time of read (:S) events"),
+ OPT_CALLBACK('k', "clockid", &record.opts.clockid,
+ "clockid", "clockid to use for events, see clock_gettime()",
+ parse_clockid),
OPT_END()
};
diff --git a/tools/perf/perf.h b/tools/perf/perf.h
index c38a085..275c0c5 100644
--- a/tools/perf/perf.h
+++ b/tools/perf/perf.h
@@ -62,6 +62,7 @@ struct record_opts {
u64 user_interval;
bool sample_transaction;
unsigned initial_delay;
+ clockid_t clockid;
};
struct option;
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index 358e595..820e789 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -32,6 +32,7 @@ static struct {
bool exclude_guest;
bool mmap2;
bool cloexec;
+ bool clockid;
} perf_missing_features;
static int perf_evsel__no_extra_init(struct perf_evsel *evsel __maybe_unused)
@@ -761,6 +762,12 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts)
attr->disabled = 0;
attr->enable_on_exec = 0;
}
+
+ if (opts->clockid >= 0) {
+ attr->use_clockid = 1;
+ attr->clockid = opts->clockid;
+ } else
+ attr->clockid = -1;
}
static int perf_evsel__alloc_fd(struct perf_evsel *evsel, int ncpus, int nthreads)
@@ -1044,6 +1051,8 @@ static size_t perf_event_attr__fprintf(struct perf_event_attr *attr, FILE *fp)
ret += PRINT_ATTR2(exclude_host, exclude_guest);
ret += PRINT_ATTR2N("excl.callchain_kern", exclude_callchain_kernel,
"excl.callchain_user", exclude_callchain_user);
+ ret += __PRINT_ATTR("%u",,use_clockid);
+
ret += PRINT_ATTR_U32(wakeup_events);
ret += PRINT_ATTR_U32(wakeup_watermark);
@@ -1056,6 +1065,7 @@ static size_t perf_event_attr__fprintf(struct perf_event_attr *attr, FILE *fp)
ret += PRINT_ATTR_X64(sample_regs_user);
ret += PRINT_ATTR_U32(sample_stack_user);
ret += PRINT_ATTR_X64(sample_regs_intr);
+ ret += PRINT_ATTR_U32(clockid);
ret += fprintf(fp, "%.60s\n", graph_dotted_line);
@@ -1122,6 +1132,16 @@ static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
goto try_fallback;
}
set_rlimit = NO_CHANGE;
+
+ /*
+ * If we succeeded but had to kill clockid, fail and
+ * have perf_evsel__open_strerror() print us a nice
+ * error.
+ */
+ if (perf_missing_features.clockid) {
+ err = -EINVAL;
+ goto out_close;
+ }
}
}
@@ -1155,7 +1175,10 @@ static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
if (err != -EINVAL || cpu > 0 || thread > 0)
goto out_close;
- if (!perf_missing_features.cloexec && (flags & PERF_FLAG_FD_CLOEXEC)) {
+ if (!perf_missing_features.clockid && evsel->attr.use_clockid) {
+ perf_missing_features.clockid = true;
+ goto fallback_missing_features;
+ } else if (!perf_missing_features.cloexec && (flags & PERF_FLAG_FD_CLOEXEC)) {
perf_missing_features.cloexec = true;
goto fallback_missing_features;
} else if (!perf_missing_features.mmap2 && evsel->attr.mmap2) {
@@ -2080,6 +2103,8 @@ int perf_evsel__fprintf(struct perf_evsel *evsel,
if_print(wakeup_events);
if_print(bp_type);
if_print(branch_sample_type);
+ if_print(clockid);
+
}
out:
fputc('\n', fp);
@@ -2158,6 +2183,10 @@ int perf_evsel__open_strerror(struct perf_evsel *evsel, struct target *target,
"The PMU counters are busy/taken by another profiler.\n"
"We found oprofile daemon running, please stop it and try again.");
break;
+ case EINVAL:
+ if (perf_missing_features.clockid)
+ return scnprintf(msg, size, "%d %s", evsel->attr.clockid, "clockid not supported.");
+ break;
default:
break;
}
--git a/tools/perf/util/header.c b/tools/perf/util/header.c
index fb43215..164441a 100644
--- a/tools/perf/util/header.c
+++ b/tools/perf/util/header.c
@@ -1098,6 +1098,8 @@ static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)
}
fprintf(fp, " }");
}
+ fprintf(fp, ", clockid = %d", evsel->attr.clockid);
+
fputc('\n', fp);
}
--
1.9.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v6 2/4] perf tools: add Java demangling support
2015-03-30 22:19 [PATCH v6 0/4] perf: add support for profiling jitted code Stephane Eranian
2015-03-30 22:19 ` [PATCH v6 1/4] perf,record: Add clockid parameter Stephane Eranian
@ 2015-03-30 22:19 ` Stephane Eranian
2015-03-31 7:00 ` Pekka Enberg
2015-03-30 22:19 ` [PATCH v6 3/4] perf inject: add jitdump mmap injection support Stephane Eranian
` (2 subsequent siblings)
4 siblings, 1 reply; 18+ messages in thread
From: Stephane Eranian @ 2015-03-30 22:19 UTC (permalink / raw)
To: linux-kernel
Cc: acme, peterz, mingo, ak, jolsa, namhyung, cel, sukadev, sonnyrao,
johnmccutchan, dsahern, adrian.hunter, pawel.moll
Add Java function descriptor demangling support.
Something bfd cannot do.
Use the JAVA_DEMANGLE_NORET flag to avoid decoding the
return type of functions.
Signed-off-by: Stephane Eranian <eranian@google.com>
---
tools/perf/util/Build | 1 +
tools/perf/util/demangle-java.c | 199 ++++++++++++++++++++++++++++++++++++++++
tools/perf/util/demangle-java.h | 10 ++
tools/perf/util/symbol-elf.c | 3 +
4 files changed, 213 insertions(+)
create mode 100644 tools/perf/util/demangle-java.c
create mode 100644 tools/perf/util/demangle-java.h
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 797490a..9bff65e 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -74,6 +74,7 @@ libperf-y += data.o
libperf-$(CONFIG_X86) += tsc.o
libperf-y += cloexec.o
libperf-y += thread-stack.o
+libperf-y += demangle-java.o
libperf-$(CONFIG_LIBELF) += symbol-elf.o
libperf-$(CONFIG_LIBELF) += probe-event.o
diff --git a/tools/perf/util/demangle-java.c b/tools/perf/util/demangle-java.c
new file mode 100644
index 0000000..19b7c06
--- /dev/null
+++ b/tools/perf/util/demangle-java.c
@@ -0,0 +1,199 @@
+#include <sys/types.h>
+#include <stdio.h>
+#include <string.h>
+#include "util.h"
+#include "debug.h"
+#include "symbol.h"
+
+#include "demangle-java.h"
+
+enum {
+ MODE_PREFIX=0,
+ MODE_CLASS=1,
+ MODE_FUNC=2,
+ MODE_TYPE=3,
+ MODE_CTYPE=3, /* class arg */
+};
+
+#define BASE_ENT(c, n) [c-'A']=n
+static const char *base_types['Z'-'A' + 1]={
+ BASE_ENT('B', "byte" ),
+ BASE_ENT('C', "char" ),
+ BASE_ENT('D', "double" ),
+ BASE_ENT('F', "float" ),
+ BASE_ENT('I', "int" ),
+ BASE_ENT('J', "long" ),
+ BASE_ENT('S', "short" ),
+ BASE_ENT('Z', "bool" ),
+};
+
+/*
+ * demangle Java symbol between str and end positions and stores
+ * up to maxlen characters into buf. The parser starts in mode.
+ *
+ * Use MODE_PREFIX to process entire prototype till end position
+ * Use MODE_TYPE to process return type if str starts on return type char
+ *
+ * Return:
+ * success: buf
+ * error : NULL
+ */
+static char *
+__demangle_java_sym(const char *str, const char *end, char *buf, int maxlen, int mode)
+{
+ int rlen = 0;
+ int array = 0;
+ int narg = 0;
+ const char *q;
+
+ if (!end)
+ end = str + strlen(str);
+
+ for (q = str; q != end; q++) {
+
+ if (rlen == (maxlen - 1))
+ break;
+
+ switch (*q) {
+ case 'L':
+ if (mode == MODE_PREFIX || mode == MODE_CTYPE) {
+ if (mode == MODE_CTYPE) {
+ if (narg)
+ rlen += scnprintf(buf + rlen, maxlen - rlen, ", ");
+ narg++;
+ }
+ rlen += scnprintf(buf + rlen, maxlen - rlen, "class ");
+ if (mode == MODE_PREFIX)
+ mode = MODE_CLASS;
+ } else
+ buf[rlen++] = *q;
+ break;
+ case 'B':
+ case 'C':
+ case 'D':
+ case 'F':
+ case 'I':
+ case 'J':
+ case 'S':
+ case 'Z':
+ if (mode == MODE_TYPE) {
+ if (narg)
+ rlen += scnprintf(buf + rlen, maxlen - rlen, ", ");
+ rlen += scnprintf(buf+rlen, maxlen - rlen, "%s", base_types[*q - 'A']);
+ while(array--)
+ rlen += scnprintf(buf + rlen, maxlen - rlen, "[]");
+ array = 0;
+ narg++;
+ } else
+ buf[rlen++] = *q;
+ break;
+ case 'V':
+ if (mode == MODE_TYPE) {
+ rlen += scnprintf(buf + rlen, maxlen - rlen, "void");
+ while(array--)
+ rlen += scnprintf(buf + rlen, maxlen - rlen, "[]");
+ array = 0;
+ } else
+ buf[rlen++] = *q;
+ break;
+ case '[':
+ if (mode != MODE_TYPE)
+ goto error;
+ array++;
+ break;
+ case '(':
+ if (mode != MODE_FUNC)
+ goto error;
+ buf[rlen++] = *q;
+ mode = MODE_TYPE;
+ break;
+ case ')':
+ if (mode != MODE_TYPE)
+ goto error;
+ buf[rlen++] = *q;
+ narg = 0;
+ break;
+ case ';':
+ if (mode != MODE_CLASS && mode != MODE_CTYPE)
+ goto error;
+ /* safe because at least one other char to process */
+ if (isalpha(*(q+1)))
+ rlen += scnprintf(buf + rlen, maxlen - rlen, ".");
+ if (mode == MODE_CLASS)
+ mode = MODE_FUNC;
+ else if (mode == MODE_CTYPE)
+ mode = MODE_TYPE;
+ break;
+ case '/':
+ if (mode != MODE_CLASS && mode != MODE_CTYPE)
+ goto error;
+ rlen += scnprintf(buf + rlen, maxlen - rlen, ".");
+ break;
+ default :
+ buf[rlen++] = *q;
+ }
+ }
+ buf[rlen] = '\0';
+ return buf;
+error:
+ return NULL;
+}
+
+/*
+ * Demangle Java function signature (openJDK, not GCJ)
+ * input:
+ * str: string to parse. String is not modified
+ * flags: comobination of JAVA_DEMANGLE_* flags to modify demangling
+ * return:
+ * if input can be demangled, then a newly allocated string is returned.
+ * if input cannot be demangled, then NULL is returned
+ *
+ * Note: caller is responsible for freeing demangled string
+ */
+char *
+java_demangle_sym(const char *str, int flags)
+{
+ char *buf, *ptr;
+ char *p;
+ size_t len, l1 = 0;
+
+ if (!str)
+ return NULL;
+
+ /* find start of retunr type */
+ p = strrchr(str, ')');
+ if (!p)
+ return NULL;
+
+ /*
+ * expansion factor estimated to 3x
+ */
+ len = strlen(str) * 3 + 1;
+ buf = malloc(len);
+ if (!buf)
+ return NULL;
+
+ buf[0] = '\0';
+ if (!(flags & JAVA_DEMANGLE_NORET)) {
+ /*
+ * get return type first
+ */
+ ptr = __demangle_java_sym(p + 1, NULL, buf, len, MODE_TYPE);
+ if (!ptr)
+ goto error;
+
+ /* add space between return type and function prototype */
+ l1 = strlen(buf);
+ buf[l1++] = ' ';
+ }
+
+ /* process function up to return type */
+ ptr = __demangle_java_sym(str, p + 1, buf + l1, len - l1, MODE_PREFIX);
+ if (!ptr)
+ goto error;
+
+ return buf;
+error:
+ free(buf);
+ return NULL;
+}
diff --git a/tools/perf/util/demangle-java.h b/tools/perf/util/demangle-java.h
new file mode 100644
index 0000000..a981c1f
--- /dev/null
+++ b/tools/perf/util/demangle-java.h
@@ -0,0 +1,10 @@
+#ifndef __PERF_DEMANGLE_JAVA
+#define __PERF_DEMANGLE_JAVA 1
+/*
+ * demangle function flags
+ */
+#define JAVA_DEMANGLE_NORET 0x1 /* do not process return type */
+
+char * java_demangle_sym(const char *str, int flags);
+
+#endif /* __PERF_DEMANGLE_JAVA */
diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
index 476268c..2174bf4 100644
--- a/tools/perf/util/symbol-elf.c
+++ b/tools/perf/util/symbol-elf.c
@@ -6,6 +6,7 @@
#include <inttypes.h>
#include "symbol.h"
+#include "demangle-java.h"
#include "machine.h"
#include "vdso.h"
#include <symbol/kallsyms.h>
@@ -1046,6 +1047,8 @@ int dso__load_sym(struct dso *dso, struct map *map,
demangle_flags = DMGL_PARAMS | DMGL_ANSI;
demangled = bfd_demangle(NULL, elf_name, demangle_flags);
+ if (demangled == NULL)
+ demangled = java_demangle_sym(elf_name, JAVA_DEMANGLE_NORET);
if (demangled != NULL)
elf_name = demangled;
}
--
1.9.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v6 3/4] perf inject: add jitdump mmap injection support
2015-03-30 22:19 [PATCH v6 0/4] perf: add support for profiling jitted code Stephane Eranian
2015-03-30 22:19 ` [PATCH v6 1/4] perf,record: Add clockid parameter Stephane Eranian
2015-03-30 22:19 ` [PATCH v6 2/4] perf tools: add Java demangling support Stephane Eranian
@ 2015-03-30 22:19 ` Stephane Eranian
2015-04-01 6:58 ` Adrian Hunter
2015-03-30 22:19 ` [PATCH v6 4/4] perf tools: add JVMTI agent library Stephane Eranian
2015-03-31 7:33 ` [PATCH v6 0/4] perf: add support for profiling jitted code Brendan Gregg
4 siblings, 1 reply; 18+ messages in thread
From: Stephane Eranian @ 2015-03-30 22:19 UTC (permalink / raw)
To: linux-kernel
Cc: acme, peterz, mingo, ak, jolsa, namhyung, cel, sukadev, sonnyrao,
johnmccutchan, dsahern, adrian.hunter, pawel.moll
This patch adds a --jit option to perf inject.
This options injects MMAP records into the perf.data
file to cover the jitted code mmaps. It also emits
ELF images for each function in the jidump file.
Those images are created where the jitdump file is.
The MMAP records point to that location as well.
Typical flow:
$ perf record -k mono -- java -agentpath:libpjvmti.so java_class
$ perf inject --jit -i perf.data -o perf.data.jitted
$ perf report -i perf.data.jitted
Note that jitdump.h support is not limited to Java, it works with
any jitted environment modified to emit the jitdump file format,
include those where code can be jitted multiple times and moved
around.
The jitdump.h format is adapted from the Oprofile project.
The genelf.c (ELF binary generation) depends on MD5 hash
encoding for the buildid. To enable this, libssl-dev must
be installed. If not, then genelf.c defaults to using
urandom to generate the buildid, which is not ideal.
The Makefile auto-detects the presence on libssl-dev.
This version mmaps the jitdump file to create a marker
MMAP record in the perf.data file. The marker is used to detect
jitdump and cause perf inject to inject the jitted mmaps and
generate ELF images for jitted functions.
Signed-off-by: Stephane Eranian <eranian@google.com>
---
tools/build/Makefile.feature | 2 +
tools/build/feature/Makefile | 4 +
tools/build/feature/test-libcrypto.c | 9 +
tools/perf/Documentation/perf-inject.txt | 6 +
tools/perf/builtin-inject.c | 60 +++-
tools/perf/config/Makefile | 11 +
tools/perf/util/Build | 2 +
tools/perf/util/genelf.c | 479 +++++++++++++++++++++++++
tools/perf/util/genelf.h | 6 +
tools/perf/util/jit.h | 15 +
tools/perf/util/jitdump.c | 588 +++++++++++++++++++++++++++++++
tools/perf/util/jitdump.h | 92 +++++
12 files changed, 1261 insertions(+), 13 deletions(-)
create mode 100644 tools/build/feature/test-libcrypto.c
create mode 100644 tools/perf/util/genelf.c
create mode 100644 tools/perf/util/genelf.h
create mode 100644 tools/perf/util/jit.h
create mode 100644 tools/perf/util/jitdump.c
create mode 100644 tools/perf/util/jitdump.h
diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
index 3a0b0ca..8b468a3 100644
--- a/tools/build/Makefile.feature
+++ b/tools/build/Makefile.feature
@@ -45,6 +45,7 @@ FEATURE_TESTS = \
libpython \
libpython-version \
libslang \
+ libcrypto \
libunwind \
pthread-attr-setaffinity-np \
stackprotector-all \
@@ -64,6 +65,7 @@ FEATURE_DISPLAY = \
libperl \
libpython \
libslang \
+ libcrypto \
libunwind \
libdw-dwarf-unwind \
zlib \
diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
index 463ed8f..ed86700 100644
--- a/tools/build/feature/Makefile
+++ b/tools/build/feature/Makefile
@@ -23,6 +23,7 @@ FILES= \
test-libpython.bin \
test-libpython-version.bin \
test-libslang.bin \
+ test-libcrypto.bin \
test-libunwind.bin \
test-libunwind-debug-frame.bin \
test-pthread-attr-setaffinity-np.bin \
@@ -93,6 +94,9 @@ __BUILD = $(CC) $(CFLAGS) -Wall -Werror -o $(OUTPUT)$@ $(patsubst %.bin,%.c,$@)
test-libslang.bin:
$(BUILD) -I/usr/include/slang -lslang
+test-libcrypto.bin:
+ $(BUILD) -lcrypto
+
test-gtk2.bin:
$(BUILD) $(shell $(PKG_CONFIG) --libs --cflags gtk+-2.0 2>/dev/null)
diff --git a/tools/build/feature/test-libcrypto.c b/tools/build/feature/test-libcrypto.c
new file mode 100644
index 0000000..8580c40
--- /dev/null
+++ b/tools/build/feature/test-libcrypto.c
@@ -0,0 +1,9 @@
+#include <openssl/md5.h>
+
+int main(void)
+{
+ MD5_CTX context;
+
+ MD5_Init(&context);
+ return 0;
+}
diff --git a/tools/perf/Documentation/perf-inject.txt b/tools/perf/Documentation/perf-inject.txt
index dc7442c..bb33fe2 100644
--- a/tools/perf/Documentation/perf-inject.txt
+++ b/tools/perf/Documentation/perf-inject.txt
@@ -44,6 +44,12 @@ OPTIONS
--kallsyms=<file>::
kallsyms pathname
+-j::
+--jit::
+ Process jitdump files by injecting the mmap records corresponding to jitted
+ functions. This option also generates the ELF images for each jitted function
+ found in the jitdumps files captured in the input perf.data file. Use this option
+ if you are monitoring environment using JIT runtimes, such as Java, DART or V8.
SEE ALSO
--------
linkperf:perf-record[1], linkperf:perf-report[1], linkperf:perf-archive[1]
diff --git a/tools/perf/builtin-inject.c b/tools/perf/builtin-inject.c
index ea46df25..af25d1a 100644
--- a/tools/perf/builtin-inject.c
+++ b/tools/perf/builtin-inject.c
@@ -16,6 +16,7 @@
#include "util/debug.h"
#include "util/build-id.h"
#include "util/data.h"
+#include "util/jit.h"
#include "util/parse-options.h"
@@ -26,6 +27,7 @@ struct perf_inject {
struct perf_session *session;
bool build_ids;
bool sched_stat;
+ bool jit_mode;
const char *input_name;
struct perf_data_file output;
u64 bytes_written;
@@ -121,12 +123,19 @@ static int perf_event__repipe_mmap(struct perf_tool *tool,
struct perf_sample *sample,
struct machine *machine)
{
- int err;
-
- err = perf_event__process_mmap(tool, event, sample, machine);
- perf_event__repipe(tool, event, sample, machine);
-
- return err;
+ struct perf_inject *inject = container_of(tool, struct perf_inject, tool);
+ u64 n = 0;
+
+ if (inject->jit_mode) {
+ /*
+ * if jit marker, then inject jit mmaps and generate ELF images
+ */
+ if (!jit_process(&inject->tool, &inject->output, machine, event->mmap.filename, sample->pid, &n)) {
+ inject->bytes_written += n;
+ return 0;
+ }
+ }
+ return perf_event__repipe(tool, event, sample, machine);
}
static int perf_event__repipe_mmap2(struct perf_tool *tool,
@@ -134,12 +143,19 @@ static int perf_event__repipe_mmap2(struct perf_tool *tool,
struct perf_sample *sample,
struct machine *machine)
{
- int err;
-
- err = perf_event__process_mmap2(tool, event, sample, machine);
- perf_event__repipe(tool, event, sample, machine);
-
- return err;
+ struct perf_inject *inject = container_of(tool, struct perf_inject, tool);
+ u64 n = 0;
+
+ if (inject->jit_mode) {
+ /*
+ * if jit marker, then inject jit mmaps and generate ELF images
+ */
+ if (!jit_process(&inject->tool, &inject->output, machine, event->mmap2.filename, sample->pid, &n)) {
+ inject->bytes_written += n;
+ return 0;
+ }
+ }
+ return perf_event__repipe(tool, event, sample, machine);
}
static int perf_event__repipe_fork(struct perf_tool *tool,
@@ -341,7 +357,6 @@ static int perf_evsel__check_stype(struct perf_evsel *evsel,
name, sample_msg);
return -EINVAL;
}
-
return 0;
}
@@ -439,6 +454,7 @@ int cmd_inject(int argc, const char **argv, const char *prefix __maybe_unused)
OPT_BOOLEAN('s', "sched-stat", &inject.sched_stat,
"Merge sched-stat and sched-switch for getting events "
"where and how long tasks slept"),
+ OPT_BOOLEAN('j', "jit", &inject.jit_mode, "merge jitdump files into perf.data file"),
OPT_INCR('v', "verbose", &verbose,
"be more verbose (show build ids, etc)"),
OPT_STRING(0, "kallsyms", &symbol_conf.kallsyms_name, "file",
@@ -470,6 +486,24 @@ int cmd_inject(int argc, const char **argv, const char *prefix __maybe_unused)
if (inject.session == NULL)
return -1;
+ if (inject.build_ids) {
+ /*
+ * to make sure the mmap records are ordered correctly
+ * and so that the correct especially due to jitted code
+ * mmaps. We cannot generate the buildid hit list and
+ * inject the jit mmaps at the same time for now.
+ */
+ inject.tool.ordered_events = true;
+ inject.tool.ordering_requires_timestamps = true;
+ }
+
+ if (inject.jit_mode) {
+ inject.tool.mmap2 = perf_event__repipe_mmap2;
+ inject.tool.mmap = perf_event__repipe_mmap;
+ inject.tool.ordered_events = true;
+ inject.tool.ordering_requires_timestamps = true;
+ }
+
if (symbol__init(&inject.session->header.env) < 0)
return -1;
diff --git a/tools/perf/config/Makefile b/tools/perf/config/Makefile
index cd121df..e3cf5e4 100644
--- a/tools/perf/config/Makefile
+++ b/tools/perf/config/Makefile
@@ -366,6 +366,17 @@ ifndef NO_LIBAUDIT
endif
endif
+ifndef NO_LIBCRYPTO
+ ifneq ($(feature-libcrypto), 1)
+ msg := $(warning No libcrypto.h found, disables jitted code injection, please install libssl-devel or libssl-dev);
+ NO_LIBCRYPTO := 1
+ else
+ CFLAGS += -DHAVE_LIBCRYPTO_SUPPORT
+ EXTLIBS += -lcrypto
+ $(call detected,CONFIG_CRYPTO)
+ endif
+endif
+
ifdef NO_NEWT
NO_SLANG=1
endif
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 9bff65e..8bc62b4 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -75,6 +75,8 @@ libperf-$(CONFIG_X86) += tsc.o
libperf-y += cloexec.o
libperf-y += thread-stack.o
libperf-y += demangle-java.o
+libperf-y += jitdump.o
+libperf-y += genelf.o
libperf-$(CONFIG_LIBELF) += symbol-elf.o
libperf-$(CONFIG_LIBELF) += probe-event.o
diff --git a/tools/perf/util/genelf.c b/tools/perf/util/genelf.c
new file mode 100644
index 0000000..a5beebe
--- /dev/null
+++ b/tools/perf/util/genelf.c
@@ -0,0 +1,479 @@
+/*
+ * genelf.c
+ * Copyright (C) 2014, Google, Inc
+ *
+ * Contributed by:
+ * Stephane Eranian <eranian@gmail.com>
+ *
+ * Released under the GPL v2. (and only v2, not any later version)
+ */
+
+#include <sys/types.h>
+#include <stdio.h>
+#include <getopt.h>
+#include <stddef.h>
+#include <libelf.h>
+#include <string.h>
+#include <stdlib.h>
+#include <fcntl.h>
+#include <err.h>
+
+#include "perf.h"
+#include "genelf.h"
+
+#define JVMTI
+
+#define BUILD_ID_URANDOM /* different uuid for each run */
+
+#ifdef HAVE_LIBCRYPTO
+
+#define BUILD_ID_MD5
+#undef BUILD_ID_SHA /* does not seem to work well when linked with Java */
+#undef BUILD_ID_URANDOM /* different uuid for each run */
+
+#ifdef BUILD_ID_SHA
+#include <openssl/sha.h>
+#endif
+
+#ifdef BUILD_ID_MD5
+#include <openssl/md5.h>
+#endif
+#endif
+
+#if defined(__arm__)
+#define GEN_ELF_ARCH EM_ARM
+#define GEN_ELF_ENDIAN ELFDATA2LSB
+#define GEN_ELF_CLASS ELFCLASS32
+#elif defined(__x86_64__)
+#define GEN_ELF_ARCH EM_X86_64
+#define GEN_ELF_ENDIAN ELFDATA2LSB
+#define GEN_ELF_CLASS ELFCLASS64
+#elif defined(__i386__)
+#define GEN_ELF_ARCH EM_386
+#define GEN_ELF_ENDIAN ELFDATA2LSB
+#define GEN_ELF_CLASS ELFCLASS32
+#elif defined(__ppcle__)
+#define GEN_ELF_ARCH EM_PPC
+#define GEN_ELF_ENDIAN ELFDATA2LSB
+#define GEN_ELF_CLASS ELFCLASS64
+#elif defined(__powerpc__)
+#define GEN_ELF_ARCH EM_PPC64
+#define GEN_ELF_ENDIAN ELFDATA2MSB
+#define GEN_ELF_CLASS ELFCLASS64
+#elif defined(__powerpcle__)
+#define GEN_ELF_ARCH EM_PPC64
+#define GEN_ELF_ENDIAN ELFDATA2LSB
+#define GEN_ELF_CLASS ELFCLASS64
+#else
+#error "unsupported architecture"
+#endif
+
+#if GEN_ELF_CLASS == ELFCLASS64
+#define elf_newehdr elf64_newehdr
+#define elf_getshdr elf64_getshdr
+#define Elf_Ehdr Elf64_Ehdr
+#define Elf_Shdr Elf64_Shdr
+#define Elf_Sym Elf64_Sym
+#define ELF_ST_TYPE(a) ELF64_ST_TYPE(a)
+#define ELF_ST_BIND(a) ELF64_ST_BIND(a)
+#define ELF_ST_VIS(a) ELF64_ST_VISIBILITY(a)
+#else
+#define elf_newehdr elf32_newehdr
+#define elf_getshdr elf32_getshdr
+#define Elf_Ehdr Elf32_Ehdr
+#define Elf_Shdr Elf32_Shdr
+#define Elf_Sym Elf32_Sym
+#define ELF_ST_TYPE(a) ELF32_ST_TYPE(a)
+#define ELF_ST_BIND(a) ELF32_ST_BIND(a)
+#define ELF_ST_VIS(a) ELF32_ST_VISIBILITY(a)
+#endif
+
+typedef struct {
+ unsigned int namesz; /* Size of entry's owner string */
+ unsigned int descsz; /* Size of the note descriptor */
+ unsigned int type; /* Interpretation of the descriptor */
+ char name[0]; /* Start of the name+desc data */
+} Elf_Note;
+
+struct options {
+ char *output;
+ int fd;
+};
+
+static char shd_string_table[] = {
+ 0,
+ '.', 't', 'e', 'x', 't', 0, /* 1 */
+ '.', 's', 'h', 's', 't', 'r', 't', 'a', 'b', 0, /* 7 */
+ '.', 's', 'y', 'm', 't', 'a', 'b', 0, /* 17 */
+ '.', 's', 't', 'r', 't', 'a', 'b', 0, /* 25 */
+ '.', 'n', 'o', 't', 'e', '.', 'g', 'n', 'u', '.', 'b', 'u', 'i', 'l', 'd', '-', 'i', 'd', 0, /* 33 */
+};
+
+
+static struct buildid_note {
+ Elf_Note desc; /* descsz: size of build-id, must be multiple of 4 */
+ char name[4]; /* GNU\0 */
+ char build_id[20];
+} bnote;
+
+static Elf_Sym symtab[]={
+ /* symbol 0 MUST be the undefined symbol */
+ { .st_name = 0, /* index in sym_string table */
+ .st_info = ELF_ST_TYPE(STT_NOTYPE),
+ .st_shndx = 0, /* for now */
+ .st_value = 0x0,
+ .st_other = ELF_ST_VIS(STV_DEFAULT),
+ .st_size = 0,
+ },
+ { .st_name = 1, /* index in sym_string table */
+ .st_info = ELF_ST_BIND(STB_LOCAL) | ELF_ST_TYPE(STT_FUNC),
+ .st_shndx = 1,
+ .st_value = 0, /* for now */
+ .st_other = ELF_ST_VIS(STV_DEFAULT),
+ .st_size = 0, /* for now */
+ }
+};
+
+#ifdef BUILD_ID_URANDOM
+static void
+gen_build_id(struct buildid_note *note,
+ unsigned long load_addr __maybe_unused,
+ const void *code __maybe_unused,
+ size_t csize __maybe_unused)
+{
+ int fd;
+ size_t sz = sizeof(note->build_id);
+ ssize_t sret;
+
+ fd = open("/dev/urandom", O_RDONLY);
+ if (fd == -1)
+ err(1, "cannot access /dev/urandom for builid");
+
+ sret = read(fd, note->build_id, sz);
+
+ close(fd);
+
+ if (sret != (ssize_t)sz)
+ memset(note->build_id, 0, sz);
+}
+#endif
+
+#ifdef BUILD_ID_SHA
+static void
+gen_build_id(struct buildid_note *note,
+ unsigned long load_addr __maybe_unused,
+ const void *code,
+ size_t csize)
+{
+ if (sizeof(note->build_id) < SHA_DIGEST_LENGTH)
+ errx(1, "build_id too small for SHA1");
+
+ SHA1(code, csize, (unsigned char *)note->build_id);
+}
+#endif
+
+#ifdef BUILD_ID_MD5
+static void
+gen_build_id(struct buildid_note *note, unsigned long load_addr, const void *code, size_t csize)
+{
+ MD5_CTX context;
+
+ if (sizeof(note->build_id) < 16)
+ errx(1, "build_id too small for MD5");
+
+ MD5_Init(&context);
+ MD5_Update(&context, &load_addr, sizeof(load_addr));
+ MD5_Update(&context, code, csize);
+ MD5_Final((unsigned char *)note->build_id, &context);
+}
+#endif
+
+/*
+ * fd: file descriptor open for writing for the output file
+ * load_addr: code load address (could be zero, just used for buildid)
+ * sym: function name (for native code - used as the symbol)
+ * code: the native code
+ * csize: the code size in bytes
+ */
+int
+jit_write_elf(int fd, unsigned long load_addr, const char *sym, const void *code, int csize)
+{
+ Elf *e;
+ Elf_Data *d;
+ Elf_Scn *scn;
+ Elf_Ehdr *ehdr;
+ Elf_Shdr *shdr;
+ char *strsym = NULL;
+ int symlen;
+ int retval = -1;
+
+ if (elf_version(EV_CURRENT) == EV_NONE) {
+ warnx("ELF initialization failed");
+ return -1;
+ }
+
+ e = elf_begin(fd, ELF_C_WRITE, NULL);
+ if (!e) {
+ warnx("elf_begin failed");
+ goto error;
+ }
+
+ /*
+ * setup ELF header
+ */
+ ehdr = elf_newehdr(e);
+ if (!ehdr) {
+ warnx("cannot get ehdr");
+ goto error;
+ }
+
+ ehdr->e_ident[EI_DATA] = GEN_ELF_ENDIAN;
+ ehdr->e_ident[EI_CLASS] = GEN_ELF_CLASS;
+ ehdr->e_machine = GEN_ELF_ARCH;
+ ehdr->e_type = ET_DYN;
+ ehdr->e_entry = 0x0;
+ ehdr->e_version = EV_CURRENT;
+ ehdr->e_shstrndx= 2; /* shdr index for section name */
+
+ /*
+ * setup text section
+ */
+ scn = elf_newscn(e);
+ if (!scn) {
+ warnx("cannot create section");
+ goto error;
+ }
+
+ d = elf_newdata(scn);
+ if (!d) {
+ warnx("cannot get new data");
+ goto error;
+ }
+
+ d->d_align = 16;
+ d->d_off = 0LL;
+ d->d_buf = (void *)code;
+ d->d_type = ELF_T_BYTE;
+ d->d_size = csize;
+ d->d_version = EV_CURRENT;
+
+ shdr = elf_getshdr(scn);
+ if (!shdr) {
+ warnx("cannot get section header");
+ goto error;
+ }
+
+ shdr->sh_name = 1;
+ shdr->sh_type = SHT_PROGBITS;
+ shdr->sh_addr = 0; /* must be zero or == sh_offset -> dynamic object */
+ shdr->sh_flags = SHF_EXECINSTR | SHF_ALLOC;
+ shdr->sh_entsize = 0;
+
+ /*
+ * setup section headers string table
+ */
+ scn = elf_newscn(e);
+ if (!scn) {
+ warnx("cannot create section");
+ goto error;
+ }
+
+ d = elf_newdata(scn);
+ if (!d) {
+ warnx("cannot get new data");
+ goto error;
+ }
+
+ d->d_align = 1;
+ d->d_off = 0LL;
+ d->d_buf = shd_string_table;
+ d->d_type = ELF_T_BYTE;
+ d->d_size = sizeof(shd_string_table);
+ d->d_version = EV_CURRENT;
+
+ shdr = elf_getshdr(scn);
+ if (!shdr) {
+ warnx("cannot get section header");
+ goto error;
+ }
+
+ shdr->sh_name = 7; /* offset of '.shstrtab' in shd_string_table */
+ shdr->sh_type = SHT_STRTAB;
+ shdr->sh_flags = 0;
+ shdr->sh_entsize = 0;
+
+ /*
+ * setup symtab section
+ */
+ symtab[1].st_size = csize;
+
+ scn = elf_newscn(e);
+ if (!scn) {
+ warnx("cannot create section");
+ goto error;
+ }
+
+ d = elf_newdata(scn);
+ if (!d) {
+ warnx("cannot get new data");
+ goto error;
+ }
+
+ d->d_align = 8;
+ d->d_off = 0LL;
+ d->d_buf = symtab;
+ d->d_type = ELF_T_SYM;
+ d->d_size = sizeof(symtab);
+ d->d_version = EV_CURRENT;
+
+ shdr = elf_getshdr(scn);
+ if (!shdr) {
+ warnx("cannot get section header");
+ goto error;
+ }
+
+ shdr->sh_name = 17; /* offset of '.symtab' in shd_string_table */
+ shdr->sh_type = SHT_SYMTAB;
+ shdr->sh_flags = 0;
+ shdr->sh_entsize = sizeof(Elf_Sym);
+ shdr->sh_link = 4; /* index of .strtab section */
+
+ /*
+ * setup symbols string table
+ * 2 = 1 for 0 in 1st entry, 1 for the 0 at end of symbol for 2nd entry
+ */
+ symlen = 2 + strlen(sym);
+ strsym = calloc(1, symlen);
+ if (!strsym) {
+ warnx("cannot allocate strsym");
+ goto error;
+ }
+ strcpy(strsym + 1, sym);
+
+ scn = elf_newscn(e);
+ if (!scn) {
+ warnx("cannot create section");
+ goto error;
+ }
+
+ d = elf_newdata(scn);
+ if (!d) {
+ warnx("cannot get new data");
+ goto error;
+ }
+
+ d->d_align = 1;
+ d->d_off = 0LL;
+ d->d_buf = strsym;
+ d->d_type = ELF_T_BYTE;
+ d->d_size = symlen;
+ d->d_version = EV_CURRENT;
+
+ shdr = elf_getshdr(scn);
+ if (!shdr) {
+ warnx("cannot get section header");
+ goto error;
+ }
+
+ shdr->sh_name = 25; /* offset in shd_string_table */
+ shdr->sh_type = SHT_STRTAB;
+ shdr->sh_flags = 0;
+ shdr->sh_entsize = 0;
+
+ /*
+ * setup build-id section
+ */
+ scn = elf_newscn(e);
+ if (!scn) {
+ warnx("cannot create section");
+ goto error;
+ }
+
+ d = elf_newdata(scn);
+ if (!d) {
+ warnx("cannot get new data");
+ goto error;
+ }
+
+ /*
+ * build-id generation
+ */
+ gen_build_id(&bnote, load_addr, code, csize);
+ bnote.desc.namesz = sizeof(bnote.name); /* must include 0 termination */
+ bnote.desc.descsz = sizeof(bnote.build_id);
+ bnote.desc.type = NT_GNU_BUILD_ID;
+ strcpy(bnote.name, "GNU");
+
+ d->d_align = 4;
+ d->d_off = 0LL;
+ d->d_buf = &bnote;
+ d->d_type = ELF_T_BYTE;
+ d->d_size = sizeof(bnote);
+ d->d_version = EV_CURRENT;
+
+ shdr = elf_getshdr(scn);
+ if (!shdr) {
+ warnx("cannot get section header");
+ goto error;
+ }
+
+ shdr->sh_name = 33; /* offset in shd_string_table */
+ shdr->sh_type = SHT_NOTE;
+ shdr->sh_addr = 0x0;
+ shdr->sh_flags = SHF_ALLOC;
+ shdr->sh_size = sizeof(bnote);
+ shdr->sh_entsize = 0;
+
+ if (elf_update(e, ELF_C_WRITE) < 0) {
+ warnx("elf_update 4 failed");
+ goto error;
+ }
+ (void)elf_end(e);
+
+ retval = 0;
+error:
+ free(strsym);
+
+ return retval;
+}
+
+#ifndef JVMTI
+
+static unsigned char x86_code[] = {
+ 0xBB, 0x2A, 0x00, 0x00, 0x00, /* movl $42, %ebx */
+ 0xB8, 0x01, 0x00, 0x00, 0x00, /* movl $1, %eax */
+ 0xCD, 0x80 /* int $0x80 */
+};
+
+static struct options options;
+
+int main(int argc, char **argv)
+{
+ int c, fd, ret;
+
+ while ((c = getopt(argc, argv, "o:h")) != -1) {
+ switch (c) {
+ case 'o':
+ options.output = optarg;
+ break;
+ case 'h':
+ printf("Usage: genelf -o output_file [-h]\n");
+ return 0;
+ default:
+ errx(1, "unknown option");
+ }
+ }
+
+ fd = open(options.output, O_CREAT|O_TRUNC|O_RDWR, 0666);
+ if (fd == -1)
+ err(1, "cannot create file %s", options.output);
+
+ ret = jit_write_elf(fd, "main", x86_code, sizeof(x86_code));
+ close(fd);
+
+ if (ret != 0)
+ unlink(options.output);
+
+ return ret;
+}
+#endif
diff --git a/tools/perf/util/genelf.h b/tools/perf/util/genelf.h
new file mode 100644
index 0000000..11307a2
--- /dev/null
+++ b/tools/perf/util/genelf.h
@@ -0,0 +1,6 @@
+#ifndef __GENELF_H__
+#define __GENELF_H__
+
+extern int jit_write_elf(int fd, unsigned long code_addr, const char *sym, const void *code, int csize);
+
+#endif
diff --git a/tools/perf/util/jit.h b/tools/perf/util/jit.h
new file mode 100644
index 0000000..2a812c0
--- /dev/null
+++ b/tools/perf/util/jit.h
@@ -0,0 +1,15 @@
+#ifndef __JIT_H__
+#define __JIT_H__
+
+#include <data.h>
+
+extern int jit_process(struct perf_tool *tool,
+ struct perf_data_file *output,
+ struct machine *machine,
+ char *filename,
+ pid_t pid,
+ u64 *nbytes);
+
+extern int jit_inject_record(const char *filename);
+
+#endif /* __JIT_H__ */
diff --git a/tools/perf/util/jitdump.c b/tools/perf/util/jitdump.c
new file mode 100644
index 0000000..9c5fafe
--- /dev/null
+++ b/tools/perf/util/jitdump.c
@@ -0,0 +1,588 @@
+#include <sys/types.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <inttypes.h>
+#include <byteswap.h>
+#include <sys/stat.h>
+#include <sys/mman.h>
+
+#include "util.h"
+#include "event.h"
+#include "debug.h"
+#include "symbol.h"
+#include "strlist.h"
+#include <elf.h>
+
+#include "session.h"
+#include "jit.h"
+#include "jitdump.h"
+#include "genelf.h"
+#include "../builtin.h"
+
+struct jit_buf_desc {
+ struct perf_data_file *output;
+ struct perf_tool *tool;
+ struct machine *machine;
+ union jr_entry *entry;
+ void *buf;
+ size_t bufsize;
+ FILE *in;
+ int needs_bswap; /* handles cross-endianess */
+ uint32_t code_load_count;
+ u64 bytes_written;
+ struct rb_root code_root;
+ char dir[PATH_MAX];
+};
+
+struct debug_line_info {
+ unsigned long vma;
+ unsigned int lineno;
+ /* The filename format is unspecified, absolute path, relative etc. */
+ char const filename[0];
+};
+
+struct jit_tool {
+ struct perf_tool tool;
+ struct perf_data_file output;
+ struct perf_data_file input;
+ u64 bytes_written;
+};
+
+#define hmax(a, b) ((a) > (b) ? (a) : (b))
+#define get_jit_tool(t) (container_of(tool, struct jit_tool, tool))
+
+static int
+jit_emit_elf(char *filename,
+ const char *sym,
+ unsigned long code,
+ int csize)
+{
+ int ret, fd;
+ unsigned long addr = (unsigned long)code;
+
+ if (verbose > 0)
+ fprintf(stderr, "write ELF image %s\n", filename);
+
+ fd = open(filename, O_CREAT|O_TRUNC|O_WRONLY, 0644);
+ if (fd == -1) {
+ pr_warning("cannot create jit ELF %s: %s\n", filename, strerror(errno));
+ return -1;
+ }
+
+ ret = jit_write_elf(fd, addr, sym, (const void *)code, csize);
+
+ close(fd);
+
+ if (ret)
+ unlink(filename);
+
+ return ret;
+}
+
+static void
+jit_close(struct jit_buf_desc *jd)
+{
+ if (!(jd && jd->in))
+ return;
+ fclose(jd->in);
+ jd->in = NULL;
+}
+
+static int
+jit_open(struct jit_buf_desc *jd, const char *name)
+{
+ struct jitheader header;
+ struct jr_prefix *prefix;
+ ssize_t bs, bsz = 0;
+ void *n, *buf = NULL;
+ int ret, retval = -1;
+
+ jd->in = fopen(name, "r");
+ if (!jd->in)
+ return -1;
+
+ bsz = hmax(sizeof(header), sizeof(*prefix));
+
+ buf = malloc(bsz);
+ if (!buf)
+ goto error;
+
+ ret = fread(buf, sizeof(header), 1, jd->in);
+ if (ret != 1)
+ goto error;
+
+ memcpy(&header, buf, sizeof(header));
+
+ if (header.magic != JITHEADER_MAGIC) {
+ if (header.magic != JITHEADER_MAGIC_SW)
+ goto error;
+ jd->needs_bswap = 1;
+ }
+
+ if (jd->needs_bswap) {
+ header.version = bswap_32(header.version);
+ header.total_size = bswap_32(header.total_size);
+ header.pid = bswap_32(header.pid);
+ header.elf_mach = bswap_32(header.elf_mach);
+ header.timestamp = bswap_64(header.timestamp);
+ }
+
+ if (verbose > 2)
+ pr_debug("version=%u\nhdr.size=%u\nts=0x%llx\npid=%d\nelf_mach=%d\n",
+ header.version,
+ header.total_size,
+ (unsigned long long)header.timestamp,
+ header.pid,
+ header.elf_mach);
+
+ bs = header.total_size - sizeof(header);
+
+ if (bs > bsz) {
+ n = realloc(buf, bs);
+ if (!n)
+ goto error;
+ bsz = bs;
+ buf = n;
+ /* read extra we do not know about */
+ ret = fread(buf, bs - bsz, 1, jd->in);
+ if (ret != 1)
+ goto error;
+ }
+ /*
+ * keep dirname for generating files and mmap records
+ */
+ strcpy(jd->dir, name);
+ dirname(jd->dir);
+
+ return 0;
+error:
+ fclose(jd->in);
+ return retval;
+}
+
+static union jr_entry *
+jit_get_next_entry(struct jit_buf_desc *jd)
+{
+ struct jr_prefix *prefix;
+ union jr_entry *jr;
+ void *addr;
+ size_t bs, size;
+ int id, ret;
+
+ if (!(jd && jd->in))
+ return NULL;
+
+ if (jd->buf == NULL) {
+ size_t sz = getpagesize();
+ if (sz < sizeof(*prefix))
+ sz = sizeof(*prefix);
+
+ jd->buf = malloc(sz);
+ if (jd->buf == NULL)
+ return NULL;
+
+ jd->bufsize = sz;
+ }
+
+ prefix = jd->buf;
+
+ ret = fread(prefix, sizeof(*prefix), 1, jd->in);
+ if (ret != 1)
+ return NULL;
+
+ if (jd->needs_bswap) {
+ prefix->id = bswap_32(prefix->id);
+ prefix->total_size = bswap_32(prefix->total_size);
+ prefix->timestamp = bswap_64(prefix->timestamp);
+ }
+ id = prefix->id;
+ size = prefix->total_size;
+
+ bs = (size_t)size;
+ if (bs < sizeof(*prefix))
+ return NULL;
+
+ if (id >= JIT_CODE_MAX) {
+ pr_warning("next_entry: unknown prefix %d, skipping\n", id);
+ return NULL;
+ }
+ if (bs > jd->bufsize) {
+ void *n;
+ n = realloc(jd->buf, bs);
+ if (!n)
+ return NULL;
+ jd->buf = n;
+ jd->bufsize = bs;
+ }
+
+ addr = ((void *)jd->buf) + sizeof(*prefix);
+
+ ret = fread(addr, bs - sizeof(*prefix), 1, jd->in);
+ if (ret != 1)
+ return NULL;
+
+ jr = (union jr_entry *)jd->buf;
+
+ switch(id) {
+ case JIT_CODE_DEBUG_INFO:
+ case JIT_CODE_CLOSE:
+ break;
+ case JIT_CODE_LOAD:
+ if (jd->needs_bswap) {
+ jr->load.pid = bswap_32(jr->load.pid);
+ jr->load.tid = bswap_32(jr->load.tid);
+ jr->load.vma = bswap_64(jr->load.vma);
+ jr->load.code_addr = bswap_64(jr->load.code_addr);
+ jr->load.code_size = bswap_64(jr->load.code_size);
+ jr->load.code_index= bswap_64(jr->load.code_index);
+ }
+ jd->code_load_count++;
+ break;
+ case JIT_CODE_MOVE:
+ if (jd->needs_bswap) {
+ jr->move.pid = bswap_32(jr->move.pid);
+ jr->move.tid = bswap_32(jr->move.tid);
+ jr->move.vma = bswap_64(jr->move.vma);
+ jr->move.old_code_addr = bswap_64(jr->move.old_code_addr);
+ jr->move.new_code_addr = bswap_64(jr->move.new_code_addr);
+ jr->move.code_size = bswap_64(jr->move.code_size);
+ jr->move.code_index = bswap_64(jr->move.code_index);
+ }
+ break;
+ case JIT_CODE_MAX:
+ default:
+ return NULL;
+ }
+ return jr;
+}
+
+static int
+jit_inject_event(struct jit_buf_desc *jd, union perf_event *event)
+{
+ ssize_t size;
+
+ size = perf_data_file__write(jd->output, event, event->header.size);
+ if (size < 0)
+ return -1;
+
+ jd->bytes_written += size;
+ return 0;
+}
+
+static int jit_repipe_code_load(struct jit_buf_desc *jd, union jr_entry *jr)
+{
+ struct perf_sample sample;
+ union perf_event *event;
+ unsigned long code, addr;
+ char *filename;
+ struct stat st;
+ size_t size;
+ u16 idr_size;
+ const char *sym;
+ uint32_t count;
+ int ret, csize;
+ pid_t pid, tid;
+ struct {
+ u32 pid, tid;
+ u64 time;
+ } *id;
+
+ pid = jr->load.pid;
+ tid = jr->load.tid;
+ csize = jr->load.code_size;
+ addr = jr->load.code_addr;
+ sym = (void *)((unsigned long)jr + sizeof(jr->load));
+ code = (unsigned long)jr + jr->load.p.total_size - csize;
+ count = jr->load.code_index;
+ idr_size = jd->machine->id_hdr_size;
+ /*
+ * +16 to account for sample_id_all (hack)
+ */
+ event = calloc(1, sizeof(*event) + 16);
+ if (!event)
+ return -1;
+
+ filename = event->mmap2.filename;
+ size = snprintf(filename, PATH_MAX, "%s/jitted-%d-%u",
+ jd->dir,
+ pid,
+ count);
+
+ size++; /* for \0 */
+
+ size = PERF_ALIGN(size, sizeof(u64));
+
+ ret = jit_emit_elf(filename, sym, code, csize);
+ if (ret) {
+ free(event);
+ return -1;
+ }
+
+ if (stat(filename, &st))
+ memset(&st, 0, sizeof(stat));
+
+ event->mmap2.header.type = PERF_RECORD_MMAP2;
+ event->mmap2.header.misc = PERF_RECORD_MISC_USER;
+ event->mmap2.header.size = (sizeof(event->mmap2) -
+ (sizeof(event->mmap2.filename) - size) + idr_size);
+
+ event->mmap2.pgoff = 0;
+ event->mmap2.start = addr;
+ event->mmap2.len = csize;
+ event->mmap2.pid = pid;
+ event->mmap2.tid = tid;
+ event->mmap2.ino = st.st_ino;
+ event->mmap2.maj = major(st.st_dev);
+ event->mmap2.min = minor(st.st_dev);
+ event->mmap2.prot = st.st_mode;
+ event->mmap2.flags = MAP_SHARED;
+ event->mmap2.ino_generation = 1;
+
+ id = (void *)((unsigned long)event + event->mmap.header.size - idr_size);
+ id->pid = pid;
+ id->tid = tid;
+ id->time = jr->load.p.timestamp;
+
+ /*
+ * create pseudo sample to induce dso hit increment
+ * use first address as sample address
+ */
+ memset(&sample, 0, sizeof(sample));
+ sample.pid = pid;
+ sample.tid = tid;
+ sample.time = id->time;
+ sample.ip = addr;
+
+ ret = perf_event__process_mmap2(jd->tool, event, &sample, jd->machine);
+ if (ret)
+ return ret;
+
+ ret = jit_inject_event(jd, event);
+ /*
+ * mark dso as use to generate buildid in the header
+ */
+ if (!ret)
+ build_id__mark_dso_hit(jd->tool, event, &sample, NULL, jd->machine);
+
+ return ret;
+}
+
+static int jit_repipe_code_move(struct jit_buf_desc *jd, union jr_entry *jr)
+{
+ struct perf_sample sample;
+ union perf_event *event;
+ char *filename;
+ size_t size;
+ struct stat st;
+ u16 idr_size;
+ int ret;
+ pid_t pid, tid;
+ struct {
+ u32 pid, tid;
+ u64 time;
+ } *id;
+
+ pid = jr->move.pid;
+ tid = jr->move.tid;
+ idr_size = jd->machine->id_hdr_size;
+
+ /*
+ * +16 to account for sample_id_all (hack)
+ */
+ event = calloc(1, sizeof(*event) + 16);
+ if (!event)
+ return -1;
+
+ filename = event->mmap2.filename;
+ size = snprintf(filename, PATH_MAX, "%s/jitted-%d-%"PRIu64,
+ jd->dir,
+ pid,
+ jr->move.code_index);
+
+ size++; /* for \0 */
+
+ if (stat(filename, &st))
+ memset(&st, 0, sizeof(stat));
+
+ size = PERF_ALIGN(size, sizeof(u64));
+
+ event->mmap2.header.type = PERF_RECORD_MMAP2;
+ event->mmap2.header.misc = PERF_RECORD_MISC_USER;
+ event->mmap2.header.size = (sizeof(event->mmap2) -
+ (sizeof(event->mmap2.filename) - size) + idr_size);
+ event->mmap2.pgoff = 0;
+ event->mmap2.start = jr->move.new_code_addr;
+ event->mmap2.len = jr->move.code_size;
+ event->mmap2.pid = pid;
+ event->mmap2.tid = tid;
+ event->mmap2.ino = st.st_ino;
+ event->mmap2.maj = major(st.st_dev);
+ event->mmap2.min = minor(st.st_dev);
+ event->mmap2.prot = st.st_mode;
+ event->mmap2.flags = MAP_SHARED;
+ event->mmap2.ino_generation = 1;
+
+ id = (void *)((unsigned long)event + event->mmap.header.size - idr_size);
+ id->pid = pid;
+ id->tid = tid;
+ id->time = jr->move.p.timestamp;
+
+ /*
+ * create pseudo sample to induce dso hit increment
+ * use first address as sample address
+ */
+ memset(&sample, 0, sizeof(sample));
+ sample.pid = pid;
+ sample.tid = tid;
+ sample.time = id->time;
+ sample.ip = jr->move.new_code_addr;
+
+ ret = perf_event__process_mmap2(jd->tool, event, &sample, jd->machine);
+ if (ret)
+ return ret;
+
+ ret = jit_inject_event(jd, event);
+ if (!ret)
+ build_id__mark_dso_hit(jd->tool, event, &sample, NULL, jd->machine);
+
+ return ret;
+}
+
+static int
+jit_process_dump(struct jit_buf_desc *jd)
+{
+ union jr_entry *jr;
+ int ret;
+
+ while ((jr = jit_get_next_entry(jd))) {
+ switch(jr->prefix.id) {
+ case JIT_CODE_LOAD:
+ ret = jit_repipe_code_load(jd, jr);
+ break;
+ case JIT_CODE_MOVE:
+ ret = jit_repipe_code_move(jd, jr);
+ break;
+ default:
+ ret = 0;
+ continue;
+ }
+ }
+ return ret;
+}
+
+static int
+jit_inject(struct jit_buf_desc *jd, char *path)
+{
+ int ret;
+
+ if (verbose > 0)
+ fprintf(stderr, "injecting: %s\n", path);
+
+ ret = jit_open(jd, path);
+ if (ret)
+ return -1;
+
+ ret = jit_process_dump(jd);
+
+ jit_close(jd);
+
+ if (verbose > 0)
+ fprintf(stderr, "injected: %s (%d)\n", path, ret);
+
+ return 0;
+}
+
+/*
+ * File must be with pattern .../jit-XXXX.dump
+ * where XXXX is the PID of the process which did the mmap()
+ * as captured in the RECORD_MMAP record
+ */
+static int
+jit_detect(char *mmap_name, pid_t pid)
+ {
+ char *p;
+ char *end = NULL;
+ pid_t pid2;
+
+ if (verbose > 2)
+ fprintf(stderr, "jit marker trying : %s\n", mmap_name);
+ /*
+ * get file name
+ */
+ p = strrchr(mmap_name, '/');
+ if (!p)
+ return -1;
+
+ /*
+ * match prefix
+ */
+ if (strncmp(p, "/jit-", 5))
+ return -1;
+
+ /*
+ * skip prefix
+ */
+ p += 5;
+
+ /*
+ * must be followed by a pid
+ */
+ if (!isdigit(*p))
+ return -1;
+
+ pid2 = (int)strtol(p, &end, 10);
+ if (!end)
+ return -1;
+
+ /*
+ * pid does not match mmap pid
+ * pid==0 in system-wide mode (synthesized)
+ */
+ if (pid && pid2 != pid)
+ return -1;
+ /*
+ * validate suffix
+ */
+ if (strcmp(end, ".dump"))
+ return -1;
+
+ if (verbose > 0)
+ fprintf(stderr, "jit marker found: %s\n", mmap_name);
+
+ return 0;
+}
+
+int
+jit_process(struct perf_tool *tool,
+ struct perf_data_file *output,
+ struct machine *machine,
+ char *filename,
+ pid_t pid,
+ u64 *nbytes)
+{
+ struct jit_buf_desc jd;
+ int ret;
+
+ memset(&jd, 0, sizeof(jd));
+
+ jd.tool = tool;
+ jd.output = output;
+ jd.machine = machine;
+
+ *nbytes = 0;
+
+ /*
+ * detect marker mmap (i.e., the jitdump mmap)
+ */
+ if (jit_detect(filename, pid))
+ return -1;
+
+ ret = jit_inject(&jd, filename);
+ if (!ret)
+ *nbytes = jd.bytes_written;
+
+ return ret;
+}
diff --git a/tools/perf/util/jitdump.h b/tools/perf/util/jitdump.h
new file mode 100644
index 0000000..120bdcf
--- /dev/null
+++ b/tools/perf/util/jitdump.h
@@ -0,0 +1,92 @@
+/*
+ * jitdump.h: jitted code info encapsulation file format
+ *
+ * Adapted from OProfile GPLv2 support jidump.h:
+ * Copyright 2007 OProfile authors
+ * Jens Wilke
+ * Daniel Hansel
+ * Copyright IBM Corporation 2007
+ */
+#ifndef JITDUMP_H
+#define JITDUMP_H
+
+#include <sys/time.h>
+#include <time.h>
+#include <stdint.h>
+
+/* JiTD */
+#define JITHEADER_MAGIC 0x4A695444
+#define JITHEADER_MAGIC_SW 0x4454694A
+
+#define PADDING_8ALIGNED(x) ((((x) + 7) & 7) ^ 7)
+
+#define JITHEADER_VERSION 1
+
+struct jitheader {
+ uint32_t magic; /* characters "jItD" */
+ uint32_t version; /* header version */
+ uint32_t total_size; /* total size of header */
+ uint32_t elf_mach; /* elf mach target */
+ uint32_t pad1; /* reserved */
+ uint32_t pid; /* JIT process id */
+ uint64_t timestamp; /* timestamp */
+};
+
+enum jit_record_type {
+ JIT_CODE_LOAD = 0,
+ JIT_CODE_MOVE = 1,
+ JIT_CODE_DEBUG_INFO = 2,
+ JIT_CODE_CLOSE = 3,
+
+ JIT_CODE_MAX,
+};
+
+/* record prefix (mandatory in each record) */
+struct jr_prefix {
+ uint32_t id;
+ uint32_t total_size;
+ uint64_t timestamp;
+};
+
+struct jr_code_load {
+ struct jr_prefix p;
+
+ uint32_t pid;
+ uint32_t tid;
+ uint64_t vma;
+ uint64_t code_addr;
+ uint64_t code_size;
+ uint64_t code_index;
+};
+
+struct jr_code_close {
+ struct jr_prefix p;
+};
+
+struct jr_code_move {
+ struct jr_prefix p;
+
+ uint32_t pid;
+ uint32_t tid;
+ uint64_t vma;
+ uint64_t old_code_addr;
+ uint64_t new_code_addr;
+ uint64_t code_size;
+ uint64_t code_index;
+};
+
+struct jr_code_debug_info {
+ struct jr_prefix p;
+
+ uint64_t code_addr;
+ uint64_t nr_entry;
+};
+
+union jr_entry {
+ struct jr_code_debug_info info;
+ struct jr_code_close close;
+ struct jr_code_load load;
+ struct jr_code_move move;
+ struct jr_prefix prefix;
+};
+#endif /* !JITDUMP_H */
--
1.9.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH v6 4/4] perf tools: add JVMTI agent library
2015-03-30 22:19 [PATCH v6 0/4] perf: add support for profiling jitted code Stephane Eranian
` (2 preceding siblings ...)
2015-03-30 22:19 ` [PATCH v6 3/4] perf inject: add jitdump mmap injection support Stephane Eranian
@ 2015-03-30 22:19 ` Stephane Eranian
2015-03-31 7:33 ` [PATCH v6 0/4] perf: add support for profiling jitted code Brendan Gregg
4 siblings, 0 replies; 18+ messages in thread
From: Stephane Eranian @ 2015-03-30 22:19 UTC (permalink / raw)
To: linux-kernel
Cc: acme, peterz, mingo, ak, jolsa, namhyung, cel, sukadev, sonnyrao,
johnmccutchan, dsahern, adrian.hunter, pawel.moll
This is a standalone JVMTI library to help profile Java jitted
code with perf record/perf report. The library is not installed
or compiled automatically by perf Makefile. It is not used
directly by perf. It is arch agnostic and has been tested on
X86 and ARM. It needs to be used with a Java runtime, such
as OpenJDK, as follows:
$ java -agentpath:libjvmti.so .......
When used this way, java will generate a jitdump binary file in
$HOME/.debug/java/jit/java-jit-*
This binary dump file contains information to help symbolize and
annotate jitted code.
The jitdump information must be injected into the perf.data file
using:
$ perf inject --jit -i perf.data -o perf.data.jitted
This injects the MMAP records to cover the jitted code and also generates
one ELF image for each jitted function. The ELF images are created in the
same subdir as the jitdump file. The MMAP records point there too.
Then, to visualize the function or asm profile, simply use the regular
perf commands:
$ perf report -i perf.data.jitted
or
$ perf annotate -i perf.data.jitted
JVMTI agent code adapted from the OProfile's opagent code.
This version of the JVMTI agent is using the CLOCK_MONOTIC
as the time source to timestamp jit samples. To correlate
with perf_events samples, it needs to run on kernel 4.0.0-rc5+
or later with the following commit from Peter Zijlstra:
34f4392 perf: Add per event clockid support
With this patch recording jitted code is done as follows:
$ perf record -k mono -- java -agentpath:libjvmti.so .......
Signed-off-by: Stephane Eranian <eranian@google.com>
---
tools/perf/jvmti/Makefile | 70 ++++++++
tools/perf/jvmti/jvmti_agent.c | 367 +++++++++++++++++++++++++++++++++++++++++
tools/perf/jvmti/jvmti_agent.h | 23 +++
tools/perf/jvmti/libjvmti.c | 149 +++++++++++++++++
4 files changed, 609 insertions(+)
create mode 100644 tools/perf/jvmti/Makefile
create mode 100644 tools/perf/jvmti/jvmti_agent.c
create mode 100644 tools/perf/jvmti/jvmti_agent.h
create mode 100644 tools/perf/jvmti/libjvmti.c
diff --git a/tools/perf/jvmti/Makefile b/tools/perf/jvmti/Makefile
new file mode 100644
index 0000000..22efd13
--- /dev/null
+++ b/tools/perf/jvmti/Makefile
@@ -0,0 +1,70 @@
+ARCH=$(shell uname -m)
+
+ifeq ($(ARCH), x86_64)
+JARCH=amd64
+endif
+ifeq ($(ARCH), armv7l)
+JARCH=armhf
+endif
+ifeq ($(ARCH), armv6l)
+JARCH=armhf
+endif
+ifeq ($(ARCH), ppc64)
+JARCH=powerpc
+endif
+ifeq ($(ARCH), ppc64le)
+JARCH=powerpc
+endif
+
+DESTDIR=/usr/local
+
+VERSION=1
+REVISION=0
+AGE=0
+
+LN=ln -sf
+RM=rm
+
+SLIBJVMTI=libjvmti.so.$(VERSION).$(REVISION).$(AGE)
+VLIBJVMTI=libjvmti.so.$(VERSION)
+SLDFLAGS=-shared -Wl,-soname -Wl,$(VLIBJVMTI)
+SOLIBEXT=so
+
+JDIR=$(shell /usr/sbin/update-java-alternatives -l | head -1 | cut -d ' ' -f 3)
+# -lrt required in 32-bit mode for clock_gettime()
+LIBS=-lelf -lrt
+INCDIR=-I $(JDIR)/include -I $(JDIR)/include/linux
+
+TARGETS=$(SLIBJVMTI)
+
+SRCS=libjvmti.c jvmti_agent.c
+OBJS=$(SRCS:.c=.o)
+SOBJS=$(OBJS:.o=.lo)
+OPT=-O2 -g -Werror -Wall
+
+CFLAGS=$(INCDIR) $(OPT)
+
+all: $(TARGETS)
+
+.c.o:
+ $(CC) $(CFLAGS) -c $*.c
+.c.lo:
+ $(CC) -fPIC -DPIC $(CFLAGS) -c $*.c -o $*.lo
+
+$(OBJS) $(SOBJS): Makefile jvmti_agent.h ../util/jitdump.h
+
+$(SLIBJVMTI): $(SOBJS)
+ $(CC) $(CFLAGS) $(SLDFLAGS) -o $@ $(SOBJS) $(LIBS)
+ $(LN) $@ libjvmti.$(SOLIBEXT)
+
+clean:
+ $(RM) -f *.o *.so.* *.so *.lo
+
+install:
+ -mkdir -p $(DESTDIR)/lib
+ install -m 755 $(SLIBJVMTI) $(DESTDIR)/lib/
+ (cd $(DESTDIR)/lib; $(LN) $(SLIBJVMTI) $(VLIBJVMTI))
+ (cd $(DESTDIR)/lib; $(LN) $(SLIBJVMTI) libjvmti.$(SOLIBEXT))
+ ldconfig
+
+.SUFFIXES: .c .S .o .lo
diff --git a/tools/perf/jvmti/jvmti_agent.c b/tools/perf/jvmti/jvmti_agent.c
new file mode 100644
index 0000000..4e034cb
--- /dev/null
+++ b/tools/perf/jvmti/jvmti_agent.c
@@ -0,0 +1,367 @@
+/*
+ * jvmti_agent.c: JVMTI agent interface
+ *
+ * Adapted from the Oprofile code in opagent.c:
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ *
+ * Copyright 2007 OProfile authors
+ * Jens Wilke
+ * Daniel Hansel
+ * Copyright IBM Corporation 2007
+ */
+#include <sys/types.h>
+#include <sys/stat.h> /* for mkdir() */
+#include <stdio.h>
+#include <errno.h>
+#include <string.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <limits.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <time.h>
+#include <sys/mman.h>
+#include <syscall.h> /* for gettid() */
+#include <err.h>
+
+#include "jvmti_agent.h"
+#include "../util/jitdump.h"
+
+#define JIT_LANG "java"
+
+static char jit_path[PATH_MAX];
+static void *marker_addr;
+
+/*
+ * padding buffer
+ */
+static const char pad_bytes[7];
+
+static inline pid_t gettid(void)
+{
+ return (pid_t)syscall(__NR_gettid);
+}
+
+static int get_e_machine(struct jitheader *hdr)
+{
+ ssize_t sret;
+ char id[16];
+ int fd, ret = -1;
+ int m = -1;
+ struct {
+ uint16_t e_type;
+ uint16_t e_machine;
+ } info;
+
+ fd = open("/proc/self/exe", O_RDONLY);
+ if (fd == -1)
+ return -1;
+
+ sret = read(fd, id, sizeof(id));
+ if (sret != sizeof(id))
+ goto error;
+
+ /* check ELF signature */
+ if (id[0] != 0x7f || id[1] != 'E' || id[2] != 'L' || id[3] != 'F')
+ goto error;
+
+ sret = read(fd, &info, sizeof(info));
+ if (sret != sizeof(info))
+ goto error;
+
+ m = info.e_machine;
+ if (m < 0)
+ m = 0; /* ELF EM_NONE */
+
+ hdr->elf_mach = m;
+ ret = 0;
+error:
+ close(fd);
+ return ret;
+}
+
+#define NSEC_PER_SEC 1000000000
+static int perf_clk_id = CLOCK_MONOTONIC;
+
+static inline uint64_t
+timespec_to_ns(const struct timespec *ts)
+{
+ return ((uint64_t) ts->tv_sec * NSEC_PER_SEC) + ts->tv_nsec;
+}
+
+static inline uint64_t
+perf_get_timestamp(void)
+{
+ struct timespec ts;
+ int ret;
+
+ ret = clock_gettime(perf_clk_id, &ts);
+ if (ret)
+ return 0;
+
+ return timespec_to_ns(&ts);
+}
+
+static int
+debug_cache_init(void)
+{
+ char str[32];
+ char *base, *p;
+ struct tm tm;
+ time_t t;
+ int ret;
+
+ time(&t);
+ localtime_r(&t, &tm);
+
+ base = getenv("JITDUMPDIR");
+ if (!base)
+ base = getenv("HOME");
+ if (!base)
+ base = ".";
+
+ strftime(str, sizeof(str), JIT_LANG"-jit-%Y%m%d", &tm);
+
+ snprintf(jit_path, PATH_MAX - 1, "%s/.debug/", base);
+
+ ret = mkdir(jit_path, 0755);
+ if (ret == -1) {
+ if (errno != EEXIST) {
+ warn("jvmti: cannot create jit cache dir %s", jit_path);
+ return -1;
+ }
+ }
+
+ snprintf(jit_path, PATH_MAX - 1, "%s/.debug/jit", base);
+ ret = mkdir(jit_path, 0755);
+ if (ret == -1) {
+ if (errno != EEXIST) {
+ warn("cannot create jit cache dir %s", jit_path);
+ return -1;
+ }
+ }
+
+ snprintf(jit_path, PATH_MAX - 1, "%s/.debug/jit/%s.XXXXXXXX", base, str);
+
+ p = mkdtemp(jit_path);
+ if (p != jit_path) {
+ warn("cannot create jit cache dir %s", jit_path);
+ return -1;
+ }
+
+ return 0;
+}
+
+static int
+perf_open_marker_file(int fd)
+{
+ long pgsz;
+
+ pgsz = sysconf(_SC_PAGESIZE);
+ if (pgsz == -1)
+ return -1;
+
+ /*
+ * we mmap the jitdump to create an MMAP RECORD in perf.data file.
+ * The mmap is captured either live (perf record running when we mmap)
+ * or in deferred mode, via /proc/PID/maps
+ * the MMAP record is used as a marker of a jitdump file for more meta
+ * data info about the jitted code. Perf report/annotate detect this
+ * special filename and process the jitdump file.
+ *
+ * mapping must be PROT_EXEC to ensure it is captured by perf record
+ * even when not using -d option
+ */
+ marker_addr = mmap(NULL, pgsz, PROT_READ|PROT_EXEC, MAP_PRIVATE, fd, 0);
+ return (marker_addr == MAP_FAILED) ? -1 : 0;
+}
+
+static void
+perf_close_marker_file(void)
+{
+ long pgsz;
+
+ if (!marker_addr)
+ return;
+
+ pgsz = sysconf(_SC_PAGESIZE);
+ if (pgsz == -1)
+ return;
+
+ munmap(marker_addr, pgsz);
+}
+
+void *jvmti_open(void)
+{
+ int pad_cnt;
+ char dump_path[PATH_MAX];
+ struct jitheader header;
+ int fd;
+ FILE *fp;
+
+ /*
+ * check if clockid is supported
+ */
+ if (!perf_get_timestamp())
+ warnx("jvmti: kernel does not support %d clock id", perf_clk_id);
+
+ memset(&header, 0, sizeof(header));
+
+ debug_cache_init();
+
+ /*
+ * jitdump file name
+ */
+ snprintf(dump_path, PATH_MAX, "%s/jit-%i.dump", jit_path, getpid());
+
+ fd = open(dump_path, O_CREAT|O_TRUNC|O_RDWR, 0666);
+ if (fd == -1)
+ return NULL;
+
+ /*
+ * create perf.data maker for the jitdump file
+ */
+ if (perf_open_marker_file(fd)) {
+ warnx("jvmti: failed to create marker file");
+ return NULL;
+ }
+
+ fp = fdopen(fd, "w+");
+ if (!fp) {
+ warn("jvmti: cannot create %s", dump_path);
+ close(fd);
+ goto error;
+ }
+
+ warnx("jvmti: jitdump in %s", dump_path);
+
+ if (get_e_machine(&header)) {
+ warn("get_e_machine failed\n");
+ goto error;
+ }
+
+ header.magic = JITHEADER_MAGIC;
+ header.version = JITHEADER_VERSION;
+ header.total_size = sizeof(header);
+ header.pid = getpid();
+
+ /* calculate amount of padding '\0' */
+ pad_cnt = PADDING_8ALIGNED(header.total_size);
+ header.total_size += pad_cnt;
+
+ header.timestamp = perf_get_timestamp();
+
+ if (!fwrite(&header, sizeof(header), 1, fp)) {
+ warn("jvmti: cannot write dumpfile header");
+ goto error;
+ }
+
+ /* write padding '\0' if necessary */
+ if (pad_cnt && !fwrite(pad_bytes, pad_cnt, 1, fp)) {
+ warn("jvmti: cannot write dumpfile header padding");
+ goto error;
+ }
+
+ return fp;
+error:
+ fclose(fp);
+ return NULL;
+}
+
+int
+jvmti_close(void *agent)
+{
+ struct jr_code_close rec;
+ FILE *fp = agent;
+
+ if (!fp) {
+ warnx("jvmti: incalid fd in close_agent");
+ return -1;
+ }
+
+ rec.p.id = JIT_CODE_CLOSE;
+ rec.p.total_size = sizeof(rec);
+
+ rec.p.timestamp = perf_get_timestamp();
+
+ if (!fwrite(&rec, sizeof(rec), 1, fp))
+ return -1;
+
+ fclose(fp);
+
+ fp = NULL;
+
+ perf_close_marker_file();
+
+ return 0;
+}
+
+int jvmti_write_code(void *agent, char const *sym,
+ uint64_t vma, void const *code, unsigned int const size)
+{
+ static int code_generation = 1;
+ struct jr_code_load rec;
+ size_t sym_len;
+ size_t padding_count;
+ FILE *fp = agent;
+ int ret = -1;
+
+ /* don't care about 0 length function, no samples */
+ if (size == 0)
+ return 0;
+
+ if (!fp) {
+ warnx("jvmti: invalid fd in write_native_code");
+ return -1;
+ }
+
+ sym_len = strlen(sym) + 1;
+
+ rec.p.id = JIT_CODE_LOAD;
+ rec.p.total_size = sizeof(rec) + sym_len;
+ padding_count = PADDING_8ALIGNED(rec.p.total_size);
+ rec.p. total_size += padding_count;
+ rec.p.timestamp = perf_get_timestamp();
+
+ rec.code_size = size;
+ rec.vma = vma;
+ rec.code_addr = vma;
+ rec.pid = getpid();
+ rec.tid = gettid();
+ rec.code_index = code_generation++;
+
+ if (code)
+ rec.p.total_size += size;
+
+ /*
+ * If JVM is multi-threaded, nultiple concurrent calls to agent
+ * may be possible, so protect file writes
+ */
+ flockfile(fp);
+
+ ret = fwrite_unlocked(&rec, sizeof(rec), 1, fp);
+ fwrite_unlocked(sym, sym_len, 1, fp);
+ if (code)
+ fwrite_unlocked(code, size, 1, fp);
+
+ if (padding_count)
+ fwrite_unlocked(pad_bytes, padding_count, 1, fp);
+
+ funlockfile(fp);
+
+ ret = 0;
+
+ return ret;
+}
diff --git a/tools/perf/jvmti/jvmti_agent.h b/tools/perf/jvmti/jvmti_agent.h
new file mode 100644
index 0000000..54e5c5e
--- /dev/null
+++ b/tools/perf/jvmti/jvmti_agent.h
@@ -0,0 +1,23 @@
+#ifndef __JVMTI_AGENT_H__
+#define __JVMTI_AGENT_H__
+
+#include <sys/types.h>
+#include <stdint.h>
+
+#define __unused __attribute__((unused))
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+void *jvmti_open(void);
+int jvmti_close(void *agent);
+int jvmti_write_code(void *agent, char const *symbol_name,
+ uint64_t vma, void const *code,
+ const unsigned int code_size);
+
+#if defined(__cplusplus)
+}
+
+#endif
+#endif /* __JVMTI_H__ */
diff --git a/tools/perf/jvmti/libjvmti.c b/tools/perf/jvmti/libjvmti.c
new file mode 100644
index 0000000..8b8d782
--- /dev/null
+++ b/tools/perf/jvmti/libjvmti.c
@@ -0,0 +1,149 @@
+#include <sys/types.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <err.h>
+#include <jvmti.h>
+
+#include "jvmti_agent.h"
+
+void *jvmti_agent;
+
+static void JNICALL
+compiled_method_load_cb(jvmtiEnv *jvmti,
+ jmethodID method,
+ jint code_size,
+ void const *code_addr,
+ jint map_length,
+ jvmtiAddrLocationMap const *map,
+ void const *compile_info __unused)
+{
+ jclass decl_class;
+ char *class_sign = NULL;
+ char *func_name = NULL;
+ char *func_sign = NULL;
+ jvmtiError ret;
+ size_t len;
+
+ ret = (*jvmti)->GetMethodDeclaringClass(jvmti, method,
+ &decl_class);
+ if (ret != JVMTI_ERROR_NONE) {
+ warnx("jvmti: getmethoddeclaringclass failed");
+ return;
+ }
+
+ ret = (*jvmti)->GetClassSignature(jvmti, decl_class,
+ &class_sign, NULL);
+ if (ret != JVMTI_ERROR_NONE) {
+ warnx("jvmti: getclassignature failed");
+ goto error;
+ }
+
+ ret = (*jvmti)->GetMethodName(jvmti, method, &func_name,
+ &func_sign, NULL);
+ if (ret != JVMTI_ERROR_NONE) {
+ warnx("jvmti: failed getmethodname");
+ goto error;
+ }
+
+ len = strlen(func_name) + strlen(class_sign) + strlen(func_sign) + 2;
+
+ {
+ char str[len];
+ uint64_t addr = (uint64_t)(unsigned long)code_addr;
+ snprintf(str, len, "%s%s%s", class_sign, func_name, func_sign);
+ ret = jvmti_write_code(jvmti_agent, str, addr, code_addr, code_size);
+ if (ret)
+ warnx("jvmti: write_code() failed");
+ }
+error:
+ (*jvmti)->Deallocate(jvmti, (unsigned char *)func_name);
+ (*jvmti)->Deallocate(jvmti, (unsigned char *)func_sign);
+ (*jvmti)->Deallocate(jvmti, (unsigned char *)class_sign);
+}
+
+static void JNICALL
+code_generated_cb(jvmtiEnv *jvmti,
+ char const *name,
+ void const *code_addr,
+ jint code_size)
+{
+ uint64_t addr = (uint64_t)(unsigned long)code_addr;
+ int ret;
+
+ ret = jvmti_write_code(jvmti_agent, name, addr, code_addr, code_size);
+ if (ret)
+ warnx("jvmti: write_code() failed for code_generated");
+}
+
+JNIEXPORT jint JNICALL
+Agent_OnLoad(JavaVM *jvm, char *options, void *reserved __unused)
+{
+ jvmtiEventCallbacks cb;
+ jvmtiCapabilities caps1;
+ jvmtiEnv *jvmti = NULL;
+ jint ret;
+
+ jvmti_agent = jvmti_open();
+ if (!jvmti_agent) {
+ warnx("jvmti: open_agent failed");
+ return -1;
+ }
+
+ /*
+ * Request a JVMTI interface version 1 environment
+ */
+ ret = (*jvm)->GetEnv(jvm, (void *)&jvmti, JVMTI_VERSION_1);
+ if (ret != JNI_OK) {
+ warnx("jvmti: jvmti version 1 not supported");
+ return -1;
+ }
+
+ /*
+ * acquire method_load capability, we require it
+ */
+ memset(&caps1, 0, sizeof(caps1));
+ caps1.can_generate_compiled_method_load_events = 1;
+
+ ret = (*jvmti)->AddCapabilities(jvmti, &caps1);
+ if (ret != JVMTI_ERROR_NONE) {
+ warnx("jvmti: acquire compiled_method capability failed");
+ return -1;
+ }
+
+ memset(&cb, 0, sizeof(cb));
+
+ cb.CompiledMethodLoad = compiled_method_load_cb;
+ cb.DynamicCodeGenerated = code_generated_cb;
+
+ ret = (*jvmti)->SetEventCallbacks(jvmti, &cb, sizeof(cb));
+ if (ret != JVMTI_ERROR_NONE) {
+ warnx("jvmti: cannot set event callbacks");
+ return -1;
+ }
+
+ ret = (*jvmti)->SetEventNotificationMode(jvmti, JVMTI_ENABLE,
+ JVMTI_EVENT_COMPILED_METHOD_LOAD, NULL);
+ if (ret != JVMTI_ERROR_NONE) {
+ warnx("jvmti: setnotification failed for method_load");
+ return -1;
+ }
+
+ ret = (*jvmti)->SetEventNotificationMode(jvmti, JVMTI_ENABLE,
+ JVMTI_EVENT_DYNAMIC_CODE_GENERATED, NULL);
+ if (ret != JVMTI_ERROR_NONE) {
+ warnx("jvmti: setnotification failed on code_generated");
+ return -1;
+ }
+ return 0;
+}
+
+JNIEXPORT void JNICALL
+Agent_OnUnload(JavaVM *jvm __unused)
+{
+ int ret;
+
+ ret = jvmti_close(jvmti_agent);
+ if (ret)
+ errx(1, "Error: op_close_agent()");
+}
--
1.9.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH v6 1/4] perf,record: Add clockid parameter
2015-03-30 22:19 ` [PATCH v6 1/4] perf,record: Add clockid parameter Stephane Eranian
@ 2015-03-30 22:24 ` David Ahern
2015-03-30 22:27 ` Stephane Eranian
2015-03-31 7:16 ` Peter Zijlstra
2015-03-31 7:28 ` Peter Zijlstra
2 siblings, 1 reply; 18+ messages in thread
From: David Ahern @ 2015-03-30 22:24 UTC (permalink / raw)
To: Stephane Eranian, linux-kernel
Cc: acme, peterz, mingo, ak, jolsa, namhyung, cel, sukadev, sonnyrao,
johnmccutchan, adrian.hunter, pawel.moll
On 3/30/15 4:19 PM, Stephane Eranian wrote:
> @@ -761,6 +762,12 @@ void perf_evsel__config(struct perf_evsel *evsel, struct record_opts *opts)
> attr->disabled = 0;
> attr->enable_on_exec = 0;
> }
> +
> + if (opts->clockid >= 0) {
> + attr->use_clockid = 1;
> + attr->clockid = opts->clockid;
> + } else
> + attr->clockid = -1;
> }
The else part needs to be removed (can't set clockid to -1). And then ...
> diff --git a/tools/perf/util/header.c b/tools/perf/util/header.c
> index fb43215..164441a 100644
> --- a/tools/perf/util/header.c
> +++ b/tools/perf/util/header.c
> @@ -1098,6 +1098,8 @@ static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)
> }
> fprintf(fp, " }");
> }
> + fprintf(fp, ", clockid = %d", evsel->attr.clockid);
> +
this should be prefaced with:
if (evsel->attr.use_clockid)
(use_clockid bit says whether the clockid value was set)
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v6 1/4] perf,record: Add clockid parameter
2015-03-30 22:24 ` David Ahern
@ 2015-03-30 22:27 ` Stephane Eranian
0 siblings, 0 replies; 18+ messages in thread
From: Stephane Eranian @ 2015-03-30 22:27 UTC (permalink / raw)
To: David Ahern
Cc: LKML, Arnaldo Carvalho de Melo, Peter Zijlstra, mingo, ak,
Jiri Olsa, Namhyung Kim, Rose Belcher, Sukadev Bhattiprolu,
Sonny Rao, John Mccutchan, Adrian Hunter, Pawel Moll
On Mon, Mar 30, 2015 at 3:24 PM, David Ahern <dsahern@gmail.com> wrote:
> On 3/30/15 4:19 PM, Stephane Eranian wrote:
>>
>> @@ -761,6 +762,12 @@ void perf_evsel__config(struct perf_evsel *evsel,
>> struct record_opts *opts)
>> attr->disabled = 0;
>> attr->enable_on_exec = 0;
>> }
>> +
>> + if (opts->clockid >= 0) {
>> + attr->use_clockid = 1;
>> + attr->clockid = opts->clockid;
>> + } else
>> + attr->clockid = -1;
>> }
>
>
> The else part needs to be removed (can't set clockid to -1). And then ...
>
>> diff --git a/tools/perf/util/header.c b/tools/perf/util/header.c
>> index fb43215..164441a 100644
>> --- a/tools/perf/util/header.c
>> +++ b/tools/perf/util/header.c
>> @@ -1098,6 +1098,8 @@ static void print_event_desc(struct perf_header *ph,
>> int fd, FILE *fp)
>> }
>> fprintf(fp, " }");
>> }
>> + fprintf(fp, ", clockid = %d", evsel->attr.clockid);
>> +
>
>
> this should be prefaced with:
>
> if (evsel->attr.use_clockid)
>
> (use_clockid bit says whether the clockid value was set)
Ok, will fix in V7 or Arnaldo grabs the patch and merges it.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v6 2/4] perf tools: add Java demangling support
2015-03-30 22:19 ` [PATCH v6 2/4] perf tools: add Java demangling support Stephane Eranian
@ 2015-03-31 7:00 ` Pekka Enberg
0 siblings, 0 replies; 18+ messages in thread
From: Pekka Enberg @ 2015-03-31 7:00 UTC (permalink / raw)
To: Stephane Eranian
Cc: LKML, Arnaldo Carvalho de Melo, Peter Zijlstra, Ingo Molnar,
Andi Kleen, Jiri Olsa, Namhyung Kim, cel, sukadev, sonnyrao,
johnmccutchan, David Ahern, adrian.hunter, Pawel Moll
Hi Stephane,
On Tue, Mar 31, 2015 at 1:19 AM, Stephane Eranian <eranian@google.com> wrote:
> +#define BASE_ENT(c, n) [c-'A']=n
> +static const char *base_types['Z'-'A' + 1]={
> + BASE_ENT('B', "byte" ),
> + BASE_ENT('C', "char" ),
> + BASE_ENT('D', "double" ),
> + BASE_ENT('F', "float" ),
> + BASE_ENT('I', "int" ),
> + BASE_ENT('J', "long" ),
> + BASE_ENT('S', "short" ),
> + BASE_ENT('Z', "bool" ),
It's "boolean", not "bool" in JVM speak.
> +static char *
> +__demangle_java_sym(const char *str, const char *end, char *buf, int maxlen, int mode)
> +{
> + int rlen = 0;
> + int array = 0;
> + int narg = 0;
> + const char *q;
> +
> + if (!end)
> + end = str + strlen(str);
> +
> + for (q = str; q != end; q++) {
> +
> + if (rlen == (maxlen - 1))
> + break;
> +
> + switch (*q) {
> + case 'L':
> + if (mode == MODE_PREFIX || mode == MODE_CTYPE) {
> + if (mode == MODE_CTYPE) {
> + if (narg)
> + rlen += scnprintf(buf + rlen, maxlen - rlen, ", ");
> + narg++;
> + }
> + rlen += scnprintf(buf + rlen, maxlen - rlen, "class ");
> + if (mode == MODE_PREFIX)
> + mode = MODE_CLASS;
> + } else
> + buf[rlen++] = *q;
> + break;
This looks odd to me. "L" marks the beginning of an class name and
it's terminated by ";". You could just strhchr() the terminator and
simply copy the name to "buf" and drop cases ';', '/', and the default
label fro the switch statement.
> +char *
> +java_demangle_sym(const char *str, int flags)
> +{
[snip]
> + /*
> + * expansion factor estimated to 3x
> + */
> + len = strlen(str) * 3 + 1;
> + buf = malloc(len);
> + if (!buf)
> + return NULL;
Truncated symbols are lame. Can't you use realloc() to ensure that
never happens?
- Pekka
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v6 1/4] perf,record: Add clockid parameter
2015-03-30 22:19 ` [PATCH v6 1/4] perf,record: Add clockid parameter Stephane Eranian
2015-03-30 22:24 ` David Ahern
@ 2015-03-31 7:16 ` Peter Zijlstra
2015-03-31 7:28 ` Peter Zijlstra
2 siblings, 0 replies; 18+ messages in thread
From: Peter Zijlstra @ 2015-03-31 7:16 UTC (permalink / raw)
To: Stephane Eranian
Cc: linux-kernel, acme, mingo, ak, jolsa, namhyung, cel, sukadev,
sonnyrao, johnmccutchan, dsahern, adrian.hunter, pawel.moll
On Tue, Mar 31, 2015 at 12:19:31AM +0200, Stephane Eranian wrote:
You seem to have misplaced the hunk that goes with the
fallback_missing_features:
label.
> @@ -1122,6 +1132,16 @@ static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
> goto try_fallback;
> }
> set_rlimit = NO_CHANGE;
> +
> + /*
> + * If we succeeded but had to kill clockid, fail and
> + * have perf_evsel__open_strerror() print us a nice
> + * error.
> + */
> + if (perf_missing_features.clockid) {
> + err = -EINVAL;
> + goto out_close;
> + }
> }
> }
>
> @@ -1155,7 +1175,10 @@ static int __perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,
> if (err != -EINVAL || cpu > 0 || thread > 0)
> goto out_close;
>
> - if (!perf_missing_features.cloexec && (flags & PERF_FLAG_FD_CLOEXEC)) {
> + if (!perf_missing_features.clockid && evsel->attr.use_clockid) {
> + perf_missing_features.clockid = true;
> + goto fallback_missing_features;
> + } else if (!perf_missing_features.cloexec && (flags & PERF_FLAG_FD_CLOEXEC)) {
> perf_missing_features.cloexec = true;
> goto fallback_missing_features;
> } else if (!perf_missing_features.mmap2 && evsel->attr.mmap2) {
> @@ -2080,6 +2103,8 @@ int perf_evsel__fprintf(struct perf_evsel *evsel,
> if_print(wakeup_events);
> if_print(bp_type);
> if_print(branch_sample_type);
> + if_print(clockid);
> +
> }
> out:
> fputc('\n', fp);
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v6 1/4] perf,record: Add clockid parameter
2015-03-30 22:19 ` [PATCH v6 1/4] perf,record: Add clockid parameter Stephane Eranian
2015-03-30 22:24 ` David Ahern
2015-03-31 7:16 ` Peter Zijlstra
@ 2015-03-31 7:28 ` Peter Zijlstra
2 siblings, 0 replies; 18+ messages in thread
From: Peter Zijlstra @ 2015-03-31 7:28 UTC (permalink / raw)
To: Stephane Eranian
Cc: linux-kernel, acme, mingo, ak, jolsa, namhyung, cel, sukadev,
sonnyrao, johnmccutchan, dsahern, adrian.hunter, pawel.moll
On Tue, Mar 31, 2015 at 12:19:31AM +0200, Stephane Eranian wrote:
> @@ -1044,6 +1051,8 @@ static size_t perf_event_attr__fprintf(struct perf_event_attr *attr, FILE *fp)
> ret += PRINT_ATTR2(exclude_host, exclude_guest);
> ret += PRINT_ATTR2N("excl.callchain_kern", exclude_callchain_kernel,
> "excl.callchain_user", exclude_callchain_user);
> + ret += __PRINT_ATTR("%u",,use_clockid);
> +
>
> ret += PRINT_ATTR_U32(wakeup_events);
> ret += PRINT_ATTR_U32(wakeup_watermark);
> @@ -1056,6 +1065,7 @@ static size_t perf_event_attr__fprintf(struct perf_event_attr *attr, FILE *fp)
> ret += PRINT_ATTR_X64(sample_regs_user);
> ret += PRINT_ATTR_U32(sample_stack_user);
> ret += PRINT_ATTR_X64(sample_regs_intr);
> + ret += PRINT_ATTR_U32(clockid);
>
> ret += fprintf(fp, "%.60s\n", graph_dotted_line);
>
You also misplaced a bunch of changes here.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v6 0/4] perf: add support for profiling jitted code
2015-03-30 22:19 [PATCH v6 0/4] perf: add support for profiling jitted code Stephane Eranian
` (3 preceding siblings ...)
2015-03-30 22:19 ` [PATCH v6 4/4] perf tools: add JVMTI agent library Stephane Eranian
@ 2015-03-31 7:33 ` Brendan Gregg
2015-03-31 21:31 ` Brendan Gregg
4 siblings, 1 reply; 18+ messages in thread
From: Brendan Gregg @ 2015-03-31 7:33 UTC (permalink / raw)
To: Stephane Eranian
Cc: LKML, acme, Peter Zijlstra, Ingo Molnar, Andi Kleen, Jiri Olsa,
Namhyung Kim, Rose Belcher, sukadev, sonnyrao, johnmccutchan,
David Ahern, adrian.hunter, Pawel Moll
G'Day Stephane,
On Mon, Mar 30, 2015 at 3:19 PM, Stephane Eranian <eranian@google.com> wrote:
[...]
> The current support only works when the runtime is monitored from
> start to finish: perf record java --agentpath:libpfmjvmti.so my_class.
>
> Once the run is completed, the jitdump file needs to be injected into
> the perf.data file. This is accomplished by using the perf inject command.
> This will also generate an ELF image for each jitted function. The
> inject MMAP records will point to those ELF images. The reasoning
> behind using ELF images is that it makes processing for perf report
> and annotate automatic and transparent. It also makes it easier to
> package and analyze on a remote machine.
[...]
This is really impressive work. Do we have an idea of the overhead for
running the java agent?
Today, I'm using perf-map-agent, loaded dynamically, to dump a
/tmp/perf*.map file as needed. My company has tens of thousands of
Linux instances running Java, but very few need profiling, and we
don't know which beforehand. So a snapshot-on-demand approach is
ideal. An always-on approach, well, we'd have to know the overhead (I
can build the agent and test...).
During a profile, our symbols can change, but it's not been a serious
problem. I can take a before and after snapshot of symbols
(/tmp/perf.map files), then highlight which have moved and are
therefore suspicious (not perfect, but ok). Yes, having a timestamped
symbol log would be most accurate, but I wouldn't pay a lot of
overhead for it. Is there a (horror) story behind why this was so
necessary?
Minor comment: I didn't notice an fsync() in the jvmti code. Can
symbol updates be buffered and not written out by the time perf is
reading them?
Again, this is impressive work. Just wanted to share comments from a
heavy user of perf with Java. You may already have other, better,
ideas for doing on-demand symbol dumps anyway.
Brendan
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v6 0/4] perf: add support for profiling jitted code
2015-03-31 7:33 ` [PATCH v6 0/4] perf: add support for profiling jitted code Brendan Gregg
@ 2015-03-31 21:31 ` Brendan Gregg
0 siblings, 0 replies; 18+ messages in thread
From: Brendan Gregg @ 2015-03-31 21:31 UTC (permalink / raw)
To: Stephane Eranian
Cc: LKML, acme, Peter Zijlstra, Ingo Molnar, Andi Kleen, Jiri Olsa,
Namhyung Kim, Rose Belcher, sukadev, sonnyrao, johnmccutchan,
David Ahern, adrian.hunter, Pawel Moll
On Tue, Mar 31, 2015 at 12:33 AM, Brendan Gregg
<brendan.d.gregg@gmail.com> wrote:
> G'Day Stephane,
>
> On Mon, Mar 30, 2015 at 3:19 PM, Stephane Eranian <eranian@google.com> wrote:
> [...]
>> The current support only works when the runtime is monitored from
>> start to finish: perf record java --agentpath:libpfmjvmti.so my_class.
>>
>> Once the run is completed, the jitdump file needs to be injected into
>> the perf.data file. This is accomplished by using the perf inject command.
>> This will also generate an ELF image for each jitted function. The
>> inject MMAP records will point to those ELF images. The reasoning
>> behind using ELF images is that it makes processing for perf report
>> and annotate automatic and transparent. It also makes it easier to
>> package and analyze on a remote machine.
> [...]
>
> This is really impressive work. Do we have an idea of the overhead for
> running the java agent?
>
> Today, I'm using perf-map-agent, loaded dynamically, to dump a
> /tmp/perf*.map file as needed. My company has tens of thousands of
> Linux instances running Java, but very few need profiling, and we
> don't know which beforehand. So a snapshot-on-demand approach is
> ideal. An always-on approach, well, we'd have to know the overhead (I
> can build the agent and test...).
I built the agent and tested with an application framework
micro-benchmark, and saw the performance overhead drop after start
from about 13% initially (measured as a reduction in maximum req/sec
given fixed CPU capacity), to 1.1% after a minute, and then 0.13%
(which is really just noise) after several minutes of high load.
So the overhead is basically zero after (minutes of) warmup, at least
for my test. My jit.dump file reached 8 Mbytes, and was growing by a
tiny amount every 30 seconds or so (hence the near-zero overhead). I'm
much less concerned about overheads now.
I'll test with a production workload if I can... But I'm still curious
about why we're even doing this, instead of the previous method of
taking symbol snapshots. Is there a backstory? If it involves a case
of high symbol churn, then this should also mean non-zero overhead to
constantly log.
Brendan
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v6 3/4] perf inject: add jitdump mmap injection support
2015-03-30 22:19 ` [PATCH v6 3/4] perf inject: add jitdump mmap injection support Stephane Eranian
@ 2015-04-01 6:58 ` Adrian Hunter
[not found] ` <CABPqkBRd9+Ystsb-6gOn0Pni37BOc4uTGkj7DHfKbBvBCU9E7A@mail.gmail.com>
0 siblings, 1 reply; 18+ messages in thread
From: Adrian Hunter @ 2015-04-01 6:58 UTC (permalink / raw)
To: Stephane Eranian, linux-kernel
Cc: acme, peterz, mingo, ak, jolsa, namhyung, cel, sukadev, sonnyrao,
johnmccutchan, dsahern, pawel.moll
On 31/03/15 01:19, Stephane Eranian wrote:
> This patch adds a --jit option to perf inject.
>
> This options injects MMAP records into the perf.data
> file to cover the jitted code mmaps. It also emits
> ELF images for each function in the jidump file.
> Those images are created where the jitdump file is.
> The MMAP records point to that location as well.
>
> Typical flow:
> $ perf record -k mono -- java -agentpath:libpjvmti.so java_class
> $ perf inject --jit -i perf.data -o perf.data.jitted
> $ perf report -i perf.data.jitted
>
> Note that jitdump.h support is not limited to Java, it works with
> any jitted environment modified to emit the jitdump file format,
> include those where code can be jitted multiple times and moved
> around.
>
> The jitdump.h format is adapted from the Oprofile project.
>
> The genelf.c (ELF binary generation) depends on MD5 hash
> encoding for the buildid. To enable this, libssl-dev must
> be installed. If not, then genelf.c defaults to using
> urandom to generate the buildid, which is not ideal.
> The Makefile auto-detects the presence on libssl-dev.
>
> This version mmaps the jitdump file to create a marker
> MMAP record in the perf.data file. The marker is used to detect
> jitdump and cause perf inject to inject the jitted mmaps and
> generate ELF images for jitted functions.
>
> Signed-off-by: Stephane Eranian <eranian@google.com>
> ---
> tools/build/Makefile.feature | 2 +
> tools/build/feature/Makefile | 4 +
> tools/build/feature/test-libcrypto.c | 9 +
> tools/perf/Documentation/perf-inject.txt | 6 +
> tools/perf/builtin-inject.c | 60 +++-
> tools/perf/config/Makefile | 11 +
> tools/perf/util/Build | 2 +
> tools/perf/util/genelf.c | 479 +++++++++++++++++++++++++
> tools/perf/util/genelf.h | 6 +
> tools/perf/util/jit.h | 15 +
> tools/perf/util/jitdump.c | 588 +++++++++++++++++++++++++++++++
> tools/perf/util/jitdump.h | 92 +++++
> 12 files changed, 1261 insertions(+), 13 deletions(-)
> create mode 100644 tools/build/feature/test-libcrypto.c
> create mode 100644 tools/perf/util/genelf.c
> create mode 100644 tools/perf/util/genelf.h
> create mode 100644 tools/perf/util/jit.h
> create mode 100644 tools/perf/util/jitdump.c
> create mode 100644 tools/perf/util/jitdump.h
>
> diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
> index 3a0b0ca..8b468a3 100644
> --- a/tools/build/Makefile.feature
> +++ b/tools/build/Makefile.feature
> @@ -45,6 +45,7 @@ FEATURE_TESTS = \
> libpython \
> libpython-version \
> libslang \
> + libcrypto \
> libunwind \
> pthread-attr-setaffinity-np \
> stackprotector-all \
> @@ -64,6 +65,7 @@ FEATURE_DISPLAY = \
> libperl \
> libpython \
> libslang \
> + libcrypto \
> libunwind \
> libdw-dwarf-unwind \
> zlib \
> diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
> index 463ed8f..ed86700 100644
> --- a/tools/build/feature/Makefile
> +++ b/tools/build/feature/Makefile
> @@ -23,6 +23,7 @@ FILES= \
> test-libpython.bin \
> test-libpython-version.bin \
> test-libslang.bin \
> + test-libcrypto.bin \
> test-libunwind.bin \
> test-libunwind-debug-frame.bin \
> test-pthread-attr-setaffinity-np.bin \
> @@ -93,6 +94,9 @@ __BUILD = $(CC) $(CFLAGS) -Wall -Werror -o $(OUTPUT)$@ $(patsubst %.bin,%.c,$@)
> test-libslang.bin:
> $(BUILD) -I/usr/include/slang -lslang
>
> +test-libcrypto.bin:
> + $(BUILD) -lcrypto
> +
> test-gtk2.bin:
> $(BUILD) $(shell $(PKG_CONFIG) --libs --cflags gtk+-2.0 2>/dev/null)
>
> diff --git a/tools/build/feature/test-libcrypto.c b/tools/build/feature/test-libcrypto.c
> new file mode 100644
> index 0000000..8580c40
> --- /dev/null
> +++ b/tools/build/feature/test-libcrypto.c
> @@ -0,0 +1,9 @@
> +#include <openssl/md5.h>
> +
> +int main(void)
> +{
> + MD5_CTX context;
> +
> + MD5_Init(&context);
> + return 0;
> +}
> diff --git a/tools/perf/Documentation/perf-inject.txt b/tools/perf/Documentation/perf-inject.txt
> index dc7442c..bb33fe2 100644
> --- a/tools/perf/Documentation/perf-inject.txt
> +++ b/tools/perf/Documentation/perf-inject.txt
> @@ -44,6 +44,12 @@ OPTIONS
> --kallsyms=<file>::
> kallsyms pathname
>
> +-j::
> +--jit::
> + Process jitdump files by injecting the mmap records corresponding to jitted
> + functions. This option also generates the ELF images for each jitted function
> + found in the jitdumps files captured in the input perf.data file. Use this option
> + if you are monitoring environment using JIT runtimes, such as Java, DART or V8.
> SEE ALSO
> --------
> linkperf:perf-record[1], linkperf:perf-report[1], linkperf:perf-archive[1]
> diff --git a/tools/perf/builtin-inject.c b/tools/perf/builtin-inject.c
> index ea46df25..af25d1a 100644
> --- a/tools/perf/builtin-inject.c
> +++ b/tools/perf/builtin-inject.c
> @@ -16,6 +16,7 @@
> #include "util/debug.h"
> #include "util/build-id.h"
> #include "util/data.h"
> +#include "util/jit.h"
>
> #include "util/parse-options.h"
>
> @@ -26,6 +27,7 @@ struct perf_inject {
> struct perf_session *session;
> bool build_ids;
> bool sched_stat;
> + bool jit_mode;
> const char *input_name;
> struct perf_data_file output;
> u64 bytes_written;
> @@ -121,12 +123,19 @@ static int perf_event__repipe_mmap(struct perf_tool *tool,
> struct perf_sample *sample,
> struct machine *machine)
> {
> - int err;
> -
> - err = perf_event__process_mmap(tool, event, sample, machine);
> - perf_event__repipe(tool, event, sample, machine);
> -
> - return err;
> + struct perf_inject *inject = container_of(tool, struct perf_inject, tool);
> + u64 n = 0;
> +
> + if (inject->jit_mode) {
> + /*
> + * if jit marker, then inject jit mmaps and generate ELF images
> + */
> + if (!jit_process(&inject->tool, &inject->output, machine, event->mmap.filename, sample->pid, &n)) {
> + inject->bytes_written += n;
> + return 0;
> + }
> + }
You have dropped perf_event__process_mmap() from the !inject->jit_mode case.
But it would be nicer for jit_mode to have its own function.
> + return perf_event__repipe(tool, event, sample, machine);
> }
>
> static int perf_event__repipe_mmap2(struct perf_tool *tool,
> @@ -134,12 +143,19 @@ static int perf_event__repipe_mmap2(struct perf_tool *tool,
> struct perf_sample *sample,
> struct machine *machine)
> {
> - int err;
> -
> - err = perf_event__process_mmap2(tool, event, sample, machine);
> - perf_event__repipe(tool, event, sample, machine);
> -
> - return err;
> + struct perf_inject *inject = container_of(tool, struct perf_inject, tool);
> + u64 n = 0;
> +
> + if (inject->jit_mode) {
> + /*
> + * if jit marker, then inject jit mmaps and generate ELF images
> + */
> + if (!jit_process(&inject->tool, &inject->output, machine, event->mmap2.filename, sample->pid, &n)) {
> + inject->bytes_written += n;
> + return 0;
> + }
> + }
Ditto
> + return perf_event__repipe(tool, event, sample, machine);
> }
>
> static int perf_event__repipe_fork(struct perf_tool *tool,
> @@ -341,7 +357,6 @@ static int perf_evsel__check_stype(struct perf_evsel *evsel,
> name, sample_msg);
> return -EINVAL;
> }
> -
> return 0;
> }
>
> @@ -439,6 +454,7 @@ int cmd_inject(int argc, const char **argv, const char *prefix __maybe_unused)
> OPT_BOOLEAN('s', "sched-stat", &inject.sched_stat,
> "Merge sched-stat and sched-switch for getting events "
> "where and how long tasks slept"),
> + OPT_BOOLEAN('j', "jit", &inject.jit_mode, "merge jitdump files into perf.data file"),
> OPT_INCR('v', "verbose", &verbose,
> "be more verbose (show build ids, etc)"),
> OPT_STRING(0, "kallsyms", &symbol_conf.kallsyms_name, "file",
> @@ -470,6 +486,24 @@ int cmd_inject(int argc, const char **argv, const char *prefix __maybe_unused)
> if (inject.session == NULL)
> return -1;
>
> + if (inject.build_ids) {
> + /*
> + * to make sure the mmap records are ordered correctly
> + * and so that the correct especially due to jitted code
> + * mmaps. We cannot generate the buildid hit list and
> + * inject the jit mmaps at the same time for now.
> + */
> + inject.tool.ordered_events = true;
> + inject.tool.ordering_requires_timestamps = true;
> + }
> +
> + if (inject.jit_mode) {
> + inject.tool.mmap2 = perf_event__repipe_mmap2;
> + inject.tool.mmap = perf_event__repipe_mmap;
As suggested above, why not make your own tool fns e.g.
inject.tool.mmap2 = perf_event__jit_mode_mmap2;
inject.tool.mmap = perf_event__jit_mode_mmap;
> + inject.tool.ordered_events = true;
> + inject.tool.ordering_requires_timestamps = true;
You are taking advantage of a bug in perf-inject, that is the
"finished_round" events are not being processed. Really they should be
processed and you should inject in time order.
> + }
> +
> if (symbol__init(&inject.session->header.env) < 0)
> return -1;
>
> diff --git a/tools/perf/config/Makefile b/tools/perf/config/Makefile
> index cd121df..e3cf5e4 100644
> --- a/tools/perf/config/Makefile
> +++ b/tools/perf/config/Makefile
> @@ -366,6 +366,17 @@ ifndef NO_LIBAUDIT
> endif
> endif
>
> +ifndef NO_LIBCRYPTO
> + ifneq ($(feature-libcrypto), 1)
> + msg := $(warning No libcrypto.h found, disables jitted code injection, please install libssl-devel or libssl-dev);
> + NO_LIBCRYPTO := 1
> + else
> + CFLAGS += -DHAVE_LIBCRYPTO_SUPPORT
> + EXTLIBS += -lcrypto
> + $(call detected,CONFIG_CRYPTO)
> + endif
> +endif
> +
> ifdef NO_NEWT
> NO_SLANG=1
> endif
> diff --git a/tools/perf/util/Build b/tools/perf/util/Build
> index 9bff65e..8bc62b4 100644
> --- a/tools/perf/util/Build
> +++ b/tools/perf/util/Build
> @@ -75,6 +75,8 @@ libperf-$(CONFIG_X86) += tsc.o
> libperf-y += cloexec.o
> libperf-y += thread-stack.o
> libperf-y += demangle-java.o
> +libperf-y += jitdump.o
> +libperf-y += genelf.o
>
> libperf-$(CONFIG_LIBELF) += symbol-elf.o
> libperf-$(CONFIG_LIBELF) += probe-event.o
> diff --git a/tools/perf/util/genelf.c b/tools/perf/util/genelf.c
> new file mode 100644
> index 0000000..a5beebe
> --- /dev/null
> +++ b/tools/perf/util/genelf.c
> @@ -0,0 +1,479 @@
> +/*
> + * genelf.c
> + * Copyright (C) 2014, Google, Inc
> + *
> + * Contributed by:
> + * Stephane Eranian <eranian@gmail.com>
> + *
> + * Released under the GPL v2. (and only v2, not any later version)
> + */
> +
> +#include <sys/types.h>
> +#include <stdio.h>
> +#include <getopt.h>
> +#include <stddef.h>
> +#include <libelf.h>
> +#include <string.h>
> +#include <stdlib.h>
> +#include <fcntl.h>
> +#include <err.h>
> +
> +#include "perf.h"
> +#include "genelf.h"
> +
> +#define JVMTI
> +
> +#define BUILD_ID_URANDOM /* different uuid for each run */
> +
> +#ifdef HAVE_LIBCRYPTO
> +
> +#define BUILD_ID_MD5
> +#undef BUILD_ID_SHA /* does not seem to work well when linked with Java */
> +#undef BUILD_ID_URANDOM /* different uuid for each run */
> +
> +#ifdef BUILD_ID_SHA
> +#include <openssl/sha.h>
> +#endif
> +
> +#ifdef BUILD_ID_MD5
> +#include <openssl/md5.h>
> +#endif
> +#endif
> +
> +#if defined(__arm__)
> +#define GEN_ELF_ARCH EM_ARM
> +#define GEN_ELF_ENDIAN ELFDATA2LSB
> +#define GEN_ELF_CLASS ELFCLASS32
> +#elif defined(__x86_64__)
> +#define GEN_ELF_ARCH EM_X86_64
> +#define GEN_ELF_ENDIAN ELFDATA2LSB
> +#define GEN_ELF_CLASS ELFCLASS64
> +#elif defined(__i386__)
> +#define GEN_ELF_ARCH EM_386
> +#define GEN_ELF_ENDIAN ELFDATA2LSB
> +#define GEN_ELF_CLASS ELFCLASS32
> +#elif defined(__ppcle__)
> +#define GEN_ELF_ARCH EM_PPC
> +#define GEN_ELF_ENDIAN ELFDATA2LSB
> +#define GEN_ELF_CLASS ELFCLASS64
> +#elif defined(__powerpc__)
> +#define GEN_ELF_ARCH EM_PPC64
> +#define GEN_ELF_ENDIAN ELFDATA2MSB
> +#define GEN_ELF_CLASS ELFCLASS64
> +#elif defined(__powerpcle__)
> +#define GEN_ELF_ARCH EM_PPC64
> +#define GEN_ELF_ENDIAN ELFDATA2LSB
> +#define GEN_ELF_CLASS ELFCLASS64
> +#else
> +#error "unsupported architecture"
> +#endif
> +
> +#if GEN_ELF_CLASS == ELFCLASS64
> +#define elf_newehdr elf64_newehdr
> +#define elf_getshdr elf64_getshdr
> +#define Elf_Ehdr Elf64_Ehdr
> +#define Elf_Shdr Elf64_Shdr
> +#define Elf_Sym Elf64_Sym
> +#define ELF_ST_TYPE(a) ELF64_ST_TYPE(a)
> +#define ELF_ST_BIND(a) ELF64_ST_BIND(a)
> +#define ELF_ST_VIS(a) ELF64_ST_VISIBILITY(a)
> +#else
> +#define elf_newehdr elf32_newehdr
> +#define elf_getshdr elf32_getshdr
> +#define Elf_Ehdr Elf32_Ehdr
> +#define Elf_Shdr Elf32_Shdr
> +#define Elf_Sym Elf32_Sym
> +#define ELF_ST_TYPE(a) ELF32_ST_TYPE(a)
> +#define ELF_ST_BIND(a) ELF32_ST_BIND(a)
> +#define ELF_ST_VIS(a) ELF32_ST_VISIBILITY(a)
> +#endif
> +
> +typedef struct {
> + unsigned int namesz; /* Size of entry's owner string */
> + unsigned int descsz; /* Size of the note descriptor */
> + unsigned int type; /* Interpretation of the descriptor */
> + char name[0]; /* Start of the name+desc data */
> +} Elf_Note;
> +
> +struct options {
> + char *output;
> + int fd;
> +};
> +
> +static char shd_string_table[] = {
> + 0,
> + '.', 't', 'e', 'x', 't', 0, /* 1 */
> + '.', 's', 'h', 's', 't', 'r', 't', 'a', 'b', 0, /* 7 */
> + '.', 's', 'y', 'm', 't', 'a', 'b', 0, /* 17 */
> + '.', 's', 't', 'r', 't', 'a', 'b', 0, /* 25 */
> + '.', 'n', 'o', 't', 'e', '.', 'g', 'n', 'u', '.', 'b', 'u', 'i', 'l', 'd', '-', 'i', 'd', 0, /* 33 */
> +};
> +
> +
> +static struct buildid_note {
> + Elf_Note desc; /* descsz: size of build-id, must be multiple of 4 */
> + char name[4]; /* GNU\0 */
> + char build_id[20];
> +} bnote;
> +
> +static Elf_Sym symtab[]={
> + /* symbol 0 MUST be the undefined symbol */
> + { .st_name = 0, /* index in sym_string table */
> + .st_info = ELF_ST_TYPE(STT_NOTYPE),
> + .st_shndx = 0, /* for now */
> + .st_value = 0x0,
> + .st_other = ELF_ST_VIS(STV_DEFAULT),
> + .st_size = 0,
> + },
> + { .st_name = 1, /* index in sym_string table */
> + .st_info = ELF_ST_BIND(STB_LOCAL) | ELF_ST_TYPE(STT_FUNC),
> + .st_shndx = 1,
> + .st_value = 0, /* for now */
> + .st_other = ELF_ST_VIS(STV_DEFAULT),
> + .st_size = 0, /* for now */
> + }
> +};
> +
> +#ifdef BUILD_ID_URANDOM
> +static void
> +gen_build_id(struct buildid_note *note,
> + unsigned long load_addr __maybe_unused,
> + const void *code __maybe_unused,
> + size_t csize __maybe_unused)
> +{
> + int fd;
> + size_t sz = sizeof(note->build_id);
> + ssize_t sret;
> +
> + fd = open("/dev/urandom", O_RDONLY);
> + if (fd == -1)
> + err(1, "cannot access /dev/urandom for builid");
> +
> + sret = read(fd, note->build_id, sz);
> +
> + close(fd);
> +
> + if (sret != (ssize_t)sz)
> + memset(note->build_id, 0, sz);
> +}
> +#endif
> +
> +#ifdef BUILD_ID_SHA
> +static void
> +gen_build_id(struct buildid_note *note,
> + unsigned long load_addr __maybe_unused,
> + const void *code,
> + size_t csize)
> +{
> + if (sizeof(note->build_id) < SHA_DIGEST_LENGTH)
> + errx(1, "build_id too small for SHA1");
> +
> + SHA1(code, csize, (unsigned char *)note->build_id);
> +}
> +#endif
> +
> +#ifdef BUILD_ID_MD5
> +static void
> +gen_build_id(struct buildid_note *note, unsigned long load_addr, const void *code, size_t csize)
> +{
> + MD5_CTX context;
> +
> + if (sizeof(note->build_id) < 16)
> + errx(1, "build_id too small for MD5");
> +
> + MD5_Init(&context);
> + MD5_Update(&context, &load_addr, sizeof(load_addr));
> + MD5_Update(&context, code, csize);
> + MD5_Final((unsigned char *)note->build_id, &context);
> +}
> +#endif
> +
> +/*
> + * fd: file descriptor open for writing for the output file
> + * load_addr: code load address (could be zero, just used for buildid)
> + * sym: function name (for native code - used as the symbol)
> + * code: the native code
> + * csize: the code size in bytes
> + */
> +int
> +jit_write_elf(int fd, unsigned long load_addr, const char *sym, const void *code, int csize)
> +{
> + Elf *e;
> + Elf_Data *d;
> + Elf_Scn *scn;
> + Elf_Ehdr *ehdr;
> + Elf_Shdr *shdr;
> + char *strsym = NULL;
> + int symlen;
> + int retval = -1;
> +
> + if (elf_version(EV_CURRENT) == EV_NONE) {
> + warnx("ELF initialization failed");
> + return -1;
> + }
> +
> + e = elf_begin(fd, ELF_C_WRITE, NULL);
> + if (!e) {
> + warnx("elf_begin failed");
> + goto error;
> + }
> +
> + /*
> + * setup ELF header
> + */
> + ehdr = elf_newehdr(e);
> + if (!ehdr) {
> + warnx("cannot get ehdr");
> + goto error;
> + }
> +
> + ehdr->e_ident[EI_DATA] = GEN_ELF_ENDIAN;
> + ehdr->e_ident[EI_CLASS] = GEN_ELF_CLASS;
> + ehdr->e_machine = GEN_ELF_ARCH;
> + ehdr->e_type = ET_DYN;
> + ehdr->e_entry = 0x0;
> + ehdr->e_version = EV_CURRENT;
> + ehdr->e_shstrndx= 2; /* shdr index for section name */
> +
> + /*
> + * setup text section
> + */
> + scn = elf_newscn(e);
> + if (!scn) {
> + warnx("cannot create section");
> + goto error;
> + }
> +
> + d = elf_newdata(scn);
> + if (!d) {
> + warnx("cannot get new data");
> + goto error;
> + }
> +
> + d->d_align = 16;
> + d->d_off = 0LL;
> + d->d_buf = (void *)code;
> + d->d_type = ELF_T_BYTE;
> + d->d_size = csize;
> + d->d_version = EV_CURRENT;
> +
> + shdr = elf_getshdr(scn);
> + if (!shdr) {
> + warnx("cannot get section header");
> + goto error;
> + }
> +
> + shdr->sh_name = 1;
> + shdr->sh_type = SHT_PROGBITS;
> + shdr->sh_addr = 0; /* must be zero or == sh_offset -> dynamic object */
> + shdr->sh_flags = SHF_EXECINSTR | SHF_ALLOC;
> + shdr->sh_entsize = 0;
> +
> + /*
> + * setup section headers string table
> + */
> + scn = elf_newscn(e);
> + if (!scn) {
> + warnx("cannot create section");
> + goto error;
> + }
> +
> + d = elf_newdata(scn);
> + if (!d) {
> + warnx("cannot get new data");
> + goto error;
> + }
> +
> + d->d_align = 1;
> + d->d_off = 0LL;
> + d->d_buf = shd_string_table;
> + d->d_type = ELF_T_BYTE;
> + d->d_size = sizeof(shd_string_table);
> + d->d_version = EV_CURRENT;
> +
> + shdr = elf_getshdr(scn);
> + if (!shdr) {
> + warnx("cannot get section header");
> + goto error;
> + }
> +
> + shdr->sh_name = 7; /* offset of '.shstrtab' in shd_string_table */
> + shdr->sh_type = SHT_STRTAB;
> + shdr->sh_flags = 0;
> + shdr->sh_entsize = 0;
> +
> + /*
> + * setup symtab section
> + */
> + symtab[1].st_size = csize;
> +
> + scn = elf_newscn(e);
> + if (!scn) {
> + warnx("cannot create section");
> + goto error;
> + }
> +
> + d = elf_newdata(scn);
> + if (!d) {
> + warnx("cannot get new data");
> + goto error;
> + }
> +
> + d->d_align = 8;
> + d->d_off = 0LL;
> + d->d_buf = symtab;
> + d->d_type = ELF_T_SYM;
> + d->d_size = sizeof(symtab);
> + d->d_version = EV_CURRENT;
> +
> + shdr = elf_getshdr(scn);
> + if (!shdr) {
> + warnx("cannot get section header");
> + goto error;
> + }
> +
> + shdr->sh_name = 17; /* offset of '.symtab' in shd_string_table */
> + shdr->sh_type = SHT_SYMTAB;
> + shdr->sh_flags = 0;
> + shdr->sh_entsize = sizeof(Elf_Sym);
> + shdr->sh_link = 4; /* index of .strtab section */
> +
> + /*
> + * setup symbols string table
> + * 2 = 1 for 0 in 1st entry, 1 for the 0 at end of symbol for 2nd entry
> + */
> + symlen = 2 + strlen(sym);
> + strsym = calloc(1, symlen);
> + if (!strsym) {
> + warnx("cannot allocate strsym");
> + goto error;
> + }
> + strcpy(strsym + 1, sym);
> +
> + scn = elf_newscn(e);
> + if (!scn) {
> + warnx("cannot create section");
> + goto error;
> + }
> +
> + d = elf_newdata(scn);
> + if (!d) {
> + warnx("cannot get new data");
> + goto error;
> + }
> +
> + d->d_align = 1;
> + d->d_off = 0LL;
> + d->d_buf = strsym;
> + d->d_type = ELF_T_BYTE;
> + d->d_size = symlen;
> + d->d_version = EV_CURRENT;
> +
> + shdr = elf_getshdr(scn);
> + if (!shdr) {
> + warnx("cannot get section header");
> + goto error;
> + }
> +
> + shdr->sh_name = 25; /* offset in shd_string_table */
> + shdr->sh_type = SHT_STRTAB;
> + shdr->sh_flags = 0;
> + shdr->sh_entsize = 0;
> +
> + /*
> + * setup build-id section
> + */
> + scn = elf_newscn(e);
> + if (!scn) {
> + warnx("cannot create section");
> + goto error;
> + }
> +
> + d = elf_newdata(scn);
> + if (!d) {
> + warnx("cannot get new data");
> + goto error;
> + }
> +
> + /*
> + * build-id generation
> + */
> + gen_build_id(&bnote, load_addr, code, csize);
> + bnote.desc.namesz = sizeof(bnote.name); /* must include 0 termination */
> + bnote.desc.descsz = sizeof(bnote.build_id);
> + bnote.desc.type = NT_GNU_BUILD_ID;
> + strcpy(bnote.name, "GNU");
> +
> + d->d_align = 4;
> + d->d_off = 0LL;
> + d->d_buf = &bnote;
> + d->d_type = ELF_T_BYTE;
> + d->d_size = sizeof(bnote);
> + d->d_version = EV_CURRENT;
> +
> + shdr = elf_getshdr(scn);
> + if (!shdr) {
> + warnx("cannot get section header");
> + goto error;
> + }
> +
> + shdr->sh_name = 33; /* offset in shd_string_table */
> + shdr->sh_type = SHT_NOTE;
> + shdr->sh_addr = 0x0;
> + shdr->sh_flags = SHF_ALLOC;
> + shdr->sh_size = sizeof(bnote);
> + shdr->sh_entsize = 0;
> +
> + if (elf_update(e, ELF_C_WRITE) < 0) {
> + warnx("elf_update 4 failed");
> + goto error;
> + }
> + (void)elf_end(e);
> +
> + retval = 0;
> +error:
> + free(strsym);
> +
> + return retval;
> +}
> +
> +#ifndef JVMTI
> +
> +static unsigned char x86_code[] = {
> + 0xBB, 0x2A, 0x00, 0x00, 0x00, /* movl $42, %ebx */
> + 0xB8, 0x01, 0x00, 0x00, 0x00, /* movl $1, %eax */
> + 0xCD, 0x80 /* int $0x80 */
> +};
> +
> +static struct options options;
> +
> +int main(int argc, char **argv)
> +{
> + int c, fd, ret;
> +
> + while ((c = getopt(argc, argv, "o:h")) != -1) {
> + switch (c) {
> + case 'o':
> + options.output = optarg;
> + break;
> + case 'h':
> + printf("Usage: genelf -o output_file [-h]\n");
> + return 0;
> + default:
> + errx(1, "unknown option");
> + }
> + }
> +
> + fd = open(options.output, O_CREAT|O_TRUNC|O_RDWR, 0666);
> + if (fd == -1)
> + err(1, "cannot create file %s", options.output);
> +
> + ret = jit_write_elf(fd, "main", x86_code, sizeof(x86_code));
> + close(fd);
> +
> + if (ret != 0)
> + unlink(options.output);
> +
> + return ret;
> +}
> +#endif
> diff --git a/tools/perf/util/genelf.h b/tools/perf/util/genelf.h
> new file mode 100644
> index 0000000..11307a2
> --- /dev/null
> +++ b/tools/perf/util/genelf.h
> @@ -0,0 +1,6 @@
> +#ifndef __GENELF_H__
> +#define __GENELF_H__
> +
> +extern int jit_write_elf(int fd, unsigned long code_addr, const char *sym, const void *code, int csize);
> +
> +#endif
> diff --git a/tools/perf/util/jit.h b/tools/perf/util/jit.h
> new file mode 100644
> index 0000000..2a812c0
> --- /dev/null
> +++ b/tools/perf/util/jit.h
> @@ -0,0 +1,15 @@
> +#ifndef __JIT_H__
> +#define __JIT_H__
> +
> +#include <data.h>
> +
> +extern int jit_process(struct perf_tool *tool,
> + struct perf_data_file *output,
> + struct machine *machine,
> + char *filename,
> + pid_t pid,
> + u64 *nbytes);
> +
> +extern int jit_inject_record(const char *filename);
> +
> +#endif /* __JIT_H__ */
> diff --git a/tools/perf/util/jitdump.c b/tools/perf/util/jitdump.c
> new file mode 100644
> index 0000000..9c5fafe
> --- /dev/null
> +++ b/tools/perf/util/jitdump.c
> @@ -0,0 +1,588 @@
> +#include <sys/types.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <fcntl.h>
> +#include <unistd.h>
> +#include <inttypes.h>
> +#include <byteswap.h>
> +#include <sys/stat.h>
> +#include <sys/mman.h>
> +
> +#include "util.h"
> +#include "event.h"
> +#include "debug.h"
> +#include "symbol.h"
> +#include "strlist.h"
> +#include <elf.h>
> +
> +#include "session.h"
> +#include "jit.h"
> +#include "jitdump.h"
> +#include "genelf.h"
> +#include "../builtin.h"
> +
> +struct jit_buf_desc {
> + struct perf_data_file *output;
> + struct perf_tool *tool;
> + struct machine *machine;
> + union jr_entry *entry;
> + void *buf;
> + size_t bufsize;
> + FILE *in;
> + int needs_bswap; /* handles cross-endianess */
> + uint32_t code_load_count;
> + u64 bytes_written;
> + struct rb_root code_root;
> + char dir[PATH_MAX];
> +};
> +
> +struct debug_line_info {
> + unsigned long vma;
> + unsigned int lineno;
> + /* The filename format is unspecified, absolute path, relative etc. */
> + char const filename[0];
> +};
> +
> +struct jit_tool {
> + struct perf_tool tool;
> + struct perf_data_file output;
> + struct perf_data_file input;
> + u64 bytes_written;
> +};
> +
> +#define hmax(a, b) ((a) > (b) ? (a) : (b))
> +#define get_jit_tool(t) (container_of(tool, struct jit_tool, tool))
> +
> +static int
> +jit_emit_elf(char *filename,
> + const char *sym,
> + unsigned long code,
> + int csize)
> +{
> + int ret, fd;
> + unsigned long addr = (unsigned long)code;
> +
> + if (verbose > 0)
> + fprintf(stderr, "write ELF image %s\n", filename);
> +
> + fd = open(filename, O_CREAT|O_TRUNC|O_WRONLY, 0644);
> + if (fd == -1) {
> + pr_warning("cannot create jit ELF %s: %s\n", filename, strerror(errno));
> + return -1;
> + }
> +
> + ret = jit_write_elf(fd, addr, sym, (const void *)code, csize);
> +
> + close(fd);
> +
> + if (ret)
> + unlink(filename);
> +
> + return ret;
> +}
> +
> +static void
> +jit_close(struct jit_buf_desc *jd)
> +{
> + if (!(jd && jd->in))
> + return;
> + fclose(jd->in);
> + jd->in = NULL;
> +}
> +
> +static int
> +jit_open(struct jit_buf_desc *jd, const char *name)
> +{
> + struct jitheader header;
> + struct jr_prefix *prefix;
> + ssize_t bs, bsz = 0;
> + void *n, *buf = NULL;
> + int ret, retval = -1;
> +
> + jd->in = fopen(name, "r");
> + if (!jd->in)
> + return -1;
> +
> + bsz = hmax(sizeof(header), sizeof(*prefix));
> +
> + buf = malloc(bsz);
> + if (!buf)
> + goto error;
> +
> + ret = fread(buf, sizeof(header), 1, jd->in);
> + if (ret != 1)
> + goto error;
> +
> + memcpy(&header, buf, sizeof(header));
> +
> + if (header.magic != JITHEADER_MAGIC) {
> + if (header.magic != JITHEADER_MAGIC_SW)
> + goto error;
> + jd->needs_bswap = 1;
> + }
> +
> + if (jd->needs_bswap) {
> + header.version = bswap_32(header.version);
> + header.total_size = bswap_32(header.total_size);
> + header.pid = bswap_32(header.pid);
> + header.elf_mach = bswap_32(header.elf_mach);
> + header.timestamp = bswap_64(header.timestamp);
> + }
> +
> + if (verbose > 2)
> + pr_debug("version=%u\nhdr.size=%u\nts=0x%llx\npid=%d\nelf_mach=%d\n",
> + header.version,
> + header.total_size,
> + (unsigned long long)header.timestamp,
> + header.pid,
> + header.elf_mach);
> +
> + bs = header.total_size - sizeof(header);
> +
> + if (bs > bsz) {
> + n = realloc(buf, bs);
> + if (!n)
> + goto error;
> + bsz = bs;
> + buf = n;
> + /* read extra we do not know about */
> + ret = fread(buf, bs - bsz, 1, jd->in);
> + if (ret != 1)
> + goto error;
> + }
> + /*
> + * keep dirname for generating files and mmap records
> + */
> + strcpy(jd->dir, name);
> + dirname(jd->dir);
> +
> + return 0;
> +error:
> + fclose(jd->in);
> + return retval;
> +}
> +
> +static union jr_entry *
> +jit_get_next_entry(struct jit_buf_desc *jd)
> +{
> + struct jr_prefix *prefix;
> + union jr_entry *jr;
> + void *addr;
> + size_t bs, size;
> + int id, ret;
> +
> + if (!(jd && jd->in))
> + return NULL;
> +
> + if (jd->buf == NULL) {
> + size_t sz = getpagesize();
> + if (sz < sizeof(*prefix))
> + sz = sizeof(*prefix);
> +
> + jd->buf = malloc(sz);
> + if (jd->buf == NULL)
> + return NULL;
> +
> + jd->bufsize = sz;
> + }
> +
> + prefix = jd->buf;
> +
> + ret = fread(prefix, sizeof(*prefix), 1, jd->in);
> + if (ret != 1)
> + return NULL;
> +
> + if (jd->needs_bswap) {
> + prefix->id = bswap_32(prefix->id);
> + prefix->total_size = bswap_32(prefix->total_size);
> + prefix->timestamp = bswap_64(prefix->timestamp);
> + }
> + id = prefix->id;
> + size = prefix->total_size;
> +
> + bs = (size_t)size;
> + if (bs < sizeof(*prefix))
> + return NULL;
> +
> + if (id >= JIT_CODE_MAX) {
> + pr_warning("next_entry: unknown prefix %d, skipping\n", id);
> + return NULL;
> + }
> + if (bs > jd->bufsize) {
> + void *n;
> + n = realloc(jd->buf, bs);
> + if (!n)
> + return NULL;
> + jd->buf = n;
> + jd->bufsize = bs;
> + }
> +
> + addr = ((void *)jd->buf) + sizeof(*prefix);
> +
> + ret = fread(addr, bs - sizeof(*prefix), 1, jd->in);
> + if (ret != 1)
> + return NULL;
> +
> + jr = (union jr_entry *)jd->buf;
> +
> + switch(id) {
> + case JIT_CODE_DEBUG_INFO:
> + case JIT_CODE_CLOSE:
> + break;
> + case JIT_CODE_LOAD:
> + if (jd->needs_bswap) {
> + jr->load.pid = bswap_32(jr->load.pid);
> + jr->load.tid = bswap_32(jr->load.tid);
> + jr->load.vma = bswap_64(jr->load.vma);
> + jr->load.code_addr = bswap_64(jr->load.code_addr);
> + jr->load.code_size = bswap_64(jr->load.code_size);
> + jr->load.code_index= bswap_64(jr->load.code_index);
> + }
> + jd->code_load_count++;
> + break;
> + case JIT_CODE_MOVE:
> + if (jd->needs_bswap) {
> + jr->move.pid = bswap_32(jr->move.pid);
> + jr->move.tid = bswap_32(jr->move.tid);
> + jr->move.vma = bswap_64(jr->move.vma);
> + jr->move.old_code_addr = bswap_64(jr->move.old_code_addr);
> + jr->move.new_code_addr = bswap_64(jr->move.new_code_addr);
> + jr->move.code_size = bswap_64(jr->move.code_size);
> + jr->move.code_index = bswap_64(jr->move.code_index);
> + }
> + break;
> + case JIT_CODE_MAX:
> + default:
> + return NULL;
> + }
> + return jr;
> +}
> +
> +static int
> +jit_inject_event(struct jit_buf_desc *jd, union perf_event *event)
> +{
> + ssize_t size;
> +
> + size = perf_data_file__write(jd->output, event, event->header.size);
> + if (size < 0)
> + return -1;
> +
> + jd->bytes_written += size;
> + return 0;
> +}
> +
> +static int jit_repipe_code_load(struct jit_buf_desc *jd, union jr_entry *jr)
> +{
> + struct perf_sample sample;
> + union perf_event *event;
> + unsigned long code, addr;
> + char *filename;
> + struct stat st;
> + size_t size;
> + u16 idr_size;
> + const char *sym;
> + uint32_t count;
> + int ret, csize;
> + pid_t pid, tid;
> + struct {
> + u32 pid, tid;
> + u64 time;
> + } *id;
> +
> + pid = jr->load.pid;
> + tid = jr->load.tid;
> + csize = jr->load.code_size;
> + addr = jr->load.code_addr;
> + sym = (void *)((unsigned long)jr + sizeof(jr->load));
> + code = (unsigned long)jr + jr->load.p.total_size - csize;
> + count = jr->load.code_index;
> + idr_size = jd->machine->id_hdr_size;
> + /*
> + * +16 to account for sample_id_all (hack)
> + */
> + event = calloc(1, sizeof(*event) + 16);
> + if (!event)
> + return -1;
> +
> + filename = event->mmap2.filename;
> + size = snprintf(filename, PATH_MAX, "%s/jitted-%d-%u",
> + jd->dir,
> + pid,
> + count);
> +
> + size++; /* for \0 */
> +
> + size = PERF_ALIGN(size, sizeof(u64));
> +
> + ret = jit_emit_elf(filename, sym, code, csize);
> + if (ret) {
> + free(event);
> + return -1;
> + }
> +
> + if (stat(filename, &st))
> + memset(&st, 0, sizeof(stat));
> +
> + event->mmap2.header.type = PERF_RECORD_MMAP2;
> + event->mmap2.header.misc = PERF_RECORD_MISC_USER;
> + event->mmap2.header.size = (sizeof(event->mmap2) -
> + (sizeof(event->mmap2.filename) - size) + idr_size);
> +
> + event->mmap2.pgoff = 0;
> + event->mmap2.start = addr;
> + event->mmap2.len = csize;
> + event->mmap2.pid = pid;
> + event->mmap2.tid = tid;
> + event->mmap2.ino = st.st_ino;
> + event->mmap2.maj = major(st.st_dev);
> + event->mmap2.min = minor(st.st_dev);
> + event->mmap2.prot = st.st_mode;
> + event->mmap2.flags = MAP_SHARED;
> + event->mmap2.ino_generation = 1;
> +
> + id = (void *)((unsigned long)event + event->mmap.header.size - idr_size);
> + id->pid = pid;
> + id->tid = tid;
> + id->time = jr->load.p.timestamp;
You need to take acount of the sample_type here instead of hard-coding the
format of the id sample.
> +
> + /*
> + * create pseudo sample to induce dso hit increment
> + * use first address as sample address
> + */
> + memset(&sample, 0, sizeof(sample));
> + sample.pid = pid;
> + sample.tid = tid;
> + sample.time = id->time;
> + sample.ip = addr;
> +
> + ret = perf_event__process_mmap2(jd->tool, event, &sample, jd->machine);
> + if (ret)
> + return ret;
> +
> + ret = jit_inject_event(jd, event);
> + /*
> + * mark dso as use to generate buildid in the header
> + */
> + if (!ret)
> + build_id__mark_dso_hit(jd->tool, event, &sample, NULL, jd->machine);
> +
> + return ret;
> +}
> +
> +static int jit_repipe_code_move(struct jit_buf_desc *jd, union jr_entry *jr)
> +{
> + struct perf_sample sample;
> + union perf_event *event;
> + char *filename;
> + size_t size;
> + struct stat st;
> + u16 idr_size;
> + int ret;
> + pid_t pid, tid;
> + struct {
> + u32 pid, tid;
> + u64 time;
> + } *id;
> +
> + pid = jr->move.pid;
> + tid = jr->move.tid;
> + idr_size = jd->machine->id_hdr_size;
> +
> + /*
> + * +16 to account for sample_id_all (hack)
> + */
> + event = calloc(1, sizeof(*event) + 16);
> + if (!event)
> + return -1;
> +
> + filename = event->mmap2.filename;
> + size = snprintf(filename, PATH_MAX, "%s/jitted-%d-%"PRIu64,
> + jd->dir,
> + pid,
> + jr->move.code_index);
> +
> + size++; /* for \0 */
> +
> + if (stat(filename, &st))
> + memset(&st, 0, sizeof(stat));
> +
> + size = PERF_ALIGN(size, sizeof(u64));
> +
> + event->mmap2.header.type = PERF_RECORD_MMAP2;
> + event->mmap2.header.misc = PERF_RECORD_MISC_USER;
> + event->mmap2.header.size = (sizeof(event->mmap2) -
> + (sizeof(event->mmap2.filename) - size) + idr_size);
> + event->mmap2.pgoff = 0;
> + event->mmap2.start = jr->move.new_code_addr;
> + event->mmap2.len = jr->move.code_size;
> + event->mmap2.pid = pid;
> + event->mmap2.tid = tid;
> + event->mmap2.ino = st.st_ino;
> + event->mmap2.maj = major(st.st_dev);
> + event->mmap2.min = minor(st.st_dev);
> + event->mmap2.prot = st.st_mode;
> + event->mmap2.flags = MAP_SHARED;
> + event->mmap2.ino_generation = 1;
> +
> + id = (void *)((unsigned long)event + event->mmap.header.size - idr_size);
> + id->pid = pid;
> + id->tid = tid;
> + id->time = jr->move.p.timestamp;
> +
> + /*
> + * create pseudo sample to induce dso hit increment
> + * use first address as sample address
> + */
> + memset(&sample, 0, sizeof(sample));
> + sample.pid = pid;
> + sample.tid = tid;
> + sample.time = id->time;
> + sample.ip = jr->move.new_code_addr;
> +
> + ret = perf_event__process_mmap2(jd->tool, event, &sample, jd->machine);
> + if (ret)
> + return ret;
> +
> + ret = jit_inject_event(jd, event);
> + if (!ret)
> + build_id__mark_dso_hit(jd->tool, event, &sample, NULL, jd->machine);
> +
> + return ret;
> +}
> +
> +static int
> +jit_process_dump(struct jit_buf_desc *jd)
> +{
> + union jr_entry *jr;
> + int ret;
> +
> + while ((jr = jit_get_next_entry(jd))) {
> + switch(jr->prefix.id) {
> + case JIT_CODE_LOAD:
> + ret = jit_repipe_code_load(jd, jr);
> + break;
> + case JIT_CODE_MOVE:
> + ret = jit_repipe_code_move(jd, jr);
> + break;
> + default:
> + ret = 0;
> + continue;
> + }
> + }
> + return ret;
> +}
> +
> +static int
> +jit_inject(struct jit_buf_desc *jd, char *path)
> +{
> + int ret;
> +
> + if (verbose > 0)
> + fprintf(stderr, "injecting: %s\n", path);
> +
> + ret = jit_open(jd, path);
> + if (ret)
> + return -1;
> +
> + ret = jit_process_dump(jd);
> +
> + jit_close(jd);
> +
> + if (verbose > 0)
> + fprintf(stderr, "injected: %s (%d)\n", path, ret);
> +
> + return 0;
> +}
> +
> +/*
> + * File must be with pattern .../jit-XXXX.dump
> + * where XXXX is the PID of the process which did the mmap()
> + * as captured in the RECORD_MMAP record
> + */
> +static int
> +jit_detect(char *mmap_name, pid_t pid)
> + {
> + char *p;
> + char *end = NULL;
> + pid_t pid2;
> +
> + if (verbose > 2)
> + fprintf(stderr, "jit marker trying : %s\n", mmap_name);
> + /*
> + * get file name
> + */
> + p = strrchr(mmap_name, '/');
> + if (!p)
> + return -1;
> +
> + /*
> + * match prefix
> + */
> + if (strncmp(p, "/jit-", 5))
> + return -1;
> +
> + /*
> + * skip prefix
> + */
> + p += 5;
> +
> + /*
> + * must be followed by a pid
> + */
> + if (!isdigit(*p))
> + return -1;
> +
> + pid2 = (int)strtol(p, &end, 10);
> + if (!end)
> + return -1;
> +
> + /*
> + * pid does not match mmap pid
> + * pid==0 in system-wide mode (synthesized)
> + */
> + if (pid && pid2 != pid)
> + return -1;
> + /*
> + * validate suffix
> + */
> + if (strcmp(end, ".dump"))
> + return -1;
> +
> + if (verbose > 0)
> + fprintf(stderr, "jit marker found: %s\n", mmap_name);
> +
> + return 0;
> +}
> +
> +int
> +jit_process(struct perf_tool *tool,
> + struct perf_data_file *output,
> + struct machine *machine,
> + char *filename,
> + pid_t pid,
> + u64 *nbytes)
> +{
> + struct jit_buf_desc jd;
> + int ret;
> +
> + memset(&jd, 0, sizeof(jd));
> +
> + jd.tool = tool;
> + jd.output = output;
> + jd.machine = machine;
> +
> + *nbytes = 0;
> +
> + /*
> + * detect marker mmap (i.e., the jitdump mmap)
> + */
> + if (jit_detect(filename, pid))
> + return -1;
> +
> + ret = jit_inject(&jd, filename);
> + if (!ret)
> + *nbytes = jd.bytes_written;
> +
> + return ret;
> +}
> diff --git a/tools/perf/util/jitdump.h b/tools/perf/util/jitdump.h
> new file mode 100644
> index 0000000..120bdcf
> --- /dev/null
> +++ b/tools/perf/util/jitdump.h
> @@ -0,0 +1,92 @@
> +/*
> + * jitdump.h: jitted code info encapsulation file format
> + *
> + * Adapted from OProfile GPLv2 support jidump.h:
> + * Copyright 2007 OProfile authors
> + * Jens Wilke
> + * Daniel Hansel
> + * Copyright IBM Corporation 2007
> + */
> +#ifndef JITDUMP_H
> +#define JITDUMP_H
> +
> +#include <sys/time.h>
> +#include <time.h>
> +#include <stdint.h>
> +
> +/* JiTD */
> +#define JITHEADER_MAGIC 0x4A695444
> +#define JITHEADER_MAGIC_SW 0x4454694A
> +
> +#define PADDING_8ALIGNED(x) ((((x) + 7) & 7) ^ 7)
> +
> +#define JITHEADER_VERSION 1
> +
> +struct jitheader {
> + uint32_t magic; /* characters "jItD" */
> + uint32_t version; /* header version */
> + uint32_t total_size; /* total size of header */
> + uint32_t elf_mach; /* elf mach target */
> + uint32_t pad1; /* reserved */
> + uint32_t pid; /* JIT process id */
> + uint64_t timestamp; /* timestamp */
> +};
> +
> +enum jit_record_type {
> + JIT_CODE_LOAD = 0,
> + JIT_CODE_MOVE = 1,
> + JIT_CODE_DEBUG_INFO = 2,
> + JIT_CODE_CLOSE = 3,
> +
> + JIT_CODE_MAX,
> +};
> +
> +/* record prefix (mandatory in each record) */
> +struct jr_prefix {
> + uint32_t id;
> + uint32_t total_size;
> + uint64_t timestamp;
> +};
> +
> +struct jr_code_load {
> + struct jr_prefix p;
> +
> + uint32_t pid;
> + uint32_t tid;
> + uint64_t vma;
> + uint64_t code_addr;
> + uint64_t code_size;
> + uint64_t code_index;
> +};
> +
> +struct jr_code_close {
> + struct jr_prefix p;
> +};
> +
> +struct jr_code_move {
> + struct jr_prefix p;
> +
> + uint32_t pid;
> + uint32_t tid;
> + uint64_t vma;
> + uint64_t old_code_addr;
> + uint64_t new_code_addr;
> + uint64_t code_size;
> + uint64_t code_index;
> +};
> +
> +struct jr_code_debug_info {
> + struct jr_prefix p;
> +
> + uint64_t code_addr;
> + uint64_t nr_entry;
> +};
> +
> +union jr_entry {
> + struct jr_code_debug_info info;
> + struct jr_code_close close;
> + struct jr_code_load load;
> + struct jr_code_move move;
> + struct jr_prefix prefix;
> +};
> +#endif /* !JITDUMP_H */
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v6 3/4] perf inject: add jitdump mmap injection support
[not found] ` <CABPqkBRd9+Ystsb-6gOn0Pni37BOc4uTGkj7DHfKbBvBCU9E7A@mail.gmail.com>
@ 2015-04-08 12:15 ` Adrian Hunter
2015-04-08 14:12 ` Stephane Eranian
0 siblings, 1 reply; 18+ messages in thread
From: Adrian Hunter @ 2015-04-08 12:15 UTC (permalink / raw)
To: Stephane Eranian
Cc: LKML, Arnaldo Carvalho de Melo, Peter Zijlstra, mingo, ak,
Jiri Olsa, Namhyung Kim, Rose Belcher, Sukadev Bhattiprolu,
Sonny Rao, John Mccutchan, David Ahern, Pawel Moll
On 06/04/15 22:41, Stephane Eranian wrote:
> > + if (inject.build_ids) {
> > + /*
> > + * to make sure the mmap records are ordered correctly
> > + * and so that the correct especially due to jitted code
> > + * mmaps. We cannot generate the buildid hit list and
> > + * inject the jit mmaps at the same time for now.
> > + */
> > + inject.tool.ordered_events = true;
> > + inject.tool.ordering_requires_timestamps = true;
> > + }
> > +
> > + if (inject.jit_mode) {
> > + inject.tool.mmap2 = perf_event__repipe_mmap2;
> > + inject.tool.mmap = perf_event__repipe_mmap;
>
> As suggested above, why not make your own tool fns e.g.
>
> inject.tool.mmap2 = perf_event__jit_mode_mmap2;
> inject.tool.mmap = perf_event__jit_mode_mmap;
>
>
> > + inject.tool.ordered_events = true;
> > + inject.tool.ordering_requires_timestamps = true;
>
> You are taking advantage of a bug in perf-inject, that is the
> "finished_round" events are not being processed. Really they should be
> processed and you should inject in time order.
>
> I am not sure I understand.
> Yes, I am trying to have inject reorder the samples instead of perf report.
> You are likely to run inject once, and report many times. Also avoids a
> warning in report about out-of-order events.
Well forgetting about "finished_round", it seems to me you need to intercept
all the delivered events (which will be in time order) and inject your own
events at the right time.
At the moment it seems to me you are injecting all your events in one go
when you see the special jitdump mmap. So I would not expect the injected
events to be ordered with respect to other events that come later. But
maybe I misunderstand that?
As I confusingly tried to suggest earlier, one way to see all the
delivered events is to hook the ordered_events "deliver" callback. That
will mean injecting one mmap event at a time.
Here is just an idea.
struct perf_inject {
...
ordered_events__deliver_t deliver;
};
int cmd_inject(int argc, const char **argv, const char *prefix __maybe_unused)
{
...
inject.deliver = inject.session->ordered_events.deliver;
inject.session->ordered_events.deliver = inject_jit_mmap;
...
}
int inject_jit_mmap(struct ordered_events *oe, struct ordered_event *event)
{
struct perf_session *session = container_of(oe, struct perf_session, ordered_events);
struct perf_inject *inject = container_of(session->tool, struct perf_inject, tool);
/* Is it time to inject an event */
if (jit_next_timestamp(inject) < event->timestamp) {
/* Yes, so inject it by delivery */
perf_session__deliver_synth_event(...);
}
inject->deliver(oe, event);
}
You would also need to inject any remaining events right at the end.
Note the advantage of "delivering" events is that the same approach works whatever the tool.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v6 3/4] perf inject: add jitdump mmap injection support
2015-04-08 12:15 ` Adrian Hunter
@ 2015-04-08 14:12 ` Stephane Eranian
2015-04-10 12:51 ` Adrian Hunter
0 siblings, 1 reply; 18+ messages in thread
From: Stephane Eranian @ 2015-04-08 14:12 UTC (permalink / raw)
To: Adrian Hunter
Cc: LKML, Arnaldo Carvalho de Melo, Peter Zijlstra, mingo, ak,
Jiri Olsa, Namhyung Kim, Rose Belcher, Sukadev Bhattiprolu,
Sonny Rao, John Mccutchan, David Ahern, Pawel Moll
On Wed, Apr 8, 2015 at 5:15 AM, Adrian Hunter <adrian.hunter@intel.com> wrote:
> On 06/04/15 22:41, Stephane Eranian wrote:
>> > + if (inject.build_ids) {
>> > + /*
>> > + * to make sure the mmap records are ordered correctly
>> > + * and so that the correct especially due to jitted code
>> > + * mmaps. We cannot generate the buildid hit list and
>> > + * inject the jit mmaps at the same time for now.
>> > + */
>> > + inject.tool.ordered_events = true;
>> > + inject.tool.ordering_requires_timestamps = true;
>> > + }
>> > +
>> > + if (inject.jit_mode) {
>> > + inject.tool.mmap2 = perf_event__repipe_mmap2;
>> > + inject.tool.mmap = perf_event__repipe_mmap;
>>
>> As suggested above, why not make your own tool fns e.g.
>>
>> inject.tool.mmap2 = perf_event__jit_mode_mmap2;
>> inject.tool.mmap = perf_event__jit_mode_mmap;
>>
>>
>> > + inject.tool.ordered_events = true;
>> > + inject.tool.ordering_requires_timestamps = true;
>>
>> You are taking advantage of a bug in perf-inject, that is the
>> "finished_round" events are not being processed. Really they should be
>> processed and you should inject in time order.
>>
>> I am not sure I understand.
>> Yes, I am trying to have inject reorder the samples instead of perf report.
>> You are likely to run inject once, and report many times. Also avoids a
>> warning in report about out-of-order events.
>
> Well forgetting about "finished_round", it seems to me you need to intercept
> all the delivered events (which will be in time order) and inject your own
> events at the right time.
>
> At the moment it seems to me you are injecting all your events in one go
> when you see the special jitdump mmap. So I would not expect the injected
> events to be ordered with respect to other events that come later. But
> maybe I misunderstand that?
>
My understanding is that if I set ordered_events = true, then events
are not delivered immediately to the callbacks. They are queued and
sorted and then passed to the callbacks. And yes, there is the finished
round mechanism of which I don't quite fully understand the logic in this
case.
> As I confusingly tried to suggest earlier, one way to see all the
> delivered events is to hook the ordered_events "deliver" callback. That
> will mean injecting one mmap event at a time.
>
> Here is just an idea.
>
> struct perf_inject {
> ...
> ordered_events__deliver_t deliver;
> };
>
> int cmd_inject(int argc, const char **argv, const char *prefix __maybe_unused)
> {
> ...
> inject.deliver = inject.session->ordered_events.deliver;
> inject.session->ordered_events.deliver = inject_jit_mmap;
> ...
> }
>
ok on that.
> int inject_jit_mmap(struct ordered_events *oe, struct ordered_event *event)
> {
> struct perf_session *session = container_of(oe, struct perf_session, ordered_events);
> struct perf_inject *inject = container_of(session->tool, struct perf_inject, tool);
>
> /* Is it time to inject an event */
> if (jit_next_timestamp(inject) < event->timestamp) {
> /* Yes, so inject it by delivery */
> perf_session__deliver_synth_event(...);
> }
> inject->deliver(oe, event);
> }
>
That suggests I have buffered all the MMAPs synthesized from the jitdump and
I have some sorted queue based on jitdump timestamps. The test would have to
be more sophisticated that this. You'd want to insert at the right
time, i.e., you'd
have to track the previous and next timestamp in the stream:
retry:
j = jit_next_timestamp(inject->jit);
if (prev_event->timestamp <= j && j > next_event->timestamp) {
deliver_synth_event(inject->jit);
jit_next(inject->jit);
goto retry;
}
All of this is required because the finished round logic is such that perf
cannot sort all records at once. It does sort them by chunks. Thus I need
to make sure I inject in the right chunk otherwise it won't work. Am I reading
this right?
> You would also need to inject any remaining events right at the end.
>
> Note the advantage of "delivering" events is that the same approach works whatever the tool.
>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v6 3/4] perf inject: add jitdump mmap injection support
2015-04-08 14:12 ` Stephane Eranian
@ 2015-04-10 12:51 ` Adrian Hunter
2015-04-13 0:37 ` Stephane Eranian
0 siblings, 1 reply; 18+ messages in thread
From: Adrian Hunter @ 2015-04-10 12:51 UTC (permalink / raw)
To: Stephane Eranian
Cc: LKML, Arnaldo Carvalho de Melo, Peter Zijlstra, mingo, ak,
Jiri Olsa, Namhyung Kim, Rose Belcher, Sukadev Bhattiprolu,
Sonny Rao, John Mccutchan, David Ahern, Pawel Moll
On 08/04/15 17:12, Stephane Eranian wrote:
> On Wed, Apr 8, 2015 at 5:15 AM, Adrian Hunter <adrian.hunter@intel.com> wrote:
>> On 06/04/15 22:41, Stephane Eranian wrote:
>>> > + if (inject.build_ids) {
>>> > + /*
>>> > + * to make sure the mmap records are ordered correctly
>>> > + * and so that the correct especially due to jitted code
>>> > + * mmaps. We cannot generate the buildid hit list and
>>> > + * inject the jit mmaps at the same time for now.
>>> > + */
>>> > + inject.tool.ordered_events = true;
>>> > + inject.tool.ordering_requires_timestamps = true;
>>> > + }
>>> > +
>>> > + if (inject.jit_mode) {
>>> > + inject.tool.mmap2 = perf_event__repipe_mmap2;
>>> > + inject.tool.mmap = perf_event__repipe_mmap;
>>>
>>> As suggested above, why not make your own tool fns e.g.
>>>
>>> inject.tool.mmap2 = perf_event__jit_mode_mmap2;
>>> inject.tool.mmap = perf_event__jit_mode_mmap;
>>>
>>>
>>> > + inject.tool.ordered_events = true;
>>> > + inject.tool.ordering_requires_timestamps = true;
>>>
>>> You are taking advantage of a bug in perf-inject, that is the
>>> "finished_round" events are not being processed. Really they should be
>>> processed and you should inject in time order.
>>>
>>> I am not sure I understand.
>>> Yes, I am trying to have inject reorder the samples instead of perf report.
>>> You are likely to run inject once, and report many times. Also avoids a
>>> warning in report about out-of-order events.
>>
>> Well forgetting about "finished_round", it seems to me you need to intercept
>> all the delivered events (which will be in time order) and inject your own
>> events at the right time.
>>
>> At the moment it seems to me you are injecting all your events in one go
>> when you see the special jitdump mmap. So I would not expect the injected
>> events to be ordered with respect to other events that come later. But
>> maybe I misunderstand that?
>>
> My understanding is that if I set ordered_events = true, then events
> are not delivered immediately to the callbacks. They are queued and
> sorted and then passed to the callbacks. And yes, there is the finished
> round mechanism of which I don't quite fully understand the logic in this
> case.
>
>> As I confusingly tried to suggest earlier, one way to see all the
>> delivered events is to hook the ordered_events "deliver" callback. That
>> will mean injecting one mmap event at a time.
>>
>> Here is just an idea.
>>
>> struct perf_inject {
>> ...
>> ordered_events__deliver_t deliver;
>> };
>>
>> int cmd_inject(int argc, const char **argv, const char *prefix __maybe_unused)
>> {
>> ...
>> inject.deliver = inject.session->ordered_events.deliver;
>> inject.session->ordered_events.deliver = inject_jit_mmap;
>> ...
>> }
>>
> ok on that.
>
>> int inject_jit_mmap(struct ordered_events *oe, struct ordered_event *event)
>> {
>> struct perf_session *session = container_of(oe, struct perf_session, ordered_events);
>> struct perf_inject *inject = container_of(session->tool, struct perf_inject, tool);
>>
>> /* Is it time to inject an event */
>> if (jit_next_timestamp(inject) < event->timestamp) {
>> /* Yes, so inject it by delivery */
>> perf_session__deliver_synth_event(...);
>> }
>> inject->deliver(oe, event);
>> }
>>
> That suggests I have buffered all the MMAPs synthesized from the jitdump and
> I have some sorted queue based on jitdump timestamps.
Yes, but won't the jitdump timestamps already be in order?
> The test would have to
> be more sophisticated that this. You'd want to insert at the right
> time, i.e., you'd
> have to track the previous and next timestamp in the stream:
>
> retry:
> j = jit_next_timestamp(inject->jit);
> if (prev_event->timestamp <= j && j > next_event->timestamp) {
(prev_event->timestamp <= j) must be always true because otherwise you would
have injected the jit mmap event already.
So perf is about to deliver an event with timestamp 'event->timestamp', so
you just need to deliver all your events that have not already been
delivered but have a timestamp < event->timestamp
> deliver_synth_event(inject->jit);
> jit_next(inject->jit);
> goto retry;
> }
>
> All of this is required because the finished round logic is such that perf
> cannot sort all records at once.
It can sort them all at once, but finished_round is an optimization that
starts delivering "early" events before having to sort the "later" events.
The result is still that the delivered events are in sorted order.
> It does sort them by chunks. Thus I need
> to make sure I inject in the right chunk otherwise it won't work. Am I reading
> this right?
If you were to queue the events for sorting, yes. What I suggested was
delivering your events directly at the "right time".
>
>> You would also need to inject any remaining events right at the end.
>>
>> Note the advantage of "delivering" events is that the same approach works whatever the tool.
>>
>>
>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v6 3/4] perf inject: add jitdump mmap injection support
2015-04-10 12:51 ` Adrian Hunter
@ 2015-04-13 0:37 ` Stephane Eranian
2015-04-13 7:03 ` Adrian Hunter
0 siblings, 1 reply; 18+ messages in thread
From: Stephane Eranian @ 2015-04-13 0:37 UTC (permalink / raw)
To: Adrian Hunter
Cc: LKML, Arnaldo Carvalho de Melo, Peter Zijlstra, mingo, ak,
Jiri Olsa, Namhyung Kim, Rose Belcher, Sukadev Bhattiprolu,
Sonny Rao, John Mccutchan, David Ahern, Pawel Moll
On Fri, Apr 10, 2015 at 5:51 AM, Adrian Hunter <adrian.hunter@intel.com> wrote:
> On 08/04/15 17:12, Stephane Eranian wrote:
>> On Wed, Apr 8, 2015 at 5:15 AM, Adrian Hunter <adrian.hunter@intel.com> wrote:
>>> On 06/04/15 22:41, Stephane Eranian wrote:
>>>> > + if (inject.build_ids) {
>>>> > + /*
>>>> > + * to make sure the mmap records are ordered correctly
>>>> > + * and so that the correct especially due to jitted code
>>>> > + * mmaps. We cannot generate the buildid hit list and
>>>> > + * inject the jit mmaps at the same time for now.
>>>> > + */
>>>> > + inject.tool.ordered_events = true;
>>>> > + inject.tool.ordering_requires_timestamps = true;
>>>> > + }
>>>> > +
>>>> > + if (inject.jit_mode) {
>>>> > + inject.tool.mmap2 = perf_event__repipe_mmap2;
>>>> > + inject.tool.mmap = perf_event__repipe_mmap;
>>>>
>>>> As suggested above, why not make your own tool fns e.g.
>>>>
>>>> inject.tool.mmap2 = perf_event__jit_mode_mmap2;
>>>> inject.tool.mmap = perf_event__jit_mode_mmap;
>>>>
>>>>
>>>> > + inject.tool.ordered_events = true;
>>>> > + inject.tool.ordering_requires_timestamps = true;
>>>>
>>>> You are taking advantage of a bug in perf-inject, that is the
>>>> "finished_round" events are not being processed. Really they should be
>>>> processed and you should inject in time order.
>>>>
>>>> I am not sure I understand.
>>>> Yes, I am trying to have inject reorder the samples instead of perf report.
>>>> You are likely to run inject once, and report many times. Also avoids a
>>>> warning in report about out-of-order events.
>>>
>>> Well forgetting about "finished_round", it seems to me you need to intercept
>>> all the delivered events (which will be in time order) and inject your own
>>> events at the right time.
>>>
>>> At the moment it seems to me you are injecting all your events in one go
>>> when you see the special jitdump mmap. So I would not expect the injected
>>> events to be ordered with respect to other events that come later. But
>>> maybe I misunderstand that?
>>>
>> My understanding is that if I set ordered_events = true, then events
>> are not delivered immediately to the callbacks. They are queued and
>> sorted and then passed to the callbacks. And yes, there is the finished
>> round mechanism of which I don't quite fully understand the logic in this
>> case.
>>
>>> As I confusingly tried to suggest earlier, one way to see all the
>>> delivered events is to hook the ordered_events "deliver" callback. That
>>> will mean injecting one mmap event at a time.
>>>
>>> Here is just an idea.
>>>
>>> struct perf_inject {
>>> ...
>>> ordered_events__deliver_t deliver;
>>> };
>>>
>>> int cmd_inject(int argc, const char **argv, const char *prefix __maybe_unused)
>>> {
>>> ...
>>> inject.deliver = inject.session->ordered_events.deliver;
>>> inject.session->ordered_events.deliver = inject_jit_mmap;
>>> ...
>>> }
>>>
>> ok on that.
>>
>>> int inject_jit_mmap(struct ordered_events *oe, struct ordered_event *event)
>>> {
>>> struct perf_session *session = container_of(oe, struct perf_session, ordered_events);
>>> struct perf_inject *inject = container_of(session->tool, struct perf_inject, tool);
>>>
>>> /* Is it time to inject an event */
>>> if (jit_next_timestamp(inject) < event->timestamp) {
>>> /* Yes, so inject it by delivery */
>>> perf_session__deliver_synth_event(...);
>>> }
>>> inject->deliver(oe, event);
>>> }
>>>
>> That suggests I have buffered all the MMAPs synthesized from the jitdump and
>> I have some sorted queue based on jitdump timestamps.
>
> Yes, but won't the jitdump timestamps already be in order?
>
I looked at implementing your approach and it turns out to be much
more complicated
than what is already there.
I do not inject anything if I don't have to. I am waiting to see a
special MMAP of the
jitdump file to start inject jit mmaps. That could not change in your
scheme. I would
need to track MMAP records and check if they correspond to a jitdump. Note that
I may have to inject multiple jitdumps in one run of perf inject. In
that case, in the
deliver function, I would have to (1) either have all the jitdump MMAPs prepared
in order for all the jitdumps I have detected or (2) go through all
the jitdump current
head records to see which one matches. In (2) I would have to do this
not just when
I see an MMAP but for any event delivered: for each new event, compare its
timestamp to the timestamps of jitdump current record for ALL jitdump
files. For (1),
I need to build a global list of all jitdump records in for each
jitdump file, i.e, another
RB tree. And then pull them one by one out if I find an event with the
right timestamp.
I think (2) i more reasonable but requires quite some infrastructure
work in the jitdump
code. Most of the pieces are there, the RB tree needs to be added
obviously. I will
experiment with (2).
>> The test would have to
>> be more sophisticated that this. You'd want to insert at the right
>> time, i.e., you'd
>> have to track the previous and next timestamp in the stream:
>>
>> retry:
>> j = jit_next_timestamp(inject->jit);
>> if (prev_event->timestamp <= j && j > next_event->timestamp) {
>
> (prev_event->timestamp <= j) must be always true because otherwise you would
> have injected the jit mmap event already.
>
> So perf is about to deliver an event with timestamp 'event->timestamp', so
> you just need to deliver all your events that have not already been
> delivered but have a timestamp < event->timestamp
>
>
>> deliver_synth_event(inject->jit);
>> jit_next(inject->jit);
>> goto retry;
>> }
>>
>> All of this is required because the finished round logic is such that perf
>> cannot sort all records at once.
>
> It can sort them all at once, but finished_round is an optimization that
> starts delivering "early" events before having to sort the "later" events.
> The result is still that the delivered events are in sorted order.
>
>> It does sort them by chunks. Thus I need
>> to make sure I inject in the right chunk otherwise it won't work. Am I reading
>> this right?
>
> If you were to queue the events for sorting, yes. What I suggested was
> delivering your events directly at the "right time".
>
>>
>>> You would also need to inject any remaining events right at the end.
>>>
>>> Note the advantage of "delivering" events is that the same approach works whatever the tool.
>>>
>>>
>>
>>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v6 3/4] perf inject: add jitdump mmap injection support
2015-04-13 0:37 ` Stephane Eranian
@ 2015-04-13 7:03 ` Adrian Hunter
0 siblings, 0 replies; 18+ messages in thread
From: Adrian Hunter @ 2015-04-13 7:03 UTC (permalink / raw)
To: Stephane Eranian
Cc: LKML, Arnaldo Carvalho de Melo, Peter Zijlstra, mingo, ak,
Jiri Olsa, Namhyung Kim, Rose Belcher, Sukadev Bhattiprolu,
Sonny Rao, John Mccutchan, David Ahern, Pawel Moll
On 13/04/15 03:37, Stephane Eranian wrote:
> On Fri, Apr 10, 2015 at 5:51 AM, Adrian Hunter <adrian.hunter@intel.com> wrote:
>> On 08/04/15 17:12, Stephane Eranian wrote:
>>> On Wed, Apr 8, 2015 at 5:15 AM, Adrian Hunter <adrian.hunter@intel.com> wrote:
>>>> On 06/04/15 22:41, Stephane Eranian wrote:
>>>>> > + if (inject.build_ids) {
>>>>> > + /*
>>>>> > + * to make sure the mmap records are ordered correctly
>>>>> > + * and so that the correct especially due to jitted code
>>>>> > + * mmaps. We cannot generate the buildid hit list and
>>>>> > + * inject the jit mmaps at the same time for now.
>>>>> > + */
>>>>> > + inject.tool.ordered_events = true;
>>>>> > + inject.tool.ordering_requires_timestamps = true;
>>>>> > + }
>>>>> > +
>>>>> > + if (inject.jit_mode) {
>>>>> > + inject.tool.mmap2 = perf_event__repipe_mmap2;
>>>>> > + inject.tool.mmap = perf_event__repipe_mmap;
>>>>>
>>>>> As suggested above, why not make your own tool fns e.g.
>>>>>
>>>>> inject.tool.mmap2 = perf_event__jit_mode_mmap2;
>>>>> inject.tool.mmap = perf_event__jit_mode_mmap;
>>>>>
>>>>>
>>>>> > + inject.tool.ordered_events = true;
>>>>> > + inject.tool.ordering_requires_timestamps = true;
>>>>>
>>>>> You are taking advantage of a bug in perf-inject, that is the
>>>>> "finished_round" events are not being processed. Really they should be
>>>>> processed and you should inject in time order.
>>>>>
>>>>> I am not sure I understand.
>>>>> Yes, I am trying to have inject reorder the samples instead of perf report.
>>>>> You are likely to run inject once, and report many times. Also avoids a
>>>>> warning in report about out-of-order events.
>>>>
>>>> Well forgetting about "finished_round", it seems to me you need to intercept
>>>> all the delivered events (which will be in time order) and inject your own
>>>> events at the right time.
>>>>
>>>> At the moment it seems to me you are injecting all your events in one go
>>>> when you see the special jitdump mmap. So I would not expect the injected
>>>> events to be ordered with respect to other events that come later. But
>>>> maybe I misunderstand that?
>>>>
>>> My understanding is that if I set ordered_events = true, then events
>>> are not delivered immediately to the callbacks. They are queued and
>>> sorted and then passed to the callbacks. And yes, there is the finished
>>> round mechanism of which I don't quite fully understand the logic in this
>>> case.
>>>
>>>> As I confusingly tried to suggest earlier, one way to see all the
>>>> delivered events is to hook the ordered_events "deliver" callback. That
>>>> will mean injecting one mmap event at a time.
>>>>
>>>> Here is just an idea.
>>>>
>>>> struct perf_inject {
>>>> ...
>>>> ordered_events__deliver_t deliver;
>>>> };
>>>>
>>>> int cmd_inject(int argc, const char **argv, const char *prefix __maybe_unused)
>>>> {
>>>> ...
>>>> inject.deliver = inject.session->ordered_events.deliver;
>>>> inject.session->ordered_events.deliver = inject_jit_mmap;
>>>> ...
>>>> }
>>>>
>>> ok on that.
>>>
>>>> int inject_jit_mmap(struct ordered_events *oe, struct ordered_event *event)
>>>> {
>>>> struct perf_session *session = container_of(oe, struct perf_session, ordered_events);
>>>> struct perf_inject *inject = container_of(session->tool, struct perf_inject, tool);
>>>>
>>>> /* Is it time to inject an event */
>>>> if (jit_next_timestamp(inject) < event->timestamp) {
>>>> /* Yes, so inject it by delivery */
>>>> perf_session__deliver_synth_event(...);
>>>> }
>>>> inject->deliver(oe, event);
>>>> }
>>>>
>>> That suggests I have buffered all the MMAPs synthesized from the jitdump and
>>> I have some sorted queue based on jitdump timestamps.
>>
>> Yes, but won't the jitdump timestamps already be in order?
>>
> I looked at implementing your approach and it turns out to be much
> more complicated
> than what is already there.
> I do not inject anything if I don't have to. I am waiting to see a
> special MMAP of the
> jitdump file to start inject jit mmaps. That could not change in your
> scheme. I would
> need to track MMAP records and check if they correspond to a jitdump. Note that
> I may have to inject multiple jitdumps in one run of perf inject. In
> that case, in the
> deliver function, I would have to (1) either have all the jitdump MMAPs prepared
> in order for all the jitdumps I have detected or (2) go through all
> the jitdump current
> head records to see which one matches. In (2) I would have to do this
> not just when
> I see an MMAP but for any event delivered: for each new event, compare its
> timestamp to the timestamps of jitdump current record for ALL jitdump
> files. For (1),
> I need to build a global list of all jitdump records in for each
> jitdump file, i.e, another
> RB tree. And then pull them one by one out if I find an event with the
> right timestamp.
>
> I think (2) i more reasonable but requires quite some infrastructure
> work in the jitdump
> code. Most of the pieces are there, the RB tree needs to be added
> obviously. I will
> experiment with (2).
I do something like (2) for Intel PT, but I use a heap to hold the next
lowest timestamp for each "queue" of data that needs to be processed -
because I have to allow for the possibility of a large number of "queues".
>
>
>>> The test would have to
>>> be more sophisticated that this. You'd want to insert at the right
>>> time, i.e., you'd
>>> have to track the previous and next timestamp in the stream:
>>>
>>> retry:
>>> j = jit_next_timestamp(inject->jit);
>>> if (prev_event->timestamp <= j && j > next_event->timestamp) {
>>
>> (prev_event->timestamp <= j) must be always true because otherwise you would
>> have injected the jit mmap event already.
>>
>> So perf is about to deliver an event with timestamp 'event->timestamp', so
>> you just need to deliver all your events that have not already been
>> delivered but have a timestamp < event->timestamp
>>
>>
>>> deliver_synth_event(inject->jit);
>>> jit_next(inject->jit);
>>> goto retry;
>>> }
>>>
>>> All of this is required because the finished round logic is such that perf
>>> cannot sort all records at once.
>>
>> It can sort them all at once, but finished_round is an optimization that
>> starts delivering "early" events before having to sort the "later" events.
>> The result is still that the delivered events are in sorted order.
>>
>>> It does sort them by chunks. Thus I need
>>> to make sure I inject in the right chunk otherwise it won't work. Am I reading
>>> this right?
>>
>> If you were to queue the events for sorting, yes. What I suggested was
>> delivering your events directly at the "right time".
>>
>>>
>>>> You would also need to inject any remaining events right at the end.
>>>>
>>>> Note the advantage of "delivering" events is that the same approach works whatever the tool.
>>>>
>>>>
>>>
>>>
>>
>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2015-04-13 7:06 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-03-30 22:19 [PATCH v6 0/4] perf: add support for profiling jitted code Stephane Eranian
2015-03-30 22:19 ` [PATCH v6 1/4] perf,record: Add clockid parameter Stephane Eranian
2015-03-30 22:24 ` David Ahern
2015-03-30 22:27 ` Stephane Eranian
2015-03-31 7:16 ` Peter Zijlstra
2015-03-31 7:28 ` Peter Zijlstra
2015-03-30 22:19 ` [PATCH v6 2/4] perf tools: add Java demangling support Stephane Eranian
2015-03-31 7:00 ` Pekka Enberg
2015-03-30 22:19 ` [PATCH v6 3/4] perf inject: add jitdump mmap injection support Stephane Eranian
2015-04-01 6:58 ` Adrian Hunter
[not found] ` <CABPqkBRd9+Ystsb-6gOn0Pni37BOc4uTGkj7DHfKbBvBCU9E7A@mail.gmail.com>
2015-04-08 12:15 ` Adrian Hunter
2015-04-08 14:12 ` Stephane Eranian
2015-04-10 12:51 ` Adrian Hunter
2015-04-13 0:37 ` Stephane Eranian
2015-04-13 7:03 ` Adrian Hunter
2015-03-30 22:19 ` [PATCH v6 4/4] perf tools: add JVMTI agent library Stephane Eranian
2015-03-31 7:33 ` [PATCH v6 0/4] perf: add support for profiling jitted code Brendan Gregg
2015-03-31 21:31 ` Brendan Gregg
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).