* [PATCHv6 0/7] perf tools: Add perf data CTF conversion
@ 2015-04-18 15:50 Jiri Olsa
2015-04-18 15:50 ` [PATCH 1/7] perf data: Show error message when conversion failed Jiri Olsa
` (7 more replies)
0 siblings, 8 replies; 29+ messages in thread
From: Jiri Olsa @ 2015-04-18 15:50 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: David Ahern, Frederic Weisbecker, He Kuang, Ingo Molnar,
Jeremie Galarneau, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan, lkml
hi,
this is follow up on original RFC patchset:
http://marc.info/?t=140732735600004&r=1&w=2
Basically we are adding 'perf data convert' command to
allow conversion of perf data file into CTF [1] data.
v6 changes:
- first bunch of patches got accepted, adding the
rest plus additional fixes
- rebased to latest acme's perf/core
- removed acked/reviewed tags from patch 2, due to small
change - changed ctf_writer__flush_streams to return
error value
- it all works with recent tracecompass and libbabeltrace git tree
- after recent changes in build system, it needs to be compiled with:
$ make LIBBABELTRACE=1 LIBBABELTRACE_DIR=/opt/libbabeltrace/
this will change once new version libabeltrace is released
and adopted by common distros
v5 changes:
- added struct field_format::alias instead of priv void*
- rebased to latest acme's perf/core
v4 changes:
- sorted out babeltrace new code regressions
- rebased to latest acme's perf/core
- includes fixes from Wang Nan, patches 9/10/11
- patch 1/11 changelog updated with babeltrace
compilation instructions [Arnaldo]
v3 changes:
- rebased to latest acme's perf/core
v2 changes:
- addressed comments from Namhyung
- rebased to latest acme's perf/core
Changes from RFC:
- able to generate CTF data, that are possible to be displayed under
tracecompas GUI [3], please check several screenshots in here [4]
- storing CTF data streams per cpu
- several cleanups
Examples:
- Catch default perf data (cycles event):
$ perf record ls
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.012 MB perf.data (~546 samples) ]
- To display converted CTF data run [2]:
$ babeltrace ./ctf-data/
[03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
[03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
[03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 }
[03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 }
[03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 }
[03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 }
[03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 }
[03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 }
[03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }
- To get some nice output in tracecompas GUI [3], please capture sched:*
and syscall tracepoints like:
# perf record -e 'sched:*,raw_syscalls:*' -a
^C[ perf record: Woken up 0 times to write data ]
[ perf record: Captured and wrote 412.347 MB perf.data (~18015721 samples) ]
- To convert perf data file run:
# perf data convert --to-ctf=./ctf
[ perf data convert: Converted 'perf.data' into CTF data './ctf' ]
[ perf data convert: Converted and wrote 408.421 MB (3964792 samples) ]
- To display converted CTF data run [2]:
# babeltrace ./ctf-data/
[23:32:20.165354855] (+0.000000507) sched:sched_wakeup: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810BCA72, perf_tid = 0, perf_pid = 0, perf_id = 462554, perf_period = 1, common_type = 265, ...
[23:32:20.165359078] (+0.000001181) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 0, perf_pid = 0, perf_id = 462562, perf_period = 1, common_type = 263, ...
[23:32:20.165364686] (+0.000000328) sched:sched_stat_runtime: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 5326, perf_pid = 5326, perf_id = 462610, perf_period = 11380, ...
[23:32:20.165366067] (+0.000001205) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 5326, perf_pid = 5326, perf_id = 462562, perf_period = 1, common_type ...
[23:32:20.165723312] (+0.000001479) sched:sched_stat_runtime: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 11821, perf_pid = 11821, perf_id = 462612, perf_period = 1000265, ...
[23:32:20.065282391] (+?.?????????) raw_syscalls:sys_enter: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810230AF, perf_tid = 26155, perf_pid = 26155, perf_id = 462635, perf_period = 1, ...
[23:32:20.065286422] (+0.000004031) raw_syscalls:sys_exit: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810231D8, perf_tid = 26155, perf_pid = 26155, perf_id = 462639, perf_period = 1, ...
- Or run tracecompass and open the CTF data ;-)
Changes are also reachable in here:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/ctf
thanks,
jirka
[1] Common Trace Format - http://www.efficios.com/ctf
[2] babeltrace - http://www.efficios.com/babeltrace
[3] Trace compass - http://projects.eclipse.org/projects/tools.tracecompass
[4] screenshots - http://people.redhat.com/~jolsa/tracecompass-perf/
---
He Kuang (1):
perf data: Show error message when conversion failed
Jiri Olsa (3):
perf data: Enable stream flush within processing
perf data: Add support for setting ordered_events queue size
tools lib traceevent: Add alias field to struct format_field
Sebastian Andrzej Siewior (1):
perf data: Switch to multiple cpu stream files
Wang Nan (2):
perf data: Fix duplicate field names and avoid reserved keywords
perf data: Fix signess of value
tools/lib/traceevent/event-parse.c | 6 +-
tools/lib/traceevent/event-parse.h | 1 +
tools/perf/util/data-convert-bt.c | 410 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------
3 files changed, 373 insertions(+), 44 deletions(-)
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH 1/7] perf data: Show error message when conversion failed
2015-04-18 15:50 [PATCHv6 0/7] perf tools: Add perf data CTF conversion Jiri Olsa
@ 2015-04-18 15:50 ` Jiri Olsa
2015-05-06 2:55 ` [tip:perf/core] " tip-bot for He Kuang
2015-04-18 15:50 ` [PATCH 2/7] perf data: Switch to multiple cpu stream files Jiri Olsa
` (6 subsequent siblings)
7 siblings, 1 reply; 29+ messages in thread
From: Jiri Olsa @ 2015-04-18 15:50 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: David Ahern, Frederic Weisbecker, He Kuang, Ingo Molnar,
Jeremie Galarneau, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan, lkml
From: He Kuang <hekuang@huawei.com>
Show message when errors occurred during conversion setup and conversion
process.
Before this patch:
$ ./perf data convert --to-ctf=ctf
$ echo $?
255
After this patch:
$ ./perf data convert --to-ctf=ctf
Error during conversion setup.
Signed-off-by: He Kuang <hekuang@huawei.com>
Link: http://lkml.kernel.org/n/tip-xvhr1vf7zav9kkeo9w1hv4uk@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
tools/perf/util/data-convert-bt.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index dd17c9a32fbc..db4f709af18d 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -837,6 +837,8 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
err = perf_session__process_events(session);
if (!err)
err = bt_ctf_stream_flush(cw->stream);
+ else
+ pr_err("Error during conversion.\n");
fprintf(stderr,
"[ perf data convert: Converted '%s' into CTF data '%s' ]\n",
@@ -847,11 +849,15 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
(double) c.events_size / 1024.0 / 1024.0,
c.events_count);
- /* its all good */
-free_session:
perf_session__delete(session);
+ ctf_writer__cleanup(cw);
+ return err;
+
+free_session:
+ perf_session__delete(session);
free_writer:
ctf_writer__cleanup(cw);
+ pr_err("Error during conversion setup.\n");
return err;
}
--
1.9.3
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 2/7] perf data: Switch to multiple cpu stream files
2015-04-18 15:50 [PATCHv6 0/7] perf tools: Add perf data CTF conversion Jiri Olsa
2015-04-18 15:50 ` [PATCH 1/7] perf data: Show error message when conversion failed Jiri Olsa
@ 2015-04-18 15:50 ` Jiri Olsa
2015-04-20 19:58 ` Arnaldo Carvalho de Melo
2015-05-06 2:57 ` [tip:perf/core] " tip-bot for Sebastian Andrzej Siewior
2015-04-18 15:50 ` [PATCH 3/7] perf data: Enable stream flush within processing Jiri Olsa
` (5 subsequent siblings)
7 siblings, 2 replies; 29+ messages in thread
From: Jiri Olsa @ 2015-04-18 15:50 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: David Ahern, Frederic Weisbecker, He Kuang, Ingo Molnar,
Jeremie Galarneau, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan, lkml
From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Currently we store the data into single data strea/file. The cpu
if data is stored within the event sample. The lttng puts the CPU
number that belongs to the event into the packet context instead
into the event.
This patch makes sure that the trace produce by perf does look the
same way. We now use one stream per-CPU. Having it all in one stream
increased the total size of the resulting file. The test went from
416KiB (with perf_cpu event member) to 24MiB due to the required
(and pointless) flush. With the per-cpu streams the total size went
up to 588KiB.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: http://lkml.kernel.org/n/tip-fco07vxi6yx4m9et2aimm4u0@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
tools/perf/util/data-convert-bt.c | 206 +++++++++++++++++++++++++++++++++-----
1 file changed, 182 insertions(+), 24 deletions(-)
diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index db4f709af18d..0a2d7a5b70f9 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -38,12 +38,20 @@ struct evsel_priv {
struct bt_ctf_event_class *event_class;
};
+#define MAX_CPUS 4096
+
+struct ctf_stream {
+ struct bt_ctf_stream *stream;
+ int cpu;
+};
+
struct ctf_writer {
/* writer primitives */
- struct bt_ctf_writer *writer;
- struct bt_ctf_stream *stream;
- struct bt_ctf_stream_class *stream_class;
- struct bt_ctf_clock *clock;
+ struct bt_ctf_writer *writer;
+ struct ctf_stream **stream;
+ int stream_cnt;
+ struct bt_ctf_stream_class *stream_class;
+ struct bt_ctf_clock *clock;
/* data types */
union {
@@ -346,12 +354,6 @@ static int add_generic_values(struct ctf_writer *cw,
return -1;
}
- if (type & PERF_SAMPLE_CPU) {
- ret = value_set_u32(cw, event, "perf_cpu", sample->cpu);
- if (ret)
- return -1;
- }
-
if (type & PERF_SAMPLE_PERIOD) {
ret = value_set_u64(cw, event, "perf_period", sample->period);
if (ret)
@@ -381,6 +383,113 @@ static int add_generic_values(struct ctf_writer *cw,
return 0;
}
+static int ctf_stream__flush(struct ctf_stream *cs)
+{
+ int err = 0;
+
+ if (cs) {
+ err = bt_ctf_stream_flush(cs->stream);
+ if (err)
+ pr_err("CTF stream %d flush failed\n", cs->cpu);
+
+ pr("Flush stream for cpu %d\n", cs->cpu);
+ }
+
+ return err;
+}
+
+static struct ctf_stream *ctf_stream__create(struct ctf_writer *cw, int cpu)
+{
+ struct ctf_stream *cs;
+ struct bt_ctf_field *pkt_ctx = NULL;
+ struct bt_ctf_field *cpu_field = NULL;
+ struct bt_ctf_stream *stream = NULL;
+ int ret;
+
+ cs = zalloc(sizeof(*cs));
+ if (!cs) {
+ pr_err("Failed to allocate ctf stream\n");
+ return NULL;
+ }
+
+ stream = bt_ctf_writer_create_stream(cw->writer, cw->stream_class);
+ if (!stream) {
+ pr_err("Failed to create CTF stream\n");
+ goto out;
+ }
+
+ pkt_ctx = bt_ctf_stream_get_packet_context(stream);
+ if (!pkt_ctx) {
+ pr_err("Failed to obtain packet context\n");
+ goto out;
+ }
+
+ cpu_field = bt_ctf_field_structure_get_field(pkt_ctx, "cpu_id");
+ bt_ctf_field_put(pkt_ctx);
+ if (!cpu_field) {
+ pr_err("Failed to obtain cpu field\n");
+ goto out;
+ }
+
+ ret = bt_ctf_field_unsigned_integer_set_value(cpu_field, (u32) cpu);
+ if (ret) {
+ pr_err("Failed to update CPU number\n");
+ goto out;
+ }
+
+ bt_ctf_field_put(cpu_field);
+
+ cs->cpu = cpu;
+ cs->stream = stream;
+ return cs;
+
+out:
+ if (cpu_field)
+ bt_ctf_field_put(cpu_field);
+ if (stream)
+ bt_ctf_stream_put(stream);
+
+ free(cs);
+ return NULL;
+}
+
+static void ctf_stream__delete(struct ctf_stream *cs)
+{
+ if (cs) {
+ bt_ctf_stream_put(cs->stream);
+ free(cs);
+ }
+}
+
+static struct ctf_stream *ctf_stream(struct ctf_writer *cw, int cpu)
+{
+ struct ctf_stream *cs = cw->stream[cpu];
+
+ if (!cs) {
+ cs = ctf_stream__create(cw, cpu);
+ cw->stream[cpu] = cs;
+ }
+
+ return cs;
+}
+
+static int get_sample_cpu(struct ctf_writer *cw, struct perf_sample *sample,
+ struct perf_evsel *evsel)
+{
+ int cpu = 0;
+
+ if (evsel->attr.sample_type & PERF_SAMPLE_CPU)
+ cpu = sample->cpu;
+
+ if (cpu > cw->stream_cnt) {
+ pr_err("Event was recorded for CPU %d, limit is at %d.\n",
+ cpu, cw->stream_cnt);
+ cpu = 0;
+ }
+
+ return cpu;
+}
+
static int process_sample_event(struct perf_tool *tool,
union perf_event *_event __maybe_unused,
struct perf_sample *sample,
@@ -390,6 +499,7 @@ static int process_sample_event(struct perf_tool *tool,
struct convert *c = container_of(tool, struct convert, tool);
struct evsel_priv *priv = evsel->priv;
struct ctf_writer *cw = &c->writer;
+ struct ctf_stream *cs;
struct bt_ctf_event_class *event_class;
struct bt_ctf_event *event;
int ret;
@@ -424,9 +534,12 @@ static int process_sample_event(struct perf_tool *tool,
return -1;
}
- bt_ctf_stream_append_event(cw->stream, event);
+ cs = ctf_stream(cw, get_sample_cpu(cw, sample, evsel));
+ if (cs)
+ bt_ctf_stream_append_event(cs->stream, event);
+
bt_ctf_event_put(event);
- return 0;
+ return cs ? 0 : -1;
}
static int add_tracepoint_fields_types(struct ctf_writer *cw,
@@ -528,9 +641,6 @@ static int add_generic_types(struct ctf_writer *cw, struct perf_evsel *evsel,
if (type & PERF_SAMPLE_STREAM_ID)
ADD_FIELD(event_class, cw->data.u64, "perf_stream_id");
- if (type & PERF_SAMPLE_CPU)
- ADD_FIELD(event_class, cw->data.u32, "perf_cpu");
-
if (type & PERF_SAMPLE_PERIOD)
ADD_FIELD(event_class, cw->data.u64, "perf_period");
@@ -604,6 +714,39 @@ static int setup_events(struct ctf_writer *cw, struct perf_session *session)
return 0;
}
+static int setup_streams(struct ctf_writer *cw, struct perf_session *session)
+{
+ struct ctf_stream **stream;
+ struct perf_header *ph = &session->header;
+ int ncpus;
+
+ /*
+ * Try to get the number of cpus used in the data file,
+ * if not present fallback to the MAX_CPUS.
+ */
+ ncpus = ph->env.nr_cpus_avail ?: MAX_CPUS;
+
+ stream = zalloc(sizeof(*stream) * ncpus);
+ if (!stream) {
+ pr_err("Failed to allocate streams.\n");
+ return -ENOMEM;
+ }
+
+ cw->stream = stream;
+ cw->stream_cnt = ncpus;
+ return 0;
+}
+
+static void free_streams(struct ctf_writer *cw)
+{
+ int cpu;
+
+ for (cpu = 0; cpu < cw->stream_cnt; cpu++)
+ ctf_stream__delete(cw->stream[cpu]);
+
+ free(cw->stream);
+}
+
static int ctf_writer__setup_env(struct ctf_writer *cw,
struct perf_session *session)
{
@@ -713,7 +856,7 @@ static void ctf_writer__cleanup(struct ctf_writer *cw)
ctf_writer__cleanup_data(cw);
bt_ctf_clock_put(cw->clock);
- bt_ctf_stream_put(cw->stream);
+ free_streams(cw);
bt_ctf_stream_class_put(cw->stream_class);
bt_ctf_writer_put(cw->writer);
@@ -725,8 +868,9 @@ static int ctf_writer__init(struct ctf_writer *cw, const char *path)
{
struct bt_ctf_writer *writer;
struct bt_ctf_stream_class *stream_class;
- struct bt_ctf_stream *stream;
struct bt_ctf_clock *clock;
+ struct bt_ctf_field_type *pkt_ctx_type;
+ int ret;
/* CTF writer */
writer = bt_ctf_writer_create(path);
@@ -767,14 +911,15 @@ static int ctf_writer__init(struct ctf_writer *cw, const char *path)
if (ctf_writer__init_data(cw))
goto err_cleanup;
- /* CTF stream instance */
- stream = bt_ctf_writer_create_stream(writer, stream_class);
- if (!stream) {
- pr("Failed to create CTF stream.\n");
+ /* Add cpu_id for packet context */
+ pkt_ctx_type = bt_ctf_stream_class_get_packet_context_type(stream_class);
+ if (!pkt_ctx_type)
goto err_cleanup;
- }
- cw->stream = stream;
+ ret = bt_ctf_field_type_structure_add_field(pkt_ctx_type, cw->data.u32, "cpu_id");
+ bt_ctf_field_type_put(pkt_ctx_type);
+ if (ret)
+ goto err_cleanup;
/* CTF clock writer setup */
if (bt_ctf_writer_add_clock(writer, clock)) {
@@ -791,6 +936,16 @@ err:
return -1;
}
+static int ctf_writer__flush_streams(struct ctf_writer *cw)
+{
+ int cpu, ret = 0;
+
+ for (cpu = 0; cpu < cw->stream_cnt && !ret; cpu++)
+ ret = ctf_stream__flush(cw->stream[cpu]);
+
+ return ret;
+}
+
int bt_convert__perf2ctf(const char *input, const char *path, bool force)
{
struct perf_session *session;
@@ -834,9 +989,12 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
if (setup_events(cw, session))
goto free_session;
+ if (setup_streams(cw, session))
+ goto free_session;
+
err = perf_session__process_events(session);
if (!err)
- err = bt_ctf_stream_flush(cw->stream);
+ err = ctf_writer__flush_streams(cw);
else
pr_err("Error during conversion.\n");
--
1.9.3
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 3/7] perf data: Enable stream flush within processing
2015-04-18 15:50 [PATCHv6 0/7] perf tools: Add perf data CTF conversion Jiri Olsa
2015-04-18 15:50 ` [PATCH 1/7] perf data: Show error message when conversion failed Jiri Olsa
2015-04-18 15:50 ` [PATCH 2/7] perf data: Switch to multiple cpu stream files Jiri Olsa
@ 2015-04-18 15:50 ` Jiri Olsa
2015-05-06 2:57 ` [tip:perf/core] " tip-bot for Jiri Olsa
2015-04-18 15:50 ` [PATCH 4/7] perf data: Add support for setting ordered_events queue size Jiri Olsa
` (4 subsequent siblings)
7 siblings, 1 reply; 29+ messages in thread
From: Jiri Olsa @ 2015-04-18 15:50 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: David Ahern, Frederic Weisbecker, He Kuang, Ingo Molnar,
Jeremie Galarneau, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan, lkml
For big data files the size of data allocated for stream
instance could get really high. It's needed to flush the
data out of the stream once in a while.
Unfortunately there's no size indication in the stream
object, so we govern the flush based on the number of
stored events. Current flush limit is set ot 100000
events.
Acked-by: Namhyung Kim <namhyung@kernel.org>
Reviewed-by: David Ahern <dsahern@gmail.com>
Link: http://lkml.kernel.org/n/tip-4tlh4s2u3xnxr76d1w7qmckb@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
tools/perf/util/data-convert-bt.c | 26 ++++++++++++++++++++++++--
1 file changed, 24 insertions(+), 2 deletions(-)
diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index 0a2d7a5b70f9..4f942b37067b 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -43,6 +43,7 @@ struct evsel_priv {
struct ctf_stream {
struct bt_ctf_stream *stream;
int cpu;
+ u32 count;
};
struct ctf_writer {
@@ -392,7 +393,10 @@ static int ctf_stream__flush(struct ctf_stream *cs)
if (err)
pr_err("CTF stream %d flush failed\n", cs->cpu);
- pr("Flush stream for cpu %d\n", cs->cpu);
+ pr("Flush stream for cpu %d (%u samples)\n",
+ cs->cpu, cs->count);
+
+ cs->count = 0;
}
return err;
@@ -490,6 +494,19 @@ static int get_sample_cpu(struct ctf_writer *cw, struct perf_sample *sample,
return cpu;
}
+#define STREAM_FLUSH_COUNT 100000
+
+/*
+ * Currently we have no other way to determine the
+ * time for the stream flush other than keep track
+ * of the number of events and check it against
+ * threshold.
+ */
+static bool is_flush_needed(struct ctf_stream *cs)
+{
+ return cs->count >= STREAM_FLUSH_COUNT;
+}
+
static int process_sample_event(struct perf_tool *tool,
union perf_event *_event __maybe_unused,
struct perf_sample *sample,
@@ -535,8 +552,13 @@ static int process_sample_event(struct perf_tool *tool,
}
cs = ctf_stream(cw, get_sample_cpu(cw, sample, evsel));
- if (cs)
+ if (cs) {
+ if (is_flush_needed(cs))
+ ctf_stream__flush(cs);
+
+ cs->count++;
bt_ctf_stream_append_event(cs->stream, event);
+ }
bt_ctf_event_put(event);
return cs ? 0 : -1;
--
1.9.3
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 4/7] perf data: Add support for setting ordered_events queue size
2015-04-18 15:50 [PATCHv6 0/7] perf tools: Add perf data CTF conversion Jiri Olsa
` (2 preceding siblings ...)
2015-04-18 15:50 ` [PATCH 3/7] perf data: Enable stream flush within processing Jiri Olsa
@ 2015-04-18 15:50 ` Jiri Olsa
2015-05-06 2:57 ` [tip:perf/core] " tip-bot for Jiri Olsa
2015-04-18 15:50 ` [PATCH 5/7] tools lib traceevent: Add alias field to struct format_field Jiri Olsa
` (3 subsequent siblings)
7 siblings, 1 reply; 29+ messages in thread
From: Jiri Olsa @ 2015-04-18 15:50 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: David Ahern, Frederic Weisbecker, He Kuang, Ingo Molnar,
Jeremie Galarneau, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan, lkml
Adding support to limit the size of ordered_events queue,
so we could control allocation size of perf data files
without proper finished round events.
Acked-by: Namhyung Kim <namhyung@kernel.org>
Reviewed-by: David Ahern <dsahern@gmail.com>
Link: http://lkml.kernel.org/n/tip-lo2z8stxkj6n42065zahe39i@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
tools/perf/util/data-convert-bt.c | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index 4f942b37067b..b35c8d6c291d 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -74,6 +74,9 @@ struct convert {
u64 events_size;
u64 events_count;
+
+ /* Ordered events configured queue size. */
+ u64 queue_size;
};
static int value_set(struct bt_ctf_field_type *type,
@@ -968,6 +971,18 @@ static int ctf_writer__flush_streams(struct ctf_writer *cw)
return ret;
}
+static int convert__config(const char *var, const char *value, void *cb)
+{
+ struct convert *c = cb;
+
+ if (!strcmp(var, "convert.queue-size")) {
+ c->queue_size = perf_config_u64(var, value);
+ return 0;
+ }
+
+ return perf_default_config(var, value, cb);
+}
+
int bt_convert__perf2ctf(const char *input, const char *path, bool force)
{
struct perf_session *session;
@@ -994,6 +1009,8 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
struct ctf_writer *cw = &c.writer;
int err = -1;
+ perf_config(convert__config, &c);
+
/* CTF writer */
if (ctf_writer__init(cw, path))
return -1;
@@ -1003,6 +1020,11 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
if (!session)
goto free_writer;
+ if (c.queue_size) {
+ ordered_events__set_alloc_size(&session->ordered_events,
+ c.queue_size);
+ }
+
/* CTF writer env/clock setup */
if (ctf_writer__setup_env(cw, session))
goto free_session;
--
1.9.3
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 5/7] tools lib traceevent: Add alias field to struct format_field
2015-04-18 15:50 [PATCHv6 0/7] perf tools: Add perf data CTF conversion Jiri Olsa
` (3 preceding siblings ...)
2015-04-18 15:50 ` [PATCH 4/7] perf data: Add support for setting ordered_events queue size Jiri Olsa
@ 2015-04-18 15:50 ` Jiri Olsa
2015-04-20 14:24 ` Steven Rostedt
2015-05-06 2:55 ` [tip:perf/core] " tip-bot for Jiri Olsa
2015-04-18 15:50 ` [PATCH 6/7] perf data: Fix duplicate field names and avoid reserved keywords Jiri Olsa
` (2 subsequent siblings)
7 siblings, 2 replies; 29+ messages in thread
From: Jiri Olsa @ 2015-04-18 15:50 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: David Ahern, Frederic Weisbecker, He Kuang, Ingo Molnar,
Jeremie Galarneau, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan, lkml
Introduce an 'alias' field to 'struct format_field' to be able
to use alternative name for the field.
It is initialized with same string pointer as 'name' field.
The free logic checks the 'alias' pointer being reset by user
and frees it.
This will be handy when converting data into CTF, where each
field within event needs to have a unique name (while this
is not required for tracepoint). Converter can easily assign
unique name into the format_field struct.
Suggested-by: Steven Rostedt <rostedt@goodmis.org>
Link: http://lkml.kernel.org/n/tip-qwyq8blnfkg6s5vlbrvn1en3@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
tools/lib/traceevent/event-parse.c | 6 ++++--
tools/lib/traceevent/event-parse.h | 1 +
2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/tools/lib/traceevent/event-parse.c b/tools/lib/traceevent/event-parse.c
index 12a7e2a40c89..1ec56b459bc0 100644
--- a/tools/lib/traceevent/event-parse.c
+++ b/tools/lib/traceevent/event-parse.c
@@ -1387,7 +1387,7 @@ static int event_read_fields(struct event_format *event, struct format_field **f
do_warning_event(event, "%s: no type found", __func__);
goto fail;
}
- field->name = last_token;
+ field->name = field->alias = last_token;
if (test_type(type, EVENT_OP))
goto fail;
@@ -1469,7 +1469,7 @@ static int event_read_fields(struct event_format *event, struct format_field **f
size_dynamic = type_size(field->name);
free_token(field->name);
strcat(field->type, brackets);
- field->name = token;
+ field->name = field->alias = token;
type = read_token(&token);
} else {
char *new_type;
@@ -6444,6 +6444,8 @@ void pevent_ref(struct pevent *pevent)
void pevent_free_format_field(struct format_field *field)
{
free(field->type);
+ if (field->alias != field->name)
+ free(field->alias);
free(field->name);
free(field);
}
diff --git a/tools/lib/traceevent/event-parse.h b/tools/lib/traceevent/event-parse.h
index 86a5839fb048..063b1971eb35 100644
--- a/tools/lib/traceevent/event-parse.h
+++ b/tools/lib/traceevent/event-parse.h
@@ -191,6 +191,7 @@ struct format_field {
struct event_format *event;
char *type;
char *name;
+ char *alias;
int offset;
int size;
unsigned int arraylen;
--
1.9.3
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 6/7] perf data: Fix duplicate field names and avoid reserved keywords
2015-04-18 15:50 [PATCHv6 0/7] perf tools: Add perf data CTF conversion Jiri Olsa
` (4 preceding siblings ...)
2015-04-18 15:50 ` [PATCH 5/7] tools lib traceevent: Add alias field to struct format_field Jiri Olsa
@ 2015-04-18 15:50 ` Jiri Olsa
2015-04-18 18:56 ` Jérémie Galarneau
2015-05-06 2:57 ` [tip:perf/core] " tip-bot for Wang Nan
2015-04-18 15:50 ` [PATCH 7/7] perf data: Fix signess of value Jiri Olsa
2015-04-20 20:05 ` [PATCHv6 0/7] perf tools: Add perf data CTF conversion Arnaldo Carvalho de Melo
7 siblings, 2 replies; 29+ messages in thread
From: Jiri Olsa @ 2015-04-18 15:50 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: David Ahern, Frederic Weisbecker, He Kuang, Ingo Molnar,
Jeremie Galarneau, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan, lkml
From: Wang Nan <wangnan0@huawei.com>
Some parameters of syscall tracepoints named as 'nr', 'event', etc.
When dealing with them, perf convert to ctf meets some problem:
1. If a parameter with name 'nr', it will duplicate syscall's
common field 'nr'. One such syscall is io_submit().
2. If a parameter with name 'event', it is denied to be inserted
because 'event' is a babeltrace keywork. One such syscall is
epoll_ctl.
This patch appends '_dupl_X' suffix to avoid problem 1, prepend a '_'
prefix to avoid problem 2.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-4tlh4s2u3xnxr76d1w7qmckb@git.kernel.org
[ changed to use format_file::alias ]
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
tools/perf/util/data-convert-bt.c | 86 +++++++++++++++++++++++++++++++++++++--
1 file changed, 82 insertions(+), 4 deletions(-)
diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index b35c8d6c291d..8eda4ed628e7 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -14,6 +14,7 @@
#include <babeltrace/ctf-writer/event.h>
#include <babeltrace/ctf-writer/event-types.h>
#include <babeltrace/ctf-writer/event-fields.h>
+#include <babeltrace/ctf-ir/utils.h>
#include <babeltrace/ctf/events.h>
#include <traceevent/event-parse.h>
#include "asm/bug.h"
@@ -184,6 +185,7 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
unsigned int len;
int ret;
+ name = fmtf->alias;
offset = fmtf->offset;
len = fmtf->size;
if (flags & FIELD_IS_STRING)
@@ -567,6 +569,82 @@ static int process_sample_event(struct perf_tool *tool,
return cs ? 0 : -1;
}
+/* If dup < 0, add a prefix. Else, add _dupl_X suffix. */
+static char *change_name(char *name, char *orig_name, int dup)
+{
+ char *new_name = NULL;
+ size_t len;
+
+ if (!name)
+ name = orig_name;
+
+ if (dup >= 10)
+ goto out;
+ /*
+ * Add '_' prefix to potential keywork. According to
+ * Mathieu Desnoyers (https://lkml.org/lkml/2015/1/23/652),
+ * futher CTF spec updating may require us to use '$'.
+ */
+ if (dup < 0)
+ len = strlen(name) + sizeof("_");
+ else
+ len = strlen(orig_name) + sizeof("_dupl_X");
+
+ new_name = malloc(len);
+ if (!new_name)
+ goto out;
+
+ if (dup < 0)
+ snprintf(new_name, len, "_%s", name);
+ else
+ snprintf(new_name, len, "%s_dupl_%d", orig_name, dup);
+
+out:
+ if (name != orig_name)
+ free(name);
+ return new_name;
+}
+
+static int event_class_add_field(struct bt_ctf_event_class *event_class,
+ struct bt_ctf_field_type *type,
+ struct format_field *field)
+{
+ struct bt_ctf_field_type *t = NULL;
+ char *name;
+ int dup = 1;
+ int ret;
+
+ /* alias was already assigned */
+ if (field->alias != field->name)
+ return bt_ctf_event_class_add_field(event_class, type,
+ (char *)field->alias);
+
+ name = field->name;
+
+ /* If 'name' is a keywork, add prefix. */
+ if (bt_ctf_validate_identifier(name))
+ name = change_name(name, field->name, -1);
+
+ if (!name) {
+ pr_err("Failed to fix invalid identifier.");
+ return -1;
+ }
+ while ((t = bt_ctf_event_class_get_field_by_name(event_class, name))) {
+ bt_ctf_field_type_put(t);
+ name = change_name(name, field->name, dup++);
+ if (!name) {
+ pr_err("Failed to create dup name for '%s'\n", field->name);
+ return -1;
+ }
+ }
+
+ ret = bt_ctf_event_class_add_field(event_class, type, name);
+ if (!ret)
+ field->alias = name;
+
+ return ret;
+}
+
static int add_tracepoint_fields_types(struct ctf_writer *cw,
struct format_field *fields,
struct bt_ctf_event_class *event_class)
@@ -595,14 +673,14 @@ static int add_tracepoint_fields_types(struct ctf_writer *cw,
if (flags & FIELD_IS_ARRAY)
type = bt_ctf_field_type_array_create(type, field->arraylen);
- ret = bt_ctf_event_class_add_field(event_class, type,
- field->name);
+ ret = event_class_add_field(event_class, type, field);
if (flags & FIELD_IS_ARRAY)
bt_ctf_field_type_put(type);
if (ret) {
- pr_err("Failed to add field '%s\n", field->name);
+ pr_err("Failed to add field '%s': %d\n",
+ field->name, ret);
return -1;
}
}
@@ -646,7 +724,7 @@ static int add_generic_types(struct ctf_writer *cw, struct perf_evsel *evsel,
do { \
pr2(" field '%s'\n", n); \
if (bt_ctf_event_class_add_field(cl, t, n)) { \
- pr_err("Failed to add field '%s;\n", n); \
+ pr_err("Failed to add field '%s';\n", n); \
return -1; \
} \
} while (0)
--
1.9.3
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 7/7] perf data: Fix signess of value
2015-04-18 15:50 [PATCHv6 0/7] perf tools: Add perf data CTF conversion Jiri Olsa
` (5 preceding siblings ...)
2015-04-18 15:50 ` [PATCH 6/7] perf data: Fix duplicate field names and avoid reserved keywords Jiri Olsa
@ 2015-04-18 15:50 ` Jiri Olsa
2015-04-20 21:23 ` Arnaldo Carvalho de Melo
2015-05-06 2:58 ` [tip:perf/core] perf data: Fix signedness " tip-bot for Wang Nan
2015-04-20 20:05 ` [PATCHv6 0/7] perf tools: Add perf data CTF conversion Arnaldo Carvalho de Melo
7 siblings, 2 replies; 29+ messages in thread
From: Jiri Olsa @ 2015-04-18 15:50 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: David Ahern, Frederic Weisbecker, He Kuang, Ingo Molnar,
Jeremie Galarneau, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan, lkml
From: Wang Nan <wangnan0@huawei.com>
When converting int values, perf first extractes it to a ulonglong, then
feeds it to babeltrace as a signed value. For negative 32 bit values
(for example, return values of failed syscalls), the extracted data
should be something like 0xfffffffe (-2). It becomes a large int64
value. Babeltrace denies to insert it with
bt_ctf_field_signed_integer_set_value() because it is larger than
0x7fffffff, the largest positive value a signed 32 bit int can be.
This patch introduces adjust_signess(), which fills high bits of
ulonglong with 1 if the value is negative.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-bq2fguw8pm8s5rr741mmgias@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
tools/perf/util/data-convert-bt.c | 64 +++++++++++++++++++++++++++++++--------
1 file changed, 52 insertions(+), 12 deletions(-)
diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index 8eda4ed628e7..977cc3f98d8f 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -166,6 +166,43 @@ get_tracepoint_field_type(struct ctf_writer *cw, struct format_field *field)
return cw->data.u32;
}
+static unsigned long long adjust_signess(unsigned long long value_int, int size)
+{
+ unsigned long long value_mask;
+
+ /*
+ * value_mask = (1 << (size * 8 - 1)) - 1.
+ * Directly set value_mask for code readers.
+ */
+ switch (size) {
+ case 1:
+ value_mask = 0x7fULL;
+ break;
+ case 2:
+ value_mask = 0x7fffULL;
+ break;
+ case 4:
+ value_mask = 0x7fffffffULL;
+ break;
+ case 8:
+ /*
+ * For 64 bit value, return it self. There is no need
+ * to fill high bit.
+ */
+ /* Fall through */
+ default:
+ /* BUG! */
+ return value_int;
+ }
+
+ /* If it is a positive value, don't adjust. */
+ if ((value_int & (~0ULL - value_mask)) == 0)
+ return value_int;
+
+ /* Fill upper part of value_int with 1 to make it a negative long long. */
+ return (value_int & value_mask) | ~value_mask;
+}
+
static int add_tracepoint_field_value(struct ctf_writer *cw,
struct bt_ctf_event_class *event_class,
struct bt_ctf_event *event,
@@ -177,7 +214,6 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
struct bt_ctf_field *field;
const char *name = fmtf->name;
void *data = sample->raw_data;
- unsigned long long value_int;
unsigned long flags = fmtf->flags;
unsigned int n_items;
unsigned int i;
@@ -222,11 +258,6 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
type = get_tracepoint_field_type(cw, fmtf);
for (i = 0; i < n_items; i++) {
- if (!(flags & FIELD_IS_STRING))
- value_int = pevent_read_number(
- fmtf->event->pevent,
- data + offset + i * len, len);
-
if (flags & FIELD_IS_ARRAY)
field = bt_ctf_field_array_get_field(array_field, i);
else
@@ -240,12 +271,21 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
if (flags & FIELD_IS_STRING)
ret = bt_ctf_field_string_set_value(field,
data + offset + i * len);
- else if (!(flags & FIELD_IS_SIGNED))
- ret = bt_ctf_field_unsigned_integer_set_value(
- field, value_int);
- else
- ret = bt_ctf_field_signed_integer_set_value(
- field, value_int);
+ else {
+ unsigned long long value_int;
+
+ value_int = pevent_read_number(
+ fmtf->event->pevent,
+ data + offset + i * len, len);
+
+ if (!(flags & FIELD_IS_SIGNED))
+ ret = bt_ctf_field_unsigned_integer_set_value(
+ field, value_int);
+ else
+ ret = bt_ctf_field_signed_integer_set_value(
+ field, adjust_signess(value_int, len));
+ }
+
if (ret) {
pr_err("failed to set file value %s\n", name);
goto err_put_field;
--
1.9.3
^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH 6/7] perf data: Fix duplicate field names and avoid reserved keywords
2015-04-18 15:50 ` [PATCH 6/7] perf data: Fix duplicate field names and avoid reserved keywords Jiri Olsa
@ 2015-04-18 18:56 ` Jérémie Galarneau
2015-04-18 20:45 ` Jiri Olsa
2015-05-06 2:57 ` [tip:perf/core] " tip-bot for Wang Nan
1 sibling, 1 reply; 29+ messages in thread
From: Jérémie Galarneau @ 2015-04-18 18:56 UTC (permalink / raw)
To: Jiri Olsa
Cc: Arnaldo Carvalho de Melo, David Ahern, Frederic Weisbecker,
He Kuang, Ingo Molnar, Jeremie Galarneau, Namhyung Kim,
Paul Mackerras, Peter Zijlstra, Sebastian Andrzej Siewior,
Tom Zanussi, Wang Nan, lkml
On Sat, Apr 18, 2015 at 11:50 AM, Jiri Olsa <jolsa@kernel.org> wrote:
> From: Wang Nan <wangnan0@huawei.com>
>
> Some parameters of syscall tracepoints named as 'nr', 'event', etc.
> When dealing with them, perf convert to ctf meets some problem:
>
> 1. If a parameter with name 'nr', it will duplicate syscall's
> common field 'nr'. One such syscall is io_submit().
>
> 2. If a parameter with name 'event', it is denied to be inserted
> because 'event' is a babeltrace keywork. One such syscall is
> epoll_ctl.
Minor nitpick: "event" is a reserved keyword in the CTF specification
[1], the rule is only enforced by Babeltrace. Also, keywork ->
keyword.
Jérémie
[1] http://diamon.org/docs/ctf/v1.8.2/#specC.1.2
>
> This patch appends '_dupl_X' suffix to avoid problem 1, prepend a '_'
> prefix to avoid problem 2.
>
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Link: http://lkml.kernel.org/n/tip-4tlh4s2u3xnxr76d1w7qmckb@git.kernel.org
> [ changed to use format_file::alias ]
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
> tools/perf/util/data-convert-bt.c | 86 +++++++++++++++++++++++++++++++++++++--
> 1 file changed, 82 insertions(+), 4 deletions(-)
>
> diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
> index b35c8d6c291d..8eda4ed628e7 100644
> --- a/tools/perf/util/data-convert-bt.c
> +++ b/tools/perf/util/data-convert-bt.c
> @@ -14,6 +14,7 @@
> #include <babeltrace/ctf-writer/event.h>
> #include <babeltrace/ctf-writer/event-types.h>
> #include <babeltrace/ctf-writer/event-fields.h>
> +#include <babeltrace/ctf-ir/utils.h>
> #include <babeltrace/ctf/events.h>
> #include <traceevent/event-parse.h>
> #include "asm/bug.h"
> @@ -184,6 +185,7 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
> unsigned int len;
> int ret;
>
> + name = fmtf->alias;
> offset = fmtf->offset;
> len = fmtf->size;
> if (flags & FIELD_IS_STRING)
> @@ -567,6 +569,82 @@ static int process_sample_event(struct perf_tool *tool,
> return cs ? 0 : -1;
> }
>
> +/* If dup < 0, add a prefix. Else, add _dupl_X suffix. */
> +static char *change_name(char *name, char *orig_name, int dup)
> +{
> + char *new_name = NULL;
> + size_t len;
> +
> + if (!name)
> + name = orig_name;
> +
> + if (dup >= 10)
> + goto out;
> + /*
> + * Add '_' prefix to potential keywork. According to
> + * Mathieu Desnoyers (https://lkml.org/lkml/2015/1/23/652),
> + * futher CTF spec updating may require us to use '$'.
> + */
> + if (dup < 0)
> + len = strlen(name) + sizeof("_");
> + else
> + len = strlen(orig_name) + sizeof("_dupl_X");
> +
> + new_name = malloc(len);
> + if (!new_name)
> + goto out;
> +
> + if (dup < 0)
> + snprintf(new_name, len, "_%s", name);
> + else
> + snprintf(new_name, len, "%s_dupl_%d", orig_name, dup);
> +
> +out:
> + if (name != orig_name)
> + free(name);
> + return new_name;
> +}
> +
> +static int event_class_add_field(struct bt_ctf_event_class *event_class,
> + struct bt_ctf_field_type *type,
> + struct format_field *field)
> +{
> + struct bt_ctf_field_type *t = NULL;
> + char *name;
> + int dup = 1;
> + int ret;
> +
> + /* alias was already assigned */
> + if (field->alias != field->name)
> + return bt_ctf_event_class_add_field(event_class, type,
> + (char *)field->alias);
> +
> + name = field->name;
> +
> + /* If 'name' is a keywork, add prefix. */
> + if (bt_ctf_validate_identifier(name))
> + name = change_name(name, field->name, -1);
> +
> + if (!name) {
> + pr_err("Failed to fix invalid identifier.");
> + return -1;
> + }
> + while ((t = bt_ctf_event_class_get_field_by_name(event_class, name))) {
> + bt_ctf_field_type_put(t);
> + name = change_name(name, field->name, dup++);
> + if (!name) {
> + pr_err("Failed to create dup name for '%s'\n", field->name);
> + return -1;
> + }
> + }
> +
> + ret = bt_ctf_event_class_add_field(event_class, type, name);
> + if (!ret)
> + field->alias = name;
> +
> + return ret;
> +}
> +
> static int add_tracepoint_fields_types(struct ctf_writer *cw,
> struct format_field *fields,
> struct bt_ctf_event_class *event_class)
> @@ -595,14 +673,14 @@ static int add_tracepoint_fields_types(struct ctf_writer *cw,
> if (flags & FIELD_IS_ARRAY)
> type = bt_ctf_field_type_array_create(type, field->arraylen);
>
> - ret = bt_ctf_event_class_add_field(event_class, type,
> - field->name);
> + ret = event_class_add_field(event_class, type, field);
>
> if (flags & FIELD_IS_ARRAY)
> bt_ctf_field_type_put(type);
>
> if (ret) {
> - pr_err("Failed to add field '%s\n", field->name);
> + pr_err("Failed to add field '%s': %d\n",
> + field->name, ret);
> return -1;
> }
> }
> @@ -646,7 +724,7 @@ static int add_generic_types(struct ctf_writer *cw, struct perf_evsel *evsel,
> do { \
> pr2(" field '%s'\n", n); \
> if (bt_ctf_event_class_add_field(cl, t, n)) { \
> - pr_err("Failed to add field '%s;\n", n); \
> + pr_err("Failed to add field '%s';\n", n); \
> return -1; \
> } \
> } while (0)
> --
> 1.9.3
>
--
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 6/7] perf data: Fix duplicate field names and avoid reserved keywords
2015-04-18 18:56 ` Jérémie Galarneau
@ 2015-04-18 20:45 ` Jiri Olsa
2015-04-20 14:02 ` Arnaldo Carvalho de Melo
0 siblings, 1 reply; 29+ messages in thread
From: Jiri Olsa @ 2015-04-18 20:45 UTC (permalink / raw)
To: Jérémie Galarneau
Cc: Jiri Olsa, Arnaldo Carvalho de Melo, David Ahern,
Frederic Weisbecker, He Kuang, Ingo Molnar, Jeremie Galarneau,
Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan, lkml
On Sat, Apr 18, 2015 at 02:56:00PM -0400, Jérémie Galarneau wrote:
> On Sat, Apr 18, 2015 at 11:50 AM, Jiri Olsa <jolsa@kernel.org> wrote:
> > From: Wang Nan <wangnan0@huawei.com>
> >
> > Some parameters of syscall tracepoints named as 'nr', 'event', etc.
> > When dealing with them, perf convert to ctf meets some problem:
> >
> > 1. If a parameter with name 'nr', it will duplicate syscall's
> > common field 'nr'. One such syscall is io_submit().
> >
> > 2. If a parameter with name 'event', it is denied to be inserted
> > because 'event' is a babeltrace keywork. One such syscall is
> > epoll_ctl.
>
> Minor nitpick: "event" is a reserved keyword in the CTF specification
> [1], the rule is only enforced by Babeltrace. Also, keywork ->
> keyword.
thanks,
Arnaldo, any chance you could change it? (with ack from
He Kuang of course), or do you need a respin..
jirka
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 6/7] perf data: Fix duplicate field names and avoid reserved keywords
2015-04-18 20:45 ` Jiri Olsa
@ 2015-04-20 14:02 ` Arnaldo Carvalho de Melo
0 siblings, 0 replies; 29+ messages in thread
From: Arnaldo Carvalho de Melo @ 2015-04-20 14:02 UTC (permalink / raw)
To: Jiri Olsa
Cc: Jérémie Galarneau, Jiri Olsa, David Ahern,
Frederic Weisbecker, He Kuang, Ingo Molnar, Jeremie Galarneau,
Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan, lkml
Em Sat, Apr 18, 2015 at 10:45:36PM +0200, Jiri Olsa escreveu:
> On Sat, Apr 18, 2015 at 02:56:00PM -0400, Jérémie Galarneau wrote:
> > On Sat, Apr 18, 2015 at 11:50 AM, Jiri Olsa <jolsa@kernel.org> wrote:
> > > From: Wang Nan <wangnan0@huawei.com>
> > >
> > > Some parameters of syscall tracepoints named as 'nr', 'event', etc.
> > > When dealing with them, perf convert to ctf meets some problem:
> > >
> > > 1. If a parameter with name 'nr', it will duplicate syscall's
> > > common field 'nr'. One such syscall is io_submit().
> > >
> > > 2. If a parameter with name 'event', it is denied to be inserted
> > > because 'event' is a babeltrace keywork. One such syscall is
> > > epoll_ctl.
> >
> > Minor nitpick: "event" is a reserved keyword in the CTF specification
> > [1], the rule is only enforced by Babeltrace. Also, keywork ->
> > keyword.
>
> thanks,
>
> Arnaldo, any chance you could change it? (with ack from
I'll do it
> He Kuang of course), or do you need a respin..
>
> jirka
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 5/7] tools lib traceevent: Add alias field to struct format_field
2015-04-18 15:50 ` [PATCH 5/7] tools lib traceevent: Add alias field to struct format_field Jiri Olsa
@ 2015-04-20 14:24 ` Steven Rostedt
2015-05-06 2:55 ` [tip:perf/core] " tip-bot for Jiri Olsa
1 sibling, 0 replies; 29+ messages in thread
From: Steven Rostedt @ 2015-04-20 14:24 UTC (permalink / raw)
To: Jiri Olsa
Cc: Arnaldo Carvalho de Melo, David Ahern, Frederic Weisbecker,
He Kuang, Ingo Molnar, Jeremie Galarneau, Namhyung Kim,
Paul Mackerras, Peter Zijlstra, Sebastian Andrzej Siewior,
Tom Zanussi, Wang Nan, lkml
On Sat, Apr 18, 2015 at 05:50:18PM +0200, Jiri Olsa wrote:
> Introduce an 'alias' field to 'struct format_field' to be able
> to use alternative name for the field.
>
> It is initialized with same string pointer as 'name' field.
> The free logic checks the 'alias' pointer being reset by user
> and frees it.
>
> This will be handy when converting data into CTF, where each
> field within event needs to have a unique name (while this
> is not required for tracepoint). Converter can easily assign
> unique name into the format_field struct.
>
> Suggested-by: Steven Rostedt <rostedt@goodmis.org>
> Link: http://lkml.kernel.org/n/tip-qwyq8blnfkg6s5vlbrvn1en3@git.kernel.org
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acke-by: Steven Rostedt <rostedt@goodmis.org>
-- Steve
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 2/7] perf data: Switch to multiple cpu stream files
2015-04-18 15:50 ` [PATCH 2/7] perf data: Switch to multiple cpu stream files Jiri Olsa
@ 2015-04-20 19:58 ` Arnaldo Carvalho de Melo
2015-04-20 20:01 ` Arnaldo Carvalho de Melo
2015-04-20 20:19 ` Jiri Olsa
2015-05-06 2:57 ` [tip:perf/core] " tip-bot for Sebastian Andrzej Siewior
1 sibling, 2 replies; 29+ messages in thread
From: Arnaldo Carvalho de Melo @ 2015-04-20 19:58 UTC (permalink / raw)
To: Jiri Olsa
Cc: David Ahern, Frederic Weisbecker, He Kuang, Ingo Molnar,
Jeremie Galarneau, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan, lkml
Em Sat, Apr 18, 2015 at 05:50:15PM +0200, Jiri Olsa escreveu:
> From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
>
> Currently we store the data into single data strea/file. The cpu
> if data is stored within the event sample. The lttng puts the CPU
> number that belongs to the event into the packet context instead
> into the event.
>
> This patch makes sure that the trace produce by perf does look the
> same way. We now use one stream per-CPU. Having it all in one stream
> increased the total size of the resulting file. The test went from
> 416KiB (with perf_cpu event member) to 24MiB due to the required
> (and pointless) flush. With the per-cpu streams the total size went
> up to 588KiB.
>
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> Link: http://lkml.kernel.org/n/tip-fco07vxi6yx4m9et2aimm4u0@git.kernel.org
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
With latest babeltrace.git:
[acme@zoo babeltrace]$ git log --oneline | head -1
48d711a204f6 Fix: Missing stdint.h in stream-class.h
But I noticed I have this installed, as well:
[acme@zoo babeltrace]$ rpm -qa | grep babeltr
libbabeltrace-1.2.1-1.fc20.x86_64
[acme@zoo babeltrace]$
But:
[acme@zoo babeltrace]$ ldd ~/bin/perf | grep babeltrace
libbabeltrace-ctf.so.1 => /opt/libbabeltrace/lib/libbabeltrace-ctf.so.1 (0x00007fb52713b000)
libbabeltrace.so.1 => /opt/libbabeltrace/lib/libbabeltrace.so.1 (0x00007fb526889000)
[acme@zoo babeltrace]$
Will try removing the rpm as well...
[acme@zoo linux]$ perf data convert --to-ctf=ctf
perf: event-types.c:1855: bt_ctf_field_type_set_native_byte_order: Assertion `byte_order == 1234 || byte_order == 4321' failed.
Aborted (core dumped)
[acme@zoo linux]$
Core was generated by `perf data convert --to-ctf=ctf'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f843378a877 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-9.fc20.x86_64 libunwind-1.1-3.fc20.x86_64 libuuid-2.24.2-2.fc20.x86_64 nss-softokn-freebl-3.18.0-1.fc20.x86_64 numactl-libs-2.0.9-2.fc20.x86_64 popt-1.16-2.fc20.x86_64 python-libs-2.7.5-15.fc20.x86_64 slang-2.2.4-11.fc20.x86_64 xz-libs-5.1.2-12alpha.fc20.x86_64
(gdb) bt
#0 0x00007f843378a877 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007f843378bf68 in __GI_abort () at abort.c:89
#2 0x00007f84337837d6 in __assert_fail_base (fmt=0x7f84338d0ce8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
assertion=assertion@entry=0x7f8432b3f510 "byte_order == 1234 || byte_order == 4321", file=file@entry=0x7f8432b3f3ba "event-types.c",
line=line@entry=1855, function=function@entry=0x7f8432b3f540 <__PRETTY_FUNCTION__.12860> "bt_ctf_field_type_set_native_byte_order")
at assert.c:92
#3 0x00007f8433783882 in __GI___assert_fail (assertion=assertion@entry=0x7f8432b3f510 "byte_order == 1234 || byte_order == 4321",
file=file@entry=0x7f8432b3f3ba "event-types.c", line=line@entry=1855,
function=function@entry=0x7f8432b3f540 <__PRETTY_FUNCTION__.12860> "bt_ctf_field_type_set_native_byte_order") at assert.c:101
#4 0x00007f8432b313fa in bt_ctf_field_type_set_native_byte_order (type=<optimized out>, byte_order=<optimized out>) at event-types.c:1855
#5 0x00007f8432b32c09 in bt_ctf_stream_class_freeze (stream_class=0x23a5b50) at stream-class.c:478
#6 0x00007f8432b2ce75 in bt_ctf_event_create (event_class=event_class@entry=0x23aabc0) at event.c:324
#7 0x00000000004e71bd in process_sample_event (tool=0x7ffe2413eaa0, _event=<optimized out>, sample=0x7ffe2413e650, evsel=0x23a9eb0,
machine=<optimized out>) at util/data-convert-bt.c:518
#8 0x00000000004b0b35 in perf_evlist__deliver_sample (machine=0x23a92e0, evsel=0x23a9eb0, sample=0x7ffe2413e650, event=0x7f84360962b0,
tool=0x7ffe2413eaa0, evlist=0x23a94d0) at util/session.c:883
#9 machines__deliver_event (file_offset=<optimized out>, tool=<optimized out>, sample=0x7ffe2413e650, event=<optimized out>,
evlist=<optimized out>, machines=<optimized out>) at util/session.c:920
#10 ordered_events__deliver_event (oe=<optimized out>, event=<optimized out>) at util/session.c:108
#11 0x00000000004b4e54 in __ordered_events__flush (oe=0x23a9410) at util/ordered-events.c:207
#12 ordered_events__flush (oe=oe@entry=0x23a9410, how=how@entry=OE_FLUSH__FINAL) at util/ordered-events.c:271
#13 0x00000000004b3ad8 in __perf_session__process_events (file_size=18224, data_size=<optimized out>, data_offset=<optimized out>, session=0x23a9220)
at util/session.c:1404
#14 perf_session__process_events (session=session@entry=0x23a9220) at util/session.c:1422
#15 0x00000000004e81db in bt_convert__perf2ctf (input=<optimized out>, path=0x7ffe241402ff "ctf", force=<optimized out>)
at util/data-convert-bt.c:995
#16 0x00000000004497c5 in cmd_data_convert (argc=<optimized out>, argv=<optimized out>, prefix=<optimized out>) at builtin-data.c:81
#17 0x0000000000472165 in run_builtin (p=p@entry=0x838640 <commands+576>, argc=argc@entry=3, argv=argv@entry=0x7ffe2413f190) at perf.c:370
#18 0x000000000041e660 in handle_internal_command (argv=0x7ffe2413f190, argc=3) at perf.c:429
#19 run_argv (argv=0x7ffe2413ef10, argcp=0x7ffe2413ef1c) at perf.c:473
#20 main (argc=3, argv=0x7ffe2413f190) at perf.c:588
(gdb)
[acme@zoo linux]$ ls -la ctf
total 60
drwxrwx---. 2 acme acme 4096 Abr 20 11:57 .
drwxrwxr-x. 30 acme acme 20480 Abr 20 16:54 ..
-rw-rw----. 1 acme acme 0 Abr 20 16:54 metadata
-rw-rw----. 1 acme acme 32768 Abr 20 16:49 perf_stream_0
[acme@zoo linux]$ babeltrace ctf
[warning] Ill-shapen or missing "/* CTF x.y" header for text-only metadata.
[warning] Unsupported CTF specification version 32547.6313276. Trying anyway.
[error] at line 1: token "": syntax error, unexpected $end
[error] Error creating AST
[warning] Unable to open trace metadata for path "ctf".
[warning] [Context] Cannot open_trace of format ctf at path ctf.
[warning] [Context] cannot open trace "ctf" from ctf for reading.
[error] Cannot open any trace for reading.
[error] opening trace "ctf" for reading.
[error] none of the specified trace paths could be opened.
[acme@zoo linux]$
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 2/7] perf data: Switch to multiple cpu stream files
2015-04-20 19:58 ` Arnaldo Carvalho de Melo
@ 2015-04-20 20:01 ` Arnaldo Carvalho de Melo
2015-04-20 20:19 ` Jiri Olsa
1 sibling, 0 replies; 29+ messages in thread
From: Arnaldo Carvalho de Melo @ 2015-04-20 20:01 UTC (permalink / raw)
To: Jiri Olsa
Cc: David Ahern, Frederic Weisbecker, He Kuang, Ingo Molnar,
Jeremie Galarneau, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan, lkml
Em Mon, Apr 20, 2015 at 04:58:54PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Sat, Apr 18, 2015 at 05:50:15PM +0200, Jiri Olsa escreveu:
> > From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> With latest babeltrace.git:
>
> [acme@zoo babeltrace]$ git log --oneline | head -1
> 48d711a204f6 Fix: Missing stdint.h in stream-class.h
>
> But I noticed I have this installed, as well:
>
> [acme@zoo babeltrace]$ rpm -qa | grep babeltr
> libbabeltrace-1.2.1-1.fc20.x86_64
> [acme@zoo babeltrace]$
>
> But:
>
> [acme@zoo babeltrace]$ ldd ~/bin/perf | grep babeltrace
> libbabeltrace-ctf.so.1 => /opt/libbabeltrace/lib/libbabeltrace-ctf.so.1 (0x00007fb52713b000)
> libbabeltrace.so.1 => /opt/libbabeltrace/lib/libbabeltrace.so.1 (0x00007fb526889000)
> [acme@zoo babeltrace]$
>
> Will try removing the rpm as well...
Didn't help, removing this patch makes it work again:
[acme@zoo linux]$ perf data convert --to-ctf=ctf
[ perf data convert: Converted 'perf.data' into CTF data 'ctf' ]
[ perf data convert: Converted and wrote 0.000 MB (8 samples) ]
[acme@zoo linux]$ babeltrace ctf
[20:29:41.303589157] (+?.?????????) cycles: { }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 23321, perf_pid = 23321, perf_period = 1 }
[20:29:41.303591929] (+0.000002772) cycles: { }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 23321, perf_pid = 23321, perf_period = 1 }
[20:29:41.303592989] (+0.000001060) cycles: { }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 23321, perf_pid = 23321, perf_period = 11 }
[20:29:41.303594530] (+0.000001541) cycles: { }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 23321, perf_pid = 23321, perf_period = 275 }
[20:29:41.303873972] (+0.000279442) cycles: { }, { perf_ip = 0xFFFFFFFF810D659E, perf_tid = 23321, perf_pid = 23321, perf_period = 5458 }
[20:29:41.303876335] (+0.000002363) cycles: { }, { perf_ip = 0xFFFFFFFF8109F508, perf_tid = 23321, perf_pid = 23321, perf_period = 5388 }
[20:29:41.303878723] (+0.000002388) cycles: { }, { perf_ip = 0x7F19FA50063F, perf_tid = 23321, perf_pid = 23321, perf_period = 73411 }
[20:29:41.303902598] (+0.000023875) cycles: { }, { perf_ip = 0xFFFFFFFF810DB934, perf_tid = 23321, perf_pid = 23321, perf_period = 1026523 }
[acme@zoo linux]$
- Arnaldo
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCHv6 0/7] perf tools: Add perf data CTF conversion
2015-04-18 15:50 [PATCHv6 0/7] perf tools: Add perf data CTF conversion Jiri Olsa
` (6 preceding siblings ...)
2015-04-18 15:50 ` [PATCH 7/7] perf data: Fix signess of value Jiri Olsa
@ 2015-04-20 20:05 ` Arnaldo Carvalho de Melo
7 siblings, 0 replies; 29+ messages in thread
From: Arnaldo Carvalho de Melo @ 2015-04-20 20:05 UTC (permalink / raw)
To: Jiri Olsa
Cc: David Ahern, Frederic Weisbecker, He Kuang, Ingo Molnar,
Jeremie Galarneau, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan, lkml
Em Sat, Apr 18, 2015 at 05:50:13PM +0200, Jiri Olsa escreveu:
> this is follow up on original RFC patchset:
> http://marc.info/?t=140732735600004&r=1&w=2
>
> Basically we are adding 'perf data convert' command to
> allow conversion of perf data file into CTF [1] data.
Patches 1 and 5 applied, the others don't apply because patch 2
failed my testing, reported.
- Arnaldo
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 2/7] perf data: Switch to multiple cpu stream files
2015-04-20 19:58 ` Arnaldo Carvalho de Melo
2015-04-20 20:01 ` Arnaldo Carvalho de Melo
@ 2015-04-20 20:19 ` Jiri Olsa
2015-04-20 20:41 ` Arnaldo Carvalho de Melo
1 sibling, 1 reply; 29+ messages in thread
From: Jiri Olsa @ 2015-04-20 20:19 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: Jiri Olsa, David Ahern, Frederic Weisbecker, He Kuang,
Ingo Molnar, Jeremie Galarneau, Namhyung Kim, Paul Mackerras,
Peter Zijlstra, Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan,
lkml
On Mon, Apr 20, 2015 at 04:58:54PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Sat, Apr 18, 2015 at 05:50:15PM +0200, Jiri Olsa escreveu:
> > From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> >
> > Currently we store the data into single data strea/file. The cpu
> > if data is stored within the event sample. The lttng puts the CPU
> > number that belongs to the event into the packet context instead
> > into the event.
> >
> > This patch makes sure that the trace produce by perf does look the
> > same way. We now use one stream per-CPU. Having it all in one stream
> > increased the total size of the resulting file. The test went from
> > 416KiB (with perf_cpu event member) to 24MiB due to the required
> > (and pointless) flush. With the per-cpu streams the total size went
> > up to 588KiB.
> >
> > Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> > Link: http://lkml.kernel.org/n/tip-fco07vxi6yx4m9et2aimm4u0@git.kernel.org
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
>
> With latest babeltrace.git:
>
> [acme@zoo babeltrace]$ git log --oneline | head -1
> 48d711a204f6 Fix: Missing stdint.h in stream-class.h
hum, this is rather old one (Mar 10th), current is:
[jolsa@krava babeltrace]$ git show --oneline | head -1
d8f190b24d52 Add tests for bt_ctf_field_string_append_len()
jirka
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 2/7] perf data: Switch to multiple cpu stream files
2015-04-20 20:19 ` Jiri Olsa
@ 2015-04-20 20:41 ` Arnaldo Carvalho de Melo
2015-04-20 20:49 ` Arnaldo Carvalho de Melo
0 siblings, 1 reply; 29+ messages in thread
From: Arnaldo Carvalho de Melo @ 2015-04-20 20:41 UTC (permalink / raw)
To: Jiri Olsa
Cc: Jiri Olsa, David Ahern, Frederic Weisbecker, He Kuang,
Ingo Molnar, Jeremie Galarneau, Namhyung Kim, Paul Mackerras,
Peter Zijlstra, Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan,
lkml
Em Mon, Apr 20, 2015 at 10:19:44PM +0200, Jiri Olsa escreveu:
> On Mon, Apr 20, 2015 at 04:58:54PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Sat, Apr 18, 2015 at 05:50:15PM +0200, Jiri Olsa escreveu:
> > > From: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> > >
> > > Currently we store the data into single data strea/file. The cpu
> > > if data is stored within the event sample. The lttng puts the CPU
> > > number that belongs to the event into the packet context instead
> > > into the event.
> > >
> > > This patch makes sure that the trace produce by perf does look the
> > > same way. We now use one stream per-CPU. Having it all in one stream
> > > increased the total size of the resulting file. The test went from
> > > 416KiB (with perf_cpu event member) to 24MiB due to the required
> > > (and pointless) flush. With the per-cpu streams the total size went
> > > up to 588KiB.
> > >
> > > Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> > > Link: http://lkml.kernel.org/n/tip-fco07vxi6yx4m9et2aimm4u0@git.kernel.org
> > > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> >
> > With latest babeltrace.git:
> >
> > [acme@zoo babeltrace]$ git log --oneline | head -1
> > 48d711a204f6 Fix: Missing stdint.h in stream-class.h
>
> hum, this is rather old one (Mar 10th), current is:
>
> [jolsa@krava babeltrace]$ git show --oneline | head -1
> d8f190b24d52 Add tests for bt_ctf_field_string_append_len()
I did a git remove update...
Lemme double check:
[acme@zoo babeltrace]$ git remote update
Fetching origin
[acme@zoo babeltrace]$ git remote -v
origin git://git.efficios.com/babeltrace.git (fetch)
origin git://git.efficios.com/babeltrace.git (push)
[acme@zoo babeltrace]$ git log --oneline | head -1
48d711a204f6 Fix: Missing stdint.h in stream-class.h
[acme@zoo babeltrace]$
What is wrong here? Is it in a different branch?
/me checks again that... Yeah, I probably created my local branch from
some other branch, now that I did:
[acme@zoo babeltrace]$ git checkout -b master origin/master
Branch master set up to track remote branch master from origin.
Switched to a new branch 'master'
[acme@zoo babeltrace]$ git log --oneline | head -1
d8f190b24d52 Add tests for bt_ctf_field_string_append_len()
[acme@zoo babeltrace]$
will try again those patches...
- ARnaldo
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 2/7] perf data: Switch to multiple cpu stream files
2015-04-20 20:41 ` Arnaldo Carvalho de Melo
@ 2015-04-20 20:49 ` Arnaldo Carvalho de Melo
0 siblings, 0 replies; 29+ messages in thread
From: Arnaldo Carvalho de Melo @ 2015-04-20 20:49 UTC (permalink / raw)
To: Jiri Olsa
Cc: Jiri Olsa, David Ahern, Frederic Weisbecker, He Kuang,
Ingo Molnar, Jeremie Galarneau, Namhyung Kim, Paul Mackerras,
Peter Zijlstra, Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan,
lkml
Em Mon, Apr 20, 2015 at 05:41:29PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Mon, Apr 20, 2015 at 10:19:44PM +0200, Jiri Olsa escreveu:
> > On Mon, Apr 20, 2015 at 04:58:54PM -0300, Arnaldo Carvalho de Melo wrote:
> > > With latest babeltrace.git:
> > >
> > > [acme@zoo babeltrace]$ git log --oneline | head -1
> > > 48d711a204f6 Fix: Missing stdint.h in stream-class.h
> >
> > hum, this is rather old one (Mar 10th), current is:
> >
> > [jolsa@krava babeltrace]$ git show --oneline | head -1
> > d8f190b24d52 Add tests for bt_ctf_field_string_append_len()
<SNIP>
> [acme@zoo babeltrace]$ git checkout -b master origin/master
> Branch master set up to track remote branch master from origin.
> Switched to a new branch 'master'
> [acme@zoo babeltrace]$ git log --oneline | head -1
> d8f190b24d52 Add tests for bt_ctf_field_string_append_len()
> [acme@zoo babeltrace]$
>
> will try again those patches...
Works now, continuing to process the others.o
- Arnaldo
[root@zoo ~]# perf record -a usleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 1.343 MB perf.data (24 samples) ]
[root@zoo ~]#
[root@zoo ~]# perf data convert --to-ctf=ctf
[ perf data convert: Converted 'perf.data' into CTF data 'ctf' ]
[ perf data convert: Converted and wrote 0.001 MB (24 samples) ]
[root@zoo ~]# ls -la ctf
total 176
drwxr-x---. 2 root root 4096 Apr 20 17:48 .
dr-xr-x---. 21 root root 36864 Apr 20 17:48 ..
-rw-r-----. 1 root root 2468 Apr 20 17:48 metadata
-rw-r-----. 1 root root 32768 Apr 20 17:48 perf_stream_0
-rw-r-----. 1 root root 32768 Apr 20 17:48 perf_stream_1
-rw-r-----. 1 root root 32768 Apr 20 17:48 perf_stream_2
-rw-r-----. 1 root root 32768 Apr 20 17:48 perf_stream_3
[root@zoo ~]# babeltrace ctf
bash: babeltrace: command not found...
[root@zoo ~]# /opt/libbabeltrace/bin/babeltrace ctf
[23:15:23.559883150] (+?.?????????) cycles: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 2344, perf_pid = 2344, perf_period = 1 }
[23:15:23.559885913] (+0.000002763) cycles: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 2344, perf_pid = 2344, perf_period = 1 }
[23:15:23.559886994] (+0.000001081) cycles: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 2344, perf_pid = 2344, perf_period = 11 }
[23:15:23.559889237] (+0.000002243) cycles: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 2344, perf_pid = 2344, perf_period = 281 }
[23:15:23.559928349] (+0.000039112) cycles: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 1 }
[23:15:23.559930427] (+0.000002078) cycles: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 1 }
[23:15:23.559931532] (+0.000001105) cycles: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 14 }
[23:15:23.559932945] (+0.000001413) cycles: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 382 }
[23:15:23.559940818] (+0.000007873) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 1 }
[23:15:23.559942796] (+0.000001978) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 1 }
[23:15:23.559944288] (+0.000001492) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 16 }
[23:15:23.559946113] (+0.000001825) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 344 }
[23:15:23.559964274] (+0.000018161) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 1 }
[23:15:23.559966375] (+0.000002101) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 1 }
[23:15:23.559967491] (+0.000001116) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 15 }
[23:15:23.559968894] (+0.000001403) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF8105E8AA, perf_tid = 0, perf_pid = 0, perf_period = 404 }
[23:15:23.560102516] (+0.000133622) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF810D659E, perf_tid = 2345, perf_pid = 2345, perf_period = 8401 }
[23:15:23.560105701] (+0.000003185) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF810CDDC0, perf_tid = 2345, perf_pid = 2345, perf_period = 9317 }
[23:15:23.560109298] (+0.000003597) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF8131793E, perf_tid = 2345, perf_pid = 2345, perf_period = 97191 }
[23:15:23.560140949] (+0.000031651) cycles: { cpu_id = 3 }, { perf_ip = 0xFFFFFFFF81100D70, perf_tid = 0, perf_pid = 0, perf_period = 929183 }
[23:15:23.562123834] (+0.001982885) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF8109F5F5, perf_tid = 2345, perf_pid = 2345, perf_period = 5662 }
[23:15:23.562133948] (+0.000010114) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF8104C859, perf_tid = 2345, perf_pid = 2345, perf_period = 5662 }
[23:15:23.562143711] (+0.000009763) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF810CDFA5, perf_tid = 2345, perf_pid = 2345, perf_period = 21836 }
[23:15:23.562174958] (+0.000031247) cycles: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF813A2B50, perf_tid = 2345, perf_pid = 2345, perf_period = 88041 }
[root@zoo ~]#
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 7/7] perf data: Fix signess of value
2015-04-18 15:50 ` [PATCH 7/7] perf data: Fix signess of value Jiri Olsa
@ 2015-04-20 21:23 ` Arnaldo Carvalho de Melo
2015-04-20 21:52 ` Jiri Olsa
2015-04-21 2:41 ` Wang Nan
2015-05-06 2:58 ` [tip:perf/core] perf data: Fix signedness " tip-bot for Wang Nan
1 sibling, 2 replies; 29+ messages in thread
From: Arnaldo Carvalho de Melo @ 2015-04-20 21:23 UTC (permalink / raw)
To: Jiri Olsa
Cc: David Ahern, Frederic Weisbecker, He Kuang, Ingo Molnar,
Jeremie Galarneau, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan, lkml
Em Sat, Apr 18, 2015 at 05:50:20PM +0200, Jiri Olsa escreveu:
> From: Wang Nan <wangnan0@huawei.com>
>
> When converting int values, perf first extractes it to a ulonglong, then
> feeds it to babeltrace as a signed value. For negative 32 bit values
> (for example, return values of failed syscalls), the extracted data
> should be something like 0xfffffffe (-2). It becomes a large int64
> value. Babeltrace denies to insert it with
> bt_ctf_field_signed_integer_set_value() because it is larger than
> 0x7fffffff, the largest positive value a signed 32 bit int can be.
There is no such word "signess", it is "signedness", fixing this up.
Humm, it seems there is such a word indeed:
http://www.urbandictionary.com/define.php?term=Signess
But I bet this is the one we want:
http://en.wikipedia.org/wiki/Signedness
Right? :-)
- Arnaldo
> This patch introduces adjust_signess(), which fills high bits of
> ulonglong with 1 if the value is negative.
>
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Link: http://lkml.kernel.org/n/tip-bq2fguw8pm8s5rr741mmgias@git.kernel.org
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
> tools/perf/util/data-convert-bt.c | 64 +++++++++++++++++++++++++++++++--------
> 1 file changed, 52 insertions(+), 12 deletions(-)
>
> diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
> index 8eda4ed628e7..977cc3f98d8f 100644
> --- a/tools/perf/util/data-convert-bt.c
> +++ b/tools/perf/util/data-convert-bt.c
> @@ -166,6 +166,43 @@ get_tracepoint_field_type(struct ctf_writer *cw, struct format_field *field)
> return cw->data.u32;
> }
>
> +static unsigned long long adjust_signess(unsigned long long value_int, int size)
> +{
> + unsigned long long value_mask;
> +
> + /*
> + * value_mask = (1 << (size * 8 - 1)) - 1.
> + * Directly set value_mask for code readers.
> + */
> + switch (size) {
> + case 1:
> + value_mask = 0x7fULL;
> + break;
> + case 2:
> + value_mask = 0x7fffULL;
> + break;
> + case 4:
> + value_mask = 0x7fffffffULL;
> + break;
> + case 8:
> + /*
> + * For 64 bit value, return it self. There is no need
> + * to fill high bit.
> + */
> + /* Fall through */
> + default:
> + /* BUG! */
> + return value_int;
> + }
> +
> + /* If it is a positive value, don't adjust. */
> + if ((value_int & (~0ULL - value_mask)) == 0)
> + return value_int;
> +
> + /* Fill upper part of value_int with 1 to make it a negative long long. */
> + return (value_int & value_mask) | ~value_mask;
> +}
> +
> static int add_tracepoint_field_value(struct ctf_writer *cw,
> struct bt_ctf_event_class *event_class,
> struct bt_ctf_event *event,
> @@ -177,7 +214,6 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
> struct bt_ctf_field *field;
> const char *name = fmtf->name;
> void *data = sample->raw_data;
> - unsigned long long value_int;
> unsigned long flags = fmtf->flags;
> unsigned int n_items;
> unsigned int i;
> @@ -222,11 +258,6 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
> type = get_tracepoint_field_type(cw, fmtf);
>
> for (i = 0; i < n_items; i++) {
> - if (!(flags & FIELD_IS_STRING))
> - value_int = pevent_read_number(
> - fmtf->event->pevent,
> - data + offset + i * len, len);
> -
> if (flags & FIELD_IS_ARRAY)
> field = bt_ctf_field_array_get_field(array_field, i);
> else
> @@ -240,12 +271,21 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
> if (flags & FIELD_IS_STRING)
> ret = bt_ctf_field_string_set_value(field,
> data + offset + i * len);
> - else if (!(flags & FIELD_IS_SIGNED))
> - ret = bt_ctf_field_unsigned_integer_set_value(
> - field, value_int);
> - else
> - ret = bt_ctf_field_signed_integer_set_value(
> - field, value_int);
> + else {
> + unsigned long long value_int;
> +
> + value_int = pevent_read_number(
> + fmtf->event->pevent,
> + data + offset + i * len, len);
> +
> + if (!(flags & FIELD_IS_SIGNED))
> + ret = bt_ctf_field_unsigned_integer_set_value(
> + field, value_int);
> + else
> + ret = bt_ctf_field_signed_integer_set_value(
> + field, adjust_signess(value_int, len));
> + }
> +
> if (ret) {
> pr_err("failed to set file value %s\n", name);
> goto err_put_field;
> --
> 1.9.3
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 7/7] perf data: Fix signess of value
2015-04-20 21:23 ` Arnaldo Carvalho de Melo
@ 2015-04-20 21:52 ` Jiri Olsa
2015-04-21 2:41 ` Wang Nan
1 sibling, 0 replies; 29+ messages in thread
From: Jiri Olsa @ 2015-04-20 21:52 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: Jiri Olsa, David Ahern, Frederic Weisbecker, He Kuang,
Ingo Molnar, Jeremie Galarneau, Namhyung Kim, Paul Mackerras,
Peter Zijlstra, Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan,
lkml
On Mon, Apr 20, 2015 at 06:23:14PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Sat, Apr 18, 2015 at 05:50:20PM +0200, Jiri Olsa escreveu:
> > From: Wang Nan <wangnan0@huawei.com>
> >
> > When converting int values, perf first extractes it to a ulonglong, then
> > feeds it to babeltrace as a signed value. For negative 32 bit values
> > (for example, return values of failed syscalls), the extracted data
> > should be something like 0xfffffffe (-2). It becomes a large int64
> > value. Babeltrace denies to insert it with
> > bt_ctf_field_signed_integer_set_value() because it is larger than
> > 0x7fffffff, the largest positive value a signed 32 bit int can be.
>
> There is no such word "signess", it is "signedness", fixing this up.
> Humm, it seems there is such a word indeed:
>
> http://www.urbandictionary.com/define.php?term=Signess
>
> But I bet this is the one we want:
>
> http://en.wikipedia.org/wiki/Signedness
>
> Right? :-)
right ;-) thanks
jirka
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 7/7] perf data: Fix signess of value
2015-04-20 21:23 ` Arnaldo Carvalho de Melo
2015-04-20 21:52 ` Jiri Olsa
@ 2015-04-21 2:41 ` Wang Nan
2015-04-21 14:25 ` Arnaldo Carvalho de Melo
1 sibling, 1 reply; 29+ messages in thread
From: Wang Nan @ 2015-04-21 2:41 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo, Jiri Olsa
Cc: David Ahern, Frederic Weisbecker, He Kuang, Ingo Molnar,
Jeremie Galarneau, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
Sebastian Andrzej Siewior, Tom Zanussi, lkml
On 2015/4/21 5:23, Arnaldo Carvalho de Melo wrote:
> Em Sat, Apr 18, 2015 at 05:50:20PM +0200, Jiri Olsa escreveu:
>> From: Wang Nan <wangnan0@huawei.com>
>>
>> When converting int values, perf first extractes it to a ulonglong, then
>> feeds it to babeltrace as a signed value. For negative 32 bit values
>> (for example, return values of failed syscalls), the extracted data
>> should be something like 0xfffffffe (-2). It becomes a large int64
>> value. Babeltrace denies to insert it with
>> bt_ctf_field_signed_integer_set_value() because it is larger than
>> 0x7fffffff, the largest positive value a signed 32 bit int can be.
>
> There is no such word "signess", it is "signedness", fixing this up.
> Humm, it seems there is such a word indeed:
>
> http://www.urbandictionary.com/define.php?term=Signess
>
> But I bet this is the one we want:
>
> http://en.wikipedia.org/wiki/Signedness
>
> Right? :-)
>
> - Arnaldo
>
Sorry for the bad English. Please help me to fix it. Thank you.
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 7/7] perf data: Fix signess of value
2015-04-21 2:41 ` Wang Nan
@ 2015-04-21 14:25 ` Arnaldo Carvalho de Melo
0 siblings, 0 replies; 29+ messages in thread
From: Arnaldo Carvalho de Melo @ 2015-04-21 14:25 UTC (permalink / raw)
To: Wang Nan
Cc: Jiri Olsa, David Ahern, Frederic Weisbecker, He Kuang,
Ingo Molnar, Jeremie Galarneau, Namhyung Kim, Paul Mackerras,
Peter Zijlstra, Sebastian Andrzej Siewior, Tom Zanussi, lkml
Em Tue, Apr 21, 2015 at 10:41:55AM +0800, Wang Nan escreveu:
> On 2015/4/21 5:23, Arnaldo Carvalho de Melo wrote:
> > Em Sat, Apr 18, 2015 at 05:50:20PM +0200, Jiri Olsa escreveu:
> >> From: Wang Nan <wangnan0@huawei.com>
> > There is no such word "signess", it is "signedness", fixing this up.
> > Humm, it seems there is such a word indeed:
> > http://www.urbandictionary.com/define.php?term=Signess
> > But I bet this is the one we want:
> > http://en.wikipedia.org/wiki/Signedness
> > Right? :-)
> Sorry for the bad English. Please help me to fix it. Thank you.
It is not a problem, thanks for your contributions, it all works that
way, we find mistakes, we fix mistakes, then we make some more...
I only mentioned that urban dictionary link because I am not a native
speaker as well, we speak portuguese down here, so I went to check if my
feeling was right and found it, felt it was funny and unexpected,
mentioned here, no offense intended :-)
- Arnaldo
^ permalink raw reply [flat|nested] 29+ messages in thread
* [tip:perf/core] perf data: Show error message when conversion failed
2015-04-18 15:50 ` [PATCH 1/7] perf data: Show error message when conversion failed Jiri Olsa
@ 2015-05-06 2:55 ` tip-bot for He Kuang
0 siblings, 0 replies; 29+ messages in thread
From: tip-bot for He Kuang @ 2015-05-06 2:55 UTC (permalink / raw)
To: linux-tip-commits
Cc: bigeasy, wangnan0, tzanussi, fweisbec, paulus, mingo,
a.p.zijlstra, jolsa, hekuang, jgalar, hpa, tglx, dsahern, acme,
linux-kernel, namhyung
Commit-ID: c21410554ee1a8e12fbc99f578d720b641cd1c74
Gitweb: http://git.kernel.org/tip/c21410554ee1a8e12fbc99f578d720b641cd1c74
Author: He Kuang <hekuang@huawei.com>
AuthorDate: Sat, 18 Apr 2015 17:50:14 +0200
Committer: Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 29 Apr 2015 10:37:45 -0300
perf data: Show error message when conversion failed
Show message when errors occurred during conversion setup and conversion
process.
Before this patch:
$ ./perf data convert --to-ctf=ctf
$ echo $?
255
After this patch:
$ ./perf data convert --to-ctf=ctf
Error during conversion setup.
Signed-off-by: He Kuang <hekuang@huawei.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jeremie Galarneau <jgalar@efficios.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-xvhr1vf7zav9kkeo9w1hv4uk@git.kernel.org
Link: http://lkml.kernel.org/r/1429372220-6406-2-git-send-email-jolsa@kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
tools/perf/util/data-convert-bt.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index dd17c9a..db4f709 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -837,6 +837,8 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
err = perf_session__process_events(session);
if (!err)
err = bt_ctf_stream_flush(cw->stream);
+ else
+ pr_err("Error during conversion.\n");
fprintf(stderr,
"[ perf data convert: Converted '%s' into CTF data '%s' ]\n",
@@ -847,11 +849,15 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
(double) c.events_size / 1024.0 / 1024.0,
c.events_count);
- /* its all good */
-free_session:
perf_session__delete(session);
+ ctf_writer__cleanup(cw);
+ return err;
+
+free_session:
+ perf_session__delete(session);
free_writer:
ctf_writer__cleanup(cw);
+ pr_err("Error during conversion setup.\n");
return err;
}
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [tip:perf/core] tools lib traceevent: Add alias field to struct format_field
2015-04-18 15:50 ` [PATCH 5/7] tools lib traceevent: Add alias field to struct format_field Jiri Olsa
2015-04-20 14:24 ` Steven Rostedt
@ 2015-05-06 2:55 ` tip-bot for Jiri Olsa
1 sibling, 0 replies; 29+ messages in thread
From: tip-bot for Jiri Olsa @ 2015-05-06 2:55 UTC (permalink / raw)
To: linux-tip-commits
Cc: hekuang, wangnan0, acme, namhyung, fweisbec, paulus, tglx,
jgalar, tzanussi, linux-kernel, jolsa, a.p.zijlstra, dsahern,
mingo, hpa, rostedt, bigeasy
Commit-ID: d3542436c9a8d114bf81c0742e67ff8ce8be9324
Gitweb: http://git.kernel.org/tip/d3542436c9a8d114bf81c0742e67ff8ce8be9324
Author: Jiri Olsa <jolsa@kernel.org>
AuthorDate: Sat, 18 Apr 2015 17:50:18 +0200
Committer: Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 29 Apr 2015 10:37:45 -0300
tools lib traceevent: Add alias field to struct format_field
Introduce an 'alias' field to 'struct format_field' to be able
to use alternative name for the field.
It is initialized with same string pointer as 'name' field.
The free logic checks the 'alias' pointer being reset by user
and frees it.
This will be handy when converting data into CTF, where each
field within event needs to have a unique name (while this
is not required for tracepoint). Converter can easily assign
unique name into the format_field struct.
Suggested-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Steven Rostedt <rostedt@goodmis.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jeremie Galarneau <jgalar@efficios.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-qwyq8blnfkg6s5vlbrvn1en3@git.kernel.org
Link: http://lkml.kernel.org/r/1429372220-6406-6-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
tools/lib/traceevent/event-parse.c | 6 ++++--
tools/lib/traceevent/event-parse.h | 1 +
2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/tools/lib/traceevent/event-parse.c b/tools/lib/traceevent/event-parse.c
index 12a7e2a..1ec56b4 100644
--- a/tools/lib/traceevent/event-parse.c
+++ b/tools/lib/traceevent/event-parse.c
@@ -1387,7 +1387,7 @@ static int event_read_fields(struct event_format *event, struct format_field **f
do_warning_event(event, "%s: no type found", __func__);
goto fail;
}
- field->name = last_token;
+ field->name = field->alias = last_token;
if (test_type(type, EVENT_OP))
goto fail;
@@ -1469,7 +1469,7 @@ static int event_read_fields(struct event_format *event, struct format_field **f
size_dynamic = type_size(field->name);
free_token(field->name);
strcat(field->type, brackets);
- field->name = token;
+ field->name = field->alias = token;
type = read_token(&token);
} else {
char *new_type;
@@ -6444,6 +6444,8 @@ void pevent_ref(struct pevent *pevent)
void pevent_free_format_field(struct format_field *field)
{
free(field->type);
+ if (field->alias != field->name)
+ free(field->alias);
free(field->name);
free(field);
}
diff --git a/tools/lib/traceevent/event-parse.h b/tools/lib/traceevent/event-parse.h
index 86a5839..063b197 100644
--- a/tools/lib/traceevent/event-parse.h
+++ b/tools/lib/traceevent/event-parse.h
@@ -191,6 +191,7 @@ struct format_field {
struct event_format *event;
char *type;
char *name;
+ char *alias;
int offset;
int size;
unsigned int arraylen;
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [tip:perf/core] perf data: Switch to multiple cpu stream files
2015-04-18 15:50 ` [PATCH 2/7] perf data: Switch to multiple cpu stream files Jiri Olsa
2015-04-20 19:58 ` Arnaldo Carvalho de Melo
@ 2015-05-06 2:57 ` tip-bot for Sebastian Andrzej Siewior
1 sibling, 0 replies; 29+ messages in thread
From: tip-bot for Sebastian Andrzej Siewior @ 2015-05-06 2:57 UTC (permalink / raw)
To: linux-tip-commits
Cc: paulus, hekuang, namhyung, linux-kernel, dsahern, bigeasy,
a.p.zijlstra, fweisbec, wangnan0, hpa, jolsa, tzanussi, jgalar,
mingo, acme, tglx
Commit-ID: 90e129ff602672c4d5a631b2564411e19de36e15
Gitweb: http://git.kernel.org/tip/90e129ff602672c4d5a631b2564411e19de36e15
Author: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
AuthorDate: Sat, 18 Apr 2015 17:50:15 +0200
Committer: Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 29 Apr 2015 10:37:47 -0300
perf data: Switch to multiple cpu stream files
Currently we store the data into single data strea/file. The cpu if data
is stored within the event sample. The lttng puts the CPU number that
belongs to the event into the packet context instead into the event.
This patch makes sure that the trace produce by perf does look the same
way. We now use one stream per-CPU. Having it all in one stream
increased the total size of the resulting file. The test went from
416KiB (with perf_cpu event member) to 24MiB due to the required (and
pointless) flush. With the per-cpu streams the total size went up to
588KiB.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jeremie Galarneau <jgalar@efficios.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Tom Zanussi <tzanussi@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1429372220-6406-3-git-send-email-jolsa@kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
tools/perf/util/data-convert-bt.c | 206 +++++++++++++++++++++++++++++++++-----
1 file changed, 182 insertions(+), 24 deletions(-)
diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index db4f709..0a2d7a5 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -38,12 +38,20 @@ struct evsel_priv {
struct bt_ctf_event_class *event_class;
};
+#define MAX_CPUS 4096
+
+struct ctf_stream {
+ struct bt_ctf_stream *stream;
+ int cpu;
+};
+
struct ctf_writer {
/* writer primitives */
- struct bt_ctf_writer *writer;
- struct bt_ctf_stream *stream;
- struct bt_ctf_stream_class *stream_class;
- struct bt_ctf_clock *clock;
+ struct bt_ctf_writer *writer;
+ struct ctf_stream **stream;
+ int stream_cnt;
+ struct bt_ctf_stream_class *stream_class;
+ struct bt_ctf_clock *clock;
/* data types */
union {
@@ -346,12 +354,6 @@ static int add_generic_values(struct ctf_writer *cw,
return -1;
}
- if (type & PERF_SAMPLE_CPU) {
- ret = value_set_u32(cw, event, "perf_cpu", sample->cpu);
- if (ret)
- return -1;
- }
-
if (type & PERF_SAMPLE_PERIOD) {
ret = value_set_u64(cw, event, "perf_period", sample->period);
if (ret)
@@ -381,6 +383,113 @@ static int add_generic_values(struct ctf_writer *cw,
return 0;
}
+static int ctf_stream__flush(struct ctf_stream *cs)
+{
+ int err = 0;
+
+ if (cs) {
+ err = bt_ctf_stream_flush(cs->stream);
+ if (err)
+ pr_err("CTF stream %d flush failed\n", cs->cpu);
+
+ pr("Flush stream for cpu %d\n", cs->cpu);
+ }
+
+ return err;
+}
+
+static struct ctf_stream *ctf_stream__create(struct ctf_writer *cw, int cpu)
+{
+ struct ctf_stream *cs;
+ struct bt_ctf_field *pkt_ctx = NULL;
+ struct bt_ctf_field *cpu_field = NULL;
+ struct bt_ctf_stream *stream = NULL;
+ int ret;
+
+ cs = zalloc(sizeof(*cs));
+ if (!cs) {
+ pr_err("Failed to allocate ctf stream\n");
+ return NULL;
+ }
+
+ stream = bt_ctf_writer_create_stream(cw->writer, cw->stream_class);
+ if (!stream) {
+ pr_err("Failed to create CTF stream\n");
+ goto out;
+ }
+
+ pkt_ctx = bt_ctf_stream_get_packet_context(stream);
+ if (!pkt_ctx) {
+ pr_err("Failed to obtain packet context\n");
+ goto out;
+ }
+
+ cpu_field = bt_ctf_field_structure_get_field(pkt_ctx, "cpu_id");
+ bt_ctf_field_put(pkt_ctx);
+ if (!cpu_field) {
+ pr_err("Failed to obtain cpu field\n");
+ goto out;
+ }
+
+ ret = bt_ctf_field_unsigned_integer_set_value(cpu_field, (u32) cpu);
+ if (ret) {
+ pr_err("Failed to update CPU number\n");
+ goto out;
+ }
+
+ bt_ctf_field_put(cpu_field);
+
+ cs->cpu = cpu;
+ cs->stream = stream;
+ return cs;
+
+out:
+ if (cpu_field)
+ bt_ctf_field_put(cpu_field);
+ if (stream)
+ bt_ctf_stream_put(stream);
+
+ free(cs);
+ return NULL;
+}
+
+static void ctf_stream__delete(struct ctf_stream *cs)
+{
+ if (cs) {
+ bt_ctf_stream_put(cs->stream);
+ free(cs);
+ }
+}
+
+static struct ctf_stream *ctf_stream(struct ctf_writer *cw, int cpu)
+{
+ struct ctf_stream *cs = cw->stream[cpu];
+
+ if (!cs) {
+ cs = ctf_stream__create(cw, cpu);
+ cw->stream[cpu] = cs;
+ }
+
+ return cs;
+}
+
+static int get_sample_cpu(struct ctf_writer *cw, struct perf_sample *sample,
+ struct perf_evsel *evsel)
+{
+ int cpu = 0;
+
+ if (evsel->attr.sample_type & PERF_SAMPLE_CPU)
+ cpu = sample->cpu;
+
+ if (cpu > cw->stream_cnt) {
+ pr_err("Event was recorded for CPU %d, limit is at %d.\n",
+ cpu, cw->stream_cnt);
+ cpu = 0;
+ }
+
+ return cpu;
+}
+
static int process_sample_event(struct perf_tool *tool,
union perf_event *_event __maybe_unused,
struct perf_sample *sample,
@@ -390,6 +499,7 @@ static int process_sample_event(struct perf_tool *tool,
struct convert *c = container_of(tool, struct convert, tool);
struct evsel_priv *priv = evsel->priv;
struct ctf_writer *cw = &c->writer;
+ struct ctf_stream *cs;
struct bt_ctf_event_class *event_class;
struct bt_ctf_event *event;
int ret;
@@ -424,9 +534,12 @@ static int process_sample_event(struct perf_tool *tool,
return -1;
}
- bt_ctf_stream_append_event(cw->stream, event);
+ cs = ctf_stream(cw, get_sample_cpu(cw, sample, evsel));
+ if (cs)
+ bt_ctf_stream_append_event(cs->stream, event);
+
bt_ctf_event_put(event);
- return 0;
+ return cs ? 0 : -1;
}
static int add_tracepoint_fields_types(struct ctf_writer *cw,
@@ -528,9 +641,6 @@ static int add_generic_types(struct ctf_writer *cw, struct perf_evsel *evsel,
if (type & PERF_SAMPLE_STREAM_ID)
ADD_FIELD(event_class, cw->data.u64, "perf_stream_id");
- if (type & PERF_SAMPLE_CPU)
- ADD_FIELD(event_class, cw->data.u32, "perf_cpu");
-
if (type & PERF_SAMPLE_PERIOD)
ADD_FIELD(event_class, cw->data.u64, "perf_period");
@@ -604,6 +714,39 @@ static int setup_events(struct ctf_writer *cw, struct perf_session *session)
return 0;
}
+static int setup_streams(struct ctf_writer *cw, struct perf_session *session)
+{
+ struct ctf_stream **stream;
+ struct perf_header *ph = &session->header;
+ int ncpus;
+
+ /*
+ * Try to get the number of cpus used in the data file,
+ * if not present fallback to the MAX_CPUS.
+ */
+ ncpus = ph->env.nr_cpus_avail ?: MAX_CPUS;
+
+ stream = zalloc(sizeof(*stream) * ncpus);
+ if (!stream) {
+ pr_err("Failed to allocate streams.\n");
+ return -ENOMEM;
+ }
+
+ cw->stream = stream;
+ cw->stream_cnt = ncpus;
+ return 0;
+}
+
+static void free_streams(struct ctf_writer *cw)
+{
+ int cpu;
+
+ for (cpu = 0; cpu < cw->stream_cnt; cpu++)
+ ctf_stream__delete(cw->stream[cpu]);
+
+ free(cw->stream);
+}
+
static int ctf_writer__setup_env(struct ctf_writer *cw,
struct perf_session *session)
{
@@ -713,7 +856,7 @@ static void ctf_writer__cleanup(struct ctf_writer *cw)
ctf_writer__cleanup_data(cw);
bt_ctf_clock_put(cw->clock);
- bt_ctf_stream_put(cw->stream);
+ free_streams(cw);
bt_ctf_stream_class_put(cw->stream_class);
bt_ctf_writer_put(cw->writer);
@@ -725,8 +868,9 @@ static int ctf_writer__init(struct ctf_writer *cw, const char *path)
{
struct bt_ctf_writer *writer;
struct bt_ctf_stream_class *stream_class;
- struct bt_ctf_stream *stream;
struct bt_ctf_clock *clock;
+ struct bt_ctf_field_type *pkt_ctx_type;
+ int ret;
/* CTF writer */
writer = bt_ctf_writer_create(path);
@@ -767,14 +911,15 @@ static int ctf_writer__init(struct ctf_writer *cw, const char *path)
if (ctf_writer__init_data(cw))
goto err_cleanup;
- /* CTF stream instance */
- stream = bt_ctf_writer_create_stream(writer, stream_class);
- if (!stream) {
- pr("Failed to create CTF stream.\n");
+ /* Add cpu_id for packet context */
+ pkt_ctx_type = bt_ctf_stream_class_get_packet_context_type(stream_class);
+ if (!pkt_ctx_type)
goto err_cleanup;
- }
- cw->stream = stream;
+ ret = bt_ctf_field_type_structure_add_field(pkt_ctx_type, cw->data.u32, "cpu_id");
+ bt_ctf_field_type_put(pkt_ctx_type);
+ if (ret)
+ goto err_cleanup;
/* CTF clock writer setup */
if (bt_ctf_writer_add_clock(writer, clock)) {
@@ -791,6 +936,16 @@ err:
return -1;
}
+static int ctf_writer__flush_streams(struct ctf_writer *cw)
+{
+ int cpu, ret = 0;
+
+ for (cpu = 0; cpu < cw->stream_cnt && !ret; cpu++)
+ ret = ctf_stream__flush(cw->stream[cpu]);
+
+ return ret;
+}
+
int bt_convert__perf2ctf(const char *input, const char *path, bool force)
{
struct perf_session *session;
@@ -834,9 +989,12 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
if (setup_events(cw, session))
goto free_session;
+ if (setup_streams(cw, session))
+ goto free_session;
+
err = perf_session__process_events(session);
if (!err)
- err = bt_ctf_stream_flush(cw->stream);
+ err = ctf_writer__flush_streams(cw);
else
pr_err("Error during conversion.\n");
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [tip:perf/core] perf data: Enable stream flush within processing
2015-04-18 15:50 ` [PATCH 3/7] perf data: Enable stream flush within processing Jiri Olsa
@ 2015-05-06 2:57 ` tip-bot for Jiri Olsa
0 siblings, 0 replies; 29+ messages in thread
From: tip-bot for Jiri Olsa @ 2015-05-06 2:57 UTC (permalink / raw)
To: linux-tip-commits
Cc: tglx, wangnan0, namhyung, paulus, linux-kernel, hpa, jgalar,
acme, fweisbec, a.p.zijlstra, mingo, hekuang, bigeasy, jolsa,
tzanussi, dsahern
Commit-ID: 89e5fa884df8cd72e81c3727cf1f0d1c7a9c7641
Gitweb: http://git.kernel.org/tip/89e5fa884df8cd72e81c3727cf1f0d1c7a9c7641
Author: Jiri Olsa <jolsa@kernel.org>
AuthorDate: Sat, 18 Apr 2015 17:50:16 +0200
Committer: Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 29 Apr 2015 10:37:48 -0300
perf data: Enable stream flush within processing
For big data files the size of data allocated for stream instance could
get really high. It's needed to flush the data out of the stream once in
a while.
Unfortunately there's no size indication in the stream object, so we
govern the flush based on the number of stored events. Current flush
limit is set ot 100000 events.
Reviewed-by: David Ahern <dsahern@gmail.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jeremie Galarneau <jgalar@efficios.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1429372220-6406-4-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
tools/perf/util/data-convert-bt.c | 26 ++++++++++++++++++++++++--
1 file changed, 24 insertions(+), 2 deletions(-)
diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index 0a2d7a5..4f942b3 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -43,6 +43,7 @@ struct evsel_priv {
struct ctf_stream {
struct bt_ctf_stream *stream;
int cpu;
+ u32 count;
};
struct ctf_writer {
@@ -392,7 +393,10 @@ static int ctf_stream__flush(struct ctf_stream *cs)
if (err)
pr_err("CTF stream %d flush failed\n", cs->cpu);
- pr("Flush stream for cpu %d\n", cs->cpu);
+ pr("Flush stream for cpu %d (%u samples)\n",
+ cs->cpu, cs->count);
+
+ cs->count = 0;
}
return err;
@@ -490,6 +494,19 @@ static int get_sample_cpu(struct ctf_writer *cw, struct perf_sample *sample,
return cpu;
}
+#define STREAM_FLUSH_COUNT 100000
+
+/*
+ * Currently we have no other way to determine the
+ * time for the stream flush other than keep track
+ * of the number of events and check it against
+ * threshold.
+ */
+static bool is_flush_needed(struct ctf_stream *cs)
+{
+ return cs->count >= STREAM_FLUSH_COUNT;
+}
+
static int process_sample_event(struct perf_tool *tool,
union perf_event *_event __maybe_unused,
struct perf_sample *sample,
@@ -535,8 +552,13 @@ static int process_sample_event(struct perf_tool *tool,
}
cs = ctf_stream(cw, get_sample_cpu(cw, sample, evsel));
- if (cs)
+ if (cs) {
+ if (is_flush_needed(cs))
+ ctf_stream__flush(cs);
+
+ cs->count++;
bt_ctf_stream_append_event(cs->stream, event);
+ }
bt_ctf_event_put(event);
return cs ? 0 : -1;
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [tip:perf/core] perf data: Add support for setting ordered_events queue size
2015-04-18 15:50 ` [PATCH 4/7] perf data: Add support for setting ordered_events queue size Jiri Olsa
@ 2015-05-06 2:57 ` tip-bot for Jiri Olsa
0 siblings, 0 replies; 29+ messages in thread
From: tip-bot for Jiri Olsa @ 2015-05-06 2:57 UTC (permalink / raw)
To: linux-tip-commits
Cc: bigeasy, paulus, dsahern, jolsa, jgalar, tzanussi, tglx,
fweisbec, a.p.zijlstra, namhyung, mingo, hpa, wangnan0, hekuang,
acme, linux-kernel
Commit-ID: 8fa46753fe2ca96b0fc175efb6c9ed5c001fb554
Gitweb: http://git.kernel.org/tip/8fa46753fe2ca96b0fc175efb6c9ed5c001fb554
Author: Jiri Olsa <jolsa@kernel.org>
AuthorDate: Sat, 18 Apr 2015 17:50:17 +0200
Committer: Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 29 Apr 2015 10:37:48 -0300
perf data: Add support for setting ordered_events queue size
Adding support to limit the size of ordered_events queue, so we could
control allocation size of perf data files without proper finished round
events.
Reviewed-by: David Ahern <dsahern@gmail.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jeremie Galarneau <jgalar@efficios.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1429372220-6406-5-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
tools/perf/util/data-convert-bt.c | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index 4f942b3..b35c8d6 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -74,6 +74,9 @@ struct convert {
u64 events_size;
u64 events_count;
+
+ /* Ordered events configured queue size. */
+ u64 queue_size;
};
static int value_set(struct bt_ctf_field_type *type,
@@ -968,6 +971,18 @@ static int ctf_writer__flush_streams(struct ctf_writer *cw)
return ret;
}
+static int convert__config(const char *var, const char *value, void *cb)
+{
+ struct convert *c = cb;
+
+ if (!strcmp(var, "convert.queue-size")) {
+ c->queue_size = perf_config_u64(var, value);
+ return 0;
+ }
+
+ return perf_default_config(var, value, cb);
+}
+
int bt_convert__perf2ctf(const char *input, const char *path, bool force)
{
struct perf_session *session;
@@ -994,6 +1009,8 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
struct ctf_writer *cw = &c.writer;
int err = -1;
+ perf_config(convert__config, &c);
+
/* CTF writer */
if (ctf_writer__init(cw, path))
return -1;
@@ -1003,6 +1020,11 @@ int bt_convert__perf2ctf(const char *input, const char *path, bool force)
if (!session)
goto free_writer;
+ if (c.queue_size) {
+ ordered_events__set_alloc_size(&session->ordered_events,
+ c.queue_size);
+ }
+
/* CTF writer env/clock setup */
if (ctf_writer__setup_env(cw, session))
goto free_session;
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [tip:perf/core] perf data: Fix duplicate field names and avoid reserved keywords
2015-04-18 15:50 ` [PATCH 6/7] perf data: Fix duplicate field names and avoid reserved keywords Jiri Olsa
2015-04-18 18:56 ` Jérémie Galarneau
@ 2015-05-06 2:57 ` tip-bot for Wang Nan
1 sibling, 0 replies; 29+ messages in thread
From: tip-bot for Wang Nan @ 2015-05-06 2:57 UTC (permalink / raw)
To: linux-tip-commits
Cc: mingo, wangnan0, fweisbec, tzanussi, dsahern, a.p.zijlstra,
paulus, namhyung, bigeasy, hekuang, jolsa, linux-kernel, hpa,
tglx, acme, jgalar
Commit-ID: e0a7cce5366dbd22391e09a83827ba5b4491cd2f
Gitweb: http://git.kernel.org/tip/e0a7cce5366dbd22391e09a83827ba5b4491cd2f
Author: Wang Nan <wangnan0@huawei.com>
AuthorDate: Sat, 18 Apr 2015 17:50:19 +0200
Committer: Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 29 Apr 2015 10:37:48 -0300
perf data: Fix duplicate field names and avoid reserved keywords
Some parameters of syscall tracepoints named as 'nr', 'event', etc.
When dealing with them, perf convert to ctf meets some problem:
1. If a parameter with name 'nr', it will duplicate syscall's
common field 'nr'. One such syscall is io_submit().
2. If a parameter with name 'event', it is denied to be inserted
because 'event' is a CTF spec keyword[1]. One such syscall is
epoll_ctl.
This patch appends '_dupl_X' suffix to avoid problem 1, prepend a '_'
prefix to avoid problem 2.
[1] http://diamon.org/docs/ctf/v1.8.2/#specC.1.2
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jeremie Galarneau <jgalar@efficios.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
Link: http://lkml.kernel.org/r/1429372220-6406-7-git-send-email-jolsa@kernel.org
[ changed to use format_file::alias ]
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
tools/perf/util/data-convert-bt.c | 86 +++++++++++++++++++++++++++++++++++++--
1 file changed, 82 insertions(+), 4 deletions(-)
diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index b35c8d6..8eda4ed 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -14,6 +14,7 @@
#include <babeltrace/ctf-writer/event.h>
#include <babeltrace/ctf-writer/event-types.h>
#include <babeltrace/ctf-writer/event-fields.h>
+#include <babeltrace/ctf-ir/utils.h>
#include <babeltrace/ctf/events.h>
#include <traceevent/event-parse.h>
#include "asm/bug.h"
@@ -184,6 +185,7 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
unsigned int len;
int ret;
+ name = fmtf->alias;
offset = fmtf->offset;
len = fmtf->size;
if (flags & FIELD_IS_STRING)
@@ -567,6 +569,82 @@ static int process_sample_event(struct perf_tool *tool,
return cs ? 0 : -1;
}
+/* If dup < 0, add a prefix. Else, add _dupl_X suffix. */
+static char *change_name(char *name, char *orig_name, int dup)
+{
+ char *new_name = NULL;
+ size_t len;
+
+ if (!name)
+ name = orig_name;
+
+ if (dup >= 10)
+ goto out;
+ /*
+ * Add '_' prefix to potential keywork. According to
+ * Mathieu Desnoyers (https://lkml.org/lkml/2015/1/23/652),
+ * futher CTF spec updating may require us to use '$'.
+ */
+ if (dup < 0)
+ len = strlen(name) + sizeof("_");
+ else
+ len = strlen(orig_name) + sizeof("_dupl_X");
+
+ new_name = malloc(len);
+ if (!new_name)
+ goto out;
+
+ if (dup < 0)
+ snprintf(new_name, len, "_%s", name);
+ else
+ snprintf(new_name, len, "%s_dupl_%d", orig_name, dup);
+
+out:
+ if (name != orig_name)
+ free(name);
+ return new_name;
+}
+
+static int event_class_add_field(struct bt_ctf_event_class *event_class,
+ struct bt_ctf_field_type *type,
+ struct format_field *field)
+{
+ struct bt_ctf_field_type *t = NULL;
+ char *name;
+ int dup = 1;
+ int ret;
+
+ /* alias was already assigned */
+ if (field->alias != field->name)
+ return bt_ctf_event_class_add_field(event_class, type,
+ (char *)field->alias);
+
+ name = field->name;
+
+ /* If 'name' is a keywork, add prefix. */
+ if (bt_ctf_validate_identifier(name))
+ name = change_name(name, field->name, -1);
+
+ if (!name) {
+ pr_err("Failed to fix invalid identifier.");
+ return -1;
+ }
+ while ((t = bt_ctf_event_class_get_field_by_name(event_class, name))) {
+ bt_ctf_field_type_put(t);
+ name = change_name(name, field->name, dup++);
+ if (!name) {
+ pr_err("Failed to create dup name for '%s'\n", field->name);
+ return -1;
+ }
+ }
+
+ ret = bt_ctf_event_class_add_field(event_class, type, name);
+ if (!ret)
+ field->alias = name;
+
+ return ret;
+}
+
static int add_tracepoint_fields_types(struct ctf_writer *cw,
struct format_field *fields,
struct bt_ctf_event_class *event_class)
@@ -595,14 +673,14 @@ static int add_tracepoint_fields_types(struct ctf_writer *cw,
if (flags & FIELD_IS_ARRAY)
type = bt_ctf_field_type_array_create(type, field->arraylen);
- ret = bt_ctf_event_class_add_field(event_class, type,
- field->name);
+ ret = event_class_add_field(event_class, type, field);
if (flags & FIELD_IS_ARRAY)
bt_ctf_field_type_put(type);
if (ret) {
- pr_err("Failed to add field '%s\n", field->name);
+ pr_err("Failed to add field '%s': %d\n",
+ field->name, ret);
return -1;
}
}
@@ -646,7 +724,7 @@ static int add_generic_types(struct ctf_writer *cw, struct perf_evsel *evsel,
do { \
pr2(" field '%s'\n", n); \
if (bt_ctf_event_class_add_field(cl, t, n)) { \
- pr_err("Failed to add field '%s;\n", n); \
+ pr_err("Failed to add field '%s';\n", n); \
return -1; \
} \
} while (0)
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [tip:perf/core] perf data: Fix signedness of value
2015-04-18 15:50 ` [PATCH 7/7] perf data: Fix signess of value Jiri Olsa
2015-04-20 21:23 ` Arnaldo Carvalho de Melo
@ 2015-05-06 2:58 ` tip-bot for Wang Nan
1 sibling, 0 replies; 29+ messages in thread
From: tip-bot for Wang Nan @ 2015-05-06 2:58 UTC (permalink / raw)
To: linux-tip-commits
Cc: linux-kernel, paulus, dsahern, namhyung, hekuang, wangnan0,
jgalar, mingo, tglx, tzanussi, a.p.zijlstra, hpa, jolsa, bigeasy,
acme, fweisbec
Commit-ID: d4ae42139289cbe38d5b84fa1558161d330d6e54
Gitweb: http://git.kernel.org/tip/d4ae42139289cbe38d5b84fa1558161d330d6e54
Author: Wang Nan <wangnan0@huawei.com>
AuthorDate: Sat, 18 Apr 2015 17:50:20 +0200
Committer: Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 29 Apr 2015 10:37:49 -0300
perf data: Fix signedness of value
When converting int values, perf first extractes it to a ulonglong, then
feeds it to babeltrace as a signed value.
For negative 32 bit values (for example, return values of failed
syscalls), the extracted data should be something like 0xfffffffe (-2).
It becomes a large int64 value.
Babeltrace denies to insert it with bt_ctf_field_signed_integer_set_value()
because it is larger than 0x7fffffff, the largest positive value a
32 bit int can be.
This patch introduces adjust_signedness(), which fills high bits of
ulonglong with 1 if the value is negative.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jeremie Galarneau <jgalar@efficios.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
Link: http://lkml.kernel.org/r/1429372220-6406-8-git-send-email-jolsa@kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
[ s/signess/signedness/g ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
tools/perf/util/data-convert-bt.c | 64 +++++++++++++++++++++++++++++++--------
1 file changed, 52 insertions(+), 12 deletions(-)
diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index 8eda4ed..5bfc119 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -166,6 +166,43 @@ get_tracepoint_field_type(struct ctf_writer *cw, struct format_field *field)
return cw->data.u32;
}
+static unsigned long long adjust_signedness(unsigned long long value_int, int size)
+{
+ unsigned long long value_mask;
+
+ /*
+ * value_mask = (1 << (size * 8 - 1)) - 1.
+ * Directly set value_mask for code readers.
+ */
+ switch (size) {
+ case 1:
+ value_mask = 0x7fULL;
+ break;
+ case 2:
+ value_mask = 0x7fffULL;
+ break;
+ case 4:
+ value_mask = 0x7fffffffULL;
+ break;
+ case 8:
+ /*
+ * For 64 bit value, return it self. There is no need
+ * to fill high bit.
+ */
+ /* Fall through */
+ default:
+ /* BUG! */
+ return value_int;
+ }
+
+ /* If it is a positive value, don't adjust. */
+ if ((value_int & (~0ULL - value_mask)) == 0)
+ return value_int;
+
+ /* Fill upper part of value_int with 1 to make it a negative long long. */
+ return (value_int & value_mask) | ~value_mask;
+}
+
static int add_tracepoint_field_value(struct ctf_writer *cw,
struct bt_ctf_event_class *event_class,
struct bt_ctf_event *event,
@@ -177,7 +214,6 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
struct bt_ctf_field *field;
const char *name = fmtf->name;
void *data = sample->raw_data;
- unsigned long long value_int;
unsigned long flags = fmtf->flags;
unsigned int n_items;
unsigned int i;
@@ -222,11 +258,6 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
type = get_tracepoint_field_type(cw, fmtf);
for (i = 0; i < n_items; i++) {
- if (!(flags & FIELD_IS_STRING))
- value_int = pevent_read_number(
- fmtf->event->pevent,
- data + offset + i * len, len);
-
if (flags & FIELD_IS_ARRAY)
field = bt_ctf_field_array_get_field(array_field, i);
else
@@ -240,12 +271,21 @@ static int add_tracepoint_field_value(struct ctf_writer *cw,
if (flags & FIELD_IS_STRING)
ret = bt_ctf_field_string_set_value(field,
data + offset + i * len);
- else if (!(flags & FIELD_IS_SIGNED))
- ret = bt_ctf_field_unsigned_integer_set_value(
- field, value_int);
- else
- ret = bt_ctf_field_signed_integer_set_value(
- field, value_int);
+ else {
+ unsigned long long value_int;
+
+ value_int = pevent_read_number(
+ fmtf->event->pevent,
+ data + offset + i * len, len);
+
+ if (!(flags & FIELD_IS_SIGNED))
+ ret = bt_ctf_field_unsigned_integer_set_value(
+ field, value_int);
+ else
+ ret = bt_ctf_field_signed_integer_set_value(
+ field, adjust_signedness(value_int, len));
+ }
+
if (ret) {
pr_err("failed to set file value %s\n", name);
goto err_put_field;
^ permalink raw reply related [flat|nested] 29+ messages in thread
end of thread, other threads:[~2015-05-06 2:58 UTC | newest]
Thread overview: 29+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-04-18 15:50 [PATCHv6 0/7] perf tools: Add perf data CTF conversion Jiri Olsa
2015-04-18 15:50 ` [PATCH 1/7] perf data: Show error message when conversion failed Jiri Olsa
2015-05-06 2:55 ` [tip:perf/core] " tip-bot for He Kuang
2015-04-18 15:50 ` [PATCH 2/7] perf data: Switch to multiple cpu stream files Jiri Olsa
2015-04-20 19:58 ` Arnaldo Carvalho de Melo
2015-04-20 20:01 ` Arnaldo Carvalho de Melo
2015-04-20 20:19 ` Jiri Olsa
2015-04-20 20:41 ` Arnaldo Carvalho de Melo
2015-04-20 20:49 ` Arnaldo Carvalho de Melo
2015-05-06 2:57 ` [tip:perf/core] " tip-bot for Sebastian Andrzej Siewior
2015-04-18 15:50 ` [PATCH 3/7] perf data: Enable stream flush within processing Jiri Olsa
2015-05-06 2:57 ` [tip:perf/core] " tip-bot for Jiri Olsa
2015-04-18 15:50 ` [PATCH 4/7] perf data: Add support for setting ordered_events queue size Jiri Olsa
2015-05-06 2:57 ` [tip:perf/core] " tip-bot for Jiri Olsa
2015-04-18 15:50 ` [PATCH 5/7] tools lib traceevent: Add alias field to struct format_field Jiri Olsa
2015-04-20 14:24 ` Steven Rostedt
2015-05-06 2:55 ` [tip:perf/core] " tip-bot for Jiri Olsa
2015-04-18 15:50 ` [PATCH 6/7] perf data: Fix duplicate field names and avoid reserved keywords Jiri Olsa
2015-04-18 18:56 ` Jérémie Galarneau
2015-04-18 20:45 ` Jiri Olsa
2015-04-20 14:02 ` Arnaldo Carvalho de Melo
2015-05-06 2:57 ` [tip:perf/core] " tip-bot for Wang Nan
2015-04-18 15:50 ` [PATCH 7/7] perf data: Fix signess of value Jiri Olsa
2015-04-20 21:23 ` Arnaldo Carvalho de Melo
2015-04-20 21:52 ` Jiri Olsa
2015-04-21 2:41 ` Wang Nan
2015-04-21 14:25 ` Arnaldo Carvalho de Melo
2015-05-06 2:58 ` [tip:perf/core] perf data: Fix signedness " tip-bot for Wang Nan
2015-04-20 20:05 ` [PATCHv6 0/7] perf tools: Add perf data CTF conversion Arnaldo Carvalho de Melo
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.