* Re: [PATCH v2 0/3] perf script python: Add more PMU fields
2018-06-01 9:01 [PATCH v2 0/3] perf script python: Add more PMU fields Jin Yao
@ 2018-06-01 1:55 ` Andi Kleen
2018-06-06 12:45 ` Arnaldo Carvalho de Melo
2018-06-01 9:01 ` [PATCH v2 1/3] perf script python: Move dsoname code to a new function Jin Yao
` (2 subsequent siblings)
3 siblings, 1 reply; 11+ messages in thread
From: Andi Kleen @ 2018-06-01 1:55 UTC (permalink / raw)
To: Jin Yao
Cc: acme, jolsa, peterz, mingo, alexander.shishkin, Linux-kernel,
kan.liang, yao.jin
On Fri, Jun 01, 2018 at 05:01:00PM +0800, Jin Yao wrote:
> When doing pmu sampling and then running a script with
> perf script -s script.py, the process_event function gets
> dictionary with some fields from the perf ring buffer
> (like ip, sym, callchain etc).
>
> But we miss quite a few fields we report now, for example,
> LBRs,data source,weight,transaction,iregs,uregs,and etc.
>
> This patch adds these fields for perf script python.
Patches look good to me.
Reviewed-by: Andi Kleen <ak@linux.intel.com>
-Andi
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v2 0/3] perf script python: Add more PMU fields
@ 2018-06-01 9:01 Jin Yao
2018-06-01 1:55 ` Andi Kleen
` (3 more replies)
0 siblings, 4 replies; 11+ messages in thread
From: Jin Yao @ 2018-06-01 9:01 UTC (permalink / raw)
To: acme, jolsa, peterz, mingo, alexander.shishkin
Cc: Linux-kernel, ak, kan.liang, yao.jin, Jin Yao
When doing pmu sampling and then running a script with
perf script -s script.py, the process_event function gets
dictionary with some fields from the perf ring buffer
(like ip, sym, callchain etc).
But we miss quite a few fields we report now, for example,
LBRs,data source,weight,transaction,iregs,uregs,and etc.
This patch adds these fields for perf script python.
Jin Yao (3):
perf script python: Move dsoname code to a new function
perf script python: Add more PMU fields
perf script python: Add fields introduction to Documentation
tools/perf/Documentation/perf-script-python.txt | 26 +++
.../util/scripting-engines/trace-event-python.c | 250 ++++++++++++++++++++-
2 files changed, 267 insertions(+), 9 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v2 1/3] perf script python: Move dsoname code to a new function
2018-06-01 9:01 [PATCH v2 0/3] perf script python: Add more PMU fields Jin Yao
2018-06-01 1:55 ` Andi Kleen
@ 2018-06-01 9:01 ` Jin Yao
2018-06-07 8:25 ` [tip:perf/urgent] " tip-bot for Jin Yao
2018-06-01 9:01 ` [PATCH v2 2/3] perf script python: Add more PMU fields Jin Yao
2018-06-01 9:01 ` [PATCH v2 3/3] perf script python: Add fields introduction to Documentation Jin Yao
3 siblings, 1 reply; 11+ messages in thread
From: Jin Yao @ 2018-06-01 9:01 UTC (permalink / raw)
To: acme, jolsa, peterz, mingo, alexander.shishkin
Cc: Linux-kernel, ak, kan.liang, yao.jin, Jin Yao
This patch creates a new function get_dsoname() and move the
code which gets the dsoname string to this function.
That's because in next patch, when we process LBR data, we will
also need get_dsoname() to return dsoname for branch from/to.
Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
---
.../util/scripting-engines/trace-event-python.c | 23 ++++++++++++++--------
1 file changed, 15 insertions(+), 8 deletions(-)
diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index 7f8afac..f863e96 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -372,6 +372,19 @@ static PyObject *get_field_numeric_entry(struct event_format *event,
return obj;
}
+static const char *get_dsoname(struct map *map)
+{
+ const char *dsoname = "[unknown]";
+
+ if (map && map->dso) {
+ if (symbol_conf.show_kernel_path && map->dso->long_name)
+ dsoname = map->dso->long_name;
+ else
+ dsoname = map->dso->name;
+ }
+
+ return dsoname;
+}
static PyObject *python_process_callchain(struct perf_sample *sample,
struct perf_evsel *evsel,
@@ -427,14 +440,8 @@ static PyObject *python_process_callchain(struct perf_sample *sample,
}
if (node->map) {
- struct map *map = node->map;
- const char *dsoname = "[unknown]";
- if (map && map->dso) {
- if (symbol_conf.show_kernel_path && map->dso->long_name)
- dsoname = map->dso->long_name;
- else
- dsoname = map->dso->name;
- }
+ const char *dsoname = get_dsoname(node->map);
+
pydict_set_item_string_decref(pyelem, "dso",
_PyUnicode_FromString(dsoname));
}
--
2.7.4
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 2/3] perf script python: Add more PMU fields
2018-06-01 9:01 [PATCH v2 0/3] perf script python: Add more PMU fields Jin Yao
2018-06-01 1:55 ` Andi Kleen
2018-06-01 9:01 ` [PATCH v2 1/3] perf script python: Move dsoname code to a new function Jin Yao
@ 2018-06-01 9:01 ` Jin Yao
2018-06-06 18:36 ` Arnaldo Carvalho de Melo
2018-06-07 8:25 ` [tip:perf/urgent] perf script python: Add more PMU fields to event handler dict tip-bot for Jin Yao
2018-06-01 9:01 ` [PATCH v2 3/3] perf script python: Add fields introduction to Documentation Jin Yao
3 siblings, 2 replies; 11+ messages in thread
From: Jin Yao @ 2018-06-01 9:01 UTC (permalink / raw)
To: acme, jolsa, peterz, mingo, alexander.shishkin
Cc: Linux-kernel, ak, kan.liang, yao.jin, Jin Yao
When doing pmu sampling and then running a script with
perf script -s script.py, the process_event function gets
dictionary with some fields from the perf ring buffer
(like ip, sym, callchain etc).
But we miss quite a few fields we report now, for example,
LBRs,data source,weight,transaction,iregs,uregs,and etc.
This patch reports these fields for perf script python
processing.
New created keys/items:
-----------------------
key : brstack
items: from, to, from_dsoname, to_dsoname, mispred,
predicted, in_tx, abort, cycles.
key : brstacksym
items: from, to, pred, in_tx, abort (converted string)
key : datasrc
key : datasrc_decode (decoded string)
key : iregs
key : uregs
key : weight
key : transaction
v2:
---
Add new fields for dso.
Use PyBool_FromLong() for mispred/predicted/in_tx/abort
Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
---
.../util/scripting-engines/trace-event-python.c | 227 ++++++++++++++++++++-
1 file changed, 226 insertions(+), 1 deletion(-)
diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index f863e96..9c29cfa3 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -48,6 +48,7 @@
#include "cpumap.h"
#include "print_binary.h"
#include "stat.h"
+#include "mem-events.h"
#if PY_MAJOR_VERSION < 3
#define _PyUnicode_FromString(arg) \
@@ -455,6 +456,166 @@ static PyObject *python_process_callchain(struct perf_sample *sample,
return pylist;
}
+static PyObject *python_process_brstack(struct perf_sample *sample,
+ struct thread *thread)
+{
+ struct branch_stack *br = sample->branch_stack;
+ PyObject *pylist;
+ u64 i;
+
+ pylist = PyList_New(0);
+ if (!pylist)
+ Py_FatalError("couldn't create Python list");
+
+ if (!(br && br->nr))
+ goto exit;
+
+ for (i = 0; i < br->nr; i++) {
+ PyObject *pyelem;
+ struct addr_location al;
+ const char *dsoname;
+
+ pyelem = PyDict_New();
+ if (!pyelem)
+ Py_FatalError("couldn't create Python dictionary");
+
+ pydict_set_item_string_decref(pyelem, "from",
+ PyLong_FromUnsignedLongLong(br->entries[i].from));
+ pydict_set_item_string_decref(pyelem, "to",
+ PyLong_FromUnsignedLongLong(br->entries[i].to));
+ pydict_set_item_string_decref(pyelem, "mispred",
+ PyBool_FromLong(br->entries[i].flags.mispred));
+ pydict_set_item_string_decref(pyelem, "predicted",
+ PyBool_FromLong(br->entries[i].flags.predicted));
+ pydict_set_item_string_decref(pyelem, "in_tx",
+ PyBool_FromLong(br->entries[i].flags.in_tx));
+ pydict_set_item_string_decref(pyelem, "abort",
+ PyBool_FromLong(br->entries[i].flags.abort));
+ pydict_set_item_string_decref(pyelem, "cycles",
+ PyLong_FromUnsignedLongLong(br->entries[i].flags.cycles));
+
+ thread__find_map(thread, sample->cpumode,
+ br->entries[i].from, &al);
+ dsoname = get_dsoname(al.map);
+ pydict_set_item_string_decref(pyelem, "from_dsoname",
+ _PyUnicode_FromString(dsoname));
+
+ thread__find_map(thread, sample->cpumode,
+ br->entries[i].to, &al);
+ dsoname = get_dsoname(al.map);
+ pydict_set_item_string_decref(pyelem, "to_dsoname",
+ _PyUnicode_FromString(dsoname));
+
+ PyList_Append(pylist, pyelem);
+ Py_DECREF(pyelem);
+ }
+
+exit:
+ return pylist;
+}
+
+static unsigned long get_offset(struct symbol *sym, struct addr_location *al)
+{
+ unsigned long offset;
+
+ if (al->addr < sym->end)
+ offset = al->addr - sym->start;
+ else
+ offset = al->addr - al->map->start - sym->start;
+
+ return offset;
+}
+
+static int get_symoff(struct symbol *sym, struct addr_location *al,
+ bool print_off, char *bf, int size)
+{
+ unsigned long offset;
+
+ if (!sym || !sym->name)
+ return scnprintf(bf, size, "%s", "[unknown]");
+
+ if (!print_off)
+ return scnprintf(bf, size, "%s", sym->name);
+
+ offset = get_offset(sym, al);
+
+ return scnprintf(bf, size, "%s+0x%x", sym->name, offset);
+}
+
+static int get_br_mspred(struct branch_flags *flags, char *bf, int size)
+{
+ if (!flags->mispred && !flags->predicted)
+ return scnprintf(bf, size, "%s", "-");
+
+ if (flags->mispred)
+ return scnprintf(bf, size, "%s", "M");
+
+ return scnprintf(bf, size, "%s", "P");
+}
+
+static PyObject *python_process_brstacksym(struct perf_sample *sample,
+ struct thread *thread)
+{
+ struct branch_stack *br = sample->branch_stack;
+ PyObject *pylist;
+ u64 i;
+ char bf[512];
+ struct addr_location al;
+
+ pylist = PyList_New(0);
+ if (!pylist)
+ Py_FatalError("couldn't create Python list");
+
+ if (!(br && br->nr))
+ goto exit;
+
+ for (i = 0; i < br->nr; i++) {
+ PyObject *pyelem;
+
+ pyelem = PyDict_New();
+ if (!pyelem)
+ Py_FatalError("couldn't create Python dictionary");
+
+ thread__find_symbol(thread, sample->cpumode,
+ br->entries[i].from, &al);
+ get_symoff(al.sym, &al, true, bf, sizeof(bf));
+ pydict_set_item_string_decref(pyelem, "from",
+ _PyUnicode_FromString(bf));
+
+ thread__find_symbol(thread, sample->cpumode,
+ br->entries[i].to, &al);
+ get_symoff(al.sym, &al, true, bf, sizeof(bf));
+ pydict_set_item_string_decref(pyelem, "to",
+ _PyUnicode_FromString(bf));
+
+ get_br_mspred(&br->entries[i].flags, bf, sizeof(bf));
+ pydict_set_item_string_decref(pyelem, "pred",
+ _PyUnicode_FromString(bf));
+
+ if (br->entries[i].flags.in_tx) {
+ pydict_set_item_string_decref(pyelem, "in_tx",
+ _PyUnicode_FromString("X"));
+ } else {
+ pydict_set_item_string_decref(pyelem, "in_tx",
+ _PyUnicode_FromString("-"));
+ }
+
+ if (br->entries[i].flags.abort) {
+ pydict_set_item_string_decref(pyelem, "abort",
+ _PyUnicode_FromString("A"));
+ } else {
+ pydict_set_item_string_decref(pyelem, "abort",
+ _PyUnicode_FromString("-"));
+ }
+
+ PyList_Append(pylist, pyelem);
+ Py_DECREF(pyelem);
+ }
+
+exit:
+ return pylist;
+}
+
static PyObject *get_sample_value_as_tuple(struct sample_read_value *value)
{
PyObject *t;
@@ -505,12 +666,63 @@ static void set_sample_read_in_dict(PyObject *dict_sample,
pydict_set_item_string_decref(dict_sample, "values", values);
}
+static void set_sample_datasrc_in_dict(PyObject *dict,
+ struct perf_sample *sample)
+{
+ struct mem_info mi = { .data_src.val = sample->data_src };
+ char decode[100];
+
+ pydict_set_item_string_decref(dict, "datasrc",
+ PyLong_FromUnsignedLongLong(sample->data_src));
+
+ perf_script__meminfo_scnprintf(decode, 100, &mi);
+
+ pydict_set_item_string_decref(dict, "datasrc_decode",
+ _PyUnicode_FromString(decode));
+}
+
+static int regs_map(struct regs_dump *regs, uint64_t mask, char *bf, int size)
+{
+ unsigned int i = 0, r;
+ int printed = 0;
+
+ bf[0] = 0;
+
+ for_each_set_bit(r, (unsigned long *) &mask, sizeof(mask) * 8) {
+ u64 val = regs->regs[i++];
+
+ printed += scnprintf(bf + printed, size - printed,
+ "%5s:0x%" PRIx64 " ",
+ perf_reg_name(r), val);
+ }
+
+ return printed;
+}
+
+static void set_regs_in_dict(PyObject *dict,
+ struct perf_sample *sample,
+ struct perf_evsel *evsel)
+{
+ struct perf_event_attr *attr = &evsel->attr;
+ char bf[512];
+
+ regs_map(&sample->intr_regs, attr->sample_regs_intr, bf, sizeof(bf));
+
+ pydict_set_item_string_decref(dict, "iregs",
+ _PyUnicode_FromString(bf));
+
+ regs_map(&sample->user_regs, attr->sample_regs_user, bf, sizeof(bf));
+
+ pydict_set_item_string_decref(dict, "uregs",
+ _PyUnicode_FromString(bf));
+}
+
static PyObject *get_perf_sample_dict(struct perf_sample *sample,
struct perf_evsel *evsel,
struct addr_location *al,
PyObject *callchain)
{
- PyObject *dict, *dict_sample;
+ PyObject *dict, *dict_sample, *brstack, *brstacksym;
dict = PyDict_New();
if (!dict)
@@ -541,6 +753,11 @@ static PyObject *get_perf_sample_dict(struct perf_sample *sample,
pydict_set_item_string_decref(dict_sample, "addr",
PyLong_FromUnsignedLongLong(sample->addr));
set_sample_read_in_dict(dict_sample, sample, evsel);
+ pydict_set_item_string_decref(dict_sample, "weight",
+ PyLong_FromUnsignedLongLong(sample->weight));
+ pydict_set_item_string_decref(dict_sample, "transaction",
+ PyLong_FromUnsignedLongLong(sample->transaction));
+ set_sample_datasrc_in_dict(dict_sample, sample);
pydict_set_item_string_decref(dict, "sample", dict_sample);
pydict_set_item_string_decref(dict, "raw_buf", _PyBytes_FromStringAndSize(
@@ -558,6 +775,14 @@ static PyObject *get_perf_sample_dict(struct perf_sample *sample,
pydict_set_item_string_decref(dict, "callchain", callchain);
+ brstack = python_process_brstack(sample, al->thread);
+ pydict_set_item_string_decref(dict, "brstack", brstack);
+
+ brstacksym = python_process_brstacksym(sample, al->thread);
+ pydict_set_item_string_decref(dict, "brstacksym", brstacksym);
+
+ set_regs_in_dict(dict, sample, evsel);
+
return dict;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 3/3] perf script python: Add fields introduction to Documentation
2018-06-01 9:01 [PATCH v2 0/3] perf script python: Add more PMU fields Jin Yao
` (2 preceding siblings ...)
2018-06-01 9:01 ` [PATCH v2 2/3] perf script python: Add more PMU fields Jin Yao
@ 2018-06-01 9:01 ` Jin Yao
2018-06-07 8:26 ` [tip:perf/urgent] perf script python: Add dict " tip-bot for Jin Yao
3 siblings, 1 reply; 11+ messages in thread
From: Jin Yao @ 2018-06-01 9:01 UTC (permalink / raw)
To: acme, jolsa, peterz, mingo, alexander.shishkin
Cc: Linux-kernel, ak, kan.liang, yao.jin, Jin Yao
Add the brief introduction of fields to perf-script-python.txt.
It may help python script developer easily finding what fields
are supported.
Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
---
tools/perf/Documentation/perf-script-python.txt | 26 +++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/tools/perf/Documentation/perf-script-python.txt b/tools/perf/Documentation/perf-script-python.txt
index 51ec2d2..0fb9eda 100644
--- a/tools/perf/Documentation/perf-script-python.txt
+++ b/tools/perf/Documentation/perf-script-python.txt
@@ -610,6 +610,32 @@ Various utility functions for use with perf script:
nsecs_str(nsecs) - returns printable string in the form secs.nsecs
avg(total, n) - returns average given a sum and a total number of values
+SUPPORTED FIELDS
+----------------
+
+Currently supported fields:
+
+ev_name, comm, pid, tid, cpu, ip, time, period, phys_addr, addr,
+symbol, dso, time_enabled, time_running, values, callchain,
+brstack, brstacksym, datasrc, datasrc_decode, iregs, uregs,
+weight, transaction, raw_buf, attr.
+
+Some fields have sub items:
+
+brstack:
+ from, to, from_dsoname, to_dsoname, mispred,
+ predicted, in_tx, abort, cycles.
+
+brstacksym:
+ items: from, to, pred, in_tx, abort (converted string)
+
+For example,
+We can use this code to print brstack "from", "to", "cycles".
+
+if 'brstack' in dict:
+ for entry in dict['brstack']:
+ print "from %s, to %s, cycles %s" % (entry["from"], entry["to"], entry["cycles"])
+
SEE ALSO
--------
linkperf:perf-script[1]
--
2.7.4
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v2 0/3] perf script python: Add more PMU fields
2018-06-01 1:55 ` Andi Kleen
@ 2018-06-06 12:45 ` Arnaldo Carvalho de Melo
0 siblings, 0 replies; 11+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-06 12:45 UTC (permalink / raw)
To: Andi Kleen
Cc: Jin Yao, jolsa, peterz, mingo, alexander.shishkin, Linux-kernel,
kan.liang, yao.jin
Em Thu, May 31, 2018 at 06:55:01PM -0700, Andi Kleen escreveu:
> On Fri, Jun 01, 2018 at 05:01:00PM +0800, Jin Yao wrote:
> > When doing pmu sampling and then running a script with
> > perf script -s script.py, the process_event function gets
> > dictionary with some fields from the perf ring buffer
> > (like ip, sym, callchain etc).
> >
> > But we miss quite a few fields we report now, for example,
> > LBRs,data source,weight,transaction,iregs,uregs,and etc.
> >
> > This patch adds these fields for perf script python.
>
> Patches look good to me.
>
> Reviewed-by: Andi Kleen <ak@linux.intel.com>
Thanks, applied.
- Arnaldo
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 2/3] perf script python: Add more PMU fields
2018-06-01 9:01 ` [PATCH v2 2/3] perf script python: Add more PMU fields Jin Yao
@ 2018-06-06 18:36 ` Arnaldo Carvalho de Melo
2018-06-07 1:00 ` Jin, Yao
2018-06-07 8:25 ` [tip:perf/urgent] perf script python: Add more PMU fields to event handler dict tip-bot for Jin Yao
1 sibling, 1 reply; 11+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-06 18:36 UTC (permalink / raw)
To: Jin Yao
Cc: Jiri Olsa, Peter Zijlstra, Ingo Molnar, Alexander Shishkin,
Linux Kernel Mailing List, Andi Kleen, Kan Liang
Em Fri, Jun 01, 2018 at 05:01:02PM +0800, Jin Yao escreveu:
<SNIP>
> +static int get_symoff(struct symbol *sym, struct addr_location *al,
> + bool print_off, char *bf, int size)
> +{
> + unsigned long offset;
> +
> + if (!sym || !sym->name)
> + return scnprintf(bf, size, "%s", "[unknown]");
52 54.22 ubuntu:17.04 : FAIL gcc (Ubuntu 6.3.0-12ubuntu2) 6.3.0 20170406
CC /tmp/build/perf/util/scripting-engines/trace-event-python.o
util/scripting-engines/trace-event-python.c:534:20: error: address of array 'sym->name' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
if (!sym || !sym->name)
~~~~~~^~~~
1 error generated.
mv: cannot stat '/tmp/build/perf/util/scripting-engines/.trace-event-python.o.tmp': No such file or directory
/git/linux/tools/build/Makefile.build:96: recipe for target '/tmp/build/perf/util/scripting-engines/trace-event-python.o' failed
make[5]: *** [/tmp/build/perf/util/scripting-engines/trace-event-python.o] Error 1
Because:
struct symbol {
struct rb_node rb_node;
u64 start;
u64 end;
u16 namelen;
<SNIP>
char name[0];
};
It sym->name is not a pointer, in symbol's constructor we have:
struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *name)
{
size_t namelen = strlen(name) + 1;
struct symbol *sym = calloc(1, (symbol_conf.priv_size +
sizeof(*sym) + namelen));
if (sym == NULL)
return NULL;
<SNIP>
sym->namelen = namelen - 1;
memcpy(sym->name, name, namelen);
return sym;
}
So it is at least 1 char long, the test above should be:
if (!sym || !sym->name[0])
return scnprintf(bf, size, "%s", "[unknown]");
I'm fixing this up here.
- Arnaldo
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 2/3] perf script python: Add more PMU fields
2018-06-06 18:36 ` Arnaldo Carvalho de Melo
@ 2018-06-07 1:00 ` Jin, Yao
0 siblings, 0 replies; 11+ messages in thread
From: Jin, Yao @ 2018-06-07 1:00 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: Jiri Olsa, Peter Zijlstra, Ingo Molnar, Alexander Shishkin,
Linux Kernel Mailing List, Andi Kleen, Kan Liang
On 6/7/2018 2:36 AM, Arnaldo Carvalho de Melo wrote:
> Em Fri, Jun 01, 2018 at 05:01:02PM +0800, Jin Yao escreveu:
>
> <SNIP>
>
>> +static int get_symoff(struct symbol *sym, struct addr_location *al,
>> + bool print_off, char *bf, int size)
>> +{
>> + unsigned long offset;
>> +
>> + if (!sym || !sym->name)
>> + return scnprintf(bf, size, "%s", "[unknown]");
>
> 52 54.22 ubuntu:17.04 : FAIL gcc (Ubuntu 6.3.0-12ubuntu2) 6.3.0 20170406
>
> CC /tmp/build/perf/util/scripting-engines/trace-event-python.o
> util/scripting-engines/trace-event-python.c:534:20: error: address of array 'sym->name' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
> if (!sym || !sym->name)
> ~~~~~~^~~~
> 1 error generated.
> mv: cannot stat '/tmp/build/perf/util/scripting-engines/.trace-event-python.o.tmp': No such file or directory
> /git/linux/tools/build/Makefile.build:96: recipe for target '/tmp/build/perf/util/scripting-engines/trace-event-python.o' failed
> make[5]: *** [/tmp/build/perf/util/scripting-engines/trace-event-python.o] Error 1
>
> Because:
>
> struct symbol {
> struct rb_node rb_node;
> u64 start;
> u64 end;
> u16 namelen;
> <SNIP>
> char name[0];
> };
>
> It sym->name is not a pointer, in symbol's constructor we have:
>
> struct symbol *symbol__new(u64 start, u64 len, u8 binding, u8 type, const char *name)
> {
> size_t namelen = strlen(name) + 1;
> struct symbol *sym = calloc(1, (symbol_conf.priv_size +
> sizeof(*sym) + namelen));
> if (sym == NULL)
> return NULL;
> <SNIP>
> sym->namelen = namelen - 1;
> memcpy(sym->name, name, namelen);
>
> return sym;
> }
>
> So it is at least 1 char long, the test above should be:
>
> if (!sym || !sym->name[0])
> return scnprintf(bf, size, "%s", "[unknown]");
>
> I'm fixing this up here.
>
> - Arnaldo
>
Oh, yes, my mistake, very sorry about that.
Thanks for helping me to fix this.
Thanks
Jin Yao
^ permalink raw reply [flat|nested] 11+ messages in thread
* [tip:perf/urgent] perf script python: Move dsoname code to a new function
2018-06-01 9:01 ` [PATCH v2 1/3] perf script python: Move dsoname code to a new function Jin Yao
@ 2018-06-07 8:25 ` tip-bot for Jin Yao
0 siblings, 0 replies; 11+ messages in thread
From: tip-bot for Jin Yao @ 2018-06-07 8:25 UTC (permalink / raw)
To: linux-tip-commits
Cc: ak, hpa, acme, kan.liang, yao.jin, linux-kernel,
alexander.shishkin, tglx, peterz, jolsa, mingo
Commit-ID: 5f9e0f3158a5cd0ef7bb205b9f1826b2ec1893a9
Gitweb: https://git.kernel.org/tip/5f9e0f3158a5cd0ef7bb205b9f1826b2ec1893a9
Author: Jin Yao <yao.jin@linux.intel.com>
AuthorDate: Fri, 1 Jun 2018 17:01:01 +0800
Committer: Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 6 Jun 2018 12:52:09 -0300
perf script python: Move dsoname code to a new function
This patch creates a new function get_dsoname() and move the code which
gets the dsoname string to this function.
That's because in next patch, when we process LBR data, we will also
need get_dsoname() to return dsoname for branch from/to.
Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1527843663-32288-2-git-send-email-yao.jin@linux.intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
.../util/scripting-engines/trace-event-python.c | 23 ++++++++++++++--------
1 file changed, 15 insertions(+), 8 deletions(-)
diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index 7f8afacd08ee..f863e96fb7bc 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -372,6 +372,19 @@ static PyObject *get_field_numeric_entry(struct event_format *event,
return obj;
}
+static const char *get_dsoname(struct map *map)
+{
+ const char *dsoname = "[unknown]";
+
+ if (map && map->dso) {
+ if (symbol_conf.show_kernel_path && map->dso->long_name)
+ dsoname = map->dso->long_name;
+ else
+ dsoname = map->dso->name;
+ }
+
+ return dsoname;
+}
static PyObject *python_process_callchain(struct perf_sample *sample,
struct perf_evsel *evsel,
@@ -427,14 +440,8 @@ static PyObject *python_process_callchain(struct perf_sample *sample,
}
if (node->map) {
- struct map *map = node->map;
- const char *dsoname = "[unknown]";
- if (map && map->dso) {
- if (symbol_conf.show_kernel_path && map->dso->long_name)
- dsoname = map->dso->long_name;
- else
- dsoname = map->dso->name;
- }
+ const char *dsoname = get_dsoname(node->map);
+
pydict_set_item_string_decref(pyelem, "dso",
_PyUnicode_FromString(dsoname));
}
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [tip:perf/urgent] perf script python: Add more PMU fields to event handler dict
2018-06-01 9:01 ` [PATCH v2 2/3] perf script python: Add more PMU fields Jin Yao
2018-06-06 18:36 ` Arnaldo Carvalho de Melo
@ 2018-06-07 8:25 ` tip-bot for Jin Yao
1 sibling, 0 replies; 11+ messages in thread
From: tip-bot for Jin Yao @ 2018-06-07 8:25 UTC (permalink / raw)
To: linux-tip-commits
Cc: jolsa, yao.jin, mingo, tglx, peterz, kan.liang, hpa,
linux-kernel, alexander.shishkin, acme, yao.jin, ak
Commit-ID: 48a1f565261d2ab1e17f9a3ad532cf6d9e07748d
Gitweb: https://git.kernel.org/tip/48a1f565261d2ab1e17f9a3ad532cf6d9e07748d
Author: Jin Yao <yao.jin@linux.intel.com>
AuthorDate: Fri, 1 Jun 2018 17:01:02 +0800
Committer: Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 6 Jun 2018 15:38:26 -0300
perf script python: Add more PMU fields to event handler dict
When doing pmu sampling and then running a script with perf script -s
script.py, the process_event function gets dictionary with some fields
from the perf ring buffer (like ip, sym, callchain etc).
But we miss quite a few fields we report now, for example, LBRs, data
source, weight, transaction, iregs, uregs, etc.
This patch reports these fields for perf script python processing.
New keys/items:
---------------
key : brstack
items: from, to, from_dsoname, to_dsoname, mispred,
predicted, in_tx, abort, cycles.
key : brstacksym
items: from, to, pred, in_tx, abort (converted string)
key : datasrc
key : datasrc_decode (decoded string)
key : iregs
key : uregs
key : weight
key : transaction
v2:
---
Add new fields for dso.
Use PyBool_FromLong() for mispred/predicted/in_tx/abort
Committer notes:
!sym->name isn't valid, as its not a pointer, its a [0] array, use
!sym->name[0] instead, guaranteed to be the case by symbol__new.
This was caught by just one of the containers:
52 54.22 ubuntu:17.04 : FAIL gcc (Ubuntu 6.3.0-12ubuntu2) 6.3.0 20170406
CC /tmp/build/perf/util/scripting-engines/trace-event-python.o
util/scripting-engines/trace-event-python.c:534:20: error: address of array 'sym->name' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
if (!sym || !sym->name)
~~~~~~^~~~
1 error generated.
mv: cannot stat '/tmp/build/perf/util/scripting-engines/.trace-event-python.o.tmp': No such file or directory
/git/linux/tools/build/Makefile.build:96: recipe for target '/tmp/build/perf/util/scripting-engines/trace-event-python.o' failed
make[5]: *** [/tmp/build/perf/util/scripting-engines/trace-event-python.o] Error 1
Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Jin Yao <yao.jin@intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1527843663-32288-3-git-send-email-yao.jin@linux.intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
.../util/scripting-engines/trace-event-python.c | 227 ++++++++++++++++++++-
1 file changed, 226 insertions(+), 1 deletion(-)
diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index f863e96fb7bc..46e9e19ab1ac 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -48,6 +48,7 @@
#include "cpumap.h"
#include "print_binary.h"
#include "stat.h"
+#include "mem-events.h"
#if PY_MAJOR_VERSION < 3
#define _PyUnicode_FromString(arg) \
@@ -455,6 +456,166 @@ exit:
return pylist;
}
+static PyObject *python_process_brstack(struct perf_sample *sample,
+ struct thread *thread)
+{
+ struct branch_stack *br = sample->branch_stack;
+ PyObject *pylist;
+ u64 i;
+
+ pylist = PyList_New(0);
+ if (!pylist)
+ Py_FatalError("couldn't create Python list");
+
+ if (!(br && br->nr))
+ goto exit;
+
+ for (i = 0; i < br->nr; i++) {
+ PyObject *pyelem;
+ struct addr_location al;
+ const char *dsoname;
+
+ pyelem = PyDict_New();
+ if (!pyelem)
+ Py_FatalError("couldn't create Python dictionary");
+
+ pydict_set_item_string_decref(pyelem, "from",
+ PyLong_FromUnsignedLongLong(br->entries[i].from));
+ pydict_set_item_string_decref(pyelem, "to",
+ PyLong_FromUnsignedLongLong(br->entries[i].to));
+ pydict_set_item_string_decref(pyelem, "mispred",
+ PyBool_FromLong(br->entries[i].flags.mispred));
+ pydict_set_item_string_decref(pyelem, "predicted",
+ PyBool_FromLong(br->entries[i].flags.predicted));
+ pydict_set_item_string_decref(pyelem, "in_tx",
+ PyBool_FromLong(br->entries[i].flags.in_tx));
+ pydict_set_item_string_decref(pyelem, "abort",
+ PyBool_FromLong(br->entries[i].flags.abort));
+ pydict_set_item_string_decref(pyelem, "cycles",
+ PyLong_FromUnsignedLongLong(br->entries[i].flags.cycles));
+
+ thread__find_map(thread, sample->cpumode,
+ br->entries[i].from, &al);
+ dsoname = get_dsoname(al.map);
+ pydict_set_item_string_decref(pyelem, "from_dsoname",
+ _PyUnicode_FromString(dsoname));
+
+ thread__find_map(thread, sample->cpumode,
+ br->entries[i].to, &al);
+ dsoname = get_dsoname(al.map);
+ pydict_set_item_string_decref(pyelem, "to_dsoname",
+ _PyUnicode_FromString(dsoname));
+
+ PyList_Append(pylist, pyelem);
+ Py_DECREF(pyelem);
+ }
+
+exit:
+ return pylist;
+}
+
+static unsigned long get_offset(struct symbol *sym, struct addr_location *al)
+{
+ unsigned long offset;
+
+ if (al->addr < sym->end)
+ offset = al->addr - sym->start;
+ else
+ offset = al->addr - al->map->start - sym->start;
+
+ return offset;
+}
+
+static int get_symoff(struct symbol *sym, struct addr_location *al,
+ bool print_off, char *bf, int size)
+{
+ unsigned long offset;
+
+ if (!sym || !sym->name[0])
+ return scnprintf(bf, size, "%s", "[unknown]");
+
+ if (!print_off)
+ return scnprintf(bf, size, "%s", sym->name);
+
+ offset = get_offset(sym, al);
+
+ return scnprintf(bf, size, "%s+0x%x", sym->name, offset);
+}
+
+static int get_br_mspred(struct branch_flags *flags, char *bf, int size)
+{
+ if (!flags->mispred && !flags->predicted)
+ return scnprintf(bf, size, "%s", "-");
+
+ if (flags->mispred)
+ return scnprintf(bf, size, "%s", "M");
+
+ return scnprintf(bf, size, "%s", "P");
+}
+
+static PyObject *python_process_brstacksym(struct perf_sample *sample,
+ struct thread *thread)
+{
+ struct branch_stack *br = sample->branch_stack;
+ PyObject *pylist;
+ u64 i;
+ char bf[512];
+ struct addr_location al;
+
+ pylist = PyList_New(0);
+ if (!pylist)
+ Py_FatalError("couldn't create Python list");
+
+ if (!(br && br->nr))
+ goto exit;
+
+ for (i = 0; i < br->nr; i++) {
+ PyObject *pyelem;
+
+ pyelem = PyDict_New();
+ if (!pyelem)
+ Py_FatalError("couldn't create Python dictionary");
+
+ thread__find_symbol(thread, sample->cpumode,
+ br->entries[i].from, &al);
+ get_symoff(al.sym, &al, true, bf, sizeof(bf));
+ pydict_set_item_string_decref(pyelem, "from",
+ _PyUnicode_FromString(bf));
+
+ thread__find_symbol(thread, sample->cpumode,
+ br->entries[i].to, &al);
+ get_symoff(al.sym, &al, true, bf, sizeof(bf));
+ pydict_set_item_string_decref(pyelem, "to",
+ _PyUnicode_FromString(bf));
+
+ get_br_mspred(&br->entries[i].flags, bf, sizeof(bf));
+ pydict_set_item_string_decref(pyelem, "pred",
+ _PyUnicode_FromString(bf));
+
+ if (br->entries[i].flags.in_tx) {
+ pydict_set_item_string_decref(pyelem, "in_tx",
+ _PyUnicode_FromString("X"));
+ } else {
+ pydict_set_item_string_decref(pyelem, "in_tx",
+ _PyUnicode_FromString("-"));
+ }
+
+ if (br->entries[i].flags.abort) {
+ pydict_set_item_string_decref(pyelem, "abort",
+ _PyUnicode_FromString("A"));
+ } else {
+ pydict_set_item_string_decref(pyelem, "abort",
+ _PyUnicode_FromString("-"));
+ }
+
+ PyList_Append(pylist, pyelem);
+ Py_DECREF(pyelem);
+ }
+
+exit:
+ return pylist;
+}
+
static PyObject *get_sample_value_as_tuple(struct sample_read_value *value)
{
PyObject *t;
@@ -505,12 +666,63 @@ static void set_sample_read_in_dict(PyObject *dict_sample,
pydict_set_item_string_decref(dict_sample, "values", values);
}
+static void set_sample_datasrc_in_dict(PyObject *dict,
+ struct perf_sample *sample)
+{
+ struct mem_info mi = { .data_src.val = sample->data_src };
+ char decode[100];
+
+ pydict_set_item_string_decref(dict, "datasrc",
+ PyLong_FromUnsignedLongLong(sample->data_src));
+
+ perf_script__meminfo_scnprintf(decode, 100, &mi);
+
+ pydict_set_item_string_decref(dict, "datasrc_decode",
+ _PyUnicode_FromString(decode));
+}
+
+static int regs_map(struct regs_dump *regs, uint64_t mask, char *bf, int size)
+{
+ unsigned int i = 0, r;
+ int printed = 0;
+
+ bf[0] = 0;
+
+ for_each_set_bit(r, (unsigned long *) &mask, sizeof(mask) * 8) {
+ u64 val = regs->regs[i++];
+
+ printed += scnprintf(bf + printed, size - printed,
+ "%5s:0x%" PRIx64 " ",
+ perf_reg_name(r), val);
+ }
+
+ return printed;
+}
+
+static void set_regs_in_dict(PyObject *dict,
+ struct perf_sample *sample,
+ struct perf_evsel *evsel)
+{
+ struct perf_event_attr *attr = &evsel->attr;
+ char bf[512];
+
+ regs_map(&sample->intr_regs, attr->sample_regs_intr, bf, sizeof(bf));
+
+ pydict_set_item_string_decref(dict, "iregs",
+ _PyUnicode_FromString(bf));
+
+ regs_map(&sample->user_regs, attr->sample_regs_user, bf, sizeof(bf));
+
+ pydict_set_item_string_decref(dict, "uregs",
+ _PyUnicode_FromString(bf));
+}
+
static PyObject *get_perf_sample_dict(struct perf_sample *sample,
struct perf_evsel *evsel,
struct addr_location *al,
PyObject *callchain)
{
- PyObject *dict, *dict_sample;
+ PyObject *dict, *dict_sample, *brstack, *brstacksym;
dict = PyDict_New();
if (!dict)
@@ -541,6 +753,11 @@ static PyObject *get_perf_sample_dict(struct perf_sample *sample,
pydict_set_item_string_decref(dict_sample, "addr",
PyLong_FromUnsignedLongLong(sample->addr));
set_sample_read_in_dict(dict_sample, sample, evsel);
+ pydict_set_item_string_decref(dict_sample, "weight",
+ PyLong_FromUnsignedLongLong(sample->weight));
+ pydict_set_item_string_decref(dict_sample, "transaction",
+ PyLong_FromUnsignedLongLong(sample->transaction));
+ set_sample_datasrc_in_dict(dict_sample, sample);
pydict_set_item_string_decref(dict, "sample", dict_sample);
pydict_set_item_string_decref(dict, "raw_buf", _PyBytes_FromStringAndSize(
@@ -558,6 +775,14 @@ static PyObject *get_perf_sample_dict(struct perf_sample *sample,
pydict_set_item_string_decref(dict, "callchain", callchain);
+ brstack = python_process_brstack(sample, al->thread);
+ pydict_set_item_string_decref(dict, "brstack", brstack);
+
+ brstacksym = python_process_brstacksym(sample, al->thread);
+ pydict_set_item_string_decref(dict, "brstacksym", brstacksym);
+
+ set_regs_in_dict(dict, sample, evsel);
+
return dict;
}
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [tip:perf/urgent] perf script python: Add dict fields introduction to Documentation
2018-06-01 9:01 ` [PATCH v2 3/3] perf script python: Add fields introduction to Documentation Jin Yao
@ 2018-06-07 8:26 ` tip-bot for Jin Yao
0 siblings, 0 replies; 11+ messages in thread
From: tip-bot for Jin Yao @ 2018-06-07 8:26 UTC (permalink / raw)
To: linux-tip-commits
Cc: ak, tglx, yao.jin, mingo, jolsa, hpa, linux-kernel, peterz,
yao.jin, kan.liang, acme, alexander.shishkin
Commit-ID: ac56aa4549cdfd9c56387b35e99e3c868cfc7bd0
Gitweb: https://git.kernel.org/tip/ac56aa4549cdfd9c56387b35e99e3c868cfc7bd0
Author: Jin Yao <yao.jin@linux.intel.com>
AuthorDate: Fri, 1 Jun 2018 17:01:03 +0800
Committer: Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Wed, 6 Jun 2018 15:40:10 -0300
perf script python: Add dict fields introduction to Documentation
Add a brief introduction about fields to perf-script-python.txt.
It should help python script developers in easily finding what fields
are supported.
Signed-off-by: Jin Yao <yao.jin@linux.intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Jin Yao <yao.jin@intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1527843663-32288-4-git-send-email-yao.jin@linux.intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
tools/perf/Documentation/perf-script-python.txt | 26 +++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/tools/perf/Documentation/perf-script-python.txt b/tools/perf/Documentation/perf-script-python.txt
index 51ec2d20068a..0fb9eda3cbca 100644
--- a/tools/perf/Documentation/perf-script-python.txt
+++ b/tools/perf/Documentation/perf-script-python.txt
@@ -610,6 +610,32 @@ Various utility functions for use with perf script:
nsecs_str(nsecs) - returns printable string in the form secs.nsecs
avg(total, n) - returns average given a sum and a total number of values
+SUPPORTED FIELDS
+----------------
+
+Currently supported fields:
+
+ev_name, comm, pid, tid, cpu, ip, time, period, phys_addr, addr,
+symbol, dso, time_enabled, time_running, values, callchain,
+brstack, brstacksym, datasrc, datasrc_decode, iregs, uregs,
+weight, transaction, raw_buf, attr.
+
+Some fields have sub items:
+
+brstack:
+ from, to, from_dsoname, to_dsoname, mispred,
+ predicted, in_tx, abort, cycles.
+
+brstacksym:
+ items: from, to, pred, in_tx, abort (converted string)
+
+For example,
+We can use this code to print brstack "from", "to", "cycles".
+
+if 'brstack' in dict:
+ for entry in dict['brstack']:
+ print "from %s, to %s, cycles %s" % (entry["from"], entry["to"], entry["cycles"])
+
SEE ALSO
--------
linkperf:perf-script[1]
^ permalink raw reply related [flat|nested] 11+ messages in thread
end of thread, other threads:[~2018-06-07 8:27 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-01 9:01 [PATCH v2 0/3] perf script python: Add more PMU fields Jin Yao
2018-06-01 1:55 ` Andi Kleen
2018-06-06 12:45 ` Arnaldo Carvalho de Melo
2018-06-01 9:01 ` [PATCH v2 1/3] perf script python: Move dsoname code to a new function Jin Yao
2018-06-07 8:25 ` [tip:perf/urgent] " tip-bot for Jin Yao
2018-06-01 9:01 ` [PATCH v2 2/3] perf script python: Add more PMU fields Jin Yao
2018-06-06 18:36 ` Arnaldo Carvalho de Melo
2018-06-07 1:00 ` Jin, Yao
2018-06-07 8:25 ` [tip:perf/urgent] perf script python: Add more PMU fields to event handler dict tip-bot for Jin Yao
2018-06-01 9:01 ` [PATCH v2 3/3] perf script python: Add fields introduction to Documentation Jin Yao
2018-06-07 8:26 ` [tip:perf/urgent] perf script python: Add dict " tip-bot for Jin Yao
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).