[PATCH 1/2] perf script: move the number processing into its own function

[PATCH 1/2] perf script: move the number processing into its own function

[Sebastian Andrzej Siewior]

I was going to change something here and the result was so much on the
right side of the screen that I decided to move that piece into its own
function.
This patch should make no function change except the moving the code
into its own function.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 .../util/scripting-engines/trace-event-python.c    | 38 +++++++++++++---------
 1 file changed, 23 insertions(+), 15 deletions(-)

diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index cd9774d..aa11790 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -230,6 +230,28 @@ static inline struct event_format *find_cache_event(struct perf_evsel *evsel)
 	return event;
 }
 
+static PyObject *get_field_numeric_entry(struct event_format *event,
+		struct format_field *field, void *data)
+{
+	PyObject *obj;
+	unsigned long long val;
+
+	val = read_size(event, data + field->offset, field->size);
+	if (field->flags & FIELD_IS_SIGNED) {
+		if ((long long)val >= LONG_MIN &&
+				(long long)val <= LONG_MAX)
+			obj = PyInt_FromLong(val);
+		else
+			obj = PyLong_FromLongLong(val);
+	} else {
+		if (val <= LONG_MAX)
+			obj = PyInt_FromLong(val);
+		else
+			obj = PyLong_FromUnsignedLongLong(val);
+	}
+	return obj;
+}
+
 static void python_process_tracepoint(struct perf_sample *sample,
 				      struct perf_evsel *evsel,
 				      struct thread *thread,
@@ -238,7 +260,6 @@ static void python_process_tracepoint(struct perf_sample *sample,
 	PyObject *handler, *retval, *context, *t, *obj, *dict = NULL;
 	static char handler_name[256];
 	struct format_field *field;
-	unsigned long long val;
 	unsigned long s, ns;
 	struct event_format *event;
 	unsigned n = 0;
@@ -302,20 +323,7 @@ static void python_process_tracepoint(struct perf_sample *sample,
 				offset = field->offset;
 			obj = PyString_FromString((char *)data + offset);
 		} else { /* FIELD_IS_NUMERIC */
-			val = read_size(event, data + field->offset,
-					field->size);
-			if (field->flags & FIELD_IS_SIGNED) {
-				if ((long long)val >= LONG_MIN &&
-				    (long long)val <= LONG_MAX)
-					obj = PyInt_FromLong(val);
-				else
-					obj = PyLong_FromLongLong(val);
-			} else {
-				if (val <= LONG_MAX)
-					obj = PyInt_FromLong(val);
-				else
-					obj = PyLong_FromUnsignedLongLong(val);
-			}
+			obj = get_field_numeric_entry(event, field, data);
 		}
 		if (handler)
 			PyTuple_SetItem(t, n++, obj);
-- 
2.0.0.rc4

[PATCH 2/2] perf script: handle the num array type in python properly

[Sebastian Andrzej Siewior]

The raw_syscalls:sys_enter tracer for instance passes has one argument
named 'arg' which is an array of 6 integers. Right the python scripts
gets only 0 passed as an argument. The reason is that
pevent_read_number() can not handle data types of 48 and returns always
0.
This patch changes this by passing num array as list of nums which fit
the description. As a result python will now see a list named arg which
contains 6 (integer) items.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 .../util/scripting-engines/trace-event-python.c    | 43 ++++++++++++++++------
 1 file changed, 31 insertions(+), 12 deletions(-)

diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index aa11790..40a815a 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -233,22 +233,41 @@ static inline struct event_format *find_cache_event(struct perf_evsel *evsel)
 static PyObject *get_field_numeric_entry(struct event_format *event,
 		struct format_field *field, void *data)
 {
-	PyObject *obj;
+	bool is_array = field->flags & FIELD_IS_ARRAY;
+	PyObject *obj, *list = NULL;
 	unsigned long long val;
+	unsigned int item_size, n_items, i;
 
-	val = read_size(event, data + field->offset, field->size);
-	if (field->flags & FIELD_IS_SIGNED) {
-		if ((long long)val >= LONG_MIN &&
-				(long long)val <= LONG_MAX)
-			obj = PyInt_FromLong(val);
-		else
-			obj = PyLong_FromLongLong(val);
+	if (is_array) {
+		list = PyList_New(field->arraylen);
+		item_size = field->size / field->arraylen;
+		n_items = field->arraylen;
 	} else {
-		if (val <= LONG_MAX)
-			obj = PyInt_FromLong(val);
-		else
-			obj = PyLong_FromUnsignedLongLong(val);
+		item_size = field->size;
+		n_items = 1;
+	}
+
+	for (i = 0; i < n_items; i++) {
+
+		val = read_size(event, data + field->offset + i * item_size,
+				item_size);
+		if (field->flags & FIELD_IS_SIGNED) {
+			if ((long long)val >= LONG_MIN &&
+					(long long)val <= LONG_MAX)
+				obj = PyInt_FromLong(val);
+			else
+				obj = PyLong_FromLongLong(val);
+		} else {
+			if (val <= LONG_MAX)
+				obj = PyInt_FromLong(val);
+			else
+				obj = PyLong_FromUnsignedLongLong(val);
+		}
+		if (is_array)
+			PyList_SET_ITEM(list, i, obj);
 	}
+	if (is_array)
+		obj = list;
 	return obj;
 }
 
-- 
2.0.0.rc4

Re: [PATCH 1/2] perf script: move the number processing into its own function

[Namhyung Kim]

Hi Sebastian,

On Tue, 27 May 2014 18:14:33 +0200, Sebastian Andrzej Siewior wrote:
> I was going to change something here and the result was so much on the
> right side of the screen that I decided to move that piece into its own
> function.
> This patch should make no function change except the moving the code
> into its own function.

Acked-by: Namhyung Kim <namhyung@kernel.org>

Thanks,
Namhyung

Re: [PATCH 2/2] perf script: handle the num array type in python properly

[Namhyung Kim]

On Tue, 27 May 2014 18:14:34 +0200, Sebastian Andrzej Siewior wrote:
> The raw_syscalls:sys_enter tracer for instance passes has one argument
> named 'arg' which is an array of 6 integers. Right the python scripts
> gets only 0 passed as an argument. The reason is that
> pevent_read_number() can not handle data types of 48 and returns always
> 0.
> This patch changes this by passing num array as list of nums which fit
> the description. As a result python will now see a list named arg which
> contains 6 (integer) items.

Acked-by: Namhyung Kim <namhyung@kernel.org>

But this requires a change in generation time as I faced a type error
when try to print an array argument as number.  I'll post a separate
patch for this.

Thanks,
Namhyung

[PATCH] perf script/python: Print array argument as string

[Namhyung Kim]

With the Sebastian's change of handling num array argument (of raw
syscall enter), the script still failed to work like this:

  $ perf record -e raw_syscalls:* sleep 1
  $ perf script -g python
  $ perf script -s perf-script.py
  ...
  Traceback (most recent call last):
    File "perf-script.py", line 42, in raw_syscalls__sys_enter
      (id, args),
  TypeError: %u format: a number is required, not list
  Fatal Python error: problem in Python trace event handler
  Aborted (core dumped)

This is because the generated script tries to print the array arg as
unsigned integer (%u).  Since the python seems to convert arguments to
strings by default, just using %s solved the problem for me.

Cc: Tom Zanussi <tzanussi@gmail.com>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/util/scripting-engines/trace-event-python.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index 40a815a5360b..a22ab53788bf 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -649,6 +649,7 @@ static int python_generate_script(struct pevent *pevent, const char *outfile)
 			fprintf(ofp, "%s=", f->name);
 			if (f->flags & FIELD_IS_STRING ||
 			    f->flags & FIELD_IS_FLAG ||
+			    f->flags & FIELD_IS_ARRAY ||
 			    f->flags & FIELD_IS_SYMBOLIC)
 				fprintf(ofp, "%%s");
 			else if (f->flags & FIELD_IS_SIGNED)
-- 
1.9.2

Re: [PATCH 2/2] perf script: handle the num array type in python properly

[Sebastian Andrzej Siewior]

On 05/29/2014 06:43 AM, Namhyung Kim wrote:
> But this requires a change in generation time as I faced a type error
> when try to print an array argument as number.  I'll post a separate
> patch for this.

Thank you.

> 
> Thanks,
> Namhyung
> 
Sebastian

Re: [PATCH] perf script/python: Print array argument as string

[Jiri Olsa]

On Thu, May 29, 2014 at 01:44:55PM +0900, Namhyung Kim wrote:
> With the Sebastian's change of handling num array argument (of raw
> syscall enter), the script still failed to work like this:
> 
>   $ perf record -e raw_syscalls:* sleep 1
>   $ perf script -g python
>   $ perf script -s perf-script.py
>   ...
>   Traceback (most recent call last):
>     File "perf-script.py", line 42, in raw_syscalls__sys_enter
>       (id, args),
>   TypeError: %u format: a number is required, not list
>   Fatal Python error: problem in Python trace event handler
>   Aborted (core dumped)
> 
> This is because the generated script tries to print the array arg as
> unsigned integer (%u).  Since the python seems to convert arguments to
> strings by default, just using %s solved the problem for me.
> 
> Cc: Tom Zanussi <tzanussi@gmail.com>
> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> Signed-off-by: Namhyung Kim <namhyung@kernel.org>

ack/test anyone? ;-)

thanks,
jirka

> ---
>  tools/perf/util/scripting-engines/trace-event-python.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
> index 40a815a5360b..a22ab53788bf 100644
> --- a/tools/perf/util/scripting-engines/trace-event-python.c
> +++ b/tools/perf/util/scripting-engines/trace-event-python.c
> @@ -649,6 +649,7 @@ static int python_generate_script(struct pevent *pevent, const char *outfile)
>  			fprintf(ofp, "%s=", f->name);
>  			if (f->flags & FIELD_IS_STRING ||
>  			    f->flags & FIELD_IS_FLAG ||
> +			    f->flags & FIELD_IS_ARRAY ||
>  			    f->flags & FIELD_IS_SYMBOLIC)
>  				fprintf(ofp, "%%s");
>  			else if (f->flags & FIELD_IS_SIGNED)
> -- 
> 1.9.2
> 

[tip:perf/core] perf script/python: Print array argument as string

[tip-bot for Namhyung Kim]

Commit-ID:  e646fe730a324098a718f1c9b2f349efb99d5457
Gitweb:     http://git.kernel.org/tip/e646fe730a324098a718f1c9b2f349efb99d5457
Author:     Namhyung Kim <namhyung@kernel.org>
AuthorDate: Thu, 29 May 2014 13:44:55 +0900
Committer:  Jiri Olsa <jolsa@kernel.org>
CommitDate: Mon, 9 Jun 2014 12:21:03 +0200

perf script/python: Print array argument as string

With the Sebastian's change of handling num array argument (of raw
syscall enter), the script still failed to work like this:

  $ perf record -e raw_syscalls:* sleep 1
  $ perf script -g python
  $ perf script -s perf-script.py
  ...
  Traceback (most recent call last):
    File "perf-script.py", line 42, in raw_syscalls__sys_enter
      (id, args),
  TypeError: %u format: a number is required, not list
  Fatal Python error: problem in Python trace event handler
  Aborted (core dumped)

This is because the generated script tries to print the array arg as
unsigned integer (%u).  Since the python seems to convert arguments to
strings by default, just using %s solved the problem for me.

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Tom Zanussi <tzanussi@gmail.com>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: http://lkml.kernel.org/r/1401338695-18837-1-git-send-email-namhyung@kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/scripting-engines/trace-event-python.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
index c3de097..1c41932 100644
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ b/tools/perf/util/scripting-engines/trace-event-python.c
@@ -623,6 +623,7 @@ static int python_generate_script(struct pevent *pevent, const char *outfile)
 			fprintf(ofp, "%s=", f->name);
 			if (f->flags & FIELD_IS_STRING ||
 			    f->flags & FIELD_IS_FLAG ||
+			    f->flags & FIELD_IS_ARRAY ||
 			    f->flags & FIELD_IS_SYMBOLIC)
 				fprintf(ofp, "%%s");
 			else if (f->flags & FIELD_IS_SIGNED)

Re: [tip:perf/core] perf script/python: Print array argument as string

[Sebastian Andrzej Siewior]

On 06/12/2014 02:01 PM, tip-bot for Namhyung Kim wrote:

> Commit-ID:  e646fe730a324098a718f1c9b2f349efb99d5457
> Gitweb:     http://git.kernel.org/tip/e646fe730a324098a718f1c9b2f349efb99d5457
> Author:     Namhyung Kim <namhyung@kernel.org>
> AuthorDate: Thu, 29 May 2014 13:44:55 +0900
> Committer:  Jiri Olsa <jolsa@kernel.org>
> CommitDate: Mon, 9 Jun 2014 12:21:03 +0200
> 
> perf script/python: Print array argument as string
> 
> With the Sebastian's change of handling num array argument (of raw
> syscall enter), the script still failed to work like this:

This patch got merged. It fixes something that was introduced by
  "perf script: move the number processing into its own function" [0]
  "perf script: handle the num array type in python properly" [1]

which was not yet merged.

[0] lkml.kernel.org/r/1401207274-8170-1-git-send-email-bigeasy@linutronix.de
[1] lkml.kernel.org/r/1401207274-8170-2-git-send-email-bigeasy@linutronix.de

>   $ perf record -e raw_syscalls:* sleep 1
>   $ perf script -g python
>   $ perf script -s perf-script.py
>   ...
>   Traceback (most recent call last):
>     File "perf-script.py", line 42, in raw_syscalls__sys_enter
>       (id, args),
>   TypeError: %u format: a number is required, not list
>   Fatal Python error: problem in Python trace event handler
>   Aborted (core dumped)
> 
> This is because the generated script tries to print the array arg as
> unsigned integer (%u).  Since the python seems to convert arguments to
> strings by default, just using %s solved the problem for me.

Sebastian

Re: [tip:perf/core] perf script/python: Print array argument as string

[Jiri Olsa]

On Thu, Jun 26, 2014 at 05:37:27PM +0200, Sebastian Andrzej Siewior wrote:
> On 06/12/2014 02:01 PM, tip-bot for Namhyung Kim wrote:
> 
> > Commit-ID:  e646fe730a324098a718f1c9b2f349efb99d5457
> > Gitweb:     http://git.kernel.org/tip/e646fe730a324098a718f1c9b2f349efb99d5457
> > Author:     Namhyung Kim <namhyung@kernel.org>
> > AuthorDate: Thu, 29 May 2014 13:44:55 +0900
> > Committer:  Jiri Olsa <jolsa@kernel.org>
> > CommitDate: Mon, 9 Jun 2014 12:21:03 +0200
> > 
> > perf script/python: Print array argument as string
> > 
> > With the Sebastian's change of handling num array argument (of raw
> > syscall enter), the script still failed to work like this:
> 
> This patch got merged. It fixes something that was introduced by
>   "perf script: move the number processing into its own function" [0]
>   "perf script: handle the num array type in python properly" [1]
> 
> which was not yet merged.

yea, we are fast.. fixing bugs even before making them ;-)

I'll check/queue those 2

thanks,
jirka