[Qemu-devel] [PATCH v4 00/11] tracetool: Improvements for future expansion

[Qemu-devel] [PATCH v4 00/11] tracetool: Improvements for future expansion

[Lluís Vilanova]

NOTE: Depend's on Harsh's port of tractool to python.

A general overhaul of the pythonic tracetool script to allow simpler future
extensions.

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---

Changes in v4:

* Remove debugging message.


Changes in v3:

* Fix arg counting in intermediate patch (so that git bisect will not blame me).


Changes in v2:

* Minor changes.


Lluís Vilanova (11):
      [trivial] Fix a compiler warning
      trace: [tracetool] Do not rebuild event list in backend code
      trace: [tracetool] Simplify event line parsing
      trace: [ŧracetool] Do not precompute the event number
      trace: [tracetool] Add support for event properties
      trace: [tracetool] Process the "disable" event property
      trace: [tracetool] Rewrite event argument parsing
      trace: [tracetool] Make format-specific code optional and with access to event information
      trace: [tracetool] Automatically establish available backends and formats
      trace: Provide a per-event status define for conditional compilation
      trace: [tracetool] Add error-reporting functions


 Makefile.objs        |    6 
 Makefile.target      |    3 
 docs/tracing.txt     |   46 +++-
 scripts/tracetool.py |  627 ++++++++++++++++++++++++++++----------------------
 4 files changed, 397 insertions(+), 285 deletions(-)


To: qemu-devel@nongnu.org
Cc: Stefan Hajnoczi <stefanha@gmail.com>
Cc: Harsh Prateek Bora <harsh@linux.vnet.ibm.com>
Cc: Aneesh Kumar K. V <aneesh.kumar@linux.vnet.ibm.com>

[Qemu-devel] [PATCH v4 01/11] [trivial] Fix a compiler warning

[Lluís Vilanova]

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
 scripts/tracetool.py |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/scripts/tracetool.py b/scripts/tracetool.py
index 6874f66..f0d7e1e 100755
--- a/scripts/tracetool.py
+++ b/scripts/tracetool.py
@@ -183,7 +183,7 @@ def simple_c(events):
         argc = event.argc
         print '''void trace_%(name)s(%(args)s)
 {
-    unsigned int tbuf_idx, rec_off;
+    unsigned int tbuf_idx, rec_off __attribute__((unused));
     uint64_t var64 __attribute__ ((unused));
     uint64_t pvar64 __attribute__ ((unused));
     uint32_t slen __attribute__ ((unused));

[Qemu-devel] [PATCH v4 02/11] trace: [tracetool] Do not rebuild event list in backend code

[Lluís Vilanova]

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
 scripts/tracetool.py |   14 +++++++-------
 1 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/scripts/tracetool.py b/scripts/tracetool.py
index f0d7e1e..7042728 100755
--- a/scripts/tracetool.py
+++ b/scripts/tracetool.py
@@ -171,15 +171,14 @@ def simple_c(events):
     print
     print 'TraceEvent trace_list[] = {'
     print
-    eventlist = list(events)
-    for event in eventlist:
+    for event in events:
         print '{.tp_name = "%(name)s", .state=0},' % {
     'name': event.name
 }
         print
     print '};'
     print
-    for event in eventlist:
+    for event in events:
         argc = event.argc
         print '''void trace_%(name)s(%(args)s)
 {
@@ -311,8 +310,7 @@ def ust_c(events):
 #undef inline
 #undef wmb
 #include "trace.h"'''
-    eventlist = list(events)
-    for event in eventlist:
+    for event in events:
         argnames = event.argnames
         if event.argc > 0:
             argnames = ', ' + event.argnames
@@ -344,7 +342,7 @@ static void ust_%(name)s_probe(%(args)s)
     print '''
 static void __attribute__((constructor)) trace_init(void)
 {'''
-    for event in eventlist:
+    for event in events:
         print '    register_trace_ust_%(name)s(ust_%(name)s_probe);' % {
     'name': event.name
 }
@@ -510,14 +508,16 @@ class Event(object):
 
 # Generator that yields Event objects given a trace-events file object
 def read_events(fobj):
+    res = []
     event_num = 0
     for line in fobj:
         if not line.strip():
             continue
         if line.lstrip().startswith('#'):
 	    continue
-	yield Event(event_num, line)
+	res.append(Event(event_num, line))
 	event_num += 1
+    return res
 
 backend = ""
 output = ""

[Qemu-devel] [PATCH v4 03/11] trace: [tracetool] Simplify event line parsing

[Lluís Vilanova]

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
 scripts/tracetool.py |   49 +++++++++++++++++--------------------------------
 1 files changed, 17 insertions(+), 32 deletions(-)

diff --git a/scripts/tracetool.py b/scripts/tracetool.py
index 7042728..b0c1904 100755
--- a/scripts/tracetool.py
+++ b/scripts/tracetool.py
@@ -38,19 +38,9 @@ Options:
 '''
     sys.exit(1)
 
-def get_name(line, sep='('):
-    head, sep, tail = line.partition(sep)
-    return head
-
-def get_args(line, sep1='(', sep2=')'):
-    head, sep1, tail = line.partition(sep1)
-    args, sep2, fmt_str = tail.partition(sep2)
-    return args
-
-def get_argnames(line, sep=','):
+def get_argnames(args):
     nfields = 0
     str = []
-    args = get_args(line)
     for field in args.split():
       nfields = nfields + 1
       # Drop pointer star
@@ -71,21 +61,7 @@ def get_argnames(line, sep=','):
     else:
       return ''
 
-def get_argc(line):
-    argc = 0
-    argnames = get_argnames(line)
-    if argnames:
-      for name in argnames.split(','):
-        argc = argc + 1
-    return argc
-
-def get_fmt(line, sep=')'):
-    event, sep, fmt = line.partition(sep)
-    return fmt
-
-def calc_sizeofargs(line):
-    args = get_args(line)
-    argc = get_argc(line)
+def calc_sizeofargs(args, argc):
     strtype = ('const char*', 'char*', 'const char *', 'char *')
     str = []
     newstr = ""
@@ -495,16 +471,25 @@ trace_gen = {
 }
 
 # A trace event
+import re
+cre = re.compile("(?P<name>[^(\s]+)\((?P<args>[^)]*)\)\s*(?P<fmt>\".*)?")
+
 class Event(object):
     def __init__(self, num, line):
         self.num = num
-        self.args = get_args(line)
+        m = cre.match(line)
+        assert m is not None
+        groups = m.groupdict('')
+        self.args = groups["args"]
         self.arglist = self.args.split(',')
-        self.name = get_name(line)
-        self.argc = get_argc(line)
-        self.argnames = get_argnames(line)
-        self.sizestr = calc_sizeofargs(line)
-        self.fmt = get_fmt(line)
+        self.name = groups["name"]
+        if len(self.arglist) == 1 and self.arglist[0] == "void":
+            self.argc = 0
+        else:
+            self.argc = len(self.arglist)
+        self.argnames = get_argnames(self.args)
+        self.sizestr = calc_sizeofargs(self.args, self.argc)
+        self.fmt = groups["fmt"]
 
 # Generator that yields Event objects given a trace-events file object
 def read_events(fobj):

[Qemu-devel] [PATCH v4 04/11] trace: [ŧracetool] Do not precompute the event number

[Lluís Vilanova]

This would otherwise break event numbering when actually using the "disable"
property.

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
 scripts/tracetool.py |   21 +++++++++------------
 1 files changed, 9 insertions(+), 12 deletions(-)

diff --git a/scripts/tracetool.py b/scripts/tracetool.py
index b0c1904..07563f0 100755
--- a/scripts/tracetool.py
+++ b/scripts/tracetool.py
@@ -128,7 +128,7 @@ def simple_h(events):
     'args': event.args
 }
     print
-    print '#define NR_TRACE_EVENTS %d' % (event.num + 1)
+    print '#define NR_TRACE_EVENTS %d' % len(events)
     print 'extern TraceEvent trace_list[NR_TRACE_EVENTS];'
 
     return
@@ -154,7 +154,7 @@ def simple_c(events):
         print
     print '};'
     print
-    for event in events:
+    for num, event in enumerate(events):
         argc = event.argc
         print '''void trace_%(name)s(%(args)s)
 {
@@ -169,12 +169,12 @@ def simple_c(events):
 ''' % {
     'name': event.name,
     'args': event.args,
-    'event_id': event.num,
+    'event_id': num,
 }
         print '''
     tbuf_idx = trace_alloc_record(%(event_id)s, %(sizestr)s);
     rec_off = (tbuf_idx + ST_V2_REC_HDR_LEN) %% TRACE_BUF_LEN; /* seek record header */
-''' % {'event_id': event.num, 'sizestr': event.sizestr,}
+''' % {'event_id': num, 'sizestr': event.sizestr,}
 
         if argc > 0:
             str = event.arglist
@@ -220,7 +220,7 @@ def stderr_h(events):
 #include "trace/stderr.h"
 
 extern TraceEvent trace_list[];'''
-    for event in events:
+    for num, event in enumerate(events):
         argnames = event.argnames
         if event.argc > 0:
             argnames = ', ' + event.argnames
@@ -235,12 +235,12 @@ static inline void trace_%(name)s(%(args)s)
 }''' % {
     'name': event.name,
     'args': event.args,
-    'event_num': event.num,
+    'event_num': num,
     'fmt': event.fmt.rstrip('\n'),
     'argnames': argnames
 }
     print
-    print '#define NR_TRACE_EVENTS %d' % (event.num + 1)
+    print '#define NR_TRACE_EVENTS %d' % len(events)
 
 def stderr_c(events):
     print '''#include "trace.h"
@@ -475,8 +475,7 @@ import re
 cre = re.compile("(?P<name>[^(\s]+)\((?P<args>[^)]*)\)\s*(?P<fmt>\".*)?")
 
 class Event(object):
-    def __init__(self, num, line):
-        self.num = num
+    def __init__(self, line):
         m = cre.match(line)
         assert m is not None
         groups = m.groupdict('')
@@ -494,14 +493,12 @@ class Event(object):
 # Generator that yields Event objects given a trace-events file object
 def read_events(fobj):
     res = []
-    event_num = 0
     for line in fobj:
         if not line.strip():
             continue
         if line.lstrip().startswith('#'):
 	    continue
-	res.append(Event(event_num, line))
-	event_num += 1
+	res.append(Event(line))
     return res
 
 backend = ""

[Qemu-devel] [PATCH v4 05/11] trace: [tracetool] Add support for event properties

[Lluís Vilanova]

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
 scripts/tracetool.py |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)

diff --git a/scripts/tracetool.py b/scripts/tracetool.py
index 07563f0..f369579 100755
--- a/scripts/tracetool.py
+++ b/scripts/tracetool.py
@@ -472,7 +472,9 @@ trace_gen = {
 
 # A trace event
 import re
-cre = re.compile("(?P<name>[^(\s]+)\((?P<args>[^)]*)\)\s*(?P<fmt>\".*)?")
+cre = re.compile("((?P<props>.*)\s+)?(?P<name>[^(\s]+)\((?P<args>[^)]*)\)\s*(?P<fmt>\".*)?")
+
+VALID_PROPS = set(["disable"])
 
 class Event(object):
     def __init__(self, line):
@@ -489,6 +491,10 @@ class Event(object):
         self.argnames = get_argnames(self.args)
         self.sizestr = calc_sizeofargs(self.args, self.argc)
         self.fmt = groups["fmt"]
+        self.properties = groups["props"].split()
+        unknown_props = set(self.properties) - VALID_PROPS
+        if len(unknown_props) > 0:
+            raise ValueError("Unknown properties: %s" % ", ".join(unknown_props))
 
 # Generator that yields Event objects given a trace-events file object
 def read_events(fobj):

[Qemu-devel] [PATCH v4 06/11] trace: [tracetool] Process the "disable" event property

[Lluís Vilanova]

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
 scripts/tracetool.py |   13 +++++++++++--
 1 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/scripts/tracetool.py b/scripts/tracetool.py
index f369579..f2bcb65 100755
--- a/scripts/tracetool.py
+++ b/scripts/tracetool.py
@@ -365,6 +365,9 @@ def dtrace_d(events):
     print '};'
     return
 
+def dtrace_nop_d(events):
+    pass
+
 def dtrace_stp(events):
     for event in events:
         # Define prototype for probe arguments
@@ -387,6 +390,9 @@ probe %(probeprefix)s.%(name)s = process("%(binary)s").mark("%(name)s")
     print
     return
 
+def dtrace_nop_stp(events):
+    pass
+
 def trace_stap_begin():
     global probeprefix
     if backend != "dtrace":
@@ -429,6 +435,8 @@ converters = {
     'nop': {
         'h': nop_h,
         'c': nop_c,
+        'd': dtrace_nop_d,
+        'stap': dtrace_nop_stp,
     },
 
     'stderr': {
@@ -563,10 +571,11 @@ def main():
         sys.exit(0)
 
     events = read_events(sys.stdin)
+
     trace_gen[output]['begin']()
-    converters[backend][output](events)
+    converters[backend][output]([ e for e in events if 'disable' not in e.properties ])
+    converters['nop'][output]([ e for e in events if 'disable' in e.properties ])
     trace_gen[output]['end']()
-    return
 
 if __name__ == "__main__":
     main()

[Qemu-devel] [PATCH v4 07/11] trace: [tracetool] Rewrite event argument parsing

[Lluís Vilanova]

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
 scripts/tracetool.py |  190 ++++++++++++++++++++++++--------------------------
 1 files changed, 91 insertions(+), 99 deletions(-)

diff --git a/scripts/tracetool.py b/scripts/tracetool.py
index f2bcb65..cd1c29d 100755
--- a/scripts/tracetool.py
+++ b/scripts/tracetool.py
@@ -38,49 +38,6 @@ Options:
 '''
     sys.exit(1)
 
-def get_argnames(args):
-    nfields = 0
-    str = []
-    for field in args.split():
-      nfields = nfields + 1
-      # Drop pointer star
-      type, ptr, tail = field.partition('*')
-      if type != field:
-        field = tail
-
-      name, sep, tail = field.partition(',')
-
-      if name == field:
-        continue
-      str.append(name)
-      str.append(", ")
-
-    if nfields > 1:
-      str.append(name)
-      return ''.join(str)
-    else:
-      return ''
-
-def calc_sizeofargs(args, argc):
-    strtype = ('const char*', 'char*', 'const char *', 'char *')
-    str = []
-    newstr = ""
-    if argc > 0:
-      str = args.split(',')
-      for elem in str:
-        if elem.lstrip().startswith(strtype): #strings
-          type, sep, var = elem.rpartition('*')
-          newstr = newstr+"4 + strlen("+var.lstrip()+") + "
-        #elif '*' in elem:
-        #  newstr = newstr + "4 + " # pointer vars
-        else:
-          #type, sep, var = elem.rpartition(' ')
-          #newstr = newstr+"sizeof("+type.lstrip()+") + "
-          newstr = newstr + '8 + '
-    newstr = newstr + '0' # for last +
-    return newstr
-
-
 def trace_h_begin():
     print '''#ifndef TRACE_H
 #define TRACE_H
@@ -133,13 +90,6 @@ def simple_h(events):
 
     return
 
-def is_string(arg):
-    strtype = ('const char*', 'char*', 'const char *', 'char *')
-    if arg.lstrip().startswith(strtype):
-        return True
-    else:
-        return False
-
 def simple_c(events):
     rec_off = 0
     print '#include "trace.h"'
@@ -154,8 +104,16 @@ def simple_c(events):
         print
     print '};'
     print
+
     for num, event in enumerate(events):
-        argc = event.argc
+        sizes = []
+        for type_, name in event.args:
+            if type_is_string(type_):
+                sizes.append("4 + strlen(%s)" % name)
+            else:
+                sizes.append("8 + sizeof(%s)" % type_)
+        sizestr = " + ".join(sizes)
+
         print '''void trace_%(name)s(%(args)s)
 {
     unsigned int tbuf_idx, rec_off __attribute__((unused));
@@ -166,52 +124,52 @@ def simple_c(events):
     if (!trace_list[%(event_id)s].state) {
         return;
     }
+
+    tbuf_idx = trace_alloc_record(%(event_id)s, %(sizestr)s);
+    rec_off = (tbuf_idx + ST_V2_REC_HDR_LEN) %% TRACE_BUF_LEN; /* seek record header */
 ''' % {
     'name': event.name,
     'args': event.args,
     'event_id': num,
+    'sizestr' : sizestr,
 }
-        print '''
-    tbuf_idx = trace_alloc_record(%(event_id)s, %(sizestr)s);
-    rec_off = (tbuf_idx + ST_V2_REC_HDR_LEN) %% TRACE_BUF_LEN; /* seek record header */
-''' % {'event_id': num, 'sizestr': event.sizestr,}
 
-        if argc > 0:
-            str = event.arglist
-            for elem in str:
-                if is_string(elem): # if string
-                    type, sep, var = elem.rpartition('*')
+        if len(event.args) > 0:
+            for type_, name in event.args:
+                # string
+                if type_is_string(type_):
                     print '''
-    slen = strlen(%(var)s);
+    slen = strlen(%(name)s);
     write_to_buffer(rec_off, (uint8_t*)&slen, sizeof(slen));
     rec_off += sizeof(slen);''' % {
-    'var': var.lstrip()
+    'name': name
 }
                     print '''
-    write_to_buffer(rec_off, (uint8_t*)%(var)s, slen);
+    write_to_buffer(rec_off, (uint8_t*)%(name)s, slen);
     rec_off += slen;''' % {
-    'var': var.lstrip()
+    'name': name
 }
-                elif '*' in elem: # pointer var (not string)
-                    type, sep, var = elem.rpartition('*')
+                # pointer var (not string)
+                elif type_.endswith('*'):
                     print '''
-    pvar64 = (uint64_t)(uint64_t*)%(var)s;
+    pvar64 = (uint64_t)(uint64_t*)%(name)s;
     write_to_buffer(rec_off, (uint8_t*)&pvar64, sizeof(uint64_t));
     rec_off += sizeof(uint64_t);''' % {
-    'var': var.lstrip()
+    'name': name
 }
-                else: # primitive data type
-                    type, sep, var = elem.rpartition(' ')
+                # primitive data type
+                else:
                     print '''
-    var64 = (uint64_t)%(var)s;
+    var64 = (uint64_t)%(name)s;
     write_to_buffer(rec_off, (uint8_t*)&var64, sizeof(uint64_t));
     rec_off += sizeof(uint64_t);''' % {
-    'var': var.lstrip()
+    'name': name
 }
         print '''
-    trace_mark_record_complete(tbuf_idx);'''
-        print '}'
-        print
+    trace_mark_record_complete(tbuf_idx);
+}
+
+'''
 
     return
 
@@ -220,12 +178,11 @@ def stderr_h(events):
 #include "trace/stderr.h"
 
 extern TraceEvent trace_list[];'''
+
     for num, event in enumerate(events):
-        argnames = event.argnames
-        if event.argc > 0:
-            argnames = ', ' + event.argnames
-        else:
-            argnames = ''
+        argnames = ", ".join(event.args.names())
+        if len(event.args) > 0:
+            argnames = ", "+argnames
         print '''
 static inline void trace_%(name)s(%(args)s)
 {
@@ -262,13 +219,13 @@ def ust_h(events):
 #undef wmb'''
 
     for event in events:
-        if event.argc > 0:
+        if len(event.args) > 0:
             print '''
 DECLARE_TRACE(ust_%(name)s, TP_PROTO(%(args)s), TP_ARGS(%(argnames)s));
 #define trace_%(name)s trace_ust_%(name)s''' % {
     'name': event.name,
     'args': event.args,
-    'argnames': event.argnames
+    'argnames': ", ".join(event.args.names())
 }
         else:
             print '''
@@ -287,9 +244,9 @@ def ust_c(events):
 #undef wmb
 #include "trace.h"'''
     for event in events:
-        argnames = event.argnames
-        if event.argc > 0:
-            argnames = ', ' + event.argnames
+        argnames = ", ".join(event.args.names())
+        if len(event.args) > 0:
+            argnames = ', ' + argnames
             print '''
 DEFINE_TRACE(ust_%(name)s);
 
@@ -339,7 +296,7 @@ def dtrace_h(events):
     'name': event.name,
     'args': event.args,
     'uppername': event.name.upper(),
-    'argnames': event.argnames,
+    'argnames': ", ".join(event.args.names()),
 }
 
 def dtrace_c(events):
@@ -379,12 +336,12 @@ probe %(probeprefix)s.%(name)s = process("%(binary)s").mark("%(name)s")
     'binary': binary
 }
         i = 1
-        if event.argc > 0:
-            for arg in event.argnames.split(','):
+        if len(event.args) > 0:
+            for name in event.args.names():
                 # 'limit' is a reserved keyword
-                if arg == 'limit':
-                    arg = '_limit'
-                print '  %s = $arg%d;' % (arg.lstrip(), i)
+                if name == 'limit':
+                    name = '_limit'
+                print '  %s = $arg%d;' % (name.lstrip(), i)
                 i += 1
         print '}'
     print
@@ -478,6 +435,47 @@ trace_gen = {
     },
 }
 
+# Event arguments
+def type_is_string(type_):
+    strtype = ('const char*', 'char*', 'const char *', 'char *')
+    return type_.startswith(strtype)
+
+class Arguments:
+    def __init__ (self, arg_str):
+        self._args = []
+        for arg in arg_str.split(","):
+            arg = arg.strip()
+            parts = arg.split()
+            head, sep, tail = parts[-1].rpartition("*")
+            parts = parts[:-1]
+            if tail == "void":
+                assert len(parts) == 0 and sep == ""
+                continue
+            arg_type = " ".join(parts + [ " ".join([head, sep]).strip() ]).strip()
+            self._args.append((arg_type, tail))
+
+    def __iter__(self):
+        return iter(self._args)
+
+    def __len__(self):
+        return len(self._args)
+
+    def __str__(self):
+        if len(self._args) == 0:
+            return "void"
+        else:
+            return ", ".join([ " ".join([t, n]) for t,n in self._args ])
+
+    def names(self):
+        return [ name for _, name in self._args ]
+
+    def types(self):
+        return [ type_ for type_, _ in self._args ]
+
+    def size_str(self):
+        res = ""
+        return res
+
 # A trace event
 import re
 cre = re.compile("((?P<props>.*)\s+)?(?P<name>[^(\s]+)\((?P<args>[^)]*)\)\s*(?P<fmt>\".*)?")
@@ -489,17 +487,11 @@ class Event(object):
         m = cre.match(line)
         assert m is not None
         groups = m.groupdict('')
-        self.args = groups["args"]
-        self.arglist = self.args.split(',')
         self.name = groups["name"]
-        if len(self.arglist) == 1 and self.arglist[0] == "void":
-            self.argc = 0
-        else:
-            self.argc = len(self.arglist)
-        self.argnames = get_argnames(self.args)
-        self.sizestr = calc_sizeofargs(self.args, self.argc)
         self.fmt = groups["fmt"]
         self.properties = groups["props"].split()
+        self.args = Arguments(groups["args"])
+
         unknown_props = set(self.properties) - VALID_PROPS
         if len(unknown_props) > 0:
             raise ValueError("Unknown properties: %s" % ", ".join(unknown_props))

[Qemu-devel] [PATCH v4 08/11] trace: [tracetool] Make format-specific code optional and with access to event information

[Lluís Vilanova]

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
 scripts/tracetool.py |   35 ++++++++++-------------------------
 1 files changed, 10 insertions(+), 25 deletions(-)

diff --git a/scripts/tracetool.py b/scripts/tracetool.py
index cd1c29d..91e7620 100755
--- a/scripts/tracetool.py
+++ b/scripts/tracetool.py
@@ -38,26 +38,19 @@ Options:
 '''
     sys.exit(1)
 
-def trace_h_begin():
+def trace_h_begin(events):
     print '''#ifndef TRACE_H
 #define TRACE_H
 
 /* This file is autogenerated by tracetool, do not edit. */
 
 #include "qemu-common.h"'''
-    return
 
-def trace_h_end():
+def trace_h_end(events):
     print '#endif /* TRACE_H */'
-    return
 
-def trace_c_begin():
+def trace_c_begin(events):
     print '/* This file is autogenerated by tracetool, do not edit. */'
-    return
-
-def trace_c_end():
-    # nop, required for trace_gen
-    return
 
 def nop_h(events):
     print
@@ -350,7 +343,7 @@ probe %(probeprefix)s.%(name)s = process("%(binary)s").mark("%(name)s")
 def dtrace_nop_stp(events):
     pass
 
-def trace_stap_begin():
+def trace_stap_begin(events):
     global probeprefix
     if backend != "dtrace":
         print 'SystemTAP tapset generator not applicable to %s backend' % backend
@@ -367,20 +360,13 @@ def trace_stap_begin():
     if probeprefix == "":
         probeprefix = 'qemu.' + targettype + '.' + targetarch
     print '/* This file is autogenerated by tracetool, do not edit. */'
-    return
 
-def trace_stap_end():
-    return #nop, reqd for trace_gen
-
-def trace_d_begin():
+def trace_d_begin(events):
     if backend != 'dtrace':
         print 'DTrace probe generator not applicable to %s backend' % backend
         sys.exit(1)
     print '/* This file is autogenerated by tracetool, do not edit. */'
 
-def trace_d_end():
-    return #nop, reqd for trace_gen
-
 
 # Registry of backends and their converter functions
 converters = {
@@ -416,22 +402,19 @@ converters = {
 }
 
 # Trace file header and footer code generators
-trace_gen = {
+formats = {
     'h': {
         'begin': trace_h_begin,
         'end': trace_h_end,
     },
     'c': {
         'begin': trace_c_begin,
-        'end': trace_c_end,
     },
     'd': {
         'begin': trace_d_begin,
-        'end': trace_d_end,
     },
     'stap': {
         'begin': trace_stap_begin,
-        'end': trace_stap_end,
     },
 }
 
@@ -564,10 +547,12 @@ def main():
 
     events = read_events(sys.stdin)
 
-    trace_gen[output]['begin']()
+    if 'begin' in formats[output]:
+        formats[output]['begin'](events)
     converters[backend][output]([ e for e in events if 'disable' not in e.properties ])
     converters['nop'][output]([ e for e in events if 'disable' in e.properties ])
-    trace_gen[output]['end']()
+    if 'end' in formats[output]:
+        formats[output]['end'](events)
 
 if __name__ == "__main__":
     main()

[Qemu-devel] [PATCH v4 09/11] trace: [tracetool] Automatically establish available backends and formats

[Lluís Vilanova]

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
 Makefile.objs        |    6 -
 Makefile.target      |    3 
 scripts/tracetool.py |  357 ++++++++++++++++++++++++++++++++------------------
 3 files changed, 230 insertions(+), 136 deletions(-)

diff --git a/Makefile.objs b/Makefile.objs
index 2b68739..941386b 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -354,12 +354,12 @@ else
 trace.h: trace.h-timestamp
 endif
 trace.h-timestamp: $(SRC_PATH)/trace-events $(BUILD_DIR)/config-host.mak
-	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py --backend=$(TRACE_BACKEND) -h < $< > $@,"  GEN   trace.h")
+	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py --format=h --backend=$(TRACE_BACKEND) < $< > $@,"  GEN   trace.h")
 	@cmp -s $@ trace.h || cp $@ trace.h
 
 trace.c: trace.c-timestamp
 trace.c-timestamp: $(SRC_PATH)/trace-events $(BUILD_DIR)/config-host.mak
-	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py --backend=$(TRACE_BACKEND) -c < $< > $@,"  GEN   trace.c")
+	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py --format=c --backend=$(TRACE_BACKEND) < $< > $@,"  GEN   trace.c")
 	@cmp -s $@ trace.c || cp $@ trace.c
 
 trace.o: trace.c $(GENERATED_HEADERS)
@@ -372,7 +372,7 @@ trace-dtrace.h: trace-dtrace.dtrace
 # rule file. So we use '.dtrace' instead
 trace-dtrace.dtrace: trace-dtrace.dtrace-timestamp
 trace-dtrace.dtrace-timestamp: $(SRC_PATH)/trace-events $(BUILD_DIR)/config-host.mak
-	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py --backend=$(TRACE_BACKEND) -d < $< > $@,"  GEN   trace-dtrace.dtrace")
+	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py --format=d --backend=$(TRACE_BACKEND) < $< > $@,"  GEN   trace-dtrace.dtrace")
 	@cmp -s $@ trace-dtrace.dtrace || cp $@ trace-dtrace.dtrace
 
 trace-dtrace.o: trace-dtrace.dtrace $(GENERATED_HEADERS)
diff --git a/Makefile.target b/Makefile.target
index 2b24ea1..2bdf955 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -53,11 +53,12 @@ endif
 
 $(QEMU_PROG).stp:
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/tracetool.py \
+		--format=stap \
 		--backend=$(TRACE_BACKEND) \
 		--binary=$(bindir)/$(QEMU_PROG) \
 		--target-arch=$(TARGET_ARCH) \
 		--target-type=$(TARGET_TYPE) \
-		--stap < $(SRC_PATH)/trace-events > $(QEMU_PROG).stp,"  GEN   $(QEMU_PROG).stp")
+		< $(SRC_PATH)/trace-events > $(QEMU_PROG).stp,"  GEN   $(QEMU_PROG).stp")
 else
 stap:
 endif
diff --git a/scripts/tracetool.py b/scripts/tracetool.py
index 91e7620..8ce39df 100755
--- a/scripts/tracetool.py
+++ b/scripts/tracetool.py
@@ -11,33 +11,109 @@
 import sys
 import getopt
 
-def usage():
-    print "Tracetool: Generate tracing code for trace events file on stdin"
-    print "Usage:"
-    print sys.argv[0], " --backend=[nop|simple|stderr|dtrace|ust] [-h|-c|-d|--stap]"
-    print '''
-Backends:
-  --nop     Tracing disabled
-  --simple  Simple built-in backend
-  --stderr  Stderr built-in backend
-  --dtrace  DTrace/SystemTAP backend
-  --ust     LTTng User Space Tracing backend
-
-Output formats:
-  -h     Generate .h file
-  -c     Generate .c file
-  -d     Generate .d file (DTrace only)
-  --stap Generate .stp file (DTrace with SystemTAP only)
 
-Options:
-  --binary       [path]    Full path to QEMU binary
-  --target-arch  [arch]    QEMU emulator target arch
-  --target-type  [type]    QEMU emulator target type ('system' or 'user')
-  --probe-prefix [prefix]  Prefix for dtrace probe names
-                           (default: qemu-targettype-targetarch)
-'''
-    sys.exit(1)
 
+######################################################################
+# format auto-registration
+
+class _Tag:
+    pass
+
+_formats = {}
+
+BEGIN = _Tag()
+END = _Tag()
+_DESCR = _Tag()
+
+def for_format(format_, when, descr = None):
+    """Decorator for format generator functions."""
+
+    if when is not BEGIN and when is not END:
+        raise ValueError("Invalid 'when' tag")
+    if format_ in _formats and when in _formats[format_]:
+        raise ValueError("Format '%s' already set for given 'when' tag" % format_)
+
+    if format_ not in _formats:
+        _formats[format_] = {}
+    if descr is not None:
+        if _DESCR in _formats[format_]:
+            raise ValueError("Description already set")
+        _formats[format_][_DESCR] = descr
+
+    def func(f):
+        _formats[format_][when] = f
+        return f
+    return func
+
+def get_format(format_, when):
+    """Get a format generator function."""
+
+    def nop(*args, **kwargs):
+        pass
+    if format_ in _formats and when in _formats[format_]:
+        return _formats[format_][when]
+    else:
+        return nop
+
+def get_format_descr(format_):
+    """Get the description of a format generator."""
+
+    if format_ in _formats and _DESCR in _formats[format_]:
+        return _formats[format_][_DESCR]
+    else:
+        return ""
+
+
+
+######################################################################
+# backend auto-registration and format compatibility
+
+_backends = {}
+
+def for_backend(backend, format_, descr = None):
+    if backend not in _backends:
+        _backends[backend] = {}
+    if format_ in _backends[backend]:
+        raise ValueError("Backend '%s' already set for backend '%s'" % (backend, format_))
+    if format_ not in _formats:
+        raise ValueError("Unknown format '%s'" % format_)
+
+    if descr is not None:
+        if _DESCR in _backends[backend]:
+            raise ValueError("Description already set")
+        _backends[backend][_DESCR] = descr
+
+    def func(f):
+        _backends[backend][format_] = f
+        return f
+    return func
+
+def get_backend(format_, backend):
+    if backend not in _backends:
+        raise ValueError("Unknown backend '%s'" % backend)
+    if format_ not in _formats:
+        raise ValueError("Unknown format '%s'" % format_)
+    if format_ not in _backends[backend]:
+        raise ValueError("Format '%s' not supported with backend '%s'" % (format_, backend))
+    return _backends[backend][format_]
+
+def get_backend_descr(backend):
+    """Get the description of a backend."""
+
+    if backend in _backends and _DESCR in _backends[backend]:
+        return _backends[backend][_DESCR]
+    else:
+        return ""
+
+
+
+######################################################################
+# formats
+
+##################################################
+# format: h
+
+@for_format("h", BEGIN, "Generate .h file")
 def trace_h_begin(events):
     print '''#ifndef TRACE_H
 #define TRACE_H
@@ -46,12 +122,27 @@ def trace_h_begin(events):
 
 #include "qemu-common.h"'''
 
+@for_format("h", END)
 def trace_h_end(events):
     print '#endif /* TRACE_H */'
 
+
+##################################################
+# format: c
+
+@for_format("c", BEGIN, "Generate .c file")
 def trace_c_begin(events):
     print '/* This file is autogenerated by tracetool, do not edit. */'
 
+
+
+######################################################################
+# backends
+
+##################################################
+# backend: nop
+
+@for_backend("nop", "h", "Tracing disabled")
 def nop_h(events):
     print
     for event in events:
@@ -62,13 +153,16 @@ def nop_h(events):
     'name': event.name,
     'args': event.args
 }
-    return
 
+@for_backend("nop", "c")
 def nop_c(events):
-    # nop, reqd for converters
-    return
+    pass
+
 
+##################################################
+# backend: simple
 
+@for_backend("simple", "h", "Simple built-in backend")
 def simple_h(events):
     print '#include "trace/simple.h"'
     print
@@ -81,8 +175,7 @@ def simple_h(events):
     print '#define NR_TRACE_EVENTS %d' % len(events)
     print 'extern TraceEvent trace_list[NR_TRACE_EVENTS];'
 
-    return
-
+@for_backend("simple", "c")
 def simple_c(events):
     rec_off = 0
     print '#include "trace.h"'
@@ -164,8 +257,11 @@ def simple_c(events):
 
 '''
 
-    return
 
+##################################################
+# backend: stderr
+
+@for_backend("stderr", "h", "Stderr built-in backend")
 def stderr_h(events):
     print '''#include <stdio.h>
 #include "trace/stderr.h"
@@ -192,6 +288,7 @@ static inline void trace_%(name)s(%(args)s)
     print
     print '#define NR_TRACE_EVENTS %d' % len(events)
 
+@for_backend("stderr", "c")
 def stderr_c(events):
     print '''#include "trace.h"
 
@@ -204,6 +301,11 @@ TraceEvent trace_list[] = {
         print
     print '};'
 
+
+##################################################
+# backend: ust
+
+@for_backend("ust", "h", "LTTng User Space Tracing backend")
 def ust_h(events):
     print '''#include <ust/tracepoint.h>
 #undef mutex_lock
@@ -227,8 +329,8 @@ _DECLARE_TRACEPOINT_NOARGS(ust_%(name)s);
     'name': event.name,
 }
     print
-    return
 
+@for_backend("ust", "c")
 def ust_c(events):
     print '''#include <ust/marker.h>
 #undef mutex_lock
@@ -274,8 +376,11 @@ static void __attribute__((constructor)) trace_init(void)
 }
     print '}'
 
-    return
 
+##################################################
+# backend: dtrace
+
+@for_backend("dtrace", "h", "DTrace/SystemTAP backend")
 def dtrace_h(events):
     print '#include "trace-dtrace.h"'
     print
@@ -292,9 +397,16 @@ def dtrace_h(events):
     'argnames': ", ".join(event.args.names()),
 }
 
+@for_backend("dtrace", "c")
 def dtrace_c(events):
-    return # No need for function definitions in dtrace backend
+    pass
+
+
+@for_format("d", BEGIN, "Generate .d file (DTrace probes)")
+def trace_d_begin(events):
+    print '/* This file is autogenerated by tracetool, do not edit. */'
 
+@for_backend("dtrace", "d")
 def dtrace_d(events):
     print 'provider qemu {'
     for event in events:
@@ -315,9 +427,27 @@ def dtrace_d(events):
     print '};'
     return
 
+@for_backend("nop", "d")
 def dtrace_nop_d(events):
     pass
 
+
+@for_format("stap", BEGIN, "Generate .stp file (SystemTAP tapsets)")
+def trace_stap_begin(events):
+    global probeprefix
+    if binary == "":
+        print '--binary is required for SystemTAP tapset generator'
+        sys.exit(1)
+    if ((probeprefix == "") and (targettype == "")):
+        print '--target-type is required for SystemTAP tapset generator'
+        sys.exit(1)
+    if ((probeprefix == "") and (targetarch == "")):
+        print '--target-arch is required for SystemTAP tapset generator'
+        sys.exit(1)
+    if probeprefix == "":
+        probeprefix = 'qemu.' + targettype + '.' + targetarch
+    print '/* This file is autogenerated by tracetool, do not edit. */'
+
 def dtrace_stp(events):
     for event in events:
         # Define prototype for probe arguments
@@ -343,82 +473,11 @@ probe %(probeprefix)s.%(name)s = process("%(binary)s").mark("%(name)s")
 def dtrace_nop_stp(events):
     pass
 
-def trace_stap_begin(events):
-    global probeprefix
-    if backend != "dtrace":
-        print 'SystemTAP tapset generator not applicable to %s backend' % backend
-        sys.exit(1)
-    if binary == "":
-        print '--binary is required for SystemTAP tapset generator'
-        sys.exit(1)
-    if ((probeprefix == "") and (targettype == "")):
-        print '--target-type is required for SystemTAP tapset generator'
-        sys.exit(1)
-    if ((probeprefix == "") and (targetarch == "")):
-        print '--target-arch is required for SystemTAP tapset generator'
-        sys.exit(1)
-    if probeprefix == "":
-        probeprefix = 'qemu.' + targettype + '.' + targetarch
-    print '/* This file is autogenerated by tracetool, do not edit. */'
-
-def trace_d_begin(events):
-    if backend != 'dtrace':
-        print 'DTrace probe generator not applicable to %s backend' % backend
-        sys.exit(1)
-    print '/* This file is autogenerated by tracetool, do not edit. */'
 
 
-# Registry of backends and their converter functions
-converters = {
-    'simple': {
-        'h': simple_h,
-	'c': simple_c,
-    },
-
-    'nop': {
-        'h': nop_h,
-        'c': nop_c,
-        'd': dtrace_nop_d,
-        'stap': dtrace_nop_stp,
-    },
-
-    'stderr': {
-        'h': stderr_h,
-        'c': stderr_c,
-    },
-
-    'dtrace': {
-        'h': dtrace_h,
-        'c': dtrace_c,
-        'd': dtrace_d,
-        'stap': dtrace_stp
-    },
-
-    'ust': {
-        'h': ust_h,
-        'c': ust_c,
-    },
-
-}
-
-# Trace file header and footer code generators
-formats = {
-    'h': {
-        'begin': trace_h_begin,
-        'end': trace_h_end,
-    },
-    'c': {
-        'begin': trace_c_begin,
-    },
-    'd': {
-        'begin': trace_d_begin,
-    },
-    'stap': {
-        'begin': trace_stap_begin,
-    },
-}
-
+######################################################################
 # Event arguments
+
 def type_is_string(type_):
     strtype = ('const char*', 'char*', 'const char *', 'char *')
     return type_.startswith(strtype)
@@ -459,7 +518,11 @@ class Arguments:
         res = ""
         return res
 
+
+
+######################################################################
 # A trace event
+
 import re
 cre = re.compile("((?P<props>.*)\s+)?(?P<name>[^(\s]+)\((?P<args>[^)]*)\)\s*(?P<fmt>\".*)?")
 
@@ -490,34 +553,54 @@ def read_events(fobj):
 	res.append(Event(line))
     return res
 
+
+######################################################################
+# Main
+
+format_ = ""
 backend = ""
-output = ""
 binary = ""
 targettype = ""
 targetarch = ""
 probeprefix = ""
 
+def usage():
+    print "Tracetool: Generate tracing code for trace events file on stdin"
+    print "Usage:"
+    print sys.argv[0], " --format=<format> --backend=<backend>"
+    print
+    print "Output formats:"
+    for f in _formats:
+        print "   %-10s %s" % (f, get_format_descr(f))
+    print
+    print "Backends:"
+    for b in _backends:
+        print "   %-10s %s" % (b, get_backend_descr(b))
+    print """
+Options:
+  --binary       [path]    Full path to QEMU binary
+  --target-arch  [arch]    QEMU emulator target arch
+  --target-type  [type]    QEMU emulator target type ('system' or 'user')
+  --probe-prefix [prefix]  Prefix for dtrace probe names
+                           (default: qemu-targettype-targetarch)
+"""
+
+    sys.exit(1)
+
 def main():
-    global backend, output, binary, targettype, targetarch, probeprefix
-    supported_backends = ["simple", "nop", "stderr", "dtrace", "ust"]
-    short_options = "hcd"
-    long_options = ["stap", "backend=", "binary=", "target-arch=", "target-type=", "probe-prefix=", "list-backends", "check-backend"]
+    global format_, backend, binary, targettype, targetarch, probeprefix
+
+    long_options = ["stap", "format=", "backend=", "binary=", "target-arch=", "target-type=", "probe-prefix=", "list-backends", "check-backend"]
     try:
-        opts, args = getopt.getopt(sys.argv[1:], short_options, long_options)
+        opts, args = getopt.getopt(sys.argv[1:], "", long_options)
     except getopt.GetoptError, err:
         # print help information and exit:
         print str(err) # will print something like "option -a not recognized"
         usage()
         sys.exit(2)
     for opt, arg in opts:
-        if opt == '-h':
-            output = 'h'
-        elif opt == '-c':
-            output = 'c'
-        elif opt == '-d':
-            output = 'd'
-        elif opt == '--stap':
-            output = 'stap'
+        if opt == '--format':
+            format_ = arg
         elif opt == '--backend':
             backend = arg
         elif opt == '--binary':
@@ -529,30 +612,40 @@ def main():
         elif opt == '--probe-prefix':
             probeprefix = arg
         elif opt == '--list-backends':
-            print 'simple, nop, stderr, dtrace'
+            print ', '.join(_backends)
             sys.exit(0)
         elif opt == "--check-backend":
-            if any(backend in s for s in supported_backends):
+            if backend in _backends:
                 sys.exit(0)
             else:
                 sys.exit(1)
         else:
-            #assert False, "unhandled option"
             print "unhandled option: ", opt
             usage()
 
-    if backend == "" or output == "":
+    if format_ not in _formats:
+        print "Unknown format: %s" % format_
+        print
+        usage()
+    if backend not in _backends:
+        print "Unknown backend: %s" % backend
+        print
         usage()
-        sys.exit(0)
 
     events = read_events(sys.stdin)
 
-    if 'begin' in formats[output]:
-        formats[output]['begin'](events)
-    converters[backend][output]([ e for e in events if 'disable' not in e.properties ])
-    converters['nop'][output]([ e for e in events if 'disable' in e.properties ])
-    if 'end' in formats[output]:
-        formats[output]['end'](events)
+    try:
+        # just force format/backend compatibility check
+        bfun = get_backend(format_, backend)
+        bnop = get_backend(format_, "nop")
+    except Exception as e:
+        sys.stderr.write(str(e) + "\n\n")
+        usage()
+
+    get_format(format_, BEGIN)(events)
+    bfun([ e for e in events if "disable" not in e.properties ])
+    bnop([ e for e in events if "disable" in e.properties ])
+    get_format(format_, END)(events)
 
 if __name__ == "__main__":
     main()

[Qemu-devel] [PATCH v4 10/11] trace: Provide a per-event status define for conditional compilation

[Lluís Vilanova]

NOTE: This is a port of a patch in Stefanha's tracing tree to the new pythonic
      tracetool version.

Adds a 'TRACE_${NAME}_ENABLED' preprocessor define for each tracing event in
"trace.h".

This lets the user conditionally compile code with a relatively high execution
cost that is only necessary when producing the tracing information for an event
that is enabled.

Note that events using this define will probably have the "disable" property by
default, in order to avoid such costs on regular builds.

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
 docs/tracing.txt     |   46 ++++++++++++++++++++++++++++++++++++++++------
 scripts/tracetool.py |    7 +++++++
 2 files changed, 47 insertions(+), 6 deletions(-)

diff --git a/docs/tracing.txt b/docs/tracing.txt
index ea29f2c..a92716f 100644
--- a/docs/tracing.txt
+++ b/docs/tracing.txt
@@ -98,12 +98,6 @@ respectively.  This ensures portability between 32- and 64-bit platforms.
 4. Name trace events after their function.  If there are multiple trace events
    in one function, append a unique distinguisher at the end of the name.
 
-5. If specific trace events are going to be called a huge number of times, this
-   might have a noticeable performance impact even when the trace events are
-   programmatically disabled. In this case you should declare the trace event
-   with the "disable" property, which will effectively disable it at compile
-   time (using the "nop" backend).
-
 == Generic interface and monitor commands ==
 
 You can programmatically query and control the dynamic state of trace events
@@ -234,3 +228,43 @@ probes:
                       --target-type system \
                       --target-arch x86_64 \
                       <trace-events >qemu.stp
+
+== Trace event properties ==
+
+Each event in the "trace-events" file can be prefixed with a space-separated
+list of zero or more of the following event properties.
+
+=== "disable" ===
+
+If a specific trace event is going to be invoked a huge number of times, this
+might have a noticeable performance impact even when the event is
+programmatically disabled.
+
+In this case you should declare such event with the "disable" property. This
+will effectively disable the event at compile time (by using the "nop" backend),
+thus having no performance impact at all on regular builds (i.e., unless you
+edit the "trace-events" file).
+
+In addition, there might be cases where relatively complex computations must be
+performed to generate values that are only used as arguments for a trace
+function. In these cases you can use the macro 'TRACE_${EVENT_NAME}_ENABLED' to
+guard such computations and avoid its compilation when the event is disabled:
+
+    #include "trace.h"  /* needed for trace event prototype */
+    
+    void *qemu_vmalloc(size_t size)
+    {
+        void *ptr;
+        size_t align = QEMU_VMALLOC_ALIGN;
+    
+        if (size < align) {
+            align = getpagesize();
+        }
+        ptr = qemu_memalign(align, size);
+        if (TRACE_QEMU_VMALLOC_ENABLED) { /* preprocessor macro */
+            void *complex;
+            /* some complex computations to produce the 'complex' value */
+            trace_qemu_vmalloc(size, ptr, complex);
+        }
+        return ptr;
+    }
diff --git a/scripts/tracetool.py b/scripts/tracetool.py
index 8ce39df..1134544 100755
--- a/scripts/tracetool.py
+++ b/scripts/tracetool.py
@@ -124,6 +124,13 @@ def trace_h_begin(events):
 
 @for_format("h", END)
 def trace_h_end(events):
+    for e in events:
+        if 'disable' in e.properties:
+            enabled = 0
+        else:
+            enabled = 1
+        print "#define TRACE_%s_ENABLED %d" % (e.name.upper(), enabled)
+    print
     print '#endif /* TRACE_H */'
 
 

[Qemu-devel] [PATCH v4 11/11] trace: [tracetool] Add error-reporting functions

[Lluís Vilanova]

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
 scripts/tracetool.py |   27 +++++++++++++++------------
 1 files changed, 15 insertions(+), 12 deletions(-)

diff --git a/scripts/tracetool.py b/scripts/tracetool.py
index 1134544..7053a74 100755
--- a/scripts/tracetool.py
+++ b/scripts/tracetool.py
@@ -12,6 +12,13 @@ import sys
 import getopt
 
 
+def error_write(*lines):
+    sys.stderr.writelines(lines)
+
+def error(*lines):
+    error_write(*lines)
+    sys.exit(1)
+
 
 ######################################################################
 # format auto-registration
@@ -443,14 +450,11 @@ def dtrace_nop_d(events):
 def trace_stap_begin(events):
     global probeprefix
     if binary == "":
-        print '--binary is required for SystemTAP tapset generator'
-        sys.exit(1)
+        error("--binary is required for SystemTAP tapset generator\n")
     if ((probeprefix == "") and (targettype == "")):
-        print '--target-type is required for SystemTAP tapset generator'
-        sys.exit(1)
+        error("--target-type is required for SystemTAP tapset generator\n")
     if ((probeprefix == "") and (targetarch == "")):
-        print '--target-arch is required for SystemTAP tapset generator'
-        sys.exit(1)
+        error("--target-arch is required for SystemTAP tapset generator\n")
     if probeprefix == "":
         probeprefix = 'qemu.' + targettype + '.' + targetarch
     print '/* This file is autogenerated by tracetool, do not edit. */'
@@ -601,8 +605,9 @@ def main():
     try:
         opts, args = getopt.getopt(sys.argv[1:], "", long_options)
     except getopt.GetoptError, err:
-        # print help information and exit:
-        print str(err) # will print something like "option -a not recognized"
+        # print help information and exit
+        # will print something like "option -a not recognized"
+        error_write(str(err)+"\n")
         usage()
         sys.exit(2)
     for opt, arg in opts:
@@ -631,12 +636,10 @@ def main():
             usage()
 
     if format_ not in _formats:
-        print "Unknown format: %s" % format_
-        print
+        error_write("Unknown format: %s\n\n" % format_)
         usage()
     if backend not in _backends:
-        print "Unknown backend: %s" % backend
-        print
+        error_write("Unknown backend: %s\n\n" % backend)
         usage()
 
     events = read_events(sys.stdin)

Re: [Qemu-devel] [PATCH v4 07/11] trace: [tracetool] Rewrite event argument parsing

[Harsh Bora]

On 02/10/2012 05:25 PM, Lluís Vilanova wrote:
> Signed-off-by: Lluís Vilanova<vilanova@ac.upc.edu>
> ---
>   scripts/tracetool.py |  190 ++++++++++++++++++++++++--------------------------
>   1 files changed, 91 insertions(+), 99 deletions(-)
>
> diff --git a/scripts/tracetool.py b/scripts/tracetool.py
> index f2bcb65..cd1c29d 100755
> --- a/scripts/tracetool.py
> +++ b/scripts/tracetool.py
> @@ -38,49 +38,6 @@ Options:
>   '''
>       sys.exit(1)
>
> -def get_argnames(args):
> -    nfields = 0
> -    str = []
> -    for field in args.split():
> -      nfields = nfields + 1
> -      # Drop pointer star
> -      type, ptr, tail = field.partition('*')
> -      if type != field:
> -        field = tail
> -
> -      name, sep, tail = field.partition(',')
> -
> -      if name == field:
> -        continue
> -      str.append(name)
> -      str.append(", ")
> -
> -    if nfields>  1:
> -      str.append(name)
> -      return ''.join(str)
> -    else:
> -      return ''
> -
> -def calc_sizeofargs(args, argc):
> -    strtype = ('const char*', 'char*', 'const char *', 'char *')
> -    str = []
> -    newstr = ""
> -    if argc>  0:
> -      str = args.split(',')
> -      for elem in str:
> -        if elem.lstrip().startswith(strtype): #strings
> -          type, sep, var = elem.rpartition('*')
> -          newstr = newstr+"4 + strlen("+var.lstrip()+") + "
> -        #elif '*' in elem:
> -        #  newstr = newstr + "4 + " # pointer vars
> -        else:
> -          #type, sep, var = elem.rpartition(' ')
> -          #newstr = newstr+"sizeof("+type.lstrip()+") + "
> -          newstr = newstr + '8 + '
> -    newstr = newstr + '0' # for last +
> -    return newstr
> -
> -
>   def trace_h_begin():
>       print '''#ifndef TRACE_H
>   #define TRACE_H
> @@ -133,13 +90,6 @@ def simple_h(events):
>
>       return
>
> -def is_string(arg):
> -    strtype = ('const char*', 'char*', 'const char *', 'char *')
> -    if arg.lstrip().startswith(strtype):
> -        return True
> -    else:
> -        return False
> -
>   def simple_c(events):
>       rec_off = 0
>       print '#include "trace.h"'
> @@ -154,8 +104,16 @@ def simple_c(events):
>           print
>       print '};'
>       print
> +
>       for num, event in enumerate(events):
> -        argc = event.argc
> +        sizes = []
> +        for type_, name in event.args:
> +            if type_is_string(type_):
> +                sizes.append("4 + strlen(%s)" % name)



> +            else:
> +                sizes.append("8 + sizeof(%s)" % type_)
> +        sizestr = " + ".join(sizes)
> +

Applied with a small change as reqd:

+            else:
+                sizes.append("8")
+        sizestr = " + ".join(sizes)
+        if len(event.args) == 0:
+            sizestr = '0'
+

BTW, I am manually applying your changes on top of my patches as there 
were significant changes in my patches also. I will include your patches 
in my next series.

- Harsh


>           print '''void trace_%(name)s(%(args)s)
>   {
>       unsigned int tbuf_idx, rec_off __attribute__((unused));
> @@ -166,52 +124,52 @@ def simple_c(events):
>       if (!trace_list[%(event_id)s].state) {
>           return;
>       }
> +
> +    tbuf_idx = trace_alloc_record(%(event_id)s, %(sizestr)s);
> +    rec_off = (tbuf_idx + ST_V2_REC_HDR_LEN) %% TRACE_BUF_LEN; /* seek record header */
>   ''' % {
>       'name': event.name,
>       'args': event.args,
>       'event_id': num,
> +    'sizestr' : sizestr,
>   }
> -        print '''
> -    tbuf_idx = trace_alloc_record(%(event_id)s, %(sizestr)s);
> -    rec_off = (tbuf_idx + ST_V2_REC_HDR_LEN) %% TRACE_BUF_LEN; /* seek record header */
> -''' % {'event_id': num, 'sizestr': event.sizestr,}
>
> -        if argc>  0:
> -            str = event.arglist
> -            for elem in str:
> -                if is_string(elem): # if string
> -                    type, sep, var = elem.rpartition('*')
> +        if len(event.args)>  0:
> +            for type_, name in event.args:
> +                # string
> +                if type_is_string(type_):
>                       print '''
> -    slen = strlen(%(var)s);
> +    slen = strlen(%(name)s);
>       write_to_buffer(rec_off, (uint8_t*)&slen, sizeof(slen));
>       rec_off += sizeof(slen);''' % {
> -    'var': var.lstrip()
> +    'name': name
>   }
>                       print '''
> -    write_to_buffer(rec_off, (uint8_t*)%(var)s, slen);
> +    write_to_buffer(rec_off, (uint8_t*)%(name)s, slen);
>       rec_off += slen;''' % {
> -    'var': var.lstrip()
> +    'name': name
>   }
> -                elif '*' in elem: # pointer var (not string)
> -                    type, sep, var = elem.rpartition('*')
> +                # pointer var (not string)
> +                elif type_.endswith('*'):
>                       print '''
> -    pvar64 = (uint64_t)(uint64_t*)%(var)s;
> +    pvar64 = (uint64_t)(uint64_t*)%(name)s;
>       write_to_buffer(rec_off, (uint8_t*)&pvar64, sizeof(uint64_t));
>       rec_off += sizeof(uint64_t);''' % {
> -    'var': var.lstrip()
> +    'name': name
>   }
> -                else: # primitive data type
> -                    type, sep, var = elem.rpartition(' ')
> +                # primitive data type
> +                else:
>                       print '''
> -    var64 = (uint64_t)%(var)s;
> +    var64 = (uint64_t)%(name)s;
>       write_to_buffer(rec_off, (uint8_t*)&var64, sizeof(uint64_t));
>       rec_off += sizeof(uint64_t);''' % {
> -    'var': var.lstrip()
> +    'name': name
>   }
>           print '''
> -    trace_mark_record_complete(tbuf_idx);'''
> -        print '}'
> -        print
> +    trace_mark_record_complete(tbuf_idx);
> +}
> +
> +'''
>
>       return
>
> @@ -220,12 +178,11 @@ def stderr_h(events):
>   #include "trace/stderr.h"
>
>   extern TraceEvent trace_list[];'''
> +
>       for num, event in enumerate(events):
> -        argnames = event.argnames
> -        if event.argc>  0:
> -            argnames = ', ' + event.argnames
> -        else:
> -            argnames = ''
> +        argnames = ", ".join(event.args.names())
> +        if len(event.args)>  0:
> +            argnames = ", "+argnames
>           print '''
>   static inline void trace_%(name)s(%(args)s)
>   {
> @@ -262,13 +219,13 @@ def ust_h(events):
>   #undef wmb'''
>
>       for event in events:
> -        if event.argc>  0:
> +        if len(event.args)>  0:
>               print '''
>   DECLARE_TRACE(ust_%(name)s, TP_PROTO(%(args)s), TP_ARGS(%(argnames)s));
>   #define trace_%(name)s trace_ust_%(name)s''' % {
>       'name': event.name,
>       'args': event.args,
> -    'argnames': event.argnames
> +    'argnames': ", ".join(event.args.names())
>   }
>           else:
>               print '''
> @@ -287,9 +244,9 @@ def ust_c(events):
>   #undef wmb
>   #include "trace.h"'''
>       for event in events:
> -        argnames = event.argnames
> -        if event.argc>  0:
> -            argnames = ', ' + event.argnames
> +        argnames = ", ".join(event.args.names())
> +        if len(event.args)>  0:
> +            argnames = ', ' + argnames
>               print '''
>   DEFINE_TRACE(ust_%(name)s);
>
> @@ -339,7 +296,7 @@ def dtrace_h(events):
>       'name': event.name,
>       'args': event.args,
>       'uppername': event.name.upper(),
> -    'argnames': event.argnames,
> +    'argnames': ", ".join(event.args.names()),
>   }
>
>   def dtrace_c(events):
> @@ -379,12 +336,12 @@ probe %(probeprefix)s.%(name)s = process("%(binary)s").mark("%(name)s")
>       'binary': binary
>   }
>           i = 1
> -        if event.argc>  0:
> -            for arg in event.argnames.split(','):
> +        if len(event.args)>  0:
> +            for name in event.args.names():
>                   # 'limit' is a reserved keyword
> -                if arg == 'limit':
> -                    arg = '_limit'
> -                print '  %s = $arg%d;' % (arg.lstrip(), i)
> +                if name == 'limit':
> +                    name = '_limit'
> +                print '  %s = $arg%d;' % (name.lstrip(), i)
>                   i += 1
>           print '}'
>       print
> @@ -478,6 +435,47 @@ trace_gen = {
>       },
>   }
>
> +# Event arguments
> +def type_is_string(type_):
> +    strtype = ('const char*', 'char*', 'const char *', 'char *')
> +    return type_.startswith(strtype)
> +
> +class Arguments:
> +    def __init__ (self, arg_str):
> +        self._args = []
> +        for arg in arg_str.split(","):
> +            arg = arg.strip()
> +            parts = arg.split()
> +            head, sep, tail = parts[-1].rpartition("*")
> +            parts = parts[:-1]
> +            if tail == "void":
> +                assert len(parts) == 0 and sep == ""
> +                continue
> +            arg_type = " ".join(parts + [ " ".join([head, sep]).strip() ]).strip()
> +            self._args.append((arg_type, tail))
> +
> +    def __iter__(self):
> +        return iter(self._args)
> +
> +    def __len__(self):
> +        return len(self._args)
> +
> +    def __str__(self):
> +        if len(self._args) == 0:
> +            return "void"
> +        else:
> +            return ", ".join([ " ".join([t, n]) for t,n in self._args ])
> +
> +    def names(self):
> +        return [ name for _, name in self._args ]
> +
> +    def types(self):
> +        return [ type_ for type_, _ in self._args ]
> +
> +    def size_str(self):
> +        res = ""
> +        return res
> +
>   # A trace event
>   import re
>   cre = re.compile("((?P<props>.*)\s+)?(?P<name>[^(\s]+)\((?P<args>[^)]*)\)\s*(?P<fmt>\".*)?")
> @@ -489,17 +487,11 @@ class Event(object):
>           m = cre.match(line)
>           assert m is not None
>           groups = m.groupdict('')
> -        self.args = groups["args"]
> -        self.arglist = self.args.split(',')
>           self.name = groups["name"]
> -        if len(self.arglist) == 1 and self.arglist[0] == "void":
> -            self.argc = 0
> -        else:
> -            self.argc = len(self.arglist)
> -        self.argnames = get_argnames(self.args)
> -        self.sizestr = calc_sizeofargs(self.args, self.argc)
>           self.fmt = groups["fmt"]
>           self.properties = groups["props"].split()
> +        self.args = Arguments(groups["args"])
> +
>           unknown_props = set(self.properties) - VALID_PROPS
>           if len(unknown_props)>  0:
>               raise ValueError("Unknown properties: %s" % ", ".join(unknown_props))
>

Re: [Qemu-devel] [PATCH v4 07/11] trace: [tracetool] Rewrite event argument parsing

[Lluís Vilanova]

Harsh Bora writes:

> BTW, I am manually applying your changes on top of my patches as there were
> significant changes in my patches also. I will include your patches in my next
> series.

Excellent.


Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

Re: [Qemu-devel] [PATCH v4 00/11] tracetool: Improvements for future expansion

[Lluís Vilanova]

Lluís Vilanova writes:

> NOTE: Depend's on Harsh's port of tractool to python.
> A general overhaul of the pythonic tracetool script to allow simpler future
> extensions.

Please disregard this series; it has been superseded by the latest tracetool
rewrite series.


Thanks,
    Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth