[Qemu-devel] [RFC PATCH v2 0/3] Fix UST backend for LTTng 2.x

[Qemu-devel] [RFC PATCH v2 0/3] Fix UST backend for LTTng 2.x

[Mohamad Gebai]

Version 2 

* Fix tracepoint generation error in first version.
* Avoid warnings and errors specific to LTTng ust 2.0.
* If using LTTng ust 2.0, we can't avoid getting warnings because of the
  --warn-common option given to the linker. This is fixed in more recent
  versions of LTTng ust.

Mohamad

Mohamad Gebai (3):
  Fix configure script for LTTng 2.x
  Modified the tracetool framework for LTTng 2.x.
  Adapt Makefiles to the new LTTng ust interface.

 Makefile                                 |    5 ++
 configure                                |    9 ++--
 scripts/tracetool/backend/events.py      |   44 ++++++++++++++++
 scripts/tracetool/backend/ust.py         |   82 ++++++------------------------
 scripts/tracetool/format/ust_events_c.py |   30 +++++++++++
 scripts/tracetool/format/ust_events_h.py |   57 +++++++++++++++++++++
 trace/Makefile.objs                      |   29 ++++++++++-
 7 files changed, 183 insertions(+), 73 deletions(-)
 create mode 100644 scripts/tracetool/format/ust_events_c.py
 create mode 100644 scripts/tracetool/format/ust_events_h.py

-- 
1.7.10.4

[Qemu-devel] [RFC PATCH v2 1/3] Fix configure script for LTTng 2.x

[Mohamad Gebai]

Signed-off-by: Mohamad Gebai <mohamad.gebai@polymtl.ca>
---
 configure |    9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/configure b/configure
index 23dbaaf..627054c 100755
--- a/configure
+++ b/configure
@@ -3287,15 +3287,14 @@ fi
 # For 'ust' backend, test if ust headers are present
 if test "$trace_backend" = "ust"; then
   cat > $TMPC << EOF
-#include <ust/tracepoint.h>
-#include <ust/marker.h>
+#include <lttng/tracepoint.h>
 int main(void) { return 0; }
 EOF
   if compile_prog "" "" ; then
-    LIBS="-lust -lurcu-bp $LIBS"
-    libs_qga="-lust -lurcu-bp $libs_qga"
+    LIBS="-llttng-ust -lurcu-bp -ldl $LIBS"
+    libs_qga="-llttng-ust -lurcu-bp -ldl $libs_qga"
   else
-    error_exit "Trace backend 'ust' missing libust header files"
+    error_exit "Trace backend 'ust' missing lttng-ust header files"
   fi
 fi
 
-- 
1.7.10.4

[Qemu-devel] [RFC PATCH v2 2/3] Modified the tracetool framework for LTTng 2.x.

[Mohamad Gebai]

* A new format is required to generate definitions for ust tracepoints.
  Files ust_events_h.py and ust_events_c.py define common macros, while
  new function ust_events_h in events.py does the actual definition of
  each tracepoint.
* ust.py generates the new interface for calling userspace tracepoints
  with LTTng 2.x, replacing trace_name(args) to tracepoint(name, args).
* As explained in ust_events_c.py, -Wredundant-decls gives a warning
  when compiling with gcc 4.7 or older. This is specific to lttng-ust so
  for now use a pragma clause to avoid getting a warning.

Signed-off-by: Mohamad Gebai <mohamad.gebai@polymtl.ca>
---
 scripts/tracetool/backend/events.py      |   44 ++++++++++++++++
 scripts/tracetool/backend/ust.py         |   82 ++++++------------------------
 scripts/tracetool/format/ust_events_c.py |   30 +++++++++++
 scripts/tracetool/format/ust_events_h.py |   57 +++++++++++++++++++++
 4 files changed, 147 insertions(+), 66 deletions(-)
 create mode 100644 scripts/tracetool/format/ust_events_c.py
 create mode 100644 scripts/tracetool/format/ust_events_h.py

diff --git a/scripts/tracetool/backend/events.py b/scripts/tracetool/backend/events.py
index 5afce3e..49b7c65 100644
--- a/scripts/tracetool/backend/events.py
+++ b/scripts/tracetool/backend/events.py
@@ -6,6 +6,8 @@ Generic event description.
 
 This is a dummy backend to establish appropriate frontend/backend compatibility
 checks.
+
+Generates tracepoint definitions for LTTng userspace tracing when using ust as a backend.
 """
 
 __author__     = "Lluís Vilanova <vilanova@ac.upc.edu>"
@@ -16,8 +18,50 @@ __maintainer__ = "Stefan Hajnoczi"
 __email__      = "stefanha@linux.vnet.ibm.com"
 
 
+from tracetool import out
+
 def events_h(events):
     pass
 
 def events_c(events):
     pass
+
+def ust_events_c(events):
+    pass
+
+def ust_events_h(events):
+    for e in events:
+        if len(e.args) > 0:
+            out('TRACEPOINT_EVENT(',
+                '   ust,',
+                '   %(name)s,',
+                '   TP_ARGS(%(args)s),',
+                '   TP_FIELDS(',
+                name = e.name,
+                args = ", ".join(", ".join(i) for i in e.args),
+                )
+
+            for t,n in e.args:
+                if ('int' in t) or ('long' in t) or ('unsigned' in t) or ('size_t' in t):
+                    out('       ctf_integer(' + t + ', ' + n + ', ' + n + ')')
+                elif ('double' in t) or ('float' in t):
+                    out('       ctf_float(' + t + ', ' + n + ', ' + n + ')')
+                elif ('char *' in t) or ('char*' in t):
+                    out('       ctf_string(' + n + ', ' + n + ')')
+                elif ('void *' in t) or ('void*' in t):
+                    out('       ctf_integer_hex(unsigned long, ' + n + ', ' + n + ')')
+
+            out('   )',
+                ')',
+                '')
+
+        else:
+            out('TRACEPOINT_EVENT(',
+                '   ust,',
+                '   %(name)s,',
+                '   TP_ARGS(void),',
+                '   TP_FIELDS()',
+                ')',
+                '',
+                name = e.name,
+                )
diff --git a/scripts/tracetool/backend/ust.py b/scripts/tracetool/backend/ust.py
index ea36995..876b6fd 100644
--- a/scripts/tracetool/backend/ust.py
+++ b/scripts/tracetool/backend/ust.py
@@ -18,76 +18,26 @@ from tracetool import out
 
 PUBLIC = True
 
-
 def c(events):
-    out('#include <ust/marker.h>',
-        '#undef mutex_lock',
-        '#undef mutex_unlock',
-        '#undef inline',
-        '#undef wmb',
-        '#include "trace.h"')
+    pass
+
 
+def h(events):
+    out('#include <lttng/tracepoint.h>',
+        '#include "trace/generated-ust-provider.h"',
+        '')
     for e in events:
         argnames = ", ".join(e.args.names())
         if len(e.args) > 0:
-            argnames = ', ' + argnames
-
-            out('DEFINE_TRACE(ust_%(name)s);',
-                '',
-                'static void ust_%(name)s_probe(%(args)s)',
-                '{',
-                '    trace_mark(ust, %(name)s, %(fmt)s%(argnames)s);',
-                '}',
-                name = e.name,
-                args = e.args,
-                fmt = e.fmt,
-                argnames = argnames,
-                )
-
-        else:
-            out('DEFINE_TRACE(ust_%(name)s);',
-                '',
-                'static void ust_%(name)s_probe(%(args)s)',
-                '{',
-                '    trace_mark(ust, %(name)s, UST_MARKER_NOARGS);',
-                '}',
-                name = e.name,
-                args = e.args,
-                )
-
-    # register probes
-    out('',
-        'static void __attribute__((constructor)) trace_init(void)',
-        '{')
-
-    for e in events:
-        out('    register_trace_ust_%(name)s(ust_%(name)s_probe);',
+            argnames = ", " + argnames
+
+        out('static inline void trace_%(name)s(%(args)s)',
+            '{',
+            '    tracepoint(ust, %(name)s%(tp_args)s);'
+            '',
+            '}',
+            '',
             name = e.name,
+            args = e.args,
+            tp_args = argnames,
             )
-
-    out('}')
-
-
-def h(events):
-    out('#include <ust/tracepoint.h>',
-        '#undef mutex_lock',
-        '#undef mutex_unlock',
-        '#undef inline',
-        '#undef wmb')
-
-    for e in events:
-        if len(e.args) > 0:
-            out('DECLARE_TRACE(ust_%(name)s, TP_PROTO(%(args)s), TP_ARGS(%(argnames)s));',
-                '#define trace_%(name)s trace_ust_%(name)s',
-                name = e.name,
-                args = e.args,
-                argnames = ", ".join(e.args.names()),
-                )
-
-        else:
-            out('_DECLARE_TRACEPOINT_NOARGS(ust_%(name)s);',
-                '#define trace_%(name)s trace_ust_%(name)s',
-                name = e.name,
-                )
-
-    out()
diff --git a/scripts/tracetool/format/ust_events_c.py b/scripts/tracetool/format/ust_events_c.py
new file mode 100644
index 0000000..71328f3
--- /dev/null
+++ b/scripts/tracetool/format/ust_events_c.py
@@ -0,0 +1,30 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+"""
+Generate .c for LTTng ust event description.
+"""
+
+__author__     = "Lluís Vilanova <vilanova@ac.upc.edu>"
+__copyright__  = "Copyright 2012, Lluís Vilanova <vilanova@ac.upc.edu>"
+__license__    = "GPL version 2 or (at your option) any later version"
+
+__maintainer__ = "Stefan Hajnoczi"
+__email__      = "stefanha@linux.vnet.ibm.com"
+
+
+from tracetool import out
+
+
+def begin(events):
+    out('/* This file is autogenerated by tracetool, do not edit. */',
+        '',
+        '#define TRACEPOINT_DEFINE',
+        '#define TRACEPOINT_CREATE_PROBES',
+        '',
+        '/* If gcc version 4.7 or older is used, LTTng ust gives a warning when compiling with',
+        '   -Wredundant-decls.',
+        ' */',
+        '#pragma GCC diagnostic ignored "-Wredundant-decls"',
+        '',
+        '#include "generated-ust-provider.h"')
diff --git a/scripts/tracetool/format/ust_events_h.py b/scripts/tracetool/format/ust_events_h.py
new file mode 100644
index 0000000..6f26b9b
--- /dev/null
+++ b/scripts/tracetool/format/ust_events_h.py
@@ -0,0 +1,57 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+"""
+Generate .h for LTTng ust event description.
+"""
+
+__author__     = "Lluís Vilanova <vilanova@ac.upc.edu>"
+__copyright__  = "Copyright 2012, Lluís Vilanova <vilanova@ac.upc.edu>"
+__license__    = "GPL version 2 or (at your option) any later version"
+
+__maintainer__ = "Stefan Hajnoczi"
+__email__      = "stefanha@linux.vnet.ibm.com"
+
+
+from tracetool import out
+
+
+def begin(events):
+    out('/* This file is autogenerated by tracetool, do not edit. */',
+        '',
+        '#undef TRACEPOINT_PROVIDER',
+        '#define TRACEPOINT_PROVIDER ust',
+        '',
+        '#undef TRACEPOINT_INCLUDE_FILE',
+        '#define TRACEPOINT_INCLUDE_FILE ./generated-ust-provider.h',
+        '',
+        '#if !defined (TRACE__GENERATED_UST_H) || defined(TRACEPOINT_HEADER_MULTI_READ)',
+        '#define TRACE__GENERATED_UST_H',
+        '',
+        '#include "qemu-common.h"',
+        '#include <lttng/tracepoint.h>',
+        '',
+        '/*',
+        ' * LTTng ust 2.0 does not allow you to use TP_ARGS(void) for tracepoints',
+        ' * requiring no arguments. We define these macros introduced in more recent'
+        ' * versions of LTTng ust as a workaround',
+        ' */',
+        '#ifndef _TP_EXPROTO1',
+        '#define _TP_EXPROTO1(a)               void',
+        '#endif',
+        '#ifndef _TP_EXDATA_PROTO1',
+        '#define _TP_EXDATA_PROTO1(a)          void *__tp_data',
+        '#endif',
+        '#ifndef _TP_EXDATA_VAR1',
+        '#define _TP_EXDATA_VAR1(a)            __tp_data',
+        '#endif',
+        '#ifndef _TP_EXVAR1',
+        '#define _TP_EXVAR1(a)',
+        '#endif',
+        '')
+
+def end(events):
+    out('#endif /* TRACE__GENERATED_UST_H */',
+        '',
+        '/* This part must be outside ifdef protection */',
+        '#include <lttng/tracepoint-event.h>')
-- 
1.7.10.4

[Qemu-devel] [RFC PATCH v2 3/3] Adapt Makefiles to the new LTTng ust interface.

[Mohamad Gebai]

Add generation of new files for LTTng ust.

Signed-off-by: Mohamad Gebai <mohamad.gebai@polymtl.ca>
---
 Makefile            |    5 +++++
 trace/Makefile.objs |   29 +++++++++++++++++++++++++++--
 2 files changed, 32 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 60fb87e..0c797c5 100644
--- a/Makefile
+++ b/Makefile
@@ -50,6 +50,11 @@ GENERATED_HEADERS += trace/generated-tracers-dtrace.h
 endif
 GENERATED_SOURCES += trace/generated-tracers.c
 
+ifeq ($(TRACE_BACKEND),ust)
+GENERATED_HEADERS += trace/generated-ust-provider.h
+GENERATED_SOURCES += trace/generated-ust.c
+endif
+
 # Don't try to regenerate Makefile or configure
 # We don't generate any of them
 Makefile: ;
diff --git a/trace/Makefile.objs b/trace/Makefile.objs
index 3b88e49..00880b3 100644
--- a/trace/Makefile.objs
+++ b/trace/Makefile.objs
@@ -1,9 +1,33 @@
 # -*- mode: makefile -*-
 
 ######################################################################
+# Auto-generated event descriptions for LTTng ust code
+
+ifeq ($(TRACE_BACKEND),ust)
+$(obj)/generated-ust-provider.h: $(obj)/generated-ust-provider.h-timestamp
+$(obj)/generated-ust-provider.h-timestamp: $(SRC_PATH)/trace-events
+	$(call quiet-command,$(TRACETOOL) \
+		--format=ust-events-h \
+		--backend=events \
+		< $< > $@,"  GEN   $(patsubst %-timestamp,%,$@)")
+	@cmp -s $@ $(patsubst %-timestamp,%,$@) || cp $@ $(patsubst %-timestamp,%,$@)
+
+$(obj)/generated-ust.c: $(obj)/generated-ust.c-timestamp $(BUILD_DIR)/config-host.mak
+$(obj)/generated-ust.c-timestamp: $(SRC_PATH)/trace-events
+	$(call quiet-command,$(TRACETOOL) \
+		--format=ust-events-c \
+		--backend=events \
+		< $< > $@,"  GEN   $(patsubst %-timestamp,%,$@)")
+	@cmp -s $@ $(patsubst %-timestamp,%,$@) || cp $@ $(patsubst %-timestamp,%,$@)
+
+ust_provider=$(obj)/generated-ust-provider.h
+ust=$(obj)/generated-ust.c
+endif
+
+######################################################################
 # Auto-generated event descriptions
 
-$(obj)/generated-events.h: $(obj)/generated-events.h-timestamp
+$(obj)/generated-events.h: $(obj)/generated-events.h-timestamp $(ust_provider)
 $(obj)/generated-events.h-timestamp: $(SRC_PATH)/trace-events
 	$(call quiet-command,$(TRACETOOL) \
 		--format=events-h \
@@ -11,7 +35,7 @@ $(obj)/generated-events.h-timestamp: $(SRC_PATH)/trace-events
 		< $< > $@,"  GEN   $(patsubst %-timestamp,%,$@)")
 	@cmp -s $@ $(patsubst %-timestamp,%,$@) || cp $@ $(patsubst %-timestamp,%,$@)
 
-$(obj)/generated-events.c: $(obj)/generated-events.c-timestamp $(BUILD_DIR)/config-host.mak
+$(obj)/generated-events.c: $(obj)/generated-events.c-timestamp $(BUILD_DIR)/config-host.mak $(ust)
 $(obj)/generated-events.c-timestamp: $(SRC_PATH)/trace-events
 	$(call quiet-command,$(TRACETOOL) \
 		--format=events-c \
@@ -77,5 +101,6 @@ util-obj-$(CONFIG_TRACE_DEFAULT) += default.o
 util-obj-$(CONFIG_TRACE_SIMPLE) += simple.o
 util-obj-$(CONFIG_TRACE_STDERR) += stderr.o
 util-obj-$(CONFIG_TRACE_FTRACE) += ftrace.o
+util-obj-$(CONFIG_TRACE_UST) += generated-ust.o
 util-obj-y += control.o
 util-obj-y += generated-tracers.o
-- 
1.7.10.4

Re: [Qemu-devel] [RFC PATCH v2 0/3] Fix UST backend for LTTng 2.x

[Alex Bennée]

mohamad.gebai@gmail.com writes:

> Version 2 
>
> * Fix tracepoint generation error in first version.
> * Avoid warnings and errors specific to LTTng ust 2.0.
> * If using LTTng ust 2.0, we can't avoid getting warnings because of the
>   --warn-common option given to the linker. This is fixed in more recent
>   versions of LTTng ust.

Looking good. Builds for me now although I need to do some reading to
actually test it out. One omission is docs/tracing.txt gives examples
for the other tracing backend but just refers to "UST utilities should
be used to list, enable/disable, and dump traces.". Perhaps a few simple
examples could be added to the document so people can at least confirm
everything is working as it should be. Otherwise I'm happy.

Reviewed-by: Alex Bennée <alex@bennee.com>

>
> Mohamad
>
> Mohamad Gebai (3):
>   Fix configure script for LTTng 2.x
>   Modified the tracetool framework for LTTng 2.x.
>   Adapt Makefiles to the new LTTng ust interface.
>
>  Makefile                                 |    5 ++
>  configure                                |    9 ++--
>  scripts/tracetool/backend/events.py      |   44 ++++++++++++++++
>  scripts/tracetool/backend/ust.py         |   82 ++++++------------------------
>  scripts/tracetool/format/ust_events_c.py |   30 +++++++++++
>  scripts/tracetool/format/ust_events_h.py |   57 +++++++++++++++++++++
>  trace/Makefile.objs                      |   29 ++++++++++-
>  7 files changed, 183 insertions(+), 73 deletions(-)
>  create mode 100644 scripts/tracetool/format/ust_events_c.py
>  create mode 100644 scripts/tracetool/format/ust_events_h.py


-- 
Alex Bennée

Re: [Qemu-devel] [RFC PATCH v2 0/3] Fix UST backend for LTTng 2.x

[Mohamad Gebai]

Sorry for the delay, just submitted the updated documentation.

Thanks,
Mohamad

On 13-10-07 08:36 AM, Alex Bennée wrote:
> mohamad.gebai@gmail.com writes:
>
>> Version 2
>>
>> * Fix tracepoint generation error in first version.
>> * Avoid warnings and errors specific to LTTng ust 2.0.
>> * If using LTTng ust 2.0, we can't avoid getting warnings because of the
>>    --warn-common option given to the linker. This is fixed in more recent
>>    versions of LTTng ust.
> Looking good. Builds for me now although I need to do some reading to
> actually test it out. One omission is docs/tracing.txt gives examples
> for the other tracing backend but just refers to "UST utilities should
> be used to list, enable/disable, and dump traces.". Perhaps a few simple
> examples could be added to the document so people can at least confirm
> everything is working as it should be. Otherwise I'm happy.
>
> Reviewed-by: Alex Bennée <alex@bennee.com>
>
>> Mohamad
>>
>> Mohamad Gebai (3):
>>    Fix configure script for LTTng 2.x
>>    Modified the tracetool framework for LTTng 2.x.
>>    Adapt Makefiles to the new LTTng ust interface.
>>
>>   Makefile                                 |    5 ++
>>   configure                                |    9 ++--
>>   scripts/tracetool/backend/events.py      |   44 ++++++++++++++++
>>   scripts/tracetool/backend/ust.py         |   82 ++++++------------------------
>>   scripts/tracetool/format/ust_events_c.py |   30 +++++++++++
>>   scripts/tracetool/format/ust_events_h.py |   57 +++++++++++++++++++++
>>   trace/Makefile.objs                      |   29 ++++++++++-
>>   7 files changed, 183 insertions(+), 73 deletions(-)
>>   create mode 100644 scripts/tracetool/format/ust_events_c.py
>>   create mode 100644 scripts/tracetool/format/ust_events_h.py
>

Re: [Qemu-devel] [RFC PATCH v2 0/3] Fix UST backend for LTTng 2.x

[Stefan Hajnoczi]

On Mon, Oct 14, 2013 at 01:05:14PM -0400, Mohamad Gebai wrote:
> Sorry for the delay, just submitted the updated documentation.

Great, thanks.  Things are busy here due to KVM Forum/LinuxCon Europe
next week.  I'll try to take a look this week though so we can merge
this.

Stefan