[Qemu-devel] [PATCH v6 0/5] Fix UST backend for LTTng 2.x

[Qemu-devel] [PATCH v6 0/5] Fix UST backend for LTTng 2.x

[Mohamad Gebai]

Version 6

* Move ust specific code from tracetool/backend/events.py to
  tracetool/backend/ust.py

Mohamad Gebai (5):
  Fix configure script for LTTng 2.x
  Modified the tracetool framework for LTTng 2.x
  Adapt Makefiles to the new LTTng ust interface
  Update documentation for LTTng ust tracing
  Add ust generated files to .gitignore

 .gitignore                               |   2 +
 Makefile                                 |   5 ++
 configure                                |  20 ++++--
 docs/tracing.txt                         |  36 +++++++++++
 scripts/tracetool/backend/ust.py         | 101 ++++++++++++++-----------------
 scripts/tracetool/format/ust_events_c.py |  30 +++++++++
 scripts/tracetool/format/ust_events_h.py |  57 +++++++++++++++++
 trace/Makefile.objs                      |  25 ++++++++
 8 files changed, 215 insertions(+), 61 deletions(-)
 create mode 100644 scripts/tracetool/format/ust_events_c.py
 create mode 100644 scripts/tracetool/format/ust_events_h.py

-- 
1.8.5.2

[Qemu-devel] [PATCH v6 1/5] Fix configure script for LTTng 2.x

[Mohamad Gebai]

Signed-off-by: Mohamad Gebai <mohamad.gebai@polymtl.ca>
---
 configure | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)

diff --git a/configure b/configure
index b472694..2a14014 100755
--- a/configure
+++ b/configure
@@ -3365,15 +3365,25 @@ fi
 # For 'ust' backend, test if ust headers are present
 if test "$trace_backend" = "ust"; then
   cat > $TMPC << EOF
-#include <ust/tracepoint.h>
-#include <ust/marker.h>
+#include <lttng/tracepoint.h>
 int main(void) { return 0; }
 EOF
   if compile_prog "" "" ; then
-    LIBS="-lust -lurcu-bp $LIBS"
-    libs_qga="-lust -lurcu-bp $libs_qga"
+    if $pkg_config lttng-ust --exists; then
+      lttng_ust_libs=`$pkg_config --libs lttng-ust`
+    else
+      lttng_ust_libs="-llttng-ust"
+    fi
+    if $pkg_config liburcu-bp --exists; then
+      urcu_bp_libs=`$pkg_config --libs liburcu-bp`
+    else
+      urcu_bp_libs="-lurcu-bp"
+    fi
+
+    LIBS="$lttng_ust_libs $urcu_bp_libs $LIBS"
+    libs_qga="$lttng_ust_libs $urcu_bp_libs $libs_qga"
   else
-    error_exit "Trace backend 'ust' missing libust header files"
+    error_exit "Trace backend 'ust' missing lttng-ust header files"
   fi
 fi
 
-- 
1.8.5.2

[Qemu-devel] [PATCH v6 2/5] Modified the tracetool framework for LTTng 2.x

[Mohamad Gebai]

* A new format is required to generate definitions for ust tracepoints.
  Files ust_events_h.py and ust_events_c.py define common macros, while
  new function ust_events_h in events.py does the actual definition of
  each tracepoint.
* ust.py generates the new interface for calling userspace tracepoints
  with LTTng 2.x, replacing trace_name(args) to tracepoint(name, args).
* As explained in ust_events_c.py, -Wredundant-decls gives a warning
  when compiling with gcc 4.7 or older. This is specific to lttng-ust so
  for now use a pragma clause to avoid getting a warning.

Signed-off-by: Mohamad Gebai <mohamad.gebai@polymtl.ca>
Reviewed-by: Alex Bennée <alex@bennee.com>
---
 scripts/tracetool/backend/ust.py         | 101 ++++++++++++++-----------------
 scripts/tracetool/format/ust_events_c.py |  30 +++++++++
 scripts/tracetool/format/ust_events_h.py |  57 +++++++++++++++++
 3 files changed, 132 insertions(+), 56 deletions(-)
 create mode 100644 scripts/tracetool/format/ust_events_c.py
 create mode 100644 scripts/tracetool/format/ust_events_h.py

diff --git a/scripts/tracetool/backend/ust.py b/scripts/tracetool/backend/ust.py
index ea36995..41c1c75 100644
--- a/scripts/tracetool/backend/ust.py
+++ b/scripts/tracetool/backend/ust.py
@@ -18,76 +18,65 @@ from tracetool import out
 
 PUBLIC = True
 
-
 def c(events):
-    out('#include <ust/marker.h>',
-        '#undef mutex_lock',
-        '#undef mutex_unlock',
-        '#undef inline',
-        '#undef wmb',
-        '#include "trace.h"')
+    pass
+
 
+def h(events):
+    out('#include <lttng/tracepoint.h>',
+        '#include "trace/generated-ust-provider.h"',
+        '')
     for e in events:
         argnames = ", ".join(e.args.names())
         if len(e.args) > 0:
-            argnames = ', ' + argnames
-
-            out('DEFINE_TRACE(ust_%(name)s);',
-                '',
-                'static void ust_%(name)s_probe(%(args)s)',
-                '{',
-                '    trace_mark(ust, %(name)s, %(fmt)s%(argnames)s);',
-                '}',
-                name = e.name,
-                args = e.args,
-                fmt = e.fmt,
-                argnames = argnames,
-                )
-
-        else:
-            out('DEFINE_TRACE(ust_%(name)s);',
-                '',
-                'static void ust_%(name)s_probe(%(args)s)',
-                '{',
-                '    trace_mark(ust, %(name)s, UST_MARKER_NOARGS);',
-                '}',
-                name = e.name,
-                args = e.args,
-                )
-
-    # register probes
-    out('',
-        'static void __attribute__((constructor)) trace_init(void)',
-        '{')
+            argnames = ", " + argnames
 
-    for e in events:
-        out('    register_trace_ust_%(name)s(ust_%(name)s_probe);',
+        out('static inline void trace_%(name)s(%(args)s)',
+            '{',
+            '    tracepoint(qemu, %(name)s%(tp_args)s);',
+            '}',
+            '',
             name = e.name,
+            args = e.args,
+            tp_args = argnames,
             )
 
-    out('}')
-
-
-def h(events):
-    out('#include <ust/tracepoint.h>',
-        '#undef mutex_lock',
-        '#undef mutex_unlock',
-        '#undef inline',
-        '#undef wmb')
+def ust_events_c(events):
+    pass
 
+def ust_events_h(events):
     for e in events:
         if len(e.args) > 0:
-            out('DECLARE_TRACE(ust_%(name)s, TP_PROTO(%(args)s), TP_ARGS(%(argnames)s));',
-                '#define trace_%(name)s trace_ust_%(name)s',
+            out('TRACEPOINT_EVENT(',
+                '   qemu,',
+                '   %(name)s,',
+                '   TP_ARGS(%(args)s),',
+                '   TP_FIELDS(',
                 name = e.name,
-                args = e.args,
-                argnames = ", ".join(e.args.names()),
+                args = ", ".join(", ".join(i) for i in e.args),
                 )
 
+            for t,n in e.args:
+                if ('int' in t) or ('long' in t) or ('unsigned' in t) or ('size_t' in t):
+                    out('       ctf_integer(' + t + ', ' + n + ', ' + n + ')')
+                elif ('double' in t) or ('float' in t):
+                    out('       ctf_float(' + t + ', ' + n + ', ' + n + ')')
+                elif ('char *' in t) or ('char*' in t):
+                    out('       ctf_string(' + n + ', ' + n + ')')
+                elif ('void *' in t) or ('void*' in t):
+                    out('       ctf_integer_hex(unsigned long, ' + n + ', ' + n + ')')
+
+            out('   )',
+                ')',
+                '')
+
         else:
-            out('_DECLARE_TRACEPOINT_NOARGS(ust_%(name)s);',
-                '#define trace_%(name)s trace_ust_%(name)s',
+            out('TRACEPOINT_EVENT(',
+                '   qemu,',
+                '   %(name)s,',
+                '   TP_ARGS(void),',
+                '   TP_FIELDS()',
+                ')',
+                '',
                 name = e.name,
-                )
-
-    out()
+                )
\ No newline at end of file
diff --git a/scripts/tracetool/format/ust_events_c.py b/scripts/tracetool/format/ust_events_c.py
new file mode 100644
index 0000000..116e713
--- /dev/null
+++ b/scripts/tracetool/format/ust_events_c.py
@@ -0,0 +1,30 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+"""
+Generate .c for LTTng ust event description.
+"""
+
+__author__     = "Mohamad Gebai <mohamad.gebai@polymtl.ca>"
+__copyright__  = "Copyright 2012, Mohamad Gebai <mohamad.gebai@polymtl.ca>"
+__license__    = "GPL version 2 or (at your option) any later version"
+
+__maintainer__ = "Stefan Hajnoczi"
+__email__      = "stefanha@redhat.com"
+
+
+from tracetool import out
+
+
+def begin(events):
+    out('/* This file is autogenerated by tracetool, do not edit. */',
+        '',
+        '#define TRACEPOINT_DEFINE',
+        '#define TRACEPOINT_CREATE_PROBES',
+        '',
+        '/* If gcc version 4.7 or older is used, LTTng ust gives a warning when compiling with',
+        '   -Wredundant-decls.',
+        ' */',
+        '#pragma GCC diagnostic ignored "-Wredundant-decls"',
+        '',
+        '#include "generated-ust-provider.h"')
diff --git a/scripts/tracetool/format/ust_events_h.py b/scripts/tracetool/format/ust_events_h.py
new file mode 100644
index 0000000..f206eca
--- /dev/null
+++ b/scripts/tracetool/format/ust_events_h.py
@@ -0,0 +1,57 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+"""
+Generate .h for LTTng ust event description.
+"""
+
+__author__     = "Mohamad Gebai <mohamad.gebai@polymtl.ca>"
+__copyright__  = "Copyright 2012, Mohamad Gebai <mohamad.gebai@polymtl.ca>"
+__license__    = "GPL version 2 or (at your option) any later version"
+
+__maintainer__ = "Stefan Hajnoczi"
+__email__      = "stefanha@redhat.com"
+
+
+from tracetool import out
+
+
+def begin(events):
+    out('/* This file is autogenerated by tracetool, do not edit. */',
+        '',
+        '#undef TRACEPOINT_PROVIDER',
+        '#define TRACEPOINT_PROVIDER qemu',
+        '',
+        '#undef TRACEPOINT_INCLUDE_FILE',
+        '#define TRACEPOINT_INCLUDE_FILE ./generated-ust-provider.h',
+        '',
+        '#if !defined (TRACE__GENERATED_UST_H) || defined(TRACEPOINT_HEADER_MULTI_READ)',
+        '#define TRACE__GENERATED_UST_H',
+        '',
+        '#include "qemu-common.h"',
+        '#include <lttng/tracepoint.h>',
+        '',
+        '/*',
+        ' * LTTng ust 2.0 does not allow you to use TP_ARGS(void) for tracepoints',
+        ' * requiring no arguments. We define these macros introduced in more recent'
+        ' * versions of LTTng ust as a workaround',
+        ' */',
+        '#ifndef _TP_EXPROTO1',
+        '#define _TP_EXPROTO1(a)               void',
+        '#endif',
+        '#ifndef _TP_EXDATA_PROTO1',
+        '#define _TP_EXDATA_PROTO1(a)          void *__tp_data',
+        '#endif',
+        '#ifndef _TP_EXDATA_VAR1',
+        '#define _TP_EXDATA_VAR1(a)            __tp_data',
+        '#endif',
+        '#ifndef _TP_EXVAR1',
+        '#define _TP_EXVAR1(a)',
+        '#endif',
+        '')
+
+def end(events):
+    out('#endif /* TRACE__GENERATED_UST_H */',
+        '',
+        '/* This part must be outside ifdef protection */',
+        '#include <lttng/tracepoint-event.h>')
-- 
1.8.5.2

[Qemu-devel] [PATCH v6 3/5] Adapt Makefiles to the new LTTng ust interface

[Mohamad Gebai]

Add generation of new files for LTTng ust.

Signed-off-by: Mohamad Gebai <mohamad.gebai@polymtl.ca>
---
 Makefile            |  5 +++++
 trace/Makefile.objs | 25 +++++++++++++++++++++++++
 2 files changed, 30 insertions(+)

diff --git a/Makefile b/Makefile
index bdff4e4..ca51193 100644
--- a/Makefile
+++ b/Makefile
@@ -57,6 +57,11 @@ GENERATED_HEADERS += trace/generated-tracers-dtrace.h
 endif
 GENERATED_SOURCES += trace/generated-tracers.c
 
+ifeq ($(TRACE_BACKEND),ust)
+GENERATED_HEADERS += trace/generated-ust-provider.h
+GENERATED_SOURCES += trace/generated-ust.c
+endif
+
 # Don't try to regenerate Makefile or configure
 # We don't generate any of them
 Makefile: ;
diff --git a/trace/Makefile.objs b/trace/Makefile.objs
index 3b88e49..d321946 100644
--- a/trace/Makefile.objs
+++ b/trace/Makefile.objs
@@ -1,6 +1,30 @@
 # -*- mode: makefile -*-
 
 ######################################################################
+# Auto-generated event descriptions for LTTng ust code
+
+ifeq ($(TRACE_BACKEND),ust)
+$(obj)/generated-ust-provider.h: $(obj)/generated-ust-provider.h-timestamp
+$(obj)/generated-ust-provider.h-timestamp: $(SRC_PATH)/trace-events
+	$(call quiet-command,$(TRACETOOL) \
+		--format=ust-events-h \
+		--backend=$(TRACE_BACKEND) \
+		< $< > $@,"  GEN   $(patsubst %-timestamp,%,$@)")
+	@cmp -s $@ $(patsubst %-timestamp,%,$@) || cp $@ $(patsubst %-timestamp,%,$@)
+
+$(obj)/generated-ust.c: $(obj)/generated-ust.c-timestamp $(BUILD_DIR)/config-host.mak
+$(obj)/generated-ust.c-timestamp: $(SRC_PATH)/trace-events
+	$(call quiet-command,$(TRACETOOL) \
+		--format=ust-events-c \
+		--backend=$(TRACE_BACKEND) \
+		< $< > $@,"  GEN   $(patsubst %-timestamp,%,$@)")
+	@cmp -s $@ $(patsubst %-timestamp,%,$@) || cp $@ $(patsubst %-timestamp,%,$@)
+
+$(obj)/generated-events.h: $(obj)/generated-ust-provider.h
+$(obj)/generated-events.c: $(obj)/generated-ust.c
+endif
+
+######################################################################
 # Auto-generated event descriptions
 
 $(obj)/generated-events.h: $(obj)/generated-events.h-timestamp
@@ -77,5 +101,6 @@ util-obj-$(CONFIG_TRACE_DEFAULT) += default.o
 util-obj-$(CONFIG_TRACE_SIMPLE) += simple.o
 util-obj-$(CONFIG_TRACE_STDERR) += stderr.o
 util-obj-$(CONFIG_TRACE_FTRACE) += ftrace.o
+util-obj-$(CONFIG_TRACE_UST) += generated-ust.o
 util-obj-y += control.o
 util-obj-y += generated-tracers.o
-- 
1.8.5.2

[Qemu-devel] [PATCH v6 4/5] Update documentation for LTTng ust tracing

[Mohamad Gebai]

Signed-off-by: Mohamad Gebai <mohamad.gebai@polymtl.ca>
---
 docs/tracing.txt | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/docs/tracing.txt b/docs/tracing.txt
index bfc261b..bf2e15c 100644
--- a/docs/tracing.txt
+++ b/docs/tracing.txt
@@ -214,6 +214,42 @@ The "ust" backend uses the LTTng Userspace Tracer library.  There are no
 monitor commands built into QEMU, instead UST utilities should be used to list,
 enable/disable, and dump traces.
 
+Package lttng-tools is required for userspace tracing. You must ensure that the
+current user belongs to the "tracing" group, or manually launch the
+lttng-sessiond daemon for the current user prior to running any instance of
+QEMU.
+
+While running an instrumented QEMU, LTTng should be able to list all available
+events:
+
+    lttng list -u
+
+Create tracing session:
+
+    lttng create mysession
+
+Enable events:
+
+    lttng enable-event qemu:g_malloc -u
+
+Where the events can either be a comma-separated list of events, or "-a" to
+enable all tracepoint events. Start and stop tracing as needed:
+
+    lttng start
+    lttng stop
+
+View the trace:
+
+    lttng view
+
+Destroy tracing session:
+
+    lttng destroy
+
+Babeltrace can be used at any later time to view the trace:
+
+    babeltrace $HOME/lttng-traces/mysession-<date>-<time>
+
 === SystemTap ===
 
 The "dtrace" backend uses DTrace sdt probes but has only been tested with
-- 
1.8.5.2

[Qemu-devel] [PATCH v6 5/5] Add ust generated files to .gitignore

[Mohamad Gebai]

Signed-off-by: Mohamad Gebai <mohamad.gebai@polymtl.ca>
Reviewed-by: Alex Bennée <alex@bennee.com>
---
 .gitignore | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/.gitignore b/.gitignore
index 1c9d63d..bed8dcf 100644
--- a/.gitignore
+++ b/.gitignore
@@ -10,6 +10,8 @@ trace/generated-tracers-dtrace.h
 trace/generated-tracers.dtrace
 trace/generated-events.h
 trace/generated-events.c
+trace/generated-ust-provider.h
+trace/generated-ust.c
 libcacard/trace/generated-tracers.c
 *-timestamp
 *-softmmu
-- 
1.8.5.2

Re: [Qemu-devel] [PATCH v6 4/5] Update documentation for LTTng ust tracing

[Alex Bennée]

mohamad.gebai@gmail.com writes:

> Signed-off-by: Mohamad Gebai <mohamad.gebai@polymtl.ca>
> ---
>  docs/tracing.txt | 36 ++++++++++++++++++++++++++++++++++++
>  1 file changed, 36 insertions(+)
<snip>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée

Re: [Qemu-devel] [PATCH v6 0/5] Fix UST backend for LTTng 2.x

[Stefan Hajnoczi]

On Wed, Jan 29, 2014 at 10:47:53PM -0500, Mohamad Gebai wrote:
> Version 6
> 
> * Move ust specific code from tracetool/backend/events.py to
>   tracetool/backend/ust.py
> 
> Mohamad Gebai (5):
>   Fix configure script for LTTng 2.x
>   Modified the tracetool framework for LTTng 2.x
>   Adapt Makefiles to the new LTTng ust interface
>   Update documentation for LTTng ust tracing
>   Add ust generated files to .gitignore
> 
>  .gitignore                               |   2 +
>  Makefile                                 |   5 ++
>  configure                                |  20 ++++--
>  docs/tracing.txt                         |  36 +++++++++++
>  scripts/tracetool/backend/ust.py         | 101 ++++++++++++++-----------------
>  scripts/tracetool/format/ust_events_c.py |  30 +++++++++
>  scripts/tracetool/format/ust_events_h.py |  57 +++++++++++++++++
>  trace/Makefile.objs                      |  25 ++++++++
>  8 files changed, 215 insertions(+), 61 deletions(-)
>  create mode 100644 scripts/tracetool/format/ust_events_c.py
>  create mode 100644 scripts/tracetool/format/ust_events_h.py

Thanks for your efforts to bring the LTTng UST tracer back to life!

Applied to my tracing tree:
https://github.com/stefanha/qemu/commits/tracing

Stefan

Re: [Qemu-devel] [PATCH v6 0/5] Fix UST backend for LTTng 2.x

[Mohamad Gebai]

On 01/30/2014 09:02 AM, Stefan Hajnoczi wrote:
> Thanks for your efforts to bring the LTTng UST tracer back to life!
>
> Applied to my tracing tree:
> https://github.com/stefanha/qemu/commits/tracing
>
> Stefan

Great, thank you!

Mohamad