[Qemu-devel] [PATCH 01/13] instrument: Add documentation

From: "Lluís Vilanova" <vilanova@ac.upc.edu>
To: qemu-devel@nongnu.org
Cc: "Emilio G. Cota" <cota@braap.org>, Stefan Hajnoczi <stefanha@redhat.com>
Subject: [Qemu-devel] [PATCH 01/13] instrument: Add documentation
Date: Mon, 24 Jul 2017 20:06:26 +0300	[thread overview]
Message-ID: <150091598647.30739.2420187318162958941.stgit@frigg.lan> (raw)
In-Reply-To: <150091574424.30739.4131793221953168474.stgit@frigg.lan>

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
---
 docs/devel/tracing.txt   |    9 ++
 docs/instrumentation.txt |  264 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 273 insertions(+)
 create mode 100644 docs/instrumentation.txt

diff --git a/docs/devel/tracing.txt b/docs/devel/tracing.txt
index 5768a0b7a2..17fcd12469 100644
--- a/docs/devel/tracing.txt
+++ b/docs/devel/tracing.txt
@@ -439,3 +439,12 @@ If the translating vCPU has address 0xc1 and code is later executed by vCPU
     baz_trans cpu=0xc1 a=0xd3
     // at guest code execution
     baz_exec cpu=0xc2 a=0xd3
+
+=== "instrument" ===
+
+When compiling QEMU with trace instrumentation enabled, the "instrument"
+property lets you provide your own implementation for that trace event. This
+implementation can override and/or wrap the backend-specific tracing code
+(regardless of the tracing backend).
+
+See "docs/instrumentation.txt" for more information.
diff --git a/docs/instrumentation.txt b/docs/instrumentation.txt
new file mode 100644
index 0000000000..3d650ed09e
--- /dev/null
+++ b/docs/instrumentation.txt
@@ -0,0 +1,264 @@
+= Trace instrumentation =
+
+== Introduction ==
+
+Trace instrumentation allows users to execute their own code when QEMU raises
+one of its tracing events (see "docs/devel/tracing.txt"). This is more efficient
+than instrumenting events with the "dtrace" backend, since the user will run
+native instrumentation code and has more options to interact with the dynamic
+tracing and instrumentation facilities of QEMU.
+
+When applied to guest code events (i.e., those with the "guest_" prefix, like
+guest memory accesses), this turns QEMU into a fairly efficient and guest
+architecture-agnostic dynamic binary instrumentation framework. It works on all
+QEMU-supported architectures, as well as works in both 'user' (standalone
+application) and 'system' (full-system emulation) modes.
+
+Look at the headers installed by QEMU on the "qemu-instr" directory for further
+information beyond this document.
+
+
+== Selecting the events to instrument ==
+
+You must first select which events must be instrumentable before compiling QEMU
+by prefixing them with the "instrument" property, and removing the "disable"
+property if it is present.
+
+To get the full list of files defining events:
+
+    find /path/to/qemu-source -name trace-events
+
+To avoid modifying QEMU's sources, you can pass the "--with-instrument-events"
+argument to configure with one event name per line.
+
+
+== Instrumenting guest code ==
+
+QEMU emulates all guest instructions when executing in TCG mode (as opposed to
+using native hardware virtualization with KVM). Instructions are decompiled and
+translated into the intermediate TCG language. Then, the TCG compiler translates
+TCG code into the native host code that QEMU will execute.
+
+All events relating to guest code are named "guest_*". In addition, all events
+with the "tcg" property (see "docs/devel/tracing.txt") can be instrumented at
+two levels:
+
+* Translation
+
+  Raised when generating TCG code (e.g., translate a memory access instruction
+  from the guest).
+
+  Note: This level only exists for events with the "tcg" property.
+
+* Execution
+
+  Raised when executing the native host code generated by the TCG compiler
+  (e.g., execute a memory access instruction from the guest).
+
+  Note: All events without the "tcg" property are raised at execution time
+        (e.g., CPU hotplug).
+
+Note: Events with the "tcg" property (e.g., 'guest_mem_before') are internally
+      translated into two events to differentiate the translation and execution
+      levels (e.g., 'guest_mem_before_trans' and 'guest_mem_before_exec').
+
+Note: All guest events have a "Mode" and "Target" line describing when they are
+      available (e.g., TCG, KVM, etc.).
+
+
+== Setting instrumentation callbacks ==
+
+Function qi_ctrl_event_set() in "qemu-instr/control.h" can be used to set the
+instrumentation callback on each event to a user-specified function. Header
+"qemu-instr/events.h" provides the event identifiers and some pre-defined
+callbacks:
+
+* QI_EVENT_${EVENT}
+
+  Event identifier, passed to functions in "qemu-instr/control.h".
+
+* qi_event_${event}_nop
+
+  Do nothing.
+
+* qi_event_${event}_trace
+
+  Trace the event using whatever tracing backend QEMU has been configured with.
+
+* qi_event_${event}_gen_exec
+
+  Generate TCG code to raise the corresponding event when the TCG-generated code
+  is executed. Otherwise, the event will not be instrumented at execution time,
+  resulting in zero-overhead when executing the guest code.
+
+  Only available for translation-time events.
+
+* qi_event_${event}_trace_and_gen_exec
+
+  Combines 'qi_event_${event}_trace' and 'qi_event_${event}_gen_exec' in a
+  single call.
+
+  Only available for translation-time events.
+
+
+== Loading an instrumentation library ==
+
+There are two ways two load an instrumentation library:
+
+* Using the command-line "-instr" argument.
+
+* Using the "instr-load" and "instr-unload" commands in the HMP and QMP
+  interfaces.
+
+
+=== Example ===
+
+1. Configure QEMU with the selected events to instrument:
+
+    # instrument guest_cpu_enter and guest_mem_before
+    cat >/tmp/my-events <<EOF
+    guest_cpu_enter
+    guest_mem_before
+    EOF
+    mkdir -p /path/to/qemu-build
+    cd /path/to/qemu-build
+    /path/to/qemu-source/configure \
+      --enable-trace-instrument \
+      --with-instrument-events=/tmp/my-events \
+      --prefix=/path/to/qemu-install
+
+2. Build and install QEMU:
+
+    make install
+
+3. Create the "Makefile" to build the instrumentation library:
+
+    mkdir -p /tmp/my-instrument
+    
+    cat > /tmp/my-instrument/Makefile <<EOF
+    QEMU_PATH=/tmp/qemu-install/
+    
+    CFLAGS += -g
+    CFLAGS += -O3
+    CFLAGS += -Werror -Wall
+    CFLAGS += -I$(QEMU_PATH)/include
+    
+    all: libtrace-instrument.la
+    
+    libtrace-instrument.la: instrument.lo
+            libtool --mode=link --tag=CC $(CC) -module -rpath /usr/local/lib -o $@ $^
+    
+    %.lo: %.c
+            libtool --mode=compile --tag=CC $(CC) $(CFLAGS) -c $^
+    
+    clean:
+            $(RM) -f *.o *.so *.lo
+            $(RM) -Rf .libs
+    EOF
+
+4. Write your instrumentation library:
+
+    cat > /tmp/my-instrument/instrument.c <<EOF
+    #include <stdio.h>
+    #include <assert.h>
+    
+    #include <qemu-instr/events.h>      /* get event declarations */
+    #include <qemu-instr/control.h>     /* manipulate events */
+    
+    /* as documented in QEMU's event description */
+    struct mem_info {
+        uint8_t size_shift : 2;
+        bool    sign_extend: 1;
+        uint8_t endianness : 1;
+        bool    store      : 1;
+    };
+    
+    /* the address for the memory access is not known at translation time */
+    void guest_mem_before_trans(QICPU *cpu, QITCGv_cpu tcg_cpu,
+                                QITCGv vaddr, uint8_t info)
+    {
+        struct mem_info *mi = (struct mem_info*)&info;
+        qi_event_guest_mem_before_trans_trace(cpu, tcg_cpu, vaddr, info);
+        if (mi->store) {
+            /* generate at execution time only for memory writes */
+            qi_event_guest_mem_before_trans_gen_exec(cpu, tcg_cpu, vaddr, info);
+        }
+    }
+    
+    /* called when QEMU executes a memory access */
+    void guest_mem_before_exec(QICPU *cpu, uint64_t vaddr, uint8_t info)
+    {
+        struct mem_info *mi = (struct mem_info*)&info;
+        if (mi->store) {
+            /* if called by TCG code, we'll only get writes (see above) */
+            qi_event_guest_mem_before_exec_trace(cpu, vaddr, info);
+        }
+    }
+    
+    /* called every time QEMU hotplugs a CPU */
+    void guest_cpu_enter(QICPU *cpu)
+    {
+        /* call the original tracing routine */
+        qi_event_guest_cpu_enter_trace(cpu);
+    
+        /* disable instrumentation and tracing after the first call */
+        static bool found = false;
+        if (found) {
+            QIEvent *ev = QI_EVENT_GUEST_CPU_ENTER;
+            qi_ctrl_event_set(ev, NULL);
+            qi_trace_event_set_state_dynamic(ev, false);
+        } else {
+            found = true;
+        }
+    }
+    
+    
+    /* mandatory initialization callback */
+    void qi_init(int argc, const char **argv)
+    {
+        int i;
+        printf("init!\n");
+        printf("    argc :: %d\n", argc);
+        for (i = 0; i < argc; i++) {
+            printf("            -> %s\n", argv[i]);
+        }
+
+        /* instrument and trace events */
+        QIEvent *ev;
+
+        ev = QI_EVENT_GUEST_CPU_ENTER;
+        qi_ctrl_event_set(ev, guest_cpu_enter);
+        qi_trace_event_set_state_dynamic(ev, true);
+    
+        ev = QI_EVENT_GUEST_MEM_BEFORE_TRANS;
+        qi_ctrl_event_set(ev, guest_mem_before_trans);
+        qi_trace_event_set_state_dynamic(ev, true);
+    
+        ev = QI_EVENT_GUEST_MEM_BEFORE_EXEC;
+        qi_ctrl_event_set(ev, guest_mem_before_exec);
+        qi_trace_event_set_state_dynamic(ev, true);
+    }
+    
+    /* mandatory finalization callback */
+    void qi_fini(void)
+    {
+        fprintf(stderr, "fini!\n");
+    
+        /* ensure all tracing is disabled */
+        qi_trace_event_set_state_dynamic(QI_EVENT_GUEST_CPU_ENTER, false);
+        qi_trace_event_set_state_dynamic(QI_EVENT_GUEST_MEM_BEFORE_TRANS, false);
+        qi_trace_event_set_state_dynamic(QI_EVENT_GUEST_MEM_BEFORE_EXEC, false);
+    
+        /* instrumentation callbacks are automatically reset by QEMU */
+    }
+    EOF
+
+5. Compile the instrumentation library:
+
+    make -C /tmp/my-instrument
+
+6. Start QEMU with the instrumentation library:
+
+    /tmp/qemu-install/bin/qemu-system-x86_64 \
+        -instr file=/tmp/my-dinstrument/.libs/libtrace-instrument.so, \
+               arg=foo,arg=bar
