[Qemu-devel] [PATCH v5 0/5] Coroutines for better asynchronous programming

[Qemu-devel] [PATCH v5 0/5] Coroutines for better asynchronous programming

[Stefan Hajnoczi]

QEMU is event-driven and suffers when blocking operations are performed because
VM execution may be stopped until the operation completes.  Therefore many
operations that could block are performed asynchronously and a callback is
invoked when the operation has completed.  This allows QEMU to continue
executing while the operation is pending.

The downside to callbacks is that they split up code into many smaller
functions, each of which is a single step in a state machine that quickly
becomes complex and hard to understand.  Callback functions also result in lots
of noise as variables are packed and unpacked into temporary structs that pass
state to the callback function.

This patch series introduces coroutines as a solution for writing asynchronous
code while still having a nice sequential control flow.  The semantics are
explained in the second patch.  The fourth patch adds automated tests.

A nice feature of coroutines is that it is relatively easy to take synchronous
code and lift it into a coroutine to make it asynchronous.  Work has been done
to move qcow2 request processing into coroutines and thereby make it
asynchronous (today qcow2 will perform synchronous metadata accesses).  This
qcow2 work is still ongoing and not quite ready for mainline yet.

Coroutines are also being used for virtfs (virtio-9p) so I have submitted this
patch now because virtfs patches that depend on coroutines are being published.

v5:
 * GThread-based implementation for platforms without makecontext(3) (Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>)
 * Switch to gtester test framework

v4:
 * Windows Fibers support (Paolo Bonzini <pbonzini@redhat.com>)
 * Return-after-setjmp() fix (Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>)
 * Re-entrancy for multi-threaded coroutines support
 * qemu-coroutine.h cleanup and documentation

v3:
 * Updated LGPL v2 license header to use web link
 * Removed atexit(3) pool freeing
 * Removed thread-local current/leader
 * Documented thread-safety limitation
 * Disabled trace events

v2:
 * Added ./check-coroutine --lifecycle-benchmark for performance measurement
 * Split pooling into a separate patch with performance justification
 * Set maximum pool size to prevent holding onto too many free coroutines
 * Added atexit(3) handler to free pool
 * Coding style cleanups

Aneesh Kumar K.V (1):
  coroutine: implement coroutines using gthread

Anthony Liguori (1):
  Add hard build dependency on glib

Kevin Wolf (1):
  coroutine: introduce coroutines

Stefan Hajnoczi (2):
  coroutine: add check-coroutine automated tests
  coroutine: add check-coroutine --benchmark-lifecycle

 .gitignore           |    1 +
 Makefile             |    5 +-
 Makefile.objs        |   13 +++
 Makefile.target      |    1 +
 configure            |   29 +++++++
 coroutine-gthread.c  |  131 ++++++++++++++++++++++++++++
 coroutine-ucontext.c |  229 ++++++++++++++++++++++++++++++++++++++++++++++++++
 coroutine-win32.c    |   92 ++++++++++++++++++++
 qemu-coroutine-int.h |   48 +++++++++++
 qemu-coroutine.c     |   75 ++++++++++++++++
 qemu-coroutine.h     |   95 +++++++++++++++++++++
 test-coroutine.c     |  192 ++++++++++++++++++++++++++++++++++++++++++
 trace-events         |    5 +
 13 files changed, 915 insertions(+), 1 deletions(-)
 create mode 100644 coroutine-gthread.c
 create mode 100644 coroutine-ucontext.c
 create mode 100644 coroutine-win32.c
 create mode 100644 qemu-coroutine-int.h
 create mode 100644 qemu-coroutine.c
 create mode 100644 qemu-coroutine.h
 create mode 100644 test-coroutine.c

-- 
1.7.5.3

[Qemu-devel] [PATCH v5 1/5] Add hard build dependency on glib

[Stefan Hajnoczi]

From: Anthony Liguori <aliguori@us.ibm.com>

GLib is an extremely common library that has a portable thread implementation
along with tons of other goodies.

GLib and GObject have a fantastic amount of infrastructure we can leverage in
QEMU including an object oriented programming infrastructure.

Short term, it has a very nice thread pool implementation that we could leverage
in something like virtio-9p.  It also has a test harness implementation that
this series will use.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
 Makefile        |    2 ++
 Makefile.objs   |    2 ++
 Makefile.target |    1 +
 configure       |   13 +++++++++++++
 4 files changed, 18 insertions(+), 0 deletions(-)

diff --git a/Makefile b/Makefile
index 096480b..306cd9b 100644
--- a/Makefile
+++ b/Makefile
@@ -106,6 +106,8 @@ audio/audio.o audio/fmodaudio.o: QEMU_CFLAGS += $(FMOD_CFLAGS)
 
 QEMU_CFLAGS+=$(CURL_CFLAGS)
 
+QEMU_CFLAGS+=$(GLIB_CFLAGS)
+
 ui/cocoa.o: ui/cocoa.m
 
 ui/sdl.o audio/sdlaudio.o ui/sdl_zoom.o baum.o: QEMU_CFLAGS += $(SDL_CFLAGS)
diff --git a/Makefile.objs b/Makefile.objs
index 52d8b23..a7807e8 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -368,3 +368,5 @@ vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
 
 vl.o: QEMU_CFLAGS+=$(SDL_CFLAGS)
 
+vl.o: QEMU_CFLAGS+=$(GLIB_CFLAGS)
+
diff --git a/Makefile.target b/Makefile.target
index b1a0f6d..0d900e2 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -202,6 +202,7 @@ QEMU_CFLAGS += $(VNC_TLS_CFLAGS)
 QEMU_CFLAGS += $(VNC_SASL_CFLAGS)
 QEMU_CFLAGS += $(VNC_JPEG_CFLAGS)
 QEMU_CFLAGS += $(VNC_PNG_CFLAGS)
+QEMU_CFLAGS += $(GLIB_CFLAGS)
 
 # xen backend driver support
 obj-i386-$(CONFIG_XEN) += xen_machine_pv.o xen_domainbuild.o
diff --git a/configure b/configure
index 663b557..3697eba 100755
--- a/configure
+++ b/configure
@@ -1767,6 +1767,18 @@ EOF
 fi
 
 ##########################################
+# glib support probe
+if $pkg_config --modversion gthread-2.0 gio-2.0 > /dev/null 2>&1 ; then
+    glib_cflags=`$pkg_config --cflags gthread-2.0 gio-2.0 2>/dev/null`
+    glib_libs=`$pkg_config --libs gthread-2.0 gio-2.0 2>/dev/null`
+    libs_softmmu="$glib_libs $libs_softmmu"
+    libs_tools="$glib_libs $libs_tools"
+else
+    echo "glib-2.0 required to compile QEMU"
+    exit 1
+fi
+
+##########################################
 # kvm probe
 if test "$kvm" != "no" ; then
     cat > $TMPC <<EOF
@@ -2923,6 +2935,7 @@ if test "$bluez" = "yes" ; then
   echo "CONFIG_BLUEZ=y" >> $config_host_mak
   echo "BLUEZ_CFLAGS=$bluez_cflags" >> $config_host_mak
 fi
+echo "GLIB_CFLAGS=$glib_cflags" >> $config_host_mak
 if test "$xen" = "yes" ; then
   echo "CONFIG_XEN=y" >> $config_host_mak
   echo "CONFIG_XEN_CTRL_INTERFACE_VERSION=$xen_ctrl_version" >> $config_host_mak
-- 
1.7.5.3

[Qemu-devel] [PATCH v5 2/5] coroutine: introduce coroutines

[Stefan Hajnoczi]

From: Kevin Wolf <kwolf@redhat.com>

Asynchronous code is becoming very complex.  At the same time
synchronous code is growing because it is convenient to write.
Sometimes duplicate code paths are even added, one synchronous and the
other asynchronous.  This patch introduces coroutines which allow code
that looks synchronous but is asynchronous under the covers.

A coroutine has its own stack and is therefore able to preserve state
across blocking operations, which traditionally require callback
functions and manual marshalling of parameters.

Creating and starting a coroutine is easy:

  coroutine = qemu_coroutine_create(my_coroutine);
  qemu_coroutine_enter(coroutine, my_data);

The coroutine then executes until it returns or yields:

  void coroutine_fn my_coroutine(void *opaque) {
      MyData *my_data = opaque;

      /* do some work */

      qemu_coroutine_yield();

      /* do some more work */
  }

Yielding switches control back to the caller of qemu_coroutine_enter().
This is typically used to switch back to the main thread's event loop
after issuing an asynchronous I/O request.  The request callback will
then invoke qemu_coroutine_enter() once more to switch back to the
coroutine.

Note that if coroutines are used only from threads which hold the global
mutex they will never execute concurrently.  This makes programming with
coroutines easier than with threads.  Race conditions cannot occur since
only one coroutine may be active at any time.  Other coroutines can only
run across yield.

This coroutines implementation is based on the gtk-vnc implementation
written by Anthony Liguori <anthony@codemonkey.ws> but it has been
significantly rewritten by Kevin Wolf <kwolf@redhat.com> to use
setjmp()/longjmp() instead of the more expensive swapcontext() and by
Paolo Bonzini <pbonzini@redhat.com> for Windows Fibers support.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
 Makefile.objs        |    7 ++
 coroutine-ucontext.c |  229 ++++++++++++++++++++++++++++++++++++++++++++++++++
 coroutine-win32.c    |   92 ++++++++++++++++++++
 qemu-coroutine-int.h |   48 +++++++++++
 qemu-coroutine.c     |   75 ++++++++++++++++
 qemu-coroutine.h     |   95 +++++++++++++++++++++
 trace-events         |    5 +
 7 files changed, 551 insertions(+), 0 deletions(-)
 create mode 100644 coroutine-ucontext.c
 create mode 100644 coroutine-win32.c
 create mode 100644 qemu-coroutine-int.h
 create mode 100644 qemu-coroutine.c
 create mode 100644 qemu-coroutine.h

diff --git a/Makefile.objs b/Makefile.objs
index a7807e8..1890944 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -11,6 +11,12 @@ oslib-obj-$(CONFIG_WIN32) += oslib-win32.o qemu-thread-win32.o
 oslib-obj-$(CONFIG_POSIX) += oslib-posix.o qemu-thread-posix.o
 
 #######################################################################
+# coroutines
+coroutine-obj-y = qemu-coroutine.o
+coroutine-obj-$(CONFIG_POSIX) += coroutine-ucontext.o
+coroutine-obj-$(CONFIG_WIN32) += coroutine-win32.o
+
+#######################################################################
 # block-obj-y is code used by both qemu system emulation and qemu-img
 
 block-obj-y = cutils.o cache-utils.o qemu-malloc.o qemu-option.o module.o async.o
@@ -69,6 +75,7 @@ common-obj-y += readline.o console.o cursor.o qemu-error.o
 common-obj-y += $(oslib-obj-y)
 common-obj-$(CONFIG_WIN32) += os-win32.o
 common-obj-$(CONFIG_POSIX) += os-posix.o
+common-obj-y += $(coroutine-obj-y)
 
 common-obj-y += tcg-runtime.o host-utils.o
 common-obj-y += irq.o ioport.o input.o
diff --git a/coroutine-ucontext.c b/coroutine-ucontext.c
new file mode 100644
index 0000000..bcea2bd
--- /dev/null
+++ b/coroutine-ucontext.c
@@ -0,0 +1,229 @@
+/*
+ * ucontext coroutine initialization code
+ *
+ * Copyright (C) 2006  Anthony Liguori <anthony@codemonkey.ws>
+ * Copyright (C) 2011  Kevin Wolf <kwolf@redhat.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.0 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+/* XXX Is there a nicer way to disable glibc's stack check for longjmp? */
+#ifdef _FORTIFY_SOURCE
+#undef _FORTIFY_SOURCE
+#endif
+#include <setjmp.h>
+#include <stdint.h>
+#include <pthread.h>
+#include <ucontext.h>
+#include "qemu-common.h"
+#include "qemu-coroutine-int.h"
+
+enum {
+    /* Maximum free pool size prevents holding too many freed coroutines */
+    POOL_MAX_SIZE = 64,
+};
+
+typedef struct {
+    Coroutine base;
+    void *stack;
+    jmp_buf env;
+} CoroutineUContext;
+
+/**
+ * Per-thread coroutine bookkeeping
+ */
+typedef struct {
+    /** Currently executing coroutine */
+    Coroutine *current;
+
+    /** Free list to speed up creation */
+    QLIST_HEAD(, Coroutine) pool;
+    unsigned int pool_size;
+
+    /** The default coroutine */
+    CoroutineUContext leader;
+} CoroutineThreadState;
+
+static pthread_key_t thread_state_key;
+
+/*
+ * va_args to makecontext() must be type 'int', so passing
+ * the pointer we need may require several int args. This
+ * union is a quick hack to let us do that
+ */
+union cc_arg {
+    void *p;
+    int i[2];
+};
+
+static CoroutineThreadState *coroutine_get_thread_state(void)
+{
+    CoroutineThreadState *s = pthread_getspecific(thread_state_key);
+
+    if (!s) {
+        s = qemu_mallocz(sizeof(*s));
+        s->current = &s->leader.base;
+        QLIST_INIT(&s->pool);
+        pthread_setspecific(thread_state_key, s);
+    }
+    return s;
+}
+
+static void qemu_coroutine_thread_cleanup(void *opaque)
+{
+    CoroutineThreadState *s = opaque;
+    Coroutine *co;
+    Coroutine *tmp;
+
+    QLIST_FOREACH_SAFE(co, &s->pool, pool_next, tmp) {
+        qemu_free(DO_UPCAST(CoroutineUContext, base, co)->stack);
+        qemu_free(co);
+    }
+    qemu_free(s);
+}
+
+static void __attribute__((constructor)) coroutine_init(void)
+{
+    int ret;
+
+    ret = pthread_key_create(&thread_state_key, qemu_coroutine_thread_cleanup);
+    if (ret != 0) {
+        fprintf(stderr, "unable to create leader key: %s\n", strerror(errno));
+        abort();
+    }
+}
+
+static void coroutine_trampoline(int i0, int i1)
+{
+    union cc_arg arg;
+    CoroutineUContext *self;
+    Coroutine *co;
+
+    arg.i[0] = i0;
+    arg.i[1] = i1;
+    self = arg.p;
+    co = &self->base;
+
+    /* Initialize longjmp environment and switch back the caller */
+    if (!setjmp(self->env)) {
+        longjmp(*(jmp_buf *)co->entry_arg, 1);
+    }
+
+    while (true) {
+        co->entry(co->entry_arg);
+        qemu_coroutine_switch(co, co->caller, COROUTINE_TERMINATE);
+    }
+}
+
+static Coroutine *coroutine_new(void)
+{
+    const size_t stack_size = 4 << 20;
+    CoroutineUContext *co;
+    ucontext_t old_uc, uc;
+    jmp_buf old_env;
+    union cc_arg arg;
+
+    /* The ucontext functions preserve signal masks which incurs a system call
+     * overhead.  setjmp()/longjmp() does not preserve signal masks but only
+     * works on the current stack.  Since we need a way to create and switch to
+     * a new stack, use the ucontext functions for that but setjmp()/longjmp()
+     * for everything else.
+     */
+
+    if (getcontext(&uc) == -1) {
+        return NULL;
+    }
+
+    co = qemu_mallocz(sizeof(*co));
+    co->stack = qemu_malloc(stack_size);
+    co->base.entry_arg = &old_env; /* stash away our jmp_buf */
+
+    uc.uc_link = &old_uc;
+    uc.uc_stack.ss_sp = co->stack;
+    uc.uc_stack.ss_size = stack_size;
+    uc.uc_stack.ss_flags = 0;
+
+    arg.p = co;
+
+    makecontext(&uc, (void (*)(void))coroutine_trampoline,
+                2, arg.i[0], arg.i[1]);
+
+    /* swapcontext() in, longjmp() back out */
+    if (!setjmp(old_env)) {
+        swapcontext(&old_uc, &uc);
+    }
+    return &co->base;
+}
+
+Coroutine *qemu_coroutine_new(void)
+{
+    CoroutineThreadState *s = coroutine_get_thread_state();
+    Coroutine *co;
+
+    co = QLIST_FIRST(&s->pool);
+    if (co) {
+        QLIST_REMOVE(co, pool_next);
+        s->pool_size--;
+    } else {
+        co = coroutine_new();
+    }
+    return co;
+}
+
+void qemu_coroutine_delete(Coroutine *co_)
+{
+    CoroutineThreadState *s = coroutine_get_thread_state();
+    CoroutineUContext *co = DO_UPCAST(CoroutineUContext, base, co_);
+
+    if (s->pool_size < POOL_MAX_SIZE) {
+        QLIST_INSERT_HEAD(&s->pool, &co->base, pool_next);
+        co->base.caller = NULL;
+        s->pool_size++;
+        return;
+    }
+
+    qemu_free(co->stack);
+    qemu_free(co);
+}
+
+CoroutineAction qemu_coroutine_switch(Coroutine *from_, Coroutine *to_,
+                                      CoroutineAction action)
+{
+    CoroutineUContext *from = DO_UPCAST(CoroutineUContext, base, from_);
+    CoroutineUContext *to = DO_UPCAST(CoroutineUContext, base, to_);
+    CoroutineThreadState *s = coroutine_get_thread_state();
+    int ret;
+
+    s->current = to_;
+
+    ret = setjmp(from->env);
+    if (ret == 0) {
+        longjmp(to->env, action);
+    }
+    return ret;
+}
+
+Coroutine *qemu_coroutine_self(void)
+{
+    CoroutineThreadState *s = coroutine_get_thread_state();
+
+    return s->current;
+}
+
+bool qemu_in_coroutine(void)
+{
+    CoroutineThreadState *s = pthread_getspecific(thread_state_key);
+
+    return s && s->current->caller;
+}
diff --git a/coroutine-win32.c b/coroutine-win32.c
new file mode 100644
index 0000000..2215ae5
--- /dev/null
+++ b/coroutine-win32.c
@@ -0,0 +1,92 @@
+/*
+ * Win32 coroutine initialization code
+ *
+ * Copyright (c) 2011 Kevin Wolf <kwolf@redhat.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu-common.h"
+#include "qemu-coroutine-int.h"
+
+typedef struct
+{
+    Coroutine base;
+
+    LPVOID fiber;
+    CoroutineAction action;
+} CoroutineWin32;
+
+static __thread CoroutineWin32 leader;
+static __thread Coroutine *current;
+
+CoroutineAction qemu_coroutine_switch(Coroutine *from_, Coroutine *to_,
+                                      CoroutineAction action)
+{
+    CoroutineWin32 *from = DO_UPCAST(CoroutineWin32, base, from_);
+    CoroutineWin32 *to = DO_UPCAST(CoroutineWin32, base, to_);
+
+    current = to_;
+
+    to->action = action;
+    SwitchToFiber(to->fiber);
+    return from->action;
+}
+
+static void CALLBACK coroutine_trampoline(void *co_)
+{
+    Coroutine *co = co_;
+
+    while (true) {
+        co->entry(co->entry_arg);
+        qemu_coroutine_switch(co, co->caller, COROUTINE_TERMINATE);
+    }
+}
+
+Coroutine *qemu_coroutine_new(void)
+{
+    const size_t stack_size = 4 << 20;
+    CoroutineWin32 *co;
+
+    co = qemu_mallocz(sizeof(*co));
+    co->fiber = CreateFiber(stack_size, coroutine_trampoline, &co->base);
+    return &co->base;
+}
+
+void qemu_coroutine_delete(Coroutine *co_)
+{
+    CoroutineWin32 *co = DO_UPCAST(CoroutineWin32, base, co_);
+
+    DeleteFiber(co->fiber);
+    qemu_free(co);
+}
+
+Coroutine *qemu_coroutine_self(void)
+{
+    if (!current) {
+        current = &leader.base;
+        leader.fiber = ConvertThreadToFiber(NULL);
+    }
+    return current;
+}
+
+bool qemu_in_coroutine(void)
+{
+    return current && current->caller;
+}
diff --git a/qemu-coroutine-int.h b/qemu-coroutine-int.h
new file mode 100644
index 0000000..64915c2
--- /dev/null
+++ b/qemu-coroutine-int.h
@@ -0,0 +1,48 @@
+/*
+ * Coroutine internals
+ *
+ * Copyright (c) 2011 Kevin Wolf <kwolf@redhat.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#ifndef QEMU_COROUTINE_INT_H
+#define QEMU_COROUTINE_INT_H
+
+#include "qemu-queue.h"
+#include "qemu-coroutine.h"
+
+typedef enum {
+    COROUTINE_YIELD = 1,
+    COROUTINE_TERMINATE = 2,
+} CoroutineAction;
+
+struct Coroutine {
+    CoroutineEntry *entry;
+    void *entry_arg;
+    Coroutine *caller;
+    QLIST_ENTRY(Coroutine) pool_next;
+};
+
+Coroutine *qemu_coroutine_new(void);
+void qemu_coroutine_delete(Coroutine *co);
+CoroutineAction qemu_coroutine_switch(Coroutine *from, Coroutine *to,
+                                      CoroutineAction action);
+
+#endif
diff --git a/qemu-coroutine.c b/qemu-coroutine.c
new file mode 100644
index 0000000..600be26
--- /dev/null
+++ b/qemu-coroutine.c
@@ -0,0 +1,75 @@
+/*
+ * QEMU coroutines
+ *
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ *  Stefan Hajnoczi    <stefanha@linux.vnet.ibm.com>
+ *  Kevin Wolf         <kwolf@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include "trace.h"
+#include "qemu-common.h"
+#include "qemu-coroutine.h"
+#include "qemu-coroutine-int.h"
+
+Coroutine *qemu_coroutine_create(CoroutineEntry *entry)
+{
+    Coroutine *co = qemu_coroutine_new();
+    co->entry = entry;
+    return co;
+}
+
+static void coroutine_swap(Coroutine *from, Coroutine *to)
+{
+    CoroutineAction ret;
+
+    ret = qemu_coroutine_switch(from, to, COROUTINE_YIELD);
+
+    switch (ret) {
+    case COROUTINE_YIELD:
+        return;
+    case COROUTINE_TERMINATE:
+        trace_qemu_coroutine_terminate(to);
+        qemu_coroutine_delete(to);
+        return;
+    default:
+        abort();
+    }
+}
+
+void qemu_coroutine_enter(Coroutine *co, void *opaque)
+{
+    Coroutine *self = qemu_coroutine_self();
+
+    trace_qemu_coroutine_enter(self, co, opaque);
+
+    if (co->caller) {
+        fprintf(stderr, "Co-routine re-entered recursively\n");
+        abort();
+    }
+
+    co->caller = self;
+    co->entry_arg = opaque;
+    coroutine_swap(self, co);
+}
+
+void coroutine_fn qemu_coroutine_yield(void)
+{
+    Coroutine *self = qemu_coroutine_self();
+    Coroutine *to = self->caller;
+
+    trace_qemu_coroutine_yield(self, to);
+
+    if (!to) {
+        fprintf(stderr, "Co-routine is yielding to no one\n");
+        abort();
+    }
+
+    self->caller = NULL;
+    coroutine_swap(self, to);
+}
diff --git a/qemu-coroutine.h b/qemu-coroutine.h
new file mode 100644
index 0000000..08255c7
--- /dev/null
+++ b/qemu-coroutine.h
@@ -0,0 +1,95 @@
+/*
+ * QEMU coroutine implementation
+ *
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ *  Stefan Hajnoczi    <stefanha@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef QEMU_COROUTINE_H
+#define QEMU_COROUTINE_H
+
+#include <stdbool.h>
+
+/**
+ * Coroutines are a mechanism for stack switching and can be used for
+ * cooperative userspace threading.  These functions provide a simple but
+ * useful flavor of coroutines that is suitable for writing sequential code,
+ * rather than callbacks, for operations that need to give up control while
+ * waiting for events to complete.
+ *
+ * These functions are re-entrant and may be used outside the global mutex.
+ */
+
+/**
+ * Mark a function that executes in coroutine context
+ *
+ * Functions that execute in coroutine context cannot be called directly from
+ * normal functions.  In the future it would be nice to enable compiler or
+ * static checker support for catching such errors.  This annotation might make
+ * it possible and in the meantime it serves as documentation.
+ *
+ * For example:
+ *
+ *   static void coroutine_fn foo(void) {
+ *       ....
+ *   }
+ */
+#define coroutine_fn
+
+typedef struct Coroutine Coroutine;
+
+/**
+ * Coroutine entry point
+ *
+ * When the coroutine is entered for the first time, opaque is passed in as an
+ * argument.
+ *
+ * When this function returns, the coroutine is destroyed automatically and
+ * execution continues in the caller who last entered the coroutine.
+ */
+typedef void coroutine_fn CoroutineEntry(void *opaque);
+
+/**
+ * Create a new coroutine
+ *
+ * Use qemu_coroutine_enter() to actually transfer control to the coroutine.
+ */
+Coroutine *qemu_coroutine_create(CoroutineEntry *entry);
+
+/**
+ * Transfer control to a coroutine
+ *
+ * The opaque argument is passed as the argument to the entry point when
+ * entering the coroutine for the first time.  It is subsequently ignored.
+ */
+void qemu_coroutine_enter(Coroutine *coroutine, void *opaque);
+
+/**
+ * Transfer control back to a coroutine's caller
+ *
+ * This function does not return until the coroutine is re-entered using
+ * qemu_coroutine_enter().
+ */
+void coroutine_fn qemu_coroutine_yield(void);
+
+/**
+ * Get the currently executing coroutine
+ */
+Coroutine *coroutine_fn qemu_coroutine_self(void);
+
+/**
+ * Return whether or not currently inside a coroutine
+ *
+ * This can be used to write functions that work both when in coroutine context
+ * and when not in coroutine context.  Note that such functions cannot use the
+ * coroutine_fn annotation since they work outside coroutine context.
+ */
+bool qemu_in_coroutine(void);
+
+#endif /* QEMU_COROUTINE_H */
diff --git a/trace-events b/trace-events
index e0e9574..e0a4d83 100644
--- a/trace-events
+++ b/trace-events
@@ -386,3 +386,8 @@ disable xen_unmap_block(void* addr, unsigned long size) "%p, size %#lx"
 
 # exec.c
 disable qemu_put_ram_ptr(void* addr) "%p"
+
+# qemu-coroutine.c
+disable qemu_coroutine_enter(void *from, void *to, void *opaque) "from %p to %p opaque %p"
+disable qemu_coroutine_yield(void *from, void *to) "from %p to %p"
+disable qemu_coroutine_terminate(void *co) "self %p"
-- 
1.7.5.3

[Qemu-devel] [PATCH v5 3/5] coroutine: implement coroutines using gthread

[Stefan Hajnoczi]

From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>

On platforms that don't support makecontext(3) use gthread based
coroutine implementation.

[Original patch by Aneesh, made consistent with coroutine-ucontext.c and
switched to GStaticPrivate by Stefan.  Tested on Linux and OpenBSD.]

Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
 Makefile.objs       |    4 ++
 configure           |   16 ++++++
 coroutine-gthread.c |  131 +++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 151 insertions(+), 0 deletions(-)
 create mode 100644 coroutine-gthread.c

diff --git a/Makefile.objs b/Makefile.objs
index 1890944..8333f5a 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -13,7 +13,11 @@ oslib-obj-$(CONFIG_POSIX) += oslib-posix.o qemu-thread-posix.o
 #######################################################################
 # coroutines
 coroutine-obj-y = qemu-coroutine.o
+ifeq ($(CONFIG_UCONTEXT_COROUTINE),y)
 coroutine-obj-$(CONFIG_POSIX) += coroutine-ucontext.o
+else
+coroutine-obj-$(CONFIG_POSIX) += coroutine-gthread.o
+endif
 coroutine-obj-$(CONFIG_WIN32) += coroutine-win32.o
 
 #######################################################################
diff --git a/configure b/configure
index 3697eba..df63403 100755
--- a/configure
+++ b/configure
@@ -2552,6 +2552,18 @@ if test "$trace_backend" = "dtrace"; then
 fi
 
 ##########################################
+# check if we have makecontext
+
+ucontext_coroutine=no
+cat > $TMPC << EOF
+#include <ucontext.h>
+int main(void) { makecontext(0, 0, 0); }
+EOF
+if compile_prog "" "" ; then
+    ucontext_coroutine=yes
+fi
+
+##########################################
 # End of CC checks
 # After here, no more $cc or $ld runs
 
@@ -3015,6 +3027,10 @@ if test "$rbd" = "yes" ; then
   echo "CONFIG_RBD=y" >> $config_host_mak
 fi
 
+if test "$ucontext_coroutine" = "yes" ; then
+  echo "CONFIG_UCONTEXT_COROUTINE=y" >> $config_host_mak
+fi
+
 # USB host support
 case "$usb" in
 linux)
diff --git a/coroutine-gthread.c b/coroutine-gthread.c
new file mode 100644
index 0000000..f09877e
--- /dev/null
+++ b/coroutine-gthread.c
@@ -0,0 +1,131 @@
+/*
+ * GThread coroutine initialization code
+ *
+ * Copyright (C) 2006  Anthony Liguori <anthony@codemonkey.ws>
+ * Copyright (C) 2011  Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.0 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <glib.h>
+#include "qemu-common.h"
+#include "qemu-coroutine-int.h"
+
+typedef struct {
+    Coroutine base;
+    GThread *thread;
+    bool runnable;
+    CoroutineAction action;
+} CoroutineGThread;
+
+static GCond *coroutine_cond;
+static GStaticMutex coroutine_lock = G_STATIC_MUTEX_INIT;
+static GStaticPrivate coroutine_key = G_STATIC_PRIVATE_INIT;
+
+static void __attribute__((constructor)) coroutine_init(void)
+{
+    if (!g_thread_supported()) {
+        g_thread_init(NULL);
+    }
+
+    coroutine_cond = g_cond_new();
+}
+
+static void coroutine_wait_runnable_locked(CoroutineGThread *co)
+{
+    while (!co->runnable) {
+        g_cond_wait(coroutine_cond, g_static_mutex_get_mutex(&coroutine_lock));
+    }
+}
+
+static void coroutine_wait_runnable(CoroutineGThread *co)
+{
+    g_static_mutex_lock(&coroutine_lock);
+    coroutine_wait_runnable_locked(co);
+    g_static_mutex_unlock(&coroutine_lock);
+}
+
+static gpointer coroutine_thread(gpointer opaque)
+{
+    CoroutineGThread *co = opaque;
+
+    g_static_private_set(&coroutine_key, co, NULL);
+    coroutine_wait_runnable(co);
+    co->base.entry(co->base.entry_arg);
+    qemu_coroutine_switch(&co->base, co->base.caller, COROUTINE_TERMINATE);
+    return NULL;
+}
+
+Coroutine *qemu_coroutine_new(void)
+{
+    CoroutineGThread *co;
+
+    co = qemu_mallocz(sizeof(*co));
+    co->thread = g_thread_create_full(coroutine_thread, co, 0, TRUE, TRUE,
+                                      G_THREAD_PRIORITY_NORMAL, NULL);
+    if (!co->thread) {
+        qemu_free(co);
+        return NULL;
+    }
+    return &co->base;
+}
+
+void qemu_coroutine_delete(Coroutine *co_)
+{
+    CoroutineGThread *co = DO_UPCAST(CoroutineGThread, base, co_);
+
+    g_thread_join(co->thread);
+    qemu_free(co);
+}
+
+CoroutineAction qemu_coroutine_switch(Coroutine *from_,
+                                      Coroutine *to_,
+                                      CoroutineAction action)
+{
+    CoroutineGThread *from = DO_UPCAST(CoroutineGThread, base, from_);
+    CoroutineGThread *to = DO_UPCAST(CoroutineGThread, base, to_);
+
+    g_static_mutex_lock(&coroutine_lock);
+    from->runnable = false;
+    from->action = action;
+    to->runnable = true;
+    to->action = action;
+    g_cond_broadcast(coroutine_cond);
+
+    if (action != COROUTINE_TERMINATE) {
+        coroutine_wait_runnable_locked(from);
+    }
+    g_static_mutex_unlock(&coroutine_lock);
+    return from->action;
+}
+
+Coroutine *qemu_coroutine_self(void)
+{
+    CoroutineGThread *co = g_static_private_get(&coroutine_key);
+
+    if (!co) {
+        co = qemu_mallocz(sizeof(*co));
+        co->runnable = true;
+        g_static_private_set(&coroutine_key, co, (GDestroyNotify)qemu_free);
+    }
+
+    return &co->base;
+}
+
+bool qemu_in_coroutine(void)
+{
+    CoroutineGThread *co = g_static_private_get(&coroutine_key);
+
+    return co && co->base.caller;
+}
-- 
1.7.5.3

[Qemu-devel] [PATCH v5 4/5] coroutine: add check-coroutine automated tests

[Stefan Hajnoczi]

To run automated tests for coroutines:

  make check-coroutine
  ./check-coroutine

On success the program terminates with exit status 0.  On failure an
error message is written to stderr and the program exits with exit
status 1.

Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
 .gitignore       |    1 +
 Makefile         |    3 +-
 test-coroutine.c |  162 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 165 insertions(+), 1 deletions(-)
 create mode 100644 test-coroutine.c

diff --git a/.gitignore b/.gitignore
index 08013fc..3ad334a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -34,6 +34,7 @@ qemu-img-cmds.h
 qemu-io
 qemu-monitor.texi
 QMP/qmp-commands.txt
+test-coroutine
 .gdbinit
 *.a
 *.aux
diff --git a/Makefile b/Makefile
index 306cd9b..6b5899a 100644
--- a/Makefile
+++ b/Makefile
@@ -134,7 +134,7 @@ qemu-io$(EXESUF): qemu-io.o cmd.o qemu-tool.o qemu-error.o $(oslib-obj-y) $(trac
 qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.hx
 	$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -h < $< > $@,"  GEN   $@")
 
-check-qint.o check-qstring.o check-qdict.o check-qlist.o check-qfloat.o check-qjson.o: $(GENERATED_HEADERS)
+check-qint.o check-qstring.o check-qdict.o check-qlist.o check-qfloat.o check-qjson.o test-coroutine.o: $(GENERATED_HEADERS)
 
 CHECK_PROG_DEPS = qemu-malloc.o $(oslib-obj-y) $(trace-obj-y) qemu-tool.o
 
@@ -144,6 +144,7 @@ check-qdict: check-qdict.o qdict.o qfloat.o qint.o qstring.o qbool.o qlist.o $(C
 check-qlist: check-qlist.o qlist.o qint.o $(CHECK_PROG_DEPS)
 check-qfloat: check-qfloat.o qfloat.o $(CHECK_PROG_DEPS)
 check-qjson: check-qjson.o qfloat.o qint.o qdict.o qstring.o qlist.o qbool.o qjson.o json-streamer.o json-lexer.o json-parser.o error.o qerror.o qemu-error.o $(CHECK_PROG_DEPS)
+test-coroutine: test-coroutine.o $(coroutine-obj-y) $(CHECK_PROG_DEPS)
 
 QEMULIBS=libhw32 libhw64 libuser libdis libdis-user
 
diff --git a/test-coroutine.c b/test-coroutine.c
new file mode 100644
index 0000000..9e9d3c9
--- /dev/null
+++ b/test-coroutine.c
@@ -0,0 +1,162 @@
+/*
+ * Coroutine tests
+ *
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ *  Stefan Hajnoczi    <stefanha@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include <glib.h>
+#include "qemu-coroutine.h"
+
+/*
+ * Check that qemu_in_coroutine() works
+ */
+
+static void coroutine_fn verify_in_coroutine(void *opaque)
+{
+    g_assert(qemu_in_coroutine());
+}
+
+static void test_in_coroutine(void)
+{
+    Coroutine *coroutine;
+
+    g_assert(!qemu_in_coroutine());
+
+    coroutine = qemu_coroutine_create(verify_in_coroutine);
+    qemu_coroutine_enter(coroutine, NULL);
+}
+
+/*
+ * Check that qemu_coroutine_self() works
+ */
+
+static void coroutine_fn verify_self(void *opaque)
+{
+    g_assert(qemu_coroutine_self() == opaque);
+}
+
+static void test_self(void)
+{
+    Coroutine *coroutine;
+
+    coroutine = qemu_coroutine_create(verify_self);
+    qemu_coroutine_enter(coroutine, coroutine);
+}
+
+/*
+ * Check that coroutines may nest multiple levels
+ */
+
+typedef struct {
+    unsigned int n_enter;   /* num coroutines entered */
+    unsigned int n_return;  /* num coroutines returned */
+    unsigned int max;       /* maximum level of nesting */
+} NestData;
+
+static void coroutine_fn nest(void *opaque)
+{
+    NestData *nd = opaque;
+
+    nd->n_enter++;
+
+    if (nd->n_enter < nd->max) {
+        Coroutine *child;
+
+        child = qemu_coroutine_create(nest);
+        qemu_coroutine_enter(child, nd);
+    }
+
+    nd->n_return++;
+}
+
+static void test_nesting(void)
+{
+    Coroutine *root;
+    NestData nd = {
+        .n_enter  = 0,
+        .n_return = 0,
+        .max      = 128,
+    };
+
+    root = qemu_coroutine_create(nest);
+    qemu_coroutine_enter(root, &nd);
+
+    /* Must enter and return from max nesting level */
+    g_assert_cmpint(nd.n_enter, ==, nd.max);
+    g_assert_cmpint(nd.n_return, ==, nd.max);
+}
+
+/*
+ * Check that yield/enter transfer control correctly
+ */
+
+static void coroutine_fn yield_5_times(void *opaque)
+{
+    bool *done = opaque;
+    int i;
+
+    for (i = 0; i < 5; i++) {
+        qemu_coroutine_yield();
+    }
+    *done = true;
+}
+
+static void test_yield(void)
+{
+    Coroutine *coroutine;
+    bool done = false;
+    int i = -1; /* one extra time to return from coroutine */
+
+    coroutine = qemu_coroutine_create(yield_5_times);
+    while (!done) {
+        qemu_coroutine_enter(coroutine, &done);
+        i++;
+    }
+    g_assert_cmpint(i, ==, 5); /* coroutine must yield 5 times */
+}
+
+/*
+ * Check that creation, enter, and return work
+ */
+
+static void coroutine_fn set_and_exit(void *opaque)
+{
+    bool *done = opaque;
+
+    *done = true;
+}
+
+static void test_lifecycle(void)
+{
+    Coroutine *coroutine;
+    bool done = false;
+
+    /* Create, enter, and return from coroutine */
+    coroutine = qemu_coroutine_create(set_and_exit);
+    qemu_coroutine_enter(coroutine, &done);
+    g_assert(done); /* expect done to be true (first time) */
+
+    /* Repeat to check that no state affects this test */
+    done = false;
+    coroutine = qemu_coroutine_create(set_and_exit);
+    qemu_coroutine_enter(coroutine, &done);
+    g_assert(done); /* expect done to be true (second time) */
+}
+
+int main(int argc, char **argv)
+{
+    g_test_init(&argc, &argv, NULL);
+    g_test_add_func("/basic/lifecycle", test_lifecycle);
+    g_test_add_func("/basic/yield", test_yield);
+    g_test_add_func("/basic/nesting", test_nesting);
+    g_test_add_func("/basic/self", test_self);
+    g_test_add_func("/basic/in_coroutine", test_in_coroutine);
+    return g_test_run();
+}
-- 
1.7.5.3

[Qemu-devel] [PATCH v5 5/5] coroutine: add check-coroutine --benchmark-lifecycle

[Stefan Hajnoczi]

Add a microbenchmark for coroutine create, enter, and return (aka
lifecycle).  This is a useful benchmark because users are expected to
create many coroutines, one per I/O request for example, and we
therefore need to provide good performance in that scenario.

To run:

  make check-coroutine
  ./check-coroutine --benchmark-lifecycle 20000000

This will do 20,000,000 coroutine create, enter, return iterations and
print the resulting time.

Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
---
 test-coroutine.c |   30 ++++++++++++++++++++++++++++++
 1 files changed, 30 insertions(+), 0 deletions(-)

diff --git a/test-coroutine.c b/test-coroutine.c
index 9e9d3c9..bf9f3e9 100644
--- a/test-coroutine.c
+++ b/test-coroutine.c
@@ -150,6 +150,33 @@ static void test_lifecycle(void)
     g_assert(done); /* expect done to be true (second time) */
 }
 
+/*
+ * Lifecycle benchmark
+ */
+
+static void coroutine_fn empty_coroutine(void *opaque)
+{
+    /* Do nothing */
+}
+
+static void perf_lifecycle(void)
+{
+    Coroutine *coroutine;
+    unsigned int i, max;
+    double duration;
+
+    max = 1000000;
+
+    g_test_timer_start();
+    for (i = 0; i < max; i++) {
+        coroutine = qemu_coroutine_create(empty_coroutine);
+        qemu_coroutine_enter(coroutine, NULL);
+    }
+    duration = g_test_timer_elapsed();
+
+    g_test_message("Lifecycle %u iterations: %f s\n", max, duration);
+}
+
 int main(int argc, char **argv)
 {
     g_test_init(&argc, &argv, NULL);
@@ -158,5 +185,8 @@ int main(int argc, char **argv)
     g_test_add_func("/basic/nesting", test_nesting);
     g_test_add_func("/basic/self", test_self);
     g_test_add_func("/basic/in_coroutine", test_in_coroutine);
+    if (g_test_perf()) {
+        g_test_add_func("/perf/lifecycle", perf_lifecycle);
+    }
     return g_test_run();
 }
-- 
1.7.5.3

Re: [Qemu-devel] [PATCH v5 1/5] Add hard build dependency on glib

[Andreas Färber]

Am 12.06.2011 um 22:46 schrieb Stefan Hajnoczi:

> From: Anthony Liguori <aliguori@us.ibm.com>
>
> GLib is an extremely common library that has a portable thread  
> implementation
> along with tons of other goodies.
>
> GLib and GObject have a fantastic amount of infrastructure we can  
> leverage in
> QEMU including an object oriented programming infrastructure.

GObject still doesn't build on Darwin/ppc64:

   CC     gboxed.lo
../../glib-2.28.8/gobject/gboxed.c: In function 'g_closure_get_type':
../../glib-2.28.8/gobject/gboxed.c:120: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:120: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:120: error: incompatible type for  
argument 2 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c:120: error: incompatible type for  
argument 3 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c: In function 'g_value_get_type':
../../glib-2.28.8/gobject/gboxed.c:121: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:121: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:121: error: incompatible type for  
argument 2 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c:121: error: incompatible type for  
argument 3 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c: In function  
'g_value_array_get_type':
../../glib-2.28.8/gobject/gboxed.c:122: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:122: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:122: error: incompatible type for  
argument 2 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c:122: error: incompatible type for  
argument 3 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c: In function 'g_date_get_type':
../../glib-2.28.8/gobject/gboxed.c:123: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:123: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:123: error: incompatible type for  
argument 2 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c:123: error: incompatible type for  
argument 3 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c: In function 'g_gstring_get_type':
../../glib-2.28.8/gobject/gboxed.c:125: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:125: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:125: error: incompatible type for  
argument 2 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c:125: error: incompatible type for  
argument 3 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c: In function 'g_hash_table_get_type':
../../glib-2.28.8/gobject/gboxed.c:126: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:126: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:126: error: incompatible type for  
argument 2 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c:126: error: incompatible type for  
argument 3 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c: In function 'g_array_get_type':
../../glib-2.28.8/gobject/gboxed.c:127: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:127: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:127: error: incompatible type for  
argument 2 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c:127: error: incompatible type for  
argument 3 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c: In function 'g_ptr_array_get_type':
../../glib-2.28.8/gobject/gboxed.c:128: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:128: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:128: error: incompatible type for  
argument 2 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c:128: error: incompatible type for  
argument 3 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c: In function 'g_byte_array_get_type':
../../glib-2.28.8/gobject/gboxed.c:129: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:129: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:129: error: incompatible type for  
argument 2 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c:129: error: incompatible type for  
argument 3 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c: In function 'g_regex_get_type':
../../glib-2.28.8/gobject/gboxed.c:132: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:132: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:132: error: incompatible type for  
argument 2 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c:132: error: incompatible type for  
argument 3 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c: In function  
'g_variant_type_get_gtype':
../../glib-2.28.8/gobject/gboxed.c:138: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:138: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:138: error: incompatible type for  
argument 2 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c:138: error: incompatible type for  
argument 3 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c: In function 'g_error_get_type':
../../glib-2.28.8/gobject/gboxed.c:141: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:141: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:141: error: incompatible type for  
argument 2 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c:141: error: incompatible type for  
argument 3 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c: In function 'g_date_time_get_type':
../../glib-2.28.8/gobject/gboxed.c:143: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:143: warning: union cannot be made  
transparent
../../glib-2.28.8/gobject/gboxed.c:143: error: incompatible type for  
argument 2 of '_g_register_boxed'
../../glib-2.28.8/gobject/gboxed.c:143: error: incompatible type for  
argument 3 of '_g_register_boxed'
make[4]: *** [gboxed.lo] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

With both gcc 4.0.1 and 4.2.1, using CC="gcc -arch ppc64" CPPFLAGS="- 
arch ppc64" and supplying GNU gettext and GNU libiconv (since it  
chokes on the system iconv.h). No Google hit.

Any thoughts?

Andreas

Re: [Qemu-devel] [PATCH v5 1/5] Add hard build dependency on glib

[Stefan Hajnoczi]

On Sat, Jun 18, 2011 at 5:15 PM, Andreas Färber <andreas.faerber@web.de> wrote:
> With both gcc 4.0.1 and 4.2.1, using CC="gcc -arch ppc64" CPPFLAGS="-arch
> ppc64" and supplying GNU gettext and GNU libiconv (since it chokes on the
> system iconv.h). No Google hit.

Strange, Fink seems to support it:
http://pdb.finkproject.org/pdb/package.php/glib2-dev

Stefan

Re: [Qemu-devel] [PATCH v5 1/5] Add hard build dependency on glib

[Andreas Färber]

Am 18.06.2011 um 19:21 schrieb Stefan Hajnoczi:

> On Sat, Jun 18, 2011 at 5:15 PM, Andreas Färber <andreas.faerber@web.de 
> > wrote:
>> With both gcc 4.0.1 and 4.2.1, using CC="gcc -arch ppc64"  
>> CPPFLAGS="-arch
>> ppc64" and supplying GNU gettext and GNU libiconv (since it chokes  
>> on the
>> system iconv.h). No Google hit.
>
> Strange, Fink seems to support it:
> http://pdb.finkproject.org/pdb/package.php/glib2-dev

Says nothing about powerpc64...

Andreas

Re: [Qemu-devel] [PATCH v5 1/5] Add hard build dependency on glib

[Stefan Hajnoczi]

On Sat, Jun 18, 2011 at 8:44 PM, Andreas Färber <andreas.faerber@web.de> wrote:
> Am 18.06.2011 um 19:21 schrieb Stefan Hajnoczi:
>
>> On Sat, Jun 18, 2011 at 5:15 PM, Andreas Färber <andreas.faerber@web.de>
>> wrote:
>>>
>>> With both gcc 4.0.1 and 4.2.1, using CC="gcc -arch ppc64" CPPFLAGS="-arch
>>> ppc64" and supplying GNU gettext and GNU libiconv (since it chokes on the
>>> system iconv.h). No Google hit.
>>
>> Strange, Fink seems to support it:
>> http://pdb.finkproject.org/pdb/package.php/glib2-dev
>
> Says nothing about powerpc64...

Shows my ignorance of Mac or Darwin :).  I figured 10.5/powerpc is
what you're after.

Stefan

Re: [Qemu-devel] [PATCH v5 1/5] Add hard build dependency on glib

[Andreas Färber]

Am 18.06.2011 um 22:46 schrieb Stefan Hajnoczi:

> On Sat, Jun 18, 2011 at 8:44 PM, Andreas Färber <andreas.faerber@web.de 
> > wrote:
>> Am 18.06.2011 um 19:21 schrieb Stefan Hajnoczi:
>>
>>> On Sat, Jun 18, 2011 at 5:15 PM, Andreas Färber <andreas.faerber@web.de 
>>> >
>>> wrote:
>>>>
>>>> With both gcc 4.0.1 and 4.2.1, using CC="gcc -arch ppc64"  
>>>> CPPFLAGS="-arch
>>>> ppc64" and supplying GNU gettext and GNU libiconv (since it  
>>>> chokes on the
>>>> system iconv.h). No Google hit.
>>>
>>> Strange, Fink seems to support it:
>>> http://pdb.finkproject.org/pdb/package.php/glib2-dev
>>
>> Says nothing about powerpc64...
>
> Shows my ignorance of Mac or Darwin :).  I figured 10.5/powerpc is
> what you're after.

No prob, it was a very short-lived technology - arrived in v10.4 half- 
done, became usable in v10.5 and was dropped in v10.6 in favor of Intel.
You have the same ppc vs. ppc64 split for Linux packages btw.

GLib 2.22.4 finished building okay now. I've filed a bug against GLib.  
Hope it works at runtime.

Andreas

Re: [Qemu-devel] [PATCH v5 0/5] Coroutines for better asynchronous programming

[Stefan Hajnoczi]

On Sun, Jun 12, 2011 at 9:46 PM, Stefan Hajnoczi
<stefanha@linux.vnet.ibm.com> wrote:
> QEMU is event-driven and suffers when blocking operations are performed because
> VM execution may be stopped until the operation completes.  Therefore many
> operations that could block are performed asynchronously and a callback is
> invoked when the operation has completed.  This allows QEMU to continue
> executing while the operation is pending.
>
> The downside to callbacks is that they split up code into many smaller
> functions, each of which is a single step in a state machine that quickly
> becomes complex and hard to understand.  Callback functions also result in lots
> of noise as variables are packed and unpacked into temporary structs that pass
> state to the callback function.
>
> This patch series introduces coroutines as a solution for writing asynchronous
> code while still having a nice sequential control flow.  The semantics are
> explained in the second patch.  The fourth patch adds automated tests.
>
> A nice feature of coroutines is that it is relatively easy to take synchronous
> code and lift it into a coroutine to make it asynchronous.  Work has been done
> to move qcow2 request processing into coroutines and thereby make it
> asynchronous (today qcow2 will perform synchronous metadata accesses).  This
> qcow2 work is still ongoing and not quite ready for mainline yet.
>
> Coroutines are also being used for virtfs (virtio-9p) so I have submitted this
> patch now because virtfs patches that depend on coroutines are being published.
>
> v5:
>  * GThread-based implementation for platforms without makecontext(3) (Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>)
>  * Switch to gtester test framework
>
> v4:
>  * Windows Fibers support (Paolo Bonzini <pbonzini@redhat.com>)
>  * Return-after-setjmp() fix (Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>)
>  * Re-entrancy for multi-threaded coroutines support
>  * qemu-coroutine.h cleanup and documentation
>
> v3:
>  * Updated LGPL v2 license header to use web link
>  * Removed atexit(3) pool freeing
>  * Removed thread-local current/leader
>  * Documented thread-safety limitation
>  * Disabled trace events
>
> v2:
>  * Added ./check-coroutine --lifecycle-benchmark for performance measurement
>  * Split pooling into a separate patch with performance justification
>  * Set maximum pool size to prevent holding onto too many free coroutines
>  * Added atexit(3) handler to free pool
>  * Coding style cleanups
>
> Aneesh Kumar K.V (1):
>  coroutine: implement coroutines using gthread
>
> Anthony Liguori (1):
>  Add hard build dependency on glib
>
> Kevin Wolf (1):
>  coroutine: introduce coroutines
>
> Stefan Hajnoczi (2):
>  coroutine: add check-coroutine automated tests
>  coroutine: add check-coroutine --benchmark-lifecycle
>
>  .gitignore           |    1 +
>  Makefile             |    5 +-
>  Makefile.objs        |   13 +++
>  Makefile.target      |    1 +
>  configure            |   29 +++++++
>  coroutine-gthread.c  |  131 ++++++++++++++++++++++++++++
>  coroutine-ucontext.c |  229 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  coroutine-win32.c    |   92 ++++++++++++++++++++
>  qemu-coroutine-int.h |   48 +++++++++++
>  qemu-coroutine.c     |   75 ++++++++++++++++
>  qemu-coroutine.h     |   95 +++++++++++++++++++++
>  test-coroutine.c     |  192 ++++++++++++++++++++++++++++++++++++++++++
>  trace-events         |    5 +
>  13 files changed, 915 insertions(+), 1 deletions(-)
>  create mode 100644 coroutine-gthread.c
>  create mode 100644 coroutine-ucontext.c
>  create mode 100644 coroutine-win32.c
>  create mode 100644 qemu-coroutine-int.h
>  create mode 100644 qemu-coroutine.c
>  create mode 100644 qemu-coroutine.h
>  create mode 100644 test-coroutine.c

Any objections to merging this?

Stefan

Re: [Qemu-devel] [PATCH v5 1/5] Add hard build dependency on glib

[Andreas Färber]

Am 12.06.2011 um 22:46 schrieb Stefan Hajnoczi:

> From: Anthony Liguori <aliguori@us.ibm.com>
>
> GLib is an extremely common library that has a portable thread  
> implementation
> along with tons of other goodies.
>
> GLib and GObject have a fantastic amount of infrastructure we can  
> leverage in
> QEMU including an object oriented programming infrastructure.
>
> Short term, it has a very nice thread pool implementation that we  
> could leverage
> in something like virtio-9p.  It also has a test harness  
> implementation that
> this series will use.
>
> Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
> Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> ---
> Makefile        |    2 ++
> Makefile.objs   |    2 ++
> Makefile.target |    1 +
> configure       |   13 +++++++++++++
> 4 files changed, 18 insertions(+), 0 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 096480b..306cd9b 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -106,6 +106,8 @@ audio/audio.o audio/fmodaudio.o: QEMU_CFLAGS += $ 
> (FMOD_CFLAGS)
>
> QEMU_CFLAGS+=$(CURL_CFLAGS)
>
> +QEMU_CFLAGS+=$(GLIB_CFLAGS)
> +
> ui/cocoa.o: ui/cocoa.m
>
> ui/sdl.o audio/sdlaudio.o ui/sdl_zoom.o baum.o: QEMU_CFLAGS += $ 
> (SDL_CFLAGS)
> diff --git a/Makefile.objs b/Makefile.objs
> index 52d8b23..a7807e8 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -368,3 +368,5 @@ vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
>
> vl.o: QEMU_CFLAGS+=$(SDL_CFLAGS)
>
> +vl.o: QEMU_CFLAGS+=$(GLIB_CFLAGS)
> +
> diff --git a/Makefile.target b/Makefile.target
> index b1a0f6d..0d900e2 100644
> --- a/Makefile.target
> +++ b/Makefile.target
> @@ -202,6 +202,7 @@ QEMU_CFLAGS += $(VNC_TLS_CFLAGS)
> QEMU_CFLAGS += $(VNC_SASL_CFLAGS)
> QEMU_CFLAGS += $(VNC_JPEG_CFLAGS)
> QEMU_CFLAGS += $(VNC_PNG_CFLAGS)
> +QEMU_CFLAGS += $(GLIB_CFLAGS)
>
> # xen backend driver support
> obj-i386-$(CONFIG_XEN) += xen_machine_pv.o xen_domainbuild.o
> diff --git a/configure b/configure
> index 663b557..3697eba 100755
> --- a/configure
> +++ b/configure
> @@ -1767,6 +1767,18 @@ EOF
> fi
>
> ##########################################
> +# glib support probe
> +if $pkg_config --modversion gthread-2.0 gio-2.0 > /dev/null 2>&1 ;  
> then
> +    glib_cflags=`$pkg_config --cflags gthread-2.0 gio-2.0 2>/dev/ 
> null`
> +    glib_libs=`$pkg_config --libs gthread-2.0 gio-2.0 2>/dev/null`
> +    libs_softmmu="$glib_libs $libs_softmmu"
> +    libs_tools="$glib_libs $libs_tools"
> +else
> +    echo "glib-2.0 required to compile QEMU"
> +    exit 1
> +fi
> +
> +##########################################
> # kvm probe
> if test "$kvm" != "no" ; then
>     cat > $TMPC <<EOF

This hunk no longer applies.

Andreas

> @@ -2923,6 +2935,7 @@ if test "$bluez" = "yes" ; then
>   echo "CONFIG_BLUEZ=y" >> $config_host_mak
>   echo "BLUEZ_CFLAGS=$bluez_cflags" >> $config_host_mak
> fi
> +echo "GLIB_CFLAGS=$glib_cflags" >> $config_host_mak
> if test "$xen" = "yes" ; then
>   echo "CONFIG_XEN=y" >> $config_host_mak
>   echo "CONFIG_XEN_CTRL_INTERFACE_VERSION=$xen_ctrl_version" >>  
> $config_host_mak
> -- 
> 1.7.5.3
>
>

Re: [Qemu-devel] [PATCH v5 2/5] coroutine: introduce coroutines

[Andreas Färber]

Am 12.06.2011 um 22:46 schrieb Stefan Hajnoczi:

> From: Kevin Wolf <kwolf@redhat.com>
>
> Asynchronous code is becoming very complex.  At the same time
> synchronous code is growing because it is convenient to write.
> Sometimes duplicate code paths are even added, one synchronous and the
> other asynchronous.  This patch introduces coroutines which allow code
> that looks synchronous but is asynchronous under the covers.
>
> A coroutine has its own stack and is therefore able to preserve state
> across blocking operations, which traditionally require callback
> functions and manual marshalling of parameters.
>
> Creating and starting a coroutine is easy:
>
>  coroutine = qemu_coroutine_create(my_coroutine);
>  qemu_coroutine_enter(coroutine, my_data);
>
> The coroutine then executes until it returns or yields:
>
>  void coroutine_fn my_coroutine(void *opaque) {
>      MyData *my_data = opaque;
>
>      /* do some work */
>
>      qemu_coroutine_yield();
>
>      /* do some more work */
>  }
>
> Yielding switches control back to the caller of  
> qemu_coroutine_enter().
> This is typically used to switch back to the main thread's event loop
> after issuing an asynchronous I/O request.  The request callback will
> then invoke qemu_coroutine_enter() once more to switch back to the
> coroutine.
>
> Note that if coroutines are used only from threads which hold the  
> global
> mutex they will never execute concurrently.  This makes programming  
> with
> coroutines easier than with threads.  Race conditions cannot occur  
> since
> only one coroutine may be active at any time.  Other coroutines can  
> only
> run across yield.
>
> This coroutines implementation is based on the gtk-vnc implementation
> written by Anthony Liguori <anthony@codemonkey.ws> but it has been
> significantly rewritten by Kevin Wolf <kwolf@redhat.com> to use
> setjmp()/longjmp() instead of the more expensive swapcontext() and by
> Paolo Bonzini <pbonzini@redhat.com> for Windows Fibers support.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> ---
> Makefile.objs        |    7 ++
> coroutine-ucontext.c |  229 +++++++++++++++++++++++++++++++++++++++++ 
> +++++++++
> coroutine-win32.c    |   92 ++++++++++++++++++++
> qemu-coroutine-int.h |   48 +++++++++++
> qemu-coroutine.c     |   75 ++++++++++++++++
> qemu-coroutine.h     |   95 +++++++++++++++++++++
> trace-events         |    5 +
> 7 files changed, 551 insertions(+), 0 deletions(-)
> create mode 100644 coroutine-ucontext.c
> create mode 100644 coroutine-win32.c
> create mode 100644 qemu-coroutine-int.h
> create mode 100644 qemu-coroutine.c
> create mode 100644 qemu-coroutine.h

> diff --git a/trace-events b/trace-events
> index e0e9574..e0a4d83 100644
> --- a/trace-events
> +++ b/trace-events
> @@ -386,3 +386,8 @@ disable xen_unmap_block(void* addr, unsigned  
> long size) "%p, size %#lx"
>
> # exec.c
> disable qemu_put_ram_ptr(void* addr) "%p"
> +
> +# qemu-coroutine.c
> +disable qemu_coroutine_enter(void *from, void *to, void *opaque)  
> "from %p to %p opaque %p"
> +disable qemu_coroutine_yield(void *from, void *to) "from %p to %p"
> +disable qemu_coroutine_terminate(void *co) "self %p"

This hunk no longer applies.

Andreas

Re: [Qemu-devel] [PATCH v5 4/5] coroutine: add check-coroutine automated tests

[Andreas Färber]

Am 12.06.2011 um 22:46 schrieb Stefan Hajnoczi:

> To run automated tests for coroutines:
>
>  make check-coroutine
>  ./check-coroutine

The commit message doesn't correspond to the changes, it's test- 
coroutine below.

$ ./test-coroutine
/basic/lifecycle: Segmentation fault

Compiles without warnings but crashes at runtime on Darwin/ppc64. It's  
using coroutine-ucontext.o implementation.

(gdb) run
Starting program: /Users/andreas/QEMU/qemu64/test-coroutine
warning: posix_spawn failed, trying execvp, error: 86
Reading symbols for shared libraries +++++++++++++ 
+ 
....................................................................... done
/basic/lifecycle:
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000
qemu_coroutine_create (entry=0x2b10 <set_and_exit>) at /Users/andreas/ 
QEMU/qemu/qemu-coroutine.c:23
23	    co->entry = entry;
(gdb)

Andreas

>
> On success the program terminates with exit status 0.  On failure an
> error message is written to stderr and the program exits with exit
> status 1.
>
> Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> ---
> .gitignore       |    1 +
> Makefile         |    3 +-
> test-coroutine.c |  162 +++++++++++++++++++++++++++++++++++++++++++++ 
> +++++++++
> 3 files changed, 165 insertions(+), 1 deletions(-)
> create mode 100644 test-coroutine.c
>
> diff --git a/.gitignore b/.gitignore
> index 08013fc..3ad334a 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -34,6 +34,7 @@ qemu-img-cmds.h
> qemu-io
> qemu-monitor.texi
> QMP/qmp-commands.txt
> +test-coroutine
> .gdbinit
> *.a
> *.aux
> diff --git a/Makefile b/Makefile
> index 306cd9b..6b5899a 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -134,7 +134,7 @@ qemu-io$(EXESUF): qemu-io.o cmd.o qemu-tool.o  
> qemu-error.o $(oslib-obj-y) $(trac
> qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.hx
> 	$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -h < $< > $@,"   
> GEN   $@")
>
> -check-qint.o check-qstring.o check-qdict.o check-qlist.o check- 
> qfloat.o check-qjson.o: $(GENERATED_HEADERS)
> +check-qint.o check-qstring.o check-qdict.o check-qlist.o check- 
> qfloat.o check-qjson.o test-coroutine.o: $(GENERATED_HEADERS)
>
> CHECK_PROG_DEPS = qemu-malloc.o $(oslib-obj-y) $(trace-obj-y) qemu- 
> tool.o
>
> @@ -144,6 +144,7 @@ check-qdict: check-qdict.o qdict.o qfloat.o  
> qint.o qstring.o qbool.o qlist.o $(C
> check-qlist: check-qlist.o qlist.o qint.o $(CHECK_PROG_DEPS)
> check-qfloat: check-qfloat.o qfloat.o $(CHECK_PROG_DEPS)
> check-qjson: check-qjson.o qfloat.o qint.o qdict.o qstring.o qlist.o  
> qbool.o qjson.o json-streamer.o json-lexer.o json-parser.o error.o  
> qerror.o qemu-error.o $(CHECK_PROG_DEPS)
> +test-coroutine: test-coroutine.o $(coroutine-obj-y) $ 
> (CHECK_PROG_DEPS)
>
> QEMULIBS=libhw32 libhw64 libuser libdis libdis-user
>
> diff --git a/test-coroutine.c b/test-coroutine.c
> new file mode 100644
> index 0000000..9e9d3c9
> --- /dev/null
> +++ b/test-coroutine.c
> @@ -0,0 +1,162 @@
> +/*
> + * Coroutine tests
> + *
> + * Copyright IBM, Corp. 2011
> + *
> + * Authors:
> + *  Stefan Hajnoczi    <stefanha@linux.vnet.ibm.com>
> + *
> + * This work is licensed under the terms of the GNU LGPL, version 2  
> or later.
> + * See the COPYING.LIB file in the top-level directory.
> + *
> + */
> +
> +#include <glib.h>
> +#include "qemu-coroutine.h"
> +
> +/*
> + * Check that qemu_in_coroutine() works
> + */
> +
> +static void coroutine_fn verify_in_coroutine(void *opaque)
> +{
> +    g_assert(qemu_in_coroutine());
> +}
> +
> +static void test_in_coroutine(void)
> +{
> +    Coroutine *coroutine;
> +
> +    g_assert(!qemu_in_coroutine());
> +
> +    coroutine = qemu_coroutine_create(verify_in_coroutine);
> +    qemu_coroutine_enter(coroutine, NULL);
> +}
> +
> +/*
> + * Check that qemu_coroutine_self() works
> + */
> +
> +static void coroutine_fn verify_self(void *opaque)
> +{
> +    g_assert(qemu_coroutine_self() == opaque);
> +}
> +
> +static void test_self(void)
> +{
> +    Coroutine *coroutine;
> +
> +    coroutine = qemu_coroutine_create(verify_self);
> +    qemu_coroutine_enter(coroutine, coroutine);
> +}
> +
> +/*
> + * Check that coroutines may nest multiple levels
> + */
> +
> +typedef struct {
> +    unsigned int n_enter;   /* num coroutines entered */
> +    unsigned int n_return;  /* num coroutines returned */
> +    unsigned int max;       /* maximum level of nesting */
> +} NestData;
> +
> +static void coroutine_fn nest(void *opaque)
> +{
> +    NestData *nd = opaque;
> +
> +    nd->n_enter++;
> +
> +    if (nd->n_enter < nd->max) {
> +        Coroutine *child;
> +
> +        child = qemu_coroutine_create(nest);
> +        qemu_coroutine_enter(child, nd);
> +    }
> +
> +    nd->n_return++;
> +}
> +
> +static void test_nesting(void)
> +{
> +    Coroutine *root;
> +    NestData nd = {
> +        .n_enter  = 0,
> +        .n_return = 0,
> +        .max      = 128,
> +    };
> +
> +    root = qemu_coroutine_create(nest);
> +    qemu_coroutine_enter(root, &nd);
> +
> +    /* Must enter and return from max nesting level */
> +    g_assert_cmpint(nd.n_enter, ==, nd.max);
> +    g_assert_cmpint(nd.n_return, ==, nd.max);
> +}
> +
> +/*
> + * Check that yield/enter transfer control correctly
> + */
> +
> +static void coroutine_fn yield_5_times(void *opaque)
> +{
> +    bool *done = opaque;
> +    int i;
> +
> +    for (i = 0; i < 5; i++) {
> +        qemu_coroutine_yield();
> +    }
> +    *done = true;
> +}
> +
> +static void test_yield(void)
> +{
> +    Coroutine *coroutine;
> +    bool done = false;
> +    int i = -1; /* one extra time to return from coroutine */
> +
> +    coroutine = qemu_coroutine_create(yield_5_times);
> +    while (!done) {
> +        qemu_coroutine_enter(coroutine, &done);
> +        i++;
> +    }
> +    g_assert_cmpint(i, ==, 5); /* coroutine must yield 5 times */
> +}
> +
> +/*
> + * Check that creation, enter, and return work
> + */
> +
> +static void coroutine_fn set_and_exit(void *opaque)
> +{
> +    bool *done = opaque;
> +
> +    *done = true;
> +}
> +
> +static void test_lifecycle(void)
> +{
> +    Coroutine *coroutine;
> +    bool done = false;
> +
> +    /* Create, enter, and return from coroutine */
> +    coroutine = qemu_coroutine_create(set_and_exit);
> +    qemu_coroutine_enter(coroutine, &done);
> +    g_assert(done); /* expect done to be true (first time) */
> +
> +    /* Repeat to check that no state affects this test */
> +    done = false;
> +    coroutine = qemu_coroutine_create(set_and_exit);
> +    qemu_coroutine_enter(coroutine, &done);
> +    g_assert(done); /* expect done to be true (second time) */
> +}
> +
> +int main(int argc, char **argv)
> +{
> +    g_test_init(&argc, &argv, NULL);
> +    g_test_add_func("/basic/lifecycle", test_lifecycle);
> +    g_test_add_func("/basic/yield", test_yield);
> +    g_test_add_func("/basic/nesting", test_nesting);
> +    g_test_add_func("/basic/self", test_self);
> +    g_test_add_func("/basic/in_coroutine", test_in_coroutine);
> +    return g_test_run();
> +}
> -- 
> 1.7.5.3
>
>

Re: [Qemu-devel] [PATCH v5 4/5] coroutine: add check-coroutine automated tests

[Andreas Färber]

Am 25.06.2011 um 16:11 schrieb Andreas Färber:

> Am 12.06.2011 um 22:46 schrieb Stefan Hajnoczi:
>
>> To run automated tests for coroutines:
>>
>> make check-coroutine
>> ./check-coroutine
>
> The commit message doesn't correspond to the changes, it's test- 
> coroutine below.
>
> $ ./test-coroutine
> /basic/lifecycle: Segmentation fault
>
> Compiles without warnings but crashes at runtime on Darwin/ppc64.  
> It's using coroutine-ucontext.o implementation.
>
> (gdb) run
> Starting program: /Users/andreas/QEMU/qemu64/test-coroutine
> warning: posix_spawn failed, trying execvp, error: 86
> Reading symbols for shared libraries +++++++++++++ 
> + 
> ....................................................................... done
> /basic/lifecycle:
> Program received signal EXC_BAD_ACCESS, Could not access memory.
> Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000
> qemu_coroutine_create (entry=0x2b10 <set_and_exit>) at /Users/ 
> andreas/QEMU/qemu/qemu-coroutine.c:23
> 23	    co->entry = entry;
> (gdb)

Commenting out CONFIG_UCONTEXT_COROUTINE=y in config-host.mak to force  
the coroutine-gthread.o implementation, this does not happen:

$ ./test-coroutine
/basic/lifecycle: OK
/basic/yield: OK
/basic/nesting: OK
/basic/self: OK
/basic/in_coroutine: OK

Andreas

Re: [Qemu-devel] [PATCH v5 4/5] coroutine: add check-coroutine automated tests

[Andreas Färber]

Am 25.06.2011 um 16:11 schrieb Andreas Färber:

> Am 12.06.2011 um 22:46 schrieb Stefan Hajnoczi:
>
>> To run automated tests for coroutines:
>>
>> make check-coroutine
>> ./check-coroutine
>
> The commit message doesn't correspond to the changes, it's test- 
> coroutine below.
>
> $ ./test-coroutine
> /basic/lifecycle: Segmentation fault
>
> Compiles without warnings but crashes at runtime on Darwin/ppc64.  
> It's using coroutine-ucontext.o implementation.
>
> (gdb) run
> Starting program: /Users/andreas/QEMU/qemu64/test-coroutine
> warning: posix_spawn failed, trying execvp, error: 86
> Reading symbols for shared libraries +++++++++++++ 
> + 
> ....................................................................... done
> /basic/lifecycle:
> Program received signal EXC_BAD_ACCESS, Could not access memory.
> Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000
> qemu_coroutine_create (entry=0x2b10 <set_and_exit>) at /Users/ 
> andreas/QEMU/qemu/qemu-coroutine.c:23
> 23	    co->entry = entry;
> (gdb)

(gdb) i r
r0             0x3690	13968
r1             0x7fff5fbff1e0	140734799802848
r2             0x7fff70231870	140735074736240
r3             0x0	0
r4             0x10080be00	4303404544
r5             0x0	0
r6             0x2	2
r7             0x0	0
r8             0x8	8
r9             0x0	0
r10            0x7fff80f14fac	140735356686252
r11            0x7fff70237940	140735074761024
r12            0x7fff80f14fa0	140735356686240
r13            0x7fff7028fde8	140735075122664
r14            0x0	0
r15            0x0	0
r16            0x0	0
r17            0x100240e94	4297330324
r18            0x100539040	4300443712
r19            0x100609400	4301296640
r20            0x0	0
r21            0x0	0
r22            0x100609b60	4301298528
r23            0x100240e94	4297330324
r24            0x100240e94	4297330324
r25            0x0	0
r26            0x0	0
r27            0x100205d78	4297088376
r28            0x7fff70231870	140735074736240
r29            0x2b10	11024
r30            0x2b10	11024
r31            0x0	0
pc             0x3690	13968
ps             0x900000000200f030	10376293541495238704
cr             0x24022222	604119586
lr             0x3690	13968
ctr            0x7fff80f14fa0	140735356686240
xer            0x20000000	536870912
mq             0x0	0
fpscr          0x82000000	2181038080
vscr           0x10000	65536
vrsave         0x0	0
(gdb) x/10i ($pc-8)
0x3688 <qemu_coroutine_create+40>:	li      r0,0
0x368c <qemu_coroutine_create+44>:	bl      0x3ca0 <qemu_coroutine_new>
0x3690 <qemu_coroutine_create+48>:	std     r29,0(r3)
0x3694 <qemu_coroutine_create+52>:	nop
0x3698 <qemu_coroutine_create+56>:	nop
0x369c <qemu_coroutine_create+60>:	nop
0x36a0 <qemu_coroutine_create+64>:	ld      r0,120(r1)
0x36a4 <qemu_coroutine_create+68>:	ld      r2,0(r28)
0x36a8 <qemu_coroutine_create+72>:	xor.    r0,r0,r2
0x36ac <qemu_coroutine_create+76>:	li      r2,0
(gdb) 

Re: [Qemu-devel] [PATCH v5 2/5] coroutine: introduce coroutines

[Andreas Färber]

Am 12.06.2011 um 22:46 schrieb Stefan Hajnoczi:

> From: Kevin Wolf <kwolf@redhat.com>
>
> Asynchronous code is becoming very complex.  At the same time
> synchronous code is growing because it is convenient to write.
> Sometimes duplicate code paths are even added, one synchronous and the
> other asynchronous.  This patch introduces coroutines which allow code
> that looks synchronous but is asynchronous under the covers.
>
> A coroutine has its own stack and is therefore able to preserve state
> across blocking operations, which traditionally require callback
> functions and manual marshalling of parameters.
>
> Creating and starting a coroutine is easy:
>
>  coroutine = qemu_coroutine_create(my_coroutine);
>  qemu_coroutine_enter(coroutine, my_data);
>
> The coroutine then executes until it returns or yields:
>
>  void coroutine_fn my_coroutine(void *opaque) {
>      MyData *my_data = opaque;
>
>      /* do some work */
>
>      qemu_coroutine_yield();
>
>      /* do some more work */
>  }
>
> Yielding switches control back to the caller of  
> qemu_coroutine_enter().
> This is typically used to switch back to the main thread's event loop
> after issuing an asynchronous I/O request.  The request callback will
> then invoke qemu_coroutine_enter() once more to switch back to the
> coroutine.
>
> Note that if coroutines are used only from threads which hold the  
> global
> mutex they will never execute concurrently.  This makes programming  
> with
> coroutines easier than with threads.  Race conditions cannot occur  
> since
> only one coroutine may be active at any time.  Other coroutines can  
> only
> run across yield.
>
> This coroutines implementation is based on the gtk-vnc implementation
> written by Anthony Liguori <anthony@codemonkey.ws> but it has been
> significantly rewritten by Kevin Wolf <kwolf@redhat.com> to use
> setjmp()/longjmp() instead of the more expensive swapcontext() and by
> Paolo Bonzini <pbonzini@redhat.com> for Windows Fibers support.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> ---
> Makefile.objs        |    7 ++
> coroutine-ucontext.c |  229 +++++++++++++++++++++++++++++++++++++++++ 
> +++++++++
> coroutine-win32.c    |   92 ++++++++++++++++++++
> qemu-coroutine-int.h |   48 +++++++++++
> qemu-coroutine.c     |   75 ++++++++++++++++
> qemu-coroutine.h     |   95 +++++++++++++++++++++
> trace-events         |    5 +
> 7 files changed, 551 insertions(+), 0 deletions(-)
> create mode 100644 coroutine-ucontext.c
> create mode 100644 coroutine-win32.c
> create mode 100644 qemu-coroutine-int.h
> create mode 100644 qemu-coroutine.c
> create mode 100644 qemu-coroutine.h

> diff --git a/coroutine-ucontext.c b/coroutine-ucontext.c
> new file mode 100644
> index 0000000..bcea2bd
> --- /dev/null
> +++ b/coroutine-ucontext.c

> +static Coroutine *coroutine_new(void)
> +{
> +    const size_t stack_size = 4 << 20;
> +    CoroutineUContext *co;
> +    ucontext_t old_uc, uc;
> +    jmp_buf old_env;
> +    union cc_arg arg;
> +
> +    /* The ucontext functions preserve signal masks which incurs a  
> system call
> +     * overhead.  setjmp()/longjmp() does not preserve signal masks  
> but only
> +     * works on the current stack.  Since we need a way to create  
> and switch to
> +     * a new stack, use the ucontext functions for that but  
> setjmp()/longjmp()
> +     * for everything else.
> +     */
> +
> +    if (getcontext(&uc) == -1) {
> +        return NULL;

We figured out: on Darwin/ppc64 we run into this code path, with errno  
= ENOTSUP.
Same for Darwin/ppc, v10.5.8.

> +    }
> +
> +    co = qemu_mallocz(sizeof(*co));
> +    co->stack = qemu_malloc(stack_size);
> +    co->base.entry_arg = &old_env; /* stash away our jmp_buf */
> +
> +    uc.uc_link = &old_uc;
> +    uc.uc_stack.ss_sp = co->stack;
> +    uc.uc_stack.ss_size = stack_size;
> +    uc.uc_stack.ss_flags = 0;
> +
> +    arg.p = co;
> +
> +    makecontext(&uc, (void (*)(void))coroutine_trampoline,
> +                2, arg.i[0], arg.i[1]);
> +
> +    /* swapcontext() in, longjmp() back out */
> +    if (!setjmp(old_env)) {
> +        swapcontext(&old_uc, &uc);
> +    }
> +    return &co->base;
> +}
> +
> +Coroutine *qemu_coroutine_new(void)
> +{
> +    CoroutineThreadState *s = coroutine_get_thread_state();
> +    Coroutine *co;
> +
> +    co = QLIST_FIRST(&s->pool);
> +    if (co) {
> +        QLIST_REMOVE(co, pool_next);
> +        s->pool_size--;
> +    } else {
> +        co = coroutine_new();

Return value is not checked ...

> +    }
> +    return co;

... and passed through.

> +}

Caller in test-coroutine.c doesn't check either. Maybe better abort  
instead of returning NULL?

Andreas

Re: [Qemu-devel] [PATCH v5 3/5] coroutine: implement coroutines using gthread

[Andreas Färber]

Am 12.06.2011 um 22:46 schrieb Stefan Hajnoczi:

> From: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
>
> On platforms that don't support makecontext(3) use gthread based
> coroutine implementation.
>
> [Original patch by Aneesh, made consistent with coroutine-ucontext.c  
> and
> switched to GStaticPrivate by Stefan.  Tested on Linux and OpenBSD.]
>
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> ---
> Makefile.objs       |    4 ++
> configure           |   16 ++++++
> coroutine-gthread.c |  131 ++++++++++++++++++++++++++++++++++++++++++ 
> +++++++++
> 3 files changed, 151 insertions(+), 0 deletions(-)
> create mode 100644 coroutine-gthread.c

> diff --git a/configure b/configure
> index 3697eba..df63403 100755
> --- a/configure
> +++ b/configure
> @@ -2552,6 +2552,18 @@ if test "$trace_backend" = "dtrace"; then
> fi
>
> ##########################################
> +# check if we have makecontext
> +
> +ucontext_coroutine=no
> +cat > $TMPC << EOF
> +#include <ucontext.h>
> +int main(void) { makecontext(0, 0, 0); }
> +EOF
> +if compile_prog "" "" ; then
> +    ucontext_coroutine=yes
> +fi
> +
> +##########################################
> # End of CC checks
> # After here, no more $cc or $ld runs
>

This add-on patch inserts a check for Darwin to force  
ucontext_coroutine=no there (getcontext() returns -1, with errno ==  
ENOTSUP):

diff --git a/configure b/configure
index c28ed7b..854f24a 100755
--- a/configure
+++ b/configure
@@ -2479,12 +2479,14 @@ fi
  # check if we have makecontext

  ucontext_coroutine=no
-cat > $TMPC << EOF
+if test "$darwin" != "yes"; then
+    cat > $TMPC << EOF
  #include <ucontext.h>
  int main(void) { makecontext(0, 0, 0); }
  EOF
-if compile_prog "" "" ; then
-    ucontext_coroutine=yes
+    if compile_prog "" "" ; then
+        ucontext_coroutine=yes
+    fi
  fi

  ##########################################

This fixes Darwin/ppc64 (and ppc) v10.5.
Don't know whether v10.6 / i386 might have a working implementation  
(cc'ing Alexand{re,er}).

Andreas

> @@ -3015,6 +3027,10 @@ if test "$rbd" = "yes" ; then
>   echo "CONFIG_RBD=y" >> $config_host_mak
> fi
>
> +if test "$ucontext_coroutine" = "yes" ; then
> +  echo "CONFIG_UCONTEXT_COROUTINE=y" >> $config_host_mak
> +fi
> +
> # USB host support
> case "$usb" in
> linux)

Re: [Qemu-devel] [PATCH v5 3/5] coroutine: implement coroutines using gthread

[Alexandre Raymond]

Hi Andreas,

> This fixes Darwin/ppc64 (and ppc) v10.5.
> Don't know whether v10.6 / i386 might have a working implementation (cc'ing
> Alexand{re,er}).

In 10.6, I get the following error when I try to compile a simple getcontext():
/usr/include/ucontext.h:42:2: error: #error ucontext routines are
deprecated, and require _XOPEN_SOURCE to be defined

On the other hand, if I do define _XOPEN_SOURCE, getcontext() returns 0.

Alexandre