[Qemu-devel] [PATCH v20 00/11] Shared library module support

[Qemu-devel] [PATCH v20 00/11] Shared library module support

[Fam Zheng]

Rewrote the executable directory patch and added Darwin API in
qemu_init_exec_dir().

v20:
    Dropped the argv0 passing patch from v19. Refactored qemu_get_exec_dir() in
    patch 01. Three patches are affected:

    [01/11] util: Split out exec_dir from os_find_datadir
            Rewritten. The exec_dir is a static global in os-{win32,posix}.c
            that is initialized in main() and later used with
            qemu_get_exec_dir().

    [07/11] module: implement module loading
            Use glue(). Prefix hash with "_", so the preceding digit won't
            cause compile error now.

    [11/11] oslib: port qemu_init_exec_dir to Darwin
            New.



Fam Zheng (10):
  util: Split out exec_dir from os_find_datadir
  rules.mak: fix $(obj) to a real relative path
  rules.mak: allow per object cflags and libs
  block: use per-object cflags and libs
  rules.mak: introduce DSO rules
  module: implement module loading
  Makefile: install modules with "make install"
  Makefile: introduce common-obj-m and block-obj-m for DSO
  block: convert block drivers linked with libs to modules
  oslib: port qemu_init_exec_dir to Darwin

Paolo Bonzini (1):
  darwin: do not use -mdynamic-no-pic

 .gitignore            |   3 ++
 Makefile              |  29 +++++++++-
 Makefile.objs         |  19 ++-----
 Makefile.target       |  21 ++++++--
 block/Makefile.objs   |  13 ++++-
 configure             |  93 +++++++++++++++++++++-----------
 include/qemu-common.h |   2 +-
 include/qemu/module.h |  23 +++++++-
 include/qemu/osdep.h  |   9 ++++
 module-common.c       |  10 ++++
 os-posix.c            |  42 +++------------
 os-win32.c            |  21 +-------
 qemu-img.c            |   1 +
 qemu-io.c             |   1 +
 qemu-nbd.c            |   1 +
 rules.mak             |  80 +++++++++++++++++++++++-----
 scripts/create_config |   3 ++
 util/module.c         | 145 +++++++++++++++++++++++++++++++++++++++++++++++++-
 util/oslib-posix.c    |  69 ++++++++++++++++++++++++
 util/oslib-win32.c    |  30 +++++++++++
 vl.c                  |   3 +-
 21 files changed, 494 insertions(+), 124 deletions(-)
 create mode 100644 module-common.c

-- 
1.8.5.4

[Qemu-devel] [PATCH v20 01/11] util: Split out exec_dir from os_find_datadir

[Fam Zheng]

With this change, main() calls qemu_init_exec_dir and uses argv[0] to
init exec_dir. The saved value can be retrieved with
qemu_get_exec_dir later. It will be reused by module loading.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 include/qemu-common.h |  2 +-
 include/qemu/osdep.h  |  9 +++++++++
 os-posix.c            | 42 +++++++--------------------------------
 os-win32.c            | 21 ++------------------
 qemu-img.c            |  1 +
 qemu-io.c             |  1 +
 qemu-nbd.c            |  1 +
 util/oslib-posix.c    | 54 +++++++++++++++++++++++++++++++++++++++++++++++++++
 util/oslib-win32.c    | 30 ++++++++++++++++++++++++++++
 vl.c                  |  3 ++-
 10 files changed, 108 insertions(+), 56 deletions(-)

diff --git a/include/qemu-common.h b/include/qemu-common.h
index 5054836..17c2959 100644
--- a/include/qemu-common.h
+++ b/include/qemu-common.h
@@ -360,7 +360,7 @@ char *qemu_find_file(int type, const char *name);
 
 /* OS specific functions */
 void os_setup_early_signal_handling(void);
-char *os_find_datadir(const char *argv0);
+char *os_find_datadir(void);
 void os_parse_cmd_args(int index, const char *optarg);
 void os_pidfile_error(void);
 
diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index eac7172..ffb2966 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -215,6 +215,15 @@ bool fips_get_state(void);
  */
 char *qemu_get_local_state_pathname(const char *relative_pathname);
 
+/* Find program directory, and save it for later usage with
+ * qemu_get_exec_dir().
+ * Try OS specific API first, if not working, parse from argv0. */
+void qemu_init_exec_dir(const char *argv0);
+
+/* Get the saved exec dir.
+ * Caller needs to release the returned string by g_free() */
+char *qemu_get_exec_dir(void);
+
 /**
  * qemu_getauxval:
  * @type: the auxiliary vector key to lookup
diff --git a/os-posix.c b/os-posix.c
index d39261d..6187301 100644
--- a/os-posix.c
+++ b/os-posix.c
@@ -84,46 +84,17 @@ void os_setup_signal_handling(void)
    running from the build tree this will be "$bindir/../pc-bios".  */
 #define SHARE_SUFFIX "/share/qemu"
 #define BUILD_SUFFIX "/pc-bios"
-char *os_find_datadir(const char *argv0)
+char *os_find_datadir(void)
 {
-    char *dir;
-    char *p = NULL;
+    char *dir, *exec_dir;
     char *res;
-    char buf[PATH_MAX];
     size_t max_len;
 
-#if defined(__linux__)
-    {
-        int len;
-        len = readlink("/proc/self/exe", buf, sizeof(buf) - 1);
-        if (len > 0) {
-            buf[len] = 0;
-            p = buf;
-        }
-    }
-#elif defined(__FreeBSD__)
-    {
-        static int mib[4] = {CTL_KERN, KERN_PROC, KERN_PROC_PATHNAME, -1};
-        size_t len = sizeof(buf) - 1;
-
-        *buf = '\0';
-        if (!sysctl(mib, ARRAY_SIZE(mib), buf, &len, NULL, 0) &&
-            *buf) {
-            buf[sizeof(buf) - 1] = '\0';
-            p = buf;
-        }
-    }
-#endif
-    /* If we don't have any way of figuring out the actual executable
-       location then try argv[0].  */
-    if (!p) {
-        p = realpath(argv0, buf);
-        if (!p) {
-            return NULL;
-        }
+    exec_dir = qemu_get_exec_dir();
+    if (exec_dir == NULL) {
+        return NULL;
     }
-    dir = dirname(p);
-    dir = dirname(dir);
+    dir = dirname(exec_dir);
 
     max_len = strlen(dir) +
         MAX(strlen(SHARE_SUFFIX), strlen(BUILD_SUFFIX)) + 1;
@@ -137,6 +108,7 @@ char *os_find_datadir(const char *argv0)
         }
     }
 
+    g_free(exec_dir);
     return res;
 }
 #undef SHARE_SUFFIX
diff --git a/os-win32.c b/os-win32.c
index 50b7f6f..5f95caa 100644
--- a/os-win32.c
+++ b/os-win32.c
@@ -84,26 +84,9 @@ void os_setup_early_signal_handling(void)
 }
 
 /* Look for support files in the same directory as the executable.  */
-char *os_find_datadir(const char *argv0)
+char *os_find_datadir(void)
 {
-    char *p;
-    char buf[MAX_PATH];
-    DWORD len;
-
-    len = GetModuleFileName(NULL, buf, sizeof(buf) - 1);
-    if (len == 0) {
-        return NULL;
-    }
-
-    buf[len] = 0;
-    p = buf + len - 1;
-    while (p != buf && *p != '\\')
-        p--;
-    *p = 0;
-    if (access(buf, R_OK) == 0) {
-        return g_strdup(buf);
-    }
-    return NULL;
+    return qemu_get_exec_dir();
 }
 
 void os_set_line_buffering(void)
diff --git a/qemu-img.c b/qemu-img.c
index c989850..45fcddd 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -2719,6 +2719,7 @@ int main(int argc, char **argv)
 #endif
 
     error_set_progname(argv[0]);
+    qemu_init_exec_dir(argv[0]);
 
     qemu_init_main_loop();
     bdrv_init();
diff --git a/qemu-io.c b/qemu-io.c
index 7f459d8..b74ac45 100644
--- a/qemu-io.c
+++ b/qemu-io.c
@@ -381,6 +381,7 @@ int main(int argc, char **argv)
 #endif
 
     progname = basename(argv[0]);
+    qemu_init_exec_dir(argv[0]);
 
     while ((c = getopt_long(argc, argv, sopt, lopt, &opt_index)) != -1) {
         switch (c) {
diff --git a/qemu-nbd.c b/qemu-nbd.c
index 136e8c9..2dc75aa 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -376,6 +376,7 @@ int main(int argc, char **argv)
     memset(&sa_sigterm, 0, sizeof(sa_sigterm));
     sa_sigterm.sa_handler = termsig_handler;
     sigaction(SIGTERM, &sa_sigterm, NULL);
+    qemu_init_exec_dir(argv[0]);
 
     while ((ch = getopt_long(argc, argv, sopt, lopt, &opt_ind)) != -1) {
         switch (ch) {
diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index d5dca47..c2eeb4f 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -57,6 +57,7 @@ extern int daemon(int, int);
 #include "trace.h"
 #include "qemu/sockets.h"
 #include <sys/mman.h>
+#include <libgen.h>
 
 #ifdef CONFIG_LINUX
 #include <sys/syscall.h>
@@ -274,3 +275,56 @@ void qemu_set_tty_echo(int fd, bool echo)
 
     tcsetattr(fd, TCSANOW, &tty);
 }
+
+static char exec_dir[PATH_MAX];
+
+void qemu_init_exec_dir(const char *argv0)
+{
+    char *dir;
+    char *p = NULL;
+    char buf[PATH_MAX];
+
+    assert(!exec_dir[0]);
+
+#if defined(__linux__)
+    {
+        int len;
+        len = readlink("/proc/self/exe", buf, sizeof(buf) - 1);
+        if (len > 0) {
+            buf[len] = 0;
+            p = buf;
+        }
+    }
+#elif defined(__FreeBSD__)
+    {
+        static int mib[4] = {CTL_KERN, KERN_PROC, KERN_PROC_PATHNAME, -1};
+        size_t len = sizeof(buf) - 1;
+
+        *buf = '\0';
+        if (!sysctl(mib, ARRAY_SIZE(mib), buf, &len, NULL, 0) &&
+            *buf) {
+            buf[sizeof(buf) - 1] = '\0';
+            p = buf;
+        }
+    }
+#endif
+    /* If we don't have any way of figuring out the actual executable
+       location then try argv[0].  */
+    if (!p) {
+        if (!argv0) {
+            return;
+        }
+        p = realpath(argv0, buf);
+        if (!p) {
+            return;
+        }
+    }
+    dir = dirname(p);
+
+    pstrcpy(exec_dir, sizeof(exec_dir), dir);
+}
+
+char *qemu_get_exec_dir(void)
+{
+    return g_strdup(exec_dir);
+}
diff --git a/util/oslib-win32.c b/util/oslib-win32.c
index 50be044..93f7d35 100644
--- a/util/oslib-win32.c
+++ b/util/oslib-win32.c
@@ -208,3 +208,33 @@ void qemu_set_tty_echo(int fd, bool echo)
                        dwMode & ~(ENABLE_ECHO_INPUT | ENABLE_LINE_INPUT));
     }
 }
+
+static char exec_dir[PATH_MAX];
+
+void qemu_init_exec_dir(const char *argv0)
+{
+
+    char *p;
+    char buf[MAX_PATH];
+    DWORD len;
+
+    len = GetModuleFileName(NULL, buf, sizeof(buf) - 1);
+    if (len == 0) {
+        return;
+    }
+
+    buf[len] = 0;
+    p = buf + len - 1;
+    while (p != buf && *p != '\\') {
+        p--;
+    }
+    *p = 0;
+    if (access(buf, R_OK) == 0) {
+        pstrcpy(exec_dir, sizeof(exec_dir), buf);
+    }
+}
+
+char *qemu_get_exec_dir(void)
+{
+    return g_strdup(exec_dir);
+}
diff --git a/vl.c b/vl.c
index 383be1b..e2e576c 100644
--- a/vl.c
+++ b/vl.c
@@ -2880,6 +2880,7 @@ int main(int argc, char **argv, char **envp)
 
     atexit(qemu_run_exit_notifiers);
     error_set_progname(argv[0]);
+    qemu_init_exec_dir(argv[0]);
 
     g_mem_set_vtable(&mem_trace);
     if (!g_thread_supported()) {
@@ -3908,7 +3909,7 @@ int main(int argc, char **argv, char **envp)
     /* If no data_dir is specified then try to find it relative to the
        executable path.  */
     if (data_dir_idx < ARRAY_SIZE(data_dir)) {
-        data_dir[data_dir_idx] = os_find_datadir(argv[0]);
+        data_dir[data_dir_idx] = os_find_datadir();
         if (data_dir[data_dir_idx] != NULL) {
             data_dir_idx++;
         }
-- 
1.8.5.4

[Qemu-devel] [PATCH v20 02/11] rules.mak: fix $(obj) to a real relative path

[Fam Zheng]

Makefile.target includes rule.mak and unnested common-obj-y, then prefix
them with '../', this will ignore object specific QEMU_CFLAGS in subdir
Makefile.objs:

    $(obj)/curl.o: QEMU_CFLAGS += $(CURL_CFLAGS)

Because $(obj) here is './block', instead of '../block'. This doesn't
hurt compiling because we basically build all .o from top Makefile,
before entering Makefile.target, but it will affact arriving per-object
libs support.

The starting point of $(obj) is passed in as argument of unnest-vars, as
well as nested variables, so that different Makefiles can pass in a
right value.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Makefile        | 14 ++++++++++++++
 Makefile.objs   | 17 +----------------
 Makefile.target | 17 +++++++++++++----
 configure       |  1 +
 rules.mak       | 14 +++++++++-----
 5 files changed, 38 insertions(+), 25 deletions(-)

diff --git a/Makefile b/Makefile
index 807054b..52b1958 100644
--- a/Makefile
+++ b/Makefile
@@ -122,6 +122,16 @@ defconfig:
 
 ifneq ($(wildcard config-host.mak),)
 include $(SRC_PATH)/Makefile.objs
+endif
+
+dummy := $(call unnest-vars,, \
+                stub-obj-y \
+                util-obj-y \
+                qga-obj-y \
+                block-obj-y \
+                common-obj-y)
+
+ifneq ($(wildcard config-host.mak),)
 include $(SRC_PATH)/tests/Makefile
 endif
 ifeq ($(CONFIG_SMARTCARD_NSS),y)
@@ -130,6 +140,10 @@ endif
 
 all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all
 
+vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
+
+vl.o: QEMU_CFLAGS+=$(SDL_CFLAGS)
+
 config-host.h: config-host.h-timestamp
 config-host.h-timestamp: config-host.mak
 qemu-options.def: $(SRC_PATH)/qemu-options.hx
diff --git a/Makefile.objs b/Makefile.objs
index ac1d0e1..1914080 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -41,7 +41,7 @@ libcacard-y += libcacard/vcardt.o
 # single QEMU executable should support all CPUs and machines.
 
 ifeq ($(CONFIG_SOFTMMU),y)
-common-obj-y = $(block-obj-y) blockdev.o blockdev-nbd.o block/
+common-obj-y = blockdev.o blockdev-nbd.o block/
 common-obj-y += net/
 common-obj-y += qdev-monitor.o device-hotplug.o
 common-obj-$(CONFIG_WIN32) += os-win32.o
@@ -111,18 +111,3 @@ version-lobj-$(CONFIG_WIN32) += $(BUILD_DIR)/version.lo
 # by libqemuutil.a.  These should be moved to a separate .json schema.
 qga-obj-y = qga/ qapi-types.o qapi-visit.o
 qga-vss-dll-obj-y = qga/
-
-vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
-
-vl.o: QEMU_CFLAGS+=$(SDL_CFLAGS)
-
-QEMU_CFLAGS+=$(GLIB_CFLAGS)
-
-nested-vars += \
-	stub-obj-y \
-	util-obj-y \
-	qga-obj-y \
-	qga-vss-dll-obj-y \
-	block-obj-y \
-	common-obj-y
-dummy := $(call unnest-vars)
diff --git a/Makefile.target b/Makefile.target
index af6ac7e..9a6e7dd 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -139,13 +139,22 @@ endif # CONFIG_SOFTMMU
 # Workaround for http://gcc.gnu.org/PR55489, see configure.
 %/translate.o: QEMU_CFLAGS += $(TRANSLATE_OPT_CFLAGS)
 
-nested-vars += obj-y
+dummy := $(call unnest-vars,,obj-y)
 
-# This resolves all nested paths, so it must come last
+# we are making another call to unnest-vars with different vars, protect obj-y,
+# it can be overriden in subdir Makefile.objs
+obj-y-save := $(obj-y)
+
+block-obj-y :=
+common-obj-y :=
 include $(SRC_PATH)/Makefile.objs
+dummy := $(call unnest-vars,..,block-obj-y common-obj-y)
+
+# Now restore obj-y
+obj-y := $(obj-y-save)
 
-all-obj-y = $(obj-y)
-all-obj-y += $(addprefix ../, $(common-obj-y))
+all-obj-y = $(obj-y) $(common-obj-y)
+all-obj-$(CONFIG_SOFTMMU) += $(block-obj-y)
 
 ifndef CONFIG_HAIKU
 LIBS+=-lm
diff --git a/configure b/configure
index 236764a..afff201 100755
--- a/configure
+++ b/configure
@@ -2346,6 +2346,7 @@ fi
 if $pkg_config --atleast-version=$glib_req_ver gthread-2.0; then
     glib_cflags=`$pkg_config --cflags gthread-2.0`
     glib_libs=`$pkg_config --libs gthread-2.0`
+    CFLAGS="$glib_cflags $CFLAGS"
     LIBS="$glib_libs $LIBS"
     libs_qga="$glib_libs $libs_qga"
 else
diff --git a/rules.mak b/rules.mak
index 49edb9b..7d27602 100644
--- a/rules.mak
+++ b/rules.mak
@@ -138,9 +138,6 @@ clean: clean-timestamp
 
 # magic to descend into other directories
 
-obj := .
-old-nested-dirs :=
-
 define push-var
 $(eval save-$2-$1 = $(value $1))
 $(eval $1 :=)
@@ -154,9 +151,11 @@ endef
 
 define unnest-dir
 $(foreach var,$(nested-vars),$(call push-var,$(var),$1/))
-$(eval obj := $(obj)/$1)
+$(eval obj-parent-$1 := $(obj))
+$(eval obj := $(if $(obj),$(obj)/$1,$1))
 $(eval include $(SRC_PATH)/$1/Makefile.objs)
-$(eval obj := $(patsubst %/$1,%,$(obj)))
+$(eval obj := $(obj-parent-$1))
+$(eval obj-parent-$1 := )
 $(foreach var,$(nested-vars),$(call pop-var,$(var),$1/))
 endef
 
@@ -171,7 +170,12 @@ $(if $(nested-dirs),
 endef
 
 define unnest-vars
+$(eval obj := $1)
+$(eval nested-vars := $2)
+$(eval old-nested-dirs := )
 $(call unnest-vars-1)
+$(if $1,$(foreach v,$(nested-vars),$(eval \
+	$v := $(addprefix $1/,$($v)))))
 $(foreach var,$(nested-vars),$(eval $(var) := $(filter-out %/, $($(var)))))
 $(shell mkdir -p $(sort $(foreach var,$(nested-vars),$(dir $($(var))))))
 $(foreach var,$(nested-vars), $(eval \
-- 
1.8.5.4

[Qemu-devel] [PATCH v20 03/11] rules.mak: allow per object cflags and libs

[Fam Zheng]

Adds extract-libs in LINK to expand any "per object libs", the syntax to define
such a libs options is like:

        foo.o-libs := $(CURL_LIBS)

in block/Makefile.objs.

Similarly,

        foo.o-cflags := $(FOO_CFLAGS)

is also supported.

"foo.o" must be listed in a nested var (e.g. common-obj-y) to make the
option variables effective.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 rules.mak | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/rules.mak b/rules.mak
index 7d27602..9398268 100644
--- a/rules.mak
+++ b/rules.mak
@@ -21,15 +21,17 @@ QEMU_DGFLAGS += -MMD -MP -MT $@ -MF $(*D)/$(*F).d
 # Same as -I$(SRC_PATH) -I., but for the nested source/object directories
 QEMU_INCLUDES += -I$(<D) -I$(@D)
 
+extract-libs = $(strip $(foreach o,$1,$($o-libs)))
+
 %.o: %.c
-	$(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")
+	$(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) $($@-cflags) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")
 %.o: %.rc
 	$(call quiet-command,$(WINDRES) -I. -o $@ $<,"  RC    $(TARGET_DIR)$@")
 
 ifeq ($(LIBTOOL),)
 LINK = $(call quiet-command,$(CC) $(QEMU_CFLAGS) $(CFLAGS) $(LDFLAGS) -o $@ \
        $(sort $(filter %.o, $1)) $(filter-out %.o, $1) $(version-obj-y) \
-       $(LIBS),"  LINK  $(TARGET_DIR)$@")
+       $(call extract-libs,$^) $(LIBS),"  LINK  $(TARGET_DIR)$@")
 else
 LIBTOOL += $(if $(V),,--quiet)
 %.lo: %.c
@@ -45,7 +47,7 @@ LINK = $(call quiet-command,\
        $(sort $(filter %.o, $1)) $(filter-out %.o, $1) \
        $(if $(filter %.lo %.la,$^),$(version-lobj-y),$(version-obj-y)) \
        $(if $(filter %.lo %.la,$^),$(LIBTOOLFLAGS)) \
-       $(LIBS),$(if $(filter %.lo %.la,$^),"lt LINK ", "  LINK  ")"$(TARGET_DIR)$@")
+       $(call extract-libs,$^) $(LIBS),$(if $(filter %.lo %.la,$^),"lt LINK ", "  LINK  ")"$(TARGET_DIR)$@")
 endif
 
 %.asm: %.S
@@ -149,11 +151,22 @@ $(eval $1 = $(value save-$2-$1) $$(subdir-$2-$1))
 $(eval save-$2-$1 :=)
 endef
 
+define fix-obj-vars
+$(foreach v,$($1), \
+	$(if $($v-cflags), \
+		$(eval $2$v-cflags := $($v-cflags)) \
+		$(eval $v-cflags := )) \
+	$(if $($v-libs), \
+		$(eval $2$v-libs := $($v-libs)) \
+		$(eval $v-libs := )))
+endef
+
 define unnest-dir
 $(foreach var,$(nested-vars),$(call push-var,$(var),$1/))
 $(eval obj-parent-$1 := $(obj))
 $(eval obj := $(if $(obj),$(obj)/$1,$1))
 $(eval include $(SRC_PATH)/$1/Makefile.objs)
+$(foreach v,$(nested-vars),$(call fix-obj-vars,$v,$(if $(obj),$(obj)/)))
 $(eval obj := $(obj-parent-$1))
 $(eval obj-parent-$1 := )
 $(foreach var,$(nested-vars),$(call pop-var,$(var),$1/))
-- 
1.8.5.4

[Qemu-devel] [PATCH v20 04/11] block: use per-object cflags and libs

[Fam Zheng]

No longer adds flags and libs for them to global variables, instead
create config-host.mak variables like FOO_CFLAGS and FOO_LIBS, which is
used as per object cflags and libs.

This removes unwanted dependencies from libcacard.

Signed-off-by: Fam Zheng <famz@redhat.com>
[Split from Fam's patch to enable modules. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/Makefile.objs | 13 ++++++++++++-
 configure           | 25 ++++++++++---------------
 2 files changed, 22 insertions(+), 16 deletions(-)

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 4e8c91e..a1db63f 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -23,4 +23,15 @@ common-obj-y += commit.o
 common-obj-y += mirror.o
 common-obj-y += backup.o
 
-$(obj)/curl.o: QEMU_CFLAGS+=$(CURL_CFLAGS)
+iscsi.o-cflags     := $(LIBISCSI_CFLAGS)
+iscsi.o-libs       := $(LIBISCSI_LIBS)
+curl.o-cflags      := $(CURL_CFLAGS)
+curl.o-libs        := $(CURL_LIBS)
+rbd.o-cflags       := $(RBD_CFLAGS)
+rbd.o-libs         := $(RBD_LIBS)
+gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
+gluster.o-libs     := $(GLUSTERFS_LIBS)
+ssh.o-cflags       := $(LIBSSH2_CFLAGS)
+ssh.o-libs         := $(LIBSSH2_LIBS)
+qcow.o-libs        := -lz
+linux-aio.o-libs   := -laio
diff --git a/configure b/configure
index afff201..bfc8636 100755
--- a/configure
+++ b/configure
@@ -2304,8 +2304,6 @@ EOF
   curl_libs=`$curlconfig --libs 2>/dev/null`
   if compile_prog "$curl_cflags" "$curl_libs" ; then
     curl=yes
-    libs_tools="$curl_libs $libs_tools"
-    libs_softmmu="$curl_libs $libs_softmmu"
   else
     if test "$curl" = "yes" ; then
       feature_not_found "curl"
@@ -2461,8 +2459,6 @@ EOF
   rbd_libs="-lrbd -lrados"
   if compile_prog "" "$rbd_libs" ; then
     rbd=yes
-    libs_tools="$rbd_libs $libs_tools"
-    libs_softmmu="$rbd_libs $libs_softmmu"
   else
     if test "$rbd" = "yes" ; then
       feature_not_found "rados block device"
@@ -2479,9 +2475,6 @@ if test "$libssh2" != "no" ; then
     libssh2_cflags=`$pkg_config libssh2 --cflags`
     libssh2_libs=`$pkg_config libssh2 --libs`
     libssh2=yes
-    libs_tools="$libssh2_libs $libs_tools"
-    libs_softmmu="$libssh2_libs $libs_softmmu"
-    QEMU_CFLAGS="$QEMU_CFLAGS $libssh2_cflags"
   else
     if test "$libssh2" = "yes" ; then
       error_exit "libssh2 >= $min_libssh2_version required for --enable-libssh2"
@@ -2527,8 +2520,6 @@ int main(void) { io_setup(0, NULL); io_set_eventfd(NULL, 0); eventfd(0, 0); retu
 EOF
   if compile_prog "" "-laio" ; then
     linux_aio=yes
-    libs_softmmu="$libs_softmmu -laio"
-    libs_tools="$libs_tools -laio"
   else
     if test "$linux_aio" = "yes" ; then
       feature_not_found "linux AIO"
@@ -2697,9 +2688,6 @@ if test "$glusterfs" != "no" ; then
     glusterfs="yes"
     glusterfs_cflags=`$pkg_config --cflags glusterfs-api`
     glusterfs_libs=`$pkg_config --libs glusterfs-api`
-    CFLAGS="$CFLAGS $glusterfs_cflags"
-    libs_tools="$glusterfs_libs $libs_tools"
-    libs_softmmu="$glusterfs_libs $libs_softmmu"
     if $pkg_config --atleast-version=5 glusterfs-api; then
       glusterfs_discard="yes"
     fi
@@ -3070,11 +3058,9 @@ EOF
     libiscsi="yes"
     libiscsi_cflags=$($pkg_config --cflags libiscsi)
     libiscsi_libs=$($pkg_config --libs libiscsi)
-    CFLAGS="$CFLAGS $libiscsi_cflags"
-    LIBS="$LIBS $libiscsi_libs"
   elif compile_prog "" "-liscsi" ; then
     libiscsi="yes"
-    LIBS="$LIBS -liscsi"
+    libiscsi_libs="-liscsi"
   else
     if test "$libiscsi" = "yes" ; then
       feature_not_found "libiscsi"
@@ -4072,6 +4058,7 @@ fi
 if test "$curl" = "yes" ; then
   echo "CONFIG_CURL=y" >> $config_host_mak
   echo "CURL_CFLAGS=$curl_cflags" >> $config_host_mak
+  echo "CURL_LIBS=$curl_libs" >> $config_host_mak
 fi
 if test "$brlapi" = "yes" ; then
   echo "CONFIG_BRLAPI=y" >> $config_host_mak
@@ -4164,6 +4151,8 @@ if test "$libiscsi" = "yes" ; then
   if test "$libiscsi_version" = "1.4.0"; then
     echo "CONFIG_LIBISCSI_1_4=y" >> $config_host_mak
   fi
+  echo "LIBISCSI_CFLAGS=$libiscsi_cflags" >> $config_host_mak
+  echo "LIBISCSI_LIBS=$libiscsi_libs" >> $config_host_mak
 fi
 
 if test "$seccomp" = "yes"; then
@@ -4185,6 +4174,8 @@ if test "$qom_cast_debug" = "yes" ; then
 fi
 if test "$rbd" = "yes" ; then
   echo "CONFIG_RBD=y" >> $config_host_mak
+  echo "RBD_CFLAGS=$rbd_cflags" >> $config_host_mak
+  echo "RBD_LIBS=$rbd_libs" >> $config_host_mak
 fi
 
 echo "CONFIG_COROUTINE_BACKEND=$coroutine" >> $config_host_mak
@@ -4228,6 +4219,8 @@ fi
 
 if test "$glusterfs" = "yes" ; then
   echo "CONFIG_GLUSTERFS=y" >> $config_host_mak
+  echo "GLUSTERFS_CFLAGS=$glusterfs_cflags" >> $config_host_mak
+  echo "GLUSTERFS_LIBS=$glusterfs_libs" >> $config_host_mak
 fi
 
 if test "$glusterfs_discard" = "yes" ; then
@@ -4240,6 +4233,8 @@ fi
 
 if test "$libssh2" = "yes" ; then
   echo "CONFIG_LIBSSH2=y" >> $config_host_mak
+  echo "LIBSSH2_CFLAGS=$libssh2_cflags" >> $config_host_mak
+  echo "LIBSSH2_LIBS=$libssh2_libs" >> $config_host_mak
 fi
 
 if test "$virtio_blk_data_plane" = "yes" ; then
-- 
1.8.5.4

[Qemu-devel] [PATCH v20 05/11] darwin: do not use -mdynamic-no-pic

[Fam Zheng]

From: Paolo Bonzini <pbonzini@redhat.com>

While -mdynamic-no-pic can speed up the code somewhat, it is only used
on the legacy PowerPC Mac OS X, and I am not sure if anyone is still
testing that.  Disabling PIC can cause problems when enabling modules,
so do not do that.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 configure | 2 --
 1 file changed, 2 deletions(-)

diff --git a/configure b/configure
index bfc8636..0e62059 100755
--- a/configure
+++ b/configure
@@ -517,8 +517,6 @@ Darwin)
   if [ "$cpu" = "x86_64" ] ; then
     QEMU_CFLAGS="-arch x86_64 $QEMU_CFLAGS"
     LDFLAGS="-arch x86_64 $LDFLAGS"
-  else
-    QEMU_CFLAGS="-mdynamic-no-pic $QEMU_CFLAGS"
   fi
   cocoa="yes"
   audio_drv_list="coreaudio"
-- 
1.8.5.4

[Qemu-devel] [PATCH v20 06/11] rules.mak: introduce DSO rules

[Fam Zheng]

Add necessary rules and flags for shared object generation.
The new rules introduced here are:

1) %.o in $(common-obj-m) is compiled to %.o, then linked to %.so.

2) %.mo in $(common-obj-m) is the placeholder for %.so for pattern
matching in Makefile. It's linked to "-shared" with all its dependencies
(multiple *.o) as input. Which means the list of depended objects must
be specified in each sub-Makefile.objs:

    foo.mo-objs := bar.o baz.o qux.o

in the same style with foo.o-cflags and foo.o-libs. The objects here
will be prefixed with "$(obj)/" if it's a subdirectory Makefile.objs.

3) For all files ending up in %.so, the following is added automatically:

    foo.o-cflags += -fPIC -DBUILD_DSO

Also introduce --enable-modules in configure, the option will enable
support of shared object build. Otherwise objects are static linked to
executables.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 .gitignore |  3 +++
 Makefile   |  4 +++-
 configure  | 15 +++++++++++++++
 rules.mak  | 52 +++++++++++++++++++++++++++++++++++++++++++---------
 4 files changed, 64 insertions(+), 10 deletions(-)

diff --git a/.gitignore b/.gitignore
index 1c9d63d..7702b0c 100644
--- a/.gitignore
+++ b/.gitignore
@@ -64,6 +64,9 @@ fsdev/virtfs-proxy-helper.pod
 *.cp
 *.dvi
 *.exe
+*.dll
+*.so
+*.mo
 *.fn
 *.ky
 *.log
diff --git a/Makefile b/Makefile
index 52b1958..9d53117 100644
--- a/Makefile
+++ b/Makefile
@@ -138,7 +138,7 @@ ifeq ($(CONFIG_SMARTCARD_NSS),y)
 include $(SRC_PATH)/libcacard/Makefile
 endif
 
-all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all
+all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all modules
 
 vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
 
@@ -256,6 +256,8 @@ clean:
 	rm -f qemu-options.def
 	find . -name '*.[oda]' -type f -exec rm -f {} +
 	find . -name '*.l[oa]' -type f -exec rm -f {} +
+	find . -name '*$(DSOSUF)' -type f -exec rm -f {} +
+	find . -name '*.mo' -type f -exec rm -f {} +
 	rm -f $(filter-out %.tlb,$(TOOLS)) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~
 	rm -f fsdev/*.pod
 	rm -rf .libs */.libs
diff --git a/configure b/configure
index 0e62059..14f4501 100755
--- a/configure
+++ b/configure
@@ -205,6 +205,9 @@ mingw32="no"
 gcov="no"
 gcov_tool="gcov"
 EXESUF=""
+DSOSUF=".so"
+LDFLAGS_SHARED="-shared"
+modules="no"
 prefix="/usr/local"
 mandir="\${prefix}/share/man"
 datadir="\${prefix}/share"
@@ -514,6 +517,7 @@ OpenBSD)
 Darwin)
   bsd="yes"
   darwin="yes"
+  LDFLAGS_SHARED="-bundle -undefined dynamic_lookup"
   if [ "$cpu" = "x86_64" ] ; then
     QEMU_CFLAGS="-arch x86_64 $QEMU_CFLAGS"
     LDFLAGS="-arch x86_64 $LDFLAGS"
@@ -607,6 +611,7 @@ fi
 
 if test "$mingw32" = "yes" ; then
   EXESUF=".exe"
+  DSOSUF=".dll"
   QEMU_CFLAGS="-DWIN32_LEAN_AND_MEAN -DWINVER=0x501 $QEMU_CFLAGS"
   # enable C99/POSIX format strings (needs mingw32-runtime 3.15 or later)
   QEMU_CFLAGS="-D__USE_MINGW_ANSI_STDIO=1 $QEMU_CFLAGS"
@@ -673,6 +678,9 @@ for opt do
   ;;
   --disable-debug-info)
   ;;
+  --enable-modules)
+      modules="yes"
+  ;;
   --cpu=*)
   ;;
   --target-list=*) target_list="$optarg"
@@ -1125,6 +1133,7 @@ Advanced options (experts only):
   --sysconfdir=PATH        install config in PATH$confsuffix
   --localstatedir=PATH     install local state in PATH (set at runtime on win32)
   --with-confsuffix=SUFFIX suffix for QEMU data inside datadir and sysconfdir [$confsuffix]
+  --enable-modules         enable modules support
   --enable-debug-tcg       enable TCG debugging
   --disable-debug-tcg      disable TCG debugging (default)
   --enable-debug-info       enable debugging information (default)
@@ -3740,6 +3749,7 @@ echo "python            $python"
 if test "$slirp" = "yes" ; then
     echo "smbd              $smbd"
 fi
+echo "module support    $modules"
 echo "host CPU          $cpu"
 echo "host big endian   $bigendian"
 echo "target list       $target_list"
@@ -3983,6 +3993,9 @@ echo "TARGET_DIRS=$target_list" >> $config_host_mak
 if [ "$docs" = "yes" ] ; then
   echo "BUILD_DOCS=yes" >> $config_host_mak
 fi
+if test "$modules" = "yes"; then
+  echo "CONFIG_MODULES=y" >> $config_host_mak
+fi
 if test "$sdl" = "yes" ; then
   echo "CONFIG_SDL=y" >> $config_host_mak
   echo "SDL_CFLAGS=$sdl_cflags" >> $config_host_mak
@@ -4361,6 +4374,8 @@ echo "LIBTOOLFLAGS=$LIBTOOLFLAGS" >> $config_host_mak
 echo "LIBS+=$LIBS" >> $config_host_mak
 echo "LIBS_TOOLS+=$libs_tools" >> $config_host_mak
 echo "EXESUF=$EXESUF" >> $config_host_mak
+echo "DSOSUF=$DSOSUF" >> $config_host_mak
+echo "LDFLAGS_SHARED=$LDFLAGS_SHARED" >> $config_host_mak
 echo "LIBS_QGA+=$libs_qga" >> $config_host_mak
 echo "POD2MAN=$POD2MAN" >> $config_host_mak
 echo "TRANSLATE_OPT_CFLAGS=$TRANSLATE_OPT_CFLAGS" >> $config_host_mak
diff --git a/rules.mak b/rules.mak
index 9398268..55db767 100644
--- a/rules.mak
+++ b/rules.mak
@@ -21,7 +21,12 @@ QEMU_DGFLAGS += -MMD -MP -MT $@ -MF $(*D)/$(*F).d
 # Same as -I$(SRC_PATH) -I., but for the nested source/object directories
 QEMU_INCLUDES += -I$(<D) -I$(@D)
 
-extract-libs = $(strip $(foreach o,$1,$($o-libs)))
+maybe-add = $(filter-out $1, $2) $1
+extract-libs = $(strip $(sort $(foreach o,$1,$($o-libs)) \
+                  $(foreach o,$(call expand-objs,$1),$($o-libs))))
+expand-objs = $(strip $(sort $(filter %.o,$1)) \
+                  $(foreach o,$(filter %.mo,$1),$($o-objs)) \
+                  $(filter-out %.o %.mo,$1))
 
 %.o: %.c
 	$(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) $($@-cflags) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")
@@ -30,8 +35,8 @@ extract-libs = $(strip $(foreach o,$1,$($o-libs)))
 
 ifeq ($(LIBTOOL),)
 LINK = $(call quiet-command,$(CC) $(QEMU_CFLAGS) $(CFLAGS) $(LDFLAGS) -o $@ \
-       $(sort $(filter %.o, $1)) $(filter-out %.o, $1) $(version-obj-y) \
-       $(call extract-libs,$^) $(LIBS),"  LINK  $(TARGET_DIR)$@")
+       $(call expand-objs,$1) $(version-obj-y) \
+       $(call extract-libs,$1) $(LIBS),"  LINK  $(TARGET_DIR)$@")
 else
 LIBTOOL += $(if $(V),,--quiet)
 %.lo: %.c
@@ -42,12 +47,12 @@ LIBTOOL += $(if $(V),,--quiet)
 	$(call quiet-command,$(LIBTOOL) --mode=compile --tag=CC dtrace -o $@ -G -s $<, " lt GEN $(TARGET_DIR)$@")
 
 LINK = $(call quiet-command,\
-       $(if $(filter %.lo %.la,$^),$(LIBTOOL) --mode=link --tag=CC \
+       $(if $(filter %.lo %.la,$1),$(LIBTOOL) --mode=link --tag=CC \
        )$(CC) $(QEMU_CFLAGS) $(CFLAGS) $(LDFLAGS) -o $@ \
-       $(sort $(filter %.o, $1)) $(filter-out %.o, $1) \
-       $(if $(filter %.lo %.la,$^),$(version-lobj-y),$(version-obj-y)) \
-       $(if $(filter %.lo %.la,$^),$(LIBTOOLFLAGS)) \
-       $(call extract-libs,$^) $(LIBS),$(if $(filter %.lo %.la,$^),"lt LINK ", "  LINK  ")"$(TARGET_DIR)$@")
+       $(call expand-objs,$1) \
+       $(if $(filter %.lo %.la,$1),$(version-lobj-y),$(version-obj-y)) \
+       $(if $(filter %.lo %.la,$1),$(LIBTOOLFLAGS)) \
+       $(call extract-libs,$1) $(LIBS),$(if $(filter %.lo %.la,$1),"lt LINK ", "  LINK  ")"$(TARGET_DIR)$@")
 endif
 
 %.asm: %.S
@@ -65,6 +70,14 @@ endif
 %.o: %.dtrace
 	$(call quiet-command,dtrace -o $@ -G -s $<, "  GEN   $(TARGET_DIR)$@")
 
+DSO_CFLAGS := -fPIC -DBUILD_DSO
+%$(DSOSUF): LDFLAGS += $(LDFLAGS_SHARED)
+%$(DSOSUF): %.mo libqemustub.a
+	$(call LINK,$^)
+
+.PHONY: modules
+modules:
+
 %$(EXESUF): %.o
 	$(call LINK,$^)
 
@@ -158,7 +171,10 @@ $(foreach v,$($1), \
 		$(eval $v-cflags := )) \
 	$(if $($v-libs), \
 		$(eval $2$v-libs := $($v-libs)) \
-		$(eval $v-libs := )))
+		$(eval $v-libs := )) \
+	$(if $($v-objs), \
+		$(eval $2$v-objs := $(addprefix $2,$($v-objs))) \
+		$(eval $v-objs := )))
 endef
 
 define unnest-dir
@@ -182,6 +198,22 @@ $(if $(nested-dirs),
   $(call unnest-vars-1))
 endef
 
+define process-modules
+$(foreach o,$(filter %.o,$($1)),
+	$(eval $(patsubst %.o,%.mo,$o): $o) \
+	$(eval $(patsubst %.o,%.mo,$o)-objs := $o))
+$(foreach o,$(filter-out $(modules-m), $(patsubst %.o,%.mo,$($1))), \
+    $(eval $o: $($o-objs))
+    $(eval modules-objs-m += $($o-objs))
+    $(eval modules-m += $o)
+    $(eval $o:; $$(call quiet-command,touch $$@,"  GEN   $$(TARGET_DIR)$$@"))
+    $(if $(CONFIG_MODULES),$(eval modules: $(patsubst %.mo,%$(DSOSUF),$o)))) \
+$(eval modules-objs-m := $(sort $(modules-objs-m)))
+$(foreach o,$(modules-objs-m), \
+    $(if $(CONFIG_MODULES),$(eval $o-cflags := $(call maybe-add, $(DSO_CFLAGS), $($o-cflags)))))
+$(eval $(patsubst %-m,%-$(call lnot,$(CONFIG_MODULES)),$1) += $($1))
+endef
+
 define unnest-vars
 $(eval obj := $1)
 $(eval nested-vars := $2)
@@ -193,4 +225,6 @@ $(foreach var,$(nested-vars),$(eval $(var) := $(filter-out %/, $($(var)))))
 $(shell mkdir -p $(sort $(foreach var,$(nested-vars),$(dir $($(var))))))
 $(foreach var,$(nested-vars), $(eval \
   -include $(addsuffix *.d, $(sort $(dir $($(var)))))))
+$(foreach v,$(filter %-m,$(nested-vars)), \
+    $(call process-modules,$v))
 endef
-- 
1.8.5.4

[Qemu-devel] [PATCH v20 07/11] module: implement module loading

[Fam Zheng]

This patch adds loading, stamp checking and initialization of modules.

The init function of dynamic module is no longer directly called as
__attribute__((constructor)) in static linked version, it is called
only after passed the checking of presense of stamp symbol:

    qemu_stamp_$RELEASEHASH

where $RELEASEHASH is generated by hashing version strings and content
of configure script.

With this, modules built from a different tree/version/configure will
not be loaded.

The module loading code requires gmodule-2.0.

Modules are searched under
 - CONFIG_MODDIR
 - executable folder (to allow running qemu-{img,io} in the build
   directory)
 - ../ of executable folder (to allow running system emulator in the
   build directory)

Modules are linked under their subdir respectively, then copied to top
level of build directory for above convinience, e.g.:
    $(BUILD_DIR)/block/curl.so -> $(BUILD_DIR)/block-curl.so

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 Makefile              |   3 ++
 configure             |  42 +++++++++++----
 include/qemu/module.h |  23 +++++++-
 module-common.c       |  10 ++++
 rules.mak             |   3 ++
 scripts/create_config |   3 ++
 util/module.c         | 145 +++++++++++++++++++++++++++++++++++++++++++++++++-
 7 files changed, 217 insertions(+), 12 deletions(-)
 create mode 100644 module-common.c

diff --git a/Makefile b/Makefile
index 9d53117..07d1ed7 100644
--- a/Makefile
+++ b/Makefile
@@ -201,6 +201,9 @@ Makefile: $(version-obj-y) $(version-lobj-y)
 libqemustub.a: $(stub-obj-y)
 libqemuutil.a: $(util-obj-y) qapi-types.o qapi-visit.o
 
+block-modules = $(foreach o,$(block-obj-m),"$(basename $(subst /,-,$o))",) NULL
+util/module.o-cflags = -D'CONFIG_BLOCK_MODULES=$(block-modules)'
+
 ######################################################################
 
 qemu-img.o: qemu-img-cmds.h
diff --git a/configure b/configure
index 14f4501..bd21db9 100755
--- a/configure
+++ b/configure
@@ -1132,7 +1132,7 @@ Advanced options (experts only):
   --libdir=PATH            install libraries in PATH
   --sysconfdir=PATH        install config in PATH$confsuffix
   --localstatedir=PATH     install local state in PATH (set at runtime on win32)
-  --with-confsuffix=SUFFIX suffix for QEMU data inside datadir and sysconfdir [$confsuffix]
+  --with-confsuffix=SUFFIX suffix for QEMU data inside datadir/libdir/sysconfdir [$confsuffix]
   --enable-modules         enable modules support
   --enable-debug-tcg       enable TCG debugging
   --disable-debug-tcg      disable TCG debugging (default)
@@ -2348,14 +2348,32 @@ if test "$mingw32" = yes; then
 else
     glib_req_ver=2.12
 fi
-if $pkg_config --atleast-version=$glib_req_ver gthread-2.0; then
-    glib_cflags=`$pkg_config --cflags gthread-2.0`
-    glib_libs=`$pkg_config --libs gthread-2.0`
-    CFLAGS="$glib_cflags $CFLAGS"
-    LIBS="$glib_libs $LIBS"
-    libs_qga="$glib_libs $libs_qga"
-else
-    error_exit "glib-$glib_req_ver required to compile QEMU"
+
+for i in gthread-2.0 gmodule-2.0; do
+    if $pkg_config --atleast-version=$glib_req_ver $i; then
+        glib_cflags=`$pkg_config --cflags $i`
+        glib_libs=`$pkg_config --libs $i`
+        CFLAGS="$glib_cflags $CFLAGS"
+        LIBS="$glib_libs $LIBS"
+        libs_qga="$glib_libs $libs_qga"
+    else
+        error_exit "glib-$glib_req_ver $i is required to compile QEMU"
+    fi
+done
+
+##########################################
+# SHA command probe for modules
+if test "$modules" = yes; then
+    shacmd_probe="sha1sum sha1 shasum"
+    for c in $shacmd_probe; do
+        if which $c &>/dev/null; then
+            shacmd="$c"
+            break
+        fi
+    done
+    if test "$shacmd" = ""; then
+        error_exit "one of the checksum commands is required to enable modules: $shacmd_probe"
+    fi
 fi
 
 ##########################################
@@ -3633,6 +3651,7 @@ if test "$mingw32" = "yes" ; then
 fi
 
 qemu_confdir=$sysconfdir$confsuffix
+qemu_moddir=$libdir$confsuffix
 qemu_datadir=$datadir$confsuffix
 qemu_localedir="$datadir/locale"
 
@@ -3723,6 +3742,7 @@ echo "Install prefix    $prefix"
 echo "BIOS directory    `eval echo $qemu_datadir`"
 echo "binary directory  `eval echo $bindir`"
 echo "library directory `eval echo $libdir`"
+echo "module directory  `eval echo $qemu_moddir`"
 echo "libexec directory `eval echo $libexecdir`"
 echo "include directory `eval echo $includedir`"
 echo "config directory  `eval echo $sysconfdir`"
@@ -3861,6 +3881,7 @@ echo "sysconfdir=$sysconfdir" >> $config_host_mak
 echo "qemu_confdir=$qemu_confdir" >> $config_host_mak
 echo "qemu_datadir=$qemu_datadir" >> $config_host_mak
 echo "qemu_docdir=$qemu_docdir" >> $config_host_mak
+echo "qemu_moddir=$qemu_moddir" >> $config_host_mak
 if test "$mingw32" = "no" ; then
   echo "qemu_localstatedir=$local_statedir" >> $config_host_mak
 fi
@@ -3994,6 +4015,9 @@ if [ "$docs" = "yes" ] ; then
   echo "BUILD_DOCS=yes" >> $config_host_mak
 fi
 if test "$modules" = "yes"; then
+  # $shacmd can generate a hash started with digit, which the compiler doesn't
+  # like as an symbol. So prefix it with an underscore
+  echo "CONFIG_STAMP=_`(echo $qemu_version; echo $pkgversion; cat $0) | $shacmd - | cut -f1 -d\ `" >> $config_host_mak
   echo "CONFIG_MODULES=y" >> $config_host_mak
 fi
 if test "$sdl" = "yes" ; then
diff --git a/include/qemu/module.h b/include/qemu/module.h
index c4ccd57..72d9498 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -14,11 +14,31 @@
 #ifndef QEMU_MODULE_H
 #define QEMU_MODULE_H
 
+#include "qemu/osdep.h"
+
+#define DSO_STAMP_FUN         glue(qemu_stamp, CONFIG_STAMP)
+#define DSO_STAMP_FUN_STR     stringify(DSO_STAMP_FUN)
+
+#ifdef BUILD_DSO
+void DSO_STAMP_FUN(void);
+/* This is a dummy symbol to identify a loaded DSO as a QEMU module, so we can
+ * distinguish "version mismatch" from "not a QEMU module", when the stamp
+ * check fails during module loading */
+void qemu_module_dummy(void);
+
+#define module_init(function, type)                                         \
+static void __attribute__((constructor)) do_qemu_init_ ## function(void)    \
+{                                                                           \
+    register_dso_module_init(function, type);                               \
+}
+#else
 /* This should not be used directly.  Use block_init etc. instead.  */
 #define module_init(function, type)                                         \
-static void __attribute__((constructor)) do_qemu_init_ ## function(void) {  \
+static void __attribute__((constructor)) do_qemu_init_ ## function(void)    \
+{                                                                           \
     register_module_init(function, type);                                   \
 }
+#endif
 
 typedef enum {
     MODULE_INIT_BLOCK,
@@ -34,6 +54,7 @@ typedef enum {
 #define type_init(function) module_init(function, MODULE_INIT_QOM)
 
 void register_module_init(void (*fn)(void), module_init_type type);
+void register_dso_module_init(void (*fn)(void), module_init_type type);
 
 void module_call_init(module_init_type type);
 
diff --git a/module-common.c b/module-common.c
new file mode 100644
index 0000000..50c6750
--- /dev/null
+++ b/module-common.c
@@ -0,0 +1,10 @@
+#include "config-host.h"
+#include "qemu/module.h"
+
+void qemu_module_dummy(void)
+{
+}
+
+void DSO_STAMP_FUN(void)
+{
+}
diff --git a/rules.mak b/rules.mak
index 55db767..a4d88e6 100644
--- a/rules.mak
+++ b/rules.mak
@@ -74,6 +74,8 @@ DSO_CFLAGS := -fPIC -DBUILD_DSO
 %$(DSOSUF): LDFLAGS += $(LDFLAGS_SHARED)
 %$(DSOSUF): %.mo libqemustub.a
 	$(call LINK,$^)
+	@# Copy to build root so modules can be loaded when program started without install
+	$(if $(findstring /,$@),$(call quiet-command,cp $@ $(subst /,-,$@), "  CP    $(subst /,-,$@)"))
 
 .PHONY: modules
 modules:
@@ -203,6 +205,7 @@ $(foreach o,$(filter %.o,$($1)),
 	$(eval $(patsubst %.o,%.mo,$o): $o) \
 	$(eval $(patsubst %.o,%.mo,$o)-objs := $o))
 $(foreach o,$(filter-out $(modules-m), $(patsubst %.o,%.mo,$($1))), \
+    $(eval $o-objs += module-common.o)
     $(eval $o: $($o-objs))
     $(eval modules-objs-m += $($o-objs))
     $(eval modules-m += $o)
diff --git a/scripts/create_config b/scripts/create_config
index 06f5316..546f889 100755
--- a/scripts/create_config
+++ b/scripts/create_config
@@ -108,6 +108,9 @@ case $line in
     value=${line#*=}
     echo "#define $name $value"
     ;;
+ DSOSUF=*)
+    echo "#define HOST_DSOSUF \"${line#*=}\""
+    ;;
 esac
 
 done # read
diff --git a/util/module.c b/util/module.c
index 7acc33d..42bc373 100644
--- a/util/module.c
+++ b/util/module.c
@@ -13,6 +13,8 @@
  * GNU GPL, version 2 or (at your option) any later version.
  */
 
+#include <stdlib.h>
+#include <gmodule.h>
 #include "qemu-common.h"
 #include "qemu/queue.h"
 #include "qemu/module.h"
@@ -21,13 +23,16 @@ typedef struct ModuleEntry
 {
     void (*init)(void);
     QTAILQ_ENTRY(ModuleEntry) node;
+    module_init_type type;
 } ModuleEntry;
 
 typedef QTAILQ_HEAD(, ModuleEntry) ModuleTypeList;
 
 static ModuleTypeList init_type_list[MODULE_INIT_MAX];
 
-static void init_types(void)
+static ModuleTypeList dso_init_list;
+
+static void init_lists(void)
 {
     static int inited;
     int i;
@@ -40,6 +45,8 @@ static void init_types(void)
         QTAILQ_INIT(&init_type_list[i]);
     }
 
+    QTAILQ_INIT(&dso_init_list);
+
     inited = 1;
 }
 
@@ -48,7 +55,7 @@ static ModuleTypeList *find_type(module_init_type type)
 {
     ModuleTypeList *l;
 
-    init_types();
+    init_lists();
 
     l = &init_type_list[type];
 
@@ -62,20 +69,154 @@ void register_module_init(void (*fn)(void), module_init_type type)
 
     e = g_malloc0(sizeof(*e));
     e->init = fn;
+    e->type = type;
 
     l = find_type(type);
 
     QTAILQ_INSERT_TAIL(l, e, node);
 }
 
+void register_dso_module_init(void (*fn)(void), module_init_type type)
+{
+    ModuleEntry *e;
+
+    init_lists();
+
+    e = g_malloc0(sizeof(*e));
+    e->init = fn;
+    e->type = type;
+
+    QTAILQ_INSERT_TAIL(&dso_init_list, e, node);
+}
+
+static void module_load(module_init_type type);
+
 void module_call_init(module_init_type type)
 {
     ModuleTypeList *l;
     ModuleEntry *e;
 
+    module_load(type);
     l = find_type(type);
 
     QTAILQ_FOREACH(e, l, node) {
         e->init();
     }
 }
+
+#ifdef CONFIG_MODULES
+static int module_load_file(const char *fname)
+{
+    GModule *g_module;
+    void (*sym)(void);
+    const char *dsosuf = HOST_DSOSUF;
+    int len = strlen(fname);
+    int suf_len = strlen(dsosuf);
+    ModuleEntry *e, *next;
+    int ret;
+
+    if (len <= suf_len || strcmp(&fname[len - suf_len], dsosuf)) {
+        /* wrong suffix */
+        ret = -EINVAL;
+        goto out;
+    }
+    if (access(fname, F_OK)) {
+        ret = -ENOENT;
+        goto out;
+    }
+
+    assert(QTAILQ_EMPTY(&dso_init_list));
+
+    g_module = g_module_open(fname, G_MODULE_BIND_LAZY | G_MODULE_BIND_LOCAL);
+    if (!g_module) {
+        fprintf(stderr, "Failed to open module: %s\n",
+                g_module_error());
+        ret = -EINVAL;
+        goto out;
+    }
+    if (!g_module_symbol(g_module, DSO_STAMP_FUN_STR, (gpointer *)&sym)) {
+        fprintf(stderr, "Failed to initialize module: %s\n",
+                fname);
+        /* Print some info if this is a QEMU module (but from different build),
+         * this will make debugging user problems easier. */
+        if (g_module_symbol(g_module, "qemu_module_dummy", (gpointer *)&sym)) {
+            fprintf(stderr,
+                    "Note: only modules from the same build can be loaded.\n");
+        }
+        g_module_close(g_module);
+        ret = -EINVAL;
+    } else {
+        QTAILQ_FOREACH(e, &dso_init_list, node) {
+            register_module_init(e->init, e->type);
+        }
+        ret = 0;
+    }
+
+    QTAILQ_FOREACH_SAFE(e, &dso_init_list, node, next) {
+        QTAILQ_REMOVE(&dso_init_list, e, node);
+        g_free(e);
+    }
+out:
+    return ret;
+}
+#endif
+
+void module_load(module_init_type type)
+{
+#ifdef CONFIG_MODULES
+    char *fname = NULL;
+    const char **mp;
+    static const char *block_modules[] = {
+        CONFIG_BLOCK_MODULES
+    };
+    char *exec_dir;
+    char *dirs[3];
+    int i = 0;
+    int ret;
+
+    if (!g_module_supported()) {
+        fprintf(stderr, "Module is not supported by system.\n");
+        return;
+    }
+
+    switch (type) {
+    case MODULE_INIT_BLOCK:
+        mp = block_modules;
+        break;
+    default:
+        /* no other types have dynamic modules for now*/
+        return;
+    }
+
+    exec_dir = qemu_get_exec_dir();
+    dirs[i++] = g_strdup_printf("%s", CONFIG_QEMU_MODDIR);
+    dirs[i++] = g_strdup_printf("%s/..", exec_dir ? : "");
+    dirs[i++] = g_strdup_printf("%s", exec_dir ? : "");
+    assert(i == ARRAY_SIZE(dirs));
+    g_free(exec_dir);
+    exec_dir = NULL;
+
+    for ( ; *mp; mp++) {
+        for (i = 0; i < ARRAY_SIZE(dirs); i++) {
+            fname = g_strdup_printf("%s/%s%s", dirs[i], *mp, HOST_DSOSUF);
+            ret = module_load_file(fname);
+            /* Try loading until loaded a module file */
+            if (!ret) {
+                break;
+            }
+            g_free(fname);
+            fname = NULL;
+        }
+        if (ret == -ENOENT) {
+            fprintf(stderr, "Can't find module: %s\n", *mp);
+        }
+
+        g_free(fname);
+    }
+
+    for (i = 0; i < ARRAY_SIZE(dirs); i++) {
+        g_free(dirs[i]);
+    }
+
+#endif
+}
-- 
1.8.5.4

[Qemu-devel] [PATCH v20 08/11] Makefile: install modules with "make install"

[Fam Zheng]

Install all the modules to ${MODDIR}.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Makefile | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/Makefile b/Makefile
index 07d1ed7..57d83a3 100644
--- a/Makefile
+++ b/Makefile
@@ -368,6 +368,12 @@ install-datadir install-localstatedir
 ifneq ($(TOOLS),)
 	$(INSTALL_PROG) $(STRIP_OPT) $(TOOLS) "$(DESTDIR)$(bindir)"
 endif
+ifneq ($(CONFIG_MODULES),)
+	$(INSTALL_DIR) "$(DESTDIR)$(qemu_moddir)"
+	for s in $(patsubst %.mo,%$(DSOSUF),$(modules-m)); do \
+		$(INSTALL_PROG) $(STRIP_OPT) $$s "$(DESTDIR)$(qemu_moddir)/$${s//\//-}"; \
+	done
+endif
 ifneq ($(HELPERS-y),)
 	$(INSTALL_DIR) "$(DESTDIR)$(libexecdir)"
 	$(INSTALL_PROG) $(STRIP_OPT) $(HELPERS-y) "$(DESTDIR)$(libexecdir)"
-- 
1.8.5.4

[Qemu-devel] [PATCH v20 09/11] Makefile: introduce common-obj-m and block-obj-m for DSO

[Fam Zheng]

$(common-obj-m) will include $(block-obj-m), like $(common-obj-y) does
for $(block-obj-y).

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Makefile        | 4 +++-
 Makefile.objs   | 2 ++
 Makefile.target | 6 +++++-
 3 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 57d83a3..992cd23 100644
--- a/Makefile
+++ b/Makefile
@@ -129,7 +129,9 @@ dummy := $(call unnest-vars,, \
                 util-obj-y \
                 qga-obj-y \
                 block-obj-y \
-                common-obj-y)
+                block-obj-m \
+                common-obj-y \
+                common-obj-m)
 
 ifneq ($(wildcard config-host.mak),)
 include $(SRC_PATH)/tests/Makefile
diff --git a/Makefile.objs b/Makefile.objs
index 1914080..4a62913 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -19,6 +19,8 @@ block-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o
 block-obj-y += qemu-coroutine-sleep.o
 block-obj-y += coroutine-$(CONFIG_COROUTINE_BACKEND).o
 
+block-obj-m = block/
+
 ifeq ($(CONFIG_VIRTIO)$(CONFIG_VIRTFS)$(CONFIG_PCI),yyy)
 # Lots of the fsdev/9pcode is pulled in by vl.c via qemu_fsdev_add.
 # only pull in the actual virtio-9p device if we also enabled virtio.
diff --git a/Makefile.target b/Makefile.target
index 9a6e7dd..3945260 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -148,7 +148,11 @@ obj-y-save := $(obj-y)
 block-obj-y :=
 common-obj-y :=
 include $(SRC_PATH)/Makefile.objs
-dummy := $(call unnest-vars,..,block-obj-y common-obj-y)
+dummy := $(call unnest-vars,.., \
+               block-obj-y \
+               block-obj-m \
+               common-obj-y \
+               common-obj-m)
 
 # Now restore obj-y
 obj-y := $(obj-y-save)
-- 
1.8.5.4

[Qemu-devel] [PATCH v20 10/11] block: convert block drivers linked with libs to modules

[Fam Zheng]

The converted block drivers are:

    curl
    iscsi
    rbd
    ssh
    glusterfs

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 configure | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/configure b/configure
index bd21db9..639b5c9 100755
--- a/configure
+++ b/configure
@@ -4091,7 +4091,7 @@ if test "$bswap_h" = "yes" ; then
   echo "CONFIG_MACHINE_BSWAP_H=y" >> $config_host_mak
 fi
 if test "$curl" = "yes" ; then
-  echo "CONFIG_CURL=y" >> $config_host_mak
+  echo "CONFIG_CURL=m" >> $config_host_mak
   echo "CURL_CFLAGS=$curl_cflags" >> $config_host_mak
   echo "CURL_LIBS=$curl_libs" >> $config_host_mak
 fi
@@ -4182,7 +4182,7 @@ if test "$glx" = "yes" ; then
 fi
 
 if test "$libiscsi" = "yes" ; then
-  echo "CONFIG_LIBISCSI=y" >> $config_host_mak
+  echo "CONFIG_LIBISCSI=m" >> $config_host_mak
   if test "$libiscsi_version" = "1.4.0"; then
     echo "CONFIG_LIBISCSI_1_4=y" >> $config_host_mak
   fi
@@ -4208,7 +4208,7 @@ if test "$qom_cast_debug" = "yes" ; then
   echo "CONFIG_QOM_CAST_DEBUG=y" >> $config_host_mak
 fi
 if test "$rbd" = "yes" ; then
-  echo "CONFIG_RBD=y" >> $config_host_mak
+  echo "CONFIG_RBD=m" >> $config_host_mak
   echo "RBD_CFLAGS=$rbd_cflags" >> $config_host_mak
   echo "RBD_LIBS=$rbd_libs" >> $config_host_mak
 fi
@@ -4253,7 +4253,7 @@ if test "$getauxval" = "yes" ; then
 fi
 
 if test "$glusterfs" = "yes" ; then
-  echo "CONFIG_GLUSTERFS=y" >> $config_host_mak
+  echo "CONFIG_GLUSTERFS=m" >> $config_host_mak
   echo "GLUSTERFS_CFLAGS=$glusterfs_cflags" >> $config_host_mak
   echo "GLUSTERFS_LIBS=$glusterfs_libs" >> $config_host_mak
 fi
@@ -4267,7 +4267,7 @@ if test "$glusterfs_zerofill" = "yes" ; then
 fi
 
 if test "$libssh2" = "yes" ; then
-  echo "CONFIG_LIBSSH2=y" >> $config_host_mak
+  echo "CONFIG_LIBSSH2=m" >> $config_host_mak
   echo "LIBSSH2_CFLAGS=$libssh2_cflags" >> $config_host_mak
   echo "LIBSSH2_LIBS=$libssh2_libs" >> $config_host_mak
 fi
-- 
1.8.5.4

[Qemu-devel] [PATCH v20 11/11] oslib: port qemu_init_exec_dir to Darwin

[Fam Zheng]

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
---
 util/oslib-posix.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/util/oslib-posix.c b/util/oslib-posix.c
index c2eeb4f..3214e44 100644
--- a/util/oslib-posix.c
+++ b/util/oslib-posix.c
@@ -59,6 +59,12 @@ extern int daemon(int, int);
 #include <sys/mman.h>
 #include <libgen.h>
 
+/* Get declaration of _NSGetExecutablePath on MacOS X 10.2 or newer.  */
+#if defined(__APPLE__) && defined(__MACH__)
+#define ENUM_DYLD_BOOL
+#include <mach-o/dyld.h>
+#endif
+
 #ifdef CONFIG_LINUX
 #include <sys/syscall.h>
 #endif
@@ -307,6 +313,15 @@ void qemu_init_exec_dir(const char *argv0)
             p = buf;
         }
     }
+#elif defined(__APPLE__) && defined(__MACH__)
+    {
+        char result[PATH_MAX];
+        uint32_t length = PATH_MAX;
+        if (_NSGetExecutablePath(result, &length) != 0 || result[0] != '/') {
+            return;
+        }
+        p = realpath(result, buf);
+    }
 #endif
     /* If we don't have any way of figuring out the actual executable
        location then try argv[0].  */
-- 
1.8.5.4

Re: [Qemu-devel] [PATCH v20 00/11] Shared library module support

[Fam Zheng]

Ping?

Thanks,
Fam

On Mon, 02/10 14:48, Fam Zheng wrote:
> Rewrote the executable directory patch and added Darwin API in
> qemu_init_exec_dir().
> 
> v20:
>     Dropped the argv0 passing patch from v19. Refactored qemu_get_exec_dir() in
>     patch 01. Three patches are affected:
> 
>     [01/11] util: Split out exec_dir from os_find_datadir
>             Rewritten. The exec_dir is a static global in os-{win32,posix}.c
>             that is initialized in main() and later used with
>             qemu_get_exec_dir().
> 
>     [07/11] module: implement module loading
>             Use glue(). Prefix hash with "_", so the preceding digit won't
>             cause compile error now.
> 
>     [11/11] oslib: port qemu_init_exec_dir to Darwin
>             New.
> 
> 
> 
> Fam Zheng (10):
>   util: Split out exec_dir from os_find_datadir
>   rules.mak: fix $(obj) to a real relative path
>   rules.mak: allow per object cflags and libs
>   block: use per-object cflags and libs
>   rules.mak: introduce DSO rules
>   module: implement module loading
>   Makefile: install modules with "make install"
>   Makefile: introduce common-obj-m and block-obj-m for DSO
>   block: convert block drivers linked with libs to modules
>   oslib: port qemu_init_exec_dir to Darwin
> 
> Paolo Bonzini (1):
>   darwin: do not use -mdynamic-no-pic
> 
>  .gitignore            |   3 ++
>  Makefile              |  29 +++++++++-
>  Makefile.objs         |  19 ++-----
>  Makefile.target       |  21 ++++++--
>  block/Makefile.objs   |  13 ++++-
>  configure             |  93 +++++++++++++++++++++-----------
>  include/qemu-common.h |   2 +-
>  include/qemu/module.h |  23 +++++++-
>  include/qemu/osdep.h  |   9 ++++
>  module-common.c       |  10 ++++
>  os-posix.c            |  42 +++------------
>  os-win32.c            |  21 +-------
>  qemu-img.c            |   1 +
>  qemu-io.c             |   1 +
>  qemu-nbd.c            |   1 +
>  rules.mak             |  80 +++++++++++++++++++++++-----
>  scripts/create_config |   3 ++
>  util/module.c         | 145 +++++++++++++++++++++++++++++++++++++++++++++++++-
>  util/oslib-posix.c    |  69 ++++++++++++++++++++++++
>  util/oslib-win32.c    |  30 +++++++++++
>  vl.c                  |   3 +-
>  21 files changed, 494 insertions(+), 124 deletions(-)
>  create mode 100644 module-common.c
> 
> -- 
> 1.8.5.4
> 
> 

Re: [Qemu-devel] [PATCH v20 00/11] Shared library module support

[Paolo Bonzini]

Il 17/02/2014 14:16, Fam Zheng ha scritto:
> Ping?

I'll send a pull request for patches 1-10 later.

Paolo

Re: [Qemu-devel] [PATCH v20 00/11] Shared library module support

[Fam Zheng]

On Mon, 02/17 14:34, Paolo Bonzini wrote:
> Il 17/02/2014 14:16, Fam Zheng ha scritto:
> >Ping?
> 
> I'll send a pull request for patches 1-10 later.
> 

Bravo, thanks!

Fam