[Qemu-devel] [RFC PATCH v3 0/5] Shared Library Module Support

[Qemu-devel] [RFC PATCH v3 0/5] Shared Library Module Support

[Fam Zheng]

This series implements feature of shared object building as described in:

http://wiki.qemu.org/Features/Modules

It's achieved in three steps, with extra bonus to change curl and qed to shared
library modules in the end (only to demonstrate the usage, no "make install"
support of .so files yet).

v3: Support a new consistent syntax for per object cflags, libs, and module
object list:

    foo.o-cflags := $(FOO_CFLAGS)
    foo.o-libs   := $(FOO_LIBS)
    bar.mo-objs  := biz.o qux.o

All without $(obj) now, as they are automa[gt]ically prefixed when unnesting
variables in top Makefile (foo.o should be listed in some nested var to expand
these variables correctly).

    [01/05] make.rule: fix $(obj) to a real relative path
            [Paolo] a different and cleaner fix as suggested, please review
            again.
    [02/05] rule.mak: allow per object cflags and libs
            Cleaner syntax as described above, see patch 05 for example.
    [03/05] Makefile: introduce common-obj-m and block-obj-m for DSO
            Based on the object variable syntax.
    [04/05] module: implement module loading function
            [Paolo] Use g_strdup_printf and G_MODULE_BIND_LOCAL. Remove unused
            #include's.
    [05/05] block: build qed and curl as shared library

Fam Zheng (5):
  make.rule: fix $(obj) to a real relative path
  rule.mak: allow per object cflags and libs
  Makefile: introduce common-obj-m and block-obj-m for DSO
  module: implement module loading function
  block: build qed and curl as shared library

 Makefile              | 42 ++++++++++++++++++++++++++++++++++++++--
 Makefile.objs         | 19 +++++-------------
 Makefile.target       | 10 ++++++++--
 block.c               |  1 +
 block/Makefile.objs   |  7 ++++---
 bsd-user/main.c       |  3 +++
 configure             | 32 ++++++++++++++++++++-----------
 include/qemu/module.h |  9 +++++++++
 linux-user/main.c     |  3 +++
 rules.mak             | 41 +++++++++++++++++++++++++++++++++------
 scripts/create_config |  4 ++++
 tests/Makefile        |  5 +++++
 util/module.c         | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++
 vl.c                  |  2 ++
 14 files changed, 193 insertions(+), 38 deletions(-)

-- 
1.8.3.1

[Qemu-devel] [RFC PATCH v3 1/5] make.rule: fix $(obj) to a real relative path

[Fam Zheng]

Makefile.target includes rule.mak and unnested common-obj-y, then prefix
them with '../', this will ignore object specific QEMU_CFLAGS in subdir
Makefile.objs:

    $(obj)/curl.o: QEMU_CFLAGS += $(CURL_CFLAGS)

Because $(obj) here is './block', instead of '../block'. This doesn't
hurt compiling because we basically build all .o from top Makefile,
before entering Makefile.target, but it will affact arriving per-object
libs support.

The starting point of $(obj) is fixed in $(obj-base) before including
./Makefile.objs, to get consistency with nested Makefile rules in target
rule and variable definition.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 Makefile        | 18 +++++++++++++++++-
 Makefile.objs   | 14 --------------
 Makefile.target |  6 ++++--
 configure       |  1 +
 rules.mak       |  8 +++++---
 tests/Makefile  |  4 ++++
 6 files changed, 31 insertions(+), 20 deletions(-)

diff --git a/Makefile b/Makefile
index 806946e..ef80f0d 100644
--- a/Makefile
+++ b/Makefile
@@ -115,14 +115,30 @@ defconfig:
 
 ifneq ($(wildcard config-host.mak),)
 include $(SRC_PATH)/Makefile.objs
-include $(SRC_PATH)/tests/Makefile
 endif
 ifeq ($(CONFIG_SMARTCARD_NSS),y)
 include $(SRC_PATH)/libcacard/Makefile
 endif
 
+nested-vars += \
+	stub-obj-y \
+	util-obj-y \
+	qga-obj-y \
+	block-obj-y \
+	common-obj-y
+
+dummy := $(call unnest-vars)
+
+ifneq ($(wildcard config-host.mak),)
+include $(SRC_PATH)/tests/Makefile
+endif
+
 all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all
 
+vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
+
+vl.o: QEMU_CFLAGS+=$(SDL_CFLAGS)
+
 config-host.h: config-host.h-timestamp
 config-host.h-timestamp: config-host.mak
 qemu-options.def: $(SRC_PATH)/qemu-options.hx
diff --git a/Makefile.objs b/Makefile.objs
index f46a4cd..efd5b0f 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -109,17 +109,3 @@ version-lobj-$(CONFIG_WIN32) += $(BUILD_DIR)/version.lo
 # FIXME: a few definitions from qapi-types.o/qapi-visit.o are needed
 # by libqemuutil.a.  These should be moved to a separate .json schema.
 qga-obj-y = qga/ qapi-types.o qapi-visit.o
-
-vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
-
-vl.o: QEMU_CFLAGS+=$(SDL_CFLAGS)
-
-QEMU_CFLAGS+=$(GLIB_CFLAGS)
-
-nested-vars += \
-	stub-obj-y \
-	util-obj-y \
-	qga-obj-y \
-	block-obj-y \
-	common-obj-y
-dummy := $(call unnest-vars)
diff --git a/Makefile.target b/Makefile.target
index 9a49852..381022d 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -143,13 +143,15 @@ endif # CONFIG_SOFTMMU
 # Workaround for http://gcc.gnu.org/PR55489, see configure.
 %/translate.o: QEMU_CFLAGS += $(TRANSLATE_OPT_CFLAGS)
 
-nested-vars += obj-y
+nested-vars += obj-y block-obj-y common-obj-y
 
 # This resolves all nested paths, so it must come last
 include $(SRC_PATH)/Makefile.objs
+obj-base := ..
+dummy := $(call unnest-vars)
 
 all-obj-y = $(obj-y)
-all-obj-y += $(addprefix ../, $(common-obj-y))
+all-obj-y += $(addprefix ../, $(common-obj-y) $(block-obj-y))
 
 ifndef CONFIG_HAIKU
 LIBS+=-lm
diff --git a/configure b/configure
index e989609..cc3cd4d 100755
--- a/configure
+++ b/configure
@@ -2251,6 +2251,7 @@ fi
 if $pkg_config --atleast-version=$glib_req_ver gthread-2.0; then
     glib_cflags=`$pkg_config --cflags gthread-2.0`
     glib_libs=`$pkg_config --libs gthread-2.0`
+    CFLAGS="$glib_cflags $CFLAGS"
     LIBS="$glib_libs $LIBS"
     libs_qga="$glib_libs $libs_qga"
 else
diff --git a/rules.mak b/rules.mak
index 4499745..3ff7d7a 100644
--- a/rules.mak
+++ b/rules.mak
@@ -103,7 +103,7 @@ clean: clean-timestamp
 
 # magic to descend into other directories
 
-obj := .
+obj = $(obj-base)
 old-nested-dirs :=
 
 define push-var
@@ -119,9 +119,11 @@ endef
 
 define unnest-dir
 $(foreach var,$(nested-vars),$(call push-var,$(var),$1/))
-$(eval obj := $(obj)/$1)
+$(eval obj-parent-$1 := $(obj))
+$(eval obj := $(if $(obj),$(obj)/$1,$1))
 $(eval include $(SRC_PATH)/$1/Makefile.objs)
-$(eval obj := $(patsubst %/$1,%,$(obj)))
+$(eval obj := $(obj-parent-$1))
+$(eval obj-parent-$1 := )
 $(foreach var,$(nested-vars),$(call pop-var,$(var),$1/))
 endef
 
diff --git a/tests/Makefile b/tests/Makefile
index baba9e9..15ef039 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -110,6 +110,10 @@ test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o
 $(test-obj-y): QEMU_INCLUDES += -Itests
 QEMU_CFLAGS += -I$(SRC_PATH)/tests
 
+nested-vars := block-obj-y
+obj-base := ..
+dummy := $(call unnest-vars)
+
 tests/test-x86-cpuid.o: QEMU_INCLUDES += -I$(SRC_PATH)/target-i386
 
 tests/check-qint$(EXESUF): tests/check-qint.o libqemuutil.a
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH v3 2/5] rule.mak: allow per object cflags and libs

[Fam Zheng]

Adds extract-libs in LINK to expand any "per object libs", the syntax to define
such a libs options is like:

        foo.o-libs := $(CURL_LIBS)

in block/Makefile.objs.

Similarly,

        foo.o-cflags := $(FOO_CFLAGS)

is also supported.

"foo.o" must be listed a nested var (e.g. common-obj-y) to make the
option variables effective.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 rules.mak | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/rules.mak b/rules.mak
index 3ff7d7a..84ed998 100644
--- a/rules.mak
+++ b/rules.mak
@@ -17,15 +17,17 @@ QEMU_DGFLAGS += -MMD -MP -MT $@ -MF $(*D)/$(*F).d
 # Same as -I$(SRC_PATH) -I., but for the nested source/object directories
 QEMU_INCLUDES += -I$(<D) -I$(@D)
 
+extract-libs = $(strip $(foreach o,$1,$($o-libs)))
+
 %.o: %.c
-	$(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")
+	$(call quiet-command,$(CC) $(QEMU_INCLUDES) $(QEMU_CFLAGS) $(QEMU_DGFLAGS) $(CFLAGS) $($@-cflags) -c -o $@ $<,"  CC    $(TARGET_DIR)$@")
 %.o: %.rc
 	$(call quiet-command,$(WINDRES) -I. -o $@ $<,"  RC    $(TARGET_DIR)$@")
 
 ifeq ($(LIBTOOL),)
 LINK = $(call quiet-command,$(CC) $(QEMU_CFLAGS) $(CFLAGS) $(LDFLAGS) -o $@ \
        $(sort $(filter %.o, $1)) $(filter-out %.o, $1) $(version-obj-y) \
-       $(LIBS),"  LINK  $(TARGET_DIR)$@")
+       $(call extract-libs,$^) $(LIBS),"  LINK  $(TARGET_DIR)$@")
 else
 LIBTOOL += $(if $(V),,--quiet)
 %.lo: %.c
@@ -41,7 +43,7 @@ LINK = $(call quiet-command,\
        $(sort $(filter %.o, $1)) $(filter-out %.o, $1) \
        $(if $(filter %.lo %.la,$^),$(version-lobj-y),$(version-obj-y)) \
        $(if $(filter %.lo %.la,$^),$(LIBTOOLFLAGS)) \
-       $(LIBS),$(if $(filter %.lo %.la,$^),"lt LINK ", "  LINK  ")"$(TARGET_DIR)$@")
+       $(call extract-libs,$^) $(LIBS),$(if $(filter %.lo %.la,$^),"lt LINK ", "  LINK  ")"$(TARGET_DIR)$@")
 endif
 
 %.asm: %.S
@@ -117,11 +119,22 @@ $(eval $1 = $(value save-$2-$1) $$(subdir-$2-$1))
 $(eval save-$2-$1 :=)
 endef
 
+define fix-obj-vars
+$(foreach v,$($1), \
+	$(if $($v-cflags), \
+		$(eval $2$v-cflags := $($v-cflags)) \
+		$(eval $v-cflags := )) \
+	$(if $($v-libs), \
+		$(eval $2$v-libs := $($v-libs)) \
+		$(eval $v-libs := )))
+endef
+
 define unnest-dir
 $(foreach var,$(nested-vars),$(call push-var,$(var),$1/))
 $(eval obj-parent-$1 := $(obj))
 $(eval obj := $(if $(obj),$(obj)/$1,$1))
 $(eval include $(SRC_PATH)/$1/Makefile.objs)
+$(foreach v,$(nested-vars),$(call fix-obj-vars,$v,$(if $(obj),$(obj)/)))
 $(eval obj := $(obj-parent-$1))
 $(eval obj-parent-$1 := )
 $(foreach var,$(nested-vars),$(call pop-var,$(var),$1/))
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH v3 3/5] Makefile: introduce common-obj-m and block-obj-m for DSO

[Fam Zheng]

Add necessary rules and flags for shared object generation.
$(common-obj-m) will include $(block-obj-m), like $(common-obj-y) does
for $(block-obj-y). The new rules introduced here are:

0) For all %.so compiling:

    QEMU_CFLAGS += -fPIC

1) %.o in $(common-obj-m) is compiled to %.o, then linked to %.so.

2) %.mo in $(common-obj-m) is the placeholder for %.so for pattern
matching in Makefile. It's linked to "-shared" with all its dependencies
(multiple *.o) as input. Which means the list of depended objects must
be ruled out in each sub-Makefile.objs with an variable:

    foo.mo-obj := bar.o baz.o qux.o

in the same style with foo.o-cflags and foo.o-libs.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 Makefile        | 26 ++++++++++++++++++++++++--
 Makefile.objs   |  5 +++++
 Makefile.target |  4 ++++
 configure       |  6 ++++++
 rules.mak       | 16 +++++++++++++++-
 tests/Makefile  |  1 +
 6 files changed, 55 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index ef80f0d..658c089 100644
--- a/Makefile
+++ b/Makefile
@@ -125,7 +125,9 @@ nested-vars += \
 	util-obj-y \
 	qga-obj-y \
 	block-obj-y \
-	common-obj-y
+	block-obj-m \
+	common-obj-y \
+	common-obj-m
 
 dummy := $(call unnest-vars)
 
@@ -133,7 +135,24 @@ ifneq ($(wildcard config-host.mak),)
 include $(SRC_PATH)/tests/Makefile
 endif
 
-all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all
+# static linked mods are expanded to .o list
+dummy := $(call expand-mod-obj,common-obj-y)
+dummy := $(call expand-mod-obj,block-obj-y)
+
+modules-m = $(patsubst %.o,%$(DSOSUF),$(filter %.o,$(block-obj-m) $(common-obj-m))) \
+            $(patsubst %.mo,%$(DSOSUF),$(filter %.mo,$(block-obj-m) $(common-obj-m)))
+
+all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all $(modules-m)
+
+# Generate rules for single file modules (%.so: %.o).
+$(foreach o,$(filter %.o,$(block-obj-m) $(common-obj-m)),$(eval \
+	$(patsubst %.o,%.so,$o): $o))
+
+# For multi file modules, dependencies should be listed explicitly in
+# Makefile.objs as
+#     foo.mo-objs := bar.o biz.o
+$(foreach o,$(filter %.mo,$(block-obj-m) $(common-obj-m)),$(eval \
+	$(patsubst %.mo,%.so,$o): $($o-objs)))
 
 vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
 
@@ -251,6 +270,9 @@ clean:
 	rm -f qemu-options.def
 	find . -name '*.[oda]' -type f -exec rm -f {} +
 	find . -name '*.l[oa]' -type f -exec rm -f {} +
+	find . -name '*.so' -type f -exec rm -f {} +
+	find . -name '*.dll' -type f -exec rm -f {} +
+
 	rm -f $(TOOLS) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~
 	rm -Rf .libs
 	rm -f qemu-img-cmds.h
diff --git a/Makefile.objs b/Makefile.objs
index efd5b0f..abf59e6 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -19,6 +19,8 @@ block-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o
 block-obj-y += qemu-coroutine-sleep.o
 block-obj-y += coroutine-$(CONFIG_COROUTINE_BACKEND).o
 
+block-obj-m = block/
+
 ifeq ($(CONFIG_VIRTIO)$(CONFIG_VIRTFS)$(CONFIG_PCI),yyy)
 # Lots of the fsdev/9pcode is pulled in by vl.c via qemu_fsdev_add.
 # only pull in the actual virtio-9p device if we also enabled virtio.
@@ -83,6 +85,9 @@ common-obj-$(CONFIG_SMARTCARD_NSS) += $(libcacard-y)
 
 common-obj-y += qmp-marshal.o
 common-obj-y += qmp.o hmp.o
+
+common-obj-m = $(block-obj-m)
+
 endif
 
 ######################################################################
diff --git a/Makefile.target b/Makefile.target
index 381022d..8d70560 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -150,6 +150,10 @@ include $(SRC_PATH)/Makefile.objs
 obj-base := ..
 dummy := $(call unnest-vars)
 
+# static linked mods are expanded to .o list
+dummy := $(call expand-mod-obj,common-obj-y)
+dummy := $(call expand-mod-obj,block-obj-y)
+
 all-obj-y = $(obj-y)
 all-obj-y += $(addprefix ../, $(common-obj-y) $(block-obj-y))
 
diff --git a/configure b/configure
index cc3cd4d..c6d4a62 100755
--- a/configure
+++ b/configure
@@ -190,6 +190,8 @@ mingw32="no"
 gcov="no"
 gcov_tool="gcov"
 EXESUF=""
+DSOSUF=".so"
+LDFLAGS_SHARED="-shared"
 prefix="/usr/local"
 mandir="\${prefix}/share/man"
 datadir="\${prefix}/share"
@@ -485,6 +487,7 @@ OpenBSD)
 Darwin)
   bsd="yes"
   darwin="yes"
+  LDFLAGS_SHARED="-bundle"
   if [ "$cpu" = "x86_64" ] ; then
     QEMU_CFLAGS="-arch x86_64 $QEMU_CFLAGS"
     LDFLAGS="-arch x86_64 $LDFLAGS"
@@ -584,6 +587,7 @@ fi
 
 if test "$mingw32" = "yes" ; then
   EXESUF=".exe"
+  DSOSUF=".dll"
   QEMU_CFLAGS="-DWIN32_LEAN_AND_MEAN -DWINVER=0x501 $QEMU_CFLAGS"
   # enable C99/POSIX format strings (needs mingw32-runtime 3.15 or later)
   QEMU_CFLAGS="-D__USE_MINGW_ANSI_STDIO=1 $QEMU_CFLAGS"
@@ -4175,6 +4179,8 @@ echo "LIBTOOLFLAGS=$LIBTOOLFLAGS" >> $config_host_mak
 echo "LIBS+=$LIBS" >> $config_host_mak
 echo "LIBS_TOOLS+=$libs_tools" >> $config_host_mak
 echo "EXESUF=$EXESUF" >> $config_host_mak
+echo "DSOSUF=$DSOSUF" >> $config_host_mak
+echo "LDFLAGS_SHARED=$LDFLAGS_SHARED" >> $config_host_mak
 echo "LIBS_QGA+=$libs_qga" >> $config_host_mak
 echo "POD2MAN=$POD2MAN" >> $config_host_mak
 echo "TRANSLATE_OPT_CFLAGS=$TRANSLATE_OPT_CFLAGS" >> $config_host_mak
diff --git a/rules.mak b/rules.mak
index 84ed998..b88ac75 100644
--- a/rules.mak
+++ b/rules.mak
@@ -58,6 +58,10 @@ endif
 %.o: %.dtrace
 	$(call quiet-command,dtrace -o $@ -G -s $<, "  GEN   $(TARGET_DIR)$@")
 
+%$(DSOSUF): QEMU_CFLAGS += -fPIC
+%$(DSOSUF):
+	$(call quiet-command,$(LD) $^ -o $@ $(LDFLAGS_SHARED),"  LD[M] $(TARGET_DIR)$@")
+
 %$(EXESUF): %.o
 	$(call LINK,$^)
 
@@ -126,7 +130,11 @@ $(foreach v,$($1), \
 		$(eval $v-cflags := )) \
 	$(if $($v-libs), \
 		$(eval $2$v-libs := $($v-libs)) \
-		$(eval $v-libs := )))
+		$(eval $v-libs := )) \
+	$(if $($v-objs), \
+		$(eval $2$v-objs := $(addprefix $2,$($v-objs))) \
+		$(eval $v-objs := )) \
+)
 endef
 
 define unnest-dir
@@ -157,3 +165,9 @@ $(shell mkdir -p $(sort $(foreach var,$(nested-vars),$(dir $($(var))))))
 $(foreach var,$(nested-vars), $(eval \
   -include $(addsuffix *.d, $(sort $(dir $($(var)))))))
 endef
+
+define expand-mod-obj
+$(eval pref = $(if $(obj-base),$(obj-base)/,))
+$(eval t = $(foreach o,$($1),$(if $($(pref)$o-obj),$($(pref)$o-obj),$o)))
+$(eval $1 = $t)
+endef
diff --git a/tests/Makefile b/tests/Makefile
index 15ef039..0dda538 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -113,6 +113,7 @@ QEMU_CFLAGS += -I$(SRC_PATH)/tests
 nested-vars := block-obj-y
 obj-base := ..
 dummy := $(call unnest-vars)
+dummy := $(call expand-mod-obj,block-obj-y)
 
 tests/test-x86-cpuid.o: QEMU_INCLUDES += -I$(SRC_PATH)/target-i386
 
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH v3 4/5] module: implement module loading function

[Fam Zheng]

Added three types of modules:

    typedef enum {
        MODULE_LOAD_BLOCK = 0,
        MODULE_LOAD_UI,
        MODULE_LOAD_NET,
        MODULE_LOAD_MAX,
    } module_load_type;

and their loading function:

    void module_load(module_load_type).

which loads all ".so" files in a subdir under "${PREFIX}/qemu/", e.g.
"/usr/lib/qemu/block". Modules of each type should be loaded before
respective subsystem initialization code.

Requires gmodule-2.0 from glib.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block.c               |  1 +
 bsd-user/main.c       |  3 +++
 configure             | 22 ++++++++++++---------
 include/qemu/module.h |  9 +++++++++
 linux-user/main.c     |  3 +++
 scripts/create_config |  4 ++++
 util/module.c         | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++
 vl.c                  |  2 ++
 8 files changed, 88 insertions(+), 9 deletions(-)

diff --git a/block.c b/block.c
index 26639e8..16ceaaf 100644
--- a/block.c
+++ b/block.c
@@ -4008,6 +4008,7 @@ BlockDriverAIOCB *bdrv_aio_discard(BlockDriverState *bs,
 
 void bdrv_init(void)
 {
+    module_load(MODULE_LOAD_BLOCK);
     module_call_init(MODULE_INIT_BLOCK);
 }
 
diff --git a/bsd-user/main.c b/bsd-user/main.c
index f9246aa..6cb9e35 100644
--- a/bsd-user/main.c
+++ b/bsd-user/main.c
@@ -33,6 +33,7 @@
 #include "tcg.h"
 #include "qemu/timer.h"
 #include "qemu/envlist.h"
+#include "qemu/module.h"
 
 int singlestep;
 #if defined(CONFIG_USE_GUEST_BASE)
@@ -749,6 +750,8 @@ int main(int argc, char **argv)
     if (argc <= 1)
         usage();
 
+    module_load(MODULE_LOAD_UI);
+    module_load(MODULE_LOAD_NET);
     module_call_init(MODULE_INIT_QOM);
 
     if ((envlist = envlist_create()) == NULL) {
diff --git a/configure b/configure
index c6d4a62..a2858c2 100755
--- a/configure
+++ b/configure
@@ -2252,15 +2252,19 @@ if test "$mingw32" = yes; then
 else
     glib_req_ver=2.12
 fi
-if $pkg_config --atleast-version=$glib_req_ver gthread-2.0; then
-    glib_cflags=`$pkg_config --cflags gthread-2.0`
-    glib_libs=`$pkg_config --libs gthread-2.0`
-    CFLAGS="$glib_cflags $CFLAGS"
-    LIBS="$glib_libs $LIBS"
-    libs_qga="$glib_libs $libs_qga"
-else
-    error_exit "glib-$glib_req_ver required to compile QEMU"
-fi
+
+for i in gthread-2.0 gmodule-2.0; do
+    if $pkg_config --atleast-version=$glib_req_ver $i; then
+        glib_cflags=`$pkg_config --cflags $i`
+        glib_libs=`$pkg_config --libs $i`
+        CFLAGS="$glib_cflags $CFLAGS"
+        LIBS="$glib_libs $LIBS"
+        libs_qga="$glib_libs $libs_qga"
+    else
+        error_exit "glib-$glib_req_ver required to compile QEMU"
+    fi
+done
+
 
 ##########################################
 # pixman support probe
diff --git a/include/qemu/module.h b/include/qemu/module.h
index c4ccd57..f00bc25 100644
--- a/include/qemu/module.h
+++ b/include/qemu/module.h
@@ -37,4 +37,13 @@ void register_module_init(void (*fn)(void), module_init_type type);
 
 void module_call_init(module_init_type type);
 
+typedef enum {
+    MODULE_LOAD_BLOCK = 0,
+    MODULE_LOAD_UI,
+    MODULE_LOAD_NET,
+    MODULE_LOAD_MAX,
+} module_load_type;
+
+void module_load(module_load_type type);
+
 #endif
diff --git a/linux-user/main.c b/linux-user/main.c
index 5c2f7b2..db08c23 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -34,6 +34,7 @@
 #include "qemu/timer.h"
 #include "qemu/envlist.h"
 #include "elf.h"
+#include <qemu/module.h>
 
 char *exec_path;
 
@@ -3551,6 +3552,8 @@ int main(int argc, char **argv, char **envp)
     int i;
     int ret;
 
+    module_load(MODULE_LOAD_UI);
+    module_load(MODULE_LOAD_NET);
     module_call_init(MODULE_INIT_QOM);
 
     qemu_cache_utils_init(envp);
diff --git a/scripts/create_config b/scripts/create_config
index b1adbf5..7a54f2d 100755
--- a/scripts/create_config
+++ b/scripts/create_config
@@ -25,6 +25,7 @@ case $line in
  prefix=*)
     # save for the next definitions
     prefix=${line#*=}
+    echo "#define CONFIG_PREFIX \"$prefix\""
     ;;
  CONFIG_AUDIO_DRIVERS=*)
     drivers=${line#*=}
@@ -104,6 +105,9 @@ case $line in
     value=${line#*=}
     echo "#define $name $value"
     ;;
+ DSOSUF=*)
+    echo "#define HOST_DSOSUF \"${line#*=}\""
+    ;;
 esac
 
 done # read
diff --git a/util/module.c b/util/module.c
index 7acc33d..ef75f8e 100644
--- a/util/module.c
+++ b/util/module.c
@@ -13,6 +13,8 @@
  * GNU GPL, version 2 or (at your option) any later version.
  */
 
+#include <gmodule.h>
+#include <dirent.h>
 #include "qemu-common.h"
 #include "qemu/queue.h"
 #include "qemu/module.h"
@@ -79,3 +81,54 @@ void module_call_init(module_init_type type)
         e->init();
     }
 }
+
+void module_load(module_load_type type)
+{
+    const char *path;
+    const char *dsosuf = HOST_DSOSUF;
+    char *fname;
+    int suf_len = strlen(dsosuf);
+    DIR *dp;
+    struct dirent *ep = NULL;
+    GModule *g_module;
+
+    if (!g_module_supported()) {
+        return;
+    }
+
+    switch (type) {
+    case MODULE_LOAD_BLOCK:
+        path = CONFIG_PREFIX "/qemu/block/";
+        break;
+    case MODULE_LOAD_UI:
+        path = CONFIG_PREFIX "/qemu/ui/";
+        break;
+    case MODULE_LOAD_NET:
+        path = CONFIG_PREFIX "/qemu/net/";
+        break;
+    default:
+        return;
+    }
+
+    dp = opendir(path);
+    if (!dp) {
+        fprintf(stderr, "Failed to open dir %s\n", path);
+        return;
+    }
+    for (ep = readdir(dp); ep; ep = readdir(dp)) {
+        int len = strlen(ep->d_name);
+        if (len > suf_len &&
+                !strcmp(&ep->d_name[len - suf_len], dsosuf)) {
+            fname = g_strdup_printf("%s%s", path, ep->d_name);
+            g_module = g_module_open(fname,
+                                     G_MODULE_BIND_LAZY | G_MODULE_BIND_LOCAL);
+            if (!g_module) {
+                fprintf(stderr, "Failed to open module file %s\n",
+                        g_module_error());
+                g_free(fname);
+                continue;
+            }
+            g_free(fname);
+        }
+    }
+}
diff --git a/vl.c b/vl.c
index b4b119a..659e53a 100644
--- a/vl.c
+++ b/vl.c
@@ -2940,6 +2940,8 @@ int main(int argc, char **argv, char **envp)
 #endif
     }
 
+    module_load(MODULE_LOAD_UI);
+    module_load(MODULE_LOAD_NET);
     module_call_init(MODULE_INIT_QOM);
 
     qemu_add_opts(&qemu_drive_opts);
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH v3 5/5] block: build qed and curl as shared library

[Fam Zheng]

Curl and qed block drivers are built as shared object module.  We have
per object cflags and libs support now, move CURL_CFLAGS and CURL_LIBS
from global option variables to a per object basis.

"make install" is not installing them yet, manually copy it to
${prefix}/qemu/block/ to make it loaded.

Signed-off-by: Fam Zheng <famz@redhat.com>
---
 block/Makefile.objs | 7 ++++---
 configure           | 5 ++---
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/block/Makefile.objs b/block/Makefile.objs
index 3bb85b5..741b92f 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -1,7 +1,6 @@
 block-obj-y += raw_bsd.o cow.o qcow.o vdi.o vmdk.o cloop.o dmg.o bochs.o vpc.o vvfat.o
 block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o
-block-obj-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
-block-obj-y += qed-check.o
+block-obj-m += qed.mo
 block-obj-y += vhdx.o
 block-obj-y += parallels.o blkdebug.o blkverify.o
 block-obj-y += snapshot.o qapi.o
@@ -23,4 +22,6 @@ common-obj-y += commit.o
 common-obj-y += mirror.o
 common-obj-y += backup.o
 
-$(obj)/curl.o: QEMU_CFLAGS+=$(CURL_CFLAGS)
+curl.o-cflags := $(CURL_CFLAGS)
+curl.o-libs := $(CURL_LIBS)
+qed.mo-objs := qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o qed-check.o
diff --git a/configure b/configure
index a2858c2..136ac1a 100755
--- a/configure
+++ b/configure
@@ -2213,8 +2213,6 @@ EOF
   curl_libs=`$curlconfig --libs 2>/dev/null`
   if compile_prog "$curl_cflags" "$curl_libs" ; then
     curl=yes
-    libs_tools="$curl_libs $libs_tools"
-    libs_softmmu="$curl_libs $libs_softmmu"
   else
     if test "$curl" = "yes" ; then
       feature_not_found "curl"
@@ -3893,8 +3891,9 @@ if test "$bswap_h" = "yes" ; then
   echo "CONFIG_MACHINE_BSWAP_H=y" >> $config_host_mak
 fi
 if test "$curl" = "yes" ; then
-  echo "CONFIG_CURL=y" >> $config_host_mak
+  echo "CONFIG_CURL=m" >> $config_host_mak
   echo "CURL_CFLAGS=$curl_cflags" >> $config_host_mak
+  echo "CURL_LIBS=$curl_libs" >> $config_host_mak
 fi
 if test "$brlapi" = "yes" ; then
   echo "CONFIG_BRLAPI=y" >> $config_host_mak
-- 
1.8.3.1

Re: [Qemu-devel] [RFC PATCH v3 1/5] make.rule: fix $(obj) to a real relative path

[Paolo Bonzini]

Il 10/09/2013 03:02, Fam Zheng ha scritto:
> Makefile.target includes rule.mak and unnested common-obj-y, then prefix
> them with '../', this will ignore object specific QEMU_CFLAGS in subdir
> Makefile.objs:
> 
>     $(obj)/curl.o: QEMU_CFLAGS += $(CURL_CFLAGS)
> 
> Because $(obj) here is './block', instead of '../block'. This doesn't
> hurt compiling because we basically build all .o from top Makefile,
> before entering Makefile.target, but it will affact arriving per-object
> libs support.
> 
> The starting point of $(obj) is fixed in $(obj-base) before including
> ./Makefile.objs, to get consistency with nested Makefile rules in target
> rule and variable definition.
> 
> Signed-off-by: Fam Zheng <famz@redhat.com>

Glad this worked. )

> -dummy := $(call unnest-vars)
> diff --git a/Makefile.target b/Makefile.target
> index 9a49852..381022d 100644
> --- a/Makefile.target
> +++ b/Makefile.target
> @@ -143,13 +143,15 @@ endif # CONFIG_SOFTMMU
>  # Workaround for http://gcc.gnu.org/PR55489, see configure.
>  %/translate.o: QEMU_CFLAGS += $(TRANSLATE_OPT_CFLAGS)
>  
> -nested-vars += obj-y
> +nested-vars += obj-y block-obj-y common-obj-y
>  
>  # This resolves all nested paths, so it must come last

This comment is now obsolete.

>  include $(SRC_PATH)/Makefile.objs
> +obj-base := ..

Strictly speaking obj-base=.. is incorrect for obj-y, isn't it?  Does it
work if you do

block-obj-y = ../
common-obj-y = ../

instead of including $(SRC_PATH)/Makefile.objs? Then obj-base can be
empty and is not needed.

But the patch looks good already, I'm asking mostly to get a better
understanding of the system.

Paolo

> +dummy := $(call unnest-vars)
>  
>  all-obj-y = $(obj-y)
> -all-obj-y += $(addprefix ../, $(common-obj-y))
> +all-obj-y += $(addprefix ../, $(common-obj-y) $(block-obj-y))
>  
>  ifndef CONFIG_HAIKU
>  LIBS+=-lm
> diff --git a/configure b/configure
> index e989609..cc3cd4d 100755
> --- a/configure
> +++ b/configure
> @@ -2251,6 +2251,7 @@ fi
>  if $pkg_config --atleast-version=$glib_req_ver gthread-2.0; then
>      glib_cflags=`$pkg_config --cflags gthread-2.0`
>      glib_libs=`$pkg_config --libs gthread-2.0`
> +    CFLAGS="$glib_cflags $CFLAGS"
>      LIBS="$glib_libs $LIBS"
>      libs_qga="$glib_libs $libs_qga"
>  else
> diff --git a/rules.mak b/rules.mak
> index 4499745..3ff7d7a 100644
> --- a/rules.mak
> +++ b/rules.mak
> @@ -103,7 +103,7 @@ clean: clean-timestamp
>  
>  # magic to descend into other directories
>  
> -obj := .
> +obj = $(obj-base)
>  old-nested-dirs :=
>  
>  define push-var
> @@ -119,9 +119,11 @@ endef
>  
>  define unnest-dir
>  $(foreach var,$(nested-vars),$(call push-var,$(var),$1/))
> -$(eval obj := $(obj)/$1)
> +$(eval obj-parent-$1 := $(obj))
> +$(eval obj := $(if $(obj),$(obj)/$1,$1))
>  $(eval include $(SRC_PATH)/$1/Makefile.objs)
> -$(eval obj := $(patsubst %/$1,%,$(obj)))
> +$(eval obj := $(obj-parent-$1))
> +$(eval obj-parent-$1 := )
>  $(foreach var,$(nested-vars),$(call pop-var,$(var),$1/))
>  endef
>  
> diff --git a/tests/Makefile b/tests/Makefile
> index baba9e9..15ef039 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -110,6 +110,10 @@ test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o
>  $(test-obj-y): QEMU_INCLUDES += -Itests
>  QEMU_CFLAGS += -I$(SRC_PATH)/tests
>  
> +nested-vars := block-obj-y
> +obj-base := ..
> +dummy := $(call unnest-vars)
> +
>  tests/test-x86-cpuid.o: QEMU_INCLUDES += -I$(SRC_PATH)/target-i386
>  
>  tests/check-qint$(EXESUF): tests/check-qint.o libqemuutil.a
> 

Re: [Qemu-devel] [RFC PATCH v3 3/5] Makefile: introduce common-obj-m and block-obj-m for DSO

[Paolo Bonzini]

Il 10/09/2013 03:02, Fam Zheng ha scritto:
> -all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all
> +# static linked mods are expanded to .o list
> +dummy := $(call expand-mod-obj,common-obj-y)
> +dummy := $(call expand-mod-obj,block-obj-y)
> +
> +modules-m = $(patsubst %.o,%$(DSOSUF),$(filter %.o,$(block-obj-m) $(common-obj-m))) \
> +            $(patsubst %.mo,%$(DSOSUF),$(filter %.mo,$(block-obj-m) $(common-obj-m)))
> +
> +all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all $(modules-m)
> +
> +# Generate rules for single file modules (%.so: %.o).
> +$(foreach o,$(filter %.o,$(block-obj-m) $(common-obj-m)),$(eval \
> +	$(patsubst %.o,%.so,$o): $o))
> +
> +# For multi file modules, dependencies should be listed explicitly in
> +# Makefile.objs as
> +#     foo.mo-objs := bar.o biz.o
> +$(foreach o,$(filter %.mo,$(block-obj-m) $(common-obj-m)),$(eval \
> +	$(patsubst %.mo,%.so,$o): $($o-objs)))

I agree that this foo.mo-objs variable is homogeneous with how you
handle libraries and cflags.  I like it now.

However, I don't like the many places in which you have to special-case
modules (expand-mod-obj, modules-m, etc.), and the duplication between
Makefile and Makefile.target.

I would prefer if you try doing this patch along the lines I suggested
in my review of v2, using .mo files as a placeholder and then doing the
final link either into the .so or in the executable.  This should remove
the need for at least expand-mod-obj, and probably for more of the
duplicated constructs you have.

In particular, I would like modules-m to be simply "$(block-obj-m)
$(common-obj-m)".

In the medium term, we need to find a way to avoid the duplication:

     block-obj-y = block/
     block-obj-m = block/

Perhaps by introducing a "dirs" variable that automatically triggers
recursion on all nested variables.  But this can be the topic of a
separate patch series, if you prefer.

Paolo

>  vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
>  
> @@ -251,6 +270,9 @@ clean:
>  	rm -f qemu-options.def
>  	find . -name '*.[oda]' -type f -exec rm -f {} +
>  	find . -name '*.l[oa]' -type f -exec rm -f {} +
> +	find . -name '*.so' -type f -exec rm -f {} +
> +	find . -name '*.dll' -type f -exec rm -f {} +
> +
>  	rm -f $(TOOLS) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~
>  	rm -Rf .libs
>  	rm -f qemu-img-cmds.h
> diff --git a/Makefile.objs b/Makefile.objs
> index efd5b0f..abf59e6 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> @@ -19,6 +19,8 @@ block-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o
>  block-obj-y += qemu-coroutine-sleep.o
>  block-obj-y += coroutine-$(CONFIG_COROUTINE_BACKEND).o
>  
> +block-obj-m = block/
> +
>  ifeq ($(CONFIG_VIRTIO)$(CONFIG_VIRTFS)$(CONFIG_PCI),yyy)
>  # Lots of the fsdev/9pcode is pulled in by vl.c via qemu_fsdev_add.
>  # only pull in the actual virtio-9p device if we also enabled virtio.
> @@ -83,6 +85,9 @@ common-obj-$(CONFIG_SMARTCARD_NSS) += $(libcacard-y)
>  
>  common-obj-y += qmp-marshal.o
>  common-obj-y += qmp.o hmp.o
> +
> +common-obj-m = $(block-obj-m)
> +
>  endif
>  
>  ######################################################################
> diff --git a/Makefile.target b/Makefile.target
> index 381022d..8d70560 100644
> --- a/Makefile.target
> +++ b/Makefile.target
> @@ -150,6 +150,10 @@ include $(SRC_PATH)/Makefile.objs
>  obj-base := ..
>  dummy := $(call unnest-vars)
>  
> +# static linked mods are expanded to .o list
> +dummy := $(call expand-mod-obj,common-obj-y)
> +dummy := $(call expand-mod-obj,block-obj-y)
> +
>  all-obj-y = $(obj-y)
>  all-obj-y += $(addprefix ../, $(common-obj-y) $(block-obj-y))
>  
> diff --git a/configure b/configure
> index cc3cd4d..c6d4a62 100755
> --- a/configure
> +++ b/configure
> @@ -190,6 +190,8 @@ mingw32="no"
>  gcov="no"
>  gcov_tool="gcov"
>  EXESUF=""
> +DSOSUF=".so"
> +LDFLAGS_SHARED="-shared"
>  prefix="/usr/local"
>  mandir="\${prefix}/share/man"
>  datadir="\${prefix}/share"
> @@ -485,6 +487,7 @@ OpenBSD)
>  Darwin)
>    bsd="yes"
>    darwin="yes"
> +  LDFLAGS_SHARED="-bundle"
>    if [ "$cpu" = "x86_64" ] ; then
>      QEMU_CFLAGS="-arch x86_64 $QEMU_CFLAGS"
>      LDFLAGS="-arch x86_64 $LDFLAGS"
> @@ -584,6 +587,7 @@ fi
>  
>  if test "$mingw32" = "yes" ; then
>    EXESUF=".exe"
> +  DSOSUF=".dll"
>    QEMU_CFLAGS="-DWIN32_LEAN_AND_MEAN -DWINVER=0x501 $QEMU_CFLAGS"
>    # enable C99/POSIX format strings (needs mingw32-runtime 3.15 or later)
>    QEMU_CFLAGS="-D__USE_MINGW_ANSI_STDIO=1 $QEMU_CFLAGS"
> @@ -4175,6 +4179,8 @@ echo "LIBTOOLFLAGS=$LIBTOOLFLAGS" >> $config_host_mak
>  echo "LIBS+=$LIBS" >> $config_host_mak
>  echo "LIBS_TOOLS+=$libs_tools" >> $config_host_mak
>  echo "EXESUF=$EXESUF" >> $config_host_mak
> +echo "DSOSUF=$DSOSUF" >> $config_host_mak
> +echo "LDFLAGS_SHARED=$LDFLAGS_SHARED" >> $config_host_mak
>  echo "LIBS_QGA+=$libs_qga" >> $config_host_mak
>  echo "POD2MAN=$POD2MAN" >> $config_host_mak
>  echo "TRANSLATE_OPT_CFLAGS=$TRANSLATE_OPT_CFLAGS" >> $config_host_mak
> diff --git a/rules.mak b/rules.mak
> index 84ed998..b88ac75 100644
> --- a/rules.mak
> +++ b/rules.mak
> @@ -58,6 +58,10 @@ endif
>  %.o: %.dtrace
>  	$(call quiet-command,dtrace -o $@ -G -s $<, "  GEN   $(TARGET_DIR)$@")
>  
> +%$(DSOSUF): QEMU_CFLAGS += -fPIC
> +%$(DSOSUF):
> +	$(call quiet-command,$(LD) $^ -o $@ $(LDFLAGS_SHARED),"  LD[M] $(TARGET_DIR)$@")
> +
>  %$(EXESUF): %.o
>  	$(call LINK,$^)
>  
> @@ -126,7 +130,11 @@ $(foreach v,$($1), \
>  		$(eval $v-cflags := )) \
>  	$(if $($v-libs), \
>  		$(eval $2$v-libs := $($v-libs)) \
> -		$(eval $v-libs := )))
> +		$(eval $v-libs := )) \
> +	$(if $($v-objs), \
> +		$(eval $2$v-objs := $(addprefix $2,$($v-objs))) \
> +		$(eval $v-objs := )) \
> +)
>  endef
>  
>  define unnest-dir
> @@ -157,3 +165,9 @@ $(shell mkdir -p $(sort $(foreach var,$(nested-vars),$(dir $($(var))))))
>  $(foreach var,$(nested-vars), $(eval \
>    -include $(addsuffix *.d, $(sort $(dir $($(var)))))))
>  endef
> +
> +define expand-mod-obj
> +$(eval pref = $(if $(obj-base),$(obj-base)/,))
> +$(eval t = $(foreach o,$($1),$(if $($(pref)$o-obj),$($(pref)$o-obj),$o)))
> +$(eval $1 = $t)
> +endef
> diff --git a/tests/Makefile b/tests/Makefile
> index 15ef039..0dda538 100644
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -113,6 +113,7 @@ QEMU_CFLAGS += -I$(SRC_PATH)/tests
>  nested-vars := block-obj-y
>  obj-base := ..
>  dummy := $(call unnest-vars)
> +dummy := $(call expand-mod-obj,block-obj-y)
>  
>  tests/test-x86-cpuid.o: QEMU_INCLUDES += -I$(SRC_PATH)/target-i386
>  
> 

Re: [Qemu-devel] [RFC PATCH v3 5/5] block: build qed and curl as shared library

[Paolo Bonzini]

Il 10/09/2013 03:02, Fam Zheng ha scritto:
> Curl and qed block drivers are built as shared object module.  We have
> per object cflags and libs support now, move CURL_CFLAGS and CURL_LIBS
> from global option variables to a per object basis.
> 
> "make install" is not installing them yet, manually copy it to
> ${prefix}/qemu/block/ to make it loaded.
> 
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
>  block/Makefile.objs | 7 ++++---
>  configure           | 5 ++---
>  2 files changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/block/Makefile.objs b/block/Makefile.objs
> index 3bb85b5..741b92f 100644
> --- a/block/Makefile.objs
> +++ b/block/Makefile.objs
> @@ -1,7 +1,6 @@
>  block-obj-y += raw_bsd.o cow.o qcow.o vdi.o vmdk.o cloop.o dmg.o bochs.o vpc.o vvfat.o
>  block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o
> -block-obj-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
> -block-obj-y += qed-check.o
> +block-obj-m += qed.mo
>  block-obj-y += vhdx.o
>  block-obj-y += parallels.o blkdebug.o blkverify.o
>  block-obj-y += snapshot.o qapi.o
> @@ -23,4 +22,6 @@ common-obj-y += commit.o
>  common-obj-y += mirror.o
>  common-obj-y += backup.o
>  
> -$(obj)/curl.o: QEMU_CFLAGS+=$(CURL_CFLAGS)
> +curl.o-cflags := $(CURL_CFLAGS)
> +curl.o-libs := $(CURL_LIBS)
> +qed.mo-objs := qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o qed-check.o
> diff --git a/configure b/configure
> index a2858c2..136ac1a 100755
> --- a/configure
> +++ b/configure
> @@ -2213,8 +2213,6 @@ EOF
>    curl_libs=`$curlconfig --libs 2>/dev/null`
>    if compile_prog "$curl_cflags" "$curl_libs" ; then
>      curl=yes
> -    libs_tools="$curl_libs $libs_tools"
> -    libs_softmmu="$curl_libs $libs_softmmu"
>    else
>      if test "$curl" = "yes" ; then
>        feature_not_found "curl"
> @@ -3893,8 +3891,9 @@ if test "$bswap_h" = "yes" ; then
>    echo "CONFIG_MACHINE_BSWAP_H=y" >> $config_host_mak
>  fi
>  if test "$curl" = "yes" ; then
> -  echo "CONFIG_CURL=y" >> $config_host_mak
> +  echo "CONFIG_CURL=m" >> $config_host_mak
>    echo "CURL_CFLAGS=$curl_cflags" >> $config_host_mak
> +  echo "CURL_LIBS=$curl_libs" >> $config_host_mak
>  fi
>  if test "$brlapi" = "yes" ; then
>    echo "CONFIG_BRLAPI=y" >> $config_host_mak
> 

Looks good.  For v4, however, please introduce --enable-modules so we
get closer to the final picture.  The implementation can be as simple as

if ($(CONFIG_MODULES),y)
modules-m = $(block-obj-m) $(common-obj-m)
else
block-obj-y += $(block-obj-m)
common-obj-y += $(common-obj-m)
block-obj-m =
common-obj-m
endif

Paolo

Re: [Qemu-devel] [RFC PATCH v3 1/5] make.rule: fix $(obj) to a real relative path

[Fam Zheng]

On Tue, 09/10 08:34, Paolo Bonzini wrote:
> Il 10/09/2013 03:02, Fam Zheng ha scritto:
> > Makefile.target includes rule.mak and unnested common-obj-y, then prefix
> > them with '../', this will ignore object specific QEMU_CFLAGS in subdir
> > Makefile.objs:
> > 
> >     $(obj)/curl.o: QEMU_CFLAGS += $(CURL_CFLAGS)
> > 
> > Because $(obj) here is './block', instead of '../block'. This doesn't
> > hurt compiling because we basically build all .o from top Makefile,
> > before entering Makefile.target, but it will affact arriving per-object
> > libs support.
> > 
> > The starting point of $(obj) is fixed in $(obj-base) before including
> > ./Makefile.objs, to get consistency with nested Makefile rules in target
> > rule and variable definition.
> > 
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> 
> Glad this worked. )
> 
> > -dummy := $(call unnest-vars)
> > diff --git a/Makefile.target b/Makefile.target
> > index 9a49852..381022d 100644
> > --- a/Makefile.target
> > +++ b/Makefile.target
> > @@ -143,13 +143,15 @@ endif # CONFIG_SOFTMMU
> >  # Workaround for http://gcc.gnu.org/PR55489, see configure.
> >  %/translate.o: QEMU_CFLAGS += $(TRANSLATE_OPT_CFLAGS)
> >  
> > -nested-vars += obj-y
> > +nested-vars += obj-y block-obj-y common-obj-y
> >  
> >  # This resolves all nested paths, so it must come last
> 
> This comment is now obsolete.
> 
> >  include $(SRC_PATH)/Makefile.objs
> > +obj-base := ..
> 
> Strictly speaking obj-base=.. is incorrect for obj-y, isn't it?  Does it

You are right, I'd do it respectively, since...

> work if you do
> 
> block-obj-y = ../
> common-obj-y = ../
> 
> instead of including $(SRC_PATH)/Makefile.objs? Then obj-base can be
> empty and is not needed.
> 
> But the patch looks good already, I'm asking mostly to get a better
> understanding of the system.
> 
it's not working.

The base point of unnesting is fixed to SRC_PATH:

    define unnest-dir
    ...
    $(eval include $(SRC_PATH)/$1/Makefile.objs)
    ...
    endef

So it can't unnest  "../". block-obj-y = ./ makes a little more sense, but
$(obj) can be messed, again.

Do you think two calls of unnest-vars OK?

    nested-vars = obj-y
    dummy := $(call unnest-vars)
    include $(SRC_PATH)/Makefile.objs
    obj-base := ..
    nested-vars = block-obj-y common-obj-y
    dummy := $(call unnest-vars)

> > +dummy := $(call unnest-vars)
> >  
> >  all-obj-y = $(obj-y)
> > -all-obj-y += $(addprefix ../, $(common-obj-y))
> > +all-obj-y += $(addprefix ../, $(common-obj-y) $(block-obj-y))
> >  
> >  ifndef CONFIG_HAIKU
> >  LIBS+=-lm
> > diff --git a/configure b/configure
> > index e989609..cc3cd4d 100755
> > --- a/configure
> > +++ b/configure
> > @@ -2251,6 +2251,7 @@ fi
> >  if $pkg_config --atleast-version=$glib_req_ver gthread-2.0; then
> >      glib_cflags=`$pkg_config --cflags gthread-2.0`
> >      glib_libs=`$pkg_config --libs gthread-2.0`
> > +    CFLAGS="$glib_cflags $CFLAGS"
> >      LIBS="$glib_libs $LIBS"
> >      libs_qga="$glib_libs $libs_qga"
> >  else
> > diff --git a/rules.mak b/rules.mak
> > index 4499745..3ff7d7a 100644
> > --- a/rules.mak
> > +++ b/rules.mak
> > @@ -103,7 +103,7 @@ clean: clean-timestamp
> >  
> >  # magic to descend into other directories
> >  
> > -obj := .
> > +obj = $(obj-base)
> >  old-nested-dirs :=
> >  
> >  define push-var
> > @@ -119,9 +119,11 @@ endef
> >  
> >  define unnest-dir
> >  $(foreach var,$(nested-vars),$(call push-var,$(var),$1/))
> > -$(eval obj := $(obj)/$1)
> > +$(eval obj-parent-$1 := $(obj))
> > +$(eval obj := $(if $(obj),$(obj)/$1,$1))
> >  $(eval include $(SRC_PATH)/$1/Makefile.objs)
> > -$(eval obj := $(patsubst %/$1,%,$(obj)))
> > +$(eval obj := $(obj-parent-$1))
> > +$(eval obj-parent-$1 := )
> >  $(foreach var,$(nested-vars),$(call pop-var,$(var),$1/))
> >  endef
> >  
> > diff --git a/tests/Makefile b/tests/Makefile
> > index baba9e9..15ef039 100644
> > --- a/tests/Makefile
> > +++ b/tests/Makefile
> > @@ -110,6 +110,10 @@ test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o
> >  $(test-obj-y): QEMU_INCLUDES += -Itests
> >  QEMU_CFLAGS += -I$(SRC_PATH)/tests
> >  
> > +nested-vars := block-obj-y
> > +obj-base := ..
> > +dummy := $(call unnest-vars)
> > +
> >  tests/test-x86-cpuid.o: QEMU_INCLUDES += -I$(SRC_PATH)/target-i386
> >  
> >  tests/check-qint$(EXESUF): tests/check-qint.o libqemuutil.a
> > 
> 

Re: [Qemu-devel] [RFC PATCH v3 1/5] make.rule: fix $(obj) to a real relative path

[Paolo Bonzini]

Il 10/09/2013 09:16, Fam Zheng ha scritto:
> The base point of unnesting is fixed to SRC_PATH:
> 
>     define unnest-dir
>     ...
>     $(eval include $(SRC_PATH)/$1/Makefile.objs)
>     ...
>     endef
> 
> So it can't unnest  "../". block-obj-y = ./ makes a little more sense, but
> $(obj) can be messed, again.

Right.  I was confusing the object directory with the source directory.

> Do you think two calls of unnest-vars OK?
> 
>     nested-vars = obj-y
>     dummy := $(call unnest-vars)
>     include $(SRC_PATH)/Makefile.objs
>     obj-base := ..
>     nested-vars = block-obj-y common-obj-y
>     dummy := $(call unnest-vars)

Yeah, that's fine.

Perhaps nested-vars and obj-base could become arguments to unnest-vars, too.

Paolo

>>> +dummy := $(call unnest-vars)
>>>  
>>>  all-obj-y = $(obj-y)
>>> -all-obj-y += $(addprefix ../, $(common-obj-y))
>>> +all-obj-y += $(addprefix ../, $(common-obj-y) $(block-obj-y))
>>>  
>>>  ifndef CONFIG_HAIKU
>>>  LIBS+=-lm
>>> diff --git a/configure b/configure
>>> index e989609..cc3cd4d 100755
>>> --- a/configure
>>> +++ b/configure
>>> @@ -2251,6 +2251,7 @@ fi
>>>  if $pkg_config --atleast-version=$glib_req_ver gthread-2.0; then
>>>      glib_cflags=`$pkg_config --cflags gthread-2.0`
>>>      glib_libs=`$pkg_config --libs gthread-2.0`
>>> +    CFLAGS="$glib_cflags $CFLAGS"
>>>      LIBS="$glib_libs $LIBS"
>>>      libs_qga="$glib_libs $libs_qga"
>>>  else
>>> diff --git a/rules.mak b/rules.mak
>>> index 4499745..3ff7d7a 100644
>>> --- a/rules.mak
>>> +++ b/rules.mak
>>> @@ -103,7 +103,7 @@ clean: clean-timestamp
>>>  
>>>  # magic to descend into other directories
>>>  
>>> -obj := .
>>> +obj = $(obj-base)
>>>  old-nested-dirs :=
>>>  
>>>  define push-var
>>> @@ -119,9 +119,11 @@ endef
>>>  
>>>  define unnest-dir
>>>  $(foreach var,$(nested-vars),$(call push-var,$(var),$1/))
>>> -$(eval obj := $(obj)/$1)
>>> +$(eval obj-parent-$1 := $(obj))
>>> +$(eval obj := $(if $(obj),$(obj)/$1,$1))
>>>  $(eval include $(SRC_PATH)/$1/Makefile.objs)
>>> -$(eval obj := $(patsubst %/$1,%,$(obj)))
>>> +$(eval obj := $(obj-parent-$1))
>>> +$(eval obj-parent-$1 := )
>>>  $(foreach var,$(nested-vars),$(call pop-var,$(var),$1/))
>>>  endef
>>>  
>>> diff --git a/tests/Makefile b/tests/Makefile
>>> index baba9e9..15ef039 100644
>>> --- a/tests/Makefile
>>> +++ b/tests/Makefile
>>> @@ -110,6 +110,10 @@ test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o
>>>  $(test-obj-y): QEMU_INCLUDES += -Itests
>>>  QEMU_CFLAGS += -I$(SRC_PATH)/tests
>>>  
>>> +nested-vars := block-obj-y
>>> +obj-base := ..
>>> +dummy := $(call unnest-vars)
>>> +
>>>  tests/test-x86-cpuid.o: QEMU_INCLUDES += -I$(SRC_PATH)/target-i386
>>>  
>>>  tests/check-qint$(EXESUF): tests/check-qint.o libqemuutil.a
>>>
>>
> 
> 

Re: [Qemu-devel] [RFC PATCH v3 3/5] Makefile: introduce common-obj-m and block-obj-m for DSO

[Fam Zheng]

On Tue, 09/10 08:45, Paolo Bonzini wrote:
> Il 10/09/2013 03:02, Fam Zheng ha scritto:
> > -all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all
> > +# static linked mods are expanded to .o list
> > +dummy := $(call expand-mod-obj,common-obj-y)
> > +dummy := $(call expand-mod-obj,block-obj-y)
> > +
> > +modules-m = $(patsubst %.o,%$(DSOSUF),$(filter %.o,$(block-obj-m) $(common-obj-m))) \
> > +            $(patsubst %.mo,%$(DSOSUF),$(filter %.mo,$(block-obj-m) $(common-obj-m)))
> > +
> > +all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all $(modules-m)
> > +
> > +# Generate rules for single file modules (%.so: %.o).
> > +$(foreach o,$(filter %.o,$(block-obj-m) $(common-obj-m)),$(eval \
> > +	$(patsubst %.o,%.so,$o): $o))
> > +
> > +# For multi file modules, dependencies should be listed explicitly in
> > +# Makefile.objs as
> > +#     foo.mo-objs := bar.o biz.o
> > +$(foreach o,$(filter %.mo,$(block-obj-m) $(common-obj-m)),$(eval \
> > +	$(patsubst %.mo,%.so,$o): $($o-objs)))
> 
> I agree that this foo.mo-objs variable is homogeneous with how you
> handle libraries and cflags.  I like it now.
> 
> However, I don't like the many places in which you have to special-case
> modules (expand-mod-obj, modules-m, etc.), and the duplication between
> Makefile and Makefile.target.
> 
> I would prefer if you try doing this patch along the lines I suggested
> in my review of v2, using .mo files as a placeholder and then doing the
> final link either into the .so or in the executable.  This should remove
> the need for at least expand-mod-obj, and probably for more of the
> duplicated constructs you have.
> 
OK. I'll try.

> In particular, I would like modules-m to be simply "$(block-obj-m)
> $(common-obj-m)".

There need to be some variable with %.o and %.mo subst to %.so, to become
dependency of target "all".

> 
> In the medium term, we need to find a way to avoid the duplication:
> 
>      block-obj-y = block/
>      block-obj-m = block/
> 
> Perhaps by introducing a "dirs" variable that automatically triggers
> recursion on all nested variables.  But this can be the topic of a
> separate patch series, if you prefer.
> 
Agree but prefer to do it in a separate series.

Thanks,

Fam

> >  vl.o: QEMU_CFLAGS+=$(GPROF_CFLAGS)
> >  
> > @@ -251,6 +270,9 @@ clean:
> >  	rm -f qemu-options.def
> >  	find . -name '*.[oda]' -type f -exec rm -f {} +
> >  	find . -name '*.l[oa]' -type f -exec rm -f {} +
> > +	find . -name '*.so' -type f -exec rm -f {} +
> > +	find . -name '*.dll' -type f -exec rm -f {} +
> > +
> >  	rm -f $(TOOLS) $(HELPERS-y) qemu-ga TAGS cscope.* *.pod *~ */*~
> >  	rm -Rf .libs
> >  	rm -f qemu-img-cmds.h
> > diff --git a/Makefile.objs b/Makefile.objs
> > index efd5b0f..abf59e6 100644
> > --- a/Makefile.objs
> > +++ b/Makefile.objs
> > @@ -19,6 +19,8 @@ block-obj-y += qemu-coroutine.o qemu-coroutine-lock.o qemu-coroutine-io.o
> >  block-obj-y += qemu-coroutine-sleep.o
> >  block-obj-y += coroutine-$(CONFIG_COROUTINE_BACKEND).o
> >  
> > +block-obj-m = block/
> > +
> >  ifeq ($(CONFIG_VIRTIO)$(CONFIG_VIRTFS)$(CONFIG_PCI),yyy)
> >  # Lots of the fsdev/9pcode is pulled in by vl.c via qemu_fsdev_add.
> >  # only pull in the actual virtio-9p device if we also enabled virtio.
> > @@ -83,6 +85,9 @@ common-obj-$(CONFIG_SMARTCARD_NSS) += $(libcacard-y)
> >  
> >  common-obj-y += qmp-marshal.o
> >  common-obj-y += qmp.o hmp.o
> > +
> > +common-obj-m = $(block-obj-m)
> > +
> >  endif
> >  
> >  ######################################################################
> > diff --git a/Makefile.target b/Makefile.target
> > index 381022d..8d70560 100644
> > --- a/Makefile.target
> > +++ b/Makefile.target
> > @@ -150,6 +150,10 @@ include $(SRC_PATH)/Makefile.objs
> >  obj-base := ..
> >  dummy := $(call unnest-vars)
> >  
> > +# static linked mods are expanded to .o list
> > +dummy := $(call expand-mod-obj,common-obj-y)
> > +dummy := $(call expand-mod-obj,block-obj-y)
> > +
> >  all-obj-y = $(obj-y)
> >  all-obj-y += $(addprefix ../, $(common-obj-y) $(block-obj-y))
> >  
> > diff --git a/configure b/configure
> > index cc3cd4d..c6d4a62 100755
> > --- a/configure
> > +++ b/configure
> > @@ -190,6 +190,8 @@ mingw32="no"
> >  gcov="no"
> >  gcov_tool="gcov"
> >  EXESUF=""
> > +DSOSUF=".so"
> > +LDFLAGS_SHARED="-shared"
> >  prefix="/usr/local"
> >  mandir="\${prefix}/share/man"
> >  datadir="\${prefix}/share"
> > @@ -485,6 +487,7 @@ OpenBSD)
> >  Darwin)
> >    bsd="yes"
> >    darwin="yes"
> > +  LDFLAGS_SHARED="-bundle"
> >    if [ "$cpu" = "x86_64" ] ; then
> >      QEMU_CFLAGS="-arch x86_64 $QEMU_CFLAGS"
> >      LDFLAGS="-arch x86_64 $LDFLAGS"
> > @@ -584,6 +587,7 @@ fi
> >  
> >  if test "$mingw32" = "yes" ; then
> >    EXESUF=".exe"
> > +  DSOSUF=".dll"
> >    QEMU_CFLAGS="-DWIN32_LEAN_AND_MEAN -DWINVER=0x501 $QEMU_CFLAGS"
> >    # enable C99/POSIX format strings (needs mingw32-runtime 3.15 or later)
> >    QEMU_CFLAGS="-D__USE_MINGW_ANSI_STDIO=1 $QEMU_CFLAGS"
> > @@ -4175,6 +4179,8 @@ echo "LIBTOOLFLAGS=$LIBTOOLFLAGS" >> $config_host_mak
> >  echo "LIBS+=$LIBS" >> $config_host_mak
> >  echo "LIBS_TOOLS+=$libs_tools" >> $config_host_mak
> >  echo "EXESUF=$EXESUF" >> $config_host_mak
> > +echo "DSOSUF=$DSOSUF" >> $config_host_mak
> > +echo "LDFLAGS_SHARED=$LDFLAGS_SHARED" >> $config_host_mak
> >  echo "LIBS_QGA+=$libs_qga" >> $config_host_mak
> >  echo "POD2MAN=$POD2MAN" >> $config_host_mak
> >  echo "TRANSLATE_OPT_CFLAGS=$TRANSLATE_OPT_CFLAGS" >> $config_host_mak
> > diff --git a/rules.mak b/rules.mak
> > index 84ed998..b88ac75 100644
> > --- a/rules.mak
> > +++ b/rules.mak
> > @@ -58,6 +58,10 @@ endif
> >  %.o: %.dtrace
> >  	$(call quiet-command,dtrace -o $@ -G -s $<, "  GEN   $(TARGET_DIR)$@")
> >  
> > +%$(DSOSUF): QEMU_CFLAGS += -fPIC
> > +%$(DSOSUF):
> > +	$(call quiet-command,$(LD) $^ -o $@ $(LDFLAGS_SHARED),"  LD[M] $(TARGET_DIR)$@")
> > +
> >  %$(EXESUF): %.o
> >  	$(call LINK,$^)
> >  
> > @@ -126,7 +130,11 @@ $(foreach v,$($1), \
> >  		$(eval $v-cflags := )) \
> >  	$(if $($v-libs), \
> >  		$(eval $2$v-libs := $($v-libs)) \
> > -		$(eval $v-libs := )))
> > +		$(eval $v-libs := )) \
> > +	$(if $($v-objs), \
> > +		$(eval $2$v-objs := $(addprefix $2,$($v-objs))) \
> > +		$(eval $v-objs := )) \
> > +)
> >  endef
> >  
> >  define unnest-dir
> > @@ -157,3 +165,9 @@ $(shell mkdir -p $(sort $(foreach var,$(nested-vars),$(dir $($(var))))))
> >  $(foreach var,$(nested-vars), $(eval \
> >    -include $(addsuffix *.d, $(sort $(dir $($(var)))))))
> >  endef
> > +
> > +define expand-mod-obj
> > +$(eval pref = $(if $(obj-base),$(obj-base)/,))
> > +$(eval t = $(foreach o,$($1),$(if $($(pref)$o-obj),$($(pref)$o-obj),$o)))
> > +$(eval $1 = $t)
> > +endef
> > diff --git a/tests/Makefile b/tests/Makefile
> > index 15ef039..0dda538 100644
> > --- a/tests/Makefile
> > +++ b/tests/Makefile
> > @@ -113,6 +113,7 @@ QEMU_CFLAGS += -I$(SRC_PATH)/tests
> >  nested-vars := block-obj-y
> >  obj-base := ..
> >  dummy := $(call unnest-vars)
> > +dummy := $(call expand-mod-obj,block-obj-y)
> >  
> >  tests/test-x86-cpuid.o: QEMU_INCLUDES += -I$(SRC_PATH)/target-i386
> >  
> > 
> 

Re: [Qemu-devel] [RFC PATCH v3 3/5] Makefile: introduce common-obj-m and block-obj-m for DSO

[Paolo Bonzini]

Il 10/09/2013 11:42, Fam Zheng ha scritto:
> On Tue, 09/10 08:45, Paolo Bonzini wrote:
>> Il 10/09/2013 03:02, Fam Zheng ha scritto:
>>> -all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all
>>> +# static linked mods are expanded to .o list
>>> +dummy := $(call expand-mod-obj,common-obj-y)
>>> +dummy := $(call expand-mod-obj,block-obj-y)
>>> +
>>> +modules-m = $(patsubst %.o,%$(DSOSUF),$(filter %.o,$(block-obj-m) $(common-obj-m))) \
>>> +            $(patsubst %.mo,%$(DSOSUF),$(filter %.mo,$(block-obj-m) $(common-obj-m)))
>>> +
>>> +all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all $(modules-m)
>>> +
>>> +# Generate rules for single file modules (%.so: %.o).
>>> +$(foreach o,$(filter %.o,$(block-obj-m) $(common-obj-m)),$(eval \
>>> +	$(patsubst %.o,%.so,$o): $o))
>>> +
>>> +# For multi file modules, dependencies should be listed explicitly in
>>> +# Makefile.objs as
>>> +#     foo.mo-objs := bar.o biz.o
>>> +$(foreach o,$(filter %.mo,$(block-obj-m) $(common-obj-m)),$(eval \
>>> +	$(patsubst %.mo,%.so,$o): $($o-objs)))
>>
>> I agree that this foo.mo-objs variable is homogeneous with how you
>> handle libraries and cflags.  I like it now.
>>
>> However, I don't like the many places in which you have to special-case
>> modules (expand-mod-obj, modules-m, etc.), and the duplication between
>> Makefile and Makefile.target.
>>
>> I would prefer if you try doing this patch along the lines I suggested
>> in my review of v2, using .mo files as a placeholder and then doing the
>> final link either into the .so or in the executable.  This should remove
>> the need for at least expand-mod-obj, and probably for more of the
>> duplicated constructs you have.
>>
> OK. I'll try.
> 
>> In particular, I would like modules-m to be simply "$(block-obj-m)
>> $(common-obj-m)".
> 
> There need to be some variable with %.o and %.mo subst to %.so, to become
> dependency of target "all".

I think you are putting too much weight on %.so, which complicates
things when handling both modular and non-modular builds.  Assuming you
have transformed block-obj-m and common-obj-m to only contain .mo files,
with something like this:

    define add-modules
    $(foreach o, $(filter-out %.o, $($1)), $(eval $o-objs := $o))
    $(eval modules-m += $(patsubst %.o,%.mo,$($1)))
    endif

    dummy := $(call add-modules,block-obj-m)
    dummy := $(call add-modules,common-obj-m)

then building modules can be just a bunch of extra targets:

    ifeq ($(CONFIG_MODULES),y)
    modules: $(patsubst %.mo,%$(DSOSUF),$(modules-m))
    all: modules
    endif

>> In the medium term, we need to find a way to avoid the duplication:
>>
>>      block-obj-y = block/
>>      block-obj-m = block/
>>
>> Perhaps by introducing a "dirs" variable that automatically triggers
>> recursion on all nested variables.  But this can be the topic of a
>> separate patch series, if you prefer.
>>
> Agree but prefer to do it in a separate series.

Sure.

Paolo