[kvm-unit-tests PATCH v2 0/9] Support for out-of-tree builds

[kvm-unit-tests PATCH v2 0/9] Support for out-of-tree builds

[Alex Bennée]

Hi,

Here is v2 of the out-of-tree build. There have been a number of
changes following review and also inclusion of Thomas' ppc64 patch
which completes the set.

The biggest change is the way I deal with creating build directories.
There is now a make variable called OBJDIRS which sub-builds can add
to. I had originally tried to be clever by expanding OBJDIRS into a
bunch of templated mkdir's which could then be set as
order-on-prerequisites as suggested by Drew. However it turns out to
be very hard to add the directory of the target as a prerequisite even
using hacks like GNU's secondary expansion. In the end I just created
a directories target and made sure the all: target had it as the first
thing. This breaks if someone tries to build an individual file
without first building the tree but I assume most people don't build
like that. If anyone else can come up with a neater solution I'm all
ears ;-)

Finally I've added a .travis.yml recipe. This really only works for
github hosted repos but its better than nothing. Evidently the cross
compilers complain about:

  lib/report.c:38:17: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
  assert_msg(len < sizeof(prefixes), "%d >= %lu", len, sizeof(prefixes));

and:

  lib/report.c:38:2: error: format '%lu' expects argument of type 'long unsigned int', but argument 6 has type 'unsigned int' [-Werror=format=]
  assert_msg(len < sizeof(prefixes), "%d >= %lu", len, sizeof(prefixes));

But these can be addressed with separate patches.

Alex Bennée (8):
  configure: make it run-able from outside source tree
  Makefile: ensure build-head works out-of-src-tree
  Makefile: set VPATH based on SRCDIR
  Makefiles: use explicit path for including sub-Makefiles
  Makefile: add explicit directories target
  Makefiles: fix up the x86 build dirs and include/link paths
  Makefiles: fix up the arm build dirs and include/link paths
  .travis.yml: initial build matrix

Thomas Huth (1):
  Makefiles: Fix up the powerpc build dirs and include/link paths

 .travis.yml             | 43 +++++++++++++++++++++++++++++++++++++++++++
 Makefile                | 22 +++++++++++++++++-----
 arm/Makefile            |  2 +-
 arm/Makefile.arm        |  2 +-
 arm/Makefile.arm64      |  4 +++-
 arm/Makefile.common     | 14 ++++++++------
 configure               | 36 ++++++++++++++++++++++++++++--------
 powerpc/Makefile        |  2 +-
 powerpc/Makefile.common | 15 +++++++++------
 powerpc/Makefile.ppc64  |  4 +++-
 scripts/asm-offsets.mak |  4 +++-
 x86/Makefile            |  2 +-
 x86/Makefile.common     | 16 ++++++++++------
 x86/Makefile.i386       |  2 +-
 x86/Makefile.x86_64     |  2 +-
 15 files changed, 130 insertions(+), 40 deletions(-)
 create mode 100644 .travis.yml

-- 
2.11.0

[kvm-unit-tests PATCH v2 1/9] configure: make it run-able from outside source tree

[Alex Bennée]

This is a first step to enabling out-of-tree builds for
kvm-unit-tests. When you invoke configure like this:

  ../tree.git/configure [args]

It will detect we the case and:

  - link ../tree.git/Makefile to the build-dir
  - link ../tree.git/$(arch) and test scripts
  - link ../tree.git/run_tests.sh
  - ensure lib is created with a correct lib/asm
  - set SRCDIR in the config.mk

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v2
  - ln -sf more directory structure
  - quotes around places where $srcdir is used
  - dropped pointless {}s
---
 configure | 36 ++++++++++++++++++++++++++++--------
 1 file changed, 28 insertions(+), 8 deletions(-)

diff --git a/configure b/configure
index d152414..2586131 100755
--- a/configure
+++ b/configure
@@ -1,5 +1,6 @@
 #!/usr/bin/env bash
 
+srcdir=$(cd "$(dirname "$0")"; pwd)
 prefix=/usr/local
 cc=gcc
 ld=ld
@@ -102,12 +103,12 @@ elif [ "$arch" = "ppc64" ]; then
 else
     testdir=$arch
 fi
-if [ ! -d $testdir ]; then
+if [ ! -d "$srcdir/$testdir" ]; then
     echo "$testdir does not exist!"
     exit 1
 fi
-if [ -f $testdir/run ]; then
-    ln -fs $testdir/run $testdir-run
+if [ -f "$srcdir/$testdir/run" ]; then
+    ln -fs "$srcdir/$testdir/run" $testdir-run
 fi
 
 # check if uint32_t needs a long format modifier
@@ -135,18 +136,37 @@ fi
 rm -f lib_test.c
 fi
 
+# Are we in a separate build tree? If so, link the Makefile
+# and shared stuff so that 'make' and run_tests.sh work.
+if test ! -e Makefile; then
+    echo "linking Makefile..."
+    ln -s "$srcdir/Makefile" .
+
+    echo "linking tests..."
+    mkdir -p $testdir
+    ln -sf "$srcdir/$testdir/run" $testdir/
+    ln -sf "$srcdir/$testdir/unittests.cfg" $testdir/
+    ln -sf "$srcdir/run_tests.sh"
+
+    echo "linking scripts..."
+    ln -sf "$srcdir/scripts"
+fi
+
 # link lib/asm for the architecture
 rm -f lib/asm
 asm=asm-generic
-if [ -d lib/$arch/asm ]; then
-	asm=$arch/asm
-elif [ -d lib/$testdir/asm ]; then
-	asm=$testdir/asm
+if [ -d "$srcdir/lib/$arch/asm" ]; then
+	asm="$srcdir/lib/$arch/asm"
+elif [ -d "$srcdir/lib/$testdir/asm" ]; then
+	asm="$srcdir/lib/$testdir/asm"
 fi
-ln -s $asm lib/asm
+mkdir -p lib
+ln -sf "$asm" lib/asm
+
 
 # create the config
 cat <<EOF > config.mak
+SRCDIR=$srcdir
 PREFIX=$prefix
 HOST=$host
 ARCH=$arch
-- 
2.11.0

[kvm-unit-tests PATCH v2 2/9] Makefile: ensure build-head works out-of-src-tree

[Alex Bennée]

Pass -C $(SRCDIR) to the git command that generates the build-head
stamp.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Andrew Jones <drjones@redhat.com>
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 2539f9a..5a0095a 100644
--- a/Makefile
+++ b/Makefile
@@ -80,7 +80,7 @@ $(LIBFDT_archive): $(addprefix $(LIBFDT_objdir)/,$(LIBFDT_OBJS))
 
 -include */.*.d */*/.*.d
 
-all: $(shell git rev-parse --verify --short=8 HEAD >build-head 2>/dev/null)
+all: $(shell git -C $(SRCDIR) rev-parse --verify --short=8 HEAD >build-head 2>/dev/null)
 
 standalone: all
 	@scripts/mkstandalone.sh
-- 
2.11.0

[kvm-unit-tests PATCH v2 3/9] Makefile: set VPATH based on SRCDIR

[Alex Bennée]

Setting the VPATH prompts make to search VPATH for source files. There
are still some fix-ups needed for linking and including other Makefile
fragments for each architecture.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Andrew Jones <drjones@redhat.com>

---
v2
  - minor whitespace fix
---
 Makefile | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Makefile b/Makefile
index 5a0095a..45097cd 100644
--- a/Makefile
+++ b/Makefile
@@ -7,6 +7,9 @@ endif
 
 include config.mak
 
+# Set search path for all sources
+VPATH = $(SRCDIR)
+
 libdirs-get = $(shell [ -d "lib/$(1)" ] && echo "lib/$(1) lib/$(1)/asm")
 ARCH_LIBDIRS := $(call libdirs-get,$(ARCH)) $(call libdirs-get,$(TEST_DIR))
 
-- 
2.11.0

[kvm-unit-tests PATCH v2 4/9] Makefiles: use explicit path for including sub-Makefiles

[Alex Bennée]

I would of thought VPATH took care of this but apparently not.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 Makefile               | 6 +++---
 arm/Makefile           | 2 +-
 arm/Makefile.arm       | 2 +-
 arm/Makefile.arm64     | 2 +-
 powerpc/Makefile       | 2 +-
 powerpc/Makefile.ppc64 | 2 +-
 x86/Makefile           | 2 +-
 x86/Makefile.i386      | 2 +-
 x86/Makefile.x86_64    | 2 +-
 9 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/Makefile b/Makefile
index 45097cd..a8e15b6 100644
--- a/Makefile
+++ b/Makefile
@@ -33,13 +33,13 @@ cflatobjs := \
 
 # libfdt paths
 LIBFDT_objdir = lib/libfdt
-LIBFDT_srcdir = lib/libfdt
+LIBFDT_srcdir = $(SRCDIR)/lib/libfdt
 LIBFDT_archive = $(LIBFDT_objdir)/libfdt.a
 LIBFDT_include = $(addprefix $(LIBFDT_srcdir)/,$(LIBFDT_INCLUDES))
 LIBFDT_version = $(addprefix $(LIBFDT_srcdir)/,$(LIBFDT_VERSION))
 
-#include architecure specific make rules
-include $(TEST_DIR)/Makefile
+#include architecture specific make rules
+include $(SRCDIR)/$(TEST_DIR)/Makefile
 
 # cc-option
 # Usage: OP_CFLAGS+=$(call cc-option, -falign-functions=0, -malign-functions=0)
diff --git a/arm/Makefile b/arm/Makefile
index 369a38b..8a007ab 100644
--- a/arm/Makefile
+++ b/arm/Makefile
@@ -1 +1 @@
-include $(TEST_DIR)/Makefile.$(ARCH)
+include $(SRCDIR)/$(TEST_DIR)/Makefile.$(ARCH)
diff --git a/arm/Makefile.arm b/arm/Makefile.arm
index 92f3757..6b57284 100644
--- a/arm/Makefile.arm
+++ b/arm/Makefile.arm
@@ -22,6 +22,6 @@ cflatobjs += lib/arm/stack.o
 # arm specific tests
 tests =
 
-include $(TEST_DIR)/Makefile.common
+include $(SRCDIR)/$(TEST_DIR)/Makefile.common
 
 arch_clean: arm_clean
diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index 0b0761c..b5fc1ec 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -14,7 +14,7 @@ cflatobjs += lib/arm64/spinlock.o
 # arm64 specific tests
 tests =
 
-include $(TEST_DIR)/Makefile.common
+include $(SRCDIR)/$(TEST_DIR)/Makefile.common
 
 arch_clean: arm_clean
 	$(RM) lib/arm64/.*.d
diff --git a/powerpc/Makefile b/powerpc/Makefile
index 369a38b..8a007ab 100644
--- a/powerpc/Makefile
+++ b/powerpc/Makefile
@@ -1 +1 @@
-include $(TEST_DIR)/Makefile.$(ARCH)
+include $(SRCDIR)/$(TEST_DIR)/Makefile.$(ARCH)
diff --git a/powerpc/Makefile.ppc64 b/powerpc/Makefile.ppc64
index 3da3a83..3c0294a 100644
--- a/powerpc/Makefile.ppc64
+++ b/powerpc/Makefile.ppc64
@@ -20,7 +20,7 @@ cflatobjs += lib/ppc64/spinlock.o
 # ppc64 specific tests
 tests =
 
-include $(TEST_DIR)/Makefile.common
+include $(SRCDIR)/$(TEST_DIR)/Makefile.common
 
 arch_clean: powerpc_clean
 	$(RM) lib/ppc64/.*.d
diff --git a/x86/Makefile b/x86/Makefile
index 369a38b..8a007ab 100644
--- a/x86/Makefile
+++ b/x86/Makefile
@@ -1 +1 @@
-include $(TEST_DIR)/Makefile.$(ARCH)
+include $(SRCDIR)/$(TEST_DIR)/Makefile.$(ARCH)
diff --git a/x86/Makefile.i386 b/x86/Makefile.i386
index 284b9e5..d801b80 100644
--- a/x86/Makefile.i386
+++ b/x86/Makefile.i386
@@ -7,4 +7,4 @@ cflatobjs += lib/x86/setjmp32.o
 tests = $(TEST_DIR)/taskswitch.flat $(TEST_DIR)/taskswitch2.flat \
 	$(TEST_DIR)/cmpxchg8b.flat
 
-include $(TEST_DIR)/Makefile.common
+include $(SRCDIR)/$(TEST_DIR)/Makefile.common
diff --git a/x86/Makefile.x86_64 b/x86/Makefile.x86_64
index 3e2821e..30f82a6 100644
--- a/x86/Makefile.x86_64
+++ b/x86/Makefile.x86_64
@@ -17,7 +17,7 @@ tests += $(TEST_DIR)/vmx.flat
 tests += $(TEST_DIR)/tscdeadline_latency.flat
 tests += $(TEST_DIR)/intel-iommu.flat
 
-include $(TEST_DIR)/Makefile.common
+include $(SRCDIR)/$(TEST_DIR)/Makefile.common
 
 $(TEST_DIR)/hyperv_clock.elf: $(TEST_DIR)/hyperv_clock.o
 
-- 
2.11.0

[kvm-unit-tests PATCH v2 5/9] Makefile: add explicit directories target

[Alex Bennée]

This is slightly clumsy way to ensure the directory structure for the
out-of-tree build is made first. The PHONY target is depended on by
make all first which runs mkdir for all OBJDIRS.

If an architecture target needs deeper nesting it needs to add
directories to OBJDIRS so it can be picked up by the directories rule.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v2
 - use explicit PHONY directories target
 - use @mkdir
---
 Makefile | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index a8e15b6..881bc78 100644
--- a/Makefile
+++ b/Makefile
@@ -12,6 +12,7 @@ VPATH = $(SRCDIR)
 
 libdirs-get = $(shell [ -d "lib/$(1)" ] && echo "lib/$(1) lib/$(1)/asm")
 ARCH_LIBDIRS := $(call libdirs-get,$(ARCH)) $(call libdirs-get,$(TEST_DIR))
+OBJDIRS := $(ARCH_LIBDIRS)
 
 DESTDIR := $(PREFIX)/share/kvm-unit-tests/
 
@@ -38,6 +39,8 @@ LIBFDT_archive = $(LIBFDT_objdir)/libfdt.a
 LIBFDT_include = $(addprefix $(LIBFDT_srcdir)/,$(LIBFDT_INCLUDES))
 LIBFDT_version = $(addprefix $(LIBFDT_srcdir)/,$(LIBFDT_VERSION))
 
+OBJDIRS += $(LIBFDT_objdir)
+
 #include architecture specific make rules
 include $(SRCDIR)/$(TEST_DIR)/Makefile
 
@@ -78,12 +81,18 @@ $(LIBFDT_archive): CFLAGS += -ffreestanding -I lib -I lib/libfdt -Wno-sign-compa
 $(LIBFDT_archive): $(addprefix $(LIBFDT_objdir)/,$(LIBFDT_OBJS))
 	$(AR) rcs $@ $^
 
+
+# Build directory target
+.PHONY: directories
+directories:
+	@mkdir -p $(OBJDIRS)
+
 %.o: %.S
 	$(CC) $(CFLAGS) -c -nostdlib -o $@ $<
 
 -include */.*.d */*/.*.d
 
-all: $(shell git -C $(SRCDIR) rev-parse --verify --short=8 HEAD >build-head 2>/dev/null)
+all: directories $(shell git -C $(SRCDIR) rev-parse --verify --short=8 HEAD >build-head 2>/dev/null)
 
 standalone: all
 	@scripts/mkstandalone.sh
-- 
2.11.0

[kvm-unit-tests PATCH v2 6/9] Makefiles: fix up the x86 build dirs and include/link paths

[Alex Bennée]

We still need to tell the compiler the correct search path for finding
headers and the like. This is slightly complicated by the "dynamic"
asm search path which is in our build tree but (may be) symlinked to
the right architectures headers.

Also we explicitly include SRCDIR for the linking scripts as VPATH
doesn't seem to find them well enough.

Also update OBJDIRS to include lib/x86.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v2
  - update OBJDIRS
---
 Makefile            |  2 +-
 x86/Makefile.common | 16 ++++++++++------
 2 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/Makefile b/Makefile
index 881bc78..af9695a 100644
--- a/Makefile
+++ b/Makefile
@@ -77,7 +77,7 @@ $(libcflat): $(cflatobjs)
 	$(AR) rcs $@ $^
 
 include $(LIBFDT_srcdir)/Makefile.libfdt
-$(LIBFDT_archive): CFLAGS += -ffreestanding -I lib -I lib/libfdt -Wno-sign-compare
+$(LIBFDT_archive): CFLAGS += -ffreestanding -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -Wno-sign-compare
 $(LIBFDT_archive): $(addprefix $(LIBFDT_objdir)/,$(LIBFDT_OBJS))
 	$(AR) rcs $@ $^
 
diff --git a/x86/Makefile.common b/x86/Makefile.common
index ec50926..57e8e45 100644
--- a/x86/Makefile.common
+++ b/x86/Makefile.common
@@ -1,6 +1,6 @@
 #This is a make file with common rules for both x86 & x86-64
 
-all: test_cases
+all: directories test_cases
 
 cflatobjs += lib/pci.o
 cflatobjs += lib/pci-edu.o
@@ -16,8 +16,10 @@ cflatobjs += lib/x86/isr.o
 cflatobjs += lib/x86/acpi.o
 cflatobjs += lib/x86/stack.o
 
+OBJDIRS += lib/x86
+
 $(libcflat): LDFLAGS += -nostdlib
-$(libcflat): CFLAGS += -ffreestanding -I lib
+$(libcflat): CFLAGS += -ffreestanding -I $(SRCDIR)/lib -I lib
 
 CFLAGS += -m$(bits)
 CFLAGS += -O1
@@ -31,8 +33,8 @@ libgcc := $(shell $(CC) -m$(bits) --print-libgcc-file-name)
 .PRECIOUS: %.elf %.o
 
 FLATLIBS = lib/libcflat.a $(libgcc)
-%.elf: %.o $(FLATLIBS) x86/flat.lds $(cstart.o)
-	$(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,x86/flat.lds \
+%.elf: %.o $(FLATLIBS) $(SRCDIR)/x86/flat.lds $(cstart.o)
+	$(CC) $(CFLAGS) -nostdlib -o $@ -Wl,-T,$(SRCDIR)/x86/flat.lds \
 		$(filter %.o, $^) $(FLATLIBS)
 
 %.flat: %.elf
@@ -52,14 +54,16 @@ ifdef API
 tests-common += api/api-sample
 tests-common += api/dirty-log
 tests-common += api/dirty-log-perf
+
+OBJDIRS += api
 endif
 
 test_cases: $(tests-common) $(tests)
 
-$(TEST_DIR)/%.o: CFLAGS += -std=gnu99 -ffreestanding -I lib -I lib/x86
+$(TEST_DIR)/%.o: CFLAGS += -std=gnu99 -ffreestanding -I $(SRCDIR)/lib -I $(SRCDIR)/lib/x86 -I lib
 
 $(TEST_DIR)/realmode.elf: $(TEST_DIR)/realmode.o
-	$(CC) -m32 -nostdlib -o $@ -Wl,-T,$(TEST_DIR)/realmode.lds $^
+	$(CC) -m32 -nostdlib -o $@ -Wl,-T,$(SRCDIR)/$(TEST_DIR)/realmode.lds $^
 
 $(TEST_DIR)/realmode.o: bits = 32
 
-- 
2.11.0

[kvm-unit-tests PATCH v2 7/9] Makefiles: fix up the arm build dirs and include/link paths

[Alex Bennée]

We still need to tell the compiler the correct search path for finding
headers and the like. This is slightly complicated by the "dynamic"
asm search path which is in our build tree but (may be) symlinked to
the right architectures headers.

Also we explicitly include SRCDIR for the linking scripts as VPATH
doesn't seem to find them well enough.

Also we set OBJDIRS for the lib/arm, lib/arm64 and generated
asm-offset files.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v2
  - Setup OBJDIRS
---
 arm/Makefile.arm64      |  2 ++
 arm/Makefile.common     | 14 ++++++++------
 scripts/asm-offsets.mak |  4 +++-
 3 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/arm/Makefile.arm64 b/arm/Makefile.arm64
index b5fc1ec..24319d6 100644
--- a/arm/Makefile.arm64
+++ b/arm/Makefile.arm64
@@ -11,6 +11,8 @@ cstart.o = $(TEST_DIR)/cstart64.o
 cflatobjs += lib/arm64/processor.o
 cflatobjs += lib/arm64/spinlock.o
 
+OBJDIRS += lib/arm64
+
 # arm64 specific tests
 tests =
 
diff --git a/arm/Makefile.common b/arm/Makefile.common
index 74c7394..03b497b 100644
--- a/arm/Makefile.common
+++ b/arm/Makefile.common
@@ -17,7 +17,7 @@ tests-common += $(TEST_DIR)/gic.flat
 tests-common += $(TEST_DIR)/psci.flat
 
 tests-all = $(tests-common) $(tests)
-all: $(tests-all)
+all: directories $(tests-all)
 
 ##################################################################
 phys_base = $(LOADADDR)
@@ -26,13 +26,13 @@ CFLAGS += -std=gnu99
 CFLAGS += -ffreestanding
 CFLAGS += -Wextra
 CFLAGS += -O2
-CFLAGS += -I lib -I lib/libfdt
+CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
 
 # We want to keep intermediate files
 .PRECIOUS: %.elf %.o
 
 asm-offsets = lib/$(ARCH)/asm-offsets.h
-include scripts/asm-offsets.mak
+include $(SRCDIR)/scripts/asm-offsets.mak
 
 cflatobjs += lib/util.o
 cflatobjs += lib/alloc.o
@@ -52,6 +52,8 @@ cflatobjs += lib/arm/smp.o
 cflatobjs += lib/arm/delay.o
 cflatobjs += lib/arm/gic.o lib/arm/gic-v2.o lib/arm/gic-v3.o
 
+OBJDIRS += lib/arm
+
 libeabi = lib/arm/libeabi.a
 eabiobjs = lib/arm/eabi_compat.o
 
@@ -60,11 +62,11 @@ start_addr := $(shell printf "%x\n" $$(( $(phys_base) + $(kernel_offset) )))
 
 FLATLIBS = $(libcflat) $(LIBFDT_archive) $(libgcc) $(libeabi)
 %.elf: LDFLAGS = $(CFLAGS) -nostdlib
-%.elf: %.o $(FLATLIBS) arm/flat.lds $(cstart.o)
+%.elf: %.o $(FLATLIBS) $(SRCDIR)/arm/flat.lds $(cstart.o)
 	$(CC) $(LDFLAGS) -o $@ \
-		-Wl,-T,arm/flat.lds,--build-id=none,-Ttext=$(start_addr) \
+		-Wl,-T,$(SRCDIR)/arm/flat.lds,--build-id=none,-Ttext=$(start_addr) \
 		$(filter %.o, $^) $(FLATLIBS) \
-		lib/auxinfo.c -DPROGNAME=\"$(@:.elf=.flat)\"
+		$(SRCDIR)/lib/auxinfo.c -DPROGNAME=\"$(@:.elf=.flat)\"
 
 %.flat: %.elf
 	$(OBJCOPY) -O binary $^ $@
diff --git a/scripts/asm-offsets.mak b/scripts/asm-offsets.mak
index b2578a6..b35da09 100644
--- a/scripts/asm-offsets.mak
+++ b/scripts/asm-offsets.mak
@@ -33,7 +33,9 @@ $(asm-offsets:.h=.s): $(asm-offsets:.h=.c)
 
 $(asm-offsets): $(asm-offsets:.h=.s)
 	$(call make_asm_offsets)
-	cp -f $(asm-offsets) lib/generated
+	cp -f $(asm-offsets) lib/generated/
+
+OBJDIRS += lib/generated
 
 asm_offsets_clean:
 	$(RM) $(asm-offsets) $(asm-offsets:.h=.s) \
-- 
2.11.0

[kvm-unit-tests PATCH v2 8/9] Makefiles: Fix up the powerpc build dirs and include/link paths

[Alex Bennée]

From: Thomas Huth <thuth@redhat.com>

The changes for out-of-tree builds are here pretty much the same
as for the ARM Makefile.common.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v2
  - OBJDIRS
---
 powerpc/Makefile.common | 15 +++++++++------
 powerpc/Makefile.ppc64  |  2 ++
 2 files changed, 11 insertions(+), 6 deletions(-)

diff --git a/powerpc/Makefile.common b/powerpc/Makefile.common
index e536ffe..db5ba62 100644
--- a/powerpc/Makefile.common
+++ b/powerpc/Makefile.common
@@ -13,7 +13,7 @@ tests-common = \
 	$(TEST_DIR)/sprs.elf
 
 tests-all = $(tests-common) $(tests)
-all: $(TEST_DIR)/boot_rom.bin $(tests-all)
+all: directories $(TEST_DIR)/boot_rom.bin $(tests-all)
 
 ##################################################################
 
@@ -21,14 +21,14 @@ CFLAGS += -std=gnu99
 CFLAGS += -ffreestanding
 CFLAGS += -Wextra
 CFLAGS += -O2
-CFLAGS += -I lib -I lib/libfdt
+CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
 CFLAGS += -Wa,-mregnames
 
 # We want to keep intermediate files
 .PRECIOUS: %.o
 
 asm-offsets = lib/$(ARCH)/asm-offsets.h
-include scripts/asm-offsets.mak
+include $(SRCDIR)/scripts/asm-offsets.mak
 
 cflatobjs += lib/util.o
 cflatobjs += lib/alloc.o
@@ -41,13 +41,16 @@ cflatobjs += lib/powerpc/processor.o
 cflatobjs += lib/powerpc/handlers.o
 cflatobjs += lib/powerpc/smp.o
 
+OBJDIRS += lib/powerpc
+
 FLATLIBS = $(libcflat) $(LIBFDT_archive)
 %.elf: CFLAGS += $(arch_CFLAGS)
 %.elf: LDFLAGS = $(arch_LDFLAGS) -nostdlib -pie -n
-%.elf: %.o $(FLATLIBS) powerpc/flat.lds $(cstart.o) $(reloc.o)
-	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) lib/auxinfo.c -DPROGNAME=\"$@\"
+%.elf: %.o $(FLATLIBS) $(SRCDIR)/powerpc/flat.lds $(cstart.o) $(reloc.o)
+	$(CC) $(CFLAGS) -c -o $(@:.elf=.aux.o) $(SRCDIR)/lib/auxinfo.c \
+		-DPROGNAME=\"$@\"
 	$(LD) $(LDFLAGS) -o $@ \
-		-T powerpc/flat.lds --build-id=none \
+		-T $(SRCDIR)/powerpc/flat.lds --build-id=none \
 		$(filter %.o, $^) $(FLATLIBS) $(@:.elf=.aux.o)
 	$(RM) $(@:.elf=.aux.o)
 	@echo -n Checking $@ for unsupported reloc types...
diff --git a/powerpc/Makefile.ppc64 b/powerpc/Makefile.ppc64
index 3c0294a..dc4e088 100644
--- a/powerpc/Makefile.ppc64
+++ b/powerpc/Makefile.ppc64
@@ -17,6 +17,8 @@ cstart.o = $(TEST_DIR)/cstart64.o
 reloc.o  = $(TEST_DIR)/reloc64.o
 cflatobjs += lib/ppc64/spinlock.o
 
+OBJDIRS += lib/ppc64
+
 # ppc64 specific tests
 tests =
 
-- 
2.11.0

[kvm-unit-tests PATCH v2 9/9] .travis.yml: initial build matrix

[Alex Bennée]

This adds a TravisCI build matrix to test all the various build
combinations of architecture and with/without out-of-tree builds. It
currently fails because the Trusty compilers have some issues with
various format strings and unsigned/signed comparisions but these can
be fixed up later.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .travis.yml | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
 create mode 100644 .travis.yml

diff --git a/.travis.yml b/.travis.yml
new file mode 100644
index 0000000..edccfe0
--- /dev/null
+++ b/.travis.yml
@@ -0,0 +1,43 @@
+sudo: false
+dist: trusty
+language: c
+compiler:
+  - gcc
+cache: ccache
+addons:
+  apt:
+    packages:
+      # Cross Toolchains
+      - gcc-arm-linux-gnueabihf
+      - gcc-aarch64-linux-gnu
+      - gcc-powerpc64le-linux-gnu
+      # Run dependencies
+      - qemu-system
+git:
+  submodules: false
+env:
+  matrix:
+    - CONFIG=""
+      BUILD_DIR="."
+    - CONFIG=""
+      BUILD_DIR="x86-builddir"
+    - CONFIG="--arch=arm --cross-prefix=arm-linux-gnueabihf-"
+      BUILD_DIR="."
+    - CONFIG="--arch=arm --cross-prefix=arm-linux-gnueabihf-"
+      BUILD_DIR="arm-buildir"
+    - CONFIG="--arch=arm64 --cross-prefix=aarch64-linux-gnu-"
+      BUILD_DIR="."
+    - CONFIG="--arch=arm64 --cross-prefix=aarch64-linux-gnu-"
+      BUILD_DIR="arm64-buildir"
+    - CONFIG="--arch=ppc64 --endian=little --cross-prefix=powerpc64le-linux-gnu-"
+      BUILD_DIR="."
+    - CONFIG="--arch=ppc64 --endian=little --cross-prefix=powerpc64le-linux-gnu-"
+      BUILD_DIR="ppc64le-buildir"
+
+before_script:
+  - mkdir -p $BUILD_DIR && cd $BUILD_DIR
+  - if [ -e ./configure ]; then ./configure $CONFIG ; fi
+  - if [ -e ../configure ]; then ../configure $CONFIG ; fi
+script:
+  - make -j3
+  - ./run_tests.sh || true
-- 
2.11.0

Re: [kvm-unit-tests PATCH v2 1/9] configure: make it run-able from outside source tree

[Radim Krčmář]

2017-05-12 17:39+0100, Alex Bennée:
> This is a first step to enabling out-of-tree builds for
> kvm-unit-tests. When you invoke configure like this:
> 
>   ../tree.git/configure [args]
> 
> It will detect we the case and:
> 
>   - link ../tree.git/Makefile to the build-dir
>   - link ../tree.git/$(arch) and test scripts
>   - link ../tree.git/run_tests.sh
>   - ensure lib is created with a correct lib/asm
>   - set SRCDIR in the config.mk
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> 
> ---
> v2
>   - ln -sf more directory structure
>   - quotes around places where $srcdir is used
>   - dropped pointless {}s
> ---
> diff --git a/configure b/configure
> +srcdir=$(cd "$(dirname "$0")"; pwd)
> +# Are we in a separate build tree? If so, link the Makefile
> +# and shared stuff so that 'make' and run_tests.sh work.
> +if test ! -e Makefile; then
> +    echo "linking Makefile..."
> +    ln -s "$srcdir/Makefile" .
> +
> +    echo "linking tests..."
> +    mkdir -p $testdir
> +    ln -sf "$srcdir/$testdir/run" $testdir/
> +    ln -sf "$srcdir/$testdir/unittests.cfg" $testdir/
> +    ln -sf "$srcdir/run_tests.sh"
> +
> +    echo "linking scripts..."
> +    ln -sf "$srcdir/scripts"
> +fi
> +
>  
>  # create the config
>  cat <<EOF > config.mak
> +SRCDIR=$srcdir

Whitespace in $srcdir is going to break the build.  I think that we
cannot prevent that by escaping as `make` is notoriously bad in that
regard.  Aborting sounds better and maybe something like this could
work:

  ln -sf "$srcdir" srcdir
  srcdir=srcdir

I'm ok with this version if the solution would not be simple. :)

Re: [kvm-unit-tests PATCH v2 0/9] Support for out-of-tree builds

[Radim Krčmář]

2017-05-12 17:39+0100, Alex Bennée:
> Hi,
> 
> Here is v2 of the out-of-tree build. There have been a number of
> changes following review and also inclusion of Thomas' ppc64 patch
> which completes the set.
> 
> The biggest change is the way I deal with creating build directories.
> There is now a make variable called OBJDIRS which sub-builds can add
> to. I had originally tried to be clever by expanding OBJDIRS into a
> bunch of templated mkdir's which could then be set as
> order-on-prerequisites as suggested by Drew. However it turns out to
> be very hard to add the directory of the target as a prerequisite even
> using hacks like GNU's secondary expansion. In the end I just created
> a directories target and made sure the all: target had it as the first
> thing. This breaks if someone tries to build an individual file
> without first building the tree but I assume most people don't build
> like that. If anyone else can come up with a neater solution I'm all
> ears ;-)

I noticed few cases that don't work:
 - SRC_DIR path contains whitespace
 - SRC_DIR was `./configure`d for another arch
 - SRC_DIR isn't clean (up-to-date results won't be rebuilt out-of-tree)
 - reconfiguring --arch in a out-of-tree directory

The in-tree build still works, out-of-tree in decent cases, Makefile
changes are reasonable, and reviewers seem content in front of a
first-grade bikeshedding material ...

Applied, thanks.

> Finally I've added a .travis.yml recipe. This really only works for
> github hosted repos but its better than nothing. Evidently the cross
> compilers complain about:
> 
>   lib/report.c:38:17: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
>   assert_msg(len < sizeof(prefixes), "%d >= %lu", len, sizeof(prefixes));
> 
> and:
> 
>   lib/report.c:38:2: error: format '%lu' expects argument of type 'long unsigned int', but argument 6 has type 'unsigned int' [-Werror=format=]
>   assert_msg(len < sizeof(prefixes), "%d >= %lu", len, sizeof(prefixes));
> 
> But these can be addressed with separate patches.

(Should be fixed now.)

Re: [kvm-unit-tests PATCH v2 1/9] configure: make it run-able from outside source tree

[Paolo Bonzini]

On 17/05/2017 16:50, Radim Krčmář wrote:
>>  # create the config
>>  cat <<EOF > config.mak
>> +SRCDIR=$srcdir
> Whitespace in $srcdir is going to break the build.

I think it's pretty common to have things break horribly if there's
whitespace in $srcdir.

Just don't call your srcdir "little bobby tables; rm -rf /".

Paolo

  I think that we
> cannot prevent that by escaping as `make` is notoriously bad in that
> regard.  Aborting sounds better and maybe something like this could
> work:
> 
>   ln -sf "$srcdir" srcdir
>   srcdir=srcdir
> 
> I'm ok with this version if the solution would not be simple. :)

Re: [kvm-unit-tests PATCH v2 5/9] Makefile: add explicit directories target

[Paolo Bonzini]

On 12/05/2017 18:39, Alex Bennée wrote:
> This is slightly clumsy way to ensure the directory structure for the
> out-of-tree build is made first. The PHONY target is depended on by
> make all first which runs mkdir for all OBJDIRS.
> 
> If an architecture target needs deeper nesting it needs to add
> directories to OBJDIRS so it can be picked up by the directories rule.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> 
> ---
> v2
>  - use explicit PHONY directories target
>  - use @mkdir
> ---
>  Makefile | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/Makefile b/Makefile
> index a8e15b6..881bc78 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -12,6 +12,7 @@ VPATH = $(SRCDIR)
>  
>  libdirs-get = $(shell [ -d "lib/$(1)" ] && echo "lib/$(1) lib/$(1)/asm")
>  ARCH_LIBDIRS := $(call libdirs-get,$(ARCH)) $(call libdirs-get,$(TEST_DIR))
> +OBJDIRS := $(ARCH_LIBDIRS)
>  
>  DESTDIR := $(PREFIX)/share/kvm-unit-tests/
>  
> @@ -38,6 +39,8 @@ LIBFDT_archive = $(LIBFDT_objdir)/libfdt.a
>  LIBFDT_include = $(addprefix $(LIBFDT_srcdir)/,$(LIBFDT_INCLUDES))
>  LIBFDT_version = $(addprefix $(LIBFDT_srcdir)/,$(LIBFDT_VERSION))
>  
> +OBJDIRS += $(LIBFDT_objdir)
> +
>  #include architecture specific make rules
>  include $(SRCDIR)/$(TEST_DIR)/Makefile
>  
> @@ -78,12 +81,18 @@ $(LIBFDT_archive): CFLAGS += -ffreestanding -I lib -I lib/libfdt -Wno-sign-compa
>  $(LIBFDT_archive): $(addprefix $(LIBFDT_objdir)/,$(LIBFDT_OBJS))
>  	$(AR) rcs $@ $^
>  
> +
> +# Build directory target
> +.PHONY: directories
> +directories:
> +	@mkdir -p $(OBJDIRS)
> +
>  %.o: %.S
>  	$(CC) $(CFLAGS) -c -nostdlib -o $@ $<

Could you add a $(shell $(mkdir -p $(@D)) here instead, and a $(shell
$(mkdir $(LIBFDT_objdir)) to cater for the nested libfdt makefile?

Sorry if I have asked this already.

Paolo

>  
>  -include */.*.d */*/.*.d
>  
> -all: $(shell git -C $(SRCDIR) rev-parse --verify --short=8 HEAD >build-head 2>/dev/null)
> +all: directories $(shell git -C $(SRCDIR) rev-parse --verify --short=8 HEAD >build-head 2>/dev/null)
>  
>  standalone: all
>  	@scripts/mkstandalone.sh
>