[PATCH v1 00/15] testing/gdbstub/docs pre-PR

[PATCH v1 00/15] testing/gdbstub/docs pre-PR

[Alex Bennée]

Hi,

Here is the pre-PR roll-up of my testing tree (and other misc bits).
We have updates to docker's binfmt handling, gdbstub testing as well
as some documentation tweaks. Last chance to review/object to any
patches before I roll the PR, the following are still unreviewed:

 - docs/system: document an example booting the versatilepb machine
 - docs/system: document an example vexpress-a15 invocation
 - tests/tcg: don't silently skip the gdb tests
 - configure: bump the minimum gdb version for check-tcg to 9.1
 - tests/docker: add a docker-exec-copy-test
 - tests/docker: preserve original name when copying libs

Alex Bennée (11):
  tests/docker: make _copy_with_mkdir accept missing files
  tests/docker: preserve original name when copying libs
  tests/docker: alias docker-help target for consistency
  tests/docker: add a docker-exec-copy-test
  configure: make version_ge more tolerant of shady version input
  configure: bump the minimum gdb version for check-tcg to 9.1
  tests/tcg: don't silently skip the gdb tests
  scripts/mtest2make.py: export all-%s-targets variable and use it
  tests/Makefile.include: don't use TARGET_DIRS for check-tcg
  docs/system: document an example vexpress-a15 invocation
  docs/system: document an example booting the versatilepb machine

Philippe Mathieu-Daudé (2):
  tests/docker: Fix _get_so_libs() for docker-binfmt-image
  tests/docker: Fix typo in help message

Richard Henderson (1):
  gdbstub: Fix handle_query_xfer_auxv

Stefan Weil (1):
  tests/tcg: Replace /bin/true by true (required on macOS)

 docs/system/arm/versatile.rst         | 34 +++++++++++++++++++++++++++
 docs/system/arm/vexpress.rst          | 28 ++++++++++++++++++++++
 configure                             |  6 ++---
 Makefile                              |  2 +-
 gdbstub.c                             | 17 ++++++++++----
 scripts/mtest2make.py                 |  1 +
 tests/Makefile.include                | 12 ++++++----
 tests/docker/Makefile.include         | 26 +++++++++++++++++---
 tests/docker/docker.py                | 23 +++++++++++++-----
 tests/docker/dockerfiles/empty.docker |  8 +++++++
 tests/tcg/Makefile.qemu               |  4 ++--
 tests/tcg/multiarch/Makefile.target   |  5 +++-
 12 files changed, 140 insertions(+), 26 deletions(-)
 create mode 100644 tests/docker/dockerfiles/empty.docker

-- 
2.20.1

[PATCH v1 01/15] tests/docker: Fix _get_so_libs() for docker-binfmt-image

[Alex Bennée]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

Fix a variable rename mistake from commit 5e33f7fead5:

  Traceback (most recent call last):
    File "./tests/docker/docker.py", line 710, in <module>
      sys.exit(main())
    File "./tests/docker/docker.py", line 706, in main
      return args.cmdobj.run(args, argv)
    File "./tests/docker/docker.py", line 489, in run
      _copy_binary_with_libs(args.include_executable,
    File "./tests/docker/docker.py", line 149, in _copy_binary_with_libs
      libs = _get_so_libs(src)
    File "./tests/docker/docker.py", line 123, in _get_so_libs
      libs.append(s.group(1))
  NameError: name 's' is not defined

Fixes: 5e33f7fead5 ("tests/docker: better handle symlinked libs")
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210119050149.516910-1-f4bug@amsat.org>
---
 tests/docker/docker.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 884dfeb29c..0b4f6167b3 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -120,7 +120,7 @@ def _get_so_libs(executable):
             search = ldd_re.search(line)
             if search:
                 try:
-                    libs.append(s.group(1))
+                    libs.append(search.group(1))
                 except IndexError:
                     pass
     except subprocess.CalledProcessError:
-- 
2.20.1

[PATCH v1 02/15] tests/docker: Fix typo in help message

[Alex Bennée]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

To have the variable properly passed, we need to set it,
ie. NOUSER=1. Fix the message displayed by 'make docker'.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210119052120.522069-1-f4bug@amsat.org>
---
 tests/docker/Makefile.include | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 0779dab5b9..bdc53ddfcf 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -209,7 +209,7 @@ endif
 	@echo '                         before running the command.'
 	@echo '    NETWORK=1            Enable virtual network interface with default backend.'
 	@echo '    NETWORK=$$BACKEND     Enable virtual network interface with $$BACKEND.'
-	@echo '    NOUSER               Define to disable adding current user to containers passwd.'
+	@echo '    NOUSER=1             Define to disable adding current user to containers passwd.'
 	@echo '    NOCACHE=1            Ignore cache when build images.'
 	@echo '    EXECUTABLE=<path>    Include executable in image.'
 	@echo '    EXTRA_FILES="<path> [... <path>]"'
-- 
2.20.1

[PATCH v1 03/15] tests/docker: make _copy_with_mkdir accept missing files

[Alex Bennée]

Depending on the linker/ldd setup we might get a file with no path.
Typically this is the psuedo library linux-vdso.so which doesn't
actually exist on the disk. Rather than try and catch these distro
specific edge cases just shout about it and try and continue.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 tests/docker/docker.py | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 0b4f6167b3..fb3de41c0b 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -103,7 +103,12 @@ def _copy_with_mkdir(src, root_dir, sub_path='.'):
         pass
 
     dest_file = "%s/%s" % (dest_dir, os.path.basename(src))
-    copy(src, dest_file)
+
+    try:
+        copy(src, dest_file)
+    except FileNotFoundError:
+        print("Couldn't copy %s to %s" % (src, dest_file))
+        pass
 
 
 def _get_so_libs(executable):
-- 
2.20.1

[PATCH v1 04/15] tests/docker: preserve original name when copying libs

[Alex Bennée]

While it is important we chase down the symlinks to copy the correct
data we can confuse the kernel by renaming the interpreter to what is
in the binary. Extend _copy_with_mkdir to preserve the original name
of the file when asked.

Fixes: 5e33f7fead ("tests/docker: better handle symlinked libs")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/docker.py | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index fb3de41c0b..39da3fefcf 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -93,7 +93,7 @@ def _guess_engine_command():
                     commands_txt)
 
 
-def _copy_with_mkdir(src, root_dir, sub_path='.'):
+def _copy_with_mkdir(src, root_dir, sub_path='.', name=None):
     """Copy src into root_dir, creating sub_path as needed."""
     dest_dir = os.path.normpath("%s/%s" % (root_dir, sub_path))
     try:
@@ -102,7 +102,7 @@ def _copy_with_mkdir(src, root_dir, sub_path='.'):
         # we can safely ignore already created directories
         pass
 
-    dest_file = "%s/%s" % (dest_dir, os.path.basename(src))
+    dest_file = "%s/%s" % (dest_dir, name if name else os.path.basename(src))
 
     try:
         copy(src, dest_file)
@@ -155,8 +155,9 @@ def _copy_binary_with_libs(src, bin_dest, dest_dir):
     if libs:
         for l in libs:
             so_path = os.path.dirname(l)
+            name = os.path.basename(l)
             real_l = os.path.realpath(l)
-            _copy_with_mkdir(real_l, dest_dir, so_path)
+            _copy_with_mkdir(real_l, dest_dir, so_path, name)
 
 
 def _check_binfmt_misc(executable):
-- 
2.20.1

[PATCH v1 05/15] tests/docker: alias docker-help target for consistency

[Alex Bennée]

We have a bunch of -help targets so this will save some cognitive
dissonance. Keep the original for those with muscle memory.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>

---
v2
  - also fix help-on-help text
---
 Makefile                      | 2 +-
 tests/docker/Makefile.include | 4 +++-
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index b0dff73904..d7fb6b270e 100644
--- a/Makefile
+++ b/Makefile
@@ -305,7 +305,7 @@ endif
 	@echo  'Test targets:'
 	$(call print-help,check,Run all tests (check-help for details))
 	$(call print-help,bench,Run all benchmarks)
-	$(call print-help,docker,Help about targets running tests inside containers)
+	$(call print-help,docker-help,Help about targets running tests inside containers)
 	$(call print-help,vm-help,Help about targets running tests inside VM)
 	@echo  ''
 	@echo  'Documentation targets:'
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index bdc53ddfcf..a5c1e4a615 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -1,6 +1,6 @@
 # Makefile for Docker tests
 
-.PHONY: docker docker-test docker-clean docker-image docker-qemu-src
+.PHONY: docker docker-help docker-test docker-clean docker-image docker-qemu-src
 
 NULL :=
 SPACE := $(NULL) #
@@ -218,6 +218,8 @@ endif
 	@echo '                         Specify which container engine to run.'
 	@echo '    REGISTRY=url         Cache builds from registry (default:$(DOCKER_REGISTRY))'
 
+docker-help: docker
+
 # This rule if for directly running against an arbitrary docker target.
 # It is called by the expanded docker targets (e.g. make
 # docker-test-foo@bar) which will do additional verification.
-- 
2.20.1

[PATCH v1 06/15] tests/docker: add a docker-exec-copy-test

[Alex Bennée]

This provides test machinery for checking the QEMU copying logic works
properly. It takes considerably less time to run than starting a
debootstrap only for it to fail later. I considered adding a remove
command to docker.py but figured that might be gold plating given the
relative size of the containers compared to the ones with actual stuff
in them.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include         | 20 +++++++++++++++++++-
 tests/docker/docker.py                |  7 ++++++-
 tests/docker/dockerfiles/empty.docker |  8 ++++++++
 3 files changed, 33 insertions(+), 2 deletions(-)
 create mode 100644 tests/docker/dockerfiles/empty.docker

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index a5c1e4a615..93b29ad823 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -11,7 +11,7 @@ HOST_ARCH = $(if $(ARCH),$(ARCH),$(shell uname -m))
 DOCKER_SUFFIX := .docker
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
 # we don't run tests on intermediate images (used as base by another image)
-DOCKER_PARTIAL_IMAGES := debian10 debian11 debian-bootstrap
+DOCKER_PARTIAL_IMAGES := debian10 debian11 debian-bootstrap empty
 DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker))))
 DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
 # Use a global constant ccache directory to speed up repetitive builds
@@ -92,6 +92,24 @@ docker-binfmt-image-debian-%: $(DOCKER_FILES_DIR)/debian-bootstrap.docker
 			{ echo "You will need to build $(EXECUTABLE)"; exit 1;},\
 			"CHECK", "debian-$* exists"))
 
+# These are test targets
+USER_TCG_TARGETS=$(patsubst %-linux-user,qemu-%,$(filter %-linux-user,$(TARGET_DIRS)))
+EXEC_COPY_TESTS=$(patsubst %,docker-exec-copy-test-%, $(USER_TCG_TARGETS))
+
+$(EXEC_COPY_TESTS): docker-exec-copy-test-%: $(DOCKER_FILES_DIR)/empty.docker
+	$(call quiet-command,							\
+		$(DOCKER_SCRIPT) build -t qemu/exec-copy-test-$* -f $< 		\
+			$(if $V,,--quiet) --no-cache 				\
+			--include-executable=$*					\
+			--skip-binfmt,						\
+			"TEST","copy $* to container")
+	$(call quiet-command,							\
+		$(DOCKER_SCRIPT) run qemu/exec-copy-test-$* 			\
+			/$* -version > tests/docker-exec-copy-test-$*.out,	\
+			"TEST","check $* works in container")
+
+docker-exec-copy-test: $(EXEC_COPY_TESTS)
+
 endif
 
 # Enforce dependencies for composite images
diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 39da3fefcf..d28df4c140 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -438,6 +438,9 @@ class BuildCommand(SubCommand):
                             help="""Specify a binary that will be copied to the
                             container together with all its dependent
                             libraries""")
+        parser.add_argument("--skip-binfmt",
+                            action="store_true",
+                            help="""Skip binfmt entry check (used for testing)""")
         parser.add_argument("--extra-files", nargs='*',
                             help="""Specify files that will be copied in the
                             Docker image, fulfilling the ADD directive from the
@@ -466,7 +469,9 @@ class BuildCommand(SubCommand):
             docker_dir = tempfile.mkdtemp(prefix="docker_build")
 
             # Validate binfmt_misc will work
-            if args.include_executable:
+            if args.skip_binfmt:
+                qpath = args.include_executable
+            elif args.include_executable:
                 qpath, enabled = _check_binfmt_misc(args.include_executable)
                 if not enabled:
                     return 1
diff --git a/tests/docker/dockerfiles/empty.docker b/tests/docker/dockerfiles/empty.docker
new file mode 100644
index 0000000000..9ba980f1a8
--- /dev/null
+++ b/tests/docker/dockerfiles/empty.docker
@@ -0,0 +1,8 @@
+#
+# Empty Dockerfile
+#
+
+FROM scratch
+
+# Add everything from the context into the container
+ADD . /
-- 
2.20.1

[PATCH v1 07/15] configure: make version_ge more tolerant of shady version input

[Alex Bennée]

When checking GDB versions we have to tolerate all sorts of random
distro extensions to the version string. While we already attempt to
do some of that before we call version_ge is makes sense to try and
regularise the first input by stripping extraneous -'s. While we at it
convert the old-style shell quoting into a cleaner form t shut up my
editors linter lest it confuse me by underlining the whole line.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Thomas Huth <thuth@redhat.com>
---
 configure | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/configure b/configure
index e85d6baf8f..432b83fadf 100755
--- a/configure
+++ b/configure
@@ -198,8 +198,8 @@ has() {
 }
 
 version_ge () {
-    local_ver1=`echo $1 | tr . ' '`
-    local_ver2=`echo $2 | tr . ' '`
+    local_ver1=$(expr "$1" : '\([0-9.]*\)' | tr . ' ')
+    local_ver2=$(echo "$2" | tr . ' ')
     while true; do
         set x $local_ver1
         local_first=${2-0}
-- 
2.20.1

[PATCH v1 08/15] configure: bump the minimum gdb version for check-tcg to 9.1

[Alex Bennée]

For SVE, currently the bulk of the GDB TCG tests, we need at least GDB
9.1 to support the "ieee_half" data type we report. This only affects
when GDB tests are run; users can still use lower versions of gdb as
long as they aren't talking to an SVE enabled model. The work around
is to either get a newer gdb or disable SVE for their CPU model.

Reported-by: Claudio Fontana <cfontana@suse.de>
Cc: Luis Machado <luis.machado@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 configure | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure b/configure
index 432b83fadf..6d419aac79 100755
--- a/configure
+++ b/configure
@@ -6115,7 +6115,7 @@ fi
 
 if test -n "$gdb_bin"; then
     gdb_version=$($gdb_bin --version | head -n 1)
-    if version_ge ${gdb_version##* } 8.3.1; then
+    if version_ge ${gdb_version##* } 9.1; then
         echo "HAVE_GDB_BIN=$gdb_bin" >> $config_host_mak
     fi
 fi
-- 
2.20.1

[PATCH v1 09/15] tests/tcg: don't silently skip the gdb tests

[Alex Bennée]

Otherwise people won't know what they are missing.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/tcg/multiarch/Makefile.target | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target
index 1dd0f64d23..abbdb2e126 100644
--- a/tests/tcg/multiarch/Makefile.target
+++ b/tests/tcg/multiarch/Makefile.target
@@ -63,8 +63,11 @@ run-gdbstub-qxfer-auxv-read: sha1
 		--bin $< --test $(MULTIARCH_SRC)/gdbstub/test-qxfer-auxv-read.py, \
 	"basic gdbstub qXfer:auxv:read support")
 
-EXTRA_RUNS += run-gdbstub-sha1 run-gdbstub-qxfer-auxv-read
+else
+run-gdbstub-%:
+	$(call skip-test, "gdbstub test $*", "need working gdb")
 endif
+EXTRA_RUNS += run-gdbstub-sha1 run-gdbstub-qxfer-auxv-read
 
 
 # Update TESTS
-- 
2.20.1

[PATCH v1 10/15] gdbstub: Fix handle_query_xfer_auxv

[Alex Bennée]

From: Richard Henderson <richard.henderson@linaro.org>

The main problem was that we were treating a guest address
as a host address with a mere cast.

Use the correct interface for accessing guest memory.  Do not
allow offset == auxv_len, which would result in an empty packet.

Fixes: 51c623b0de1 ("gdbstub: add support to Xfer:auxv:read: packet")
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210128201831.534033-1-richard.henderson@linaro.org>
---
 gdbstub.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/gdbstub.c b/gdbstub.c
index c7ca7e9f88..759bb00bcf 100644
--- a/gdbstub.c
+++ b/gdbstub.c
@@ -2245,7 +2245,6 @@ static void handle_query_xfer_auxv(GdbCmdContext *gdb_ctx, void *user_ctx)
 {
     TaskState *ts;
     unsigned long offset, len, saved_auxv, auxv_len;
-    const char *mem;
 
     if (gdb_ctx->num_params < 2) {
         put_packet("E22");
@@ -2257,8 +2256,8 @@ static void handle_query_xfer_auxv(GdbCmdContext *gdb_ctx, void *user_ctx)
     ts = gdbserver_state.c_cpu->opaque;
     saved_auxv = ts->info->saved_auxv;
     auxv_len = ts->info->auxv_len;
-    mem = (const char *)(saved_auxv + offset);
-    if (offset > auxv_len) {
+
+    if (offset >= auxv_len) {
         put_packet("E00");
         return;
     }
@@ -2269,12 +2268,20 @@ static void handle_query_xfer_auxv(GdbCmdContext *gdb_ctx, void *user_ctx)
 
     if (len < auxv_len - offset) {
         g_string_assign(gdbserver_state.str_buf, "m");
-        memtox(gdbserver_state.str_buf, mem, len);
     } else {
         g_string_assign(gdbserver_state.str_buf, "l");
-        memtox(gdbserver_state.str_buf, mem, auxv_len - offset);
+        len = auxv_len - offset;
+    }
+
+    g_byte_array_set_size(gdbserver_state.mem_buf, len);
+    if (target_memory_rw_debug(gdbserver_state.g_cpu, saved_auxv + offset,
+                               gdbserver_state.mem_buf->data, len, false)) {
+        put_packet("E14");
+        return;
     }
 
+    memtox(gdbserver_state.str_buf,
+           (const char *)gdbserver_state.mem_buf->data, len);
     put_packet_binary(gdbserver_state.str_buf->str,
                       gdbserver_state.str_buf->len, true);
 }
-- 
2.20.1

[PATCH v1 11/15] tests/tcg: Replace /bin/true by true (required on macOS)

[Alex Bennée]

From: Stefan Weil <sw@weilnetz.de>

/bin/true is missing on macOS, but simply "true" is available as a shell builtin.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210128135627.2067003-1-sw@weilnetz.de>
---
 tests/tcg/Makefile.qemu | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/tcg/Makefile.qemu b/tests/tcg/Makefile.qemu
index c096c611a2..a56564660c 100644
--- a/tests/tcg/Makefile.qemu
+++ b/tests/tcg/Makefile.qemu
@@ -90,11 +90,11 @@ run-guest-tests: guest-tests
 
 else
 guest-tests:
-	$(call quiet-command, /bin/true, "BUILD", \
+	$(call quiet-command, true, "BUILD", \
 		"$(TARGET) guest-tests SKIPPED")
 
 run-guest-tests:
-	$(call quiet-command, /bin/true, "RUN", \
+	$(call quiet-command, true, "RUN", \
 		"tests for $(TARGET) SKIPPED")
 endif
 
-- 
2.20.1

[PATCH v1 12/15] scripts/mtest2make.py: export all-%s-targets variable and use it

[Alex Bennée]

There are some places where the conditional makefile support is the
simplest solution. Now we don't expose CONFIG_TCG as a variable create
a new one that can be checked for the check-help output.

As check-tcg is a PHONY target we re-use check-softfloat to gate that
as well.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210126145356.7860-2-alex.bennee@linaro.org>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
---
 scripts/mtest2make.py  | 1 +
 tests/Makefile.include | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
index 25ee6887cf..cbbcba100d 100644
--- a/scripts/mtest2make.py
+++ b/scripts/mtest2make.py
@@ -110,6 +110,7 @@ def emit_suite(name, suite, prefix):
     print('ifneq ($(filter %s %s, $(MAKECMDGOALS)),)' % (target, prefix))
     print('.tests += $(.test.$(SPEED).%s)' % (target, ))
     print('endif')
+    print('all-%s-targets += %s' % (prefix, target))
 
 targets = {t['id']: [os.path.relpath(f) for f in t['filename']]
            for t in introspect['targets']}
diff --git a/tests/Makefile.include b/tests/Makefile.include
index ceaf3f0d6e..17dafdfe98 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -12,7 +12,7 @@ check-help:
 	@echo " $(MAKE) check-speed          Run qobject speed tests"
 	@echo " $(MAKE) check-qapi-schema    Run QAPI schema tests"
 	@echo " $(MAKE) check-block          Run block tests"
-ifeq ($(CONFIG_TCG),y)
+ifneq ($(filter $(all-check-targets), check-softfloat),)
 	@echo " $(MAKE) check-tcg            Run TCG tests"
 	@echo " $(MAKE) check-softfloat      Run FPU emulation tests"
 endif
-- 
2.20.1

[PATCH v1 13/15] tests/Makefile.include: don't use TARGET_DIRS for check-tcg

[Alex Bennée]

TARGET_DIRS reflects what we wanted to configure which in the normal
case is all our targets. However once meson has pared-down our target
list due to missing features we need to check the final list of
ninja-targets. This prevents check-tcg barfing on a --disable-tcg
build.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210126145356.7860-3-alex.bennee@linaro.org>

---
v2
  - move everything to Makefile.include
---
 tests/Makefile.include | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 17dafdfe98..d34254fb29 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -40,11 +40,13 @@ SYSEMU_TARGET_LIST := $(subst -softmmu.mak,,$(notdir \
 
 SPEED = quick
 
-# Per guest TCG tests
+# Build up our target list from the filtered list of ninja targets
+TARGETS=$(patsubst libqemu-%.fa, %, $(filter libqemu-%.fa, $(ninja-targets)))
 
-BUILD_TCG_TARGET_RULES=$(patsubst %,build-tcg-tests-%, $(TARGET_DIRS))
-CLEAN_TCG_TARGET_RULES=$(patsubst %,clean-tcg-tests-%, $(TARGET_DIRS))
-RUN_TCG_TARGET_RULES=$(patsubst %,run-tcg-tests-%, $(TARGET_DIRS))
+# Per guest TCG tests
+BUILD_TCG_TARGET_RULES=$(patsubst %,build-tcg-tests-%, $(TARGETS))
+CLEAN_TCG_TARGET_RULES=$(patsubst %,clean-tcg-tests-%, $(TARGETS))
+RUN_TCG_TARGET_RULES=$(patsubst %,run-tcg-tests-%, $(TARGETS))
 
 # Probe for the Docker Builds needed for each build
 $(foreach PROBE_TARGET,$(TARGET_DIRS), 				\
-- 
2.20.1

[PATCH v1 14/15] docs/system: document an example vexpress-a15 invocation

[Alex Bennée]

The wiki and the web are curiously absent of the right runes to boot a
vexpress model so I had to work from first principles to work it out.
Use the more modern -drive notation so alternative backends can be
used (unlike the hardwired -sd mode).

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Anders Roxell <anders.roxell@linaro.org>

---
v2
  - reword kernel build.
---
 docs/system/arm/vexpress.rst | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/docs/system/arm/vexpress.rst b/docs/system/arm/vexpress.rst
index 7f1bcbef07..3e3839e923 100644
--- a/docs/system/arm/vexpress.rst
+++ b/docs/system/arm/vexpress.rst
@@ -58,3 +58,31 @@ Other differences between the hardware and the QEMU model:
   ``vexpress-a15``, and have IRQs from 40 upwards. If a dtb is
   provided on the command line then QEMU will edit it to include
   suitable entries describing these transports for the guest.
+
+Booting a Linux kernel
+----------------------
+
+Building a current Linux kernel with ``multi_v7_defconfig`` should be
+enough to get something running. Nowadays an out-of-tree build is
+recommended (and also useful if you build a lot of different targets).
+In the following example $BLD points to the build directory and $SRC
+points to the root of the Linux source tree. You can drop $SRC if you
+are running from there.
+
+.. code-block:: bash
+
+  $ make O=$BLD -C $SRC ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- multi_v7_defconfig
+  $ make O=$BLD -C $SRC ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf-
+
+By default you will want to boot your rootfs off the sdcard interface.
+Your rootfs will need to be padded to the right size. With a suitable
+DTB you could also add devices to the virtio-mmio bus.
+
+.. code-block:: bash
+
+  $ qemu-system-arm -cpu cortex-a15 -smp 4 -m 4096 \
+      -machine type=vexpress-a15 -serial mon:stdio \
+      -drive if=sd,driver=file,filename=armel-rootfs.ext4 \
+      -kernel zImage  \
+      -dtb vexpress-v2p-ca15-tc1.dtb \
+      -append "console=ttyAMA0 root=/dev/mmcblk0 ro"
-- 
2.20.1

[PATCH v1 15/15] docs/system: document an example booting the versatilepb machine

[Alex Bennée]

There is a bit more out there including Aurelien's excellent write up
and older Debian images here:

  https://www.aurel32.net/info/debian_arm_qemu.php
  https://people.debian.org/~aurel32/qemu/armel/

However the web is transitory and git is forever so lets add something
to the fine manual.

Cc: Anders Roxell <anders.roxell@linaro.org>
Cc: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v2
  -reword kernel build
---
 docs/system/arm/versatile.rst | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/docs/system/arm/versatile.rst b/docs/system/arm/versatile.rst
index 51221c30a4..2ae792bac3 100644
--- a/docs/system/arm/versatile.rst
+++ b/docs/system/arm/versatile.rst
@@ -27,3 +27,37 @@ The Arm Versatile baseboard is emulated with the following devices:
    devices.
 
 -  PL181 MultiMedia Card Interface with SD card.
+
+Booting a Linux kernel
+----------------------
+
+Building a current Linux kernel with ``versatile_defconfig`` should be
+enough to get something running. Nowadays an out-of-tree build is
+recommended (and also useful if you build a lot of different targets).
+In the following example $BLD points to the build directory and $SRC
+points to the root of the Linux source tree. You can drop $SRC if you
+are running from there.
+
+.. code-block:: bash
+
+  $ make O=$BLD -C $SRC ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- versatile_defconfig
+  $ make O=$BLD -C $SRC ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf-
+
+You may want to enable some additional modules if you want to boot
+something from the SCSI interface::
+
+  CONFIG_PCI=y
+  CONFIG_PCI_VERSATILE=y
+  CONFIG_SCSI=y
+  CONFIG_SCSI_SYM53C8XX_2=y
+
+You can then boot with a command line like:
+
+.. code-block:: bash
+
+  $ qemu-system-arm -machine type=versatilepb \
+      -serial mon:stdio \
+      -drive if=scsi,driver=file,filename=debian-buster-armel-rootfs.ext4 \
+      -kernel zImage \
+      -dtb versatile-pb.dtb  \
+      -append "console=ttyAMA0 ro root=/dev/sda"
-- 
2.20.1

Re: [PATCH v1 07/15] configure: make version_ge more tolerant of shady version input

[Eric Blake]

On 2/2/21 7:39 AM, Alex Bennée wrote:
> When checking GDB versions we have to tolerate all sorts of random
> distro extensions to the version string. While we already attempt to
> do some of that before we call version_ge is makes sense to try and
> regularise the first input by stripping extraneous -'s. While we at it
> convert the old-style shell quoting into a cleaner form t shut up my
> editors linter lest it confuse me by underlining the whole line.
> 
> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Tested-by: Thomas Huth <thuth@redhat.com>
> ---
>  configure | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/configure b/configure
> index e85d6baf8f..432b83fadf 100755
> --- a/configure
> +++ b/configure
> @@ -198,8 +198,8 @@ has() {
>  }
>  
>  version_ge () {
> -    local_ver1=`echo $1 | tr . ' '`
> -    local_ver2=`echo $2 | tr . ' '`
> +    local_ver1=$(expr "$1" : '\([0-9.]*\)' | tr . ' ')
> +    local_ver2=$(echo "$2" | tr . ' ')
>      while true; do
>          set x $local_ver1
>          local_first=${2-0}
> 

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [PATCH v1 05/15] tests/docker: alias docker-help target for consistency

[Philippe Mathieu-Daudé]

On 2/2/21 2:39 PM, Alex Bennée wrote:
> We have a bunch of -help targets so this will save some cognitive
> dissonance. Keep the original for those with muscle memory.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> 
> ---
> v2
>   - also fix help-on-help text
> ---
>  Makefile                      | 2 +-
>  tests/docker/Makefile.include | 4 +++-
>  2 files changed, 4 insertions(+), 2 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Re: [PATCH v1 11/15] tests/tcg: Replace /bin/true by true (required on macOS)

[Philippe Mathieu-Daudé]

On 2/2/21 2:39 PM, Alex Bennée wrote:
> From: Stefan Weil <sw@weilnetz.de>
> 
> /bin/true is missing on macOS, but simply "true" is available as a shell builtin.
> 
> Signed-off-by: Stefan Weil <sw@weilnetz.de>

Reviewed ... :
https://www.mail-archive.com/qemu-devel@nongnu.org/msg777004.html
... before Peter:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg777010.html

> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>

His tag is and mine isn't... Is it possible this is a b4 bug when
names use UTF-8 encoding?

Again:
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20210128135627.2067003-1-sw@weilnetz.de>
> ---
>  tests/tcg/Makefile.qemu | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/tcg/Makefile.qemu b/tests/tcg/Makefile.qemu
> index c096c611a2..a56564660c 100644
> --- a/tests/tcg/Makefile.qemu
> +++ b/tests/tcg/Makefile.qemu
> @@ -90,11 +90,11 @@ run-guest-tests: guest-tests
>  
>  else
>  guest-tests:
> -	$(call quiet-command, /bin/true, "BUILD", \
> +	$(call quiet-command, true, "BUILD", \
>  		"$(TARGET) guest-tests SKIPPED")
>  
>  run-guest-tests:
> -	$(call quiet-command, /bin/true, "RUN", \
> +	$(call quiet-command, true, "RUN", \
>  		"tests for $(TARGET) SKIPPED")
>  endif
>  
> 

Re: [PATCH v1 11/15] tests/tcg: Replace /bin/true by true (required on macOS)

[Philippe Mathieu-Daudé]

On Tue, Feb 2, 2021 at 3:23 PM Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> On 2/2/21 2:39 PM, Alex Bennée wrote:
> > From: Stefan Weil <sw@weilnetz.de>
> >
> > /bin/true is missing on macOS, but simply "true" is available as a shell builtin.
> >
> > Signed-off-by: Stefan Weil <sw@weilnetz.de>
>
> Reviewed ... :
> https://www.mail-archive.com/qemu-devel@nongnu.org/msg777004.html
> ... before Peter:
> https://www.mail-archive.com/qemu-devel@nongnu.org/msg777010.html
>
> > Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
>
> His tag is and mine isn't... Is it possible this is a b4 bug when
> names use UTF-8 encoding?

FWIW the version I'm using works:

$ b4 --version
0.5.2

$ b4 am 20210128135627.2067003-1-sw@weilnetz.de
Looking up https://lore.kernel.org/r/20210128135627.2067003-1-sw%40weilnetz.de
Grabbing thread from lore.kernel.org/qemu-devel
Analyzing 4 messages in the thread
---
Writing ./20210128_sw_tests_tcg_replace_bin_true_by_true_required_on_macos.mbx
 [PATCH] tests/tcg: Replace /bin/true by true (required on macOS)
   + Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
   + Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
Total patches: 1
---
Link: https://lore.kernel.org/r/20210128135627.2067003-1-sw@weilnetz.de
Base: not found
      git am ./20210128_sw_tests_tcg_replace_bin_true_by_true_required_on_macos.mbx

> Again:
> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>
> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> > Message-Id: <20210128135627.2067003-1-sw@weilnetz.de>
> > ---
> >  tests/tcg/Makefile.qemu | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)

Re: [PATCH v1 11/15] tests/tcg: Replace /bin/true by true (required on macOS)

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> On Tue, Feb 2, 2021 at 3:23 PM Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>> On 2/2/21 2:39 PM, Alex Bennée wrote:
>> > From: Stefan Weil <sw@weilnetz.de>
>> >
>> > /bin/true is missing on macOS, but simply "true" is available as a shell builtin.
>> >
>> > Signed-off-by: Stefan Weil <sw@weilnetz.de>
>>
>> Reviewed ... :
>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg777004.html
>> ... before Peter:
>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg777010.html
>>
>> > Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
>>
>> His tag is and mine isn't... Is it possible this is a b4 bug when
>> names use UTF-8 encoding?
>
> FWIW the version I'm using works:
>
> $ b4 --version
> 0.5.2

Same here... the only thing I can think of was I applied it old style
and your email hadn't been indexed yet (I have Peter's rb in my
review.org). I do generally use b4 to apply series now (as I will this
one when I roll). Sometimes however individual patches get applied
directly if I don't see anything in my email.

It would be nice if b4 could run on across an existing git tree to see
if things can be applied based on tags that may have arrived in the
interim. I think there is a potential race there while a series is
prepared.

> $ b4 am 20210128135627.2067003-1-sw@weilnetz.de
> Looking up https://lore.kernel.org/r/20210128135627.2067003-1-sw%40weilnetz.de
> Grabbing thread from lore.kernel.org/qemu-devel
> Analyzing 4 messages in the thread
> ---
> Writing ./20210128_sw_tests_tcg_replace_bin_true_by_true_required_on_macos.mbx
>  [PATCH] tests/tcg: Replace /bin/true by true (required on macOS)
>    + Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
>    + Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> Total patches: 1
> ---
> Link: https://lore.kernel.org/r/20210128135627.2067003-1-sw@weilnetz.de
> Base: not found
>       git am ./20210128_sw_tests_tcg_replace_bin_true_by_true_required_on_macos.mbx
>
>> Again:
>> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>
>> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> > Message-Id: <20210128135627.2067003-1-sw@weilnetz.de>
>> > ---
>> >  tests/tcg/Makefile.qemu | 4 ++--
>> >  1 file changed, 2 insertions(+), 2 deletions(-)


-- 
Alex Bennée

Re: [PATCH v1 11/15] tests/tcg: Replace /bin/true by true (required on macOS)

[Alex Bennée]

Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> On Tue, Feb 2, 2021 at 3:23 PM Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>> On 2/2/21 2:39 PM, Alex Bennée wrote:
>> > From: Stefan Weil <sw@weilnetz.de>
>> >
>> > /bin/true is missing on macOS, but simply "true" is available as a shell builtin.
>> >
>> > Signed-off-by: Stefan Weil <sw@weilnetz.de>
>>
>> Reviewed ... :
>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg777004.html
>> ... before Peter:
>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg777010.html
>>
>> > Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
<snip>

It turns out there isn't an easy way to extract the information direct
from b4 so I just scripted it into Emacs:

  (defun my-git-check-for-updates-with-b4 (id subject)
    "Fetch `id' via the b4 tool and check for new tags."
    (let ((tags)
          (add-dco (rx (: "+ " (group (regexp my-bare-dco-tag-re))))))
      (with-temp-buffer
        (call-process "b4" nil t t "am" id "-o" "-")
        (goto-char 0)
        (when (re-search-forward subject nil t)
          (forward-line)
          (beginning-of-line)
          (while (string-match add-dco (thing-at-point 'line))
            (push (match-string-no-properties 1 (thing-at-point 'line)) tags)
            (forward-line))))
      tags))

  (defun my-commit-update-with-b4 ()
    "Check if the current commit has tags from it's last posting.

  This only works if there is a message id in the buffer to search for."
    (interactive)
    (let ((subj) (id))
      (save-excursion
        (goto-char 0)
        (setq subj (substring-no-properties (thing-at-point 'line)))
        (if (re-search-forward my-capture-msgid-re nil t)
            (setq id (match-string-no-properties 1))))
      (when (and subj id)
        (let ((tags (my-git-check-for-updates-with-b4 id subj)))
          (--map
           (save-excursion
             (goto-char 0)
             (when (not (re-search-forward it nil t))
               (re-search-forward my-capture-msgid-re nil t)
               (beginning-of-line)
               (insert it ?\n))) tags)))))


-- 
Alex Bennée