[PATCH v1 00/15] testing and plugin updates

[PATCH v1 00/15] testing and plugin updates

[Alex Bennée]

Hi,

Here is the current status of various testing and plugin related
patches. Most of the patches should be uncontroversial although I
would draw attention to the cpu and linux-user QOM object lifetime
patches. I think properly de-referencing the vCPU object associated
with the linux-user thread is the cleaner solution but it does involve
also changing some assumptions about the cpu indexing.

The following patches need review:
  - plugins: new lockstep plugin for debugging TCG changes
  - linux-user: properly "unrealize" vCPU object
  - cpus-common: ensure auto-assigned cpu_indexes don't clash
  - tests/docker: use a gcc-10 based image for arm64 tests
  - tests/docker: add debian11 base image
  - tests/docker: bump fedora to 32
  - tests/tcg: better detect confused gdb which can't connect
  - tests/fp: split and audit the conversion tests
  - tests/fp: enable extf80_le_quite tests
  - configure: add alternate binary for genisoimage

Alex Bennée (13):
  configure: add alternate binary for genisoimage
  tests/vm: pass --genisoimage to basevm script
  tests/tcg: fix invocation of the memory record/replay tests
  tests/fp: enable extf80_le_quite tests
  tests/fp: split and audit the conversion tests
  tests/tcg: better detect confused gdb which can't connect
  tests/docker: bump fedora to 32
  tests/docker: add debian11 base image
  tests/docker: use a gcc-10 based image for arm64 tests
  cpus-common: ensure auto-assigned cpu_indexes don't clash
  linux-user: properly "unrealize" vCPU object
  tests/tcg: add new threadcount test
  plugins: new lockstep plugin for debugging TCG changes

Richard Henderson (1):
  tests/tcg/aarch64: Add bti smoke test

Thomas Huth (1):
  travis.yml: Use clang++ in the Clang tests

 configure                                     |   2 +-
 cpus-common.c                                 |  10 +-
 linux-user/syscall.c                          |  19 +-
 tests/plugin/lockstep.c                       | 340 ++++++++++++++++++
 tests/tcg/aarch64/bti-1.c                     |  62 ++++
 tests/tcg/aarch64/bti-crt.inc.c               |  51 +++
 tests/tcg/multiarch/threadcount.c             |  64 ++++
 .travis.yml                                   |   8 +-
 tests/Makefile.include                        |  44 ++-
 tests/docker/Makefile.include                 |   4 +-
 .../debian-arm64-test-cross.docker            |  13 +
 tests/docker/dockerfiles/debian11.docker      |  18 +
 tests/docker/dockerfiles/fedora.docker        |   2 +-
 tests/plugin/Makefile                         |   1 +
 tests/tcg/Makefile.target                     |   2 +-
 tests/tcg/aarch64/Makefile.softmmu-target     |   2 +-
 tests/tcg/aarch64/Makefile.target             |   9 +
 tests/tcg/configure.sh                        |   8 +-
 tests/tcg/multiarch/Makefile.target           |   2 +
 tests/tcg/multiarch/gdbstub/sha1.py           |   4 +
 tests/vm/Makefile.include                     |   1 +
 tests/vm/basevm.py                            |  16 +-
 22 files changed, 641 insertions(+), 41 deletions(-)
 create mode 100644 tests/plugin/lockstep.c
 create mode 100644 tests/tcg/aarch64/bti-1.c
 create mode 100644 tests/tcg/aarch64/bti-crt.inc.c
 create mode 100644 tests/tcg/multiarch/threadcount.c
 create mode 100644 tests/docker/dockerfiles/debian-arm64-test-cross.docker
 create mode 100644 tests/docker/dockerfiles/debian11.docker

-- 
2.20.1

[PATCH v1 01/15] configure: add alternate binary for genisoimage

[Alex Bennée]

Not all distros ship genisoimage which is a Debian fork from the
original cdrtools. As the options are pretty much the same support it
as a fallback binary.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 configure | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure b/configure
index 26084fc53ad..a574250524f 100755
--- a/configure
+++ b/configure
@@ -941,7 +941,7 @@ done
 
 # Check for ancillary tools used in testing
 genisoimage=
-for binary in genisoimage
+for binary in genisoimage mkisofs
 do
     if has $binary
     then
-- 
2.20.1

[PATCH v1 02/15] tests/vm: pass --genisoimage to basevm script

[Alex Bennée]

If we have an alternative to genisoimage we really need to tell the
script about it as well so it can use it. It will still default to
genisoimage in case it is run outside our build machinery.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 tests/vm/Makefile.include |  1 +
 tests/vm/basevm.py        | 16 ++++++++++------
 2 files changed, 11 insertions(+), 6 deletions(-)

diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index 1bf9693d195..74ab522c55d 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -56,6 +56,7 @@ $(IMAGES_DIR)/%.img:	$(SRC_PATH)/tests/vm/% \
 	$(call quiet-command, \
 		$(PYTHON) $< \
 		$(if $(V)$(DEBUG), --debug) \
+		$(if $(GENISOIMAGE),--genisoimage $(GENISOIMAGE)) \
 		--image "$@" \
 		--force \
 		--build-image $@, \
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 756ccf7acae..a2d4054d72b 100644
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -61,8 +61,9 @@ class BaseVM(object):
     # 4 is arbitrary, but greater than 2,
     # since we found we need to wait more than twice as long.
     tcg_ssh_timeout_multiplier = 4
-    def __init__(self, debug=False, vcpus=None):
+    def __init__(self, debug=False, vcpus=None, genisoimage=None):
         self._guest = None
+        self._genisoimage = genisoimage
         self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-",
                                                          suffix=".tmp",
                                                          dir="."))
@@ -381,12 +382,12 @@ class BaseVM(object):
             udata.writelines(["apt:\n",
                               "  proxy: %s" % proxy])
         udata.close()
-        subprocess.check_call(["genisoimage", "-output", "cloud-init.iso",
+        subprocess.check_call([self._genisoimage, "-output", "cloud-init.iso",
                                "-volid", "cidata", "-joliet", "-rock",
                                "user-data", "meta-data"],
-                               cwd=cidir,
-                               stdin=self._devnull, stdout=self._stdout,
-                               stderr=self._stdout)
+                              cwd=cidir,
+                              stdin=self._devnull, stdout=self._stdout,
+                              stderr=self._stdout)
 
         return os.path.join(cidir, "cloud-init.iso")
 
@@ -424,6 +425,8 @@ def parse_args(vmcls):
                       help="Interactively run command")
     parser.add_option("--snapshot", "-s", action="store_true",
                       help="run tests with a snapshot")
+    parser.add_option("--genisoimage", default="genisoimage",
+                      help="iso imaging tool")
     parser.disable_interspersed_args()
     return parser.parse_args()
 
@@ -435,7 +438,8 @@ def main(vmcls):
             return 1
         logging.basicConfig(level=(logging.DEBUG if args.debug
                                    else logging.WARN))
-        vm = vmcls(debug=args.debug, vcpus=args.jobs)
+        vm = vmcls(debug=args.debug, vcpus=args.jobs,
+                   genisoimage=args.genisoimage)
         if args.build_image:
             if os.path.exists(args.image) and not args.force:
                 sys.stderr.writelines(["Image file exists: %s\n" % args.image,
-- 
2.20.1

[PATCH v1 03/15] travis.yml: Use clang++ in the Clang tests

[Alex Bennée]

From: Thomas Huth <thuth@redhat.com>

Our configure script does not look for clang++ automatically, so we
should use --cxx=clang++ to make sure that we test our C++ code with
Clang, too. And while we're at it, also use --host-cc=clang here
to avoid that we use the normal "cc" as host C compiler.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200518083316.25065-1-thuth@redhat.com>
---
 .travis.yml | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 1ec8a7b4657..564be50a3c1 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -205,14 +205,15 @@ jobs:
     # Test with Clang for compile portability (Travis uses clang-5.0)
     - name: "Clang (user)"
       env:
-        - CONFIG="--disable-system"
+        - CONFIG="--disable-system --host-cc=clang --cxx=clang++"
         - CACHE_NAME="${TRAVIS_BRANCH}-linux-clang-default"
       compiler: clang
 
 
     - name: "Clang (main-softmmu)"
       env:
-        - CONFIG="--target-list=${MAIN_SOFTMMU_TARGETS} "
+        - CONFIG="--target-list=${MAIN_SOFTMMU_TARGETS}
+                  --host-cc=clang --cxx=clang++"
         - CACHE_NAME="${TRAVIS_BRANCH}-linux-clang-sanitize"
       compiler: clang
       before_script:
@@ -222,7 +223,8 @@ jobs:
 
     - name: "Clang (other-softmmu)"
       env:
-        - CONFIG="--disable-user --target-list-exclude=${MAIN_SOFTMMU_TARGETS}"
+        - CONFIG="--disable-user --target-list-exclude=${MAIN_SOFTMMU_TARGETS}
+                  --host-cc=clang --cxx=clang++"
         - CACHE_NAME="${TRAVIS_BRANCH}-linux-clang-default"
       compiler: clang
 
-- 
2.20.1

[PATCH v1 04/15] tests/tcg: fix invocation of the memory record/replay tests

[Alex Bennée]

I'm not sure when this broke but we should use EXTRA_RUNS for
"virtual" tests which are not generated from the binary names.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

---
v2
  - keep the PHONY's for the benefit of output
---
 tests/tcg/aarch64/Makefile.softmmu-target | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/tcg/aarch64/Makefile.softmmu-target b/tests/tcg/aarch64/Makefile.softmmu-target
index 71f72cfbe34..1057a8ac498 100644
--- a/tests/tcg/aarch64/Makefile.softmmu-target
+++ b/tests/tcg/aarch64/Makefile.softmmu-target
@@ -61,7 +61,7 @@ run-memory-replay: memory-replay run-memory-record
 	   	  $(QEMU_OPTS) memory, \
 	  "$< on $(TARGET_NAME)")
 
-EXTRA_TESTS+=memory-record memory-replay
+EXTRA_RUNS+=run-memory-replay
 
 ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_ARMV8_3),)
 pauth-3: CFLAGS += -march=armv8.3-a
-- 
2.20.1

[PATCH v1 05/15] tests/fp: enable extf80_le_quite tests

[Alex Bennée]

These have been fixed now so we no longer need a special version of
the le_quiet rule to skip the test.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/Makefile.include | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 03a74b60f6b..e6d87fcbf0e 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -741,13 +741,6 @@ check-softfloat-%: $(FP_TEST_BIN)
 SF_COMPARE_OPS=eq eq_signaling le le_quiet lt_quiet
 SF_COMPARE_RULES=$(patsubst %,check-softfloat-%, $(SF_COMPARE_OPS))
 
-# FIXME: extF80_le_quiet (broken)
-check-softfloat-le_quiet: $(FP_TEST_BIN)
-	$(call test-softfloat, 				\
-		f16_le_quiet f32_le_quiet f64_le_quiet  \
-		f128_le_quiet, 				\
-		le_quiet)
-
 # FIXME: extF80_lt_quiet (broken)
 check-softfloat-lt_quiet: $(FP_TEST_BIN)
 	$(call test-softfloat, 				\
-- 
2.20.1

[PATCH v1 06/15] tests/fp: split and audit the conversion tests

[Alex Bennée]

Split the float conversion tests into separate groups and audit the
tests to check what is still broken. I was able to enable a bunch of
tests that had been missed before:

  all the float to float conversions
  ui32_to_extF80
  ui64_to_extF80
  extF80_to_ui32
  extF80_to_ui32_r_minMag
  extF80_to_ui64
  extF80_to_ui64_r_minMag

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/Makefile.include | 37 ++++++++++++++++++++++++++++++++-----
 1 file changed, 32 insertions(+), 5 deletions(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index e6d87fcbf0e..a00ccc94b8b 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -687,11 +687,26 @@ test-softfloat = $(call quiet-command, \
 			(cat $2.out && exit 1;), \
 			"FLOAT TEST", $2)
 
-# Conversion Routines:
+# Conversion Routines: Float to Float
+# FIXME: f32_to_f128 (broken), f64_to_f128 (broken)
+# FIXME: f128_to_f32(broken), f128_to_f64 (broken)
+# FIXME: f128_to_extF80 (broken)
+check-softfloat-conv-f2f: $(FP_TEST_BIN)
+	$(call test-softfloat, \
+		f16_to_f32 f16_to_f64 \
+		f16_to_extF80 f16_to_f128 \
+		f32_to_f16 f32_to_f64 \
+		f32_to_extF80 \
+		f64_to_f16 f64_to_f32 \
+		extF80_to_f16 extF80_to_f32 \
+		extF80_to_f64 extF80_to_f128 \
+		f128_to_f16, \
+		float-to-float)
+
+# Conversion Routines: Int and Uint to Float
 # FIXME: i32_to_extF80 (broken), i64_to_extF80 (broken)
-#        ui32_to_f128 (not implemented), extF80_roundToInt (broken)
-#
-check-softfloat-conv: $(FP_TEST_BIN)
+#        ui32_to_f128 (not implemented)
+check-softfloat-conv-to-float: $(FP_TEST_BIN)
 	$(call test-softfloat, \
 		i32_to_f16 i64_to_f16 \
 		i32_to_f32 i64_to_f32 \
@@ -701,7 +716,12 @@ check-softfloat-conv: $(FP_TEST_BIN)
 		ui32_to_f16 ui64_to_f16 \
 		ui32_to_f32 ui64_to_f32 \
 		ui32_to_f64 ui64_to_f64 \
+		ui32_to_extF80 ui64_to_extF80 \
 		ui64_to_f128, uint-to-float)
+
+# Conversion Routines: Float to integers
+# FIXME: extF80_roundToInt (broken)
+check-softfloat-conv-to-int: $(FP_TEST_BIN)
 	$(call test-softfloat, \
 		f16_to_i32 f16_to_i32_r_minMag \
 		f32_to_i32 f32_to_i32_r_minMag \
@@ -718,10 +738,12 @@ check-softfloat-conv: $(FP_TEST_BIN)
 		f16_to_ui32 f16_to_ui32_r_minMag \
 		f32_to_ui32 f32_to_ui32_r_minMag \
 		f64_to_ui32 f64_to_ui32_r_minMag \
+		extF80_to_ui32 extF80_to_ui32_r_minMag \
 		f128_to_ui32 f128_to_ui32_r_minMag \
 		f16_to_ui64 f16_to_ui64_r_minMag \
 		f32_to_ui64 f32_to_ui64_r_minMag \
 		f64_to_ui64 f64_to_ui64_r_minMag \
+		extF80_to_ui64 extF80_to_ui64_r_minMag \
 		f128_to_ui64 f128_to_ui64_r_minMag, \
 		float-to-uint)
 	$(call test-softfloat, \
@@ -729,9 +751,14 @@ check-softfloat-conv: $(FP_TEST_BIN)
 		f64_roundToInt f128_roundToInt, \
 		round-to-integer)
 
+.PHONY: check-softfloat-conv
+check-softfloat-conv: check-softfloat-conv-f2f
+check-softfloat-conv: check-softfloat-conv-to-float
+check-softfloat-conv: check-softfloat-conv-to-int
+
 # Generic rule for all float operations
 #
-# Some patterns are overidden due to broken or missing tests.
+# Some patterns are overridden due to broken or missing tests.
 # Hopefully these can be removed over time.
 
 check-softfloat-%: $(FP_TEST_BIN)
-- 
2.20.1

[PATCH v1 07/15] tests/tcg: better detect confused gdb which can't connect

[Alex Bennée]

While we may gamely give the right information it can still confuse
the wide range of GDBs out there. For example ppc64abi32-linux-user
reports:

  warning: Selected architecture powerpc:common is not compatible with reported target architecture powerpc:common64
  warning: Architecture rejected target-supplied description

but still connects. Add a test for a 0 pc and exit early if that is
the case. This may actually be a bug we need to fix?

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Richard Henderson <rth@twiddle.net>
---
 tests/tcg/multiarch/gdbstub/sha1.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tests/tcg/multiarch/gdbstub/sha1.py b/tests/tcg/multiarch/gdbstub/sha1.py
index 734553b98bb..2bfde49633c 100644
--- a/tests/tcg/multiarch/gdbstub/sha1.py
+++ b/tests/tcg/multiarch/gdbstub/sha1.py
@@ -65,6 +65,10 @@ except (gdb.error, AttributeError):
     print("SKIPPING (not connected)", file=sys.stderr)
     exit(0)
 
+if gdb.parse_and_eval('$pc') == 0:
+    print("SKIP: PC not set")
+    exit(0)
+
 try:
     # These are not very useful in scripts
     gdb.execute("set pagination off")
-- 
2.20.1

[PATCH v1 08/15] tests/docker: bump fedora to 32

[Alex Bennée]

We should be keeping this up to date as Fedora goes out of support
quite quickly.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/dockerfiles/fedora.docker | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker
index 179575ecaaa..92204dc79eb 100644
--- a/tests/docker/dockerfiles/fedora.docker
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -1,4 +1,4 @@
-FROM fedora:30
+FROM fedora:32
 
 # Please keep this list sorted alphabetically
 ENV PACKAGES \
-- 
2.20.1

[PATCH v1 09/15] tests/docker: add debian11 base image

[Alex Bennée]

We won't use this for building QEMU but we do need newer GCC's and
binutils for building some of our test cases.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include            |  2 +-
 tests/docker/dockerfiles/debian11.docker | 18 ++++++++++++++++++
 2 files changed, 19 insertions(+), 1 deletion(-)
 create mode 100644 tests/docker/dockerfiles/debian11.docker

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 43a86786882..3596b589307 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -7,7 +7,7 @@ HOST_ARCH = $(if $(ARCH),$(ARCH),$(shell uname -m))
 DOCKER_SUFFIX := .docker
 DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
 # we don't run tests on intermediate images (used as base by another image)
-DOCKER_PARTIAL_IMAGES := debian9 debian10
+DOCKER_PARTIAL_IMAGES := debian9 debian10 debian11
 DOCKER_PARTIAL_IMAGES += debian9-mxe debian-bootstrap
 DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker))))
 DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
diff --git a/tests/docker/dockerfiles/debian11.docker b/tests/docker/dockerfiles/debian11.docker
new file mode 100644
index 00000000000..5adfd62d553
--- /dev/null
+++ b/tests/docker/dockerfiles/debian11.docker
@@ -0,0 +1,18 @@
+#
+# Docker multiarch cross-compiler target
+#
+# This docker target uses the current development version of Debian as
+# a base for cross compilers for building test binaries. We won't
+# attempt to build QEMU on it yet given it is still in development.
+#
+# On its own you can't build much but the docker-foo-cross targets
+# build on top of the base debian image.
+#
+FROM debian:bullseye-slim
+
+# Duplicate deb line as deb-src
+RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
+
+# Install common build utilities
+RUN apt update && \
+    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata
-- 
2.20.1

[PATCH v1 10/15] tests/docker: use a gcc-10 based image for arm64 tests

[Alex Bennée]

As we enable newer features that we want to test on arm64 targets we
need newer compilers. Split off a new debian-arm64-test-cross image
which we can use to build these new tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/Makefile.include                       |  2 ++
 .../dockerfiles/debian-arm64-test-cross.docker      | 13 +++++++++++++
 tests/tcg/configure.sh                              |  4 ++--
 3 files changed, 17 insertions(+), 2 deletions(-)
 create mode 100644 tests/docker/dockerfiles/debian-arm64-test-cross.docker

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 3596b589307..ed46bd98eb5 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -131,9 +131,11 @@ docker-image-travis: NOUSER=1
 
 # Specialist build images, sometimes very limited tools
 docker-image-tricore-cross: docker-image-debian9
+docker-image-debian-arm64-test-cross: docker-image-debian11
 
 # These images may be good enough for building tests but not for test builds
 DOCKER_PARTIAL_IMAGES += debian-alpha-cross
+DOCKER_PARTIAL_IMAGES += debian-arm64-test-cross
 DOCKER_PARTIAL_IMAGES += debian-hppa-cross
 DOCKER_PARTIAL_IMAGES += debian-m68k-cross debian-mips64-cross
 DOCKER_PARTIAL_IMAGES += debian-powerpc-cross debian-ppc64-cross
diff --git a/tests/docker/dockerfiles/debian-arm64-test-cross.docker b/tests/docker/dockerfiles/debian-arm64-test-cross.docker
new file mode 100644
index 00000000000..a44e76d9421
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-arm64-test-cross.docker
@@ -0,0 +1,13 @@
+#
+# Docker arm64 cross-compiler target (tests only)
+#
+# This docker target builds on the debian Bullseye base image.
+#
+FROM qemu:debian11
+
+# Add the foreign architecture we want and install dependencies
+RUN dpkg --add-architecture arm64
+RUN apt update && \
+    DEBIAN_FRONTEND=noninteractive eatmydata \
+        apt install -y --no-install-recommends \
+        crossbuild-essential-arm64 gcc-10-aarch64-linux-gnu
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index eaaaff6233a..2326f978562 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -97,8 +97,8 @@ for target in $target_list; do
   case $target in
     aarch64-*)
       # We don't have any bigendian build tools so we only use this for AArch64
-      container_image=debian-arm64-cross
-      container_cross_cc=aarch64-linux-gnu-gcc
+      container_image=debian-arm64-test-cross
+      container_cross_cc=aarch64-linux-gnu-gcc-10
       ;;
     alpha-*)
       container_image=debian-alpha-cross
-- 
2.20.1

[PATCH v1 11/15] tests/tcg/aarch64: Add bti smoke test

[Alex Bennée]

From: Richard Henderson <richard.henderson@linaro.org>

Requires gcc 10 for -mbranch-protection=standard.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200520023139.9812-3-richard.henderson@linaro.org>
[AJB: add CROSS_CC_HAS_ARMV8_BTI check]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/tcg/aarch64/bti-1.c         | 62 +++++++++++++++++++++++++++++++
 tests/tcg/aarch64/bti-crt.inc.c   | 51 +++++++++++++++++++++++++
 tests/tcg/aarch64/Makefile.target |  9 +++++
 tests/tcg/configure.sh            |  4 ++
 4 files changed, 126 insertions(+)
 create mode 100644 tests/tcg/aarch64/bti-1.c
 create mode 100644 tests/tcg/aarch64/bti-crt.inc.c

diff --git a/tests/tcg/aarch64/bti-1.c b/tests/tcg/aarch64/bti-1.c
new file mode 100644
index 00000000000..61924f0d7a4
--- /dev/null
+++ b/tests/tcg/aarch64/bti-1.c
@@ -0,0 +1,62 @@
+/*
+ * Branch target identification, basic notskip cases.
+ */
+
+#include "bti-crt.inc.c"
+
+static void skip2_sigill(int sig, siginfo_t *info, ucontext_t *uc)
+{
+    uc->uc_mcontext.pc += 8;
+    uc->uc_mcontext.pstate = 1;
+}
+
+#define NOP       "nop"
+#define BTI_N     "hint #32"
+#define BTI_C     "hint #34"
+#define BTI_J     "hint #36"
+#define BTI_JC    "hint #38"
+
+#define BTYPE_1(DEST) \
+    asm("mov %0,#1; adr x16, 1f; br x16; 1: " DEST "; mov %0,#0" \
+        : "=r"(skipped) : : "x16")
+
+#define BTYPE_2(DEST) \
+    asm("mov %0,#1; adr x16, 1f; blr x16; 1: " DEST "; mov %0,#0" \
+        : "=r"(skipped) : : "x16", "x30")
+
+#define BTYPE_3(DEST) \
+    asm("mov %0,#1; adr x15, 1f; br x15; 1: " DEST "; mov %0,#0" \
+        : "=r"(skipped) : : "x15")
+
+#define TEST(WHICH, DEST, EXPECT) \
+    do { WHICH(DEST); fail += skipped ^ EXPECT; } while (0)
+
+
+int main()
+{
+    int fail = 0;
+    int skipped;
+
+    /* Signal-like with SA_SIGINFO.  */
+    signal_info(SIGILL, skip2_sigill);
+
+    TEST(BTYPE_1, NOP, 1);
+    TEST(BTYPE_1, BTI_N, 1);
+    TEST(BTYPE_1, BTI_C, 0);
+    TEST(BTYPE_1, BTI_J, 0);
+    TEST(BTYPE_1, BTI_JC, 0);
+
+    TEST(BTYPE_2, NOP, 1);
+    TEST(BTYPE_2, BTI_N, 1);
+    TEST(BTYPE_2, BTI_C, 0);
+    TEST(BTYPE_2, BTI_J, 1);
+    TEST(BTYPE_2, BTI_JC, 0);
+
+    TEST(BTYPE_3, NOP, 1);
+    TEST(BTYPE_3, BTI_N, 1);
+    TEST(BTYPE_3, BTI_C, 1);
+    TEST(BTYPE_3, BTI_J, 0);
+    TEST(BTYPE_3, BTI_JC, 0);
+
+    return fail;
+}
diff --git a/tests/tcg/aarch64/bti-crt.inc.c b/tests/tcg/aarch64/bti-crt.inc.c
new file mode 100644
index 00000000000..ef7831ad76a
--- /dev/null
+++ b/tests/tcg/aarch64/bti-crt.inc.c
@@ -0,0 +1,51 @@
+/*
+ * Minimal user-environment for testing BTI.
+ *
+ * Normal libc is not built with BTI support enabled, and so could
+ * generate a BTI TRAP before ever reaching main.
+ */
+
+#include <stdlib.h>
+#include <signal.h>
+#include <ucontext.h>
+#include <asm/unistd.h>
+
+int main(void);
+
+void _start(void)
+{
+    exit(main());
+}
+
+void exit(int ret)
+{
+    register int x0 __asm__("x0") = ret;
+    register int x8 __asm__("x8") = __NR_exit;
+
+    asm volatile("svc #0" : : "r"(x0), "r"(x8));
+    __builtin_unreachable();
+}
+
+/*
+ * Irritatingly, the user API struct sigaction does not match the
+ * kernel API struct sigaction.  So for simplicity, isolate the
+ * kernel ABI here, and make this act like signal.
+ */
+void signal_info(int sig, void (*fn)(int, siginfo_t *, ucontext_t *))
+{
+    struct kernel_sigaction {
+        void (*handler)(int, siginfo_t *, ucontext_t *);
+        unsigned long flags;
+        unsigned long restorer;
+        unsigned long mask;
+    } sa = { fn, SA_SIGINFO, 0, 0 };
+
+    register int x0 __asm__("x0") = sig;
+    register void *x1 __asm__("x1") = &sa;
+    register void *x2 __asm__("x2") = 0;
+    register int x3 __asm__("x3") = sizeof(unsigned long);
+    register int x8 __asm__("x8") = __NR_rt_sigaction;
+
+    asm volatile("svc #0"
+                 : : "r"(x0), "r"(x1), "r"(x2), "r"(x3), "r"(x8) : "memory");
+}
diff --git a/tests/tcg/aarch64/Makefile.target b/tests/tcg/aarch64/Makefile.target
index 312f36cde5f..8d39f4b2f45 100644
--- a/tests/tcg/aarch64/Makefile.target
+++ b/tests/tcg/aarch64/Makefile.target
@@ -24,6 +24,15 @@ run-pauth-%: QEMU_OPTS += -cpu max
 pauth-%: CFLAGS += -march=armv8.3-a
 endif
 
+# We need binutils-2.32 to link this test case without warnings.
+# And if we know that we have 2.32, we also don't need the linker script.
+# In the meantime, disable the test by default.
+ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_ARMV8_BTI),)
+AARCH64_TESTS += bti-1
+bti-%: CFLAGS += -mbranch-protection=standard
+bti-1: LDFLAGS += -nostdlib
+endif
+
 # Semihosting smoke test for linux-user
 AARCH64_TESTS += semihosting
 run-semihosting: semihosting
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index 2326f978562..ebfe3e2ce5a 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -231,6 +231,10 @@ for target in $target_list; do
                -march=armv8.3-a -o $TMPE $TMPC; then
                 echo "CROSS_CC_HAS_ARMV8_3=y" >> $config_target_mak
             fi
+            if do_compiler "$target_compiler" $target_compiler_cflags \
+               -mbranch-protection=standard -o $TMPE $TMPC; then
+                echo "CROSS_CC_HAS_ARMV8_BTI=y" >> $config_target_mak
+            fi
         ;;
     esac
 
-- 
2.20.1

[PATCH v1 12/15] cpus-common: ensure auto-assigned cpu_indexes don't clash

[Alex Bennée]

Basing the cpu_index on the number of currently allocated vCPUs fails
when vCPUs aren't removed in a LIFO manner. This is especially true
when we are allocating a cpu_index for each guest thread in
linux-user where there is no ordering constraint on their allocation
and de-allocation.

[I've dropped the assert which is there to guard against out-of-order
removal as this should probably be caught higher up the stack. Maybe
we could just ifdef CONFIG_SOFTTMU it?]

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Nikolay Igotti <igotti@gmail.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>

---
v2
  - slightly tweak the index algorithm to preserve cpu_index = 0
---
 cpus-common.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/cpus-common.c b/cpus-common.c
index 55d5df89237..70a9d12981a 100644
--- a/cpus-common.c
+++ b/cpus-common.c
@@ -61,13 +61,15 @@ static bool cpu_index_auto_assigned;
 static int cpu_get_free_index(void)
 {
     CPUState *some_cpu;
-    int cpu_index = 0;
+    int max_cpu_index = 0;
 
     cpu_index_auto_assigned = true;
     CPU_FOREACH(some_cpu) {
-        cpu_index++;
+        if (some_cpu->cpu_index >= max_cpu_index) {
+            max_cpu_index = some_cpu->cpu_index + 1;
+        }
     }
-    return cpu_index;
+    return max_cpu_index;
 }
 
 void cpu_list_add(CPUState *cpu)
@@ -90,8 +92,6 @@ void cpu_list_remove(CPUState *cpu)
         return;
     }
 
-    assert(!(cpu_index_auto_assigned && cpu != QTAILQ_LAST(&cpus)));
-
     QTAILQ_REMOVE_RCU(&cpus, cpu, node);
     cpu->cpu_index = UNASSIGNED_CPU_INDEX;
 }
-- 
2.20.1

[PATCH v1 13/15] linux-user: properly "unrealize" vCPU object

[Alex Bennée]

We shouldn't be messing around with the CPU list in linux-user save
for the very special case of do_fork(). When threads end we need to
properly follow QOM object lifetime handling and allow the eventual
cpu_common_unrealizefn to both remove the CPU and ensure any clean-up
actions are taken place, for example calling plugin exit hooks.

There is still a race condition to avoid so use the linux-user
specific clone_lock instead of the cpu_list_lock to avoid it.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Nikolay Igotti <igotti@gmail.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Daniel P. Berrange <berrange@redhat.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Markus Armbruster <armbru@redhat.com>

---
v2
  - expand Cc list to QOM/QMP object lifetime experts
---
 linux-user/syscall.c | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 05f03919ff0..7f6700c54e3 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -7635,30 +7635,33 @@ static abi_long do_syscall1(void *cpu_env, int num, abi_long arg1,
             return -TARGET_ERESTARTSYS;
         }
 
-        cpu_list_lock();
+        pthread_mutex_lock(&clone_lock);
 
         if (CPU_NEXT(first_cpu)) {
-            TaskState *ts;
+            TaskState *ts = cpu->opaque;
 
-            /* Remove the CPU from the list.  */
-            QTAILQ_REMOVE_RCU(&cpus, cpu, node);
+            object_property_set_bool(OBJECT(cpu), false, "realized", NULL);
+            object_unref(OBJECT(cpu));
+            /*
+             * At this point the CPU should be unrealized and removed
+             * from cpu lists. We can clean-up the rest of the thread
+             * data without the lock held.
+             */
 
-            cpu_list_unlock();
+            pthread_mutex_unlock(&clone_lock);
 
-            ts = cpu->opaque;
             if (ts->child_tidptr) {
                 put_user_u32(0, ts->child_tidptr);
                 do_sys_futex(g2h(ts->child_tidptr), FUTEX_WAKE, INT_MAX,
                           NULL, NULL, 0);
             }
             thread_cpu = NULL;
-            object_unref(OBJECT(cpu));
             g_free(ts);
             rcu_unregister_thread();
             pthread_exit(NULL);
         }
 
-        cpu_list_unlock();
+        pthread_mutex_unlock(&clone_lock);
         preexit_cleanup(cpu_env, arg1);
         _exit(arg1);
         return 0; /* avoid warning */
-- 
2.20.1

[PATCH v1 14/15] tests/tcg: add new threadcount test

[Alex Bennée]

Based on the original testcase by Nikolay Igotti.

Message-ID: <CAEme+7GLKg_dNsHizzTKDymX9HyD+Ph2iZ=WKhOw2XG+zhViXg@mail.gmail.com>
Signed-off-by: Nikolay Igotti <igotti@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v2
  - add Nikolay's s-o-b
  - checkpatch fixes
---
 tests/tcg/multiarch/threadcount.c   | 64 +++++++++++++++++++++++++++++
 tests/tcg/multiarch/Makefile.target |  2 +
 2 files changed, 66 insertions(+)
 create mode 100644 tests/tcg/multiarch/threadcount.c

diff --git a/tests/tcg/multiarch/threadcount.c b/tests/tcg/multiarch/threadcount.c
new file mode 100644
index 00000000000..545a1c81466
--- /dev/null
+++ b/tests/tcg/multiarch/threadcount.c
@@ -0,0 +1,64 @@
+/*
+ * Thread Exerciser
+ *
+ * Unlike testthread which is mainly concerned about testing thread
+ * semantics this test is used to exercise the thread creation and
+ * accounting. A version of this test found a problem with clashing
+ * cpu_indexes which caused a break in plugin handling.
+ *
+ * Based on the original test case by Nikolay Igotti.
+ *
+ * Copyright (c) 2020 Linaro Ltd
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <pthread.h>
+
+int max_threads = 10;
+
+typedef struct {
+    int delay;
+} ThreadArg;
+
+static void *thread_fn(void* varg)
+{
+    ThreadArg *arg = varg;
+    usleep(arg->delay);
+    free(arg);
+    return NULL;
+}
+
+int main(int argc, char **argv)
+{
+    int i;
+    pthread_t *threads;
+
+    if (argc > 1) {
+        max_threads = atoi(argv[1]);
+    }
+    threads = calloc(sizeof(pthread_t), max_threads);
+
+    for (i = 0; i < max_threads; i++) {
+        ThreadArg *arg = calloc(sizeof(ThreadArg), 1);
+        arg->delay = i * 100;
+        pthread_create(threads + i, NULL, thread_fn, arg);
+    }
+
+    printf("Created %d threads\n", max_threads);
+
+    /* sleep until roughly half the threads have "finished" */
+    usleep(max_threads * 50);
+
+    for (i = 0; i < max_threads; i++) {
+        pthread_join(threads[i], NULL);
+    }
+
+    printf("Done\n");
+
+    return 0;
+}
diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target
index 51fb75ecfdd..cb49cc9ccb2 100644
--- a/tests/tcg/multiarch/Makefile.target
+++ b/tests/tcg/multiarch/Makefile.target
@@ -28,6 +28,8 @@ run-float_%: float_%
 
 testthread: LDFLAGS+=-lpthread
 
+threadcount: LDFLAGS+=-lpthread
+
 # We define the runner for test-mmap after the individual
 # architectures have defined their supported pages sizes. If no
 # additional page sizes are defined we only run the default test.
-- 
2.20.1

[PATCH v1 15/15] plugins: new lockstep plugin for debugging TCG changes

[Alex Bennée]

When we make changes to the TCG we sometimes cause regressions that
are deep into the execution cycle of the guest. Debugging this often
requires comparing large volumes of trace information to figure out
where behaviour has diverged.

The lockstep plugin utilises a shared socket so two QEMU's running
with the plugin will write their current execution position and wait
to receive the position of their partner process. When execution
diverges the plugins output where they were and the previous few
blocks before unloading themselves and letting execution continue.

Originally I planned for this to be most useful with -icount but it
turns out you can get divergence pretty quickly due to asynchronous
qemu_cpu_kick_rr_cpus() events causing one side to eventually run into
a short block a few cycles before the other side. For this reason I've
added a bit of tracking and I think the divergence reporting could be
finessed to report only if we really start to diverge in execution.

An example run would be:

  qemu-system-sparc -monitor none -parallel none -net none \
    -M SS-20 -m 256 -kernel day11/zImage.elf \
    -plugin ./tests/plugin/liblockstep.so,arg=lockstep-sparc.sock \
    -d plugin,nochain

with an identical command in another window in the same working
directory.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Message-Id: <20200429200754.18327-1-alex.bennee@linaro.org>

---
v3
  - added verbose flag
  - basic heuristics to detect "real" divergence
  - checkpatch tweaks
---
 tests/plugin/lockstep.c   | 340 ++++++++++++++++++++++++++++++++++++++
 tests/plugin/Makefile     |   1 +
 tests/tcg/Makefile.target |   2 +-
 3 files changed, 342 insertions(+), 1 deletion(-)
 create mode 100644 tests/plugin/lockstep.c

diff --git a/tests/plugin/lockstep.c b/tests/plugin/lockstep.c
new file mode 100644
index 00000000000..a696673dff3
--- /dev/null
+++ b/tests/plugin/lockstep.c
@@ -0,0 +1,340 @@
+/*
+ * Lockstep Execution Plugin
+ *
+ * Allows you to execute two QEMU instances in lockstep and report
+ * when their execution diverges. This is mainly useful for developers
+ * who want to see where a change to TCG code generation has
+ * introduced a subtle and hard to find bug.
+ *
+ * Caveats:
+ *   - single-threaded linux-user apps only with non-deterministic syscalls
+ *   - no MTTCG enabled system emulation (icount may help)
+ *
+ * While icount makes things more deterministic it doesn't mean a
+ * particular run may execute the exact same sequence of blocks. An
+ * asynchronous event (for example X11 graphics update) may cause a
+ * block to end early and a new partial block to start. This means
+ * serial only test cases are a better bet. -d nochain may also help.
+ *
+ * This code is not thread safe!
+ *
+ * Copyright (c) 2020 Linaro Ltd
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include <glib.h>
+#include <inttypes.h>
+#include <unistd.h>
+#include <sys/socket.h>
+#include <sys/un.h>
+#include <stdio.h>
+#include <errno.h>
+
+#include <qemu-plugin.h>
+
+QEMU_PLUGIN_EXPORT int qemu_plugin_version = QEMU_PLUGIN_VERSION;
+
+/* saved so we can uninstall later */
+static qemu_plugin_id_t our_id;
+
+static unsigned long bb_count;
+static unsigned long insn_count;
+
+/* Information about a translated block */
+typedef struct {
+    uint64_t pc;
+    uint64_t insns;
+} BlockInfo;
+
+/* Information about an execution state in the log */
+typedef struct {
+    BlockInfo *block;
+    unsigned long insn_count;
+    unsigned long block_count;
+} ExecInfo;
+
+/* The execution state we compare */
+typedef struct {
+    uint64_t pc;
+    unsigned long insn_count;
+} ExecState;
+
+typedef struct {
+    GSList *log_pos;
+    int distance;
+} DivergeState;
+
+/* list of translated block info */
+static GSList *blocks;
+
+/* execution log and points of divergence */
+static GSList *log, *divergence_log;
+
+static int socket_fd;
+static char *path_to_unlink;
+
+static bool verbose;
+
+static void plugin_cleanup(qemu_plugin_id_t id)
+{
+    /* Free our block data */
+    g_slist_free_full(blocks, &g_free);
+    g_slist_free_full(log, &g_free);
+    g_slist_free(divergence_log);
+
+    close(socket_fd);
+    if (path_to_unlink) {
+        unlink(path_to_unlink);
+    }
+}
+
+static void plugin_exit(qemu_plugin_id_t id, void *p)
+{
+    g_autoptr(GString) out = g_string_new("No divergence :-)\n");
+    g_string_append_printf(out, "Executed %ld/%d blocks\n",
+                           bb_count, g_slist_length(log));
+    g_string_append_printf(out, "Executed ~%ld instructions\n", insn_count);
+    qemu_plugin_outs(out->str);
+
+    plugin_cleanup(id);
+}
+
+static void report_divergance(ExecState *us, ExecState *them)
+{
+    DivergeState divrec = { log, 0 };
+    g_autoptr(GString) out = g_string_new("");
+    bool diverged = false;
+
+    /*
+     * If we have diverged before did we get back on track or are we
+     * totally loosing it?
+     */
+    if (divergence_log) {
+        DivergeState *last = (DivergeState *) divergence_log->data;
+        GSList *entry;
+
+        for (entry = log; g_slist_next(entry); entry = g_slist_next(entry)) {
+            if (entry == last->log_pos) {
+                break;
+            }
+            divrec.distance++;
+        }
+
+        /*
+         * If the last two records are so close it is likely we will
+         * not recover synchronisation with the other end.
+         */
+        if (divrec.distance == 1 && last->distance == 1) {
+            diverged = true;
+        }
+    }
+    divergence_log = g_slist_prepend(divergence_log,
+                                     g_memdup(&divrec, sizeof(divrec)));
+
+    /* Output short log entry of going out of sync... */
+    if (verbose || divrec.distance == 1 || diverged) {
+        g_string_printf(out, "@ %#016lx vs %#016lx (%d/%d since last)\n",
+                        us->pc, them->pc, g_slist_length(divergence_log),
+                        divrec.distance);
+        qemu_plugin_outs(out->str);
+    }
+
+    if (diverged) {
+        int i;
+        GSList *entry;
+
+        g_string_printf(out, "Δ insn_count @ %#016lx (%ld) vs %#016lx (%ld)\n",
+                        us->pc, us->insn_count, them->pc, them->insn_count);
+
+        for (entry = log, i = 0;
+             g_slist_next(entry) && i < 5;
+             entry = g_slist_next(entry), i++) {
+            ExecInfo *prev = (ExecInfo *) entry->data;
+            g_string_append_printf(out,
+                                   "  previously @ %#016lx/%ld (%ld insns)\n",
+                                   prev->block->pc, prev->block->insns,
+                                   prev->insn_count);
+        }
+        qemu_plugin_outs(out->str);
+        qemu_plugin_outs("too much divergence... giving up.");
+        qemu_plugin_uninstall(our_id, plugin_cleanup);
+    }
+}
+
+static void vcpu_tb_exec(unsigned int cpu_index, void *udata)
+{
+    BlockInfo *bi = (BlockInfo *) udata;
+    ExecState us, them;
+    ssize_t bytes;
+    ExecInfo *exec;
+
+    us.pc = bi->pc;
+    us.insn_count = insn_count;
+
+    /*
+     * Write our current position to the other end. If we fail the
+     * other end has probably died and we should shut down gracefully.
+     */
+    bytes = write(socket_fd, &us, sizeof(ExecState));
+    if (bytes < sizeof(ExecState)) {
+        qemu_plugin_outs(bytes < 0 ?
+                         "problem writing to socket" :
+                         "wrote less than expected to socket");
+        qemu_plugin_uninstall(our_id, plugin_cleanup);
+        return;
+    }
+
+    /*
+     * Now read where our peer has reached. Again a failure probably
+     * indicates the other end died and we should close down cleanly.
+     */
+    bytes = read(socket_fd, &them, sizeof(ExecState));
+    if (bytes < sizeof(ExecState)) {
+        qemu_plugin_outs(bytes < 0 ?
+                         "problem reading from socket" :
+                         "read less than expected");
+        qemu_plugin_uninstall(our_id, plugin_cleanup);
+        return;
+    }
+
+    /*
+     * Compare and report if we have diverged.
+     */
+    if (us.pc != them.pc) {
+        report_divergance(&us, &them);
+    }
+
+    /*
+     * Assume this block will execute fully and record it
+     * in the execution log.
+     */
+    insn_count += bi->insns;
+    bb_count++;
+    exec = g_new0(ExecInfo, 1);
+    exec->block = bi;
+    exec->insn_count = insn_count;
+    exec->block_count = bb_count;
+    log = g_slist_prepend(log, exec);
+}
+
+static void vcpu_tb_trans(qemu_plugin_id_t id, struct qemu_plugin_tb *tb)
+{
+    BlockInfo *bi = g_new0(BlockInfo, 1);
+    bi->pc = qemu_plugin_tb_vaddr(tb);
+    bi->insns = qemu_plugin_tb_n_insns(tb);
+
+    /* save a reference so we can free later */
+    blocks = g_slist_prepend(blocks, bi);
+    qemu_plugin_register_vcpu_tb_exec_cb(tb, vcpu_tb_exec,
+                                         QEMU_PLUGIN_CB_NO_REGS, (void *)bi);
+}
+
+
+/*
+ * Instead of encoding master/slave status into what is essentially
+ * two peers we shall just take the simple approach of checking for
+ * the existence of the pipe and assuming if it's not there we are the
+ * first process.
+ */
+static bool setup_socket(const char *path)
+{
+    struct sockaddr_un sockaddr;
+    int fd;
+
+    fd = socket(AF_UNIX, SOCK_STREAM, 0);
+    if (fd < 0) {
+        perror("create socket");
+        return false;
+    }
+
+    sockaddr.sun_family = AF_UNIX;
+    g_strlcpy(sockaddr.sun_path, path, sizeof(sockaddr.sun_path) - 1);
+    if (bind(fd, (struct sockaddr *)&sockaddr, sizeof(sockaddr)) < 0) {
+        perror("bind socket");
+        close(fd);
+        return false;
+    }
+
+    /* remember to clean-up */
+    path_to_unlink = g_strdup(path);
+
+    if (listen(fd, 1) < 0) {
+        perror("listen socket");
+        close(fd);
+        return false;
+    }
+
+    socket_fd = accept(fd, NULL, NULL);
+    if (socket_fd < 0 && errno != EINTR) {
+        perror("accept socket");
+        return false;
+    }
+
+    qemu_plugin_outs("setup_socket::ready\n");
+
+    return true;
+}
+
+static bool connect_socket(const char *path)
+{
+    int fd;
+    struct sockaddr_un sockaddr;
+
+    fd = socket(AF_UNIX, SOCK_STREAM, 0);
+    if (fd < 0) {
+        perror("create socket");
+        return false;
+    }
+
+    sockaddr.sun_family = AF_UNIX;
+    g_strlcpy(sockaddr.sun_path, path, sizeof(sockaddr.sun_path) - 1);
+
+    if (connect(fd, (struct sockaddr *)&sockaddr, sizeof(sockaddr)) < 0) {
+        perror("failed to connect");
+        return false;
+    }
+
+    qemu_plugin_outs("connect_socket::ready\n");
+
+    socket_fd = fd;
+    return true;
+}
+
+static bool setup_unix_socket(const char *path)
+{
+    if (g_file_test(path, G_FILE_TEST_EXISTS)) {
+        return connect_socket(path);
+    } else {
+        return setup_socket(path);
+    }
+}
+
+
+QEMU_PLUGIN_EXPORT int qemu_plugin_install(qemu_plugin_id_t id,
+                                           const qemu_info_t *info,
+                                           int argc, char **argv)
+{
+    int i;
+
+    if (!argc || !argv[0]) {
+        qemu_plugin_outs("Need a socket path to talk to other instance.");
+        return -1;
+    }
+
+    for (i = 0; i < argc; i++) {
+        char *p = argv[i];
+        if (strcmp(p, "verbose") == 0) {
+            verbose = true;
+        } else if (!setup_unix_socket(argv[0])) {
+            qemu_plugin_outs("Failed to setup socket for communications.");
+            return -1;
+        }
+    }
+
+    our_id = id;
+
+    qemu_plugin_register_vcpu_tb_trans_cb(id, vcpu_tb_trans);
+    qemu_plugin_register_atexit_cb(id, plugin_exit, NULL);
+    return 0;
+}
diff --git a/tests/plugin/Makefile b/tests/plugin/Makefile
index 75467b6db85..b3250e2504c 100644
--- a/tests/plugin/Makefile
+++ b/tests/plugin/Makefile
@@ -13,6 +13,7 @@ NAMES += mem
 NAMES += hotblocks
 NAMES += howvec
 NAMES += hotpages
+NAMES += lockstep
 
 SONAMES := $(addsuffix .so,$(addprefix lib,$(NAMES)))
 
diff --git a/tests/tcg/Makefile.target b/tests/tcg/Makefile.target
index b3cff3cad1a..075daf3d22d 100644
--- a/tests/tcg/Makefile.target
+++ b/tests/tcg/Makefile.target
@@ -128,7 +128,7 @@ RUN_TESTS=$(patsubst %,run-%, $(TESTS))
 ifeq ($(CONFIG_PLUGIN),y)
 PLUGIN_DIR=../../plugin
 VPATH+=$(PLUGIN_DIR)
-PLUGINS=$(notdir $(wildcard $(PLUGIN_DIR)/*.so))
+PLUGINS=$(filter-out liblockstep.so,$(notdir $(wildcard $(PLUGIN_DIR)/*.so)))
 
 # We need to ensure expand the run-plugin-TEST-with-PLUGIN
 # pre-requistes manually here as we can't use stems to handle it. We
-- 
2.20.1

Re: [PATCH v1 00/15] testing and plugin updates

[no-reply]

Patchew URL: https://patchew.org/QEMU/20200520140541.30256-1-alex.bennee@linaro.org/



Hi,

This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1
=== TEST SCRIPT END ===

 perl-Encode-devel                       x86_64  4:3.06-445.fc32                    updates   42 k
 perl-Env                                noarch  1.04-440.fc32                      fedora    19 k
 perl-Errno                              x86_64  1.30-452.fc32                      fedora    24 k
 perl-Error                              noarch  1:0.17029-1.fc32                   fedora    42 k
 perl-Exporter                           noarch  5.74-2.fc32                        fedora    32 k
 perl-Exporter-Tiny                      noarch  1.002001-5.fc32                    fedora    51 k
 perl-ExtUtils-CBuilder                  noarch  1:0.280234-2.fc32                  fedora    47 k
---
(497/699): perl-Env-1.04-440.fc32.noarch.rpm    198 kB/s |  19 kB     00:00    
(498/699): perl-Digest-SHA-6.02-442.fc32.x86_64 325 kB/s |  64 kB     00:00    
(499/699): perl-Errno-1.30-452.fc32.x86_64.rpm  245 kB/s |  24 kB     00:00    
(500/699): perl-Error-0.17029-1.fc32.noarch.rpm 415 kB/s |  42 kB     00:00    
(501/699): perl-Exporter-5.74-2.fc32.noarch.rpm 318 kB/s |  32 kB     00:00    
(502/699): perl-Exporter-Tiny-1.002001-5.fc32.n 497 kB/s |  51 kB     00:00    
(503/699): perl-ExtUtils-CBuilder-0.280234-2.fc 455 kB/s |  47 kB     00:00    
---
  Installing       : perl-DB_File-1.853-2.fc32.x86_64                   175/700 
  Installing       : perl-Devel-Size-0.83-5.fc32.x86_64                 176/700 
  Installing       : perl-Env-1.04-440.fc32.noarch                      177/700 
  Installing       : perl-Error-1:0.17029-1.fc32.noarch                 178/700 
  Installing       : perl-ExtUtils-MM-Utils-2:7.44-2.fc32.noarch        179/700 
  Installing       : perl-IPC-SysV-2.07-442.fc32.x86_64                 180/700 
  Installing       : perl-IPC-System-Simple-1.30-1.fc32.noarch          181/700 
---
  Verifying        : perl-Encode-Locale-1.05-15.fc32.noarch             497/700 
  Verifying        : perl-Env-1.04-440.fc32.noarch                      498/700 
  Verifying        : perl-Errno-1.30-452.fc32.x86_64                    499/700 
  Verifying        : perl-Error-1:0.17029-1.fc32.noarch                 500/700 
  Verifying        : perl-Exporter-5.74-2.fc32.noarch                   501/700 
  Verifying        : perl-Exporter-Tiny-1.002001-5.fc32.noarch          502/700 
  Verifying        : perl-ExtUtils-CBuilder-1:0.280234-2.fc32.noarch    503/700 
---
  perl-Encode-devel-4:3.06-445.fc32.x86_64                                      
  perl-Env-1.04-440.fc32.noarch                                                 
  perl-Errno-1.30-452.fc32.x86_64                                               
  perl-Error-1:0.17029-1.fc32.noarch                                            
  perl-Exporter-5.74-2.fc32.noarch                                              
  perl-Exporter-Tiny-1.002001-5.fc32.noarch                                     
  perl-ExtUtils-CBuilder-1:0.280234-2.fc32.noarch                               
---
  CC      backends/tpm.o
  CC      backends/hostmem.o
  CC      backends/hostmem-ram.o
/tmp/qemu-test/src/audio/mixeng.c:274:34: error: implicit conversion from 'unsigned int' to 'float' changes value from 4294967295 to 4294967296 [-Werror,-Wimplicit-int-float-conversion]
static const float float_scale = UINT_MAX / 2.f;
                                 ^~~~~~~~ ~
/usr/lib64/clang/10.0.0/include/limits.h:56:37: note: expanded from macro 'UINT_MAX'
#define UINT_MAX  (__INT_MAX__  *2U +1U)
                   ~~~~~~~~~~~~~~~~~^~~
1 error generated.
make: *** [/tmp/qemu-test/src/rules.mak:69: audio/mixeng.o] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
  File "./tests/docker/docker.py", line 664, in <module>
---
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=07d8739029c2460480f2e8f49878e61e', '-u', '1003', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=x86_64-softmmu', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew2/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-juuwuycm/src/docker-src.2020-05-20-12.32.55.601:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-debug']' returned non-zero exit status 2.
filter=--filter=label=com.qemu.instance.uuid=07d8739029c2460480f2e8f49878e61e
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-juuwuycm/src'
make: *** [docker-run-test-debug@fedora] Error 2

real    4m16.571s
user    0m9.425s


The full log is available at
http://patchew.org/logs/20200520140541.30256-1-alex.bennee@linaro.org/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [PATCH v1 00/15] testing and plugin updates

[no-reply]

Patchew URL: https://patchew.org/QEMU/20200520140541.30256-1-alex.bennee@linaro.org/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Message-id: 20200520140541.30256-1-alex.bennee@linaro.org
Subject: [PATCH  v1 00/15] testing and plugin updates
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Switched to a new branch 'test'
a1f3009 plugins: new lockstep plugin for debugging TCG changes
f8ab9f1 tests/tcg: add new threadcount test
bbaf5c2 linux-user: properly "unrealize" vCPU object
8d602ed cpus-common: ensure auto-assigned cpu_indexes don't clash
a31d8dd tests/tcg/aarch64: Add bti smoke test
46efafc tests/docker: use a gcc-10 based image for arm64 tests
c62d664 tests/docker: add debian11 base image
c5590fe tests/docker: bump fedora to 32
5eea6f1 tests/tcg: better detect confused gdb which can't connect
63db898 tests/fp: split and audit the conversion tests
d484715 tests/fp: enable extf80_le_quite tests
da123f1 tests/tcg: fix invocation of the memory record/replay tests
0667cc3 travis.yml: Use clang++ in the Clang tests
03334db tests/vm: pass --genisoimage to basevm script
f49c9ff configure: add alternate binary for genisoimage

=== OUTPUT BEGIN ===
1/15 Checking commit f49c9ff28400 (configure: add alternate binary for genisoimage)
2/15 Checking commit 03334db3deeb (tests/vm: pass --genisoimage to basevm script)
3/15 Checking commit 0667cc3737a6 (travis.yml: Use clang++ in the Clang tests)
4/15 Checking commit da123f1fc347 (tests/tcg: fix invocation of the memory record/replay tests)
5/15 Checking commit d484715faabe (tests/fp: enable extf80_le_quite tests)
6/15 Checking commit 63db8983c952 (tests/fp: split and audit the conversion tests)
7/15 Checking commit 5eea6f17d0e4 (tests/tcg: better detect confused gdb which can't connect)
8/15 Checking commit c5590fe1a910 (tests/docker: bump fedora to 32)
9/15 Checking commit c62d664e2b07 (tests/docker: add debian11 base image)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#27: 
new file mode 100644

total: 0 errors, 1 warnings, 26 lines checked

Patch 9/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
10/15 Checking commit 46efafc4e520 (tests/docker: use a gcc-10 based image for arm64 tests)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#31: 
new file mode 100644

total: 0 errors, 1 warnings, 34 lines checked

Patch 10/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
11/15 Checking commit a31d8ddc748b (tests/tcg/aarch64: Add bti smoke test)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#36: 
new file mode 100644

ERROR: externs should be avoided in .c files
#121: FILE: tests/tcg/aarch64/bti-crt.inc.c:13:
+int main(void);

total: 1 errors, 1 warnings, 138 lines checked

Patch 11/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

12/15 Checking commit 8d602ed77a25 (cpus-common: ensure auto-assigned cpu_indexes don't clash)
13/15 Checking commit bbaf5c2f62be (linux-user: properly "unrealize" vCPU object)
14/15 Checking commit f8ab9f1fcc96 (tests/tcg: add new threadcount test)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#29: 
new file mode 100644

total: 0 errors, 1 warnings, 72 lines checked

Patch 14/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
15/15 Checking commit a1f3009ce1f2 (plugins: new lockstep plugin for debugging TCG changes)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#55: 
new file mode 100644

total: 0 errors, 1 warnings, 355 lines checked

Patch 15/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20200520140541.30256-1-alex.bennee@linaro.org/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [PATCH v1 05/15] tests/fp: enable extf80_le_quite tests

[Richard Henderson]

On 5/20/20 7:05 AM, Alex Bennée wrote:
> These have been fixed now so we no longer need a special version of
> the le_quiet rule to skip the test.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/Makefile.include | 7 -------
>  1 file changed, 7 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~

Re: [PATCH v1 06/15] tests/fp: split and audit the conversion tests

[Richard Henderson]

On 5/20/20 7:05 AM, Alex Bennée wrote:
> Split the float conversion tests into separate groups and audit the
> tests to check what is still broken. I was able to enable a bunch of
> tests that had been missed before:
> 
>   all the float to float conversions
>   ui32_to_extF80
>   ui64_to_extF80
>   extF80_to_ui32
>   extF80_to_ui32_r_minMag
>   extF80_to_ui64
>   extF80_to_ui64_r_minMag
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/Makefile.include | 37 ++++++++++++++++++++++++++++++++-----
>  1 file changed, 32 insertions(+), 5 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~

Re: [PATCH v1 07/15] tests/tcg: better detect confused gdb which can't connect

[Richard Henderson]

On 5/20/20 7:05 AM, Alex Bennée wrote:
> While we may gamely give the right information it can still confuse
> the wide range of GDBs out there. For example ppc64abi32-linux-user
> reports:
> 
>   warning: Selected architecture powerpc:common is not compatible with reported target architecture powerpc:common64
>   warning: Architecture rejected target-supplied description
> 
> but still connects. Add a test for a 0 pc and exit early if that is
> the case. This may actually be a bug we need to fix?
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Cc: Richard Henderson <rth@twiddle.net>
> ---
>  tests/tcg/multiarch/gdbstub/sha1.py | 4 ++++
>  1 file changed, 4 insertions(+)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~

Re: [PATCH v1 08/15] tests/docker: bump fedora to 32

[Richard Henderson]

On 5/20/20 7:05 AM, Alex Bennée wrote:
> We should be keeping this up to date as Fedora goes out of support
> quite quickly.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/dockerfiles/fedora.docker | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~

Re: [PATCH v1 09/15] tests/docker: add debian11 base image

[Richard Henderson]

On 5/20/20 7:05 AM, Alex Bennée wrote:
> We won't use this for building QEMU but we do need newer GCC's and
> binutils for building some of our test cases.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/Makefile.include            |  2 +-
>  tests/docker/dockerfiles/debian11.docker | 18 ++++++++++++++++++
>  2 files changed, 19 insertions(+), 1 deletion(-)
>  create mode 100644 tests/docker/dockerfiles/debian11.docker

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~

Re: [PATCH v1 10/15] tests/docker: use a gcc-10 based image for arm64 tests

[Richard Henderson]

On 5/20/20 7:05 AM, Alex Bennée wrote:
> As we enable newer features that we want to test on arm64 targets we
> need newer compilers. Split off a new debian-arm64-test-cross image
> which we can use to build these new tests.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/Makefile.include                       |  2 ++
>  .../dockerfiles/debian-arm64-test-cross.docker      | 13 +++++++++++++
>  tests/tcg/configure.sh                              |  4 ++--
>  3 files changed, 17 insertions(+), 2 deletions(-)
>  create mode 100644 tests/docker/dockerfiles/debian-arm64-test-cross.docker

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~

Re: [PATCH v1 11/15] tests/tcg/aarch64: Add bti smoke test

[Richard Henderson]

On 5/20/20 7:05 AM, Alex Bennée wrote:
> From: Richard Henderson <richard.henderson@linaro.org>
> 
> Requires gcc 10 for -mbranch-protection=standard.
> 
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> Message-Id: <20200520023139.9812-3-richard.henderson@linaro.org>
> [AJB: add CROSS_CC_HAS_ARMV8_BTI check]
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/tcg/aarch64/bti-1.c         | 62 +++++++++++++++++++++++++++++++
>  tests/tcg/aarch64/bti-crt.inc.c   | 51 +++++++++++++++++++++++++
>  tests/tcg/aarch64/Makefile.target |  9 +++++
>  tests/tcg/configure.sh            |  4 ++
>  4 files changed, 126 insertions(+)
>  create mode 100644 tests/tcg/aarch64/bti-1.c
>  create mode 100644 tests/tcg/aarch64/bti-crt.inc.c

This should be dropped from this patch set.  I have adjusted it and it and
posted as

https://lists.gnu.org/archive/html/qemu-devel/2020-05/msg05891.html


r~

Re: [PATCH v1 12/15] cpus-common: ensure auto-assigned cpu_indexes don't clash

[Richard Henderson]

On 5/20/20 7:05 AM, Alex Bennée wrote:
> Basing the cpu_index on the number of currently allocated vCPUs fails
> when vCPUs aren't removed in a LIFO manner. This is especially true
> when we are allocating a cpu_index for each guest thread in
> linux-user where there is no ordering constraint on their allocation
> and de-allocation.
> 
> [I've dropped the assert which is there to guard against out-of-order
> removal as this should probably be caught higher up the stack. Maybe
> we could just ifdef CONFIG_SOFTTMU it?]
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Cc: Nikolay Igotti <igotti@gmail.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Igor Mammedov <imammedo@redhat.com>
> Cc: Eduardo Habkost <ehabkost@redhat.com>
> 
> ---
> v2
>   - slightly tweak the index algorithm to preserve cpu_index = 0
> ---
>  cpus-common.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~