[PULL 00/12] some testing and plugin updates

[PULL 00/12] some testing and plugin updates

[Alex Bennée]

The following changes since commit 14f12119aa675e9e28207a48b0728a2daa5b88d6:

  Merge remote-tracking branch 'remotes/vsementsov/tags/pull-jobs-2021-10-07-v2' into staging (2021-10-07 10:26:35 -0700)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-for-6.2-081021-1

for you to fetch changes up to 755c4aefd86f8b8eda1eb37f20024765c70ebbcb:

  tests/docker: add a debian-native image and make available (2021-10-08 12:55:27 +0100)

----------------------------------------------------------------
Some testing and plugin updates:

  - don't override the test compiler when specified
  - split some multiarch tests by guest OS
  - add riscv64 docker image and cross-compile tests
  - drop release tarball test from Travis
  - skip check-patch on master repo
  - fix passing of TEST_TARGETS to cirrus
  - fix missing symbols in plugins
  - refactor plugin instruction boundary detection
  - update github repo lockdown
  - add a debian-native test image for multi-arch builds

----------------------------------------------------------------
Alex Bennée (7):
      configure: don't override the selected host test compiler if defined
      tests/tcg/sha1: remove endian include
      tests/tcg: move some multiarch files and make conditional
      tests/docker: promote debian-riscv64-cross to a full image
      accel/tcg: re-factor plugin_inject_cb so we can assert insn_idx is valid
      .github: move repo lockdown to the v2 configuration
      tests/docker: add a debian-native image and make available

Daniel P. Berrangé (2):
      gitlab: skip the check-patch job on the upstream repo
      gitlab: fix passing of TEST_TARGETS env to cirrus

Lukas Jünger (1):
      plugins/: Add missing functions to symbol list

Richard Henderson (1):
      gitlab: Add cross-riscv64-system, cross-riscv64-user

Thomas Huth (1):
      travis.yml: Remove the "Release tarball" job

 configure                                          |   6 +-
 include/tcg/tcg.h                                  |   6 -
 accel/tcg/plugin-gen.c                             | 157 +++++++++++----------
 tests/tcg/multiarch/{ => libs}/float_helpers.c     |   2 +-
 tests/tcg/multiarch/{ => linux}/linux-test.c       |   0
 tests/tcg/multiarch/sha1.c                         |   1 -
 .github/lockdown.yml                               |  34 -----
 .github/workflows/lockdown.yml                     |  30 ++++
 .gitlab-ci.d/cirrus.yml                            |   2 +-
 .gitlab-ci.d/cirrus/build.yml                      |   1 +
 .gitlab-ci.d/container-cross.yml                   |   3 +-
 .gitlab-ci.d/crossbuilds.yml                       |  19 +++
 .gitlab-ci.d/static_checks.yml                     |   2 +-
 .travis.yml                                        |  23 ---
 plugins/qemu-plugins.symbols                       |   3 +
 tests/docker/Makefile.include                      |   6 +-
 tests/docker/common.rc                             |  10 +-
 tests/docker/dockerfiles/debian-native.docker      |  49 +++++++
 .../docker/dockerfiles/debian-riscv64-cross.docker |  46 +++++-
 tests/tcg/multiarch/Makefile.target                |  15 +-
 tests/tcg/x86_64/Makefile.target                   |   4 +
 21 files changed, 263 insertions(+), 156 deletions(-)
 rename tests/tcg/multiarch/{ => libs}/float_helpers.c (99%)
 rename tests/tcg/multiarch/{ => linux}/linux-test.c (100%)
 delete mode 100644 .github/lockdown.yml
 create mode 100644 .github/workflows/lockdown.yml
 create mode 100644 tests/docker/dockerfiles/debian-native.docker

-- 
2.30.2

[PULL 01/12] configure: don't override the selected host test compiler if defined

[Alex Bennée]

There are not many cases you would want to do this but one is if you
want to use a test friendly compiler like gcc instead of a system
compiler like clang. Either way we should honour the users choice if
they have made it.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Warner Losh <imp@bsdimp.com>
Reviewed-by: Warner Losh <imp@bsdimp.com>
Message-Id: <20210917162332.3511179-2-alex.bennee@linaro.org>

diff --git a/configure b/configure
index 877bf3d76a..e2750810e2 100755
--- a/configure
+++ b/configure
@@ -1686,8 +1686,10 @@ case "$cpu" in
     # No special flags required for other host CPUs
 esac
 
-eval "cross_cc_${cpu}=\$cc"
-cross_cc_vars="$cross_cc_vars cross_cc_${cpu}"
+if eval test -z "\${cross_cc_$cpu}"; then
+    eval "cross_cc_${cpu}=\$cc"
+    cross_cc_vars="$cross_cc_vars cross_cc_${cpu}"
+fi
 
 # For user-mode emulation the host arch has to be one we explicitly
 # support, even if we're using TCI.
-- 
2.30.2

[PULL 02/12] tests/tcg/sha1: remove endian include

[Alex Bennée]

This doesn't exist in BSD world and doesn't seem to be needed by
either.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Warner Losh <imp@bsdimp.com>
Message-Id: <20210917162332.3511179-3-alex.bennee@linaro.org>

diff --git a/tests/tcg/multiarch/sha1.c b/tests/tcg/multiarch/sha1.c
index 87bfbcdf52..0081bd7657 100644
--- a/tests/tcg/multiarch/sha1.c
+++ b/tests/tcg/multiarch/sha1.c
@@ -43,7 +43,6 @@ void SHA1Init(SHA1_CTX* context);
 void SHA1Update(SHA1_CTX* context, const unsigned char* data, uint32_t len);
 void SHA1Final(unsigned char digest[20], SHA1_CTX* context);
 /* ================ end of sha1.h ================ */
-#include <endian.h>
 
 #define rol(value, bits) (((value) << (bits)) | ((value) >> (32 - (bits))))
 
-- 
2.30.2

[PULL 03/12] tests/tcg: move some multiarch files and make conditional

[Alex Bennée]

We had some messy code to filter out stuff we can't build. Lets junk
that and simplify the logic by pushing some stuff into subdirs. In
particular we move:

  float_helpers into libs - not a standalone test
  linux-test into linux - so we only build on Linux hosts

This allows for at least some of the tests to be nominally usable
by *BSD user builds.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Warner Losh <imp@bsdimp.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Warner Losh <imp@bsdimp.com>
Message-Id: <20210917162332.3511179-4-alex.bennee@linaro.org>

diff --git a/tests/tcg/multiarch/float_helpers.c b/tests/tcg/multiarch/libs/float_helpers.c
similarity index 99%
rename from tests/tcg/multiarch/float_helpers.c
rename to tests/tcg/multiarch/libs/float_helpers.c
index bc530e5732..4e68d2b659 100644
--- a/tests/tcg/multiarch/float_helpers.c
+++ b/tests/tcg/multiarch/libs/float_helpers.c
@@ -22,7 +22,7 @@
 #include <float.h>
 #include <fenv.h>
 
-#include "float_helpers.h"
+#include "../float_helpers.h"
 
 #define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
 
diff --git a/tests/tcg/multiarch/linux-test.c b/tests/tcg/multiarch/linux/linux-test.c
similarity index 100%
rename from tests/tcg/multiarch/linux-test.c
rename to tests/tcg/multiarch/linux/linux-test.c
diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target
index 3f283eabe6..6ccb592aac 100644
--- a/tests/tcg/multiarch/Makefile.target
+++ b/tests/tcg/multiarch/Makefile.target
@@ -8,18 +8,23 @@
 MULTIARCH_SRC=$(SRC_PATH)/tests/tcg/multiarch
 
 # Set search path for all sources
-VPATH 		+= $(MULTIARCH_SRC)
-MULTIARCH_SRCS   =$(notdir $(wildcard $(MULTIARCH_SRC)/*.c))
-MULTIARCH_TESTS  =$(filter-out float_helpers, $(MULTIARCH_SRCS:.c=))
+VPATH 	       += $(MULTIARCH_SRC)
+MULTIARCH_SRCS =  $(notdir $(wildcard $(MULTIARCH_SRC)/*.c))
+ifneq ($(CONFIG_LINUX),)
+VPATH 	       += $(MULTIARCH_SRC)/linux
+MULTIARCH_SRCS += $(notdir $(wildcard $(MULTIARCH_SRC)/linux/*.c))
+endif
+MULTIARCH_TESTS = $(MULTIARCH_SRCS:.c=)
 
+$(info SRCS=${MULTIARCH_SRCS} and ${MULTIARCH_TESTS})
 #
 # The following are any additional rules needed to build things
 #
 
 
 float_%: LDFLAGS+=-lm
-float_%: float_%.c float_helpers.c
-	$(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< $(MULTIARCH_SRC)/float_helpers.c -o $@ $(LDFLAGS)
+float_%: float_%.c libs/float_helpers.c
+	$(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< $(MULTIARCH_SRC)/libs/float_helpers.c -o $@ $(LDFLAGS)
 
 run-float_%: float_%
 	$(call run-test,$<, $(QEMU) $(QEMU_OPTS) $<,"$< on $(TARGET_NAME)")
diff --git a/tests/tcg/x86_64/Makefile.target b/tests/tcg/x86_64/Makefile.target
index 2151ea6302..d7a7385583 100644
--- a/tests/tcg/x86_64/Makefile.target
+++ b/tests/tcg/x86_64/Makefile.target
@@ -8,8 +8,12 @@
 
 include $(SRC_PATH)/tests/tcg/i386/Makefile.target
 
+ifneq ($(CONFIG_LINUX),)
 X86_64_TESTS += vsyscall
 TESTS=$(MULTIARCH_TESTS) $(X86_64_TESTS) test-x86_64
+else
+TESTS=$(MULTIARCH_TESTS)
+endif
 QEMU_OPTS += -cpu max
 
 test-x86_64: LDFLAGS+=-lm -lc
-- 
2.30.2

[PULL 04/12] tests/docker: promote debian-riscv64-cross to a full image

[Alex Bennée]

To be able to cross build QEMU itself we need to include a few more
libraries. These are only available in Debian's unstable ports repo
for now so we need to base the riscv64 image on sid with the the
minimal libs needed to build QEMU (glib/pixman).

The result works but is not as clean as using build-dep to bring in
more dependencies. However sid is by definition a shifting pile of
sand and by keeping the list of libs minimal we reduce the chance of
having an image we can't build. It's good enough for a basic cross
build testing of TCG.

Cc: "Daniel P. Berrangé" <berrange@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20210914185830.1378771-2-richard.henderson@linaro.org>
[AJB: tweak allow_failure]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210917162332.3511179-5-alex.bennee@linaro.org>

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 0fcebe363a..a3b5b90552 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -134,7 +134,8 @@ ppc64el-debian-cross-container:
 riscv64-debian-cross-container:
   extends: .container_job_template
   stage: containers-layer2
-  needs: ['amd64-debian10-container']
+  # as we are currently based on 'sid/unstable' we may break so...
+  allow_failure: true
   variables:
     NAME: debian-riscv64-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 0806c6f726..450c76a3ca 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -141,7 +141,6 @@ docker-image-debian-mips64-cross: docker-image-debian10
 docker-image-debian-mips64el-cross: docker-image-debian10
 docker-image-debian-mipsel-cross: docker-image-debian10
 docker-image-debian-ppc64el-cross: docker-image-debian10
-docker-image-debian-riscv64-cross: docker-image-debian10
 docker-image-debian-s390x-cross: docker-image-debian10
 docker-image-debian-sh4-cross: docker-image-debian10
 docker-image-debian-sparc64-cross: docker-image-debian10
@@ -180,7 +179,6 @@ DOCKER_PARTIAL_IMAGES += debian-arm64-test-cross
 DOCKER_PARTIAL_IMAGES += debian-powerpc-test-cross
 DOCKER_PARTIAL_IMAGES += debian-hppa-cross
 DOCKER_PARTIAL_IMAGES += debian-m68k-cross debian-mips64-cross
-DOCKER_PARTIAL_IMAGES += debian-riscv64-cross
 DOCKER_PARTIAL_IMAGES += debian-sh4-cross debian-sparc64-cross
 DOCKER_PARTIAL_IMAGES += debian-tricore-cross
 DOCKER_PARTIAL_IMAGES += debian-xtensa-cross
diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker
index 2bbff19772..594d97982c 100644
--- a/tests/docker/dockerfiles/debian-riscv64-cross.docker
+++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker
@@ -1,12 +1,48 @@
 #
-# Docker cross-compiler target
+# Docker cross-compiler target for riscv64
 #
-# This docker target builds on the debian Buster base image.
+# Currently the only distro that gets close to cross compiling riscv64
+# images is Debian Sid (with unofficial ports). As this is a moving
+# target we keep the library list minimal and are aiming to migrate
+# from this hack as soon as we are able.
 #
-FROM qemu/debian10
+FROM docker.io/library/debian:sid-slim
+
+# Add ports
+RUN apt update && \
+    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
+    DEBIAN_FRONTEND=noninteractive eatmydata apt update -yy && \
+    DEBIAN_FRONTEND=noninteractive eatmydata apt upgrade -yy
+
+# Install common build utilities
+RUN DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy \
+    bc \
+    build-essential \
+    ca-certificates \
+    debian-ports-archive-keyring \
+    dpkg-dev \
+    gettext \
+    git \
+    ninja-build \
+    pkg-config \
+    python3
+
+# Add ports and riscv64 architecture
+RUN echo "deb http://ftp.ports.debian.org/debian-ports/ sid main" >> /etc/apt/sources.list
+RUN dpkg --add-architecture riscv64
+
+# Duplicate deb line as deb-src
+RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
 
 RUN apt update && \
     DEBIAN_FRONTEND=noninteractive eatmydata \
     apt install -y --no-install-recommends \
-        gcc-riscv64-linux-gnu \
-        libc6-dev-riscv64-cross
+         gcc-riscv64-linux-gnu \
+         libc6-dev-riscv64-cross \
+         libffi-dev:riscv64 \
+         libglib2.0-dev:riscv64 \
+         libpixman-1-dev:riscv64
+
+# Specify the cross prefix for this image (see tests/docker/common.rc)
+ENV QEMU_CONFIGURE_OPTS --cross-prefix=riscv64-linux-gnu-
+ENV DEF_TARGET_LIST riscv64-softmmu,riscv64-linux-user
-- 
2.30.2

[PULL 05/12] gitlab: Add cross-riscv64-system, cross-riscv64-user

[Alex Bennée]

From: Richard Henderson <richard.henderson@linaro.org>

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20210914185830.1378771-3-richard.henderson@linaro.org>
[AJB: add allow_failure]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210917162332.3511179-6-alex.bennee@linaro.org>

diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index f10168db2e..17d6cb3e45 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -124,6 +124,25 @@ cross-ppc64el-user:
   variables:
     IMAGE: debian-ppc64el-cross
 
+# The riscv64 cross-builds currently use a 'sid' container to get
+# compilers and libraries. Until something more stable is found we
+# allow_failure so as not to block CI.
+cross-riscv64-system:
+  extends: .cross_system_build_job
+  allow_failure: true
+  needs:
+    job: riscv64-debian-cross-container
+  variables:
+    IMAGE: debian-riscv64-cross
+
+cross-riscv64-user:
+  extends: .cross_user_build_job
+  allow_failure: true
+  needs:
+    job: riscv64-debian-cross-container
+  variables:
+    IMAGE: debian-riscv64-cross
+
 cross-s390x-system:
   extends: .cross_system_build_job
   needs:
-- 
2.30.2

[PULL 06/12] travis.yml: Remove the "Release tarball" job

[Alex Bennée]

From: Thomas Huth <thuth@redhat.com>

This is a leftover from the days when we were using Travis excessively,
but since x86 jobs are not really usable there anymore, this job has
likely never been used since many months. Let's simply remove it now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210917094826.466047-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210917162332.3511179-8-alex.bennee@linaro.org>

diff --git a/.travis.yml b/.travis.yml
index 0faddf7b4e..41010ebe6b 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -305,26 +305,3 @@ jobs:
         - CONFIG="--disable-containers --disable-tcg --enable-kvm
                   --disable-tools --host-cc=clang --cxx=clang++"
         - UNRELIABLE=true
-
-    # Release builds
-    # The make-release script expect a QEMU version, so our tag must start with a 'v'.
-    # This is the case when release candidate tags are created.
-    - name: "Release tarball"
-      if: tag IS present AND tag =~ /^v\d+\.\d+(\.\d+)?(-\S*)?$/
-      env:
-        # We want to build from the release tarball
-        - BUILD_DIR="release/build/dir" SRC_DIR="../../.."
-        - BASE_CONFIG="--prefix=$PWD/dist"
-        - CONFIG="--target-list=x86_64-softmmu,aarch64-softmmu,armeb-linux-user,ppc-linux-user"
-        - TEST_CMD="make install -j${JOBS}"
-        - QEMU_VERSION="${TRAVIS_TAG:1}"
-        - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
-      script:
-        - make -C ${SRC_DIR} qemu-${QEMU_VERSION}.tar.bz2
-        - ls -l ${SRC_DIR}/qemu-${QEMU_VERSION}.tar.bz2
-        - tar -xf ${SRC_DIR}/qemu-${QEMU_VERSION}.tar.bz2 && cd qemu-${QEMU_VERSION}
-        - mkdir -p release-build && cd release-build
-        - ../configure ${BASE_CONFIG} ${CONFIG} || { cat config.log meson-logs/meson-log.txt && exit 1; }
-        - make install
-  allow_failures:
-    - env: UNRELIABLE=true
-- 
2.30.2

[PULL 07/12] gitlab: skip the check-patch job on the upstream repo

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

The check-patch job is intended to be used by contributors or
subsystem maintainers to see if there are style mistakes. The
false positive rate is too high to be used in a gating scenario
so should not run it on the upstream repo ever.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20210915125452.1704899-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210917162332.3511179-9-alex.bennee@linaro.org>

diff --git a/.gitlab-ci.d/static_checks.yml b/.gitlab-ci.d/static_checks.yml
index 96dbd9e310..902843f8b3 100644
--- a/.gitlab-ci.d/static_checks.yml
+++ b/.gitlab-ci.d/static_checks.yml
@@ -8,7 +8,7 @@ check-patch:
   variables:
     GIT_DEPTH: 1000
   rules:
-    - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
+    - if: '$CI_PROJECT_NAMESPACE == "qemu-project"'
       when: never
     - when: on_success
       allow_failure: true
-- 
2.30.2

[PULL 08/12] gitlab: fix passing of TEST_TARGETS env to cirrus

[Alex Bennée]

From: Daniel P. Berrangé <berrange@redhat.com>

A typo meant the substitution would not work, and the placeholder in the
target file didn't even exist.

The result was that tests were never run on the FreeBSD and macOS jobs,
only a basic build.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20210915125452.1704899-3-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210917162332.3511179-10-alex.bennee@linaro.org>

diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
index 675db69622..e7b25e7427 100644
--- a/.gitlab-ci.d/cirrus.yml
+++ b/.gitlab-ci.d/cirrus.yml
@@ -35,7 +35,7 @@
           -e "s|[@]PIP3@|$PIP3|g"
           -e "s|[@]PYPI_PKGS@|$PYPI_PKGS|g"
           -e "s|[@]CONFIGURE_ARGS@|$CONFIGURE_ARGS|g"
-          -e "s|[@]TEST_TARGETSS@|$TEST_TARGETSS|g"
+          -e "s|[@]TEST_TARGETS@|$TEST_TARGETS|g"
       <.gitlab-ci.d/cirrus/build.yml >.gitlab-ci.d/cirrus/$NAME.yml
     - cat .gitlab-ci.d/cirrus/$NAME.yml
     - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
diff --git a/.gitlab-ci.d/cirrus/build.yml b/.gitlab-ci.d/cirrus/build.yml
index 857bdc5536..c555f5d36e 100644
--- a/.gitlab-ci.d/cirrus/build.yml
+++ b/.gitlab-ci.d/cirrus/build.yml
@@ -13,6 +13,7 @@ env:
   PYTHON: "@PYTHON@"
   MAKE: "@MAKE@"
   CONFIGURE_ARGS: "@CONFIGURE_ARGS@"
+  TEST_TARGETS: "@TEST_TARGETS@"
 
 build_task:
   install_script:
-- 
2.30.2

[PULL 09/12] plugins/: Add missing functions to symbol list

[Alex Bennée]

From: Lukas Jünger <lukas.junger@greensocs.com>

Some functions of the plugin API were missing in
the symbol list. However, they are all used by
the contributed example plugins. QEMU fails to
load the plugin if the function symbol is not
exported.

Signed-off-by: Lukas Jünger <lukas.junger@greensocs.com>
Message-Id: <20210905140939.638928-2-lukas.junger@greensocs.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210917162332.3511179-11-alex.bennee@linaro.org>

diff --git a/plugins/qemu-plugins.symbols b/plugins/qemu-plugins.symbols
index 67b309ea2a..4834756ba3 100644
--- a/plugins/qemu-plugins.symbols
+++ b/plugins/qemu-plugins.symbols
@@ -1,11 +1,14 @@
 {
   qemu_plugin_bool_parse;
   qemu_plugin_get_hwaddr;
+  qemu_plugin_hwaddr_device_name;
   qemu_plugin_hwaddr_is_io;
+  qemu_plugin_hwaddr_phys_addr;
   qemu_plugin_insn_data;
   qemu_plugin_insn_disas;
   qemu_plugin_insn_haddr;
   qemu_plugin_insn_size;
+  qemu_plugin_insn_symbol;
   qemu_plugin_insn_vaddr;
   qemu_plugin_mem_is_big_endian;
   qemu_plugin_mem_is_sign_extended;
-- 
2.30.2

[PULL 10/12] accel/tcg: re-factor plugin_inject_cb so we can assert insn_idx is valid

[Alex Bennée]

Coverity doesn't know enough about how we have arranged our plugin TCG
ops to know we will always have incremented insn_idx before injecting
the callback. Let us assert it for the benefit of Coverity and protect
ourselves from accidentally breaking the assumption and triggering
harder to grok errors deeper in the code if we attempt a negative
indexed array lookup.

However to get to this point we re-factor the code and remove the
second hand instruction boundary detection in favour of scanning the
full set of ops and using the existing INDEX_op_insn_start to cleanly
detect when the instruction has started. As we no longer need the
plugin specific list of ops we delete that.

My initial benchmarks shows no discernible impact of dropping the
plugin specific ops list.

Fixes: Coverity 1459509
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Cc: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20210917162332.3511179-12-alex.bennee@linaro.org>

diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index ba13ab1151..9f398b9afe 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -493,9 +493,6 @@ typedef struct TCGOp {
 
     /* Next and previous opcodes.  */
     QTAILQ_ENTRY(TCGOp) link;
-#ifdef CONFIG_PLUGIN
-    QSIMPLEQ_ENTRY(TCGOp) plugin_link;
-#endif
 
     /* Arguments for the opcode.  */
     TCGArg args[MAX_OPC_PARAM];
@@ -605,9 +602,6 @@ struct TCGContext {
 
     /* descriptor of the instruction being translated */
     struct qemu_plugin_insn *plugin_insn;
-
-    /* list to quickly access the injected ops */
-    QSIMPLEQ_HEAD(, TCGOp) plugin_ops;
 #endif
 
     GHashTable *const_table[TCG_TYPE_COUNT];
diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index f5fd5f279c..61be64b78c 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -162,11 +162,7 @@ static void gen_empty_mem_helper(void)
 static void gen_plugin_cb_start(enum plugin_gen_from from,
                                 enum plugin_gen_cb type, unsigned wr)
 {
-    TCGOp *op;
-
     tcg_gen_plugin_cb_start(from, type, wr);
-    op = tcg_last_op();
-    QSIMPLEQ_INSERT_TAIL(&tcg_ctx->plugin_ops, op, plugin_link);
 }
 
 static void gen_wrapped(enum plugin_gen_from from,
@@ -706,62 +702,6 @@ static void plugin_gen_disable_mem_helper(const struct qemu_plugin_tb *ptb,
     inject_mem_disable_helper(insn, begin_op);
 }
 
-static void plugin_inject_cb(const struct qemu_plugin_tb *ptb, TCGOp *begin_op,
-                             int insn_idx)
-{
-    enum plugin_gen_from from = begin_op->args[0];
-    enum plugin_gen_cb type = begin_op->args[1];
-
-    switch (from) {
-    case PLUGIN_GEN_FROM_TB:
-        switch (type) {
-        case PLUGIN_GEN_CB_UDATA:
-            plugin_gen_tb_udata(ptb, begin_op);
-            return;
-        case PLUGIN_GEN_CB_INLINE:
-            plugin_gen_tb_inline(ptb, begin_op);
-            return;
-        default:
-            g_assert_not_reached();
-        }
-    case PLUGIN_GEN_FROM_INSN:
-        switch (type) {
-        case PLUGIN_GEN_CB_UDATA:
-            plugin_gen_insn_udata(ptb, begin_op, insn_idx);
-            return;
-        case PLUGIN_GEN_CB_INLINE:
-            plugin_gen_insn_inline(ptb, begin_op, insn_idx);
-            return;
-        case PLUGIN_GEN_ENABLE_MEM_HELPER:
-            plugin_gen_enable_mem_helper(ptb, begin_op, insn_idx);
-            return;
-        default:
-            g_assert_not_reached();
-        }
-    case PLUGIN_GEN_FROM_MEM:
-        switch (type) {
-        case PLUGIN_GEN_CB_MEM:
-            plugin_gen_mem_regular(ptb, begin_op, insn_idx);
-            return;
-        case PLUGIN_GEN_CB_INLINE:
-            plugin_gen_mem_inline(ptb, begin_op, insn_idx);
-            return;
-        default:
-            g_assert_not_reached();
-        }
-    case PLUGIN_GEN_AFTER_INSN:
-        switch (type) {
-        case PLUGIN_GEN_DISABLE_MEM_HELPER:
-            plugin_gen_disable_mem_helper(ptb, begin_op, insn_idx);
-            return;
-        default:
-            g_assert_not_reached();
-        }
-    default:
-        g_assert_not_reached();
-    }
-}
-
 /* #define DEBUG_PLUGIN_GEN_OPS */
 static void pr_ops(void)
 {
@@ -819,21 +759,95 @@ static void pr_ops(void)
 static void plugin_gen_inject(const struct qemu_plugin_tb *plugin_tb)
 {
     TCGOp *op;
-    int insn_idx;
+    int insn_idx = -1;
 
     pr_ops();
-    insn_idx = -1;
-    QSIMPLEQ_FOREACH(op, &tcg_ctx->plugin_ops, plugin_link) {
-        enum plugin_gen_from from = op->args[0];
-        enum plugin_gen_cb type = op->args[1];
-
-        tcg_debug_assert(op->opc == INDEX_op_plugin_cb_start);
-        /* ENABLE_MEM_HELPER is the first callback of an instruction */
-        if (from == PLUGIN_GEN_FROM_INSN &&
-            type == PLUGIN_GEN_ENABLE_MEM_HELPER) {
+
+    QTAILQ_FOREACH(op, &tcg_ctx->ops, link) {
+        switch (op->opc) {
+        case INDEX_op_insn_start:
             insn_idx++;
+            break;
+        case INDEX_op_plugin_cb_start:
+        {
+            enum plugin_gen_from from = op->args[0];
+            enum plugin_gen_cb type = op->args[1];
+
+            switch (from) {
+            case PLUGIN_GEN_FROM_TB:
+            {
+                g_assert(insn_idx == -1);
+
+                switch (type) {
+                case PLUGIN_GEN_CB_UDATA:
+                    plugin_gen_tb_udata(plugin_tb, op);
+                    break;
+                case PLUGIN_GEN_CB_INLINE:
+                    plugin_gen_tb_inline(plugin_tb, op);
+                    break;
+                default:
+                    g_assert_not_reached();
+                }
+                break;
+            }
+            case PLUGIN_GEN_FROM_INSN:
+            {
+                g_assert(insn_idx >= 0);
+
+                switch (type) {
+                case PLUGIN_GEN_CB_UDATA:
+                    plugin_gen_insn_udata(plugin_tb, op, insn_idx);
+                    break;
+                case PLUGIN_GEN_CB_INLINE:
+                    plugin_gen_insn_inline(plugin_tb, op, insn_idx);
+                    break;
+                case PLUGIN_GEN_ENABLE_MEM_HELPER:
+                    plugin_gen_enable_mem_helper(plugin_tb, op, insn_idx);
+                    break;
+                default:
+                    g_assert_not_reached();
+                }
+                break;
+            }
+            case PLUGIN_GEN_FROM_MEM:
+            {
+                g_assert(insn_idx >= 0);
+
+                switch (type) {
+                case PLUGIN_GEN_CB_MEM:
+                    plugin_gen_mem_regular(plugin_tb, op, insn_idx);
+                    break;
+                case PLUGIN_GEN_CB_INLINE:
+                    plugin_gen_mem_inline(plugin_tb, op, insn_idx);
+                    break;
+                default:
+                    g_assert_not_reached();
+                }
+
+                break;
+            }
+            case PLUGIN_GEN_AFTER_INSN:
+            {
+                g_assert(insn_idx >= 0);
+
+                switch (type) {
+                case PLUGIN_GEN_DISABLE_MEM_HELPER:
+                    plugin_gen_disable_mem_helper(plugin_tb, op, insn_idx);
+                    break;
+                default:
+                    g_assert_not_reached();
+                }
+                break;
+            }
+            default:
+                g_assert_not_reached();
+            }
+            break;
+        }
+        default:
+            /* plugins don't care about any other ops */
+            break;
         }
-        plugin_inject_cb(plugin_tb, op, insn_idx);
     }
     pr_ops();
 }
@@ -846,7 +860,6 @@ bool plugin_gen_tb_start(CPUState *cpu, const TranslationBlock *tb, bool mem_onl
     if (test_bit(QEMU_PLUGIN_EV_VCPU_TB_TRANS, cpu->plugin_mask)) {
         ret = true;
 
-        QSIMPLEQ_INIT(&tcg_ctx->plugin_ops);
         ptb->vaddr = tb->pc;
         ptb->vaddr2 = -1;
         get_page_addr_code_hostp(cpu->env_ptr, tb->pc, &ptb->haddr1);
-- 
2.30.2

[PULL 11/12] .github: move repo lockdown to the v2 configuration

[Alex Bennée]

I was getting prompted by GitHub for new permissions but it turns out
per https://github.com/dessant/repo-lockdown/issues/6:

  Repo Lockdown has been rewritten for GitHub Actions, offering new
  features and better control over your automation presets. The legacy
  GitHub App has been deprecated, and the public instance of the app
  has been shut down.

So this is what I've done. As the issues tab is disabled I've removed
the handling for issues from the new version.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20211004154308.2114870-1-alex.bennee@linaro.org>

diff --git a/.github/lockdown.yml b/.github/lockdown.yml
deleted file mode 100644
index d3546bd2bc..0000000000
--- a/.github/lockdown.yml
+++ /dev/null
@@ -1,34 +0,0 @@
-# Configuration for Repo Lockdown - https://github.com/dessant/repo-lockdown
-
-# Close issues and pull requests
-close: true
-
-# Lock issues and pull requests
-lock: true
-
-issues:
-  comment: |
-    Thank you for your interest in the QEMU project.
-
-    This repository is a read-only mirror of the project's repostories hosted
-    at https://gitlab.com/qemu-project/qemu.git.
-    The project does not process issues filed on GitHub.
-
-    The project issues are tracked on GitLab:
-    https://gitlab.com/qemu-project/qemu/-/issues
-
-    QEMU welcomes bug report contributions. You can file new ones on:
-    https://gitlab.com/qemu-project/qemu/-/issues/new
-
-pulls:
-  comment: |
-    Thank you for your interest in the QEMU project.
-
-    This repository is a read-only mirror of the project's repostories hosted
-    on https://gitlab.com/qemu-project/qemu.git.
-    The project does not process merge requests filed on GitHub.
-
-    QEMU welcomes contributions of code (either fixing bugs or adding new
-    functionality). However, we get a lot of patches, and so we have some
-    guidelines about contributing on the project website:
-    https://www.qemu.org/contribute/
diff --git a/.github/workflows/lockdown.yml b/.github/workflows/lockdown.yml
new file mode 100644
index 0000000000..ad8b8f7e30
--- /dev/null
+++ b/.github/workflows/lockdown.yml
@@ -0,0 +1,30 @@
+# Configuration for Repo Lockdown - https://github.com/dessant/repo-lockdown
+
+name: 'Repo Lockdown'
+
+on:
+  pull_request_target:
+    types: opened
+
+permissions:
+  pull-requests: write
+
+jobs:
+  action:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: dessant/repo-lockdown@v2
+        with:
+          pull-comment: |
+            Thank you for your interest in the QEMU project.
+
+            This repository is a read-only mirror of the project's repostories hosted
+            on https://gitlab.com/qemu-project/qemu.git.
+            The project does not process merge requests filed on GitHub.
+
+            QEMU welcomes contributions of code (either fixing bugs or adding new
+            functionality). However, we get a lot of patches, and so we have some
+            guidelines about contributing on the project website:
+            https://www.qemu.org/contribute/
+          lock-pull: true
+          close-pull: true
-- 
2.30.2

[PULL 12/12] tests/docker: add a debian-native image and make available

[Alex Bennée]

This image is intended for building whatever the native versions of
QEMU are for the host architecture. This will hopefully be an aid for
3rd parties who want to be able to build QEMU themselves without
redoing all the dependencies themselves.

We disable the registry because we currently don't have multi-arch
support there.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Anders Roxell <anders.roxell@linaro.org>
Acked-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210922151528.2192966-1-alex.bennee@linaro.org>

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 450c76a3ca..b9d4094c2e 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -145,6 +145,10 @@ docker-image-debian-s390x-cross: docker-image-debian10
 docker-image-debian-sh4-cross: docker-image-debian10
 docker-image-debian-sparc64-cross: docker-image-debian10
 
+# The native build should never use the registry
+docker-image-debian-native: DOCKER_REGISTRY=
+
+
 #
 # The build rule for hexagon-cross is special in so far for most of
 # the time we don't want to build it. While dockers caching does avoid
diff --git a/tests/docker/common.rc b/tests/docker/common.rc
index c5cc33d366..e6f8cee0d6 100755
--- a/tests/docker/common.rc
+++ b/tests/docker/common.rc
@@ -12,8 +12,14 @@
 # the top-level directory.
 
 # This might be set by ENV of a docker container... it is always
-# overriden by TARGET_LIST if the user sets it.
-DEF_TARGET_LIST=${DEF_TARGET_LIST:-"x86_64-softmmu,aarch64-softmmu"}
+# overriden by TARGET_LIST if the user sets it. We special case
+# "none" to allow for other options like --disable-tcg to restrict the
+# builds we eventually do.
+if test "$DEF_TARGET_LIST" = "none"; then
+    DEF_TARGET_LIST=""
+else
+    DEF_TARGET_LIST=${DEF_TARGET_LIST:-"x86_64-softmmu,aarch64-softmmu"}
+fi
 
 requires_binary()
 {
diff --git a/tests/docker/dockerfiles/debian-native.docker b/tests/docker/dockerfiles/debian-native.docker
new file mode 100644
index 0000000000..efd55cb6e0
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-native.docker
@@ -0,0 +1,49 @@
+#
+# Docker Debian Native
+#
+# This this intended to build QEMU on native host systems. Debian is
+# chosen due to the broadest range on supported host systems for QEMU.
+#
+# This docker target is based on the docker.io Debian Bullseye base
+# image rather than QEMU's base because we would otherwise confuse the
+# build grabbing stuff from the registry built for other
+# architectures.
+#
+FROM docker.io/library/debian:bullseye-slim
+MAINTAINER Alex Bennée <alex.bennee@linaro.org>
+
+# Duplicate deb line as deb-src
+RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
+
+# Install common build utilities
+RUN apt update && \
+    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata
+
+RUN apt update && \
+    DEBIAN_FRONTEND=noninteractive eatmydata \
+    apt build-dep -yy --arch-only qemu
+
+RUN apt update && \
+    DEBIAN_FRONTEND=noninteractive eatmydata \
+    apt install -y --no-install-recommends \
+        cscope \
+        genisoimage \
+        exuberant-ctags \
+        global \
+        libbz2-dev \
+        liblzo2-dev \
+        libgcrypt20-dev \
+        libfdt-dev \
+        librdmacm-dev \
+        libsasl2-dev \
+        libsnappy-dev \
+        libvte-dev \
+        netcat-openbsd \
+        ninja-build \
+        openssh-client \
+        python3-numpy \
+        python3-opencv \
+        python3-venv
+
+ENV QEMU_CONFIGURE_OPTS $QEMU_CONFIGURE_OPTS
+ENV DEF_TARGET_LIST "none"
-- 
2.30.2

Re: [PULL 00/12] some testing and plugin updates

[Richard Henderson]

On 10/8/21 5:25 AM, Alex Bennée wrote:
> The following changes since commit 14f12119aa675e9e28207a48b0728a2daa5b88d6:
> 
>    Merge remote-tracking branch 'remotes/vsementsov/tags/pull-jobs-2021-10-07-v2' into staging (2021-10-07 10:26:35 -0700)
> 
> are available in the Git repository at:
> 
>    https://github.com/stsquad/qemu.git tags/pull-for-6.2-081021-1
> 
> for you to fetch changes up to 755c4aefd86f8b8eda1eb37f20024765c70ebbcb:
> 
>    tests/docker: add a debian-native image and make available (2021-10-08 12:55:27 +0100)
> 
> ----------------------------------------------------------------
> Some testing and plugin updates:
> 
>    - don't override the test compiler when specified
>    - split some multiarch tests by guest OS
>    - add riscv64 docker image and cross-compile tests
>    - drop release tarball test from Travis
>    - skip check-patch on master repo
>    - fix passing of TEST_TARGETS to cirrus
>    - fix missing symbols in plugins
>    - refactor plugin instruction boundary detection
>    - update github repo lockdown
>    - add a debian-native test image for multi-arch builds
> 
> ----------------------------------------------------------------
> Alex Bennée (7):
>        configure: don't override the selected host test compiler if defined
>        tests/tcg/sha1: remove endian include
>        tests/tcg: move some multiarch files and make conditional
>        tests/docker: promote debian-riscv64-cross to a full image
>        accel/tcg: re-factor plugin_inject_cb so we can assert insn_idx is valid
>        .github: move repo lockdown to the v2 configuration
>        tests/docker: add a debian-native image and make available
> 
> Daniel P. Berrangé (2):
>        gitlab: skip the check-patch job on the upstream repo
>        gitlab: fix passing of TEST_TARGETS env to cirrus
> 
> Lukas Jünger (1):
>        plugins/: Add missing functions to symbol list
> 
> Richard Henderson (1):
>        gitlab: Add cross-riscv64-system, cross-riscv64-user
> 
> Thomas Huth (1):
>        travis.yml: Remove the "Release tarball" job
> 
>   configure                                          |   6 +-
>   include/tcg/tcg.h                                  |   6 -
>   accel/tcg/plugin-gen.c                             | 157 +++++++++++----------
>   tests/tcg/multiarch/{ => libs}/float_helpers.c     |   2 +-
>   tests/tcg/multiarch/{ => linux}/linux-test.c       |   0
>   tests/tcg/multiarch/sha1.c                         |   1 -
>   .github/lockdown.yml                               |  34 -----
>   .github/workflows/lockdown.yml                     |  30 ++++
>   .gitlab-ci.d/cirrus.yml                            |   2 +-
>   .gitlab-ci.d/cirrus/build.yml                      |   1 +
>   .gitlab-ci.d/container-cross.yml                   |   3 +-
>   .gitlab-ci.d/crossbuilds.yml                       |  19 +++
>   .gitlab-ci.d/static_checks.yml                     |   2 +-
>   .travis.yml                                        |  23 ---
>   plugins/qemu-plugins.symbols                       |   3 +
>   tests/docker/Makefile.include                      |   6 +-
>   tests/docker/common.rc                             |  10 +-
>   tests/docker/dockerfiles/debian-native.docker      |  49 +++++++
>   .../docker/dockerfiles/debian-riscv64-cross.docker |  46 +++++-
>   tests/tcg/multiarch/Makefile.target                |  15 +-
>   tests/tcg/x86_64/Makefile.target                   |   4 +
>   21 files changed, 263 insertions(+), 156 deletions(-)
>   rename tests/tcg/multiarch/{ => libs}/float_helpers.c (99%)
>   rename tests/tcg/multiarch/{ => linux}/linux-test.c (100%)
>   delete mode 100644 .github/lockdown.yml
>   create mode 100644 .github/workflows/lockdown.yml
>   create mode 100644 tests/docker/dockerfiles/debian-native.docker

I'm seeing:

   TEST    threadcount-with-libmem.so on s390x
**
ERROR:../accel/tcg/plugin-gen.c:795:plugin_gen_inject: assertion failed: (insn_idx >= 0)
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
timeout: the monitored command dumped core
Segmentation fault

on the build-user job.

https://gitlab.com/qemu-project/qemu/-/jobs/1662530140


r~