* [PULL 01/21] tests/docker: Update debian-hexagon-cross to a newer toolchain
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 02/21] gitlab-ci: Remove special casing for hexagon testing Alex Bennée
` (20 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Brian Cain, Thomas Huth, richard.henderson, qemu-devel,
Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
Willian Rampazzo, Taylor Simpson, Alex Bennée
From: Richard Henderson <richard.henderson@linaro.org>
The new dockerfile uses a complete toolchain built by CodeLinaro.
Remove the special casing that was necessary when the dockerfile
built the toolchain from source.
Suggested-by: Brian Cain <bcain@quicinc.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
[AJB: fix MAINTAINERS, restore local build-dep, skip signals]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211014224435.2539547-4-richard.henderson@linaro.org>
diff --git a/MAINTAINERS b/MAINTAINERS
index 7cbae14422..b7ef31e701 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -205,7 +205,6 @@ F: tests/tcg/hexagon/
F: disas/hexagon.c
F: configs/targets/hexagon-linux-user/default.mak
F: docker/dockerfiles/debian-hexagon-cross.docker
-F: docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
HPPA (PA-RISC) TCG CPUs
M: Richard Henderson <richard.henderson@linaro.org>
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index b9d4094c2e..d223100dff 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -148,33 +148,11 @@ docker-image-debian-sparc64-cross: docker-image-debian10
# The native build should never use the registry
docker-image-debian-native: DOCKER_REGISTRY=
-
-#
-# The build rule for hexagon-cross is special in so far for most of
-# the time we don't want to build it. While dockers caching does avoid
-# this most of the time sometimes we want to force the issue.
-#
-docker-image-debian-hexagon-cross: $(DOCKER_FILES_DIR)/debian-hexagon-cross.docker
- $(if $(NOCACHE), \
- $(call quiet-command, \
- $(DOCKER_SCRIPT) build -t qemu/debian-hexagon-cross -f $< \
- $(if $V,,--quiet) --no-cache \
- --registry $(DOCKER_REGISTRY) --extra-files \
- $(DOCKER_FILES_DIR)/debian-hexagon-cross.docker.d/build-toolchain.sh, \
- "BUILD", "debian-hexagon-cross"), \
- $(call quiet-command, \
- $(DOCKER_SCRIPT) fetch $(if $V,,--quiet) \
- qemu/debian-hexagon-cross $(DOCKER_REGISTRY), \
- "FETCH", "debian-hexagon-cross") \
- $(call quiet-command, \
- $(DOCKER_SCRIPT) update $(if $V,,--quiet) \
- qemu/debian-hexagon-cross --add-current-user, \
- "PREPARE", "debian-hexagon-cross"))
-
# Specialist build images, sometimes very limited tools
docker-image-debian-tricore-cross: docker-image-debian10
docker-image-debian-all-test-cross: docker-image-debian10
docker-image-debian-arm64-test-cross: docker-image-debian11
+docker-image-debian-hexagon-cross: docker-image-debian10
docker-image-debian-powerpc-test-cross: docker-image-debian11
# These images may be good enough for building tests but not for test builds
diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.docker b/tests/docker/dockerfiles/debian-hexagon-cross.docker
index d5dc299dc1..f6f8898645 100644
--- a/tests/docker/dockerfiles/debian-hexagon-cross.docker
+++ b/tests/docker/dockerfiles/debian-hexagon-cross.docker
@@ -1,45 +1,18 @@
#
# Docker Hexagon cross-compiler target
#
-# This docker target is used for building hexagon tests. As it also
-# needs to be able to build QEMU itself in CI we include it's
-# build-deps. It is also a "stand-alone" image so as not to be
-# triggered by re-builds on other base images given it takes a long
-# time to build.
-#
FROM qemu/debian10
-# Install common build utilities
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
+RUN DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends \
- bison \
- cmake \
- flex \
- lld \
- rsync \
- wget
-
-ENV TOOLCHAIN_INSTALL /usr/local
-ENV ROOTFS /usr/local
-
-ENV LLVM_URL https://github.com/llvm/llvm-project/archive/bfcd21876adc3498065e4da92799f613e730d475.tar.gz
-ENV MUSL_URL https://github.com/quic/musl/archive/aff74b395fbf59cd7e93b3691905aa1af6c0778c.tar.gz
-ENV LINUX_URL https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.6.18.tar.xz
-
-ADD build-toolchain.sh /root/hexagon-toolchain/build-toolchain.sh
+ curl \
+ xz-utils
+RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+ apt build-dep -yy --arch-only qemu
-RUN cd /root/hexagon-toolchain && ./build-toolchain.sh
+RUN curl -#SL https://codelinaro.jfrog.io/artifactory/codelinaro-toolchain-for-hexagon/v2021.09.10/clang+llvm-Sept-2021-cross-hexagon-unknown-linux-musl.tar.xz \
+ | tar -xJC /opt
-FROM debian:buster-slim
-# Duplicate deb line as deb-src
-RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
-# Install QEMU build deps for use in CI
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
- DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy git ninja-build && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt build-dep -yy --arch-only qemu
-COPY --from=0 /usr/local /usr/local
-ENV PATH $PATH:/usr/local/bin/
+ENV PATH $PATH:/opt/clang+llvm-Sept-2021-cross-hexagon-unknown-linux-musl/x86_64-linux-gnu/bin
+ENV DEF_TARGET_LIST hexagon-linux-user
+ENV QEMU_CONFIGURE_OPTS --disable-docs --disable-tools --cross-cc-hexagon=hexagon-unknown-linux-musl-clang
diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh b/tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
deleted file mode 100755
index 19b1c9f83e..0000000000
--- a/tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
+++ /dev/null
@@ -1,141 +0,0 @@
-#!/bin/bash
-
-set -e
-
-BASE=$(readlink -f ${PWD})
-
-TOOLCHAIN_INSTALL=$(readlink -f "$TOOLCHAIN_INSTALL")
-ROOTFS=$(readlink -f "$ROOTFS")
-
-TOOLCHAIN_BIN=${TOOLCHAIN_INSTALL}/bin
-HEX_SYSROOT=${TOOLCHAIN_INSTALL}/hexagon-unknown-linux-musl
-HEX_TOOLS_TARGET_BASE=${HEX_SYSROOT}/usr
-
-function cdp() {
- DIR="$1"
- mkdir -p "$DIR"
- cd "$DIR"
-}
-
-function fetch() {
- DIR="$1"
- URL="$2"
- TEMP="$(readlink -f "$PWD/tmp.tar.gz")"
- wget --quiet "$URL" -O "$TEMP"
- cdp "$DIR"
- tar xaf "$TEMP" --strip-components=1
- rm "$TEMP"
- cd -
-}
-
-build_llvm_clang() {
- fetch "$BASE/llvm-project" "$LLVM_URL"
- cdp "$BASE/build-llvm"
-
- cmake -G Ninja \
- -DCMAKE_BUILD_TYPE=Release \
- -DCMAKE_INSTALL_PREFIX=${TOOLCHAIN_INSTALL} \
- -DLLVM_ENABLE_LLD=ON \
- -DLLVM_TARGETS_TO_BUILD="Hexagon" \
- -DLLVM_ENABLE_PROJECTS="clang;lld" \
- "$BASE/llvm-project/llvm"
- ninja all install
- cd ${TOOLCHAIN_BIN}
- ln -sf clang hexagon-unknown-linux-musl-clang
- ln -sf clang++ hexagon-unknown-linux-musl-clang++
- ln -sf llvm-ar hexagon-unknown-linux-musl-ar
- ln -sf llvm-objdump hexagon-unknown-linux-musl-objdump
- ln -sf llvm-objcopy hexagon-unknown-linux-musl-objcopy
- ln -sf llvm-readelf hexagon-unknown-linux-musl-readelf
- ln -sf llvm-ranlib hexagon-unknown-linux-musl-ranlib
-
- # workaround for now:
- cat <<EOF > hexagon-unknown-linux-musl.cfg
--G0 --sysroot=${HEX_SYSROOT}
-EOF
-}
-
-build_clang_rt() {
- cdp "$BASE/build-clang_rt"
- cmake -G Ninja \
- -DCMAKE_BUILD_TYPE=Release \
- -DLLVM_CONFIG_PATH="$BASE/build-llvm/bin/llvm-config" \
- -DCMAKE_ASM_FLAGS="-G0 -mlong-calls -fno-pic --target=hexagon-unknown-linux-musl " \
- -DCMAKE_SYSTEM_NAME=Linux \
- -DCMAKE_C_COMPILER="${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang" \
- -DCMAKE_ASM_COMPILER="${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang" \
- -DCMAKE_INSTALL_PREFIX=${HEX_TOOLS_TARGET_BASE} \
- -DCMAKE_CROSSCOMPILING=ON \
- -DCMAKE_C_COMPILER_FORCED=ON \
- -DCMAKE_CXX_COMPILER_FORCED=ON \
- -DCOMPILER_RT_BUILD_BUILTINS=ON \
- -DCOMPILER_RT_BUILTINS_ENABLE_PIC=OFF \
- -DCMAKE_SIZEOF_VOID_P=4 \
- -DCOMPILER_RT_OS_DIR= \
- -DCAN_TARGET_hexagon=1 \
- -DCAN_TARGET_x86_64=0 \
- -DCOMPILER_RT_SUPPORTED_ARCH=hexagon \
- -DLLVM_ENABLE_PROJECTS="compiler-rt" \
- "$BASE/llvm-project/compiler-rt"
- ninja install-compiler-rt
-}
-
-build_musl_headers() {
- fetch "$BASE/musl" "$MUSL_URL"
- cd "$BASE/musl"
- make clean
- CC=${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang \
- CROSS_COMPILE=hexagon-unknown-linux-musl \
- LIBCC=${HEX_TOOLS_TARGET_BASE}/lib/libclang_rt.builtins-hexagon.a \
- CROSS_CFLAGS="-G0 -O0 -mv65 -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl" \
- ./configure --target=hexagon --prefix=${HEX_TOOLS_TARGET_BASE}
- PATH=${TOOLCHAIN_BIN}:$PATH make CROSS_COMPILE= install-headers
-
- cd ${HEX_SYSROOT}/..
- ln -sf hexagon-unknown-linux-musl hexagon
-}
-
-build_kernel_headers() {
- fetch "$BASE/linux" "$LINUX_URL"
- mkdir -p "$BASE/build-linux"
- cd "$BASE/linux"
- make O=../build-linux ARCH=hexagon \
- KBUILD_CFLAGS_KERNEL="-mlong-calls" \
- CC=${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang \
- LD=${TOOLCHAIN_BIN}/ld.lld \
- KBUILD_VERBOSE=1 comet_defconfig
- make mrproper
-
- cd "$BASE/build-linux"
- make \
- ARCH=hexagon \
- CC=${TOOLCHAIN_BIN}/clang \
- INSTALL_HDR_PATH=${HEX_TOOLS_TARGET_BASE} \
- V=1 \
- headers_install
-}
-
-build_musl() {
- cd "$BASE/musl"
- make clean
- CROSS_COMPILE=hexagon-unknown-linux-musl- \
- AR=llvm-ar \
- RANLIB=llvm-ranlib \
- STRIP=llvm-strip \
- CC=clang \
- LIBCC=${HEX_TOOLS_TARGET_BASE}/lib/libclang_rt.builtins-hexagon.a \
- CFLAGS="-G0 -O0 -mv65 -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl" \
- ./configure --target=hexagon --prefix=${HEX_TOOLS_TARGET_BASE}
- PATH=${TOOLCHAIN_BIN}/:$PATH make CROSS_COMPILE= install
- cd ${HEX_TOOLS_TARGET_BASE}/lib
- ln -sf libc.so ld-musl-hexagon.so
- ln -sf ld-musl-hexagon.so ld-musl-hexagon.so.1
- cdp ${HEX_TOOLS_TARGET_BASE}/../lib
- ln -sf ../usr/lib/ld-musl-hexagon.so.1
-}
-
-build_llvm_clang
-build_kernel_headers
-build_musl_headers
-build_clang_rt
-build_musl
diff --git a/tests/tcg/hexagon/Makefile.target b/tests/tcg/hexagon/Makefile.target
index 8b07a28166..84a722e1bc 100644
--- a/tests/tcg/hexagon/Makefile.target
+++ b/tests/tcg/hexagon/Makefile.target
@@ -21,6 +21,12 @@ EXTRA_RUNS =
CFLAGS += -Wno-incompatible-pointer-types -Wno-undefined-internal
CFLAGS += -fno-unroll-loops
+# There is a 4-7% failure rate which becomes 100% during CI
+run-signals: signals
+ $(call skip-test, $<, "BROKEN")
+run-plugin-signals-with-%:
+ $(call skip-test, $<, "BROKEN")
+
HEX_SRC=$(SRC_PATH)/tests/tcg/hexagon
VPATH += $(HEX_SRC)
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 02/21] gitlab-ci: Remove special casing for hexagon testing
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
2021-11-03 17:05 ` [PULL 01/21] tests/docker: Update debian-hexagon-cross to a newer toolchain Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 03/21] tests/docker: Add debian-nios2-cross image Alex Bennée
` (19 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Thomas Huth, richard.henderson, qemu-devel,
Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
Willian Rampazzo, Alex Bennée
From: Richard Henderson <richard.henderson@linaro.org>
Now that the hexagon container is using a pre-built
toolchain, we do not need to make the build optional.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211014224435.2539547-5-richard.henderson@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20211026102234.3961636-5-alex.bennee@linaro.org>
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 6c1301e912..3031917840 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -249,14 +249,10 @@ build-user-static:
CONFIGURE_ARGS: --disable-tools --disable-system --static
MAKE_CHECK_ARGS: check-tcg
-# Because the hexagon cross-compiler takes so long to build we don't rely
-# on the CI system to build it and hence this job has an optional dependency
-# declared. The image is manually uploaded.
build-user-hexagon:
extends: .native_build_job_template
needs:
job: hexagon-cross-container
- optional: true
variables:
IMAGE: debian-hexagon-cross
TARGETS: hexagon-linux-user
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index a3b5b90552..ed7a849a69 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -47,33 +47,12 @@ armhf-debian-cross-container:
variables:
NAME: debian-armhf-cross
-# We never want to build hexagon in the CI system and by default we
-# always want to refer to the master registry where it lives.
hexagon-cross-container:
- image: docker:stable
- stage: containers
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project"'
- when: never
- - when: always
+ extends: .container_job_template
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
variables:
NAME: debian-hexagon-cross
- GIT_DEPTH: 1
- services:
- - docker:dind
- before_script:
- - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
- - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/qemu/$NAME:latest"
- - docker info
- - docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD"
- script:
- - echo "TAG:$TAG"
- - echo "COMMON_TAG:$COMMON_TAG"
- - docker pull $COMMON_TAG
- - docker tag $COMMON_TAG $TAG
- - docker push "$TAG"
- after_script:
- - docker logout
hppa-debian-cross-container:
extends: .container_job_template
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 03/21] tests/docker: Add debian-nios2-cross image
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
2021-11-03 17:05 ` [PULL 01/21] tests/docker: Update debian-hexagon-cross to a newer toolchain Alex Bennée
2021-11-03 17:05 ` [PULL 02/21] gitlab-ci: Remove special casing for hexagon testing Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 04/21] tests/docker: Add debian-microblaze-cross image Alex Bennée
` (18 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Marek Vasut, Thomas Huth, Chris Wulff, richard.henderson,
qemu-devel, Wainer dos Santos Moschetta,
Philippe Mathieu-Daudé,
Willian Rampazzo, Alex Bennée
From: Richard Henderson <richard.henderson@linaro.org>
Build the entire cross tool chain from source.
For this reason, default to caching.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
[AJB: honour NOUSER in cached fetch and build, update MAINTAINERS]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211014224435.2539547-6-richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20211026102234.3961636-6-alex.bennee@linaro.org>
diff --git a/MAINTAINERS b/MAINTAINERS
index b7ef31e701..e07134aa28 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -252,6 +252,7 @@ F: target/nios2/
F: hw/nios2/
F: disas/nios2.c
F: configs/devices/nios2-softmmu/default.mak
+F: tests/docker/dockerfiles/debian-nios2-cross.d/build-toolchain.sh
OpenRISC TCG CPUs
M: Stafford Horne <shorne@gmail.com>
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index d223100dff..24ed5c1f1d 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -148,11 +148,35 @@ docker-image-debian-sparc64-cross: docker-image-debian10
# The native build should never use the registry
docker-image-debian-native: DOCKER_REGISTRY=
+debian-toolchain-run = \
+ $(if $(NOCACHE), \
+ $(call quiet-command, \
+ $(DOCKER_SCRIPT) build -t qemu/$1 -f $< \
+ $(if $V,,--quiet) --no-cache \
+ --registry $(DOCKER_REGISTRY) --extra-files \
+ $(DOCKER_FILES_DIR)/$1.d/build-toolchain.sh, \
+ "BUILD", $1), \
+ $(call quiet-command, \
+ $(DOCKER_SCRIPT) fetch $(if $V,,--quiet) \
+ qemu/$1 $(DOCKER_REGISTRY), \
+ "FETCH", $1) \
+ $(call quiet-command, \
+ $(DOCKER_SCRIPT) update $(if $V,,--quiet) \
+ qemu/$1 \
+ $(if $(NOUSER),,--add-current-user) \
+ "PREPARE", $1))
+debian-toolchain = $(call debian-toolchain-run,$(patsubst docker-image-%,%,$1))
+
+docker-image-debian-nios2-cross: $(DOCKER_FILES_DIR)/debian-toolchain.docker \
+ $(DOCKER_FILES_DIR)/debian-nios2-cross.d/build-toolchain.sh
+ $(call debian-toolchain, $@)
+
# Specialist build images, sometimes very limited tools
docker-image-debian-tricore-cross: docker-image-debian10
docker-image-debian-all-test-cross: docker-image-debian10
docker-image-debian-arm64-test-cross: docker-image-debian11
docker-image-debian-hexagon-cross: docker-image-debian10
+docker-image-debian-nios2-cross: docker-image-debian10
docker-image-debian-powerpc-test-cross: docker-image-debian11
# These images may be good enough for building tests but not for test builds
@@ -161,6 +185,7 @@ DOCKER_PARTIAL_IMAGES += debian-arm64-test-cross
DOCKER_PARTIAL_IMAGES += debian-powerpc-test-cross
DOCKER_PARTIAL_IMAGES += debian-hppa-cross
DOCKER_PARTIAL_IMAGES += debian-m68k-cross debian-mips64-cross
+DOCKER_PARTIAL_IMAGES += debian-nios2-cross
DOCKER_PARTIAL_IMAGES += debian-sh4-cross debian-sparc64-cross
DOCKER_PARTIAL_IMAGES += debian-tricore-cross
DOCKER_PARTIAL_IMAGES += debian-xtensa-cross
diff --git a/tests/docker/dockerfiles/debian-nios2-cross.d/build-toolchain.sh b/tests/docker/dockerfiles/debian-nios2-cross.d/build-toolchain.sh
new file mode 100755
index 0000000000..ba3c9d8aff
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-nios2-cross.d/build-toolchain.sh
@@ -0,0 +1,87 @@
+#!/bin/bash
+
+set -e
+
+TARGET=nios2-linux-gnu
+LINUX_ARCH=nios2
+
+J=$(expr $(nproc) / 2)
+TOOLCHAIN_INSTALL=/usr/local
+TOOLCHAIN_BIN=${TOOLCHAIN_INSTALL}/bin
+CROSS_SYSROOT=${TOOLCHAIN_INSTALL}/$TARGET/sys-root
+
+export PATH=${TOOLCHAIN_BIN}:$PATH
+
+#
+# Grab all of the source for the toolchain bootstrap.
+#
+
+wget https://ftp.gnu.org/gnu/binutils/binutils-2.37.tar.xz
+wget https://ftp.gnu.org/gnu/gcc/gcc-11.2.0/gcc-11.2.0.tar.xz
+wget https://ftp.gnu.org/gnu/glibc/glibc-2.34.tar.xz
+wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.10.70.tar.xz
+
+tar axf binutils-2.37.tar.xz
+tar axf gcc-11.2.0.tar.xz
+tar axf glibc-2.34.tar.xz
+tar axf linux-5.10.70.tar.xz
+
+mv binutils-2.37 src-binu
+mv gcc-11.2.0 src-gcc
+mv glibc-2.34 src-glibc
+mv linux-5.10.70 src-linux
+
+mkdir -p bld-hdr bld-binu bld-gcc bld-glibc
+mkdir -p ${CROSS_SYSROOT}/usr/include
+
+#
+# Install kernel and glibc headers
+#
+
+cd src-linux
+make headers_install ARCH=${LINUX_ARCH} INSTALL_HDR_PATH=${CROSS_SYSROOT}/usr
+cd ..
+
+cd bld-hdr
+../src-glibc/configure --prefix=/usr --host=${TARGET}
+make install-headers DESTDIR=${CROSS_SYSROOT}
+touch ${CROSS_SYSROOT}/usr/include/gnu/stubs.h
+cd ..
+
+#
+# Build binutils
+#
+
+cd bld-binu
+../src-binu/configure --disable-werror \
+ --prefix=${TOOLCHAIN_INSTALL} --with-sysroot --target=${TARGET}
+make -j${J}
+make install
+cd ..
+
+#
+# Build gcc, without shared libraries, because we do not yet
+# have a shared libc against which to link.
+#
+
+cd bld-gcc
+../src-gcc/configure --disable-werror --disable-shared \
+ --prefix=${TOOLCHAIN_INSTALL} --with-sysroot --target=${TARGET} \
+ --enable-languages=c --disable-libssp --disable-libsanitizer \
+ --disable-libatomic --disable-libgomp --disable-libquadmath
+make -j${J}
+make install
+cd ..
+
+#
+# Build glibc
+# There are a few random things that use c++ but we didn't build that
+# cross-compiler. We can get away without them. Disable CXX so that
+# glibc doesn't try to use the host c++ compiler.
+#
+
+cd bld-glibc
+CXX=false ../src-glibc/configure --prefix=/usr --host=${TARGET}
+make -j${j}
+make install DESTDIR=${CROSS_SYSROOT}
+cd ..
diff --git a/tests/docker/dockerfiles/debian-toolchain.docker b/tests/docker/dockerfiles/debian-toolchain.docker
new file mode 100644
index 0000000000..738d808aa6
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-toolchain.docker
@@ -0,0 +1,36 @@
+#
+# Docker toolchain cross-compiler
+#
+# This dockerfile is used for building a cross-compiler toolchain.
+# The script for building the toolchain is supplied via extra-files.
+#
+FROM qemu/debian10
+
+# Install build utilities for building gcc and glibc.
+# ??? The build-dep isn't working, missing a number of
+# minimal build dependiencies, e.g. libmpc.
+
+RUN apt update && \
+ DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
+ DEBIAN_FRONTEND=noninteractive eatmydata \
+ apt install -y --no-install-recommends \
+ bison \
+ flex \
+ gawk \
+ libmpc-dev \
+ libmpfr-dev \
+ rsync \
+ texinfo \
+ wget && \
+ DEBIAN_FRONTEND=noninteractive eatmydata \
+ apt build-dep -yy --arch-only gcc glibc
+
+ADD build-toolchain.sh /root/build-toolchain.sh
+
+RUN cd /root && ./build-toolchain.sh
+
+# Throw away the extra toolchain build deps, the downloaded source,
+# and the build trees by restoring the original debian10 image,
+# then copying the built toolchain from stage 0.
+FROM qemu/debian10
+COPY --from=0 /usr/local /usr/local
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 04/21] tests/docker: Add debian-microblaze-cross image
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (2 preceding siblings ...)
2021-11-03 17:05 ` [PULL 03/21] tests/docker: Add debian-nios2-cross image Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 05/21] tests/tcg: Enable container_cross_cc for microblaze Alex Bennée
` (17 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Thomas Huth, richard.henderson, qemu-devel,
Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
Willian Rampazzo, Edgar E. Iglesias, Alex Bennée
From: Richard Henderson <richard.henderson@linaro.org>
Build the entire cross tool chain from source.
For this reason, default to caching.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
[AJB: Update MAINTAINERS]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211014224435.2539547-7-richard.henderson@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20211026102234.3961636-7-alex.bennee@linaro.org>
diff --git a/MAINTAINERS b/MAINTAINERS
index e07134aa28..a4d7a6f2e5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -227,6 +227,7 @@ S: Maintained
F: target/microblaze/
F: hw/microblaze/
F: disas/microblaze.c
+F: tests/docker/dockerfiles/debian-microblaze-cross.d/build-toolchain.sh
MIPS TCG CPUs
M: Philippe Mathieu-Daudé <f4bug@amsat.org>
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 24ed5c1f1d..e943ac33b0 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -167,6 +167,10 @@ debian-toolchain-run = \
"PREPARE", $1))
debian-toolchain = $(call debian-toolchain-run,$(patsubst docker-image-%,%,$1))
+docker-image-debian-microblaze-cross: $(DOCKER_FILES_DIR)/debian-toolchain.docker \
+ $(DOCKER_FILES_DIR)/debian-microblaze-cross.d/build-toolchain.sh
+ $(call debian-toolchain, $@)
+
docker-image-debian-nios2-cross: $(DOCKER_FILES_DIR)/debian-toolchain.docker \
$(DOCKER_FILES_DIR)/debian-nios2-cross.d/build-toolchain.sh
$(call debian-toolchain, $@)
@@ -176,6 +180,7 @@ docker-image-debian-tricore-cross: docker-image-debian10
docker-image-debian-all-test-cross: docker-image-debian10
docker-image-debian-arm64-test-cross: docker-image-debian11
docker-image-debian-hexagon-cross: docker-image-debian10
+docker-image-debian-microblaze-cross: docker-image-debian10
docker-image-debian-nios2-cross: docker-image-debian10
docker-image-debian-powerpc-test-cross: docker-image-debian11
@@ -185,6 +190,7 @@ DOCKER_PARTIAL_IMAGES += debian-arm64-test-cross
DOCKER_PARTIAL_IMAGES += debian-powerpc-test-cross
DOCKER_PARTIAL_IMAGES += debian-hppa-cross
DOCKER_PARTIAL_IMAGES += debian-m68k-cross debian-mips64-cross
+DOCKER_PARTIAL_IMAGES += debian-microblaze-cross
DOCKER_PARTIAL_IMAGES += debian-nios2-cross
DOCKER_PARTIAL_IMAGES += debian-sh4-cross debian-sparc64-cross
DOCKER_PARTIAL_IMAGES += debian-tricore-cross
diff --git a/tests/docker/dockerfiles/debian-microblaze-cross.d/build-toolchain.sh b/tests/docker/dockerfiles/debian-microblaze-cross.d/build-toolchain.sh
new file mode 100755
index 0000000000..23ec0aa9a7
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-microblaze-cross.d/build-toolchain.sh
@@ -0,0 +1,88 @@
+#!/bin/bash
+
+set -e
+
+TARGET=microblaze-linux-musl
+LINUX_ARCH=microblaze
+
+J=$(expr $(nproc) / 2)
+TOOLCHAIN_INSTALL=/usr/local
+TOOLCHAIN_BIN=${TOOLCHAIN_INSTALL}/bin
+CROSS_SYSROOT=${TOOLCHAIN_INSTALL}/$TARGET/sys-root
+
+export PATH=${TOOLCHAIN_BIN}:$PATH
+
+#
+# Grab all of the source for the toolchain bootstrap.
+#
+
+wget https://ftp.gnu.org/gnu/binutils/binutils-2.37.tar.xz
+wget https://ftp.gnu.org/gnu/gcc/gcc-11.2.0/gcc-11.2.0.tar.xz
+wget https://www.musl-libc.org/releases/musl-1.2.2.tar.gz
+wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.10.70.tar.xz
+
+tar axf binutils-2.37.tar.xz
+tar axf gcc-11.2.0.tar.xz
+tar axf musl-1.2.2.tar.gz
+tar axf linux-5.10.70.tar.xz
+
+mv binutils-2.37 src-binu
+mv gcc-11.2.0 src-gcc
+mv musl-1.2.2 src-musl
+mv linux-5.10.70 src-linux
+
+mkdir -p bld-hdr bld-binu bld-gcc bld-musl
+mkdir -p ${CROSS_SYSROOT}/usr/include
+
+#
+# Install kernel headers
+#
+
+cd src-linux
+make headers_install ARCH=${LINUX_ARCH} INSTALL_HDR_PATH=${CROSS_SYSROOT}/usr
+cd ..
+
+#
+# Build binutils
+#
+
+cd bld-binu
+../src-binu/configure --disable-werror \
+ --prefix=${TOOLCHAIN_INSTALL} --with-sysroot --target=${TARGET}
+make -j${J}
+make install
+cd ..
+
+#
+# Build gcc, just the compiler so far.
+#
+
+cd bld-gcc
+../src-gcc/configure --disable-werror --disable-shared \
+ --prefix=${TOOLCHAIN_INSTALL} --with-sysroot --target=${TARGET} \
+ --enable-languages=c --disable-libssp --disable-libsanitizer \
+ --disable-libatomic --disable-libgomp --disable-libquadmath
+make -j${J} all-gcc
+make install-gcc
+cd ..
+
+#
+# Build musl.
+# We won't go through the extra step of building shared libraries
+# because we don't actually use them in QEMU docker testing.
+#
+
+cd bld-musl
+../src-musl/configure --prefix=/usr --host=${TARGET} --disable-shared
+make -j${j}
+make install DESTDIR=${CROSS_SYSROOT}
+cd ..
+
+#
+# Go back and build the compiler runtime
+#
+
+cd bld-gcc
+make -j${j}
+make install
+cd ..
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 05/21] tests/tcg: Enable container_cross_cc for microblaze
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (3 preceding siblings ...)
2021-11-03 17:05 ` [PULL 04/21] tests/docker: Add debian-microblaze-cross image Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 06/21] tests/tcg: Fix some targets default cross compiler path Alex Bennée
` (16 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Alex Bennée, richard.henderson, qemu-devel,
Philippe Mathieu-Daudé
From: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211014224435.2539547-8-richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20211026102234.3961636-8-alex.bennee@linaro.org>
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index 1f985ccfc0..ccf84e5559 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -55,6 +55,7 @@ fi
: ${cross_cc_i386="i686-linux-gnu-gcc"}
: ${cross_cc_cflags_i386="-m32"}
: ${cross_cc_m68k="m68k-linux-gnu-gcc"}
+: ${cross_cc_microblaze="microblaze-linux-musl-gcc"}
: $(cross_cc_mips64el="mips64el-linux-gnuabi64-gcc")
: $(cross_cc_mips64="mips64-linux-gnuabi64-gcc")
: $(cross_cc_mipsel="mipsel-linux-gnu-gcc")
@@ -133,6 +134,11 @@ for target in $target_list; do
container_image=debian-m68k-cross
container_cross_cc=m68k-linux-gnu-gcc
;;
+ microblaze-*)
+ container_hosts=x86_64
+ container_image=debian-microblaze-cross
+ container_cross_cc=microblaze-linux-musl-gcc
+ ;;
mips64el-*)
container_hosts=x86_64
container_image=debian-mips64el-cross
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 06/21] tests/tcg: Fix some targets default cross compiler path
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (4 preceding siblings ...)
2021-11-03 17:05 ` [PULL 05/21] tests/tcg: Enable container_cross_cc for microblaze Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 07/21] tests/docker: split PARTIAL into PARTIAL and VIRTUAL images Alex Bennée
` (15 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Alex Bennée, richard.henderson, qemu-devel,
Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <f4bug@amsat.org>
We do not want a shell command substitution, but a parameter
substitution (with assignment). Replace $() -> ${}, otherwise
the expanded command return an empty string and the $cross_cc
variable is not set.
Fixes: 634ef789f8e ("tests/tcg: add more default compilers to configure.sh")
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: disable sh4 linux-test]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211023164329.328137-1-f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20211026102234.3961636-9-alex.bennee@linaro.org>
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index ccf84e5559..d74ed50c18 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -46,7 +46,7 @@ fi
: ${cross_cc_aarch64="aarch64-linux-gnu-gcc"}
: ${cross_cc_aarch64_be="$cross_cc_aarch64"}
: ${cross_cc_cflags_aarch64_be="-mbig-endian"}
-: $(cross_cc_alpha="alpha-linux-gnu-gcc")
+: ${cross_cc_alpha="alpha-linux-gnu-gcc"}
: ${cross_cc_arm="arm-linux-gnueabihf-gcc"}
: ${cross_cc_cflags_armeb="-mbig-endian"}
: ${cross_cc_hexagon="hexagon-unknown-linux-musl-clang"}
@@ -56,17 +56,17 @@ fi
: ${cross_cc_cflags_i386="-m32"}
: ${cross_cc_m68k="m68k-linux-gnu-gcc"}
: ${cross_cc_microblaze="microblaze-linux-musl-gcc"}
-: $(cross_cc_mips64el="mips64el-linux-gnuabi64-gcc")
-: $(cross_cc_mips64="mips64-linux-gnuabi64-gcc")
-: $(cross_cc_mipsel="mipsel-linux-gnu-gcc")
-: $(cross_cc_mips="mips-linux-gnu-gcc")
+: ${cross_cc_mips64el="mips64el-linux-gnuabi64-gcc"}
+: ${cross_cc_mips64="mips64-linux-gnuabi64-gcc"}
+: ${cross_cc_mipsel="mipsel-linux-gnu-gcc"}
+: ${cross_cc_mips="mips-linux-gnu-gcc"}
: ${cross_cc_ppc="powerpc-linux-gnu-gcc"}
: ${cross_cc_cflags_ppc="-m32"}
: ${cross_cc_ppc64="powerpc64-linux-gnu-gcc"}
: ${cross_cc_ppc64le="powerpc64le-linux-gnu-gcc"}
-: $(cross_cc_riscv64="riscv64-linux-gnu-gcc")
+: ${cross_cc_riscv64="riscv64-linux-gnu-gcc"}
: ${cross_cc_s390x="s390x-linux-gnu-gcc"}
-: $(cross_cc_sh4="sh4-linux-gnu-gcc")
+: ${cross_cc_sh4="sh4-linux-gnu-gcc"}
: ${cross_cc_cflags_sparc="-m32 -mv8plus -mcpu=ultrasparc"}
: ${cross_cc_sparc64="sparc64-linux-gnu-gcc"}
: ${cross_cc_cflags_sparc64="-m64 -mcpu=ultrasparc"}
diff --git a/tests/tcg/sh4/Makefile.target b/tests/tcg/sh4/Makefile.target
index 47c39a44b6..0e96aeff16 100644
--- a/tests/tcg/sh4/Makefile.target
+++ b/tests/tcg/sh4/Makefile.target
@@ -12,3 +12,9 @@ run-signals: signals
$(call skip-test, $<, "BROKEN")
run-plugin-signals-with-%:
$(call skip-test, $<, "BROKEN")
+
+# This test is currently broken: https://gitlab.com/qemu-project/qemu/-/issues/704
+run-linux-test: linux-test
+ $(call skip-test, $<, "BROKEN")
+run-plugin-linux-test-with-%:
+ $(call skip-test, $<, "BROKEN")
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 07/21] tests/docker: split PARTIAL into PARTIAL and VIRTUAL images
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (5 preceding siblings ...)
2021-11-03 17:05 ` [PULL 06/21] tests/tcg: Fix some targets default cross compiler path Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 08/21] tests/tcg: enable debian-nios2-cross for test building Alex Bennée
` (14 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Thomas Huth, richard.henderson, qemu-devel,
Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
Willian Rampazzo, Alex Bennée
This is mostly to ensure we don't include the toolchain and bootstrap
builds in DOCKER_IMAGES which is useful when verifying all images
still build.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20211026102234.3961636-10-alex.bennee@linaro.org>
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index e943ac33b0..5bbbaceed1 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -11,8 +11,10 @@ HOST_ARCH = $(if $(ARCH),$(ARCH),$(shell uname -m))
DOCKER_SUFFIX := .docker
DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
# we don't run tests on intermediate images (used as base by another image)
-DOCKER_PARTIAL_IMAGES := debian10 debian11 debian-bootstrap empty
-DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker))))
+DOCKER_PARTIAL_IMAGES := debian10 debian11
+# we don't directly build virtual images (they are used to build other images)
+DOCKER_VIRTUAL_IMAGES := debian-bootstrap debian-toolchain empty
+DOCKER_IMAGES := $(sort $(filter-out $(DOCKER_VIRTUAL_IMAGES), $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))))
DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
# Use a global constant ccache directory to speed up repetitive builds
DOCKER_CCACHE_DIR := $$HOME/.cache/qemu-docker-ccache
@@ -204,7 +206,7 @@ DOCKER_PARTIAL_IMAGES += fedora-cris-cross
# packages.
# Expand all the pre-requistes for each docker image and test combination
-$(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES)), \
+$(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES) $(DOCKER_VIRTUAL_IMAGES),$(DOCKER_IMAGES)), \
$(foreach t,$(DOCKER_TESTS), \
$(eval .PHONY: docker-$t@$i) \
$(eval docker-$t@$i: docker-image-$i docker-run-$t@$i) \
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 08/21] tests/tcg: enable debian-nios2-cross for test building
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (6 preceding siblings ...)
2021-11-03 17:05 ` [PULL 07/21] tests/docker: split PARTIAL into PARTIAL and VIRTUAL images Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 09/21] ebpf: really include it only in system emulators Alex Bennée
` (13 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Edgar E. Iglesias, Alex Bennée, richard.henderson,
qemu-devel, Philippe Mathieu-Daudé
Now we have a nios2 test image we can start using it to build tests.
However signal handling in nios2 is still broken so we disable the
signals and linux-test tests that trigger the bug.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20211026102234.3961636-12-alex.bennee@linaro.org>
diff --git a/MAINTAINERS b/MAINTAINERS
index a4d7a6f2e5..a2ad6f71b0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -228,6 +228,7 @@ F: target/microblaze/
F: hw/microblaze/
F: disas/microblaze.c
F: tests/docker/dockerfiles/debian-microblaze-cross.d/build-toolchain.sh
+F: tests/tcg/nios2/Makefile.target
MIPS TCG CPUs
M: Philippe Mathieu-Daudé <f4bug@amsat.org>
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index d74ed50c18..9b76f58258 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -60,6 +60,7 @@ fi
: ${cross_cc_mips64="mips64-linux-gnuabi64-gcc"}
: ${cross_cc_mipsel="mipsel-linux-gnu-gcc"}
: ${cross_cc_mips="mips-linux-gnu-gcc"}
+: ${cross_cc_nios2="nios2-linux-gnu-gcc"}
: ${cross_cc_ppc="powerpc-linux-gnu-gcc"}
: ${cross_cc_cflags_ppc="-m32"}
: ${cross_cc_ppc64="powerpc64-linux-gnu-gcc"}
@@ -159,6 +160,11 @@ for target in $target_list; do
container_image=debian-mips-cross
container_cross_cc=mips-linux-gnu-gcc
;;
+ nios2-*)
+ container_hosts=x86_64
+ container_image=debian-nios2-cross
+ container_cross_cc=nios2-linux-gnu-gcc
+ ;;
ppc-*|ppc64abi32-*)
container_hosts=x86_64
container_image=debian-powerpc-test-cross
diff --git a/tests/tcg/nios2/Makefile.target b/tests/tcg/nios2/Makefile.target
new file mode 100644
index 0000000000..b38e2352b7
--- /dev/null
+++ b/tests/tcg/nios2/Makefile.target
@@ -0,0 +1,11 @@
+# nios2 specific test tweaks
+
+# Currently nios2 signal handling is broken
+run-signals: signals
+ $(call skip-test, $<, "BROKEN")
+run-plugin-signals-with-%:
+ $(call skip-test, $<, "BROKEN")
+run-linux-test: linux-test
+ $(call skip-test, $<, "BROKEN")
+run-plugin-linux-test-with-%:
+ $(call skip-test, $<, "BROKEN")
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 09/21] ebpf: really include it only in system emulators
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (7 preceding siblings ...)
2021-11-03 17:05 ` [PULL 08/21] tests/tcg: enable debian-nios2-cross for test building Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 10/21] plugins/cache: freed heap-allocated mutexes Alex Bennée
` (12 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Alex Bennée, Paolo Bonzini, richard.henderson, qemu-devel
From: Paolo Bonzini <pbonzini@redhat.com>
eBPF libraries are being included in user emulators, which is useless and
also breaks --static compilation if a shared library for libbpf is
present in the system.
Reported-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211012162252.263933-1-pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-bt: Warner Losh <imp@bsdimp.com>
Message-Id: <20211026102234.3961636-13-alex.bennee@linaro.org>
diff --git a/meson.build b/meson.build
index 26c58123e9..fa2e6cceff 100644
--- a/meson.build
+++ b/meson.build
@@ -2610,8 +2610,6 @@ subdir('bsd-user')
subdir('linux-user')
subdir('ebpf')
-common_ss.add(libbpf)
-
specific_ss.add_all(when: 'CONFIG_BSD_USER', if_true: bsd_user_ss)
linux_user_ss.add(files('thunk.c'))
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 10/21] plugins/cache: freed heap-allocated mutexes
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (8 preceding siblings ...)
2021-11-03 17:05 ` [PULL 09/21] ebpf: really include it only in system emulators Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 11/21] plugins/cache: implement unified L2 cache emulation Alex Bennée
` (11 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: richard.henderson, qemu-devel, Philippe Mathieu-Daudé,
Alexandre Iooss, Mahmoud Mandour, Alex Bennée
From: Mahmoud Mandour <ma.mandourr@gmail.com>
Signed-off-by: Mahmoud Mandour <ma.mandourr@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210810134844.166490-2-ma.mandourr@gmail.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20211026102234.3961636-14-alex.bennee@linaro.org>
diff --git a/contrib/plugins/cache.c b/contrib/plugins/cache.c
index a1e03ca882..a255e26e25 100644
--- a/contrib/plugins/cache.c
+++ b/contrib/plugins/cache.c
@@ -614,6 +614,9 @@ static void plugin_exit(qemu_plugin_id_t id, void *p)
caches_free(dcaches);
caches_free(icaches);
+ g_free(dcache_locks);
+ g_free(icache_locks);
+
g_hash_table_destroy(miss_ht);
}
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 11/21] plugins/cache: implement unified L2 cache emulation
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (9 preceding siblings ...)
2021-11-03 17:05 ` [PULL 10/21] plugins/cache: freed heap-allocated mutexes Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 12/21] plugins/cache: split command line arguments into name and value Alex Bennée
` (10 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Alex Bennée, Mahmoud Mandour, richard.henderson, qemu-devel,
Alexandre Iooss
From: Mahmoud Mandour <ma.mandourr@gmail.com>
This adds an implementation of a simple L2 configuration, in which a
unified L2 cache (stores both blocks of instructions and data) is
maintained for each core separately, with no inter-core interaction
taken in account. The L2 cache is used as a backup for L1 and is only
accessed if the wanted block does not exist in L1.
In terms of multi-threaded user-space emulation, the same approximation
of L1 is done, a static number of caches is maintained, and each and
every memory access initiated by a thread will have to go through one of
the available caches.
An atomic increment is used to maintain the number of L2 misses per
instruction.
The default cache parameters of L2 caches is:
2MB cache size
16-way associativity
64-byte blocks
Signed-off-by: Mahmoud Mandour <ma.mandourr@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210810134844.166490-3-ma.mandourr@gmail.com>
Message-Id: <20211026102234.3961636-15-alex.bennee@linaro.org>
diff --git a/contrib/plugins/cache.c b/contrib/plugins/cache.c
index a255e26e25..908c967a09 100644
--- a/contrib/plugins/cache.c
+++ b/contrib/plugins/cache.c
@@ -82,8 +82,9 @@ typedef struct {
char *disas_str;
const char *symbol;
uint64_t addr;
- uint64_t dmisses;
- uint64_t imisses;
+ uint64_t l1_dmisses;
+ uint64_t l1_imisses;
+ uint64_t l2_misses;
} InsnData;
void (*update_hit)(Cache *cache, int set, int blk);
@@ -93,15 +94,20 @@ void (*metadata_init)(Cache *cache);
void (*metadata_destroy)(Cache *cache);
static int cores;
-static Cache **dcaches, **icaches;
+static Cache **l1_dcaches, **l1_icaches;
+static Cache **l2_ucaches;
-static GMutex *dcache_locks;
-static GMutex *icache_locks;
+static GMutex *l1_dcache_locks;
+static GMutex *l1_icache_locks;
+static GMutex *l2_ucache_locks;
-static uint64_t all_dmem_accesses;
-static uint64_t all_imem_accesses;
-static uint64_t all_imisses;
-static uint64_t all_dmisses;
+static uint64_t l1_dmem_accesses;
+static uint64_t l1_imem_accesses;
+static uint64_t l1_imisses;
+static uint64_t l1_dmisses;
+
+static uint64_t l2_mem_accesses;
+static uint64_t l2_misses;
static int pow_of_two(int num)
{
@@ -382,6 +388,7 @@ static void vcpu_mem_access(unsigned int vcpu_index, qemu_plugin_meminfo_t info,
struct qemu_plugin_hwaddr *hwaddr;
int cache_idx;
InsnData *insn;
+ bool hit_in_l1;
hwaddr = qemu_plugin_get_hwaddr(info, vaddr);
if (hwaddr && qemu_plugin_hwaddr_is_io(hwaddr)) {
@@ -391,14 +398,29 @@ static void vcpu_mem_access(unsigned int vcpu_index, qemu_plugin_meminfo_t info,
effective_addr = hwaddr ? qemu_plugin_hwaddr_phys_addr(hwaddr) : vaddr;
cache_idx = vcpu_index % cores;
- g_mutex_lock(&dcache_locks[cache_idx]);
- if (!access_cache(dcaches[cache_idx], effective_addr)) {
+ g_mutex_lock(&l1_dcache_locks[cache_idx]);
+ hit_in_l1 = access_cache(l1_dcaches[cache_idx], effective_addr);
+ if (!hit_in_l1) {
+ insn = (InsnData *) userdata;
+ __atomic_fetch_add(&insn->l1_dmisses, 1, __ATOMIC_SEQ_CST);
+ l1_dcaches[cache_idx]->misses++;
+ }
+ l1_dcaches[cache_idx]->accesses++;
+ g_mutex_unlock(&l1_dcache_locks[cache_idx]);
+
+ if (hit_in_l1) {
+ /* No need to access L2 */
+ return;
+ }
+
+ g_mutex_lock(&l2_ucache_locks[cache_idx]);
+ if (!access_cache(l2_ucaches[cache_idx], effective_addr)) {
insn = (InsnData *) userdata;
- __atomic_fetch_add(&insn->dmisses, 1, __ATOMIC_SEQ_CST);
- dcaches[cache_idx]->misses++;
+ __atomic_fetch_add(&insn->l2_misses, 1, __ATOMIC_SEQ_CST);
+ l2_ucaches[cache_idx]->misses++;
}
- dcaches[cache_idx]->accesses++;
- g_mutex_unlock(&dcache_locks[cache_idx]);
+ l2_ucaches[cache_idx]->accesses++;
+ g_mutex_unlock(&l2_ucache_locks[cache_idx]);
}
static void vcpu_insn_exec(unsigned int vcpu_index, void *userdata)
@@ -406,18 +428,34 @@ static void vcpu_insn_exec(unsigned int vcpu_index, void *userdata)
uint64_t insn_addr;
InsnData *insn;
int cache_idx;
+ bool hit_in_l1;
insn_addr = ((InsnData *) userdata)->addr;
cache_idx = vcpu_index % cores;
- g_mutex_lock(&icache_locks[cache_idx]);
- if (!access_cache(icaches[cache_idx], insn_addr)) {
+ g_mutex_lock(&l1_icache_locks[cache_idx]);
+ hit_in_l1 = access_cache(l1_icaches[cache_idx], insn_addr);
+ if (!hit_in_l1) {
+ insn = (InsnData *) userdata;
+ __atomic_fetch_add(&insn->l1_imisses, 1, __ATOMIC_SEQ_CST);
+ l1_icaches[cache_idx]->misses++;
+ }
+ l1_icaches[cache_idx]->accesses++;
+ g_mutex_unlock(&l1_icache_locks[cache_idx]);
+
+ if (hit_in_l1) {
+ /* No need to access L2 */
+ return;
+ }
+
+ g_mutex_lock(&l2_ucache_locks[cache_idx]);
+ if (!access_cache(l2_ucaches[cache_idx], insn_addr)) {
insn = (InsnData *) userdata;
- __atomic_fetch_add(&insn->imisses, 1, __ATOMIC_SEQ_CST);
- icaches[cache_idx]->misses++;
+ __atomic_fetch_add(&insn->l2_misses, 1, __ATOMIC_SEQ_CST);
+ l2_ucaches[cache_idx]->misses++;
}
- icaches[cache_idx]->accesses++;
- g_mutex_unlock(&icache_locks[cache_idx]);
+ l2_ucaches[cache_idx]->accesses++;
+ g_mutex_unlock(&l2_ucache_locks[cache_idx]);
}
static void vcpu_tb_trans(qemu_plugin_id_t id, struct qemu_plugin_tb *tb)
@@ -493,30 +531,28 @@ static void caches_free(Cache **caches)
}
}
-static int dcmp(gconstpointer a, gconstpointer b)
-{
- InsnData *insn_a = (InsnData *) a;
- InsnData *insn_b = (InsnData *) b;
-
- return insn_a->dmisses < insn_b->dmisses ? 1 : -1;
-}
-
-static void append_stats_line(GString *line, uint64_t daccess, uint64_t dmisses,
- uint64_t iaccess, uint64_t imisses)
+static void append_stats_line(GString *line, uint64_t l1_daccess,
+ uint64_t l1_dmisses, uint64_t l1_iaccess,
+ uint64_t l1_imisses, uint64_t l2_access,
+ uint64_t l2_misses)
{
- double dmiss_rate, imiss_rate;
+ double l1_dmiss_rate, l1_imiss_rate, l2_miss_rate;
- dmiss_rate = ((double) dmisses) / (daccess) * 100.0;
- imiss_rate = ((double) imisses) / (iaccess) * 100.0;
+ l1_dmiss_rate = ((double) l1_dmisses) / (l1_daccess) * 100.0;
+ l1_imiss_rate = ((double) l1_imisses) / (l1_iaccess) * 100.0;
+ l2_miss_rate = ((double) l2_misses) / (l2_access) * 100.0;
g_string_append_printf(line, "%-14lu %-12lu %9.4lf%% %-14lu %-12lu"
- " %9.4lf%%\n",
- daccess,
- dmisses,
- daccess ? dmiss_rate : 0.0,
- iaccess,
- imisses,
- iaccess ? imiss_rate : 0.0);
+ " %9.4lf%% %-12lu %-11lu %10.4lf%%\n",
+ l1_daccess,
+ l1_dmisses,
+ l1_daccess ? l1_dmiss_rate : 0.0,
+ l1_iaccess,
+ l1_imisses,
+ l1_iaccess ? l1_imiss_rate : 0.0,
+ l2_access,
+ l2_misses,
+ l2_access ? l2_miss_rate : 0.0);
}
static void sum_stats(void)
@@ -525,43 +561,66 @@ static void sum_stats(void)
g_assert(cores > 1);
for (i = 0; i < cores; i++) {
- all_imisses += icaches[i]->misses;
- all_dmisses += dcaches[i]->misses;
- all_imem_accesses += icaches[i]->accesses;
- all_dmem_accesses += dcaches[i]->accesses;
+ l1_imisses += l1_icaches[i]->misses;
+ l1_dmisses += l1_dcaches[i]->misses;
+ l1_imem_accesses += l1_icaches[i]->accesses;
+ l1_dmem_accesses += l1_dcaches[i]->accesses;
+
+ l2_misses += l2_ucaches[i]->misses;
+ l2_mem_accesses += l2_ucaches[i]->accesses;
}
}
+static int dcmp(gconstpointer a, gconstpointer b)
+{
+ InsnData *insn_a = (InsnData *) a;
+ InsnData *insn_b = (InsnData *) b;
+
+ return insn_a->l1_dmisses < insn_b->l1_dmisses ? 1 : -1;
+}
+
static int icmp(gconstpointer a, gconstpointer b)
{
InsnData *insn_a = (InsnData *) a;
InsnData *insn_b = (InsnData *) b;
- return insn_a->imisses < insn_b->imisses ? 1 : -1;
+ return insn_a->l1_imisses < insn_b->l1_imisses ? 1 : -1;
+}
+
+static int l2_cmp(gconstpointer a, gconstpointer b)
+{
+ InsnData *insn_a = (InsnData *) a;
+ InsnData *insn_b = (InsnData *) b;
+
+ return insn_a->l2_misses < insn_b->l2_misses ? 1 : -1;
}
static void log_stats(void)
{
int i;
- Cache *icache, *dcache;
+ Cache *icache, *dcache, *l2_cache;
g_autoptr(GString) rep = g_string_new("core #, data accesses, data misses,"
" dmiss rate, insn accesses,"
- " insn misses, imiss rate\n");
+ " insn misses, imiss rate,"
+ " l2 accesses, l2 misses,"
+ " l2 miss rate\n");
for (i = 0; i < cores; i++) {
g_string_append_printf(rep, "%-8d", i);
- dcache = dcaches[i];
- icache = icaches[i];
+ dcache = l1_dcaches[i];
+ icache = l1_icaches[i];
+ l2_cache = l2_ucaches[i];
append_stats_line(rep, dcache->accesses, dcache->misses,
- icache->accesses, icache->misses);
+ icache->accesses, icache->misses, l2_cache->accesses,
+ l2_cache->misses);
}
if (cores > 1) {
sum_stats();
g_string_append_printf(rep, "%-8s", "sum");
- append_stats_line(rep, all_dmem_accesses, all_dmisses,
- all_imem_accesses, all_imisses);
+ append_stats_line(rep, l1_dmem_accesses, l1_dmisses,
+ l1_imem_accesses, l1_imisses, l2_mem_accesses, l2_misses);
}
g_string_append(rep, "\n");
@@ -585,7 +644,7 @@ static void log_top_insns(void)
if (insn->symbol) {
g_string_append_printf(rep, " (%s)", insn->symbol);
}
- g_string_append_printf(rep, ", %ld, %s\n", insn->dmisses,
+ g_string_append_printf(rep, ", %ld, %s\n", insn->l1_dmisses,
insn->disas_str);
}
@@ -598,7 +657,20 @@ static void log_top_insns(void)
if (insn->symbol) {
g_string_append_printf(rep, " (%s)", insn->symbol);
}
- g_string_append_printf(rep, ", %ld, %s\n", insn->imisses,
+ g_string_append_printf(rep, ", %ld, %s\n", insn->l1_imisses,
+ insn->disas_str);
+ }
+
+ miss_insns = g_list_sort(miss_insns, l2_cmp);
+ g_string_append_printf(rep, "%s", "\naddress, L2 misses, instruction\n");
+
+ for (curr = miss_insns, i = 0; curr && i < limit; i++, curr = curr->next) {
+ insn = (InsnData *) curr->data;
+ g_string_append_printf(rep, "0x%" PRIx64, insn->addr);
+ if (insn->symbol) {
+ g_string_append_printf(rep, " (%s)", insn->symbol);
+ }
+ g_string_append_printf(rep, ", %ld, %s\n", insn->l2_misses,
insn->disas_str);
}
@@ -611,11 +683,13 @@ static void plugin_exit(qemu_plugin_id_t id, void *p)
log_stats();
log_top_insns();
- caches_free(dcaches);
- caches_free(icaches);
+ caches_free(l1_dcaches);
+ caches_free(l1_icaches);
+ caches_free(l2_ucaches);
- g_free(dcache_locks);
- g_free(icache_locks);
+ g_free(l1_dcache_locks);
+ g_free(l1_icache_locks);
+ g_free(l2_ucache_locks);
g_hash_table_destroy(miss_ht);
}
@@ -647,19 +721,24 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
int argc, char **argv)
{
int i;
- int iassoc, iblksize, icachesize;
- int dassoc, dblksize, dcachesize;
+ int l1_iassoc, l1_iblksize, l1_icachesize;
+ int l1_dassoc, l1_dblksize, l1_dcachesize;
+ int l2_assoc, l2_blksize, l2_cachesize;
limit = 32;
sys = info->system_emulation;
- dassoc = 8;
- dblksize = 64;
- dcachesize = dblksize * dassoc * 32;
+ l1_dassoc = 8;
+ l1_dblksize = 64;
+ l1_dcachesize = l1_dblksize * l1_dassoc * 32;
+
+ l1_iassoc = 8;
+ l1_iblksize = 64;
+ l1_icachesize = l1_iblksize * l1_iassoc * 32;
- iassoc = 8;
- iblksize = 64;
- icachesize = iblksize * iassoc * 32;
+ l2_assoc = 16;
+ l2_blksize = 64;
+ l2_cachesize = l2_assoc * l2_blksize * 2048;
policy = LRU;
@@ -668,21 +747,27 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
for (i = 0; i < argc; i++) {
char *opt = argv[i];
if (g_str_has_prefix(opt, "iblksize=")) {
- iblksize = g_ascii_strtoll(opt + 9, NULL, 10);
+ l1_iblksize = g_ascii_strtoll(opt + 9, NULL, 10);
} else if (g_str_has_prefix(opt, "iassoc=")) {
- iassoc = g_ascii_strtoll(opt + 7, NULL, 10);
+ l1_iassoc = g_ascii_strtoll(opt + 7, NULL, 10);
} else if (g_str_has_prefix(opt, "icachesize=")) {
- icachesize = g_ascii_strtoll(opt + 11, NULL, 10);
+ l1_icachesize = g_ascii_strtoll(opt + 11, NULL, 10);
} else if (g_str_has_prefix(opt, "dblksize=")) {
- dblksize = g_ascii_strtoll(opt + 9, NULL, 10);
+ l1_dblksize = g_ascii_strtoll(opt + 9, NULL, 10);
} else if (g_str_has_prefix(opt, "dassoc=")) {
- dassoc = g_ascii_strtoll(opt + 7, NULL, 10);
+ l1_dassoc = g_ascii_strtoll(opt + 7, NULL, 10);
} else if (g_str_has_prefix(opt, "dcachesize=")) {
- dcachesize = g_ascii_strtoll(opt + 11, NULL, 10);
+ l1_dcachesize = g_ascii_strtoll(opt + 11, NULL, 10);
} else if (g_str_has_prefix(opt, "limit=")) {
limit = g_ascii_strtoll(opt + 6, NULL, 10);
} else if (g_str_has_prefix(opt, "cores=")) {
cores = g_ascii_strtoll(opt + 6, NULL, 10);
+ } else if (g_str_has_prefix(opt, "l2cachesize=")) {
+ l2_cachesize = g_ascii_strtoll(opt + 6, NULL, 10);
+ } else if (g_str_has_prefix(opt, "l2blksize=")) {
+ l2_blksize = g_ascii_strtoll(opt + 6, NULL, 10);
+ } else if (g_str_has_prefix(opt, "l2assoc=")) {
+ l2_assoc = g_ascii_strtoll(opt + 6, NULL, 10);
} else if (g_str_has_prefix(opt, "evict=")) {
gchar *p = opt + 6;
if (g_strcmp0(p, "rand") == 0) {
@@ -703,24 +788,33 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
policy_init();
- dcaches = caches_init(dblksize, dassoc, dcachesize);
- if (!dcaches) {
- const char *err = cache_config_error(dblksize, dassoc, dcachesize);
+ l1_dcaches = caches_init(l1_dblksize, l1_dassoc, l1_dcachesize);
+ if (!l1_dcaches) {
+ const char *err = cache_config_error(l1_dblksize, l1_dassoc, l1_dcachesize);
fprintf(stderr, "dcache cannot be constructed from given parameters\n");
fprintf(stderr, "%s\n", err);
return -1;
}
- icaches = caches_init(iblksize, iassoc, icachesize);
- if (!icaches) {
- const char *err = cache_config_error(iblksize, iassoc, icachesize);
+ l1_icaches = caches_init(l1_iblksize, l1_iassoc, l1_icachesize);
+ if (!l1_icaches) {
+ const char *err = cache_config_error(l1_iblksize, l1_iassoc, l1_icachesize);
fprintf(stderr, "icache cannot be constructed from given parameters\n");
fprintf(stderr, "%s\n", err);
return -1;
}
- dcache_locks = g_new0(GMutex, cores);
- icache_locks = g_new0(GMutex, cores);
+ l2_ucaches = caches_init(l2_blksize, l2_assoc, l2_cachesize);
+ if (!l2_ucaches) {
+ const char *err = cache_config_error(l2_blksize, l2_assoc, l2_cachesize);
+ fprintf(stderr, "L2 cache cannot be constructed from given parameters\n");
+ fprintf(stderr, "%s\n", err);
+ return -1;
+ }
+
+ l1_dcache_locks = g_new0(GMutex, cores);
+ l1_icache_locks = g_new0(GMutex, cores);
+ l2_ucache_locks = g_new0(GMutex, cores);
qemu_plugin_register_vcpu_tb_trans_cb(id, vcpu_tb_trans);
qemu_plugin_register_atexit_cb(id, plugin_exit, NULL);
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 12/21] plugins/cache: split command line arguments into name and value
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (10 preceding siblings ...)
2021-11-03 17:05 ` [PULL 11/21] plugins/cache: implement unified L2 cache emulation Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 13/21] plugins/cache: make L2 emulation optional through args Alex Bennée
` (9 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Alex Bennée, Mahmoud Mandour, richard.henderson, qemu-devel,
Alexandre Iooss
From: Mahmoud Mandour <ma.mandourr@gmail.com>
This way of handling args is more lenient and sets a better framework to
parse boolean command line arguments.
Signed-off-by: Mahmoud Mandour <ma.mandourr@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210810134844.166490-4-ma.mandourr@gmail.com>
Message-Id: <20211026102234.3961636-16-alex.bennee@linaro.org>
diff --git a/contrib/plugins/cache.c b/contrib/plugins/cache.c
index 908c967a09..ff325beb9f 100644
--- a/contrib/plugins/cache.c
+++ b/contrib/plugins/cache.c
@@ -11,6 +11,8 @@
#include <qemu-plugin.h>
+#define STRTOLL(x) g_ascii_strtoll(x, NULL, 10)
+
QEMU_PLUGIN_EXPORT int qemu_plugin_version = QEMU_PLUGIN_VERSION;
static enum qemu_plugin_mem_rw rw = QEMU_PLUGIN_MEM_RW;
@@ -746,35 +748,36 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
for (i = 0; i < argc; i++) {
char *opt = argv[i];
- if (g_str_has_prefix(opt, "iblksize=")) {
- l1_iblksize = g_ascii_strtoll(opt + 9, NULL, 10);
- } else if (g_str_has_prefix(opt, "iassoc=")) {
- l1_iassoc = g_ascii_strtoll(opt + 7, NULL, 10);
- } else if (g_str_has_prefix(opt, "icachesize=")) {
- l1_icachesize = g_ascii_strtoll(opt + 11, NULL, 10);
- } else if (g_str_has_prefix(opt, "dblksize=")) {
- l1_dblksize = g_ascii_strtoll(opt + 9, NULL, 10);
- } else if (g_str_has_prefix(opt, "dassoc=")) {
- l1_dassoc = g_ascii_strtoll(opt + 7, NULL, 10);
- } else if (g_str_has_prefix(opt, "dcachesize=")) {
- l1_dcachesize = g_ascii_strtoll(opt + 11, NULL, 10);
- } else if (g_str_has_prefix(opt, "limit=")) {
- limit = g_ascii_strtoll(opt + 6, NULL, 10);
- } else if (g_str_has_prefix(opt, "cores=")) {
- cores = g_ascii_strtoll(opt + 6, NULL, 10);
- } else if (g_str_has_prefix(opt, "l2cachesize=")) {
- l2_cachesize = g_ascii_strtoll(opt + 6, NULL, 10);
- } else if (g_str_has_prefix(opt, "l2blksize=")) {
- l2_blksize = g_ascii_strtoll(opt + 6, NULL, 10);
- } else if (g_str_has_prefix(opt, "l2assoc=")) {
- l2_assoc = g_ascii_strtoll(opt + 6, NULL, 10);
- } else if (g_str_has_prefix(opt, "evict=")) {
- gchar *p = opt + 6;
- if (g_strcmp0(p, "rand") == 0) {
+ g_autofree char **tokens = g_strsplit(opt, "=", 2);
+
+ if (g_strcmp0(tokens[0], "iblksize") == 0) {
+ l1_iblksize = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "iassoc") == 0) {
+ l1_iassoc = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "icachesize") == 0) {
+ l1_icachesize = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "dblksize") == 0) {
+ l1_dblksize = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "dassoc") == 0) {
+ l1_dassoc = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "dcachesize") == 0) {
+ l1_dcachesize = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "limit") == 0) {
+ limit = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "cores") == 0) {
+ cores = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "l2cachesize") == 0) {
+ l2_cachesize = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "l2blksize") == 0) {
+ l2_blksize = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "l2assoc") == 0) {
+ l2_assoc = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "evict") == 0) {
+ if (g_strcmp0(tokens[1], "rand") == 0) {
policy = RAND;
- } else if (g_strcmp0(p, "lru") == 0) {
+ } else if (g_strcmp0(tokens[1], "lru") == 0) {
policy = LRU;
- } else if (g_strcmp0(p, "fifo") == 0) {
+ } else if (g_strcmp0(tokens[1], "fifo") == 0) {
policy = FIFO;
} else {
fprintf(stderr, "invalid eviction policy: %s\n", opt);
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 13/21] plugins/cache: make L2 emulation optional through args
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (11 preceding siblings ...)
2021-11-03 17:05 ` [PULL 12/21] plugins/cache: split command line arguments into name and value Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 14/21] docs/tcg-plugins: add L2 arguments to cache docs Alex Bennée
` (8 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Alex Bennée, Mahmoud Mandour, richard.henderson, qemu-devel,
Alexandre Iooss
From: Mahmoud Mandour <ma.mandourr@gmail.com>
By default L2 is not enabled and is enabled by either using the
newly-introduced "l2" boolean argument, or by setting any of the L2
cache parameters using args. On specifying "l2=on", the default cache
configuration is used.
Signed-off-by: Mahmoud Mandour <ma.mandourr@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210810134844.166490-5-ma.mandourr@gmail.com>
Message-Id: <20211026102234.3961636-17-alex.bennee@linaro.org>
diff --git a/contrib/plugins/cache.c b/contrib/plugins/cache.c
index ff325beb9f..b9226e7c40 100644
--- a/contrib/plugins/cache.c
+++ b/contrib/plugins/cache.c
@@ -97,6 +97,8 @@ void (*metadata_destroy)(Cache *cache);
static int cores;
static Cache **l1_dcaches, **l1_icaches;
+
+static bool use_l2;
static Cache **l2_ucaches;
static GMutex *l1_dcache_locks;
@@ -410,7 +412,7 @@ static void vcpu_mem_access(unsigned int vcpu_index, qemu_plugin_meminfo_t info,
l1_dcaches[cache_idx]->accesses++;
g_mutex_unlock(&l1_dcache_locks[cache_idx]);
- if (hit_in_l1) {
+ if (hit_in_l1 || !use_l2) {
/* No need to access L2 */
return;
}
@@ -445,7 +447,7 @@ static void vcpu_insn_exec(unsigned int vcpu_index, void *userdata)
l1_icaches[cache_idx]->accesses++;
g_mutex_unlock(&l1_icache_locks[cache_idx]);
- if (hit_in_l1) {
+ if (hit_in_l1 || !use_l2) {
/* No need to access L2 */
return;
}
@@ -542,19 +544,25 @@ static void append_stats_line(GString *line, uint64_t l1_daccess,
l1_dmiss_rate = ((double) l1_dmisses) / (l1_daccess) * 100.0;
l1_imiss_rate = ((double) l1_imisses) / (l1_iaccess) * 100.0;
- l2_miss_rate = ((double) l2_misses) / (l2_access) * 100.0;
g_string_append_printf(line, "%-14lu %-12lu %9.4lf%% %-14lu %-12lu"
- " %9.4lf%% %-12lu %-11lu %10.4lf%%\n",
+ " %9.4lf%%",
l1_daccess,
l1_dmisses,
l1_daccess ? l1_dmiss_rate : 0.0,
l1_iaccess,
l1_imisses,
- l1_iaccess ? l1_imiss_rate : 0.0,
- l2_access,
- l2_misses,
- l2_access ? l2_miss_rate : 0.0);
+ l1_iaccess ? l1_imiss_rate : 0.0);
+
+ if (use_l2) {
+ l2_miss_rate = ((double) l2_misses) / (l2_access) * 100.0;
+ g_string_append_printf(line, " %-12lu %-11lu %10.4lf%%",
+ l2_access,
+ l2_misses,
+ l2_access ? l2_miss_rate : 0.0);
+ }
+
+ g_string_append(line, "\n");
}
static void sum_stats(void)
@@ -568,8 +576,10 @@ static void sum_stats(void)
l1_imem_accesses += l1_icaches[i]->accesses;
l1_dmem_accesses += l1_dcaches[i]->accesses;
- l2_misses += l2_ucaches[i]->misses;
- l2_mem_accesses += l2_ucaches[i]->accesses;
+ if (use_l2) {
+ l2_misses += l2_ucaches[i]->misses;
+ l2_mem_accesses += l2_ucaches[i]->accesses;
+ }
}
}
@@ -604,25 +614,31 @@ static void log_stats(void)
g_autoptr(GString) rep = g_string_new("core #, data accesses, data misses,"
" dmiss rate, insn accesses,"
- " insn misses, imiss rate,"
- " l2 accesses, l2 misses,"
- " l2 miss rate\n");
+ " insn misses, imiss rate");
+
+ if (use_l2) {
+ g_string_append(rep, ", l2 accesses, l2 misses, l2 miss rate");
+ }
+
+ g_string_append(rep, "\n");
for (i = 0; i < cores; i++) {
g_string_append_printf(rep, "%-8d", i);
dcache = l1_dcaches[i];
icache = l1_icaches[i];
- l2_cache = l2_ucaches[i];
+ l2_cache = use_l2 ? l2_ucaches[i] : NULL;
append_stats_line(rep, dcache->accesses, dcache->misses,
- icache->accesses, icache->misses, l2_cache->accesses,
- l2_cache->misses);
+ icache->accesses, icache->misses,
+ l2_cache ? l2_cache->accesses : 0,
+ l2_cache ? l2_cache->misses : 0);
}
if (cores > 1) {
sum_stats();
g_string_append_printf(rep, "%-8s", "sum");
append_stats_line(rep, l1_dmem_accesses, l1_dmisses,
- l1_imem_accesses, l1_imisses, l2_mem_accesses, l2_misses);
+ l1_imem_accesses, l1_imisses,
+ l2_cache ? l2_mem_accesses : 0, l2_cache ? l2_misses : 0);
}
g_string_append(rep, "\n");
@@ -663,6 +679,10 @@ static void log_top_insns(void)
insn->disas_str);
}
+ if (!use_l2) {
+ goto finish;
+ }
+
miss_insns = g_list_sort(miss_insns, l2_cmp);
g_string_append_printf(rep, "%s", "\naddress, L2 misses, instruction\n");
@@ -676,6 +696,7 @@ static void log_top_insns(void)
insn->disas_str);
}
+finish:
qemu_plugin_outs(rep->str);
g_list_free(miss_insns);
}
@@ -687,11 +708,14 @@ static void plugin_exit(qemu_plugin_id_t id, void *p)
caches_free(l1_dcaches);
caches_free(l1_icaches);
- caches_free(l2_ucaches);
g_free(l1_dcache_locks);
g_free(l1_icache_locks);
- g_free(l2_ucache_locks);
+
+ if (use_l2) {
+ caches_free(l2_ucaches);
+ g_free(l2_ucache_locks);
+ }
g_hash_table_destroy(miss_ht);
}
@@ -767,11 +791,19 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
} else if (g_strcmp0(tokens[0], "cores") == 0) {
cores = STRTOLL(tokens[1]);
} else if (g_strcmp0(tokens[0], "l2cachesize") == 0) {
+ use_l2 = true;
l2_cachesize = STRTOLL(tokens[1]);
} else if (g_strcmp0(tokens[0], "l2blksize") == 0) {
+ use_l2 = true;
l2_blksize = STRTOLL(tokens[1]);
} else if (g_strcmp0(tokens[0], "l2assoc") == 0) {
+ use_l2 = true;
l2_assoc = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "l2") == 0) {
+ if (!qemu_plugin_bool_parse(tokens[0], tokens[1], &use_l2)) {
+ fprintf(stderr, "boolean argument parsing failed: %s\n", opt);
+ return -1;
+ }
} else if (g_strcmp0(tokens[0], "evict") == 0) {
if (g_strcmp0(tokens[1], "rand") == 0) {
policy = RAND;
@@ -807,8 +839,8 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
return -1;
}
- l2_ucaches = caches_init(l2_blksize, l2_assoc, l2_cachesize);
- if (!l2_ucaches) {
+ l2_ucaches = use_l2 ? caches_init(l2_blksize, l2_assoc, l2_cachesize) : NULL;
+ if (!l2_ucaches && use_l2) {
const char *err = cache_config_error(l2_blksize, l2_assoc, l2_cachesize);
fprintf(stderr, "L2 cache cannot be constructed from given parameters\n");
fprintf(stderr, "%s\n", err);
@@ -817,7 +849,7 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
l1_dcache_locks = g_new0(GMutex, cores);
l1_icache_locks = g_new0(GMutex, cores);
- l2_ucache_locks = g_new0(GMutex, cores);
+ l2_ucache_locks = use_l2 ? g_new0(GMutex, cores) : NULL;
qemu_plugin_register_vcpu_tb_trans_cb(id, vcpu_tb_trans);
qemu_plugin_register_atexit_cb(id, plugin_exit, NULL);
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 14/21] docs/tcg-plugins: add L2 arguments to cache docs
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (12 preceding siblings ...)
2021-11-03 17:05 ` [PULL 13/21] plugins/cache: make L2 emulation optional through args Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 15/21] chardev: don't exit() straight away on C-a x Alex Bennée
` (7 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Alex Bennée, Mahmoud Mandour, richard.henderson, qemu-devel,
Alexandre Iooss
From: Mahmoud Mandour <ma.mandourr@gmail.com>
cache plugin now allows optional L2 per-core cache emulation that can be
configured through plugin arguments, this commit adds this functionality
to the docs.
While I'm at it, I editted the bullet point for cache plugin to say:
contrib/plugins/cache.c
instead of
contrib/plugins/cache
to match other plugins.
Signed-off-by: Mahmoud Mandour <ma.mandourr@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210810134844.166490-6-ma.mandourr@gmail.com>
Message-Id: <20211026102234.3961636-18-alex.bennee@linaro.org>
diff --git a/docs/devel/tcg-plugins.rst b/docs/devel/tcg-plugins.rst
index 842ae01a4c..59a7d838be 100644
--- a/docs/devel/tcg-plugins.rst
+++ b/docs/devel/tcg-plugins.rst
@@ -361,8 +361,9 @@ which will output an execution trace following this structure::
- contrib/plugins/cache.c
-Cache modelling plugin that measures the performance of a given cache
-configuration when a given working set is run::
+Cache modelling plugin that measures the performance of a given L1 cache
+configuration, and optionally a unified L2 per-core cache when a given working
+set is run::
qemu-x86_64 -plugin ./contrib/plugins/libcache.so \
-d plugin -D cache.log ./tests/tcg/x86_64-linux-user/float_convs
@@ -420,3 +421,18 @@ The plugin has a number of arguments, all of them are optional:
Sets the number of cores for which we maintain separate icache and dcache.
(default: for linux-user, N = 1, for full system emulation: N = cores
available to guest)
+
+ * l2=on
+
+ Simulates a unified L2 cache (stores blocks for both instructions and data)
+ using the default L2 configuration (cache size = 2MB, associativity = 16-way,
+ block size = 64B).
+
+ * l2cachesize=N
+ * l2blksize=B
+ * l2assoc=A
+
+ L2 cache configuration arguments. They specify the cache size, block size, and
+ associativity of the L2 cache, respectively. Setting any of the L2
+ configuration arguments implies ``l2=on``.
+ (default: N = 2097152 (2MB), B = 64, A = 16)
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 15/21] chardev: don't exit() straight away on C-a x
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (13 preceding siblings ...)
2021-11-03 17:05 ` [PULL 14/21] docs/tcg-plugins: add L2 arguments to cache docs Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 16/21] tests/plugins: extend the insn plugin to track opcode sizes Alex Bennée
` (6 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Philippe Mathieu-Daudé,
richard.henderson, qemu-devel, Lukas Jünger, Paolo Bonzini,
Marc-André Lureau, Alex Bennée
While there are a number of uses in the code-base of the exit(0)
pattern it gets in the way of clean exit which can do all of it's
house-keeping. In particular it was reported that you can crash
plugins this way because TCG can still be running on other threads
when the atexit callback is called.
Use qmp_quit() instead which takes care of some housekeeping before
triggering the shutdown.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reported-by: Lukas Jünger <lukas.junger@greensocs.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20211026102234.3961636-19-alex.bennee@linaro.org>
diff --git a/chardev/char-mux.c b/chardev/char-mux.c
index ada0c6866f..ee2d47b20d 100644
--- a/chardev/char-mux.c
+++ b/chardev/char-mux.c
@@ -28,6 +28,7 @@
#include "qemu/option.h"
#include "chardev/char.h"
#include "sysemu/block-backend.h"
+#include "qapi/qapi-commands-control.h"
#include "chardev-internal.h"
/* MUX driver for serial I/O splitting */
@@ -157,7 +158,7 @@ static int mux_proc_byte(Chardev *chr, MuxChardev *d, int ch)
{
const char *term = "QEMU: Terminated\n\r";
qemu_chr_write_all(chr, (uint8_t *)term, strlen(term));
- exit(0);
+ qmp_quit(NULL);
break;
}
case 's':
diff --git a/stubs/qmp-quit.c b/stubs/qmp-quit.c
new file mode 100644
index 0000000000..a3ff47f7bd
--- /dev/null
+++ b/stubs/qmp-quit.c
@@ -0,0 +1,8 @@
+#include "qemu/osdep.h"
+#include "qapi/qapi-commands-control.h"
+#include "qapi/qmp/dispatch.h"
+
+void qmp_quit(Error **errp)
+{
+ g_assert_not_reached();
+}
diff --git a/stubs/meson.build b/stubs/meson.build
index f6aa3aa94f..71469c1d50 100644
--- a/stubs/meson.build
+++ b/stubs/meson.build
@@ -31,6 +31,7 @@ stub_ss.add(files('pci-bus.c'))
stub_ss.add(files('qemu-timer-notify-cb.c'))
stub_ss.add(files('qmp_memory_device.c'))
stub_ss.add(files('qmp-command-available.c'))
+stub_ss.add(files('qmp-quit.c'))
stub_ss.add(files('qtest.c'))
stub_ss.add(files('ram-block.c'))
stub_ss.add(files('ramfb.c'))
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 16/21] tests/plugins: extend the insn plugin to track opcode sizes
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (14 preceding siblings ...)
2021-11-03 17:05 ` [PULL 15/21] chardev: don't exit() straight away on C-a x Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 17/21] plugins: try and make plugin_insn_append more ergonomic Alex Bennée
` (5 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Alex Bennée, Mahmoud Mandour, richard.henderson, qemu-devel,
Alexandre Iooss
This is mostly a convenience feature for identifying frontends that do
multiple repeat loads so I can test changes to the instruction
tracking interface.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211026102234.3961636-20-alex.bennee@linaro.org>
diff --git a/tests/plugin/insn.c b/tests/plugin/insn.c
index 0f6a1938c1..d229fdc001 100644
--- a/tests/plugin/insn.c
+++ b/tests/plugin/insn.c
@@ -18,6 +18,8 @@ QEMU_PLUGIN_EXPORT int qemu_plugin_version = QEMU_PLUGIN_VERSION;
static uint64_t insn_count;
static bool do_inline;
+static bool do_size;
+static GArray *sizes;
static void vcpu_insn_exec_before(unsigned int cpu_index, void *udata)
{
@@ -49,13 +51,35 @@ static void vcpu_tb_trans(qemu_plugin_id_t id, struct qemu_plugin_tb *tb)
insn, vcpu_insn_exec_before, QEMU_PLUGIN_CB_NO_REGS,
GUINT_TO_POINTER(vaddr));
}
+
+ if (do_size) {
+ size_t sz = qemu_plugin_insn_size(insn);
+ if (sz > sizes->len) {
+ g_array_set_size(sizes, sz);
+ }
+ unsigned long *cnt = &g_array_index(sizes, unsigned long, sz);
+ (*cnt)++;
+ }
}
}
static void plugin_exit(qemu_plugin_id_t id, void *p)
{
- g_autofree gchar *out = g_strdup_printf("insns: %" PRIu64 "\n", insn_count);
- qemu_plugin_outs(out);
+ g_autoptr(GString) out = g_string_new(NULL);
+
+ if (do_size) {
+ int i;
+ for (i = 0; i <= sizes->len; i++) {
+ unsigned long *cnt = &g_array_index(sizes, unsigned long, i);
+ if (*cnt) {
+ g_string_append_printf(out,
+ "len %d bytes: %ld insns\n", i, *cnt);
+ }
+ }
+ } else {
+ g_string_append_printf(out, "insns: %" PRIu64 "\n", insn_count);
+ }
+ qemu_plugin_outs(out->str);
}
QEMU_PLUGIN_EXPORT int qemu_plugin_install(qemu_plugin_id_t id,
@@ -70,12 +94,21 @@ QEMU_PLUGIN_EXPORT int qemu_plugin_install(qemu_plugin_id_t id,
fprintf(stderr, "boolean argument parsing failed: %s\n", opt);
return -1;
}
+ } else if (g_strcmp0(tokens[0], "sizes") == 0) {
+ if (!qemu_plugin_bool_parse(tokens[0], tokens[1], &do_size)) {
+ fprintf(stderr, "boolean argument parsing failed: %s\n", opt);
+ return -1;
+ }
} else {
fprintf(stderr, "option parsing failed: %s\n", opt);
return -1;
}
}
+ if (do_size) {
+ sizes = g_array_new(true, true, sizeof(unsigned long));
+ }
+
qemu_plugin_register_vcpu_tb_trans_cb(id, vcpu_tb_trans);
qemu_plugin_register_atexit_cb(id, plugin_exit, NULL);
return 0;
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 17/21] plugins: try and make plugin_insn_append more ergonomic
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (15 preceding siblings ...)
2021-11-03 17:05 ` [PULL 16/21] tests/plugins: extend the insn plugin to track opcode sizes Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 18/21] tests/tcg: remove duplicate EXTRA_RUNS Alex Bennée
` (4 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Alex Bennée, richard.henderson, qemu-devel, Paolo Bonzini
Currently we make the assumption that the guest frontend loads all
op code bytes sequentially. This mostly holds up for regular fixed
encodings but some architectures like s390x like to re-read the
instruction which causes weirdness to occur. Rather than changing the
frontends make the plugin API a little more ergonomic and able to
handle the re-read case.
Stuff will still get strange if we read ahead of the opcode but so far
no front ends have done that and this patch asserts the case so we can
catch it early if they do.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20211026102234.3961636-21-alex.bennee@linaro.org>
diff --git a/include/exec/plugin-gen.h b/include/exec/plugin-gen.h
index b1b72b5d90..f92f169739 100644
--- a/include/exec/plugin-gen.h
+++ b/include/exec/plugin-gen.h
@@ -27,13 +27,21 @@ void plugin_gen_insn_end(void);
void plugin_gen_disable_mem_helpers(void);
void plugin_gen_empty_mem_callback(TCGv addr, uint32_t info);
-static inline void plugin_insn_append(const void *from, size_t size)
+static inline void plugin_insn_append(abi_ptr pc, const void *from, size_t size)
{
struct qemu_plugin_insn *insn = tcg_ctx->plugin_insn;
+ abi_ptr off;
if (insn == NULL) {
return;
}
+ off = pc - insn->vaddr;
+ if (off < insn->data->len) {
+ g_byte_array_set_size(insn->data, off);
+ } else if (off > insn->data->len) {
+ /* we have an unexpected gap */
+ g_assert_not_reached();
+ }
insn->data = g_byte_array_append(insn->data, from, size);
}
@@ -62,7 +70,7 @@ static inline void plugin_gen_disable_mem_helpers(void)
static inline void plugin_gen_empty_mem_callback(TCGv addr, uint32_t info)
{ }
-static inline void plugin_insn_append(const void *from, size_t size)
+static inline void plugin_insn_append(abi_ptr pc, const void *from, size_t size)
{ }
#endif /* CONFIG_PLUGIN */
diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
index b3172b147f..145f8a221a 100644
--- a/include/qemu/plugin.h
+++ b/include/qemu/plugin.h
@@ -163,10 +163,12 @@ struct qemu_plugin_tb {
/**
* qemu_plugin_tb_insn_get(): get next plugin record for translation.
- *
+ * @tb: the internal tb context
+ * @pc: address of instruction
*/
static inline
-struct qemu_plugin_insn *qemu_plugin_tb_insn_get(struct qemu_plugin_tb *tb)
+struct qemu_plugin_insn *qemu_plugin_tb_insn_get(struct qemu_plugin_tb *tb,
+ uint64_t pc)
{
struct qemu_plugin_insn *insn;
int i, j;
@@ -179,6 +181,7 @@ struct qemu_plugin_insn *qemu_plugin_tb_insn_get(struct qemu_plugin_tb *tb)
g_byte_array_set_size(insn->data, 0);
insn->calls_helpers = false;
insn->mem_helper = false;
+ insn->vaddr = pc;
for (i = 0; i < PLUGIN_N_CB_TYPES; i++) {
for (j = 0; j < PLUGIN_N_CB_SUBTYPES; j++) {
diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index 61be64b78c..22d95fe1c3 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -876,9 +876,8 @@ void plugin_gen_insn_start(CPUState *cpu, const DisasContextBase *db)
struct qemu_plugin_tb *ptb = tcg_ctx->plugin_tb;
struct qemu_plugin_insn *pinsn;
- pinsn = qemu_plugin_tb_insn_get(ptb);
+ pinsn = qemu_plugin_tb_insn_get(ptb, db->pc_next);
tcg_ctx->plugin_insn = pinsn;
- pinsn->vaddr = db->pc_next;
plugin_gen_empty_callback(PLUGIN_GEN_FROM_INSN);
/*
diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
index 390bd9db0a..f06c314266 100644
--- a/accel/tcg/translator.c
+++ b/accel/tcg/translator.c
@@ -169,7 +169,7 @@ static inline void translator_maybe_page_protect(DisasContextBase *dcbase,
if (do_swap) { \
ret = swap_fn(ret); \
} \
- plugin_insn_append(&ret, sizeof(ret)); \
+ plugin_insn_append(pc, &ret, sizeof(ret)); \
return ret; \
}
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 18/21] tests/tcg: remove duplicate EXTRA_RUNS
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (16 preceding siblings ...)
2021-11-03 17:05 ` [PULL 17/21] plugins: try and make plugin_insn_append more ergonomic Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 19/21] gdbstub: Switch to the thread receiving a signal Alex Bennée
` (3 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Alex Bennée, richard.henderson, qemu-devel,
Philippe Mathieu-Daudé
We set it bellow outside the #if leg.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20211026102234.3961636-28-alex.bennee@linaro.org>
diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target
index 6ccb592aac..c0d9e638e9 100644
--- a/tests/tcg/multiarch/Makefile.target
+++ b/tests/tcg/multiarch/Makefile.target
@@ -61,8 +61,6 @@ run-gdbstub-sha1: sha1
--bin $< --test $(MULTIARCH_SRC)/gdbstub/sha1.py, \
"basic gdbstub support")
-EXTRA_RUNS += run-gdbstub-sha1
-
run-gdbstub-qxfer-auxv-read: sha1
$(call run-test, $@, $(GDB_SCRIPT) \
--gdb $(HAVE_GDB_BIN) \
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 19/21] gdbstub: Switch to the thread receiving a signal
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (17 preceding siblings ...)
2021-11-03 17:05 ` [PULL 18/21] tests/tcg: remove duplicate EXTRA_RUNS Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 20/21] tests/tcg: remove debug polluting make output Alex Bennée
` (2 subsequent siblings)
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Alex Bennée, Pavel Labath, richard.henderson, qemu-devel,
Philippe Mathieu-Daudé
From: Pavel Labath <pavel@labath.sk>
Respond with Txxthread:yyyy; instead of a plain Sxx to indicate which
thread received the signal. Otherwise, the debugger will associate it
with the main one. Also automatically select this thread, as that is
what gdb expects.
Signed-off-by: Pavel Labath <pavel@labath.sk>
Message-Id: <20211019174953.36560-1-pavel@labath.sk>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20211026102234.3961636-29-alex.bennee@linaro.org>
diff --git a/gdbstub.c b/gdbstub.c
index 36b85aa50e..23baaef40e 100644
--- a/gdbstub.c
+++ b/gdbstub.c
@@ -3138,8 +3138,12 @@ gdb_handlesig(CPUState *cpu, int sig)
tb_flush(cpu);
if (sig != 0) {
- snprintf(buf, sizeof(buf), "S%02x", target_signal_to_gdb(sig));
- put_packet(buf);
+ gdb_set_stop_cpu(cpu);
+ g_string_printf(gdbserver_state.str_buf,
+ "T%02xthread:", target_signal_to_gdb(sig));
+ gdb_append_thread_id(cpu, gdbserver_state.str_buf);
+ g_string_append_c(gdbserver_state.str_buf, ';');
+ put_strbuf();
}
/* put_packet() might have detected that the peer terminated the
connection. */
diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target
index c0d9e638e9..b962ed8236 100644
--- a/tests/tcg/multiarch/Makefile.target
+++ b/tests/tcg/multiarch/Makefile.target
@@ -68,11 +68,19 @@ run-gdbstub-qxfer-auxv-read: sha1
--bin $< --test $(MULTIARCH_SRC)/gdbstub/test-qxfer-auxv-read.py, \
"basic gdbstub qXfer:auxv:read support")
+run-gdbstub-thread-breakpoint: testthread
+ $(call run-test, $@, $(GDB_SCRIPT) \
+ --gdb $(HAVE_GDB_BIN) \
+ --qemu $(QEMU) --qargs "$(QEMU_OPTS)" \
+ --bin $< --test $(MULTIARCH_SRC)/gdbstub/test-thread-breakpoint.py, \
+ "hitting a breakpoint on non-main thread")
+
else
run-gdbstub-%:
$(call skip-test, "gdbstub test $*", "need working gdb")
endif
-EXTRA_RUNS += run-gdbstub-sha1 run-gdbstub-qxfer-auxv-read
+EXTRA_RUNS += run-gdbstub-sha1 run-gdbstub-qxfer-auxv-read \
+ run-gdbstub-thread-breakpoint
# ARM Compatible Semi Hosting Tests
#
diff --git a/tests/tcg/multiarch/gdbstub/test-thread-breakpoint.py b/tests/tcg/multiarch/gdbstub/test-thread-breakpoint.py
new file mode 100644
index 0000000000..798d508bc7
--- /dev/null
+++ b/tests/tcg/multiarch/gdbstub/test-thread-breakpoint.py
@@ -0,0 +1,60 @@
+from __future__ import print_function
+#
+# Test auxiliary vector is loaded via gdbstub
+#
+# This is launched via tests/guest-debug/run-test.py
+#
+
+import gdb
+import sys
+
+failcount = 0
+
+def report(cond, msg):
+ "Report success/fail of test"
+ if cond:
+ print ("PASS: %s" % (msg))
+ else:
+ print ("FAIL: %s" % (msg))
+ global failcount
+ failcount += 1
+
+def run_test():
+ "Run through the tests one by one"
+
+ sym, ok = gdb.lookup_symbol("thread1_func")
+ gdb.execute("b thread1_func")
+ gdb.execute("c")
+
+ frame = gdb.selected_frame()
+ report(str(frame.function()) == "thread1_func", "break @ %s"%frame)
+
+#
+# This runs as the script it sourced (via -x, via run-test.py)
+#
+try:
+ inferior = gdb.selected_inferior()
+ arch = inferior.architecture()
+ print("ATTACHED: %s" % arch.name())
+except (gdb.error, AttributeError):
+ print("SKIPPING (not connected)", file=sys.stderr)
+ exit(0)
+
+if gdb.parse_and_eval('$pc') == 0:
+ print("SKIP: PC not set")
+ exit(0)
+
+try:
+ # These are not very useful in scripts
+ gdb.execute("set pagination off")
+ gdb.execute("set confirm off")
+
+ # Run the actual tests
+ run_test()
+except (gdb.error):
+ print ("GDB Exception: %s" % (sys.exc_info()[0]))
+ failcount += 1
+ pass
+
+print("All tests complete: %d failures" % failcount)
+exit(failcount)
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 20/21] tests/tcg: remove debug polluting make output
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (18 preceding siblings ...)
2021-11-03 17:05 ` [PULL 19/21] gdbstub: Switch to the thread receiving a signal Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:05 ` [PULL 21/21] tests/vm/openbsd: Update to release 7.0 Alex Bennée
2021-11-03 17:20 ` [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Alex Bennée, richard.henderson, qemu-devel,
Philippe Mathieu-Daudé
Fixes: 5343a837cd ("tests/tcg: move some multiarch files and make conditional")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20211026173914.79377-1-alex.bennee@linaro.org>
diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target
index b962ed8236..a83efb4a9d 100644
--- a/tests/tcg/multiarch/Makefile.target
+++ b/tests/tcg/multiarch/Makefile.target
@@ -16,7 +16,6 @@ MULTIARCH_SRCS += $(notdir $(wildcard $(MULTIARCH_SRC)/linux/*.c))
endif
MULTIARCH_TESTS = $(MULTIARCH_SRCS:.c=)
-$(info SRCS=${MULTIARCH_SRCS} and ${MULTIARCH_TESTS})
#
# The following are any additional rules needed to build things
#
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PULL 21/21] tests/vm/openbsd: Update to release 7.0
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (19 preceding siblings ...)
2021-11-03 17:05 ` [PULL 20/21] tests/tcg: remove debug polluting make output Alex Bennée
@ 2021-11-03 17:05 ` Alex Bennée
2021-11-03 17:20 ` [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
21 siblings, 0 replies; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:05 UTC (permalink / raw)
To: peter.maydell
Cc: Thomas Huth, richard.henderson, qemu-devel,
Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
Willian Rampazzo, Alex Bennée, Brad Smith
From: Richard Henderson <richard.henderson@linaro.org>
There are two minor changes required in the script for the
network configuration of the newer release.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20211018205313.3526915-1-richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index c4c78a80f1..337fe7c303 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -22,8 +22,8 @@ class OpenBSDVM(basevm.BaseVM):
name = "openbsd"
arch = "x86_64"
- link = "https://cdn.openbsd.org/pub/OpenBSD/6.9/amd64/install69.iso"
- csum = "140d26548aec680e34bb5f82295414228e7f61e4f5e7951af066014fda2d6e43"
+ link = "https://cdn.openbsd.org/pub/OpenBSD/7.0/amd64/install70.iso"
+ csum = "1882f9a23c9800e5dba3dbd2cf0126f552605c915433ef4c5bb672610a4ca3a4"
size = "20G"
pkgs = [
# tools
@@ -95,10 +95,9 @@ class OpenBSDVM(basevm.BaseVM):
self.console_wait_send("Terminal type", "xterm\n")
self.console_wait_send("System hostname", "openbsd\n")
self.console_wait_send("Which network interface", "vio0\n")
- self.console_wait_send("IPv4 address", "dhcp\n")
+ self.console_wait_send("IPv4 address", "autoconf\n")
self.console_wait_send("IPv6 address", "none\n")
self.console_wait_send("Which network interface", "done\n")
- self.console_wait_send("DNS domain name", "localnet\n")
self.console_wait("Password for root account")
self.console_send("%s\n" % self._config["root_pass"])
self.console_wait("Password for root account")
--
2.30.2
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PULL for 6.2 00/21] testing, plugin and gdbstub updates
2021-11-03 17:05 [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
` (20 preceding siblings ...)
2021-11-03 17:05 ` [PULL 21/21] tests/vm/openbsd: Update to release 7.0 Alex Bennée
@ 2021-11-03 17:20 ` Alex Bennée
2021-11-03 19:32 ` Taylor Simpson
21 siblings, 1 reply; 25+ messages in thread
From: Alex Bennée @ 2021-11-03 17:20 UTC (permalink / raw)
To: peter.maydell
Cc: Alex Bennée, Taylor Simpson, richard.henderson, qemu-devel
Alex Bennée <alex.bennee@linaro.org> writes:
> The following changes since commit e86e00a2493254d072581960b48461eb96481e45:
>
> Merge remote-tracking branch 'remotes/berrange/tags/hmp-x-qmp-620-pull-request' into staging (2021-11-03 08:04:32 -0400)
>
<snip>
> tests/docker: Update debian-hexagon-cross to a newer toolchain
Hmm it seems like we can blame the toolchain:
Subject: Hexagon toolchain update vs linux-user signals
From: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <4f65337b-b9ed-dc4c-ac09-025bef5eaa4c@linaro.org>
Date: Wed, 3 Nov 2021 11:22:03 -0400
NACK this PR and I'll roll v2 tomorrow....
--
Alex Bennée
^ permalink raw reply [flat|nested] 25+ messages in thread* RE: [PULL for 6.2 00/21] testing, plugin and gdbstub updates
2021-11-03 17:20 ` [PULL for 6.2 00/21] testing, plugin and gdbstub updates Alex Bennée
@ 2021-11-03 19:32 ` Taylor Simpson
2021-11-04 4:45 ` Richard Henderson
0 siblings, 1 reply; 25+ messages in thread
From: Taylor Simpson @ 2021-11-03 19:32 UTC (permalink / raw)
To: Alex Bennée, peter.maydell; +Cc: Brian Cain, richard.henderson, qemu-devel
We'll investigate the toolchain issue. This is compiling tests/tcg/multiarch/signals.c, right?
Also, I'll do as Richard suggests in his email and respin the pull with the HVX tests disabled in the makefile.
Thanks,
Taylor
> -----Original Message-----
> From: Alex Bennée <alex.bennee@linaro.org>
> Sent: Wednesday, November 3, 2021 12:21 PM
> To: peter.maydell@linaro.org
> Cc: richard.henderson@linaro.org; qemu-devel@nongnu.org; Alex Bennée
> <alex.bennee@linaro.org>; Taylor Simpson <tsimpson@quicinc.com>
> Subject: Re: [PULL for 6.2 00/21] testing, plugin and gdbstub updates
>
> WARNING: This email originated from outside of Qualcomm. Please be wary
> of any links or attachments, and do not enable macros.
>
> Alex Bennée <alex.bennee@linaro.org> writes:
>
> > The following changes since commit
> e86e00a2493254d072581960b48461eb96481e45:
> >
> > Merge remote-tracking branch 'remotes/berrange/tags/hmp-x-qmp-620-
> pull-request' into staging (2021-11-03 08:04:32 -0400)
> >
> <snip>
> > tests/docker: Update debian-hexagon-cross to a newer toolchain
>
> Hmm it seems like we can blame the toolchain:
>
> Subject: Hexagon toolchain update vs linux-user signals
> From: Richard Henderson <richard.henderson@linaro.org>
> Message-ID: <4f65337b-b9ed-dc4c-ac09-025bef5eaa4c@linaro.org>
> Date: Wed, 3 Nov 2021 11:22:03 -0400
>
> NACK this PR and I'll roll v2 tomorrow....
>
> --
> Alex Bennée
^ permalink raw reply [flat|nested] 25+ messages in thread