* [PATCH v1 01/28] tests/docker: Use apt build-dep in debian10
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 10:22 ` [PATCH v1 02/28] tests/docker: Simplify debian-all-test-cross Alex Bennée
` (26 subsequent siblings)
27 siblings, 0 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, Thomas Huth, minyihh, berrange, kuhn.chenqun,
Richard Henderson, f4bug, robhenry, Willian Rampazzo,
mahmoudabdalghany, aaron, cota, Wainer dos Santos Moschetta,
stefanha, crosa, pbonzini, ma.mandourr, Alex Bennée,
aurelien
From: Richard Henderson <richard.henderson@linaro.org>
Replace the complex apt-get subshell with apt build-dep.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211014224435.2539547-2-richard.henderson@linaro.org>
---
tests/docker/dockerfiles/debian10.docker | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/tests/docker/dockerfiles/debian10.docker b/tests/docker/dockerfiles/debian10.docker
index b414af1b9f..d3bbb90c64 100644
--- a/tests/docker/dockerfiles/debian10.docker
+++ b/tests/docker/dockerfiles/debian10.docker
@@ -14,7 +14,8 @@ RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.lis
# Install common build utilities
RUN apt update && \
- DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
+ DEBIAN_FRONTEND=noninteractive \
+ apt install -yy eatmydata && \
DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends \
bc \
@@ -33,5 +34,7 @@ RUN apt update && \
psmisc \
python3 \
python3-sphinx \
- python3-sphinx-rtd-theme \
- $(apt-get -s build-dep --arch-only qemu | egrep ^Inst | fgrep '[all]' | cut -d\ -f2)
+ python3-sphinx-rtd-theme
+
+RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+ apt build-dep -yy qemu
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* [PATCH v1 02/28] tests/docker: Simplify debian-all-test-cross
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
2021-10-26 10:22 ` [PATCH v1 01/28] tests/docker: Use apt build-dep in debian10 Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 10:22 ` [PATCH v1 03/28] tests/docker: Update debian-hexagon-cross to a newer toolchain Alex Bennée
` (25 subsequent siblings)
27 siblings, 0 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, Thomas Huth, minyihh, berrange, kuhn.chenqun,
Richard Henderson, f4bug, robhenry, Willian Rampazzo,
mahmoudabdalghany, aaron, cota, Wainer dos Santos Moschetta,
stefanha, crosa, pbonzini, ma.mandourr, Alex Bennée,
aurelien
From: Richard Henderson <richard.henderson@linaro.org>
The base debian10 image contains enough to build qemu;
we do not need to repeat that within the new image.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211014224435.2539547-3-richard.henderson@linaro.org>
---
tests/docker/dockerfiles/debian-all-test-cross.docker | 5 -----
1 file changed, 5 deletions(-)
diff --git a/tests/docker/dockerfiles/debian-all-test-cross.docker b/tests/docker/dockerfiles/debian-all-test-cross.docker
index dedcea58b4..b185b7c15a 100644
--- a/tests/docker/dockerfiles/debian-all-test-cross.docker
+++ b/tests/docker/dockerfiles/debian-all-test-cross.docker
@@ -8,11 +8,6 @@
#
FROM qemu/debian10
-# What we need to build QEMU itself
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt build-dep -yy qemu
-
# Add the foreign architecture we want and install dependencies
RUN DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends \
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* [PATCH v1 03/28] tests/docker: Update debian-hexagon-cross to a newer toolchain
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
2021-10-26 10:22 ` [PATCH v1 01/28] tests/docker: Use apt build-dep in debian10 Alex Bennée
2021-10-26 10:22 ` [PATCH v1 02/28] tests/docker: Simplify debian-all-test-cross Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 14:58 ` Philippe Mathieu-Daudé
2021-10-26 10:22 ` [PATCH v1 04/28] gitlab-ci: Remove special casing for hexagon testing Alex Bennée
` (24 subsequent siblings)
27 siblings, 1 reply; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, Brian Cain, Thomas Huth, minyihh, berrange, kuhn.chenqun,
Brian Cain, Richard Henderson, f4bug, robhenry, Willian Rampazzo,
mahmoudabdalghany, aaron, cota, Wainer dos Santos Moschetta,
stefanha, crosa, pbonzini, ma.mandourr, Alex Bennée,
aurelien
From: Brian Cain <brian.cain@gmail.com>
The new dockerfile uses a complete toolchain built by CodeLinaro.
Remove the special casing that was necessary when the dockerfile
built the toolchain from source.
Suggested-by: Brian Cain <bcain@quicinc.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
[AJB: fix MAINTAINERS]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211014224435.2539547-4-richard.henderson@linaro.org>
squash! tests/docker: Update debian-hexagon-cross to a newer toolchain
---
MAINTAINERS | 1 -
tests/docker/Makefile.include | 24 +--
.../dockerfiles/debian-hexagon-cross.docker | 45 +-----
.../build-toolchain.sh | 141 ------------------
4 files changed, 9 insertions(+), 202 deletions(-)
delete mode 100755 tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
diff --git a/MAINTAINERS b/MAINTAINERS
index 894dc43105..5423ae0cd4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -199,7 +199,6 @@ F: tests/tcg/hexagon/
F: disas/hexagon.c
F: configs/targets/hexagon-linux-user/default.mak
F: docker/dockerfiles/debian-hexagon-cross.docker
-F: docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
HPPA (PA-RISC) TCG CPUs
M: Richard Henderson <richard.henderson@linaro.org>
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index b9d4094c2e..d223100dff 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -148,33 +148,11 @@ docker-image-debian-sparc64-cross: docker-image-debian10
# The native build should never use the registry
docker-image-debian-native: DOCKER_REGISTRY=
-
-#
-# The build rule for hexagon-cross is special in so far for most of
-# the time we don't want to build it. While dockers caching does avoid
-# this most of the time sometimes we want to force the issue.
-#
-docker-image-debian-hexagon-cross: $(DOCKER_FILES_DIR)/debian-hexagon-cross.docker
- $(if $(NOCACHE), \
- $(call quiet-command, \
- $(DOCKER_SCRIPT) build -t qemu/debian-hexagon-cross -f $< \
- $(if $V,,--quiet) --no-cache \
- --registry $(DOCKER_REGISTRY) --extra-files \
- $(DOCKER_FILES_DIR)/debian-hexagon-cross.docker.d/build-toolchain.sh, \
- "BUILD", "debian-hexagon-cross"), \
- $(call quiet-command, \
- $(DOCKER_SCRIPT) fetch $(if $V,,--quiet) \
- qemu/debian-hexagon-cross $(DOCKER_REGISTRY), \
- "FETCH", "debian-hexagon-cross") \
- $(call quiet-command, \
- $(DOCKER_SCRIPT) update $(if $V,,--quiet) \
- qemu/debian-hexagon-cross --add-current-user, \
- "PREPARE", "debian-hexagon-cross"))
-
# Specialist build images, sometimes very limited tools
docker-image-debian-tricore-cross: docker-image-debian10
docker-image-debian-all-test-cross: docker-image-debian10
docker-image-debian-arm64-test-cross: docker-image-debian11
+docker-image-debian-hexagon-cross: docker-image-debian10
docker-image-debian-powerpc-test-cross: docker-image-debian11
# These images may be good enough for building tests but not for test builds
diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.docker b/tests/docker/dockerfiles/debian-hexagon-cross.docker
index d5dc299dc1..f3d7110814 100644
--- a/tests/docker/dockerfiles/debian-hexagon-cross.docker
+++ b/tests/docker/dockerfiles/debian-hexagon-cross.docker
@@ -1,45 +1,16 @@
#
# Docker Hexagon cross-compiler target
#
-# This docker target is used for building hexagon tests. As it also
-# needs to be able to build QEMU itself in CI we include it's
-# build-deps. It is also a "stand-alone" image so as not to be
-# triggered by re-builds on other base images given it takes a long
-# time to build.
-#
FROM qemu/debian10
-# Install common build utilities
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
+RUN DEBIAN_FRONTEND=noninteractive eatmydata \
apt install -y --no-install-recommends \
- bison \
- cmake \
- flex \
- lld \
- rsync \
- wget
-
-ENV TOOLCHAIN_INSTALL /usr/local
-ENV ROOTFS /usr/local
-
-ENV LLVM_URL https://github.com/llvm/llvm-project/archive/bfcd21876adc3498065e4da92799f613e730d475.tar.gz
-ENV MUSL_URL https://github.com/quic/musl/archive/aff74b395fbf59cd7e93b3691905aa1af6c0778c.tar.gz
-ENV LINUX_URL https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.6.18.tar.xz
-
-ADD build-toolchain.sh /root/hexagon-toolchain/build-toolchain.sh
+ curl \
+ xz-utils
-RUN cd /root/hexagon-toolchain && ./build-toolchain.sh
+RUN curl -#SL https://codelinaro.jfrog.io/artifactory/codelinaro-toolchain-for-hexagon/v2021.09.10/clang+llvm-Sept-2021-cross-hexagon-unknown-linux-musl.tar.xz \
+ | tar -xJC /opt
-FROM debian:buster-slim
-# Duplicate deb line as deb-src
-RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
-# Install QEMU build deps for use in CI
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
- DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy git ninja-build && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt build-dep -yy --arch-only qemu
-COPY --from=0 /usr/local /usr/local
-ENV PATH $PATH:/usr/local/bin/
+ENV PATH $PATH:/opt/clang+llvm-Sept-2021-cross-hexagon-unknown-linux-musl/x86_64-linux-gnu/bin
+ENV DEF_TARGET_LIST hexagon-linux-user
+ENV QEMU_CONFIGURE_OPTS --disable-docs --disable-tools --cross-cc-hexagon=hexagon-unknown-linux-musl-clang
diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh b/tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
deleted file mode 100755
index 19b1c9f83e..0000000000
--- a/tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
+++ /dev/null
@@ -1,141 +0,0 @@
-#!/bin/bash
-
-set -e
-
-BASE=$(readlink -f ${PWD})
-
-TOOLCHAIN_INSTALL=$(readlink -f "$TOOLCHAIN_INSTALL")
-ROOTFS=$(readlink -f "$ROOTFS")
-
-TOOLCHAIN_BIN=${TOOLCHAIN_INSTALL}/bin
-HEX_SYSROOT=${TOOLCHAIN_INSTALL}/hexagon-unknown-linux-musl
-HEX_TOOLS_TARGET_BASE=${HEX_SYSROOT}/usr
-
-function cdp() {
- DIR="$1"
- mkdir -p "$DIR"
- cd "$DIR"
-}
-
-function fetch() {
- DIR="$1"
- URL="$2"
- TEMP="$(readlink -f "$PWD/tmp.tar.gz")"
- wget --quiet "$URL" -O "$TEMP"
- cdp "$DIR"
- tar xaf "$TEMP" --strip-components=1
- rm "$TEMP"
- cd -
-}
-
-build_llvm_clang() {
- fetch "$BASE/llvm-project" "$LLVM_URL"
- cdp "$BASE/build-llvm"
-
- cmake -G Ninja \
- -DCMAKE_BUILD_TYPE=Release \
- -DCMAKE_INSTALL_PREFIX=${TOOLCHAIN_INSTALL} \
- -DLLVM_ENABLE_LLD=ON \
- -DLLVM_TARGETS_TO_BUILD="Hexagon" \
- -DLLVM_ENABLE_PROJECTS="clang;lld" \
- "$BASE/llvm-project/llvm"
- ninja all install
- cd ${TOOLCHAIN_BIN}
- ln -sf clang hexagon-unknown-linux-musl-clang
- ln -sf clang++ hexagon-unknown-linux-musl-clang++
- ln -sf llvm-ar hexagon-unknown-linux-musl-ar
- ln -sf llvm-objdump hexagon-unknown-linux-musl-objdump
- ln -sf llvm-objcopy hexagon-unknown-linux-musl-objcopy
- ln -sf llvm-readelf hexagon-unknown-linux-musl-readelf
- ln -sf llvm-ranlib hexagon-unknown-linux-musl-ranlib
-
- # workaround for now:
- cat <<EOF > hexagon-unknown-linux-musl.cfg
--G0 --sysroot=${HEX_SYSROOT}
-EOF
-}
-
-build_clang_rt() {
- cdp "$BASE/build-clang_rt"
- cmake -G Ninja \
- -DCMAKE_BUILD_TYPE=Release \
- -DLLVM_CONFIG_PATH="$BASE/build-llvm/bin/llvm-config" \
- -DCMAKE_ASM_FLAGS="-G0 -mlong-calls -fno-pic --target=hexagon-unknown-linux-musl " \
- -DCMAKE_SYSTEM_NAME=Linux \
- -DCMAKE_C_COMPILER="${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang" \
- -DCMAKE_ASM_COMPILER="${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang" \
- -DCMAKE_INSTALL_PREFIX=${HEX_TOOLS_TARGET_BASE} \
- -DCMAKE_CROSSCOMPILING=ON \
- -DCMAKE_C_COMPILER_FORCED=ON \
- -DCMAKE_CXX_COMPILER_FORCED=ON \
- -DCOMPILER_RT_BUILD_BUILTINS=ON \
- -DCOMPILER_RT_BUILTINS_ENABLE_PIC=OFF \
- -DCMAKE_SIZEOF_VOID_P=4 \
- -DCOMPILER_RT_OS_DIR= \
- -DCAN_TARGET_hexagon=1 \
- -DCAN_TARGET_x86_64=0 \
- -DCOMPILER_RT_SUPPORTED_ARCH=hexagon \
- -DLLVM_ENABLE_PROJECTS="compiler-rt" \
- "$BASE/llvm-project/compiler-rt"
- ninja install-compiler-rt
-}
-
-build_musl_headers() {
- fetch "$BASE/musl" "$MUSL_URL"
- cd "$BASE/musl"
- make clean
- CC=${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang \
- CROSS_COMPILE=hexagon-unknown-linux-musl \
- LIBCC=${HEX_TOOLS_TARGET_BASE}/lib/libclang_rt.builtins-hexagon.a \
- CROSS_CFLAGS="-G0 -O0 -mv65 -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl" \
- ./configure --target=hexagon --prefix=${HEX_TOOLS_TARGET_BASE}
- PATH=${TOOLCHAIN_BIN}:$PATH make CROSS_COMPILE= install-headers
-
- cd ${HEX_SYSROOT}/..
- ln -sf hexagon-unknown-linux-musl hexagon
-}
-
-build_kernel_headers() {
- fetch "$BASE/linux" "$LINUX_URL"
- mkdir -p "$BASE/build-linux"
- cd "$BASE/linux"
- make O=../build-linux ARCH=hexagon \
- KBUILD_CFLAGS_KERNEL="-mlong-calls" \
- CC=${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang \
- LD=${TOOLCHAIN_BIN}/ld.lld \
- KBUILD_VERBOSE=1 comet_defconfig
- make mrproper
-
- cd "$BASE/build-linux"
- make \
- ARCH=hexagon \
- CC=${TOOLCHAIN_BIN}/clang \
- INSTALL_HDR_PATH=${HEX_TOOLS_TARGET_BASE} \
- V=1 \
- headers_install
-}
-
-build_musl() {
- cd "$BASE/musl"
- make clean
- CROSS_COMPILE=hexagon-unknown-linux-musl- \
- AR=llvm-ar \
- RANLIB=llvm-ranlib \
- STRIP=llvm-strip \
- CC=clang \
- LIBCC=${HEX_TOOLS_TARGET_BASE}/lib/libclang_rt.builtins-hexagon.a \
- CFLAGS="-G0 -O0 -mv65 -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl" \
- ./configure --target=hexagon --prefix=${HEX_TOOLS_TARGET_BASE}
- PATH=${TOOLCHAIN_BIN}/:$PATH make CROSS_COMPILE= install
- cd ${HEX_TOOLS_TARGET_BASE}/lib
- ln -sf libc.so ld-musl-hexagon.so
- ln -sf ld-musl-hexagon.so ld-musl-hexagon.so.1
- cdp ${HEX_TOOLS_TARGET_BASE}/../lib
- ln -sf ../usr/lib/ld-musl-hexagon.so.1
-}
-
-build_llvm_clang
-build_kernel_headers
-build_musl_headers
-build_clang_rt
-build_musl
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 03/28] tests/docker: Update debian-hexagon-cross to a newer toolchain
2021-10-26 10:22 ` [PATCH v1 03/28] tests/docker: Update debian-hexagon-cross to a newer toolchain Alex Bennée
@ 2021-10-26 14:58 ` Philippe Mathieu-Daudé
2021-10-26 14:59 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 61+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-10-26 14:58 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Brian Cain
Cc: fam, Thomas Huth, Brian Cain, pbonzini, Richard Henderson,
berrange, robhenry, aaron, Willian Rampazzo, mahmoudabdalghany,
minyihh, cota, Wainer dos Santos Moschetta, stefanha, crosa,
kuhn.chenqun, ma.mandourr, aurelien
On 10/26/21 12:22, Alex Bennée wrote:
> From: Brian Cain <brian.cain@gmail.com>
>
> The new dockerfile uses a complete toolchain built by CodeLinaro.
> Remove the special casing that was necessary when the dockerfile
> built the toolchain from source.
Can we start using .container_job_template for the
currently special-cased hexagon-cross-container job then?
> Suggested-by: Brian Cain <bcain@quicinc.com>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> [AJB: fix MAINTAINERS]
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20211014224435.2539547-4-richard.henderson@linaro.org>
>
> squash! tests/docker: Update debian-hexagon-cross to a newer toolchain
> ---
> MAINTAINERS | 1 -
> tests/docker/Makefile.include | 24 +--
> .../dockerfiles/debian-hexagon-cross.docker | 45 +-----
> .../build-toolchain.sh | 141 ------------------
> 4 files changed, 9 insertions(+), 202 deletions(-)
> delete mode 100755 tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [PATCH v1 03/28] tests/docker: Update debian-hexagon-cross to a newer toolchain
2021-10-26 14:58 ` Philippe Mathieu-Daudé
@ 2021-10-26 14:59 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 61+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-10-26 14:59 UTC (permalink / raw)
To: Alex Bennée, qemu-devel@nongnu.org Developers, Brian Cain
Cc: Fam Zheng, Thomas Huth, Brian Cain, Paolo Bonzini,
Richard Henderson, Daniel P. Berrange, robhenry, aaron,
Willian Rampazzo, Mahmoud Abd Al Ghany, minyihh, Emilio G . Cota,
Wainer dos Santos Moschetta, Stefan Hajnoczi, Cleber Rosa,
Chen Qun, Mahmoud Mandour, Aurelien Jarno
On Tue, Oct 26, 2021 at 4:58 PM Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> On 10/26/21 12:22, Alex Bennée wrote:
> > From: Brian Cain <brian.cain@gmail.com>
> >
> > The new dockerfile uses a complete toolchain built by CodeLinaro.
> > Remove the special casing that was necessary when the dockerfile
> > built the toolchain from source.
>
> Can we start using .container_job_template for the
> currently special-cased hexagon-cross-container job then?
Doh whatever this is exactly what the following patch does...
> > Suggested-by: Brian Cain <bcain@quicinc.com>
> > Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> > [AJB: fix MAINTAINERS]
> > Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> > Message-Id: <20211014224435.2539547-4-richard.henderson@linaro.org>
> >
> > squash! tests/docker: Update debian-hexagon-cross to a newer toolchain
> > ---
> > MAINTAINERS | 1 -
> > tests/docker/Makefile.include | 24 +--
> > .../dockerfiles/debian-hexagon-cross.docker | 45 +-----
> > .../build-toolchain.sh | 141 ------------------
> > 4 files changed, 9 insertions(+), 202 deletions(-)
> > delete mode 100755 tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 04/28] gitlab-ci: Remove special casing for hexagon testing
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (2 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 03/28] tests/docker: Update debian-hexagon-cross to a newer toolchain Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 15:00 ` Philippe Mathieu-Daudé
2021-10-26 20:46 ` Willian Rampazzo
2021-10-26 10:22 ` [PATCH v1 05/28] tests/docker: Add debian-nios2-cross image Alex Bennée
` (23 subsequent siblings)
27 siblings, 2 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, Thomas Huth, minyihh, berrange, kuhn.chenqun,
Richard Henderson, f4bug, robhenry, Willian Rampazzo,
mahmoudabdalghany, aaron, cota, Wainer dos Santos Moschetta,
stefanha, crosa, pbonzini, ma.mandourr, Alex Bennée,
aurelien
From: Richard Henderson <richard.henderson@linaro.org>
Now that the hexagon container is using a pre-built
toolchain, we do not need to make the build optional.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211014224435.2539547-5-richard.henderson@linaro.org>
---
.gitlab-ci.d/buildtest.yml | 4 ----
.gitlab-ci.d/container-cross.yml | 27 +++------------------------
2 files changed, 3 insertions(+), 28 deletions(-)
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 5c378e35f9..27b9b6be6b 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -249,14 +249,10 @@ build-user-static:
CONFIGURE_ARGS: --disable-tools --disable-system --static
MAKE_CHECK_ARGS: check-tcg
-# Because the hexagon cross-compiler takes so long to build we don't rely
-# on the CI system to build it and hence this job has an optional dependency
-# declared. The image is manually uploaded.
build-user-hexagon:
extends: .native_build_job_template
needs:
job: hexagon-cross-container
- optional: true
variables:
IMAGE: debian-hexagon-cross
TARGETS: hexagon-linux-user
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index a3b5b90552..ed7a849a69 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -47,33 +47,12 @@ armhf-debian-cross-container:
variables:
NAME: debian-armhf-cross
-# We never want to build hexagon in the CI system and by default we
-# always want to refer to the master registry where it lives.
hexagon-cross-container:
- image: docker:stable
- stage: containers
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project"'
- when: never
- - when: always
+ extends: .container_job_template
+ stage: containers-layer2
+ needs: ['amd64-debian10-container']
variables:
NAME: debian-hexagon-cross
- GIT_DEPTH: 1
- services:
- - docker:dind
- before_script:
- - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
- - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/qemu/$NAME:latest"
- - docker info
- - docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD"
- script:
- - echo "TAG:$TAG"
- - echo "COMMON_TAG:$COMMON_TAG"
- - docker pull $COMMON_TAG
- - docker tag $COMMON_TAG $TAG
- - docker push "$TAG"
- after_script:
- - docker logout
hppa-debian-cross-container:
extends: .container_job_template
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 04/28] gitlab-ci: Remove special casing for hexagon testing
2021-10-26 10:22 ` [PATCH v1 04/28] gitlab-ci: Remove special casing for hexagon testing Alex Bennée
@ 2021-10-26 15:00 ` Philippe Mathieu-Daudé
2021-10-26 20:46 ` Willian Rampazzo
1 sibling, 0 replies; 61+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-10-26 15:00 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, Thomas Huth, berrange, pbonzini, Richard Henderson, aaron,
robhenry, Willian Rampazzo, mahmoudabdalghany, minyihh, cota,
Wainer dos Santos Moschetta, stefanha, crosa, kuhn.chenqun,
ma.mandourr, aurelien
On 10/26/21 12:22, Alex Bennée wrote:
> From: Richard Henderson <richard.henderson@linaro.org>
>
> Now that the hexagon container is using a pre-built
> toolchain, we do not need to make the build optional.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20211014224435.2539547-5-richard.henderson@linaro.org>
> ---
> .gitlab-ci.d/buildtest.yml | 4 ----
> .gitlab-ci.d/container-cross.yml | 27 +++------------------------
> 2 files changed, 3 insertions(+), 28 deletions(-)
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [PATCH v1 04/28] gitlab-ci: Remove special casing for hexagon testing
2021-10-26 10:22 ` [PATCH v1 04/28] gitlab-ci: Remove special casing for hexagon testing Alex Bennée
2021-10-26 15:00 ` Philippe Mathieu-Daudé
@ 2021-10-26 20:46 ` Willian Rampazzo
1 sibling, 0 replies; 61+ messages in thread
From: Willian Rampazzo @ 2021-10-26 20:46 UTC (permalink / raw)
To: Alex Bennée
Cc: Fam Zheng, Thomas Huth, minyihh, Daniel Berrange, kuhn.chenqun,
Richard Henderson, qemu-devel, robhenry,
Philippe Mathieu-Daudé,
mahmoudabdalghany, aaron, cota, Wainer dos Santos Moschetta,
Stefan Hajnoczi, Cleber Rosa Junior, Paolo Bonzini,
Mahmoud Mandour, Aurelien Jarno
On Tue, Oct 26, 2021 at 7:22 AM Alex Bennée <alex.bennee@linaro.org> wrote:
>
> From: Richard Henderson <richard.henderson@linaro.org>
>
> Now that the hexagon container is using a pre-built
> toolchain, we do not need to make the build optional.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20211014224435.2539547-5-richard.henderson@linaro.org>
> ---
> .gitlab-ci.d/buildtest.yml | 4 ----
> .gitlab-ci.d/container-cross.yml | 27 +++------------------------
> 2 files changed, 3 insertions(+), 28 deletions(-)
>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 05/28] tests/docker: Add debian-nios2-cross image
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (3 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 04/28] gitlab-ci: Remove special casing for hexagon testing Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 15:01 ` Philippe Mathieu-Daudé
2021-10-26 20:49 ` Willian Rampazzo
2021-10-26 10:22 ` [PATCH v1 06/28] tests/docker: Add debian-microblaze-cross image Alex Bennée
` (22 subsequent siblings)
27 siblings, 2 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, Marek Vasut, Thomas Huth, minyihh, berrange, kuhn.chenqun,
Chris Wulff, Richard Henderson, f4bug, robhenry,
Willian Rampazzo, mahmoudabdalghany, aaron, cota,
Wainer dos Santos Moschetta, stefanha, crosa, pbonzini,
ma.mandourr, Alex Bennée, aurelien
From: Richard Henderson <richard.henderson@linaro.org>
Build the entire cross tool chain from source.
For this reason, default to caching.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
[AJB: honour NOUSER in cached fetch and build, update MAINTAINERS]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211014224435.2539547-6-richard.henderson@linaro.org>
---
MAINTAINERS | 1 +
tests/docker/Makefile.include | 25 ++++++
.../debian-nios2-cross.d/build-toolchain.sh | 87 +++++++++++++++++++
.../dockerfiles/debian-toolchain.docker | 36 ++++++++
4 files changed, 149 insertions(+)
create mode 100755 tests/docker/dockerfiles/debian-nios2-cross.d/build-toolchain.sh
create mode 100644 tests/docker/dockerfiles/debian-toolchain.docker
diff --git a/MAINTAINERS b/MAINTAINERS
index 5423ae0cd4..aa515bd313 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -256,6 +256,7 @@ F: target/nios2/
F: hw/nios2/
F: disas/nios2.c
F: configs/devices/nios2-softmmu/default.mak
+F: tests/docker/dockerfiles/debian-nios2-cross.d/build-toolchain.sh
OpenRISC TCG CPUs
M: Stafford Horne <shorne@gmail.com>
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index d223100dff..24ed5c1f1d 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -148,11 +148,35 @@ docker-image-debian-sparc64-cross: docker-image-debian10
# The native build should never use the registry
docker-image-debian-native: DOCKER_REGISTRY=
+debian-toolchain-run = \
+ $(if $(NOCACHE), \
+ $(call quiet-command, \
+ $(DOCKER_SCRIPT) build -t qemu/$1 -f $< \
+ $(if $V,,--quiet) --no-cache \
+ --registry $(DOCKER_REGISTRY) --extra-files \
+ $(DOCKER_FILES_DIR)/$1.d/build-toolchain.sh, \
+ "BUILD", $1), \
+ $(call quiet-command, \
+ $(DOCKER_SCRIPT) fetch $(if $V,,--quiet) \
+ qemu/$1 $(DOCKER_REGISTRY), \
+ "FETCH", $1) \
+ $(call quiet-command, \
+ $(DOCKER_SCRIPT) update $(if $V,,--quiet) \
+ qemu/$1 \
+ $(if $(NOUSER),,--add-current-user) \
+ "PREPARE", $1))
+debian-toolchain = $(call debian-toolchain-run,$(patsubst docker-image-%,%,$1))
+
+docker-image-debian-nios2-cross: $(DOCKER_FILES_DIR)/debian-toolchain.docker \
+ $(DOCKER_FILES_DIR)/debian-nios2-cross.d/build-toolchain.sh
+ $(call debian-toolchain, $@)
+
# Specialist build images, sometimes very limited tools
docker-image-debian-tricore-cross: docker-image-debian10
docker-image-debian-all-test-cross: docker-image-debian10
docker-image-debian-arm64-test-cross: docker-image-debian11
docker-image-debian-hexagon-cross: docker-image-debian10
+docker-image-debian-nios2-cross: docker-image-debian10
docker-image-debian-powerpc-test-cross: docker-image-debian11
# These images may be good enough for building tests but not for test builds
@@ -161,6 +185,7 @@ DOCKER_PARTIAL_IMAGES += debian-arm64-test-cross
DOCKER_PARTIAL_IMAGES += debian-powerpc-test-cross
DOCKER_PARTIAL_IMAGES += debian-hppa-cross
DOCKER_PARTIAL_IMAGES += debian-m68k-cross debian-mips64-cross
+DOCKER_PARTIAL_IMAGES += debian-nios2-cross
DOCKER_PARTIAL_IMAGES += debian-sh4-cross debian-sparc64-cross
DOCKER_PARTIAL_IMAGES += debian-tricore-cross
DOCKER_PARTIAL_IMAGES += debian-xtensa-cross
diff --git a/tests/docker/dockerfiles/debian-nios2-cross.d/build-toolchain.sh b/tests/docker/dockerfiles/debian-nios2-cross.d/build-toolchain.sh
new file mode 100755
index 0000000000..ba3c9d8aff
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-nios2-cross.d/build-toolchain.sh
@@ -0,0 +1,87 @@
+#!/bin/bash
+
+set -e
+
+TARGET=nios2-linux-gnu
+LINUX_ARCH=nios2
+
+J=$(expr $(nproc) / 2)
+TOOLCHAIN_INSTALL=/usr/local
+TOOLCHAIN_BIN=${TOOLCHAIN_INSTALL}/bin
+CROSS_SYSROOT=${TOOLCHAIN_INSTALL}/$TARGET/sys-root
+
+export PATH=${TOOLCHAIN_BIN}:$PATH
+
+#
+# Grab all of the source for the toolchain bootstrap.
+#
+
+wget https://ftp.gnu.org/gnu/binutils/binutils-2.37.tar.xz
+wget https://ftp.gnu.org/gnu/gcc/gcc-11.2.0/gcc-11.2.0.tar.xz
+wget https://ftp.gnu.org/gnu/glibc/glibc-2.34.tar.xz
+wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.10.70.tar.xz
+
+tar axf binutils-2.37.tar.xz
+tar axf gcc-11.2.0.tar.xz
+tar axf glibc-2.34.tar.xz
+tar axf linux-5.10.70.tar.xz
+
+mv binutils-2.37 src-binu
+mv gcc-11.2.0 src-gcc
+mv glibc-2.34 src-glibc
+mv linux-5.10.70 src-linux
+
+mkdir -p bld-hdr bld-binu bld-gcc bld-glibc
+mkdir -p ${CROSS_SYSROOT}/usr/include
+
+#
+# Install kernel and glibc headers
+#
+
+cd src-linux
+make headers_install ARCH=${LINUX_ARCH} INSTALL_HDR_PATH=${CROSS_SYSROOT}/usr
+cd ..
+
+cd bld-hdr
+../src-glibc/configure --prefix=/usr --host=${TARGET}
+make install-headers DESTDIR=${CROSS_SYSROOT}
+touch ${CROSS_SYSROOT}/usr/include/gnu/stubs.h
+cd ..
+
+#
+# Build binutils
+#
+
+cd bld-binu
+../src-binu/configure --disable-werror \
+ --prefix=${TOOLCHAIN_INSTALL} --with-sysroot --target=${TARGET}
+make -j${J}
+make install
+cd ..
+
+#
+# Build gcc, without shared libraries, because we do not yet
+# have a shared libc against which to link.
+#
+
+cd bld-gcc
+../src-gcc/configure --disable-werror --disable-shared \
+ --prefix=${TOOLCHAIN_INSTALL} --with-sysroot --target=${TARGET} \
+ --enable-languages=c --disable-libssp --disable-libsanitizer \
+ --disable-libatomic --disable-libgomp --disable-libquadmath
+make -j${J}
+make install
+cd ..
+
+#
+# Build glibc
+# There are a few random things that use c++ but we didn't build that
+# cross-compiler. We can get away without them. Disable CXX so that
+# glibc doesn't try to use the host c++ compiler.
+#
+
+cd bld-glibc
+CXX=false ../src-glibc/configure --prefix=/usr --host=${TARGET}
+make -j${j}
+make install DESTDIR=${CROSS_SYSROOT}
+cd ..
diff --git a/tests/docker/dockerfiles/debian-toolchain.docker b/tests/docker/dockerfiles/debian-toolchain.docker
new file mode 100644
index 0000000000..738d808aa6
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-toolchain.docker
@@ -0,0 +1,36 @@
+#
+# Docker toolchain cross-compiler
+#
+# This dockerfile is used for building a cross-compiler toolchain.
+# The script for building the toolchain is supplied via extra-files.
+#
+FROM qemu/debian10
+
+# Install build utilities for building gcc and glibc.
+# ??? The build-dep isn't working, missing a number of
+# minimal build dependiencies, e.g. libmpc.
+
+RUN apt update && \
+ DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
+ DEBIAN_FRONTEND=noninteractive eatmydata \
+ apt install -y --no-install-recommends \
+ bison \
+ flex \
+ gawk \
+ libmpc-dev \
+ libmpfr-dev \
+ rsync \
+ texinfo \
+ wget && \
+ DEBIAN_FRONTEND=noninteractive eatmydata \
+ apt build-dep -yy --arch-only gcc glibc
+
+ADD build-toolchain.sh /root/build-toolchain.sh
+
+RUN cd /root && ./build-toolchain.sh
+
+# Throw away the extra toolchain build deps, the downloaded source,
+# and the build trees by restoring the original debian10 image,
+# then copying the built toolchain from stage 0.
+FROM qemu/debian10
+COPY --from=0 /usr/local /usr/local
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 05/28] tests/docker: Add debian-nios2-cross image
2021-10-26 10:22 ` [PATCH v1 05/28] tests/docker: Add debian-nios2-cross image Alex Bennée
@ 2021-10-26 15:01 ` Philippe Mathieu-Daudé
2021-10-26 20:49 ` Willian Rampazzo
1 sibling, 0 replies; 61+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-10-26 15:01 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, Marek Vasut, Thomas Huth, berrange, pbonzini, Chris Wulff,
Richard Henderson, aaron, robhenry, Willian Rampazzo,
mahmoudabdalghany, minyihh, cota, Wainer dos Santos Moschetta,
stefanha, crosa, kuhn.chenqun, ma.mandourr, aurelien
On 10/26/21 12:22, Alex Bennée wrote:
> From: Richard Henderson <richard.henderson@linaro.org>
>
> Build the entire cross tool chain from source.
> For this reason, default to caching.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> [AJB: honour NOUSER in cached fetch and build, update MAINTAINERS]
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20211014224435.2539547-6-richard.henderson@linaro.org>
> ---
> MAINTAINERS | 1 +
> tests/docker/Makefile.include | 25 ++++++
> .../debian-nios2-cross.d/build-toolchain.sh | 87 +++++++++++++++++++
> .../dockerfiles/debian-toolchain.docker | 36 ++++++++
> 4 files changed, 149 insertions(+)
> create mode 100755 tests/docker/dockerfiles/debian-nios2-cross.d/build-toolchain.sh
> create mode 100644 tests/docker/dockerfiles/debian-toolchain.docker
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [PATCH v1 05/28] tests/docker: Add debian-nios2-cross image
2021-10-26 10:22 ` [PATCH v1 05/28] tests/docker: Add debian-nios2-cross image Alex Bennée
2021-10-26 15:01 ` Philippe Mathieu-Daudé
@ 2021-10-26 20:49 ` Willian Rampazzo
1 sibling, 0 replies; 61+ messages in thread
From: Willian Rampazzo @ 2021-10-26 20:49 UTC (permalink / raw)
To: Alex Bennée
Cc: Fam Zheng, Marek Vasut, Thomas Huth, minyihh, Daniel Berrange,
kuhn.chenqun, Chris Wulff, Richard Henderson, qemu-devel,
robhenry, Philippe Mathieu-Daudé,
mahmoudabdalghany, aaron, cota, Wainer dos Santos Moschetta,
Stefan Hajnoczi, Cleber Rosa Junior, Paolo Bonzini,
Mahmoud Mandour, Aurelien Jarno
On Tue, Oct 26, 2021 at 7:22 AM Alex Bennée <alex.bennee@linaro.org> wrote:
>
> From: Richard Henderson <richard.henderson@linaro.org>
>
> Build the entire cross tool chain from source.
> For this reason, default to caching.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> [AJB: honour NOUSER in cached fetch and build, update MAINTAINERS]
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20211014224435.2539547-6-richard.henderson@linaro.org>
> ---
> MAINTAINERS | 1 +
> tests/docker/Makefile.include | 25 ++++++
> .../debian-nios2-cross.d/build-toolchain.sh | 87 +++++++++++++++++++
> .../dockerfiles/debian-toolchain.docker | 36 ++++++++
> 4 files changed, 149 insertions(+)
> create mode 100755 tests/docker/dockerfiles/debian-nios2-cross.d/build-toolchain.sh
> create mode 100644 tests/docker/dockerfiles/debian-toolchain.docker
>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 06/28] tests/docker: Add debian-microblaze-cross image
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (4 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 05/28] tests/docker: Add debian-nios2-cross image Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 15:02 ` Philippe Mathieu-Daudé
2021-10-26 20:50 ` Willian Rampazzo
2021-10-26 10:22 ` [PATCH v1 07/28] tests/tcg: Enable container_cross_cc for microblaze Alex Bennée
` (21 subsequent siblings)
27 siblings, 2 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, Thomas Huth, minyihh, berrange, Edgar E. Iglesias,
kuhn.chenqun, Richard Henderson, f4bug, robhenry,
Willian Rampazzo, mahmoudabdalghany, aaron, cota,
Wainer dos Santos Moschetta, stefanha, crosa, pbonzini,
ma.mandourr, Alex Bennée, aurelien
From: Richard Henderson <richard.henderson@linaro.org>
Build the entire cross tool chain from source.
For this reason, default to caching.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
[AJB: Update MAINTAINERS]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211014224435.2539547-7-richard.henderson@linaro.org>
---
MAINTAINERS | 1 +
tests/docker/Makefile.include | 6 ++
.../build-toolchain.sh | 88 +++++++++++++++++++
3 files changed, 95 insertions(+)
create mode 100755 tests/docker/dockerfiles/debian-microblaze-cross.d/build-toolchain.sh
diff --git a/MAINTAINERS b/MAINTAINERS
index aa515bd313..108fcb799b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -221,6 +221,7 @@ S: Maintained
F: target/microblaze/
F: hw/microblaze/
F: disas/microblaze.c
+F: tests/docker/dockerfiles/debian-microblaze-cross.d/build-toolchain.sh
MIPS TCG CPUs
M: Philippe Mathieu-Daudé <f4bug@amsat.org>
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 24ed5c1f1d..e943ac33b0 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -167,6 +167,10 @@ debian-toolchain-run = \
"PREPARE", $1))
debian-toolchain = $(call debian-toolchain-run,$(patsubst docker-image-%,%,$1))
+docker-image-debian-microblaze-cross: $(DOCKER_FILES_DIR)/debian-toolchain.docker \
+ $(DOCKER_FILES_DIR)/debian-microblaze-cross.d/build-toolchain.sh
+ $(call debian-toolchain, $@)
+
docker-image-debian-nios2-cross: $(DOCKER_FILES_DIR)/debian-toolchain.docker \
$(DOCKER_FILES_DIR)/debian-nios2-cross.d/build-toolchain.sh
$(call debian-toolchain, $@)
@@ -176,6 +180,7 @@ docker-image-debian-tricore-cross: docker-image-debian10
docker-image-debian-all-test-cross: docker-image-debian10
docker-image-debian-arm64-test-cross: docker-image-debian11
docker-image-debian-hexagon-cross: docker-image-debian10
+docker-image-debian-microblaze-cross: docker-image-debian10
docker-image-debian-nios2-cross: docker-image-debian10
docker-image-debian-powerpc-test-cross: docker-image-debian11
@@ -185,6 +190,7 @@ DOCKER_PARTIAL_IMAGES += debian-arm64-test-cross
DOCKER_PARTIAL_IMAGES += debian-powerpc-test-cross
DOCKER_PARTIAL_IMAGES += debian-hppa-cross
DOCKER_PARTIAL_IMAGES += debian-m68k-cross debian-mips64-cross
+DOCKER_PARTIAL_IMAGES += debian-microblaze-cross
DOCKER_PARTIAL_IMAGES += debian-nios2-cross
DOCKER_PARTIAL_IMAGES += debian-sh4-cross debian-sparc64-cross
DOCKER_PARTIAL_IMAGES += debian-tricore-cross
diff --git a/tests/docker/dockerfiles/debian-microblaze-cross.d/build-toolchain.sh b/tests/docker/dockerfiles/debian-microblaze-cross.d/build-toolchain.sh
new file mode 100755
index 0000000000..23ec0aa9a7
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-microblaze-cross.d/build-toolchain.sh
@@ -0,0 +1,88 @@
+#!/bin/bash
+
+set -e
+
+TARGET=microblaze-linux-musl
+LINUX_ARCH=microblaze
+
+J=$(expr $(nproc) / 2)
+TOOLCHAIN_INSTALL=/usr/local
+TOOLCHAIN_BIN=${TOOLCHAIN_INSTALL}/bin
+CROSS_SYSROOT=${TOOLCHAIN_INSTALL}/$TARGET/sys-root
+
+export PATH=${TOOLCHAIN_BIN}:$PATH
+
+#
+# Grab all of the source for the toolchain bootstrap.
+#
+
+wget https://ftp.gnu.org/gnu/binutils/binutils-2.37.tar.xz
+wget https://ftp.gnu.org/gnu/gcc/gcc-11.2.0/gcc-11.2.0.tar.xz
+wget https://www.musl-libc.org/releases/musl-1.2.2.tar.gz
+wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.10.70.tar.xz
+
+tar axf binutils-2.37.tar.xz
+tar axf gcc-11.2.0.tar.xz
+tar axf musl-1.2.2.tar.gz
+tar axf linux-5.10.70.tar.xz
+
+mv binutils-2.37 src-binu
+mv gcc-11.2.0 src-gcc
+mv musl-1.2.2 src-musl
+mv linux-5.10.70 src-linux
+
+mkdir -p bld-hdr bld-binu bld-gcc bld-musl
+mkdir -p ${CROSS_SYSROOT}/usr/include
+
+#
+# Install kernel headers
+#
+
+cd src-linux
+make headers_install ARCH=${LINUX_ARCH} INSTALL_HDR_PATH=${CROSS_SYSROOT}/usr
+cd ..
+
+#
+# Build binutils
+#
+
+cd bld-binu
+../src-binu/configure --disable-werror \
+ --prefix=${TOOLCHAIN_INSTALL} --with-sysroot --target=${TARGET}
+make -j${J}
+make install
+cd ..
+
+#
+# Build gcc, just the compiler so far.
+#
+
+cd bld-gcc
+../src-gcc/configure --disable-werror --disable-shared \
+ --prefix=${TOOLCHAIN_INSTALL} --with-sysroot --target=${TARGET} \
+ --enable-languages=c --disable-libssp --disable-libsanitizer \
+ --disable-libatomic --disable-libgomp --disable-libquadmath
+make -j${J} all-gcc
+make install-gcc
+cd ..
+
+#
+# Build musl.
+# We won't go through the extra step of building shared libraries
+# because we don't actually use them in QEMU docker testing.
+#
+
+cd bld-musl
+../src-musl/configure --prefix=/usr --host=${TARGET} --disable-shared
+make -j${j}
+make install DESTDIR=${CROSS_SYSROOT}
+cd ..
+
+#
+# Go back and build the compiler runtime
+#
+
+cd bld-gcc
+make -j${j}
+make install
+cd ..
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 06/28] tests/docker: Add debian-microblaze-cross image
2021-10-26 10:22 ` [PATCH v1 06/28] tests/docker: Add debian-microblaze-cross image Alex Bennée
@ 2021-10-26 15:02 ` Philippe Mathieu-Daudé
2021-10-26 20:50 ` Willian Rampazzo
1 sibling, 0 replies; 61+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-10-26 15:02 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, Thomas Huth, berrange, pbonzini, crosa, Richard Henderson,
aaron, robhenry, Willian Rampazzo, mahmoudabdalghany, minyihh,
cota, Wainer dos Santos Moschetta, stefanha, kuhn.chenqun,
Edgar E. Iglesias, ma.mandourr, aurelien
On 10/26/21 12:22, Alex Bennée wrote:
> From: Richard Henderson <richard.henderson@linaro.org>
>
> Build the entire cross tool chain from source.
> For this reason, default to caching.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> [AJB: Update MAINTAINERS]
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20211014224435.2539547-7-richard.henderson@linaro.org>
> ---
> MAINTAINERS | 1 +
> tests/docker/Makefile.include | 6 ++
> .../build-toolchain.sh | 88 +++++++++++++++++++
> 3 files changed, 95 insertions(+)
> create mode 100755 tests/docker/dockerfiles/debian-microblaze-cross.d/build-toolchain.sh
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [PATCH v1 06/28] tests/docker: Add debian-microblaze-cross image
2021-10-26 10:22 ` [PATCH v1 06/28] tests/docker: Add debian-microblaze-cross image Alex Bennée
2021-10-26 15:02 ` Philippe Mathieu-Daudé
@ 2021-10-26 20:50 ` Willian Rampazzo
1 sibling, 0 replies; 61+ messages in thread
From: Willian Rampazzo @ 2021-10-26 20:50 UTC (permalink / raw)
To: Alex Bennée
Cc: Fam Zheng, Thomas Huth, minyihh, Daniel Berrange,
Edgar E. Iglesias, kuhn.chenqun, Richard Henderson, qemu-devel,
robhenry, Philippe Mathieu-Daudé,
mahmoudabdalghany, aaron, cota, Wainer dos Santos Moschetta,
Stefan Hajnoczi, Cleber Rosa Junior, Paolo Bonzini,
Mahmoud Mandour, Aurelien Jarno
On Tue, Oct 26, 2021 at 7:22 AM Alex Bennée <alex.bennee@linaro.org> wrote:
>
> From: Richard Henderson <richard.henderson@linaro.org>
>
> Build the entire cross tool chain from source.
> For this reason, default to caching.
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> [AJB: Update MAINTAINERS]
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20211014224435.2539547-7-richard.henderson@linaro.org>
> ---
> MAINTAINERS | 1 +
> tests/docker/Makefile.include | 6 ++
> .../build-toolchain.sh | 88 +++++++++++++++++++
> 3 files changed, 95 insertions(+)
> create mode 100755 tests/docker/dockerfiles/debian-microblaze-cross.d/build-toolchain.sh
>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 07/28] tests/tcg: Enable container_cross_cc for microblaze
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (5 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 06/28] tests/docker: Add debian-microblaze-cross image Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 15:03 ` Philippe Mathieu-Daudé
2021-10-26 10:22 ` [PATCH v1 08/28] tests/tcg: Fix some targets default cross compiler path Alex Bennée
` (20 subsequent siblings)
27 siblings, 1 reply; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, Richard Henderson, f4bug,
robhenry, mahmoudabdalghany, aaron, cota, stefanha, crosa,
pbonzini, ma.mandourr, Alex Bennée, aurelien
From: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211014224435.2539547-8-richard.henderson@linaro.org>
---
tests/tcg/configure.sh | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index 1f985ccfc0..149eeb15ff 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -55,6 +55,7 @@ fi
: ${cross_cc_i386="i686-linux-gnu-gcc"}
: ${cross_cc_cflags_i386="-m32"}
: ${cross_cc_m68k="m68k-linux-gnu-gcc"}
+: $(cross_cc_microblaze="microblaze-linux-musl-gcc")
: $(cross_cc_mips64el="mips64el-linux-gnuabi64-gcc")
: $(cross_cc_mips64="mips64-linux-gnuabi64-gcc")
: $(cross_cc_mipsel="mipsel-linux-gnu-gcc")
@@ -133,6 +134,11 @@ for target in $target_list; do
container_image=debian-m68k-cross
container_cross_cc=m68k-linux-gnu-gcc
;;
+ microblaze-*)
+ container_hosts=x86_64
+ container_image=debian-microblaze-cross
+ container_cross_cc=microblaze-linux-musl-gcc
+ ;;
mips64el-*)
container_hosts=x86_64
container_image=debian-mips64el-cross
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 07/28] tests/tcg: Enable container_cross_cc for microblaze
2021-10-26 10:22 ` [PATCH v1 07/28] tests/tcg: Enable container_cross_cc for microblaze Alex Bennée
@ 2021-10-26 15:03 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 61+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-10-26 15:03 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, pbonzini, Richard Henderson, aaron, robhenry,
mahmoudabdalghany, minyihh, cota, stefanha, crosa, kuhn.chenqun,
ma.mandourr, aurelien
On 10/26/21 12:22, Alex Bennée wrote:
> From: Richard Henderson <richard.henderson@linaro.org>
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20211014224435.2539547-8-richard.henderson@linaro.org>
> ---
> tests/tcg/configure.sh | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
> index 1f985ccfc0..149eeb15ff 100755
> --- a/tests/tcg/configure.sh
> +++ b/tests/tcg/configure.sh
> @@ -55,6 +55,7 @@ fi
> : ${cross_cc_i386="i686-linux-gnu-gcc"}
> : ${cross_cc_cflags_i386="-m32"}
> : ${cross_cc_m68k="m68k-linux-gnu-gcc"}
> +: $(cross_cc_microblaze="microblaze-linux-musl-gcc")
${ }
Otherwise,
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 08/28] tests/tcg: Fix some targets default cross compiler path
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (6 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 07/28] tests/tcg: Enable container_cross_cc for microblaze Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 10:41 ` Philippe Mathieu-Daudé
2021-10-26 10:22 ` [PATCH v1 09/28] tests/docker: split PARTIAL into PARTIAL and VIRTUAL images Alex Bennée
` (19 subsequent siblings)
27 siblings, 1 reply; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
mahmoudabdalghany, aaron, cota, stefanha, crosa, pbonzini,
ma.mandourr, Alex Bennée, aurelien
From: Philippe Mathieu-Daudé <f4bug@amsat.org>
We do not want a shell command substitution, but a parameter
substitution (with assignment). Replace $() -> ${}, otherwise
the expanded command return an empty string and the $cross_cc
variable is not set.
Fixes: 634ef789f8e ("tests/tcg: add more default compilers to configure.sh")
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211023164329.328137-1-f4bug@amsat.org>
---
tests/tcg/configure.sh | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index 149eeb15ff..0d3ac839ee 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -46,7 +46,7 @@ fi
: ${cross_cc_aarch64="aarch64-linux-gnu-gcc"}
: ${cross_cc_aarch64_be="$cross_cc_aarch64"}
: ${cross_cc_cflags_aarch64_be="-mbig-endian"}
-: $(cross_cc_alpha="alpha-linux-gnu-gcc")
+: ${cross_cc_alpha="alpha-linux-gnu-gcc"}
: ${cross_cc_arm="arm-linux-gnueabihf-gcc"}
: ${cross_cc_cflags_armeb="-mbig-endian"}
: ${cross_cc_hexagon="hexagon-unknown-linux-musl-clang"}
@@ -56,17 +56,17 @@ fi
: ${cross_cc_cflags_i386="-m32"}
: ${cross_cc_m68k="m68k-linux-gnu-gcc"}
: $(cross_cc_microblaze="microblaze-linux-musl-gcc")
-: $(cross_cc_mips64el="mips64el-linux-gnuabi64-gcc")
-: $(cross_cc_mips64="mips64-linux-gnuabi64-gcc")
-: $(cross_cc_mipsel="mipsel-linux-gnu-gcc")
-: $(cross_cc_mips="mips-linux-gnu-gcc")
+: ${cross_cc_mips64el="mips64el-linux-gnuabi64-gcc"}
+: ${cross_cc_mips64="mips64-linux-gnuabi64-gcc"}
+: ${cross_cc_mipsel="mipsel-linux-gnu-gcc"}
+: ${cross_cc_mips="mips-linux-gnu-gcc"}
: ${cross_cc_ppc="powerpc-linux-gnu-gcc"}
: ${cross_cc_cflags_ppc="-m32"}
: ${cross_cc_ppc64="powerpc64-linux-gnu-gcc"}
: ${cross_cc_ppc64le="powerpc64le-linux-gnu-gcc"}
-: $(cross_cc_riscv64="riscv64-linux-gnu-gcc")
+: ${cross_cc_riscv64="riscv64-linux-gnu-gcc"}
: ${cross_cc_s390x="s390x-linux-gnu-gcc"}
-: $(cross_cc_sh4="sh4-linux-gnu-gcc")
+: ${cross_cc_sh4="sh4-linux-gnu-gcc"}
: ${cross_cc_cflags_sparc="-m32 -mv8plus -mcpu=ultrasparc"}
: ${cross_cc_sparc64="sparc64-linux-gnu-gcc"}
: ${cross_cc_cflags_sparc64="-m64 -mcpu=ultrasparc"}
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 08/28] tests/tcg: Fix some targets default cross compiler path
2021-10-26 10:22 ` [PATCH v1 08/28] tests/tcg: Fix some targets default cross compiler path Alex Bennée
@ 2021-10-26 10:41 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 61+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-10-26 10:41 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, pbonzini, aaron, robhenry, mahmoudabdalghany,
minyihh, cota, stefanha, crosa, kuhn.chenqun, ma.mandourr,
aurelien
On 10/26/21 12:22, Alex Bennée wrote:
> From: Philippe Mathieu-Daudé <f4bug@amsat.org>
>
> We do not want a shell command substitution, but a parameter
> substitution (with assignment). Replace $() -> ${}, otherwise
> the expanded command return an empty string and the $cross_cc
> variable is not set.
>
> Fixes: 634ef789f8e ("tests/tcg: add more default compilers to configure.sh")
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20211023164329.328137-1-f4bug@amsat.org>
> ---
> tests/tcg/configure.sh | 14 +++++++-------
> 1 file changed, 7 insertions(+), 7 deletions(-)
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 09/28] tests/docker: split PARTIAL into PARTIAL and VIRTUAL images
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (7 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 08/28] tests/tcg: Fix some targets default cross compiler path Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 20:01 ` Richard Henderson
2021-10-26 20:52 ` Willian Rampazzo
2021-10-26 10:22 ` [PATCH v1 10/28] tests/docker: allow non-unique userid Alex Bennée
` (18 subsequent siblings)
27 siblings, 2 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, Thomas Huth, minyihh, berrange, kuhn.chenqun, f4bug,
robhenry, Willian Rampazzo, mahmoudabdalghany, aaron, cota,
Wainer dos Santos Moschetta, stefanha, crosa, pbonzini,
ma.mandourr, Alex Bennée, aurelien
This is mostly to ensure we don't include the toolchain and bootstrap
builds in DOCKER_IMAGES which is useful when verifying all images
still build.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/docker/Makefile.include | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index e943ac33b0..5bbbaceed1 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -11,8 +11,10 @@ HOST_ARCH = $(if $(ARCH),$(ARCH),$(shell uname -m))
DOCKER_SUFFIX := .docker
DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
# we don't run tests on intermediate images (used as base by another image)
-DOCKER_PARTIAL_IMAGES := debian10 debian11 debian-bootstrap empty
-DOCKER_IMAGES := $(sort $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker))))
+DOCKER_PARTIAL_IMAGES := debian10 debian11
+# we don't directly build virtual images (they are used to build other images)
+DOCKER_VIRTUAL_IMAGES := debian-bootstrap debian-toolchain empty
+DOCKER_IMAGES := $(sort $(filter-out $(DOCKER_VIRTUAL_IMAGES), $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))))
DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
# Use a global constant ccache directory to speed up repetitive builds
DOCKER_CCACHE_DIR := $$HOME/.cache/qemu-docker-ccache
@@ -204,7 +206,7 @@ DOCKER_PARTIAL_IMAGES += fedora-cris-cross
# packages.
# Expand all the pre-requistes for each docker image and test combination
-$(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES)), \
+$(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES) $(DOCKER_VIRTUAL_IMAGES),$(DOCKER_IMAGES)), \
$(foreach t,$(DOCKER_TESTS), \
$(eval .PHONY: docker-$t@$i) \
$(eval docker-$t@$i: docker-image-$i docker-run-$t@$i) \
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 09/28] tests/docker: split PARTIAL into PARTIAL and VIRTUAL images
2021-10-26 10:22 ` [PATCH v1 09/28] tests/docker: split PARTIAL into PARTIAL and VIRTUAL images Alex Bennée
@ 2021-10-26 20:01 ` Richard Henderson
2021-10-26 20:52 ` Willian Rampazzo
1 sibling, 0 replies; 61+ messages in thread
From: Richard Henderson @ 2021-10-26 20:01 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, Thomas Huth, berrange, pbonzini, f4bug, robhenry, aaron,
Willian Rampazzo, mahmoudabdalghany, minyihh, cota,
Wainer dos Santos Moschetta, stefanha, crosa, kuhn.chenqun,
ma.mandourr, aurelien
On 10/26/21 3:22 AM, Alex Bennée wrote:
> This is mostly to ensure we don't include the toolchain and bootstrap
> builds in DOCKER_IMAGES which is useful when verifying all images
> still build.
>
> Signed-off-by: Alex Bennée<alex.bennee@linaro.org>
> ---
> tests/docker/Makefile.include | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
r~
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [PATCH v1 09/28] tests/docker: split PARTIAL into PARTIAL and VIRTUAL images
2021-10-26 10:22 ` [PATCH v1 09/28] tests/docker: split PARTIAL into PARTIAL and VIRTUAL images Alex Bennée
2021-10-26 20:01 ` Richard Henderson
@ 2021-10-26 20:52 ` Willian Rampazzo
1 sibling, 0 replies; 61+ messages in thread
From: Willian Rampazzo @ 2021-10-26 20:52 UTC (permalink / raw)
To: Alex Bennée
Cc: Fam Zheng, Thomas Huth, minyihh, Daniel Berrange, kuhn.chenqun,
qemu-devel, robhenry, Philippe Mathieu-Daudé,
mahmoudabdalghany, aaron, cota, Wainer dos Santos Moschetta,
Stefan Hajnoczi, Cleber Rosa Junior, Paolo Bonzini,
Mahmoud Mandour, Aurelien Jarno
On Tue, Oct 26, 2021 at 7:22 AM Alex Bennée <alex.bennee@linaro.org> wrote:
>
> This is mostly to ensure we don't include the toolchain and bootstrap
> builds in DOCKER_IMAGES which is useful when verifying all images
> still build.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/docker/Makefile.include | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 10/28] tests/docker: allow non-unique userid
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (8 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 09/28] tests/docker: split PARTIAL into PARTIAL and VIRTUAL images Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 20:42 ` Richard Henderson
2021-10-26 10:22 ` [PATCH v1 11/28] tests/tcg: enable debian-nios2-cross for test building Alex Bennée
` (17 subsequent siblings)
27 siblings, 1 reply; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, Thomas Huth, minyihh, berrange, kuhn.chenqun, f4bug,
robhenry, Willian Rampazzo, mahmoudabdalghany, aaron, cota,
Wainer dos Santos Moschetta, stefanha, crosa, pbonzini,
ma.mandourr, Alex Bennée, aurelien
This is a bit of a band-aid against hand-built images that have been
accidentally polluted by a user build. All images pulled from the
registry shouldn't have the user defined.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/docker/docker.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 78dd13171e..5eadf0bb7b 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -336,7 +336,7 @@ def build_image(self, tag, docker_dir, dockerfile,
uid = os.getuid()
uname = getpwuid(uid).pw_name
tmp_df.write("\n")
- tmp_df.write("RUN id %s 2>/dev/null || useradd -u %d -U %s" %
+ tmp_df.write("RUN id %s 2>/dev/null || useradd -o -u %d -U %s" %
(uname, uid, uname))
tmp_df.write("\n")
@@ -590,7 +590,7 @@ def run(self, args, argv):
uid = os.getuid()
uname = getpwuid(uid).pw_name
df.write("\n")
- df.write("RUN id %s 2>/dev/null || useradd -u %d -U %s" %
+ df.write("RUN id %s 2>/dev/null || useradd -o -u %d -U %s" %
(uname, uid, uname))
df_bytes = BytesIO(bytes(df.getvalue(), "UTF-8"))
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 10/28] tests/docker: allow non-unique userid
2021-10-26 10:22 ` [PATCH v1 10/28] tests/docker: allow non-unique userid Alex Bennée
@ 2021-10-26 20:42 ` Richard Henderson
2021-11-01 15:10 ` Alex Bennée
0 siblings, 1 reply; 61+ messages in thread
From: Richard Henderson @ 2021-10-26 20:42 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, Thomas Huth, berrange, pbonzini, f4bug, robhenry, aaron,
Willian Rampazzo, mahmoudabdalghany, minyihh, cota,
Wainer dos Santos Moschetta, stefanha, crosa, kuhn.chenqun,
ma.mandourr, aurelien
On 10/26/21 3:22 AM, Alex Bennée wrote:
> This is a bit of a band-aid against hand-built images that have been
> accidentally polluted by a user build. All images pulled from the
> registry shouldn't have the user defined.
I believe that I've fixed the images currently uploaded.
I think we ought to totally rewrite now we add users to images. I don't think we can
reasonably layer one image upon another without getting the user added at an intermediate
image. At present the only way to do this is NOCACHE=1 NOUSER=1, so that all of the
intermediate images are also rebuilt, and also force nouser.
Does this patch really help in the meantime?
r~
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [PATCH v1 10/28] tests/docker: allow non-unique userid
2021-10-26 20:42 ` Richard Henderson
@ 2021-11-01 15:10 ` Alex Bennée
0 siblings, 0 replies; 61+ messages in thread
From: Alex Bennée @ 2021-11-01 15:10 UTC (permalink / raw)
To: Richard Henderson
Cc: fam, Thomas Huth, berrange, pbonzini, aaron, qemu-devel,
robhenry, f4bug, Willian Rampazzo, mahmoudabdalghany, minyihh,
cota, Wainer dos Santos Moschetta, stefanha, crosa, kuhn.chenqun,
ma.mandourr, aurelien
Richard Henderson <richard.henderson@linaro.org> writes:
> On 10/26/21 3:22 AM, Alex Bennée wrote:
>> This is a bit of a band-aid against hand-built images that have been
>> accidentally polluted by a user build. All images pulled from the
>> registry shouldn't have the user defined.
>
> I believe that I've fixed the images currently uploaded.
>
> I think we ought to totally rewrite now we add users to images. I
> don't think we can reasonably layer one image upon another without
> getting the user added at an intermediate image.
We could just force NOUSER=1 on the base images. The only images users
will actually want with their IDs are the final ones anyway. I'll drop
this patch from the upcoming PR and see if I can rework stuff.
> At present the only
> way to do this is NOCACHE=1 NOUSER=1, so that all of the intermediate
> images are also rebuilt, and also force nouser.
>
> Does this patch really help in the meantime?
>
>
> r~
--
Alex Bennée
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 11/28] tests/tcg: enable debian-nios2-cross for test building
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (9 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 10/28] tests/docker: allow non-unique userid Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 15:04 ` Philippe Mathieu-Daudé
2021-10-26 20:02 ` Richard Henderson
2021-10-26 10:22 ` [PATCH v1 12/28] ebpf: really include it only in system emulators Alex Bennée
` (16 subsequent siblings)
27 siblings, 2 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, Edgar E. Iglesias, f4bug,
robhenry, mahmoudabdalghany, aaron, cota, stefanha, crosa,
pbonzini, ma.mandourr, Alex Bennée, aurelien
Now we have a nios2 test image we can start using it to build tests.
However signal handling in nios2 is still broken so we disable the
signals and linux-test tests that trigger the bug.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
MAINTAINERS | 1 +
tests/tcg/configure.sh | 6 ++++++
tests/tcg/nios2/Makefile.target | 11 +++++++++++
3 files changed, 18 insertions(+)
create mode 100644 tests/tcg/nios2/Makefile.target
diff --git a/MAINTAINERS b/MAINTAINERS
index 108fcb799b..87d67f19ce 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -222,6 +222,7 @@ F: target/microblaze/
F: hw/microblaze/
F: disas/microblaze.c
F: tests/docker/dockerfiles/debian-microblaze-cross.d/build-toolchain.sh
+F: tests/tcg/nios2/Makefile.target
MIPS TCG CPUs
M: Philippe Mathieu-Daudé <f4bug@amsat.org>
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index 0d3ac839ee..34f6dbbc20 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -60,6 +60,7 @@ fi
: ${cross_cc_mips64="mips64-linux-gnuabi64-gcc"}
: ${cross_cc_mipsel="mipsel-linux-gnu-gcc"}
: ${cross_cc_mips="mips-linux-gnu-gcc"}
+: ${cross_cc_nios2="nios2-linux-gnu-gcc"}
: ${cross_cc_ppc="powerpc-linux-gnu-gcc"}
: ${cross_cc_cflags_ppc="-m32"}
: ${cross_cc_ppc64="powerpc64-linux-gnu-gcc"}
@@ -159,6 +160,11 @@ for target in $target_list; do
container_image=debian-mips-cross
container_cross_cc=mips-linux-gnu-gcc
;;
+ nios2-*)
+ container_hosts=x86_64
+ container_image=debian-nios2-cross
+ container_cross_cc=nios2-linux-gnu-gcc
+ ;;
ppc-*|ppc64abi32-*)
container_hosts=x86_64
container_image=debian-powerpc-test-cross
diff --git a/tests/tcg/nios2/Makefile.target b/tests/tcg/nios2/Makefile.target
new file mode 100644
index 0000000000..b38e2352b7
--- /dev/null
+++ b/tests/tcg/nios2/Makefile.target
@@ -0,0 +1,11 @@
+# nios2 specific test tweaks
+
+# Currently nios2 signal handling is broken
+run-signals: signals
+ $(call skip-test, $<, "BROKEN")
+run-plugin-signals-with-%:
+ $(call skip-test, $<, "BROKEN")
+run-linux-test: linux-test
+ $(call skip-test, $<, "BROKEN")
+run-plugin-linux-test-with-%:
+ $(call skip-test, $<, "BROKEN")
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 11/28] tests/tcg: enable debian-nios2-cross for test building
2021-10-26 10:22 ` [PATCH v1 11/28] tests/tcg: enable debian-nios2-cross for test building Alex Bennée
@ 2021-10-26 15:04 ` Philippe Mathieu-Daudé
2021-10-26 20:02 ` Richard Henderson
1 sibling, 0 replies; 61+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-10-26 15:04 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, pbonzini, crosa, aaron, robhenry,
mahmoudabdalghany, minyihh, cota, stefanha, Edgar E. Iglesias,
kuhn.chenqun, ma.mandourr, aurelien
On 10/26/21 12:22, Alex Bennée wrote:
> Now we have a nios2 test image we can start using it to build tests.
> However signal handling in nios2 is still broken so we disable the
> signals and linux-test tests that trigger the bug.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> MAINTAINERS | 1 +
> tests/tcg/configure.sh | 6 ++++++
> tests/tcg/nios2/Makefile.target | 11 +++++++++++
> 3 files changed, 18 insertions(+)
> create mode 100644 tests/tcg/nios2/Makefile.target
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [PATCH v1 11/28] tests/tcg: enable debian-nios2-cross for test building
2021-10-26 10:22 ` [PATCH v1 11/28] tests/tcg: enable debian-nios2-cross for test building Alex Bennée
2021-10-26 15:04 ` Philippe Mathieu-Daudé
@ 2021-10-26 20:02 ` Richard Henderson
1 sibling, 0 replies; 61+ messages in thread
From: Richard Henderson @ 2021-10-26 20:02 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, pbonzini, crosa, f4bug, robhenry, aaron,
mahmoudabdalghany, minyihh, cota, stefanha, Edgar E. Iglesias,
kuhn.chenqun, ma.mandourr, aurelien
On 10/26/21 3:22 AM, Alex Bennée wrote:
> Now we have a nios2 test image we can start using it to build tests.
> However signal handling in nios2 is still broken so we disable the
> signals and linux-test tests that trigger the bug.
>
> Signed-off-by: Alex Bennée<alex.bennee@linaro.org>
> ---
> MAINTAINERS | 1 +
> tests/tcg/configure.sh | 6 ++++++
> tests/tcg/nios2/Makefile.target | 11 +++++++++++
> 3 files changed, 18 insertions(+)
> create mode 100644 tests/tcg/nios2/Makefile.target
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
r~
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 12/28] ebpf: really include it only in system emulators
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (10 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 11/28] tests/tcg: enable debian-nios2-cross for test building Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 20:03 ` Richard Henderson
2021-10-27 16:10 ` Warner Losh
2021-10-26 10:22 ` [PATCH v1 13/28] plugins/cache: freed heap-allocated mutexes Alex Bennée
` (15 subsequent siblings)
27 siblings, 2 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
mahmoudabdalghany, aaron, cota, stefanha, crosa, pbonzini,
ma.mandourr, Alex Bennée, aurelien
From: Paolo Bonzini <pbonzini@redhat.com>
eBPF libraries are being included in user emulators, which is useless and
also breaks --static compilation if a shared library for libbpf is
present in the system.
Reported-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211012162252.263933-1-pbonzini@redhat.com>
---
meson.build | 2 --
1 file changed, 2 deletions(-)
diff --git a/meson.build b/meson.build
index 2c5b53cbe2..bc520b579c 100644
--- a/meson.build
+++ b/meson.build
@@ -2610,8 +2610,6 @@ subdir('bsd-user')
subdir('linux-user')
subdir('ebpf')
-common_ss.add(libbpf)
-
specific_ss.add_all(when: 'CONFIG_BSD_USER', if_true: bsd_user_ss)
linux_user_ss.add(files('thunk.c'))
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 12/28] ebpf: really include it only in system emulators
2021-10-26 10:22 ` [PATCH v1 12/28] ebpf: really include it only in system emulators Alex Bennée
@ 2021-10-26 20:03 ` Richard Henderson
2021-10-27 16:10 ` Warner Losh
1 sibling, 0 replies; 61+ messages in thread
From: Richard Henderson @ 2021-10-26 20:03 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, pbonzini, f4bug, robhenry, aaron,
mahmoudabdalghany, minyihh, cota, stefanha, crosa, kuhn.chenqun,
ma.mandourr, aurelien
On 10/26/21 3:22 AM, Alex Bennée wrote:
> From: Paolo Bonzini<pbonzini@redhat.com>
>
> eBPF libraries are being included in user emulators, which is useless and
> also breaks --static compilation if a shared library for libbpf is
> present in the system.
>
> Reported-by: Alex Bennée<alex.bennee@linaro.org>
> Signed-off-by: Paolo Bonzini<pbonzini@redhat.com>
> Signed-off-by: Alex Bennée<alex.bennee@linaro.org>
> Message-Id:<20211012162252.263933-1-pbonzini@redhat.com>
> ---
> meson.build | 2 --
> 1 file changed, 2 deletions(-)
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
r~
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [PATCH v1 12/28] ebpf: really include it only in system emulators
2021-10-26 10:22 ` [PATCH v1 12/28] ebpf: really include it only in system emulators Alex Bennée
2021-10-26 20:03 ` Richard Henderson
@ 2021-10-27 16:10 ` Warner Losh
1 sibling, 0 replies; 61+ messages in thread
From: Warner Losh @ 2021-10-27 16:10 UTC (permalink / raw)
To: Alex Bennée
Cc: fam, berrange, pbonzini, aaron, qemu-devel, robhenry, f4bug,
mahmoudabdalghany, minyihh, cota, stefanha, crosa, kuhn.chenqun,
ma.mandourr, aurelien
> On Oct 26, 2021, at 4:22 AM, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> From: Paolo Bonzini <pbonzini@redhat.com>
>
> eBPF libraries are being included in user emulators, which is useless and
> also breaks --static compilation if a shared library for libbpf is
> present in the system.
>
> Reported-by: Alex Bennée <alex.bennee@linaro.org>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20211012162252.263933-1-pbonzini@redhat.com>
Reviewed-bt: Warner Losh <imp@bsdimp.com>
> ---
> meson.build | 2 --
> 1 file changed, 2 deletions(-)
>
> diff --git a/meson.build b/meson.build
> index 2c5b53cbe2..bc520b579c 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -2610,8 +2610,6 @@ subdir('bsd-user')
> subdir('linux-user')
> subdir('ebpf')
>
> -common_ss.add(libbpf)
> -
> specific_ss.add_all(when: 'CONFIG_BSD_USER', if_true: bsd_user_ss)
>
> linux_user_ss.add(files('thunk.c'))
> --
> 2.30.2
>
>
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 13/28] plugins/cache: freed heap-allocated mutexes
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (11 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 12/28] ebpf: really include it only in system emulators Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 15:06 ` Philippe Mathieu-Daudé
2021-10-26 10:22 ` [PATCH v1 14/28] plugins/cache: implement unified L2 cache emulation Alex Bennée
` (14 subsequent siblings)
27 siblings, 1 reply; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
mahmoudabdalghany, aaron, cota, stefanha, crosa, pbonzini,
ma.mandourr, Alexandre Iooss, Alex Bennée, aurelien
From: Mahmoud Mandour <ma.mandourr@gmail.com>
Signed-off-by: Mahmoud Mandour <ma.mandourr@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210810134844.166490-2-ma.mandourr@gmail.com>
---
contrib/plugins/cache.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/contrib/plugins/cache.c b/contrib/plugins/cache.c
index a1e03ca882..a255e26e25 100644
--- a/contrib/plugins/cache.c
+++ b/contrib/plugins/cache.c
@@ -614,6 +614,9 @@ static void plugin_exit(qemu_plugin_id_t id, void *p)
caches_free(dcaches);
caches_free(icaches);
+ g_free(dcache_locks);
+ g_free(icache_locks);
+
g_hash_table_destroy(miss_ht);
}
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 13/28] plugins/cache: freed heap-allocated mutexes
2021-10-26 10:22 ` [PATCH v1 13/28] plugins/cache: freed heap-allocated mutexes Alex Bennée
@ 2021-10-26 15:06 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 61+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-10-26 15:06 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, pbonzini, aaron, robhenry, mahmoudabdalghany,
minyihh, cota, stefanha, crosa, kuhn.chenqun, ma.mandourr,
Alexandre Iooss, aurelien
On 10/26/21 12:22, Alex Bennée wrote:
> From: Mahmoud Mandour <ma.mandourr@gmail.com>
>
> Signed-off-by: Mahmoud Mandour <ma.mandourr@gmail.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20210810134844.166490-2-ma.mandourr@gmail.com>
> ---
> contrib/plugins/cache.c | 3 +++
> 1 file changed, 3 insertions(+)
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 14/28] plugins/cache: implement unified L2 cache emulation
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (12 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 13/28] plugins/cache: freed heap-allocated mutexes Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 10:22 ` [PATCH v1 15/28] plugins/cache: split command line arguments into name and value Alex Bennée
` (13 subsequent siblings)
27 siblings, 0 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
mahmoudabdalghany, aaron, cota, stefanha, crosa, pbonzini,
ma.mandourr, Alexandre Iooss, Alex Bennée, aurelien
From: Mahmoud Mandour <ma.mandourr@gmail.com>
This adds an implementation of a simple L2 configuration, in which a
unified L2 cache (stores both blocks of instructions and data) is
maintained for each core separately, with no inter-core interaction
taken in account. The L2 cache is used as a backup for L1 and is only
accessed if the wanted block does not exist in L1.
In terms of multi-threaded user-space emulation, the same approximation
of L1 is done, a static number of caches is maintained, and each and
every memory access initiated by a thread will have to go through one of
the available caches.
An atomic increment is used to maintain the number of L2 misses per
instruction.
The default cache parameters of L2 caches is:
2MB cache size
16-way associativity
64-byte blocks
Signed-off-by: Mahmoud Mandour <ma.mandourr@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210810134844.166490-3-ma.mandourr@gmail.com>
---
contrib/plugins/cache.c | 256 +++++++++++++++++++++++++++-------------
1 file changed, 175 insertions(+), 81 deletions(-)
diff --git a/contrib/plugins/cache.c b/contrib/plugins/cache.c
index a255e26e25..908c967a09 100644
--- a/contrib/plugins/cache.c
+++ b/contrib/plugins/cache.c
@@ -82,8 +82,9 @@ typedef struct {
char *disas_str;
const char *symbol;
uint64_t addr;
- uint64_t dmisses;
- uint64_t imisses;
+ uint64_t l1_dmisses;
+ uint64_t l1_imisses;
+ uint64_t l2_misses;
} InsnData;
void (*update_hit)(Cache *cache, int set, int blk);
@@ -93,15 +94,20 @@ void (*metadata_init)(Cache *cache);
void (*metadata_destroy)(Cache *cache);
static int cores;
-static Cache **dcaches, **icaches;
+static Cache **l1_dcaches, **l1_icaches;
+static Cache **l2_ucaches;
-static GMutex *dcache_locks;
-static GMutex *icache_locks;
+static GMutex *l1_dcache_locks;
+static GMutex *l1_icache_locks;
+static GMutex *l2_ucache_locks;
-static uint64_t all_dmem_accesses;
-static uint64_t all_imem_accesses;
-static uint64_t all_imisses;
-static uint64_t all_dmisses;
+static uint64_t l1_dmem_accesses;
+static uint64_t l1_imem_accesses;
+static uint64_t l1_imisses;
+static uint64_t l1_dmisses;
+
+static uint64_t l2_mem_accesses;
+static uint64_t l2_misses;
static int pow_of_two(int num)
{
@@ -382,6 +388,7 @@ static void vcpu_mem_access(unsigned int vcpu_index, qemu_plugin_meminfo_t info,
struct qemu_plugin_hwaddr *hwaddr;
int cache_idx;
InsnData *insn;
+ bool hit_in_l1;
hwaddr = qemu_plugin_get_hwaddr(info, vaddr);
if (hwaddr && qemu_plugin_hwaddr_is_io(hwaddr)) {
@@ -391,14 +398,29 @@ static void vcpu_mem_access(unsigned int vcpu_index, qemu_plugin_meminfo_t info,
effective_addr = hwaddr ? qemu_plugin_hwaddr_phys_addr(hwaddr) : vaddr;
cache_idx = vcpu_index % cores;
- g_mutex_lock(&dcache_locks[cache_idx]);
- if (!access_cache(dcaches[cache_idx], effective_addr)) {
+ g_mutex_lock(&l1_dcache_locks[cache_idx]);
+ hit_in_l1 = access_cache(l1_dcaches[cache_idx], effective_addr);
+ if (!hit_in_l1) {
+ insn = (InsnData *) userdata;
+ __atomic_fetch_add(&insn->l1_dmisses, 1, __ATOMIC_SEQ_CST);
+ l1_dcaches[cache_idx]->misses++;
+ }
+ l1_dcaches[cache_idx]->accesses++;
+ g_mutex_unlock(&l1_dcache_locks[cache_idx]);
+
+ if (hit_in_l1) {
+ /* No need to access L2 */
+ return;
+ }
+
+ g_mutex_lock(&l2_ucache_locks[cache_idx]);
+ if (!access_cache(l2_ucaches[cache_idx], effective_addr)) {
insn = (InsnData *) userdata;
- __atomic_fetch_add(&insn->dmisses, 1, __ATOMIC_SEQ_CST);
- dcaches[cache_idx]->misses++;
+ __atomic_fetch_add(&insn->l2_misses, 1, __ATOMIC_SEQ_CST);
+ l2_ucaches[cache_idx]->misses++;
}
- dcaches[cache_idx]->accesses++;
- g_mutex_unlock(&dcache_locks[cache_idx]);
+ l2_ucaches[cache_idx]->accesses++;
+ g_mutex_unlock(&l2_ucache_locks[cache_idx]);
}
static void vcpu_insn_exec(unsigned int vcpu_index, void *userdata)
@@ -406,18 +428,34 @@ static void vcpu_insn_exec(unsigned int vcpu_index, void *userdata)
uint64_t insn_addr;
InsnData *insn;
int cache_idx;
+ bool hit_in_l1;
insn_addr = ((InsnData *) userdata)->addr;
cache_idx = vcpu_index % cores;
- g_mutex_lock(&icache_locks[cache_idx]);
- if (!access_cache(icaches[cache_idx], insn_addr)) {
+ g_mutex_lock(&l1_icache_locks[cache_idx]);
+ hit_in_l1 = access_cache(l1_icaches[cache_idx], insn_addr);
+ if (!hit_in_l1) {
+ insn = (InsnData *) userdata;
+ __atomic_fetch_add(&insn->l1_imisses, 1, __ATOMIC_SEQ_CST);
+ l1_icaches[cache_idx]->misses++;
+ }
+ l1_icaches[cache_idx]->accesses++;
+ g_mutex_unlock(&l1_icache_locks[cache_idx]);
+
+ if (hit_in_l1) {
+ /* No need to access L2 */
+ return;
+ }
+
+ g_mutex_lock(&l2_ucache_locks[cache_idx]);
+ if (!access_cache(l2_ucaches[cache_idx], insn_addr)) {
insn = (InsnData *) userdata;
- __atomic_fetch_add(&insn->imisses, 1, __ATOMIC_SEQ_CST);
- icaches[cache_idx]->misses++;
+ __atomic_fetch_add(&insn->l2_misses, 1, __ATOMIC_SEQ_CST);
+ l2_ucaches[cache_idx]->misses++;
}
- icaches[cache_idx]->accesses++;
- g_mutex_unlock(&icache_locks[cache_idx]);
+ l2_ucaches[cache_idx]->accesses++;
+ g_mutex_unlock(&l2_ucache_locks[cache_idx]);
}
static void vcpu_tb_trans(qemu_plugin_id_t id, struct qemu_plugin_tb *tb)
@@ -493,30 +531,28 @@ static void caches_free(Cache **caches)
}
}
-static int dcmp(gconstpointer a, gconstpointer b)
-{
- InsnData *insn_a = (InsnData *) a;
- InsnData *insn_b = (InsnData *) b;
-
- return insn_a->dmisses < insn_b->dmisses ? 1 : -1;
-}
-
-static void append_stats_line(GString *line, uint64_t daccess, uint64_t dmisses,
- uint64_t iaccess, uint64_t imisses)
+static void append_stats_line(GString *line, uint64_t l1_daccess,
+ uint64_t l1_dmisses, uint64_t l1_iaccess,
+ uint64_t l1_imisses, uint64_t l2_access,
+ uint64_t l2_misses)
{
- double dmiss_rate, imiss_rate;
+ double l1_dmiss_rate, l1_imiss_rate, l2_miss_rate;
- dmiss_rate = ((double) dmisses) / (daccess) * 100.0;
- imiss_rate = ((double) imisses) / (iaccess) * 100.0;
+ l1_dmiss_rate = ((double) l1_dmisses) / (l1_daccess) * 100.0;
+ l1_imiss_rate = ((double) l1_imisses) / (l1_iaccess) * 100.0;
+ l2_miss_rate = ((double) l2_misses) / (l2_access) * 100.0;
g_string_append_printf(line, "%-14lu %-12lu %9.4lf%% %-14lu %-12lu"
- " %9.4lf%%\n",
- daccess,
- dmisses,
- daccess ? dmiss_rate : 0.0,
- iaccess,
- imisses,
- iaccess ? imiss_rate : 0.0);
+ " %9.4lf%% %-12lu %-11lu %10.4lf%%\n",
+ l1_daccess,
+ l1_dmisses,
+ l1_daccess ? l1_dmiss_rate : 0.0,
+ l1_iaccess,
+ l1_imisses,
+ l1_iaccess ? l1_imiss_rate : 0.0,
+ l2_access,
+ l2_misses,
+ l2_access ? l2_miss_rate : 0.0);
}
static void sum_stats(void)
@@ -525,43 +561,66 @@ static void sum_stats(void)
g_assert(cores > 1);
for (i = 0; i < cores; i++) {
- all_imisses += icaches[i]->misses;
- all_dmisses += dcaches[i]->misses;
- all_imem_accesses += icaches[i]->accesses;
- all_dmem_accesses += dcaches[i]->accesses;
+ l1_imisses += l1_icaches[i]->misses;
+ l1_dmisses += l1_dcaches[i]->misses;
+ l1_imem_accesses += l1_icaches[i]->accesses;
+ l1_dmem_accesses += l1_dcaches[i]->accesses;
+
+ l2_misses += l2_ucaches[i]->misses;
+ l2_mem_accesses += l2_ucaches[i]->accesses;
}
}
+static int dcmp(gconstpointer a, gconstpointer b)
+{
+ InsnData *insn_a = (InsnData *) a;
+ InsnData *insn_b = (InsnData *) b;
+
+ return insn_a->l1_dmisses < insn_b->l1_dmisses ? 1 : -1;
+}
+
static int icmp(gconstpointer a, gconstpointer b)
{
InsnData *insn_a = (InsnData *) a;
InsnData *insn_b = (InsnData *) b;
- return insn_a->imisses < insn_b->imisses ? 1 : -1;
+ return insn_a->l1_imisses < insn_b->l1_imisses ? 1 : -1;
+}
+
+static int l2_cmp(gconstpointer a, gconstpointer b)
+{
+ InsnData *insn_a = (InsnData *) a;
+ InsnData *insn_b = (InsnData *) b;
+
+ return insn_a->l2_misses < insn_b->l2_misses ? 1 : -1;
}
static void log_stats(void)
{
int i;
- Cache *icache, *dcache;
+ Cache *icache, *dcache, *l2_cache;
g_autoptr(GString) rep = g_string_new("core #, data accesses, data misses,"
" dmiss rate, insn accesses,"
- " insn misses, imiss rate\n");
+ " insn misses, imiss rate,"
+ " l2 accesses, l2 misses,"
+ " l2 miss rate\n");
for (i = 0; i < cores; i++) {
g_string_append_printf(rep, "%-8d", i);
- dcache = dcaches[i];
- icache = icaches[i];
+ dcache = l1_dcaches[i];
+ icache = l1_icaches[i];
+ l2_cache = l2_ucaches[i];
append_stats_line(rep, dcache->accesses, dcache->misses,
- icache->accesses, icache->misses);
+ icache->accesses, icache->misses, l2_cache->accesses,
+ l2_cache->misses);
}
if (cores > 1) {
sum_stats();
g_string_append_printf(rep, "%-8s", "sum");
- append_stats_line(rep, all_dmem_accesses, all_dmisses,
- all_imem_accesses, all_imisses);
+ append_stats_line(rep, l1_dmem_accesses, l1_dmisses,
+ l1_imem_accesses, l1_imisses, l2_mem_accesses, l2_misses);
}
g_string_append(rep, "\n");
@@ -585,7 +644,7 @@ static void log_top_insns(void)
if (insn->symbol) {
g_string_append_printf(rep, " (%s)", insn->symbol);
}
- g_string_append_printf(rep, ", %ld, %s\n", insn->dmisses,
+ g_string_append_printf(rep, ", %ld, %s\n", insn->l1_dmisses,
insn->disas_str);
}
@@ -598,7 +657,20 @@ static void log_top_insns(void)
if (insn->symbol) {
g_string_append_printf(rep, " (%s)", insn->symbol);
}
- g_string_append_printf(rep, ", %ld, %s\n", insn->imisses,
+ g_string_append_printf(rep, ", %ld, %s\n", insn->l1_imisses,
+ insn->disas_str);
+ }
+
+ miss_insns = g_list_sort(miss_insns, l2_cmp);
+ g_string_append_printf(rep, "%s", "\naddress, L2 misses, instruction\n");
+
+ for (curr = miss_insns, i = 0; curr && i < limit; i++, curr = curr->next) {
+ insn = (InsnData *) curr->data;
+ g_string_append_printf(rep, "0x%" PRIx64, insn->addr);
+ if (insn->symbol) {
+ g_string_append_printf(rep, " (%s)", insn->symbol);
+ }
+ g_string_append_printf(rep, ", %ld, %s\n", insn->l2_misses,
insn->disas_str);
}
@@ -611,11 +683,13 @@ static void plugin_exit(qemu_plugin_id_t id, void *p)
log_stats();
log_top_insns();
- caches_free(dcaches);
- caches_free(icaches);
+ caches_free(l1_dcaches);
+ caches_free(l1_icaches);
+ caches_free(l2_ucaches);
- g_free(dcache_locks);
- g_free(icache_locks);
+ g_free(l1_dcache_locks);
+ g_free(l1_icache_locks);
+ g_free(l2_ucache_locks);
g_hash_table_destroy(miss_ht);
}
@@ -647,19 +721,24 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
int argc, char **argv)
{
int i;
- int iassoc, iblksize, icachesize;
- int dassoc, dblksize, dcachesize;
+ int l1_iassoc, l1_iblksize, l1_icachesize;
+ int l1_dassoc, l1_dblksize, l1_dcachesize;
+ int l2_assoc, l2_blksize, l2_cachesize;
limit = 32;
sys = info->system_emulation;
- dassoc = 8;
- dblksize = 64;
- dcachesize = dblksize * dassoc * 32;
+ l1_dassoc = 8;
+ l1_dblksize = 64;
+ l1_dcachesize = l1_dblksize * l1_dassoc * 32;
+
+ l1_iassoc = 8;
+ l1_iblksize = 64;
+ l1_icachesize = l1_iblksize * l1_iassoc * 32;
- iassoc = 8;
- iblksize = 64;
- icachesize = iblksize * iassoc * 32;
+ l2_assoc = 16;
+ l2_blksize = 64;
+ l2_cachesize = l2_assoc * l2_blksize * 2048;
policy = LRU;
@@ -668,21 +747,27 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
for (i = 0; i < argc; i++) {
char *opt = argv[i];
if (g_str_has_prefix(opt, "iblksize=")) {
- iblksize = g_ascii_strtoll(opt + 9, NULL, 10);
+ l1_iblksize = g_ascii_strtoll(opt + 9, NULL, 10);
} else if (g_str_has_prefix(opt, "iassoc=")) {
- iassoc = g_ascii_strtoll(opt + 7, NULL, 10);
+ l1_iassoc = g_ascii_strtoll(opt + 7, NULL, 10);
} else if (g_str_has_prefix(opt, "icachesize=")) {
- icachesize = g_ascii_strtoll(opt + 11, NULL, 10);
+ l1_icachesize = g_ascii_strtoll(opt + 11, NULL, 10);
} else if (g_str_has_prefix(opt, "dblksize=")) {
- dblksize = g_ascii_strtoll(opt + 9, NULL, 10);
+ l1_dblksize = g_ascii_strtoll(opt + 9, NULL, 10);
} else if (g_str_has_prefix(opt, "dassoc=")) {
- dassoc = g_ascii_strtoll(opt + 7, NULL, 10);
+ l1_dassoc = g_ascii_strtoll(opt + 7, NULL, 10);
} else if (g_str_has_prefix(opt, "dcachesize=")) {
- dcachesize = g_ascii_strtoll(opt + 11, NULL, 10);
+ l1_dcachesize = g_ascii_strtoll(opt + 11, NULL, 10);
} else if (g_str_has_prefix(opt, "limit=")) {
limit = g_ascii_strtoll(opt + 6, NULL, 10);
} else if (g_str_has_prefix(opt, "cores=")) {
cores = g_ascii_strtoll(opt + 6, NULL, 10);
+ } else if (g_str_has_prefix(opt, "l2cachesize=")) {
+ l2_cachesize = g_ascii_strtoll(opt + 6, NULL, 10);
+ } else if (g_str_has_prefix(opt, "l2blksize=")) {
+ l2_blksize = g_ascii_strtoll(opt + 6, NULL, 10);
+ } else if (g_str_has_prefix(opt, "l2assoc=")) {
+ l2_assoc = g_ascii_strtoll(opt + 6, NULL, 10);
} else if (g_str_has_prefix(opt, "evict=")) {
gchar *p = opt + 6;
if (g_strcmp0(p, "rand") == 0) {
@@ -703,24 +788,33 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
policy_init();
- dcaches = caches_init(dblksize, dassoc, dcachesize);
- if (!dcaches) {
- const char *err = cache_config_error(dblksize, dassoc, dcachesize);
+ l1_dcaches = caches_init(l1_dblksize, l1_dassoc, l1_dcachesize);
+ if (!l1_dcaches) {
+ const char *err = cache_config_error(l1_dblksize, l1_dassoc, l1_dcachesize);
fprintf(stderr, "dcache cannot be constructed from given parameters\n");
fprintf(stderr, "%s\n", err);
return -1;
}
- icaches = caches_init(iblksize, iassoc, icachesize);
- if (!icaches) {
- const char *err = cache_config_error(iblksize, iassoc, icachesize);
+ l1_icaches = caches_init(l1_iblksize, l1_iassoc, l1_icachesize);
+ if (!l1_icaches) {
+ const char *err = cache_config_error(l1_iblksize, l1_iassoc, l1_icachesize);
fprintf(stderr, "icache cannot be constructed from given parameters\n");
fprintf(stderr, "%s\n", err);
return -1;
}
- dcache_locks = g_new0(GMutex, cores);
- icache_locks = g_new0(GMutex, cores);
+ l2_ucaches = caches_init(l2_blksize, l2_assoc, l2_cachesize);
+ if (!l2_ucaches) {
+ const char *err = cache_config_error(l2_blksize, l2_assoc, l2_cachesize);
+ fprintf(stderr, "L2 cache cannot be constructed from given parameters\n");
+ fprintf(stderr, "%s\n", err);
+ return -1;
+ }
+
+ l1_dcache_locks = g_new0(GMutex, cores);
+ l1_icache_locks = g_new0(GMutex, cores);
+ l2_ucache_locks = g_new0(GMutex, cores);
qemu_plugin_register_vcpu_tb_trans_cb(id, vcpu_tb_trans);
qemu_plugin_register_atexit_cb(id, plugin_exit, NULL);
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* [PATCH v1 15/28] plugins/cache: split command line arguments into name and value
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (13 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 14/28] plugins/cache: implement unified L2 cache emulation Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 10:22 ` [PATCH v1 16/28] plugins/cache: make L2 emulation optional through args Alex Bennée
` (12 subsequent siblings)
27 siblings, 0 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
mahmoudabdalghany, aaron, cota, stefanha, crosa, pbonzini,
ma.mandourr, Alexandre Iooss, Alex Bennée, aurelien
From: Mahmoud Mandour <ma.mandourr@gmail.com>
This way of handling args is more lenient and sets a better framework to
parse boolean command line arguments.
Signed-off-by: Mahmoud Mandour <ma.mandourr@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210810134844.166490-4-ma.mandourr@gmail.com>
---
contrib/plugins/cache.c | 57 ++++++++++++++++++++++-------------------
1 file changed, 30 insertions(+), 27 deletions(-)
diff --git a/contrib/plugins/cache.c b/contrib/plugins/cache.c
index 908c967a09..ff325beb9f 100644
--- a/contrib/plugins/cache.c
+++ b/contrib/plugins/cache.c
@@ -11,6 +11,8 @@
#include <qemu-plugin.h>
+#define STRTOLL(x) g_ascii_strtoll(x, NULL, 10)
+
QEMU_PLUGIN_EXPORT int qemu_plugin_version = QEMU_PLUGIN_VERSION;
static enum qemu_plugin_mem_rw rw = QEMU_PLUGIN_MEM_RW;
@@ -746,35 +748,36 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
for (i = 0; i < argc; i++) {
char *opt = argv[i];
- if (g_str_has_prefix(opt, "iblksize=")) {
- l1_iblksize = g_ascii_strtoll(opt + 9, NULL, 10);
- } else if (g_str_has_prefix(opt, "iassoc=")) {
- l1_iassoc = g_ascii_strtoll(opt + 7, NULL, 10);
- } else if (g_str_has_prefix(opt, "icachesize=")) {
- l1_icachesize = g_ascii_strtoll(opt + 11, NULL, 10);
- } else if (g_str_has_prefix(opt, "dblksize=")) {
- l1_dblksize = g_ascii_strtoll(opt + 9, NULL, 10);
- } else if (g_str_has_prefix(opt, "dassoc=")) {
- l1_dassoc = g_ascii_strtoll(opt + 7, NULL, 10);
- } else if (g_str_has_prefix(opt, "dcachesize=")) {
- l1_dcachesize = g_ascii_strtoll(opt + 11, NULL, 10);
- } else if (g_str_has_prefix(opt, "limit=")) {
- limit = g_ascii_strtoll(opt + 6, NULL, 10);
- } else if (g_str_has_prefix(opt, "cores=")) {
- cores = g_ascii_strtoll(opt + 6, NULL, 10);
- } else if (g_str_has_prefix(opt, "l2cachesize=")) {
- l2_cachesize = g_ascii_strtoll(opt + 6, NULL, 10);
- } else if (g_str_has_prefix(opt, "l2blksize=")) {
- l2_blksize = g_ascii_strtoll(opt + 6, NULL, 10);
- } else if (g_str_has_prefix(opt, "l2assoc=")) {
- l2_assoc = g_ascii_strtoll(opt + 6, NULL, 10);
- } else if (g_str_has_prefix(opt, "evict=")) {
- gchar *p = opt + 6;
- if (g_strcmp0(p, "rand") == 0) {
+ g_autofree char **tokens = g_strsplit(opt, "=", 2);
+
+ if (g_strcmp0(tokens[0], "iblksize") == 0) {
+ l1_iblksize = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "iassoc") == 0) {
+ l1_iassoc = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "icachesize") == 0) {
+ l1_icachesize = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "dblksize") == 0) {
+ l1_dblksize = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "dassoc") == 0) {
+ l1_dassoc = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "dcachesize") == 0) {
+ l1_dcachesize = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "limit") == 0) {
+ limit = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "cores") == 0) {
+ cores = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "l2cachesize") == 0) {
+ l2_cachesize = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "l2blksize") == 0) {
+ l2_blksize = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "l2assoc") == 0) {
+ l2_assoc = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "evict") == 0) {
+ if (g_strcmp0(tokens[1], "rand") == 0) {
policy = RAND;
- } else if (g_strcmp0(p, "lru") == 0) {
+ } else if (g_strcmp0(tokens[1], "lru") == 0) {
policy = LRU;
- } else if (g_strcmp0(p, "fifo") == 0) {
+ } else if (g_strcmp0(tokens[1], "fifo") == 0) {
policy = FIFO;
} else {
fprintf(stderr, "invalid eviction policy: %s\n", opt);
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* [PATCH v1 16/28] plugins/cache: make L2 emulation optional through args
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (14 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 15/28] plugins/cache: split command line arguments into name and value Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 10:22 ` [PATCH v1 17/28] docs/tcg-plugins: add L2 arguments to cache docs Alex Bennée
` (11 subsequent siblings)
27 siblings, 0 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
mahmoudabdalghany, aaron, cota, stefanha, crosa, pbonzini,
ma.mandourr, Alexandre Iooss, Alex Bennée, aurelien
From: Mahmoud Mandour <ma.mandourr@gmail.com>
By default L2 is not enabled and is enabled by either using the
newly-introduced "l2" boolean argument, or by setting any of the L2
cache parameters using args. On specifying "l2=on", the default cache
configuration is used.
Signed-off-by: Mahmoud Mandour <ma.mandourr@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210810134844.166490-5-ma.mandourr@gmail.com>
---
contrib/plugins/cache.c | 76 +++++++++++++++++++++++++++++------------
1 file changed, 54 insertions(+), 22 deletions(-)
diff --git a/contrib/plugins/cache.c b/contrib/plugins/cache.c
index ff325beb9f..b9226e7c40 100644
--- a/contrib/plugins/cache.c
+++ b/contrib/plugins/cache.c
@@ -97,6 +97,8 @@ void (*metadata_destroy)(Cache *cache);
static int cores;
static Cache **l1_dcaches, **l1_icaches;
+
+static bool use_l2;
static Cache **l2_ucaches;
static GMutex *l1_dcache_locks;
@@ -410,7 +412,7 @@ static void vcpu_mem_access(unsigned int vcpu_index, qemu_plugin_meminfo_t info,
l1_dcaches[cache_idx]->accesses++;
g_mutex_unlock(&l1_dcache_locks[cache_idx]);
- if (hit_in_l1) {
+ if (hit_in_l1 || !use_l2) {
/* No need to access L2 */
return;
}
@@ -445,7 +447,7 @@ static void vcpu_insn_exec(unsigned int vcpu_index, void *userdata)
l1_icaches[cache_idx]->accesses++;
g_mutex_unlock(&l1_icache_locks[cache_idx]);
- if (hit_in_l1) {
+ if (hit_in_l1 || !use_l2) {
/* No need to access L2 */
return;
}
@@ -542,19 +544,25 @@ static void append_stats_line(GString *line, uint64_t l1_daccess,
l1_dmiss_rate = ((double) l1_dmisses) / (l1_daccess) * 100.0;
l1_imiss_rate = ((double) l1_imisses) / (l1_iaccess) * 100.0;
- l2_miss_rate = ((double) l2_misses) / (l2_access) * 100.0;
g_string_append_printf(line, "%-14lu %-12lu %9.4lf%% %-14lu %-12lu"
- " %9.4lf%% %-12lu %-11lu %10.4lf%%\n",
+ " %9.4lf%%",
l1_daccess,
l1_dmisses,
l1_daccess ? l1_dmiss_rate : 0.0,
l1_iaccess,
l1_imisses,
- l1_iaccess ? l1_imiss_rate : 0.0,
- l2_access,
- l2_misses,
- l2_access ? l2_miss_rate : 0.0);
+ l1_iaccess ? l1_imiss_rate : 0.0);
+
+ if (use_l2) {
+ l2_miss_rate = ((double) l2_misses) / (l2_access) * 100.0;
+ g_string_append_printf(line, " %-12lu %-11lu %10.4lf%%",
+ l2_access,
+ l2_misses,
+ l2_access ? l2_miss_rate : 0.0);
+ }
+
+ g_string_append(line, "\n");
}
static void sum_stats(void)
@@ -568,8 +576,10 @@ static void sum_stats(void)
l1_imem_accesses += l1_icaches[i]->accesses;
l1_dmem_accesses += l1_dcaches[i]->accesses;
- l2_misses += l2_ucaches[i]->misses;
- l2_mem_accesses += l2_ucaches[i]->accesses;
+ if (use_l2) {
+ l2_misses += l2_ucaches[i]->misses;
+ l2_mem_accesses += l2_ucaches[i]->accesses;
+ }
}
}
@@ -604,25 +614,31 @@ static void log_stats(void)
g_autoptr(GString) rep = g_string_new("core #, data accesses, data misses,"
" dmiss rate, insn accesses,"
- " insn misses, imiss rate,"
- " l2 accesses, l2 misses,"
- " l2 miss rate\n");
+ " insn misses, imiss rate");
+
+ if (use_l2) {
+ g_string_append(rep, ", l2 accesses, l2 misses, l2 miss rate");
+ }
+
+ g_string_append(rep, "\n");
for (i = 0; i < cores; i++) {
g_string_append_printf(rep, "%-8d", i);
dcache = l1_dcaches[i];
icache = l1_icaches[i];
- l2_cache = l2_ucaches[i];
+ l2_cache = use_l2 ? l2_ucaches[i] : NULL;
append_stats_line(rep, dcache->accesses, dcache->misses,
- icache->accesses, icache->misses, l2_cache->accesses,
- l2_cache->misses);
+ icache->accesses, icache->misses,
+ l2_cache ? l2_cache->accesses : 0,
+ l2_cache ? l2_cache->misses : 0);
}
if (cores > 1) {
sum_stats();
g_string_append_printf(rep, "%-8s", "sum");
append_stats_line(rep, l1_dmem_accesses, l1_dmisses,
- l1_imem_accesses, l1_imisses, l2_mem_accesses, l2_misses);
+ l1_imem_accesses, l1_imisses,
+ l2_cache ? l2_mem_accesses : 0, l2_cache ? l2_misses : 0);
}
g_string_append(rep, "\n");
@@ -663,6 +679,10 @@ static void log_top_insns(void)
insn->disas_str);
}
+ if (!use_l2) {
+ goto finish;
+ }
+
miss_insns = g_list_sort(miss_insns, l2_cmp);
g_string_append_printf(rep, "%s", "\naddress, L2 misses, instruction\n");
@@ -676,6 +696,7 @@ static void log_top_insns(void)
insn->disas_str);
}
+finish:
qemu_plugin_outs(rep->str);
g_list_free(miss_insns);
}
@@ -687,11 +708,14 @@ static void plugin_exit(qemu_plugin_id_t id, void *p)
caches_free(l1_dcaches);
caches_free(l1_icaches);
- caches_free(l2_ucaches);
g_free(l1_dcache_locks);
g_free(l1_icache_locks);
- g_free(l2_ucache_locks);
+
+ if (use_l2) {
+ caches_free(l2_ucaches);
+ g_free(l2_ucache_locks);
+ }
g_hash_table_destroy(miss_ht);
}
@@ -767,11 +791,19 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
} else if (g_strcmp0(tokens[0], "cores") == 0) {
cores = STRTOLL(tokens[1]);
} else if (g_strcmp0(tokens[0], "l2cachesize") == 0) {
+ use_l2 = true;
l2_cachesize = STRTOLL(tokens[1]);
} else if (g_strcmp0(tokens[0], "l2blksize") == 0) {
+ use_l2 = true;
l2_blksize = STRTOLL(tokens[1]);
} else if (g_strcmp0(tokens[0], "l2assoc") == 0) {
+ use_l2 = true;
l2_assoc = STRTOLL(tokens[1]);
+ } else if (g_strcmp0(tokens[0], "l2") == 0) {
+ if (!qemu_plugin_bool_parse(tokens[0], tokens[1], &use_l2)) {
+ fprintf(stderr, "boolean argument parsing failed: %s\n", opt);
+ return -1;
+ }
} else if (g_strcmp0(tokens[0], "evict") == 0) {
if (g_strcmp0(tokens[1], "rand") == 0) {
policy = RAND;
@@ -807,8 +839,8 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
return -1;
}
- l2_ucaches = caches_init(l2_blksize, l2_assoc, l2_cachesize);
- if (!l2_ucaches) {
+ l2_ucaches = use_l2 ? caches_init(l2_blksize, l2_assoc, l2_cachesize) : NULL;
+ if (!l2_ucaches && use_l2) {
const char *err = cache_config_error(l2_blksize, l2_assoc, l2_cachesize);
fprintf(stderr, "L2 cache cannot be constructed from given parameters\n");
fprintf(stderr, "%s\n", err);
@@ -817,7 +849,7 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
l1_dcache_locks = g_new0(GMutex, cores);
l1_icache_locks = g_new0(GMutex, cores);
- l2_ucache_locks = g_new0(GMutex, cores);
+ l2_ucache_locks = use_l2 ? g_new0(GMutex, cores) : NULL;
qemu_plugin_register_vcpu_tb_trans_cb(id, vcpu_tb_trans);
qemu_plugin_register_atexit_cb(id, plugin_exit, NULL);
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* [PATCH v1 17/28] docs/tcg-plugins: add L2 arguments to cache docs
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (15 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 16/28] plugins/cache: make L2 emulation optional through args Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 10:22 ` [PATCH v1 18/28] chardev: don't exit() straight away on C-a x Alex Bennée
` (10 subsequent siblings)
27 siblings, 0 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
mahmoudabdalghany, aaron, cota, stefanha, crosa, pbonzini,
ma.mandourr, Alexandre Iooss, Alex Bennée, aurelien
From: Mahmoud Mandour <ma.mandourr@gmail.com>
cache plugin now allows optional L2 per-core cache emulation that can be
configured through plugin arguments, this commit adds this functionality
to the docs.
While I'm at it, I editted the bullet point for cache plugin to say:
contrib/plugins/cache.c
instead of
contrib/plugins/cache
to match other plugins.
Signed-off-by: Mahmoud Mandour <ma.mandourr@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210810134844.166490-6-ma.mandourr@gmail.com>
---
docs/devel/tcg-plugins.rst | 20 ++++++++++++++++++--
1 file changed, 18 insertions(+), 2 deletions(-)
diff --git a/docs/devel/tcg-plugins.rst b/docs/devel/tcg-plugins.rst
index 842ae01a4c..59a7d838be 100644
--- a/docs/devel/tcg-plugins.rst
+++ b/docs/devel/tcg-plugins.rst
@@ -361,8 +361,9 @@ which will output an execution trace following this structure::
- contrib/plugins/cache.c
-Cache modelling plugin that measures the performance of a given cache
-configuration when a given working set is run::
+Cache modelling plugin that measures the performance of a given L1 cache
+configuration, and optionally a unified L2 per-core cache when a given working
+set is run::
qemu-x86_64 -plugin ./contrib/plugins/libcache.so \
-d plugin -D cache.log ./tests/tcg/x86_64-linux-user/float_convs
@@ -420,3 +421,18 @@ The plugin has a number of arguments, all of them are optional:
Sets the number of cores for which we maintain separate icache and dcache.
(default: for linux-user, N = 1, for full system emulation: N = cores
available to guest)
+
+ * l2=on
+
+ Simulates a unified L2 cache (stores blocks for both instructions and data)
+ using the default L2 configuration (cache size = 2MB, associativity = 16-way,
+ block size = 64B).
+
+ * l2cachesize=N
+ * l2blksize=B
+ * l2assoc=A
+
+ L2 cache configuration arguments. They specify the cache size, block size, and
+ associativity of the L2 cache, respectively. Setting any of the L2
+ configuration arguments implies ``l2=on``.
+ (default: N = 2097152 (2MB), B = 64, A = 16)
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* [PATCH v1 18/28] chardev: don't exit() straight away on C-a x
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (16 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 17/28] docs/tcg-plugins: add L2 arguments to cache docs Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 10:22 ` [PATCH v1 19/28] tests/plugins: extend the insn plugin to track opcode sizes Alex Bennée
` (9 subsequent siblings)
27 siblings, 0 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, Lukas Jünger,
Philippe Mathieu-Daudé,
f4bug, robhenry, mahmoudabdalghany, aaron, cota, stefanha, crosa,
pbonzini, ma.mandourr, Marc-André Lureau, Alex Bennée,
aurelien
While there are a number of uses in the code-base of the exit(0)
pattern it gets in the way of clean exit which can do all of it's
house-keeping. In particular it was reported that you can crash
plugins this way because TCG can still be running on other threads
when the atexit callback is called.
Use qmp_quit() instead which takes care of some housekeeping before
triggering the shutdown.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reported-by: Lukas Jünger <lukas.junger@greensocs.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20211018140226.838137-1-alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20211019092901.1519570-1-alex.bennee@linaro.org>
---
chardev/char-mux.c | 3 ++-
stubs/qmp-quit.c | 8 ++++++++
stubs/meson.build | 1 +
3 files changed, 11 insertions(+), 1 deletion(-)
create mode 100644 stubs/qmp-quit.c
diff --git a/chardev/char-mux.c b/chardev/char-mux.c
index ada0c6866f..ee2d47b20d 100644
--- a/chardev/char-mux.c
+++ b/chardev/char-mux.c
@@ -28,6 +28,7 @@
#include "qemu/option.h"
#include "chardev/char.h"
#include "sysemu/block-backend.h"
+#include "qapi/qapi-commands-control.h"
#include "chardev-internal.h"
/* MUX driver for serial I/O splitting */
@@ -157,7 +158,7 @@ static int mux_proc_byte(Chardev *chr, MuxChardev *d, int ch)
{
const char *term = "QEMU: Terminated\n\r";
qemu_chr_write_all(chr, (uint8_t *)term, strlen(term));
- exit(0);
+ qmp_quit(NULL);
break;
}
case 's':
diff --git a/stubs/qmp-quit.c b/stubs/qmp-quit.c
new file mode 100644
index 0000000000..a3ff47f7bd
--- /dev/null
+++ b/stubs/qmp-quit.c
@@ -0,0 +1,8 @@
+#include "qemu/osdep.h"
+#include "qapi/qapi-commands-control.h"
+#include "qapi/qmp/dispatch.h"
+
+void qmp_quit(Error **errp)
+{
+ g_assert_not_reached();
+}
diff --git a/stubs/meson.build b/stubs/meson.build
index f6aa3aa94f..71469c1d50 100644
--- a/stubs/meson.build
+++ b/stubs/meson.build
@@ -31,6 +31,7 @@ stub_ss.add(files('pci-bus.c'))
stub_ss.add(files('qemu-timer-notify-cb.c'))
stub_ss.add(files('qmp_memory_device.c'))
stub_ss.add(files('qmp-command-available.c'))
+stub_ss.add(files('qmp-quit.c'))
stub_ss.add(files('qtest.c'))
stub_ss.add(files('ram-block.c'))
stub_ss.add(files('ramfb.c'))
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* [PATCH v1 19/28] tests/plugins: extend the insn plugin to track opcode sizes
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (17 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 18/28] chardev: don't exit() straight away on C-a x Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 10:22 ` [PATCH v1 20/28] plugins: try and make plugin_insn_append more ergonomic Alex Bennée
` (8 subsequent siblings)
27 siblings, 0 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
mahmoudabdalghany, aaron, cota, stefanha, crosa, pbonzini,
ma.mandourr, Alexandre Iooss, Alex Bennée, aurelien
This is mostly a convenience feature for identifying frontends that do
multiple repeat loads so I can test changes to the instruction
tracking interface.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/plugin/insn.c | 37 +++++++++++++++++++++++++++++++++++--
1 file changed, 35 insertions(+), 2 deletions(-)
diff --git a/tests/plugin/insn.c b/tests/plugin/insn.c
index 0f6a1938c1..d229fdc001 100644
--- a/tests/plugin/insn.c
+++ b/tests/plugin/insn.c
@@ -18,6 +18,8 @@ QEMU_PLUGIN_EXPORT int qemu_plugin_version = QEMU_PLUGIN_VERSION;
static uint64_t insn_count;
static bool do_inline;
+static bool do_size;
+static GArray *sizes;
static void vcpu_insn_exec_before(unsigned int cpu_index, void *udata)
{
@@ -49,13 +51,35 @@ static void vcpu_tb_trans(qemu_plugin_id_t id, struct qemu_plugin_tb *tb)
insn, vcpu_insn_exec_before, QEMU_PLUGIN_CB_NO_REGS,
GUINT_TO_POINTER(vaddr));
}
+
+ if (do_size) {
+ size_t sz = qemu_plugin_insn_size(insn);
+ if (sz > sizes->len) {
+ g_array_set_size(sizes, sz);
+ }
+ unsigned long *cnt = &g_array_index(sizes, unsigned long, sz);
+ (*cnt)++;
+ }
}
}
static void plugin_exit(qemu_plugin_id_t id, void *p)
{
- g_autofree gchar *out = g_strdup_printf("insns: %" PRIu64 "\n", insn_count);
- qemu_plugin_outs(out);
+ g_autoptr(GString) out = g_string_new(NULL);
+
+ if (do_size) {
+ int i;
+ for (i = 0; i <= sizes->len; i++) {
+ unsigned long *cnt = &g_array_index(sizes, unsigned long, i);
+ if (*cnt) {
+ g_string_append_printf(out,
+ "len %d bytes: %ld insns\n", i, *cnt);
+ }
+ }
+ } else {
+ g_string_append_printf(out, "insns: %" PRIu64 "\n", insn_count);
+ }
+ qemu_plugin_outs(out->str);
}
QEMU_PLUGIN_EXPORT int qemu_plugin_install(qemu_plugin_id_t id,
@@ -70,12 +94,21 @@ QEMU_PLUGIN_EXPORT int qemu_plugin_install(qemu_plugin_id_t id,
fprintf(stderr, "boolean argument parsing failed: %s\n", opt);
return -1;
}
+ } else if (g_strcmp0(tokens[0], "sizes") == 0) {
+ if (!qemu_plugin_bool_parse(tokens[0], tokens[1], &do_size)) {
+ fprintf(stderr, "boolean argument parsing failed: %s\n", opt);
+ return -1;
+ }
} else {
fprintf(stderr, "option parsing failed: %s\n", opt);
return -1;
}
}
+ if (do_size) {
+ sizes = g_array_new(true, true, sizeof(unsigned long));
+ }
+
qemu_plugin_register_vcpu_tb_trans_cb(id, vcpu_tb_trans);
qemu_plugin_register_atexit_cb(id, plugin_exit, NULL);
return 0;
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* [PATCH v1 20/28] plugins: try and make plugin_insn_append more ergonomic
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (18 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 19/28] tests/plugins: extend the insn plugin to track opcode sizes Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 20:09 ` Richard Henderson
2021-10-26 10:22 ` [PATCH v1 21/28] docs: remove references to TCG tracing Alex Bennée
` (7 subsequent siblings)
27 siblings, 1 reply; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, Richard Henderson, f4bug,
robhenry, mahmoudabdalghany, aaron, cota, stefanha, crosa,
pbonzini, ma.mandourr, Alex Bennée, aurelien
Currently we make the assumption that the guest frontend loads all
op code bytes sequentially. This mostly holds up for regular fixed
encodings but some architectures like s390x like to re-read the
instruction which causes weirdness to occur. Rather than changing the
frontends make the plugin API a little more ergonomic and able to
handle the re-read case.
Stuff will still get strange if we read ahead of the opcode but so far
no front ends have done that and this patch asserts the case so we can
catch it early if they do.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Suggested-by: Richard Henderson <richard.henderson@linaro.org>
---
include/exec/plugin-gen.h | 12 ++++++++++--
include/qemu/plugin.h | 7 +++++--
accel/tcg/plugin-gen.c | 3 +--
accel/tcg/translator.c | 2 +-
4 files changed, 17 insertions(+), 7 deletions(-)
diff --git a/include/exec/plugin-gen.h b/include/exec/plugin-gen.h
index b1b72b5d90..f92f169739 100644
--- a/include/exec/plugin-gen.h
+++ b/include/exec/plugin-gen.h
@@ -27,13 +27,21 @@ void plugin_gen_insn_end(void);
void plugin_gen_disable_mem_helpers(void);
void plugin_gen_empty_mem_callback(TCGv addr, uint32_t info);
-static inline void plugin_insn_append(const void *from, size_t size)
+static inline void plugin_insn_append(abi_ptr pc, const void *from, size_t size)
{
struct qemu_plugin_insn *insn = tcg_ctx->plugin_insn;
+ abi_ptr off;
if (insn == NULL) {
return;
}
+ off = pc - insn->vaddr;
+ if (off < insn->data->len) {
+ g_byte_array_set_size(insn->data, off);
+ } else if (off > insn->data->len) {
+ /* we have an unexpected gap */
+ g_assert_not_reached();
+ }
insn->data = g_byte_array_append(insn->data, from, size);
}
@@ -62,7 +70,7 @@ static inline void plugin_gen_disable_mem_helpers(void)
static inline void plugin_gen_empty_mem_callback(TCGv addr, uint32_t info)
{ }
-static inline void plugin_insn_append(const void *from, size_t size)
+static inline void plugin_insn_append(abi_ptr pc, const void *from, size_t size)
{ }
#endif /* CONFIG_PLUGIN */
diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
index b3172b147f..145f8a221a 100644
--- a/include/qemu/plugin.h
+++ b/include/qemu/plugin.h
@@ -163,10 +163,12 @@ struct qemu_plugin_tb {
/**
* qemu_plugin_tb_insn_get(): get next plugin record for translation.
- *
+ * @tb: the internal tb context
+ * @pc: address of instruction
*/
static inline
-struct qemu_plugin_insn *qemu_plugin_tb_insn_get(struct qemu_plugin_tb *tb)
+struct qemu_plugin_insn *qemu_plugin_tb_insn_get(struct qemu_plugin_tb *tb,
+ uint64_t pc)
{
struct qemu_plugin_insn *insn;
int i, j;
@@ -179,6 +181,7 @@ struct qemu_plugin_insn *qemu_plugin_tb_insn_get(struct qemu_plugin_tb *tb)
g_byte_array_set_size(insn->data, 0);
insn->calls_helpers = false;
insn->mem_helper = false;
+ insn->vaddr = pc;
for (i = 0; i < PLUGIN_N_CB_TYPES; i++) {
for (j = 0; j < PLUGIN_N_CB_SUBTYPES; j++) {
diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index 61be64b78c..22d95fe1c3 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -876,9 +876,8 @@ void plugin_gen_insn_start(CPUState *cpu, const DisasContextBase *db)
struct qemu_plugin_tb *ptb = tcg_ctx->plugin_tb;
struct qemu_plugin_insn *pinsn;
- pinsn = qemu_plugin_tb_insn_get(ptb);
+ pinsn = qemu_plugin_tb_insn_get(ptb, db->pc_next);
tcg_ctx->plugin_insn = pinsn;
- pinsn->vaddr = db->pc_next;
plugin_gen_empty_callback(PLUGIN_GEN_FROM_INSN);
/*
diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
index 390bd9db0a..f06c314266 100644
--- a/accel/tcg/translator.c
+++ b/accel/tcg/translator.c
@@ -169,7 +169,7 @@ static inline void translator_maybe_page_protect(DisasContextBase *dcbase,
if (do_swap) { \
ret = swap_fn(ret); \
} \
- plugin_insn_append(&ret, sizeof(ret)); \
+ plugin_insn_append(pc, &ret, sizeof(ret)); \
return ret; \
}
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 20/28] plugins: try and make plugin_insn_append more ergonomic
2021-10-26 10:22 ` [PATCH v1 20/28] plugins: try and make plugin_insn_append more ergonomic Alex Bennée
@ 2021-10-26 20:09 ` Richard Henderson
0 siblings, 0 replies; 61+ messages in thread
From: Richard Henderson @ 2021-10-26 20:09 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
mahmoudabdalghany, aaron, cota, stefanha, crosa, pbonzini,
ma.mandourr, aurelien
On 10/26/21 3:22 AM, Alex Bennée wrote:
> Currently we make the assumption that the guest frontend loads all
> op code bytes sequentially. This mostly holds up for regular fixed
> encodings but some architectures like s390x like to re-read the
> instruction which causes weirdness to occur. Rather than changing the
> frontends make the plugin API a little more ergonomic and able to
> handle the re-read case.
>
> Stuff will still get strange if we read ahead of the opcode but so far
> no front ends have done that and this patch asserts the case so we can
> catch it early if they do.
>
> Signed-off-by: Alex Bennée<alex.bennee@linaro.org>
> Suggested-by: Richard Henderson<richard.henderson@linaro.org>
> ---
> include/exec/plugin-gen.h | 12 ++++++++++--
> include/qemu/plugin.h | 7 +++++--
> accel/tcg/plugin-gen.c | 3 +--
> accel/tcg/translator.c | 2 +-
> 4 files changed, 17 insertions(+), 7 deletions(-)
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
r~
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 21/28] docs: remove references to TCG tracing
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (19 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 20/28] plugins: try and make plugin_insn_append more ergonomic Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 20:10 ` Richard Henderson
2021-10-26 10:22 ` [PATCH v1 22/28] tracing: remove TCG memory access tracing Alex Bennée
` (6 subsequent siblings)
27 siblings, 1 reply; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
Luis Vilanova, mahmoudabdalghany, aaron, cota, stefanha, crosa,
pbonzini, ma.mandourr, Alex Bennée, aurelien
Users wanting this sort of functionality should turn to TCG plugins
instead.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Luis Vilanova <vilanova@imperial.ac.uk>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
---
docs/about/removed-features.rst | 13 +++++
docs/devel/tracing.rst | 85 ---------------------------------
2 files changed, 13 insertions(+), 85 deletions(-)
diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst
index 9d0d90c90d..641aa22112 100644
--- a/docs/about/removed-features.rst
+++ b/docs/about/removed-features.rst
@@ -594,6 +594,19 @@ the upstream Linux kernel in 2018, and it has also been dropped from glibc, so
there is no new Linux development taking place with this architecture. For
running the old binaries, you can use older versions of QEMU.
+TCG introspection features
+--------------------------
+
+TCG trace-events (since 6.2)
+''''''''''''''''''''''''''''
+
+The ability to add new TCG trace points had bit rotted and as the
+feature can be replicated with TCG plugins it was removed. If
+any user is currently using this feature and needs help with
+converting to using TCG plugins they should contact the qemu-devel
+mailing list.
+
+
System emulator devices
-----------------------
diff --git a/docs/devel/tracing.rst b/docs/devel/tracing.rst
index ba83954899..ef67bc9d70 100644
--- a/docs/devel/tracing.rst
+++ b/docs/devel/tracing.rst
@@ -411,88 +411,3 @@ disabled, this check will have no performance impact.
return ptr;
}
-"tcg"
------
-
-Guest code generated by TCG can be traced by defining an event with the "tcg"
-event property. Internally, this property generates two events:
-"<eventname>_trans" to trace the event at translation time, and
-"<eventname>_exec" to trace the event at execution time.
-
-Instead of using these two events, you should instead use the function
-"trace_<eventname>_tcg" during translation (TCG code generation). This function
-will automatically call "trace_<eventname>_trans", and will generate the
-necessary TCG code to call "trace_<eventname>_exec" during guest code execution.
-
-Events with the "tcg" property can be declared in the "trace-events" file with a
-mix of native and TCG types, and "trace_<eventname>_tcg" will gracefully forward
-them to the "<eventname>_trans" and "<eventname>_exec" events. Since TCG values
-are not known at translation time, these are ignored by the "<eventname>_trans"
-event. Because of this, the entry in the "trace-events" file needs two printing
-formats (separated by a comma)::
-
- tcg foo(uint8_t a1, TCGv_i32 a2) "a1=%d", "a1=%d a2=%d"
-
-For example::
-
- #include "trace-tcg.h"
-
- void some_disassembly_func (...)
- {
- uint8_t a1 = ...;
- TCGv_i32 a2 = ...;
- trace_foo_tcg(a1, a2);
- }
-
-This will immediately call::
-
- void trace_foo_trans(uint8_t a1);
-
-and will generate the TCG code to call::
-
- void trace_foo(uint8_t a1, uint32_t a2);
-
-"vcpu"
-------
-
-Identifies events that trace vCPU-specific information. It implicitly adds a
-"CPUState*" argument, and extends the tracing print format to show the vCPU
-information. If used together with the "tcg" property, it adds a second
-"TCGv_env" argument that must point to the per-target global TCG register that
-points to the vCPU when guest code is executed (usually the "cpu_env" variable).
-
-The "tcg" and "vcpu" properties are currently only honored in the root
-./trace-events file.
-
-The following example events::
-
- foo(uint32_t a) "a=%x"
- vcpu bar(uint32_t a) "a=%x"
- tcg vcpu baz(uint32_t a) "a=%x", "a=%x"
-
-Can be used as::
-
- #include "trace-tcg.h"
-
- CPUArchState *env;
- TCGv_ptr cpu_env;
-
- void some_disassembly_func(...)
- {
- /* trace emitted at this point */
- trace_foo(0xd1);
- /* trace emitted at this point */
- trace_bar(env_cpu(env), 0xd2);
- /* trace emitted at this point (env) and when guest code is executed (cpu_env) */
- trace_baz_tcg(env_cpu(env), cpu_env, 0xd3);
- }
-
-If the translating vCPU has address 0xc1 and code is later executed by vCPU
-0xc2, this would be an example output::
-
- // at guest code translation
- foo a=0xd1
- bar cpu=0xc1 a=0xd2
- baz_trans cpu=0xc1 a=0xd3
- // at guest code execution
- baz_exec cpu=0xc2 a=0xd3
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 21/28] docs: remove references to TCG tracing
2021-10-26 10:22 ` [PATCH v1 21/28] docs: remove references to TCG tracing Alex Bennée
@ 2021-10-26 20:10 ` Richard Henderson
0 siblings, 0 replies; 61+ messages in thread
From: Richard Henderson @ 2021-10-26 20:10 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, pbonzini, f4bug, robhenry, aaron, Luis Vilanova,
mahmoudabdalghany, minyihh, cota, stefanha, crosa, kuhn.chenqun,
ma.mandourr, aurelien
On 10/26/21 3:22 AM, Alex Bennée wrote:
> Users wanting this sort of functionality should turn to TCG plugins
> instead.
>
> Signed-off-by: Alex Bennée<alex.bennee@linaro.org>
> Cc: Luis Vilanova<vilanova@imperial.ac.uk>
> Cc: Stefan Hajnoczi<stefanha@redhat.com>
> ---
> docs/about/removed-features.rst | 13 +++++
> docs/devel/tracing.rst | 85 ---------------------------------
> 2 files changed, 13 insertions(+), 85 deletions(-)
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
r~
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 22/28] tracing: remove TCG memory access tracing
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (20 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 21/28] docs: remove references to TCG tracing Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 20:15 ` Richard Henderson
2021-10-26 10:22 ` [PATCH v1 23/28] tracing: remove the trace-tcg includes from the build Alex Bennée
` (5 subsequent siblings)
27 siblings, 1 reply; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, Riku Voipio,
Richard Henderson, f4bug, robhenry, Luis Vilanova,
mahmoudabdalghany, aaron, cota, stefanha, crosa, pbonzini,
ma.mandourr, Alex Bennée, aurelien
If you really want to trace all memory operations TCG plugins gives
you a more flexible interface for doing so.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Luis Vilanova <vilanova@imperial.ac.uk>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
---
accel/tcg/atomic_template.h | 12 ------------
accel/tcg/cputlb.c | 2 --
accel/tcg/user-exec.c | 15 +--------------
tcg/tcg-op.c | 5 -----
accel/tcg/atomic_common.c.inc | 20 --------------------
trace-events | 14 --------------
6 files changed, 1 insertion(+), 67 deletions(-)
diff --git a/accel/tcg/atomic_template.h b/accel/tcg/atomic_template.h
index 2d917b6b1f..fc165031e8 100644
--- a/accel/tcg/atomic_template.h
+++ b/accel/tcg/atomic_template.h
@@ -77,7 +77,6 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr,
PAGE_READ | PAGE_WRITE, retaddr);
DATA_TYPE ret;
- atomic_trace_rmw_pre(env, addr, oi);
#if DATA_SIZE == 16
ret = atomic16_cmpxchg(haddr, cmpv, newv);
#else
@@ -97,7 +96,6 @@ ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr,
PAGE_READ, retaddr);
DATA_TYPE val;
- atomic_trace_ld_pre(env, addr, oi);
val = atomic16_read(haddr);
ATOMIC_MMU_CLEANUP;
atomic_trace_ld_post(env, addr, oi);
@@ -110,7 +108,6 @@ void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, ABI_TYPE val,
DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE,
PAGE_WRITE, retaddr);
- atomic_trace_st_pre(env, addr, oi);
atomic16_set(haddr, val);
ATOMIC_MMU_CLEANUP;
atomic_trace_st_post(env, addr, oi);
@@ -124,7 +121,6 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val,
PAGE_READ | PAGE_WRITE, retaddr);
DATA_TYPE ret;
- atomic_trace_rmw_pre(env, addr, oi);
ret = qatomic_xchg__nocheck(haddr, val);
ATOMIC_MMU_CLEANUP;
atomic_trace_rmw_post(env, addr, oi);
@@ -138,7 +134,6 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr, \
DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, \
PAGE_READ | PAGE_WRITE, retaddr); \
DATA_TYPE ret; \
- atomic_trace_rmw_pre(env, addr, oi); \
ret = qatomic_##X(haddr, val); \
ATOMIC_MMU_CLEANUP; \
atomic_trace_rmw_post(env, addr, oi); \
@@ -171,7 +166,6 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr, \
XDATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, \
PAGE_READ | PAGE_WRITE, retaddr); \
XDATA_TYPE cmp, old, new, val = xval; \
- atomic_trace_rmw_pre(env, addr, oi); \
smp_mb(); \
cmp = qatomic_read__nocheck(haddr); \
do { \
@@ -216,7 +210,6 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr,
PAGE_READ | PAGE_WRITE, retaddr);
DATA_TYPE ret;
- atomic_trace_rmw_pre(env, addr, oi);
#if DATA_SIZE == 16
ret = atomic16_cmpxchg(haddr, BSWAP(cmpv), BSWAP(newv));
#else
@@ -236,7 +229,6 @@ ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr,
PAGE_READ, retaddr);
DATA_TYPE val;
- atomic_trace_ld_pre(env, addr, oi);
val = atomic16_read(haddr);
ATOMIC_MMU_CLEANUP;
atomic_trace_ld_post(env, addr, oi);
@@ -249,7 +241,6 @@ void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, ABI_TYPE val,
DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE,
PAGE_WRITE, retaddr);
- atomic_trace_st_pre(env, addr, oi);
val = BSWAP(val);
atomic16_set(haddr, val);
ATOMIC_MMU_CLEANUP;
@@ -264,7 +255,6 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val,
PAGE_READ | PAGE_WRITE, retaddr);
ABI_TYPE ret;
- atomic_trace_rmw_pre(env, addr, oi);
ret = qatomic_xchg__nocheck(haddr, BSWAP(val));
ATOMIC_MMU_CLEANUP;
atomic_trace_rmw_post(env, addr, oi);
@@ -278,7 +268,6 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr, \
DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, \
PAGE_READ | PAGE_WRITE, retaddr); \
DATA_TYPE ret; \
- atomic_trace_rmw_pre(env, addr, oi); \
ret = qatomic_##X(haddr, BSWAP(val)); \
ATOMIC_MMU_CLEANUP; \
atomic_trace_rmw_post(env, addr, oi); \
@@ -308,7 +297,6 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr, \
XDATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, \
PAGE_READ | PAGE_WRITE, retaddr); \
XDATA_TYPE ldo, ldn, old, new, val = xval; \
- atomic_trace_rmw_pre(env, addr, oi); \
smp_mb(); \
ldn = qatomic_read__nocheck(haddr); \
do { \
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index b69a953447..6899e4fe72 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -2140,7 +2140,6 @@ static inline uint64_t cpu_load_helper(CPUArchState *env, abi_ptr addr,
{
uint64_t ret;
- trace_guest_ld_before_exec(env_cpu(env), addr, oi);
ret = full_load(env, addr, oi, retaddr);
qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R);
return ret;
@@ -2487,7 +2486,6 @@ static inline void cpu_store_helper(CPUArchState *env, target_ulong addr,
uint64_t val, MemOpIdx oi, uintptr_t ra,
FullStoreHelper *full_store)
{
- trace_guest_st_before_exec(env_cpu(env), addr, oi);
full_store(env, addr, val, oi, ra);
qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W);
}
diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
index e6bb29b42d..7e10634580 100644
--- a/accel/tcg/user-exec.c
+++ b/accel/tcg/user-exec.c
@@ -920,7 +920,6 @@ uint8_t cpu_ldb_mmu(CPUArchState *env, abi_ptr addr,
uint8_t ret;
validate_memop(oi, MO_UB);
- trace_guest_ld_before_exec(env_cpu(env), addr, oi);
haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD);
ret = ldub_p(haddr);
clear_helper_retaddr();
@@ -935,7 +934,6 @@ uint16_t cpu_ldw_be_mmu(CPUArchState *env, abi_ptr addr,
uint16_t ret;
validate_memop(oi, MO_BEUW);
- trace_guest_ld_before_exec(env_cpu(env), addr, oi);
haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD);
ret = lduw_be_p(haddr);
clear_helper_retaddr();
@@ -950,7 +948,6 @@ uint32_t cpu_ldl_be_mmu(CPUArchState *env, abi_ptr addr,
uint32_t ret;
validate_memop(oi, MO_BEUL);
- trace_guest_ld_before_exec(env_cpu(env), addr, oi);
haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD);
ret = ldl_be_p(haddr);
clear_helper_retaddr();
@@ -965,7 +962,7 @@ uint64_t cpu_ldq_be_mmu(CPUArchState *env, abi_ptr addr,
uint64_t ret;
validate_memop(oi, MO_BEQ);
- trace_guest_ld_before_exec(env_cpu(env), addr, oi);
+
haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD);
ret = ldq_be_p(haddr);
clear_helper_retaddr();
@@ -980,7 +977,6 @@ uint16_t cpu_ldw_le_mmu(CPUArchState *env, abi_ptr addr,
uint16_t ret;
validate_memop(oi, MO_LEUW);
- trace_guest_ld_before_exec(env_cpu(env), addr, oi);
haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD);
ret = lduw_le_p(haddr);
clear_helper_retaddr();
@@ -995,7 +991,6 @@ uint32_t cpu_ldl_le_mmu(CPUArchState *env, abi_ptr addr,
uint32_t ret;
validate_memop(oi, MO_LEUL);
- trace_guest_ld_before_exec(env_cpu(env), addr, oi);
haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD);
ret = ldl_le_p(haddr);
clear_helper_retaddr();
@@ -1010,7 +1005,6 @@ uint64_t cpu_ldq_le_mmu(CPUArchState *env, abi_ptr addr,
uint64_t ret;
validate_memop(oi, MO_LEQ);
- trace_guest_ld_before_exec(env_cpu(env), addr, oi);
haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD);
ret = ldq_le_p(haddr);
clear_helper_retaddr();
@@ -1024,7 +1018,6 @@ void cpu_stb_mmu(CPUArchState *env, abi_ptr addr, uint8_t val,
void *haddr;
validate_memop(oi, MO_UB);
- trace_guest_st_before_exec(env_cpu(env), addr, oi);
haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE);
stb_p(haddr, val);
clear_helper_retaddr();
@@ -1037,7 +1030,6 @@ void cpu_stw_be_mmu(CPUArchState *env, abi_ptr addr, uint16_t val,
void *haddr;
validate_memop(oi, MO_BEUW);
- trace_guest_st_before_exec(env_cpu(env), addr, oi);
haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE);
stw_be_p(haddr, val);
clear_helper_retaddr();
@@ -1050,7 +1042,6 @@ void cpu_stl_be_mmu(CPUArchState *env, abi_ptr addr, uint32_t val,
void *haddr;
validate_memop(oi, MO_BEUL);
- trace_guest_st_before_exec(env_cpu(env), addr, oi);
haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE);
stl_be_p(haddr, val);
clear_helper_retaddr();
@@ -1063,7 +1054,6 @@ void cpu_stq_be_mmu(CPUArchState *env, abi_ptr addr, uint64_t val,
void *haddr;
validate_memop(oi, MO_BEQ);
- trace_guest_st_before_exec(env_cpu(env), addr, oi);
haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE);
stq_be_p(haddr, val);
clear_helper_retaddr();
@@ -1076,7 +1066,6 @@ void cpu_stw_le_mmu(CPUArchState *env, abi_ptr addr, uint16_t val,
void *haddr;
validate_memop(oi, MO_LEUW);
- trace_guest_st_before_exec(env_cpu(env), addr, oi);
haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE);
stw_le_p(haddr, val);
clear_helper_retaddr();
@@ -1089,7 +1078,6 @@ void cpu_stl_le_mmu(CPUArchState *env, abi_ptr addr, uint32_t val,
void *haddr;
validate_memop(oi, MO_LEUL);
- trace_guest_st_before_exec(env_cpu(env), addr, oi);
haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE);
stl_le_p(haddr, val);
clear_helper_retaddr();
@@ -1102,7 +1090,6 @@ void cpu_stq_le_mmu(CPUArchState *env, abi_ptr addr, uint64_t val,
void *haddr;
validate_memop(oi, MO_LEQ);
- trace_guest_st_before_exec(env_cpu(env), addr, oi);
haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_STORE);
stq_le_p(haddr, val);
clear_helper_retaddr();
diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c
index 61b492d89f..65e1c94c2d 100644
--- a/tcg/tcg-op.c
+++ b/tcg/tcg-op.c
@@ -27,7 +27,6 @@
#include "tcg/tcg.h"
#include "tcg/tcg-op.h"
#include "tcg/tcg-mo.h"
-#include "trace-tcg.h"
#include "exec/plugin-gen.h"
/* Reduce the number of ifdefs below. This assumes that all uses of
@@ -2877,7 +2876,6 @@ void tcg_gen_qemu_ld_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop)
tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD);
memop = tcg_canonicalize_memop(memop, 0, 0);
oi = make_memop_idx(memop, idx);
- trace_guest_ld_before_tcg(tcg_ctx->cpu, cpu_env, addr, oi);
orig_memop = memop;
if (!TCG_TARGET_HAS_MEMORY_BSWAP && (memop & MO_BSWAP)) {
@@ -2916,7 +2914,6 @@ void tcg_gen_qemu_st_i32(TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop)
tcg_gen_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST);
memop = tcg_canonicalize_memop(memop, 0, 1);
oi = make_memop_idx(memop, idx);
- trace_guest_st_before_tcg(tcg_ctx->cpu, cpu_env, addr, oi);
if (!TCG_TARGET_HAS_MEMORY_BSWAP && (memop & MO_BSWAP)) {
swap = tcg_temp_new_i32();
@@ -2965,7 +2962,6 @@ void tcg_gen_qemu_ld_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop)
tcg_gen_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD);
memop = tcg_canonicalize_memop(memop, 1, 0);
oi = make_memop_idx(memop, idx);
- trace_guest_ld_before_tcg(tcg_ctx->cpu, cpu_env, addr, oi);
orig_memop = memop;
if (!TCG_TARGET_HAS_MEMORY_BSWAP && (memop & MO_BSWAP)) {
@@ -3013,7 +3009,6 @@ void tcg_gen_qemu_st_i64(TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop)
tcg_gen_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST);
memop = tcg_canonicalize_memop(memop, 1, 1);
oi = make_memop_idx(memop, idx);
- trace_guest_st_before_tcg(tcg_ctx->cpu, cpu_env, addr, oi);
if (!TCG_TARGET_HAS_MEMORY_BSWAP && (memop & MO_BSWAP)) {
swap = tcg_temp_new_i64();
diff --git a/accel/tcg/atomic_common.c.inc b/accel/tcg/atomic_common.c.inc
index 1df1f243e9..6602d7689f 100644
--- a/accel/tcg/atomic_common.c.inc
+++ b/accel/tcg/atomic_common.c.inc
@@ -13,14 +13,6 @@
* See the COPYING file in the top-level directory.
*/
-static void atomic_trace_rmw_pre(CPUArchState *env, target_ulong addr,
- MemOpIdx oi)
-{
- CPUState *cpu = env_cpu(env);
-
- trace_guest_rmw_before_exec(cpu, addr, oi);
-}
-
static void atomic_trace_rmw_post(CPUArchState *env, target_ulong addr,
MemOpIdx oi)
{
@@ -28,24 +20,12 @@ static void atomic_trace_rmw_post(CPUArchState *env, target_ulong addr,
}
#if HAVE_ATOMIC128
-static void atomic_trace_ld_pre(CPUArchState *env, target_ulong addr,
- MemOpIdx oi)
-{
- trace_guest_ld_before_exec(env_cpu(env), addr, oi);
-}
-
static void atomic_trace_ld_post(CPUArchState *env, target_ulong addr,
MemOpIdx oi)
{
qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R);
}
-static void atomic_trace_st_pre(CPUArchState *env, target_ulong addr,
- MemOpIdx oi)
-{
- trace_guest_st_before_exec(env_cpu(env), addr, oi);
-}
-
static void atomic_trace_st_post(CPUArchState *env, target_ulong addr,
MemOpIdx oi)
{
diff --git a/trace-events b/trace-events
index a637a61eba..bc71006675 100644
--- a/trace-events
+++ b/trace-events
@@ -117,20 +117,6 @@ vcpu guest_cpu_exit(void)
# Targets: all
vcpu guest_cpu_reset(void)
-# tcg/tcg-op.c
-
-# @vaddr: Access' virtual address.
-# @memopidx: Access' information (see below).
-#
-# Start virtual memory access (before any potential access violation).
-# Does not include memory accesses performed by devices.
-#
-# Mode: user, softmmu
-# Targets: TCG(all)
-vcpu tcg guest_ld_before(TCGv vaddr, uint32_t memopidx) "info=%d", "vaddr=0x%016"PRIx64" memopidx=0x%x"
-vcpu tcg guest_st_before(TCGv vaddr, uint32_t memopidx) "info=%d", "vaddr=0x%016"PRIx64" memopidx=0x%x"
-vcpu tcg guest_rmw_before(TCGv vaddr, uint32_t memopidx) "info=%d", "vaddr=0x%016"PRIx64" memopidx=0x%x"
-
# include/user/syscall-trace.h
# @num: System call number.
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 22/28] tracing: remove TCG memory access tracing
2021-10-26 10:22 ` [PATCH v1 22/28] tracing: remove TCG memory access tracing Alex Bennée
@ 2021-10-26 20:15 ` Richard Henderson
0 siblings, 0 replies; 61+ messages in thread
From: Richard Henderson @ 2021-10-26 20:15 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, Riku Voipio, f4bug,
robhenry, Luis Vilanova, mahmoudabdalghany, aaron, cota,
stefanha, crosa, pbonzini, ma.mandourr, aurelien
On 10/26/21 3:22 AM, Alex Bennée wrote:
> @@ -950,7 +948,6 @@ uint32_t cpu_ldl_be_mmu(CPUArchState *env, abi_ptr addr,
> uint32_t ret;
>
> validate_memop(oi, MO_BEUL);
> - trace_guest_ld_before_exec(env_cpu(env), addr, oi);
> haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD);
> ret = ldl_be_p(haddr);
> clear_helper_retaddr();
> @@ -965,7 +962,7 @@ uint64_t cpu_ldq_be_mmu(CPUArchState *env, abi_ptr addr,
> uint64_t ret;
>
> validate_memop(oi, MO_BEQ);
> - trace_guest_ld_before_exec(env_cpu(env), addr, oi);
> +
> haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD);
> ret = ldq_be_p(haddr);
> clear_helper_retaddr();
> @@ -980,7 +977,6 @@ uint16_t cpu_ldw_le_mmu(CPUArchState *env, abi_ptr addr,
> uint16_t ret;
>
> validate_memop(oi, MO_LEUW);
> - trace_guest_ld_before_exec(env_cpu(env), addr, oi);
> haddr = cpu_mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD);
> ret = lduw_le_p(haddr);
> clear_helper_retaddr();
In that one instance you add an extra line. Otherwise,
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
r~
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 23/28] tracing: remove the trace-tcg includes from the build
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (21 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 22/28] tracing: remove TCG memory access tracing Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 20:20 ` Richard Henderson
2021-10-26 10:22 ` [PATCH v1 24/28] tracing: excise the tcg related from tracetool Alex Bennée
` (4 subsequent siblings)
27 siblings, 1 reply; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
Luis Vilanova, mahmoudabdalghany, aaron, cota, stefanha, crosa,
pbonzini, ma.mandourr, Alex Bennée, aurelien
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Luis Vilanova <vilanova@imperial.ac.uk>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
---
include/trace-tcg.h | 6 ------
trace/meson.build | 1 -
2 files changed, 7 deletions(-)
delete mode 100644 include/trace-tcg.h
diff --git a/include/trace-tcg.h b/include/trace-tcg.h
deleted file mode 100644
index da68608c85..0000000000
--- a/include/trace-tcg.h
+++ /dev/null
@@ -1,6 +0,0 @@
-#ifndef TRACE_TCG_H
-#define TRACE_TCG_H
-
-#include "trace/generated-tcg-tracers.h"
-
-#endif /* TRACE_TCG_H */
diff --git a/trace/meson.build b/trace/meson.build
index 573dd699c6..e279e33d4d 100644
--- a/trace/meson.build
+++ b/trace/meson.build
@@ -62,7 +62,6 @@ trace_events_all = custom_target('trace-events-all',
install_dir: qemu_datadir)
foreach d : [
- ['generated-tcg-tracers.h', 'tcg-h'],
['generated-helpers.c', 'tcg-helper-c'],
['generated-helpers.h', 'tcg-helper-h'],
['generated-helpers-wrappers.h', 'tcg-helper-wrapper-h'],
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 23/28] tracing: remove the trace-tcg includes from the build
2021-10-26 10:22 ` [PATCH v1 23/28] tracing: remove the trace-tcg includes from the build Alex Bennée
@ 2021-10-26 20:20 ` Richard Henderson
0 siblings, 0 replies; 61+ messages in thread
From: Richard Henderson @ 2021-10-26 20:20 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, pbonzini, f4bug, robhenry, aaron, Luis Vilanova,
mahmoudabdalghany, minyihh, cota, stefanha, crosa, kuhn.chenqun,
ma.mandourr, aurelien
On 10/26/21 3:22 AM, Alex Bennée wrote:
> Signed-off-by: Alex Bennée<alex.bennee@linaro.org>
> Cc: Luis Vilanova<vilanova@imperial.ac.uk>
> Cc: Stefan Hajnoczi<stefanha@redhat.com>
> ---
> include/trace-tcg.h | 6 ------
> trace/meson.build | 1 -
> 2 files changed, 7 deletions(-)
> delete mode 100644 include/trace-tcg.h
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
r~
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 24/28] tracing: excise the tcg related from tracetool
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (22 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 23/28] tracing: remove the trace-tcg includes from the build Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 20:23 ` Richard Henderson
2021-10-26 10:22 ` [PATCH v1 25/28] plugins: add helper functions for coverage plugins Alex Bennée
` (3 subsequent siblings)
27 siblings, 1 reply; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, Richard Henderson, f4bug,
robhenry, Luis Vilanova, mahmoudabdalghany, aaron, cota,
stefanha, crosa, pbonzini, ma.mandourr, Alex Bennée,
aurelien
Now we have no TCG trace events and no longer handle them in the code
we can remove the handling from the tracetool to generate them. vcpu
tracing is still available although the existing syscall event is an
exercise in redundancy (plugins and -strace can also get the
information).
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Luis Vilanova <vilanova@imperial.ac.uk>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
---
meson.build | 4 -
include/exec/helper-gen.h | 2 -
include/exec/helper-proto.h | 1 -
include/exec/helper-tcg.h | 1 -
scripts/tracetool/__init__.py | 41 +--------
scripts/tracetool/format/tcg_h.py | 83 -------------------
scripts/tracetool/format/tcg_helper_c.py | 79 ------------------
scripts/tracetool/format/tcg_helper_h.py | 48 -----------
.../tracetool/format/tcg_helper_wrapper_h.py | 70 ----------------
scripts/tracetool/vcpu.py | 14 +---
trace/meson.build | 13 ---
11 files changed, 4 insertions(+), 352 deletions(-)
delete mode 100644 scripts/tracetool/format/tcg_h.py
delete mode 100644 scripts/tracetool/format/tcg_helper_c.py
delete mode 100644 scripts/tracetool/format/tcg_helper_h.py
delete mode 100644 scripts/tracetool/format/tcg_helper_wrapper_h.py
diff --git a/meson.build b/meson.build
index bc520b579c..c2d0d064cd 100644
--- a/meson.build
+++ b/meson.build
@@ -2304,19 +2304,15 @@ tracetool_depends = files(
'scripts/tracetool/backend/simple.py',
'scripts/tracetool/backend/syslog.py',
'scripts/tracetool/backend/ust.py',
- 'scripts/tracetool/format/tcg_h.py',
'scripts/tracetool/format/ust_events_c.py',
'scripts/tracetool/format/ust_events_h.py',
'scripts/tracetool/format/__init__.py',
'scripts/tracetool/format/d.py',
- 'scripts/tracetool/format/tcg_helper_c.py',
'scripts/tracetool/format/simpletrace_stap.py',
'scripts/tracetool/format/c.py',
'scripts/tracetool/format/h.py',
- 'scripts/tracetool/format/tcg_helper_h.py',
'scripts/tracetool/format/log_stap.py',
'scripts/tracetool/format/stap.py',
- 'scripts/tracetool/format/tcg_helper_wrapper_h.py',
'scripts/tracetool/__init__.py',
'scripts/tracetool/transform.py',
'scripts/tracetool/vcpu.py'
diff --git a/include/exec/helper-gen.h b/include/exec/helper-gen.h
index 1c2e7a8ed3..7b6ca975ef 100644
--- a/include/exec/helper-gen.h
+++ b/include/exec/helper-gen.h
@@ -79,8 +79,6 @@ static inline void glue(gen_helper_, name)(dh_retvar_decl(ret) \
}
#include "helper.h"
-#include "trace/generated-helpers.h"
-#include "trace/generated-helpers-wrappers.h"
#include "accel/tcg/tcg-runtime.h"
#include "accel/tcg/plugin-helpers.h"
diff --git a/include/exec/helper-proto.h b/include/exec/helper-proto.h
index ba100793a7..c4b1bda632 100644
--- a/include/exec/helper-proto.h
+++ b/include/exec/helper-proto.h
@@ -38,7 +38,6 @@ dh_ctype(ret) HELPER(name) (dh_ctype(t1), dh_ctype(t2), dh_ctype(t3), \
#define IN_HELPER_PROTO
#include "helper.h"
-#include "trace/generated-helpers.h"
#include "accel/tcg/tcg-runtime.h"
#include "accel/tcg/plugin-helpers.h"
diff --git a/include/exec/helper-tcg.h b/include/exec/helper-tcg.h
index 16cd318b83..3933258f1a 100644
--- a/include/exec/helper-tcg.h
+++ b/include/exec/helper-tcg.h
@@ -59,7 +59,6 @@
| dh_typemask(t5, 5) | dh_typemask(t6, 6) | dh_typemask(t7, 7) },
#include "helper.h"
-#include "trace/generated-helpers.h"
#include "accel/tcg/tcg-runtime.h"
#include "accel/tcg/plugin-helpers.h"
diff --git a/scripts/tracetool/__init__.py b/scripts/tracetool/__init__.py
index 5bc94d95cf..5393c7fc5c 100644
--- a/scripts/tracetool/__init__.py
+++ b/scripts/tracetool/__init__.py
@@ -87,8 +87,6 @@ def out(*lines, **kwargs):
"ssize_t",
"uintptr_t",
"ptrdiff_t",
- # Magic substitution is done by tracetool
- "TCGv",
]
def validate_type(name):
@@ -232,7 +230,7 @@ class Event(object):
"(?:(?:(?P<fmt_trans>\".+),)?\s*(?P<fmt>\".+))?"
"\s*")
- _VALID_PROPS = set(["disable", "tcg", "tcg-trans", "tcg-exec", "vcpu"])
+ _VALID_PROPS = set(["disable", "vcpu"])
def __init__(self, name, props, fmt, args, lineno, filename, orig=None,
event_trans=None, event_exec=None):
@@ -321,15 +319,6 @@ def build(line_str, lineno, filename):
fmt = [fmt_trans, fmt]
args = Arguments.build(groups["args"])
- if "tcg-trans" in props:
- raise ValueError("Invalid property 'tcg-trans'")
- if "tcg-exec" in props:
- raise ValueError("Invalid property 'tcg-exec'")
- if "tcg" not in props and not isinstance(fmt, str):
- raise ValueError("Only events with 'tcg' property can have two format strings")
- if "tcg" in props and isinstance(fmt, str):
- raise ValueError("Events with 'tcg' property must have two format strings")
-
event = Event(name, props, fmt, args, lineno, filename)
# add implicit arguments when using the 'vcpu' property
@@ -409,33 +398,7 @@ def read_events(fobj, fname):
e.args = (arg0,) + e.args[1:]
raise
- # transform TCG-enabled events
- if "tcg" not in event.properties:
- events.append(event)
- else:
- event_trans = event.copy()
- event_trans.name += "_trans"
- event_trans.properties += ["tcg-trans"]
- event_trans.fmt = event.fmt[0]
- # ignore TCG arguments
- args_trans = []
- for atrans, aorig in zip(
- event_trans.transform(tracetool.transform.TCG_2_HOST).args,
- event.args):
- if atrans == aorig:
- args_trans.append(atrans)
- event_trans.args = Arguments(args_trans)
-
- event_exec = event.copy()
- event_exec.name += "_exec"
- event_exec.properties += ["tcg-exec"]
- event_exec.fmt = event.fmt[1]
- event_exec.args = event_exec.args.transform(tracetool.transform.TCG_2_HOST)
-
- new_event = [event_trans, event_exec]
- event.event_trans, event.event_exec = new_event
-
- events.extend(new_event)
+ events.append(event)
return events
diff --git a/scripts/tracetool/format/tcg_h.py b/scripts/tracetool/format/tcg_h.py
deleted file mode 100644
index 4d84440aff..0000000000
--- a/scripts/tracetool/format/tcg_h.py
+++ /dev/null
@@ -1,83 +0,0 @@
-# -*- coding: utf-8 -*-
-
-"""
-Generate .h file for TCG code generation.
-"""
-
-__author__ = "Lluís Vilanova <vilanova@ac.upc.edu>"
-__copyright__ = "Copyright 2012-2017, Lluís Vilanova <vilanova@ac.upc.edu>"
-__license__ = "GPL version 2 or (at your option) any later version"
-
-__maintainer__ = "Stefan Hajnoczi"
-__email__ = "stefanha@redhat.com"
-
-
-from tracetool import out, Arguments
-import tracetool.vcpu
-
-
-def vcpu_transform_args(args):
- assert len(args) == 1
- return Arguments([
- args,
- # NOTE: this name must be kept in sync with the one in "tcg_h"
- # NOTE: Current helper code uses TCGv_env (CPUArchState*)
- ("TCGv_env", "__tcg_" + args.names()[0]),
- ])
-
-
-def generate(events, backend, group):
- if group == "root":
- header = "trace/trace-root.h"
- else:
- header = "trace.h"
-
- out('/* This file is autogenerated by tracetool, do not edit. */',
- '/* You must include this file after the inclusion of helper.h */',
- '',
- '#ifndef TRACE_%s_GENERATED_TCG_TRACERS_H' % group.upper(),
- '#define TRACE_%s_GENERATED_TCG_TRACERS_H' % group.upper(),
- '',
- '#include "exec/helper-proto.h"',
- '#include "%s"' % header,
- '',
- )
-
- for e in events:
- # just keep one of them
- if "tcg-exec" not in e.properties:
- continue
-
- out('static inline void %(name_tcg)s(%(args)s)',
- '{',
- name_tcg=e.original.api(e.QEMU_TRACE_TCG),
- args=tracetool.vcpu.transform_args("tcg_h", e.original))
-
- if "disable" not in e.properties:
- args_trans = e.original.event_trans.args
- args_exec = tracetool.vcpu.transform_args(
- "tcg_helper_c", e.original.event_exec, "wrapper")
- if "vcpu" in e.properties:
- trace_cpu = e.args.names()[0]
- cond = "trace_event_get_vcpu_state(%(cpu)s,"\
- " TRACE_%(id)s)"\
- % dict(
- cpu=trace_cpu,
- id=e.original.event_exec.name.upper())
- else:
- cond = "true"
-
- out(' %(name_trans)s(%(argnames_trans)s);',
- ' if (%(cond)s) {',
- ' gen_helper_%(name_exec)s(%(argnames_exec)s);',
- ' }',
- name_trans=e.original.event_trans.api(e.QEMU_TRACE),
- name_exec=e.original.event_exec.api(e.QEMU_TRACE),
- argnames_trans=", ".join(args_trans.names()),
- argnames_exec=", ".join(args_exec.names()),
- cond=cond)
-
- out('}')
-
- out('',
- '#endif /* TRACE_%s_GENERATED_TCG_TRACERS_H */' % group.upper())
diff --git a/scripts/tracetool/format/tcg_helper_c.py b/scripts/tracetool/format/tcg_helper_c.py
deleted file mode 100644
index 72576e67d1..0000000000
--- a/scripts/tracetool/format/tcg_helper_c.py
+++ /dev/null
@@ -1,79 +0,0 @@
-# -*- coding: utf-8 -*-
-
-"""
-Generate trace/generated-helpers.c.
-"""
-
-__author__ = "Lluís Vilanova <vilanova@ac.upc.edu>"
-__copyright__ = "Copyright 2012-2017, Lluís Vilanova <vilanova@ac.upc.edu>"
-__license__ = "GPL version 2 or (at your option) any later version"
-
-__maintainer__ = "Stefan Hajnoczi"
-__email__ = "stefanha@redhat.com"
-
-
-from tracetool import Arguments, out
-from tracetool.transform import *
-import tracetool.vcpu
-
-
-def vcpu_transform_args(args, mode):
- assert len(args) == 1
- # NOTE: this name must be kept in sync with the one in "tcg_h"
- args = Arguments([(args.types()[0], "__tcg_" + args.names()[0])])
- if mode == "code":
- return Arguments([
- # Does cast from helper requirements to tracing types
- ("CPUState *", "env_cpu(%s)" % args.names()[0]),
- ])
- else:
- args = Arguments([
- # NOTE: Current helper code uses TCGv_env (CPUArchState*)
- ("CPUArchState *", args.names()[0]),
- ])
- if mode == "header":
- return args
- elif mode == "wrapper":
- return args.transform(HOST_2_TCG)
- else:
- assert False
-
-
-def generate(events, backend, group):
- if group == "root":
- header = "trace/trace-root.h"
- else:
- header = "trace.h"
-
- events = [e for e in events
- if "disable" not in e.properties]
-
- out('/* This file is autogenerated by tracetool, do not edit. */',
- '',
- '#include "qemu/osdep.h"',
- '#include "cpu.h"',
- '#include "exec/helper-proto.h"',
- '#include "%s"' % header,
- '',
- )
-
- for e in events:
- if "tcg-exec" not in e.properties:
- continue
-
- e_args_api = tracetool.vcpu.transform_args(
- "tcg_helper_c", e.original, "header").transform(
- HOST_2_TCG_COMPAT, TCG_2_TCG_HELPER_DEF)
- e_args_call = tracetool.vcpu.transform_args(
- "tcg_helper_c", e, "code")
-
- out('void %(name_tcg)s(%(args_api)s)',
- '{',
- # NOTE: the check was already performed at TCG-generation time
- ' %(name)s(%(args_call)s);',
- '}',
- name_tcg="helper_%s_proxy" % e.api(),
- name=e.api(e.QEMU_TRACE_NOCHECK),
- args_api=e_args_api,
- args_call=", ".join(e_args_call.casted()),
- )
diff --git a/scripts/tracetool/format/tcg_helper_h.py b/scripts/tracetool/format/tcg_helper_h.py
deleted file mode 100644
index 08554fbc85..0000000000
--- a/scripts/tracetool/format/tcg_helper_h.py
+++ /dev/null
@@ -1,48 +0,0 @@
-# -*- coding: utf-8 -*-
-
-"""
-Generate trace/generated-helpers.h.
-"""
-
-__author__ = "Lluís Vilanova <vilanova@ac.upc.edu>"
-__copyright__ = "Copyright 2012-2016, Lluís Vilanova <vilanova@ac.upc.edu>"
-__license__ = "GPL version 2 or (at your option) any later version"
-
-__maintainer__ = "Stefan Hajnoczi"
-__email__ = "stefanha@redhat.com"
-
-
-from tracetool import out
-from tracetool.transform import *
-import tracetool.vcpu
-
-
-def generate(events, backend, group):
- events = [e for e in events
- if "disable" not in e.properties]
-
- out('/* This file is autogenerated by tracetool, do not edit. */',
- '',
- )
-
- for e in events:
- if "tcg-exec" not in e.properties:
- continue
-
- # TCG helper proxy declaration
- fmt = "DEF_HELPER_FLAGS_%(argc)d(%(name)s, %(flags)svoid%(types)s)"
- e_args = tracetool.vcpu.transform_args("tcg_helper_c", e.original, "header")
- args = e_args.transform(HOST_2_TCG_COMPAT, HOST_2_TCG,
- TCG_2_TCG_HELPER_DECL)
- types = ", ".join(args.types())
- if types != "":
- types = ", " + types
-
- flags = "TCG_CALL_NO_RWG, "
-
- out(fmt,
- flags=flags,
- argc=len(args),
- name=e.api() + "_proxy",
- types=types,
- )
diff --git a/scripts/tracetool/format/tcg_helper_wrapper_h.py b/scripts/tracetool/format/tcg_helper_wrapper_h.py
deleted file mode 100644
index 0c5a9797d1..0000000000
--- a/scripts/tracetool/format/tcg_helper_wrapper_h.py
+++ /dev/null
@@ -1,70 +0,0 @@
-# -*- coding: utf-8 -*-
-
-"""
-Generate trace/generated-helpers-wrappers.h.
-"""
-
-__author__ = "Lluís Vilanova <vilanova@ac.upc.edu>"
-__copyright__ = "Copyright 2012-2016, Lluís Vilanova <vilanova@ac.upc.edu>"
-__license__ = "GPL version 2 or (at your option) any later version"
-
-__maintainer__ = "Stefan Hajnoczi"
-__email__ = "stefanha@redhat.com"
-
-
-from tracetool import out
-from tracetool.transform import *
-import tracetool.vcpu
-
-
-def generate(events, backend, group):
- events = [e for e in events
- if "disable" not in e.properties]
-
- out('/* This file is autogenerated by tracetool, do not edit. */',
- '',
- '#define tcg_temp_new_nop(v) (v)',
- '#define tcg_temp_free_nop(v)',
- '',
- )
-
- for e in events:
- if "tcg-exec" not in e.properties:
- continue
-
- # tracetool.generate always transforms types to host
- e_args = tracetool.vcpu.transform_args("tcg_helper_c", e.original, "wrapper")
-
- # mixed-type to TCG helper bridge
- args_tcg_compat = e_args.transform(HOST_2_TCG_COMPAT)
-
- code_new = [
- "%(tcg_type)s __%(name)s = %(tcg_func)s(%(name)s);" %
- {"tcg_type": transform_type(type_, HOST_2_TCG),
- "tcg_func": transform_type(type_, HOST_2_TCG_TMP_NEW),
- "name": name}
- for (type_, name) in args_tcg_compat
- ]
-
- code_free = [
- "%(tcg_func)s(__%(name)s);" %
- {"tcg_func": transform_type(type_, HOST_2_TCG_TMP_FREE),
- "name": name}
- for (type_, name) in args_tcg_compat
- ]
-
- gen_name = "gen_helper_" + e.api()
-
- out('static inline void %(name)s(%(args)s)',
- '{',
- ' %(code_new)s',
- ' %(proxy_name)s(%(tmp_names)s);',
- ' %(code_free)s',
- '}',
- name=gen_name,
- args=e_args,
- proxy_name=gen_name + "_proxy",
- code_new="\n ".join(code_new),
- code_free="\n ".join(code_free),
- tmp_names=", ".join(["__%s" % name for _, name in e_args]),
- )
diff --git a/scripts/tracetool/vcpu.py b/scripts/tracetool/vcpu.py
index 868b4cb04c..d232cb1d06 100644
--- a/scripts/tracetool/vcpu.py
+++ b/scripts/tracetool/vcpu.py
@@ -19,19 +19,9 @@
def transform_event(event):
"""Transform event to comply with the 'vcpu' property (if present)."""
if "vcpu" in event.properties:
- # events with 'tcg-trans' and 'tcg-exec' are auto-generated from
- # already-patched events
- assert "tcg-trans" not in event.properties
- assert "tcg-exec" not in event.properties
-
event.args = Arguments([("void *", "__cpu"), event.args])
- if "tcg" in event.properties:
- fmt = "\"cpu=%p \""
- event.fmt = [fmt + event.fmt[0],
- fmt + event.fmt[1]]
- else:
- fmt = "\"cpu=%p \""
- event.fmt = fmt + event.fmt
+ fmt = "\"cpu=%p \""
+ event.fmt = fmt + event.fmt
return event
diff --git a/trace/meson.build b/trace/meson.build
index e279e33d4d..298926036a 100644
--- a/trace/meson.build
+++ b/trace/meson.build
@@ -61,19 +61,6 @@ trace_events_all = custom_target('trace-events-all',
install: true,
install_dir: qemu_datadir)
-foreach d : [
- ['generated-helpers.c', 'tcg-helper-c'],
- ['generated-helpers.h', 'tcg-helper-h'],
- ['generated-helpers-wrappers.h', 'tcg-helper-wrapper-h'],
-]
- gen = custom_target(d[0],
- output: d[0],
- input: meson.project_source_root() / 'trace-events',
- command: [ tracetool, '--group=root', '--format=@0@'.format(d[1]), '@INPUT@', '@OUTPUT@' ],
- depend_files: tracetool_depends)
- specific_ss.add(when: 'CONFIG_TCG', if_true: gen)
-endforeach
-
if 'ust' in get_option('trace_backends')
trace_ust_all_h = custom_target('trace-ust-all.h',
output: 'trace-ust-all.h',
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 24/28] tracing: excise the tcg related from tracetool
2021-10-26 10:22 ` [PATCH v1 24/28] tracing: excise the tcg related from tracetool Alex Bennée
@ 2021-10-26 20:23 ` Richard Henderson
0 siblings, 0 replies; 61+ messages in thread
From: Richard Henderson @ 2021-10-26 20:23 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
Luis Vilanova, mahmoudabdalghany, aaron, cota, stefanha, crosa,
pbonzini, ma.mandourr, aurelien
On 10/26/21 3:22 AM, Alex Bennée wrote:
> diff --git a/include/exec/helper-gen.h b/include/exec/helper-gen.h
> index 1c2e7a8ed3..7b6ca975ef 100644
> --- a/include/exec/helper-gen.h
> +++ b/include/exec/helper-gen.h
> @@ -79,8 +79,6 @@ static inline void glue(gen_helper_, name)(dh_retvar_decl(ret) \
> }
>
> #include "helper.h"
> -#include "trace/generated-helpers.h"
> -#include "trace/generated-helpers-wrappers.h"
> #include "accel/tcg/tcg-runtime.h"
> #include "accel/tcg/plugin-helpers.h"
>
> diff --git a/include/exec/helper-proto.h b/include/exec/helper-proto.h
> index ba100793a7..c4b1bda632 100644
> --- a/include/exec/helper-proto.h
> +++ b/include/exec/helper-proto.h
> @@ -38,7 +38,6 @@ dh_ctype(ret) HELPER(name) (dh_ctype(t1), dh_ctype(t2), dh_ctype(t3), \
> #define IN_HELPER_PROTO
>
> #include "helper.h"
> -#include "trace/generated-helpers.h"
> #include "accel/tcg/tcg-runtime.h"
> #include "accel/tcg/plugin-helpers.h"
>
> diff --git a/include/exec/helper-tcg.h b/include/exec/helper-tcg.h
> index 16cd318b83..3933258f1a 100644
> --- a/include/exec/helper-tcg.h
> +++ b/include/exec/helper-tcg.h
> @@ -59,7 +59,6 @@
> | dh_typemask(t5, 5) | dh_typemask(t6, 6) | dh_typemask(t7, 7) },
>
> #include "helper.h"
> -#include "trace/generated-helpers.h"
> #include "accel/tcg/tcg-runtime.h"
> #include "accel/tcg/plugin-helpers.h"
>
These should have gone with the previous patch.
Otherwise,
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
r~
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 25/28] plugins: add helper functions for coverage plugins
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (23 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 24/28] tracing: excise the tcg related from tracetool Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 20:25 ` Richard Henderson
2021-10-26 10:22 ` [PATCH v1 26/28] contrib/plugins: add a drcov plugin Alex Bennée
` (2 subsequent siblings)
27 siblings, 1 reply; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
Ivanov Arkady, mahmoudabdalghany, aaron, cota, stefanha, crosa,
pbonzini, ma.mandourr, Alexandre Iooss, Alex Bennée,
aurelien
From: Ivanov Arkady <arkadiy.ivanov@ispras.ru>
Which provide information about:
- start_code.
- end_code.
- entry.
- path to the executable binary.
Signed-off-by: Ivanov Arkady <arkadiy.ivanov@ispras.ru>
Message-Id: <163491883461.304355.8210754161847179432.stgit@pc-System-Product-Name>
[AJB: reword title, better descriptions, defaults, rm export, fix include]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
include/qemu/qemu-plugin.h | 34 ++++++++++++++++++++++++++
plugins/api.c | 46 ++++++++++++++++++++++++++++++++++++
plugins/qemu-plugins.symbols | 4 ++++
3 files changed, 84 insertions(+)
diff --git a/include/qemu/qemu-plugin.h b/include/qemu/qemu-plugin.h
index 5f1017201f..535ddbf0ae 100644
--- a/include/qemu/qemu-plugin.h
+++ b/include/qemu/qemu-plugin.h
@@ -590,4 +590,38 @@ void qemu_plugin_outs(const char *string);
*/
bool qemu_plugin_bool_parse(const char *name, const char *val, bool *ret);
+/**
+ * qemu_plugin_path_to_binary() - path to binary file being executed
+ *
+ * Return a string representing the path to the binary. For user-mode
+ * this is the main executable. For system emulation we currently
+ * return NULL. The user should g_free() the string once no longer
+ * needed.
+ */
+const char *qemu_plugin_path_to_binary(void);
+
+/**
+ * qemu_plugin_start_code() - returns start of text segment
+ *
+ * Returns the nominal start address of the main text segment in
+ * user-mode. Currently returns 0 for system emulation.
+ */
+uint64_t qemu_plugin_start_code(void);
+
+/**
+ * qemu_plugin_end_code() - returns end of text segment
+ *
+ * Returns the nominal end address of the main text segment in
+ * user-mode. Currently returns 0 for system emulation.
+ */
+uint64_t qemu_plugin_end_code(void);
+
+/**
+ * qemu_plugin_entry_code() - returns start address for module
+ *
+ * Returns the nominal entry address of the main text segment in
+ * user-mode. Currently returns 0 for system emulation.
+ */
+uint64_t qemu_plugin_entry_code(void);
+
#endif /* QEMU_PLUGIN_API_H */
diff --git a/plugins/api.c b/plugins/api.c
index b143b09ce9..e6a2c9dde3 100644
--- a/plugins/api.c
+++ b/plugins/api.c
@@ -44,6 +44,9 @@
#ifndef CONFIG_USER_ONLY
#include "qemu/plugin-memory.h"
#include "hw/boards.h"
+#else
+#include "qemu.h"
+#include "loader.h"
#endif
/* Uninstall and Reset handlers */
@@ -391,3 +394,46 @@ bool qemu_plugin_bool_parse(const char *name, const char *value, bool *ret)
{
return name && value && qapi_bool_parse(name, value, ret, NULL);
}
+
+/*
+ * Binary path, start and end locations
+ */
+const char *qemu_plugin_path_to_binary(void)
+{
+ char *path = NULL;
+#ifdef CONFIG_USER_ONLY
+ TaskState *ts = (TaskState *) current_cpu->opaque;
+ path = g_strdup(ts->bprm->filename);
+#endif
+ return path;
+}
+
+uint64_t qemu_plugin_start_code(void)
+{
+ uint64_t start = 0;
+#ifdef CONFIG_USER_ONLY
+ TaskState *ts = (TaskState *) current_cpu->opaque;
+ start = ts->info->start_code;
+#endif
+ return start;
+}
+
+uint64_t qemu_plugin_end_code(void)
+{
+ uint64_t end = 0;
+#ifdef CONFIG_USER_ONLY
+ TaskState *ts = (TaskState *) current_cpu->opaque;
+ end = ts->info->end_code;
+#endif
+ return end;
+}
+
+uint64_t qemu_plugin_entry_code(void)
+{
+ uint64_t entry = 0;
+#ifdef CONFIG_USER_ONLY
+ TaskState *ts = (TaskState *) current_cpu->opaque;
+ entry = ts->info->entry;
+#endif
+ return entry;
+}
diff --git a/plugins/qemu-plugins.symbols b/plugins/qemu-plugins.symbols
index 4834756ba3..71f6c90549 100644
--- a/plugins/qemu-plugins.symbols
+++ b/plugins/qemu-plugins.symbols
@@ -1,5 +1,7 @@
{
qemu_plugin_bool_parse;
+ qemu_plugin_end_code;
+ qemu_plugin_entry_code;
qemu_plugin_get_hwaddr;
qemu_plugin_hwaddr_device_name;
qemu_plugin_hwaddr_is_io;
@@ -17,6 +19,7 @@
qemu_plugin_n_max_vcpus;
qemu_plugin_n_vcpus;
qemu_plugin_outs;
+ qemu_plugin_path_to_binary;
qemu_plugin_register_atexit_cb;
qemu_plugin_register_flush_cb;
qemu_plugin_register_vcpu_exit_cb;
@@ -33,6 +36,7 @@
qemu_plugin_register_vcpu_tb_exec_inline;
qemu_plugin_register_vcpu_tb_trans_cb;
qemu_plugin_reset;
+ qemu_plugin_start_code;
qemu_plugin_tb_get_insn;
qemu_plugin_tb_n_insns;
qemu_plugin_tb_vaddr;
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 25/28] plugins: add helper functions for coverage plugins
2021-10-26 10:22 ` [PATCH v1 25/28] plugins: add helper functions for coverage plugins Alex Bennée
@ 2021-10-26 20:25 ` Richard Henderson
2021-10-27 16:09 ` Warner Losh
0 siblings, 1 reply; 61+ messages in thread
From: Richard Henderson @ 2021-10-26 20:25 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, pbonzini, crosa, f4bug, robhenry, aaron,
mahmoudabdalghany, minyihh, cota, stefanha, Ivanov Arkady,
kuhn.chenqun, ma.mandourr, Alexandre Iooss, aurelien
On 10/26/21 3:22 AM, Alex Bennée wrote:
> +#else
> +#include "qemu.h"
> +#include "loader.h"
There is no bsd-user/loader.h.
r~
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [PATCH v1 25/28] plugins: add helper functions for coverage plugins
2021-10-26 20:25 ` Richard Henderson
@ 2021-10-27 16:09 ` Warner Losh
2021-10-27 18:58 ` Richard Henderson
0 siblings, 1 reply; 61+ messages in thread
From: Warner Losh @ 2021-10-27 16:09 UTC (permalink / raw)
To: Richard Henderson
Cc: fam, minyihh, berrange, kuhn.chenqun, qemu-devel, robhenry,
f4bug, mahmoudabdalghany, aaron, cota, Ivanov Arkady, stefanha,
crosa, pbonzini, ma.mandourr, Alexandre Iooss, Alex Bennée,
aurelien
> On Oct 26, 2021, at 2:25 PM, Richard Henderson <richard.henderson@linaro.org> wrote:
>
> On 10/26/21 3:22 AM, Alex Bennée wrote:
>> +#else
>> +#include "qemu.h"
>> +#include "loader.h"
>
> There is no bsd-user/loader.h.
Should there be?
Warner
> r~
>
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [PATCH v1 25/28] plugins: add helper functions for coverage plugins
2021-10-27 16:09 ` Warner Losh
@ 2021-10-27 18:58 ` Richard Henderson
2021-10-28 17:09 ` Warner Losh
0 siblings, 1 reply; 61+ messages in thread
From: Richard Henderson @ 2021-10-27 18:58 UTC (permalink / raw)
To: Warner Losh
Cc: fam, minyihh, berrange, kuhn.chenqun, qemu-devel, robhenry,
f4bug, mahmoudabdalghany, aaron, cota, Ivanov Arkady, stefanha,
crosa, pbonzini, ma.mandourr, Alexandre Iooss, Alex Bennée,
aurelien
On 10/27/21 9:09 AM, Warner Losh wrote:
>
>
>> On Oct 26, 2021, at 2:25 PM, Richard Henderson <richard.henderson@linaro.org> wrote:
>>
>> On 10/26/21 3:22 AM, Alex Bennée wrote:
>>> +#else
>>> +#include "qemu.h"
>>> +#include "loader.h"
>>
>> There is no bsd-user/loader.h.
>
> Should there be?
Probably. It came with Peter's cleanup of linux-user/qemu.h at
3ad0a76928df01726e5872b8530d8e1eaa1a971d.
What I meant with my over-terse response is that Alex's patch is incomplete and cannot be
applied as-is.
I think we should introduce a new header for sharing these new functions with plugins/.
Perhaps something shared, in include/user/.
r~
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [PATCH v1 25/28] plugins: add helper functions for coverage plugins
2021-10-27 18:58 ` Richard Henderson
@ 2021-10-28 17:09 ` Warner Losh
0 siblings, 0 replies; 61+ messages in thread
From: Warner Losh @ 2021-10-28 17:09 UTC (permalink / raw)
To: Richard Henderson
Cc: Fam Zheng, minyihh, Daniel P. Berrange, Chen Qun,
QEMU Developers, robhenry, Philippe Mathieu-Daudé,
mahmoudabdalghany, aaron, cota, Ivanov Arkady, Stefan Hajnoczi,
Cleber Rosa, Paolo Bonzini, ma.mandourr, Alexandre Iooss,
Alex Bennée, Aurelien Jarno
[-- Attachment #1: Type: text/plain, Size: 1189 bytes --]
On Wed, Oct 27, 2021 at 12:58 PM Richard Henderson <
richard.henderson@linaro.org> wrote:
> On 10/27/21 9:09 AM, Warner Losh wrote:
> >
> >
> >> On Oct 26, 2021, at 2:25 PM, Richard Henderson <
> richard.henderson@linaro.org> wrote:
> >>
> >> On 10/26/21 3:22 AM, Alex Bennée wrote:
> >>> +#else
> >>> +#include "qemu.h"
> >>> +#include "loader.h"
> >>
> >> There is no bsd-user/loader.h.
> >
> > Should there be?
>
> Probably. It came with Peter's cleanup of linux-user/qemu.h at
> 3ad0a76928df01726e5872b8530d8e1eaa1a971d.
>
> What I meant with my over-terse response is that Alex's patch is
> incomplete and cannot be
> applied as-is.
>
> I think we should introduce a new header for sharing these new functions
> with plugins/.
> Perhaps something shared, in include/user/.
>
I like that idea. I think there'd need to be some level of effort to rework
bsd-user
to use a common header (though if it's just the header, not much). What's
harder
is sharing the bulk of the ELF loading code because it's the same
everywhere.
Or I should, sadly, say mostly the same everywhere but the 'mostly' can be
worked out w/o a huge amount of fuss.
Warner
[-- Attachment #2: Type: text/html, Size: 1809 bytes --]
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 26/28] contrib/plugins: add a drcov plugin
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (24 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 25/28] plugins: add helper functions for coverage plugins Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 10:22 ` [PATCH v1 27/28] tests/tcg: remove duplicate EXTRA_RUNS Alex Bennée
2021-10-26 10:22 ` [PATCH v1 28/28] gdbstub: Switch to the thread receiving a signal Alex Bennée
27 siblings, 0 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
Ivanov Arkady, mahmoudabdalghany, aaron, cota, stefanha, crosa,
pbonzini, ma.mandourr, Alexandre Iooss, Alex Bennée,
aurelien
From: Ivanov Arkady <arkadiy.ivanov@ispras.ru>
This patch adds the ability to generate files in drcov format. Primary
goal this script is to have coverage logfiles thatwork in Lighthouse.
Signed-off-by: Ivanov Arkady <arkadiy.ivanov@ispras.ru>
Message-Id: <163491884553.304355.13246023070235438959.stgit@pc-System-Product-Name>
[AJB: use g_ptr_array instead of slist]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
contrib/plugins/drcov.c | 163 +++++++++++++++++++++++++++++++++++++++
contrib/plugins/Makefile | 1 +
2 files changed, 164 insertions(+)
create mode 100644 contrib/plugins/drcov.c
diff --git a/contrib/plugins/drcov.c b/contrib/plugins/drcov.c
new file mode 100644
index 0000000000..b4a855adaf
--- /dev/null
+++ b/contrib/plugins/drcov.c
@@ -0,0 +1,163 @@
+/*
+ * Copyright (C) 2021, Ivanov Arkady <arkadiy.ivanov@ispras.ru>
+ *
+ * Drcov - a DynamoRIO-based tool that collects coverage information
+ * from a binary. Primary goal this script is to have coverage log
+ * files that work in Lighthouse.
+ *
+ * License: GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include <inttypes.h>
+#include <assert.h>
+#include <stdlib.h>
+#include <inttypes.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdio.h>
+#include <glib.h>
+
+#include <qemu-plugin.h>
+
+QEMU_PLUGIN_EXPORT int qemu_plugin_version = QEMU_PLUGIN_VERSION;
+
+static char header[] = "DRCOV VERSION: 2\n"
+ "DRCOV FLAVOR: drcov-64\n"
+ "Module Table: version 2, count 1\n"
+ "Columns: id, base, end, entry, path\n";
+
+static FILE *fp;
+static const char *file_name = "file.drcov.trace";
+static GMutex lock;
+
+typedef struct {
+ uint32_t start;
+ uint16_t size;
+ uint16_t mod_id;
+ bool exec;
+} bb_entry_t;
+
+/* Translated blocks */
+static GPtrArray *blocks;
+
+static void printf_header(unsigned long count)
+{
+ fprintf(fp, "%s", header);
+ const char *path = qemu_plugin_path_to_binary();
+ uint64_t start_code = qemu_plugin_start_code();
+ uint64_t end_code = qemu_plugin_end_code();
+ uint64_t entry = qemu_plugin_entry_code();
+ fprintf(fp, "0, 0x%lx, 0x%lx, 0x%lx, %s\n",
+ start_code, end_code, entry, path);
+ fprintf(fp, "BB Table: %ld bbs\n", count);
+}
+
+static void printf_char_array32(uint32_t data)
+{
+ const uint8_t *bytes = (const uint8_t *)(&data);
+ fwrite(bytes, sizeof(char), sizeof(data), fp);
+}
+
+static void printf_char_array16(uint16_t data)
+{
+ const uint8_t *bytes = (const uint8_t *)(&data);
+ fwrite(bytes, sizeof(char), sizeof(data), fp);
+}
+
+
+static void printf_el(gpointer data, gpointer user_data)
+{
+ bb_entry_t *bb = (bb_entry_t *)data;
+ if (bb->exec) {
+ printf_char_array32(bb->start);
+ printf_char_array16(bb->size);
+ printf_char_array16(bb->mod_id);
+ }
+ g_free(bb);
+}
+
+static void count_block(gpointer data, gpointer user_data)
+{
+ unsigned long *count = (unsigned long *) user_data;
+ bb_entry_t *bb = (bb_entry_t *)data;
+ if (bb->exec) {
+ *count = *count + 1;
+ }
+}
+
+static void plugin_exit(qemu_plugin_id_t id, void *p)
+{
+ unsigned long count = 0;
+ g_mutex_lock(&lock);
+ g_ptr_array_foreach(blocks, count_block, &count);
+
+ /* Print function */
+ printf_header(count);
+ g_ptr_array_foreach(blocks, printf_el, NULL);
+
+ /* Clear */
+ g_ptr_array_free(blocks, true);
+
+ fclose(fp);
+
+ g_mutex_unlock(&lock);
+}
+
+static void plugin_init(void)
+{
+ fp = fopen(file_name, "wb");
+ blocks = g_ptr_array_sized_new(128);
+}
+
+static void vcpu_tb_exec(unsigned int cpu_index, void *udata)
+{
+ bb_entry_t *bb = (bb_entry_t *) udata;
+
+ g_mutex_lock(&lock);
+ bb->exec = true;
+ g_mutex_unlock(&lock);
+}
+
+static void vcpu_tb_trans(qemu_plugin_id_t id, struct qemu_plugin_tb *tb)
+{
+ uint64_t pc = qemu_plugin_tb_vaddr(tb);
+ size_t n = qemu_plugin_tb_n_insns(tb);
+
+ g_mutex_lock(&lock);
+
+ bb_entry_t *bb = g_new0(bb_entry_t, 1);
+ for (int i = 0; i < n; i++) {
+ bb->size += qemu_plugin_insn_size(qemu_plugin_tb_get_insn(tb, i));
+ }
+
+ bb->start = pc;
+ bb->mod_id = 0;
+ bb->exec = false;
+ g_ptr_array_add(blocks, bb);
+
+ g_mutex_unlock(&lock);
+ qemu_plugin_register_vcpu_tb_exec_cb(tb, vcpu_tb_exec,
+ QEMU_PLUGIN_CB_NO_REGS,
+ (void *)bb);
+
+}
+
+QEMU_PLUGIN_EXPORT
+int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
+ int argc, char **argv)
+{
+ for (int i = 0; i < argc; i++) {
+ g_autofree char **tokens = g_strsplit(argv[i], "=", 2);
+ if (g_strcmp0(tokens[0], "filename") == 0) {
+ file_name = g_strdup(tokens[1]);
+ }
+ }
+
+ plugin_init();
+
+ qemu_plugin_register_vcpu_tb_trans_cb(id, vcpu_tb_trans);
+ qemu_plugin_register_atexit_cb(id, plugin_exit, NULL);
+
+ return 0;
+}
diff --git a/contrib/plugins/Makefile b/contrib/plugins/Makefile
index 54ac5ccd9f..df3499f4f2 100644
--- a/contrib/plugins/Makefile
+++ b/contrib/plugins/Makefile
@@ -20,6 +20,7 @@ NAMES += howvec
NAMES += lockstep
NAMES += hwprofile
NAMES += cache
+NAMES += drcov
SONAMES := $(addsuffix .so,$(addprefix lib,$(NAMES)))
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* [PATCH v1 27/28] tests/tcg: remove duplicate EXTRA_RUNS
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (25 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 26/28] contrib/plugins: add a drcov plugin Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 15:11 ` Philippe Mathieu-Daudé
2021-10-26 20:27 ` Richard Henderson
2021-10-26 10:22 ` [PATCH v1 28/28] gdbstub: Switch to the thread receiving a signal Alex Bennée
27 siblings, 2 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, f4bug, robhenry,
mahmoudabdalghany, aaron, cota, stefanha, crosa, pbonzini,
ma.mandourr, Alex Bennée, aurelien
We set it bellow outside the #if leg.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/tcg/multiarch/Makefile.target | 2 --
1 file changed, 2 deletions(-)
diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target
index 6ccb592aac..c0d9e638e9 100644
--- a/tests/tcg/multiarch/Makefile.target
+++ b/tests/tcg/multiarch/Makefile.target
@@ -61,8 +61,6 @@ run-gdbstub-sha1: sha1
--bin $< --test $(MULTIARCH_SRC)/gdbstub/sha1.py, \
"basic gdbstub support")
-EXTRA_RUNS += run-gdbstub-sha1
-
run-gdbstub-qxfer-auxv-read: sha1
$(call run-test, $@, $(GDB_SCRIPT) \
--gdb $(HAVE_GDB_BIN) \
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 27/28] tests/tcg: remove duplicate EXTRA_RUNS
2021-10-26 10:22 ` [PATCH v1 27/28] tests/tcg: remove duplicate EXTRA_RUNS Alex Bennée
@ 2021-10-26 15:11 ` Philippe Mathieu-Daudé
2021-10-26 20:27 ` Richard Henderson
1 sibling, 0 replies; 61+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-10-26 15:11 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, pbonzini, aaron, robhenry, mahmoudabdalghany,
minyihh, cota, stefanha, crosa, kuhn.chenqun, ma.mandourr,
aurelien
On 10/26/21 12:22, Alex Bennée wrote:
> We set it bellow outside the #if leg.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/tcg/multiarch/Makefile.target | 2 --
> 1 file changed, 2 deletions(-)
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [PATCH v1 27/28] tests/tcg: remove duplicate EXTRA_RUNS
2021-10-26 10:22 ` [PATCH v1 27/28] tests/tcg: remove duplicate EXTRA_RUNS Alex Bennée
2021-10-26 15:11 ` Philippe Mathieu-Daudé
@ 2021-10-26 20:27 ` Richard Henderson
2021-10-27 14:48 ` Alex Bennée
1 sibling, 1 reply; 61+ messages in thread
From: Richard Henderson @ 2021-10-26 20:27 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, pbonzini, f4bug, robhenry, aaron,
mahmoudabdalghany, minyihh, cota, stefanha, crosa, kuhn.chenqun,
ma.mandourr, aurelien
On 10/26/21 3:22 AM, Alex Bennée wrote:
> We set it bellow outside the #if leg.
>
> Signed-off-by: Alex Bennée<alex.bennee@linaro.org>
> ---
> tests/tcg/multiarch/Makefile.target | 2 --
> 1 file changed, 2 deletions(-)
Surely it's better to have them inside the IF?
Why add tests that we're definitely going to SKIP?
r~
^ permalink raw reply [flat|nested] 61+ messages in thread
* Re: [PATCH v1 27/28] tests/tcg: remove duplicate EXTRA_RUNS
2021-10-26 20:27 ` Richard Henderson
@ 2021-10-27 14:48 ` Alex Bennée
0 siblings, 0 replies; 61+ messages in thread
From: Alex Bennée @ 2021-10-27 14:48 UTC (permalink / raw)
To: Richard Henderson
Cc: fam, berrange, pbonzini, aaron, qemu-devel, robhenry, f4bug,
mahmoudabdalghany, minyihh, cota, stefanha, crosa, kuhn.chenqun,
ma.mandourr, aurelien
Richard Henderson <richard.henderson@linaro.org> writes:
> On 10/26/21 3:22 AM, Alex Bennée wrote:
>> We set it bellow outside the #if leg.
>> Signed-off-by: Alex Bennée<alex.bennee@linaro.org>
>> ---
>> tests/tcg/multiarch/Makefile.target | 2 --
>> 1 file changed, 2 deletions(-)
>
> Surely it's better to have them inside the IF?
> Why add tests that we're definitely going to SKIP?
At least this way you get a visual indication of tests that are skipped
(and you might be able to do something about that).
--
Alex Bennée
^ permalink raw reply [flat|nested] 61+ messages in thread
* [PATCH v1 28/28] gdbstub: Switch to the thread receiving a signal
2021-10-26 10:22 [PATCH v1 00/28] testing, plugins and gdbstub for 6.2 Alex Bennée
` (26 preceding siblings ...)
2021-10-26 10:22 ` [PATCH v1 27/28] tests/tcg: remove duplicate EXTRA_RUNS Alex Bennée
@ 2021-10-26 10:22 ` Alex Bennée
2021-10-26 20:34 ` Richard Henderson
27 siblings, 1 reply; 61+ messages in thread
From: Alex Bennée @ 2021-10-26 10:22 UTC (permalink / raw)
To: qemu-devel
Cc: fam, minyihh, berrange, kuhn.chenqun, Philippe Mathieu-Daudé,
f4bug, robhenry, mahmoudabdalghany, aaron, cota, Pavel Labath,
stefanha, crosa, pbonzini, ma.mandourr, Alex Bennée,
aurelien
From: Pavel Labath <pavel@labath.sk>
Respond with Txxthread:yyyy; instead of a plain Sxx to indicate which
thread received the signal. Otherwise, the debugger will associate it
with the main one. Also automatically select this thread, as that is
what gdb expects.
Signed-off-by: Pavel Labath <pavel@labath.sk>
Message-Id: <20211019174953.36560-1-pavel@labath.sk>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
gdbstub.c | 8 ++-
tests/tcg/multiarch/Makefile.target | 10 +++-
| 60 +++++++++++++++++++
3 files changed, 75 insertions(+), 3 deletions(-)
create mode 100644 tests/tcg/multiarch/gdbstub/test-thread-breakpoint.py
diff --git a/gdbstub.c b/gdbstub.c
index 36b85aa50e..23baaef40e 100644
--- a/gdbstub.c
+++ b/gdbstub.c
@@ -3138,8 +3138,12 @@ gdb_handlesig(CPUState *cpu, int sig)
tb_flush(cpu);
if (sig != 0) {
- snprintf(buf, sizeof(buf), "S%02x", target_signal_to_gdb(sig));
- put_packet(buf);
+ gdb_set_stop_cpu(cpu);
+ g_string_printf(gdbserver_state.str_buf,
+ "T%02xthread:", target_signal_to_gdb(sig));
+ gdb_append_thread_id(cpu, gdbserver_state.str_buf);
+ g_string_append_c(gdbserver_state.str_buf, ';');
+ put_strbuf();
}
/* put_packet() might have detected that the peer terminated the
connection. */
diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target
index c0d9e638e9..b962ed8236 100644
--- a/tests/tcg/multiarch/Makefile.target
+++ b/tests/tcg/multiarch/Makefile.target
@@ -68,11 +68,19 @@ run-gdbstub-qxfer-auxv-read: sha1
--bin $< --test $(MULTIARCH_SRC)/gdbstub/test-qxfer-auxv-read.py, \
"basic gdbstub qXfer:auxv:read support")
+run-gdbstub-thread-breakpoint: testthread
+ $(call run-test, $@, $(GDB_SCRIPT) \
+ --gdb $(HAVE_GDB_BIN) \
+ --qemu $(QEMU) --qargs "$(QEMU_OPTS)" \
+ --bin $< --test $(MULTIARCH_SRC)/gdbstub/test-thread-breakpoint.py, \
+ "hitting a breakpoint on non-main thread")
+
else
run-gdbstub-%:
$(call skip-test, "gdbstub test $*", "need working gdb")
endif
-EXTRA_RUNS += run-gdbstub-sha1 run-gdbstub-qxfer-auxv-read
+EXTRA_RUNS += run-gdbstub-sha1 run-gdbstub-qxfer-auxv-read \
+ run-gdbstub-thread-breakpoint
# ARM Compatible Semi Hosting Tests
#
--git a/tests/tcg/multiarch/gdbstub/test-thread-breakpoint.py b/tests/tcg/multiarch/gdbstub/test-thread-breakpoint.py
new file mode 100644
index 0000000000..798d508bc7
--- /dev/null
+++ b/tests/tcg/multiarch/gdbstub/test-thread-breakpoint.py
@@ -0,0 +1,60 @@
+from __future__ import print_function
+#
+# Test auxiliary vector is loaded via gdbstub
+#
+# This is launched via tests/guest-debug/run-test.py
+#
+
+import gdb
+import sys
+
+failcount = 0
+
+def report(cond, msg):
+ "Report success/fail of test"
+ if cond:
+ print ("PASS: %s" % (msg))
+ else:
+ print ("FAIL: %s" % (msg))
+ global failcount
+ failcount += 1
+
+def run_test():
+ "Run through the tests one by one"
+
+ sym, ok = gdb.lookup_symbol("thread1_func")
+ gdb.execute("b thread1_func")
+ gdb.execute("c")
+
+ frame = gdb.selected_frame()
+ report(str(frame.function()) == "thread1_func", "break @ %s"%frame)
+
+#
+# This runs as the script it sourced (via -x, via run-test.py)
+#
+try:
+ inferior = gdb.selected_inferior()
+ arch = inferior.architecture()
+ print("ATTACHED: %s" % arch.name())
+except (gdb.error, AttributeError):
+ print("SKIPPING (not connected)", file=sys.stderr)
+ exit(0)
+
+if gdb.parse_and_eval('$pc') == 0:
+ print("SKIP: PC not set")
+ exit(0)
+
+try:
+ # These are not very useful in scripts
+ gdb.execute("set pagination off")
+ gdb.execute("set confirm off")
+
+ # Run the actual tests
+ run_test()
+except (gdb.error):
+ print ("GDB Exception: %s" % (sys.exc_info()[0]))
+ failcount += 1
+ pass
+
+print("All tests complete: %d failures" % failcount)
+exit(failcount)
--
2.30.2
^ permalink raw reply related [flat|nested] 61+ messages in thread* Re: [PATCH v1 28/28] gdbstub: Switch to the thread receiving a signal
2021-10-26 10:22 ` [PATCH v1 28/28] gdbstub: Switch to the thread receiving a signal Alex Bennée
@ 2021-10-26 20:34 ` Richard Henderson
0 siblings, 0 replies; 61+ messages in thread
From: Richard Henderson @ 2021-10-26 20:34 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, Pavel Labath, berrange, pbonzini, f4bug, robhenry, aaron,
mahmoudabdalghany, minyihh, cota, stefanha, crosa, kuhn.chenqun,
ma.mandourr, Philippe Mathieu-Daudé,
aurelien
On 10/26/21 3:22 AM, Alex Bennée wrote:
> From: Pavel Labath<pavel@labath.sk>
>
> Respond with Txxthread:yyyy; instead of a plain Sxx to indicate which
> thread received the signal. Otherwise, the debugger will associate it
> with the main one. Also automatically select this thread, as that is
> what gdb expects.
>
> Signed-off-by: Pavel Labath<pavel@labath.sk>
> Message-Id:<20211019174953.36560-1-pavel@labath.sk>
> Signed-off-by: Alex Bennée<alex.bennee@linaro.org>
> ---
> gdbstub.c | 8 ++-
> tests/tcg/multiarch/Makefile.target | 10 +++-
> .../gdbstub/test-thread-breakpoint.py | 60 +++++++++++++++++++
> 3 files changed, 75 insertions(+), 3 deletions(-)
> create mode 100644 tests/tcg/multiarch/gdbstub/test-thread-breakpoint.py
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
r~
^ permalink raw reply [flat|nested] 61+ messages in thread