All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH  v1 0/9] testing/next (docs, hexagon, cfi, docker)
@ 2021-03-05  9:23 Alex Bennée
  2021-03-05  9:23 ` [PATCH v1 1/9] docs/devel: re-organise the developers guide into sections Alex Bennée
                   ` (10 more replies)
  0 siblings, 11 replies; 19+ messages in thread
From: Alex Bennée @ 2021-03-05  9:23 UTC (permalink / raw)
  To: qemu-devel
  Cc: fam, berrange, f4bug, stefanha, crosa, pbonzini,
	Alex Bennée, aurelien

Hi,

Another week another testing/next roll. The series includes a couple
of my proposed documentation tweaks (including the re-org of the devel
manual). We also enable testing for the hexagon linux-user target to
avoid bitrot. I've documented the process for manually updating the
registry at:

 https://wiki.qemu.org/Testing/DockerBuild#The_GitLab_Registry

There are also jobs for testing CFI and some other minor tweaks and
clean-ups.

The following still need review:

 - gitlab: add build-user-hexagon test
 - tests/docker: add a test-tcg for building then running check-tcg
 - docs/devel: re-organise the developers guide into sections

Alessandro Di Federico (2):
  docker: Add Hexagon image
  tests/tcg: Use Hexagon Docker image

Alex Bennée (4):
  docs/devel: re-organise the developers guide into sections
  docs/system: add a gentle prompt for the complexity to come
  tests/docker: add a test-tcg for building then running check-tcg
  gitlab: add build-user-hexagon test

Daniele Buono (2):
  gitlab-ci.yml: Allow custom # of parallel linkers
  gitlab-ci.yml: Add jobs to test CFI flags

Philippe Mathieu-Daudé (1):
  tests/docker: Use --arch-only when building Debian cross image

 docs/devel/index.rst                          |  32 +---
 docs/devel/multi-thread-tcg.rst               |   5 +-
 docs/devel/section-apis.rst                   |  16 ++
 docs/devel/section-building.rst               |  13 ++
 docs/devel/section-concepts.rst               |  21 +++
 docs/devel/section-process.rst                |  11 ++
 docs/devel/section-tcg-emulation.rst          |  19 +++
 docs/devel/section-testing.rst                |  20 +++
 docs/devel/tcg-icount.rst                     |   6 +-
 docs/devel/testing.rst                        |   6 +-
 docs/system/quickstart.rst                    |   8 +
 docs/system/targets.rst                       |   2 +
 .gitlab-ci.yml                                | 134 +++++++++++++++++
 MAINTAINERS                                   |   2 +
 tests/docker/Makefile.include                 |   2 +
 .../dockerfiles/debian-hexagon-cross.docker   |  27 ++++
 .../build-toolchain.sh                        | 141 ++++++++++++++++++
 tests/docker/dockerfiles/debian10.docker      |   2 +-
 tests/docker/test-tcg                         |  22 +++
 tests/tcg/configure.sh                        |   4 +
 20 files changed, 458 insertions(+), 35 deletions(-)
 create mode 100644 docs/devel/section-apis.rst
 create mode 100644 docs/devel/section-building.rst
 create mode 100644 docs/devel/section-concepts.rst
 create mode 100644 docs/devel/section-process.rst
 create mode 100644 docs/devel/section-tcg-emulation.rst
 create mode 100644 docs/devel/section-testing.rst
 create mode 100644 tests/docker/dockerfiles/debian-hexagon-cross.docker
 create mode 100755 tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
 create mode 100755 tests/docker/test-tcg

-- 
2.20.1



^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH v1 1/9] docs/devel: re-organise the developers guide into sections
  2021-03-05  9:23 [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) Alex Bennée
@ 2021-03-05  9:23 ` Alex Bennée
  2021-03-05  9:40   ` Philippe Mathieu-Daudé
  2021-03-05  9:23 ` [PATCH v1 2/9] docs/system: add a gentle prompt for the complexity to come Alex Bennée
                   ` (9 subsequent siblings)
  10 siblings, 1 reply; 19+ messages in thread
From: Alex Bennée @ 2021-03-05  9:23 UTC (permalink / raw)
  To: qemu-devel
  Cc: fam, berrange, f4bug, stefanha, crosa, pbonzini,
	Alex Bennée, aurelien

The list of sub-sections was getting a bit long and sporadically
organised. Let's try and impose some order on this hairball of
documentation.

[AJB: RFC because I wonder if we should make a more concerted effort
to move bits of the wiki into a canonical maintained document. There
is also probably a need for a quickbuild or tldr section of the
build-system for users who just want to build something.]

Based-on: 20210223095931.16908-1-alex.bennee@linaro.org
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 docs/devel/index.rst                 | 32 ++++++----------------------
 docs/devel/multi-thread-tcg.rst      |  5 +++--
 docs/devel/section-apis.rst          | 16 ++++++++++++++
 docs/devel/section-building.rst      | 13 +++++++++++
 docs/devel/section-concepts.rst      | 21 ++++++++++++++++++
 docs/devel/section-process.rst       | 11 ++++++++++
 docs/devel/section-tcg-emulation.rst | 19 +++++++++++++++++
 docs/devel/section-testing.rst       | 20 +++++++++++++++++
 docs/devel/tcg-icount.rst            |  6 +++---
 docs/devel/testing.rst               |  6 +++---
 10 files changed, 115 insertions(+), 34 deletions(-)
 create mode 100644 docs/devel/section-apis.rst
 create mode 100644 docs/devel/section-building.rst
 create mode 100644 docs/devel/section-concepts.rst
 create mode 100644 docs/devel/section-process.rst
 create mode 100644 docs/devel/section-tcg-emulation.rst
 create mode 100644 docs/devel/section-testing.rst

diff --git a/docs/devel/index.rst b/docs/devel/index.rst
index ae664da00c..2af7bf8736 100644
--- a/docs/devel/index.rst
+++ b/docs/devel/index.rst
@@ -13,29 +13,9 @@ Contents:
 .. toctree::
    :maxdepth: 2
 
-   build-system
-   style
-   kconfig
-   testing
-   fuzzing
-   control-flow-integrity
-   loads-stores
-   memory
-   migration
-   atomics
-   stable-process
-   qtest
-   decodetree
-   secure-coding-practices
-   tcg
-   tcg-icount
-   tracing
-   multi-thread-tcg
-   tcg-plugins
-   bitops
-   reset
-   s390-dasd-ipl
-   clocks
-   qom
-   block-coroutine-wrapper
-   multi-process
+   section-building
+   section-testing
+   section-concepts
+   section-apis
+   section-tcg-emulation
+   section-process
diff --git a/docs/devel/multi-thread-tcg.rst b/docs/devel/multi-thread-tcg.rst
index 92a9eba13c..5b446ee08b 100644
--- a/docs/devel/multi-thread-tcg.rst
+++ b/docs/devel/multi-thread-tcg.rst
@@ -4,8 +4,9 @@
   This work is licensed under the terms of the GNU GPL, version 2 or
   later. See the COPYING file in the top-level directory.
 
-Introduction
-============
+==================
+Multi-threaded TCG
+==================
 
 This document outlines the design for multi-threaded TCG (a.k.a MTTCG)
 system-mode emulation. user-mode emulation has always mirrored the
diff --git a/docs/devel/section-apis.rst b/docs/devel/section-apis.rst
new file mode 100644
index 0000000000..ab1f6bed60
--- /dev/null
+++ b/docs/devel/section-apis.rst
@@ -0,0 +1,16 @@
+*********
+QEMU APIs
+*********
+
+There are a number of APIs in QEMU and the following sections document
+some of the most important ones. For tose that aren't documented here
+you can also find notes on usage in the header definitions.
+
+.. toctree::
+   :maxdepth: 2
+
+   atomics
+   bitops
+   loads-stores
+   memory
+   tracing
diff --git a/docs/devel/section-building.rst b/docs/devel/section-building.rst
new file mode 100644
index 0000000000..e6e9fa1d6a
--- /dev/null
+++ b/docs/devel/section-building.rst
@@ -0,0 +1,13 @@
+*************
+Building QEMU
+*************
+
+The following sections deal with how the build system works, how it is
+configured and some basic guidance on how code should be written.
+
+.. toctree::
+   :maxdepth: 2
+
+   build-system
+   kconfig
+   style
diff --git a/docs/devel/section-concepts.rst b/docs/devel/section-concepts.rst
new file mode 100644
index 0000000000..566c52e90a
--- /dev/null
+++ b/docs/devel/section-concepts.rst
@@ -0,0 +1,21 @@
+*************
+QEMU Concepts
+*************
+
+There are a number of high level concepts that are useful to
+understand when working with the code base. Perhaps the most pervasive
+is the QEMU Object Model (QOM) which underpins much of the flexibility
+and configurable of the project. The following sections document that
+as well as diving into other concepts that are useful to know if
+working on some areas of the code.
+
+.. toctree::
+   :maxdepth: 2
+
+   qom
+   clocks
+   reset
+   block-coroutine-wrapper
+   migration
+   multi-process
+   s390-dasd-ipl
diff --git a/docs/devel/section-process.rst b/docs/devel/section-process.rst
new file mode 100644
index 0000000000..3b0ae4f19b
--- /dev/null
+++ b/docs/devel/section-process.rst
@@ -0,0 +1,11 @@
+===================
+Development Process
+===================
+
+The following sections detail aspects of the development process.
+
+.. toctree::
+   :maxdepth: 2
+
+   secure-coding-practices
+   stable-process
diff --git a/docs/devel/section-tcg-emulation.rst b/docs/devel/section-tcg-emulation.rst
new file mode 100644
index 0000000000..81fa131de7
--- /dev/null
+++ b/docs/devel/section-tcg-emulation.rst
@@ -0,0 +1,19 @@
+*************
+TCG Emulation
+*************
+
+QEMU was originally built as an emulator capable of running binaries
+for one architecture on another. The following sections describe the
+internals of how the Just In Time (JIT) Tiny Code Generator (TCG)
+works. You only really need to read this if you are interested in
+adding new architectures or fixing existing architecture emulation.
+
+
+.. toctree::
+   :maxdepth: 2
+
+   tcg
+   multi-thread-tcg
+   tcg-icount
+   decodetree
+   tcg-plugins
diff --git a/docs/devel/section-testing.rst b/docs/devel/section-testing.rst
new file mode 100644
index 0000000000..c93ff4c4dd
--- /dev/null
+++ b/docs/devel/section-testing.rst
@@ -0,0 +1,20 @@
+***************
+Testing in QEMU
+***************
+
+QEMU is a large and complex project which can be configured in a
+multitude of ways. As it's impossible for an individual developer to
+manually test all of these we rely on a whole suite of automated
+testing approaches to ensure regressions are kept to a minimum.
+
+The following sections give a broad overview of the testing
+infrastructure as well as some detailed introductions into more
+advanced testing topics.
+
+.. toctree::
+   :maxdepth: 2
+
+   testing
+   fuzzing
+   control-flow-integrity
+   qtest
diff --git a/docs/devel/tcg-icount.rst b/docs/devel/tcg-icount.rst
index 8d67b6c076..eb990b0976 100644
--- a/docs/devel/tcg-icount.rst
+++ b/docs/devel/tcg-icount.rst
@@ -3,9 +3,9 @@
    Written by Alex Bennée
 
 
-========================
-TCG Instruction Counting
-========================
+=================================
+TCG Instruction Counting (icount)
+=================================
 
 TCG has long supported a feature known as icount which allows for
 instruction counting during execution. This should not be confused
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 1434a50cc4..af98efec8c 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -1,6 +1,6 @@
-===============
-Testing in QEMU
-===============
+=======
+Testing
+=======
 
 This document describes the testing infrastructure in QEMU.
 
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v1 2/9] docs/system: add a gentle prompt for the complexity to come
  2021-03-05  9:23 [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) Alex Bennée
  2021-03-05  9:23 ` [PATCH v1 1/9] docs/devel: re-organise the developers guide into sections Alex Bennée
@ 2021-03-05  9:23 ` Alex Bennée
  2021-03-05  9:23 ` [PATCH v1 3/9] tests/docker: add a test-tcg for building then running check-tcg Alex Bennée
                   ` (8 subsequent siblings)
  10 siblings, 0 replies; 19+ messages in thread
From: Alex Bennée @ 2021-03-05  9:23 UTC (permalink / raw)
  To: qemu-devel
  Cc: fam, Thomas Huth, berrange, John Snow, f4bug, stefanha, crosa,
	pbonzini, Alex Bennée, aurelien

We all know the QEMU command line can become a fiendishly complex
beast. Lets gently prepare our user for the horrors to come by
referencing where other example command lines can be found in the
manual.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210301112741.31047-1-alex.bennee@linaro.org>

---
v2
  - minor grammar tweaks suggested by Thomas and John
---
 docs/system/quickstart.rst | 8 ++++++++
 docs/system/targets.rst    | 2 ++
 2 files changed, 10 insertions(+)

diff --git a/docs/system/quickstart.rst b/docs/system/quickstart.rst
index 3a3acab5e7..681678c86e 100644
--- a/docs/system/quickstart.rst
+++ b/docs/system/quickstart.rst
@@ -11,3 +11,11 @@ Download and uncompress a PC hard disk image with Linux installed (e.g.
    |qemu_system| linux.img
 
 Linux should boot and give you a prompt.
+
+Users should be aware the above example elides a lot of the complexity
+of setting up a VM with x86_64 specific defaults and assumes the
+first non switch argument is a PC compatible disk image with a boot
+sector. For a non-x86 system where we emulate a broad range of machine
+types, the command lines are generally more explicit in defining the
+machine and boot behaviour. You will find more example command lines
+in the :ref:`system-targets-ref` section of the manual.
diff --git a/docs/system/targets.rst b/docs/system/targets.rst
index 560783644d..145cc64551 100644
--- a/docs/system/targets.rst
+++ b/docs/system/targets.rst
@@ -1,3 +1,5 @@
+.. _system-targets-ref:
+
 QEMU System Emulator Targets
 ============================
 
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v1 3/9] tests/docker: add a test-tcg for building then running check-tcg
  2021-03-05  9:23 [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) Alex Bennée
  2021-03-05  9:23 ` [PATCH v1 1/9] docs/devel: re-organise the developers guide into sections Alex Bennée
  2021-03-05  9:23 ` [PATCH v1 2/9] docs/system: add a gentle prompt for the complexity to come Alex Bennée
@ 2021-03-05  9:23 ` Alex Bennée
  2021-03-05  9:41   ` Philippe Mathieu-Daudé
  2021-03-05  9:23 ` [PATCH v1 4/9] docker: Add Hexagon image Alex Bennée
                   ` (7 subsequent siblings)
  10 siblings, 1 reply; 19+ messages in thread
From: Alex Bennée @ 2021-03-05  9:23 UTC (permalink / raw)
  To: qemu-devel
  Cc: fam, berrange, Philippe Mathieu-Daudé,
	f4bug, stefanha, crosa, pbonzini, Alex Bennée, aurelien

This is mostly useful for verifying containers will work on the CI
setup.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/test-tcg | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100755 tests/docker/test-tcg

diff --git a/tests/docker/test-tcg b/tests/docker/test-tcg
new file mode 100755
index 0000000000..00993b73ba
--- /dev/null
+++ b/tests/docker/test-tcg
@@ -0,0 +1,22 @@
+#!/bin/bash -e
+#
+# Build and run the TCG tests
+#
+# Copyright (c) 2021 Linaro Ltd.
+#
+# Authors:
+#  Alex Bennée <alex.bennee@linaro.org>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+cd "$BUILD_DIR"
+
+# although we are not building QEMU itself we still need a configured
+# build for the unit tests to be built and run
+TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
+build_qemu
+check_qemu check-tcg
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH  v1 4/9] docker: Add Hexagon image
  2021-03-05  9:23 [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) Alex Bennée
                   ` (2 preceding siblings ...)
  2021-03-05  9:23 ` [PATCH v1 3/9] tests/docker: add a test-tcg for building then running check-tcg Alex Bennée
@ 2021-03-05  9:23 ` Alex Bennée
  2021-03-05  9:23 ` [PATCH v1 5/9] tests/tcg: Use Hexagon Docker image Alex Bennée
                   ` (6 subsequent siblings)
  10 siblings, 0 replies; 19+ messages in thread
From: Alex Bennée @ 2021-03-05  9:23 UTC (permalink / raw)
  To: qemu-devel
  Cc: fam, Alessandro Di Federico, berrange, Brian Cain,
	Philippe Mathieu-Daudé,
	f4bug, stefanha, crosa, pbonzini, Alex Bennée, aurelien

From: Alessandro Di Federico <ale@rev.ng>

[PMD: Base on qemu/debian10, add missing EXTRA_FILES, remove X86]

Signed-off-by: Alessandro Di Federico <ale@rev.ng>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Brian Cain <bcain@quicinc.com>
Message-Id: <20210228222314.304787-2-f4bug@amsat.org>
[AJB: add build-dep for QEMU, include in MAINTAINERS]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

squash! docker: Add Hexagon image
---
 MAINTAINERS                                   |   2 +
 tests/docker/Makefile.include                 |   2 +
 .../dockerfiles/debian-hexagon-cross.docker   |  27 ++++
 .../build-toolchain.sh                        | 141 ++++++++++++++++++
 4 files changed, 172 insertions(+)
 create mode 100644 tests/docker/dockerfiles/debian-hexagon-cross.docker
 create mode 100755 tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh

diff --git a/MAINTAINERS b/MAINTAINERS
index 9b2aa18e1f..1443278059 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -196,6 +196,8 @@ F: linux-user/hexagon/
 F: tests/tcg/hexagon/
 F: disas/hexagon.c
 F: default-configs/targets/hexagon-linux-user.mak
+F: docker/dockerfiles/debian-hexagon-cross.docker
+F: docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
 
 HPPA (PA-RISC) TCG CPUs
 M: Richard Henderson <richard.henderson@linaro.org>
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 7cab761bf5..3aee031c4e 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -144,6 +144,8 @@ docker-image-debian-riscv64-cross: docker-image-debian10
 docker-image-debian-s390x-cross: docker-image-debian10
 docker-image-debian-sh4-cross: docker-image-debian10
 docker-image-debian-sparc64-cross: docker-image-debian10
+docker-image-debian-hexagon-cross: \
+	EXTRA_FILES:=$(SRC_PATH)/tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
 
 # Specialist build images, sometimes very limited tools
 docker-image-debian-tricore-cross: docker-image-debian10
diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.docker b/tests/docker/dockerfiles/debian-hexagon-cross.docker
new file mode 100644
index 0000000000..bf32536896
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-hexagon-cross.docker
@@ -0,0 +1,27 @@
+FROM qemu/debian10
+
+# What we need to build QEMU itself (as this is used in CI as well)
+RUN apt update && \
+    DEBIAN_FRONTEND=noninteractive eatmydata \
+    apt build-dep -yy qemu
+
+RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+    apt install -y --no-install-recommends \
+        bison \
+        cmake \
+        flex \
+        lld \
+        rsync \
+        wget
+
+ENV TOOLCHAIN_INSTALL /usr
+ENV ROOTFS /usr
+
+ENV LLVM_URL https://github.com/llvm/llvm-project/archive/3d8149c2a1228609fd7d7c91a04681304a2f0ca9.tar.gz
+ENV MUSL_URL https://github.com/quic/musl/archive/aff74b395fbf59cd7e93b3691905aa1af6c0778c.tar.gz
+ENV LINUX_URL https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.6.18.tar.xz
+
+ADD build-toolchain.sh /root/hexagon-toolchain/build-toolchain.sh
+
+RUN ( cd /root/hexagon-toolchain && ./build-toolchain.sh ) && \
+    rm -rf /root/hexagon-toolchain
diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh b/tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
new file mode 100755
index 0000000000..19b1c9f83e
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh
@@ -0,0 +1,141 @@
+#!/bin/bash
+
+set -e
+
+BASE=$(readlink -f ${PWD})
+
+TOOLCHAIN_INSTALL=$(readlink -f "$TOOLCHAIN_INSTALL")
+ROOTFS=$(readlink -f "$ROOTFS")
+
+TOOLCHAIN_BIN=${TOOLCHAIN_INSTALL}/bin
+HEX_SYSROOT=${TOOLCHAIN_INSTALL}/hexagon-unknown-linux-musl
+HEX_TOOLS_TARGET_BASE=${HEX_SYSROOT}/usr
+
+function cdp() {
+  DIR="$1"
+  mkdir -p "$DIR"
+  cd "$DIR"
+}
+
+function fetch() {
+  DIR="$1"
+  URL="$2"
+  TEMP="$(readlink -f "$PWD/tmp.tar.gz")"
+  wget --quiet "$URL" -O "$TEMP"
+  cdp "$DIR"
+  tar xaf "$TEMP" --strip-components=1
+  rm "$TEMP"
+  cd -
+}
+
+build_llvm_clang() {
+  fetch "$BASE/llvm-project" "$LLVM_URL"
+  cdp "$BASE/build-llvm"
+
+  cmake -G Ninja \
+    -DCMAKE_BUILD_TYPE=Release \
+    -DCMAKE_INSTALL_PREFIX=${TOOLCHAIN_INSTALL} \
+    -DLLVM_ENABLE_LLD=ON \
+    -DLLVM_TARGETS_TO_BUILD="Hexagon" \
+    -DLLVM_ENABLE_PROJECTS="clang;lld" \
+    "$BASE/llvm-project/llvm"
+  ninja all install
+  cd ${TOOLCHAIN_BIN}
+  ln -sf clang hexagon-unknown-linux-musl-clang
+  ln -sf clang++ hexagon-unknown-linux-musl-clang++
+  ln -sf llvm-ar hexagon-unknown-linux-musl-ar
+  ln -sf llvm-objdump hexagon-unknown-linux-musl-objdump
+  ln -sf llvm-objcopy hexagon-unknown-linux-musl-objcopy
+  ln -sf llvm-readelf hexagon-unknown-linux-musl-readelf
+  ln -sf llvm-ranlib hexagon-unknown-linux-musl-ranlib
+
+  # workaround for now:
+  cat <<EOF > hexagon-unknown-linux-musl.cfg
+-G0 --sysroot=${HEX_SYSROOT}
+EOF
+}
+
+build_clang_rt() {
+  cdp "$BASE/build-clang_rt"
+  cmake -G Ninja \
+    -DCMAKE_BUILD_TYPE=Release \
+    -DLLVM_CONFIG_PATH="$BASE/build-llvm/bin/llvm-config" \
+    -DCMAKE_ASM_FLAGS="-G0 -mlong-calls -fno-pic --target=hexagon-unknown-linux-musl " \
+    -DCMAKE_SYSTEM_NAME=Linux \
+    -DCMAKE_C_COMPILER="${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang" \
+    -DCMAKE_ASM_COMPILER="${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang" \
+    -DCMAKE_INSTALL_PREFIX=${HEX_TOOLS_TARGET_BASE} \
+    -DCMAKE_CROSSCOMPILING=ON \
+    -DCMAKE_C_COMPILER_FORCED=ON \
+    -DCMAKE_CXX_COMPILER_FORCED=ON \
+    -DCOMPILER_RT_BUILD_BUILTINS=ON \
+    -DCOMPILER_RT_BUILTINS_ENABLE_PIC=OFF \
+    -DCMAKE_SIZEOF_VOID_P=4 \
+    -DCOMPILER_RT_OS_DIR= \
+    -DCAN_TARGET_hexagon=1 \
+    -DCAN_TARGET_x86_64=0 \
+    -DCOMPILER_RT_SUPPORTED_ARCH=hexagon \
+    -DLLVM_ENABLE_PROJECTS="compiler-rt" \
+    "$BASE/llvm-project/compiler-rt"
+  ninja install-compiler-rt
+}
+
+build_musl_headers() {
+  fetch "$BASE/musl" "$MUSL_URL"
+  cd "$BASE/musl"
+  make clean
+  CC=${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang \
+    CROSS_COMPILE=hexagon-unknown-linux-musl \
+    LIBCC=${HEX_TOOLS_TARGET_BASE}/lib/libclang_rt.builtins-hexagon.a \
+    CROSS_CFLAGS="-G0 -O0 -mv65 -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl" \
+    ./configure --target=hexagon --prefix=${HEX_TOOLS_TARGET_BASE}
+  PATH=${TOOLCHAIN_BIN}:$PATH make CROSS_COMPILE= install-headers
+
+  cd ${HEX_SYSROOT}/..
+  ln -sf hexagon-unknown-linux-musl hexagon
+}
+
+build_kernel_headers() {
+  fetch "$BASE/linux" "$LINUX_URL"
+  mkdir -p "$BASE/build-linux"
+  cd "$BASE/linux"
+  make O=../build-linux ARCH=hexagon \
+   KBUILD_CFLAGS_KERNEL="-mlong-calls" \
+   CC=${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang \
+   LD=${TOOLCHAIN_BIN}/ld.lld \
+   KBUILD_VERBOSE=1 comet_defconfig
+  make mrproper
+
+  cd "$BASE/build-linux"
+  make \
+    ARCH=hexagon \
+    CC=${TOOLCHAIN_BIN}/clang \
+    INSTALL_HDR_PATH=${HEX_TOOLS_TARGET_BASE} \
+    V=1 \
+    headers_install
+}
+
+build_musl() {
+  cd "$BASE/musl"
+  make clean
+  CROSS_COMPILE=hexagon-unknown-linux-musl- \
+    AR=llvm-ar \
+    RANLIB=llvm-ranlib \
+    STRIP=llvm-strip \
+    CC=clang \
+    LIBCC=${HEX_TOOLS_TARGET_BASE}/lib/libclang_rt.builtins-hexagon.a \
+    CFLAGS="-G0 -O0 -mv65 -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl" \
+    ./configure --target=hexagon --prefix=${HEX_TOOLS_TARGET_BASE}
+  PATH=${TOOLCHAIN_BIN}/:$PATH make CROSS_COMPILE= install
+  cd ${HEX_TOOLS_TARGET_BASE}/lib
+  ln -sf libc.so ld-musl-hexagon.so
+  ln -sf ld-musl-hexagon.so ld-musl-hexagon.so.1
+  cdp ${HEX_TOOLS_TARGET_BASE}/../lib
+  ln -sf ../usr/lib/ld-musl-hexagon.so.1
+}
+
+build_llvm_clang
+build_kernel_headers
+build_musl_headers
+build_clang_rt
+build_musl
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH  v1 5/9] tests/tcg: Use Hexagon Docker image
  2021-03-05  9:23 [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) Alex Bennée
                   ` (3 preceding siblings ...)
  2021-03-05  9:23 ` [PATCH v1 4/9] docker: Add Hexagon image Alex Bennée
@ 2021-03-05  9:23 ` Alex Bennée
  2021-03-05  9:23 ` [PATCH v1 6/9] gitlab: add build-user-hexagon test Alex Bennée
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 19+ messages in thread
From: Alex Bennée @ 2021-03-05  9:23 UTC (permalink / raw)
  To: qemu-devel
  Cc: fam, Alessandro Di Federico, berrange, f4bug, stefanha, crosa,
	pbonzini, Alex Bennée, aurelien

From: Alessandro Di Federico <ale@rev.ng>

[PMD: Split from 'Add Hexagon Docker image' patch]

Signed-off-by: Alessandro Di Federico <ale@rev.ng>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210228222314.304787-5-f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/tcg/configure.sh | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index 36b8a73a54..f70fd7435d 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -124,6 +124,10 @@ for target in $target_list; do
       container_image=fedora-cris-cross
       container_cross_cc=cris-linux-gnu-gcc
       ;;
+    hexagon-*)
+      container_image=debian-hexagon-cross
+      container_cross_cc=hexagon-unknown-linux-musl-clang
+      ;;
     hppa-*)
       container_image=debian-hppa-cross
       container_cross_cc=hppa-linux-gnu-gcc
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH  v1 6/9] gitlab: add build-user-hexagon test
  2021-03-05  9:23 [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) Alex Bennée
                   ` (4 preceding siblings ...)
  2021-03-05  9:23 ` [PATCH v1 5/9] tests/tcg: Use Hexagon Docker image Alex Bennée
@ 2021-03-05  9:23 ` Alex Bennée
  2021-03-05  9:43   ` Philippe Mathieu-Daudé
  2021-03-05 15:37   ` Wainer dos Santos Moschetta
  2021-03-05  9:23 ` [PATCH v1 7/9] gitlab-ci.yml: Allow custom # of parallel linkers Alex Bennée
                   ` (4 subsequent siblings)
  10 siblings, 2 replies; 19+ messages in thread
From: Alex Bennée @ 2021-03-05  9:23 UTC (permalink / raw)
  To: qemu-devel
  Cc: fam, Thomas Huth, berrange, Philippe Mathieu-Daudé,
	f4bug, Wainer dos Santos Moschetta, stefanha, crosa, pbonzini,
	Alex Bennée, aurelien

We special case this as the container with the cross compiler for the
tests takes so long to build it is manually uploaded into the
registry.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .gitlab-ci.yml | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 8b6d495288..c2c5196b8b 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -417,6 +417,17 @@ build-user-static:
     CONFIGURE_ARGS: --disable-tools --disable-system --static
     MAKE_CHECK_ARGS: check-tcg
 
+# Because the hexagon cross-compiler takes so long to build we don't rely
+# on the CI system to build it and hence this job has no dependency
+# declared. The image is manually uploaded.
+build-user-hexagon:
+  <<: *native_build_job_definition
+  variables:
+    IMAGE: debian-hexagon-cross
+    TARGETS: hexagon-linux-user
+    CONFIGURE_ARGS: --disable-tools --disable-docs --enable-debug-tcg
+    MAKE_CHECK_ARGS: check-tcg
+
 # Only build the softmmu targets we have check-tcg tests for
 build-some-softmmu:
   <<: *native_build_job_definition
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH  v1 7/9] gitlab-ci.yml: Allow custom # of parallel linkers
  2021-03-05  9:23 [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) Alex Bennée
                   ` (5 preceding siblings ...)
  2021-03-05  9:23 ` [PATCH v1 6/9] gitlab: add build-user-hexagon test Alex Bennée
@ 2021-03-05  9:23 ` Alex Bennée
  2021-03-05  9:23 ` [PATCH v1 8/9] gitlab-ci.yml: Add jobs to test CFI flags Alex Bennée
                   ` (3 subsequent siblings)
  10 siblings, 0 replies; 19+ messages in thread
From: Alex Bennée @ 2021-03-05  9:23 UTC (permalink / raw)
  To: qemu-devel
  Cc: fam, Thomas Huth, berrange, Philippe Mathieu-Daudé,
	f4bug, Wainer dos Santos Moschetta, Daniele Buono, stefanha,
	crosa, pbonzini, Alex Bennée, aurelien

From: Daniele Buono <dbuono@linux.vnet.ibm.com>

Define a new variable LD_JOBS, that can be used to select
the maximum number of linking jobs to be executed in parallel.
If the variable is not defined, maintain the default given by
make -j

Currently, make parallelism at build time is based on the number
of cpus available.

This doesn't work well with LTO at linking, because with LTO the
linker has to load in memory all the intermediate object files
for optimization.
The end result is that, if the gitlab runner happens to run two
linking processes at the same time, the job will fail with an
out-of-memory error,

This patch leverages the ability to maintain high parallelism at
compile time, but limit the number of linkers executed in parallel.

Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210304030948.9367-2-dbuono@linux.vnet.ibm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .gitlab-ci.yml | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index c2c5196b8b..1ea944eb91 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -27,6 +27,10 @@ include:
       else
         ../configure --enable-werror $CONFIGURE_ARGS ;
       fi || { cat config.log meson-logs/meson-log.txt && exit 1; }
+    - if test -n "$LD_JOBS";
+      then
+        meson configure . -Dbackend_max_links="$LD_JOBS" ;
+      fi || exit 1;
     - make -j"$JOBS"
     - if test -n "$MAKE_CHECK_ARGS";
       then
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH  v1 8/9] gitlab-ci.yml: Add jobs to test CFI flags
  2021-03-05  9:23 [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) Alex Bennée
                   ` (6 preceding siblings ...)
  2021-03-05  9:23 ` [PATCH v1 7/9] gitlab-ci.yml: Allow custom # of parallel linkers Alex Bennée
@ 2021-03-05  9:23 ` Alex Bennée
  2021-03-05  9:43   ` Philippe Mathieu-Daudé
  2021-03-05  9:23 ` [PATCH v1 9/9] tests/docker: Use --arch-only when building Debian cross image Alex Bennée
                   ` (2 subsequent siblings)
  10 siblings, 1 reply; 19+ messages in thread
From: Alex Bennée @ 2021-03-05  9:23 UTC (permalink / raw)
  To: qemu-devel
  Cc: fam, Thomas Huth, berrange, Philippe Mathieu-Daudé,
	f4bug, Wainer dos Santos Moschetta, Daniele Buono, stefanha,
	crosa, pbonzini, Alex Bennée, aurelien

From: Daniele Buono <dbuono@linux.vnet.ibm.com>

QEMU has had options to enable control-flow integrity features
for a few months now. Add two sets of build/check/acceptance
jobs to ensure the binary produced is working fine.

The three sets allow testing of x86_64 binaries for x86_64, s390x,
ppc64 and aarch64 targets

Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com>
Message-Id: <20210304030948.9367-3-dbuono@linux.vnet.ibm.com>
[AJB: tweak job names to avoid brands]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 .gitlab-ci.yml | 119 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 119 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 1ea944eb91..5625265ef8 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -494,6 +494,125 @@ clang-user:
       --extra-cflags=-fsanitize=undefined --extra-cflags=-fno-sanitize-recover=undefined
     MAKE_CHECK_ARGS: check-unit check-tcg
 
+# Set LD_JOBS=1 because this requires LTO and ld consumes a large amount of memory.
+# On gitlab runners, default value sometimes end up calling 2 lds concurrently and
+# triggers an Out-Of-Memory error
+#
+# Since slirp callbacks are used in QEMU Timers, slirp needs to be compiled together
+# with QEMU and linked as a static library to avoid false positives in CFI checks.
+# This can be accomplished by using -enable-slirp=git, which avoids the use of
+# a system-wide version of the library
+#
+# Split in three sets of build/check/acceptance to limit the execution time of each
+# job
+build-cfi-aarch64:
+  <<: *native_build_job_definition
+  needs:
+  - job: amd64-fedora-container
+  variables:
+    LD_JOBS: 1
+    AR: llvm-ar
+    IMAGE: fedora
+    CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug
+      --enable-safe-stack --enable-slirp=git
+    TARGETS: aarch64-softmmu
+    MAKE_CHECK_ARGS: check-build
+  artifacts:
+    expire_in: 2 days
+    paths:
+      - build
+
+check-cfi-aarch64:
+  <<: *native_test_job_definition
+  needs:
+    - job: build-cfi-aarch64
+      artifacts: true
+  variables:
+    IMAGE: fedora
+    MAKE_CHECK_ARGS: check
+
+acceptance-cfi-aarch64:
+  <<: *native_test_job_definition
+  needs:
+    - job: build-cfi-aarch64
+      artifacts: true
+  variables:
+    IMAGE: fedora
+    MAKE_CHECK_ARGS: check-acceptance
+  <<: *acceptance_definition
+
+build-cfi-ppc64-s390x:
+  <<: *native_build_job_definition
+  needs:
+  - job: amd64-fedora-container
+  variables:
+    LD_JOBS: 1
+    AR: llvm-ar
+    IMAGE: fedora
+    CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug
+      --enable-safe-stack --enable-slirp=git
+    TARGETS: ppc64-softmmu s390x-softmmu
+    MAKE_CHECK_ARGS: check-build
+  artifacts:
+    expire_in: 2 days
+    paths:
+      - build
+
+check-cfi-ppc64-s390x:
+  <<: *native_test_job_definition
+  needs:
+    - job: build-cfi-ppc64-s390x
+      artifacts: true
+  variables:
+    IMAGE: fedora
+    MAKE_CHECK_ARGS: check
+
+acceptance-cfi-ppc64-s390x:
+  <<: *native_test_job_definition
+  needs:
+    - job: build-cfi-ppc64-s390x
+      artifacts: true
+  variables:
+    IMAGE: fedora
+    MAKE_CHECK_ARGS: check-acceptance
+  <<: *acceptance_definition
+
+build-cfi-x86_64:
+  <<: *native_build_job_definition
+  needs:
+  - job: amd64-fedora-container
+  variables:
+    LD_JOBS: 1
+    AR: llvm-ar
+    IMAGE: fedora
+    CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug
+      --enable-safe-stack --enable-slirp=git
+    TARGETS: x86_64-softmmu
+    MAKE_CHECK_ARGS: check-build
+  artifacts:
+    expire_in: 2 days
+    paths:
+      - build
+
+check-cfi-x86_64:
+  <<: *native_test_job_definition
+  needs:
+    - job: build-cfi-x86_64
+      artifacts: true
+  variables:
+    IMAGE: fedora
+    MAKE_CHECK_ARGS: check
+
+acceptance-cfi-x86_64:
+  <<: *native_test_job_definition
+  needs:
+    - job: build-cfi-x86_64
+      artifacts: true
+  variables:
+    IMAGE: fedora
+    MAKE_CHECK_ARGS: check-acceptance
+  <<: *acceptance_definition
+
 tsan-build:
   <<: *native_build_job_definition
   variables:
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v1 9/9] tests/docker: Use --arch-only when building Debian cross image
  2021-03-05  9:23 [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) Alex Bennée
                   ` (7 preceding siblings ...)
  2021-03-05  9:23 ` [PATCH v1 8/9] gitlab-ci.yml: Add jobs to test CFI flags Alex Bennée
@ 2021-03-05  9:23 ` Alex Bennée
  2021-03-05  9:42 ` [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) no-reply
  2021-03-05 19:37 ` Willian Rampazzo
  10 siblings, 0 replies; 19+ messages in thread
From: Alex Bennée @ 2021-03-05  9:23 UTC (permalink / raw)
  To: qemu-devel
  Cc: fam, berrange, Philippe Mathieu-Daudé,
	f4bug, Christian Ehrhardt, stefanha, crosa, pbonzini,
	Alex Bennée, aurelien

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

When building a Docker image based on debian10.docker on
a non-x86 host, we get:

 [2/4] RUN apt update &&     DEBIAN_FRONTEND=noninteractive eatmydata     apt build-dep -yy qemu
 Reading package lists... Done
 Building dependency tree
 Reading state information... Done
 Some packages could not be installed. This may mean that you have
 requested an impossible situation or if you are using the unstable
 distribution that some required packages have not yet been created
 or been moved out of Incoming.
 The following information may help to resolve the situation:

 The following packages have unmet dependencies:
  builddeps:qemu : Depends: gcc-s390x-linux-gnu but it is not installable
                   Depends: gcc-alpha-linux-gnu but it is not installable
 E: Unable to correct problems, you have held broken packages.

Fix by using the --arch-only option suggested here:
https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1866032/comments/1

Suggested-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210223211115.2971565-1-f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
 tests/docker/dockerfiles/debian10.docker | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/docker/dockerfiles/debian10.docker b/tests/docker/dockerfiles/debian10.docker
index 9d42b5a4b8..d034acbd25 100644
--- a/tests/docker/dockerfiles/debian10.docker
+++ b/tests/docker/dockerfiles/debian10.docker
@@ -32,6 +32,6 @@ RUN apt update && \
         psmisc \
         python3 \
         python3-sphinx \
-        $(apt-get -s build-dep qemu | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)
+        $(apt-get -s build-dep --arch-only qemu | egrep ^Inst | fgrep '[all]' | cut -d\  -f2)
 
 ENV FEATURES docs
-- 
2.20.1



^ permalink raw reply related	[flat|nested] 19+ messages in thread

* Re: [PATCH v1 1/9] docs/devel: re-organise the developers guide into sections
  2021-03-05  9:23 ` [PATCH v1 1/9] docs/devel: re-organise the developers guide into sections Alex Bennée
@ 2021-03-05  9:40   ` Philippe Mathieu-Daudé
  2021-03-05 13:37     ` Alex Bennée
  0 siblings, 1 reply; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-03-05  9:40 UTC (permalink / raw)
  To: Alex Bennée, qemu-devel
  Cc: fam, berrange, stefanha, crosa, pbonzini, aurelien

On 3/5/21 10:23 AM, Alex Bennée wrote:
> The list of sub-sections was getting a bit long and sporadically
> organised. Let's try and impose some order on this hairball of
> documentation.
> 
> [AJB: RFC because I wonder if we should make a more concerted effort
> to move bits of the wiki into a canonical maintained document. There
> is also probably a need for a quickbuild or tldr section of the
> build-system for users who just want to build something.]
> 
> Based-on: 20210223095931.16908-1-alex.bennee@linaro.org

^ IMO this hint for git bots should be removed from git history.

> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  docs/devel/index.rst                 | 32 ++++++----------------------
>  docs/devel/multi-thread-tcg.rst      |  5 +++--
>  docs/devel/section-apis.rst          | 16 ++++++++++++++
>  docs/devel/section-building.rst      | 13 +++++++++++
>  docs/devel/section-concepts.rst      | 21 ++++++++++++++++++
>  docs/devel/section-process.rst       | 11 ++++++++++
>  docs/devel/section-tcg-emulation.rst | 19 +++++++++++++++++
>  docs/devel/section-testing.rst       | 20 +++++++++++++++++
>  docs/devel/tcg-icount.rst            |  6 +++---
>  docs/devel/testing.rst               |  6 +++---
>  10 files changed, 115 insertions(+), 34 deletions(-)
>  create mode 100644 docs/devel/section-apis.rst
>  create mode 100644 docs/devel/section-building.rst
>  create mode 100644 docs/devel/section-concepts.rst
>  create mode 100644 docs/devel/section-process.rst
>  create mode 100644 docs/devel/section-tcg-emulation.rst
>  create mode 100644 docs/devel/section-testing.rst


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v1 3/9] tests/docker: add a test-tcg for building then running check-tcg
  2021-03-05  9:23 ` [PATCH v1 3/9] tests/docker: add a test-tcg for building then running check-tcg Alex Bennée
@ 2021-03-05  9:41   ` Philippe Mathieu-Daudé
  0 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-03-05  9:41 UTC (permalink / raw)
  To: Alex Bennée, qemu-devel
  Cc: fam, berrange, stefanha, crosa, pbonzini,
	Philippe Mathieu-Daudé,
	aurelien

On 3/5/21 10:23 AM, Alex Bennée wrote:
> This is mostly useful for verifying containers will work on the CI
> setup.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  tests/docker/test-tcg | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
>  create mode 100755 tests/docker/test-tcg

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH  v1 0/9] testing/next (docs, hexagon, cfi, docker)
  2021-03-05  9:23 [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) Alex Bennée
                   ` (8 preceding siblings ...)
  2021-03-05  9:23 ` [PATCH v1 9/9] tests/docker: Use --arch-only when building Debian cross image Alex Bennée
@ 2021-03-05  9:42 ` no-reply
  2021-03-05 19:37 ` Willian Rampazzo
  10 siblings, 0 replies; 19+ messages in thread
From: no-reply @ 2021-03-05  9:42 UTC (permalink / raw)
  To: alex.bennee
  Cc: fam, berrange, qemu-devel, f4bug, stefanha, crosa, pbonzini,
	alex.bennee, aurelien

Patchew URL: https://patchew.org/QEMU/20210305092328.31792-1-alex.bennee@linaro.org/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20210305092328.31792-1-alex.bennee@linaro.org
Subject: [PATCH  v1 0/9] testing/next (docs, hexagon, cfi, docker)

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]         patchew/20210305092328.31792-1-alex.bennee@linaro.org -> patchew/20210305092328.31792-1-alex.bennee@linaro.org
Switched to a new branch 'test'
91a7ef1 tests/docker: Use --arch-only when building Debian cross image
e0a95da gitlab-ci.yml: Add jobs to test CFI flags
360c5ee gitlab-ci.yml: Allow custom # of parallel linkers
222a269 gitlab: add build-user-hexagon test
e226953 tests/tcg: Use Hexagon Docker image
ffa599c docker: Add Hexagon image
14ff621 tests/docker: add a test-tcg for building then running check-tcg
c540ed9 docs/system: add a gentle prompt for the complexity to come
a190f84 docs/devel: re-organise the developers guide into sections

=== OUTPUT BEGIN ===
1/9 Checking commit a190f84a7607 (docs/devel: re-organise the developers guide into sections)
Use of uninitialized value $acpi_testexpected in string eq at ./scripts/checkpatch.pl line 1529.
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#89: 
new file mode 100644

total: 0 errors, 1 warnings, 167 lines checked

Patch 1/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
2/9 Checking commit c540ed9c6222 (docs/system: add a gentle prompt for the complexity to come)
3/9 Checking commit 14ff6214abdf (tests/docker: add a test-tcg for building then running check-tcg)
Use of uninitialized value $acpi_testexpected in string eq at ./scripts/checkpatch.pl line 1529.
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#17: 
new file mode 100755

total: 0 errors, 1 warnings, 22 lines checked

Patch 3/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
4/9 Checking commit ffa599c3946f (docker: Add Hexagon image)
WARNING: line over 80 characters
#152: FILE: tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh:63:
+    -DCMAKE_ASM_FLAGS="-G0 -mlong-calls -fno-pic --target=hexagon-unknown-linux-musl " \

ERROR: line over 90 characters
#179: FILE: tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh:90:
+    CROSS_CFLAGS="-G0 -O0 -mv65 -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl" \

ERROR: line over 90 characters
#216: FILE: tests/docker/dockerfiles/debian-hexagon-cross.docker.d/build-toolchain.sh:127:
+    CFLAGS="-G0 -O0 -mv65 -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl" \

total: 2 errors, 1 warnings, 184 lines checked

Patch 4/9 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

5/9 Checking commit e2269539f712 (tests/tcg: Use Hexagon Docker image)
6/9 Checking commit 222a269a7bf9 (gitlab: add build-user-hexagon test)
7/9 Checking commit 360c5ee0193d (gitlab-ci.yml: Allow custom # of parallel linkers)
8/9 Checking commit e0a95daa34d9 (gitlab-ci.yml: Add jobs to test CFI flags)
9/9 Checking commit 91a7ef19c6c1 (tests/docker: Use --arch-only when building Debian cross image)
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20210305092328.31792-1-alex.bennee@linaro.org/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v1 6/9] gitlab: add build-user-hexagon test
  2021-03-05  9:23 ` [PATCH v1 6/9] gitlab: add build-user-hexagon test Alex Bennée
@ 2021-03-05  9:43   ` Philippe Mathieu-Daudé
  2021-03-05 15:37   ` Wainer dos Santos Moschetta
  1 sibling, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-03-05  9:43 UTC (permalink / raw)
  To: Alex Bennée, qemu-devel
  Cc: fam, Thomas Huth, berrange, Wainer dos Santos Moschetta,
	stefanha, crosa, pbonzini, Philippe Mathieu-Daudé,
	aurelien

On 3/5/21 10:23 AM, Alex Bennée wrote:
> We special case this as the container with the cross compiler for the
> tests takes so long to build it is manually uploaded into the
> registry.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  .gitlab-ci.yml | 11 +++++++++++
>  1 file changed, 11 insertions(+)

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v1 8/9] gitlab-ci.yml: Add jobs to test CFI flags
  2021-03-05  9:23 ` [PATCH v1 8/9] gitlab-ci.yml: Add jobs to test CFI flags Alex Bennée
@ 2021-03-05  9:43   ` Philippe Mathieu-Daudé
  0 siblings, 0 replies; 19+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-03-05  9:43 UTC (permalink / raw)
  To: Alex Bennée, qemu-devel
  Cc: fam, Thomas Huth, berrange, Wainer dos Santos Moschetta,
	Daniele Buono, stefanha, crosa, pbonzini,
	Philippe Mathieu-Daudé,
	aurelien

On 3/5/21 10:23 AM, Alex Bennée wrote:
> From: Daniele Buono <dbuono@linux.vnet.ibm.com>
> 
> QEMU has had options to enable control-flow integrity features
> for a few months now. Add two sets of build/check/acceptance
> jobs to ensure the binary produced is working fine.
> 
> The three sets allow testing of x86_64 binaries for x86_64, s390x,
> ppc64 and aarch64 targets
> 
> Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com>
> Message-Id: <20210304030948.9367-3-dbuono@linux.vnet.ibm.com>
> [AJB: tweak job names to avoid brands]
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>  .gitlab-ci.yml | 119 +++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 119 insertions(+)

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v1 1/9] docs/devel: re-organise the developers guide into sections
  2021-03-05  9:40   ` Philippe Mathieu-Daudé
@ 2021-03-05 13:37     ` Alex Bennée
  0 siblings, 0 replies; 19+ messages in thread
From: Alex Bennée @ 2021-03-05 13:37 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé
  Cc: fam, berrange, qemu-devel, stefanha, crosa, pbonzini, aurelien


Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> On 3/5/21 10:23 AM, Alex Bennée wrote:
>> The list of sub-sections was getting a bit long and sporadically
>> organised. Let's try and impose some order on this hairball of
>> documentation.
>> 
>> [AJB: RFC because I wonder if we should make a more concerted effort
>> to move bits of the wiki into a canonical maintained document. There
>> is also probably a need for a quickbuild or tldr section of the
>> build-system for users who just want to build something.]
>> 
>> Based-on: 20210223095931.16908-1-alex.bennee@linaro.org
>
> ^ IMO this hint for git bots should be removed from git history.

I'll add a check for that in my patch scripts.

>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
>>  docs/devel/index.rst                 | 32 ++++++----------------------
>>  docs/devel/multi-thread-tcg.rst      |  5 +++--
>>  docs/devel/section-apis.rst          | 16 ++++++++++++++
>>  docs/devel/section-building.rst      | 13 +++++++++++
>>  docs/devel/section-concepts.rst      | 21 ++++++++++++++++++
>>  docs/devel/section-process.rst       | 11 ++++++++++
>>  docs/devel/section-tcg-emulation.rst | 19 +++++++++++++++++
>>  docs/devel/section-testing.rst       | 20 +++++++++++++++++
>>  docs/devel/tcg-icount.rst            |  6 +++---
>>  docs/devel/testing.rst               |  6 +++---
>>  10 files changed, 115 insertions(+), 34 deletions(-)
>>  create mode 100644 docs/devel/section-apis.rst
>>  create mode 100644 docs/devel/section-building.rst
>>  create mode 100644 docs/devel/section-concepts.rst
>>  create mode 100644 docs/devel/section-process.rst
>>  create mode 100644 docs/devel/section-tcg-emulation.rst
>>  create mode 100644 docs/devel/section-testing.rst


-- 
Alex Bennée


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v1 6/9] gitlab: add build-user-hexagon test
  2021-03-05  9:23 ` [PATCH v1 6/9] gitlab: add build-user-hexagon test Alex Bennée
  2021-03-05  9:43   ` Philippe Mathieu-Daudé
@ 2021-03-05 15:37   ` Wainer dos Santos Moschetta
  1 sibling, 0 replies; 19+ messages in thread
From: Wainer dos Santos Moschetta @ 2021-03-05 15:37 UTC (permalink / raw)
  To: Alex Bennée, qemu-devel
  Cc: fam, Thomas Huth, berrange, f4bug, stefanha, crosa, pbonzini,
	Philippe Mathieu-Daudé,
	aurelien


On 3/5/21 6:23 AM, Alex Bennée wrote:
> We special case this as the container with the cross compiler for the
> tests takes so long to build it is manually uploaded into the
> registry.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
>   .gitlab-ci.yml | 11 +++++++++++
>   1 file changed, 11 insertions(+)


Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>


>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 8b6d495288..c2c5196b8b 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -417,6 +417,17 @@ build-user-static:
>       CONFIGURE_ARGS: --disable-tools --disable-system --static
>       MAKE_CHECK_ARGS: check-tcg
>   
> +# Because the hexagon cross-compiler takes so long to build we don't rely
> +# on the CI system to build it and hence this job has no dependency
> +# declared. The image is manually uploaded.
> +build-user-hexagon:
> +  <<: *native_build_job_definition
> +  variables:
> +    IMAGE: debian-hexagon-cross
> +    TARGETS: hexagon-linux-user
> +    CONFIGURE_ARGS: --disable-tools --disable-docs --enable-debug-tcg
> +    MAKE_CHECK_ARGS: check-tcg
> +
>   # Only build the softmmu targets we have check-tcg tests for
>   build-some-softmmu:
>     <<: *native_build_job_definition



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker)
  2021-03-05  9:23 [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) Alex Bennée
                   ` (9 preceding siblings ...)
  2021-03-05  9:42 ` [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) no-reply
@ 2021-03-05 19:37 ` Willian Rampazzo
  2021-03-07 17:29   ` Alex Bennée
  10 siblings, 1 reply; 19+ messages in thread
From: Willian Rampazzo @ 2021-03-05 19:37 UTC (permalink / raw)
  To: Alex Bennée
  Cc: Fam Zheng, Daniel Berrange, qemu-devel,
	Philippe Mathieu-Daudé,
	Stefan Hajnoczi, Cleber Rosa Junior, Paolo Bonzini,
	Aurelien Jarno

On Fri, Mar 5, 2021 at 6:31 AM Alex Bennée <alex.bennee@linaro.org> wrote:
>
> Hi,
>
> Another week another testing/next roll. The series includes a couple
> of my proposed documentation tweaks (including the re-org of the devel
> manual). We also enable testing for the hexagon linux-user target to
> avoid bitrot. I've documented the process for manually updating the
> registry at:
>
>  https://wiki.qemu.org/Testing/DockerBuild#The_GitLab_Registry
>

I see a mix of valid/invalid URLs, like `registry.gitlab.example.com`
and then `registry.gitlab.com`. For consistency, my suggestion is to
use either the example URL or a valid URL mentioning that is an
example for a specific docker image.



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker)
  2021-03-05 19:37 ` Willian Rampazzo
@ 2021-03-07 17:29   ` Alex Bennée
  0 siblings, 0 replies; 19+ messages in thread
From: Alex Bennée @ 2021-03-07 17:29 UTC (permalink / raw)
  To: Willian Rampazzo
  Cc: Fam Zheng, Daniel Berrange, qemu-devel,
	Philippe Mathieu-Daudé,
	Stefan Hajnoczi, Cleber Rosa Junior, Paolo Bonzini,
	Aurelien Jarno


Willian Rampazzo <wrampazz@redhat.com> writes:

> On Fri, Mar 5, 2021 at 6:31 AM Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> Hi,
>>
>> Another week another testing/next roll. The series includes a couple
>> of my proposed documentation tweaks (including the re-org of the devel
>> manual). We also enable testing for the hexagon linux-user target to
>> avoid bitrot. I've documented the process for manually updating the
>> registry at:
>>
>>  https://wiki.qemu.org/Testing/DockerBuild#The_GitLab_Registry
>>
>
> I see a mix of valid/invalid URLs, like `registry.gitlab.example.com`
> and then `registry.gitlab.com`. For consistency, my suggestion is to
> use either the example URL or a valid URL mentioning that is an
> example for a specific docker image.

Oops bad copy/paste from my shell history. Fixed now.


-- 
Alex Bennée


^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2021-03-07 17:31 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-05  9:23 [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) Alex Bennée
2021-03-05  9:23 ` [PATCH v1 1/9] docs/devel: re-organise the developers guide into sections Alex Bennée
2021-03-05  9:40   ` Philippe Mathieu-Daudé
2021-03-05 13:37     ` Alex Bennée
2021-03-05  9:23 ` [PATCH v1 2/9] docs/system: add a gentle prompt for the complexity to come Alex Bennée
2021-03-05  9:23 ` [PATCH v1 3/9] tests/docker: add a test-tcg for building then running check-tcg Alex Bennée
2021-03-05  9:41   ` Philippe Mathieu-Daudé
2021-03-05  9:23 ` [PATCH v1 4/9] docker: Add Hexagon image Alex Bennée
2021-03-05  9:23 ` [PATCH v1 5/9] tests/tcg: Use Hexagon Docker image Alex Bennée
2021-03-05  9:23 ` [PATCH v1 6/9] gitlab: add build-user-hexagon test Alex Bennée
2021-03-05  9:43   ` Philippe Mathieu-Daudé
2021-03-05 15:37   ` Wainer dos Santos Moschetta
2021-03-05  9:23 ` [PATCH v1 7/9] gitlab-ci.yml: Allow custom # of parallel linkers Alex Bennée
2021-03-05  9:23 ` [PATCH v1 8/9] gitlab-ci.yml: Add jobs to test CFI flags Alex Bennée
2021-03-05  9:43   ` Philippe Mathieu-Daudé
2021-03-05  9:23 ` [PATCH v1 9/9] tests/docker: Use --arch-only when building Debian cross image Alex Bennée
2021-03-05  9:42 ` [PATCH v1 0/9] testing/next (docs, hexagon, cfi, docker) no-reply
2021-03-05 19:37 ` Willian Rampazzo
2021-03-07 17:29   ` Alex Bennée

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.