* [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests
@ 2018-04-04 2:51 Fam Zheng
2018-04-04 2:51 ` [Qemu-devel] [PATCH 1/3] archive-source.sh: Drop submodule code Fam Zheng
` (7 more replies)
0 siblings, 8 replies; 15+ messages in thread
From: Fam Zheng @ 2018-04-04 2:51 UTC (permalink / raw)
To: qemu-devel
Cc: pbonzini, Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé
Docker testing on patchew has long suffered from 'make check' hangings. The
cleanness of VM testing is the cure. Now let's add a CentOS 7 image to run the
tests. It's purely ad-hoc, but hopefully still easy to understand and use for
everyone.
The first patch makes passing source code from host to the container in VM
working, and is a nice clean up.
The second patch makes caches work, to speed up repetitive runs like on
patchew.
The last patch adds the new image that does the job. Two out of three docker
tests running on patchew.org are added to the image. I'll wait for Peter to fix
the 'docker-test-quick@centos6' hanging (oob test) before adding it too.
Fam
Fam Zheng (3):
archive-source.sh: Drop submodule code
tests: Add an option for snapshot (default: off)
tests: Add centos VM testing
scripts/archive-source.sh | 47 +++------------------------
tests/vm/basevm.py | 7 +++-
tests/vm/centos | 82 +++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 92 insertions(+), 44 deletions(-)
create mode 100755 tests/vm/centos
--
2.14.3
^ permalink raw reply [flat|nested] 15+ messages in thread
* [Qemu-devel] [PATCH 1/3] archive-source.sh: Drop submodule code
2018-04-04 2:51 [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests Fam Zheng
@ 2018-04-04 2:51 ` Fam Zheng
2018-04-04 8:23 ` Daniel P. Berrangé
2018-04-04 2:52 ` [Qemu-devel] [PATCH 2/3] tests: Add an option for snapshot (default: off) Fam Zheng
` (6 subsequent siblings)
7 siblings, 1 reply; 15+ messages in thread
From: Fam Zheng @ 2018-04-04 2:51 UTC (permalink / raw)
To: qemu-devel
Cc: pbonzini, Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé
./configure and make now take care of submodules, we only need to clone
the top tree.
Signed-off-by: Fam Zheng <famz@redhat.com>
---
scripts/archive-source.sh | 47 ++++-------------------------------------------
1 file changed, 4 insertions(+), 43 deletions(-)
diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
index 4e63774f9a..d53774d507 100755
--- a/scripts/archive-source.sh
+++ b/scripts/archive-source.sh
@@ -18,51 +18,12 @@ if test $# -lt 1; then
error "Usage: $0 <output tarball>"
fi
-tar_file=`realpath "$1"`
-list_file="${tar_file}.list"
-vroot_dir="${tar_file}.vroot"
+tar_file="$1"
+list_file="$1.list"
-# We want a predictable list of submodules for builds, that is
-# independent of what the developer currently has initialized
-# in their checkout, because the build environment is completely
-# different to the host OS.
-submodules="dtc ui/keycodemapdb"
+trap "status=$?; rm -f \"$list_file\"; exit \$status" 0 1 2 3 15
-trap "status=$?; rm -rf \"$list_file\" \"$vroot_dir\"; exit \$status" 0 1 2 3 15
-
-if git diff-index --quiet HEAD -- &>/dev/null
-then
- HEAD=HEAD
-else
- HEAD=`git stash create`
-fi
-git clone --shared . "$vroot_dir"
-test $? -ne 0 && error "failed to clone into '$vroot_dir'"
-
-cd "$vroot_dir"
-test $? -ne 0 && error "failed to change into '$vroot_dir'"
-
-git checkout $HEAD
-test $? -ne 0 && error "failed to checkout $HEAD revision"
-
-for sm in $submodules; do
- git submodule update --init $sm
- test $? -ne 0 && error "failed to init submodule $sm"
-done
-
-if test -n "$submodules"; then
- {
- git ls-files || error "git ls-files failed"
- for sm in $submodules; do
- (cd $sm; git ls-files) | sed "s:^:$sm/:"
- if test "${PIPESTATUS[*]}" != "0 0"; then
- error "git ls-files in submodule $sm failed"
- fi
- done
- } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$list_file"
-else
- git ls-files > "$list_file"
-fi
+( git ls-files && echo '.git' ) > "$list_file"
if test $? -ne 0; then
error "failed to generate list file"
--
2.14.3
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [Qemu-devel] [PATCH 2/3] tests: Add an option for snapshot (default: off)
2018-04-04 2:51 [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests Fam Zheng
2018-04-04 2:51 ` [Qemu-devel] [PATCH 1/3] archive-source.sh: Drop submodule code Fam Zheng
@ 2018-04-04 2:52 ` Fam Zheng
2018-04-04 2:52 ` [Qemu-devel] [PATCH 3/3] tests: Add centos VM testing Fam Zheng
` (5 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: Fam Zheng @ 2018-04-04 2:52 UTC (permalink / raw)
To: qemu-devel
Cc: pbonzini, Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé
Not using snapshot has the benefit of automatically persisting useful
test harnesses, such as docker images and ccache database. Although it
will lose some cleanness, it is imaginably useful for patchew.
Signed-off-by: Fam Zheng <famz@redhat.com>
---
tests/vm/basevm.py | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 686d88decf..960e13f349 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -218,6 +218,8 @@ def parse_args(vm_name):
help="build QEMU from source in guest")
parser.add_option("--interactive", "-I", action="store_true",
help="Interactively run command")
+ parser.add_option("--snapshot", "-s", action="store_true",
+ help="run tests with a snapshot")
parser.disable_interspersed_args()
return parser.parse_args()
@@ -243,7 +245,10 @@ def main(vmcls):
jobs=args.jobs)]
else:
cmd = argv
- vm.boot(args.image + ",snapshot=on")
+ img = args.image
+ if args.snapshot:
+ img += ",snapshot=on"
+ vm.boot(img)
vm.wait_ssh()
except Exception as e:
if isinstance(e, SystemExit) and e.code == 0:
--
2.14.3
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [Qemu-devel] [PATCH 3/3] tests: Add centos VM testing
2018-04-04 2:51 [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests Fam Zheng
2018-04-04 2:51 ` [Qemu-devel] [PATCH 1/3] archive-source.sh: Drop submodule code Fam Zheng
2018-04-04 2:52 ` [Qemu-devel] [PATCH 2/3] tests: Add an option for snapshot (default: off) Fam Zheng
@ 2018-04-04 2:52 ` Fam Zheng
2018-04-04 3:01 ` [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests no-reply
` (4 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: Fam Zheng @ 2018-04-04 2:52 UTC (permalink / raw)
To: qemu-devel
Cc: pbonzini, Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé
This one does docker testing in the VM. It is intended to replace the
native docker testing on patchew testers.
Signed-off-by: Fam Zheng <famz@redhat.com>
---
tests/vm/centos | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 82 insertions(+)
create mode 100755 tests/vm/centos
diff --git a/tests/vm/centos b/tests/vm/centos
new file mode 100755
index 0000000000..1a0d1fc0a2
--- /dev/null
+++ b/tests/vm/centos
@@ -0,0 +1,82 @@
+#!/usr/bin/env python
+#
+# CentOS image
+#
+# Copyright 2018 Red Hat Inc.
+#
+# Authors:
+# Fam Zheng <famz@redhat.com>
+#
+# This code is licensed under the GPL version 2 or later. See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import sys
+import subprocess
+import basevm
+import time
+
+class CentosVM(basevm.BaseVM):
+ name = "centos"
+ BUILD_SCRIPT = """
+ set -e;
+ cd $(mktemp -d);
+ sudo chmod a+r /dev/vdb;
+ tar -xf /dev/vdb;
+ make docker-test-mingw@fedora V=1 J=8;
+ make docker-test-build@min-glib V=1 J=8;
+ """
+
+ def _gen_cloud_init_iso(self):
+ cidir = self._tmpdir
+ mdata = open(os.path.join(cidir, "meta-data"), "w")
+ mdata.writelines(["instance-id: centos-vm-0\n",
+ "local-hostname: centos-guest\n"])
+ mdata.close()
+ udata = open(os.path.join(cidir, "user-data"), "w")
+ udata.writelines(["#cloud-config\n",
+ "chpasswd:\n",
+ " list: |\n",
+ " root:%s\n" % self.ROOT_PASS,
+ " %s:%s\n" % (self.GUEST_USER, self.GUEST_PASS),
+ " expire: False\n",
+ "users:\n",
+ " - name: %s\n" % self.GUEST_USER,
+ " sudo: ALL=(ALL) NOPASSWD:ALL\n",
+ " ssh-authorized-keys:\n",
+ " - %s\n" % basevm.SSH_PUB_KEY,
+ " - name: root\n",
+ " ssh-authorized-keys:\n",
+ " - %s\n" % basevm.SSH_PUB_KEY,
+ "locale: en_US.UTF-8\n"])
+ udata.close()
+ subprocess.check_call(["genisoimage", "-output", "cloud-init.iso",
+ "-volid", "cidata", "-joliet", "-rock",
+ "user-data", "meta-data"],
+ cwd=cidir,
+ stdin=self._devnull, stdout=self._stdout,
+ stderr=self._stdout)
+ return os.path.join(cidir, "cloud-init.iso")
+
+ def build_image(self, img):
+ cimg = self._download_with_cache("https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1802.qcow2.xz")
+ img_tmp = img + ".tmp"
+ subprocess.check_call(["cp", "-f", cimg, img_tmp + ".xz"])
+ subprocess.check_call(["xz", "-df", img_tmp + ".xz"])
+ subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
+ self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
+ self.wait_ssh()
+ self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")
+ self.ssh_root_check("yum update -y")
+ self.ssh_root_check("yum install -y docker make git")
+ self.ssh_root_check("systemctl enable docker")
+ self.ssh_root("poweroff")
+ self.wait()
+ if os.path.exists(img):
+ os.remove(img)
+ os.rename(img_tmp, img)
+ return 0
+
+if __name__ == "__main__":
+ sys.exit(basevm.main(CentosVM))
--
2.14.3
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests
2018-04-04 2:51 [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests Fam Zheng
` (2 preceding siblings ...)
2018-04-04 2:52 ` [Qemu-devel] [PATCH 3/3] tests: Add centos VM testing Fam Zheng
@ 2018-04-04 3:01 ` no-reply
2018-04-04 3:04 ` no-reply
` (3 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: no-reply @ 2018-04-04 3:01 UTC (permalink / raw)
To: famz; +Cc: qemu-devel, alex.bennee, pbonzini, f4bug
Hi,
This series failed docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
Type: series
Message-id: 20180404025201.11921-1-famz@redhat.com
Subject: [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests
=== TEST SCRIPT BEGIN ===
#!/bin/bash
set -e
git submodule update --init dtc
# Let docker tests dump environment info
export SHOW_ENV=1
export J=8
time make docker-test-mingw@fedora
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
99d3cb1498 tests: Add centos VM testing
a2d747b22b tests: Add an option for snapshot (default: off)
9410826528 archive-source.sh: Drop submodule code
=== OUTPUT BEGIN ===
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-d015hcwx/src/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
BUILD fedora
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-d015hcwx/src'
GEN /var/tmp/patchew-tester-tmp-d015hcwx/src/docker-src.2018-04-03-22.58.48.13379/qemu.tar
COPY RUNNER
RUN test-mingw in qemu:fedora
Packages installed:
PyYAML-3.12-5.fc27.x86_64
SDL-devel-1.2.15-29.fc27.x86_64
bc-1.07.1-3.fc27.x86_64
bison-3.0.4-8.fc27.x86_64
bzip2-1.0.6-24.fc27.x86_64
ccache-3.3.6-1.fc27.x86_64
clang-5.0.1-3.fc27.x86_64
findutils-4.6.0-16.fc27.x86_64
flex-2.6.1-5.fc27.x86_64
gcc-7.3.1-5.fc27.x86_64
gcc-c++-7.3.1-5.fc27.x86_64
gettext-0.19.8.1-12.fc27.x86_64
git-2.14.3-3.fc27.x86_64
glib2-devel-2.54.3-2.fc27.x86_64
hostname-3.18-4.fc27.x86_64
libaio-devel-0.3.110-9.fc27.x86_64
libasan-7.3.1-5.fc27.x86_64
libfdt-devel-1.4.6-1.fc27.x86_64
libubsan-7.3.1-5.fc27.x86_64
llvm-5.0.1-3.fc27.x86_64
make-4.2.1-4.fc27.x86_64
mingw32-SDL-1.2.15-9.fc27.noarch
mingw32-bzip2-1.0.6-9.fc27.noarch
mingw32-curl-7.54.1-2.fc27.noarch
mingw32-glib2-2.54.1-1.fc27.noarch
mingw32-gmp-6.1.2-2.fc27.noarch
mingw32-gnutls-3.5.13-2.fc27.noarch
mingw32-gtk2-2.24.31-4.fc27.noarch
mingw32-gtk3-3.22.16-1.fc27.noarch
mingw32-libjpeg-turbo-1.5.1-3.fc27.noarch
mingw32-libpng-1.6.29-2.fc27.noarch
mingw32-libssh2-1.8.0-3.fc27.noarch
mingw32-libtasn1-4.13-1.fc27.noarch
mingw32-nettle-3.3-3.fc27.noarch
mingw32-pixman-0.34.0-3.fc27.noarch
mingw32-pkg-config-0.28-9.fc27.x86_64
mingw64-SDL-1.2.15-9.fc27.noarch
mingw64-bzip2-1.0.6-9.fc27.noarch
mingw64-curl-7.54.1-2.fc27.noarch
mingw64-glib2-2.54.1-1.fc27.noarch
mingw64-gmp-6.1.2-2.fc27.noarch
mingw64-gnutls-3.5.13-2.fc27.noarch
mingw64-gtk2-2.24.31-4.fc27.noarch
mingw64-gtk3-3.22.16-1.fc27.noarch
mingw64-libjpeg-turbo-1.5.1-3.fc27.noarch
mingw64-libpng-1.6.29-2.fc27.noarch
mingw64-libssh2-1.8.0-3.fc27.noarch
mingw64-libtasn1-4.13-1.fc27.noarch
mingw64-nettle-3.3-3.fc27.noarch
mingw64-pixman-0.34.0-3.fc27.noarch
mingw64-pkg-config-0.28-9.fc27.x86_64
nettle-devel-3.4-1.fc27.x86_64
perl-5.26.1-403.fc27.x86_64
pixman-devel-0.34.0-4.fc27.x86_64
python3-3.6.2-13.fc27.x86_64
sparse-0.5.1-2.fc27.x86_64
tar-1.29-7.fc27.x86_64
which-2.21-4.fc27.x86_64
zlib-devel-1.2.11-4.fc27.x86_64
Environment variables:
TARGET_LIST=
PACKAGES=ccache gettext git tar PyYAML sparse flex bison python3 bzip2 hostname glib2-devel pixman-devel zlib-devel SDL-devel libfdt-devel gcc gcc-c++ llvm clang make perl which bc findutils libaio-devel nettle-devel libasan libubsan mingw32-pixman mingw32-glib2 mingw32-gmp mingw32-SDL mingw32-pkg-config mingw32-gtk2 mingw32-gtk3 mingw32-gnutls mingw32-nettle mingw32-libtasn1 mingw32-libjpeg-turbo mingw32-libpng mingw32-curl mingw32-libssh2 mingw32-bzip2 mingw64-pixman mingw64-glib2 mingw64-gmp mingw64-SDL mingw64-pkg-config mingw64-gtk2 mingw64-gtk3 mingw64-gnutls mingw64-nettle mingw64-libtasn1 mingw64-libjpeg-turbo mingw64-libpng mingw64-curl mingw64-libssh2 mingw64-bzip2
J=8
V=
HOSTNAME=79564714ea88
DEBUG=
SHOW_ENV=1
PWD=/
HOME=/root
CCACHE_DIR=/var/tmp/ccache
DISTTAG=f27container
QEMU_CONFIGURE_OPTS=--python=/usr/bin/python3
FGC=f27
TEST_DIR=/tmp/qemu-test
SHLVL=1
FEATURES=mingw clang pyyaml asan dtc
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
MAKEFLAGS= -j8
EXTRA_CONFIGURE_OPTS=
_=/usr/bin/env
Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install --python=/usr/bin/python3 --cross-prefix=x86_64-w64-mingw32- --enable-trace-backends=simple --enable-gnutls --enable-nettle --enable-curl --enable-vnc --enable-bzip2 --enable-guest-agent --with-sdlabi=1.2 --with-gtkabi=2.0
Install prefix /tmp/qemu-test/install
BIOS directory /tmp/qemu-test/install
firmware path /tmp/qemu-test/install/share/qemu-firmware
binary directory /tmp/qemu-test/install
library directory /tmp/qemu-test/install/lib
module directory /tmp/qemu-test/install/lib
libexec directory /tmp/qemu-test/install/libexec
include directory /tmp/qemu-test/install/include
config directory /tmp/qemu-test/install
local state directory queried at runtime
Windows SDK no
Source path /tmp/qemu-test/src
GIT binary git
GIT submodules ui/keycodemapdb dtc capstone
C compiler x86_64-w64-mingw32-gcc
Host C compiler cc
C++ compiler x86_64-w64-mingw32-g++
Objective-C compiler clang
ARFLAGS rv
CFLAGS -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
QEMU_CFLAGS -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/pixman-1 -I$(SRC_PATH)/dtc/libfdt -Werror -DHAS_LIBSSH2_SFTP_FSYNC -mms-bitfields -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/glib-2.0 -I/usr/x86_64-w64-mingw32/sys-root/mingw/lib/glib-2.0/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -m64 -mcx16 -mthreads -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/p11-kit-1 -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/libpng16 -I$(SRC_PATH)/capstone/include
LDFLAGS -Wl,--nxcompat -Wl,--no-seh -Wl,--dynamicbase -Wl,--warn-common -m64 -g
make make
install install
python /usr/bin/python3 -B
smbd /usr/sbin/smbd
module support no
host CPU x86_64
host big endian no
target list x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabled no
strip binaries yes
profiler no
static build no
SDL support yes (1.2.15)
GTK support yes (2.24.31)
GTK GL support no
VTE support no
TLS priority NORMAL
GNUTLS support yes
GNUTLS rnd yes
libgcrypt no
libgcrypt kdf no
nettle yes (3.3)
nettle kdf yes
libtasn1 yes
curses support no
virgl support no
curl support yes
mingw32 support yes
Audio drivers dsound
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
Multipath support no
VNC support yes
VNC SASL support no
VNC JPEG support yes
VNC PNG support yes
xen support no
brlapi support no
bluez support no
Documentation no
PIE no
vde support no
netmap support no
Linux AIO support no
ATTR/XATTR support no
Install blobs yes
KVM support no
HAX support yes
HVF support no
WHPX support no
TCG support yes
TCG debug enabled no
TCG interpreter no
malloc trim support no
RDMA support no
fdt support yes
membarrier no
preadv support no
fdatasync no
madvise no
posix_madvise no
posix_memalign no
libcap-ng support no
vhost-net support no
vhost-crypto support no
vhost-scsi support no
vhost-vsock support no
vhost-user support no
Trace backends simple
Trace output file trace-<pid>
spice support no
rbd support no
xfsctl support no
smartcard support no
libusb no
usb net redir no
OpenGL support no
OpenGL dmabufs no
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info yes
QGA MSI support no
seccomp support no
coroutine backend win32
coroutine pool yes
debug stack usage no
crypto afalg no
GlusterFS support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support yes
TPM passthrough no
TPM emulator no
QOM debugging yes
Live block migration yes
lzo support no
snappy support no
bzip2 support yes
NUMA host support no
libxml2 no
tcmalloc support no
jemalloc support no
avx2 optimization yes
replication support yes
VxHS block device no
capstone git
WARNING: Use of GTK 2.0 is deprecated and will be removed in
WARNING: future releases. Please switch to using GTK 3.0
WARNING: Use of SDL 1.2 is deprecated and will be removed in
WARNING: future releases. Please switch to using SDL 2.0
GEN x86_64-softmmu/config-devices.mak.tmp
GEN aarch64-softmmu/config-devices.mak.tmp
GEN config-host.h
GIT ui/keycodemapdb dtc capstone
GEN qemu-options.def
GEN qapi-gen
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers-wrappers.h
GEN x86_64-softmmu/config-devices.mak
GEN aarch64-softmmu/config-devices.mak
GEN trace/generated-helpers.h
GEN trace/generated-helpers.c
GEN module_block.h
GEN tests/test-qapi-gen
GEN trace-root.h
GEN util/trace.h
GEN crypto/trace.h
GEN io/trace.h
GEN migration/trace.h
Submodule 'capstone' (git://git.qemu.org/capstone.git) registered for path 'capstone'
Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Cloning into '/tmp/qemu-test/src/capstone'...
GEN block/trace.h
GEN chardev/trace.h
GEN hw/block/trace.h
GEN hw/block/dataplane/trace.h
GEN hw/char/trace.h
GEN hw/intc/trace.h
GEN hw/net/trace.h
fatal: Unable to look up git.qemu.org (port 9418) (Name or service not known)
GEN hw/rdma/trace.h
GEN hw/rdma/vmw/trace.h
GEN hw/virtio/trace.h
fatal: clone of 'git://git.qemu.org/capstone.git' into submodule path '/tmp/qemu-test/src/capstone' failed
Failed to clone 'capstone'. Retry scheduled
Cloning into '/tmp/qemu-test/src/ui/keycodemapdb'...
fatal: Unable to look up git.qemu.org (port 9418) (Name or service not known)
GEN hw/audio/trace.h
GEN hw/misc/trace.h
GEN hw/misc/macio/trace.h
GEN hw/usb/trace.h
GEN hw/scsi/trace.h
fatal: clone of 'git://git.qemu.org/keycodemapdb.git' into submodule path '/tmp/qemu-test/src/ui/keycodemapdb' failed
Failed to clone 'ui/keycodemapdb'. Retry scheduled
Cloning into '/tmp/qemu-test/src/capstone'...
fatal: Unable to look up git.qemu.org (port 9418) (Name or service not known)
GEN hw/nvram/trace.h
GEN hw/display/trace.h
GEN hw/input/trace.h
fatal: clone of 'git://git.qemu.org/capstone.git' into submodule path '/tmp/qemu-test/src/capstone' failed
Failed to clone 'capstone' a second time, aborting
GEN hw/timer/trace.h
./scripts/git-submodule.sh: failed to update modules
Unable to automatically checkout GIT submodules ' ui/keycodemapdb dtc capstone'.
If you require use of an alternative GIT binary (for example to
enable use of a transparent proxy), then please specify it by
running configure by with the '--with-git' argument. e.g.
$ ./configure --with-git='tsocks git'
Alternatively you may disable automatic GIT submodule checkout
with:
$ ./configure --disable-git-update
and then manually update submodules prior to running make, with:
$ scripts/git-submodule.sh update ui/keycodemapdb dtc capstone
make: *** [Makefile:47: git-submodule-update] Error 1
make: *** Waiting for unfinished jobs....
Traceback (most recent call last):
File "./tests/docker/docker.py", line 407, in <module>
sys.exit(main())
File "./tests/docker/docker.py", line 404, in main
return args.cmdobj.run(args, argv)
File "./tests/docker/docker.py", line 261, in run
return Docker().run(argv, args.keep, quiet=args.quiet)
File "./tests/docker/docker.py", line 229, in run
quiet=quiet)
File "./tests/docker/docker.py", line 147, in _do_check
return subprocess.check_call(self._command + cmd, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker', 'run', '--label', 'com.qemu.instance.uuid=3c2ec30637b411e89cb952540069c830', '-u', '0', '--security-opt', 'seccomp=unconfined', '--rm', '--net=none', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=8', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/root/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-d015hcwx/src/docker-src.2018-04-03-22.58.48.13379:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2
make[1]: *** [tests/docker/Makefile.include:129: docker-run] Error 1
make[1]: Leaving directory '/var/tmp/patchew-tester-tmp-d015hcwx/src'
make: *** [tests/docker/Makefile.include:163: docker-run-test-mingw@fedora] Error 2
real 2m41.698s
user 0m1.618s
sys 0m3.855s
=== OUTPUT END ===
Test command exited with code: 2
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests
2018-04-04 2:51 [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests Fam Zheng
` (3 preceding siblings ...)
2018-04-04 3:01 ` [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests no-reply
@ 2018-04-04 3:04 ` no-reply
2018-04-04 5:03 ` Peter Xu
` (2 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: no-reply @ 2018-04-04 3:04 UTC (permalink / raw)
To: famz; +Cc: qemu-devel, alex.bennee, pbonzini, f4bug
Hi,
This series failed docker-build@min-glib build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
Type: series
Message-id: 20180404025201.11921-1-famz@redhat.com
Subject: [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests
=== TEST SCRIPT BEGIN ===
#!/bin/bash
set -e
git submodule update --init dtc
# Let docker tests dump environment info
export SHOW_ENV=1
export J=8
time make docker-test-build@min-glib
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
99d3cb1498 tests: Add centos VM testing
a2d747b22b tests: Add an option for snapshot (default: off)
9410826528 archive-source.sh: Drop submodule code
=== OUTPUT BEGIN ===
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-_g2ozc9x/src/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
BUILD min-glib
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-_g2ozc9x/src'
GEN /var/tmp/patchew-tester-tmp-_g2ozc9x/src/docker-src.2018-04-03-23.02.08.16350/qemu.tar
COPY RUNNER
RUN test-build in qemu:min-glib
Environment variables:
HOSTNAME=c7583a32ef00
MAKEFLAGS= -j8
J=8
CCACHE_DIR=/var/tmp/ccache
EXTRA_CONFIGURE_OPTS=
V=
SHOW_ENV=1
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
TARGET_LIST=
SHLVL=1
HOME=/root
TEST_DIR=/tmp/qemu-test
FEATURES= dtc
DEBUG=
_=/usr/bin/env
Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install
No C++ compiler available; disabling C++ specific optional code
Install prefix /tmp/qemu-test/install
BIOS directory /tmp/qemu-test/install/share/qemu
firmware path /tmp/qemu-test/install/share/qemu-firmware
binary directory /tmp/qemu-test/install/bin
library directory /tmp/qemu-test/install/lib
module directory /tmp/qemu-test/install/lib/qemu
libexec directory /tmp/qemu-test/install/libexec
include directory /tmp/qemu-test/install/include
config directory /tmp/qemu-test/install/etc
local state directory /tmp/qemu-test/install/var
Manual directory /tmp/qemu-test/install/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path /tmp/qemu-test/src
GIT binary git
GIT submodules ui/keycodemapdb dtc capstone
C compiler cc
Host C compiler cc
C++ compiler
Objective-C compiler cc
ARFLAGS rv
CFLAGS -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
QEMU_CFLAGS -I/usr/include/pixman-1 -I$(SRC_PATH)/dtc/libfdt -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wendif-labels -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-all -Wno-missing-braces -I$(SRC_PATH)/capstone/include
LDFLAGS -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g
make make
install install
python python -B
smbd /usr/sbin/smbd
module support no
host CPU x86_64
host big endian no
target list x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabled no
strip binaries yes
profiler no
static build no
SDL support yes (1.2.14)
GTK support no
GTK GL support no
VTE support no
TLS priority NORMAL
GNUTLS support no
GNUTLS rnd no
libgcrypt no
libgcrypt kdf no
nettle no
nettle kdf no
libtasn1 no
curses support no
virgl support no
curl support no
mingw32 support no
Audio drivers oss
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
Multipath support no
VNC support yes
VNC SASL support no
VNC JPEG support no
VNC PNG support no
xen support no
brlapi support no
bluez support no
Documentation no
PIE yes
vde support no
netmap support no
Linux AIO support no
ATTR/XATTR support yes
Install blobs yes
KVM support yes
HAX support no
HVF support no
WHPX support no
TCG support yes
TCG debug enabled no
TCG interpreter no
malloc trim support yes
RDMA support no
fdt support yes
membarrier no
preadv support yes
fdatasync yes
madvise yes
posix_madvise yes
posix_memalign yes
libcap-ng support no
vhost-net support yes
vhost-crypto support yes
vhost-scsi support yes
vhost-vsock support yes
vhost-user support yes
Trace backends log
spice support no
rbd support no
xfsctl support no
smartcard support no
libusb no
usb net redir no
OpenGL support no
OpenGL dmabufs no
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info no
QGA MSI support no
seccomp support no
coroutine backend ucontext
coroutine pool yes
debug stack usage no
crypto afalg no
GlusterFS support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support no
TPM passthrough yes
TPM emulator yes
QOM debugging yes
Live block migration yes
lzo support no
snappy support no
bzip2 support no
NUMA host support no
libxml2 no
tcmalloc support no
jemalloc support no
avx2 optimization no
replication support yes
VxHS block device no
capstone git
WARNING: Use of SDL 1.2 is deprecated and will be removed in
WARNING: future releases. Please switch to using SDL 2.0
GEN x86_64-softmmu/config-devices.mak.tmp
GEN aarch64-softmmu/config-devices.mak.tmp
GEN config-host.h
GIT ui/keycodemapdb dtc capstone
GEN qemu-options.def
GEN qapi-gen
GEN trace/generated-tcg-tracers.h
GEN aarch64-softmmu/config-devices.mak
GEN x86_64-softmmu/config-devices.mak
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-helpers.h
GEN trace/generated-helpers.c
GEN module_block.h
GEN tests/test-qapi-gen
GEN trace-root.h
GEN util/trace.h
GEN crypto/trace.h
GEN io/trace.h
GEN migration/trace.h
GEN block/trace.h
GEN chardev/trace.h
GEN hw/block/trace.h
GEN hw/block/dataplane/trace.h
GEN hw/char/trace.h
GEN hw/intc/trace.h
GEN hw/net/trace.h
GEN hw/rdma/trace.h
GEN hw/rdma/vmw/trace.h
GEN hw/virtio/trace.h
GEN hw/audio/trace.h
GEN hw/misc/trace.h
GEN hw/misc/macio/trace.h
GEN hw/usb/trace.h
GEN hw/scsi/trace.h
GEN hw/nvram/trace.h
GEN hw/display/trace.h
GEN hw/input/trace.h
GEN hw/timer/trace.h
GEN hw/dma/trace.h
GEN hw/sparc/trace.h
GEN hw/sparc64/trace.h
GEN hw/sd/trace.h
GEN hw/isa/trace.h
GEN hw/mem/trace.h
GEN hw/i386/trace.h
GEN hw/i386/xen/trace.h
GEN hw/9pfs/trace.h
GEN hw/ppc/trace.h
GEN hw/pci/trace.h
GEN hw/pci-host/trace.h
GEN hw/s390x/trace.h
GEN hw/vfio/trace.h
GEN hw/acpi/trace.h
GEN hw/arm/trace.h
GEN hw/alpha/trace.h
GEN hw/hppa/trace.h
fatal: Unable to look up git.qemu.org (port 9418) (Temporary failure in name resolution)
GEN hw/xen/trace.h
GEN hw/ide/trace.h
GEN hw/tpm/trace.h
Clone of 'git://git.qemu.org/capstone.git' into submodule path 'capstone' failed
./scripts/git-submodule.sh: failed to update modules
Unable to automatically checkout GIT submodules ' ui/keycodemapdb dtc capstone'.
If you require use of an alternative GIT binary (for example to
enable use of a transparent proxy), then please specify it by
running configure by with the '--with-git' argument. e.g.
$ ./configure --with-git='tsocks git'
Alternatively you may disable automatic GIT submodule checkout
with:
$ ./configure --disable-git-update
and then manually update submodules prior to running make, with:
$ scripts/git-submodule.sh update ui/keycodemapdb dtc capstone
make: *** [git-submodule-update] Error 1
make: *** Waiting for unfinished jobs....
make: *** wait: No child processes. Stop.
Traceback (most recent call last):
File "./tests/docker/docker.py", line 407, in <module>
sys.exit(main())
File "./tests/docker/docker.py", line 404, in main
return args.cmdobj.run(args, argv)
File "./tests/docker/docker.py", line 261, in run
return Docker().run(argv, args.keep, quiet=args.quiet)
File "./tests/docker/docker.py", line 229, in run
quiet=quiet)
File "./tests/docker/docker.py", line 147, in _do_check
return subprocess.check_call(self._command + cmd, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker', 'run', '--label', 'com.qemu.instance.uuid=abc5394837b411e8965d52540069c830', '-u', '0', '--security-opt', 'seccomp=unconfined', '--rm', '--net=none', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=8', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/root/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-_g2ozc9x/src/docker-src.2018-04-03-23.02.08.16350:/var/tmp/qemu:z,ro', 'qemu:min-glib', '/var/tmp/qemu/run', 'test-build']' returned non-zero exit status 2
make[1]: *** [tests/docker/Makefile.include:129: docker-run] Error 1
make[1]: Leaving directory '/var/tmp/patchew-tester-tmp-_g2ozc9x/src'
make: *** [tests/docker/Makefile.include:163: docker-run-test-build@min-glib] Error 2
real 2m17.711s
user 0m1.534s
sys 0m3.511s
=== OUTPUT END ===
Test command exited with code: 2
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests
2018-04-04 2:51 [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests Fam Zheng
` (4 preceding siblings ...)
2018-04-04 3:04 ` no-reply
@ 2018-04-04 5:03 ` Peter Xu
2018-04-04 6:56 ` Peter Xu
2018-04-04 8:19 ` Daniel P. Berrangé
2018-04-06 16:47 ` Paolo Bonzini
7 siblings, 1 reply; 15+ messages in thread
From: Peter Xu @ 2018-04-04 5:03 UTC (permalink / raw)
To: Fam Zheng
Cc: qemu-devel, Alex Bennée, pbonzini, Philippe Mathieu-Daudé
On Wed, Apr 04, 2018 at 10:51:58AM +0800, Fam Zheng wrote:
> Docker testing on patchew has long suffered from 'make check' hangings. The
> cleanness of VM testing is the cure. Now let's add a CentOS 7 image to run the
> tests. It's purely ad-hoc, but hopefully still easy to understand and use for
> everyone.
>
> The first patch makes passing source code from host to the container in VM
> working, and is a nice clean up.
>
> The second patch makes caches work, to speed up repetitive runs like on
> patchew.
>
> The last patch adds the new image that does the job. Two out of three docker
> tests running on patchew.org are added to the image. I'll wait for Peter to fix
> the 'docker-test-quick@centos6' hanging (oob test) before adding it too.
This hang does not happen on centos7 (tested myself manually). I
consider this can be related to different default versions of glib2:
centos6: glib2-2.28.8-9.el6.x86_64
centos7: glib2-2.50.3-3.el7.x86_64
(both fetched from docker.io)
qmp-test hangs at oob-test, while QEMU failed to respond to SIGTERM
when trying to take a lock:
#0 0x00007f18eed02334 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007f18eecfd5d8 in _L_lock_854 () from /lib64/libpthread.so.0
#2 0x00007f18eecfd4a7 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00007f18eb476c9d in ?? () from /lib64/libglib-2.0.so.0
#4 0x0000000000aa6672 in aio_context_unref (ctx=0x225b600) at /root/qemu/util/async.c:497
#5 0x000000000065851c in iothread_instance_finalize (obj=0x225b380) at /root/qemu/iothread.c:129
#6 0x0000000000962d79 in object_deinit (obj=0x225b380, type=0x2199960) at /root/qemu/qom/object.c:462
#7 0x0000000000962e0d in object_finalize (data=0x225b380) at /root/qemu/qom/object.c:476
#8 0x0000000000964146 in object_unref (obj=0x225b380) at /root/qemu/qom/object.c:924
#9 0x0000000000965880 in object_finalize_child_property (obj=0x2257640, name=0x225aca0 "mon_iothread", opaque=0x225b380) at /root/qemu/qom/object.c:1436
#10 0x0000000000962c33 in object_property_del_child (obj=0x2257640, child=0x225b380, errp=0x0) at /root/qemu/qom/object.c:436
#11 0x0000000000962d26 in object_unparent (obj=0x225b380) at /root/qemu/qom/object.c:455
#12 0x0000000000658f00 in iothread_destroy (iothread=0x225b380) at /root/qemu/iothread.c:365
#13 0x00000000004c67a8 in monitor_cleanup () at /root/qemu/monitor.c:4663
#14 0x0000000000669e27 in main (argc=16, argv=0x7ffe33ce3a08, envp=0x7ffe33ce3a90) at /root/qemu/vl.c:4749
Currently I have no quick idea on why the lock was held by anyone
else (I suppose that's the GMainContext's lock), and I'm not sure
whether it could be a glib bug.
--
Peter Xu
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests
2018-04-04 5:03 ` Peter Xu
@ 2018-04-04 6:56 ` Peter Xu
0 siblings, 0 replies; 15+ messages in thread
From: Peter Xu @ 2018-04-04 6:56 UTC (permalink / raw)
To: Fam Zheng
Cc: qemu-devel, Alex Bennée, pbonzini, Philippe Mathieu-Daudé
On Wed, Apr 04, 2018 at 01:03:47PM +0800, Peter Xu wrote:
> On Wed, Apr 04, 2018 at 10:51:58AM +0800, Fam Zheng wrote:
> > Docker testing on patchew has long suffered from 'make check' hangings. The
> > cleanness of VM testing is the cure. Now let's add a CentOS 7 image to run the
> > tests. It's purely ad-hoc, but hopefully still easy to understand and use for
> > everyone.
> >
> > The first patch makes passing source code from host to the container in VM
> > working, and is a nice clean up.
> >
> > The second patch makes caches work, to speed up repetitive runs like on
> > patchew.
> >
> > The last patch adds the new image that does the job. Two out of three docker
> > tests running on patchew.org are added to the image. I'll wait for Peter to fix
> > the 'docker-test-quick@centos6' hanging (oob test) before adding it too.
>
> This hang does not happen on centos7 (tested myself manually). I
> consider this can be related to different default versions of glib2:
>
> centos6: glib2-2.28.8-9.el6.x86_64
> centos7: glib2-2.50.3-3.el7.x86_64
>
> (both fetched from docker.io)
>
> qmp-test hangs at oob-test, while QEMU failed to respond to SIGTERM
> when trying to take a lock:
>
> #0 0x00007f18eed02334 in __lll_lock_wait () from /lib64/libpthread.so.0
> #1 0x00007f18eecfd5d8 in _L_lock_854 () from /lib64/libpthread.so.0
> #2 0x00007f18eecfd4a7 in pthread_mutex_lock () from /lib64/libpthread.so.0
> #3 0x00007f18eb476c9d in ?? () from /lib64/libglib-2.0.so.0
> #4 0x0000000000aa6672 in aio_context_unref (ctx=0x225b600) at /root/qemu/util/async.c:497
> #5 0x000000000065851c in iothread_instance_finalize (obj=0x225b380) at /root/qemu/iothread.c:129
> #6 0x0000000000962d79 in object_deinit (obj=0x225b380, type=0x2199960) at /root/qemu/qom/object.c:462
> #7 0x0000000000962e0d in object_finalize (data=0x225b380) at /root/qemu/qom/object.c:476
> #8 0x0000000000964146 in object_unref (obj=0x225b380) at /root/qemu/qom/object.c:924
> #9 0x0000000000965880 in object_finalize_child_property (obj=0x2257640, name=0x225aca0 "mon_iothread", opaque=0x225b380) at /root/qemu/qom/object.c:1436
> #10 0x0000000000962c33 in object_property_del_child (obj=0x2257640, child=0x225b380, errp=0x0) at /root/qemu/qom/object.c:436
> #11 0x0000000000962d26 in object_unparent (obj=0x225b380) at /root/qemu/qom/object.c:455
> #12 0x0000000000658f00 in iothread_destroy (iothread=0x225b380) at /root/qemu/iothread.c:365
> #13 0x00000000004c67a8 in monitor_cleanup () at /root/qemu/monitor.c:4663
> #14 0x0000000000669e27 in main (argc=16, argv=0x7ffe33ce3a08, envp=0x7ffe33ce3a90) at /root/qemu/vl.c:4749
>
> Currently I have no quick idea on why the lock was held by anyone
> else (I suppose that's the GMainContext's lock), and I'm not sure
> whether it could be a glib bug.
I posted a patch for this to workaround the possible glib bug:
[PATCH for-2.12] iothread: workaround glib bug which hangs qmp-test
Let's see whether we can accept that. Thanks,
--
Peter Xu
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests
2018-04-04 2:51 [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests Fam Zheng
` (5 preceding siblings ...)
2018-04-04 5:03 ` Peter Xu
@ 2018-04-04 8:19 ` Daniel P. Berrangé
2018-04-04 8:24 ` Fam Zheng
2018-04-06 16:47 ` Paolo Bonzini
7 siblings, 1 reply; 15+ messages in thread
From: Daniel P. Berrangé @ 2018-04-04 8:19 UTC (permalink / raw)
To: Fam Zheng
Cc: qemu-devel, Alex Bennée, pbonzini, Philippe Mathieu-Daudé
On Wed, Apr 04, 2018 at 10:51:58AM +0800, Fam Zheng wrote:
> Docker testing on patchew has long suffered from 'make check' hangings. The
> cleanness of VM testing is the cure. Now let's add a CentOS 7 image to run the
> tests. It's purely ad-hoc, but hopefully still easy to understand and use for
> everyone.
If our automated tests are hanging, then developers could hit those hangs
to, and depending on where the problem is, end users could hit the hangs
in production deployment.
IOW, hangs in the testing aren't something we should just hide away by
changing the way we run tests - we need to identify & fix the hangs.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Qemu-devel] [PATCH 1/3] archive-source.sh: Drop submodule code
2018-04-04 2:51 ` [Qemu-devel] [PATCH 1/3] archive-source.sh: Drop submodule code Fam Zheng
@ 2018-04-04 8:23 ` Daniel P. Berrangé
2018-04-04 9:07 ` Fam Zheng
0 siblings, 1 reply; 15+ messages in thread
From: Daniel P. Berrangé @ 2018-04-04 8:23 UTC (permalink / raw)
To: Fam Zheng
Cc: qemu-devel, Alex Bennée, pbonzini, Philippe Mathieu-Daudé
On Wed, Apr 04, 2018 at 10:51:59AM +0800, Fam Zheng wrote:
> ./configure and make now take care of submodules, we only need to clone
> the top tree.
If you don't include the submodules in the tar.gz archive, then the
test system needs to be able to clone them from qemu.org, which
requires networking.
The VM tests have networking by default, but the docker tests do
not have networking - you need to opt-in by setting NETWORK=1
make variable. So AFAICT, this will break docker tests.
>
> Signed-off-by: Fam Zheng <famz@redhat.com>
> ---
> scripts/archive-source.sh | 47 ++++-------------------------------------------
> 1 file changed, 4 insertions(+), 43 deletions(-)
>
> diff --git a/scripts/archive-source.sh b/scripts/archive-source.sh
> index 4e63774f9a..d53774d507 100755
> --- a/scripts/archive-source.sh
> +++ b/scripts/archive-source.sh
> @@ -18,51 +18,12 @@ if test $# -lt 1; then
> error "Usage: $0 <output tarball>"
> fi
>
> -tar_file=`realpath "$1"`
> -list_file="${tar_file}.list"
> -vroot_dir="${tar_file}.vroot"
> +tar_file="$1"
> +list_file="$1.list"
>
> -# We want a predictable list of submodules for builds, that is
> -# independent of what the developer currently has initialized
> -# in their checkout, because the build environment is completely
> -# different to the host OS.
> -submodules="dtc ui/keycodemapdb"
> +trap "status=$?; rm -f \"$list_file\"; exit \$status" 0 1 2 3 15
>
> -trap "status=$?; rm -rf \"$list_file\" \"$vroot_dir\"; exit \$status" 0 1 2 3 15
> -
> -if git diff-index --quiet HEAD -- &>/dev/null
> -then
> - HEAD=HEAD
> -else
> - HEAD=`git stash create`
> -fi
> -git clone --shared . "$vroot_dir"
> -test $? -ne 0 && error "failed to clone into '$vroot_dir'"
> -
> -cd "$vroot_dir"
> -test $? -ne 0 && error "failed to change into '$vroot_dir'"
> -
> -git checkout $HEAD
> -test $? -ne 0 && error "failed to checkout $HEAD revision"
> -
> -for sm in $submodules; do
> - git submodule update --init $sm
> - test $? -ne 0 && error "failed to init submodule $sm"
> -done
> -
> -if test -n "$submodules"; then
> - {
> - git ls-files || error "git ls-files failed"
> - for sm in $submodules; do
> - (cd $sm; git ls-files) | sed "s:^:$sm/:"
> - if test "${PIPESTATUS[*]}" != "0 0"; then
> - error "git ls-files in submodule $sm failed"
> - fi
> - done
> - } | grep -x -v $(for sm in $submodules; do echo "-e $sm"; done) > "$list_file"
> -else
> - git ls-files > "$list_file"
> -fi
> +( git ls-files && echo '.git' ) > "$list_file"
>
> if test $? -ne 0; then
> error "failed to generate list file"
> --
> 2.14.3
>
>
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests
2018-04-04 8:19 ` Daniel P. Berrangé
@ 2018-04-04 8:24 ` Fam Zheng
0 siblings, 0 replies; 15+ messages in thread
From: Fam Zheng @ 2018-04-04 8:24 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: qemu-devel, Alex Bennée, pbonzini, Philippe Mathieu-Daudé
On Wed, 04/04 09:19, Daniel P. Berrangé wrote:
> On Wed, Apr 04, 2018 at 10:51:58AM +0800, Fam Zheng wrote:
> > Docker testing on patchew has long suffered from 'make check' hangings. The
> > cleanness of VM testing is the cure. Now let's add a CentOS 7 image to run the
> > tests. It's purely ad-hoc, but hopefully still easy to understand and use for
> > everyone.
>
> If our automated tests are hanging, then developers could hit those hangs
> to, and depending on where the problem is, end users could hit the hangs
> in production deployment.
>
> IOW, hangs in the testing aren't something we should just hide away by
> changing the way we run tests - we need to identify & fix the hangs.
This will not hide anything. It just helps how we clean up everything
(especially the hanging test process in patchew) when timeout, and doesn't
affect how we identify and report timeouts.
Fam
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Qemu-devel] [PATCH 1/3] archive-source.sh: Drop submodule code
2018-04-04 8:23 ` Daniel P. Berrangé
@ 2018-04-04 9:07 ` Fam Zheng
2018-04-04 9:24 ` Daniel P. Berrangé
0 siblings, 1 reply; 15+ messages in thread
From: Fam Zheng @ 2018-04-04 9:07 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: qemu-devel, Alex Bennée, pbonzini, Philippe Mathieu-Daudé
On Wed, 04/04 09:23, Daniel P. Berrangé wrote:
> On Wed, Apr 04, 2018 at 10:51:59AM +0800, Fam Zheng wrote:
> > ./configure and make now take care of submodules, we only need to clone
> > the top tree.
>
> If you don't include the submodules in the tar.gz archive, then the
> test system needs to be able to clone them from qemu.org, which
> requires networking.
>
> The VM tests have networking by default, but the docker tests do
> not have networking - you need to opt-in by setting NETWORK=1
> make variable. So AFAICT, this will break docker tests.
Yes, I'll revise the patch.
Fam
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Qemu-devel] [PATCH 1/3] archive-source.sh: Drop submodule code
2018-04-04 9:07 ` Fam Zheng
@ 2018-04-04 9:24 ` Daniel P. Berrangé
2018-04-04 9:37 ` Fam Zheng
0 siblings, 1 reply; 15+ messages in thread
From: Daniel P. Berrangé @ 2018-04-04 9:24 UTC (permalink / raw)
To: Fam Zheng
Cc: qemu-devel, Alex Bennée, pbonzini, Philippe Mathieu-Daudé
On Wed, Apr 04, 2018 at 05:07:56PM +0800, Fam Zheng wrote:
> On Wed, 04/04 09:23, Daniel P. Berrangé wrote:
> > On Wed, Apr 04, 2018 at 10:51:59AM +0800, Fam Zheng wrote:
> > > ./configure and make now take care of submodules, we only need to clone
> > > the top tree.
> >
> > If you don't include the submodules in the tar.gz archive, then the
> > test system needs to be able to clone them from qemu.org, which
> > requires networking.
> >
> > The VM tests have networking by default, but the docker tests do
> > not have networking - you need to opt-in by setting NETWORK=1
> > make variable. So AFAICT, this will break docker tests.
>
> Yes, I'll revise the patch.
BTW, I can't help feeling that scripts/archive-source.sh should not even
exist. We already have scripts/make-release that has significant overlap
of functionality and is used by 'make dist'. Ideally imho we should be
able to either invoke 'make dist' for testing, or call make-release.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Qemu-devel] [PATCH 1/3] archive-source.sh: Drop submodule code
2018-04-04 9:24 ` Daniel P. Berrangé
@ 2018-04-04 9:37 ` Fam Zheng
0 siblings, 0 replies; 15+ messages in thread
From: Fam Zheng @ 2018-04-04 9:37 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: qemu-devel, Alex Bennée, pbonzini, Philippe Mathieu-Daudé
On Wed, 04/04 10:24, Daniel P. Berrangé wrote:
> On Wed, Apr 04, 2018 at 05:07:56PM +0800, Fam Zheng wrote:
> > On Wed, 04/04 09:23, Daniel P. Berrangé wrote:
> > > On Wed, Apr 04, 2018 at 10:51:59AM +0800, Fam Zheng wrote:
> > > > ./configure and make now take care of submodules, we only need to clone
> > > > the top tree.
> > >
> > > If you don't include the submodules in the tar.gz archive, then the
> > > test system needs to be able to clone them from qemu.org, which
> > > requires networking.
> > >
> > > The VM tests have networking by default, but the docker tests do
> > > not have networking - you need to opt-in by setting NETWORK=1
> > > make variable. So AFAICT, this will break docker tests.
> >
> > Yes, I'll revise the patch.
>
> BTW, I can't help feeling that scripts/archive-source.sh should not even
> exist. We already have scripts/make-release that has significant overlap
> of functionality and is used by 'make dist'. Ideally imho we should be
> able to either invoke 'make dist' for testing, or call make-release.
But it has the advantage that it, as test code, never messes with the process to
release QEMU. :)
Fam
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests
2018-04-04 2:51 [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests Fam Zheng
` (6 preceding siblings ...)
2018-04-04 8:19 ` Daniel P. Berrangé
@ 2018-04-06 16:47 ` Paolo Bonzini
7 siblings, 0 replies; 15+ messages in thread
From: Paolo Bonzini @ 2018-04-06 16:47 UTC (permalink / raw)
To: Fam Zheng, qemu-devel; +Cc: Alex Bennée, Philippe Mathieu-Daudé
On 04/04/2018 04:51, Fam Zheng wrote:
> Docker testing on patchew has long suffered from 'make check' hangings. The
> cleanness of VM testing is the cure. Now let's add a CentOS 7 image to run the
> tests. It's purely ad-hoc, but hopefully still easy to understand and use for
> everyone.
>
> The first patch makes passing source code from host to the container in VM
> working, and is a nice clean up.
>
> The second patch makes caches work, to speed up repetitive runs like on
> patchew.
>
> The last patch adds the new image that does the job. Two out of three docker
> tests running on patchew.org are added to the image. I'll wait for Peter to fix
> the 'docker-test-quick@centos6' hanging (oob test) before adding it too.
I think it's a good idea. The tests are the same as before, only the
environment is more self-contained.
Paolo
> Fam
>
> Fam Zheng (3):
> archive-source.sh: Drop submodule code
> tests: Add an option for snapshot (default: off)
> tests: Add centos VM testing
>
> scripts/archive-source.sh | 47 +++------------------------
> tests/vm/basevm.py | 7 +++-
> tests/vm/centos | 82 +++++++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 92 insertions(+), 44 deletions(-)
> create mode 100755 tests/vm/centos
>
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2018-04-06 16:47 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-04-04 2:51 [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests Fam Zheng
2018-04-04 2:51 ` [Qemu-devel] [PATCH 1/3] archive-source.sh: Drop submodule code Fam Zheng
2018-04-04 8:23 ` Daniel P. Berrangé
2018-04-04 9:07 ` Fam Zheng
2018-04-04 9:24 ` Daniel P. Berrangé
2018-04-04 9:37 ` Fam Zheng
2018-04-04 2:52 ` [Qemu-devel] [PATCH 2/3] tests: Add an option for snapshot (default: off) Fam Zheng
2018-04-04 2:52 ` [Qemu-devel] [PATCH 3/3] tests: Add centos VM testing Fam Zheng
2018-04-04 3:01 ` [Qemu-devel] [PATCH 0/3] Add a CentOS test image to run docker tests no-reply
2018-04-04 3:04 ` no-reply
2018-04-04 5:03 ` Peter Xu
2018-04-04 6:56 ` Peter Xu
2018-04-04 8:19 ` Daniel P. Berrangé
2018-04-04 8:24 ` Fam Zheng
2018-04-06 16:47 ` Paolo Bonzini
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.