* [PATCH v1 00/10] testing/next updates (tweaks and re-greening)
@ 2020-03-02 18:18 Alex Bennée
2020-03-02 18:18 ` [PATCH v1 01/10] tests/vm: use $(PYTHON) consistently Alex Bennée
` (9 more replies)
0 siblings, 10 replies; 23+ messages in thread
From: Alex Bennée @ 2020-03-02 18:18 UTC (permalink / raw)
To: qemu-devel; +Cc: Alex Bennée
Hi,
I was reviewing Roberts vm-build.aarch64 patches when I ran into some
problems with testing. Then I noticed the CI had turned red again so I
went to fix those problems and here we are.
The following patches are mine and need review:
- travis.yml: install python3 numpy and opencv libraries
- tests/acceptance: move @skipUnless decoration to test itself
- configure: detect and report genisoimage
- configure: fix check for libzstd
Alex Bennée (4):
configure: fix check for libzstd
configure: detect and report genisoimage
tests/acceptance: move @skipUnless decoration to test itself
travis.yml: install python3 numpy and opencv libraries
Laurent Vivier (1):
travis: enable tools build on OS X
Robert Foley (5):
tests/vm: use $(PYTHON) consistently
tests/vm: Debug mode shows ssh output.
tests/vm: increased max timeout for vm boot.
tests/vm: give wait_ssh() option to wait for root
tests/vm: Added gen_cloud_init_iso() to basevm.py
configure | 15 ++++++-
.travis.yml | 3 ++
tests/acceptance/machine_mips_malta.py | 4 +-
tests/vm/Makefile.include | 16 +++++--
tests/vm/basevm.py | 61 ++++++++++++++++++++++++--
tests/vm/centos | 33 +-------------
tests/vm/ubuntu.i386 | 37 +---------------
7 files changed, 91 insertions(+), 78 deletions(-)
--
2.20.1
^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH v1 01/10] tests/vm: use $(PYTHON) consistently
2020-03-02 18:18 [PATCH v1 00/10] testing/next updates (tweaks and re-greening) Alex Bennée
@ 2020-03-02 18:18 ` Alex Bennée
2020-03-02 18:29 ` Philippe Mathieu-Daudé
2020-03-02 18:18 ` [PATCH v1 02/10] tests/vm: Debug mode shows ssh output Alex Bennée
` (8 subsequent siblings)
9 siblings, 1 reply; 23+ messages in thread
From: Alex Bennée @ 2020-03-02 18:18 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Puhov, Alex Bennée, Robert Foley,
Philippe Mathieu-Daudé
From: Robert Foley <robert.foley@linaro.org>
Change Makefile.include to use $(PYTHON) so for vm-boot-ssh to be
consistent with other cases like vm-build.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200219163537.22098-2-robert.foley@linaro.org>
---
tests/vm/Makefile.include | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index 9e7c46a4735..778e5067554 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -80,7 +80,7 @@ vm-boot-serial-%: $(IMAGES_DIR)/%.img
vm-boot-ssh-%: $(IMAGES_DIR)/%.img
$(call quiet-command, \
- $(SRC_PATH)/tests/vm/$* \
+ $(PYTHON) $(SRC_PATH)/tests/vm/$* \
$(if $(J),--jobs $(J)) \
--image "$<" \
--interactive \
--
2.20.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v1 02/10] tests/vm: Debug mode shows ssh output.
2020-03-02 18:18 [PATCH v1 00/10] testing/next updates (tweaks and re-greening) Alex Bennée
2020-03-02 18:18 ` [PATCH v1 01/10] tests/vm: use $(PYTHON) consistently Alex Bennée
@ 2020-03-02 18:18 ` Alex Bennée
2020-03-02 18:30 ` Philippe Mathieu-Daudé
2020-03-02 18:19 ` [PATCH v1 03/10] tests/vm: increased max timeout for vm boot Alex Bennée
` (7 subsequent siblings)
9 siblings, 1 reply; 23+ messages in thread
From: Alex Bennée @ 2020-03-02 18:18 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Puhov, Alex Bennée, Robert Foley,
Philippe Mathieu-Daudé
From: Robert Foley <robert.foley@linaro.org>
Add changes to tests/vm/basevm.py so that during debug mode we show ssh output.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200219163537.22098-3-robert.foley@linaro.org>
---
tests/vm/basevm.py | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 4dee6647e6e..c99725b8c0d 100644
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -120,11 +120,16 @@ class BaseVM(object):
return fname
def _ssh_do(self, user, cmd, check):
- ssh_cmd = ["ssh", "-q", "-t",
+ ssh_cmd = ["ssh",
+ "-t",
"-o", "StrictHostKeyChecking=no",
"-o", "UserKnownHostsFile=" + os.devnull,
"-o", "ConnectTimeout=1",
"-p", self.ssh_port, "-i", self._ssh_key_file]
+ # If not in debug mode, set ssh to quiet mode to
+ # avoid printing the results of commands.
+ if not self.debug:
+ ssh_cmd.append("-q")
for var in self.envvars:
ssh_cmd += ['-o', "SendEnv=%s" % var ]
assert not isinstance(cmd, str)
--
2.20.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v1 03/10] tests/vm: increased max timeout for vm boot.
2020-03-02 18:18 [PATCH v1 00/10] testing/next updates (tweaks and re-greening) Alex Bennée
2020-03-02 18:18 ` [PATCH v1 01/10] tests/vm: use $(PYTHON) consistently Alex Bennée
2020-03-02 18:18 ` [PATCH v1 02/10] tests/vm: Debug mode shows ssh output Alex Bennée
@ 2020-03-02 18:19 ` Alex Bennée
2020-03-02 18:32 ` Philippe Mathieu-Daudé
2020-03-02 18:19 ` [PATCH v1 04/10] tests/vm: give wait_ssh() option to wait for root Alex Bennée
` (6 subsequent siblings)
9 siblings, 1 reply; 23+ messages in thread
From: Alex Bennée @ 2020-03-02 18:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Puhov, Alex Bennée, Robert Foley,
Philippe Mathieu-Daudé
From: Robert Foley <robert.foley@linaro.org>
Add change to increase timeout waiting for VM to boot.
Needed for some emulation cases where it can take longer
than 5 minutes to boot.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200219163537.22098-4-robert.foley@linaro.org>
---
tests/vm/basevm.py | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index c99725b8c0d..5ca445e29af 100644
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -57,6 +57,10 @@ class BaseVM(object):
poweroff = "poweroff"
# enable IPv6 networking
ipv6 = True
+ # Scale up some timeouts under TCG.
+ # 4 is arbitrary, but greater than 2,
+ # since we found we need to wait more than twice as long.
+ tcg_ssh_timeout_multiplier = 4
def __init__(self, debug=False, vcpus=None):
self._guest = None
self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-",
@@ -309,6 +313,9 @@ class BaseVM(object):
sys.stderr.write("### %s ...\n" % text)
def wait_ssh(self, seconds=300):
+ # Allow more time for VM to boot under TCG.
+ if not kvm_available(self.arch):
+ seconds *= self.tcg_ssh_timeout_multiplier
starttime = datetime.datetime.now()
endtime = starttime + datetime.timedelta(seconds=seconds)
guest_up = False
--
2.20.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v1 04/10] tests/vm: give wait_ssh() option to wait for root
2020-03-02 18:18 [PATCH v1 00/10] testing/next updates (tweaks and re-greening) Alex Bennée
` (2 preceding siblings ...)
2020-03-02 18:19 ` [PATCH v1 03/10] tests/vm: increased max timeout for vm boot Alex Bennée
@ 2020-03-02 18:19 ` Alex Bennée
2020-03-02 18:19 ` [PATCH v1 05/10] tests/vm: Added gen_cloud_init_iso() to basevm.py Alex Bennée
` (5 subsequent siblings)
9 siblings, 0 replies; 23+ messages in thread
From: Alex Bennée @ 2020-03-02 18:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Puhov, Alex Bennée, Robert Foley,
Philippe Mathieu-Daudé
From: Robert Foley <robert.foley@linaro.org>
Allow wait_ssh to wait for root user to be ready.
This solves the issue where we perform a wait_ssh()
successfully, but the root user is not yet ready
to be logged in.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200219163537.22098-5-robert.foley@linaro.org>
---
tests/vm/basevm.py | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 5ca445e29af..7f268922685 100644
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -312,7 +312,7 @@ class BaseVM(object):
def print_step(self, text):
sys.stderr.write("### %s ...\n" % text)
- def wait_ssh(self, seconds=300):
+ def wait_ssh(self, wait_root=False, seconds=300):
# Allow more time for VM to boot under TCG.
if not kvm_available(self.arch):
seconds *= self.tcg_ssh_timeout_multiplier
@@ -320,7 +320,10 @@ class BaseVM(object):
endtime = starttime + datetime.timedelta(seconds=seconds)
guest_up = False
while datetime.datetime.now() < endtime:
- if self.ssh("exit 0") == 0:
+ if wait_root and self.ssh_root("exit 0") == 0:
+ guest_up = True
+ break
+ elif self.ssh("exit 0") == 0:
guest_up = True
break
seconds = (endtime - datetime.datetime.now()).total_seconds()
--
2.20.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v1 05/10] tests/vm: Added gen_cloud_init_iso() to basevm.py
2020-03-02 18:18 [PATCH v1 00/10] testing/next updates (tweaks and re-greening) Alex Bennée
` (3 preceding siblings ...)
2020-03-02 18:19 ` [PATCH v1 04/10] tests/vm: give wait_ssh() option to wait for root Alex Bennée
@ 2020-03-02 18:19 ` Alex Bennée
2020-03-02 18:32 ` Philippe Mathieu-Daudé
2020-03-02 18:19 ` [PATCH v1 06/10] travis: enable tools build on OS X Alex Bennée
` (4 subsequent siblings)
9 siblings, 1 reply; 23+ messages in thread
From: Alex Bennée @ 2020-03-02 18:19 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Peter Puhov, Alex Bennée, Robert Foley,
Philippe Mathieu-Daudé
From: Robert Foley <robert.foley@linaro.org>
This method was located in both centos and ubuntu.i386.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200219163537.22098-6-robert.foley@linaro.org>
---
tests/vm/basevm.py | 40 ++++++++++++++++++++++++++++++++++++++++
tests/vm/centos | 33 +--------------------------------
tests/vm/ubuntu.i386 | 37 +------------------------------------
3 files changed, 42 insertions(+), 68 deletions(-)
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 7f268922685..8400b0e07f6 100644
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -345,6 +345,46 @@ class BaseVM(object):
def qmp(self, *args, **kwargs):
return self._guest.qmp(*args, **kwargs)
+ def gen_cloud_init_iso(self):
+ cidir = self._tmpdir
+ mdata = open(os.path.join(cidir, "meta-data"), "w")
+ name = self.name.replace(".","-")
+ mdata.writelines(["instance-id: {}-vm-0\n".format(name),
+ "local-hostname: {}-guest\n".format(name)])
+ mdata.close()
+ udata = open(os.path.join(cidir, "user-data"), "w")
+ print("guest user:pw {}:{}".format(self._config['guest_user'],
+ self._config['guest_pass']))
+ udata.writelines(["#cloud-config\n",
+ "chpasswd:\n",
+ " list: |\n",
+ " root:%s\n" % self._config['root_pass'],
+ " %s:%s\n" % (self._config['guest_user'],
+ self._config['guest_pass']),
+ " expire: False\n",
+ "users:\n",
+ " - name: %s\n" % self._config['guest_user'],
+ " sudo: ALL=(ALL) NOPASSWD:ALL\n",
+ " ssh-authorized-keys:\n",
+ " - %s\n" % self._config['ssh_pub_key'],
+ " - name: root\n",
+ " ssh-authorized-keys:\n",
+ " - %s\n" % self._config['ssh_pub_key'],
+ "locale: en_US.UTF-8\n"])
+ proxy = os.environ.get("http_proxy")
+ if not proxy is None:
+ udata.writelines(["apt:\n",
+ " proxy: %s" % proxy])
+ udata.close()
+ subprocess.check_call(["genisoimage", "-output", "cloud-init.iso",
+ "-volid", "cidata", "-joliet", "-rock",
+ "user-data", "meta-data"],
+ cwd=cidir,
+ stdin=self._devnull, stdout=self._stdout,
+ stderr=self._stdout)
+
+ return os.path.join(cidir, "cloud-init.iso")
+
def parse_args(vmcls):
def get_default_jobs():
diff --git a/tests/vm/centos b/tests/vm/centos
index a41ff109eb5..0ad4ecf4190 100755
--- a/tests/vm/centos
+++ b/tests/vm/centos
@@ -31,37 +31,6 @@ class CentosVM(basevm.BaseVM):
make docker-test-mingw@fedora {verbose} J={jobs} NETWORK=1;
"""
- def _gen_cloud_init_iso(self):
- cidir = self._tmpdir
- mdata = open(os.path.join(cidir, "meta-data"), "w")
- mdata.writelines(["instance-id: centos-vm-0\n",
- "local-hostname: centos-guest\n"])
- mdata.close()
- udata = open(os.path.join(cidir, "user-data"), "w")
- udata.writelines(["#cloud-config\n",
- "chpasswd:\n",
- " list: |\n",
- " root:%s\n" % self.ROOT_PASS,
- " %s:%s\n" % (self.GUEST_USER, self.GUEST_PASS),
- " expire: False\n",
- "users:\n",
- " - name: %s\n" % self.GUEST_USER,
- " sudo: ALL=(ALL) NOPASSWD:ALL\n",
- " ssh-authorized-keys:\n",
- " - %s\n" % basevm.SSH_PUB_KEY,
- " - name: root\n",
- " ssh-authorized-keys:\n",
- " - %s\n" % basevm.SSH_PUB_KEY,
- "locale: en_US.UTF-8\n"])
- udata.close()
- subprocess.check_call(["genisoimage", "-output", "cloud-init.iso",
- "-volid", "cidata", "-joliet", "-rock",
- "user-data", "meta-data"],
- cwd=cidir,
- stdin=self._devnull, stdout=self._stdout,
- stderr=self._stdout)
- return os.path.join(cidir, "cloud-init.iso")
-
def build_image(self, img):
cimg = self._download_with_cache("https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1802.qcow2.xz")
img_tmp = img + ".tmp"
@@ -69,7 +38,7 @@ class CentosVM(basevm.BaseVM):
subprocess.check_call(["ln", "-f", cimg, img_tmp + ".xz"])
subprocess.check_call(["xz", "--keep", "-dvf", img_tmp + ".xz"])
self.exec_qemu_img("resize", img_tmp, "50G")
- self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
+ self.boot(img_tmp, extra_args = ["-cdrom", self.gen_cloud_init_iso()])
self.wait_ssh()
self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")
self.ssh_root_check("yum update -y")
diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 48e9cb1ad3d..3266038fbde 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -29,41 +29,6 @@ class UbuntuX86VM(basevm.BaseVM):
make --output-sync {target} -j{jobs} {verbose};
"""
- def _gen_cloud_init_iso(self):
- cidir = self._tmpdir
- mdata = open(os.path.join(cidir, "meta-data"), "w")
- mdata.writelines(["instance-id: ubuntu-vm-0\n",
- "local-hostname: ubuntu-guest\n"])
- mdata.close()
- udata = open(os.path.join(cidir, "user-data"), "w")
- udata.writelines(["#cloud-config\n",
- "chpasswd:\n",
- " list: |\n",
- " root:%s\n" % self.ROOT_PASS,
- " %s:%s\n" % (self.GUEST_USER, self.GUEST_PASS),
- " expire: False\n",
- "users:\n",
- " - name: %s\n" % self.GUEST_USER,
- " sudo: ALL=(ALL) NOPASSWD:ALL\n",
- " ssh-authorized-keys:\n",
- " - %s\n" % basevm.SSH_PUB_KEY,
- " - name: root\n",
- " ssh-authorized-keys:\n",
- " - %s\n" % basevm.SSH_PUB_KEY,
- "locale: en_US.UTF-8\n"])
- proxy = os.environ.get("http_proxy")
- if not proxy is None:
- udata.writelines(["apt:\n",
- " proxy: %s" % proxy])
- udata.close()
- subprocess.check_call(["genisoimage", "-output", "cloud-init.iso",
- "-volid", "cidata", "-joliet", "-rock",
- "user-data", "meta-data"],
- cwd=cidir,
- stdin=self._devnull, stdout=self._stdout,
- stderr=self._stdout)
- return os.path.join(cidir, "cloud-init.iso")
-
def build_image(self, img):
cimg = self._download_with_cache(
"https://cloud-images.ubuntu.com/releases/bionic/release-20191114/ubuntu-18.04-server-cloudimg-i386.img",
@@ -71,7 +36,7 @@ class UbuntuX86VM(basevm.BaseVM):
img_tmp = img + ".tmp"
subprocess.check_call(["cp", "-f", cimg, img_tmp])
self.exec_qemu_img("resize", img_tmp, "50G")
- self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
+ self.boot(img_tmp, extra_args = ["-cdrom", self.gen_cloud_init_iso()])
self.wait_ssh()
self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")
self.ssh_root_check("apt-get update")
--
2.20.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v1 06/10] travis: enable tools build on OS X
2020-03-02 18:18 [PATCH v1 00/10] testing/next updates (tweaks and re-greening) Alex Bennée
` (4 preceding siblings ...)
2020-03-02 18:19 ` [PATCH v1 05/10] tests/vm: Added gen_cloud_init_iso() to basevm.py Alex Bennée
@ 2020-03-02 18:19 ` Alex Bennée
2020-03-02 18:46 ` Philippe Mathieu-Daudé
2020-03-02 18:19 ` [PATCH v1 07/10] configure: fix check for libzstd Alex Bennée
` (3 subsequent siblings)
9 siblings, 1 reply; 23+ messages in thread
From: Alex Bennée @ 2020-03-02 18:19 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé
From: Laurent Vivier <lvivier@redhat.com>
As we can build tools on OS X we should check we don't break build
when we submit new codes.
Signed-off-by: Laurent Vivier <lvivier@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200302154630.45620-3-lvivier@redhat.com>
---
.travis.yml | 1 +
1 file changed, 1 insertion(+)
diff --git a/.travis.yml b/.travis.yml
index 9867272177d..70a24bf2fc8 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -273,6 +273,7 @@ jobs:
- name: "OSX Xcode 10.3"
env:
+ - BASE_CONFIG="--disable-docs --enable-tools"
- CONFIG="--target-list=i386-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,x86_64-softmmu"
os: osx
osx_image: xcode10.3
--
2.20.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v1 07/10] configure: fix check for libzstd
2020-03-02 18:18 [PATCH v1 00/10] testing/next updates (tweaks and re-greening) Alex Bennée
` (5 preceding siblings ...)
2020-03-02 18:19 ` [PATCH v1 06/10] travis: enable tools build on OS X Alex Bennée
@ 2020-03-02 18:19 ` Alex Bennée
2020-03-02 22:02 ` Alex Bennée
2020-03-02 18:19 ` [PATCH v1 08/10] configure: detect and report genisoimage Alex Bennée
` (2 subsequent siblings)
9 siblings, 1 reply; 23+ messages in thread
From: Alex Bennée @ 2020-03-02 18:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Alex Bennée
Fixes: 3a67848134d0
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
configure | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/configure b/configure
index 7b373bc0bb8..caa65f58831 100755
--- a/configure
+++ b/configure
@@ -2464,7 +2464,7 @@ fi
# zstd check
if test "$zstd" != "no" ; then
- if $pkg_config --exist libzstd ; then
+ if $pkg_config --exists libzstd ; then
zstd_cflags="$($pkg_config --cflags libzstd)"
zstd_libs="$($pkg_config --libs libzstd)"
LIBS="$zstd_libs $LIBS"
--
2.20.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v1 08/10] configure: detect and report genisoimage
2020-03-02 18:18 [PATCH v1 00/10] testing/next updates (tweaks and re-greening) Alex Bennée
` (6 preceding siblings ...)
2020-03-02 18:19 ` [PATCH v1 07/10] configure: fix check for libzstd Alex Bennée
@ 2020-03-02 18:19 ` Alex Bennée
2020-03-02 18:25 ` Philippe Mathieu-Daudé
2020-03-02 18:19 ` [PATCH v1 09/10] tests/acceptance: move @skipUnless decoration to test itself Alex Bennée
2020-03-02 18:19 ` [PATCH v1 10/10] travis.yml: install python3 numpy and opencv libraries Alex Bennée
9 siblings, 1 reply; 23+ messages in thread
From: Alex Bennée @ 2020-03-02 18:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Fam Zheng, Philippe Mathieu-Daudé, Alex Bennée
This is used for some of the vm-build tests so lets detect it and
behave sanely when it is not installed.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
configure | 13 +++++++++++++
tests/vm/Makefile.include | 14 +++++++++++---
2 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/configure b/configure
index caa65f58831..376ccb25623 100755
--- a/configure
+++ b/configure
@@ -936,6 +936,17 @@ do
fi
done
+# Check for ancillary tools used in testing
+genisoimage=
+for binary in genisoimage
+do
+ if has $binary
+ then
+ genisoimage=$(command -v "$binary")
+ break
+ fi
+done
+
: ${smbd=${SMBD-/usr/sbin/smbd}}
# Default objcc to clang if available, otherwise use CC
@@ -6567,6 +6578,7 @@ echo "python $python ($python_version)"
if test "$docs" != "no"; then
echo "sphinx-build $sphinx_build"
fi
+echo "genisoimage $genisoimage"
echo "slirp support $slirp $(echo_version $slirp $slirp_version)"
if test "$slirp" != "no" ; then
echo "smbd $smbd"
@@ -7616,6 +7628,7 @@ echo "INSTALL_PROG=$install -c -m 0755" >> $config_host_mak
echo "INSTALL_LIB=$install -c -m 0644" >> $config_host_mak
echo "PYTHON=$python" >> $config_host_mak
echo "SPHINX_BUILD=$sphinx_build" >> $config_host_mak
+echo "GENISOIMAGE=$genisoimage" >> $config_host_mak
echo "CC=$cc" >> $config_host_mak
if $iasl -h > /dev/null 2>&1; then
echo "IASL=$iasl" >> $config_host_mak
diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index 778e5067554..1bf9693d195 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -2,7 +2,11 @@
.PHONY: vm-build-all vm-clean-all
-IMAGES := ubuntu.i386 freebsd netbsd openbsd centos fedora
+IMAGES := freebsd netbsd openbsd centos fedora
+ifneq ($(GENISOIMAGE),)
+IMAGES += ubuntu.i386 centos
+endif
+
IMAGES_DIR := $(HOME)/.cache/qemu-vm/images
IMAGE_FILES := $(patsubst %, $(IMAGES_DIR)/%.img, $(IMAGES))
@@ -12,12 +16,16 @@ IMAGE_FILES := $(patsubst %, $(IMAGES_DIR)/%.img, $(IMAGES))
vm-help vm-test:
@echo "vm-help: Test QEMU in preconfigured virtual machines"
@echo
- @echo " vm-build-ubuntu.i386 - Build QEMU in ubuntu i386 VM"
@echo " vm-build-freebsd - Build QEMU in FreeBSD VM"
@echo " vm-build-netbsd - Build QEMU in NetBSD VM"
@echo " vm-build-openbsd - Build QEMU in OpenBSD VM"
- @echo " vm-build-centos - Build QEMU in CentOS VM, with Docker"
@echo " vm-build-fedora - Build QEMU in Fedora VM"
+ifneq ($(GENISOIMAGE),)
+ @echo " vm-build-centos - Build QEMU in CentOS VM, with Docker"
+ @echo " vm-build-ubuntu.i386 - Build QEMU in ubuntu i386 VM"
+else
+ @echo " (install genisoimage to build centos/ubuntu images)"
+endif
@echo ""
@echo " vm-build-all - Build QEMU in all VMs"
@echo " vm-clean-all - Clean up VM images"
--
2.20.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v1 09/10] tests/acceptance: move @skipUnless decoration to test itself
2020-03-02 18:18 [PATCH v1 00/10] testing/next updates (tweaks and re-greening) Alex Bennée
` (7 preceding siblings ...)
2020-03-02 18:19 ` [PATCH v1 08/10] configure: detect and report genisoimage Alex Bennée
@ 2020-03-02 18:19 ` Alex Bennée
2020-03-02 18:19 ` [PATCH v1 10/10] travis.yml: install python3 numpy and opencv libraries Alex Bennée
9 siblings, 0 replies; 23+ messages in thread
From: Alex Bennée @ 2020-03-02 18:19 UTC (permalink / raw)
To: qemu-devel
Cc: Philippe Mathieu-Daudé,
Alex Bennée, Aurelien Jarno, Aleksandar Markovic
It appears ignore the decoration if just applied to the class.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/acceptance/machine_mips_malta.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tests/acceptance/machine_mips_malta.py b/tests/acceptance/machine_mips_malta.py
index 92b4f28a112..b8fac2a44d5 100644
--- a/tests/acceptance/machine_mips_malta.py
+++ b/tests/acceptance/machine_mips_malta.py
@@ -30,14 +30,14 @@ except ImportError:
CV2_AVAILABLE = False
-@skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
-@skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
class MaltaMachineFramebuffer(Test):
timeout = 30
KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
+ @skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
+ @skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
def do_test_i6400_framebuffer_logo(self, cpu_cores_count):
"""
Boot Linux kernel and check Tux logo is displayed on the framebuffer.
--
2.20.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH v1 10/10] travis.yml: install python3 numpy and opencv libraries
2020-03-02 18:18 [PATCH v1 00/10] testing/next updates (tweaks and re-greening) Alex Bennée
` (8 preceding siblings ...)
2020-03-02 18:19 ` [PATCH v1 09/10] tests/acceptance: move @skipUnless decoration to test itself Alex Bennée
@ 2020-03-02 18:19 ` Alex Bennée
9 siblings, 0 replies; 23+ messages in thread
From: Alex Bennée @ 2020-03-02 18:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Fam Zheng, Philippe Mathieu-Daudé, Alex Bennée
These are used for the acceptance framebuffer tests to count Tux.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
.travis.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.travis.yml b/.travis.yml
index 70a24bf2fc8..9db6c1557f1 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -323,6 +323,8 @@ jobs:
packages:
- python3-pil
- python3-pip
+ - python3-numpy
+ - python3-opencv
- python3.5-venv
- rpm2cpio
- tesseract-ocr
--
2.20.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [PATCH v1 08/10] configure: detect and report genisoimage
2020-03-02 18:19 ` [PATCH v1 08/10] configure: detect and report genisoimage Alex Bennée
@ 2020-03-02 18:25 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 23+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-03-02 18:25 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng, Wainer dos Santos Moschetta
Cc'ing Wainer
On 3/2/20 7:19 PM, Alex Bennée wrote:
> This is used for some of the vm-build tests so lets detect it and
> behave sanely when it is not installed.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> configure | 13 +++++++++++++
> tests/vm/Makefile.include | 14 +++++++++++---
> 2 files changed, 24 insertions(+), 3 deletions(-)
>
> diff --git a/configure b/configure
> index caa65f58831..376ccb25623 100755
> --- a/configure
> +++ b/configure
> @@ -936,6 +936,17 @@ do
> fi
> done
>
> +# Check for ancillary tools used in testing
> +genisoimage=
> +for binary in genisoimage
> +do
> + if has $binary
> + then
> + genisoimage=$(command -v "$binary")
> + break
> + fi
> +done
> +
> : ${smbd=${SMBD-/usr/sbin/smbd}}
>
> # Default objcc to clang if available, otherwise use CC
> @@ -6567,6 +6578,7 @@ echo "python $python ($python_version)"
> if test "$docs" != "no"; then
> echo "sphinx-build $sphinx_build"
> fi
> +echo "genisoimage $genisoimage"
> echo "slirp support $slirp $(echo_version $slirp $slirp_version)"
> if test "$slirp" != "no" ; then
> echo "smbd $smbd"
> @@ -7616,6 +7628,7 @@ echo "INSTALL_PROG=$install -c -m 0755" >> $config_host_mak
> echo "INSTALL_LIB=$install -c -m 0644" >> $config_host_mak
> echo "PYTHON=$python" >> $config_host_mak
> echo "SPHINX_BUILD=$sphinx_build" >> $config_host_mak
> +echo "GENISOIMAGE=$genisoimage" >> $config_host_mak
> echo "CC=$cc" >> $config_host_mak
> if $iasl -h > /dev/null 2>&1; then
> echo "IASL=$iasl" >> $config_host_mak
> diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
> index 778e5067554..1bf9693d195 100644
> --- a/tests/vm/Makefile.include
> +++ b/tests/vm/Makefile.include
> @@ -2,7 +2,11 @@
>
> .PHONY: vm-build-all vm-clean-all
>
> -IMAGES := ubuntu.i386 freebsd netbsd openbsd centos fedora
> +IMAGES := freebsd netbsd openbsd centos fedora
> +ifneq ($(GENISOIMAGE),)
> +IMAGES += ubuntu.i386 centos
> +endif
> +
> IMAGES_DIR := $(HOME)/.cache/qemu-vm/images
> IMAGE_FILES := $(patsubst %, $(IMAGES_DIR)/%.img, $(IMAGES))
>
> @@ -12,12 +16,16 @@ IMAGE_FILES := $(patsubst %, $(IMAGES_DIR)/%.img, $(IMAGES))
> vm-help vm-test:
> @echo "vm-help: Test QEMU in preconfigured virtual machines"
> @echo
> - @echo " vm-build-ubuntu.i386 - Build QEMU in ubuntu i386 VM"
> @echo " vm-build-freebsd - Build QEMU in FreeBSD VM"
> @echo " vm-build-netbsd - Build QEMU in NetBSD VM"
> @echo " vm-build-openbsd - Build QEMU in OpenBSD VM"
> - @echo " vm-build-centos - Build QEMU in CentOS VM, with Docker"
> @echo " vm-build-fedora - Build QEMU in Fedora VM"
> +ifneq ($(GENISOIMAGE),)
> + @echo " vm-build-centos - Build QEMU in CentOS VM, with Docker"
> + @echo " vm-build-ubuntu.i386 - Build QEMU in ubuntu i386 VM"
> +else
> + @echo " (install genisoimage to build centos/ubuntu images)"
> +endif
> @echo ""
> @echo " vm-build-all - Build QEMU in all VMs"
> @echo " vm-clean-all - Clean up VM images"
>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v1 01/10] tests/vm: use $(PYTHON) consistently
2020-03-02 18:18 ` [PATCH v1 01/10] tests/vm: use $(PYTHON) consistently Alex Bennée
@ 2020-03-02 18:29 ` Philippe Mathieu-Daudé
2020-03-02 18:41 ` Alex Bennée
0 siblings, 1 reply; 23+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-03-02 18:29 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng, Peter Puhov, Robert Foley
On 3/2/20 7:18 PM, Alex Bennée wrote:
> From: Robert Foley <robert.foley@linaro.org>
>
> Change Makefile.include to use $(PYTHON) so for vm-boot-ssh to be
> consistent with other cases like vm-build.
So what you are trying to fix here is when you have two python3
installed (one from the distribution and one manually built), and you
want to use the manually built?
>
> Signed-off-by: Robert Foley <robert.foley@linaro.org>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20200219163537.22098-2-robert.foley@linaro.org>
> ---
> tests/vm/Makefile.include | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
> index 9e7c46a4735..778e5067554 100644
> --- a/tests/vm/Makefile.include
> +++ b/tests/vm/Makefile.include
> @@ -80,7 +80,7 @@ vm-boot-serial-%: $(IMAGES_DIR)/%.img
>
> vm-boot-ssh-%: $(IMAGES_DIR)/%.img
> $(call quiet-command, \
> - $(SRC_PATH)/tests/vm/$* \
> + $(PYTHON) $(SRC_PATH)/tests/vm/$* \
> $(if $(J),--jobs $(J)) \
> --image "$<" \
> --interactive \
>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v1 02/10] tests/vm: Debug mode shows ssh output.
2020-03-02 18:18 ` [PATCH v1 02/10] tests/vm: Debug mode shows ssh output Alex Bennée
@ 2020-03-02 18:30 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 23+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-03-02 18:30 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng, Peter Puhov, Robert Foley
On 3/2/20 7:18 PM, Alex Bennée wrote:
> From: Robert Foley <robert.foley@linaro.org>
>
> Add changes to tests/vm/basevm.py so that during debug mode we show ssh output.
>
> Signed-off-by: Robert Foley <robert.foley@linaro.org>
> Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20200219163537.22098-3-robert.foley@linaro.org>
> ---
> tests/vm/basevm.py | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
> index 4dee6647e6e..c99725b8c0d 100644
> --- a/tests/vm/basevm.py
> +++ b/tests/vm/basevm.py
> @@ -120,11 +120,16 @@ class BaseVM(object):
> return fname
>
> def _ssh_do(self, user, cmd, check):
> - ssh_cmd = ["ssh", "-q", "-t",
> + ssh_cmd = ["ssh",
> + "-t",
> "-o", "StrictHostKeyChecking=no",
> "-o", "UserKnownHostsFile=" + os.devnull,
> "-o", "ConnectTimeout=1",
> "-p", self.ssh_port, "-i", self._ssh_key_file]
> + # If not in debug mode, set ssh to quiet mode to
> + # avoid printing the results of commands.
> + if not self.debug:
> + ssh_cmd.append("-q")
> for var in self.envvars:
> ssh_cmd += ['-o', "SendEnv=%s" % var ]
> assert not isinstance(cmd, str)
>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v1 03/10] tests/vm: increased max timeout for vm boot.
2020-03-02 18:19 ` [PATCH v1 03/10] tests/vm: increased max timeout for vm boot Alex Bennée
@ 2020-03-02 18:32 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 23+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-03-02 18:32 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng, Peter Puhov, Robert Foley
On 3/2/20 7:19 PM, Alex Bennée wrote:
> From: Robert Foley <robert.foley@linaro.org>
>
> Add change to increase timeout waiting for VM to boot.
> Needed for some emulation cases where it can take longer
> than 5 minutes to boot.
>
> Signed-off-by: Robert Foley <robert.foley@linaro.org>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20200219163537.22098-4-robert.foley@linaro.org>
> ---
> tests/vm/basevm.py | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
> index c99725b8c0d..5ca445e29af 100644
> --- a/tests/vm/basevm.py
> +++ b/tests/vm/basevm.py
> @@ -57,6 +57,10 @@ class BaseVM(object):
> poweroff = "poweroff"
> # enable IPv6 networking
> ipv6 = True
> + # Scale up some timeouts under TCG.
> + # 4 is arbitrary, but greater than 2,
> + # since we found we need to wait more than twice as long.
> + tcg_ssh_timeout_multiplier = 4
> def __init__(self, debug=False, vcpus=None):
> self._guest = None
> self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-",
> @@ -309,6 +313,9 @@ class BaseVM(object):
> sys.stderr.write("### %s ...\n" % text)
>
> def wait_ssh(self, seconds=300):
> + # Allow more time for VM to boot under TCG.
> + if not kvm_available(self.arch):
> + seconds *= self.tcg_ssh_timeout_multiplier
> starttime = datetime.datetime.now()
> endtime = starttime + datetime.timedelta(seconds=seconds)
> guest_up = False
>
A clean way might be to use a QMP socket and poll for networking events,
then start counting when the network is up.
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v1 05/10] tests/vm: Added gen_cloud_init_iso() to basevm.py
2020-03-02 18:19 ` [PATCH v1 05/10] tests/vm: Added gen_cloud_init_iso() to basevm.py Alex Bennée
@ 2020-03-02 18:32 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 23+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-03-02 18:32 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng, Peter Puhov, Robert Foley
On 3/2/20 7:19 PM, Alex Bennée wrote:
> From: Robert Foley <robert.foley@linaro.org>
>
> This method was located in both centos and ubuntu.i386.
>
> Signed-off-by: Robert Foley <robert.foley@linaro.org>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20200219163537.22098-6-robert.foley@linaro.org>
> ---
> tests/vm/basevm.py | 40 ++++++++++++++++++++++++++++++++++++++++
> tests/vm/centos | 33 +--------------------------------
> tests/vm/ubuntu.i386 | 37 +------------------------------------
> 3 files changed, 42 insertions(+), 68 deletions(-)
>
> diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
> index 7f268922685..8400b0e07f6 100644
> --- a/tests/vm/basevm.py
> +++ b/tests/vm/basevm.py
> @@ -345,6 +345,46 @@ class BaseVM(object):
> def qmp(self, *args, **kwargs):
> return self._guest.qmp(*args, **kwargs)
>
> + def gen_cloud_init_iso(self):
> + cidir = self._tmpdir
> + mdata = open(os.path.join(cidir, "meta-data"), "w")
> + name = self.name.replace(".","-")
> + mdata.writelines(["instance-id: {}-vm-0\n".format(name),
> + "local-hostname: {}-guest\n".format(name)])
> + mdata.close()
> + udata = open(os.path.join(cidir, "user-data"), "w")
> + print("guest user:pw {}:{}".format(self._config['guest_user'],
> + self._config['guest_pass']))
> + udata.writelines(["#cloud-config\n",
> + "chpasswd:\n",
> + " list: |\n",
> + " root:%s\n" % self._config['root_pass'],
> + " %s:%s\n" % (self._config['guest_user'],
> + self._config['guest_pass']),
> + " expire: False\n",
> + "users:\n",
> + " - name: %s\n" % self._config['guest_user'],
> + " sudo: ALL=(ALL) NOPASSWD:ALL\n",
> + " ssh-authorized-keys:\n",
> + " - %s\n" % self._config['ssh_pub_key'],
> + " - name: root\n",
> + " ssh-authorized-keys:\n",
> + " - %s\n" % self._config['ssh_pub_key'],
> + "locale: en_US.UTF-8\n"])
> + proxy = os.environ.get("http_proxy")
> + if not proxy is None:
> + udata.writelines(["apt:\n",
> + " proxy: %s" % proxy])
> + udata.close()
> + subprocess.check_call(["genisoimage", "-output", "cloud-init.iso",
> + "-volid", "cidata", "-joliet", "-rock",
> + "user-data", "meta-data"],
> + cwd=cidir,
> + stdin=self._devnull, stdout=self._stdout,
> + stderr=self._stdout)
> +
> + return os.path.join(cidir, "cloud-init.iso")
> +
> def parse_args(vmcls):
>
> def get_default_jobs():
> diff --git a/tests/vm/centos b/tests/vm/centos
> index a41ff109eb5..0ad4ecf4190 100755
> --- a/tests/vm/centos
> +++ b/tests/vm/centos
> @@ -31,37 +31,6 @@ class CentosVM(basevm.BaseVM):
> make docker-test-mingw@fedora {verbose} J={jobs} NETWORK=1;
> """
>
> - def _gen_cloud_init_iso(self):
> - cidir = self._tmpdir
> - mdata = open(os.path.join(cidir, "meta-data"), "w")
> - mdata.writelines(["instance-id: centos-vm-0\n",
> - "local-hostname: centos-guest\n"])
> - mdata.close()
> - udata = open(os.path.join(cidir, "user-data"), "w")
> - udata.writelines(["#cloud-config\n",
> - "chpasswd:\n",
> - " list: |\n",
> - " root:%s\n" % self.ROOT_PASS,
> - " %s:%s\n" % (self.GUEST_USER, self.GUEST_PASS),
> - " expire: False\n",
> - "users:\n",
> - " - name: %s\n" % self.GUEST_USER,
> - " sudo: ALL=(ALL) NOPASSWD:ALL\n",
> - " ssh-authorized-keys:\n",
> - " - %s\n" % basevm.SSH_PUB_KEY,
> - " - name: root\n",
> - " ssh-authorized-keys:\n",
> - " - %s\n" % basevm.SSH_PUB_KEY,
> - "locale: en_US.UTF-8\n"])
> - udata.close()
> - subprocess.check_call(["genisoimage", "-output", "cloud-init.iso",
> - "-volid", "cidata", "-joliet", "-rock",
> - "user-data", "meta-data"],
> - cwd=cidir,
> - stdin=self._devnull, stdout=self._stdout,
> - stderr=self._stdout)
> - return os.path.join(cidir, "cloud-init.iso")
> -
> def build_image(self, img):
> cimg = self._download_with_cache("https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1802.qcow2.xz")
> img_tmp = img + ".tmp"
> @@ -69,7 +38,7 @@ class CentosVM(basevm.BaseVM):
> subprocess.check_call(["ln", "-f", cimg, img_tmp + ".xz"])
> subprocess.check_call(["xz", "--keep", "-dvf", img_tmp + ".xz"])
> self.exec_qemu_img("resize", img_tmp, "50G")
> - self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
> + self.boot(img_tmp, extra_args = ["-cdrom", self.gen_cloud_init_iso()])
> self.wait_ssh()
> self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")
> self.ssh_root_check("yum update -y")
> diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
> index 48e9cb1ad3d..3266038fbde 100755
> --- a/tests/vm/ubuntu.i386
> +++ b/tests/vm/ubuntu.i386
> @@ -29,41 +29,6 @@ class UbuntuX86VM(basevm.BaseVM):
> make --output-sync {target} -j{jobs} {verbose};
> """
>
> - def _gen_cloud_init_iso(self):
> - cidir = self._tmpdir
> - mdata = open(os.path.join(cidir, "meta-data"), "w")
> - mdata.writelines(["instance-id: ubuntu-vm-0\n",
> - "local-hostname: ubuntu-guest\n"])
> - mdata.close()
> - udata = open(os.path.join(cidir, "user-data"), "w")
> - udata.writelines(["#cloud-config\n",
> - "chpasswd:\n",
> - " list: |\n",
> - " root:%s\n" % self.ROOT_PASS,
> - " %s:%s\n" % (self.GUEST_USER, self.GUEST_PASS),
> - " expire: False\n",
> - "users:\n",
> - " - name: %s\n" % self.GUEST_USER,
> - " sudo: ALL=(ALL) NOPASSWD:ALL\n",
> - " ssh-authorized-keys:\n",
> - " - %s\n" % basevm.SSH_PUB_KEY,
> - " - name: root\n",
> - " ssh-authorized-keys:\n",
> - " - %s\n" % basevm.SSH_PUB_KEY,
> - "locale: en_US.UTF-8\n"])
> - proxy = os.environ.get("http_proxy")
> - if not proxy is None:
> - udata.writelines(["apt:\n",
> - " proxy: %s" % proxy])
> - udata.close()
> - subprocess.check_call(["genisoimage", "-output", "cloud-init.iso",
> - "-volid", "cidata", "-joliet", "-rock",
> - "user-data", "meta-data"],
> - cwd=cidir,
> - stdin=self._devnull, stdout=self._stdout,
> - stderr=self._stdout)
> - return os.path.join(cidir, "cloud-init.iso")
> -
> def build_image(self, img):
> cimg = self._download_with_cache(
> "https://cloud-images.ubuntu.com/releases/bionic/release-20191114/ubuntu-18.04-server-cloudimg-i386.img",
> @@ -71,7 +36,7 @@ class UbuntuX86VM(basevm.BaseVM):
> img_tmp = img + ".tmp"
> subprocess.check_call(["cp", "-f", cimg, img_tmp])
> self.exec_qemu_img("resize", img_tmp, "50G")
> - self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
> + self.boot(img_tmp, extra_args = ["-cdrom", self.gen_cloud_init_iso()])
> self.wait_ssh()
> self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")
> self.ssh_root_check("apt-get update")
>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v1 01/10] tests/vm: use $(PYTHON) consistently
2020-03-02 18:29 ` Philippe Mathieu-Daudé
@ 2020-03-02 18:41 ` Alex Bennée
2020-03-02 18:50 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 23+ messages in thread
From: Alex Bennée @ 2020-03-02 18:41 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Fam Zheng, Peter Puhov, qemu-devel, Robert Foley
Philippe Mathieu-Daudé <philmd@redhat.com> writes:
> On 3/2/20 7:18 PM, Alex Bennée wrote:
>> From: Robert Foley <robert.foley@linaro.org>
>> Change Makefile.include to use $(PYTHON) so for vm-boot-ssh to be
>> consistent with other cases like vm-build.
>
> So what you are trying to fix here is when you have two python3
> installed (one from the distribution and one manually built), and you
> want to use the manually built?
No - just consistency with the rest of the build system.
>
>> Signed-off-by: Robert Foley <robert.foley@linaro.org>
>> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>> Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> Message-Id: <20200219163537.22098-2-robert.foley@linaro.org>
>> ---
>> tests/vm/Makefile.include | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>> diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
>> index 9e7c46a4735..778e5067554 100644
>> --- a/tests/vm/Makefile.include
>> +++ b/tests/vm/Makefile.include
>> @@ -80,7 +80,7 @@ vm-boot-serial-%: $(IMAGES_DIR)/%.img
>> vm-boot-ssh-%: $(IMAGES_DIR)/%.img
>> $(call quiet-command, \
>> - $(SRC_PATH)/tests/vm/$* \
>> + $(PYTHON) $(SRC_PATH)/tests/vm/$* \
>> $(if $(J),--jobs $(J)) \
>> --image "$<" \
>> --interactive \
>>
--
Alex Bennée
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v1 06/10] travis: enable tools build on OS X
2020-03-02 18:19 ` [PATCH v1 06/10] travis: enable tools build on OS X Alex Bennée
@ 2020-03-02 18:46 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 23+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-03-02 18:46 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Laurent Vivier, Fam Zheng
On 3/2/20 7:19 PM, Alex Bennée wrote:
> From: Laurent Vivier <lvivier@redhat.com>
>
> As we can build tools on OS X we should check we don't break build
> when we submit new codes.
>
> Signed-off-by: Laurent Vivier <lvivier@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20200302154630.45620-3-lvivier@redhat.com>
> ---
> .travis.yml | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index 9867272177d..70a24bf2fc8 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -273,6 +273,7 @@ jobs:
>
> - name: "OSX Xcode 10.3"
> env:
> + - BASE_CONFIG="--disable-docs --enable-tools"
> - CONFIG="--target-list=i386-softmmu,ppc-softmmu,ppc64-softmmu,m68k-softmmu,x86_64-softmmu"
> os: osx
> osx_image: xcode10.3
>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v1 01/10] tests/vm: use $(PYTHON) consistently
2020-03-02 18:41 ` Alex Bennée
@ 2020-03-02 18:50 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 23+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-03-02 18:50 UTC (permalink / raw)
To: Alex Bennée; +Cc: Fam Zheng, Peter Puhov, qemu-devel, Robert Foley
On 3/2/20 7:41 PM, Alex Bennée wrote:
>
> Philippe Mathieu-Daudé <philmd@redhat.com> writes:
>
>> On 3/2/20 7:18 PM, Alex Bennée wrote:
>>> From: Robert Foley <robert.foley@linaro.org>
>>> Change Makefile.include to use $(PYTHON) so for vm-boot-ssh to be
>>> consistent with other cases like vm-build.
>>
>> So what you are trying to fix here is when you have two python3
>> installed (one from the distribution and one manually built), and you
>> want to use the manually built?
>
> No - just consistency with the rest of the build system.
This patch was first posted on 2/5/20 but it shouldn't be necessary since:
commit c88ee46cdb9888fbe9ea45c174743d6f7544b737
Date: Thu Jan 30 17:32:24 2020 +0100
tests: Explicit usage of Python 3
Use the program search path to find the Python 3 interpreter.
Anyway consistency is good:
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>
>>
>>> Signed-off-by: Robert Foley <robert.foley@linaro.org>
>>> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>>> Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>> Message-Id: <20200219163537.22098-2-robert.foley@linaro.org>
>>> ---
>>> tests/vm/Makefile.include | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>> diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
>>> index 9e7c46a4735..778e5067554 100644
>>> --- a/tests/vm/Makefile.include
>>> +++ b/tests/vm/Makefile.include
>>> @@ -80,7 +80,7 @@ vm-boot-serial-%: $(IMAGES_DIR)/%.img
>>> vm-boot-ssh-%: $(IMAGES_DIR)/%.img
>>> $(call quiet-command, \
>>> - $(SRC_PATH)/tests/vm/$* \
>>> + $(PYTHON) $(SRC_PATH)/tests/vm/$* \
>>> $(if $(J),--jobs $(J)) \
>>> --image "$<" \
>>> --interactive \
>>>
>
>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v1 07/10] configure: fix check for libzstd
2020-03-02 18:19 ` [PATCH v1 07/10] configure: fix check for libzstd Alex Bennée
@ 2020-03-02 22:02 ` Alex Bennée
2020-03-05 9:43 ` Juan Quintela
0 siblings, 1 reply; 23+ messages in thread
From: Alex Bennée @ 2020-03-02 22:02 UTC (permalink / raw)
To: qemu-devel; +Cc: Alex Bennée
Alex Bennée <alex.bennee@linaro.org> writes:
> Fixes: 3a67848134d0
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> configure | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/configure b/configure
> index 7b373bc0bb8..caa65f58831 100755
> --- a/configure
> +++ b/configure
> @@ -2464,7 +2464,7 @@ fi
> # zstd check
>
> if test "$zstd" != "no" ; then
> - if $pkg_config --exist libzstd ; then
> + if $pkg_config --exists libzstd ; then
Dropping this patch as it breaks the build even more!
CC migration/block.o
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:24:5: error: unknown type name ‘ZSTD_CStream’
ZSTD_CStream *zcs;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:26:5: error: unknown type name ‘ZSTD_DStream’
ZSTD_DStream *zds;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:28:5: error: unknown type name ‘ZSTD_inBuffer’
ZSTD_inBuffer in;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:29:5: error: unknown type name ‘ZSTD_outBuffer’
ZSTD_outBuffer out;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c: In function ‘zstd_send_setup’:
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:55:14: error: implicit declaration of function ‘ZSTD_createCStream’ [-Werror=implicit-function-declaration]
z->zcs = ZSTD_createCStream();
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:55:5: error: nested extern declaration of ‘ZSTD_createCStream’ [-Werror=nested-externs]
z->zcs = ZSTD_createCStream();
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:55:12: error: assignment makes pointer from integer without a cast [-Werror=int-conversion]
z->zcs = ZSTD_createCStream();
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:62:11: error: implicit declaration of function ‘ZSTD_initCStream’ [-Werror=implicit-function-declaration]
res = ZSTD_initCStream(z->zcs, migrate_multifd_zstd_level());
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:62:5: error: nested extern declaration of ‘ZSTD_initCStream’ [-Werror=nested-externs]
res = ZSTD_initCStream(z->zcs, migrate_multifd_zstd_level());
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:64:9: error: implicit declaration of function ‘ZSTD_freeCStream’ [-Werror=implicit-function-declaration]
ZSTD_freeCStream(z->zcs);
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:64:9: error: nested extern declaration of ‘ZSTD_freeCStream’ [-Werror=nested-externs]
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c: In function ‘zstd_send_prepare’:
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:120:11: error: request for member ‘dst’ in something not a structure or union
z->out.dst = z->zbuff;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:121:11: error: request for member ‘size’ in something not a structure or union
z->out.size = z->zbuff_len;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:122:11: error: request for member ‘pos’ in something not a structure or union
z->out.pos = 0;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:125:9: error: unknown type name ‘ZSTD_EndDirective’
ZSTD_EndDirective flush = ZSTD_e_continue;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:125:35: error: ‘ZSTD_e_continue’ undeclared (first use in this function)
ZSTD_EndDirective flush = ZSTD_e_continue;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:125:35: note: each undeclared identifier is reported only once for each function it appears in
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:128:21: error: ‘ZSTD_e_flush’ undeclared (first use in this function)
flush = ZSTD_e_flush;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:130:14: error: request for member ‘src’ in something not a structure or union
z->in.src = iov[i].iov_base;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:131:14: error: request for member ‘size’ in something not a structure or union
z->in.size = iov[i].iov_len;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:132:14: error: request for member ‘pos’ in something not a structure or union
z->in.pos = 0;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:143:19: error: implicit declaration of function ‘ZSTD_compressStream2’ [-Werror=implicit-function-declaration]
ret = ZSTD_compressStream2(z->zcs, &z->out, &z->in, flush);
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:143:13: error: nested extern declaration of ‘ZSTD_compressStream2’ [-Werror=nested-externs]
ret = ZSTD_compressStream2(z->zcs, &z->out, &z->in, flush);
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:144:35: error: request for member ‘size’ in something not a structure or union
} while (ret > 0 && (z->in.size - z->in.pos > 0)
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:144:48: error: request for member ‘pos’ in something not a structure or union
} while (ret > 0 && (z->in.size - z->in.pos > 0)
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:145:36: error: request for member ‘size’ in something not a structure or union
&& (z->out.size - z->out.pos > 0));
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:145:50: error: request for member ‘pos’ in something not a structure or union
&& (z->out.size - z->out.pos > 0));
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:146:30: error: request for member ‘size’ in something not a structure or union
if (ret > 0 && (z->in.size - z->in.pos > 0)) {
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:146:43: error: request for member ‘pos’ in something not a structure or union
if (ret > 0 && (z->in.size - z->in.pos > 0)) {
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:157:33: error: request for member ‘pos’ in something not a structure or union
p->next_packet_size = z->out.pos;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c: In function ‘zstd_recv_setup’:
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:199:14: error: implicit declaration of function ‘ZSTD_createDStream’ [-Werror=implicit-function-declaration]
z->zds = ZSTD_createDStream();
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:199:5: error: nested extern declaration of ‘ZSTD_createDStream’ [-Werror=nested-externs]
z->zds = ZSTD_createDStream();
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:199:12: error: assignment makes pointer from integer without a cast [-Werror=int-conversion]
z->zds = ZSTD_createDStream();
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:206:11: error: implicit declaration of function ‘ZSTD_initDStream’ [-Werror=implicit-function-declaration]
ret = ZSTD_initDStream(z->zds);
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:206:5: error: nested extern declaration of ‘ZSTD_initDStream’ [-Werror=nested-externs]
ret = ZSTD_initDStream(z->zds);
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:208:9: error: implicit declaration of function ‘ZSTD_freeDStream’ [-Werror=implicit-function-declaration]
ZSTD_freeDStream(z->zds);
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:208:9: error: nested extern declaration of ‘ZSTD_freeDStream’ [-Werror=nested-externs]
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c: In function ‘zstd_recv_pages’:
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:281:10: error: request for member ‘src’ in something not a structure or union
z->in.src = z->zbuff;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:282:10: error: request for member ‘size’ in something not a structure or union
z->in.size = in_size;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:283:10: error: request for member ‘pos’ in something not a structure or union
z->in.pos = 0;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:288:15: error: request for member ‘dst’ in something not a structure or union
z->out.dst = iov->iov_base;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:289:15: error: request for member ‘size’ in something not a structure or union
z->out.size = iov->iov_len;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:290:15: error: request for member ‘pos’ in something not a structure or union
z->out.pos = 0;
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:301:19: error: implicit declaration of function ‘ZSTD_decompressStream’ [-Werror=implicit-function-declaration]
ret = ZSTD_decompressStream(z->zds, &z->out, &z->in);
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:301:13: error: nested extern declaration of ‘ZSTD_decompressStream’ [-Werror=nested-externs]
ret = ZSTD_decompressStream(z->zds, &z->out, &z->in);
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:302:35: error: request for member ‘size’ in something not a structure or union
} while (ret > 0 && (z->in.size - z->in.pos > 0)
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:302:48: error: request for member ‘pos’ in something not a structure or union
} while (ret > 0 && (z->in.size - z->in.pos > 0)
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:303:36: error: request for member ‘pos’ in something not a structure or union
&& (z->out.pos < iov->iov_len));
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:304:31: error: request for member ‘pos’ in something not a structure or union
if (ret > 0 && (z->out.pos < iov->iov_len)) {
^
/home/travis/build/stsquad/qemu/migration/multifd-zstd.c:314:27: error: request for member ‘pos’ in something not a structure or union
out_size += z->out.pos;
^
cc1: all warnings being treated as errors
/home/travis/build/stsquad/qemu/rules.mak:69: recipe for target 'migration/multifd-zstd.o' failed
make: *** [migration/multifd-zstd.o] Error 1
make: *** Waiting for unfinished jobs....
rm tests/qemu-iotests/socket_scm_helper.o
> zstd_cflags="$($pkg_config --cflags libzstd)"
> zstd_libs="$($pkg_config --libs libzstd)"
> LIBS="$zstd_libs $LIBS"
--
Alex Bennée
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v1 07/10] configure: fix check for libzstd
2020-03-02 22:02 ` Alex Bennée
@ 2020-03-05 9:43 ` Juan Quintela
2020-03-05 10:09 ` Alex Bennée
0 siblings, 1 reply; 23+ messages in thread
From: Juan Quintela @ 2020-03-05 9:43 UTC (permalink / raw)
To: Alex Bennée; +Cc: qemu-devel
Alex Bennée <alex.bennee@linaro.org> wrote:
> Alex Bennée <alex.bennee@linaro.org> writes:
>
>> Fixes: 3a67848134d0
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
>> configure | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/configure b/configure
>> index 7b373bc0bb8..caa65f58831 100755
>> --- a/configure
>> +++ b/configure
>> @@ -2464,7 +2464,7 @@ fi
>> # zstd check
>>
>> if test "$zstd" != "no" ; then
>> - if $pkg_config --exist libzstd ; then
>> + if $pkg_config --exists libzstd ; then
Hi
several things:
a- I found why I didn't get the error. Fedora pkg-config is really
"smart":
b- I have tried (with this patch), the following configurations:
* --enable-zstd
* --disable-zstd
with both libzstd-devel installed and not installed. Everything
worked as expected.
(BTW, I tested that before submmiting the patch in the first place, I
thought that I had done all testing needed for such a check).
Can you told me what architecture/distro/os are you using.
> Dropping this patch as it breaks the build even more!
>
> CC migration/block.o
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:24:5: error: unknown type name ‘ZSTD_CStream’
>
> ZSTD_CStream *zcs;
>
> ^
This is really weird. if you arrive here, that means:
- you have zstd devel installed (whatever is that called for your
os/distro/whatever).
- pkg-config has found zstd devel packages and configured them (that
file depends on CONFiG_ZSTD beoing defined)
- gcc has found <zstd.h> (i.e. it don't give one error about that
include file not found).
And zstd don't have ZSTD_CStream defined? What is going on here?
Can you post/show what is on your zstd.h file?
What zstd library version do you have?
I thought that zstd was a new library, and that we didn't need to check
for versions. It appears that I was wrong. And no, the include file
don't show what features are new/old.
Sorry for the inconveniences.
Later, Juan.
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:26:5: error: unknown type name ‘ZSTD_DStream’
>
> ZSTD_DStream *zds;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:28:5: error: unknown type name ‘ZSTD_inBuffer’
>
> ZSTD_inBuffer in;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:29:5: error: unknown type name ‘ZSTD_outBuffer’
>
> ZSTD_outBuffer out;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c: In function ‘zstd_send_setup’:
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:55:14: error: implicit declaration of function ‘ZSTD_createCStream’ [-Werror=implicit-function-declaration]
>
> z->zcs = ZSTD_createCStream();
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:55:5: error: nested extern declaration of ‘ZSTD_createCStream’ [-Werror=nested-externs]
>
> z->zcs = ZSTD_createCStream();
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:55:12: error: assignment makes pointer from integer without a cast [-Werror=int-conversion]
>
> z->zcs = ZSTD_createCStream();
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:62:11: error: implicit declaration of function ‘ZSTD_initCStream’ [-Werror=implicit-function-declaration]
>
> res = ZSTD_initCStream(z->zcs, migrate_multifd_zstd_level());
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:62:5: error: nested extern declaration of ‘ZSTD_initCStream’ [-Werror=nested-externs]
>
> res = ZSTD_initCStream(z->zcs, migrate_multifd_zstd_level());
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:64:9: error: implicit declaration of function ‘ZSTD_freeCStream’ [-Werror=implicit-function-declaration]
>
> ZSTD_freeCStream(z->zcs);
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:64:9: error: nested extern declaration of ‘ZSTD_freeCStream’ [-Werror=nested-externs]
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c: In function ‘zstd_send_prepare’:
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:120:11: error: request for member ‘dst’ in something not a structure or union
>
> z->out.dst = z->zbuff;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:121:11: error: request for member ‘size’ in something not a structure or union
>
> z->out.size = z->zbuff_len;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:122:11: error: request for member ‘pos’ in something not a structure or union
>
> z->out.pos = 0;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:125:9: error: unknown type name ‘ZSTD_EndDirective’
>
> ZSTD_EndDirective flush = ZSTD_e_continue;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:125:35: error: ‘ZSTD_e_continue’ undeclared (first use in this function)
>
> ZSTD_EndDirective flush = ZSTD_e_continue;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:125:35: note: each undeclared identifier is reported only once for each function it appears in
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:128:21: error: ‘ZSTD_e_flush’ undeclared (first use in this function)
>
> flush = ZSTD_e_flush;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:130:14: error: request for member ‘src’ in something not a structure or union
>
> z->in.src = iov[i].iov_base;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:131:14: error: request for member ‘size’ in something not a structure or union
>
> z->in.size = iov[i].iov_len;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:132:14: error: request for member ‘pos’ in something not a structure or union
>
> z->in.pos = 0;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:143:19: error: implicit declaration of function ‘ZSTD_compressStream2’ [-Werror=implicit-function-declaration]
>
> ret = ZSTD_compressStream2(z->zcs, &z->out, &z->in, flush);
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:143:13: error: nested extern declaration of ‘ZSTD_compressStream2’ [-Werror=nested-externs]
>
> ret = ZSTD_compressStream2(z->zcs, &z->out, &z->in, flush);
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:144:35: error: request for member ‘size’ in something not a structure or union
>
> } while (ret > 0 && (z->in.size - z->in.pos > 0)
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:144:48: error: request for member ‘pos’ in something not a structure or union
>
> } while (ret > 0 && (z->in.size - z->in.pos > 0)
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:145:36: error: request for member ‘size’ in something not a structure or union
>
> && (z->out.size - z->out.pos > 0));
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:145:50: error: request for member ‘pos’ in something not a structure or union
>
> && (z->out.size - z->out.pos > 0));
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:146:30: error: request for member ‘size’ in something not a structure or union
>
> if (ret > 0 && (z->in.size - z->in.pos > 0)) {
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:146:43: error: request for member ‘pos’ in something not a structure or union
>
> if (ret > 0 && (z->in.size - z->in.pos > 0)) {
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:157:33: error: request for member ‘pos’ in something not a structure or union
>
> p->next_packet_size = z->out.pos;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c: In function ‘zstd_recv_setup’:
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:199:14: error: implicit declaration of function ‘ZSTD_createDStream’ [-Werror=implicit-function-declaration]
>
> z->zds = ZSTD_createDStream();
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:199:5: error: nested extern declaration of ‘ZSTD_createDStream’ [-Werror=nested-externs]
>
> z->zds = ZSTD_createDStream();
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:199:12: error: assignment makes pointer from integer without a cast [-Werror=int-conversion]
>
> z->zds = ZSTD_createDStream();
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:206:11: error: implicit declaration of function ‘ZSTD_initDStream’ [-Werror=implicit-function-declaration]
>
> ret = ZSTD_initDStream(z->zds);
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:206:5: error: nested extern declaration of ‘ZSTD_initDStream’ [-Werror=nested-externs]
>
> ret = ZSTD_initDStream(z->zds);
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:208:9: error: implicit declaration of function ‘ZSTD_freeDStream’ [-Werror=implicit-function-declaration]
>
> ZSTD_freeDStream(z->zds);
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:208:9: error: nested extern declaration of ‘ZSTD_freeDStream’ [-Werror=nested-externs]
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c: In function ‘zstd_recv_pages’:
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:281:10: error: request for member ‘src’ in something not a structure or union
>
> z->in.src = z->zbuff;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:282:10: error: request for member ‘size’ in something not a structure or union
>
> z->in.size = in_size;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:283:10: error: request for member ‘pos’ in something not a structure or union
>
> z->in.pos = 0;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:288:15: error: request for member ‘dst’ in something not a structure or union
>
> z->out.dst = iov->iov_base;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:289:15: error: request for member ‘size’ in something not a structure or union
>
> z->out.size = iov->iov_len;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:290:15: error: request for member ‘pos’ in something not a structure or union
>
> z->out.pos = 0;
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:301:19: error: implicit declaration of function ‘ZSTD_decompressStream’ [-Werror=implicit-function-declaration]
>
> ret = ZSTD_decompressStream(z->zds, &z->out, &z->in);
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:301:13: error: nested extern declaration of ‘ZSTD_decompressStream’ [-Werror=nested-externs]
>
> ret = ZSTD_decompressStream(z->zds, &z->out, &z->in);
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:302:35: error: request for member ‘size’ in something not a structure or union
>
> } while (ret > 0 && (z->in.size - z->in.pos > 0)
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:302:48: error: request for member ‘pos’ in something not a structure or union
>
> } while (ret > 0 && (z->in.size - z->in.pos > 0)
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:303:36: error: request for member ‘pos’ in something not a structure or union
>
> && (z->out.pos < iov->iov_len));
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:304:31: error: request for member ‘pos’ in something not a structure or union
>
> if (ret > 0 && (z->out.pos < iov->iov_len)) {
>
> ^
>
> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:314:27: error: request for member ‘pos’ in something not a structure or union
>
> out_size += z->out.pos;
>
> ^
>
> cc1: all warnings being treated as errors
>
> /home/travis/build/stsquad/qemu/rules.mak:69: recipe for target 'migration/multifd-zstd.o' failed
>
> make: *** [migration/multifd-zstd.o] Error 1
>
> make: *** Waiting for unfinished jobs....
>
> rm tests/qemu-iotests/socket_scm_helper.o
>
>> zstd_cflags="$($pkg_config --cflags libzstd)"
>> zstd_libs="$($pkg_config --libs libzstd)"
>> LIBS="$zstd_libs $LIBS"
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v1 07/10] configure: fix check for libzstd
2020-03-05 9:43 ` Juan Quintela
@ 2020-03-05 10:09 ` Alex Bennée
2020-03-05 10:28 ` Juan Quintela
0 siblings, 1 reply; 23+ messages in thread
From: Alex Bennée @ 2020-03-05 10:09 UTC (permalink / raw)
To: quintela; +Cc: qemu-devel
Juan Quintela <quintela@redhat.com> writes:
> Alex Bennée <alex.bennee@linaro.org> wrote:
>> Alex Bennée <alex.bennee@linaro.org> writes:
>>
>>> Fixes: 3a67848134d0
>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>> ---
>>> configure | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/configure b/configure
>>> index 7b373bc0bb8..caa65f58831 100755
>>> --- a/configure
>>> +++ b/configure
>>> @@ -2464,7 +2464,7 @@ fi
>>> # zstd check
>>>
>>> if test "$zstd" != "no" ; then
>>> - if $pkg_config --exist libzstd ; then
>>> + if $pkg_config --exists libzstd ; then
>
> Hi
>
> several things:
>
> a- I found why I didn't get the error. Fedora pkg-config is really
> "smart":
>
> b- I have tried (with this patch), the following configurations:
> * --enable-zstd
> * --disable-zstd
> with both libzstd-devel installed and not installed. Everything
> worked as expected.
> (BTW, I tested that before submmiting the patch in the first place, I
> thought that I had done all testing needed for such a check).
>
> Can you told me what architecture/distro/os are you using.
It broke on the CI setup - it could be another issue with the ageing
Travis images (Ubuntu 16.04).
>> Dropping this patch as it breaks the build even more!
>>
>> CC migration/block.o
>>
>> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:24:5: error: unknown type name ‘ZSTD_CStream’
>>
>> ZSTD_CStream *zcs;
>>
>> ^
>
> This is really weird. if you arrive here, that means:
> - you have zstd devel installed (whatever is that called for your
> os/distro/whatever).
Well it detected it:
zstd support yes
>
> - pkg-config has found zstd devel packages and configured them (that
> file depends on CONFiG_ZSTD beoing defined)
>
> - gcc has found <zstd.h> (i.e. it don't give one error about that
> include file not found).
>
> And zstd don't have ZSTD_CStream defined? What is going on here?
> Can you post/show what is on your zstd.h file?
> What zstd library version do you have?
>
> I thought that zstd was a new library, and that we didn't need to check
> for versions. It appears that I was wrong. And no, the include file
> don't show what features are new/old.
Obviously not that new but has changed since it first got introduced.
--
Alex Bennée
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH v1 07/10] configure: fix check for libzstd
2020-03-05 10:09 ` Alex Bennée
@ 2020-03-05 10:28 ` Juan Quintela
0 siblings, 0 replies; 23+ messages in thread
From: Juan Quintela @ 2020-03-05 10:28 UTC (permalink / raw)
To: Alex Bennée; +Cc: qemu-devel
Alex Bennée <alex.bennee@linaro.org> wrote:
> Juan Quintela <quintela@redhat.com> writes:
>
>> Alex Bennée <alex.bennee@linaro.org> wrote:
>>> Alex Bennée <alex.bennee@linaro.org> writes:
>>>
>>
>> Can you told me what architecture/distro/os are you using.
>
> It broke on the CI setup - it could be another issue with the ageing
> Travis images (Ubuntu 16.04).
>
>>> Dropping this patch as it breaks the build even more!
>>>
>>> CC migration/block.o
>>>
>>> /home/travis/build/stsquad/qemu/migration/multifd-zstd.c:24:5: error: unknown type name ‘ZSTD_CStream’
>>>
>>> ZSTD_CStream *zcs;
>>>
>>> ^
>>
>> This is really weird. if you arrive here, that means:
>> - you have zstd devel installed (whatever is that called for your
>> os/distro/whatever).
>
> Well it detected it:
>
> zstd support yes
>
>>
>> - pkg-config has found zstd devel packages and configured them (that
>> file depends on CONFiG_ZSTD beoing defined)
>>
>> - gcc has found <zstd.h> (i.e. it don't give one error about that
>> include file not found).
>>
>> And zstd don't have ZSTD_CStream defined? What is going on here?
>> Can you post/show what is on your zstd.h file?
>> What zstd library version do you have?
>>
>> I thought that zstd was a new library, and that we didn't need to check
>> for versions. It appears that I was wrong. And no, the include file
>> don't show what features are new/old.
>
> Obviously not that new but has changed since it first got introduced.
I put my archeology hat, and went digging.
Streaming API was introduced on this commit:
commit 5a0c8e24395079f8e8cdc90aa1659cd5ab1b7427
Author: Yann Collet <yann.collet.73@gmail.com>
Date: Fri Aug 12 01:20:36 2016 +0200
new streaming API (compression)
And it first appears on v0.8.1 version.
Posting a better fix on toplevel. If you want to try, I have changed it
to:
diff --git a/configure b/configure
index 7b373bc0bb..1bf48df1ef 100755
--- a/configure
+++ b/configure
@@ -2464,7 +2464,8 @@ fi
# zstd check
if test "$zstd" != "no" ; then
- if $pkg_config --exist libzstd ; then
+ libzstd_minver="0.8.1"
+ if $pkg_config --atleast-version=$libzstd_minver libzstd ; then
zstd_cflags="$($pkg_config --cflags libzstd)"
zstd_libs="$($pkg_config --libs libzstd)"
LIBS="$zstd_libs $LIBS"
If you can check that this works for you, thanks.
^ permalink raw reply related [flat|nested] 23+ messages in thread
end of thread, other threads:[~2020-03-05 10:29 UTC | newest]
Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-03-02 18:18 [PATCH v1 00/10] testing/next updates (tweaks and re-greening) Alex Bennée
2020-03-02 18:18 ` [PATCH v1 01/10] tests/vm: use $(PYTHON) consistently Alex Bennée
2020-03-02 18:29 ` Philippe Mathieu-Daudé
2020-03-02 18:41 ` Alex Bennée
2020-03-02 18:50 ` Philippe Mathieu-Daudé
2020-03-02 18:18 ` [PATCH v1 02/10] tests/vm: Debug mode shows ssh output Alex Bennée
2020-03-02 18:30 ` Philippe Mathieu-Daudé
2020-03-02 18:19 ` [PATCH v1 03/10] tests/vm: increased max timeout for vm boot Alex Bennée
2020-03-02 18:32 ` Philippe Mathieu-Daudé
2020-03-02 18:19 ` [PATCH v1 04/10] tests/vm: give wait_ssh() option to wait for root Alex Bennée
2020-03-02 18:19 ` [PATCH v1 05/10] tests/vm: Added gen_cloud_init_iso() to basevm.py Alex Bennée
2020-03-02 18:32 ` Philippe Mathieu-Daudé
2020-03-02 18:19 ` [PATCH v1 06/10] travis: enable tools build on OS X Alex Bennée
2020-03-02 18:46 ` Philippe Mathieu-Daudé
2020-03-02 18:19 ` [PATCH v1 07/10] configure: fix check for libzstd Alex Bennée
2020-03-02 22:02 ` Alex Bennée
2020-03-05 9:43 ` Juan Quintela
2020-03-05 10:09 ` Alex Bennée
2020-03-05 10:28 ` Juan Quintela
2020-03-02 18:19 ` [PATCH v1 08/10] configure: detect and report genisoimage Alex Bennée
2020-03-02 18:25 ` Philippe Mathieu-Daudé
2020-03-02 18:19 ` [PATCH v1 09/10] tests/acceptance: move @skipUnless decoration to test itself Alex Bennée
2020-03-02 18:19 ` [PATCH v1 10/10] travis.yml: install python3 numpy and opencv libraries Alex Bennée
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.