* [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm)
@ 2019-06-19 19:40 Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 01/17] tests/vm: avoid extra compressed image copy Alex Bennée
` (16 more replies)
0 siblings, 17 replies; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel; +Cc: Alex Bennée
Hi,
This is the current state of testing/next which has fixes for the
MacOS Travis failures. We also have the return of the bsd-over-serial
autosetup patches which I've tested on the previously failing
platform. The one thing I've not included is adding check-block to the
default set of check tests. The blocker for that is it still bails on
some iotests when the source tree filesystem (e.g. ZFS) doesn't
support features it needs for some tests. It seems that they should
skip a bit more gracefully:
Not run: 045 059 064 070 075 077 078 081 084 088 092 094 101 106 113 116 119 123 128 131 146 148 160 162 171 175 211 212 213 221 225 231 233 237 239 241 253
Some cases not run in: 139
Failures: 102 243 250
Failed 3 of 138 tests
I'll try and send the PR for this out by the end of the week.
The following patches need review
patch 0014/tests vm fedora autoinstall using serial console.patch
patch 0016/.travis.yml default the disable system build to s.patch
patch 0017/.travis.yml force a brew update for MacOS builds.patch
Alex Bennée (2):
.travis.yml: default the --disable-system build to --static
.travis.yml: force a brew update for MacOS builds
Cleber Rosa (4):
tests/vm: avoid extra compressed image copy
tests/vm: avoid image presence check and removal
tests/vm: pin ubuntu.i386 image
tests/vm: add source repos on ubuntu.i386
Gerd Hoffmann (11):
tests/vm: send proxy environment variables over ssh
tests/vm: use ssh with pty unconditionally
tests/vm: run test builds on snapshot
tests/vm: proper guest shutdown
tests/vm: add vm-boot-{ssh,serial}-<guest> targets
tests/vm: serial console support helpers
tests/vm: openbsd autoinstall, using serial console
tests/vm: freebsd autoinstall, using serial console
tests/vm: netbsd autoinstall, using serial console
tests/vm: fedora autoinstall, using serial console
tests/vm: ubuntu.i386: apt proxy setup
.travis.yml | 3 +-
tests/vm/Makefile.include | 23 ++++-
tests/vm/basevm.py | 135 +++++++++++++++++++++++----
tests/vm/centos | 6 +-
tests/vm/fedora | 189 ++++++++++++++++++++++++++++++++++++++
tests/vm/freebsd | 180 ++++++++++++++++++++++++++++++++++--
tests/vm/netbsd | 187 +++++++++++++++++++++++++++++++++++--
tests/vm/openbsd | 159 +++++++++++++++++++++++++++++---
tests/vm/ubuntu.i386 | 11 ++-
9 files changed, 833 insertions(+), 60 deletions(-)
create mode 100755 tests/vm/fedora
--
2.20.1
^ permalink raw reply [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 01/17] tests/vm: avoid extra compressed image copy
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-20 11:07 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 02/17] tests/vm: avoid image presence check and removal Alex Bennée
` (15 subsequent siblings)
16 siblings, 1 reply; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Philippe Mathieu-Daudé, Alex Bennée, Cleber Rosa
From: Cleber Rosa <crosa@redhat.com>
The image copy is only really needed because xz doesn't know to
properly decompress a file not named properly. Instead of
decompressing to stdout, and having to rely on a shell, let's just
create a link instead of copying the file.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20190613130718.3763-2-crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/vm/centos | 4 ++--
tests/vm/freebsd | 4 ++--
tests/vm/netbsd | 4 ++--
tests/vm/openbsd | 4 ++--
4 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/tests/vm/centos b/tests/vm/centos
index 7417b50af4..b00b46a8dc 100755
--- a/tests/vm/centos
+++ b/tests/vm/centos
@@ -66,8 +66,8 @@ class CentosVM(basevm.BaseVM):
cimg = self._download_with_cache("https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1802.qcow2.xz")
img_tmp = img + ".tmp"
sys.stderr.write("Extracting the image...\n")
- subprocess.check_call(["cp", "-f", cimg, img_tmp + ".xz"])
- subprocess.check_call(["xz", "-dvf", img_tmp + ".xz"])
+ subprocess.check_call(["ln", "-f", cimg, img_tmp + ".xz"])
+ subprocess.check_call(["xz", "--keep", "-dvf", img_tmp + ".xz"])
subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
self.wait_ssh()
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index b0066017a6..5575c23a6f 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -34,8 +34,8 @@ class FreeBSDVM(basevm.BaseVM):
img_tmp_xz = img + ".tmp.xz"
img_tmp = img + ".tmp"
sys.stderr.write("Extracting the image...\n")
- subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
- subprocess.check_call(["xz", "-dvf", img_tmp_xz])
+ subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
+ subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
if os.path.exists(img):
os.remove(img)
os.rename(img_tmp, img)
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index 4c6624ea5e..d0508f4465 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -34,8 +34,8 @@ class NetBSDVM(basevm.BaseVM):
img_tmp_xz = img + ".tmp.xz"
img_tmp = img + ".tmp"
sys.stderr.write("Extracting the image...\n")
- subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
- subprocess.check_call(["xz", "-dvf", img_tmp_xz])
+ subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
+ subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
if os.path.exists(img):
os.remove(img)
os.rename(img_tmp, img)
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 2105c01a26..87ec982489 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -36,8 +36,8 @@ class OpenBSDVM(basevm.BaseVM):
img_tmp_xz = img + ".tmp.xz"
img_tmp = img + ".tmp"
sys.stderr.write("Extracting the image...\n")
- subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
- subprocess.check_call(["xz", "-dvf", img_tmp_xz])
+ subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
+ subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
if os.path.exists(img):
os.remove(img)
os.rename(img_tmp, img)
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 02/17] tests/vm: avoid image presence check and removal
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 01/17] tests/vm: avoid extra compressed image copy Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-20 10:29 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 03/17] tests/vm: pin ubuntu.i386 image Alex Bennée
` (14 subsequent siblings)
16 siblings, 1 reply; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Philippe Mathieu-Daudé, Alex Bennée, Cleber Rosa
From: Cleber Rosa <crosa@redhat.com>
Python's os.rename() will silently replace an existing file,
so there's no need for the extra check and removal.
Reference: https://docs.python.org/3/library/os.html#os.rename
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20190613130718.3763-3-crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/vm/centos | 2 --
tests/vm/freebsd | 2 --
tests/vm/netbsd | 2 --
tests/vm/openbsd | 2 --
tests/vm/ubuntu.i386 | 2 --
5 files changed, 10 deletions(-)
diff --git a/tests/vm/centos b/tests/vm/centos
index b00b46a8dc..53976f1c4c 100755
--- a/tests/vm/centos
+++ b/tests/vm/centos
@@ -77,8 +77,6 @@ class CentosVM(basevm.BaseVM):
self.ssh_root_check("systemctl enable docker")
self.ssh_root("poweroff")
self.wait()
- if os.path.exists(img):
- os.remove(img)
os.rename(img_tmp, img)
return 0
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 5575c23a6f..091be1a065 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -36,8 +36,6 @@ class FreeBSDVM(basevm.BaseVM):
sys.stderr.write("Extracting the image...\n")
subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
- if os.path.exists(img):
- os.remove(img)
os.rename(img_tmp, img)
if __name__ == "__main__":
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index d0508f4465..ee9eaeab50 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -36,8 +36,6 @@ class NetBSDVM(basevm.BaseVM):
sys.stderr.write("Extracting the image...\n")
subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
- if os.path.exists(img):
- os.remove(img)
os.rename(img_tmp, img)
if __name__ == "__main__":
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 87ec982489..28c7d25e29 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -38,8 +38,6 @@ class OpenBSDVM(basevm.BaseVM):
sys.stderr.write("Extracting the image...\n")
subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
- if os.path.exists(img):
- os.remove(img)
os.rename(img_tmp, img)
if __name__ == "__main__":
diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index a22d137e76..12867b193f 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -80,8 +80,6 @@ class UbuntuX86VM(basevm.BaseVM):
self.ssh_root_check("apt-get install -y libfdt-dev flex bison")
self.ssh_root("poweroff")
self.wait()
- if os.path.exists(img):
- os.remove(img)
os.rename(img_tmp, img)
return 0
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 03/17] tests/vm: pin ubuntu.i386 image
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 01/17] tests/vm: avoid extra compressed image copy Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 02/17] tests/vm: avoid image presence check and removal Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-20 10:31 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 04/17] tests/vm: add source repos on ubuntu.i386 Alex Bennée
` (13 subsequent siblings)
16 siblings, 1 reply; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Philippe Mathieu-Daudé, Alex Bennée, Cleber Rosa
From: Cleber Rosa <crosa@redhat.com>
It's a good practice to always have the same components used in tests.
According to:
https://cloud-images.ubuntu.com/releases/16.04/
New images are released from time to time, and the "release/"
directory points to the latest release. Let's pin to the latest
available version, and while at it, set a hash for verification.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20190613130718.3763-4-crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/vm/ubuntu.i386 | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 12867b193f..7017e6d388 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -61,7 +61,9 @@ class UbuntuX86VM(basevm.BaseVM):
return os.path.join(cidir, "cloud-init.iso")
def build_image(self, img):
- cimg = self._download_with_cache("https://cloud-images.ubuntu.com/releases/16.04/release/ubuntu-16.04-server-cloudimg-i386-disk1.img")
+ cimg = self._download_with_cache(
+ "https://cloud-images.ubuntu.com/releases/16.04/release-20190605/ubuntu-16.04-server-cloudimg-i386-disk1.img",
+ sha256sum="e30091144c73483822b7c27193e9d47346dd1064229da577c3fedcf943f7cfcc")
img_tmp = img + ".tmp"
subprocess.check_call(["cp", "-f", cimg, img_tmp])
subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 04/17] tests/vm: add source repos on ubuntu.i386
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
` (2 preceding siblings ...)
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 03/17] tests/vm: pin ubuntu.i386 image Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-20 11:06 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 05/17] tests/vm: send proxy environment variables over ssh Alex Bennée
` (12 subsequent siblings)
16 siblings, 1 reply; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Philippe Mathieu-Daudé, Alex Bennée, Cleber Rosa
From: Cleber Rosa <crosa@redhat.com>
Possibly because of different behavior on the newly update
cloud-image, trying to run 'apt-get build-dep' results in:
E: You must put some 'source' URIs in your sources.list
This enables all source repos (even though some are not
needed) for simplicity sake.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20190613130718.3763-5-crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/vm/ubuntu.i386 | 1 +
1 file changed, 1 insertion(+)
diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 7017e6d388..3ea459ee20 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -77,6 +77,7 @@ class UbuntuX86VM(basevm.BaseVM):
time.sleep(5)
self.wait_ssh()
# The previous update sometimes doesn't survive a reboot, so do it again
+ self.ssh_root_check("sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sources.list")
self.ssh_root_check("apt-get update")
self.ssh_root_check("apt-get build-dep -y qemu")
self.ssh_root_check("apt-get install -y libfdt-dev flex bison")
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 05/17] tests/vm: send proxy environment variables over ssh
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
` (3 preceding siblings ...)
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 04/17] tests/vm: add source repos on ubuntu.i386 Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-20 10:38 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 06/17] tests/vm: use ssh with pty unconditionally Alex Bennée
` (11 subsequent siblings)
16 siblings, 1 reply; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Alex Bennée, Thomas Huth,
Philippe Mathieu-Daudé,
Gerd Hoffmann
From: Gerd Hoffmann <kraxel@redhat.com>
Packages are fetched via proxy that way, if configured on the host.
That might be required to pass firewalls, and it allows to route
package downloads through a caching proxy server.
Needs AcceptEnv setup in sshd_config on the guest side to work.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190617043858.8290-2-kraxel@redhat.com>
---
tests/vm/basevm.py | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 4847549592..5e30bac661 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -38,6 +38,13 @@ class BaseVM(object):
GUEST_PASS = "qemupass"
ROOT_PASS = "qemupass"
+ envvars = [
+ "https_proxy",
+ "http_proxy",
+ "ftp_proxy",
+ "no_proxy",
+ ]
+
# The script to run in the guest that builds QEMU
BUILD_SCRIPT = ""
# The guest name, to be overridden by subclasses
@@ -106,6 +113,8 @@ class BaseVM(object):
"-o", "UserKnownHostsFile=" + os.devnull,
"-o", "ConnectTimeout=1",
"-p", self.ssh_port, "-i", self._ssh_key_file]
+ for var in self.envvars:
+ ssh_cmd += ['-o', "SendEnv=%s" % var ]
if interactive:
ssh_cmd += ['-t']
assert not isinstance(cmd, str)
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 06/17] tests/vm: use ssh with pty unconditionally
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
` (4 preceding siblings ...)
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 05/17] tests/vm: send proxy environment variables over ssh Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 07/17] tests/vm: run test builds on snapshot Alex Bennée
` (10 subsequent siblings)
16 siblings, 0 replies; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Alex Bennée, Thomas Huth,
Philippe Mathieu-Daudé,
Gerd Hoffmann
From: Gerd Hoffmann <kraxel@redhat.com>
Allways ask ssh to run with a pseudo terminal.
Not having a terminal causes problems now and then.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-3-kraxel@redhat.com>
[AJB: added PMD's r-b/t-b tags]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/vm/basevm.py | 13 ++++---------
1 file changed, 4 insertions(+), 9 deletions(-)
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 5e30bac661..395eefaec9 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -107,16 +107,14 @@ class BaseVM(object):
os.rename(fname + ".download", fname)
return fname
- def _ssh_do(self, user, cmd, check, interactive=False):
- ssh_cmd = ["ssh", "-q",
+ def _ssh_do(self, user, cmd, check):
+ ssh_cmd = ["ssh", "-q", "-t",
"-o", "StrictHostKeyChecking=no",
"-o", "UserKnownHostsFile=" + os.devnull,
"-o", "ConnectTimeout=1",
"-p", self.ssh_port, "-i", self._ssh_key_file]
for var in self.envvars:
ssh_cmd += ['-o', "SendEnv=%s" % var ]
- if interactive:
- ssh_cmd += ['-t']
assert not isinstance(cmd, str)
ssh_cmd += ["%s@127.0.0.1" % user] + list(cmd)
logging.debug("ssh_cmd: %s", " ".join(ssh_cmd))
@@ -128,9 +126,6 @@ class BaseVM(object):
def ssh(self, *cmd):
return self._ssh_do(self.GUEST_USER, cmd, False)
- def ssh_interactive(self, *cmd):
- return self._ssh_do(self.GUEST_USER, cmd, False, True)
-
def ssh_root(self, *cmd):
return self._ssh_do("root", cmd, False)
@@ -284,9 +279,9 @@ def main(vmcls):
return 2
if args.interactive:
- if vm.ssh_interactive(*cmd) == 0:
+ if vm.ssh(*cmd) == 0:
return 0
- vm.ssh_interactive()
+ vm.ssh()
return 3
else:
if vm.ssh(*cmd) != 0:
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 07/17] tests/vm: run test builds on snapshot
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
` (5 preceding siblings ...)
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 06/17] tests/vm: use ssh with pty unconditionally Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 08/17] tests/vm: proper guest shutdown Alex Bennée
` (9 subsequent siblings)
16 siblings, 0 replies; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Thomas Huth, Ed Maste, Philippe Mathieu-Daudé,
Gerd Hoffmann, Alex Bennée
From: Gerd Hoffmann <kraxel@redhat.com>
The build script doesn't shutdown the guest VMs properly,
which results in filesystem corruption and guest boot
failures sooner or later.
Use the --snapshot to run builds on a snapshot,
That way killing the VM doesn't corrupt the base image.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Acked-by: Ed Maste <emaste@freebsd.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-4-kraxel@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/vm/Makefile.include | 1 +
1 file changed, 1 insertion(+)
diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index c59411bee0..276b870216 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -57,6 +57,7 @@ vm-build-%: $(IMAGES_DIR)/%.img
$(if $(V),--verbose) \
--image "$<" \
$(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \
+ --snapshot \
--build-qemu $(SRC_PATH) -- \
$(if $(TARGET_LIST),--target-list=$(TARGET_LIST)) \
$(if $(EXTRA_CONFIGURE_OPTS),$(EXTRA_CONFIGURE_OPTS)), \
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 08/17] tests/vm: proper guest shutdown
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
` (6 preceding siblings ...)
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 07/17] tests/vm: run test builds on snapshot Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 09/17] tests/vm: add vm-boot-{ssh, serial}-<guest> targets Alex Bennée
` (8 subsequent siblings)
16 siblings, 0 replies; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Alex Bennée, Thomas Huth,
Philippe Mathieu-Daudé,
Gerd Hoffmann
From: Gerd Hoffmann <kraxel@redhat.com>
When not running in snapshot mode ask the guest to poweroff and wait for
this to finish instead of simply quitting qemu, so the guest can flush
pending updates to disk.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-5-kraxel@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/vm/basevm.py | 22 +++++++++++++++-------
1 file changed, 15 insertions(+), 7 deletions(-)
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 395eefaec9..f27178f3c7 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -51,6 +51,8 @@ class BaseVM(object):
name = "#base"
# The guest architecture, to be overridden by subclasses
arch = "#arch"
+ # command to halt the guest, can be overridden by subclasses
+ poweroff = "poweroff"
def __init__(self, debug=False, vcpus=None):
self._guest = None
self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-",
@@ -202,6 +204,10 @@ class BaseVM(object):
def wait(self):
self._guest.wait()
+ def graceful_shutdown(self):
+ self.ssh_root(self.poweroff)
+ self._guest.wait()
+
def qmp(self, *args, **kwargs):
return self._guest.qmp(*args, **kwargs)
@@ -278,11 +284,13 @@ def main(vmcls):
traceback.print_exc()
return 2
- if args.interactive:
- if vm.ssh(*cmd) == 0:
- return 0
+ exitcode = 0
+ if vm.ssh(*cmd) != 0:
+ exitcode = 3
+ if exitcode != 0 and args.interactive:
vm.ssh()
- return 3
- else:
- if vm.ssh(*cmd) != 0:
- return 3
+
+ if not args.snapshot:
+ vm.graceful_shutdown()
+
+ return exitcode
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 09/17] tests/vm: add vm-boot-{ssh, serial}-<guest> targets
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
` (7 preceding siblings ...)
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 08/17] tests/vm: proper guest shutdown Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 10/17] tests/vm: serial console support helpers Alex Bennée
` (7 subsequent siblings)
16 siblings, 0 replies; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Alex Bennée, Thomas Huth,
Philippe Mathieu-Daudé,
Gerd Hoffmann
From: Gerd Hoffmann <kraxel@redhat.com>
For testing/troubleshooting convenience.
make vm-boot-serial-<guest>
Boot guest, with the serial console on stdio.
make vm-boot-ssh-<guest>
Boot guest, login via ssh.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-6-kraxel@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/vm/Makefile.include | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index 276b870216..a61c236b8e 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -20,6 +20,10 @@ vm-test:
@echo " vm-build-all - Build QEMU in all VMs"
@echo " vm-clean-all - Clean up VM images"
@echo
+ @echo "For trouble-shooting:"
+ @echo " vm-boot-serial-<guest> - Boot guest, serial console on stdio"
+ @echo " vm-boot-ssh-<guest> - Boot guest and login via ssh"
+ @echo
@echo "Special variables:"
@echo " BUILD_TARGET=foo - Override the build target"
@echo " TARGET_LIST=a,b,c - Override target list in builds"
@@ -63,3 +67,18 @@ vm-build-%: $(IMAGES_DIR)/%.img
$(if $(EXTRA_CONFIGURE_OPTS),$(EXTRA_CONFIGURE_OPTS)), \
" VM-BUILD $*")
+vm-boot-serial-%: $(IMAGES_DIR)/%.img
+ qemu-system-x86_64 -enable-kvm -m 4G -smp 2 -nographic \
+ -drive if=none,id=vblk,cache=writeback,file="$<" \
+ -netdev user,id=vnet \
+ -device virtio-blk-pci,drive=vblk \
+ -device virtio-net-pci,netdev=vnet \
+ || true
+
+vm-boot-ssh-%: $(IMAGES_DIR)/%.img
+ $(call quiet-command, \
+ $(SRC_PATH)/tests/vm/$* \
+ --image "$<" \
+ --interactive \
+ false, \
+ " VM-BOOT-SSH $*") || true
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 10/17] tests/vm: serial console support helpers
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
` (8 preceding siblings ...)
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 09/17] tests/vm: add vm-boot-{ssh, serial}-<guest> targets Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 11/17] tests/vm: openbsd autoinstall, using serial console Alex Bennée
` (6 subsequent siblings)
16 siblings, 0 replies; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Alex Bennée, Thomas Huth,
Philippe Mathieu-Daudé,
Gerd Hoffmann
From: Gerd Hoffmann <kraxel@redhat.com>
Add a bunch of helpers to talk to the guest using the
serial console.
Also drop the hard-coded -serial parameter for the vm
so QEMUMachine.set_console() actually works.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-7-kraxel@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/vm/basevm.py | 86 ++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 83 insertions(+), 3 deletions(-)
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index f27178f3c7..592a344773 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -2,10 +2,11 @@
#
# VM testing base class
#
-# Copyright 2017 Red Hat Inc.
+# Copyright 2017-2019 Red Hat Inc.
#
# Authors:
# Fam Zheng <famz@redhat.com>
+# Gerd Hoffmann <kraxel@redhat.com>
#
# This code is licensed under the GPL version 2 or later. See
# the COPYING file in the top-level directory.
@@ -13,7 +14,9 @@
from __future__ import print_function
import os
+import re
import sys
+import socket
import logging
import time
import datetime
@@ -79,8 +82,7 @@ class BaseVM(object):
"-cpu", "max",
"-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22",
"-device", "virtio-net-pci,netdev=vnet",
- "-vnc", "127.0.0.1:0,to=20",
- "-serial", "file:%s" % os.path.join(self._tmpdir, "serial.out")]
+ "-vnc", "127.0.0.1:0,to=20"]
if vcpus and vcpus > 1:
self._args += ["-smp", "%d" % vcpus]
if kvm_available(self.arch):
@@ -162,6 +164,8 @@ class BaseVM(object):
logging.debug("QEMU args: %s", " ".join(args))
qemu_bin = os.environ.get("QEMU", "qemu-system-" + self.arch)
guest = QEMUMachine(binary=qemu_bin, args=args)
+ guest.set_machine('pc')
+ guest.set_console()
try:
guest.launch()
except:
@@ -184,6 +188,82 @@ class BaseVM(object):
raise Exception("Cannot find ssh port from 'info usernet':\n%s" % \
usernet_info)
+ def console_init(self, timeout = 120):
+ vm = self._guest
+ vm.console_socket.settimeout(timeout)
+
+ def console_log(self, text):
+ for line in re.split("[\r\n]", text):
+ # filter out terminal escape sequences
+ line = re.sub("\x1b\[[0-9;?]*[a-zA-Z]", "", line)
+ line = re.sub("\x1b\([0-9;?]*[a-zA-Z]", "", line)
+ # replace unprintable chars
+ line = re.sub("\x1b", "<esc>", line)
+ line = re.sub("[\x00-\x1f]", ".", line)
+ line = re.sub("[\x80-\xff]", ".", line)
+ if line == "":
+ continue
+ # log console line
+ sys.stderr.write("con recv: %s\n" % line)
+
+ def console_wait(self, expect):
+ vm = self._guest
+ output = ""
+ while True:
+ try:
+ chars = vm.console_socket.recv(1)
+ except socket.timeout:
+ sys.stderr.write("console: *** read timeout ***\n")
+ sys.stderr.write("console: waiting for: '%s'\n" % expect)
+ sys.stderr.write("console: line buffer:\n")
+ sys.stderr.write("\n")
+ self.console_log(output.rstrip())
+ sys.stderr.write("\n")
+ raise
+ output += chars.decode("latin1")
+ if expect in output:
+ break
+ if "\r" in output or "\n" in output:
+ lines = re.split("[\r\n]", output)
+ output = lines.pop()
+ if self.debug:
+ self.console_log("\n".join(lines))
+ if self.debug:
+ self.console_log(output)
+
+ def console_send(self, command):
+ vm = self._guest
+ if self.debug:
+ logline = re.sub("\n", "<enter>", command)
+ logline = re.sub("[\x00-\x1f]", ".", logline)
+ sys.stderr.write("con send: %s\n" % logline)
+ for char in list(command):
+ vm.console_socket.send(char.encode("utf-8"))
+ time.sleep(0.01)
+
+ def console_wait_send(self, wait, command):
+ self.console_wait(wait)
+ self.console_send(command)
+
+ def console_ssh_init(self, prompt, user, pw):
+ sshkey_cmd = "echo '%s' > .ssh/authorized_keys\n" % SSH_PUB_KEY.rstrip()
+ self.console_wait_send("login:", "%s\n" % user)
+ self.console_wait_send("Password:", "%s\n" % pw)
+ self.console_wait_send(prompt, "mkdir .ssh\n")
+ self.console_wait_send(prompt, sshkey_cmd)
+ self.console_wait_send(prompt, "chmod 755 .ssh\n")
+ self.console_wait_send(prompt, "chmod 644 .ssh/authorized_keys\n")
+
+ def console_sshd_config(self, prompt):
+ self.console_wait(prompt)
+ self.console_send("echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config\n")
+ for var in self.envvars:
+ self.console_wait(prompt)
+ self.console_send("echo 'AcceptEnv %s' >> /etc/ssh/sshd_config\n" % var)
+
+ def print_step(self, text):
+ sys.stderr.write("### %s ...\n" % text)
+
def wait_ssh(self, seconds=300):
starttime = datetime.datetime.now()
endtime = starttime + datetime.timedelta(seconds=seconds)
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 11/17] tests/vm: openbsd autoinstall, using serial console
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
` (9 preceding siblings ...)
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 10/17] tests/vm: serial console support helpers Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 12/17] tests/vm: freebsd " Alex Bennée
` (5 subsequent siblings)
16 siblings, 0 replies; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé,
Gerd Hoffmann, Brad Smith
From: Gerd Hoffmann <kraxel@redhat.com>
Instead of fetching the prebuilt image from patchew download the install
iso and prepare the image locally. Install to disk, using the serial
console. Create qemu user, configure ssh login. Install packages
needed for qemu builds.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-8-kraxel@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/vm/openbsd | 161 +++++++++++++++++++++++++++++++++++++++++++----
1 file changed, 148 insertions(+), 13 deletions(-)
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index 28c7d25e29..b92c39f89a 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -2,10 +2,11 @@
#
# OpenBSD VM image
#
-# Copyright 2017 Red Hat Inc.
+# Copyright 2017-2019 Red Hat Inc.
#
# Authors:
# Fam Zheng <famz@redhat.com>
+# Gerd Hoffmann <kraxel@redhat.com>
#
# This code is licensed under the GPL version 2 or later. See
# the COPYING file in the top-level directory.
@@ -13,32 +14,166 @@
import os
import sys
+import socket
import subprocess
import basevm
class OpenBSDVM(basevm.BaseVM):
name = "openbsd"
arch = "x86_64"
+
+ link = "https://cdn.openbsd.org/pub/OpenBSD/6.5/amd64/install65.iso"
+ csum = "38d1f8cadd502f1c27bf05c5abde6cc505dd28f3f34f8a941048ff9a54f9f608"
+ size = "20G"
+ pkgs = [
+ # tools
+ "git",
+ "pkgconf",
+ "bzip2", "xz",
+
+ # gnu tools
+ "bash",
+ "gmake",
+ "gsed",
+ "bison",
+
+ # libs: usb
+ "libusb1",
+
+ # libs: crypto
+ "gnutls",
+
+ # libs: images
+ "jpeg",
+ "png",
+
+ # libs: ui
+ "sdl2",
+ "gtk+3",
+ "libxkbcommon",
+ ]
+
BUILD_SCRIPT = """
set -e;
- rm -rf /var/tmp/qemu-test.*
- cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
+ rm -rf /home/qemu/qemu-test.*
+ cd $(mktemp -d /home/qemu/qemu-test.XXXXXX);
+ mkdir src build; cd src;
tar -xf /dev/rsd1c;
- ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts};
- gmake --output-sync -j{jobs} {verbose};
- # XXX: "gmake check" seems to always hang or fail
- #gmake --output-sync -j{jobs} check {verbose};
+ cd ../build
+ ../src/configure --cc=cc --python=python3 {configure_opts};
+ gmake --output-sync -j{jobs} {target} {verbose};
"""
+ poweroff = "halt -p"
def build_image(self, img):
- cimg = self._download_with_cache("http://download.patchew.org/openbsd-6.1-amd64.img.xz",
- sha256sum='8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf')
- img_tmp_xz = img + ".tmp.xz"
+ self.print_step("Downloading install iso")
+ cimg = self._download_with_cache(self.link, sha256sum=self.csum)
img_tmp = img + ".tmp"
- sys.stderr.write("Extracting the image...\n")
- subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
- subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
+ iso = img + ".install.iso"
+
+ self.print_step("Preparing iso and disk image")
+ subprocess.check_call(["cp", "-f", cimg, iso])
+ subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
+ img_tmp, self.size])
+
+ self.print_step("Booting installer")
+ self.boot(img_tmp, extra_args = [
+ "-bios", "pc-bios/bios-256k.bin",
+ "-machine", "graphics=off",
+ "-cdrom", iso
+ ])
+ self.console_init()
+ self.console_wait_send("boot>", "set tty com0\n")
+ self.console_wait_send("boot>", "\n")
+
+ # pre-install configuration
+ self.console_wait_send("(I)nstall", "i\n")
+ self.console_wait_send("Terminal type", "xterm\n")
+ self.console_wait_send("System hostname", "openbsd\n")
+ self.console_wait_send("Which network interface", "vio0\n")
+ self.console_wait_send("IPv4 address", "dhcp\n")
+ self.console_wait_send("IPv6 address", "none\n")
+ self.console_wait_send("Which network interface", "done\n")
+ self.console_wait_send("DNS domain name", "localnet\n")
+ self.console_wait("Password for root account")
+ self.console_send("%s\n" % self.ROOT_PASS)
+ self.console_wait("Password for root account")
+ self.console_send("%s\n" % self.ROOT_PASS)
+ self.console_wait_send("Start sshd(8)", "yes\n")
+ self.console_wait_send("X Window System", "\n")
+ self.console_wait_send("xenodm", "\n")
+ self.console_wait_send("console to com0", "\n")
+ self.console_wait_send("Which speed", "\n")
+
+ self.console_wait("Setup a user")
+ self.console_send("%s\n" % self.GUEST_USER)
+ self.console_wait("Full name")
+ self.console_send("%s\n" % self.GUEST_USER)
+ self.console_wait("Password")
+ self.console_send("%s\n" % self.GUEST_PASS)
+ self.console_wait("Password")
+ self.console_send("%s\n" % self.GUEST_PASS)
+
+ self.console_wait_send("Allow root ssh login", "yes\n")
+ self.console_wait_send("timezone", "UTC\n")
+ self.console_wait_send("root disk", "\n")
+ self.console_wait_send("(W)hole disk", "\n")
+ self.console_wait_send("(A)uto layout", "\n")
+ self.console_wait_send("Location of sets", "cd0\n")
+ self.console_wait_send("Pathname to the sets", "\n")
+ self.console_wait_send("Set name(s)", "\n")
+ self.console_wait_send("without verification", "yes\n")
+
+ self.print_step("Installation started now, this will take a while")
+ self.console_wait_send("Location of sets", "done\n")
+
+ self.console_wait("successfully completed")
+ self.print_step("Installation finished, rebooting")
+ self.console_wait_send("(R)eboot", "reboot\n")
+
+ # setup qemu user
+ prompt = "$"
+ self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS)
+ self.console_wait_send(prompt, "exit\n")
+
+ # setup root user
+ prompt = "openbsd#"
+ self.console_ssh_init(prompt, "root", self.ROOT_PASS)
+ self.console_sshd_config(prompt)
+
+ # setup virtio-blk #1 (tarfile)
+ self.console_wait(prompt)
+ self.console_send("echo 'chmod 666 /dev/rsd1c' >> /etc/rc.local\n")
+
+ # enable w+x for /home
+ self.console_wait(prompt)
+ self.console_send("sed -i -e '/home/s/rw,/rw,wxallowed,/' /etc/fstab\n")
+
+ # tweak datasize limit
+ self.console_wait(prompt)
+ self.console_send("sed -i -e 's/\\(datasize[^=]*\\)=[^:]*/\\1=infinity/' /etc/login.conf\n")
+
+ # use http (be proxy cache friendly)
+ self.console_wait(prompt)
+ self.console_send("sed -i -e 's/https/http/' /etc/installurl\n")
+
+ self.print_step("Configuration finished, rebooting")
+ self.console_wait_send(prompt, "reboot\n")
+ self.console_wait("login:")
+ self.wait_ssh()
+
+ self.print_step("Installing packages")
+ self.ssh_root_check("pkg_add %s\n" % " ".join(self.pkgs))
+
+ # shutdown
+ self.ssh_root(self.poweroff)
+ self.wait()
+
+ if os.path.exists(img):
+ os.remove(img)
os.rename(img_tmp, img)
+ os.remove(iso)
+ self.print_step("All done")
if __name__ == "__main__":
sys.exit(basevm.main(OpenBSDVM))
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 12/17] tests/vm: freebsd autoinstall, using serial console
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
` (10 preceding siblings ...)
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 11/17] tests/vm: openbsd autoinstall, using serial console Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-20 10:39 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 13/17] tests/vm: netbsd " Alex Bennée
` (4 subsequent siblings)
16 siblings, 1 reply; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Alex Bennée, Thomas Huth,
Philippe Mathieu-Daudé,
Gerd Hoffmann
From: Gerd Hoffmann <kraxel@redhat.com>
Instead of fetching the prebuilt image from patchew download the install
iso and prepare the image locally. Install to disk, using the serial
console. Create qemu user, configure ssh login. Install packages
needed for qemu builds.
Note that freebsd package downloads are delivered as non-cachable
content, so I had to configure squid with "ignore-no-store
ignore-private ignore-reload" for pkgmir.geo.freebsd.org to make the
caching actually work.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190617043858.8290-9-kraxel@redhat.com>
---
tests/vm/freebsd | 182 ++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 172 insertions(+), 10 deletions(-)
diff --git a/tests/vm/freebsd b/tests/vm/freebsd
index 091be1a065..2a19461a90 100755
--- a/tests/vm/freebsd
+++ b/tests/vm/freebsd
@@ -2,41 +2,203 @@
#
# FreeBSD VM image
#
-# Copyright 2017 Red Hat Inc.
+# Copyright 2017-2019 Red Hat Inc.
#
# Authors:
# Fam Zheng <famz@redhat.com>
+# Gerd Hoffmann <kraxel@redhat.com>
#
# This code is licensed under the GPL version 2 or later. See
# the COPYING file in the top-level directory.
#
import os
+import re
import sys
+import time
+import socket
import subprocess
import basevm
class FreeBSDVM(basevm.BaseVM):
name = "freebsd"
arch = "x86_64"
+
+ link = "https://download.freebsd.org/ftp/releases/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-disc1.iso.xz"
+ csum = "1d40015bea89d05b8bd13e2ed80c40b522a9ec1abd8e7c8b80954fb485fb99db"
+ size = "20G"
+ pkgs = [
+ # build tools
+ "git",
+ "pkgconf",
+ "bzip2",
+
+ # gnu tools
+ "bash",
+ "gmake",
+ "gsed",
+ "flex", "bison",
+
+ # libs: crypto
+ "gnutls",
+
+ # libs: images
+ "jpeg-turbo",
+ "png",
+
+ # libs: ui
+ "sdl2",
+ "gtk3",
+ "libxkbcommon",
+
+ # libs: opengl
+ "libepoxy",
+ "mesa-libs",
+ ]
+
BUILD_SCRIPT = """
set -e;
- rm -rf /var/tmp/qemu-test.*
- cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
+ rm -rf /home/qemu/qemu-test.*
+ cd $(mktemp -d /home/qemu/qemu-test.XXXXXX);
+ mkdir src build; cd src;
tar -xf /dev/vtbd1;
- ./configure {configure_opts};
+ cd ../build
+ ../src/configure --python=python3.6 {configure_opts};
gmake --output-sync -j{jobs} {target} {verbose};
"""
+ def console_boot_serial(self):
+ self.console_wait_send("Autoboot", "3")
+ self.console_wait_send("OK", "set console=comconsole\n")
+ self.console_wait_send("OK", "boot\n")
+
def build_image(self, img):
- cimg = self._download_with_cache("http://download.patchew.org/freebsd-11.1-amd64.img.xz",
- sha256sum='adcb771549b37bc63826c501f05121a206ed3d9f55f49145908f7e1432d65891')
- img_tmp_xz = img + ".tmp.xz"
+ self.print_step("Downloading install iso")
+ cimg = self._download_with_cache(self.link, sha256sum=self.csum)
img_tmp = img + ".tmp"
- sys.stderr.write("Extracting the image...\n")
- subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
- subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
+ iso = img + ".install.iso"
+ iso_xz = iso + ".xz"
+
+ self.print_step("Preparing iso and disk image")
+ subprocess.check_call(["cp", "-f", cimg, iso_xz])
+ subprocess.check_call(["xz", "-dvf", iso_xz])
+ subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
+ img_tmp, self.size])
+
+ self.print_step("Booting installer")
+ self.boot(img_tmp, extra_args = [
+ "-bios", "pc-bios/bios-256k.bin",
+ "-machine", "graphics=off",
+ "-cdrom", iso
+ ])
+ self.console_init()
+ self.console_boot_serial()
+ self.console_wait_send("Console type", "xterm\n")
+
+ # pre-install configuration
+ self.console_wait_send("Welcome", "\n")
+ self.console_wait_send("Keymap Selection", "\n")
+ self.console_wait_send("Set Hostname", "freebsd\n")
+ self.console_wait_send("Distribution Select", "\n")
+ self.console_wait_send("Partitioning", "\n")
+ self.console_wait_send("Partition", "\n")
+ self.console_wait_send("Scheme", "\n")
+ self.console_wait_send("Editor", "f")
+ self.console_wait_send("Confirmation", "c")
+
+ self.print_step("Installation started now, this will take a while")
+
+ # post-install configuration
+ self.console_wait("New Password:")
+ self.console_send("%s\n" % self.ROOT_PASS)
+ self.console_wait("Retype New Password:")
+ self.console_send("%s\n" % self.ROOT_PASS)
+
+ self.console_wait_send("Network Configuration", "\n")
+ self.console_wait_send("IPv4", "y")
+ self.console_wait_send("DHCP", "y")
+ self.console_wait_send("IPv6", "n")
+ self.console_wait_send("Resolver", "\n")
+
+ self.console_wait_send("Time Zone Selector", "a\n")
+ self.console_wait_send("Confirmation", "y")
+ self.console_wait_send("Time & Date", "\n")
+ self.console_wait_send("Time & Date", "\n")
+
+ self.console_wait_send("System Configuration", "\n")
+ self.console_wait_send("System Hardening", "\n")
+
+ # qemu user
+ self.console_wait_send("Add User Accounts", "y")
+ self.console_wait("Username")
+ self.console_send("%s\n" % self.GUEST_USER)
+ self.console_wait("Full name")
+ self.console_send("%s\n" % self.GUEST_USER)
+ self.console_wait_send("Uid", "\n")
+ self.console_wait_send("Login group", "\n")
+ self.console_wait_send("Login group", "\n")
+ self.console_wait_send("Login class", "\n")
+ self.console_wait_send("Shell", "\n")
+ self.console_wait_send("Home directory", "\n")
+ self.console_wait_send("Home directory perm", "\n")
+ self.console_wait_send("Use password", "\n")
+ self.console_wait_send("Use an empty password", "\n")
+ self.console_wait_send("Use a random password", "\n")
+ self.console_wait("Enter password:")
+ self.console_send("%s\n" % self.GUEST_PASS)
+ self.console_wait("Enter password again:")
+ self.console_send("%s\n" % self.GUEST_PASS)
+ self.console_wait_send("Lock out", "\n")
+ self.console_wait_send("OK", "yes\n")
+ self.console_wait_send("Add another user", "no\n")
+
+ self.console_wait_send("Final Configuration", "\n")
+ self.console_wait_send("Manual Configuration", "\n")
+ self.console_wait_send("Complete", "\n")
+
+ self.print_step("Installation finished, rebooting")
+ self.console_boot_serial()
+
+ # setup qemu user
+ prompt = "$"
+ self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS)
+ self.console_wait_send(prompt, "exit\n")
+
+ # setup root user
+ prompt = "root@freebsd:~ #"
+ self.console_ssh_init(prompt, "root", self.ROOT_PASS)
+ self.console_sshd_config(prompt)
+
+ # setup serial console
+ self.console_wait(prompt)
+ self.console_send("echo 'console=comconsole' >> /boot/loader.conf\n")
+
+ # setup boot delay
+ self.console_wait(prompt)
+ self.console_send("echo 'autoboot_delay=1' >> /boot/loader.conf\n")
+
+ # setup virtio-blk #1 (tarfile)
+ self.console_wait(prompt)
+ self.console_send("echo 'chmod 666 /dev/vtbd1' >> /etc/rc.local\n")
+
+ self.print_step("Configuration finished, rebooting")
+ self.console_wait_send(prompt, "reboot\n")
+ self.console_wait("login:")
+ self.wait_ssh()
+
+ self.print_step("Installing packages")
+ self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
+
+ # shutdown
+ self.ssh_root(self.poweroff)
+ self.console_wait("Uptime:")
+ self.wait()
+
+ if os.path.exists(img):
+ os.remove(img)
os.rename(img_tmp, img)
+ os.remove(iso)
+ self.print_step("All done")
if __name__ == "__main__":
sys.exit(basevm.main(FreeBSDVM))
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 13/17] tests/vm: netbsd autoinstall, using serial console
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
` (11 preceding siblings ...)
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 12/17] tests/vm: freebsd " Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 14/17] tests/vm: fedora " Alex Bennée
` (3 subsequent siblings)
16 siblings, 0 replies; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Thomas Huth, Philippe Mathieu-Daudé,
Kamil Rytarowski, Gerd Hoffmann, Kamil Rytarowski,
Alex Bennée
From: Gerd Hoffmann <kraxel@redhat.com>
Instead of fetching the prebuilt image from patchew download the install
iso and prepare the image locally. Install to disk, using the serial
console. Create qemu user, configure ssh login. Install packages
needed for qemu builds.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Kamil Rytarowski <n54@gmx.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-10-kraxel@redhat.com>
[AJB: added tags]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/vm/netbsd | 189 +++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 179 insertions(+), 10 deletions(-)
diff --git a/tests/vm/netbsd b/tests/vm/netbsd
index ee9eaeab50..be59a2c1da 100755
--- a/tests/vm/netbsd
+++ b/tests/vm/netbsd
@@ -2,10 +2,11 @@
#
# NetBSD VM image
#
-# Copyright 2017 Red Hat Inc.
+# Copyright 2017-2019 Red Hat Inc.
#
# Authors:
# Fam Zheng <famz@redhat.com>
+# Gerd Hoffmann <kraxel@redhat.com>
#
# This code is licensed under the GPL version 2 or later. See
# the COPYING file in the top-level directory.
@@ -13,30 +14,198 @@
import os
import sys
+import time
import subprocess
import basevm
class NetBSDVM(basevm.BaseVM):
name = "netbsd"
arch = "x86_64"
+
+ link = "https://cdn.netbsd.org/pub/NetBSD/NetBSD-8.0/images/NetBSD-8.0-amd64.iso"
+ size = "20G"
+ pkgs = [
+ # tools
+ "git-base",
+ "pkgconf",
+ "xz",
+ "python37",
+
+ # gnu tools
+ "bash",
+ "gmake",
+ "gsed",
+ "flex", "bison",
+
+ # libs: crypto
+ "gnutls",
+
+ # libs: images
+ "jpeg",
+ "png",
+
+ # libs: ui
+ "SDL2",
+ "gtk3+",
+ "libxkbcommon",
+ ]
+
BUILD_SCRIPT = """
set -e;
- rm -rf /var/tmp/qemu-test.*
- cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
+ rm -rf /home/qemu/qemu-test.*
+ cd $(mktemp -d /home/qemu/qemu-test.XXXXXX);
+ mkdir src build; cd src;
tar -xf /dev/rld1a;
- ./configure --python=python2.7 {configure_opts};
+ cd ../build
+ ../src/configure --python=python3.7 --disable-opengl {configure_opts};
gmake --output-sync -j{jobs} {target} {verbose};
"""
+ poweroff = "/sbin/poweroff"
def build_image(self, img):
- cimg = self._download_with_cache("http://download.patchew.org/netbsd-7.1-amd64.img.xz",
- sha256sum='b633d565b0eac3d02015cd0c81440bd8a7a8df8512615ac1ee05d318be015732')
- img_tmp_xz = img + ".tmp.xz"
+ cimg = self._download_with_cache(self.link)
img_tmp = img + ".tmp"
- sys.stderr.write("Extracting the image...\n")
- subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
- subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
+ iso = img + ".install.iso"
+
+ self.print_step("Preparing iso and disk image")
+ subprocess.check_call(["cp", "-f", cimg, iso])
+ subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
+ img_tmp, self.size])
+
+ self.print_step("Booting installer")
+ self.boot(img_tmp, extra_args = [
+ "-bios", "pc-bios/bios-256k.bin",
+ "-machine", "graphics=off",
+ "-cdrom", iso
+ ])
+ self.console_init()
+ self.console_wait("Primary Bootstrap")
+
+ # serial console boot menu output doesn't work for some
+ # reason, so we have to fly blind ...
+ for char in list("5consdev com0\n"):
+ time.sleep(0.2)
+ self.console_send(char)
+ self.console_wait("")
+ self.console_wait_send("> ", "boot\n")
+
+ self.console_wait_send("Terminal type", "xterm\n")
+ self.console_wait_send("a: Installation messages", "a\n")
+ self.console_wait_send("b: US-English", "b\n")
+ self.console_wait_send("a: Install NetBSD", "a\n")
+ self.console_wait("Shall we continue?")
+ self.console_wait_send("b: Yes", "b\n")
+
+ self.console_wait_send("a: ld0", "a\n")
+ self.console_wait_send("a: This is the correct", "a\n")
+ self.console_wait_send("b: Use the entire disk", "b\n")
+ self.console_wait("NetBSD bootcode")
+ self.console_wait_send("a: Yes", "a\n")
+ self.console_wait_send("b: Use existing part", "b\n")
+ self.console_wait_send("x: Partition sizes ok", "x\n")
+ self.console_wait_send("for your NetBSD disk", "\n")
+ self.console_wait("Shall we continue?")
+ self.console_wait_send("b: Yes", "b\n")
+
+ self.console_wait_send("b: Use serial port com0", "b\n")
+ self.console_wait_send("f: Set serial baud rate", "f\n")
+ self.console_wait_send("a: 9600", "a\n")
+ self.console_wait_send("x: Exit", "x\n")
+
+ self.console_wait_send("a: Full installation", "a\n")
+ self.console_wait_send("a: CD-ROM", "a\n")
+
+ self.print_step("Installation started now, this will take a while")
+ self.console_wait_send("Hit enter to continue", "\n")
+
+ self.console_wait_send("d: Change root password", "d\n")
+ self.console_wait_send("a: Yes", "a\n")
+ self.console_wait("New password:")
+ self.console_send("%s\n" % self.ROOT_PASS)
+ self.console_wait("New password:")
+ self.console_send("%s\n" % self.ROOT_PASS)
+ self.console_wait("Retype new password:")
+ self.console_send("%s\n" % self.ROOT_PASS)
+
+ self.console_wait_send("o: Add a user", "o\n")
+ self.console_wait("username")
+ self.console_send("%s\n" % self.GUEST_USER)
+ self.console_wait("to group wheel")
+ self.console_wait_send("a: Yes", "a\n")
+ self.console_wait_send("a: /bin/sh", "a\n")
+ self.console_wait("New password:")
+ self.console_send("%s\n" % self.GUEST_PASS)
+ self.console_wait("New password:")
+ self.console_send("%s\n" % self.GUEST_PASS)
+ self.console_wait("Retype new password:")
+ self.console_send("%s\n" % self.GUEST_PASS)
+
+ self.console_wait_send("a: Configure network", "a\n")
+ self.console_wait_send("a: vioif0", "a\n")
+ self.console_wait_send("Network media type", "\n")
+ self.console_wait("autoconfiguration")
+ self.console_wait_send("a: Yes", "a\n")
+ self.console_wait_send("DNS domain", "localnet\n")
+ self.console_wait("Are they OK?")
+ self.console_wait_send("a: Yes", "a\n")
+ self.console_wait("installed in /etc")
+ self.console_wait_send("a: Yes", "a\n")
+
+ self.console_wait_send("e: Enable install", "e\n")
+ proxy = os.environ.get("http_proxy")
+ if not proxy is None:
+ self.console_wait_send("f: Proxy", "f\n")
+ self.console_wait("Proxy")
+ self.console_send("%s\n" % proxy)
+ self.console_wait_send("x: Install pkgin", "x\n")
+ self.console_init(1200)
+ self.console_wait_send("Hit enter to continue", "\n")
+ self.console_init()
+
+ self.console_wait_send("g: Enable sshd", "g\n")
+ self.console_wait_send("x: Finished conf", "x\n")
+ self.console_wait_send("Hit enter to continue", "\n")
+
+ self.print_step("Installation finished, rebooting")
+ self.console_wait_send("d: Reboot the computer", "d\n")
+
+ # setup qemu user
+ prompt = "localhost$"
+ self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS)
+ self.console_wait_send(prompt, "exit\n")
+
+ # setup root user
+ prompt = "localhost#"
+ self.console_ssh_init(prompt, "root", self.ROOT_PASS)
+ self.console_sshd_config(prompt)
+
+ # setup virtio-blk #1 (tarfile)
+ self.console_wait(prompt)
+ self.console_send("echo 'chmod 666 /dev/rld1a' >> /etc/rc.local\n")
+
+ # turn off mprotect (conflicts with tcg)
+ self.console_wait(prompt)
+ self.console_send("echo security.pax.mprotect.enabled=0 >> /etc/sysctl.conf\n")
+
+ self.print_step("Configuration finished, rebooting")
+ self.console_wait_send(prompt, "reboot\n")
+ self.console_wait("login:")
+ self.wait_ssh()
+
+ self.print_step("Installing packages")
+ self.ssh_root_check("pkgin update\n")
+ self.ssh_root_check("pkgin -y install %s\n" % " ".join(self.pkgs))
+
+ # shutdown
+ self.ssh_root(self.poweroff)
+ self.console_wait("entering state S5")
+ self.wait()
+
+ if os.path.exists(img):
+ os.remove(img)
os.rename(img_tmp, img)
+ os.remove(iso)
+ self.print_step("All done")
if __name__ == "__main__":
sys.exit(basevm.main(NetBSDVM))
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 14/17] tests/vm: fedora autoinstall, using serial console
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
` (12 preceding siblings ...)
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 13/17] tests/vm: netbsd " Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-20 10:36 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 15/17] tests/vm: ubuntu.i386: apt proxy setup Alex Bennée
` (2 subsequent siblings)
16 siblings, 1 reply; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Philippe Mathieu-Daudé,
Thomas Huth, Alex Bennée, Gerd Hoffmann
From: Gerd Hoffmann <kraxel@redhat.com>
Download the install iso and prepare the image locally. Install to
disk, using the serial console. Create qemu user, configure ssh login.
Install packages needed for qemu builds.
Yes, we have docker images for fedora. But for trouble-shooting it
might be helpful to have a vm too. When vm builds fail you can use
it to figure whenever the vm setup or the guest os is the problem.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190617043858.8290-11-kraxel@redhat.com>
---
tests/vm/Makefile.include | 3 +-
tests/vm/basevm.py | 9 +-
tests/vm/fedora | 189 ++++++++++++++++++++++++++++++++++++++
3 files changed, 199 insertions(+), 2 deletions(-)
create mode 100755 tests/vm/fedora
diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
index a61c236b8e..809b80e2e5 100644
--- a/tests/vm/Makefile.include
+++ b/tests/vm/Makefile.include
@@ -2,7 +2,7 @@
.PHONY: vm-build-all vm-clean-all
-IMAGES := ubuntu.i386 freebsd netbsd openbsd centos
+IMAGES := ubuntu.i386 freebsd netbsd openbsd centos fedora
IMAGES_DIR := $(HOME)/.cache/qemu-vm/images
IMAGE_FILES := $(patsubst %, $(IMAGES_DIR)/%.img, $(IMAGES))
@@ -16,6 +16,7 @@ vm-test:
@echo " vm-build-netbsd - Build QEMU in NetBSD VM"
@echo " vm-build-openbsd - Build QEMU in OpenBSD VM"
@echo " vm-build-centos - Build QEMU in CentOS VM, with Docker"
+ @echo " vm-build-fedora - Build QEMU in Fedora VM"
@echo ""
@echo " vm-build-all - Build QEMU in all VMs"
@echo " vm-clean-all - Clean up VM images"
diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 592a344773..3d717da498 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -206,7 +206,7 @@ class BaseVM(object):
# log console line
sys.stderr.write("con recv: %s\n" % line)
- def console_wait(self, expect):
+ def console_wait(self, expect, expectalt = None):
vm = self._guest
output = ""
while True:
@@ -215,6 +215,8 @@ class BaseVM(object):
except socket.timeout:
sys.stderr.write("console: *** read timeout ***\n")
sys.stderr.write("console: waiting for: '%s'\n" % expect)
+ if not expectalt is None:
+ sys.stderr.write("console: waiting for: '%s' (alt)\n" % expectalt)
sys.stderr.write("console: line buffer:\n")
sys.stderr.write("\n")
self.console_log(output.rstrip())
@@ -223,6 +225,8 @@ class BaseVM(object):
output += chars.decode("latin1")
if expect in output:
break
+ if not expectalt is None and expectalt in output:
+ break
if "\r" in output or "\n" in output:
lines = re.split("[\r\n]", output)
output = lines.pop()
@@ -230,6 +234,9 @@ class BaseVM(object):
self.console_log("\n".join(lines))
if self.debug:
self.console_log(output)
+ if not expectalt is None and expectalt in output:
+ return False
+ return True
def console_send(self, command):
vm = self._guest
diff --git a/tests/vm/fedora b/tests/vm/fedora
new file mode 100755
index 0000000000..e8fa5bf0d2
--- /dev/null
+++ b/tests/vm/fedora
@@ -0,0 +1,189 @@
+#!/usr/bin/env python
+#
+# Fedora VM image
+#
+# Copyright 2019 Red Hat Inc.
+#
+# Authors:
+# Gerd Hoffmann <kraxel@redhat.com>
+#
+# This code is licensed under the GPL version 2 or later. See
+# the COPYING file in the top-level directory.
+#
+
+import os
+import re
+import sys
+import time
+import socket
+import subprocess
+import basevm
+
+class FedoraVM(basevm.BaseVM):
+ name = "fedora"
+ arch = "x86_64"
+
+ base = "http://dl.fedoraproject.org/pub/fedora/linux/releases/30/"
+ link = base + "Server/x86_64/iso/Fedora-Server-netinst-x86_64-30-1.2.iso"
+ repo = base + "Server/x86_64/os/"
+ full = base + "Everything/x86_64/os/"
+ csum = "5e4eac4566d8c572bfb3bcf54b7d6c82006ec3c6c882a2c9235c6d3494d7b100"
+ size = "20G"
+ pkgs = [
+ # tools
+ 'git-core',
+ 'flex', 'bison',
+ 'gcc', 'binutils', 'make',
+
+ # perl
+ 'perl-Test-Harness',
+
+ # libs: usb
+ '"pkgconfig(libusb-1.0)"',
+ '"pkgconfig(libusbredirparser-0.5)"',
+
+ # libs: crypto
+ '"pkgconfig(gnutls)"',
+
+ # libs: ui
+ '"pkgconfig(sdl2)"',
+ '"pkgconfig(gtk+-3.0)"',
+ '"pkgconfig(ncursesw)"',
+
+ # libs: audio
+ '"pkgconfig(libpulse)"',
+ '"pkgconfig(alsa)"',
+ ]
+
+ BUILD_SCRIPT = """
+ set -e;
+ rm -rf /home/qemu/qemu-test.*
+ cd $(mktemp -d /home/qemu/qemu-test.XXXXXX);
+ mkdir src build; cd src;
+ tar -xf /dev/vdb;
+ cd ../build
+ ../src/configure --python=python3 {configure_opts};
+ gmake --output-sync -j{jobs} {target} {verbose};
+ """
+
+ def build_image(self, img):
+ self.print_step("Downloading install iso")
+ cimg = self._download_with_cache(self.link, sha256sum=self.csum)
+ img_tmp = img + ".tmp"
+ iso = img + ".install.iso"
+
+ self.print_step("Preparing iso and disk image")
+ subprocess.check_call(["cp", "-f", cimg, iso])
+ subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
+ img_tmp, self.size])
+
+ self.print_step("Booting installer")
+ self.boot(img_tmp, extra_args = [
+ "-bios", "pc-bios/bios-256k.bin",
+ "-machine", "graphics=off",
+ "-cdrom", iso
+ ])
+ self.console_init(300)
+ self.console_wait("installation process.")
+ time.sleep(0.3)
+ self.console_send("\t")
+ time.sleep(0.3)
+ self.console_send(" console=ttyS0")
+ proxy = os.environ.get("http_proxy")
+ if not proxy is None:
+ self.console_send(" proxy=%s" % proxy)
+ self.console_send(" inst.proxy=%s" % proxy)
+ self.console_send(" inst.repo=%s" % self.repo)
+ self.console_send("\n")
+
+ self.console_wait_send("2) Use text mode", "2\n")
+
+ self.console_wait_send("5) [!] Installation Dest", "5\n")
+ self.console_wait_send("1) [x]", "c\n")
+ self.console_wait_send("2) [ ] Use All Space", "2\n")
+ self.console_wait_send("2) [x] Use All Space", "c\n")
+ self.console_wait_send("1) [ ] Standard Part", "1\n")
+ self.console_wait_send("1) [x] Standard Part", "c\n")
+
+ self.console_wait_send("7) [!] Root password", "7\n")
+ self.console_wait("Password:")
+ self.console_send("%s\n" % self.ROOT_PASS)
+ self.console_wait("Password (confirm):")
+ self.console_send("%s\n" % self.ROOT_PASS)
+
+ self.console_wait_send("8) [ ] User creation", "8\n")
+ self.console_wait_send("1) [ ] Create user", "1\n")
+ self.console_wait_send("3) User name", "3\n")
+ self.console_wait_send("ENTER:", "%s\n" % self.GUEST_USER)
+ self.console_wait_send("4) [ ] Use password", "4\n")
+ self.console_wait_send("5) Password", "5\n")
+ self.console_wait("Password:")
+ self.console_send("%s\n" % self.GUEST_PASS)
+ self.console_wait("Password (confirm):")
+ self.console_send("%s\n" % self.GUEST_PASS)
+ self.console_wait_send("7) Groups", "c\n")
+
+ while True:
+ good = self.console_wait("3) [x] Installation",
+ "3) [!] Installation")
+ self.console_send("r\n")
+ if good:
+ break
+ time.sleep(10)
+
+ while True:
+ good = self.console_wait("4) [x] Software",
+ "4) [!] Software")
+ self.console_send("r\n")
+ if good:
+ break
+ time.sleep(10)
+ self.console_send("r\n" % self.GUEST_PASS)
+
+ self.console_wait_send("'b' to begin install", "b\n")
+
+ self.print_step("Installation started now, this will take a while")
+
+ self.console_wait_send("Installation complete", "\n")
+ self.print_step("Installation finished, rebooting")
+
+ # setup qemu user
+ prompt = " ~]$"
+ self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS)
+ self.console_wait_send(prompt, "exit\n")
+
+ # setup root user
+ prompt = " ~]#"
+ self.console_ssh_init(prompt, "root", self.ROOT_PASS)
+ self.console_sshd_config(prompt)
+
+ # setup virtio-blk #1 (tarfile)
+ self.console_wait(prompt)
+ self.console_send("echo 'KERNEL==\"vdb\" MODE=\"666\"' >> %s\n" %
+ "/etc/udev/rules.d/99-qemu.rules")
+
+ self.print_step("Configuration finished, rebooting")
+ self.console_wait_send(prompt, "reboot\n")
+ self.console_wait("login:")
+ self.wait_ssh()
+
+ self.print_step("Installing packages")
+ self.ssh_root_check("rm -vf /etc/yum.repos.d/fedora*.repo\n")
+ self.ssh_root_check("echo '[fedora]' >> /etc/yum.repos.d/qemu.repo\n")
+ self.ssh_root_check("echo 'baseurl=%s' >> /etc/yum.repos.d/qemu.repo\n" % self.full)
+ self.ssh_root_check("echo 'gpgcheck=0' >> /etc/yum.repos.d/qemu.repo\n")
+ self.ssh_root_check("dnf install -y %s\n" % " ".join(self.pkgs))
+
+ # shutdown
+ self.ssh_root(self.poweroff)
+ self.console_wait("sleep state S5")
+ self.wait()
+
+ if os.path.exists(img):
+ os.remove(img)
+ os.rename(img_tmp, img)
+ os.remove(iso)
+ self.print_step("All done")
+
+if __name__ == "__main__":
+ sys.exit(basevm.main(FedoraVM))
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 15/17] tests/vm: ubuntu.i386: apt proxy setup
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
` (13 preceding siblings ...)
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 14/17] tests/vm: fedora " Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 16/17] .travis.yml: default the --disable-system build to --static Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 17/17] .travis.yml: force a brew update for MacOS builds Alex Bennée
16 siblings, 0 replies; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Alex Bennée, Philippe Mathieu-Daudé, Gerd Hoffmann
From: Gerd Hoffmann <kraxel@redhat.com>
Configure apt proxy so package downloads
can be cached and can pass firewalls.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190617043858.8290-12-kraxel@redhat.com>
---
tests/vm/ubuntu.i386 | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
index 3ea459ee20..38f740eabf 100755
--- a/tests/vm/ubuntu.i386
+++ b/tests/vm/ubuntu.i386
@@ -51,6 +51,10 @@ class UbuntuX86VM(basevm.BaseVM):
" ssh-authorized-keys:\n",
" - %s\n" % basevm.SSH_PUB_KEY,
"locale: en_US.UTF-8\n"])
+ proxy = os.environ.get("http_proxy")
+ if not proxy is None:
+ udata.writelines(["apt:\n",
+ " proxy: %s" % proxy])
udata.close()
subprocess.check_call(["genisoimage", "-output", "cloud-init.iso",
"-volid", "cidata", "-joliet", "-rock",
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 16/17] .travis.yml: default the --disable-system build to --static
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
` (14 preceding siblings ...)
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 15/17] tests/vm: ubuntu.i386: apt proxy setup Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-20 10:37 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 17/17] .travis.yml: force a brew update for MacOS builds Alex Bennée
16 siblings, 1 reply; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel; +Cc: Fam Zheng, Philippe Mathieu-Daudé, Alex Bennée
It's fairly common to build qemu-user binaries with --static linking
so the binary can be copied around without libraries. Enable --static
in the default qemu-user build to cover this.
There are other qemu-user builds that use dynamic linking so they
should catch any problems there.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
.travis.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index aeb9b211cd..9750dc905c 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -80,7 +80,7 @@ script:
matrix:
include:
- env:
- - CONFIG="--disable-system"
+ - CONFIG="--disable-system --static"
# we split the system builds as it takes a while to build them all
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [Qemu-devel] [PATCH v1 17/17] .travis.yml: force a brew update for MacOS builds
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
` (15 preceding siblings ...)
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 16/17] .travis.yml: default the --disable-system build to --static Alex Bennée
@ 2019-06-19 19:40 ` Alex Bennée
2019-06-20 10:37 ` Philippe Mathieu-Daudé
16 siblings, 1 reply; 29+ messages in thread
From: Alex Bennée @ 2019-06-19 19:40 UTC (permalink / raw)
To: qemu-devel; +Cc: Fam Zheng, Philippe Mathieu-Daudé, Alex Bennée
It looks like the Travis image package databases are out of date
causing the build to error with:
Error: Your Homebrew is outdated. Please run `brew update`.
Error: Kernel.exit
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
.travis.yml | 1 +
1 file changed, 1 insertion(+)
diff --git a/.travis.yml b/.travis.yml
index 9750dc905c..a891ce2485 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -43,6 +43,7 @@ addons:
- glib
- pixman
- gnu-sed
+ update: true
# The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu
--
2.20.1
^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH v1 02/17] tests/vm: avoid image presence check and removal
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 02/17] tests/vm: avoid image presence check and removal Alex Bennée
@ 2019-06-20 10:29 ` Philippe Mathieu-Daudé
2019-06-20 11:07 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2019-06-20 10:29 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng, Cleber Rosa
On 6/19/19 9:40 PM, Alex Bennée wrote:
> From: Cleber Rosa <crosa@redhat.com>
>
> Python's os.rename() will silently replace an existing file,
> so there's no need for the extra check and removal.
>
> Reference: https://docs.python.org/3/library/os.html#os.rename
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Message-Id: <20190613130718.3763-3-crosa@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/vm/centos | 2 --
> tests/vm/freebsd | 2 --
> tests/vm/netbsd | 2 --
> tests/vm/openbsd | 2 --
> tests/vm/ubuntu.i386 | 2 --
> 5 files changed, 10 deletions(-)
>
> diff --git a/tests/vm/centos b/tests/vm/centos
> index b00b46a8dc..53976f1c4c 100755
> --- a/tests/vm/centos
> +++ b/tests/vm/centos
> @@ -77,8 +77,6 @@ class CentosVM(basevm.BaseVM):
> self.ssh_root_check("systemctl enable docker")
> self.ssh_root("poweroff")
> self.wait()
> - if os.path.exists(img):
> - os.remove(img)
> os.rename(img_tmp, img)
> return 0
>
> diff --git a/tests/vm/freebsd b/tests/vm/freebsd
> index 5575c23a6f..091be1a065 100755
> --- a/tests/vm/freebsd
> +++ b/tests/vm/freebsd
> @@ -36,8 +36,6 @@ class FreeBSDVM(basevm.BaseVM):
> sys.stderr.write("Extracting the image...\n")
> subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
> subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
> - if os.path.exists(img):
> - os.remove(img)
> os.rename(img_tmp, img)
>
> if __name__ == "__main__":
> diff --git a/tests/vm/netbsd b/tests/vm/netbsd
> index d0508f4465..ee9eaeab50 100755
> --- a/tests/vm/netbsd
> +++ b/tests/vm/netbsd
> @@ -36,8 +36,6 @@ class NetBSDVM(basevm.BaseVM):
> sys.stderr.write("Extracting the image...\n")
> subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
> subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
> - if os.path.exists(img):
> - os.remove(img)
> os.rename(img_tmp, img)
>
> if __name__ == "__main__":
> diff --git a/tests/vm/openbsd b/tests/vm/openbsd
> index 87ec982489..28c7d25e29 100755
> --- a/tests/vm/openbsd
> +++ b/tests/vm/openbsd
> @@ -38,8 +38,6 @@ class OpenBSDVM(basevm.BaseVM):
> sys.stderr.write("Extracting the image...\n")
> subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
> subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
> - if os.path.exists(img):
> - os.remove(img)
> os.rename(img_tmp, img)
>
> if __name__ == "__main__":
> diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
> index a22d137e76..12867b193f 100755
> --- a/tests/vm/ubuntu.i386
> +++ b/tests/vm/ubuntu.i386
> @@ -80,8 +80,6 @@ class UbuntuX86VM(basevm.BaseVM):
> self.ssh_root_check("apt-get install -y libfdt-dev flex bison")
> self.ssh_root("poweroff")
> self.wait()
> - if os.path.exists(img):
> - os.remove(img)
> os.rename(img_tmp, img)
> return 0
>
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH v1 03/17] tests/vm: pin ubuntu.i386 image
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 03/17] tests/vm: pin ubuntu.i386 image Alex Bennée
@ 2019-06-20 10:31 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2019-06-20 10:31 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng, Cleber Rosa
On 6/19/19 9:40 PM, Alex Bennée wrote:
> From: Cleber Rosa <crosa@redhat.com>
>
> It's a good practice to always have the same components used in tests.
> According to:
>
> https://cloud-images.ubuntu.com/releases/16.04/
>
> New images are released from time to time, and the "release/"
> directory points to the latest release. Let's pin to the latest
> available version, and while at it, set a hash for verification.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Message-Id: <20190613130718.3763-4-crosa@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/vm/ubuntu.i386 | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
> index 12867b193f..7017e6d388 100755
> --- a/tests/vm/ubuntu.i386
> +++ b/tests/vm/ubuntu.i386
> @@ -61,7 +61,9 @@ class UbuntuX86VM(basevm.BaseVM):
> return os.path.join(cidir, "cloud-init.iso")
>
> def build_image(self, img):
> - cimg = self._download_with_cache("https://cloud-images.ubuntu.com/releases/16.04/release/ubuntu-16.04-server-cloudimg-i386-disk1.img")
> + cimg = self._download_with_cache(
> + "https://cloud-images.ubuntu.com/releases/16.04/release-20190605/ubuntu-16.04-server-cloudimg-i386-disk1.img",
> + sha256sum="e30091144c73483822b7c27193e9d47346dd1064229da577c3fedcf943f7cfcc")
> img_tmp = img + ".tmp"
> subprocess.check_call(["cp", "-f", cimg, img_tmp])
> subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH v1 14/17] tests/vm: fedora autoinstall, using serial console
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 14/17] tests/vm: fedora " Alex Bennée
@ 2019-06-20 10:36 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2019-06-20 10:36 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng, Thomas Huth, Gerd Hoffmann
On 6/19/19 9:40 PM, Alex Bennée wrote:
> From: Gerd Hoffmann <kraxel@redhat.com>
>
> Download the install iso and prepare the image locally. Install to
> disk, using the serial console. Create qemu user, configure ssh login.
> Install packages needed for qemu builds.
>
> Yes, we have docker images for fedora. But for trouble-shooting it
> might be helpful to have a vm too. When vm builds fail you can use
> it to figure whenever the vm setup or the guest os is the problem.
>
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> Tested-by: Thomas Huth <thuth@redhat.com>
> Message-Id: <20190617043858.8290-11-kraxel@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/vm/Makefile.include | 3 +-
> tests/vm/basevm.py | 9 +-
> tests/vm/fedora | 189 ++++++++++++++++++++++++++++++++++++++
> 3 files changed, 199 insertions(+), 2 deletions(-)
> create mode 100755 tests/vm/fedora
>
> diff --git a/tests/vm/Makefile.include b/tests/vm/Makefile.include
> index a61c236b8e..809b80e2e5 100644
> --- a/tests/vm/Makefile.include
> +++ b/tests/vm/Makefile.include
> @@ -2,7 +2,7 @@
>
> .PHONY: vm-build-all vm-clean-all
>
> -IMAGES := ubuntu.i386 freebsd netbsd openbsd centos
> +IMAGES := ubuntu.i386 freebsd netbsd openbsd centos fedora
> IMAGES_DIR := $(HOME)/.cache/qemu-vm/images
> IMAGE_FILES := $(patsubst %, $(IMAGES_DIR)/%.img, $(IMAGES))
>
> @@ -16,6 +16,7 @@ vm-test:
> @echo " vm-build-netbsd - Build QEMU in NetBSD VM"
> @echo " vm-build-openbsd - Build QEMU in OpenBSD VM"
> @echo " vm-build-centos - Build QEMU in CentOS VM, with Docker"
> + @echo " vm-build-fedora - Build QEMU in Fedora VM"
> @echo ""
> @echo " vm-build-all - Build QEMU in all VMs"
> @echo " vm-clean-all - Clean up VM images"
> diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
> index 592a344773..3d717da498 100755
> --- a/tests/vm/basevm.py
> +++ b/tests/vm/basevm.py
> @@ -206,7 +206,7 @@ class BaseVM(object):
> # log console line
> sys.stderr.write("con recv: %s\n" % line)
>
> - def console_wait(self, expect):
> + def console_wait(self, expect, expectalt = None):
> vm = self._guest
> output = ""
> while True:
> @@ -215,6 +215,8 @@ class BaseVM(object):
> except socket.timeout:
> sys.stderr.write("console: *** read timeout ***\n")
> sys.stderr.write("console: waiting for: '%s'\n" % expect)
> + if not expectalt is None:
> + sys.stderr.write("console: waiting for: '%s' (alt)\n" % expectalt)
> sys.stderr.write("console: line buffer:\n")
> sys.stderr.write("\n")
> self.console_log(output.rstrip())
> @@ -223,6 +225,8 @@ class BaseVM(object):
> output += chars.decode("latin1")
> if expect in output:
> break
> + if not expectalt is None and expectalt in output:
> + break
> if "\r" in output or "\n" in output:
> lines = re.split("[\r\n]", output)
> output = lines.pop()
> @@ -230,6 +234,9 @@ class BaseVM(object):
> self.console_log("\n".join(lines))
> if self.debug:
> self.console_log(output)
> + if not expectalt is None and expectalt in output:
> + return False
> + return True
>
> def console_send(self, command):
> vm = self._guest
> diff --git a/tests/vm/fedora b/tests/vm/fedora
> new file mode 100755
> index 0000000000..e8fa5bf0d2
> --- /dev/null
> +++ b/tests/vm/fedora
> @@ -0,0 +1,189 @@
> +#!/usr/bin/env python
> +#
> +# Fedora VM image
> +#
> +# Copyright 2019 Red Hat Inc.
> +#
> +# Authors:
> +# Gerd Hoffmann <kraxel@redhat.com>
> +#
> +# This code is licensed under the GPL version 2 or later. See
> +# the COPYING file in the top-level directory.
> +#
> +
> +import os
> +import re
> +import sys
> +import time
> +import socket
> +import subprocess
> +import basevm
> +
> +class FedoraVM(basevm.BaseVM):
> + name = "fedora"
> + arch = "x86_64"
> +
> + base = "http://dl.fedoraproject.org/pub/fedora/linux/releases/30/"
> + link = base + "Server/x86_64/iso/Fedora-Server-netinst-x86_64-30-1.2.iso"
> + repo = base + "Server/x86_64/os/"
> + full = base + "Everything/x86_64/os/"
> + csum = "5e4eac4566d8c572bfb3bcf54b7d6c82006ec3c6c882a2c9235c6d3494d7b100"
> + size = "20G"
> + pkgs = [
> + # tools
> + 'git-core',
> + 'flex', 'bison',
> + 'gcc', 'binutils', 'make',
> +
> + # perl
> + 'perl-Test-Harness',
> +
> + # libs: usb
> + '"pkgconfig(libusb-1.0)"',
> + '"pkgconfig(libusbredirparser-0.5)"',
> +
> + # libs: crypto
> + '"pkgconfig(gnutls)"',
> +
> + # libs: ui
> + '"pkgconfig(sdl2)"',
> + '"pkgconfig(gtk+-3.0)"',
> + '"pkgconfig(ncursesw)"',
> +
> + # libs: audio
> + '"pkgconfig(libpulse)"',
> + '"pkgconfig(alsa)"',
> + ]
> +
> + BUILD_SCRIPT = """
> + set -e;
> + rm -rf /home/qemu/qemu-test.*
> + cd $(mktemp -d /home/qemu/qemu-test.XXXXXX);
> + mkdir src build; cd src;
> + tar -xf /dev/vdb;
> + cd ../build
> + ../src/configure --python=python3 {configure_opts};
> + gmake --output-sync -j{jobs} {target} {verbose};
> + """
> +
> + def build_image(self, img):
> + self.print_step("Downloading install iso")
> + cimg = self._download_with_cache(self.link, sha256sum=self.csum)
> + img_tmp = img + ".tmp"
> + iso = img + ".install.iso"
> +
> + self.print_step("Preparing iso and disk image")
> + subprocess.check_call(["cp", "-f", cimg, iso])
> + subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
> + img_tmp, self.size])
> +
> + self.print_step("Booting installer")
> + self.boot(img_tmp, extra_args = [
> + "-bios", "pc-bios/bios-256k.bin",
> + "-machine", "graphics=off",
> + "-cdrom", iso
> + ])
> + self.console_init(300)
> + self.console_wait("installation process.")
> + time.sleep(0.3)
> + self.console_send("\t")
> + time.sleep(0.3)
> + self.console_send(" console=ttyS0")
> + proxy = os.environ.get("http_proxy")
> + if not proxy is None:
> + self.console_send(" proxy=%s" % proxy)
> + self.console_send(" inst.proxy=%s" % proxy)
> + self.console_send(" inst.repo=%s" % self.repo)
> + self.console_send("\n")
> +
> + self.console_wait_send("2) Use text mode", "2\n")
> +
> + self.console_wait_send("5) [!] Installation Dest", "5\n")
> + self.console_wait_send("1) [x]", "c\n")
> + self.console_wait_send("2) [ ] Use All Space", "2\n")
> + self.console_wait_send("2) [x] Use All Space", "c\n")
> + self.console_wait_send("1) [ ] Standard Part", "1\n")
> + self.console_wait_send("1) [x] Standard Part", "c\n")
> +
> + self.console_wait_send("7) [!] Root password", "7\n")
> + self.console_wait("Password:")
> + self.console_send("%s\n" % self.ROOT_PASS)
> + self.console_wait("Password (confirm):")
> + self.console_send("%s\n" % self.ROOT_PASS)
> +
> + self.console_wait_send("8) [ ] User creation", "8\n")
> + self.console_wait_send("1) [ ] Create user", "1\n")
> + self.console_wait_send("3) User name", "3\n")
> + self.console_wait_send("ENTER:", "%s\n" % self.GUEST_USER)
> + self.console_wait_send("4) [ ] Use password", "4\n")
> + self.console_wait_send("5) Password", "5\n")
> + self.console_wait("Password:")
> + self.console_send("%s\n" % self.GUEST_PASS)
> + self.console_wait("Password (confirm):")
> + self.console_send("%s\n" % self.GUEST_PASS)
> + self.console_wait_send("7) Groups", "c\n")
> +
> + while True:
> + good = self.console_wait("3) [x] Installation",
> + "3) [!] Installation")
> + self.console_send("r\n")
> + if good:
> + break
> + time.sleep(10)
> +
> + while True:
> + good = self.console_wait("4) [x] Software",
> + "4) [!] Software")
> + self.console_send("r\n")
> + if good:
> + break
> + time.sleep(10)
> + self.console_send("r\n" % self.GUEST_PASS)
> +
> + self.console_wait_send("'b' to begin install", "b\n")
> +
> + self.print_step("Installation started now, this will take a while")
> +
> + self.console_wait_send("Installation complete", "\n")
> + self.print_step("Installation finished, rebooting")
> +
> + # setup qemu user
> + prompt = " ~]$"
> + self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS)
> + self.console_wait_send(prompt, "exit\n")
> +
> + # setup root user
> + prompt = " ~]#"
> + self.console_ssh_init(prompt, "root", self.ROOT_PASS)
> + self.console_sshd_config(prompt)
> +
> + # setup virtio-blk #1 (tarfile)
> + self.console_wait(prompt)
> + self.console_send("echo 'KERNEL==\"vdb\" MODE=\"666\"' >> %s\n" %
> + "/etc/udev/rules.d/99-qemu.rules")
> +
> + self.print_step("Configuration finished, rebooting")
> + self.console_wait_send(prompt, "reboot\n")
> + self.console_wait("login:")
> + self.wait_ssh()
> +
> + self.print_step("Installing packages")
> + self.ssh_root_check("rm -vf /etc/yum.repos.d/fedora*.repo\n")
> + self.ssh_root_check("echo '[fedora]' >> /etc/yum.repos.d/qemu.repo\n")
> + self.ssh_root_check("echo 'baseurl=%s' >> /etc/yum.repos.d/qemu.repo\n" % self.full)
> + self.ssh_root_check("echo 'gpgcheck=0' >> /etc/yum.repos.d/qemu.repo\n")
> + self.ssh_root_check("dnf install -y %s\n" % " ".join(self.pkgs))
> +
> + # shutdown
> + self.ssh_root(self.poweroff)
> + self.console_wait("sleep state S5")
> + self.wait()
> +
> + if os.path.exists(img):
> + os.remove(img)
> + os.rename(img_tmp, img)
> + os.remove(iso)
> + self.print_step("All done")
> +
> +if __name__ == "__main__":
> + sys.exit(basevm.main(FedoraVM))
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH v1 16/17] .travis.yml: default the --disable-system build to --static
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 16/17] .travis.yml: default the --disable-system build to --static Alex Bennée
@ 2019-06-20 10:37 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2019-06-20 10:37 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng
On 6/19/19 9:40 PM, Alex Bennée wrote:
> It's fairly common to build qemu-user binaries with --static linking
> so the binary can be copied around without libraries. Enable --static
> in the default qemu-user build to cover this.
>
> There are other qemu-user builds that use dynamic linking so they
> should catch any problems there.
>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> .travis.yml | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index aeb9b211cd..9750dc905c 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -80,7 +80,7 @@ script:
> matrix:
> include:
> - env:
> - - CONFIG="--disable-system"
> + - CONFIG="--disable-system --static"
>
>
> # we split the system builds as it takes a while to build them all
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH v1 17/17] .travis.yml: force a brew update for MacOS builds
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 17/17] .travis.yml: force a brew update for MacOS builds Alex Bennée
@ 2019-06-20 10:37 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2019-06-20 10:37 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng
On 6/19/19 9:40 PM, Alex Bennée wrote:
> It looks like the Travis image package databases are out of date
> causing the build to error with:
>
> Error: Your Homebrew is outdated. Please run `brew update`.
> Error: Kernel.exit
>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> .travis.yml | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/.travis.yml b/.travis.yml
> index 9750dc905c..a891ce2485 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -43,6 +43,7 @@ addons:
> - glib
> - pixman
> - gnu-sed
> + update: true
>
>
> # The channel name "irc.oftc.net#qemu" is encrypted against qemu/qemu
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH v1 05/17] tests/vm: send proxy environment variables over ssh
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 05/17] tests/vm: send proxy environment variables over ssh Alex Bennée
@ 2019-06-20 10:38 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2019-06-20 10:38 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng, Thomas Huth, Gerd Hoffmann
On 6/19/19 9:40 PM, Alex Bennée wrote:
> From: Gerd Hoffmann <kraxel@redhat.com>
>
> Packages are fetched via proxy that way, if configured on the host.
> That might be required to pass firewalls, and it allows to route
> package downloads through a caching proxy server.
>
> Needs AcceptEnv setup in sshd_config on the guest side to work.
>
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Tested-by: Thomas Huth <thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Message-Id: <20190617043858.8290-2-kraxel@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/vm/basevm.py | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
> index 4847549592..5e30bac661 100755
> --- a/tests/vm/basevm.py
> +++ b/tests/vm/basevm.py
> @@ -38,6 +38,13 @@ class BaseVM(object):
> GUEST_PASS = "qemupass"
> ROOT_PASS = "qemupass"
>
> + envvars = [
> + "https_proxy",
> + "http_proxy",
> + "ftp_proxy",
> + "no_proxy",
> + ]
> +
> # The script to run in the guest that builds QEMU
> BUILD_SCRIPT = ""
> # The guest name, to be overridden by subclasses
> @@ -106,6 +113,8 @@ class BaseVM(object):
> "-o", "UserKnownHostsFile=" + os.devnull,
> "-o", "ConnectTimeout=1",
> "-p", self.ssh_port, "-i", self._ssh_key_file]
> + for var in self.envvars:
> + ssh_cmd += ['-o', "SendEnv=%s" % var ]
> if interactive:
> ssh_cmd += ['-t']
> assert not isinstance(cmd, str)
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH v1 12/17] tests/vm: freebsd autoinstall, using serial console
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 12/17] tests/vm: freebsd " Alex Bennée
@ 2019-06-20 10:39 ` Philippe Mathieu-Daudé
2019-06-20 11:01 ` Alex Bennée
0 siblings, 1 reply; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2019-06-20 10:39 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng, Thomas Huth, Gerd Hoffmann
On 6/19/19 9:40 PM, Alex Bennée wrote:
> From: Gerd Hoffmann <kraxel@redhat.com>
>
> Instead of fetching the prebuilt image from patchew download the install
> iso and prepare the image locally. Install to disk, using the serial
> console. Create qemu user, configure ssh login. Install packages
> needed for qemu builds.
>
> Note that freebsd package downloads are delivered as non-cachable
> content, so I had to configure squid with "ignore-no-store
> ignore-private ignore-reload" for pkgmir.geo.freebsd.org to make the
> caching actually work.
>
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Tested-by: Thomas Huth <thuth@redhat.com>
> Message-Id: <20190617043858.8290-9-kraxel@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/vm/freebsd | 182 ++++++++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 172 insertions(+), 10 deletions(-)
>
> diff --git a/tests/vm/freebsd b/tests/vm/freebsd
> index 091be1a065..2a19461a90 100755
> --- a/tests/vm/freebsd
> +++ b/tests/vm/freebsd
> @@ -2,41 +2,203 @@
> #
> # FreeBSD VM image
> #
> -# Copyright 2017 Red Hat Inc.
> +# Copyright 2017-2019 Red Hat Inc.
> #
> # Authors:
> # Fam Zheng <famz@redhat.com>
> +# Gerd Hoffmann <kraxel@redhat.com>
> #
> # This code is licensed under the GPL version 2 or later. See
> # the COPYING file in the top-level directory.
> #
>
> import os
> +import re
> import sys
> +import time
> +import socket
> import subprocess
> import basevm
>
> class FreeBSDVM(basevm.BaseVM):
> name = "freebsd"
> arch = "x86_64"
> +
> + link = "https://download.freebsd.org/ftp/releases/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-disc1.iso.xz"
> + csum = "1d40015bea89d05b8bd13e2ed80c40b522a9ec1abd8e7c8b80954fb485fb99db"
> + size = "20G"
> + pkgs = [
> + # build tools
> + "git",
> + "pkgconf",
> + "bzip2",
> +
> + # gnu tools
> + "bash",
> + "gmake",
> + "gsed",
> + "flex", "bison",
> +
> + # libs: crypto
> + "gnutls",
> +
> + # libs: images
> + "jpeg-turbo",
> + "png",
> +
> + # libs: ui
> + "sdl2",
> + "gtk3",
> + "libxkbcommon",
> +
> + # libs: opengl
> + "libepoxy",
> + "mesa-libs",
> + ]
> +
> BUILD_SCRIPT = """
> set -e;
> - rm -rf /var/tmp/qemu-test.*
> - cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
> + rm -rf /home/qemu/qemu-test.*
> + cd $(mktemp -d /home/qemu/qemu-test.XXXXXX);
> + mkdir src build; cd src;
> tar -xf /dev/vtbd1;
> - ./configure {configure_opts};
> + cd ../build
> + ../src/configure --python=python3.6 {configure_opts};
> gmake --output-sync -j{jobs} {target} {verbose};
> """
>
> + def console_boot_serial(self):
> + self.console_wait_send("Autoboot", "3")
> + self.console_wait_send("OK", "set console=comconsole\n")
> + self.console_wait_send("OK", "boot\n")
> +
> def build_image(self, img):
> - cimg = self._download_with_cache("http://download.patchew.org/freebsd-11.1-amd64.img.xz",
> - sha256sum='adcb771549b37bc63826c501f05121a206ed3d9f55f49145908f7e1432d65891')
> - img_tmp_xz = img + ".tmp.xz"
> + self.print_step("Downloading install iso")
> + cimg = self._download_with_cache(self.link, sha256sum=self.csum)
> img_tmp = img + ".tmp"
> - sys.stderr.write("Extracting the image...\n")
> - subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
> - subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
> + iso = img + ".install.iso"
> + iso_xz = iso + ".xz"
> +
> + self.print_step("Preparing iso and disk image")
> + subprocess.check_call(["cp", "-f", cimg, iso_xz])
> + subprocess.check_call(["xz", "-dvf", iso_xz])
> + subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
> + img_tmp, self.size])
> +
> + self.print_step("Booting installer")
> + self.boot(img_tmp, extra_args = [
> + "-bios", "pc-bios/bios-256k.bin",
> + "-machine", "graphics=off",
> + "-cdrom", iso
> + ])
> + self.console_init()
> + self.console_boot_serial()
> + self.console_wait_send("Console type", "xterm\n")
> +
> + # pre-install configuration
> + self.console_wait_send("Welcome", "\n")
> + self.console_wait_send("Keymap Selection", "\n")
> + self.console_wait_send("Set Hostname", "freebsd\n")
> + self.console_wait_send("Distribution Select", "\n")
> + self.console_wait_send("Partitioning", "\n")
> + self.console_wait_send("Partition", "\n")
> + self.console_wait_send("Scheme", "\n")
> + self.console_wait_send("Editor", "f")
> + self.console_wait_send("Confirmation", "c")
> +
> + self.print_step("Installation started now, this will take a while")
> +
> + # post-install configuration
> + self.console_wait("New Password:")
> + self.console_send("%s\n" % self.ROOT_PASS)
> + self.console_wait("Retype New Password:")
> + self.console_send("%s\n" % self.ROOT_PASS)
> +
> + self.console_wait_send("Network Configuration", "\n")
> + self.console_wait_send("IPv4", "y")
> + self.console_wait_send("DHCP", "y")
> + self.console_wait_send("IPv6", "n")
> + self.console_wait_send("Resolver", "\n")
> +
> + self.console_wait_send("Time Zone Selector", "a\n")
> + self.console_wait_send("Confirmation", "y")
> + self.console_wait_send("Time & Date", "\n")
> + self.console_wait_send("Time & Date", "\n")
> +
> + self.console_wait_send("System Configuration", "\n")
> + self.console_wait_send("System Hardening", "\n")
> +
> + # qemu user
> + self.console_wait_send("Add User Accounts", "y")
> + self.console_wait("Username")
> + self.console_send("%s\n" % self.GUEST_USER)
> + self.console_wait("Full name")
> + self.console_send("%s\n" % self.GUEST_USER)
> + self.console_wait_send("Uid", "\n")
> + self.console_wait_send("Login group", "\n")
> + self.console_wait_send("Login group", "\n")
> + self.console_wait_send("Login class", "\n")
> + self.console_wait_send("Shell", "\n")
> + self.console_wait_send("Home directory", "\n")
> + self.console_wait_send("Home directory perm", "\n")
> + self.console_wait_send("Use password", "\n")
> + self.console_wait_send("Use an empty password", "\n")
> + self.console_wait_send("Use a random password", "\n")
> + self.console_wait("Enter password:")
> + self.console_send("%s\n" % self.GUEST_PASS)
> + self.console_wait("Enter password again:")
> + self.console_send("%s\n" % self.GUEST_PASS)
> + self.console_wait_send("Lock out", "\n")
> + self.console_wait_send("OK", "yes\n")
> + self.console_wait_send("Add another user", "no\n")
> +
> + self.console_wait_send("Final Configuration", "\n")
> + self.console_wait_send("Manual Configuration", "\n")
> + self.console_wait_send("Complete", "\n")
> +
> + self.print_step("Installation finished, rebooting")
> + self.console_boot_serial()
> +
> + # setup qemu user
> + prompt = "$"
> + self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS)
> + self.console_wait_send(prompt, "exit\n")
> +
> + # setup root user
> + prompt = "root@freebsd:~ #"
> + self.console_ssh_init(prompt, "root", self.ROOT_PASS)
> + self.console_sshd_config(prompt)
> +
> + # setup serial console
> + self.console_wait(prompt)
> + self.console_send("echo 'console=comconsole' >> /boot/loader.conf\n")
> +
> + # setup boot delay
> + self.console_wait(prompt)
> + self.console_send("echo 'autoboot_delay=1' >> /boot/loader.conf\n")
> +
> + # setup virtio-blk #1 (tarfile)
> + self.console_wait(prompt)
> + self.console_send("echo 'chmod 666 /dev/vtbd1' >> /etc/rc.local\n")
> +
> + self.print_step("Configuration finished, rebooting")
> + self.console_wait_send(prompt, "reboot\n")
> + self.console_wait("login:")
> + self.wait_ssh()
> +
> + self.print_step("Installing packages")
> + self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
> +
> + # shutdown
> + self.ssh_root(self.poweroff)
> + self.console_wait("Uptime:")
> + self.wait()
> +
> + if os.path.exists(img):
> + os.remove(img)
> os.rename(img_tmp, img)
> + os.remove(iso)
> + self.print_step("All done")
>
> if __name__ == "__main__":
> sys.exit(basevm.main(FreeBSDVM))
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH v1 12/17] tests/vm: freebsd autoinstall, using serial console
2019-06-20 10:39 ` Philippe Mathieu-Daudé
@ 2019-06-20 11:01 ` Alex Bennée
0 siblings, 0 replies; 29+ messages in thread
From: Alex Bennée @ 2019-06-20 11:01 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Fam Zheng, Thomas Huth, qemu-devel, Gerd Hoffmann
Philippe Mathieu-Daudé <philmd@redhat.com> writes:
> On 6/19/19 9:40 PM, Alex Bennée wrote:
>> From: Gerd Hoffmann <kraxel@redhat.com>
>>
>> Instead of fetching the prebuilt image from patchew download the install
>> iso and prepare the image locally. Install to disk, using the serial
>> console. Create qemu user, configure ssh login. Install packages
>> needed for qemu builds.
>>
>> Note that freebsd package downloads are delivered as non-cachable
>> content, so I had to configure squid with "ignore-no-store
>> ignore-private ignore-reload" for pkgmir.geo.freebsd.org to make the
>> caching actually work.
>>
>> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
>> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>> Tested-by: Thomas Huth <thuth@redhat.com>
>> Message-Id: <20190617043858.8290-9-kraxel@redhat.com>
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
So I do have a check for this in my PR scripts. I guess I should
replicate it in my patch series workflow.
>
>> ---
>> tests/vm/freebsd | 182 ++++++++++++++++++++++++++++++++++++++++++++---
>> 1 file changed, 172 insertions(+), 10 deletions(-)
>>
>> diff --git a/tests/vm/freebsd b/tests/vm/freebsd
>> index 091be1a065..2a19461a90 100755
>> --- a/tests/vm/freebsd
>> +++ b/tests/vm/freebsd
>> @@ -2,41 +2,203 @@
>> #
>> # FreeBSD VM image
>> #
>> -# Copyright 2017 Red Hat Inc.
>> +# Copyright 2017-2019 Red Hat Inc.
>> #
>> # Authors:
>> # Fam Zheng <famz@redhat.com>
>> +# Gerd Hoffmann <kraxel@redhat.com>
>> #
>> # This code is licensed under the GPL version 2 or later. See
>> # the COPYING file in the top-level directory.
>> #
>>
>> import os
>> +import re
>> import sys
>> +import time
>> +import socket
>> import subprocess
>> import basevm
>>
>> class FreeBSDVM(basevm.BaseVM):
>> name = "freebsd"
>> arch = "x86_64"
>> +
>> + link = "https://download.freebsd.org/ftp/releases/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-disc1.iso.xz"
>> + csum = "1d40015bea89d05b8bd13e2ed80c40b522a9ec1abd8e7c8b80954fb485fb99db"
>> + size = "20G"
>> + pkgs = [
>> + # build tools
>> + "git",
>> + "pkgconf",
>> + "bzip2",
>> +
>> + # gnu tools
>> + "bash",
>> + "gmake",
>> + "gsed",
>> + "flex", "bison",
>> +
>> + # libs: crypto
>> + "gnutls",
>> +
>> + # libs: images
>> + "jpeg-turbo",
>> + "png",
>> +
>> + # libs: ui
>> + "sdl2",
>> + "gtk3",
>> + "libxkbcommon",
>> +
>> + # libs: opengl
>> + "libepoxy",
>> + "mesa-libs",
>> + ]
>> +
>> BUILD_SCRIPT = """
>> set -e;
>> - rm -rf /var/tmp/qemu-test.*
>> - cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
>> + rm -rf /home/qemu/qemu-test.*
>> + cd $(mktemp -d /home/qemu/qemu-test.XXXXXX);
>> + mkdir src build; cd src;
>> tar -xf /dev/vtbd1;
>> - ./configure {configure_opts};
>> + cd ../build
>> + ../src/configure --python=python3.6 {configure_opts};
>> gmake --output-sync -j{jobs} {target} {verbose};
>> """
>>
>> + def console_boot_serial(self):
>> + self.console_wait_send("Autoboot", "3")
>> + self.console_wait_send("OK", "set console=comconsole\n")
>> + self.console_wait_send("OK", "boot\n")
>> +
>> def build_image(self, img):
>> - cimg = self._download_with_cache("http://download.patchew.org/freebsd-11.1-amd64.img.xz",
>> - sha256sum='adcb771549b37bc63826c501f05121a206ed3d9f55f49145908f7e1432d65891')
>> - img_tmp_xz = img + ".tmp.xz"
>> + self.print_step("Downloading install iso")
>> + cimg = self._download_with_cache(self.link, sha256sum=self.csum)
>> img_tmp = img + ".tmp"
>> - sys.stderr.write("Extracting the image...\n")
>> - subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
>> - subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
>> + iso = img + ".install.iso"
>> + iso_xz = iso + ".xz"
>> +
>> + self.print_step("Preparing iso and disk image")
>> + subprocess.check_call(["cp", "-f", cimg, iso_xz])
>> + subprocess.check_call(["xz", "-dvf", iso_xz])
>> + subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
>> + img_tmp, self.size])
>> +
>> + self.print_step("Booting installer")
>> + self.boot(img_tmp, extra_args = [
>> + "-bios", "pc-bios/bios-256k.bin",
>> + "-machine", "graphics=off",
>> + "-cdrom", iso
>> + ])
>> + self.console_init()
>> + self.console_boot_serial()
>> + self.console_wait_send("Console type", "xterm\n")
>> +
>> + # pre-install configuration
>> + self.console_wait_send("Welcome", "\n")
>> + self.console_wait_send("Keymap Selection", "\n")
>> + self.console_wait_send("Set Hostname", "freebsd\n")
>> + self.console_wait_send("Distribution Select", "\n")
>> + self.console_wait_send("Partitioning", "\n")
>> + self.console_wait_send("Partition", "\n")
>> + self.console_wait_send("Scheme", "\n")
>> + self.console_wait_send("Editor", "f")
>> + self.console_wait_send("Confirmation", "c")
>> +
>> + self.print_step("Installation started now, this will take a while")
>> +
>> + # post-install configuration
>> + self.console_wait("New Password:")
>> + self.console_send("%s\n" % self.ROOT_PASS)
>> + self.console_wait("Retype New Password:")
>> + self.console_send("%s\n" % self.ROOT_PASS)
>> +
>> + self.console_wait_send("Network Configuration", "\n")
>> + self.console_wait_send("IPv4", "y")
>> + self.console_wait_send("DHCP", "y")
>> + self.console_wait_send("IPv6", "n")
>> + self.console_wait_send("Resolver", "\n")
>> +
>> + self.console_wait_send("Time Zone Selector", "a\n")
>> + self.console_wait_send("Confirmation", "y")
>> + self.console_wait_send("Time & Date", "\n")
>> + self.console_wait_send("Time & Date", "\n")
>> +
>> + self.console_wait_send("System Configuration", "\n")
>> + self.console_wait_send("System Hardening", "\n")
>> +
>> + # qemu user
>> + self.console_wait_send("Add User Accounts", "y")
>> + self.console_wait("Username")
>> + self.console_send("%s\n" % self.GUEST_USER)
>> + self.console_wait("Full name")
>> + self.console_send("%s\n" % self.GUEST_USER)
>> + self.console_wait_send("Uid", "\n")
>> + self.console_wait_send("Login group", "\n")
>> + self.console_wait_send("Login group", "\n")
>> + self.console_wait_send("Login class", "\n")
>> + self.console_wait_send("Shell", "\n")
>> + self.console_wait_send("Home directory", "\n")
>> + self.console_wait_send("Home directory perm", "\n")
>> + self.console_wait_send("Use password", "\n")
>> + self.console_wait_send("Use an empty password", "\n")
>> + self.console_wait_send("Use a random password", "\n")
>> + self.console_wait("Enter password:")
>> + self.console_send("%s\n" % self.GUEST_PASS)
>> + self.console_wait("Enter password again:")
>> + self.console_send("%s\n" % self.GUEST_PASS)
>> + self.console_wait_send("Lock out", "\n")
>> + self.console_wait_send("OK", "yes\n")
>> + self.console_wait_send("Add another user", "no\n")
>> +
>> + self.console_wait_send("Final Configuration", "\n")
>> + self.console_wait_send("Manual Configuration", "\n")
>> + self.console_wait_send("Complete", "\n")
>> +
>> + self.print_step("Installation finished, rebooting")
>> + self.console_boot_serial()
>> +
>> + # setup qemu user
>> + prompt = "$"
>> + self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS)
>> + self.console_wait_send(prompt, "exit\n")
>> +
>> + # setup root user
>> + prompt = "root@freebsd:~ #"
>> + self.console_ssh_init(prompt, "root", self.ROOT_PASS)
>> + self.console_sshd_config(prompt)
>> +
>> + # setup serial console
>> + self.console_wait(prompt)
>> + self.console_send("echo 'console=comconsole' >> /boot/loader.conf\n")
>> +
>> + # setup boot delay
>> + self.console_wait(prompt)
>> + self.console_send("echo 'autoboot_delay=1' >> /boot/loader.conf\n")
>> +
>> + # setup virtio-blk #1 (tarfile)
>> + self.console_wait(prompt)
>> + self.console_send("echo 'chmod 666 /dev/vtbd1' >> /etc/rc.local\n")
>> +
>> + self.print_step("Configuration finished, rebooting")
>> + self.console_wait_send(prompt, "reboot\n")
>> + self.console_wait("login:")
>> + self.wait_ssh()
>> +
>> + self.print_step("Installing packages")
>> + self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
>> +
>> + # shutdown
>> + self.ssh_root(self.poweroff)
>> + self.console_wait("Uptime:")
>> + self.wait()
>> +
>> + if os.path.exists(img):
>> + os.remove(img)
>> os.rename(img_tmp, img)
>> + os.remove(iso)
>> + self.print_step("All done")
>>
>> if __name__ == "__main__":
>> sys.exit(basevm.main(FreeBSDVM))
>>
--
Alex Bennée
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH v1 04/17] tests/vm: add source repos on ubuntu.i386
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 04/17] tests/vm: add source repos on ubuntu.i386 Alex Bennée
@ 2019-06-20 11:06 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2019-06-20 11:06 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng, Cleber Rosa
On 6/19/19 9:40 PM, Alex Bennée wrote:
> From: Cleber Rosa <crosa@redhat.com>
>
> Possibly because of different behavior on the newly update
> cloud-image, trying to run 'apt-get build-dep' results in:
>
> E: You must put some 'source' URIs in your sources.list
>
> This enables all source repos (even though some are not
> needed) for simplicity sake.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Message-Id: <20190613130718.3763-5-crosa@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/vm/ubuntu.i386 | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386
> index 7017e6d388..3ea459ee20 100755
> --- a/tests/vm/ubuntu.i386
> +++ b/tests/vm/ubuntu.i386
> @@ -77,6 +77,7 @@ class UbuntuX86VM(basevm.BaseVM):
> time.sleep(5)
> self.wait_ssh()
> # The previous update sometimes doesn't survive a reboot, so do it again
> + self.ssh_root_check("sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sources.list")
> self.ssh_root_check("apt-get update")
> self.ssh_root_check("apt-get build-dep -y qemu")
> self.ssh_root_check("apt-get install -y libfdt-dev flex bison")
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH v1 02/17] tests/vm: avoid image presence check and removal
2019-06-20 10:29 ` Philippe Mathieu-Daudé
@ 2019-06-20 11:07 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2019-06-20 11:07 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng, Cleber Rosa
On 6/20/19 12:29 PM, Philippe Mathieu-Daudé wrote:
> On 6/19/19 9:40 PM, Alex Bennée wrote:
>> From: Cleber Rosa <crosa@redhat.com>
>>
>> Python's os.rename() will silently replace an existing file,
>> so there's no need for the extra check and removal.
>>
>> Reference: https://docs.python.org/3/library/os.html#os.rename
>> Signed-off-by: Cleber Rosa <crosa@redhat.com>
>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
And now:
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>
>> Message-Id: <20190613130718.3763-3-crosa@redhat.com>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [Qemu-devel] [PATCH v1 01/17] tests/vm: avoid extra compressed image copy
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 01/17] tests/vm: avoid extra compressed image copy Alex Bennée
@ 2019-06-20 11:07 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2019-06-20 11:07 UTC (permalink / raw)
To: Alex Bennée, qemu-devel; +Cc: Fam Zheng, Cleber Rosa
On 6/19/19 9:40 PM, Alex Bennée wrote:
> From: Cleber Rosa <crosa@redhat.com>
>
> The image copy is only really needed because xz doesn't know to
> properly decompress a file not named properly. Instead of
> decompressing to stdout, and having to rely on a shell, let's just
> create a link instead of copying the file.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Message-Id: <20190613130718.3763-2-crosa@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> tests/vm/centos | 4 ++--
> tests/vm/freebsd | 4 ++--
> tests/vm/netbsd | 4 ++--
> tests/vm/openbsd | 4 ++--
> 4 files changed, 8 insertions(+), 8 deletions(-)
>
> diff --git a/tests/vm/centos b/tests/vm/centos
> index 7417b50af4..b00b46a8dc 100755
> --- a/tests/vm/centos
> +++ b/tests/vm/centos
> @@ -66,8 +66,8 @@ class CentosVM(basevm.BaseVM):
> cimg = self._download_with_cache("https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1802.qcow2.xz")
> img_tmp = img + ".tmp"
> sys.stderr.write("Extracting the image...\n")
> - subprocess.check_call(["cp", "-f", cimg, img_tmp + ".xz"])
> - subprocess.check_call(["xz", "-dvf", img_tmp + ".xz"])
> + subprocess.check_call(["ln", "-f", cimg, img_tmp + ".xz"])
> + subprocess.check_call(["xz", "--keep", "-dvf", img_tmp + ".xz"])
> subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
> self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
> self.wait_ssh()
> diff --git a/tests/vm/freebsd b/tests/vm/freebsd
> index b0066017a6..5575c23a6f 100755
> --- a/tests/vm/freebsd
> +++ b/tests/vm/freebsd
> @@ -34,8 +34,8 @@ class FreeBSDVM(basevm.BaseVM):
> img_tmp_xz = img + ".tmp.xz"
> img_tmp = img + ".tmp"
> sys.stderr.write("Extracting the image...\n")
> - subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
> - subprocess.check_call(["xz", "-dvf", img_tmp_xz])
> + subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
> + subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
> if os.path.exists(img):
> os.remove(img)
> os.rename(img_tmp, img)
> diff --git a/tests/vm/netbsd b/tests/vm/netbsd
> index 4c6624ea5e..d0508f4465 100755
> --- a/tests/vm/netbsd
> +++ b/tests/vm/netbsd
> @@ -34,8 +34,8 @@ class NetBSDVM(basevm.BaseVM):
> img_tmp_xz = img + ".tmp.xz"
> img_tmp = img + ".tmp"
> sys.stderr.write("Extracting the image...\n")
> - subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
> - subprocess.check_call(["xz", "-dvf", img_tmp_xz])
> + subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
> + subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
> if os.path.exists(img):
> os.remove(img)
> os.rename(img_tmp, img)
> diff --git a/tests/vm/openbsd b/tests/vm/openbsd
> index 2105c01a26..87ec982489 100755
> --- a/tests/vm/openbsd
> +++ b/tests/vm/openbsd
> @@ -36,8 +36,8 @@ class OpenBSDVM(basevm.BaseVM):
> img_tmp_xz = img + ".tmp.xz"
> img_tmp = img + ".tmp"
> sys.stderr.write("Extracting the image...\n")
> - subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
> - subprocess.check_call(["xz", "-dvf", img_tmp_xz])
> + subprocess.check_call(["ln", "-f", cimg, img_tmp_xz])
> + subprocess.check_call(["xz", "--keep", "-dvf", img_tmp_xz])
> if os.path.exists(img):
> os.remove(img)
> os.rename(img_tmp, img)
>
^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2019-06-20 11:56 UTC | newest]
Thread overview: 29+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-06-19 19:40 [Qemu-devel] [PATCH v1 00/17] testing/next (Travis fixes, more tests/vm) Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 01/17] tests/vm: avoid extra compressed image copy Alex Bennée
2019-06-20 11:07 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 02/17] tests/vm: avoid image presence check and removal Alex Bennée
2019-06-20 10:29 ` Philippe Mathieu-Daudé
2019-06-20 11:07 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 03/17] tests/vm: pin ubuntu.i386 image Alex Bennée
2019-06-20 10:31 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 04/17] tests/vm: add source repos on ubuntu.i386 Alex Bennée
2019-06-20 11:06 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 05/17] tests/vm: send proxy environment variables over ssh Alex Bennée
2019-06-20 10:38 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 06/17] tests/vm: use ssh with pty unconditionally Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 07/17] tests/vm: run test builds on snapshot Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 08/17] tests/vm: proper guest shutdown Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 09/17] tests/vm: add vm-boot-{ssh, serial}-<guest> targets Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 10/17] tests/vm: serial console support helpers Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 11/17] tests/vm: openbsd autoinstall, using serial console Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 12/17] tests/vm: freebsd " Alex Bennée
2019-06-20 10:39 ` Philippe Mathieu-Daudé
2019-06-20 11:01 ` Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 13/17] tests/vm: netbsd " Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 14/17] tests/vm: fedora " Alex Bennée
2019-06-20 10:36 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 15/17] tests/vm: ubuntu.i386: apt proxy setup Alex Bennée
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 16/17] .travis.yml: default the --disable-system build to --static Alex Bennée
2019-06-20 10:37 ` Philippe Mathieu-Daudé
2019-06-19 19:40 ` [Qemu-devel] [PATCH v1 17/17] .travis.yml: force a brew update for MacOS builds Alex Bennée
2019-06-20 10:37 ` Philippe Mathieu-Daudé
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).