[Qemu-devel] [PATCH v4] tests: vm: auto_install OpenBSD

[Qemu-devel] [PATCH v4] tests: vm: auto_install OpenBSD

[Fam Zheng]

Upgrade OpenBSD to 6.4 using auto_install. Especially, drop SDL1,
include SDL2.

Also do the build in $HOME since both /var/tmp and /tmp are tmpfs with
limited capacities.

Signed-off-by: Fam Zheng <famz@redhat.com>

---

v4: Use 6.4. [Brad]
---
 tests/vm/basevm.py |  6 ++--
 tests/vm/openbsd   | 85 +++++++++++++++++++++++++++++++++++++++-------
 2 files changed, 76 insertions(+), 15 deletions(-)

diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
index 5caf77d6b8..6fb446d4c5 100755
--- a/tests/vm/basevm.py
+++ b/tests/vm/basevm.py
@@ -68,8 +68,6 @@ class BaseVM(object):
         self._args = [ \
             "-nodefaults", "-m", "4G",
             "-cpu", "max",
-            "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22",
-            "-device", "virtio-net-pci,netdev=vnet",
             "-vnc", "127.0.0.1:0,to=20",
             "-serial", "file:%s" % os.path.join(self._tmpdir, "serial.out")]
         if vcpus and vcpus > 1:
@@ -146,8 +144,10 @@ class BaseVM(object):
                             "-device",
                             "virtio-blk,drive=%s,serial=%s,bootindex=1" % (name, name)]
 
-    def boot(self, img, extra_args=[]):
+    def boot(self, img, extra_args=[], extra_usernet_args=""):
         args = self._args + [
+            "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22" + extra_usernet_args,
+            "-device", "virtio-net-pci,netdev=vnet",
             "-device", "VGA",
             "-drive", "file=%s,if=none,id=drive0,cache=writeback" % img,
             "-device", "virtio-blk,drive=drive0,bootindex=0"]
diff --git a/tests/vm/openbsd b/tests/vm/openbsd
index cfe0572c59..99a7e98d80 100755
--- a/tests/vm/openbsd
+++ b/tests/vm/openbsd
@@ -14,6 +14,9 @@
 import os
 import sys
 import subprocess
+import time
+import atexit
+import tempfile
 import basevm
 
 class OpenBSDVM(basevm.BaseVM):
@@ -21,25 +24,83 @@ class OpenBSDVM(basevm.BaseVM):
     arch = "x86_64"
     BUILD_SCRIPT = """
         set -e;
-        rm -rf /var/tmp/qemu-test.*
-        cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
+        rm -rf $HOME/qemu-test.*
+        cd $(mktemp -d $HOME/qemu-test.XXXXXX);
         tar -xf /dev/rsd1c;
-        ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts};
+        ./configure {configure_opts};
         gmake --output-sync -j{jobs} {verbose};
         # XXX: "gmake check" seems to always hang or fail
         #gmake --output-sync -j{jobs} check {verbose};
     """
 
+    def _install_os(self, img):
+        tmpdir = tempfile.mkdtemp()
+        pxeboot = self._download_with_cache("https://fastly.cdn.openbsd.org/pub/OpenBSD/6.4/amd64/pxeboot",
+                sha256sum="d87ab39d941ff926d693943a927585945456ccedb76ea504a251b4b93cd4c266")
+        bsd_rd = self._download_with_cache("https://fastly.cdn.openbsd.org/pub/OpenBSD/6.4/amd64/bsd.rd",
+                sha256sum="89505c683cbcd75582fe475e847ed53d89e2b8180c3e3d61f4eb4b76b5e11f5c")
+        install = self._download_with_cache("https://fastly.cdn.openbsd.org/pub/OpenBSD/6.4/amd64/install64.iso",
+                sha256sum='81833b79e23dc0f961ac5fb34484bca66386deb3181ddb8236870fa4f488cdd2')
+        subprocess.check_call(["qemu-img", "create", img, "32G"])
+        subprocess.check_call(["cp", pxeboot, os.path.join(tmpdir, "auto_install")])
+        subprocess.check_call(["cp", bsd_rd, os.path.join(tmpdir, "bsd")])
+
+        self._gen_install_conf(tmpdir)
+        # BOOTP filename being auto_install makes sure OpenBSD installer
+        # not prompt for "auto install mode"
+        usernet_args = ",tftp=%s,bootfile=/auto_install" % tmpdir
+        usernet_args += ",tftp-server-name=10.0.2.4"
+        usernet_args += ",guestfwd=tcp:10.0.2.4:80-cmd:cat %s" % \
+            os.path.join(tmpdir, "install.conf")
+        self.boot(img,
+                  extra_args=["-boot", "once=n", "-no-reboot",
+                              "-cdrom", install],
+                  extra_usernet_args=usernet_args)
+        self.wait()
+
+    def _gen_install_conf(self, tmpdir):
+        contents = """\
+HTTP/1.0 200 OK
+
+System hostname = qemu-openbsd
+Password for root = qemupass
+Public ssh key for root = {pub_key}
+Allow root ssh login = yes
+Network interfaces = vio0
+IPv4 address for vio0 = dhcp
+Setup a user = qemu
+Password for user = qemupass
+Public ssh key for user = {pub_key}
+What timezone are you in = US/Eastern
+Server = fastly.cdn.openbsd.org
+Use http = yes
+Default IPv4 route = 10.0.2.2
+Location of sets = cd0
+Set name(s) = all
+Continue without verification = yes
+""".format(pub_key=basevm.SSH_PUB_KEY)
+        with open(os.path.join(tmpdir, "install.conf"), "w") as f:
+            f.write(contents)
+
     def build_image(self, img):
-        cimg = self._download_with_cache("http://download.patchew.org/openbsd-6.1-amd64.img.xz",
-                sha256sum='8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf')
-        img_tmp_xz = img + ".tmp.xz"
-        img_tmp = img + ".tmp"
-        subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
-        subprocess.check_call(["xz", "-df", img_tmp_xz])
-        if os.path.exists(img):
-            os.remove(img)
-        os.rename(img_tmp, img)
+
+        self._install_os(img + ".tmp")
+
+        self.boot(img + ".tmp")
+        self.wait_ssh()
+
+        self.ssh_root("usermod -G operator qemu")
+        self.ssh_root("echo https://fastly.cdn.openbsd.org/pub/OpenBSD > /etc/installurl")
+        for pkg in ["git", "gmake", "glib2", "bison", "sdl2"]:
+            self.ssh_root("pkg_add " + pkg)
+        self.ssh_root("ln -sf /usr/local/bin/python2.7 /usr/local/bin/python")
+        self.ssh_root("ln -sf /usr/local/bin/python2.7-2to3 /usr/local/bin/2to3")
+        self.ssh_root("ln -sf /usr/local/bin/python2.7-config /usr/local/bin/python-config")
+        self.ssh_root("ln -sf /usr/local/bin/pydoc2.7 /usr/local/bin/pydoc")
+        self.ssh_root("shutdown -p now")
+        self.wait()
+
+        subprocess.check_call(["mv", img + ".tmp", img])
 
 if __name__ == "__main__":
     sys.exit(basevm.main(OpenBSDVM))
-- 
2.17.2

Re: [Qemu-devel] [PATCH v4] tests: vm: auto_install OpenBSD

[Brad Smith]

ping.

On 10/30/2018 10:57 PM, Fam Zheng wrote:
> Upgrade OpenBSD to 6.4 using auto_install. Especially, drop SDL1,
> include SDL2.
>
> Also do the build in $HOME since both /var/tmp and /tmp are tmpfs with
> limited capacities.
>
> Signed-off-by: Fam Zheng <famz@redhat.com>
>
> ---
>
> v4: Use 6.4. [Brad]
> ---
>   tests/vm/basevm.py |  6 ++--
>   tests/vm/openbsd   | 85 +++++++++++++++++++++++++++++++++++++++-------
>   2 files changed, 76 insertions(+), 15 deletions(-)
>
> diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
> index 5caf77d6b8..6fb446d4c5 100755
> --- a/tests/vm/basevm.py
> +++ b/tests/vm/basevm.py
> @@ -68,8 +68,6 @@ class BaseVM(object):
>           self._args = [ \
>               "-nodefaults", "-m", "4G",
>               "-cpu", "max",
> -            "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22",
> -            "-device", "virtio-net-pci,netdev=vnet",
>               "-vnc", "127.0.0.1:0,to=20",
>               "-serial", "file:%s" % os.path.join(self._tmpdir, "serial.out")]
>           if vcpus and vcpus > 1:
> @@ -146,8 +144,10 @@ class BaseVM(object):
>                               "-device",
>                               "virtio-blk,drive=%s,serial=%s,bootindex=1" % (name, name)]
>   
> -    def boot(self, img, extra_args=[]):
> +    def boot(self, img, extra_args=[], extra_usernet_args=""):
>           args = self._args + [
> +            "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22" + extra_usernet_args,
> +            "-device", "virtio-net-pci,netdev=vnet",
>               "-device", "VGA",
>               "-drive", "file=%s,if=none,id=drive0,cache=writeback" % img,
>               "-device", "virtio-blk,drive=drive0,bootindex=0"]
> diff --git a/tests/vm/openbsd b/tests/vm/openbsd
> index cfe0572c59..99a7e98d80 100755
> --- a/tests/vm/openbsd
> +++ b/tests/vm/openbsd
> @@ -14,6 +14,9 @@
>   import os
>   import sys
>   import subprocess
> +import time
> +import atexit
> +import tempfile
>   import basevm
>   
>   class OpenBSDVM(basevm.BaseVM):
> @@ -21,25 +24,83 @@ class OpenBSDVM(basevm.BaseVM):
>       arch = "x86_64"
>       BUILD_SCRIPT = """
>           set -e;
> -        rm -rf /var/tmp/qemu-test.*
> -        cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
> +        rm -rf $HOME/qemu-test.*
> +        cd $(mktemp -d $HOME/qemu-test.XXXXXX);
>           tar -xf /dev/rsd1c;
> -        ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts};
> +        ./configure {configure_opts};
>           gmake --output-sync -j{jobs} {verbose};
>           # XXX: "gmake check" seems to always hang or fail
>           #gmake --output-sync -j{jobs} check {verbose};
>       """
>   
> +    def _install_os(self, img):
> +        tmpdir = tempfile.mkdtemp()
> +        pxeboot = self._download_with_cache("https://fastly.cdn.openbsd.org/pub/OpenBSD/6.4/amd64/pxeboot",
> +                sha256sum="d87ab39d941ff926d693943a927585945456ccedb76ea504a251b4b93cd4c266")
> +        bsd_rd = self._download_with_cache("https://fastly.cdn.openbsd.org/pub/OpenBSD/6.4/amd64/bsd.rd",
> +                sha256sum="89505c683cbcd75582fe475e847ed53d89e2b8180c3e3d61f4eb4b76b5e11f5c")
> +        install = self._download_with_cache("https://fastly.cdn.openbsd.org/pub/OpenBSD/6.4/amd64/install64.iso",
> +                sha256sum='81833b79e23dc0f961ac5fb34484bca66386deb3181ddb8236870fa4f488cdd2')
> +        subprocess.check_call(["qemu-img", "create", img, "32G"])
> +        subprocess.check_call(["cp", pxeboot, os.path.join(tmpdir, "auto_install")])
> +        subprocess.check_call(["cp", bsd_rd, os.path.join(tmpdir, "bsd")])
> +
> +        self._gen_install_conf(tmpdir)
> +        # BOOTP filename being auto_install makes sure OpenBSD installer
> +        # not prompt for "auto install mode"
> +        usernet_args = ",tftp=%s,bootfile=/auto_install" % tmpdir
> +        usernet_args += ",tftp-server-name=10.0.2.4"
> +        usernet_args += ",guestfwd=tcp:10.0.2.4:80-cmd:cat %s" % \
> +            os.path.join(tmpdir, "install.conf")
> +        self.boot(img,
> +                  extra_args=["-boot", "once=n", "-no-reboot",
> +                              "-cdrom", install],
> +                  extra_usernet_args=usernet_args)
> +        self.wait()
> +
> +    def _gen_install_conf(self, tmpdir):
> +        contents = """\
> +HTTP/1.0 200 OK
> +
> +System hostname = qemu-openbsd
> +Password for root = qemupass
> +Public ssh key for root = {pub_key}
> +Allow root ssh login = yes
> +Network interfaces = vio0
> +IPv4 address for vio0 = dhcp
> +Setup a user = qemu
> +Password for user = qemupass
> +Public ssh key for user = {pub_key}
> +What timezone are you in = US/Eastern
> +Server = fastly.cdn.openbsd.org
> +Use http = yes
> +Default IPv4 route = 10.0.2.2
> +Location of sets = cd0
> +Set name(s) = all
> +Continue without verification = yes
> +""".format(pub_key=basevm.SSH_PUB_KEY)
> +        with open(os.path.join(tmpdir, "install.conf"), "w") as f:
> +            f.write(contents)
> +
>       def build_image(self, img):
> -        cimg = self._download_with_cache("http://download.patchew.org/openbsd-6.1-amd64.img.xz",
> -                sha256sum='8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf')
> -        img_tmp_xz = img + ".tmp.xz"
> -        img_tmp = img + ".tmp"
> -        subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
> -        subprocess.check_call(["xz", "-df", img_tmp_xz])
> -        if os.path.exists(img):
> -            os.remove(img)
> -        os.rename(img_tmp, img)
> +
> +        self._install_os(img + ".tmp")
> +
> +        self.boot(img + ".tmp")
> +        self.wait_ssh()
> +
> +        self.ssh_root("usermod -G operator qemu")
> +        self.ssh_root("echo https://fastly.cdn.openbsd.org/pub/OpenBSD > /etc/installurl")
> +        for pkg in ["git", "gmake", "glib2", "bison", "sdl2"]:
> +            self.ssh_root("pkg_add " + pkg)
> +        self.ssh_root("ln -sf /usr/local/bin/python2.7 /usr/local/bin/python")
> +        self.ssh_root("ln -sf /usr/local/bin/python2.7-2to3 /usr/local/bin/2to3")
> +        self.ssh_root("ln -sf /usr/local/bin/python2.7-config /usr/local/bin/python-config")
> +        self.ssh_root("ln -sf /usr/local/bin/pydoc2.7 /usr/local/bin/pydoc")
> +        self.ssh_root("shutdown -p now")
> +        self.wait()
> +
> +        subprocess.check_call(["mv", img + ".tmp", img])
>   
>   if __name__ == "__main__":
>       sys.exit(basevm.main(OpenBSDVM))

Re: [Qemu-devel] [PATCH v4] tests: vm: auto_install OpenBSD

[Fam Zheng]

On Sun, 11/11 18:20, Brad Smith wrote:
> ping.

Queued. Will send a pull request soon.

Fam

> 
> On 10/30/2018 10:57 PM, Fam Zheng wrote:
> > Upgrade OpenBSD to 6.4 using auto_install. Especially, drop SDL1,
> > include SDL2.
> > 
> > Also do the build in $HOME since both /var/tmp and /tmp are tmpfs with
> > limited capacities.
> > 
> > Signed-off-by: Fam Zheng <famz@redhat.com>
> > 
> > ---
> > 
> > v4: Use 6.4. [Brad]
> > ---
> >   tests/vm/basevm.py |  6 ++--
> >   tests/vm/openbsd   | 85 +++++++++++++++++++++++++++++++++++++++-------
> >   2 files changed, 76 insertions(+), 15 deletions(-)
> > 
> > diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
> > index 5caf77d6b8..6fb446d4c5 100755
> > --- a/tests/vm/basevm.py
> > +++ b/tests/vm/basevm.py
> > @@ -68,8 +68,6 @@ class BaseVM(object):
> >           self._args = [ \
> >               "-nodefaults", "-m", "4G",
> >               "-cpu", "max",
> > -            "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22",
> > -            "-device", "virtio-net-pci,netdev=vnet",
> >               "-vnc", "127.0.0.1:0,to=20",
> >               "-serial", "file:%s" % os.path.join(self._tmpdir, "serial.out")]
> >           if vcpus and vcpus > 1:
> > @@ -146,8 +144,10 @@ class BaseVM(object):
> >                               "-device",
> >                               "virtio-blk,drive=%s,serial=%s,bootindex=1" % (name, name)]
> > -    def boot(self, img, extra_args=[]):
> > +    def boot(self, img, extra_args=[], extra_usernet_args=""):
> >           args = self._args + [
> > +            "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22" + extra_usernet_args,
> > +            "-device", "virtio-net-pci,netdev=vnet",
> >               "-device", "VGA",
> >               "-drive", "file=%s,if=none,id=drive0,cache=writeback" % img,
> >               "-device", "virtio-blk,drive=drive0,bootindex=0"]
> > diff --git a/tests/vm/openbsd b/tests/vm/openbsd
> > index cfe0572c59..99a7e98d80 100755
> > --- a/tests/vm/openbsd
> > +++ b/tests/vm/openbsd
> > @@ -14,6 +14,9 @@
> >   import os
> >   import sys
> >   import subprocess
> > +import time
> > +import atexit
> > +import tempfile
> >   import basevm
> >   class OpenBSDVM(basevm.BaseVM):
> > @@ -21,25 +24,83 @@ class OpenBSDVM(basevm.BaseVM):
> >       arch = "x86_64"
> >       BUILD_SCRIPT = """
> >           set -e;
> > -        rm -rf /var/tmp/qemu-test.*
> > -        cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
> > +        rm -rf $HOME/qemu-test.*
> > +        cd $(mktemp -d $HOME/qemu-test.XXXXXX);
> >           tar -xf /dev/rsd1c;
> > -        ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts};
> > +        ./configure {configure_opts};
> >           gmake --output-sync -j{jobs} {verbose};
> >           # XXX: "gmake check" seems to always hang or fail
> >           #gmake --output-sync -j{jobs} check {verbose};
> >       """
> > +    def _install_os(self, img):
> > +        tmpdir = tempfile.mkdtemp()
> > +        pxeboot = self._download_with_cache("https://fastly.cdn.openbsd.org/pub/OpenBSD/6.4/amd64/pxeboot",
> > +                sha256sum="d87ab39d941ff926d693943a927585945456ccedb76ea504a251b4b93cd4c266")
> > +        bsd_rd = self._download_with_cache("https://fastly.cdn.openbsd.org/pub/OpenBSD/6.4/amd64/bsd.rd",
> > +                sha256sum="89505c683cbcd75582fe475e847ed53d89e2b8180c3e3d61f4eb4b76b5e11f5c")
> > +        install = self._download_with_cache("https://fastly.cdn.openbsd.org/pub/OpenBSD/6.4/amd64/install64.iso",
> > +                sha256sum='81833b79e23dc0f961ac5fb34484bca66386deb3181ddb8236870fa4f488cdd2')
> > +        subprocess.check_call(["qemu-img", "create", img, "32G"])
> > +        subprocess.check_call(["cp", pxeboot, os.path.join(tmpdir, "auto_install")])
> > +        subprocess.check_call(["cp", bsd_rd, os.path.join(tmpdir, "bsd")])
> > +
> > +        self._gen_install_conf(tmpdir)
> > +        # BOOTP filename being auto_install makes sure OpenBSD installer
> > +        # not prompt for "auto install mode"
> > +        usernet_args = ",tftp=%s,bootfile=/auto_install" % tmpdir
> > +        usernet_args += ",tftp-server-name=10.0.2.4"
> > +        usernet_args += ",guestfwd=tcp:10.0.2.4:80-cmd:cat %s" % \
> > +            os.path.join(tmpdir, "install.conf")
> > +        self.boot(img,
> > +                  extra_args=["-boot", "once=n", "-no-reboot",
> > +                              "-cdrom", install],
> > +                  extra_usernet_args=usernet_args)
> > +        self.wait()
> > +
> > +    def _gen_install_conf(self, tmpdir):
> > +        contents = """\
> > +HTTP/1.0 200 OK
> > +
> > +System hostname = qemu-openbsd
> > +Password for root = qemupass
> > +Public ssh key for root = {pub_key}
> > +Allow root ssh login = yes
> > +Network interfaces = vio0
> > +IPv4 address for vio0 = dhcp
> > +Setup a user = qemu
> > +Password for user = qemupass
> > +Public ssh key for user = {pub_key}
> > +What timezone are you in = US/Eastern
> > +Server = fastly.cdn.openbsd.org
> > +Use http = yes
> > +Default IPv4 route = 10.0.2.2
> > +Location of sets = cd0
> > +Set name(s) = all
> > +Continue without verification = yes
> > +""".format(pub_key=basevm.SSH_PUB_KEY)
> > +        with open(os.path.join(tmpdir, "install.conf"), "w") as f:
> > +            f.write(contents)
> > +
> >       def build_image(self, img):
> > -        cimg = self._download_with_cache("http://download.patchew.org/openbsd-6.1-amd64.img.xz",
> > -                sha256sum='8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf')
> > -        img_tmp_xz = img + ".tmp.xz"
> > -        img_tmp = img + ".tmp"
> > -        subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
> > -        subprocess.check_call(["xz", "-df", img_tmp_xz])
> > -        if os.path.exists(img):
> > -            os.remove(img)
> > -        os.rename(img_tmp, img)
> > +
> > +        self._install_os(img + ".tmp")
> > +
> > +        self.boot(img + ".tmp")
> > +        self.wait_ssh()
> > +
> > +        self.ssh_root("usermod -G operator qemu")
> > +        self.ssh_root("echo https://fastly.cdn.openbsd.org/pub/OpenBSD > /etc/installurl")
> > +        for pkg in ["git", "gmake", "glib2", "bison", "sdl2"]:
> > +            self.ssh_root("pkg_add " + pkg)
> > +        self.ssh_root("ln -sf /usr/local/bin/python2.7 /usr/local/bin/python")
> > +        self.ssh_root("ln -sf /usr/local/bin/python2.7-2to3 /usr/local/bin/2to3")
> > +        self.ssh_root("ln -sf /usr/local/bin/python2.7-config /usr/local/bin/python-config")
> > +        self.ssh_root("ln -sf /usr/local/bin/pydoc2.7 /usr/local/bin/pydoc")
> > +        self.ssh_root("shutdown -p now")
> > +        self.wait()
> > +
> > +        subprocess.check_call(["mv", img + ".tmp", img])
> >   if __name__ == "__main__":
> >       sys.exit(basevm.main(OpenBSDVM))

Re: [Qemu-devel] [PATCH v4] tests: vm: auto_install OpenBSD

[Daniel P. Berrangé]

On Wed, Nov 14, 2018 at 05:58:07PM +0800, Fam Zheng wrote:
> On Sun, 11/11 18:20, Brad Smith wrote:
> > ping.
> 
> Queued. Will send a pull request soon.

ping, this seems to have got lost, and is blocking us deleting SDL 1.2
support from QEMU.

> > On 10/30/2018 10:57 PM, Fam Zheng wrote:
> > > Upgrade OpenBSD to 6.4 using auto_install. Especially, drop SDL1,
> > > include SDL2.
> > > 
> > > Also do the build in $HOME since both /var/tmp and /tmp are tmpfs with
> > > limited capacities.
> > > 
> > > Signed-off-by: Fam Zheng <famz@redhat.com>
> > > 
> > > ---
> > > 
> > > v4: Use 6.4. [Brad]
> > > ---
> > >   tests/vm/basevm.py |  6 ++--
> > >   tests/vm/openbsd   | 85 +++++++++++++++++++++++++++++++++++++++-------
> > >   2 files changed, 76 insertions(+), 15 deletions(-)
> > > 
> > > diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py
> > > index 5caf77d6b8..6fb446d4c5 100755
> > > --- a/tests/vm/basevm.py
> > > +++ b/tests/vm/basevm.py
> > > @@ -68,8 +68,6 @@ class BaseVM(object):
> > >           self._args = [ \
> > >               "-nodefaults", "-m", "4G",
> > >               "-cpu", "max",
> > > -            "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22",
> > > -            "-device", "virtio-net-pci,netdev=vnet",
> > >               "-vnc", "127.0.0.1:0,to=20",
> > >               "-serial", "file:%s" % os.path.join(self._tmpdir, "serial.out")]
> > >           if vcpus and vcpus > 1:
> > > @@ -146,8 +144,10 @@ class BaseVM(object):
> > >                               "-device",
> > >                               "virtio-blk,drive=%s,serial=%s,bootindex=1" % (name, name)]
> > > -    def boot(self, img, extra_args=[]):
> > > +    def boot(self, img, extra_args=[], extra_usernet_args=""):
> > >           args = self._args + [
> > > +            "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22" + extra_usernet_args,
> > > +            "-device", "virtio-net-pci,netdev=vnet",
> > >               "-device", "VGA",
> > >               "-drive", "file=%s,if=none,id=drive0,cache=writeback" % img,
> > >               "-device", "virtio-blk,drive=drive0,bootindex=0"]
> > > diff --git a/tests/vm/openbsd b/tests/vm/openbsd
> > > index cfe0572c59..99a7e98d80 100755
> > > --- a/tests/vm/openbsd
> > > +++ b/tests/vm/openbsd
> > > @@ -14,6 +14,9 @@
> > >   import os
> > >   import sys
> > >   import subprocess
> > > +import time
> > > +import atexit
> > > +import tempfile
> > >   import basevm
> > >   class OpenBSDVM(basevm.BaseVM):
> > > @@ -21,25 +24,83 @@ class OpenBSDVM(basevm.BaseVM):
> > >       arch = "x86_64"
> > >       BUILD_SCRIPT = """
> > >           set -e;
> > > -        rm -rf /var/tmp/qemu-test.*
> > > -        cd $(mktemp -d /var/tmp/qemu-test.XXXXXX);
> > > +        rm -rf $HOME/qemu-test.*
> > > +        cd $(mktemp -d $HOME/qemu-test.XXXXXX);
> > >           tar -xf /dev/rsd1c;
> > > -        ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts};
> > > +        ./configure {configure_opts};
> > >           gmake --output-sync -j{jobs} {verbose};
> > >           # XXX: "gmake check" seems to always hang or fail
> > >           #gmake --output-sync -j{jobs} check {verbose};
> > >       """
> > > +    def _install_os(self, img):
> > > +        tmpdir = tempfile.mkdtemp()
> > > +        pxeboot = self._download_with_cache("https://fastly.cdn.openbsd.org/pub/OpenBSD/6.4/amd64/pxeboot",
> > > +                sha256sum="d87ab39d941ff926d693943a927585945456ccedb76ea504a251b4b93cd4c266")
> > > +        bsd_rd = self._download_with_cache("https://fastly.cdn.openbsd.org/pub/OpenBSD/6.4/amd64/bsd.rd",
> > > +                sha256sum="89505c683cbcd75582fe475e847ed53d89e2b8180c3e3d61f4eb4b76b5e11f5c")
> > > +        install = self._download_with_cache("https://fastly.cdn.openbsd.org/pub/OpenBSD/6.4/amd64/install64.iso",
> > > +                sha256sum='81833b79e23dc0f961ac5fb34484bca66386deb3181ddb8236870fa4f488cdd2')
> > > +        subprocess.check_call(["qemu-img", "create", img, "32G"])
> > > +        subprocess.check_call(["cp", pxeboot, os.path.join(tmpdir, "auto_install")])
> > > +        subprocess.check_call(["cp", bsd_rd, os.path.join(tmpdir, "bsd")])
> > > +
> > > +        self._gen_install_conf(tmpdir)
> > > +        # BOOTP filename being auto_install makes sure OpenBSD installer
> > > +        # not prompt for "auto install mode"
> > > +        usernet_args = ",tftp=%s,bootfile=/auto_install" % tmpdir
> > > +        usernet_args += ",tftp-server-name=10.0.2.4"
> > > +        usernet_args += ",guestfwd=tcp:10.0.2.4:80-cmd:cat %s" % \
> > > +            os.path.join(tmpdir, "install.conf")
> > > +        self.boot(img,
> > > +                  extra_args=["-boot", "once=n", "-no-reboot",
> > > +                              "-cdrom", install],
> > > +                  extra_usernet_args=usernet_args)
> > > +        self.wait()
> > > +
> > > +    def _gen_install_conf(self, tmpdir):
> > > +        contents = """\
> > > +HTTP/1.0 200 OK
> > > +
> > > +System hostname = qemu-openbsd
> > > +Password for root = qemupass
> > > +Public ssh key for root = {pub_key}
> > > +Allow root ssh login = yes
> > > +Network interfaces = vio0
> > > +IPv4 address for vio0 = dhcp
> > > +Setup a user = qemu
> > > +Password for user = qemupass
> > > +Public ssh key for user = {pub_key}
> > > +What timezone are you in = US/Eastern
> > > +Server = fastly.cdn.openbsd.org
> > > +Use http = yes
> > > +Default IPv4 route = 10.0.2.2
> > > +Location of sets = cd0
> > > +Set name(s) = all
> > > +Continue without verification = yes
> > > +""".format(pub_key=basevm.SSH_PUB_KEY)
> > > +        with open(os.path.join(tmpdir, "install.conf"), "w") as f:
> > > +            f.write(contents)
> > > +
> > >       def build_image(self, img):
> > > -        cimg = self._download_with_cache("http://download.patchew.org/openbsd-6.1-amd64.img.xz",
> > > -                sha256sum='8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf')
> > > -        img_tmp_xz = img + ".tmp.xz"
> > > -        img_tmp = img + ".tmp"
> > > -        subprocess.check_call(["cp", "-f", cimg, img_tmp_xz])
> > > -        subprocess.check_call(["xz", "-df", img_tmp_xz])
> > > -        if os.path.exists(img):
> > > -            os.remove(img)
> > > -        os.rename(img_tmp, img)
> > > +
> > > +        self._install_os(img + ".tmp")
> > > +
> > > +        self.boot(img + ".tmp")
> > > +        self.wait_ssh()
> > > +
> > > +        self.ssh_root("usermod -G operator qemu")
> > > +        self.ssh_root("echo https://fastly.cdn.openbsd.org/pub/OpenBSD > /etc/installurl")
> > > +        for pkg in ["git", "gmake", "glib2", "bison", "sdl2"]:
> > > +            self.ssh_root("pkg_add " + pkg)
> > > +        self.ssh_root("ln -sf /usr/local/bin/python2.7 /usr/local/bin/python")
> > > +        self.ssh_root("ln -sf /usr/local/bin/python2.7-2to3 /usr/local/bin/2to3")
> > > +        self.ssh_root("ln -sf /usr/local/bin/python2.7-config /usr/local/bin/python-config")
> > > +        self.ssh_root("ln -sf /usr/local/bin/pydoc2.7 /usr/local/bin/pydoc")
> > > +        self.ssh_root("shutdown -p now")
> > > +        self.wait()
> > > +
> > > +        subprocess.check_call(["mv", img + ".tmp", img])
> > >   if __name__ == "__main__":
> > >       sys.exit(basevm.main(OpenBSDVM))
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH v4] tests: vm: auto_install OpenBSD

[Daniel P. Berrangé]

On Wed, Jan 09, 2019 at 01:55:04PM +0000, Daniel P. Berrangé wrote:
> On Wed, Nov 14, 2018 at 05:58:07PM +0800, Fam Zheng wrote:
> > On Sun, 11/11 18:20, Brad Smith wrote:
> > > ping.
> > 
> > Queued. Will send a pull request soon.
> 
> ping, this seems to have got lost, and is blocking us deleting SDL 1.2
> support from QEMU.

Sigh, never mind, I found the PR is here:

https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg00732.html

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH v4] tests: vm: auto_install OpenBSD

[Fam Zheng]

> On Jan 9, 2019, at 21:59, Daniel P. Berrangé <berrange@redhat.com> wrote:
> 
> On Wed, Jan 09, 2019 at 01:55:04PM +0000, Daniel P. Berrangé wrote:
>> On Wed, Nov 14, 2018 at 05:58:07PM +0800, Fam Zheng wrote:
>>> On Sun, 11/11 18:20, Brad Smith wrote:
>>>> ping.
>>> 
>>> Queued. Will send a pull request soon.
>> 
>> ping, this seems to have got lost, and is blocking us deleting SDL 1.2
>> support from QEMU.
> 
> Sigh, never mind, I found the PR is here:
> 
> https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg00732.html

I had to drop it from the queue because Peter didn’t like the QEMU version requirement (for the slirp tftp parameters). :(

What is the blocker for delete SDL 1.2? Is it breaking tests/vm/openbsd? If so maybe we can add —disable-sdl to its configure command.

Fam

> 
> Regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
> 

Re: [Qemu-devel] [PATCH v4] tests: vm: auto_install OpenBSD

[Daniel P. Berrangé]

On Mon, Jan 14, 2019 at 11:50:08AM +0800, Fam Zheng wrote:
> 
> 
> > On Jan 9, 2019, at 21:59, Daniel P. Berrangé <berrange@redhat.com> wrote:
> > 
> > On Wed, Jan 09, 2019 at 01:55:04PM +0000, Daniel P. Berrangé wrote:
> >> On Wed, Nov 14, 2018 at 05:58:07PM +0800, Fam Zheng wrote:
> >>> On Sun, 11/11 18:20, Brad Smith wrote:
> >>>> ping.
> >>> 
> >>> Queued. Will send a pull request soon.
> >> 
> >> ping, this seems to have got lost, and is blocking us deleting SDL 1.2
> >> support from QEMU.
> > 
> > Sigh, never mind, I found the PR is here:
> > 
> > https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg00732.html
> 
> I had to drop it from the queue because Peter didn’t like the QEMU version
> requirement (for the slirp tftp parameters). :(

IIUC, currently we're just downloading

   http://download.patchew.org/openbsd-6.1-amd64.img.xz

can't you just refresh that pre-built image to have SDL 2, based on your
auto-install patches, so we don't have to wait for that to merge.

> What is the blocker for delete SDL 1.2? Is it breaking tests/vm/openbsd?
> If so maybe we can add —disable-sdl to its configure command.

If you don't have SDL then there's no audio driver impl available for
QEMU, so configure fails with:

  ERROR: sdl not found or disabled, can not use sdl audio driver


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH v4] tests: vm: auto_install OpenBSD

[Peter Maydell]

On Mon, 14 Jan 2019 at 10:20, Daniel P. Berrangé <berrange@redhat.com> wrote:
> If you don't have SDL then there's no audio driver impl available for
> QEMU, so configure fails with:
>
>   ERROR: sdl not found or disabled, can not use sdl audio driver

Ideally we should fix configure/the audio driver selection
logic so that if SDL is not available or disabled then
'sdl' is removed from the set of possible audio devices
and not selected as the default for OpenBSD (and if necessary
fix things so we can build a QEMU with no audio drivers at all,
falling back to the null driver, ie no sound).

It would also be good to make the various BSD images have
SDL2 installed, for the compile coverage.

thanks
-- PMM

Re: [Qemu-devel] [PATCH v4] tests: vm: auto_install OpenBSD

[Daniel P. Berrangé]

On Mon, Jan 14, 2019 at 10:15:15AM +0000, Daniel P. Berrangé wrote:
> On Mon, Jan 14, 2019 at 11:50:08AM +0800, Fam Zheng wrote:
> > 
> > 
> > > On Jan 9, 2019, at 21:59, Daniel P. Berrangé <berrange@redhat.com> wrote:
> > > 
> > > On Wed, Jan 09, 2019 at 01:55:04PM +0000, Daniel P. Berrangé wrote:
> > >> On Wed, Nov 14, 2018 at 05:58:07PM +0800, Fam Zheng wrote:
> > >>> On Sun, 11/11 18:20, Brad Smith wrote:
> > >>>> ping.
> > >>> 
> > >>> Queued. Will send a pull request soon.
> > >> 
> > >> ping, this seems to have got lost, and is blocking us deleting SDL 1.2
> > >> support from QEMU.
> > > 
> > > Sigh, never mind, I found the PR is here:
> > > 
> > > https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg00732.html
> > 
> > I had to drop it from the queue because Peter didn’t like the QEMU version
> > requirement (for the slirp tftp parameters). :(
> 
> IIUC, currently we're just downloading
> 
>    http://download.patchew.org/openbsd-6.1-amd64.img.xz
> 
> can't you just refresh that pre-built image to have SDL 2, based on your
> auto-install patches, so we don't have to wait for that to merge.

Please could we just get this pre-built img updated so that we are
not blocked on the SDL 1 removal anymore.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH v4] tests: vm: auto_install OpenBSD

[Fam Zheng]

On Tue, 01/22 16:12, Daniel P. Berrangé wrote:
> > can't you just refresh that pre-built image to have SDL 2, based on your
> > auto-install patches, so we don't have to wait for that to merge.
> 
> Please could we just get this pre-built img updated so that we are
> not blocked on the SDL 1 removal anymore.

Oops. I was seriously distracted from the external mail folders recently..
Updated the image, please test it.

Fam

Re: [Qemu-devel] [PATCH v4] tests: vm: auto_install OpenBSD

[Fam Zheng]

On Tue, 01/22 16:12, Daniel P. Berrangé wrote:
> > can't you just refresh that pre-built image to have SDL 2, based on your
> > auto-install patches, so we don't have to wait for that to merge.
> 
> Please could we just get this pre-built img updated so that we are
> not blocked on the SDL 1 removal anymore.

Oops. I was seriously distracted from the external mail folders recently..
Updated the image, please test it.

Fam