[PATCH OSSTEST 0/4] Have OpenStack tested on top of xen's master and libvirt's master.

[PATCH OSSTEST 0/4] Have OpenStack tested on top of xen's master and libvirt's master.

[Anthony PERARD]

Hi,

I have looked into getting OpenStack been tested on the latest Xen via
osstest. There is an adjustement in common code to accomodate OpenStack,
this is an extra Linux config option. Otherwise anything else should be
done only if a Toolstack named "openstack" is to be used for testing.

I've introduce an extra Osstest::Toolstack which help to install extra
package, and use ballonning for Dom0, 500MB for Dom0 is definetly not
enough.

The ts-devstack script does prepare a bit more the host, clone devstack,
then run ./stack.sh, which is a bit like raisin. Once the machine ready,
the integration test suite from OpenStack, Tempest, is started. Do you
think those two step should be in separate test, one for devstack, and one
for Tempest?  I have not done it, but we could have some smoke test before
Tempest where osstest tryied to start a guest.

For the last patch, I've tried to have ./standalone make-flight have only
the necessary jobs to run test-amd64-amd64-devstack, but there is still
build-xsm in the list.

Then later, there will be the question of which tree to track, devstack?
nova? Or don't track any and just test with the master branch from time to
time.

Thanks,

Anthony PERARD (4):
  ts-kernel-build: Enable CONFIG_NETFILTER_XT_TARGET_CHECKSUM
  Toolstack: Add OpenStack as a toolstack.
  ts-devstack: Deploy OpenStack then test it with Tempest
  Create a flight to test OpenStack with xen-unstable and libvirt.

 Osstest/Toolstack/openstack.pm |  81 +++++++++++
 ap-common                      |   4 +
 ap-fetch-version               |   4 +
 ap-fetch-version-old           |   5 +
 ap-print-url                   |   3 +
 cri-common                     |   1 +
 make-flight                    |  17 +++
 mfi-common                     |   7 +
 sg-run-job                     |   5 +
 ts-devstack                    | 305 +++++++++++++++++++++++++++++++++++++++++
 ts-kernel-build                |   5 +
 ts-xen-install                 |   1 +
 12 files changed, 438 insertions(+)
 create mode 100644 Osstest/Toolstack/openstack.pm
 create mode 100755 ts-devstack

-- 
Anthony PERARD

[PATCH OSSTEST 1/4] ts-kernel-build: Enable CONFIG_NETFILTER_XT_TARGET_CHECKSUM

[Anthony PERARD]

This iptables target CHECKSUM is used by OpenStack.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-kernel-build | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/ts-kernel-build b/ts-kernel-build
index 28bead4..d5defaa 100755
--- a/ts-kernel-build
+++ b/ts-kernel-build
@@ -248,6 +248,11 @@ setopt CONFIG_SYSVIPC y
 setopt CONFIG_BLK_DEV_LOOP y
 
 setopt CONFIG_PACKET y
+
+# needed for OpenStack
+# because: https://bugzilla.redhat.com/show_bug.cgi?id=910619#c6
+setopt CONFIG_NETFILTER_ADVANCED y
+setopt CONFIG_NETFILTER_XT_TARGET_CHECKSUM m
 END
 sub stash_config_edscript ($) {
     my ($settings) = @_;
-- 
Anthony PERARD

Re: [PATCH OSSTEST 1/4] ts-kernel-build: Enable CONFIG_NETFILTER_XT_TARGET_CHECKSUM

[Ian Campbell]

On Thu, 2015-07-16 at 12:18 +0100, Anthony PERARD wrote:
> This iptables target CHECKSUM is used by OpenStack.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

> ---
>  ts-kernel-build | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/ts-kernel-build b/ts-kernel-build
> index 28bead4..d5defaa 100755
> --- a/ts-kernel-build
> +++ b/ts-kernel-build
> @@ -248,6 +248,11 @@ setopt CONFIG_SYSVIPC y
>  setopt CONFIG_BLK_DEV_LOOP y
>  
>  setopt CONFIG_PACKET y
> +
> +# needed for OpenStack
> +# because: https://bugzilla.redhat.com/show_bug.cgi?id=910619#c6

How exciting!

> +setopt CONFIG_NETFILTER_ADVANCED y
> +setopt CONFIG_NETFILTER_XT_TARGET_CHECKSUM m
>  END
>  sub stash_config_edscript ($) {
>      my ($settings) = @_;

[PATCH OSSTEST 2/4] Toolstack: Add OpenStack as a toolstack.

[Anthony PERARD]

Having a separate toolstack help to configure the host specificly for
OpenStack, and also helps to have a separate flight.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/Toolstack/openstack.pm | 81 ++++++++++++++++++++++++++++++++++++++++++
 ts-xen-install                 |  1 +
 2 files changed, 82 insertions(+)
 create mode 100644 Osstest/Toolstack/openstack.pm

diff --git a/Osstest/Toolstack/openstack.pm b/Osstest/Toolstack/openstack.pm
new file mode 100644
index 0000000..4529845
--- /dev/null
+++ b/Osstest/Toolstack/openstack.pm
@@ -0,0 +1,81 @@
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+package Osstest::Toolstack::openstack;
+
+use strict;
+use warnings;
+
+use Osstest::TestSupport;
+
+sub new {
+    my ($class, $ho, $methname,$asset) = @_;
+    my @extra_packages = qw(git pwgen sudo);
+    # package from libvirt
+    push(@extra_packages, qw(libavahi-client3));
+    my $nl_lib = "libnl-3-200";
+    $nl_lib = "libnl1" if ($ho->{Suite} =~ m/wheezy/);
+    push(@extra_packages, $nl_lib);
+    return bless { Name => "openstack",
+		   Host => $ho,
+                   NewDaemons => [qw(libvirtd)],
+		   Dom0MemFixed => 0,
+                   ExtraPackages => \@extra_packages,
+    }, $class;
+}
+
+sub destroy ($$) {
+    my ($self,$gho) = @_;
+    die "Guest control not yet supported via OpenStack";
+}
+
+sub create ($$) {
+    my ($self,$gho) = @_;
+    die "Guest control not yet supported via OpenStack";
+}
+
+sub consolecmd ($$) {
+    my ($self,$gho) = @_;
+    die "Guest control not yet supported via OpenStack";
+}
+
+sub shutdown_wait ($$$) {
+    my ($self,$gho,$timeout) = @_;
+    die "Guest control not yet supported via OpenStack";
+}
+
+sub migrate_check ($) {
+    my ($self) = @_;
+    die "Guest control not yet supported via OpenStack";
+}
+
+sub migrate ($) {
+    my ($self,$gho,$dst,$timeout) = @_;
+    die "Guest control not yet supported via OpenStack";
+}
+
+sub save ($$$$) {
+    my ($self,$gho,$f,$timeout) = @_;
+    die "Guest control not yet supported via OpenStack";
+}
+
+sub restore ($$$$) {
+    my ($self,$gho,$f,$timeout) = @_;
+    die "Guest control not yet supported via OpenStack";
+}
+
+1;
diff --git a/ts-xen-install b/ts-xen-install
index 0f53382..e7e823d 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -72,6 +72,7 @@ sub packages () {
 sub extract () {
     my @parts = ('', 'kern', 'xen');
     push @parts, 'libvirt' if $r{toolstack} eq "libvirt";
+    push @parts, 'libvirt' if $r{toolstack} eq "openstack";
 
     foreach my $part (@parts) {
         target_extract_jobdistpath($ho, $part, "path_${part}dist",
-- 
Anthony PERARD

Re: [PATCH OSSTEST 2/4] Toolstack: Add OpenStack as a toolstack.

[Ian Campbell]

On Thu, 2015-07-16 at 12:18 +0100, Anthony PERARD wrote:
> Having a separate toolstack help to configure the host specificly for

"specifically"

> OpenStack, and also helps to have a separate flight.

Will any of the die()s here ever become real commands? I think not
because that's not how OpenStack works? i.e. how it is driven doesn't
fit into Osstest::Toolstack's model of the world at all.

Given that and given that you introduce your own recipe to sg-run-job
which doesn't/can't/won't try to do any of the usual stuff, I can't see
any reason not to simply go with toolstack=libvirt for these test cases.
That it is the most accurate reflection of what we want and if some
openstack test case did want to talk to the underlying toolstack for
some reason libvirt would be it.

That would also save duplicating the functionality of libvirt.pm's new
function, which would normally call for a subclass of some sort (e.g.
how xend is a subclass of xl). Please don't do this though ;-)

> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
>  Osstest/Toolstack/openstack.pm | 81 ++++++++++++++++++++++++++++++++++++++++++
>  ts-xen-install                 |  1 +
>  2 files changed, 82 insertions(+)
>  create mode 100644 Osstest/Toolstack/openstack.pm
> 
> diff --git a/Osstest/Toolstack/openstack.pm b/Osstest/Toolstack/openstack.pm
> new file mode 100644
> index 0000000..4529845
> --- /dev/null
> +++ b/Osstest/Toolstack/openstack.pm
> @@ -0,0 +1,81 @@
> +# This is part of "osstest", an automated testing framework for Xen.
> +# Copyright (C) 2015 Citrix Inc.
> +#
> +# This program is free software: you can redistribute it and/or modify
> +# it under the terms of the GNU Affero General Public License as published by
> +# the Free Software Foundation, either version 3 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU Affero General Public License for more details.
> +#
> +# You should have received a copy of the GNU Affero General Public License
> +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
> +
> +
> +package Osstest::Toolstack::openstack;
> +
> +use strict;
> +use warnings;
> +
> +use Osstest::TestSupport;
> +
> +sub new {
> +    my ($class, $ho, $methname,$asset) = @_;
> +    my @extra_packages = qw(git pwgen sudo);
> +    # package from libvirt
> +    push(@extra_packages, qw(libavahi-client3));
> +    my $nl_lib = "libnl-3-200";
> +    $nl_lib = "libnl1" if ($ho->{Suite} =~ m/wheezy/);
> +    push(@extra_packages, $nl_lib);
> +    return bless { Name => "openstack",
> +		   Host => $ho,
> +                   NewDaemons => [qw(libvirtd)],
> +		   Dom0MemFixed => 0,
> +                   ExtraPackages => \@extra_packages,
> +    }, $class;
> +}
> +
> +sub destroy ($$) {
> +    my ($self,$gho) = @_;
> +    die "Guest control not yet supported via OpenStack";
> +}
> +
> +sub create ($$) {
> +    my ($self,$gho) = @_;
> +    die "Guest control not yet supported via OpenStack";
> +}
> +
> +sub consolecmd ($$) {
> +    my ($self,$gho) = @_;
> +    die "Guest control not yet supported via OpenStack";
> +}
> +
> +sub shutdown_wait ($$$) {
> +    my ($self,$gho,$timeout) = @_;
> +    die "Guest control not yet supported via OpenStack";
> +}
> +
> +sub migrate_check ($) {
> +    my ($self) = @_;
> +    die "Guest control not yet supported via OpenStack";
> +}
> +
> +sub migrate ($) {
> +    my ($self,$gho,$dst,$timeout) = @_;
> +    die "Guest control not yet supported via OpenStack";
> +}
> +
> +sub save ($$$$) {
> +    my ($self,$gho,$f,$timeout) = @_;
> +    die "Guest control not yet supported via OpenStack";
> +}
> +
> +sub restore ($$$$) {
> +    my ($self,$gho,$f,$timeout) = @_;
> +    die "Guest control not yet supported via OpenStack";
> +}
> +
> +1;
> diff --git a/ts-xen-install b/ts-xen-install
> index 0f53382..e7e823d 100755
> --- a/ts-xen-install
> +++ b/ts-xen-install
> @@ -72,6 +72,7 @@ sub packages () {
>  sub extract () {
>      my @parts = ('', 'kern', 'xen');
>      push @parts, 'libvirt' if $r{toolstack} eq "libvirt";
> +    push @parts, 'libvirt' if $r{toolstack} eq "openstack";
>  
>      foreach my $part (@parts) {
>          target_extract_jobdistpath($ho, $part, "path_${part}dist",

Re: [PATCH OSSTEST 2/4] Toolstack: Add OpenStack as a toolstack.

[Anthony PERARD]

On Fri, Jul 17, 2015 at 04:58:16PM +0100, Ian Campbell wrote:
> On Thu, 2015-07-16 at 12:18 +0100, Anthony PERARD wrote:
> > Having a separate toolstack help to configure the host specificly for
> 
> "specifically"
> 
> > OpenStack, and also helps to have a separate flight.
> 
> Will any of the die()s here ever become real commands? I think not
> because that's not how OpenStack works? i.e. how it is driven doesn't
> fit into Osstest::Toolstack's model of the world at all.

The main issue would be create an instance from a config file. The
Toolstack.create() would need to parse the cfg, extract the image that we
want to boot, upload it to Glance, extract other parameter, update the
image in Glance, prepare a flavor (disk size, ram, ..) and boot that image
with that flavor via Nova.

I think that would be possible since everything can be done from the
command line but it would not be as easy as with libvirt.

Or not...
I've look at the doc to create a new image and it's basicly use libvirt to
install an OS, install cloud-init in it, and upload the result to Glance.

So, it's look like you're right, Osstest::Toolstack does not fit well with
OpenStack.

> Given that and given that you introduce your own recipe to sg-run-job
> which doesn't/can't/won't try to do any of the usual stuff, I can't see
> any reason not to simply go with toolstack=libvirt for these test cases.
> That it is the most accurate reflection of what we want and if some
> openstack test case did want to talk to the underlying toolstack for
> some reason libvirt would be it.

Ok, I'll just have to find another way to get more RAM into dom0.

> That would also save duplicating the functionality of libvirt.pm's new
> function, which would normally call for a subclass of some sort (e.g.
> how xend is a subclass of xl). Please don't do this though ;-)

-- 
Anthony PERARD

Re: [PATCH OSSTEST 2/4] Toolstack: Add OpenStack as a toolstack.

[Ian Campbell]

On Fri, 2015-07-17 at 17:32 +0100, Anthony PERARD wrote:
> > Given that and given that you introduce your own recipe to sg-run-job
> > which doesn't/can't/won't try to do any of the usual stuff, I can't see
> > any reason not to simply go with toolstack=libvirt for these test cases.
> > That it is the most accurate reflection of what we want and if some
> > openstack test case did want to talk to the underlying toolstack for
> > some reason libvirt would be it.
> 
> Ok, I'll just have to find another way to get more RAM into dom0.

In ts-xen-install I think:
    if (toolstack($ho)->{Dom0MemFixed}) {
        $xenhopt .= " dom0_mem=512M,max:512M";
    }
should become sthg like:
    if (toolstack($ho)->{Dom0MemFixed}) {
	my $mem = $r{'dom0_mem'} // 512;
        $xenhopt .= " dom0_mem=${mem}M,max:${mem}M";
    }
and make-flight should set this new runvar only for the openstack job.

Ian.

[PATCH OSSTEST 3/4] ts-devstack: Deploy OpenStack then test it with Tempest

[Anthony PERARD]

This test configure configure a host via the ./stack.sh from devstack, it
then test it with Tempest.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 sg-run-job  |   5 +
 ts-devstack | 305 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 310 insertions(+)
 create mode 100755 ts-devstack

diff --git a/sg-run-job b/sg-run-job
index e44f79c..109f5ad 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -340,6 +340,11 @@ proc run-job/test-rumpuserxen {} {
                  ts-guest-destroy-hard            host   $g   +
 }
 
+proc need-hosts/test-devstack {} { return host }
+proc run-job/test-devstack {} {
+  run-ts . = ts-devstack
+}
+
 #---------- builds ----------
 
 proc need-hosts/build {} { return BUILD }
diff --git a/ts-devstack b/ts-devstack
new file mode 100755
index 0000000..ed6d180
--- /dev/null
+++ b/ts-devstack
@@ -0,0 +1,305 @@
+#!/usr/bin/perl
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+use strict qw(vars);
+use Osstest;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+our ($ho,$gho) = ts_get_host_guest(@ARGV);
+builddirsprops();
+
+sub tgt_init ();
+
+sub checkout () {
+  prepbuilddirs();
+  build_clone($ho, 'devstack', $builddir, 'devstack');
+  my $openstack_git_base = git_massage_url('git://git.openstack.org');
+  my $vg = target_choose_vg($ho, 10*1024); # 10GB
+  target_cmd($ho, <<END, 100);
+    cd $builddir/devstack
+    >local.conf
+    echo >>local.conf '[[local|localrc]]'
+    echo >>local.conf ADMIN_PASSWORD=`pwgen 20 1`
+    echo >>local.conf DATABASE_PASSWORD=`pwgen 20 1`
+    echo >>local.conf RABBIT_PASSWORD=`pwgen 20 1`
+    echo >>local.conf SERVICE_PASSWORD=`pwgen 20 1`
+    echo >>local.conf SERVICE_TOKEN=`pwgen 20 1`
+    echo >>local.conf \\\# make it small because there is no way to not
+    echo >>local.conf \\\# have this lvm volume created
+    echo >>local.conf VOLUME_BACKING_FILE_SIZE=500M
+    echo >>local.conf DEST=/opt/stack
+    echo >>local.conf LOGFILE=\\\$DEST/logs/stack.sh.log
+    echo >>local.conf LOG_COLOR=False
+    echo >>local.conf LIBVIRT_TYPE=xen
+    echo >>local.conf GIT_BASE="$openstack_git_base"
+    echo >>local.conf disable_service horizon
+    echo >>local.conf disable_service n-novnc
+    echo >>local.conf enable_service n-obj
+    echo >>local.conf '[[post-config|\$CINDER_CONF]]'
+    echo >>local.conf '[lvmdriver-1]'
+    echo >>local.conf volume_group = $vg
+END
+
+  # libvirt is already install, but not as a package, so avoid installation of
+  # the libvirt package with devstack
+  target_editfile($ho, "$builddir/devstack/files/debs/nova", sub {
+      while (<EI>) {
+        next if m/.*libvirt.*/;
+        print EO or die $!;
+      }
+  });
+  target_editfile($ho, "$builddir/devstack/lib/nova_plugins/functions-libvirt", sub {
+      while (<EI>) {
+        next if m/install_package.*libvirt.*/;
+        print EO or die $!;
+      }
+  });
+
+  # OpenStack needs access to libvirt from a user.
+  target_cmd_root($ho, <<END
+    echo >>/etc/libvirt/libvirtd.conf 'unix_sock_group = "libvirt"'
+    echo >>/etc/libvirt/libvirtd.conf 'unix_sock_ro_perms = "0777"'
+    echo >>/etc/libvirt/libvirtd.conf 'unix_sock_rw_perms = "0770"'
+END
+  );
+
+  # devstack is going to setup the host, install some dependency, clone
+  # openstack in /opt/stack
+  target_putfilecontents_root_stash($ho, 100, <<'END',"/etc/sudoers.d/devstack");
+osstest ALL=(ALL) NOPASSWD:ALL
+END
+
+  target_putfilecontents_root_stash($ho, 60, tgt_init(), "/etc/init.d/tgt");
+  target_cmd_root($ho, <<END);
+      chmod +x /etc/init.d/tgt
+END
+}
+
+sub deploy() {
+  # This clone many repos and may take sometime until the GitProxyCache is
+  # filled
+  target_cmd($ho, <<END, 3600);
+    cd $builddir/devstack
+    ./stack.sh
+END
+}
+
+sub tempest() {
+  target_cmd($ho, <<END, 7200);
+    /opt/stack/tempest/run_tempest.sh -V --serial -- --concurrency=2 '(?!.*\\[.*\\bslow\\b.*\\]|.*test_volume_boot_pattern)(^tempest\\.(api|scenario|thirdparty))'
+END
+}
+
+sub cleanup() {
+  # Try to have less leaked stuff.
+  target_cmd($ho, <<END, 300);
+    cd $builddir/devstack
+    ./unstack.sh
+END
+}
+
+checkout();
+deploy();
+tempest();
+cleanup();
+
+# This is missing from Debian but required by devstack
+# Got it from https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=577925
+sub tgt_init () {
+  return <<'END';
+#!/bin/bash
+### BEGIN INIT INFO
+# Provides:          tgtd
+# Required-Start:    $remote_fs $syslog
+# Required-Stop:     $remote_fs $syslog
+# Should-Start:      zfs
+# Should-Stop:       zfs
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: iscsi target daemon
+# Description:       iscsi target daemon
+### END INIT INFO
+
+DESC="target framework daemon"
+NAME=tgtd
+DAEMON=/usr/sbin/${NAME}
+
+TGTD_CONFIG=/etc/tgt/targets.conf
+
+TASK=$1
+
+. /lib/lsb/init-functions
+
+[ -x $DAEMON ] || exit 0
+
+start()
+{
+        log_daemon_msg "Starting $DESC" "$NAME"
+        # Start tgtd first.
+        tgtd &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -ne 0 ] ; then
+                log_end_msg 1
+                exit 1
+        else
+                log_end_msg 0
+        fi
+        # Put tgtd into "offline" state until all the targets are configured.
+        # We don't want initiators to (re)connect and fail the connection
+        # if it's not ready.
+        tgtadm --op update --mode sys --name State -v offline
+        # Configure the targets.
+        tgt-admin -e -c $TGTD_CONFIG
+        # Put tgtd into "ready" state.
+        tgtadm --op update --mode sys --name State -v ready
+}
+
+stop()
+{
+        if [ "$RUNLEVEL" == 0 -o "$RUNLEVEL" == 6 ] ; then
+            forcedstop
+        fi
+        log_daemon_msg "Stopping $DESC" "$NAME"
+        # Remove all targets. It only removes targets which are not in use.
+        tgt-admin --update ALL -c /dev/null &>/dev/null
+        # tgtd will exit if all targets were removed
+        tgtadm --op delete --mode system &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -eq 107 ] ; then
+                if [ "$TASK" != "restart" ] ; then
+                        log_end_msg 1
+                        exit 1
+                else
+                        log_end_msg 0
+                fi
+        elif [ "$RETVAL" -ne 0 ] ; then
+                log_end_msg 1
+                echo "Some initiators are still connected - could not stop tgtd"
+                exit 2
+        else
+                log_end_msg 0
+        fi
+        echo -n
+}
+
+forcedstop()
+{
+        # NOTE: Forced shutdown of the iscsi target may cause data corruption
+        # for initiators that are connected.
+        echo "Force-stopping target framework daemon"
+        # Offline everything first. May be needed if we're rebooting, but
+        # expect the initiators to reconnect cleanly when we boot again
+        # (i.e. we don't want them to reconnect to a tgtd which is still
+        # working, but the target is gone).
+        tgtadm --op update --mode sys --name State -v offline &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -eq 107 ] ; then
+            echo "tgtd is not running"
+            [ "$TASK" != "restart" ] && exit 1
+        else
+            tgt-admin --offline ALL
+            # Remove all targets, even if they are still in use.
+            tgt-admin --update ALL -c /dev/null -f
+            # It will shut down tgtd only after all targets were removed.
+            tgtadm --op delete --mode system
+            RETVAL=$?
+            if [ "$RETVAL" -ne 0 ] ; then
+                echo "Failed to shutdown tgtd"
+                exit 1
+            fi
+        fi
+        echo -n
+}
+
+reload()
+{
+        log_daemon_msg "Reloading configuration of $DESC" "$NAME"
+        # Update configuration for targets. Only targets which
+        # are not in use will be updated.
+        tgt-admin --update ALL -c $TGTD_CONFIG &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -eq 107 ] ; then
+                log_end_msg 1
+                echo "tgtd is not running"
+                exit 1
+        fi
+        log_end_msg 0
+}
+
+forcedreload()
+{
+        log_daemon_msg "Forced-reload configuration of $DESC" "$NAME"
+        # Update configuration for targets, even those in use.
+        tgt-admin --update ALL -f -c $TGTD_CONFIG &>/dev/null
+        RETVAL=$?
+        if [ "$RETVAL" -eq 107 ] ; then
+                log_end_msg 1
+                echo "tgtd is not running"
+                exit 1
+        else
+                log_end_msg 0
+        fi
+}
+
+status()
+{
+        # Don't name this script "tgtd"...
+        TGTD_PROC=$(ps -C tgtd | grep -c tgtd)
+        if [ "$TGTD_PROC" -eq 2 ] ; then
+            echo "tgtd is running. Run 'tgt-admin -s' to see detailed target info."
+        else
+            echo "tgtd is NOT running."
+        fi
+}
+
+case $1 in
+        start)
+                start
+                ;;
+        stop)
+                stop
+                ;;
+        forcedstop)
+                forcedstop
+                ;;
+        restart)
+                TASK=restart
+                stop && start
+                ;;
+        forcedrestart)
+                TASK=restart
+                forcedstop && start
+                ;;
+        reload)
+                reload
+                ;;
+        force-reload)
+                forcedreload
+                ;;
+        status)
+                status
+                ;;
+        *)
+                echo "Usage: $0 {start|stop|forcedstop|restart|forcedrestart|reload|force-reload|status}"
+                exit 2
+                ;;
+esac
+END
+}
-- 
Anthony PERARD

Re: [PATCH OSSTEST 3/4] ts-devstack: Deploy OpenStack then test it with Tempest

[Ian Campbell]

On Thu, 2015-07-16 at 12:18 +0100, Anthony PERARD wrote:
> +    cd $builddir/devstack
> +    >local.conf
> +    echo >>local.conf '[[local|localrc]]'
> +    echo >>local.conf ADMIN_PASSWORD=`pwgen 20 1`
> +    echo >>local.conf DATABASE_PASSWORD=`pwgen 20 1`
> +    echo >>local.conf RABBIT_PASSWORD=`pwgen 20 1`
> +    echo >>local.conf SERVICE_PASSWORD=`pwgen 20 1`
> +    echo >>local.conf SERVICE_TOKEN=`pwgen 20 1`
> +    echo >>local.conf \\\# make it small because there is no way to not
> +    echo >>local.conf \\\# have this lvm volume created
> +    echo >>local.conf VOLUME_BACKING_FILE_SIZE=500M
> +    echo >>local.conf DEST=/opt/stack
> +    echo >>local.conf LOGFILE=\\\$DEST/logs/stack.sh.log
> +    echo >>local.conf LOG_COLOR=False
> +    echo >>local.conf LIBVIRT_TYPE=xen
> +    echo >>local.conf GIT_BASE="$openstack_git_base"
> +    echo >>local.conf disable_service horizon
> +    echo >>local.conf disable_service n-novnc
> +    echo >>local.conf enable_service n-obj
> +    echo >>local.conf '[[post-config|\$CINDER_CONF]]'
> +    echo >>local.conf '[lvmdriver-1]'
> +    echo >>local.conf volume_group = $vg

target_putfilecontents_root_stash with a Perl here doc would be  better
I think?

> +[...]+
> +  # OpenStack needs access to libvirt from a user.
> +  target_cmd_root($ho, <<END
> +    echo >>/etc/libvirt/libvirtd.conf 'unix_sock_group = "libvirt"'
> +    echo >>/etc/libvirt/libvirtd.conf 'unix_sock_ro_perms = "0777"'
> +    echo >>/etc/libvirt/libvirtd.conf 'unix_sock_rw_perms = "0770"'

This one should be a bash heredoc, I think (can't be a Perl one because
this is an append?).

> +sub cleanup() {
> +  # Try to have less leaked stuff.

Leaked as in "discovered by ts-leak-check" or just a general tidy up?

If the latter I wouldn't bother.

If the former then won't this hide real issues?

Ian.

Re: [PATCH OSSTEST 3/4] ts-devstack: Deploy OpenStack then test it with Tempest

[Anthony PERARD]

On Fri, Jul 17, 2015 at 05:04:03PM +0100, Ian Campbell wrote:
> On Thu, 2015-07-16 at 12:18 +0100, Anthony PERARD wrote:
> > +    cd $builddir/devstack
> > +    >local.conf
> > +    echo >>local.conf '[[local|localrc]]'
> > +    echo >>local.conf ADMIN_PASSWORD=`pwgen 20 1`
> > +    echo >>local.conf DATABASE_PASSWORD=`pwgen 20 1`
> > +    echo >>local.conf RABBIT_PASSWORD=`pwgen 20 1`
> > +    echo >>local.conf SERVICE_PASSWORD=`pwgen 20 1`
> > +    echo >>local.conf SERVICE_TOKEN=`pwgen 20 1`
> > +    echo >>local.conf \\\# make it small because there is no way to not
> > +    echo >>local.conf \\\# have this lvm volume created
> > +    echo >>local.conf VOLUME_BACKING_FILE_SIZE=500M
> > +    echo >>local.conf DEST=/opt/stack
> > +    echo >>local.conf LOGFILE=\\\$DEST/logs/stack.sh.log
> > +    echo >>local.conf LOG_COLOR=False
> > +    echo >>local.conf LIBVIRT_TYPE=xen
> > +    echo >>local.conf GIT_BASE="$openstack_git_base"
> > +    echo >>local.conf disable_service horizon
> > +    echo >>local.conf disable_service n-novnc
> > +    echo >>local.conf enable_service n-obj
> > +    echo >>local.conf '[[post-config|\$CINDER_CONF]]'
> > +    echo >>local.conf '[lvmdriver-1]'
> > +    echo >>local.conf volume_group = $vg
> 
> target_putfilecontents_root_stash with a Perl here doc would be  better
> I think?

Will use the function. I guest I just need to replace pwgen by something in
perl. I think services are accessible from the network, which is why I'm
using pwgen.

> > +[...]+
> > +  # OpenStack needs access to libvirt from a user.
> > +  target_cmd_root($ho, <<END
> > +    echo >>/etc/libvirt/libvirtd.conf 'unix_sock_group = "libvirt"'
> > +    echo >>/etc/libvirt/libvirtd.conf 'unix_sock_ro_perms = "0777"'
> > +    echo >>/etc/libvirt/libvirtd.conf 'unix_sock_rw_perms = "0770"'
> 
> This one should be a bash heredoc, I think (can't be a Perl one because
> this is an append?).

This is append to the existing libvirtd.conf, yes. I'll use bash heredoc.
Or is target_editfile_root() can be used to add lines at the end?

> > +sub cleanup() {
> > +  # Try to have less leaked stuff.
> 
> Leaked as in "discovered by ts-leak-check" or just a general tidy up?
> 
> If the latter I wouldn't bother.
> 
> If the former then won't this hide real issues?

I've added this when I've seen many "leaked" process from OpenStack by
ts-leak-check. I can try to teach the ts- script to not considered as
leaked, process and other files from openstack.

-- 
Anthony PERARD

Re: [PATCH OSSTEST 3/4] ts-devstack: Deploy OpenStack then test it with Tempest

[Ian Campbell]

On Mon, 2015-07-20 at 15:12 +0100, Anthony PERARD wrote:
> On Fri, Jul 17, 2015 at 05:04:03PM +0100, Ian Campbell wrote:
> > On Thu, 2015-07-16 at 12:18 +0100, Anthony PERARD wrote:
> > > +    cd $builddir/devstack
> > > +    >local.conf
> > > +    echo >>local.conf '[[local|localrc]]'
> > > +    echo >>local.conf ADMIN_PASSWORD=`pwgen 20 1`
> > > +    echo >>local.conf DATABASE_PASSWORD=`pwgen 20 1`
> > > +    echo >>local.conf RABBIT_PASSWORD=`pwgen 20 1`
> > > +    echo >>local.conf SERVICE_PASSWORD=`pwgen 20 1`
> > > +    echo >>local.conf SERVICE_TOKEN=`pwgen 20 1`
> > > +    echo >>local.conf \\\# make it small because there is no way to not
> > > +    echo >>local.conf \\\# have this lvm volume created
> > > +    echo >>local.conf VOLUME_BACKING_FILE_SIZE=500M
> > > +    echo >>local.conf DEST=/opt/stack
> > > +    echo >>local.conf LOGFILE=\\\$DEST/logs/stack.sh.log
> > > +    echo >>local.conf LOG_COLOR=False
> > > +    echo >>local.conf LIBVIRT_TYPE=xen
> > > +    echo >>local.conf GIT_BASE="$openstack_git_base"
> > > +    echo >>local.conf disable_service horizon
> > > +    echo >>local.conf disable_service n-novnc
> > > +    echo >>local.conf enable_service n-obj
> > > +    echo >>local.conf '[[post-config|\$CINDER_CONF]]'
> > > +    echo >>local.conf '[lvmdriver-1]'
> > > +    echo >>local.conf volume_group = $vg
> > 
> > target_putfilecontents_root_stash with a Perl here doc would be  better
> > I think?
> 
> Will use the function. I guest I just need to replace pwgen by something in
> perl. I think services are accessible from the network, which is why I'm
> using pwgen.

Only internally within the test COLO. Given that all the hosts have the
same root password I wouldn't worry too much :-)

> 
> > > +[...]+
> > > +  # OpenStack needs access to libvirt from a user.
> > > +  target_cmd_root($ho, <<END
> > > +    echo >>/etc/libvirt/libvirtd.conf 'unix_sock_group = "libvirt"'
> > > +    echo >>/etc/libvirt/libvirtd.conf 'unix_sock_ro_perms = "0777"'
> > > +    echo >>/etc/libvirt/libvirtd.conf 'unix_sock_rw_perms = "0770"'
> > 
> > This one should be a bash heredoc, I think (can't be a Perl one because
> > this is an append?).
> 
> This is append to the existing libvirtd.conf, yes. I'll use bash heredoc.
> Or is target_editfile_root() can be used to add lines at the end?

I'd expect so, but it'll probably involve more Perl-fu than I can
summon. I think a HEREDOC is the right answer anyway.

> 
> > > +sub cleanup() {
> > > +  # Try to have less leaked stuff.
> > 
> > Leaked as in "discovered by ts-leak-check" or just a general tidy up?
> > 
> > If the latter I wouldn't bother.
> > 
> > If the former then won't this hide real issues?
> 
> I've added this when I've seen many "leaked" process from OpenStack by
> ts-leak-check. I can try to teach the ts- script to not considered as
> leaked, process and other files from openstack.

Be sure to distinguish between "expected" leaks (like a daemon which
should be running after the test) and unexpected rubbish. You'll
probably want to justify anything in the former category in the commit
log, and not to include the latter of course.

Ian.

Re: [PATCH OSSTEST 3/4] ts-devstack: Deploy OpenStack then test it with Tempest

[Ian Campbell]

On Thu, 2015-07-16 at 12:18 +0100, Anthony PERARD wrote:

> +sub deploy() {
> +  # This clone many repos and may take sometime until the GitProxyCache is
> +  # filled

Can we find those repos and therefore record the version actual in use
via store_vcs_revision?

Even without the ability to control which things get used I think this
would be useful data to have when looking into errors etc...

Re: [PATCH OSSTEST 3/4] ts-devstack: Deploy OpenStack then test it with Tempest

[Anthony PERARD]

On Fri, Jul 17, 2015 at 05:10:18PM +0100, Ian Campbell wrote:
> On Thu, 2015-07-16 at 12:18 +0100, Anthony PERARD wrote:
> 
> > +sub deploy() {
> > +  # This clone many repos and may take sometime until the GitProxyCache is
> > +  # filled
> 
> Can we find those repos and therefore record the version actual in use
> via store_vcs_revision?

Yes, that can be done. I'll look into using store_vcs_revision.

> Even without the ability to control which things get used I think this
> would be useful data to have when looking into errors etc...

-- 
Anthony PERARD

[PATCH OSSTEST 4/4] Create a flight to test OpenStack with xen-unstable and libvirt.

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ap-common            |  4 ++++
 ap-fetch-version     |  4 ++++
 ap-fetch-version-old |  5 +++++
 ap-print-url         |  3 +++
 cri-common           |  1 +
 make-flight          | 17 +++++++++++++++++
 mfi-common           |  7 +++++++
 7 files changed, 41 insertions(+)

diff --git a/ap-common b/ap-common
index dfeab9e..cb2e91a 100644
--- a/ap-common
+++ b/ap-common
@@ -57,6 +57,9 @@
 : ${PUSH_TREE_OVMF:=$XENBITS:/home/xen/git/osstest/ovmf.git}
 : ${BASE_TREE_OVMF:=git://xenbits.xen.org/osstest/ovmf.git}
 
+: ${TREE_DEVSTACK:=git://git.openstack.org/openstack-dev/devstack.git}
+#: ${BASE_TREE_DEVSTACK:=git://xenbits.xen.org/openstack-dev/devstack.git}
+
 : ${TREE_LINUXFIRMWARE:=git://xenbits.xen.org/osstest/linux-firmware.git}
 : ${PUSH_TREE_LINUXFIRMWARE:=$XENBITS:/home/osstest/ext/linux-firmware.git}
 : ${UPSTREAM_TREE_LINUXFIRMWARE:=$GIT_KERNEL_ORG/pub/scm/linux/kernel/git/firmware/linux-firmware.git}
@@ -84,6 +87,7 @@ fi
 : ${LOCALREV_RUMPUSERXEN:=daily-cron.$branch}
 : ${LOCALREV_SEABIOS:=daily-cron.$branch}
 : ${LOCALREV_OVMF:=daily-cron.$branch}
+: ${LOCALREV_DEVSTACK:=daily-cron.$branch}
 
 : ${TREEBASE_LINUX_XCP:=http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27}
 
diff --git a/ap-fetch-version b/ap-fetch-version
index 8b41acf..9aba511 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -89,6 +89,10 @@ ovmf)
 	repo_tree_rev_fetch_git ovmf \
 		$TREE_OVMF_UPSTREAM master $LOCALREV_OVMF
 	;;
+openstack)
+	repo_tree_rev_fetch_git devstack \
+		$TREE_DEVSTACK master $LOCALREV_DEVSTACK
+	;;
 osstest)
         if [ "x$OSSTEST_USE_HEAD" = "xy" ] ; then
 	    git update-ref -m "Arranging to test HEAD" \
diff --git a/ap-fetch-version-old b/ap-fetch-version-old
index 0b4739b..d12c753 100755
--- a/ap-fetch-version-old
+++ b/ap-fetch-version-old
@@ -31,6 +31,7 @@ select_xenbranch
 : ${BASE_LOCALREV_RUMPUSERXEN:=daily-cron.$branch.old}
 : ${BASE_LOCALREV_SEABIOS:=daily-cron.$branch.old}
 : ${BASE_LOCALREV_OVMF:=daily-cron.$branch.old}
+: ${BASE_LOCALREV_DEVSTACK:=daily-cron.$branch.old}
 
 : ${BASE_TREE_QEMU_UPSTREAM:=${TREE_QEMU_UPSTREAM/\/staging\//\/}}
 
@@ -97,6 +98,10 @@ ovmf)
 	repo_tree_rev_fetch_git ovmf \
 		$BASE_TREE_OVMF xen-tested-master $BASE_LOCALREV_OVMF
 	;;
+openstack)
+	repo_tree_rev_fetch_git devstack \
+		$TREE_DEVSTACK master $BASE_LOCALREV_DEVSTACK
+	;;
 osstest)
 	if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
 	    git fetch -f $HOME/testing.git production:ap-fetch
diff --git a/ap-print-url b/ap-print-url
index 3db2375..72a09c5 100755
--- a/ap-print-url
+++ b/ap-print-url
@@ -64,6 +64,9 @@ ovmf)
 osstest)
 	echo none:;
 	;;
+openstack)
+	echo $TREE_DEVSTACK
+	;;
 *)
 	echo >&2 "branch $branch ?"
 	exit 1
diff --git a/cri-common b/cri-common
index ad44546..3e817b6 100644
--- a/cri-common
+++ b/cri-common
@@ -73,6 +73,7 @@ select_xenbranch () {
 	seabios)		tree=seabios;	xenbranch=xen-unstable ;;
 	ovmf)			tree=ovmf;	xenbranch=xen-unstable ;;
 	osstest)		tree=osstest;	xenbranch=xen-unstable ;;
+	openstack)		tree=devstack;	xenbranch=xen-unstable ;;
 	esac
 	if [ "x$tree" = xlinux ]; then
 		linuxbranch=$branch
diff --git a/make-flight b/make-flight
index 3b31939..a99dfe5 100755
--- a/make-flight
+++ b/make-flight
@@ -97,6 +97,17 @@ job_create_test_filter_callback () {
         *) return 1;;
       esac
       ;;
+    openstack)
+      if [ x$toolstack != xopenstack ] ; then return 1; fi
+      case $xenarch in
+          amd64) ;;
+          *) return 1;;
+      esac
+      case $dom0arch in
+          amd64) ;;
+          *) return 1;;
+      esac
+      ;;
     *)
       case "$job" in
         *-qemuu-*)
@@ -473,6 +484,12 @@ test_matrix_do_one () {
 
   fi
   #do_passthrough_tests
+
+  job_create_test test-$xenarch$kern-$dom0arch-devstack \
+    test-devstack openstack $xenarch $dom0arch \
+    tree_devstack=$TREE_DEVSTACK \
+    revision_devstack=master \
+    all_hostflags=$most_hostflags
 }
 
 test_matrix_iterate
diff --git a/mfi-common b/mfi-common
index a9e966f..1438b66 100644
--- a/mfi-common
+++ b/mfi-common
@@ -50,6 +50,7 @@ xenbranch_xsm_variants () {
     xen-4.3-testing) echo "false";;
     xen-4.4-testing) echo "false";;
     xen-4.5-testing) echo "false";;
+    openstack)       echo "false";;
     *) echo "false true";
     esac
 }
@@ -84,6 +85,7 @@ create_build_jobs () {
       rumpuserxen) continue;;
       seabios) continue;;
       ovmf) continue;;
+      openstack) continue;;
       esac
       case "$xenbranch" in
       xen-3.*-testing) continue;;
@@ -109,6 +111,9 @@ create_build_jobs () {
       "
       ;;
     esac
+    if [ "$arch" = i386 ] && [ "$branch" = openstack ]; then
+      continue
+    fi
 
     case "$arch" in
     *)     suite=$defsuite;;
@@ -291,6 +296,8 @@ job_create_test () {
     xen-4.1-testing:*) ;;
     *:libvirt) tsbuildjob="libvirtbuildjob=${bfi}build-$dom0arch-libvirt"
             ;;
+    *:openstack) tsbuildjob="libvirtbuildjob=${bfi}build-$dom0arch-libvirt"
+            ;;
     xen-4.2-testing:*) ;;
     xen-4.3-testing:*) ;;
     *:xend) xenbuildjob="$xenbuildjob-xend"
-- 
Anthony PERARD

Re: [PATCH OSSTEST 4/4] Create a flight to test OpenStack with xen-unstable and libvirt.

[Ian Campbell]

On Thu, 2015-07-16 at 12:18 +0100, Anthony PERARD wrote:
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
>  ap-common            |  4 ++++
>  ap-fetch-version     |  4 ++++
>  ap-fetch-version-old |  5 +++++
>  ap-print-url         |  3 +++

This all looks correct, but since you hardcode master in make-flight and
haven't patches cr-daily-branch I don't think they are used? They
probably should, I suspect this is related to the queries in the cover
letter which I left until I'd read the series? Which I should now go
digest...

Ian.

Re: [PATCH OSSTEST 0/4] Have OpenStack tested on top of xen's master and libvirt's master.

[Ian Campbell]

On Thu, 2015-07-16 at 12:18 +0100, Anthony PERARD wrote:
> For the last patch, I've tried to have ./standalone make-flight have only
> the necessary jobs to run test-amd64-amd64-devstack, but there is still
> build-xsm in the list.

In my Debian distro series there is patch to add "mfi-common: Allow
make-*flight to filter the set of build jobs to include" which will be
useful for this. Ian wanted it for something too.

Last posting was
<1436358662-12886-1-git-send-email-ian.campbell@citrix.com>

Ian.

Re: [PATCH OSSTEST 0/4] Have OpenStack tested on top of xen's master and libvirt's master.

[Ian Campbell]

On Thu, 2015-07-16 at 12:18 +0100, Anthony PERARD wrote:

> I've introduce an extra Osstest::Toolstack which help to install extra
> package, 

I've commented on this.

> and use ballonning for Dom0, 500MB for Dom0 is definetly not
> enough.

This is for overriding Dom0MemFixed I think?

I think going to ballooning is a bit too far, I think instead the actual
size of dom0 ought to be something which can be overridden via a runvar,
so that the openstack tests can ask for a bigger one.

> The ts-devstack script does prepare a bit more the host, clone devstack,
> then run ./stack.sh, which is a bit like raisin. Once the machine ready,
> the integration test suite from OpenStack, Tempest, is started. Do you
> think those two step should be in separate test, one for devstack, and one
> for Tempest?

I don't really understand the difference between them well enough, but I
would say if they are separate things then different steps would be good
-- since if nothing else it gives a clearer indication in the test
report what had failed.

>   I have not done it, but we could have some smoke test before
> Tempest where osstest tryied to start a guest.

A test can have a dependency on a build job, but I'm not sure about
another test, but that would seem generally useful and would allow your
new test to depend on test-ARCH-ARCH-libvirt.

However I think we don't actually want to do that:

For the openstack branch we will be using known good versions of all the
other branches, so that test-ARCH-ARCH-libvirt will have happened
already (when the libvirt branch got pushed).

For the other branches then we already have loads of tests which might
all fail for the same reason which you could serialise to prevent this,
but it would make the whole flight take much longer and it isn't really
needed.

> Then later, there will be the question of which tree to track, devstack?
> nova? Or don't track any and just test with the master branch from time to
> time.

This is a complicated one, especially for things which don't fit into
the "one tree one branch" model of things...

In terms of gating (which matters to us for regression tracking even if
we don't care about the output of the gate) is it the case that if you
clone $rootthing (whatever that is) and select a given revision of that
you will always get the same thing?

Or is it like raisin where you clone $rootthing and select a revision of
that and it will clone the latest version of everything at that time?

I'm expecting the second one?

I think the important thing is we would want to be testing stuff which
has already gone through openstack testing?

Maybe we want to track particular OStack releases?

Ian.

Re: [PATCH OSSTEST 0/4] Have OpenStack tested on top of xen's master and libvirt's master.

[Anthony PERARD]

On Fri, Jul 17, 2015 at 05:22:10PM +0100, Ian Campbell wrote:
> On Thu, 2015-07-16 at 12:18 +0100, Anthony PERARD wrote:
> 
> > I've introduce an extra Osstest::Toolstack which help to install extra
> > package, 
> 
> I've commented on this.
> 
> > and use ballonning for Dom0, 500MB for Dom0 is definetly not
> > enough.
> 
> This is for overriding Dom0MemFixed I think?

Yes.

> I think going to ballooning is a bit too far, I think instead the actual
> size of dom0 ought to be something which can be overridden via a runvar,
> so that the openstack tests can ask for a bigger one.

I will look into that.

> > The ts-devstack script does prepare a bit more the host, clone devstack,
> > then run ./stack.sh, which is a bit like raisin. Once the machine ready,
> > the integration test suite from OpenStack, Tempest, is started. Do you
> > think those two step should be in separate test, one for devstack, and one
> > for Tempest?
> 
> I don't really understand the difference between them well enough, but I
> would say if they are separate things then different steps would be good
> -- since if nothing else it gives a clearer indication in the test
> report what had failed.

I'll seperated them in different step.

> >   I have not done it, but we could have some smoke test before
> > Tempest where osstest tryied to start a guest.
> 
> A test can have a dependency on a build job, but I'm not sure about
> another test, but that would seem generally useful and would allow your
> new test to depend on test-ARCH-ARCH-libvirt.

I was thinking of an extra steps within the test which would just start a
guest via OpenStack/Nova before running the full test suite from Tempest.

> However I think we don't actually want to do that:
> 
> For the openstack branch we will be using known good versions of all the
> other branches, so that test-ARCH-ARCH-libvirt will have happened
> already (when the libvirt branch got pushed).
> 
> For the other branches then we already have loads of tests which might
> all fail for the same reason which you could serialise to prevent this,
> but it would make the whole flight take much longer and it isn't really
> needed.
> 
> > Then later, there will be the question of which tree to track, devstack?
> > nova? Or don't track any and just test with the master branch from time to
> > time.
> 
> This is a complicated one, especially for things which don't fit into
> the "one tree one branch" model of things...
> 
> In terms of gating (which matters to us for regression tracking even if
> we don't care about the output of the gate) is it the case that if you
> clone $rootthing (whatever that is) and select a given revision of that
> you will always get the same thing?

> Or is it like raisin where you clone $rootthing and select a revision of
> that and it will clone the latest version of everything at that time?
> 
> I'm expecting the second one?

Yes, I think your description of raisin would match the description of
devstack, the $rootthing.
So, devstack will clone master of every other tree by default. But we
can select a specific revision for everything that is going to be cloned.

Also, if one clone a stable/* branch of devstack, then we'll get the same
stable branch of every other trees.

> I think the important thing is we would want to be testing stuff which
> has already gone through openstack testing?

Yes. Everything in OpenStack trees have been tested and have gone through
the gate anyway.

> Maybe we want to track particular OStack releases?

I think tracking master at first would be fine.

-- 
Anthony PERARD

Re: [PATCH OSSTEST 0/4] Have OpenStack tested on top of xen's master and libvirt's master.

[Ian Jackson]

Anthony PERARD writes ("Re: [PATCH OSSTEST 0/4] Have OpenStack tested on top of xen's master and libvirt's master."):
> Yes, I think your description of raisin would match the description of
> devstack, the $rootthing.
> So, devstack will clone master of every other tree by default. But we
> can select a specific revision for everything that is going to be cloned.

Are we trying to make a push gate out of this ?

In any case at the very least I think we need to be able to have
osstest's runvars control the versions of the subtrees.  That way the
bisector can work properly.

Ian.

Re: [PATCH OSSTEST 0/4] Have OpenStack tested on top of xen's master and libvirt's master.

[Ian Campbell]

On Mon, 2015-07-20 at 16:07 +0100, Anthony PERARD wrote:
> > >   I have not done it, but we could have some smoke test before
> > > Tempest where osstest tryied to start a guest.
> > 
> > A test can have a dependency on a build job, but I'm not sure about
> > another test, but that would seem generally useful and would allow your
> > new test to depend on test-ARCH-ARCH-libvirt.
> 
> I was thinking of an extra steps within the test which would just start a
> guest via OpenStack/Nova before running the full test suite from Tempest.

ISWYM.

That might be nice, if it were very little osstest-development effort to
make it happen, but I think it isn't going to be a small effort and
running tempest and seeing what happens seems like it would be good
enough anyway.

> > > Then later, there will be the question of which tree to track, devstack?
> > > nova? Or don't track any and just test with the master branch from time to
> > > time.
> > 
> > This is a complicated one, especially for things which don't fit into
> > the "one tree one branch" model of things...
> > 
> > In terms of gating (which matters to us for regression tracking even if
> > we don't care about the output of the gate) is it the case that if you
> > clone $rootthing (whatever that is) and select a given revision of that
> > you will always get the same thing?
> 
> > Or is it like raisin where you clone $rootthing and select a revision of
> > that and it will clone the latest version of everything at that time?
> > 
> > I'm expecting the second one?
> 
> Yes, I think your description of raisin would match the description of
> devstack, the $rootthing.
> So, devstack will clone master of every other tree by default. But we
> can select a specific revision for everything that is going to be cloned.
> 
> Also, if one clone a stable/* branch of devstack, then we'll get the same
> stable branch of every other trees.

But in the presence of stable backports we may not get the exact same
set of commits from one day to the next, despite cloning the exact same
devstack revision, I think?

> > I think the important thing is we would want to be testing stuff which
> > has already gone through openstack testing?
> 
> Yes. Everything in OpenStack trees have been tested and have gone through
> the gate anyway.
> 
> > Maybe we want to track particular OStack releases?
> 
> I think tracking master at first would be fine.

Right.