[OSSTEST PATCH v13 00/24] Have OpenStack tested on top of xen's master and libvirt's master.

[OSSTEST PATCH v13 00/24] Have OpenStack tested on top of xen's master and libvirt's master.

[Anthony PERARD]

Now powered with subunit-to-substep engine.

The Tempest test names reported via subunit are in the form:
tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario[compute,id-bdbb5441-9204-419d-a225-b4fdbfb1a1a8,image,network,volume]

so very long. Sometime, it in the form: "setUpClass ($classname)" for skipped
or failed tests preparation.

git tree:
https://xenbits.xen.org/git-http/people/aperard/osstest.git
tag: openstack-v13

(Acked, New):
A  ts-openstack-deploy: Deploy OpenStack on a host with devstack
A  ts-openstack-tempest: Run Tempest to check OpenStack
A  ts-openstack-deploy: Set http proxy
A  TestSupport: provide target_https_mitm_proxy_cert_path
A  ts-openstack-deploy: set CURL_CA_BUNDLE
A  ts-openstack-deploy: Keep CURL_CA_BUNDLE when sudo is called
A  ts-openstack-deploy: Try to disable use of SYSTEMD
A  ts-kernel-build: Enable network related modules for Neutron
A  ts-openstack-deploy: Switch to Neutron for network
   ts-openstack-deploy: Increase open fd limit for RabbitMQ
A  ts-openstack-deploy: Apply a Tempest patch
A  ts-openstack-deploy: Ignore libvirt-python version and use latest
A  ts-openstack-tempest: Fix tempest invocation
A  ts-openstack-tempest: Update list of skipped tests
A  ts-openstack-deploy: Move logs to /var/log/openstack
A  ts-logs-capture: Capture OpenStack logs
A  ts-openstack-deploy: Increase devstack timeout
 N TestSupport: Introduce target_cmd_stashed
   TestSupport: Implement target_subunit_cmd a subunit stream parser into substeps
A  ts-openstack-tempest: Use target_subunit_cmd
   Create a flight to test OpenStack with xen-unstable and libvirt
 N New branch openstack-ocata
A  make-flight: Increase dom0_mem for openstack flight
 N openstack tests: Don't run them on arm*

Changes in V13:
- parse more from subunit.
- new target_cmd_stashed
- details in patch notes

Changes in V12:
- new patches to introduce a subunit stream parser and have subunit tests
  appears as substeps.
- rework openstack flight generation, get rid of the patch that introduced
  'openstack' branch which is not used, have the flight test a stable branch of
  openstack (release Ocata), branch name: openstack-ocata

Changes in V11:
- plenty of new patches, on top of the original 3 patches that were acked.
- and an attempt at creating a flight for a stable branch of openstack. But
  there is many git tree to pull the branch from.

Anthony PERARD (21):
  ts-openstack-deploy: Deploy OpenStack on a host with devstack
  ts-openstack-tempest: Run Tempest to check OpenStack
  ts-openstack-deploy: Keep CURL_CA_BUNDLE when sudo is called
  ts-openstack-deploy: Try to disable use of SYSTEMD
  ts-kernel-build: Enable network related modules for Neutron
  ts-openstack-deploy: Switch to Neutron for network
  ts-openstack-deploy: Increase open fd limit for RabbitMQ
  ts-openstack-deploy: Apply a Tempest patch
  ts-openstack-deploy: Ignore libvirt-python version and use latest
  ts-openstack-tempest: Fix tempest invocation
  ts-openstack-tempest: Update list of skipped tests
  ts-openstack-deploy: Move logs to /var/log/openstack
  ts-logs-capture: Capture OpenStack logs
  ts-openstack-deploy: Increase devstack timeout
  TestSupport: Introduce target_cmd_stashed
  TestSupport: Implement target_subunit_cmd a subunit stream parser into
    substeps
  ts-openstack-tempest: Use target_subunit_cmd
  Create a flight to test OpenStack with xen-unstable and libvirt
  New branch openstack-ocata
  make-flight: Increase dom0_mem for openstack flight
  openstack tests: Don't run them on arm*

Ian Jackson (3):
  ts-openstack-deploy: Set http proxy
  TestSupport: provide target_https_mitm_proxy_cert_path
  ts-openstack-deploy: set CURL_CA_BUNDLE

 Osstest/TestSupport.pm | 133 ++++++++++++++++++++++++++++++++++++
 ap-common              |  19 ++++++
 ap-fetch-version       |   6 ++
 ap-fetch-version-old   |   8 +++
 ap-print-url           |   3 +
 ap-push                |   7 ++
 cr-daily-branch        |  31 +++++++++
 cr-for-branches        |   2 +-
 cri-common             |   1 +
 make-flight            |  47 ++++++++++++-
 sg-run-job             |   6 ++
 ts-kernel-build        |  17 ++++-
 ts-logs-capture        |   6 ++
 ts-openstack-deploy    | 182 +++++++++++++++++++++++++++++++++++++++++++++++++
 ts-openstack-tempest   |  63 +++++++++++++++++
 15 files changed, 527 insertions(+), 4 deletions(-)
 create mode 100755 ts-openstack-deploy
 create mode 100755 ts-openstack-tempest

-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 01/24] ts-openstack-deploy: Deploy OpenStack on a host with devstack

[Anthony PERARD]

This script installs any necessary packages and clones all of the OpenStack
trees which are used by devstack to deploy OpenStack.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 sg-run-job          |   5 ++
 ts-openstack-deploy | 148 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 153 insertions(+)
 create mode 100755 ts-openstack-deploy

diff --git a/sg-run-job b/sg-run-job
index b1f94f4..6092384 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -629,6 +629,11 @@ proc run-job/test-rumprun {} {
                  ts-guest-destroy-hard        host   $g   +
 }
 
+proc need-hosts/test-devstack {} { return host }
+proc run-job/test-devstack {} {
+    run-ts . = ts-openstack-deploy + host
+}
+
 if {[file exists sg-run-job-adhoc]} {
     source sg-run-job-adhoc
 }
diff --git a/ts-openstack-deploy b/ts-openstack-deploy
new file mode 100755
index 0000000..6f061eb
--- /dev/null
+++ b/ts-openstack-deploy
@@ -0,0 +1,148 @@
+#!/usr/bin/perl
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2016 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use Osstest;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+our ($whhost) = @ARGV;
+$whhost ||= 'host';
+our $ho = selecthost($whhost);
+our $builddir = target_jobdir($ho);
+
+sub packages () {
+    target_install_packages($ho, qw(git sudo));
+}
+sub checkout () {
+    prepbuilddirs();
+    build_clone($ho, 'openstack_cinder',       $builddir, 'cinder');
+    build_clone($ho, 'openstack_devstack',     $builddir, 'devstack');
+    build_clone($ho, 'openstack_glance',       $builddir, 'glance');
+    build_clone($ho, 'openstack_keystone',     $builddir, 'keystone');
+    build_clone($ho, 'openstack_nova',         $builddir, 'nova');
+    build_clone($ho, 'openstack_requirements', $builddir, 'requirements');
+    build_clone($ho, 'openstack_tempest',      $builddir, 'tempest');
+
+    my $vg = target_choose_vg($ho, 10*1024); # 10GB
+    target_putfilecontents_stash($ho, 60,
+        <<END, $builddir.'/devstack/local.conf');
+[[local|localrc]]
+# Everything should be cloned by osstest, so devstack don't have to do it
+ERROR_ON_CLONE=True
+USE_SCREEN=False
+ADMIN_PASSWORD=secretadmin
+DATABASE_PASSWORD=secretdatabase
+RABBIT_PASSWORD=secretrabbit
+SERVICE_PASSWORD=secretservice
+SERVICE_TOKEN=atokenserviced
+# make it small because there is no way to not have this lvm volume created
+VOLUME_BACKING_FILE_SIZE=50M
+CINDER_LVM_TYPE=default
+DEST=$builddir
+DATA_DIR=\$DEST/data
+SERVICE_DIR=\$DEST/status
+SUBUNIT_OUTPUT=\$DEST/devstack.subunit
+LOGFILE=\$DEST/logs/stack.sh.log
+LOG_COLOR=False
+LIBVIRT_TYPE=xen
+disable_service horizon
+disable_service n-novnc
+disable_service dstat
+# Disable neutron and switch back to nova-network
+disable_service q-svc
+disable_service q-dhcp
+disable_service q-meta
+disable_service q-agt
+disable_service q-l3
+enable_service n-net
+[[post-config|\$CINDER_CONF]]
+[lvmdriver-1]
+volume_group = $vg
+END
+
+    # libvirt is already installed, but not as a package, so avoid installation
+    # of the libvirt package with devstack
+    # https://bugs.launchpad.net/devstack/+bug/1641144
+    target_editfile($ho, "$builddir/devstack/files/debs/nova", sub {
+        while (<EI>) {
+            next if m/.*libvirt.*/;
+            print EO or die $!;
+        }
+    });
+    target_editfile($ho,
+        "$builddir/devstack/lib/nova_plugins/functions-libvirt",
+        sub {
+            while (<EI>) {
+                if (m/install_package.*libvirt.*/) {
+                    s#install_package.*#:#
+                }
+                print EO or die $!;
+            }
+        }
+    );
+
+    # Package python-systemd does not exist in Debian installed by osstest
+    target_editfile($ho, "$builddir/devstack/files/debs/general", sub {
+        while (<EI>) {
+            next if m/^python-systemd$/;
+            print EO or die $!;
+        }
+    });
+
+    # devstack blindly assume that systemd is used if systemctl is present
+    # https://bugs.launchpad.net/devstack/+bug/1641112
+    target_editfile($ho, "$builddir/devstack/functions-common", sub {
+        while (<EI>) {
+            if (m#\[ -x /bin/systemctl#) {
+                s#\[ -x /bin/systemctl \]#false#
+            }
+            print EO or die $!;
+        }
+    });
+
+    # OpenStack needs access to libvirt from a user.
+    target_cmd_root($ho, <<END);
+        set -e
+        addgroup --system libvirt
+        cat >> /etc/libvirt/libvirtd.conf <<EOF
+unix_sock_group = "libvirt"
+unix_sock_ro_perms = "0777"
+unix_sock_rw_perms = "0770"
+EOF
+        service libvirtd restart
+END
+
+    # devstack is going to setup the host, install some dependency.
+    target_putfilecontents_root_stash($ho, 100,
+        <<END, "/etc/sudoers.d/devstack");
+osstest ALL=(ALL) NOPASSWD:ALL
+END
+}
+
+sub deploy() {
+    target_cmd($ho, <<END, 1800);
+        set -e
+        cd $builddir/devstack
+        ./stack.sh
+END
+}
+
+packages();
+checkout();
+deploy();
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 02/24] ts-openstack-tempest: Run Tempest to check OpenStack

[Anthony PERARD]

This script runs the OpenStack integration test suite, Tempest.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 sg-run-job           |  1 +
 ts-openstack-tempest | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+)
 create mode 100755 ts-openstack-tempest

diff --git a/sg-run-job b/sg-run-job
index 6092384..5f15821 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -632,6 +632,7 @@ proc run-job/test-rumprun {} {
 proc need-hosts/test-devstack {} { return host }
 proc run-job/test-devstack {} {
     run-ts . = ts-openstack-deploy + host
+    run-ts . = ts-openstack-tempest + host
 }
 
 if {[file exists sg-run-job-adhoc]} {
diff --git a/ts-openstack-tempest b/ts-openstack-tempest
new file mode 100755
index 0000000..82e9a71
--- /dev/null
+++ b/ts-openstack-tempest
@@ -0,0 +1,65 @@
+#!/usr/bin/perl
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2016 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use Osstest;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+our ($whhost) = @ARGV;
+$whhost ||= 'host';
+our $ho = selecthost($whhost);
+our $builddir = target_jobdir($ho);
+
+sub tempest() {
+    my @ignored_tests;
+    my $scenario = 'tempest.scenario';
+    my $volume_boot_pattern =
+        "$scenario.test_volume_boot_pattern.TestVolumeBootPattern";
+    my $shelve_instance = "$scenario.test_shelve_instance.TestShelveInstance";
+
+    # Ignore tests which try to boot a guest with /dev/vda as boot device name.
+    push @ignored_tests,
+        "^\Q$volume_boot_pattern.test_volume_boot_pattern\E";
+    push @ignored_tests,
+        "^\Q$volume_boot_pattern.test_create_ebs_image_and_check_boot\E";
+    push @ignored_tests,
+        "^\Q$shelve_instance.test_shelve_volume_backed_instance\E";
+
+    # Those tests access a volume through iSCSI. This does not work when both
+    # the server and client of iSCSI are on the same Xen host (both in dom0),
+    # Linux 4.0 is the first Linux to have a fix.
+    push @ignored_tests,
+        "^\Q${volume_boot_pattern}V2.test_volume_boot_pattern\E";
+    push @ignored_tests,
+        "^\Q${volume_boot_pattern}V2.test_create_ebs_image_and_check_boot\E";
+
+    # This regex below select the tests to run and exclude the ones marked as
+    # slow as well as the explicit tests listed above.  It is based on the one
+    # that can be found in tempest.git/tox.ini in the section [testenv:full].
+    my $ignored_tests = join("|", @ignored_tests);
+    my $regex = "(?!.*\\[.*\\bslow\\b.*\\]|$ignored_tests)".
+        "(^tempest\\.(api|scenario|thirdparty))";
+
+    target_cmd($ho, <<END, 7200);
+set -e
+$builddir/tempest/run_tempest.sh --virtual-env -- --concurrency=2 '$regex'
+END
+}
+
+tempest();
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 03/24] ts-openstack-deploy: Set http proxy

[Anthony PERARD]

From: Ian Jackson <ian.jackson@eu.citrix.com>

This allows ./stack.sh to access the global internet.

CC: Anthony PERARD <anthony.perard@citrix.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 ts-openstack-deploy | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index 6f061eb..d2971f5 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -136,8 +136,11 @@ END
 }
 
 sub deploy() {
+    my $httpproxy = http_proxy_envsettings($ho);
+
     target_cmd($ho, <<END, 1800);
         set -e
+        $httpproxy
         cd $builddir/devstack
         ./stack.sh
 END
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 04/24] TestSupport: provide target_https_mitm_proxy_cert_path

[Anthony PERARD]

From: Ian Jackson <ian.jackson@eu.citrix.com>

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 6e19b28..7215156 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -64,6 +64,7 @@ BEGIN {
                       target_put_guest_image target_editfile
                       target_editfile_cancel target_fetchurl
                       http_proxy_envsettings
+                      target_https_mitm_proxy_cert_path
                       target_editfile_root target_file_exists
                       target_editfile_kvp_replace
                       target_run_apt
@@ -2714,4 +2715,10 @@ sub target_https_mitm_proxy_setup ($) {
     target_cmd_root($ho, 'update-ca-certificates', 300);
 }
 
+sub target_https_mitm_proxy_cert_path ($) {
+    my ($ho) = @_;
+    return undef unless length $c{HttpsProxyMITMCert};
+    return '/etc/ssl/certs/osstest.pem';
+}
+
 1;
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 05/24] ts-openstack-deploy: set CURL_CA_BUNDLE

[Anthony PERARD]

From: Ian Jackson <ian.jackson@eu.citrix.com>

This overrides pip's attempt to specify a specific certificate bundle,
and is necessary if we have a MITM SSL proxy.

The security implications are not ideal, because the MITM proxy will
allow any X.509 cert from any CA, whereas pip would only allow an
expected cert.  But we got pip via plain https to start with...

CC: Anthony PERARD <anthony.perard@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-openstack-deploy | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index d2971f5..6d7de1c 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -137,7 +137,10 @@ END
 
 sub deploy() {
     my $httpproxy = http_proxy_envsettings($ho);
-
+    my $mitmcert = target_https_mitm_proxy_cert_path($ho);
+    $httpproxy .=
+        "\n        CURL_CA_BUNDLE=$mitmcert; export CURL_CA_BUNDLE"
+        if $mitmcert;
     target_cmd($ho, <<END, 1800);
         set -e
         $httpproxy
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 06/24] ts-openstack-deploy: Keep CURL_CA_BUNDLE when sudo is called

[Anthony PERARD]

This is part of commit "ts-openstack-deploy: set CURL_CA_BUNDLE" but
also allow pip to work when it is called via sudo without preserving the
existing environment variables.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-openstack-deploy | 1 +
 1 file changed, 1 insertion(+)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index 6d7de1c..147071f 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -132,6 +132,7 @@ END
     target_putfilecontents_root_stash($ho, 100,
         <<END, "/etc/sudoers.d/devstack");
 osstest ALL=(ALL) NOPASSWD:ALL
+Defaults:osstest env_keep += "CURL_CA_BUNDLE"
 END
 }
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 07/24] ts-openstack-deploy: Try to disable use of SYSTEMD

[Anthony PERARD]

There is USE_SYSTEMD off by default, but it is now turned on if
USE_SCREEN if off. Try to keep use of systemd disabled.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-openstack-deploy | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index 147071f..1349009 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -45,7 +45,6 @@ sub checkout () {
 [[local|localrc]]
 # Everything should be cloned by osstest, so devstack don't have to do it
 ERROR_ON_CLONE=True
-USE_SCREEN=False
 ADMIN_PASSWORD=secretadmin
 DATABASE_PASSWORD=secretdatabase
 RABBIT_PASSWORD=secretrabbit
@@ -71,6 +70,9 @@ disable_service q-meta
 disable_service q-agt
 disable_service q-l3
 enable_service n-net
+USE_SYSTEMD=False
+# To keep systemd off, we need to enable use of screen
+USE_SCREEN=True
 [[post-config|\$CINDER_CONF]]
 [lvmdriver-1]
 volume_group = $vg
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 08/24] ts-kernel-build: Enable network related modules for Neutron

[Anthony PERARD]

Those options/modules are needed to run OpenStack Neutron with Open
vSwitch.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-kernel-build | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/ts-kernel-build b/ts-kernel-build
index 94e67a4..0bcd340 100755
--- a/ts-kernel-build
+++ b/ts-kernel-build
@@ -252,10 +252,23 @@ setopt CONFIG_BLK_DEV_LOOP y
 
 setopt CONFIG_PACKET y
 
-# needed for OpenStack
-# because: https://bugzilla.redhat.com/show_bug.cgi?id=910619#c6
+# Used by OpenStack Neutron with Open vSwitch
+setopt CONFIG_OPENVSWITCH m
+setopt CONFIG_IP6_NF_RAW m
+setopt CONFIG_IP_NF_RAW m
+setopt CONFIG_IP_SET m
+setopt CONFIG_IP_SET_HASH_NET m
 setopt CONFIG_NETFILTER_ADVANCED y
+setopt CONFIG_NETFILTER_XT_CONNMARK m
+setopt CONFIG_NETFILTER_XT_MATCH_COMMENT m
+setopt CONFIG_NETFILTER_XT_MATCH_MAC m
+setopt CONFIG_NETFILTER_XT_MATCH_PHYSDEV m
+setopt CONFIG_NETFILTER_XT_SET m
 setopt CONFIG_NETFILTER_XT_TARGET_CHECKSUM m
+setopt CONFIG_NETFILTER_XT_TARGET_CT m
+setopt CONFIG_NETFILTER_XT_TARGET_REDIRECT m
+setopt CONFIG_NF_CONNTRACK_ZONES y
+setopt CONFIG_VETH m
 
 # Used by OpenStack Tempest to test encrypted volume
 setopt CONFIG_CRYPTO_XTS m
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 09/24] ts-openstack-deploy: Switch to Neutron for network

[Anthony PERARD]

nova-network is not supported anymore and Neutron is the default.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-openstack-deploy | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index 1349009..2107760 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -35,6 +35,7 @@ sub checkout () {
     build_clone($ho, 'openstack_devstack',     $builddir, 'devstack');
     build_clone($ho, 'openstack_glance',       $builddir, 'glance');
     build_clone($ho, 'openstack_keystone',     $builddir, 'keystone');
+    build_clone($ho, 'openstack_neutron',      $builddir, 'neutron');
     build_clone($ho, 'openstack_nova',         $builddir, 'nova');
     build_clone($ho, 'openstack_requirements', $builddir, 'requirements');
     build_clone($ho, 'openstack_tempest',      $builddir, 'tempest');
@@ -63,13 +64,6 @@ LIBVIRT_TYPE=xen
 disable_service horizon
 disable_service n-novnc
 disable_service dstat
-# Disable neutron and switch back to nova-network
-disable_service q-svc
-disable_service q-dhcp
-disable_service q-meta
-disable_service q-agt
-disable_service q-l3
-enable_service n-net
 USE_SYSTEMD=False
 # To keep systemd off, we need to enable use of screen
 USE_SCREEN=True
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 10/24] ts-openstack-deploy: Increase open fd limit for RabbitMQ

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-openstack-deploy | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index 2107760..f677513 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -130,6 +130,13 @@ END
 osstest ALL=(ALL) NOPASSWD:ALL
 Defaults:osstest env_keep += "CURL_CA_BUNDLE"
 END
+
+    # Increase open fd limit of RabbitMQ server (message broker)
+    # https://bugs.launchpad.net/devstack/+bug/1703651
+    target_putfilecontents_root_stash($ho, 100,
+        <<END, "/etc/default/rabbitmq-server");
+ulimit -n 65536
+END
 }
 
 sub deploy() {
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 11/24] ts-openstack-deploy: Apply a Tempest patch

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-openstack-deploy | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index f677513..befe3d3 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -137,6 +137,15 @@ END
         <<END, "/etc/default/rabbitmq-server");
 ulimit -n 65536
 END
+
+    # Apply https://review.openstack.org/449695/ to tempest to workaround an
+    # issue. Check comments for more information
+    target_cmd($ho, <<END, 120);
+set -e
+cd $builddir/tempest
+git fetch origin refs/changes/95/449695/1
+git cherry-pick FETCH_HEAD
+END
 }
 
 sub deploy() {
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 12/24] ts-openstack-deploy: Ignore libvirt-python version and use latest

[Anthony PERARD]

Devstack is going to try to install a specific version of libvirt-python
(currently 2.5.0) but this fail with libvirt installed by osstest.
Remove the requirement and use the latest available instead.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-openstack-deploy | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index befe3d3..00f262f 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -93,6 +93,21 @@ END
         }
     );
 
+    target_editfile($ho,
+        "$builddir/requirements/upper-constraints.txt",
+        sub {
+            while (<EI>) {
+                # Ignore libvirt-python requirement and install latest,
+                # otherwise it's not going to work with latest libvirt
+                # installed by osstest.
+                if (m/^libvirt-python===.*$/) {
+                    next;
+                }
+                print EO or die $!;
+            }
+        }
+    );
+
     # Package python-systemd does not exist in Debian installed by osstest
     target_editfile($ho, "$builddir/devstack/files/debs/general", sub {
         while (<EI>) {
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 13/24] ts-openstack-tempest: Fix tempest invocation

[Anthony PERARD]

./run_tempest.sh is deprecated.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-openstack-tempest | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/ts-openstack-tempest b/ts-openstack-tempest
index 82e9a71..b95043a 100755
--- a/ts-openstack-tempest
+++ b/ts-openstack-tempest
@@ -58,7 +58,8 @@ sub tempest() {
 
     target_cmd($ho, <<END, 7200);
 set -e
-$builddir/tempest/run_tempest.sh --virtual-env -- --concurrency=2 '$regex'
+cd $builddir/tempest
+tempest run --concurrency=2 --regex '$regex'
 END
 }
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 14/24] ts-openstack-tempest: Update list of skipped tests

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-openstack-tempest | 19 ++++++++-----------
 1 file changed, 8 insertions(+), 11 deletions(-)

diff --git a/ts-openstack-tempest b/ts-openstack-tempest
index b95043a..ae3662f 100755
--- a/ts-openstack-tempest
+++ b/ts-openstack-tempest
@@ -31,23 +31,20 @@ sub tempest() {
     my $scenario = 'tempest.scenario';
     my $volume_boot_pattern =
         "$scenario.test_volume_boot_pattern.TestVolumeBootPattern";
-    my $shelve_instance = "$scenario.test_shelve_instance.TestShelveInstance";
-
-    # Ignore tests which try to boot a guest with /dev/vda as boot device name.
-    push @ignored_tests,
-        "^\Q$volume_boot_pattern.test_volume_boot_pattern\E";
-    push @ignored_tests,
-        "^\Q$volume_boot_pattern.test_create_ebs_image_and_check_boot\E";
-    push @ignored_tests,
-        "^\Q$shelve_instance.test_shelve_volume_backed_instance\E";
 
     # Those tests access a volume through iSCSI. This does not work when both
     # the server and client of iSCSI are on the same Xen host (both in dom0),
     # Linux 4.0 is the first Linux to have a fix.
     push @ignored_tests,
-        "^\Q${volume_boot_pattern}V2.test_volume_boot_pattern\E";
+        "^\Q${volume_boot_pattern}.test_volume_boot_pattern\E";
+    push @ignored_tests,
+        "^\Q${volume_boot_pattern}.test_create_ebs_image_and_check_boot\E";
+
+    # See nova.git:devstack/tempest-dsvm-tempest-xen-rc
+    push @ignored_tests,
+        "^\Qtempest.api.compute.admin.test_volume_swap.TestVolumeSwap.test_volume_swap\E";
     push @ignored_tests,
-        "^\Q${volume_boot_pattern}V2.test_create_ebs_image_and_check_boot\E";
+        "^\Qtempest.api.compute.images.test_images.ImagesTestJSON.test_create_image_from_paused_server\E";
 
     # This regex below select the tests to run and exclude the ones marked as
     # slow as well as the explicit tests listed above.  It is based on the one
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 15/24] ts-openstack-deploy: Move logs to /var/log/openstack

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-openstack-deploy | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index 00f262f..e7c94a5 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -58,7 +58,7 @@ DEST=$builddir
 DATA_DIR=\$DEST/data
 SERVICE_DIR=\$DEST/status
 SUBUNIT_OUTPUT=\$DEST/devstack.subunit
-LOGFILE=\$DEST/logs/stack.sh.log
+LOGDIR=/var/log/openstack
 LOG_COLOR=False
 LIBVIRT_TYPE=xen
 disable_service horizon
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 16/24] ts-logs-capture: Capture OpenStack logs

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-logs-capture | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/ts-logs-capture b/ts-logs-capture
index 061a118..0e3d267 100755
--- a/ts-logs-capture
+++ b/ts-logs-capture
@@ -171,6 +171,12 @@ sub fetch_logs_host () {
 
                   /var/core/*.core
 
+                  /var/log/openstack/*.log
+                  /etc/nova/*
+                  /etc/neutron/*
+                  /etc/cinder/*
+                  /home/osstest/build.*.test-*-devstack/tempest/etc/tempest.conf
+
                   )];
     if (!try_fetch_logs($ho, $logs)) {
         logm("log fetching failed, trying hard host reboot...");
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 17/24] ts-openstack-deploy: Increase devstack timeout

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-openstack-deploy | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index e7c94a5..875c4a7 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -169,7 +169,7 @@ sub deploy() {
     $httpproxy .=
         "\n        CURL_CA_BUNDLE=$mitmcert; export CURL_CA_BUNDLE"
         if $mitmcert;
-    target_cmd($ho, <<END, 1800);
+    target_cmd($ho, <<END, 3600);
         set -e
         $httpproxy
         cd $builddir/devstack
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 18/24] TestSupport: Introduce target_cmd_stashed

[Anthony PERARD]

This works like target_cmd, but takes a ref to a filename as argument
and stash the output of the command then return a path to the stashed
output.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/TestSupport.pm | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 7215156..135289b 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -55,6 +55,7 @@ BEGIN {
 
                       target_cmd_root_status target_cmd_output_root_status
                       target_cmd_root target_cmd target_cmd_build
+                      target_cmd_stashed
                       target_cmd_output_root target_cmd_output
                       target_cmd_inputfh_root sshuho
                       target_getfile target_getfile_root
@@ -764,6 +765,16 @@ sub target_cmd_inputfh_root ($$$;$$) {
     tcmd($stdinfh,undef,0,'root',$tho,$tcmd,@rest);
 }
 
+# Like target_cmd, but stash cmd stdout and return a path to it.
+sub target_cmd_stashed ($$$;$$) {
+    my ($tho,$leafref,$tcmd,$timeout,$extrasshopts) = @_;
+    my $stdout = open_unique_stashfile($leafref);
+    my $rc = tcmd(undef, $stdout, 0, 'osstest', $tho, $tcmd, $timeout,
+        $extrasshopts);
+    die "$stdout $!" if $stdout->error or !close $stdout;
+    return "$stash/$$leafref";
+}
+
 sub poll_loop ($$$&) {
     my ($maxwait, $interval, $what, $code) = @_;
     # $code should return undef when all is well
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 19/24] TestSupport: Implement target_subunit_cmd a subunit stream parser into substeps

[Anthony PERARD]

target_subunit_cmd can be used like target_cmd, but the command would
needs to output a subunit v1 stream, which will be parsed and turned
into osstest substeps. The command can be `| subunit-2to1` in order to
turn a subunit v2 stream into v1.

Currently, time is not taken into account, and all substeps will have
bogus timestamp as the output of the command is parsed after it has
runned.

This is a description of the subunit v1 protocol, taken from
python-subunit README, or https://pypi.python.org/pypi/python-subunit

test|testing|test:|testing: test LABEL
success|success:|successful|successful: test LABEL
success|success:|successful|successful: test LABEL DETAILS
failure: test LABEL
failure: test LABEL DETAILS
error: test LABEL
error: test LABEL DETAILS
skip[:] test LABEL
skip[:] test LABEL DETAILS
xfail[:] test LABEL
xfail[:] test LABEL DETAILS
uxsuccess[:] test LABEL
uxsuccess[:] test LABEL DETAILS
progress: [+|-]X
progress: push
progress: pop
tags: [-]TAG ...
time: YYYY-MM-DD HH:MM:SSZ

LABEL: UTF8*
NAME: UTF8*
DETAILS ::= BRACKETED | MULTIPART
BRACKETED ::= '[' CR UTF8-lines ']' CR
MULTIPART ::= '[ multipart' CR PART* ']' CR
PART ::= PART_TYPE CR NAME CR PART_BYTES CR
PART_TYPE ::= Content-Type: type/sub-type(;parameter=value,parameter=value)
PART_BYTES ::= (DIGITS CR LF BYTE{DIGITS})* '0' CR LF

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    Changes in v13:
    - also parse multipart output
    - add every possible test result
    - use target_cmd_stashed

 Osstest/TestSupport.pm | 117 ++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 116 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 135289b..ba55967 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -55,7 +55,7 @@ BEGIN {
 
                       target_cmd_root_status target_cmd_output_root_status
                       target_cmd_root target_cmd target_cmd_build
-                      target_cmd_stashed
+                      target_cmd_stashed target_subunit_cmd
                       target_cmd_output_root target_cmd_output
                       target_cmd_inputfh_root sshuho
                       target_getfile target_getfile_root
@@ -775,6 +775,121 @@ sub target_cmd_stashed ($$$;$$) {
     return "$stash/$$leafref";
 }
 
+sub subunit_result_to_osstest_result ($) {
+    my ($ret) = @_;
+    return "pass" if $ret eq "success" or $ret eq "successful";
+    return "fail" if $ret eq "failure";
+    return "skip" if $ret eq "skip";
+    return "fail" if $ret eq "error";
+    # expected failure
+    return "pass" if $ret eq "xfail";
+    # unexpected success
+    return "fail" if $ret eq "uxsuccess";
+    die "subunit_result_to_osstest_result unexpected result $ret";
+}
+sub subunit_sanitize_testname ($) {
+    my ($testname) = @_;
+    $testname =~ s'[^_.()\[\]/~0-9a-zA-Z-]'_'g;
+    return $testname;
+}
+
+# Like target_cmd, but parse the command output as a subunit v1 stream and make
+# a substep out of each subunit test.
+sub target_subunit_cmd ($$;$$) {
+    my ($tho,$tcmd,$timeout,$extrasshopts) = @_;
+    my $filename = "subunit-output";
+    my $path = target_cmd_stashed($tho, \$filename, $tcmd, $timeout,
+        $extrasshopts);
+
+    open my $stdout, "$path" or die "$path: $!";
+
+    my $logfilename = undef;
+    my $fh = undef;
+
+    while (<$stdout>) {
+        if (/^time: (\d+)-(\d+)-(\d+) (\d+):(\d+):(\d+)(\.\d+)?Z$/) {
+            # This is the timestamp for the next events
+        } elsif (/^test(?:ing)?:? (.+)\n/) {
+            # Start of a new test.
+            $logfilename = subunit_sanitize_testname($1) . '.log';
+            $fh = open_unique_stashfile(\$logfilename);
+            substep_start(subunit_sanitize_testname($1), $logfilename);
+        } elsif (/^(success(?:ful)?|failure|skip|error|xfail|uxsuccess):
+                   \ (.+?)(\ \[(\ multipart)?)?$/x) {
+            # Result of a test, with its output.
+            my $result = $1;
+            my $testname = $2;
+            my $have_details = $3;
+            my $is_multipart = $4;
+
+            if ($have_details) {
+                if ($is_multipart) {
+                    # Test output
+                    while (<$stdout>) {
+                        # part content-type
+                        # from https://tools.ietf.org/html/rfc6838#section-4.2
+                        my $restricted_name = qr'[a-zA-Z0-9][a-zA-Z0-9!#$&^_.+-]*';
+                        if (m{ ^Content-Type:\s+
+                                $restricted_name/$restricted_name # type/sub-type
+                                # parameters
+                                (?:\s*;\s*$restricted_name=[^,]+
+                                  (?:,\s*$restricted_name=[^,]+)*)
+                                \s*$
+                            }xi) {
+                            print $fh $_ or die $!;
+
+                            # part name
+                            my $line = <$stdout>;
+                            print $fh $line or die $!;
+
+                            # Read chunks of a part
+                            while (<$stdout>) {
+                                if (/^([0-9A-F]+)\r$/i) {
+                                    my $chunk_size = hex($1);
+                                    my $chunk;
+
+                                    last if $chunk_size == 0;
+                                    read $stdout, $chunk, $chunk_size;
+                                    print $fh $chunk or die $!;
+                                } else {
+                                    # Unexpected output
+                                    chomp;
+                                    logm("*** $_");
+                                }
+                            }
+                        } elsif (/^\]$/) {
+                            last;
+                        } else {
+                            # Unexpected output
+                            chomp;
+                            logm("*** $_");
+                        }
+                    }
+                } else {
+                    # Simple non-multipart test output.
+                    while (<$stdout>) {
+                        last if (/^\]$/);
+                        print $fh $_ or die $!;
+                    }
+                }
+            }
+            close $fh or die $!;
+            substep_finish(subunit_sanitize_testname($testname),
+                subunit_result_to_osstest_result($result));
+        } elsif (/^tags: .+/) {
+            # unused
+        } elsif (/^progress: (?:[+-]?\d+|push|pop)$/) {
+            # unused
+        } else {
+            # Unexpected output
+            chomp;
+            logm("*** $_");
+        }
+    }
+
+    close $stdout or die $!;
+}
+
 sub poll_loop ($$$&) {
     my ($maxwait, $interval, $what, $code) = @_;
     # $code should return undef when all is well
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 20/24] ts-openstack-tempest: Use target_subunit_cmd

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-openstack-tempest | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ts-openstack-tempest b/ts-openstack-tempest
index ae3662f..596142a 100755
--- a/ts-openstack-tempest
+++ b/ts-openstack-tempest
@@ -53,10 +53,10 @@ sub tempest() {
     my $regex = "(?!.*\\[.*\\bslow\\b.*\\]|$ignored_tests)".
         "(^tempest\\.(api|scenario|thirdparty))";
 
-    target_cmd($ho, <<END, 7200);
+    target_subunit_cmd($ho, <<END, 7200);
 set -e
 cd $builddir/tempest
-tempest run --concurrency=2 --regex '$regex'
+tempest run --concurrency=2 --subunit --regex '$regex' | subunit-2to1
 END
 }
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 21/24] Create a flight to test OpenStack with xen-unstable and libvirt

[Anthony PERARD]

This patch creates a flight "openstack*", with those jobs:
  build-amd64
  build-amd64-libvirt
  build-amd64-pvops
  build-amd64-xsm
  build-arm64
  build-arm64-libvirt
  build-arm64-pvops
  build-arm64-xsm
  build-armhf
  build-armhf-libvirt
  build-armhf-pvops
  build-armhf-xsm
  test-amd64-amd64-devstack
  test-amd64-amd64-devstack-xsm
  test-arm64-arm64-devstack
  test-arm64-arm64-devstack-xsm
  test-armhf-armhf-devstack
  test-armhf-armhf-devstack-xsm

This would be a flight to test a stable release of OpenStack against
Xen.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    Changes in V13:
    - split changes in ap-* and cr-* into a different patch
    - rename os_* to openstack_*
    - rename target_cmd_subunit to target_subunit_cmd

 make-flight | 46 +++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 45 insertions(+), 1 deletion(-)

diff --git a/make-flight b/make-flight
index db124fe..e03aacc 100755
--- a/make-flight
+++ b/make-flight
@@ -183,6 +183,16 @@ job_create_test_filter_callback () {
         *) return 1;;
       esac
       ;;
+    openstack*)
+      case "$job" in
+        *-devstack) ;;
+        *-devstack-xsm) ;;
+        *) return 1;;
+      esac
+      case $dom0arch in
+          i386) return 1 ;;
+      esac
+      ;;
     *)
       case "$job" in
         *-qemuu-*)
@@ -216,7 +226,13 @@ arch_branch_filter_callback () {
         ovmf) return 1;;
         esac
         ;;
-  i386|amd64)
+  i386)
+        case "$branch" in
+        linux-arm-xen) return 1;;
+        openstack*) return 1;;
+        esac
+        ;;
+  amd64)
         case "$branch" in
         linux-arm-xen) return 1;;
         esac
@@ -680,6 +696,33 @@ do_examine_one () {
                   all_hostflags=$most_hostflags
 }
 
+do_openstack_tests () {
+    local xsms=$(xenbranch_xsm_variants)
+    local openstack_trees=(cinder devstack glance keystone neutron nova
+        requirements tempest)
+
+    case "$branch" in
+        openstack*)          ;;
+        *)           return  ;;
+    esac
+
+    local openstack_runvars=""
+    for tree in "${openstack_trees[@]}"; do
+        tree="openstack_$tree"
+        eval "openstack_runvars+=\" tree_$tree=\${TREE_${tree^^}}\""
+        eval "openstack_runvars+=\" revision_$tree=\${REVISION_${tree^^}}\""
+    done
+
+    for xsm in $xsms ; do
+        job_create_test test-$xenarch$kern-$dom0arch-devstack \
+            test-devstack libvirt $xenarch $dom0arch \
+            $openstack_runvars \
+            dom0_mem=4000 \
+            enable_xsm=$xsm \
+            all_hostflags=$most_hostflags
+    done
+}
+
 test_matrix_do_one () {
 
   do_pv_debian_tests
@@ -700,6 +743,7 @@ test_matrix_do_one () {
 
   do_rtds_tests
   do_credit2_tests
+  do_openstack_tests
 
   do_examine_one
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 22/24] New branch openstack-ocata

[Anthony PERARD]

Testing of the Ocata stable branch of OpenStack against Xen unstable.

OpenStack have many different repo which should be in sync, so we should
attempd to grab the revisions of the stable branch of every OpenStack
tree, but for now, the runvars REVISION_* of tree other than nova is set
to "origin/stable/ocata", except Tempest does not have stable branch and
should be able to test any OpenStack version.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ap-common            | 19 +++++++++++++++++++
 ap-fetch-version     |  6 ++++++
 ap-fetch-version-old |  8 ++++++++
 ap-print-url         |  3 +++
 ap-push              |  7 +++++++
 cr-daily-branch      | 31 +++++++++++++++++++++++++++++++
 cr-for-branches      |  2 +-
 cri-common           |  1 +
 8 files changed, 76 insertions(+), 1 deletion(-)

diff --git a/ap-common b/ap-common
index bc7c03c..72496fd 100644
--- a/ap-common
+++ b/ap-common
@@ -54,6 +54,24 @@
 : ${PUSH_TREE_OVMF:=$XENBITS:/home/xen/git/osstest/ovmf.git}
 : ${BASE_TREE_OVMF:=git://xenbits.xen.org/osstest/ovmf.git}
 
+define_openstack_trees() {
+    local openstack_trees=(cinder glance keystone neutron nova requirements
+        tempest)
+    local tree
+    local url
+
+    : ${GIT_OPENSTACK_ORG:=git://git.openstack.org}
+    : ${TREE_OPENSTACK_DEVSTACK:=$GIT_OPENSTACK_ORG/openstack-dev/devstack.git}
+    for tree in "${openstack_trees[@]}"; do
+        url=$GIT_OPENSTACK_ORG/openstack/$tree.git
+        eval ": \${TREE_OPENSTACK_${tree^^}:=$url}"
+    done
+}
+
+define_openstack_trees
+: ${PUSH_TREE_OPENSTACK_NOVA:=$XENBITS:/home/xen/git/osstest/openstack-nova.git}
+: ${BASE_TREE_OPENSTACK_NOVA:=git://xenbits.xen.org/osstest/openstack-nova.git}
+
 : ${TREE_LINUXFIRMWARE:=git://xenbits.xen.org/osstest/linux-firmware.git}
 : ${PUSH_TREE_LINUXFIRMWARE:=$XENBITS:/home/osstest/ext/linux-firmware.git}
 : ${UPSTREAM_TREE_LINUXFIRMWARE:=$GIT_KERNEL_ORG/pub/scm/linux/kernel/git/firmware/linux-firmware.git}
@@ -82,6 +100,7 @@ fi
 : ${LOCALREV_SEABIOS:=daily-cron.$branch}
 : ${LOCALREV_OVMF:=daily-cron.$branch}
 : ${LOCALREV_XTF:=daily-cron.$branch}
+: ${LOCALREV_OPENSTACK_NOVA:=daily-cron.$branch}
 
 : ${TREEBASE_LINUX_XCP:=http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27}
 
diff --git a/ap-fetch-version b/ap-fetch-version
index a107c93..1d86351 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -106,6 +106,12 @@ ovmf)
 	repo_tree_rev_fetch_git ovmf \
 		$TREE_OVMF_UPSTREAM master $LOCALREV_OVMF
 	;;
+openstack-ocata)
+        openstack_release=${branch#openstack-}
+        repo_tree_rev_fetch_git "openstack-nova" \
+                "$TREE_OPENSTACK_NOVA" "stable/$openstack_release" \
+                "$LOCALREV_OPENSTACK_NOVA"
+        ;;
 osstest)
         if [ "x$OSSTEST_USE_HEAD" = "xy" ] ; then
 	    git update-ref -m "Arranging to test HEAD" \
diff --git a/ap-fetch-version-old b/ap-fetch-version-old
index 3cbc176..a0b8c08 100755
--- a/ap-fetch-version-old
+++ b/ap-fetch-version-old
@@ -35,6 +35,7 @@ check_ap_fetch_placeholders
 : ${BASE_LOCALREV_XTF:=daily-cron.$branch.old}
 : ${BASE_LOCALREV_OVMF:=daily-cron.$branch.old}
 : ${BASE_TAG_LIBVIRT:=xen-tested-master}
+: ${BASE_LOCALREV_OPENSTACK_NOVA:=daily-cron.$branch.old}
 
 if info_linux_tree "$branch"; then
 	repo_tree_rev_fetch_git linux \
@@ -114,6 +115,13 @@ ovmf)
 	repo_tree_rev_fetch_git ovmf \
 		$BASE_TREE_OVMF xen-tested-master $BASE_LOCALREV_OVMF
 	;;
+openstack-ocata)
+        openstack_release="${branch##*-}"
+        repo_tree_rev_fetch_git openstack-nova \
+                "$BASE_TREE_OPENSTACK_NOVA" \
+                "xen-tested-stable-$openstack_release" \
+                "$BASE_LOCALREV_OPENSTACK_NOVA"
+        ;;
 osstest)
 	if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
 	    git fetch -f $HOME/testing.git production:ap-fetch
diff --git a/ap-print-url b/ap-print-url
index 93c14b3..cfba1d4 100755
--- a/ap-print-url
+++ b/ap-print-url
@@ -67,6 +67,9 @@ ovmf)
 osstest)
 	echo none:;
 	;;
+openstack-ocata)
+	echo $TREE_OPENSTACK_NOVA
+	;;
 *)
 	echo >&2 "branch $branch ?"
 	exit 1
diff --git a/ap-push b/ap-push
index a27ccc2..e5c061a 100755
--- a/ap-push
+++ b/ap-push
@@ -41,6 +41,7 @@ TREE_RUMPRUN=$PUSH_TREE_RUMPRUN
 TREE_SEABIOS=$PUSH_TREE_SEABIOS
 TREE_OVMF=$PUSH_TREE_OVMF
 TREE_XTF=$PUSH_TREE_XTF
+TREE_OPENSTACK_NOVA=$PUSH_TREE_OPENSTACK_NOVA
 
 if info_linux_tree "$branch"; then
 	cd $repos/linux
@@ -129,6 +130,12 @@ ovmf)
 	cd $repos/ovmf
 	git push $TREE_OVMF $revision:refs/heads/xen-tested-master
 	;;
+openstack-ocata)
+        cd $repos/openstack-nova
+        openstack_release=${branch#openstack-}
+        git push $TREE_OPENSTACK_NOVA \
+                $revision:refs/heads/xen-tested-stable-$openstack_release
+        ;;
 osstest)
 	git push $HOME/testing.git $revision:production
 	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
diff --git a/cr-daily-branch b/cr-daily-branch
index 39483cd..c3de124 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -207,6 +207,33 @@ if [ "x$REVISION_LINUXFIRMWARE" = x ]; then
 	determine_version REVISION_LINUXFIRMWARE linuxfirmware LINUXFIRMWARE
         export REVISION_LINUXFIRMWARE
 fi
+if [ "x$REVISION_OPENSTACK_NOVA" = x ]; then
+        determine_version REVISION_OPENSTACK_NOVA openstack-ocata OPENSTACK_NOVA
+        export REVISION_OPENSTACK_NOVA
+fi
+# Set REVISION for every tree that openstack is going to use and that have a
+# stable branch (tempest does not)
+determine_openstack_version() {
+        local openstack_tree="$1"
+        local openstack_release
+        local openstack_branch
+
+        if ! eval [ "x\$REVISION_OPENSTACK_${openstack_tree^^}" = x ]; then
+                return
+        fi
+        case "$branch" in
+        openstack-ocata)
+                openstack_release=${branch#openstack-}
+                openstack_branch="origin/stable/${branch##*-}"
+                eval "REVISION_OPENSTACK_${openstack_tree^^}=$openstack_branch"
+                export "REVISION_OPENSTACK_${openstack_tree^^}"
+                ;;
+        esac
+}
+for openstack_tree in cinder devstack glance keystone neutron requirements; do
+        determine_openstack_version "$openstack_tree"
+done
+
 
 case "$tree" in
 xen)
@@ -255,6 +282,10 @@ ovmf)
 	realtree=ovmf
 	NEW_REVISION=$REVISION_OVMF
 	;;
+openstack-nova)
+	realtree=openstack-nova
+	NEW_REVISION=$REVISION_OPENSTACK_NOVA
+	;;
 *)
 	NEW_REVISION=''
 	wantpush=false
diff --git a/cr-for-branches b/cr-for-branches
index a691273..ed76d70 100755
--- a/cr-for-branches
+++ b/cr-for-branches
@@ -31,7 +31,7 @@ scriptoptions="$1"; shift
 LOGFILE=tmp/cr-for-branches.log
 export LOGFILE
 
-: ${BRANCHES:=osstest xen-4.0-testing xen-4.1-testing xen-4.2-testing xen-4.3-testing xen-4.4-testing xen-4.5-testing xen-4.6-testing xen-4.7-testing xen-4.8-testing xen-4.9-testing xen-unstable qemu-mainline qemu-upstream-unstable qemu-upstream-4.2-testing qemu-upstream-4.3-testing qemu-upstream-4.4-testing qemu-upstream-4.5-testing qemu-upstream-4.6-testing qemu-upstream-4.7-testing qemu-upstream-4.8-testing qemu-upstream-4.9-testing linux-linus linux-4.9 linux-4.1 linux-3.18 linux-3.16 linux-3.14 linux-3.10 linux-3.4 linux-arm-xen seabios ovmf xtf ${EXTRA_BRANCHES}}
+: ${BRANCHES:=osstest xen-4.0-testing xen-4.1-testing xen-4.2-testing xen-4.3-testing xen-4.4-testing xen-4.5-testing xen-4.6-testing xen-4.7-testing xen-4.8-testing xen-4.9-testing xen-unstable qemu-mainline qemu-upstream-unstable qemu-upstream-4.2-testing qemu-upstream-4.3-testing qemu-upstream-4.4-testing qemu-upstream-4.5-testing qemu-upstream-4.6-testing qemu-upstream-4.7-testing qemu-upstream-4.8-testing qemu-upstream-4.9-testing linux-linus linux-4.9 linux-4.1 linux-3.18 linux-3.16 linux-3.14 linux-3.10 linux-3.4 linux-arm-xen seabios ovmf xtf openstack-ocata ${EXTRA_BRANCHES}}
 export BRANCHES
 
 fetchwlem=$wlem
diff --git a/cri-common b/cri-common
index 903fb4e..751a362 100644
--- a/cri-common
+++ b/cri-common
@@ -82,6 +82,7 @@ select_xenbranch () {
 	ovmf)			tree=ovmf;	xenbranch=xen-unstable ;;
 	distros-*|examine)	tree=none;	xenbranch=xen-unstable ;;
 	osstest)		tree=osstest;	xenbranch=xen-unstable ;;
+	openstack-ocata)   tree=openstack-nova;	xenbranch=xen-unstable ;;
 	esac
 	if [ "x$tree" = xlinux ]; then
 		linuxbranch=$branch
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 23/24] make-flight: Increase dom0_mem for openstack flight

[Anthony PERARD]

With 4G for dom0_mem, a host running devstack is using about 1.5G of
swap.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 make-flight | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/make-flight b/make-flight
index e03aacc..cc95529 100755
--- a/make-flight
+++ b/make-flight
@@ -717,7 +717,7 @@ do_openstack_tests () {
         job_create_test test-$xenarch$kern-$dom0arch-devstack \
             test-devstack libvirt $xenarch $dom0arch \
             $openstack_runvars \
-            dom0_mem=4000 \
+            dom0_mem=6000 \
             enable_xsm=$xsm \
             all_hostflags=$most_hostflags
     done
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v13 24/24] openstack tests: Don't run them on arm*

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 make-flight | 1 +
 1 file changed, 1 insertion(+)

diff --git a/make-flight b/make-flight
index cc95529..f615155 100755
--- a/make-flight
+++ b/make-flight
@@ -224,6 +224,7 @@ arch_branch_filter_callback () {
         rumprun) return 1;;
         seabios) return 1;;
         ovmf) return 1;;
+        openstack*) return 1;;
         esac
         ;;
   i386)
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 10/24] ts-openstack-deploy: Increase open fd limit for RabbitMQ

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v13 10/24] ts-openstack-deploy: Increase open fd limit for RabbitMQ"):
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 19/24] TestSupport: Implement target_subunit_cmd a subunit stream parser into substeps

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v13 19/24] TestSupport: Implement target_subunit_cmd a subunit stream parser into substeps"):
> target_subunit_cmd can be used like target_cmd, but the command would
> needs to output a subunit v1 stream, which will be parsed and turned
> into osstest substeps. The command can be `| subunit-2to1` in order to
> turn a subunit v2 stream into v1.
> 
> Currently, time is not taken into account, and all substeps will have
> bogus timestamp as the output of the command is parsed after it has
> runned.
> 
> This is a description of the subunit v1 protocol, taken from
> python-subunit README, or https://pypi.python.org/pypi/python-subunit

What a lot of code!

> +    while (<$stdout>) {
> +        if (/^time: (\d+)-(\d+)-(\d+) (\d+):(\d+):(\d+)(\.\d+)?Z$/) {
> +            # This is the timestamp for the next events

I'm not sure what your ( ) are doing here.

> +        } elsif (/^test(?:ing)?:? (.+)\n/) {
> +            # Start of a new test.
> +            $logfilename = subunit_sanitize_testname($1) . '.log';
> +            $fh = open_unique_stashfile(\$logfilename);

This name might clash with existing logfile names, which might be
generated later.  Can you put "subunit-" on the front maybe ?

> +            substep_start(subunit_sanitize_testname($1), $logfilename);

And here, I think you should start the parameter you pass to
substep_start with '/' so that it gets appended to the testid for the
whole script, for a similar reason.

I think it would be better to call subunit_sanitize_testname only
once.

> +        } elsif (/^(success(?:ful)?|failure|skip|error|xfail|uxsuccess):
> +                   \ (.+?)(\ \[(\ multipart)?)?$/x) {
> +            # Result of a test, with its output.
> +            my $result = $1;
> +            my $testname = $2;
> +            my $have_details = $3;
> +            my $is_multipart = $4;

I would normally write this:
               my ($result, $testname, $have_... ) = ($1,$2,$3,$4,$5)
although I don't really mind much that you have written it as you
have.

> +            if ($have_details) {
> +                if ($is_multipart) {
> +                    # Test output
> +                    while (<$stdout>) {
> +                        # part content-type
> +                        # from https://tools.ietf.org/html/rfc6838#section-4.2
> +                        my $restricted_name = qr'[a-zA-Z0-9][a-zA-Z0-9!#$&^_.+-]*';
> +                        if (m{ ^Content-Type:\s+
> +                                $restricted_name/$restricted_name # type/sub-type
> +                                # parameters
> +                                (?:\s*;\s*$restricted_name=[^,]+
> +                                  (?:,\s*$restricted_name=[^,]+)*)
> +                                \s*$
> +                            }xi) {

I don't understand why you are trying to match this Content-Type so
precisely.  AFAICT from the grammar, all you need to do is see whether
there is something vaguely like a c-t header.

> +                            print $fh $_ or die $!;
> +
> +                            # part name
> +                            my $line = <$stdout>;
> +                            print $fh $line or die $!;
> +
> +                            # Read chunks of a part
> +                            while (<$stdout>) {
> +                                if (/^([0-9A-F]+)\r$/i) {
> +                                    my $chunk_size = hex($1);

What makes you think the digits are in hex ?

Since you have to go to the effort of separating out all of this
stuff, it might be worth printing these multipart objects with one
object per logfile.  Although I won't insist on that because I suspect
that multipart results are rare.

> +                                } else {
> +                                    # Unexpected output
> +                                    chomp;
> +                                    logm("*** $_");

I guess the error recovery is to continue until you see "]"
and hope.  Fair enough.

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 21/24] Create a flight to test OpenStack with xen-unstable and libvirt

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v13 21/24] Create a flight to test OpenStack with xen-unstable and libvirt"):
> This patch creates a flight "openstack*", with those jobs:

Do you mean it creates a "branch" ?  But I don't think it does.  I
predict no changes to the output of mg-list-all-branches.  You
probably want to edit cr-for-branches.

>   build-amd64
>   build-amd64-libvirt
>   build-amd64-pvops
>   build-amd64-xsm
>   build-arm64
>   build-arm64-libvirt
>   build-arm64-pvops
>   build-arm64-xsm
>   build-armhf
>   build-armhf-libvirt
>   build-armhf-pvops
>   build-armhf-xsm
>   test-amd64-amd64-devstack
>   test-amd64-amd64-devstack-xsm
>   test-arm64-arm64-devstack
>   test-arm64-arm64-devstack-xsm
>   test-armhf-armhf-devstack
>   test-armhf-armhf-devstack-xsm

Does it add these jobs to existing flights ?  I think it probably
does.  Is that intentional ?  I think it probably isn't.  If it is you
should explain it in the commit message - and also explain why this is
OK despite you not keeping tested versions of all the input branches
(ie, risking uncontrolled regressions in openstack components breaking
the xen pushes...)

You can see what it does with standalone-dump-all-flight-runvars
(which is best run with eatmydata).

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 24/24] openstack tests: Don't run them on arm*

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v13 24/24] openstack tests: Don't run them on arm*"):
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

You should probably mention the existence of this patch in the other
one's commit message...

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 22/24] New branch openstack-ocata

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v13 22/24] New branch openstack-ocata"):
> Testing of the Ocata stable branch of OpenStack against Xen unstable.
> 
> OpenStack have many different repo which should be in sync, so we should
> attempd to grab the revisions of the stable branch of every OpenStack
> tree, but for now, the runvars REVISION_* of tree other than nova is set
> to "origin/stable/ocata", except Tempest does not have stable branch and
> should be able to test any OpenStack version.

Ah I see this is where the new branch is created.  You will want to
run standalone-generate-dump-flight-runvars after this, not after the
previous patch (which is fine as it is - sorry, please forget my
comment about editing cr-for-branches there).

I'm afraid I don't understand your explanation about stable branches.
Earlier I asked:

  > Do you intend to provide a version of this patch which maintains a
  > tested branch for all of these different trees ?

  No, I don't. This would be a different patch (and maybe different patch
  series).

So you don't intend to maintain a tested branch for each of these
trees.  But you do intend, I think, to maintain a tested branch of the
main tree.  You say the subtrees "should be in sync", which I take to
mean that openstack upstream only expect it to work if you grab a
revision from all of these trees at "roughty the same time" or
something.

I don't think this will necessarily work properly.  The most obvious
problem I see is that regressions which are introduced in the subtrees
will be picked up by even flights which are attempting to use a
working (ie osstest-tested) version of "openstack".

This may not be critical if openstack jobs appear only on the flights
for openstack branches.  openstack branches will occasionally suffer
trouble but things will probably be BALGE.

But we definitely won't be able to add openstack tests to the other
branches without doing something different.

I have a very strong feeling we have discussed this before but I'm
afraid the answer doesn't seem to have been written down somewhere I
can easily find it.  It should be explained in your series somewhere,
in a comment or a commit message.

It would also be nice to have a theory about how this could be
improved in the future.  That would mean we could be more confident
that we're not painting ourselves into a corner.

Having said all that, with a suitable explanation, I think the code is
probably about right.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 18/24] TestSupport: Introduce target_cmd_stashed

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v13 18/24] TestSupport: Introduce target_cmd_stashed"):
> This works like target_cmd, but takes a ref to a filename as argument
> and stash the output of the command then return a path to the stashed
> output.
...
> +# Like target_cmd, but stash cmd stdout and return a path to it.
> +sub target_cmd_stashed ($$$;$$) {
> +    my ($tho,$leafref,$tcmd,$timeout,$extrasshopts) = @_;
> +    my $stdout = open_unique_stashfile($leafref);
> +    my $rc = tcmd(undef, $stdout, 0, 'osstest', $tho, $tcmd, $timeout,
> +        $extrasshopts);
> +    die "$stdout $!" if $stdout->error or !close $stdout;

You can't sensibly interpolate a filehandle into a "" string.
You should use the filename (probably, the whole filename).

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 00/24] Have OpenStack tested on top of xen's master and libvirt's master.

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v13 00/24] Have OpenStack tested on top of xen's master and libvirt's master."):
> Now powered with subunit-to-substep engine.
> 
> The Tempest test names reported via subunit are in the form:
> tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario[compute,id-bdbb5441-9204-419d-a225-b4fdbfb1a1a8,image,network,volume]

Blimey.  Well, I guess we will put up with that.

Thanks.  I think I have been through all the patches now.  I hope you
find my comments helpful.

Regards,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 19/24] TestSupport: Implement target_subunit_cmd a subunit stream parser into substeps

[Anthony PERARD]

On Tue, Jul 25, 2017 at 07:00:47PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[OSSTEST PATCH v13 19/24] TestSupport: Implement target_subunit_cmd a subunit stream parser into substeps"):
> > target_subunit_cmd can be used like target_cmd, but the command would
> > needs to output a subunit v1 stream, which will be parsed and turned
> > into osstest substeps. The command can be `| subunit-2to1` in order to
> > turn a subunit v2 stream into v1.
> > 
> > Currently, time is not taken into account, and all substeps will have
> > bogus timestamp as the output of the command is parsed after it has
> > runned.
> > 
> > This is a description of the subunit v1 protocol, taken from
> > python-subunit README, or https://pypi.python.org/pypi/python-subunit
> 
> What a lot of code!
> 
> > +    while (<$stdout>) {
> > +        if (/^time: (\d+)-(\d+)-(\d+) (\d+):(\d+):(\d+)(\.\d+)?Z$/) {
> > +            # This is the timestamp for the next events
> 
> I'm not sure what your ( ) are doing here.

No real reason. I'll remove them.

> > +        } elsif (/^test(?:ing)?:? (.+)\n/) {
> > +            # Start of a new test.
> > +            $logfilename = subunit_sanitize_testname($1) . '.log';
> > +            $fh = open_unique_stashfile(\$logfilename);
> 
> This name might clash with existing logfile names, which might be
> generated later.  Can you put "subunit-" on the front maybe ?

Will do.

> > +            substep_start(subunit_sanitize_testname($1), $logfilename);
> 
> And here, I think you should start the parameter you pass to
> substep_start with '/' so that it gets appended to the testid for the
> whole script, for a similar reason.

OK.

> I think it would be better to call subunit_sanitize_testname only
> once.

OK.

> > +        } elsif (/^(success(?:ful)?|failure|skip|error|xfail|uxsuccess):
> > +                   \ (.+?)(\ \[(\ multipart)?)?$/x) {
> > +            # Result of a test, with its output.
> > +            my $result = $1;
> > +            my $testname = $2;
> > +            my $have_details = $3;
> > +            my $is_multipart = $4;
> 
> I would normally write this:
>                my ($result, $testname, $have_... ) = ($1,$2,$3,$4,$5)
> although I don't really mind much that you have written it as you
> have.
> 
> > +            if ($have_details) {
> > +                if ($is_multipart) {
> > +                    # Test output
> > +                    while (<$stdout>) {
> > +                        # part content-type
> > +                        # from https://tools.ietf.org/html/rfc6838#section-4.2
> > +                        my $restricted_name = qr'[a-zA-Z0-9][a-zA-Z0-9!#$&^_.+-]*';
> > +                        if (m{ ^Content-Type:\s+
> > +                                $restricted_name/$restricted_name # type/sub-type
> > +                                # parameters
> > +                                (?:\s*;\s*$restricted_name=[^,]+
> > +                                  (?:,\s*$restricted_name=[^,]+)*)
> > +                                \s*$
> > +                            }xi) {
> 
> I don't understand why you are trying to match this Content-Type so
> precisely.  AFAICT from the grammar, all you need to do is see whether
> there is something vaguely like a c-t header.

I think I start by looking at what kind of characters could be part of
type and sub-type, and just start writing a more complicated regex.

So is the following would be enough for you?
m{^Content-Type: [^/ ]+/[^/ ]+(?:;.+)?$}


> > +                            print $fh $_ or die $!;
> > +
> > +                            # part name
> > +                            my $line = <$stdout>;
> > +                            print $fh $line or die $!;
> > +
> > +                            # Read chunks of a part
> > +                            while (<$stdout>) {
> > +                                if (/^([0-9A-F]+)\r$/i) {
> > +                                    my $chunk_size = hex($1);
> 
> What makes you think the digits are in hex ?

I tried with [0-9] (because DIGITS), but that was not enought. Then I've
check the subunit implementation, there are using "%X" which is hex.

> Since you have to go to the effort of separating out all of this
> stuff, it might be worth printing these multipart objects with one
> object per logfile.  Although I won't insist on that because I suspect
> that multipart results are rare.

There are usually 3 part per tests, with those names:
  pythonlogging:''
  stdout
  stderr
And sometime, there is also one name 'traceback'.
I think stdout and stderr are usually empty.

I think having one file per part will make it more complicated to
read logs of a failed test.

> > +                                } else {
> > +                                    # Unexpected output
> > +                                    chomp;
> > +                                    logm("*** $_");
> 
> I guess the error recovery is to continue until you see "]"
> and hope.  Fair enough.

That one of the reason for the subunit-v2, with a binary protocol,
better recovery.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 21/24] Create a flight to test OpenStack with xen-unstable and libvirt

[Anthony PERARD]

On Tue, Jul 25, 2017 at 07:09:06PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[OSSTEST PATCH v13 21/24] Create a flight to test OpenStack with xen-unstable and libvirt"):
> > This patch creates a flight "openstack*", with those jobs:
> 
> Do you mean it creates a "branch" ?  But I don't think it does.  I
> predict no changes to the output of mg-list-all-branches.  You
> probably want to edit cr-for-branches.

Maybe branch is the right word, but this patch does not change anything
anymore. I've try to split changes in "make-fligh" from the on in ap-*
and cr-*, but the description is just not clear enough.

> >   build-amd64
> >   build-amd64-libvirt
> >   build-amd64-pvops
> >   build-amd64-xsm
> >   build-arm64
> >   build-arm64-libvirt
> >   build-arm64-pvops
> >   build-arm64-xsm
> >   build-armhf
> >   build-armhf-libvirt
> >   build-armhf-pvops
> >   build-armhf-xsm
> >   test-amd64-amd64-devstack
> >   test-amd64-amd64-devstack-xsm
> >   test-arm64-arm64-devstack
> >   test-arm64-arm64-devstack-xsm
> >   test-armhf-armhf-devstack
> >   test-armhf-armhf-devstack-xsm
> 
> Does it add these jobs to existing flights ?  I think it probably
> does.  Is that intentional ?  I think it probably isn't.  If it is you
> should explain it in the commit message - and also explain why this is
> OK despite you not keeping tested versions of all the input branches
> (ie, risking uncontrolled regressions in openstack components breaking
> the xen pushes...)

Last time I've check, by testing this patch with the next one, those
jobs where only part of a branch names "openstack-ocata" and did not
change anything else.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 24/24] openstack tests: Don't run them on arm*

[Anthony PERARD]

On Tue, Jul 25, 2017 at 07:18:21PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[OSSTEST PATCH v13 24/24] openstack tests: Don't run them on arm*"):
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
> 
> You should probably mention the existence of this patch in the other
> one's commit message...

The goal of this patch was to not use ARM capacity while there is not
enough. And also to be reverted once there there is plenty of capacity.

So I did not think it was usefull to say anything in the other patch.
Or maybe under the ---.

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 18/24] TestSupport: Introduce target_cmd_stashed

[Anthony PERARD]

On Tue, Jul 25, 2017 at 07:26:07PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[OSSTEST PATCH v13 18/24] TestSupport: Introduce target_cmd_stashed"):
> > This works like target_cmd, but takes a ref to a filename as argument
> > and stash the output of the command then return a path to the stashed
> > output.
> ...
> > +# Like target_cmd, but stash cmd stdout and return a path to it.
> > +sub target_cmd_stashed ($$$;$$) {
> > +    my ($tho,$leafref,$tcmd,$timeout,$extrasshopts) = @_;
> > +    my $stdout = open_unique_stashfile($leafref);
> > +    my $rc = tcmd(undef, $stdout, 0, 'osstest', $tho, $tcmd, $timeout,
> > +        $extrasshopts);
> > +    die "$stdout $!" if $stdout->error or !close $stdout;
> 
> You can't sensibly interpolate a filehandle into a "" string.
> You should use the filename (probably, the whole filename).

Right, I did not think about that. I've just copy/past the line from the
function tcmdout(). I'll use the filename.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 19/24] TestSupport: Implement target_subunit_cmd a subunit stream parser into substeps

[Ian Jackson]

Anthony PERARD writes ("Re: [OSSTEST PATCH v13 19/24] TestSupport: Implement target_subunit_cmd a subunit stream parser into substeps"):
> I think I start by looking at what kind of characters could be part of
> type and sub-type, and just start writing a more complicated regex.
> 
> So is the following would be enough for you?
> m{^Content-Type: [^/ ]+/[^/ ]+(?:;.+)?$}

Why do you need to check the at all ?  I think, according to the spec,
that the only thing which can occur here is "Content-Type: something"
or "]".  What would be wrong with
   m{^content-type:}i
?

> > > +                            # Read chunks of a part
> > > +                            while (<$stdout>) {
> > > +                                if (/^([0-9A-F]+)\r$/i) {
> > > +                                    my $chunk_size = hex($1);
> > 
> > What makes you think the digits are in hex ?
> 
> I tried with [0-9] (because DIGITS), but that was not enought. Then I've
> check the subunit implementation, there are using "%X" which is hex.

Wow.  Can you put a comment next to this please ?  Something like

 # The chunk size is in hex, even though this does not seem to be
 # documented in the subunit specification.

perhaps.

> > Since you have to go to the effort of separating out all of this
> > stuff, it might be worth printing these multipart objects with one
> > object per logfile.  Although I won't insist on that because I suspect
> > that multipart results are rare.
> 
> There are usually 3 part per tests, with those names:
>   pythonlogging:''
>   stdout
>   stderr
> And sometime, there is also one name 'traceback'.
> I think stdout and stderr are usually empty.
> 
> I think having one file per part will make it more complicated to
> read logs of a failed test.

OK, leave it as-is then.  (Also, "pythonlogging:''" ?!)

> > I guess the error recovery is to continue until you see "]"
> > and hope.  Fair enough.
> 
> That one of the reason for the subunit-v2, with a binary protocol,
> better recovery.

I don't think that's a good reason.  But this ranting is quite
off-topic now :-).

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 21/24] Create a flight to test OpenStack with xen-unstable and libvirt

[Ian Jackson]

Anthony PERARD writes ("Re: [OSSTEST PATCH v13 21/24] Create a flight to test OpenStack with xen-unstable and libvirt"):
> On Tue, Jul 25, 2017 at 07:09:06PM +0100, Ian Jackson wrote:
> > Anthony PERARD writes ("[OSSTEST PATCH v13 21/24] Create a flight to test OpenStack with xen-unstable and libvirt"):
> > > This patch creates a flight "openstack*", with those jobs:
> > 
> > Do you mean it creates a "branch" ?  But I don't think it does.  I
> > predict no changes to the output of mg-list-all-branches.  You
> > probably want to edit cr-for-branches.
> 
> Maybe branch is the right word, but this patch does not change anything
> anymore. I've try to split changes in "make-fligh" from the on in ap-*
> and cr-*, but the description is just not clear enough.

Maybe

   Subject: Create devstack jobs (on openstack branches only)

   blah blah blah

   The new jobs appear only in the branches openstack-*, which
   will be introduced in the next patch.  So, for now, no
   overall functional change.

?

> Last time I've check, by testing this patch with the next one, those
> jobs where only part of a branch names "openstack-ocata" and did not
> change anything else.

Oh, good.  You might mention having done this check, in the commit
message.  That would save me from being confused, at least.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 24/24] openstack tests: Don't run them on arm*

[Ian Jackson]

Anthony PERARD writes ("Re: [OSSTEST PATCH v13 24/24] openstack tests: Don't run them on arm*"):
> On Tue, Jul 25, 2017 at 07:18:21PM +0100, Ian Jackson wrote:
> > Anthony PERARD writes ("[OSSTEST PATCH v13 24/24] openstack tests: Don't run them on arm*"):
> > > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> > 
> > Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
> > 
> > You should probably mention the existence of this patch in the other
> > one's commit message...
> 
> The goal of this patch was to not use ARM capacity while there is not
> enough. And also to be reverted once there there is plenty of capacity.
> 
> So I did not think it was usefull to say anything in the other patch.
> Or maybe under the ---.

Either way is fine.  I don't have a strong opinion.

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 19/24] TestSupport: Implement target_subunit_cmd a subunit stream parser into substeps

[Anthony PERARD]

On Wed, Jul 26, 2017 at 04:41:58PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("Re: [OSSTEST PATCH v13 19/24] TestSupport: Implement target_subunit_cmd a subunit stream parser into substeps"):
> > I think I start by looking at what kind of characters could be part of
> > type and sub-type, and just start writing a more complicated regex.
> > 
> > So is the following would be enough for you?
> > m{^Content-Type: [^/ ]+/[^/ ]+(?:;.+)?$}
> 
> Why do you need to check the at all ?  I think, according to the spec,
> that the only thing which can occur here is "Content-Type: something"
> or "]".  What would be wrong with
>    m{^content-type:}i
> ?

Ok, will go with that.

> > > > +                            # Read chunks of a part
> > > > +                            while (<$stdout>) {
> > > > +                                if (/^([0-9A-F]+)\r$/i) {
> > > > +                                    my $chunk_size = hex($1);
> > > 
> > > What makes you think the digits are in hex ?
> > 
> > I tried with [0-9] (because DIGITS), but that was not enought. Then I've
> > check the subunit implementation, there are using "%X" which is hex.
> 
> Wow.  Can you put a comment next to this please ?  Something like
> 
>  # The chunk size is in hex, even though this does not seem to be
>  # documented in the subunit specification.
> 
> perhaps.

I don't think there is a specification for subunit, beside the source
code. In the README of the project, it is called a "description", and
there is even "When in doubt, refer the source".

But I guess I can add the comment.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v13 19/24] TestSupport: Implement target_subunit_cmd a subunit stream parser into substeps

[Anthony PERARD]

On Tue, Jul 25, 2017 at 07:00:47PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[OSSTEST PATCH v13 19/24] TestSupport: Implement target_subunit_cmd a subunit stream parser into substeps"):
> > +                            # Read chunks of a part
> > +                            while (<$stdout>) {
> > +                                if (/^([0-9A-F]+)\r$/i) {
> > +                                    my $chunk_size = hex($1);
[...]
> > +                                } else {
> > +                                    # Unexpected output
> > +                                    chomp;
> > +                                    logm("*** $_");
> 
> I guess the error recovery is to continue until you see "]"
> and hope.  Fair enough.

Actually, in the chunk parser, the only expected output is an hex number
for a chunk size. There is nothing that parse "]". So the only error
recovery would be to find a line that only contain a hex number, this
could be the next "0\r\n" if part ends with a "\n", which I think is
likely.

Only the loop that parse the multipart check for "]".

Maybe I could end the chunks parser loop and go back to the multipart
parser loop.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel