[OSSTEST PATCH v11 00/20] Have OpenStack tested on top of xen's master and libvirt's master.

[OSSTEST PATCH v11 00/20] Have OpenStack tested on top of xen's master and libvirt's master.

[Anthony PERARD]

Changes in V11:
- plenty of new patches, on top of the original 3 patches that were acked.
- and an attempt at creating a flight for a stable branch of openstack. But
  there is many git tree to pull the branch from.

Anthony PERARD (17):
  ts-openstack-deploy: Deploy OpenStack on a host with devstack
  ts-openstack-tempest: Run Tempest to check OpenStack
  Create a flight to test OpenStack with xen-unstable and libvirt
  ts-openstack-deploy: Keep CURL_CA_BUNDLE when sudo is called
  ts-openstack-deploy: Try to disable use of SYSTEMD
  ts-kernel-build: Enable network related modules for Neutron
  ts-openstack-deploy: Switch to Neutron for network
  ts-openstack-deploy: Increase fd and memory limits for rabbitmq
  make-flight: Increase dom0_mem for openstack flight
  ts-openstack-deploy: Apply a Tempest patch
  ts-openstack-deploy: Ignore libvirt-python version and use latest
  ts-openstack-tempest: Fix tempest invocation
  ts-openstack-tempest: Update list of skipped tests
  ts-openstack-deploy: Move logs to /var/log/openstack
  ts-logs-capture: Capture OpenStack logs
  ts-openstack-deploy: Increase devstack timeout
  Introduce flight for stable branches of OpenStack

Ian Jackson (3):
  ts-openstack-deploy: Set http proxy
  TestSupport: provide target_https_mitm_proxy_cert_path
  ts-openstack-deploy: set CURL_CA_BUNDLE

 Osstest/TestSupport.pm |   7 +
 ap-common              |  19 ++
 ap-fetch-version       |  20 +++
 ap-fetch-version-old   |  13 ++
 ap-print-url           |   3 +
 ap-push                |  11 ++
 cr-daily-branch        |  36 ++++
 cr-for-branches        |   2 +-
 cri-common             |   1 +
 make-flight            |  46 ++++-
 sg-run-job             |   6 +
 ts-kernel-build        |  17 +-
 ts-logs-capture        |   6 +
 ts-openstack-deploy    | 190 ++++++++++++++++++++
 ts-openstack-tempest   |  63 +++++++
 15 files changed, 436 insertions(+), 4 deletions(-)
 create mode 100755 ts-openstack-deploy
 create mode 100755 ts-openstack-tempest

-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 01/20] ts-openstack-deploy: Deploy OpenStack on a host with devstack

[Anthony PERARD]

This script installs any necessary packages and clones all of the OpenStack
trees which are used by devstack to deploy OpenStack.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 sg-run-job          |   5 +
 ts-openstack-deploy | 148 ++++++++++++++++++++
 2 files changed, 153 insertions(+)

diff --git a/sg-run-job b/sg-run-job
index 4b98e2b..4635b65 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -501,6 +501,11 @@ proc run-job/test-rumprun {} {
                  ts-guest-destroy-hard        host   $g   +
 }
 
+proc need-hosts/test-devstack {} { return host }
+proc run-job/test-devstack {} {
+    run-ts . = ts-openstack-deploy + host
+}
+
 if {[file exists sg-run-job-adhoc]} {
     source sg-run-job-adhoc
 }
diff --git a/ts-openstack-deploy b/ts-openstack-deploy
new file mode 100755
index 0000000..6f061eb
--- /dev/null
+++ b/ts-openstack-deploy
@@ -0,0 +1,148 @@
+#!/usr/bin/perl
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2016 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use Osstest;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+our ($whhost) = @ARGV;
+$whhost ||= 'host';
+our $ho = selecthost($whhost);
+our $builddir = target_jobdir($ho);
+
+sub packages () {
+    target_install_packages($ho, qw(git sudo));
+}
+sub checkout () {
+    prepbuilddirs();
+    build_clone($ho, 'openstack_cinder',       $builddir, 'cinder');
+    build_clone($ho, 'openstack_devstack',     $builddir, 'devstack');
+    build_clone($ho, 'openstack_glance',       $builddir, 'glance');
+    build_clone($ho, 'openstack_keystone',     $builddir, 'keystone');
+    build_clone($ho, 'openstack_nova',         $builddir, 'nova');
+    build_clone($ho, 'openstack_requirements', $builddir, 'requirements');
+    build_clone($ho, 'openstack_tempest',      $builddir, 'tempest');
+
+    my $vg = target_choose_vg($ho, 10*1024); # 10GB
+    target_putfilecontents_stash($ho, 60,
+        <<END, $builddir.'/devstack/local.conf');
+[[local|localrc]]
+# Everything should be cloned by osstest, so devstack don't have to do it
+ERROR_ON_CLONE=True
+USE_SCREEN=False
+ADMIN_PASSWORD=secretadmin
+DATABASE_PASSWORD=secretdatabase
+RABBIT_PASSWORD=secretrabbit
+SERVICE_PASSWORD=secretservice
+SERVICE_TOKEN=atokenserviced
+# make it small because there is no way to not have this lvm volume created
+VOLUME_BACKING_FILE_SIZE=50M
+CINDER_LVM_TYPE=default
+DEST=$builddir
+DATA_DIR=\$DEST/data
+SERVICE_DIR=\$DEST/status
+SUBUNIT_OUTPUT=\$DEST/devstack.subunit
+LOGFILE=\$DEST/logs/stack.sh.log
+LOG_COLOR=False
+LIBVIRT_TYPE=xen
+disable_service horizon
+disable_service n-novnc
+disable_service dstat
+# Disable neutron and switch back to nova-network
+disable_service q-svc
+disable_service q-dhcp
+disable_service q-meta
+disable_service q-agt
+disable_service q-l3
+enable_service n-net
+[[post-config|\$CINDER_CONF]]
+[lvmdriver-1]
+volume_group = $vg
+END
+
+    # libvirt is already installed, but not as a package, so avoid installation
+    # of the libvirt package with devstack
+    # https://bugs.launchpad.net/devstack/+bug/1641144
+    target_editfile($ho, "$builddir/devstack/files/debs/nova", sub {
+        while (<EI>) {
+            next if m/.*libvirt.*/;
+            print EO or die $!;
+        }
+    });
+    target_editfile($ho,
+        "$builddir/devstack/lib/nova_plugins/functions-libvirt",
+        sub {
+            while (<EI>) {
+                if (m/install_package.*libvirt.*/) {
+                    s#install_package.*#:#
+                }
+                print EO or die $!;
+            }
+        }
+    );
+
+    # Package python-systemd does not exist in Debian installed by osstest
+    target_editfile($ho, "$builddir/devstack/files/debs/general", sub {
+        while (<EI>) {
+            next if m/^python-systemd$/;
+            print EO or die $!;
+        }
+    });
+
+    # devstack blindly assume that systemd is used if systemctl is present
+    # https://bugs.launchpad.net/devstack/+bug/1641112
+    target_editfile($ho, "$builddir/devstack/functions-common", sub {
+        while (<EI>) {
+            if (m#\[ -x /bin/systemctl#) {
+                s#\[ -x /bin/systemctl \]#false#
+            }
+            print EO or die $!;
+        }
+    });
+
+    # OpenStack needs access to libvirt from a user.
+    target_cmd_root($ho, <<END);
+        set -e
+        addgroup --system libvirt
+        cat >> /etc/libvirt/libvirtd.conf <<EOF
+unix_sock_group = "libvirt"
+unix_sock_ro_perms = "0777"
+unix_sock_rw_perms = "0770"
+EOF
+        service libvirtd restart
+END
+
+    # devstack is going to setup the host, install some dependency.
+    target_putfilecontents_root_stash($ho, 100,
+        <<END, "/etc/sudoers.d/devstack");
+osstest ALL=(ALL) NOPASSWD:ALL
+END
+}
+
+sub deploy() {
+    target_cmd($ho, <<END, 1800);
+        set -e
+        cd $builddir/devstack
+        ./stack.sh
+END
+}
+
+packages();
+checkout();
+deploy();
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 02/20] ts-openstack-tempest: Run Tempest to check OpenStack

[Anthony PERARD]

This script runs the OpenStack integration test suite, Tempest.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 sg-run-job           |  1 +
 ts-openstack-tempest | 65 ++++++++++++++++++++
 2 files changed, 66 insertions(+)

diff --git a/sg-run-job b/sg-run-job
index 4635b65..cd077fe 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -504,6 +504,7 @@ proc run-job/test-rumprun {} {
 proc need-hosts/test-devstack {} { return host }
 proc run-job/test-devstack {} {
     run-ts . = ts-openstack-deploy + host
+    run-ts . = ts-openstack-tempest + host
 }
 
 if {[file exists sg-run-job-adhoc]} {
diff --git a/ts-openstack-tempest b/ts-openstack-tempest
new file mode 100755
index 0000000..82e9a71
--- /dev/null
+++ b/ts-openstack-tempest
@@ -0,0 +1,65 @@
+#!/usr/bin/perl
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2016 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use Osstest;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+our ($whhost) = @ARGV;
+$whhost ||= 'host';
+our $ho = selecthost($whhost);
+our $builddir = target_jobdir($ho);
+
+sub tempest() {
+    my @ignored_tests;
+    my $scenario = 'tempest.scenario';
+    my $volume_boot_pattern =
+        "$scenario.test_volume_boot_pattern.TestVolumeBootPattern";
+    my $shelve_instance = "$scenario.test_shelve_instance.TestShelveInstance";
+
+    # Ignore tests which try to boot a guest with /dev/vda as boot device name.
+    push @ignored_tests,
+        "^\Q$volume_boot_pattern.test_volume_boot_pattern\E";
+    push @ignored_tests,
+        "^\Q$volume_boot_pattern.test_create_ebs_image_and_check_boot\E";
+    push @ignored_tests,
+        "^\Q$shelve_instance.test_shelve_volume_backed_instance\E";
+
+    # Those tests access a volume through iSCSI. This does not work when both
+    # the server and client of iSCSI are on the same Xen host (both in dom0),
+    # Linux 4.0 is the first Linux to have a fix.
+    push @ignored_tests,
+        "^\Q${volume_boot_pattern}V2.test_volume_boot_pattern\E";
+    push @ignored_tests,
+        "^\Q${volume_boot_pattern}V2.test_create_ebs_image_and_check_boot\E";
+
+    # This regex below select the tests to run and exclude the ones marked as
+    # slow as well as the explicit tests listed above.  It is based on the one
+    # that can be found in tempest.git/tox.ini in the section [testenv:full].
+    my $ignored_tests = join("|", @ignored_tests);
+    my $regex = "(?!.*\\[.*\\bslow\\b.*\\]|$ignored_tests)".
+        "(^tempest\\.(api|scenario|thirdparty))";
+
+    target_cmd($ho, <<END, 7200);
+set -e
+$builddir/tempest/run_tempest.sh --virtual-env -- --concurrency=2 '$regex'
+END
+}
+
+tempest();
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 03/20] Create a flight to test OpenStack with xen-unstable and libvirt

[Anthony PERARD]

This patch should create a flight "openstack-nova", with those jobs:
  build-amd64
  build-amd64-xsm
  build-amd64-pvops
  build-amd64-libvirt
  test-amd64-amd64-devstack
  test-amd64-amd64-devstack-xsm
  build-armhf
  build-armhf-libvirt
  build-armhf-pvops
  build-armhf-xsm
  test-armhf-armhf-devstack
  test-armhf-armhf-devstack-xsm

About the runvars revision_* of test-*-*-devstack:
  only REVISION_OPENSTACK_NOVA is set, the others are unset.
  Empty revision_* runvar would clone the default branch, which should
  be master for every openstack repos.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ap-common            | 18 ++++++++
 ap-fetch-version     |  4 ++
 ap-fetch-version-old |  5 +++
 ap-print-url         |  3 ++
 ap-push              |  5 +++
 cr-daily-branch      |  8 ++++
 cr-for-branches      |  2 +-
 cri-common           |  1 +
 make-flight          | 45 +++++++++++++++++++-
 9 files changed, 89 insertions(+), 2 deletions(-)

diff --git a/ap-common b/ap-common
index cbb815c..7e84f15 100644
--- a/ap-common
+++ b/ap-common
@@ -54,6 +54,23 @@
 : ${PUSH_TREE_OVMF:=$XENBITS:/home/xen/git/osstest/ovmf.git}
 : ${BASE_TREE_OVMF:=git://xenbits.xen.org/osstest/ovmf.git}
 
+define_openstack_trees() {
+    local openstack_trees=(cinder glance keystone nova requirements tempest)
+    local tree
+    local url
+
+    : ${GIT_OPENSTACK_ORG:=git://git.openstack.org}
+    : ${TREE_OPENSTACK_DEVSTACK:=$GIT_OPENSTACK_ORG/openstack-dev/devstack.git}
+    for tree in "${openstack_trees[@]}"; do
+        url=$GIT_OPENSTACK_ORG/openstack/$tree.git
+        eval ": \${TREE_OPENSTACK_${tree^^}:=$url}"
+    done
+}
+
+define_openstack_trees
+: ${PUSH_TREE_OPENSTACK_NOVA:=$XENBITS:/home/xen/git/osstest/openstack-nova.git}
+: ${BASE_TREE_OPENSTACK_NOVA:=git://xenbits.xen.org/osstest/openstack-nova.git}
+
 : ${TREE_LINUXFIRMWARE:=git://xenbits.xen.org/osstest/linux-firmware.git}
 : ${PUSH_TREE_LINUXFIRMWARE:=$XENBITS:/home/osstest/ext/linux-firmware.git}
 : ${UPSTREAM_TREE_LINUXFIRMWARE:=$GIT_KERNEL_ORG/pub/scm/linux/kernel/git/firmware/linux-firmware.git}
@@ -82,6 +99,7 @@ fi
 : ${LOCALREV_SEABIOS:=daily-cron.$branch}
 : ${LOCALREV_OVMF:=daily-cron.$branch}
 : ${LOCALREV_XTF:=daily-cron.$branch}
+: ${LOCALREV_OPENSTACK_NOVA:=daily-cron.$branch}
 
 : ${TREEBASE_LINUX_XCP:=http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27}
 
diff --git a/ap-fetch-version b/ap-fetch-version
index a107c93..a714ee2 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -106,6 +106,10 @@ ovmf)
 	repo_tree_rev_fetch_git ovmf \
 		$TREE_OVMF_UPSTREAM master $LOCALREV_OVMF
 	;;
+openstack-nova)
+	repo_tree_rev_fetch_git openstack-nova \
+		$TREE_OPENSTACK_NOVA master $LOCALREV_OPENSTACK_NOVA
+	;;
 osstest)
         if [ "x$OSSTEST_USE_HEAD" = "xy" ] ; then
 	    git update-ref -m "Arranging to test HEAD" \
diff --git a/ap-fetch-version-old b/ap-fetch-version-old
index 3cbc176..6dddbb7 100755
--- a/ap-fetch-version-old
+++ b/ap-fetch-version-old
@@ -35,6 +35,7 @@ check_ap_fetch_placeholders
 : ${BASE_LOCALREV_XTF:=daily-cron.$branch.old}
 : ${BASE_LOCALREV_OVMF:=daily-cron.$branch.old}
 : ${BASE_TAG_LIBVIRT:=xen-tested-master}
+: ${BASE_LOCALREV_OPENSTACK_NOVA:=daily-cron.$branch.old}
 
 if info_linux_tree "$branch"; then
 	repo_tree_rev_fetch_git linux \
@@ -114,6 +115,10 @@ ovmf)
 	repo_tree_rev_fetch_git ovmf \
 		$BASE_TREE_OVMF xen-tested-master $BASE_LOCALREV_OVMF
 	;;
+openstack-nova)
+	repo_tree_rev_fetch_git openstack-nova \
+		$BASE_TREE_OPENSTACK_NOVA xen-tested-master $BASE_LOCALREV_OPENSTACK_NOVA
+	;;
 osstest)
 	if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
 	    git fetch -f $HOME/testing.git production:ap-fetch
diff --git a/ap-print-url b/ap-print-url
index 93c14b3..6f4e6b1 100755
--- a/ap-print-url
+++ b/ap-print-url
@@ -67,6 +67,9 @@ ovmf)
 osstest)
 	echo none:;
 	;;
+openstack-nova)
+	echo $TREE_OPENSTACK_NOVA
+	;;
 *)
 	echo >&2 "branch $branch ?"
 	exit 1
diff --git a/ap-push b/ap-push
index a27ccc2..136d1b6 100755
--- a/ap-push
+++ b/ap-push
@@ -41,6 +41,7 @@ TREE_RUMPRUN=$PUSH_TREE_RUMPRUN
 TREE_SEABIOS=$PUSH_TREE_SEABIOS
 TREE_OVMF=$PUSH_TREE_OVMF
 TREE_XTF=$PUSH_TREE_XTF
+TREE_OPENSTACK_NOVA=$PUSH_TREE_OPENSTACK_NOVA
 
 if info_linux_tree "$branch"; then
 	cd $repos/linux
@@ -129,6 +130,10 @@ ovmf)
 	cd $repos/ovmf
 	git push $TREE_OVMF $revision:refs/heads/xen-tested-master
 	;;
+openstack-nova)
+	cd $repos/openstack-nova
+	git push $TREE_OPENSTACK_NOVA $revision:refs/heads/xen-tested-master
+	;;
 osstest)
 	git push $HOME/testing.git $revision:production
 	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
diff --git a/cr-daily-branch b/cr-daily-branch
index 39483cd..5b24c47 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -207,6 +207,10 @@ if [ "x$REVISION_LINUXFIRMWARE" = x ]; then
 	determine_version REVISION_LINUXFIRMWARE linuxfirmware LINUXFIRMWARE
         export REVISION_LINUXFIRMWARE
 fi
+if [ "x$REVISION_OPENSTACK_NOVA" = x ]; then
+        determine_version REVISION_OPENSTACK_NOVA openstack-nova OPENSTACK_NOVA
+        export REVISION_OPENSTACK_NOVA
+fi
 
 case "$tree" in
 xen)
@@ -255,6 +259,10 @@ ovmf)
 	realtree=ovmf
 	NEW_REVISION=$REVISION_OVMF
 	;;
+openstack-nova)
+	realtree=openstack-nova
+	NEW_REVISION=$REVISION_OPENSTACK_NOVA
+	;;
 *)
 	NEW_REVISION=''
 	wantpush=false
diff --git a/cr-for-branches b/cr-for-branches
index a691273..7096e03 100755
--- a/cr-for-branches
+++ b/cr-for-branches
@@ -31,7 +31,7 @@ scriptoptions="$1"; shift
 LOGFILE=tmp/cr-for-branches.log
 export LOGFILE
 
-: ${BRANCHES:=osstest xen-4.0-testing xen-4.1-testing xen-4.2-testing xen-4.3-testing xen-4.4-testing xen-4.5-testing xen-4.6-testing xen-4.7-testing xen-4.8-testing xen-4.9-testing xen-unstable qemu-mainline qemu-upstream-unstable qemu-upstream-4.2-testing qemu-upstream-4.3-testing qemu-upstream-4.4-testing qemu-upstream-4.5-testing qemu-upstream-4.6-testing qemu-upstream-4.7-testing qemu-upstream-4.8-testing qemu-upstream-4.9-testing linux-linus linux-4.9 linux-4.1 linux-3.18 linux-3.16 linux-3.14 linux-3.10 linux-3.4 linux-arm-xen seabios ovmf xtf ${EXTRA_BRANCHES}}
+: ${BRANCHES:=osstest xen-4.0-testing xen-4.1-testing xen-4.2-testing xen-4.3-testing xen-4.4-testing xen-4.5-testing xen-4.6-testing xen-4.7-testing xen-4.8-testing xen-4.9-testing xen-unstable qemu-mainline qemu-upstream-unstable qemu-upstream-4.2-testing qemu-upstream-4.3-testing qemu-upstream-4.4-testing qemu-upstream-4.5-testing qemu-upstream-4.6-testing qemu-upstream-4.7-testing qemu-upstream-4.8-testing qemu-upstream-4.9-testing linux-linus linux-4.9 linux-4.1 linux-3.18 linux-3.16 linux-3.14 linux-3.10 linux-3.4 linux-arm-xen seabios ovmf xtf openstack-nova ${EXTRA_BRANCHES}}
 export BRANCHES
 
 fetchwlem=$wlem
diff --git a/cri-common b/cri-common
index 903fb4e..126f90c 100644
--- a/cri-common
+++ b/cri-common
@@ -82,6 +82,7 @@ select_xenbranch () {
 	ovmf)			tree=ovmf;	xenbranch=xen-unstable ;;
 	distros-*|examine)	tree=none;	xenbranch=xen-unstable ;;
 	osstest)		tree=osstest;	xenbranch=xen-unstable ;;
+	openstack-nova)	   tree=openstack-nova;	xenbranch=xen-unstable ;;
 	esac
 	if [ "x$tree" = xlinux ]; then
 		linuxbranch=$branch
diff --git a/make-flight b/make-flight
index 11cc01e..3235cca 100755
--- a/make-flight
+++ b/make-flight
@@ -183,6 +183,16 @@ job_create_test_filter_callback () {
         *) return 1;;
       esac
       ;;
+    openstack-nova)
+      case "$job" in
+        *-devstack) ;;
+        *-devstack-xsm) ;;
+        *) return 1;;
+      esac
+      case $dom0arch in
+          i386) return 1 ;;
+      esac
+      ;;
     *)
       case "$job" in
         *-qemuu-*)
@@ -216,7 +226,13 @@ arch_branch_filter_callback () {
         ovmf) return 1;;
         esac
         ;;
-  i386|amd64)
+  i386)
+        case "$branch" in
+        linux-arm-xen) return 1;;
+        openstack-nova) return 1;;
+        esac
+        ;;
+  amd64)
         case "$branch" in
         linux-arm-xen) return 1;;
         esac
@@ -678,6 +694,32 @@ do_examine_one () {
                   all_hostflags=$most_hostflags
 }
 
+do_openstack_tests () {
+    local xsms=$(xenbranch_xsm_variants)
+    local openstack_trees=(cinder devstack glance keystone nova
+        requirements tempest)
+
+    if [ $branch != openstack-nova ]; then
+        return
+    fi
+
+    local os_runvars=""
+    for tree in "${openstack_trees[@]}"; do
+        tree="openstack_$tree"
+        eval "os_runvars+=\" tree_$tree=\${TREE_${tree^^}}\""
+        eval "os_runvars+=\" revision_$tree=\${REVISION_${tree^^}}\""
+    done
+
+    for xsm in $xsms ; do
+        job_create_test test-$xenarch$kern-$dom0arch-devstack \
+            test-devstack libvirt $xenarch $dom0arch \
+            $os_runvars \
+            dom0_mem=4000 \
+            enable_xsm=$xsm \
+            all_hostflags=$most_hostflags
+    done
+}
+
 test_matrix_do_one () {
 
   do_pv_debian_tests
@@ -698,6 +740,7 @@ test_matrix_do_one () {
 
   do_rtds_tests
   do_credit2_tests
+  do_openstack_tests
 
   do_examine_one
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 04/20] ts-openstack-deploy: Set http proxy

[Anthony PERARD]

From: Ian Jackson <ian.jackson@eu.citrix.com>

This allows ./stack.sh to access the global internet.

CC: Anthony PERARD <anthony.perard@citrix.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 ts-openstack-deploy | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index 6f061eb..d2971f5 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -136,8 +136,11 @@ END
 }
 
 sub deploy() {
+    my $httpproxy = http_proxy_envsettings($ho);
+
     target_cmd($ho, <<END, 1800);
         set -e
+        $httpproxy
         cd $builddir/devstack
         ./stack.sh
 END
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 05/20] TestSupport: provide target_https_mitm_proxy_cert_path

[Anthony PERARD]

From: Ian Jackson <ian.jackson@eu.citrix.com>

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 Osstest/TestSupport.pm | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index ce9ffaa..80b97f8 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -64,6 +64,7 @@ BEGIN {
                       target_put_guest_image target_editfile
                       target_editfile_cancel target_fetchurl
                       http_proxy_envsettings
+                      target_https_mitm_proxy_cert_path
                       target_editfile_root target_file_exists
                       target_editfile_kvp_replace
                       target_run_apt
@@ -2695,4 +2696,10 @@ sub target_https_mitm_proxy_setup ($) {
     target_cmd_root($ho, 'update-ca-certificates', 300);
 }
 
+sub target_https_mitm_proxy_cert_path ($) {
+    my ($ho) = @_;
+    return undef unless length $c{HttpsProxyMITMCert};
+    return '/etc/ssl/certs/osstest.pem';
+}
+
 1;
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 06/20] ts-openstack-deploy: set CURL_CA_BUNDLE

[Anthony PERARD]

From: Ian Jackson <ian.jackson@eu.citrix.com>

This overrides pip's attempt to specify a specific certificate bundle,
and is necessary if we have a MITM SSL proxy.

The security implications are not ideal, because the MITM proxy will
allow any X.509 cert from any CA, whereas pip would only allow an
expected cert.  But we got pip via plain https to start with...

CC: Anthony PERARD <anthony.perard@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-openstack-deploy | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index d2971f5..6d7de1c 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -137,7 +137,10 @@ END
 
 sub deploy() {
     my $httpproxy = http_proxy_envsettings($ho);
-
+    my $mitmcert = target_https_mitm_proxy_cert_path($ho);
+    $httpproxy .=
+        "\n        CURL_CA_BUNDLE=$mitmcert; export CURL_CA_BUNDLE"
+        if $mitmcert;
     target_cmd($ho, <<END, 1800);
         set -e
         $httpproxy
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 07/20] ts-openstack-deploy: Keep CURL_CA_BUNDLE when sudo is called

[Anthony PERARD]

This is part of commit "ts-openstack-deploy: set CURL_CA_BUNDLE" but
also allow pip to work when it is called via sudo without preserving the
existing environment variables.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-openstack-deploy | 1 +
 1 file changed, 1 insertion(+)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index 6d7de1c..147071f 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -132,6 +132,7 @@ END
     target_putfilecontents_root_stash($ho, 100,
         <<END, "/etc/sudoers.d/devstack");
 osstest ALL=(ALL) NOPASSWD:ALL
+Defaults:osstest env_keep += "CURL_CA_BUNDLE"
 END
 }
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 08/20] ts-openstack-deploy: Try to disable use of SYSTEMD

[Anthony PERARD]

There is USE_SYSTEMD off by default, but it is now turn on if USE_SCREEN
if off. Try to keep use of systemd disable.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-openstack-deploy | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index 147071f..1349009 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -45,7 +45,6 @@ sub checkout () {
 [[local|localrc]]
 # Everything should be cloned by osstest, so devstack don't have to do it
 ERROR_ON_CLONE=True
-USE_SCREEN=False
 ADMIN_PASSWORD=secretadmin
 DATABASE_PASSWORD=secretdatabase
 RABBIT_PASSWORD=secretrabbit
@@ -71,6 +70,9 @@ disable_service q-meta
 disable_service q-agt
 disable_service q-l3
 enable_service n-net
+USE_SYSTEMD=False
+# To keep systemd off, we need to enable use of screen
+USE_SCREEN=True
 [[post-config|\$CINDER_CONF]]
 [lvmdriver-1]
 volume_group = $vg
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 09/20] ts-kernel-build: Enable network related modules for Neutron

[Anthony PERARD]

Those options/modules are needed to run OpenStack Neutron with Open
vSwitch.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-kernel-build | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/ts-kernel-build b/ts-kernel-build
index 94e67a4..0bcd340 100755
--- a/ts-kernel-build
+++ b/ts-kernel-build
@@ -252,10 +252,23 @@ setopt CONFIG_BLK_DEV_LOOP y
 
 setopt CONFIG_PACKET y
 
-# needed for OpenStack
-# because: https://bugzilla.redhat.com/show_bug.cgi?id=910619#c6
+# Used by OpenStack Neutron with Open vSwitch
+setopt CONFIG_OPENVSWITCH m
+setopt CONFIG_IP6_NF_RAW m
+setopt CONFIG_IP_NF_RAW m
+setopt CONFIG_IP_SET m
+setopt CONFIG_IP_SET_HASH_NET m
 setopt CONFIG_NETFILTER_ADVANCED y
+setopt CONFIG_NETFILTER_XT_CONNMARK m
+setopt CONFIG_NETFILTER_XT_MATCH_COMMENT m
+setopt CONFIG_NETFILTER_XT_MATCH_MAC m
+setopt CONFIG_NETFILTER_XT_MATCH_PHYSDEV m
+setopt CONFIG_NETFILTER_XT_SET m
 setopt CONFIG_NETFILTER_XT_TARGET_CHECKSUM m
+setopt CONFIG_NETFILTER_XT_TARGET_CT m
+setopt CONFIG_NETFILTER_XT_TARGET_REDIRECT m
+setopt CONFIG_NF_CONNTRACK_ZONES y
+setopt CONFIG_VETH m
 
 # Used by OpenStack Tempest to test encrypted volume
 setopt CONFIG_CRYPTO_XTS m
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 10/20] ts-openstack-deploy: Switch to Neutron for network

[Anthony PERARD]

nova-network is not supported anymore and Neutron is the default.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ap-common           | 3 ++-
 make-flight         | 2 +-
 ts-openstack-deploy | 8 +-------
 3 files changed, 4 insertions(+), 9 deletions(-)

diff --git a/ap-common b/ap-common
index 7e84f15..adfdc11 100644
--- a/ap-common
+++ b/ap-common
@@ -55,7 +55,8 @@
 : ${BASE_TREE_OVMF:=git://xenbits.xen.org/osstest/ovmf.git}
 
 define_openstack_trees() {
-    local openstack_trees=(cinder glance keystone nova requirements tempest)
+    local openstack_trees=(cinder glance keystone neutron nova requirements
+        tempest)
     local tree
     local url
 
diff --git a/make-flight b/make-flight
index 3235cca..ff4f17e 100755
--- a/make-flight
+++ b/make-flight
@@ -696,7 +696,7 @@ do_examine_one () {
 
 do_openstack_tests () {
     local xsms=$(xenbranch_xsm_variants)
-    local openstack_trees=(cinder devstack glance keystone nova
+    local openstack_trees=(cinder devstack glance keystone neutron nova
         requirements tempest)
 
     if [ $branch != openstack-nova ]; then
diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index 1349009..2107760 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -35,6 +35,7 @@ sub checkout () {
     build_clone($ho, 'openstack_devstack',     $builddir, 'devstack');
     build_clone($ho, 'openstack_glance',       $builddir, 'glance');
     build_clone($ho, 'openstack_keystone',     $builddir, 'keystone');
+    build_clone($ho, 'openstack_neutron',      $builddir, 'neutron');
     build_clone($ho, 'openstack_nova',         $builddir, 'nova');
     build_clone($ho, 'openstack_requirements', $builddir, 'requirements');
     build_clone($ho, 'openstack_tempest',      $builddir, 'tempest');
@@ -63,13 +64,6 @@ LIBVIRT_TYPE=xen
 disable_service horizon
 disable_service n-novnc
 disable_service dstat
-# Disable neutron and switch back to nova-network
-disable_service q-svc
-disable_service q-dhcp
-disable_service q-meta
-disable_service q-agt
-disable_service q-l3
-enable_service n-net
 USE_SYSTEMD=False
 # To keep systemd off, we need to enable use of screen
 USE_SCREEN=True
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 11/20] ts-openstack-deploy: Increase fd and memory limits for rabbitmq

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-openstack-deploy | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index 2107760..04317a0 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -130,6 +130,20 @@ END
 osstest ALL=(ALL) NOPASSWD:ALL
 Defaults:osstest env_keep += "CURL_CA_BUNDLE"
 END
+
+    # Increase some limits of rabbit server (message broker)
+    target_cmd_root($ho, <<END);
+        set -e
+        mkdir -p /etc/rabbitmq
+END
+    target_putfilecontents_root_stash($ho, 100,
+        <<END, "/etc/default/rabbitmq-server");
+ulimit -n 65536
+END
+    target_putfilecontents_root_stash($ho, 100,
+        <<END, "/etc/rabbitmq/rabbitmq.config");
+[{rabbit, [{vm_memory_high_watermark, 0.8}]}].
+END
 }
 
 sub deploy() {
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 12/20] make-flight: Increase dom0_mem for openstack flight

[Anthony PERARD]

With 4G for dom0_mem, a host running devstack is using about 1.5G of
swap.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 make-flight | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/make-flight b/make-flight
index ff4f17e..a0c9d2b 100755
--- a/make-flight
+++ b/make-flight
@@ -714,7 +714,7 @@ do_openstack_tests () {
         job_create_test test-$xenarch$kern-$dom0arch-devstack \
             test-devstack libvirt $xenarch $dom0arch \
             $os_runvars \
-            dom0_mem=4000 \
+            dom0_mem=6000 \
             enable_xsm=$xsm \
             all_hostflags=$most_hostflags
     done
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 13/20] ts-openstack-deploy: Apply a Tempest patch

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-openstack-deploy | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index 04317a0..04053de 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -144,6 +144,16 @@ END
         <<END, "/etc/rabbitmq/rabbitmq.config");
 [{rabbit, [{vm_memory_high_watermark, 0.8}]}].
 END
+
+    # Apply https://review.openstack.org/449695/ to tempest to workaround an
+    # issue. Check comments for more information
+    target_cmd($ho, <<END, 120);
+set -e
+cd $builddir/tempest
+git fetch origin refs/changes/95/449695/1
+git cherry-pick FETCH_HEAD
+END
+
 }
 
 sub deploy() {
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 14/20] ts-openstack-deploy: Ignore libvirt-python version and use latest

[Anthony PERARD]

Devstack is going to try to install a specific version of libvirt-python
(currently 2.5.0) but this fail with libvirt installed by osstest.
Remove the requirement and use the latest available instead.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-openstack-deploy | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index 04053de..8f6c7a2 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -93,6 +93,21 @@ END
         }
     );
 
+    target_editfile($ho,
+        "$builddir/requirements/upper-constraints.txt",
+        sub {
+            while (<EI>) {
+                # Ignore libvirt-python requirement and install latest,
+                # otherwise it's not going to work with latest libvirt
+                # installed by osstest.
+                if (m/^libvirt-python===.*$/) {
+                    next;
+                }
+                print EO or die $!;
+            }
+        }
+    );
+
     # Package python-systemd does not exist in Debian installed by osstest
     target_editfile($ho, "$builddir/devstack/files/debs/general", sub {
         while (<EI>) {
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 15/20] ts-openstack-tempest: Fix tempest invocation

[Anthony PERARD]

./run_tempest.sh is deprecated.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-openstack-tempest | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/ts-openstack-tempest b/ts-openstack-tempest
index 82e9a71..b95043a 100755
--- a/ts-openstack-tempest
+++ b/ts-openstack-tempest
@@ -58,7 +58,8 @@ sub tempest() {
 
     target_cmd($ho, <<END, 7200);
 set -e
-$builddir/tempest/run_tempest.sh --virtual-env -- --concurrency=2 '$regex'
+cd $builddir/tempest
+tempest run --concurrency=2 --regex '$regex'
 END
 }
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-openstack-tempest | 19 ++++++++-----------
 1 file changed, 8 insertions(+), 11 deletions(-)

diff --git a/ts-openstack-tempest b/ts-openstack-tempest
index b95043a..ae3662f 100755
--- a/ts-openstack-tempest
+++ b/ts-openstack-tempest
@@ -31,23 +31,20 @@ sub tempest() {
     my $scenario = 'tempest.scenario';
     my $volume_boot_pattern =
         "$scenario.test_volume_boot_pattern.TestVolumeBootPattern";
-    my $shelve_instance = "$scenario.test_shelve_instance.TestShelveInstance";
-
-    # Ignore tests which try to boot a guest with /dev/vda as boot device name.
-    push @ignored_tests,
-        "^\Q$volume_boot_pattern.test_volume_boot_pattern\E";
-    push @ignored_tests,
-        "^\Q$volume_boot_pattern.test_create_ebs_image_and_check_boot\E";
-    push @ignored_tests,
-        "^\Q$shelve_instance.test_shelve_volume_backed_instance\E";
 
     # Those tests access a volume through iSCSI. This does not work when both
     # the server and client of iSCSI are on the same Xen host (both in dom0),
     # Linux 4.0 is the first Linux to have a fix.
     push @ignored_tests,
-        "^\Q${volume_boot_pattern}V2.test_volume_boot_pattern\E";
+        "^\Q${volume_boot_pattern}.test_volume_boot_pattern\E";
+    push @ignored_tests,
+        "^\Q${volume_boot_pattern}.test_create_ebs_image_and_check_boot\E";
+
+    # See nova.git:devstack/tempest-dsvm-tempest-xen-rc
+    push @ignored_tests,
+        "^\Qtempest.api.compute.admin.test_volume_swap.TestVolumeSwap.test_volume_swap\E";
     push @ignored_tests,
-        "^\Q${volume_boot_pattern}V2.test_create_ebs_image_and_check_boot\E";
+        "^\Qtempest.api.compute.images.test_images.ImagesTestJSON.test_create_image_from_paused_server\E";
 
     # This regex below select the tests to run and exclude the ones marked as
     # slow as well as the explicit tests listed above.  It is based on the one
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 17/20] ts-openstack-deploy: Move logs to /var/log/openstack

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-openstack-deploy | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index 8f6c7a2..cffbb5d 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -58,7 +58,7 @@ DEST=$builddir
 DATA_DIR=\$DEST/data
 SERVICE_DIR=\$DEST/status
 SUBUNIT_OUTPUT=\$DEST/devstack.subunit
-LOGFILE=\$DEST/logs/stack.sh.log
+LOGDIR=/var/log/openstack
 LOG_COLOR=False
 LIBVIRT_TYPE=xen
 disable_service horizon
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 18/20] ts-logs-capture: Capture OpenStack logs

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-logs-capture | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/ts-logs-capture b/ts-logs-capture
index 061a118..0e3d267 100755
--- a/ts-logs-capture
+++ b/ts-logs-capture
@@ -171,6 +171,12 @@ sub fetch_logs_host () {
 
                   /var/core/*.core
 
+                  /var/log/openstack/*.log
+                  /etc/nova/*
+                  /etc/neutron/*
+                  /etc/cinder/*
+                  /home/osstest/build.*.test-*-devstack/tempest/etc/tempest.conf
+
                   )];
     if (!try_fetch_logs($ho, $logs)) {
         logm("log fetching failed, trying hard host reboot...");
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 19/20] ts-openstack-deploy: Increase devstack timeout

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-openstack-deploy | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ts-openstack-deploy b/ts-openstack-deploy
index cffbb5d..c21689d 100755
--- a/ts-openstack-deploy
+++ b/ts-openstack-deploy
@@ -177,7 +177,7 @@ sub deploy() {
     $httpproxy .=
         "\n        CURL_CA_BUNDLE=$mitmcert; export CURL_CA_BUNDLE"
         if $mitmcert;
-    target_cmd($ho, <<END, 1800);
+    target_cmd($ho, <<END, 3600);
         set -e
         $httpproxy
         cd $builddir/devstack
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 20/20] Introduce flight for stable branches of OpenStack

[Anthony PERARD]

OpenStack have many different repo which should be in sync, so this
patch should grab the revisions of the stable branch of every OpenStack
tree. Tempest does not have stable branch and should be able to test any
OpenStack version.

This patch also create flight for the latest release of OpenStack named
Ocata instead of testing branch master, as the scripts are not yet ready
to test it.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ap-fetch-version     | 18 +++++++++-
 ap-fetch-version-old |  8 +++++
 ap-print-url         |  2 +-
 ap-push              |  6 ++++
 cr-daily-branch      | 36 +++++++++++++++++---
 cr-for-branches      |  2 +-
 cri-common           |  2 +-
 make-flight          | 11 +++---
 8 files changed, 72 insertions(+), 13 deletions(-)

diff --git a/ap-fetch-version b/ap-fetch-version
index a714ee2..54265f4 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -106,10 +106,26 @@ ovmf)
 	repo_tree_rev_fetch_git ovmf \
 		$TREE_OVMF_UPSTREAM master $LOCALREV_OVMF
 	;;
-openstack-nova)
+openstack-tempest*)
+        # OpenStack Tempest does not have stable branches and should work with any
+        # version of OpenStack
 	repo_tree_rev_fetch_git openstack-nova \
 		$TREE_OPENSTACK_NOVA master $LOCALREV_OPENSTACK_NOVA
 	;;
+openstack-*-*)
+	os_tree="${branch#openstack-}"
+	os_tree="${os_tree%-*}"
+	branchcore="${branch##*-}"
+	eval repo_tree_rev_fetch_git "openstack-$os_tree" \
+		"\$TREE_OPENSTACK_${os_tree^^}" "stable/$branchcore" \
+                "\$LOCALREV_OPENSTACK_${os_tree^^}"
+	;;
+openstack-*)
+	os_tree="${branch#openstack-}"
+	eval repo_tree_rev_fetch_git "openstack-$os_tree" \
+		"\$TREE_OPENSTACK_${os_tree^^}" "master" \
+                "\$LOCALREV_OPENSTACK_${os_tree^^}"
+	;;
 osstest)
         if [ "x$OSSTEST_USE_HEAD" = "xy" ] ; then
 	    git update-ref -m "Arranging to test HEAD" \
diff --git a/ap-fetch-version-old b/ap-fetch-version-old
index 6dddbb7..00af8fa 100755
--- a/ap-fetch-version-old
+++ b/ap-fetch-version-old
@@ -115,6 +115,14 @@ ovmf)
 	repo_tree_rev_fetch_git ovmf \
 		$BASE_TREE_OVMF xen-tested-master $BASE_LOCALREV_OVMF
 	;;
+openstack-nova-*)
+	os_tree="${branch#openstack-}"
+	os_tree="${os_tree%-*}"
+	branchcore="${branch##*-}"
+	eval repo_tree_rev_fetch_git "openstack-$os_tree" \
+		"\$BASE_TREE_OPENSTACK_${os_tree^^}" "xen-tested-$branchcore" \
+                "\$BASE_LOCALREV_OPENSTACK_${os_tree^^}"
+	;;
 openstack-nova)
 	repo_tree_rev_fetch_git openstack-nova \
 		$BASE_TREE_OPENSTACK_NOVA xen-tested-master $BASE_LOCALREV_OPENSTACK_NOVA
diff --git a/ap-print-url b/ap-print-url
index 6f4e6b1..6d42ddf 100755
--- a/ap-print-url
+++ b/ap-print-url
@@ -67,7 +67,7 @@ ovmf)
 osstest)
 	echo none:;
 	;;
-openstack-nova)
+openstack-nova*)
 	echo $TREE_OPENSTACK_NOVA
 	;;
 *)
diff --git a/ap-push b/ap-push
index 136d1b6..0cac8a6 100755
--- a/ap-push
+++ b/ap-push
@@ -134,6 +134,12 @@ openstack-nova)
 	cd $repos/openstack-nova
 	git push $TREE_OPENSTACK_NOVA $revision:refs/heads/xen-tested-master
 	;;
+openstack-nova-*)
+        cd $repos/openstack-nova
+        openstack_nova_branch=${branch#openstack-nova-}
+        git push $TREE_OPENSTACK_NOVA \
+                $revision:refs/heads/xen-tested-stable-$openstack_nova_branch
+        ;;
 osstest)
 	git push $HOME/testing.git $revision:production
 	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
diff --git a/cr-daily-branch b/cr-daily-branch
index 5b24c47..20c95fc 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -207,10 +207,38 @@ if [ "x$REVISION_LINUXFIRMWARE" = x ]; then
 	determine_version REVISION_LINUXFIRMWARE linuxfirmware LINUXFIRMWARE
         export REVISION_LINUXFIRMWARE
 fi
-if [ "x$REVISION_OPENSTACK_NOVA" = x ]; then
-        determine_version REVISION_OPENSTACK_NOVA openstack-nova OPENSTACK_NOVA
-        export REVISION_OPENSTACK_NOVA
-fi
+openstack_rev() {
+        local os_tree="$1"
+        local os_branch
+
+        if eval [ "x\$REVISION_OPENSTACK_${os_tree^^}" = x ]; then
+                case "$branch" in
+                openstack-*-*)
+                        os_branch="openstack-$os_tree-${branch##*-}"
+                        os_git_branch="origin/stable/${branch##*-}"
+                        ;;
+                *)
+                        os_branch="openstack-$os_tree"
+                        os_git_branch="origin/master"
+                        ;;
+                esac
+
+                # Use latest version, even for other openstack trees
+                # so branch openstack-nova-ocata should have other trees like
+                # openstack-neutron have the revision of the same branch fetch
+                # at the same time
+                if [ "$branch" != "$os_branch" ]; then
+                        eval "export REVISION_OPENSTACK_${os_tree^^}=$os_git_branch"
+                        return
+                fi
+                determine_version "REVISION_OPENSTACK_${os_tree^^}" \
+                        "$os_branch" "OPENSTACK_${os_tree^^}"
+                eval "export REVISION_OPENSTACK_${os_tree^^}"
+        fi
+}
+for os_tree in cinder devstack glance keystone neutron nova requirements; do
+        openstack_rev "$os_tree"
+done
 
 case "$tree" in
 xen)
diff --git a/cr-for-branches b/cr-for-branches
index 7096e03..17f8b58 100755
--- a/cr-for-branches
+++ b/cr-for-branches
@@ -31,7 +31,7 @@ scriptoptions="$1"; shift
 LOGFILE=tmp/cr-for-branches.log
 export LOGFILE
 
-: ${BRANCHES:=osstest xen-4.0-testing xen-4.1-testing xen-4.2-testing xen-4.3-testing xen-4.4-testing xen-4.5-testing xen-4.6-testing xen-4.7-testing xen-4.8-testing xen-4.9-testing xen-unstable qemu-mainline qemu-upstream-unstable qemu-upstream-4.2-testing qemu-upstream-4.3-testing qemu-upstream-4.4-testing qemu-upstream-4.5-testing qemu-upstream-4.6-testing qemu-upstream-4.7-testing qemu-upstream-4.8-testing qemu-upstream-4.9-testing linux-linus linux-4.9 linux-4.1 linux-3.18 linux-3.16 linux-3.14 linux-3.10 linux-3.4 linux-arm-xen seabios ovmf xtf openstack-nova ${EXTRA_BRANCHES}}
+: ${BRANCHES:=osstest xen-4.0-testing xen-4.1-testing xen-4.2-testing xen-4.3-testing xen-4.4-testing xen-4.5-testing xen-4.6-testing xen-4.7-testing xen-4.8-testing xen-4.9-testing xen-unstable qemu-mainline qemu-upstream-unstable qemu-upstream-4.2-testing qemu-upstream-4.3-testing qemu-upstream-4.4-testing qemu-upstream-4.5-testing qemu-upstream-4.6-testing qemu-upstream-4.7-testing qemu-upstream-4.8-testing qemu-upstream-4.9-testing linux-linus linux-4.9 linux-4.1 linux-3.18 linux-3.16 linux-3.14 linux-3.10 linux-3.4 linux-arm-xen seabios ovmf xtf openstack-nova-ocata ${EXTRA_BRANCHES}}
 export BRANCHES
 
 fetchwlem=$wlem
diff --git a/cri-common b/cri-common
index 126f90c..ae091fb 100644
--- a/cri-common
+++ b/cri-common
@@ -82,7 +82,7 @@ select_xenbranch () {
 	ovmf)			tree=ovmf;	xenbranch=xen-unstable ;;
 	distros-*|examine)	tree=none;	xenbranch=xen-unstable ;;
 	osstest)		tree=osstest;	xenbranch=xen-unstable ;;
-	openstack-nova)	   tree=openstack-nova;	xenbranch=xen-unstable ;;
+	openstack-nova*)   tree=openstack-nova;	xenbranch=xen-unstable ;;
 	esac
 	if [ "x$tree" = xlinux ]; then
 		linuxbranch=$branch
diff --git a/make-flight b/make-flight
index a0c9d2b..14b27c4 100755
--- a/make-flight
+++ b/make-flight
@@ -183,7 +183,7 @@ job_create_test_filter_callback () {
         *) return 1;;
       esac
       ;;
-    openstack-nova)
+    openstack-nova*)
       case "$job" in
         *-devstack) ;;
         *-devstack-xsm) ;;
@@ -229,7 +229,7 @@ arch_branch_filter_callback () {
   i386)
         case "$branch" in
         linux-arm-xen) return 1;;
-        openstack-nova) return 1;;
+        openstack-nova*) return 1;;
         esac
         ;;
   amd64)
@@ -699,9 +699,10 @@ do_openstack_tests () {
     local openstack_trees=(cinder devstack glance keystone neutron nova
         requirements tempest)
 
-    if [ $branch != openstack-nova ]; then
-        return
-    fi
+    case "$branch" in
+        openstack-nova*)          ;;
+        *)                return  ;;
+    esac
 
     local os_runvars=""
     for tree in "${openstack_trees[@]}"; do
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 00/20] Have OpenStack tested on top of xen's master and libvirt's master.

[Anthony PERARD]

On Fri, Jun 16, 2017 at 06:56:44PM +0100, Anthony PERARD wrote:
> Changes in V11:
> - plenty of new patches, on top of the original 3 patches that were acked.
> - and an attempt at creating a flight for a stable branch of openstack. But
>   there is many git tree to pull the branch from.

And here is a branch(tag) for all those commits:
https://xenbits.xen.org/git-http/people/aperard/osstest.git
tag: openstack-v11

> Anthony PERARD (17):
>   ts-openstack-deploy: Deploy OpenStack on a host with devstack
>   ts-openstack-tempest: Run Tempest to check OpenStack
>   Create a flight to test OpenStack with xen-unstable and libvirt
>   ts-openstack-deploy: Keep CURL_CA_BUNDLE when sudo is called
>   ts-openstack-deploy: Try to disable use of SYSTEMD
>   ts-kernel-build: Enable network related modules for Neutron
>   ts-openstack-deploy: Switch to Neutron for network
>   ts-openstack-deploy: Increase fd and memory limits for rabbitmq
>   make-flight: Increase dom0_mem for openstack flight
>   ts-openstack-deploy: Apply a Tempest patch
>   ts-openstack-deploy: Ignore libvirt-python version and use latest
>   ts-openstack-tempest: Fix tempest invocation
>   ts-openstack-tempest: Update list of skipped tests
>   ts-openstack-deploy: Move logs to /var/log/openstack
>   ts-logs-capture: Capture OpenStack logs
>   ts-openstack-deploy: Increase devstack timeout
>   Introduce flight for stable branches of OpenStack
> 
> Ian Jackson (3):
>   ts-openstack-deploy: Set http proxy
>   TestSupport: provide target_https_mitm_proxy_cert_path
>   ts-openstack-deploy: set CURL_CA_BUNDLE
> 
>  Osstest/TestSupport.pm |   7 +
>  ap-common              |  19 ++
>  ap-fetch-version       |  20 +++
>  ap-fetch-version-old   |  13 ++
>  ap-print-url           |   3 +
>  ap-push                |  11 ++
>  cr-daily-branch        |  36 ++++
>  cr-for-branches        |   2 +-
>  cri-common             |   1 +
>  make-flight            |  46 ++++-
>  sg-run-job             |   6 +
>  ts-kernel-build        |  17 +-
>  ts-logs-capture        |   6 +
>  ts-openstack-deploy    | 190 ++++++++++++++++++++
>  ts-openstack-tempest   |  63 +++++++
>  15 files changed, 436 insertions(+), 4 deletions(-)
>  create mode 100755 ts-openstack-deploy
>  create mode 100755 ts-openstack-tempest
> 
> -- 
> Anthony PERARD
> 

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 07/20] ts-openstack-deploy: Keep CURL_CA_BUNDLE when sudo is called

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 07/20] ts-openstack-deploy: Keep CURL_CA_BUNDLE when sudo is called"):
> This is part of commit "ts-openstack-deploy: set CURL_CA_BUNDLE" but
> also allow pip to work when it is called via sudo without preserving the
> existing environment variables.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 08/20] ts-openstack-deploy: Try to disable use of SYSTEMD

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 08/20] ts-openstack-deploy: Try to disable use of SYSTEMD"):
> There is USE_SYSTEMD off by default, but it is now turn on if USE_SCREEN
                                                         ^ed
> if off. Try to keep use of systemd disable.
                                            ^d

> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 09/20] ts-kernel-build: Enable network related modules for Neutron

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 09/20] ts-kernel-build: Enable network related modules for Neutron"):
> Those options/modules are needed to run OpenStack Neutron with Open
> vSwitch.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 10/20] ts-openstack-deploy: Switch to Neutron for network

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 10/20] ts-openstack-deploy: Switch to Neutron for network"):
> nova-network is not supported anymore and Neutron is the default.

Is this kind of thing going to be common ?  If so then it will be a
constant maintenance burden in osstest.

Is there some way we can get the list of components out of the
top-level tree ?

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 11/20] ts-openstack-deploy: Increase fd and memory limits for rabbitmq

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 11/20] ts-openstack-deploy: Increase fd and memory limits for rabbitmq"):
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Does this not mean that the upstream defaults are wrong ?

In general I like to see at least an upstream bug report url or number
mentioned in a comment near the workaround.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 12/20] make-flight: Increase dom0_mem for openstack flight

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 12/20] make-flight: Increase dom0_mem for openstack flight"):
> With 4G for dom0_mem, a host running devstack is using about 1.5G of
> swap.

Is this going to work properly on 8G hosts ?

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 13/20] ts-openstack-deploy: Apply a Tempest patch

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 13/20] ts-openstack-deploy: Apply a Tempest patch"):
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

>  ts-openstack-deploy | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/ts-openstack-deploy b/ts-openstack-deploy
> index 04317a0..04053de 100755
> --- a/ts-openstack-deploy
> +++ b/ts-openstack-deploy
> @@ -144,6 +144,16 @@ END
>          <<END, "/etc/rabbitmq/rabbitmq.config");
>  [{rabbit, [{vm_memory_high_watermark, 0.8}]}].
>  END
> +
> +    # Apply https://review.openstack.org/449695/ to tempest to workaround an
> +    # issue. Check comments for more information
> +    target_cmd($ho, <<END, 120);
> +set -e
> +cd $builddir/tempest
> +git fetch origin refs/changes/95/449695/1
> +git cherry-pick FETCH_HEAD
> +END

I foresee the need to generalise this...

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 14/20] ts-openstack-deploy: Ignore libvirt-python version and use latest

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 14/20] ts-openstack-deploy: Ignore libvirt-python version and use latest"):
> Devstack is going to try to install a specific version of libvirt-python
> (currently 2.5.0) but this fail with libvirt installed by osstest.
> Remove the requirement and use the latest available instead.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 15/20] ts-openstack-tempest: Fix tempest invocation

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 15/20] ts-openstack-tempest: Fix tempest invocation"):
> ./run_tempest.sh is deprecated.
...
>      target_cmd($ho, <<END, 7200);
>  set -e
> -$builddir/tempest/run_tempest.sh --virtual-env -- --concurrency=2 '$regex'
> +cd $builddir/tempest
> +tempest run --concurrency=2 --regex '$regex'

Has /usr/local/bin/tempest or something been created by
ts-openstay-deploy, then ?

If so,

  Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

Otherwise I wonder how this works, since I don't see how `tempest'
would be on PATH.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests"):
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Again, does this not mean we're going to suffer a maintenance burden
as tempest grows new inapplicable tests ?

Other possibilities that come to my mind: ideally the tempest tests
would have metadata so that particular classes of tests could be
skipped.  Alternatively, if they aren't disruptive or very slow, if
run, we could run them anyway as substeps.

Which makes me think: maybe the tempest tests want to be substeps
anyway.  Is that possible ?  (Does tempest speak subunit or
something ?)

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 17/20] ts-openstack-deploy: Move logs to /var/log/openstack

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 17/20] ts-openstack-deploy: Move logs to /var/log/openstack"):
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[OSSTEST PATCH v11 18/20] ts-logs-capture: Capture OpenStack logs

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 18/20] ts-logs-capture: Capture OpenStack logs"):
> +                  /var/log/openstack/*.log
> +                  /etc/nova/*
> +                  /etc/neutron/*
> +                  /etc/cinder/*

This is fine:

> +                  /home/osstest/build.*.test-*-devstack/tempest/etc/tempest.conf

This is not fine.  If a build host is shared, it will collect all the
logs from all of the builds.

I think build ts-* scripts ought to collect their own logs.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 19/20] ts-openstack-deploy: Increase devstack timeout

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 19/20] ts-openstack-deploy: Increase devstack timeout"):
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 20/20] Introduce flight for stable branches of OpenStack

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 20/20] Introduce flight for stable branches of OpenStack"):
> OpenStack have many different repo which should be in sync, so this
> patch should grab the revisions of the stable branch of every OpenStack
> tree. Tempest does not have stable branch and should be able to test any
> OpenStack version.

I'm afraid I don't understand this patch.

Partly, I think it needs to be squashed with the original patch
introducing `openstack-nova' as a branch.  While your series has a
number of things where a thing is introduced and then later patched,
and this is largely OK, I think it is too confusing to have a whole
branch appear and disappear like this (without ever having been
run for real).

> -openstack-nova)
> +openstack-tempest*)
> +        # OpenStack Tempest does not have stable branches and should work with any

Your comment lines need wrapping to ~70-75 (here and later).

> +        # version of OpenStack
>  	repo_tree_rev_fetch_git openstack-nova \
>  		$TREE_OPENSTACK_NOVA master $LOCALREV_OPENSTACK_NOVA
>  	;;
> +openstack-*-*)

I think you need to document what your openstack-*-* branch names are
going to be.

And, you make provision here for branches openstack-tempest* but you
don't add any such branches to cr-for-branches.

I forget what decisions we made about the push gate logic for the
various openstack branches.  I think it would be worth explicitly
writing this down in tree (in a comment in ap-fetch-* perhaps).

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 10/20] ts-openstack-deploy: Switch to Neutron for network

[Anthony PERARD]

On Fri, Jun 23, 2017 at 02:41:02PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[OSSTEST PATCH v11 10/20] ts-openstack-deploy: Switch to Neutron for network"):
> > nova-network is not supported anymore and Neutron is the default.
> 
> Is this kind of thing going to be common ?  If so then it will be a
> constant maintenance burden in osstest.

No, that's not common. I don't think it's going to happen again anytime
soon.

> Is there some way we can get the list of components out of the
> top-level tree ?

I don't know if there is a simple way to get this list. And if we start
looking at the list of all OpenStack git trees, there are hundreds of
them.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 10/20] ts-openstack-deploy: Switch to Neutron for network

[Ian Jackson]

Anthony PERARD writes ("Re: [OSSTEST PATCH v11 10/20] ts-openstack-deploy: Switch to Neutron for network"):
> On Fri, Jun 23, 2017 at 02:41:02PM +0100, Ian Jackson wrote:
> > Is this kind of thing going to be common ?  If so then it will be a
> > constant maintenance burden in osstest.
> 
> No, that's not common. I don't think it's going to happen again anytime
> soon.

OK, fine.

> > Is there some way we can get the list of components out of the
> > top-level tree ?
> 
> I don't know if there is a simple way to get this list. And if we start
> looking at the list of all OpenStack git trees, there are hundreds of
> them.

Yow.  Let's not.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 11/20] ts-openstack-deploy: Increase fd and memory limits for rabbitmq

[Anthony PERARD]

On Fri, Jun 23, 2017 at 02:41:59PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[OSSTEST PATCH v11 11/20] ts-openstack-deploy: Increase fd and memory limits for rabbitmq"):
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> Does this not mean that the upstream defaults are wrong ?

That depends on what you meant by upstream.

On Ubuntu, the limit of open fd is set to an higher value via the
systemd unit file. It does not look like devstack (OpenStack) is
changing this.

In devstack, I've seen an increate of the limit of open fd, but via
systemd, so for all systemd services.


As for the memory limit, it was necessary with a host of 4G of RAM. But
I did not try again with the default limit and a host with 6G of RAM.
(The default is 0.4, here I've set it to 0.8)

> In general I like to see at least an upstream bug report url or number
> mentioned in a comment near the workaround.
> 
> Ian.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 12/20] make-flight: Increase dom0_mem for openstack flight

[Anthony PERARD]

On Fri, Jun 23, 2017 at 02:42:53PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[OSSTEST PATCH v11 12/20] make-flight: Increase dom0_mem for openstack flight"):
> > With 4G for dom0_mem, a host running devstack is using about 1.5G of
> > swap.
> 
> Is this going to work properly on 8G hosts ?

Yes, it is fine. I usually do my OpenStack testing on 8G hosts. The CI
loop is running with testing on 8G, and have a bit more than 7G for
dom0. Tempest only run small VM (64MB or 128MB) and only a few at a
time.

So yes.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 15/20] ts-openstack-tempest: Fix tempest invocation

[Anthony PERARD]

On Fri, Jun 23, 2017 at 02:45:12PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[OSSTEST PATCH v11 15/20] ts-openstack-tempest: Fix tempest invocation"):
> > ./run_tempest.sh is deprecated.
> ...
> >      target_cmd($ho, <<END, 7200);
> >  set -e
> > -$builddir/tempest/run_tempest.sh --virtual-env -- --concurrency=2 '$regex'
> > +cd $builddir/tempest
> > +tempest run --concurrency=2 --regex '$regex'
> 
> Has /usr/local/bin/tempest or something been created by
> ts-openstay-deploy, then ?

Yes.

> If so,
> 
>   Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
> 
> Otherwise I wonder how this works, since I don't see how `tempest'
> would be on PATH.
> 
> Ian.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests

[Anthony PERARD]

On Fri, Jun 23, 2017 at 02:47:37PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests"):
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> 
> Again, does this not mean we're going to suffer a maintenance burden
> as tempest grows new inapplicable tests ?

That exactly what is happening with the OpenStack CI loop, from time to
time, there are new tests, that can fail, and diffenrent ways to fix
this (fix the bug, fix a configuration, or just skip the tests).

Recently, we've actually push the list of tests to skip into nova.git,
but I think it is only available in master.

> Other possibilities that come to my mind: ideally the tempest tests
> would have metadata so that particular classes of tests could be
> skipped.  Alternatively, if they aren't disruptive or very slow, if
> run, we could run them anyway as substeps.
> 
> Which makes me think: maybe the tempest tests want to be substeps
> anyway.  Is that possible ?  (Does tempest speak subunit or
> something ?)

I think it is subunit, yes. And I've got the command that is been run by
tempest (as it's printed on stdout).

running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover} --list

then several:
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover}  --load-list /tmp/tmpzSNsrB



-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 18/20] ts-logs-capture: Capture OpenStack logs

[Anthony PERARD]

On Fri, Jun 23, 2017 at 02:49:11PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[OSSTEST PATCH v11 18/20] ts-logs-capture: Capture OpenStack logs"):
> > +                  /var/log/openstack/*.log
> > +                  /etc/nova/*
> > +                  /etc/neutron/*
> > +                  /etc/cinder/*
> 
> This is fine:
> 
> > +                  /home/osstest/build.*.test-*-devstack/tempest/etc/tempest.conf
> 
> This is not fine.  If a build host is shared, it will collect all the
> logs from all of the builds.

This file should not exist on a build host, as it would be created by
ts-openstack-deploy which I hope is not run on a build host.

We are only using using target_jobdir() to have a directory to clone all
the OpenStack repo and run ./devstack.

But I can find a way to copy this file somewhere else.

> I think build ts-* scripts ought to collect their own logs.

Do you mean I can capture logs like ts-logs-capture would do, but before
it is been run?

ts-openstack-deploy could collect all the configuration file then, like
/etc/nova/*.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 11/20] ts-openstack-deploy: Increase fd and memory limits for rabbitmq

[Ian Jackson]

Anthony PERARD writes ("Re: [OSSTEST PATCH v11 11/20] ts-openstack-deploy: Increase fd and memory limits for rabbitmq"):
> On Fri, Jun 23, 2017 at 02:41:59PM +0100, Ian Jackson wrote:
> > Anthony PERARD writes ("[OSSTEST PATCH v11 11/20] ts-openstack-deploy: Increase fd and memory limits for rabbitmq"):
> > > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> > 
> > Does this not mean that the upstream defaults are wrong ?
> 
> That depends on what you meant by upstream.
> 
> On Ubuntu, the limit of open fd is set to an higher value via the
> systemd unit file. It does not look like devstack (OpenStack) is
> changing this.
> 
> In devstack, I've seen an increate of the limit of open fd, but via
> systemd, so for all systemd services.

And if one isn't using systemd ?

> As for the memory limit, it was necessary with a host of 4G of RAM. But
> I did not try again with the default limit and a host with 6G of RAM.
> (The default is 0.4, here I've set it to 0.8)

It sounds like the default calculation is not right, then ?

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 12/20] make-flight: Increase dom0_mem for openstack flight

[Ian Jackson]

Anthony PERARD writes ("Re: [OSSTEST PATCH v11 12/20] make-flight: Increase dom0_mem for openstack flight"):
> On Fri, Jun 23, 2017 at 02:42:53PM +0100, Ian Jackson wrote:
> > Anthony PERARD writes ("[OSSTEST PATCH v11 12/20] make-flight: Increase dom0_mem for openstack flight"):
> > > With 4G for dom0_mem, a host running devstack is using about 1.5G of
> > > swap.
> > 
> > Is this going to work properly on 8G hosts ?
> 
> Yes, it is fine. I usually do my OpenStack testing on 8G hosts. The CI
> loop is running with testing on 8G, and have a bit more than 7G for
> dom0. Tempest only run small VM (64MB or 128MB) and only a few at a
> time.

OK, great.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests

[Ian Jackson]

Anthony PERARD writes ("Re: [OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests"):
> On Fri, Jun 23, 2017 at 02:47:37PM +0100, Ian Jackson wrote:
> > Again, does this not mean we're going to suffer a maintenance burden
> > as tempest grows new inapplicable tests ?
> 
> That exactly what is happening with the OpenStack CI loop, from time to
> time, there are new tests, that can fail, and diffenrent ways to fix
> this (fix the bug, fix a configuration, or just skip the tests).
> 
> Recently, we've actually push the list of tests to skip into nova.git,
> but I think it is only available in master.

Aha.  Well, if the stable branch is stable then the set of tests to
skip there is probably stable too ?  And on master it's built-in ?  So
this actually won't be a problem - in the sense that this will be
approximately the last necessary update to the list of tests to skip ?

> > Which makes me think: maybe the tempest tests want to be substeps
> > anyway.  Is that possible ?  (Does tempest speak subunit or
> > something ?)
> 
> I think it is subunit, yes. And I've got the command that is been run by
> tempest (as it's printed on stdout).
> 
> running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
> OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
> OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
> OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
> ${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover} --list
> 
> then several:
> running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
> OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
> OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
> OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
> ${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover}  --load-list /tmp/tmpzSNsrB

It would be really good to have those individual subunit results as
substeps.  There is no subunit parser in osstest yet but we could have
one.

What version of subunit does it print out ?  The subunit v1 protocol
is lovely and simple but there is a Second System :-/.

I guess that tempest doesn't stop on the first failed test ?  So
perhaps we could just tolerate the failed-but-not-skipped tests ?

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 18/20] ts-logs-capture: Capture OpenStack logs

[Ian Jackson]

Anthony PERARD writes ("Re: [OSSTEST PATCH v11 18/20] ts-logs-capture: Capture OpenStack logs"):
> On Fri, Jun 23, 2017 at 02:49:11PM +0100, Ian Jackson wrote:
> > This is not fine.  If a build host is shared, it will collect all the
> > logs from all of the builds.
> 
> This file should not exist on a build host, as it would be created by
> ts-openstack-deploy which I hope is not run on a build host.

Ah.  Sorry.  I think I got confused.

In the future even test hosts might be shared sequentially by
different jobs but I think given what your ts-openstack-deploy does,
we can't make that true for these jobs.

> We are only using using target_jobdir() to have a directory to clone all
> the OpenStack repo and run ./devstack.

Aha.

> But I can find a way to copy this file somewhere else.

I now see that you have an openstack-tempest-test-specific filename so
it is fine to try to collect this file in all jobs.

> > I think build ts-* scripts ought to collect their own logs.
> 
> Do you mean I can capture logs like ts-logs-capture would do, but before
> it is been run?
> 
> ts-openstack-deploy could collect all the configuration file then, like
> /etc/nova/*.

I'm sorry, forget what I said about build ts-* scripts.


So, the upshot is:

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

and sorry for not paying proper attention.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 20/20] Introduce flight for stable branches of OpenStack

[Anthony PERARD]

On Fri, Jun 23, 2017 at 02:58:25PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[OSSTEST PATCH v11 20/20] Introduce flight for stable branches of OpenStack"):
> > OpenStack have many different repo which should be in sync, so this
> > patch should grab the revisions of the stable branch of every OpenStack
> > tree. Tempest does not have stable branch and should be able to test any
> > OpenStack version.
> 
> I'm afraid I don't understand this patch.
> 
> Partly, I think it needs to be squashed with the original patch
> introducing `openstack-nova' as a branch.  While your series has a
> number of things where a thing is introduced and then later patched,
> and this is largely OK, I think it is too confusing to have a whole
> branch appear and disappear like this (without ever having been
> run for real).

Yes, I can do that.

> > -openstack-nova)
> > +openstack-tempest*)
> > +        # OpenStack Tempest does not have stable branches and should work with any
> 
> Your comment lines need wrapping to ~70-75 (here and later).
> 
> > +        # version of OpenStack
> >  	repo_tree_rev_fetch_git openstack-nova \
> >  		$TREE_OPENSTACK_NOVA master $LOCALREV_OPENSTACK_NOVA
> >  	;;
> > +openstack-*-*)
> 
> I think you need to document what your openstack-*-* branch names are
> going to be.
> 
> And, you make provision here for branches openstack-tempest* but you
> don't add any such branches to cr-for-branches.
> 
> I forget what decisions we made about the push gate logic for the
> various openstack branches.  I think it would be worth explicitly
> writing this down in tree (in a comment in ap-fetch-* perhaps).

We decided to track only nova.git, and let osstest clone other trees and
checkout the HEAD (by having REVISION_* empty). That is fine if we track
"master" of nova.git.

The issue now is if we want to track a branch of OpenStack, like
"stable/ocata" (which is the latest release), we need to have all trees
(nova.git, devstack.git, ...) checkout the same branch. (tempest.git
does not have release specific branch.)

I think this patch is confusing because I originally try to use osstest
scripts to find which commit to use for every trees and so have add the
necessary into ./ap-fetch-version. But I could not make that works
without duplicating some functions and so went with writing
'origin/stable/ocata' into REVISION_*.


In term of osstest branches, in cr-for-branches, I would like only one
branch. 'openstack-nova-ocata' would be a the branch that test the
OpenStack release named 'Ocata'. Later, once this branch run smoothly,
we can add a branch to test 'master', the development branch of
OpenStack, it was called 'openstack-nova' in the original 3 patch of
this patch series.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 20/20] Introduce flight for stable branches of OpenStack

[Ian Jackson]

Anthony PERARD writes ("Re: [OSSTEST PATCH v11 20/20] Introduce flight for stable branches of OpenStack"):
> We decided to track only nova.git, and let osstest clone other trees and
> checkout the HEAD (by having REVISION_* empty). That is fine if we track
> "master" of nova.git.

Right.

> The issue now is if we want to track a branch of OpenStack, like
> "stable/ocata" (which is the latest release), we need to have all trees
> (nova.git, devstack.git, ...) checkout the same branch. (tempest.git
> does not have release specific branch.)

Ah.

> I think this patch is confusing because I originally try to use osstest
> scripts to find which commit to use for every trees and so have add the
> necessary into ./ap-fetch-version. But I could not make that works
> without duplicating some functions and so went with writing
> 'origin/stable/ocata' into REVISION_*.

OIC.

I think it is fine to write origin/stable/ocata into REVISION_*, in
this case.  Of course it means that the sub-trees are not revision
controlled by the osstest push-gate.  IIRC you said that bugs due to
those sub-trees would not be very likely.

> In term of osstest branches, in cr-for-branches, I would like only one
> branch. 'openstack-nova-ocata' would be a the branch that test the
> OpenStack release named 'Ocata'. Later, once this branch run smoothly,
> we can add a branch to test 'master', the development branch of
> OpenStack, it was called 'openstack-nova' in the original 3 patch of
> this patch series.

Aha.

I will go and read the patch again.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 11/20] ts-openstack-deploy: Increase fd and memory limits for rabbitmq

[Anthony PERARD]

On Fri, Jun 23, 2017 at 04:34:41PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("Re: [OSSTEST PATCH v11 11/20] ts-openstack-deploy: Increase fd and memory limits for rabbitmq"):
> > On Fri, Jun 23, 2017 at 02:41:59PM +0100, Ian Jackson wrote:
> > > Anthony PERARD writes ("[OSSTEST PATCH v11 11/20] ts-openstack-deploy: Increase fd and memory limits for rabbitmq"):
> > > > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> > > 
> > > Does this not mean that the upstream defaults are wrong ?
> > 
> > That depends on what you meant by upstream.
> > 
> > On Ubuntu, the limit of open fd is set to an higher value via the
> > systemd unit file. It does not look like devstack (OpenStack) is
> > changing this.
> > 
> > In devstack, I've seen an increate of the limit of open fd, but via
> > systemd, so for all systemd services.
> 
> And if one isn't using systemd ?

I guess it would have to fix devstack. Also, the default is 1k, devstack
increase it to 2k, but rabbitmq on Ubuntu have a limit at 65k.

> > As for the memory limit, it was necessary with a host of 4G of RAM. But
> > I did not try again with the default limit and a host with 6G of RAM.
> > (The default is 0.4, here I've set it to 0.8)
> 
> It sounds like the default calculation is not right, then ?

I think 0.4 means 40% of the RAM. Sorry, I should have said so. Also,
0.4 is the same on the CI loop, with a dom0 of 7G or so.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 20/20] Introduce flight for stable branches of OpenStack

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST PATCH v11 20/20] Introduce flight for stable branches of OpenStack"):
> OpenStack have many different repo which should be in sync, so this
> patch should grab the revisions of the stable branch of every OpenStack
> tree. Tempest does not have stable branch and should be able to test any
> OpenStack version.
...
> +openstack-*-*)
> +	os_tree="${branch#openstack-}"
> +	os_tree="${os_tree%-*}"
> +	branchcore="${branch##*-}"
> +	eval repo_tree_rev_fetch_git "openstack-$os_tree" \
> +		"\$TREE_OPENSTACK_${os_tree^^}" "stable/$branchcore" \
> +                "\$LOCALREV_OPENSTACK_${os_tree^^}"

From your previous email:

  I think this patch is confusing because I originally try to use osstest
  scripts to find which commit to use for every trees and so have add the
  necessary into ./ap-fetch-version. But I could not make that works
  without duplicating some functions and so went with writing
  'origin/stable/ocata' into REVISION_*.

I think I am indeed still confused by some of it.

For example:

> +openstack_rev() {
> +        local os_tree="$1"
> +        local os_branch
> +
> +        if eval [ "x\$REVISION_OPENSTACK_${os_tree^^}" = x ]; then
> +                case "$branch" in
> +                openstack-*-*)
> +                        os_branch="openstack-$os_tree-${branch##*-}"
> +                        os_git_branch="origin/stable/${branch##*-}"
> +                        ;;
> +                *)
> +                        os_branch="openstack-$os_tree"
> +                        os_git_branch="origin/master"
> +                        ;;
> +                esac
> +
> +                # Use latest version, even for other openstack
> +                # trees so branch openstack-nova-ocata should have
> +                # other trees like openstack-neutron have the
> +                # revision of the same branch fetch at the same
> +                # time
> +                if [ "$branch" != "$os_branch" ]; then
> +                        eval "export REVISION_OPENSTACK_${os_tree^^}=$os_git_branch"
> +                        return
> +                fi
> +                determine_version "REVISION_OPENSTACK_${os_tree^^}" \
> +                        "$os_branch" "OPENSTACK_${os_tree^^}"
> +                eval "export REVISION_OPENSTACK_${os_tree^^}"
> +        fi
> +}
> +for os_tree in cinder devstack glance keystone neutron nova requirements; do
> +        openstack_rev "$os_tree"
> +done

I wonder if this full generality is really necessary ?  If you don't
intend branches like   openstack-ocata-neutron   then it would be
sufficient to call one function for nova and another for the other
trees.

And, frankly, I don't think we could have branches like
`openstack-ocata-neutron'.  That would be too many branches.

So perhaps the branch `openstack-ocata-nova' should be called
`openstack-ocata' ?


Also, right now I think it's clear that we're not intending to add
openstack jobs to existing branches' flights.  But if we were to do
that in the future, we would want all the subtrees to be tracked.

Maybe we should have a way for cr-daily-branch to fetch, and push,
multiple trees.  We could call ap-fetch-version on every tree,
and set the appropriate variable (with determine_version, as you have
above).  And then set a variable to call ap-push multiple times, if we
get a pass.


Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 11/20] ts-openstack-deploy: Increase fd and memory limits for rabbitmq

[Ian Jackson]

Anthony PERARD writes ("Re: [OSSTEST PATCH v11 11/20] ts-openstack-deploy: Increase fd and memory limits for rabbitmq"):
> On Fri, Jun 23, 2017 at 04:34:41PM +0100, Ian Jackson wrote:
> > And if one isn't using systemd ?
> 
> I guess it would have to fix devstack. Also, the default is 1k, devstack
> increase it to 2k, but rabbitmq on Ubuntu have a limit at 65k.

My point is this: unless openstack intend to support only systemd, it
is a bug that devstack is not fixed in this way.  So that bug ought to
be filed and referenced in the workaround in openstack.

I'm not sure I'm qualified to comment on the distinction between
devstack's and ubuntu's fd limits, but shouldn't there be an openstack
bug about that too ?

> > > As for the memory limit, it was necessary with a host of 4G of RAM. But
> > > I did not try again with the default limit and a host with 6G of RAM.
> > > (The default is 0.4, here I've set it to 0.8)
> > 
> > It sounds like the default calculation is not right, then ?
> 
> I think 0.4 means 40% of the RAM. Sorry, I should have said so. Also,
> 0.4 is the same on the CI loop, with a dom0 of 7G or so.

Yes, I understood that 0.4 to mean 40% of RAM.

My point is that a calculation which gives too-small a value on a 4G
host could be improved.  May the calculation should be "40% of RAM,
but at least 3G" or something ?  Again, this might warrant an upstream bug.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests

[Anthony PERARD]

On Fri, Jun 23, 2017 at 04:38:05PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("Re: [OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests"):
> > On Fri, Jun 23, 2017 at 02:47:37PM +0100, Ian Jackson wrote:
> > > Again, does this not mean we're going to suffer a maintenance burden
> > > as tempest grows new inapplicable tests ?
> > 
> > That exactly what is happening with the OpenStack CI loop, from time to
> > time, there are new tests, that can fail, and diffenrent ways to fix
> > this (fix the bug, fix a configuration, or just skip the tests).
> > 
> > Recently, we've actually push the list of tests to skip into nova.git,
> > but I think it is only available in master.
> 
> Aha.  Well, if the stable branch is stable then the set of tests to
> skip there is probably stable too ?  And on master it's built-in ?  So
> this actually won't be a problem - in the sense that this will be
> approximately the last necessary update to the list of tests to skip ?

The built-in list is to be consume by the CI, it is not used
automatically. It a bash script to be sourced. I have not yet try to
have osstest use it. That will be for later.

> > > Which makes me think: maybe the tempest tests want to be substeps
> > > anyway.  Is that possible ?  (Does tempest speak subunit or
> > > something ?)
> > 
> > I think it is subunit, yes. And I've got the command that is been run by
> > tempest (as it's printed on stdout).
> > 
> > running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
> > OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
> > OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
> > OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
> > ${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover} --list
> > 
> > then several:
> > running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
> > OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
> > OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} \
> > OS_TEST_LOCK_PATH=${OS_TEST_LOCK_PATH:-${TMPDIR:-'/tmp'}} \
> > ${PYTHON:-python} -m subunit.run discover -t ${OS_TOP_LEVEL:-./} ${OS_TEST_PATH:-./tempest/test_discover}  --load-list /tmp/tmpzSNsrB
> 
> It would be really good to have those individual subunit results as
> substeps.  There is no subunit parser in osstest yet but we could have
> one.
> 
> What version of subunit does it print out ?  The subunit v1 protocol
> is lovely and simple but there is a Second System :-/.

Right now, tempest print on stdout something much simpler, lines liked
this:
{1} tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario [71.009969s] ... ok
Followed by summary of all test, and the output of failed tests.

But it can be told to print subunit v2, which can be pipped to a different
program to transform into want we want, including simple cvs, subunit v1, and
other.

I thing we could try to parse subunit v1, grab the start and finish time
of a test, the result, and the output of the tests. And have all that as
subunit. During my manual tests, `tempest run --subunit ... | subunit-2to1`
only print the stuff about a tests once it is finish, so I guess we'll
have to rewrite substep_start() (or another function) to take time as
parameter.

> I guess that tempest doesn't stop on the first failed test ?  So
> perhaps we could just tolerate the failed-but-not-skipped tests ?

It does not, it run everything, no matter how many test fails.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests

[Ian Jackson]

Anthony PERARD writes ("Re: [OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests"):
> On Fri, Jun 23, 2017 at 04:38:05PM +0100, Ian Jackson wrote:
> > Aha.  Well, if the stable branch is stable then the set of tests to
> > skip there is probably stable too ?  And on master it's built-in ?  So
> > this actually won't be a problem - in the sense that this will be
> > approximately the last necessary update to the list of tests to skip ?
> 
> The built-in list is to be consume by the CI, it is not used
> automatically. It a bash script to be sourced. I have not yet try to
> have osstest use it. That will be for later.

Hah.

> Right now, tempest print on stdout something much simpler, lines liked
> this:
> {1} tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario [71.009969s] ... ok
> Followed by summary of all test, and the output of failed tests.
> 
> But it can be told to print subunit v2, which can be pipped to a different
> program to transform into want we want, including simple cvs, subunit v1, and
> other.
> 
> I thing we could try to parse subunit v1, grab the start and finish time
> of a test, the result, and the output of the tests. And have all that as
> subunit. During my manual tests, `tempest run --subunit ... | subunit-2to1`
> only print the stuff about a tests once it is finish, so I guess we'll
> have to rewrite substep_start() (or another function) to take time as
> parameter.

That sounds quite good really.

I guess you don't want to write a parser for subunit v2 but if there
is one in Perl already in some package in Debian we could use it.  Or
subunit-2to1 is fine.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 20/20] Introduce flight for stable branches of OpenStack

[Anthony PERARD]

On Fri, Jun 23, 2017 at 06:00:29PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("[OSSTEST PATCH v11 20/20] Introduce flight for stable branches of OpenStack"):
> > OpenStack have many different repo which should be in sync, so this
> > patch should grab the revisions of the stable branch of every OpenStack
> > tree. Tempest does not have stable branch and should be able to test any
> > OpenStack version.
> ...
> > +openstack-*-*)
> > +	os_tree="${branch#openstack-}"
> > +	os_tree="${os_tree%-*}"
> > +	branchcore="${branch##*-}"
> > +	eval repo_tree_rev_fetch_git "openstack-$os_tree" \
> > +		"\$TREE_OPENSTACK_${os_tree^^}" "stable/$branchcore" \
> > +                "\$LOCALREV_OPENSTACK_${os_tree^^}"
> 
> From your previous email:
> 
>   I think this patch is confusing because I originally try to use osstest
>   scripts to find which commit to use for every trees and so have add the
>   necessary into ./ap-fetch-version. But I could not make that works
>   without duplicating some functions and so went with writing
>   'origin/stable/ocata' into REVISION_*.
> 
> I think I am indeed still confused by some of it.
> 
> For example:
> 
> > +openstack_rev() {
> > +        local os_tree="$1"
> > +        local os_branch
> > +
> > +        if eval [ "x\$REVISION_OPENSTACK_${os_tree^^}" = x ]; then
> > +                case "$branch" in
> > +                openstack-*-*)
> > +                        os_branch="openstack-$os_tree-${branch##*-}"
> > +                        os_git_branch="origin/stable/${branch##*-}"
> > +                        ;;
> > +                *)
> > +                        os_branch="openstack-$os_tree"
> > +                        os_git_branch="origin/master"
> > +                        ;;
> > +                esac
> > +
> > +                # Use latest version, even for other openstack
> > +                # trees so branch openstack-nova-ocata should have
> > +                # other trees like openstack-neutron have the
> > +                # revision of the same branch fetch at the same
> > +                # time
> > +                if [ "$branch" != "$os_branch" ]; then
> > +                        eval "export REVISION_OPENSTACK_${os_tree^^}=$os_git_branch"
> > +                        return
> > +                fi
> > +                determine_version "REVISION_OPENSTACK_${os_tree^^}" \
> > +                        "$os_branch" "OPENSTACK_${os_tree^^}"
> > +                eval "export REVISION_OPENSTACK_${os_tree^^}"
> > +        fi
> > +}
> > +for os_tree in cinder devstack glance keystone neutron nova requirements; do
> > +        openstack_rev "$os_tree"
> > +done
> 
> I wonder if this full generality is really necessary ?  If you don't
> intend branches like   openstack-ocata-neutron   then it would be
> sufficient to call one function for nova and another for the other
> trees.

I'll see what I can do.

> And, frankly, I don't think we could have branches like
> `openstack-ocata-neutron'.  That would be too many branches.

Yes, it was to keep a door open.

> So perhaps the branch `openstack-ocata-nova' should be called
> `openstack-ocata' ?

Yes, I think that would be enough. You mean "openstack-$version" right?
(Or with other words "openstack-$release_name".)

> Also, right now I think it's clear that we're not intending to add
> openstack jobs to existing branches' flights.  But if we were to do
> that in the future, we would want all the subtrees to be tracked.
> 
> Maybe we should have a way for cr-daily-branch to fetch, and push,
> multiple trees.  We could call ap-fetch-version on every tree,
> and set the appropriate variable (with determine_version, as you have
> above).  And then set a variable to call ap-push multiple times, if we
> get a pass.

Yes, I first do the work to have the branches been properly setup to
test openstack of a release, then later I can look into pushing multiple
branches.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests

[Anthony PERARD]

On Fri, Jun 30, 2017 at 05:32:56PM +0100, Ian Jackson wrote:
> Anthony PERARD writes ("Re: [OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests"):
> > But it can be told to print subunit v2, which can be pipped to a different
> > program to transform into want we want, including simple cvs, subunit v1, and
> > other.
> > 
> > I thing we could try to parse subunit v1, grab the start and finish time
> > of a test, the result, and the output of the tests. And have all that as
> > subunit. During my manual tests, `tempest run --subunit ... | subunit-2to1`
> > only print the stuff about a tests once it is finish, so I guess we'll
> > have to rewrite substep_start() (or another function) to take time as
> > parameter.
> 
> That sounds quite good really.
> 
> I guess you don't want to write a parser for subunit v2 but if there
> is one in Perl already in some package in Debian we could use it.  Or
> subunit-2to1 is fine.

There is one, libsubunit-perl, but its only for subunit v1 :-(. So I
guess I'll use both, the package and subunit-2to1.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 20/20] Introduce flight for stable branches of OpenStack

[Ian Jackson]

Anthony PERARD writes ("Re: [OSSTEST PATCH v11 20/20] Introduce flight for stable branches of OpenStack"):
> On Fri, Jun 23, 2017 at 06:00:29PM +0100, Ian Jackson wrote:
> > Anthony PERARD writes ("[OSSTEST PATCH v11 20/20] Introduce flight for stable branches of OpenStack"):
> > So perhaps the branch `openstack-ocata-nova' should be called
> > `openstack-ocata' ?
> 
> Yes, I think that would be enough. You mean "openstack-$version" right?
> (Or with other words "openstack-$release_name".)

Yes.

> > Also, right now I think it's clear that we're not intending to add
> > openstack jobs to existing branches' flights.  But if we were to do
> > that in the future, we would want all the subtrees to be tracked.
> > 
> > Maybe we should have a way for cr-daily-branch to fetch, and push,
> > multiple trees.  We could call ap-fetch-version on every tree,
> > and set the appropriate variable (with determine_version, as you have
> > above).  And then set a variable to call ap-push multiple times, if we
> > get a pass.
> 
> Yes, I first do the work to have the branches been properly setup to
> test openstack of a release, then later I can look into pushing multiple
> branches.

OK.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests

[Anthony PERARD]

On Fri, Jun 30, 2017 at 06:58:43PM +0100, Anthony PERARD wrote:
> On Fri, Jun 30, 2017 at 05:32:56PM +0100, Ian Jackson wrote:
> > Anthony PERARD writes ("Re: [OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests"):
> > > But it can be told to print subunit v2, which can be pipped to a different
> > > program to transform into want we want, including simple cvs, subunit v1, and
> > > other.
> > > 
> > > I thing we could try to parse subunit v1, grab the start and finish time
> > > of a test, the result, and the output of the tests. And have all that as
> > > subunit. During my manual tests, `tempest run --subunit ... | subunit-2to1`
> > > only print the stuff about a tests once it is finish, so I guess we'll
> > > have to rewrite substep_start() (or another function) to take time as
> > > parameter.
> > 
> > That sounds quite good really.
> > 
> > I guess you don't want to write a parser for subunit v2 but if there
> > is one in Perl already in some package in Debian we could use it.  Or
> > subunit-2to1 is fine.
> 
> There is one, libsubunit-perl, but its only for subunit v1 :-(. So I
> guess I'll use both, the package and subunit-2to1.

:( the package does not work with the output of `Tempest | subunit-2to1`
there are a few things that are not parsed properly.

That the parser:
https://github.com/testing-cabal/subunit/blob/master/perl/lib/Subunit.pm

I'm kind of tempted to write one that's will be easier to integrate with
Osstest, especially because subunit v1 is simple enough.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel