[OSSTEST RFC 00/16] Testing CentOS as dom0 with osstest

[OSSTEST RFC 00/16] Testing CentOS as dom0 with osstest

[Anthony PERARD]

Hi,

This patch series teach osstest how to install CentOS 7 on a host and how to
install Xen via packages that have been built in the CentOS Community Build
Service (or CBS).

For now, this is only usefull in standalone or to start ad-hoc flight, as there
is nothing to build Xen.

It is based on Roger FreeBSD patch series, with the new all_host_os runvar.

The series is missing a lot of configuration options, it can only test the
Xen 4.8 package on CentOS 7.

A branch can be found:
https://xenbits.xen.org/git-http/people/aperard/osstest.git
tag: centos-v1

In standalone mode, the different jobs can be created with:
OSSTEST_FLIGHT=standalone ./make-centos-flight centos xen-4.8-testing play

In an osstest instance, create a flight simply with:
./make-centos-flight centos xen-4.8-testing play

Anthony PERARD (16):
  JobDB-Standalone.tcl: Fix read-runvar
  Osstest/TestSupport: In teditfileex, get the file with the requested
    user
  Osstest/TestSupport: Add centos to package_install_cmd
  TestSupport: In host_install_postboot, don't call update-rc.d on
    CentOS
  TestSupport: Adapt target_https_mitm_proxy_setup to CentOS
  Osstest/CentOS.pm: Introduce CentOS support, starting with
    kickstart_installcmdline_core
  Osstest/CentOS: kickstart_create to generate an autoinstall recipe
  ts-centos-host-install: Install CentOS on a host
  CentOS: Setup osstest-confirm-booted.service
  ts-centos-xen-pkg-install: Install of Xen package on CentOS
  ts-centos-xen-pkg-install: Adjust daemons configuration
  ts-centos-xen-pkg-install: Create bridge config
  WORKAROUND: Osstest/TestSupport: Make target_reboot works with systemd
  sg-run-job: Select host install script based on all_host_os runvar
  make-centos-flight: Create a flight with CentOS as dom0
  Osstest/TestSupport: Handle qemu-img location on CentOS

 Osstest/CentOS.pm                                  | 273 +++++++++++
 Osstest/TestSupport.pm                             |  27 +-
 make-centos-flight                                 | 524 +++++++++++++++++++++
 .../systemd/system/osstest-confirm-booted.service  |  28 ++
 overlay-centos/etc/systemd/system/osstest.target   |  19 +
 sg-run-job                                         |  16 +-
 tcl/JobDB-Standalone.tcl                           |   2 +-
 ts-centos-host-install                             | 154 ++++++
 ts-centos-xen-pkg-install                          | 111 +++++
 9 files changed, 1144 insertions(+), 10 deletions(-)
 create mode 100644 Osstest/CentOS.pm
 create mode 100755 make-centos-flight
 create mode 100644 overlay-centos/etc/systemd/system/osstest-confirm-booted.service
 create mode 100644 overlay-centos/etc/systemd/system/osstest.target
 create mode 100755 ts-centos-host-install
 create mode 100755 ts-centos-xen-pkg-install

-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 01/16] JobDB-Standalone.tcl: Fix read-runvar

[Anthony PERARD]

This fix the error bellow, when the runvar exist:
can't read "runvarinfo()": no such element in array
    while executing
"set val $runvarinfo($val)"
    invoked from within
"osstestdb eval {
        SELECT val FROM runvars
         WHERE flight = $flight
           AND job = $job
           AND name = $name
    } runvarinfo {
        se..."
    (procedure "jobdb::read-runvar" line 3)
    invoked from within
"jobdb::read-runvar $flight $job all_host_os"
    (procedure "run-job" line 13)
    invoked from within
"run-job $job"
    (file "./sg-run-job" line 767)

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 tcl/JobDB-Standalone.tcl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tcl/JobDB-Standalone.tcl b/tcl/JobDB-Standalone.tcl
index 09eac6e..306e855 100644
--- a/tcl/JobDB-Standalone.tcl
+++ b/tcl/JobDB-Standalone.tcl
@@ -52,7 +52,7 @@ proc read-runvar {flight job name {val {}}} {
            AND job = $job
            AND name = $name
     } runvarinfo {
-	set val $runvarinfo($val)
+	set val $runvarinfo(val)
     }
     return $val
 }
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 02/16] Osstest/TestSupport: In teditfileex, get the file with the requested user

[Anthony PERARD]

e.g. If the file to be edited is only accessible to the root user,
target_getfile() would not work.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/TestSupport.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index c9dada3..65324fb 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -638,7 +638,7 @@ sub teditfileex {
         logm("editing $rfile to $rdest as $lfile".'{,.new}');
     }
 
-    target_getfile($ho, 60, $rfile, $lfile);
+    tgetfileex($user, $ho, 60, $rfile, $lfile);
     open '::EI', "$lfile" or die "$lfile: $!";
     open '::EO', "> $lfile.new" or die "$lfile.new: $!";
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 03/16] Osstest/TestSupport: Add centos to package_install_cmd

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/TestSupport.pm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 65324fb..13e4360 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -530,6 +530,8 @@ sub package_install_cmd ($;$) {
 
     if ($ho->{OS} eq "freebsd") {
         push @cmd, qw(lockf /var/run/osstest-pkg-lock pkg-static install);
+    } elsif ($ho->{OS} eq "centos") {
+        push @cmd, qw(yum -y install);
     } else {
         push @cmd, qw(DEBIAN_PRIORITY=critical UCF_FORCE_CONFFOLD=y
                       with-lock-ex -w /var/lock/osstest-apt apt-get);
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 04/16] TestSupport: In host_install_postboot, don't call update-rc.d on CentOS

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/TestSupport.pm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 13e4360..b0e21bf 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -2818,7 +2818,8 @@ sub guest_editconfig_nocd ($$) {
 sub host_install_postboot_complete ($) {
     my ($ho) = @_;
     target_core_dump_setup($ho);
-    target_cmd_root($ho, "update-rc.d osstest-confirm-booted start 99 2 .");
+    target_cmd_root($ho, "update-rc.d osstest-confirm-booted start 99 2 .")
+      unless $ho->{OS} eq "centos";
     target_https_mitm_proxy_setup($ho);
 }
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 05/16] TestSupport: Adapt target_https_mitm_proxy_setup to CentOS

[Anthony PERARD]

The location for new certificates is different, and
update-ca-certificates is Debian specific.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/TestSupport.pm | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index b0e21bf..2b56c26 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -2845,9 +2845,18 @@ sub target_https_mitm_proxy_setup ($) {
     my ($ho) = @_;
     my $cert = $c{HttpsProxyMITMCert};
     return unless length $cert;
+    my $dest;
+    my $update_ca_cmd;
+    if ($ho->{OS} eq "centos" ) {
+        $dest = '/etc/pki/ca-trust/source/anchors';
+        $update_ca_cmd = 'update-ca-trust extract';
+    } else {
+        $dest = '/usr/local/share/ca-certificates';
+        $update_ca_cmd = 'update-ca-certificates';
+    }
     target_putfilecontents_root_stash($ho,30,$cert,
-                  '/usr/local/share/ca-certificates/osstest.crt');
-    target_cmd_root($ho, 'update-ca-certificates', 300);
+        $dest.'/osstest.crt');
+    target_cmd_root($ho, $update_ca_cmd, 300);
 }
 
 sub sha256file ($;$) {
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 06/16] Osstest/CentOS.pm: Introduce CentOS support, starting with kickstart_installcmdline_core

[Anthony PERARD]

This function will be use later to install CentOS on a host.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/CentOS.pm | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)
 create mode 100644 Osstest/CentOS.pm

diff --git a/Osstest/CentOS.pm b/Osstest/CentOS.pm
new file mode 100644
index 0000000..33479b1
--- /dev/null
+++ b/Osstest/CentOS.pm
@@ -0,0 +1,51 @@
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2017 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+package Osstest::CentOS;
+
+use strict;
+use warnings;
+
+use Osstest;
+use Osstest::TestSupport;
+
+BEGIN {
+    use Exporter ();
+    our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS);
+    $VERSION     = 1.00;
+    @ISA         = qw(Exporter);
+    @EXPORT      = qw(
+                      kickstart_installcmdline_core
+                      );
+    %EXPORT_TAGS = ( );
+
+    @EXPORT_OK   = qw();
+}
+
+sub kickstart_installcmdline_core ($$;@) {
+    my ($tho, $ks_url, %xopts) = @_;
+
+    my @cl= qw(debug inst.text);
+    push @cl, "inst.ks=$ks_url";
+
+    # FIXME: CentOS installer forward syslog only via TCP, osstest listen on UPD.
+    push @cl, "inst.syslog=$r{syslog_server}"
+        if ($r{syslog_server});
+
+    return @cl;
+}
+
+1;
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 07/16] Osstest/CentOS: kickstart_create to generate an autoinstall recipe

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/CentOS.pm | 215 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 215 insertions(+)

diff --git a/Osstest/CentOS.pm b/Osstest/CentOS.pm
index 33479b1..dbba354 100644
--- a/Osstest/CentOS.pm
+++ b/Osstest/CentOS.pm
@@ -28,13 +28,22 @@ BEGIN {
     $VERSION     = 1.00;
     @ISA         = qw(Exporter);
     @EXPORT      = qw(
+                      kickstart_create
+                      kickstart_hook_command
                       kickstart_installcmdline_core
+                      ks_vg_name
                       );
     %EXPORT_TAGS = ( );
 
     @EXPORT_OK   = qw();
 }
 
+# -Y off disables any proxy, since there is no point going through the
+# proxy when fetching stuff from the local controller or the local cache.
+our $kickstart_wget = 'wget -Y off';
+
+our %kickstart_cmds;
+
 sub kickstart_installcmdline_core ($$;@) {
     my ($tho, $ks_url, %xopts) = @_;
 
@@ -48,4 +57,210 @@ sub kickstart_installcmdline_core ($$;@) {
     return @cl;
 }
 
+sub ks_vg_name($) {
+    my ($ho) = @_;
+
+    return "$ho->{Name}-vg";
+}
+
+sub kickstart_ssh ($$) {
+    my ($ho,$sfx) = @_;
+
+    my $authkeys_url= create_webfile($ho, "authkeys$sfx", authorized_keys());
+    my $knownhosts_url= create_webfile($ho, "known_hosts$sfx", known_hosts());
+
+    kickstart_hook_command($ho, 'post', $sfx, <<END);
+#!/bin/sh
+set -ex
+
+r=/root
+cd \$r
+
+umask 022
+mkdir -p .ssh
+$kickstart_wget -O .ssh/authorized_keys '$authkeys_url'
+$kickstart_wget -O .ssh/known_hosts     '$knownhosts_url'
+
+u=osstest
+h=/home/\$u
+mkdir -p \$h/.ssh
+cp .ssh/authorized_keys \$h/.ssh
+chown -R \$u.\$u \$h/.ssh
+END
+}
+
+sub kickstart_hook_overlay ($$$$) {
+    my ($ho, $sfx, $srcdir, $tfilename) = @_;
+    my $url= create_webfile($ho, "$tfilename$sfx", sub {
+        my ($fh) = @_;
+        contents_make_cpio($fh, 'ustar', $srcdir);
+    });
+    kickstart_hook_command($ho, 'post', $sfx, <<END);
+#!/bin/sh
+set -ex
+
+r=/root
+cd \$r
+
+umask 022
+
+$kickstart_wget -O overlay.tar '$url'
+cd /
+tar xf \$r/overlay.tar
+cd \$r
+rm overlay.tar
+
+END
+}
+
+sub kickstart_base ($$;@) {
+    my ($ho,$sfx,%xopts) = @_;
+
+    kickstart_ssh($ho, $sfx);
+
+    kickstart_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar');
+
+    my $ntpserver = get_target_property($ho,'NtpServer');
+    if ($ntpserver) {
+        $ntpserver = "--ntpservers=$ntpserver";
+    }
+    my $kickstart_file = <<"END";
+#version=RHEL7
+# System authorization information
+auth --enableshadow --passalgo=sha512
+# License agreement
+eula --agreed
+# Use text mode install
+text
+# Keyboard layouts
+keyboard --vckeymap=us --xlayouts='us'
+# System language
+lang en_GB.UTF-8
+# Do not configure the X Window System
+skipx
+# System timezone
+timezone $c{Timezone} --isUtc $ntpserver
+# Reboot after installation
+reboot
+
+# Root password
+rootpw --plaintext xenroot
+# osstest user
+user --name=osstest --plaintext --password=osstest --gecos="FLOSS Xen Test"
+END
+
+    return $kickstart_file;
+}
+
+sub kickstart_create ($$;@) {
+    my ($ho, $sfx, %xopts) = @_;
+
+    my $disk= $xopts{DiskDevice} || '/dev/sda';
+
+    # Start an update, which pulls packages from the updates repo
+    push @{ $kickstart_cmds{post} }, "yum -y update";
+
+    my $releasever = '7';
+    my $basearch = 'x86_64';
+
+    my $proxy = '';
+    $proxy = '--proxy='. $c{HttpProxy}
+      if $c{HttpProxy};
+
+    # CentOS mirror found on this list:
+    # http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
+    my $mirror = 'http://centos.mirroring.pulsant.co.uk/7.4.1708/os/x86_64/';
+
+    our $vgname = ks_vg_name($ho);
+
+    my $kickstart_file = kickstart_base($ho, $sfx, %xopts);
+    $kickstart_file .= <<"END";
+# Use network installation
+url --url=$mirror $proxy
+
+# Network information
+network --bootproto=dhcp --device=bootif --hostname=$xopts{Hostname}.$c{TestHostDomain}
+
+# Clear the Master Boot Record
+zerombr
+# Partition clearing information
+clearpart --all --drives=$disk
+# Disk partitioning information
+part /boot --asprimary --size=$c{HostDiskBoot} --fstype=ext3 --ondrive=$disk
+part pv.01 --grow --ondrive=$disk
+volgroup $vgname pv.01
+logvol /    --name=root --size=$c{HostDiskRoot} --vgname=$vgname --fstype=ext4
+logvol swap --name=swap --size=$c{HostDiskSwap} --vgname=$vgname
+
+# System bootloader configuration
+bootloader --location=mbr --boot-drive=$disk
+END
+
+    kickstart_hook_command($ho, 'post', $sfx, <<END);
+#!/bin/sh
+set -ex
+cmdline=\$(cat /proc/cmdline)
+bootif=\${cmdline#*BOOTIF=}
+[ "\$cmdline" != "\$bootif" ]
+bootif=\${bootif% *}
+bootif=\${bootif#01-}
+bootif=\${bootif//-/:}
+
+# Remove generated network profiles
+rm -v /etc/sysconfig/network-scripts/ifcfg-*
+
+# Replace them with one based on MAC rather than interface name
+tee /etc/sysconfig/network-scripts/ifcfg-osstest-if0 <<ENDCFG
+TYPE=Ethernet
+HWADDR=\$bootif
+ONBOOT=yes
+BOOTPROTO=dhcp
+ENDCFG
+END
+
+    # pre & post sections
+    $kickstart_file .= kickstart_hook_cmds();
+
+    # packages section
+    $kickstart_file .= <<"END";
+%packages
+\@core
+wget
+ed
+%end
+END
+
+    # Disable kdump
+    $kickstart_file .= <<END;
+%addon com_redhat_kdump --disable
+%end
+END
+
+    return create_webfile($ho, "kickstart$sfx", $kickstart_file);
+}
+
+
+sub kickstart_hook_command ($$$$) {
+    my ($ho, $ks_section, $sfx, $text) = @_;
+    my $ix= $#{ $kickstart_cmds{$ks_section} } + 1;
+    my $url= create_webfile($ho, "$ks_section-$ix$sfx", $text);
+    my $file= "/tmp/$ks_section-$ix";
+    my $cmd_cmd= "$kickstart_wget -O $file '$url' && chmod +x $file && $file";
+    push @{ $kickstart_cmds{$ks_section} }, $cmd_cmd;
+}
+
+sub kickstart_hook_cmds () {
+    my $kickstart;
+    foreach my $ks_section (keys %kickstart_cmds) {
+        my $cmds = join("\n", @{ $kickstart_cmds{$ks_section} });
+        $kickstart .= <<"END";
+%$ks_section --log=/root/ks-$ks_section.log
+set -x
+$cmds
+%end
+END
+    }
+    return $kickstart;
+}
+
 1;
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 08/16] ts-centos-host-install: Install CentOS on a host

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-centos-host-install | 154 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 154 insertions(+)
 create mode 100755 ts-centos-host-install

diff --git a/ts-centos-host-install b/ts-centos-host-install
new file mode 100755
index 0000000..943002d
--- /dev/null
+++ b/ts-centos-host-install
@@ -0,0 +1,154 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+use POSIX;
+
+BEGIN { unshift @INC, qw(.); }
+use Osstest;
+use Osstest::CentOS;
+use Osstest::TestSupport;
+use Osstest::Logtailer;
+
+tsreadconfig();
+
+my $poweron_test_only;
+
+our %xopts;
+
+while (@ARGV and $ARGV[0] =~ m/^-/) {
+    $_= shift @ARGV;
+    last if m/^--$/;
+    if  (m/^--poweron-test-only$/) {
+	$poweron_test_only= 1;
+    } else {
+	die "$_ $!";
+    }
+}
+
+our ($whhost) = @ARGV;
+$whhost ||= 'host';
+our $ho= selecthost($whhost);
+exit 0 if $ho->{Flags}{'no-reinstall'};
+exit 0 if $ho->{SharedReady};
+
+our %timeout= qw(ReadPreseed  350
+                 Sshd        2400);
+
+sub install () {
+    power_state($ho, 0);
+
+    my ($ks_url,$ps_file) = kickstart_create
+        ($ho, '',
+         DiskDevice => $ho->{DiskDevice},
+         Properties => $ho->{Properties},
+         Hostname   => $ho->{Name},
+         IpAddress  => $ho->{Ip}
+         );
+
+    setup_netboot_firstboot($ks_url);
+    power_cycle_sleep($ho);
+
+    my $logtailer = Osstest::Logtailer->new($c{WebspaceLog});
+    power_state($ho, 1);
+
+    await_webspace_fetch_byleaf(get_timeout($ho,'reboot',$timeout{ReadPreseed})
+                                , 1,
+                                $logtailer, $ho, $ks_url);
+
+    if ($poweron_test_only) {
+	logm("Hooray, power on worked.");
+	power_state($ho, 0);
+	exit 0;
+    }
+
+    setup_netboot_local($ho);
+
+    await_tcp(get_timeout($ho, 'reboot', $timeout{Sshd}), 14, $ho);
+
+    our $vgname = ks_vg_name($ho);
+
+    my $partitions= target_cmd_output_root($ho, 'cat /proc/partitions');
+    foreach my $l (split /\n/, $partitions) {
+        logm("| $l");
+        next unless $l =~ m/^\s*\d+\s+\d+\s+\d+\s+((?:sd|hd)[b-z])\s*$/;
+        my $dev= "/dev/$1";
+        target_cmd_root($ho, "pvcreate $dev");
+        target_cmd_root($ho, "vgextend $vgname $dev");
+    }
+
+    my $kpath= $c{TestHostKeypairPath};
+    my $kleaf= $kpath;
+    $kleaf =~ s,.*/,,;
+    my $ktarg= $kleaf;
+    $ktarg =~ s/^(id_[a-z]+)_.*/$1/;
+    foreach my $ext ('', '.pub') {
+	target_putfile     ($ho,10, "${kpath}${ext}", ".ssh/${ktarg}${ext}");
+	target_putfile_root($ho,10, "${kpath}${ext}", ".ssh/${ktarg}${ext}");
+    }
+    target_cmd     ($ho, "chmod g-r .ssh/${ktarg}");
+    target_cmd_root($ho, "chmod g-r .ssh/${ktarg}");
+
+    target_cmd_root($ho, "chmod 2755 /root");
+
+    if ($c{HttpProxy}) {
+        target_editfile_root($ho, '/etc/yum.conf',
+            sub { target_editfile_kvp_replace("proxy", $c{HttpProxy}) });
+    }
+
+    host_install_postboot_complete($ho);
+
+    logm('OK: install completed');
+}
+
+sub setup_netboot_firstboot ($) {
+    my ($ks_url) = @_;
+
+    my $TftpCiBase = "$c{TftpPlayDir}centos";
+    my $CiVersion = 'current';
+    my $Suite = 'centos7';
+    my $c_i = "$TftpCiBase/$r{arch}/$CiVersion-$Suite";
+
+    my @dicmdline= qw(vga=normal);
+    push @dicmdline, kickstart_installcmdline_core($ho, $ks_url, %xopts);
+
+    my $kernel= "$c_i/vmlinuz";
+    my $initrd= "$c_i/initrd.img";
+
+    my $ipappend = 2;
+
+    push @dicmdline,
+        get_host_property($ho, "install-append $ho->{Suite}", ''),
+        get_host_property($ho, "install-append $ho->{Suite} $r{arch}", '');
+
+    my $console = get_host_native_linux_console($ho);
+
+    push @dicmdline, "console=$console" unless $console eq "NONE";
+
+    my @hocmdline;
+
+    push @hocmdline,
+        get_host_property($ho, "linux-boot-append $ho->{Suite}", ''),
+        get_host_property($ho, "linux-boot-append $ho->{Suite} $r{arch}", '');
+
+    $xopts{ipappend} = $ipappend;
+    setup_netboot_di($ho, $kernel, "/$initrd", \@dicmdline, \@hocmdline,
+		     %xopts);
+}
+
+install();
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 09/16] CentOS: Setup osstest-confirm-booted.service

[Anthony PERARD]

This emulate the osstest-confirm-booted service that is done on Debian.

In order to have the service been started last:
- it is made dependent on multi-user.target which is the default target on systemd.
- it is part of osstest.target which depend on multi-user.target.
- the osstest.target is set as the new default.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/CentOS.pm                                  |  7 ++++++
 .../systemd/system/osstest-confirm-booted.service  | 28 ++++++++++++++++++++++
 overlay-centos/etc/systemd/system/osstest.target   | 19 +++++++++++++++
 3 files changed, 54 insertions(+)
 create mode 100644 overlay-centos/etc/systemd/system/osstest-confirm-booted.service
 create mode 100644 overlay-centos/etc/systemd/system/osstest.target

diff --git a/Osstest/CentOS.pm b/Osstest/CentOS.pm
index dbba354..28c8193 100644
--- a/Osstest/CentOS.pm
+++ b/Osstest/CentOS.pm
@@ -119,6 +119,13 @@ sub kickstart_base ($$;@) {
     kickstart_ssh($ho, $sfx);
 
     kickstart_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar');
+    kickstart_hook_overlay($ho, $sfx, 'overlay-centos', 'overlay.tar');
+    kickstart_hook_command($ho, 'post', $sfx, <<END);
+#!/bin/sh
+set -ex
+systemctl enable osstest-confirm-booted.service
+systemctl set-default osstest.target
+END
 
     my $ntpserver = get_target_property($ho,'NtpServer');
     if ($ntpserver) {
diff --git a/overlay-centos/etc/systemd/system/osstest-confirm-booted.service b/overlay-centos/etc/systemd/system/osstest-confirm-booted.service
new file mode 100644
index 0000000..eae79ed
--- /dev/null
+++ b/overlay-centos/etc/systemd/system/osstest-confirm-booted.service
@@ -0,0 +1,28 @@
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2017 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+[Unit]
+Description=Confirm fully booted
+After=multi-user.target
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+ExecStart=/usr/bin/touch /dev/shm/osstest-confirm-booted
+ExecStop=/usr/bin/rm -f /dev/shm/osstest-confirm-booted
+
+[Install]
+WantedBy=osstest.target
diff --git a/overlay-centos/etc/systemd/system/osstest.target b/overlay-centos/etc/systemd/system/osstest.target
new file mode 100644
index 0000000..7b98fc0
--- /dev/null
+++ b/overlay-centos/etc/systemd/system/osstest.target
@@ -0,0 +1,19 @@
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2017 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+[Unit]
+Requires=multi-user.target
+After=multi-user.target
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 10/16] ts-centos-xen-pkg-install: Install of Xen package on CentOS

[Anthony PERARD]

Install candidate packages that have been built by CBS, the CentOS
Community Build Service.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-centos-xen-pkg-install | 79 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)
 create mode 100755 ts-centos-xen-pkg-install

diff --git a/ts-centos-xen-pkg-install b/ts-centos-xen-pkg-install
new file mode 100755
index 0000000..e10456d
--- /dev/null
+++ b/ts-centos-xen-pkg-install
@@ -0,0 +1,79 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2017 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+BEGIN { unshift @INC, qw(.); }
+use Osstest;
+use File::Path;
+use POSIX;
+use Osstest::TestSupport;
+
+
+tsreadconfig();
+
+our @hos;
+
+if (!@ARGV) {
+    push @ARGV, 'host';
+}
+foreach my $k (@ARGV) {
+    push @hos, selecthost($k);
+}
+
+our $ho;
+
+sub packages_setup () {
+    my $subtag = $r{centos_cbs_repo} // 'candidate';
+    die unless $subtag =~ m/^(candidate|testing|release)$/;
+    my $repos = <<END;
+[virt-xen-48-$subtag]
+name=VirtSIG-\$releasever - Xen 4.8 CBS $subtag
+baseurl=http://cbs.centos.org/repos/virt\$releasever-xen-48-$subtag/\$basearch/os/
+gpgcheck=0
+[virt-xen-common-$subtag]
+name=VirtSIG-\$releasever - Xen common CBS
+baseurl=http://cbs.centos.org/repos/virt\$releasever-xen-common-$subtag/\$basearch/os/
+gpgcheck=0
+END
+    target_putfilecontents_root_stash($ho, 10, $repos,
+        "/etc/yum.repos.d/Osstest.repo");
+
+    # This packages in necessary, in order to get the right kernel
+    # It pull /usr/bin/grub-bootxen.sh (helper to boot Xen by default) and
+    # the CentOS Virt7 repos.
+    target_install_packages($ho, qw(centos-release-xen));
+
+    target_cmd_root($ho, 'yum -y update', 10*60);
+
+    my @xen_packages = qw(xen);
+    # For vncpasswd
+    push @xen_packages, 'tigervnc-server-minimal';
+    push @xen_packages, qw(libvirt-daemon-xen libvirt-client)
+	if toolstack($ho)->{Name} eq "libvirt";
+    target_install_packages($ho, @xen_packages);
+}
+
+
+sub setupboot () {
+    logm("ready to boot Xen");
+}
+
+die if @hos > 1;
+$ho= $hos[0];
+packages_setup();
+setupboot();
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 11/16] ts-centos-xen-pkg-install: Adjust daemons configuration

[Anthony PERARD]

Ajust configuration of xenconsoled and libvirtd.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-centos-xen-pkg-install | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/ts-centos-xen-pkg-install b/ts-centos-xen-pkg-install
index e10456d..0327b72 100755
--- a/ts-centos-xen-pkg-install
+++ b/ts-centos-xen-pkg-install
@@ -68,6 +68,15 @@ END
     target_install_packages($ho, @xen_packages);
 }
 
+sub adjustconfig () {
+    target_editfile_root($ho, '/etc/sysconfig/xencommons',
+	sub { target_editfile_kvp_replace("XENCONSOLED_TRACE", "guest") });
+
+    target_editfile_root($ho, '/etc/libvirt/libvirtd.conf',
+		sub { target_editfile_kvp_replace("log_level", "1") })
+	if toolstack($ho)->{Name} eq "libvirt";
+}
+
 
 sub setupboot () {
     logm("ready to boot Xen");
@@ -76,4 +85,5 @@ sub setupboot () {
 die if @hos > 1;
 $ho= $hos[0];
 packages_setup();
+adjustconfig();
 setupboot();
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 12/16] ts-centos-xen-pkg-install: Create bridge config

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-centos-xen-pkg-install | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/ts-centos-xen-pkg-install b/ts-centos-xen-pkg-install
index 0327b72..9a4dcdd 100755
--- a/ts-centos-xen-pkg-install
+++ b/ts-centos-xen-pkg-install
@@ -82,8 +82,30 @@ sub setupboot () {
     logm("ready to boot Xen");
 }
 
+sub setupbridge () {
+    my $bridge_xenbr0 = <<END;
+DEVICE=xenbr0
+STP=no
+BRIDGING_OPTS="hello_time=0"
+TYPE=Bridge
+BOOTPROTO=dhcp
+ONBOOT=yes
+END
+    target_putfilecontents_root_stash($ho, 10, $bridge_xenbr0,
+        "/etc/sysconfig/network-scripts/ifcfg-xenbr0");
+
+    # Should be setup by install
+    target_editfile_root($ho, "/etc/sysconfig/network-scripts/ifcfg-osstest-if0",
+	sub { target_editfile_kvp_replace("BRIDGE", "xenbr0") });
+    target_editfile_root($ho, "/etc/sysconfig/network-scripts/ifcfg-osstest-if0",
+	sub { target_editfile_kvp_replace("BOOTPROTO", "none") });
+
+
+}
+
 die if @hos > 1;
 $ho= $hos[0];
 packages_setup();
 adjustconfig();
 setupboot();
+setupbridge();
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 13/16] WORKAROUND: Osstest/TestSupport: Make target_reboot works with systemd

[Anthony PERARD]

On host running with systemd as init, doing `ssh host reboot` will
result in ssh returning an error.
This patch works around by not waiting for the reboot command to return.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/TestSupport.pm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 2b56c26..40a5c5a 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1318,7 +1318,8 @@ sub host_get_free_memory($) {
 
 sub target_reboot ($) {
     my ($ho) = @_;
-    target_cmd_root($ho, "init 6");
+    # Don't wait for systemd to reap off the socket
+    target_cmd_root($ho, "init 6 & disown");
     target_await_down($ho, $timeout{RebootDown});
     await_tcp(get_timeout($ho,'reboot',$timeout{RebootUp}), 5,$ho);
 }
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 14/16] sg-run-job: Select host install script based on all_host_os runvar

[Anthony PERARD]

This also select a different xen installation script.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 sg-run-job | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/sg-run-job b/sg-run-job
index f6e8340..b324101 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -25,8 +25,9 @@ readconfig
 source-method JobDB
 
 proc per-host-prep {} {
+    global ts_xen_install
     per-host-ts .       host-ping-check-native/@ ts-host-ping-check
-    per-host-ts .       xen-install/@     ts-xen-install
+    per-host-ts .       xen-install/@     $ts_xen_install
     per-host-ts .       xen-boot/@        ts-host-reboot
 
     per-host-ts .       host-ping-check-xen/@ ts-host-ping-check
@@ -42,6 +43,7 @@ proc per-host-finish {} {
 proc run-job {job} {
     global jobinfo builds flight ok truncate need_xen_hosts
     global nested_layers_hosts truncate_globs skip_globs anyskipped
+    global ts_xen_install
 
     set ok 1
     set truncate 0
@@ -84,7 +86,17 @@ proc run-job {job} {
         }
     }
 
-    per-host-ts broken  host-install/@(*) ts-host-install-twice
+    switch -exact [jobdb::read-runvar $flight $job all_host_os] {
+        centos {
+          set ts_host_install ts-centos-host-install
+          set ts_xen_install  ts-centos-xen-pkg-install
+        }
+        default {
+          set ts_host_install ts-host-install-twice
+          set ts_xen_install  ts-xen-install
+        }
+    }
+    per-host-ts broken  host-install/@(*) $ts_host_install
 
     per-host-prep
 
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 15/16] make-centos-flight: Create a flight with CentOS as dom0

[Anthony PERARD]

This is based on make-flight, with the added all_host_os=centos runvar,
and without test that can not be run.

Anything based on the recipe "test-debian" or "test-pair" is remove, as
they require xen-tools. There is no XSM tests as the CentOS packages is
built without. There is no build jobs as the Xen packages been tested
are built in the CentOS CBS.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 make-centos-flight | 524 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 524 insertions(+)
 create mode 100755 make-centos-flight

diff --git a/make-centos-flight b/make-centos-flight
new file mode 100755
index 0000000..a41887f
--- /dev/null
+++ b/make-centos-flight
@@ -0,0 +1,524 @@
+#!/bin/bash
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2017 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+set -e -o posix
+
+branch=$1
+xenbranch=$2
+blessing=$3
+
+flight=`./cs-flight-create $blessing $branch`
+
+. ./cri-common
+. ./cri-getplatforms # getplatforms()
+. ./ap-common
+. ./mfi-common
+
+# Older versions of Xen may not build with the current default.  Note
+# that branches older than 4.3 might need something even older than
+# Wheezy, but we have not done the archaeology to figure out what they
+# require.
+case "$xenbranch" in
+  *)
+    defsuite=`getconfig DebianSuite`
+    defguestsuite=`getconfig GuestDebianSuite`
+    ;;
+esac
+
+# Pick default Debian Installer version to correspond to the chosen
+# suite.
+if [ -z "$defdi_version" ] ; then
+    defdi_version=`getconfig_TftpDiVersion_suite $defsuite`
+fi
+
+guest_di_version=$defdi_version
+
+job_create_test_filter_callback () {
+  local job=$1; shift
+  local recipe=$1; shift
+  local toolstack=$1; shift
+  local xenarch=$1; shift
+  local dom0arch=$1; shift
+
+  # cut down the arm64 tests temporarily, due to lack of capacity
+  case "$branch" in
+    *arm*|*libvirt*) ;;
+    *)
+      case "$job" in
+        test-arm64-arm64-xl-multivcpu)		return 1 ;;
+        test-arm64-arm64-xl-rtds)		return 1 ;;
+        test-arm64-arm64-libvirt)		return 1 ;;
+        test-arm64-arm64-libvirt-qcow2)		return 1 ;;
+      esac
+      ;;
+  esac
+
+  case "$branch" in
+    xen-unstable-smoke)
+      case "$job" in
+        test-amd64-amd64-libvirt)                  return 0 ;;
+        test-armhf-armhf-xl)                       return 0 ;;
+        test-arm64-arm64-xl-xsm)                   return 0 ;;
+        test-amd64-amd64-xl-qemuu-debianhvm-i386)  return 0 ;;
+        *)                                         return 1 ;;
+      esac
+      ;;
+    qemu-upstream-4.2-testing)
+      case " $* " in
+        *" device_model_version=qemu-xen "*)
+          ;;
+        *)
+          : "suppressed $job"
+          return 1;;
+      esac
+      ;;
+    libvirt)
+      if [ x$toolstack != xlibvirt ] ; then return 1; fi
+      ;;
+    rumprun)
+      case "$job" in
+      *-rumprun-*)
+          ;;
+      *)
+          : "suppressed $job"
+          return 1;;
+      esac
+      ;;
+    xtf)
+      case "$xenarch:$dom0arch" in
+          amd64:amd64) ;;
+          *) return 1;;
+      esac
+      case $job in
+          *-xtf-*) ;;
+          *) return 1;;
+      esac
+      ;;
+    seabios)
+      case $xenarch in
+          amd64|i386) ;;
+          *) return 1;;
+      esac
+      case $job in
+          *-qemuu-ovmf-*) return 1;;
+          *-qemuu-*) ;;
+          *) return 1;;
+      esac
+      ;;
+    ovmf)
+      case $xenarch in
+          amd64|i386) ;;
+          *) return 1;;
+      esac
+      case $job in
+          *-qemuu-*) ;;
+          *) return 1;;
+      esac
+      case " $* " in
+        *" bios=ovmf "*) ;;
+        *) return 1;;
+      esac
+      ;;
+    *)
+      case "$job" in
+        *-qemuu-*)
+          if [ x$toolstack != xxl -a x$toolstack != xlibvirt ];then
+              return 1;
+          fi
+          ;;
+      esac
+      ;;
+  esac
+
+  return 0;
+}
+
+do_hvm_winxp_tests () {
+  case $xenbranch in
+    xen-3.*-testing)      ;;
+    xen-4.0-testing)      ;;
+    xen-4.1-testing)      ;;
+    xen-4.2-testing)      ;;
+    xen-4.3-testing)      ;;
+    xen-4.4-testing)      ;;
+    xen-4.5-testing)      ;;
+    *)                    return;;
+  esac
+
+  for vcpus in '' 1; do
+    case "$vcpus" in
+      '') vcpus_runvars=''; vcpus_suffix='' ;;
+      *) vcpus_runvars=guests_vcpus=$vcpus; vcpus_suffix=-vcpus$vcpus ;;
+    esac
+
+    if [ "x$vcpus" != x ] && \
+       [ "$xenarch$kern-$dom0arch" != "amd64-i386" ]; then
+      continue
+    fi
+
+    toolstack="xl"
+
+    if [ x$toolstack = x ] ; then
+      stripy toolstack xend xl \
+            "$vcpus" 1 \
+            "$kern" '' \
+            "$xenarch" i386 \
+            "$dom0arch" i386
+    fi
+
+    toolstack_runvars="toolstack=$toolstack"
+
+    job_create_test \
+            test-$xenarch$kern-$dom0arch-$toolstack$qemuu_suffix-winxpsp3$vcpus_suffix \
+            test-win $toolstack $xenarch $dom0arch $qemuu_runvar \
+            win_image=winxpsp3.iso $vcpus_runvars   \
+            win_acpi_shutdown=true \
+            $centos_test_runvars \
+            all_hostflags=$most_hostflags,hvm
+
+  done
+}
+
+do_hvm_win_test_one () {
+  local testidpart=$1
+  local isobase=$2
+  local guestarch=$3
+  shift;shift;shift
+
+  if [ $xenarch != amd64 ]; then
+    return
+  fi
+
+  case "$guestarch" in
+    amd64)  win_arch=x64 ;;
+    i386)   win_arch=x86 ;;
+    *)      win_arch=$guestarch ;; # probably wrong
+  esac
+
+  local iso=$isobase-$win_arch.iso
+
+  job_create_test \
+      test-$xenarch$kern-$dom0arch-xl$qemuu_suffix-$testidpart-$guestarch \
+            test-win xl $xenarch $dom0arch $qemuu_runvar \
+            win_image=$iso \
+            win_acpi_shutdown=true \
+            $centos_test_runvars \
+            all_hostflags=$most_hostflags,hvm \
+            "$@"
+}
+
+do_hvm_win7_x64_tests () {
+  do_hvm_win_test_one win7 win7 amd64
+}
+
+do_hvm_win_2017_tests () {
+  do_hvm_win_test_one ws16  ws16       amd64 guests_memsize=3584
+  do_hvm_win_test_one win10 win10v1703 i386  guests_memsize=3584
+}
+
+branch_debianhvm_arch () {
+  case $branch in
+    xen-unstable-smoke) echo i386;;
+    *) echo amd64;;
+  esac
+}
+
+do_hvm_debian_test_one () {
+  testname=$1
+  toolstack=$2
+  bios=$3
+  xsm=$4 # 'false' or 'true'
+  stubdom=$5 # '' (or unset) or 'true'
+
+  local arch=$(branch_debianhvm_arch)
+  local testvars
+
+  case "$arch" in
+    amd64) iso_dir='install.amd' ;;
+    i386)  iso_dir='install.386'
+           testvars+=' debianhvm_install_timeoutfactor=1.5'
+           testvars+=' debianhvm_diskdevice=/dev/sda' ;;
+    *)     iso_dir="install.$arch" ;;
+  esac
+
+  stubdom_suffix=""
+  stubdom_runvar=""
+  if [ x$stubdom = xtrue ]; then
+      stubdom_suffix="-stubdom"
+      stubdom_runvar="debianhvm_stubdom=$stubdom"
+  fi
+
+  job_create_test test-$xenarch$kern-$dom0arch-$toolstack$qemuu_suffix$stubdom_suffix-$testname-$arch\
+    test-debianhvm $toolstack $xenarch $dom0arch $qemuu_runvar \
+    $stubdom_runvar $testvars                   \
+    debianhvm_suite=$guestsuite			\
+    debianhvm_image=$(usual_debianhvm_image $arch) \
+    debianhvm_iso_kernel=/$iso_dir/vmlinuz \
+    debianhvm_iso_ramdisk=/$iso_dir/initrd.gz \
+    bios=$bios \
+    $centos_test_runvars \
+    all_hostflags=$most_hostflags,hvm
+}
+
+do_hvm_debian_tests() {
+  if [ $xenarch != amd64 ]; then
+    return
+  fi
+
+  # QEMU upstream supports
+  #   1. ovmf + xl
+  #   2. seabios + xl
+  #   3. seabios + libvirt
+  # For libvirt we only generate -xsm test case.
+  if [ "x$qemuu_suffix" == "x-qemuu" ]; then
+    do_hvm_debian_test_one ovmf xl ovmf false
+    do_hvm_debian_test_one debianhvm xl seabios false
+  fi
+
+  # QEMU traditional supports rombios and stubdom
+  # Only test xl with QEMU traditional
+  if [ "x$qemuu_suffix" == "x-qemut" ]; then
+    do_hvm_debian_test_one debianhvm xl rombios false
+  fi
+}
+
+do_hvm_rhel6_tests () {
+  if [ $xenarch != amd64 -o $dom0arch != i386 -o "$kern" != "" ]; then
+    return
+  fi
+
+  for cpuvendor in amd intel; do
+
+    job_create_test test-$xenarch$kern-$dom0arch$qemuu_suffix-rhel6hvm-$cpuvendor \
+                                            test-rhelhvm xl $xenarch $dom0arch \
+            redhat_image=rhel-server-6.1-i386-dvd.iso \
+            $centos_test_runvars \
+            all_hostflags=$most_hostflags,hvm-$cpuvendor \
+            $qemuu_runvar
+
+  done
+}
+
+do_pygrub_tests () {
+  if [ $xenarch != amd64 -o $dom0arch != amd64 -o "$kern" != "" ]; then
+    return
+  fi
+
+  job_create_test test-$xenarch$kern-$dom0arch-pygrub   \
+    test-debian-di xl $xenarch $dom0arch                \
+      debian_arch=amd64                                 \
+      debian_suite=$guestsuite                          \
+      debian_di_version=$guest_di_version               \
+      debian_method=netboot                             \
+      debian_bootloader=pygrub                          \
+      $centos_test_runvars                              \
+      all_hostflags=$most_hostflags
+}
+
+do_pv_debian_test_one () {
+  testname=$1; shift
+  recipe_sfx=$1; shift
+  toolstack=$1; shift
+  platform=$1; shift
+
+  case "$recipe_sfx" in
+    # test-debian recipe requires xen-tools
+    '') return ;;
+  esac
+
+  suffix=${platform:+-$platform}
+  hostflags=${most_hostflags}${platform:+,platform-$platform}
+
+  job_create_test test-$xenarch$kern-$dom0arch-$testname$suffix \
+     test-debian$recipe_sfx $toolstack                          \
+            $xenarch $dom0arch                                  \
+            $centos_test_runvars                                \
+            $debian_runvars all_hostflags=$hostflags $@
+}
+
+do_pv_debian_tests () {
+  # Basic PV Linux test with xl
+  for platform in '' `getplatforms $xenarch` ; do
+    do_pv_debian_test_one xl '' xl "$platform"
+  done
+
+  do_pv_debian_test_one libvirt '' libvirt ''
+
+  # We compute a desired architecture for each combination of ts and fmt
+  # Within each fmt we rotate through the list of arches
+  # The starting list rotates once per ts, so that we try to
+  # exercise each fmt on each arch family.
+  local fmtarches_outer="i386 armhf amd64"
+  local endfmt="do_pv_debian_tests-missing-ts-fmt-for-dom0arch="
+
+  for ts in xl libvirt ; do
+
+    local fmtarches=$fmtarches_outer
+    fmtarches_outer="${fmtarches_outer#* } ${fmtarches_outer%% *}"
+
+    for fmt in raw vhd qcow2 ; do
+
+      local fmtarch="${fmtarches%% *}"
+      fmtarches="${fmtarches#* } $fmtarch"
+
+      if [ "x$fmtarch" != "x$dom0arch" ]; then
+        continue
+      else
+        endfmt=": "
+      fi
+
+      fmt_runvar="debian_diskfmt=$fmt"
+
+      do_pv_debian_test_one $ts-$fmt '-di' $ts '' \
+          debian_arch=$dom0arch                 \
+          debian_suite=$guestsuite              \
+          debian_method=netboot                 \
+          debian_bootloader=pygrub              \
+          $fmt_runvar
+
+    done
+  done
+
+  $endfmt$dom0arch
+}
+
+do_centos_cbs_package () {
+  case "$branch" in
+    centos)   ;;
+    *) return ;;
+  esac
+  # test-centos doesn't do anything
+  # This could be a guest test.
+  job_create_test test-$xenarch$kern-$dom0arch-centos-devel \
+      test-centos xl $xenarch $dom0arch \
+      centos_cbs_repo=candidate \
+      bios=seabios \
+      $centos_test_runvars \
+      all_hostflags=$most_hostflags \
+      all_host_os=centos
+}
+
+# Override job_create_test from mfi-common
+job_create_test () {
+  local job=$1; shift
+  local recipe=$1; shift
+  local toolstack=$1; shift
+  local xenarch=$1; shift
+  local dom0arch=$1; shift
+
+  job="$job"
+
+  job_create_test_filter_callback \
+    "$job" "$recipe" "$toolstack" "$xenarch" "$dom0arch" "$@" || return 0
+
+  ./cs-job-create $flight $job $recipe toolstack=$toolstack       \
+    $RUNVARS $TEST_RUNVARS $global_runvars $most_runvars          \
+    "$@"
+}
+
+test_matrix_do_one () {
+
+  do_pv_debian_tests
+
+  # RTDS came in 4.5
+  case "$xenbranch" in
+  xen-3.*-testing) test_rtds=n ;;
+  xen-4.0-testing) test_rtds=n ;;
+  xen-4.1-testing) test_rtds=n ;;
+  xen-4.2-testing) test_rtds=n ;;
+  xen-4.3-testing) test_rtds=n ;;
+  xen-4.4-testing) test_rtds=n ;;
+  xen-4.5-testing) test_rtds=y ;;
+  *)               test_rtds=y ;;
+  esac
+
+  for qemuu_suffix in '' -qemut -qemuu; do
+    case "$qemuu_suffix" in
+    '')
+          case $xenbranch in
+          xen-3.*-testing) ;;
+          xen-4.0-testing) ;;
+          xen-4.1-testing) ;;
+          xen-4.2-testing) ;;
+          *) continue ;;
+          esac
+          qemuu_runvar=''
+          ;;
+    -qemut)
+          qemuu_runvar=device_model_version=qemu-xen-traditional
+          case $branch in
+          qemu-mainline*) continue ;;
+          qemu-upstream*) continue ;;
+          esac
+          ;;
+    -qemuu)
+          case $xenbranch in
+          xen-3.*-testing) continue;;
+          xen-4.0-testing) continue;;
+          xen-4.1-testing) continue;;
+          esac
+          qemuu_runvar=device_model_version=qemu-xen
+          ;;
+    esac
+
+    do_hvm_winxp_tests
+    do_hvm_win7_x64_tests
+    do_hvm_win_2017_tests
+    do_hvm_rhel6_tests
+
+    do_hvm_debian_tests
+
+  done # qemuu_suffix
+
+  do_pygrub_tests
+}
+
+xenarch=amd64
+dom0arch=amd64
+guestsuite=$defguestsuite
+di_version=$defdi_version
+pairtoolstack="xl libvirt"
+
+hostos_runvars="
+  all_host_os=centos        \
+  "
+
+eval "
+    arch_runvars=\"\$ARCH_RUNVARS_$dom0arch\"
+"
+kernkind=pvops
+debian_runvars="debian_kernkind=$kernkind \
+                debian_arch=$dom0arch \
+                debian_suite=$guestsuite \
+                "
+most_hostflags="arch-$dom0arch,arch-xen-$xenarch,purpose-test"
+most_runvars="
+          arch=$dom0arch                                  \
+          kernkind=$kernkind                              \
+          $arch_runvars $hostos_runvars
+          "
+
+test_matrix_do_one
+
+echo $flight
+
+# Local variables:
+# mode: sh
+# sh-basic-offset: 2
+# indent-tabs-mode: nil
+# End:
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[OSSTEST RFC 16/16] Osstest/TestSupport: Handle qemu-img location on CentOS

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/TestSupport.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 40a5c5a..c82ba96 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1949,8 +1949,8 @@ sub make_qcow2 ($$$) {
     my ($ho, $gho, $disk_mb) = @_;
     # upstream qemu's version. Seems preferable to qemu-xen-img from qemu-trad.
     my $qemu_img;
-    foreach (qw(/usr/local /usr)) {
-	my $try = "$_/lib/xen/bin/qemu-img";
+    foreach (qw(/usr/local/lib /usr/lib /usr/lib64)) {
+	my $try = "$_/xen/bin/qemu-img";
         if (target_file_exists($ho, $try)) {
             $qemu_img=$try;
             last;
-- 
Anthony PERARD


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [OSSTEST RFC 01/16] JobDB-Standalone.tcl: Fix read-runvar

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST RFC 01/16] JobDB-Standalone.tcl: Fix read-runvar"):
> This fix the error bellow, when the runvar exist:
> can't read "runvarinfo()": no such element in array

*boggle*

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [OSSTEST RFC 02/16] Osstest/TestSupport: In teditfileex, get the file with the requested user

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST RFC 02/16] Osstest/TestSupport: In teditfileex, get the file with the requested user"):
> e.g. If the file to be edited is only accessible to the root user,
> target_getfile() would not work.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [OSSTEST RFC 03/16] Osstest/TestSupport: Add centos to package_install_cmd

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST RFC 03/16] Osstest/TestSupport: Add centos to package_install_cmd"):
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

But, are we going to invent a package name translation layer, or
something ?  Or are we going to do this ad-hoc in each call to
target_install_packages ?

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [OSSTEST RFC 04/16] TestSupport: In host_install_postboot, don't call update-rc.d on CentOS

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST RFC 04/16] TestSupport: In host_install_postboot, don't call update-rc.d on CentOS"):
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>

At the very least, this needs an explanation of how, on CentOS, the
same effect is achieved.

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [OSSTEST RFC 06/16] Osstest/CentOS.pm: Introduce CentOS support, starting with kickstart_installcmdline_core

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST RFC 06/16] Osstest/CentOS.pm: Introduce CentOS support, starting with kickstart_installcmdline_core"):
> This function will be use later to install CentOS on a host.

How does any of this relate to ts-redhat-install ?  I think that
really, ts-redhat-install needs to share a lot of this.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [OSSTEST RFC 09/16] CentOS: Setup osstest-confirm-booted.service

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST RFC 09/16] CentOS: Setup osstest-confirm-booted.service"):
> This emulate the osstest-confirm-booted service that is done on Debian.
> 
> In order to have the service been started last:
> - it is made dependent on multi-user.target which is the default target on systemd.
> - it is part of osstest.target which depend on multi-user.target.
> - the osstest.target is set as the new default.

This seems quite fragile.  In particular, two scripts which both tried
to do this wouldn't compose.  But if there isn't a better way in
CentOS's provided systemd setup, then so be it.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [OSSTEST RFC 14/16] sg-run-job: Select host install script based on all_host_os runvar

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST RFC 14/16] sg-run-job: Select host install script based on all_host_os runvar"):
> This also select a different xen installation script.

This is wrong, I'm afraid.  In general, the recipes should not read
runvars (with the exception perhaps of runvars that are specifically
targeted at sg-run-job).

I think this should be done either by (a) having ts-xen-install be
able to install packages rather than xen.git-built tarballs, according
to runvars or (b) inventing a new recipe-specific proc a la
run-job/RECIPE, need-hosts/RECIPE, etc. (xen-install/RECIPE, maybe).

I think (a) is likely to be better because some of the same
configuration will be needed.  For example, I don't know who does
bridge setup on CentOS but the Debian Xen packages expect the user to
do it, just like the upstream ones, so a ts- which installs Debian Xen
packages needs to reuse the Debian bridge creation functionality.

What do you think ?

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [OSSTEST RFC 10/16] ts-centos-xen-pkg-install: Install of Xen package on CentOS

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST RFC 10/16] ts-centos-xen-pkg-install: Install of Xen package on CentOS"):
> Install candidate packages that have been built by CBS, the CentOS
> Community Build Service.
...
> +name=VirtSIG-\$releasever - Xen 4.8 CBS $subtag
> +baseurl=http://cbs.centos.org/repos/virt\$releasever-xen-48-$subtag/\$basearch/os/
> +gpgcheck=0
> +[virt-xen-common-$subtag]
> +name=VirtSIG-\$releasever - Xen common CBS
> +baseurl=http://cbs.centos.org/repos/virt\$releasever-xen-common-$subtag/\$basearch/os/
> +gpgcheck=0

URLs must not be hardcoded in ts-* scripts.  They must come from
config or runvars.  In this case, runvars, which should come from the
config.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [OSSTEST RFC 11/16] ts-centos-xen-pkg-install: Adjust daemons configuration

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST RFC 11/16] ts-centos-xen-pkg-install: Adjust daemons configuration"):
> Ajust configuration of xenconsoled and libvirtd.

See my comments about ts-xen-install.  It already does this.  Please
don't duplicate things.

Thanks,
Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [OSSTEST RFC 13/16] WORKAROUND: Osstest/TestSupport: Make target_reboot works with systemd

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST RFC 13/16] WORKAROUND: Osstest/TestSupport: Make target_reboot works with systemd"):
> On host running with systemd as init, doing `ssh host reboot` will
> result in ssh returning an error.
> This patch works around by not waiting for the reboot command to return.

Erk.  What is your plan for fixing this properly ?

Ina.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [OSSTEST RFC 16/16] Osstest/TestSupport: Handle qemu-img location on CentOS

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST RFC 16/16] Osstest/TestSupport: Handle qemu-img location on CentOS"):
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
...
> +    foreach (qw(/usr/local/lib /usr/lib /usr/lib64)) {

Yuk.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

But, shouldn't it include /usr/local/lib64 too ?

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [OSSTEST RFC 15/16] make-centos-flight: Create a flight with CentOS as dom0

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST RFC 15/16] make-centos-flight: Create a flight with CentOS as dom0"):
> This is based on make-flight, with the added all_host_os=centos runvar,
> and without test that can not be run.
> 
> Anything based on the recipe "test-debian" or "test-pair" is remove, as
> they require xen-tools. There is no XSM tests as the CentOS packages is
> built without. There is no build jobs as the Xen packages been tested
> are built in the CentOS CBS.

Please provide output (diff, or extract) from
standalone-dump-all-flight-runvars.

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [OSSTEST RFC 05/16] TestSupport: Adapt target_https_mitm_proxy_setup to CentOS

[Ian Jackson]

Anthony PERARD writes ("[OSSTEST RFC 05/16] TestSupport: Adapt target_https_mitm_proxy_setup to CentOS"):
> The location for new certificates is different, and
> update-ca-certificates is Debian specific.
...
> +    my $dest;
> +    my $update_ca_cmd;
> +    if ($ho->{OS} eq "centos" ) {

I'm not sure "centos" is the right key here.  Maybe we need
   is_host_redhat_derived($ho)
?

> +        $dest = '/etc/pki/ca-trust/source/anchors';
> +        $update_ca_cmd = 'update-ca-trust extract';
> +    } else {
> +        $dest = '/usr/local/share/ca-certificates';
> +        $update_ca_cmd = 'update-ca-certificates';
> +    }
>      target_putfilecontents_root_stash($ho,30,$cert,
> -                  '/usr/local/share/ca-certificates/osstest.crt');
> -    target_cmd_root($ho, 'update-ca-certificates', 300);
> +        $dest.'/osstest.crt');
> +    target_cmd_root($ho, $update_ca_cmd, 300);

But the general approach is right.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel