[OSSTEST PATCH v17 00/24] Nested HVM

[OSSTEST PATCH v17 00/24] Nested HVM

[Ian Jackson]

This is, I think, a fully-working series to support Nested HVM testing
in osstest.  It can be found here:

git://xenbits.xen.org/people/iwj/osstest.git
  http://xenbits.xen.org/git-http/people/iwj/osstest.git
in base.nested-hvm.v17..wip.nested-hvm.v17

 a  01  cs-adjust-flight: Add some missing doc comment info
*   02  cs-adjust-flight: Allow adjusting "this" flight
 aT 03  selecthost: Minor cleanups
+   04  make-flight: Break out usual_debianhvm_image and honour ...
 aT 05  selecthost: Support nested hosts (guests which are also hosts)
 aT 06  Nested hosts: Provide PDU power method
 aT 07  DhcpWatch::leases: Fix a reporting message
 aT 08  target_check_ip: Rename and improve from guest_check_ip
 aT 09  await_tcp(): Run check_ip on each loop iteration
 aT 10  LVM: Break out lv_create
 aT 11  Toolstack::xl: Provide block_attach method
 at 12  sg-run-job: Break out per-host-prep and per-host-finish
 AT 13  sg-run-job: Provide infrastructure for layers of nesting
 aT 14  Nested HVM: Provide ts-nested-setup to help make L1 usable as a host
 aT 15  Nested HVM: Provide test-nested recipe
* T 16  Nested HVM: Add test job to appropriate flights
 aT 17  ts-xen-install: Properly handle hosts without a static IP address
 aT 18  ts-xen-install: networking: Rename `nodhcp' to `ensurebridge'
 aT 19  Osstest/Testsupport.pm: change target's default kernkind to 'pvops'
 aT 20  Osstest/Testsupport.pm: use get_target_property() for some host setup
 at 21  HVM guests: Use qemu "pipe:" for serial output logging
 at 22  Serial: Factor out Osstest::Serial::keys_real
* t 23  Serial: Add new serial method object for `guest' type
+   24  Serial::xenuse: Send xenuse output to /dev/null

Key:
  a   acked by Ian Campbell
  A   acked by Ian Jackson
  T   formally Tested-by Robert Hu
  t   informal test report in email from Robert Hu

Thanks,
Ian.

[OSSTEST PATCH 01/24] cs-adjust-flight: Add some missing doc comment info

[Ian Jackson]

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v14: New patch
---
 cs-adjust-flight |    8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/cs-adjust-flight b/cs-adjust-flight
index 4bfef48..d70abf7 100755
--- a/cs-adjust-flight
+++ b/cs-adjust-flight
@@ -24,6 +24,14 @@
 #   .                 means all jobs
 #   ^<pcre>           means $foo =~ m/^<pcre>/
 #   /<pcre>           means $foo =~ m/<pcre>/
+#
+# <dst-flight>:
+#   <flight>
+#   new:<intended-blessing>
+#
+# options:
+#   -v              verbose - list changes to stderr
+#   -D              debugging output to stderr
 
 # This is part of "osstest", an automated testing framework for Xen.
 # Copyright (C) 2009-2013 Citrix Inc.
-- 
1.7.10.4

[OSSTEST PATCH 02/24] cs-adjust-flight: Allow adjusting "this" flight

[Ian Jackson]

This allows cs-adjust-flight to be run by hand to adjust runvars, in a
flight being used with hand-invocation of ./ts-* scripts.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Tested-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v17: Better documentation of running: flight status
v14: New patch
---
 cs-adjust-flight |    9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/cs-adjust-flight b/cs-adjust-flight
index d70abf7..a72cd88 100755
--- a/cs-adjust-flight
+++ b/cs-adjust-flight
@@ -28,6 +28,8 @@
 # <dst-flight>:
 #   <flight>
 #   new:<intended-blessing>
+#   running:                 uses OSSTEST_FLIGHT (flight referred to
+#                             must be blessed `running' or `constructing')
 #
 # options:
 #   -v              verbose - list changes to stderr
@@ -395,6 +397,13 @@ sub main () {
             verbose_discard();
             changes();
         });
+    } elsif ($dstflightspec =~ m/^running:$/) {
+        $dstflight = $ENV{OSSTEST_FLIGHT} // die;
+        db_retry($dstflight,[qw(constructing running)],
+                 $dbh_tests, [qw(flights)], sub {
+            verbose_discard();
+            changes();
+        });
     } elsif ($dstflightspec =~ m/^new:/) {
         my $intended = $'; #';
         db_retry($dbh_tests, [qw(flights)], sub {
-- 
1.7.10.4

[OSSTEST PATCH 03/24] selecthost: Minor cleanups

[Ian Jackson]

Document the syntax for $ident.

Log the ident as well as the selected hostname.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Tested-by: Robert Ho <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v17: Fix typo in doc message.
v14: New patch
---
 Osstest/TestSupport.pm |   16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index aa41952..8c3612c 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -810,6 +810,20 @@ sub power_state ($$) {
 sub selecthost ($) {
     my ($ident) = @_;
     # must be run outside transaction
+
+    # $ident is <identspec>
+    #
+    # <identspec> can be <ident>, typically "host" or "xxx_host"
+    # which means look up the runvar $r{$ident} which
+    # contains <hostspec>
+    # OR
+    # <identspec> can be <ident>=<hostspec>
+    # which means ignore <ident> except for logging purposes etc.
+    # and use <hostspec>
+    #
+    # <hostspec> is <hostname> which means use that host (and all
+    # its flags and properties from the configuration and database)
+
     my $name;
     if ($ident =~ m/=/) {
         $ident= $`;
@@ -917,7 +931,7 @@ sub selecthost ($) {
 
     $mjobdb->host_check_allocated($ho);
 
-    logm("host: selected $ho->{Name} ".
+    logm("host $ho->{Ident}: selected $ho->{Name} ".
 	 (defined $ho->{Ether} ? $ho->{Ether} : '<unknown-ether>').
 	 " $ho->{Ip}".
          (!$ho->{Shared} ? '' :
-- 
1.7.10.4

[OSSTEST PATCH 04/24] make-flight: Break out usual_debianhvm_image and honour DEBIAN_IMAGE_VERSION

[Ian Jackson]

No functional change.  (Verified for xen-unstable with
standalone-generate-dump-flight-runvars.)

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Tested-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v17: New patch.
---
 make-flight |    2 +-
 mfi-common  |    5 +++++
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/make-flight b/make-flight
index 28820f5..f908017 100755
--- a/make-flight
+++ b/make-flight
@@ -285,7 +285,7 @@ do_hvm_debian_test_one () {
     test-debianhvm $toolstack $xenarch $dom0arch $qemuu_runvar \
     enable_xsm=$xsm                             \
     $stubdom_runvar $testvars                   \
-    debianhvm_image=debian-7.2.0-$arch-CD-1.iso \
+    debianhvm_image=$(usual_debianhvm_image $arch) \
     debianhvm_iso_kernel=/$iso_dir/vmlinuz \
     debianhvm_iso_ramdisk=/$iso_dir/initrd.gz \
     bios=$bios \
diff --git a/mfi-common b/mfi-common
index fab4aa3..5fbe195 100644
--- a/mfi-common
+++ b/mfi-common
@@ -351,6 +351,11 @@ job_create_test () {
     xenbuildjob=$xenbuildjob buildjob=$buildjob $tsbuildjob "$@"
 }
 
+usual_debianhvm_image () {
+  local arch=$1; shift
+  echo debian-${DEBIAN_IMAGE_VERSION-7.2.0}-$arch-CD-1.iso
+}
+
 # Iterate over xenarch, dom0arch and kernel calling test_matrix_do_one
 # for each combination.
 #
-- 
1.7.10.4

[OSSTEST PATCH 05/24] selecthost: Support nested hosts (guests which are also hosts)

[Ian Jackson]

We introduce a new syntax: instead of a hostname (which might appear
in a command line argument to a ts-* script and hence be passed to
selecthost, or which might be in a runvar), we now support
<hostspec>:<domname>.

Such `hosts' (let us refer to such a thing as an L1, although in
principle further nesting may be possible) are expected to be
dynamically created.  So they do not have flags and properties in the
configuration (or in an Executive instance's database).

The IP address is determined dynamically from the leases file.  If the
L1 is not running, then no IP address may be found.  This is not an
error.  Users of this facility will need to make sure that ts-*
scripts which are unaware of the L1's special status are only invoked
when it is known that the L1 is up and has obtained its IP address.

`Power cycling' the L1 will be done by VM control operations in the
L0; this will come in a subsequent patch.

`Serial access' to the L1 guest will likewise need to be done via the
console arrangements in L0.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Tested-by: Robert Ho <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v14: New patch
---
 Osstest/TestSupport.pm |   37 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 36 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 8c3612c..3145040 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -807,6 +807,7 @@ sub power_state ($$) {
 
 #---------- host selection and properties ----------
 
+sub selecthost ($);
 sub selecthost ($) {
     my ($ident) = @_;
     # must be run outside transaction
@@ -821,8 +822,12 @@ sub selecthost ($) {
     # which means ignore <ident> except for logging purposes etc.
     # and use <hostspec>
     #
-    # <hostspec> is <hostname> which means use that host (and all
+    # <hostspec> can be <hostname> which means use that host (and all
     # its flags and properties from the configuration and database)
+    # OR
+    # <hostspec> can be <parent_identspec>:<domname> meaning use the
+    # Xen domain name <domname> on the host specified by
+    # <parent_identspec>, which is an <identspec> as above.
 
     my $name;
     if ($ident =~ m/=/) {
@@ -838,6 +843,7 @@ sub selecthost ($) {
         Ident => $ident,
         Name => $name,
         TcpCheckPort => 22,
+        NestingLevel => 0,
         Info => [],
     };
     if (defined $job) {
@@ -845,6 +851,35 @@ sub selecthost ($) {
 					  $c{DebianSuite});
     }
 
+    #----- handle hosts which are themselves guests (nested) -----
+
+    if ($name =~ s/^(.*)://) {
+	my $parentname = $1;
+	my $parent = selecthost($parentname);
+	my $child = selectguest($name,$parent);
+	$child->{Ident} = $ho->{Ident};
+	$child->{Info} = [ "in", $parent->{Name}, @{ $parent->{Info} } ];
+	$child->{NestingLevel} = $parent->{NestingLevel}+1;
+
+	# $child->{Power} = 'guest';   todo
+	power_cycle_host_setup($child);
+
+	$child->{Properties}{Serial} = 'noop'; # todo
+	serial_host_setup($child);
+
+	my $msg = "L$child->{NestingLevel} host $child->{Ident}:";
+	$msg .= " guest $child->{Guest} (@{ $child->{Info} })";
+	$msg .= " $child->{Ether}";
+
+	my $err = guest_check_ip($child);
+	$msg .= " ".(defined $err ? "<no-ip> $err" : $child->{Ip});
+
+	logm($msg);
+
+	# all the rest of selecthost is wrong for this case
+	return $child;
+    }
+
     #----- calculation of the host's properties -----
 
     $ho->{Properties} = { };
-- 
1.7.10.4

[OSSTEST PATCH 06/24] Nested hosts: Provide PDU power method

[Ian Jackson]

From: Robert Ho <robert.hu@intel.com>

This `guest' power method uses VM create/destroy.  It is automatically
used for nested hosts.  It would not make much sense to configure it
manually.

For nested host/guest, its power on/off method shall be
its host invoke $(toolstack)->create/destroy method.

Signed-off-by: Robert Ho <robert.hu@intel.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Tested-by: Robert Ho <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v14: Mostly rewritten by iwj
---
 Osstest/PDU/guest.pm   |   59 ++++++++++++++++++++++++++++++++++++++++++++++++
 Osstest/TestSupport.pm |    2 +-
 2 files changed, 60 insertions(+), 1 deletion(-)
 create mode 100755 Osstest/PDU/guest.pm

diff --git a/Osstest/PDU/guest.pm b/Osstest/PDU/guest.pm
new file mode 100755
index 0000000..b6bf9a1
--- /dev/null
+++ b/Osstest/PDU/guest.pm
@@ -0,0 +1,59 @@
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Intel.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+package Osstest::PDU::guest;
+
+use strict;
+use warnings;
+use Switch;
+
+use Osstest;
+use Osstest::TestSupport;
+use IO::File;
+
+BEGIN {
+    use Exporter ();
+    our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS);
+    $VERSION     = 1.00;
+    @ISA         = qw(Exporter);
+    @EXPORT      = qw();
+    %EXPORT_TAGS = ( );
+
+    @EXPORT_OK   = qw();
+}
+
+sub new {
+    my ($class, $ho) = @_;
+    return bless { Target => $ho }, $class;
+}
+
+sub pdu_power_state {
+    my ($mo, $on) = @_;
+
+    my $child = $mo->{Target};
+    my $parent = $child->{Host};
+    die "$child->{Name} ?" unless $parent;
+
+    logm("power $child->{Name} nested on $parent->{Name} ".($on+0));
+    if ($on) {
+	toolstack($parent)->create($child);
+    } else {
+	toolstack($parent)->destroy($child);
+    }
+}
+
+1;
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 3145040..9a256a9 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -861,7 +861,7 @@ sub selecthost ($) {
 	$child->{Info} = [ "in", $parent->{Name}, @{ $parent->{Info} } ];
 	$child->{NestingLevel} = $parent->{NestingLevel}+1;
 
-	# $child->{Power} = 'guest';   todo
+	$child->{Power} = 'guest';
 	power_cycle_host_setup($child);
 
 	$child->{Properties}{Serial} = 'noop'; # todo
-- 
1.7.10.4

[OSSTEST PATCH 07/24] DhcpWatch::leases: Fix a reporting message

[Ian Jackson]

This talks about `guest_check_ip', but this code is now factored out
into a method.  Use the correct method name in reporting.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Tested-by: Robert Ho <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v14: New patch
---
 Osstest/DhcpWatch/leases.pm |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/DhcpWatch/leases.pm b/Osstest/DhcpWatch/leases.pm
index 9a74c40..b74ebf0 100644
--- a/Osstest/DhcpWatch/leases.pm
+++ b/Osstest/DhcpWatch/leases.pm
@@ -171,7 +171,7 @@ sub check_ip ($$) {
     }
     $gho->{Ip}= $best->{' addr'};
 
-    report_once($gho, 'guest_check_ip', 
+    report_once($gho, 'leases::check_ip', 
 		"guest $gho->{Name}: $gho->{Ether} $gho->{Ip}");
     return undef;
 }
-- 
1.7.10.4

[OSSTEST PATCH 08/24] target_check_ip: Rename and improve from guest_check_ip

[Ian Jackson]

Make this function suitable for running on targets with static IP
addresses.  (Ie, on physical hosts.)  Accordingly, rename it and
adjust all call sites.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Tested-by: Robert Ho <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v14: New patch
---
 Osstest/TestSupport.pm |   11 ++++++-----
 ts-guest-localmigrate  |    2 +-
 ts-guest-migrate       |    2 +-
 ts-guest-saverestore   |    2 +-
 4 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 9a256a9..325e2e3 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -95,7 +95,7 @@ BEGIN {
                       prepareguest_part_lvmdisk prepareguest_part_diskimg
                       prepareguest_part_xencfg
                       guest_umount_lv guest_await guest_await_dhcp_tcp
-                      guest_checkrunning guest_check_ip guest_find_ether
+                      guest_checkrunning target_check_ip guest_find_ether
                       guest_find_domid guest_check_up guest_check_up_quick
                       guest_get_state guest_await_reboot
                       guest_await_shutdown guest_await_destroy guest_destroy
@@ -730,8 +730,9 @@ sub dhcp_watch_setup ($$) {
     $gho->{DhcpWatch} = get_host_method_object($ho, 'DhcpWatch', $meth);
 }
 
-sub guest_check_ip ($) {
-    my ($gho) = @_;
+sub target_check_ip ($) {
+    my ($gho) = @_; # returns error message or undef
+    return undef if $gho->{IpStatic};
     guest_find_ether($gho);
     $gho->{DhcpWatch}->check_ip($gho);
 }
@@ -871,7 +872,7 @@ sub selecthost ($) {
 	$msg .= " guest $child->{Guest} (@{ $child->{Info} })";
 	$msg .= " $child->{Ether}";
 
-	my $err = guest_check_ip($child);
+	my $err = target_check_ip($child);
 	$msg .= " ".(defined $err ? "<no-ip> $err" : $child->{Ip});
 
 	logm($msg);
@@ -1956,7 +1957,7 @@ sub guest_await_dhcp_tcp ($$) {
 	      " $gho->{TcpCheckPort}".
               " link/ip/tcp",
               sub {
-        my $err= guest_check_ip($gho);
+        my $err= target_check_ip($gho);
         return $err if defined $err;
 
         return
diff --git a/ts-guest-localmigrate b/ts-guest-localmigrate
index 8fe986d..85a0887 100755
--- a/ts-guest-localmigrate
+++ b/ts-guest-localmigrate
@@ -40,7 +40,7 @@ sub migrate () {
 guest_await_dhcp_tcp($gho, 5);
 guest_check_up($gho);
 
-my $err= guest_check_ip($gho);  die "$err $gho->{Name}" if defined $err;
+my $err= target_check_ip($gho);  die "$err $gho->{Name}" if defined $err;
 
 for (my $rep=1; $rep<=$reps; $rep++) {
     logm("========== rep $rep ==========");
diff --git a/ts-guest-migrate b/ts-guest-migrate
index b77d0de..505fab2 100755
--- a/ts-guest-migrate
+++ b/ts-guest-migrate
@@ -30,7 +30,7 @@ our $gho = selectguest($ARGV[2],$sho);
 
 sub migrate () {
     guest_checkrunning($sho,$gho) or die $gho->{Name};
-    my $err= guest_check_ip($gho);  die "$err $gho->{Name}" if defined $err;
+    my $err= target_check_ip($gho);  die "$err $gho->{Name}" if defined $err;
     toolstack($sho)->migrate($gho, $dho, $timeout{Migrate});
 }
 
diff --git a/ts-guest-saverestore b/ts-guest-saverestore
index 7e13d90..73883ef 100755
--- a/ts-guest-saverestore
+++ b/ts-guest-saverestore
@@ -26,7 +26,7 @@ our ($ho,$gho) = ts_get_host_guest(@ARGV);
 
 sub save () {
     guest_checkrunning($ho,$gho) or die $gho->{Name};
-    my $err= guest_check_ip($gho);  die "$err $gho->{Name}" if defined $err;
+    my $err= target_check_ip($gho);  die "$err $gho->{Name}" if defined $err;
     toolstack($ho)->save($gho,"image",200);
     target_ping_check_down($gho);
 }
-- 
1.7.10.4

[OSSTEST PATCH 09/24] await_tcp(): Run check_ip on each loop iteration

[Ian Jackson]

From: Robert Ho <robert.hu@intel.com>

await_tcp is often invoked after a reboot.

In this situation the target's IP address may change.  If this happens
while await_tcp is running, we would continue to poll the old IP address.
Fix this by running target_check_ip on each iteration.

Signed-off-by: Robert Ho <robert.hu@intel.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Tested-by: Robert Ho <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v14: Dropped change to selecthost, which was in code which is no
      longer present in this version of the series.
     Rewritten to use target_check_ip.
     Dropped IMO-unnecessary comment.
---
 Osstest/TestSupport.pm |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 325e2e3..b70f19f 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -2029,9 +2029,10 @@ sub await_tcp ($$$) {
     my ($maxwait,$interval,$ho) = @_;
     target_adjust_timeout($ho,\$maxwait);
     poll_loop($maxwait,$interval,
-              "await tcp $ho->{Name} $ho->{TcpCheckPort}",
+              "await tcp $ho->{Name} $ho->{Ip} $ho->{TcpCheckPort}",
               sub {
-        return target_tcp_check($ho,$interval);
+	return target_check_ip($ho) //
+               target_tcp_check($ho,$interval);
     });
 }
 
-- 
1.7.10.4

[OSSTEST PATCH 10/24] LVM: Break out lv_create

[Ian Jackson]

We are going to want to reuse this.

lv_create doesn't (want to) take a $gho, but the $vg and $lv names
directly (so that callers can use it when they don't have a suitable
$gho whose $gho->{Lvdev} they want to use).

In the one existing call site we pass $gho->{Vg} and $gho->{Lv} so
that the effect is the same.

There is a minor functional change: $gho->{Lvdev} has been put through
lv_dev_mapper.  But we don't care about that in lv_create (since the
LVM operations, and dd, are perfectly happy to use the `real',
non-/dev/mapper, names).  So we can just use /dev/$vg/$lv.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Signed-off-by: Robert Ho <robert.hu@intel.com>
Tested-by: Robert Ho <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v17: Discuss change to /dev/$vg/$lv from $gho->{Lvdev}.
v14: New patch
v15: Change some trivial typo, so to resolve conflicts with
     production tree.
---
 Osstest/TestSupport.pm |   15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index b70f19f..b507ada 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -63,7 +63,7 @@ BEGIN {
                       target_install_packages target_install_packages_norec
                       target_jobdir target_extract_jobdistpath_subdir
                       target_extract_jobdistpath
-                      lv_dev_mapper
+                      lv_create lv_dev_mapper
 
                       poll_loop tcpconnect await_tcp
                       contents_make_cpio file_simple_write_contents
@@ -713,6 +713,15 @@ sub poll_loop ($$$&) {
     logm("$what: ok. (${waited}s)");
 }
 
+sub lv_create ($$$$) {
+    my ($ho, $vg, $lv, $mb) = @_;
+    my $lvdev = "/dev/$vg/$lv";
+    target_cmd_root($ho, "lvremove -f $lvdev ||:");
+    target_cmd_root($ho, "lvcreate -L ${mb}M -n $lv $vg");
+    target_cmd_root($ho, "dd if=/dev/zero of=$lvdev count=10");
+    return $lvdev;
+}
+
 sub lv_dev_mapper ($$) {
     my ($vg,$lv) = @_;
     $vg =~ s/-/--/g;
@@ -1692,9 +1701,7 @@ sub prepareguest ($$$$$$) {
 
 sub prepareguest_part_lvmdisk ($$$) {
     my ($ho, $gho, $disk_mb) = @_;
-    target_cmd_root($ho, "lvremove -f $gho->{Lvdev} ||:");
-    target_cmd_root($ho, "lvcreate -L ${disk_mb}M -n $gho->{Lv} $gho->{Vg}");
-    target_cmd_root($ho, "dd if=/dev/zero of=$gho->{Lvdev} count=10");
+    lv_create($ho, $gho->{Vg}, $gho->{Lv}, $disk_mb);
 }
 
 sub make_vhd ($$$) {
-- 
1.7.10.4

[OSSTEST PATCH 11/24] Toolstack::xl: Provide block_attach method

[Ian Jackson]

It is possible that this may work some of the time with xm, so I have
taken no measures to prevent it running then.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Signed-off-by: Robert Hu <robert.hu@intel.com>
Tested-by: Robert Hu <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
----
v14: New patch
v15: Fix missing $ho assignment in block-attach method
---
 Osstest/Toolstack/xl.pm |    9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/Osstest/Toolstack/xl.pm b/Osstest/Toolstack/xl.pm
index 0f8abed..d956bcd 100644
--- a/Osstest/Toolstack/xl.pm
+++ b/Osstest/Toolstack/xl.pm
@@ -109,4 +109,13 @@ sub restore ($$$$) {
 		    ." $f", $timeout);
 }
 
+sub block_attach ($$$$) {
+    my ($self,$gho,$xldiskspec) = @_;
+    die "quotes in $xldiskspec ?" if $xldiskspec =~ m/'/;
+    my $ho = $self->{Host};
+    my $gn = $gho->{Name};
+    my $cmd = $self->{_VerboseCommand}." block-attach $gn '$xldiskspec'";
+    target_cmd_root($ho, $cmd, 100);
+}
+
 1;
-- 
1.7.10.4

[OSSTEST PATCH 12/24] sg-run-job: Break out per-host-prep and per-host-finish

[Ian Jackson]

No functional change.

We now call the per-host-ts finish steps unconditionally, rather than
only if !$need_build_host, per-host-ts is (complicated) no-op if
$need_build_host, since in that case $need_xen_hosts is {}.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Tested by: Robert Ho <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v14: Squash typo fix from Robert into this patch
---
 sg-run-job |   31 +++++++++++++++++++------------
 1 file changed, 19 insertions(+), 12 deletions(-)

diff --git a/sg-run-job b/sg-run-job
index 66145b8..884a21d 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -23,6 +23,20 @@ source ./tcl/osstestlib.tcl
 readconfig
 source-method JobDB
 
+proc per-host-prep {} {
+    per-host-ts .       host-ping-check-native/@ ts-host-ping-check
+    per-host-ts .       xen-install/@     ts-xen-install
+    per-host-ts .       xen-boot/@        ts-host-reboot
+
+    per-host-ts .       host-ping-check-xen/@ ts-host-ping-check
+    per-host-ts .       =(*)             {ts-leak-check basis}
+}
+
+proc per-host-finish {} {
+    per-host-ts .       =                {ts-leak-check check}
+    per-host-ts !broken capture-logs/@(*) ts-logs-capture
+}
+
 proc run-job {job} {
     global jobinfo builds flight ok need_xen_hosts anyfailed
 
@@ -51,22 +65,15 @@ proc run-job {job} {
     if {$ok} { setstatus running                                          }
 
     per-host-ts broken  host-install/@(*) ts-host-install-twice
-    per-host-ts .       host-ping-check-native/@ ts-host-ping-check
-    per-host-ts .       xen-install/@     ts-xen-install
-    per-host-ts .       xen-boot/@        ts-host-reboot
-    per-host-ts .       host-ping-check-xen/@ ts-host-ping-check
 
-    per-host-ts .       =(*)             {ts-leak-check basis}
+    per-host-prep
 
     if {$ok} { catching-otherwise fail      run-job/$jobinfo(recipe)      }
-    per-host-ts .       =                {ts-leak-check check}
 
-    if {!$need_build_host} {
-        per-host-ts !broken capture-logs/@(*) ts-logs-capture
-    } else {
-        if {$anyfailed} {
-            run-ts  !broken capture-logs      ts-logs-capture + host
-        }
+    per-host-finish
+
+    if {$need_build_host && $anyfailed} {
+	run-ts  !broken capture-logs      ts-logs-capture + host
     }
 
     if {$ok} { setstatus pass                                             }
-- 
1.7.10.4

[OSSTEST PATCH 13/24] sg-run-job: Provide infrastructure for layers of nesting

[Ian Jackson]

Provides nested-layer-descend, which can be called in an individual
test job at the appropriate point (after the L1 has been set up).

The inner host is a guest of the outer host; powering it off means
destroying it.  Putting the poweroff at this point in the loop, rather
than in per-host-finish, avoids powering off physical servers.  The
use of `.'  rather than `!.' for iffail means we do not power off
after failures (as we might want to preserve the state for debugging
etc).

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Tested-by: Robert Ho <robert.hu@intel.com>
Signed-off-by: Robert Ho <robert.hu@intel.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v14: Squash syntax fix from Robert Ho into this patch
v15: Remove spurious "=" from final-poweroff step invocation
---
 sg-run-job |   21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/sg-run-job b/sg-run-job
index 884a21d..172c87e 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -39,6 +39,7 @@ proc per-host-finish {} {
 
 proc run-job {job} {
     global jobinfo builds flight ok need_xen_hosts anyfailed
+    global nested_layers_hosts
 
     set ok 1
     set anyfailed 0
@@ -52,6 +53,7 @@ proc run-job {job} {
         set need_xen_hosts $nh
         set need_build_host 0
     }
+    set nested_layers_hosts {}
 
     catching-otherwise blocked            check-not-blocked
     if {!$ok} return
@@ -70,7 +72,15 @@ proc run-job {job} {
 
     if {$ok} { catching-otherwise fail      run-job/$jobinfo(recipe)      }
 
-    per-host-finish
+    while 1 {
+	per-host-finish
+	
+	if {![llength $nested_layers_hosts]} break
+
+	per-host-ts . final-poweroff {ts-host-powercycle --power=0}
+
+        set need_xen_hosts [lunappend nested_layers_hosts]
+    }
 
     if {$need_build_host && $anyfailed} {
 	run-ts  !broken capture-logs      ts-logs-capture + host
@@ -247,6 +257,15 @@ proc per-host-ts {iffail ident script args} {
     }
 }
 
+proc nested-layer-descend {nested_hosts} {
+    # We save need_xen_hosts on a stack in nested_layers_hosts
+    # It gets popped again during the cleanup part of run-job
+    global nested_layers_hosts need_xen_hosts
+    lappend nested_layers_hosts $need_xen_hosts
+    set need_xen_hosts $nested_hosts
+    per-host-prep
+}
+
 #---------- test recipes ----------
 
 proc need-hosts/test-debian-nomigr {} { return host }
-- 
1.7.10.4

[OSSTEST PATCH 14/24] Nested HVM: Provide ts-nested-setup to help make L1 usable as a host

[Ian Jackson]

From: Robert Ho <robert.hu@intel.com>

* Provide the L1 with some storage for its own guests' disks
* Install some packages in the L1
* Optionally, set a runvar defining the L1 for the rest of the job

The recipe is going to run ts-xen-install etc.

Signed-off-by: longtao.pang <longtaox.pang@intel.com>
Signed-off-by: Robert Ho <robert.hu@intel.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Tested-by: Robert Ho <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v14: Use target_check_ip (renamed, earlier, in a new patch)
     ts-nested-setup now has a default gueststorage_size of 20G
      and this is implicitly used by make-flight.
     Adjusted for new selecthost nested host syntax and correspondingly
      completely changed invocation syntax.
     Only optionally sets the runvar, if you pass --define.  (This
      will make it easier to play around with interactively.)
     Broken the pieces of work out into subroutines for clarity.
     Comment about guest storage slightly edited and rewrapped.
     Guest storage runvars and perl variable names etc. renamed to
      `gueststorage' rather than `guest_storage'.
     LVM and VG names and perl variable names changed to be clearer.
     Install `ed' too.
     Use lv_create and toolstack()->block_attach.
     Dropped ack from Ian Campbell.
---
 ts-nested-setup |   99 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 99 insertions(+)
 create mode 100755 ts-nested-setup

diff --git a/ts-nested-setup b/ts-nested-setup
new file mode 100755
index 0000000..f7e0ebd
--- /dev/null
+++ b/ts-nested-setup
@@ -0,0 +1,99 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Intel Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use Osstest::Debian;
+use Osstest::TestSupport;
+
+tsreadconfig();
+
+our $dodefine;
+if (@ARGV && $ARGV[0] eq '--define') { $dodefine=1; shift @ARGV; }
+
+our ($l1_identspec) = @ARGV;
+
+my $l1 = selecthost($l1_identspec);
+my $l0 = $l1->{Host};
+
+die unless $l0;
+
+our $def_gueststorage_size = 20000; # Mby
+
+sub packages () {
+    # Faster to install packages while L1 is still running PVHVM
+    target_install_packages_norec($l1, qw(lvm2 rsync ed genisoimage));
+}
+
+sub guest_storage () {
+    # We need to attach an extra disk to the L1 guest to be used as L2
+    # guest storage:
+    #
+    # When running in a nested HVM environment the L1 domain is acting
+    # as both a guest to L0 and a host to L2 guests and therefore
+    # potentially sees connections to two independent xenstore
+    # instances, one provided by the L0 host and one which is provided
+    # by the L1 instance of xenstore.
+    #
+    # Unfortunately the kernel is not capable of dealing with this and
+    # is only able to cope with a single xenstore connection. Since
+    # the L1 toolstack and L2 guests absolutely require xenstore to
+    # function we therefore cannot use the L0 xenstore and therefore
+    # cannot use PV devices (xvdX etc) in the L1 guest and must use
+    # emulated devices (sdX etc).
+    #
+    # However at the moment we have not yet rebooted L1 into Xen and
+    # so it does have PV devices available and sdb actually appears as
+    # xvdb.  We could disable the Xen platform device and use emulated
+    # devices for the install phase too but that would be needlessly
+    # slow.
+    # 
+    # Instead, to avoid needing to even mention the name of xvdb or
+    # sdb, we create the vg in l0.  When the l1 reboots it will
+    # automatically find the empty vg we have created for it, and
+    # target_choose_vg on l1 (which is used by all the guest creation
+    # ts-* scripts) will use it since it has plenty of space.
+
+    my $size = guest_var($l1,'gueststorage_size',$def_gueststorage_size);
+    die "gueststorage_size is undefined" unless $size;
+
+    my $outer_vg = target_choose_vg($l0, $size);
+    my $outer_lv = "$l1->{Ident}_gueststorage_outer_lv";
+    my $inner_vg = "$l1->{Ident}_gueststorage_vg";
+
+    target_cmd_root($l0, "vgremove -f $inner_vg ||:");
+    my $outer_lvdev = lv_create($l0, $outer_vg, $outer_lv, $size);
+
+    target_cmd_root($l0, <<END);
+        pvcreate $outer_lvdev
+	vgcreate $inner_vg $outer_lvdev
+END
+
+    toolstack($l0)->block_attach($l1, "$outer_lvdev,raw,sdb,rw");
+    # NB this does not update the l1 guest config so if the l1 is shut
+    # down and recreated in the l0, this will vanish.
+}
+
+packages();
+guest_storage();
+host_install_postboot_complete($l1);
+
+if ($dodefine) {
+    $l1_identspec =~ m/^(.*)=/ or die "$l1_identspec ?";
+    store_runvar($1, $'); #';
+}
-- 
1.7.10.4

[OSSTEST PATCH 15/24] Nested HVM: Provide test-nested recipe

[Ian Jackson]

From: Robert Ho <robert.hu@intel.com>

Signed-off-by: Robert Ho <robert.hu@intel.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Tested-by: Robert Ho <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v14: ts-nested-setup command line syntax updated.
v15: (Robert Ho) remove the unnecessary l1 destroy; as it will
     implicitly powered off by framework as a nested host.
     This change hasn't been confirmed by Ian Jackson yet; I
     plan to separate it as fix patch but by mistake squashed
     it in. Ian Jackson may want to revert this if he dosn't
     agree.
---
 sg-run-job |    9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/sg-run-job b/sg-run-job
index 172c87e..a2527ca 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -348,6 +348,15 @@ proc run-job/test-pair-oneway {} {
     run-ts . =              ts-guest-stop          dst_host          + debian
 }
 
+proc need-hosts/test-nested {} {return host}
+proc run-job/test-nested {} {
+    run-ts . = ts-debian-hvm-install + host l1
+    run-ts . = ts-nested-setup + --define l1=host:l1
+    nested-layer-descend l1
+    run-ts . = ts-debian-hvm-install l1 l2
+    run-ts . = ts-guest-stop l1 l2
+}
+
 proc test-guest-migr {g} {
     set to_reap [spawn-ts . = ts-migrate-support-check + host $g 1]
     set can_migrate [reap-ts $to_reap]
-- 
1.7.10.4

[OSSTEST PATCH 16/24] Nested HVM: Add test job to appropriate flights

[Ian Jackson]

From: Robert Ho <robert.hu@intel.com>

Signed-off-by: longtao.pang <longtaox.pang@intel.com>
Signed-off-by: Robert Ho <robert.hu@intel.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Tested-by: Robert Ho <robert.hu@intel.com>
---
v17: Use usual_debianhvm_image
v14: Use default gueststorage_size, rather than setting runvar.
     Dropped acked from Ian Campbell.
---
 make-flight |   28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/make-flight b/make-flight
index f908017..b59b87d 100755
--- a/make-flight
+++ b/make-flight
@@ -249,6 +249,33 @@ do_hvm_win7_x64_tests () {
             all_hostflags=$most_hostflags,hvm
 }
 
+do_hvm_debian_nested_tests () {
+  bios=$1
+
+  if [ $xenarch != amd64 -o $dom0arch != amd64 \
+      -o "x$qemuu_suffix" != "x-qemuu" ]; then
+    return
+  fi
+
+  case $xenbranch in
+    xen-3.*-testing)      return;;
+    xen-4.0-testing)      return;;
+    xen-4.1-testing)      return;;
+    xen-4.2-testing)      return;;
+    xen-4.3-testing)      return;;
+  esac
+
+  job_create_test test-$xenarch$kern-$dom0arch$qemuu_suffix-nested \
+            test-nested xl $xenarch $dom0arch $qemuu_runvar        \
+            l1_image=$(usual_debianhvm_image amd64)                \
+            l1_vifmodel='e1000'                                    \
+            l1_memsize='3072'                                      \
+            l1_enable_nestedhvm='true'                             \
+            l2_image=$(usual_debianhvm_image amd64)                \
+            bios=$bios                                             \
+            all_hostflags=$most_hostflags,hvm
+}
+
 branch_debianhvm_arch () {
   case $branch in
     xen-unstable-smoke) echo i386;;
@@ -592,6 +619,7 @@ test_matrix_do_one () {
     do_hvm_rhel6_tests
 
     do_hvm_debian_tests
+    do_hvm_debian_nested_tests seabios
 
   done # qemuu_suffix
 
-- 
1.7.10.4

[OSSTEST PATCH 17/24] ts-xen-install: Properly handle hosts without a static IP address

[Ian Jackson]

From: Robert Ho <robert.hu@intel.com>

Check IpStatic, and if it is not set, provide a dhcp stanza in
/etc/network/interfaces, rather than an `inet static' one.

This is necessary for L1 nested hosts, because they don't have a
static IP address.

In principle this makes matters more correct for physical hosts
without static IP addresses, but these are currently not supported
by selecthost().

Signed-off-by: Robert Ho <robert.hu@intel.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Tested-by: Robert Ho <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v14: Only use `inet dhcp' if !$ho->{IpStatic}.
---
 ts-xen-install |    5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/ts-xen-install b/ts-xen-install
index b511e2b..d9aa694 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -320,11 +320,14 @@ END
                 /auto $iface\n/x;
             if (m/^\s* iface \s+ (?: $physif | xenbr0 ) \s+ inet \s /x) {
                 $suppress= 1;
-                print EO <<END;
+                print EO ($ho->{IpStatic} ? <<END : <<END), <<END;
 iface $iface inet static
     address $ho->{Ip}
     netmask $netmask
     gateway $gateway
+END
+iface $iface inet dhcp
+END
 $bridgex
 END
             }
-- 
1.7.10.4

[OSSTEST PATCH 18/24] ts-xen-install: networking: Rename `nodhcp' to `ensurebridge'

[Ian Jackson]

This function does not (now) always undo the DHCP configuration.
Sometimes it leaves it.  Its main function is to ensure that we have
a bridge for use by guests.

So rename the function.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Tested-by: Robert Ho <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v14: This patch was previously 4/4 of a miniature series containing
      a different way of dealing with the Nested HVM L1 DHCP problem.
---
 ts-xen-install |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ts-xen-install b/ts-xen-install
index d9aa694..3d0f394 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -247,7 +247,7 @@ sub hosts () {
     });
 }
 
-sub nodhcp () {
+sub ensurebridge () {
     target_editfile_root($ho, "/etc/network/interfaces",
                          "etc-network-interfaces", sub {
         my $physif= get_host_property($ho,'interface force',undef);
@@ -370,6 +370,6 @@ if ($checkmode) {
     adjustconfig();
     setupboot();
     setupinitd();
-    nodhcp();
+    ensurebridge();
     hosts();
 }
-- 
1.7.10.4

[OSSTEST PATCH 19/24] Osstest/Testsupport.pm: change target's default kernkind to 'pvops'

[Ian Jackson]

This is safe only if no existing flights would be affected.  (That is,
the meaning of no existing sets of runvars would be changed.)

To check whether this would make any difference I did some database
searches.  Since any time target_kernkind_check is called it sets a
corresponding `console' runvar, I can search for `console' without a
corresponding `kernkind'.  I ran this query:

  select * from (select *, (select name from runvars r2 where
  r2.flight=r1.flight and r2.job=r1.job and r2.name=
  replace(r1.name,'console','kernkind')) kk from runvars r1 where
  r1.name like '%console') iq where kk is null order by flight desc;

and it found nothing since flight 7682.  So I think we can change the
default.

Signed-off-by: 	Ian Jackson <ian.jackson@eu.citrix.com>
Signed-off-by: 	Robert Ho <robert.hu@intel.com>
Tested-by: 	Robert Ho <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v15: New patch
---
 Osstest/TestSupport.pm |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index b507ada..cb496d2 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -2065,7 +2065,7 @@ sub target_var ($$) {
 sub target_kernkind_check ($) {
     my ($gho) = @_;
     my $pfx= target_var_prefix($gho);
-    my $kernkind= $r{$pfx."kernkind"};
+    my $kernkind= $r{$pfx."kernkind"} // 'pvops';
     my $isguest= exists $gho->{Guest};
     if ($kernkind eq 'pvops') {
         store_runvar($pfx."rootdev", 'xvda') if $isguest;
-- 
1.7.10.4

[OSSTEST PATCH 20/24] Osstest/Testsupport.pm: use get_target_property() for some host setup

[Ian Jackson]

From: Robert Ho <robert.hu@intel.com>

For nested cases, nested host can inherit its host's property for
dhcp watch setup and ether_prefix property setup.

Signed-off-by: Robert Ho <robert.hu@intel.com>
Tested-by: Robert Ho <robert.hu@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v15: New patch
---
 Osstest/TestSupport.pm |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index cb496d2..32f2956 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -735,7 +735,7 @@ sub lv_dev_mapper ($$) {
 sub dhcp_watch_setup ($$) {
     my ($ho,$gho) = @_;
 
-    my $meth = get_host_property($ho,'dhcp-watch-method',undef);
+    my $meth = get_target_property($ho,'dhcp-watch-method',undef);
     $gho->{DhcpWatch} = get_host_method_object($ho, 'DhcpWatch', $meth);
 }
 
@@ -1583,7 +1583,7 @@ sub target_choose_vg ($$) {
 
 sub ether_prefix($) {
     my ($ho) = @_;
-    my $prefix = get_host_property($ho, 'gen-ether-prefix-base');
+    my $prefix = get_target_property($ho, 'gen-ether-prefix-base');
     $prefix =~ m/^(\w+:\w+):(\w+):(\w+)$/ or die "$prefix ?";
     my $lhs = $1;
     my $pv = (hex($2)<<8) | (hex($3));
-- 
1.7.10.4

[OSSTEST PATCH 21/24] HVM guests: Use qemu "pipe:" for serial output logging

[Ian Jackson]

Modern qemu has the "pipe:/PATH" character driver.  This opens
/PATH.in for reading and /PATH.out for writing.  In my tests, I found
that:
  - contrary to the documentation, they do not need to be pipes
    (at least, /PATH.out can be a file)
  - but they must both already exist
  - qemu will follow symlinks, so /PATH.out can be a symlink to
    a file
  - if /PATH.in is a fifo, qemu will tolerate other processes opening
    it for writing, and writing things, only occasionally.  (Probably,
    qemu opens it O_RDWR; or perhaps it reopens it after EOF.)

Use this feature to achieve the following:
  - guest serial output ends up in /var/log/xen/osstest-serial-GUEST.log
    (which is already captured by ts-logs-capture) rather than
    interleaved with qemu's stderr output (in the libxl-created logfile)
  - guest serial input comes from a pipe in /root which we can open
    and write to if we want to talk to the guest

We are mostly interested in the final bullet point, because that will
allow us to send debug keys to the emulated serial port of an L1
nested HVM guest.

Looking at the source code of qemu in 4.2 and 4.6 I think the above
approach will work with all relevant qemu-xen's.

If the device model version is qemu-xen-traditional, `pipe:' is not
supported.  If device_model_version is not set, we will be using
whatever the xen.git we used defaults to.  For Xen 4.1 and earlier
that is qemu-xen-traditional, and I'm slightly loathe to break osstest
for those earlier versions.  There doesn't seem to be anything else in
the runvars that would clue us in.  So be cautious and do not use the
new feature unless device_model_version is explicitly set.

The nested tests are all -qemuu so set device_model_version.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v17: Fix fifo mode to be 600, not 700
v16: New patch
---
 Osstest/TestSupport.pm |   12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 32f2956..47fc1b1 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1884,8 +1884,18 @@ END
 	# fd pointing to mini-os's console. IOW any such path used
 	# here ends up in the host logs in /var/log/xen/qemu-dm-$guest.log
 	$cfg .= "serial='file:/var/log/dm-serial.log'\n";
-    } else {
+    } elsif (!(defined $devmodel) || $devmodel =~ /traditional/) {
 	$cfg .= "serial='file:/dev/stderr'\n";
+    } else {
+	my $logpath = "/var/log/xen/osstest-serial-$gho->{Name}.log";
+	my $basepath = "/root/$flight.$job.$gho->{Name}.serial";
+	target_cmd_root($ho, <<END);
+            set -ex
+            test -e $logpath      || touch $logpath
+            test -e $basepath.out || ln -s $logpath $basepath.out
+            test -e $basepath.in  || mkfifo -m600 $basepath.in
+END
+	$cfg .= "serial='pipe:$basepath'\n";
     }
 
     $xopts{VifType} ||= "ioemu";
-- 
1.7.10.4

[OSSTEST PATCH 22/24] Serial: Factor out Osstest::Serial::keys_real

[Ian Jackson]

The sympathy and xenuse serial modules had too much in common.  Factor
out the common code, which is now responsible for
  - knowledge of the Xen console switch
  - splitting strings up into individual keys
  - timing decisions
  - error trapping and logging

This new class is an abstract base class for the concrete serial
method classes, and calls back to its derived class to prepare, send
each actual key, and shut down.

There is some functional change: notably, after failure to send the
first debug key, sending the remainder will not be attempted.

While we're here, fix a typo `dettach' to `detach'.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v17: Move `force attach' and `force detach' writes into xenuse.pm
      where they belong.
     Add typo fix.
---
 Osstest/Serial/keys_real.pm |   65 +++++++++++++++++++++++++++++++++++++++++++
 Osstest/Serial/sympathy.pm  |   55 ++++++++++++------------------------
 Osstest/Serial/xenuse.pm    |   57 ++++++++++++++-----------------------
 3 files changed, 104 insertions(+), 73 deletions(-)
 create mode 100644 Osstest/Serial/keys_real.pm

diff --git a/Osstest/Serial/keys_real.pm b/Osstest/Serial/keys_real.pm
new file mode 100644
index 0000000..80a159d
--- /dev/null
+++ b/Osstest/Serial/keys_real.pm
@@ -0,0 +1,65 @@
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Citrix Inc.
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+# 
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+package Osstest::Serial::keys_real;
+
+# Base class providing debug keys for real serial ports.
+# Derived class is expected to provide:
+#   $mo->keys_prepare();
+#   $mo->keys_prepare($what,$str,$pause);
+#   $mo->keys_shutdown();
+
+
+use strict;
+use warnings;
+
+use Osstest::TestSupport;
+
+sub request_debug {
+    my ($mo,$conswitch,$xenkeys,$guestkeys) = @_;
+
+    if (!eval {
+	local ($SIG{'PIPE'}) = 'IGNORE';
+
+	$mo->keys_prepare();
+
+	my $debugkeys= sub {
+	    my ($what, $keys) = @_;
+	    foreach my $k (split //, $keys) {
+		$mo->keys_write("$what debug info request, debug key $k",
+				$k, 2);
+	    }
+	};
+
+	$mo->keys_write('request for input to Xen', $conswitch, 1);
+	$debugkeys->('Xen', $xenkeys);
+	sleep(10);
+	$debugkeys->('guest', $guestkeys);
+	sleep(10);
+	$mo->keys_write("RET to dom0","$conswitch\r", 5);
+
+	$mo->keys_shutdown();
+
+	1;
+    }) {
+	warn "failed to send debug key(s): $@\n";
+	return 0;
+    }
+    return 1;
+}
+
+1;
diff --git a/Osstest/Serial/sympathy.pm b/Osstest/Serial/sympathy.pm
index d6bf425..fa34143 100644
--- a/Osstest/Serial/sympathy.pm
+++ b/Osstest/Serial/sympathy.pm
@@ -22,12 +22,13 @@ use warnings;
 
 use Osstest;
 use Osstest::TestSupport;
+use Osstest::Serial::keys_real;
 
 BEGIN {
     use Exporter ();
     our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS);
     $VERSION     = 1.00;
-    @ISA         = qw(Exporter);
+    @ISA         = qw(Exporter Osstest::Serial::keys_real);
     @EXPORT      = qw();
     %EXPORT_TAGS = ( );
 
@@ -57,48 +58,28 @@ sub new {
     return bless $mo, $class;
 }
 
-sub request_debug {
-    my ($mo,$conswitch,$xenkeys,$guestkeys) = @_;
+sub keys_prepare {
+}
 
-    my $targhost= $mo->{Server};
+sub keys_write {
+    my ($mo, $what,$str,$pause) = @_;
 
+    my $targhost= $mo->{Server};
     my ($sshopts) = sshopts();
-    my $sympwrite= sub {
-        my ($what,$str,$pause) = @_;
-        logm("sympathy sending $what");
-        if (!eval {
-            local ($SIG{'PIPE'}) = 'IGNORE';
-            my $sock= $mo->{Socket};
-            my $rcmd= "sympathy -c -k $sock -N >/dev/null";
-            $rcmd= "alarm 5 $rcmd";
-            open SYMPWRITE, "|ssh @$sshopts root\@$targhost '$rcmd'" or die $!;
-            autoflush SYMPWRITE 1;
-            print SYMPWRITE $str or die $!;
-            sleep($pause);
-            close SYMPWRITE or die "$? $!";
-            1;
-        }) {
-            warn "failed to send $what: $@\n";
-            return 0;
-        }
-        return 1;
-    };
 
-    my $debugkeys= sub {
-	my ($what, $keys) = @_;
-	foreach my $k (split //, $keys) {
-	    $sympwrite->("$what debug info request, debug key $k", $k, 2);
-	}
-    };
+    logm("sympathy sending $what");
 
-    $sympwrite->('request for input to Xen', $conswitch, 1);
-    $debugkeys->('Xen', $xenkeys);
-    sleep(10);
-    $debugkeys->('guest', $guestkeys);
-    sleep(10);
-    $sympwrite->("RET to dom0","$conswitch\r", 5);
+    my $sock= $mo->{Socket};
+    my $rcmd= "sympathy -c -k $sock -N >/dev/null";
+    $rcmd= "alarm 5 $rcmd";
+    open SYMPWRITE, "|ssh @$sshopts root\@$targhost '$rcmd'" or die $!;
+    autoflush SYMPWRITE 1;
+    print SYMPWRITE $str or die $!;
+    sleep($pause);
+    close SYMPWRITE or die "$? $!";
+}
 
-    return 1;
+sub keys_shutdown {
 }
 
 sub fetch_logs {
diff --git a/Osstest/Serial/xenuse.pm b/Osstest/Serial/xenuse.pm
index 6b9d955..e1270e0 100644
--- a/Osstest/Serial/xenuse.pm
+++ b/Osstest/Serial/xenuse.pm
@@ -23,6 +23,7 @@ use warnings;
 
 use Osstest;
 use Osstest::TestSupport;
+use Osstest::Serial::keys_real;
 
 use File::Temp;
 use File::Copy;
@@ -31,7 +32,7 @@ BEGIN {
     use Exporter ();
     our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS);
     $VERSION     = 1.00;
-    @ISA         = qw(Exporter);
+    @ISA         = qw(Exporter Osstest::Serial::keys_real);
     @EXPORT      = qw();
     %EXPORT_TAGS = ( );
 
@@ -46,51 +47,35 @@ sub new {
     return bless $mo, $class;
 }
 
-sub request_debug {
-    my ($mo,$conswitch,$xenkeys,$guestkeys) = @_;
-    my $xenuse= $c{XenUsePath} || "xenuse";
+sub keys_prepare {
+    my ($mo) = @_;
 
     my $ho= $mo->{Host};
 
-    my $writer= sub {
-        my ($what,$str,$pause) = @_;
-        logm("xenuse sending $what");
-        if (!eval {
-            print XENUSEWRITE $str or die $!;
-            sleep($pause);
-            1;
-        }) {
-            warn "failed to send $what: $@\n";
-            return 0;
-        }
-        return 1;
-    };
-
-    my $debugkeys= sub {
-	my ($what, $keys) = @_;
-	foreach my $k (split //, $keys) {
-	    $writer->("$what debug info request, debug key $k", $k, 2);
-	}
-    };
-
-    local ($SIG{'PIPE'}) = 'IGNORE';
+    my $xenuse= $c{XenUsePath} || "xenuse";
+
     open XENUSEWRITE, "|$xenuse -t $ho->{Name}" or die $!;
     autoflush XENUSEWRITE 1;
 
-    $writer->('force attach', "\x05cf", 1); # ^E c f == force attach
+    $mo->keys_write('force attach', "\x05cf", 1); # ^E c f == force attach
 
-    $writer->('request for input to Xen', $conswitch, 1);
-    $debugkeys->('Xen', $xenkeys);
-    sleep(10);
-    $debugkeys->('guest', $guestkeys);
-    sleep(10);
-    $writer->("RET to dom0","$conswitch\r", 5);
+    sleep 5;
+}
 
-    $writer->('dettach', "\x05c.", 1); # ^E c . == disconnect
+sub keys_write {
+    my ($mo, $what,$str,$pause) = @_;
+    logm("xenuse sending $what");
 
-    close XENUSEWRITE or die "$? $!";
+    print XENUSEWRITE $str or die $!;
+    sleep($pause);
+}
+
+sub keys_shutdown {
+    my ($mo) = @_;
 
-    return 1;
+    $mo->keys_write('dettach', "\x05c.", 1); # ^E c . == disconnect
+
+    close XENUSEWRITE or die "$? $!";
 }
 
 sub fetch_logs {
-- 
1.7.10.4

[OSSTEST PATCH 23/24] Serial: Add new serial method object for `guest' type

[Ian Jackson]

From: Robert Ho <robert.hu@intel.com>

L1 guests' serial ports are owned by qemu in L0.  We can send them
debug keys by writing to the qemu pipe.

(xl debug-key looks like it would be useful but it actually sends
debug keys to the hypervisor of the host it is running on.  We want to
send the debug keys to the hypervisor and kernel from the outside.)

Log fetching is not needed because from the POV of the L0 the L1 is a
guest, so the L0's log capture will already fetch the L1's serial
console output.

Signed-off-by: Robert Ho <robert.hu@intel.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>

---
v17: Get FIFO path right.
     Log the ssh command.
     Put quotes around the redirection, as required.
     Export sshuho from TestSupport.
     Make ::guest->fetch_logs actually use @_ so that it works.
v16: Mostly rewritten: now uses new keys_real base class, and
     uses the qemu pipe rather than xl debug-keys.
v15: New patch.
---
 Osstest/Serial/guest.pm |   96 +++++++++++++++++++++++++++++++++++++++++++++++
 Osstest/TestSupport.pm  |    4 +-
 2 files changed, 98 insertions(+), 2 deletions(-)
 create mode 100644 Osstest/Serial/guest.pm

diff --git a/Osstest/Serial/guest.pm b/Osstest/Serial/guest.pm
new file mode 100644
index 0000000..2511556
--- /dev/null
+++ b/Osstest/Serial/guest.pm
@@ -0,0 +1,96 @@
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2015 Intel Inc.
+# Copyright (C) 2015 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Send debug keys to nested host (L1).
+
+package Osstest::Serial::guest;
+
+use strict;
+use warnings;
+
+use Osstest;
+use Osstest::TestSupport;
+use Osstest::Serial::keys_real;
+
+BEGIN {
+    use Exporter ();
+    our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS);
+    $VERSION     = 1.00;
+    @ISA         = qw(Exporter Osstest::Serial::keys_real);
+    @EXPORT      = qw();
+    %EXPORT_TAGS = ( );
+
+    @EXPORT_OK   = qw();
+}
+
+sub new {
+    my ($class, $l1ho, $methname, @args) = @_;
+    my $mo = { Target => $l1ho, Parent => $l1ho->{Host} };
+
+    logm("serial method $methname $mo->{Target}{Name}: @args");
+    return bless $mo, $class;
+}
+
+sub keys_prepare {
+    my ($mo) = @_;
+
+    my $gho = $mo->{Target};
+    my $pho = $mo->{Parent};
+    my $puho = sshuho('root',$pho);
+    my $domname = $r{"$gho->{Guest}_domname"};
+
+    my $fifo = "/root/$flight.$job.$domname.serial.in";
+
+    # NB this by-hand construction and execution of an
+    # ssh command line bypasses the usual timeout arrangements.
+    my ($sshopts) = sshopts();
+    my $cmd = "ssh @$sshopts $puho 'cat >$fifo'";
+    logm("spawning $cmd");
+
+    # timeouts: open will carry on regardless even if the command hangs
+    open SERIALWRITE, "|$cmd" or die $!;
+    autoflush SERIALWRITE 1;
+}
+
+sub keys_write {
+    my ($mo, $what,$str,$pause) = @_;
+    logm("xenuse sending $what");
+
+    # timeouts: we are going to write much less than any plausible
+    # PIPE_MAX so there is no risk that we will block on write
+    print SERIALWRITE $str or die $!;
+    sleep($pause);
+}
+
+sub keys_shutdown {
+    my ($mo) = @_;
+
+    # timeouts: close waits for the child to exit, so set an alarm
+    alarm(15);
+    $!=0; $?=0; close SERIALWRITE or die "$? $!";
+    alarm(0);
+}
+
+sub fetch_logs {
+    my ($mo) = @_;
+
+    logm("$mo->{Target}{Name} (nested host) serial console logs".
+	 " will be found in guest logs from $mo->{Parent}{Name} (parent)");
+    return;
+}
+
+1;
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 47fc1b1..6be50e3 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -50,7 +50,7 @@ BEGIN {
 
                       target_cmd_root target_cmd target_cmd_build
                       target_cmd_output_root target_cmd_output
-                      target_cmd_inputfh_root
+                      target_cmd_inputfh_root sshuho
                       target_getfile target_getfile_root
                       target_putfile target_putfile_root
                       target_putfilecontents_stash
@@ -874,7 +874,7 @@ sub selecthost ($) {
 	$child->{Power} = 'guest';
 	power_cycle_host_setup($child);
 
-	$child->{Properties}{Serial} = 'noop'; # todo
+	$child->{Properties}{Serial} = 'guest';
 	serial_host_setup($child);
 
 	my $msg = "L$child->{NestingLevel} host $child->{Ident}:";
-- 
1.7.10.4

[OSSTEST PATCH 24/24] Serial::xenuse: Send xenuse output to /dev/null

[Ian Jackson]

Like sympathy, attaching via xenuse causes xenuse to send output from
the host to its own stdout.

But we don't want the ts-logs-capture stdout to contain this serial
output, interleaved with its own log messages.  We'll capture the
whole serial log from the xenuse logfile.  So redirect it to /dev/null.

I have checked that xenuse does (at least sometimes, eg when given a
nonexistent hostname) use stderr when something actually goes wrong.

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Tested-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
v17: New patch
---
 Osstest/Serial/xenuse.pm |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Serial/xenuse.pm b/Osstest/Serial/xenuse.pm
index e1270e0..c79d986 100644
--- a/Osstest/Serial/xenuse.pm
+++ b/Osstest/Serial/xenuse.pm
@@ -54,7 +54,7 @@ sub keys_prepare {
 
     my $xenuse= $c{XenUsePath} || "xenuse";
 
-    open XENUSEWRITE, "|$xenuse -t $ho->{Name}" or die $!;
+    open XENUSEWRITE, "|$xenuse -t $ho->{Name} >/dev/null" or die $!;
     autoflush XENUSEWRITE 1;
 
     $mo->keys_write('force attach', "\x05cf", 1); # ^E c f == force attach
-- 
1.7.10.4

Re: [OSSTEST PATCH 02/24] cs-adjust-flight: Allow adjusting "this" flight

[Ian Campbell]

On Fri, 2015-11-13 at 12:03 +0000, Ian Jackson wrote:
> This allows cs-adjust-flight to be run by hand to adjust runvars, in a
> flight being used with hand-invocation of ./ts-* scripts.
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
> Tested-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>

Re: [OSSTEST PATCH 04/24] make-flight: Break out usual_debianhvm_image and honour DEBIAN_IMAGE_VERSION

[Ian Campbell]

On Fri, 2015-11-13 at 12:03 +0000, Ian Jackson wrote:
> No functional change.  (Verified for xen-unstable with
> standalone-generate-dump-flight-runvars.)
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
> Tested-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [OSSTEST PATCH 23/24] Serial: Add new serial method object for `guest' type

[Ian Campbell]

On Fri, 2015-11-13 at 12:03 +0000, Ian Jackson wrote:
> From: Robert Ho <robert.hu@intel.com>
> 
> L1 guests' serial ports are owned by qemu in L0.  We can send them
> debug keys by writing to the qemu pipe.
> 
> (xl debug-key looks like it would be useful but it actually sends
> debug keys to the hypervisor of the host it is running on.  We want to
> send the debug keys to the hypervisor and kernel from the outside.)
> 
> Log fetching is not needed because from the POV of the L0 the L1 is a
> guest, so the L0's log capture will already fetch the L1's serial
> console output.
> 
> Signed-off-by: Robert Ho <robert.hu@intel.com>
> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [OSSTEST PATCH 24/24] Serial::xenuse: Send xenuse output to /dev/null

[Ian Campbell]

On Fri, 2015-11-13 at 12:03 +0000, Ian Jackson wrote:
> Like sympathy, attaching via xenuse causes xenuse to send output from
> the host to its own stdout.
> 
> But we don't want the ts-logs-capture stdout to contain this serial
> output, interleaved with its own log messages.  We'll capture the
> whole serial log from the xenuse logfile.  So redirect it to /dev/null.
> 
> I have checked that xenuse does (at least sometimes, eg when given a
> nonexistent hostname) use stderr when something actually goes wrong.
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
> Tested-by: Ian Jackson <Ian.Jackson@eu.citrix.com>

Acked-by: Ian Campbell <ian.campbell@citrix.com>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel