* [OSSTEST v6 01/24] TestSupport: Add helper to fetch a URL on a host
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-12 15:08 ` Ian Jackson
2015-05-06 9:56 ` [OSSTEST v6 02/24] TestSupport: allow caller of prepareguest_part_xencfg to specify viftype Ian Campbell
` (24 subsequent siblings)
25 siblings, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v5: Support http_proxy via $c{HttpProxy}
v3: Make sure wget is installed
---
Osstest/Debian.pm | 2 +-
Osstest/TestSupport.pm | 10 +++++++++-
2 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index a577d1d..acdaaa8 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -617,7 +617,7 @@ d-i apt-setup/another boolean false
d-i apt-setup/non-free boolean false
d-i apt-setup/contrib boolean false
-d-i pkgsel/include string openssh-server, ntp, ntpdate, ethtool, chiark-utils-bin, $extra_packages
+d-i pkgsel/include string openssh-server, ntp, ntpdate, ethtool, chiark-utils-bin, wget, $extra_packages
$xopts{ExtraPreseed}
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 1053016..3621bf8 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -55,7 +55,7 @@ BEGIN {
target_putfilecontents_stash
target_putfilecontents_root_stash
target_put_guest_image target_editfile
- target_editfile_cancel
+ target_editfile_cancel target_fetchurl
target_editfile_root target_file_exists
target_run_apt
target_install_packages target_install_packages_norec
@@ -1560,6 +1560,14 @@ END
return $cfgpath;
}
+sub target_fetchurl($$$;$) {
+ my ($ho, $url, $path, $timeo) = @_;
+ $timeo ||= 2000;
+ my $useproxy = "export http_proxy=$c{HttpProxy};" if $c{HttpProxy};
+ target_cmd_root($ho, "$useproxy wget --progress=dot:mega -O $path $url", $timeo);
+}
+
+
sub target_put_guest_image ($$$) {
my ($ho, $gho, $default) = @_;
my $specimage = $r{"$gho->{Guest}_image"};
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 01/24] TestSupport: Add helper to fetch a URL on a host
2015-05-06 9:56 ` [OSSTEST v6 01/24] TestSupport: Add helper to fetch a URL on a host Ian Campbell
@ 2015-05-12 15:08 ` Ian Jackson
2015-05-12 15:17 ` Ian Campbell
0 siblings, 1 reply; 53+ messages in thread
From: Ian Jackson @ 2015-05-12 15:08 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("[OSSTEST v6 01/24] TestSupport: Add helper to fetch a URL on a host"):
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> +sub target_fetchurl($$$;$) {
> + my ($ho, $url, $path, $timeo) = @_;
> + $timeo ||= 2000;
> + my $useproxy = "export http_proxy=$c{HttpProxy};" if $c{HttpProxy};
> + target_cmd_root($ho, "$useproxy wget --progress=dot:mega -O $path $url", $timeo);
Do we care that this will break badly if the url contains shell
metacharacters ? I think we may do.
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 01/24] TestSupport: Add helper to fetch a URL on a host
2015-05-12 15:08 ` Ian Jackson
@ 2015-05-12 15:17 ` Ian Campbell
0 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-12 15:17 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
On Tue, 2015-05-12 at 16:08 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[OSSTEST v6 01/24] TestSupport: Add helper to fetch a URL on a host"):
> > Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
>
> > +sub target_fetchurl($$$;$) {
> > + my ($ho, $url, $path, $timeo) = @_;
> > + $timeo ||= 2000;
> > + my $useproxy = "export http_proxy=$c{HttpProxy};" if $c{HttpProxy};
> > + target_cmd_root($ho, "$useproxy wget --progress=dot:mega -O $path $url", $timeo);
>
> Do we care that this will break badly if the url contains shell
> metacharacters ? I think we may do.
I agree.
Is adding \" around the $url is all which is needed there, or are there
enough layers of quoting that it is more complicated than that?
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread
* [OSSTEST v6 02/24] TestSupport: allow caller of prepareguest_part_xencfg to specify viftype
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 01/24] TestSupport: Add helper to fetch a URL on a host Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 03/24] create_webfile: Support use with guests as well as hosts Ian Campbell
` (23 subsequent siblings)
25 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
--
v3: Use CamelCase for xopts, use the correct condition
---
Osstest/TestSupport.pm | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 3621bf8..b6fc09a 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1533,12 +1533,13 @@ sub prepareguest_part_xencfg ($$$$$) {
my $onpoweroff= $xopts->{OnPowerOff} || 'destroy';
my $oncrash= $xopts->{OnCrash} || 'preserve';
my $vcpus= guest_var($gho, 'vcpus', $xopts->{DefVcpus} || 2);
+ my $viftype= $xopts->{VifType} ? "type=$xopts->{VifType}," : "";
my $xoptcfg= $xopts->{ExtraConfig};
$xoptcfg='' unless defined $xoptcfg;
my $xencfg= <<END;
name = '$gho->{Name}'
memory = ${ram_mb}
-vif = [ 'type=ioemu,mac=$gho->{Ether}' ]
+vif = [ '${viftype}mac=$gho->{Ether}' ]
#
on_poweroff = '$onpoweroff'
on_reboot = '$onreboot'
@@ -1634,6 +1635,7 @@ END
$cfg .= "bios='$bios'\n";
}
+ $xopts{VifType} ||= "ioemu";
my $cfgpath= prepareguest_part_xencfg($ho, $gho, $ram_mb, \%xopts, $cfg);
target_cmd_root($ho, <<END);
(echo $passwd; echo $passwd) | vncpasswd $gho->{Guest}.vncpw
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 03/24] create_webfile: Support use with guests as well as hosts.
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 01/24] TestSupport: Add helper to fetch a URL on a host Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 02/24] TestSupport: allow caller of prepareguest_part_xencfg to specify viftype Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 04/24] Debian: refactor code to add preseed commands to the preseed file Ian Campbell
` (22 subsequent siblings)
25 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
In particular make the path unique by ensuring it includes the host
and guest name in the guest case.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v4: Dropped redundant -> from $ho->{Host}->{Name}.
v3: New patch
---
Osstest/TestSupport.pm | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index b6fc09a..8d6b349 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1980,6 +1980,8 @@ sub await_webspace_fetch_byleaf ($$$$$) {
sub create_webfile ($$$) {
my ($ho, $tail, $contents) = @_; # $contents as for file_link_contents
my $wf_rhs= $ho->{Name}."_".$tail;
+ # $ho->{Host} is set if $ho is a guest.
+ $wf_rhs= $ho->{Host}{Name}."_${wf_rhs}" if $ho->{Host};
my $wf_common= $c{WebspaceCommon}.$wf_rhs;
my $wf_url= $c{WebspaceUrl}.$wf_common;
my $wf_file= $c{WebspaceFile}.$wf_common;
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 04/24] Debian: refactor code to add preseed commands to the preseed file
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (2 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 03/24] create_webfile: Support use with guests as well as hosts Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-12 15:11 ` Ian Jackson
2015-05-06 9:56 ` [OSSTEST v6 05/24] Debian: refactor preseeding of .ssh directories Ian Campbell
` (21 subsequent siblings)
25 siblings, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: longtaox.pang, Ian Campbell, xen-devel
Call it from ts-debian-hvm-install.
This means that, in future, ts-debian-di-install can use
preseed_hook_command and preseed_hook_installscript.
The existing opencoded use of d-i/late_command in the guest preseed
needs to become a preseed_hook_command so as not to clash with the use
of preseed_hook_cmds().
This requires also adding a #! line and the "set -ex" boilerplate
which in turn requires slightly rewriting the /boot/EFI handling part
to also work if the system is not installed for EFI (in which case
grubx64.efi isn't installed). Previously this would have needlessly
created the directory and then ignored the error from cp.
The ssh authorized keys bit isn't touched since it works as is and
will go away in a subsequent patch.
Apart from no longer creating /target/boot/efi/EFI/boot when it is not
needed there is no functional change for now.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: longtaox.pang@intel.com
---
v6: Move existing d-i/late_command preseed to use preseed_hook_command
v5: Extended commit message
v3: New patch
---
Osstest/Debian.pm | 16 +++++++++++-----
ts-debian-hvm-install | 20 +++++++++++++++-----
2 files changed, 26 insertions(+), 10 deletions(-)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index acdaaa8..0656ba4 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -37,7 +37,7 @@ BEGIN {
%preseed_cmds
preseed_base
preseed_create
- preseed_hook_command preseed_hook_installscript
+ preseed_hook_command preseed_hook_installscript preseed_hook_cmds
di_installcmdline_core
);
%EXPORT_TAGS = ( );
@@ -913,10 +913,7 @@ d-i partman-auto/expert_recipe string \\
END
- foreach my $di_key (keys %preseed_cmds) {
- $preseed_file .= "d-i preseed/$di_key string ".
- (join ' && ', @{ $preseed_cmds{$di_key} }). "\n";
- }
+ $preseed_file .= preseed_hook_cmds();
if ($ho->{Flags}{'no-di-kernel'}) {
$preseed_file .= <<END;
@@ -960,4 +957,13 @@ chmod +x '$installer_pathname'
END
}
+sub preseed_hook_cmds () {
+ my $preseed;
+ foreach my $di_key (keys %preseed_cmds) {
+ $preseed .= "d-i preseed/$di_key string ".
+ (join ' && ', @{ $preseed_cmds{$di_key} }). "\n";
+ }
+ return $preseed;
+}
+
1;
diff --git a/ts-debian-hvm-install b/ts-debian-hvm-install
index cfd5144..c5e094c 100755
--- a/ts-debian-hvm-install
+++ b/ts-debian-hvm-install
@@ -70,13 +70,23 @@ d-i partman-auto/expert_recipe string \\
.
d-i apt-setup/cdrom/set-first boolean false
+END
-d-i preseed/late_command string \\
- in-target mkdir -p /boot/efi/EFI/boot; \\
- in-target cp /boot/efi/EFI/debian/grubx64.efi /boot/efi/EFI/boot/bootx64.efi ;\\
- in-target mkdir -p /root/.ssh; \\
- in-target sh -c "echo -e '$authkeys'> /root/.ssh/authorized_keys";
+preseed_hook_command($gho, 'late_command', '', <<END);
+#!/bin/sh
+set -ex
+
+b=/target/boot/efi/EFI
+if [ -e \$b/debian/grubx64.efi ] ; then
+ mkdir -p \$b/boot
+ cp \$b/debian/grubx64.efi \$b/boot/bootx64.efi
+fi
+in-target mkdir -p /root/.ssh
+in-target sh -c "echo -e '$authkeys'> /root/.ssh/authorized_keys"
END
+
+ $preseed_file .= preseed_hook_cmds();
+
return $preseed_file;
}
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 05/24] Debian: refactor preseeding of .ssh directories
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (3 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 04/24] Debian: refactor code to add preseed commands to the preseed file Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-12 15:13 ` Ian Jackson
2015-05-06 9:56 ` [OSSTEST v6 06/24] Debian: Refactor installation of overlays, so it can be used for guests too Ian Campbell
` (20 subsequent siblings)
25 siblings, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: longtaox.pang, Ian Campbell, xen-devel
Causes known_hosts to be consistently created as well as ~osstest/.ssh
to be consistently populated (it previsouly wasn't for HVM guests).
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: longtaox.pang@intel.com
---
v5: Call preseed_ssh from preseed_base
v3: New patch
---
Osstest/Debian.pm | 117 ++++++++++++++++++++++++++++----------------------
ts-debian-hvm-install | 5 +--
2 files changed, 67 insertions(+), 55 deletions(-)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 0656ba4..56223f4 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -37,6 +37,7 @@ BEGIN {
%preseed_cmds
preseed_base
preseed_create
+ preseed_ssh
preseed_hook_command preseed_hook_installscript preseed_hook_cmds
di_installcmdline_core
);
@@ -551,8 +552,70 @@ sub di_installcmdline_core ($$;@) {
return @cl;
}
-sub preseed_base ($$$;@) {
- my ($ho,$suite,$extra_packages,%xopts) = @_;
+sub preseed_ssh ($$) {
+ my ($ho,$sfx) = @_;
+
+ my $authkeys_url= create_webfile($ho, "authkeys$sfx", authorized_keys());
+
+ my $hostkeyfile= "$c{OverlayLocal}/etc/ssh/ssh_host_rsa_key.pub";
+ my $hostkey= get_filecontents($hostkeyfile);
+ chomp($hostkey); $hostkey.="\n";
+ my $knownhosts= '';
+
+ my $hostsq= $dbh_tests->prepare(<<END);
+ SELECT val FROM runvars
+ WHERE flight=? AND name LIKE '%host'
+ GROUP BY val
+END
+ $hostsq->execute($flight);
+ while (my ($node) = $hostsq->fetchrow_array()) {
+ my $defaultfqdn = $node;
+ $defaultfqdn .= ".$c{TestHostDomain}" unless $defaultfqdn =~ m/\./;
+
+ my %props;
+ $mhostdb->get_properties($node, \%props);
+
+ my $longname= $props{Fqdn} // $defaultfqdn;
+ my (@hostent)= gethostbyname($longname);
+ if (!@hostent) {
+ logm("skipping host key for nonexistent host $longname");
+ next;
+ }
+ my $specs= join ',', $longname, $node, map {
+ join '.', unpack 'W4', $_;
+ } @hostent[4..$#hostent];
+ logm("adding host key for $specs");
+ $knownhosts.= "$specs ".$hostkey;
+ }
+ $hostsq->finish();
+
+ $knownhosts.= "localhost,127.0.0.1 ".$hostkey;
+ my $knownhosts_url= create_webfile($ho, "known_hosts$sfx", $knownhosts);
+
+ preseed_hook_command($ho, 'late_command', $sfx, <<END);
+#!/bin/sh
+set -ex
+
+r=/target/root
+cd \$r
+
+umask 022
+mkdir .ssh
+wget -O .ssh/authorized_keys '$authkeys_url'
+wget -O .ssh/known_hosts '$knownhosts_url'
+
+u=osstest
+h=/home/\$u
+mkdir /target\$h/.ssh
+cp .ssh/authorized_keys /target\$h/.ssh
+chroot /target chown -R \$u.\$u \$h/.ssh
+END
+}
+
+sub preseed_base ($$$$;@) {
+ my ($ho,$suite,$sfx,$extra_packages,%xopts) = @_;
+
+ preseed_ssh($ho, $sfx);
my $preseed = <<"END";
d-i mirror/suite string $suite
@@ -639,49 +702,12 @@ END
sub preseed_create ($$;@) {
my ($ho, $sfx, %xopts) = @_;
- my $authkeys_url= create_webfile($ho, "authkeys$sfx", authorized_keys());
-
- my $hostkeyfile= "$c{OverlayLocal}/etc/ssh/ssh_host_rsa_key.pub";
- my $hostkey= get_filecontents($hostkeyfile);
- chomp($hostkey); $hostkey.="\n";
- my $knownhosts= '';
-
my $disk= $xopts{DiskDevice} || '/dev/sda';
my $suite= $xopts{Suite} || $c{DebianSuite};
my $d_i= $ho->{Tftp}{Path}.'/'.$ho->{Tftp}{DiBase}.'/'.$r{arch}.'/'.
$c{TftpDiVersion}.'-'.$ho->{Suite};
- my $hostsq= $dbh_tests->prepare(<<END);
- SELECT val FROM runvars
- WHERE flight=? AND name LIKE '%host'
- GROUP BY val
-END
- $hostsq->execute($flight);
- while (my ($node) = $hostsq->fetchrow_array()) {
- my $defaultfqdn = $node;
- $defaultfqdn .= ".$c{TestHostDomain}" unless $defaultfqdn =~ m/\./;
-
- my %props;
- $mhostdb->get_properties($node, \%props);
-
- my $longname= $props{Fqdn} // $defaultfqdn;
- my (@hostent)= gethostbyname($longname);
- if (!@hostent) {
- logm("skipping host key for nonexistent host $longname");
- next;
- }
- my $specs= join ',', $longname, $node, map {
- join '.', unpack 'W4', $_;
- } @hostent[4..$#hostent];
- logm("adding host key for $specs");
- $knownhosts.= "$specs ".$hostkey;
- }
- $hostsq->finish();
-
- $knownhosts.= "localhost,127.0.0.1 ".$hostkey;
- my $knownhosts_url= create_webfile($ho, "known_hosts$sfx", $knownhosts);
-
my $overlays= '';
my $create_overlay= sub {
my ($srcdir, $tfilename) = @_;
@@ -753,17 +779,6 @@ set -ex
r=/target/root
cd \$r
-umask 022
-mkdir .ssh
-wget -O .ssh/authorized_keys '$authkeys_url'
-wget -O .ssh/known_hosts '$knownhosts_url'
-
-u=osstest
-h=/home/\$u
-mkdir /target\$h/.ssh
-cp .ssh/authorized_keys /target\$h/.ssh
-chroot /target chown -R \$u.\$u \$h/.ssh
-
echo FANCYTTY=0 >> /target/etc/lsb-base-logging.sh
$overlays
@@ -878,7 +893,7 @@ END
my $extra_packages = join(",",@extra_packages);
- my $preseed_file= preseed_base($ho,$suite,$extra_packages,%xopts);
+ my $preseed_file= preseed_base($ho,$suite,$sfx,$extra_packages,%xopts);
$preseed_file .= (<<END);
d-i partman-auto/method string lvm
diff --git a/ts-debian-hvm-install b/ts-debian-hvm-install
index c5e094c..b37f0f9 100755
--- a/ts-debian-hvm-install
+++ b/ts-debian-hvm-install
@@ -43,8 +43,7 @@ our $gho;
sub preseed () {
- my $preseed_file = preseed_base($gho,'wheezy','',());
- my $authkeys = join('\\n', split(/\n/, authorized_keys()));
+ my $preseed_file = preseed_base($gho,'wheezy','','',());
$preseed_file .= (<<END);
d-i netcfg/get_hostname string $gn
@@ -81,8 +80,6 @@ if [ -e \$b/debian/grubx64.efi ] ; then
mkdir -p \$b/boot
cp \$b/debian/grubx64.efi \$b/boot/bootx64.efi
fi
-in-target mkdir -p /root/.ssh
-in-target sh -c "echo -e '$authkeys'> /root/.ssh/authorized_keys"
END
$preseed_file .= preseed_hook_cmds();
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 06/24] Debian: Refactor installation of overlays, so it can be used for guests too
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (4 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 05/24] Debian: refactor preseeding of .ssh directories Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 07/24] Debian: add preseed_create_guest helper Ian Campbell
` (19 subsequent siblings)
25 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: longtaox.pang, Ian Campbell, xen-devel
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: longtaox.pang@intel.com
---
v5: OverLay local now handled preseed_base since all callers called it
v3: New patch
fixup: Move OverlayLocal to preseed_base
---
Osstest/Debian.pm | 58 ++++++++++++++++++++++++++++---------------------------
1 file changed, 30 insertions(+), 28 deletions(-)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 56223f4..5ca93db 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -38,7 +38,9 @@ BEGIN {
preseed_base
preseed_create
preseed_ssh
- preseed_hook_command preseed_hook_installscript preseed_hook_cmds
+ preseed_hook_command preseed_hook_installscript
+ preseed_hook_overlay
+ preseed_hook_cmds
di_installcmdline_core
);
%EXPORT_TAGS = ( );
@@ -616,6 +618,7 @@ sub preseed_base ($$$$;@) {
my ($ho,$suite,$sfx,$extra_packages,%xopts) = @_;
preseed_ssh($ho, $sfx);
+ preseed_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar');
my $preseed = <<"END";
d-i mirror/suite string $suite
@@ -708,26 +711,6 @@ sub preseed_create ($$;@) {
my $d_i= $ho->{Tftp}{Path}.'/'.$ho->{Tftp}{DiBase}.'/'.$r{arch}.'/'.
$c{TftpDiVersion}.'-'.$ho->{Suite};
- my $overlays= '';
- my $create_overlay= sub {
- my ($srcdir, $tfilename) = @_;
- my $url= create_webfile($ho, "$tfilename$sfx", sub {
- my ($fh) = @_;
- contents_make_cpio($fh, 'ustar', $srcdir);
- });
- $overlays .= <<END;
-wget -O overlay.tar '$url'
-cd /target
-tar xf \$r/overlay.tar
-cd \$r
-rm overlay.tar
-
-END
- };
-
- $create_overlay->('overlay', 'overlay.tar');
- $create_overlay->($c{OverlayLocal}, 'overlay-local.tar');
-
preseed_hook_installscript($ho, $sfx,
'/lib/partman/init.d', '000override-parted-devices', <<END);
#!/bin/sh
@@ -772,18 +755,13 @@ ls -l /dev/sd*
true
END
+ preseed_hook_overlay($ho, $sfx, 'overlay', 'overlay.tar');
+
preseed_hook_command($ho, 'late_command', $sfx, <<END);
#!/bin/sh
set -ex
-r=/target/root
-cd \$r
-
echo FANCYTTY=0 >> /target/etc/lsb-base-logging.sh
-
-$overlays
-
-echo latecmd done.
END
my $dtbs = "$d_i/dtbs.tar.gz";
@@ -972,6 +950,30 @@ chmod +x '$installer_pathname'
END
}
+sub preseed_hook_overlay ($$$$) {
+ my ($ho, $sfx, $srcdir, $tfilename) = @_;
+ my $url= create_webfile($ho, "$tfilename$sfx", sub {
+ my ($fh) = @_;
+ contents_make_cpio($fh, 'ustar', $srcdir);
+ });
+ preseed_hook_command($ho, 'late_command', $sfx, <<END);
+#!/bin/sh
+set -ex
+
+r=/target/root
+cd \$r
+
+umask 022
+
+wget -O overlay.tar '$url'
+cd /target
+tar xf \$r/overlay.tar
+cd \$r
+rm overlay.tar
+
+END
+}
+
sub preseed_hook_cmds () {
my $preseed;
foreach my $di_key (keys %preseed_cmds) {
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 07/24] Debian: add preseed_create_guest helper
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (5 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 06/24] Debian: Refactor installation of overlays, so it can be used for guests too Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-12 15:16 ` Ian Jackson
2015-05-06 9:56 ` [OSSTEST v6 08/24] ts-debian-hvm-install: Only apply EFI workaround for Wheezy Ian Campbell
` (18 subsequent siblings)
25 siblings, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Creates a preseed file suitable for use in a PV guest
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v5: preseed_ssh now called from preseed_base, so not here
OverlayLocal is now called from preseed base, so not here
No need to set a default for $extra_packages in preseed_base,
everyone passes something.
v4: Rebase, pass $ho to preseed_base
v3: Handle $xopts{ExtraPreseed} undefined in preseed_base
fixup: No need to default
fixup: Move OverlayLocal to preseed_base
---
Osstest/Debian.pm | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 5ca93db..f7ef1d9 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -37,6 +37,7 @@ BEGIN {
%preseed_cmds
preseed_base
preseed_create
+ preseed_create_guest
preseed_ssh
preseed_hook_command preseed_hook_installscript
preseed_hook_overlay
@@ -617,6 +618,8 @@ END
sub preseed_base ($$$$;@) {
my ($ho,$suite,$sfx,$extra_packages,%xopts) = @_;
+ $xopts{ExtraPreseed} ||= '';
+
preseed_ssh($ho, $sfx);
preseed_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar');
@@ -702,6 +705,28 @@ END
return $preseed;
}
+sub preseed_create_guest ($$;@) {
+ my ($ho, $sfx, %xopts) = @_;
+
+ my $suite= $xopts{Suite} || $c{DebianSuite};
+
+ my $extra_packages;
+
+ my $preseed_file= preseed_base($ho, $suite, $sfx, $extra_packages, %xopts);
+ $preseed_file.= (<<END);
+d-i partman-auto/method string regular
+d-i partman-auto/choose_recipe \\
+ select All files in one partition (recommended for new users)
+
+d-i grub-installer/bootdev string /dev/xvda
+
+END
+
+ $preseed_file .= preseed_hook_cmds();
+
+ return create_webfile($ho, "preseed$sfx", $preseed_file);
+}
+
sub preseed_create ($$;@) {
my ($ho, $sfx, %xopts) = @_;
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 07/24] Debian: add preseed_create_guest helper
2015-05-06 9:56 ` [OSSTEST v6 07/24] Debian: add preseed_create_guest helper Ian Campbell
@ 2015-05-12 15:16 ` Ian Jackson
2015-05-12 15:22 ` Ian Campbell
0 siblings, 1 reply; 53+ messages in thread
From: Ian Jackson @ 2015-05-12 15:16 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("[OSSTEST v6 07/24] Debian: add preseed_create_guest helper"):
> Creates a preseed file suitable for use in a PV guest
...
> + my $extra_packages;
Doesn't this need a value from somewhere ?
The rest looks plausible.
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 07/24] Debian: add preseed_create_guest helper
2015-05-12 15:16 ` Ian Jackson
@ 2015-05-12 15:22 ` Ian Campbell
0 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-12 15:22 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
On Tue, 2015-05-12 at 16:16 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[OSSTEST v6 07/24] Debian: add preseed_create_guest helper"):
> > Creates a preseed file suitable for use in a PV guest
> ...
> > + my $extra_packages;
>
> Doesn't this need a value from somewhere ?
It's given one in "distros: support booting Debian PV (d-i installed)
guests with pvgrub." later on, which is clearly too late.
I'll move the definition and use to the patch which actually needs it.
>
> The rest looks plausible.
>
> Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread
* [OSSTEST v6 08/24] ts-debian-hvm-install: Only apply EFI workaround for Wheezy
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (6 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 07/24] Debian: add preseed_create_guest helper Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-12 15:17 ` Ian Jackson
2015-05-06 9:56 ` [OSSTEST v6 09/24] make-flight: Handle $BUILD_LVEXTEND_MAX in mfi-common:create_build_jobs() Ian Campbell
` (17 subsequent siblings)
25 siblings, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: wei.liu2, Ian Campbell, xen-devel
The previous refactoring of preseed hooks makes this easy to do.
The underlying issue is lack of persistent variable store in our OVMF
setup, which we workaround by placing a copy of grub at the removable
media path. Add a comment saying this since I initially thought this
was just a Wheezy bug.
In Jessie the extra copy of grub can be achieved by preseeding
grub-installer/force-efi-extra-removable (since various real h/w has
similar limitations/bugs) however I haven't tested that so I didn't
add it to the preseed yet, I just mention it in the code comment.
Currently this script hardcodes Wheezy, refactor to use the 'suite'
guest_var (or $c{GuestDebianSuite})
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: wei.liu2@citrix.com
---
v6: New patch
---
ts-debian-hvm-install | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/ts-debian-hvm-install b/ts-debian-hvm-install
index b37f0f9..f3629e1 100755
--- a/ts-debian-hvm-install
+++ b/ts-debian-hvm-install
@@ -41,9 +41,11 @@ our $disk_mb= 10000;
our $guesthost= "$gn.guest.osstest";
our $gho;
+our $gsuite= guest_var($gho,'suite',$c{GuestDebianSuite});
+
sub preseed () {
- my $preseed_file = preseed_base($gho,'wheezy','','',());
+ my $preseed_file = preseed_base($gho,$gsuite,'','',());
$preseed_file .= (<<END);
d-i netcfg/get_hostname string $gn
@@ -71,7 +73,15 @@ d-i partman-auto/expert_recipe string \\
d-i apt-setup/cdrom/set-first boolean false
END
-preseed_hook_command($gho, 'late_command', '', <<END);
+
+ # Our OVMF does not have a persistent variable store and therefore
+ # cannot remember the boot entry added by the Debian
+ # installer. Therefore we ensure that a copy of the bootloader is
+ # copied to the removable media path, which is the EFI fallback.
+ #
+ # From Jessie onwards this can be worked around by using the fix to
+ # #767037 i.e. preseeding grub-installer/force-efi-extra-removable.
+preseed_hook_command($gho, 'late_command', '', <<END) if $gsuite =~ /wheezy/;
#!/bin/sh
set -ex
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 08/24] ts-debian-hvm-install: Only apply EFI workaround for Wheezy
2015-05-06 9:56 ` [OSSTEST v6 08/24] ts-debian-hvm-install: Only apply EFI workaround for Wheezy Ian Campbell
@ 2015-05-12 15:17 ` Ian Jackson
0 siblings, 0 replies; 53+ messages in thread
From: Ian Jackson @ 2015-05-12 15:17 UTC (permalink / raw)
To: Ian Campbell; +Cc: wei.liu2, xen-devel
Ian Campbell writes ("[OSSTEST v6 08/24] ts-debian-hvm-install: Only apply EFI workaround for Wheezy"):
> The previous refactoring of preseed hooks makes this easy to do.
>
> The underlying issue is lack of persistent variable store in our OVMF
> setup, which we workaround by placing a copy of grub at the removable
> media path. Add a comment saying this since I initially thought this
> was just a Wheezy bug.
>
> In Jessie the extra copy of grub can be achieved by preseeding
> grub-installer/force-efi-extra-removable (since various real h/w has
> similar limitations/bugs) however I haven't tested that so I didn't
> add it to the preseed yet, I just mention it in the code comment.
>
> Currently this script hardcodes Wheezy, refactor to use the 'suite'
> guest_var (or $c{GuestDebianSuite})
>
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> Cc: wei.liu2@citrix.com
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
^ permalink raw reply [flat|nested] 53+ messages in thread
* [OSSTEST v6 09/24] make-flight: Handle $BUILD_LVEXTEND_MAX in mfi-common:create_build_jobs()
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (7 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 08/24] ts-debian-hvm-install: Only apply EFI workaround for Wheezy Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 10/24] distros: add support for installing Debian PV guests via d-i, flight and jobs Ian Campbell
` (16 subsequent siblings)
25 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
v3: New patch
---
make-flight | 4 ----
mfi-common | 4 ++++
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/make-flight b/make-flight
index cc8ecdb..ec0a20c 100755
--- a/make-flight
+++ b/make-flight
@@ -36,10 +36,6 @@ defguestsuite=`getconfig GuestDebianSuite`
if [ x$buildflight = x ]; then
- if [ "x$BUILD_LVEXTEND_MAX" != x ]; then
- BUILD_RUNVARS+=" build_lvextend_max=$BUILD_LVEXTEND_MAX "
- fi
-
create_build_jobs
else
diff --git a/mfi-common b/mfi-common
index 16fc8c5..a9e966f 100644
--- a/mfi-common
+++ b/mfi-common
@@ -63,6 +63,10 @@ create_build_jobs () {
local enable_ovmf
local build_hostflags
+ if [ "x$BUILD_LVEXTEND_MAX" != x ]; then
+ BUILD_RUNVARS+=" build_lvextend_max=$BUILD_LVEXTEND_MAX "
+ fi
+
for arch in ${BUILD_ARCHES- i386 amd64 armhf }; do
if [ "x$arch" = xdisable ]; then continue; fi
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 10/24] distros: add support for installing Debian PV guests via d-i, flight and jobs
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (8 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 09/24] make-flight: Handle $BUILD_LVEXTEND_MAX in mfi-common:create_build_jobs() Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-12 15:25 ` Ian Jackson
2015-05-06 9:56 ` [OSSTEST v6 11/24] distros: support booting Debian PV (d-i installed) guests with pvgrub Ian Campbell
` (15 subsequent siblings)
25 siblings, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
This patch introduces ts-debian-di-install which can install Debian
from a netboot (PXE) debian installer image. By default it installs
from the d-i image used by osstest (using the special Xen PV guest
enabled flavour where necessary) but it can also install the current
release versions from Squeeze onwards (up to and including Sid) and
the d-i daily builds. This is controlled by runvars {Guest}_diver =
osstest|current and {Guest}_dist = squeeze|...|sid. The resulting
guests boot the distro kernel using pygrub (pvgrub will follow).
The distros flights differ substantially from the existing flights.
Introduce make-distros-flight using the functionality previously
refactored into mfi-common. The new flight tests all versions of
Debian from Squeeze onward as an amd64, i386 and armhf guests (armhf
from Jessie onwards only) using the usual smoke tests.
Test names are suffixed -pygrub pending the addition of pvgrub
variants in a future commit.
Add the new cases to sg-run-job
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v6: Only apply -xen suffix to x86 images when doing a netboot using
the osstest version of d-i, since that is the only arch where we
create such files, other arches can use the bare names.
Use the guest $arch not the host $r{arch} when finding the
kernel+initrd to use for d-i install using the osstest d-i.
v4: use guest create
v3: $BUILD_LVEXTEND_MAX now handled in mfi-common
Consolidate setting of ruvars
Include $flight and $job in tmpdir name
Use Osstest::Debian::di_installcmdline_core
Document the usage of get_host_property on a guest object
Correct ARM netboot paths
Include bootloader in test name
Should include -pv too?
console= repetition for Jessie onwards.
Wait for up to an hour for the install. I'd seen timeouts right at
the end of the install with the previous value
---
Osstest/TestSupport.pm | 3 +
make-distros-flight | 98 ++++++++++++++++++++++++++++
sg-run-job | 11 ++++
ts-debian-di-install | 170 +++++++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 282 insertions(+)
create mode 100755 make-distros-flight
create mode 100755 ts-debian-di-install
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 8d6b349..b3d85d6 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -906,8 +906,11 @@ sub propname_massage ($) {
return $prop;
}
+# It is fine to call this on a guest object too, in which case it will
+# always return $defval.
sub get_host_property ($$;$) {
my ($ho, $prop, $defval) = @_;
+ return $defval unless $ho->{Properties};
my $val = $ho->{Properties}{propname_massage($prop)};
return defined($val) ? $val : $defval;
}
diff --git a/make-distros-flight b/make-distros-flight
new file mode 100755
index 0000000..5067342
--- /dev/null
+++ b/make-distros-flight
@@ -0,0 +1,98 @@
+#!/bin/bash
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+
+set -e
+
+branch=$1
+xenbranch=$2
+blessing=$3
+buildflight=$4
+
+flight=`./cs-flight-create $blessing $branch`
+
+. cri-common
+. ap-common
+. mfi-common
+
+defsuite=`getconfig DebianSuite`
+defguestsuite=`getconfig GuestDebianSuite`
+
+if [ x$buildflight = x ]; then
+
+ WANT_XEND=false REVISION_LINUX_OLD=disable
+
+ create_build_jobs
+
+else
+
+ bfi=$buildflight.
+
+fi
+
+job_create_test_filter_callback () {
+ if [ "$xenarch" = "i386" ]; then return 1; fi
+ return 0
+}
+
+test_matrix_branch_filter_callback () {
+ :
+}
+
+test_do_one_netboot () {
+ job_create_test \
+ test-$xenarch$kern-$dom0arch-$domU-$dist-netboot-pygrub \
+ test-debian-di xl $xenarch $dom0arch \
+ kernbuildjob=${bfi}build-$dom0arch-$kernbuild \
+ debian_arch=$domU \
+ debian_dist=$dist \
+ debian_method=netboot \
+ debian_diver=current \
+ all_hostflags=$most_hostflags
+}
+
+test_matrix_do_one () {
+ case ${xenarch} in
+ amd64) domUarches="amd64 i386";;
+ armhf) domUarches="armhf";;
+ esac
+
+ for domU in $domUarches ; do
+ for dist in squeeze wheezy jessie sid daily ; do
+ case ${domU}_${dist} in
+ armhf_squeeze) continue;; # No armhf in Squeeze
+ armhf_wheezy) continue;; # No armhf guest support in Wheezy
+ *) ;;
+ esac
+
+ test_do_one_netboot
+
+ done
+
+ done
+}
+
+test_matrix_iterate
+
+echo $flight
+
+# Local variables:
+# mode: sh
+# sh-basic-offset: 2
+# indent-tabs-mode: nil
+# End:
diff --git a/sg-run-job b/sg-run-job
index eae159d..e4a02cf 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -258,6 +258,17 @@ proc run-job/test-debian {} {
test-guest debian
}
+proc install-guest-debian-di {} {
+ run-ts . = ts-debian-di-install
+ run-ts . = ts-guest-start + debian
+}
+
+proc need-hosts/test-debian-di {} { return host }
+proc run-job/test-debian-di {} {
+ install-guest-debian-di
+ test-guest debian
+}
+
proc need-hosts/test-freebsd {} { return host }
proc run-job/test-freebsd {} {
run-ts . = ts-freebsd-install
diff --git a/ts-debian-di-install b/ts-debian-di-install
new file mode 100755
index 0000000..9fabd56
--- /dev/null
+++ b/ts-debian-di-install
@@ -0,0 +1,170 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use Osstest::Debian;
+use Osstest::TestSupport;
+
+tsreadconfig();
+
+our ($whhost,$gn) = @ARGV;
+$whhost ||= 'host';
+$gn ||= 'debian';
+
+our $ho= selecthost($whhost);
+
+our $ram_mb= 512;
+our $disk_mb= 10000;
+
+our $guesthost= "$gn.guest.osstest";
+our $gho;
+
+sub prep () {
+ target_install_packages_norec($ho, qw(lvm2));
+
+ $gho= prepareguest($ho, $gn, $guesthost, 22,
+ $disk_mb, 40);
+
+ prepareguest_part_lvmdisk($ho, $gho, $disk_mb);
+
+ target_cmd_root($ho, "umount $gho->{Lvdev} ||:");
+}
+
+sub setup_netboot($$$)
+{
+ my ($didir, $arch, $suite) = @_;
+
+ my $di_ver= $r{"$gho->{Guest}_diver"} || "osstest";
+
+ my ($kernel,$initrd);
+
+ if ( $di_ver eq "osstest" ) {
+ my $di_path = $c{TftpPath}.'/'.$ho->{Tftp}{DiBase}.'/'.${arch}.'/'.$c{TftpDiVersion}.'-'.$ho->{Suite};
+
+ my $suffix = '';
+ $suffix .= "-xen" if ${arch} =~ m/amd64|i386/;
+ $kernel = "$di_path/vmlinuz$suffix";
+ $initrd = "$di_path/initrd.gz$suffix";
+
+ target_putfile_root($ho, 60, $kernel, "$didir/kernel_${suite}_${arch}");
+ target_putfile_root($ho, 60, $initrd, "$didir/initrd_${suite}_${arch}");
+
+ } else {
+ my $mirror = "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath}";
+
+ my $di_url = $suite eq "daily" ?
+ "http://d-i.debian.org/daily-images/$arch/daily/netboot" :
+ "$mirror/dists/$suite/main/installer-$arch/$di_ver/images/netboot";
+
+ $di_url .= "/xen" if $arch =~ m/amd64|i386/;
+ $di_url .= "/debian-installer/arm64" if $arch =~ /arm64/;
+
+ $kernel = "$di_url/vmlinuz";
+ $initrd = "$di_url/initrd.gz";
+
+ target_fetchurl($ho, $kernel, "$didir/kernel_${suite}_${arch}");
+ target_fetchurl($ho, $initrd, "$didir/initrd_${suite}_${arch}");
+ }
+
+ store_runvar("$gho->{Guest}_netboot_kernel", $kernel);
+ store_runvar("$gho->{Guest}_netboot_initrd", $initrd);
+
+ return <<END;
+kernel = "$didir/kernel_${suite}_${arch}"
+ramdisk = "$didir/initrd_${suite}_${arch}"
+END
+}
+sub ginstall () {
+ my $arch= $r{"$gho->{Guest}_arch"};
+ my $method= $r{"$gho->{Guest}_method"};
+
+ my $tmpdir= "/root/$flight-$job-di";
+ target_cmd_root($ho, <<END);
+rm -rf $tmpdir
+mkdir $tmpdir
+END
+
+ my ($method_cfg, $ps_url, $extra_disk);
+
+ if ( $method eq "netboot" )
+ {
+ my $suite= $r{"$gho->{Guest}_dist"};
+ logm("$method $suite/$arch");
+
+ $method_cfg = setup_netboot($tmpdir, $arch, $suite);
+
+ $suite = "sid" if $suite eq "daily";
+
+ $ps_url = preseed_create_guest($gho, '', Suite=>$suite);
+
+ $extra_disk = "";
+ }
+ else
+ {
+ die "$method";
+ }
+
+ my @cmdline = ();
+ push @cmdline, "debian-installer/exit/always_halt=true";
+ push @cmdline, "domain=$c{TestHostDomain}";
+ push @cmdline, "console=hvc0";
+ push @cmdline, di_installcmdline_core($gho, $ps_url);
+ push @cmdline, "--";
+ # See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=762007 for
+ # why this is repeated.
+ push @cmdline, "console=hvc0";
+
+ my $cmdline = join(" ", @cmdline);
+
+ my %install_xopts = (
+ OnPowerOff => "preserve"
+ );
+
+ prepareguest_part_xencfg($ho, $gho, $ram_mb, \%install_xopts, <<END);
+$method_cfg
+extra = "$cmdline"
+#
+disk = [
+ $extra_disk 'phy:$gho->{Lvdev},xvda,w'
+ ]
+END
+
+ guest_create($gho);
+
+ guest_checkrunning($ho, $gho) or die "$gho->{Name} not running";
+
+ guest_await_shutdown($ho,$gho,3600);
+ guest_destroy($gho);
+
+ my $blcfg = <<END;
+bootloader = "pygrub"
+END
+
+ prepareguest_part_xencfg($ho, $gho, $ram_mb, {}, <<END);
+$blcfg
+#
+disk = [
+ 'phy:$gho->{Lvdev},xvda,w'
+ ]
+END
+ return;
+}
+
+prep();
+ginstall();
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 10/24] distros: add support for installing Debian PV guests via d-i, flight and jobs
2015-05-06 9:56 ` [OSSTEST v6 10/24] distros: add support for installing Debian PV guests via d-i, flight and jobs Ian Campbell
@ 2015-05-12 15:25 ` Ian Jackson
2015-05-12 15:40 ` Ian Campbell
0 siblings, 1 reply; 53+ messages in thread
From: Ian Jackson @ 2015-05-12 15:25 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("[OSSTEST v6 10/24] distros: add support for installing Debian PV guests via d-i, flight and jobs"):
> This patch introduces ts-debian-di-install which can install Debian
> from a netboot (PXE) debian installer image. By default it installs
> from the d-i image used by osstest (using the special Xen PV guest
> enabled flavour where necessary) but it can also install the current
> release versions from Squeeze onwards (up to and including Sid) and
> the d-i daily builds. This is controlled by runvars {Guest}_diver =
> osstest|current and {Guest}_dist = squeeze|...|sid. The resulting
> guests boot the distro kernel using pygrub (pvgrub will follow).
Shouldn't <GUEST>_dist be called <GUEST>_suite (which is probably
already set in various places) ?
> The distros flights differ substantially from the existing flights.
> Introduce make-distros-flight using the functionality previously
> refactored into mfi-common. The new flight tests all versions of
> Debian from Squeeze onward as an amd64, i386 and armhf guests (armhf
> from Jessie onwards only) using the usual smoke tests.
Do you have an output from mg-show-flight-runvars you could share with
us ?
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 10/24] distros: add support for installing Debian PV guests via d-i, flight and jobs
2015-05-12 15:25 ` Ian Jackson
@ 2015-05-12 15:40 ` Ian Campbell
2015-05-12 16:08 ` Ian Jackson
0 siblings, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-12 15:40 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
On Tue, 2015-05-12 at 16:25 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[OSSTEST v6 10/24] distros: add support for installing Debian PV guests via d-i, flight and jobs"):
> > This patch introduces ts-debian-di-install which can install Debian
> > from a netboot (PXE) debian installer image. By default it installs
> > from the d-i image used by osstest (using the special Xen PV guest
> > enabled flavour where necessary) but it can also install the current
> > release versions from Squeeze onwards (up to and including Sid) and
> > the d-i daily builds. This is controlled by runvars {Guest}_diver =
> > osstest|current and {Guest}_dist = squeeze|...|sid. The resulting
> > guests boot the distro kernel using pygrub (pvgrub will follow).
>
> Shouldn't <GUEST>_dist be called <GUEST>_suite (which is probably
> already set in various places) ?
Hrm, yes I think so.
Note that "set in various places" doesn't help here since our aim is to
enumerate all the versions, not just use the one from the config. But I
can at least use consistent naming for sure.
> > The distros flights differ substantially from the existing flights.
> > Introduce make-distros-flight using the functionality previously
> > refactored into mfi-common. The new flight tests all versions of
> > Debian from Squeeze onward as an amd64, i386 and armhf guests (armhf
> > from Jessie onwards only) using the usual smoke tests.
>
> Do you have an output from mg-show-flight-runvars you could share with
> us ?
>From a random (bt fairly recent) adhoc flight 37300 (sorted by job
name). Looks like the results are still at
http://www.chiark.greenend.org.uk/~xensrcts/logs/37300/ too
NB this would be from the end of the series not this exact point.
build-amd64 arch amd64
build-amd64 enable_ovmf true
build-amd64 enable_xend false
build-amd64 enable_xsm false
build-amd64 host_hostflags share-build-wheezy-amd64,arch-amd64,suite-wheezy,purpose-build
build-amd64-libvirt arch amd64
build-amd64-libvirt buildjob build-amd64
build-amd64-libvirt host_hostflags share-build-wheezy-amd64,arch-amd64,suite-wheezy,purpose-build
build-amd64-libvirt revision_libvirt 63a368012df8009023dc63b2b2a58f1ca2668f97
build-amd64-libvirt tree_libvirt git://xenbits.xen.org/libvirt.git
build-amd64-libvirt tree_xen git://xenbits.xen.org/xen.git
build-amd64-pvops arch amd64
build-amd64-pvops host_hostflags share-build-wheezy-amd64,arch-amd64,suite-wheezy,purpose-build
build-amd64-pvops kconfighow xen-enable-xen-config
build-amd64-pvops revision_linux 8a5f782c33c04ea5c9b3ca6fb32d6039e2e5c0c9
build-amd64-pvops revision_linuxfirmware c530a75c1e6a472b0eb9558310b518f0dfcd8860
build-amd64-pvops tree_linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
build-amd64-pvops tree_linux git://xenbits.xen.org/linux-pvops.git
build-amd64-pvops treevcs_linux git
build-amd64 revision_ovmf
build-amd64 revision_qemu
build-amd64 revision_qemuu 727b998448e852a5e8eb570ac3a259ef62fbdacb
build-amd64 revision_seabios
build-amd64 revision_xen 123c7793797502b222300eb710cd3873dcca41ee
build-amd64 tree_ovmf
build-amd64 tree_qemu git://xenbits.xen.org/staging/qemu-xen-unstable.git
build-amd64 tree_qemuu git://xenbits.xen.org/staging/qemu-upstream-unstable.git
build-amd64 tree_seabios
build-amd64 tree_xen git://xenbits.xen.org/xen.git
build-armhf arch armhf
build-armhf enable_ovmf true
build-armhf enable_xend false
build-armhf enable_xsm false
build-armhf host_hostflags share-build-wheezy-armhf,arch-armhf,suite-wheezy,purpose-build
build-armhf-libvirt arch armhf
build-armhf-libvirt buildjob build-armhf
build-armhf-libvirt host_hostflags share-build-wheezy-armhf,arch-armhf,suite-wheezy,purpose-build
build-armhf-libvirt revision_libvirt 63a368012df8009023dc63b2b2a58f1ca2668f97
build-armhf-libvirt tree_libvirt git://xenbits.xen.org/libvirt.git
build-armhf-libvirt tree_xen git://xenbits.xen.org/xen.git
build-armhf-pvops arch armhf
build-armhf-pvops host_hostflags share-build-wheezy-armhf,arch-armhf,suite-wheezy,purpose-build
build-armhf-pvops kconfighow xen-enable-xen-config
build-armhf-pvops kconfig_override_y CONFIG_EXT4_FS
build-armhf-pvops revision_linux ec18c9fc039971041d854e0d58551f1f1a32ff8f
build-armhf-pvops revision_linuxfirmware c530a75c1e6a472b0eb9558310b518f0dfcd8860
build-armhf-pvops tree_linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
build-armhf-pvops tree_linux git://xenbits.xen.org/linux-pvops.git
build-armhf-pvops treevcs_linux git
build-armhf revision_ovmf
build-armhf revision_qemu
build-armhf revision_qemuu 727b998448e852a5e8eb570ac3a259ef62fbdacb
build-armhf revision_seabios
build-armhf revision_xen 123c7793797502b222300eb710cd3873dcca41ee
build-armhf tree_ovmf
build-armhf tree_qemu git://xenbits.xen.org/staging/qemu-xen-unstable.git
build-armhf tree_qemuu git://xenbits.xen.org/staging/qemu-upstream-unstable.git
build-armhf tree_seabios
build-armhf tree_xen git://xenbits.xen.org/xen.git
build-i386 arch i386
build-i386 enable_ovmf true
build-i386 enable_xend false
build-i386 enable_xsm false
build-i386 host_hostflags share-build-wheezy-i386,arch-i386,suite-wheezy,purpose-build
build-i386-libvirt arch i386
build-i386-libvirt buildjob build-i386
build-i386-libvirt host_hostflags share-build-wheezy-i386,arch-i386,suite-wheezy,purpose-build
build-i386-libvirt revision_libvirt 63a368012df8009023dc63b2b2a58f1ca2668f97
build-i386-libvirt tree_libvirt git://xenbits.xen.org/libvirt.git
build-i386-libvirt tree_xen git://xenbits.xen.org/xen.git
build-i386-pvops arch i386
build-i386-pvops host_hostflags share-build-wheezy-i386,arch-i386,suite-wheezy,purpose-build
build-i386-pvops kconfighow xen-enable-xen-config
build-i386-pvops revision_linux 8a5f782c33c04ea5c9b3ca6fb32d6039e2e5c0c9
build-i386-pvops revision_linuxfirmware c530a75c1e6a472b0eb9558310b518f0dfcd8860
build-i386-pvops tree_linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
build-i386-pvops tree_linux git://xenbits.xen.org/linux-pvops.git
build-i386-pvops treevcs_linux git
build-i386 revision_ovmf
build-i386 revision_qemu
build-i386 revision_qemuu 727b998448e852a5e8eb570ac3a259ef62fbdacb
build-i386 revision_seabios
build-i386 revision_xen 123c7793797502b222300eb710cd3873dcca41ee
build-i386 tree_ovmf
build-i386 tree_qemu git://xenbits.xen.org/staging/qemu-xen-unstable.git
build-i386 tree_qemuu git://xenbits.xen.org/staging/qemu-upstream-unstable.git
build-i386 tree_seabios
build-i386 tree_xen git://xenbits.xen.org/xen.git
test-amd64-amd64-amd64-current-netinst-pygrub all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-amd64-amd64-current-netinst-pygrub arch amd64
test-amd64-amd64-amd64-current-netinst-pygrub buildjob build-amd64
test-amd64-amd64-amd64-current-netinst-pygrub debian_arch amd64
test-amd64-amd64-amd64-current-netinst-pygrub debian_bootloader pygrub
test-amd64-amd64-amd64-current-netinst-pygrub debian_cd current
test-amd64-amd64-amd64-current-netinst-pygrub debian_method netinst
test-amd64-amd64-amd64-current-netinst-pygrub kernbuildjob build-amd64-pvops
test-amd64-amd64-amd64-current-netinst-pygrub kernkind pvops
test-amd64-amd64-amd64-current-netinst-pygrub toolstack xl
test-amd64-amd64-amd64-current-netinst-pygrub xenbuildjob build-amd64
test-amd64-amd64-amd64-daily-netboot-pvgrub all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-amd64-amd64-daily-netboot-pvgrub arch amd64
test-amd64-amd64-amd64-daily-netboot-pvgrub buildjob build-amd64
test-amd64-amd64-amd64-daily-netboot-pvgrub debian_arch amd64
test-amd64-amd64-amd64-daily-netboot-pvgrub debian_bootloader pvgrub
test-amd64-amd64-amd64-daily-netboot-pvgrub debian_dist daily
test-amd64-amd64-amd64-daily-netboot-pvgrub debian_diver current
test-amd64-amd64-amd64-daily-netboot-pvgrub debian_method netboot
test-amd64-amd64-amd64-daily-netboot-pvgrub kernbuildjob build-amd64-pvops
test-amd64-amd64-amd64-daily-netboot-pvgrub kernkind pvops
test-amd64-amd64-amd64-daily-netboot-pvgrub toolstack xl
test-amd64-amd64-amd64-daily-netboot-pvgrub xenbuildjob build-amd64
test-amd64-amd64-amd64-jessie-netboot-pvgrub all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-amd64-amd64-jessie-netboot-pvgrub arch amd64
test-amd64-amd64-amd64-jessie-netboot-pvgrub buildjob build-amd64
test-amd64-amd64-amd64-jessie-netboot-pvgrub debian_arch amd64
test-amd64-amd64-amd64-jessie-netboot-pvgrub debian_bootloader pvgrub
test-amd64-amd64-amd64-jessie-netboot-pvgrub debian_dist jessie
test-amd64-amd64-amd64-jessie-netboot-pvgrub debian_diver current
test-amd64-amd64-amd64-jessie-netboot-pvgrub debian_method netboot
test-amd64-amd64-amd64-jessie-netboot-pvgrub kernbuildjob build-amd64-pvops
test-amd64-amd64-amd64-jessie-netboot-pvgrub kernkind pvops
test-amd64-amd64-amd64-jessie-netboot-pvgrub toolstack xl
test-amd64-amd64-amd64-jessie-netboot-pvgrub xenbuildjob build-amd64
test-amd64-amd64-amd64-sid-netboot-pygrub all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-amd64-amd64-sid-netboot-pygrub arch amd64
test-amd64-amd64-amd64-sid-netboot-pygrub buildjob build-amd64
test-amd64-amd64-amd64-sid-netboot-pygrub debian_arch amd64
test-amd64-amd64-amd64-sid-netboot-pygrub debian_bootloader pygrub
test-amd64-amd64-amd64-sid-netboot-pygrub debian_dist sid
test-amd64-amd64-amd64-sid-netboot-pygrub debian_diver current
test-amd64-amd64-amd64-sid-netboot-pygrub debian_method netboot
test-amd64-amd64-amd64-sid-netboot-pygrub kernbuildjob build-amd64-pvops
test-amd64-amd64-amd64-sid-netboot-pygrub kernkind pvops
test-amd64-amd64-amd64-sid-netboot-pygrub toolstack xl
test-amd64-amd64-amd64-sid-netboot-pygrub xenbuildjob build-amd64
test-amd64-amd64-amd64-squeeze-netboot-pygrub all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-amd64-amd64-squeeze-netboot-pygrub arch amd64
test-amd64-amd64-amd64-squeeze-netboot-pygrub buildjob build-amd64
test-amd64-amd64-amd64-squeeze-netboot-pygrub debian_arch amd64
test-amd64-amd64-amd64-squeeze-netboot-pygrub debian_bootloader pygrub
test-amd64-amd64-amd64-squeeze-netboot-pygrub debian_dist squeeze
test-amd64-amd64-amd64-squeeze-netboot-pygrub debian_diver current
test-amd64-amd64-amd64-squeeze-netboot-pygrub debian_method netboot
test-amd64-amd64-amd64-squeeze-netboot-pygrub kernbuildjob build-amd64-pvops
test-amd64-amd64-amd64-squeeze-netboot-pygrub kernkind pvops
test-amd64-amd64-amd64-squeeze-netboot-pygrub toolstack xl
test-amd64-amd64-amd64-squeeze-netboot-pygrub xenbuildjob build-amd64
test-amd64-amd64-amd64-weekly-netinst-pygrub all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-amd64-amd64-weekly-netinst-pygrub arch amd64
test-amd64-amd64-amd64-weekly-netinst-pygrub buildjob build-amd64
test-amd64-amd64-amd64-weekly-netinst-pygrub debian_arch amd64
test-amd64-amd64-amd64-weekly-netinst-pygrub debian_bootloader pygrub
test-amd64-amd64-amd64-weekly-netinst-pygrub debian_cd weekly
test-amd64-amd64-amd64-weekly-netinst-pygrub debian_method netinst
test-amd64-amd64-amd64-weekly-netinst-pygrub kernbuildjob build-amd64-pvops
test-amd64-amd64-amd64-weekly-netinst-pygrub kernkind pvops
test-amd64-amd64-amd64-weekly-netinst-pygrub toolstack xl
test-amd64-amd64-amd64-weekly-netinst-pygrub xenbuildjob build-amd64
test-amd64-amd64-amd64-wheezy-netboot-pvgrub all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-amd64-amd64-wheezy-netboot-pvgrub arch amd64
test-amd64-amd64-amd64-wheezy-netboot-pvgrub buildjob build-amd64
test-amd64-amd64-amd64-wheezy-netboot-pvgrub debian_arch amd64
test-amd64-amd64-amd64-wheezy-netboot-pvgrub debian_bootloader pvgrub
test-amd64-amd64-amd64-wheezy-netboot-pvgrub debian_dist wheezy
test-amd64-amd64-amd64-wheezy-netboot-pvgrub debian_diver current
test-amd64-amd64-amd64-wheezy-netboot-pvgrub debian_method netboot
test-amd64-amd64-amd64-wheezy-netboot-pvgrub kernbuildjob build-amd64-pvops
test-amd64-amd64-amd64-wheezy-netboot-pvgrub kernkind pvops
test-amd64-amd64-amd64-wheezy-netboot-pvgrub toolstack xl
test-amd64-amd64-amd64-wheezy-netboot-pvgrub xenbuildjob build-amd64
test-amd64-amd64-i386-current-netinst-pygrub all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-amd64-i386-current-netinst-pygrub arch amd64
test-amd64-amd64-i386-current-netinst-pygrub buildjob build-amd64
test-amd64-amd64-i386-current-netinst-pygrub debian_arch i386
test-amd64-amd64-i386-current-netinst-pygrub debian_bootloader pygrub
test-amd64-amd64-i386-current-netinst-pygrub debian_cd current
test-amd64-amd64-i386-current-netinst-pygrub debian_method netinst
test-amd64-amd64-i386-current-netinst-pygrub kernbuildjob build-amd64-pvops
test-amd64-amd64-i386-current-netinst-pygrub kernkind pvops
test-amd64-amd64-i386-current-netinst-pygrub toolstack xl
test-amd64-amd64-i386-current-netinst-pygrub xenbuildjob build-amd64
test-amd64-amd64-i386-daily-netboot-pygrub all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-amd64-i386-daily-netboot-pygrub arch amd64
test-amd64-amd64-i386-daily-netboot-pygrub buildjob build-amd64
test-amd64-amd64-i386-daily-netboot-pygrub debian_arch i386
test-amd64-amd64-i386-daily-netboot-pygrub debian_bootloader pygrub
test-amd64-amd64-i386-daily-netboot-pygrub debian_dist daily
test-amd64-amd64-i386-daily-netboot-pygrub debian_diver current
test-amd64-amd64-i386-daily-netboot-pygrub debian_method netboot
test-amd64-amd64-i386-daily-netboot-pygrub kernbuildjob build-amd64-pvops
test-amd64-amd64-i386-daily-netboot-pygrub kernkind pvops
test-amd64-amd64-i386-daily-netboot-pygrub toolstack xl
test-amd64-amd64-i386-daily-netboot-pygrub xenbuildjob build-amd64
test-amd64-amd64-i386-jessie-netboot-pygrub all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-amd64-i386-jessie-netboot-pygrub arch amd64
test-amd64-amd64-i386-jessie-netboot-pygrub buildjob build-amd64
test-amd64-amd64-i386-jessie-netboot-pygrub debian_arch i386
test-amd64-amd64-i386-jessie-netboot-pygrub debian_bootloader pygrub
test-amd64-amd64-i386-jessie-netboot-pygrub debian_dist jessie
test-amd64-amd64-i386-jessie-netboot-pygrub debian_diver current
test-amd64-amd64-i386-jessie-netboot-pygrub debian_method netboot
test-amd64-amd64-i386-jessie-netboot-pygrub kernbuildjob build-amd64-pvops
test-amd64-amd64-i386-jessie-netboot-pygrub kernkind pvops
test-amd64-amd64-i386-jessie-netboot-pygrub toolstack xl
test-amd64-amd64-i386-jessie-netboot-pygrub xenbuildjob build-amd64
test-amd64-amd64-i386-sid-netboot-pvgrub all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-amd64-i386-sid-netboot-pvgrub arch amd64
test-amd64-amd64-i386-sid-netboot-pvgrub buildjob build-amd64
test-amd64-amd64-i386-sid-netboot-pvgrub debian_arch i386
test-amd64-amd64-i386-sid-netboot-pvgrub debian_bootloader pvgrub
test-amd64-amd64-i386-sid-netboot-pvgrub debian_dist sid
test-amd64-amd64-i386-sid-netboot-pvgrub debian_diver current
test-amd64-amd64-i386-sid-netboot-pvgrub debian_method netboot
test-amd64-amd64-i386-sid-netboot-pvgrub kernbuildjob build-amd64-pvops
test-amd64-amd64-i386-sid-netboot-pvgrub kernkind pvops
test-amd64-amd64-i386-sid-netboot-pvgrub toolstack xl
test-amd64-amd64-i386-sid-netboot-pvgrub xenbuildjob build-amd64
test-amd64-amd64-i386-squeeze-netboot-pygrub all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-amd64-i386-squeeze-netboot-pygrub arch amd64
test-amd64-amd64-i386-squeeze-netboot-pygrub buildjob build-amd64
test-amd64-amd64-i386-squeeze-netboot-pygrub debian_arch i386
test-amd64-amd64-i386-squeeze-netboot-pygrub debian_bootloader pygrub
test-amd64-amd64-i386-squeeze-netboot-pygrub debian_dist squeeze
test-amd64-amd64-i386-squeeze-netboot-pygrub debian_diver current
test-amd64-amd64-i386-squeeze-netboot-pygrub debian_method netboot
test-amd64-amd64-i386-squeeze-netboot-pygrub kernbuildjob build-amd64-pvops
test-amd64-amd64-i386-squeeze-netboot-pygrub kernkind pvops
test-amd64-amd64-i386-squeeze-netboot-pygrub toolstack xl
test-amd64-amd64-i386-squeeze-netboot-pygrub xenbuildjob build-amd64
test-amd64-amd64-i386-weekly-netinst-pygrub all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-amd64-i386-weekly-netinst-pygrub arch amd64
test-amd64-amd64-i386-weekly-netinst-pygrub buildjob build-amd64
test-amd64-amd64-i386-weekly-netinst-pygrub debian_arch i386
test-amd64-amd64-i386-weekly-netinst-pygrub debian_bootloader pygrub
test-amd64-amd64-i386-weekly-netinst-pygrub debian_cd weekly
test-amd64-amd64-i386-weekly-netinst-pygrub debian_method netinst
test-amd64-amd64-i386-weekly-netinst-pygrub kernbuildjob build-amd64-pvops
test-amd64-amd64-i386-weekly-netinst-pygrub kernkind pvops
test-amd64-amd64-i386-weekly-netinst-pygrub toolstack xl
test-amd64-amd64-i386-weekly-netinst-pygrub xenbuildjob build-amd64
test-amd64-amd64-i386-wheezy-netboot-pygrub all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-amd64-i386-wheezy-netboot-pygrub arch amd64
test-amd64-amd64-i386-wheezy-netboot-pygrub buildjob build-amd64
test-amd64-amd64-i386-wheezy-netboot-pygrub debian_arch i386
test-amd64-amd64-i386-wheezy-netboot-pygrub debian_bootloader pygrub
test-amd64-amd64-i386-wheezy-netboot-pygrub debian_dist wheezy
test-amd64-amd64-i386-wheezy-netboot-pygrub debian_diver current
test-amd64-amd64-i386-wheezy-netboot-pygrub debian_method netboot
test-amd64-amd64-i386-wheezy-netboot-pygrub kernbuildjob build-amd64-pvops
test-amd64-amd64-i386-wheezy-netboot-pygrub kernkind pvops
test-amd64-amd64-i386-wheezy-netboot-pygrub toolstack xl
test-amd64-amd64-i386-wheezy-netboot-pygrub xenbuildjob build-amd64
test-amd64-i386-amd64-current-netinst-pygrub all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-i386-amd64-current-netinst-pygrub arch i386
test-amd64-i386-amd64-current-netinst-pygrub buildjob build-i386
test-amd64-i386-amd64-current-netinst-pygrub debian_arch amd64
test-amd64-i386-amd64-current-netinst-pygrub debian_bootloader pygrub
test-amd64-i386-amd64-current-netinst-pygrub debian_cd current
test-amd64-i386-amd64-current-netinst-pygrub debian_method netinst
test-amd64-i386-amd64-current-netinst-pygrub kernbuildjob build-i386-pvops
test-amd64-i386-amd64-current-netinst-pygrub kernkind pvops
test-amd64-i386-amd64-current-netinst-pygrub toolstack xl
test-amd64-i386-amd64-current-netinst-pygrub xenbuildjob build-amd64
test-amd64-i386-amd64-daily-netboot-pygrub all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-i386-amd64-daily-netboot-pygrub arch i386
test-amd64-i386-amd64-daily-netboot-pygrub buildjob build-i386
test-amd64-i386-amd64-daily-netboot-pygrub debian_arch amd64
test-amd64-i386-amd64-daily-netboot-pygrub debian_bootloader pygrub
test-amd64-i386-amd64-daily-netboot-pygrub debian_dist daily
test-amd64-i386-amd64-daily-netboot-pygrub debian_diver current
test-amd64-i386-amd64-daily-netboot-pygrub debian_method netboot
test-amd64-i386-amd64-daily-netboot-pygrub kernbuildjob build-i386-pvops
test-amd64-i386-amd64-daily-netboot-pygrub kernkind pvops
test-amd64-i386-amd64-daily-netboot-pygrub toolstack xl
test-amd64-i386-amd64-daily-netboot-pygrub xenbuildjob build-amd64
test-amd64-i386-amd64-jessie-netboot-pygrub all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-i386-amd64-jessie-netboot-pygrub arch i386
test-amd64-i386-amd64-jessie-netboot-pygrub buildjob build-i386
test-amd64-i386-amd64-jessie-netboot-pygrub debian_arch amd64
test-amd64-i386-amd64-jessie-netboot-pygrub debian_bootloader pygrub
test-amd64-i386-amd64-jessie-netboot-pygrub debian_dist jessie
test-amd64-i386-amd64-jessie-netboot-pygrub debian_diver current
test-amd64-i386-amd64-jessie-netboot-pygrub debian_method netboot
test-amd64-i386-amd64-jessie-netboot-pygrub kernbuildjob build-i386-pvops
test-amd64-i386-amd64-jessie-netboot-pygrub kernkind pvops
test-amd64-i386-amd64-jessie-netboot-pygrub toolstack xl
test-amd64-i386-amd64-jessie-netboot-pygrub xenbuildjob build-amd64
test-amd64-i386-amd64-sid-netboot-pygrub all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-i386-amd64-sid-netboot-pygrub arch i386
test-amd64-i386-amd64-sid-netboot-pygrub buildjob build-i386
test-amd64-i386-amd64-sid-netboot-pygrub debian_arch amd64
test-amd64-i386-amd64-sid-netboot-pygrub debian_bootloader pygrub
test-amd64-i386-amd64-sid-netboot-pygrub debian_dist sid
test-amd64-i386-amd64-sid-netboot-pygrub debian_diver current
test-amd64-i386-amd64-sid-netboot-pygrub debian_method netboot
test-amd64-i386-amd64-sid-netboot-pygrub kernbuildjob build-i386-pvops
test-amd64-i386-amd64-sid-netboot-pygrub kernkind pvops
test-amd64-i386-amd64-sid-netboot-pygrub toolstack xl
test-amd64-i386-amd64-sid-netboot-pygrub xenbuildjob build-amd64
test-amd64-i386-amd64-squeeze-netboot-pygrub all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-i386-amd64-squeeze-netboot-pygrub arch i386
test-amd64-i386-amd64-squeeze-netboot-pygrub buildjob build-i386
test-amd64-i386-amd64-squeeze-netboot-pygrub debian_arch amd64
test-amd64-i386-amd64-squeeze-netboot-pygrub debian_bootloader pygrub
test-amd64-i386-amd64-squeeze-netboot-pygrub debian_dist squeeze
test-amd64-i386-amd64-squeeze-netboot-pygrub debian_diver current
test-amd64-i386-amd64-squeeze-netboot-pygrub debian_method netboot
test-amd64-i386-amd64-squeeze-netboot-pygrub kernbuildjob build-i386-pvops
test-amd64-i386-amd64-squeeze-netboot-pygrub kernkind pvops
test-amd64-i386-amd64-squeeze-netboot-pygrub toolstack xl
test-amd64-i386-amd64-squeeze-netboot-pygrub xenbuildjob build-amd64
test-amd64-i386-amd64-weekly-netinst-pygrub all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-i386-amd64-weekly-netinst-pygrub arch i386
test-amd64-i386-amd64-weekly-netinst-pygrub buildjob build-i386
test-amd64-i386-amd64-weekly-netinst-pygrub debian_arch amd64
test-amd64-i386-amd64-weekly-netinst-pygrub debian_bootloader pygrub
test-amd64-i386-amd64-weekly-netinst-pygrub debian_cd weekly
test-amd64-i386-amd64-weekly-netinst-pygrub debian_method netinst
test-amd64-i386-amd64-weekly-netinst-pygrub kernbuildjob build-i386-pvops
test-amd64-i386-amd64-weekly-netinst-pygrub kernkind pvops
test-amd64-i386-amd64-weekly-netinst-pygrub toolstack xl
test-amd64-i386-amd64-weekly-netinst-pygrub xenbuildjob build-amd64
test-amd64-i386-amd64-wheezy-netboot-pygrub all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-i386-amd64-wheezy-netboot-pygrub arch i386
test-amd64-i386-amd64-wheezy-netboot-pygrub buildjob build-i386
test-amd64-i386-amd64-wheezy-netboot-pygrub debian_arch amd64
test-amd64-i386-amd64-wheezy-netboot-pygrub debian_bootloader pygrub
test-amd64-i386-amd64-wheezy-netboot-pygrub debian_dist wheezy
test-amd64-i386-amd64-wheezy-netboot-pygrub debian_diver current
test-amd64-i386-amd64-wheezy-netboot-pygrub debian_method netboot
test-amd64-i386-amd64-wheezy-netboot-pygrub kernbuildjob build-i386-pvops
test-amd64-i386-amd64-wheezy-netboot-pygrub kernkind pvops
test-amd64-i386-amd64-wheezy-netboot-pygrub toolstack xl
test-amd64-i386-amd64-wheezy-netboot-pygrub xenbuildjob build-amd64
test-amd64-i386-i386-current-netinst-pygrub all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-i386-i386-current-netinst-pygrub arch i386
test-amd64-i386-i386-current-netinst-pygrub buildjob build-i386
test-amd64-i386-i386-current-netinst-pygrub debian_arch i386
test-amd64-i386-i386-current-netinst-pygrub debian_bootloader pygrub
test-amd64-i386-i386-current-netinst-pygrub debian_cd current
test-amd64-i386-i386-current-netinst-pygrub debian_method netinst
test-amd64-i386-i386-current-netinst-pygrub kernbuildjob build-i386-pvops
test-amd64-i386-i386-current-netinst-pygrub kernkind pvops
test-amd64-i386-i386-current-netinst-pygrub toolstack xl
test-amd64-i386-i386-current-netinst-pygrub xenbuildjob build-amd64
test-amd64-i386-i386-daily-netboot-pvgrub all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-i386-i386-daily-netboot-pvgrub arch i386
test-amd64-i386-i386-daily-netboot-pvgrub buildjob build-i386
test-amd64-i386-i386-daily-netboot-pvgrub debian_arch i386
test-amd64-i386-i386-daily-netboot-pvgrub debian_bootloader pvgrub
test-amd64-i386-i386-daily-netboot-pvgrub debian_dist daily
test-amd64-i386-i386-daily-netboot-pvgrub debian_diver current
test-amd64-i386-i386-daily-netboot-pvgrub debian_method netboot
test-amd64-i386-i386-daily-netboot-pvgrub kernbuildjob build-i386-pvops
test-amd64-i386-i386-daily-netboot-pvgrub kernkind pvops
test-amd64-i386-i386-daily-netboot-pvgrub toolstack xl
test-amd64-i386-i386-daily-netboot-pvgrub xenbuildjob build-amd64
test-amd64-i386-i386-jessie-netboot-pvgrub all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-i386-i386-jessie-netboot-pvgrub arch i386
test-amd64-i386-i386-jessie-netboot-pvgrub buildjob build-i386
test-amd64-i386-i386-jessie-netboot-pvgrub debian_arch i386
test-amd64-i386-i386-jessie-netboot-pvgrub debian_bootloader pvgrub
test-amd64-i386-i386-jessie-netboot-pvgrub debian_dist jessie
test-amd64-i386-i386-jessie-netboot-pvgrub debian_diver current
test-amd64-i386-i386-jessie-netboot-pvgrub debian_method netboot
test-amd64-i386-i386-jessie-netboot-pvgrub kernbuildjob build-i386-pvops
test-amd64-i386-i386-jessie-netboot-pvgrub kernkind pvops
test-amd64-i386-i386-jessie-netboot-pvgrub toolstack xl
test-amd64-i386-i386-jessie-netboot-pvgrub xenbuildjob build-amd64
test-amd64-i386-i386-sid-netboot-pygrub all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-i386-i386-sid-netboot-pygrub arch i386
test-amd64-i386-i386-sid-netboot-pygrub buildjob build-i386
test-amd64-i386-i386-sid-netboot-pygrub debian_arch i386
test-amd64-i386-i386-sid-netboot-pygrub debian_bootloader pygrub
test-amd64-i386-i386-sid-netboot-pygrub debian_dist sid
test-amd64-i386-i386-sid-netboot-pygrub debian_diver current
test-amd64-i386-i386-sid-netboot-pygrub debian_method netboot
test-amd64-i386-i386-sid-netboot-pygrub kernbuildjob build-i386-pvops
test-amd64-i386-i386-sid-netboot-pygrub kernkind pvops
test-amd64-i386-i386-sid-netboot-pygrub toolstack xl
test-amd64-i386-i386-sid-netboot-pygrub xenbuildjob build-amd64
test-amd64-i386-i386-squeeze-netboot-pygrub all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-i386-i386-squeeze-netboot-pygrub arch i386
test-amd64-i386-i386-squeeze-netboot-pygrub buildjob build-i386
test-amd64-i386-i386-squeeze-netboot-pygrub debian_arch i386
test-amd64-i386-i386-squeeze-netboot-pygrub debian_bootloader pygrub
test-amd64-i386-i386-squeeze-netboot-pygrub debian_dist squeeze
test-amd64-i386-i386-squeeze-netboot-pygrub debian_diver current
test-amd64-i386-i386-squeeze-netboot-pygrub debian_method netboot
test-amd64-i386-i386-squeeze-netboot-pygrub kernbuildjob build-i386-pvops
test-amd64-i386-i386-squeeze-netboot-pygrub kernkind pvops
test-amd64-i386-i386-squeeze-netboot-pygrub toolstack xl
test-amd64-i386-i386-squeeze-netboot-pygrub xenbuildjob build-amd64
test-amd64-i386-i386-weekly-netinst-pygrub all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-i386-i386-weekly-netinst-pygrub arch i386
test-amd64-i386-i386-weekly-netinst-pygrub buildjob build-i386
test-amd64-i386-i386-weekly-netinst-pygrub debian_arch i386
test-amd64-i386-i386-weekly-netinst-pygrub debian_bootloader pygrub
test-amd64-i386-i386-weekly-netinst-pygrub debian_cd weekly
test-amd64-i386-i386-weekly-netinst-pygrub debian_method netinst
test-amd64-i386-i386-weekly-netinst-pygrub kernbuildjob build-i386-pvops
test-amd64-i386-i386-weekly-netinst-pygrub kernkind pvops
test-amd64-i386-i386-weekly-netinst-pygrub toolstack xl
test-amd64-i386-i386-weekly-netinst-pygrub xenbuildjob build-amd64
test-amd64-i386-i386-wheezy-netboot-pvgrub all_hostflags arch-i386,arch-xen-amd64,suite-wheezy,purpose-test
test-amd64-i386-i386-wheezy-netboot-pvgrub arch i386
test-amd64-i386-i386-wheezy-netboot-pvgrub buildjob build-i386
test-amd64-i386-i386-wheezy-netboot-pvgrub debian_arch i386
test-amd64-i386-i386-wheezy-netboot-pvgrub debian_bootloader pvgrub
test-amd64-i386-i386-wheezy-netboot-pvgrub debian_dist wheezy
test-amd64-i386-i386-wheezy-netboot-pvgrub debian_diver current
test-amd64-i386-i386-wheezy-netboot-pvgrub debian_method netboot
test-amd64-i386-i386-wheezy-netboot-pvgrub kernbuildjob build-i386-pvops
test-amd64-i386-i386-wheezy-netboot-pvgrub kernkind pvops
test-amd64-i386-i386-wheezy-netboot-pvgrub toolstack xl
test-amd64-i386-i386-wheezy-netboot-pvgrub xenbuildjob build-amd64
test-armhf-armhf-armhf-daily-netboot-pygrub all_hostflags arch-armhf,arch-xen-armhf,suite-wheezy,purpose-test
test-armhf-armhf-armhf-daily-netboot-pygrub arch armhf
test-armhf-armhf-armhf-daily-netboot-pygrub buildjob build-armhf
test-armhf-armhf-armhf-daily-netboot-pygrub debian_arch armhf
test-armhf-armhf-armhf-daily-netboot-pygrub debian_bootloader pygrub
test-armhf-armhf-armhf-daily-netboot-pygrub debian_dist daily
test-armhf-armhf-armhf-daily-netboot-pygrub debian_diver current
test-armhf-armhf-armhf-daily-netboot-pygrub debian_method netboot
test-armhf-armhf-armhf-daily-netboot-pygrub kernbuildjob build-armhf-pvops
test-armhf-armhf-armhf-daily-netboot-pygrub kernkind pvops
test-armhf-armhf-armhf-daily-netboot-pygrub toolstack xl
test-armhf-armhf-armhf-daily-netboot-pygrub xenbuildjob build-armhf
test-armhf-armhf-armhf-jessie-netboot-pygrub all_hostflags arch-armhf,arch-xen-armhf,suite-wheezy,purpose-test
test-armhf-armhf-armhf-jessie-netboot-pygrub arch armhf
test-armhf-armhf-armhf-jessie-netboot-pygrub buildjob build-armhf
test-armhf-armhf-armhf-jessie-netboot-pygrub debian_arch armhf
test-armhf-armhf-armhf-jessie-netboot-pygrub debian_bootloader pygrub
test-armhf-armhf-armhf-jessie-netboot-pygrub debian_dist jessie
test-armhf-armhf-armhf-jessie-netboot-pygrub debian_diver current
test-armhf-armhf-armhf-jessie-netboot-pygrub debian_method netboot
test-armhf-armhf-armhf-jessie-netboot-pygrub kernbuildjob build-armhf-pvops
test-armhf-armhf-armhf-jessie-netboot-pygrub kernkind pvops
test-armhf-armhf-armhf-jessie-netboot-pygrub toolstack xl
test-armhf-armhf-armhf-jessie-netboot-pygrub xenbuildjob build-armhf
test-armhf-armhf-armhf-sid-netboot-pygrub all_hostflags arch-armhf,arch-xen-armhf,suite-wheezy,purpose-test
test-armhf-armhf-armhf-sid-netboot-pygrub arch armhf
test-armhf-armhf-armhf-sid-netboot-pygrub buildjob build-armhf
test-armhf-armhf-armhf-sid-netboot-pygrub debian_arch armhf
test-armhf-armhf-armhf-sid-netboot-pygrub debian_bootloader pygrub
test-armhf-armhf-armhf-sid-netboot-pygrub debian_dist sid
test-armhf-armhf-armhf-sid-netboot-pygrub debian_diver current
test-armhf-armhf-armhf-sid-netboot-pygrub debian_method netboot
test-armhf-armhf-armhf-sid-netboot-pygrub kernbuildjob build-armhf-pvops
test-armhf-armhf-armhf-sid-netboot-pygrub kernkind pvops
test-armhf-armhf-armhf-sid-netboot-pygrub toolstack xl
test-armhf-armhf-armhf-sid-netboot-pygrub xenbuildjob build-armhf
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 10/24] distros: add support for installing Debian PV guests via d-i, flight and jobs
2015-05-12 15:40 ` Ian Campbell
@ 2015-05-12 16:08 ` Ian Jackson
0 siblings, 0 replies; 53+ messages in thread
From: Ian Jackson @ 2015-05-12 16:08 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("Re: [OSSTEST v6 10/24] distros: add support for installing Debian PV guests via d-i, flight and jobs"):
> On Tue, 2015-05-12 at 16:25 +0100, Ian Jackson wrote:
> > Shouldn't <GUEST>_dist be called <GUEST>_suite (which is probably
> > already set in various places) ?
>
> Hrm, yes I think so.
>
> Note that "set in various places" doesn't help here since our aim is to
> enumerate all the versions, not just use the one from the config. But I
> can at least use consistent naming for sure.
Indeed. Thanks.
> > > The distros flights differ substantially from the existing flights.
> > > Introduce make-distros-flight using the functionality previously
> > > refactored into mfi-common. The new flight tests all versions of
> > > Debian from Squeeze onward as an amd64, i386 and armhf guests (armhf
> > > from Jessie onwards only) using the usual smoke tests.
> >
> > Do you have an output from mg-show-flight-runvars you could share with
> > us ?
>
> >From a random (bt fairly recent) adhoc flight 37300 (sorted by job
> name). Looks like the results are still at
> http://www.chiark.greenend.org.uk/~xensrcts/logs/37300/ too
Thanks. That looks plausible.
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread
* [OSSTEST v6 11/24] distros: support booting Debian PV (d-i installed) guests with pvgrub.
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (9 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 10/24] distros: add support for installing Debian PV guests via d-i, flight and jobs Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-12 15:32 ` Ian Jackson
2015-05-06 9:56 ` [OSSTEST v6 12/24] distros: Support pvgrub for Wheezy too Ian Campbell
` (14 subsequent siblings)
25 siblings, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
This requires the use of the pv-grub-menu package which is in Jessie
onwards. (it is in wheezy-backports which is the subject of a
subsequent patch).
The bootloader to use is specified via a runvar {Guest}_bootloader.
Adjust make-distros-flight to use pvgrub for some subset of i386 and
amd64 guests to get coverage.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v3: Define and use arch_debian2xen and arch_xen2debian
Avoid pv-grub-x86_64.gz on i386 dom0, we don't built it there.
Fiddle with py vs pv grub stripy a bit.
---
Osstest.pm | 7 +++++++
Osstest/Debian.pm | 2 +-
make-distros-flight | 21 ++++++++++++++++++++-
ts-debian-di-install | 11 +++++++++--
4 files changed, 37 insertions(+), 4 deletions(-)
diff --git a/Osstest.pm b/Osstest.pm
index 7f72bc6..7ee00b1 100644
--- a/Osstest.pm
+++ b/Osstest.pm
@@ -38,6 +38,7 @@ BEGIN {
db_begin_work
ensuredir get_filecontents_core_quiet system_checked
nonempty visible_undef show_abs_time
+ %arch_debian2xen %arch_xen2debian
);
%EXPORT_TAGS = ( );
@@ -49,6 +50,12 @@ our $mjobdb;
our $dbh_tests;
+our %arch_debian2xen = qw(i386 x86_32
+ amd64 x86_64
+ armhf armhf);
+our %arch_xen2debian;
+$arch_xen2debian{$arch_debian2xen{$_}} = $_ foreach keys %arch_debian2xen;
+
#---------- static default config settings ----------
our %c = qw(
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index f7ef1d9..c3ef93f 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -710,7 +710,7 @@ sub preseed_create_guest ($$;@) {
my $suite= $xopts{Suite} || $c{DebianSuite};
- my $extra_packages;
+ my $extra_packages = "pv-grub-menu" if $xopts{PvMenuLst};
my $preseed_file= preseed_base($ho, $suite, $sfx, $extra_packages, %xopts);
$preseed_file.= (<<END);
diff --git a/make-distros-flight b/make-distros-flight
index 5067342..80df21b 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -55,14 +55,33 @@ test_matrix_branch_filter_callback () {
}
test_do_one_netboot () {
+ stripy bootloader pvgrub pygrub \
+ "$xenarch" "amd64" \
+ "$dom0arch" "i386" \
+ "$domU" "amd64" \
+ "$dist" "sid"
+
+ case ${dom0arch}_${domU}_${dist} in
+ arm*_arm*_*) bootloader="pygrub";; # no pvgrub for arm
+
+ # Needs a menu.lst, not present in Squeeze+ due to switch to grub2,
+ # workedaround in Jessie+ with pv-grub-menu package.
+ *_squeeze) bootloader="pygrub";;
+ *_wheezy) bootloader="pygrub";;
+
+ # pv-grub-x86_64.gz is not built by 32-bit dom0 userspace build.
+ i386_amd64_*) bootloader="pygrub";;
+ esac
+
job_create_test \
- test-$xenarch$kern-$dom0arch-$domU-$dist-netboot-pygrub \
+ test-$xenarch$kern-$dom0arch-$domU-$dist-netboot-$bootloader \
test-debian-di xl $xenarch $dom0arch \
kernbuildjob=${bfi}build-$dom0arch-$kernbuild \
debian_arch=$domU \
debian_dist=$dist \
debian_method=netboot \
debian_diver=current \
+ debian_bootloader=$bootloader \
all_hostflags=$most_hostflags
}
diff --git a/ts-debian-di-install b/ts-debian-di-install
index 9fabd56..07ad7e8 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -95,6 +95,8 @@ sub ginstall () {
my $method= $r{"$gho->{Guest}_method"};
my $tmpdir= "/root/$flight-$job-di";
+ my $bl= $r{"$gho->{Guest}_bootloader"};
+
target_cmd_root($ho, <<END);
rm -rf $tmpdir
mkdir $tmpdir
@@ -111,7 +113,7 @@ END
$suite = "sid" if $suite eq "daily";
- $ps_url = preseed_create_guest($gho, '', Suite=>$suite);
+ $ps_url = preseed_create_guest($gho, '', Suite=>$suite, PvMenuLst=>($bl eq "pvgrub"));
$extra_disk = "";
}
@@ -152,7 +154,12 @@ END
guest_await_shutdown($ho,$gho,3600);
guest_destroy($gho);
- my $blcfg = <<END;
+ my $xenarch = $arch_debian2xen{$arch};
+ my $pvgrub = "/usr/local/lib/xen/boot/pv-grub-$xenarch.gz";
+ my $blcfg = $bl eq "pvgrub" ? <<END : <<END;
+kernel = "$pvgrub"
+extra = "(hd0,0)/boot/grub/menu.lst"
+END
bootloader = "pygrub"
END
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 12/24] distros: Support pvgrub for Wheezy too.
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (10 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 11/24] distros: support booting Debian PV (d-i installed) guests with pvgrub Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-12 15:35 ` Ian Jackson
2015-05-06 9:56 ` [OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media Ian Campbell
` (13 subsequent siblings)
25 siblings, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
This requires us to install pv-grub-menu from backports, which we do
using a late_command.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v3:
- Remove spurious () from <<(END) (and the prexisting "" too)
- Remove $xopts{EnableBackports} and automatically handle the need to add
backports in preseed_base.
- Install via late_command not apt-setup, since the former has
issues, hence subject drops "attempt to..."
---
Osstest/Debian.pm | 44 +++++++++++++++++++++++++++++++++++++++++++-
make-distros-flight | 4 ++--
2 files changed, 45 insertions(+), 3 deletions(-)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index c3ef93f..5ec1126 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -697,6 +697,17 @@ END
d-i clock-setup/ntp-server string $ntpserver
END
+ # deb http://ftp.debian.org/debian/ wheezy-backports main
+ $preseed .= <<END if $extra_packages =~ m#\b/$suite-backports\b#;
+d-i apt-setup/local0/repository string \\
+ http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} $suite-backports main
+d-i apt-setup/local0/comment string $suite backports
+END
+# Jessie onwards...
+# $preseed .= <<END if $extra_packages =~ m#\b/$suite-backports\b#;
+#d-i apt-setup/services-select multiselect security, updates, backports
+#END
+
$preseed .= <<"END";
### END OF DEBIAN PRESEED BASE
@@ -710,7 +721,38 @@ sub preseed_create_guest ($$;@) {
my $suite= $xopts{Suite} || $c{DebianSuite};
- my $extra_packages = "pv-grub-menu" if $xopts{PvMenuLst};
+ my $extra_packages = "";
+ if ($xopts{PvMenuLst}) {
+ if ($suite =~ m/wheezy/) {
+ # pv-grub-menu/wheezy-backports + using apt-setup to add
+ # backports results in iproute, ifupdown and
+ # isc-dhcp-client getting removed because tasksel's
+ # invocation of apt-get install somehow decides the
+ # iproute2 from wheezy-backports is a thing it wants to
+ # install. So instead lets fake it with a late command...
+ #
+ # This also has the bonus of working round an issue with
+ # 1.2.1~bpo70+1 which created an invalid menu.lst using
+ # "root(/dev/xvda,0)" which pvgrub cannot parse because
+ # the Grub device.map isn't present at pkgsel/include time
+ # but it is by late_command time. This was fixed by
+ # version 1.3 which is in Jessie onwards.
+ preseed_hook_command($ho, 'late_command', $sfx, <<END);
+#!/bin/sh
+set -ex
+
+(
+ echo
+ echo \\\# $suite backports
+ echo deb http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} $suite-backports main
+) >> /target/etc/apt/sources.list
+in-target apt-get update
+in-target apt-get install -y -t wheezy-backports pv-grub-menu
+END
+ } else {
+ $extra_packages = "pv-grub-menu";
+ }
+ }
my $preseed_file= preseed_base($ho, $suite, $sfx, $extra_packages, %xopts);
$preseed_file.= (<<END);
diff --git a/make-distros-flight b/make-distros-flight
index 80df21b..97df89a 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -65,9 +65,9 @@ test_do_one_netboot () {
arm*_arm*_*) bootloader="pygrub";; # no pvgrub for arm
# Needs a menu.lst, not present in Squeeze+ due to switch to grub2,
- # workedaround in Jessie+ with pv-grub-menu package.
+ # workedaround in Wheezy+ with pv-grub-menu package (backports in Wheezy,
+ # in Jessie+ main).
*_squeeze) bootloader="pygrub";;
- *_wheezy) bootloader="pygrub";;
# pv-grub-x86_64.gz is not built by 32-bit dom0 userspace build.
i386_amd64_*) bootloader="pygrub";;
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 12/24] distros: Support pvgrub for Wheezy too.
2015-05-06 9:56 ` [OSSTEST v6 12/24] distros: Support pvgrub for Wheezy too Ian Campbell
@ 2015-05-12 15:35 ` Ian Jackson
2015-05-12 15:49 ` Ian Campbell
0 siblings, 1 reply; 53+ messages in thread
From: Ian Jackson @ 2015-05-12 15:35 UTC (permalink / raw)
To: Ian Campbell; +Cc: ian.jackson, xen-devel
Ian Campbell writes ("[OSSTEST v6 12/24] distros: Support pvgrub for Wheezy too."):
> This requires us to install pv-grub-menu from backports, which we do
> using a late_command.
...
> +# $preseed .= <<END if $extra_packages =~ m#\b/$suite-backports\b#;
This is quite odd. It expects the _suite_ name in $extra_packages.
But AFAICT nothing ever passes that anyway.
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 12/24] distros: Support pvgrub for Wheezy too.
2015-05-12 15:35 ` Ian Jackson
@ 2015-05-12 15:49 ` Ian Campbell
0 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-12 15:49 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
On Tue, 2015-05-12 at 16:35 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[OSSTEST v6 12/24] distros: Support pvgrub for Wheezy too."):
> > This requires us to install pv-grub-menu from backports, which we do
> > using a late_command.
> ...
> > +# $preseed .= <<END if $extra_packages =~ m#\b/$suite-backports\b#;
>
> This is quite odd. It expects the _suite_ name in $extra_packages.
> But AFAICT nothing ever passes that anyway.
It was checking for uses of "$package/$suite-backports" which is a
syntax supported by apt (and hence d-i) for installing things from the
non-default suite.
I think that functionality isn't actually used any more, because it
didn't actually work in the d-i context (I've forgotten why) and so we
do it via a late_command now instead (later in this patch).
So I think I should ditch that whole bit.
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread
* [OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media.
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (11 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 12/24] distros: Support pvgrub for Wheezy too Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-12 15:52 ` Ian Jackson
2015-05-06 9:56 ` [OSSTEST v6 14/24] Test pygrub and pvgrub on the regular flights Ian Campbell
` (12 subsequent siblings)
25 siblings, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
The netinst media are iso images containing a base Debian install and
some (image size dependent) additional tasks.
On x86 the "multiarch" iso flavour contains a Xen capable kernel for
both i386 and amd64 so use that.
This adds support for two classes of ISO, the CD sized ones which are
built nightly and the DVD sized ones which are built weekly.
The images are downloaded using jigdo which sources the majority of
the data from a local Debian mirror, for this reason I have not
worried about the fact that the i386 and amd64 tests are downloading
the same thing (adding a specific download job would require finding
up to 4GB of scratch space for each flight).
The ISOs booted using pygrub which can extract the kernel and initrd
from a ISO image. The resulting guests are also booted with pygrub
since the pv-grub-menu package is not available on the ISO images and
we have pvgrub coverage from the netboot tests.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v4: Support http_proxy in jigdo download
v3:
- Indent jigdo-lite script line to improve readability
- Wrap bootloader_args
- Fetch URL on target, to get a timeout. Use wget since that is arranged to
be present.
- include bootloader in test name.
---
Osstest/Debian.pm | 7 ++++--
make-distros-flight | 23 ++++++++++++++++++
ts-debian-di-install | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 95 insertions(+), 2 deletions(-)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 5ec1126..a153f34 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -624,8 +624,6 @@ sub preseed_base ($$$$;@) {
preseed_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar');
my $preseed = <<"END";
-d-i mirror/suite string $suite
-
d-i debian-installer/locale string en_GB
d-i console-keymaps-at/keymap select gb
d-i keyboard-configuration/xkb-keymap string en_GB
@@ -697,6 +695,11 @@ END
d-i clock-setup/ntp-server string $ntpserver
END
+ # For CDROM the suite is part of the image
+ $preseed .= <<END unless $xopts{CDROM};
+d-i mirror/suite string $suite
+END
+
# deb http://ftp.debian.org/debian/ wheezy-backports main
$preseed .= <<END if $extra_packages =~ m#\b/$suite-backports\b#;
d-i apt-setup/local0/repository string \\
diff --git a/make-distros-flight b/make-distros-flight
index 97df89a..e3ec8f7 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -85,6 +85,20 @@ test_do_one_netboot () {
all_hostflags=$most_hostflags
}
+test_do_one_netinst () {
+ # Always pygrub since no pv-grub-menu on CD
+ job_create_test \
+ test-$xenarch$kern-$dom0arch-$domU-$cd-netinst-pygrub \
+ test-debian-di xl $xenarch $dom0arch \
+ kernbuildjob=${bfi}build-$dom0arch-$kernbuild \
+ debian_arch=$domU \
+ debian_cd=$cd \
+ debian_method=netinst \
+ debian_bootloader=pygrub \
+ all_hostflags=$most_hostflags
+
+}
+
test_matrix_do_one () {
case ${xenarch} in
amd64) domUarches="amd64 i386";;
@@ -103,6 +117,15 @@ test_matrix_do_one () {
done
+ for cd in current weekly ; do
+ case ${domU}_${dist} in
+ armhf_*) continue;; # No iso targets for armhf
+ *) ;;
+ esac
+
+ test_do_one_netinst
+
+ done
done
}
diff --git a/ts-debian-di-install b/ts-debian-di-install
index 07ad7e8..5dfb0a1 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -46,6 +46,63 @@ sub prep () {
target_cmd_root($ho, "umount $gho->{Lvdev} ||:");
}
+sub setup_netinst($$$)
+{
+ my ($didir, $arch, $cd) = @_;
+ my %arch_props = (
+ amd64 => { PathArch => "multi-arch", FileArch => "amd64-i386", IsoPath => "/install.amd/xen" },
+ i386 => { PathArch => "multi-arch", FileArch => "amd64-i386", IsoPath => "/install.386/xen" },
+ armhf => { PathArch => "armhf", FileArch => "armhf", IsoPath => "/install.armhf" }
+ );
+ my $props = $arch_props{$arch} or die "Unknown arch $arch";
+
+ target_install_packages($ho, qw(jigdo-file));
+
+ my $baseurl = $cd eq "current" ?
+ "http://cdimage.debian.org/debian-cd/current/$props->{PathArch}/jigdo-cd" :
+ "http://cdimage.debian.org/cdimage/weekly-builds/$props->{PathArch}/jigdo-cd";
+
+ my $filebase;
+
+ # Use the MD5SUMs file as an index
+ logm("Fetch index from $baseurl/MD5SUMS");
+ my $idx = target_cmd_output_root($ho, "wget --quiet -O - $baseurl/MD5SUMS");
+ foreach (split /\n/, $idx) {
+ m/^[0-9a-f]{32} (debian-.*-$props->{FileArch}-netinst)\.iso$/ or next;
+ $filebase = $1;
+ last;
+ }
+
+ die unless $filebase;
+
+ logm("Downloading $baseurl/$filebase.jigdo");
+ my $netinst_jigdo = "$baseurl/$filebase.jigdo";
+ # Jigdo uses wget internally, and so obeys $http_proxy. This seems
+ # simpler than the advice at
+ # https://www.debian.org/CD/jigdo-cd/#faq which is to edit
+ # ~/.jigdo-lite and change the wgetOpts setting.
+ my $useproxy = $c{DebianMirrorProxy} // $c{HttpProxy};
+ my $cmd = '';
+ $cmd .= <<END if $useproxy;
+ export http_proxy=$useproxy
+END
+ $cmd .= <<END;
+ cd $didir && jigdo-lite --noask $netinst_jigdo
+END
+ # Jigdo seems to use /etc/apt/sources.list or something, so this
+ # just works using the already configured mirror without
+ # additional configuration, which is good because there doesn't
+ # seem to be any support for such things, at least in Squeeze.
+ target_cmd_root($ho, $cmd, 3600);
+ store_runvar("$gho->{Guest}_netinst_jigdo", $netinst_jigdo);
+
+ return (<<END, "\"file:$didir/$filebase.iso,xvdd:cdrom,r\",");
+bootloader = "pygrub"
+bootloader_args = ["--kernel=$props->{IsoPath}/vmlinuz",
+ "--ramdisk=$props->{IsoPath}/initrd.gz"]
+END
+}
+
sub setup_netboot($$$)
{
my ($didir, $arch, $suite) = @_;
@@ -117,6 +174,16 @@ END
$extra_disk = "";
}
+ elsif ($method eq "netinst" )
+ {
+ my $cd = $r{"$gho->{Guest}_cd"};
+
+ logm("$method $cd/$arch");
+
+ ($method_cfg,$extra_disk) = setup_netinst($tmpdir, $arch, $cd);
+
+ $ps_url = preseed_create_guest($gho, '', CDROM=>1);
+ }
else
{
die "$method";
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media.
2015-05-06 9:56 ` [OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media Ian Campbell
@ 2015-05-12 15:52 ` Ian Jackson
2015-05-12 16:14 ` Ian Jackson
2015-05-13 9:06 ` Ian Campbell
0 siblings, 2 replies; 53+ messages in thread
From: Ian Jackson @ 2015-05-12 15:52 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("[OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media."):
> The netinst media are iso images containing a base Debian install and
> some (image size dependent) additional tasks.
Most of this looks good to me.
> + amd64 => { PathArch => "multi-arch", FileArch => "amd64-i386", IsoPath => "/install.amd/xen" },
> + i386 => { PathArch => "multi-arch", FileArch => "amd64-i386", IsoPath => "/install.386/xen" },
> + armhf => { PathArch => "armhf", FileArch => "armhf", IsoPath => "/install.armhf" }
Wrap damage on my screen. I appreciate that you want to retain the
tabular nature, but perhaps
my %arch_props = (
amd64 => [ "multi-arch", "amd64-i386", "/install.amd/xen" },
i386 => [ "multi-arch", "amd64-i386", "/install.386/xen" },
armhf => [ "armhf", "armhf", "/install.armhf" },
)
my ( $path_arch, $file_arch, $iso_path ) =
@{ $arch_props{$arch} };
Or something ?
> + my $baseurl = $cd eq "current" ?
> + "http://cdimage.debian.org/debian-cd/current/$props->{PathArch}/jigdo-cd" :
> + "http://cdimage.debian.org/cdimage/weekly-builds/$props->{PathArch}/jigdo-cd";
This should surely come from a runvar (or perhaps a config option)
rather than being hardcoded.
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media.
2015-05-12 15:52 ` Ian Jackson
@ 2015-05-12 16:14 ` Ian Jackson
2015-05-13 9:16 ` Ian Campbell
2015-05-13 9:06 ` Ian Campbell
1 sibling, 1 reply; 53+ messages in thread
From: Ian Jackson @ 2015-05-12 16:14 UTC (permalink / raw)
To: Ian Campbell, xen-devel
Ian Jackson writes ("Re: [OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media."):
> Ian Campbell writes ("[OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media."):
> > + my $baseurl = $cd eq "current" ?
> > + "http://cdimage.debian.org/debian-cd/current/$props->{PathArch}/jigdo-cd" :
> > + "http://cdimage.debian.org/cdimage/weekly-builds/$props->{PathArch}/jigdo-cd";
>
> This should surely come from a runvar (or perhaps a config option)
> rather than being hardcoded.
It occurs to me that doing this in a runvar with the url would mean
that the runvar "<GUEST>_cd" which currently contains a string like
`current' would go away.
I wonder if a similar consideration ought to apply to the
`<GUEST>_diver' thing. (Why `diver' rather than `divert', JOOI?)
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media.
2015-05-12 16:14 ` Ian Jackson
@ 2015-05-13 9:16 ` Ian Campbell
0 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-13 9:16 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
On Tue, 2015-05-12 at 17:14 +0100, Ian Jackson wrote:
> Ian Jackson writes ("Re: [OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media."):
> > Ian Campbell writes ("[OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media."):
> > > + my $baseurl = $cd eq "current" ?
> > > + "http://cdimage.debian.org/debian-cd/current/$props->{PathArch}/jigdo-cd" :
> > > + "http://cdimage.debian.org/cdimage/weekly-builds/$props->{PathArch}/jigdo-cd";
> >
> > This should surely come from a runvar (or perhaps a config option)
> > rather than being hardcoded.
>
> It occurs to me that doing this in a runvar with the url would mean
> that the runvar "<GUEST>_cd" which currently contains a string like
> `current' would go away.
If you prefer, sure.
> I wonder if a similar consideration ought to apply to the
> `<GUEST>_diver' thing. (Why `diver' rather than `divert', JOOI?)
It diver for debian-installer version (cf TftpDiVersion).
It can either be "osstest", which means "use $c{TftpDiVersion}" (i.e.
the same version as host installs use, updated with
mg-update-debian-installer) or it can be "daily", meaning the daily
images from [0] or a string which is any directory entry in [1], most
commonly "current" (and not many historical versions are kept there by
Debian).
Ian.
[0] http://d-i.debian.org/daily-images/$arch/daily/netboot
[1] ftp://ftp.debian.org/debian/dists/wheezy/main/installer-amd64/
^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media.
2015-05-12 15:52 ` Ian Jackson
2015-05-12 16:14 ` Ian Jackson
@ 2015-05-13 9:06 ` Ian Campbell
2015-05-14 18:01 ` Ian Jackson
1 sibling, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-13 9:06 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
On Tue, 2015-05-12 at 16:52 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media."):
> > The netinst media are iso images containing a base Debian install and
> > some (image size dependent) additional tasks.
>
> Most of this looks good to me.
>
> > + amd64 => { PathArch => "multi-arch", FileArch => "amd64-i386", IsoPath => "/install.amd/xen" },
> > + i386 => { PathArch => "multi-arch", FileArch => "amd64-i386", IsoPath => "/install.386/xen" },
> > + armhf => { PathArch => "armhf", FileArch => "armhf", IsoPath => "/install.armhf" }
>
> Wrap damage on my screen. I appreciate that you want to retain the
> tabular nature, but perhaps
>
> my %arch_props = (
> amd64 => [ "multi-arch", "amd64-i386", "/install.amd/xen" },
> i386 => [ "multi-arch", "amd64-i386", "/install.386/xen" },
> armhf => [ "armhf", "armhf", "/install.armhf" },
> )
> my ( $path_arch, $file_arch, $iso_path ) =
> @{ $arch_props{$arch} };
>
> Or something ?
More Perl-fu than I could muster by myself, but looks good to me ;-)
> > + my $baseurl = $cd eq "current" ?
> > + "http://cdimage.debian.org/debian-cd/current/$props->{PathArch}/jigdo-cd" :
> > + "http://cdimage.debian.org/cdimage/weekly-builds/$props->{PathArch}/jigdo-cd";
>
> This should surely come from a runvar (or perhaps a config option)
> rather than being hardcoded.
$cd above came from $r{"$gho->{Guest}_cd"} and takes the values either
"current" or "weekly".
I could push the logic which mas those to an actual URL into
make-distros-flight if you would prefer? Or perhaps push the two
prefixes (up to but not including "$props->.../jigdo-cd") into the cfg
file? So it would become "$c{DebianCdURL_
$cd}/$props->{PathArch}/jigdo-cd"
The latter sounds a bit preferable I think, hardcoding URLs in
make-flight seems as wrong as hardcoding them here.
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media.
2015-05-13 9:06 ` Ian Campbell
@ 2015-05-14 18:01 ` Ian Jackson
2015-05-15 8:58 ` Ian Campbell
0 siblings, 1 reply; 53+ messages in thread
From: Ian Jackson @ 2015-05-14 18:01 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("Re: [OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media."):
> On Tue, 2015-05-12 at 16:52 +0100, Ian Jackson wrote:
> > Wrap damage on my screen. I appreciate that you want to retain the
> > tabular nature, but perhaps
> >
> > my %arch_props = (
> > amd64 => [ "multi-arch", "amd64-i386", "/install.amd/xen" },
> > i386 => [ "multi-arch", "amd64-i386", "/install.386/xen" },
> > armhf => [ "armhf", "armhf", "/install.armhf" },
> > )
> > my ( $path_arch, $file_arch, $iso_path ) =
> > @{ $arch_props{$arch} };
...
> More Perl-fu than I could muster by myself, but looks good to me ;-)
You didn't spot the [...} then ? :-)
> > > + my $baseurl = $cd eq "current" ?
> > > + "http://cdimage.debian.org/debian-cd/current/$props->{PathArch}/jigdo-cd" :
> > > + "http://cdimage.debian.org/cdimage/weekly-builds/$props->{PathArch}/jigdo-cd";
> >
> > This should surely come from a runvar (or perhaps a config option)
> > rather than being hardcoded.
>
> $cd above came from $r{"$gho->{Guest}_cd"} and takes the values either
> "current" or "weekly".
>
> I could push the logic which mas those to an actual URL into
> make-distros-flight if you would prefer? Or perhaps push the two
> prefixes (up to but not including "$props->.../jigdo-cd") into the cfg
> file? So it would become "$c{DebianCdURL_
> $cd}/$props->{PathArch}/jigdo-cd"
(We had a conversation IRL about some of this, but:)
Hardcoding something in make-flight is not as bad as hardcoding it in
the script, IMO, but:
> The latter sounds a bit preferable I think, hardcoding URLs in
> make-flight seems as wrong as hardcoding them here.
Having it in the config is best. You might want to consider whether
the config ought to be plumbed through a runvar by make-flight, so
that it is possible to (eg) use cs-adjust-flight or whatever to do
tests of different things.
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media.
2015-05-14 18:01 ` Ian Jackson
@ 2015-05-15 8:58 ` Ian Campbell
0 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-15 8:58 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
On Thu, 2015-05-14 at 19:01 +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media."):
> > On Tue, 2015-05-12 at 16:52 +0100, Ian Jackson wrote:
> > > Wrap damage on my screen. I appreciate that you want to retain the
> > > tabular nature, but perhaps
> > >
> > > my %arch_props = (
> > > amd64 => [ "multi-arch", "amd64-i386", "/install.amd/xen" },
> > > i386 => [ "multi-arch", "amd64-i386", "/install.386/xen" },
> > > armhf => [ "armhf", "armhf", "/install.armhf" },
> > > )
> > > my ( $path_arch, $file_arch, $iso_path ) =
> > > @{ $arch_props{$arch} };
> ...
> > More Perl-fu than I could muster by myself, but looks good to me ;-)
>
> You didn't spot the [...} then ? :-)
I did when I copied it, before I ran it even.
Actually, this all goes away (or rather becomes case..esac in the
make-flight bash script) with the proposal below.
> > > > + my $baseurl = $cd eq "current" ?
> > > > + "http://cdimage.debian.org/debian-cd/current/$props->{PathArch}/jigdo-cd" :
> > > > + "http://cdimage.debian.org/cdimage/weekly-builds/$props->{PathArch}/jigdo-cd";
> > >
> > > This should surely come from a runvar (or perhaps a config option)
> > > rather than being hardcoded.
> >
> > $cd above came from $r{"$gho->{Guest}_cd"} and takes the values either
> > "current" or "weekly".
> >
> > I could push the logic which mas those to an actual URL into
> > make-distros-flight if you would prefer? Or perhaps push the two
> > prefixes (up to but not including "$props->.../jigdo-cd") into the cfg
> > file? So it would become "$c{DebianCdURL_
> > $cd}/$props->{PathArch}/jigdo-cd"
>
> (We had a conversation IRL about some of this, but:)
>
> Hardcoding something in make-flight is not as bad as hardcoding it in
> the script, IMO, but:
>
> > The latter sounds a bit preferable I think, hardcoding URLs in
> > make-flight seems as wrong as hardcoding them here.
>
> Having it in the config is best. You might want to consider whether
> the config ought to be plumbed through a runvar by make-flight, so
> that it is possible to (eg) use cs-adjust-flight or whatever to do
> tests of different things.
After our RL conversation I've moved the URLs to make-flight and
actually it looks ok, I've got a few kinks to work out but I think it
should work ok.
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread
* [OSSTEST v6 14/24] Test pygrub and pvgrub on the regular flights
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (12 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 13/24] distros: support PV guest install from Debian netinst media Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-12 15:54 ` Ian Jackson
2015-05-06 9:56 ` [OSSTEST v6 15/24] distros: add branch infrastructure Ian Campbell
` (11 subsequent siblings)
25 siblings, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Since we now have the ability to test these drop one of each of
pygrub, pvgrub-32 and pvgrub-64 into the standard flights. Omitting
the {Guest}_diver runvar causes ts-debian-di-install to use the d-i
images in the location configured via TftpDiVersion, so they are
Version Controlled along with the d-i version used for the host.
This adds three new jobs:
test-amd64-amd64-amd64-pvgrub:
all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
arch amd64
buildjob build-amd64
debian_arch amd64
debian_bootloader pvgrub
debian_dist wheezy
debian_method netboot
kernbuildjob build-amd64-pvops
kernkind pvops
toolstack xl
xenbuildjob build-amd64
test-amd64-amd64-i386-pvgrub:
all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
arch amd64
buildjob build-amd64
debian_arch i386
debian_bootloader pvgrub
debian_dist wheezy
debian_method netboot
kernbuildjob build-amd64-pvops
kernkind pvops
toolstack xl
xenbuildjob build-amd64
test-amd64-amd64-pygrub:
all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
arch amd64
buildjob build-amd64
debian_arch amd64
debian_bootloader pygrub
debian_dist wheezy
debian_method netboot
kernbuildjob build-amd64-pvops
kernkind pvops
toolstack xl
xenbuildjob build-amd64
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v3: added runvar details
---
make-flight | 39 +++++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)
diff --git a/make-flight b/make-flight
index ec0a20c..ca48b81 100755
--- a/make-flight
+++ b/make-flight
@@ -303,6 +303,42 @@ do_passthrough_tests () {
done
}
+do_pygrub_tests () {
+ if [ $xenarch != amd64 -o $dom0arch != amd64 -o "$kern" != "" ]; then
+ return
+ fi
+
+ job_create_test test-$xenarch$kern-$dom0arch-pygrub \
+ test-debian-di xl $xenarch $dom0arch \
+ debian_arch=amd64 \
+ debian_dist=$guestsuite \
+ debian_method=netboot \
+ debian_bootloader=pygrub \
+ all_hostflags=$most_hostflags
+}
+
+do_pvgrub_tests () {
+ if [ $xenarch != amd64 -o $dom0arch != amd64 -o "$kern" != "" ]; then
+ return
+ fi
+
+ job_create_test test-$xenarch$kern-$dom0arch-amd64-pvgrub \
+ test-debian-di xl $xenarch $dom0arch \
+ debian_arch=amd64 \
+ debian_dist=$guestsuite \
+ debian_method=netboot \
+ debian_bootloader=pvgrub \
+ all_hostflags=$most_hostflags \
+
+ job_create_test test-$xenarch$kern-$dom0arch-i386-pvgrub \
+ test-debian-di xl $xenarch $dom0arch \
+ debian_arch=i386 \
+ debian_dist=$guestsuite \
+ debian_method=netboot \
+ debian_bootloader=pvgrub \
+ all_hostflags=$most_hostflags
+}
+
do_pv_debian_tests () {
xsms=$(xenbranch_xsm_variants)
@@ -427,6 +463,9 @@ test_matrix_do_one () {
fi
#do_passthrough_tests
+
+ do_pygrub_tests
+ do_pvgrub_tests
}
test_matrix_iterate
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 14/24] Test pygrub and pvgrub on the regular flights
2015-05-06 9:56 ` [OSSTEST v6 14/24] Test pygrub and pvgrub on the regular flights Ian Campbell
@ 2015-05-12 15:54 ` Ian Jackson
2015-05-13 9:11 ` Ian Campbell
0 siblings, 1 reply; 53+ messages in thread
From: Ian Jackson @ 2015-05-12 15:54 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("[OSSTEST v6 14/24] Test pygrub and pvgrub on the regular flights"):
> Since we now have the ability to test these drop one of each of
> pygrub, pvgrub-32 and pvgrub-64 into the standard flights. Omitting
> the {Guest}_diver runvar causes ts-debian-di-install to use the d-i
> images in the location configured via TftpDiVersion, so they are
> Version Controlled along with the d-i version used for the host.
>
> This adds three new jobs:
I think we should consider at this point which jobs we can drop as a
result of these new ones being added. Ideally this patch would
contain hunks adding "delete this later" style comments to some
existing tests.
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 14/24] Test pygrub and pvgrub on the regular flights
2015-05-12 15:54 ` Ian Jackson
@ 2015-05-13 9:11 ` Ian Campbell
2015-05-14 17:58 ` Ian Jackson
0 siblings, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-13 9:11 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
On Tue, 2015-05-12 at 16:54 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[OSSTEST v6 14/24] Test pygrub and pvgrub on the regular flights"):
> > Since we now have the ability to test these drop one of each of
> > pygrub, pvgrub-32 and pvgrub-64 into the standard flights. Omitting
> > the {Guest}_diver runvar causes ts-debian-di-install to use the d-i
> > images in the location configured via TftpDiVersion, so they are
> > Version Controlled along with the d-i version used for the host.
> >
> > This adds three new jobs:
>
> I think we should consider at this point which jobs we can drop as a
> result of these new ones being added. Ideally this patch would
> contain hunks adding "delete this later" style comments to some
> existing tests.
After these tests are added the set of test-* jobs (taken from a random
adhoc flight I ran) are below. I'm not sure which could sensibly be
dropped. I appreciate this is adding a lot of jobs though :-(
test-amd64-amd64-amd64-pvgrub
test-amd64-amd64-i386-pvgrub
test-amd64-amd64-libvirt
test-amd64-amd64-libvirt-qcow2
test-amd64-amd64-libvirt-raw
test-amd64-amd64-libvirt-vhd
test-amd64-amd64-libvirt-xsm
test-amd64-amd64-pair
test-amd64-amd64-pygrub
test-amd64-amd64-rumpuserxen-amd64
test-amd64-amd64-xl
test-amd64-amd64-xl-credit2
test-amd64-amd64-xl-multivcpu
test-amd64-amd64-xl-pvh-amd
test-amd64-amd64-xl-pvh-intel
test-amd64-amd64-xl-qcow2
test-amd64-amd64-xl-qemut-debianhvm-amd64
test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm
test-amd64-amd64-xl-qemut-win7-amd64
test-amd64-amd64-xl-qemut-winxpsp3
test-amd64-amd64-xl-qemuu-debianhvm-amd64
test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm
test-amd64-amd64-xl-qemuu-ovmf-amd64
test-amd64-amd64-xl-qemuu-win7-amd64
test-amd64-amd64-xl-qemuu-winxpsp3
test-amd64-amd64-xl-raw
test-amd64-amd64-xl-sedf
test-amd64-amd64-xl-sedf-pin
test-amd64-amd64-xl-vhd
test-amd64-amd64-xl-win7-amd64
test-amd64-amd64-xl-winxpsp3
test-amd64-amd64-xl-xsm
test-amd64-i386-freebsd10-amd64
test-amd64-i386-freebsd10-i386
test-amd64-i386-libvirt
test-amd64-i386-libvirt-qcow2
test-amd64-i386-libvirt-raw
test-amd64-i386-libvirt-vhd
test-amd64-i386-libvirt-xsm
test-amd64-i386-pair
test-amd64-i386-qemut-rhel6hvm-amd
test-amd64-i386-qemut-rhel6hvm-intel
test-amd64-i386-qemuu-rhel6hvm-amd
test-amd64-i386-qemuu-rhel6hvm-intel
test-amd64-i386-rhel6hvm-amd
test-amd64-i386-rhel6hvm-intel
test-amd64-i386-rumpuserxen-i386
test-amd64-i386-xl
test-amd64-i386-xl-qcow2
test-amd64-i386-xl-qemut-debianhvm-amd64
test-amd64-i386-xl-qemut-debianhvm-amd64-xsm
test-amd64-i386-xl-qemut-win7-amd64
test-amd64-i386-xl-qemut-winxpsp3
test-amd64-i386-xl-qemut-winxpsp3-vcpus1
test-amd64-i386-xl-qemuu-debianhvm-amd64
test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm
test-amd64-i386-xl-qemuu-ovmf-amd64
test-amd64-i386-xl-qemuu-win7-amd64
test-amd64-i386-xl-qemuu-winxpsp3
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1
test-amd64-i386-xl-raw
test-amd64-i386-xl-vhd
test-amd64-i386-xl-win7-amd64
test-amd64-i386-xl-winxpsp3
test-amd64-i386-xl-winxpsp3-vcpus1
test-amd64-i386-xl-xsm
test-armhf-armhf-libvirt
test-armhf-armhf-libvirt-qcow2
test-armhf-armhf-libvirt-raw
test-armhf-armhf-libvirt-vhd
test-armhf-armhf-libvirt-xsm
test-armhf-armhf-xl
test-armhf-armhf-xl-credit2
test-armhf-armhf-xl-midway
test-armhf-armhf-xl-multivcpu
test-armhf-armhf-xl-qcow2
test-armhf-armhf-xl-raw
test-armhf-armhf-xl-sedf
test-armhf-armhf-xl-sedf-pin
test-armhf-armhf-xl-vhd
test-armhf-armhf-xl-xsm
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 14/24] Test pygrub and pvgrub on the regular flights
2015-05-13 9:11 ` Ian Campbell
@ 2015-05-14 17:58 ` Ian Jackson
0 siblings, 0 replies; 53+ messages in thread
From: Ian Jackson @ 2015-05-14 17:58 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("Re: [OSSTEST v6 14/24] Test pygrub and pvgrub on the regular flights"):
> On Tue, 2015-05-12 at 16:54 +0100, Ian Jackson wrote:
> > I think we should consider at this point which jobs we can drop as a
> > result of these new ones being added. Ideally this patch would
> > contain hunks adding "delete this later" style comments to some
> > existing tests.
>
> After these tests are added the set of test-* jobs (taken from a random
> adhoc flight I ran) are below. I'm not sure which could sensibly be
> dropped. I appreciate this is adding a lot of jobs though :-(
Hmm. Well, fair enough, then, I guess.
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread
* [OSSTEST v6 15/24] distros: add branch infrastructure
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (13 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 14/24] Test pygrub and pvgrub on the regular flights Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-12 16:03 ` Ian Jackson
2015-05-06 9:56 ` [OSSTEST v6 16/24] distros: Run a flight over the weekend Ian Campbell
` (10 subsequent siblings)
25 siblings, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Since the distro nightlies are not version controlled we cannot use
the usual mechanisms for detecting regressions. Special case things
appropriately. We use an OLD_REVISION of "flight-NNN" to signify that
the old revision is another flight and not a tree revision.
A grep over $NEW_REVISION needed adjusting since NEW_REVISION is empty
in this mode, leading to "grep <filename>" which hangs waiting for
stdin.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v3:
Handle within cr-daily-branch, since ap-fetch-version* don't make sense for
a branch such as this.
---
cr-daily-branch | 36 ++++++++++++++++++++++++++++--------
cri-common | 1 +
2 files changed, 29 insertions(+), 8 deletions(-)
diff --git a/cr-daily-branch b/cr-daily-branch
index c7d1071..55afadf 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -68,23 +68,34 @@ fetch_version () {
printf '%s\n' "$fetch_version_result"
}
-treeurl=`./ap-print-url $branch`
+case $branch in
+ distros)
+ treeurl=none;;
+ *)
+ treeurl=`./ap-print-url $branch`;;
+esac
force_baseline=false
skipidentical=true
wantpush=$OSSTEST_PUSH
-if [ "x$OLD_REVISION" = x ]; then
- OLD_REVISION="`./ap-fetch-version-old $branch`"
- export OLD_REVISION
-fi
-
check_tested () {
./sg-check-tested --debug --branch=$branch \
--blessings=${DAILY_BRANCH_TESTED_BLESSING:-$OSSTEST_BLESSING} \
"$@"
}
+if [ "x$OLD_REVISION" = x ]; then
+ case $branch in
+ distros)
+ OSSTEST_NO_BASELINE=y
+ OLD_REVISION=flight-`check_tested`
+ ;;
+ *) OLD_REVISION="`./ap-fetch-version-old $branch`";;
+ esac
+ export OLD_REVISION
+fi
+
if [ "x$OSSTEST_NO_BASELINE" != xy ] ; then
testedflight=`check_tested --revision-$tree="$OLD_REVISION"`
@@ -227,6 +238,11 @@ if [ "x$OLD_REVISION" = xdetermine-late ]; then
OLD_REVISION="`./ap-fetch-version-baseline-late $branch $NEW_REVISION`"
fi
+case $branch in
+distros) makeflight=./make-distros-flight ;;
+*) makeflight=./make-flight ;;
+esac
+
if [ "x$NEW_REVISION" = "x$OLD_REVISION" ]; then
wantpush=false
for checkbranch in x $BRANCHES_ALWAYS; do
@@ -241,7 +257,7 @@ if [ "x$NEW_REVISION" = "x$OLD_REVISION" ]; then
fi
$DAILY_BRANCH_PREMAKE_HOOK
-flight=`./make-flight $branch $xenbranch $OSSTEST_BLESSING "$@"`
+flight=`$makeflight $branch $xenbranch $OSSTEST_BLESSING "$@"`
$DAILY_BRANCH_POSTMAKE_HOOK
heading=tmp/$flight.heading-info
@@ -261,6 +277,10 @@ fi
revlog=tmp/$flight.revision-log
case "$NEW_REVISION/$OLD_REVISION" in
+/flight-[0-9]*)
+ echo >&2 "SGR COMPARISON AGAINST ${OLD_REVISION}"
+ sgr_args+=" --that-flight=${OLD_REVISION#flight-}"
+ ;;
*/*[^0-9a-f]* | *[^0-9a-f]*/*)
echo >&2 "NO SGR COMPARISON badchar $NEW_REVISION/$OLD_REVISION"
;;
@@ -321,7 +341,7 @@ start_email $flight $branch "$sgr_args" "$subject_prefix"
push=false
if grep '^tolerable$' $mrof >/dev/null 2>&1; then push=$wantpush; fi
if test -f $branch.force; then push=$OSSTEST_PUSH; fi
-if grep -xF $NEW_REVISION $branch.force-rev; then push=$OSSTEST_PUSH; fi
+if test -n "$NEW_REVISION" && grep -xF $NEW_REVISION $branch.force-rev; then push=$OSSTEST_PUSH; fi
if test -f $branch.block; then push=false; fi
if test -e $mrof && test -e $tree_bisect && ! grep '^broken' $mrof; then
diff --git a/cri-common b/cri-common
index ad44546..4bf7548 100644
--- a/cri-common
+++ b/cri-common
@@ -72,6 +72,7 @@ select_xenbranch () {
rumpuserxen) tree=rumpuserxen; xenbranch=xen-unstable ;;
seabios) tree=seabios; xenbranch=xen-unstable ;;
ovmf) tree=ovmf; xenbranch=xen-unstable ;;
+ distros) tree=none; xenbranch=xen-unstable ;;
osstest) tree=osstest; xenbranch=xen-unstable ;;
esac
if [ "x$tree" = xlinux ]; then
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 15/24] distros: add branch infrastructure
2015-05-06 9:56 ` [OSSTEST v6 15/24] distros: add branch infrastructure Ian Campbell
@ 2015-05-12 16:03 ` Ian Jackson
2015-05-13 9:12 ` Ian Campbell
0 siblings, 1 reply; 53+ messages in thread
From: Ian Jackson @ 2015-05-12 16:03 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("[OSSTEST v6 15/24] distros: add branch infrastructure"):
> Since the distro nightlies are not version controlled we cannot use
> the usual mechanisms for detecting regressions. Special case things
> appropriately. We use an OLD_REVISION of "flight-NNN" to signify that
> the old revision is another flight and not a tree revision.
This all looks pretty good, except:
> -if grep -xF $NEW_REVISION $branch.force-rev; then push=$OSSTEST_PUSH; fi
> +if test -n "$NEW_REVISION" && grep -xF $NEW_REVISION $branch.force-rev; then push=$OSSTEST_PUSH; fi
Why is it not sufficient to simply add " " around $NEW_REVISION ?
Thanks,
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [OSSTEST v6 15/24] distros: add branch infrastructure
2015-05-12 16:03 ` Ian Jackson
@ 2015-05-13 9:12 ` Ian Campbell
0 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-13 9:12 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
On Tue, 2015-05-12 at 17:03 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[OSSTEST v6 15/24] distros: add branch infrastructure"):
> > Since the distro nightlies are not version controlled we cannot use
> > the usual mechanisms for detecting regressions. Special case things
> > appropriately. We use an OLD_REVISION of "flight-NNN" to signify that
> > the old revision is another flight and not a tree revision.
>
> This all looks pretty good, except:
>
> > -if grep -xF $NEW_REVISION $branch.force-rev; then push=$OSSTEST_PUSH; fi
> > +if test -n "$NEW_REVISION" && grep -xF $NEW_REVISION $branch.force-rev; then push=$OSSTEST_PUSH; fi
>
> Why is it not sufficient to simply add " " around $NEW_REVISION ?
Uh, I think it is!
^ permalink raw reply [flat|nested] 53+ messages in thread
* [OSSTEST v6 16/24] distros: Run a flight over the weekend.
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (14 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 15/24] distros: add branch infrastructure Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 17/24] Debian: Handle lack of bootloader support in d-i on ARM Ian Campbell
` (9 subsequent siblings)
25 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Once a week should be sufficient for this test. It involves quite a
large number of jobs so schedule it to start early on Saturday so it
can run over the weekend when, in theory, things should be quieter.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
crontab | 1 +
1 file changed, 1 insertion(+)
diff --git a/crontab b/crontab
index 5dc8675..1917ec4 100644
--- a/crontab
+++ b/crontab
@@ -6,5 +6,6 @@ MAILTO=ian.jackson@citrix.com,ian.campbell@eu.citrix.com
18 9 * * 1,3,5 cd testing.git && BRANCHES=linux-next ./cr-for-branches branches -w "./cr-daily-branch --real"
18 4 * * * cd testing.git && BRANCHES='linux-linus linux-mingo-tip-master linux-3.0 libvirt rumpuserxen' ./cr-for-branches branches -w "./cr-daily-branch --real"
6-59/15 * * * * cd testing.git && EXTRA_BRANCHES='linux-linus linux-3.0 rumpuserxen libvirt' ./cr-for-branches bisects -w "./cr-try-bisect --real"
+46 7 * * 6 cd testing.git && BRANCHES=distros ./cr-for-branches branches -w "./cr-daily-branch --real"
#8-59/5 * * * * cd bisects/adhoc.git && with-lock-ex -q data-tree-lock bash -c "./cr-try-bisect-adhoc; exit $?"
3 4 * * * savelog -c28 testing.git/tmp/cr-for-branches.log >/dev/null
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 17/24] Debian: Handle lack of bootloader support in d-i on ARM.
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (15 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 16/24] distros: Run a flight over the weekend Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 18/24] ts-debian-di-install: Refactor root_disk specification Ian Campbell
` (8 subsequent siblings)
25 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Debian doesn't currently know what bootloader to install in a Xen
guest on ARM. We install pv-grub-menu above which actually does what
we need, but the installer doesn't treat that as a "bootloader".
Most ARM platforms end up installing a u-boot boot.scr, based on a
platform whitelist. This doesn't seem appropriate for us. Grub is not
available for arm32. For arm64 we will eventually end up with in-guest
UEFI and therefore grub-efi and things will work normally. I'm not
sure what the answer is going to be for arm32.
This patch enables the workaround for Wheezy, Jessie and Sid,
post-Jessie should be enabled as we add them. (Pre-wheezy does not
support running as a Xen guest on ARM so we don't test them at all).
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v4: Handle sid too
v3: New
---
Osstest/Debian.pm | 14 ++++++++++++--
ts-debian-di-install | 6 ++++--
2 files changed, 16 insertions(+), 4 deletions(-)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index a153f34..129226f 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -719,8 +719,8 @@ END
return $preseed;
}
-sub preseed_create_guest ($$;@) {
- my ($ho, $sfx, %xopts) = @_;
+sub preseed_create_guest ($$$;@) {
+ my ($ho, $arch, $sfx, %xopts) = @_;
my $suite= $xopts{Suite} || $c{DebianSuite};
@@ -767,6 +767,16 @@ d-i grub-installer/bootdev string /dev/xvda
END
+ # Debian doesn't currently know what bootloader to install in a
+ # Xen guest on ARM. We install pv-grub-menu above which actually
+ # does what we need, but the installer doesn't treat that as a
+ # "bootloader".
+ logm("\$arch is $arch, \$suite is $suite");
+ $preseed_file.= (<<END) if $arch =~ /^arm/ && $suite =~ /wheezy|jessie|sid/;
+d-i nobootloader/confirmation_common boolean true
+
+END
+
$preseed_file .= preseed_hook_cmds();
return create_webfile($ho, "preseed$sfx", $preseed_file);
diff --git a/ts-debian-di-install b/ts-debian-di-install
index 5dfb0a1..1228f15 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -170,7 +170,9 @@ END
$suite = "sid" if $suite eq "daily";
- $ps_url = preseed_create_guest($gho, '', Suite=>$suite, PvMenuLst=>($bl eq "pvgrub"));
+ $ps_url = preseed_create_guest($gho, $arch, '',
+ Suite=>$suite,
+ PvMenuLst=>($bl eq "pvgrub"));
$extra_disk = "";
}
@@ -182,7 +184,7 @@ END
($method_cfg,$extra_disk) = setup_netinst($tmpdir, $arch, $cd);
- $ps_url = preseed_create_guest($gho, '', CDROM=>1);
+ $ps_url = preseed_create_guest($gho, $arch, '', CDROM=>1);
}
else
{
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 18/24] ts-debian-di-install: Refactor root_disk specification
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (16 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 17/24] Debian: Handle lack of bootloader support in d-i on ARM Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 19/24] make-flight: refactor PV debian tests Ian Campbell
` (7 subsequent siblings)
25 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v4: new patch
---
ts-debian-di-install | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/ts-debian-di-install b/ts-debian-di-install
index 1228f15..9ae3d6d 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -207,12 +207,14 @@ END
OnPowerOff => "preserve"
);
+ my $root_disk = "'phy:$gho->{Lvdev},xvda,w'";
+
prepareguest_part_xencfg($ho, $gho, $ram_mb, \%install_xopts, <<END);
$method_cfg
extra = "$cmdline"
#
disk = [
- $extra_disk 'phy:$gho->{Lvdev},xvda,w'
+ $extra_disk $root_disk
]
END
@@ -236,7 +238,7 @@ END
$blcfg
#
disk = [
- 'phy:$gho->{Lvdev},xvda,w'
+ $root_disk
]
END
return;
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 19/24] make-flight: refactor PV debian tests
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (17 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 18/24] ts-debian-di-install: Refactor root_disk specification Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 20/24] Add testing of non-LVM/phy disk backends Ian Campbell
` (6 subsequent siblings)
25 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
No functional change, standalong-generate-dump-flight-runvars confirms
no change to the runvars.
Includes a hook which is not used yet, $recipe_sfx.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v4: new patch
---
make-flight | 24 ++++++++++++++----------
1 file changed, 14 insertions(+), 10 deletions(-)
diff --git a/make-flight b/make-flight
index ca48b81..909cebe 100755
--- a/make-flight
+++ b/make-flight
@@ -339,6 +339,17 @@ do_pvgrub_tests () {
all_hostflags=$most_hostflags
}
+do_pv_debian_test_one () {
+ testname=$1; shift
+ recipe_sfx=$1; shift
+ toolstack=$1; shift
+
+ job_create_test test-$xenarch$kern-$dom0arch-$testname \
+ test-debian$recipe_sfx $toolstack \
+ $xenarch $dom0arch \
+ $debian_runvars all_hostflags=$most_hostflags $@
+}
+
do_pv_debian_tests () {
xsms=$(xenbranch_xsm_variants)
@@ -354,20 +365,13 @@ do_pv_debian_tests () {
suffix=${platform:+-$platform}
hostflags=${most_hostflags}${platform:+,platform-$platform}
- job_create_test test-$xenarch$kern-$dom0arch-xl$suffix \
- test-debian xl \
- $xenarch $dom0arch \
- enable_xsm=$xsm \
- $debian_runvars all_hostflags=$hostflags
+ do_pv_debian_test_one xl$suffix '' xl enable_xsm=$xsm
+
done
done
for xsm in $xsms ; do
- job_create_test test-$xenarch$kern-$dom0arch-libvirt \
- test-debian libvirt \
- $xenarch $dom0arch \
- enable_xsm=$xsm \
- $debian_runvars all_hostflags=$most_hostflags
+ do_pv_debian_test_one libvirt '' libvirt enable_xsm=$xsm
done
}
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 20/24] Add testing of non-LVM/phy disk backends.
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (18 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 19/24] make-flight: refactor PV debian tests Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 21/24] mfi-common: Allow make-*flight to filter the set of build jobs to include Ian Campbell
` (5 subsequent siblings)
25 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
xen-create-image makes this tricky to do since it is rather LVM
centric. Now that we have the ability to install from d-i it's
possible to arrange fairly easily that they use non-LVM disk backend
arrangements.
Here we add support to the test script and infra and create a bunch of
new jobs testing the cross product of {xl,libvirt} x {raw,qcow2,vhd}.
The test scripts are modified such that when constructing a domain
with a non-LVM diskfmt runvar:
- the LVM device is slightly enlarged to account for file format
headers (1M should be plenty).
- the LVM device will have an ext3 filesystem created on it instead
of being used as a phy device for the guest. Reusing the LVM volume
in this way means we don't need to do more storage management in
dom0 (i.e. arranging for / to be large enough, or managing a
special "images" LV)
- the relevant type of container is created within the filesystem
using the appropriate tool.
- New properties Disk{fmt,spec} are added to all $gho, containing
the format used for the root disk and the xl diskspec to load it.
- lvm backed guests use a xend/xm compatible spec, everything
else uses the improved xl syntax which libvirt also supports.
We won't test non-LVM on xend.
- New properties Disk{mnt,img} are added to $gho which are not using
LVM. These contain the mount point to use (configurable via
OSSTEST_CONFIG and runvars) and the full path (including mount
point) to the image itself.
- When starting or stopping a guest we arrange for the filesystem to
be (u)mounted.
- The prepearation when starting a guest copes gracefully with
the disk already being prepared.
- Hooks are called from guest_create() and guest_destroy() to
manipulate the disk as needed.
Using standalong-generate-dump-flight-runvars a representative set of
runvars is:
xen-unstable test-amd64-amd64-xl-qcow2 all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
xen-unstable test-amd64-amd64-xl-qcow2 arch amd64
xen-unstable test-amd64-amd64-xl-qcow2 buildjob build-amd64
xen-unstable test-amd64-amd64-xl-qcow2 debian_arch amd64
xen-unstable test-amd64-amd64-xl-qcow2 debian_bootloader pygrub
xen-unstable test-amd64-amd64-xl-qcow2 debian_diskfmt qcow2
xen-unstable test-amd64-amd64-xl-qcow2 debian_dist wheezy
xen-unstable test-amd64-amd64-xl-qcow2 debian_method netboot
xen-unstable test-amd64-amd64-xl-qcow2 kernbuildjob build-amd64-pvops
xen-unstable test-amd64-amd64-xl-qcow2 kernkind pvops
xen-unstable test-amd64-amd64-xl-qcow2 toolstack xl
xen-unstable test-amd64-amd64-xl-qcow2 xenbuildjob build-amd64
Compared to test-amd64-amd64-pygrub (which is the most similar job) and
normalising the test name the difference is:
xen-unstable test-amd64-amd64-SUFFIX all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
xen-unstable test-amd64-amd64-SUFFIX arch amd64
xen-unstable test-amd64-amd64-SUFFIX buildjob build-amd64
xen-unstable test-amd64-amd64-SUFFIX debian_arch amd64
xen-unstable test-amd64-amd64-SUFFIX debian_bootloader pygrub
+xen-unstable test-amd64-amd64-SUFFIX debian_diskfmt qcow2
xen-unstable test-amd64-amd64-SUFFIX debian_dist wheezy
xen-unstable test-amd64-amd64-SUFFIX debian_method netboot
xen-unstable test-amd64-amd64-SUFFIX kernbuildjob build-amd64-pvops
xen-unstable test-amd64-amd64-SUFFIX kernkind pvops
xen-unstable test-amd64-amd64-SUFFIX toolstack xl
xen-unstable test-amd64-amd64-SUFFIX xenbuildjob build-amd64
These are added to the following flights:
+libvirt
+linux-3.0
+linux-3.10
+linux-3.14
+linux-3.4
+linux-arm-xen
+linux-linus
+linux-mingo-tip-master
+linux-next
+osstest
+qemu-mainline
+qemu-upstream-4.3-testing
+qemu-upstream-4.4-testing
+qemu-upstream-unstable
+seabios
+xen-4.0-testing
+xen-4.1-testing
+xen-4.2-testing
+xen-4.3-testing
+xen-4.4-testing
+xen-unstable
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v6: Use bs=1MB (=1*1000) when creating the raw images instead of bs=1M
(=1*1024), this matches the units used by lvcreate's -L option and
therefore arranges that the imaage actually fits.
v5: Assume 100MB/s dd from /dev/zero when creating a raw disk image
Allow 10M of slack on filesystem for raw, qcow and vhd. 1M wasn't
enough in practice for raw.
v4: new patch
---
Osstest/TestSupport.pm | 97 ++++++++++++++++++++++++++++++++++++++++++++++++--
make-flight | 16 +++++++++
ts-debian-di-install | 10 ++----
ts-guest-start | 1 -
4 files changed, 114 insertions(+), 10 deletions(-)
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index b3d85d6..a979223 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -89,7 +89,8 @@ BEGIN {
target_var target_var_prefix
selectguest prepareguest more_prepareguest_hvm
guest_var guest_var_commalist
- prepareguest_part_lvmdisk prepareguest_part_xencfg
+ prepareguest_part_lvmdisk prepareguest_part_diskimg
+ prepareguest_part_xencfg
guest_umount_lv guest_await guest_await_dhcp_tcp
guest_checkrunning guest_check_ip guest_find_ether
guest_find_domid guest_check_up guest_check_up_quick
@@ -97,6 +98,7 @@ BEGIN {
guest_await_shutdown guest_await_destroy guest_destroy
guest_vncsnapshot_begin guest_vncsnapshot_stash
guest_check_remus_ok guest_editconfig
+ guest_prepare_disk guest_unprepare_disk
host_involves_pcipassthrough host_get_pcipassthrough_devs
toolstack guest_create
@@ -1319,6 +1321,7 @@ sub selectguest ($$) {
}
logm("guest: using $gn on $gho->{Host}{Name}");
guest_find_lv($gho);
+ guest_find_diskimg($gho);
guest_find_ether($gho);
guest_find_tcpcheckport($gho);
dhcp_watch_setup($ho,$gho);
@@ -1334,6 +1337,22 @@ sub guest_find_lv ($) {
? '/dev/'.$gho->{Vg}.'/'.$gho->{Lv} : undef;
}
+sub guest_find_diskimg($)
+{
+ my ($gho) = @_;
+ $gho->{Diskfmt} = $r{"$gho->{Guest}_diskfmt"} // "lvm";
+ $gho->{Diskspec} = "phy:$gho->{Lvdev},xvda,w";
+
+ return if $gho->{Diskfmt} eq "lvm";
+
+ my $mntroot = get_host_property($gho->{Host}, "DiskImageMount",
+ $c{DiskImageMount} // "/var/lib/xen/images");
+
+ $gho->{Diskmnt} = "$mntroot/$gho->{Guest}";
+ $gho->{Diskimg} = "$gho->{Diskmnt}/disk.$gho->{Diskfmt}";
+ $gho->{Diskspec} = "format=$gho->{Diskfmt},vdev=xvda,target=$gho->{Diskimg}";
+}
+
sub guest_find_ether ($) {
my ($gho) = @_;
$gho->{Ether}= $r{"$gho->{Guest}_ether"};
@@ -1373,6 +1392,7 @@ sub guest_destroy ($) {
my ($gho) = @_;
my $ho = $gho->{Host};
toolstack($ho)->destroy($gho);
+ guest_unprepare_disk($gho);
}
sub guest_await_destroy ($$) {
@@ -1384,9 +1404,32 @@ sub guest_await_destroy ($$) {
sub guest_create ($) {
my ($gho) = @_;
my $ho = $gho->{Host};
+ guest_prepare_disk($gho);
toolstack($ho)->create($gho);
}
+sub guest_prepare_disk ($) {
+ my ($gho) = @_;
+
+ guest_umount_lv($gho->{Host}, $gho);
+
+ return if $gho->{Diskfmt} eq "lvm";
+
+ target_cmd_root($gho->{Host}, <<END);
+mkdir -p $gho->{Diskmnt}
+mount $gho->{Lvdev} $gho->{Diskmnt};
+END
+}
+
+sub guest_unprepare_disk ($) {
+ my ($gho) = @_;
+ return if $gho->{Diskfmt} eq "lvm";
+ target_cmd_root($gho->{Host}, <<END);
+umount $gho->{Lvdev} || :
+END
+}
+
+
sub target_choose_vg ($$) {
my ($ho, $mbneeded) = @_;
@@ -1518,6 +1561,7 @@ sub prepareguest ($$$$$$) {
}
guest_find_lv($gho);
+ guest_find_diskimg($gho);
guest_find_ether($gho);
guest_find_tcpcheckport($gho);
return $gho;
@@ -1528,7 +1572,56 @@ sub prepareguest_part_lvmdisk ($$$) {
target_cmd_root($ho, "lvremove -f $gho->{Lvdev} ||:");
target_cmd_root($ho, "lvcreate -L ${disk_mb}M -n $gho->{Lv} $gho->{Vg}");
target_cmd_root($ho, "dd if=/dev/zero of=$gho->{Lvdev} count=10");
-}
+}
+
+sub make_vhd ($$$) {
+ my ($ho, $gho, $disk_mb) = @_;
+ target_cmd_root($ho, "vhd-util create -n $gho->{Rootimg} -s $disk_mb");
+}
+sub make_qcow2 ($$$) {
+ my ($ho, $gho, $disk_mb) = @_;
+ # upstream qemu's version. Seems preferable to qemu-xen-img from qemu-trad.
+ my $qemu_img = "/usr/local/lib/xen/bin/qemu-img";
+ target_cmd_root($ho, "$qemu_img create -f qcow2 $gho->{Rootimg} ${disk_mb}M");
+}
+sub make_raw ($$$) {
+ my ($ho, $gho, $disk_mb) = @_;
+ # In local tests this reported 130MB/s, so calculate a timeout assuming 100MB/s.
+ target_cmd_root($ho, "dd if=/dev/zero of=$gho->{Rootimg} bs=1MB count=${disk_mb}",
+ ${disk_mb} / 100);
+}
+
+sub prepareguest_part_diskimg ($$$) {
+ my ($ho, $gho, $disk_mb) = @_;
+
+ my $diskfmt = $gho->{Diskfmt};
+ # Allow an extra 10 megabytes for image format headers
+ my $disk_overhead = $diskfmt eq "lvm" ? 0 : 10;
+
+ logm("preparing guest disks in $diskfmt format");
+
+ target_cmd_root($ho, "umount $gho->{Lvdev} ||:");
+
+ prepareguest_part_lvmdisk($ho, $gho, $disk_mb + $disk_overhead);
+
+ if ($diskfmt ne "lvm") {
+
+ $gho->{Rootimg} = "$gho->{Diskmnt}/disk.$diskfmt";
+ $gho->{Rootcfg} = "format=$diskfmt,vdev=xvda,target=$gho->{Rootimg}";
+
+ target_cmd_root($ho, <<END);
+mkfs.ext3 $gho->{Lvdev}
+mkdir -p $gho->{Diskmnt}
+mount $gho->{Lvdev} $gho->{Diskmnt}
+END
+ no strict qw(refs);
+ &{"make_$diskfmt"}($ho, $gho, $disk_mb);
+
+ target_cmd_root($ho, <<END);
+umount $gho->{Lvdev}
+END
+ }
+}
sub prepareguest_part_xencfg ($$$$$) {
my ($ho, $gho, $ram_mb, $xopts, $cfgrest) = @_;
diff --git a/make-flight b/make-flight
index 909cebe..c923f21 100755
--- a/make-flight
+++ b/make-flight
@@ -373,6 +373,22 @@ do_pv_debian_tests () {
for xsm in $xsms ; do
do_pv_debian_test_one libvirt '' libvirt enable_xsm=$xsm
done
+
+ for ts in xl libvirt ; do
+
+ for fmt in raw vhd qcow2 ; do
+
+ fmt_runvar="debian_diskfmt=$fmt"
+
+ do_pv_debian_test_one $ts-$fmt '-di' $ts \
+ debian_arch=amd64 \
+ debian_dist=$guestsuite \
+ debian_method=netboot \
+ debian_bootloader=pygrub \
+ $fmt_runvar
+
+ done
+ done
}
test_matrix_do_one () {
diff --git a/ts-debian-di-install b/ts-debian-di-install
index 9ae3d6d..ecdb6a1 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -41,9 +41,7 @@ sub prep () {
$gho= prepareguest($ho, $gn, $guesthost, 22,
$disk_mb, 40);
- prepareguest_part_lvmdisk($ho, $gho, $disk_mb);
-
- target_cmd_root($ho, "umount $gho->{Lvdev} ||:");
+ prepareguest_part_diskimg($ho, $gho, $disk_mb);
}
sub setup_netinst($$$)
@@ -207,14 +205,12 @@ END
OnPowerOff => "preserve"
);
- my $root_disk = "'phy:$gho->{Lvdev},xvda,w'";
-
prepareguest_part_xencfg($ho, $gho, $ram_mb, \%install_xopts, <<END);
$method_cfg
extra = "$cmdline"
#
disk = [
- $extra_disk $root_disk
+ $extra_disk '$gho->{Diskspec}'
]
END
@@ -238,7 +234,7 @@ END
$blcfg
#
disk = [
- $root_disk
+ '$gho->{Diskspec}'
]
END
return;
diff --git a/ts-guest-start b/ts-guest-start
index 1aa9e69..a434720 100755
--- a/ts-guest-start
+++ b/ts-guest-start
@@ -25,7 +25,6 @@ tsreadconfig();
our ($ho,$gho) = ts_get_host_guest(@ARGV);
sub start () {
- guest_umount_lv($ho, $gho);
guest_create($gho);
}
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 21/24] mfi-common: Allow make-*flight to filter the set of build jobs to include
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (19 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 20/24] Add testing of non-LVM/phy disk backends Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 22/24] make-distros-flight: don't bother building for XSM Ian Campbell
` (4 subsequent siblings)
25 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
By using the same job_create_build(_filter_callback) scheme used for
the test jobs.
Will be used in make-distros-flight.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
make-distros-flight | 4 ++++
make-flight | 4 ++++
mfi-common | 21 +++++++++++++++------
3 files changed, 23 insertions(+), 6 deletions(-)
diff --git a/make-distros-flight b/make-distros-flight
index e3ec8f7..2724dd2 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -33,6 +33,10 @@ flight=`./cs-flight-create $blessing $branch`
defsuite=`getconfig DebianSuite`
defguestsuite=`getconfig GuestDebianSuite`
+job_create_build_filter_callback () {
+ :
+}
+
if [ x$buildflight = x ]; then
WANT_XEND=false REVISION_LINUX_OLD=disable
diff --git a/make-flight b/make-flight
index c923f21..ef1e422 100755
--- a/make-flight
+++ b/make-flight
@@ -34,6 +34,10 @@ flight=`./cs-flight-create $blessing $branch`
defsuite=`getconfig DebianSuite`
defguestsuite=`getconfig GuestDebianSuite`
+job_create_build_filter_callback () {
+ :
+}
+
if [ x$buildflight = x ]; then
create_build_jobs
diff --git a/mfi-common b/mfi-common
index a9e966f..a100afb 100644
--- a/mfi-common
+++ b/mfi-common
@@ -54,6 +54,15 @@ xenbranch_xsm_variants () {
esac
}
+job_create_build () {
+ job_create_build_filter_callback "$@" || return 0
+
+ local job=$1; shift
+ local recipe=$1; shift
+
+ ./cs-job-create $flight $job $recipe "$@"
+}
+
create_build_jobs () {
local arch
@@ -164,7 +173,7 @@ create_build_jobs () {
else
xsm_suffix=""
fi
- ./cs-job-create $flight build-$arch$xsm_suffix build \
+ job_create_build build-$arch$xsm_suffix build \
arch=$arch enable_xend=$build_defxend enable_ovmf=$enable_ovmf\
enable_xsm=$enable_xsm \
tree_qemu=$TREE_QEMU \
@@ -183,7 +192,7 @@ create_build_jobs () {
done
if [ $build_extraxend = "true" ] ; then
- ./cs-job-create $flight build-$arch-xend build \
+ job_create_build build-$arch-xend build \
arch=$arch enable_xend=true enable_ovmf=$enable_ovmf \
tree_qemu=$TREE_QEMU \
tree_qemuu=$TREE_QEMU_UPSTREAM \
@@ -196,7 +205,7 @@ create_build_jobs () {
revision_qemuu=$REVISION_QEMU_UPSTREAM
fi
- ./cs-job-create $flight build-$arch-pvops build-kern \
+ job_create_build build-$arch-pvops build-kern \
arch=$arch kconfighow=xen-enable-xen-config \
$RUNVARS $BUILD_RUNVARS $BUILD_LINUX_RUNVARS $arch_runvars \
$suite_runvars \
@@ -208,7 +217,7 @@ create_build_jobs () {
if [ "x$REVISION_LIBVIRT" != xdisable ]; then
- ./cs-job-create $flight build-$arch-libvirt build-libvirt \
+ job_create_build build-$arch-libvirt build-libvirt \
arch=$arch \
tree_xen=$TREE_XEN \
$RUNVARS $BUILD_RUNVARS $BUILD_LIBVIRT_RUNVARS $arch_runvars \
@@ -223,7 +232,7 @@ create_build_jobs () {
case $arch in
i386|amd64)
- ./cs-job-create $flight build-$arch-rumpuserxen build-rumpuserxen \
+ job_create_build build-$arch-rumpuserxen build-rumpuserxen \
arch=$arch \
tree_xen=$TREE_XEN \
$RUNVARS $BUILD_RUNVARS $BUILD_RUMPUSERXEN_RUNVARS $arch_runvars \
@@ -252,7 +261,7 @@ create_build_jobs () {
if [ "x$REVISION_LINUX_OLD" != xdisable ]; then
- ./cs-job-create $flight build-$arch-oldkern build-kern \
+ job_create_build build-$arch-oldkern build-kern \
arch=$arch kconfighow=create-config-sh \
kimagefile=vmlinux \
$RUNVARS $BUILD_RUNVARS $BUILD_LINUX_OLD_RUNVARS \
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 22/24] make-distros-flight: don't bother building for XSM.
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (20 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 21/24] mfi-common: Allow make-*flight to filter the set of build jobs to include Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 23/24] distros: email only me on play flights Ian Campbell
` (3 subsequent siblings)
25 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
make-distros-flight | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/make-distros-flight b/make-distros-flight
index 2724dd2..62c527b 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -34,7 +34,10 @@ defsuite=`getconfig DebianSuite`
defguestsuite=`getconfig GuestDebianSuite`
job_create_build_filter_callback () {
- :
+ case " $* " in
+ *" enable_xsm=true "*) return 1;;
+ esac
+ return 0
}
if [ x$buildflight = x ]; then
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 23/24] distros: email only me on play flights
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (21 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 22/24] make-distros-flight: don't bother building for XSM Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-06 9:56 ` [OSSTEST v6 24/24] ts-debian-di-install: Use ftp.debian.org directly Ian Campbell
` (2 subsequent siblings)
25 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
daily-cron-email-play--distros | 1 +
1 file changed, 1 insertion(+)
create mode 100644 daily-cron-email-play--distros
diff --git a/daily-cron-email-play--distros b/daily-cron-email-play--distros
new file mode 100644
index 0000000..0d82e65
--- /dev/null
+++ b/daily-cron-email-play--distros
@@ -0,0 +1 @@
+To: ian.campbell@citrix.com
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* [OSSTEST v6 24/24] ts-debian-di-install: Use ftp.debian.org directly
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (22 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 23/24] distros: email only me on play flights Ian Campbell
@ 2015-05-06 9:56 ` Ian Campbell
2015-05-06 9:57 ` [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
2015-05-12 16:06 ` Ian Jackson
25 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:56 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
The local proxy seems to serve stale packages for Jessie etc, I blame
the intercepting cache on the way out of our network, similar to
b5f15136900d "mg-debian-installer-update: workaround caching proxies",
except it is between the apt-cache and the world not the osstest vm
and the world.
Since the netboot kernel+initrd are reasonably small, these flights
are infrequent and they are intended to test the current upstream
version I think this is tollerable.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
ts-debian-di-install | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/ts-debian-di-install b/ts-debian-di-install
index ecdb6a1..6a3b39a 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -121,7 +121,9 @@ sub setup_netboot($$$)
target_putfile_root($ho, 60, $initrd, "$didir/initrd_${suite}_${arch}");
} else {
- my $mirror = "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath}";
+ #my $mirror = "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath}";
+ # XXX local mirror seems to serve up stale files.
+ my $mirror = "http://ftp.debian.org/debian";
my $di_url = $suite eq "daily" ?
"http://d-i.debian.org/daily-images/$arch/daily/netboot" :
--
2.1.4
^ permalink raw reply related [flat|nested] 53+ messages in thread* Re: [PATCH OSSTEST v5 00/24] add distro domU testing flight
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (23 preceding siblings ...)
2015-05-06 9:56 ` [OSSTEST v6 24/24] ts-debian-di-install: Use ftp.debian.org directly Ian Campbell
@ 2015-05-06 9:57 ` Ian Campbell
2015-05-12 16:06 ` Ian Jackson
25 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-06 9:57 UTC (permalink / raw)
To: Ian Jackson; +Cc: longtaox.pang, xen-devel
On Wed, 2015-05-06 at 10:46 +0100, Ian Campbell wrote:
> Hi,
>
> Since v5
Subject should have said v6 and I accidentally sent the patches
themselves to xendevel and not xen-devel@lists. I've resent, sorry to
Ian especially for the extra spam :-(
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [PATCH OSSTEST v5 00/24] add distro domU testing flight
2015-05-06 9:46 [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
` (24 preceding siblings ...)
2015-05-06 9:57 ` [PATCH OSSTEST v5 00/24] add distro domU testing flight Ian Campbell
@ 2015-05-12 16:06 ` Ian Jackson
2015-05-13 9:19 ` Ian Campbell
25 siblings, 1 reply; 53+ messages in thread
From: Ian Jackson @ 2015-05-12 16:06 UTC (permalink / raw)
To: Ian Campbell; +Cc: longtaox.pang, xen-devel
Ian Campbell writes ("[Xen-devel] [PATCH OSSTEST v5 00/24] add distro domU testing flight"):
> As last time there are some patches in here which I think will be useful
> to the Intel folks doing the nested virt testing, specifically the
> refactoring of how overlays and ssh host keys are done will be useful
> for installing a guest to be treated as the L1 host. (Note that this has
> changed a bit, again, since last time)
I think I have reviewed all of those now.
> When I originally started this work I envisioned a flight running on the
> main production instance (in Cambridge at the time). Now that we have
> the new colo I would still consider that the main production instance is
> the best home, however given the new colo is not yet up to full capacity
> we could also consider running this flight in Cambridge for the time
> being. What do you think?
I think we should use the resources we have available, and that
includes the Cambridge instance (which is hopefully less broken now).
Until the Xen Project colo instance is _much_ bigger than Cambridge, I
think we should run it here in Cambridge.
> Also when running the adhoc tests the sheer number of jobs which are
> involved (with my hope there will be more in the future as other distros
> get in on the act) I was considering splitting it into multiple
> distros-debian-{squeeze,wheezy,jessie,sid,daily} flights. Thoughts?
As discussed, we think this is a good idea. In particular, it would
allow running stable releases less often. (As a result, this will
produce a whole lot of distros-<distro>-<revision> flights.)
Thanks,
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [PATCH OSSTEST v5 00/24] add distro domU testing flight
2015-05-12 16:06 ` Ian Jackson
@ 2015-05-13 9:19 ` Ian Campbell
2015-05-13 16:09 ` Ian Jackson
0 siblings, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-05-13 9:19 UTC (permalink / raw)
To: Ian Jackson; +Cc: longtaox.pang, xen-devel
On Tue, 2015-05-12 at 17:06 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[Xen-devel] [PATCH OSSTEST v5 00/24] add distro domU testing flight"):
> > As last time there are some patches in here which I think will be useful
> > to the Intel folks doing the nested virt testing, specifically the
> > refactoring of how overlays and ssh host keys are done will be useful
> > for installing a guest to be treated as the L1 host. (Note that this has
> > changed a bit, again, since last time)
>
> I think I have reviewed all of those now.
Thanks!
I think you have acked all of the ones I identified as being an
initially commitable chunk except one "TestSupport: Add helper to fetch
a URL on a host" which I think is not used by the nested virt series and
so can wait.
Assuming that turns out to be the case are you OK with me pushing that
subset to pretest?
> > When I originally started this work I envisioned a flight running on the
> > main production instance (in Cambridge at the time). Now that we have
> > the new colo I would still consider that the main production instance is
> > the best home, however given the new colo is not yet up to full capacity
> > we could also consider running this flight in Cambridge for the time
> > being. What do you think?
>
> I think we should use the resources we have available, and that
> includes the Cambridge instance (which is hopefully less broken now).
> Until the Xen Project colo instance is _much_ bigger than Cambridge, I
> think we should run it here in Cambridge.
OK, will do. Should I arrange for reports to go to xen-devel even though
logs will only be available internally?
> > Also when running the adhoc tests the sheer number of jobs which are
> > involved (with my hope there will be more in the future as other distros
> > get in on the act) I was considering splitting it into multiple
> > distros-debian-{squeeze,wheezy,jessie,sid,daily} flights. Thoughts?
>
> As discussed, we think this is a good idea. In particular, it would
> allow running stable releases less often. (As a result, this will
> produce a whole lot of distros-<distro>-<revision> flights.)
Correct, I'll refactor along those lines.
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread* Re: [PATCH OSSTEST v5 00/24] add distro domU testing flight
2015-05-13 9:19 ` Ian Campbell
@ 2015-05-13 16:09 ` Ian Jackson
0 siblings, 0 replies; 53+ messages in thread
From: Ian Jackson @ 2015-05-13 16:09 UTC (permalink / raw)
To: Ian Campbell; +Cc: longtaox.pang, xen-devel
Ian Campbell writes ("Re: [Xen-devel] [PATCH OSSTEST v5 00/24] add distro domU testing flight"):
> I think you have acked all of the ones I identified as being an
> initially commitable chunk except one "TestSupport: Add helper to fetch
> a URL on a host" which I think is not used by the nested virt series and
> so can wait.
Right.
> Assuming that turns out to be the case are you OK with me pushing that
> subset to pretest?
Yes. You might like to combine it with my LVM resize/mkfs series (if
you think the latter is ready), which I have put here for your
convenience:
xenbits.xen.org:/home/iwj/ext/osstest.git#wip.build-prep-no-fs-resize.v1
> > I think we should use the resources we have available, and that
> > includes the Cambridge instance (which is hopefully less broken now).
> > Until the Xen Project colo instance is _much_ bigger than Cambridge, I
> > think we should run it here in Cambridge.
>
> OK, will do. Should I arrange for reports to go to xen-devel even though
> logs will only be available internally?
Good question. Probably, I think.
Ian.
^ permalink raw reply [flat|nested] 53+ messages in thread