* [PATCH OSSTEST v4 01/25] TestSupport: Add helper to fetch a URL on a host
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 11:24 ` Ian Jackson
2015-04-02 9:16 ` [PATCH OSSTEST v4 02/25] TestSupport: allow caller of prepareguest_part_xencfg to specify viftype Ian Campbell
` (24 subsequent siblings)
25 siblings, 1 reply; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v3: Make sure wget is installed
---
Osstest/Debian.pm | 2 +-
Osstest/TestSupport.pm | 9 ++++++++-
2 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 69530fb..5e6bcba 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -617,7 +617,7 @@ d-i apt-setup/another boolean false
d-i apt-setup/non-free boolean false
d-i apt-setup/contrib boolean false
-d-i pkgsel/include string openssh-server, ntp, ntpdate, ethtool, chiark-utils-bin, $extra_packages
+d-i pkgsel/include string openssh-server, ntp, ntpdate, ethtool, chiark-utils-bin, wget, $extra_packages
$xopts{ExtraPreseed}
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 1053016..c4adc48 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -55,7 +55,7 @@ BEGIN {
target_putfilecontents_stash
target_putfilecontents_root_stash
target_put_guest_image target_editfile
- target_editfile_cancel
+ target_editfile_cancel target_fetchurl
target_editfile_root target_file_exists
target_run_apt
target_install_packages target_install_packages_norec
@@ -1560,6 +1560,13 @@ END
return $cfgpath;
}
+sub target_fetchurl($$$;$) {
+ my ($ho, $url, $path, $timeo) = @_;
+ $timeo ||= 2000;
+ target_cmd_root($ho, "wget --progress=dot:mega -O $path $url", $timeo);
+}
+
+
sub target_put_guest_image ($$$) {
my ($ho, $gho, $default) = @_;
my $specimage = $r{"$gho->{Guest}_image"};
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* Re: [PATCH OSSTEST v4 01/25] TestSupport: Add helper to fetch a URL on a host
2015-04-02 9:16 ` [PATCH OSSTEST v4 01/25] TestSupport: Add helper to fetch a URL on a host Ian Campbell
@ 2015-04-02 11:24 ` Ian Jackson
2015-04-02 12:26 ` Ian Campbell
0 siblings, 1 reply; 40+ messages in thread
From: Ian Jackson @ 2015-04-02 11:24 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("[PATCH OSSTEST v4 01/25] TestSupport: Add helper to fetch a URL on a host"):
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
We're not using this to fetch URLs on the public internet, are we ?
If we are then we need to honour $c{HttpProxy}.
Perhaps you could add a comment to the effect that target_fetch_url
should be used for URLs within the test environment, only.
Ian.
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [PATCH OSSTEST v4 01/25] TestSupport: Add helper to fetch a URL on a host
2015-04-02 11:24 ` Ian Jackson
@ 2015-04-02 12:26 ` Ian Campbell
0 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 12:26 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
On Thu, 2015-04-02 at 12:24 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[PATCH OSSTEST v4 01/25] TestSupport: Add helper to fetch a URL on a host"):
> > Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
>
> We're not using this to fetch URLs on the public internet, are we ?
> If we are then we need to honour $c{HttpProxy}.
It is used to fetch the kernel+initrd to test, which previously came
from the apt cache, via $c{DebianMirrorHost}.
Would you recommend $c{HttpProxy} or ${DebianMirrorProxy} for this? I
think given the generic nature only the former, even though the only
caller is currently grabbing apt-cachable things.
> Perhaps you could add a comment to the effect that target_fetch_url
> should be used for URLs within the test environment, only.
>
> Ian.
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 02/25] TestSupport: allow caller of prepareguest_part_xencfg to specify viftype
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 01/25] TestSupport: Add helper to fetch a URL on a host Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 11:25 ` Ian Jackson
2015-04-02 9:16 ` [PATCH OSSTEST v4 03/25] create_webfile: Support use with guests as well as hosts Ian Campbell
` (23 subsequent siblings)
25 siblings, 1 reply; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
--
v3: Use CamelCase for xopts, use the correct condition
---
Osstest/TestSupport.pm | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index c4adc48..490e16d 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1533,12 +1533,13 @@ sub prepareguest_part_xencfg ($$$$$) {
my $onpoweroff= $xopts->{OnPowerOff} || 'destroy';
my $oncrash= $xopts->{OnCrash} || 'preserve';
my $vcpus= guest_var($gho, 'vcpus', $xopts->{DefVcpus} || 2);
+ my $viftype= $xopts->{VifType} ? "type=$xopts->{VifType}," : "";
my $xoptcfg= $xopts->{ExtraConfig};
$xoptcfg='' unless defined $xoptcfg;
my $xencfg= <<END;
name = '$gho->{Name}'
memory = ${ram_mb}
-vif = [ 'type=ioemu,mac=$gho->{Ether}' ]
+vif = [ '${viftype}mac=$gho->{Ether}' ]
#
on_poweroff = '$onpoweroff'
on_reboot = '$onreboot'
@@ -1633,6 +1634,7 @@ END
$cfg .= "bios='$bios'\n";
}
+ $xopts{VifType} ||= "ioemu";
my $cfgpath= prepareguest_part_xencfg($ho, $gho, $ram_mb, \%xopts, $cfg);
target_cmd_root($ho, <<END);
(echo $passwd; echo $passwd) | vncpasswd $gho->{Guest}.vncpw
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 03/25] create_webfile: Support use with guests as well as hosts.
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 01/25] TestSupport: Add helper to fetch a URL on a host Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 02/25] TestSupport: allow caller of prepareguest_part_xencfg to specify viftype Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 11:26 ` Ian Jackson
2015-04-02 9:16 ` [PATCH OSSTEST v4 04/25] Debian: refactor code to add preseed commands to the preseed file Ian Campbell
` (22 subsequent siblings)
25 siblings, 1 reply; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
In particular make the path unique by ensuring it includes the host
and guest name in the guest case.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v3: New patch
---
Osstest/TestSupport.pm | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 490e16d..8f67f7e 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1979,6 +1979,8 @@ sub await_webspace_fetch_byleaf ($$$$$) {
sub create_webfile ($$$) {
my ($ho, $tail, $contents) = @_; # $contents as for file_link_contents
my $wf_rhs= $ho->{Name}."_".$tail;
+ # $ho->{Host} is set if $ho is a guest.
+ $wf_rhs= $ho->{Host}->{Name}."_${wf_rhs}" if $ho->{Host};
my $wf_common= $c{WebspaceCommon}.$wf_rhs;
my $wf_url= $c{WebspaceUrl}.$wf_common;
my $wf_file= $c{WebspaceFile}.$wf_common;
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* Re: [PATCH OSSTEST v4 03/25] create_webfile: Support use with guests as well as hosts.
2015-04-02 9:16 ` [PATCH OSSTEST v4 03/25] create_webfile: Support use with guests as well as hosts Ian Campbell
@ 2015-04-02 11:26 ` Ian Jackson
0 siblings, 0 replies; 40+ messages in thread
From: Ian Jackson @ 2015-04-02 11:26 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("[PATCH OSSTEST v4 03/25] create_webfile: Support use with guests as well as hosts."):
> In particular make the path unique by ensuring it includes the host
> and guest name in the guest case.
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
But I have a minor style quibble:
> + # $ho->{Host} is set if $ho is a guest.
> + $wf_rhs= $ho->{Host}->{Name}."_${wf_rhs}" if $ho->{Host};
It would be usual to write
+ $wf_rhs= $ho->{Host}{Name}."_${wf_rhs}" if $ho->{Host};
since only the first -> is required.
Thanks,
Ian.
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 04/25] Debian: refactor code to add preseed commands to the preseed file
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (2 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 03/25] create_webfile: Support use with guests as well as hosts Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 11:31 ` Ian Jackson
2015-04-02 9:16 ` [PATCH OSSTEST v4 05/25] Debian: refactor preseeding of .ssh directories Ian Campbell
` (21 subsequent siblings)
25 siblings, 1 reply; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: longtaox.pang, Ian Campbell, xen-devel
Call it from ts-debian-hvm-install.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: longtaox.pang@intel.com
---
v3: New patch
---
Osstest/Debian.pm | 16 +++++++++++-----
ts-debian-hvm-install | 3 +++
2 files changed, 14 insertions(+), 5 deletions(-)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 5e6bcba..ef602b1 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -37,7 +37,7 @@ BEGIN {
%preseed_cmds
preseed_base
preseed_create
- preseed_hook_command preseed_hook_installscript
+ preseed_hook_command preseed_hook_installscript preseed_hook_cmds
di_installcmdline_core
);
%EXPORT_TAGS = ( );
@@ -907,10 +907,7 @@ d-i partman-auto/expert_recipe string \\
END
- foreach my $di_key (keys %preseed_cmds) {
- $preseed_file .= "d-i preseed/$di_key string ".
- (join ' && ', @{ $preseed_cmds{$di_key} }). "\n";
- }
+ $preseed_file .= preseed_hook_cmds();
if ($ho->{Flags}{'no-di-kernel'}) {
$preseed_file .= <<END;
@@ -954,4 +951,13 @@ chmod +x '$installer_pathname'
END
}
+sub preseed_hook_cmds () {
+ my $preseed;
+ foreach my $di_key (keys %preseed_cmds) {
+ $preseed .= "d-i preseed/$di_key string ".
+ (join ' && ', @{ $preseed_cmds{$di_key} }). "\n";
+ }
+ return $preseed;
+}
+
1;
diff --git a/ts-debian-hvm-install b/ts-debian-hvm-install
index cfd5144..95fce9a 100755
--- a/ts-debian-hvm-install
+++ b/ts-debian-hvm-install
@@ -77,6 +77,9 @@ d-i preseed/late_command string \\
in-target mkdir -p /root/.ssh; \\
in-target sh -c "echo -e '$authkeys'> /root/.ssh/authorized_keys";
END
+
+ $preseed_file .= preseed_hook_cmds();
+
return $preseed_file;
}
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* Re: [PATCH OSSTEST v4 04/25] Debian: refactor code to add preseed commands to the preseed file
2015-04-02 9:16 ` [PATCH OSSTEST v4 04/25] Debian: refactor code to add preseed commands to the preseed file Ian Campbell
@ 2015-04-02 11:31 ` Ian Jackson
2015-04-02 12:29 ` Ian Campbell
0 siblings, 1 reply; 40+ messages in thread
From: Ian Jackson @ 2015-04-02 11:31 UTC (permalink / raw)
To: Ian Campbell; +Cc: longtaox.pang, xen-devel
Ian Campbell writes ("[PATCH OSSTEST v4 04/25] Debian: refactor code to add preseed commands to the preseed file"):
> Call it from ts-debian-hvm-install.
Please expand the commit message. Something like:
This means that, in future, ts-debian-hvm-install can use
preseed_hook_command and preseed_hook_installscript. No functional
change for now.
Thanks,
Ian.
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [PATCH OSSTEST v4 04/25] Debian: refactor code to add preseed commands to the preseed file
2015-04-02 11:31 ` Ian Jackson
@ 2015-04-02 12:29 ` Ian Campbell
0 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 12:29 UTC (permalink / raw)
To: Ian Jackson; +Cc: longtaox.pang, xen-devel
On Thu, 2015-04-02 at 12:31 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[PATCH OSSTEST v4 04/25] Debian: refactor code to add preseed commands to the preseed file"):
> > Call it from ts-debian-hvm-install.
>
> Please expand the commit message. Something like:
>
> This means that, in future, ts-debian-hvm-install can use
> preseed_hook_command and preseed_hook_installscript. No functional
> change for now.
I assumed you meant s/hvm/di/ since the hvm case is handled right here,
but done.
Ian
>
> Thanks,
> Ian.
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 05/25] Debian: refactor preseeding of .ssh directories
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (3 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 04/25] Debian: refactor code to add preseed commands to the preseed file Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 11:33 ` Ian Jackson
2015-04-02 9:16 ` [PATCH OSSTEST v4 06/25] Debian: Refactor installation of overlays, so it can be used for guests too Ian Campbell
` (20 subsequent siblings)
25 siblings, 1 reply; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: longtaox.pang, Ian Campbell, xen-devel
Causes known_hosts to be consistently created as well as ~osstest/.ssh
to be consistently populated (it previsouly wasn't for HVM guests).
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: longtaox.pang@intel.com
---
v3: New patch
---
Osstest/Debian.pm | 99 +++++++++++++++++++++++++++++----------------------
ts-debian-hvm-install | 5 ++-
2 files changed, 59 insertions(+), 45 deletions(-)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index ef602b1..db89326 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -37,6 +37,7 @@ BEGIN {
%preseed_cmds
preseed_base
preseed_create
+ preseed_ssh
preseed_hook_command preseed_hook_installscript preseed_hook_cmds
di_installcmdline_core
);
@@ -551,6 +552,60 @@ sub di_installcmdline_core ($$;@) {
return @cl;
}
+sub preseed_ssh ($$) {
+ my ($ho,$sfx) = @_;
+
+ my $authkeys_url= create_webfile($ho, "authkeys$sfx", authorized_keys());
+
+ my $hostkeyfile= "$c{OverlayLocal}/etc/ssh/ssh_host_rsa_key.pub";
+ my $hostkey= get_filecontents($hostkeyfile);
+ chomp($hostkey); $hostkey.="\n";
+ my $knownhosts= '';
+
+ my $hostsq= $dbh_tests->prepare(<<END);
+ SELECT val FROM runvars
+ WHERE flight=? AND name LIKE '%host'
+ GROUP BY val
+END
+ $hostsq->execute($flight);
+ while (my ($node) = $hostsq->fetchrow_array()) {
+ my $longname= "$node.$c{TestHostDomain}";
+ my (@hostent)= gethostbyname($longname);
+ if (!@hostent) {
+ logm("skipping host key for nonexistent host $longname");
+ next;
+ }
+ my $specs= join ',', $longname, $node, map {
+ join '.', unpack 'W4', $_;
+ } @hostent[4..$#hostent];
+ logm("adding host key for $specs");
+ $knownhosts.= "$specs ".$hostkey;
+ }
+ $hostsq->finish();
+
+ $knownhosts.= "localhost,127.0.0.1 ".$hostkey;
+ my $knownhosts_url= create_webfile($ho, "known_hosts$sfx", $knownhosts);
+
+ preseed_hook_command($ho, 'late_command', $sfx, <<END);
+#!/bin/sh
+set -ex
+
+r=/target/root
+cd \$r
+
+umask 022
+mkdir .ssh
+wget -O .ssh/authorized_keys '$authkeys_url'
+wget -O .ssh/known_hosts '$knownhosts_url'
+
+u=osstest
+h=/home/\$u
+mkdir /target\$h/.ssh
+cp .ssh/authorized_keys /target\$h/.ssh
+chroot /target chown -R \$u.\$u \$h/.ssh
+END
+}
+
sub preseed_base ($$$;@) {
my ($ho,$suite,$extra_packages,%xopts) = @_;
@@ -639,43 +694,12 @@ END
sub preseed_create ($$;@) {
my ($ho, $sfx, %xopts) = @_;
- my $authkeys_url= create_webfile($ho, "authkeys$sfx", authorized_keys());
-
- my $hostkeyfile= "$c{OverlayLocal}/etc/ssh/ssh_host_rsa_key.pub";
- my $hostkey= get_filecontents($hostkeyfile);
- chomp($hostkey); $hostkey.="\n";
- my $knownhosts= '';
-
my $disk= $xopts{DiskDevice} || '/dev/sda';
my $suite= $xopts{Suite} || $c{DebianSuite};
my $d_i= $ho->{Tftp}{Path}.'/'.$ho->{Tftp}{DiBase}.'/'.$r{arch}.'/'.
$c{TftpDiVersion}.'-'.$ho->{Suite};
- my $hostsq= $dbh_tests->prepare(<<END);
- SELECT val FROM runvars
- WHERE flight=? AND name LIKE '%host'
- GROUP BY val
-END
- $hostsq->execute($flight);
- while (my ($node) = $hostsq->fetchrow_array()) {
- my $longname= "$node.$c{TestHostDomain}";
- my (@hostent)= gethostbyname($longname);
- if (!@hostent) {
- logm("skipping host key for nonexistent host $longname");
- next;
- }
- my $specs= join ',', $longname, $node, map {
- join '.', unpack 'W4', $_;
- } @hostent[4..$#hostent];
- logm("adding host key for $specs");
- $knownhosts.= "$specs ".$hostkey;
- }
- $hostsq->finish();
-
- $knownhosts.= "localhost,127.0.0.1 ".$hostkey;
- my $knownhosts_url= create_webfile($ho, "known_hosts$sfx", $knownhosts);
-
my $overlays= '';
my $create_overlay= sub {
my ($srcdir, $tfilename) = @_;
@@ -740,6 +764,8 @@ ls -l /dev/sd*
true
END
+ preseed_ssh($ho, $sfx);
+
preseed_hook_command($ho, 'late_command', $sfx, <<END);
#!/bin/sh
set -ex
@@ -747,17 +773,6 @@ set -ex
r=/target/root
cd \$r
-umask 022
-mkdir .ssh
-wget -O .ssh/authorized_keys '$authkeys_url'
-wget -O .ssh/known_hosts '$knownhosts_url'
-
-u=osstest
-h=/home/\$u
-mkdir /target\$h/.ssh
-cp .ssh/authorized_keys /target\$h/.ssh
-chroot /target chown -R \$u.\$u \$h/.ssh
-
echo FANCYTTY=0 >> /target/etc/lsb-base-logging.sh
$overlays
diff --git a/ts-debian-hvm-install b/ts-debian-hvm-install
index 95fce9a..02c142b 100755
--- a/ts-debian-hvm-install
+++ b/ts-debian-hvm-install
@@ -44,7 +44,6 @@ our $gho;
sub preseed () {
my $preseed_file = preseed_base($gho,'wheezy','',());
- my $authkeys = join('\\n', split(/\n/, authorized_keys()));
$preseed_file .= (<<END);
d-i netcfg/get_hostname string $gn
@@ -74,10 +73,10 @@ d-i apt-setup/cdrom/set-first boolean false
d-i preseed/late_command string \\
in-target mkdir -p /boot/efi/EFI/boot; \\
in-target cp /boot/efi/EFI/debian/grubx64.efi /boot/efi/EFI/boot/bootx64.efi ;\\
- in-target mkdir -p /root/.ssh; \\
- in-target sh -c "echo -e '$authkeys'> /root/.ssh/authorized_keys";
END
+ preseed_ssh($ho,'');
+
$preseed_file .= preseed_hook_cmds();
return $preseed_file;
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* Re: [PATCH OSSTEST v4 05/25] Debian: refactor preseeding of .ssh directories
2015-04-02 9:16 ` [PATCH OSSTEST v4 05/25] Debian: refactor preseeding of .ssh directories Ian Campbell
@ 2015-04-02 11:33 ` Ian Jackson
2015-04-02 12:39 ` Ian Campbell
0 siblings, 1 reply; 40+ messages in thread
From: Ian Jackson @ 2015-04-02 11:33 UTC (permalink / raw)
To: Ian Campbell; +Cc: longtaox.pang, xen-devel
Ian Campbell writes ("[PATCH OSSTEST v4 05/25] Debian: refactor preseeding of .ssh directories"):
> Causes known_hosts to be consistently created as well as ~osstest/.ssh
> to be consistently populated (it previsouly wasn't for HVM guests).
Most of this is code motion, AFAICT. The difference is just that
preseed_ssh is now called from ts-debian-hvm-install, too ?
Why should preseed_ssh not be used by preseed_base ?
Ian.
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [PATCH OSSTEST v4 05/25] Debian: refactor preseeding of .ssh directories
2015-04-02 11:33 ` Ian Jackson
@ 2015-04-02 12:39 ` Ian Campbell
0 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 12:39 UTC (permalink / raw)
To: Ian Jackson; +Cc: longtaox.pang, xen-devel
On Thu, 2015-04-02 at 12:33 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[PATCH OSSTEST v4 05/25] Debian: refactor preseeding of .ssh directories"):
> > Causes known_hosts to be consistently created as well as ~osstest/.ssh
> > to be consistently populated (it previsouly wasn't for HVM guests).
>
> Most of this is code motion, AFAICT. The difference is just that
> preseed_ssh is now called from ts-debian-hvm-install, too ?
>
> Why should preseed_ssh not be used by preseed_base ?
Good question, I think it could be. At least I can't think of any sort
of preseedable thing we would want to install which doesn't want this.
I'll try it for next time.
preseed_base today is only concerned with creating the base text for the
pressed file, whereas preseed_ssh does more stuff like adding webfiles
and adding preseed hooks.
I take it you don't see that as a problem?
Ian.
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 06/25] Debian: Refactor installation of overlays, so it can be used for guests too
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (4 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 05/25] Debian: refactor preseeding of .ssh directories Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 11:35 ` Ian Jackson
2015-04-02 9:16 ` [PATCH OSSTEST v4 07/25] Debian: add preseed_create_guest helper Ian Campbell
` (19 subsequent siblings)
25 siblings, 1 reply; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: longtaox.pang, Ian Campbell, xen-devel
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: longtaox.pang@intel.com
---
v3: New patch
---
Osstest/Debian.pm | 57 ++++++++++++++++++++++++++++---------------------------
1 file changed, 29 insertions(+), 28 deletions(-)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index db89326..13cd147 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -38,7 +38,9 @@ BEGIN {
preseed_base
preseed_create
preseed_ssh
- preseed_hook_command preseed_hook_installscript preseed_hook_cmds
+ preseed_hook_command preseed_hook_installscript
+ preseed_hook_overlay
+ preseed_hook_cmds
di_installcmdline_core
);
%EXPORT_TAGS = ( );
@@ -700,26 +702,6 @@ sub preseed_create ($$;@) {
my $d_i= $ho->{Tftp}{Path}.'/'.$ho->{Tftp}{DiBase}.'/'.$r{arch}.'/'.
$c{TftpDiVersion}.'-'.$ho->{Suite};
- my $overlays= '';
- my $create_overlay= sub {
- my ($srcdir, $tfilename) = @_;
- my $url= create_webfile($ho, "$tfilename$sfx", sub {
- my ($fh) = @_;
- contents_make_cpio($fh, 'ustar', $srcdir);
- });
- $overlays .= <<END;
-wget -O overlay.tar '$url'
-cd /target
-tar xf \$r/overlay.tar
-cd \$r
-rm overlay.tar
-
-END
- };
-
- $create_overlay->('overlay', 'overlay.tar');
- $create_overlay->($c{OverlayLocal}, 'overlay-local.tar');
-
preseed_hook_installscript($ho, $sfx,
'/lib/partman/init.d', '000override-parted-devices', <<END);
#!/bin/sh
@@ -765,19 +747,14 @@ true
END
preseed_ssh($ho, $sfx);
+ preseed_hook_overlay($ho, $sfx, 'overlay', 'overlay.tar');
+ preseed_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar');
preseed_hook_command($ho, 'late_command', $sfx, <<END);
#!/bin/sh
set -ex
-r=/target/root
-cd \$r
-
echo FANCYTTY=0 >> /target/etc/lsb-base-logging.sh
-
-$overlays
-
-echo latecmd done.
END
my $dtbs = "$d_i/dtbs.tar.gz";
@@ -966,6 +943,30 @@ chmod +x '$installer_pathname'
END
}
+sub preseed_hook_overlay ($$$$) {
+ my ($ho, $sfx, $srcdir, $tfilename) = @_;
+ my $url= create_webfile($ho, "$tfilename$sfx", sub {
+ my ($fh) = @_;
+ contents_make_cpio($fh, 'ustar', $srcdir);
+ });
+ preseed_hook_command($ho, 'late_command', $sfx, <<END);
+#!/bin/sh
+set -ex
+
+r=/target/root
+cd \$r
+
+umask 022
+
+wget -O overlay.tar '$url'
+cd /target
+tar xf \$r/overlay.tar
+cd \$r
+rm overlay.tar
+
+END
+}
+
sub preseed_hook_cmds () {
my $preseed;
foreach my $di_key (keys %preseed_cmds) {
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 07/25] Debian: add preseed_create_guest helper
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (5 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 06/25] Debian: Refactor installation of overlays, so it can be used for guests too Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 11:40 ` Ian Jackson
2015-04-02 9:16 ` [PATCH OSSTEST v4 08/25] make-flight: Handle $BUILD_LVEXTEND_MAX in mfi-common:create_build_jobs() Ian Campbell
` (18 subsequent siblings)
25 siblings, 1 reply; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Creates a preseed file suitable for use in a PV guest
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v4: Rebase, pass $ho to preseed_base
v3: Handle $xopts{ExtraPreseed} undefined in preseed_base
---
Osstest/Debian.pm | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 13cd147..993acc7 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -37,6 +37,7 @@ BEGIN {
%preseed_cmds
preseed_base
preseed_create
+ preseed_create_guest
preseed_ssh
preseed_hook_command preseed_hook_installscript
preseed_hook_overlay
@@ -611,6 +612,9 @@ END
sub preseed_base ($$$;@) {
my ($ho,$suite,$extra_packages,%xopts) = @_;
+ $extra_packages ||= '';
+ $xopts{ExtraPreseed} ||= '';
+
my $preseed = <<"END";
d-i mirror/suite string $suite
@@ -693,6 +697,31 @@ END
return $preseed;
}
+sub preseed_create_guest ($$;@) {
+ my ($ho, $sfx, %xopts) = @_;
+
+ my $suite= $xopts{Suite} || $c{DebianSuite};
+
+ my $extra_packages;
+
+ my $preseed_file= preseed_base($ho, $suite, $extra_packages, %xopts);
+ $preseed_file.= (<<END);
+d-i partman-auto/method string regular
+d-i partman-auto/choose_recipe \\
+ select All files in one partition (recommended for new users)
+
+d-i grub-installer/bootdev string /dev/xvda
+
+END
+
+ preseed_ssh($ho, $sfx);
+ preseed_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar');
+
+ $preseed_file .= preseed_hook_cmds();
+
+ return create_webfile($ho, "preseed$sfx", $preseed_file);
+}
+
sub preseed_create ($$;@) {
my ($ho, $sfx, %xopts) = @_;
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* Re: [PATCH OSSTEST v4 07/25] Debian: add preseed_create_guest helper
2015-04-02 9:16 ` [PATCH OSSTEST v4 07/25] Debian: add preseed_create_guest helper Ian Campbell
@ 2015-04-02 11:40 ` Ian Jackson
2015-04-02 12:44 ` Ian Campbell
0 siblings, 1 reply; 40+ messages in thread
From: Ian Jackson @ 2015-04-02 11:40 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel
Ian Campbell writes ("[PATCH OSSTEST v4 07/25] Debian: add preseed_create_guest helper"):
> Creates a preseed file suitable for use in a PV guest
>
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> ---
> v4: Rebase, pass $ho to preseed_base
> v3: Handle $xopts{ExtraPreseed} undefined in preseed_base
> ---
> Osstest/Debian.pm | 29 +++++++++++++++++++++++++++++
> 1 file changed, 29 insertions(+)
>
> diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
> index 13cd147..993acc7 100644
> --- a/Osstest/Debian.pm
> +++ b/Osstest/Debian.pm
> @@ -37,6 +37,7 @@ BEGIN {
> %preseed_cmds
> preseed_base
> preseed_create
> + preseed_create_guest
> preseed_ssh
> preseed_hook_command preseed_hook_installscript
> preseed_hook_overlay
> @@ -611,6 +612,9 @@ END
> sub preseed_base ($$$;@) {
> my ($ho,$suite,$extra_packages,%xopts) = @_;
>
> + $extra_packages ||= '';
This is a little odd. You are changing preseed_base to tolerate an
unspecified $extra_packages, but it's still a mandatory argument.
Maybe you should mvoe the `;' in the sub prototype.
Or maybe you just wanted to write:
+ my $extra_packages = '';
instead of:
> + my $extra_packages;
> + preseed_ssh($ho, $sfx);
> + preseed_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar');
Perhaps the OverlayLocal should always be done, rather than having to
be explicitly specified by all the callers of preseed_base ?
Thanks,
Ian.
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [PATCH OSSTEST v4 07/25] Debian: add preseed_create_guest helper
2015-04-02 11:40 ` Ian Jackson
@ 2015-04-02 12:44 ` Ian Campbell
0 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 12:44 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-devel
On Thu, 2015-04-02 at 12:40 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[PATCH OSSTEST v4 07/25] Debian: add preseed_create_guest helper"):
> > Creates a preseed file suitable for use in a PV guest
> >
> > Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> > ---
> > v4: Rebase, pass $ho to preseed_base
> > v3: Handle $xopts{ExtraPreseed} undefined in preseed_base
> > ---
> > Osstest/Debian.pm | 29 +++++++++++++++++++++++++++++
> > 1 file changed, 29 insertions(+)
> >
> > diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
> > index 13cd147..993acc7 100644
> > --- a/Osstest/Debian.pm
> > +++ b/Osstest/Debian.pm
> > @@ -37,6 +37,7 @@ BEGIN {
> > %preseed_cmds
> > preseed_base
> > preseed_create
> > + preseed_create_guest
> > preseed_ssh
> > preseed_hook_command preseed_hook_installscript
> > preseed_hook_overlay
> > @@ -611,6 +612,9 @@ END
> > sub preseed_base ($$$;@) {
> > my ($ho,$suite,$extra_packages,%xopts) = @_;
> >
> > + $extra_packages ||= '';
>
> This is a little odd. You are changing preseed_base to tolerate an
> unspecified $extra_packages, but it's still a mandatory argument.
> Maybe you should mvoe the `;' in the sub prototype.
>
> Or maybe you just wanted to write:
> + my $extra_packages = '';
> instead of:
> > + my $extra_packages;
It looks like I actually do, I suspect that defaulting must be a
leftover.
>
> > + preseed_ssh($ho, $sfx);
> > + preseed_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar');
>
> Perhaps the OverlayLocal should always be done, rather than having to
> be explicitly specified by all the callers of preseed_base ?
Having moved preseed_ssh to preseed_base this certainly makes sense.
Ian.
^ permalink raw reply [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 08/25] make-flight: Handle $BUILD_LVEXTEND_MAX in mfi-common:create_build_jobs()
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (6 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 07/25] Debian: add preseed_create_guest helper Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-14 15:45 ` Ian Jackson
2015-04-02 9:16 ` [PATCH OSSTEST v4 09/25] distros: add support for installing Debian PV guests via d-i, flight and jobs Ian Campbell
` (17 subsequent siblings)
25 siblings, 1 reply; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v3: New patch
---
make-flight | 4 ----
mfi-common | 4 ++++
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/make-flight b/make-flight
index cc8ecdb..ec0a20c 100755
--- a/make-flight
+++ b/make-flight
@@ -36,10 +36,6 @@ defguestsuite=`getconfig GuestDebianSuite`
if [ x$buildflight = x ]; then
- if [ "x$BUILD_LVEXTEND_MAX" != x ]; then
- BUILD_RUNVARS+=" build_lvextend_max=$BUILD_LVEXTEND_MAX "
- fi
-
create_build_jobs
else
diff --git a/mfi-common b/mfi-common
index 16fc8c5..a9e966f 100644
--- a/mfi-common
+++ b/mfi-common
@@ -63,6 +63,10 @@ create_build_jobs () {
local enable_ovmf
local build_hostflags
+ if [ "x$BUILD_LVEXTEND_MAX" != x ]; then
+ BUILD_RUNVARS+=" build_lvextend_max=$BUILD_LVEXTEND_MAX "
+ fi
+
for arch in ${BUILD_ARCHES- i386 amd64 armhf }; do
if [ "x$arch" = xdisable ]; then continue; fi
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 09/25] distros: add support for installing Debian PV guests via d-i, flight and jobs
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (7 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 08/25] make-flight: Handle $BUILD_LVEXTEND_MAX in mfi-common:create_build_jobs() Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 10/25] distros: support booting Debian PV (d-i installed) guests with pvgrub Ian Campbell
` (16 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
This patch introduces ts-debian-di-install which can install Debian
from a netboot (PXE) debian installer image. By default it installs
from the d-i image used by osstest (using the special Xen PV guest
enabled flavour where necessary) but it can also install the current
release versions from Squeeze onwards (up to and including Sid) and
the d-i daily builds. This is controlled by runvars {Guest}_diver =
osstest|current and {Guest}_dist = squeeze|...|sid. The resulting
guests boot the distro kernel using pygrub (pvgrub will follow).
The distros flights differ substantially from the existing flights.
Introduce make-distros-flight using the functionality previously
refactored into mfi-common. The new flight tests all versions of
Debian from Squeeze onward as an amd64, i386 and armhf guests (armhf
from Jessie onwards only) using the usual smoke tests.
Test names are suffixed -pygrub pending the addition of pvgrub
variants in a future commit.
Add the new cases to sg-run-job
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v4: use guest create
v3: $BUILD_LVEXTEND_MAX now handled in mfi-common
Consolidate setting of ruvars
Include $flight and $job in tmpdir name
Use Osstest::Debian::di_installcmdline_core
Document the usage of get_host_property on a guest object
Correct ARM netboot paths
Include bootloader in test name
Should include -pv too?
console= repetition for Jessie onwards.
Wait for up to an hour for the install. I'd seen timeouts right at
the end of the install with the previous value
fixup: distros: add support for installing Debian PV guests via d-i, flight and jobs
---
Osstest/TestSupport.pm | 3 +
make-distros-flight | 98 +++++++++++++++++++++++++++++
sg-run-job | 11 ++++
ts-debian-di-install | 168 +++++++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 280 insertions(+)
create mode 100755 make-distros-flight
create mode 100755 ts-debian-di-install
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 8f67f7e..3cd8d22 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -906,8 +906,11 @@ sub propname_massage ($) {
return $prop;
}
+# It is fine to call this on a guest object too, in which case it will
+# always return $defval.
sub get_host_property ($$;$) {
my ($ho, $prop, $defval) = @_;
+ return $defval unless $ho->{Properties};
my $val = $ho->{Properties}{propname_massage($prop)};
return defined($val) ? $val : $defval;
}
diff --git a/make-distros-flight b/make-distros-flight
new file mode 100755
index 0000000..5067342
--- /dev/null
+++ b/make-distros-flight
@@ -0,0 +1,98 @@
+#!/bin/bash
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+
+set -e
+
+branch=$1
+xenbranch=$2
+blessing=$3
+buildflight=$4
+
+flight=`./cs-flight-create $blessing $branch`
+
+. cri-common
+. ap-common
+. mfi-common
+
+defsuite=`getconfig DebianSuite`
+defguestsuite=`getconfig GuestDebianSuite`
+
+if [ x$buildflight = x ]; then
+
+ WANT_XEND=false REVISION_LINUX_OLD=disable
+
+ create_build_jobs
+
+else
+
+ bfi=$buildflight.
+
+fi
+
+job_create_test_filter_callback () {
+ if [ "$xenarch" = "i386" ]; then return 1; fi
+ return 0
+}
+
+test_matrix_branch_filter_callback () {
+ :
+}
+
+test_do_one_netboot () {
+ job_create_test \
+ test-$xenarch$kern-$dom0arch-$domU-$dist-netboot-pygrub \
+ test-debian-di xl $xenarch $dom0arch \
+ kernbuildjob=${bfi}build-$dom0arch-$kernbuild \
+ debian_arch=$domU \
+ debian_dist=$dist \
+ debian_method=netboot \
+ debian_diver=current \
+ all_hostflags=$most_hostflags
+}
+
+test_matrix_do_one () {
+ case ${xenarch} in
+ amd64) domUarches="amd64 i386";;
+ armhf) domUarches="armhf";;
+ esac
+
+ for domU in $domUarches ; do
+ for dist in squeeze wheezy jessie sid daily ; do
+ case ${domU}_${dist} in
+ armhf_squeeze) continue;; # No armhf in Squeeze
+ armhf_wheezy) continue;; # No armhf guest support in Wheezy
+ *) ;;
+ esac
+
+ test_do_one_netboot
+
+ done
+
+ done
+}
+
+test_matrix_iterate
+
+echo $flight
+
+# Local variables:
+# mode: sh
+# sh-basic-offset: 2
+# indent-tabs-mode: nil
+# End:
diff --git a/sg-run-job b/sg-run-job
index 0ec314f..284cd50 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -258,6 +258,17 @@ proc run-job/test-debian {} {
test-guest debian
}
+proc install-guest-debian-di {} {
+ run-ts . = ts-debian-di-install
+ run-ts . = ts-guest-start + debian
+}
+
+proc need-hosts/test-debian-di {} { return host }
+proc run-job/test-debian-di {} {
+ install-guest-debian-di
+ test-guest debian
+}
+
proc need-hosts/test-freebsd {} { return host }
proc run-job/test-freebsd {} {
run-ts . = ts-freebsd-install
diff --git a/ts-debian-di-install b/ts-debian-di-install
new file mode 100755
index 0000000..a829185
--- /dev/null
+++ b/ts-debian-di-install
@@ -0,0 +1,168 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use Osstest::Debian;
+use Osstest::TestSupport;
+
+tsreadconfig();
+
+our ($whhost,$gn) = @ARGV;
+$whhost ||= 'host';
+$gn ||= 'debian';
+
+our $ho= selecthost($whhost);
+
+our $ram_mb= 512;
+our $disk_mb= 10000;
+
+our $guesthost= "$gn.guest.osstest";
+our $gho;
+
+sub prep () {
+ target_install_packages_norec($ho, qw(lvm2));
+
+ $gho= prepareguest($ho, $gn, $guesthost, 22,
+ $disk_mb, 40);
+
+ prepareguest_part_lvmdisk($ho, $gho, $disk_mb);
+
+ target_cmd_root($ho, "umount $gho->{Lvdev} ||:");
+}
+
+sub setup_netboot($$$)
+{
+ my ($didir, $arch, $suite) = @_;
+
+ my $di_ver= $r{"$gho->{Guest}_diver"} || "osstest";
+
+ my ($kernel,$initrd);
+
+ if ( $di_ver eq "osstest" ) {
+ my $di_path = $c{TftpPath}.'/'.$ho->{Tftp}{DiBase}.'/'.$r{arch}.'/'.$c{TftpDiVersion}.'-'.$ho->{Suite};
+
+ $kernel = "$di_path/vmlinuz-xen";
+ $initrd = "$di_path/initrd.gz-xen";
+
+ target_putfile_root($ho, 60, $kernel, "$didir/kernel_${suite}_${arch}");
+ target_putfile_root($ho, 60, $initrd, "$didir/initrd_${suite}_${arch}");
+
+ } else {
+ my $mirror = "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath}";
+
+ my $di_url = $suite eq "daily" ?
+ "http://d-i.debian.org/daily-images/$arch/daily/netboot" :
+ "$mirror/dists/$suite/main/installer-$arch/$di_ver/images/netboot";
+
+ $di_url .= "/xen" if $arch =~ m/amd64|i386/;
+ $di_url .= "/debian-installer/arm64" if $arch =~ /arm64/;
+
+ $kernel = "$di_url/vmlinuz";
+ $initrd = "$di_url/initrd.gz";
+
+ target_fetchurl($ho, $kernel, "$didir/kernel_${suite}_${arch}");
+ target_fetchurl($ho, $initrd, "$didir/initrd_${suite}_${arch}");
+ }
+
+ store_runvar("$gho->{Guest}_netboot_kernel", $kernel);
+ store_runvar("$gho->{Guest}_netboot_initrd", $initrd);
+
+ return <<END;
+kernel = "$didir/kernel_${suite}_${arch}"
+ramdisk = "$didir/initrd_${suite}_${arch}"
+END
+}
+sub ginstall () {
+ my $arch= $r{"$gho->{Guest}_arch"};
+ my $method= $r{"$gho->{Guest}_method"};
+
+ my $tmpdir= "/root/$flight-$job-di";
+ target_cmd_root($ho, <<END);
+rm -rf $tmpdir
+mkdir $tmpdir
+END
+
+ my ($method_cfg, $ps_url, $extra_disk);
+
+ if ( $method eq "netboot" )
+ {
+ my $suite= $r{"$gho->{Guest}_dist"};
+ logm("$method $suite/$arch");
+
+ $method_cfg = setup_netboot($tmpdir, $arch, $suite);
+
+ $suite = "sid" if $suite eq "daily";
+
+ $ps_url = preseed_create_guest($gho, '', Suite=>$suite);
+
+ $extra_disk = "";
+ }
+ else
+ {
+ die "$method";
+ }
+
+ my @cmdline = ();
+ push @cmdline, "debian-installer/exit/always_halt=true";
+ push @cmdline, "domain=$c{TestHostDomain}";
+ push @cmdline, "console=hvc0";
+ push @cmdline, di_installcmdline_core($gho, $ps_url);
+ push @cmdline, "--";
+ # See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=762007 for
+ # why this is repeated.
+ push @cmdline, "console=hvc0";
+
+ my $cmdline = join(" ", @cmdline);
+
+ my %install_xopts = (
+ OnPowerOff => "preserve"
+ );
+
+ prepareguest_part_xencfg($ho, $gho, $ram_mb, \%install_xopts, <<END);
+$method_cfg
+extra = "$cmdline"
+#
+disk = [
+ $extra_disk 'phy:$gho->{Lvdev},xvda,w'
+ ]
+END
+
+ guest_create($gho);
+
+ guest_checkrunning($ho, $gho) or die "$gho->{Name} not running";
+
+ guest_await_shutdown($ho,$gho,3600);
+ guest_destroy($gho);
+
+ my $blcfg = <<END;
+bootloader = "pygrub"
+END
+
+ prepareguest_part_xencfg($ho, $gho, $ram_mb, {}, <<END);
+$blcfg
+#
+disk = [
+ 'phy:$gho->{Lvdev},xvda,w'
+ ]
+END
+ return;
+}
+
+prep();
+ginstall();
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 10/25] distros: support booting Debian PV (d-i installed) guests with pvgrub.
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (8 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 09/25] distros: add support for installing Debian PV guests via d-i, flight and jobs Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 11/25] distros: Support pvgrub for Wheezy too Ian Campbell
` (15 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
This requires the use of the pv-grub-menu package which is in Jessie
onwards. (it is in wheezy-backports which is the subject of a
subsequent patch).
The bootloader to use is specified via a runvar {Guest}_bootloader.
Adjust make-distros-flight to use pvgrub for some subset of i386 and
amd64 guests to get coverage.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v3: Define and use arch_debian2xen and arch_xen2debian
Avoid pv-grub-x86_64.gz on i386 dom0, we don't built it there.
Fiddle with py vs pv grub stripy a bit.
---
Osstest.pm | 7 +++++++
Osstest/Debian.pm | 2 +-
make-distros-flight | 21 ++++++++++++++++++++-
ts-debian-di-install | 11 +++++++++--
4 files changed, 37 insertions(+), 4 deletions(-)
diff --git a/Osstest.pm b/Osstest.pm
index 7989129..c0e3ee7 100644
--- a/Osstest.pm
+++ b/Osstest.pm
@@ -38,6 +38,7 @@ BEGIN {
db_begin_work
ensuredir get_filecontents_core_quiet system_checked
nonempty visible_undef show_abs_time
+ %arch_debian2xen %arch_xen2debian
);
%EXPORT_TAGS = ( );
@@ -49,6 +50,12 @@ our $mjobdb;
our $dbh_tests;
+our %arch_debian2xen = qw(i386 x86_32
+ amd64 x86_64
+ armhf armhf);
+our %arch_xen2debian;
+$arch_xen2debian{$arch_debian2xen{$_}} = $_ foreach keys %arch_debian2xen;
+
#---------- static default config settings ----------
our %c = qw(
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 993acc7..cbc7172 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -702,7 +702,7 @@ sub preseed_create_guest ($$;@) {
my $suite= $xopts{Suite} || $c{DebianSuite};
- my $extra_packages;
+ my $extra_packages = "pv-grub-menu" if $xopts{PvMenuLst};
my $preseed_file= preseed_base($ho, $suite, $extra_packages, %xopts);
$preseed_file.= (<<END);
diff --git a/make-distros-flight b/make-distros-flight
index 5067342..80df21b 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -55,14 +55,33 @@ test_matrix_branch_filter_callback () {
}
test_do_one_netboot () {
+ stripy bootloader pvgrub pygrub \
+ "$xenarch" "amd64" \
+ "$dom0arch" "i386" \
+ "$domU" "amd64" \
+ "$dist" "sid"
+
+ case ${dom0arch}_${domU}_${dist} in
+ arm*_arm*_*) bootloader="pygrub";; # no pvgrub for arm
+
+ # Needs a menu.lst, not present in Squeeze+ due to switch to grub2,
+ # workedaround in Jessie+ with pv-grub-menu package.
+ *_squeeze) bootloader="pygrub";;
+ *_wheezy) bootloader="pygrub";;
+
+ # pv-grub-x86_64.gz is not built by 32-bit dom0 userspace build.
+ i386_amd64_*) bootloader="pygrub";;
+ esac
+
job_create_test \
- test-$xenarch$kern-$dom0arch-$domU-$dist-netboot-pygrub \
+ test-$xenarch$kern-$dom0arch-$domU-$dist-netboot-$bootloader \
test-debian-di xl $xenarch $dom0arch \
kernbuildjob=${bfi}build-$dom0arch-$kernbuild \
debian_arch=$domU \
debian_dist=$dist \
debian_method=netboot \
debian_diver=current \
+ debian_bootloader=$bootloader \
all_hostflags=$most_hostflags
}
diff --git a/ts-debian-di-install b/ts-debian-di-install
index a829185..73cb43d 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -93,6 +93,8 @@ sub ginstall () {
my $method= $r{"$gho->{Guest}_method"};
my $tmpdir= "/root/$flight-$job-di";
+ my $bl= $r{"$gho->{Guest}_bootloader"};
+
target_cmd_root($ho, <<END);
rm -rf $tmpdir
mkdir $tmpdir
@@ -109,7 +111,7 @@ END
$suite = "sid" if $suite eq "daily";
- $ps_url = preseed_create_guest($gho, '', Suite=>$suite);
+ $ps_url = preseed_create_guest($gho, '', Suite=>$suite, PvMenuLst=>($bl eq "pvgrub"));
$extra_disk = "";
}
@@ -150,7 +152,12 @@ END
guest_await_shutdown($ho,$gho,3600);
guest_destroy($gho);
- my $blcfg = <<END;
+ my $xenarch = $arch_debian2xen{$arch};
+ my $pvgrub = "/usr/local/lib/xen/boot/pv-grub-$xenarch.gz";
+ my $blcfg = $bl eq "pvgrub" ? <<END : <<END;
+kernel = "$pvgrub"
+extra = "(hd0,0)/boot/grub/menu.lst"
+END
bootloader = "pygrub"
END
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 11/25] distros: Support pvgrub for Wheezy too.
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (9 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 10/25] distros: support booting Debian PV (d-i installed) guests with pvgrub Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 12/25] distros: support PV guest install from Debian netinst media Ian Campbell
` (14 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
This requires us to install pv-grub-menu from backports, which we do
using a late_command.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v3:
- Remove spurious () from <<(END) (and the prexisting "" too)
- Remove $xopts{EnableBackports} and automatically handle the need to add
backports in preseed_base.
- Install via late_command not apt-setup, since the former has
issues, hence subject drops "attempt to..."
---
Osstest/Debian.pm | 44 +++++++++++++++++++++++++++++++++++++++++++-
make-distros-flight | 4 ++--
2 files changed, 45 insertions(+), 3 deletions(-)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index cbc7172..b37540e 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -689,6 +689,17 @@ END
d-i clock-setup/ntp-server string $ntpserver
END
+ # deb http://ftp.debian.org/debian/ wheezy-backports main
+ $preseed .= <<END if $extra_packages =~ m#\b/$suite-backports\b#;
+d-i apt-setup/local0/repository string \\
+ http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} $suite-backports main
+d-i apt-setup/local0/comment string $suite backports
+END
+# Jessie onwards...
+# $preseed .= <<END if $extra_packages =~ m#\b/$suite-backports\b#;
+#d-i apt-setup/services-select multiselect security, updates, backports
+#END
+
$preseed .= <<"END";
### END OF DEBIAN PRESEED BASE
@@ -702,7 +713,38 @@ sub preseed_create_guest ($$;@) {
my $suite= $xopts{Suite} || $c{DebianSuite};
- my $extra_packages = "pv-grub-menu" if $xopts{PvMenuLst};
+ my $extra_packages = "";
+ if ($xopts{PvMenuLst}) {
+ if ($suite =~ m/wheezy/) {
+ # pv-grub-menu/wheezy-backports + using apt-setup to add
+ # backports results in iproute, ifupdown and
+ # isc-dhcp-client getting removed because tasksel's
+ # invocation of apt-get install somehow decides the
+ # iproute2 from wheezy-backports is a thing it wants to
+ # install. So instead lets fake it with a late command...
+ #
+ # This also has the bonus of working round an issue with
+ # 1.2.1~bpo70+1 which created an invalid menu.lst using
+ # "root(/dev/xvda,0)" which pvgrub cannot parse because
+ # the Grub device.map isn't present at pkgsel/include time
+ # but it is by late_command time. This was fixed by
+ # version 1.3 which is in Jessie onwards.
+ preseed_hook_command($ho, 'late_command', $sfx, <<END);
+#!/bin/sh
+set -ex
+
+(
+ echo
+ echo \\\# $suite backports
+ echo deb http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} $suite-backports main
+) >> /target/etc/apt/sources.list
+in-target apt-get update
+in-target apt-get install -y -t wheezy-backports pv-grub-menu
+END
+ } else {
+ $extra_packages = "pv-grub-menu";
+ }
+ }
my $preseed_file= preseed_base($ho, $suite, $extra_packages, %xopts);
$preseed_file.= (<<END);
diff --git a/make-distros-flight b/make-distros-flight
index 80df21b..97df89a 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -65,9 +65,9 @@ test_do_one_netboot () {
arm*_arm*_*) bootloader="pygrub";; # no pvgrub for arm
# Needs a menu.lst, not present in Squeeze+ due to switch to grub2,
- # workedaround in Jessie+ with pv-grub-menu package.
+ # workedaround in Wheezy+ with pv-grub-menu package (backports in Wheezy,
+ # in Jessie+ main).
*_squeeze) bootloader="pygrub";;
- *_wheezy) bootloader="pygrub";;
# pv-grub-x86_64.gz is not built by 32-bit dom0 userspace build.
i386_amd64_*) bootloader="pygrub";;
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 12/25] distros: support PV guest install from Debian netinst media.
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (10 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 11/25] distros: Support pvgrub for Wheezy too Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 13/25] Test pygrub and pvgrub on the regular flights Ian Campbell
` (13 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
The netinst media are iso images containing a base Debian install and
some (image size dependent) additional tasks.
On x86 the "multiarch" iso flavour contains a Xen capable kernel for
both i386 and amd64 so use that.
This adds support for two classes of ISO, the CD sized ones which are
built nightly and the DVD sized ones which are built weekly.
The images are downloaded using jigdo which sources the majority of
the data from a local Debian mirror, for this reason I have not
worried about the fact that the i386 and amd64 tests are downloading
the same thing (adding a specific download job would require finding
up to 4GB of scratch space for each flight).
The ISOs booted using pygrub which can extract the kernel and initrd
from a ISO image. The resulting guests are also booted with pygrub
since the pv-grub-menu package is not available on the ISO images and
we have pvgrub coverage from the netboot tests.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v4: Support http_proxy in jigdo download
v3:
- Indent jigdo-lite script line to improve readability
- Wrap bootloader_args
- Fetch URL on target, to get a timeout. Use wget since that is arranged to
be present.
- include bootloader in test name.
---
Osstest/Debian.pm | 7 ++++--
make-distros-flight | 23 ++++++++++++++++++
ts-debian-di-install | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 95 insertions(+), 2 deletions(-)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index b37540e..831290c 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -616,8 +616,6 @@ sub preseed_base ($$$;@) {
$xopts{ExtraPreseed} ||= '';
my $preseed = <<"END";
-d-i mirror/suite string $suite
-
d-i debian-installer/locale string en_GB
d-i console-keymaps-at/keymap select gb
d-i keyboard-configuration/xkb-keymap string en_GB
@@ -689,6 +687,11 @@ END
d-i clock-setup/ntp-server string $ntpserver
END
+ # For CDROM the suite is part of the image
+ $preseed .= <<END unless $xopts{CDROM};
+d-i mirror/suite string $suite
+END
+
# deb http://ftp.debian.org/debian/ wheezy-backports main
$preseed .= <<END if $extra_packages =~ m#\b/$suite-backports\b#;
d-i apt-setup/local0/repository string \\
diff --git a/make-distros-flight b/make-distros-flight
index 97df89a..e3ec8f7 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -85,6 +85,20 @@ test_do_one_netboot () {
all_hostflags=$most_hostflags
}
+test_do_one_netinst () {
+ # Always pygrub since no pv-grub-menu on CD
+ job_create_test \
+ test-$xenarch$kern-$dom0arch-$domU-$cd-netinst-pygrub \
+ test-debian-di xl $xenarch $dom0arch \
+ kernbuildjob=${bfi}build-$dom0arch-$kernbuild \
+ debian_arch=$domU \
+ debian_cd=$cd \
+ debian_method=netinst \
+ debian_bootloader=pygrub \
+ all_hostflags=$most_hostflags
+
+}
+
test_matrix_do_one () {
case ${xenarch} in
amd64) domUarches="amd64 i386";;
@@ -103,6 +117,15 @@ test_matrix_do_one () {
done
+ for cd in current weekly ; do
+ case ${domU}_${dist} in
+ armhf_*) continue;; # No iso targets for armhf
+ *) ;;
+ esac
+
+ test_do_one_netinst
+
+ done
done
}
diff --git a/ts-debian-di-install b/ts-debian-di-install
index 73cb43d..62e9bb5 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -46,6 +46,63 @@ sub prep () {
target_cmd_root($ho, "umount $gho->{Lvdev} ||:");
}
+sub setup_netinst($$$)
+{
+ my ($didir, $arch, $cd) = @_;
+ my %arch_props = (
+ amd64 => { PathArch => "multi-arch", FileArch => "amd64-i386", IsoPath => "/install.amd/xen" },
+ i386 => { PathArch => "multi-arch", FileArch => "amd64-i386", IsoPath => "/install.386/xen" },
+ armhf => { PathArch => "armhf", FileArch => "armhf", IsoPath => "/install.armhf" }
+ );
+ my $props = $arch_props{$arch} or die "Unknown arch $arch";
+
+ target_install_packages($ho, qw(jigdo-file));
+
+ my $baseurl = $cd eq "current" ?
+ "http://cdimage.debian.org/debian-cd/current/$props->{PathArch}/jigdo-cd" :
+ "http://cdimage.debian.org/cdimage/weekly-builds/$props->{PathArch}/jigdo-cd";
+
+ my $filebase;
+
+ # Use the MD5SUMs file as an index
+ logm("Fetch index from $baseurl/MD5SUMS");
+ my $idx = target_cmd_output_root($ho, "wget --quiet -O - $baseurl/MD5SUMS");
+ foreach (split /\n/, $idx) {
+ m/^[0-9a-f]{32} (debian-.*-$props->{FileArch}-netinst)\.iso$/ or next;
+ $filebase = $1;
+ last;
+ }
+
+ die unless $filebase;
+
+ logm("Downloading $baseurl/$filebase.jigdo");
+ my $netinst_jigdo = "$baseurl/$filebase.jigdo";
+ # Jigdo uses wget internally, and so obeys $http_proxy. This seems
+ # simpler than the advice at
+ # https://www.debian.org/CD/jigdo-cd/#faq which is to edit
+ # ~/.jigdo-lite and change the wgetOpts setting.
+ my $useproxy = $c{DebianMirrorProxy} // $c{HttpProxy};
+ my $cmd = '';
+ $cmd .= <<END if $useproxy;
+ export http_proxy=$useproxy
+END
+ $cmd .= <<END;
+ cd $didir && jigdo-lite --noask $netinst_jigdo
+END
+ # Jigdo seems to use /etc/apt/sources.list or something, so this
+ # just works using the already configured mirror without
+ # additional configuration, which is good because there doesn't
+ # seem to be any support for such things, at least in Squeeze.
+ target_cmd_root($ho, $cmd, 3600);
+ store_runvar("$gho->{Guest}_netinst_jigdo", $netinst_jigdo);
+
+ return (<<END, "\"file:$didir/$filebase.iso,xvdd:cdrom,r\",");
+bootloader = "pygrub"
+bootloader_args = ["--kernel=$props->{IsoPath}/vmlinuz",
+ "--ramdisk=$props->{IsoPath}/initrd.gz"]
+END
+}
+
sub setup_netboot($$$)
{
my ($didir, $arch, $suite) = @_;
@@ -115,6 +172,16 @@ END
$extra_disk = "";
}
+ elsif ($method eq "netinst" )
+ {
+ my $cd = $r{"$gho->{Guest}_cd"};
+
+ logm("$method $cd/$arch");
+
+ ($method_cfg,$extra_disk) = setup_netinst($tmpdir, $arch, $cd);
+
+ $ps_url = preseed_create_guest($gho, '', CDROM=>1);
+ }
else
{
die "$method";
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 13/25] Test pygrub and pvgrub on the regular flights
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (11 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 12/25] distros: support PV guest install from Debian netinst media Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 14/25] distros: add branch infrastructure Ian Campbell
` (12 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Since we now have the ability to test these drop one of each of
pygrub, pvgrub-32 and pvgrub-64 into the standard flights. Omitting
the {Guest}_diver runvar causes ts-debian-di-install to use the d-i
images in the location configured via TftpDiVersion, so they are
Version Controlled along with the d-i version used for the host.
This adds three new jobs:
test-amd64-amd64-amd64-pvgrub:
all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
arch amd64
buildjob build-amd64
debian_arch amd64
debian_bootloader pvgrub
debian_dist wheezy
debian_method netboot
kernbuildjob build-amd64-pvops
kernkind pvops
toolstack xl
xenbuildjob build-amd64
test-amd64-amd64-i386-pvgrub:
all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
arch amd64
buildjob build-amd64
debian_arch i386
debian_bootloader pvgrub
debian_dist wheezy
debian_method netboot
kernbuildjob build-amd64-pvops
kernkind pvops
toolstack xl
xenbuildjob build-amd64
test-amd64-amd64-pygrub:
all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
arch amd64
buildjob build-amd64
debian_arch amd64
debian_bootloader pygrub
debian_dist wheezy
debian_method netboot
kernbuildjob build-amd64-pvops
kernkind pvops
toolstack xl
xenbuildjob build-amd64
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v3: added runvar details
---
make-flight | 39 +++++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)
diff --git a/make-flight b/make-flight
index ec0a20c..ca48b81 100755
--- a/make-flight
+++ b/make-flight
@@ -303,6 +303,42 @@ do_passthrough_tests () {
done
}
+do_pygrub_tests () {
+ if [ $xenarch != amd64 -o $dom0arch != amd64 -o "$kern" != "" ]; then
+ return
+ fi
+
+ job_create_test test-$xenarch$kern-$dom0arch-pygrub \
+ test-debian-di xl $xenarch $dom0arch \
+ debian_arch=amd64 \
+ debian_dist=$guestsuite \
+ debian_method=netboot \
+ debian_bootloader=pygrub \
+ all_hostflags=$most_hostflags
+}
+
+do_pvgrub_tests () {
+ if [ $xenarch != amd64 -o $dom0arch != amd64 -o "$kern" != "" ]; then
+ return
+ fi
+
+ job_create_test test-$xenarch$kern-$dom0arch-amd64-pvgrub \
+ test-debian-di xl $xenarch $dom0arch \
+ debian_arch=amd64 \
+ debian_dist=$guestsuite \
+ debian_method=netboot \
+ debian_bootloader=pvgrub \
+ all_hostflags=$most_hostflags \
+
+ job_create_test test-$xenarch$kern-$dom0arch-i386-pvgrub \
+ test-debian-di xl $xenarch $dom0arch \
+ debian_arch=i386 \
+ debian_dist=$guestsuite \
+ debian_method=netboot \
+ debian_bootloader=pvgrub \
+ all_hostflags=$most_hostflags
+}
+
do_pv_debian_tests () {
xsms=$(xenbranch_xsm_variants)
@@ -427,6 +463,9 @@ test_matrix_do_one () {
fi
#do_passthrough_tests
+
+ do_pygrub_tests
+ do_pvgrub_tests
}
test_matrix_iterate
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 14/25] distros: add branch infrastructure
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (12 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 13/25] Test pygrub and pvgrub on the regular flights Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 15/25] distros: Run a flight over the weekend Ian Campbell
` (11 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Since the distro nightlies are not version controlled we cannot use
the usual mechanisms for detecting regressions. Special case things
appropriately. We use an OLD_REVISION of "flight-NNN" to signify that
the old revision is another flight and not a tree revision.
A grep over $NEW_REVISION needed adjusting since NEW_REVISION is empty
in this mode, leading to "grep <filename>" which hangs waiting for
stdin.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v3:
Handle within cr-daily-branch, since ap-fetch-version* don't make sense for
a branch such as this.
---
cr-daily-branch | 36 ++++++++++++++++++++++++++++--------
cri-common | 1 +
2 files changed, 29 insertions(+), 8 deletions(-)
diff --git a/cr-daily-branch b/cr-daily-branch
index c7d1071..55afadf 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -68,23 +68,34 @@ fetch_version () {
printf '%s\n' "$fetch_version_result"
}
-treeurl=`./ap-print-url $branch`
+case $branch in
+ distros)
+ treeurl=none;;
+ *)
+ treeurl=`./ap-print-url $branch`;;
+esac
force_baseline=false
skipidentical=true
wantpush=$OSSTEST_PUSH
-if [ "x$OLD_REVISION" = x ]; then
- OLD_REVISION="`./ap-fetch-version-old $branch`"
- export OLD_REVISION
-fi
-
check_tested () {
./sg-check-tested --debug --branch=$branch \
--blessings=${DAILY_BRANCH_TESTED_BLESSING:-$OSSTEST_BLESSING} \
"$@"
}
+if [ "x$OLD_REVISION" = x ]; then
+ case $branch in
+ distros)
+ OSSTEST_NO_BASELINE=y
+ OLD_REVISION=flight-`check_tested`
+ ;;
+ *) OLD_REVISION="`./ap-fetch-version-old $branch`";;
+ esac
+ export OLD_REVISION
+fi
+
if [ "x$OSSTEST_NO_BASELINE" != xy ] ; then
testedflight=`check_tested --revision-$tree="$OLD_REVISION"`
@@ -227,6 +238,11 @@ if [ "x$OLD_REVISION" = xdetermine-late ]; then
OLD_REVISION="`./ap-fetch-version-baseline-late $branch $NEW_REVISION`"
fi
+case $branch in
+distros) makeflight=./make-distros-flight ;;
+*) makeflight=./make-flight ;;
+esac
+
if [ "x$NEW_REVISION" = "x$OLD_REVISION" ]; then
wantpush=false
for checkbranch in x $BRANCHES_ALWAYS; do
@@ -241,7 +257,7 @@ if [ "x$NEW_REVISION" = "x$OLD_REVISION" ]; then
fi
$DAILY_BRANCH_PREMAKE_HOOK
-flight=`./make-flight $branch $xenbranch $OSSTEST_BLESSING "$@"`
+flight=`$makeflight $branch $xenbranch $OSSTEST_BLESSING "$@"`
$DAILY_BRANCH_POSTMAKE_HOOK
heading=tmp/$flight.heading-info
@@ -261,6 +277,10 @@ fi
revlog=tmp/$flight.revision-log
case "$NEW_REVISION/$OLD_REVISION" in
+/flight-[0-9]*)
+ echo >&2 "SGR COMPARISON AGAINST ${OLD_REVISION}"
+ sgr_args+=" --that-flight=${OLD_REVISION#flight-}"
+ ;;
*/*[^0-9a-f]* | *[^0-9a-f]*/*)
echo >&2 "NO SGR COMPARISON badchar $NEW_REVISION/$OLD_REVISION"
;;
@@ -321,7 +341,7 @@ start_email $flight $branch "$sgr_args" "$subject_prefix"
push=false
if grep '^tolerable$' $mrof >/dev/null 2>&1; then push=$wantpush; fi
if test -f $branch.force; then push=$OSSTEST_PUSH; fi
-if grep -xF $NEW_REVISION $branch.force-rev; then push=$OSSTEST_PUSH; fi
+if test -n "$NEW_REVISION" && grep -xF $NEW_REVISION $branch.force-rev; then push=$OSSTEST_PUSH; fi
if test -f $branch.block; then push=false; fi
if test -e $mrof && test -e $tree_bisect && ! grep '^broken' $mrof; then
diff --git a/cri-common b/cri-common
index ad44546..4bf7548 100644
--- a/cri-common
+++ b/cri-common
@@ -72,6 +72,7 @@ select_xenbranch () {
rumpuserxen) tree=rumpuserxen; xenbranch=xen-unstable ;;
seabios) tree=seabios; xenbranch=xen-unstable ;;
ovmf) tree=ovmf; xenbranch=xen-unstable ;;
+ distros) tree=none; xenbranch=xen-unstable ;;
osstest) tree=osstest; xenbranch=xen-unstable ;;
esac
if [ "x$tree" = xlinux ]; then
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 15/25] distros: Run a flight over the weekend.
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (13 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 14/25] distros: add branch infrastructure Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 16/25] Debian: Handle lack of bootloader support in d-i on ARM Ian Campbell
` (10 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Once a week should be sufficient for this test. It involves quite a
large number of jobs so schedule it to start early on Saturday so it
can run over the weekend when, in theory, things should be quieter.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
crontab | 1 +
1 file changed, 1 insertion(+)
diff --git a/crontab b/crontab
index 5dc8675..1917ec4 100644
--- a/crontab
+++ b/crontab
@@ -6,5 +6,6 @@ MAILTO=ian.jackson@citrix.com,ian.campbell@eu.citrix.com
18 9 * * 1,3,5 cd testing.git && BRANCHES=linux-next ./cr-for-branches branches -w "./cr-daily-branch --real"
18 4 * * * cd testing.git && BRANCHES='linux-linus linux-mingo-tip-master linux-3.0 libvirt rumpuserxen' ./cr-for-branches branches -w "./cr-daily-branch --real"
6-59/15 * * * * cd testing.git && EXTRA_BRANCHES='linux-linus linux-3.0 rumpuserxen libvirt' ./cr-for-branches bisects -w "./cr-try-bisect --real"
+46 7 * * 6 cd testing.git && BRANCHES=distros ./cr-for-branches branches -w "./cr-daily-branch --real"
#8-59/5 * * * * cd bisects/adhoc.git && with-lock-ex -q data-tree-lock bash -c "./cr-try-bisect-adhoc; exit $?"
3 4 * * * savelog -c28 testing.git/tmp/cr-for-branches.log >/dev/null
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 16/25] Debian: Handle lack of bootloader support in d-i on ARM.
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (14 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 15/25] distros: Run a flight over the weekend Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 17/25] standalone: propagate result of command from with_logging Ian Campbell
` (9 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Debian doesn't currently know what bootloader to install in a Xen
guest on ARM. We install pv-grub-menu above which actually does what
we need, but the installer doesn't treat that as a "bootloader".
Most ARM platforms end up installing a u-boot boot.scr, based on a
platform whitelist. This doesn't seem appropriate for us. Grub is not
available for arm32. For arm64 we will eventually end up with in-guest
UEFI and therefore grub-efi and things will work normally. I'm not
sure what the answer is going to be for arm32.
This patch enables the workaround for Wheezy, Jessie and Sid,
post-Jessie should be enabled as we add them. (Pre-wheezy does not
support running as a Xen guest on ARM so we don't test them at all).
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v4: Handle sid too
v3: New
---
Osstest/Debian.pm | 14 ++++++++++++--
ts-debian-di-install | 6 ++++--
2 files changed, 16 insertions(+), 4 deletions(-)
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 831290c..7f9deaa 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -711,8 +711,8 @@ END
return $preseed;
}
-sub preseed_create_guest ($$;@) {
- my ($ho, $sfx, %xopts) = @_;
+sub preseed_create_guest ($$$;@) {
+ my ($ho, $arch, $sfx, %xopts) = @_;
my $suite= $xopts{Suite} || $c{DebianSuite};
@@ -759,6 +759,16 @@ d-i grub-installer/bootdev string /dev/xvda
END
+ # Debian doesn't currently know what bootloader to install in a
+ # Xen guest on ARM. We install pv-grub-menu above which actually
+ # does what we need, but the installer doesn't treat that as a
+ # "bootloader".
+ logm("\$arch is $arch, \$suite is $suite");
+ $preseed_file.= (<<END) if $arch =~ /^arm/ && $suite =~ /wheezy|jessie|sid/;
+d-i nobootloader/confirmation_common boolean true
+
+END
+
preseed_ssh($ho, $sfx);
preseed_hook_overlay($ho, $sfx, $c{OverlayLocal}, 'overlay-local.tar');
diff --git a/ts-debian-di-install b/ts-debian-di-install
index 62e9bb5..56b6ad8 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -168,7 +168,9 @@ END
$suite = "sid" if $suite eq "daily";
- $ps_url = preseed_create_guest($gho, '', Suite=>$suite, PvMenuLst=>($bl eq "pvgrub"));
+ $ps_url = preseed_create_guest($gho, $arch, '',
+ Suite=>$suite,
+ PvMenuLst=>($bl eq "pvgrub"));
$extra_disk = "";
}
@@ -180,7 +182,7 @@ END
($method_cfg,$extra_disk) = setup_netinst($tmpdir, $arch, $cd);
- $ps_url = preseed_create_guest($gho, '', CDROM=>1);
+ $ps_url = preseed_create_guest($gho, $arch, '', CDROM=>1);
}
else
{
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 17/25] standalone: propagate result of command from with_logging
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (15 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 16/25] Debian: Handle lack of bootloader support in d-i on ARM Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 18/25] ts-debian-di-install: Refactor root_disk specification Ian Campbell
` (8 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Such that the overall script gets the return code of the test/job
which was run.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
standalone | 1 +
1 file changed, 1 insertion(+)
diff --git a/standalone b/standalone
index caf3fd5..3189e84 100755
--- a/standalone
+++ b/standalone
@@ -156,6 +156,7 @@ with_logging() {
if [ $rc -ne 0 ] ; then
echo "FAILED rc=${rc}" >&2
fi
+ return $rc
}
# other potential ops:
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 18/25] ts-debian-di-install: Refactor root_disk specification
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (16 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 17/25] standalone: propagate result of command from with_logging Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 19/25] make-flight: refactor PV debian tests Ian Campbell
` (7 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v4: new patch
---
ts-debian-di-install | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/ts-debian-di-install b/ts-debian-di-install
index 56b6ad8..281e7cb 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -205,12 +205,14 @@ END
OnPowerOff => "preserve"
);
+ my $root_disk = "'phy:$gho->{Lvdev},xvda,w'";
+
prepareguest_part_xencfg($ho, $gho, $ram_mb, \%install_xopts, <<END);
$method_cfg
extra = "$cmdline"
#
disk = [
- $extra_disk 'phy:$gho->{Lvdev},xvda,w'
+ $extra_disk $root_disk
]
END
@@ -234,7 +236,7 @@ END
$blcfg
#
disk = [
- 'phy:$gho->{Lvdev},xvda,w'
+ $root_disk
]
END
return;
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 19/25] make-flight: refactor PV debian tests
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (17 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 18/25] ts-debian-di-install: Refactor root_disk specification Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 20/25] Add testing of non-LVM/phy disk backends Ian Campbell
` (6 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
No functional change, standalong-generate-dump-flight-runvars confirms
no change to the runvars.
Includes a hook which is not used yet, $recipe_sfx.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v4: new patch
---
make-flight | 24 ++++++++++++++----------
1 file changed, 14 insertions(+), 10 deletions(-)
diff --git a/make-flight b/make-flight
index ca48b81..909cebe 100755
--- a/make-flight
+++ b/make-flight
@@ -339,6 +339,17 @@ do_pvgrub_tests () {
all_hostflags=$most_hostflags
}
+do_pv_debian_test_one () {
+ testname=$1; shift
+ recipe_sfx=$1; shift
+ toolstack=$1; shift
+
+ job_create_test test-$xenarch$kern-$dom0arch-$testname \
+ test-debian$recipe_sfx $toolstack \
+ $xenarch $dom0arch \
+ $debian_runvars all_hostflags=$most_hostflags $@
+}
+
do_pv_debian_tests () {
xsms=$(xenbranch_xsm_variants)
@@ -354,20 +365,13 @@ do_pv_debian_tests () {
suffix=${platform:+-$platform}
hostflags=${most_hostflags}${platform:+,platform-$platform}
- job_create_test test-$xenarch$kern-$dom0arch-xl$suffix \
- test-debian xl \
- $xenarch $dom0arch \
- enable_xsm=$xsm \
- $debian_runvars all_hostflags=$hostflags
+ do_pv_debian_test_one xl$suffix '' xl enable_xsm=$xsm
+
done
done
for xsm in $xsms ; do
- job_create_test test-$xenarch$kern-$dom0arch-libvirt \
- test-debian libvirt \
- $xenarch $dom0arch \
- enable_xsm=$xsm \
- $debian_runvars all_hostflags=$most_hostflags
+ do_pv_debian_test_one libvirt '' libvirt enable_xsm=$xsm
done
}
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 20/25] Add testing of non-LVM/phy disk backends.
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (18 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 19/25] make-flight: refactor PV debian tests Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 21/25] mfi-common: Allow make-*flight to filter the set of build jobs to include Ian Campbell
` (5 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
xen-create-image makes this tricky to do since it is rather LVM
centric. Now that we have the ability to install from d-i it's
possible to arrange fairly easily that they use non-LVM disk backend
arrangements.
Here we add support to the test script and infra and create a bunch of
new jobs testing the cross product of {xl,libvirt} x {raw,qcow2,vhd}.
The test scripts are modified such that when constructing a domain
with a non-LVM diskfmt runvar:
- the LVM device is slightly enlarged to account for file format
headers (1M should be plenty).
- the LVM device will have an ext3 filesystem created on it instead
of being used as a phy device for the guest. Reusing the LVM volume
in this way means we don't need to do more storage management in
dom0 (i.e. arranging for / to be large enough, or managing a
special "images" LV)
- the relevant type of container is created within the filesystem
using the appropriate tool.
- New properties Disk{fmt,spec} are added to all $gho, containing
the format used for the root disk and the xl diskspec to load it.
- lvm backed guests use a xend/xm compatible spec, everything
else uses the improved xl syntax which libvirt also supports.
We won't test non-LVM on xend.
- New properties Disk{mnt,img} are added to $gho which are not using
LVM. These contain the mount point to use (configurable via
OSSTEST_CONFIG and runvars) and the full path (including mount
point) to the image itself.
- When starting or stopping a guest we arrange for the filesystem to
be (u)mounted.
- The prepearation when starting a guest copes gracefully with
the disk already being prepared.
- Hooks are called from guest_create() and guest_destroy() to
manipulate the disk as needed.
Using standalong-generate-dump-flight-runvars a representative set of
runvars is:
xen-unstable test-amd64-amd64-xl-qcow2 all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
xen-unstable test-amd64-amd64-xl-qcow2 arch amd64
xen-unstable test-amd64-amd64-xl-qcow2 buildjob build-amd64
xen-unstable test-amd64-amd64-xl-qcow2 debian_arch amd64
xen-unstable test-amd64-amd64-xl-qcow2 debian_bootloader pygrub
xen-unstable test-amd64-amd64-xl-qcow2 debian_diskfmt qcow2
xen-unstable test-amd64-amd64-xl-qcow2 debian_dist wheezy
xen-unstable test-amd64-amd64-xl-qcow2 debian_method netboot
xen-unstable test-amd64-amd64-xl-qcow2 kernbuildjob build-amd64-pvops
xen-unstable test-amd64-amd64-xl-qcow2 kernkind pvops
xen-unstable test-amd64-amd64-xl-qcow2 toolstack xl
xen-unstable test-amd64-amd64-xl-qcow2 xenbuildjob build-amd64
Compared to test-amd64-amd64-pygrub (which is the most similar job) and
normalising the test name the difference is:
xen-unstable test-amd64-amd64-SUFFIX all_hostflags arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test
xen-unstable test-amd64-amd64-SUFFIX arch amd64
xen-unstable test-amd64-amd64-SUFFIX buildjob build-amd64
xen-unstable test-amd64-amd64-SUFFIX debian_arch amd64
xen-unstable test-amd64-amd64-SUFFIX debian_bootloader pygrub
+xen-unstable test-amd64-amd64-SUFFIX debian_diskfmt qcow2
xen-unstable test-amd64-amd64-SUFFIX debian_dist wheezy
xen-unstable test-amd64-amd64-SUFFIX debian_method netboot
xen-unstable test-amd64-amd64-SUFFIX kernbuildjob build-amd64-pvops
xen-unstable test-amd64-amd64-SUFFIX kernkind pvops
xen-unstable test-amd64-amd64-SUFFIX toolstack xl
xen-unstable test-amd64-amd64-SUFFIX xenbuildjob build-amd64
These are added to the following flights:
+libvirt
+linux-3.0
+linux-3.10
+linux-3.14
+linux-3.4
+linux-arm-xen
+linux-linus
+linux-mingo-tip-master
+linux-next
+osstest
+qemu-mainline
+qemu-upstream-4.3-testing
+qemu-upstream-4.4-testing
+qemu-upstream-unstable
+seabios
+xen-4.0-testing
+xen-4.1-testing
+xen-4.2-testing
+xen-4.3-testing
+xen-4.4-testing
+xen-unstable
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
v4: new patch
---
Osstest/TestSupport.pm | 95 ++++++++++++++++++++++++++++++++++++++++++++++++--
make-flight | 16 +++++++++
ts-debian-di-install | 10 ++----
ts-guest-start | 1 -
4 files changed, 112 insertions(+), 10 deletions(-)
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 3cd8d22..61fd7bc 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -89,7 +89,8 @@ BEGIN {
target_var target_var_prefix
selectguest prepareguest more_prepareguest_hvm
guest_var guest_var_commalist
- prepareguest_part_lvmdisk prepareguest_part_xencfg
+ prepareguest_part_lvmdisk prepareguest_part_diskimg
+ prepareguest_part_xencfg
guest_umount_lv guest_await guest_await_dhcp_tcp
guest_checkrunning guest_check_ip guest_find_ether
guest_find_domid guest_check_up guest_check_up_quick
@@ -97,6 +98,7 @@ BEGIN {
guest_await_shutdown guest_await_destroy guest_destroy
guest_vncsnapshot_begin guest_vncsnapshot_stash
guest_check_remus_ok guest_editconfig
+ guest_prepare_disk guest_unprepare_disk
host_involves_pcipassthrough host_get_pcipassthrough_devs
toolstack guest_create
@@ -1319,6 +1321,7 @@ sub selectguest ($$) {
}
logm("guest: using $gn on $gho->{Host}{Name}");
guest_find_lv($gho);
+ guest_find_diskimg($gho);
guest_find_ether($gho);
guest_find_tcpcheckport($gho);
dhcp_watch_setup($ho,$gho);
@@ -1334,6 +1337,22 @@ sub guest_find_lv ($) {
? '/dev/'.$gho->{Vg}.'/'.$gho->{Lv} : undef;
}
+sub guest_find_diskimg($)
+{
+ my ($gho) = @_;
+ $gho->{Diskfmt} = $r{"$gho->{Guest}_diskfmt"} // "lvm";
+ $gho->{Diskspec} = "phy:$gho->{Lvdev},xvda,w";
+
+ return if $gho->{Diskfmt} eq "lvm";
+
+ my $mntroot = get_host_property($gho->{Host}, "DiskImageMount",
+ $c{DiskImageMount} // "/var/lib/xen/images");
+
+ $gho->{Diskmnt} = "$mntroot/$gho->{Guest}";
+ $gho->{Diskimg} = "$gho->{Diskmnt}/disk.$gho->{Diskfmt}";
+ $gho->{Diskspec} = "format=$gho->{Diskfmt},vdev=xvda,target=$gho->{Diskimg}";
+}
+
sub guest_find_ether ($) {
my ($gho) = @_;
$gho->{Ether}= $r{"$gho->{Guest}_ether"};
@@ -1373,6 +1392,7 @@ sub guest_destroy ($) {
my ($gho) = @_;
my $ho = $gho->{Host};
toolstack($ho)->destroy($gho);
+ guest_unprepare_disk($gho);
}
sub guest_await_destroy ($$) {
@@ -1384,9 +1404,32 @@ sub guest_await_destroy ($$) {
sub guest_create ($) {
my ($gho) = @_;
my $ho = $gho->{Host};
+ guest_prepare_disk($gho);
toolstack($ho)->create($gho);
}
+sub guest_prepare_disk ($) {
+ my ($gho) = @_;
+
+ guest_umount_lv($gho->{Host}, $gho);
+
+ return if $gho->{Diskfmt} eq "lvm";
+
+ target_cmd_root($gho->{Host}, <<END);
+mkdir -p $gho->{Diskmnt}
+mount $gho->{Lvdev} $gho->{Diskmnt};
+END
+}
+
+sub guest_unprepare_disk ($) {
+ my ($gho) = @_;
+ return if $gho->{Diskfmt} eq "lvm";
+ target_cmd_root($gho->{Host}, <<END);
+umount $gho->{Lvdev} || :
+END
+}
+
+
sub target_choose_vg ($$) {
my ($ho, $mbneeded) = @_;
@@ -1518,6 +1561,7 @@ sub prepareguest ($$$$$$) {
}
guest_find_lv($gho);
+ guest_find_diskimg($gho);
guest_find_ether($gho);
guest_find_tcpcheckport($gho);
return $gho;
@@ -1528,7 +1572,54 @@ sub prepareguest_part_lvmdisk ($$$) {
target_cmd_root($ho, "lvremove -f $gho->{Lvdev} ||:");
target_cmd_root($ho, "lvcreate -L ${disk_mb}M -n $gho->{Lv} $gho->{Vg}");
target_cmd_root($ho, "dd if=/dev/zero of=$gho->{Lvdev} count=10");
-}
+}
+
+sub make_vhd ($$$) {
+ my ($ho, $gho, $disk_mb) = @_;
+ target_cmd_root($ho, "vhd-util create -n $gho->{Rootimg} -s $disk_mb");
+}
+sub make_qcow2 ($$$) {
+ my ($ho, $gho, $disk_mb) = @_;
+ # upstream qemu's version. Seems preferable to qemu-xen-img from qemu-trad.
+ my $qemu_img = "/usr/local/lib/xen/bin/qemu-img";
+ target_cmd_root($ho, "$qemu_img create -f qcow2 $gho->{Rootimg} ${disk_mb}M");
+}
+sub make_raw ($$$) {
+ my ($ho, $gho, $disk_mb) = @_;
+ target_cmd_root($ho, "dd if=/dev/zero of=$gho->{Rootimg} bs=1M count=${disk_mb}");
+}
+
+sub prepareguest_part_diskimg ($$$) {
+ my ($ho, $gho, $disk_mb) = @_;
+
+ my $diskfmt = $gho->{Diskfmt};
+ # Allow an extra megabyte for image format headers
+ my $disk_overhead = $diskfmt eq "lvm" ? 0 : 1;
+
+ logm("preparing guest disks in $diskfmt format");
+
+ target_cmd_root($ho, "umount $gho->{Lvdev} ||:");
+
+ prepareguest_part_lvmdisk($ho, $gho, $disk_mb + $disk_overhead);
+
+ if ($diskfmt ne "lvm") {
+
+ $gho->{Rootimg} = "$gho->{Diskmnt}/disk.$diskfmt";
+ $gho->{Rootcfg} = "format=$diskfmt,vdev=xvda,target=$gho->{Rootimg}";
+
+ target_cmd_root($ho, <<END);
+mkfs.ext3 $gho->{Lvdev}
+mkdir -p $gho->{Diskmnt}
+mount $gho->{Lvdev} $gho->{Diskmnt}
+END
+ no strict qw(refs);
+ &{"make_$diskfmt"}($ho, $gho, $disk_mb);
+
+ target_cmd_root($ho, <<END);
+umount $gho->{Lvdev}
+END
+ }
+}
sub prepareguest_part_xencfg ($$$$$) {
my ($ho, $gho, $ram_mb, $xopts, $cfgrest) = @_;
diff --git a/make-flight b/make-flight
index 909cebe..c923f21 100755
--- a/make-flight
+++ b/make-flight
@@ -373,6 +373,22 @@ do_pv_debian_tests () {
for xsm in $xsms ; do
do_pv_debian_test_one libvirt '' libvirt enable_xsm=$xsm
done
+
+ for ts in xl libvirt ; do
+
+ for fmt in raw vhd qcow2 ; do
+
+ fmt_runvar="debian_diskfmt=$fmt"
+
+ do_pv_debian_test_one $ts-$fmt '-di' $ts \
+ debian_arch=amd64 \
+ debian_dist=$guestsuite \
+ debian_method=netboot \
+ debian_bootloader=pygrub \
+ $fmt_runvar
+
+ done
+ done
}
test_matrix_do_one () {
diff --git a/ts-debian-di-install b/ts-debian-di-install
index 281e7cb..839cc27 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -41,9 +41,7 @@ sub prep () {
$gho= prepareguest($ho, $gn, $guesthost, 22,
$disk_mb, 40);
- prepareguest_part_lvmdisk($ho, $gho, $disk_mb);
-
- target_cmd_root($ho, "umount $gho->{Lvdev} ||:");
+ prepareguest_part_diskimg($ho, $gho, $disk_mb);
}
sub setup_netinst($$$)
@@ -205,14 +203,12 @@ END
OnPowerOff => "preserve"
);
- my $root_disk = "'phy:$gho->{Lvdev},xvda,w'";
-
prepareguest_part_xencfg($ho, $gho, $ram_mb, \%install_xopts, <<END);
$method_cfg
extra = "$cmdline"
#
disk = [
- $extra_disk $root_disk
+ $extra_disk '$gho->{Diskspec}'
]
END
@@ -236,7 +232,7 @@ END
$blcfg
#
disk = [
- $root_disk
+ '$gho->{Diskspec}'
]
END
return;
diff --git a/ts-guest-start b/ts-guest-start
index 1aa9e69..a434720 100755
--- a/ts-guest-start
+++ b/ts-guest-start
@@ -25,7 +25,6 @@ tsreadconfig();
our ($ho,$gho) = ts_get_host_guest(@ARGV);
sub start () {
- guest_umount_lv($ho, $gho);
guest_create($gho);
}
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 21/25] mfi-common: Allow make-*flight to filter the set of build jobs to include
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (19 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 20/25] Add testing of non-LVM/phy disk backends Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 22/25] make-distros-flight: don't bother building for XSM Ian Campbell
` (4 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
By using the same job_create_build(_filter_callback) scheme used for
the test jobs.
Will be used in make-distros-flight.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
make-distros-flight | 4 ++++
make-flight | 4 ++++
mfi-common | 21 +++++++++++++++------
3 files changed, 23 insertions(+), 6 deletions(-)
diff --git a/make-distros-flight b/make-distros-flight
index e3ec8f7..2724dd2 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -33,6 +33,10 @@ flight=`./cs-flight-create $blessing $branch`
defsuite=`getconfig DebianSuite`
defguestsuite=`getconfig GuestDebianSuite`
+job_create_build_filter_callback () {
+ :
+}
+
if [ x$buildflight = x ]; then
WANT_XEND=false REVISION_LINUX_OLD=disable
diff --git a/make-flight b/make-flight
index c923f21..ef1e422 100755
--- a/make-flight
+++ b/make-flight
@@ -34,6 +34,10 @@ flight=`./cs-flight-create $blessing $branch`
defsuite=`getconfig DebianSuite`
defguestsuite=`getconfig GuestDebianSuite`
+job_create_build_filter_callback () {
+ :
+}
+
if [ x$buildflight = x ]; then
create_build_jobs
diff --git a/mfi-common b/mfi-common
index a9e966f..a100afb 100644
--- a/mfi-common
+++ b/mfi-common
@@ -54,6 +54,15 @@ xenbranch_xsm_variants () {
esac
}
+job_create_build () {
+ job_create_build_filter_callback "$@" || return 0
+
+ local job=$1; shift
+ local recipe=$1; shift
+
+ ./cs-job-create $flight $job $recipe "$@"
+}
+
create_build_jobs () {
local arch
@@ -164,7 +173,7 @@ create_build_jobs () {
else
xsm_suffix=""
fi
- ./cs-job-create $flight build-$arch$xsm_suffix build \
+ job_create_build build-$arch$xsm_suffix build \
arch=$arch enable_xend=$build_defxend enable_ovmf=$enable_ovmf\
enable_xsm=$enable_xsm \
tree_qemu=$TREE_QEMU \
@@ -183,7 +192,7 @@ create_build_jobs () {
done
if [ $build_extraxend = "true" ] ; then
- ./cs-job-create $flight build-$arch-xend build \
+ job_create_build build-$arch-xend build \
arch=$arch enable_xend=true enable_ovmf=$enable_ovmf \
tree_qemu=$TREE_QEMU \
tree_qemuu=$TREE_QEMU_UPSTREAM \
@@ -196,7 +205,7 @@ create_build_jobs () {
revision_qemuu=$REVISION_QEMU_UPSTREAM
fi
- ./cs-job-create $flight build-$arch-pvops build-kern \
+ job_create_build build-$arch-pvops build-kern \
arch=$arch kconfighow=xen-enable-xen-config \
$RUNVARS $BUILD_RUNVARS $BUILD_LINUX_RUNVARS $arch_runvars \
$suite_runvars \
@@ -208,7 +217,7 @@ create_build_jobs () {
if [ "x$REVISION_LIBVIRT" != xdisable ]; then
- ./cs-job-create $flight build-$arch-libvirt build-libvirt \
+ job_create_build build-$arch-libvirt build-libvirt \
arch=$arch \
tree_xen=$TREE_XEN \
$RUNVARS $BUILD_RUNVARS $BUILD_LIBVIRT_RUNVARS $arch_runvars \
@@ -223,7 +232,7 @@ create_build_jobs () {
case $arch in
i386|amd64)
- ./cs-job-create $flight build-$arch-rumpuserxen build-rumpuserxen \
+ job_create_build build-$arch-rumpuserxen build-rumpuserxen \
arch=$arch \
tree_xen=$TREE_XEN \
$RUNVARS $BUILD_RUNVARS $BUILD_RUMPUSERXEN_RUNVARS $arch_runvars \
@@ -252,7 +261,7 @@ create_build_jobs () {
if [ "x$REVISION_LINUX_OLD" != xdisable ]; then
- ./cs-job-create $flight build-$arch-oldkern build-kern \
+ job_create_build build-$arch-oldkern build-kern \
arch=$arch kconfighow=create-config-sh \
kimagefile=vmlinux \
$RUNVARS $BUILD_RUNVARS $BUILD_LINUX_OLD_RUNVARS \
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 22/25] make-distros-flight: don't bother building for XSM.
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (20 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 21/25] mfi-common: Allow make-*flight to filter the set of build jobs to include Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 23/25] Debian.pm: Assume 100MB/s dd from /dev/zero when creating a raw disk image Ian Campbell
` (3 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
make-distros-flight | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/make-distros-flight b/make-distros-flight
index 2724dd2..62c527b 100755
--- a/make-distros-flight
+++ b/make-distros-flight
@@ -34,7 +34,10 @@ defsuite=`getconfig DebianSuite`
defguestsuite=`getconfig GuestDebianSuite`
job_create_build_filter_callback () {
- :
+ case " $* " in
+ *" enable_xsm=true "*) return 1;;
+ esac
+ return 0
}
if [ x$buildflight = x ]; then
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 23/25] Debian.pm: Assume 100MB/s dd from /dev/zero when creating a raw disk image
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (21 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 22/25] make-distros-flight: don't bother building for XSM Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 24/25] distros: email only me on play flights Ian Campbell
` (2 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
In my local test:
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB) copied, 80.8332 s, 130 MB/s
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
Osstest/TestSupport.pm | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 61fd7bc..391c9b2 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1586,7 +1586,9 @@ sub make_qcow2 ($$$) {
}
sub make_raw ($$$) {
my ($ho, $gho, $disk_mb) = @_;
- target_cmd_root($ho, "dd if=/dev/zero of=$gho->{Rootimg} bs=1M count=${disk_mb}");
+ # In local tests this reported 130MB/s, so calculate a timeout assuming 100MB/s.
+ target_cmd_root($ho, "dd if=/dev/zero of=$gho->{Rootimg} bs=1M count=${disk_mb}",
+ ${disk_mb} / 100);
}
sub prepareguest_part_diskimg ($$$) {
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 24/25] distros: email only me on play flights
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (22 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 23/25] Debian.pm: Assume 100MB/s dd from /dev/zero when creating a raw disk image Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 9:16 ` [PATCH OSSTEST v4 25/25] ts-debian-di-install: Use ftp.debian.org directly Ian Campbell
2015-04-02 11:21 ` [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Jackson
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
daily-cron-email-play--distros | 1 +
1 file changed, 1 insertion(+)
create mode 100644 daily-cron-email-play--distros
diff --git a/daily-cron-email-play--distros b/daily-cron-email-play--distros
new file mode 100644
index 0000000..0d82e65
--- /dev/null
+++ b/daily-cron-email-play--distros
@@ -0,0 +1 @@
+To: ian.campbell@citrix.com
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [PATCH OSSTEST v4 25/25] ts-debian-di-install: Use ftp.debian.org directly
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (23 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 24/25] distros: email only me on play flights Ian Campbell
@ 2015-04-02 9:16 ` Ian Campbell
2015-04-02 11:21 ` [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Jackson
25 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 9:16 UTC (permalink / raw)
To: ian.jackson; +Cc: Ian Campbell, xen-devel
The local proxy seems to serve stale packages for Jessie etc, I blame
the intercepting cache on the way out of our network, similar to
b5f15136900d "mg-debian-installer-update: workaround caching proxies",
except it is between the apt-cache and the world not the osstest vm
and the world.
Since the netboot kernel+initrd are reasonably small, these flights
are infrequent and they are intended to test the current upstream
version I think this is tollerable.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
ts-debian-di-install | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/ts-debian-di-install b/ts-debian-di-install
index 839cc27..469c1b6 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -119,7 +119,9 @@ sub setup_netboot($$$)
target_putfile_root($ho, 60, $initrd, "$didir/initrd_${suite}_${arch}");
} else {
- my $mirror = "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath}";
+ #my $mirror = "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath}";
+ # XXX local mirror seems to serve up stale files.
+ my $mirror = "http://ftp.debian.org/debian";
my $di_url = $suite eq "daily" ?
"http://d-i.debian.org/daily-images/$arch/daily/netboot" :
--
2.1.4
^ permalink raw reply related [flat|nested] 40+ messages in thread
* Re: [PATCH OSSTEST v4 00/25] add distro domU testing flight
2015-04-02 9:16 [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Campbell
` (24 preceding siblings ...)
2015-04-02 9:16 ` [PATCH OSSTEST v4 25/25] ts-debian-di-install: Use ftp.debian.org directly Ian Campbell
@ 2015-04-02 11:21 ` Ian Jackson
2015-04-02 12:24 ` Ian Campbell
25 siblings, 1 reply; 40+ messages in thread
From: Ian Jackson @ 2015-04-02 11:21 UTC (permalink / raw)
To: Ian Campbell; +Cc: longtaox.pang, xen-devel
Ian Campbell writes ("[Xen-devel] [PATCH OSSTEST v4 00/25] add distro domU testing flight"):
> The failures mainly seem to be some sort of proxy issue returning old
> data (similar to what used to happen to mg-debian-installer-update),
Maybe we can set up an explicit proxy which actually works. I guess
that might need negotiation with Citrix corporate IT.
Luckily the new colo is not affected by anything like this.
> On a small number I see "The installer failed to download a file from
> the mirror." messages, which might be the cache/proxy or might be an
> actual Debian issue.
Hrm. I find these vague error messages exceptionally frustrating
(debootstrap has this problem too).
> The -raw tests all failed due to a timeout doing:
> dd if=/dev/zero of=/var/lib/xen/images/debian/disk.raw bs=1M count=10000
>
> I'm unclear if it timed out after 30 (as the code seems to say) or 60
> (as the logs seem to say) but either way something obviously needs
> adjusting. I just tried it on my local test box and it took 1m20s and
> reported 130 MB/s. I've appended an as yet untested patch which assumed
> 100 MB/s and calculates a timeout from that.
Do you intend to fold that patch in ?
> I could switch to using "count=1 seek=10000" to create a sparse file,
> which should be pretty much instantaneous, but allocating the whole
> backing file seemed like a good idea.
Right.
Thanks,
Ian.
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [PATCH OSSTEST v4 00/25] add distro domU testing flight
2015-04-02 11:21 ` [PATCH OSSTEST v4 00/25] add distro domU testing flight Ian Jackson
@ 2015-04-02 12:24 ` Ian Campbell
0 siblings, 0 replies; 40+ messages in thread
From: Ian Campbell @ 2015-04-02 12:24 UTC (permalink / raw)
To: Ian Jackson; +Cc: longtaox.pang, xen-devel
On Thu, 2015-04-02 at 12:21 +0100, Ian Jackson wrote:
> Ian Campbell writes ("[Xen-devel] [PATCH OSSTEST v4 00/25] add distro domU testing flight"):
> > The failures mainly seem to be some sort of proxy issue returning old
> > data (similar to what used to happen to mg-debian-installer-update),
>
> Maybe we can set up an explicit proxy which actually works. I guess
> that might need negotiation with Citrix corporate IT.
I think it would, in order for the proxy to get past the intercepting
thing. I suspect the best answer we could expect would be "here is how
to use the intercepting thing directly", which is unlikely to help.
> Luckily the new colo is not affected by anything like this.
Indeed, and I was thinking that this flight would normally run there,
unless capacity concerns mean not for now.
> > On a small number I see "The installer failed to download a file from
> > the mirror." messages, which might be the cache/proxy or might be an
> > actual Debian issue.
>
> Hrm. I find these vague error messages exceptionally frustrating
> (debootstrap has this problem too).
If it happened in an interactive install you could get to a more
comprehensive logs, but that's not that useful for automation.
Perhaps a wishlist against d-i such that if it is preseeding and
DEBIAN_FRONTEND=text it will dump /var/log/syslog on fail? Or at least
an option to do so.
> > The -raw tests all failed due to a timeout doing:
> > dd if=/dev/zero of=/var/lib/xen/images/debian/disk.raw bs=1M count=10000
> >
> > I'm unclear if it timed out after 30 (as the code seems to say) or 60
> > (as the logs seem to say) but either way something obviously needs
> > adjusting. I just tried it on my local test box and it took 1m20s and
> > reported 130 MB/s. I've appended an as yet untested patch which assumed
> > 100 MB/s and calculates a timeout from that.
>
> Do you intend to fold that patch in ?
Yes, I just left it separate on this iteration because it was new and
untested, even if somewhat obvious.
> > I could switch to using "count=1 seek=10000" to create a sparse file,
> > which should be pretty much instantaneous, but allocating the whole
> > backing file seemed like a good idea.
>
> Right.
>
> Thanks,
> Ian.
^ permalink raw reply [flat|nested] 40+ messages in thread