[OSSTest PATCH [RFC] 0/4] New test case: Fedora PV guests

[OSSTest PATCH [RFC] 0/4] New test case: Fedora PV guests

[Dario Faggioli]

Hi,

here they are more OSSTest bits, but this time implementing an actual test
case! ;-P

In fact, this series adds one test case (resulting in a few jobs) for a Fedora
PV guest, instead of the fairly typical Debian one. Basically, what we do is we
install a guest via the Anaconda installer with a kickstart file, allowing for
completely automated installation (much like d-i preseeds), and the run it
through the usual `test-guest' procedure from sg-run-job: start, stop,
save/restore, local migration).

Jobs are introduced, in this series, for a Fedora19 guest and for a
Fedora20-RC1 one... PV guests, in both of the cases. I'm now adding a few words
on why I think this could be useful.

Well, first of all, the more testing the better, of course. With this respect,
Fedora updates the kernel (and that would mean the guest kernel, as far as
these test jobs are concerned) fairly often, meaning we get the chance to test
how staging does with an almost current DomU kernel, independently from what
Dom0 kernel is being used. For instance, F19 had a 3.9 when released, while it
is on 3.11.10 as of now. That's the reason why I think having a F19 test case
(right now, it'll become F20 as soon as that will be released, later in
December) could be valuable.

The reason why I also included 20-RC1 jobs is the Fedora 'DomU chriterion'. So,
long story short, the Fedora release chriteria includes this: 'The release must
boot successfully as Xen DomU with releases providing a functional, supported
Xen Dom0 and widely used cloud providers utilizing Xen' [1]. If a Fedora TC
(stands for Test Compose) or RC does not boot as a PV guest, it can't be
released. Well, for the last couple of releases Konrad and I tried to cover for
the testing necessary to assess this, and we did manage quite well. Still, it
would be really great to have this integrated in OSSTest, as that would
guarantee much more coverage and reliability, for the benefit of both Xen's and
Fedora's communities.

Add to the above the fact that, if we're scarce on test hardware, there is
really no need to run neither both nor even just one of the test cases/jobs
described above in all the flights.  It may well be enough to have the
"released" (so, the F19 one right now, the F20 one from F20 on, and so on and
so forth), like, once a week, while the TC/RC (i.e., the equivalent of the
20-RC1 jobs in this series) could be scheduled to happen only when the time of
a new Fedora releas approaches, i.e., when the TCs and RCs are actually
available!  :-)

The new jobs introduced in this series are these:

 $ sqlite3 standalone.db 'select * from jobs;' | grep fedora
  standalone|test-amd64-i386-fedora19-x86_64|test-fedora|queued
  standalone|test-amd64-i386-fedora20-RC1-x86_64|test-fedora|queued
  standalone|test-amd64-i386-fedora19-i386|test-fedora|queued
  standalone|test-amd64-i386-fedora20-RC1-i386|test-fedora|queued
  standalone|test-amd64-amd64-fedora19-x86_64|test-fedora|queued
  standalone|test-amd64-amd64-fedora20-RC1-x86_64|test-fedora|queued
  standalone|test-amd64-amd64-fedora19-i386|test-fedora|queued
  standalone|test-amd64-amd64-fedora20-RC1-i386|test-fedora|queued

(it'd be possible to include armhf jobs too, but I left this commented out for
now, as I haven't tested it).

Right now, changing the releases that we want to test, e.g., switching from
20-RC1 to 20-RC2 (when available), requires actually changing 'make-flight',
which I don't think is that sexy... This is the main reason why this series is
an RFC... IanJ, IanC, are there better ways to achieve this?

I tested this in standalone mode, e.g., via:

 ./standalone-reset
 ./sg-run-job build-amd64
 ./sg-run-job build-amd64-pvops
 ./sg-run-job test-amd64-amd64-fedora19-i386

or:

 ./standalone-reset
 ./sg-run-job build-amd64
 ./sg-run-job build-amd64-pvops
 ./sg-run-job test-amd64-amd64-fedora20-RC1-x86_64

And it all seems to have worked. :-)

The series can be pulled from the following git branch:

  git://xenbits.xen.org/people/dariof/osstest.git fedora-guest

Thanks and Regards,
Dario

[1] https://fedoraproject.org/wiki/Fedora_20_Final_Release_Criteria#Xen_DomU
---

Dario Faggioli (4):
      Introducing Osstest/Fedora.pm
      ts-fedora-install: added for installing fedora guests
      sg-run-job: Fedora guest job
      make-flight: Fedora guest tests


 Osstest.pm                |    4 +
 Osstest/Fedora.pm         |   83 ++++++++++++++++++++++
 README                    |    9 ++
 make-flight               |   19 +++++
 sg-run-job                |   11 +++
 standalone-config-example |    2 +
 ts-fedora-install         |  168 +++++++++++++++++++++++++++++++++++++++++++++
 7 files changed, 296 insertions(+)
 create mode 100644 Osstest/Fedora.pm
 create mode 100755 ts-fedora-install

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

[OSSTest PATCH [RFC] 1/4] Introducing Osstest/Fedora.pm

[Dario Faggioli]

which, for now, only handles the kickstart files (the
equivalent of d-i preseeds).

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
---
 Osstest.pm                |    4 ++
 Osstest/Fedora.pm         |   83 +++++++++++++++++++++++++++++++++++++++++++++
 README                    |    9 +++++
 standalone-config-example |    2 +
 4 files changed, 98 insertions(+)
 create mode 100644 Osstest/Fedora.pm

diff --git a/Osstest.pm b/Osstest.pm
index 9ba2882..1cc5acb 100644
--- a/Osstest.pm
+++ b/Osstest.pm
@@ -62,6 +62,8 @@ our %c = qw(
     DebianSuite squeeze
     DebianMirrorSubpath debian
 
+    FedoraMirrorSubpath fedora/linux
+
     TestHostKeypairPath id_rsa_osstest
     HostProp_GenEtherPrefixBase 5e:36:0e:f5
 
@@ -179,6 +181,8 @@ sub readglobalconfig () {
     $c{OverlayLocal} ||= "overlay-local";
     $c{GuestDebianSuite} ||= $c{DebianSuite};
 
+    $c{GuestFedoraRelease} ||= 19;
+
     $c{DefaultBranch} ||= 'xen-unstable';
 }
 
diff --git a/Osstest/Fedora.pm b/Osstest/Fedora.pm
new file mode 100644
index 0000000..a5c7cec
--- /dev/null
+++ b/Osstest/Fedora.pm
@@ -0,0 +1,83 @@
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+package Osstest::Fedora;
+
+use strict;
+use warnings;
+
+use IO::File;
+use File::Copy;
+
+use Osstest;
+use Osstest::TestSupport;
+
+BEGIN {
+    use Exporter ();
+    our ($VERSION, @ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS);
+    $VERSION     = 1.00;
+    @ISA         = qw(Exporter);
+    @EXPORT      = qw(kickstart_create);
+    %EXPORT_TAGS = ( );
+
+    @EXPORT_OK   = qw();
+}
+
+#---------- installation of Fedora via kickstart file ----------
+
+sub kickstart_create($$$) {
+    my ($ho, $repourl, $sfx) = @_;
+
+    my $ks_file.= (<<END);
+install
+firewall --disabled
+keyboard 'us'
+reboot
+rootpw --plaintext xenroot
+timezone Europe/London --isUtc
+lang en_US
+auth --useshadow --passalgo=sha512
+text # text mode install
+selinux --permissive
+skipx
+
+url --url="$repourl"
+network  --bootproto=dhcp --device=eth0
+
+bootloader --location=mbr
+zerombr
+clearpart --all
+part swap --asprimary --fstype="swap" --size=1000
+part / --asprimary --fstype="ext4" --grow --size=1
+
+\%packages
+\@admin-tools
+\@standard
+\%end
+
+\%post
+exec < /dev/console > /dev/console
+echo "# Adding osstest user"
+useradd -p \`openssl passwd -1 osstest\` osstest
+echo "# Upgrading the system"
+yum -y upgrade
+\%end
+END
+    return create_webfile($ho, "ks$sfx", $ks_file);
+}
+
+1;
diff --git a/README b/README
index 29c9d45..c2ce2c1 100644
--- a/README
+++ b/README
@@ -89,6 +89,13 @@ DebianMirrorHost
    Set DebianMirrorSubpath to the path inside the mirror if
    your mirror isn't at http://<DebianMirrorHost>/debian/
 
+FedoraMirrorHost
+   Domain name or address of the Fedora mirror to use.
+   Set FedoraMirrorSubpath to the path inside the mirror.
+   http://<FedoraMirrorHost>/<FedoraMirrorSubpath>/ should
+   point to where 'releases' and/or 'development' directory
+   are.
+
 TestHost <hostname>
 TestHost_<ident> <hostname>
    Specifies the test box to use.  Should be a bare hostname,
@@ -191,6 +198,8 @@ GuestDebianSuite   defaults to DebianSuite
 
 DebianPreseed      added to existing preseed file
 
+GuestFedoraRelease defaults to 19
+
 TftpPxeTemplates
     List (space-separated) of template filenames for writing
     The templates contain variable substitutions %var%
diff --git a/standalone-config-example b/standalone-config-example
index 9b2b79b..2bc18f7 100644
--- a/standalone-config-example
+++ b/standalone-config-example
@@ -26,6 +26,8 @@ HostProp_DhcpWatchMethod leases dhcp3 dhcp.uk.xensource.com:5556
 TftpPath /usr/groups/netboot/
 
 DebianMirrorHost debian.uk.xensource.com
+FedoraMirrorHost dl.fedoraproject.org
+FedoraMirrorSubpath pub/fedora/linux
 
 DebianPreseed= <<'END'
 d-i clock-setup/ntp-server string ntp.uk.xensource.com

[OSSTest PATCH [RFC] 2/4] ts-fedora-install: added for installing fedora guests

[Dario Faggioli]

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
---
 ts-fedora-install |  168 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 168 insertions(+)
 create mode 100755 ts-fedora-install

diff --git a/ts-fedora-install b/ts-fedora-install
new file mode 100755
index 0000000..6b55091
--- /dev/null
+++ b/ts-fedora-install
@@ -0,0 +1,168 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2013 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use Osstest::Fedora;
+use Osstest::TestSupport;
+
+use Data::Dumper;
+
+tsreadconfig();
+
+our ($whhost,$gn) = @ARGV;
+$whhost ||= 'host';
+$gn ||= 'fedora';
+
+our $ho= selecthost($whhost);
+
+our $ram_mb=   512;
+our $disk_mb= 5000;
+
+our $guesthost= "$gn.guest.osstest";
+our $gho;
+
+sub prep () {
+    target_install_packages_norec($ho, qw(lvm2 kpartx));
+
+    $gho= prepareguest($ho, $gn, $guesthost, 22,
+                       $disk_mb, 40);
+
+    prepareguest_part_lvmdisk($ho, $gho, $disk_mb);
+
+    target_cmd_root($ho, "umount $gho->{Lvdev} ||:");
+}
+
+sub ginstall () {
+    my $arch= defined($r{"$gho->{Guest}_arch"}) ?
+                  $r{"$gho->{Guest}_arch"} : "x86_64";
+    my $pae= ($arch eq 'i386') ? '-PAE' : '';
+    my $fedora_release= guest_var($gho, 'fedora_release',
+                                  $c{GuestFedoraRelease});
+
+    # Paths within a mirror are different depending on whether we are dealing
+    # with an actual released distro, one in development, or one being tested
+    # right before a release (Test Composes or RC-s, see
+    # https://fedoraproject.org/wiki/Test_Results:Current_Installation_Test).
+    my $releaseurl= "http://$c{FedoraMirrorHost}/$c{FedoraMirrorSubpath}/releases/$fedora_release/Fedora/$arch/os";
+    my $develurl = "http://$c{FedoraMirrorHost}/$c{FedoraMirrorSubpath}/development/$fedora_release/$arch/os";
+    my $stageurl = "http://dl.fedoraproject.org/pub/alt/stage/$fedora_release/Fedora/$arch/os";
+    my $repourl;
+
+    if (system("wget -O /tmp/ti --quiet $releaseurl/.treeinfo") == 0) {
+        $repourl = $releaseurl;
+    } elsif (system("wget -O /tmp/ti --quiet $develurl/.treeinfo") == 0) {
+        $repourl = $develurl;
+    } elsif (system("wget -O /tmp/ti --quiet $stageurl/.treeinfo") == 0) {
+        $repourl = $stageurl;
+    } else {
+        die "unable to find images for Fedora $fedora_release for $arch"
+    }
+
+    my $fi_url = "$repourl/images/pxeboot";
+    target_cmd($ho, <<END, 2000);
+	wget --quiet -O /tmp/fi_kernel $fi_url/vmlinuz$pae
+	wget --quiet -O /tmp/fi_initrd $fi_url/initrd$pae.img
+END
+
+    my $ks_url = kickstart_create($gho, $repourl, '');
+
+    my $vcpus= guest_var($gho, 'vcpus', 1);
+
+    my $install_cfg= <<END;
+name = '$gho->{Name}'
+# Fedora installer requires no less than 1GB RAM
+memory = 1024
+#
+kernel      = "/tmp/fi_kernel"
+ramdisk     = "/tmp/fi_initrd"
+extra       = "repo=$repourl console=hvc0 text serial ks=$ks_url"
+#
+vif = [ 'mac=$gho->{Ether}' ]
+#
+on_poweroff = 'destroy'
+# the below is needed for allowing guest_await_reboot() to trigger
+on_reboot   = 'preserve'
+on_crash    = 'preserve'
+#
+vcpus = $vcpus
+#
+disk = [
+        'phy:$gho->{Lvdev},xvda,w'
+       ]
+END
+
+    my $cfgpath= "/etc/xen/$gho->{Name}.cfg";
+
+    $gho->{CfgPath}= $cfgpath;
+    store_runvar("$gho->{Guest}_cfgpath", "$cfgpath");
+    target_putfilecontents_root_stash($ho, 30, $install_cfg, $cfgpath);
+
+    my $cmd= toolstack()->{Command}." create ".
+        $r{ $gho->{Guest}.'_'. toolstack()->{CfgPathVar} };
+    target_cmd_root($ho, $cmd, 300);
+
+    guest_checkrunning($ho, $gho) or die "$gho->{Name} not running";
+
+    guest_await_reboot($ho, $gho, 2000);
+    guest_destroy($ho, $gho);
+
+    # Some fixups
+    store_runvar("$gho->{Guest}_tcpcheckport", 22);
+
+    my $authkeys= authorized_keys();
+    my $rootdev = target_guest_lv_name($ho, $gho->{Name}) . "--disk2";
+    target_cmd_root($ho, <<END);
+        set -ex
+        kpartx -a $gho->{Lvdev}
+        mount $rootdev /mnt
+        mkdir -p /mnt/root/.ssh /mnt/etc/ssh
+        cat <<'ENDKEYS' > /mnt/root/.ssh/authorized_keys
+$authkeys
+ENDKEYS
+        cp -a /etc/ssh/ssh_host_*key* /mnt/etc/ssh/
+        mkdir -p /mnt/home/osstest/.ssh
+        cp /mnt/root/.ssh/authorized_keys /mnt/home/osstest/.ssh
+        chroot /mnt chown -R osstest.osstest /home/osstest/.ssh
+        umount /mnt
+        kpartx -d $gho->{Lvdev}
+END
+
+    my $runtime_cfg= <<END;
+name = '$gho->{Name}'
+memory = ${ram_mb}
+#
+bootloader = "pygrub"
+#
+vif = [ 'mac=$gho->{Ether}' ]
+#
+on_poweroff = 'destroy'
+on_reboot   = 'restart'
+on_crash    = 'preserve'
+#
+vcpus = $vcpus
+#
+disk = [
+        'phy:$gho->{Lvdev},xvda,w'
+       ]
+END
+    target_putfilecontents_root_stash($ho, 30, $runtime_cfg, $cfgpath);
+}
+
+prep();
+ginstall();

[OSSTest PATCH [RFC] 3/4] sg-run-job: Fedora guest job

[Dario Faggioli]

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
---
 sg-run-job |   11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/sg-run-job b/sg-run-job
index db62365..d216408 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -253,6 +253,17 @@ proc run-job/test-freebsd {} {
     test-guest freebsd
 }
 
+proc install-guest-fedora {} {
+    run-ts . = ts-fedora-install
+    run-ts . = ts-guest-start + fedora
+}
+
+proc need-hosts/test-fedora {} { return host }
+proc run-job/test-fedora {} {
+    install-guest-fedora
+    test-guest fedora
+}
+
 proc need-hosts/test-win {} { return host }
 proc run-job/test-win {} {
     run-ts . = ts-windows-install

[OSSTest PATCH [RFC] 4/4] make-flight: Fedora guest tests

[Dario Faggioli]

Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
---
 make-flight |   19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/make-flight b/make-flight
index dcfdfe6..08a4a98 100755
--- a/make-flight
+++ b/make-flight
@@ -355,6 +355,25 @@ for xenarch in ${TEST_ARCHES- i386 amd64 armhf } ; do
 		debian_arch=$dom0arch   				  \
 		all_hostflags=$most_hostflags
 
+
+      case ${xenarch} in
+	  amd64) fedora_domUarches="x86_64 i386";;
+	  i386)  fedora_domUarches="";;
+	  # No ARM for now. Fedora has, from F18 on, an ARM version, and it
+	  # all should work, but I haven't had the chance to test it in
+	  # standalone mode.
+	  #armhf) fedora_domUarches="armhfp";;
+      esac
+
+      for domU in $fedora_domUarches ; do
+        for dist in "19" "20-RC1" ; do
+	  job_create_test test-$xenarch$kern-$dom0arch-fedora$dist-$domU test-fedora xl \
+	        fedora_arch=$domU \
+	        fedora_fedora_release=$dist \
+	        all_hostflags=$most_hostflags
+        done
+      done
+
       if [ $xenarch = amd64 -a $dom0arch = i386 -a "$kern" = "" ]; then
 
         for freebsdarch in amd64 i386; do

Re: [OSSTest PATCH [RFC] 1/4] Introducing Osstest/Fedora.pm

[Ian Jackson]

Dario Faggioli writes ("[OSSTest PATCH [RFC] 1/4] Introducing Osstest/Fedora.pm"):
> which, for now, only handles the kickstart files (the
> equivalent of d-i preseeds).

Doesn't this duplicate some of the code in ts-redhat-install ?

It seems to me that it would probaby be better to treat the
rpm/kickstart ecosystem as more of a single kind of thing.
So perhaps this new perl module should have a different name, too.

Ian.

Re: [OSSTest PATCH [RFC] 2/4] ts-fedora-install: added for installing fedora guests

[Ian Jackson]

Dario Faggioli writes ("[OSSTest PATCH [RFC] 2/4] ts-fedora-install: added for installing fedora guests"):
> Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
...
> +    # Paths within a mirror are different depending on whether we are dealing
> +    # with an actual released distro, one in development, or one being tested
> +    # right before a release (Test Composes or RC-s, see
> +    # https://fedoraproject.org/wiki/Test_Results:Current_Installation_Test).
> +    my $releaseurl= "http://$c{FedoraMirrorHost}/$c{FedoraMirrorSubpath}/releases/$fedora_release/Fedora/$arch/os";
> +    my $develurl = "http://$c{FedoraMirrorHost}/$c{FedoraMirrorSubpath}/development/$fedora_release/$arch/os";
> +    my $stageurl = "http://dl.fedoraproject.org/pub/alt/stage/$fedora_release/Fedora/$arch/os";

I'm not sure I like this kind of downloading of test OS code for each
platform.  This is sustainable for one distro (the Debian which most of
our tests use), but if we need to maintain a local mirror for every
test target OS the infrastructure is going to be a royal pain to
maintain.  And we'll be exposed to a much wider range of failures
which will cause spurious test failures.  This surely applies to
release images - couldn't they just be stored in the osstest images
directory ?

OTOH if we are having a Fedora-specific flight for checking RCs, I
think downloading things directly from the Fedora upstream is fine.

> +    my $fi_url = "$repourl/images/pxeboot";
> +    target_cmd($ho, <<END, 2000);

target_cmd doesn't do set -e.  Perhaps it should ?

> +	wget --quiet -O /tmp/fi_kernel $fi_url/vmlinuz$pae
> +	wget --quiet -O /tmp/fi_initrd $fi_url/initrd$pae.img

You shouldn't use /tmp really like this.  Please use a
flight-and-job-specific directory in ~.

> +    my $install_cfg= <<END;
> +name = '$gho->{Name}'
> +# Fedora installer requires no less than 1GB RAM
> +memory = 1024
> +#
> +kernel      = "/tmp/fi_kernel"
> +ramdisk     = "/tmp/fi_initrd"
> +extra       = "repo=$repourl console=hvc0 text serial ks=$ks_url"
> +#
> +vif = [ 'mac=$gho->{Ether}' ]
> +#
> +on_poweroff = 'destroy'
> +# the below is needed for allowing guest_await_reboot() to trigger
> +on_reboot   = 'preserve'
> +on_crash    = 'preserve'

Perhaps there is other similar code that this could be combined with.

> +    my $authkeys= authorized_keys();
> +    my $rootdev = target_guest_lv_name($ho, $gho->{Name}) . "--disk2";
> +    target_cmd_root($ho, <<END);
> +        set -ex
> +        kpartx -a $gho->{Lvdev}
> +        mount $rootdev /mnt
> +        mkdir -p /mnt/root/.ssh /mnt/etc/ssh
> +        cat <<'ENDKEYS' > /mnt/root/.ssh/authorized_keys
> +$authkeys
> +ENDKEYS
> +        cp -a /etc/ssh/ssh_host_*key* /mnt/etc/ssh/
> +        mkdir -p /mnt/home/osstest/.ssh
> +        cp /mnt/root/.ssh/authorized_keys /mnt/home/osstest/.ssh
> +        chroot /mnt chown -R osstest.osstest /home/osstest/.ssh
> +        umount /mnt
> +        kpartx -d $gho->{Lvdev}
> +END

This messing with authorized_keys is very similar to code that appears
in Debian.pm, TestSupport.pm and/or ts-debian-fixup.

> +    my $runtime_cfg= <<END;

This is duplicating some of the logic in ts-redhat-install ?

Ian.

Re: [OSSTest PATCH [RFC] 2/4] ts-fedora-install: added for installing fedora guests

[Ian Jackson]

Dario Faggioli writes ("[OSSTest PATCH [RFC] 2/4] ts-fedora-install: added for installing fedora guests"):

> +    guest_await_reboot($ho, $gho, 2000);
> +    guest_destroy($ho, $gho);
> +
> +    # Some fixups
> +    store_runvar("$gho->{Guest}_tcpcheckport", 22);

The Debian guest installation is split into ts-debian-install and
ts-debian-fixup, so that editing the fixup code doesn't involve
rerunning the time-consuming installation step.

Would that be worth doing here as well ?

Thanks,
Ian.

Re: [OSSTest PATCH [RFC] 4/4] make-flight: Fedora guest tests

[Ian Jackson]

Dario Faggioli writes ("[OSSTest PATCH [RFC] 4/4] make-flight: Fedora guest tests"):
> +      case ${xenarch} in
> +	  amd64) fedora_domUarches="x86_64 i386";;
> +	  i386)  fedora_domUarches="";;
> +	  # No ARM for now. Fedora has, from F18 on, an ARM version, and it
> +	  # all should work, but I haven't had the chance to test it in
> +	  # standalone mode.
> +	  #armhf) fedora_domUarches="armhfp";;
> +      esac
> +
> +      for domU in $fedora_domUarches ; do
> +        for dist in "19" "20-RC1" ; do
> +	  job_create_test test-$xenarch$kern-$dom0arch-fedora$dist-$domU test-fedora xl \

This puts the Fedora arch name in the job name.  I think it would be
better if our job names used a consistent arch naming scheme (i.e.,
the Debian one).

According to your 0/4 message, this creates a large number of new test
jobs, but you're patching the ordinary make-flight.  So I think this
isn't really suitable as it is.

Probably what we want is one or two extra jobs in the existing flights
(for testing new Xens still work with released Fedora) and a new more
comprehensive kind of flight for testing Fedora RCs.

Ian.

Re: [OSSTest PATCH [RFC] 4/4] make-flight: Fedora guest tests

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 2303 bytes --]

On ven, 2013-12-13 at 16:49 +0000, Ian Jackson wrote:
> Dario Faggioli writes ("[OSSTest PATCH [RFC] 4/4] make-flight: Fedora guest tests"):
> > +      case ${xenarch} in
> > +	  amd64) fedora_domUarches="x86_64 i386";;
> > +	  i386)  fedora_domUarches="";;
> > +	  # No ARM for now. Fedora has, from F18 on, an ARM version, and it
> > +	  # all should work, but I haven't had the chance to test it in
> > +	  # standalone mode.
> > +	  #armhf) fedora_domUarches="armhfp";;
> > +      esac
> > +
> > +      for domU in $fedora_domUarches ; do
> > +        for dist in "19" "20-RC1" ; do
> > +	  job_create_test test-$xenarch$kern-$dom0arch-fedora$dist-$domU test-fedora xl \
> 
> This puts the Fedora arch name in the job name.  I think it would be
> better if our job names used a consistent arch naming scheme (i.e.,
> the Debian one).
> 
You mean sticking with amd64 rather than x86_64? If yes, ok, I agree and
that's easy enough.

> According to your 0/4 message, this creates a large number of new test
> jobs, but you're patching the ordinary make-flight.  So I think this
> isn't really suitable as it is.
> 
So, the "lage number of new jobs" comes from the combination of Dom0
arches and DomU arches, and from the fact that I'm testing two releases
(19 and 20RC1, as per this series). I agree that all this variation is
not necessary in regular flights, as I also tried to state myself in
0/4, so, yes, I think I can fall back to a less comprehensive set of
test jobs, like only testing _one_ release, both amd64 and i386 for
DomU, but only amd64 for dom0, like it is being done for the FreeBSD
tests... Would this work?

> Probably what we want is one or two extra jobs in the existing flights
> (for testing new Xens still work with released Fedora) and a new more
> comprehensive kind of flight for testing Fedora RCs.
> 
Ok, you mean (the final part) creating 'make-fedora-flight' with all and
only the fedora jobs, right? Yes, I guess I can do that.

Thanks for the comments,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [OSSTest PATCH [RFC] 4/4] make-flight: Fedora guest tests

[Ian Jackson]

Dario Faggioli writes ("Re: [OSSTest PATCH [RFC] 4/4] make-flight: Fedora guest tests"):
> On ven, 2013-12-13 at 16:49 +0000, Ian Jackson wrote:
> > This puts the Fedora arch name in the job name.  I think it would be
> > better if our job names used a consistent arch naming scheme (i.e.,
> > the Debian one).
> 
> You mean sticking with amd64 rather than x86_64? If yes, ok, I agree and
> that's easy enough.

Exactly, thanks.

> > According to your 0/4 message, this creates a large number of new test
> > jobs, but you're patching the ordinary make-flight.  So I think this
> > isn't really suitable as it is.
> > 
> So, the "lage number of new jobs" comes from the combination of Dom0
> arches and DomU arches, and from the fact that I'm testing two releases
> (19 and 20RC1, as per this series). I agree that all this variation is
> not necessary in regular flights, as I also tried to state myself in
> 0/4, so, yes, I think I can fall back to a less comprehensive set of
> test jobs, like only testing _one_ release, both amd64 and i386 for
> DomU, but only amd64 for dom0, like it is being done for the FreeBSD
> tests... Would this work?

Right.

> > Probably what we want is one or two extra jobs in the existing flights
> > (for testing new Xens still work with released Fedora) and a new more
> > comprehensive kind of flight for testing Fedora RCs.
> > 
> Ok, you mean (the final part) creating 'make-fedora-flight' with all and
> only the fedora jobs, right? Yes, I guess I can do that.

Perhaps.  I'm not sure whether it wants to be a different script or
not.  We already have some linux-specific flights which suppress most
of the jobs.  Maybe it would be better done with some ifs in the
existing script, or something.

If you do invent a make-fedora-flight you will have to factor out the
build parts of make-flight into a separate sub-script or a function in
a sourced file, or something.

Ian.

Re: [OSSTest PATCH [RFC] 2/4] ts-fedora-install: added for installing fedora guests

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1216 bytes --]

On ven, 2013-12-13 at 16:45 +0000, Ian Jackson wrote:
> Dario Faggioli writes ("[OSSTest PATCH [RFC] 2/4] ts-fedora-install: added for installing fedora guests"):
> 
> > +    guest_await_reboot($ho, $gho, 2000);
> > +    guest_destroy($ho, $gho);
> > +
> > +    # Some fixups
> > +    store_runvar("$gho->{Guest}_tcpcheckport", 22);
> 
> The Debian guest installation is split into ts-debian-install and
> ts-debian-fixup, so that editing the fixup code doesn't involve
> rerunning the time-consuming installation step.
> 
> Would that be worth doing here as well ?
> 
Well, given the fact that Debian is installed with xen-tools, there's
probably more fixup to do there than here, and that's why it looked
sensible enough to keep the two together in this case.

Anyway, I think being able to change what we do during fixup without
necessarily having to reinstall is something valuable to have. I'll go
for it.

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [OSSTest PATCH [RFC] 1/4] Introducing Osstest/Fedora.pm

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 872 bytes --]

On ven, 2013-12-13 at 16:33 +0000, Ian Jackson wrote:
> Dario Faggioli writes ("[OSSTest PATCH [RFC] 1/4] Introducing Osstest/Fedora.pm"):
> > which, for now, only handles the kickstart files (the
> > equivalent of d-i preseeds).
> 
> Doesn't this duplicate some of the code in ts-redhat-install ?
> 
> It seems to me that it would probaby be better to treat the
> rpm/kickstart ecosystem as more of a single kind of thing.
> So perhaps this new perl module should have a different name, too.
> 
Good point. Not sure it's doable up to 100%, but I'll look into that.

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [OSSTest PATCH [RFC] 4/4] make-flight: Fedora guest tests

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1037 bytes --]

On ven, 2013-12-13 at 17:34 +0000, Ian Jackson wrote:
> Dario Faggioli writes ("Re: [OSSTest PATCH [RFC] 4/4] make-flight: Fedora guest tests"):
> > Ok, you mean (the final part) creating 'make-fedora-flight' with all and
> > only the fedora jobs, right? Yes, I guess I can do that.
> 
> Perhaps.  I'm not sure whether it wants to be a different script or
> not.  We already have some linux-specific flights which suppress most
> of the jobs.  Maybe it would be better done with some ifs in the
> existing script, or something.
> 
Aha, I was forgetting about those... So, just to check if I'm
understanding correctly, you mean, as far as make-flight is concerned, a
$branch. E.g, a 'fedora' branch, as we have a 'linux-next' branch,
right?

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [OSSTest PATCH [RFC] 2/4] ts-fedora-install: added for installing fedora guests

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 3786 bytes --]

On ven, 2013-12-13 at 16:44 +0000, Ian Jackson wrote:
> Dario Faggioli writes ("[OSSTest PATCH [RFC] 2/4] ts-fedora-install: added for installing fedora guests"):
> > Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
> ...
> > +    # Paths within a mirror are different depending on whether we are dealing
> > +    # with an actual released distro, one in development, or one being tested
> > +    # right before a release (Test Composes or RC-s, see
> > +    # https://fedoraproject.org/wiki/Test_Results:Current_Installation_Test).
> > +    my $releaseurl= "http://$c{FedoraMirrorHost}/$c{FedoraMirrorSubpath}/releases/$fedora_release/Fedora/$arch/os";
> > +    my $develurl = "http://$c{FedoraMirrorHost}/$c{FedoraMirrorSubpath}/development/$fedora_release/$arch/os";
> > +    my $stageurl = "http://dl.fedoraproject.org/pub/alt/stage/$fedora_release/Fedora/$arch/os";
> 
> I'm not sure I like this kind of downloading of test OS code for each
> platform.  This is sustainable for one distro (the Debian which most of
> our tests use), but if we need to maintain a local mirror for every
> test target OS the infrastructure is going to be a royal pain to
> maintain.  And we'll be exposed to a much wider range of failures
> which will cause spurious test failures.  This surely applies to
> release images - couldn't they just be stored in the osstest images
> directory ?
> 
I'm not sure what "osstest images directory" is; I'll check. What I care
is for these jobs to keep being usable in standalone mode, if going that
way is not a problem wrt that, I think I can.

Also, this are not .iso or disk images, they're just a kernel and a
ramdisk, but I guess that does not make much of a difference.

I'll check what RH testcases do, as well as what Roger did for FreeBSD,
as I remember seeing something about downloading images and storing them
in the proper place in those patches too.

> OTOH if we are having a Fedora-specific flight for checking RCs, I
> think downloading things directly from the Fedora upstream is fine.
> 
> > +    my $fi_url = "$repourl/images/pxeboot";
> > +    target_cmd($ho, <<END, 2000);
> 
> target_cmd doesn't do set -e.  Perhaps it should ?
> 
Ok.

> > +	wget --quiet -O /tmp/fi_kernel $fi_url/vmlinuz$pae
> > +	wget --quiet -O /tmp/fi_initrd $fi_url/initrd$pae.img
> 
> You shouldn't use /tmp really like this.  Please use a
> flight-and-job-specific directory in ~.
> 
And that would be for the sake of keeping the installer's kernel and
initrd and have them archived together with the logs?

I was under the impression that this won't be much useful, but I guess
it would, in case something goes wrong during installation and one wants
to debug... Is that the reason?

> > +    my $install_cfg= <<END;
> > +name = '$gho->{Name}'
> > +# Fedora installer requires no less than 1GB RAM
> > +memory = 1024
> > +#
> > +kernel      = "/tmp/fi_kernel"
> > +ramdisk     = "/tmp/fi_initrd"
> > +extra       = "repo=$repourl console=hvc0 text serial ks=$ks_url"
> > +#
> > +vif = [ 'mac=$gho->{Ether}' ]
> > +#
> > +on_poweroff = 'destroy'
> > +# the below is needed for allowing guest_await_reboot() to trigger
> > +on_reboot   = 'preserve'
> > +on_crash    = 'preserve'
> 
> Perhaps there is other similar code that this could be combined with.
> 
Ok, I'll see if I can factor a bit more (valid also for the other
comments that were expressed below this point for this patch).

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [OSSTest PATCH [RFC] 4/4] make-flight: Fedora guest tests

[Ian Jackson]

Dario Faggioli writes ("Re: [Xen-devel] [OSSTest PATCH [RFC] 4/4] make-flight: Fedora guest tests"):
> On ven, 2013-12-13 at 17:34 +0000, Ian Jackson wrote:
> > Perhaps.  I'm not sure whether it wants to be a different script or
> > not.  We already have some linux-specific flights which suppress most
> > of the jobs.  Maybe it would be better done with some ifs in the
> > existing script, or something.
> 
> Aha, I was forgetting about those... So, just to check if I'm
> understanding correctly, you mean, as far as make-flight is concerned, a
> $branch. E.g, a 'fedora' branch, as we have a 'linux-next' branch,
> right?

Yes.  However it's done, from the pov of the database and the cron
jobs it would probably be a new $branch, but that doesn't mean it
can't have a new script.

I don't have a fixed opinion about whether ifs in make-flight are the
right approach, compared to a separate script.  You should pick for
yourself the one that makes the code smaller and/or less complicated
:-).

Thanks,
Ian.

Re: [OSSTest PATCH [RFC] 2/4] ts-fedora-install: added for installing fedora guests

[Ian Jackson]

Dario Faggioli writes ("Re: [Xen-devel] [OSSTest PATCH [RFC] 2/4] ts-fedora-install: added for installing fedora guests"):
> On ven, 2013-12-13 at 16:44 +0000, Ian Jackson wrote:
> > I'm not sure I like this kind of downloading of test OS code for each
> > platform.  This is sustainable for one distro (the Debian which most of
> > our tests use), but if we need to maintain a local mirror for every
> > test target OS the infrastructure is going to be a royal pain to
> > maintain.  And we'll be exposed to a much wider range of failures
> > which will cause spurious test failures.  This surely applies to
> > release images - couldn't they just be stored in the osstest images
> > directory ?
> > 
> I'm not sure what "osstest images directory" is; I'll check. What I care
> is for these jobs to keep being usable in standalone mode, if going that
> way is not a problem wrt that, I think I can.

I mean $c{Images}.  You could provide a script (mg-download-fedora) to
download/update them.

> Also, this are not .iso or disk images, they're just a kernel and a
> ramdisk, but I guess that does not make much of a difference.

Indeed.  The debian-installer stuff goes in /tftpboot but is handled
much the way I'm suggesting.

> I'll check what RH testcases do, as well as what Roger did for FreeBSD,
> as I remember seeing something about downloading images and storing them
> in the proper place in those patches too.

Right.

> > You shouldn't use /tmp really like this.  Please use a
> > flight-and-job-specific directory in ~.
> 
> And that would be for the sake of keeping the installer's kernel and
> initrd and have them archived together with the logs?

No, it's because in the future I want to be able to run multiple test
jobs on a single host, so the pathnames oughtn't to be fixed.  And
there is no good reason for using /tmp here.  /tmp has a number of
problems (many of which don't apply here, admittedly) so it's not a
good habit to use it.

We're talking about paths on the test host dom0 here, right ?

> Ok, I'll see if I can factor a bit more (valid also for the other
> comments that were expressed below this point for this patch).

Thanks,
Ian.