Re: Guest Testing in OSSTEST - What distros and versions should we test against

From: "Roger Pau Monné" <roger.pau@citrix.com>
To: Jan Beulich <JBeulich@suse.com>
Cc: Lars Kurth <lars.kurth.xen@gmail.com>,
	Juergen Gross <jgross@suse.com>,
	committers@xenproject.org,
	Ian Jackson <Ian.Jackson@eu.citrix.com>,
	xen-devel <xen-devel@lists.xenproject.org>
Subject: Re: Guest Testing in OSSTEST - What distros and versions should we test against
Date: Mon, 27 May 2019 13:56:49 +0200	[thread overview]
Message-ID: <20190527115649.zhpfcskp7rtpnqdr@Air-de-Roger> (raw)
In-Reply-To: <5CD52CEA020000780022D6B1@prv1-mh.provo.novell.com>

On Fri, May 10, 2019 at 01:48:58AM -0600, Jan Beulich wrote:
> >>> On 10.05.19 at 03:28, <lars.kurth.xen@gmail.com> wrote:
> > Hi all,
> > 
> > following a discussion with committers about Guest testing in OSSTEST, it 
> > surfaced that we have not updated what distros we test in OSSTEST for a very 
> > long time. All agreed that we should regularly review what we test against: 
> > maybe at the beginning of a release cycle
> > 
> > In any case, currently we test against
> > 
> > x86 HVM guests:
> >   debian-9.4.0-{i386,amd64}-CD-1.iso
> >   rhel-server-6.1-i386-dvd.iso
> >   win10v1703-x86.iso
> >   win7-x64.iso
> >   ws16-x64.iso
> >   FreeBSD-10.1-CUSTOM-{i386,amd64}-20150525.raw.xz
> 
> Testing fixed snapshots (commonly the original release afaict) instead
> of latest updates is pretty meaningless imo. While I would like to
> recommend testing all the (active) distros of parties actively involved
> in Xen development, I have to admite that I have no idea at all how
> this would work for e.g. our SLE versions (such that latest bits would
> get tested).

FWIW, this is what osstest does in order to test FreeBSD HEAD. I'm not
sure how much of this would apply to SLE.

There's a dedicated FreeBSD flight, that does the following:

 - Install a bare-metal box using the latest known to work install
   media.
 - Fetch FreeBSD sources and build new install media from scratch.
 - Use the newly built media to install a box from scratch.
 - Build FreeBSD again to make sure the new version is capable of
   building itself at least.
 - Build Xen using the new FreeBSD version.

If all of those steps succeed the new version is anointed, and it will
be used as the install media for new flights.

Note that this is a time consuming process, so making this part of a
regular Xen flight is not feasible, hence the anointment in order to
store the binary installer so other flights can re-use it.

Also note that this process is circular, and needs an initial seed
(binary installer) to be manually anointed for all this to work.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel