From mboxrd@z Thu Jan 1 00:00:00 1970 From: Doug Goldstein Subject: Re: Test Meetup at Developer Summit Date: Tue, 6 Sep 2016 10:53:47 -0500 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5944043898131221518==" Return-path: Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bhIh9-0007SB-8c for xen-devel@lists.xenproject.org; Tue, 06 Sep 2016 15:53:55 +0000 Received: by mail-yw0-f169.google.com with SMTP id g192so70626481ywh.1 for ; Tue, 06 Sep 2016 08:53:53 -0700 (PDT) In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Lars Kurth , xen-devel Cc: Andrew Cooper , committers@xenproject.org, George Dunlap , Jonathan Creekmore List-Id: xen-devel@lists.xenproject.org This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --===============5944043898131221518== Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="oBospxjS9uk70P3Oc5cuTC3kc0fQhelWx" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --oBospxjS9uk70P3Oc5cuTC3kc0fQhelWx Content-Type: multipart/mixed; boundary="51Cs67rAplWAFEK2TRomPoVXpexr5jhST"; protected-headers="v1" From: Doug Goldstein To: Lars Kurth , xen-devel Cc: George Dunlap , Andrew Cooper , committers@xenproject.org, Jonathan Creekmore Message-ID: Subject: Re: Test Meetup at Developer Summit References: In-Reply-To: --51Cs67rAplWAFEK2TRomPoVXpexr5jhST Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 9/5/16 6:19 AM, Lars Kurth wrote: > =3D=3D Attendees =3D=3D > Lars Kurth > George Dunlap > Doug Goldstein > Andrew Cooper > Paul Durrant >=20 > There were a few others, which I may have missed I know Jonathan Creekmore was involved as well. >=20 > I tried to transcribe from a recording we had at lunch, but due to back= ground noise I didn't get everything. Please add/correct, if I got someth= ing wrong. There was a bit of discussion, but I tried to capture the key = points only (e.g. I omitted clarifying questions) Thanks Lars! Sorry for the delay in responses I went immediately from the Summit to vacation for the past week. >=20 > =3D=3D Discussion: Travis Improvements =3D=3D >=20 > Doug, wanted to discuss changes to the project's Travis set-up > * Reconfiguring Travis set-up such that it launches docker containers t= o do very basic smoke testing of individual commits on a number of differ= ent build configurations > * Right now this would require running a script running inside a script= > * The key question is what matrix to support: e.g. which compiler versi= ons, whether it should cover deb builds, ... > * It is basically just a build test, with a very basic run-test within = an emulated qemu environment to check whether we boot. Aka booting a smal= l image that is cross-compiled against various environments that produces= "Hello World" on the first available serial-port > * If "Hello World" was produced, then Travis would decide the build was= successful >=20 > George: over OSSTEST what value would this add?=20 > Doug: feedback about 7-8 minutes after a push with some confidence that= a build was not broken, faster than OSSTEST does now. Obviously this wou= ld not replace OSSTEST, but should reduce the number of build-related iss= ues in OSSTEST > Andrew: This should help quite a lot with avoiding spotting build relat= ed issues earlier > Doug: Would help avoid unnecessary manual build testing > George: Should help committers also=20 > Lars: Are there any extra requirements for people to use? > Andrew: Needs an account and set-up on GitHub If we decide to stick with GitHub and Travis then I plan on documenting this so its easier to people to do this on their own. >=20 > =3D=3D Discussion: GitHub vs. GitLab for Travis =3D=3D > The second set of topics was to move from GitHub to GitLab for Build-te= sting due to the code availability issues with GitHub. Doug uses GitLab i= nternally and is also more functional that GitHub.=20 The concern when I initiated the GitHub + Travis bits were that GitHub and Travis were not free as in speech but only free as in beer. GitLab is free as in speech [1] and if you use their hosted service [2] its free as in beer as well. They've made outstanding strides this year with their integrated CI service called GitLab CI. At my company we now use this for all of our projects since it allows you to define multiple steps and dependencies of those steps. Steps can be flagged to have to run within specific Docker containers which allows us to have a matrix of build environments to build test with. It also allows you to capture artifacts from the build which is what we do to actually run real tests against a built object (ala OSSTest). Travis + GitHub at this point allow you to specify that you need sudo and then from there you can script up Docker commands by hand which we can certainly do but there is no parallelism and its a lot more we need to maintain in our scripts. >=20 > GitLab can be used hosted (for free projects) or host it locally on a V= M >=20 > Doug: would probably require taking the github yaml and rewrite as gitl= ab yam. There are a number of advantages on OS support and docker integra= tion, which makes set-up a lot easier compared to github. >=20 > Andy: if we were going to do this, the project should probably get a su= pport license >=20 > Summary: > - Gather community input on GitHub vs. GitLab > - GitLab code is public, and thus should address=20 I think this was suppose to end with "address concerns about software licensing of the software we use" >=20 > There was also a bit of discussion about the review work-flow functiona= lity in GitLab: just to clarify, we were not discussing any changes in th= is area. Merely to use GitLab as framework for Travis / Build testing and= addressing some of the past issues that were raised around GitHub code a= vailability. I'm not proposing using any of the merge request or issue tracker bits. Just the CI portion for doing builds and allowing people to push a branch and have it built on a variety of compilers to lessen the chance that "it worked on my machine" situations strike. [1] https://gitlab.com/gitlab-org/gitlab-ce [2] https://about.gitlab.com/gitlab-com/ --=20 Doug Goldstein --51Cs67rAplWAFEK2TRomPoVXpexr5jhST-- --oBospxjS9uk70P3Oc5cuTC3kc0fQhelWx Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0 iQJ8BAEBCgBmBQJXzuaOXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNTM5MEQ2RTNFMTkyNzlCNzVDMzIwOTVB MkJDMDNEQzg3RUQxQkQ0AAoJEKK8A9yH7RvUDyEQAJHaQEUGLBV7Em/1CZlzJBw2 lO6VxkHJSRU63UhhZmLPW2ycToHSVPjZ0TbtgKN9mPCJiaJCEmPRKTfRFEx4AVua mIbrldawZZYc8moIEbseYhCVz6r5lWc9AxKTBJYvCdvT+E5i91cvsuEadDybSLXl KFwppxyTVkIPZhe/VGjtJ5+NcRBlxe66JfDlkmfvV7qzk4/OoCepCbnFr1BVGExx KT4oaeb1Bu3NzaBno9QqcTsar3YHALNto0E14JBx5j3VfFDDzCL8vpv4eWsKollq X+o57nWQa6A6drYfptSY+EcYDue1GJzZaN2hwfRggEpsaGOINTckmxyTfLeGoNKn DXn5mFDSXLfqQvB8dMt3hUXN0GkWPYy7+uSYBJeEg1Y6fpHJnFG7jJRGU5WaN31J 3kqEFcyOJiqi6ltRgw5OjapB4G6S51FoR54x+X9/dCbzBO8RoOFfcvprxw3FtT8x 4PYwlyC0ZTJmUiXGjPKx4kvRCsLpoYZMOV/3K0nYrzHkFF5nvxPq0KpPGYgAZJuP kaAgcYE+a83Rlp7AqB4KraIvFqEawOS/dxvhxLYOhD7PWAiBibkjNAj7Nt26UbEX 3fWrHltHc9mtZGBKXvxC6wVE6iIQvj6uLsiTzNO0oYWgD2UZExL2FCbc2nB3rCyI SyqWJRouhRPXNdVrLizV =J67c -----END PGP SIGNATURE----- --oBospxjS9uk70P3Oc5cuTC3kc0fQhelWx-- --===============5944043898131221518== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --===============5944043898131221518==--