Re: [OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests

[Ian Jackson <ian.jackson@eu.citrix.com>]

Anthony PERARD writes ("Re: [OSSTEST PATCH v11 16/20] ts-openstack-tempest: Update list of skipped tests"):
> On Fri, Jun 23, 2017 at 04:38:05PM +0100, Ian Jackson wrote:
> > Aha.  Well, if the stable branch is stable then the set of tests to
> > skip there is probably stable too ?  And on master it's built-in ?  So
> > this actually won't be a problem - in the sense that this will be
> > approximately the last necessary update to the list of tests to skip ?
> 
> The built-in list is to be consume by the CI, it is not used
> automatically. It a bash script to be sourced. I have not yet try to
> have osstest use it. That will be for later.

Hah.

> Right now, tempest print on stdout something much simpler, lines liked
> this:
> {1} tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario [71.009969s] ... ok
> Followed by summary of all test, and the output of failed tests.
> 
> But it can be told to print subunit v2, which can be pipped to a different
> program to transform into want we want, including simple cvs, subunit v1, and
> other.
> 
> I thing we could try to parse subunit v1, grab the start and finish time
> of a test, the result, and the output of the tests. And have all that as
> subunit. During my manual tests, `tempest run --subunit ... | subunit-2to1`
> only print the stuff about a tests once it is finish, so I guess we'll
> have to rewrite substep_start() (or another function) to take time as
> parameter.

That sounds quite good really.

I guess you don't want to write a parser for subunit v2 but if there
is one in Perl already in some package in Debian we could use it.  Or
subunit-2to1 is fine.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel