From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Dlba3zz8TBDeviNrw7HDDWB7rPtW00/YtckKADyOOR4=; b=bSQhI2Bfhz7BKmm5245MiKAK9ubq4JumHb7fetLfnvd1YhVhHyX0TosMdp3imKogBH cMkpoQ+yMif4WBsgfQQD2i1mf6a+XWPlm8pPReAeAgFpY/gKXnOdYuiZ0B/9ndbNZ9zh N6UrLKBERTXrR5nRWMRFbtKM7G1MerRayQAkjfCVPAmrx5TQE1Zhht4NekQkMPTy80xw j2EcPAE14s6+edcNHSE9m6zne8iY4d9kbvEc1gdRezUNMTzCl6Pn6Cv/Y1ThsuFdBiIT ECxH9P+/KPtnnOLbePKlzaHBnkTMDUaCwnFQ2poHko+Kh6T6DuncPaFaOwgmtsBN/nGs P3sw== MIME-Version: 1.0 In-Reply-To: References: From: dhinakar k Date: Wed, 29 Mar 2017 08:04:13 +0530 Message-ID: Content-Type: multipart/alternative; boundary=001a113ad4aadfa7ee054bd56bcc Subject: Re: [Fuego] UART/USB Connectivity, History Database, Test result format List-Id: Mailing list for the Fuego test framework List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Bird, Timothy" Cc: "fuego@lists.linuxfoundation.org" --001a113ad4aadfa7ee054bd56bcc Content-Type: text/plain; charset=UTF-8 Hi Tim, That was a very detailed reply. Thank you very much. Just one follow up question. How does Fuego compare with Avocado? If you had checked it out already please let me know. Regards, Dhinakar On Tue, Mar 28, 2017 at 11:54 PM, Bird, Timothy wrote: > > > > -----Original Message----- > > From dhinakar k on Tuesday, March 28, 2017 3:26 AM > > Hi, > > > > I would like to know how the following are handled by Fuego. > > > > 1.Connectivity to targets which support UART or USB > > - If Fuego supports it, then how is it done? (e.g. via Perl Serial > module?) > > - How are the Request/Response handled via UART or USB connectivity? > > UART connectivity is handled with the 'serial' transport, using a program > called 'serio'. Support for this is currently in what I'd call 'alpha' > format. > It works, for the most part, with a few issues that need to be resolved > before we can call it an official feature. > > For USB there are two methods of support (one existing and one intended): > * network over USB - that is supported today, but the configuration of the > USB networking is left as an exercise for the user > * ADB - There are patches available to support ADB, but they are not > currently integrated. > > > 2.How can we archive job details, execution results etc.? Does Fuego > already > > have support for database? If so, which database? (e.g. MySQL?) > > Currently, at a local site, job results are saved in XML in the Jenkins > build.xml > file. Fuego saves it's own parallel set of results to a run.json file > (when the test > is executed with 'ftc run-test'). This will be changing shortly (likely > in the 1.1 release), > to have all tests (including those executed by Jenkins) create a run.json > file. > > We have not decided on a centralized database yet, although there is > prototype > code in the fuego wiki software to manage json files like a database (e.g. > supporting filters > and queries and table generation). However, this wiki code is not > finished yet, > and is lower priority. > > > 3.What format(s) supported for test results? (json, xunit, yaml, TAP?) > > All tests integrated into Fuego will output results in the above > format? > > The target is json, with converters to other formats as needed. Currently, > benchmarks produce json output, and there is prototype code to have > functional tests produce json output (for sub-tests) as well. The AGL > group has code > to produce XML, but it appears they are interested in switching to json > output, > and the intent is to unify around this format. > > The intent is to have all tests integrated into Fuego produce the json > output, > and have report generators that take the format as input. We have report > generators > that take the benchmark data, and functional data (in json) and produce pdf > reports (with xml and tex as intermediate formats). However, with a big > refactoring > that we're doing in the format output, there will be some churn on this > part of > Fuego over the next 2 months or so. > > > Please answer the above queries. > > I hope this is helpful. > -- Tim > > --001a113ad4aadfa7ee054bd56bcc Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Tim,

That was a very deta= iled reply.
Thank you very much.

Just on= e follow up question.
How does Fuego compare with Avocado?
<= div>If you had checked it out already please let me know.

Regards,
Dhinakar
<= br>
On Tue, Mar 28, 2017 at 11:54 PM, Bird, Timot= hy <Tim.Bird@sony.com> wrote:


> -----Original Message-----
> From dhinakar k on Tuesday, March 28, 2017 3:26 AM
> Hi,
>
> I would like to know how the following are handled by Fuego.
>
> 1.Connectivity to targets which support UART or USB
>=C2=A0 =C2=A0 - If Fuego supports it, then how is it done? (e.g. via Pe= rl Serial module?)
>=C2=A0 =C2=A0 - How are the Request/Response handled via UART or USB co= nnectivity?

UART connectivity is handled with the 'serial' transport, us= ing a program
called 'serio'.=C2=A0 Support for this is currently in what I'd= call 'alpha' format.
It works, for the most part, with a few issues that need to be resolved
before we can call it an official feature.

For USB there are two methods of support (one existing and one intended): =C2=A0* network over USB - that is supported today, but the configuration o= f the
USB networking is left as an exercise for the user
=C2=A0* ADB - There are patches available to support ADB, but they are not<= br> currently integrated.

> 2.How can we archive job details, execution results etc.? Does Fuego a= lready
> have support for database? If so, which database? (e.g. MySQL?)

Currently, at a local site, job results are saved in XML in the Jenk= ins build.xml
file.=C2=A0 Fuego saves it's own parallel set of results to a run.json = file (when the test
is executed with 'ftc run-test').=C2=A0 This will be changing short= ly (likely in the 1.1 release),
to have all tests (including those executed by Jenkins) create a run.json f= ile.

We have not decided on a centralized database yet, although there is protot= ype
code in the fuego wiki software to manage json files like a database (e.g. = supporting filters
and queries and table generation).=C2=A0 However, this wiki code is not fin= ished yet,
and is lower priority.

> 3.What format(s) supported for test results? (json, xunit, yaml, TAP?)=
>=C2=A0 =C2=A0 All tests integrated into Fuego will output results in th= e above format?

The target is json, with converters to other formats as needed.=C2= =A0 Currently,
benchmarks produce json output, and there is prototype code to have
functional tests produce json output (for sub-tests) as well. The AGL group= has code
to produce XML, but it appears they are interested in switching to json out= put,
and the intent is to unify around this format.

The intent is to have all tests integrated into Fuego produce the json outp= ut,
and have report generators that take the format as input.=C2=A0 We have rep= ort generators
that take the benchmark data, and functional data (in json) and produce pdf=
reports (with xml and tex as intermediate formats).=C2=A0 However, with a b= ig refactoring
that we're doing in the format output, there will be some churn on this= part of
Fuego over the next 2 months or so.

> Please answer the above queries.

I hope this is helpful.
=C2=A0-- Tim


--001a113ad4aadfa7ee054bd56bcc--