On the whole, I'd like to extend our current charting mechanisms to support what you need. scripts/mod.js can support arbitrary HTML generated by prepare_chart_data.py, so we should be able to make it work.

With regard to batch_id, that feature is not finished yet. Filtering on it is trivial to add to ftc and prepare_chart_data.py.

But I don't think the field is recorded in run.json and the other files used for charting. The key issue outstanding was the mechanism for assigning the batch_id for runs. The idea was to have a test plan job set the batch_id for the jobs it started. But there's not a good Jenkins mechanism to do this. When we finished having Jenkins call ftc run-test to run test jobs, my plan was to add a new ftc run-plan, and have Jenkins call that for executing batch jobs. Thus would allow me to set a batch ID for the jobs associated with the batch run, I think.

Anyway, there just needs to be some batch_id on the job command line, or in an environment variable, that gets put into the run.json file.

I'll take a look at this next week.

Regards,
- - Tim


-------- Original Message --------
Subject: Re: [Fuego] Problem of adding a benchmark result table
From: "Li, Xiaoming" <lixm.fnst@cn.fujitsu.com>
Date: Aug 24, 2018, 12:57 AM
To: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>,fuego@lists.linuxfoundation.org
Hi Daniel,

> -----Original Message-----
> From: Daniel Sangorrin [mailto:daniel.sangorrin@toshiba.co.jp]
> Sent: Friday, August 24, 2018 11:11 AM
> To: Li, Xiaoming/李 霄鸣; fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] Problem of adding a benchmark result table
>
> Hi Li,
>
> > <fuego-bounces@lists.linuxfoundation.org> On Behalf Of Li, Xiaoming
>  > I want to add a table of building results for every single benchmark in
> Jenkins.
>
> If you want to use a table to show the building results instead of a plot, you
> can change the chart_type from chart_config.json to testset_summary_table.
> If you run the benchmark again, you should get a table instead of a plot.

>
> That functionality is implemented at:
> scripts/parser/prepare_chart_data.py
> scripts/mod.js
>
> I believe that code needs some improvements:
>    - ability to override chart_config.json with a per-board configuration
>    - I have seen in Dhrystone that the table shows pass=1 but also err=1
>    - The tables only show building results (pass/fail), you may want to add the
> benchmark measures
>
> > My solutions is to use jenkins's plugin groovy postbuild, which can
> > add a hyperlink in build history.
> > But groovy postbuild requies jenkins‘s version of 2.6.0 or later, as
> > fuego's default is 2.32.1.
>
> Updating jenkins regularly is important. However, we have had bad
> experiences with Jenkins' plugins. Each time we upgraded Jenkins a number
> of plugins where not available anymore causing many breakages. For that
> reason, we decided to limit the usage of plugins to the minimum necessary.
>
> > I also have to install  java8 to run higher version jenkins.
> This would require changing the Dockerfile. It might be a hard job if it can't be
> done with Debian Jessie.
>
> > Could anyone give me a better idea?
>
> Apart from using chart_config.son, there is another option. We have an SQL-
> like interface to fetch results from Fuego. You can use that interface with ftc
> gen-report and create an html/xls/pdf report with all of the build results that
> you want. Maybe you could even call to ftc gen-report in a postbuild action
> and then add a link to the generated html from the "Set build description"
> post action.

I have tried it , and it works.
But the  --where batch_id arg  can not work temporarily,   I will  investigate it sometime later.
On the whole,  it is a  wonderful  method.



>
> Also, soon we will be able to post test results to kernelci and squad instances.
>
> Best regards,
> Daniel
>
> > --------------------------------------------------
> >
> > Li, Xiaoming
> >
> > Development Dept.III
> >
> > Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST) No. 6 Wenzhu
> > Road, Nanjing, 210012, China
> >
> > TEL: +86+25-86630566-8451
> >
> > FAX: +86+25-83317685
> >
> > MAIL: lixm.fnst@cn.fujitsu.com <mailto:lixm.fnst@cn.fujitsu.com>
> >
> > --------------------------------------------------
> >
> >
>
>
>


Thx
Li


_______________________________________________
Fuego mailing list
Fuego@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/fuego