[Fuego] Slides, presentations, and notes from Fuego Jamboree

[Fuego] Slides, presentations, and notes from Fuego Jamboree

[Tim.Bird]

Hello Fuego community,

I have uploaded pictures from the Fuego Jamboree to the wiki page.
We had about 15 people there, and had some good discussions about
the status of Fuego and things that different groups are working on
to extend or improve it.

The slides are available on the wiki page as well, and 
I took some notes on issues that were discussed.

I'm including the notes below for discussion on the e-mail list.
Please see the wiki page for the materials from the event:
http://fuegotest.org/wiki/Fuego_Jamboree_2

Let me know your feedback from the event, or input about the
discussion items mentioned below.  If you have additional
notes, please add them to the page, or send them to the
list.

Regards,
 -- Tim

------ (start of notes)
= Notes for discussion of Roadmap and Priorities =

Thanks to everyone who came to the event.  It was valuable having you there!

Here are a few notes that Tim took from the meeting:

== ftc run-test issues ==
 * Daniel proposed isolating Jenkins code in ftc, so that commands that don't need to access Jenkins don't have dependencies on it (e.g. don't do 'import jenkins' unless needed)
 * Khiem noted that there may be concurrency issues using the command line tool only
   * Jenkins handles job concurrency if the job goes through it.  It won't schedule another job for the board, if one is currently running.  ftc run-test doesn't have a run-time concurrency check, only a build-time concurrency check.
 * We definitely need to add concurrency control for a board, for ftc run-test.

== fuego run scalability ==
 * Khiem also noted that he sees a lot of buildup of run data that clogs the Jenkins interface
   * Tim noted that there are mechanisms for the tables to only show the last 40 entries in a chart, but nothing in Fuego to remove old run data
   * Khiem says he runs hundreds of tests a week, and the number of runs ends up clogging up the system.
   * Tim says that certain things in Fuego, like the report generation, can get really slow if there is too much run data.  This is a fundamental scalability issue that needs to be addressed.
      * some --where clauses require reading every run's run.json file and some don'), so you may only see slowness on some queries.
 * one potential solutions would be to make a script which deletes old run data
 * maybe add an ftc command to remove runs.
   * could be something that uses --where clauses:
     * 'ftc rm-runs --where start_time=<"1 week ago"'

== stand-alone layers ==
Based on discussions with Hiramatsu-san (who was representing the LKFT project
at this event), it seems like improving our layering would be good.
Daniel has already (or will shortly) layer the code between Fuego and 
the aggregation/presentation layer - supporting kernelci, fuego and squad
servers.  Daniel mentioned isolating the parsing layer, as that seems to be
something other systems don't have, that would be good to make available
to other projects. 

Tim wants to layer the software as well - including separating the parsing
layer (he agrees with Daniel on this), as well as formalizing the
board control and provisioning layer.  Maybe we could have some preliminary
stuff in the 1.4 release

== release timing ==
Khiem asked about the timing of releases.  Tim said that 6 months is too long,
which is what each of the last 2 releases have been.  We should shoot for 3-month releases, even if it reduces the number of features in each release.
The group discussed trying to achieve the 1.4 release in early October,
prior to Embedded Linux Conference Europe, and the Automated Testing Summit.
Tim will be busy most of July, and won't make much progress, but we should
still be able to do something in that time period.

One thing that has stalled releases is release testing. We continue to increase
the amount of Fuego self-test and release-test code, so the overhead of making
a release should decrease over time.

== program binary cache ==
Tim mentioned that he has 2 reasons to want to do a test program binary cache:
 * to support the LAVA integration work by providing a way for LAVA worker nodes to download needed test program code independently from having a Fuego builder node
 * to support users who don't have an SDK for their board.  This would include people using Fuego to self-test their host machine
Due to this, Tim thinks it would be good to work on the test program binary cache.  (''Note: I have a prototype written that I worked on on the plane home, that might be part of this'')

== Priorities ==
The following seem to be important priorities, based on discussions at the Jamboree:
 * ftc run-test concurrency (some kind of board lock)
 * ftc rm-run command, with where-clause support
 * isolate the parser system and make it standalone
 * create a test program binary cache

As well as all the other stuff that was mentioned in the slides.  :-)
------ (end of notes)