[Fuego] How run Fuego tools on a Jenkins slave node

[Fuego] How run Fuego tools on a Jenkins slave node

[Michael Nazzareno Trimarchi]

[-- Attachment #1: Type: text/plain, Size: 157 bytes --]

Hi

I have already a ci setup but i would like to use Fuego and run script on
slave and get results on master node. Is there a way to configure it?

Michael

[-- Attachment #2: Type: text/html, Size: 272 bytes --]

Re: [Fuego] How run Fuego tools on a Jenkins slave node

[Tim.Bird]

> -----Original Message-----
> From: Fuego <fuego-bounces@lists.linuxfoundation.org> On Behalf Of Michael Nazzareno Trimarchi
> Sent: Thursday, August 26, 2021 11:38 PM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] How run Fuego tools on a Jenkins slave node
> 
> Hi
> 
> I have already a ci setup but i would like to use Fuego and run script on slave and get results on master node. Is there a way to configure it?

I believe so.  Other Fuego users have used Fuego with CI systems other than the
default one included (the Jenkins instance inside the Fuego docker container).

You can perform tests using Fuego, using the 'ftc' command line tool.

The default installation of Fuego creates a docker container where Fuego core
scripts and Jenkins are installed.  You can run 'ftc' from inside this container
(which is where the included Jenkins will execute test jobs from), or from outside
the container, from the command line.

When you install Fuego, you have the option to build the container without Jenkins
(so it just consist of the build environment for the tests, and the Fuego core scripts).
See http://fuegotest.org/wiki/Installing_Fuego#without_Jenkins for a description
of the install option used for this.

By way of example, when Fuego is run with Jenkins, it creates Jenkins jobs with
the following command sequence for the "build" attribute (this example is for
the test Functional.hello_world):


#logging areas=pre_test,pre_check,build,makepkg,deploy,snapshot,run,
#    post_test,processing, parser,criteria,charting
#logging levels=debug,verbose,info,warning,error
#export FUEGO_LOGLEVELS="run:debug,parser:verbose"
export FUEGO_CALLER="jenkins"
ftc run-test -b $NODE_NAME -t Functional.hello_world -s default \
    --timeout 6m \
    --reboot false \
    --rebuild false \
    --precleanup true \
    --postcleanup true

If the environment variable FUEGO_CALLER is set to "jenkins", then ftc and the
core test system perform additional steps with the integrated
Jenkins system (such as augmenting the Jenkins build with pointers to Fuego 
test artifacts, so they are accessible in the Jenkins interface).  However, if you
are running Jenkins somewhere else (not in the
Fuego container), then I think the jobs and build data will not be in the
expected directories for Fuego to operate on.  So this aspect of Fuego/Jenkins
integration would be lost (or you would have to recreate it).

If FUEGO_CALLER is not set, then ftc and the core test system will just output
the test execution to stdout, and deposit test artifacts in the fuego-rw/logs
directory.  Build artifacts (e.g. the test program source and build directories
are under fuego-rw/buildzone).

See 'ftc run-test help' for information about the arguments used in the above
example ftc invocation. Not all of these arguments are required (in fact, most
of those will default to the values shown above, but they are placed in the Jenkins
job definition to allow users to more easily remember and edit them for a particular job.)

If I had a BeagleBone Black board in my lab, named 'bbb', I could execute
a test at the command line with: "ftc run-test -b bbb -t hello_world"

By default, visualization of Fuego results (tables and plots) are prepared
for use in the integrated Jenkins interface.  However, you can also generate reports
in various formats, that might be useful with your Jenkins (or whatever other CI
system one might use).

Look at the options for 'ftc gen-report', particularly the 'txt', 'html', or 'csv' output
formats, which might be import-able into your CI system, for generating your
own results data displays.

The graphical results (measure plots) for Benchmark data is generated in a json
format which is interpreted by a javascript module called 'mod.js', which uses
a graphing module called 'flot' that is installed as a Jenkins plugin.  'flot' and 'mod.js'
might be usable with other CI systems that have a web-based user interface
(as they are just javascript (well, jquery)) but it would take some work to perform
that integration.

It sounds like part of your question is about how the host/target relationship is
handled in Fuego. By default, a Fuego test runs on a host, and the test programs
are actually transferred to the device under test and executed there, during test
execution.  However, some Fuego users install the Fuego core system on the actual
device under test, and execute it locally.  In this case, the remote aspect of the
test would be handled by the calling CI system, not by Fuego.

Normally, with the integrated version of Jenkins (the one in the Fuego docker container),
there is a slave used that is assigned to each node (board).  This ends up serializing
the access to that board.  But this slave runs on the host machine, where Jenkins
and Fuego are installed.  Are you asking about executing a Jenkins slave on a remote
machine (perhaps the device under test), and having Fuego installed there?

Hopefully the information above  is helpful.

I'm not sure exactly what you are asking, but please let me know if you need more
information.  I am happy to discuss your configuration, and see if Fuego would work
for you.
 -- Tim
  

Re: [Fuego] How run Fuego tools on a Jenkins slave node

[Tim.Bird]

I just realized you might be asking about how to make the Fuego Jenkins instance a slave
to your already-existing Jenkins instance master.  I have not done a configuration of
Jenkins master/slave like this, but here are some instructions for doing this.
https://baykara.medium.com/part-ii-add-a-jenkins-slave-to-jenkins-master-857ef055122d


> -----Original Message-----
> From: Fuego <fuego-bounces@lists.linuxfoundation.org> On Behalf Of Tim.Bird@sony.com
> 
> > -----Original Message-----
> > From: Fuego <fuego-bounces@lists.linuxfoundation.org> On Behalf Of Michael Nazzareno Trimarchi
> >
> > Hi
> >
> > I have already a ci setup but i would like to use Fuego and run script on slave and get results on master node. Is there a way to configure
> it?
> 
> I believe so.  Other Fuego users have used Fuego with CI systems other than the
> default one included (the Jenkins instance inside the Fuego docker container).
> 
> You can perform tests using Fuego, using the 'ftc' command line tool.
> 
> The default installation of Fuego creates a docker container where Fuego core
> scripts and Jenkins are installed.  You can run 'ftc' from inside this container
> (which is where the included Jenkins will execute test jobs from), or from outside
> the container, from the command line.
> 
> When you install Fuego, you have the option to build the container without Jenkins
> (so it just consist of the build environment for the tests, and the Fuego core scripts).
> See http://fuegotest.org/wiki/Installing_Fuego#without_Jenkins for a description
> of the install option used for this.
> 
> By way of example, when Fuego is run with Jenkins, it creates Jenkins jobs with
> the following command sequence for the "build" attribute (this example is for
> the test Functional.hello_world):
> 
> 
> #logging areas=pre_test,pre_check,build,makepkg,deploy,snapshot,run,
> #    post_test,processing, parser,criteria,charting
> #logging levels=debug,verbose,info,warning,error
> #export FUEGO_LOGLEVELS="run:debug,parser:verbose"
> export FUEGO_CALLER="jenkins"
> ftc run-test -b $NODE_NAME -t Functional.hello_world -s default \
>     --timeout 6m \
>     --reboot false \
>     --rebuild false \
>     --precleanup true \
>     --postcleanup true
> 
> If the environment variable FUEGO_CALLER is set to "jenkins", then ftc and the
> core test system perform additional steps with the integrated
> Jenkins system (such as augmenting the Jenkins build with pointers to Fuego
> test artifacts, so they are accessible in the Jenkins interface).  However, if you
> are running Jenkins somewhere else (not in the
> Fuego container), then I think the jobs and build data will not be in the
> expected directories for Fuego to operate on.  So this aspect of Fuego/Jenkins
> integration would be lost (or you would have to recreate it).
> 
> If FUEGO_CALLER is not set, then ftc and the core test system will just output
> the test execution to stdout, and deposit test artifacts in the fuego-rw/logs
> directory.  Build artifacts (e.g. the test program source and build directories
> are under fuego-rw/buildzone).
> 
> See 'ftc run-test help' for information about the arguments used in the above
> example ftc invocation. Not all of these arguments are required (in fact, most
> of those will default to the values shown above, but they are placed in the Jenkins
> job definition to allow users to more easily remember and edit them for a particular job.)
> 
> If I had a BeagleBone Black board in my lab, named 'bbb', I could execute
> a test at the command line with: "ftc run-test -b bbb -t hello_world"
> 
> By default, visualization of Fuego results (tables and plots) are prepared
> for use in the integrated Jenkins interface.  However, you can also generate reports
> in various formats, that might be useful with your Jenkins (or whatever other CI
> system one might use).
> 
> Look at the options for 'ftc gen-report', particularly the 'txt', 'html', or 'csv' output
> formats, which might be import-able into your CI system, for generating your
> own results data displays.
> 
> The graphical results (measure plots) for Benchmark data is generated in a json
> format which is interpreted by a javascript module called 'mod.js', which uses
> a graphing module called 'flot' that is installed as a Jenkins plugin.  'flot' and 'mod.js'
> might be usable with other CI systems that have a web-based user interface
> (as they are just javascript (well, jquery)) but it would take some work to perform
> that integration.
> 
> It sounds like part of your question is about how the host/target relationship is
> handled in Fuego. By default, a Fuego test runs on a host, and the test programs
> are actually transferred to the device under test and executed there, during test
> execution.  However, some Fuego users install the Fuego core system on the actual
> device under test, and execute it locally.  In this case, the remote aspect of the
> test would be handled by the calling CI system, not by Fuego.
> 
> Normally, with the integrated version of Jenkins (the one in the Fuego docker container),
> there is a slave used that is assigned to each node (board).  This ends up serializing
> the access to that board.  But this slave runs on the host machine, where Jenkins
> and Fuego are installed.  Are you asking about executing a Jenkins slave on a remote
> machine (perhaps the device under test), and having Fuego installed there?
> 
> Hopefully the information above  is helpful.
> 
> I'm not sure exactly what you are asking, but please let me know if you need more
> information.  I am happy to discuss your configuration, and see if Fuego would work
> for you.
>  -- Tim
> 
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] How run Fuego tools on a Jenkins slave node

[Michael Nazzareno Trimarchi]

[-- Attachment #1: Type: text/plain, Size: 6588 bytes --]

Hi

On Fri, Aug 27, 2021, 6:17 PM <Tim.Bird@sony.com> wrote:

> I just realized you might be asking about how to make the Fuego Jenkins
> instance a slave
> to your already-existing Jenkins instance master.  I have not done a
> configuration of
> Jenkins master/slave like this, but here are some instructions for doing
> this.
>
> https://baykara.medium.com/part-ii-add-a-jenkins-slave-to-jenkins-master-857ef055122d


When I read the code the FTC command run only locally. Create pipeline
using FTC is not possible. What i would like to have is the FTC script on
master and the executor on slave.

Do I need to split the logic?

I have already a complex multislave configuration but i have a lot of
hardware and slave but i decided to go to Fuego as regression testing
machine

Michael

>
>
>
> > -----Original Message-----
> > From: Fuego <fuego-bounces@lists.linuxfoundation.org> On Behalf Of
> Tim.Bird@sony.com
> >
> > > -----Original Message-----
> > > From: Fuego <fuego-bounces@lists.linuxfoundation.org> On Behalf Of
> Michael Nazzareno Trimarchi
> > >
> > > Hi
> > >
> > > I have already a ci setup but i would like to use Fuego and run script
> on slave and get results on master node. Is there a way to configure
> > it?
> >
> > I believe so.  Other Fuego users have used Fuego with CI systems other
> than the
> > default one included (the Jenkins instance inside the Fuego docker
> container).
> >
> > You can perform tests using Fuego, using the 'ftc' command line tool.
> >
> > The default installation of Fuego creates a docker container where Fuego
> core
> > scripts and Jenkins are installed.  You can run 'ftc' from inside this
> container
> > (which is where the included Jenkins will execute test jobs from), or
> from outside
> > the container, from the command line.
> >
> > When you install Fuego, you have the option to build the container
> without Jenkins
> > (so it just consist of the build environment for the tests, and the
> Fuego core scripts).
> > See http://fuegotest.org/wiki/Installing_Fuego#without_Jenkins for a
> description
> > of the install option used for this.
> >
> > By way of example, when Fuego is run with Jenkins, it creates Jenkins
> jobs with
> > the following command sequence for the "build" attribute (this example
> is for
> > the test Functional.hello_world):
> >
> >
> > #logging areas=pre_test,pre_check,build,makepkg,deploy,snapshot,run,
> > #    post_test,processing, parser,criteria,charting
> > #logging levels=debug,verbose,info,warning,error
> > #export FUEGO_LOGLEVELS="run:debug,parser:verbose"
> > export FUEGO_CALLER="jenkins"
> > ftc run-test -b $NODE_NAME -t Functional.hello_world -s default \
> >     --timeout 6m \
> >     --reboot false \
> >     --rebuild false \
> >     --precleanup true \
> >     --postcleanup true
> >
> > If the environment variable FUEGO_CALLER is set to "jenkins", then ftc
> and the
> > core test system perform additional steps with the integrated
> > Jenkins system (such as augmenting the Jenkins build with pointers to
> Fuego
> > test artifacts, so they are accessible in the Jenkins interface).
> However, if you
> > are running Jenkins somewhere else (not in the
> > Fuego container), then I think the jobs and build data will not be in the
> > expected directories for Fuego to operate on.  So this aspect of
> Fuego/Jenkins
> > integration would be lost (or you would have to recreate it).
> >
> > If FUEGO_CALLER is not set, then ftc and the core test system will just
> output
> > the test execution to stdout, and deposit test artifacts in the
> fuego-rw/logs
> > directory.  Build artifacts (e.g. the test program source and build
> directories
> > are under fuego-rw/buildzone).
> >
> > See 'ftc run-test help' for information about the arguments used in the
> above
> > example ftc invocation. Not all of these arguments are required (in
> fact, most
> > of those will default to the values shown above, but they are placed in
> the Jenkins
> > job definition to allow users to more easily remember and edit them for
> a particular job.)
> >
> > If I had a BeagleBone Black board in my lab, named 'bbb', I could execute
> > a test at the command line with: "ftc run-test -b bbb -t hello_world"
> >
> > By default, visualization of Fuego results (tables and plots) are
> prepared
> > for use in the integrated Jenkins interface.  However, you can also
> generate reports
> > in various formats, that might be useful with your Jenkins (or whatever
> other CI
> > system one might use).
> >
> > Look at the options for 'ftc gen-report', particularly the 'txt',
> 'html', or 'csv' output
> > formats, which might be import-able into your CI system, for generating
> your
> > own results data displays.
> >
> > The graphical results (measure plots) for Benchmark data is generated in
> a json
> > format which is interpreted by a javascript module called 'mod.js',
> which uses
> > a graphing module called 'flot' that is installed as a Jenkins plugin.
> 'flot' and 'mod.js'
> > might be usable with other CI systems that have a web-based user
> interface
> > (as they are just javascript (well, jquery)) but it would take some work
> to perform
> > that integration.
> >
> > It sounds like part of your question is about how the host/target
> relationship is
> > handled in Fuego. By default, a Fuego test runs on a host, and the test
> programs
> > are actually transferred to the device under test and executed there,
> during test
> > execution.  However, some Fuego users install the Fuego core system on
> the actual
> > device under test, and execute it locally.  In this case, the remote
> aspect of the
> > test would be handled by the calling CI system, not by Fuego.
> >
> > Normally, with the integrated version of Jenkins (the one in the Fuego
> docker container),
> > there is a slave used that is assigned to each node (board).  This ends
> up serializing
> > the access to that board.  But this slave runs on the host machine,
> where Jenkins
> > and Fuego are installed.  Are you asking about executing a Jenkins slave
> on a remote
> > machine (perhaps the device under test), and having Fuego installed
> there?
> >
> > Hopefully the information above  is helpful.
> >
> > I'm not sure exactly what you are asking, but please let me know if you
> need more
> > information.  I am happy to discuss your configuration, and see if Fuego
> would work
> > for you.
> >  -- Tim
> >
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
>

[-- Attachment #2: Type: text/html, Size: 8727 bytes --]

Re: [Fuego] How run Fuego tools on a Jenkins slave node

[Tim.Bird]

> -----Original Message-----
> From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> 
> Hi
> 
> 
> On Fri, Aug 27, 2021, 6:17 PM <Tim.Bird@sony.com <mailto:Tim.Bird@sony.com> > wrote:
> 
> 
> 	I just realized you might be asking about how to make the Fuego Jenkins instance a slave
> 	to your already-existing Jenkins instance master.  I have not done a configuration of
> 	Jenkins master/slave like this, but here are some instructions for doing this.
> 	https://baykara.medium.com/part-ii-add-a-jenkins-slave-to-jenkins-master-857ef055122d
> 
> 
> When I read the code the FTC command run only locally.

The default configuration of ftc is that it run on the host where Fuego is installed, and 
the test programs (that it builds from source), run on the device under test, which
is usually a remote board accessed via ssh.

> Create pipeline using FTC is not possible.
I don't understand why a pipeline can't use ftc.  It's just a Linux command line
tool, like any other. What is the operating system of your Jenkins master?

>What i would like to have is the FTC script
> on master and the executor on slave.
When you say "slave" are you referring to a Jenkins process, or the device under test.
ftc already executes it's tests on the device under test, which is usually a separate
machine from where Fuego and ftc are installed.

> 
> Do I need to split the logic?
I think the logic is already split, but I'm still not sure I understand your use case.

Please see the architecture diagram on this page:
http://fuegotest.org/wiki/Architecture

I guess my question is: Where is your Jenkins master running?  It is on the same machine
as the Fuego installation?

> 
> I have already a complex multislave configuration but i have a lot of hardware and slave but i decided to go to Fuego as regression testing
> machine

ftc doesn't talk to a Jenkins slave.  It is usually executed by a Jenkins slave.
However, in the default configuration of the instance of Jenkins integrated
into the docker container, the slaves are processes that run on the same
host as the Jenkins master process.  There is one slave (aka 'node') allocated
for each board installed in the system.  This is done with 'ftc add-node'.
When the master executes a job, it is assigned to the slave/node that
corresponds to the board related to that node.  That slave runs ftc, and

ftc (usually) uses ssh to directly access and manage the device under test
(the 'target board' in the Architecture Diagram).

I'm afraid I still don't understand the configuration you'd like to use.

 -- Tim

> 
> 	> -----Original Message-----
> 	> From: Fuego <fuego-bounces@lists.linuxfoundation.org <mailto:fuego-bounces@lists.linuxfoundation.org> > On Behalf Of
> Tim.Bird@sony.com <mailto:Tim.Bird@sony.com>
> 	>
> 	> > -----Original Message-----
> 	> > From: Fuego <fuego-bounces@lists.linuxfoundation.org <mailto:fuego-bounces@lists.linuxfoundation.org> > On Behalf Of
> Michael Nazzareno Trimarchi
> 	> >
> 	> > Hi
> 	> >
> 	> > I have already a ci setup but i would like to use Fuego and run script on slave and get results on master node. Is there a way to
> configure
> 	> it?
> 	>
> 	> I believe so.  Other Fuego users have used Fuego with CI systems other than the
> 	> default one included (the Jenkins instance inside the Fuego docker container).
> 	>
> 	> You can perform tests using Fuego, using the 'ftc' command line tool.
> 	>
> 	> The default installation of Fuego creates a docker container where Fuego core
> 	> scripts and Jenkins are installed.  You can run 'ftc' from inside this container
> 	> (which is where the included Jenkins will execute test jobs from), or from outside
> 	> the container, from the command line.
> 	>
> 	> When you install Fuego, you have the option to build the container without Jenkins
> 	> (so it just consist of the build environment for the tests, and the Fuego core scripts).
> 	> See http://fuegotest.org/wiki/Installing_Fuego#without_Jenkins for a description
> 	> of the install option used for this.
> 	>
> 	> By way of example, when Fuego is run with Jenkins, it creates Jenkins jobs with
> 	> the following command sequence for the "build" attribute (this example is for
> 	> the test Functional.hello_world):
> 	>
> 	>
> 	> #logging areas=pre_test,pre_check,build,makepkg,deploy,snapshot,run,
> 	> #    post_test,processing, parser,criteria,charting
> 	> #logging levels=debug,verbose,info,warning,error
> 	> #export FUEGO_LOGLEVELS="run:debug,parser:verbose"
> 	> export FUEGO_CALLER="jenkins"
> 	> ftc run-test -b $NODE_NAME -t Functional.hello_world -s default \
> 	>     --timeout 6m \
> 	>     --reboot false \
> 	>     --rebuild false \
> 	>     --precleanup true \
> 	>     --postcleanup true
> 	>
> 	> If the environment variable FUEGO_CALLER is set to "jenkins", then ftc and the
> 	> core test system perform additional steps with the integrated
> 	> Jenkins system (such as augmenting the Jenkins build with pointers to Fuego
> 	> test artifacts, so they are accessible in the Jenkins interface).  However, if you
> 	> are running Jenkins somewhere else (not in the
> 	> Fuego container), then I think the jobs and build data will not be in the
> 	> expected directories for Fuego to operate on.  So this aspect of Fuego/Jenkins
> 	> integration would be lost (or you would have to recreate it).
> 	>
> 	> If FUEGO_CALLER is not set, then ftc and the core test system will just output
> 	> the test execution to stdout, and deposit test artifacts in the fuego-rw/logs
> 	> directory.  Build artifacts (e.g. the test program source and build directories
> 	> are under fuego-rw/buildzone).
> 	>
> 	> See 'ftc run-test help' for information about the arguments used in the above
> 	> example ftc invocation. Not all of these arguments are required (in fact, most
> 	> of those will default to the values shown above, but they are placed in the Jenkins
> 	> job definition to allow users to more easily remember and edit them for a particular job.)
> 	>
> 	> If I had a BeagleBone Black board in my lab, named 'bbb', I could execute
> 	> a test at the command line with: "ftc run-test -b bbb -t hello_world"
> 	>
> 	> By default, visualization of Fuego results (tables and plots) are prepared
> 	> for use in the integrated Jenkins interface.  However, you can also generate reports
> 	> in various formats, that might be useful with your Jenkins (or whatever other CI
> 	> system one might use).
> 	>
> 	> Look at the options for 'ftc gen-report', particularly the 'txt', 'html', or 'csv' output
> 	> formats, which might be import-able into your CI system, for generating your
> 	> own results data displays.
> 	>
> 	> The graphical results (measure plots) for Benchmark data is generated in a json
> 	> format which is interpreted by a javascript module called 'mod.js', which uses
> 	> a graphing module called 'flot' that is installed as a Jenkins plugin.  'flot' and 'mod.js'
> 	> might be usable with other CI systems that have a web-based user interface
> 	> (as they are just javascript (well, jquery)) but it would take some work to perform
> 	> that integration.
> 	>
> 	> It sounds like part of your question is about how the host/target relationship is
> 	> handled in Fuego. By default, a Fuego test runs on a host, and the test programs
> 	> are actually transferred to the device under test and executed there, during test
> 	> execution.  However, some Fuego users install the Fuego core system on the actual
> 	> device under test, and execute it locally.  In this case, the remote aspect of the
> 	> test would be handled by the calling CI system, not by Fuego.
> 	>
> 	> Normally, with the integrated version of Jenkins (the one in the Fuego docker container),
> 	> there is a slave used that is assigned to each node (board).  This ends up serializing
> 	> the access to that board.  But this slave runs on the host machine, where Jenkins
> 	> and Fuego are installed.  Are you asking about executing a Jenkins slave on a remote
> 	> machine (perhaps the device under test), and having Fuego installed there?
> 	>
> 	> Hopefully the information above  is helpful.
> 	>
> 	> I'm not sure exactly what you are asking, but please let me know if you need more
> 	> information.  I am happy to discuss your configuration, and see if Fuego would work
> 	> for you.
> 	>  -- Tim
> 	>
> 	>
> 	> _______________________________________________
> 	> Fuego mailing list
> 	> Fuego@lists.linuxfoundation.org <mailto:Fuego@lists.linuxfoundation.org>
> 	> https://lists.linuxfoundation.org/mailman/listinfo/fuego
> 

Re: [Fuego] How run Fuego tools on a Jenkins slave node

[Michael Nazzareno Trimarchi]

Hi

On Fri, Aug 27, 2021 at 11:10 PM <Tim.Bird@sony.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> >
> > Hi
> >
> >
> > On Fri, Aug 27, 2021, 6:17 PM <Tim.Bird@sony.com <mailto:Tim.Bird@sony.com> > wrote:
> >
> >
> >       I just realized you might be asking about how to make the Fuego Jenkins instance a slave
> >       to your already-existing Jenkins instance master.  I have not done a configuration of
> >       Jenkins master/slave like this, but here are some instructions for doing this.
> >       https://baykara.medium.com/part-ii-add-a-jenkins-slave-to-jenkins-master-857ef055122d
> >
> >
> > When I read the code the FTC command run only locally.
>
> The default configuration of ftc is that it run on the host where Fuego is installed, and
> the test programs (that it builds from source), run on the device under test, which
> is usually a remote board accessed via ssh.
>
> > Create pipeline using FTC is not possible.
> I don't understand why a pipeline can't use ftc.  It's just a Linux command line
> tool, like any other. What is the operating system of your Jenkins master?
>
> >What i would like to have is the FTC script
> > on master and the executor on slave.
> When you say "slave" are you referring to a Jenkins process, or the device under test.
> ftc already executes it's tests on the device under test, which is usually a separate
> machine from where Fuego and ftc are installed.
>
> >
> > Do I need to split the logic?
> I think the logic is already split, but I'm still not sure I understand your use case.
>
> Please see the architecture diagram on this page:
> http://fuegotest.org/wiki/Architecture
>
> I guess my question is: Where is your Jenkins master running?  It is on the same machine
> as the Fuego installation?
>
> >
> > I have already a complex multislave configuration but i have a lot of hardware and slave but i decided to go to Fuego as regression testing
> > machine
>
> ftc doesn't talk to a Jenkins slave.  It is usually executed by a Jenkins slave.
> However, in the default configuration of the instance of Jenkins integrated
> into the docker container, the slaves are processes that run on the same
> host as the Jenkins master process.  There is one slave (aka 'node') allocated
> for each board installed in the system.  This is done with 'ftc add-node'.
> When the master executes a job, it is assigned to the slave/node that
> corresponds to the board related to that node.  That slave runs ftc, and
>
> ftc (usually) uses ssh to directly access and manage the device under test
> (the 'target board' in the Architecture Diagram).

This is what I understand. I can build a docker image just using the fuego.
From a jenkins pipeline I can use docker.inside("fuego") docker image.
From fuego docker I can launch tools from the slave to the hardware
connected to the slave
that can offer me ssh or any kind of connection. I notice that you
have a slave.jar
in your bitbucket. What is it?

jenkins_master\->slave1 -> hw1,hw2
                        \->slave2 -> hw3

Michael


>
> I'm afraid I still don't understand the configuration you'd like to use.
>
>  -- Tim
>
> >
> >       > -----Original Message-----
> >       > From: Fuego <fuego-bounces@lists.linuxfoundation.org <mailto:fuego-bounces@lists.linuxfoundation.org> > On Behalf Of
> > Tim.Bird@sony.com <mailto:Tim.Bird@sony.com>
> >       >
> >       > > -----Original Message-----
> >       > > From: Fuego <fuego-bounces@lists.linuxfoundation.org <mailto:fuego-bounces@lists.linuxfoundation.org> > On Behalf Of
> > Michael Nazzareno Trimarchi
> >       > >
> >       > > Hi
> >       > >
> >       > > I have already a ci setup but i would like to use Fuego and run script on slave and get results on master node. Is there a way to
> > configure
> >       > it?
> >       >
> >       > I believe so.  Other Fuego users have used Fuego with CI systems other than the
> >       > default one included (the Jenkins instance inside the Fuego docker container).
> >       >
> >       > You can perform tests using Fuego, using the 'ftc' command line tool.
> >       >
> >       > The default installation of Fuego creates a docker container where Fuego core
> >       > scripts and Jenkins are installed.  You can run 'ftc' from inside this container
> >       > (which is where the included Jenkins will execute test jobs from), or from outside
> >       > the container, from the command line.
> >       >
> >       > When you install Fuego, you have the option to build the container without Jenkins
> >       > (so it just consist of the build environment for the tests, and the Fuego core scripts).
> >       > See http://fuegotest.org/wiki/Installing_Fuego#without_Jenkins for a description
> >       > of the install option used for this.
> >       >
> >       > By way of example, when Fuego is run with Jenkins, it creates Jenkins jobs with
> >       > the following command sequence for the "build" attribute (this example is for
> >       > the test Functional.hello_world):
> >       >
> >       >
> >       > #logging areas=pre_test,pre_check,build,makepkg,deploy,snapshot,run,
> >       > #    post_test,processing, parser,criteria,charting
> >       > #logging levels=debug,verbose,info,warning,error
> >       > #export FUEGO_LOGLEVELS="run:debug,parser:verbose"
> >       > export FUEGO_CALLER="jenkins"
> >       > ftc run-test -b $NODE_NAME -t Functional.hello_world -s default \
> >       >     --timeout 6m \
> >       >     --reboot false \
> >       >     --rebuild false \
> >       >     --precleanup true \
> >       >     --postcleanup true
> >       >
> >       > If the environment variable FUEGO_CALLER is set to "jenkins", then ftc and the
> >       > core test system perform additional steps with the integrated
> >       > Jenkins system (such as augmenting the Jenkins build with pointers to Fuego
> >       > test artifacts, so they are accessible in the Jenkins interface).  However, if you
> >       > are running Jenkins somewhere else (not in the
> >       > Fuego container), then I think the jobs and build data will not be in the
> >       > expected directories for Fuego to operate on.  So this aspect of Fuego/Jenkins
> >       > integration would be lost (or you would have to recreate it).
> >       >
> >       > If FUEGO_CALLER is not set, then ftc and the core test system will just output
> >       > the test execution to stdout, and deposit test artifacts in the fuego-rw/logs
> >       > directory.  Build artifacts (e.g. the test program source and build directories
> >       > are under fuego-rw/buildzone).
> >       >
> >       > See 'ftc run-test help' for information about the arguments used in the above
> >       > example ftc invocation. Not all of these arguments are required (in fact, most
> >       > of those will default to the values shown above, but they are placed in the Jenkins
> >       > job definition to allow users to more easily remember and edit them for a particular job.)
> >       >
> >       > If I had a BeagleBone Black board in my lab, named 'bbb', I could execute
> >       > a test at the command line with: "ftc run-test -b bbb -t hello_world"
> >       >
> >       > By default, visualization of Fuego results (tables and plots) are prepared
> >       > for use in the integrated Jenkins interface.  However, you can also generate reports
> >       > in various formats, that might be useful with your Jenkins (or whatever other CI
> >       > system one might use).
> >       >
> >       > Look at the options for 'ftc gen-report', particularly the 'txt', 'html', or 'csv' output
> >       > formats, which might be import-able into your CI system, for generating your
> >       > own results data displays.
> >       >
> >       > The graphical results (measure plots) for Benchmark data is generated in a json
> >       > format which is interpreted by a javascript module called 'mod.js', which uses
> >       > a graphing module called 'flot' that is installed as a Jenkins plugin.  'flot' and 'mod.js'
> >       > might be usable with other CI systems that have a web-based user interface
> >       > (as they are just javascript (well, jquery)) but it would take some work to perform
> >       > that integration.
> >       >
> >       > It sounds like part of your question is about how the host/target relationship is
> >       > handled in Fuego. By default, a Fuego test runs on a host, and the test programs
> >       > are actually transferred to the device under test and executed there, during test
> >       > execution.  However, some Fuego users install the Fuego core system on the actual
> >       > device under test, and execute it locally.  In this case, the remote aspect of the
> >       > test would be handled by the calling CI system, not by Fuego.
> >       >
> >       > Normally, with the integrated version of Jenkins (the one in the Fuego docker container),
> >       > there is a slave used that is assigned to each node (board).  This ends up serializing
> >       > the access to that board.  But this slave runs on the host machine, where Jenkins
> >       > and Fuego are installed.  Are you asking about executing a Jenkins slave on a remote
> >       > machine (perhaps the device under test), and having Fuego installed there?
> >       >
> >       > Hopefully the information above  is helpful.
> >       >
> >       > I'm not sure exactly what you are asking, but please let me know if you need more
> >       > information.  I am happy to discuss your configuration, and see if Fuego would work
> >       > for you.
> >       >  -- Tim
> >       >
> >       >
> >       > _______________________________________________
> >       > Fuego mailing list
> >       > Fuego@lists.linuxfoundation.org <mailto:Fuego@lists.linuxfoundation.org>
> >       > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> >
>


-- 
Michael Nazzareno Trimarchi
Co-Founder & Chief Executive Officer
M. +39 347 913 2170
michael@amarulasolutions.com
__________________________________

Amarula Solutions BV
Joop Geesinkweg 125, 1114 AB, Amsterdam, NL
T. +31 (0)85 111 9172
info@amarulasolutions.com
www.amarulasolutions.com

Re: [Fuego] How run Fuego tools on a Jenkins slave node

[Tim.Bird]

> -----Original Message-----
> From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> 
> Hi
> 
> On Fri, Aug 27, 2021 at 11:10 PM <Tim.Bird@sony.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > >
> > > Hi
> > >
> > >
> > > On Fri, Aug 27, 2021, 6:17 PM <Tim.Bird@sony.com <mailto:Tim.Bird@sony.com> > wrote:
> > >
> > >
> > >       I just realized you might be asking about how to make the Fuego Jenkins instance a slave
> > >       to your already-existing Jenkins instance master.  I have not done a configuration of
> > >       Jenkins master/slave like this, but here are some instructions for doing this.
> > >       https://baykara.medium.com/part-ii-add-a-jenkins-slave-to-jenkins-master-857ef055122d
> > >
> > >
> > > When I read the code the FTC command run only locally.
> >
> > The default configuration of ftc is that it run on the host where Fuego is installed, and
> > the test programs (that it builds from source), run on the device under test, which
> > is usually a remote board accessed via ssh.
> >
> > > Create pipeline using FTC is not possible.
> > I don't understand why a pipeline can't use ftc.  It's just a Linux command line
> > tool, like any other. What is the operating system of your Jenkins master?
> >
> > >What i would like to have is the FTC script
> > > on master and the executor on slave.
> > When you say "slave" are you referring to a Jenkins process, or the device under test.
> > ftc already executes it's tests on the device under test, which is usually a separate
> > machine from where Fuego and ftc are installed.
> >
> > >
> > > Do I need to split the logic?
> > I think the logic is already split, but I'm still not sure I understand your use case.
> >
> > Please see the architecture diagram on this page:
> > http://fuegotest.org/wiki/Architecture
> >
> > I guess my question is: Where is your Jenkins master running?  It is on the same machine
> > as the Fuego installation?
> >
> > >
> > > I have already a complex multislave configuration but i have a lot of hardware and slave but i decided to go to Fuego as regression
> testing
> > > machine
> >
> > ftc doesn't talk to a Jenkins slave.  It is usually executed by a Jenkins slave.
> > However, in the default configuration of the instance of Jenkins integrated
> > into the docker container, the slaves are processes that run on the same
> > host as the Jenkins master process.  There is one slave (aka 'node') allocated
> > for each board installed in the system.  This is done with 'ftc add-node'.
> > When the master executes a job, it is assigned to the slave/node that
> > corresponds to the board related to that node.  That slave runs ftc, and
> >
> > ftc (usually) uses ssh to directly access and manage the device under test
> > (the 'target board' in the Architecture Diagram).
> 
> This is what I understand. I can build a docker image just using the fuego.
> From a jenkins pipeline I can use docker.inside("fuego") docker image.
> From fuego docker I can launch tools from the slave to the hardware
> connected to the slave
> that can offer me ssh or any kind of connection. I notice that you
> have a slave.jar
> in your bitbucket. What is it?

This is used by ftc to issues CLI commands to the Fuego instance
of Jenkins.   Previously, it was used much more than it is currently.
A few years ago we added support in ftc for using the 'jenkins' module.
Now, I believe the only operation in 'ftc' that uses this is add-nodes,
which adds a Fuego board as a node in the Jenkins system. 

> 
> jenkins_master\->slave1 -> hw1,hw2
>                         \->slave2 -> hw3

Yes, but I would add a bit more detail here, for clarity:

jenkins_master\->slave1\-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw1
                                         \-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw2
                          \-> slave2-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw3

90% of the time, FUEGO_TRANSPORT is ssh, but it could be some other protocol
for executing commands on the board and transferring files (like 'serial', or 'ebf')

Note that the call to 'docker ftc' is transparent.  On a system where Fuego is installed,
you can just run 'ftc', and it figures our whether to run natively (if installed natively)
or by doing a second invocation inside the container (if installed with a container).

 -- Tim
 
> >
> > >
> > >       > -----Original Message-----
> > >       > From: Fuego <fuego-bounces@lists.linuxfoundation.org <mailto:fuego-bounces@lists.linuxfoundation.org> > On Behalf Of
> > > Tim.Bird@sony.com <mailto:Tim.Bird@sony.com>
> > >       >
> > >       > > -----Original Message-----
> > >       > > From: Fuego <fuego-bounces@lists.linuxfoundation.org <mailto:fuego-bounces@lists.linuxfoundation.org> > On Behalf Of
> > > Michael Nazzareno Trimarchi
> > >       > >
> > >       > > Hi
> > >       > >
> > >       > > I have already a ci setup but i would like to use Fuego and run script on slave and get results on master node. Is there a way to
> > > configure
> > >       > it?
> > >       >
> > >       > I believe so.  Other Fuego users have used Fuego with CI systems other than the
> > >       > default one included (the Jenkins instance inside the Fuego docker container).
> > >       >
> > >       > You can perform tests using Fuego, using the 'ftc' command line tool.
> > >       >
> > >       > The default installation of Fuego creates a docker container where Fuego core
> > >       > scripts and Jenkins are installed.  You can run 'ftc' from inside this container
> > >       > (which is where the included Jenkins will execute test jobs from), or from outside
> > >       > the container, from the command line.
> > >       >
> > >       > When you install Fuego, you have the option to build the container without Jenkins
> > >       > (so it just consist of the build environment for the tests, and the Fuego core scripts).
> > >       > See http://fuegotest.org/wiki/Installing_Fuego#without_Jenkins for a description
> > >       > of the install option used for this.
> > >       >
> > >       > By way of example, when Fuego is run with Jenkins, it creates Jenkins jobs with
> > >       > the following command sequence for the "build" attribute (this example is for
> > >       > the test Functional.hello_world):
> > >       >
> > >       >
> > >       > #logging areas=pre_test,pre_check,build,makepkg,deploy,snapshot,run,
> > >       > #    post_test,processing, parser,criteria,charting
> > >       > #logging levels=debug,verbose,info,warning,error
> > >       > #export FUEGO_LOGLEVELS="run:debug,parser:verbose"
> > >       > export FUEGO_CALLER="jenkins"
> > >       > ftc run-test -b $NODE_NAME -t Functional.hello_world -s default \
> > >       >     --timeout 6m \
> > >       >     --reboot false \
> > >       >     --rebuild false \
> > >       >     --precleanup true \
> > >       >     --postcleanup true
> > >       >
> > >       > If the environment variable FUEGO_CALLER is set to "jenkins", then ftc and the
> > >       > core test system perform additional steps with the integrated
> > >       > Jenkins system (such as augmenting the Jenkins build with pointers to Fuego
> > >       > test artifacts, so they are accessible in the Jenkins interface).  However, if you
> > >       > are running Jenkins somewhere else (not in the
> > >       > Fuego container), then I think the jobs and build data will not be in the
> > >       > expected directories for Fuego to operate on.  So this aspect of Fuego/Jenkins
> > >       > integration would be lost (or you would have to recreate it).
> > >       >
> > >       > If FUEGO_CALLER is not set, then ftc and the core test system will just output
> > >       > the test execution to stdout, and deposit test artifacts in the fuego-rw/logs
> > >       > directory.  Build artifacts (e.g. the test program source and build directories
> > >       > are under fuego-rw/buildzone).
> > >       >
> > >       > See 'ftc run-test help' for information about the arguments used in the above
> > >       > example ftc invocation. Not all of these arguments are required (in fact, most
> > >       > of those will default to the values shown above, but they are placed in the Jenkins
> > >       > job definition to allow users to more easily remember and edit them for a particular job.)
> > >       >
> > >       > If I had a BeagleBone Black board in my lab, named 'bbb', I could execute
> > >       > a test at the command line with: "ftc run-test -b bbb -t hello_world"
> > >       >
> > >       > By default, visualization of Fuego results (tables and plots) are prepared
> > >       > for use in the integrated Jenkins interface.  However, you can also generate reports
> > >       > in various formats, that might be useful with your Jenkins (or whatever other CI
> > >       > system one might use).
> > >       >
> > >       > Look at the options for 'ftc gen-report', particularly the 'txt', 'html', or 'csv' output
> > >       > formats, which might be import-able into your CI system, for generating your
> > >       > own results data displays.
> > >       >
> > >       > The graphical results (measure plots) for Benchmark data is generated in a json
> > >       > format which is interpreted by a javascript module called 'mod.js', which uses
> > >       > a graphing module called 'flot' that is installed as a Jenkins plugin.  'flot' and 'mod.js'
> > >       > might be usable with other CI systems that have a web-based user interface
> > >       > (as they are just javascript (well, jquery)) but it would take some work to perform
> > >       > that integration.
> > >       >
> > >       > It sounds like part of your question is about how the host/target relationship is
> > >       > handled in Fuego. By default, a Fuego test runs on a host, and the test programs
> > >       > are actually transferred to the device under test and executed there, during test
> > >       > execution.  However, some Fuego users install the Fuego core system on the actual
> > >       > device under test, and execute it locally.  In this case, the remote aspect of the
> > >       > test would be handled by the calling CI system, not by Fuego.
> > >       >
> > >       > Normally, with the integrated version of Jenkins (the one in the Fuego docker container),
> > >       > there is a slave used that is assigned to each node (board).  This ends up serializing
> > >       > the access to that board.  But this slave runs on the host machine, where Jenkins
> > >       > and Fuego are installed.  Are you asking about executing a Jenkins slave on a remote
> > >       > machine (perhaps the device under test), and having Fuego installed there?
> > >       >
> > >       > Hopefully the information above  is helpful.
> > >       >
> > >       > I'm not sure exactly what you are asking, but please let me know if you need more
> > >       > information.  I am happy to discuss your configuration, and see if Fuego would work
> > >       > for you.
> > >       >  -- Tim
> > >       >
> > >       >
> > >       > _______________________________________________
> > >       > Fuego mailing list
> > >       > Fuego@lists.linuxfoundation.org <mailto:Fuego@lists.linuxfoundation.org>
> > >       > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > >
> >
> 
> 
> --
> Michael Nazzareno Trimarchi
> Co-Founder & Chief Executive Officer
> M. +39 347 913 2170
> michael@amarulasolutions.com
> __________________________________
> 
> Amarula Solutions BV
> Joop Geesinkweg 125, 1114 AB, Amsterdam, NL
> T. +31 (0)85 111 9172
> info@amarulasolutions.com
> www.amarulasolutions.com

Re: [Fuego] How run Fuego tools on a Jenkins slave node

[Michael Nazzareno Trimarchi]

Hi

On Mon, Aug 30, 2021 at 6:35 PM <Tim.Bird@sony.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> >
> > Hi
> >
> > On Fri, Aug 27, 2021 at 11:10 PM <Tim.Bird@sony.com> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > > >
> > > > Hi
> > > >
> > > >
> > > > On Fri, Aug 27, 2021, 6:17 PM <Tim.Bird@sony.com <mailto:Tim.Bird@sony.com> > wrote:
> > > >
> > > >
> > > >       I just realized you might be asking about how to make the Fuego Jenkins instance a slave
> > > >       to your already-existing Jenkins instance master.  I have not done a configuration of
> > > >       Jenkins master/slave like this, but here are some instructions for doing this.
> > > >       https://baykara.medium.com/part-ii-add-a-jenkins-slave-to-jenkins-master-857ef055122d
> > > >
> > > >
> > > > When I read the code the FTC command run only locally.
> > >
> > > The default configuration of ftc is that it run on the host where Fuego is installed, and
> > > the test programs (that it builds from source), run on the device under test, which
> > > is usually a remote board accessed via ssh.
> > >
> > > > Create pipeline using FTC is not possible.
> > > I don't understand why a pipeline can't use ftc.  It's just a Linux command line
> > > tool, like any other. What is the operating system of your Jenkins master?
> > >
> > > >What i would like to have is the FTC script
> > > > on master and the executor on slave.
> > > When you say "slave" are you referring to a Jenkins process, or the device under test.
> > > ftc already executes it's tests on the device under test, which is usually a separate
> > > machine from where Fuego and ftc are installed.
> > >
> > > >
> > > > Do I need to split the logic?
> > > I think the logic is already split, but I'm still not sure I understand your use case.
> > >
> > > Please see the architecture diagram on this page:
> > > http://fuegotest.org/wiki/Architecture
> > >
> > > I guess my question is: Where is your Jenkins master running?  It is on the same machine
> > > as the Fuego installation?
> > >
> > > >
> > > > I have already a complex multislave configuration but i have a lot of hardware and slave but i decided to go to Fuego as regression
> > testing
> > > > machine
> > >
> > > ftc doesn't talk to a Jenkins slave.  It is usually executed by a Jenkins slave.
> > > However, in the default configuration of the instance of Jenkins integrated
> > > into the docker container, the slaves are processes that run on the same
> > > host as the Jenkins master process.  There is one slave (aka 'node') allocated
> > > for each board installed in the system.  This is done with 'ftc add-node'.
> > > When the master executes a job, it is assigned to the slave/node that
> > > corresponds to the board related to that node.  That slave runs ftc, and
> > >
> > > ftc (usually) uses ssh to directly access and manage the device under test
> > > (the 'target board' in the Architecture Diagram).
> >
> > This is what I understand. I can build a docker image just using the fuego.
> > From a jenkins pipeline I can use docker.inside("fuego") docker image.
> > From fuego docker I can launch tools from the slave to the hardware
> > connected to the slave
> > that can offer me ssh or any kind of connection. I notice that you
> > have a slave.jar
> > in your bitbucket. What is it?
>
> This is used by ftc to issues CLI commands to the Fuego instance
> of Jenkins.   Previously, it was used much more than it is currently.
> A few years ago we added support in ftc for using the 'jenkins' module.
> Now, I believe the only operation in 'ftc' that uses this is add-nodes,
> which adds a Fuego board as a node in the Jenkins system.
>
> >
> > jenkins_master\->slave1 -> hw1,hw2
> >                         \->slave2 -> hw3
>
> Yes, but I would add a bit more detail here, for clarity:
>
> jenkins_master\->slave1\-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw1
>                                          \-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw2
>                           \-> slave2-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw3
>
> 90% of the time, FUEGO_TRANSPORT is ssh, but it could be some other protocol
> for executing commands on the board and transferring files (like 'serial', or 'ebf')
>

Ok

> Note that the call to 'docker ftc' is transparent.  On a system where Fuego is installed,
> you can just run 'ftc', and it figures our whether to run natively (if installed natively)
> or by doing a second invocation inside the container (if installed with a container).

Ok, so I can clone it on my local node and use it

Michael

>
>  -- Tim
>
> > >
> > > >
> > > >       > -----Original Message-----
> > > >       > From: Fuego <fuego-bounces@lists.linuxfoundation.org <mailto:fuego-bounces@lists.linuxfoundation.org> > On Behalf Of
> > > > Tim.Bird@sony.com <mailto:Tim.Bird@sony.com>
> > > >       >
> > > >       > > -----Original Message-----
> > > >       > > From: Fuego <fuego-bounces@lists.linuxfoundation.org <mailto:fuego-bounces@lists.linuxfoundation.org> > On Behalf Of
> > > > Michael Nazzareno Trimarchi
> > > >       > >
> > > >       > > Hi
> > > >       > >
> > > >       > > I have already a ci setup but i would like to use Fuego and run script on slave and get results on master node. Is there a way to
> > > > configure
> > > >       > it?
> > > >       >
> > > >       > I believe so.  Other Fuego users have used Fuego with CI systems other than the
> > > >       > default one included (the Jenkins instance inside the Fuego docker container).
> > > >       >
> > > >       > You can perform tests using Fuego, using the 'ftc' command line tool.
> > > >       >
> > > >       > The default installation of Fuego creates a docker container where Fuego core
> > > >       > scripts and Jenkins are installed.  You can run 'ftc' from inside this container
> > > >       > (which is where the included Jenkins will execute test jobs from), or from outside
> > > >       > the container, from the command line.
> > > >       >
> > > >       > When you install Fuego, you have the option to build the container without Jenkins
> > > >       > (so it just consist of the build environment for the tests, and the Fuego core scripts).
> > > >       > See http://fuegotest.org/wiki/Installing_Fuego#without_Jenkins for a description
> > > >       > of the install option used for this.
> > > >       >
> > > >       > By way of example, when Fuego is run with Jenkins, it creates Jenkins jobs with
> > > >       > the following command sequence for the "build" attribute (this example is for
> > > >       > the test Functional.hello_world):
> > > >       >
> > > >       >
> > > >       > #logging areas=pre_test,pre_check,build,makepkg,deploy,snapshot,run,
> > > >       > #    post_test,processing, parser,criteria,charting
> > > >       > #logging levels=debug,verbose,info,warning,error
> > > >       > #export FUEGO_LOGLEVELS="run:debug,parser:verbose"
> > > >       > export FUEGO_CALLER="jenkins"
> > > >       > ftc run-test -b $NODE_NAME -t Functional.hello_world -s default \
> > > >       >     --timeout 6m \
> > > >       >     --reboot false \
> > > >       >     --rebuild false \
> > > >       >     --precleanup true \
> > > >       >     --postcleanup true
> > > >       >
> > > >       > If the environment variable FUEGO_CALLER is set to "jenkins", then ftc and the
> > > >       > core test system perform additional steps with the integrated
> > > >       > Jenkins system (such as augmenting the Jenkins build with pointers to Fuego
> > > >       > test artifacts, so they are accessible in the Jenkins interface).  However, if you
> > > >       > are running Jenkins somewhere else (not in the
> > > >       > Fuego container), then I think the jobs and build data will not be in the
> > > >       > expected directories for Fuego to operate on.  So this aspect of Fuego/Jenkins
> > > >       > integration would be lost (or you would have to recreate it).
> > > >       >
> > > >       > If FUEGO_CALLER is not set, then ftc and the core test system will just output
> > > >       > the test execution to stdout, and deposit test artifacts in the fuego-rw/logs
> > > >       > directory.  Build artifacts (e.g. the test program source and build directories
> > > >       > are under fuego-rw/buildzone).
> > > >       >
> > > >       > See 'ftc run-test help' for information about the arguments used in the above
> > > >       > example ftc invocation. Not all of these arguments are required (in fact, most
> > > >       > of those will default to the values shown above, but they are placed in the Jenkins
> > > >       > job definition to allow users to more easily remember and edit them for a particular job.)
> > > >       >
> > > >       > If I had a BeagleBone Black board in my lab, named 'bbb', I could execute
> > > >       > a test at the command line with: "ftc run-test -b bbb -t hello_world"
> > > >       >
> > > >       > By default, visualization of Fuego results (tables and plots) are prepared
> > > >       > for use in the integrated Jenkins interface.  However, you can also generate reports
> > > >       > in various formats, that might be useful with your Jenkins (or whatever other CI
> > > >       > system one might use).
> > > >       >
> > > >       > Look at the options for 'ftc gen-report', particularly the 'txt', 'html', or 'csv' output
> > > >       > formats, which might be import-able into your CI system, for generating your
> > > >       > own results data displays.
> > > >       >
> > > >       > The graphical results (measure plots) for Benchmark data is generated in a json
> > > >       > format which is interpreted by a javascript module called 'mod.js', which uses
> > > >       > a graphing module called 'flot' that is installed as a Jenkins plugin.  'flot' and 'mod.js'
> > > >       > might be usable with other CI systems that have a web-based user interface
> > > >       > (as they are just javascript (well, jquery)) but it would take some work to perform
> > > >       > that integration.
> > > >       >
> > > >       > It sounds like part of your question is about how the host/target relationship is
> > > >       > handled in Fuego. By default, a Fuego test runs on a host, and the test programs
> > > >       > are actually transferred to the device under test and executed there, during test
> > > >       > execution.  However, some Fuego users install the Fuego core system on the actual
> > > >       > device under test, and execute it locally.  In this case, the remote aspect of the
> > > >       > test would be handled by the calling CI system, not by Fuego.
> > > >       >
> > > >       > Normally, with the integrated version of Jenkins (the one in the Fuego docker container),
> > > >       > there is a slave used that is assigned to each node (board).  This ends up serializing
> > > >       > the access to that board.  But this slave runs on the host machine, where Jenkins
> > > >       > and Fuego are installed.  Are you asking about executing a Jenkins slave on a remote
> > > >       > machine (perhaps the device under test), and having Fuego installed there?
> > > >       >
> > > >       > Hopefully the information above  is helpful.
> > > >       >
> > > >       > I'm not sure exactly what you are asking, but please let me know if you need more
> > > >       > information.  I am happy to discuss your configuration, and see if Fuego would work
> > > >       > for you.
> > > >       >  -- Tim
> > > >       >
> > > >       >
> > > >       > _______________________________________________
> > > >       > Fuego mailing list
> > > >       > Fuego@lists.linuxfoundation.org <mailto:Fuego@lists.linuxfoundation.org>
> > > >       > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > > >
> > >
> >
> >
> > --
> > Michael Nazzareno Trimarchi
> > Co-Founder & Chief Executive Officer
> > M. +39 347 913 2170
> > michael@amarulasolutions.com
> > __________________________________
> >
> > Amarula Solutions BV
> > Joop Geesinkweg 125, 1114 AB, Amsterdam, NL
> > T. +31 (0)85 111 9172
> > info@amarulasolutions.com
> > www.amarulasolutions.com



-- 
Michael Nazzareno Trimarchi
Co-Founder & Chief Executive Officer
M. +39 347 913 2170
michael@amarulasolutions.com
__________________________________

Amarula Solutions BV
Joop Geesinkweg 125, 1114 AB, Amsterdam, NL
T. +31 (0)85 111 9172
info@amarulasolutions.com
www.amarulasolutions.com

Re: [Fuego] How run Fuego tools on a Jenkins slave node

[Tim.Bird]

> -----Original Message-----
> From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> 
> Hi
> 
> On Mon, Aug 30, 2021 at 6:35 PM <Tim.Bird@sony.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > >
> > > Hi
> > >
> > > On Fri, Aug 27, 2021 at 11:10 PM <Tim.Bird@sony.com> wrote:
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > > > >
> > > > > Hi
> > > > >
> > > > >
> > > > > On Fri, Aug 27, 2021, 6:17 PM <Tim.Bird@sony.com <mailto:Tim.Bird@sony.com> > wrote:
[...snipping a lot...]
> >
> > >
> > > jenkins_master\->slave1 -> hw1,hw2
> > >                         \->slave2 -> hw3
> >
> > Yes, but I would add a bit more detail here, for clarity:
> >
> > jenkins_master\->slave1\-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw1
> >                                          \-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw2
> >                           \-> slave2-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw3
> >
> > 90% of the time, FUEGO_TRANSPORT is ssh, but it could be some other protocol
> > for executing commands on the board and transferring files (like 'serial', or 'ebf')
> >
> 
> Ok
> 
> > Note that the call to 'docker ftc' is transparent.  On a system where Fuego is installed,
> > you can just run 'ftc', and it figures our whether to run natively (if installed natively)
> > or by doing a second invocation inside the container (if installed with a container).
> 
> Ok, so I can clone it on my local node and use it
> 

Yes.   And if hw4 happens to be the machine where all this is running, then you
can use 'local' instead of 'ssh' as the Fuego transport for that 'board'.
 -- Tim

Re: [Fuego] How run Fuego tools on a Jenkins slave node

[Michael Nazzareno Trimarchi]

Hi

On Mon, Aug 30, 2021 at 7:54 PM <Tim.Bird@sony.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> >
> > Hi
> >
> > On Mon, Aug 30, 2021 at 6:35 PM <Tim.Bird@sony.com> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > > >
> > > > Hi
> > > >
> > > > On Fri, Aug 27, 2021 at 11:10 PM <Tim.Bird@sony.com> wrote:
> > > > >
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > > > > >
> > > > > > Hi
> > > > > >
> > > > > >
> > > > > > On Fri, Aug 27, 2021, 6:17 PM <Tim.Bird@sony.com <mailto:Tim.Bird@sony.com> > wrote:
> [...snipping a lot...]
> > >
> > > >
> > > > jenkins_master\->slave1 -> hw1,hw2
> > > >                         \->slave2 -> hw3
> > >
> > > Yes, but I would add a bit more detail here, for clarity:
> > >
> > > jenkins_master\->slave1\-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw1
> > >                                          \-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw2
> > >                           \-> slave2-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw3
> > >
> > > 90% of the time, FUEGO_TRANSPORT is ssh, but it could be some other protocol
> > > for executing commands on the board and transferring files (like 'serial', or 'ebf')
> > >
> >
> > Ok
> >
> > > Note that the call to 'docker ftc' is transparent.  On a system where Fuego is installed,
> > > you can just run 'ftc', and it figures our whether to run natively (if installed natively)
> > > or by doing a second invocation inside the container (if installed with a container).
> >
> > Ok, so I can clone it on my local node and use it
> >
>
> Yes.   And if hw4 happens to be the machine where all this is running, then you
> can use 'local' instead of 'ssh' as the Fuego transport for that 'board'.

I think that this is most of the use cases. In this cases the pipeline
result should be shown
import the html report and visualize in master. I think that this
should have a section on documentation.

I will ask someone in the company to play a bit. I'm always short in time

Michael

>  -- Tim
>


-- 
Michael Nazzareno Trimarchi
Co-Founder & Chief Executive Officer
M. +39 347 913 2170
michael@amarulasolutions.com
__________________________________

Amarula Solutions BV
Joop Geesinkweg 125, 1114 AB, Amsterdam, NL
T. +31 (0)85 111 9172
info@amarulasolutions.com
www.amarulasolutions.com

Re: [Fuego] How run Fuego tools on a Jenkins slave node

[Michael Nazzareno Trimarchi]

Hi Tim

On Mon, Aug 30, 2021 at 9:14 PM Michael Nazzareno Trimarchi
<michael@amarulasolutions.com> wrote:
>
> Hi
>
> On Mon, Aug 30, 2021 at 7:54 PM <Tim.Bird@sony.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > >
> > > Hi
> > >
> > > On Mon, Aug 30, 2021 at 6:35 PM <Tim.Bird@sony.com> wrote:
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > > > >
> > > > > Hi
> > > > >
> > > > > On Fri, Aug 27, 2021 at 11:10 PM <Tim.Bird@sony.com> wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > > > > > >
> > > > > > > Hi
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Aug 27, 2021, 6:17 PM <Tim.Bird@sony.com <mailto:Tim.Bird@sony.com> > wrote:
> > [...snipping a lot...]
> > > >
> > > > >
> > > > > jenkins_master\->slave1 -> hw1,hw2
> > > > >                         \->slave2 -> hw3
> > > >
> > > > Yes, but I would add a bit more detail here, for clarity:
> > > >
> > > > jenkins_master\->slave1\-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw1
> > > >                                          \-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw2
> > > >                           \-> slave2-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw3
> > > >
> > > > 90% of the time, FUEGO_TRANSPORT is ssh, but it could be some other protocol
> > > > for executing commands on the board and transferring files (like 'serial', or 'ebf')
> > > >
> > >
> > > Ok
> > >
> > > > Note that the call to 'docker ftc' is transparent.  On a system where Fuego is installed,
> > > > you can just run 'ftc', and it figures our whether to run natively (if installed natively)
> > > > or by doing a second invocation inside the container (if installed with a container).
> > >
> > > Ok, so I can clone it on my local node and use it
> > >
> >
> > Yes.   And if hw4 happens to be the machine where all this is running, then you
> > can use 'local' instead of 'ssh' as the Fuego transport for that 'board'.
>
> I think that this is most of the use cases. In this cases the pipeline
> result should be shown
> import the html report and visualize in master. I think that this
> should have a section on documentation.
>
> I will ask someone in the company to play a bit. I'm always short in time
>

I have started by myself. I have few questions:
sudo is required only if your are not in docker users and in general
is not needed.
I have done a patch to pass as optional when I create the container.

I have used ./instal.sh --nojenkins but the volume are not mounted
when the container
start. Do you have the same problem?

Michael

> Michael
>
> >  -- Tim
> >
>
>
> --
> Michael Nazzareno Trimarchi
> Co-Founder & Chief Executive Officer
> M. +39 347 913 2170
> michael@amarulasolutions.com
> __________________________________
>
> Amarula Solutions BV
> Joop Geesinkweg 125, 1114 AB, Amsterdam, NL
> T. +31 (0)85 111 9172
> info@amarulasolutions.com
> www.amarulasolutions.com



-- 
Michael Nazzareno Trimarchi
Co-Founder & Chief Executive Officer
M. +39 347 913 2170
michael@amarulasolutions.com
__________________________________

Amarula Solutions BV
Joop Geesinkweg 125, 1114 AB, Amsterdam, NL
T. +31 (0)85 111 9172
info@amarulasolutions.com
www.amarulasolutions.com

Re: [Fuego] How run Fuego tools on a Jenkins slave node

[Michael Nazzareno Trimarchi]

HI

On Fri, Sep 3, 2021 at 7:33 PM Michael Nazzareno Trimarchi
<michael@amarulasolutions.com> wrote:
>
> Hi Tim
>
> On Mon, Aug 30, 2021 at 9:14 PM Michael Nazzareno Trimarchi
> <michael@amarulasolutions.com> wrote:
> >
> > Hi
> >
> > On Mon, Aug 30, 2021 at 7:54 PM <Tim.Bird@sony.com> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > > >
> > > > Hi
> > > >
> > > > On Mon, Aug 30, 2021 at 6:35 PM <Tim.Bird@sony.com> wrote:
> > > > >
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > > > > >
> > > > > > Hi
> > > > > >
> > > > > > On Fri, Aug 27, 2021 at 11:10 PM <Tim.Bird@sony.com> wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > > > > > > >
> > > > > > > > Hi
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Aug 27, 2021, 6:17 PM <Tim.Bird@sony.com <mailto:Tim.Bird@sony.com> > wrote:
> > > [...snipping a lot...]
> > > > >
> > > > > >
> > > > > > jenkins_master\->slave1 -> hw1,hw2
> > > > > >                         \->slave2 -> hw3
> > > > >
> > > > > Yes, but I would add a bit more detail here, for clarity:
> > > > >
> > > > > jenkins_master\->slave1\-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw1
> > > > >                                          \-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw2
> > > > >                           \-> slave2-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw3
> > > > >
> > > > > 90% of the time, FUEGO_TRANSPORT is ssh, but it could be some other protocol
> > > > > for executing commands on the board and transferring files (like 'serial', or 'ebf')
> > > > >
> > > >
> > > > Ok
> > > >
> > > > > Note that the call to 'docker ftc' is transparent.  On a system where Fuego is installed,
> > > > > you can just run 'ftc', and it figures our whether to run natively (if installed natively)
> > > > > or by doing a second invocation inside the container (if installed with a container).
> > > >
> > > > Ok, so I can clone it on my local node and use it
> > > >
> > >
> > > Yes.   And if hw4 happens to be the machine where all this is running, then you
> > > can use 'local' instead of 'ssh' as the Fuego transport for that 'board'.
> >
> > I think that this is most of the use cases. In this cases the pipeline
> > result should be shown
> > import the html report and visualize in master. I think that this
> > should have a section on documentation.
> >
> > I will ask someone in the company to play a bit. I'm always short in time
> >
>
> I have started by myself. I have few questions:
> sudo is required only if your are not in docker users and in general
> is not needed.
> I have done a patch to pass as optional when I create the container.
>
> I have used ./instal.sh --nojenkins but the volume are not mounted
> when the container
> start. Do you have the same problem?

Solved I have some older installation when container name was in use

Michael
>
> Michael
>
> > Michael
> >
> > >  -- Tim
> > >
> >
> >
> > --
> > Michael Nazzareno Trimarchi
> > Co-Founder & Chief Executive Officer
> > M. +39 347 913 2170
> > michael@amarulasolutions.com
> > __________________________________
> >
> > Amarula Solutions BV
> > Joop Geesinkweg 125, 1114 AB, Amsterdam, NL
> > T. +31 (0)85 111 9172
> > info@amarulasolutions.com
> > www.amarulasolutions.com
>
>
>
> --
> Michael Nazzareno Trimarchi
> Co-Founder & Chief Executive Officer
> M. +39 347 913 2170
> michael@amarulasolutions.com
> __________________________________
>
> Amarula Solutions BV
> Joop Geesinkweg 125, 1114 AB, Amsterdam, NL
> T. +31 (0)85 111 9172
> info@amarulasolutions.com
> www.amarulasolutions.com



-- 
Michael Nazzareno Trimarchi
Co-Founder & Chief Executive Officer
M. +39 347 913 2170
michael@amarulasolutions.com
__________________________________

Amarula Solutions BV
Joop Geesinkweg 125, 1114 AB, Amsterdam, NL
T. +31 (0)85 111 9172
info@amarulasolutions.com
www.amarulasolutions.com

Re: [Fuego] How run Fuego tools on a Jenkins slave node

[Tim.Bird]

> -----Original Message-----
> From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> 
> HI
> 
> On Fri, Sep 3, 2021 at 7:33 PM Michael Nazzareno Trimarchi
> <michael@amarulasolutions.com> wrote:
> >
> > Hi Tim
> >
> > On Mon, Aug 30, 2021 at 9:14 PM Michael Nazzareno Trimarchi
> > <michael@amarulasolutions.com> wrote:
> > >
> > > Hi
> > >
> > > On Mon, Aug 30, 2021 at 7:54 PM <Tim.Bird@sony.com> wrote:
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > > > >
> > > > > Hi
> > > > >
> > > > > On Mon, Aug 30, 2021 at 6:35 PM <Tim.Bird@sony.com> wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > > > > > >
> > > > > > > Hi
> > > > > > >
> > > > > > > On Fri, Aug 27, 2021 at 11:10 PM <Tim.Bird@sony.com> wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: Michael Nazzareno Trimarchi <michael@amarulasolutions.com>
> > > > > > > > >
> > > > > > > > > Hi
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Aug 27, 2021, 6:17 PM <Tim.Bird@sony.com <mailto:Tim.Bird@sony.com> > wrote:
> > > > [...snipping a lot...]
> > > > > >
> > > > > > >
> > > > > > > jenkins_master\->slave1 -> hw1,hw2
> > > > > > >                         \->slave2 -> hw3
> > > > > >
> > > > > > Yes, but I would add a bit more detail here, for clarity:
> > > > > >
> > > > > > jenkins_master\->slave1\-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw1
> > > > > >                                          \-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw2
> > > > > >                           \-> slave2-> ftc -> docker ftc -> FUEGO_TRANSPORT -> hw3
> > > > > >
> > > > > > 90% of the time, FUEGO_TRANSPORT is ssh, but it could be some other protocol
> > > > > > for executing commands on the board and transferring files (like 'serial', or 'ebf')
> > > > > >
> > > > >
> > > > > Ok
> > > > >
> > > > > > Note that the call to 'docker ftc' is transparent.  On a system where Fuego is installed,
> > > > > > you can just run 'ftc', and it figures our whether to run natively (if installed natively)
> > > > > > or by doing a second invocation inside the container (if installed with a container).
> > > > >
> > > > > Ok, so I can clone it on my local node and use it
> > > > >
> > > >
> > > > Yes.   And if hw4 happens to be the machine where all this is running, then you
> > > > can use 'local' instead of 'ssh' as the Fuego transport for that 'board'.
> > >
> > > I think that this is most of the use cases. In this cases the pipeline
> > > result should be shown
> > > import the html report and visualize in master. I think that this
> > > should have a section on documentation.
> > >
> > > I will ask someone in the company to play a bit. I'm always short in time
> > >
> >
> > I have started by myself. I have few questions:
> > sudo is required only if your are not in docker users and in general
> > is not needed.
> > I have done a patch to pass as optional when I create the container.
> >
> > I have used ./instal.sh --nojenkins but the volume are not mounted
> > when the container
> > start. Do you have the same problem?
> 
> Solved I have some older installation when container name was in use

Good to hear you found the problem and were able to solve it.
 -- Tim