* [Fuego] RFC: Fuego's version up and other changes
@ 2017-01-11 0:28 Daniel Sangorrin
2017-01-12 5:51 ` Ibe.Kengo
2017-01-20 0:53 ` [Fuego] " Bird, Timothy
0 siblings, 2 replies; 43+ messages in thread
From: Daniel Sangorrin @ 2017-01-11 0:28 UTC (permalink / raw)
To: 'Bird, Timothy', fuego
Hi everyone,
I've been working on Fuego's upgrade for a few weeks, and in the process I have made quite a few changes.
You can check it by following these instructions. I'd really appreciate your feedback:
Quickstart:
$ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
$ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
$ cd fuego/
$ ./install.sh
$ ./fuego-host-scripts/docker-create-container.sh
$ ./fuego-host-scripts/docker-start-container.sh
docker# fuego-create-node --board docker
docker# fuego-create-jobs --board docker --testplan testplan_docker --distrib nosyslogd.dist
Then, start the "docker.testplan_docker.batch" job which will trigger the rest of jobs.
Here is a list of the most fundamental modifications I've added:
- I automatized the version upgrade process. So far I have updated Jenkins 3 times quite smoothly.
The current version is the latest LTS version (2.31.1 LTS)
- I reduced plugin dependencies to the minimum (only 2, and they can be made optional).
- I removed dependencies on all groovy scripts.
- Userdata is gone. Instead we now have fuego-ro and fuego-rw, which together with fuego-core
are mounted as external docker volumes. Only fuego-rw, which contains logs and build folders,
is mounted in RW mode. I also fixed the uid/gid of jenkins so that it matches the one in the host.
This means you can develop comfortably on your host, with GUI tools, and don't fear a buggy
script deleting your folders.
- I added a tool for generating jobs from testplans. Creating jobs from testplans is powerful.
It allows you, for example, to easily specify the timeout for each test on your board.
The tool also generates the corresponding batch job. For example, you can easily trigger all
tests for your board and a specific testplan periodically.
[Note] This will also enable the creation of scripts for comparing test results across
different boards and test parameters (on my TODO list).
Other less fundamental changes include:
- I added support for using the docker container itself as a target board (e.g.: for a quickstart or unit tests)
- I fixed the flot plugin (java and javascript fixes)
- Now you can click on a node and see which jobs (and testplans) are assigned to your board. This was
one of my old feature requests and makes Fuego's GUI easier to use.
- I added quite a few fixes and improvements to the core engine scripts and tests (too detailed to describe here).
- I added Excel output support for IOzone
- Faster docker build time (ARMhf compiler installation is now optional, no latex..)
- I put a fixed name to the fuego container, instead of the "last_container_id" file.
- I removed the inotify script
- I added scripts for easily removing the docker container/image.
I have to split the current "huge" patch into smaller chunks and test it further but it would
be nice to get your feedback about the approach in general.
Tim: I haven't updated ftc nor your "Christmas commits" yet.
Best regards
Daniel
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] RFC: Fuego's version up and other changes
2017-01-11 0:28 [Fuego] RFC: Fuego's version up and other changes Daniel Sangorrin
@ 2017-01-12 5:51 ` Ibe.Kengo
2017-01-12 6:44 ` Daniel Sangorrin
2017-01-20 0:53 ` [Fuego] " Bird, Timothy
1 sibling, 1 reply; 43+ messages in thread
From: Ibe.Kengo @ 2017-01-12 5:51 UTC (permalink / raw)
To: Daniel Sangorrin; +Cc: fuego
Hi, Daniel-san.
I’m very interested in your work!
So I tried to run your build scripts to create Docker image today,
then I could run new Fuego.
And, I added Docker node and test cases to use following command.
==
$ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
$ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
$ cd fuego/
$ ./install.sh
$ ./fuego-host-scripts/docker-create-container.sh
$ ./fuego-host-scripts/docker-start-container.sh
docker# fuego-create-node --board docker
docker# fuego-create-jobs --board docker --testplan testplan_docker
--distrib nosyslogd.dist
==
I can see new node and new test jobs.
But, I can’t run tests like Dhrystone on Docker.
So far, I had one small problem that is “Failed to mkdirs: /fuego-rw/
buildzone”
This reason is the permission or owner of “fuego-rw”,
so I added the write permission to “fuego-rw” diredtory.
I have solved this problem.
After that, I got the error message like following in console log.
====
Started by user anonymous
Building remotely on docker in workspace /fuego-rw/buildzone
[buildzone] $ /bin/sh -xe /tmp/hudson1255554151521556820.sh
+ export Reboot=false
+ Reboot=false
+ export Rebuild=true
+ Rebuild=true
+ export Target_Cleanup=true
+ Target_Cleanup=true
+ export TESTDIR=Benchmark.Dhrystone
+ TESTDIR=Benchmark.Dhrystone
+ export TESTNAME=Dhrystone
+ TESTNAME=Dhrystone
+ export DISTRIB=nosyslogd.dist
+ DISTRIB=nosyslogd.dist
+ export TESTPLAN=testplan_docker
+ TESTPLAN=testplan_docker
+ timeout --signal=9 3m /bin/bash
/fuego-core/engine/tests/Benchmark.Dhrystone/Dhrystone.sh
board overlay: boards/docker.board
using boards/docker.board board overlay
using nosyslogd.dist overlay
BATCH_TESTPLAN is not set, using testplan_docker.json testplan
Traceback (most recent call last):
File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 615, in <module
>
run()
File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 607, in run
generateProlog(output, ofcls, classes, testPlans, testSpecs)
File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 375, in
generateProlog
outfile = open(outFilePath, "w")
IOError: [Errno 2] No such file or directory: '/fuego-rw/work/docker_
prolog.sh'
*** ABORTED ***
Fuego error reason: Error while prolog.sh file generation
Build step 'Execute shell' marked build as failure
[description-setter] Description set:
Finished: FAILURE
====
Did I miss to run test cases on Docker node?
If you know this solution, could you give me some advice?
Thank you.
Best Regard.
Kengo Ibe
----- Original Message -----
件名: [Fuego] RFC: Fuego's version up and other changes
送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
送信日時: 2017年01月11日(水) 09:28:07
宛先: "Bird, Timothy" <Tim.Bird@am.sony.com>, fuego@lists.
linuxfoundation.org
> Hi everyone,
>
> I've been working on Fuego's upgrade for a few weeks, and in the
process I
> have made quite a few changes.
> You can check it by following these instructions. I'd really
appreciate
> your feedback:
>
> Quickstart:
> $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
> $ cd fuego/
> $ ./install.sh
> $ ./fuego-host-scripts/docker-create-container.sh
> $ ./fuego-host-scripts/docker-start-container.sh
> docker# fuego-create-node --board docker
> docker# fuego-create-jobs --board docker --testplan testplan_docker
> --distrib nosyslogd.dist
> Then, start the "docker.testplan_docker.batch" job which will
trigger
> the rest of jobs.
>
> Here is a list of the most fundamental modifications I've added:
> - I automatized the version upgrade process. So far I have updated
Jenkins
> 3 times quite smoothly.
> The current version is the latest LTS version (2.31.1 LTS)
> - I reduced plugin dependencies to the minimum (only 2, and they can
be
> made optional).
> - I removed dependencies on all groovy scripts.
> - Userdata is gone. Instead we now have fuego-ro and fuego-rw, which
> together with fuego-core
> are mounted as external docker volumes. Only fuego-rw, which
contains
> logs and build folders,
> is mounted in RW mode. I also fixed the uid/gid of jenkins so that
it
> matches the one in the host.
> This means you can develop comfortably on your host, with GUI tools,
> and don't fear a buggy
> script deleting your folders.
> - I added a tool for generating jobs from testplans. Creating jobs
from
> testplans is powerful.
> It allows you, for example, to easily specify the timeout for each
test
> on your board.
> The tool also generates the corresponding batch job. For example,
you
> can easily trigger all
> tests for your board and a specific testplan periodically.
> [Note] This will also enable the creation of scripts for comparing
test
> results across
> different boards and test parameters (on my TODO list).
>
> Other less fundamental changes include:
> - I added support for using the docker container itself as a target
board
> (e.g.: for a quickstart or unit tests)
> - I fixed the flot plugin (java and javascript fixes)
> - Now you can click on a node and see which jobs (and testplans) are
> assigned to your board. This was
> one of my old feature requests and makes Fuego's GUI easier to use.
> - I added quite a few fixes and improvements to the core engine
scripts
> and tests (too detailed to describe here).
> - I added Excel output support for IOzone
> - Faster docker build time (ARMhf compiler installation is now
optional,
> no latex..)
> - I put a fixed name to the fuego container, instead of the
> "last_container_id" file.
> - I removed the inotify script
> - I added scripts for easily removing the docker container/image.
>
> I have to split the current "huge" patch into smaller chunks and test
it
> further but it would
> be nice to get your feedback about the approach in general.
>
> Tim: I haven't updated ftc nor your "Christmas commits" yet.
>
> Best regards
> Daniel
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] RFC: Fuego's version up and other changes
2017-01-12 5:51 ` Ibe.Kengo
@ 2017-01-12 6:44 ` Daniel Sangorrin
2017-01-12 8:15 ` Ibe.Kengo
2017-01-13 1:02 ` Bird, Timothy
0 siblings, 2 replies; 43+ messages in thread
From: Daniel Sangorrin @ 2017-01-12 6:44 UTC (permalink / raw)
To: Ibe.Kengo; +Cc: fuego
Hi Kengo-san,
Thanks a lot for your feedback.
> -----Original Message-----
> From: Ibe.Kengo@bx.MitsubishiElectric.co.jp [mailto:Ibe.Kengo@bx.MitsubishiElectric.co.jp]
> Sent: Thursday, January 12, 2017 2:52 PM
> To: Daniel Sangorrin
> Cc: fuego@lists.linuxfoundation.org
> Subject: Re: [Fuego] RFC: Fuego's version up and other changes
>
> Hi, Daniel-san.
>
> I’m very interested in your work!
> So I tried to run your build scripts to create Docker image today,
> then I could run new Fuego.
> And, I added Docker node and test cases to use following command.
> ==
> $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
>
> $ cd fuego/
> $ ./install.sh
> $ ./fuego-host-scripts/docker-create-container.sh
> $ ./fuego-host-scripts/docker-start-container.sh
> docker# fuego-create-node --board docker
> docker# fuego-create-jobs --board docker --testplan testplan_docker
> --distrib nosyslogd.dist
> ==
>
> I can see new node and new test jobs.
> But, I can’t run tests like Dhrystone on Docker.
>
> So far, I had one small problem that is “Failed to mkdirs: /fuego-rw/
> buildzone”
> This reason is the permission or owner of “fuego-rw”,
> so I added the write permission to “fuego-rw” diredtory.
Humm You should not need to change any permissions.
The script is supposed to use your user's uid/gid for the jenkins daemon.
However, I have only tested it on my pc which has a single user.
Do you have more than one user? Could you check that the jenkins uid/gid
is the same as yours? (you can check on /etc/passwd inside and outside docker)
Please also check that the fuego-rw's owner is your user (e.g.: don't git clone with
a different user or root)
> I have solved this problem.
>
> After that, I got the error message like following in console log.
> ====
> Started by user anonymous
> Building remotely on docker in workspace /fuego-rw/buildzone
> [buildzone] $ /bin/sh -xe /tmp/hudson1255554151521556820.sh
> + export Reboot=false
> + Reboot=false
> + export Rebuild=true
> + Rebuild=true
> + export Target_Cleanup=true
> + Target_Cleanup=true
> + export TESTDIR=Benchmark.Dhrystone
> + TESTDIR=Benchmark.Dhrystone
> + export TESTNAME=Dhrystone
> + TESTNAME=Dhrystone
> + export DISTRIB=nosyslogd.dist
> + DISTRIB=nosyslogd.dist
> + export TESTPLAN=testplan_docker
> + TESTPLAN=testplan_docker
> + timeout --signal=9 3m /bin/bash
> /fuego-core/engine/tests/Benchmark.Dhrystone/Dhrystone.sh
> board overlay: boards/docker.board
> using boards/docker.board board overlay
> using nosyslogd.dist overlay
> BATCH_TESTPLAN is not set, using testplan_docker.json testplan
> Traceback (most recent call last):
> File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 615, in <module
> >
> run()
> File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 607, in run
> generateProlog(output, ofcls, classes, testPlans, testSpecs)
> File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 375, in
> generateProlog
> outfile = open(outFilePath, "w")
> IOError: [Errno 2] No such file or directory: '/fuego-rw/work/docker_
> prolog.sh'
>
> *** ABORTED ***
>
> Fuego error reason: Error while prolog.sh file generation
>
> Build step 'Execute shell' marked build as failure
> [description-setter] Description set:
> Finished: FAILURE
> ====
>
> Did I miss to run test cases on Docker node?
> If you know this solution, could you give me some advice?
Could you check that fuego-rw is correctly mounted inside docker?
outside-docker$ touch fuego-rw/hoge
inside-docker$ ls /fuego-rw
hoge <-- does it appear?
On my PC (ubuntu 16.04, docker version 1.12.3 ) it's always succesful so I will
try on a different one. Which OS and docker version are you using?
Thanks,
Daniel
> Thank you.
>
> Best Regard.
> Kengo Ibe
>
> ----- Original Message -----
> 件名: [Fuego] RFC: Fuego's version up and other changes
> 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> 送信日時: 2017年01月11日(水) 09:28:07
> 宛先: "Bird, Timothy" <Tim.Bird@am.sony.com>, fuego@lists.
> linuxfoundation.org
>
> > Hi everyone,
> >
> > I've been working on Fuego's upgrade for a few weeks, and in the
> process I
> > have made quite a few changes.
> > You can check it by following these instructions. I'd really
> appreciate
> > your feedback:
> >
> > Quickstart:
> > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
>
> > $ cd fuego/
> > $ ./install.sh
> > $ ./fuego-host-scripts/docker-create-container.sh
> > $ ./fuego-host-scripts/docker-start-container.sh
> > docker# fuego-create-node --board docker
> > docker# fuego-create-jobs --board docker --testplan testplan_docker
> > --distrib nosyslogd.dist
> > Then, start the "docker.testplan_docker.batch" job which will
> trigger
> > the rest of jobs.
> >
> > Here is a list of the most fundamental modifications I've added:
> > - I automatized the version upgrade process. So far I have updated
> Jenkins
> > 3 times quite smoothly.
> > The current version is the latest LTS version (2.31.1 LTS)
> > - I reduced plugin dependencies to the minimum (only 2, and they can
> be
> > made optional).
> > - I removed dependencies on all groovy scripts.
> > - Userdata is gone. Instead we now have fuego-ro and fuego-rw, which
> > together with fuego-core
> > are mounted as external docker volumes. Only fuego-rw, which
> contains
> > logs and build folders,
> > is mounted in RW mode. I also fixed the uid/gid of jenkins so that
> it
> > matches the one in the host.
> > This means you can develop comfortably on your host, with GUI tools,
> > and don't fear a buggy
> > script deleting your folders.
> > - I added a tool for generating jobs from testplans. Creating jobs
> from
> > testplans is powerful.
> > It allows you, for example, to easily specify the timeout for each
> test
> > on your board.
> > The tool also generates the corresponding batch job. For example,
> you
> > can easily trigger all
> > tests for your board and a specific testplan periodically.
> > [Note] This will also enable the creation of scripts for comparing
> test
> > results across
> > different boards and test parameters (on my TODO list).
> >
> > Other less fundamental changes include:
> > - I added support for using the docker container itself as a target
> board
> > (e.g.: for a quickstart or unit tests)
> > - I fixed the flot plugin (java and javascript fixes)
> > - Now you can click on a node and see which jobs (and testplans) are
> > assigned to your board. This was
> > one of my old feature requests and makes Fuego's GUI easier to use.
> > - I added quite a few fixes and improvements to the core engine
> scripts
> > and tests (too detailed to describe here).
> > - I added Excel output support for IOzone
> > - Faster docker build time (ARMhf compiler installation is now
> optional,
> > no latex..)
> > - I put a fixed name to the fuego container, instead of the
> > "last_container_id" file.
> > - I removed the inotify script
> > - I added scripts for easily removing the docker container/image.
> >
> > I have to split the current "huge" patch into smaller chunks and test
> it
> > further but it would
> > be nice to get your feedback about the approach in general.
> >
> > Tim: I haven't updated ftc nor your "Christmas commits" yet.
> >
> > Best regards
> > Daniel
> >
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> >
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] RFC: Fuego's version up and other changes
2017-01-12 6:44 ` Daniel Sangorrin
@ 2017-01-12 8:15 ` Ibe.Kengo
2017-01-12 8:45 ` Daniel Sangorrin
2017-01-13 1:02 ` Bird, Timothy
1 sibling, 1 reply; 43+ messages in thread
From: Ibe.Kengo @ 2017-01-12 8:15 UTC (permalink / raw)
To: Daniel Sangorrin; +Cc: fuego
Hi, Daniel-san
Thank you for your prompt reply.
I have more a few question.
> Humm You should not need to change any permissions.
OK.
I removed the Fuego and the Fuego-core,
after that, I redownloaded the Fuego and the Fuego-core and rebuilded
and start Docker.
I can see following permissions of directories. Is it correct?
==
melco@debian8:~/$ ./fuego/fuego-host-scripts/docker-start-container.sh
Starting Fuego container (fuego-container)
[ ok ] Starting Jenkins Continuous Integration Server: jenkins.
root@debian8:/# ls -al
total 84
drwxr-xr-x 65 root root 4096 Jan 12 07:26 .
drwxr-xr-x 65 root root 4096 Jan 12 07:26 ..
-rwxr-xr-x 1 root root 0 Jan 12 07:26 .dockerenv
drwxr-xr-x 2 root root 4096 Jan 12 00:53 bin
drwxr-xr-x 2 root root 4096 Sep 12 04:09 boot
drwxr-xr-x 5 root root 380 Jan 12 07:26 dev
drwxr-xr-x 96 root root 4096 Jan 12 07:26 etc
drwxr-xr-x 4 jenkins jenkins 4096 Jan 12 07:07 fuego-core
drwxr-xr-x 5 jenkins jenkins 4096 Jan 12 07:03 fuego-ro
drwxr-xr-x 2 root root 4096 Jan 12 07:26 fuego-rw
drwxr-xr-x 2 root root 4096 Sep 12 04:09 home
~
melco@debian8:~/$ ls -al fuego
合計 68
drwxr-xr-x 8 melco melco 4096 1月 12 16:26 .
drwxr-xr-x 4 melco melco 4096 1月 12 16:02 ..
-rw-r--r-- 1 melco melco 42 1月 12 16:03 .dockerignore
drwxr-xr-x 8 melco melco 4096 1月 12 16:03 .git
-rw-r--r-- 1 melco melco 9 1月 12 16:03 .gitignore
-rw-r--r-- 1 melco melco 1482 1月 12 16:03 CHANGELOG
-rw-r--r-- 1 melco melco 4810 1月 12 16:03 Dockerfile
-rw-r--r-- 1 melco melco 1492 1月 12 16:03 LICENSE
-rw-r--r-- 1 melco melco 3414 1月 12 16:03 README
-rw-r--r-- 1 melco melco 85 1月 12 16:03 VERSION
drwxr-xr-x 3 melco melco 4096 1月 12 16:03 docs
drwxr-xr-x 3 melco melco 4096 1月 12 16:03 frontend-install
drwxr-xr-x 2 melco melco 4096 1月 12 16:03 fuego-host-scripts
drwxr-xr-x 5 melco melco 4096 1月 12 16:03 fuego-ro
drwxr-xr-x 2 root root 4096 1月 12 16:26 fuego-rw
-rwxr-xr-x 1 melco melco 48 1月 12 16:03 install.sh
==
“melco” user is only user on host PC which is Debian8.
I’m concerned about fuego-rw permissions and owner.
> The script is supposed to use your user's uid/gid for the jenkins
daemon.
Do you mean that I need to make “Jenkins” user on host PC?
Or
Can I use “melco” user inside Docker?
> However, I have only tested it on my pc which has a single user.
> Do you have more than one user?
I have one user. I'v not made "jenkins" user on host PC.
I’ll check other point following your advice.
Thank you so much.
Kengo
----- Original Message -----
件名: RE: [Fuego] RFC: Fuego's version up and other changes
送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
送信日時: 2017年01月12日(木) 15:44:34
宛先: Ibe.Kengo@bx.MitsubishiElectric.co.jp
Cc: fuego@lists.linuxfoundation.org
> Hi Kengo-san,
>
> Thanks a lot for your feedback.
>
> > -----Original Message-----
> > From: Ibe.Kengo@bx.MitsubishiElectric.co.jp [mailto:Ibe.Kengo@bx.
MitsubishiElectric.co.jp]
> > Sent: Thursday, January 12, 2017 2:52 PM
> > To: Daniel Sangorrin
> > Cc: fuego@lists.linuxfoundation.org
> > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> >
> > Hi, Daniel-san.
> >
> > I’m very interested in your work!
> > So I tried to run your build scripts to create Docker image today,
> > then I could run new Fuego.
> > And, I added Docker node and test cases to use following command.
> > ==
> > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
> >
> > $ cd fuego/
> > $ ./install.sh
> > $ ./fuego-host-scripts/docker-create-container.sh
> > $ ./fuego-host-scripts/docker-start-container.sh
> > docker# fuego-create-node --board docker
> > docker# fuego-create-jobs --board docker --testplan testplan_
docker
> > --distrib nosyslogd.dist
> > ==
> >
> > I can see new node and new test jobs.
> > But, I can’t run tests like Dhrystone on Docker.
> >
> > So far, I had one small problem that is “Failed to mkdirs: /fuego-rw
/
> > buildzone”
> > This reason is the permission or owner of “fuego-rw”,
> > so I added the write permission to “fuego-rw” diredtory.
>
> Humm You should not need to change any permissions.
> The script is supposed to use your user's uid/gid for the jenkins
daemon.
> However, I have only tested it on my pc which has a single user.
> Do you have more than one user? Could you check that the jenkins uid/
gid
> is the same as yours? (you can check on /etc/passwd inside and outside
docker)
> Please also check that the fuego-rw's owner is your user (e.g.: don't
git clone with
> a different user or root)
>
> > I have solved this problem.
> >
> > After that, I got the error message like following in console log.
> > ====
> > Started by user anonymous
> > Building remotely on docker in workspace /fuego-rw/buildzone
> > [buildzone] $ /bin/sh -xe /tmp/hudson1255554151521556820.sh
> > + export Reboot=false
> > + Reboot=false
> > + export Rebuild=true
> > + Rebuild=true
> > + export Target_Cleanup=true
> > + Target_Cleanup=true
> > + export TESTDIR=Benchmark.Dhrystone
> > + TESTDIR=Benchmark.Dhrystone
> > + export TESTNAME=Dhrystone
> > + TESTNAME=Dhrystone
> > + export DISTRIB=nosyslogd.dist
> > + DISTRIB=nosyslogd.dist
> > + export TESTPLAN=testplan_docker
> > + TESTPLAN=testplan_docker
> > + timeout --signal=9 3m /bin/bash
> > /fuego-core/engine/tests/Benchmark.Dhrystone/Dhrystone.sh
> > board overlay: boards/docker.board
> > using boards/docker.board board overlay
> > using nosyslogd.dist overlay
> > BATCH_TESTPLAN is not set, using testplan_docker.json testplan
> > Traceback (most recent call last):
> > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 615, in <
module
> > >
> > run()
> > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 607, in run
> > generateProlog(output, ofcls, classes, testPlans, testSpecs)
> > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 375, in
> > generateProlog
> > outfile = open(outFilePath, "w")
> > IOError: [Errno 2] No such file or directory: '/fuego-rw/work/docker
_
> > prolog.sh'
> >
> > *** ABORTED ***
> >
> > Fuego error reason: Error while prolog.sh file generation
> >
> > Build step 'Execute shell' marked build as failure
> > [description-setter] Description set:
> > Finished: FAILURE
> > ====
> >
> > Did I miss to run test cases on Docker node?
> > If you know this solution, could you give me some advice?
>
> Could you check that fuego-rw is correctly mounted inside docker?
> outside-docker$ touch fuego-rw/hoge
> inside-docker$ ls /fuego-rw
> hoge <-- does it appear?
>
> On my PC (ubuntu 16.04, docker version 1.12.3 ) it's always succesful
so I will
> try on a different one. Which OS and docker version are you using?
>
> Thanks,
> Daniel
>
> > Thank you.
> >
> > Best Regard.
> > Kengo Ibe
> >
> > ----- Original Message -----
> > 件名: [Fuego] RFC: Fuego's version up and other changes
> > 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> > 送信日時: 2017年01月11日(水) 09:28:07
> > 宛先: "Bird, Timothy" <Tim.Bird@am.sony.com>, fuego@lists.
> > linuxfoundation.org
> >
> > > Hi everyone,
> > >
> > > I've been working on Fuego's upgrade for a few weeks, and in the
> > process I
> > > have made quite a few changes.
> > > You can check it by following these instructions. I'd really
> > appreciate
> > > your feedback:
> > >
> > > Quickstart:
> > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
> >
> > > $ cd fuego/
> > > $ ./install.sh
> > > $ ./fuego-host-scripts/docker-create-container.sh
> > > $ ./fuego-host-scripts/docker-start-container.sh
> > > docker# fuego-create-node --board docker
> > > docker# fuego-create-jobs --board docker --testplan testplan_
docker
> > > --distrib nosyslogd.dist
> > > Then, start the "docker.testplan_docker.batch" job which will
> > trigger
> > > the rest of jobs.
> > >
> > > Here is a list of the most fundamental modifications I've added:
> > > - I automatized the version upgrade process. So far I have updated
> > Jenkins
> > > 3 times quite smoothly.
> > > The current version is the latest LTS version (2.31.1 LTS)
> > > - I reduced plugin dependencies to the minimum (only 2, and they
can
> > be
> > > made optional).
> > > - I removed dependencies on all groovy scripts.
> > > - Userdata is gone. Instead we now have fuego-ro and fuego-rw,
which
> > > together with fuego-core
> > > are mounted as external docker volumes. Only fuego-rw, which
> > contains
> > > logs and build folders,
> > > is mounted in RW mode. I also fixed the uid/gid of jenkins so
that
> > it
> > > matches the one in the host.
> > > This means you can develop comfortably on your host, with GUI
tools,
> > > and don't fear a buggy
> > > script deleting your folders.
> > > - I added a tool for generating jobs from testplans. Creating
jobs
> > from
> > > testplans is powerful.
> > > It allows you, for example, to easily specify the timeout for
each
> > test
> > > on your board.
> > > The tool also generates the corresponding batch job. For
example,
> > you
> > > can easily trigger all
> > > tests for your board and a specific testplan periodically.
> > > [Note] This will also enable the creation of scripts for
comparing
> > test
> > > results across
> > > different boards and test parameters (on my TODO list).
> > >
> > > Other less fundamental changes include:
> > > - I added support for using the docker container itself as a
target
> > board
> > > (e.g.: for a quickstart or unit tests)
> > > - I fixed the flot plugin (java and javascript fixes)
> > > - Now you can click on a node and see which jobs (and testplans)
are
> > > assigned to your board. This was
> > > one of my old feature requests and makes Fuego's GUI easier to
use.
> > > - I added quite a few fixes and improvements to the core engine
> > scripts
> > > and tests (too detailed to describe here).
> > > - I added Excel output support for IOzone
> > > - Faster docker build time (ARMhf compiler installation is now
> > optional,
> > > no latex..)
> > > - I put a fixed name to the fuego container, instead of the
> > > "last_container_id" file.
> > > - I removed the inotify script
> > > - I added scripts for easily removing the docker container/image.
> > >
> > > I have to split the current "huge" patch into smaller chunks and
test
> > it
> > > further but it would
> > > be nice to get your feedback about the approach in general.
> > >
> > > Tim: I haven't updated ftc nor your "Christmas commits" yet.
> > >
> > > Best regards
> > > Daniel
> > >
> > >
> > > _______________________________________________
> > > Fuego mailing list
> > > Fuego@lists.linuxfoundation.org
> > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > >
> >
>
>
>
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] RFC: Fuego's version up and other changes
2017-01-12 8:15 ` Ibe.Kengo
@ 2017-01-12 8:45 ` Daniel Sangorrin
2017-01-12 9:04 ` Ibe.Kengo
0 siblings, 1 reply; 43+ messages in thread
From: Daniel Sangorrin @ 2017-01-12 8:45 UTC (permalink / raw)
To: Ibe.Kengo; +Cc: fuego
Hi Kengo-san,
I'm sorry, it seems that fuego-rw didn't get pushed. The reason is because it's included in .gitignore!!
I have pushed it now. Please remove your current fuego-rw and do a git pull. It should work properly this time.
Thanks
Daniel
> -----Original Message-----
> From: Ibe.Kengo@bx.MitsubishiElectric.co.jp [mailto:Ibe.Kengo@bx.MitsubishiElectric.co.jp]
> Sent: Thursday, January 12, 2017 5:16 PM
> To: Daniel Sangorrin
> Cc: fuego@lists.linuxfoundation.org
> Subject: Re: [Fuego] RFC: Fuego's version up and other changes
>
> Hi, Daniel-san
>
> Thank you for your prompt reply.
> I have more a few question.
>
> > Humm You should not need to change any permissions.
> OK.
> I removed the Fuego and the Fuego-core,
> after that, I redownloaded the Fuego and the Fuego-core and rebuilded
> and start Docker.
> I can see following permissions of directories. Is it correct?
> ==
> melco@debian8:~/$ ./fuego/fuego-host-scripts/docker-start-container.sh
> Starting Fuego container (fuego-container)
> [ ok ] Starting Jenkins Continuous Integration Server: jenkins.
>
> root@debian8:/# ls -al
> total 84
> drwxr-xr-x 65 root root 4096 Jan 12 07:26 .
> drwxr-xr-x 65 root root 4096 Jan 12 07:26 ..
> -rwxr-xr-x 1 root root 0 Jan 12 07:26 .dockerenv
> drwxr-xr-x 2 root root 4096 Jan 12 00:53 bin
> drwxr-xr-x 2 root root 4096 Sep 12 04:09 boot
> drwxr-xr-x 5 root root 380 Jan 12 07:26 dev
> drwxr-xr-x 96 root root 4096 Jan 12 07:26 etc
> drwxr-xr-x 4 jenkins jenkins 4096 Jan 12 07:07 fuego-core
> drwxr-xr-x 5 jenkins jenkins 4096 Jan 12 07:03 fuego-ro
> drwxr-xr-x 2 root root 4096 Jan 12 07:26 fuego-rw
> drwxr-xr-x 2 root root 4096 Sep 12 04:09 home
> ~
>
> melco@debian8:~/$ ls -al fuego
> 合計 68
> drwxr-xr-x 8 melco melco 4096 1月 12 16:26 .
> drwxr-xr-x 4 melco melco 4096 1月 12 16:02 ..
> -rw-r--r-- 1 melco melco 42 1月 12 16:03 .dockerignore
> drwxr-xr-x 8 melco melco 4096 1月 12 16:03 .git
> -rw-r--r-- 1 melco melco 9 1月 12 16:03 .gitignore
> -rw-r--r-- 1 melco melco 1482 1月 12 16:03 CHANGELOG
> -rw-r--r-- 1 melco melco 4810 1月 12 16:03 Dockerfile
> -rw-r--r-- 1 melco melco 1492 1月 12 16:03 LICENSE
> -rw-r--r-- 1 melco melco 3414 1月 12 16:03 README
> -rw-r--r-- 1 melco melco 85 1月 12 16:03 VERSION
> drwxr-xr-x 3 melco melco 4096 1月 12 16:03 docs
> drwxr-xr-x 3 melco melco 4096 1月 12 16:03 frontend-install
> drwxr-xr-x 2 melco melco 4096 1月 12 16:03 fuego-host-scripts
> drwxr-xr-x 5 melco melco 4096 1月 12 16:03 fuego-ro
> drwxr-xr-x 2 root root 4096 1月 12 16:26 fuego-rw
> -rwxr-xr-x 1 melco melco 48 1月 12 16:03 install.sh
>
> ==
> “melco” user is only user on host PC which is Debian8.
> I’m concerned about fuego-rw permissions and owner.
>
> > The script is supposed to use your user's uid/gid for the jenkins
> daemon.
> Do you mean that I need to make “Jenkins” user on host PC?
> Or
> Can I use “melco” user inside Docker?
>
> > However, I have only tested it on my pc which has a single user.
> > Do you have more than one user?
> I have one user. I'v not made "jenkins" user on host PC.
>
> I’ll check other point following your advice.
> Thank you so much.
> Kengo
>
>
> ----- Original Message -----
> 件名: RE: [Fuego] RFC: Fuego's version up and other changes
> 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> 送信日時: 2017年01月12日(木) 15:44:34
> 宛先: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> Cc: fuego@lists.linuxfoundation.org
>
> > Hi Kengo-san,
> >
> > Thanks a lot for your feedback.
> >
> > > -----Original Message-----
> > > From: Ibe.Kengo@bx.MitsubishiElectric.co.jp [mailto:Ibe.Kengo@bx.
> MitsubishiElectric.co.jp]
> > > Sent: Thursday, January 12, 2017 2:52 PM
> > > To: Daniel Sangorrin
> > > Cc: fuego@lists.linuxfoundation.org
> > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > >
> > > Hi, Daniel-san.
> > >
> > > I’m very interested in your work!
> > > So I tried to run your build scripts to create Docker image today,
> > > then I could run new Fuego.
> > > And, I added Docker node and test cases to use following command.
> > > ==
> > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
>
> > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
>
> > >
> > > $ cd fuego/
> > > $ ./install.sh
> > > $ ./fuego-host-scripts/docker-create-container.sh
> > > $ ./fuego-host-scripts/docker-start-container.sh
> > > docker# fuego-create-node --board docker
> > > docker# fuego-create-jobs --board docker --testplan testplan_
> docker
> > > --distrib nosyslogd.dist
> > > ==
> > >
> > > I can see new node and new test jobs.
> > > But, I can’t run tests like Dhrystone on Docker.
> > >
> > > So far, I had one small problem that is “Failed to mkdirs: /fuego-rw
> /
> > > buildzone”
> > > This reason is the permission or owner of “fuego-rw”,
> > > so I added the write permission to “fuego-rw” diredtory.
> >
> > Humm You should not need to change any permissions.
> > The script is supposed to use your user's uid/gid for the jenkins
> daemon.
> > However, I have only tested it on my pc which has a single user.
> > Do you have more than one user? Could you check that the jenkins uid/
> gid
> > is the same as yours? (you can check on /etc/passwd inside and outside
> docker)
> > Please also check that the fuego-rw's owner is your user (e.g.: don't
> git clone with
> > a different user or root)
> >
> > > I have solved this problem.
> > >
> > > After that, I got the error message like following in console log.
> > > ====
> > > Started by user anonymous
> > > Building remotely on docker in workspace /fuego-rw/buildzone
> > > [buildzone] $ /bin/sh -xe /tmp/hudson1255554151521556820.sh
> > > + export Reboot=false
> > > + Reboot=false
> > > + export Rebuild=true
> > > + Rebuild=true
> > > + export Target_Cleanup=true
> > > + Target_Cleanup=true
> > > + export TESTDIR=Benchmark.Dhrystone
> > > + TESTDIR=Benchmark.Dhrystone
> > > + export TESTNAME=Dhrystone
> > > + TESTNAME=Dhrystone
> > > + export DISTRIB=nosyslogd.dist
> > > + DISTRIB=nosyslogd.dist
> > > + export TESTPLAN=testplan_docker
> > > + TESTPLAN=testplan_docker
> > > + timeout --signal=9 3m /bin/bash
> > > /fuego-core/engine/tests/Benchmark.Dhrystone/Dhrystone.sh
> > > board overlay: boards/docker.board
> > > using boards/docker.board board overlay
> > > using nosyslogd.dist overlay
> > > BATCH_TESTPLAN is not set, using testplan_docker.json testplan
> > > Traceback (most recent call last):
> > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 615, in <
> module
> > > >
> > > run()
> > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 607, in run
> > > generateProlog(output, ofcls, classes, testPlans, testSpecs)
> > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 375, in
> > > generateProlog
> > > outfile = open(outFilePath, "w")
> > > IOError: [Errno 2] No such file or directory: '/fuego-rw/work/docker
> _
> > > prolog.sh'
> > >
> > > *** ABORTED ***
> > >
> > > Fuego error reason: Error while prolog.sh file generation
> > >
> > > Build step 'Execute shell' marked build as failure
> > > [description-setter] Description set:
> > > Finished: FAILURE
> > > ====
> > >
> > > Did I miss to run test cases on Docker node?
> > > If you know this solution, could you give me some advice?
> >
> > Could you check that fuego-rw is correctly mounted inside docker?
> > outside-docker$ touch fuego-rw/hoge
> > inside-docker$ ls /fuego-rw
> > hoge <-- does it appear?
> >
> > On my PC (ubuntu 16.04, docker version 1.12.3 ) it's always succesful
> so I will
> > try on a different one. Which OS and docker version are you using?
> >
> > Thanks,
> > Daniel
> >
> > > Thank you.
> > >
> > > Best Regard.
> > > Kengo Ibe
> > >
> > > ----- Original Message -----
> > > 件名: [Fuego] RFC: Fuego's version up and other changes
> > > 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> > > 送信日時: 2017年01月11日(水) 09:28:07
> > > 宛先: "Bird, Timothy" <Tim.Bird@am.sony.com>, fuego@lists.
> > > linuxfoundation.org
> > >
> > > > Hi everyone,
> > > >
> > > > I've been working on Fuego's upgrade for a few weeks, and in the
> > > process I
> > > > have made quite a few changes.
> > > > You can check it by following these instructions. I'd really
> > > appreciate
> > > > your feedback:
> > > >
> > > > Quickstart:
> > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
>
> > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
>
> > >
> > > > $ cd fuego/
> > > > $ ./install.sh
> > > > $ ./fuego-host-scripts/docker-create-container.sh
> > > > $ ./fuego-host-scripts/docker-start-container.sh
> > > > docker# fuego-create-node --board docker
> > > > docker# fuego-create-jobs --board docker --testplan testplan_
> docker
> > > > --distrib nosyslogd.dist
> > > > Then, start the "docker.testplan_docker.batch" job which will
> > > trigger
> > > > the rest of jobs.
> > > >
> > > > Here is a list of the most fundamental modifications I've added:
> > > > - I automatized the version upgrade process. So far I have updated
> > > Jenkins
> > > > 3 times quite smoothly.
> > > > The current version is the latest LTS version (2.31.1 LTS)
> > > > - I reduced plugin dependencies to the minimum (only 2, and they
> can
> > > be
> > > > made optional).
> > > > - I removed dependencies on all groovy scripts.
> > > > - Userdata is gone. Instead we now have fuego-ro and fuego-rw,
> which
> > > > together with fuego-core
> > > > are mounted as external docker volumes. Only fuego-rw, which
> > > contains
> > > > logs and build folders,
> > > > is mounted in RW mode. I also fixed the uid/gid of jenkins so
> that
> > > it
> > > > matches the one in the host.
> > > > This means you can develop comfortably on your host, with GUI
> tools,
> > > > and don't fear a buggy
> > > > script deleting your folders.
> > > > - I added a tool for generating jobs from testplans. Creating
> jobs
> > > from
> > > > testplans is powerful.
> > > > It allows you, for example, to easily specify the timeout for
> each
> > > test
> > > > on your board.
> > > > The tool also generates the corresponding batch job. For
> example,
> > > you
> > > > can easily trigger all
> > > > tests for your board and a specific testplan periodically.
> > > > [Note] This will also enable the creation of scripts for
> comparing
> > > test
> > > > results across
> > > > different boards and test parameters (on my TODO list).
> > > >
> > > > Other less fundamental changes include:
> > > > - I added support for using the docker container itself as a
> target
> > > board
> > > > (e.g.: for a quickstart or unit tests)
> > > > - I fixed the flot plugin (java and javascript fixes)
> > > > - Now you can click on a node and see which jobs (and testplans)
> are
> > > > assigned to your board. This was
> > > > one of my old feature requests and makes Fuego's GUI easier to
> use.
> > > > - I added quite a few fixes and improvements to the core engine
> > > scripts
> > > > and tests (too detailed to describe here).
> > > > - I added Excel output support for IOzone
> > > > - Faster docker build time (ARMhf compiler installation is now
> > > optional,
> > > > no latex..)
> > > > - I put a fixed name to the fuego container, instead of the
> > > > "last_container_id" file.
> > > > - I removed the inotify script
> > > > - I added scripts for easily removing the docker container/image.
> > > >
> > > > I have to split the current "huge" patch into smaller chunks and
> test
> > > it
> > > > further but it would
> > > > be nice to get your feedback about the approach in general.
> > > >
> > > > Tim: I haven't updated ftc nor your "Christmas commits" yet.
> > > >
> > > > Best regards
> > > > Daniel
> > > >
> > > >
> > > > _______________________________________________
> > > > Fuego mailing list
> > > > Fuego@lists.linuxfoundation.org
> > > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > > >
> > >
> >
> >
> >
> >
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] RFC: Fuego's version up and other changes
2017-01-12 8:45 ` Daniel Sangorrin
@ 2017-01-12 9:04 ` Ibe.Kengo
2017-01-12 9:18 ` Daniel Sangorrin
0 siblings, 1 reply; 43+ messages in thread
From: Ibe.Kengo @ 2017-01-12 9:04 UTC (permalink / raw)
To: Daniel Sangorrin; +Cc: fuego
Hi, Daniel-san,
Thank you for prompt help.
I was able to run tests in my Docker!! :)
I'm going to try to use some function of your Fuego.
If I have some question, let me ask you some question.
Thanks,
Kengo
----- Original Message -----
件名: RE: [Fuego] RFC: Fuego's version up and other changes
送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
送信日時: 2017年01月12日(木) 17:45:20
宛先: Ibe.Kengo@bx.MitsubishiElectric.co.jp
Cc: fuego@lists.linuxfoundation.org
> Hi Kengo-san,
>
> I'm sorry, it seems that fuego-rw didn't get pushed. The reason is
because it's included in .gitignore!!
> I have pushed it now. Please remove your current fuego-rw and do a git
pull. It should work properly this time.
>
> Thanks
> Daniel
>
>
> > -----Original Message-----
> > From: Ibe.Kengo@bx.MitsubishiElectric.co.jp [mailto:Ibe.Kengo@bx.
MitsubishiElectric.co.jp]
> > Sent: Thursday, January 12, 2017 5:16 PM
> > To: Daniel Sangorrin
> > Cc: fuego@lists.linuxfoundation.org
> > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> >
> > Hi, Daniel-san
> >
> > Thank you for your prompt reply.
> > I have more a few question.
> >
> > > Humm You should not need to change any permissions.
> > OK.
> > I removed the Fuego and the Fuego-core,
> > after that, I redownloaded the Fuego and the Fuego-core and
rebuilded
> > and start Docker.
> > I can see following permissions of directories. Is it correct?
> > ==
> > melco@debian8:~/$ ./fuego/fuego-host-scripts/docker-start-container.
sh
> > Starting Fuego container (fuego-container)
> > [ ok ] Starting Jenkins Continuous Integration Server: jenkins.
> >
> > root@debian8:/# ls -al
> > total 84
> > drwxr-xr-x 65 root root 4096 Jan 12 07:26 .
> > drwxr-xr-x 65 root root 4096 Jan 12 07:26 ..
> > -rwxr-xr-x 1 root root 0 Jan 12 07:26 .dockerenv
> > drwxr-xr-x 2 root root 4096 Jan 12 00:53 bin
> > drwxr-xr-x 2 root root 4096 Sep 12 04:09 boot
> > drwxr-xr-x 5 root root 380 Jan 12 07:26 dev
> > drwxr-xr-x 96 root root 4096 Jan 12 07:26 etc
> > drwxr-xr-x 4 jenkins jenkins 4096 Jan 12 07:07 fuego-core
> > drwxr-xr-x 5 jenkins jenkins 4096 Jan 12 07:03 fuego-ro
> > drwxr-xr-x 2 root root 4096 Jan 12 07:26 fuego-rw
> > drwxr-xr-x 2 root root 4096 Sep 12 04:09 home
> > ~
> >
> > melco@debian8:~/$ ls -al fuego
> > 合計 68
> > drwxr-xr-x 8 melco melco 4096 1月 12 16:26 .
> > drwxr-xr-x 4 melco melco 4096 1月 12 16:02 ..
> > -rw-r--r-- 1 melco melco 42 1月 12 16:03 .dockerignore
> > drwxr-xr-x 8 melco melco 4096 1月 12 16:03 .git
> > -rw-r--r-- 1 melco melco 9 1月 12 16:03 .gitignore
> > -rw-r--r-- 1 melco melco 1482 1月 12 16:03 CHANGELOG
> > -rw-r--r-- 1 melco melco 4810 1月 12 16:03 Dockerfile
> > -rw-r--r-- 1 melco melco 1492 1月 12 16:03 LICENSE
> > -rw-r--r-- 1 melco melco 3414 1月 12 16:03 README
> > -rw-r--r-- 1 melco melco 85 1月 12 16:03 VERSION
> > drwxr-xr-x 3 melco melco 4096 1月 12 16:03 docs
> > drwxr-xr-x 3 melco melco 4096 1月 12 16:03 frontend-install
> > drwxr-xr-x 2 melco melco 4096 1月 12 16:03 fuego-host-scripts
> > drwxr-xr-x 5 melco melco 4096 1月 12 16:03 fuego-ro
> > drwxr-xr-x 2 root root 4096 1月 12 16:26 fuego-rw
> > -rwxr-xr-x 1 melco melco 48 1月 12 16:03 install.sh
> >
> > ==
> > “melco” user is only user on host PC which is Debian8.
> > I’m concerned about fuego-rw permissions and owner.
> >
> > > The script is supposed to use your user's uid/gid for the jenkins
> > daemon.
> > Do you mean that I need to make “Jenkins” user on host PC?
> > Or
> > Can I use “melco” user inside Docker?
> >
> > > However, I have only tested it on my pc which has a single user.
> > > Do you have more than one user?
> > I have one user. I'v not made "jenkins" user on host PC.
> >
> > I’ll check other point following your advice.
> > Thank you so much.
> > Kengo
> >
> >
> > ----- Original Message -----
> > 件名: RE: [Fuego] RFC: Fuego's version up and other changes
> > 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> > 送信日時: 2017年01月12日(木) 15:44:34
> > 宛先: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > Cc: fuego@lists.linuxfoundation.org
> >
> > > Hi Kengo-san,
> > >
> > > Thanks a lot for your feedback.
> > >
> > > > -----Original Message-----
> > > > From: Ibe.Kengo@bx.MitsubishiElectric.co.jp [mailto:Ibe.Kengo@bx.
> > MitsubishiElectric.co.jp]
> > > > Sent: Thursday, January 12, 2017 2:52 PM
> > > > To: Daniel Sangorrin
> > > > Cc: fuego@lists.linuxfoundation.org
> > > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > > >
> > > > Hi, Daniel-san.
> > > >
> > > > I’m very interested in your work!
> > > > So I tried to run your build scripts to create Docker image
today,
> > > > then I could run new Fuego.
> > > > And, I added Docker node and test cases to use following command.
> > > > ==
> > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> >
> > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
> >
> > > >
> > > > $ cd fuego/
> > > > $ ./install.sh
> > > > $ ./fuego-host-scripts/docker-create-container.sh
> > > > $ ./fuego-host-scripts/docker-start-container.sh
> > > > docker# fuego-create-node --board docker
> > > > docker# fuego-create-jobs --board docker --testplan testplan_
> > docker
> > > > --distrib nosyslogd.dist
> > > > ==
> > > >
> > > > I can see new node and new test jobs.
> > > > But, I can’t run tests like Dhrystone on Docker.
> > > >
> > > > So far, I had one small problem that is “Failed to mkdirs: /
fuego-rw
> > /
> > > > buildzone”
> > > > This reason is the permission or owner of “fuego-rw”,
> > > > so I added the write permission to “fuego-rw” diredtory.
> > >
> > > Humm You should not need to change any permissions.
> > > The script is supposed to use your user's uid/gid for the jenkins
> > daemon.
> > > However, I have only tested it on my pc which has a single user.
> > > Do you have more than one user? Could you check that the jenkins
uid/
> > gid
> > > is the same as yours? (you can check on /etc/passwd inside and
outside
> > docker)
> > > Please also check that the fuego-rw's owner is your user (e.g.:
don't
> > git clone with
> > > a different user or root)
> > >
> > > > I have solved this problem.
> > > >
> > > > After that, I got the error message like following in console
log.
> > > > ====
> > > > Started by user anonymous
> > > > Building remotely on docker in workspace /fuego-rw/buildzone
> > > > [buildzone] $ /bin/sh -xe /tmp/hudson1255554151521556820.sh
> > > > + export Reboot=false
> > > > + Reboot=false
> > > > + export Rebuild=true
> > > > + Rebuild=true
> > > > + export Target_Cleanup=true
> > > > + Target_Cleanup=true
> > > > + export TESTDIR=Benchmark.Dhrystone
> > > > + TESTDIR=Benchmark.Dhrystone
> > > > + export TESTNAME=Dhrystone
> > > > + TESTNAME=Dhrystone
> > > > + export DISTRIB=nosyslogd.dist
> > > > + DISTRIB=nosyslogd.dist
> > > > + export TESTPLAN=testplan_docker
> > > > + TESTPLAN=testplan_docker
> > > > + timeout --signal=9 3m /bin/bash
> > > > /fuego-core/engine/tests/Benchmark.Dhrystone/Dhrystone.sh
> > > > board overlay: boards/docker.board
> > > > using boards/docker.board board overlay
> > > > using nosyslogd.dist overlay
> > > > BATCH_TESTPLAN is not set, using testplan_docker.json testplan
> > > > Traceback (most recent call last):
> > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 615, in
<
> > module
> > > > >
> > > > run()
> > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 607, in
run
> > > > generateProlog(output, ofcls, classes, testPlans, testSpecs)
> > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 375, in
> > > > generateProlog
> > > > outfile = open(outFilePath, "w")
> > > > IOError: [Errno 2] No such file or directory: '/fuego-rw/work/
docker
> > _
> > > > prolog.sh'
> > > >
> > > > *** ABORTED ***
> > > >
> > > > Fuego error reason: Error while prolog.sh file generation
> > > >
> > > > Build step 'Execute shell' marked build as failure
> > > > [description-setter] Description set:
> > > > Finished: FAILURE
> > > > ====
> > > >
> > > > Did I miss to run test cases on Docker node?
> > > > If you know this solution, could you give me some advice?
> > >
> > > Could you check that fuego-rw is correctly mounted inside docker?
> > > outside-docker$ touch fuego-rw/hoge
> > > inside-docker$ ls /fuego-rw
> > > hoge <-- does it appear?
> > >
> > > On my PC (ubuntu 16.04, docker version 1.12.3 ) it's always
succesful
> > so I will
> > > try on a different one. Which OS and docker version are you using?
> > >
> > > Thanks,
> > > Daniel
> > >
> > > > Thank you.
> > > >
> > > > Best Regard.
> > > > Kengo Ibe
> > > >
> > > > ----- Original Message -----
> > > > 件名: [Fuego] RFC: Fuego's version up and other changes
> > > > 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> > > > 送信日時: 2017年01月11日(水) 09:28:07
> > > > 宛先: "Bird, Timothy" <Tim.Bird@am.sony.com>, fuego@lists.
> > > > linuxfoundation.org
> > > >
> > > > > Hi everyone,
> > > > >
> > > > > I've been working on Fuego's upgrade for a few weeks, and in
the
> > > > process I
> > > > > have made quite a few changes.
> > > > > You can check it by following these instructions. I'd really
> > > > appreciate
> > > > > your feedback:
> > > > >
> > > > > Quickstart:
> > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> >
> > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
> >
> > > >
> > > > > $ cd fuego/
> > > > > $ ./install.sh
> > > > > $ ./fuego-host-scripts/docker-create-container.sh
> > > > > $ ./fuego-host-scripts/docker-start-container.sh
> > > > > docker# fuego-create-node --board docker
> > > > > docker# fuego-create-jobs --board docker --testplan testplan
_
> > docker
> > > > > --distrib nosyslogd.dist
> > > > > Then, start the "docker.testplan_docker.batch" job which
will
> > > > trigger
> > > > > the rest of jobs.
> > > > >
> > > > > Here is a list of the most fundamental modifications I've
added:
> > > > > - I automatized the version upgrade process. So far I have
updated
> > > > Jenkins
> > > > > 3 times quite smoothly.
> > > > > The current version is the latest LTS version (2.31.1 LTS)
> > > > > - I reduced plugin dependencies to the minimum (only 2, and
they
> > can
> > > > be
> > > > > made optional).
> > > > > - I removed dependencies on all groovy scripts.
> > > > > - Userdata is gone. Instead we now have fuego-ro and fuego-rw,
> > which
> > > > > together with fuego-core
> > > > > are mounted as external docker volumes. Only fuego-rw,
which
> > > > contains
> > > > > logs and build folders,
> > > > > is mounted in RW mode. I also fixed the uid/gid of jenkins
so
> > that
> > > > it
> > > > > matches the one in the host.
> > > > > This means you can develop comfortably on your host, with
GUI
> > tools,
> > > > > and don't fear a buggy
> > > > > script deleting your folders.
> > > > > - I added a tool for generating jobs from testplans.
Creating
> > jobs
> > > > from
> > > > > testplans is powerful.
> > > > > It allows you, for example, to easily specify the timeout
for
> > each
> > > > test
> > > > > on your board.
> > > > > The tool also generates the corresponding batch job. For
> > example,
> > > > you
> > > > > can easily trigger all
> > > > > tests for your board and a specific testplan periodically.
> > > > > [Note] This will also enable the creation of scripts for
> > comparing
> > > > test
> > > > > results across
> > > > > different boards and test parameters (on my TODO list).
> > > > >
> > > > > Other less fundamental changes include:
> > > > > - I added support for using the docker container itself as a
> > target
> > > > board
> > > > > (e.g.: for a quickstart or unit tests)
> > > > > - I fixed the flot plugin (java and javascript fixes)
> > > > > - Now you can click on a node and see which jobs (and
testplans)
> > are
> > > > > assigned to your board. This was
> > > > > one of my old feature requests and makes Fuego's GUI easier
to
> > use.
> > > > > - I added quite a few fixes and improvements to the core
engine
> > > > scripts
> > > > > and tests (too detailed to describe here).
> > > > > - I added Excel output support for IOzone
> > > > > - Faster docker build time (ARMhf compiler installation is now
> > > > optional,
> > > > > no latex..)
> > > > > - I put a fixed name to the fuego container, instead of the
> > > > > "last_container_id" file.
> > > > > - I removed the inotify script
> > > > > - I added scripts for easily removing the docker container/
image.
> > > > >
> > > > > I have to split the current "huge" patch into smaller chunks
and
> > test
> > > > it
> > > > > further but it would
> > > > > be nice to get your feedback about the approach in general.
> > > > >
> > > > > Tim: I haven't updated ftc nor your "Christmas commits" yet.
> > > > >
> > > > > Best regards
> > > > > Daniel
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Fuego mailing list
> > > > > Fuego@lists.linuxfoundation.org
> > > > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > > > >
> > > >
> > >
> > >
> > >
> > >
> >
>
>
>
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] RFC: Fuego's version up and other changes
2017-01-12 9:04 ` Ibe.Kengo
@ 2017-01-12 9:18 ` Daniel Sangorrin
0 siblings, 0 replies; 43+ messages in thread
From: Daniel Sangorrin @ 2017-01-12 9:18 UTC (permalink / raw)
To: Ibe.Kengo; +Cc: fuego
> From: Ibe.Kengo@bx.MitsubishiElectric.co.jp [mailto:Ibe.Kengo@bx.MitsubishiElectric.co.jp]
> Hi, Daniel-san,
>
> Thank you for prompt help.
> I was able to run tests in my Docker!! :)
>
> I'm going to try to use some function of your Fuego.
> If I have some question, let me ask you some question.
Ok, that's great!
I'll be glad to answer your questions or ideas for improving it.
Thanks,
Daniel
> ----- Original Message -----
> 件名: RE: [Fuego] RFC: Fuego's version up and other changes
> 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> 送信日時: 2017年01月12日(木) 17:45:20
> 宛先: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> Cc: fuego@lists.linuxfoundation.org
>
> > Hi Kengo-san,
> >
> > I'm sorry, it seems that fuego-rw didn't get pushed. The reason is
> because it's included in .gitignore!!
> > I have pushed it now. Please remove your current fuego-rw and do a git
> pull. It should work properly this time.
> >
> > Thanks
> > Daniel
> >
> >
> > > -----Original Message-----
> > > From: Ibe.Kengo@bx.MitsubishiElectric.co.jp [mailto:Ibe.Kengo@bx.
> MitsubishiElectric.co.jp]
> > > Sent: Thursday, January 12, 2017 5:16 PM
> > > To: Daniel Sangorrin
> > > Cc: fuego@lists.linuxfoundation.org
> > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > >
> > > Hi, Daniel-san
> > >
> > > Thank you for your prompt reply.
> > > I have more a few question.
> > >
> > > > Humm You should not need to change any permissions.
> > > OK.
> > > I removed the Fuego and the Fuego-core,
> > > after that, I redownloaded the Fuego and the Fuego-core and
> rebuilded
> > > and start Docker.
> > > I can see following permissions of directories. Is it correct?
> > > ==
> > > melco@debian8:~/$ ./fuego/fuego-host-scripts/docker-start-container.
> sh
> > > Starting Fuego container (fuego-container)
> > > [ ok ] Starting Jenkins Continuous Integration Server: jenkins.
> > >
> > > root@debian8:/# ls -al
> > > total 84
> > > drwxr-xr-x 65 root root 4096 Jan 12 07:26 .
> > > drwxr-xr-x 65 root root 4096 Jan 12 07:26 ..
> > > -rwxr-xr-x 1 root root 0 Jan 12 07:26 .dockerenv
> > > drwxr-xr-x 2 root root 4096 Jan 12 00:53 bin
> > > drwxr-xr-x 2 root root 4096 Sep 12 04:09 boot
> > > drwxr-xr-x 5 root root 380 Jan 12 07:26 dev
> > > drwxr-xr-x 96 root root 4096 Jan 12 07:26 etc
> > > drwxr-xr-x 4 jenkins jenkins 4096 Jan 12 07:07 fuego-core
> > > drwxr-xr-x 5 jenkins jenkins 4096 Jan 12 07:03 fuego-ro
> > > drwxr-xr-x 2 root root 4096 Jan 12 07:26 fuego-rw
> > > drwxr-xr-x 2 root root 4096 Sep 12 04:09 home
> > > ~
> > >
> > > melco@debian8:~/$ ls -al fuego
> > > 合計 68
> > > drwxr-xr-x 8 melco melco 4096 1月 12 16:26 .
> > > drwxr-xr-x 4 melco melco 4096 1月 12 16:02 ..
> > > -rw-r--r-- 1 melco melco 42 1月 12 16:03 .dockerignore
> > > drwxr-xr-x 8 melco melco 4096 1月 12 16:03 .git
> > > -rw-r--r-- 1 melco melco 9 1月 12 16:03 .gitignore
> > > -rw-r--r-- 1 melco melco 1482 1月 12 16:03 CHANGELOG
> > > -rw-r--r-- 1 melco melco 4810 1月 12 16:03 Dockerfile
> > > -rw-r--r-- 1 melco melco 1492 1月 12 16:03 LICENSE
> > > -rw-r--r-- 1 melco melco 3414 1月 12 16:03 README
> > > -rw-r--r-- 1 melco melco 85 1月 12 16:03 VERSION
> > > drwxr-xr-x 3 melco melco 4096 1月 12 16:03 docs
> > > drwxr-xr-x 3 melco melco 4096 1月 12 16:03 frontend-install
> > > drwxr-xr-x 2 melco melco 4096 1月 12 16:03 fuego-host-scripts
> > > drwxr-xr-x 5 melco melco 4096 1月 12 16:03 fuego-ro
> > > drwxr-xr-x 2 root root 4096 1月 12 16:26 fuego-rw
> > > -rwxr-xr-x 1 melco melco 48 1月 12 16:03 install.sh
> > >
> > > ==
> > > “melco” user is only user on host PC which is Debian8.
> > > I’m concerned about fuego-rw permissions and owner.
> > >
> > > > The script is supposed to use your user's uid/gid for the jenkins
> > > daemon.
> > > Do you mean that I need to make “Jenkins” user on host PC?
> > > Or
> > > Can I use “melco” user inside Docker?
> > >
> > > > However, I have only tested it on my pc which has a single user.
> > > > Do you have more than one user?
> > > I have one user. I'v not made "jenkins" user on host PC.
> > >
> > > I’ll check other point following your advice.
> > > Thank you so much.
> > > Kengo
> > >
> > >
> > > ----- Original Message -----
> > > 件名: RE: [Fuego] RFC: Fuego's version up and other changes
> > > 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> > > 送信日時: 2017年01月12日(木) 15:44:34
> > > 宛先: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > Cc: fuego@lists.linuxfoundation.org
> > >
> > > > Hi Kengo-san,
> > > >
> > > > Thanks a lot for your feedback.
> > > >
> > > > > -----Original Message-----
> > > > > From: Ibe.Kengo@bx.MitsubishiElectric.co.jp [mailto:Ibe.Kengo@bx.
> > > MitsubishiElectric.co.jp]
> > > > > Sent: Thursday, January 12, 2017 2:52 PM
> > > > > To: Daniel Sangorrin
> > > > > Cc: fuego@lists.linuxfoundation.org
> > > > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > > > >
> > > > > Hi, Daniel-san.
> > > > >
> > > > > I’m very interested in your work!
> > > > > So I tried to run your build scripts to create Docker image
> today,
> > > > > then I could run new Fuego.
> > > > > And, I added Docker node and test cases to use following command.
> > > > > ==
> > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
>
> > >
> > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
>
> > >
> > > > >
> > > > > $ cd fuego/
> > > > > $ ./install.sh
> > > > > $ ./fuego-host-scripts/docker-create-container.sh
> > > > > $ ./fuego-host-scripts/docker-start-container.sh
> > > > > docker# fuego-create-node --board docker
> > > > > docker# fuego-create-jobs --board docker --testplan testplan_
> > > docker
> > > > > --distrib nosyslogd.dist
> > > > > ==
> > > > >
> > > > > I can see new node and new test jobs.
> > > > > But, I can’t run tests like Dhrystone on Docker.
> > > > >
> > > > > So far, I had one small problem that is “Failed to mkdirs: /
> fuego-rw
> > > /
> > > > > buildzone”
> > > > > This reason is the permission or owner of “fuego-rw”,
> > > > > so I added the write permission to “fuego-rw” diredtory.
> > > >
> > > > Humm You should not need to change any permissions.
> > > > The script is supposed to use your user's uid/gid for the jenkins
> > > daemon.
> > > > However, I have only tested it on my pc which has a single user.
> > > > Do you have more than one user? Could you check that the jenkins
> uid/
> > > gid
> > > > is the same as yours? (you can check on /etc/passwd inside and
> outside
> > > docker)
> > > > Please also check that the fuego-rw's owner is your user (e.g.:
> don't
> > > git clone with
> > > > a different user or root)
> > > >
> > > > > I have solved this problem.
> > > > >
> > > > > After that, I got the error message like following in console
> log.
> > > > > ====
> > > > > Started by user anonymous
> > > > > Building remotely on docker in workspace /fuego-rw/buildzone
> > > > > [buildzone] $ /bin/sh -xe /tmp/hudson1255554151521556820.sh
> > > > > + export Reboot=false
> > > > > + Reboot=false
> > > > > + export Rebuild=true
> > > > > + Rebuild=true
> > > > > + export Target_Cleanup=true
> > > > > + Target_Cleanup=true
> > > > > + export TESTDIR=Benchmark.Dhrystone
> > > > > + TESTDIR=Benchmark.Dhrystone
> > > > > + export TESTNAME=Dhrystone
> > > > > + TESTNAME=Dhrystone
> > > > > + export DISTRIB=nosyslogd.dist
> > > > > + DISTRIB=nosyslogd.dist
> > > > > + export TESTPLAN=testplan_docker
> > > > > + TESTPLAN=testplan_docker
> > > > > + timeout --signal=9 3m /bin/bash
> > > > > /fuego-core/engine/tests/Benchmark.Dhrystone/Dhrystone.sh
> > > > > board overlay: boards/docker.board
> > > > > using boards/docker.board board overlay
> > > > > using nosyslogd.dist overlay
> > > > > BATCH_TESTPLAN is not set, using testplan_docker.json testplan
> > > > > Traceback (most recent call last):
> > > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 615, in
> <
> > > module
> > > > > >
> > > > > run()
> > > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 607, in
> run
> > > > > generateProlog(output, ofcls, classes, testPlans, testSpecs)
> > > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 375, in
> > > > > generateProlog
> > > > > outfile = open(outFilePath, "w")
> > > > > IOError: [Errno 2] No such file or directory: '/fuego-rw/work/
> docker
> > > _
> > > > > prolog.sh'
> > > > >
> > > > > *** ABORTED ***
> > > > >
> > > > > Fuego error reason: Error while prolog.sh file generation
> > > > >
> > > > > Build step 'Execute shell' marked build as failure
> > > > > [description-setter] Description set:
> > > > > Finished: FAILURE
> > > > > ====
> > > > >
> > > > > Did I miss to run test cases on Docker node?
> > > > > If you know this solution, could you give me some advice?
> > > >
> > > > Could you check that fuego-rw is correctly mounted inside docker?
> > > > outside-docker$ touch fuego-rw/hoge
> > > > inside-docker$ ls /fuego-rw
> > > > hoge <-- does it appear?
> > > >
> > > > On my PC (ubuntu 16.04, docker version 1.12.3 ) it's always
> succesful
> > > so I will
> > > > try on a different one. Which OS and docker version are you using?
> > > >
> > > > Thanks,
> > > > Daniel
> > > >
> > > > > Thank you.
> > > > >
> > > > > Best Regard.
> > > > > Kengo Ibe
> > > > >
> > > > > ----- Original Message -----
> > > > > 件名: [Fuego] RFC: Fuego's version up and other changes
> > > > > 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> > > > > 送信日時: 2017年01月11日(水) 09:28:07
> > > > > 宛先: "Bird, Timothy" <Tim.Bird@am.sony.com>, fuego@lists.
> > > > > linuxfoundation.org
> > > > >
> > > > > > Hi everyone,
> > > > > >
> > > > > > I've been working on Fuego's upgrade for a few weeks, and in
> the
> > > > > process I
> > > > > > have made quite a few changes.
> > > > > > You can check it by following these instructions. I'd really
> > > > > appreciate
> > > > > > your feedback:
> > > > > >
> > > > > > Quickstart:
> > > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
>
> > >
> > > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
>
> > >
> > > > >
> > > > > > $ cd fuego/
> > > > > > $ ./install.sh
> > > > > > $ ./fuego-host-scripts/docker-create-container.sh
> > > > > > $ ./fuego-host-scripts/docker-start-container.sh
> > > > > > docker# fuego-create-node --board docker
> > > > > > docker# fuego-create-jobs --board docker --testplan testplan
> _
> > > docker
> > > > > > --distrib nosyslogd.dist
> > > > > > Then, start the "docker.testplan_docker.batch" job which
> will
> > > > > trigger
> > > > > > the rest of jobs.
> > > > > >
> > > > > > Here is a list of the most fundamental modifications I've
> added:
> > > > > > - I automatized the version upgrade process. So far I have
> updated
> > > > > Jenkins
> > > > > > 3 times quite smoothly.
> > > > > > The current version is the latest LTS version (2.31.1 LTS)
> > > > > > - I reduced plugin dependencies to the minimum (only 2, and
> they
> > > can
> > > > > be
> > > > > > made optional).
> > > > > > - I removed dependencies on all groovy scripts.
> > > > > > - Userdata is gone. Instead we now have fuego-ro and fuego-rw,
> > > which
> > > > > > together with fuego-core
> > > > > > are mounted as external docker volumes. Only fuego-rw,
> which
> > > > > contains
> > > > > > logs and build folders,
> > > > > > is mounted in RW mode. I also fixed the uid/gid of jenkins
> so
> > > that
> > > > > it
> > > > > > matches the one in the host.
> > > > > > This means you can develop comfortably on your host, with
> GUI
> > > tools,
> > > > > > and don't fear a buggy
> > > > > > script deleting your folders.
> > > > > > - I added a tool for generating jobs from testplans.
> Creating
> > > jobs
> > > > > from
> > > > > > testplans is powerful.
> > > > > > It allows you, for example, to easily specify the timeout
> for
> > > each
> > > > > test
> > > > > > on your board.
> > > > > > The tool also generates the corresponding batch job. For
> > > example,
> > > > > you
> > > > > > can easily trigger all
> > > > > > tests for your board and a specific testplan periodically.
> > > > > > [Note] This will also enable the creation of scripts for
> > > comparing
> > > > > test
> > > > > > results across
> > > > > > different boards and test parameters (on my TODO list).
> > > > > >
> > > > > > Other less fundamental changes include:
> > > > > > - I added support for using the docker container itself as a
> > > target
> > > > > board
> > > > > > (e.g.: for a quickstart or unit tests)
> > > > > > - I fixed the flot plugin (java and javascript fixes)
> > > > > > - Now you can click on a node and see which jobs (and
> testplans)
> > > are
> > > > > > assigned to your board. This was
> > > > > > one of my old feature requests and makes Fuego's GUI easier
> to
> > > use.
> > > > > > - I added quite a few fixes and improvements to the core
> engine
> > > > > scripts
> > > > > > and tests (too detailed to describe here).
> > > > > > - I added Excel output support for IOzone
> > > > > > - Faster docker build time (ARMhf compiler installation is now
> > > > > optional,
> > > > > > no latex..)
> > > > > > - I put a fixed name to the fuego container, instead of the
> > > > > > "last_container_id" file.
> > > > > > - I removed the inotify script
> > > > > > - I added scripts for easily removing the docker container/
> image.
> > > > > >
> > > > > > I have to split the current "huge" patch into smaller chunks
> and
> > > test
> > > > > it
> > > > > > further but it would
> > > > > > be nice to get your feedback about the approach in general.
> > > > > >
> > > > > > Tim: I haven't updated ftc nor your "Christmas commits" yet.
> > > > > >
> > > > > > Best regards
> > > > > > Daniel
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > Fuego mailing list
> > > > > > Fuego@lists.linuxfoundation.org
> > > > > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> >
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] RFC: Fuego's version up and other changes
2017-01-12 6:44 ` Daniel Sangorrin
2017-01-12 8:15 ` Ibe.Kengo
@ 2017-01-13 1:02 ` Bird, Timothy
2017-01-13 1:29 ` Daniel Sangorrin
1 sibling, 1 reply; 43+ messages in thread
From: Bird, Timothy @ 2017-01-13 1:02 UTC (permalink / raw)
To: Daniel Sangorrin, Ibe.Kengo; +Cc: fuego
Daniel,
Thanks for all this work. I'm having some issues, which may be related to the user and group, so
I'm replying to the message rather than start a new thread.
Basically I'm having two problems, both during ./install.sh.
1) I very often have the debian update and extra file installation fail (Step 10 in the Docker image build).
(noninteractive apt-get update && apt-get -yV install ...)
There are 352 'get' operations during this secondary installation, and if even a single one fails,
the docker build fails. I kept having problems getting all the way through this step. I would
have network failures from the debian servers. It seemed to be intermittent, but overall it
required that I try to build the docker container about 10 times for it to get past that step.
Here's a sample set of messages:
Get:95 http://httpredir.debian.org/debian/ jessie/main libatk-wrapper-java all 0.30.5-1 [30.3 kB]
Err http://httpredir.debian.org/debian/ jessie/main libavahi-glib1 amd64 0.6.31-5
Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
And, at the end of the failed docker build, this:
-------
Fetched 231 MB in 1min 43s (2226 kB/s)
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/a/avahi/libavahi-glib1_0.6.31-5_amd64.deb Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
The command '/bin/sh -c DEBIAN_FRONTEND=noninteractive apt-get update && apt-get -yV install apt-utils daemon gcc make python-paramiko python-lxml python-simplejson python-matplotlib libtool xmlstarlet autoconf automake rsync openjdk-7-jre openjdk-7-jdk iperf netperf netpipe-tcp sshpass wget git diffstat sudo net-tools vim curl inotify-tools python-openpyxl g++ bzip2 bc libaio-dev gettext pkg-config libglib2.0-dev' returned a non-zero code: 100
--------
It was not the same package failing each time - but the times it failed some of the
same packages occurred (so, it's hard to pin it on the package or the server).
Maybe this is just some bad networking at my site, but it did cause me problems.
(Or maybe the Debian servers are just having a bad day today).
More problematical is a hang at Step 21:
Removing intermediate container a522905e9fc2
Step 21 : RUN groupadd -g ${gid} ${group} && useradd -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
---> Running in 106f8bf82d61
The build just hangs there, every time. The dockerd is running on my desktop,
and consuming 110% cpu.
I'm not sure if this is hung in the 'groupadd' step or the 'wget -nv ${JENKINS_URL}' step.
I'm on Ubuntu 14.04.5 LTS using docker version 1.12.1.
dockerd appears to be stuck waiting for a futex.
I'm not sure that the line 'ARG uid=1000' is going to work in my case. Here's my setup:
$ id
uid=2060932290(1000072457) gid=2060452353(domain_users) groups=2060452353(domain_users),4(adm),20(dialout),24(cdrom),27(sudo),46(plugdev),108(lpadmin),124(sambashare),999(docker)
Don’t ask - this is due to my Linux machine being previously administered by corporate types.
By the way, when I kill the build, dockerd continues to dominate my CPU usage in top
(and I can hear my desktop cpu fan running).
I think this might be a docker bug, but I'm not sure.
I'll play around with the Dockerfile (maybe altering the user configuration), but I thought I should report my status so far.
Any ideas?
-- Tim
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Wednesday, January 11, 2017 10:45 PM
> To: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> Cc: fuego@lists.linuxfoundation.org
> Subject: Re: [Fuego] RFC: Fuego's version up and other changes
>
> Hi Kengo-san,
>
> Thanks a lot for your feedback.
>
> > -----Original Message-----
> > From: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> [mailto:Ibe.Kengo@bx.MitsubishiElectric.co.jp]
> > Sent: Thursday, January 12, 2017 2:52 PM
> > To: Daniel Sangorrin
> > Cc: fuego@lists.linuxfoundation.org
> > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> >
> > Hi, Daniel-san.
> >
> > I’m very interested in your work!
> > So I tried to run your build scripts to create Docker image today,
> > then I could run new Fuego.
> > And, I added Docker node and test cases to use following command.
> > ==
> > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
> >
> > $ cd fuego/
> > $ ./install.sh
> > $ ./fuego-host-scripts/docker-create-container.sh
> > $ ./fuego-host-scripts/docker-start-container.sh
> > docker# fuego-create-node --board docker
> > docker# fuego-create-jobs --board docker --testplan testplan_docker
> > --distrib nosyslogd.dist
> > ==
> >
> > I can see new node and new test jobs.
> > But, I can’t run tests like Dhrystone on Docker.
> >
> > So far, I had one small problem that is “Failed to mkdirs: /fuego-rw/
> > buildzone”
> > This reason is the permission or owner of “fuego-rw”,
> > so I added the write permission to “fuego-rw” diredtory.
>
> Humm You should not need to change any permissions.
> The script is supposed to use your user's uid/gid for the jenkins daemon.
> However, I have only tested it on my pc which has a single user.
> Do you have more than one user? Could you check that the jenkins uid/gid
> is the same as yours? (you can check on /etc/passwd inside and outside
> docker)
> Please also check that the fuego-rw's owner is your user (e.g.: don't git clone
> with
> a different user or root)
>
> > I have solved this problem.
> >
> > After that, I got the error message like following in console log.
> > ====
> > Started by user anonymous
> > Building remotely on docker in workspace /fuego-rw/buildzone
> > [buildzone] $ /bin/sh -xe /tmp/hudson1255554151521556820.sh
> > + export Reboot=false
> > + Reboot=false
> > + export Rebuild=true
> > + Rebuild=true
> > + export Target_Cleanup=true
> > + Target_Cleanup=true
> > + export TESTDIR=Benchmark.Dhrystone
> > + TESTDIR=Benchmark.Dhrystone
> > + export TESTNAME=Dhrystone
> > + TESTNAME=Dhrystone
> > + export DISTRIB=nosyslogd.dist
> > + DISTRIB=nosyslogd.dist
> > + export TESTPLAN=testplan_docker
> > + TESTPLAN=testplan_docker
> > + timeout --signal=9 3m /bin/bash
> > /fuego-core/engine/tests/Benchmark.Dhrystone/Dhrystone.sh
> > board overlay: boards/docker.board
> > using boards/docker.board board overlay
> > using nosyslogd.dist overlay
> > BATCH_TESTPLAN is not set, using testplan_docker.json testplan
> > Traceback (most recent call last):
> > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 615, in <module
> > >
> > run()
> > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 607, in run
> > generateProlog(output, ofcls, classes, testPlans, testSpecs)
> > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 375, in
> > generateProlog
> > outfile = open(outFilePath, "w")
> > IOError: [Errno 2] No such file or directory: '/fuego-rw/work/docker_
> > prolog.sh'
> >
> > *** ABORTED ***
> >
> > Fuego error reason: Error while prolog.sh file generation
> >
> > Build step 'Execute shell' marked build as failure
> > [description-setter] Description set:
> > Finished: FAILURE
> > ====
> >
> > Did I miss to run test cases on Docker node?
> > If you know this solution, could you give me some advice?
>
> Could you check that fuego-rw is correctly mounted inside docker?
> outside-docker$ touch fuego-rw/hoge
> inside-docker$ ls /fuego-rw
> hoge <-- does it appear?
>
> On my PC (ubuntu 16.04, docker version 1.12.3 ) it's always succesful so I will
> try on a different one. Which OS and docker version are you using?
>
> Thanks,
> Daniel
>
> > Thank you.
> >
> > Best Regard.
> > Kengo Ibe
> >
> > ----- Original Message -----
> > 件名: [Fuego] RFC: Fuego's version up and other changes
> > 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> > 送信日時: 2017年01月11日(水) 09:28:07
> > 宛先: "Bird, Timothy" <Tim.Bird@am.sony.com>, fuego@lists.
> > linuxfoundation.org
> >
> > > Hi everyone,
> > >
> > > I've been working on Fuego's upgrade for a few weeks, and in the
> > process I
> > > have made quite a few changes.
> > > You can check it by following these instructions. I'd really
> > appreciate
> > > your feedback:
> > >
> > > Quickstart:
> > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
> >
> > > $ cd fuego/
> > > $ ./install.sh
> > > $ ./fuego-host-scripts/docker-create-container.sh
> > > $ ./fuego-host-scripts/docker-start-container.sh
> > > docker# fuego-create-node --board docker
> > > docker# fuego-create-jobs --board docker --testplan testplan_docker
> > > --distrib nosyslogd.dist
> > > Then, start the "docker.testplan_docker.batch" job which will
> > trigger
> > > the rest of jobs.
> > >
> > > Here is a list of the most fundamental modifications I've added:
> > > - I automatized the version upgrade process. So far I have updated
> > Jenkins
> > > 3 times quite smoothly.
> > > The current version is the latest LTS version (2.31.1 LTS)
> > > - I reduced plugin dependencies to the minimum (only 2, and they can
> > be
> > > made optional).
> > > - I removed dependencies on all groovy scripts.
> > > - Userdata is gone. Instead we now have fuego-ro and fuego-rw, which
> > > together with fuego-core
> > > are mounted as external docker volumes. Only fuego-rw, which
> > contains
> > > logs and build folders,
> > > is mounted in RW mode. I also fixed the uid/gid of jenkins so that
> > it
> > > matches the one in the host.
> > > This means you can develop comfortably on your host, with GUI tools,
> > > and don't fear a buggy
> > > script deleting your folders.
> > > - I added a tool for generating jobs from testplans. Creating jobs
> > from
> > > testplans is powerful.
> > > It allows you, for example, to easily specify the timeout for each
> > test
> > > on your board.
> > > The tool also generates the corresponding batch job. For example,
> > you
> > > can easily trigger all
> > > tests for your board and a specific testplan periodically.
> > > [Note] This will also enable the creation of scripts for comparing
> > test
> > > results across
> > > different boards and test parameters (on my TODO list).
> > >
> > > Other less fundamental changes include:
> > > - I added support for using the docker container itself as a target
> > board
> > > (e.g.: for a quickstart or unit tests)
> > > - I fixed the flot plugin (java and javascript fixes)
> > > - Now you can click on a node and see which jobs (and testplans) are
> > > assigned to your board. This was
> > > one of my old feature requests and makes Fuego's GUI easier to use.
> > > - I added quite a few fixes and improvements to the core engine
> > scripts
> > > and tests (too detailed to describe here).
> > > - I added Excel output support for IOzone
> > > - Faster docker build time (ARMhf compiler installation is now
> > optional,
> > > no latex..)
> > > - I put a fixed name to the fuego container, instead of the
> > > "last_container_id" file.
> > > - I removed the inotify script
> > > - I added scripts for easily removing the docker container/image.
> > >
> > > I have to split the current "huge" patch into smaller chunks and test
> > it
> > > further but it would
> > > be nice to get your feedback about the approach in general.
> > >
> > > Tim: I haven't updated ftc nor your "Christmas commits" yet.
> > >
> > > Best regards
> > > Daniel
> > >
> > >
> > > _______________________________________________
> > > Fuego mailing list
> > > Fuego@lists.linuxfoundation.org
> > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > >
> >
>
>
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] RFC: Fuego's version up and other changes
2017-01-13 1:02 ` Bird, Timothy
@ 2017-01-13 1:29 ` Daniel Sangorrin
2017-01-13 1:37 ` Bird, Timothy
0 siblings, 1 reply; 43+ messages in thread
From: Daniel Sangorrin @ 2017-01-13 1:29 UTC (permalink / raw)
To: 'Bird, Timothy', Ibe.Kengo; +Cc: fuego
Hi Tim,
> -----Original Message-----
> From: Bird, Timothy [mailto:Tim.Bird@am.sony.com]
> Sent: Friday, January 13, 2017 10:02 AM
> To: Daniel Sangorrin; Ibe.Kengo@bx.MitsubishiElectric.co.jp
> Cc: fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] RFC: Fuego's version up and other changes
>
> Daniel,
>
> Thanks for all this work. I'm having some issues, which may be related to the user and group, so
> I'm replying to the message rather than start a new thread.
>
> Basically I'm having two problems, both during ./install.sh.
>
> 1) I very often have the debian update and extra file installation fail (Step 10 in the Docker image build).
> (noninteractive apt-get update && apt-get -yV install ...)
>
> There are 352 'get' operations during this secondary installation, and if even a single one fails,
> the docker build fails. I kept having problems getting all the way through this step. I would
> have network failures from the debian servers. It seemed to be intermittent, but overall it
> required that I try to build the docker container about 10 times for it to get past that step.
> Here's a sample set of messages:
>
> Get:95 http://httpredir.debian.org/debian/ jessie/main libatk-wrapper-java all 0.30.5-1 [30.3 kB]
> Err http://httpredir.debian.org/debian/ jessie/main libavahi-glib1 amd64 0.6.31-5
> Error reading from server. Remote end closed connection [IP: 5.153.231.35 80]
Yes, I had the same problem yesterday when I installed it on a different machine.
It seems that httpredir.debian.org is not as reliable as using a local mirror directly.
I'm not sure if I should add retries or put a default debian server as it used to be.
Could you change httpredir by the US mirror (http://ftp.us.debian.org/debian) and try again?
> And, at the end of the failed docker build, this:
> -------
> Fetched 231 MB in 1min 43s (2226 kB/s)
> E: Failed to fetch http://httpredir.debian.org/debian/pool/main/a/avahi/libavahi-glib1_0.6.31-5_amd64.deb Error reading from
> server. Remote end closed connection [IP: 5.153.231.35 80]
>
> E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
> The command '/bin/sh -c DEBIAN_FRONTEND=noninteractive apt-get update && apt-get -yV install apt-utils daemon gcc make
> python-paramiko python-lxml python-simplejson python-matplotlib libtool xmlstarlet autoconf automake rsync openjdk-7-jre
> openjdk-7-jdk iperf netperf netpipe-tcp sshpass wget git diffstat sudo net-tools vim curl inotify-tools python-openpyxl g++ bzip2
> bc libaio-dev gettext pkg-config libglib2.0-dev' returned a non-zero code: 100
> --------
>
> It was not the same package failing each time - but the times it failed some of the
> same packages occurred (so, it's hard to pin it on the package or the server).
>
> Maybe this is just some bad networking at my site, but it did cause me problems.
> (Or maybe the Debian servers are just having a bad day today).
>
> More problematical is a hang at Step 21:
> Removing intermediate container a522905e9fc2
> Step 21 : RUN groupadd -g ${gid} ${group} && useradd -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
> ---> Running in 106f8bf82d61
>
> The build just hangs there, every time. The dockerd is running on my desktop,
> and consuming 110% cpu.
>
> I'm not sure if this is hung in the 'groupadd' step or the 'wget -nv ${JENKINS_URL}' step.
>
> I'm on Ubuntu 14.04.5 LTS using docker version 1.12.1.
The wget does take a couple of minutes. But I hadn't experienced the groupadd hang.
> dockerd appears to be stuck waiting for a futex.
>
> I'm not sure that the line 'ARG uid=1000' is going to work in my case. Here's my setup:
> $ id
> uid=2060932290(1000072457) gid=2060452353(domain_users)
> groups=2060452353(domain_users),4(adm),20(dialout),24(cdrom),27(sudo),46(plugdev),108(lpadmin),124(sambashare),999(docker)
>
> Don’t ask - this is due to my Linux machine being previously administered by corporate types.
Ok, I think I know what the problem is. I had assumed that the gid would be the same as the uid
in the general case, and other users could just pass their uid/gid in the docker-build-image.sh
script. I have fixed docker-build-image.sh and added the definition of gid.
Please, could you do a git pull and try again?
https://bitbucket.org/nirrognas/fuego/commits/9d8d1426c326575af8467f73bce7773d1f45a76c
> By the way, when I kill the build, dockerd continues to dominate my CPU usage in top
> (and I can hear my desktop cpu fan running).
>
> I think this might be a docker bug, but I'm not sure.
Sorry about that!
Thanks for your feedback!!
Daniel
>
> I'll play around with the Dockerfile (maybe altering the user configuration), but I thought I should report my status so far.
>
> Any ideas?
> -- Tim
>
> > -----Original Message-----
> > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> > bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> > Sent: Wednesday, January 11, 2017 10:45 PM
> > To: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > Cc: fuego@lists.linuxfoundation.org
> > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> >
> > Hi Kengo-san,
> >
> > Thanks a lot for your feedback.
> >
> > > -----Original Message-----
> > > From: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > [mailto:Ibe.Kengo@bx.MitsubishiElectric.co.jp]
> > > Sent: Thursday, January 12, 2017 2:52 PM
> > > To: Daniel Sangorrin
> > > Cc: fuego@lists.linuxfoundation.org
> > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > >
> > > Hi, Daniel-san.
> > >
> > > I’m very interested in your work!
> > > So I tried to run your build scripts to create Docker image today,
> > > then I could run new Fuego.
> > > And, I added Docker node and test cases to use following command.
> > > ==
> > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
> > >
> > > $ cd fuego/
> > > $ ./install.sh
> > > $ ./fuego-host-scripts/docker-create-container.sh
> > > $ ./fuego-host-scripts/docker-start-container.sh
> > > docker# fuego-create-node --board docker
> > > docker# fuego-create-jobs --board docker --testplan testplan_docker
> > > --distrib nosyslogd.dist
> > > ==
> > >
> > > I can see new node and new test jobs.
> > > But, I can’t run tests like Dhrystone on Docker.
> > >
> > > So far, I had one small problem that is “Failed to mkdirs: /fuego-rw/
> > > buildzone”
> > > This reason is the permission or owner of “fuego-rw”,
> > > so I added the write permission to “fuego-rw” diredtory.
> >
> > Humm You should not need to change any permissions.
> > The script is supposed to use your user's uid/gid for the jenkins daemon.
> > However, I have only tested it on my pc which has a single user.
> > Do you have more than one user? Could you check that the jenkins uid/gid
> > is the same as yours? (you can check on /etc/passwd inside and outside
> > docker)
> > Please also check that the fuego-rw's owner is your user (e.g.: don't git clone
> > with
> > a different user or root)
> >
> > > I have solved this problem.
> > >
> > > After that, I got the error message like following in console log.
> > > ====
> > > Started by user anonymous
> > > Building remotely on docker in workspace /fuego-rw/buildzone
> > > [buildzone] $ /bin/sh -xe /tmp/hudson1255554151521556820.sh
> > > + export Reboot=false
> > > + Reboot=false
> > > + export Rebuild=true
> > > + Rebuild=true
> > > + export Target_Cleanup=true
> > > + Target_Cleanup=true
> > > + export TESTDIR=Benchmark.Dhrystone
> > > + TESTDIR=Benchmark.Dhrystone
> > > + export TESTNAME=Dhrystone
> > > + TESTNAME=Dhrystone
> > > + export DISTRIB=nosyslogd.dist
> > > + DISTRIB=nosyslogd.dist
> > > + export TESTPLAN=testplan_docker
> > > + TESTPLAN=testplan_docker
> > > + timeout --signal=9 3m /bin/bash
> > > /fuego-core/engine/tests/Benchmark.Dhrystone/Dhrystone.sh
> > > board overlay: boards/docker.board
> > > using boards/docker.board board overlay
> > > using nosyslogd.dist overlay
> > > BATCH_TESTPLAN is not set, using testplan_docker.json testplan
> > > Traceback (most recent call last):
> > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 615, in <module
> > > >
> > > run()
> > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 607, in run
> > > generateProlog(output, ofcls, classes, testPlans, testSpecs)
> > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 375, in
> > > generateProlog
> > > outfile = open(outFilePath, "w")
> > > IOError: [Errno 2] No such file or directory: '/fuego-rw/work/docker_
> > > prolog.sh'
> > >
> > > *** ABORTED ***
> > >
> > > Fuego error reason: Error while prolog.sh file generation
> > >
> > > Build step 'Execute shell' marked build as failure
> > > [description-setter] Description set:
> > > Finished: FAILURE
> > > ====
> > >
> > > Did I miss to run test cases on Docker node?
> > > If you know this solution, could you give me some advice?
> >
> > Could you check that fuego-rw is correctly mounted inside docker?
> > outside-docker$ touch fuego-rw/hoge
> > inside-docker$ ls /fuego-rw
> > hoge <-- does it appear?
> >
> > On my PC (ubuntu 16.04, docker version 1.12.3 ) it's always succesful so I will
> > try on a different one. Which OS and docker version are you using?
> >
> > Thanks,
> > Daniel
> >
> > > Thank you.
> > >
> > > Best Regard.
> > > Kengo Ibe
> > >
> > > ----- Original Message -----
> > > 件名: [Fuego] RFC: Fuego's version up and other changes
> > > 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> > > 送信日時: 2017年01月11日(水) 09:28:07
> > > 宛先: "Bird, Timothy" <Tim.Bird@am.sony.com>, fuego@lists.
> > > linuxfoundation.org
> > >
> > > > Hi everyone,
> > > >
> > > > I've been working on Fuego's upgrade for a few weeks, and in the
> > > process I
> > > > have made quite a few changes.
> > > > You can check it by following these instructions. I'd really
> > > appreciate
> > > > your feedback:
> > > >
> > > > Quickstart:
> > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-core.git
> > >
> > > > $ cd fuego/
> > > > $ ./install.sh
> > > > $ ./fuego-host-scripts/docker-create-container.sh
> > > > $ ./fuego-host-scripts/docker-start-container.sh
> > > > docker# fuego-create-node --board docker
> > > > docker# fuego-create-jobs --board docker --testplan testplan_docker
> > > > --distrib nosyslogd.dist
> > > > Then, start the "docker.testplan_docker.batch" job which will
> > > trigger
> > > > the rest of jobs.
> > > >
> > > > Here is a list of the most fundamental modifications I've added:
> > > > - I automatized the version upgrade process. So far I have updated
> > > Jenkins
> > > > 3 times quite smoothly.
> > > > The current version is the latest LTS version (2.31.1 LTS)
> > > > - I reduced plugin dependencies to the minimum (only 2, and they can
> > > be
> > > > made optional).
> > > > - I removed dependencies on all groovy scripts.
> > > > - Userdata is gone. Instead we now have fuego-ro and fuego-rw, which
> > > > together with fuego-core
> > > > are mounted as external docker volumes. Only fuego-rw, which
> > > contains
> > > > logs and build folders,
> > > > is mounted in RW mode. I also fixed the uid/gid of jenkins so that
> > > it
> > > > matches the one in the host.
> > > > This means you can develop comfortably on your host, with GUI tools,
> > > > and don't fear a buggy
> > > > script deleting your folders.
> > > > - I added a tool for generating jobs from testplans. Creating jobs
> > > from
> > > > testplans is powerful.
> > > > It allows you, for example, to easily specify the timeout for each
> > > test
> > > > on your board.
> > > > The tool also generates the corresponding batch job. For example,
> > > you
> > > > can easily trigger all
> > > > tests for your board and a specific testplan periodically.
> > > > [Note] This will also enable the creation of scripts for comparing
> > > test
> > > > results across
> > > > different boards and test parameters (on my TODO list).
> > > >
> > > > Other less fundamental changes include:
> > > > - I added support for using the docker container itself as a target
> > > board
> > > > (e.g.: for a quickstart or unit tests)
> > > > - I fixed the flot plugin (java and javascript fixes)
> > > > - Now you can click on a node and see which jobs (and testplans) are
> > > > assigned to your board. This was
> > > > one of my old feature requests and makes Fuego's GUI easier to use.
> > > > - I added quite a few fixes and improvements to the core engine
> > > scripts
> > > > and tests (too detailed to describe here).
> > > > - I added Excel output support for IOzone
> > > > - Faster docker build time (ARMhf compiler installation is now
> > > optional,
> > > > no latex..)
> > > > - I put a fixed name to the fuego container, instead of the
> > > > "last_container_id" file.
> > > > - I removed the inotify script
> > > > - I added scripts for easily removing the docker container/image.
> > > >
> > > > I have to split the current "huge" patch into smaller chunks and test
> > > it
> > > > further but it would
> > > > be nice to get your feedback about the approach in general.
> > > >
> > > > Tim: I haven't updated ftc nor your "Christmas commits" yet.
> > > >
> > > > Best regards
> > > > Daniel
> > > >
> > > >
> > > > _______________________________________________
> > > > Fuego mailing list
> > > > Fuego@lists.linuxfoundation.org
> > > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > > >
> > >
> >
> >
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] RFC: Fuego's version up and other changes
2017-01-13 1:29 ` Daniel Sangorrin
@ 2017-01-13 1:37 ` Bird, Timothy
2017-01-13 1:45 ` Daniel Sangorrin
0 siblings, 1 reply; 43+ messages in thread
From: Bird, Timothy @ 2017-01-13 1:37 UTC (permalink / raw)
To: Daniel Sangorrin, Ibe.Kengo; +Cc: fuego
I just pulled it. Is this right?
if [ "$(id -u)" == "0" ]; then
JENKINS_UID=$(id -u $SUDO_USER)
+ JENKINS_GID=$(id -g $SUDO_USER)
else
JENKINS_UID=$(id -u $USER)
+ JENKINS_GID=$(id -g $SUDO_USER)
fi
I'm not sure exactly what's going on, but shouldn't that second
changed line be?
JENKINS_GID=$(id -g $USER)
-- Tim
> -----Original Message-----
> From: Daniel Sangorrin [mailto:daniel.sangorrin@toshiba.co.jp]
> Sent: Thursday, January 12, 2017 5:30 PM
> To: Bird, Timothy <Tim.Bird@am.sony.com>;
> Ibe.Kengo@bx.MitsubishiElectric.co.jp
> Cc: fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] RFC: Fuego's version up and other changes
>
> Hi Tim,
>
> > -----Original Message-----
> > From: Bird, Timothy [mailto:Tim.Bird@am.sony.com]
> > Sent: Friday, January 13, 2017 10:02 AM
> > To: Daniel Sangorrin; Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > Cc: fuego@lists.linuxfoundation.org
> > Subject: RE: [Fuego] RFC: Fuego's version up and other changes
> >
> > Daniel,
> >
> > Thanks for all this work. I'm having some issues, which may be related to
> the user and group, so
> > I'm replying to the message rather than start a new thread.
> >
> > Basically I'm having two problems, both during ./install.sh.
> >
> > 1) I very often have the debian update and extra file installation fail (Step
> 10 in the Docker image build).
> > (noninteractive apt-get update && apt-get -yV install ...)
> >
> > There are 352 'get' operations during this secondary installation, and if even
> a single one fails,
> > the docker build fails. I kept having problems getting all the way through
> this step. I would
> > have network failures from the debian servers. It seemed to be
> intermittent, but overall it
> > required that I try to build the docker container about 10 times for it to get
> past that step.
> > Here's a sample set of messages:
> >
> > Get:95 http://httpredir.debian.org/debian/ jessie/main libatk-wrapper-
> java all 0.30.5-1 [30.3 kB]
> > Err http://httpredir.debian.org/debian/ jessie/main libavahi-glib1 amd64
> 0.6.31-5
> > Error reading from server. Remote end closed connection [IP: 5.153.231.35
> 80]
>
> Yes, I had the same problem yesterday when I installed it on a different
> machine.
> It seems that httpredir.debian.org is not as reliable as using a local mirror
> directly.
> I'm not sure if I should add retries or put a default debian server as it used to
> be.
> Could you change httpredir by the US mirror
> (http://ftp.us.debian.org/debian) and try again?
>
> > And, at the end of the failed docker build, this:
> > -------
> > Fetched 231 MB in 1min 43s (2226 kB/s)
> > E: Failed to fetch
> http://httpredir.debian.org/debian/pool/main/a/avahi/libavahi-glib1_0.6.31-
> 5_amd64.deb Error reading from
> > server. Remote end closed connection [IP: 5.153.231.35 80]
> >
> > E: Unable to fetch some archives, maybe run apt-get update or try with --
> fix-missing?
> > The command '/bin/sh -c DEBIAN_FRONTEND=noninteractive apt-get
> update && apt-get -yV install apt-utils daemon gcc make
> > python-paramiko python-lxml python-simplejson python-matplotlib
> libtool xmlstarlet autoconf automake rsync openjdk-7-jre
> > openjdk-7-jdk iperf netperf netpipe-tcp sshpass wget git diffstat sudo
> net-tools vim curl inotify-tools python-openpyxl g++ bzip2
> > bc libaio-dev gettext pkg-config libglib2.0-dev' returned a non-zero code:
> 100
> > --------
> >
> > It was not the same package failing each time - but the times it failed some
> of the
> > same packages occurred (so, it's hard to pin it on the package or the
> server).
> >
> > Maybe this is just some bad networking at my site, but it did cause me
> problems.
> > (Or maybe the Debian servers are just having a bad day today).
> >
> > More problematical is a hang at Step 21:
> > Removing intermediate container a522905e9fc2
> > Step 21 : RUN groupadd -g ${gid} ${group} && useradd -m -d
> "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
> > ---> Running in 106f8bf82d61
> >
> > The build just hangs there, every time. The dockerd is running on my
> desktop,
> > and consuming 110% cpu.
> >
> > I'm not sure if this is hung in the 'groupadd' step or the 'wget -nv
> ${JENKINS_URL}' step.
> >
> > I'm on Ubuntu 14.04.5 LTS using docker version 1.12.1.
>
> The wget does take a couple of minutes. But I hadn't experienced the
> groupadd hang.
>
> > dockerd appears to be stuck waiting for a futex.
> >
> > I'm not sure that the line 'ARG uid=1000' is going to work in my case. Here's
> my setup:
> > $ id
> > uid=2060932290(1000072457) gid=2060452353(domain_users)
> >
> groups=2060452353(domain_users),4(adm),20(dialout),24(cdrom),27(sudo),
> 46(plugdev),108(lpadmin),124(sambashare),999(docker)
> >
> > Don’t ask - this is due to my Linux machine being previously administered
> by corporate types.
>
> Ok, I think I know what the problem is. I had assumed that the gid would be
> the same as the uid
> in the general case, and other users could just pass their uid/gid in the
> docker-build-image.sh
> script. I have fixed docker-build-image.sh and added the definition of gid.
> Please, could you do a git pull and try again?
>
> https://bitbucket.org/nirrognas/fuego/commits/9d8d1426c326575af8467f73
> bce7773d1f45a76c
>
> > By the way, when I kill the build, dockerd continues to dominate my CPU
> usage in top
> > (and I can hear my desktop cpu fan running).
> >
> > I think this might be a docker bug, but I'm not sure.
>
> Sorry about that!
>
> Thanks for your feedback!!
> Daniel
>
> >
> > I'll play around with the Dockerfile (maybe altering the user configuration),
> but I thought I should report my status so far.
> >
> > Any ideas?
> > -- Tim
> >
> > > -----Original Message-----
> > > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> > > bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> > > Sent: Wednesday, January 11, 2017 10:45 PM
> > > To: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > Cc: fuego@lists.linuxfoundation.org
> > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > >
> > > Hi Kengo-san,
> > >
> > > Thanks a lot for your feedback.
> > >
> > > > -----Original Message-----
> > > > From: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > [mailto:Ibe.Kengo@bx.MitsubishiElectric.co.jp]
> > > > Sent: Thursday, January 12, 2017 2:52 PM
> > > > To: Daniel Sangorrin
> > > > Cc: fuego@lists.linuxfoundation.org
> > > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > > >
> > > > Hi, Daniel-san.
> > > >
> > > > I’m very interested in your work!
> > > > So I tried to run your build scripts to create Docker image today,
> > > > then I could run new Fuego.
> > > > And, I added Docker node and test cases to use following command.
> > > > ==
> > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-
> core.git
> > > >
> > > > $ cd fuego/
> > > > $ ./install.sh
> > > > $ ./fuego-host-scripts/docker-create-container.sh
> > > > $ ./fuego-host-scripts/docker-start-container.sh
> > > > docker# fuego-create-node --board docker
> > > > docker# fuego-create-jobs --board docker --testplan testplan_docker
> > > > --distrib nosyslogd.dist
> > > > ==
> > > >
> > > > I can see new node and new test jobs.
> > > > But, I can’t run tests like Dhrystone on Docker.
> > > >
> > > > So far, I had one small problem that is “Failed to mkdirs: /fuego-rw/
> > > > buildzone”
> > > > This reason is the permission or owner of “fuego-rw”,
> > > > so I added the write permission to “fuego-rw” diredtory.
> > >
> > > Humm You should not need to change any permissions.
> > > The script is supposed to use your user's uid/gid for the jenkins daemon.
> > > However, I have only tested it on my pc which has a single user.
> > > Do you have more than one user? Could you check that the jenkins
> uid/gid
> > > is the same as yours? (you can check on /etc/passwd inside and outside
> > > docker)
> > > Please also check that the fuego-rw's owner is your user (e.g.: don't git
> clone
> > > with
> > > a different user or root)
> > >
> > > > I have solved this problem.
> > > >
> > > > After that, I got the error message like following in console log.
> > > > ====
> > > > Started by user anonymous
> > > > Building remotely on docker in workspace /fuego-rw/buildzone
> > > > [buildzone] $ /bin/sh -xe /tmp/hudson1255554151521556820.sh
> > > > + export Reboot=false
> > > > + Reboot=false
> > > > + export Rebuild=true
> > > > + Rebuild=true
> > > > + export Target_Cleanup=true
> > > > + Target_Cleanup=true
> > > > + export TESTDIR=Benchmark.Dhrystone
> > > > + TESTDIR=Benchmark.Dhrystone
> > > > + export TESTNAME=Dhrystone
> > > > + TESTNAME=Dhrystone
> > > > + export DISTRIB=nosyslogd.dist
> > > > + DISTRIB=nosyslogd.dist
> > > > + export TESTPLAN=testplan_docker
> > > > + TESTPLAN=testplan_docker
> > > > + timeout --signal=9 3m /bin/bash
> > > > /fuego-core/engine/tests/Benchmark.Dhrystone/Dhrystone.sh
> > > > board overlay: boards/docker.board
> > > > using boards/docker.board board overlay
> > > > using nosyslogd.dist overlay
> > > > BATCH_TESTPLAN is not set, using testplan_docker.json testplan
> > > > Traceback (most recent call last):
> > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 615, in
> <module
> > > > >
> > > > run()
> > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 607, in run
> > > > generateProlog(output, ofcls, classes, testPlans, testSpecs)
> > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 375, in
> > > > generateProlog
> > > > outfile = open(outFilePath, "w")
> > > > IOError: [Errno 2] No such file or directory: '/fuego-rw/work/docker_
> > > > prolog.sh'
> > > >
> > > > *** ABORTED ***
> > > >
> > > > Fuego error reason: Error while prolog.sh file generation
> > > >
> > > > Build step 'Execute shell' marked build as failure
> > > > [description-setter] Description set:
> > > > Finished: FAILURE
> > > > ====
> > > >
> > > > Did I miss to run test cases on Docker node?
> > > > If you know this solution, could you give me some advice?
> > >
> > > Could you check that fuego-rw is correctly mounted inside docker?
> > > outside-docker$ touch fuego-rw/hoge
> > > inside-docker$ ls /fuego-rw
> > > hoge <-- does it appear?
> > >
> > > On my PC (ubuntu 16.04, docker version 1.12.3 ) it's always succesful so I
> will
> > > try on a different one. Which OS and docker version are you using?
> > >
> > > Thanks,
> > > Daniel
> > >
> > > > Thank you.
> > > >
> > > > Best Regard.
> > > > Kengo Ibe
> > > >
> > > > ----- Original Message -----
> > > > 件名: [Fuego] RFC: Fuego's version up and other changes
> > > > 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> > > > 送信日時: 2017年01月11日(水) 09:28:07
> > > > 宛先: "Bird, Timothy" <Tim.Bird@am.sony.com>, fuego@lists.
> > > > linuxfoundation.org
> > > >
> > > > > Hi everyone,
> > > > >
> > > > > I've been working on Fuego's upgrade for a few weeks, and in the
> > > > process I
> > > > > have made quite a few changes.
> > > > > You can check it by following these instructions. I'd really
> > > > appreciate
> > > > > your feedback:
> > > > >
> > > > > Quickstart:
> > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-
> core.git
> > > >
> > > > > $ cd fuego/
> > > > > $ ./install.sh
> > > > > $ ./fuego-host-scripts/docker-create-container.sh
> > > > > $ ./fuego-host-scripts/docker-start-container.sh
> > > > > docker# fuego-create-node --board docker
> > > > > docker# fuego-create-jobs --board docker --testplan
> testplan_docker
> > > > > --distrib nosyslogd.dist
> > > > > Then, start the "docker.testplan_docker.batch" job which will
> > > > trigger
> > > > > the rest of jobs.
> > > > >
> > > > > Here is a list of the most fundamental modifications I've added:
> > > > > - I automatized the version upgrade process. So far I have updated
> > > > Jenkins
> > > > > 3 times quite smoothly.
> > > > > The current version is the latest LTS version (2.31.1 LTS)
> > > > > - I reduced plugin dependencies to the minimum (only 2, and they can
> > > > be
> > > > > made optional).
> > > > > - I removed dependencies on all groovy scripts.
> > > > > - Userdata is gone. Instead we now have fuego-ro and fuego-rw,
> which
> > > > > together with fuego-core
> > > > > are mounted as external docker volumes. Only fuego-rw, which
> > > > contains
> > > > > logs and build folders,
> > > > > is mounted in RW mode. I also fixed the uid/gid of jenkins so that
> > > > it
> > > > > matches the one in the host.
> > > > > This means you can develop comfortably on your host, with GUI
> tools,
> > > > > and don't fear a buggy
> > > > > script deleting your folders.
> > > > > - I added a tool for generating jobs from testplans. Creating jobs
> > > > from
> > > > > testplans is powerful.
> > > > > It allows you, for example, to easily specify the timeout for each
> > > > test
> > > > > on your board.
> > > > > The tool also generates the corresponding batch job. For example,
> > > > you
> > > > > can easily trigger all
> > > > > tests for your board and a specific testplan periodically.
> > > > > [Note] This will also enable the creation of scripts for comparing
> > > > test
> > > > > results across
> > > > > different boards and test parameters (on my TODO list).
> > > > >
> > > > > Other less fundamental changes include:
> > > > > - I added support for using the docker container itself as a target
> > > > board
> > > > > (e.g.: for a quickstart or unit tests)
> > > > > - I fixed the flot plugin (java and javascript fixes)
> > > > > - Now you can click on a node and see which jobs (and testplans) are
> > > > > assigned to your board. This was
> > > > > one of my old feature requests and makes Fuego's GUI easier to
> use.
> > > > > - I added quite a few fixes and improvements to the core engine
> > > > scripts
> > > > > and tests (too detailed to describe here).
> > > > > - I added Excel output support for IOzone
> > > > > - Faster docker build time (ARMhf compiler installation is now
> > > > optional,
> > > > > no latex..)
> > > > > - I put a fixed name to the fuego container, instead of the
> > > > > "last_container_id" file.
> > > > > - I removed the inotify script
> > > > > - I added scripts for easily removing the docker container/image.
> > > > >
> > > > > I have to split the current "huge" patch into smaller chunks and test
> > > > it
> > > > > further but it would
> > > > > be nice to get your feedback about the approach in general.
> > > > >
> > > > > Tim: I haven't updated ftc nor your "Christmas commits" yet.
> > > > >
> > > > > Best regards
> > > > > Daniel
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Fuego mailing list
> > > > > Fuego@lists.linuxfoundation.org
> > > > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > > > >
> > > >
> > >
> > >
> > >
> > > _______________________________________________
> > > Fuego mailing list
> > > Fuego@lists.linuxfoundation.org
> > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] RFC: Fuego's version up and other changes
2017-01-13 1:37 ` Bird, Timothy
@ 2017-01-13 1:45 ` Daniel Sangorrin
2017-01-13 1:57 ` Bird, Timothy
0 siblings, 1 reply; 43+ messages in thread
From: Daniel Sangorrin @ 2017-01-13 1:45 UTC (permalink / raw)
To: 'Bird, Timothy', Ibe.Kengo; +Cc: fuego
oops, sorry I wrote it too quickly. I just fixed it.
> -----Original Message-----
> From: Bird, Timothy [mailto:Tim.Bird@am.sony.com]
> Sent: Friday, January 13, 2017 10:38 AM
> To: Daniel Sangorrin; Ibe.Kengo@bx.MitsubishiElectric.co.jp
> Cc: fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] RFC: Fuego's version up and other changes
>
> I just pulled it. Is this right?
>
> if [ "$(id -u)" == "0" ]; then
> JENKINS_UID=$(id -u $SUDO_USER)
> + JENKINS_GID=$(id -g $SUDO_USER)
> else
> JENKINS_UID=$(id -u $USER)
> + JENKINS_GID=$(id -g $SUDO_USER)
> fi
>
> I'm not sure exactly what's going on, but shouldn't that second
> changed line be?
> JENKINS_GID=$(id -g $USER)
>
> -- Tim
>
> > -----Original Message-----
> > From: Daniel Sangorrin [mailto:daniel.sangorrin@toshiba.co.jp]
> > Sent: Thursday, January 12, 2017 5:30 PM
> > To: Bird, Timothy <Tim.Bird@am.sony.com>;
> > Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > Cc: fuego@lists.linuxfoundation.org
> > Subject: RE: [Fuego] RFC: Fuego's version up and other changes
> >
> > Hi Tim,
> >
> > > -----Original Message-----
> > > From: Bird, Timothy [mailto:Tim.Bird@am.sony.com]
> > > Sent: Friday, January 13, 2017 10:02 AM
> > > To: Daniel Sangorrin; Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > Cc: fuego@lists.linuxfoundation.org
> > > Subject: RE: [Fuego] RFC: Fuego's version up and other changes
> > >
> > > Daniel,
> > >
> > > Thanks for all this work. I'm having some issues, which may be related to
> > the user and group, so
> > > I'm replying to the message rather than start a new thread.
> > >
> > > Basically I'm having two problems, both during ./install.sh.
> > >
> > > 1) I very often have the debian update and extra file installation fail (Step
> > 10 in the Docker image build).
> > > (noninteractive apt-get update && apt-get -yV install ...)
> > >
> > > There are 352 'get' operations during this secondary installation, and if even
> > a single one fails,
> > > the docker build fails. I kept having problems getting all the way through
> > this step. I would
> > > have network failures from the debian servers. It seemed to be
> > intermittent, but overall it
> > > required that I try to build the docker container about 10 times for it to get
> > past that step.
> > > Here's a sample set of messages:
> > >
> > > Get:95 http://httpredir.debian.org/debian/ jessie/main libatk-wrapper-
> > java all 0.30.5-1 [30.3 kB]
> > > Err http://httpredir.debian.org/debian/ jessie/main libavahi-glib1 amd64
> > 0.6.31-5
> > > Error reading from server. Remote end closed connection [IP: 5.153.231.35
> > 80]
> >
> > Yes, I had the same problem yesterday when I installed it on a different
> > machine.
> > It seems that httpredir.debian.org is not as reliable as using a local mirror
> > directly.
> > I'm not sure if I should add retries or put a default debian server as it used to
> > be.
> > Could you change httpredir by the US mirror
> > (http://ftp.us.debian.org/debian) and try again?
> >
> > > And, at the end of the failed docker build, this:
> > > -------
> > > Fetched 231 MB in 1min 43s (2226 kB/s)
> > > E: Failed to fetch
> > http://httpredir.debian.org/debian/pool/main/a/avahi/libavahi-glib1_0.6.31-
> > 5_amd64.deb Error reading from
> > > server. Remote end closed connection [IP: 5.153.231.35 80]
> > >
> > > E: Unable to fetch some archives, maybe run apt-get update or try with --
> > fix-missing?
> > > The command '/bin/sh -c DEBIAN_FRONTEND=noninteractive apt-get
> > update && apt-get -yV install apt-utils daemon gcc make
> > > python-paramiko python-lxml python-simplejson python-matplotlib
> > libtool xmlstarlet autoconf automake rsync openjdk-7-jre
> > > openjdk-7-jdk iperf netperf netpipe-tcp sshpass wget git diffstat sudo
> > net-tools vim curl inotify-tools python-openpyxl g++ bzip2
> > > bc libaio-dev gettext pkg-config libglib2.0-dev' returned a non-zero code:
> > 100
> > > --------
> > >
> > > It was not the same package failing each time - but the times it failed some
> > of the
> > > same packages occurred (so, it's hard to pin it on the package or the
> > server).
> > >
> > > Maybe this is just some bad networking at my site, but it did cause me
> > problems.
> > > (Or maybe the Debian servers are just having a bad day today).
> > >
> > > More problematical is a hang at Step 21:
> > > Removing intermediate container a522905e9fc2
> > > Step 21 : RUN groupadd -g ${gid} ${group} && useradd -m -d
> > "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
> > > ---> Running in 106f8bf82d61
> > >
> > > The build just hangs there, every time. The dockerd is running on my
> > desktop,
> > > and consuming 110% cpu.
> > >
> > > I'm not sure if this is hung in the 'groupadd' step or the 'wget -nv
> > ${JENKINS_URL}' step.
> > >
> > > I'm on Ubuntu 14.04.5 LTS using docker version 1.12.1.
> >
> > The wget does take a couple of minutes. But I hadn't experienced the
> > groupadd hang.
> >
> > > dockerd appears to be stuck waiting for a futex.
> > >
> > > I'm not sure that the line 'ARG uid=1000' is going to work in my case. Here's
> > my setup:
> > > $ id
> > > uid=2060932290(1000072457) gid=2060452353(domain_users)
> > >
> > groups=2060452353(domain_users),4(adm),20(dialout),24(cdrom),27(sudo),
> > 46(plugdev),108(lpadmin),124(sambashare),999(docker)
> > >
> > > Don’t ask - this is due to my Linux machine being previously administered
> > by corporate types.
> >
> > Ok, I think I know what the problem is. I had assumed that the gid would be
> > the same as the uid
> > in the general case, and other users could just pass their uid/gid in the
> > docker-build-image.sh
> > script. I have fixed docker-build-image.sh and added the definition of gid.
> > Please, could you do a git pull and try again?
> >
> > https://bitbucket.org/nirrognas/fuego/commits/9d8d1426c326575af8467f73
> > bce7773d1f45a76c
> >
> > > By the way, when I kill the build, dockerd continues to dominate my CPU
> > usage in top
> > > (and I can hear my desktop cpu fan running).
> > >
> > > I think this might be a docker bug, but I'm not sure.
> >
> > Sorry about that!
> >
> > Thanks for your feedback!!
> > Daniel
> >
> > >
> > > I'll play around with the Dockerfile (maybe altering the user configuration),
> > but I thought I should report my status so far.
> > >
> > > Any ideas?
> > > -- Tim
> > >
> > > > -----Original Message-----
> > > > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> > > > bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> > > > Sent: Wednesday, January 11, 2017 10:45 PM
> > > > To: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > > Cc: fuego@lists.linuxfoundation.org
> > > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > > >
> > > > Hi Kengo-san,
> > > >
> > > > Thanks a lot for your feedback.
> > > >
> > > > > -----Original Message-----
> > > > > From: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > > [mailto:Ibe.Kengo@bx.MitsubishiElectric.co.jp]
> > > > > Sent: Thursday, January 12, 2017 2:52 PM
> > > > > To: Daniel Sangorrin
> > > > > Cc: fuego@lists.linuxfoundation.org
> > > > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > > > >
> > > > > Hi, Daniel-san.
> > > > >
> > > > > I’m very interested in your work!
> > > > > So I tried to run your build scripts to create Docker image today,
> > > > > then I could run new Fuego.
> > > > > And, I added Docker node and test cases to use following command.
> > > > > ==
> > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-
> > core.git
> > > > >
> > > > > $ cd fuego/
> > > > > $ ./install.sh
> > > > > $ ./fuego-host-scripts/docker-create-container.sh
> > > > > $ ./fuego-host-scripts/docker-start-container.sh
> > > > > docker# fuego-create-node --board docker
> > > > > docker# fuego-create-jobs --board docker --testplan testplan_docker
> > > > > --distrib nosyslogd.dist
> > > > > ==
> > > > >
> > > > > I can see new node and new test jobs.
> > > > > But, I can’t run tests like Dhrystone on Docker.
> > > > >
> > > > > So far, I had one small problem that is “Failed to mkdirs: /fuego-rw/
> > > > > buildzone”
> > > > > This reason is the permission or owner of “fuego-rw”,
> > > > > so I added the write permission to “fuego-rw” diredtory.
> > > >
> > > > Humm You should not need to change any permissions.
> > > > The script is supposed to use your user's uid/gid for the jenkins daemon.
> > > > However, I have only tested it on my pc which has a single user.
> > > > Do you have more than one user? Could you check that the jenkins
> > uid/gid
> > > > is the same as yours? (you can check on /etc/passwd inside and outside
> > > > docker)
> > > > Please also check that the fuego-rw's owner is your user (e.g.: don't git
> > clone
> > > > with
> > > > a different user or root)
> > > >
> > > > > I have solved this problem.
> > > > >
> > > > > After that, I got the error message like following in console log.
> > > > > ====
> > > > > Started by user anonymous
> > > > > Building remotely on docker in workspace /fuego-rw/buildzone
> > > > > [buildzone] $ /bin/sh -xe /tmp/hudson1255554151521556820.sh
> > > > > + export Reboot=false
> > > > > + Reboot=false
> > > > > + export Rebuild=true
> > > > > + Rebuild=true
> > > > > + export Target_Cleanup=true
> > > > > + Target_Cleanup=true
> > > > > + export TESTDIR=Benchmark.Dhrystone
> > > > > + TESTDIR=Benchmark.Dhrystone
> > > > > + export TESTNAME=Dhrystone
> > > > > + TESTNAME=Dhrystone
> > > > > + export DISTRIB=nosyslogd.dist
> > > > > + DISTRIB=nosyslogd.dist
> > > > > + export TESTPLAN=testplan_docker
> > > > > + TESTPLAN=testplan_docker
> > > > > + timeout --signal=9 3m /bin/bash
> > > > > /fuego-core/engine/tests/Benchmark.Dhrystone/Dhrystone.sh
> > > > > board overlay: boards/docker.board
> > > > > using boards/docker.board board overlay
> > > > > using nosyslogd.dist overlay
> > > > > BATCH_TESTPLAN is not set, using testplan_docker.json testplan
> > > > > Traceback (most recent call last):
> > > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 615, in
> > <module
> > > > > >
> > > > > run()
> > > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 607, in run
> > > > > generateProlog(output, ofcls, classes, testPlans, testSpecs)
> > > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 375, in
> > > > > generateProlog
> > > > > outfile = open(outFilePath, "w")
> > > > > IOError: [Errno 2] No such file or directory: '/fuego-rw/work/docker_
> > > > > prolog.sh'
> > > > >
> > > > > *** ABORTED ***
> > > > >
> > > > > Fuego error reason: Error while prolog.sh file generation
> > > > >
> > > > > Build step 'Execute shell' marked build as failure
> > > > > [description-setter] Description set:
> > > > > Finished: FAILURE
> > > > > ====
> > > > >
> > > > > Did I miss to run test cases on Docker node?
> > > > > If you know this solution, could you give me some advice?
> > > >
> > > > Could you check that fuego-rw is correctly mounted inside docker?
> > > > outside-docker$ touch fuego-rw/hoge
> > > > inside-docker$ ls /fuego-rw
> > > > hoge <-- does it appear?
> > > >
> > > > On my PC (ubuntu 16.04, docker version 1.12.3 ) it's always succesful so I
> > will
> > > > try on a different one. Which OS and docker version are you using?
> > > >
> > > > Thanks,
> > > > Daniel
> > > >
> > > > > Thank you.
> > > > >
> > > > > Best Regard.
> > > > > Kengo Ibe
> > > > >
> > > > > ----- Original Message -----
> > > > > 件名: [Fuego] RFC: Fuego's version up and other changes
> > > > > 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> > > > > 送信日時: 2017年01月11日(水) 09:28:07
> > > > > 宛先: "Bird, Timothy" <Tim.Bird@am.sony.com>, fuego@lists.
> > > > > linuxfoundation.org
> > > > >
> > > > > > Hi everyone,
> > > > > >
> > > > > > I've been working on Fuego's upgrade for a few weeks, and in the
> > > > > process I
> > > > > > have made quite a few changes.
> > > > > > You can check it by following these instructions. I'd really
> > > > > appreciate
> > > > > > your feedback:
> > > > > >
> > > > > > Quickstart:
> > > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> > > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-
> > core.git
> > > > >
> > > > > > $ cd fuego/
> > > > > > $ ./install.sh
> > > > > > $ ./fuego-host-scripts/docker-create-container.sh
> > > > > > $ ./fuego-host-scripts/docker-start-container.sh
> > > > > > docker# fuego-create-node --board docker
> > > > > > docker# fuego-create-jobs --board docker --testplan
> > testplan_docker
> > > > > > --distrib nosyslogd.dist
> > > > > > Then, start the "docker.testplan_docker.batch" job which will
> > > > > trigger
> > > > > > the rest of jobs.
> > > > > >
> > > > > > Here is a list of the most fundamental modifications I've added:
> > > > > > - I automatized the version upgrade process. So far I have updated
> > > > > Jenkins
> > > > > > 3 times quite smoothly.
> > > > > > The current version is the latest LTS version (2.31.1 LTS)
> > > > > > - I reduced plugin dependencies to the minimum (only 2, and they can
> > > > > be
> > > > > > made optional).
> > > > > > - I removed dependencies on all groovy scripts.
> > > > > > - Userdata is gone. Instead we now have fuego-ro and fuego-rw,
> > which
> > > > > > together with fuego-core
> > > > > > are mounted as external docker volumes. Only fuego-rw, which
> > > > > contains
> > > > > > logs and build folders,
> > > > > > is mounted in RW mode. I also fixed the uid/gid of jenkins so that
> > > > > it
> > > > > > matches the one in the host.
> > > > > > This means you can develop comfortably on your host, with GUI
> > tools,
> > > > > > and don't fear a buggy
> > > > > > script deleting your folders.
> > > > > > - I added a tool for generating jobs from testplans. Creating jobs
> > > > > from
> > > > > > testplans is powerful.
> > > > > > It allows you, for example, to easily specify the timeout for each
> > > > > test
> > > > > > on your board.
> > > > > > The tool also generates the corresponding batch job. For example,
> > > > > you
> > > > > > can easily trigger all
> > > > > > tests for your board and a specific testplan periodically.
> > > > > > [Note] This will also enable the creation of scripts for comparing
> > > > > test
> > > > > > results across
> > > > > > different boards and test parameters (on my TODO list).
> > > > > >
> > > > > > Other less fundamental changes include:
> > > > > > - I added support for using the docker container itself as a target
> > > > > board
> > > > > > (e.g.: for a quickstart or unit tests)
> > > > > > - I fixed the flot plugin (java and javascript fixes)
> > > > > > - Now you can click on a node and see which jobs (and testplans) are
> > > > > > assigned to your board. This was
> > > > > > one of my old feature requests and makes Fuego's GUI easier to
> > use.
> > > > > > - I added quite a few fixes and improvements to the core engine
> > > > > scripts
> > > > > > and tests (too detailed to describe here).
> > > > > > - I added Excel output support for IOzone
> > > > > > - Faster docker build time (ARMhf compiler installation is now
> > > > > optional,
> > > > > > no latex..)
> > > > > > - I put a fixed name to the fuego container, instead of the
> > > > > > "last_container_id" file.
> > > > > > - I removed the inotify script
> > > > > > - I added scripts for easily removing the docker container/image.
> > > > > >
> > > > > > I have to split the current "huge" patch into smaller chunks and test
> > > > > it
> > > > > > further but it would
> > > > > > be nice to get your feedback about the approach in general.
> > > > > >
> > > > > > Tim: I haven't updated ftc nor your "Christmas commits" yet.
> > > > > >
> > > > > > Best regards
> > > > > > Daniel
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > Fuego mailing list
> > > > > > Fuego@lists.linuxfoundation.org
> > > > > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Fuego mailing list
> > > > Fuego@lists.linuxfoundation.org
> > > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> >
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] RFC: Fuego's version up and other changes
2017-01-13 1:45 ` Daniel Sangorrin
@ 2017-01-13 1:57 ` Bird, Timothy
2017-01-13 2:05 ` Daniel Sangorrin
0 siblings, 1 reply; 43+ messages in thread
From: Bird, Timothy @ 2017-01-13 1:57 UTC (permalink / raw)
To: Daniel Sangorrin, Ibe.Kengo; +Cc: fuego
OK - I think there's some bug in docker. I added some debug lines to the Dockerfile,
and split the 'groupadd...' line. I found that the hang is with the 'useradd' line,
which is weird as this should be a fairly straightforward operation inside the image.
Here's what I get:
Step 20 : ENV JENKINS_HOME /var/lib/jenkins
---> Using cache
---> b26e2d078353
Step 21 : RUN echo "uid=${uid}"
---> Running in b22a2010fff2
uid=2060932290
---> 5dcc36f214af
Removing intermediate container b22a2010fff2
Step 22 : RUN echo "gid=${gid}"
---> Running in ed562da957b6
gid=2060452353
---> 228948aad0df
Removing intermediate container ed562da957b6
Step 23 : RUN groupadd -g ${gid} ${group}
---> Running in 7db50dee926e
---> 33a79d51f82e
Removing intermediate container 7db50dee926e
Step 24 : RUN useradd -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
---> Running in f75c9c00ef86
(hangs here, with dockerd at 100% of CPU)
^C1000072457@tlinux:~/work/daniel-fuego/fuego$
I'm not sure what's going on here. But if I comment out the 'RUN useradd...' line, the image
builds and I get what appear to be default (auto-generated) uid and gid numbers for the 'jenkins' user
in the image.
I'll just do that for now, and try some other operations to see if the image is working as intended.
Or will the permission difference between container and host give me more problems?
-- Tim
> -----Original Message-----
> From: Daniel Sangorrin [mailto:daniel.sangorrin@toshiba.co.jp]
> Sent: Thursday, January 12, 2017 5:45 PM
> To: Bird, Timothy <Tim.Bird@am.sony.com>;
> Ibe.Kengo@bx.MitsubishiElectric.co.jp
> Cc: fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] RFC: Fuego's version up and other changes
>
> oops, sorry I wrote it too quickly. I just fixed it.
>
> > -----Original Message-----
> > From: Bird, Timothy [mailto:Tim.Bird@am.sony.com]
> > Sent: Friday, January 13, 2017 10:38 AM
> > To: Daniel Sangorrin; Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > Cc: fuego@lists.linuxfoundation.org
> > Subject: RE: [Fuego] RFC: Fuego's version up and other changes
> >
> > I just pulled it. Is this right?
> >
> > if [ "$(id -u)" == "0" ]; then
> > JENKINS_UID=$(id -u $SUDO_USER)
> > + JENKINS_GID=$(id -g $SUDO_USER)
> > else
> > JENKINS_UID=$(id -u $USER)
> > + JENKINS_GID=$(id -g $SUDO_USER)
> > fi
> >
> > I'm not sure exactly what's going on, but shouldn't that second
> > changed line be?
> > JENKINS_GID=$(id -g $USER)
> >
> > -- Tim
> >
> > > -----Original Message-----
> > > From: Daniel Sangorrin [mailto:daniel.sangorrin@toshiba.co.jp]
> > > Sent: Thursday, January 12, 2017 5:30 PM
> > > To: Bird, Timothy <Tim.Bird@am.sony.com>;
> > > Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > Cc: fuego@lists.linuxfoundation.org
> > > Subject: RE: [Fuego] RFC: Fuego's version up and other changes
> > >
> > > Hi Tim,
> > >
> > > > -----Original Message-----
> > > > From: Bird, Timothy [mailto:Tim.Bird@am.sony.com]
> > > > Sent: Friday, January 13, 2017 10:02 AM
> > > > To: Daniel Sangorrin; Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > > Cc: fuego@lists.linuxfoundation.org
> > > > Subject: RE: [Fuego] RFC: Fuego's version up and other changes
> > > >
> > > > Daniel,
> > > >
> > > > Thanks for all this work. I'm having some issues, which may be related
> to
> > > the user and group, so
> > > > I'm replying to the message rather than start a new thread.
> > > >
> > > > Basically I'm having two problems, both during ./install.sh.
> > > >
> > > > 1) I very often have the debian update and extra file installation fail
> (Step
> > > 10 in the Docker image build).
> > > > (noninteractive apt-get update && apt-get -yV install ...)
> > > >
> > > > There are 352 'get' operations during this secondary installation, and if
> even
> > > a single one fails,
> > > > the docker build fails. I kept having problems getting all the way
> through
> > > this step. I would
> > > > have network failures from the debian servers. It seemed to be
> > > intermittent, but overall it
> > > > required that I try to build the docker container about 10 times for it to
> get
> > > past that step.
> > > > Here's a sample set of messages:
> > > >
> > > > Get:95 http://httpredir.debian.org/debian/ jessie/main libatk-wrapper-
> > > java all 0.30.5-1 [30.3 kB]
> > > > Err http://httpredir.debian.org/debian/ jessie/main libavahi-glib1
> amd64
> > > 0.6.31-5
> > > > Error reading from server. Remote end closed connection [IP:
> 5.153.231.35
> > > 80]
> > >
> > > Yes, I had the same problem yesterday when I installed it on a different
> > > machine.
> > > It seems that httpredir.debian.org is not as reliable as using a local mirror
> > > directly.
> > > I'm not sure if I should add retries or put a default debian server as it used
> to
> > > be.
> > > Could you change httpredir by the US mirror
> > > (http://ftp.us.debian.org/debian) and try again?
> > >
> > > > And, at the end of the failed docker build, this:
> > > > -------
> > > > Fetched 231 MB in 1min 43s (2226 kB/s)
> > > > E: Failed to fetch
> > > http://httpredir.debian.org/debian/pool/main/a/avahi/libavahi-
> glib1_0.6.31-
> > > 5_amd64.deb Error reading from
> > > > server. Remote end closed connection [IP: 5.153.231.35 80]
> > > >
> > > > E: Unable to fetch some archives, maybe run apt-get update or try with
> --
> > > fix-missing?
> > > > The command '/bin/sh -c DEBIAN_FRONTEND=noninteractive apt-get
> > > update && apt-get -yV install apt-utils daemon gcc make
> > > > python-paramiko python-lxml python-simplejson python-
> matplotlib
> > > libtool xmlstarlet autoconf automake rsync openjdk-7-jre
> > > > openjdk-7-jdk iperf netperf netpipe-tcp sshpass wget git diffstat
> sudo
> > > net-tools vim curl inotify-tools python-openpyxl g++ bzip2
> > > > bc libaio-dev gettext pkg-config libglib2.0-dev' returned a non-zero
> code:
> > > 100
> > > > --------
> > > >
> > > > It was not the same package failing each time - but the times it failed
> some
> > > of the
> > > > same packages occurred (so, it's hard to pin it on the package or the
> > > server).
> > > >
> > > > Maybe this is just some bad networking at my site, but it did cause me
> > > problems.
> > > > (Or maybe the Debian servers are just having a bad day today).
> > > >
> > > > More problematical is a hang at Step 21:
> > > > Removing intermediate container a522905e9fc2
> > > > Step 21 : RUN groupadd -g ${gid} ${group} && useradd -m -d
> > > "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
> > > > ---> Running in 106f8bf82d61
> > > >
> > > > The build just hangs there, every time. The dockerd is running on my
> > > desktop,
> > > > and consuming 110% cpu.
> > > >
> > > > I'm not sure if this is hung in the 'groupadd' step or the 'wget -nv
> > > ${JENKINS_URL}' step.
> > > >
> > > > I'm on Ubuntu 14.04.5 LTS using docker version 1.12.1.
> > >
> > > The wget does take a couple of minutes. But I hadn't experienced the
> > > groupadd hang.
> > >
> > > > dockerd appears to be stuck waiting for a futex.
> > > >
> > > > I'm not sure that the line 'ARG uid=1000' is going to work in my case.
> Here's
> > > my setup:
> > > > $ id
> > > > uid=2060932290(1000072457) gid=2060452353(domain_users)
> > > >
> > >
> groups=2060452353(domain_users),4(adm),20(dialout),24(cdrom),27(sudo),
> > > 46(plugdev),108(lpadmin),124(sambashare),999(docker)
> > > >
> > > > Don’t ask - this is due to my Linux machine being previously
> administered
> > > by corporate types.
> > >
> > > Ok, I think I know what the problem is. I had assumed that the gid would
> be
> > > the same as the uid
> > > in the general case, and other users could just pass their uid/gid in the
> > > docker-build-image.sh
> > > script. I have fixed docker-build-image.sh and added the definition of gid.
> > > Please, could you do a git pull and try again?
> > >
> > >
> https://bitbucket.org/nirrognas/fuego/commits/9d8d1426c326575af8467f73
> > > bce7773d1f45a76c
> > >
> > > > By the way, when I kill the build, dockerd continues to dominate my
> CPU
> > > usage in top
> > > > (and I can hear my desktop cpu fan running).
> > > >
> > > > I think this might be a docker bug, but I'm not sure.
> > >
> > > Sorry about that!
> > >
> > > Thanks for your feedback!!
> > > Daniel
> > >
> > > >
> > > > I'll play around with the Dockerfile (maybe altering the user
> configuration),
> > > but I thought I should report my status so far.
> > > >
> > > > Any ideas?
> > > > -- Tim
> > > >
> > > > > -----Original Message-----
> > > > > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> > > > > bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> > > > > Sent: Wednesday, January 11, 2017 10:45 PM
> > > > > To: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > > > Cc: fuego@lists.linuxfoundation.org
> > > > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > > > >
> > > > > Hi Kengo-san,
> > > > >
> > > > > Thanks a lot for your feedback.
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > > > [mailto:Ibe.Kengo@bx.MitsubishiElectric.co.jp]
> > > > > > Sent: Thursday, January 12, 2017 2:52 PM
> > > > > > To: Daniel Sangorrin
> > > > > > Cc: fuego@lists.linuxfoundation.org
> > > > > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > > > > >
> > > > > > Hi, Daniel-san.
> > > > > >
> > > > > > I’m very interested in your work!
> > > > > > So I tried to run your build scripts to create Docker image today,
> > > > > > then I could run new Fuego.
> > > > > > And, I added Docker node and test cases to use following
> command.
> > > > > > ==
> > > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> > > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-
> > > core.git
> > > > > >
> > > > > > $ cd fuego/
> > > > > > $ ./install.sh
> > > > > > $ ./fuego-host-scripts/docker-create-container.sh
> > > > > > $ ./fuego-host-scripts/docker-start-container.sh
> > > > > > docker# fuego-create-node --board docker
> > > > > > docker# fuego-create-jobs --board docker --testplan
> testplan_docker
> > > > > > --distrib nosyslogd.dist
> > > > > > ==
> > > > > >
> > > > > > I can see new node and new test jobs.
> > > > > > But, I can’t run tests like Dhrystone on Docker.
> > > > > >
> > > > > > So far, I had one small problem that is “Failed to mkdirs: /fuego-rw/
> > > > > > buildzone”
> > > > > > This reason is the permission or owner of “fuego-rw”,
> > > > > > so I added the write permission to “fuego-rw” diredtory.
> > > > >
> > > > > Humm You should not need to change any permissions.
> > > > > The script is supposed to use your user's uid/gid for the jenkins
> daemon.
> > > > > However, I have only tested it on my pc which has a single user.
> > > > > Do you have more than one user? Could you check that the jenkins
> > > uid/gid
> > > > > is the same as yours? (you can check on /etc/passwd inside and
> outside
> > > > > docker)
> > > > > Please also check that the fuego-rw's owner is your user (e.g.: don't
> git
> > > clone
> > > > > with
> > > > > a different user or root)
> > > > >
> > > > > > I have solved this problem.
> > > > > >
> > > > > > After that, I got the error message like following in console log.
> > > > > > ====
> > > > > > Started by user anonymous
> > > > > > Building remotely on docker in workspace /fuego-rw/buildzone
> > > > > > [buildzone] $ /bin/sh -xe /tmp/hudson1255554151521556820.sh
> > > > > > + export Reboot=false
> > > > > > + Reboot=false
> > > > > > + export Rebuild=true
> > > > > > + Rebuild=true
> > > > > > + export Target_Cleanup=true
> > > > > > + Target_Cleanup=true
> > > > > > + export TESTDIR=Benchmark.Dhrystone
> > > > > > + TESTDIR=Benchmark.Dhrystone
> > > > > > + export TESTNAME=Dhrystone
> > > > > > + TESTNAME=Dhrystone
> > > > > > + export DISTRIB=nosyslogd.dist
> > > > > > + DISTRIB=nosyslogd.dist
> > > > > > + export TESTPLAN=testplan_docker
> > > > > > + TESTPLAN=testplan_docker
> > > > > > + timeout --signal=9 3m /bin/bash
> > > > > > /fuego-core/engine/tests/Benchmark.Dhrystone/Dhrystone.sh
> > > > > > board overlay: boards/docker.board
> > > > > > using boards/docker.board board overlay
> > > > > > using nosyslogd.dist overlay
> > > > > > BATCH_TESTPLAN is not set, using testplan_docker.json testplan
> > > > > > Traceback (most recent call last):
> > > > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 615, in
> > > <module
> > > > > > >
> > > > > > run()
> > > > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 607, in run
> > > > > > generateProlog(output, ofcls, classes, testPlans, testSpecs)
> > > > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 375, in
> > > > > > generateProlog
> > > > > > outfile = open(outFilePath, "w")
> > > > > > IOError: [Errno 2] No such file or directory: '/fuego-
> rw/work/docker_
> > > > > > prolog.sh'
> > > > > >
> > > > > > *** ABORTED ***
> > > > > >
> > > > > > Fuego error reason: Error while prolog.sh file generation
> > > > > >
> > > > > > Build step 'Execute shell' marked build as failure
> > > > > > [description-setter] Description set:
> > > > > > Finished: FAILURE
> > > > > > ====
> > > > > >
> > > > > > Did I miss to run test cases on Docker node?
> > > > > > If you know this solution, could you give me some advice?
> > > > >
> > > > > Could you check that fuego-rw is correctly mounted inside docker?
> > > > > outside-docker$ touch fuego-rw/hoge
> > > > > inside-docker$ ls /fuego-rw
> > > > > hoge <-- does it appear?
> > > > >
> > > > > On my PC (ubuntu 16.04, docker version 1.12.3 ) it's always succesful
> so I
> > > will
> > > > > try on a different one. Which OS and docker version are you using?
> > > > >
> > > > > Thanks,
> > > > > Daniel
> > > > >
> > > > > > Thank you.
> > > > > >
> > > > > > Best Regard.
> > > > > > Kengo Ibe
> > > > > >
> > > > > > ----- Original Message -----
> > > > > > 件名: [Fuego] RFC: Fuego's version up and other changes
> > > > > > 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> > > > > > 送信日時: 2017年01月11日(水) 09:28:07
> > > > > > 宛先: "Bird, Timothy" <Tim.Bird@am.sony.com>, fuego@lists.
> > > > > > linuxfoundation.org
> > > > > >
> > > > > > > Hi everyone,
> > > > > > >
> > > > > > > I've been working on Fuego's upgrade for a few weeks, and in the
> > > > > > process I
> > > > > > > have made quite a few changes.
> > > > > > > You can check it by following these instructions. I'd really
> > > > > > appreciate
> > > > > > > your feedback:
> > > > > > >
> > > > > > > Quickstart:
> > > > > > > $ git clone --branch next
> https://bitbucket.org/nirrognas/fuego.git
> > > > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-
> > > core.git
> > > > > >
> > > > > > > $ cd fuego/
> > > > > > > $ ./install.sh
> > > > > > > $ ./fuego-host-scripts/docker-create-container.sh
> > > > > > > $ ./fuego-host-scripts/docker-start-container.sh
> > > > > > > docker# fuego-create-node --board docker
> > > > > > > docker# fuego-create-jobs --board docker --testplan
> > > testplan_docker
> > > > > > > --distrib nosyslogd.dist
> > > > > > > Then, start the "docker.testplan_docker.batch" job which will
> > > > > > trigger
> > > > > > > the rest of jobs.
> > > > > > >
> > > > > > > Here is a list of the most fundamental modifications I've added:
> > > > > > > - I automatized the version upgrade process. So far I have
> updated
> > > > > > Jenkins
> > > > > > > 3 times quite smoothly.
> > > > > > > The current version is the latest LTS version (2.31.1 LTS)
> > > > > > > - I reduced plugin dependencies to the minimum (only 2, and they
> can
> > > > > > be
> > > > > > > made optional).
> > > > > > > - I removed dependencies on all groovy scripts.
> > > > > > > - Userdata is gone. Instead we now have fuego-ro and fuego-rw,
> > > which
> > > > > > > together with fuego-core
> > > > > > > are mounted as external docker volumes. Only fuego-rw, which
> > > > > > contains
> > > > > > > logs and build folders,
> > > > > > > is mounted in RW mode. I also fixed the uid/gid of jenkins so
> that
> > > > > > it
> > > > > > > matches the one in the host.
> > > > > > > This means you can develop comfortably on your host, with GUI
> > > tools,
> > > > > > > and don't fear a buggy
> > > > > > > script deleting your folders.
> > > > > > > - I added a tool for generating jobs from testplans. Creating jobs
> > > > > > from
> > > > > > > testplans is powerful.
> > > > > > > It allows you, for example, to easily specify the timeout for each
> > > > > > test
> > > > > > > on your board.
> > > > > > > The tool also generates the corresponding batch job. For
> example,
> > > > > > you
> > > > > > > can easily trigger all
> > > > > > > tests for your board and a specific testplan periodically.
> > > > > > > [Note] This will also enable the creation of scripts for comparing
> > > > > > test
> > > > > > > results across
> > > > > > > different boards and test parameters (on my TODO list).
> > > > > > >
> > > > > > > Other less fundamental changes include:
> > > > > > > - I added support for using the docker container itself as a target
> > > > > > board
> > > > > > > (e.g.: for a quickstart or unit tests)
> > > > > > > - I fixed the flot plugin (java and javascript fixes)
> > > > > > > - Now you can click on a node and see which jobs (and testplans)
> are
> > > > > > > assigned to your board. This was
> > > > > > > one of my old feature requests and makes Fuego's GUI easier to
> > > use.
> > > > > > > - I added quite a few fixes and improvements to the core engine
> > > > > > scripts
> > > > > > > and tests (too detailed to describe here).
> > > > > > > - I added Excel output support for IOzone
> > > > > > > - Faster docker build time (ARMhf compiler installation is now
> > > > > > optional,
> > > > > > > no latex..)
> > > > > > > - I put a fixed name to the fuego container, instead of the
> > > > > > > "last_container_id" file.
> > > > > > > - I removed the inotify script
> > > > > > > - I added scripts for easily removing the docker container/image.
> > > > > > >
> > > > > > > I have to split the current "huge" patch into smaller chunks and
> test
> > > > > > it
> > > > > > > further but it would
> > > > > > > be nice to get your feedback about the approach in general.
> > > > > > >
> > > > > > > Tim: I haven't updated ftc nor your "Christmas commits" yet.
> > > > > > >
> > > > > > > Best regards
> > > > > > > Daniel
> > > > > > >
> > > > > > >
> > > > > > > _______________________________________________
> > > > > > > Fuego mailing list
> > > > > > > Fuego@lists.linuxfoundation.org
> > > > > > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > Fuego mailing list
> > > > > Fuego@lists.linuxfoundation.org
> > > > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > >
>
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] RFC: Fuego's version up and other changes
2017-01-13 1:57 ` Bird, Timothy
@ 2017-01-13 2:05 ` Daniel Sangorrin
2017-01-13 3:34 ` Bird, Timothy
0 siblings, 1 reply; 43+ messages in thread
From: Daniel Sangorrin @ 2017-01-13 2:05 UTC (permalink / raw)
To: 'Bird, Timothy', Ibe.Kengo; +Cc: fuego
Hi Tim,
> -----Original Message-----
> From: Bird, Timothy [mailto:Tim.Bird@am.sony.com]
> Sent: Friday, January 13, 2017 10:58 AM
> To: Daniel Sangorrin; Ibe.Kengo@bx.MitsubishiElectric.co.jp
> Cc: fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] RFC: Fuego's version up and other changes
>
> OK - I think there's some bug in docker. I added some debug lines to the Dockerfile,
> and split the 'groupadd...' line. I found that the hang is with the 'useradd' line,
> which is weird as this should be a fairly straightforward operation inside the image.
>
> Here's what I get:
> Step 20 : ENV JENKINS_HOME /var/lib/jenkins
> ---> Using cache
> ---> b26e2d078353
> Step 21 : RUN echo "uid=${uid}"
> ---> Running in b22a2010fff2
> uid=2060932290
> ---> 5dcc36f214af
> Removing intermediate container b22a2010fff2
> Step 22 : RUN echo "gid=${gid}"
> ---> Running in ed562da957b6
> gid=2060452353
> ---> 228948aad0df
> Removing intermediate container ed562da957b6
> Step 23 : RUN groupadd -g ${gid} ${group}
> ---> Running in 7db50dee926e
> ---> 33a79d51f82e
> Removing intermediate container 7db50dee926e
> Step 24 : RUN useradd -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
> ---> Running in f75c9c00ef86
> (hangs here, with dockerd at 100% of CPU)
> ^C1000072457@tlinux:~/work/daniel-fuego/fuego$
>
> I'm not sure what's going on here. But if I comment out the 'RUN useradd...' line, the image
> builds and I get what appear to be default (auto-generated) uid and gid numbers for the 'jenkins' user
> in the image.
>
It seems to be related to this (docker build hangs/crashes when useradd with large UID):
https://github.com/docker/docker/issues/5419
Can you add the -l option to useradd and see if it fixes it?
> I'll just do that for now, and try some other operations to see if the image is working as intended.
> Or will the permission difference between container and host give me more problems?
If the folders are owned by jenkins inside docker there shouldn't be any problem.
Thanks,
Daniel
> -- Tim
>
>
> > -----Original Message-----
> > From: Daniel Sangorrin [mailto:daniel.sangorrin@toshiba.co.jp]
> > Sent: Thursday, January 12, 2017 5:45 PM
> > To: Bird, Timothy <Tim.Bird@am.sony.com>;
> > Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > Cc: fuego@lists.linuxfoundation.org
> > Subject: RE: [Fuego] RFC: Fuego's version up and other changes
> >
> > oops, sorry I wrote it too quickly. I just fixed it.
> >
> > > -----Original Message-----
> > > From: Bird, Timothy [mailto:Tim.Bird@am.sony.com]
> > > Sent: Friday, January 13, 2017 10:38 AM
> > > To: Daniel Sangorrin; Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > Cc: fuego@lists.linuxfoundation.org
> > > Subject: RE: [Fuego] RFC: Fuego's version up and other changes
> > >
> > > I just pulled it. Is this right?
> > >
> > > if [ "$(id -u)" == "0" ]; then
> > > JENKINS_UID=$(id -u $SUDO_USER)
> > > + JENKINS_GID=$(id -g $SUDO_USER)
> > > else
> > > JENKINS_UID=$(id -u $USER)
> > > + JENKINS_GID=$(id -g $SUDO_USER)
> > > fi
> > >
> > > I'm not sure exactly what's going on, but shouldn't that second
> > > changed line be?
> > > JENKINS_GID=$(id -g $USER)
> > >
> > > -- Tim
> > >
> > > > -----Original Message-----
> > > > From: Daniel Sangorrin [mailto:daniel.sangorrin@toshiba.co.jp]
> > > > Sent: Thursday, January 12, 2017 5:30 PM
> > > > To: Bird, Timothy <Tim.Bird@am.sony.com>;
> > > > Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > > Cc: fuego@lists.linuxfoundation.org
> > > > Subject: RE: [Fuego] RFC: Fuego's version up and other changes
> > > >
> > > > Hi Tim,
> > > >
> > > > > -----Original Message-----
> > > > > From: Bird, Timothy [mailto:Tim.Bird@am.sony.com]
> > > > > Sent: Friday, January 13, 2017 10:02 AM
> > > > > To: Daniel Sangorrin; Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > > > Cc: fuego@lists.linuxfoundation.org
> > > > > Subject: RE: [Fuego] RFC: Fuego's version up and other changes
> > > > >
> > > > > Daniel,
> > > > >
> > > > > Thanks for all this work. I'm having some issues, which may be related
> > to
> > > > the user and group, so
> > > > > I'm replying to the message rather than start a new thread.
> > > > >
> > > > > Basically I'm having two problems, both during ./install.sh.
> > > > >
> > > > > 1) I very often have the debian update and extra file installation fail
> > (Step
> > > > 10 in the Docker image build).
> > > > > (noninteractive apt-get update && apt-get -yV install ...)
> > > > >
> > > > > There are 352 'get' operations during this secondary installation, and if
> > even
> > > > a single one fails,
> > > > > the docker build fails. I kept having problems getting all the way
> > through
> > > > this step. I would
> > > > > have network failures from the debian servers. It seemed to be
> > > > intermittent, but overall it
> > > > > required that I try to build the docker container about 10 times for it to
> > get
> > > > past that step.
> > > > > Here's a sample set of messages:
> > > > >
> > > > > Get:95 http://httpredir.debian.org/debian/ jessie/main libatk-wrapper-
> > > > java all 0.30.5-1 [30.3 kB]
> > > > > Err http://httpredir.debian.org/debian/ jessie/main libavahi-glib1
> > amd64
> > > > 0.6.31-5
> > > > > Error reading from server. Remote end closed connection [IP:
> > 5.153.231.35
> > > > 80]
> > > >
> > > > Yes, I had the same problem yesterday when I installed it on a different
> > > > machine.
> > > > It seems that httpredir.debian.org is not as reliable as using a local mirror
> > > > directly.
> > > > I'm not sure if I should add retries or put a default debian server as it used
> > to
> > > > be.
> > > > Could you change httpredir by the US mirror
> > > > (http://ftp.us.debian.org/debian) and try again?
> > > >
> > > > > And, at the end of the failed docker build, this:
> > > > > -------
> > > > > Fetched 231 MB in 1min 43s (2226 kB/s)
> > > > > E: Failed to fetch
> > > > http://httpredir.debian.org/debian/pool/main/a/avahi/libavahi-
> > glib1_0.6.31-
> > > > 5_amd64.deb Error reading from
> > > > > server. Remote end closed connection [IP: 5.153.231.35 80]
> > > > >
> > > > > E: Unable to fetch some archives, maybe run apt-get update or try with
> > --
> > > > fix-missing?
> > > > > The command '/bin/sh -c DEBIAN_FRONTEND=noninteractive apt-get
> > > > update && apt-get -yV install apt-utils daemon gcc make
> > > > > python-paramiko python-lxml python-simplejson python-
> > matplotlib
> > > > libtool xmlstarlet autoconf automake rsync openjdk-7-jre
> > > > > openjdk-7-jdk iperf netperf netpipe-tcp sshpass wget git diffstat
> > sudo
> > > > net-tools vim curl inotify-tools python-openpyxl g++ bzip2
> > > > > bc libaio-dev gettext pkg-config libglib2.0-dev' returned a non-zero
> > code:
> > > > 100
> > > > > --------
> > > > >
> > > > > It was not the same package failing each time - but the times it failed
> > some
> > > > of the
> > > > > same packages occurred (so, it's hard to pin it on the package or the
> > > > server).
> > > > >
> > > > > Maybe this is just some bad networking at my site, but it did cause me
> > > > problems.
> > > > > (Or maybe the Debian servers are just having a bad day today).
> > > > >
> > > > > More problematical is a hang at Step 21:
> > > > > Removing intermediate container a522905e9fc2
> > > > > Step 21 : RUN groupadd -g ${gid} ${group} && useradd -m -d
> > > > "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
> > > > > ---> Running in 106f8bf82d61
> > > > >
> > > > > The build just hangs there, every time. The dockerd is running on my
> > > > desktop,
> > > > > and consuming 110% cpu.
> > > > >
> > > > > I'm not sure if this is hung in the 'groupadd' step or the 'wget -nv
> > > > ${JENKINS_URL}' step.
> > > > >
> > > > > I'm on Ubuntu 14.04.5 LTS using docker version 1.12.1.
> > > >
> > > > The wget does take a couple of minutes. But I hadn't experienced the
> > > > groupadd hang.
> > > >
> > > > > dockerd appears to be stuck waiting for a futex.
> > > > >
> > > > > I'm not sure that the line 'ARG uid=1000' is going to work in my case.
> > Here's
> > > > my setup:
> > > > > $ id
> > > > > uid=2060932290(1000072457) gid=2060452353(domain_users)
> > > > >
> > > >
> > groups=2060452353(domain_users),4(adm),20(dialout),24(cdrom),27(sudo),
> > > > 46(plugdev),108(lpadmin),124(sambashare),999(docker)
> > > > >
> > > > > Don’t ask - this is due to my Linux machine being previously
> > administered
> > > > by corporate types.
> > > >
> > > > Ok, I think I know what the problem is. I had assumed that the gid would
> > be
> > > > the same as the uid
> > > > in the general case, and other users could just pass their uid/gid in the
> > > > docker-build-image.sh
> > > > script. I have fixed docker-build-image.sh and added the definition of gid.
> > > > Please, could you do a git pull and try again?
> > > >
> > > >
> > https://bitbucket.org/nirrognas/fuego/commits/9d8d1426c326575af8467f73
> > > > bce7773d1f45a76c
> > > >
> > > > > By the way, when I kill the build, dockerd continues to dominate my
> > CPU
> > > > usage in top
> > > > > (and I can hear my desktop cpu fan running).
> > > > >
> > > > > I think this might be a docker bug, but I'm not sure.
> > > >
> > > > Sorry about that!
> > > >
> > > > Thanks for your feedback!!
> > > > Daniel
> > > >
> > > > >
> > > > > I'll play around with the Dockerfile (maybe altering the user
> > configuration),
> > > > but I thought I should report my status so far.
> > > > >
> > > > > Any ideas?
> > > > > -- Tim
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> > > > > > bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> > > > > > Sent: Wednesday, January 11, 2017 10:45 PM
> > > > > > To: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > > > > Cc: fuego@lists.linuxfoundation.org
> > > > > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > > > > >
> > > > > > Hi Kengo-san,
> > > > > >
> > > > > > Thanks a lot for your feedback.
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Ibe.Kengo@bx.MitsubishiElectric.co.jp
> > > > > > [mailto:Ibe.Kengo@bx.MitsubishiElectric.co.jp]
> > > > > > > Sent: Thursday, January 12, 2017 2:52 PM
> > > > > > > To: Daniel Sangorrin
> > > > > > > Cc: fuego@lists.linuxfoundation.org
> > > > > > > Subject: Re: [Fuego] RFC: Fuego's version up and other changes
> > > > > > >
> > > > > > > Hi, Daniel-san.
> > > > > > >
> > > > > > > I’m very interested in your work!
> > > > > > > So I tried to run your build scripts to create Docker image today,
> > > > > > > then I could run new Fuego.
> > > > > > > And, I added Docker node and test cases to use following
> > command.
> > > > > > > ==
> > > > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego.git
> > > > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-
> > > > core.git
> > > > > > >
> > > > > > > $ cd fuego/
> > > > > > > $ ./install.sh
> > > > > > > $ ./fuego-host-scripts/docker-create-container.sh
> > > > > > > $ ./fuego-host-scripts/docker-start-container.sh
> > > > > > > docker# fuego-create-node --board docker
> > > > > > > docker# fuego-create-jobs --board docker --testplan
> > testplan_docker
> > > > > > > --distrib nosyslogd.dist
> > > > > > > ==
> > > > > > >
> > > > > > > I can see new node and new test jobs.
> > > > > > > But, I can’t run tests like Dhrystone on Docker.
> > > > > > >
> > > > > > > So far, I had one small problem that is “Failed to mkdirs: /fuego-rw/
> > > > > > > buildzone”
> > > > > > > This reason is the permission or owner of “fuego-rw”,
> > > > > > > so I added the write permission to “fuego-rw” diredtory.
> > > > > >
> > > > > > Humm You should not need to change any permissions.
> > > > > > The script is supposed to use your user's uid/gid for the jenkins
> > daemon.
> > > > > > However, I have only tested it on my pc which has a single user.
> > > > > > Do you have more than one user? Could you check that the jenkins
> > > > uid/gid
> > > > > > is the same as yours? (you can check on /etc/passwd inside and
> > outside
> > > > > > docker)
> > > > > > Please also check that the fuego-rw's owner is your user (e.g.: don't
> > git
> > > > clone
> > > > > > with
> > > > > > a different user or root)
> > > > > >
> > > > > > > I have solved this problem.
> > > > > > >
> > > > > > > After that, I got the error message like following in console log.
> > > > > > > ====
> > > > > > > Started by user anonymous
> > > > > > > Building remotely on docker in workspace /fuego-rw/buildzone
> > > > > > > [buildzone] $ /bin/sh -xe /tmp/hudson1255554151521556820.sh
> > > > > > > + export Reboot=false
> > > > > > > + Reboot=false
> > > > > > > + export Rebuild=true
> > > > > > > + Rebuild=true
> > > > > > > + export Target_Cleanup=true
> > > > > > > + Target_Cleanup=true
> > > > > > > + export TESTDIR=Benchmark.Dhrystone
> > > > > > > + TESTDIR=Benchmark.Dhrystone
> > > > > > > + export TESTNAME=Dhrystone
> > > > > > > + TESTNAME=Dhrystone
> > > > > > > + export DISTRIB=nosyslogd.dist
> > > > > > > + DISTRIB=nosyslogd.dist
> > > > > > > + export TESTPLAN=testplan_docker
> > > > > > > + TESTPLAN=testplan_docker
> > > > > > > + timeout --signal=9 3m /bin/bash
> > > > > > > /fuego-core/engine/tests/Benchmark.Dhrystone/Dhrystone.sh
> > > > > > > board overlay: boards/docker.board
> > > > > > > using boards/docker.board board overlay
> > > > > > > using nosyslogd.dist overlay
> > > > > > > BATCH_TESTPLAN is not set, using testplan_docker.json testplan
> > > > > > > Traceback (most recent call last):
> > > > > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 615, in
> > > > <module
> > > > > > > >
> > > > > > > run()
> > > > > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 607, in run
> > > > > > > generateProlog(output, ofcls, classes, testPlans, testSpecs)
> > > > > > > File "/fuego-core/engine/scripts/ovgen/ovgen.py", line 375, in
> > > > > > > generateProlog
> > > > > > > outfile = open(outFilePath, "w")
> > > > > > > IOError: [Errno 2] No such file or directory: '/fuego-
> > rw/work/docker_
> > > > > > > prolog.sh'
> > > > > > >
> > > > > > > *** ABORTED ***
> > > > > > >
> > > > > > > Fuego error reason: Error while prolog.sh file generation
> > > > > > >
> > > > > > > Build step 'Execute shell' marked build as failure
> > > > > > > [description-setter] Description set:
> > > > > > > Finished: FAILURE
> > > > > > > ====
> > > > > > >
> > > > > > > Did I miss to run test cases on Docker node?
> > > > > > > If you know this solution, could you give me some advice?
> > > > > >
> > > > > > Could you check that fuego-rw is correctly mounted inside docker?
> > > > > > outside-docker$ touch fuego-rw/hoge
> > > > > > inside-docker$ ls /fuego-rw
> > > > > > hoge <-- does it appear?
> > > > > >
> > > > > > On my PC (ubuntu 16.04, docker version 1.12.3 ) it's always succesful
> > so I
> > > > will
> > > > > > try on a different one. Which OS and docker version are you using?
> > > > > >
> > > > > > Thanks,
> > > > > > Daniel
> > > > > >
> > > > > > > Thank you.
> > > > > > >
> > > > > > > Best Regard.
> > > > > > > Kengo Ibe
> > > > > > >
> > > > > > > ----- Original Message -----
> > > > > > > 件名: [Fuego] RFC: Fuego's version up and other changes
> > > > > > > 送信者: "Daniel Sangorrin" <daniel.sangorrin@toshiba.co.jp>
> > > > > > > 送信日時: 2017年01月11日(水) 09:28:07
> > > > > > > 宛先: "Bird, Timothy" <Tim.Bird@am.sony.com>, fuego@lists.
> > > > > > > linuxfoundation.org
> > > > > > >
> > > > > > > > Hi everyone,
> > > > > > > >
> > > > > > > > I've been working on Fuego's upgrade for a few weeks, and in the
> > > > > > > process I
> > > > > > > > have made quite a few changes.
> > > > > > > > You can check it by following these instructions. I'd really
> > > > > > > appreciate
> > > > > > > > your feedback:
> > > > > > > >
> > > > > > > > Quickstart:
> > > > > > > > $ git clone --branch next
> > https://bitbucket.org/nirrognas/fuego.git
> > > > > > > > $ git clone --branch next https://bitbucket.org/nirrognas/fuego-
> > > > core.git
> > > > > > >
> > > > > > > > $ cd fuego/
> > > > > > > > $ ./install.sh
> > > > > > > > $ ./fuego-host-scripts/docker-create-container.sh
> > > > > > > > $ ./fuego-host-scripts/docker-start-container.sh
> > > > > > > > docker# fuego-create-node --board docker
> > > > > > > > docker# fuego-create-jobs --board docker --testplan
> > > > testplan_docker
> > > > > > > > --distrib nosyslogd.dist
> > > > > > > > Then, start the "docker.testplan_docker.batch" job which will
> > > > > > > trigger
> > > > > > > > the rest of jobs.
> > > > > > > >
> > > > > > > > Here is a list of the most fundamental modifications I've added:
> > > > > > > > - I automatized the version upgrade process. So far I have
> > updated
> > > > > > > Jenkins
> > > > > > > > 3 times quite smoothly.
> > > > > > > > The current version is the latest LTS version (2.31.1 LTS)
> > > > > > > > - I reduced plugin dependencies to the minimum (only 2, and they
> > can
> > > > > > > be
> > > > > > > > made optional).
> > > > > > > > - I removed dependencies on all groovy scripts.
> > > > > > > > - Userdata is gone. Instead we now have fuego-ro and fuego-rw,
> > > > which
> > > > > > > > together with fuego-core
> > > > > > > > are mounted as external docker volumes. Only fuego-rw, which
> > > > > > > contains
> > > > > > > > logs and build folders,
> > > > > > > > is mounted in RW mode. I also fixed the uid/gid of jenkins so
> > that
> > > > > > > it
> > > > > > > > matches the one in the host.
> > > > > > > > This means you can develop comfortably on your host, with GUI
> > > > tools,
> > > > > > > > and don't fear a buggy
> > > > > > > > script deleting your folders.
> > > > > > > > - I added a tool for generating jobs from testplans. Creating jobs
> > > > > > > from
> > > > > > > > testplans is powerful.
> > > > > > > > It allows you, for example, to easily specify the timeout for each
> > > > > > > test
> > > > > > > > on your board.
> > > > > > > > The tool also generates the corresponding batch job. For
> > example,
> > > > > > > you
> > > > > > > > can easily trigger all
> > > > > > > > tests for your board and a specific testplan periodically.
> > > > > > > > [Note] This will also enable the creation of scripts for comparing
> > > > > > > test
> > > > > > > > results across
> > > > > > > > different boards and test parameters (on my TODO list).
> > > > > > > >
> > > > > > > > Other less fundamental changes include:
> > > > > > > > - I added support for using the docker container itself as a target
> > > > > > > board
> > > > > > > > (e.g.: for a quickstart or unit tests)
> > > > > > > > - I fixed the flot plugin (java and javascript fixes)
> > > > > > > > - Now you can click on a node and see which jobs (and testplans)
> > are
> > > > > > > > assigned to your board. This was
> > > > > > > > one of my old feature requests and makes Fuego's GUI easier to
> > > > use.
> > > > > > > > - I added quite a few fixes and improvements to the core engine
> > > > > > > scripts
> > > > > > > > and tests (too detailed to describe here).
> > > > > > > > - I added Excel output support for IOzone
> > > > > > > > - Faster docker build time (ARMhf compiler installation is now
> > > > > > > optional,
> > > > > > > > no latex..)
> > > > > > > > - I put a fixed name to the fuego container, instead of the
> > > > > > > > "last_container_id" file.
> > > > > > > > - I removed the inotify script
> > > > > > > > - I added scripts for easily removing the docker container/image.
> > > > > > > >
> > > > > > > > I have to split the current "huge" patch into smaller chunks and
> > test
> > > > > > > it
> > > > > > > > further but it would
> > > > > > > > be nice to get your feedback about the approach in general.
> > > > > > > >
> > > > > > > > Tim: I haven't updated ftc nor your "Christmas commits" yet.
> > > > > > > >
> > > > > > > > Best regards
> > > > > > > > Daniel
> > > > > > > >
> > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > Fuego mailing list
> > > > > > > > Fuego@lists.linuxfoundation.org
> > > > > > > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > Fuego mailing list
> > > > > > Fuego@lists.linuxfoundation.org
> > > > > > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> > > >
> >
> >
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] RFC: Fuego's version up and other changes
2017-01-13 2:05 ` Daniel Sangorrin
@ 2017-01-13 3:34 ` Bird, Timothy
2017-01-13 4:36 ` Daniel Sangorrin
0 siblings, 1 reply; 43+ messages in thread
From: Bird, Timothy @ 2017-01-13 3:34 UTC (permalink / raw)
To: Daniel Sangorrin, Ibe.Kengo; +Cc: fuego
> -----Original Message-----
> From: Daniel Sangorrin on Thursday, January 12, 2017 6:06 PM
>
> It seems to be related to this (docker build hangs/crashes when useradd with
> large UID):
> https://github.com/docker/docker/issues/5419
>
> Can you add the -l option to useradd and see if it fixes it?
That fixes the problem. Thanks!
I'll do some more testing and tell you anything more I find.
-- Tim
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] RFC: Fuego's version up and other changes
2017-01-13 3:34 ` Bird, Timothy
@ 2017-01-13 4:36 ` Daniel Sangorrin
0 siblings, 0 replies; 43+ messages in thread
From: Daniel Sangorrin @ 2017-01-13 4:36 UTC (permalink / raw)
To: 'Bird, Timothy', Ibe.Kengo; +Cc: fuego
> -----Original Message-----
> From: Bird, Timothy [mailto:Tim.Bird@am.sony.com]
> Sent: Friday, January 13, 2017 12:35 PM
> To: Daniel Sangorrin; Ibe.Kengo@bx.MitsubishiElectric.co.jp
> Cc: fuego@lists.linuxfoundation.org
> Subject: RE: [Fuego] RFC: Fuego's version up and other changes
>
> > -----Original Message-----
> > From: Daniel Sangorrin on Thursday, January 12, 2017 6:06 PM
> >
> > It seems to be related to this (docker build hangs/crashes when useradd with
> > large UID):
> > https://github.com/docker/docker/issues/5419
> >
> > Can you add the -l option to useradd and see if it fixes it?
>
> That fixes the problem. Thanks!
>
> I'll do some more testing and tell you anything more I find.
OK, thank you!!
I'm also testing it and already found some bugs. I will push them as quickly as I can.
Thanks
Daniel
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-01-11 0:28 [Fuego] RFC: Fuego's version up and other changes Daniel Sangorrin
2017-01-12 5:51 ` Ibe.Kengo
@ 2017-01-20 0:53 ` Bird, Timothy
2017-01-20 7:09 ` Daniel Sangorrin
1 sibling, 1 reply; 43+ messages in thread
From: Bird, Timothy @ 2017-01-20 0:53 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From: Daniel Sangorrin on Tuesday, January 10, 2017 4:28 PM
> I've been working on Fuego's upgrade for a few weeks, and in the process I
> have made quite a few changes.
> You can check it by following these instructions. I'd really appreciate your
> feedback:
...
>
This is really awesome work, and I'm very excited about the improvements.
I do have some questions and comments, which I'll inline below.
> Here is a list of the most fundamental modifications I've added:
> - I automatized the version upgrade process. So far I have updated Jenkins 3
> times quite smoothly.
> The current version is the latest LTS version (2.31.1 LTS)
This is very welcome, and something that should help us keep synchronized with
Jenkins going forward.
> - I reduced plugin dependencies to the minimum (only 2, and they can be
> made optional).
This is good, but it dramatically changes the default visual look of fuego
(which was based on some Jenkins plugins).
This will require a substantial re-write of some of the docs, and a revamping
of our tutorials. Luckily (??), we don't have a whole lot of tutorials and screenshots
to redo. This will obsolete some of the existing presentations and videos, but
only certain portions, I think. And it's early enough in Fuego's lifetime that it's
better to do it now than later.
We will need to make sure that previous functionality is available,
(or alternatively, that we can live without that functionality). I assume
some cosmetic things (like, say, the "greenballs" Jenkins plugin)
should work without problem. More problematical is dealing with changes
in the workflow caused by missing Jenkins modules that have been assumed
by Fuego users to be present.
> - I removed dependencies on all groovy scripts.
I'm still processing this. I believe it means that targets are not first-class
objects inside of Jenkins anymore. I'll have to investigate how to use
Jenkins to configure attributes of the target, or see target-specific test
results, etc, based on the new interface.
I haven't had time yet to play with the newer code and find out what
features or workflows are sacrificed by this, but I hope to do so next
week.
I plan to come up with a list of tasks and operations that I did in the
old interface, that are not obvious how to do in the new interface.
> - Userdata is gone. Instead we now have fuego-ro and fuego-rw, which
> together with fuego-core
> are mounted as external docker volumes. Only fuego-rw, which contains
> logs and build folders,
> is mounted in RW mode. I also fixed the uid/gid of jenkins so that it
> matches the one in the host.
> This means you can develop comfortably on your host, with GUI tools, and
> don't fear a buggy
> script deleting your folders.
This is super awesome, and resolves a lot of my own development headaches
with Fuego.
> - I added a tool for generating jobs from testplans. Creating jobs from
> testplans is powerful.
> It allows you, for example, to easily specify the timeout for each test on
> your board.
> The tool also generates the corresponding batch job. For example, you can
> easily trigger all
> tests for your board and a specific testplan periodically.
Sounds good, and similar to what 0day does, but I'm not sure I understand
completely. I believe you mean that you can use the test materials and testplans
to create separate Jenkins job for each plan? Is that right?
It looks like you also create a separate Jenkins job for each combination of
test and target? Is that right? The default Jenkins interface is so sparse
that I'm a little lost what to do with it. (Maybe I should not have comingled
my learning of Jenkins with my learning of Fuego - I think I've made assumptions
about base Jenkins features based on the plugins Cogent added by default
in JTA.)
> [Note] This will also enable the creation of scripts for comparing test results
> across
> different boards and test parameters (on my TODO list).
This sounds good.
> Other less fundamental changes include:
> - I added support for using the docker container itself as a target board (e.g.:
> for a quickstart or unit tests)
That's very handy, for a quickstart, and for testing Fuego itself. Good idea!
> - I fixed the flot plugin (java and javascript fixes)
Thanks!!
> - Now you can click on a node and see which jobs (and testplans) are
> assigned to your board. This was
> one of my old feature requests and makes Fuego's GUI easier to use.
> - I added quite a few fixes and improvements to the core engine scripts and
> tests (too detailed to describe here).
This one worries me. I haven't done a diff yet, but I hope we're not
diverging too far to make integration difficult.
> - I added Excel output support for IOzone
> - Faster docker build time (ARMhf compiler installation is now optional, no
> latex..)
These both sound good. The excel output format possibly ties
into discussions we've had about canonical output formats,
results saving and comparisons, etc. But I'd like to get more details
to see what's up with this.
> - I put a fixed name to the fuego container, instead of the
> "last_container_id" file.
I tend to have several containers lying around, for testing multiple
fuego versions. I'm not sure if this will work if docker is using a fixed
name for the container.
> - I removed the inotify script
One of my first contributions to fuego! This makes me a little sad,
but happy if it's not needed anymore. :-)
> - I added scripts for easily removing the docker container/image.
>
> I have to split the current "huge" patch into smaller chunks and test it further
> but it would
> be nice to get your feedback about the approach in general.
>
> Tim: I haven't updated ftc nor your "Christmas commits" yet.
I can help with updating ftc and with more recent commits.
I have yet to really dig into the code changes to see how it affects
the overall architecture. But I really like the list of changes.
I do want to make sure that we haven't lost any important features
going forward, and that the changes fit with my longterm goals
for the project.
Let me know what would be best to look at, for planning future
integration of these changes.
-- Tim
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-01-20 0:53 ` [Fuego] " Bird, Timothy
@ 2017-01-20 7:09 ` Daniel Sangorrin
2017-01-28 2:14 ` Bird, Timothy
0 siblings, 1 reply; 43+ messages in thread
From: Daniel Sangorrin @ 2017-01-20 7:09 UTC (permalink / raw)
To: 'Bird, Timothy', fuego
> > - I removed dependencies on all groovy scripts.
>
> I'm still processing this. I believe it means that targets are not first-class
> objects inside of Jenkins anymore. I'll have to investigate how to use
> Jenkins to configure attributes of the target, or see target-specific test
> results, etc, based on the new interface.
I'll try to explain this.
In the current version of Fuego:
- Nodes and its attributes (e.g.: which board and dist file to use) are created
manually through the jenkins interface.
- To run a job on a node, you open a job and then select your node (as well
as some variables such as 'rebuild' etc..). The problem is that you can only
run one node at a time, unless you copy the job and change the name
manually, and that you have to trigger them manually as well.
In my version of Fuego:
- Nodes and its attributes are created through the fuego-create-node script.
You can configure the attributes of the target as usual from the jenkins interface
or from the script interface (delete the node and create it with different attributes).
- Jobs are created through a script called fuego-create-jobs. Jobs are not
dynamic anymore (although you can do the same just by clicking on configure)
and are assigned a specific node, testplan and other attributes. If you want
to see a target-specific test result, you click on the corresponding node to display
jobs assigned to your target/node. Then you click on the job you want.
By the way, if you still want to dynamically change the parameters of a job, all you
have to do is click on "configure" and then change the variable, testplan, etc.. as
you please.
> I haven't had time yet to play with the newer code and find out what
> features or workflows are sacrificed by this, but I hope to do so next
> week.
>
> I plan to come up with a list of tasks and operations that I did in the
> old interface, that are not obvious how to do in the new interface.
Sure, that would be great thank you!
> > - I added a tool for generating jobs from testplans. Creating jobs from
> > testplans is powerful.
> > It allows you, for example, to easily specify the timeout for each test on
> > your board.
> > The tool also generates the corresponding batch job. For example, you can
> > easily trigger all
> > tests for your board and a specific testplan periodically.
>
> Sounds good, and similar to what 0day does, but I'm not sure I understand
> completely. I believe you mean that you can use the test materials and testplans
> to create separate Jenkins job for each plan? Is that right?
I am assuming a workflow where you have a testplan for each of your targets (otherwise you'll get the default one).
In a testplan you define which tests you want to run on your target. For example, you may only want to run
8 tests on a specific board. You also define the spec to use for each test (by 'spec' I mean the word that gets mapped to a
specific set of parameters for the test).
When you create the jobs for your target, you pass the corresponding testplan as a parameter
to fuego-create-jobs. This tool will traverse the testplan and create the jobs on the jenkins interface.
[Note] At the moment, if you want to use two different specs for the same target-test combination you have to
define it in a separate testplan. But I will add support for testplans that contain the same test with different specs.
> It looks like you also create a separate Jenkins job for each combination of
> test and target? Is that right?
More exactly, for each combination of target.testplan.test.
[Note] When I add support for multiple specs inside a testplan it will become target.testplan.testspec.test.
> The default Jenkins interface is so sparse
> that I'm a little lost what to do with it. (Maybe I should not have comingled
> my learning of Jenkins with my learning of Fuego - I think I've made assumptions
> about base Jenkins features based on the plugins Cogent added by default
> in JTA.)
Actually, it's much easier with my version. You only need to click on your node and you will
only see the jobs assigned to your node/board (for example, 8 tests).
In the current version it is impossible to differentiate which job is for which node, all you can
do is check the history of which jobs were executed on each node.
> > - I added Excel output support for IOzone
> > - Faster docker build time (ARMhf compiler installation is now optional, no
> > latex..)
> These both sound good. The excel output format possibly ties
> into discussions we've had about canonical output formats,
> results saving and comparisons, etc. But I'd like to get more details
> to see what's up with this.
Well, IOzone already supported the excel output (-b).
One more important change is that now the results can be linked to each build. So
if you look to the left sidebar, each successful build/test will preserve the results of
that specific test at that specific time. Also, you can arbitrarily set these links in the
test specs (e.g.: plot.png, the log of the test, excel, ..and whatever file you want).
This results are suffixed with the build number so that they don't collide with each other.
The flot plugin (and plot.png) allows you to compare the results over time.
> > - I put a fixed name to the fuego container, instead of the
> > "last_container_id" file.
> I tend to have several containers lying around, for testing multiple
> fuego versions. I'm not sure if this will work if docker is using a fixed
> name for the container.
OK, I can add a parameter to the script so that you can overwrite the default
name. Still I think it will be better than "last_container_id".
> > - I removed the inotify script
> One of my first contributions to fuego! This makes me a little sad,
> but happy if it's not needed anymore. :-)
Haha I'm sorry. Actually, jenkins changed and now most of the information
is not stored in config.xml. In fact, for certain setups it won't even generate
any config.xml. I didn't know how to set the global variables without config.xml
so I left it there, but if we write the global variables in a script we can get rid
of the config.xml for good.
> > - I added scripts for easily removing the docker container/image.
> >
> > I have to split the current "huge" patch into smaller chunks and test it further
> > but it would
> > be nice to get your feedback about the approach in general.
> >
> > Tim: I haven't updated ftc nor your "Christmas commits" yet.
>
> I can help with updating ftc and with more recent commits.
>
> I have yet to really dig into the code changes to see how it affects
> the overall architecture. But I really like the list of changes.
> I do want to make sure that we haven't lost any important features
> going forward, and that the changes fit with my longterm goals
> for the project.
That would be great.
By the way, make sure you git pull the latest version because I added
a few fixes last week.
Thanks,
Daniel
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-01-20 7:09 ` Daniel Sangorrin
@ 2017-01-28 2:14 ` Bird, Timothy
2017-01-31 1:56 ` Daniel Sangorrin
0 siblings, 1 reply; 43+ messages in thread
From: Bird, Timothy @ 2017-01-28 2:14 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
Sorry for the slow response. I was out-of-office most of this week.
> From: Daniel Sangorrin on Thursday, January 19, 2017 11:10 PM
> I am assuming a workflow where you have a testplan for each of your targets
> (otherwise you'll get the default one).
> In a testplan you define which tests you want to run on your target. For
> example, you may only want to run
> 8 tests on a specific board. You also define the spec to use for each test (by
> 'spec' I mean the word that gets mapped to a
> specific set of parameters for the test).
OK - that's actually a bit different than how testplans are currently used
in Fuego. Weirdly (IMHO), they don't actually specify the set of tests
to run, but only the list of specs to use for each test indicated in the plan.
So, the 'testplan_mmc' specifies that each test it mentions should use
the 'mmc' spec, when used with that plan. But Fuego does not currently
have a feature (that I can see) to run all of the tests mentioned in the testplan.
In terms of separation, I believe the Jenkins job was intended to provide the
actual tests to perform, while the parameters for each test were come from
the testplan/spec system. But Fuego doesn't actually ship with any pre-defined
Jenkins jobs that run multiple tests that are statically defined.
The only multi-test jobs are the 3 tests (in the "batch runs" tab):
* run all test on all targets
* run all tests on selected targets
* run selected tests on selected targets
The second and third of these are interactive, and I can find no feature
that saves the information so you can re-run the test with your previous
selections whenever you want. That is, you have to manually select
the targets and tests each time.
>
> When you create the jobs for your target, you pass the corresponding
> testplan as a parameter
> to fuego-create-jobs. This tool will traverse the testplan and create the jobs
> on the jenkins interface.
>
> [Note] At the moment, if you want to use two different specs for the same
> target-test combination you have to
> define it in a separate testplan. But I will add support for testplans that
> contain the same test with different specs.
That sounds good and superior to what we have now.
>
> > It looks like you also create a separate Jenkins job for each combination of
> > test and target? Is that right?
>
> More exactly, for each combination of target.testplan.test.
> [Note] When I add support for multiple specs inside a testplan it will become
> target.testplan.testspec.test.
>
I've been looking at how testplans are currently defined in Jenkins, and
frankly they could use an overhaul. They don't do what you describe
in your refactoring, but I think they should. The ability to run multiple
tests in a predefined configuration and order should be a feature of
fuego-core, and not reliant on Jenkins.
As part of my work towards supporting test packaging,
I have modified ovgen.py to not require testplan_default, and to read
all the spec files from their respective test directories (instead of a single
centralized location).
There are two kinds of testplans in current fuego:
- single-test: this allows you to choose a single spec for an individual test
- multi-test: this allows you to choose a single spec name for multiple tests
There are currently no instances of testplans that indicate different specs
for different tests. But this is what you are describing, and I think this
was a feature originally envisioned for this system.
For Jenkins jobs, I think composing them may be the way to go, but I've got
to finish my ovgen.py rewrite first. I'd like the architecture to support
both jenkins and command line test invocation equally well.
I'd like to do away with the need to edit testplans when a test is installed,
so I think they should be more loosely coupled than they are now. I'm
looking at adding a wildcard capability to the test name. With this, we could have
something like testplan_board_smoketest that would look like:
{
"testPlanName": "testplan_board_smoketest",
"tests": [
{
"testName": "*",
"spec": "smoketest"
}
]
}
That way, if someone adds a new smoketest test to the system, all they
need to provide is the spec 'smoketest', and they would run with those
testplans that mention that spec.
Alternatively, when running an individual test, one could just list the available
specs, without the need for a testplan at all.
I had hoped to come up with modifications to ovgen.py that worked with
both the current fuego system and your new one, but to be honest I've been
stumped. Everything I do ends up requiring edits to the groovy scripts, which
you have eliminated in your system.
Ultimately, I want people to be able to share testplans, specs, and tests
freely. But since plans mention specs and tests, they are dependent on
those being installed. So, if someone installed something like "testplan_basic_distro_check",
it would have a list of tests and specs that should be run to validate a basic
embedded Linux distribution. Maybe an installer for that plan could
automatically find and load the tests that were referenced by it.
I think that specs should obviously be in the test directory, as they are very
test-specific, but it would be nice if people could provide different
specs very easily for the same test (for example, a real-time developer
could provide different specs for cyclictest, to examine or test different
aspects of the system with the same tool). Having the specs in separate
files is already a nice feature for this decoupling.
Anyway, I'll try to get something that handles the current system, then
quickly move on to supporting your new model.
Any feedback would be welcome.
-- Tim
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-01-28 2:14 ` Bird, Timothy
@ 2017-01-31 1:56 ` Daniel Sangorrin
2017-01-31 20:54 ` Jan-Simon Möller
` (2 more replies)
0 siblings, 3 replies; 43+ messages in thread
From: Daniel Sangorrin @ 2017-01-31 1:56 UTC (permalink / raw)
To: 'Bird, Timothy', fuego
Hi Tim,
> -----Original Message-----
> From: Bird, Timothy [mailto:Tim.Bird@sony.com]
> Sent: Saturday, January 28, 2017 11:15 AM
> I'd like to do away with the need to edit testplans when a test is installed,
> so I think they should be more loosely coupled than they are now. I'm
> looking at adding a wildcard capability to the test name. With this, we could have
> something like testplan_board_smoketest that would look like:
>
> {
> "testPlanName": "testplan_board_smoketest",
> "tests": [
> {
> "testName": "*",
> "spec": "smoketest"
> }
> ]
> }
> That way, if someone adds a new smoketest test to the system, all they
> need to provide is the spec 'smoketest', and they would run with those
> testplans that mention that spec.
I like the idea of using a wildcard (or regex) in the testplans. We can also create
a script to generate testplans as well:
$ fuego-create-testplan -t Functional.* -s smoketest
[Note] I'm starting to think that we need to create a single tool (fuego) with subcommands.
We could also support several specs for each test as follows:
{
"testPlanName": "testplan_board ",
"tests": [
{
"testName": "mytest",
"spec": "default, stress, lightcheck, with_option_*"
}
]
}
> Alternatively, when running an individual test, one could just list the available
> specs, without the need for a testplan at all.
>
> I had hoped to come up with modifications to ovgen.py that worked with
> both the current fuego system and your new one, but to be honest I've been
> stumped. Everything I do ends up requiring edits to the groovy scripts, which
> you have eliminated in your system.
Maybe we could put the master branch in maintenance mode (add bug fixes but
no new features) and do all new development on the next branch. That would
save us a lot of headaches.
> Ultimately, I want people to be able to share testplans, specs, and tests
> freely. But since plans mention specs and tests, they are dependent on
> those being installed. So, if someone installed something like "testplan_basic_distro_check",
> it would have a list of tests and specs that should be run to validate a basic
> embedded Linux distribution. Maybe an installer for that plan could
> automatically find and load the tests that were referenced by it.
>
> I think that specs should obviously be in the test directory, as they are very
> test-specific, but it would be nice if people could provide different
> specs very easily for the same test (for example, a real-time developer
> could provide different specs for cyclictest, to examine or test different
> aspects of the system with the same tool). Having the specs in separate
> files is already a nice feature for this decoupling.
I agree with putting test specs in the same directory as the tests.
However, I think it is already pretty easy to add a new entry to a test spec as it is.
For example, consider the IOZone spec:
https://bitbucket.org/nirrognas/fuego-core/src/8db15809ea20e123062264ed52cde34f6fa9a7ae/engine/overlays/test_specs/Benchmark.IOzone.spec
You can easily create a new entry (copy&paste) and tune the test parameters.
{
"name":"default",
"MOUNT_BLOCKDEV":"ROOT",
"MOUNT_POINT":"$BOARD_TESTDIR/work",
"FILE_SIZE":"2M",
"TESTS":"'-i 0 -i 1 -i 2 -i 6 -i 7 -i 8 -i 9'"
}
Note: cyclictest is a bad example because the test script doesn't offer enough tuning knobs.
Note: sharing can be done by sending a patch to the test maintainer for example.
> Anyway, I'll try to get something that handles the current system, then
> quickly move on to supporting your new model.
Ok, great. I will be polishing my branch in the meantime.
Thanks,
Daniel
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-01-31 1:56 ` Daniel Sangorrin
@ 2017-01-31 20:54 ` Jan-Simon Möller
2017-02-02 1:25 ` Bird, Timothy
2017-01-31 23:15 ` Jan-Simon Möller
2017-02-02 1:17 ` Bird, Timothy
2 siblings, 1 reply; 43+ messages in thread
From: Jan-Simon Möller @ 2017-01-31 20:54 UTC (permalink / raw)
To: fuego; +Cc: 'Bird, Timothy'
[-- Attachment #1: Type: text/plain, Size: 1875 bytes --]
Hi Daniel, Tim, * !
I've setup an instance of the -next branch and I'm using and working on it right now.
Let me add a few notes and observations:
* I like the create-job / delete-job and the node scripts. They simplify
things a lot.
* A extension to this approach IMHO would be the use of jenkins-job-builder.
The beauty of this is that we do not have to deal with the xml ourselves.
Instead we define yaml files and let jjb create the necessary xml.
Another plus is that such an approach would also allow ppl to add
the testcases to an *existing* jenkins using the jjb templates.
Let me illustrate this with a jump-start example:
Installation of jjb (in fuego container):
$ cd ~/
$ sudo apt-get install python-pip
$ sudo pip install jenkins-job-builder
# should be 1.6.x
$ cat <<EOF > jjb-conf.ini
[job_builder]
ignore_cache=True
keep_descriptions=False
include_path=.:scripts:~/git/
recursive=True
[jenkins]
user=Anonymous
password=
url=https://localhost/
##### This is deprecated, use job_builder section instead
#ignore_cache=True
query_plugins_info=False
EOF
$ mkdir jjb
$ cd jjb/
$ mkdir testjob
$ cat <<EOF > testjob/testjob.yaml
- project:
name: fuego-tests-smoke
machine:
- raspberrypi3
testname:
-
-
-
-
-
jobs:
- {machine}.smoke.batch
- {machine}.smoke.{testname}
- job-template:
name: '{machine}.smoke.batch'
project-type: freestyle
concurrent: false
node: master
builders:
- orchestrator-script ON
- trigger '{machine}.smoke.*' BLOCKING
- orchestrator-script OFF ALWAYS
- job-template:
name: '{machine}.smoke.{testname}
project-type: freestyle
concurrent: false
node: {machine}
--
--
Jan-Simon Möller
dl9pf@gmx.de
[-- Attachment #2: Type: text/html, Size: 12459 bytes --]
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-01-31 1:56 ` Daniel Sangorrin
2017-01-31 20:54 ` Jan-Simon Möller
@ 2017-01-31 23:15 ` Jan-Simon Möller
2017-02-01 0:56 ` Daniel Sangorrin
2017-02-02 1:35 ` Bird, Timothy
2017-02-02 1:17 ` Bird, Timothy
2 siblings, 2 replies; 43+ messages in thread
From: Jan-Simon Möller @ 2017-01-31 23:15 UTC (permalink / raw)
To: fuego; +Cc: 'Bird, Timothy'
[-- Attachment #1: Type: text/plain, Size: 4484 bytes --]
Hi Daniel, Tim, * !
v2: now complete and w/o thick finger.
I've setup an instance of the -next branch and I'm using and working on it right now.
Let me add a few notes and observations:
* I like the create-job / delete-job and the node scripts. They simplify
things a lot.
* A extension to this approach IMHO would be the use of jenkins-job-builder.
The beauty of this is that we do not have to deal with the xml ourselves.
Instead we define yaml files and let jjb create the necessary xml.
Another plus is that such an approach would also allow ppl to add
the testcases to an *existing* jenkins using the jjb templates.
Let me illustrate this with a jump-start example:
Installation of jjb (in fuego container):
$ cd ~/
$ sudo apt-get install python-pip
$ sudo pip install jenkins-job-builder
# should be 1.6.x
$ cat <<EOF > jjb-conf.ini
[job_builder]
ignore_cache=True
keep_descriptions=False
include_path=.:scripts:~/git/
recursive=True
[jenkins]
#user=
#password=
url=http://localhost:8080/fuego
##### This is deprecated, use job_builder section instead
#ignore_cache=True
query_plugins_info=False
EOF
$ mkdir jjb
$ cd jjb/
$ mkdir testjob
$ cat <<EOF > testjob/testjob.yaml
- project:
name: fuego-tests-smoke
testplan:
- smoke
machine:
- raspberrypi3
testname:
- Benchmark.Dhrystone
- Benchmark.IOzone
- Functional.hello_world
- Functional.ipv6connect
- Functional.stress
jobs:
- fuego-{machine}-{testplan}-batch
- fuego-{machine}-{testplan}-{testname}
- job-template:
name: fuego-{machine}-{testplan}-batch
project-type: freestyle
concurrent: false
node: master
builders:
- shell: 'echo "Run Orchestrator hook - action to enable board"'
- shell: 'echo "Next section should also be autogenerated - just dont know the kong-fu, yet."'
- trigger-builds:
- project: fuego-{machine}-{testplan}-Benchmark.Dhrystone
current-parameters: true
block: true
- project: fuego-{machine}-{testplan}-Benchmark.IOzone
current-parameters: true
block: true
- project: fuego-{machine}-{testplan}-Functional.hello_world
current-parameters: true
block: true
- project: fuego-{machine}-{testplan}-Functional.ipv6connect
current-parameters: true
block: true
- project: fuego-{machine}-{testplan}-Functional.stress
current-parameters: true
block: true
- conditional-step:
condition-kind: always
steps:
- shell: 'echo "Run orchestrator hook - action do disable board"'
- job-template:
name: 'fuego-{machine}-{testplan}-{testname}'
project-type: freestyle
concurrent: false
node: '{machine}'
builders:
- shell: |
export Reboot=false
export Rebuild=true
export Target_Cleanup=true
export TESTDIR={testname} # yep, mixed up testname here with testdir and vice-versa. anyway.
export TESTNAME=$(echo "{testname}" | sed -e "s#.*\.##")
export DISTRIB="{{distrib}}"
export TESTPLAN="{testplan}"
timeout --signal=9 10m /bin/bash $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNAME}}.sh
EOF
$ jenkins-jobs --conf jjb-conf.ini test -r testjob/
# to create the jobs within jenkins:$ jenkins-jobs --conf jjb-conf.ini update -r testjob/
The idea would be that fuego-core would provide the scripts and the yaml file in the same git.
While running tests I also found:
- we need to split the build of the test and the postprocessing to be run separately
-- this is b/c it is not useful to keep to board waiting on us all the time.
- Integration of board up/down could happen in a few ways:
-- Either ppl should just generate a wrapper around the (blocking) batch job and trigger it
with their pre/post to their needs. Done.
-- Or we allow hooks like the current "TARGET_SETUP_LINK" and amend it with a matching "TARGET_TEARDOWN_LINK"
- If a model like the "TARGET_SETUP_LINK" is used, we add a delay here as this call must block until the board is up.
In this case our predifined timeouts are bogus as they track not just the test run, but all processing.
Thats my 0,02 - please comment.
Best,
--
--
Jan-Simon Möller
dl9pf@gmx.de
[-- Attachment #2: Type: text/html, Size: 22225 bytes --]
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-01-31 23:15 ` Jan-Simon Möller
@ 2017-02-01 0:56 ` Daniel Sangorrin
2017-02-01 10:46 ` Jan-Simon Möller
2017-02-02 1:35 ` Bird, Timothy
1 sibling, 1 reply; 43+ messages in thread
From: Daniel Sangorrin @ 2017-02-01 0:56 UTC (permalink / raw)
To: 'Jan-Simon Möller', fuego; +Cc: 'Bird, Timothy'
Hi Jan-Simon,
> -----Original Message-----
> From: Jan-Simon Möller [mailto:dl9pf@gmx.de]
...
> Let me add a few notes and observations:
>
> * I like the create-job / delete-job and the node scripts. They simplify
>
> things a lot.
>
> * A extension to this approach IMHO would be the use of jenkins-job-builder.
Wow honestly, I didn't know about that application. It looks quite mature.
> The beauty of this is that we do not have to deal with the xml ourselves.
That's great because I noticed that the XML format changes a bit every time I upgrade.
> Instead we define yaml files and let jjb create the necessary xml.
Cool, I'm ok with YAML but I see it also supports JSON.
http://docs.openstack.org/infra/jenkins-job-builder/
> Another plus is that such an approach would also allow ppl to add
>
> the testcases to an *existing* jenkins using the jjb templates.
>
>
>
> Let me illustrate this with a jump-start example:
>
>
>
> Installation of jjb (in fuego container):
>
> $ cd ~/
>
> $ sudo apt-get install python-pip
>
> $ sudo pip install jenkins-job-builder
>
> # should be 1.6.x
>
>
>
> $ cat <<EOF > jjb-conf.ini
>
> [job_builder]
>
> ignore_cache=True
>
> keep_descriptions=False
>
> include_path=.:scripts:~/git/
>
> recursive=True
>
> [jenkins]
>
> #user=
>
> #password=
>
> url=http://localhost:8080/fuego
>
> ##### This is deprecated, use job_builder section instead
>
> #ignore_cache=True
>
> query_plugins_info=False
>
> EOF
>
>
>
> $ mkdir jjb
>
> $ cd jjb/
>
>
>
> $ mkdir testjob
>
> $ cat <<EOF > testjob/testjob.yaml
>
> - project:
>
> name: fuego-tests-smoke
>
> testplan:
>
> - smoke
>
> machine:
>
> - raspberrypi3
>
> testname:
>
> - Benchmark.Dhrystone
>
> - Benchmark.IOzone
>
> - Functional.hello_world
>
> - Functional.ipv6connect
>
> - Functional.stress
>
> jobs:
>
> - fuego-{machine}-{testplan}-batch
>
> - fuego-{machine}-{testplan}-{testname}
>
>
>
> - job-template:
>
> name: fuego-{machine}-{testplan}-batch
>
> project-type: freestyle
>
> concurrent: false
>
> node: master
>
>
>
> builders:
>
> - shell: 'echo "Run Orchestrator hook - action to enable board"'
>
> - shell: 'echo "Next section should also be autogenerated - just dont know the kong-fu, yet."'
>
> - trigger-builds:
>
> - project: fuego-{machine}-{testplan}-Benchmark.Dhrystone
>
> current-parameters: true
>
> block: true
>
> - project: fuego-{machine}-{testplan}-Benchmark.IOzone
>
> current-parameters: true
>
> block: true
>
> - project: fuego-{machine}-{testplan}-Functional.hello_world
>
> current-parameters: true
>
> block: true
>
> - project: fuego-{machine}-{testplan}-Functional.ipv6connect
>
> current-parameters: true
>
> block: true
>
> - project: fuego-{machine}-{testplan}-Functional.stress
>
> current-parameters: true
>
> block: true
>
> - conditional-step:
>
> condition-kind: always
>
> steps:
>
> - shell: 'echo "Run orchestrator hook - action do disable board"'
>
>
>
> - job-template:
>
> name: 'fuego-{machine}-{testplan}-{testname}'
>
> project-type: freestyle
>
> concurrent: false
>
> node: '{machine}'
>
> builders:
>
> - shell: |
>
> export Reboot=false
>
> export Rebuild=true
>
> export Target_Cleanup=true
>
> export TESTDIR={testname} # yep, mixed up testname here with testdir and vice-versa. anyway.
>
> export TESTNAME=$(echo "{testname}" | sed -e "s#.*\.##")
>
> export DISTRIB="{{distrib}}"
>
> export TESTPLAN="{testplan}"
>
> timeout --signal=9 10m /bin/bash $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNAME}}.sh
>
>
>
>
>
> EOF
>
>
>
> $ jenkins-jobs --conf jjb-conf.ini test -r testjob/
>
> # to create the jobs within jenkins:
> $ jenkins-jobs --conf jjb-conf.ini update -r testjob/
>
>
>
>
>
> The idea would be that fuego-core would provide the scripts and the yaml file in the same git.
>
Thanks for the detailed example. A few observations:
- Normally the timeout value depends on the board's performance and the test duration.
In my next branch, timeouts are defined for each test in the testplan_board.json file.
Here it looks like the timeout needs to be the same for all of the tests. I was wondering
if it is possible to put a list of timeouts (same amount as the number of tests) and then
use timeout --signal=9 {timeout}. Doing this by hand would be too tedious but we could
generate the yaml files from the testplans using a script.
- What would we do if we want to execute the same test with several test_specs? I was
thinking about creating separate jobs for each combination of test-test_spec because
it's the simple but it wonder if that can be implemented with jjb. Another approach
would be to create a single job and then have fuego-core take care of reading the
tests specs of each test from the testplan, and repeat the test for each test spec. Then
a modified parser would display and compare the results of the test across the
different test specs.
[Note] I think that AGL's fuego has a similar parser.
- In my next branch, test specs have the ability to define which links should be available
after the test is complete. I'm using the Description Setter plugin for that. Do you think
that would be possible with the jjb approach?.
> While running tests I also found:
>
> - we need to split the build of the test and the postprocessing to be run separately
>
> -- this is b/c it is not useful to keep to board waiting on us all the time.
I wasn't aware that the board is waiting on us while we do the post-processing.
Could you elaborate a bit more?
> - Integration of board up/down could happen in a few ways:
>
> -- Either ppl should just generate a wrapper around the (blocking) batch job and trigger it
>
> with their pre/post to their needs. Done.
>
> -- Or we allow hooks like the current "TARGET_SETUP_LINK" and amend it with a matching "TARGET_TEARDOWN_LINK"
>
> - If a model like the "TARGET_SETUP_LINK" is used, we add a delay here as this call must block until the board is up.
>
> In this case our predifined timeouts are bogus as they track not just the test run, but all processing.
Good point. I wasn't using the reboot feature so I hadn't thought about it.
I guess we would need to define a timeout for the board to reboot, and then the timeout for each test.
The timeout for the board could be called BOARD_REBOOT_TIMEOUT for example and be defined
in the asdf.board file. And the timeout for each test defined in another variable such as TEST_TIMEOUT.
BOARD_REBOOT_TIMEOUT=1m
TEST_TIMEOUT=15m
/bin/bash $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNAME}}.sh
Then, internally the timeouts would be used separately.
# we could also split timeouts between execution time and post-processing time
> Thats my 0,02 - please comment.
>
> Best,
>
> --
>
> --
>
> Jan-Simon Möller
>
> dl9pf@gmx.de
Thanks a lot.
Cheers,
Daniel
PS: sorry, my mailer converts html mails into text automatically.
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-01 0:56 ` Daniel Sangorrin
@ 2017-02-01 10:46 ` Jan-Simon Möller
2017-02-02 1:57 ` Bird, Timothy
2017-02-02 5:45 ` Daniel Sangorrin
0 siblings, 2 replies; 43+ messages in thread
From: Jan-Simon Möller @ 2017-02-01 10:46 UTC (permalink / raw)
To: Daniel Sangorrin; +Cc: 'Bird, Timothy', fuego
Hi !
Am Mittwoch, 1. Februar 2017, 09:56:03 schrieb Daniel Sangorrin:
> Thanks for the detailed example. A few observations:
>
> - Normally the timeout value depends on the board's performance and the test
> duration. In my next branch, timeouts are defined for each test in the
> testplan_board.json file. Here it looks like the timeout needs to be the
> same for all of the tests. I was wondering if it is possible to put a list
> of timeouts (same amount as the number of tests) and then use timeout
> --signal=9 {timeout}. Doing this by hand would be too tedious but we could
> generate the yaml files from the testplans using a script.
Oh, that was just /me taking a shortcut. But yes, the template wouldn't know directly the board/test specific values.
One way would be to make these bash env files and source them before running the script.
The 'same' number of timeouts would create a matrix [ tests x timeouts ], that is not what we're looking for.
> - What would we do if we want to execute the same test with several
> test_specs? I was thinking about creating separate jobs for each
> combination of test-test_spec because it's the simple but it wonder if that
> can be implemented with jjb.
That would then be:
- project:
name: fuego-tests-smoke
testplan:
- smoke
testspec:
- testspec1
- testspec2
machine:
- raspberrypi3
testname:
- Benchmark.Dhrystone
- Benchmark.IOzone
- Functional.hello_world
- Functional.ipv6connect
- Functional.stress
jobs:
- fuego-{machine}-{testplan}-{testspec}-batch
- fuego-{machine}-{testplan}-{testspec}-{testname}
> Another approach would be to create a single
> job and then have fuego-core take care of reading the tests specs of each
> test from the testplan, and repeat the test for each test spec. Then a
> modified parser would display and compare the results of the test across
> the different test specs.
> [Note] I think that AGL's fuego has a similar parser.
Having all in one job let's you not execute one specific test easily.
But if we simplify the shell call that jenkins has to do, creating jobs
with specific names and parameters/env-variables is trivial.
E.g.:
builders:
- shell: |
#
source $FUEGO_RO/conf/boards/{machine}.sh
source $FUEGO_CORE/engine/env/${{DISTRO}}.sh # DISTRO set in machine.sh
source $FUEGO_CORE/engine/testplans/{testplan}.sh # taken from jjb
#
export Reboot=false
export Rebuild=true
export Target_Cleanup=true
export TESTDIR={testname}
export TESTNAME=$(echo "{testname}" | sed -e "s#.*\.##")
timeout --signal=9 ${{JOB_TIMEOUT}} /bin/bash $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNAME}}.sh
In this case, ${{JOB_TIMEOUT}} can be defined in the $FUEGO_RO/conf/boards/{machine}.sh env script.
> - In my next branch, test specs have the ability to define which links
> should be available after the test is complete. I'm using the Description
> Setter plugin for that. Do you think that would be possible with the jjb
> approach?.
Links ... let me see
http://docs.openstack.org/infra/jenkins-job-builder/publishers.html#publishers.description-setter ?
should work
For reference:
http://docs.openstack.org/infra/jenkins-job-builder/definition.html
http://docs.openstack.org/infra/jenkins-job-builder/builders.html
http://docs.openstack.org/infra/jenkins-job-builder/publishers.html
>
> > While running tests I also found:
> >
> >
> > - we need to split the build of the test and the postprocessing to be run
> > separately>
> >
> >
> > -- this is b/c it is not useful to keep to board waiting on us all the
> > time.
> I wasn't aware that the board is waiting on us while we do the
> post-processing. Could you elaborate a bit more?
We access the board very early even before the build (pre_test).
The we build, deploy, run, fetch results, analyse. At this point
> > - Integration of board up/down could happen in a few ways:
> > -- Either ppl should just generate a wrapper around the (blocking) batch
> > job and trigger it>
> > with their pre/post to their needs. Done.
> > -- Or we allow hooks like the current "TARGET_SETUP_LINK" and amend it
> > with a matching "TARGET_TEARDOWN_LINK">
> > - If a model like the "TARGET_SETUP_LINK" is used, we add a delay here as
> > this call must block until the board is up.>
> > In this case our predifined timeouts are bogus as they track not just the
> > test run, but all processing.
> Good point. I wasn't using the reboot feature so I hadn't thought about it.
> I guess we would need to define a timeout for the board to reboot, and then
> the timeout for each test. The timeout for the board could be called
> BOARD_REBOOT_TIMEOUT for example and be defined in the asdf.board file. And
> the timeout for each test defined in another variable such as TEST_TIMEOUT.
As I said, there are multiple ways to integrate that.
a) The wrapper-job triggering the batch leaves fuego out of the picture. But we need to block the whole chain to make flow-control possible.
b) TARGET_SETUP_LINK gives us a _per job_ way of brining up the board. Not just for the wrapp'ed batch.
Most users might get away and be happy with a) . While a per-job control as in b) might scale better if you think of the case that you could have
multiple target boards of the same type (multiple executors!) and could run jobs 'in parallel' !
> BOARD_REBOOT_TIMEOUT=1m
> TEST_TIMEOUT=15m
> /bin/bash $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNAME}}.sh
>
> Then, internally the timeouts would be used separately.
> # we could also split timeouts between execution time and post-processing
> time
Yep ... we probably should split the build, test and postproc into separate scripts.
or instead of $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNAME}}.sh in jenkins, we could call ftc as you would do on the terminal.
Also the above should be in separate jobs ... think of it:
- the build phase doesn't need to block, can run in parallel,
e.g. on master node!
- the test batch needs to block to allow flow-control and
run only on the specified executor. No concurrency but multiple jobs
of the *same* batch could run in parallel, given we can handle multiple
boards of the same type at the same time (which is true with an 'orchestrator' like lava).
- the batch is only deploy/run/fetch result but can trigger postprocessing
- the postprocessing is another job, can run in parallel, independent of the board, e.g. on master
JS
--
--
Jan-Simon Möller
dl9pf@gmx.de
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-01-31 1:56 ` Daniel Sangorrin
2017-01-31 20:54 ` Jan-Simon Möller
2017-01-31 23:15 ` Jan-Simon Möller
@ 2017-02-02 1:17 ` Bird, Timothy
2 siblings, 0 replies; 43+ messages in thread
From: Bird, Timothy @ 2017-02-02 1:17 UTC (permalink / raw)
To: Daniel Sangorrin, fuego
> -----Original Message-----
> From: Daniel Sangorrin on Monday, January 30, 2017 5:56 PM
>...
> [Note] I'm starting to think that we need to create a single tool (fuego) with
> subcommands.
I agree. I intended 'ftc' to be that single tool. I originally wanted to avoid
using the name 'fuego', as that would make the tool name the same as
the project name, which I think can be confusing in documentation
and tutorials.
However, many projects have this ('git', 'avacado', 'rpm'), and it doesn't
seem to have hurt them too much.
So, anyway, renaming 'ftc' to 'fuego' and having it be something of a 'borg'
command that assimilates other commands, is now on my todo list.
-- Tim
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-01-31 20:54 ` Jan-Simon Möller
@ 2017-02-02 1:25 ` Bird, Timothy
2017-02-02 5:49 ` Daniel Sangorrin
0 siblings, 1 reply; 43+ messages in thread
From: Bird, Timothy @ 2017-02-02 1:25 UTC (permalink / raw)
To: Jan-Simon Möller, fuego
> -----Original Message-----
> From: Jan-Simon Möller on Tuesday, January 31, 2017 12:55 PM
>...
> * A extension to this approach IMHO would be the use of jenkins-job-
> builder.
>
> The beauty of this is that we do not have to deal with the xml ourselves.
> Instead we define yaml files and let jjb create the necessary xml.
> Another plus is that such an approach would also allow ppl to add
> the testcases to an *existing* jenkins using the jjb templates.
>
> Let me illustrate this with a jump-start example:
>
>
> Installation of jjb (in fuego container):
> $ cd ~/
> $ sudo apt-get install python-pip
> $ sudo pip install jenkins-job-builder
> # should be 1.6.x
>
> $ cat <<EOF > jjb-conf.ini
> [job_builder]
> ignore_cache=True
> keep_descriptions=False
> include_path=.:scripts:~/git/
> recursive=True
> [jenkins]
> user=Anonymous
> password=
> url=https://localhost/
> ##### This is deprecated, use job_builder section instead
> #ignore_cache=True
> query_plugins_info=False
> EOF
>
> $ mkdir jjb
> $ cd jjb/
> $ mkdir testjob
> $ cat <<EOF > testjob/testjob.yaml
> - project:
> name: fuego-tests-smoke
> machine:
> - raspberrypi3
> testname:
> -
> -
> -
> -
> -
> jobs:
> - {machine}.smoke.batch
> - {machine}.smoke.{testname}
>
> - job-template:
> name: '{machine}.smoke.batch'
> project-type: freestyle
> concurrent: false
> node: master
>
> builders:
> - orchestrator-script ON
> - trigger '{machine}.smoke.*' BLOCKING
> - orchestrator-script OFF ALWAYS
>
> - job-template:
> name: '{machine}.smoke.{testname}
> project-type: freestyle
> concurrent: false
> node: {machine}
I like the idea of using a tool maintained by others
to handle the interaction with Jenkins. This will
insulate us from additional Jenkins changes. I do
worry a bit, however, about exposing Fuego users
to a whole bunch of extraneous Jenkins or jjb
info that they don't need just to install and run tests
on their boards.
If the complexity of the above can be hidden from
the 'casual' user, who has a board, and just wants to
run an existing test on it, then that would be good.
For me, this includes the whole issue of plans and
specs as well. I'd like to make awareness of those
optional for Fuego users who don't need them.
Daniel's scripts take a minimum of arguments, and
do a pretty good job in this regards. If they can
use jjb behind the scenes, making it so that end users
don't have to fiddle with YAML files full of weird,
Jenkins parameters, that would be good, IMHO.
-- Tim
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-01-31 23:15 ` Jan-Simon Möller
2017-02-01 0:56 ` Daniel Sangorrin
@ 2017-02-02 1:35 ` Bird, Timothy
2017-02-02 5:52 ` Daniel Sangorrin
` (2 more replies)
1 sibling, 3 replies; 43+ messages in thread
From: Bird, Timothy @ 2017-02-02 1:35 UTC (permalink / raw)
To: Jan-Simon Möller, fuego
> -----Original Message-----
> From: Jan-Simon Möller on Tuesday, January 31, 2017 3:16 PM
...
> While running tests I also found:
>
> - we need to split the build of the test and the postprocessing to be run
> separately
> -- this is b/c it is not useful to keep to board waiting on us all the time.
> - Integration of board up/down could happen in a few ways:
> -- Either ppl should just generate a wrapper around the (blocking) batch job
> and trigger it with their pre/post to their needs. Done.
> -- Or we allow hooks like the current "TARGET_SETUP_LINK" and amend it
> with a matching "TARGET_TEARDOWN_LINK"
>
> - If a model like the "TARGET_SETUP_LINK" is used, we add a delay here as
> this call must block until the board is up.
After thinking about this, I believe that link setup and teardown should
be built into the transport layer, with the possibility of defining board-specific override
functions. I've been working on the serial transport feature, and it
has some (possibly time-consuming) operations that are needed
on the first interaction with the target, and not afterwards (during
every put, get or cmd).
I haven't seen your lava integration slides yet, but it sounds like
they have some elaborate setup and teardown operations as well.
Functionally, doing it in the transport layer would work the same,
but just be placed in the base-board.fuegoclass file, with stubs for
transports that don't require it. e.g. ov_transport_setup() and
ov_transport_teardown()
> In this case our predifined timeouts are bogus as they track not just the test
> run, but all processing.
Agreed. In general the timeouts need some better definition, as different
tests and different transports (and different boards) will need different
timeouts.
-- Tim
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-01 10:46 ` Jan-Simon Möller
@ 2017-02-02 1:57 ` Bird, Timothy
2017-02-02 6:06 ` Daniel Sangorrin
2017-02-02 9:57 ` Jan-Simon Möller
2017-02-02 5:45 ` Daniel Sangorrin
1 sibling, 2 replies; 43+ messages in thread
From: Bird, Timothy @ 2017-02-02 1:57 UTC (permalink / raw)
To: Jan-Simon Möller, Daniel Sangorrin; +Cc: fuego
> -----Original Message-----
> From: Jan-Simon Möller on Wednesday, February 01, 2017 2:46 AM
>
> Am Mittwoch, 1. Februar 2017, 09:56:03 schrieb Daniel Sangorrin:
>
> > Another approach would be to create a single
> > job and then have fuego-core take care of reading the tests specs of each
> > test from the testplan, and repeat the test for each test spec. Then a
> > modified parser would display and compare the results of the test across
> > the different test specs.
> > [Note] I think that AGL's fuego has a similar parser.
This seems to be getting into the territory of matrix testing, which is
what Avacado seems pretty good at (they use YAML to specify
the set of parameters for varying in the matrix of tests to run.
I'd like to have the dust settle on basic testing (execute a single
test, or a set of tests on a board) before we get into how to
effectively handle matrix testing. I think Jenkins has some stuff
for this as well, which I haven't looked into yet.
In any event, I want to avoid too much complexity in Fuego
that can be done by another tool.
> Having all in one job let's you not execute one specific test easily.
> But if we simplify the shell call that jenkins has to do, creating jobs
> with specific names and parameters/env-variables is trivial.
Agreed.
> E.g.:
> builders:
> - shell: |
> #
> source $FUEGO_RO/conf/boards/{machine}.sh
> source $FUEGO_CORE/engine/env/${{DISTRO}}.sh # DISTRO set in
> machine.sh
> source $FUEGO_CORE/engine/testplans/{testplan}.sh # taken from jjb
> #
> export Reboot=false
> export Rebuild=true
> export Target_Cleanup=true
> export TESTDIR={testname}
> export TESTNAME=$(echo "{testname}" | sed -e "s#.*\.##")
> timeout --signal=9 ${{JOB_TIMEOUT}} /bin/bash
> $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNAME}}.sh
>
> In this case, ${{JOB_TIMEOUT}} can be defined in the
> $FUEGO_RO/conf/boards/{machine}.sh env script.
I'd like to cut down the amount of stuff in the Jenkins job configuration,
while still allowing some parameters to be managed with the Jenkins
interface. Optimally, we could have something as simple as:
shell: fuego run ${{TESTNAME}}
with TESTDIR implied and other parameters like the JOB_TIMEOUT
being auto-calculated from data in the transport, job file, distribution,
test itself, and board file.
...
>
> We access the board very early even before the build (pre_test).
> The we build, deploy, run, fetch results, analyse. At this point
Jenkins basically has the board busy during the entire sequence.
But if fuego could run the phases independently (like bitbake
or rpm can do for the phases of their operations), then
we could have the board busy during less time.
Possibly pretest needs to be split into two parts. One that
tests the link, and the other that gathers data to save as part
of the test results.
For example, if we had something like this:
$ fuego --board beaglebone --phase build Functional.LTP
$ fuego --board beaglebone --phase "pretest,deploy,run,gather" Functional.LTP
$ fuego --board beaglebone --phase post_test Functional.LTP
I don't know if there's a way to have Jenkins only lock the node
during the middle of those operations. The last can be put in the
Jenkins post_test. I don't see any post_test operation in Daniel's
new system. Did I miss something? Where does cleanup occur?
>
> > > - Integration of board up/down could happen in a few ways:
> > > -- Either ppl should just generate a wrapper around the (blocking) batch
> > > job and trigger it>
> > > with their pre/post to their needs. Done.
> > > -- Or we allow hooks like the current "TARGET_SETUP_LINK" and amend
> it
> > > with a matching "TARGET_TEARDOWN_LINK">
> > > - If a model like the "TARGET_SETUP_LINK" is used, we add a delay here
> as
> > > this call must block until the board is up.>
> > > In this case our predifined timeouts are bogus as they track not just the
> > > test run, but all processing.
> > Good point. I wasn't using the reboot feature so I hadn't thought about it.
> > I guess we would need to define a timeout for the board to reboot, and
> then
> > the timeout for each test. The timeout for the board could be called
> > BOARD_REBOOT_TIMEOUT for example and be defined in the asdf.board
> file. And
> > the timeout for each test defined in another variable such as
> TEST_TIMEOUT.
>
> As I said, there are multiple ways to integrate that.
> a) The wrapper-job triggering the batch leaves fuego out of the picture. But
> we need to block the whole chain to make flow-control possible.
> b) TARGET_SETUP_LINK gives us a _per job_ way of brining up the board.
> Not just for the wrapp'ed batch.
>
> Most users might get away and be happy with a) . While a per-job control as
> in b) might scale better if you think of the case that you could have
> multiple target boards of the same type (multiple executors!) and could run
> jobs 'in parallel' !
>
> > BOARD_REBOOT_TIMEOUT=1m
> > TEST_TIMEOUT=15m
> > /bin/bash $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNAME}}.sh
> >
> > Then, internally the timeouts would be used separately.
> > # we could also split timeouts between execution time and post-processing
> > time
>
> Yep ... we probably should split the build, test and postproc into separate
> scripts.
> or instead of $FUEGO_CORE/engine/tests/${{TESTDIR}}/${{TESTNAME}}.sh in
> jenkins, we could call ftc as you would do on the terminal.
>
> Also the above should be in separate jobs ... think of it:
> - the build phase doesn't need to block, can run in parallel,
> e.g. on master node!
> - the test batch needs to block to allow flow-control and
> run only on the specified executor. No concurrency but multiple jobs
> of the *same* batch could run in parallel, given we can handle multiple
> boards of the same type at the same time (which is true with an
> 'orchestrator' like lava).
> - the batch is only deploy/run/fetch result but can trigger postprocessing
> - the postprocessing is another job, can run in parallel, independent of the
> board, e.g. on master
I should have read farther... this is what I describe above, but
you beat me to it.
I agree.
-- Tim
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-01 10:46 ` Jan-Simon Möller
2017-02-02 1:57 ` Bird, Timothy
@ 2017-02-02 5:45 ` Daniel Sangorrin
1 sibling, 0 replies; 43+ messages in thread
From: Daniel Sangorrin @ 2017-02-02 5:45 UTC (permalink / raw)
To: 'Jan-Simon Möller'; +Cc: 'Bird, Timothy', fuego
Hi Jan-Simon,
> -----Original Message-----
> From: Jan-Simon Möller [mailto:dl9pf@gmx.de]
> > - In my next branch, test specs have the ability to define which links
> > should be available after the test is complete. I'm using the Description
> > Setter plugin for that. Do you think that would be possible with the jjb
> > approach?.
>
> Links ... let me see
> http://docs.openstack.org/infra/jenkins-job-builder/publishers.html#publishers.description-setter ?
> should work
>
> For reference:
> http://docs.openstack.org/infra/jenkins-job-builder/definition.html
> http://docs.openstack.org/infra/jenkins-job-builder/builders.html
> http://docs.openstack.org/infra/jenkins-job-builder/publishers.html
Nice.
I think we can use it as the back end of the fuego xxx scripts. Hopefully jjb
also handles different versions of jenkins for us.
> We access the board very early even before the build (pre_test).
> The we build, deploy, run, fetch results, analyse. At this point
Ok, I know what you mean now. I think there is a trade-off between
complexity and performance. In this particular case, IMHO I think we
won't gain much for the complexity that having multiple timeouts would add.
One timeout for the whole job is simple and easy to set by the user.
Suppose a test costs 20m to complete. Separating 20 seconds for the reboot
time is not really that important.
Also running jobs in parallel on the same board is not a good idea, even if you
schedule them in phases because some tests may affect other tests
in different ways.
> > > - Integration of board up/down could happen in a few ways:
> > > -- Either ppl should just generate a wrapper around the (blocking) batch
> > > job and trigger it>
> > > with their pre/post to their needs. Done.
> > > -- Or we allow hooks like the current "TARGET_SETUP_LINK" and amend it
> > > with a matching "TARGET_TEARDOWN_LINK">
> > > - If a model like the "TARGET_SETUP_LINK" is used, we add a delay here as
> > > this call must block until the board is up.>
> > > In this case our predifined timeouts are bogus as they track not just the
> > > test run, but all processing.
> > Good point. I wasn't using the reboot feature so I hadn't thought about it.
> > I guess we would need to define a timeout for the board to reboot, and then
> > the timeout for each test. The timeout for the board could be called
> > BOARD_REBOOT_TIMEOUT for example and be defined in the asdf.board file. And
> > the timeout for each test defined in another variable such as TEST_TIMEOUT.
>
> As I said, there are multiple ways to integrate that.
> a) The wrapper-job triggering the batch leaves fuego out of the picture. But we need to block the whole chain to make flow-control
> possible.
> b) TARGET_SETUP_LINK gives us a _per job_ way of brining up the board. Not just for the wrapp'ed batch.
Not sure I understand you but if b) means that when you click on the batch job, all the dependent individual jobs are triggered and execute with their own timeouts (and possibly reboot).
This is actually how it is in my next branch. For the reboots this could be easily done by adding the reboot flag to each test (otherwise by default it doesn't reboot) on the testplan.
a) seems like a subset of b) where all REBOOT flags are set to 0 except the one for the batch job.
> Most users might get away and be happy with a) . While a per-job control as in b) might scale better if you think of the case that you
> could have
> multiple target boards of the same type (multiple executors!) and could run jobs 'in parallel' !
I would prefer b). And a) being a subset of b) we can make everyone happy.
Cheers
Daniel
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-02 1:25 ` Bird, Timothy
@ 2017-02-02 5:49 ` Daniel Sangorrin
2017-02-02 9:50 ` Jan-Simon Möller
0 siblings, 1 reply; 43+ messages in thread
From: Daniel Sangorrin @ 2017-02-02 5:49 UTC (permalink / raw)
To: 'Bird, Timothy', 'Jan-Simon Möller', fuego
> -----Original Message-----
> From: Bird, Timothy [mailto:Tim.Bird@sony.com]
> Sent: Thursday, February 02, 2017 10:26 AM
> To: Jan-Simon Möller; fuego@lists.linuxfoundation.org
> Cc: Daniel Sangorrin
> Subject: RE: [Fuego] Fuego's version up and other changes
>
>
>
> > -----Original Message-----
> > From: Jan-Simon Möller on Tuesday, January 31, 2017 12:55 PM
> >...
> > * A extension to this approach IMHO would be the use of jenkins-job-
> > builder.
> >
> > The beauty of this is that we do not have to deal with the xml ourselves.
> > Instead we define yaml files and let jjb create the necessary xml.
> > Another plus is that such an approach would also allow ppl to add
> > the testcases to an *existing* jenkins using the jjb templates.
> >
> > Let me illustrate this with a jump-start example:
> >
> >
> > Installation of jjb (in fuego container):
> > $ cd ~/
> > $ sudo apt-get install python-pip
> > $ sudo pip install jenkins-job-builder
> > # should be 1.6.x
> >
> > $ cat <<EOF > jjb-conf.ini
> > [job_builder]
> > ignore_cache=True
> > keep_descriptions=False
> > include_path=.:scripts:~/git/
> > recursive=True
> > [jenkins]
> > user=Anonymous
> > password=
> > url=https://localhost/
> > ##### This is deprecated, use job_builder section instead
> > #ignore_cache=True
> > query_plugins_info=False
> > EOF
> >
> > $ mkdir jjb
> > $ cd jjb/
> > $ mkdir testjob
> > $ cat <<EOF > testjob/testjob.yaml
> > - project:
> > name: fuego-tests-smoke
> > machine:
> > - raspberrypi3
> > testname:
> > -
> > -
> > -
> > -
> > -
> > jobs:
> > - {machine}.smoke.batch
> > - {machine}.smoke.{testname}
> >
> > - job-template:
> > name: '{machine}.smoke.batch'
> > project-type: freestyle
> > concurrent: false
> > node: master
> >
> > builders:
> > - orchestrator-script ON
> > - trigger '{machine}.smoke.*' BLOCKING
> > - orchestrator-script OFF ALWAYS
> >
> > - job-template:
> > name: '{machine}.smoke.{testname}
> > project-type: freestyle
> > concurrent: false
> > node: {machine}
>
> I like the idea of using a tool maintained by others
> to handle the interaction with Jenkins. This will
> insulate us from additional Jenkins changes. I do
> worry a bit, however, about exposing Fuego users
> to a whole bunch of extraneous Jenkins or jjb
> info that they don't need just to install and run tests
> on their boards.
>
> If the complexity of the above can be hidden from
> the 'casual' user, who has a board, and just wants to
> run an existing test on it, then that would be good.
> For me, this includes the whole issue of plans and
> specs as well. I'd like to make awareness of those
> optional for Fuego users who don't need them.
>
> Daniel's scripts take a minimum of arguments, and
> do a pretty good job in this regards. If they can
> use jjb behind the scenes, making it so that end users
> don't have to fiddle with YAML files full of weird,
> Jenkins parameters, that would be good, IMHO.
Yeah, I agree with that. The simpler for the users the better.
Also for the poor developers.
Daniel
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-02 1:35 ` Bird, Timothy
@ 2017-02-02 5:52 ` Daniel Sangorrin
2017-02-02 10:04 ` Jan-Simon Möller
2017-02-02 20:51 ` Kevin Hilman
2 siblings, 0 replies; 43+ messages in thread
From: Daniel Sangorrin @ 2017-02-02 5:52 UTC (permalink / raw)
To: 'Bird, Timothy', 'Jan-Simon Möller', fuego
> -----Original Message-----
> From: Bird, Timothy [mailto:Tim.Bird@sony.com]
> > In this case our predifined timeouts are bogus as they track not just the test
> > run, but all processing.
> Agreed. In general the timeouts need some better definition, as different
> tests and different transports (and different boards) will need different
> timeouts.
And the OS etc..
In the end timeouts need to be tuned by the user when a test fails by timeout
but it is not that hard.
Cheers
Daniel
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-02 1:57 ` Bird, Timothy
@ 2017-02-02 6:06 ` Daniel Sangorrin
2017-02-03 1:04 ` Daniel Sangorrin
2017-02-02 9:57 ` Jan-Simon Möller
1 sibling, 1 reply; 43+ messages in thread
From: Daniel Sangorrin @ 2017-02-02 6:06 UTC (permalink / raw)
To: 'Bird, Timothy', 'Jan-Simon Möller'; +Cc: fuego
> -----Original Message-----
> From: Bird, Timothy [mailto:Tim.Bird@sony.com]
> I don't know if there's a way to have Jenkins only lock the node
> during the middle of those operations. The last can be put in the
> Jenkins post_test. I don't see any post_test operation in Daniel's
> new system. Did I miss something? Where does cleanup occur?
Oops My bad, I will fix it.
Thanks,
Daniel
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-02 5:49 ` Daniel Sangorrin
@ 2017-02-02 9:50 ` Jan-Simon Möller
0 siblings, 0 replies; 43+ messages in thread
From: Jan-Simon Möller @ 2017-02-02 9:50 UTC (permalink / raw)
To: Daniel Sangorrin; +Cc: 'Bird, Timothy', fuego
Am Donnerstag, 2. Februar 2017, 14:49:08 schrieb Daniel Sangorrin:
> >
> > If the complexity of the above can be hidden from
> > the 'casual' user, who has a board, and just wants to
> > run an existing test on it, then that would be good.
> > For me, this includes the whole issue of plans and
> > specs as well. I'd like to make awareness of those
> > optional for Fuego users who don't need them.
> >
> > Daniel's scripts take a minimum of arguments, and
> > do a pretty good job in this regards. If they can
> > use jjb behind the scenes, making it so that end users
> > don't have to fiddle with YAML files full of weird,
> > Jenkins parameters, that would be good, IMHO.
>
> Yeah, I agree with that. The simpler for the users the better.
> Also for the poor developers.
>
I fully agree, it should be simpler to users and the developer.
I think this can be done and it will open-up cycles as we have to deal less
with jenkins itself and more with the tests.
Best,
JS
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-02 1:57 ` Bird, Timothy
2017-02-02 6:06 ` Daniel Sangorrin
@ 2017-02-02 9:57 ` Jan-Simon Möller
1 sibling, 0 replies; 43+ messages in thread
From: Jan-Simon Möller @ 2017-02-02 9:57 UTC (permalink / raw)
To: Bird, Timothy; +Cc: fuego
Am Donnerstag, 2. Februar 2017, 01:57:34 schrieb Bird, Timothy:
> I'd like to cut down the amount of stuff in the Jenkins job configuration,
> while still allowing some parameters to be managed with the Jenkins
> interface. Optimally, we could have something as simple as:
> shell: fuego run ${{TESTNAME}}
> with TESTDIR implied and other parameters like the JOB_TIMEOUT
> being auto-calculated from data in the transport, job file, distribution,
> test itself, and board file.
Yes, if we can reduce this to just the minimum required parameters, it would
simplify a lot.
The folders are implied and the rest comes out of the conf files.
There are 2 directions as we discussed previously:
- fuego being the central wrapper for a whole series of tests
shell: fuego run -m {machine} -t {testspec}
-- in this case we'd likely just have one jenkins job to create
and the loop + flow-control is all on fuego
-- downside is that one error will crash the whole series and there
is no way to restart a single test easily
- fuego starting single tests
shell: fuego run {testname}
-- multiple jenkins jobs, flow-control in jenkins
-- crash limited to one test
Leaning towards "fuego run {testname}" b/c of the finer control imho.
Best,
--
Jan-Simon Möller
dl9pf@gmx.de
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-02 1:35 ` Bird, Timothy
2017-02-02 5:52 ` Daniel Sangorrin
@ 2017-02-02 10:04 ` Jan-Simon Möller
2017-02-02 20:51 ` Kevin Hilman
2 siblings, 0 replies; 43+ messages in thread
From: Jan-Simon Möller @ 2017-02-02 10:04 UTC (permalink / raw)
To: Bird, Timothy; +Cc: fuego
Am Donnerstag, 2. Februar 2017, 01:35:58 schrieb Bird, Timothy:
> > - If a model like the "TARGET_SETUP_LINK" is used, we add a delay here as
> > this call must block until the board is up.
>
> After thinking about this, I believe that link setup and teardown should
> be built into the transport layer, with the possibility of defining
> board-specific override functions. I've been working on the serial
> transport feature, and it has some (possibly time-consuming) operations
> that are needed
> on the first interaction with the target, and not afterwards (during
> every put, get or cmd).
Agree, but there is a catch: if I put the lava-support into the transport
(e.g. on first cmd/get/put) - I've no knowledge about the job and how long it
might take and we'd have to add some state or timeout in the background to
release the connection as we don't know what the last "cmd" for the session
is.
So just in the transport layer is not enough for this type of integration.
Currently I'm using the "ssh" transport. The job setup/teardown is done
pre/post the test job.
>
> I haven't seen your lava integration slides yet, but it sounds like
> they have some elaborate setup and teardown operations as well.
Still hacking, I'll send them on friday but don't expect miracles.
> Functionally, doing it in the transport layer would work the same,
> but just be placed in the base-board.fuegoclass file, with stubs for
> transports that don't require it. e.g. ov_transport_setup() and
> ov_transport_teardown()
Ok, that is a possibility, but see my previous comment. We don't have any
knowledge about the session right now.
There might also be the question if we pool multiple tests and run them in one
setup/teardown instead of doing the setup/teardown for _each_ test
individually. So there might also be the need for a "keepalive" flag or
counter.
> > In this case our predifined timeouts are bogus as they track not just the
> > test run, but all processing.
>
> Agreed. In general the timeouts need some better definition, as different
> tests and different transports (and different boards) will need different
> timeouts.
We need to calculate them for each of the identified phases (setup, build,
deploy, run, ...)
Best,
--
--
Jan-Simon Möller
dl9pf@gmx.de
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-02 1:35 ` Bird, Timothy
2017-02-02 5:52 ` Daniel Sangorrin
2017-02-02 10:04 ` Jan-Simon Möller
@ 2017-02-02 20:51 ` Kevin Hilman
2017-02-02 22:16 ` Bird, Timothy
2 siblings, 1 reply; 43+ messages in thread
From: Kevin Hilman @ 2017-02-02 20:51 UTC (permalink / raw)
To: Bird, Timothy; +Cc: fuego
"Bird, Timothy" <Tim.Bird@sony.com> writes:
[...]
> I haven't seen your lava integration slides yet, but it sounds like
> they have some elaborate setup and teardown operations as well.
Speaking of LAVA (and hijacking the thread a bit, since I just
subscribed...)
I'm new to fuego and would love to see/hear more about the LAVA
integration status/plans/future etc. (Unfortunately, I won't be a FOSDEM
to hear Jan-Simon's talk.)
Fuego seems OK in terms of running tests, but actually adding boards is,
well... less than what I was hoping for.
There are many labs out there with lots of different boards hooked up to
LAVA already, so having a way for Fuego to submit tests and extract
results from LAVA seems like a good area for future work.
Kevin
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-02 20:51 ` Kevin Hilman
@ 2017-02-02 22:16 ` Bird, Timothy
2017-02-02 23:37 ` Kevin Hilman
2017-02-04 0:07 ` Jan-Simon Möller
0 siblings, 2 replies; 43+ messages in thread
From: Bird, Timothy @ 2017-02-02 22:16 UTC (permalink / raw)
To: Kevin Hilman; +Cc: fuego
> -----Original Message-----
> From: Kevin Hilman on Thursday, February 02, 2017 12:52 PM
> "Bird, Timothy" <Tim.Bird@sony.com> writes:
>
> [...]
>
> > I haven't seen your lava integration slides yet, but it sounds like
> > they have some elaborate setup and teardown operations as well.
>
> Speaking of LAVA (and hijacking the thread a bit, since I just
> subscribed...)
>
> I'm new to fuego and would love to see/hear more about the LAVA
> integration status/plans/future etc. (Unfortunately, I won't be a FOSDEM
> to hear Jan-Simon's talk.)
He'll be giving the talk at ELC, so if you're coming to that you can
see it there (or a few weeks later on video).
> Fuego seems OK in terms of running tests, but actually adding boards is,
> well... less than what I was hoping for.
>
> There are many labs out there with lots of different boards hooked up to
> LAVA already, so having a way for Fuego to submit tests and extract
> results from LAVA seems like a good area for future work.
Ok - now seems like as good a time as any to start rambling on about the
Fuego roadmap and priorities... :-)
Yeah - I've consciously NOT put a lot of work into Fuego's board handling,
since there are other systems (including LAVA) that do this already.
Right now, Fuego supports software reboot, but doesn't have an API
for the kind of hardware control of power, reboot, reflashing, serial console,
etc. needed for kernel bring-up testing, like what KernelCI does.
Since KernelCI is doing a good job in that space (Continuous Integration
kernel boot testing at top of tree on a matrix of boards), I've been trying
to position Fuego as more of a distribution tester. So, for now we're assuming
that the kernel boot and networking are stable, and Fuego is focused on
testing features and packages.
Right now, we're pretty reliant on a stable ssh, shell and filesystem. We've
lately been workingon improving our Jenkins integration (decoupling from a
specific version of Jenkins, or even from Jenkins itself, with our own CLI
test tool), and migrating towards a standalone test package format
(so all tests don't have to be in our repository, and we can distribute
tests independently from the Fuego core system).
Big areas to follow are:
- canonical test output format
- ability to gather test results from nodes
- ability to download test packages from a "store"
The area I'd like Fuego to focus on, that I don't see other Linux test
frameworks doing, is actually building out a complement of tests.
I'd like Fuego to be a "test distribution" with lots of tests for different
aspects of the system (file system, networking, realtime, power management,
etc.) I see it as important to get the package system and test API
right, before doing a concerted push to create all the different tests
in different areas. This is why the current focus is where it is.
Our first priority is as a distribution tester, where someone (a Linux
Foundation project, a semiconductor vendor, or an internal team
at a big company) has produced a distribution of Linux, and wants to
validate that it has few bugs, and continues working as intended when
delivered to other groups. The ideal is that the whole test framework
can be delivered with the distribution, and the distro customer can
continue to validate for themselves that nothing has broken in the distro
while thy develop their end product (and modify parts of the distro).
Anyway - I'm also interested in Jan-Simon's LAVA integration.
It would be great to be able to run Fuego tests in current KernelCI
labs.
-- Tim
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-02 22:16 ` Bird, Timothy
@ 2017-02-02 23:37 ` Kevin Hilman
2017-02-04 0:07 ` Jan-Simon Möller
1 sibling, 0 replies; 43+ messages in thread
From: Kevin Hilman @ 2017-02-02 23:37 UTC (permalink / raw)
To: Bird, Timothy; +Cc: fuego
"Bird, Timothy" <Tim.Bird@sony.com> writes:
>> -----Original Message-----
>> From: Kevin Hilman on Thursday, February 02, 2017 12:52 PM
>> "Bird, Timothy" <Tim.Bird@sony.com> writes:
>>
>> [...]
>>
>> > I haven't seen your lava integration slides yet, but it sounds like
>> > they have some elaborate setup and teardown operations as well.
>>
>> Speaking of LAVA (and hijacking the thread a bit, since I just
>> subscribed...)
>>
>> I'm new to fuego and would love to see/hear more about the LAVA
>> integration status/plans/future etc. (Unfortunately, I won't be a FOSDEM
>> to hear Jan-Simon's talk.)
>
> He'll be giving the talk at ELC, so if you're coming to that you can
> see it there (or a few weeks later on video).
>
>> Fuego seems OK in terms of running tests, but actually adding boards is,
>> well... less than what I was hoping for.
>>
>> There are many labs out there with lots of different boards hooked up to
>> LAVA already, so having a way for Fuego to submit tests and extract
>> results from LAVA seems like a good area for future work.
>
> Ok - now seems like as good a time as any to start rambling on about the
> Fuego roadmap and priorities... :-)
>
> Yeah - I've consciously NOT put a lot of work into Fuego's board handling,
> since there are other systems (including LAVA) that do this already.
> Right now, Fuego supports software reboot, but doesn't have an API
> for the kind of hardware control of power, reboot, reflashing, serial console,
> etc. needed for kernel bring-up testing, like what KernelCI does.
>
> Since KernelCI is doing a good job in that space (Continuous Integration
> kernel boot testing at top of tree on a matrix of boards), I've been trying
> to position Fuego as more of a distribution tester. So, for now we're assuming
> that the kernel boot and networking are stable, and Fuego is focused on
> testing features and packages.
Does Fuego handle the distro install/upgrade piece? I couldn't tell so
far if it does.
Anyways, I agree that getting a broad suite of tests on a stable
kernel/rootfs is the right focus for testing, at least for starters.
While kernel CI does lots of upstream trees, it also does stable trees
(and the stable-rc tree) where things should, by definition, be
mostly stable, so we've been aiming at stable for broad testing, but
also care about a rather wide variety of hardware, across several arch
(currently x86, arm, arm64 and MIPS.)
Also, focusing testing on stable means better starting point for distro
kernels too. In addition, being able to compare a test report from a
distro kernel to the stable kernel it was based on would be very useful
data.
> Right now, we're pretty reliant on a stable ssh, shell and filesystem. We've
> lately been workingon improving our Jenkins integration (decoupling from a
> specific version of Jenkins, or even from Jenkins itself, with our own CLI
> test tool), and migrating towards a standalone test package format
> (so all tests don't have to be in our repository, and we can distribute
> tests independently from the Fuego core system).
In scaling up kernel CI over the last year or so, de-coupling from
Jenkins is a very good idea. ;)
> Big areas to follow are:
> - canonical test output format
> - ability to gather test results from nodes
> - ability to download test packages from a "store"
Thanks to a pointer from Jan-Simon, I started looking at Avocado
recently. In terms of a standalone CLI tool, it seems to do all of this
(except maybe the store part.)
Avocado also seems ot have a great handle on testing where there's more
than one "entity" involved (e.g. KVM boot with a host and multiple
guests.) Not sure if Fuego has that yet?
Curious what your opinions of Avocado are and how it might fit into the
Fuego vision? I've only briefly started looking at both Fuego and
Avocado, so I don't really have a good grip yet on where there may be
overlap.
> The area I'd like Fuego to focus on, that I don't see other Linux test
> frameworks doing, is actually building out a complement of tests.
> I'd like Fuego to be a "test distribution" with lots of tests for different
> aspects of the system (file system, networking, realtime, power management,
> etc.) I see it as important to get the package system and test API
> right, before doing a concerted push to create all the different tests
> in different areas. This is why the current focus is where it is.
I fully agree. Writing and running tests is easy. The hard part is
getting them packaged in a way that's usuable across a wide range of
hardware, and parsing/processing the results into useful reports that
include historical/regression analysis. Also making the tests tolerant
of interfaces/modules etc. that may or may not be present.
> Our first priority is as a distribution tester, where someone (a Linux
> Foundation project, a semiconductor vendor, or an internal team
> at a big company) has produced a distribution of Linux, and wants to
> validate that it has few bugs, and continues working as intended when
> delivered to other groups. The ideal is that the whole test framework
> can be delivered with the distribution, and the distro customer can
> continue to validate for themselves that nothing has broken in the distro
> while thy develop their end product (and modify parts of the distro).
That sounds like a good priority.
What distro(s) are you focused on now? It seems like there should be
some definition of minimal/baseline distro (e.g. yocto
core-image-minimal) for starters.
> Anyway - I'm also interested in Jan-Simon's LAVA integration.
> It would be great to be able to run Fuego tests in current KernelCI
> labs.
Exactly. If that was a possibility, we could offer a broad range of
hardware for experimentation with Fuego very easily.
Kevin
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-02 6:06 ` Daniel Sangorrin
@ 2017-02-03 1:04 ` Daniel Sangorrin
2017-02-13 3:51 ` Daniel Sangorrin
0 siblings, 1 reply; 43+ messages in thread
From: Daniel Sangorrin @ 2017-02-03 1:04 UTC (permalink / raw)
To: 'Bird, Timothy', 'Jan-Simon Möller'; +Cc: fuego
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Thursday, February 02, 2017 3:07 PM
> > -----Original Message-----
> > From: Bird, Timothy [mailto:Tim.Bird@sony.com]
> > I don't know if there's a way to have Jenkins only lock the node
> > during the middle of those operations. The last can be put in the
> > Jenkins post_test. I don't see any post_test operation in Daniel's
> > new system. Did I miss something? Where does cleanup occur?
>
> Oops My bad, I will fix it.
Ok, it's now fixed:
https://bitbucket.org/nirrognas/fuego-core/commits/03cabaeefa86a7e1d02dac8146d08f22db9c6a21?at=next
https://bitbucket.org/nirrognas/fuego/commits/c37f721173f705c66432a1feedf05eeaabf3d9c6?at=next
Additionally I have fixed the target_cleanup function:
function target_cleanup {
- cmd "rm -rf $BOARD_TESTDIR/* /tmp/* $BOARD_TESTDIR/.local"
+ cmd "rm -rf $BOARD_TESTDIR/fuego.$1 /tmp/fuego.$1 $BOARD_TESTDIR/.local"
}
Cleaning the whole tmp directory seems quite harsh, so now it only cleans up the
directory corresponding to the current test. I also deleted the .local reference since
I couldn't find any reference to it in the code (if you have objections let me know).
Also, notice that "target_cleanup" really means "target_pre_cleanup". So it's not
related with post_test. I think we should also add a target_post_cleanup inside
post_test that can be optionally disabled. When a test fails
I often ssh into the target and try to run the test manually. If we cleanup the
test by default, it becomes harder to debug it.
What do you think?
Cheers
Daniel
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-02 22:16 ` Bird, Timothy
2017-02-02 23:37 ` Kevin Hilman
@ 2017-02-04 0:07 ` Jan-Simon Möller
2017-02-07 1:56 ` Daniel Sangorrin
1 sibling, 1 reply; 43+ messages in thread
From: Jan-Simon Möller @ 2017-02-04 0:07 UTC (permalink / raw)
To: Bird, Timothy, Daniel Sangorrin; +Cc: fuego
Hi Tim, Kevin, Daniel ...
Slides are boring ;) - let's talk shop on some code.
But bear with me:
it's already after midnight, so any nonsense is due to the late hour.
See:
https://bitbucket.org/dl9pf/fuego
https://bitbucket.org/dl9pf/fuego-core
This is the "every test job does his own lava job" version.
Quite robust, but not as speedy as you'd like it.
@Daniel: I had to completely rm the timeouts, they were way to short.
@ALL if you want to try the example with the rpi3, ping me via PM and I can
send you lava credentials for the raspberrypi3.lava file or the jenkins in
reach of the board.
Best,
--
Dipl.-Ing.
Jan-Simon Möller
jansimon.moeller@gmx.de
On Thursday 02 February 2017 22:16:17 Bird, Timothy wrote:
> > -----Original Message-----
> > From: Kevin Hilman on Thursday, February 02, 2017 12:52 PM
> > "Bird, Timothy" <Tim.Bird@sony.com> writes:
> >
> > [...]
> >
> > > I haven't seen your lava integration slides yet, but it sounds like
> > > they have some elaborate setup and teardown operations as well.
> >
> > Speaking of LAVA (and hijacking the thread a bit, since I just
> > subscribed...)
> >
> > I'm new to fuego and would love to see/hear more about the LAVA
> > integration status/plans/future etc. (Unfortunately, I won't be a FOSDEM
> > to hear Jan-Simon's talk.)
>
> He'll be giving the talk at ELC, so if you're coming to that you can
> see it there (or a few weeks later on video).
>
> > Fuego seems OK in terms of running tests, but actually adding boards is,
> > well... less than what I was hoping for.
> >
> > There are many labs out there with lots of different boards hooked up to
> > LAVA already, so having a way for Fuego to submit tests and extract
> > results from LAVA seems like a good area for future work.
>
> Ok - now seems like as good a time as any to start rambling on about the
> Fuego roadmap and priorities... :-)
>
> Yeah - I've consciously NOT put a lot of work into Fuego's board handling,
> since there are other systems (including LAVA) that do this already.
> Right now, Fuego supports software reboot, but doesn't have an API
> for the kind of hardware control of power, reboot, reflashing, serial
> console, etc. needed for kernel bring-up testing, like what KernelCI does.
>
> Since KernelCI is doing a good job in that space (Continuous Integration
> kernel boot testing at top of tree on a matrix of boards), I've been trying
> to position Fuego as more of a distribution tester. So, for now we're
> assuming that the kernel boot and networking are stable, and Fuego is
> focused on testing features and packages.
>
> Right now, we're pretty reliant on a stable ssh, shell and filesystem.
> We've lately been workingon improving our Jenkins integration (decoupling
> from a specific version of Jenkins, or even from Jenkins itself, with our
> own CLI test tool), and migrating towards a standalone test package format
> (so all tests don't have to be in our repository, and we can distribute
> tests independently from the Fuego core system).
>
> Big areas to follow are:
> - canonical test output format
> - ability to gather test results from nodes
> - ability to download test packages from a "store"
>
> The area I'd like Fuego to focus on, that I don't see other Linux test
> frameworks doing, is actually building out a complement of tests.
> I'd like Fuego to be a "test distribution" with lots of tests for different
> aspects of the system (file system, networking, realtime, power management,
> etc.) I see it as important to get the package system and test API
> right, before doing a concerted push to create all the different tests
> in different areas. This is why the current focus is where it is.
>
> Our first priority is as a distribution tester, where someone (a Linux
> Foundation project, a semiconductor vendor, or an internal team
> at a big company) has produced a distribution of Linux, and wants to
> validate that it has few bugs, and continues working as intended when
> delivered to other groups. The ideal is that the whole test framework
> can be delivered with the distribution, and the distro customer can
> continue to validate for themselves that nothing has broken in the distro
> while thy develop their end product (and modify parts of the distro).
>
> Anyway - I'm also interested in Jan-Simon's LAVA integration.
> It would be great to be able to run Fuego tests in current KernelCI
> labs.
> -- Tim
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-04 0:07 ` Jan-Simon Möller
@ 2017-02-07 1:56 ` Daniel Sangorrin
0 siblings, 0 replies; 43+ messages in thread
From: Daniel Sangorrin @ 2017-02-07 1:56 UTC (permalink / raw)
To: 'Jan-Simon Möller', 'Bird, Timothy'; +Cc: fuego
Hi Jan-Simon,
> -----Original Message-----
> From: Jan-Simon Möller [mailto:dl9pf@gmx.de]
> Sent: Saturday, February 04, 2017 9:07 AM
> To: Bird, Timothy; Daniel Sangorrin
> Cc: Kevin Hilman; fuego@lists.linuxfoundation.org
> Subject: Re: [Fuego] Fuego's version up and other changes
>
> Hi Tim, Kevin, Daniel ...
>
> Slides are boring ;) - let's talk shop on some code.
>
> But bear with me:
> it's already after midnight, so any nonsense is due to the late hour.
>
> See:
> https://bitbucket.org/dl9pf/fuego
> https://bitbucket.org/dl9pf/fuego-core
>
> This is the "every test job does his own lava job" version.
> Quite robust, but not as speedy as you'd like it.
Thanks for the code!!
I have little experience with LAVA but from what I understand, you
are creating a "LAVA Hacking session" (which just means that you
install a public SSH KEY on the target) and let Fuego access the board
directly.
+-- kernel/rootfs---> [HTTP server]----+
| |
[FUEGO] ---------lava-tool --------------->[LAVA]---------[BOARD]
| |
+-------------------------------SSH---------------------------+
Note 1: The board is required to have sshd installed and
be accessible from FUEGO (may require a VPN to bypass firewalls).
Note 2: LAVA's role in this case consists of scheduling (orchestrating?) the board
among the submitted jobs. And also it can automatically setup the kernel/nfs
root filesystem I guess.
I think this approach integrates well with Fuego because it basically
works as usual. Also I can see the value in the fact that there are board
farms ready to use through LAVA.
Do you think this would be enough or do you want to add more functionality?
By the way, I was just asking myself as an exercise, could we do
the same with the functionality we already have?.
I guess it would look like this:
[FTC/CURL] ---------REST-API--------------->[FUEGO]---ssh/serial---[BOARD]
|
[WEB BROWSER] ---------------HTTPS-----------+
- Orchestrator functionality: I think Jenkins already schedules jobs in a FIFO
fashion by default.
- Job submission: lava-tool's submit-job, job-status, and cancel-job
could be replaced by the Jenkins REST API.
- Developers shell (for troubleshooting): there is this plugin for jenkins that seems to do the job
https://wiki.jenkins-ci.org/display/JENKINS/Remote+Terminal+Access+Plugin
My opinion is that we should support both approaches so that people
that want to use LAVA can use it, and people that want something more
simple to install can go with the default installation.
> @Daniel: I had to completely rm the timeouts, they were way to short.
Hehe sorry about that.
# You could have made them longer instead.
Cheers,
Daniel
> @ALL if you want to try the example with the rpi3, ping me via PM and I can
> send you lava credentials for the raspberrypi3.lava file or the jenkins in
> reach of the board.
>
> Dipl.-Ing.
> Jan-Simon Möller
>
> jansimon.moeller@gmx.de
> On Thursday 02 February 2017 22:16:17 Bird, Timothy wrote:
> > > -----Original Message-----
> > > From: Kevin Hilman on Thursday, February 02, 2017 12:52 PM
> > > "Bird, Timothy" <Tim.Bird@sony.com> writes:
> > >
> > > [...]
> > >
> > > > I haven't seen your lava integration slides yet, but it sounds like
> > > > they have some elaborate setup and teardown operations as well.
> > >
> > > Speaking of LAVA (and hijacking the thread a bit, since I just
> > > subscribed...)
> > >
> > > I'm new to fuego and would love to see/hear more about the LAVA
> > > integration status/plans/future etc. (Unfortunately, I won't be a FOSDEM
> > > to hear Jan-Simon's talk.)
> >
> > He'll be giving the talk at ELC, so if you're coming to that you can
> > see it there (or a few weeks later on video).
> >
> > > Fuego seems OK in terms of running tests, but actually adding boards is,
> > > well... less than what I was hoping for.
> > >
> > > There are many labs out there with lots of different boards hooked up to
> > > LAVA already, so having a way for Fuego to submit tests and extract
> > > results from LAVA seems like a good area for future work.
> >
> > Ok - now seems like as good a time as any to start rambling on about the
> > Fuego roadmap and priorities... :-)
> >
> > Yeah - I've consciously NOT put a lot of work into Fuego's board handling,
> > since there are other systems (including LAVA) that do this already.
> > Right now, Fuego supports software reboot, but doesn't have an API
> > for the kind of hardware control of power, reboot, reflashing, serial
> > console, etc. needed for kernel bring-up testing, like what KernelCI does.
> >
> > Since KernelCI is doing a good job in that space (Continuous Integration
> > kernel boot testing at top of tree on a matrix of boards), I've been trying
> > to position Fuego as more of a distribution tester. So, for now we're
> > assuming that the kernel boot and networking are stable, and Fuego is
> > focused on testing features and packages.
> >
> > Right now, we're pretty reliant on a stable ssh, shell and filesystem.
> > We've lately been workingon improving our Jenkins integration (decoupling
> > from a specific version of Jenkins, or even from Jenkins itself, with our
> > own CLI test tool), and migrating towards a standalone test package format
> > (so all tests don't have to be in our repository, and we can distribute
> > tests independently from the Fuego core system).
> >
> > Big areas to follow are:
> > - canonical test output format
> > - ability to gather test results from nodes
> > - ability to download test packages from a "store"
> >
> > The area I'd like Fuego to focus on, that I don't see other Linux test
> > frameworks doing, is actually building out a complement of tests.
> > I'd like Fuego to be a "test distribution" with lots of tests for different
> > aspects of the system (file system, networking, realtime, power management,
> > etc.) I see it as important to get the package system and test API
> > right, before doing a concerted push to create all the different tests
> > in different areas. This is why the current focus is where it is.
> >
> > Our first priority is as a distribution tester, where someone (a Linux
> > Foundation project, a semiconductor vendor, or an internal team
> > at a big company) has produced a distribution of Linux, and wants to
> > validate that it has few bugs, and continues working as intended when
> > delivered to other groups. The ideal is that the whole test framework
> > can be delivered with the distribution, and the distro customer can
> > continue to validate for themselves that nothing has broken in the distro
> > while thy develop their end product (and modify parts of the distro).
> >
> > Anyway - I'm also interested in Jan-Simon's LAVA integration.
> > It would be great to be able to run Fuego tests in current KernelCI
> > labs.
> > -- Tim
>
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-03 1:04 ` Daniel Sangorrin
@ 2017-02-13 3:51 ` Daniel Sangorrin
2017-02-14 19:55 ` Bird, Timothy
0 siblings, 1 reply; 43+ messages in thread
From: Daniel Sangorrin @ 2017-02-13 3:51 UTC (permalink / raw)
To: 'Bird, Timothy', 'Jan-Simon Möller'; +Cc: fuego
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> Sent: Friday, February 03, 2017 10:05 AM
> To: 'Bird, Timothy'; 'Jan-Simon Möller'
> Cc: fuego@lists.linuxfoundation.org
> Subject: Re: [Fuego] Fuego's version up and other changes
>
> > -----Original Message-----
> > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-bounces@lists.linuxfoundation.org] On Behalf Of Daniel Sangorrin
> > Sent: Thursday, February 02, 2017 3:07 PM
> > > -----Original Message-----
> > > From: Bird, Timothy [mailto:Tim.Bird@sony.com]
> > > I don't know if there's a way to have Jenkins only lock the node
> > > during the middle of those operations. The last can be put in the
> > > Jenkins post_test. I don't see any post_test operation in Daniel's
> > > new system. Did I miss something? Where does cleanup occur?
> >
> > Oops My bad, I will fix it.
>
> Ok, it's now fixed:
> https://bitbucket.org/nirrognas/fuego-core/commits/03cabaeefa86a7e1d02dac8146d08f22db9c6a21?at=next
> https://bitbucket.org/nirrognas/fuego/commits/c37f721173f705c66432a1feedf05eeaabf3d9c6?at=next
>
> Additionally I have fixed the target_cleanup function:
>
> function target_cleanup {
> - cmd "rm -rf $BOARD_TESTDIR/* /tmp/* $BOARD_TESTDIR/.local"
> + cmd "rm -rf $BOARD_TESTDIR/fuego.$1 /tmp/fuego.$1 $BOARD_TESTDIR/.local"
> }
>
> Cleaning the whole tmp directory seems quite harsh, so now it only cleans up the
> directory corresponding to the current test. I also deleted the .local reference since
> I couldn't find any reference to it in the code (if you have objections let me know).
>
> Also, notice that "target_cleanup" really means "target_pre_cleanup". So it's not
> related with post_test. I think we should also add a target_post_cleanup inside
> post_test that can be optionally disabled. When a test fails
> I often ssh into the target and try to run the test manually. If we cleanup the
> test by default, it becomes harder to debug it.
> What do you think?
Ok, so I have split target PRE and POST cleanup. They can be now individually
enabled or disabled through the corresponding entries in your testplan (check
testplan_default.json on my next branch).
I also enabled support for the "Rebuild" flag, and the "Reboot" flag.
However, I noticed that nobody is using the "Reboot" flag!! :O
I was thinking to add it to pre_tests, but I'm not sure how well LAVA would
integrate with such changes. In the end, I decided to leave it for future work.
Thanks,
Daniel
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-13 3:51 ` Daniel Sangorrin
@ 2017-02-14 19:55 ` Bird, Timothy
2017-02-15 18:39 ` Kevin Hilman
0 siblings, 1 reply; 43+ messages in thread
From: Bird, Timothy @ 2017-02-14 19:55 UTC (permalink / raw)
To: Daniel Sangorrin, 'Jan-Simon Möller'; +Cc: fuego
> -----Original Message-----
> From: Daniel Sangorrin on Sunday, February 12, 2017 7:51 PM
> > -----Original Message-----
> > From: Daniel Sangorrin on Friday, February 03, 2017 10:05 AM
> > > -----Original Message-----
> > > From: Daniel Sangorrin on Thursday, February 02, 2017 3:07 PM
> > > > -----Original Message-----
> > > > From: Bird, Timothy [mailto:Tim.Bird@sony.com]
> > > > I don't know if there's a way to have Jenkins only lock the node
> > > > during the middle of those operations. The last can be put in the
> > > > Jenkins post_test. I don't see any post_test operation in Daniel's
> > > > new system. Did I miss something? Where does cleanup occur?
> > >
> > > Oops My bad, I will fix it.
> >
> > Ok, it's now fixed:
> > https://bitbucket.org/nirrognas/fuego-
> core/commits/03cabaeefa86a7e1d02dac8146d08f22db9c6a21?at=next
> >
> https://bitbucket.org/nirrognas/fuego/commits/c37f721173f705c66432a1fee
> df05eeaabf3d9c6?at=next
> >
> > Additionally I have fixed the target_cleanup function:
> >
> > function target_cleanup {
> > - cmd "rm -rf $BOARD_TESTDIR/* /tmp/* $BOARD_TESTDIR/.local"
> > + cmd "rm -rf $BOARD_TESTDIR/fuego.$1 /tmp/fuego.$1
> $BOARD_TESTDIR/.local"
> > }
> >
> > Cleaning the whole tmp directory seems quite harsh, so now it only cleans
> up the
> > directory corresponding to the current test. I also deleted the .local
> reference since
> > I couldn't find any reference to it in the code (if you have objections let me
> know).
I did a similar change in my 'next' branch, but also added some code to
handle a board-specific temp directory, specified with
FUEGO_TARGET_TMP. This was needed to use a persistent tmp directory
for the Benchmark.reboot test. Some boards clear the /tmp directory on a boot,
and Fuego doesn't like it when the log files disappear in the middle of the test.
See https://bitbucket.org/tbird20d/fuego-core/commits/0adfb8f56688ab3af5e83bf885674b22b2f62b28
When we merge our code based, we'll have to get these different features integrated.
I was not as brave as you, so I didn't remove the .local dir. But I agree that it should go.
We should start removing things that are not used anywhere. Well, maybe we should
actually implement support for the Reboot flag. This seems like a handy thing to use
to ensure the machine starts in a pristine state.
> >
> > Also, notice that "target_cleanup" really means "target_pre_cleanup". So
> it's not
> > related with post_test. I think we should also add a target_post_cleanup
> inside
> > post_test that can be optionally disabled. When a test fails
> > I often ssh into the target and try to run the test manually. If we cleanup
> the
> > test by default, it becomes harder to debug it.
> > What do you think?
I totally agree. These are different operations.
>
> Ok, so I have split target PRE and POST cleanup. They can be now individually
> enabled or disabled through the corresponding entries in your testplan
> (check
> testplan_default.json on my next branch).
That's great stuff. Thanks very much. I was resorting to my own kludges to preserve
test materials on target after a test. Having a simple flag to control this is much better.
>
> I also enabled support for the "Rebuild" flag, and the "Reboot" flag.
> However, I noticed that nobody is using the "Reboot" flag!! :O
This is odd. I think a lot of the things that look half-baked are from features
that Cogent maybe had and removed, or were in process of being changed when
they made JTA public. I think we should still support the Reboot flag, and add
the implementation for it. I've been recently working on the serial support, and it
requires some knowledge of when a reboot occurs, to re-sync (ie, get back to the shell prompt)
after a reboot. So it might be a little more complicated than one might first think.
> I was thinking to add it to pre_tests, but I'm not sure how well LAVA would
> integrate with such changes. In the end, I decided to leave it for future work.
Sounds good. This is one unimplemented thing I think it worth retaining for
future implementation.
----
FYI - I have requested a dedicated BOF session at ELC for the "Status and Roadmap of Fuego".
I'll announce the date, time and room when I hear back.
I'd like to discuss the status of our different trees, and work out a plan for merging
stuff at ELC. I'd like to get the merge done as soon after ELC as possible, to prevent further diverging.
Jan-Simon will be at ELC as well. I believe the three of us are key drivers of Fuego
at this point, and if we can hammer out a plan for moving forward, that would be great.
Anyone else on this list going to ELC? If so, please bring your Fuego feedback and ideas
to the BOF.
^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [Fuego] Fuego's version up and other changes
2017-02-14 19:55 ` Bird, Timothy
@ 2017-02-15 18:39 ` Kevin Hilman
0 siblings, 0 replies; 43+ messages in thread
From: Kevin Hilman @ 2017-02-15 18:39 UTC (permalink / raw)
To: Bird, Timothy; +Cc: fuego
"Bird, Timothy" <Tim.Bird@sony.com> writes:
[...]
> FYI - I have requested a dedicated BOF session at ELC for the "Status and Roadmap of Fuego".
> I'll announce the date, time and room when I hear back.
>
> I'd like to discuss the status of our different trees, and work out a plan for merging
> stuff at ELC. I'd like to get the merge done as soon after ELC as possible, to prevent further diverging.
> Jan-Simon will be at ELC as well. I believe the three of us are key drivers of Fuego
> at this point, and if we can hammer out a plan for moving forward, that would be great.
>
> Anyone else on this list going to ELC? If so, please bring your Fuego feedback and ideas
> to the BOF.
Unforunately, I won't be at ELC (first one I've missed in many years) :(
My feature request: Fuego create & submit LAVA v2 jobs and post-process results.
Kevin
^ permalink raw reply [flat|nested] 43+ messages in thread
end of thread, other threads:[~2017-02-15 18:39 UTC | newest]
Thread overview: 43+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-01-11 0:28 [Fuego] RFC: Fuego's version up and other changes Daniel Sangorrin
2017-01-12 5:51 ` Ibe.Kengo
2017-01-12 6:44 ` Daniel Sangorrin
2017-01-12 8:15 ` Ibe.Kengo
2017-01-12 8:45 ` Daniel Sangorrin
2017-01-12 9:04 ` Ibe.Kengo
2017-01-12 9:18 ` Daniel Sangorrin
2017-01-13 1:02 ` Bird, Timothy
2017-01-13 1:29 ` Daniel Sangorrin
2017-01-13 1:37 ` Bird, Timothy
2017-01-13 1:45 ` Daniel Sangorrin
2017-01-13 1:57 ` Bird, Timothy
2017-01-13 2:05 ` Daniel Sangorrin
2017-01-13 3:34 ` Bird, Timothy
2017-01-13 4:36 ` Daniel Sangorrin
2017-01-20 0:53 ` [Fuego] " Bird, Timothy
2017-01-20 7:09 ` Daniel Sangorrin
2017-01-28 2:14 ` Bird, Timothy
2017-01-31 1:56 ` Daniel Sangorrin
2017-01-31 20:54 ` Jan-Simon Möller
2017-02-02 1:25 ` Bird, Timothy
2017-02-02 5:49 ` Daniel Sangorrin
2017-02-02 9:50 ` Jan-Simon Möller
2017-01-31 23:15 ` Jan-Simon Möller
2017-02-01 0:56 ` Daniel Sangorrin
2017-02-01 10:46 ` Jan-Simon Möller
2017-02-02 1:57 ` Bird, Timothy
2017-02-02 6:06 ` Daniel Sangorrin
2017-02-03 1:04 ` Daniel Sangorrin
2017-02-13 3:51 ` Daniel Sangorrin
2017-02-14 19:55 ` Bird, Timothy
2017-02-15 18:39 ` Kevin Hilman
2017-02-02 9:57 ` Jan-Simon Möller
2017-02-02 5:45 ` Daniel Sangorrin
2017-02-02 1:35 ` Bird, Timothy
2017-02-02 5:52 ` Daniel Sangorrin
2017-02-02 10:04 ` Jan-Simon Möller
2017-02-02 20:51 ` Kevin Hilman
2017-02-02 22:16 ` Bird, Timothy
2017-02-02 23:37 ` Kevin Hilman
2017-02-04 0:07 ` Jan-Simon Möller
2017-02-07 1:56 ` Daniel Sangorrin
2017-02-02 1:17 ` Bird, Timothy
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.