Re: Toaster Weekly Meeting Minutes 24/08/16

Re: Toaster Weekly Meeting Minutes 24/08/16

[Barros Pena, Belen]

On 24/08/2016 17:12, "toaster-bounces@yoctoproject.org on behalf of Barros
Pena, Belen" <toaster-bounces@yoctoproject.org on behalf of
belen.barros.pena@intel.com> wrote:

>Belen: open a bug for the meta-raspberrypi images not finishing problem.
>Assign it to Brian

There it goes

https://bugzilla.yoctoproject.org/show_bug.cgi?id=10185

Re: Toaster Weekly Meeting Minutes 24/08/16

[Brian Avery]

[-- Attachment #1: Type: text/plain, Size: 1351 bytes --]

yes, That's from the change we did on how we interpret the webport argument
and whether it binds to all interfaces (old behaviour) or only to localhost
(new behaviour).  It's the same whether we run toaster from w/in a
container or not.
-b

On Fri, Aug 26, 2016 at 5:59 AM, Barros Pena, Belen <
belen.barros.pena@intel.com> wrote:

>
>
> On 25/08/2016 17:26, "Brian Avery" <avery.brian@gmail.com> wrote:
>
> >So gave it another try with
> >
> >docker run -it --rm -p 127.0.0.1:18000:8000 -v
> >/home/<user>/toaster-workdir:/toaster-workdir crops/toaster-master
> >
> >
> >That failed saying the workdir directory didn't exist. So I deleted the
> >/home/<user>/toaster-workdir directory and created /home/<user>/workdir
> >instead. Then the run command as above worked like a charm, and the
> >container came up. I was able to access the container from another
> >computer at
> >
> >name_of_server_running_the_container:18000
>
> Just a small correction. The instructions in GitHub say to run
>
> docker run -it --rm -p 127.0.0.1:18000:8000 -v
> /home/myuser/workdir:/workdir crops/toaster-master
>
>
> But in order to access the container from a different machine I had to run
>
> docker run -it --rm -p 0.0.0.0:18000:8000 -v /home/myuser/workdir:/workdir
> crops/toaster-master
>
>
> Cheers
>
> Belén
>
>

[-- Attachment #2: Type: text/html, Size: 1913 bytes --]

Re: Toaster Weekly Meeting Minutes 24/08/16

[Barros Pena, Belen]

On 25/08/2016 17:26, "Brian Avery" <avery.brian@gmail.com> wrote:

>So gave it another try with
>
>docker run -it --rm -p 127.0.0.1:18000:8000 -v
>/home/<user>/toaster-workdir:/toaster-workdir crops/toaster-master
>
>
>That failed saying the workdir directory didn't exist. So I deleted the
>/home/<user>/toaster-workdir directory and created /home/<user>/workdir
>instead. Then the run command as above worked like a charm, and the
>container came up. I was able to access the container from another
>computer at
>
>name_of_server_running_the_container:18000

Just a small correction. The instructions in GitHub say to run

docker run -it --rm -p 127.0.0.1:18000:8000 -v
/home/myuser/workdir:/workdir crops/toaster-master


But in order to access the container from a different machine I had to run

docker run -it --rm -p 0.0.0.0:18000:8000 -v /home/myuser/workdir:/workdir
crops/toaster-master


Cheers

Belén

Re: Toaster Weekly Meeting Minutes 24/08/16

[Brian Avery]

[-- Attachment #1: Type: text/plain, Size: 9237 bytes --]

Thanks for trying it out and for the feedback :).  I'll try to address the
various points.

Issue1) scary install instructions
  this is a fairly standard way to install packages where they are quickly
moving and you want to be able to easily keep up with the changes. For
example, here's the one for google-chrome
http://www.ubuntuupdates.org/ppa/google_chrome.
Here's an explanation of ppa's and what they are:
http://askubuntu.com/questions/4983/what-are-ppas-and-how-do-i-use-them

The digital ocean approach works, but it will not be something you can
easily do an apt-get update/upgrade with.  I'll go through the docker
install steps here to explain them a little:

I) you need the base packages and repo:
Steps 1-3 are just adding some dependencies for pkg management and
Step 4 is probably the scary one, it's adding the key for the repo so that
the signing can be checked.  This is actully a good thing :).   Here's a
bit of an explanation on repo keys:
http://askubuntu.com/questions/131397/what-is-a-repository-key-under-ubuntu-and-how-do-they-work
Step 7 - basically adds the repo by name for what release (14,15,16...) you
are on.  If you look in etc/apt/sources.list you will see many entries like
it. For example, 'deb http://us.archive.ubuntu.com/ubuntu/ trusty
multiverse'                                                II)  You need
the kernel support for containers:
Basically pull in the required kernel and reboot.
III) Install docker.  Now that everything is set up, this is back to the
standard apt-get install docker-engine. update/upgrade will also work now
:).
IV) Add a docker group.  This makes using the containers easier, so I'd
recommend it.
V) Add dns servers for your company or for google (8.8.8.8) in the docker
startup file.  Fox example in /etc/default/docker I have
DOCKER_OPTS="--dns 10.x.x.y --dns 10.x.x.x"  Note, it can cake a couple.


Issue 2) Running the container
So, a couple of things to address here.
a) The naming convention is
toaster-release - for example crops/toaster-krogoth and
crops/toaster-master.
toaster defaults to the last release version on the assumption that this is
what a naive user might use and we wanted them to get the last release
version since master is a tad cutting edge.
b) running and workdir
So when you run the container you should do something like
$ mkdir ~/AnyDirname
docker run -it --rm -p 127.0.0.1:18000:8000 -v ~/AnyDirname:/workdir
crops/toaster-master
the -v line maps whatever dir you want on your host to /workdir inside the
container. In general, the first part of the -v command will vary as you
use different host side directories.  the second argument will always be
/workdir unless you are using the container in a very advanced way.
---
The master project build issue:
Running master and getting git to work behind a proxy. I'd suggest people
try to use chameleonsocks:
https://hub.docker.com/r/crops/chameleonsocks/
https://github.com/crops/chameleonsocks

A tldr version of installing this is

https_proxy=https:// <https://proxy-chain.intel.com:912/>
<comapnyproxy>:<port> wget
https://raw.githubusercontent.com/crops/chameleonsocks/master/chameleonsocks.sh
&&
chmod 755 chameleonsocks.sh

https_proxy=https:// <https://proxy-chain.intel.com:912/>
<companyproxy>:<port> PROXY=<companyproxy>  PAC_URL=<company wpad
file> ./chameleonsocks.sh --install
The <company wpad file> is often something like
http://wpad.mycompany.com/wpad.dat <http://wpad.intel.com/wpad.dat>
This variable can also take a proxy.pac file instead

Chameleonsocks is a container that encapsulates a project called redsocks
that redirects all traffic through the SOCKS5 proxy.  Once installed it
proxies traffic for both the containers and the host. Note, for speed &
load reasons, it is a good idea to set the http/https proxy in the browser
separately rather than have the browser rely on chameleonsocks.  Note, in
order for it to work it is running in privileged mode and as net=host this
means it looks more like a program/service and is not really containerized
from the point of view of host isolation.
-----
The local project build issue:
This is a bug:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=10163

----

Your takeaways make sense to me.

Your questions:
1) the workdir can be names anything on the host side. so I can have
$ mkdir ~/Duck
 and pass -v ~/Duck:/workdir to the container. This maps ~/Duck on the host
so it looks like /workdir from inside the container.
2) Yes, we need this bug fixed ;)

Hope this helped,
-bavery
an Intel employee

On Thu, Aug 25, 2016 at 7:21 AM, Barros Pena, Belen <
belen.barros.pena@intel.com> wrote:

>
>
> On 24/08/2016 18:03, "toaster-bounces@yoctoproject.org on behalf of Barros
> Pena, Belen" <toaster-bounces@yoctoproject.org on behalf of
> belen.barros.pena@intel.com> wrote:
>
> >
> >
> >On 24/08/2016 17:12, "toaster-bounces@yoctoproject.org on behalf of
> Barros
> >Pena, Belen" <toaster-bounces@yoctoproject.org on behalf of
> >belen.barros.pena@intel.com> wrote:
> >
> >>Brian: send a link to a CROPS instance we can try and the document on how
> >>to run the Toaster Docker container
> >
> >Here come the links (thanks Brian):
> >
> >Toaster Docker containers:
> >https://hub.docker.com/r/crops/toaster-krogoth/
> >https://hub.docker.com/r/crops/toaster-master/
> >
> >Instructions:
> >https://github.com/crops/toaster-container
>
> I gave the container a go this morning, and I wanted to tell you all about
> it ... mainly for toaster manual writing purposes.
>
> First, I had to install Docker in Ubuntu 14.04. The Docker instructions
> looked very scary
>
> https://docs.docker.com/engine/installation/linux/ubuntulinux/
>
> Then someone told me they have a magic script that automates all those
> steps
>
> https://www.digitalocean.com/community/tutorials/how-to-
> install-and-use-doc
> ker-compose-on-ubuntu-14-04
>
> This works really well, but you do need to remember to add your user to
> the docker group before you do anything else
>
> sudo usermod -aG docker $(whoami)
>
>
> Then you have to log out and log back in for the group changes to take
> effect.
>
> Since I am behind a proxy, I had to edit the /etc/default/docker file to
> set http_proxy and https_proxy. I would have never managed this without
> help.
>
> Then I pulled the container I wanted to try using the command provided at
>
> https://hub.docker.com/r/crops/toaster-master/
>
> To run it, I followed the instructions here
>
> https://github.com/crops/toaster-container
>
> I created a workdir as /home/<user>/toaster-workdir
>
> Then I run the run command in the readme file like this
>
> docker run -it --rm -p 127.0.0.1:18000:8000 -v
> /home/<user>/toaster-workdir:/toaster-workdir crops/toaster
>
>
> That made Docker pull a different container. I didn't realise that
> crops/toaster should have been crops/toaster-master in my case :/
>
> So gave it another try with
>
> docker run -it --rm -p 127.0.0.1:18000:8000 -v
> /home/<user>/toaster-workdir:/toaster-workdir crops/toaster-master
>
>
> That failed saying the workdir directory didn't exist. So I deleted the
> /home/<user>/toaster-workdir directory and created /home/<user>/workdir
> instead. Then the run command as above worked like a charm, and the
> container came up. I was able to access the container from another
> computer at
>
> name_of_server_running_the_container:18000
>
>
> I created a project with the master release and started a build. Sadly
> that failed during the cloning stage because I am behind a proxy, and I
> was told I would need to configure the proxy in the container. I have no
> idea how to do that.
>
> Then I thought I should try the local release instead. But once again, the
> build failed, this time with this message:
>
> Existing git repository at /home/usersetup/poky, but with different
> remotes ('origin        http://git.yoctoproject.org/git/poky (fetch),
> origin  http://git.yoctoproject.org/git/poky (push), ', expected
> 'git://git.yoctoproject.org/poky'). Toaster will not continue out of fear
> of damaging something.
>
>
> I stopped at this point, and wrote this email :)
>
> From the experience so far, when writing the content for the manual, it
> might be handy to provide
>
> 1. Some instructions on how to easily install docker
> 2. What to do if you are behind a proxy
>
> I also have a couple of questions:
>
> 1. Does the workdir have to be called "workdir"? Or did I do something
> wrong?
> 2. Do we need to fix the issue with the local project builds?
>
> Cheers
>
> Belén
>
> >
> >Instructions for running tests (in progress):
> >https://wiki.yoctoproject.org/wiki/TipsAndTricks/
> TestingToasterWithContain
> >e
> >rs
> >
> >
> >--
> >_______________________________________________
> >toaster mailing list
> >toaster@yoctoproject.org
> >https://lists.yoctoproject.org/listinfo/toaster
>
> --
> _______________________________________________
> toaster mailing list
> toaster@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster
>

[-- Attachment #2: Type: text/html, Size: 13990 bytes --]

Re: Toaster Weekly Meeting Minutes 24/08/16

[Barros Pena, Belen]

On 24/08/2016 18:03, "toaster-bounces@yoctoproject.org on behalf of Barros
Pena, Belen" <toaster-bounces@yoctoproject.org on behalf of
belen.barros.pena@intel.com> wrote:

>
>
>On 24/08/2016 17:12, "toaster-bounces@yoctoproject.org on behalf of Barros
>Pena, Belen" <toaster-bounces@yoctoproject.org on behalf of
>belen.barros.pena@intel.com> wrote:
>
>>Brian: send a link to a CROPS instance we can try and the document on how
>>to run the Toaster Docker container
>
>Here come the links (thanks Brian):
>
>Toaster Docker containers:
>https://hub.docker.com/r/crops/toaster-krogoth/
>https://hub.docker.com/r/crops/toaster-master/
>
>Instructions:
>https://github.com/crops/toaster-container

I gave the container a go this morning, and I wanted to tell you all about
it ... mainly for toaster manual writing purposes.

First, I had to install Docker in Ubuntu 14.04. The Docker instructions
looked very scary

https://docs.docker.com/engine/installation/linux/ubuntulinux/

Then someone told me they have a magic script that automates all those
steps

https://www.digitalocean.com/community/tutorials/how-to-install-and-use-doc
ker-compose-on-ubuntu-14-04

This works really well, but you do need to remember to add your user to
the docker group before you do anything else

sudo usermod -aG docker $(whoami)


Then you have to log out and log back in for the group changes to take
effect. 

Since I am behind a proxy, I had to edit the /etc/default/docker file to
set http_proxy and https_proxy. I would have never managed this without
help. 

Then I pulled the container I wanted to try using the command provided at

https://hub.docker.com/r/crops/toaster-master/

To run it, I followed the instructions here

https://github.com/crops/toaster-container

I created a workdir as /home/<user>/toaster-workdir

Then I run the run command in the readme file like this

docker run -it --rm -p 127.0.0.1:18000:8000 -v
/home/<user>/toaster-workdir:/toaster-workdir crops/toaster


That made Docker pull a different container. I didn't realise that
crops/toaster should have been crops/toaster-master in my case :/

So gave it another try with

docker run -it --rm -p 127.0.0.1:18000:8000 -v
/home/<user>/toaster-workdir:/toaster-workdir crops/toaster-master


That failed saying the workdir directory didn't exist. So I deleted the
/home/<user>/toaster-workdir directory and created /home/<user>/workdir
instead. Then the run command as above worked like a charm, and the
container came up. I was able to access the container from another
computer at

name_of_server_running_the_container:18000


I created a project with the master release and started a build. Sadly
that failed during the cloning stage because I am behind a proxy, and I
was told I would need to configure the proxy in the container. I have no
idea how to do that.

Then I thought I should try the local release instead. But once again, the
build failed, this time with this message:

Existing git repository at /home/usersetup/poky, but with different
remotes ('origin	http://git.yoctoproject.org/git/poky (fetch),
origin	http://git.yoctoproject.org/git/poky (push), ', expected
'git://git.yoctoproject.org/poky'). Toaster will not continue out of fear
of damaging something.


I stopped at this point, and wrote this email :)

From the experience so far, when writing the content for the manual, it
might be handy to provide

1. Some instructions on how to easily install docker
2. What to do if you are behind a proxy

I also have a couple of questions:

1. Does the workdir have to be called "workdir"? Or did I do something
wrong?
2. Do we need to fix the issue with the local project builds?

Cheers

Belén

>
>Instructions for running tests (in progress):
>https://wiki.yoctoproject.org/wiki/TipsAndTricks/TestingToasterWithContain
>e
>rs
>
>
>-- 
>_______________________________________________
>toaster mailing list
>toaster@yoctoproject.org
>https://lists.yoctoproject.org/listinfo/toaster

Re: Toaster Weekly Meeting Minutes 24/08/16

[Barros Pena, Belen]

On 25/08/2016 02:22, "Paul Eggleton" <paul.eggleton@linux.intel.com> wrote:

>>
>>This also means we need someone else as the default
>>assignee for the layer index bugs. Brian will talk to Paul Eggleton to
>>see
>>what he recommends.
>
>I can be the default assignee again, I may need some help from someone
>else 
>(perhaps Michael?) on urgent issues though.

Thanks Paul. 

Cheers

Belén

Re: Toaster Weekly Meeting Minutes 24/08/16

[Paul Eggleton]

On Wed, 24 Aug 2016 16:12:18 Barros Pena, Belen wrote:
> Belen: Elliot is leaving the team, which is really sad because he is an
> excellent developer, a great contributor, and a really nice guy. Thanks
> Elliot for all your work over the past couple of releases. You will be
> sorely missed. 

Sad news indeed :( Thanks Elliot for all of your hard work, and all the very 
best.

> This also means we need someone else as the default
> assignee for the layer index bugs. Brian will talk to Paul Eggleton to see
> what he recommends.

I can be the default assignee again, I may need some help from someone else 
(perhaps Michael?) on urgent issues though.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Toaster Weekly Meeting Minutes 24/08/16

[Barros Pena, Belen]

On 24/08/2016 17:12, "toaster-bounces@yoctoproject.org on behalf of Barros
Pena, Belen" <toaster-bounces@yoctoproject.org on behalf of
belen.barros.pena@intel.com> wrote:

>Brian: send a link to a CROPS instance we can try and the document on how
>to run the Toaster Docker container

Here come the links (thanks Brian):

Toaster Docker containers:
https://hub.docker.com/r/crops/toaster-krogoth/
https://hub.docker.com/r/crops/toaster-master/

Instructions:
https://github.com/crops/toaster-container

Instructions for running tests (in progress):
https://wiki.yoctoproject.org/wiki/TipsAndTricks/TestingToasterWithContaine
rs

Toaster Weekly Meeting Minutes 24/08/16

[Barros Pena, Belen]

Attendees: David, Libby, Ed, Brian and Belén

Updates:

David: patch sent for 9936, which gets the 'add' button for layer
dependencies working again. Patch passed UI review: waiting for code
review. Taking 8188 and 8190 about variable history next.

Libby: testing master for 2.2m3

Ed: working on 9869 (artifact events) and on reliably killing toaster
(7973). For the latter, after submitting a series for review we found that
Ubuntu is missing some utilities he is relying on to isolate the running
of build processes. So he will reimplement and resend.

Belén: on holiday last week. Spent this week looking at people's patches.
Submitted a patch for 10179

Brian: working on Randy¹s toaster container to adapt it to running local
tests. When ready will create a tips and tricks document explaining how to
run tests for Toaster with nothing but Docker

Triage bugs:
https://wiki.yoctoproject.org/wiki/Bug_Triage#YP_Toaster_and_Layer_Index_Bu
gs_needing_Triage

Link to the wiki docs:
https://wiki.yoctoproject.org/wiki/Toaster
https://wiki.yoctoproject.org/wiki/Contribute_to_Toaster
https://wiki.yoctoproject.org/wiki/Toaster_and_bitbake_communications

YP 2.2 Planning:
Please capture future items you want at the wiki:
https://wiki.yoctoproject.org/wiki/Toaster_future_release_planning

Toaster Docker container:
The link to Randy's toaster container:
https://hub.docker.com/r/crops/toaster/

AR's: 

Brian: send the ARs from last week to Belén so she can add them to the
notes

Brian: send a link to a CROPS instance we can try and the document on how
to run the Toaster Docker container

Brian: talk to Paul Eggleton about the new default assignee for the layer
index bugs

Brian: build one of the images in meta-raspberrypi and dump the events to
check if any missing ones might be causing the builds not finishing
problem. 

Belen: open a bug for the meta-raspberrypi images not finishing problem.
Assign it to Brian

Ed: submit the patch for 10170

Opens:

David: question about starting up Toaster specifying an ip address and
webport. It has been changed to the Django standard way, so to access a
remote Toaster instance you need to pass webport=0.0.0.0:port_number

Belén: meta-raspberrypi. Image builds still show as never completing in
the Toaster UI. Brian looked at the events and there is one that we are
not using named cooker.cookerexit. When the cooker exists, it emits this
event always, independently of build outcome and the metadata. Since this
event cannot be masked by the metadata, we should probably be tying the
Toaster end of builds to this. We have added a couple of ARs to deal with
the issue (see the list of ARs above)

Belén: question about Docker containers and manual. We'll recommend the
docker container as the way to set up toaster in the manual. We will also
include the other way as a secondary way. The container is using sqlite.
Brian is creating the mysql one on top of that.

Belen: Elliot is leaving the team, which is really sad because he is an
excellent developer, a great contributor, and a really nice guy. Thanks
Elliot for all your work over the past couple of releases. You will be
sorely missed. This also means we need someone else as the default
assignee for the layer index bugs. Brian will talk to Paul Eggleton to see
what he recommends.