Second design proposal - building non-Git layers

Second design proposal - building non-Git layers

[Barros Pena, Belen]

I spent last week gathering feedback from my first design proposal about
how to build non-Git layers with Toaster. For some background, the first
design proposal is here

https://lists.yoctoproject.org/pipermail/toaster/2016-June/004855.html

The main point from the feedback was that the design was unnecessarily
complicated: it provided options that were unlikely to be widely used. It
also allowed users to change the data coming from the layer index, and
that made some people a bit uncomfortable: they thought that data
constitutes a baseline and a safe point of return for users, so it should
be kept "sane".

I agree with both points, so I set to work on a version 2 that addressed
them. A new explanatory video is available at

https://youtu.be/z5wVjBwJDzY

We will be discussing both design proposals in the upcoming design review
call, which is open to everybody. This is your chance to get your hands
dirty with design stuff, if you are so inclined. These are the call
details:

Date: Thursday, July 7th
Time: 4:00 PM BST (8:00 AM PST, 8:30 PM IST)
Tel: 1-888-875-9370/916-356-2663

Bridge: 4
Passcode: 1434913

If you are coming, please make sure to watch both videos beforehand (they
amount to about 15 minutes, so I hope it's not too much to ask).

Video 1: https://youtu.be/N6gvTtZUP3Y
Video 2: https://youtu.be/z5wVjBwJDzY

See you there.

Belén

 

Re: Second design proposal - building non-Git layers

[Reyna, David]

Hi Belen,

We are very excited about this feature. It is something we have wanted for a long time.

Here are my questions.

1) In your example, you are using a local path "home/users/mklayers/meta-path" that does not start with a "/". Can I assume that it is just missing that "/" prefix, and that all local layers must be full absolute paths to a local or NFS directory?

2) When you switch a layer from a local path to a git path (or the other way), does Toaster remember the other values so that you can switch back and forth without reentering all the data? 

That would be handy if you are testing a local development layer versus the formal git layer and are switching back and forth, plus that hidden persistent effectively provides the feature from your previous version without the visual overhead that this second version is avoiding.

3) The layers that you cannot change (but can provide your own version), are these all the layers defined in the official Layer Index, or just those designated in the JSON file default layer list? I presume the former.

4) If you do add a copy of say "openembedded-core" and forget to remove the original one, how does Toaster react? Just curious.

5) I was a little confused on the "Layer Revision" pulldown that you show at the end. 

I think that you were saying that normally you would use layers from the Layer Index, and would normally have them all use the same global branch, and that the usual selections would be either "master" or the most recent stable branch. 

Here are my questions:
  * What appears when you only checkbox "Other"? Do all possible layers and branches appear, or only those layers that have a selected revision value that is not "master" nor the latest stable branch name?

  * I would think that a local layer would always use a locally named branch that has its own revision naming model, and (except for say "master") would never fit the normal global pattern. Are these then only listed in the "Other" category? If you uncheck "Other", do these local layers effectively disappear it you select the latest YP branch because these local layers do not have that specific named branch?

6) Remind me, does Toaster support git paths to local repos (i.e. "file:///opt/git/project.git")? In other words, if you have your git layers local to your machine (and not in the network), you can use the "file://" protocol and everything is hunky and dory?

7) What about batches of local layers? 

Thanks,
David


-----Original Message-----
From: toaster-bounces@yoctoproject.org [mailto:toaster-bounces@yoctoproject.org] On Behalf Of Barros Pena, Belen
Sent: Wednesday, July 06, 2016 4:48 AM
To: toaster@yoctoproject.org
Cc: Tufto, Kathleen
Subject: [Toaster] Second design proposal - building non-Git layers

I spent last week gathering feedback from my first design proposal about how to build non-Git layers with Toaster. For some background, the first design proposal is here

https://lists.yoctoproject.org/pipermail/toaster/2016-June/004855.html

The main point from the feedback was that the design was unnecessarily
complicated: it provided options that were unlikely to be widely used. It also allowed users to change the data coming from the layer index, and that made some people a bit uncomfortable: they thought that data constitutes a baseline and a safe point of return for users, so it should be kept "sane".

I agree with both points, so I set to work on a version 2 that addressed them. A new explanatory video is available at

https://youtu.be/z5wVjBwJDzY

We will be discussing both design proposals in the upcoming design review call, which is open to everybody. This is your chance to get your hands dirty with design stuff, if you are so inclined. These are the call
details:

Date: Thursday, July 7th
Time: 4:00 PM BST (8:00 AM PST, 8:30 PM IST)
Tel: 1-888-875-9370/916-356-2663

Bridge: 4
Passcode: 1434913

If you are coming, please make sure to watch both videos beforehand (they amount to about 15 minutes, so I hope it's not too much to ask).

Video 1: https://youtu.be/N6gvTtZUP3Y
Video 2: https://youtu.be/z5wVjBwJDzY

See you there.

Belén

 

--
_______________________________________________
toaster mailing list
toaster@yoctoproject.org
https://lists.yoctoproject.org/listinfo/toaster

Re: Second design proposal - building non-Git layers

[Barros Pena, Belen]

Hi David, 

Thanks for the questions. Some answers inline.

On 07/07/2016 08:37, "Reyna, David" <david.reyna@windriver.com> wrote:

>Hi Belen,
>
>We are very excited about this feature. It is something we have wanted
>for a long time.

Glad to hear. It looks like everybody is happy we are working on this :)

>
>Here are my questions.
>
>1) In your example, you are using a local path
>"home/users/mklayers/meta-path" that does not start with a "/". Can I
>assume that it is just missing that "/" prefix, and that all local layers
>must be full absolute paths to a local or NFS directory?

Sorry about that: the prototype should have shown the starting "/".

Your question also brings up something we have been discussing about the
design: the fact that, whichever way we implement the feature, we should
make sure it works for both local installations (Toaster running on your
own development machine), and remote installations (Toaster running on a
different machine). We are not sure how to handle this just yet, but we
might need to tar an upload the layer directory or something like that.

>
>2) When you switch a layer from a local path to a git path (or the other
>way), does Toaster remember the other values so that you can switch back
>and forth without reentering all the data?
>
>That would be handy if you are testing a local development layer versus
>the formal git layer and are switching back and forth, plus that hidden
>persistent effectively provides the feature from your previous version
>without the visual overhead that this second version is avoiding.

That would be a nice feature from my perspective. Whether it's possible or
not, and how hard it would be to implement, is a question for the dev
team. 

A couple of people have questioned the utility of changing an imported
layer from pointing to a directory to pointing to a git repository, since
the same could be achieved by importing a second version of the layer. Any
thoughts on that?

>
>3) The layers that you cannot change (but can provide your own version),
>are these all the layers defined in the official Layer Index, or just
>those designated in the JSON file default layer list? I presume the
>former.

It is indeed the former. The idea is to give more visibility to the fact
that you can happily import your own versions of existing layers, while
keeping the original data from the layer index unchanged.

>
>4) If you do add a copy of say "openembedded-core" and forget to remove
>the original one, how does Toaster react? Just curious.

It would be neat if we could tell you: "hey, these 2 layers are the same,
you should remove one", but I didn't include any validation of this kind
in the design because I fret it would fiendishly complicated to do. When
you import a layer, Toaster knows nothing about it, so I am not even sure
what we could use to reliably check if 2 layers are versions of the same
one. If you import your own version of a layer and the version from the
layer index is also added to the project configuration, I suspect the
builds will fail, and that's what Toaster will show you.

As implementation progresses we will start coming across cases like this:
seeing the outcome might suggest some possible solutions.

>
>5) I was a little confused on the "Layer Revision" pulldown that you show
>at the end.
>
>I think that you were saying that normally you would use layers from the
>Layer Index, and would normally have them all use the same global branch,
>and that the usual selections would be either "master" or the most recent
>stable branch.

I am not sure I explained this very well in the video: apologies. I have
received complaints from people about the fact that Toaster only shows you
layers with branches that are compatible with the selected project
release. So, if you create a Toaster project and you choose to use the
"krogoth" release, the list of layers Toaster provides you in that project
will only include layers that, in the layer index, have a krogoth branch.
If the layer you want to use doesn't have a krogoth branch, but it has a
master branch (meta-raspberrypi being currently an example of this), you
will not find meta-raspberrypi anywhere in your krogoth project, and you
might assume it doesn't exist.

The design tries to solve that issue, by defaulting to the layers with the
compatible branch, but giving you an option to see everything that exists
in all branches. I am personally not fully convinced about any of the 2
options I show in that video: we might still come up with something
completely different.

>
>Here are my questions:
>  * What appears when you only checkbox "Other"? Do all possible layers
>and branches appear, or only those layers that have a selected revision
>value that is not "master" nor the latest stable branch name?

My initial thinking was the latter, but as I mentioned above, I am still
not sure this design is the way to go.

>
>  * I would think that a local layer would always use a locally named
>branch that has its own revision naming model, and (except for say
>"master") would never fit the normal global pattern. Are these then only
>listed in the "Other" category?

Well, yes ...

>If you uncheck "Other", do these local layers effectively disappear it
>you select the latest YP branch because these local layers do not have
>that specific named branch?

Yes again ... did I mention I am not very convinced about this design? ;)

>
>6) Remind me, does Toaster support git paths to local repos (i.e.
>"file:///opt/git/project.git")? In other words, if you have your git
>layers local to your machine (and not in the network), you can use the
>"file://" protocol and everything is hunky and dory?

Yes, Toaster supports paths to local git repos. And it works wonderfully,
as long as the layer source is in the same machine that runs Toaster of
course.

>
>7) What about batches of local layers?

Someone else asked the same question. Here is my answer:

https://lists.yoctoproject.org/pipermail/toaster/2016-July/004941.html

So I guess what I am asking is: what is the need behind the request? What
would users be trying to do?

Cheers,

Belén

>
>Thanks,
>David
>
>
>-----Original Message-----
>From: toaster-bounces@yoctoproject.org
>[mailto:toaster-bounces@yoctoproject.org] On Behalf Of Barros Pena, Belen
>Sent: Wednesday, July 06, 2016 4:48 AM
>To: toaster@yoctoproject.org
>Cc: Tufto, Kathleen
>Subject: [Toaster] Second design proposal - building non-Git layers
>
>I spent last week gathering feedback from my first design proposal about
>how to build non-Git layers with Toaster. For some background, the first
>design proposal is here
>
>https://lists.yoctoproject.org/pipermail/toaster/2016-June/004855.html
>
>The main point from the feedback was that the design was unnecessarily
>complicated: it provided options that were unlikely to be widely used. It
>also allowed users to change the data coming from the layer index, and
>that made some people a bit uncomfortable: they thought that data
>constitutes a baseline and a safe point of return for users, so it should
>be kept "sane".
>
>I agree with both points, so I set to work on a version 2 that addressed
>them. A new explanatory video is available at
>
>https://youtu.be/z5wVjBwJDzY
>
>We will be discussing both design proposals in the upcoming design review
>call, which is open to everybody. This is your chance to get your hands
>dirty with design stuff, if you are so inclined. These are the call
>details:
>
>Date: Thursday, July 7th
>Time: 4:00 PM BST (8:00 AM PST, 8:30 PM IST)
>Tel: 1-888-875-9370/916-356-2663
>
>Bridge: 4
>Passcode: 1434913
>
>If you are coming, please make sure to watch both videos beforehand (they
>amount to about 15 minutes, so I hope it's not too much to ask).
>
>Video 1: https://youtu.be/N6gvTtZUP3Y
>Video 2: https://youtu.be/z5wVjBwJDzY
>
>See you there.
>
>Belén
>
>
>
>--
>_______________________________________________
>toaster mailing list
>toaster@yoctoproject.org
>https://lists.yoctoproject.org/listinfo/toaster