Question on toaster bringup

Question on toaster bringup

[Olaf Mandel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hello,

I want to initialize toaster from fixtures alone, if possible. It should
only provide a "local" release (no network dependency at all). Here I
ran into two to three problems:

* the oe-core.xml fixture adds three releases. Is it possible to
remove/delete objects in the custom.xml fixture that I do not want?
Currently I am working around that by just configuring out the line
call_command("loaddata", "oe-core") in checksettings.py.

* the custom.xml fixture needs to reside in the
bitbake/lib/toaster/orm/fixtures directory to be found. How can I search
at other places? Setting an environment variable FIXTURE_DIRS didn't
seem to help.

* minor point: would it make sense to detect if
call_command("lsupdates") is needed? One does not need that for a purely
local release and I have it commented out.

Thank you for your time,
Olaf
- -- 
Olaf Mandel

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJZprHhAAoJEFS9nbRiUh3/YeEIAIWvZIb0gECYO+mTrcZu+7w7
Zmtg2ZcYM2BWwthzJl4aA48vVjwYOpY74oQnByorEJh6yjSob/GqAFgPKbNGVF+a
zH1BCkzao7oAdYtY5naYfYyaD5iDvStl/pfxCoxujh7wC9k7o11gBXO2HIp4U/Ww
8kiNlDpjI2dOciuanYB9hluOtwJ+pBJsuFhktTRKn3f7hSkPq1Q/C+L8dJEBc9sl
JyQ5yNdEhXhrvCPT1wyJtxnHiAXIK/nQrXb0uEFzLv1ENxgCiGVxY6GsHm7KSXf/
I53I+m5kWiBozgo5bXPfj6W0se/v7EsU30kOCilUCAf6bl65LS2EyqVndTJCHZ4=
=GW/i
-----END PGP SIGNATURE-----

Re: Question on toaster bringup

[Reyna, David]

Hi Olaf,

Thank you for using Toaster! There is a Toaster team and we have a meeting each week, for which I am the lead and the Toaster maintainer. I will forward you the link (it is in about 10 minutes :-)

1) See 11957 for a resolution for your question. The fixtures are strictly additive, so 11957 provides the way to disable the default fixtures.

2) Yes, create and enhancement request and we can address that. It is too late for enhancements for YP-2.4 but we can look at that for 2.5.

3) Ideally you would provide your own Layer Index server so that you can get the full power of the interface. We can certainly look at alternatives for your reduced use case.

- David Reyna

-----Original Message-----
From: bitbake-devel-bounces@lists.openembedded.org [mailto:bitbake-devel-bounces@lists.openembedded.org] On Behalf Of Olaf Mandel
Sent: Wednesday, August 30, 2017 5:39 AM
To: bitbake-devel@lists.openembedded.org
Subject: [bitbake-devel] Question on toaster bringup

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hello,

I want to initialize toaster from fixtures alone, if possible. It should
only provide a "local" release (no network dependency at all). Here I
ran into two to three problems:

* the oe-core.xml fixture adds three releases. Is it possible to
remove/delete objects in the custom.xml fixture that I do not want?
Currently I am working around that by just configuring out the line
call_command("loaddata", "oe-core") in checksettings.py.

* the custom.xml fixture needs to reside in the
bitbake/lib/toaster/orm/fixtures directory to be found. How can I search
at other places? Setting an environment variable FIXTURE_DIRS didn't
seem to help.

* minor point: would it make sense to detect if
call_command("lsupdates") is needed? One does not need that for a purely
local release and I have it commented out.

Thank you for your time,
Olaf
- -- 
Olaf Mandel

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJZprHhAAoJEFS9nbRiUh3/YeEIAIWvZIb0gECYO+mTrcZu+7w7
Zmtg2ZcYM2BWwthzJl4aA48vVjwYOpY74oQnByorEJh6yjSob/GqAFgPKbNGVF+a
zH1BCkzao7oAdYtY5naYfYyaD5iDvStl/pfxCoxujh7wC9k7o11gBXO2HIp4U/Ww
8kiNlDpjI2dOciuanYB9hluOtwJ+pBJsuFhktTRKn3f7hSkPq1Q/C+L8dJEBc9sl
JyQ5yNdEhXhrvCPT1wyJtxnHiAXIK/nQrXb0uEFzLv1ENxgCiGVxY6GsHm7KSXf/
I53I+m5kWiBozgo5bXPfj6W0se/v7EsU30kOCilUCAf6bl65LS2EyqVndTJCHZ4=
=GW/i
-----END PGP SIGNATURE-----
-- 
_______________________________________________
bitbake-devel mailing list
bitbake-devel@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/bitbake-devel

Re: [bitbake-devel] Question on toaster bringup

[Olaf Mandel]

[-- Attachment #1.1: Type: text/plain, Size: 2972 bytes --]

Hi,

Am 30.08.2017 um 16:51 schrieb Reyna, David:
> [Am 30.08.2017 um 14:38 schrieb Olaf Mandel:]
>> I want to initialize toaster from fixtures alone, if possible. It should
>> only provide a "local" release (no network dependency at all). Here I
>> ran into two to three problems:
>> 
> Thank you for using Toaster! There is a Toaster team [...]

I obviously posted to the wrong forum, sorry: this should be the last
email there.

> [...] and we have a meeting each week, for which I am the lead and the Toaster maintainer. I will forward you the link (it is in about 10 minutes :-)

Thanks, but I missed it. Probably would not have had much to contribute,
anyway.

>> * the oe-core.xml fixture adds three releases. Is it possible to
>> remove/delete objects in the custom.xml fixture that I do not want?
>> Currently I am working around that by just configuring out the line
>> call_command("loaddata", "oe-core") in checksettings.py.
>> 
> 1) See 11957 for a resolution for your question. The fixtures are strictly additive, so 11957 provides the way to disable the default fixtures.

Thank you. Just for anyone else who got as confused as me: the commit-id
got rewritten from 254e2debe5705f4fe34307378a8c1457e4327d1c to
ac29d4a9078494544a2627d8b6b021096b49cb34 (at the moment).

One question for that: where does ToasterSetting get an object with
name=='CUSTOM_XML_ONLY' from? Does this require editing the settings.xml
file?

>> * the custom.xml fixture needs to reside in the
>> bitbake/lib/toaster/orm/fixtures directory to be found. How can I search
>> at other places? Setting an environment variable FIXTURE_DIRS didn't
>> seem to help.
>> 
> 2) Yes, create and enhancement request and we can address that. It is too late for enhancements for YP-2.4 but we can look at that for 2.5.
> 
Request number is #12007. Thank you for considering it.

>> * minor point: would it make sense to detect if
>> call_command("lsupdates") is needed? One does not need that for a purely
>> local release and I have it commented out.
>> 
> 3) Ideally you would provide your own Layer Index server so that you can get the full power of the interface. We can certainly look at alternatives for your reduced use case.
> 
I guess I could. But even without it, I can still build. The only
inconveniences I see are:

* machine is not selectable / detectable
* recipes / images are only known after they were build once
* because of above, need to put the image recipes into the custom.xml
fixture to allow building them
* links to recipe files are dead

I can live with all of the above, so I will probably not set up a local
layer index.

Thank you,
Olaf
-- 
Olaf Mandel
phone: +49-89-189166-185
fax:   +49-89-189166-111
Menlo Systems GmbH
Am Klopferspitz 19a, D-82152 Martinsried
Amtsgericht München HRB 138145
Geschäftsführung: Dr Michael Mei, Dr Ronald Holzwarth
USt-IdNr. DE217772017, St.-Nr. 14316170324


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 484 bytes --]

Re: Question on toaster bringup

[Olaf Mandel]

[-- Attachment #1.1: Type: text/plain, Size: 2972 bytes --]

Hi,

Am 30.08.2017 um 16:51 schrieb Reyna, David:
> [Am 30.08.2017 um 14:38 schrieb Olaf Mandel:]
>> I want to initialize toaster from fixtures alone, if possible. It should
>> only provide a "local" release (no network dependency at all). Here I
>> ran into two to three problems:
>> 
> Thank you for using Toaster! There is a Toaster team [...]

I obviously posted to the wrong forum, sorry: this should be the last
email there.

> [...] and we have a meeting each week, for which I am the lead and the Toaster maintainer. I will forward you the link (it is in about 10 minutes :-)

Thanks, but I missed it. Probably would not have had much to contribute,
anyway.

>> * the oe-core.xml fixture adds three releases. Is it possible to
>> remove/delete objects in the custom.xml fixture that I do not want?
>> Currently I am working around that by just configuring out the line
>> call_command("loaddata", "oe-core") in checksettings.py.
>> 
> 1) See 11957 for a resolution for your question. The fixtures are strictly additive, so 11957 provides the way to disable the default fixtures.

Thank you. Just for anyone else who got as confused as me: the commit-id
got rewritten from 254e2debe5705f4fe34307378a8c1457e4327d1c to
ac29d4a9078494544a2627d8b6b021096b49cb34 (at the moment).

One question for that: where does ToasterSetting get an object with
name=='CUSTOM_XML_ONLY' from? Does this require editing the settings.xml
file?

>> * the custom.xml fixture needs to reside in the
>> bitbake/lib/toaster/orm/fixtures directory to be found. How can I search
>> at other places? Setting an environment variable FIXTURE_DIRS didn't
>> seem to help.
>> 
> 2) Yes, create and enhancement request and we can address that. It is too late for enhancements for YP-2.4 but we can look at that for 2.5.
> 
Request number is #12007. Thank you for considering it.

>> * minor point: would it make sense to detect if
>> call_command("lsupdates") is needed? One does not need that for a purely
>> local release and I have it commented out.
>> 
> 3) Ideally you would provide your own Layer Index server so that you can get the full power of the interface. We can certainly look at alternatives for your reduced use case.
> 
I guess I could. But even without it, I can still build. The only
inconveniences I see are:

* machine is not selectable / detectable
* recipes / images are only known after they were build once
* because of above, need to put the image recipes into the custom.xml
fixture to allow building them
* links to recipe files are dead

I can live with all of the above, so I will probably not set up a local
layer index.

Thank you,
Olaf
-- 
Olaf Mandel
phone: +49-89-189166-185
fax:   +49-89-189166-111
Menlo Systems GmbH
Am Klopferspitz 19a, D-82152 Martinsried
Amtsgericht München HRB 138145
Geschäftsführung: Dr Michael Mei, Dr Ronald Holzwarth
USt-IdNr. DE217772017, St.-Nr. 14316170324


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 484 bytes --]

Re: [bitbake-devel] Question on toaster bringup

[Reyna, David]

Hi Olaf,

> I obviously posted to the wrong forum,

No problem! We appreciate your patches.

FYI, there is a wiki page that describes our process for creating and sharing Toaster patches.
   https://wiki.yoctoproject.org/wiki/Contribute_to_Toaster#Submitting_patch_sets_for_integration_upstream

We follow this process to (a) share the information with our community and provide review and insights, and (b) help the bitbake maintainers process our patches so that they can be confidently and quickly pushed.

> Probably would not have had much to contribute, anyway.

You are welcome to join anytime you have suggestions of questions, and especially if you have more patches that you wish to share. I have invited you to the toaster mailing list in case you are not already a member.

> One question for that: where does ToasterSetting get an object with
> name=='CUSTOM_XML_ONLY' from? Does this require editing the settings.xml
> file?

Good question. You would normally define that in "settings.xml" because that is always read first. I put that information into the patch header but have not yet added it to the official Toaster documentation (which is a work in progress).

In thinking about your suggestions, I think that allowing this value to be defined in the environment as well would be a good idea, allow people to leave "setting.xml" un-edited.

* machine is not selectable / detectable

You can actually define machines in custom.xml if want to provide a selection. 

You can in fact define anything you want pre-populated in the database that is normally done by "lsupdate.py".

* links to recipe files are dead

You can define them in the custom.xml as well. I plan to do this to help resolve the related issue that I raised in Bug 11956 for the pre-populated layers in the various default fixture files.

- David



-----Original Message-----
From: Olaf Mandel [mailto:o.mandel@menlosystems.com] 
Sent: Wednesday, August 30, 2017 9:09 AM
To: Reyna, David; toaster@yoctoproject.org
Subject: Re: [bitbake-devel] Question on toaster bringup

Hi,

Am 30.08.2017 um 16:51 schrieb Reyna, David:
> [Am 30.08.2017 um 14:38 schrieb Olaf Mandel:]
>> I want to initialize toaster from fixtures alone, if possible. It should
>> only provide a "local" release (no network dependency at all). Here I
>> ran into two to three problems:
>> 
> Thank you for using Toaster! There is a Toaster team [...]

I obviously posted to the wrong forum, sorry: this should be the last
email there.

> [...] and we have a meeting each week, for which I am the lead and the Toaster maintainer. I will forward you the link (it is in about 10 minutes :-)

Thanks, but I missed it. Probably would not have had much to contribute,
anyway.

>> * the oe-core.xml fixture adds three releases. Is it possible to
>> remove/delete objects in the custom.xml fixture that I do not want?
>> Currently I am working around that by just configuring out the line
>> call_command("loaddata", "oe-core") in checksettings.py.
>> 
> 1) See 11957 for a resolution for your question. The fixtures are strictly additive, so 11957 provides the way to disable the default fixtures.

Thank you. Just for anyone else who got as confused as me: the commit-id
got rewritten from 254e2debe5705f4fe34307378a8c1457e4327d1c to
ac29d4a9078494544a2627d8b6b021096b49cb34 (at the moment).

One question for that: where does ToasterSetting get an object with
name=='CUSTOM_XML_ONLY' from? Does this require editing the settings.xml
file?

>> * the custom.xml fixture needs to reside in the
>> bitbake/lib/toaster/orm/fixtures directory to be found. How can I search
>> at other places? Setting an environment variable FIXTURE_DIRS didn't
>> seem to help.
>> 
> 2) Yes, create and enhancement request and we can address that. It is too late for enhancements for YP-2.4 but we can look at that for 2.5.
> 
Request number is #12007. Thank you for considering it.

>> * minor point: would it make sense to detect if
>> call_command("lsupdates") is needed? One does not need that for a purely
>> local release and I have it commented out.
>> 
> 3) Ideally you would provide your own Layer Index server so that you can get the full power of the interface. We can certainly look at alternatives for your reduced use case.
> 
I guess I could. But even without it, I can still build. The only
inconveniences I see are:

* machine is not selectable / detectable
* recipes / images are only known after they were build once
* because of above, need to put the image recipes into the custom.xml
fixture to allow building them
* links to recipe files are dead

I can live with all of the above, so I will probably not set up a local
layer index.

Thank you,
Olaf
-- 
Olaf Mandel
phone: +49-89-189166-185
fax:   +49-89-189166-111
Menlo Systems GmbH
Am Klopferspitz 19a, D-82152 Martinsried
Amtsgericht München HRB 138145
Geschäftsführung: Dr Michael Mei, Dr Ronald Holzwarth
USt-IdNr. DE217772017, St.-Nr. 14316170324

Re: [bitbake-devel] Question on toaster bringup

[Olaf Mandel]

[-- Attachment #1.1: Type: text/plain, Size: 1730 bytes --]

Hello,

Am 31.08.2017 um 04:30 schrieb Reyna, David:
> FYI, there is a wiki page that describes our process for creating and sharing Toaster patches.
>    https://wiki.yoctoproject.org/wiki/Contribute_to_Toaster#Submitting_patch_sets_for_integration_upstream
> 
oh. Will follow it next time, as far as possible. I will have to skip
running the tests, though:

The tests don't run for me (unmodified poky master): locally, I get
"EEEEEEEEEEEEEEEEEEEE.EEEEEEEEEEEEEEEEEEEEEEEEEEEEE". This seems to
mostly be "Unable to load Atom 'find_element' from file
':/ghostdriver/./third_party/webdriver-atoms/find_element.js'" or the
same with "'find_elements'".

When I try to run the tests using the
selenium/standalone-firefox-debug:2.53.0 container, it does not get
better. I get "Exception: The path to the driver executable must be set
by the webdriver.gecko.driver system property" errors in the container
and the tests fail as "EEEEEEEEEEEEEEEEEE" with first the same error as
above and then lots of "Address already in use".

>> One question for that: where does ToasterSetting get an object with
>> name=='CUSTOM_XML_ONLY' from? Does this require editing the settings.xml
>> file?
-Snipp-
> In thinking about your suggestions, I think that allowing this value to be defined in the environment as well would be a good idea, allow people to leave "setting.xml" un-edited.
> 
Yes, please: that would strongly simplify things. In fact,
CUSTOM_XML_ONLY is only of limited use if it could only be set by
editing the settings.xml file: if you have to editing settings.xml, then
you may as well edit oe-core.xml and put the contents of your custom.xml
in there.

Best regards,
Olaf
-- 
Olaf Mandel



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 484 bytes --]