Buildbot / Autobuilder / custom?

Buildbot / Autobuilder / custom?

[Alain Achkar]

[-- Attachment #1: Type: text/plain, Size: 884 bytes --]

Hello Joshua!

We are using Yocto to build for the Variscite DART-6UL SoM which is based
on Freescale/NXP i.MX6UL (
http://variwiki.com/index.php?title=DART-6UL_Yocto_Jethro_R1_build).

Currently, developers run builds manually on their desktops. We are
considering using a continuous integration server. I found out that The
Yocto Project uses Autobuilder which in turn uses BuildBot. I read most of
the introductory documentation.

Would you recommend that we install only BuildBot and configure it to build
what we need?

OR

Would you recommend that we install Autobuilder (and BuildBot) and
configure it to build what we need?

What is the faster path to get up and running?

Thanks!
Alain.

-- 

Alain Achkar, M.Eng. | Software Engineer

*Triacta Power Solutions*

(O) 613.256.2868 (C) 613.406.6520

*www.triacta.com  <http://www.triacta.com/>*

[-- Attachment #2: Type: text/html, Size: 2173 bytes --]

Re: Buildbot / Autobuilder / custom?

[Beth 'pidge' Flanagan]

On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
> Hello Joshua!
> 
> We are using Yocto to build for the Variscite DART-6UL SoM which is
> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.php?title=D
> ART-6UL_Yocto_Jethro_R1_build).
> 
> Currently, developers run builds manually on their desktops. We are
> considering using a continuous integration server. I found out that
> The Yocto Project uses Autobuilder which in turn uses BuildBot. I
> read most of the introductory documentation.
> 
> Would you recommend that we install only BuildBot and configure it to
> build what we need?
> 
> OR
> 
> Would you recommend that we install Autobuilder (and BuildBot) and
> configure it to build what we need?

Actually, the yocto-autobuilder already contains buildbot so no need
for both.

If you read:

http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/README-QUIC
KSTART

and probably some of the other READMEs and the documentation in the
docs directory of the yocto-autobuilder directory, it should get you
started in the right direction.

Feel free to ask questions if you need help.

-b

Elizabeth 'pidge' Flanagan
Founder/CTO Togán Labs
www.toganlabs.com

> 
> What is the faster path to get up and running?
> 
> Thanks!
> Alain.
> 
> -- 
> Alain Achkar, M.Eng. | Software Engineer
> Triacta Power Solutions
> (O) 613.256.2868 (C) 613.406.6520
> www.triacta.com  

Re: Buildbot / Autobuilder / custom?

[Bill Randle]

Also, be sure to check the Yocto Project wiki pages:
    https://wiki.yoctoproject.org/wiki/The_Yocto_Autobuilder
in particular, the AB cluster setup and AB maintenance links. Even
though the one link refers to setting up an entire cluster, I've used
that procedure to setup a single autobuilder, as well.

    -Bill

On Fri, Nov 11, 2016 at 10:37 AM, Beth 'pidge' Flanagan
<pidge@toganlabs.com> wrote:
> On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
>> Hello Joshua!
>>
>> We are using Yocto to build for the Variscite DART-6UL SoM which is
>> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.php?title=D
>> ART-6UL_Yocto_Jethro_R1_build).
>>
>> Currently, developers run builds manually on their desktops. We are
>> considering using a continuous integration server. I found out that
>> The Yocto Project uses Autobuilder which in turn uses BuildBot. I
>> read most of the introductory documentation.
>>
>> Would you recommend that we install only BuildBot and configure it to
>> build what we need?
>>
>> OR
>>
>> Would you recommend that we install Autobuilder (and BuildBot) and
>> configure it to build what we need?
>
> Actually, the yocto-autobuilder already contains buildbot so no need
> for both.
>
> If you read:
>
> http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/README-QUIC
> KSTART
>
> and probably some of the other READMEs and the documentation in the
> docs directory of the yocto-autobuilder directory, it should get you
> started in the right direction.
>
> Feel free to ask questions if you need help.
>
> -b
>
> Elizabeth 'pidge' Flanagan
> Founder/CTO Togán Labs
> www.toganlabs.com
>
>>
>> What is the faster path to get up and running?
>>
>> Thanks!
>> Alain.
>>
>> --
>> Alain Achkar, M.Eng. | Software Engineer
>> Triacta Power Solutions
>> (O) 613.256.2868 (C) 613.406.6520
>> www.triacta.com
>
>
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

Re: Buildbot / Autobuilder / custom?

[Alain Achkar]

[-- Attachment #1: Type: text/plain, Size: 3580 bytes --]

Thanks for your answers! From reading these links, it is still not clear to
me if this might be overkill for my requirements. AB Cluster Setup talks
about "*the worker requires 2+ TB to hold all the build temp files and git
repos. If build artifacts and a local sstate mirror are included,
additional worker space is required.*"

Currently, my build only takes 35GB, so I think what these links are
talking about is how to replicate what the Autobuilder project
https://autobuilder.yoctoproject.org/ already does.

To clarify, I am not interested in running builds and tests for everything
that Yocto already builds and tests (i.e. all the processor architectures,
all the machine types, etc.).  I am interested in running one build for one
machine type (the Variscite DART-6UL i.MX6UL arm-based processor, for which
NXP/Freescale and Variscite have provided recipes and layers for).

I know that autobuilder includes BuildBot (this is why I specified it in
parentheses) but my question remains: do I only install BuildBot and try to
build my machine type with it, or do I install Autobuilder?

On Mon, Nov 14, 2016 at 10:10 AM, Bill Randle <bill.randle@gmail.com> wrote:

> Also, be sure to check the Yocto Project wiki pages:
>     https://wiki.yoctoproject.org/wiki/The_Yocto_Autobuilder
> in particular, the AB cluster setup and AB maintenance links. Even
> though the one link refers to setting up an entire cluster, I've used
> that procedure to setup a single autobuilder, as well.
>
>     -Bill
>
> On Fri, Nov 11, 2016 at 10:37 AM, Beth 'pidge' Flanagan
> <pidge@toganlabs.com> wrote:
> > On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
> >> Hello Joshua!
> >>
> >> We are using Yocto to build for the Variscite DART-6UL SoM which is
> >> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.php?title=D
> >> ART-6UL_Yocto_Jethro_R1_build).
> >>
> >> Currently, developers run builds manually on their desktops. We are
> >> considering using a continuous integration server. I found out that
> >> The Yocto Project uses Autobuilder which in turn uses BuildBot. I
> >> read most of the introductory documentation.
> >>
> >> Would you recommend that we install only BuildBot and configure it to
> >> build what we need?
> >>
> >> OR
> >>
> >> Would you recommend that we install Autobuilder (and BuildBot) and
> >> configure it to build what we need?
> >
> > Actually, the yocto-autobuilder already contains buildbot so no need
> > for both.
> >
> > If you read:
> >
> > http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/README-QUIC
> > KSTART
> >
> > and probably some of the other READMEs and the documentation in the
> > docs directory of the yocto-autobuilder directory, it should get you
> > started in the right direction.
> >
> > Feel free to ask questions if you need help.
> >
> > -b
> >
> > Elizabeth 'pidge' Flanagan
> > Founder/CTO Togán Labs
> > www.toganlabs.com
> >
> >>
> >> What is the faster path to get up and running?
> >>
> >> Thanks!
> >> Alain.
> >>
> >> --
> >> Alain Achkar, M.Eng. | Software Engineer
> >> Triacta Power Solutions
> >> (O) 613.256.2868 (C) 613.406.6520
> >> www.triacta.com
> >
> >
> >
> > --
> > _______________________________________________
> > yocto mailing list
> > yocto@yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/yocto
>



-- 

Alain Achkar, M.Eng. | Software Engineer

*Triacta Power Solutions*

(O) 613.256.2868 (C) 613.406.6520

*www.triacta.com  <http://www.triacta.com/>*

[-- Attachment #2: Type: text/html, Size: 6990 bytes --]

Re: Buildbot / Autobuilder / custom?

[Bill Randle]

Yes, the 2 TB is for everything you might do with an AutoBuilder. The
actual space used will be closer to your 35 GB, since you're building
only a single image / work product. People have used BuilBot directly
for automated builds, but using the AutoBuilder code (which runs
BuildBot underneath) gives you added flexibility and capability
without much additional overhead.

    -Bill

On Mon, Nov 14, 2016 at 8:13 AM, Alain Achkar <alain.achkar@triacta.com> wrote:
> Thanks for your answers! From reading these links, it is still not clear to
> me if this might be overkill for my requirements. AB Cluster Setup talks
> about "the worker requires 2+ TB to hold all the build temp files and git
> repos. If build artifacts and a local sstate mirror are included, additional
> worker space is required."
>
> Currently, my build only takes 35GB, so I think what these links are talking
> about is how to replicate what the Autobuilder project
> https://autobuilder.yoctoproject.org/ already does.
>
> To clarify, I am not interested in running builds and tests for everything
> that Yocto already builds and tests (i.e. all the processor architectures,
> all the machine types, etc.).  I am interested in running one build for one
> machine type (the Variscite DART-6UL i.MX6UL arm-based processor, for which
> NXP/Freescale and Variscite have provided recipes and layers for).
>
> I know that autobuilder includes BuildBot (this is why I specified it in
> parentheses) but my question remains: do I only install BuildBot and try to
> build my machine type with it, or do I install Autobuilder?
>
> On Mon, Nov 14, 2016 at 10:10 AM, Bill Randle <bill.randle@gmail.com> wrote:
>>
>> Also, be sure to check the Yocto Project wiki pages:
>>     https://wiki.yoctoproject.org/wiki/The_Yocto_Autobuilder
>> in particular, the AB cluster setup and AB maintenance links. Even
>> though the one link refers to setting up an entire cluster, I've used
>> that procedure to setup a single autobuilder, as well.
>>
>>     -Bill
>>
>> On Fri, Nov 11, 2016 at 10:37 AM, Beth 'pidge' Flanagan
>> <pidge@toganlabs.com> wrote:
>> > On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
>> >> Hello Joshua!
>> >>
>> >> We are using Yocto to build for the Variscite DART-6UL SoM which is
>> >> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.php?title=D
>> >> ART-6UL_Yocto_Jethro_R1_build).
>> >>
>> >> Currently, developers run builds manually on their desktops. We are
>> >> considering using a continuous integration server. I found out that
>> >> The Yocto Project uses Autobuilder which in turn uses BuildBot. I
>> >> read most of the introductory documentation.
>> >>
>> >> Would you recommend that we install only BuildBot and configure it to
>> >> build what we need?
>> >>
>> >> OR
>> >>
>> >> Would you recommend that we install Autobuilder (and BuildBot) and
>> >> configure it to build what we need?
>> >
>> > Actually, the yocto-autobuilder already contains buildbot so no need
>> > for both.
>> >
>> > If you read:
>> >
>> > http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/README-QUIC
>> > KSTART
>> >
>> > and probably some of the other READMEs and the documentation in the
>> > docs directory of the yocto-autobuilder directory, it should get you
>> > started in the right direction.
>> >
>> > Feel free to ask questions if you need help.
>> >
>> > -b
>> >
>> > Elizabeth 'pidge' Flanagan
>> > Founder/CTO Togán Labs
>> > www.toganlabs.com
>> >
>> >>
>> >> What is the faster path to get up and running?
>> >>
>> >> Thanks!
>> >> Alain.
>> >>
>> >> --
>> >> Alain Achkar, M.Eng. | Software Engineer
>> >> Triacta Power Solutions
>> >> (O) 613.256.2868 (C) 613.406.6520
>> >> www.triacta.com
>> >
>> >
>> >
>> > --
>> > _______________________________________________
>> > yocto mailing list
>> > yocto@yoctoproject.org
>> > https://lists.yoctoproject.org/listinfo/yocto
>
>
>
>
> --
>
> Alain Achkar, M.Eng. | Software Engineer
>
> Triacta Power Solutions
>
> (O) 613.256.2868 (C) 613.406.6520
>
> www.triacta.com

Re: Buildbot / Autobuilder / custom?

[Alain Achkar]

[-- Attachment #1: Type: text/plain, Size: 9476 bytes --]

I followed
http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/README-QUICKSTART
and got:

>0< alain@esxi-ub1 Thu Mar 23 05:34 PM [master]
/media/data/yocto-autobuilder >  *. ./yocto-autobuilder-setup*

Creating yocto-controller/buildbot.tac from an example buildbot.tac


Creating yocto-worker/buildbot.tac from an example buildbot.tac


Creating yocto-controller/controller.cfg from an example controller.cfg

#########################################################################
 Setting envvars.
 In the future though please add the following to your shell environment:
 PYTHONPATH=/media/data/yocto-autobuilder/lib/python2.7/site-packages/:/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/:/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/buildsteps:$PYTHONPATH
 PATH=/media/data/yocto-autobuilder/bin:/media/data/yocto-autobuilder/yocto-autobuilder/scripts:$PATH
 YOCTO_AB_CONFIG=/media/data/yocto-autobuilder/buildset-config/yoctoAB.conf

 If you don't, you should source this script everytime you want start the
 autobuilder.

Creating config/autobuilder.conf from an example autobuilder.conf


Creating buildset-config from buildset-config.controller


 You've not setup the autobuilder before. Generating a server/client
password
 combo for you.
 Client/Server Password = (deleted, on purpose)

 Modifying the following files with this password:

 /media/data/yocto-autobuilder/yocto-controller/controller.cfg
 /media/data/yocto-autobuilder/yocto-controller/buildbot.tac
 /media/data/yocto-autobuilder/yocto-worker/buildbot.tac

 Updating worker-init script used for google cloud building.

 If you wish to use your own generated username and password please
 modify the above files as needed. Please see the README for more
 information.

 Generating an .htpasswd file using your current username and (deleted, on
purpose) at /media/data/yocto-autobuilder/.htpasswd
#########################################################################

 Please modify /media/data/yocto-autobuilder/config/autobuilder.conf if you
wish to specify
 a different location in which to publish build artifacts, etc.

#########################################################################

 Ready to start the yocto autobuilder.

 The yocto-autobuilder runs buildbot 0.8.8 with some modifications and
 a different git fetcher (yoctogit.py)

#########################################################################
checking basedir
checking for running master
checking controller.cfg
LOADING CONFIG FILE
*error while parsing config file:*
Traceback (most recent call last):
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py",
line 1187, in unwindGenerator
    return _inlineCallbacks(None, gen, Deferred())
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py",
line 1045, in _inlineCallbacks
    result = g.send(result)
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/buildbot-0.8.8-py2.7.egg/buildbot/scripts/upgrade_master.py",
line 169, in upgradeMaster
    master_cfg = loadConfig(config, configFile)
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/buildbot-0.8.8-py2.7.egg/buildbot/scripts/upgrade_master.py",
line 53, in loadConfig
    config['basedir'], configFileName)
--- <exception caught here> ---
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/buildbot-0.8.8-py2.7.egg/buildbot/config.py",
line 149, in loadConfig
    exec f in localDict
  File "/media/data/yocto-autobuilder/yocto-controller/controller.cfg",
line 94, in <module>
    yocto_buildsets.createBuildsets()
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/Autobuilder.py",
line 65, in createBuildsets
    self.parseBuildSet(buildset)
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/Autobuilder.py",
line 102, in parseBuildSet
    self.parseRepos(buildset)
  File
"/media/data/yocto-autobuilder/lib/python2.7/site-packages/autobuilder/Autobuilder.py",
line 127, in parseRepos
    if layer.iterkeys().next() not in self.repos:
*exceptions.AttributeError: 'list' object has no attribute 'iterkeys'*
Errors loading configuration:
  error while parsing config file: 'list' object has no attribute
'iterkeys' (traceback in logfile)
 To start the autobuilder:
 ./yocto-start-autobuilder <worker|controller|both>

 To stop the autobuilder:
 ./yocto-stop-autobuilder <worker|controller|both>

>0< alain@esxi-ub1 Thu Mar 23 05:34 PM [master !?]
/media/data/yocto-autobuilder >

Any ideas on how to fix this?

On Mon, Nov 14, 2016 at 1:40 PM, Bill Randle <bill.randle@gmail.com> wrote:

> Yes, the 2 TB is for everything you might do with an AutoBuilder. The
> actual space used will be closer to your 35 GB, since you're building
> only a single image / work product. People have used BuilBot directly
> for automated builds, but using the AutoBuilder code (which runs
> BuildBot underneath) gives you added flexibility and capability
> without much additional overhead.
>
>     -Bill
>
> On Mon, Nov 14, 2016 at 8:13 AM, Alain Achkar <alain.achkar@triacta.com>
> wrote:
> > Thanks for your answers! From reading these links, it is still not clear
> to
> > me if this might be overkill for my requirements. AB Cluster Setup talks
> > about "the worker requires 2+ TB to hold all the build temp files and git
> > repos. If build artifacts and a local sstate mirror are included,
> additional
> > worker space is required."
> >
> > Currently, my build only takes 35GB, so I think what these links are
> talking
> > about is how to replicate what the Autobuilder project
> > https://autobuilder.yoctoproject.org/ already does.
> >
> > To clarify, I am not interested in running builds and tests for
> everything
> > that Yocto already builds and tests (i.e. all the processor
> architectures,
> > all the machine types, etc.).  I am interested in running one build for
> one
> > machine type (the Variscite DART-6UL i.MX6UL arm-based processor, for
> which
> > NXP/Freescale and Variscite have provided recipes and layers for).
> >
> > I know that autobuilder includes BuildBot (this is why I specified it in
> > parentheses) but my question remains: do I only install BuildBot and try
> to
> > build my machine type with it, or do I install Autobuilder?
> >
> > On Mon, Nov 14, 2016 at 10:10 AM, Bill Randle <bill.randle@gmail.com>
> wrote:
> >>
> >> Also, be sure to check the Yocto Project wiki pages:
> >>     https://wiki.yoctoproject.org/wiki/The_Yocto_Autobuilder
> >> in particular, the AB cluster setup and AB maintenance links. Even
> >> though the one link refers to setting up an entire cluster, I've used
> >> that procedure to setup a single autobuilder, as well.
> >>
> >>     -Bill
> >>
> >> On Fri, Nov 11, 2016 at 10:37 AM, Beth 'pidge' Flanagan
> >> <pidge@toganlabs.com> wrote:
> >> > On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
> >> >> Hello Joshua!
> >> >>
> >> >> We are using Yocto to build for the Variscite DART-6UL SoM which is
> >> >> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.
> php?title=D
> >> >> ART-6UL_Yocto_Jethro_R1_build).
> >> >>
> >> >> Currently, developers run builds manually on their desktops. We are
> >> >> considering using a continuous integration server. I found out that
> >> >> The Yocto Project uses Autobuilder which in turn uses BuildBot. I
> >> >> read most of the introductory documentation.
> >> >>
> >> >> Would you recommend that we install only BuildBot and configure it to
> >> >> build what we need?
> >> >>
> >> >> OR
> >> >>
> >> >> Would you recommend that we install Autobuilder (and BuildBot) and
> >> >> configure it to build what we need?
> >> >
> >> > Actually, the yocto-autobuilder already contains buildbot so no need
> >> > for both.
> >> >
> >> > If you read:
> >> >
> >> > http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/
> tree/README-QUIC
> >> > KSTART
> >> >
> >> > and probably some of the other READMEs and the documentation in the
> >> > docs directory of the yocto-autobuilder directory, it should get you
> >> > started in the right direction.
> >> >
> >> > Feel free to ask questions if you need help.
> >> >
> >> > -b
> >> >
> >> > Elizabeth 'pidge' Flanagan
> >> > Founder/CTO Togán Labs
> >> > www.toganlabs.com
> >> >
> >> >>
> >> >> What is the faster path to get up and running?
> >> >>
> >> >> Thanks!
> >> >> Alain.
> >> >>
> >> >> --
> >> >> Alain Achkar, M.Eng. | Software Engineer
> >> >> Triacta Power Solutions
> >> >> (O) 613.256.2868 (C) 613.406.6520
> >> >> www.triacta.com
> >> >
> >> >
> >> >
> >> > --
> >> > _______________________________________________
> >> > yocto mailing list
> >> > yocto@yoctoproject.org
> >> > https://lists.yoctoproject.org/listinfo/yocto
> >
> >
> >
> >
> > --
> >
> > Alain Achkar, M.Eng. | Software Engineer
> >
> > Triacta Power Solutions
> >
> > (O) 613.256.2868 (C) 613.406.6520
> >
> > www.triacta.com
>



-- 

Alain Achkar, M.Eng. | Software Engineer

*Triacta Power Solutions*

(O) 613.256.2868 (C) 613.406.6520

*www.triacta.com  <http://www.triacta.com/>*

[-- Attachment #2: Type: text/html, Size: 18250 bytes --]

Re: Buildbot / Autobuilder / custom?

[Joshua Lock]

[-- Attachment #1: Type: text/plain, Size: 8762 bytes --]

Hi Alain,
On Thu, 2017-03-23 at 17:40 -0400, Alain Achkar wrote:
> I followed http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tre
> e/README-QUICKSTARTand got:

<snip>
> checking controller.cfg
> LOADING CONFIG FILE
> error while parsing config file:
> Traceback (most recent call last):
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/Twisted-12.2.0-py2.7-linux-
> x86_64.egg/twisted/internet/defer.py", line 1187, in unwindGenerator
>     return _inlineCallbacks(None, gen, Deferred())
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/Twisted-12.2.0-py2.7-linux-
> x86_64.egg/twisted/internet/defer.py", line 1045, in _inlineCallbacks
>     result = g.send(result)
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/buildbot-0.8.8-
> py2.7.egg/buildbot/scripts/upgrade_master.py", line 169, in
> upgradeMaster
>     master_cfg = loadConfig(config, configFile)
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/buildbot-0.8.8-
> py2.7.egg/buildbot/scripts/upgrade_master.py", line 53, in loadConfig
>     config['basedir'], configFileName)
> --- <exception caught here> ---
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/buildbot-0.8.8-py2.7.egg/buildbot/config.py", line 149, in
> loadConfig
>     exec f in localDict
>   File "/media/data/yocto-autobuilder/yocto-
> controller/controller.cfg", line 94, in <module>
>     yocto_buildsets.createBuildsets()
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/autobuilder/Autobuilder.py", line 65, in createBuildsets
>     self.parseBuildSet(buildset)
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/autobuilder/Autobuilder.py", line 102, in parseBuildSet
>     self.parseRepos(buildset)
>   File "/media/data/yocto-autobuilder/lib/python2.7/site-
> packages/autobuilder/Autobuilder.py", line 127, in parseRepos
>     if layer.iterkeys().next() not in self.repos:
> exceptions.AttributeError: 'list' object has no attribute 'iterkeys'

I appear to have left a stray comma in buildset-
config.controller/nightly.conf which causes the parser to choke. I've
fixed this in the master branch of yocto-autobuilder: http://git.yoctop
roject.org/clean/cgit.cgi/yocto-
autobuilder/commit/?id=239eeaa1e2cf3a985e75f8b75dedfe704a7d6991

You'll need to edit buildset-config/nightly.conf on your local instance
similarly to remove the comma at the end of the repos list, around line
38.
Thanks for reporting this error!
Joshua
> Errors loading configuration:
>   error while parsing config file: 'list' object has no attribute
> 'iterkeys' (traceback in logfile)
>  To start the autobuilder:
>  ./yocto-start-autobuilder <worker|controller|both>
> 
> 
>  To stop the autobuilder:
>  ./yocto-stop-autobuilder <worker|controller|both>
> 
> 
> >0< alain@esxi-ub1 Thu Mar 23 05:34 PM [master !?] /media/data/yocto-
> autobuilder >
> 
> 
> Any ideas on how to fix this?
> On Mon, Nov 14, 2016 at 1:40 PM, Bill Randle <bill.randle@gmail.com>
> wrote:
> > Yes, the 2 TB is for everything you might do with an AutoBuilder.
> > The
> > 
> > actual space used will be closer to your 35 GB, since you're
> > building
> > 
> > only a single image / work product. People have used BuilBot
> > directly
> > 
> > for automated builds, but using the AutoBuilder code (which runs
> > 
> > BuildBot underneath) gives you added flexibility and capability
> > 
> > without much additional overhead.
> > 
> > 
> > 
> >     -Bill
> > 
> > 
> > 
> > On Mon, Nov 14, 2016 at 8:13 AM, Alain Achkar <alain.achkar@triacta
> > .com> wrote:
> > 
> > > Thanks for your answers! From reading these links, it is still
> > not clear to
> > 
> > > me if this might be overkill for my requirements. AB Cluster
> > Setup talks
> > 
> > > about "the worker requires 2+ TB to hold all the build temp files
> > and git
> > 
> > > repos. If build artifacts and a local sstate mirror are included,
> > additional
> > 
> > > worker space is required."
> > 
> > >
> > 
> > > Currently, my build only takes 35GB, so I think what these links
> > are talking
> > 
> > > about is how to replicate what the Autobuilder project
> > 
> > > https://autobuilder.yoctoproject.org/ already does.
> > 
> > >
> > 
> > > To clarify, I am not interested in running builds and tests for
> > everything
> > 
> > > that Yocto already builds and tests (i.e. all the processor
> > architectures,
> > 
> > > all the machine types, etc.).  I am interested in running one
> > build for one
> > 
> > > machine type (the Variscite DART-6UL i.MX6UL arm-based processor,
> > for which
> > 
> > > NXP/Freescale and Variscite have provided recipes and layers
> > for).
> > 
> > >
> > 
> > > I know that autobuilder includes BuildBot (this is why I
> > specified it in
> > 
> > > parentheses) but my question remains: do I only install BuildBot
> > and try to
> > 
> > > build my machine type with it, or do I install Autobuilder?
> > 
> > >
> > 
> > > On Mon, Nov 14, 2016 at 10:10 AM, Bill Randle <bill.randle@gmail.
> > com> wrote:
> > 
> > >>
> > 
> > >> Also, be sure to check the Yocto Project wiki pages:
> > 
> > >>     https://wiki.yoctoproject.org/wiki/The_Yocto_Autobuilder
> > 
> > >> in particular, the AB cluster setup and AB maintenance links.
> > Even
> > 
> > >> though the one link refers to setting up an entire cluster, I've
> > used
> > 
> > >> that procedure to setup a single autobuilder, as well.
> > 
> > >>
> > 
> > >>     -Bill
> > 
> > >>
> > 
> > >> On Fri, Nov 11, 2016 at 10:37 AM, Beth 'pidge' Flanagan
> > 
> > >> <pidge@toganlabs.com> wrote:
> > 
> > >> > On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
> > 
> > >> >> Hello Joshua!
> > 
> > >> >>
> > 
> > >> >> We are using Yocto to build for the Variscite DART-6UL SoM
> > which is
> > 
> > >> >> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.php
> > ?title=D
> > 
> > >> >> ART-6UL_Yocto_Jethro_R1_build).
> > 
> > >> >>
> > 
> > >> >> Currently, developers run builds manually on their desktops.
> > We are
> > 
> > >> >> considering using a continuous integration server. I found
> > out that
> > 
> > >> >> The Yocto Project uses Autobuilder which in turn uses
> > BuildBot. I
> > 
> > >> >> read most of the introductory documentation.
> > 
> > >> >>
> > 
> > >> >> Would you recommend that we install only BuildBot and
> > configure it to
> > 
> > >> >> build what we need?
> > 
> > >> >>
> > 
> > >> >> OR
> > 
> > >> >>
> > 
> > >> >> Would you recommend that we install Autobuilder (and
> > BuildBot) and
> > 
> > >> >> configure it to build what we need?
> > 
> > >> >
> > 
> > >> > Actually, the yocto-autobuilder already contains buildbot so
> > no need
> > 
> > >> > for both.
> > 
> > >> >
> > 
> > >> > If you read:
> > 
> > >> >
> > 
> > >> > http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/RE
> > ADME-QUIC
> > 
> > >> > KSTART
> > 
> > >> >
> > 
> > >> > and probably some of the other READMEs and the documentation
> > in the
> > 
> > >> > docs directory of the yocto-autobuilder directory, it should
> > get you
> > 
> > >> > started in the right direction.
> > 
> > >> >
> > 
> > >> > Feel free to ask questions if you need help.
> > 
> > >> >
> > 
> > >> > -b
> > 
> > >> >
> > 
> > >> > Elizabeth 'pidge' Flanagan
> > 
> > >> > Founder/CTO Togán Labs
> > 
> > >> > www.toganlabs.com
> > 
> > >> >
> > 
> > >> >>
> > 
> > >> >> What is the faster path to get up and running?
> > 
> > >> >>
> > 
> > >> >> Thanks!
> > 
> > >> >> Alain.
> > 
> > >> >>
> > 
> > >> >> --
> > 
> > >> >> Alain Achkar, M.Eng. | Software Engineer
> > 
> > >> >> Triacta Power Solutions
> > 
> > >> >> (O) 613.256.2868 (C) 613.406.6520
> > 
> > >> >> www.triacta.com
> > 
> > >> >
> > 
> > >> >
> > 
> > >> >
> > 
> > >> > --
> > 
> > >> > _______________________________________________
> > 
> > >> > yocto mailing list
> > 
> > >> > yocto@yoctoproject.org
> > 
> > >> > https://lists.yoctoproject.org/listinfo/yocto
> > 
> > >
> > 
> > >
> > 
> > >
> > 
> > >
> > 
> > > --
> > 
> > >
> > 
> > > Alain Achkar, M.Eng. | Software Engineer
> > 
> > >
> > 
> > > Triacta Power Solutions
> > 
> > >
> > 
> > > (O) 613.256.2868 (C) 613.406.6520
> > 
> > >
> > 
> > > www.triacta.com
> > 
> > 
> 
> 
> -- 
> Alain Achkar, M.Eng. | Software Engineer
> Triacta Power Solutions
> (O) 613.256.2868 (C) 613.406.6520
> 
> www.triacta.com  
> 

[-- Attachment #2: Type: text/html, Size: 13732 bytes --]

Re: Buildbot / Autobuilder / custom?

[Alain Achkar]

[-- Attachment #1: Type: text/plain, Size: 7411 bytes --]

Thanks Joshua! This fix worked.

On Fri, Mar 24, 2017 at 6:44 AM, Joshua Lock <joshua.g.lock@linux.intel.com>
wrote:

> Hi Alain,
>
> On Thu, 2017-03-23 at 17:40 -0400, Alain Achkar wrote:
>
> I followed http://git.yoctoproject.org/cgit.cgi/
> yocto-autobuilder/tree/README-QUICKSTART
> and got:
>
>
> <snip>
>
> checking controller.cfg
> LOADING CONFIG FILE
> *error while parsing config file:*
> Traceback (most recent call last):
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/Twisted-12.2.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py",
> line 1187, in unwindGenerator
>     return _inlineCallbacks(None, gen, Deferred())
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/Twisted-12.2.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py",
> line 1045, in _inlineCallbacks
>     result = g.send(result)
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/buildbot-0.8.8-py2.7.egg/buildbot/scripts/upgrade_master.py",
> line 169, in upgradeMaster
>     master_cfg = loadConfig(config, configFile)
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/buildbot-0.8.8-py2.7.egg/buildbot/scripts/upgrade_master.py",
> line 53, in loadConfig
>     config['basedir'], configFileName)
> --- <exception caught here> ---
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/buildbot-0.8.8-py2.7.egg/buildbot/config.py", line 149, in
> loadConfig
>     exec f in localDict
>   File "/media/data/yocto-autobuilder/yocto-controller/controller.cfg",
> line 94, in <module>
>     yocto_buildsets.createBuildsets()
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/autobuilder/Autobuilder.py", line 65, in createBuildsets
>     self.parseBuildSet(buildset)
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/autobuilder/Autobuilder.py", line 102, in parseBuildSet
>     self.parseRepos(buildset)
>   File "/media/data/yocto-autobuilder/lib/python2.7/
> site-packages/autobuilder/Autobuilder.py", line 127, in parseRepos
>     if layer.iterkeys().next() not in self.repos:
> *exceptions.AttributeError: 'list' object has no attribute 'iterkeys'*
>
>
> I appear to have left a stray comma in buildset-config.controller/nightly.conf
> which causes the parser to choke. I've fixed this in the master branch of
> yocto-autobuilder: http://git.yoctoproject.org/clean/cgit.cgi/yocto-
> autobuilder/commit/?id=239eeaa1e2cf3a985e75f8b75dedfe704a7d6991
>
>
> You'll need to edit buildset-config/nightly.conf on your local instance
> similarly to remove the comma at the end of the repos list, around line 38.
>
> Thanks for reporting this error!
>
> Joshua
>
> Errors loading configuration:
>   error while parsing config file: 'list' object has no attribute
> 'iterkeys' (traceback in logfile)
>  To start the autobuilder:
>  ./yocto-start-autobuilder <worker|controller|both>
>
>  To stop the autobuilder:
>  ./yocto-stop-autobuilder <worker|controller|both>
>
> >0< alain@esxi-ub1 Thu Mar 23 05:34 PM [master !?]
> /media/data/yocto-autobuilder >
>
> Any ideas on how to fix this?
>
> On Mon, Nov 14, 2016 at 1:40 PM, Bill Randle <bill.randle@gmail.com>
> wrote:
>
> Yes, the 2 TB is for everything you might do with an AutoBuilder. The
> actual space used will be closer to your 35 GB, since you're building
> only a single image / work product. People have used BuilBot directly
> for automated builds, but using the AutoBuilder code (which runs
> BuildBot underneath) gives you added flexibility and capability
> without much additional overhead.
>
>     -Bill
>
> On Mon, Nov 14, 2016 at 8:13 AM, Alain Achkar <alain.achkar@triacta.com>
> wrote:
> > Thanks for your answers! From reading these links, it is still not clear
> to
> > me if this might be overkill for my requirements. AB Cluster Setup talks
> > about "the worker requires 2+ TB to hold all the build temp files and git
> > repos. If build artifacts and a local sstate mirror are included,
> additional
> > worker space is required."
> >
> > Currently, my build only takes 35GB, so I think what these links are
> talking
> > about is how to replicate what the Autobuilder project
> > https://autobuilder.yoctoproject.org/ already does.
> >
> > To clarify, I am not interested in running builds and tests for
> everything
> > that Yocto already builds and tests (i.e. all the processor
> architectures,
> > all the machine types, etc.).  I am interested in running one build for
> one
> > machine type (the Variscite DART-6UL i.MX6UL arm-based processor, for
> which
> > NXP/Freescale and Variscite have provided recipes and layers for).
> >
> > I know that autobuilder includes BuildBot (this is why I specified it in
> > parentheses) but my question remains: do I only install BuildBot and try
> to
> > build my machine type with it, or do I install Autobuilder?
> >
> > On Mon, Nov 14, 2016 at 10:10 AM, Bill Randle <bill.randle@gmail.com>
> wrote:
> >>
> >> Also, be sure to check the Yocto Project wiki pages:
> >>     https://wiki.yoctoproject.org/wiki/The_Yocto_Autobuilder
> >> in particular, the AB cluster setup and AB maintenance links. Even
> >> though the one link refers to setting up an entire cluster, I've used
> >> that procedure to setup a single autobuilder, as well.
> >>
> >>     -Bill
> >>
> >> On Fri, Nov 11, 2016 at 10:37 AM, Beth 'pidge' Flanagan
> >> <pidge@toganlabs.com> wrote:
> >> > On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
> >> >> Hello Joshua!
> >> >>
> >> >> We are using Yocto to build for the Variscite DART-6UL SoM which is
> >> >> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.php
> ?title=D
> >> >> ART-6UL_Yocto_Jethro_R1_build).
> >> >>
> >> >> Currently, developers run builds manually on their desktops. We are
> >> >> considering using a continuous integration server. I found out that
> >> >> The Yocto Project uses Autobuilder which in turn uses BuildBot. I
> >> >> read most of the introductory documentation.
> >> >>
> >> >> Would you recommend that we install only BuildBot and configure it to
> >> >> build what we need?
> >> >>
> >> >> OR
> >> >>
> >> >> Would you recommend that we install Autobuilder (and BuildBot) and
> >> >> configure it to build what we need?
> >> >
> >> > Actually, the yocto-autobuilder already contains buildbot so no need
> >> > for both.
> >> >
> >> > If you read:
> >> >
> >> > http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/
> README-QUIC
> >> > KSTART
> >> >
> >> > and probably some of the other READMEs and the documentation in the
> >> > docs directory of the yocto-autobuilder directory, it should get you
> >> > started in the right direction.
> >> >
> >> > Feel free to ask questions if you need help.
> >> >
> >> > -b
> >> >
> >> > Elizabeth 'pidge' Flanagan
> >> > Founder/CTO Togán Labs
> >> > www.toganlabs.com
> >> >
> >> >>
> >> >> What is the faster path to get up and running?
> >> >>
> >> >> Thanks!
> >> >> Alain.
> >> >>
> >> >> --
>
> >> >
> >> >
> >> >
> >> > --
> >> > _______________________________________________
> >> > yocto mailing list
> >> > yocto@yoctoproject.org
> >> > https://lists.yoctoproject.org/listinfo/yocto
> >
> >
> >
> >
> > --
>
>
>
>
> --
>
>
>

[-- Attachment #2: Type: text/html, Size: 12743 bytes --]

Re: Buildbot / Autobuilder / custom?

[Alain Achkar]

[-- Attachment #1: Type: text/plain, Size: 6412 bytes --]

Hi Chris!

I had to put this task on hold and I am re-visiting it now. I followed the
instructions here:  http://git.yoctoproject.org/cgit.cgi/
yocto-autobuilder/tree/README-QUICKSTART and managed to force one of the
builds that comes with Yocto. I got errors but I'm not interested in the
Yocto-provided nightly builds anyway. (The errors are related to
buildhistory - it seems that some builds set Buildhistory=True but I
haven't set up a git repo for build history)

Now, I am still confused how to build the Variscite image for the i.MX6UL.
We are still at Jethro but don't mind upgrading to Krogoth.

I would appreciate some general guidelines on how to take the instructions
in here:
http://variwiki.com/index.php?title=DART-6UL_Yocto_Jethro_R1_build
and transform them to the config format that Autobuilder understands.

Mainly, I understand that I have to:

1) checkout the source code

$ mkdir ~/var-mx6ul-mx7-yocto-jethro
$ cd ~/var-mx6ul-mx7-yocto-jethro
$ repo init -u git://git.freescale.com/imx/fsl-arm-yocto-bsp.git -b
imx-4.1.15-1.0.0_ga
$ repo sync

$ cd ~/var-mx6ul-mx7-yocto-jethro/sources
$ git clone https://github.com/varigit/meta-variscite-mx6ul-mx7.git -b
imx_4.1.15_ga-var01


2) copy and patch

$ cp meta-variscite-mx6ul-mx7/scripts/var-setup-release.sh ../
$ patch -p1 < meta-variscite-mx6ul-mx7/patch/Fix-FSL-multi-patch-append-bugs.patch


3) setup a download directory

$ sudo mkdir /opt/yocto_downloads
$ sudo chmod 777 /opt/yocto_downloads/
$ sed -i 's/DL_DIR ?= "${BSPDIR}\/downloads/DL_DIR =
"\/opt\/yocto_downloads/g' conf/local.conf


4) setup the environment

$ cd ~/var-mx6ul-mx7-yocto-jethro
$ MACHINE=imx6ul-var-dart DISTRO=fsl-imx-x11 source
var-setup-release.sh -b build_x11


5) build

$ bitbake fsl-image-gui


I looked at some nightly-*.conf files, for example nightly-x86-64.conf to
try to guess what to do.

I know I have to create a file like nightly-var-mx6ul.conf but given that
Variscite is not using the 'repo' command that Freescale is using (they do
a git clone) and given step (2) of copying their own setup script and
patching the whole freescale meta- layers, I am not sure about what to put
instead of the line:

        {'CheckOutLayers': {}},


I also can't figure out how the info in Variscite's build-x11/local.conf
can be translated to the Autobuilder format, i.e. where to specify all
these config parameters.

Any help would be greatly appreciated.


On Mon, Nov 14, 2016 at 12:35 PM, Alain Achkar <alain.achkar@triacta.com>
wrote:

> Thanks Chris!
>
> This is the answer I was looking for.
>
> On Mon, Nov 14, 2016 at 11:58 AM, Chris Whittenburg <whittenburg@gmail.com
> > wrote:
>
>>
>> Hi Alain,
>>
>> We use the same module-- Variscite DART MX6, and I was able to hack the
>> yocto autobuilder to build our image nightly.  I host it all on a single
>> machine.
>>
>> -chris
>>
>>
>> On Mon, Nov 14, 2016 at 10:13 AM, Alain Achkar <alain.achkar@triacta.com>
>> wrote:
>>
>>> Thanks for your answers! From reading these links, it is still not clear
>>> to me if this might be overkill for my requirements. AB Cluster Setup talks
>>> about "*the worker requires 2+ TB to hold all the build temp files and
>>> git repos. If build artifacts and a local sstate mirror are included,
>>> additional worker space is required.*"
>>>
>>> Currently, my build only takes 35GB, so I think what these links are
>>> talking about is how to replicate what the Autobuilder project
>>> https://autobuilder.yoctoproject.org/ already does.
>>>
>>> To clarify, I am not interested in running builds and tests for
>>> everything that Yocto already builds and tests (i.e. all the processor
>>> architectures, all the machine types, etc.).  I am interested in running
>>> one build for one machine type (the Variscite DART-6UL i.MX6UL arm-based
>>> processor, for which NXP/Freescale and Variscite have provided recipes and
>>> layers for).
>>>
>>> I know that autobuilder includes BuildBot (this is why I specified it in
>>> parentheses) but my question remains: do I only install BuildBot and try to
>>> build my machine type with it, or do I install Autobuilder?
>>>
>>> On Mon, Nov 14, 2016 at 10:10 AM, Bill Randle <bill.randle@gmail.com>
>>> wrote:
>>>
>>>> Also, be sure to check the Yocto Project wiki pages:
>>>>     https://wiki.yoctoproject.org/wiki/The_Yocto_Autobuilder
>>>> in particular, the AB cluster setup and AB maintenance links. Even
>>>> though the one link refers to setting up an entire cluster, I've used
>>>> that procedure to setup a single autobuilder, as well.
>>>>
>>>>     -Bill
>>>>
>>>> On Fri, Nov 11, 2016 at 10:37 AM, Beth 'pidge' Flanagan
>>>> <pidge@toganlabs.com> wrote:
>>>> > On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
>>>> >> Hello Joshua!
>>>> >>
>>>> >> We are using Yocto to build for the Variscite DART-6UL SoM which is
>>>> >> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.php
>>>> ?title=D
>>>> >> ART-6UL_Yocto_Jethro_R1_build).
>>>> >>
>>>> >> Currently, developers run builds manually on their desktops. We are
>>>> >> considering using a continuous integration server. I found out that
>>>> >> The Yocto Project uses Autobuilder which in turn uses BuildBot. I
>>>> >> read most of the introductory documentation.
>>>> >>
>>>> >> Would you recommend that we install only BuildBot and configure it to
>>>> >> build what we need?
>>>> >>
>>>> >> OR
>>>> >>
>>>> >> Would you recommend that we install Autobuilder (and BuildBot) and
>>>> >> configure it to build what we need?
>>>> >
>>>> > Actually, the yocto-autobuilder already contains buildbot so no need
>>>> > for both.
>>>> >
>>>> > If you read:
>>>> >
>>>> > http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/
>>>> README-QUIC
>>>> > KSTART
>>>> >
>>>> > and probably some of the other READMEs and the documentation in the
>>>> > docs directory of the yocto-autobuilder directory, it should get you
>>>> > started in the right direction.
>>>> >
>>>> > Feel free to ask questions if you need help.
>>>> >
>>>> > -b
>>>> >
>>>> > Elizabeth 'pidge' Flanagan
>>>> > Founder/CTO Togán Labs
>>>> > www.toganlabs.com
>>>> >
>>>> >>
>>>> >> What is the faster path to get up and running?
>>>> >>
>>>> >> Thanks!
>>>> >> Alain.
>>>> >>
>>>>
>>>
>>>

[-- Attachment #2: Type: text/html, Size: 12248 bytes --]

Re: Buildbot / Autobuilder / custom?

[Alain Achkar]

[-- Attachment #1: Type: text/plain, Size: 6962 bytes --]

Kindly requesting help again, please.
I understand that Variscite might not be following the "Yocto way" 100% but
surely we can run custom commands using Autobuilder?

Please see detailed description in my last post.
Thanks!

On Mon, Apr 3, 2017 at 9:43 PM, Alain Achkar <alain.achkar@triacta.com>
wrote:

> Hi Chris!
>
> I had to put this task on hold and I am re-visiting it now. I followed the
> instructions here:  http://git.yoctoproject.org/cgit.cgi/yocto-
> autobuilder/tree/README-QUICKSTART and managed to force one of the builds
> that comes with Yocto. I got errors but I'm not interested in the
> Yocto-provided nightly builds anyway. (The errors are related to
> buildhistory - it seems that some builds set Buildhistory=True but I
> haven't set up a git repo for build history)
>
> Now, I am still confused how to build the Variscite image for the i.MX6UL.
> We are still at Jethro but don't mind upgrading to Krogoth.
>
> I would appreciate some general guidelines on how to take the instructions
> in here:
> http://variwiki.com/index.php?title=DART-6UL_Yocto_Jethro_R1_build
> and transform them to the config format that Autobuilder understands.
>
> Mainly, I understand that I have to:
>
> 1) checkout the source code
>
> $ mkdir ~/var-mx6ul-mx7-yocto-jethro
> $ cd ~/var-mx6ul-mx7-yocto-jethro
> $ repo init -u git://git.freescale.com/imx/fsl-arm-yocto-bsp.git -b imx-4.1.15-1.0.0_ga
> $ repo sync
>
> $ cd ~/var-mx6ul-mx7-yocto-jethro/sources
> $ git clone https://github.com/varigit/meta-variscite-mx6ul-mx7.git -b imx_4.1.15_ga-var01
>
>
> 2) copy and patch
>
> $ cp meta-variscite-mx6ul-mx7/scripts/var-setup-release.sh ../
> $ patch -p1 < meta-variscite-mx6ul-mx7/patch/Fix-FSL-multi-patch-append-bugs.patch
>
>
> 3) setup a download directory
>
> $ sudo mkdir /opt/yocto_downloads
> $ sudo chmod 777 /opt/yocto_downloads/
> $ sed -i 's/DL_DIR ?= "${BSPDIR}\/downloads/DL_DIR = "\/opt\/yocto_downloads/g' conf/local.conf
>
>
> 4) setup the environment
>
> $ cd ~/var-mx6ul-mx7-yocto-jethro
> $ MACHINE=imx6ul-var-dart DISTRO=fsl-imx-x11 source var-setup-release.sh -b build_x11
>
>
> 5) build
>
> $ bitbake fsl-image-gui
>
>
> I looked at some nightly-*.conf files, for example nightly-x86-64.conf to
> try to guess what to do.
>
> I know I have to create a file like nightly-var-mx6ul.conf but given that
> Variscite is not using the 'repo' command that Freescale is using (they do
> a git clone) and given step (2) of copying their own setup script and
> patching the whole freescale meta- layers, I am not sure about what to put
> instead of the line:
>
>         {'CheckOutLayers': {}},
>
>
> I also can't figure out how the info in Variscite's build-x11/local.conf
> can be translated to the Autobuilder format, i.e. where to specify all
> these config parameters.
>
> Any help would be greatly appreciated.
>
>
> On Mon, Nov 14, 2016 at 12:35 PM, Alain Achkar <alain.achkar@triacta.com>
> wrote:
>
>> Thanks Chris!
>>
>> This is the answer I was looking for.
>>
>> On Mon, Nov 14, 2016 at 11:58 AM, Chris Whittenburg <
>> whittenburg@gmail.com> wrote:
>>
>>>
>>> Hi Alain,
>>>
>>> We use the same module-- Variscite DART MX6, and I was able to hack the
>>> yocto autobuilder to build our image nightly.  I host it all on a single
>>> machine.
>>>
>>> -chris
>>>
>>>
>>> On Mon, Nov 14, 2016 at 10:13 AM, Alain Achkar <alain.achkar@triacta.com
>>> > wrote:
>>>
>>>> Thanks for your answers! From reading these links, it is still not
>>>> clear to me if this might be overkill for my requirements. AB Cluster Setup
>>>> talks about "*the worker requires 2+ TB to hold all the build temp
>>>> files and git repos. If build artifacts and a local sstate mirror are
>>>> included, additional worker space is required.*"
>>>>
>>>> Currently, my build only takes 35GB, so I think what these links are
>>>> talking about is how to replicate what the Autobuilder project
>>>> https://autobuilder.yoctoproject.org/ already does.
>>>>
>>>> To clarify, I am not interested in running builds and tests for
>>>> everything that Yocto already builds and tests (i.e. all the processor
>>>> architectures, all the machine types, etc.).  I am interested in running
>>>> one build for one machine type (the Variscite DART-6UL i.MX6UL arm-based
>>>> processor, for which NXP/Freescale and Variscite have provided recipes and
>>>> layers for).
>>>>
>>>> I know that autobuilder includes BuildBot (this is why I specified it
>>>> in parentheses) but my question remains: do I only install BuildBot and try
>>>> to build my machine type with it, or do I install Autobuilder?
>>>>
>>>> On Mon, Nov 14, 2016 at 10:10 AM, Bill Randle <bill.randle@gmail.com>
>>>> wrote:
>>>>
>>>>> Also, be sure to check the Yocto Project wiki pages:
>>>>>     https://wiki.yoctoproject.org/wiki/The_Yocto_Autobuilder
>>>>> in particular, the AB cluster setup and AB maintenance links. Even
>>>>> though the one link refers to setting up an entire cluster, I've used
>>>>> that procedure to setup a single autobuilder, as well.
>>>>>
>>>>>     -Bill
>>>>>
>>>>> On Fri, Nov 11, 2016 at 10:37 AM, Beth 'pidge' Flanagan
>>>>> <pidge@toganlabs.com> wrote:
>>>>> > On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
>>>>> >> Hello Joshua!
>>>>> >>
>>>>> >> We are using Yocto to build for the Variscite DART-6UL SoM which is
>>>>> >> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.php
>>>>> ?title=D
>>>>> >> ART-6UL_Yocto_Jethro_R1_build).
>>>>> >>
>>>>> >> Currently, developers run builds manually on their desktops. We are
>>>>> >> considering using a continuous integration server. I found out that
>>>>> >> The Yocto Project uses Autobuilder which in turn uses BuildBot. I
>>>>> >> read most of the introductory documentation.
>>>>> >>
>>>>> >> Would you recommend that we install only BuildBot and configure it
>>>>> to
>>>>> >> build what we need?
>>>>> >>
>>>>> >> OR
>>>>> >>
>>>>> >> Would you recommend that we install Autobuilder (and BuildBot) and
>>>>> >> configure it to build what we need?
>>>>> >
>>>>> > Actually, the yocto-autobuilder already contains buildbot so no need
>>>>> > for both.
>>>>> >
>>>>> > If you read:
>>>>> >
>>>>> > http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/
>>>>> README-QUIC
>>>>> > KSTART
>>>>> >
>>>>> > and probably some of the other READMEs and the documentation in the
>>>>> > docs directory of the yocto-autobuilder directory, it should get you
>>>>> > started in the right direction.
>>>>> >
>>>>> > Feel free to ask questions if you need help.
>>>>> >
>>>>> > -b
>>>>> >
>>>>> > Elizabeth 'pidge' Flanagan
>>>>> > Founder/CTO Togán Labs
>>>>> > www.toganlabs.com
>>>>> >
>>>>> >>
>>>>> >> What is the faster path to get up and running?
>>>>> >>
>>>>> >> Thanks!
>>>>> >> Alain.
>>>>> >>
>>>>>
>>>>
>>>>

[-- Attachment #2: Type: text/html, Size: 13219 bytes --]

Re: Buildbot / Autobuilder / custom?

[Alain Achkar]

[-- Attachment #1: Type: text/plain, Size: 10793 bytes --]

Hello Beth and/or Joshua,

I would really appreciate some pointers on how to get the Variscite builds
to work with Autobuilder.

Since my last email, I was able to setup the Yocto Autobuilder, add the
sample hello.conf and run it.
I was also able to add another file, nightly-variscite-x11.conf and I can
see it in the Waterfall view. Initially, I started by filling it with
content similar to hello.conf and this worked. Now, I need to be able to
add the Variscite build.

Since my last email, Variscite have moved to the morty branch, since
Freescale/NXP have moved to the morty branch. So, the wiki instructions
here
http://variwiki.com/index.php?title=Yocto_Build_Release&release=RELEASE_MORTY_BETA_DART-6UL
are now simpler:

1) Checkout the source code

$ mkdir ~/var-fslc-yocto
$ cd ~/var-fslc-yocto
$ repo init -u https://github.com/varigit/variscite-bsp-platform.git -b morty
$ repo sync -j4


2) Run the setup script (which calls oe-init-environment)

$ cd ~/var-fslc-yocto
$ MACHINE=imx6ul-var-dart DISTRO=fslc-x11 . setup-environment build_x11


3) Build X11 demo images

# Without Qt content:
$ bitbake fsl-image-gui

# Or with Qt content:
$ bitbake fsl-image-qt5


The 'repo' command reads the file default.xml which has the following
contents:

<?xml version="1.0" encoding="UTF-8"?>
<manifest>

  <default sync-j="4" revision="morty"/>

  <remote fetch="https://git.yoctoproject.org/git" name="yocto"/>
  <remote fetch="https://github.com/Freescale"     name="freescale"/>
  <remote fetch="https://github.com/openembedded"  name="oe"/>
  <remote fetch="https://github.com/varigit"       name="variscite"/>
  <remote fetch="https://github.com/OSSystems"     name="OSSystems"/>
  <remote fetch="git://github.com/meta-qt5"        name="QT5"/>

  <project remote="yocto"
revision="6a1f33cc40bfac33cf030fe41e1a8efd1e5fad6f" upstream="morty"
name="poky" path="sources/poky"/>
  <project remote="yocto"
revision="56a1460777d1e24b1dfb426cad2b103740936a83" upstream="morty"
name="meta-freescale" path="sources/meta-freescale" />

  <project remote="oe"
revision="fe5c83312de11e80b85680ef237f8acb04b4b26e" upstream="morty"
name="meta-openembedded" path="sources/meta-openembedded" />

  <project remote="variscite" revision="morty"
name="variscite-bsp-base" path="sources/base">
	<copyfile dest="README" src="README"/>
	<copyfile dest="setup-environment" src="setup-environment"/>
  </project>

  <project remote="freescale"
revision="4be3db25959e741ac9164a32c82b8c09b59213ad" upstream="morty"
name="meta-freescale-3rdparty"
path="sources/meta-freescale-3rdparty"/>
  <project remote="freescale"
revision="dd482aeb7913c099c69f82362776bd92b584553c" upstream="morty"
name="meta-freescale-distro" path="sources/meta-freescale-distro"/>
  <project remote="QT5"
revision="3601fd2c5306ac6d5d0d536e0be8cbb90da9b4c1" upstream="morty"
name="meta-qt5" path="sources/meta-qt5"/>
  <project remote="OSSystems"
revision="7d37ae2adec5cc5ea27367ce125bc7bce2310ca1" upstream="morty"
name="meta-browser" path="sources/meta-browser"/>
  <project remote="variscite"
revision="7a214e7579869be810aa8992c59627ba064db076" upstream="morty"
name="meta-variscite-fslc" path="sources/meta-variscite-fslc"  />

</manifest>


So, AFAIK, I need to either (a) translate these into the 'repos' property
in my buildset, or write a custom buildstep that can use the 'repo' tool to
clone the needed repositories.

Any help would be greatly appreciated.
Thanks!


On Fri, Apr 21, 2017 at 11:39 AM, Alain Achkar <alain.achkar@triacta.com>
wrote:

> Kindly requesting help again, please.
> I understand that Variscite might not be following the "Yocto way" 100%
> but surely we can run custom commands using Autobuilder?
>
> Please see detailed description in my last post.
> Thanks!
>
> On Mon, Apr 3, 2017 at 9:43 PM, Alain Achkar <alain.achkar@triacta.com>
> wrote:
>
>> Hi Chris!
>>
>> I had to put this task on hold and I am re-visiting it now. I followed
>> the instructions here:  http://git.yoctoproject
>> .org/cgit.cgi/yocto-autobuilder/tree/README-QUICKSTART and managed to
>> force one of the builds that comes with Yocto. I got errors but I'm not
>> interested in the Yocto-provided nightly builds anyway. (The errors are
>> related to buildhistory - it seems that some builds set Buildhistory=True
>> but I haven't set up a git repo for build history)
>>
>> Now, I am still confused how to build the Variscite image for the
>> i.MX6UL. We are still at Jethro but don't mind upgrading to Krogoth.
>>
>> I would appreciate some general guidelines on how to take the
>> instructions in here:
>> http://variwiki.com/index.php?title=DART-6UL_Yocto_Jethro_R1_build
>> and transform them to the config format that Autobuilder understands.
>>
>> Mainly, I understand that I have to:
>>
>> 1) checkout the source code
>>
>> $ mkdir ~/var-mx6ul-mx7-yocto-jethro
>> $ cd ~/var-mx6ul-mx7-yocto-jethro
>> $ repo init -u git://git.freescale.com/imx/fsl-arm-yocto-bsp.git -b imx-4.1.15-1.0.0_ga
>> $ repo sync
>>
>> $ cd ~/var-mx6ul-mx7-yocto-jethro/sources
>> $ git clone https://github.com/varigit/meta-variscite-mx6ul-mx7.git -b imx_4.1.15_ga-var01
>>
>>
>> 2) copy and patch
>>
>> $ cp meta-variscite-mx6ul-mx7/scripts/var-setup-release.sh ../
>> $ patch -p1 < meta-variscite-mx6ul-mx7/patch/Fix-FSL-multi-patch-append-bugs.patch
>>
>>
>> 3) setup a download directory
>>
>> $ sudo mkdir /opt/yocto_downloads
>> $ sudo chmod 777 /opt/yocto_downloads/
>> $ sed -i 's/DL_DIR ?= "${BSPDIR}\/downloads/DL_DIR = "\/opt\/yocto_downloads/g' conf/local.conf
>>
>>
>> 4) setup the environment
>>
>> $ cd ~/var-mx6ul-mx7-yocto-jethro
>> $ MACHINE=imx6ul-var-dart DISTRO=fsl-imx-x11 source var-setup-release.sh -b build_x11
>>
>>
>> 5) build
>>
>> $ bitbake fsl-image-gui
>>
>>
>> I looked at some nightly-*.conf files, for example nightly-x86-64.conf to
>> try to guess what to do.
>>
>> I know I have to create a file like nightly-var-mx6ul.conf but given that
>> Variscite is not using the 'repo' command that Freescale is using (they do
>> a git clone) and given step (2) of copying their own setup script and
>> patching the whole freescale meta- layers, I am not sure about what to put
>> instead of the line:
>>
>>         {'CheckOutLayers': {}},
>>
>>
>> I also can't figure out how the info in Variscite's build-x11/local.conf
>> can be translated to the Autobuilder format, i.e. where to specify all
>> these config parameters.
>>
>> Any help would be greatly appreciated.
>>
>>
>> On Mon, Nov 14, 2016 at 12:35 PM, Alain Achkar <alain.achkar@triacta.com>
>> wrote:
>>
>>> Thanks Chris!
>>>
>>> This is the answer I was looking for.
>>>
>>> On Mon, Nov 14, 2016 at 11:58 AM, Chris Whittenburg <
>>> whittenburg@gmail.com> wrote:
>>>
>>>>
>>>> Hi Alain,
>>>>
>>>> We use the same module-- Variscite DART MX6, and I was able to hack the
>>>> yocto autobuilder to build our image nightly.  I host it all on a single
>>>> machine.
>>>>
>>>> -chris
>>>>
>>>>
>>>> On Mon, Nov 14, 2016 at 10:13 AM, Alain Achkar <
>>>> alain.achkar@triacta.com> wrote:
>>>>
>>>>> Thanks for your answers! From reading these links, it is still not
>>>>> clear to me if this might be overkill for my requirements. AB Cluster Setup
>>>>> talks about "*the worker requires 2+ TB to hold all the build temp
>>>>> files and git repos. If build artifacts and a local sstate mirror are
>>>>> included, additional worker space is required.*"
>>>>>
>>>>> Currently, my build only takes 35GB, so I think what these links are
>>>>> talking about is how to replicate what the Autobuilder project
>>>>> https://autobuilder.yoctoproject.org/ already does.
>>>>>
>>>>> To clarify, I am not interested in running builds and tests for
>>>>> everything that Yocto already builds and tests (i.e. all the processor
>>>>> architectures, all the machine types, etc.).  I am interested in running
>>>>> one build for one machine type (the Variscite DART-6UL i.MX6UL arm-based
>>>>> processor, for which NXP/Freescale and Variscite have provided recipes and
>>>>> layers for).
>>>>>
>>>>> I know that autobuilder includes BuildBot (this is why I specified it
>>>>> in parentheses) but my question remains: do I only install BuildBot and try
>>>>> to build my machine type with it, or do I install Autobuilder?
>>>>>
>>>>> On Mon, Nov 14, 2016 at 10:10 AM, Bill Randle <bill.randle@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Also, be sure to check the Yocto Project wiki pages:
>>>>>>     https://wiki.yoctoproject.org/wiki/The_Yocto_Autobuilder
>>>>>> in particular, the AB cluster setup and AB maintenance links. Even
>>>>>> though the one link refers to setting up an entire cluster, I've used
>>>>>> that procedure to setup a single autobuilder, as well.
>>>>>>
>>>>>>     -Bill
>>>>>>
>>>>>> On Fri, Nov 11, 2016 at 10:37 AM, Beth 'pidge' Flanagan
>>>>>> <pidge@toganlabs.com> wrote:
>>>>>> > On Fri, 2016-11-11 at 12:35 -0500, Alain Achkar wrote:
>>>>>> >> Hello Joshua!
>>>>>> >>
>>>>>> >> We are using Yocto to build for the Variscite DART-6UL SoM which is
>>>>>> >> based on Freescale/NXP i.MX6UL (http://variwiki.com/index.php
>>>>>> ?title=D
>>>>>> >> ART-6UL_Yocto_Jethro_R1_build).
>>>>>> >>
>>>>>> >> Currently, developers run builds manually on their desktops. We are
>>>>>> >> considering using a continuous integration server. I found out that
>>>>>> >> The Yocto Project uses Autobuilder which in turn uses BuildBot. I
>>>>>> >> read most of the introductory documentation.
>>>>>> >>
>>>>>> >> Would you recommend that we install only BuildBot and configure it
>>>>>> to
>>>>>> >> build what we need?
>>>>>> >>
>>>>>> >> OR
>>>>>> >>
>>>>>> >> Would you recommend that we install Autobuilder (and BuildBot) and
>>>>>> >> configure it to build what we need?
>>>>>> >
>>>>>> > Actually, the yocto-autobuilder already contains buildbot so no need
>>>>>> > for both.
>>>>>> >
>>>>>> > If you read:
>>>>>> >
>>>>>> > http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder/tree/
>>>>>> README-QUIC
>>>>>> > KSTART
>>>>>> >
>>>>>> > and probably some of the other READMEs and the documentation in the
>>>>>> > docs directory of the yocto-autobuilder directory, it should get you
>>>>>> > started in the right direction.
>>>>>> >
>>>>>> > Feel free to ask questions if you need help.
>>>>>> >
>>>>>> > -b
>>>>>> >
>>>>>> > Elizabeth 'pidge' Flanagan
>>>>>> > Founder/CTO Togán Labs
>>>>>> > www.toganlabs.com
>>>>>> >
>>>>>> >>
>>>>>> >> What is the faster path to get up and running?
>>>>>> >>
>>>>>> >> Thanks!
>>>>>> >> Alain.
>>>>>> >>
>>>>>>
>>>>>
>>>>>
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 22126 bytes --]

Re: Buildbot / Autobuilder / custom?

[Joshua Lock]

[-- Attachment #1: Type: text/plain, Size: 4315 bytes --]

On Mon, 2017-05-29 at 17:01 -0400, Alain Achkar wrote:
> Hello Beth and/or Joshua,
> I would really appreciate some pointers on how to get the Variscite
> builds to work with Autobuilder.
> 
> Since my last email, I was able to setup the Yocto Autobuilder, add
> the sample hello.conf and run it.
> I was also able to add another file, nightly-variscite-x11.conf and I
> can see it in the Waterfall view. Initially, I started by filling it
> with content similar to hello.conf and this worked. Now, I need to be
> able to add the Variscite build.
> 
> Since my last email, Variscite have moved to the morty branch, since
> Freescale/NXP have moved to the morty branch. So, the wiki
> instructions here http://variwiki.com/index.php?title=Yocto_Build_Rel
> ease&release=RELEASE_MORTY_BETA_DART-6UL are now simpler:
> 
> 1) Checkout the source code
> $ mkdir ~/var-fslc-yocto
> $ cd ~/var-fslc-yocto
> $ repo init -u https://github.com/varigit/variscite-bsp-platform.git
> -b morty
> $ repo sync -j4
> 
> 2) Run the setup script (which calls oe-init-environment)
> $ cd ~/var-fslc-yocto
> $ MACHINE=imx6ul-var-dart DISTRO=fslc-x11 . setup-environment
> build_x11
> 
> 3) Build X11 demo images
> # Without Qt content:
> $ bitbake fsl-image-gui
> 
> # Or with Qt content:
> $ bitbake fsl-image-qt5
> 
> The 'repo' command reads the file default.xml which has the following
> contents:
> 
> <?xml version="1.0" encoding="UTF-8"?>
> <manifest>
> 
> 
>   <default sync-j="4" revision="morty"/>
> 
> 
>   <remote fetch="https://git.yoctoproject.org/git" name="yocto"/>
>   <remote fetch="https://github.com/Freescale"     name="freescale"/>
>   <remote fetch="https://github.com/openembedded"  name="oe"/>
>   <remote fetch="https://github.com/varigit"       name="variscite"/>
>   <remote fetch="https://github.com/OSSystems"     name="OSSystems"/>
>   <remote fetch="git://github.com/meta-qt5"        name="QT5"/>
> 
> 
>   <project remote="yocto"
> revision="6a1f33cc40bfac33cf030fe41e1a8efd1e5fad6f" upstream="morty"
>  name="poky" path="sources/poky"/>
>   <project remote="yocto"
> revision="56a1460777d1e24b1dfb426cad2b103740936a83" upstream="morty"
>  name="meta-freescale" path="sources/meta-freescale" />
> 
> 
>   <project remote="oe"
> revision="fe5c83312de11e80b85680ef237f8acb04b4b26e" upstream="morty"
> name="meta-openembedded" path="sources/meta-openembedded" />
> 
> 
>   <project remote="variscite" revision="morty" name="variscite-bsp-
> base" path="sources/base">
> 	<copyfile dest="README" src="README"/>
> 	<copyfile dest="setup-environment" src="setup-environment"/>
>   </project>
> 
> 
>   <project remote="freescale"
> revision="4be3db25959e741ac9164a32c82b8c09b59213ad" upstream="morty"
> name="meta-freescale-3rdparty" path="sources/meta-freescale-
> 3rdparty"/>
>   <project remote="freescale"
> revision="dd482aeb7913c099c69f82362776bd92b584553c" upstream="morty"
> name="meta-freescale-distro" path="sources/meta-freescale-distro"/>
>   <project remote="QT5"
> revision="3601fd2c5306ac6d5d0d536e0be8cbb90da9b4c1" upstream="morty"
> name="meta-qt5" path="sources/meta-qt5"/>
>   <project remote="OSSystems"
> revision="7d37ae2adec5cc5ea27367ce125bc7bce2310ca1" upstream="morty"
> name="meta-browser" path="sources/meta-browser"/>
>   <project remote="variscite"
> revision="7a214e7579869be810aa8992c59627ba064db076" upstream="morty"
> name="meta-variscite-fslc" path="sources/meta-variscite-fslc"  />
> 
> 
> </manifest>
> 
> 
> 
> So, AFAIK, I need to either (a) translate these into the 'repos'
> property in my buildset, or write a custom buildstep that can use the
> 'repo' tool to clone the needed repositories.

You are correct.

The other step you might have issues with is the setup environment
script, yocto-autobuilder currently expects to source oe-init-build-env 
and doesn't support other init scripts presently.

I have a WIP patch to support alternative environment scripts, but it
doesn't fully resolve support for your BSP as it seems from the example
above that MACHINE and DISTRO need to be set in the environment when
the script is sourced.
Will those values be picked up from auto.conf if not set in the
environment?

Joshua

[-- Attachment #2: Type: text/html, Size: 8360 bytes --]