[Buildroot] Tesla is using Buildroot

[Buildroot] Tesla is using Buildroot

[Thomas Petazzoni]

Hello,

I met a few engineers from Tesla at the Embedded Linux Conference in
March, who told me they are using Buildroot. Now that their tree is
publicly available online, I can share this information.

Their Buildroot tree is at:

  https://github.com/teslamotors/buildroot

Unfortunately, it looks a bit ugly in terms of commit history: just
a few huge comments that mix tons of changes together. I was told the
autopilot configurations are there for now, but infotainment
configurations should be added in the near future.

It's of course very nice to see Buildroot being used on board of those
vehicles!

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] Tesla is using Buildroot

[anisse at astier.eu]

Hi,

On Fri, May 11, 2018 at 05:23:14PM +0200, Thomas Petazzoni wrote:
> Hello,
> 
> I met a few engineers from Tesla at the Embedded Linux Conference in
> March, who told me they are using Buildroot. Now that their tree is
> publicly available online, I can share this information.
> 
> Their Buildroot tree is at:
> 
>   https://github.com/teslamotors/buildroot
> 
> Unfortunately, it looks a bit ugly in terms of commit history: just
> a few huge comments that mix tons of changes together. I was told the
> autopilot configurations are there for now, but infotainment
> configurations should be added in the near future.
> 
> It's of course very nice to see Buildroot being used on board of those
> vehicles!
> 
> Best regards,
> 
> Thomas
> -- 
> Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
> Embedded Linux and Kernel engineering
> https://bootlin.com


I've had a quick look at what's inside. Here is what I found:
 - it seems based on buildroot 2016.05, with backports from more recent
   versions; but at its core it's still a 2016.05
 - there are a few packages tesla-{findutils, grep, bash, gzip, which, rsync}
   that are here with old versions to work around GPLv3
 - there's a tesla-verity package which seems to be a custom init that
   checks the validity of the verity metadata and interacts with
   firmware to check soc lock status before calling dmsetup.
 - there are a few packages that look like backports: python-dateutil,
   nanomsg, python-pytz, python-jsonschema
 - tesla-binutils is a "real" host binutils (non-cross)
 - there's tesla-libsystemd stub that builds a libsystemd with stubbed
   functions
 - it has its own parallel building infrastructure, using the loglinear
   tool, first introduced in google fiber's buildroot implementation
   https://gfiber.googlesource.com/buildroot/
 - many packages are patched to modify behaviour, customize build
   options: business as usual
 - probably many things I've missed


I've added Olof in cc. 

Regards,

Anisse

[Buildroot] Tesla is using Buildroot

[ratbert90]

This is pretty neat! The main website should really have a prominent list of companies that use Buildroot.

Google/Tesla/GoPro etc etc. It would be good advertisement!

Adam

________________________________
From: buildroot <buildroot-bounces@busybox.net> on behalf of anisse at astier.eu <anisse@astier.eu>
Sent: Friday, May 11, 2018 5:55:08 PM
To: Thomas Petazzoni
Cc: Olof Johansson; buildroot at uclibc.org
Subject: Re: [Buildroot] Tesla is using Buildroot

Hi,

On Fri, May 11, 2018 at 05:23:14PM +0200, Thomas Petazzoni wrote:
> Hello,
>
> I met a few engineers from Tesla at the Embedded Linux Conference in
> March, who told me they are using Buildroot. Now that their tree is
> publicly available online, I can share this information.
>
> Their Buildroot tree is at:
>
>   https://github.com/teslamotors/buildroot
>
> Unfortunately, it looks a bit ugly in terms of commit history: just
> a few huge comments that mix tons of changes together. I was told the
> autopilot configurations are there for now, but infotainment
> configurations should be added in the near future.
>
> It's of course very nice to see Buildroot being used on board of those
> vehicles!
>
> Best regards,
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
> Embedded Linux and Kernel engineering
> https://bootlin.com


I've had a quick look at what's inside. Here is what I found:
 - it seems based on buildroot 2016.05, with backports from more recent
   versions; but at its core it's still a 2016.05
 - there are a few packages tesla-{findutils, grep, bash, gzip, which, rsync}
   that are here with old versions to work around GPLv3
 - there's a tesla-verity package which seems to be a custom init that
   checks the validity of the verity metadata and interacts with
   firmware to check soc lock status before calling dmsetup.
 - there are a few packages that look like backports: python-dateutil,
   nanomsg, python-pytz, python-jsonschema
 - tesla-binutils is a "real" host binutils (non-cross)
 - there's tesla-libsystemd stub that builds a libsystemd with stubbed
   functions
 - it has its own parallel building infrastructure, using the loglinear
   tool, first introduced in google fiber's buildroot implementation
   https://gfiber.googlesource.com/buildroot/
 - many packages are patched to modify behaviour, customize build
   options: business as usual
 - probably many things I've missed


I've added Olof in cc.

Regards,

Anisse
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180512/83146a35/attachment.html>

[Buildroot] Tesla is using Buildroot

[Carlos Santos]

> From: "ratbert90" <aduskett@gmail.com>
> To: anisse at astier.eu, "Thomas Petazzoni" <thomas.petazzoni@bootlin.com>
> Cc: "Olof Johansson" <olof@lixom.net>, buildroot at uclibc.org
> Sent: Friday, May 11, 2018 10:22:49 PM
> Subject: Re: [Buildroot] Tesla is using Buildroot

> This is pretty neat! The main website should really have a prominent list of
> companies that use Buildroot.

> Google/Tesla/GoPro etc etc. It would be good advertisement!

> Adam

> From: buildroot <buildroot-bounces@busybox.net> on behalf of anisse at astier.eu
> <anisse@astier.eu>
> Sent: Friday, May 11, 2018 5:55:08 PM
> To: Thomas Petazzoni
> Cc: Olof Johansson; buildroot at uclibc.org
> Subject: Re: [Buildroot] Tesla is using Buildroot
> Hi,

> On Fri, May 11, 2018 at 05:23:14PM +0200, Thomas Petazzoni wrote:
> > Hello,

> > I met a few engineers from Tesla at the Embedded Linux Conference in
> > March, who told me they are using Buildroot. Now that their tree is
> > publicly available online, I can share this information.

> > Their Buildroot tree is at:

>> [ https://github.com/teslamotors/buildroot |
> > https://github.com/teslamotors/buildroot ]

> > Unfortunately, it looks a bit ugly in terms of commit history: just
> > a few huge comments that mix tons of changes together. I was told the
> > autopilot configurations are there for now, but infotainment
> > configurations should be added in the near future.

> > It's of course very nice to see Buildroot being used on board of those
> > vehicles!

> > Best regards,

> > Thomas
> > --
> > Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
> > Embedded Linux and Kernel engineering
> > [ https://bootlin.com/ | https://bootlin.com ]

> I've had a quick look at what's inside. Here is what I found:
> - it seems based on buildroot 2016.05, with backports from more recent
> versions; but at its core it's still a 2016.05
> - there are a few packages tesla-{findutils, grep, bash, gzip, which, rsync}
> that are here with old versions to work around GPLv3

... which highlights the need for some mechanism to blacklist licenses
and warn the user in the configuration menu that a package cannot be
selected because of license restrictions.

> - there's a tesla-verity package which seems to be a custom init that
> checks the validity of the verity metadata and interacts with
> firmware to check soc lock status before calling dmsetup.
> - there are a few packages that look like backports: python-dateutil,
> nanomsg, python-pytz, python-jsonschema
> - tesla-binutils is a "real" host binutils (non-cross)
> - there's tesla-libsystemd stub that builds a libsystemd with stubbed
> functions

Makes me wonder why they don't use a BR2_EXTERNAL.

> - it has its own parallel building infrastructure, using the loglinear
> tool, first introduced in google fiber's buildroot implementation
> [ https://gfiber.googlesource.com/buildroot/ |
> https://gfiber.googlesource.com/buildroot/ ]
> - many packages are patched to modify behaviour, customize build
> options: business as usual
> - probably many things I've missed

> I've added Olof in cc.

> Regards,

> Anisse

-- 
Carlos Santos (Casantos) - DATACOM, P&D
?Marched towards the enemy, spear upright, armed with the certainty
that only the ignorant can have.? ? Epitaph of a volunteer

[Buildroot] Tesla is using Buildroot

[Adrian Perez de Castro]

On Fri, 11 May 2018 22:42:45 -0300 (BRT), Carlos Santos <casantos@datacom.ind.br> wrote:
> > From: "ratbert90" <aduskett@gmail.com>
> > To: anisse at astier.eu, "Thomas Petazzoni" <thomas.petazzoni@bootlin.com>
> > Cc: "Olof Johansson" <olof@lixom.net>, buildroot at uclibc.org
> > Sent: Friday, May 11, 2018 10:22:49 PM
> > Subject: Re: [Buildroot] Tesla is using Buildroot
> >
> > [...]
> >
> > - there's a tesla-verity package which seems to be a custom init that
> > checks the validity of the verity metadata and interacts with
> > firmware to check soc lock status before calling dmsetup.
> > - there are a few packages that look like backports: python-dateutil,
> > nanomsg, python-pytz, python-jsonschema
> > - tesla-binutils is a "real" host binutils (non-cross)
> > - there's tesla-libsystemd stub that builds a libsystemd with stubbed
> > functions
> 
> Makes me wonder why they don't use a BR2_EXTERNAL.

Probably because (AFAIK) it is not yet possible to override base package in
a BR2_EXTERNAL. I have tripped on this myself a couple of times, and ended
up providing a top-level Makefile in the repo for my BR2_EXTERNAL which
downloads a certain version of the Buildroot tarball and applies a couple
of patches over it, then proceeds to chain up to the Buildroot Makefiles
passing the path BR2_EXTERNAL path down to them (so from the point of view
of somebody building, they just download the BR2_EXTERNAL repo and do ?make
foo_defconfig && make?).

It would be great to allow overriding base packages in a BR2_EXTERNAL ?

Cheers,


--
 Adri?n ?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180512/cfb2c8e2/attachment.asc>

[Buildroot] Tesla is using Buildroot

[Adam Duskett]

On Sat, May 12, 2018 at 9:28 AM Adrian Perez de Castro <aperez@igalia.com>
wrote:

> On Fri, 11 May 2018 22:42:45 -0300 (BRT), Carlos Santos <
> casantos at datacom.ind.br> wrote:
> > > From: "ratbert90" <aduskett@gmail.com>
> > > To: anisse at astier.eu, "Thomas Petazzoni" <thomas.petazzoni@bootlin.com
> >
> > > Cc: "Olof Johansson" <olof@lixom.net>, buildroot at uclibc.org
> > > Sent: Friday, May 11, 2018 10:22:49 PM
> > > Subject: Re: [Buildroot] Tesla is using Buildroot
> > >
> > > [...]
> > >
> > > - there's a tesla-verity package which seems to be a custom init that
> > > checks the validity of the verity metadata and interacts with
> > > firmware to check soc lock status before calling dmsetup.
> > > - there are a few packages that look like backports: python-dateutil,
> > > nanomsg, python-pytz, python-jsonschema
> > > - tesla-binutils is a "real" host binutils (non-cross)
> > > - there's tesla-libsystemd stub that builds a libsystemd with stubbed
> > > functions
> >
> > Makes me wonder why they don't use a BR2_EXTERNAL.
>
> Probably because (AFAIK) it is not yet possible to override base package in
> a BR2_EXTERNAL. I have tripped on this myself a couple of times, and ended
> up providing a top-level Makefile in the repo for my BR2_EXTERNAL which
> downloads a certain version of the Buildroot tarball and applies a couple
> of patches over it, then proceeds to chain up to the Buildroot Makefiles
> passing the path BR2_EXTERNAL path down to them (so from the point of view
> of somebody building, they just download the BR2_EXTERNAL repo and do ?make
> foo_defconfig && make?).
>
> It would be great to allow overriding base packages in a BR2_EXTERNAL ?
>
> Cheers,
>
>
> --
>  Adri?n ?
>
Interesting. My solution is to have a base stock BuildRoot folder and then
seperate company folder submodule.
In that submodule is a package directory and a patch directory. Any
packages I want to overwrite I just move to
the company/packages folder and then remove the base package from the base
package directory.

It's worked great for years that way. With the added benefit that when a
new BuildRoot is released, it's as easy as
removing everything but the company folder, dumping the new buildroot into
the base directory, and reapplying the
patches.

Adam


> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180512/b665d91a/attachment.html>

[Buildroot] Tesla is using Buildroot

[Joseph Kogut]

On Sat, May 12, 2018 at 6:27 AM, Adrian Perez de Castro
<aperez@igalia.com> wrote:
> On Fri, 11 May 2018 22:42:45 -0300 (BRT), Carlos Santos <casantos@datacom.ind.br> wrote:
>> > From: "ratbert90" <aduskett@gmail.com>
>> > To: anisse at astier.eu, "Thomas Petazzoni" <thomas.petazzoni@bootlin.com>
>> > Cc: "Olof Johansson" <olof@lixom.net>, buildroot at uclibc.org
>> > Sent: Friday, May 11, 2018 10:22:49 PM
>> > Subject: Re: [Buildroot] Tesla is using Buildroot
>> >
>> > [...]
>> >
>> > - there's a tesla-verity package which seems to be a custom init that
>> > checks the validity of the verity metadata and interacts with
>> > firmware to check soc lock status before calling dmsetup.
>> > - there are a few packages that look like backports: python-dateutil,
>> > nanomsg, python-pytz, python-jsonschema
>> > - tesla-binutils is a "real" host binutils (non-cross)
>> > - there's tesla-libsystemd stub that builds a libsystemd with stubbed
>> > functions
>>
>> Makes me wonder why they don't use a BR2_EXTERNAL.
>
> Probably because (AFAIK) it is not yet possible to override base package in
> a BR2_EXTERNAL. I have tripped on this myself a couple of times, and ended
> up providing a top-level Makefile in the repo for my BR2_EXTERNAL which
> downloads a certain version of the Buildroot tarball and applies a couple
> of patches over it, then proceeds to chain up to the Buildroot Makefiles
> passing the path BR2_EXTERNAL path down to them (so from the point of view
> of somebody building, they just download the BR2_EXTERNAL repo and do ?make
> foo_defconfig && make?).

I do the same thing. The top level Makefile exports BR2_EXTERNAL="..",
there's a target for cloning and checking out a specific Buildroot
revision, then there's a wildcard pattern at the end to pass any
unrecognized targets up to Buildroot's Makefile, so things like
"linux-menuconfig" still work.

It keeps my project repos nicely separated from upstream, and makes it
easy to pull in upstream changes.

I also have a few custom targets for taking the generated filesystem
images, and packing them up with an installer and manifest.

It would be nice if the manual covered some of these approaches,
because when I first started using Buildroot, it wasn't immediately
apparent how to do what I needed. Some examples would go a long way in
that regard, I might see what I can do about that later.

>
> It would be great to allow overriding base packages in a BR2_EXTERNAL ?
>
> Cheers,
>
>
> --
>  Adri?n ?
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
>

[Buildroot] Tesla is using Buildroot

[Olof Johansson]

On Sat, May 12, 2018 at 10:06 AM, Joseph Kogut <joseph.kogut@gmail.com> wrote:
> On Sat, May 12, 2018 at 6:27 AM, Adrian Perez de Castro
> <aperez@igalia.com> wrote:
>> On Fri, 11 May 2018 22:42:45 -0300 (BRT), Carlos Santos <casantos@datacom.ind.br> wrote:
>>> > From: "ratbert90" <aduskett@gmail.com>
>>> > To: anisse at astier.eu, "Thomas Petazzoni" <thomas.petazzoni@bootlin.com>
>>> > Cc: "Olof Johansson" <olof@lixom.net>, buildroot at uclibc.org
>>> > Sent: Friday, May 11, 2018 10:22:49 PM
>>> > Subject: Re: [Buildroot] Tesla is using Buildroot
>>> >
>>> > [...]
>>> >
>>> > - there's a tesla-verity package which seems to be a custom init that
>>> > checks the validity of the verity metadata and interacts with
>>> > firmware to check soc lock status before calling dmsetup.
>>> > - there are a few packages that look like backports: python-dateutil,
>>> > nanomsg, python-pytz, python-jsonschema
>>> > - tesla-binutils is a "real" host binutils (non-cross)
>>> > - there's tesla-libsystemd stub that builds a libsystemd with stubbed
>>> > functions
>>>
>>> Makes me wonder why they don't use a BR2_EXTERNAL.
>>
>> Probably because (AFAIK) it is not yet possible to override base package in
>> a BR2_EXTERNAL. I have tripped on this myself a couple of times, and ended
>> up providing a top-level Makefile in the repo for my BR2_EXTERNAL which
>> downloads a certain version of the Buildroot tarball and applies a couple
>> of patches over it, then proceeds to chain up to the Buildroot Makefiles
>> passing the path BR2_EXTERNAL path down to them (so from the point of view
>> of somebody building, they just download the BR2_EXTERNAL repo and do ?make
>> foo_defconfig && make?).
>
> I do the same thing. The top level Makefile exports BR2_EXTERNAL="..",
> there's a target for cloning and checking out a specific Buildroot
> revision, then there's a wildcard pattern at the end to pass any
> unrecognized targets up to Buildroot's Makefile, so things like
> "linux-menuconfig" still work.
>
> It keeps my project repos nicely separated from upstream, and makes it
> easy to pull in upstream changes.
>
> I also have a few custom targets for taking the generated filesystem
> images, and packing them up with an installer and manifest.
>
> It would be nice if the manual covered some of these approaches,
> because when I first started using Buildroot, it wasn't immediately
> apparent how to do what I needed. Some examples would go a long way in
> that regard, I might see what I can do about that later.

I liked the split of having third party software in an upstream-based
separate buildroot repo, and only proprietary components in
BR2_EXTERNAL. That way it's easier to separate out the portion you
need to share for compliance (i.e. see current github contents), while
having a place to keep confidential material, work on new
configs/products/prototypes/internal uses that are not yet released,
etc in the BR2_EXTERNAL. As expressed above, wrapping it with a second
layer of makefiles makes it relatively easy to deal with.

Removing the upstream package and adding it locally really is no
different from replacing the contents in the repo. A rebase would do
the same.


-Olof

[Buildroot] Tesla is using Buildroot

[Olof Johansson]

On Fri, May 11, 2018 at 6:42 PM, Carlos Santos <casantos@datacom.ind.br> wrote:
>> From: "ratbert90" <aduskett@gmail.com>
>> To: anisse at astier.eu, "Thomas Petazzoni" <thomas.petazzoni@bootlin.com>
>> Cc: "Olof Johansson" <olof@lixom.net>, buildroot at uclibc.org
>> Sent: Friday, May 11, 2018 10:22:49 PM
>> Subject: Re: [Buildroot] Tesla is using Buildroot
>
>> This is pretty neat! The main website should really have a prominent list of
>> companies that use Buildroot.
>
>> Google/Tesla/GoPro etc etc. It would be good advertisement!
>
>> Adam
>
>> From: buildroot <buildroot-bounces@busybox.net> on behalf of anisse at astier.eu
>> <anisse@astier.eu>
>> Sent: Friday, May 11, 2018 5:55:08 PM
>> To: Thomas Petazzoni
>> Cc: Olof Johansson; buildroot at uclibc.org
>> Subject: Re: [Buildroot] Tesla is using Buildroot
>> Hi,
>
>> On Fri, May 11, 2018 at 05:23:14PM +0200, Thomas Petazzoni wrote:
>> > Hello,
>
>> > I met a few engineers from Tesla at the Embedded Linux Conference in
>> > March, who told me they are using Buildroot. Now that their tree is
>> > publicly available online, I can share this information.
>
>> > Their Buildroot tree is at:
>
>>> [ https://github.com/teslamotors/buildroot |
>> > https://github.com/teslamotors/buildroot ]
>
>> > Unfortunately, it looks a bit ugly in terms of commit history: just
>> > a few huge comments that mix tons of changes together. I was told the
>> > autopilot configurations are there for now, but infotainment
>> > configurations should be added in the near future.
>
>> > It's of course very nice to see Buildroot being used on board of those
>> > vehicles!
>
>> > Best regards,
>
>> > Thomas
>> > --
>> > Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
>> > Embedded Linux and Kernel engineering
>> > [ https://bootlin.com/ | https://bootlin.com ]
>
>> I've had a quick look at what's inside. Here is what I found:
>> - it seems based on buildroot 2016.05, with backports from more recent
>> versions; but at its core it's still a 2016.05
>> - there are a few packages tesla-{findutils, grep, bash, gzip, which, rsync}
>> that are here with old versions to work around GPLv3
>
> ... which highlights the need for some mechanism to blacklist licenses
> and warn the user in the configuration menu that a package cannot be
> selected because of license restrictions.

I think Kconfig could solve this quite nicely. Make a selection menu
such that you can choose what licenses you're willing to accept, and
then make the respective packages depend on the license config
(depends on LICENSE_GPLV3 etc)

>
>> - there's a tesla-verity package which seems to be a custom init that
>> checks the validity of the verity metadata and interacts with
>> firmware to check soc lock status before calling dmsetup.
>> - there are a few packages that look like backports: python-dateutil,
>> nanomsg, python-pytz, python-jsonschema
>> - tesla-binutils is a "real" host binutils (non-cross)
>> - there's tesla-libsystemd stub that builds a libsystemd with stubbed
>> functions
>
> Makes me wonder why they don't use a BR2_EXTERNAL.

BR2_EXTERNAL is used but by splitting the contents between the
buildroot repo and external, and keeping only things you don't want to
share in the external one (proprietary apps, experimental work, random
internal stuff that won't ship to the outside world and work on
unreleased stuff) works well. The way the (separate but corresponding)
configuration that's published is generated, the reference to
BR2_EXTERNAL doesn't show up.

>> - it has its own parallel building infrastructure, using the loglinear
>> tool, first introduced in google fiber's buildroot implementation
>> [ https://gfiber.googlesource.com/buildroot/ |
>> https://gfiber.googlesource.com/buildroot/ ]
>> - many packages are patched to modify behaviour, customize build
>> options: business as usual
>> - probably many things I've missed

One thing that I'm not sure has been used all that much elsewhere is
that there's a recursive invocation to generate the initramfs, and
then build that into the kernel. Could maybe have been done as two
separate toplevel builds with postprocessing, but this worked quite
well.



-Olof

[Buildroot] Tesla is using Buildroot

[Trent Piepho]

On Sat, 2018-05-12 at 10:51 -0700, Olof Johansson wrote:
> On Sat, May 12, 2018 at 10:06 AM, Joseph Kogut <joseph.kogut@gmail.com> wrote:
> > On Sat, May 12, 2018 at 6:27 AM, Adrian Perez de Castro
> > <aperez@igalia.com> wrote:
> > > On Fri, 11 May 2018 22:42:45 -0300 (BRT), Carlos Santos <casantos@datacom.ind.br> wrote:
> > > > > From: "ratbert90" <aduskett@gmail.com>
> > > > > To: anisse at astier.eu, "Thomas Petazzoni" <thomas.petazzoni@bootlin.com>
> > > > > 
> > > > 
> > > > Makes me wonder why they don't use a BR2_EXTERNAL.
> > > 
> > > Probably because (AFAIK) it is not yet possible to override base package in
> > > a BR2_EXTERNAL. I have tripped on this myself a couple of times, and ended
> > > up providing a top-level Makefile in the repo for my BR2_EXTERNAL which
> > > downloads a certain version of the Buildroot tarball and applies a couple
> > > of patches over it, then proceeds to chain up to the Buildroot Makefiles
> > > passing the path BR2_EXTERNAL path down to them (so from the point of view
> > > of somebody building, they just download the BR2_EXTERNAL repo and do ?make
> > > foo_defconfig && make?).

Since the external mk file is included after all of buildroot's
internal packages and infrastructure files, it's possible to re-define
variables from buildroot packages in external.  In some case this could
can be a way to override or patch a buildroot package via BR2_EXTERNAL
instead of as a patch to buildroot.

> > I do the same thing. The top level Makefile exports BR2_EXTERNAL="..",
> > there's a target for cloning and checking out a specific Buildroot
> > revision, then there's a wildcard pattern at the end to pass any
> > unrecognized targets up to Buildroot's Makefile, so things like
> > "linux-menuconfig" still work.

> I liked the split of having third party software in an upstream-based
> separate buildroot repo, and only proprietary components in
> BR2_EXTERNAL. That way it's easier to separate out the portion you
> need to share for compliance (i.e. see current github contents), while
> having a place to keep confidential material, work on new
> configs/products/prototypes/internal uses that are not yet released,
> etc in the BR2_EXTERNAL. As expressed above, wrapping it with a second
> layer of makefiles makes it relatively easy to deal with.

We have a top-level project with a top level Makefile that contains the
 BR2_EXTERNAL tree.  I.e., the BR2_EXTERNAL tree is the top level
project.  Buildroot exists as a git submodule in a directory of this
project.  We can update buildroot by pulling from upstream and rebasing
and can git format-patch a patch to send to the list.

We patch buildroot if:
1. We think the patch is upstreamable.
2. There appears to be no reasonable way to accomplish what we want
otherwise.

The top level makefile uses a pattern rule to provide a target for
every defconfig that exists in br2-external/configs directory.  It'll
call buildroot's build with BR2_EXTERNAL and O set build that defconfig
into an output directory.  Or just do a buildroot *_defconfig call but
not actually build.

buildroot sets itself up so that once you go to the output directory,
there is a Makefile that will execute any buildroot target (menuconfig,
pkg-rebuild, all, etc.) with BR2_EXTERNAL configured.

[Buildroot] Tesla is using Buildroot

[Arnout Vandecappelle]

On 14-05-18 20:00, Trent Piepho wrote:
> On Sat, 2018-05-12 at 10:51 -0700, Olof Johansson wrote:
>> On Sat, May 12, 2018 at 10:06 AM, Joseph Kogut <joseph.kogut@gmail.com> wrote:
>>> On Sat, May 12, 2018 at 6:27 AM, Adrian Perez de Castro <aperez@igalia.com> wrote:
[snip]
>>> I do the same thing. The top level Makefile exports BR2_EXTERNAL="..",
>>> there's a target for cloning and checking out a specific Buildroot
>>> revision, then there's a wildcard pattern at the end to pass any
>>> unrecognized targets up to Buildroot's Makefile, so things like
>>> "linux-menuconfig" still work.
> 
>> I liked the split of having third party software in an upstream-based
>> separate buildroot repo, and only proprietary components in
>> BR2_EXTERNAL. That way it's easier to separate out the portion you
>> need to share for compliance (i.e. see current github contents), while
>> having a place to keep confidential material, work on new
>> configs/products/prototypes/internal uses that are not yet released,
>> etc in the BR2_EXTERNAL. As expressed above, wrapping it with a second
>> layer of makefiles makes it relatively easy to deal with.
> 
> We have a top-level project with a top level Makefile that contains the
>  BR2_EXTERNAL tree.  I.e., the BR2_EXTERNAL tree is the top level
> project.  Buildroot exists as a git submodule in a directory of this
> project.  We can update buildroot by pulling from upstream and rebasing
> and can git format-patch a patch to send to the list.
> 
> We patch buildroot if:
> 1. We think the patch is upstreamable.
> 2. There appears to be no reasonable way to accomplish what we want
> otherwise.
> 
> The top level makefile uses a pattern rule to provide a target for
> every defconfig that exists in br2-external/configs directory.  It'll
> call buildroot's build with BR2_EXTERNAL and O set build that defconfig
> into an output directory.  Or just do a buildroot *_defconfig call but
> not actually build.
> 
> buildroot sets itself up so that once you go to the output directory,
> there is a Makefile that will execute any buildroot target (menuconfig,
> pkg-rebuild, all, etc.) with BR2_EXTERNAL configured.

 That's *exactly* the setup I've developed for one of my projects. I've been
meaning to export this to a buildroot-external superproject that people could
reuse...

 One thing though: I recently switched from git submodule to git subtree.
Submodules are really painful to work with when there are multiple developers
who need to change both the supermodule and submodule. I haven't yet gotten
around to using the subtree split functionality to export the upstreamable
changes again, but it looks pretty convenient to use.

 Regards,
 Arnout
-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] Tesla is using Buildroot

[Angelo Compagnucci]

2018-05-15 22:18 GMT+02:00 Arnout Vandecappelle <arnout@mind.be>:
>
>
> On 14-05-18 20:00, Trent Piepho wrote:
>> On Sat, 2018-05-12 at 10:51 -0700, Olof Johansson wrote:
>>> On Sat, May 12, 2018 at 10:06 AM, Joseph Kogut <joseph.kogut@gmail.com> wrote:
>>>> On Sat, May 12, 2018 at 6:27 AM, Adrian Perez de Castro <aperez@igalia.com> wrote:
> [snip]
>>>> I do the same thing. The top level Makefile exports BR2_EXTERNAL="..",
>>>> there's a target for cloning and checking out a specific Buildroot
>>>> revision, then there's a wildcard pattern at the end to pass any
>>>> unrecognized targets up to Buildroot's Makefile, so things like
>>>> "linux-menuconfig" still work.
>>
>>> I liked the split of having third party software in an upstream-based
>>> separate buildroot repo, and only proprietary components in
>>> BR2_EXTERNAL. That way it's easier to separate out the portion you
>>> need to share for compliance (i.e. see current github contents), while
>>> having a place to keep confidential material, work on new
>>> configs/products/prototypes/internal uses that are not yet released,
>>> etc in the BR2_EXTERNAL. As expressed above, wrapping it with a second
>>> layer of makefiles makes it relatively easy to deal with.
>>
>> We have a top-level project with a top level Makefile that contains the
>>  BR2_EXTERNAL tree.  I.e., the BR2_EXTERNAL tree is the top level
>> project.  Buildroot exists as a git submodule in a directory of this
>> project.  We can update buildroot by pulling from upstream and rebasing
>> and can git format-patch a patch to send to the list.
>>
>> We patch buildroot if:
>> 1. We think the patch is upstreamable.
>> 2. There appears to be no reasonable way to accomplish what we want
>> otherwise.
>>
>> The top level makefile uses a pattern rule to provide a target for
>> every defconfig that exists in br2-external/configs directory.  It'll
>> call buildroot's build with BR2_EXTERNAL and O set build that defconfig
>> into an output directory.  Or just do a buildroot *_defconfig call but
>> not actually build.
>>
>> buildroot sets itself up so that once you go to the output directory,
>> there is a Makefile that will execute any buildroot target (menuconfig,
>> pkg-rebuild, all, etc.) with BR2_EXTERNAL configured.
>
>  That's *exactly* the setup I've developed for one of my projects. I've been
> meaning to export this to a buildroot-external superproject that people could
> reuse...

Please do! I'd really would like to have a "best practice" or a sort
of boilerplate to use when setting up a new project!

Thanks!

>
>  One thing though: I recently switched from git submodule to git subtree.
> Submodules are really painful to work with when there are multiple developers
> who need to change both the supermodule and submodule. I haven't yet gotten
> around to using the subtree split functionality to export the upstreamable
> changes again, but it looks pretty convenient to use.
>
>  Regards,
>  Arnout
> --
> Arnout Vandecappelle                          arnout at mind be
> Senior Embedded Software Architect            +32-16-286500
> Essensium/Mind                                http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot



-- 
Profile: http://it.linkedin.com/in/compagnucciangelo

[Buildroot] Tesla is using Buildroot

[Andreas Naumann]

Am 19.05.2018 um 13:08 schrieb Angelo Compagnucci:
> 2018-05-15 22:18 GMT+02:00 Arnout Vandecappelle <arnout@mind.be>:
>> On 14-05-18 20:00, Trent Piepho wrote:
>>> On Sat, 2018-05-12 at 10:51 -0700, Olof Johansson wrote:
>>>> On Sat, May 12, 2018 at 10:06 AM, Joseph Kogut <joseph.kogut@gmail.com> wrote:
>>>>> On Sat, May 12, 2018 at 6:27 AM, Adrian Perez de Castro <aperez@igalia.com> wrote:
>> [snip]
>>>>> I do the same thing. The top level Makefile exports BR2_EXTERNAL="..",
>>>>> there's a target for cloning and checking out a specific Buildroot
>>>>> revision, then there's a wildcard pattern at the end to pass any
>>>>> unrecognized targets up to Buildroot's Makefile, so things like
>>>>> "linux-menuconfig" still work.
>>>
>>>> I liked the split of having third party software in an upstream-based
>>>> separate buildroot repo, and only proprietary components in
>>>> BR2_EXTERNAL. That way it's easier to separate out the portion you
>>>> need to share for compliance (i.e. see current github contents), while
>>>> having a place to keep confidential material, work on new
>>>> configs/products/prototypes/internal uses that are not yet released,
>>>> etc in the BR2_EXTERNAL. As expressed above, wrapping it with a second
>>>> layer of makefiles makes it relatively easy to deal with.
>>>
>>> We have a top-level project with a top level Makefile that contains the
>>>   BR2_EXTERNAL tree.  I.e., the BR2_EXTERNAL tree is the top level
>>> project.  Buildroot exists as a git submodule in a directory of this
>>> project.  We can update buildroot by pulling from upstream and rebasing
>>> and can git format-patch a patch to send to the list.
>>>
>>> We patch buildroot if:
>>> 1. We think the patch is upstreamable.
>>> 2. There appears to be no reasonable way to accomplish what we want
>>> otherwise.
>>>
>>> The top level makefile uses a pattern rule to provide a target for
>>> every defconfig that exists in br2-external/configs directory.  It'll
>>> call buildroot's build with BR2_EXTERNAL and O set build that defconfig
>>> into an output directory.  Or just do a buildroot *_defconfig call but
>>> not actually build.
>>>
>>> buildroot sets itself up so that once you go to the output directory,
>>> there is a Makefile that will execute any buildroot target (menuconfig,
>>> pkg-rebuild, all, etc.) with BR2_EXTERNAL configured.
>>
>>   That's *exactly* the setup I've developed for one of my projects. I've been
>> meaning to export this to a buildroot-external superproject that people could
>> reuse...

I use a similar setup, but use the android repo-tool to checkout 
buildroot itself, additional BR2_EXTERNAL repos and 
<PACKAGE>_OVERRIDE_SRCDIR repos which are under development (e.g. main 
application).
The repo-manifest provides the option to copy files from one of the 
repos to toplevel, which I use (just like android) for the main Makefile.
This Makefile then points to our wrapper-Makefile which facilitates 
building multiple buildroot defconfigs, packing those into an update 
package, running legal-info and other steps and even trying to "save" an 
intermediate state of the output-dir just before target-finalize in 
order to reuse it for continuous application integration.
To keep things clean everything is built in out-of-tree folders, where 
Buildroot Make-targets can be called as usual.

We have chosen the repo-tool after being frustrated with git-submodules, 
however it's not without problems either, especially if you work in a 
team an try to avoid setting up a gerrit-instance.
So one of those day I'll have to look into git subtree...


regards,
Andreas


> 
> Please do! I'd really would like to have a "best practice" or a sort
> of boilerplate to use when setting up a new project!
> 
> Thanks!
> 
>>
>>   One thing though: I recently switched from git submodule to git subtree.
>> Submodules are really painful to work with when there are multiple developers
>> who need to change both the supermodule and submodule. I haven't yet gotten
>> around to using the subtree split functionality to export the upstreamable
>> changes again, but it looks pretty convenient to use.
>>
>>   Regards,
>>   Arnout
>> --
>> Arnout Vandecappelle                          arnout at mind be
>> Senior Embedded Software Architect            +32-16-286500
>> Essensium/Mind                                http://www.mind.be
>> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
>> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
>> GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
>> _______________________________________________
>> buildroot mailing list
>> buildroot at busybox.net
>> http://lists.busybox.net/mailman/listinfo/buildroot
> 
> 
> 

[Buildroot] Tesla is using Buildroot

[Trent Piepho]

On Tue, 2018-05-22 at 09:51 +0200, Andreas Naumann wrote:
> Am 19.05.2018 um 13:08 schrieb Angelo Compagnucci:
> > 
> > > > 
> > > > The top level makefile uses a pattern rule to provide a target for
> > > > every defconfig that exists in br2-external/configs directory.  It'll
> > > > call buildroot's build with BR2_EXTERNAL and O set build that defconfig
> > > > into an output directory.  Or just do a buildroot *_defconfig call but
> > > > not actually build.
> > > > 
> > > > buildroot sets itself up so that once you go to the output directory,
> > > > there is a Makefile that will execute any buildroot target (menuconfig,
> > > > pkg-rebuild, all, etc.) with BR2_EXTERNAL configured.
> > > 
> > >   That's *exactly* the setup I've developed for one of my projects. I've been
> > > meaning to export this to a buildroot-external superproject that people could
> > > reuse...
> 
> I use a similar setup, but use the android repo-tool to checkout 
> buildroot itself, additional BR2_EXTERNAL repos and 
> <PACKAGE>_OVERRIDE_SRCDIR repos which are under development (e.g. main 
> application).

I've used OVERRIDE_SRCDIR for the packages under development, which
were themselves git repositories that were submodules of the master
project.  Sounds like you've got the same thing but with repo-tool
instead of submodules.  There were a couple things that weren't that
good.

After a updating the source, by working on it or via git, buildroot
doesn't detect a change and rebuild it.  It is necessary to <pkg>-
rebuild it manually.  Tracking down which packages to rebuild was a
real pain for developers to do after every pull.  It seems like
buildroot could check srcdir timestamp vs build dir rsync stamp file
and do this automatically.

Another problem that the rsync of srcdir into the build dir will not
delete files that have been removed from srcdir.  If someone used a
wildcard in the makefile, e.g SRSCS := $(wildcard *.c), and then
renamed a source, the build fails because the build dir will have the
old and new source file and try to compile both of them.  This could be
avoided if buildroot didn't need to rsync the srcdir into the build
dir, but instead used the package's (assuming it has one) out-of-tree
build feature to compile directly from srcdir with output to the build
dir.