[U-Boot] SWUpdate - U-Boot environment library dependency

[U-Boot] SWUpdate - U-Boot environment library dependency

[Andreas Reichel]

Hi all,

as Stefano Babic was so friendly and pointed out a few things already,
we come the following problematic points:

For SWupdate to access U-Boot's environment, it uses code from U-Boot.
Before 2015, fw_env.c was copied - hence out of version control,
afterwards and since then, a lib.a produced by U-Boot has been used
and renamed to libubootenv.a to link against.

However, this approach creates several difficulties:

* Distros like Debian cannot provide a devel package for SWUpdate like
  u-boot-dev, since U-Boot has its default environment code compiled
  board-dependently and Debian needed it board-independent.
  If a board-independent libubootenv.a was provided without
  a specific default environment, the first update process would destroy
  the U-Boot environment if it had had bad CRC before.
  (Thanks Stefano for this good point).

* If we have a board with U-Boot already preinstalled and want to
  compile SWUpdate for it, we need the sources of this very U-Boot to
  get the propper lib. For example in a debian-like build system we had to
  compile U-Boot again, although it is already installed since we lack
  a dev package.

* U-Boot does not provide any default means to install a development
  library. Thus anything we modify on-top might break with the next
  version.

First proposal by Stefano and me would be to somehow split the default
environment from the library to have a board-independent component and
board specific data that is passed to U-Boot and SWUpdate somehow.

Goal is to factor out U-Boot environment support for other software like
SWUpdate and not patching and hacking like its the case with recipes as in
openembedded atm.

Thanks,
Andreas

-- 
Andreas Reichel
Dipl.-Phys. (Univ.)
Software Consultant

Andreas.Reichel at tngtech.com, +49-174-3180074
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring
Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller
Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082

[U-Boot] SWUpdate - U-Boot environment library dependency

[Simon Goldschmidt]

On 04.09.2018 12:30, Andreas Reichel wrote:
> Hi all,
>
> as Stefano Babic was so friendly and pointed out a few things already,
> we come the following problematic points:
>
> For SWupdate to access U-Boot's environment, it uses code from U-Boot.
> Before 2015, fw_env.c was copied - hence out of version control,
> afterwards and since then, a lib.a produced by U-Boot has been used
> and renamed to libubootenv.a to link against.
>
> However, this approach creates several difficulties:
>
> * Distros like Debian cannot provide a devel package for SWUpdate like
>    u-boot-dev, since U-Boot has its default environment code compiled
>    board-dependently and Debian needed it board-independent.
>    If a board-independent libubootenv.a was provided without
>    a specific default environment, the first update process would destroy
>    the U-Boot environment if it had had bad CRC before.
>    (Thanks Stefano for this good point).
>
> * If we have a board with U-Boot already preinstalled and want to
>    compile SWUpdate for it, we need the sources of this very U-Boot to
>    get the propper lib. For example in a debian-like build system we had to
>    compile U-Boot again, although it is already installed since we lack
>    a dev package.
>
> * U-Boot does not provide any default means to install a development
>    library. Thus anything we modify on-top might break with the next
>    version.
>
> First proposal by Stefano and me would be to somehow split the default
> environment from the library to have a board-independent component and
> board specific data that is passed to U-Boot and SWUpdate somehow.
>
> Goal is to factor out U-Boot environment support for other software like
> SWUpdate and not patching and hacking like its the case with recipes as in
> openembedded atm.

Has there been any progress in the SWupdate/fw_setenv integration? I 
thought I remember reading something about letting fw_setenv extract the 
environment from the U-Boot binary in the target, but I cannot find the 
thread. I do think the current situation should be improved, making 
fw_setenv independent of the target that is built.

And as this thread is on the swupdate group as well: I would prefer 
calling an external fw_setenv tool instead of linking in the static 
library libubootenv.a...

Simon

[U-Boot] SWUpdate - U-Boot environment library dependency

[Marek Vasut]

On 11/20/2018 10:11 PM, Simon Goldschmidt wrote:
> On 04.09.2018 12:30, Andreas Reichel wrote:
>> Hi all,
>>
>> as Stefano Babic was so friendly and pointed out a few things already,
>> we come the following problematic points:
>>
>> For SWupdate to access U-Boot's environment, it uses code from U-Boot.
>> Before 2015, fw_env.c was copied - hence out of version control,
>> afterwards and since then, a lib.a produced by U-Boot has been used
>> and renamed to libubootenv.a to link against.
>>
>> However, this approach creates several difficulties:
>>
>> * Distros like Debian cannot provide a devel package for SWUpdate like
>>    u-boot-dev, since U-Boot has its default environment code compiled
>>    board-dependently and Debian needed it board-independent.
>>    If a board-independent libubootenv.a was provided without
>>    a specific default environment, the first update process would destroy
>>    the U-Boot environment if it had had bad CRC before.
>>    (Thanks Stefano for this good point).
>>
>> * If we have a board with U-Boot already preinstalled and want to
>>    compile SWUpdate for it, we need the sources of this very U-Boot to
>>    get the propper lib. For example in a debian-like build system we
>> had to
>>    compile U-Boot again, although it is already installed since we lack
>>    a dev package.
>>
>> * U-Boot does not provide any default means to install a development
>>    library. Thus anything we modify on-top might break with the next
>>    version.
>>
>> First proposal by Stefano and me would be to somehow split the default
>> environment from the library to have a board-independent component and
>> board specific data that is passed to U-Boot and SWUpdate somehow.
>>
>> Goal is to factor out U-Boot environment support for other software like
>> SWUpdate and not patching and hacking like its the case with recipes
>> as in
>> openembedded atm.
> 
> Has there been any progress in the SWupdate/fw_setenv integration? I
> thought I remember reading something about letting fw_setenv extract the
> environment from the U-Boot binary in the target, but I cannot find the
> thread. I do think the current situation should be improved, making
> fw_setenv independent of the target that is built.
> 
> And as this thread is on the swupdate group as well: I would prefer
> calling an external fw_setenv tool instead of linking in the static
> library libubootenv.a...

Wouldn't it make sense to have U-Boot build the env code as LGPL
library, in addition to executable ?

-- 
Best regards,
Marek Vasut

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Stefano Babic]

Hi Marek,

On 21/11/18 00:10, Marek Vasut wrote:
> On 11/20/2018 10:11 PM, Simon Goldschmidt wrote:
>> On 04.09.2018 12:30, Andreas Reichel wrote:
>>> Hi all,
>>>
>>> as Stefano Babic was so friendly and pointed out a few things already,
>>> we come the following problematic points:
>>>
>>> For SWupdate to access U-Boot's environment, it uses code from U-Boot.
>>> Before 2015, fw_env.c was copied - hence out of version control,
>>> afterwards and since then, a lib.a produced by U-Boot has been used
>>> and renamed to libubootenv.a to link against.
>>>
>>> However, this approach creates several difficulties:
>>>
>>> * Distros like Debian cannot provide a devel package for SWUpdate like
>>>    u-boot-dev, since U-Boot has its default environment code compiled
>>>    board-dependently and Debian needed it board-independent.
>>>    If a board-independent libubootenv.a was provided without
>>>    a specific default environment, the first update process would destroy
>>>    the U-Boot environment if it had had bad CRC before.
>>>    (Thanks Stefano for this good point).
>>>
>>> * If we have a board with U-Boot already preinstalled and want to
>>>    compile SWUpdate for it, we need the sources of this very U-Boot to
>>>    get the propper lib. For example in a debian-like build system we
>>> had to
>>>    compile U-Boot again, although it is already installed since we lack
>>>    a dev package.
>>>
>>> * U-Boot does not provide any default means to install a development
>>>    library. Thus anything we modify on-top might break with the next
>>>    version.
>>>
>>> First proposal by Stefano and me would be to somehow split the default
>>> environment from the library to have a board-independent component and
>>> board specific data that is passed to U-Boot and SWUpdate somehow.
>>>
>>> Goal is to factor out U-Boot environment support for other software like
>>> SWUpdate and not patching and hacking like its the case with recipes
>>> as in
>>> openembedded atm.
>>
>> Has there been any progress in the SWupdate/fw_setenv integration? I
>> thought I remember reading something about letting fw_setenv extract the
>> environment from the U-Boot binary in the target, but I cannot find the
>> thread. I do think the current situation should be improved, making
>> fw_setenv independent of the target that is built.
>>
>> And as this thread is on the swupdate group as well: I would prefer
>> calling an external fw_setenv tool instead of linking in the static
>> library libubootenv.a...
> 
> Wouldn't it make sense to have U-Boot build the env code as LGPL
> library, in addition to executable ?

+1

The library is already built (tools/env/lib.a), but IMHO the best thing
is to export it official and let that in OE we have a
u-boot-fw-tools-dev with header and library.

I would like to see the library under LGPL instead of GPL2, too, and I
raised this issue when I started SWUpdate, but I was not very active to
promote this. Tom, Wolfgang, is there chances to switch license ?

A env library is very welcomed by many customers, because they could
integrate it in their application if license allows it.

Best regards,
Stefano Babic


-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Wolfgang Denk]

Dear Stefano,

In message <96836cc1-e4bb-a2a2-05ac-056053b4c426@denx.de> you wrote:
> 
> I would like to see the library under LGPL instead of GPL2, too, and I
> raised this issue when I started SWUpdate, but I was not very active to
> promote this. Tom, Wolfgang, is there chances to switch license ?

Relicensing requires permission from all who contributed to that
code.

Consider mine as granted.

But someone hat to invest the efforts to analyze the code so we
know who to ask, and then collect all the permissions...

> A env library is very welcomed by many customers, because they could
> integrate it in their application if license allows it.

Agreed.


Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
If you don't have time to do it right, when will you have time to  do
it over?                                                - John Wooden

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Marek Vasut]

On 11/21/2018 10:20 AM, Wolfgang Denk wrote:
> Dear Stefano,

Hi,

> In message <96836cc1-e4bb-a2a2-05ac-056053b4c426@denx.de> you wrote:
>>
>> I would like to see the library under LGPL instead of GPL2, too, and I
>> raised this issue when I started SWUpdate, but I was not very active to
>> promote this. Tom, Wolfgang, is there chances to switch license ?
> 
> Relicensing requires permission from all who contributed to that
> code.
> 
> Consider mine as granted.
> 
> But someone hat to invest the efforts to analyze the code so we
> know who to ask, and then collect all the permissions...
> 
>> A env library is very welcomed by many customers, because they could
>> integrate it in their application if license allows it.
> 
> Agreed.

Then again, U-Boot environment structure is trivial, crc, flags, data,
there is no complexity involved. There is probably some complexity in
the backing store access stuff (MTD, block devs, legacy stuff), but that
should either use some MTD utils libs, basic block access primitives or
be given a once-over and possibly be dropped.

I think prototyping a library from scratch that's LGPL would be a few
days' work and the benefit would be tremendous all over.

-- 
Best regards,
Marek Vasut

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Stefano Babic]

Hi Marek,

On 21/11/18 15:31, Marek Vasut wrote:
> On 11/21/2018 10:20 AM, Wolfgang Denk wrote:
>> Dear Stefano,
> 
> Hi,
> 
>> In message <96836cc1-e4bb-a2a2-05ac-056053b4c426@denx.de> you wrote:
>>>
>>> I would like to see the library under LGPL instead of GPL2, too, and I
>>> raised this issue when I started SWUpdate, but I was not very active to
>>> promote this. Tom, Wolfgang, is there chances to switch license ?
>>
>> Relicensing requires permission from all who contributed to that
>> code.
>>
>> Consider mine as granted.
>>
>> But someone hat to invest the efforts to analyze the code so we
>> know who to ask, and then collect all the permissions...
>>
>>> A env library is very welcomed by many customers, because they could
>>> integrate it in their application if license allows it.
>>
>> Agreed.
> 
> Then again, U-Boot environment structure is trivial, crc, flags, data,
> there is no complexity involved. There is probably some complexity in
> the backing store access stuff (MTD, block devs, legacy stuff), but that
> should either use some MTD utils libs, basic block access primitives or
> be given a once-over and possibly be dropped.
> 
> I think prototyping a library from scratch that's LGPL would be a few
> days' work and the benefit would be tremendous all over.


I confess I had the same idea - why not ignore the code in tools/env
(they have also some drawbacks, see the locking mechanism in my previous
e-mail) and start with a new library from scratch ? Then LGPL is not an
issue anymore, it is a new development. And I already took this way for
"grubenv" (I had to, grubenv is not license compatible).

But something in my head is telling me that this is not else as a fork
of u-boot (ok, a partial fork, just tools/env). And if the U-Boot
community decides to follow other ways for the environment, the "forked"
project aka "new library" should follow. But yes, I guess it is easier,
and I agree with you this is just a few days work.

This can solve the issue with license - the other topic regarding
"default" (or "initialization") environment is not changed by an
external library.

Best regards,
Stefano Babic

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================

[U-Boot] SWUpdate - U-Boot environment library dependency

[Simon Goldschmidt]

On Wednesday, November 21, 2018, Stefano Babic <sbabic@denx.de> wrote:

> Hi Marek,
>
> On 21/11/18 15:31, Marek Vasut wrote:
> > On 11/21/2018 10:20 AM, Wolfgang Denk wrote:
> >> Dear Stefano,
> >
> > Hi,
> >
> >> In message <96836cc1-e4bb-a2a2-05ac-056053b4c426@denx.de> you wrote:
> >>>
> >>> I would like to see the library under LGPL instead of GPL2, too, and I
> >>> raised this issue when I started SWUpdate, but I was not very active to
> >>> promote this. Tom, Wolfgang, is there chances to switch license ?
> >>
> >> Relicensing requires permission from all who contributed to that
> >> code.
> >>
> >> Consider mine as granted.
> >>
> >> But someone hat to invest the efforts to analyze the code so we
> >> know who to ask, and then collect all the permissions...
> >>
> >>> A env library is very welcomed by many customers, because they could
> >>> integrate it in their application if license allows it.
> >>
> >> Agreed.
> >
> > Then again, U-Boot environment structure is trivial, crc, flags, data,
> > there is no complexity involved. There is probably some complexity in
> > the backing store access stuff (MTD, block devs, legacy stuff), but that
> > should either use some MTD utils libs, basic block access primitives or
> > be given a once-over and possibly be dropped.
> >
> > I think prototyping a library from scratch that's LGPL would be a few
> > days' work and the benefit would be tremendous all over.
>
>
> I confess I had the same idea - why not ignore the code in tools/env
> (they have also some drawbacks, see the locking mechanism in my previous
> e-mail) and start with a new library from scratch ? Then LGPL is not an
> issue anymore, it is a new development. And I already took this way for
> "grubenv" (I had to, grubenv is not license compatible).
>
> But something in my head is telling me that this is not else as a fork
> of u-boot (ok, a partial fork, just tools/env). And if the U-Boot
> community decides to follow other ways for the environment, the "forked"
> project aka "new library" should follow. But yes, I guess it is easier,
> and I agree with you this is just a few days work.


Once you had this new lgpl library, would it be a big problem to backport
it to the U-Boot tree? Wouldn't that also ensure future commiters accept
the lgpl license when adding their new code?

Simon

[U-Boot] SWUpdate - U-Boot environment library dependency

[Marek Vasut]

On 11/21/2018 04:31 PM, Simon Goldschmidt wrote:
> 
> 
> On Wednesday, November 21, 2018, Stefano Babic <sbabic@denx.de
> <mailto:sbabic@denx.de>> wrote:
> 
>     Hi Marek,
> 
>     On 21/11/18 15:31, Marek Vasut wrote:
>     > On 11/21/2018 10:20 AM, Wolfgang Denk wrote:
>     >> Dear Stefano,
>     >
>     > Hi,
>     >
>     >> In message <96836cc1-e4bb-a2a2-05ac-056053b4c426@denx.de
>     <mailto:96836cc1-e4bb-a2a2-05ac-056053b4c426@denx.de>> you wrote:
>     >>>
>     >>> I would like to see the library under LGPL instead of GPL2, too,
>     and I
>     >>> raised this issue when I started SWUpdate, but I was not very
>     active to
>     >>> promote this. Tom, Wolfgang, is there chances to switch license ?
>     >>
>     >> Relicensing requires permission from all who contributed to that
>     >> code.
>     >>
>     >> Consider mine as granted.
>     >>
>     >> But someone hat to invest the efforts to analyze the code so we
>     >> know who to ask, and then collect all the permissions...
>     >>
>     >>> A env library is very welcomed by many customers, because they could
>     >>> integrate it in their application if license allows it.
>     >>
>     >> Agreed.
>     >
>     > Then again, U-Boot environment structure is trivial, crc, flags, data,
>     > there is no complexity involved. There is probably some complexity in
>     > the backing store access stuff (MTD, block devs, legacy stuff),
>     but that
>     > should either use some MTD utils libs, basic block access
>     primitives or
>     > be given a once-over and possibly be dropped.
>     >
>     > I think prototyping a library from scratch that's LGPL would be a few
>     > days' work and the benefit would be tremendous all over.
> 
> 
>     I confess I had the same idea - why not ignore the code in tools/env
>     (they have also some drawbacks, see the locking mechanism in my previous
>     e-mail) and start with a new library from scratch ? Then LGPL is not an
>     issue anymore, it is a new development. And I already took this way for
>     "grubenv" (I had to, grubenv is not license compatible).
> 
>     But something in my head is telling me that this is not else as a fork
>     of u-boot (ok, a partial fork, just tools/env). And if the U-Boot
>     community decides to follow other ways for the environment, the "forked"
>     project aka "new library" should follow. But yes, I guess it is easier,
>     and I agree with you this is just a few days work.
> 
> 
> Once you had this new lgpl library, would it be a big problem to
> backport it to the U-Boot tree? Wouldn't that also ensure future
> commiters accept the lgpl license when adding their new code?

Maybe it can be part of the U-Boot tree and the old env tools can just
be migrated over, make them a husk which calls the new lib .

-- 
Best regards,
Marek Vasut

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Otavio Salvador]

On Wed, Nov 21, 2018 at 7:10 AM Stefano Babic <sbabic@denx.de> wrote:
> On 21/11/18 00:10, Marek Vasut wrote:
> > On 11/20/2018 10:11 PM, Simon Goldschmidt wrote:
> >> On 04.09.2018 12:30, Andreas Reichel wrote:
> The library is already built (tools/env/lib.a), but IMHO the best thing
> is to export it official and let that in OE we have a
> u-boot-fw-tools-dev with header and library.
>
> I would like to see the library under LGPL instead of GPL2, too, and I
> raised this issue when I started SWUpdate, but I was not very active to
> promote this. Tom, Wolfgang, is there chances to switch license ?

From OE perspective, it ought to be a different recipe as
libu-boot-env or similar, so it does allow to be used without being
connected to the BSP in use. Especially if the option to load default
env from a file or similar is added.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854          Mobile: +1 (347) 903-9750

[U-Boot] SWUpdate - U-Boot environment library dependency

[Wolfgang Denk]

Dear Simon,

In message <90014ca9-64d8-8e38-a6c0-4c06e991c92c@gmail.com> you wrote:
>
> > However, this approach creates several difficulties:

As usual, "difficulties" are made for solving.

I mean, none of these are technically challenging in any way.

> > * Distros like Debian cannot provide a devel package for SWUpdate like
> >    u-boot-dev, since U-Boot has its default environment code compiled
> >    board-dependently and Debian needed it board-independent.

Thta's fine.

> >    If a board-independent libubootenv.a was provided without
> >    a specific default environment, the first update process would destroy
> >    the U-Boot environment if it had had bad CRC before.
> >    (Thanks Stefano for this good point).

One possible way to fix this is not to use a compiled in default
environment for the fw_env tools (or only use this as a very last
resort), but instead provide a configuration option to read the
default settings from a file.  In this case the user could provide
his own, board specific default environment.

> > * If we have a board with U-Boot already preinstalled and want to
> >    compile SWUpdate for it, we need the sources of this very U-Boot to
> >    get the propper lib. For example in a debian-like build system we had to
> >    compile U-Boot again, although it is already installed since we lack
> >    a dev package.

- Why does building the U-Boot package not also create these
  libraries, either directly or as a separate package?
- What prevents the creation of such a dev package that only builds
  the needed lib?

> > * U-Boot does not provide any default means to install a development
> >    library. Thus anything we modify on-top might break with the next
> >    version.

Why don't you add this, then?  Patches are welcome!

> Has there been any progress in the SWupdate/fw_setenv integration? I 
> thought I remember reading something about letting fw_setenv extract the 
> environment from the U-Boot binary in the target, but I cannot find the 
> thread. I do think the current situation should be improved, making 
> fw_setenv independent of the target that is built.

To be honest, I think this is not a clever approach.  This whole
concept of the compiled in default environment is antediluvian.  It
comes from a time where we were booting from parallel NOR flasah (or
other boot ROM where you directly started executing code) and where
we wanted to make sure that we can read the console baud rate very,
very soon.  Today, many boards don't have environment access in the
SPL, and nobody complains that these run with a fixed baud rate
only.

Did anybody ever ask who actually _needs_ this compiled in default
environment?  We could provide other, more efficient configurations
if we drop a few of the old requirements (which are not met by many
recent boards anyway).

> And as this thread is on the swupdate group as well: I would prefer 
> calling an external fw_setenv tool instead of linking in the static 
> library libubootenv.a...

IMO this is just a matter of taste.  Both ways should be possible.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Anyone who isn't confused here doesn't really know what's going on.

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Stefano Babic]

Hi Wolfgang,

On 21/11/18 09:31, Wolfgang Denk wrote:
> Dear Simon,
> 
> In message <90014ca9-64d8-8e38-a6c0-4c06e991c92c@gmail.com> you wrote:
>>
>>> However, this approach creates several difficulties:
> 
> As usual, "difficulties" are made for solving.
> 
> I mean, none of these are technically challenging in any way.
> 
>>> * Distros like Debian cannot provide a devel package for SWUpdate like
>>>    u-boot-dev, since U-Boot has its default environment code compiled
>>>    board-dependently and Debian needed it board-independent.
> 
> Thta's fine.
> 
>>>    If a board-independent libubootenv.a was provided without
>>>    a specific default environment, the first update process would destroy
>>>    the U-Boot environment if it had had bad CRC before.
>>>    (Thanks Stefano for this good point).
> 
> One possible way to fix this is not to use a compiled in default
> environment for the fw_env tools (or only use this as a very last
> resort), but instead provide a configuration option to read the
> default settings from a file.  In this case the user could provide
> his own, board specific default environment.

I tried this, but it is very difficult to maintain this in sync,
specially in projects where the number of developers is > 1 (all
projects). If the file does not correspond to the linked environment,
strange behaviors are expected.

Much better will be if u-boot generates in its build process this file.
It becomes just another artifact like SPL or u-boot.img and we are sure
that it is in sync.

> 
>>> * If we have a board with U-Boot already preinstalled and want to
>>>    compile SWUpdate for it, we need the sources of this very U-Boot to
>>>    get the propper lib. For example in a debian-like build system we had to
>>>    compile U-Boot again, although it is already installed since we lack
>>>    a dev package.
> 
> - Why does building the U-Boot package not also create these
>   libraries, either directly or as a separate package?

Issue is that there is not a u-boot-dev package

> - What prevents the creation of such a dev package that only builds
>   the needed lib?

I think Simon wants to prevent to rebuild u-boot.

> 
>>> * U-Boot does not provide any default means to install a development
>>>    library. Thus anything we modify on-top might break with the next
>>>    version.
> 
> Why don't you add this, then?  Patches are welcome!

+1

> 
>> Has there been any progress in the SWupdate/fw_setenv integration? I 
>> thought I remember reading something about letting fw_setenv extract the 
>> environment from the U-Boot binary in the target, but I cannot find the 
>> thread. I do think the current situation should be improved, making 
>> fw_setenv independent of the target that is built.
> 
> To be honest, I think this is not a clever approach.  This whole
> concept of the compiled in default environment is antediluvian.

Fully agree - I am glad if there will be an improvement.

To be clear: the issue just raises if a board runs with default
environment. User Space cannot know this environment because it is
linked in the bootloader - the current way is to link the same
environment in tools/env (that means, u-boot-fw-tools in OE).

There is no issue at all for boards that are not using default
environment at all, that means they rely to have always a valid
environment in flash.

>  It
> comes from a time where we were booting from parallel NOR flasah (or
> other boot ROM where you directly started executing code) and where
> we wanted to make sure that we can read the console baud rate very,
> very soon.  Today, many boards don't have environment access in the
> SPL, and nobody complains that these run with a fixed baud rate
> only.
> 
> Did anybody ever ask who actually _needs_ this compiled in default
> environment? 

Rather, quite all boards....see CONFIG_EXTRA_ENV in include/configs/*
and the number of patches sent just to fix something in it.

UMHO the default environment should be like an extra image in u-boot,
like an header added by mkimage, because there is no changes in code and
behavior. Not to mention that CONFIG_EXTRA_ENV is really error prone
with the requirement to put each line as a C string with trailing "\0".

> We could provide other, more efficient configurations
> if we drop a few of the old requirements (which are not met by many
> recent boards anyway).

All recent boards, specially the one that switched to distro_, use the
default environment. Thing is that there is no need to add an extra
environment and the board can boot without any intervention by operator
(or by factory) simply after flashing the bootloader.

> 
>> And as this thread is on the swupdate group as well: I would prefer 
>> calling an external fw_setenv tool instead of linking in the static 
>> library libubootenv.a...
> 
> IMO this is just a matter of taste.  Both ways should be possible.
> 

Best regards,
Stefano Babic

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Simon Goldschmidt]

On Wed, Nov 21, 2018 at 10:33 AM Stefano Babic <sbabic@denx.de> wrote:
>
> Hi Wolfgang,
>
> On 21/11/18 09:31, Wolfgang Denk wrote:
> > Dear Simon,
> >
> > In message <90014ca9-64d8-8e38-a6c0-4c06e991c92c@gmail.com> you wrote:
> >>
> >>> However, this approach creates several difficulties:
> >
> > As usual, "difficulties" are made for solving.
> >
> > I mean, none of these are technically challenging in any way.
> >
> >>> * Distros like Debian cannot provide a devel package for SWUpdate like
> >>>    u-boot-dev, since U-Boot has its default environment code compiled
> >>>    board-dependently and Debian needed it board-independent.
> >
> > Thta's fine.
> >
> >>>    If a board-independent libubootenv.a was provided without
> >>>    a specific default environment, the first update process would destroy
> >>>    the U-Boot environment if it had had bad CRC before.
> >>>    (Thanks Stefano for this good point).
> >
> > One possible way to fix this is not to use a compiled in default
> > environment for the fw_env tools (or only use this as a very last
> > resort), but instead provide a configuration option to read the
> > default settings from a file.  In this case the user could provide
> > his own, board specific default environment.
>
> I tried this, but it is very difficult to maintain this in sync,
> specially in projects where the number of developers is > 1 (all
> projects). If the file does not correspond to the linked environment,
> strange behaviors are expected.
>
> Much better will be if u-boot generates in its build process this file.
> It becomes just another artifact like SPL or u-boot.img and we are sure
> that it is in sync.
>
> >
> >>> * If we have a board with U-Boot already preinstalled and want to
> >>>    compile SWUpdate for it, we need the sources of this very U-Boot to
> >>>    get the propper lib. For example in a debian-like build system we had to
> >>>    compile U-Boot again, although it is already installed since we lack
> >>>    a dev package.
> >
> > - Why does building the U-Boot package not also create these
> >   libraries, either directly or as a separate package?
>
> Issue is that there is not a u-boot-dev package
>
> > - What prevents the creation of such a dev package that only builds
> >   the needed lib?
>
> I think Simon wants to prevent to rebuild u-boot.

No, I want to prevent recompiling the u-boot-tools package (fw_setenv)
and Swupdate after changing U-Boot's default environment.

In fact, we are running a Debian-based system and I would like to run
upstream Debian u-boot-tools. And of course running an upstream
swupdate package would be preferred as well.

One way to fix this would be to let fw_setenv somehow write an
environment that doesn't need to know the default environment and is
loaded in a "add" way, not replacing the whole environment as we
normally do on load.

And security wise, I think linking in things as static libs in
Swupdate is not a good decision as you can end up with 2 different
versions of the same sources in your target. I can understand calling
external binaries is not a good solution as well. Using shared libs
would be best. That would mean u-boot-tools fw_setenv would use a
shared library to access the environment and swupdate could use it,
too.

>
> >
> >>> * U-Boot does not provide any default means to install a development
> >>>    library. Thus anything we modify on-top might break with the next
> >>>    version.
> >
> > Why don't you add this, then?  Patches are welcome!
>
> +1
>
> >
> >> Has there been any progress in the SWupdate/fw_setenv integration? I
> >> thought I remember reading something about letting fw_setenv extract the
> >> environment from the U-Boot binary in the target, but I cannot find the
> >> thread. I do think the current situation should be improved, making
> >> fw_setenv independent of the target that is built.
> >
> > To be honest, I think this is not a clever approach.  This whole
> > concept of the compiled in default environment is antediluvian.
>
> Fully agree - I am glad if there will be an improvement.
>
> To be clear: the issue just raises if a board runs with default
> environment. User Space cannot know this environment because it is
> linked in the bootloader - the current way is to link the same
> environment in tools/env (that means, u-boot-fw-tools in OE).
>
> There is no issue at all for boards that are not using default
> environment at all, that means they rely to have always a valid
> environment in flash.
>
> >  It
> > comes from a time where we were booting from parallel NOR flasah (or
> > other boot ROM where you directly started executing code) and where
> > we wanted to make sure that we can read the console baud rate very,
> > very soon.  Today, many boards don't have environment access in the
> > SPL, and nobody complains that these run with a fixed baud rate
> > only.
> >
> > Did anybody ever ask who actually _needs_ this compiled in default
> > environment?

In a secure boot environment, you cannot allow to load the environment
from an untrusted source. We need a default environment in this case.

But there are better ways than compiling it in via C headers, of course!

Simon

>
> Rather, quite all boards....see CONFIG_EXTRA_ENV in include/configs/*
> and the number of patches sent just to fix something in it.
>
> UMHO the default environment should be like an extra image in u-boot,
> like an header added by mkimage, because there is no changes in code and
> behavior. Not to mention that CONFIG_EXTRA_ENV is really error prone
> with the requirement to put each line as a C string with trailing "\0".
>
> > We could provide other, more efficient configurations
> > if we drop a few of the old requirements (which are not met by many
> > recent boards anyway).
>
> All recent boards, specially the one that switched to distro_, use the
> default environment. Thing is that there is no need to add an extra
> environment and the board can boot without any intervention by operator
> (or by factory) simply after flashing the bootloader.
>
> >
> >> And as this thread is on the swupdate group as well: I would prefer
> >> calling an external fw_setenv tool instead of linking in the static
> >> library libubootenv.a...
> >
> > IMO this is just a matter of taste.  Both ways should be possible.
> >
>
> Best regards,
> Stefano Babic
>
> --
> =====================================================================
> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
> =====================================================================

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Jan Kiszka]

On 21.11.18 11:14, Simon Goldschmidt wrote:
>>> - What prevents the creation of such a dev package that only builds
>>>    the needed lib?
>>
>> I think Simon wants to prevent to rebuild u-boot.
> 
> No, I want to prevent recompiling the u-boot-tools package (fw_setenv)
> and Swupdate after changing U-Boot's default environment.
> 
> In fact, we are running a Debian-based system and I would like to run
> upstream Debian u-boot-tools. And of course running an upstream
> swupdate package would be preferred as well.
> 

+1, same situation and requirements on our side. We work around this in [1], but 
we also have boards that we can feed from upstream Debian - except if we wanted 
SWupdate.

Jan

[1] https://github.com/ilbers/isar/tree/master/meta/recipes-bsp/u-boot

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Stefano Babic]

Hi Simon,

On 21/11/18 11:14, Simon Goldschmidt wrote:
> On Wed, Nov 21, 2018 at 10:33 AM Stefano Babic <sbabic@denx.de> wrote:
>>
>> Hi Wolfgang,
>>
>> On 21/11/18 09:31, Wolfgang Denk wrote:
>>> Dear Simon,
>>>
>>> In message <90014ca9-64d8-8e38-a6c0-4c06e991c92c@gmail.com> you wrote:
>>>>
>>>>> However, this approach creates several difficulties:
>>>
>>> As usual, "difficulties" are made for solving.
>>>
>>> I mean, none of these are technically challenging in any way.
>>>
>>>>> * Distros like Debian cannot provide a devel package for SWUpdate like
>>>>>    u-boot-dev, since U-Boot has its default environment code compiled
>>>>>    board-dependently and Debian needed it board-independent.
>>>
>>> Thta's fine.
>>>
>>>>>    If a board-independent libubootenv.a was provided without
>>>>>    a specific default environment, the first update process would destroy
>>>>>    the U-Boot environment if it had had bad CRC before.
>>>>>    (Thanks Stefano for this good point).
>>>
>>> One possible way to fix this is not to use a compiled in default
>>> environment for the fw_env tools (or only use this as a very last
>>> resort), but instead provide a configuration option to read the
>>> default settings from a file.  In this case the user could provide
>>> his own, board specific default environment.
>>
>> I tried this, but it is very difficult to maintain this in sync,
>> specially in projects where the number of developers is > 1 (all
>> projects). If the file does not correspond to the linked environment,
>> strange behaviors are expected.
>>
>> Much better will be if u-boot generates in its build process this file.
>> It becomes just another artifact like SPL or u-boot.img and we are sure
>> that it is in sync.
>>
>>>
>>>>> * If we have a board with U-Boot already preinstalled and want to
>>>>>    compile SWUpdate for it, we need the sources of this very U-Boot to
>>>>>    get the propper lib. For example in a debian-like build system we had to
>>>>>    compile U-Boot again, although it is already installed since we lack
>>>>>    a dev package.
>>>
>>> - Why does building the U-Boot package not also create these
>>>   libraries, either directly or as a separate package?
>>
>> Issue is that there is not a u-boot-dev package
>>
>>> - What prevents the creation of such a dev package that only builds
>>>   the needed lib?
>>
>> I think Simon wants to prevent to rebuild u-boot.
> 
> No, I want to prevent recompiling the u-boot-tools package (fw_setenv)
> and Swupdate after changing U-Boot's default environment.
> 

Right, I understand this.

> In fact, we are running a Debian-based system and I would like to run
> upstream Debian u-boot-tools. And of course running an upstream
> swupdate package would be preferred as well.

I am searching / waiting for a mentor:

	https://mentors.debian.net/package/swupdate

Package should be updated as well, but I'll push a new release for the
end of the month.

> 
> One way to fix this would be to let fw_setenv somehow write an
> environment that doesn't need to know the default environment and is
> loaded in a "add" way, not replacing the whole environment as we
> normally do on load.

True, but this has side effect and limitation. You cannot change a
variable in the deafult environment if you need it because you do not
know it. If some changes are needed, even if for very small things like
activating a gpio before ooting, you need to update the bootloader.

I have also thought at this but I dislike the drawbacks.

> 
> And security wise, I think linking in things as static libs in
> Swupdate is not a good decision as you can end up with 2 different
> versions of the same sources in your target.

This is a different topic - but until the library was used in SWUpdate,
there was not another user, and if SWUpdate is the only user of it,
static linking is fine.

Moving to shared library should be done in U-Boot project, then. Some
changes are then required, at least how the environment is locked (it is
not clean as it is done now - locking should be done by the library and
not by the caller).

> I can understand calling
> external binaries is not a good solution as well.

Right, I explain this in the other e-mail.

> Using shared libs
> would be best. That would mean u-boot-tools fw_setenv would use a
> shared library to access the environment and swupdate could use it,
> too.
The approach is fine with me.

> 
>>
>>>
>>>>> * U-Boot does not provide any default means to install a development
>>>>>    library. Thus anything we modify on-top might break with the next
>>>>>    version.
>>>
>>> Why don't you add this, then?  Patches are welcome!
>>
>> +1
>>
>>>
>>>> Has there been any progress in the SWupdate/fw_setenv integration? I
>>>> thought I remember reading something about letting fw_setenv extract the
>>>> environment from the U-Boot binary in the target, but I cannot find the
>>>> thread. I do think the current situation should be improved, making
>>>> fw_setenv independent of the target that is built.
>>>
>>> To be honest, I think this is not a clever approach.  This whole
>>> concept of the compiled in default environment is antediluvian.
>>
>> Fully agree - I am glad if there will be an improvement.
>>
>> To be clear: the issue just raises if a board runs with default
>> environment. User Space cannot know this environment because it is
>> linked in the bootloader - the current way is to link the same
>> environment in tools/env (that means, u-boot-fw-tools in OE).
>>
>> There is no issue at all for boards that are not using default
>> environment at all, that means they rely to have always a valid
>> environment in flash.
>>
>>>  It
>>> comes from a time where we were booting from parallel NOR flasah (or
>>> other boot ROM where you directly started executing code) and where
>>> we wanted to make sure that we can read the console baud rate very,
>>> very soon.  Today, many boards don't have environment access in the
>>> SPL, and nobody complains that these run with a fixed baud rate
>>> only.
>>>
>>> Did anybody ever ask who actually _needs_ this compiled in default
>>> environment?
> 
> In a secure boot environment, you cannot allow to load the environment
> from an untrusted source. We need a default environment in this case.

Or you make that changes are trusted.

> 
> But there are better ways than compiling it in via C headers, of course!

Exactly - another possible way is to use a configuration file, run a
sort of "mkenvimage", bound the result to u-boot and use the
configuration file as input for the libubootenv.[a|so] library.

Best regards,
Stefano

> 
> Simon
> 
>>
>> Rather, quite all boards....see CONFIG_EXTRA_ENV in include/configs/*
>> and the number of patches sent just to fix something in it.
>>
>> UMHO the default environment should be like an extra image in u-boot,
>> like an header added by mkimage, because there is no changes in code and
>> behavior. Not to mention that CONFIG_EXTRA_ENV is really error prone
>> with the requirement to put each line as a C string with trailing "\0".
>>
>>> We could provide other, more efficient configurations
>>> if we drop a few of the old requirements (which are not met by many
>>> recent boards anyway).
>>
>> All recent boards, specially the one that switched to distro_, use the
>> default environment. Thing is that there is no need to add an extra
>> environment and the board can boot without any intervention by operator
>> (or by factory) simply after flashing the bootloader.
>>
>>>
>>>> And as this thread is on the swupdate group as well: I would prefer
>>>> calling an external fw_setenv tool instead of linking in the static
>>>> library libubootenv.a...
>>>
>>> IMO this is just a matter of taste.  Both ways should be possible.
>>>
>>
>> Best regards,
>> Stefano Babic
>>
>> --
>> =====================================================================
>> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
>> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
>> Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
>> =====================================================================
> 


-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Wolfgang Denk]

Dear Stefano,

In message <9efc8990-118c-d5b9-802d-8190db2326d3@denx.de> you wrote:
>
> True, but this has side effect and limitation. You cannot change a
> variable in the deafult environment if you need it because you do not
> know it. If some changes are needed, even if for very small things like
> activating a gpio before ooting, you need to update the bootloader.

Just my words.  (Mis) using the default environment for such
purposes is broken by design.

Let's get rid of this!

> Moving to shared library should be done in U-Boot project, then. Some
> changes are then required, at least how the environment is locked (it is
> not clean as it is done now - locking should be done by the library and
> not by the caller).

Speaking of security...  shared libraries open a number of new
attack vectors, too...

> > In a secure boot environment, you cannot allow to load the environment
> > from an untrusted source. We need a default environment in this case.
>
> Or you make that changes are trusted.

Right, when we sign (and check the signatures) of all other images,
then why not do the very same for some environment image?

That would even be _better_ as currently there is no, absolutely no
check if the builtin default environment is in any way consistent.


Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"This is a test of the Emergency Broadcast System. If this had been an
actual emergency, do you really think we'd stick around to tell you?"

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Simon Goldschmidt]

On Wed, Nov 21, 2018 at 12:45 PM Wolfgang Denk <wd@denx.de> wrote:
>
> Dear Stefano,
>
> In message <9efc8990-118c-d5b9-802d-8190db2326d3@denx.de> you wrote:
> >
> > True, but this has side effect and limitation. You cannot change a
> > variable in the deafult environment if you need it because you do not
> > know it. If some changes are needed, even if for very small things like
> > activating a gpio before ooting, you need to update the bootloader.
>
> Just my words.  (Mis) using the default environment for such
> purposes is broken by design.
>
> Let's get rid of this!
>
> > Moving to shared library should be done in U-Boot project, then. Some
> > changes are then required, at least how the environment is locked (it is
> > not clean as it is done now - locking should be done by the library and
> > not by the caller).
>
> Speaking of security...  shared libraries open a number of new
> attack vectors, too...
>
> > > In a secure boot environment, you cannot allow to load the environment
> > > from an untrusted source. We need a default environment in this case.
> >
> > Or you make that changes are trusted.
>
> Right, when we sign (and check the signatures) of all other images,
> then why not do the very same for some environment image?

You normally cannot sign the environment in the target when saving it
when using private/public keys.

We are using a signed U-Boot image that apart from the default
environment only needs to load MAC addresses. I cannot do this via a
loaded environment (signed or unsigned) as the MAC addresses are
stored in production and I cannot rely on production always having an
up-to-date environment to embed their MAC addresses when programming.
To use environment loading here, I would have to implement a whitelist
that only loads the MAC addresses from the saved environment. That
sounds a bit hacked, too.

So when it comes to secure boot, I do think there's a use case for not
loading an environment. I don't currently mind how this environment is
initialized. And maybe I don't yet get what you are talking about when
trying to get rid of the default environment. I do need U-Boot to run
with a predefined environment without loading it.

Regards,
Simon

>
> That would even be _better_ as currently there is no, absolutely no
> check if the builtin default environment is in any way consistent.

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Wolfgang Denk]

Dear Simon,

In message <CAAh8qszhQARksHGQkUMk_QEyA8j6s7p_gnNCGQbZfAi_SK0W5A@mail.gmail.com> you wrote:
> >
> > Right, when we sign (and check the signatures) of all other images,
> > then why not do the very same for some environment image?
>
> You normally cannot sign the environment in the target when saving it
> when using private/public keys.

You mix things here.  I was never talking about the currently used
(and modified) envrionment "when saving it".

Discussion was about the default environment, and I suggested to
replace this with a (signed) image used to initialize the normal
environment.  ther ehas never been a request or suggestion to create
this signature on the target - do it in the very same way as you
sign all other images for the system.

> We are using a signed U-Boot image that apart from the default
> environment only needs to load MAC addresses. I cannot do this via a
> loaded environment (signed or unsigned) as the MAC addresses are
> stored in production and I cannot rely on production always having an
> up-to-date environment to embed their MAC addresses when programming.

Yes you can.  If you like, you can have the U-Boot image and the
environment image separate and signed separately,  or you can create
some form of metaimage (say, as FIT) with individual signatures, or
you can simly concatelante both and use a common signature.  There
is a zillion of ways to do it.

> To use environment loading here, I would have to implement a whitelist
> that only loads the MAC addresses from the saved environment. That
> sounds a bit hacked, too.

Why are you making things so complicated?  You can always do just
"env import ethaddr".  No need to imeplement enything.

> So when it comes to secure boot, I do think there's a use case for not
> loading an environment.

But you are loading it in any case.  Whether from a binary object
placed by the linker somewhwere in your data segment or from a
[signed or at least checksummed] image somewhere else does not make
any difference security-wise.  It is only a minimal technical
difference, i. e. using a different loading mechanism.

> I don't currently mind how this environment is
> initialized. And maybe I don't yet get what you are talking about when
> trying to get rid of the default environment. I do need U-Boot to run
> with a predefined environment without loading it.

See my previous explantion about the 3 copies of the environment.
All I suggest is to replace the binary blob by somethin gthat is not
statically linked into the U-Boot image, so it can be shared for
example with the fw_env tools.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Philosophy is a game with objectives and no rules.
Mathematics is a game with rules and no objectives.

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Stefano Babic]

Hi Wolfgang,

On 21/11/18 12:45, Wolfgang Denk wrote:
> Dear Stefano,
> 
> In message <9efc8990-118c-d5b9-802d-8190db2326d3@denx.de> you wrote:
>>
>> True, but this has side effect and limitation. You cannot change a
>> variable in the deafult environment if you need it because you do not
>> know it. If some changes are needed, even if for very small things like
>> activating a gpio before ooting, you need to update the bootloader.
> 
> Just my words.  (Mis) using the default environment for such
> purposes is broken by design.
> 
> Let's get rid of this!
> 
>> Moving to shared library should be done in U-Boot project, then. Some
>> changes are then required, at least how the environment is locked (it is
>> not clean as it is done now - locking should be done by the library and
>> not by the caller).
> 
> Speaking of security...  shared libraries open a number of new
> attack vectors, too...

This is the reason why these libraries are in meta-swupdate built as
static libraries. Up now I had no requests that someone else wants to
use them.

> 
>>> In a secure boot environment, you cannot allow to load the environment
>>> from an untrusted source. We need a default environment in this case.
>>
>> Or you make that changes are trusted.
> 
> Right, when we sign (and check the signatures) of all other images,
> then why not do the very same for some environment image?

The weird thing is with "saveenv" - if we just read the env, it is fine,
but if we want to change it, we need to sign, and this requires a
private key on target.

> 
> That would even be _better_ as currently there is no, absolutely no
> check if the builtin default environment is in any way consistent.

This is not true. If the environment is linked to u-boot, it is signed
together with u-boot and its consistency is automatically verified.

Best regards,
Stefano

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Wolfgang Denk]

Dear Stefano,

In message <7089ef62-ed0f-87f4-e979-8c18a6ae4b62@denx.de> you wrote:
>
> > Right, when we sign (and check the signatures) of all other images,
> > then why not do the very same for some environment image?
>
> The weird thing is with "saveenv" - if we just read the env, it is fine,
> but if we want to change it, we need to sign, and this requires a
> private key on target.

Agreed, but this is a totaly different issue.

The separate (potentially singed0 environment image is only the
replacement for the current "default environment", which is not
used for "env save".  In the same way, there is no need to modfy the
signed image.

But yes, it might be desirable to protect the working environment
against malicious manipulation - but this should be discussed in a
separate thread.

> > That would even be _better_ as currently there is no, absolutely no
> > check if the builtin default environment is in any way consistent.
>
> This is not true. If the environment is linked to u-boot, it is signed
> together with u-boot and its consistency is automatically verified.

Only if you use signed images.  With plain U-Boot, there is not even
a checksum for it...

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
If all you have is a hammer, everything looks like a nail.

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Simon Goldschmidt]

Am Mi., 21. Nov. 2018, 15:27 hat Wolfgang Denk <wd@denx.de> geschrieben:

> Dear Stefano,
>
> In message <7089ef62-ed0f-87f4-e979-8c18a6ae4b62@denx.de> you wrote:
> >
> > > Right, when we sign (and check the signatures) of all other images,
> > > then why not do the very same for some environment image?
> >
> > The weird thing is with "saveenv" - if we just read the env, it is fine,
> > but if we want to change it, we need to sign, and this requires a
> > private key on target.
>
> Agreed, but this is a totaly different issue.
>
> The separate (potentially singed0 environment image is only the
> replacement for the current "default environment", which is not
> used for "env save".  In the same way, there is no need to modfy the
> signed image.
>
> But yes, it might be desirable to protect the working environment
> against malicious manipulation - but this should be discussed in a
> separate thread.
>
> > > That would even be _better_ as currently there is no, absolutely no
> > > check if the builtin default environment is in any way consistent.
> >
> > This is not true. If the environment is linked to u-boot, it is signed
> > together with u-boot and its consistency is automatically verified.
>
> Only if you use signed images.  With plain U-Boot, there is not even
> a checksum for it...
>

When SPL loads U-Boot from a legacy image, isn't there a CRC involved over
the full image including the environment?

Simon

>

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Stefano Babic]

On 21/11/18 15:37, Simon Goldschmidt wrote:
> 
> 
> Am Mi., 21. Nov. 2018, 15:27 hat Wolfgang Denk <wd@denx.de
> <mailto:wd@denx.de>> geschrieben:
> 
>     Dear Stefano,
> 
>     In message <7089ef62-ed0f-87f4-e979-8c18a6ae4b62@denx.de
>     <mailto:7089ef62-ed0f-87f4-e979-8c18a6ae4b62@denx.de>> you wrote:
>     >
>     > > Right, when we sign (and check the signatures) of all other images,
>     > > then why not do the very same for some environment image?
>     >
>     > The weird thing is with "saveenv" - if we just read the env, it is
>     fine,
>     > but if we want to change it, we need to sign, and this requires a
>     > private key on target.
> 
>     Agreed, but this is a totaly different issue.
> 
>     The separate (potentially singed0 environment image is only the
>     replacement for the current "default environment", which is not
>     used for "env save".  In the same way, there is no need to modfy the
>     signed image.
> 
>     But yes, it might be desirable to protect the working environment
>     against malicious manipulation - but this should be discussed in a
>     separate thread.
> 
>     > > That would even be _better_ as currently there is no, absolutely no
>     > > check if the builtin default environment is in any way consistent.
>     >
>     > This is not true. If the environment is linked to u-boot, it is signed
>     > together with u-boot and its consistency is automatically verified.
> 
>     Only if you use signed images.  With plain U-Boot, there is not even
>     a checksum for it...
> 
> 
> When SPL loads U-Boot from a legacy image, isn't there a CRC involved
> over the full image including the environment?

I think Marek is talking about raw u-boot, not in case mkimage has put
an header at the beginning. See CONFIG_SPL_RAW_IMAGE_SUPPORT and
spl_parse_image_header(). The image is simply loaded without checks.

Best regards,
Stefano


-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Wolfgang Denk]

Dear Simon,

In message <CAAh8qsyopV-HftdCWMbMu+fdcpqWWQtBnZDDyqFyv6iC1eoCDA@mail.gmail.com> you wrote:
>
> > Only if you use signed images.  With plain U-Boot, there is not even
> > a checksum for it...
>
> When SPL loads U-Boot from a legacy image, isn't there a CRC involved over
> the full image including the environment?

Yes, but when - for example - the compiled in default environment
gets corrupted while loading from the storage device or while
writing it to RAM, there will be zero check when reading it.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Randal said it would be tough to do in sed. He didn't say  he  didn't
understand  sed.  Randal  understands sed quite well. Which is why he
uses Perl. :-)         - Larry Wall in <7874@jpl-devvax.JPL.NASA.GOV>

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Simon Goldschmidt]

On 21.11.2018 16:41, Wolfgang Denk wrote:
> Dear Simon,
>
> In message <CAAh8qsyopV-HftdCWMbMu+fdcpqWWQtBnZDDyqFyv6iC1eoCDA@mail.gmail.com> you wrote:
>>> Only if you use signed images.  With plain U-Boot, there is not even
>>> a checksum for it...
>> When SPL loads U-Boot from a legacy image, isn't there a CRC involved over
>> the full image including the environment?
> Yes, but when - for example - the compiled in default environment
> gets corrupted while loading from the storage device or while
> writing it to RAM, there will be zero check when reading it.

Hmm, on my board, SPL loads U-Boot from NOR to SDRAM, then checks the 
uimage CRC. I'd say this is enough checks. That might not hold for all 
boards though.

But I still think keeping the default environment in a different place 
would be good.

Could we put U-Boot into a fit image and put the environment in a 
subimage? But it might increase SPL code size when a fit image has to be 
parsed...

Regards,
Simon

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Wolfgang Denk]

Dear Simon,

In message <e7751e81-610b-9cdd-e14d-04fc5b7a297f@gmail.com> you wrote:
>
> Hmm, on my board, SPL loads U-Boot from NOR to SDRAM, then checks the 
> uimage CRC. I'd say this is enough checks. That might not hold for all 
> boards though.

If you check the image in RAM, then yes.  I've also seen
implementations where it gets checked on the fly (for boot time
reasons), so write corruption would not be covered.

> Could we put U-Boot into a fit image and put the environment in a 
> subimage? But it might increase SPL code size when a fit image has to be 
> parsed...

Yes, of course this can be done.  You would need such code anyway in
SPL if you gor for Falcon mode, where you would load a (signed) FIT
image with the Linux kernel and DTB (and maybe even a ramdisk
image).

But as mentioned - this depends on the use case and on available
resources.  It may not be possible on many more resource-constricted
boards - there, other mechanisms should be possible.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"Free markets select for winning solutions."        - Eric S. Raymond

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Simon Goldschmidt]

On 21.11.2018 16:48, Simon Goldschmidt wrote:
> On 21.11.2018 16:41, Wolfgang Denk wrote:
>> Dear Simon,
>>
>> In message <CAAh8qsyopV-HftdCWMbMu+fdcpqWWQtBnZDDyqFyv6iC1eoCDA@mail.gmail.com> you wrote:
>>>> Only if you use signed images.  With plain U-Boot, there is not even
>>>> a checksum for it...
>>> When SPL loads U-Boot from a legacy image, isn't there a CRC involved over
>>> the full image including the environment?
>> Yes, but when - for example - the compiled in default environment
>> gets corrupted while loading from the storage device or while
>> writing it to RAM, there will be zero check when reading it.
> Hmm, on my board, SPL loads U-Boot from NOR to SDRAM, then checks the
> uimage CRC. I'd say this is enough checks. That might not hold for all
> boards though.

Oops, I'll have to take that back. I just checked and SPL *does* boot 
the uImage U-Boot also if I deliberately corrupt the CRC of the image. 
So you're right, it seems to be not protected at all!

Is that expected behaviour or a bug?


Simon

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Wolfgang Denk]

Dear Simon,

In message <97f8b0ec-103b-e9b7-42ed-e1922f6d489a@gmail.com> you wrote:
>
> Oops, I'll have to take that back. I just checked and SPL *does* boot 
> the uImage U-Boot also if I deliberately corrupt the CRC of the image. 
> So you're right, it seems to be not protected at all!
>
> Is that expected behaviour or a bug?

I don't know your board.

It may be intentional, for example if there are size restrictions
for the SPL wich don't allow to include the CRC32 code...

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
The word "fit", as I understand it, means "appropriate to a purpose",
and I would say the body of the Dean is supremely appropriate to  the
purpose of sitting around all day and eating big heavy meals.
                                 - Terry Pratchett, _Moving Pictures_

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Simon Goldschmidt]

Wolfgang,

Am Do., 22. Nov. 2018, 15:23 hat Wolfgang Denk <wd@denx.de> geschrieben:

> Dear Simon,
>
> In message <97f8b0ec-103b-e9b7-42ed-e1922f6d489a@gmail.com> you wrote:
> >
> > Oops, I'll have to take that back. I just checked and SPL *does* boot
> > the uImage U-Boot also if I deliberately corrupt the CRC of the image.
> > So you're right, it seems to be not protected at all!
> >
> > Is that expected behaviour or a bug?
>
> I don't know your board.
>
> It may be intentional, for example if there are size restrictions
> for the SPL wich don't allow to include the CRC32 code...
>

I don't think this is intentional. I discussed this in another thread with
Marek by now. There is no reference to the CRC code in the files in
common/spl or the funtions called by it.

Searching the whole code, I cannot see anything that checks the uImage CRC
from standard SPL. And I don't see board specific controls to do this
check, did I miss something?

Regards,
Simon

>

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Wolfgang Denk]

Dear Simon,

In message <CAAh8qswJ-hf_FCvTdOggpS4MO6pThKLefjYxcquaBuhKfdQO2w@mail.gmail.com> you wrote:
>
> > I don't know your board.
> >
> > It may be intentional, for example if there are size restrictions
> > for the SPL wich don't allow to include the CRC32 code...
> >
>
> I don't think this is intentional. I discussed this in another thread with
> Marek by now. There is no reference to the CRC code in the files in
> common/spl or the funtions called by it.

Depending on hardware capabilities, this may be indeed intentional,
for example to keep the size of the SPL below a certain limit or to
allow for other, more critical features instead.

> Searching the whole code, I cannot see anything that checks the uImage CRC
> from standard SPL. And I don't see board specific controls to do this
> check, did I miss something?

This may well be possible.  But then, this is usually stage two
already - the ROM bootloader did not check any SPL checksum
either...

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Perl itself is  usually  pretty  good  about  telling  you  what  you
shouldn't do. :-)     - Larry Wall in <11091@jpl-devvax.JPL.NASA.GOV>

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Simon Goldschmidt]

Dear Wolfgang,

On 22.11.2018 17:00, Wolfgang Denk wrote:
> Dear Simon,
>
> In message <CAAh8qswJ-hf_FCvTdOggpS4MO6pThKLefjYxcquaBuhKfdQO2w@mail.gmail.com> you wrote:
>>> I don't know your board.
>>>
>>> It may be intentional, for example if there are size restrictions
>>> for the SPL wich don't allow to include the CRC32 code...
>>>
>> I don't think this is intentional. I discussed this in another thread with
>> Marek by now. There is no reference to the CRC code in the files in
>> common/spl or the funtions called by it.
> Depending on hardware capabilities, this may be indeed intentional,
> for example to keep the size of the SPL below a certain limit or to
> allow for other, more critical features instead.
>
>> Searching the whole code, I cannot see anything that checks the uImage CRC
>> from standard SPL. And I don't see board specific controls to do this
>> check, did I miss something?
> This may well be possible.  But then, this is usually stage two
> already - the ROM bootloader did not check any SPL checksum
> either...

Well, the thing is: on my platform the boot rom *does* check SPL crc :-)

I wanted to post a patch for this, but from reading your response, this 
should probably be configurable. Especially as it doesn't only increase 
size but also boot time.

Thanks for your input on this!

Regards,
Simon

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Wolfgang Denk]

Dear Stefano,

In message <6b07725e-51ba-7309-1506-e7211487b947@denx.de> you wrote:
>
> > One possible way to fix this is not to use a compiled in default
> > environment for the fw_env tools (or only use this as a very last
> > resort), but instead provide a configuration option to read the
> > default settings from a file.  In this case the user could provide
> > his own, board specific default environment.
>
> I tried this, but it is very difficult to maintain this in sync,
> specially in projects where the number of developers is > 1 (all
> projects). If the file does not correspond to the linked environment,
> strange behaviors are expected.
>
> Much better will be if u-boot generates in its build process this file.
> It becomes just another artifact like SPL or u-boot.img and we are sure
> that it is in sync.

I tend to disagree.  As mentioned, the original intent of the
default environment was only (!) to make sure that a working
baudrate setting for the console is available on boards where the
environment has never been initialized yet (fresh our of
production) or, but only as additional benefit, that have lost their
environment somehow.

Usually it is not good for any practical purposes - it is missing
any board specific settings like MAC adress, IP addresses, serial
numbers etc.

This was when booting from parallel NOR.

The whole paradigm was dropped when first implementations of SPL
became necessary to boot from NAND - they all had harcoded baud
rates, as they could not access the environment (often due to size
limitations).

My question is:  why do we need the default environment today?

It just consumes a couple of kB size, and does not bring any
measurable benefits.

> > - Why does building the U-Boot package not also create these
> >   libraries, either directly or as a separate package?
>
> Issue is that there is not a u-boot-dev package

Then why not create one?

> > - What prevents the creation of such a dev package that only builds
> >   the needed lib?
>
> I think Simon wants to prevent to rebuild u-boot.

You don't have to rebuild U-Boot - building the lib should be
sufficient.

> To be clear: the issue just raises if a board runs with default
> environment. User Space cannot know this environment because it is
> linked in the bootloader - the current way is to link the same
> environment in tools/env (that means, u-boot-fw-tools in OE).

And I ask again: what exactly would we lose if we drop the whole
default environment?  I mean, what would we lose that we have not
already given up elsewhere with nobody ever complaining about?

> There is no issue at all for boards that are not using default
> environment at all, that means they rely to have always a valid
> environment in flash.

I would be seriously surprised if a single board in real  life uses
the default environment, and only that, for normale operation.

> > Did anybody ever ask who actually _needs_ this compiled in default
> > environment? 
>
> Rather, quite all boards....see CONFIG_EXTRA_ENV in include/configs/*
> and the number of patches sent just to fix something in it.

You misunderstand.  This was not primarily intended for the
_default_ environment, but to _initialize_ the real environment.
In these olden days this would usually be placed insome smaller
sectors of the parallel NOR flash, and special (hand-tuned) linker
scripts would make sure that the respective data would be properly
placed in the U-Boot image.

At these times, a U-Boot image would usually contaim 3 copies of the
same data: two redundant copies of the normal, persistent
environment, embedded somehwere at "magic offsets" in the image,
plus the default environment (somehwere in the [read-only] data
segment, basically at an unknown address).

There would be a few boards that did not use embedded environments,
so other ways wewre selected to initialize it.

But the default environment was never ever intended to be used for
this purpose.

> UMHO the default environment should be like an extra image in u-boot,
> like an header added by mkimage, because there is no changes in code and
> behavior. Not to mention that CONFIG_EXTRA_ENV is really error prone
> with the requirement to put each line as a C string with trailing "\0".

It was 1:1 coding at the time, done by a handful of experts, when
the number of contributors was still single-digit.

I fully afree that it would be much nicer if we could just place it
as plain text (ideally even commented!) on some other file.

But then: there is no technical reason not to implement this.
Volunteers welcome!

> All recent boards, specially the one that switched to distro_, use the
> default environment. Thing is that there is no need to add an extra
> environment and the board can boot without any intervention by operator
> (or by factory) simply after flashing the bootloader.

This is a broken design, and should be fixed.  We don't need three
copies of the environment.  We should get rid of the "default"
environment and replace it with a better way to initialize the real
environment.

And yes, I really hate all these distro settings, because they are
inelegant and unflexible and the slightest change requires to rebuild
and reinstall the whole U-Boot image, when all you want to change is
just a variable setting.  This is CRAP!


IIUC, it should for example be possible to perform the initialization
of the normale environment by running the equivalent of "env import"
from some image (uImage or FIT image or even a plain text file
[though I'd not recommend the latter as it includes zero consistency
checking].


I mean: instead of complaining about the limitations of an
antediluvian construct and investing precious energy in creating
more and more ugly workarounds, we should think about ways to
replace this with a more flexible and more powerful method.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"I think trash is the most important manifestation of culture we have
in my lifetime."                                      - Johnny Legend

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Stefano Babic]

Hi Wolfgang,

On 21/11/18 12:38, Wolfgang Denk wrote:
> Dear Stefano,
> 
> In message <6b07725e-51ba-7309-1506-e7211487b947@denx.de> you wrote:
>>
>>> One possible way to fix this is not to use a compiled in default
>>> environment for the fw_env tools (or only use this as a very last
>>> resort), but instead provide a configuration option to read the
>>> default settings from a file.  In this case the user could provide
>>> his own, board specific default environment.
>>
>> I tried this, but it is very difficult to maintain this in sync,
>> specially in projects where the number of developers is > 1 (all
>> projects). If the file does not correspond to the linked environment,
>> strange behaviors are expected.
>>
>> Much better will be if u-boot generates in its build process this file.
>> It becomes just another artifact like SPL or u-boot.img and we are sure
>> that it is in sync.
> 
> I tend to disagree.  As mentioned, the original intent of the
> default environment was only (!) to make sure that a working
> baudrate setting for the console is available on boards where the
> environment has never been initialized yet (fresh our of
> production) or, but only as additional benefit, that have lost their
> environment somehow.

Current intent is to provide a "factory reset" environment and the
device must be able to be functional even if with limiutation, at least
it should be possible to be restored.

> 
> Usually it is not good for any practical purposes - it is missing
> any board specific settings like MAC adress, IP addresses, serial
> numbers etc.

This is also another aspect that it is difficult to be factorized. Many
projects do not use the env for serial numbers or MAC, but they add an
additional storage with a custom interface.

Storing MAC changed, at least for some SOC as i.MX6, because it is
stored into the SOC (fuses).

> 
> This was when booting from parallel NOR.
> 
> The whole paradigm was dropped when first implementations of SPL
> became necessary to boot from NAND - they all had harcoded baud
> rates, as they could not access the environment (often due to size
> limitations).
> 
> My question is:  why do we need the default environment today?

The "default" environemnt is the "factory reset" environment.

> 
> It just consumes a couple of kB size,

I guess more on some boards...

> and does not bring any
> measurable benefits.
> 
>>> - Why does building the U-Boot package not also create these
>>>   libraries, either directly or as a separate package?
>>
>> Issue is that there is not a u-boot-dev package
> 
> Then why not create one?

Where ? Which project ? You men OE ?

I built and include a sort of u-boot-dev in meta-swupdate, and I have
never heard of someone else who wants to use it up now.

> 
>>> - What prevents the creation of such a dev package that only builds
>>>   the needed lib?
>>
>> I think Simon wants to prevent to rebuild u-boot.
> 
> You don't have to rebuild U-Boot - building the lib should be
> sufficient.

If there is a configuration file that can be imported by the lib and it
remains in sync with U-Boot, if U-Boot still sues linked environment.

> 
>> To be clear: the issue just raises if a board runs with default
>> environment. User Space cannot know this environment because it is
>> linked in the bootloader - the current way is to link the same
>> environment in tools/env (that means, u-boot-fw-tools in OE).
> 
> And I ask again: what exactly would we lose if we drop the whole
> default environment?

Board is not able to run after flashing the bootloader. This is become
worse in the last time due to secure boot. In fact, if this "factory
environment" is linked to U-Boot, it is signed together with U-Boot and
can be trusted (this is the original topic in the thread).

If we use in U-Boot a sort of env import, we have to be sure that we can
trust it.

>  I mean, what would we lose that we have not
> already given up elsewhere with nobody ever complaining about?

I see a lot of complains..

The feature we give with it is an additional redundancy because board
comes up even if environment in flash is damaged.

> 
>> There is no issue at all for boards that are not using default
>> environment at all, that means they rely to have always a valid
>> environment in flash.
> 
> I would be seriously surprised if a single board in real  life uses
> the default environment, and only that, for normale operation.

I guess you are really surprised !

There are plenty of boards - "default environment" means "factory reset
environment".

> 
>>> Did anybody ever ask who actually _needs_ this compiled in default
>>> environment? 
>>
>> Rather, quite all boards....see CONFIG_EXTRA_ENV in include/configs/*
>> and the number of patches sent just to fix something in it.
> 
> You misunderstand.  

I guess...

>This was not primarily intended for the
> _default_ environment, but to _initialize_ the real environment.

This was maybe the initial goal, but if I look in current code there is
just the symbol "default_environment" in the ELF. It contains the
default as you mind + the "real" initialization environment from
CONFIG_EXTRA_ENV.

> In these olden days this would usually be placed insome smaller
> sectors of the parallel NOR flash, and special (hand-tuned) linker
> scripts would make sure that the respective data would be properly
> placed in the U-Boot image.

It could be, but I am talking about what I see in current code.

> 
> At these times, a U-Boot image would usually contaim 3 copies of the
> same data: two redundant copies of the normal, persistent
> environment, embedded somehwere at "magic offsets" in the image,
> plus the default environment (somehwere in the [read-only] data
> segment, basically at an unknown address).

I do not see the last one in data - there are two copies (with
CONFIG_ENV_REDUND set, of course) in flash + one (called
"default_environment") in .text segment. And yes, it is linked, and
address changes.

> 
> There would be a few boards that did not use embedded environments,
> so other ways wewre selected to initialize it.
> 
> But the default environment was never ever intended to be used for
> this purpose.
Ok - but the fact is how it is used nowadays. Just see the number of
patches sent to ML just to change something in CONFIG_EXTRA_ENV.

> 
>> UMHO the default environment should be like an extra image in u-boot,
>> like an header added by mkimage, because there is no changes in code and
>> behavior. Not to mention that CONFIG_EXTRA_ENV is really error prone
>> with the requirement to put each line as a C string with trailing "\0".
> 
> It was 1:1 coding at the time, done by a handful of experts, when
> the number of contributors was still single-digit.

Right

> 
> I fully afree that it would be much nicer if we could just place it
> as plain text (ideally even commented!) on some other file.

+1

> 
> But then: there is no technical reason not to implement this.
> Volunteers welcome!

+1

but we need some concept how to do: do we link the generated environment
as today ? Or do we put somewhere else, for example at the end of U-Boot
(and SPL, too, as some boards can read environment in SPL).

> 
>> All recent boards, specially the one that switched to distro_, use the
>> default environment. Thing is that there is no need to add an extra
>> environment and the board can boot without any intervention by operator
>> (or by factory) simply after flashing the bootloader.
> 
> This is a broken design, and should be fixed.  We don't need three
> copies of the environment.  We should get rid of the "default"
> environment and replace it with a better way to initialize the real
> environment.

We have still3 copies of the environment. Fine if we find a common way
and not board specific to load the initial environment.

> 
> And yes, I really hate all these distro settings, because they are
> inelegant and unflexible and the slightest change requires to rebuild
> and reinstall the whole U-Boot image, when all you want to change is
> just a variable setting.  This is CRAP!

There are different goals. Distros want to have exactly the same
interface for all boards and same set of scripts to boot. If ideally
there is just one u-boot at least for the same set of architecture,
distros could have just one package.

On the other side, products / devices prefer to have a specific
environment to speed up the boot process (distro setting performs
scanning) or to provide custom feature.

> 
> 
> IIUC, it should for example be possible to perform the initialization
> of the normale environment by running the equivalent of "env import"
> from some image (uImage or FIT image or even a plain text file

A signed fit image with the environment should be possible. Anyway, the
same (not signed) we have running mkenvimage on an ASCII file and then
store somewhere.

> [though I'd not recommend the latter as it includes zero consistency
> checking].
> 
> 
> I mean: instead of complaining about the limitations of an
> antediluvian construct and investing precious energy in creating
> more and more ugly workarounds, we should think about ways to
> replace this with a more flexible and more powerful method.

Agree on this, we should find an accepted and community approved method..

Best regards,
Stefano

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Wolfgang Denk]

Dear Stefano,

In message <151c30b1-d383-d212-a611-519150a80e98@denx.de> you wrote:
>
> > I tend to disagree.  As mentioned, the original intent of the
> > default environment was only (!) to make sure that a working
> > baudrate setting for the console is available on boards where the
> > environment has never been initialized yet (fresh our of
> > production) or, but only as additional benefit, that have lost their
> > environment somehow.
>
> Current intent is to provide a "factory reset" environment and the
> device must be able to be functional even if with limiutation, at least
> it should be possible to be restored.

I understand the intend.  But instead of chosing an approoriate
method (creating a separate enviornment block to be loaded with "env
import" which has been created for exactly such purposes as user
profiles or factory reset) they decided to to the Wong Thing.

This should be fixed.

> > My question is:  why do we need the default environment today?
>
> The "default" environemnt is the "factory reset" environment.

You avoid a straight and honest answer here.

It is _used_ for this purpoose, or rather _misused_.

My question is: whyu do we _need_ the default environment today?

I think we should get rid of it, and yes, obviously this means that
we should put the current (mis-) users from their heads to their
feet.

> >>> - Why does building the U-Boot package not also create these
> >>>   libraries, either directly or as a separate package?
> >>
> >> Issue is that there is not a u-boot-dev package
> > 
> > Then why not create one?
>
> Where ? Which project ? You men OE ?

You said there was no such image.  I don't know which project you
were referring to...

> > You don't have to rebuild U-Boot - building the lib should be
> > sufficient.
>
> If there is a configuration file that can be imported by the lib and it
> remains in sync with U-Boot, if U-Boot still sues linked environment.

Sorry, I don't understand what you want to tell me here.


> > And I ask again: what exactly would we lose if we drop the whole
> > default environment?
>
> Board is not able to run after flashing the bootloader. This is become
> worse in the last time due to secure boot. In fact, if this "factory
> environment" is linked to U-Boot, it is signed together with U-Boot and
> can be trusted (this is the original topic in the thread).

This is what people made out of it.  But it's not the only possible
way, and definitely not the best way.

You ignore my arguments.

Instead of using a binary blob inserted somewhere by the linker, we
could use a more suitable data structure, probably even separate and
with it's own signature so it could be used (and verified) by
external tools as well.

And get rid of this current implementation.

> If we use in U-Boot a sort of env import, we have to be sure that we can
> trust it.

Any other form of implementation is probably better and more secure.
At themoment the binary blob just gets loaded, without any checking,
not even for corruption.

> I see a lot of complains..

I don't.

> The feature we give with it is an additional redundancy because board
> comes up even if environment in flash is damaged.

If you split off the initialization, you could even have more than
one copy of the initial / factory settings if you like.

> > I would be seriously surprised if a single board in real  life uses
> > the default environment, and only that, for normale operation.
>
> I guess you are really surprised !
>
> There are plenty of boards - "default environment" means "factory reset
> environment".

s/means/has been misused/

This is not a feature, it is part of the problem that should be
solved!

> This was maybe the initial goal, but if I look in current code there is
> just the symbol "default_environment" in the ELF. It contains the
> default as you mind + the "real" initialization environment from
> CONFIG_EXTRA_ENV.

This has always been like that.  Yes of course it is a convenient
way to create4 customized initial settings.  But the method how it
is implemented does not scale, and we are running in limitations.

So let's replace it with soemthing more powerful and more flexible.

> > At these times, a U-Boot image would usually contaim 3 copies of the
> > same data: two redundant copies of the normal, persistent
> > environment, embedded somehwere at "magic offsets" in the image,
> > plus the default environment (somehwere in the [read-only] data
> > segment, basically at an unknown address).
>
> I do not see the last one in data - there are two copies (with
> CONFIG_ENV_REDUND set, of course) in flash + one (called
> "default_environment") in .text segment. And yes, it is linked, and
> address changes.

If you were using an "embedded environment" it was 3 copies.
But for initialization, only one would be needed.

> but we need some concept how to do: do we link the generated environment
> as today ? Or do we put somewhere else, for example at the end of U-Boot
> (and SPL, too, as some boards can read environment in SPL).

If we don't use a binary blob any more linking few advantages; the
only one I can think of that it is easy to get the start address in
the code - it's just a variable initialized by the linker.  the
disadvantage is that this address is more or less random unless you
play tricks in the linker script, and it;s difficult to access from
outside.

To me it seems more desirable to have a separate image, so we can
still decide what to do with it:  attach (or otherwise combine it)
with the U-Boot or SPL binary - maybe using the linker or "cat" or
"dd" or FIT images or whatever seems useful in the specific context.
I don't think we have to decide for one specific implementation - we
should allow for flexbility here.

Then we can also use other (external) tools like fw_env to access
this area easily.

> We have still3 copies of the environment. Fine if we find a common way
> and not board specific to load the initial environment.

3, or even more.  This is good, as nobody will suffere from an
increase of the memory footprint.

The difference is that an ancient crutch gets replaced by a more
powerful and flexible mnechanism - one where you can potentially
have not only one "default" (or factory) environment, but more
(think of "user profiles"), which can be changed (even securely,
i.  e. with signature checking) without having to recompile and
re-install U-Boot.

> > And yes, I really hate all these distro settings, because they are
> > inelegant and unflexible and the slightest change requires to rebuild
> > and reinstall the whole U-Boot image, when all you want to change is
> > just a variable setting.  This is CRAP!
>
> There are different goals. Distros want to have exactly the same
> interface for all boards and same set of scripts to boot. If ideally
> there is just one u-boot at least for the same set of architecture,
> distros could have just one package.

I don't think these are so different goals.  These are the result of
the current limitations and that people tend to think small - only
for their own poroject and within the current limits.

As is, each and every distribution will have their own U-Boot
binary.  And switching the distro will mean you have to update
U-Boot.

If we splitt off the defualt environment, distros could use the same
binary (at least in theory) - but in any case switching distros
should just mean you have to add the new environment settings - and
when I say "add" here I mean exactly that. Ideally there would be no
need to replace / remove an already existing (known to be working)
environment from another distro.  We could keep it to allow
switching between distros without reinstall. We could use it as a
fall-back if anything goes wrong.  And so on...

> On the other side, products / devices prefer to have a specific
> environment to speed up the boot process (distro setting performs
> scanning) or to provide custom feature.

Right, but my proposal just adds more lexibility, it does not take
away anything you have right now.

> A signed fit image with the environment should be possible. Anyway, the
> same (not signed) we have running mkenvimage on an ASCII file and then
> store somewhere.

Right.  Or, if you are satisfied with the status quo for the default
environment (no checking at all) you could just use a plain text
file.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"I like your game but we have to change the rules."

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Simon Goldschmidt]

Am Mi., 21. Nov. 2018, 15:23 hat Wolfgang Denk <wd@denx.de> geschrieben:

> Dear Stefano,
>
> In message <151c30b1-d383-d212-a611-519150a80e98@denx.de> you wrote:
> >
> > > I tend to disagree.  As mentioned, the original intent of the
> > > default environment was only (!) to make sure that a working
> > > baudrate setting for the console is available on boards where the
> > > environment has never been initialized yet (fresh our of
> > > production) or, but only as additional benefit, that have lost their
> > > environment somehow.
> >
> > Current intent is to provide a "factory reset" environment and the
> > device must be able to be functional even if with limiutation, at least
> > it should be possible to be restored.
>
> I understand the intend.  But instead of chosing an approoriate
> method (creating a separate enviornment block to be loaded with "env
> import" which has been created for exactly such purposes as user
> profiles or factory reset) they decided to to the Wong Thing.
>
> This should be fixed.
>
> > > My question is:  why do we need the default environment today?
> >
> > The "default" environemnt is the "factory reset" environment.
>
> You avoid a straight and honest answer here.
>
> It is _used_ for this purpoose, or rather _misused_.
>
> My question is: whyu do we _need_ the default environment today?
>
> I think we should get rid of it, and yes, obviously this means that
> we should put the current (mis-) users from their heads to their
> feet.
>
> > >>> - Why does building the U-Boot package not also create these
> > >>>   libraries, either directly or as a separate package?
> > >>
> > >> Issue is that there is not a u-boot-dev package
> > >
> > > Then why not create one?
> >
> > Where ? Which project ? You men OE ?
>
> You said there was no such image.  I don't know which project you
> were referring to...
>
> > > You don't have to rebuild U-Boot - building the lib should be
> > > sufficient.
> >
> > If there is a configuration file that can be imported by the lib and it
> > remains in sync with U-Boot, if U-Boot still sues linked environment.
>
> Sorry, I don't understand what you want to tell me here.
>
>
> > > And I ask again: what exactly would we lose if we drop the whole
> > > default environment?
> >
> > Board is not able to run after flashing the bootloader. This is become
> > worse in the last time due to secure boot. In fact, if this "factory
> > environment" is linked to U-Boot, it is signed together with U-Boot and
> > can be trusted (this is the original topic in the thread).
>
> This is what people made out of it.  But it's not the only possible
> way, and definitely not the best way.
>
> You ignore my arguments.
>
> Instead of using a binary blob inserted somewhere by the linker, we
> could use a more suitable data structure, probably even separate and
> with it's own signature so it could be used (and verified) by
> external tools as well.
>
> And get rid of this current implementation.
>
> > If we use in U-Boot a sort of env import, we have to be sure that we can
> > trust it.
>
> Any other form of implementation is probably better and more secure.
> At themoment the binary blob just gets loaded, without any checking,
> not even for corruption.
>
> > I see a lot of complains..
>
> I don't.
>
> > The feature we give with it is an additional redundancy because board
> > comes up even if environment in flash is damaged.
>
> If you split off the initialization, you could even have more than
> one copy of the initial / factory settings if you like.
>
> > > I would be seriously surprised if a single board in real  life uses
> > > the default environment, and only that, for normale operation.
> >
> > I guess you are really surprised !
> >
> > There are plenty of boards - "default environment" means "factory reset
> > environment".
>
> s/means/has been misused/
>
> This is not a feature, it is part of the problem that should be
> solved!
>
> > This was maybe the initial goal, but if I look in current code there is
> > just the symbol "default_environment" in the ELF. It contains the
> > default as you mind + the "real" initialization environment from
> > CONFIG_EXTRA_ENV.
>
> This has always been like that.  Yes of course it is a convenient
> way to create4 customized initial settings.  But the method how it
> is implemented does not scale, and we are running in limitations.
>
> So let's replace it with soemthing more powerful and more flexible.
>
> > > At these times, a U-Boot image would usually contaim 3 copies of the
> > > same data: two redundant copies of the normal, persistent
> > > environment, embedded somehwere at "magic offsets" in the image,
> > > plus the default environment (somehwere in the [read-only] data
> > > segment, basically at an unknown address).
> >
> > I do not see the last one in data - there are two copies (with
> > CONFIG_ENV_REDUND set, of course) in flash + one (called
> > "default_environment") in .text segment. And yes, it is linked, and
> > address changes.
>
> If you were using an "embedded environment" it was 3 copies.
> But for initialization, only one would be needed.
>
> > but we need some concept how to do: do we link the generated environment
> > as today ? Or do we put somewhere else, for example at the end of U-Boot
> > (and SPL, too, as some boards can read environment in SPL).
>
> If we don't use a binary blob any more linking few advantages; the
> only one I can think of that it is easy to get the start address in
> the code - it's just a variable initialized by the linker.  the
> disadvantage is that this address is more or less random unless you
> play tricks in the linker script, and it;s difficult to access from
> outside.
>
> To me it seems more desirable to have a separate image, so we can
> still decide what to do with it:  attach (or otherwise combine it)
> with the U-Boot or SPL binary - maybe using the linker or "cat" or
> "dd" or FIT images or whatever seems useful in the specific context.
> I don't think we have to decide for one specific implementation - we
> should allow for flexbility here.
>
> Then we can also use other (external) tools like fw_env to access
> this area easily.
>

That would make sense, yes. Seems like I really did mix things up. I think
going this way would solve things for the distro packages as well as keep
usage scenarios like mine running.

Regards,
Simon


> > We have still3 copies of the environment. Fine if we find a common way
> > and not board specific to load the initial environment.
>
> 3, or even more.  This is good, as nobody will suffere from an
> increase of the memory footprint.
>
> The difference is that an ancient crutch gets replaced by a more
> powerful and flexible mnechanism - one where you can potentially
> have not only one "default" (or factory) environment, but more
> (think of "user profiles"), which can be changed (even securely,
> i.  e. with signature checking) without having to recompile and
> re-install U-Boot.
>
> > > And yes, I really hate all these distro settings, because they are
> > > inelegant and unflexible and the slightest change requires to rebuild
> > > and reinstall the whole U-Boot image, when all you want to change is
> > > just a variable setting.  This is CRAP!
> >
> > There are different goals. Distros want to have exactly the same
> > interface for all boards and same set of scripts to boot. If ideally
> > there is just one u-boot at least for the same set of architecture,
> > distros could have just one package.
>
> I don't think these are so different goals.  These are the result of
> the current limitations and that people tend to think small - only
> for their own poroject and within the current limits.
>
> As is, each and every distribution will have their own U-Boot
> binary.  And switching the distro will mean you have to update
> U-Boot.
>
> If we splitt off the defualt environment, distros could use the same
> binary (at least in theory) - but in any case switching distros
> should just mean you have to add the new environment settings - and
> when I say "add" here I mean exactly that. Ideally there would be no
> need to replace / remove an already existing (known to be working)
> environment from another distro.  We could keep it to allow
> switching between distros without reinstall. We could use it as a
> fall-back if anything goes wrong.  And so on...
>
> > On the other side, products / devices prefer to have a specific
> > environment to speed up the boot process (distro setting performs
> > scanning) or to provide custom feature.
>
> Right, but my proposal just adds more lexibility, it does not take
> away anything you have right now.
>
> > A signed fit image with the environment should be possible. Anyway, the
> > same (not signed) we have running mkenvimage on an ASCII file and then
> > store somewhere.
>
> Right.  Or, if you are satisfied with the status quo for the default
> environment (no checking at all) you could just use a plain text
> file.
>
> Best regards,
>
> Wolfgang Denk
>
> --
> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
> "I like your game but we have to change the rules."
>

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Stefano Babic]

Hi Simon,

On 20/11/18 22:11, Simon Goldschmidt wrote:
> On 04.09.2018 12:30, Andreas Reichel wrote:
>> Hi all,
>>
>> as Stefano Babic was so friendly and pointed out a few things already,
>> we come the following problematic points:
>>
>> For SWupdate to access U-Boot's environment, it uses code from U-Boot.
>> Before 2015, fw_env.c was copied - hence out of version control,
>> afterwards and since then, a lib.a produced by U-Boot has been used
>> and renamed to libubootenv.a to link against.
>>
>> However, this approach creates several difficulties:
>>
>> * Distros like Debian cannot provide a devel package for SWUpdate like
>>    u-boot-dev, since U-Boot has its default environment code compiled
>>    board-dependently and Debian needed it board-independent.
>>    If a board-independent libubootenv.a was provided without
>>    a specific default environment, the first update process would destroy
>>    the U-Boot environment if it had had bad CRC before.
>>    (Thanks Stefano for this good point).
>>
>> * If we have a board with U-Boot already preinstalled and want to
>>    compile SWUpdate for it, we need the sources of this very U-Boot to
>>    get the propper lib. For example in a debian-like build system we
>> had to
>>    compile U-Boot again, although it is already installed since we lack
>>    a dev package.
>>
>> * U-Boot does not provide any default means to install a development
>>    library. Thus anything we modify on-top might break with the next
>>    version.
>>
>> First proposal by Stefano and me would be to somehow split the default
>> environment from the library to have a board-independent component and
>> board specific data that is passed to U-Boot and SWUpdate somehow.
>>
>> Goal is to factor out U-Boot environment support for other software like
>> SWUpdate and not patching and hacking like its the case with recipes
>> as in
>> openembedded atm.
> 
> Has there been any progress in the SWupdate/fw_setenv integration? I
> thought I remember reading something about letting fw_setenv extract the
> environment from the U-Boot binary in the target, but I cannot find the
> thread.

I did some tests, but I have found the way quite fragile. We should need
at least markers to identify begin and end of the environment because
size differs for each board.

> I do think the current situation should be improved, making
> fw_setenv independent of the target that is built.

A simple way is to let fw_setenv to get the default env via a parameter
(file), but this is also error prone if u-boot is updated and the file
with default env not.

> 
> And as this thread is on the swupdate group as well: I would prefer
> calling an external fw_setenv tool instead of linking in the static
> library libubootenv.a...

I do not prefer this - SWUpdate should be as much as possible self
contained (also for security reasons). Linking libubootenv.a is like
linking any other library. The weirdness is just with the default
environment, that should be addressed.

Best regards,
Stefano Babic

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Jan Kiszka]

On 21.11.18 10:19, Stefano Babic wrote:
> Hi Simon,
> 
> On 20/11/18 22:11, Simon Goldschmidt wrote:
>> On 04.09.2018 12:30, Andreas Reichel wrote:
>>> Hi all,
>>>
>>> as Stefano Babic was so friendly and pointed out a few things already,
>>> we come the following problematic points:
>>>
>>> For SWupdate to access U-Boot's environment, it uses code from U-Boot.
>>> Before 2015, fw_env.c was copied - hence out of version control,
>>> afterwards and since then, a lib.a produced by U-Boot has been used
>>> and renamed to libubootenv.a to link against.
>>>
>>> However, this approach creates several difficulties:
>>>
>>> * Distros like Debian cannot provide a devel package for SWUpdate like
>>>     u-boot-dev, since U-Boot has its default environment code compiled
>>>     board-dependently and Debian needed it board-independent.
>>>     If a board-independent libubootenv.a was provided without
>>>     a specific default environment, the first update process would destroy
>>>     the U-Boot environment if it had had bad CRC before.
>>>     (Thanks Stefano for this good point).
>>>
>>> * If we have a board with U-Boot already preinstalled and want to
>>>     compile SWUpdate for it, we need the sources of this very U-Boot to
>>>     get the propper lib. For example in a debian-like build system we
>>> had to
>>>     compile U-Boot again, although it is already installed since we lack
>>>     a dev package.
>>>
>>> * U-Boot does not provide any default means to install a development
>>>     library. Thus anything we modify on-top might break with the next
>>>     version.
>>>
>>> First proposal by Stefano and me would be to somehow split the default
>>> environment from the library to have a board-independent component and
>>> board specific data that is passed to U-Boot and SWUpdate somehow.
>>>
>>> Goal is to factor out U-Boot environment support for other software like
>>> SWUpdate and not patching and hacking like its the case with recipes
>>> as in
>>> openembedded atm.
>>
>> Has there been any progress in the SWupdate/fw_setenv integration? I
>> thought I remember reading something about letting fw_setenv extract the
>> environment from the U-Boot binary in the target, but I cannot find the
>> thread.
> 
> I did some tests, but I have found the way quite fragile. We should need
> at least markers to identify begin and end of the environment because
> size differs for each board.
> 
>> I do think the current situation should be improved, making
>> fw_setenv independent of the target that is built.
> 
> A simple way is to let fw_setenv to get the default env via a parameter
> (file), but this is also error prone if u-boot is updated and the file
> with default env not.
> 
>>
>> And as this thread is on the swupdate group as well: I would prefer
>> calling an external fw_setenv tool instead of linking in the static
>> library libubootenv.a...
> 
> I do not prefer this - SWUpdate should be as much as possible self
> contained (also for security reasons). Linking libubootenv.a is like
> linking any other library. The weirdness is just with the default
> environment, that should be addressed.
> 

I'm not sure if this is still on the radar as requirement, but it would be great 
if we could kick all the board dependencies out of that library, make them 
config files. That would make the lib compatible with common binary distros like 
Debian.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Simon Goldschmidt]

On Wed, Nov 21, 2018 at 10:19 AM Stefano Babic <sbabic@denx.de> wrote:
>
> Hi Simon,
>
> On 20/11/18 22:11, Simon Goldschmidt wrote:
> > On 04.09.2018 12:30, Andreas Reichel wrote:
> >> Hi all,
> >>
> >> as Stefano Babic was so friendly and pointed out a few things already,
> >> we come the following problematic points:
> >>
> >> For SWupdate to access U-Boot's environment, it uses code from U-Boot.
> >> Before 2015, fw_env.c was copied - hence out of version control,
> >> afterwards and since then, a lib.a produced by U-Boot has been used
> >> and renamed to libubootenv.a to link against.
> >>
> >> However, this approach creates several difficulties:
> >>
> >> * Distros like Debian cannot provide a devel package for SWUpdate like
> >>    u-boot-dev, since U-Boot has its default environment code compiled
> >>    board-dependently and Debian needed it board-independent.
> >>    If a board-independent libubootenv.a was provided without
> >>    a specific default environment, the first update process would destroy
> >>    the U-Boot environment if it had had bad CRC before.
> >>    (Thanks Stefano for this good point).
> >>
> >> * If we have a board with U-Boot already preinstalled and want to
> >>    compile SWUpdate for it, we need the sources of this very U-Boot to
> >>    get the propper lib. For example in a debian-like build system we
> >> had to
> >>    compile U-Boot again, although it is already installed since we lack
> >>    a dev package.
> >>
> >> * U-Boot does not provide any default means to install a development
> >>    library. Thus anything we modify on-top might break with the next
> >>    version.
> >>
> >> First proposal by Stefano and me would be to somehow split the default
> >> environment from the library to have a board-independent component and
> >> board specific data that is passed to U-Boot and SWUpdate somehow.
> >>
> >> Goal is to factor out U-Boot environment support for other software like
> >> SWUpdate and not patching and hacking like its the case with recipes
> >> as in
> >> openembedded atm.
> >
> > Has there been any progress in the SWupdate/fw_setenv integration? I
> > thought I remember reading something about letting fw_setenv extract the
> > environment from the U-Boot binary in the target, but I cannot find the
> > thread.
>
> I did some tests, but I have found the way quite fragile. We should need
> at least markers to identify begin and end of the environment because
> size differs for each board.
>
> > I do think the current situation should be improved, making
> > fw_setenv independent of the target that is built.
>
> A simple way is to let fw_setenv to get the default env via a parameter
> (file), but this is also error prone if u-boot is updated and the file
> with default env not.
>
> >
> > And as this thread is on the swupdate group as well: I would prefer
> > calling an external fw_setenv tool instead of linking in the static
> > library libubootenv.a...
>
> I do not prefer this - SWUpdate should be as much as possible self
> contained (also for security reasons). Linking libubootenv.a is like
> linking any other library. The weirdness is just with the default
> environment, that should be addressed.

For exactly these security reasons, I think it would be better to have
the code only once in your target. And this is not only true for the
u-boot env lib but also for mtd support and maybe for the webserver.

If you have the same versions twice, you need to track and fix both
versions if there should be issues.

I know these things are not available as libs right now, even if that
would be preferable. Therefore I think calling external binaries could
be better. And I don't yet understand why calling external binaries
would be a security risk, maybe you can explain this to me?

Regards,
Simon

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Stefano Babic]

On 21/11/18 11:21, Simon Goldschmidt wrote:
> On Wed, Nov 21, 2018 at 10:19 AM Stefano Babic <sbabic@denx.de> wrote:
>>
>> Hi Simon,
>>
>> On 20/11/18 22:11, Simon Goldschmidt wrote:
>>> On 04.09.2018 12:30, Andreas Reichel wrote:
>>>> Hi all,
>>>>
>>>> as Stefano Babic was so friendly and pointed out a few things already,
>>>> we come the following problematic points:
>>>>
>>>> For SWupdate to access U-Boot's environment, it uses code from U-Boot.
>>>> Before 2015, fw_env.c was copied - hence out of version control,
>>>> afterwards and since then, a lib.a produced by U-Boot has been used
>>>> and renamed to libubootenv.a to link against.
>>>>
>>>> However, this approach creates several difficulties:
>>>>
>>>> * Distros like Debian cannot provide a devel package for SWUpdate like
>>>>    u-boot-dev, since U-Boot has its default environment code compiled
>>>>    board-dependently and Debian needed it board-independent.
>>>>    If a board-independent libubootenv.a was provided without
>>>>    a specific default environment, the first update process would destroy
>>>>    the U-Boot environment if it had had bad CRC before.
>>>>    (Thanks Stefano for this good point).
>>>>
>>>> * If we have a board with U-Boot already preinstalled and want to
>>>>    compile SWUpdate for it, we need the sources of this very U-Boot to
>>>>    get the propper lib. For example in a debian-like build system we
>>>> had to
>>>>    compile U-Boot again, although it is already installed since we lack
>>>>    a dev package.
>>>>
>>>> * U-Boot does not provide any default means to install a development
>>>>    library. Thus anything we modify on-top might break with the next
>>>>    version.
>>>>
>>>> First proposal by Stefano and me would be to somehow split the default
>>>> environment from the library to have a board-independent component and
>>>> board specific data that is passed to U-Boot and SWUpdate somehow.
>>>>
>>>> Goal is to factor out U-Boot environment support for other software like
>>>> SWUpdate and not patching and hacking like its the case with recipes
>>>> as in
>>>> openembedded atm.
>>>
>>> Has there been any progress in the SWupdate/fw_setenv integration? I
>>> thought I remember reading something about letting fw_setenv extract the
>>> environment from the U-Boot binary in the target, but I cannot find the
>>> thread.
>>
>> I did some tests, but I have found the way quite fragile. We should need
>> at least markers to identify begin and end of the environment because
>> size differs for each board.
>>
>>> I do think the current situation should be improved, making
>>> fw_setenv independent of the target that is built.
>>
>> A simple way is to let fw_setenv to get the default env via a parameter
>> (file), but this is also error prone if u-boot is updated and the file
>> with default env not.
>>
>>>
>>> And as this thread is on the swupdate group as well: I would prefer
>>> calling an external fw_setenv tool instead of linking in the static
>>> library libubootenv.a...
>>
>> I do not prefer this - SWUpdate should be as much as possible self
>> contained (also for security reasons). Linking libubootenv.a is like
>> linking any other library. The weirdness is just with the default
>> environment, that should be addressed.
> 
> For exactly these security reasons, I think it would be better to have
> the code only once in your target. And this is not only true for the
> u-boot env lib but also for mtd support and maybe for the webserver.

Webserver ?

I guess you are then about libmtd.a and libubi.a. There is a ticket to
have a mtd-utils-dev package:
	
	https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911821

> 
> If you have the same versions twice, you need to track and fix both
> versions if there should be issues.

Currently, both mtd-utils and u-boot do not export a library. The only
user in OE is SWUpdate that has own rules to build it.

Of course, this could be changed (ad it is desired).

> 
> I know these things are not available as libs right now, even if that
> would be preferable. Therefore I think calling external binaries could
> be better.

No.

> And I don't yet understand why calling external binaries
> would be a security risk, maybe you can explain this to me?

If you check in SWUpdate, you see that code does not call system() at
all - the only exception is for shell scripts.

The first attack I can imagine (sure that we can find several different
if we think enough about it) is a leak in your application that let
change /bin/sh or /usr/bin/fw_setenv on the filesystem. Even if the leak
is outside SWUpdate, an attacker can use it to avoid an update, run
malicious code or install something else.

Not only, if SWUpdate remains self-contained, you can jail it in a
chroot environment - less dependencies SWUpdate has, much better.

Best regards,
Stefano Babic



-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Simon Goldschmidt]

On Wed, Nov 21, 2018 at 12:05 PM Stefano Babic <sbabic@denx.de> wrote:
>
> On 21/11/18 11:21, Simon Goldschmidt wrote:
> > On Wed, Nov 21, 2018 at 10:19 AM Stefano Babic <sbabic@denx.de> wrote:
> >>
> >> Hi Simon,
> >>
> >> On 20/11/18 22:11, Simon Goldschmidt wrote:
> >>> On 04.09.2018 12:30, Andreas Reichel wrote:
> >>>> Hi all,
> >>>>
> >>>> as Stefano Babic was so friendly and pointed out a few things already,
> >>>> we come the following problematic points:
> >>>>
> >>>> For SWupdate to access U-Boot's environment, it uses code from U-Boot.
> >>>> Before 2015, fw_env.c was copied - hence out of version control,
> >>>> afterwards and since then, a lib.a produced by U-Boot has been used
> >>>> and renamed to libubootenv.a to link against.
> >>>>
> >>>> However, this approach creates several difficulties:
> >>>>
> >>>> * Distros like Debian cannot provide a devel package for SWUpdate like
> >>>>    u-boot-dev, since U-Boot has its default environment code compiled
> >>>>    board-dependently and Debian needed it board-independent.
> >>>>    If a board-independent libubootenv.a was provided without
> >>>>    a specific default environment, the first update process would destroy
> >>>>    the U-Boot environment if it had had bad CRC before.
> >>>>    (Thanks Stefano for this good point).
> >>>>
> >>>> * If we have a board with U-Boot already preinstalled and want to
> >>>>    compile SWUpdate for it, we need the sources of this very U-Boot to
> >>>>    get the propper lib. For example in a debian-like build system we
> >>>> had to
> >>>>    compile U-Boot again, although it is already installed since we lack
> >>>>    a dev package.
> >>>>
> >>>> * U-Boot does not provide any default means to install a development
> >>>>    library. Thus anything we modify on-top might break with the next
> >>>>    version.
> >>>>
> >>>> First proposal by Stefano and me would be to somehow split the default
> >>>> environment from the library to have a board-independent component and
> >>>> board specific data that is passed to U-Boot and SWUpdate somehow.
> >>>>
> >>>> Goal is to factor out U-Boot environment support for other software like
> >>>> SWUpdate and not patching and hacking like its the case with recipes
> >>>> as in
> >>>> openembedded atm.
> >>>
> >>> Has there been any progress in the SWupdate/fw_setenv integration? I
> >>> thought I remember reading something about letting fw_setenv extract the
> >>> environment from the U-Boot binary in the target, but I cannot find the
> >>> thread.
> >>
> >> I did some tests, but I have found the way quite fragile. We should need
> >> at least markers to identify begin and end of the environment because
> >> size differs for each board.
> >>
> >>> I do think the current situation should be improved, making
> >>> fw_setenv independent of the target that is built.
> >>
> >> A simple way is to let fw_setenv to get the default env via a parameter
> >> (file), but this is also error prone if u-boot is updated and the file
> >> with default env not.
> >>
> >>>
> >>> And as this thread is on the swupdate group as well: I would prefer
> >>> calling an external fw_setenv tool instead of linking in the static
> >>> library libubootenv.a...
> >>
> >> I do not prefer this - SWUpdate should be as much as possible self
> >> contained (also for security reasons). Linking libubootenv.a is like
> >> linking any other library. The weirdness is just with the default
> >> environment, that should be addressed.
> >
> > For exactly these security reasons, I think it would be better to have
> > the code only once in your target. And this is not only true for the
> > u-boot env lib but also for mtd support and maybe for the webserver.
>
> Webserver ?

Mongoose. Right now, I'm talking about the swupdate 2017.11 branch.
Maybe there are changes regarding webserver since then. I will update
to a newer branch soon (before relase, that is :-)

> I guess you are then about libmtd.a and libubi.a. There is a ticket to
> have a mtd-utils-dev package:
>
>         https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911821
>
> >
> > If you have the same versions twice, you need to track and fix both
> > versions if there should be issues.
>
> Currently, both mtd-utils and u-boot do not export a library. The only
> user in OE is SWUpdate that has own rules to build it.
>
> Of course, this could be changed (ad it is desired).

Yes, I'm talking about libmtd.a. It's good to know there's a plan to
change that.

Thanks for your answers!

Regards,
Simon

>
> >
> > I know these things are not available as libs right now, even if that
> > would be preferable. Therefore I think calling external binaries could
> > be better.
>
> No.
>
> > And I don't yet understand why calling external binaries
> > would be a security risk, maybe you can explain this to me?
>
> If you check in SWUpdate, you see that code does not call system() at
> all - the only exception is for shell scripts.
>
> The first attack I can imagine (sure that we can find several different
> if we think enough about it) is a leak in your application that let
> change /bin/sh or /usr/bin/fw_setenv on the filesystem. Even if the leak
> is outside SWUpdate, an attacker can use it to avoid an update, run
> malicious code or install something else.
>
> Not only, if SWUpdate remains self-contained, you can jail it in a
> chroot environment - less dependencies SWUpdate has, much better.
>
> Best regards,
> Stefano Babic
>
>
>
> --
> =====================================================================
> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
> =====================================================================

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Stefano Babic]

Hi Simon,

On 21/11/18 12:13, Simon Goldschmidt wrote:
> On Wed, Nov 21, 2018 at 12:05 PM Stefano Babic <sbabic@denx.de> wrote:
>>
>> On 21/11/18 11:21, Simon Goldschmidt wrote:
>>> On Wed, Nov 21, 2018 at 10:19 AM Stefano Babic <sbabic@denx.de> wrote:
>>>>
>>>> Hi Simon,
>>>>
>>>> On 20/11/18 22:11, Simon Goldschmidt wrote:
>>>>> On 04.09.2018 12:30, Andreas Reichel wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> as Stefano Babic was so friendly and pointed out a few things already,
>>>>>> we come the following problematic points:
>>>>>>
>>>>>> For SWupdate to access U-Boot's environment, it uses code from U-Boot.
>>>>>> Before 2015, fw_env.c was copied - hence out of version control,
>>>>>> afterwards and since then, a lib.a produced by U-Boot has been used
>>>>>> and renamed to libubootenv.a to link against.
>>>>>>
>>>>>> However, this approach creates several difficulties:
>>>>>>
>>>>>> * Distros like Debian cannot provide a devel package for SWUpdate like
>>>>>>    u-boot-dev, since U-Boot has its default environment code compiled
>>>>>>    board-dependently and Debian needed it board-independent.
>>>>>>    If a board-independent libubootenv.a was provided without
>>>>>>    a specific default environment, the first update process would destroy
>>>>>>    the U-Boot environment if it had had bad CRC before.
>>>>>>    (Thanks Stefano for this good point).
>>>>>>
>>>>>> * If we have a board with U-Boot already preinstalled and want to
>>>>>>    compile SWUpdate for it, we need the sources of this very U-Boot to
>>>>>>    get the propper lib. For example in a debian-like build system we
>>>>>> had to
>>>>>>    compile U-Boot again, although it is already installed since we lack
>>>>>>    a dev package.
>>>>>>
>>>>>> * U-Boot does not provide any default means to install a development
>>>>>>    library. Thus anything we modify on-top might break with the next
>>>>>>    version.
>>>>>>
>>>>>> First proposal by Stefano and me would be to somehow split the default
>>>>>> environment from the library to have a board-independent component and
>>>>>> board specific data that is passed to U-Boot and SWUpdate somehow.
>>>>>>
>>>>>> Goal is to factor out U-Boot environment support for other software like
>>>>>> SWUpdate and not patching and hacking like its the case with recipes
>>>>>> as in
>>>>>> openembedded atm.
>>>>>
>>>>> Has there been any progress in the SWupdate/fw_setenv integration? I
>>>>> thought I remember reading something about letting fw_setenv extract the
>>>>> environment from the U-Boot binary in the target, but I cannot find the
>>>>> thread.
>>>>
>>>> I did some tests, but I have found the way quite fragile. We should need
>>>> at least markers to identify begin and end of the environment because
>>>> size differs for each board.
>>>>
>>>>> I do think the current situation should be improved, making
>>>>> fw_setenv independent of the target that is built.
>>>>
>>>> A simple way is to let fw_setenv to get the default env via a parameter
>>>> (file), but this is also error prone if u-boot is updated and the file
>>>> with default env not.
>>>>
>>>>>
>>>>> And as this thread is on the swupdate group as well: I would prefer
>>>>> calling an external fw_setenv tool instead of linking in the static
>>>>> library libubootenv.a...
>>>>
>>>> I do not prefer this - SWUpdate should be as much as possible self
>>>> contained (also for security reasons). Linking libubootenv.a is like
>>>> linking any other library. The weirdness is just with the default
>>>> environment, that should be addressed.
>>>
>>> For exactly these security reasons, I think it would be better to have
>>> the code only once in your target. And this is not only true for the
>>> u-boot env lib but also for mtd support and maybe for the webserver.
>>
>> Webserver ?
> 
> Mongoose. 

SWupdate has a modified version of the Webserver (yes, code comes from
Mongoose project) to support streaming. It is a different situation as
libmtd or libubootenv

> Right now, I'm talking about the swupdate 2017.11 branch.
> Maybe there are changes regarding webserver since then. I will update
> to a newer branch soon (before relase, that is :-)

There are big changes. Code was synchronized wit 6.11 version of the
Webserver and old static website was replaced by a responsive Web
application.

> 
>> I guess you are then about libmtd.a and libubi.a. There is a ticket to
>> have a mtd-utils-dev package:
>>
>>         https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911821
>>
>>>
>>> If you have the same versions twice, you need to track and fix both
>>> versions if there should be issues.
>>
>> Currently, both mtd-utils and u-boot do not export a library. The only
>> user in OE is SWUpdate that has own rules to build it.
>>
>> Of course, this could be changed (ad it is desired).
> 
> Yes, I'm talking about libmtd.a. It's good to know there's a plan to
> change that.

Plan depends on other projects, anyway...

> 
> Thanks for your answers!
> 

Best regards,
Stefano

> Regards,
> Simon
> 
>>
>>>
>>> I know these things are not available as libs right now, even if that
>>> would be preferable. Therefore I think calling external binaries could
>>> be better.
>>
>> No.
>>
>>> And I don't yet understand why calling external binaries
>>> would be a security risk, maybe you can explain this to me?
>>
>> If you check in SWUpdate, you see that code does not call system() at
>> all - the only exception is for shell scripts.
>>
>> The first attack I can imagine (sure that we can find several different
>> if we think enough about it) is a leak in your application that let
>> change /bin/sh or /usr/bin/fw_setenv on the filesystem. Even if the leak
>> is outside SWUpdate, an attacker can use it to avoid an update, run
>> malicious code or install something else.
>>
>> Not only, if SWUpdate remains self-contained, you can jail it in a
>> chroot environment - less dependencies SWUpdate has, much better.
>>
>> Best regards,
>> Stefano Babic
>>
>>
>>
>> --
>> =====================================================================
>> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
>> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
>> Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
>> =====================================================================


-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================

[U-Boot] [swupdate] Re: SWUpdate - U-Boot environment library dependency

[Stefano Babic]

Hi Simon,

reviving an old (dead)  thread:

On 20/11/18 22:11, Simon Goldschmidt wrote:
> On 04.09.2018 12:30, Andreas Reichel wrote:
>> Hi all,
>>
>> as Stefano Babic was so friendly and pointed out a few things already,
>> we come the following problematic points:
>>
>> For SWupdate to access U-Boot's environment, it uses code from U-Boot.
>> Before 2015, fw_env.c was copied - hence out of version control,
>> afterwards and since then, a lib.a produced by U-Boot has been used
>> and renamed to libubootenv.a to link against.
>>
>> However, this approach creates several difficulties:
>>
>> * Distros like Debian cannot provide a devel package for SWUpdate like
>>    u-boot-dev, since U-Boot has its default environment code compiled
>>    board-dependently and Debian needed it board-independent.
>>    If a board-independent libubootenv.a was provided without
>>    a specific default environment, the first update process would destroy
>>    the U-Boot environment if it had had bad CRC before.
>>    (Thanks Stefano for this good point).
>>
>> * If we have a board with U-Boot already preinstalled and want to
>>    compile SWUpdate for it, we need the sources of this very U-Boot to
>>    get the propper lib. For example in a debian-like build system we
>> had to
>>    compile U-Boot again, although it is already installed since we lack
>>    a dev package.
>>
>> * U-Boot does not provide any default means to install a development
>>    library. Thus anything we modify on-top might break with the next
>>    version.
>>
>> First proposal by Stefano and me would be to somehow split the default
>> environment from the library to have a board-independent component and
>> board specific data that is passed to U-Boot and SWUpdate somehow.
>>
>> Goal is to factor out U-Boot environment support for other software like
>> SWUpdate and not patching and hacking like its the case with recipes
>> as in
>> openembedded atm.
> 
> Has there been any progress in the SWupdate/fw_setenv integration? I
> thought I remember reading something about letting fw_setenv extract the
> environment from the U-Boot binary in the target, but I cannot find the
> thread. I do think the current situation should be improved, making
> fw_setenv independent of the target that is built.

This is done - I took some suggestions (thanks Marek, you, Wolfgang,
etc.) and I create an independent (shared) library under LGPL-2.1.
Details at https://groups.google.com/forum/#!topic/swupdate/2wcIyfexOro

> 
> And as this thread is on the swupdate group as well: I would prefer
> calling an external fw_setenv tool instead of linking in the static
> library libubootenv.a...

This is also done - the "new" fw_setenv takes as argument the initial
environment as file and it is then independent from the target.

Best regards,
Stefano

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================