[U-Boot] [RFC] Exporting defaul environment

[U-Boot] [RFC] Exporting defaul environment

[Stefano Babic]

Hi,

I am thinking about how it is possible to export in a clean way the
default environment from u-boot. The general use case happens when the
environment must be changed from user space (via fw_setenv or whatever)
and no environment was still stored - board boots with default environment.

Up now, the trick is to compile the environment tools (u-boot-fw-utils
in Yocto) with exactly the same U-Boot sources. This works, but it is
error prone when a new Yocto version is delivered (and maybe with a
different u-boot-fw-utils) or even if the bootloader is updated. I am
searching for an alterntive method.

The code in tools/env is really generic, but it becomes board specific
just for the default environment. If the default environment can read in
a reliable way from u-boot itslef (its binary), the tool can search for
the default environment instead of linking again, and tools/env becomes
"distro capable" - one binary that works on all boards.

I have some thoughts and I would like to share, maybe some of you has
better ideas. So what I would like to get directly from the binary is
the offset of "default_environment" (default_environment)
CONFIG_SYS_TEXT_BASE:

- put this offset in the .img header. Yes, it works just for SPL /
u-boot.img (drawback).

- surround default environment with some easy to recognize magic number.
Offset is not known, but user space can find it, like

	magic (long string)
	...
	default_environment
	end default environment (long string)

- adding a section in the linker. I think we tried something in the
past, see __UBOOT_ENV_SECTION__ (it replaced __PPCENV__ from the old
good mpc8xx times..) that is currently unused. I do not like this
approach, it can increase (due to alignment) footprint in SPL, where I
am starting to have issues for smaller i.MX.

- export the deafult environment with such a tool. This dump can be
passed as input to the program in user space. Well, it could be done,
but I still suffer if something is changed (sync is required to have
always the correct dump, I do not see improvement here).

Any thought ?

Best regards,
Stefano


-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================

[U-Boot] [RFC] Exporting defaul environment

[Simon Glass]

Hi Stefano,

On 2 May 2018 at 02:48, Stefano Babic <sbabic@denx.de> wrote:
>
> Hi,
>
> I am thinking about how it is possible to export in a clean way the
> default environment from u-boot. The general use case happens when the
> environment must be changed from user space (via fw_setenv or whatever)
> and no environment was still stored - board boots with default environment.
>
> Up now, the trick is to compile the environment tools (u-boot-fw-utils
> in Yocto) with exactly the same U-Boot sources. This works, but it is
> error prone when a new Yocto version is delivered (and maybe with a
> different u-boot-fw-utils) or even if the bootloader is updated. I am
> searching for an alterntive method.
>
> The code in tools/env is really generic, but it becomes board specific
> just for the default environment. If the default environment can read in
> a reliable way from u-boot itslef (its binary), the tool can search for
> the default environment instead of linking again, and tools/env becomes
> "distro capable" - one binary that works on all boards.
>
> I have some thoughts and I would like to share, maybe some of you has
> better ideas. So what I would like to get directly from the binary is
> the offset of "default_environment" (default_environment)
> CONFIG_SYS_TEXT_BASE:
>
> - put this offset in the .img header. Yes, it works just for SPL /
> u-boot.img (drawback).

How about putting the default environment in a FIT? That is typically
easy to find using the magic word.

>
> - surround default environment with some easy to recognize magic number.
> Offset is not known, but user space can find it, like
>
>         magic (long string)
>         ...
>         default_environment
>         end default environment (long string)
>

Seems like FIT would take care of this for you.

> - adding a section in the linker. I think we tried something in the
> past, see __UBOOT_ENV_SECTION__ (it replaced __PPCENV__ from the old
> good mpc8xx times..) that is currently unused. I do not like this
> approach, it can increase (due to alignment) footprint in SPL, where I
> am starting to have issues for smaller i.MX.

Also I wonder if compression might be useful? FIT supports that fairly easily.

>
> - export the deafult environment with such a tool. This dump can be
> passed as input to the program in user space. Well, it could be done,
> but I still suffer if something is changed (sync is required to have
> always the correct dump, I do not see improvement here).
>
> Any thought ?

Regards,
Simon

[U-Boot] [RFC] Exporting defaul environment

[Kristian Amlie]

On 02/05/18 21:32, Simon Glass wrote:
> Hi Stefano,
> 
> On 2 May 2018 at 02:48, Stefano Babic <sbabic@denx.de> wrote:
>>
>> Hi,
>>
>> I am thinking about how it is possible to export in a clean way the
>> default environment from u-boot. The general use case happens when the
>> environment must be changed from user space (via fw_setenv or whatever)
>> and no environment was still stored - board boots with default environment.
>>
>> Up now, the trick is to compile the environment tools (u-boot-fw-utils
>> in Yocto) with exactly the same U-Boot sources. This works, but it is
>> error prone when a new Yocto version is delivered (and maybe with a
>> different u-boot-fw-utils) or even if the bootloader is updated. I am
>> searching for an alterntive method.
>>
>> The code in tools/env is really generic, but it becomes board specific
>> just for the default environment. If the default environment can read in
>> a reliable way from u-boot itslef (its binary), the tool can search for
>> the default environment instead of linking again, and tools/env becomes
>> "distro capable" - one binary that works on all boards.
>>
>> I have some thoughts and I would like to share, maybe some of you has
>> better ideas. So what I would like to get directly from the binary is
>> the offset of "default_environment" (default_environment)
>> CONFIG_SYS_TEXT_BASE:
>>
>> - put this offset in the .img header. Yes, it works just for SPL /
>> u-boot.img (drawback).
> 
> How about putting the default environment in a FIT? That is typically
> easy to find using the magic word.
> 
>>
>> - surround default environment with some easy to recognize magic number.
>> Offset is not known, but user space can find it, like
>>
>>         magic (long string)
>>         ...
>>         default_environment
>>         end default environment (long string)
>>
> 
> Seems like FIT would take care of this for you.
> 
>> - adding a section in the linker. I think we tried something in the
>> past, see __UBOOT_ENV_SECTION__ (it replaced __PPCENV__ from the old
>> good mpc8xx times..) that is currently unused. I do not like this
>> approach, it can increase (due to alignment) footprint in SPL, where I
>> am starting to have issues for smaller i.MX.
> 
> Also I wonder if compression might be useful? FIT supports that fairly easily.

I like the FIT idea, but in all proposals so far, you still have the
problem of locating either the U-Boot binary or the FIT image. It's not
always clear where they can be found.

I've been having another idea in the back of my head for while, using a
very different approach: Instead of requiring that the tool be able to
fall back to a default environment, require that U-Boot write the
environment to storage if the CRC check fails when it boots, and remove
the ability to write to an uninitialized environment from the tools.
This would guarantee that the environment is available in its expected
storage location when entering userspace, and would also have the
benefit of exposing dangerous situations where a newly flashed root
filesystem has a /etc/fw_env.config which is wrong.

The downside is that if the environment somehow gets messed up from
userspace, you have to reboot to fix the problem. This could be
somewhat, but not completely remedied by requiring a redundant environment.

-- 
Kristian

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180503/c373cf33/attachment.sig>

[U-Boot] [RFC] Exporting defaul environment

[Stefano Babic]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="windows-1254", Size: 5233 bytes --]

Hi Kristian,

On 03/05/2018 14:57, Kristian Amlie wrote:
> On 02/05/18 21:32, Simon Glass wrote:
>> Hi Stefano,
>>
>> On 2 May 2018 at 02:48, Stefano Babic <sbabic@denx.de> wrote:
>>>
>>> Hi,
>>>
>>> I am thinking about how it is possible to export in a clean way the
>>> default environment from u-boot. The general use case happens when the
>>> environment must be changed from user space (via fw_setenv or whatever)
>>> and no environment was still stored - board boots with default environment.
>>>
>>> Up now, the trick is to compile the environment tools (u-boot-fw-utils
>>> in Yocto) with exactly the same U-Boot sources. This works, but it is
>>> error prone when a new Yocto version is delivered (and maybe with a
>>> different u-boot-fw-utils) or even if the bootloader is updated. I am
>>> searching for an alterntive method.
>>>
>>> The code in tools/env is really generic, but it becomes board specific
>>> just for the default environment. If the default environment can read in
>>> a reliable way from u-boot itslef (its binary), the tool can search for
>>> the default environment instead of linking again, and tools/env becomes
>>> "distro capable" - one binary that works on all boards.
>>>
>>> I have some thoughts and I would like to share, maybe some of you has
>>> better ideas. So what I would like to get directly from the binary is
>>> the offset of "default_environment" (default_environment)
>>> CONFIG_SYS_TEXT_BASE:
>>>
>>> - put this offset in the .img header. Yes, it works just for SPL /
>>> u-boot.img (drawback).
>>
>> How about putting the default environment in a FIT? That is typically
>> easy to find using the magic word.
>>
>>>
>>> - surround default environment with some easy to recognize magic number.
>>> Offset is not known, but user space can find it, like
>>>
>>>         magic (long string)
>>>         ...
>>>         default_environment
>>>         end default environment (long string)
>>>
>>
>> Seems like FIT would take care of this for you.
>>
>>> - adding a section in the linker. I think we tried something in the
>>> past, see __UBOOT_ENV_SECTION__ (it replaced __PPCENV__ from the old
>>> good mpc8xx times..) that is currently unused. I do not like this
>>> approach, it can increase (due to alignment) footprint in SPL, where I
>>> am starting to have issues for smaller i.MX.
>>
>> Also I wonder if compression might be useful? FIT supports that fairly easily.
> 
> I like the FIT idea, but in all proposals so far, you still have the
> problem of locating either the U-Boot binary or the FIT image. It's not
> always clear where they can be found.

Right. Things get even worse when a board can boot from several
storages, like from SD, from NOR and so on. It is then difficult to find
at runtime which is the correct image to check.

> 
> I've been having another idea in the back of my head for while, using a
> very different approach: Instead of requiring that the tool be able to
> fall back to a default environment, require that U-Boot write the
> environment to storage if the CRC check fails when it boots, and remove
> the ability to write to an uninitialized environment from the tools.

>From my experience, this is often not desired by customers that do not
like that flash is touched at first start. It does not work in case of
CONFIG_ENV_IS_EMBEDDED, where we should skip the save. Anyway, I agree
that this helps in User Space because it is guaranteed that a suitable
environment is always present.

> This would guarantee that the environment is available in its expected
> storage location when entering userspace, and would also have the
> benefit of exposing dangerous situations where a newly flashed root
> filesystem has a /etc/fw_env.config which is wrong.

Well, if /etc/fw_env.config is broken, we cannot do a lot. But this is
like a bug and should be solved during the test phase.

> 
> The downside is that if the environment somehow gets messed up from
> userspace, you have to reboot to fix the problem. This could be
> somewhat, but not completely remedied by requiring a redundant environment. 

If userspace does not know how to handle the environment and the
environemnt gets messed up, the board could not boot anymore. We do not
know. The redundant environment guarantees us that changing the
environment is power-off safe, but there is no transactions mechanism
where the bootloader can easy step back to the previous one.

Łukasz told me yesterday that he has already sent some patches to
OE-Core to extract the default environment, even if this is not suitable
for an "updater" (our use case).

http://lists.openembedded.org/pipermail/openembedded-core/2018-April/150194.html

The patch is useful to generate a ready-to-flash image, but the exported
default environment could be used as input by the env tools in user
space when no environment is found.

Regards,
Stefano

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================

[U-Boot] [RFC] Exporting defaul environment

[Simon Glass]

Hi Kristian,

On 3 May 2018 at 06:57, Kristian Amlie <kristian.amlie@northern.tech> wrote:
> On 02/05/18 21:32, Simon Glass wrote:
>> Hi Stefano,
>>
>> On 2 May 2018 at 02:48, Stefano Babic <sbabic@denx.de> wrote:
>>>
>>> Hi,
>>>
>>> I am thinking about how it is possible to export in a clean way the
>>> default environment from u-boot. The general use case happens when the
>>> environment must be changed from user space (via fw_setenv or whatever)
>>> and no environment was still stored - board boots with default environment.

Is a magic number not enough to find it?

Alternatively could we put the default environment in /chosen in the
device tree before jumping to Linux?

Regards,
Simon

[U-Boot] [RFC] Exporting defaul environment

[Kristian Amlie]

On 03/05/18 18:00, Stefano Babic wrote:
> On 03/05/2018 14:57, Kristian Amlie wrote:
>> I've been having another idea in the back of my head for while, using a
>> very different approach: Instead of requiring that the tool be able to
>> fall back to a default environment, require that U-Boot write the
>> environment to storage if the CRC check fails when it boots, and remove
>> the ability to write to an uninitialized environment from the tools.
> 
> From my experience, this is often not desired by customers that do not
> like that flash is touched at first start. It does not work in case of
> CONFIG_ENV_IS_EMBEDDED, where we should skip the save. Anyway, I agree
> that this helps in User Space because it is guaranteed that a suitable
> environment is always present.

As long as it's possible to opt out, I think resiliency and ease of use
should take priority over such needs. Making the common case easy, and
the hard case possible. Also, with the patch you mentioned from Łukasz
below, if the proper environment is already in storage, no writing
should be necessary.

>> This would guarantee that the environment is available in its expected
>> storage location when entering userspace, and would also have the
>> benefit of exposing dangerous situations where a newly flashed root
>> filesystem has a /etc/fw_env.config which is wrong.
> 
> Well, if /etc/fw_env.config is broken, we cannot do a lot. But this is
> like a bug and should be solved during the test phase.

Of course. This is just a bonus, and not an argument for choosing this
approach.

> Łukasz told me yesterday that he has already sent some patches to
> OE-Core to extract the default environment, even if this is not suitable
> for an "updater" (our use case).
> 
> http://lists.openembedded.org/pipermail/openembedded-core/2018-April/150194.html
> 
> The patch is useful to generate a ready-to-flash image, but the exported
> default environment could be used as input by the env tools in user
> space when no environment is found.

-- 
Kristian

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180507/8ba3c9aa/attachment-0001.sig>