[U-Boot] Default environment file

[U-Boot] Default environment file

[Linder Pascal]

Hi everyone,


I am currently moving the configurations of the KM boards from header files to Kconfig. But for the customly defined environment variables I did not found a decent solution until I have come across the default environment file, which seems very interesting to me.


To this day, nevertheless, it appears that noone made use of the CONFIG_USE_DEFAULT_ENV_FILE configuration defined in env/Kconfig. Does anyone still have an example for this kind of environment definition or knows how to create it?


In my opinion, this could be highly relevant for the transition away from the header files in include/configs.


Sincerely,


Pascal Linder

Student Telekommunikation Netzwerke und Sicherheit

Klasse T-3b

[U-Boot] Default environment file

[Lukasz Majewski]

Hi Pascal,

> Hi everyone,
> 
> 
> I am currently moving the configurations of the KM boards from header
> files to Kconfig. But for the customly defined environment variables
> I did not found a decent solution until I have come across the
> default environment file, which seems very interesting to me.
> 
> 
> To this day, nevertheless, it appears that noone made use of the
> CONFIG_USE_DEFAULT_ENV_FILE configuration defined in env/Kconfig.
> Does anyone still have an example for this kind of environment
> definition or knows how to create it?

I do remember that somebody wanted to use this feature.

Please look into following link - providing patch description and dome
discussion:

https://patchwork.ozlabs.org/patch/888149/


Just a tip: patchwork for U-Boot project keeps together discussion and
other patches from the series. It is very convenient to look for some
additional info regarding e.g. CONFIG_* variables.

> 
> 
> In my opinion, this could be highly relevant for the transition away
> from the header files in include/configs.
> 
> 
> Sincerely,
> 
> 
> Pascal Linder
> 
> Student Telekommunikation Netzwerke und Sicherheit
> 
> Klasse T-3b
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> https://lists.denx.de/listinfo/u-boot




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma at denx.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190612/d2115f92/attachment.sig>

[U-Boot] Default environment file

[Stefano Babic]

Hi Pascal,

On 12/06/19 10:20, Linder Pascal wrote:
> Hi everyone,
> 
> 
> I am currently moving the configurations of the KM boards from header files to Kconfig. But for the customly defined environment variables I did not found a decent solution until I have come across the default environment file, which seems very interesting to me.
> 
> 
> To this day, nevertheless, it appears that noone made use of the CONFIG_USE_DEFAULT_ENV_FILE configuration defined in env/Kconfig. Does anyone still have an example for this kind of environment definition or knows how to create it?
> 
> 
> In my opinion, this could be highly relevant for the transition away from the header files in include/configs.
> 

Fully agree. Rather, I do not think there is a relevant example. But the
environment is something like data and should not be part of the header
file as it is for histoical reason. I added some times ago a way to
extract the environment from the header and make the transition easy
(see make u-boot-initial-env). And if the environment is split from the
header as CONFIG_USE_DEFAULT_ENV_FILE allows, it is also easier to set
an own environment via OE BSP layer without pushing for each small
change to U-Boot. Not only, environments often conflict, and what is
good for a project becomes evil for another one.

Best regards,
Stefano Babic

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================

[U-Boot] Default environment file

[Rasmus Villemoes]

On 12/06/2019 10.43, Stefano Babic wrote:
> Hi Pascal,
> 
> On 12/06/19 10:20, Linder Pascal wrote:
>> Hi everyone,
>>
>>
>> I am currently moving the configurations of the KM boards from header files to Kconfig. But for the customly defined environment variables I did not found a decent solution until I have come across the default environment file, which seems very interesting to me.
>>
>>
>> To this day, nevertheless, it appears that noone made use of the CONFIG_USE_DEFAULT_ENV_FILE configuration defined in env/Kconfig. Does anyone still have an example for this kind of environment definition or knows how to create it?
>>
>>
>> In my opinion, this could be highly relevant for the transition away from the header files in include/configs.
>>
> 
> Fully agree. Rather, I do not think there is a relevant example. But the
> environment is something like data and should not be part of the header
> file as it is for histoical reason. I added some times ago a way to
> extract the environment from the header and make the transition easy
> (see make u-boot-initial-env). And if the environment is split from the
> header as CONFIG_USE_DEFAULT_ENV_FILE allows, it is also easier to set
> an own environment via OE BSP layer without pushing for each small
> change to U-Boot. Not only, environments often conflict, and what is
> good for a project becomes evil for another one.

Yup. FWIW, here's a small snippet from our Yocto recipe that creates the
default env file:

ENV_INPUT = "common.txt board.txt"
ENV_INPUT_append_basic-test = " testctrl.txt"

python do_configure() {
    import fileinput
    with open("uboot.txt", "w") as out:
        for line in fileinput.input(files=d.getVar("ENV_INPUT").split()):
            line = d.expand(line)
            # Strip backslash-newline
            if line.endswith("\\\n"):
                line = line[:-2]
            out.write(line)
}

The line = d.expand(line) allows us to use ${SOME_BITBAKE_VARIABLE} in
the input files and have them replaced appropriately - this can be quite
useful to e.g. configure a tftp server address differently.

Rasmus

[U-Boot] Default environment file

[Tom Rini]

On Wed, Jun 12, 2019 at 10:43:26AM +0200, Stefano Babic wrote:
> Hi Pascal,
> 
> On 12/06/19 10:20, Linder Pascal wrote:
> > Hi everyone,
> > 
> > 
> > I am currently moving the configurations of the KM boards from header files to Kconfig. But for the customly defined environment variables I did not found a decent solution until I have come across the default environment file, which seems very interesting to me.
> > 
> > 
> > To this day, nevertheless, it appears that noone made use of the CONFIG_USE_DEFAULT_ENV_FILE configuration defined in env/Kconfig. Does anyone still have an example for this kind of environment definition or knows how to create it?
> > 
> > 
> > In my opinion, this could be highly relevant for the transition away from the header files in include/configs.
> > 
> 
> Fully agree. Rather, I do not think there is a relevant example. But the
> environment is something like data and should not be part of the header
> file as it is for histoical reason. I added some times ago a way to
> extract the environment from the header and make the transition easy
> (see make u-boot-initial-env). And if the environment is split from the
> header as CONFIG_USE_DEFAULT_ENV_FILE allows, it is also easier to set
> an own environment via OE BSP layer without pushing for each small
> change to U-Boot. Not only, environments often conflict, and what is
> good for a project becomes evil for another one.

With the high-level goal of being able to eliminate the include/configs
file, we need to figure out a better solution to dealing with the
default environment.  Shuffling things into include/environment/ has
been the first step I've tried but I'm absolutely not tied down to this
and if people are motivated to push in a new solution to this overall
problem I'm happy to see it happen.  This sounds like a good overall
idea.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190612/27e72a5e/attachment.sig>

[U-Boot] Default environment file

[Frank Wunderlich]

Hi,

i use this option to provide a default environment:

https://github.com/frank-w/u-boot/blob/89401dc85a3d1793261024a6e4047a7e4c9d079d/configs/mt7623n_bpir2_defconfig#L64

regards Frank


> Gesendet: Mittwoch, 12. Juni 2019 um 10:43 Uhr
> Von: "Stefano Babic" <sbabic@denx.de>

> Fully agree. Rather, I do not think there is a relevant example. But the
> environment is something like data and should not be part of the header
> file as it is for histoical reason. I added some times ago a way to
> extract the environment from the header and make the transition easy
> (see make u-boot-initial-env). And if the environment is split from the
> header as CONFIG_USE_DEFAULT_ENV_FILE allows, it is also easier to set
> an own environment via OE BSP layer without pushing for each small
> change to U-Boot. Not only, environments often conflict, and what is
> good for a project becomes evil for another one.

[U-Boot] Default environment file

[Stefano Babic]

Hi Tom,

Hi everybody,

On 12/06/19 16:16, Tom Rini wrote:
> On Wed, Jun 12, 2019 at 10:43:26AM +0200, Stefano Babic wrote:
>> Hi Pascal,
>>
>> On 12/06/19 10:20, Linder Pascal wrote:
>>> Hi everyone,
>>>
>>>
>>> I am currently moving the configurations of the KM boards from header files to Kconfig. But for the customly defined environment variables I did not found a decent solution until I have come across the default environment file, which seems very interesting to me.
>>>
>>>
>>> To this day, nevertheless, it appears that noone made use of the CONFIG_USE_DEFAULT_ENV_FILE configuration defined in env/Kconfig. Does anyone still have an example for this kind of environment definition or knows how to create it?
>>>
>>>
>>> In my opinion, this could be highly relevant for the transition away from the header files in include/configs.
>>>
>>
>> Fully agree. Rather, I do not think there is a relevant example. But the
>> environment is something like data and should not be part of the header
>> file as it is for histoical reason. I added some times ago a way to
>> extract the environment from the header and make the transition easy
>> (see make u-boot-initial-env). And if the environment is split from the
>> header as CONFIG_USE_DEFAULT_ENV_FILE allows, it is also easier to set
>> an own environment via OE BSP layer without pushing for each small
>> change to U-Boot. Not only, environments often conflict, and what is
>> good for a project becomes evil for another one.
> 
> With the high-level goal of being able to eliminate the include/configs
> file, we need to figure out a better solution to dealing with the
> default environment. 

Right.

> Shuffling things into include/environment/ has
> been the first step I've tried but I'm absolutely not tied down to this
> and if people are motivated to push in a new solution to this overall
> problem I'm happy to see it happen.  This sounds like a good overall
> idea.

The default / initial environment is more a configuration data for the
bootloader as part of it. Linking it to the rest of code was done at the
beginning of U-Boot and it was never changed for historical reasons, but
the environment is just configuration data.  Theoretically, we could
have the same environment for multiple boards and we could use the same
files.

IMHO it should be more a job for binman as for the linker to put
environment and u-boot code together. My first idea could be to drop it
from code and appending it to the binary, letting the code (SPL /
u-boot) know where the initial environment is found.
CONFIG_USE_DEFAULT_ENV_FILE could be used to set which file should be
taken by binman - the result is still a single file that can be signed
in case of secure boot.

Best regards,
Stefano

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================

[U-Boot] Default environment file

[Vladimir Oltean]

On Wed, 12 Jun 2019 at 19:30, Stefano Babic <sbabic@denx.de> wrote:
>
> Hi Tom,
>
> Hi everybody,
>
> On 12/06/19 16:16, Tom Rini wrote:
> > On Wed, Jun 12, 2019 at 10:43:26AM +0200, Stefano Babic wrote:
> >> Hi Pascal,
> >>
> >> On 12/06/19 10:20, Linder Pascal wrote:
> >>> Hi everyone,
> >>>
> >>>
> >>> I am currently moving the configurations of the KM boards from header files to Kconfig. But for the customly defined environment variables I did not found a decent solution until I have come across the default environment file, which seems very interesting to me.
> >>>
> >>>
> >>> To this day, nevertheless, it appears that noone made use of the CONFIG_USE_DEFAULT_ENV_FILE configuration defined in env/Kconfig. Does anyone still have an example for this kind of environment definition or knows how to create it?
> >>>
> >>>
> >>> In my opinion, this could be highly relevant for the transition away from the header files in include/configs.
> >>>
> >>
> >> Fully agree. Rather, I do not think there is a relevant example. But the
> >> environment is something like data and should not be part of the header
> >> file as it is for histoical reason. I added some times ago a way to
> >> extract the environment from the header and make the transition easy
> >> (see make u-boot-initial-env). And if the environment is split from the
> >> header as CONFIG_USE_DEFAULT_ENV_FILE allows, it is also easier to set
> >> an own environment via OE BSP layer without pushing for each small
> >> change to U-Boot. Not only, environments often conflict, and what is
> >> good for a project becomes evil for another one.
> >
> > With the high-level goal of being able to eliminate the include/configs
> > file, we need to figure out a better solution to dealing with the
> > default environment.
>
> Right.
>
> > Shuffling things into include/environment/ has
> > been the first step I've tried but I'm absolutely not tied down to this
> > and if people are motivated to push in a new solution to this overall
> > problem I'm happy to see it happen.  This sounds like a good overall
> > idea.
>
> The default / initial environment is more a configuration data for the
> bootloader as part of it. Linking it to the rest of code was done at the
> beginning of U-Boot and it was never changed for historical reasons, but
> the environment is just configuration data.  Theoretically, we could
> have the same environment for multiple boards and we could use the same
> files.
>
> IMHO it should be more a job for binman as for the linker to put
> environment and u-boot code together. My first idea could be to drop it
> from code and appending it to the binary, letting the code (SPL /
> u-boot) know where the initial environment is found.
> CONFIG_USE_DEFAULT_ENV_FILE could be used to set which file should be
> taken by binman - the result is still a single file that can be signed
> in case of secure boot.
>
> Best regards,
> Stefano
>
> --
> =====================================================================
> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
> =====================================================================
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> https://lists.denx.de/listinfo/u-boot

Hi guys,

Chiming in to the discussion as well, as I was looking to submit a new
board port of U-boot with the distro boot feature, and I was amazed by
the stupendous amount of code duplication in these include/configs/*.h
files.
Separating the default env from the main bootloader executable image
doesn't look to me like the main problem here - but rather how to
re-use common portions of environments in a way that scales for
hundreds of boards, and at the same time allow for customization?
Maybe this is a naive question, but what if we just throw the C
preprocessor at the CONFIG_DEFAULT_ENV_FILE before including it into
the U-boot image? I don't know enough about the Hush shell to realize
at this point whether it has any other overlap with the C preprocessor
than the comments (# in Hush, // or /* */ in C).
Then per-board CONFIG_DEFAULT_ENV_FILE files could sit just fine in
include/environment, with the advantage that they can be layered in a
nice inclusion hierarchy - this I believe should addresses some of
Tom's issues with this setting at least partly.

Regards,
-Vladimir

[U-Boot] Default environment file

[Stefano Babic]

Hi Vladimir,

On 12/06/19 23:33, Vladimir Oltean wrote:
> On Wed, 12 Jun 2019 at 19:30, Stefano Babic <sbabic@denx.de> wrote:
>>
>> Hi Tom,
>>
>> Hi everybody,
>>
>> On 12/06/19 16:16, Tom Rini wrote:
>>> On Wed, Jun 12, 2019 at 10:43:26AM +0200, Stefano Babic wrote:
>>>> Hi Pascal,
>>>>
>>>> On 12/06/19 10:20, Linder Pascal wrote:
>>>>> Hi everyone,
>>>>>
>>>>>
>>>>> I am currently moving the configurations of the KM boards from header files to Kconfig. But for the customly defined environment variables I did not found a decent solution until I have come across the default environment file, which seems very interesting to me.
>>>>>
>>>>>
>>>>> To this day, nevertheless, it appears that noone made use of the CONFIG_USE_DEFAULT_ENV_FILE configuration defined in env/Kconfig. Does anyone still have an example for this kind of environment definition or knows how to create it?
>>>>>
>>>>>
>>>>> In my opinion, this could be highly relevant for the transition away from the header files in include/configs.
>>>>>
>>>>
>>>> Fully agree. Rather, I do not think there is a relevant example. But the
>>>> environment is something like data and should not be part of the header
>>>> file as it is for histoical reason. I added some times ago a way to
>>>> extract the environment from the header and make the transition easy
>>>> (see make u-boot-initial-env). And if the environment is split from the
>>>> header as CONFIG_USE_DEFAULT_ENV_FILE allows, it is also easier to set
>>>> an own environment via OE BSP layer without pushing for each small
>>>> change to U-Boot. Not only, environments often conflict, and what is
>>>> good for a project becomes evil for another one.
>>>
>>> With the high-level goal of being able to eliminate the include/configs
>>> file, we need to figure out a better solution to dealing with the
>>> default environment.
>>
>> Right.
>>
>>> Shuffling things into include/environment/ has
>>> been the first step I've tried but I'm absolutely not tied down to this
>>> and if people are motivated to push in a new solution to this overall
>>> problem I'm happy to see it happen.  This sounds like a good overall
>>> idea.
>>
>> The default / initial environment is more a configuration data for the
>> bootloader as part of it. Linking it to the rest of code was done at the
>> beginning of U-Boot and it was never changed for historical reasons, but
>> the environment is just configuration data.  Theoretically, we could
>> have the same environment for multiple boards and we could use the same
>> files.
>>
>> IMHO it should be more a job for binman as for the linker to put
>> environment and u-boot code together. My first idea could be to drop it
>> from code and appending it to the binary, letting the code (SPL /
>> u-boot) know where the initial environment is found.
>> CONFIG_USE_DEFAULT_ENV_FILE could be used to set which file should be
>> taken by binman - the result is still a single file that can be signed
>> in case of secure boot.
>>
>> Best regards,
>> Stefano
>>
>> --
>> =====================================================================
>> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
>> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
>> Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
>> =====================================================================
>> _______________________________________________
>> U-Boot mailing list
>> U-Boot at lists.denx.de
>> https://lists.denx.de/listinfo/u-boot
> 
> Hi guys,
> 
> Chiming in to the discussion as well, as I was looking to submit a new
> board port of U-boot with the distro boot feature, and I was amazed by
> the stupendous amount of code duplication in these include/configs/*.h
> files.
> Separating the default env from the main bootloader executable image
> doesn't look to me like the main problem here - 

It is part of the problem - the environment is really data for the
bootloader, and it is mixed with code. It make sense to have one
bootloader and exchange just the data, as usually for each program.

IMHO it is also quite nasty that this initial environment is set with an
array of strings, and the board maintainer should be careful adding the
'\0' to each command. How many times led this to errors due to
mishandling of the stings ?

And U-Boot does not use this internally - "env import" parses an ASCII
file from the memory.

> but rather how to
> re-use common portions of environments in a way that scales for
> hundreds of boards, and at the same time allow for customization?

Yes, and not only. This is maybe ok for board vendors, but think about
to the very common use case where a board is used for many projects.
Each project often needs its own environment for many reason. It should
be easy to exchange the environment without pushing it to u-boot, but
just use the bootloader as "code" and add the own "data".

Reusing can be reached if CONFIG_DEFAULT_ENV_FILE points to the same
file (for example) or concatenating more files into one (something is
done now with distro boot).

Just looking at i.MX, there are tons of patches just to change some
variables in the environment. They do not change code. IMHO it should be
possible to have the same "code" (linked binary) and simply exchange the
data (the initial environment).

> Maybe this is a naive question, but what if we just throw the C
> preprocessor at the CONFIG_DEFAULT_ENV_FILE before including it into
> the U-boot image? 

My question is why we need the C compiler / preprocessor at all just to
get the environment. We have also tools like "mkenvimage" that can
generate a suitable image.

> I don't know enough about the Hush shell to realize
> at this point whether it has any other overlap with the C preprocessor
> than the comments (# in Hush, // or /* */ in C).
> Then per-board CONFIG_DEFAULT_ENV_FILE files could sit just fine in
> include/environment, with the advantage that they can be layered in a
> nice inclusion hierarchy - this I believe should addresses some of
> Tom's issues with this setting at least partly.

Best regards,
Stefano


-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================