[PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Lukasz Majewski]

This commit provides the ability to generate u-boot environment(s) as
images, which afterwards can be used to produce image (with wic) for
flashing (eMMC or SPI-NOR).

This change removes the need to run "env default" during production phase,
as proper environment (including redundant one) is already stored on
persistent memory (the CRC is also correct).

Signed-off-by: Lukasz Majewski <lukma@denx.de>

---
This patch depends on "u-boot: Upgrade to 2018.03 release"
https://patchwork.openembedded.org/patch/149998/
---
 meta/recipes-bsp/u-boot/u-boot.inc | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/meta/recipes-bsp/u-boot/u-boot.inc b/meta/recipes-bsp/u-boot/u-boot.inc
index c2bcf99840..2796e503cf 100644
--- a/meta/recipes-bsp/u-boot/u-boot.inc
+++ b/meta/recipes-bsp/u-boot/u-boot.inc
@@ -305,3 +305,38 @@ do_deploy () {
 }
 
 addtask deploy before do_build after do_compile
+
+# Create new rules to extract default envs
+UBOOT_ENVS_DEFAULT ?= "uboot-envs-default"
+DEFAULT_ENVS ?= "u-boot-env-default.txt"
+DEFAULT_ENVS_SIZE ?= "65536"
+
+# Generate default environment
+do_gen_default_envs[doc] = "Generate image with default U-Boot environment(s)"
+do_gen_default_envs () {
+	${B}/source/scripts/get_default_envs.sh ${B} > ${B}/${DEFAULT_ENVS}
+
+	# Generate env image
+	${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}
+
+	# Generate redundant env image
+	${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}
+
+	rm ${B}/${DEFAULT_ENVS}
+}
+
+addtask gen_default_envs before do_deploy_default_envs after do_compile
+
+# Deploy default environment
+do_deploy_default_envs[doc] = "Copy images with default U-Boot environment to deployment directory"
+do_deploy_default_envs () {
+	install -d ${DEPLOYDIR}
+
+	install ${B}/${UBOOT_ENVS_DEFAULT} ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}
+	install ${B}/${UBOOT_ENVS_DEFAULT}_r ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r
+
+	rm ${B}/${UBOOT_ENVS_DEFAULT}
+	rm ${B}/${UBOOT_ENVS_DEFAULT}_r
+}
+
+addtask deploy_default_envs before do_deploy after do_gen_default_envs
-- 
2.11.0

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Marek Vasut]

On 04/27/2018 04:51 PM, Lukasz Majewski wrote:
> This commit provides the ability to generate u-boot environment(s) as
> images, which afterwards can be used to produce image (with wic) for
> flashing (eMMC or SPI-NOR).
> 
> This change removes the need to run "env default" during production phase,
> as proper environment (including redundant one) is already stored on
> persistent memory (the CRC is also correct).
> 
> Signed-off-by: Lukasz Majewski <lukma@denx.de>

If your default env is correct, why do you need this ? I can see some
use with non-default env, but then that can be wrapped into a separate
recipe.

> ---
> This patch depends on "u-boot: Upgrade to 2018.03 release"
> https://patchwork.openembedded.org/patch/149998/
> ---
>  meta/recipes-bsp/u-boot/u-boot.inc | 35 +++++++++++++++++++++++++++++++++++
>  1 file changed, 35 insertions(+)
> 
> diff --git a/meta/recipes-bsp/u-boot/u-boot.inc b/meta/recipes-bsp/u-boot/u-boot.inc
> index c2bcf99840..2796e503cf 100644
> --- a/meta/recipes-bsp/u-boot/u-boot.inc
> +++ b/meta/recipes-bsp/u-boot/u-boot.inc
> @@ -305,3 +305,38 @@ do_deploy () {
>  }
>  
>  addtask deploy before do_build after do_compile
> +
> +# Create new rules to extract default envs
> +UBOOT_ENVS_DEFAULT ?= "uboot-envs-default"
> +DEFAULT_ENVS ?= "u-boot-env-default.txt"
> +DEFAULT_ENVS_SIZE ?= "65536"
> +
> +# Generate default environment
> +do_gen_default_envs[doc] = "Generate image with default U-Boot environment(s)"
> +do_gen_default_envs () {
> +	${B}/source/scripts/get_default_envs.sh ${B} > ${B}/${DEFAULT_ENVS}
> +
> +	# Generate env image
> +	${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}

Does this actually work during cross build , when mkenvimage
architecture is different than host architecture ?

> +	# Generate redundant env image
> +	${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}

Is redundant env always needed on all systems ?

> +	rm ${B}/${DEFAULT_ENVS}
> +}
> +
> +addtask gen_default_envs before do_deploy_default_envs after do_compile
> +
> +# Deploy default environment
> +do_deploy_default_envs[doc] = "Copy images with default U-Boot environment to deployment directory"
> +do_deploy_default_envs () {
> +	install -d ${DEPLOYDIR}
> +
> +	install ${B}/${UBOOT_ENVS_DEFAULT} ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}
> +	install ${B}/${UBOOT_ENVS_DEFAULT}_r ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r

Does this work with multiple machines or will it overwrite the deployed
image ?


> +	rm ${B}/${UBOOT_ENVS_DEFAULT}
> +	rm ${B}/${UBOOT_ENVS_DEFAULT}_r
> +}
> +
> +addtask deploy_default_envs before do_deploy after do_gen_default_envs
> 


-- 
Best regards,
Marek Vasut

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 4020 bytes --]

Hi Marek,

Thanks for prompt feedback.

> On 04/27/2018 04:51 PM, Lukasz Majewski wrote:
> > This commit provides the ability to generate u-boot environment(s)
> > as images, which afterwards can be used to produce image (with wic)
> > for flashing (eMMC or SPI-NOR).
> > 
> > This change removes the need to run "env default" during production
> > phase, as proper environment (including redundant one) is already
> > stored on persistent memory (the CRC is also correct).
> > 
> > Signed-off-by: Lukasz Majewski <lukma@denx.de>  
> 
> If your default env is correct, why do you need this ? 

Some users wants to have the working board (i.e. without any warnings
that CRC for envs is bad) after flashing the SPI-NOR/eMMC

> I can see some
> use with non-default env, but then that can be wrapped into a separate
> recipe.

I can add this functionality as a separate recipe. This is v1, so ideas
are welcome.

> 
> > ---
> > This patch depends on "u-boot: Upgrade to 2018.03 release"
> > https://patchwork.openembedded.org/patch/149998/
> > ---
> >  meta/recipes-bsp/u-boot/u-boot.inc | 35
> > +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
> > 
> > diff --git a/meta/recipes-bsp/u-boot/u-boot.inc
> > b/meta/recipes-bsp/u-boot/u-boot.inc index c2bcf99840..2796e503cf
> > 100644 --- a/meta/recipes-bsp/u-boot/u-boot.inc
> > +++ b/meta/recipes-bsp/u-boot/u-boot.inc
> > @@ -305,3 +305,38 @@ do_deploy () {
> >  }
> >  
> >  addtask deploy before do_build after do_compile
> > +
> > +# Create new rules to extract default envs
> > +UBOOT_ENVS_DEFAULT ?= "uboot-envs-default"
> > +DEFAULT_ENVS ?= "u-boot-env-default.txt"
> > +DEFAULT_ENVS_SIZE ?= "65536"
> > +
> > +# Generate default environment
> > +do_gen_default_envs[doc] = "Generate image with default U-Boot
> > environment(s)" +do_gen_default_envs () {
> > +	${B}/source/scripts/get_default_envs.sh ${B} >
> > ${B}/${DEFAULT_ENVS} +
> > +	# Generate env image
> > +	${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o
> > ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}  
> 
> Does this actually work during cross build , when mkenvimage
> architecture is different than host architecture ?

Yes. This was tested.

> 
> > +	# Generate redundant env image
> > +	${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o
> > ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}  
> 
> Is redundant env always needed on all systems ?

No, they are not. However, it shall not be a problem if it is build
anyway - user can adjust wic to only put primary env image.

> 
> > +	rm ${B}/${DEFAULT_ENVS}
> > +}
> > +
> > +addtask gen_default_envs before do_deploy_default_envs after
> > do_compile +
> > +# Deploy default environment
> > +do_deploy_default_envs[doc] = "Copy images with default U-Boot
> > environment to deployment directory" +do_deploy_default_envs () {
> > +	install -d ${DEPLOYDIR}
> > +
> > +	install ${B}/${UBOOT_ENVS_DEFAULT}
> > ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}
> > +	install ${B}/${UBOOT_ENVS_DEFAULT}_r
> > ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r  
> 
> Does this work with multiple machines 

Unfortunately not.

For multiple machines one needs to add ${target}, which would add
machine name into path:

target=${MACHINE}_defconfig

and change ${B} -> ${B}/${target}


I suppose, that I would need to adjust this script to be similar to
do_install() and do_compile() in this matter.

> or will it overwrite the
> deployed image ?

As it is now - it will overwrite the image.

> 
> 
> > +	rm ${B}/${UBOOT_ENVS_DEFAULT}
> > +	rm ${B}/${UBOOT_ENVS_DEFAULT}_r
> > +}
> > +
> > +addtask deploy_default_envs before do_deploy after
> > do_gen_default_envs 
> 
> 




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 499 bytes --]

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Marek Vasut]

On 04/27/2018 06:15 PM, Lukasz Majewski wrote:
> Hi Marek,
> 
> Thanks for prompt feedback.
> 
>> On 04/27/2018 04:51 PM, Lukasz Majewski wrote:
>>> This commit provides the ability to generate u-boot environment(s)
>>> as images, which afterwards can be used to produce image (with wic)
>>> for flashing (eMMC or SPI-NOR).
>>>
>>> This change removes the need to run "env default" during production
>>> phase, as proper environment (including redundant one) is already
>>> stored on persistent memory (the CRC is also correct).
>>>
>>> Signed-off-by: Lukasz Majewski <lukma@denx.de>  
>>
>> If your default env is correct, why do you need this ? 
> 
> Some users wants to have the working board (i.e. without any warnings
> that CRC for envs is bad) after flashing the SPI-NOR/eMMC

And the board is not working when it tells you your env is not
populated? Then maybe the board port is broken.

>> I can see some
>> use with non-default env, but then that can be wrapped into a separate
>> recipe.
> 
> I can add this functionality as a separate recipe. This is v1, so ideas
> are welcome.

I think that only makes sense.

>>> ---
>>> This patch depends on "u-boot: Upgrade to 2018.03 release"
>>> https://patchwork.openembedded.org/patch/149998/
>>> ---
>>>  meta/recipes-bsp/u-boot/u-boot.inc | 35
>>> +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
>>>
>>> diff --git a/meta/recipes-bsp/u-boot/u-boot.inc
>>> b/meta/recipes-bsp/u-boot/u-boot.inc index c2bcf99840..2796e503cf
>>> 100644 --- a/meta/recipes-bsp/u-boot/u-boot.inc
>>> +++ b/meta/recipes-bsp/u-boot/u-boot.inc
>>> @@ -305,3 +305,38 @@ do_deploy () {
>>>  }
>>>  
>>>  addtask deploy before do_build after do_compile
>>> +
>>> +# Create new rules to extract default envs
>>> +UBOOT_ENVS_DEFAULT ?= "uboot-envs-default"
>>> +DEFAULT_ENVS ?= "u-boot-env-default.txt"
>>> +DEFAULT_ENVS_SIZE ?= "65536"
>>> +
>>> +# Generate default environment
>>> +do_gen_default_envs[doc] = "Generate image with default U-Boot
>>> environment(s)" +do_gen_default_envs () {
>>> +	${B}/source/scripts/get_default_envs.sh ${B} >
>>> ${B}/${DEFAULT_ENVS} +
>>> +	# Generate env image
>>> +	${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o
>>> ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}  
>>
>> Does this actually work during cross build , when mkenvimage
>> architecture is different than host architecture ?
> 
> Yes. This was tested.

How can it work if this binary cannot be executed ? :)

>>> +	# Generate redundant env image
>>> +	${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o
>>> ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}  
>>
>> Is redundant env always needed on all systems ?
> 
> No, they are not. However, it shall not be a problem if it is build
> anyway - user can adjust wic to only put primary env image.

So many wasted CPU cycles for a feature barely anyone needs. Nope :)

>>> +	rm ${B}/${DEFAULT_ENVS}
>>> +}
>>> +
>>> +addtask gen_default_envs before do_deploy_default_envs after
>>> do_compile +
>>> +# Deploy default environment
>>> +do_deploy_default_envs[doc] = "Copy images with default U-Boot
>>> environment to deployment directory" +do_deploy_default_envs () {
>>> +	install -d ${DEPLOYDIR}
>>> +
>>> +	install ${B}/${UBOOT_ENVS_DEFAULT}
>>> ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}
>>> +	install ${B}/${UBOOT_ENVS_DEFAULT}_r
>>> ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r  
>>
>> Does this work with multiple machines 
> 
> Unfortunately not.
> 
> For multiple machines one needs to add ${target}, which would add
> machine name into path:
> 
> target=${MACHINE}_defconfig
> 
> and change ${B} -> ${B}/${target}
> 
> 
> I suppose, that I would need to adjust this script to be similar to
> do_install() and do_compile() in this matter.

Yep

>> or will it overwrite the
>> deployed image ?
> 
> As it is now - it will overwrite the image.
Yep

-- 
Best regards,
Marek Vasut

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 5238 bytes --]

Hi Marek,

> On 04/27/2018 06:15 PM, Lukasz Majewski wrote:
> > Hi Marek,
> > 
> > Thanks for prompt feedback.
> >   
> >> On 04/27/2018 04:51 PM, Lukasz Majewski wrote:  
> >>> This commit provides the ability to generate u-boot environment(s)
> >>> as images, which afterwards can be used to produce image (with
> >>> wic) for flashing (eMMC or SPI-NOR).
> >>>
> >>> This change removes the need to run "env default" during
> >>> production phase, as proper environment (including redundant one)
> >>> is already stored on persistent memory (the CRC is also correct).
> >>>
> >>> Signed-off-by: Lukasz Majewski <lukma@denx.de>    
> >>
> >> If your default env is correct, why do you need this ?   
> > 
> > Some users wants to have the working board (i.e. without any
> > warnings that CRC for envs is bad) after flashing the SPI-NOR/eMMC  
> 
> And the board is not working when it tells you your env is not
> populated? 

It complains that CRC is wrong, which is true as without this patch
(and some wic adjustments) we don't have proper envs in expected
non-volatile area.

> Then maybe the board port is broken.

Then it proceeds with default envs - and boots.

> 
> >> I can see some
> >> use with non-default env, but then that can be wrapped into a
> >> separate recipe.  
> > 
> > I can add this functionality as a separate recipe. This is v1, so
> > ideas are welcome.  
> 
> I think that only makes sense.

Ok.

> 
> >>> ---
> >>> This patch depends on "u-boot: Upgrade to 2018.03 release"
> >>> https://patchwork.openembedded.org/patch/149998/
> >>> ---
> >>>  meta/recipes-bsp/u-boot/u-boot.inc | 35
> >>> +++++++++++++++++++++++++++++++++++ 1 file changed, 35
> >>> insertions(+)
> >>>
> >>> diff --git a/meta/recipes-bsp/u-boot/u-boot.inc
> >>> b/meta/recipes-bsp/u-boot/u-boot.inc index
> >>> c2bcf99840.cdd7703db9ea580308d4ebcd6d0ae3209b761daa.2796e503cf
> >>> 100644 --- a/meta/recipes-bsp/u-boot/u-boot.inc +++
> >>> b/meta/recipes-bsp/u-boot/u-boot.inc @@ -305,3 +305,38 @@
> >>> do_deploy () { }
> >>>  
> >>>  addtask deploy before do_build after do_compile
> >>> +
> >>> +# Create new rules to extract default envs
> >>> +UBOOT_ENVS_DEFAULT ?= "uboot-envs-default"
> >>> +DEFAULT_ENVS ?= "u-boot-env-default.txt"
> >>> +DEFAULT_ENVS_SIZE ?= "65536"
> >>> +
> >>> +# Generate default environment
> >>> +do_gen_default_envs[doc] = "Generate image with default U-Boot
> >>> environment(s)" +do_gen_default_envs () {
> >>> +	${B}/source/scripts/get_default_envs.sh ${B} >
> >>> ${B}/${DEFAULT_ENVS} +
> >>> +	# Generate env image
> >>> +	${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o
> >>> ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}    
> >>
> >> Does this actually work during cross build , when mkenvimage
> >> architecture is different than host architecture ?  
> > 
> > Yes. This was tested.  
> 
> How can it work if this binary cannot be executed ? :)

Could you be more specific here?

I do follow normal u-boot recipe build. If mkenvimage is build in a
wrong way (as other tools) - then I suppose that the whole u-boot recipe
is broken.

The get_default_envs.sh script is prepared to use either native or
cross-compiled set of binutils (that thought I had in mind when I said
that it was tested).

> 
> >>> +	# Generate redundant env image
> >>> +	${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o
> >>> ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}    
> >>
> >> Is redundant env always needed on all systems ?  
> > 
> > No, they are not. However, it shall not be a problem if it is build
> > anyway - user can adjust wic to only put primary env image.  
> 
> So many wasted CPU cycles for a feature barely anyone needs. Nope :)

I will try to craft some OE magic to make it not build when not
needed... :-)

> 
> >>> +	rm ${B}/${DEFAULT_ENVS}
> >>> +}
> >>> +
> >>> +addtask gen_default_envs before do_deploy_default_envs after
> >>> do_compile +
> >>> +# Deploy default environment
> >>> +do_deploy_default_envs[doc] = "Copy images with default U-Boot
> >>> environment to deployment directory" +do_deploy_default_envs () {
> >>> +	install -d ${DEPLOYDIR}
> >>> +
> >>> +	install ${B}/${UBOOT_ENVS_DEFAULT}
> >>> ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}
> >>> +	install ${B}/${UBOOT_ENVS_DEFAULT}_r
> >>> ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r    
> >>
> >> Does this work with multiple machines   
> > 
> > Unfortunately not.
> > 
> > For multiple machines one needs to add ${target}, which would add
> > machine name into path:
> > 
> > target=${MACHINE}_defconfig
> > 
> > and change ${B} -> ${B}/${target}
> > 
> > 
> > I suppose, that I would need to adjust this script to be similar to
> > do_install() and do_compile() in this matter.  
> 
> Yep

Ok.

> 
> >> or will it overwrite the
> >> deployed image ?  
> > 
> > As it is now - it will overwrite the image.  
> Yep
> 

Ok.


Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 499 bytes --]

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Marek Vasut]

On 04/29/2018 03:53 PM, Lukasz Majewski wrote:
> Hi Marek,

Hello Lukasz,

>> On 04/27/2018 06:15 PM, Lukasz Majewski wrote:
>>> Hi Marek,
>>>
>>> Thanks for prompt feedback.
>>>   
>>>> On 04/27/2018 04:51 PM, Lukasz Majewski wrote:  
>>>>> This commit provides the ability to generate u-boot environment(s)
>>>>> as images, which afterwards can be used to produce image (with
>>>>> wic) for flashing (eMMC or SPI-NOR).
>>>>>
>>>>> This change removes the need to run "env default" during
>>>>> production phase, as proper environment (including redundant one)
>>>>> is already stored on persistent memory (the CRC is also correct).
>>>>>
>>>>> Signed-off-by: Lukasz Majewski <lukma@denx.de>    
>>>>
>>>> If your default env is correct, why do you need this ?   
>>>
>>> Some users wants to have the working board (i.e. without any
>>> warnings that CRC for envs is bad) after flashing the SPI-NOR/eMMC  
>>
>> And the board is not working when it tells you your env is not
>> populated? 
> 
> It complains that CRC is wrong, which is true as without this patch
> (and some wic adjustments) we don't have proper envs in expected
> non-volatile area.

This is NOT an error, this is warning and a correct behavior. If your
default U-Boot environment is not correct, that is a bug in your U-Boot
port.

If you need to adjust that environment later in the product life cycle,
that's a different topic.

>> Then maybe the board port is broken.
> 
> Then it proceeds with default envs - and boots.

I think I might not quite understand the problem anymore. Is the issue
here that U-Boot informs you that it's using default environment which
you do not like ?

[...]

>>>>> +# Generate default environment
>>>>> +do_gen_default_envs[doc] = "Generate image with default U-Boot
>>>>> environment(s)" +do_gen_default_envs () {
>>>>> +	${B}/source/scripts/get_default_envs.sh ${B} >
>>>>> ${B}/${DEFAULT_ENVS} +
>>>>> +	# Generate env image
>>>>> +	${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o
>>>>> ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}    
>>>>
>>>> Does this actually work during cross build , when mkenvimage
>>>> architecture is different than host architecture ?  
>>>
>>> Yes. This was tested.  
>>
>> How can it work if this binary cannot be executed ? :)
> 
> Could you be more specific here?
> 
> I do follow normal u-boot recipe build. If mkenvimage is build in a
> wrong way (as other tools) - then I suppose that the whole u-boot recipe
> is broken.
> 
> The get_default_envs.sh script is prepared to use either native or
> cross-compiled set of binutils (that thought I had in mind when I said
> that it was tested).

Consider you crosscompile mkenvimage for ARM and try to run it on x86 host.

>>>>> +	# Generate redundant env image
>>>>> +	${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o
>>>>> ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}    
>>>>
>>>> Is redundant env always needed on all systems ?  
>>>
>>> No, they are not. However, it shall not be a problem if it is build
>>> anyway - user can adjust wic to only put primary env image.  
>>
>> So many wasted CPU cycles for a feature barely anyone needs. Nope :)
> 
> I will try to craft some OE magic to make it not build when not
> needed... :-)

Can't you just patch a correct default env into the U-Boot port ?

-- 
Best regards,
Marek Vasut

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Otavio Salvador]

On Fri, Apr 27, 2018 at 2:37 PM Marek Vasut <marex@denx.de> wrote:

> On 04/27/2018 06:15 PM, Lukasz Majewski wrote:
> > I can add this functionality as a separate recipe. This is v1, so ideas
> > are welcome.

> I think that only makes sense.

Also, we likely need to make use of the mkimage recipe so we use the tools
and not build it all again.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Marek Vasut]

On 04/30/2018 04:03 PM, Otavio Salvador wrote:
> On Fri, Apr 27, 2018 at 2:37 PM Marek Vasut <marex@denx.de> wrote:
> 
>> On 04/27/2018 06:15 PM, Lukasz Majewski wrote:
>>> I can add this functionality as a separate recipe. This is v1, so ideas
>>> are welcome.
> 
>> I think that only makes sense.
> 
> Also, we likely need to make use of the mkimage recipe so we use the tools
> and not build it all again.

And -- based on the offlist discussion -- I realized that env tools are
machine specific, they bundle default env with them.

-- 
Best regards,
Marek Vasut

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Stefano Babic]

On 27/04/2018 17:07, Marek Vasut wrote:
> On 04/27/2018 04:51 PM, Lukasz Majewski wrote:
>> This commit provides the ability to generate u-boot environment(s) as
>> images, which afterwards can be used to produce image (with wic) for
>> flashing (eMMC or SPI-NOR).
>>
>> This change removes the need to run "env default" during production phase,
>> as proper environment (including redundant one) is already stored on
>> persistent memory (the CRC is also correct).
>>
>> Signed-off-by: Lukasz Majewski <lukma@denx.de>
> 
> If your default env is correct, why do you need this ? I can see some
> use with non-default env, but then that can be wrapped into a separate
> recipe.
> 

A use case is when the environment must be changed from user space.
fw_setenv will report the CRC error and it needs the default environment
to add changes. The default environment is linked together to fw_setenv,
but this prohibites to use fw_setenv for multiple boards and must be
explicitely built for that machine and with the same sources as u-boot
(at least, they must share the same CONFIG_EXTRA_ENV). If the default
environment is extracted, we could have a general (distro ?) fw_setenv.

>> ---
>> This patch depends on "u-boot: Upgrade to 2018.03 release"
>> https://patchwork.openembedded.org/patch/149998/
>> ---
>>  meta/recipes-bsp/u-boot/u-boot.inc | 35 +++++++++++++++++++++++++++++++++++
>>  1 file changed, 35 insertions(+)
>>
>> diff --git a/meta/recipes-bsp/u-boot/u-boot.inc b/meta/recipes-bsp/u-boot/u-boot.inc
>> index c2bcf99840..2796e503cf 100644
>> --- a/meta/recipes-bsp/u-boot/u-boot.inc
>> +++ b/meta/recipes-bsp/u-boot/u-boot.inc
>> @@ -305,3 +305,38 @@ do_deploy () {
>>  }
>>  
>>  addtask deploy before do_build after do_compile
>> +
>> +# Create new rules to extract default envs
>> +UBOOT_ENVS_DEFAULT ?= "uboot-envs-default"
>> +DEFAULT_ENVS ?= "u-boot-env-default.txt"
>> +DEFAULT_ENVS_SIZE ?= "65536"
>> +
>> +# Generate default environment
>> +do_gen_default_envs[doc] = "Generate image with default U-Boot environment(s)"
>> +do_gen_default_envs () {
>> +	${B}/source/scripts/get_default_envs.sh ${B} > ${B}/${DEFAULT_ENVS}
>> +
>> +	# Generate env image
>> +	${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}
> 
> Does this actually work during cross build , when mkenvimage
> architecture is different than host architecture ?
> 
>> +	# Generate redundant env image
>> +	${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}
> 
> Is redundant env always needed on all systems ?
> 
>> +	rm ${B}/${DEFAULT_ENVS}
>> +}
>> +
>> +addtask gen_default_envs before do_deploy_default_envs after do_compile
>> +
>> +# Deploy default environment
>> +do_deploy_default_envs[doc] = "Copy images with default U-Boot environment to deployment directory"
>> +do_deploy_default_envs () {
>> +	install -d ${DEPLOYDIR}
>> +
>> +	install ${B}/${UBOOT_ENVS_DEFAULT} ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}
>> +	install ${B}/${UBOOT_ENVS_DEFAULT}_r ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r
> 
> Does this work with multiple machines or will it overwrite the deployed
> image ?
> 
> 
>> +	rm ${B}/${UBOOT_ENVS_DEFAULT}
>> +	rm ${B}/${UBOOT_ENVS_DEFAULT}_r
>> +}
>> +
>> +addtask deploy_default_envs before do_deploy after do_gen_default_envs
>>
> 
> 

Regards,
Stefano

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
=====================================================================

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Marek Vasut]

On 05/03/2018 06:28 PM, Stefano Babic wrote:
> On 27/04/2018 17:07, Marek Vasut wrote:
>> On 04/27/2018 04:51 PM, Lukasz Majewski wrote:
>>> This commit provides the ability to generate u-boot environment(s) as
>>> images, which afterwards can be used to produce image (with wic) for
>>> flashing (eMMC or SPI-NOR).
>>>
>>> This change removes the need to run "env default" during production phase,
>>> as proper environment (including redundant one) is already stored on
>>> persistent memory (the CRC is also correct).
>>>
>>> Signed-off-by: Lukasz Majewski <lukma@denx.de>
>>
>> If your default env is correct, why do you need this ? I can see some
>> use with non-default env, but then that can be wrapped into a separate
>> recipe.
>>
> 
> A use case is when the environment must be changed from user space.
> fw_setenv will report the CRC error and it needs the default environment
> to add changes. The default environment is linked together to fw_setenv,
> but this prohibites to use fw_setenv for multiple boards and must be
> explicitely built for that machine and with the same sources as u-boot
> (at least, they must share the same CONFIG_EXTRA_ENV). If the default
> environment is extracted, we could have a general (distro ?) fw_setenv.

I think in that case, the real solution is to either build fw_setenv per
machine OR fix fw_setenv to take env defaults from a file or somesuch ?

[...]
-- 
Best regards,
Marek Vasut

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Stefano Babic]

On 03/05/2018 18:36, Marek Vasut wrote:
> On 05/03/2018 06:28 PM, Stefano Babic wrote:
>> On 27/04/2018 17:07, Marek Vasut wrote:
>>> On 04/27/2018 04:51 PM, Lukasz Majewski wrote:
>>>> This commit provides the ability to generate u-boot environment(s) as
>>>> images, which afterwards can be used to produce image (with wic) for
>>>> flashing (eMMC or SPI-NOR).
>>>>
>>>> This change removes the need to run "env default" during production phase,
>>>> as proper environment (including redundant one) is already stored on
>>>> persistent memory (the CRC is also correct).
>>>>
>>>> Signed-off-by: Lukasz Majewski <lukma@denx.de>
>>>
>>> If your default env is correct, why do you need this ? I can see some
>>> use with non-default env, but then that can be wrapped into a separate
>>> recipe.
>>>
>>
>> A use case is when the environment must be changed from user space.
>> fw_setenv will report the CRC error and it needs the default environment
>> to add changes. The default environment is linked together to fw_setenv,
>> but this prohibites to use fw_setenv for multiple boards and must be
>> explicitely built for that machine and with the same sources as u-boot
>> (at least, they must share the same CONFIG_EXTRA_ENV). If the default
>> environment is extracted, we could have a general (distro ?) fw_setenv.
> 
> I think in that case, the real solution is to either build fw_setenv per
> machine 

This is how we try to do now, fw_setenv is built per machine but it is
enough that u-boot-fw-utils is built in a different version as u-boot to
get a mess.

> OR fix fw_setenv to take env defaults from a file or somesuch ?

Right, I interprete this patch as a step in this direction. This patch
generates a default that can be used as input for fw_setenv.

Regards,
Stefano

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
=====================================================================

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Marek Vasut]

On 05/03/2018 06:50 PM, Stefano Babic wrote:
> On 03/05/2018 18:36, Marek Vasut wrote:
>> On 05/03/2018 06:28 PM, Stefano Babic wrote:
>>> On 27/04/2018 17:07, Marek Vasut wrote:
>>>> On 04/27/2018 04:51 PM, Lukasz Majewski wrote:
>>>>> This commit provides the ability to generate u-boot environment(s) as
>>>>> images, which afterwards can be used to produce image (with wic) for
>>>>> flashing (eMMC or SPI-NOR).
>>>>>
>>>>> This change removes the need to run "env default" during production phase,
>>>>> as proper environment (including redundant one) is already stored on
>>>>> persistent memory (the CRC is also correct).
>>>>>
>>>>> Signed-off-by: Lukasz Majewski <lukma@denx.de>
>>>>
>>>> If your default env is correct, why do you need this ? I can see some
>>>> use with non-default env, but then that can be wrapped into a separate
>>>> recipe.
>>>>
>>>
>>> A use case is when the environment must be changed from user space.
>>> fw_setenv will report the CRC error and it needs the default environment
>>> to add changes. The default environment is linked together to fw_setenv,
>>> but this prohibites to use fw_setenv for multiple boards and must be
>>> explicitely built for that machine and with the same sources as u-boot
>>> (at least, they must share the same CONFIG_EXTRA_ENV). If the default
>>> environment is extracted, we could have a general (distro ?) fw_setenv.
>>
>> I think in that case, the real solution is to either build fw_setenv per
>> machine 
> 
> This is how we try to do now, fw_setenv is built per machine but it is
> enough that u-boot-fw-utils is built in a different version as u-boot to
> get a mess.

Well yes, if you mix and match packages, it becomes a mess. Isn't that
to be expected ?

>> OR fix fw_setenv to take env defaults from a file or somesuch ?
> 
> Right, I interprete this patch as a step in this direction. This patch
> generates a default that can be used as input for fw_setenv.

It generates environment images which can be written -- on certain
specific setups -- into the flash. It doesn't generate any sort of input
for the fw_setenv to my knowledge ?

-- 
Best regards,
Marek Vasut

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 3347 bytes --]

Hi Marek, Stefano,

> On 05/03/2018 06:50 PM, Stefano Babic wrote:
> > On 03/05/2018 18:36, Marek Vasut wrote:  
> >> On 05/03/2018 06:28 PM, Stefano Babic wrote:  
> >>> On 27/04/2018 17:07, Marek Vasut wrote:  
> >>>> On 04/27/2018 04:51 PM, Lukasz Majewski wrote:  
> >>>>> This commit provides the ability to generate u-boot
> >>>>> environment(s) as images, which afterwards can be used to
> >>>>> produce image (with wic) for flashing (eMMC or SPI-NOR).
> >>>>>
> >>>>> This change removes the need to run "env default" during
> >>>>> production phase, as proper environment (including redundant
> >>>>> one) is already stored on persistent memory (the CRC is also
> >>>>> correct).
> >>>>>
> >>>>> Signed-off-by: Lukasz Majewski <lukma@denx.de>  
> >>>>
> >>>> If your default env is correct, why do you need this ? I can see
> >>>> some use with non-default env, but then that can be wrapped into
> >>>> a separate recipe.
> >>>>  
> >>>
> >>> A use case is when the environment must be changed from user
> >>> space. fw_setenv will report the CRC error and it needs the
> >>> default environment to add changes. The default environment is
> >>> linked together to fw_setenv, but this prohibites to use
> >>> fw_setenv for multiple boards and must be explicitely built for
> >>> that machine and with the same sources as u-boot (at least, they
> >>> must share the same CONFIG_EXTRA_ENV). If the default environment
> >>> is extracted, we could have a general (distro ?) fw_setenv.  
> >>
> >> I think in that case, the real solution is to either build
> >> fw_setenv per machine   
> > 
> > This is how we try to do now, fw_setenv is built per machine but it
> > is enough that u-boot-fw-utils is built in a different version as
> > u-boot to get a mess.  
> 
> Well yes, if you mix and match packages, it becomes a mess. Isn't that
> to be expected ?
> 
> >> OR fix fw_setenv to take env defaults from a file or somesuch ?  
> > 
> > Right, I interprete this patch as a step in this direction. This
> > patch generates a default that can be used as input for fw_setenv.  
> 
> It generates environment images which can be written -- on certain
> specific setups -- into the flash. It doesn't generate any sort of
> input for the fw_setenv to my knowledge ?
> 

I think that it would be great if:

1. We would have this code as a separate recipe - as suggested by Marek
and Stefano already. This recipe would end up as a package to be
installed on the rootfs

2. As input I would use default_envs.txt (or any other name) - either
extracted from u-boot build or provided from external file

3. For now I do use mkenvimage -> and I do have env image [*] to be
flashed on the board.

However, I do wonder if for the default fw_setenv envs we could:

- modify fw_setenv to read (and store) [*] when no correct default env
  is available

I must check if fw_setenv handles images generated by mkenvimage (or
boot.src [1] generated by mkimage).


[1] - <u-boot repo>/board/samsung/common/bootscripts/autoboot.cmd

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 499 bytes --]

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Marek Vasut]

On 05/03/2018 08:15 PM, Lukasz Majewski wrote:
> Hi Marek, Stefano,
> 
>> On 05/03/2018 06:50 PM, Stefano Babic wrote:
>>> On 03/05/2018 18:36, Marek Vasut wrote:  
>>>> On 05/03/2018 06:28 PM, Stefano Babic wrote:  
>>>>> On 27/04/2018 17:07, Marek Vasut wrote:  
>>>>>> On 04/27/2018 04:51 PM, Lukasz Majewski wrote:  
>>>>>>> This commit provides the ability to generate u-boot
>>>>>>> environment(s) as images, which afterwards can be used to
>>>>>>> produce image (with wic) for flashing (eMMC or SPI-NOR).
>>>>>>>
>>>>>>> This change removes the need to run "env default" during
>>>>>>> production phase, as proper environment (including redundant
>>>>>>> one) is already stored on persistent memory (the CRC is also
>>>>>>> correct).
>>>>>>>
>>>>>>> Signed-off-by: Lukasz Majewski <lukma@denx.de>  
>>>>>>
>>>>>> If your default env is correct, why do you need this ? I can see
>>>>>> some use with non-default env, but then that can be wrapped into
>>>>>> a separate recipe.
>>>>>>  
>>>>>
>>>>> A use case is when the environment must be changed from user
>>>>> space. fw_setenv will report the CRC error and it needs the
>>>>> default environment to add changes. The default environment is
>>>>> linked together to fw_setenv, but this prohibites to use
>>>>> fw_setenv for multiple boards and must be explicitely built for
>>>>> that machine and with the same sources as u-boot (at least, they
>>>>> must share the same CONFIG_EXTRA_ENV). If the default environment
>>>>> is extracted, we could have a general (distro ?) fw_setenv.  
>>>>
>>>> I think in that case, the real solution is to either build
>>>> fw_setenv per machine   
>>>
>>> This is how we try to do now, fw_setenv is built per machine but it
>>> is enough that u-boot-fw-utils is built in a different version as
>>> u-boot to get a mess.  
>>
>> Well yes, if you mix and match packages, it becomes a mess. Isn't that
>> to be expected ?
>>
>>>> OR fix fw_setenv to take env defaults from a file or somesuch ?  
>>>
>>> Right, I interprete this patch as a step in this direction. This
>>> patch generates a default that can be used as input for fw_setenv.  
>>
>> It generates environment images which can be written -- on certain
>> specific setups -- into the flash. It doesn't generate any sort of
>> input for the fw_setenv to my knowledge ?
>>
> 
> I think that it would be great if:
> 
> 1. We would have this code as a separate recipe - as suggested by Marek
> and Stefano already. This recipe would end up as a package to be
> installed on the rootfs
> 
> 2. As input I would use default_envs.txt (or any other name) - either
> extracted from u-boot build or provided from external file
> 
> 3. For now I do use mkenvimage -> and I do have env image [*] to be
> flashed on the board.

Sounds about good. I think this approach fails with NAND, so be careful
there.

> However, I do wonder if for the default fw_setenv envs we could:
> 
> - modify fw_setenv to read (and store) [*] when no correct default env
>   is available
Or add some build-time option to build it with blank default env. Then
you can apply your file-based approach, the setenv would be
board-agnostic and it should do I guess ?

-- 
Best regards,
Marek Vasut

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 5080 bytes --]

Hi Marek, Stefano,

> On 05/03/2018 08:15 PM, Lukasz Majewski wrote:
> > Hi Marek, Stefano,
> >   
> >> On 05/03/2018 06:50 PM, Stefano Babic wrote:  
> >>> On 03/05/2018 18:36, Marek Vasut wrote:    
> >>>> On 05/03/2018 06:28 PM, Stefano Babic wrote:    
> >>>>> On 27/04/2018 17:07, Marek Vasut wrote:    
> >>>>>> On 04/27/2018 04:51 PM, Lukasz Majewski wrote:    
> >>>>>>> This commit provides the ability to generate u-boot
> >>>>>>> environment(s) as images, which afterwards can be used to
> >>>>>>> produce image (with wic) for flashing (eMMC or SPI-NOR).
> >>>>>>>
> >>>>>>> This change removes the need to run "env default" during
> >>>>>>> production phase, as proper environment (including redundant
> >>>>>>> one) is already stored on persistent memory (the CRC is also
> >>>>>>> correct).
> >>>>>>>
> >>>>>>> Signed-off-by: Lukasz Majewski <lukma@denx.de>    
> >>>>>>
> >>>>>> If your default env is correct, why do you need this ? I can
> >>>>>> see some use with non-default env, but then that can be
> >>>>>> wrapped into a separate recipe.
> >>>>>>    
> >>>>>
> >>>>> A use case is when the environment must be changed from user
> >>>>> space. fw_setenv will report the CRC error and it needs the
> >>>>> default environment to add changes. The default environment is
> >>>>> linked together to fw_setenv, but this prohibites to use
> >>>>> fw_setenv for multiple boards and must be explicitely built for
> >>>>> that machine and with the same sources as u-boot (at least, they
> >>>>> must share the same CONFIG_EXTRA_ENV). If the default
> >>>>> environment is extracted, we could have a general (distro ?)
> >>>>> fw_setenv.    
> >>>>
> >>>> I think in that case, the real solution is to either build
> >>>> fw_setenv per machine     
> >>>
> >>> This is how we try to do now, fw_setenv is built per machine but
> >>> it is enough that u-boot-fw-utils is built in a different version
> >>> as u-boot to get a mess.    
> >>
> >> Well yes, if you mix and match packages, it becomes a mess. Isn't
> >> that to be expected ?
> >>  
> >>>> OR fix fw_setenv to take env defaults from a file or
> >>>> somesuch ?    
> >>>
> >>> Right, I interprete this patch as a step in this direction. This
> >>> patch generates a default that can be used as input for
> >>> fw_setenv.    
> >>
> >> It generates environment images which can be written -- on certain
> >> specific setups -- into the flash. It doesn't generate any sort of
> >> input for the fw_setenv to my knowledge ?
> >>  
> > 
> > I think that it would be great if:
> > 
> > 1. We would have this code as a separate recipe - as suggested by
> > Marek and Stefano already. This recipe would end up as a package to
> > be installed on the rootfs

I've looked a bit more on the topic, and I do have some further
questions:

1. It seems like u-boot is compiled at least twice - once when
do_comiple() is invoked in u-boot.inc (here we support multiple
configs, and targets) and in u-boot-fw-utils (also u-boot-mkimage
compiles u-boot for sandbox_defconfig).

It seems like the synchronization of the built package (and envs) is
ensured with u-boot-common_2018.03.inc package. 

2. It seems like it would be feasible to have a separate recipe -
u-boot-envs_2018.03.bb [*] which would:

	- Use u-boot-env.txt provided as an external file (from e.g.
	  SRC += "./file/u-boot-env.txt"). This is the approach similar
	  to
	  https://github.com/webosose/meta-webosose/blob/master/meta-webos-raspberrypi/recipes-bsp/u-boot/u-boot-env.bb

	- If above input *.txt file is not present then get the envs
	  extracted from built u-boot. 

The main question for above item - how and where to extract this file?

	- The most feasible would be to extend do_compile of u-boot.inc
	  to run get_default_envs.sh script there (as it builds all
	  the kind of u-boot variants/binaries).

	- Then "export" those generated *.txt files to [*] recipe.
	  What would be the best way?

	  Via ${DEPLOYDIR} ? Install it on /boot directory (exported by
	  u-boot) ? 

> > 
> > 2. As input I would use default_envs.txt (or any other name) -
> > either extracted from u-boot build or provided from external file
> > 
> > 3. For now I do use mkenvimage -> and I do have env image [*] to be
> > flashed on the board.  
> 
> Sounds about good. I think this approach fails with NAND, so be
> careful there.
> 
> > However, I do wonder if for the default fw_setenv envs we could:
> > 
> > - modify fw_setenv to read (and store) [*] when no correct default
> > env is available  
> Or add some build-time option to build it with blank default env. Then
> you can apply your file-based approach, the setenv would be
> board-agnostic and it should do I guess ?
> 




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 499 bytes --]

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Marek Vasut]

On 05/09/2018 01:45 PM, Lukasz Majewski wrote:
> Hi Marek, Stefano,
> 
>> On 05/03/2018 08:15 PM, Lukasz Majewski wrote:
>>> Hi Marek, Stefano,
>>>   
>>>> On 05/03/2018 06:50 PM, Stefano Babic wrote:  
>>>>> On 03/05/2018 18:36, Marek Vasut wrote:    
>>>>>> On 05/03/2018 06:28 PM, Stefano Babic wrote:    
>>>>>>> On 27/04/2018 17:07, Marek Vasut wrote:    
>>>>>>>> On 04/27/2018 04:51 PM, Lukasz Majewski wrote:    
>>>>>>>>> This commit provides the ability to generate u-boot
>>>>>>>>> environment(s) as images, which afterwards can be used to
>>>>>>>>> produce image (with wic) for flashing (eMMC or SPI-NOR).
>>>>>>>>>
>>>>>>>>> This change removes the need to run "env default" during
>>>>>>>>> production phase, as proper environment (including redundant
>>>>>>>>> one) is already stored on persistent memory (the CRC is also
>>>>>>>>> correct).
>>>>>>>>>
>>>>>>>>> Signed-off-by: Lukasz Majewski <lukma@denx.de>    
>>>>>>>>
>>>>>>>> If your default env is correct, why do you need this ? I can
>>>>>>>> see some use with non-default env, but then that can be
>>>>>>>> wrapped into a separate recipe.
>>>>>>>>    
>>>>>>>
>>>>>>> A use case is when the environment must be changed from user
>>>>>>> space. fw_setenv will report the CRC error and it needs the
>>>>>>> default environment to add changes. The default environment is
>>>>>>> linked together to fw_setenv, but this prohibites to use
>>>>>>> fw_setenv for multiple boards and must be explicitely built for
>>>>>>> that machine and with the same sources as u-boot (at least, they
>>>>>>> must share the same CONFIG_EXTRA_ENV). If the default
>>>>>>> environment is extracted, we could have a general (distro ?)
>>>>>>> fw_setenv.    
>>>>>>
>>>>>> I think in that case, the real solution is to either build
>>>>>> fw_setenv per machine     
>>>>>
>>>>> This is how we try to do now, fw_setenv is built per machine but
>>>>> it is enough that u-boot-fw-utils is built in a different version
>>>>> as u-boot to get a mess.    
>>>>
>>>> Well yes, if you mix and match packages, it becomes a mess. Isn't
>>>> that to be expected ?
>>>>  
>>>>>> OR fix fw_setenv to take env defaults from a file or
>>>>>> somesuch ?    
>>>>>
>>>>> Right, I interprete this patch as a step in this direction. This
>>>>> patch generates a default that can be used as input for
>>>>> fw_setenv.    
>>>>
>>>> It generates environment images which can be written -- on certain
>>>> specific setups -- into the flash. It doesn't generate any sort of
>>>> input for the fw_setenv to my knowledge ?
>>>>  
>>>
>>> I think that it would be great if:
>>>
>>> 1. We would have this code as a separate recipe - as suggested by
>>> Marek and Stefano already. This recipe would end up as a package to
>>> be installed on the rootfs
> 
> I've looked a bit more on the topic, and I do have some further
> questions:
> 
> 1. It seems like u-boot is compiled at least twice - once when
> do_comiple() is invoked in u-boot.inc (here we support multiple
> configs, and targets) and in u-boot-fw-utils (also u-boot-mkimage
> compiles u-boot for sandbox_defconfig).
> 
> It seems like the synchronization of the built package (and envs) is
> ensured with u-boot-common_2018.03.inc package. 
> 
> 2. It seems like it would be feasible to have a separate recipe -
> u-boot-envs_2018.03.bb [*] which would:
> 
> 	- Use u-boot-env.txt provided as an external file (from e.g.
> 	  SRC += "./file/u-boot-env.txt"). This is the approach similar
> 	  to
> 	  https://github.com/webosose/meta-webosose/blob/master/meta-webos-raspberrypi/recipes-bsp/u-boot/u-boot-env.bb
> 
> 	- If above input *.txt file is not present then get the envs
> 	  extracted from built u-boot. 
> 
> The main question for above item - how and where to extract this file?
> 
> 	- The most feasible would be to extend do_compile of u-boot.inc
> 	  to run get_default_envs.sh script there (as it builds all
> 	  the kind of u-boot variants/binaries).
> 
> 	- Then "export" those generated *.txt files to [*] recipe.
> 	  What would be the best way?
> 
> 	  Via ${DEPLOYDIR} ? Install it on /boot directory (exported by
> 	  u-boot) ? 

Compile the fw env utils per-machine for now and in parallel add option
to compile it with blank env and use env from file into upstream u-boot
fwutils ?

-- 
Best regards,
Marek Vasut

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Stefano Babic]

Hi Marek,

On 03/05/2018 18:59, Marek Vasut wrote:
> On 05/03/2018 06:50 PM, Stefano Babic wrote:
>> On 03/05/2018 18:36, Marek Vasut wrote:
>>> On 05/03/2018 06:28 PM, Stefano Babic wrote:
>>>> On 27/04/2018 17:07, Marek Vasut wrote:
>>>>> On 04/27/2018 04:51 PM, Lukasz Majewski wrote:
>>>>>> This commit provides the ability to generate u-boot environment(s) as
>>>>>> images, which afterwards can be used to produce image (with wic) for
>>>>>> flashing (eMMC or SPI-NOR).
>>>>>>
>>>>>> This change removes the need to run "env default" during production phase,
>>>>>> as proper environment (including redundant one) is already stored on
>>>>>> persistent memory (the CRC is also correct).
>>>>>>
>>>>>> Signed-off-by: Lukasz Majewski <lukma@denx.de>
>>>>>
>>>>> If your default env is correct, why do you need this ? I can see some
>>>>> use with non-default env, but then that can be wrapped into a separate
>>>>> recipe.
>>>>>
>>>>
>>>> A use case is when the environment must be changed from user space.
>>>> fw_setenv will report the CRC error and it needs the default environment
>>>> to add changes. The default environment is linked together to fw_setenv,
>>>> but this prohibites to use fw_setenv for multiple boards and must be
>>>> explicitely built for that machine and with the same sources as u-boot
>>>> (at least, they must share the same CONFIG_EXTRA_ENV). If the default
>>>> environment is extracted, we could have a general (distro ?) fw_setenv.
>>>
>>> I think in that case, the real solution is to either build fw_setenv per
>>> machine 
>>
>> This is how we try to do now, fw_setenv is built per machine but it is
>> enough that u-boot-fw-utils is built in a different version as u-boot to
>> get a mess.
> 
> Well yes, if you mix and match packages, it becomes a mess. Isn't that
> to be expected ?
> 

Well, quite. But in the specific case, it is weird that the environment
tools are built by a separate recipe. u-boot-fw-utils generates binaries
from the same sources as u-boot (or it should), and building the tool
per machine means for me to have a single recipe for it, that generates
as additional package the env tools. This guarantees that they are
always in sync.

>>> OR fix fw_setenv to take env defaults from a file or somesuch ?
>>

Having a separate recipe as now means for me to try to have the tools
not per machine, and then makes sense to pass as input the default
environment to the tools.

>> Right, I interprete this patch as a step in this direction. This patch
>> generates a default that can be used as input for fw_setenv.
> 
> It generates environment images which can be written -- on certain
> specific setups -- into the flash. It doesn't generate any sort of input
> for the fw_setenv to my knowledge ?

Not the current patch - we are discussing how it could evolve ;-)

Regards,
Stefano

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
=====================================================================

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Marek Vasut]

On 05/03/2018 10:58 PM, Stefano Babic wrote:
> Hi Marek,
> 
> On 03/05/2018 18:59, Marek Vasut wrote:
>> On 05/03/2018 06:50 PM, Stefano Babic wrote:
>>> On 03/05/2018 18:36, Marek Vasut wrote:
>>>> On 05/03/2018 06:28 PM, Stefano Babic wrote:
>>>>> On 27/04/2018 17:07, Marek Vasut wrote:
>>>>>> On 04/27/2018 04:51 PM, Lukasz Majewski wrote:
>>>>>>> This commit provides the ability to generate u-boot environment(s) as
>>>>>>> images, which afterwards can be used to produce image (with wic) for
>>>>>>> flashing (eMMC or SPI-NOR).
>>>>>>>
>>>>>>> This change removes the need to run "env default" during production phase,
>>>>>>> as proper environment (including redundant one) is already stored on
>>>>>>> persistent memory (the CRC is also correct).
>>>>>>>
>>>>>>> Signed-off-by: Lukasz Majewski <lukma@denx.de>
>>>>>>
>>>>>> If your default env is correct, why do you need this ? I can see some
>>>>>> use with non-default env, but then that can be wrapped into a separate
>>>>>> recipe.
>>>>>>
>>>>>
>>>>> A use case is when the environment must be changed from user space.
>>>>> fw_setenv will report the CRC error and it needs the default environment
>>>>> to add changes. The default environment is linked together to fw_setenv,
>>>>> but this prohibites to use fw_setenv for multiple boards and must be
>>>>> explicitely built for that machine and with the same sources as u-boot
>>>>> (at least, they must share the same CONFIG_EXTRA_ENV). If the default
>>>>> environment is extracted, we could have a general (distro ?) fw_setenv.
>>>>
>>>> I think in that case, the real solution is to either build fw_setenv per
>>>> machine 
>>>
>>> This is how we try to do now, fw_setenv is built per machine but it is
>>> enough that u-boot-fw-utils is built in a different version as u-boot to
>>> get a mess.
>>
>> Well yes, if you mix and match packages, it becomes a mess. Isn't that
>> to be expected ?
>>
> 
> Well, quite. But in the specific case, it is weird that the environment
> tools are built by a separate recipe. u-boot-fw-utils generates binaries
> from the same sources as u-boot (or it should), and building the tool
> per machine means for me to have a single recipe for it, that generates
> as additional package the env tools. This guarantees that they are
> always in sync.

Patches welcome ? :)

>>>> OR fix fw_setenv to take env defaults from a file or somesuch ?
>>>
> 
> Having a separate recipe as now means for me to try to have the tools
> not per machine, and then makes sense to pass as input the default
> environment to the tools.

Cfr the conclusion (?) we reached with Lukasz I think ?

>>> Right, I interprete this patch as a step in this direction. This patch
>>> generates a default that can be used as input for fw_setenv.
>>
>> It generates environment images which can be written -- on certain
>> specific setups -- into the flash. It doesn't generate any sort of input
>> for the fw_setenv to my knowledge ?
> 
> Not the current patch - we are discussing how it could evolve ;-)

Ah, I wonder if we shouldn't start a new thread for this and crosspost
it to the U-Boot ML too then ?

-- 
Best regards,
Marek Vasut

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Martin Hundebøll]

Hi Lukasz,

On 2018-04-27 16:51, Lukasz Majewski wrote:
> This commit provides the ability to generate u-boot environment(s) as
> images, which afterwards can be used to produce image (with wic) for
> flashing (eMMC or SPI-NOR).
> 
> This change removes the need to run "env default" during production phase,
> as proper environment (including redundant one) is already stored on
> persistent memory (the CRC is also correct).

I think we should create a separate recipe to install the native 
mkenvimage binary (e.g. u-boot-mkenvimage_%.bb) or update 
u-boot-mkimage_%.bb install it.

Then a new recipe to create the environment images can depend on 
u-boot-mkenvimage-native.

Also note the recently added upstream support for external environment 
definitions:
http://git.denx.de/?p=u-boot.git;a=commit;h=f3d8f7dd73ac5dde258eb786d4a01869395b56d7

For our usecase we need the ability to generate environment images in 
yocto from such external definitions.

// Martin

> 
> Signed-off-by: Lukasz Majewski <lukma@denx.de>
> 
> ---
> This patch depends on "u-boot: Upgrade to 2018.03 release"
> https://patchwork.openembedded.org/patch/149998/
> ---
>   meta/recipes-bsp/u-boot/u-boot.inc | 35 +++++++++++++++++++++++++++++++++++
>   1 file changed, 35 insertions(+)
> 
> diff --git a/meta/recipes-bsp/u-boot/u-boot.inc b/meta/recipes-bsp/u-boot/u-boot.inc
> index c2bcf99840..2796e503cf 100644
> --- a/meta/recipes-bsp/u-boot/u-boot.inc
> +++ b/meta/recipes-bsp/u-boot/u-boot.inc
> @@ -305,3 +305,38 @@ do_deploy () {
>   }
>   
>   addtask deploy before do_build after do_compile
> +
> +# Create new rules to extract default envs
> +UBOOT_ENVS_DEFAULT ?= "uboot-envs-default"
> +DEFAULT_ENVS ?= "u-boot-env-default.txt"
> +DEFAULT_ENVS_SIZE ?= "65536"
> +
> +# Generate default environment
> +do_gen_default_envs[doc] = "Generate image with default U-Boot environment(s)"
> +do_gen_default_envs () {
> +	${B}/source/scripts/get_default_envs.sh ${B} > ${B}/${DEFAULT_ENVS}
> +
> +	# Generate env image
> +	${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}
> +
> +	# Generate redundant env image
> +	${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}
> +
> +	rm ${B}/${DEFAULT_ENVS}
> +}
> +
> +addtask gen_default_envs before do_deploy_default_envs after do_compile
> +
> +# Deploy default environment
> +do_deploy_default_envs[doc] = "Copy images with default U-Boot environment to deployment directory"
> +do_deploy_default_envs () {
> +	install -d ${DEPLOYDIR}
> +
> +	install ${B}/${UBOOT_ENVS_DEFAULT} ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}
> +	install ${B}/${UBOOT_ENVS_DEFAULT}_r ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r
> +
> +	rm ${B}/${UBOOT_ENVS_DEFAULT}
> +	rm ${B}/${UBOOT_ENVS_DEFAULT}_r
> +}
> +
> +addtask deploy_default_envs before do_deploy after do_gen_default_envs
> 

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 3897 bytes --]

Hi Martin,

> Hi Lukasz,
> 
> On 2018-04-27 16:51, Lukasz Majewski wrote:
> > This commit provides the ability to generate u-boot environment(s)
> > as images, which afterwards can be used to produce image (with wic)
> > for flashing (eMMC or SPI-NOR).
> > 
> > This change removes the need to run "env default" during production
> > phase, as proper environment (including redundant one) is already
> > stored on persistent memory (the CRC is also correct).  
> 
> I think we should create a separate recipe to install the native 
> mkenvimage binary (e.g. u-boot-mkenvimage_%.bb) or update 
> u-boot-mkimage_%.bb install it.

I see your point.
I suppose that the above idea is more OE style than:

${B}/tools/mkenvimage 
and DEPENDS = "u-boot:compile"

> 
> Then a new recipe to create the environment images can depend on 
> u-boot-mkenvimage-native.

Ok.

> 
> Also note the recently added upstream support for external
> environment definitions:
> http://git.denx.de/?p=u-boot.git;a=commit;h=f3d8f7dd73ac5dde258eb786d4a01869395b56d7

Those features are complementary.

1. Envs definition from CONFIG_EXTRA_ENV_SETTINGS are provided
externally,

2. The one generated by source/scripts/get_default_envs.sh are simply
the ones from u-boot's "default environment" (extracted from the
binary).

They both produce *.txt file which is the input for mkenvimage.

> 
> For our usecase we need the ability to generate environment images in 
> yocto from such external definitions.

Ok. I see.

> 
> // Martin
> 
> > 
> > Signed-off-by: Lukasz Majewski <lukma@denx.de>
> > 
> > ---
> > This patch depends on "u-boot: Upgrade to 2018.03 release"
> > https://patchwork.openembedded.org/patch/149998/
> > ---
> >   meta/recipes-bsp/u-boot/u-boot.inc | 35
> > +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
> > 
> > diff --git a/meta/recipes-bsp/u-boot/u-boot.inc
> > b/meta/recipes-bsp/u-boot/u-boot.inc index c2bcf99840..2796e503cf
> > 100644 --- a/meta/recipes-bsp/u-boot/u-boot.inc
> > +++ b/meta/recipes-bsp/u-boot/u-boot.inc
> > @@ -305,3 +305,38 @@ do_deploy () {
> >   }
> >   
> >   addtask deploy before do_build after do_compile
> > +
> > +# Create new rules to extract default envs
> > +UBOOT_ENVS_DEFAULT ?= "uboot-envs-default"
> > +DEFAULT_ENVS ?= "u-boot-env-default.txt"
> > +DEFAULT_ENVS_SIZE ?= "65536"
> > +
> > +# Generate default environment
> > +do_gen_default_envs[doc] = "Generate image with default U-Boot
> > environment(s)" +do_gen_default_envs () {
> > +	${B}/source/scripts/get_default_envs.sh ${B} >
> > ${B}/${DEFAULT_ENVS} +
> > +	# Generate env image
> > +	${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o
> > ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS} +
> > +	# Generate redundant env image
> > +	${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o
> > ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS} +
> > +	rm ${B}/${DEFAULT_ENVS}
> > +}
> > +
> > +addtask gen_default_envs before do_deploy_default_envs after
> > do_compile +
> > +# Deploy default environment
> > +do_deploy_default_envs[doc] = "Copy images with default U-Boot
> > environment to deployment directory" +do_deploy_default_envs () {
> > +	install -d ${DEPLOYDIR}
> > +
> > +	install ${B}/${UBOOT_ENVS_DEFAULT}
> > ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}
> > +	install ${B}/${UBOOT_ENVS_DEFAULT}_r
> > ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r +
> > +	rm ${B}/${UBOOT_ENVS_DEFAULT}
> > +	rm ${B}/${UBOOT_ENVS_DEFAULT}_r
> > +}
> > +
> > +addtask deploy_default_envs before do_deploy after
> > do_gen_default_envs 




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 499 bytes --]

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3744 bytes --]

u-boot-mkimage already builds mkenvimage, you just need to install it in
do_install:

    install -m 0755 tools/mkenvimage ${D}${bindir}/uboot-mkenvimage
    ln -sf uboot-mkenvimage ${D}${bindir}/mkenvimage

This is what we have in recipes-bsp/u-boot/u-boot-mkimage_%.bbappend and it
works fine.

On Mon, Apr 30, 2018 at 8:50 AM, Martin Hundebøll <mnhu@prevas.dk> wrote:

> Hi Lukasz,
>
> On 2018-04-27 16:51, Lukasz Majewski wrote:
>
>> This commit provides the ability to generate u-boot environment(s) as
>> images, which afterwards can be used to produce image (with wic) for
>> flashing (eMMC or SPI-NOR).
>>
>> This change removes the need to run "env default" during production phase,
>> as proper environment (including redundant one) is already stored on
>> persistent memory (the CRC is also correct).
>>
>
> I think we should create a separate recipe to install the native
> mkenvimage binary (e.g. u-boot-mkenvimage_%.bb) or update
> u-boot-mkimage_%.bb install it.
>
> Then a new recipe to create the environment images can depend on
> u-boot-mkenvimage-native.
>
> Also note the recently added upstream support for external environment
> definitions:
> http://git.denx.de/?p=u-boot.git;a=commit;h=f3d8f7dd73ac5dde
> 258eb786d4a01869395b56d7
>
> For our usecase we need the ability to generate environment images in
> yocto from such external definitions.
>
> // Martin
>
>
>> Signed-off-by: Lukasz Majewski <lukma@denx.de>
>>
>> ---
>> This patch depends on "u-boot: Upgrade to 2018.03 release"
>> https://patchwork.openembedded.org/patch/149998/
>> ---
>>   meta/recipes-bsp/u-boot/u-boot.inc | 35 ++++++++++++++++++++++++++++++
>> +++++
>>   1 file changed, 35 insertions(+)
>>
>> diff --git a/meta/recipes-bsp/u-boot/u-boot.inc
>> b/meta/recipes-bsp/u-boot/u-boot.inc
>> index c2bcf99840..2796e503cf 100644
>> --- a/meta/recipes-bsp/u-boot/u-boot.inc
>> +++ b/meta/recipes-bsp/u-boot/u-boot.inc
>> @@ -305,3 +305,38 @@ do_deploy () {
>>   }
>>     addtask deploy before do_build after do_compile
>> +
>> +# Create new rules to extract default envs
>> +UBOOT_ENVS_DEFAULT ?= "uboot-envs-default"
>> +DEFAULT_ENVS ?= "u-boot-env-default.txt"
>> +DEFAULT_ENVS_SIZE ?= "65536"
>> +
>> +# Generate default environment
>> +do_gen_default_envs[doc] = "Generate image with default U-Boot
>> environment(s)"
>> +do_gen_default_envs () {
>> +       ${B}/source/scripts/get_default_envs.sh ${B} >
>> ${B}/${DEFAULT_ENVS}
>> +
>> +       # Generate env image
>> +       ${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o
>> ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}
>> +
>> +       # Generate redundant env image
>> +       ${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o
>> ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}
>> +
>> +       rm ${B}/${DEFAULT_ENVS}
>> +}
>> +
>> +addtask gen_default_envs before do_deploy_default_envs after do_compile
>> +
>> +# Deploy default environment
>> +do_deploy_default_envs[doc] = "Copy images with default U-Boot
>> environment to deployment directory"
>> +do_deploy_default_envs () {
>> +       install -d ${DEPLOYDIR}
>> +
>> +       install ${B}/${UBOOT_ENVS_DEFAULT} ${DEPLOYDIR}/${UBOOT_ENVS_DEFA
>> ULT}
>> +       install ${B}/${UBOOT_ENVS_DEFAULT}_r
>> ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r
>> +
>> +       rm ${B}/${UBOOT_ENVS_DEFAULT}
>> +       rm ${B}/${UBOOT_ENVS_DEFAULT}_r
>> +}
>> +
>> +addtask deploy_default_envs before do_deploy after do_gen_default_envs
>>
>> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>

[-- Attachment #2: Type: text/html, Size: 5238 bytes --]

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 4538 bytes --]

And here is example how we're using it:

https://github.com/webosose/meta-webosose/blob/master/meta-webos-raspberrypi/recipes-bsp/u-boot/u-boot-env.bb

To make it compatible with raspberrypi3 and raspberrypi3-64, I've updated
it a bit to select correct IMAGETYPE and BOOTCMD based on the environment:

do_compile() {
    sed -e 's/@@KERNEL_IMAGETYPE@@/${KERNEL_IMAGETYPE}/' \
        -e 's/@@KERNEL_BOOTCMD@@/${KERNEL_BOOTCMD}/' \
        "${WORKDIR}/uboot-env.txt.in" > "${WORKDIR}/uboot-env.txt"
    uboot-mkenvimage -s 16384 -o uboot.env ${WORKDIR}/uboot-env.txt
}


On Mon, Apr 30, 2018 at 3:23 PM, Martin Jansa <martin.jansa@gmail.com>
wrote:

> u-boot-mkimage already builds mkenvimage, you just need to install it in
> do_install:
>
>     install -m 0755 tools/mkenvimage ${D}${bindir}/uboot-mkenvimage
>     ln -sf uboot-mkenvimage ${D}${bindir}/mkenvimage
>
> This is what we have in recipes-bsp/u-boot/u-boot-mkimage_%.bbappend and
> it works fine.
>
> On Mon, Apr 30, 2018 at 8:50 AM, Martin Hundebøll <mnhu@prevas.dk> wrote:
>
>> Hi Lukasz,
>>
>> On 2018-04-27 16:51, Lukasz Majewski wrote:
>>
>>> This commit provides the ability to generate u-boot environment(s) as
>>> images, which afterwards can be used to produce image (with wic) for
>>> flashing (eMMC or SPI-NOR).
>>>
>>> This change removes the need to run "env default" during production
>>> phase,
>>> as proper environment (including redundant one) is already stored on
>>> persistent memory (the CRC is also correct).
>>>
>>
>> I think we should create a separate recipe to install the native
>> mkenvimage binary (e.g. u-boot-mkenvimage_%.bb) or update
>> u-boot-mkimage_%.bb install it.
>>
>> Then a new recipe to create the environment images can depend on
>> u-boot-mkenvimage-native.
>>
>> Also note the recently added upstream support for external environment
>> definitions:
>> http://git.denx.de/?p=u-boot.git;a=commit;h=f3d8f7dd73ac5dde
>> 258eb786d4a01869395b56d7
>>
>> For our usecase we need the ability to generate environment images in
>> yocto from such external definitions.
>>
>> // Martin
>>
>>
>>> Signed-off-by: Lukasz Majewski <lukma@denx.de>
>>>
>>> ---
>>> This patch depends on "u-boot: Upgrade to 2018.03 release"
>>> https://patchwork.openembedded.org/patch/149998/
>>> ---
>>>   meta/recipes-bsp/u-boot/u-boot.inc | 35 ++++++++++++++++++++++++++++++
>>> +++++
>>>   1 file changed, 35 insertions(+)
>>>
>>> diff --git a/meta/recipes-bsp/u-boot/u-boot.inc
>>> b/meta/recipes-bsp/u-boot/u-boot.inc
>>> index c2bcf99840..2796e503cf 100644
>>> --- a/meta/recipes-bsp/u-boot/u-boot.inc
>>> +++ b/meta/recipes-bsp/u-boot/u-boot.inc
>>> @@ -305,3 +305,38 @@ do_deploy () {
>>>   }
>>>     addtask deploy before do_build after do_compile
>>> +
>>> +# Create new rules to extract default envs
>>> +UBOOT_ENVS_DEFAULT ?= "uboot-envs-default"
>>> +DEFAULT_ENVS ?= "u-boot-env-default.txt"
>>> +DEFAULT_ENVS_SIZE ?= "65536"
>>> +
>>> +# Generate default environment
>>> +do_gen_default_envs[doc] = "Generate image with default U-Boot
>>> environment(s)"
>>> +do_gen_default_envs () {
>>> +       ${B}/source/scripts/get_default_envs.sh ${B} >
>>> ${B}/${DEFAULT_ENVS}
>>> +
>>> +       # Generate env image
>>> +       ${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o
>>> ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}
>>> +
>>> +       # Generate redundant env image
>>> +       ${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o
>>> ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}
>>> +
>>> +       rm ${B}/${DEFAULT_ENVS}
>>> +}
>>> +
>>> +addtask gen_default_envs before do_deploy_default_envs after do_compile
>>> +
>>> +# Deploy default environment
>>> +do_deploy_default_envs[doc] = "Copy images with default U-Boot
>>> environment to deployment directory"
>>> +do_deploy_default_envs () {
>>> +       install -d ${DEPLOYDIR}
>>> +
>>> +       install ${B}/${UBOOT_ENVS_DEFAULT} ${DEPLOYDIR}/${UBOOT_ENVS_DEFA
>>> ULT}
>>> +       install ${B}/${UBOOT_ENVS_DEFAULT}_r
>>> ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r
>>> +
>>> +       rm ${B}/${UBOOT_ENVS_DEFAULT}
>>> +       rm ${B}/${UBOOT_ENVS_DEFAULT}_r
>>> +}
>>> +
>>> +addtask deploy_default_envs before do_deploy after do_gen_default_envs
>>>
>>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>
>
>

[-- Attachment #2: Type: text/html, Size: 6599 bytes --]

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 5159 bytes --]

Hi Martin,

> And here is example how we're using it:
> 
> https://github.com/webosose/meta-webosose/blob/master/meta-webos-raspberrypi/recipes-bsp/u-boot/u-boot-env.bb
> 
> To make it compatible with raspberrypi3 and raspberrypi3-64, I've
> updated it a bit to select correct IMAGETYPE and BOOTCMD based on the
> environment:
> 
> do_compile() {
>     sed -e 's/@@KERNEL_IMAGETYPE@@/${KERNEL_IMAGETYPE}/' \
>         -e 's/@@KERNEL_BOOTCMD@@/${KERNEL_BOOTCMD}/' \
>         "${WORKDIR}/uboot-env.txt.in" > "${WORKDIR}/uboot-env.txt"
>     uboot-mkenvimage -s 16384 -o uboot.env ${WORKDIR}/uboot-env.txt
> }
> 

Thanks for sharing the code (and detailed explanation).

> 
> On Mon, Apr 30, 2018 at 3:23 PM, Martin Jansa <martin.jansa@gmail.com>
> wrote:
> 
> > u-boot-mkimage already builds mkenvimage, you just need to install
> > it in do_install:
> >
> >     install -m 0755 tools/mkenvimage ${D}${bindir}/uboot-mkenvimage
> >     ln -sf uboot-mkenvimage ${D}${bindir}/mkenvimage
> >
> > This is what we have in
> > recipes-bsp/u-boot/u-boot-mkimage_%.bbappend and it works fine.
> >
> > On Mon, Apr 30, 2018 at 8:50 AM, Martin Hundebøll <mnhu@prevas.dk>
> > wrote: 
> >> Hi Lukasz,
> >>
> >> On 2018-04-27 16:51, Lukasz Majewski wrote:
> >>  
> >>> This commit provides the ability to generate u-boot
> >>> environment(s) as images, which afterwards can be used to produce
> >>> image (with wic) for flashing (eMMC or SPI-NOR).
> >>>
> >>> This change removes the need to run "env default" during
> >>> production phase,
> >>> as proper environment (including redundant one) is already stored
> >>> on persistent memory (the CRC is also correct).
> >>>  
> >>
> >> I think we should create a separate recipe to install the native
> >> mkenvimage binary (e.g. u-boot-mkenvimage_%.bb) or update
> >> u-boot-mkimage_%.bb install it.
> >>
> >> Then a new recipe to create the environment images can depend on
> >> u-boot-mkenvimage-native.
> >>
> >> Also note the recently added upstream support for external
> >> environment definitions:
> >> http://git.denx.de/?p=u-boot.git;a=commit;h=f3d8f7dd73ac5dde
> >> 258eb786d4a01869395b56d7
> >>
> >> For our usecase we need the ability to generate environment images
> >> in yocto from such external definitions.
> >>
> >> // Martin
> >>
> >>  
> >>> Signed-off-by: Lukasz Majewski <lukma@denx.de>
> >>>
> >>> ---
> >>> This patch depends on "u-boot: Upgrade to 2018.03 release"
> >>> https://patchwork.openembedded.org/patch/149998/
> >>> ---
> >>>   meta/recipes-bsp/u-boot/u-boot.inc | 35
> >>> ++++++++++++++++++++++++++++++ +++++
> >>>   1 file changed, 35 insertions(+)
> >>>
> >>> diff --git a/meta/recipes-bsp/u-boot/u-boot.inc
> >>> b/meta/recipes-bsp/u-boot/u-boot.inc
> >>> index c2bcf99840..2796e503cf 100644
> >>> --- a/meta/recipes-bsp/u-boot/u-boot.inc
> >>> +++ b/meta/recipes-bsp/u-boot/u-boot.inc
> >>> @@ -305,3 +305,38 @@ do_deploy () {
> >>>   }
> >>>     addtask deploy before do_build after do_compile
> >>> +
> >>> +# Create new rules to extract default envs
> >>> +UBOOT_ENVS_DEFAULT ?= "uboot-envs-default"
> >>> +DEFAULT_ENVS ?= "u-boot-env-default.txt"
> >>> +DEFAULT_ENVS_SIZE ?= "65536"
> >>> +
> >>> +# Generate default environment
> >>> +do_gen_default_envs[doc] = "Generate image with default U-Boot
> >>> environment(s)"
> >>> +do_gen_default_envs () {
> >>> +       ${B}/source/scripts/get_default_envs.sh ${B} >
> >>> ${B}/${DEFAULT_ENVS}
> >>> +
> >>> +       # Generate env image
> >>> +       ${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o
> >>> ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}
> >>> +
> >>> +       # Generate redundant env image
> >>> +       ${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o
> >>> ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}
> >>> +
> >>> +       rm ${B}/${DEFAULT_ENVS}
> >>> +}
> >>> +
> >>> +addtask gen_default_envs before do_deploy_default_envs after
> >>> do_compile +
> >>> +# Deploy default environment
> >>> +do_deploy_default_envs[doc] = "Copy images with default U-Boot
> >>> environment to deployment directory"
> >>> +do_deploy_default_envs () {
> >>> +       install -d ${DEPLOYDIR}
> >>> +
> >>> +       install ${B}/${UBOOT_ENVS_DEFAULT}
> >>> ${DEPLOYDIR}/${UBOOT_ENVS_DEFA ULT}
> >>> +       install ${B}/${UBOOT_ENVS_DEFAULT}_r
> >>> ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r
> >>> +
> >>> +       rm ${B}/${UBOOT_ENVS_DEFAULT}
> >>> +       rm ${B}/${UBOOT_ENVS_DEFAULT}_r
> >>> +}
> >>> +
> >>> +addtask deploy_default_envs before do_deploy after
> >>> do_gen_default_envs
> >>>
> >>> --  
> >> _______________________________________________
> >> Openembedded-core mailing list
> >> Openembedded-core@lists.openembedded.org
> >> http://lists.openembedded.org/mailman/listinfo/openembedded-core
> >>  
> >
> >  




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 499 bytes --]

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Stefano Babic]

Hi Lukasz,

On 27/04/2018 16:51, Lukasz Majewski wrote:
> This commit provides the ability to generate u-boot environment(s) as
> images, which afterwards can be used to produce image (with wic) for
> flashing (eMMC or SPI-NOR).
> 
> This change removes the need to run "env default" during production phase,
> as proper environment (including redundant one) is already stored on
> persistent memory (the CRC is also correct).
> 
> Signed-off-by: Lukasz Majewski <lukma@denx.de>
> 
> ---
> This patch depends on "u-boot: Upgrade to 2018.03 release"
> https://patchwork.openembedded.org/patch/149998/
> ---
>  meta/recipes-bsp/u-boot/u-boot.inc | 35 +++++++++++++++++++++++++++++++++++
>  1 file changed, 35 insertions(+)
> 
> diff --git a/meta/recipes-bsp/u-boot/u-boot.inc b/meta/recipes-bsp/u-boot/u-boot.inc
> index c2bcf99840..2796e503cf 100644
> --- a/meta/recipes-bsp/u-boot/u-boot.inc
> +++ b/meta/recipes-bsp/u-boot/u-boot.inc
> @@ -305,3 +305,38 @@ do_deploy () {
>  }
>  
>  addtask deploy before do_build after do_compile
> +
> +# Create new rules to extract default envs
> +UBOOT_ENVS_DEFAULT ?= "uboot-envs-default"
> +DEFAULT_ENVS ?= "u-boot-env-default.txt"
> +DEFAULT_ENVS_SIZE ?= "65536"
> +
> +# Generate default environment
> +do_gen_default_envs[doc] = "Generate image with default U-Boot environment(s)"
> +do_gen_default_envs () {
> +	${B}/source/scripts/get_default_envs.sh ${B} > ${B}/${DEFAULT_ENVS}
> +
> +	# Generate env image
> +	${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}
> +
> +	# Generate redundant env image
> +	${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}
> +
> +	rm ${B}/${DEFAULT_ENVS}
> +}
> +

Why do we need a separate task ? Is it not part of do_compile ?

> +addtask gen_default_envs before do_deploy_default_envs after do_compile
> +
> +# Deploy default environment
> +do_deploy_default_envs[doc] = "Copy images with default U-Boot environment to deployment directory"
> +do_deploy_default_envs () {
> +	install -d ${DEPLOYDIR}
> +
> +	install ${B}/${UBOOT_ENVS_DEFAULT} ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}
> +	install ${B}/${UBOOT_ENVS_DEFAULT}_r ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r
> +
> +	rm ${B}/${UBOOT_ENVS_DEFAULT}
> +	rm ${B}/${UBOOT_ENVS_DEFAULT}_r
> +}
> +
> +addtask deploy_default_envs before do_deploy after do_gen_default_envs
> 

I would like to see that the default environment is installed and
delivered as (additional) package. I could then add it to my rootfs and
use it to set up the environment in user space in case a CRC error is
reported by the env tools.

Regards,
Stefano

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
=====================================================================

Re: [PATCH] u-boot: Add {gen|deploy}_default_envs tasks to generate environment images

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 3400 bytes --]

On Thu, 3 May 2018 18:24:32 +0200
Stefano Babic <sbabic@denx.de> wrote:

> Hi Lukasz,
> 
> On 27/04/2018 16:51, Lukasz Majewski wrote:
> > This commit provides the ability to generate u-boot environment(s)
> > as images, which afterwards can be used to produce image (with wic)
> > for flashing (eMMC or SPI-NOR).
> > 
> > This change removes the need to run "env default" during production
> > phase, as proper environment (including redundant one) is already
> > stored on persistent memory (the CRC is also correct).
> > 
> > Signed-off-by: Lukasz Majewski <lukma@denx.de>
> > 
> > ---
> > This patch depends on "u-boot: Upgrade to 2018.03 release"
> > https://patchwork.openembedded.org/patch/149998/
> > ---
> >  meta/recipes-bsp/u-boot/u-boot.inc | 35
> > +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
> > 
> > diff --git a/meta/recipes-bsp/u-boot/u-boot.inc
> > b/meta/recipes-bsp/u-boot/u-boot.inc index c2bcf99840..2796e503cf
> > 100644 --- a/meta/recipes-bsp/u-boot/u-boot.inc
> > +++ b/meta/recipes-bsp/u-boot/u-boot.inc
> > @@ -305,3 +305,38 @@ do_deploy () {
> >  }
> >  
> >  addtask deploy before do_build after do_compile
> > +
> > +# Create new rules to extract default envs
> > +UBOOT_ENVS_DEFAULT ?= "uboot-envs-default"
> > +DEFAULT_ENVS ?= "u-boot-env-default.txt"
> > +DEFAULT_ENVS_SIZE ?= "65536"
> > +
> > +# Generate default environment
> > +do_gen_default_envs[doc] = "Generate image with default U-Boot
> > environment(s)" +do_gen_default_envs () {
> > +	${B}/source/scripts/get_default_envs.sh ${B} >
> > ${B}/${DEFAULT_ENVS} +
> > +	# Generate env image
> > +	${B}/tools/mkenvimage -s ${DEFAULT_ENVS_SIZE} -o
> > ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS} +
> > +	# Generate redundant env image
> > +	${B}/tools/mkenvimage -r -s ${DEFAULT_ENVS_SIZE} -o
> > ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS} +
> > +	rm ${B}/${DEFAULT_ENVS}
> > +}
> > +  
> 
> Why do we need a separate task ? Is it not part of do_compile ?

The get_default_envs.sh script requires the build-in.o to be already
build. For that reason I've added a separate task, which goes just after
the do_compile for u-boot.

> 
> > +addtask gen_default_envs before do_deploy_default_envs after
> > do_compile +
> > +# Deploy default environment
> > +do_deploy_default_envs[doc] = "Copy images with default U-Boot
> > environment to deployment directory" +do_deploy_default_envs () {
> > +	install -d ${DEPLOYDIR}
> > +
> > +	install ${B}/${UBOOT_ENVS_DEFAULT}
> > ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}
> > +	install ${B}/${UBOOT_ENVS_DEFAULT}_r
> > ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r +
> > +	rm ${B}/${UBOOT_ENVS_DEFAULT}
> > +	rm ${B}/${UBOOT_ENVS_DEFAULT}_r
> > +}
> > +
> > +addtask deploy_default_envs before do_deploy after
> > do_gen_default_envs 
> 
> I would like to see that the default environment is installed and
> delivered as (additional) package. I could then add it to my rootfs
> and use it to set up the environment in user space in case a CRC
> error is reported by the env tools.
> 
> Regards,
> Stefano
> 




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 499 bytes --]