imx-boot do_compile failing with custom distro #yocto

imx-boot do_compile failing with custom distro #yocto

[stefan.wenninger]

[-- Attachment #1: Type: text/plain, Size: 3583 bytes --]

Hi,
I have encountered a curious problem when trying to create my own poky-based distribution.
My distribution poky-mydist.conf in conf/distro/ of my own layer:

> 
> require conf/distro/poky.conf
> DISTRO = "poky-mydist"
> DISTRO_NAME = "Poky my dist (Custom Project Distro)"
> DISTROOVERRIDES = "poky:poky-mydist"
> 
> DISTRO_FEATURES += "ppp"
> 

This distro is basically identical to the base poky from poky.conf. I just added a feature to be able to verify that my distro is being used.

When I build core-image-base however, I get the following error:

> 
> ERROR: imx-boot-0.2-r0 do_compile: Function failed: do_compile (log file
> is located at
> /opt/yocto-dev/build_poky/tmp/work/imx8mq_var_dart-poky-linux/imx-boot/0.2-r0/temp/log.do_compile.14295)
> 
> ERROR: Logfile of failure stored in:
> /opt/yocto-dev/build_poky/tmp/work/imx8mq_var_dart-poky-linux/imx-boot/0.2-r0/temp/log.do_compile.14295
> 
> Log data follows:
> | DEBUG: Executing shell function do_compile
> | Copying DTB
> | 8MQ/8MM boot binary build
> | Copy ddr_firmware: lpddr4_pmu_train_1d_imem.bin from
> /opt/yocto-dev/build_poky/tmp/deploy/images/imx8mq-var-dart ->
> /opt/yocto-dev/build_poky/tmp/work/imx8mq_var_dart-poky-linux/imx-boot/0.2-r0/git/iMX8M
> 
> | Copy ddr_firmware: lpddr4_pmu_train_1d_dmem.bin from
> /opt/yocto-dev/build_poky/tmp/deploy/images/imx8mq-var-dart ->
> /opt/yocto-dev/build_poky/tmp/work/imx8mq_var_dart-poky-linux/imx-boot/0.2-r0/git/iMX8M
> 
> | Copy ddr_firmware: lpddr4_pmu_train_2d_imem.bin from
> /opt/yocto-dev/build_poky/tmp/deploy/images/imx8mq-var-dart ->
> /opt/yocto-dev/build_poky/tmp/work/imx8mq_var_dart-poky-linux/imx-boot/0.2-r0/git/iMX8M
> 
> | Copy ddr_firmware: lpddr4_pmu_train_2d_dmem.bin from
> /opt/yocto-dev/build_poky/tmp/deploy/images/imx8mq-var-dart ->
> /opt/yocto-dev/build_poky/tmp/work/imx8mq_var_dart-poky-linux/imx-boot/0.2-r0/git/iMX8M
> 
> | cp: cannot stat
> '/opt/yocto-dev/build_poky/tmp/deploy/images/imx8mq-var-dart/imx-boot-tools/bl31-imx8mq.bin':
> No such file or directory
> | WARNING: exit code 1 from a shell command.
> | ERROR: Function failed: do_compile (log file is located at
> /opt/yocto-dev/build_poky/tmp/work/imx8mq_var_dart-poky-linux/imx-boot/0.2-r0/temp/log.do_compile.14295)
> 
> ERROR: Task
> (/opt/yocto-dev/sources/meta-fsl-bsp-release/imx/meta-bsp/recipes-bsp/imx-mkimage/imx-boot_0.2.bb:do_compile)
> failed with exit code '1'
> 

By just switching my DISTRO to 'poky' in local.conf, it builds without error.

There are several things that surprise me about this error:
1. I find it quite strange that these ddr_firmwares are copied from the deploy directory to the working directory of imx-boot. I would expect that to be the other way around.
2. The file bl31-imx8mq.bin is not in the imx-boot-tools folder, even after a successful build with DISTRO='poky'. (Maybe it was there during build and was just removed afterwards?)

From my point of view it looks like the set of files that get put into imx-boot-tools/ is dependant on the DISTRO value? When using DISTRO='poky' the correct set is placed and copied. When using DISTRO='poky-mydist' the files that are placed into the folder are not the files that are supposed to be copied from there, hence the error.

Am I interpreting this situation correctly?
Is there an error in my poky-mydist distribution?
Is that something that needs to be fixed in the recipe for imx-boot? Or can I somehow configure poky-mydist.conf to be treated exactly like poky.conf ?

Thanks in advance
Stefan

[-- Attachment #2: Type: text/html, Size: 4133 bytes --]

Re: [yocto] imx-boot do_compile failing with custom distro #yocto

[Rudolf J Streif]

[-- Attachment #1.1.1: Type: text/plain, Size: 4850 bytes --]

Hi Stefan,

I agree that it is odd that a recipe copies a file, in this case the
bl31 bootloader image, from the deploy area to the recipe area. This
bootloader binary is created by the imx-atf recipe. imx-atf is listed as
a dependency for imx-boot which is the recipe that is failing in your
build. Therefore I would expect imx-atf to run before imx-boot. However,
a recipe's do_compile task does not depend on another recipe's do_deploy
task but on the other recipe's do_install task. What likely happened is
that imx-boot do_compile was started after imx-atf do_install completed
but before imx-atf do_deploy has completed. For imx-boot to work
correctly it should copy the files from imx-atf's staging area rather
than the deploy directory.

I suggest that you try to run imx-atf alone to see if it builds and if
the bl31 image is built and placed into the deploy directory. And then
run your build again.

:rjs

On 4/8/20 7:41 AM, stefan.wenninger@evopro-ag.de wrote:
> Hi,
> I have encountered a curious problem when trying to create my own
> poky-based distribution.
> My distribution /poky-mydist.conf/ in conf/distro/ of my own layer:
>
>     require conf/distro/poky.conf
>     DISTRO = "poky-mydist"
>     DISTRO_NAME = "Poky my dist (Custom Project Distro)"
>     DISTROOVERRIDES = "poky:poky-mydist"
>      
>     DISTRO_FEATURES += "ppp"
>
> This distro is basically identical to the base poky from /poky.conf/.
> I just added a feature to be able to verify that my distro is being used.
>
> When I build core-image-base however, I get the following error:
>
>     ERROR: imx-boot-0.2-r0 do_compile: Function failed: do_compile
>     (log file is located at
>     /opt/yocto-dev/build_poky/tmp/work/imx8mq_var_dart-poky-linux/imx-boot/0.2-r0/temp/log.do_compile.14295)
>     ERROR: Logfile of failure stored in:
>     /opt/yocto-dev/build_poky/tmp/work/imx8mq_var_dart-poky-linux/imx-boot/0.2-r0/temp/log.do_compile.14295
>     Log data follows:
>     | DEBUG: Executing shell function do_compile
>     | Copying DTB
>     | 8MQ/8MM boot binary build
>     | Copy ddr_firmware: lpddr4_pmu_train_1d_imem.bin from
>     /opt/yocto-dev/build_poky/tmp/deploy/images/imx8mq-var-dart ->
>     /opt/yocto-dev/build_poky/tmp/work/imx8mq_var_dart-poky-linux/imx-boot/0.2-r0/git/iMX8M
>     | Copy ddr_firmware: lpddr4_pmu_train_1d_dmem.bin from
>     /opt/yocto-dev/build_poky/tmp/deploy/images/imx8mq-var-dart ->
>     /opt/yocto-dev/build_poky/tmp/work/imx8mq_var_dart-poky-linux/imx-boot/0.2-r0/git/iMX8M
>     | Copy ddr_firmware: lpddr4_pmu_train_2d_imem.bin from
>     /opt/yocto-dev/build_poky/tmp/deploy/images/imx8mq-var-dart ->
>     /opt/yocto-dev/build_poky/tmp/work/imx8mq_var_dart-poky-linux/imx-boot/0.2-r0/git/iMX8M
>     | Copy ddr_firmware: lpddr4_pmu_train_2d_dmem.bin from
>     /opt/yocto-dev/build_poky/tmp/deploy/images/imx8mq-var-dart ->
>     /opt/yocto-dev/build_poky/tmp/work/imx8mq_var_dart-poky-linux/imx-boot/0.2-r0/git/iMX8M
>     | cp: cannot stat
>     '/opt/yocto-dev/build_poky/tmp/deploy/images/imx8mq-var-dart/imx-boot-tools/bl31-imx8mq.bin':
>     No such file or directory
>     | WARNING: exit code 1 from a shell command.
>     | ERROR: Function failed: do_compile (log file is located at
>     /opt/yocto-dev/build_poky/tmp/work/imx8mq_var_dart-poky-linux/imx-boot/0.2-r0/temp/log.do_compile.14295)
>     ERROR: Task
>     (/opt/yocto-dev/sources/meta-fsl-bsp-release/imx/meta-bsp/recipes-bsp/imx-mkimage/imx-boot_0.2.bb:do_compile)
>     failed with exit code '1'
>
> By just switching my DISTRO to 'poky' in local.conf, it builds without
> error.
>
> There are several things that surprise me about this error:
> 1. I find it quite strange that these ddr_firmwares are copied from
> the deploy directory to the working directory of imx-boot. I would
> expect that to be the other way around.
> 2. The file /bl31-imx8mq.bin/ is not in the imx-boot-tools folder,
> even after a successful build with DISTRO='poky'. (Maybe it was there
> during build and was just removed afterwards?)
>
> From my point of view it looks like the set of files that get put into
> imx-boot-tools/ is dependant on the DISTRO value? When using
> DISTRO='poky' the correct set is placed and copied. When using
> DISTRO='poky-mydist' the files that are placed into the folder are not
> the files that are supposed to be copied from there, hence the error.
>
> Am I interpreting this situation correctly?
> Is there an error in my /poky-mydist/ distribution?
> Is that something that needs to be fixed in the recipe for imx-boot?
> Or can I somehow configure /poky-mydist.conf/ to be treated exactly
> like /poky.conf/?
>
> Thanks in advance
> Stefan 
>
> 

-- 
-----
Rudolf J Streif
CEO/CTO ibeeto
+1.855.442.3386 x700


[-- Attachment #1.1.2: Type: text/html, Size: 6748 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 499 bytes --]

Re: imx-boot do_compile failing with custom distro #yocto

[stefan.wenninger]

[-- Attachment #1: Type: text/plain, Size: 1964 bytes --]

Hi Rudolf,
I did some digging in imx-atf_1.0.bb and saw that it copies bl31-imx8mq.bin to the imx-boot-tools folder in do_deploy().
In imx-boot_0.2.bb I can see that its do_compile is setup to run after imx-atf's do_deploy:

> 
> do_compile[depends] += " \
> virtual/bootloader:do_deploy \
> ${@' '.join('%s:do_deploy' % r for r in '${IMX_FIRMWARE}'.split() )} \
> imx-atf:do_deploy \
> ${IMX_M4_DEMOS} \
> ${@bb.utils.contains('COMBINED_FEATURES', 'optee',
> 'optee-os-imx:do_deploy', '', d)} \
> "
> 

As I have learned that is exactly the mechanism used to instruct one recipe to only be executed *after* a specific step in another recipe.
I think that if that piece of code was not there, my problem would be exactly what you described: imx-boot:do_compile executing after imx-atf:do_install but before imx-atf:do_deploy.

I followed your advice to build imx-atf on its own (which worked) and I think I know why my build failed:
I was not aware of imx-boot depending on imx-atf. While I was trying to get my new distro to work I at one point deleted the imx-boot-tools folder manually. I now realise that that is not a smart move.
Additionally I ran bitbake -c clean imx-boot. But not bitbake -c clean imx-atf , because I was not aware of it existing.
That likely led to imx-boot being rebuilt at next build, but imx-atf being considered up-to-date by bitbake. So it didn't bother building it again.
But because I manually removed some of the files imx-atf would produce, and upon which imx-boot depends, the build failed.

So in the end your explanations, hints and advice helped me understand more about how bitbake recipes interact with each other.
It would have probably been very difficult for you (or anyone) to spot my real problem since you could never have anticipated me just taking a shotgun to the files in my deploy directory :D
Nevertheless I want to thank you for your help.

I consider this issue solved.

Stefan

[-- Attachment #2: Type: text/html, Size: 2291 bytes --]