[PATCH] u-boot: do_compile depends on the value of UBOOT_LOCALVERSION

[PATCH] u-boot: do_compile depends on the value of UBOOT_LOCALVERSION

[Denys Dmytriyenko]

UBOOT_LOCALVERSION is often used to pass SCM commit SHA to the code
via populating .scmversion file at the start of do_compile(). Let's
explicitly mark do_compile() as depending on the the value of this
variable to avoid errors like this:

$ echo 'UBOOT_LOCALVERSION = "${SRCPV}"' >> conf/local.conf
$ bitbake u-boot
Loading cache: 100%
Loaded 0 entries from dependency cache.
WARNING: /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb: Exception during build_dependencies for do_compile
WARNING: /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb: Error during finalise of /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb
ERROR: ExpansionError during parsing /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb
Traceback (most recent call last):
  File "Var <SRCPV>", line 1, in <module>
  File "/OE/poky-master/bitbake/lib/bb/fetch2/__init__.py", line 784, in get_srcrev(d=<bb.data_smart.DataSmart object at 0x7fa0604a7100>, method_name='sortable_revision'):
         if not scms:
    >        raise FetchError("SRCREV was used yet no valid SCM was found in SRC_URI")

bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: SRCREV was used yet no valid SCM was found in SRC_URI
The variable dependency chain for the failure is: SRCPV -> UBOOT_LOCALVERSION -> do_compile

ERROR: Parsing halted due to errors, see error messages above

Signed-off-by: Denys Dmytriyenko <denis@denix.org>
---
 meta/recipes-bsp/u-boot/u-boot.inc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-bsp/u-boot/u-boot.inc b/meta/recipes-bsp/u-boot/u-boot.inc
index b3482dcef3..ec8108f5c9 100644
--- a/meta/recipes-bsp/u-boot/u-boot.inc
+++ b/meta/recipes-bsp/u-boot/u-boot.inc
@@ -23,6 +23,7 @@ PACKAGECONFIG[openssl] = ",,openssl-native"
 # u-boot build system and appended to the u-boot version.  If the .scmversion
 # file already exists it will not be overwritten.
 UBOOT_LOCALVERSION ?= ""
+do_compile[vardepvalue] = "${UBOOT_LOCALVERSION}"
 
 require u-boot-configure.inc
 
-- 
2.25.1

Re: [OE-core] [PATCH] u-boot: do_compile depends on the value of UBOOT_LOCALVERSION

[Richard Purdie]

On Sun, 2023-05-28 at 17:15 +0000, Denys Dmytriyenko wrote:
> UBOOT_LOCALVERSION is often used to pass SCM commit SHA to the code
> via populating .scmversion file at the start of do_compile(). Let's
> explicitly mark do_compile() as depending on the the value of this
> variable to avoid errors like this:
> 
> $ echo 'UBOOT_LOCALVERSION = "${SRCPV}"' >> conf/local.conf
> $ bitbake u-boot
> Loading cache: 100%
> Loaded 0 entries from dependency cache.
> WARNING: /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb: Exception during build_dependencies for do_compile
> WARNING: /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb: Error during finalise of /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb
> ERROR: ExpansionError during parsing /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb
> Traceback (most recent call last):
>   File "Var <SRCPV>", line 1, in <module>
>   File "/OE/poky-master/bitbake/lib/bb/fetch2/__init__.py", line 784, in get_srcrev(d=<bb.data_smart.DataSmart object at 0x7fa0604a7100>, method_name='sortable_revision'):
>          if not scms:
>     >        raise FetchError("SRCREV was used yet no valid SCM was found in SRC_URI")
> 
> bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: SRCREV was used yet no valid SCM was found in SRC_URI
> The variable dependency chain for the failure is: SRCPV -> UBOOT_LOCALVERSION -> do_compile
> 
> ERROR: Parsing halted due to errors, see error messages above
> 
> Signed-off-by: Denys Dmytriyenko <denis@denix.org>
> ---
>  meta/recipes-bsp/u-boot/u-boot.inc | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/recipes-bsp/u-boot/u-boot.inc b/meta/recipes-bsp/u-boot/u-boot.inc
> index b3482dcef3..ec8108f5c9 100644
> --- a/meta/recipes-bsp/u-boot/u-boot.inc
> +++ b/meta/recipes-bsp/u-boot/u-boot.inc
> @@ -23,6 +23,7 @@ PACKAGECONFIG[openssl] = ",,openssl-native"
>  # u-boot build system and appended to the u-boot version.  If the .scmversion
>  # file already exists it will not be overwritten.
>  UBOOT_LOCALVERSION ?= ""
> +do_compile[vardepvalue] = "${UBOOT_LOCALVERSION}"
>  
>  require u-boot-configure.inc
>  

How would I reproduce this?

The fix looks a bit like a workaround and I'd like to understand what
is really going on here.

Cheers,

Richard

Re: [OE-core] [PATCH] u-boot: do_compile depends on the value of UBOOT_LOCALVERSION

[Denys Dmytriyenko]

On Mon, May 29, 2023 at 01:41:00PM +0100, Richard Purdie wrote:
> On Sun, 2023-05-28 at 17:15 +0000, Denys Dmytriyenko wrote:
> > UBOOT_LOCALVERSION is often used to pass SCM commit SHA to the code
> > via populating .scmversion file at the start of do_compile(). Let's
> > explicitly mark do_compile() as depending on the the value of this
> > variable to avoid errors like this:
> > 
> > $ echo 'UBOOT_LOCALVERSION = "${SRCPV}"' >> conf/local.conf
> > $ bitbake u-boot

^^^^^^^^


> > Loading cache: 100%
> > Loaded 0 entries from dependency cache.
> > WARNING: /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb: Exception during build_dependencies for do_compile
> > WARNING: /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb: Error during finalise of /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb
> > ERROR: ExpansionError during parsing /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb
> > Traceback (most recent call last):
> >   File "Var <SRCPV>", line 1, in <module>
> >   File "/OE/poky-master/bitbake/lib/bb/fetch2/__init__.py", line 784, in get_srcrev(d=<bb.data_smart.DataSmart object at 0x7fa0604a7100>, method_name='sortable_revision'):
> >          if not scms:
> >     >        raise FetchError("SRCREV was used yet no valid SCM was found in SRC_URI")
> > 
> > bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: SRCREV was used yet no valid SCM was found in SRC_URI
> > The variable dependency chain for the failure is: SRCPV -> UBOOT_LOCALVERSION -> do_compile
> > 
> > ERROR: Parsing halted due to errors, see error messages above
> > 
> > Signed-off-by: Denys Dmytriyenko <denis@denix.org>
> > ---
> >  meta/recipes-bsp/u-boot/u-boot.inc | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/meta/recipes-bsp/u-boot/u-boot.inc b/meta/recipes-bsp/u-boot/u-boot.inc
> > index b3482dcef3..ec8108f5c9 100644
> > --- a/meta/recipes-bsp/u-boot/u-boot.inc
> > +++ b/meta/recipes-bsp/u-boot/u-boot.inc
> > @@ -23,6 +23,7 @@ PACKAGECONFIG[openssl] = ",,openssl-native"
> >  # u-boot build system and appended to the u-boot version.  If the .scmversion
> >  # file already exists it will not be overwritten.
> >  UBOOT_LOCALVERSION ?= ""
> > +do_compile[vardepvalue] = "${UBOOT_LOCALVERSION}"
> >  
> >  require u-boot-configure.inc
> >  
> 
> How would I reproduce this?

The instructions are in the description above.


> The fix looks a bit like a workaround and I'd like to understand what
> is really going on here.

This started happening recently after your change that empties SRC_URI for 
code parser cache handling in BB_HASH_CODEPARSER_VALS, as reported here:

https://lists.openembedded.org/g/openembedded-core/message/181726

-- 
Denys

Re: [OE-core] [PATCH] u-boot: do_compile depends on the value of UBOOT_LOCALVERSION

[Richard Purdie]

On Mon, 2023-05-29 at 12:06 -0400, Denys Dmytriyenko wrote:
> On Mon, May 29, 2023 at 01:41:00PM +0100, Richard Purdie wrote:
> > On Sun, 2023-05-28 at 17:15 +0000, Denys Dmytriyenko wrote:
> > > UBOOT_LOCALVERSION is often used to pass SCM commit SHA to the code
> > > via populating .scmversion file at the start of do_compile(). Let's
> > > explicitly mark do_compile() as depending on the the value of this
> > > variable to avoid errors like this:
> > > 
> > > $ echo 'UBOOT_LOCALVERSION = "${SRCPV}"' >> conf/local.conf
> > > $ bitbake u-boot
> 
> ^^^^^^^^
> 
> 
> > > Loading cache: 100%
> > > Loaded 0 entries from dependency cache.
> > > WARNING: /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb: Exception during build_dependencies for do_compile
> > > WARNING: /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb: Error during finalise of /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb
> > > ERROR: ExpansionError during parsing /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb
> > > Traceback (most recent call last):
> > >   File "Var <SRCPV>", line 1, in <module>
> > >   File "/OE/poky-master/bitbake/lib/bb/fetch2/__init__.py", line 784, in get_srcrev(d=<bb.data_smart.DataSmart object at 0x7fa0604a7100>, method_name='sortable_revision'):
> > >          if not scms:
> > >     >        raise FetchError("SRCREV was used yet no valid SCM was found in SRC_URI")
> > > 
> > > bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: SRCREV was used yet no valid SCM was found in SRC_URI
> > > The variable dependency chain for the failure is: SRCPV -> UBOOT_LOCALVERSION -> do_compile
> > > 
> > > ERROR: Parsing halted due to errors, see error messages above
> > > 
> > > Signed-off-by: Denys Dmytriyenko <denis@denix.org>
> > > ---
> > >  meta/recipes-bsp/u-boot/u-boot.inc | 1 +
> > >  1 file changed, 1 insertion(+)
> > > 
> > > diff --git a/meta/recipes-bsp/u-boot/u-boot.inc b/meta/recipes-bsp/u-boot/u-boot.inc
> > > index b3482dcef3..ec8108f5c9 100644
> > > --- a/meta/recipes-bsp/u-boot/u-boot.inc
> > > +++ b/meta/recipes-bsp/u-boot/u-boot.inc
> > > @@ -23,6 +23,7 @@ PACKAGECONFIG[openssl] = ",,openssl-native"
> > >  # u-boot build system and appended to the u-boot version.  If the .scmversion
> > >  # file already exists it will not be overwritten.
> > >  UBOOT_LOCALVERSION ?= ""
> > > +do_compile[vardepvalue] = "${UBOOT_LOCALVERSION}"
> > >  
> > >  require u-boot-configure.inc
> > >  
> > 
> > How would I reproduce this?
> 
> The instructions are in the description above.

Sorry, I'd assumed you'd need a specific target MACHINE so I somehow
didn't think they were complete.
> 

> > The fix looks a bit like a workaround and I'd like to understand what
> > is really going on here.
> 
> This started happening recently after your change that empties SRC_URI for 
> code parser cache handling in BB_HASH_CODEPARSER_VALS, as reported here:
> 
> https://lists.openembedded.org/g/openembedded-core/message/181726

It would be useful if the commit message had mentioned that!

I suspect the correct fix is to add SRCPV=xxx to
BB_HASH_CODEPARSER_VALS

Could you see if that works instead?

Cheers,

Richard

Re: [OE-core] [PATCH] u-boot: do_compile depends on the value of UBOOT_LOCALVERSION

[Denys Dmytriyenko]

On Tue, May 30, 2023 at 03:47:07PM +0100, Richard Purdie wrote:
> On Mon, 2023-05-29 at 12:06 -0400, Denys Dmytriyenko wrote:
> > On Mon, May 29, 2023 at 01:41:00PM +0100, Richard Purdie wrote:
> > > On Sun, 2023-05-28 at 17:15 +0000, Denys Dmytriyenko wrote:
> > > > UBOOT_LOCALVERSION is often used to pass SCM commit SHA to the code
> > > > via populating .scmversion file at the start of do_compile(). Let's
> > > > explicitly mark do_compile() as depending on the the value of this
> > > > variable to avoid errors like this:
> > > > 
> > > > $ echo 'UBOOT_LOCALVERSION = "${SRCPV}"' >> conf/local.conf
> > > > $ bitbake u-boot
> > 
> > ^^^^^^^^
> > 
> > 
> > > > Loading cache: 100%
> > > > Loaded 0 entries from dependency cache.
> > > > WARNING: /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb: Exception during build_dependencies for do_compile
> > > > WARNING: /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb: Error during finalise of /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb
> > > > ERROR: ExpansionError during parsing /OE/poky-master/meta/recipes-bsp/u-boot/u-boot_2023.04.bb
> > > > Traceback (most recent call last):
> > > >   File "Var <SRCPV>", line 1, in <module>
> > > >   File "/OE/poky-master/bitbake/lib/bb/fetch2/__init__.py", line 784, in get_srcrev(d=<bb.data_smart.DataSmart object at 0x7fa0604a7100>, method_name='sortable_revision'):
> > > >          if not scms:
> > > >     >        raise FetchError("SRCREV was used yet no valid SCM was found in SRC_URI")
> > > > 
> > > > bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: SRCREV was used yet no valid SCM was found in SRC_URI
> > > > The variable dependency chain for the failure is: SRCPV -> UBOOT_LOCALVERSION -> do_compile
> > > > 
> > > > ERROR: Parsing halted due to errors, see error messages above
> > > > 
> > > > Signed-off-by: Denys Dmytriyenko <denis@denix.org>
> > > > ---
> > > >  meta/recipes-bsp/u-boot/u-boot.inc | 1 +
> > > >  1 file changed, 1 insertion(+)
> > > > 
> > > > diff --git a/meta/recipes-bsp/u-boot/u-boot.inc b/meta/recipes-bsp/u-boot/u-boot.inc
> > > > index b3482dcef3..ec8108f5c9 100644
> > > > --- a/meta/recipes-bsp/u-boot/u-boot.inc
> > > > +++ b/meta/recipes-bsp/u-boot/u-boot.inc
> > > > @@ -23,6 +23,7 @@ PACKAGECONFIG[openssl] = ",,openssl-native"
> > > >  # u-boot build system and appended to the u-boot version.  If the .scmversion
> > > >  # file already exists it will not be overwritten.
> > > >  UBOOT_LOCALVERSION ?= ""
> > > > +do_compile[vardepvalue] = "${UBOOT_LOCALVERSION}"
> > > >  
> > > >  require u-boot-configure.inc
> > > >  
> > > 
> > > How would I reproduce this?
> > 
> > The instructions are in the description above.
> 
> Sorry, I'd assumed you'd need a specific target MACHINE so I somehow
> didn't think they were complete.

No, any qemu target should do.


> > > The fix looks a bit like a workaround and I'd like to understand what
> > > is really going on here.
> > 
> > This started happening recently after your change that empties SRC_URI for 
> > code parser cache handling in BB_HASH_CODEPARSER_VALS, as reported here:
> > 
> > https://lists.openembedded.org/g/openembedded-core/message/181726
> 
> It would be useful if the commit message had mentioned that!
> 
> I suspect the correct fix is to add SRCPV=xxx to
> BB_HASH_CODEPARSER_VALS
> 
> Could you see if that works instead?

Yes, it works. Do you need a patch?

-- 
Denys