Problems having both git and npmsw sources in recipe #bitbake

Problems having both git and npmsw sources in recipe #bitbake

[public]

[-- Attachment #1: Type: text/plain, Size: 2300 bytes --]

Hi

I have posted the following topic under the group Openembedded-issues. I am a little unsure if that is the right place, as there seems to be little or no activity in that group?

Openembedded-issues@lists.openembedded.org | Problems having both git and npmsw sources in recipe ( https://lists.openembedded.org/g/Openembedded-issues/topic/problems_having_both_git_and/98260091?p=,,,20,0,0,0::recentpostdate/sticky,,,20,2,0,98260091,previd%3D1681471653367016231,nextid%3D1328543407000000000&previd=1681471653367016231&nextid=1328543407000000000 )

Hi

I am experiencing problems with a recipe where I have both a git and a npmsw source in the SRC_URI. It seems that when the fetcher is fetching from the shrinkwrap, the SRCREV variable has been deleted but it still ends up fetching from the git source resulting in an error because SRCREV is undefined.

The problem occurs because the Fetch class defaults to the value of SRC_URI if the urls parameter contains an empty list. The following change to the npmsw.py file seems to solve the problem:

diff --git a/bitbake/lib/bb/fetch2/npmsw.py b/bitbake/lib/bb/fetch2/npmsw.py
index a8c4d3528f..4d700a537a 100644
--- a/bitbake/lib/bb/fetch2/npmsw.py
+++ b/bitbake/lib/bb/fetch2/npmsw.py
@@ -193,7 +193,9 @@ class NpmShrinkWrap(FetchMethod):
# This fetcher resolves multiple URIs from a shrinkwrap file and then
# forwards it to a proxy fetcher. The management of the donestamp file,
# the lockfile and the checksums are forwarded to the proxy fetcher.
-        ud.proxy = Fetch([dep["url"] for dep in ud.deps if dep["url"]], data)
+        shrinkwrap_urls = [dep["url"] for dep in ud.deps if dep["url"]]
+        if shrinkwrap_urls:
+            ud.proxy = Fetch(shrinkwrap_urls, data)
ud.needdonestamp = False

Are this change OK or can it result in other issues?

Also more generally why does the Fetch class default to the value of SRC_URI variable if the urls parameter contains an empty list? Isn't this a strange implementation? It also seems to result in issues if the shrinkwrap file contains no dependencies.

Please note that I am fairly new to bitbake and completely new to the source code. :)

Kind regards,
Svend Meyland Nicolaisen

[-- Attachment #2: Type: text/html, Size: 4237 bytes --]

Re: [bitbake-devel] Problems having both git and npmsw sources in recipe #bitbake

[Richard Purdie]

On Fri, 2023-04-14 at 04:41 -0700, public@smn.dk wrote:
> I have posted the following topic under the group Openembedded-issues. I am a little unsure if that is the right place, as there seems to be little or no activity in that group? 
> 
> Openembedded-issues@lists.openembedded.org | Problems having both git and npmsw sources in recipe
> 
> Hi
> 
> I am experiencing problems with a recipe where I have both a git and a npmsw source in the SRC_URI. It seems that when the fetcher is fetching from the shrinkwrap, the SRCREV variable has been deleted but it still ends up fetching from the git source resulting in an error because SRCREV is undefined.
> 
> The problem occurs because the Fetch class defaults to the value of SRC_URI if the urls parameter contains an empty list. The following change to the npmsw.py file seems to solve the problem:
> 
> diff --git a/bitbake/lib/bb/fetch2/npmsw.py b/bitbake/lib/bb/fetch2/npmsw.py
> index a8c4d3528f..4d700a537a 100644
> --- a/bitbake/lib/bb/fetch2/npmsw.py
> +++ b/bitbake/lib/bb/fetch2/npmsw.py
> @@ -193,7 +193,9 @@ class NpmShrinkWrap(FetchMethod):
>          # This fetcher resolves multiple URIs from a shrinkwrap file and then
>          # forwards it to a proxy fetcher. The management of the donestamp file,
>          # the lockfile and the checksums are forwarded to the proxy fetcher.
> -        ud.proxy = Fetch([dep["url"] for dep in ud.deps if dep["url"]], data)
> +        shrinkwrap_urls = [dep["url"] for dep in ud.deps if dep["url"]]
> +        if shrinkwrap_urls:
> +            ud.proxy = Fetch(shrinkwrap_urls, data)
>          ud.needdonestamp = False
> 
> Are this change OK or can it result in other issues?
> 
> Also more generally why does the Fetch class default to the value of SRC_URI variable if the urls parameter contains an empty list? Isn't this a strange implementation? It also seems to result in issues if the shrinkwrap file contains no dependencies.
> 
> Please note that I am fairly new to bitbake and completely new to the source code. :)

I think your patch is right. The Fetch class defaults to use SRC_URI
for legacy reasons based on how things were abstracted and extended, it
made sense at the time. I'm not sure if we still need it or not now.

If you send the above patch as a proper patch we can test it and likely
merge.

Cheers,

Richard

Re: Problems having both git and npmsw sources in recipe #bitbake

[public]

[-- Attachment #1: Type: text/plain, Size: 140 bytes --]

Hi Richard

Thank you for your answer. I have tried to send in the patch. I hope, that I have done it correctly.

Kind regards,
Svend

[-- Attachment #2: Type: text/html, Size: 166 bytes --]