All of lore.kernel.org
 help / color / mirror / Atom feed
* [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts
@ 2023-05-27  6:25 belouargamohamed
  2023-05-27  6:25 ` [kirkstone][PATCH 2/5] classes: npm: Handle peer dependencies for npm packages belouargamohamed
                   ` (4 more replies)
  0 siblings, 5 replies; 10+ messages in thread
From: belouargamohamed @ 2023-05-27  6:25 UTC (permalink / raw)
  To: openembedded-core; +Cc: e.aubineau, f.lahoudere, BELOUARGA Mohamed

From: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>

Npm packages do not have yocto friendly names. fore instance we can have names like
"@example/npmPackage"

npm fetcher has a function that convert these names to yocto friendly names.
But in recipe tool we have an other function (duplicate).

Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
---
 scripts/lib/recipetool/create_npm.py | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py
index 3394a89970..e667a4d19b 100644
--- a/scripts/lib/recipetool/create_npm.py
+++ b/scripts/lib/recipetool/create_npm.py
@@ -13,6 +13,7 @@ import sys
 import tempfile
 import bb
 from bb.fetch2.npm import NpmEnvironment
+from bb.fetch2.npm import npm_package
 from bb.fetch2.npmsw import foreach_dependencies
 from recipetool.create import RecipeHandler
 from recipetool.create import get_license_md5sums
@@ -30,15 +31,6 @@ def tinfoil_init(instance):
 class NpmRecipeHandler(RecipeHandler):
     """Class to handle the npm recipe creation"""
 
-    @staticmethod
-    def _npm_name(name):
-        """Generate a Yocto friendly npm name"""
-        name = re.sub("/", "-", name)
-        name = name.lower()
-        name = re.sub(r"[^\-a-z0-9]", "", name)
-        name = name.strip("-")
-        return name
-
     @staticmethod
     def _get_registry(lines):
         """Get the registry value from the 'npm://registry' url"""
@@ -143,7 +135,7 @@ class NpmRecipeHandler(RecipeHandler):
 
         # Handle the dependencies
         def _handle_dependency(name, params, deptree):
-            suffix = "-".join([self._npm_name(dep) for dep in deptree])
+            suffix = "-".join([npm_package(dep) for dep in deptree])
             destdirs = [os.path.join("node_modules", dep) for dep in deptree]
             destdir = os.path.join(*destdirs)
             packages["${PN}-" + suffix] = destdir
@@ -173,7 +165,7 @@ class NpmRecipeHandler(RecipeHandler):
         if "name" not in data or "version" not in data:
             return False
 
-        extravalues["PN"] = self._npm_name(data["name"])
+        extravalues["PN"] = npm_package(data["name"])
         extravalues["PV"] = data["version"]
 
         if "description" in data:
-- 
2.25.1



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [kirkstone][PATCH 2/5] classes: npm: Handle peer dependencies for npm packages
  2023-05-27  6:25 [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts belouargamohamed
@ 2023-05-27  6:25 ` belouargamohamed
  2023-05-27  6:25 ` [kirkstone][PATCH 3/5] recipetool: create: npm: Add support for the new format of the shrinkwrap file belouargamohamed
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 10+ messages in thread
From: belouargamohamed @ 2023-05-27  6:25 UTC (permalink / raw)
  To: openembedded-core; +Cc: e.aubineau, f.lahoudere, BELOUARGA Mohamed

From: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>

NPM changed its manner to handle peer dependencies over its versions:
 - Before NPM 3: NPM installs automatically peer dependencies
 - Between NPM 3 and 7: NPM shows a warning about peer dependencies
 - After NPM 3: NPM reworked its manner how to handle peer dependencies

The shrinkwrap doesn't have the parameters of the peer dependencies, so we cannot
fetch them. in the same time peer dependencies are not direct dependencies, they should
be installed as run time dependencies.

Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
---
 meta/classes/npm.bbclass | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index 8379c7b988..abc4c6f92b 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -108,6 +108,7 @@ python npm_do_configure() {
     import tempfile
     from bb.fetch2.npm import NpmEnvironment
     from bb.fetch2.npm import npm_unpack
+    from bb.fetch2.npm import npm_package
     from bb.fetch2.npmsw import foreach_dependencies
     from bb.progress import OutOfProgressHandler
     from oe.npm_registry import NpmRegistry
@@ -168,6 +169,7 @@ python npm_do_configure() {
     if has_shrinkwrap_file:
        cached_shrinkwrap = copy.deepcopy(orig_shrinkwrap)
        cached_shrinkwrap.pop("dependencies", None)
+       cached_shrinkwrap["packages"][""].pop("peerDependencies", None)
 
     # Manage the dependencies
     progress = OutOfProgressHandler(d, r"^(\d+)/(\d+)$")
@@ -202,6 +204,19 @@ python npm_do_configure() {
     if has_shrinkwrap_file:
         foreach_dependencies(orig_shrinkwrap, _count_dependency, dev)
         foreach_dependencies(orig_shrinkwrap, _cache_dependency, dev)
+    
+    # Manage Peer Dependencies
+    if has_shrinkwrap_file:
+        packages = orig_shrinkwrap.get("packages", {})
+        peer_deps = packages.get("", {}).get("peerDependencies", {})
+        package_runtime_dependencies = d.getVar("RDEPENDS:%s" % d.getVar("PN"))
+        
+        for peer_dep in peer_deps:
+            peer_dep_yocto_name = npm_package(peer_dep)
+            if peer_dep_yocto_name not in package_runtime_dependencies:
+                bb.warn(peer_dep + " is a peer dependencie that is not in RDEPENDS variable. " + 
+                "Please add this peer dependencie to the RDEPENDS variable as %s and generate its recipe with devtool"
+                % peer_dep_yocto_name)
 
     # Configure the main package
     with tempfile.TemporaryDirectory() as tmpdir:
@@ -277,6 +292,9 @@ python npm_do_compile() {
         args.append(("target_arch", d.getVar("NPM_ARCH")))
         args.append(("build-from-source", "true"))
 
+        # Don't install peer dependencies as they should be in RDEPENDS variable
+        args.append(("legacy-peer-deps", "true"))
+
         # Pack and install the main package
         (tarball, _) = npm_pack(env, d.getVar("NPM_PACKAGE"), tmpdir)
         cmd = "npm install %s %s" % (shlex.quote(tarball), d.getVar("EXTRA_OENPM"))
-- 
2.25.1



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [kirkstone][PATCH 3/5] recipetool: create: npm: Add support for the new format of the shrinkwrap file
  2023-05-27  6:25 [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts belouargamohamed
  2023-05-27  6:25 ` [kirkstone][PATCH 2/5] classes: npm: Handle peer dependencies for npm packages belouargamohamed
@ 2023-05-27  6:25 ` belouargamohamed
  2023-05-27  6:25 ` [kirkstone][PATCH 4/5] recipetool: create: npm: Add support to handle peer dependencies belouargamohamed
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 10+ messages in thread
From: belouargamohamed @ 2023-05-27  6:25 UTC (permalink / raw)
  To: openembedded-core; +Cc: e.aubineau, f.lahoudere, BELOUARGA Mohamed

From: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>

The shrinkwrap file changed its format, but npm does not version this file. So we can use it properly.
The actual changes make the script check if the npm package has dependencies in the actual shrinkwrap format.

Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
---
 scripts/lib/recipetool/create_npm.py | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py
index e667a4d19b..25e7ddb472 100644
--- a/scripts/lib/recipetool/create_npm.py
+++ b/scripts/lib/recipetool/create_npm.py
@@ -134,11 +134,10 @@ class NpmRecipeHandler(RecipeHandler):
                     licfiles.append(os.path.relpath(readme, srctree))
 
         # Handle the dependencies
-        def _handle_dependency(name, params, deptree):
+        def _handle_dependency(name, params, destdir):
+            deptree = destdir.split('node_modules/')
             suffix = "-".join([npm_package(dep) for dep in deptree])
-            destdirs = [os.path.join("node_modules", dep) for dep in deptree]
-            destdir = os.path.join(*destdirs)
-            packages["${PN}-" + suffix] = destdir
+            packages["${PN}" + suffix] = destdir
             _licfiles_append_fallback_readme_files(destdir)
 
         with open(shrinkwrap_file, "r") as f:
@@ -234,7 +233,7 @@ class NpmRecipeHandler(RecipeHandler):
             value = origvalue.replace("version=" + data["version"], "version=${PV}")
             value = value.replace("version=latest", "version=${PV}")
             values = [line.strip() for line in value.strip('\n').splitlines()]
-            if "dependencies" in shrinkwrap:
+            if "dependencies" in shrinkwrap.get("packages", {}).get("", {}):
                 values.append(url_recipe)
             return values, None, 4, False
 
-- 
2.25.1



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [kirkstone][PATCH 4/5] recipetool: create: npm: Add support to handle peer dependencies
  2023-05-27  6:25 [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts belouargamohamed
  2023-05-27  6:25 ` [kirkstone][PATCH 2/5] classes: npm: Handle peer dependencies for npm packages belouargamohamed
  2023-05-27  6:25 ` [kirkstone][PATCH 3/5] recipetool: create: npm: Add support for the new format of the shrinkwrap file belouargamohamed
@ 2023-05-27  6:25 ` belouargamohamed
  2023-05-27  6:25 ` [kirkstone][PATCH 5/5] classes: npm: Add support for the new format of the shrinkwrap file belouargamohamed
  2023-05-29 15:13 ` [OE-core] [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts Steve Sakoman
  4 siblings, 0 replies; 10+ messages in thread
From: belouargamohamed @ 2023-05-27  6:25 UTC (permalink / raw)
  To: openembedded-core; +Cc: e.aubineau, f.lahoudere, BELOUARGA Mohamed

From: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>

NPM changed its manner to handle peer dependencies over its versions:

 - Before NPM 3: NPM installs automatically peer dependencies
 - Between NPM 3 and 7: NPM shows a warning about peer dependencies
 - After NPM 3: NPM reworked its manner how to handle peer dependencies

The shrinkwrap doesn't have the parameters of the peer dependencies, so we cannot
fetch them. in the same time peer dependencies are not direct dependencies, they should
be installed as run time dependencies.

Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
---
 scripts/lib/recipetool/create_npm.py | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py
index 25e7ddb472..113a89f6a6 100644
--- a/scripts/lib/recipetool/create_npm.py
+++ b/scripts/lib/recipetool/create_npm.py
@@ -146,6 +146,23 @@ class NpmRecipeHandler(RecipeHandler):
         foreach_dependencies(shrinkwrap, _handle_dependency, dev)
 
         return licfiles, packages
+    
+    # Handle the peer dependencies   
+    def _handle_peer_dependency(self, shrinkwrap_file):
+        """Check if package has peer dependencies and show warning if it is the case"""
+        with open(shrinkwrap_file, "r") as f:
+            shrinkwrap = json.load(f)
+        
+        packages = shrinkwrap.get("packages", {})
+        peer_deps = packages.get("", {}).get("peerDependencies", {})
+        
+        for peer_dep in peer_deps:
+            peer_dep_yocto_name = npm_package(peer_dep)
+            bb.warn(peer_dep + " is a peer dependencie of the actual package. " + 
+            "Please add this peer dependencie to the RDEPENDS variable as %s and generate its recipe with devtool"
+            % peer_dep_yocto_name)
+
+
 
     def process(self, srctree, classes, lines_before, lines_after, handled, extravalues):
         """Handle the npm recipe creation"""
@@ -283,6 +300,9 @@ class NpmRecipeHandler(RecipeHandler):
         classes.append("npm")
         handled.append("buildsystem")
 
+        # Check if package has peer dependencies and inform the user
+        self._handle_peer_dependency(shrinkwrap_file)
+
         return True
 
 def register_recipe_handlers(handlers):
-- 
2.25.1



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [kirkstone][PATCH 5/5] classes: npm: Add support for the new format of the shrinkwrap file
  2023-05-27  6:25 [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts belouargamohamed
                   ` (2 preceding siblings ...)
  2023-05-27  6:25 ` [kirkstone][PATCH 4/5] recipetool: create: npm: Add support to handle peer dependencies belouargamohamed
@ 2023-05-27  6:25 ` belouargamohamed
  2023-05-29 15:13 ` [OE-core] [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts Steve Sakoman
  4 siblings, 0 replies; 10+ messages in thread
From: belouargamohamed @ 2023-05-27  6:25 UTC (permalink / raw)
  To: openembedded-core; +Cc: e.aubineau, f.lahoudere, BELOUARGA Mohamed

From: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>

1 - Adapt do_configure to the new format of the shrinkwrap

2 - Remove useless function _npmsw_dependency_dict because the dictionnary
    is already given by npmsw:foreach_dependencies

3 - Rename arguments of callback functions

Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
---
 meta/classes/npm.bbclass | 27 ++++++---------------------
 1 file changed, 6 insertions(+), 21 deletions(-)

diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index abc4c6f92b..6899a64909 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -129,22 +129,6 @@ python npm_do_configure() {
         sha512 = bb.utils.sha512_file(tarball)
         return "sha512-" + base64.b64encode(bytes.fromhex(sha512)).decode()
 
-    def _npmsw_dependency_dict(orig, deptree):
-        """
-        Return the sub dictionary in the 'orig' dictionary corresponding to the
-        'deptree' dependency tree. This function follows the shrinkwrap file
-        format.
-        """
-        ptr = orig
-        for dep in deptree:
-            if "dependencies" not in ptr:
-                ptr["dependencies"] = {}
-            ptr = ptr["dependencies"]
-            if dep not in ptr:
-                ptr[dep] = {}
-            ptr = ptr[dep]
-        return ptr
-
     # Manage the manifest file and shrinkwrap files
     orig_manifest_file = d.expand("${S}/package.json")
     orig_shrinkwrap_file = d.expand("${S}/npm-shrinkwrap.json")
@@ -176,24 +160,25 @@ python npm_do_configure() {
     progress_total = 1 # also count the main package
     progress_done = 0
 
-    def _count_dependency(name, params, deptree):
+    def _count_dependency(name, params, destsuffix):
         nonlocal progress_total
         progress_total += 1
 
-    def _cache_dependency(name, params, deptree):
-        destsubdirs = [os.path.join("node_modules", dep) for dep in deptree]
-        destsuffix = os.path.join(*destsubdirs)
+    def _cache_dependency(name, params, destsuffix):
         with tempfile.TemporaryDirectory() as tmpdir:
             # Add the dependency to the npm cache
             destdir = os.path.join(d.getVar("S"), destsuffix)
             (tarball, pkg) = npm_pack(env, destdir, tmpdir)
             _npm_cache_add(tarball, pkg)
             # Add its signature to the cached shrinkwrap
-            dep = _npmsw_dependency_dict(cached_shrinkwrap, deptree)
+            dep = params
             dep["version"] = pkg['version']
             dep["integrity"] = _npm_integrity(tarball)
             if params.get("dev", False):
                 dep["dev"] = True
+            if "dependencies" not in cached_shrinkwrap:
+                cached_shrinkwrap["dependencies"] = {}
+            cached_shrinkwrap["dependencies"][name] = dep
             # Display progress
             nonlocal progress_done
             progress_done += 1
-- 
2.25.1



^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [OE-core] [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts
  2023-05-27  6:25 [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts belouargamohamed
                   ` (3 preceding siblings ...)
  2023-05-27  6:25 ` [kirkstone][PATCH 5/5] classes: npm: Add support for the new format of the shrinkwrap file belouargamohamed
@ 2023-05-29 15:13 ` Steve Sakoman
  2023-05-29 20:16   ` belouargamohamed
  4 siblings, 1 reply; 10+ messages in thread
From: Steve Sakoman @ 2023-05-29 15:13 UTC (permalink / raw)
  To: belouargamohamed
  Cc: openembedded-core, e.aubineau, f.lahoudere, BELOUARGA Mohamed

Are these same issues present in the master branch?  If so, this
series should be submitted for the master branch first, and I will
then backport to kirkstone (and mickledore).

Steve

On Fri, May 26, 2023 at 8:25 PM <belouargamohamed@gmail.com> wrote:
>
> From: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
>
> Npm packages do not have yocto friendly names. fore instance we can have names like
> "@example/npmPackage"
>
> npm fetcher has a function that convert these names to yocto friendly names.
> But in recipe tool we have an other function (duplicate).
>
> Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
> ---
>  scripts/lib/recipetool/create_npm.py | 14 +++-----------
>  1 file changed, 3 insertions(+), 11 deletions(-)
>
> diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py
> index 3394a89970..e667a4d19b 100644
> --- a/scripts/lib/recipetool/create_npm.py
> +++ b/scripts/lib/recipetool/create_npm.py
> @@ -13,6 +13,7 @@ import sys
>  import tempfile
>  import bb
>  from bb.fetch2.npm import NpmEnvironment
> +from bb.fetch2.npm import npm_package
>  from bb.fetch2.npmsw import foreach_dependencies
>  from recipetool.create import RecipeHandler
>  from recipetool.create import get_license_md5sums
> @@ -30,15 +31,6 @@ def tinfoil_init(instance):
>  class NpmRecipeHandler(RecipeHandler):
>      """Class to handle the npm recipe creation"""
>
> -    @staticmethod
> -    def _npm_name(name):
> -        """Generate a Yocto friendly npm name"""
> -        name = re.sub("/", "-", name)
> -        name = name.lower()
> -        name = re.sub(r"[^\-a-z0-9]", "", name)
> -        name = name.strip("-")
> -        return name
> -
>      @staticmethod
>      def _get_registry(lines):
>          """Get the registry value from the 'npm://registry' url"""
> @@ -143,7 +135,7 @@ class NpmRecipeHandler(RecipeHandler):
>
>          # Handle the dependencies
>          def _handle_dependency(name, params, deptree):
> -            suffix = "-".join([self._npm_name(dep) for dep in deptree])
> +            suffix = "-".join([npm_package(dep) for dep in deptree])
>              destdirs = [os.path.join("node_modules", dep) for dep in deptree]
>              destdir = os.path.join(*destdirs)
>              packages["${PN}-" + suffix] = destdir
> @@ -173,7 +165,7 @@ class NpmRecipeHandler(RecipeHandler):
>          if "name" not in data or "version" not in data:
>              return False
>
> -        extravalues["PN"] = self._npm_name(data["name"])
> +        extravalues["PN"] = npm_package(data["name"])
>          extravalues["PV"] = data["version"]
>
>          if "description" in data:
> --
> 2.25.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#181791): https://lists.openembedded.org/g/openembedded-core/message/181791
> Mute This Topic: https://lists.openembedded.org/mt/99163838/3620601
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [steve@sakoman.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts
  2023-05-29 15:13 ` [OE-core] [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts Steve Sakoman
@ 2023-05-29 20:16   ` belouargamohamed
       [not found]     ` <CAOSpxda3AWOQfskRpCWB0zQMDHs+SMKz0RGqE5z2Nd80DRKp7g@mail.gmail.com>
  0 siblings, 1 reply; 10+ messages in thread
From: belouargamohamed @ 2023-05-29 20:16 UTC (permalink / raw)
  To: openembedded-core

[-- Attachment #1: Type: text/plain, Size: 729 bytes --]

Yes, the same issues are present in master branch.

But this series of patches won't work on master, it needs some adaptation.

Because in kirkstone we have Nodejs 16.19.1 and in master we have Nodejs 18.16, and they generate shrinkwrap files that are slightly different.

For Mickledore, I didn't test yet but I think that it will work like in the master branch.

Note: these changes work only if my patch for bitbake is accepted: https://lists.openembedded.org/g/bitbake-devel/topic/kirkstone_patch_fetch2/99163598?p=,,,20,0,0,0::recentpostdate/sticky,,,20,2,0,99163598,previd%3D1685367798112436212,nextid%3D1683629533018906083&previd=1685367798112436212&nextid=1683629533018906083

Best regards,
BELOUARGA Mohamed

[-- Attachment #2: Type: text/html, Size: 1265 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Fwd: [OE-core] [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts
       [not found]     ` <CAOSpxda3AWOQfskRpCWB0zQMDHs+SMKz0RGqE5z2Nd80DRKp7g@mail.gmail.com>
@ 2023-05-29 22:59       ` Steve Sakoman
  2023-05-31 20:27         ` belouargamohamed
  0 siblings, 1 reply; 10+ messages in thread
From: Steve Sakoman @ 2023-05-29 22:59 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer

Forgot to reply all!

---------- Forwarded message ---------
From: Steve Sakoman <steve@sakoman.com>
Date: Mon, May 29, 2023 at 12:58 PM
Subject: Re: [OE-core] [kirkstone][PATCH 1/5] recipetool: create: npm:
Remove duplicate function to not have future conflicts
To: <belouargamohamed@gmail.com>


On Mon, May 29, 2023 at 10:16 AM <belouargamohamed@gmail.com> wrote:
>
> Yes, the same issues are present in master branch.
>
> But this series of patches won't work on master, it needs some adaptation.

In that case you will also need to submit a modified series for master
(which will hopefully cherry-pick to mickledore)

At that point I can take the kirkstone version of the patch.  The
reason is that we don't want to have issues fixed in a stable branch
that aren't also fixed in master.

Thanks!

Steve

> Because in kirkstone we have Nodejs 16.19.1 and in master we have Nodejs 18.16, and they generate shrinkwrap files that are slightly different.
>
> For Mickledore, I didn't test yet but I think that it will work like in the master branch.
>
>  Note: these changes work only if my patch for bitbake is accepted: https://lists.openembedded.org/g/bitbake-devel/topic/kirkstone_patch_fetch2/99163598?p=,,,20,0,0,0::recentpostdate/sticky,,,20,2,0,99163598,previd%3D1685367798112436212,nextid%3D1683629533018906083&previd=1685367798112436212&nextid=1683629533018906083
>
> Best regards,
> BELOUARGA Mohamed
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#181892): https://lists.openembedded.org/g/openembedded-core/message/181892
> Mute This Topic: https://lists.openembedded.org/mt/99163838/3620601
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [steve@sakoman.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts
  2023-05-29 22:59       ` Fwd: [OE-core] " Steve Sakoman
@ 2023-05-31 20:27         ` belouargamohamed
  0 siblings, 0 replies; 10+ messages in thread
From: belouargamohamed @ 2023-05-31 20:27 UTC (permalink / raw)
  To: openembedded-core

[-- Attachment #1: Type: text/plain, Size: 1572 bytes --]

Hello,

I fixed the problem on master branch.
*For master:*

In order that my changes work, Some patches on bitbake should be accepted:
https://lists.openembedded.org/g/bitbake-devel/message/14815?p=%2C%2C%2C20%2C0%2C0%2C0%3A%3Arecentpostdate%2Fsticky%2C%2Cbelouarga%2C20%2C2%2C0%2C99163598
https://lists.openembedded.org/g/bitbake-devel/message/14819?p=%2C%2C%2C20%2C0%2C0%2C0%3A%3Arecentpostdate%2Fsticky%2C%2Cbelouarga%2C20%2C2%2C0%2C99230772
https://lists.openembedded.org/g/bitbake-devel/message/14820?p=%2C%2C%2C20%2C0%2C0%2C0%3A%3Arecentpostdate%2Fsticky%2C%2Cbelouarga%2C20%2C2%2C0%2C99248868
And:
https://lists.openembedded.org/g/openembedded-core/message/182022?p=%2C%2C%2C20%2C0%2C0%2C0%3A%3Arecentpostdate%2Fsticky%2C%2Cbelouarga%2C20%2C2%2C0%2C99248914
And this series of patches:
https://lists.openembedded.org/g/openembedded-core/message/181961?p=%2C%2C%2C20%2C0%2C0%2C0%3A%3Arecentpostdate%2Fsticky%2C%2Cbelouarga%2C20%2C2%2C0%2C99230551

*For kirkstone:*
In order that my changes work, Some patches on bitbake should be accepted:
https://lists.openembedded.org/g/bitbake-devel/message/14815?p=%2C%2C%2C20%2C0%2C0%2C0%3A%3Arecentpostdate%2Fsticky%2C%2Cbelouarga%2C20%2C2%2C0%2C99163598
https://lists.openembedded.org/g/bitbake-devel/message/14819?p=%2C%2C%2C20%2C0%2C0%2C0%3A%3Arecentpostdate%2Fsticky%2C%2Cbelouarga%2C20%2C2%2C0%2C99230772
https://lists.openembedded.org/g/bitbake-devel/message/14820?p=%2C%2C%2C20%2C0%2C0%2C0%3A%3Arecentpostdate%2Fsticky%2C%2Cbelouarga%2C20%2C2%2C0%2C99248868

And this series of patches.

Thanks

[-- Attachment #2: Type: text/html, Size: 3369 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts
@ 2023-05-27  6:17 belouargamohamed
  0 siblings, 0 replies; 10+ messages in thread
From: belouargamohamed @ 2023-05-27  6:17 UTC (permalink / raw)
  To: openembedded-core; +Cc: e.aubineau, f.lahoudere, BELOUARGA Mohamed

From: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>

Npm packages do not have yocto friendly names. fore instance we can have names like
"@example/npmPackage"

npm fetcher has a function that convert these names to yocto friendly names.
But in recipe tool we have an other function (duplicate).

Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
---
 scripts/lib/recipetool/create_npm.py | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py
index 3394a89970..e667a4d19b 100644
--- a/scripts/lib/recipetool/create_npm.py
+++ b/scripts/lib/recipetool/create_npm.py
@@ -13,6 +13,7 @@ import sys
 import tempfile
 import bb
 from bb.fetch2.npm import NpmEnvironment
+from bb.fetch2.npm import npm_package
 from bb.fetch2.npmsw import foreach_dependencies
 from recipetool.create import RecipeHandler
 from recipetool.create import get_license_md5sums
@@ -30,15 +31,6 @@ def tinfoil_init(instance):
 class NpmRecipeHandler(RecipeHandler):
     """Class to handle the npm recipe creation"""
 
-    @staticmethod
-    def _npm_name(name):
-        """Generate a Yocto friendly npm name"""
-        name = re.sub("/", "-", name)
-        name = name.lower()
-        name = re.sub(r"[^\-a-z0-9]", "", name)
-        name = name.strip("-")
-        return name
-
     @staticmethod
     def _get_registry(lines):
         """Get the registry value from the 'npm://registry' url"""
@@ -143,7 +135,7 @@ class NpmRecipeHandler(RecipeHandler):
 
         # Handle the dependencies
         def _handle_dependency(name, params, deptree):
-            suffix = "-".join([self._npm_name(dep) for dep in deptree])
+            suffix = "-".join([npm_package(dep) for dep in deptree])
             destdirs = [os.path.join("node_modules", dep) for dep in deptree]
             destdir = os.path.join(*destdirs)
             packages["${PN}-" + suffix] = destdir
@@ -173,7 +165,7 @@ class NpmRecipeHandler(RecipeHandler):
         if "name" not in data or "version" not in data:
             return False
 
-        extravalues["PN"] = self._npm_name(data["name"])
+        extravalues["PN"] = npm_package(data["name"])
         extravalues["PV"] = data["version"]
 
         if "description" in data:
-- 
2.25.1



^ permalink raw reply related	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2023-05-31 20:27 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-05-27  6:25 [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts belouargamohamed
2023-05-27  6:25 ` [kirkstone][PATCH 2/5] classes: npm: Handle peer dependencies for npm packages belouargamohamed
2023-05-27  6:25 ` [kirkstone][PATCH 3/5] recipetool: create: npm: Add support for the new format of the shrinkwrap file belouargamohamed
2023-05-27  6:25 ` [kirkstone][PATCH 4/5] recipetool: create: npm: Add support to handle peer dependencies belouargamohamed
2023-05-27  6:25 ` [kirkstone][PATCH 5/5] classes: npm: Add support for the new format of the shrinkwrap file belouargamohamed
2023-05-29 15:13 ` [OE-core] [kirkstone][PATCH 1/5] recipetool: create: npm: Remove duplicate function to not have future conflicts Steve Sakoman
2023-05-29 20:16   ` belouargamohamed
     [not found]     ` <CAOSpxda3AWOQfskRpCWB0zQMDHs+SMKz0RGqE5z2Nd80DRKp7g@mail.gmail.com>
2023-05-29 22:59       ` Fwd: [OE-core] " Steve Sakoman
2023-05-31 20:27         ` belouargamohamed
  -- strict thread matches above, loose matches on Subject: below --
2023-05-27  6:17 belouargamohamed

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.