Dunfell, nodejs and typescript

* Dunfell, nodejs and typescript - short experience report
@ 2021-02-12 10:02 Simon Vogl
  2021-02-12 10:12 ` [yocto] " Josef Holzmayr
  0 siblings, 1 reply; 6+ messages in thread
From: Simon Vogl @ 2021-02-12 10:02 UTC (permalink / raw)
  To: yocto

[-- Attachment #1: Type: text/plain, Size: 3507 bytes --]

I have some remarks and questions about the npm/nodejs support in 
dunfell that I wanted to share. We are creating nodejs-based IoT edge 
solutions and upgrading our build environments to Dunfell one by one. In 
the course of this, we are switching to the new npm-implementation and 
found a few small issues.

Firstly, the do_configure() task takes quite some time to complete. 
After a quick analysis, I saw that most of the time is being spent in 
creating the npmrc files while packing the dependent packages. I wrote a 
small workaround to directly create the file instead of calling 'npm 
config', which results in a 3x-4x speedup:

Signed-off-by: Simon Vogl<simon@voxel.at>
---
  lib/bb/fetch2/npm.py | 9 +++++++--
  1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/lib/bb/fetch2/npm.py b/lib/bb/fetch2/npm.py
index 4789850..2720d87 100644
--- a/lib/bb/fetch2/npm.py
+++ b/lib/bb/fetch2/npm.py
@@ -97,13 +97,18 @@ class NpmEnvironment(object):
                  cmd = "NPM_CONFIG_GLOBALCONFIG=%s " % cfgfile + cmd
                  return runfetchcmd(cmd, d, workdir=workdir)
  
+            cfg = open(cfgfile, "a")
              if self.configs:
                  for key, value in self.configs:
-                    _run("npm config set %s %s" % (key, shlex.quote(value)))
+                    cfg.write("%s=%s\n" % (key, shlex.quote(value)))
+                    #_run("npm config set %s %s" % (key, shlex.quote(value)))
  
              if configs:
                  for key, value in configs:
-                    _run("npm config set %s %s" % (key, shlex.quote(value)))
+                    cfg.write("%s=%s\n" % (key, shlex.quote(value)))
+                    # _run("npm config set %s %s" % (key, shlex.quote(value)))
+
+            cfg.close()
  
              if args:
                  for key, value in args:
-- 2.7.4

Are there any side effects that I did not stumble over yet? And I'd LOVE 
to have these calls running in a thread-pool for better performance...


Secondly, our projects are based on typescript, so a native compile step 
is necessary to create a compiled version for packing. We experimented 
with a separate release branch to check in compiled versions, but this 
is not easy to handle. I played around with npm.bbclass and found a way 
to extend configure (!) with a call to our build script before packaging:

diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index 068032a1e5..31535098cf 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -170,6 +170,11 @@ python npm_do_configure() {
  
      # Configure the main package
      with tempfile.TemporaryDirectory() as tmpdir:
+        # install all (native) build dependencies, overrides npm cache:
+        ret = os.system("npm i")
+        # run build step:
+        env.run("npm run build", args=[], workdir=d.getVar("S"))
+
          tarball = npm_pack(env, d.getVar("S"), tmpdir)
          npm_unpack(tarball, d.getVar("NPM_PACKAGE"), d)

As we have plain JS packages as well, I put the modified configure() in 
a subclass and this works for us, but it does not look like a clean 
solution to me. How do you other IoT'ers address this situation?

Simon


-- 
VoXel Interaction Design  |  www.voxel.at
DI Dr.techn. Simon Vogl   |  simon@voxel.at
Tomaschekweg 46           |  +43 650 2323 555
A-4040 Linz - Austria     |
Office address: Industriezeile 35, 4020 Linz (2nd floor)


[-- Attachment #2: Type: text/html, Size: 4353 bytes --]

^ permalink raw reply related	[flat|nested] 6+ messages in thread