Broken NPM builds

Broken NPM builds

[Davis, Michael]

[-- Attachment #1: Type: text/plain, Size: 4894 bytes --]

I have a simple grunt-cli recipe using inherit npm.
On master branch it appears to be broken and unable to find the registry.  Same recipe on Morty branch works fine.
I looked through the mailing list and didn't see anyone else seeing this issue.

On a related side note I have recipes working for node 6.10.2.  Is there a reason we are staying on the 4.x series now that 6 is LTS?

Log data follows:
| DEBUG: Executing shell function do_install
| npm ERR! Linux 3.10.0-514.10.2.el7.x86_64
| npm ERR! argv "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/recipe-sysroot-native/usr/bin/node" "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/recipe-sysroot-native/usr/bin/npm" "install" "--prefix" "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/image/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/recipe-sysroot-native/usr" "-g" "--arch=x64" "--target_arch=x64" "--production" "--no-registry"
| npm ERR! node v4.7.3
| npm ERR! npm  v2.15.11
|
| npm ERR! Registry not defined and registry files not found: "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/npm_cache/noregistry/findup-sync/.cache.json", "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/npm_cache/findup-sync/.cache.json".
| npm ERR!
| npm ERR! If you need help, you may report this error at:
| npm ERR!     <https://github.com/npm/npm/issues>
| npm ERR! Linux 3.10.0-514.10.2.el7.x86_64
| npm ERR! argv "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/recipe-sysroot-native/usr/bin/node" "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/recipe-sysroot-native/usr/bin/npm" "install" "--prefix" "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/image/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/recipe-sysroot-native/usr" "-g" "--arch=x64" "--target_arch=x64" "--production" "--no-registry"
| npm ERR! node v4.7.3
| npm ERR! npm  v2.15.11
|
| npm ERR! Registry not defined and registry files not found: "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/npm_cache/noregistry/grunt-known-options/.cache.json", "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/npm_cache/grunt-known-options/.cache.json".
| npm ERR!
| npm ERR! If you need help, you may report this error at:
| npm ERR!     <https://github.com/npm/npm/issues>
| npm ERR! Linux 3.10.0-514.10.2.el7.x86_64
| npm ERR! argv "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/recipe-sysroot-native/usr/bin/node" "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/recipe-sysroot-native/usr/bin/npm" "install" "--prefix" "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/image/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/recipe-sysroot-native/usr" "-g" "--arch=x64" "--target_arch=x64" "--production" "--no-registry"
| npm ERR! node v4.7.3
| npm ERR! npm  v2.15.11
|
| npm ERR! Registry not defined and registry files not found: "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/npm_cache/noregistry/nopt/.cache.json", "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/npm_cache/nopt/.cache.json".
| npm ERR!
| npm ERR! If you need help, you may report this error at:
| npm ERR!     <https://github.com/npm/npm/issues>
| npm ERR! Linux 3.10.0-514.10.2.el7.x86_64
| npm ERR! argv "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/recipe-sysroot-native/usr/bin/node" "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/recipe-sysroot-native/usr/bin/npm" "install" "--prefix" "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/image/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/recipe-sysroot-native/usr" "-g" "--arch=x64" "--target_arch=x64" "--production" "--no-registry"
| npm ERR! node v4.7.3
| npm ERR! npm  v2.15.11
|
| npm ERR! Registry not defined and registry files not found: "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/npm_cache/noregistry/resolve/.cache.json", "/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/npm_cache/resolve/.cache.json".
| npm ERR!
| npm ERR! If you need help, you may report this error at:
| npm ERR!     <https://github.com/npm/npm/issues>
|
| npm ERR! Please include the following file with any support request:
| npm ERR!     /build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/npmpkg/npm-debug.log
| WARNING: /build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/temp/run.do_install.88695:1 exit 1 from 'npm install --prefix /build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/image/build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/recipe-sysroot-native/usr -g --arch=x64 --target_arch=x64 --production --no-registry'
| ERROR: Function failed: do_install (log file is located at /build/test/tmp/work/x86_64-linux/grunt-cli-native/1.2.0-r0/temp/log.do_install.88695)





[-- Attachment #2: Type: text/html, Size: 9113 bytes --]

Re: Broken NPM builds

[Trevor Woerner]

On Wed 2017-04-26 @ 04:01:20 PM, Davis, Michael wrote:
> I have a simple grunt-cli recipe using inherit npm.
> On master branch it appears to be broken and unable to find the registry.  Same recipe on Morty branch works fine.
> I looked through the mailing list and didn't see anyone else seeing this issue.

There's been quite a bit of churn on master this cycle, it's not too
surprising master is struggling. Any chance you can share your recipe; or a
stripped-down, "example.com" version of your recipe so others can take a look?

> On a related side note I have recipes working for node 6.10.2.

You have a grunt-cli recipe working on master with node 6%?

> Is there a reason we are staying on the 4.x series now that 6 is LTS?

I'm guessing nobody got around to noticing the switch. I guess we could
cherry-pick node-6% from
https://layers.openembedded.org/layerindex/branch/master/layer/meta-nodejs/

Ideally we'd either merge all/most of
https://layers.openembedded.org/layerindex/branch/master/layer/meta-nodejs/
into oecore, or remove what's in oecore and tell people to use the external
layer. Why does oe-core have some, but not all, node support?

Re: Broken NPM builds

[Davis, Michael]

My recipe was just an autogenerated recipe from devtool.  
I pull this grunt recipe out of the mailing list as one that worked previously and it has the same issue.

SUMMARY = "Grunt.js command line wrapper"
HOMEPAGE = "http://gruntjs.com/"
SECTION = "js/devel"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://README.md;md5=a24cc788ab726e615aa5bf0168c5d77d"
SRC_URI = "npm://registry.npmjs.org;name=${PN};version=${PV}"                                                                                                                  
inherit npm

The 6% recipe has the same problem on master, but works fine in morty.  I can post it as a patch after I can fully test it again.

The external meta-nodejs layer is not compatible with the npm.bbclass fetcher.  I have a patch in to fix that, but I am waiting for it to be integrated.


-----Original Message-----
From: Trevor Woerner [mailto:twoerner@gmail.com] 
Sent: Wednesday, April 26, 2017 2:28 PM
To: Davis, Michael
Cc: openembedded-core@lists.openembedded.org; ilkka.myller@nodefield.com
Subject: Re: [OE-core] Broken NPM builds

On Wed 2017-04-26 @ 04:01:20 PM, Davis, Michael wrote:
> I have a simple grunt-cli recipe using inherit npm.
> On master branch it appears to be broken and unable to find the registry.  Same recipe on Morty branch works fine.
> I looked through the mailing list and didn't see anyone else seeing this issue.

There's been quite a bit of churn on master this cycle, it's not too
surprising master is struggling. Any chance you can share your recipe; or a
stripped-down, "example.com" version of your recipe so others can take a look?

> On a related side note I have recipes working for node 6.10.2.

You have a grunt-cli recipe working on master with node 6%?

> Is there a reason we are staying on the 4.x series now that 6 is LTS?

I'm guessing nobody got around to noticing the switch. I guess we could
cherry-pick node-6% from
https://layers.openembedded.org/layerindex/branch/master/layer/meta-nodejs/

Ideally we'd either merge all/most of
https://layers.openembedded.org/layerindex/branch/master/layer/meta-nodejs/
into oecore, or remove what's in oecore and tell people to use the external
layer. Why does oe-core have some, but not all, node support?

Re: Broken NPM builds

[Ilkka Myller]

[-- Attachment #1: Type: text/plain, Size: 1479 bytes --]

I’m the maintainer of meta-nodejs layer. The layer offers latest Node.js versions (current, LTS and end-of-life). As a member of Node.js core collaborator team, I’ve picked up the task of maintaining up-to-date Node.js offering for OE. 

In addition to Node.js core packages the layer offers oe_runnpm command (from npm-base class) that allows running 100% standard npm with added capability to safely cross-compile native components on build host.
This command is analogous to oe_runmake from base class in OE core.

This design allows building static, immutable, OE-proper packages for Node.js based applications while still using standard npm installation process on the build host. This is where OE core npm class differs the most by-design.

meta-nodejs has chosen not to promote non-standard use of npm utility, npm repository or npm installation process. 

However, as Michael Davis pointed out we can merge his patch allowing co-existence of both methods to meta-nodejs layer. I expect this to happen within few days if all goes well.

> Trevor Woerner <twoerner@gmail.com> kirjoitti 26.4.2017 kello 22.28:
> 
> Ideally we'd either merge all/most of
> https://layers.openembedded.org/layerindex/branch/master/layer/meta-nodejs/ <https://layers.openembedded.org/layerindex/branch/master/layer/meta-nodejs/>
> into oecore, or remove what's in oecore and tell people to use the external
> layer. Why does oe-core have some, but not all, node support?


[-- Attachment #2: Type: text/html, Size: 4353 bytes --]

Re: Broken NPM builds

[Alexander Kanavin]

On 04/26/2017 11:31 PM, Ilkka Myller wrote:
> This design allows building static, immutable, OE-proper packages for
> Node.js based applications while still using standard npm installation
> process on the build host. This is where OE core npm class differs the
> most by-design.

But are they reproducible? If the app is built one week later, is there 
a guarantee that it will pull in the same components, and a check that 
it is indeed so?

Also, are the licenses of various dependencies tracked and enforced?

> meta-nodejs has chosen not to promote non-standard use of npm utility,
> npm repository or npm installation process.

With this, I fully agree. I can only see pain and suffering with the 
current oe-core approach.


Alex

Re: Broken NPM builds

[Alexander Kanavin]

On 04/26/2017 11:31 PM, Ilkka Myller wrote:
> This design allows building static, immutable, OE-proper packages for
> Node.js based applications while still using standard npm installation
> process on the build host. This is where OE core npm class differs the
> most by-design.

But are they reproducible? If the app is built one week later, is there 
a guarantee that it will pull in the same components, and a check that 
it is indeed so?

Also, are the licenses of various dependencies tracked and enforced?

These are important concerns in the embedded world (as opposed to the 
cloud world where node.js originated).

> meta-nodejs has chosen not to promote non-standard use of npm utility,
> npm repository or npm installation process.

With this, I fully agree. I can only see pain and suffering with the 
current oe-core approach.


Alex