Re: [Openembedded-architecture] Proposal: dealing with language-specific build tools/dependency management tools

[Trevor Woerner <twoerner@gmail.com>]

Hi Alexander,

Thanks for bringing up this important topic. There is no doubt we're seeing
paradigm shifts in the way applications are written, built, and packaged;
as well as a complete lack of interest in licensing.

Although the trend is to not care about licensing, I believe it is vitally
important that we do our best to keep track of all the licensing from every
package that is pulled into an image. If we're pulling in >1000 npm packages
just for one node app, then that means we should have >1000 item list of each
dependency and their respective licenses. Although it makes a recipe look
ugly, I wouldn't want to drop this functionality due to aesthetic concerns.
Maybe the license list could be moved to another file that is required by the
"main" recipe file? Maybe the list could be moved to the bottom of the file?

In the case of node specifically, I don't think trying to create and maintain
separate recipes for each and every dependency one might find in the npm
registry would be a sane approach. Currently we embed the version info into
the recipe filename. This will simply not scale to millions of npm packages,
each with numerous versions.

I've been playing with node a fair amount lately as it relates to OE and I
have to say I've been quite impressed! These aren't easy things and I think
there's a lot of good work happening.

I've outlined some of my thoughts on my experiences[1]:
http://lists.openembedded.org/pipermail/openembedded-core/2017-February/133432.html

Other than these (short-term?) issues devtool seems to be on the right
track (?) It does, for example, generate a lockdown.json file and an
npm-shrinkwrap.json file automatically. All we need is the package.json from
the app developer, and that can be auto-generated via npm. I think we have to
accept that node developers are going to want to develop on the target device
itself, and when they're done they can hand us the package.json file which we
can run devtool on which will generate the recipe for us.

As a short-term work-around, I've simply been creating an image with node+npm,
running it on the device, copying over the package.json file, running npm
install against it, then collecting up all the extra stuff that gets added
to my image (as a result), and bundling all that into a platform-specific
"bin_package" (bbclass). It works, but it's a multi-step process. If I could
cut out some of those steps (once things from [1] are fixed), it would be an
improvement.

Best regards,
	Trevor







[1] A short recap of those emails:

	Different paths seem to be followed depending on whether you point devtool at,
	say, a github repository versus a local checkout of the same project. That
	seems wrong.

	Also (as you've pointed out) RSS is messing all this up on master at the
	moment; but I assume this can/will get fixed? Things work fine on morty.

	Also, devtool gets tripped up when it encounters a license string that isn't
	found in its list of already-known license strings. This approach seems doomed
	to failure. It has to be able to recover gracefully and continue walking the
	dependency list without having to continuously add corner cases to the code.