* [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments
@ 2021-10-08 7:42 Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 02/12] recipetool: Move license md5sums into CSV files Stefan Herbrechtsmeier
` (10 more replies)
0 siblings, 11 replies; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-08 7:42 UTC (permalink / raw)
To: openembedded-core; +Cc: Stefan Herbrechtsmeier
From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Add support for EXTRA_OENPM arguments to set node-gyp variables for
example. This allows use of shared librariess, avoid download
of external sources or build from source.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
---
(no changes since v1)
meta/classes/npm.bbclass | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index 91a2321116..477b40d921 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -22,6 +22,8 @@ inherit python3native
DEPENDS:prepend = "nodejs-native "
RDEPENDS:${PN}:append:class-target = " nodejs"
+EXTRA_OENPM = ""
+
NPM_INSTALL_DEV ?= "0"
def npm_target_arch_map(target_arch):
@@ -260,7 +262,8 @@ python npm_do_compile() {
# Pack and install the main package
tarball = npm_pack(env, d.getVar("NPM_PACKAGE"), tmpdir)
- env.run("npm install %s" % shlex.quote(tarball), args=args, configs=configs)
+ cmd = "npm install %s %s" % (shlex.quote(tarball), d.getVar("EXTRA_OENPM"))
+ env.run(cmd, args=args, configs=configs)
}
npm_do_install() {
--
2.20.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 02/12] recipetool: Move license md5sums into CSV files
2021-10-08 7:42 [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments Stefan Herbrechtsmeier
@ 2021-10-08 7:42 ` Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 03/12] recipetool: Skip common source files in guess_license Stefan Herbrechtsmeier
` (9 subsequent siblings)
10 siblings, 0 replies; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-08 7:42 UTC (permalink / raw)
To: openembedded-core; +Cc: Stefan Herbrechtsmeier
From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
---
(no changes since v1)
scripts/lib/recipetool/create.py | 52 +++++++++--------------------
scripts/lib/recipetool/licenses.csv | 37 ++++++++++++++++++++
2 files changed, 52 insertions(+), 37 deletions(-)
create mode 100644 scripts/lib/recipetool/licenses.csv
diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index 5cd72ea0a7..e88a4253da 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -1004,6 +1004,7 @@ def handle_license_vars(srctree, lines_before, handled, extravalues, d):
def get_license_md5sums(d, static_only=False):
import bb.utils
+ import csv
md5sums = {}
if not static_only:
# Gather md5sums of license files in common license dir
@@ -1011,46 +1012,23 @@ def get_license_md5sums(d, static_only=False):
for fn in os.listdir(commonlicdir):
md5value = bb.utils.md5_file(os.path.join(commonlicdir, fn))
md5sums[md5value] = fn
+
# The following were extracted from common values in various recipes
# (double checking the license against the license file itself, not just
# the LICENSE value in the recipe)
- md5sums['94d55d512a9ba36caa9b7df079bae19f'] = 'GPLv2'
- md5sums['b234ee4d69f5fce4486a80fdaf4a4263'] = 'GPLv2'
- md5sums['59530bdf33659b29e73d4adb9f9f6552'] = 'GPLv2'
- md5sums['0636e73ff0215e8d672dc4c32c317bb3'] = 'GPLv2'
- md5sums['eb723b61539feef013de476e68b5c50a'] = 'GPLv2'
- md5sums['751419260aa954499f7abaabaa882bbe'] = 'GPLv2'
- md5sums['393a5ca445f6965873eca0259a17f833'] = 'GPLv2'
- md5sums['12f884d2ae1ff87c09e5b7ccc2c4ca7e'] = 'GPLv2'
- md5sums['8ca43cbc842c2336e835926c2166c28b'] = 'GPLv2'
- md5sums['ebb5c50ab7cab4baeffba14977030c07'] = 'GPLv2'
- md5sums['c93c0550bd3173f4504b2cbd8991e50b'] = 'GPLv2'
- md5sums['9ac2e7cff1ddaf48b6eab6028f23ef88'] = 'GPLv2'
- md5sums['4325afd396febcb659c36b49533135d4'] = 'GPLv2'
- md5sums['18810669f13b87348459e611d31ab760'] = 'GPLv2'
- md5sums['d7810fab7487fb0aad327b76f1be7cd7'] = 'GPLv2' # the Linux kernel's COPYING file
- md5sums['bbb461211a33b134d42ed5ee802b37ff'] = 'LGPLv2.1'
- md5sums['7fbc338309ac38fefcd64b04bb903e34'] = 'LGPLv2.1'
- md5sums['4fbd65380cdd255951079008b364516c'] = 'LGPLv2.1'
- md5sums['2d5025d4aa3495befef8f17206a5b0a1'] = 'LGPLv2.1'
- md5sums['fbc093901857fcd118f065f900982c24'] = 'LGPLv2.1'
- md5sums['a6f89e2100d9b6cdffcea4f398e37343'] = 'LGPLv2.1'
- md5sums['d8045f3b8f929c1cb29a1e3fd737b499'] = 'LGPLv2.1'
- md5sums['fad9b3332be894bab9bc501572864b29'] = 'LGPLv2.1'
- md5sums['3bf50002aefd002f49e7bb854063f7e7'] = 'LGPLv2'
- md5sums['9f604d8a4f8e74f4f5140845a21b6674'] = 'LGPLv2'
- md5sums['5f30f0716dfdd0d91eb439ebec522ec2'] = 'LGPLv2'
- md5sums['55ca817ccb7d5b5b66355690e9abc605'] = 'LGPLv2'
- md5sums['252890d9eee26aab7b432e8b8a616475'] = 'LGPLv2'
- md5sums['3214f080875748938ba060314b4f727d'] = 'LGPLv2'
- md5sums['db979804f025cf55aabec7129cb671ed'] = 'LGPLv2'
- md5sums['d32239bcb673463ab874e80d47fae504'] = 'GPLv3'
- md5sums['f27defe1e96c2e1ecd4e0c9be8967949'] = 'GPLv3'
- md5sums['6a6a8e020838b23406c81b19c1d46df6'] = 'LGPLv3'
- md5sums['3b83ef96387f14655fc854ddc3c6bd57'] = 'Apache-2.0'
- md5sums['385c55653886acac3821999a3ccd17b3'] = 'Artistic-1.0 | GPL-2.0' # some perl modules
- md5sums['54c7042be62e169199200bc6477f04d1'] = 'BSD-3-Clause'
- md5sums['bfe1f75d606912a4111c90743d6c7325'] = 'MPL-1.1'
+
+ # Read license md5sums from csv file
+ scripts_path = os.path.dirname(os.path.realpath(__file__))
+ for path in (d.getVar('BBPATH').split(':')
+ + [os.path.join(scripts_path, '..', '..')]):
+ csv_path = os.path.join(path, 'lib', 'recipetool', 'licenses.csv')
+ if os.path.isfile(csv_path):
+ with open(csv_path, newline='') as csv_file:
+ fieldnames = ['md5sum', 'license']
+ reader = csv.DictReader(csv_file, delimiter=',', fieldnames=fieldnames)
+ for row in reader:
+ md5sums[row['md5sum']] = row['license']
+
return md5sums
def crunch_license(licfile):
diff --git a/scripts/lib/recipetool/licenses.csv b/scripts/lib/recipetool/licenses.csv
new file mode 100644
index 0000000000..0d3fb0607b
--- /dev/null
+++ b/scripts/lib/recipetool/licenses.csv
@@ -0,0 +1,37 @@
+0636e73ff0215e8d672dc4c32c317bb3,GPLv2
+12f884d2ae1ff87c09e5b7ccc2c4ca7e,GPLv2
+18810669f13b87348459e611d31ab760,GPLv2
+252890d9eee26aab7b432e8b8a616475,LGPLv2
+2d5025d4aa3495befef8f17206a5b0a1,LGPLv2.1
+3214f080875748938ba060314b4f727d,LGPLv2
+385c55653886acac3821999a3ccd17b3,Artistic-1.0 | GPL-2.0
+393a5ca445f6965873eca0259a17f833,GPLv2
+3b83ef96387f14655fc854ddc3c6bd57,Apache-2.0
+3bf50002aefd002f49e7bb854063f7e7,LGPLv2
+4325afd396febcb659c36b49533135d4,GPLv2
+4fbd65380cdd255951079008b364516c,LGPLv2.1
+54c7042be62e169199200bc6477f04d1,BSD-3-Clause
+55ca817ccb7d5b5b66355690e9abc605,LGPLv2
+59530bdf33659b29e73d4adb9f9f6552,GPLv2
+5f30f0716dfdd0d91eb439ebec522ec2,LGPLv2
+6a6a8e020838b23406c81b19c1d46df6,LGPLv3
+751419260aa954499f7abaabaa882bbe,GPLv2
+7fbc338309ac38fefcd64b04bb903e34,LGPLv2.1
+8ca43cbc842c2336e835926c2166c28b,GPLv2
+94d55d512a9ba36caa9b7df079bae19f,GPLv2
+9ac2e7cff1ddaf48b6eab6028f23ef88,GPLv2
+9f604d8a4f8e74f4f5140845a21b6674,LGPLv2
+a6f89e2100d9b6cdffcea4f398e37343,LGPLv2.1
+b234ee4d69f5fce4486a80fdaf4a4263,GPLv2
+bbb461211a33b134d42ed5ee802b37ff,LGPLv2.1
+bfe1f75d606912a4111c90743d6c7325,MPL-1.1
+c93c0550bd3173f4504b2cbd8991e50b,GPLv2
+d32239bcb673463ab874e80d47fae504,GPLv3
+d7810fab7487fb0aad327b76f1be7cd7,GPLv2
+d8045f3b8f929c1cb29a1e3fd737b499,LGPLv2.1
+db979804f025cf55aabec7129cb671ed,LGPLv2
+eb723b61539feef013de476e68b5c50a,GPLv2
+ebb5c50ab7cab4baeffba14977030c07,GPLv2
+f27defe1e96c2e1ecd4e0c9be8967949,GPLv3
+fad9b3332be894bab9bc501572864b29,LGPLv2.1
+fbc093901857fcd118f065f900982c24,LGPLv2.1
--
2.20.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 03/12] recipetool: Skip common source files in guess_license
2021-10-08 7:42 [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 02/12] recipetool: Move license md5sums into CSV files Stefan Herbrechtsmeier
@ 2021-10-08 7:42 ` Stefan Herbrechtsmeier
2021-10-08 7:45 ` [OE-core] " Konrad Weihmann
2021-10-08 7:42 ` [PATCH 04/12] recipetool: Change default paramter fallback_licenses of function split_pkg_licenses from None to [] Stefan Herbrechtsmeier
` (8 subsequent siblings)
10 siblings, 1 reply; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-08 7:42 UTC (permalink / raw)
To: openembedded-core; +Cc: Stefan Herbrechtsmeier
From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
---
(no changes since v1)
scripts/lib/recipetool/create.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index e88a4253da..83cf25d9b7 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -1114,6 +1114,8 @@ def guess_license(srctree, d):
licfiles = []
for root, dirs, files in os.walk(srctree):
for fn in files:
+ if fn.endswith(".html") or fn.endswith(".js") or fn.endswith(".json") or fn.endswith(".svg") or fn.endswith(".ts"):
+ continue
for spec in licspecs:
if fnmatch.fnmatch(fn, spec):
fullpath = os.path.join(root, fn)
--
2.20.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 04/12] recipetool: Change default paramter fallback_licenses of function split_pkg_licenses from None to []
2021-10-08 7:42 [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 02/12] recipetool: Move license md5sums into CSV files Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 03/12] recipetool: Skip common source files in guess_license Stefan Herbrechtsmeier
@ 2021-10-08 7:42 ` Stefan Herbrechtsmeier
2022-04-13 21:35 ` [OE-core] " Paul Eggleton
2021-10-08 7:42 ` [PATCH 05/12] recipetool: ignore empty license files Stefan Herbrechtsmeier
` (7 subsequent siblings)
10 siblings, 1 reply; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-08 7:42 UTC (permalink / raw)
To: openembedded-core; +Cc: Stefan Herbrechtsmeier
From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
---
(no changes since v1)
scripts/lib/recipetool/create.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index 83cf25d9b7..893980a5ce 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -1134,7 +1134,7 @@ def guess_license(srctree, d):
return licenses
-def split_pkg_licenses(licvalues, packages, outlines, fallback_licenses=None, pn='${PN}'):
+def split_pkg_licenses(licvalues, packages, outlines, fallback_licenses=[], pn='${PN}'):
"""
Given a list of (license, path, md5sum) as returned by guess_license(),
a dict of package name to path mappings, write out a set of
--
2.20.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 05/12] recipetool: ignore empty license files
2021-10-08 7:42 [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments Stefan Herbrechtsmeier
` (2 preceding siblings ...)
2021-10-08 7:42 ` [PATCH 04/12] recipetool: Change default paramter fallback_licenses of function split_pkg_licenses from None to [] Stefan Herbrechtsmeier
@ 2021-10-08 7:42 ` Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 06/12] recipetool: Add logger info for missing license entries Stefan Herbrechtsmeier
` (6 subsequent siblings)
10 siblings, 0 replies; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-08 7:42 UTC (permalink / raw)
To: openembedded-core; +Cc: Stefan Herbrechtsmeier
From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
---
(no changes since v1)
scripts/lib/recipetool/create.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index 893980a5ce..87b25ebc7e 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -1126,9 +1126,10 @@ def guess_license(srctree, d):
license = md5sums.get(md5value, None)
if not license:
license, crunched_md5, lictext = crunch_license(licfile)
- if not license:
+ if lictext and not license:
license = 'Unknown'
- licenses.append((license, os.path.relpath(licfile, srctree), md5value))
+ if license:
+ licenses.append((license, os.path.relpath(licfile, srctree), md5value))
# FIXME should we grab at least one source file with a license header and add that too?
--
2.20.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 06/12] recipetool: Add logger info for missing license entries
2021-10-08 7:42 [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments Stefan Herbrechtsmeier
` (3 preceding siblings ...)
2021-10-08 7:42 ` [PATCH 05/12] recipetool: ignore empty license files Stefan Herbrechtsmeier
@ 2021-10-08 7:42 ` Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 07/12] recipetool: Add support for linenumbers to licenses.csv Stefan Herbrechtsmeier
` (5 subsequent siblings)
10 siblings, 0 replies; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-08 7:42 UTC (permalink / raw)
To: openembedded-core; +Cc: Stefan Herbrechtsmeier
From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
---
(no changes since v1)
scripts/lib/recipetool/create.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index 87b25ebc7e..a8c4cdef4a 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -1128,6 +1128,9 @@ def guess_license(srctree, d):
license, crunched_md5, lictext = crunch_license(licfile)
if lictext and not license:
license = 'Unknown'
+ logger.info("Please add the following line for '%s' to a 'lib/recipetool/licenses.csv' " \
+ "and replace `Unknown` with the license:\n" \
+ "%s,Unknown" % (os.path.relpath(licfile, srctree), md5value))
if license:
licenses.append((license, os.path.relpath(licfile, srctree), md5value))
--
2.20.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 07/12] recipetool: Add support for linenumbers to licenses.csv
2021-10-08 7:42 [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments Stefan Herbrechtsmeier
` (4 preceding siblings ...)
2021-10-08 7:42 ` [PATCH 06/12] recipetool: Add logger info for missing license entries Stefan Herbrechtsmeier
@ 2021-10-08 7:42 ` Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 08/12] recipetool: npm: Do not add package.json files to LIC_FILES_CHKSUM Stefan Herbrechtsmeier
` (4 subsequent siblings)
10 siblings, 0 replies; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-08 7:42 UTC (permalink / raw)
To: openembedded-core; +Cc: Stefan Herbrechtsmeier
From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Add support for linenumbers (begin and end lines) to licenses.csv. Add
an optional linenumbers parameter to get_license_md5sums to support
different use cases.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
---
(no changes since v1)
scripts/lib/recipetool/create.py | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index a8c4cdef4a..277266be4e 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -1002,11 +1002,11 @@ def handle_license_vars(srctree, lines_before, handled, extravalues, d):
handled.append(('license', licvalues))
return licvalues
-def get_license_md5sums(d, static_only=False):
+def get_license_md5sums(d, static_only=False, linenumbers=False):
import bb.utils
import csv
md5sums = {}
- if not static_only:
+ if not static_only and not linenumbers:
# Gather md5sums of license files in common license dir
commonlicdir = d.getVar('COMMON_LICENSE_DIR')
for fn in os.listdir(commonlicdir):
@@ -1024,10 +1024,14 @@ def get_license_md5sums(d, static_only=False):
csv_path = os.path.join(path, 'lib', 'recipetool', 'licenses.csv')
if os.path.isfile(csv_path):
with open(csv_path, newline='') as csv_file:
- fieldnames = ['md5sum', 'license']
+ fieldnames = ['md5sum', 'license', 'beginline', 'endline', 'md5']
reader = csv.DictReader(csv_file, delimiter=',', fieldnames=fieldnames)
for row in reader:
- md5sums[row['md5sum']] = row['license']
+ if linenumbers:
+ md5sums[row['md5sum']] = (
+ row['license'], row['beginline'], row['endline'], row['md5'])
+ else:
+ md5sums[row['md5sum']] = row['license']
return md5sums
--
2.20.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 08/12] recipetool: npm: Do not add package.json files to LIC_FILES_CHKSUM
2021-10-08 7:42 [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments Stefan Herbrechtsmeier
` (5 preceding siblings ...)
2021-10-08 7:42 ` [PATCH 07/12] recipetool: Add support for linenumbers to licenses.csv Stefan Herbrechtsmeier
@ 2021-10-08 7:42 ` Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 09/12] recipetool: npm: Use README as license fallback Stefan Herbrechtsmeier
` (3 subsequent siblings)
10 siblings, 0 replies; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-08 7:42 UTC (permalink / raw)
To: openembedded-core; +Cc: Stefan Herbrechtsmeier
From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
The package.json files doesn't contain any licenses. The name of the
license doesn't comply the license requirements of most liceneses.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
---
(no changes since v1)
scripts/lib/recipetool/create_npm.py | 10 ----------
1 file changed, 10 deletions(-)
diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py
index 2bcae91dfa..c939780931 100644
--- a/scripts/lib/recipetool/create_npm.py
+++ b/scripts/lib/recipetool/create_npm.py
@@ -115,17 +115,9 @@ class NpmRecipeHandler(RecipeHandler):
def _handle_licenses(self, srctree, shrinkwrap_file, dev):
"""Return the extra license files and the list of packages"""
- licfiles = []
packages = {}
- def _licfiles_append(licfile):
- """Append 'licfile' to the license files list"""
- licfilepath = os.path.join(srctree, licfile)
- licmd5 = bb.utils.md5_file(licfilepath)
- licfiles.append("file://%s;md5=%s" % (licfile, licmd5))
-
# Handle the parent package
- _licfiles_append("package.json")
packages["${PN}"] = ""
# Handle the dependencies
@@ -133,7 +125,6 @@ class NpmRecipeHandler(RecipeHandler):
suffix = "-".join([self._npm_name(dep) for dep in deptree])
destdirs = [os.path.join("node_modules", dep) for dep in deptree]
destdir = os.path.join(*destdirs)
- _licfiles_append(os.path.join(destdir, "package.json"))
packages["${PN}-" + suffix] = destdir
with open(shrinkwrap_file, "r") as f:
@@ -246,7 +237,6 @@ class NpmRecipeHandler(RecipeHandler):
bb.note("Handling licences ...")
(licfiles, packages) = self._handle_licenses(srctree, shrinkwrap_file, dev)
- extravalues["LIC_FILES_CHKSUM"] = licfiles
split_pkg_licenses(guess_license(srctree, d), packages, lines_after, [])
classes.append("npm")
--
2.20.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 09/12] recipetool: npm: Use README as license fallback
2021-10-08 7:42 [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments Stefan Herbrechtsmeier
` (6 preceding siblings ...)
2021-10-08 7:42 ` [PATCH 08/12] recipetool: npm: Do not add package.json files to LIC_FILES_CHKSUM Stefan Herbrechtsmeier
@ 2021-10-08 7:42 ` Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 10/12] npm: Add variable NPM_NODEDIR with default value Stefan Herbrechtsmeier
` (2 subsequent siblings)
10 siblings, 0 replies; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-08 7:42 UTC (permalink / raw)
To: openembedded-core; +Cc: Stefan Herbrechtsmeier
From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Use the README as license fallback if a license file is missing. Use the
linenumbers parameter of get_license_md5sums function to determine the
license text inside the README.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
---
Changes in v2:
- Rework to add licenses to package LICENSE:${PN} and LIC_FILES_CHKSUM
scripts/lib/recipetool/create_npm.py | 52 +++++++++++++++++++++++++++-
1 file changed, 51 insertions(+), 1 deletion(-)
diff --git a/scripts/lib/recipetool/create_npm.py b/scripts/lib/recipetool/create_npm.py
index c939780931..3394a89970 100644
--- a/scripts/lib/recipetool/create_npm.py
+++ b/scripts/lib/recipetool/create_npm.py
@@ -6,6 +6,7 @@
"""Recipe creation tool - npm module support plugin"""
import json
+import logging
import os
import re
import sys
@@ -14,8 +15,10 @@ import bb
from bb.fetch2.npm import NpmEnvironment
from bb.fetch2.npmsw import foreach_dependencies
from recipetool.create import RecipeHandler
+from recipetool.create import get_license_md5sums
from recipetool.create import guess_license
from recipetool.create import split_pkg_licenses
+logger = logging.getLogger('recipetool')
TINFOIL = None
@@ -115,17 +118,36 @@ class NpmRecipeHandler(RecipeHandler):
def _handle_licenses(self, srctree, shrinkwrap_file, dev):
"""Return the extra license files and the list of packages"""
+ licfiles = []
packages = {}
# Handle the parent package
packages["${PN}"] = ""
+ def _licfiles_append_fallback_readme_files(destdir):
+ """Append README files as fallback to license files if a license files is missing"""
+
+ fallback = True
+ readmes = []
+ basedir = os.path.join(srctree, destdir)
+ for fn in os.listdir(basedir):
+ upper = fn.upper()
+ if upper.startswith("README"):
+ fullpath = os.path.join(basedir, fn)
+ readmes.append(fullpath)
+ if upper.startswith("COPYING") or "LICENCE" in upper or "LICENSE" in upper:
+ fallback = False
+ if fallback:
+ for readme in readmes:
+ licfiles.append(os.path.relpath(readme, srctree))
+
# Handle the dependencies
def _handle_dependency(name, params, deptree):
suffix = "-".join([self._npm_name(dep) for dep in deptree])
destdirs = [os.path.join("node_modules", dep) for dep in deptree]
destdir = os.path.join(*destdirs)
packages["${PN}-" + suffix] = destdir
+ _licfiles_append_fallback_readme_files(destdir)
with open(shrinkwrap_file, "r") as f:
shrinkwrap = json.load(f)
@@ -237,7 +259,35 @@ class NpmRecipeHandler(RecipeHandler):
bb.note("Handling licences ...")
(licfiles, packages) = self._handle_licenses(srctree, shrinkwrap_file, dev)
- split_pkg_licenses(guess_license(srctree, d), packages, lines_after, [])
+
+ def _guess_odd_license(licfiles):
+ import bb
+
+ md5sums = get_license_md5sums(d, linenumbers=True)
+
+ chksums = []
+ licenses = []
+ for licfile in licfiles:
+ f = os.path.join(srctree, licfile)
+ md5value = bb.utils.md5_file(f)
+ (license, beginline, endline, md5) = md5sums.get(md5value,
+ (None, "", "", ""))
+ if not license:
+ license = "Unknown"
+ logger.info("Please add the following line for '%s' to a "
+ "'lib/recipetool/licenses.csv' and replace `Unknown`, "
+ "`X`, `Y` and `MD5` with the license, begin line, "
+ "end line and partial MD5 checksum:\n" \
+ "%s,Unknown,X,Y,MD5" % (licfile, md5value))
+ chksums.append("file://%s%s%s;md5=%s" % (licfile,
+ ";beginline=%s" % (beginline) if beginline else "",
+ ";endline=%s" % (endline) if endline else "",
+ md5 if md5 else md5value))
+ licenses.append((license, licfile, md5value))
+ return (licenses, chksums)
+
+ (licenses, extravalues["LIC_FILES_CHKSUM"]) = _guess_odd_license(licfiles)
+ split_pkg_licenses([*licenses, *guess_license(srctree, d)], packages, lines_after)
classes.append("npm")
handled.append("buildsystem")
--
2.20.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 10/12] npm: Add variable NPM_NODEDIR with default value
2021-10-08 7:42 [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments Stefan Herbrechtsmeier
` (7 preceding siblings ...)
2021-10-08 7:42 ` [PATCH 09/12] recipetool: npm: Use README as license fallback Stefan Herbrechtsmeier
@ 2021-10-08 7:42 ` Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 11/12] npm: Use configs for npm environment and args for npm run command Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 12/12] recipetool: Rework crunch_license to recognize more variants Stefan Herbrechtsmeier
10 siblings, 0 replies; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-08 7:42 UTC (permalink / raw)
To: openembedded-core; +Cc: Stefan Herbrechtsmeier
From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Replace the variable NPM_NODEDIR fallback to a default value inside the
code with a variable NPM_NODEDIR with default value.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
---
(no changes since v1)
meta/classes/npm.bbclass | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index 477b40d921..05bb9f578d 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -26,6 +26,8 @@ EXTRA_OENPM = ""
NPM_INSTALL_DEV ?= "0"
+NPM_NODEDIR ?= "${RECIPE_SYSROOT_NATIVE}${prefix_native}"
+
def npm_target_arch_map(target_arch):
"""Maps arch names to npm arch names"""
import re
@@ -249,11 +251,7 @@ python npm_do_compile() {
# Add node-gyp configuration
configs.append(("arch", d.getVar("NPM_ARCH")))
configs.append(("release", "true"))
- nodedir = d.getVar("NPM_NODEDIR")
- if not nodedir:
- sysroot = d.getVar("RECIPE_SYSROOT_NATIVE")
- nodedir = os.path.join(sysroot, d.getVar("prefix_native").strip("/"))
- configs.append(("nodedir", nodedir))
+ configs.append(("nodedir", d.getVar("NPM_NODEDIR")))
configs.append(("python", d.getVar("PYTHON")))
# Add node-pre-gyp configuration
--
2.20.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 11/12] npm: Use configs for npm environment and args for npm run command
2021-10-08 7:42 [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments Stefan Herbrechtsmeier
` (8 preceding siblings ...)
2021-10-08 7:42 ` [PATCH 10/12] npm: Add variable NPM_NODEDIR with default value Stefan Herbrechtsmeier
@ 2021-10-08 7:42 ` Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 12/12] recipetool: Rework crunch_license to recognize more variants Stefan Herbrechtsmeier
10 siblings, 0 replies; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-08 7:42 UTC (permalink / raw)
To: openembedded-core; +Cc: Stefan Herbrechtsmeier
From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Use parameter configs of class NpmEnvironment and parameter args of
function run to support a common npmrc for all run calls of a single
NpmEnvironment.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
---
(no changes since v1)
meta/classes/npm.bbclass | 14 ++++++--------
1 file changed, 6 insertions(+), 8 deletions(-)
diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index 05bb9f578d..547c531883 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -61,8 +61,8 @@ def npm_pack(env, srcdir, workdir):
"""Run 'npm pack' on a specified directory"""
import shlex
cmd = "npm pack %s" % shlex.quote(srcdir)
- configs = [("ignore-scripts", "true")]
- tarball = env.run(cmd, configs=configs, workdir=workdir).strip("\n")
+ args = [("ignore-scripts", "true")]
+ tarball = env.run(cmd, args=args, workdir=workdir).strip("\n")
return os.path.join(workdir, tarball)
python npm_do_configure() {
@@ -228,15 +228,11 @@ python npm_do_compile() {
bb.utils.remove(d.getVar("NPM_BUILD"), recurse=True)
- env = NpmEnvironment(d, configs=npm_global_configs(d))
-
- dev = bb.utils.to_boolean(d.getVar("NPM_INSTALL_DEV"), False)
-
with tempfile.TemporaryDirectory() as tmpdir:
args = []
- configs = []
+ configs = npm_global_configs(d)
- if dev:
+ if bb.utils.to_boolean(d.getVar("NPM_INSTALL_DEV"), False):
configs.append(("also", "development"))
else:
configs.append(("only", "production"))
@@ -254,6 +250,8 @@ python npm_do_compile() {
configs.append(("nodedir", d.getVar("NPM_NODEDIR")))
configs.append(("python", d.getVar("PYTHON")))
+ env = NpmEnvironment(d, configs)
+
# Add node-pre-gyp configuration
args.append(("target_arch", d.getVar("NPM_ARCH")))
args.append(("build-from-source", "true"))
--
2.20.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 12/12] recipetool: Rework crunch_license to recognize more variants
2021-10-08 7:42 [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments Stefan Herbrechtsmeier
` (9 preceding siblings ...)
2021-10-08 7:42 ` [PATCH 11/12] npm: Use configs for npm environment and args for npm run command Stefan Herbrechtsmeier
@ 2021-10-08 7:42 ` Stefan Herbrechtsmeier
10 siblings, 0 replies; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-08 7:42 UTC (permalink / raw)
To: openembedded-core; +Cc: Stefan Herbrechtsmeier
From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
---
(no changes since v1)
scripts/lib/recipetool/create.py | 121 +++++++++++++++++++++++++------
1 file changed, 97 insertions(+), 24 deletions(-)
diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index 277266be4e..3b58d6fa82 100644
--- a/scripts/lib/recipetool/create.py
+++ b/scripts/lib/recipetool/create.py
@@ -1048,54 +1048,127 @@ def crunch_license(licfile):
import oe.utils
# Note: these are carefully constructed!
- license_title_re = re.compile(r'^\(?(#+ *)?(The )?.{1,10} [Ll]icen[sc]e( \(.{1,10}\))?\)?:?$')
- license_statement_re = re.compile(r'^(This (project|software) is( free software)? (released|licen[sc]ed)|(Released|Licen[cs]ed)) under the .{1,10} [Ll]icen[sc]e:?$')
- copyright_re = re.compile('^(#+)? *Copyright .*$')
+ license_title_re = re.compile(r'^#*\(? *(This is )?([Tt]he )?.{0,15} ?[Ll]icen[sc]e( \(.{1,10}\))?\)?[:\.]? ?#*$')
+ license_statement_re = re.compile(r'^((This (project|software)|.{1,10}) is( free software)? (released|licen[sc]ed)|(Released|Licen[cs]ed)) under the .{1,10} [Ll]icen[sc]e:?$')
+ copyright_re = re.compile('^ *[#\*]* *(Modified work |MIT LICENSED )?Copyright ?(\([cC]\))? .*$')
+ disclaimer_re = re.compile('^ *\*? ?All [Rr]ights [Rr]eserved\.$')
+ email_re = re.compile('^.*<[\w\.-]*@[\w\.\-]*>$')
+ header_re = re.compile('^(\/\**!?)? ?[\-=\*]* ?(\*\/)?$')
+ tag_re = re.compile('^ *@?\(?([Ll]icense|MIT)\)?$')
+ url_re = re.compile('^ *[#\*]* *https?:\/\/[\w\.\/\-]+$')
crunched_md5sums = {}
+
+ # common licenses
+ crunched_md5sums['89f3bf322f30a1dcfe952e09945842f0'] = 'Apache-2.0'
+ crunched_md5sums['13b6fe3075f8f42f2270a748965bf3a1'] = 'BSD-0-Clause'
+ crunched_md5sums['ba87a7d7c20719c8df4b8beed9b78c43'] = 'BSD-2-Clause'
+ crunched_md5sums['7f8892c03b72de419c27be4ebfa253f8'] = 'BSD-3-Clause'
+ crunched_md5sums['21128c0790b23a8a9f9e260d5f6b3619'] = 'BSL-1.0'
+ crunched_md5sums['975742a59ae1b8abdea63a97121f49f4'] = 'EDL-1.0'
+ crunched_md5sums['5322cee4433d84fb3aafc9e253116447'] = 'EPL-1.0'
+ crunched_md5sums['6922352e87de080f42419bed93063754'] = 'EPL-2.0'
+ crunched_md5sums['793475baa22295cae1d3d4046a3a0ceb'] = 'GPL-2.0-only'
+ crunched_md5sums['ff9047f969b02c20f0559470df5cb433'] = 'GPL-2.0-or-later'
+ crunched_md5sums['ea6de5453fcadf534df246e6cdafadcd'] = 'GPL-3.0-only'
+ crunched_md5sums['b419257d4d153a6fde92ddf96acf5b67'] = 'GPL-3.0-or-later'
+ crunched_md5sums['228737f4c49d3ee75b8fb3706b090b84'] = 'ISC'
+ crunched_md5sums['c6a782e826ca4e85bf7f8b89435a677d'] = 'LGPL-2.0-only'
+ crunched_md5sums['32d8f758a066752f0db09bd7624b8090'] = 'LGPL-2.0-or-later'
+ crunched_md5sums['4820937eb198b4f84c52217ed230be33'] = 'LGPL-2.1-only'
+ crunched_md5sums['db13fe9f3a13af7adab2dc7a76f9e44a'] = 'LGPL-2.1-or-later'
+ crunched_md5sums['d7a0f2e4e0950e837ac3eabf5bd1d246'] = 'LGPL-3.0-only'
+ crunched_md5sums['abbf328e2b434f9153351f06b9f79d02'] = 'LGPL-3.0-or-later'
+ crunched_md5sums['eecf6429523cbc9693547cf2db790b5c'] = 'MIT'
+ crunched_md5sums['b218b0e94290b9b818c4be67c8e1cc82'] = 'MIT-0'
+ crunched_md5sums['ddc18131d6748374f0f35a621c245b49'] = 'Unlicense'
+ crunched_md5sums['51f9570ff32571fc0a443102285c5e33'] = 'WTFPL'
+
# The following two were gleaned from the "forever" npm package
crunched_md5sums['0a97f8e4cbaf889d6fa51f84b89a79f6'] = 'ISC'
- crunched_md5sums['eecf6429523cbc9693547cf2db790b5c'] = 'MIT'
- # https://github.com/vasi/pixz/blob/master/LICENSE
- crunched_md5sums['2f03392b40bbe663597b5bd3cc5ebdb9'] = 'BSD-2-Clause'
# https://github.com/waffle-gl/waffle/blob/master/LICENSE.txt
- crunched_md5sums['e72e5dfef0b1a4ca8a3d26a60587db66'] = 'BSD-2-Clause'
+ crunched_md5sums['50fab24ce589d69af8964fdbfe414c60'] = 'BSD-2-Clause'
# https://github.com/spigwitmer/fakeds1963s/blob/master/LICENSE
- crunched_md5sums['8be76ac6d191671f347ee4916baa637e'] = 'GPLv2'
- # https://github.com/datto/dattobd/blob/master/COPYING
- # http://git.savannah.gnu.org/cgit/freetype/freetype2.git/tree/docs/GPLv2.TXT
- crunched_md5sums['1d65c5ad4bf6489f85f4812bf08ae73d'] = 'GPLv2'
+ crunched_md5sums['88a4355858a1433fea99fae34a44da88'] = 'GPLv2'
# http://www.gnu.org/licenses/old-licenses/gpl-2.0.txt
- # http://git.neil.brown.name/?p=mdadm.git;a=blob;f=COPYING;h=d159169d1050894d3ea3b98e1c965c4058208fe1;hb=HEAD
- crunched_md5sums['fb530f66a7a89ce920f0e912b5b66d4b'] = 'GPLv2'
- # https://github.com/gkos/nrf24/blob/master/COPYING
- crunched_md5sums['7b6aaa4daeafdfa6ed5443fd2684581b'] = 'GPLv2'
- # https://github.com/josch09/resetusb/blob/master/COPYING
- crunched_md5sums['8b8ac1d631a4d220342e83bcf1a1fbc3'] = 'GPLv3'
+ crunched_md5sums['063b5c3ebb5f3aa4c85a2ed18a31fbe7'] = 'GPLv2'
# https://github.com/FFmpeg/FFmpeg/blob/master/COPYING.LGPLv2.1
- crunched_md5sums['2ea316ed973ae176e502e2297b574bb3'] = 'LGPLv2.1'
+ crunched_md5sums['7f5202f4d44ed15dcd4915f5210417d8'] = 'LGPLv2.1'
# unixODBC-2.3.4 COPYING
- crunched_md5sums['1daebd9491d1e8426900b4fa5a422814'] = 'LGPLv2.1'
+ crunched_md5sums['3debde09238a8c8e1f6a847e1ec9055b'] = 'LGPLv2.1'
# https://github.com/FFmpeg/FFmpeg/blob/master/COPYING.LGPLv3
- crunched_md5sums['2ebfb3bb49b9a48a075cc1425e7f4129'] = 'LGPLv3'
+ crunched_md5sums['f90c613c51aa35da4d79dd55fc724ceb'] = 'LGPLv3'
# https://raw.githubusercontent.com/eclipse/mosquitto/v1.4.14/epl-v10
crunched_md5sums['efe2cb9a35826992b9df68224e3c2628'] = 'EPL-1.0'
- # https://raw.githubusercontent.com/eclipse/mosquitto/v1.4.14/edl-v10
- crunched_md5sums['0a9c78c0a398d1bbce4a166757d60387'] = 'EDL-1.0'
+
+ # https://raw.githubusercontent.com/jquery/esprima/3.1.3/LICENSE.BSD
+ crunched_md5sums['80fa7b56a28e8c902e6af194003220a5'] = 'BSD-2-Clause'
+ # https://raw.githubusercontent.com/npm/npm-install-checks/master/LICENSE
+ crunched_md5sums['e659f77bfd9002659e112d0d3d59b2c1'] = 'BSD-2-Clause'
+ # https://raw.githubusercontent.com/silverwind/default-gateway/4.2.0/LICENSE
+ crunched_md5sums['4c641f2d995c47f5cb08bdb4b5b6ea05'] = 'BSD-2-Clause'
+ # https://raw.githubusercontent.com/tad-lispy/node-damerau-levenshtein/v1.0.5/LICENSE
+ crunched_md5sums['2b8c039b2b9a25f0feb4410c4542d346'] = 'BSD-2-Clause'
+ # https://raw.githubusercontent.com/terser/terser/v3.17.0/LICENSE
+ crunched_md5sums['8bd23871802951c9ad63855151204c2c'] = 'BSD-2-Clause'
+ # https://raw.githubusercontent.com/alexei/sprintf.js/1.0.3/LICENSE
+ crunched_md5sums['008c22318c8ea65928bf730ddd0273e3'] = 'BSD-3-Clause'
+ # https://raw.githubusercontent.com/Caligatio/jsSHA/v3.2.0/LICENSE
+ crunched_md5sums['0e46634a01bfef056892949acaea85b1'] = 'BSD-3-Clause'
+ # https://raw.githubusercontent.com/d3/d3-path/v1.0.9/LICENSE
+ crunched_md5sums['b5f72aef53d3b2b432702c30b0215666'] = 'BSD-3-Clause'
+ # https://raw.githubusercontent.com/feross/ieee754/v1.1.13/LICENSE
+ crunched_md5sums['a39327c997c20da0937955192d86232d'] = 'BSD-3-Clause'
+ # https://raw.githubusercontent.com/joyent/node-extsprintf/v1.3.0/LICENSE
+ crunched_md5sums['721f23a96ff4161ca3a5f071bbe18108'] = 'MIT'
+ # https://raw.githubusercontent.com/pvorb/clone/v0.2.0/LICENSE
+ crunched_md5sums['b376d29a53c9573006b9970709231431'] = 'MIT'
+ # https://raw.githubusercontent.com/andris9/encoding/v0.1.12/LICENSE
+ crunched_md5sums['85d8a977ee9d7c5ab4ac03c9b95431c4'] = 'MIT-0'
+ # https://raw.githubusercontent.com/faye/websocket-driver-node/0.7.3/LICENSE.md
+ crunched_md5sums['b66384e7137e41a9b1904ef4d39703b6'] = 'Apache-2.0'
+ # https://raw.githubusercontent.com/less/less.js/v4.1.1/LICENSE
+ crunched_md5sums['b27575459e02221ccef97ec0bfd457ae'] = 'Apache-2.0'
+ # https://raw.githubusercontent.com/microsoft/TypeScript/v3.5.3/LICENSE.txt
+ crunched_md5sums['a54a1a6a39e7f9dbb4a23a42f5c7fd1c'] = 'Apache-2.0'
+ # https://raw.githubusercontent.com/request/request/v2.87.0/LICENSE
+ crunched_md5sums['1034431802e57486b393d00c5d262b8a'] = 'Apache-2.0'
+ # https://raw.githubusercontent.com/dchest/tweetnacl-js/v0.14.5/LICENSE
+ crunched_md5sums['75605e6bdd564791ab698fca65c94a4f'] = 'Unlicense'
+ # https://raw.githubusercontent.com/stackgl/gl-mat3/v2.0.0/LICENSE.md
+ crunched_md5sums['75512892d6f59dddb6d1c7e191957e9c'] = 'Zlib'
+
lictext = []
with open(licfile, 'r', errors='surrogateescape') as f:
for line in f:
# Drop opening statements
if copyright_re.match(line):
continue
+ elif disclaimer_re.match(line):
+ continue
+ elif email_re.match(line):
+ continue
+ elif header_re.match(line):
+ continue
+ elif tag_re.match(line):
+ continue
+ elif url_re.match(line):
+ continue
elif license_title_re.match(line):
continue
elif license_statement_re.match(line):
continue
- # Squash spaces, and replace smart quotes, double quotes
- # and backticks with single quotes
+ # Strip comment symbols
+ line = line.replace('*', '') \
+ .replace('#', '')
+ # Unify spelling
+ line = line.replace('sub-license', 'sublicense')
+ # Squash spaces
line = oe.utils.squashspaces(line.strip())
+ # Replace smart quotes, double quotes and backticks with single quotes
line = line.replace(u"\u2018", "'").replace(u"\u2019", "'").replace(u"\u201c","'").replace(u"\u201d", "'").replace('"', '\'').replace('`', '\'')
+ # Unify brackets
+ line = line.replace("{", "[").replace("}", "]")
if line:
lictext.append(line)
--
2.20.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [OE-core] [PATCH 03/12] recipetool: Skip common source files in guess_license
2021-10-08 7:42 ` [PATCH 03/12] recipetool: Skip common source files in guess_license Stefan Herbrechtsmeier
@ 2021-10-08 7:45 ` Konrad Weihmann
2021-10-08 7:55 ` Stefan Herbrechtsmeier
2021-10-08 12:26 ` Richard Purdie
0 siblings, 2 replies; 18+ messages in thread
From: Konrad Weihmann @ 2021-10-08 7:45 UTC (permalink / raw)
To: Stefan Herbrechtsmeier, openembedded-core; +Cc: Stefan Herbrechtsmeier
On 08.10.21 09:42, Stefan Herbrechtsmeier wrote:
> From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
>
> Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
> ---
>
> (no changes since v1)
>
> scripts/lib/recipetool/create.py | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
> index e88a4253da..83cf25d9b7 100644
> --- a/scripts/lib/recipetool/create.py
> +++ b/scripts/lib/recipetool/create.py
> @@ -1114,6 +1114,8 @@ def guess_license(srctree, d):
> licfiles = []
> for root, dirs, files in os.walk(srctree):
> for fn in files:
> + if fn.endswith(".html") or fn.endswith(".js") or fn.endswith(".json") or fn.endswith(".svg") or fn.endswith(".ts"):
Why not
_, _ext = os.path.splitext(fn)
if _ext in [".html", ".js", ".json", ".svg", ".ts"]:
?
to me that would be easier to read and maintain than this ever growing
or clause construction
> + continue
> for spec in licspecs:
> if fnmatch.fnmatch(fn, spec):
> fullpath = os.path.join(root, fn)
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#156745): https://lists.openembedded.org/g/openembedded-core/message/156745
> Mute This Topic: https://lists.openembedded.org/mt/86165223/3647476
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [kweihmann@outlook.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [OE-core] [PATCH 03/12] recipetool: Skip common source files in guess_license
2021-10-08 7:45 ` [OE-core] " Konrad Weihmann
@ 2021-10-08 7:55 ` Stefan Herbrechtsmeier
2021-10-08 12:26 ` Richard Purdie
1 sibling, 0 replies; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-08 7:55 UTC (permalink / raw)
To: Konrad Weihmann, openembedded-core; +Cc: Stefan Herbrechtsmeier
Am 08.10.2021 um 09:45 schrieb Konrad Weihmann:
>
>
> On 08.10.21 09:42, Stefan Herbrechtsmeier wrote:
>> From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
>>
>> Signed-off-by: Stefan Herbrechtsmeier
>> <stefan.herbrechtsmeier@weidmueller.com>
>> ---
>>
>> (no changes since v1)
>>
>> scripts/lib/recipetool/create.py | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>> diff --git a/scripts/lib/recipetool/create.py
>> b/scripts/lib/recipetool/create.py
>> index e88a4253da..83cf25d9b7 100644
>> --- a/scripts/lib/recipetool/create.py
>> +++ b/scripts/lib/recipetool/create.py
>> @@ -1114,6 +1114,8 @@ def guess_license(srctree, d):
>> licfiles = []
>> for root, dirs, files in os.walk(srctree):
>> for fn in files:
>> + if fn.endswith(".html") or fn.endswith(".js") or
>> fn.endswith(".json") or fn.endswith(".svg") or fn.endswith(".ts"):
>
> Why not
>
> _, _ext = os.path.splitext(fn)
> if _ext in [".html", ".js", ".json", ".svg", ".ts"]:
>
> ?
>
> to me that would be easier to read and maintain than this ever growing
> or clause construction
Will be changed in the next version.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [OE-core] [PATCH 03/12] recipetool: Skip common source files in guess_license
2021-10-08 7:45 ` [OE-core] " Konrad Weihmann
2021-10-08 7:55 ` Stefan Herbrechtsmeier
@ 2021-10-08 12:26 ` Richard Purdie
2021-10-12 12:27 ` Stefan Herbrechtsmeier
1 sibling, 1 reply; 18+ messages in thread
From: Richard Purdie @ 2021-10-08 12:26 UTC (permalink / raw)
To: Konrad Weihmann, Stefan Herbrechtsmeier, openembedded-core
Cc: Stefan Herbrechtsmeier
On Fri, 2021-10-08 at 09:45 +0200, Konrad Weihmann wrote:
>
> On 08.10.21 09:42, Stefan Herbrechtsmeier wrote:
> > From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
> >
> > Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
> > ---
> >
> > (no changes since v1)
> >
> > scripts/lib/recipetool/create.py | 2 ++
> > 1 file changed, 2 insertions(+)
> >
> > diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
> > index e88a4253da..83cf25d9b7 100644
> > --- a/scripts/lib/recipetool/create.py
> > +++ b/scripts/lib/recipetool/create.py
> > @@ -1114,6 +1114,8 @@ def guess_license(srctree, d):
> > licfiles = []
> > for root, dirs, files in os.walk(srctree):
> > for fn in files:
> > + if fn.endswith(".html") or fn.endswith(".js") or fn.endswith(".json") or fn.endswith(".svg") or fn.endswith(".ts"):
>
> Why not
>
> _, _ext = os.path.splitext(fn)
> if _ext in [".html", ".js", ".json", ".svg", ".ts"]:
you can do fn.endswith((".html", ".js", ".json", ".svg", ".ts"))
Cheers,
Richard
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [OE-core] [PATCH 03/12] recipetool: Skip common source files in guess_license
2021-10-08 12:26 ` Richard Purdie
@ 2021-10-12 12:27 ` Stefan Herbrechtsmeier
0 siblings, 0 replies; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-12 12:27 UTC (permalink / raw)
To: Richard Purdie, Konrad Weihmann, openembedded-core; +Cc: Stefan Herbrechtsmeier
Hi Richard,
Am 08.10.2021 um 14:26 schrieb Richard Purdie:
> On Fri, 2021-10-08 at 09:45 +0200, Konrad Weihmann wrote:
>>
>> On 08.10.21 09:42, Stefan Herbrechtsmeier wrote:
>>> From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
>>>
>>> Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
>>> ---
>>>
>>> (no changes since v1)
>>>
>>> scripts/lib/recipetool/create.py | 2 ++
>>> 1 file changed, 2 insertions(+)
>>>
>>> diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
>>> index e88a4253da..83cf25d9b7 100644
>>> --- a/scripts/lib/recipetool/create.py
>>> +++ b/scripts/lib/recipetool/create.py
>>> @@ -1114,6 +1114,8 @@ def guess_license(srctree, d):
>>> licfiles = []
>>> for root, dirs, files in os.walk(srctree):
>>> for fn in files:
>>> + if fn.endswith(".html") or fn.endswith(".js") or fn.endswith(".json") or fn.endswith(".svg") or fn.endswith(".ts"):
>>
>> Why not
>>
>> _, _ext = os.path.splitext(fn)
>> if _ext in [".html", ".js", ".json", ".svg", ".ts"]:
>
> you can do fn.endswith((".html", ".js", ".json", ".svg", ".ts"))
Thanks, I will post an updated series next week.
Regards
Stefan
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [OE-core] [PATCH 04/12] recipetool: Change default paramter fallback_licenses of function split_pkg_licenses from None to []
2021-10-08 7:42 ` [PATCH 04/12] recipetool: Change default paramter fallback_licenses of function split_pkg_licenses from None to [] Stefan Herbrechtsmeier
@ 2022-04-13 21:35 ` Paul Eggleton
0 siblings, 0 replies; 18+ messages in thread
From: Paul Eggleton @ 2022-04-13 21:35 UTC (permalink / raw)
To: Stefan Herbrechtsmeier, Stefan Herbrechtsmeier; +Cc: openembedded-core
Hi Stefan
Firstly, thanks for the work extending and improving this.
Secondly, apologies for dredging this up, but I just found this looking back
through the history for the release notes:
On Friday, 8 October 2021 20:42:20 NZST Stefan Herbrechtsmeier wrote:
> From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
>
> Signed-off-by: Stefan Herbrechtsmeier
> <stefan.herbrechtsmeier@weidmueller.com> ---
>
> (no changes since v1)
>
> scripts/lib/recipetool/create.py | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/lib/recipetool/create.py
> b/scripts/lib/recipetool/create.py index 83cf25d9b7..893980a5ce 100644
> --- a/scripts/lib/recipetool/create.py
> +++ b/scripts/lib/recipetool/create.py
> @@ -1134,7 +1134,7 @@ def guess_license(srctree, d):
>
> return licenses
>
> -def split_pkg_licenses(licvalues, packages, outlines,
> fallback_licenses=None, pn='${PN}'): +def split_pkg_licenses(licvalues,
> packages, outlines, fallback_licenses=[], pn='${PN}'): """
> Given a list of (license, path, md5sum) as returned by guess_license(),
> a dict of package name to path mappings, write out a set of
Note that setting a mutable type (e.g. an empty list) as a default for a
function parameter is not good practice in Python - see:
https://florimond.dev/en/posts/2018/08/python-mutable-defaults-are-the-source-of-all-evil/
I'm aware we have done this already in a few places - git grep "^def.*=\[\]"
shows a few others that we probably ought to fix.
Paul
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH 11/12] npm: Use configs for npm environment and args for npm run command
2021-10-07 10:13 [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments Stefan Herbrechtsmeier
@ 2021-10-07 10:13 ` Stefan Herbrechtsmeier
0 siblings, 0 replies; 18+ messages in thread
From: Stefan Herbrechtsmeier @ 2021-10-07 10:13 UTC (permalink / raw)
To: openembedded-core; +Cc: Stefan Herbrechtsmeier
From: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Use parameter configs of class NpmEnvironment and parameter args of
function run to support a common npmrc for all run calls of a single
NpmEnvironment.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
---
meta/classes/npm.bbclass | 14 ++++++--------
1 file changed, 6 insertions(+), 8 deletions(-)
diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index 05bb9f578d..547c531883 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -61,8 +61,8 @@ def npm_pack(env, srcdir, workdir):
"""Run 'npm pack' on a specified directory"""
import shlex
cmd = "npm pack %s" % shlex.quote(srcdir)
- configs = [("ignore-scripts", "true")]
- tarball = env.run(cmd, configs=configs, workdir=workdir).strip("\n")
+ args = [("ignore-scripts", "true")]
+ tarball = env.run(cmd, args=args, workdir=workdir).strip("\n")
return os.path.join(workdir, tarball)
python npm_do_configure() {
@@ -228,15 +228,11 @@ python npm_do_compile() {
bb.utils.remove(d.getVar("NPM_BUILD"), recurse=True)
- env = NpmEnvironment(d, configs=npm_global_configs(d))
-
- dev = bb.utils.to_boolean(d.getVar("NPM_INSTALL_DEV"), False)
-
with tempfile.TemporaryDirectory() as tmpdir:
args = []
- configs = []
+ configs = npm_global_configs(d)
- if dev:
+ if bb.utils.to_boolean(d.getVar("NPM_INSTALL_DEV"), False):
configs.append(("also", "development"))
else:
configs.append(("only", "production"))
@@ -254,6 +250,8 @@ python npm_do_compile() {
configs.append(("nodedir", d.getVar("NPM_NODEDIR")))
configs.append(("python", d.getVar("PYTHON")))
+ env = NpmEnvironment(d, configs)
+
# Add node-pre-gyp configuration
args.append(("target_arch", d.getVar("NPM_ARCH")))
args.append(("build-from-source", "true"))
--
2.20.1
^ permalink raw reply related [flat|nested] 18+ messages in thread
end of thread, other threads:[~2022-04-14 16:03 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-08 7:42 [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 02/12] recipetool: Move license md5sums into CSV files Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 03/12] recipetool: Skip common source files in guess_license Stefan Herbrechtsmeier
2021-10-08 7:45 ` [OE-core] " Konrad Weihmann
2021-10-08 7:55 ` Stefan Herbrechtsmeier
2021-10-08 12:26 ` Richard Purdie
2021-10-12 12:27 ` Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 04/12] recipetool: Change default paramter fallback_licenses of function split_pkg_licenses from None to [] Stefan Herbrechtsmeier
2022-04-13 21:35 ` [OE-core] " Paul Eggleton
2021-10-08 7:42 ` [PATCH 05/12] recipetool: ignore empty license files Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 06/12] recipetool: Add logger info for missing license entries Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 07/12] recipetool: Add support for linenumbers to licenses.csv Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 08/12] recipetool: npm: Do not add package.json files to LIC_FILES_CHKSUM Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 09/12] recipetool: npm: Use README as license fallback Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 10/12] npm: Add variable NPM_NODEDIR with default value Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 11/12] npm: Use configs for npm environment and args for npm run command Stefan Herbrechtsmeier
2021-10-08 7:42 ` [PATCH 12/12] recipetool: Rework crunch_license to recognize more variants Stefan Herbrechtsmeier
-- strict thread matches above, loose matches on Subject: below --
2021-10-07 10:13 [PATCH 01/12] npm: Add support for EXTRA_OENPM arguments Stefan Herbrechtsmeier
2021-10-07 10:13 ` [PATCH 11/12] npm: Use configs for npm environment and args for npm run command Stefan Herbrechtsmeier
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.