* [Buildroot] [PATCH 1/4] support/scripts/pkg-stats: import cve module only when needed @ 2020-11-19 14:53 Thomas Petazzoni 2020-11-19 14:53 ` [Buildroot] [PATCH 2/4] support/scripts/pkg-stats: fix Python 3.8 deprecation warning Thomas Petazzoni ` (4 more replies) 0 siblings, 5 replies; 11+ messages in thread From: Thomas Petazzoni @ 2020-11-19 14:53 UTC (permalink / raw) To: buildroot The cve module needs ijson, which may not be installed. Since cve matching is only enabled when --nvd-path is passed, it is a bit silly to error out about ijson being missing if it's not used. So instead of unconditionally importing the cve module, only do it conditionally. However, instead of doing it right at the point where it is used, we do it at the beginning of the main() function. Indeed, if the cve module is needed but cannot be imported, we want to error out immediately rather than doing a whole bunch of things, and failing on the user later on in the middle of the pkg-stats execution. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> --- support/scripts/pkg-stats | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats index d44f8241c1..2676cc49dd 100755 --- a/support/scripts/pkg-stats +++ b/support/scripts/pkg-stats @@ -32,7 +32,6 @@ brpath = os.path.normpath(os.path.join(os.path.dirname(__file__), "..", "..")) sys.path.append(os.path.join(brpath, "utils")) from getdeveloperlib import parse_developers # noqa: E402 -import cve as cvecheck # noqa: E402 INFRA_RE = re.compile(r"\$\(eval \$\(([a-z-]*)-package\)\)") @@ -950,7 +949,13 @@ def parse_args(): def __main__(): + global cvecheck + args = parse_args() + + if args.nvd_path: + import cve as cvecheck + if args.packages: package_list = args.packages.split(",") elif args.configpackages: -- 2.28.0 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 2/4] support/scripts/pkg-stats: fix Python 3.8 deprecation warning 2020-11-19 14:53 [Buildroot] [PATCH 1/4] support/scripts/pkg-stats: import cve module only when needed Thomas Petazzoni @ 2020-11-19 14:53 ` Thomas Petazzoni 2020-11-19 14:53 ` [Buildroot] [PATCH 3/4] utils/get-developers: use Developers.hasfile() where appropriate Thomas Petazzoni ` (3 subsequent siblings) 4 siblings, 0 replies; 11+ messages in thread From: Thomas Petazzoni @ 2020-11-19 14:53 UTC (permalink / raw) To: buildroot With Python 3.8, the following deprecation warnings are emitted: /home/thomas/projets/buildroot/./support/scripts/pkg-stats:418: DeprecationWarning: The explicit passing of coroutine objects to asyncio.wait() is deprecated since Python 3.8, and scheduled for removal in Python 3.11. /home/thomas/projets/buildroot/./support/scripts/pkg-stats:536: DeprecationWarning: The explicit passing of coroutine objects to asyncio.wait() is deprecated since Python 3.8, and scheduled for removal in Python 3.11. The correct way to pass coroutines is to use asyncio.create_task(), but this is rather new method (Python 3.7), and using it breaks compatibility with older Python versions. As suggested at https://docs.python.org/3/library/asyncio-task.html#asyncio.create_task, use the more cryptic, but also more compatible asyncio.ensure_future() method. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> --- support/scripts/pkg-stats | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats index 2676cc49dd..e472b67784 100755 --- a/support/scripts/pkg-stats +++ b/support/scripts/pkg-stats @@ -413,7 +413,7 @@ async def check_package_urls(packages): async with aiohttp.ClientSession(connector=connector, trust_env=True) as sess: packages = [p for p in packages if p.status['url'][0] == 'ok'] for pkg in packages: - tasks.append(check_url_status(sess, pkg, len(packages))) + tasks.append(asyncio.ensure_future(check_url_status(sess, pkg, len(packages)))) await asyncio.wait(tasks) @@ -531,7 +531,7 @@ async def check_package_latest_version(packages): async with aiohttp.ClientSession(connector=connector, trust_env=True) as sess: packages = [p for p in packages if p.has_valid_infra] for pkg in packages: - tasks.append(check_package_latest_version_get(sess, pkg, len(packages))) + tasks.append(asyncio.ensure_future(check_package_latest_version_get(sess, pkg, len(packages)))) await asyncio.wait(tasks) -- 2.28.0 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 3/4] utils/get-developers: use Developers.hasfile() where appropriate 2020-11-19 14:53 [Buildroot] [PATCH 1/4] support/scripts/pkg-stats: import cve module only when needed Thomas Petazzoni 2020-11-19 14:53 ` [Buildroot] [PATCH 2/4] support/scripts/pkg-stats: fix Python 3.8 deprecation warning Thomas Petazzoni @ 2020-11-19 14:53 ` Thomas Petazzoni 2020-11-20 10:03 ` Heiko Thiery 2020-11-19 14:53 ` [Buildroot] [PATCH 4/4] utils/getdeveloperlib.py: use relative paths for files Thomas Petazzoni ` (2 subsequent siblings) 4 siblings, 1 reply; 11+ messages in thread From: Thomas Petazzoni @ 2020-11-19 14:53 UTC (permalink / raw) To: buildroot Instead of open-coding Developers.hasfile() in utils/get-developers, use it directly. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> --- utils/get-developers | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/utils/get-developers b/utils/get-developers index 750fc556dc..20272ed60b 100755 --- a/utils/get-developers +++ b/utils/get-developers @@ -77,9 +77,8 @@ def __main__(): if args.files is not None: args.files = [os.path.abspath(f) for f in args.files] for dev in devs: - for devfile in dev.files: - commonfiles = [f for f in args.files if f.startswith(devfile)] - if commonfiles: + for f in args.files: + if dev.hasfile(f): print(dev.name) break -- 2.28.0 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 3/4] utils/get-developers: use Developers.hasfile() where appropriate 2020-11-19 14:53 ` [Buildroot] [PATCH 3/4] utils/get-developers: use Developers.hasfile() where appropriate Thomas Petazzoni @ 2020-11-20 10:03 ` Heiko Thiery 0 siblings, 0 replies; 11+ messages in thread From: Heiko Thiery @ 2020-11-20 10:03 UTC (permalink / raw) To: buildroot Hi Thomas, Am Do., 19. Nov. 2020 um 15:54 Uhr schrieb Thomas Petazzoni <thomas.petazzoni@bootlin.com>: > > Instead of open-coding Developers.hasfile() in utils/get-developers, > use it directly. > > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Tested-by: Heiko Thiery <heiko.thiery@gmail.com> > --- > utils/get-developers | 5 ++--- > 1 file changed, 2 insertions(+), 3 deletions(-) > > diff --git a/utils/get-developers b/utils/get-developers > index 750fc556dc..20272ed60b 100755 > --- a/utils/get-developers > +++ b/utils/get-developers > @@ -77,9 +77,8 @@ def __main__(): > if args.files is not None: > args.files = [os.path.abspath(f) for f in args.files] > for dev in devs: > - for devfile in dev.files: > - commonfiles = [f for f in args.files if f.startswith(devfile)] > - if commonfiles: > + for f in args.files: > + if dev.hasfile(f): > print(dev.name) > break > Thank you -- Heiko ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 4/4] utils/getdeveloperlib.py: use relative paths for files 2020-11-19 14:53 [Buildroot] [PATCH 1/4] support/scripts/pkg-stats: import cve module only when needed Thomas Petazzoni 2020-11-19 14:53 ` [Buildroot] [PATCH 2/4] support/scripts/pkg-stats: fix Python 3.8 deprecation warning Thomas Petazzoni 2020-11-19 14:53 ` [Buildroot] [PATCH 3/4] utils/get-developers: use Developers.hasfile() where appropriate Thomas Petazzoni @ 2020-11-19 14:53 ` Thomas Petazzoni 2020-11-20 10:03 ` Heiko Thiery 2020-11-20 10:02 ` [Buildroot] [PATCH 1/4] support/scripts/pkg-stats: import cve module only when needed Heiko Thiery 2021-01-17 14:32 ` Thomas Petazzoni 4 siblings, 1 reply; 11+ messages in thread From: Thomas Petazzoni @ 2020-11-19 14:53 UTC (permalink / raw) To: buildroot Using absolute paths within getdeveloperlib isn't very sensible, it makes a lot more sense to handle everything as relative paths from the top-level Buildroot source directory. parse_developers() is changed to no longer take the base path as argument: it is automatically calculated based on the location of utils/getdeveloperlib.py. Then, the rest of the logic is adjusted to use relative paths, and prepend them with the base "brpath" when needed. This commit allows pkg-stats to report correct developers information even when executed from an out of tree directory. Before this patch: $ ~/buildroot/support/scripts/pkg-stats -p ipmitool --json out.json $ cat out.json | jq '.packages.ipmitool.developers' [] $ cat out.json | jq '.defconfigs.stm32f469_disco' { "name": "stm32f469_disco", "path": "configs/stm32f469_disco_defconfig", "developers": [] } After this patch: $ ~/buildroot/support/scripts/pkg-stats -p ipmitool --json out.json $ cat out.json | jq '.packages.ipmitool.developers' [ "Floris Bos <bos@je-eigen-domein.nl>", "Heiko Thiery <heiko.thiery@gmail.com>" ] $ cat out.json | jq '.defconfigs.stm32f469_disco' { "name": "stm32f469_disco", "path": "configs/stm32f469_disco_defconfig", "developers": [ "Christophe Priouzeau <christophe.priouzeau@st.com>" ] } Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> --- support/scripts/pkg-stats | 2 +- utils/get-developers | 5 ----- utils/getdeveloperlib.py | 26 ++++++++++++-------------- 3 files changed, 13 insertions(+), 20 deletions(-) diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats index e472b67784..f6460cf6f2 100755 --- a/support/scripts/pkg-stats +++ b/support/scripts/pkg-stats @@ -969,7 +969,7 @@ def __main__(): print("Build package list ...") packages = get_pkglist(args.npackages, package_list) print("Getting developers ...") - developers = parse_developers(brpath) + developers = parse_developers() print("Build defconfig list ...") defconfigs = get_defconfig_list() for d in defconfigs: diff --git a/utils/get-developers b/utils/get-developers index 20272ed60b..e027c26562 100755 --- a/utils/get-developers +++ b/utils/get-developers @@ -45,10 +45,6 @@ def __main__(): print("No action specified") return - # getdeveloperlib expects to be executed from the toplevel buildroot - # directory, which is one level up from this script - os.chdir(os.path.join(os.path.dirname(os.path.realpath(__file__)), '..')) - devs = getdeveloperlib.parse_developers() if devs is None: sys.exit(1) @@ -75,7 +71,6 @@ def __main__(): # Handle the files action if args.files is not None: - args.files = [os.path.abspath(f) for f in args.files] for dev in devs: for f in args.files: if dev.hasfile(f): diff --git a/utils/getdeveloperlib.py b/utils/getdeveloperlib.py index f57f41887b..d7a90457ed 100644 --- a/utils/getdeveloperlib.py +++ b/utils/getdeveloperlib.py @@ -6,6 +6,8 @@ import subprocess import sys import unittest +brpath = os.path.normpath(os.path.join(os.path.dirname(__file__), "..")) + # # Patch parsing functions # @@ -94,14 +96,14 @@ def get_all_test_cases(suite): yield (suite.__module__, suite.__class__.__name__) -def list_unittests(path): +def list_unittests(): """Use the unittest module to retreive all test cases from a given directory""" loader = unittest.TestLoader() - suite = loader.discover(path) + suite = loader.discover(os.path.join(brpath, "support", "testing")) tests = {} for module, test in get_all_test_cases(suite): - module_path = os.path.join(path, *module.split('.')) + module_path = os.path.join("support", "testing", *module.split('.')) tests.setdefault(module_path, []).append('%s.%s' % (module, test)) return tests @@ -124,9 +126,7 @@ class Developer: self.defconfigs = parse_developer_defconfigs(files) def hasfile(self, f): - f = os.path.abspath(f) for fs in self.files: - fs = os.path.abspath(fs) if f.startswith(fs): return True return False @@ -158,7 +158,7 @@ def parse_developer_packages(fnames): patterns, and return a list of those packages.""" packages = set() for fname in fnames: - for root, dirs, files in os.walk(fname): + for root, dirs, files in os.walk(os.path.join(brpath, fname)): for f in files: path = os.path.join(root, f) if fname_get_package_infra(path): @@ -223,7 +223,7 @@ def parse_developer_runtime_tests(fnames): # List all files recursively for fname in fnames: if os.path.isdir(fname): - for root, _dirs, files in os.walk(fname): + for root, _dirs, files in os.walk(os.path.join(brpath, fname)): all_files += [os.path.join(root, f) for f in files] else: all_files.append(fname) @@ -237,15 +237,13 @@ def parse_developer_runtime_tests(fnames): return runtimes -def parse_developers(basepath=None): +def parse_developers(): """Parse the DEVELOPERS file and return a list of Developer objects.""" developers = [] linen = 0 - if basepath is None: - basepath = os.getcwd() global unittests - unittests = list_unittests(os.path.join(basepath, 'support/testing')) - with open(os.path.join(basepath, "DEVELOPERS"), "r") as f: + unittests = list_unittests() + with open(os.path.join(brpath, "DEVELOPERS"), "r") as f: files = [] name = None for line in f: @@ -259,11 +257,11 @@ def parse_developers(basepath=None): name = line[2:].strip() elif line.startswith("F:"): fname = line[2:].strip() - dev_files = glob.glob(os.path.join(basepath, fname)) + dev_files = glob.glob(os.path.join(brpath, fname)) if len(dev_files) == 0: print("WARNING: '%s' doesn't match any file" % fname, file=sys.stderr) - files += dev_files + files += [os.path.relpath(f, brpath) for f in dev_files] elif line == "": if not name: continue -- 2.28.0 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 4/4] utils/getdeveloperlib.py: use relative paths for files 2020-11-19 14:53 ` [Buildroot] [PATCH 4/4] utils/getdeveloperlib.py: use relative paths for files Thomas Petazzoni @ 2020-11-20 10:03 ` Heiko Thiery 2020-11-20 14:14 ` Thomas Petazzoni 0 siblings, 1 reply; 11+ messages in thread From: Heiko Thiery @ 2020-11-20 10:03 UTC (permalink / raw) To: buildroot Hi Thomas, Am Do., 19. Nov. 2020 um 15:54 Uhr schrieb Thomas Petazzoni <thomas.petazzoni@bootlin.com>: > > Using absolute paths within getdeveloperlib isn't very sensible, it > makes a lot more sense to handle everything as relative paths from the > top-level Buildroot source directory. > > parse_developers() is changed to no longer take the base path as > argument: it is automatically calculated based on the location of > utils/getdeveloperlib.py. Then, the rest of the logic is adjusted to > use relative paths, and prepend them with the base "brpath" when > needed. > > This commit allows pkg-stats to report correct developers information > even when executed from an out of tree directory. Just tried this but either I do not completely understand what it should do or there is an issue. I do as follows: --- >8 --- # mkdir b # cd b # make ../buildroot/ defconfig # ../buildroot/support/scripts/pkg-stats -c --json out.json make: *** Keine Regel, um ?show-info? zu erstellen. Schluss. Traceback (most recent call last): File "../buildroot/support/scripts/pkg-stats", line 1008, in <module> __main__() File "../buildroot/support/scripts/pkg-stats", line 962, in __main__ package_list = get_config_packages() File "../buildroot/support/scripts/pkg-stats", line 337, in get_config_packages js = json.loads(subprocess.check_output(cmd)) File "/usr/lib/python3.7/subprocess.py", line 395, in check_output **kwargs).stdout File "/usr/lib/python3.7/subprocess.py", line 487, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['make', '--no-print-directory', 'show-info']' returned non-zero exit status 2. --- >8 --- > > Before this patch: > > $ ~/buildroot/support/scripts/pkg-stats -p ipmitool --json out.json > $ cat out.json | jq '.packages.ipmitool.developers' > [] > > $ cat out.json | jq '.defconfigs.stm32f469_disco' > { > "name": "stm32f469_disco", > "path": "configs/stm32f469_disco_defconfig", > "developers": [] > } > > After this patch: > > $ ~/buildroot/support/scripts/pkg-stats -p ipmitool --json out.json > $ cat out.json | jq '.packages.ipmitool.developers' > [ > "Floris Bos <bos@je-eigen-domein.nl>", > "Heiko Thiery <heiko.thiery@gmail.com>" > ] > $ cat out.json | jq '.defconfigs.stm32f469_disco' > { > "name": "stm32f469_disco", > "path": "configs/stm32f469_disco_defconfig", > "developers": [ > "Christophe Priouzeau <christophe.priouzeau@st.com>" > ] > } In-tree this change works as expected. > > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Tested-by: Heiko Thiery <heiko.thiery@gmail.com> > --- > support/scripts/pkg-stats | 2 +- > utils/get-developers | 5 ----- > utils/getdeveloperlib.py | 26 ++++++++++++-------------- > 3 files changed, 13 insertions(+), 20 deletions(-) > > diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats > index e472b67784..f6460cf6f2 100755 > --- a/support/scripts/pkg-stats > +++ b/support/scripts/pkg-stats > @@ -969,7 +969,7 @@ def __main__(): > print("Build package list ...") > packages = get_pkglist(args.npackages, package_list) > print("Getting developers ...") > - developers = parse_developers(brpath) > + developers = parse_developers() > print("Build defconfig list ...") > defconfigs = get_defconfig_list() > for d in defconfigs: > diff --git a/utils/get-developers b/utils/get-developers > index 20272ed60b..e027c26562 100755 > --- a/utils/get-developers > +++ b/utils/get-developers > @@ -45,10 +45,6 @@ def __main__(): > print("No action specified") > return > > - # getdeveloperlib expects to be executed from the toplevel buildroot > - # directory, which is one level up from this script > - os.chdir(os.path.join(os.path.dirname(os.path.realpath(__file__)), '..')) > - > devs = getdeveloperlib.parse_developers() > if devs is None: > sys.exit(1) > @@ -75,7 +71,6 @@ def __main__(): > > # Handle the files action > if args.files is not None: > - args.files = [os.path.abspath(f) for f in args.files] > for dev in devs: > for f in args.files: > if dev.hasfile(f): > diff --git a/utils/getdeveloperlib.py b/utils/getdeveloperlib.py > index f57f41887b..d7a90457ed 100644 > --- a/utils/getdeveloperlib.py > +++ b/utils/getdeveloperlib.py > @@ -6,6 +6,8 @@ import subprocess > import sys > import unittest > > +brpath = os.path.normpath(os.path.join(os.path.dirname(__file__), "..")) > + > # > # Patch parsing functions > # > @@ -94,14 +96,14 @@ def get_all_test_cases(suite): > yield (suite.__module__, suite.__class__.__name__) > > > -def list_unittests(path): > +def list_unittests(): > """Use the unittest module to retreive all test cases from a given > directory""" > loader = unittest.TestLoader() > - suite = loader.discover(path) > + suite = loader.discover(os.path.join(brpath, "support", "testing")) > tests = {} > for module, test in get_all_test_cases(suite): > - module_path = os.path.join(path, *module.split('.')) > + module_path = os.path.join("support", "testing", *module.split('.')) > tests.setdefault(module_path, []).append('%s.%s' % (module, test)) > return tests > > @@ -124,9 +126,7 @@ class Developer: > self.defconfigs = parse_developer_defconfigs(files) > > def hasfile(self, f): > - f = os.path.abspath(f) > for fs in self.files: > - fs = os.path.abspath(fs) > if f.startswith(fs): > return True > return False > @@ -158,7 +158,7 @@ def parse_developer_packages(fnames): > patterns, and return a list of those packages.""" > packages = set() > for fname in fnames: > - for root, dirs, files in os.walk(fname): > + for root, dirs, files in os.walk(os.path.join(brpath, fname)): > for f in files: > path = os.path.join(root, f) > if fname_get_package_infra(path): > @@ -223,7 +223,7 @@ def parse_developer_runtime_tests(fnames): > # List all files recursively > for fname in fnames: > if os.path.isdir(fname): > - for root, _dirs, files in os.walk(fname): > + for root, _dirs, files in os.walk(os.path.join(brpath, fname)): > all_files += [os.path.join(root, f) for f in files] > else: > all_files.append(fname) > @@ -237,15 +237,13 @@ def parse_developer_runtime_tests(fnames): > return runtimes > > > -def parse_developers(basepath=None): > +def parse_developers(): > """Parse the DEVELOPERS file and return a list of Developer objects.""" > developers = [] > linen = 0 > - if basepath is None: > - basepath = os.getcwd() > global unittests > - unittests = list_unittests(os.path.join(basepath, 'support/testing')) > - with open(os.path.join(basepath, "DEVELOPERS"), "r") as f: > + unittests = list_unittests() > + with open(os.path.join(brpath, "DEVELOPERS"), "r") as f: > files = [] > name = None > for line in f: > @@ -259,11 +257,11 @@ def parse_developers(basepath=None): > name = line[2:].strip() > elif line.startswith("F:"): > fname = line[2:].strip() > - dev_files = glob.glob(os.path.join(basepath, fname)) > + dev_files = glob.glob(os.path.join(brpath, fname)) > if len(dev_files) == 0: > print("WARNING: '%s' doesn't match any file" % fname, > file=sys.stderr) > - files += dev_files > + files += [os.path.relpath(f, brpath) for f in dev_files] > elif line == "": > if not name: > continue Thank you. -- Heiko ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 4/4] utils/getdeveloperlib.py: use relative paths for files 2020-11-20 10:03 ` Heiko Thiery @ 2020-11-20 14:14 ` Thomas Petazzoni 2020-11-20 14:26 ` Heiko Thiery 0 siblings, 1 reply; 11+ messages in thread From: Thomas Petazzoni @ 2020-11-20 14:14 UTC (permalink / raw) To: buildroot Hello Heiko, Thanks for the review and feedback! On Fri, 20 Nov 2020 11:03:45 +0100 Heiko Thiery <heiko.thiery@gmail.com> wrote: > I do as follows: > > --- >8 --- > # mkdir b > # cd b > # make ../buildroot/ defconfig > # ../buildroot/support/scripts/pkg-stats -c --json out.json > make: *** Keine Regel, um ?show-info? zu erstellen. Schluss. The current folder must be a Builroot output directory: $ mkdir foobar/ $ cd foobar/ $ make -C /path/to/buildroot O=$(pwd) blabla_defconfig $ make pkg-stats or $ /path/to/buildroot/support/scripts/pkg-stats -c --json out.json Or this should also work (but not tested) $ cd /path/to/buildroot $ make O=/output blabla_defconfig $ make O=/output pkg-stats In any case: "pkg-stats -c" runs "make show-info", so it needs to be from a place where you can run Buildroot make targets. Does this clarifies the situation ? Thomas -- Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 4/4] utils/getdeveloperlib.py: use relative paths for files 2020-11-20 14:14 ` Thomas Petazzoni @ 2020-11-20 14:26 ` Heiko Thiery 0 siblings, 0 replies; 11+ messages in thread From: Heiko Thiery @ 2020-11-20 14:26 UTC (permalink / raw) To: buildroot Hi Thomas, Am Fr., 20. Nov. 2020 um 15:14 Uhr schrieb Thomas Petazzoni <thomas.petazzoni@bootlin.com>: > > Hello Heiko, > > Thanks for the review and feedback! > > On Fri, 20 Nov 2020 11:03:45 +0100 > Heiko Thiery <heiko.thiery@gmail.com> wrote: > > > I do as follows: > > > > --- >8 --- > > # mkdir b > > # cd b > > # make ../buildroot/ defconfig > > # ../buildroot/support/scripts/pkg-stats -c --json out.json > > make: *** Keine Regel, um ?show-info? zu erstellen. Schluss. > > The current folder must be a Builroot output directory: > > $ mkdir foobar/ > $ cd foobar/ > $ make -C /path/to/buildroot O=$(pwd) blabla_defconfig > $ make pkg-stats > or > $ /path/to/buildroot/support/scripts/pkg-stats -c --json out.json > > Or this should also work (but not tested) > > $ cd /path/to/buildroot > $ make O=/output blabla_defconfig > $ make O=/output pkg-stats > > In any case: "pkg-stats -c" runs "make show-info", so it needs to be > from a place where you can run Buildroot make targets. > > Does this clarifies the situation ? Oh yes .. that was my fault. I completely forgot to add the O=$PWD argument. Forget my previous mail ;-) -- Heiko ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/4] support/scripts/pkg-stats: import cve module only when needed 2020-11-19 14:53 [Buildroot] [PATCH 1/4] support/scripts/pkg-stats: import cve module only when needed Thomas Petazzoni ` (2 preceding siblings ...) 2020-11-19 14:53 ` [Buildroot] [PATCH 4/4] utils/getdeveloperlib.py: use relative paths for files Thomas Petazzoni @ 2020-11-20 10:02 ` Heiko Thiery 2021-01-17 14:32 ` Thomas Petazzoni 4 siblings, 0 replies; 11+ messages in thread From: Heiko Thiery @ 2020-11-20 10:02 UTC (permalink / raw) To: buildroot Hi Thomas, Am Do., 19. Nov. 2020 um 15:54 Uhr schrieb Thomas Petazzoni <thomas.petazzoni@bootlin.com>: > > The cve module needs ijson, which may not be installed. Since cve > matching is only enabled when --nvd-path is passed, it is a bit silly > to error out about ijson being missing if it's not used. > > So instead of unconditionally importing the cve module, only do it > conditionally. > > However, instead of doing it right at the point where it is used, we > do it at the beginning of the main() function. Indeed, if the cve > module is needed but cannot be imported, we want to error out > immediately rather than doing a whole bunch of things, and failing on > the user later on in the middle of the pkg-stats execution. > > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Tested-by: Heiko Thiery <heiko.thiery@gmail.com> > --- > support/scripts/pkg-stats | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats > index d44f8241c1..2676cc49dd 100755 > --- a/support/scripts/pkg-stats > +++ b/support/scripts/pkg-stats > @@ -32,7 +32,6 @@ brpath = os.path.normpath(os.path.join(os.path.dirname(__file__), "..", "..")) > > sys.path.append(os.path.join(brpath, "utils")) > from getdeveloperlib import parse_developers # noqa: E402 > -import cve as cvecheck # noqa: E402 > > > INFRA_RE = re.compile(r"\$\(eval \$\(([a-z-]*)-package\)\)") > @@ -950,7 +949,13 @@ def parse_args(): > > > def __main__(): > + global cvecheck > + > args = parse_args() > + > + if args.nvd_path: > + import cve as cvecheck > + > if args.packages: > package_list = args.packages.split(",") > elif args.configpackages: Thank you -- Heiko ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/4] support/scripts/pkg-stats: import cve module only when needed 2020-11-19 14:53 [Buildroot] [PATCH 1/4] support/scripts/pkg-stats: import cve module only when needed Thomas Petazzoni ` (3 preceding siblings ...) 2020-11-20 10:02 ` [Buildroot] [PATCH 1/4] support/scripts/pkg-stats: import cve module only when needed Heiko Thiery @ 2021-01-17 14:32 ` Thomas Petazzoni 2021-01-19 14:24 ` Peter Korsgaard 4 siblings, 1 reply; 11+ messages in thread From: Thomas Petazzoni @ 2021-01-17 14:32 UTC (permalink / raw) To: buildroot On Thu, 19 Nov 2020 15:53:51 +0100 Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote: > The cve module needs ijson, which may not be installed. Since cve > matching is only enabled when --nvd-path is passed, it is a bit silly > to error out about ijson being missing if it's not used. > > So instead of unconditionally importing the cve module, only do it > conditionally. > > However, instead of doing it right at the point where it is used, we > do it at the beginning of the main() function. Indeed, if the cve > module is needed but cannot be imported, we want to error out > immediately rather than doing a whole bunch of things, and failing on > the user later on in the middle of the pkg-stats execution. > > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> > --- > support/scripts/pkg-stats | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) Series applied. Thomas -- Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/4] support/scripts/pkg-stats: import cve module only when needed 2021-01-17 14:32 ` Thomas Petazzoni @ 2021-01-19 14:24 ` Peter Korsgaard 0 siblings, 0 replies; 11+ messages in thread From: Peter Korsgaard @ 2021-01-19 14:24 UTC (permalink / raw) To: buildroot >>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes: > On Thu, 19 Nov 2020 15:53:51 +0100 > Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote: >> The cve module needs ijson, which may not be installed. Since cve >> matching is only enabled when --nvd-path is passed, it is a bit silly >> to error out about ijson being missing if it's not used. >> >> So instead of unconditionally importing the cve module, only do it >> conditionally. >> >> However, instead of doing it right at the point where it is used, we >> do it at the beginning of the main() function. Indeed, if the cve >> module is needed but cannot be imported, we want to error out >> immediately rather than doing a whole bunch of things, and failing on >> the user later on in the middle of the pkg-stats execution. >> >> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> >> --- >> support/scripts/pkg-stats | 7 ++++++- >> 1 file changed, 6 insertions(+), 1 deletion(-) > Series applied. Committed to 2020.02.x and 2020.11.x, thanks. -- Bye, Peter Korsgaard ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2021-01-19 14:24 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-11-19 14:53 [Buildroot] [PATCH 1/4] support/scripts/pkg-stats: import cve module only when needed Thomas Petazzoni 2020-11-19 14:53 ` [Buildroot] [PATCH 2/4] support/scripts/pkg-stats: fix Python 3.8 deprecation warning Thomas Petazzoni 2020-11-19 14:53 ` [Buildroot] [PATCH 3/4] utils/get-developers: use Developers.hasfile() where appropriate Thomas Petazzoni 2020-11-20 10:03 ` Heiko Thiery 2020-11-19 14:53 ` [Buildroot] [PATCH 4/4] utils/getdeveloperlib.py: use relative paths for files Thomas Petazzoni 2020-11-20 10:03 ` Heiko Thiery 2020-11-20 14:14 ` Thomas Petazzoni 2020-11-20 14:26 ` Heiko Thiery 2020-11-20 10:02 ` [Buildroot] [PATCH 1/4] support/scripts/pkg-stats: import cve module only when needed Heiko Thiery 2021-01-17 14:32 ` Thomas Petazzoni 2021-01-19 14:24 ` Peter Korsgaard
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.