All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH 0/2] Add CVE reporting to pkg-stats
@ 2020-02-04 21:52 Thomas Petazzoni
  2020-02-04 21:52 ` [Buildroot] [PATCH 1/2] support/scripts/pkg-stats: add support for CVE reporting Thomas Petazzoni
                   ` (2 more replies)
  0 siblings, 3 replies; 11+ messages in thread
From: Thomas Petazzoni @ 2020-02-04 21:52 UTC (permalink / raw)
  To: buildroot

Hello,

This set of commit extends the pkg-stats tool to use the NVD database
(https://nvd.nist.gov/vuln/data-feeds) to see if the current version
of each Buildroot package is affected by a CVE.

An example result can be seen here:

 - Human readable HTML:       https://bootlin.com/~thomas/stats-cve.html
 - Machine parseable JSON:    https://bootlin.com/~thomas/stats-cve.json

Thanks to this, we can see that 60 of our packages are apparently
affected by a total of 185 CVEs.

A new per-package variable, <pkg>_IGNORE_CVES, is introduced, and
allows to tell the tool to ignore some CVEs, for example because it is
fixed by a local patch in Buildroot, or because the CVE does not apply
to the Buildroot package (the CVE only affects a non-Linux operating
system, or affect a functionality of the package that isn't built in
Buildroot).

Of course, the results are not perfect:

 - The NVD database product names certainly don't 100% match the
   Buildroot package names. We might have to add some extra metadata
   information in each package (CPE ID ?) to map to the correct NVD
   database product name.

 - Buildroot packages that have a version selection are not correctly
   handled.

But overall, it already provide useful results. The plan is of course
to implement e-mail notification to Buildroot developers in charge of
packages with unfixed CVEs, in a second step.

Thanks to Thomas DS and Titouan for all the help in the implementation
of this. We started at 2 PM today, and we have this first version to
show.

Thomas DS: I told you we could have something done by the end of the day!

Thomas

Thomas Petazzoni (2):
  support/scripts/pkg-stats: add support for CVE reporting
  docs/manual: describe the new <pkg>_IGNORE_CVES variable

 docs/manual/adding-packages-generic.txt |  14 +++
 support/scripts/pkg-stats               | 157 +++++++++++++++++++++++-
 2 files changed, 170 insertions(+), 1 deletion(-)

-- 
2.24.1

^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/2] support/scripts/pkg-stats: add support for CVE reporting
  2020-02-04 21:52 [Buildroot] [PATCH 0/2] Add CVE reporting to pkg-stats Thomas Petazzoni
@ 2020-02-04 21:52 ` Thomas Petazzoni
  2020-02-04 22:34   ` Matthew Weber
                     ` (2 more replies)
  2020-02-04 21:52 ` [Buildroot] [PATCH 2/2] docs/manual: describe the new <pkg>_IGNORE_CVES variable Thomas Petazzoni
  2020-02-04 22:32 ` [Buildroot] [PATCH 0/2] Add CVE reporting to pkg-stats Titouan Christophe
  2 siblings, 3 replies; 11+ messages in thread
From: Thomas Petazzoni @ 2020-02-04 21:52 UTC (permalink / raw)
  To: buildroot

This commit extends the pkg-stats script to grab information about the
CVEs affecting the Buildroot packages.

To do so, it downloads the NVD database from
https://nvd.nist.gov/vuln/data-feeds in JSON format, and processes the
JSON file to determine which of our packages is affected by which
CVE. The information is then displayed in both the HTML output and the
JSON output of pkg-stats.

To use this feature, you have to pass the new --nvd-path option,
pointing to a writable directory where pkg-stats will store the NVD
database. If the local database is less than 24 hours old, it will not
re-download it. If it is more than 24 hours old, it will re-download
only the files that have really been updated by upstream NVD.

Packages can use the newly introduced <pkg>_IGNORE_CVES variable to
tell pkg-stats that some CVEs should be ignored: it can be because a
patch we have is fixing the CVE, or because the CVE doesn't apply in
our case.

From an implementation point of view:

 - The download_nvd() and download_nvd_year() function implement the
   NVD database downloading.

 - The check_package_cves() function will go through all the CVE
   reports of the NVD database, which has one JSON file per year. For
   each CVE report it will check if we have a package with the same
   name in Buildroot. If we do, then
   check_package_cve_version_affected() verifies if the version in
   Buildroot is affected by the CVE.

 - The statistics are extended with the total number of CVEs, and the
   total number of packages that have at least one CVE pending.

 - The HTML output is extended with these new details. There are no
   changes to the code generating the JSON output because the existing
   code is smart enough to automatically expose the new information.

This development is a collective effort with Titouan Christophe
<titouan.christophe@railnova.eu> and Thomas De Schampheleire
<thomas.de_schampheleire@nokia.com>.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 support/scripts/pkg-stats | 157 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 156 insertions(+), 1 deletion(-)

diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats
index e477828f7b..23e630363b 100755
--- a/support/scripts/pkg-stats
+++ b/support/scripts/pkg-stats
@@ -26,10 +26,18 @@ import subprocess
 import requests  # URL checking
 import json
 import certifi
+import distutils.version
+import time
+import gzip
+import glob
 from urllib3 import HTTPSConnectionPool
 from urllib3.exceptions import HTTPError
 from multiprocessing import Pool
 
+NVD_START_YEAR = 2002
+NVD_JSON_VERSION = "1.0"
+NVD_BASE_URL = "https://nvd.nist.gov/feeds/json/cve/" + NVD_JSON_VERSION
+
 INFRA_RE = re.compile(r"\$\(eval \$\(([a-z-]*)-package\)\)")
 URL_RE = re.compile(r"\s*https?://\S*\s*$")
 
@@ -47,6 +55,7 @@ class Package:
     all_licenses = list()
     all_license_files = list()
     all_versions = dict()
+    all_ignored_cves = dict()
 
     def __init__(self, name, path):
         self.name = name
@@ -61,6 +70,7 @@ class Package:
         self.url = None
         self.url_status = None
         self.url_worker = None
+        self.cves = list()
         self.latest_version = (RM_API_STATUS_ERROR, None, None)
 
     def pkgvar(self):
@@ -152,6 +162,13 @@ class Package:
                 self.warnings = int(m.group(1))
                 return
 
+    def is_cve_ignored(self, cve):
+        """
+        Tells is the CVE is ignored by the package
+        """
+        return self.pkgvar() in self.all_ignored_cves and \
+            cve in self.all_ignored_cves[self.pkgvar()]
+
     def __eq__(self, other):
         return self.path == other.path
 
@@ -227,7 +244,7 @@ def get_pkglist(npackages, package_list):
 def package_init_make_info():
     # Fetch all variables at once
     variables = subprocess.check_output(["make", "BR2_HAVE_DOT_CONFIG=y", "-s", "printvars",
-                                         "VARS=%_LICENSE %_LICENSE_FILES %_VERSION"])
+                                         "VARS=%_LICENSE %_LICENSE_FILES %_VERSION %_IGNORE_CVES"])
     variable_list = variables.splitlines()
 
     # We process first the host package VERSION, and then the target
@@ -261,6 +278,10 @@ def package_init_make_info():
             pkgvar = pkgvar[:-8]
             Package.all_versions[pkgvar] = value
 
+        elif pkgvar.endswith("_IGNORE_CVES"):
+            pkgvar = pkgvar[:-12]
+            Package.all_ignored_cves[pkgvar] = value.split(" ")
+
 
 def check_url_status_worker(url, url_status):
     if url_status != "Missing" and url_status != "No Config.in":
@@ -355,6 +376,49 @@ def check_package_latest_version(packages):
     del http_pool
 
 
+def check_package_cve_version_affected(pkg, versions):
+    for v in versions:
+        if v["version_affected"] == "=":
+            return pkg.current_version == v["version_value"]
+        elif v["version_affected"] == "<=":
+            pkg_version = distutils.version.LooseVersion(pkg.current_version)
+            if not hasattr(pkg_version, "version"):
+                print("Cannot parse package '%s' version '%s'" % (pkg.name, pkg.current_version))
+                continue
+            cve_affected_version = distutils.version.LooseVersion(v["version_value"])
+            if not hasattr(cve_affected_version, "version"):
+                print("Cannot parse CVE affected version '%s'" % v["version_value"])
+                continue
+            return pkg_version < cve_affected_version
+
+    return False
+
+
+def check_package_cve_filter(packages, cve):
+    for vendor_datum in cve['cve']['affects']['vendor']['vendor_data']:
+        for product_datum in vendor_datum['product']['product_data']:
+            cve_pkg_name = product_datum['product_name']
+            for pkg in packages:
+                if cve_pkg_name == pkg.name:
+                    if check_package_cve_version_affected(pkg, product_datum["version"]["version_data"]):
+                        cveid = cve["cve"]["CVE_data_meta"]["ID"]
+                        if not pkg.is_cve_ignored(cveid):
+                            pkg.cves.append(cveid)
+
+
+def check_package_cves_year(packages, nvd_file):
+    cves = json.load(open(nvd_file))
+    for cve in cves["CVE_Items"]:
+        check_package_cve_filter(packages, cve)
+
+
+def check_package_cves(nvd_path, packages):
+    nvd_files = sorted(glob.glob(os.path.join(nvd_path, "nvdcve-%s-*.json" %
+                                              NVD_JSON_VERSION)))
+    for nvd_file in nvd_files:
+        check_package_cves_year(packages, nvd_file)
+
+
 def calculate_stats(packages):
     stats = defaultdict(int)
     for pkg in packages:
@@ -390,6 +454,9 @@ def calculate_stats(packages):
         else:
             stats["version-not-uptodate"] += 1
         stats["patches"] += pkg.patch_count
+        stats["total-cves"] += len(pkg.cves)
+        if len(pkg.cves) != 0:
+            stats["pkg-cves"] += 1
     return stats
 
 
@@ -601,6 +668,17 @@ def dump_html_pkg(f, pkg):
     f.write("  <td class=\"%s\">%s</td>\n" %
             (" ".join(td_class), url_str))
 
+    # CVEs
+    td_class = ["centered"]
+    if len(pkg.cves) == 0:
+        td_class.append("correct")
+    else:
+        td_class.append("wrong")
+    f.write("  <td class=\"%s\">\n" % " ".join(td_class))
+    for cve in pkg.cves:
+        f.write("   <a href=\"https://security-tracker.debian.org/tracker/%s\">%s<br/>\n" % (cve, cve))
+    f.write("  </td>\n")
+
     f.write(" </tr>\n")
 
 
@@ -618,6 +696,7 @@ def dump_html_all_pkgs(f, packages):
 <td class=\"centered\">Latest version</td>
 <td class=\"centered\">Warnings</td>
 <td class=\"centered\">Upstream URL</td>
+<td class=\"centered\">CVEs</td>
 </tr>
 """)
     for pkg in sorted(packages):
@@ -656,6 +735,10 @@ def dump_html_stats(f, stats):
             stats["version-not-uptodate"])
     f.write("<tr><td>Packages with no known upstream version</td><td>%s</td></tr>\n" %
             stats["version-unknown"])
+    f.write("<tr><td>Packages affected by CVEs</td><td>%s</td></tr>\n" %
+            stats["pkg-cves"])
+    f.write("<tr><td>Total number of CVEs affecting all packages</td><td>%s</td></tr>\n" %
+            stats["total-cves"])
     f.write("</table>\n")
 
 
@@ -702,6 +785,72 @@ def dump_json(packages, stats, date, commit, output):
         f.write('\n')
 
 
+def download_nvd_year(nvd_path, year):
+    metaf = "nvdcve-%s-%s.meta" % (NVD_JSON_VERSION, year)
+    path_metaf = os.path.join(nvd_path, metaf)
+
+    # If the meta file is less than a day old, we assume the NVD data
+    # locally available is recent enough.
+    if os.path.exists(path_metaf) and os.stat(path_metaf).st_mtime >= time.time() - 86400:
+        return
+
+    # If not, we download the meta file
+    url = "%s/%s" % (NVD_BASE_URL, metaf)
+    print("Getting %s" % url)
+    r = requests.get(url)
+    meta_new = dict(x.strip().split(':', 1) for x in r.text.splitlines())
+    if os.path.exists(path_metaf):
+        # If the meta file already existed, we compare the existing
+        # one with the data newly downloaded. If they are different,
+        # we need to re-download the database.
+        with open(path_metaf, "r") as f:
+            meta_old = dict(x.strip().split(':', 1) for x in f)
+        needs_download = meta_new != meta_old
+    else:
+        # If the meta file does not exist locally, of course we need
+        # to download the database
+        needs_download = True
+
+    # If we don't need to download the database, bail out
+    if not needs_download:
+        return
+
+    # Write the meta file, possibly overwriting the existing one
+    with open(path_metaf, "w") as f:
+        f.write(r.text)
+
+    # Grab the compressed JSON NVD database
+    jsonf = "nvdcve-%s-%s.json" % (NVD_JSON_VERSION, year)
+    jsonf_gz = jsonf + ".gz"
+    path_jsonf = os.path.join(nvd_path, jsonf)
+    path_jsonf_gz = os.path.join(nvd_path, jsonf_gz)
+    url = "%s/%s" % (NVD_BASE_URL, jsonf_gz)
+    print("Getting %s" % url)
+    r = requests.get(url)
+    with open(path_jsonf_gz, "wb") as f:
+        f.write(r.content)
+
+    # Uncompress and write it
+    gz = gzip.GzipFile(path_jsonf_gz)
+    with open(path_jsonf, "w") as f:
+        f.write(gz.read())
+
+    # Remove the temporary compressed file
+    os.unlink(path_jsonf_gz)
+
+
+def download_nvd(nvd_path):
+    if nvd_path is None:
+        print("NVD database path not provided, not checking CVEs")
+        return
+
+    if not os.path.isdir(nvd_path):
+        raise RuntimeError("NVD database path doesn't exist")
+
+    for year in range(NVD_START_YEAR, datetime.date.today().year + 1):
+        download_nvd_year(nvd_path, year)
+
+
 def parse_args():
     parser = argparse.ArgumentParser()
     output = parser.add_argument_group('output', 'Output file(s)')
@@ -714,6 +863,8 @@ def parse_args():
                           help='Number of packages')
     packages.add_argument('-p', dest='packages', action='store',
                           help='List of packages (comma separated)')
+    parser.add_argument('--nvd-path', dest='nvd_path',
+                        help='Path to the local NVD database')
     args = parser.parse_args()
     if not args.html and not args.json:
         parser.error('at least one of --html or --json (or both) is required')
@@ -729,6 +880,7 @@ def __main__():
     date = datetime.datetime.utcnow()
     commit = subprocess.check_output(['git', 'rev-parse',
                                       'HEAD']).splitlines()[0]
+    download_nvd(args.nvd_path)
     print("Build package list ...")
     packages = get_pkglist(args.npackages, package_list)
     print("Getting package make info ...")
@@ -746,6 +898,9 @@ def __main__():
     check_package_urls(packages)
     print("Getting latest versions ...")
     check_package_latest_version(packages)
+    if args.nvd_path:
+        print("CVE")
+        check_package_cves(args.nvd_path, packages)
     print("Calculate stats")
     stats = calculate_stats(packages)
     if args.html:
-- 
2.24.1

^ permalink raw reply related	[flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 2/2] docs/manual: describe the new <pkg>_IGNORE_CVES variable
  2020-02-04 21:52 [Buildroot] [PATCH 0/2] Add CVE reporting to pkg-stats Thomas Petazzoni
  2020-02-04 21:52 ` [Buildroot] [PATCH 1/2] support/scripts/pkg-stats: add support for CVE reporting Thomas Petazzoni
@ 2020-02-04 21:52 ` Thomas Petazzoni
  2020-02-04 22:59   ` Peter Korsgaard
  2020-02-05  5:53   ` Thomas De Schampheleire
  2020-02-04 22:32 ` [Buildroot] [PATCH 0/2] Add CVE reporting to pkg-stats Titouan Christophe
  2 siblings, 2 replies; 11+ messages in thread
From: Thomas Petazzoni @ 2020-02-04 21:52 UTC (permalink / raw)
  To: buildroot

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
 docs/manual/adding-packages-generic.txt | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt
index baa052e31c..9a77923a92 100644
--- a/docs/manual/adding-packages-generic.txt
+++ b/docs/manual/adding-packages-generic.txt
@@ -488,6 +488,20 @@ not and can not work as people would expect it should:
   locations, `/lib/firmware`, `/usr/lib/firmware`, `/lib/modules`,
   `/usr/lib/modules`, and `/usr/share`, which are automatically excluded.
 
+* +LIBFOO_IGNORE_CVES+ is a space-separated list of CVEs that tells
+  Buildroot CVE tracking tools which CVEs should be ignored for this
+  package. This is typically used when the CVE is fixed by a patch in
+  the package, or when the CVE for some reason does not affect the
+  Buildroot package. A Makefile comment must always preceed the
+  addition of a CVE to this variable. Example:
+
+----------------------
+# 0001-fix-cve-2020-12345.patch
+LIBFOO_IGNORE_CVES += CVE-2020-12345
+# only when built with libbaz, which Buildroot doesn't support
+LIBFOO_IGNORE_CVES += CVE-2020-54321
+----------------------
+
 The recommended way to define these variables is to use the following
 syntax:
 
-- 
2.24.1

^ permalink raw reply related	[flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 0/2] Add CVE reporting to pkg-stats
  2020-02-04 21:52 [Buildroot] [PATCH 0/2] Add CVE reporting to pkg-stats Thomas Petazzoni
  2020-02-04 21:52 ` [Buildroot] [PATCH 1/2] support/scripts/pkg-stats: add support for CVE reporting Thomas Petazzoni
  2020-02-04 21:52 ` [Buildroot] [PATCH 2/2] docs/manual: describe the new <pkg>_IGNORE_CVES variable Thomas Petazzoni
@ 2020-02-04 22:32 ` Titouan Christophe
  2020-02-04 22:36   ` Matthew Weber
  2 siblings, 1 reply; 11+ messages in thread
From: Titouan Christophe @ 2020-02-04 22:32 UTC (permalink / raw)
  To: buildroot

Hello Thomas^2 and all,

On 2/4/20 10:52 PM, Thomas Petazzoni wrote:
> Hello,
> 
> This set of commit extends the pkg-stats tool to use the NVD database
> (https://nvd.nist.gov/vuln/data-feeds) to see if the current version
> of each Buildroot package is affected by a CVE.
> 
> An example result can be seen here:
> 
>   - Human readable HTML:       https://bootlin.com/~thomas/stats-cve.html
>   - Machine parseable JSON:    https://bootlin.com/~thomas/stats-cve.json

Really great to see this landing !

> 
> Thanks to this, we can see that 60 of our packages are apparently
> affected by a total of 185 CVEs.
> 
> A new per-package variable, <pkg>_IGNORE_CVES, is introduced, and
> allows to tell the tool to ignore some CVEs, for example because it is
> fixed by a local patch in Buildroot, or because the CVE does not apply
> to the Buildroot package (the CVE only affects a non-Linux operating
> system, or affect a functionality of the package that isn't built in
> Buildroot).
> 
> Of course, the results are not perfect:
> 
>   - The NVD database product names certainly don't 100% match the
>     Buildroot package names. We might have to add some extra metadata
>     information in each package (CPE ID ?) to map to the correct NVD
>     database product name.
> 
>   - Buildroot packages that have a version selection are not correctly
>     handled.

In this latter case, we should maybe display a comment in the CVE column 
of the HTML report that says "CVE checking failed", because the 
"correct" CSS class could let us think that everything is fine while a 
package is on fire.

Probably bikeshed for this first iteration though.

> 
> But overall, it already provide useful results. The plan is of course
> to implement e-mail notification to Buildroot developers in charge of
> packages with unfixed CVEs, in a second step.
> 
> Thanks to Thomas DS and Titouan for all the help in the implementation
> of this. We started at 2 PM today, and we have this first version to
> show.
> 
> Thomas DS: I told you we could have something done by the end of the day!
> 
> Thomas
> 
> Thomas Petazzoni (2):
>    support/scripts/pkg-stats: add support for CVE reporting
>    docs/manual: describe the new <pkg>_IGNORE_CVES variable
> 
>   docs/manual/adding-packages-generic.txt |  14 +++
>   support/scripts/pkg-stats               | 157 +++++++++++++++++++++++-
>   2 files changed, 170 insertions(+), 1 deletion(-)
> 

I'll run once more through the code tomorrow morning with a fresh brain, 
but overall looks okay.


Best regards,

Titouan

^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/2] support/scripts/pkg-stats: add support for CVE reporting
  2020-02-04 21:52 ` [Buildroot] [PATCH 1/2] support/scripts/pkg-stats: add support for CVE reporting Thomas Petazzoni
@ 2020-02-04 22:34   ` Matthew Weber
  2020-02-04 22:56   ` Peter Korsgaard
  2020-02-05 10:48   ` Arnout Vandecappelle
  2 siblings, 0 replies; 11+ messages in thread
From: Matthew Weber @ 2020-02-04 22:34 UTC (permalink / raw)
  To: buildroot

All,


On Tue, Feb 4, 2020 at 3:53 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> This commit extends the pkg-stats script to grab information about the
> CVEs affecting the Buildroot packages.
>
> To do so, it downloads the NVD database from
> https://nvd.nist.gov/vuln/data-feeds in JSON format, and processes the
> JSON file to determine which of our packages is affected by which
> CVE. The information is then displayed in both the HTML output and the
> JSON output of pkg-stats.
>
> To use this feature, you have to pass the new --nvd-path option,
> pointing to a writable directory where pkg-stats will store the NVD
> database. If the local database is less than 24 hours old, it will not
> re-download it. If it is more than 24 hours old, it will re-download
> only the files that have really been updated by upstream NVD.
>
> Packages can use the newly introduced <pkg>_IGNORE_CVES variable to
> tell pkg-stats that some CVEs should be ignored: it can be because a
> patch we have is fixing the CVE, or because the CVE doesn't apply in
> our case.
>
> From an implementation point of view:
>
>  - The download_nvd() and download_nvd_year() function implement the
>    NVD database downloading.
>
>  - The check_package_cves() function will go through all the CVE
>    reports of the NVD database, which has one JSON file per year. For
>    each CVE report it will check if we have a package with the same
>    name in Buildroot. If we do, then
>    check_package_cve_version_affected() verifies if the version in
>    Buildroot is affected by the CVE.

Based on the previous CPE series iterations we were working to refine
the search criteria (pkg name and version) such that we could get more
exact and partial matches when you use them for a CVE search.  Do you
think there will be a package name to CVE package name translation
table required to improve matching?  I had done this in the package.mk
for CPE vendor/product fixing up to better match the CPE
dictionary....

>
>  - The statistics are extended with the total number of CVEs, and the
>    total number of packages that have at least one CVE pending.
>
>  - The HTML output is extended with these new details. There are no
>    changes to the code generating the JSON output because the existing
>    code is smart enough to automatically expose the new information.
>
> This development is a collective effort with Titouan Christophe
> <titouan.christophe@railnova.eu> and Thomas De Schampheleire
> <thomas.de_schampheleire@nokia.com>.
>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
>  support/scripts/pkg-stats | 157 +++++++++++++++++++++++++++++++++++++-
>  1 file changed, 156 insertions(+), 1 deletion(-)
>
> diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats
> index e477828f7b..23e630363b 100755
> --- a/support/scripts/pkg-stats
> +++ b/support/scripts/pkg-stats
> @@ -26,10 +26,18 @@ import subprocess
>  import requests  # URL checking
>  import json
>  import certifi
> +import distutils.version
> +import time
> +import gzip
> +import glob
>  from urllib3 import HTTPSConnectionPool
>  from urllib3.exceptions import HTTPError
>  from multiprocessing import Pool
>
> +NVD_START_YEAR = 2002
> +NVD_JSON_VERSION = "1.0"
> +NVD_BASE_URL = "https://nvd.nist.gov/feeds/json/cve/" + NVD_JSON_VERSION
> +
>  INFRA_RE = re.compile(r"\$\(eval \$\(([a-z-]*)-package\)\)")
>  URL_RE = re.compile(r"\s*https?://\S*\s*$")
>
> @@ -47,6 +55,7 @@ class Package:
>      all_licenses = list()
>      all_license_files = list()
>      all_versions = dict()
> +    all_ignored_cves = dict()
>
>      def __init__(self, name, path):
>          self.name = name
> @@ -61,6 +70,7 @@ class Package:
>          self.url = None
>          self.url_status = None
>          self.url_worker = None
> +        self.cves = list()
>          self.latest_version = (RM_API_STATUS_ERROR, None, None)
>
>      def pkgvar(self):
> @@ -152,6 +162,13 @@ class Package:
>                  self.warnings = int(m.group(1))
>                  return
>
> +    def is_cve_ignored(self, cve):
> +        """
> +        Tells is the CVE is ignored by the package
> +        """
> +        return self.pkgvar() in self.all_ignored_cves and \
> +            cve in self.all_ignored_cves[self.pkgvar()]
> +
>      def __eq__(self, other):
>          return self.path == other.path
>
> @@ -227,7 +244,7 @@ def get_pkglist(npackages, package_list):
>  def package_init_make_info():
>      # Fetch all variables at once
>      variables = subprocess.check_output(["make", "BR2_HAVE_DOT_CONFIG=y", "-s", "printvars",
> -                                         "VARS=%_LICENSE %_LICENSE_FILES %_VERSION"])
> +                                         "VARS=%_LICENSE %_LICENSE_FILES %_VERSION %_IGNORE_CVES"])
>      variable_list = variables.splitlines()
>
>      # We process first the host package VERSION, and then the target
> @@ -261,6 +278,10 @@ def package_init_make_info():
>              pkgvar = pkgvar[:-8]
>              Package.all_versions[pkgvar] = value
>
> +        elif pkgvar.endswith("_IGNORE_CVES"):
> +            pkgvar = pkgvar[:-12]
> +            Package.all_ignored_cves[pkgvar] = value.split(" ")
> +
>
>  def check_url_status_worker(url, url_status):
>      if url_status != "Missing" and url_status != "No Config.in":
> @@ -355,6 +376,49 @@ def check_package_latest_version(packages):
>      del http_pool
>
>
> +def check_package_cve_version_affected(pkg, versions):
> +    for v in versions:
> +        if v["version_affected"] == "=":
> +            return pkg.current_version == v["version_value"]
> +        elif v["version_affected"] == "<=":
> +            pkg_version = distutils.version.LooseVersion(pkg.current_version)
> +            if not hasattr(pkg_version, "version"):
> +                print("Cannot parse package '%s' version '%s'" % (pkg.name, pkg.current_version))
> +                continue
> +            cve_affected_version = distutils.version.LooseVersion(v["version_value"])
> +            if not hasattr(cve_affected_version, "version"):
> +                print("Cannot parse CVE affected version '%s'" % v["version_value"])
> +                continue
> +            return pkg_version < cve_affected_version
> +
> +    return False
> +
> +
> +def check_package_cve_filter(packages, cve):
> +    for vendor_datum in cve['cve']['affects']['vendor']['vendor_data']:
> +        for product_datum in vendor_datum['product']['product_data']:
> +            cve_pkg_name = product_datum['product_name']
> +            for pkg in packages:
> +                if cve_pkg_name == pkg.name:
> +                    if check_package_cve_version_affected(pkg, product_datum["version"]["version_data"]):
> +                        cveid = cve["cve"]["CVE_data_meta"]["ID"]
> +                        if not pkg.is_cve_ignored(cveid):
> +                            pkg.cves.append(cveid)
> +
> +
> +def check_package_cves_year(packages, nvd_file):
> +    cves = json.load(open(nvd_file))
> +    for cve in cves["CVE_Items"]:
> +        check_package_cve_filter(packages, cve)
> +
> +
> +def check_package_cves(nvd_path, packages):
> +    nvd_files = sorted(glob.glob(os.path.join(nvd_path, "nvdcve-%s-*.json" %
> +                                              NVD_JSON_VERSION)))
> +    for nvd_file in nvd_files:
> +        check_package_cves_year(packages, nvd_file)
> +
> +
>  def calculate_stats(packages):
>      stats = defaultdict(int)
>      for pkg in packages:
> @@ -390,6 +454,9 @@ def calculate_stats(packages):
>          else:
>              stats["version-not-uptodate"] += 1
>          stats["patches"] += pkg.patch_count
> +        stats["total-cves"] += len(pkg.cves)
> +        if len(pkg.cves) != 0:
> +            stats["pkg-cves"] += 1
>      return stats
>
>
> @@ -601,6 +668,17 @@ def dump_html_pkg(f, pkg):
>      f.write("  <td class=\"%s\">%s</td>\n" %
>              (" ".join(td_class), url_str))
>
> +    # CVEs
> +    td_class = ["centered"]
> +    if len(pkg.cves) == 0:
> +        td_class.append("correct")
> +    else:
> +        td_class.append("wrong")
> +    f.write("  <td class=\"%s\">\n" % " ".join(td_class))
> +    for cve in pkg.cves:
> +        f.write("   <a href=\"https://security-tracker.debian.org/tracker/%s\">%s<br/>\n" % (cve, cve))
> +    f.write("  </td>\n")
> +
>      f.write(" </tr>\n")
>
>
> @@ -618,6 +696,7 @@ def dump_html_all_pkgs(f, packages):
>  <td class=\"centered\">Latest version</td>
>  <td class=\"centered\">Warnings</td>
>  <td class=\"centered\">Upstream URL</td>
> +<td class=\"centered\">CVEs</td>
>  </tr>
>  """)
>      for pkg in sorted(packages):
> @@ -656,6 +735,10 @@ def dump_html_stats(f, stats):
>              stats["version-not-uptodate"])
>      f.write("<tr><td>Packages with no known upstream version</td><td>%s</td></tr>\n" %
>              stats["version-unknown"])
> +    f.write("<tr><td>Packages affected by CVEs</td><td>%s</td></tr>\n" %
> +            stats["pkg-cves"])
> +    f.write("<tr><td>Total number of CVEs affecting all packages</td><td>%s</td></tr>\n" %
> +            stats["total-cves"])
>      f.write("</table>\n")
>
>
> @@ -702,6 +785,72 @@ def dump_json(packages, stats, date, commit, output):
>          f.write('\n')
>
>
> +def download_nvd_year(nvd_path, year):
> +    metaf = "nvdcve-%s-%s.meta" % (NVD_JSON_VERSION, year)
> +    path_metaf = os.path.join(nvd_path, metaf)
> +
> +    # If the meta file is less than a day old, we assume the NVD data
> +    # locally available is recent enough.
> +    if os.path.exists(path_metaf) and os.stat(path_metaf).st_mtime >= time.time() - 86400:
> +        return
> +
> +    # If not, we download the meta file
> +    url = "%s/%s" % (NVD_BASE_URL, metaf)
> +    print("Getting %s" % url)
> +    r = requests.get(url)
> +    meta_new = dict(x.strip().split(':', 1) for x in r.text.splitlines())
> +    if os.path.exists(path_metaf):
> +        # If the meta file already existed, we compare the existing
> +        # one with the data newly downloaded. If they are different,
> +        # we need to re-download the database.
> +        with open(path_metaf, "r") as f:
> +            meta_old = dict(x.strip().split(':', 1) for x in f)
> +        needs_download = meta_new != meta_old
> +    else:
> +        # If the meta file does not exist locally, of course we need
> +        # to download the database
> +        needs_download = True
> +
> +    # If we don't need to download the database, bail out
> +    if not needs_download:
> +        return
> +
> +    # Write the meta file, possibly overwriting the existing one
> +    with open(path_metaf, "w") as f:
> +        f.write(r.text)
> +
> +    # Grab the compressed JSON NVD database
> +    jsonf = "nvdcve-%s-%s.json" % (NVD_JSON_VERSION, year)
> +    jsonf_gz = jsonf + ".gz"
> +    path_jsonf = os.path.join(nvd_path, jsonf)
> +    path_jsonf_gz = os.path.join(nvd_path, jsonf_gz)
> +    url = "%s/%s" % (NVD_BASE_URL, jsonf_gz)
> +    print("Getting %s" % url)
> +    r = requests.get(url)
> +    with open(path_jsonf_gz, "wb") as f:
> +        f.write(r.content)
> +
> +    # Uncompress and write it
> +    gz = gzip.GzipFile(path_jsonf_gz)
> +    with open(path_jsonf, "w") as f:
> +        f.write(gz.read())
> +
> +    # Remove the temporary compressed file
> +    os.unlink(path_jsonf_gz)
> +
> +
> +def download_nvd(nvd_path):
> +    if nvd_path is None:
> +        print("NVD database path not provided, not checking CVEs")
> +        return
> +
> +    if not os.path.isdir(nvd_path):
> +        raise RuntimeError("NVD database path doesn't exist")
> +
> +    for year in range(NVD_START_YEAR, datetime.date.today().year + 1):
> +        download_nvd_year(nvd_path, year)
> +
> +
>  def parse_args():
>      parser = argparse.ArgumentParser()
>      output = parser.add_argument_group('output', 'Output file(s)')
> @@ -714,6 +863,8 @@ def parse_args():
>                            help='Number of packages')
>      packages.add_argument('-p', dest='packages', action='store',
>                            help='List of packages (comma separated)')
> +    parser.add_argument('--nvd-path', dest='nvd_path',
> +                        help='Path to the local NVD database')
>      args = parser.parse_args()
>      if not args.html and not args.json:
>          parser.error('at least one of --html or --json (or both) is required')
> @@ -729,6 +880,7 @@ def __main__():
>      date = datetime.datetime.utcnow()
>      commit = subprocess.check_output(['git', 'rev-parse',
>                                        'HEAD']).splitlines()[0]
> +    download_nvd(args.nvd_path)
>      print("Build package list ...")
>      packages = get_pkglist(args.npackages, package_list)
>      print("Getting package make info ...")
> @@ -746,6 +898,9 @@ def __main__():
>      check_package_urls(packages)
>      print("Getting latest versions ...")
>      check_package_latest_version(packages)
> +    if args.nvd_path:
> +        print("CVE")
> +        check_package_cves(args.nvd_path, packages)
>      print("Calculate stats")
>      stats = calculate_stats(packages)
>      if args.html:
> --
> 2.24.1
>


-- 

Matthew Weber | Associate Director Software Engineer | Commercial Avionics

COLLINS AEROSPACE

400 Collins Road NE, Cedar Rapids, Iowa 52498, USA

Tel: +1 319 295 7349 | FAX: +1 319 263 6099

matthew.weber at collins.com | collinsaerospace.com



CONFIDENTIALITY WARNING: This message may contain proprietary and/or
privileged information of Collins Aerospace and its affiliated
companies. If you are not the intended recipient, please 1) Do not
disclose, copy, distribute or use this message or its contents. 2)
Advise the sender by return email. 3) Delete all copies (including all
attachments) from your computer. Your cooperation is greatly
appreciated.


Any export restricted material should be shared using my
matthew.weber at corp.rockwellcollins.com address.

^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 0/2] Add CVE reporting to pkg-stats
  2020-02-04 22:32 ` [Buildroot] [PATCH 0/2] Add CVE reporting to pkg-stats Titouan Christophe
@ 2020-02-04 22:36   ` Matthew Weber
  0 siblings, 0 replies; 11+ messages in thread
From: Matthew Weber @ 2020-02-04 22:36 UTC (permalink / raw)
  To: buildroot

On Tue, Feb 4, 2020 at 4:32 PM Titouan Christophe
<titouan.christophe@railnova.eu> wrote:
>
> Hello Thomas^2 and all,
>
> On 2/4/20 10:52 PM, Thomas Petazzoni wrote:
> > Hello,
> >
> > This set of commit extends the pkg-stats tool to use the NVD database
> > (https://nvd.nist.gov/vuln/data-feeds) to see if the current version
> > of each Buildroot package is affected by a CVE.

Would be a neat feature to also have ran against LTS

> >
> > An example result can be seen here:
> >
> >   - Human readable HTML:       https://bootlin.com/~thomas/stats-cve.html
> >   - Machine parseable JSON:    https://bootlin.com/~thomas/stats-cve.json
>
> Really great to see this landing !
>
> >
> > Thanks to this, we can see that 60 of our packages are apparently
> > affected by a total of 185 CVEs.
> >
> > A new per-package variable, <pkg>_IGNORE_CVES, is introduced, and
> > allows to tell the tool to ignore some CVEs, for example because it is
> > fixed by a local patch in Buildroot, or because the CVE does not apply
> > to the Buildroot package (the CVE only affects a non-Linux operating
> > system, or affect a functionality of the package that isn't built in
> > Buildroot).
> >
> > Of course, the results are not perfect:
> >
> >   - The NVD database product names certainly don't 100% match the
> >     Buildroot package names. We might have to add some extra metadata
> >     information in each package (CPE ID ?) to map to the correct NVD
> >     database product name.
> >
> >   - Buildroot packages that have a version selection are not correctly
> >     handled.
>
> In this latter case, we should maybe display a comment in the CVE column
> of the HTML report that says "CVE checking failed", because the
> "correct" CSS class could let us think that everything is fine while a
> package is on fire.
>
> Probably bikeshed for this first iteration though.
>
> >
> > But overall, it already provide useful results. The plan is of course
> > to implement e-mail notification to Buildroot developers in charge of
> > packages with unfixed CVEs, in a second step.
> >
> > Thanks to Thomas DS and Titouan for all the help in the implementation
> > of this. We started at 2 PM today, and we have this first version to
> > show.
> >
> > Thomas DS: I told you we could have something done by the end of the day!
> >
> > Thomas
> >
> > Thomas Petazzoni (2):
> >    support/scripts/pkg-stats: add support for CVE reporting
> >    docs/manual: describe the new <pkg>_IGNORE_CVES variable
> >
> >   docs/manual/adding-packages-generic.txt |  14 +++
> >   support/scripts/pkg-stats               | 157 +++++++++++++++++++++++-
> >   2 files changed, 170 insertions(+), 1 deletion(-)
> >
>
> I'll run once more through the code tomorrow morning with a fresh brain,
> but overall looks okay.
>
>
> Best regards,
>
> Titouan



-- 

Matthew Weber | Associate Director Software Engineer | Commercial Avionics

COLLINS AEROSPACE

400 Collins Road NE, Cedar Rapids, Iowa 52498, USA

Tel: +1 319 295 7349 | FAX: +1 319 263 6099

matthew.weber at collins.com | collinsaerospace.com



CONFIDENTIALITY WARNING: This message may contain proprietary and/or
privileged information of Collins Aerospace and its affiliated
companies. If you are not the intended recipient, please 1) Do not
disclose, copy, distribute or use this message or its contents. 2)
Advise the sender by return email. 3) Delete all copies (including all
attachments) from your computer. Your cooperation is greatly
appreciated.


Any export restricted material should be shared using my
matthew.weber at corp.rockwellcollins.com address.

^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/2] support/scripts/pkg-stats: add support for CVE reporting
  2020-02-04 21:52 ` [Buildroot] [PATCH 1/2] support/scripts/pkg-stats: add support for CVE reporting Thomas Petazzoni
  2020-02-04 22:34   ` Matthew Weber
@ 2020-02-04 22:56   ` Peter Korsgaard
  2020-02-05 10:48   ` Arnout Vandecappelle
  2 siblings, 0 replies; 11+ messages in thread
From: Peter Korsgaard @ 2020-02-04 22:56 UTC (permalink / raw)
  To: buildroot

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > This commit extends the pkg-stats script to grab information about the
 > CVEs affecting the Buildroot packages.

 > To do so, it downloads the NVD database from
 > https://nvd.nist.gov/vuln/data-feeds in JSON format, and processes the
 > JSON file to determine which of our packages is affected by which
 > CVE. The information is then displayed in both the HTML output and the
 > JSON output of pkg-stats.

 > To use this feature, you have to pass the new --nvd-path option,
 > pointing to a writable directory where pkg-stats will store the NVD
 > database. If the local database is less than 24 hours old, it will not
 > re-download it. If it is more than 24 hours old, it will re-download
 > only the files that have really been updated by upstream NVD.

 > Packages can use the newly introduced <pkg>_IGNORE_CVES variable to
 > tell pkg-stats that some CVEs should be ignored: it can be because a
 > patch we have is fixing the CVE, or because the CVE doesn't apply in
 > our case.

 > From an implementation point of view:

 >  - The download_nvd() and download_nvd_year() function implement the
 >    NVD database downloading.

 >  - The check_package_cves() function will go through all the CVE
 >    reports of the NVD database, which has one JSON file per year. For
 >    each CVE report it will check if we have a package with the same
 >    name in Buildroot. If we do, then
 >    check_package_cve_version_affected() verifies if the version in
 >    Buildroot is affected by the CVE.

 >  - The statistics are extended with the total number of CVEs, and the
 >    total number of packages that have at least one CVE pending.

 >  - The HTML output is extended with these new details. There are no
 >    changes to the code generating the JSON output because the existing
 >    code is smart enough to automatically expose the new information.

 > This development is a collective effort with Titouan Christophe
 > <titouan.christophe@railnova.eu> and Thomas De Schampheleire
 > <thomas.de_schampheleire@nokia.com>.

Neat!


 > +NVD_START_YEAR = 2002
 > +NVD_JSON_VERSION = "1.0"

Looking at https://nvd.nist.gov/vuln/data-feeds#JSON_FEED I see a 1.1
version instead of 1.0?


 > +def check_package_cves_year(packages, nvd_file):
 > +    cves = json.load(open(nvd_file))
 > +    for cve in cves["CVE_Items"]:
 > +        check_package_cve_filter(packages, cve)
 > +
 > +
 > +def check_package_cves(nvd_path, packages):
 > +    nvd_files = sorted(glob.glob(os.path.join(nvd_path, "nvdcve-%s-*.json" %
 > +                                              NVD_JSON_VERSION)))
 > +    for nvd_file in nvd_files:
 > +        check_package_cves_year(packages, nvd_file)

NIT: It is a bit confusing to swap the argument order between
check_package_cves() and check_packages_cves_year().


 > +def download_nvd_year(nvd_path, year):
 > +    metaf = "nvdcve-%s-%s.meta" % (NVD_JSON_VERSION, year)
 > +    path_metaf = os.path.join(nvd_path, metaf)
 > +
 > +    # If the meta file is less than a day old, we assume the NVD data
 > +    # locally available is recent enough.
 > +    if os.path.exists(path_metaf) and os.stat(path_metaf).st_mtime >= time.time() - 86400:
 > +        return

This could be swapped around to check for the json file so it doesn't
break if this is stopped between downloading the meta and json file.


 > +    # If not, we download the meta file
 > +    url = "%s/%s" % (NVD_BASE_URL, metaf)
 > +    print("Getting %s" % url)
 > +    r = requests.get(url)
 > +    meta_new = dict(x.strip().split(':', 1) for x in r.text.splitlines())
 > +    if os.path.exists(path_metaf):
 > +        # If the meta file already existed, we compare the existing
 > +        # one with the data newly downloaded. If they are different,
 > +        # we need to re-download the database.
 > +        with open(path_metaf, "r") as f:
 > +            meta_old = dict(x.strip().split(':', 1) for x in f)
 > +        needs_download = meta_new != meta_old

Is there any reason for doing the dict stuff instead of just comparing
the old file with the new one directly? We don't seem to really use the
content of the meta file for anything, right?


 > +    else:
 > +        # If the meta file does not exist locally, of course we need
 > +        # to download the database
 > +        needs_download = True
 > +
 > +    # If we don't need to download the database, bail out
 > +    if not needs_download:
 > +        return
 > +
 > +    # Write the meta file, possibly overwriting the existing one
 > +    with open(path_metaf, "w") as f:
 > +        f.write(r.text)
 > +
 > +    # Grab the compressed JSON NVD database
 > +    jsonf = "nvdcve-%s-%s.json" % (NVD_JSON_VERSION, year)
 > +    jsonf_gz = jsonf + ".gz"
 > +    path_jsonf = os.path.join(nvd_path, jsonf)
 > +    path_jsonf_gz = os.path.join(nvd_path, jsonf_gz)
 > +    url = "%s/%s" % (NVD_BASE_URL, jsonf_gz)
 > +    print("Getting %s" % url)
 > +    r = requests.get(url)
 > +    with open(path_jsonf_gz, "wb") as f:
 > +        f.write(r.content)
 > +
 > +    # Uncompress and write it
 > +    gz = gzip.GzipFile(path_jsonf_gz)
 > +    with open(path_jsonf, "w") as f:
 > +        f.write(gz.read())

We could also just use gzip.open() to read the .json.gz files later on
and save some disk space by never expanding the files on disk.

If we want to completely get rid of race conditions, we probably need to
write to a temporary file / sync / rename.

-- 
Bye, Peter Korsgaard

^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 2/2] docs/manual: describe the new <pkg>_IGNORE_CVES variable
  2020-02-04 21:52 ` [Buildroot] [PATCH 2/2] docs/manual: describe the new <pkg>_IGNORE_CVES variable Thomas Petazzoni
@ 2020-02-04 22:59   ` Peter Korsgaard
  2020-02-05  5:53   ` Thomas De Schampheleire
  1 sibling, 0 replies; 11+ messages in thread
From: Peter Korsgaard @ 2020-02-04 22:59 UTC (permalink / raw)
  To: buildroot

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
 > ---
 >  docs/manual/adding-packages-generic.txt | 14 ++++++++++++++
 >  1 file changed, 14 insertions(+)

 > diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt
 > index baa052e31c..9a77923a92 100644
 > --- a/docs/manual/adding-packages-generic.txt
 > +++ b/docs/manual/adding-packages-generic.txt
 > @@ -488,6 +488,20 @@ not and can not work as people would expect it should:
 >    locations, `/lib/firmware`, `/usr/lib/firmware`, `/lib/modules`,
 >    `/usr/lib/modules`, and `/usr/share`, which are automatically excluded.
 
 > +* +LIBFOO_IGNORE_CVES+ is a space-separated list of CVEs that tells

Maybe 'CVE identifiers' instead of CVEs?

 > +  Buildroot CVE tracking tools which CVEs should be ignored for this

Maybe '.. list of CVE identifiers that should be ignored by the
Buildroot CVE trackign tool for this package.' ?

-- 
Bye, Peter Korsgaard

^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 2/2] docs/manual: describe the new <pkg>_IGNORE_CVES variable
  2020-02-04 21:52 ` [Buildroot] [PATCH 2/2] docs/manual: describe the new <pkg>_IGNORE_CVES variable Thomas Petazzoni
  2020-02-04 22:59   ` Peter Korsgaard
@ 2020-02-05  5:53   ` Thomas De Schampheleire
  1 sibling, 0 replies; 11+ messages in thread
From: Thomas De Schampheleire @ 2020-02-05  5:53 UTC (permalink / raw)
  To: buildroot

On Tue, Feb 4, 2020, 22:52 Thomas Petazzoni <thomas.petazzoni@bootlin.com>
wrote:

> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
>  docs/manual/adding-packages-generic.txt | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
>
> diff --git a/docs/manual/adding-packages-generic.txt
> b/docs/manual/adding-packages-generic.txt
> index baa052e31c..9a77923a92 100644
> --- a/docs/manual/adding-packages-generic.txt
> +++ b/docs/manual/adding-packages-generic.txt
> @@ -488,6 +488,20 @@ not and can not work as people would expect it should:
>    locations, `/lib/firmware`, `/usr/lib/firmware`, `/lib/modules`,
>    `/usr/lib/modules`, and `/usr/share`, which are automatically excluded.
>
> +* +LIBFOO_IGNORE_CVES+ is a space-separated list of CVEs that tells
> +  Buildroot CVE tracking tools which CVEs should be ignored for this
> +  package. This is typically used when the CVE is fixed by a patch in
> +  the package, or when the CVE for some reason does not affect the
> +  Buildroot package. A Makefile comment must always preceed the
>

Precede

+  addition of a CVE to this variable. Example:
> +
> +----------------------
> +# 0001-fix-cve-2020-12345.patch
> +LIBFOO_IGNORE_CVES += CVE-2020-12345
> +# only when built with libbaz, which Buildroot doesn't support
> +LIBFOO_IGNORE_CVES += CVE-2020-54321
> +----------------------
> +
>  The recommended way to define these variables is to use the following
>  syntax:
>
> --
> 2.24.1
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200205/fc7d4f89/attachment.html>

^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/2] support/scripts/pkg-stats: add support for CVE reporting
  2020-02-04 21:52 ` [Buildroot] [PATCH 1/2] support/scripts/pkg-stats: add support for CVE reporting Thomas Petazzoni
  2020-02-04 22:34   ` Matthew Weber
  2020-02-04 22:56   ` Peter Korsgaard
@ 2020-02-05 10:48   ` Arnout Vandecappelle
  2020-02-13 22:03     ` Peter Korsgaard
  2 siblings, 1 reply; 11+ messages in thread
From: Arnout Vandecappelle @ 2020-02-05 10:48 UTC (permalink / raw)
  To: buildroot



On 04/02/2020 22:52, Thomas Petazzoni wrote:
> This commit extends the pkg-stats script to grab information about the
> CVEs affecting the Buildroot packages.
> 
> To do so, it downloads the NVD database from
> https://nvd.nist.gov/vuln/data-feeds in JSON format, and processes the
> JSON file to determine which of our packages is affected by which
> CVE. The information is then displayed in both the HTML output and the
> JSON output of pkg-stats.
> 
> To use this feature, you have to pass the new --nvd-path option,
> pointing to a writable directory where pkg-stats will store the NVD
> database. If the local database is less than 24 hours old, it will not
> re-download it. If it is more than 24 hours old, it will re-download
> only the files that have really been updated by upstream NVD.
> 
> Packages can use the newly introduced <pkg>_IGNORE_CVES variable to
> tell pkg-stats that some CVEs should be ignored: it can be because a
> patch we have is fixing the CVE, or because the CVE doesn't apply in
> our case.
> 
> From an implementation point of view:
> 
>  - The download_nvd() and download_nvd_year() function implement the
>    NVD database downloading.
> 
>  - The check_package_cves() function will go through all the CVE
>    reports of the NVD database, which has one JSON file per year. For
>    each CVE report it will check if we have a package with the same
>    name in Buildroot. If we do, then
>    check_package_cve_version_affected() verifies if the version in
>    Buildroot is affected by the CVE.
> 
>  - The statistics are extended with the total number of CVEs, and the
>    total number of packages that have at least one CVE pending.
> 
>  - The HTML output is extended with these new details. There are no
>    changes to the code generating the JSON output because the existing
>    code is smart enough to automatically expose the new information.
> 
> This development is a collective effort with Titouan Christophe
> <titouan.christophe@railnova.eu> and Thomas De Schampheleire
> <thomas.de_schampheleire@nokia.com>.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ---
>  support/scripts/pkg-stats | 157 +++++++++++++++++++++++++++++++++++++-
>  1 file changed, 156 insertions(+), 1 deletion(-)
> 
> diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats
> index e477828f7b..23e630363b 100755
> --- a/support/scripts/pkg-stats
> +++ b/support/scripts/pkg-stats
> @@ -26,10 +26,18 @@ import subprocess
>  import requests  # URL checking
>  import json
>  import certifi
> +import distutils.version
> +import time
> +import gzip
> +import glob
>  from urllib3 import HTTPSConnectionPool
>  from urllib3.exceptions import HTTPError
>  from multiprocessing import Pool
>  
> +NVD_START_YEAR = 2002
> +NVD_JSON_VERSION = "1.0"
> +NVD_BASE_URL = "https://nvd.nist.gov/feeds/json/cve/" + NVD_JSON_VERSION
> +
>  INFRA_RE = re.compile(r"\$\(eval \$\(([a-z-]*)-package\)\)")
>  URL_RE = re.compile(r"\s*https?://\S*\s*$")
>  
> @@ -47,6 +55,7 @@ class Package:
>      all_licenses = list()
>      all_license_files = list()
>      all_versions = dict()
> +    all_ignored_cves = dict()
>  
>      def __init__(self, name, path):
>          self.name = name
> @@ -61,6 +70,7 @@ class Package:
>          self.url = None
>          self.url_status = None
>          self.url_worker = None
> +        self.cves = list()
>          self.latest_version = (RM_API_STATUS_ERROR, None, None)
>  
>      def pkgvar(self):
> @@ -152,6 +162,13 @@ class Package:
>                  self.warnings = int(m.group(1))
>                  return
>  
> +    def is_cve_ignored(self, cve):
> +        """
> +        Tells is the CVE is ignored by the package

 Tells *if* ?

> +        """
> +        return self.pkgvar() in self.all_ignored_cves and \
> +            cve in self.all_ignored_cves[self.pkgvar()]

 A more pythonic way is:

return cve in self.all_ignored_cves.get(self.pkgvar(), [])


> +
>      def __eq__(self, other):
>          return self.path == other.path
>  
> @@ -227,7 +244,7 @@ def get_pkglist(npackages, package_list):
>  def package_init_make_info():
>      # Fetch all variables at once
>      variables = subprocess.check_output(["make", "BR2_HAVE_DOT_CONFIG=y", "-s", "printvars",
> -                                         "VARS=%_LICENSE %_LICENSE_FILES %_VERSION"])
> +                                         "VARS=%_LICENSE %_LICENSE_FILES %_VERSION %_IGNORE_CVES"])
>      variable_list = variables.splitlines()
>  
>      # We process first the host package VERSION, and then the target
> @@ -261,6 +278,10 @@ def package_init_make_info():
>              pkgvar = pkgvar[:-8]
>              Package.all_versions[pkgvar] = value
>  
> +        elif pkgvar.endswith("_IGNORE_CVES"):
> +            pkgvar = pkgvar[:-12]
> +            Package.all_ignored_cves[pkgvar] = value.split(" ")
> +
>  
>  def check_url_status_worker(url, url_status):
>      if url_status != "Missing" and url_status != "No Config.in":
> @@ -355,6 +376,49 @@ def check_package_latest_version(packages):
>      del http_pool
>  
>  
> +def check_package_cve_version_affected(pkg, versions):

 This could use some documenation, in particular of what the pkg and versions
arguments are.

> +    for v in versions:
> +        if v["version_affected"] == "=":
> +            return pkg.current_version == v["version_value"]
> +        elif v["version_affected"] == "<=":

 Isn't there a >= we need to check as well?

> +            pkg_version = distutils.version.LooseVersion(pkg.current_version)
> +            if not hasattr(pkg_version, "version"):
> +                print("Cannot parse package '%s' version '%s'" % (pkg.name, pkg.current_version))
> +                continue
> +            cve_affected_version = distutils.version.LooseVersion(v["version_value"])
> +            if not hasattr(cve_affected_version, "version"):
> +                print("Cannot parse CVE affected version '%s'" % v["version_value"])
> +                continue
> +            return pkg_version < cve_affected_version

 Maybe use packaging.version instead? [1]

[1]
https://stackoverflow.com/questions/11887762/how-do-i-compare-version-numbers-in-python

 Hm, but the packaging module is not installed on my system (which does have
setuptools), so maybe not...

> +
> +    return False
> +
> +
> +def check_package_cve_filter(packages, cve):
> +    for vendor_datum in cve['cve']['affects']['vendor']['vendor_data']:
> +        for product_datum in vendor_datum['product']['product_data']:
> +            cve_pkg_name = product_datum['product_name']
> +            for pkg in packages:
> +                if cve_pkg_name == pkg.name:

 I think it would be useful (as in, speed things up) to convert the package list
into a dict at the top level, because then here we can just do

pkg = packages.get(cve_pkg_name):
if pkg:
   if check_package_cve_version_affected(pkg, ...):

 The dictionary would be created like this:

cve_packages = {pkg.name: pkg for pkg in packages}

And if we ever support a _CVENAME in the .mk file to improve the mapping, it
would become:

cve_packages = {pkg.cvename or pkg.name: pkg for pkg in packages}

> +                    if check_package_cve_version_affected(pkg, product_datum["version"]["version_data"]):
> +                        cveid = cve["cve"]["CVE_data_meta"]["ID"]
> +                        if not pkg.is_cve_ignored(cveid):
> +                            pkg.cves.append(cveid)
> +
> +
> +def check_package_cves_year(packages, nvd_file):

 AFAIU, the "year" part here is not really relevant. It just happens to be that
the json files are organised per year. So I'd just call it check_package_cves_file.

> +    cves = json.load(open(nvd_file))
> +    for cve in cves["CVE_Items"]:
> +        check_package_cve_filter(packages, cve)
> +
> +
> +def check_package_cves(nvd_path, packages):
> +    nvd_files = sorted(glob.glob(os.path.join(nvd_path, "nvdcve-%s-*.json" %

 Maybe it would be better if the nvd_download step would make an explicit list
of json files.

> +                                              NVD_JSON_VERSION)))
> +    for nvd_file in nvd_files:
> +        check_package_cves_year(packages, nvd_file)
> +
> +
>  def calculate_stats(packages):
>      stats = defaultdict(int)
>      for pkg in packages:
> @@ -390,6 +454,9 @@ def calculate_stats(packages):
>          else:
>              stats["version-not-uptodate"] += 1
>          stats["patches"] += pkg.patch_count
> +        stats["total-cves"] += len(pkg.cves)
> +        if len(pkg.cves) != 0:
> +            stats["pkg-cves"] += 1
>      return stats
>  
>  
> @@ -601,6 +668,17 @@ def dump_html_pkg(f, pkg):
>      f.write("  <td class=\"%s\">%s</td>\n" %
>              (" ".join(td_class), url_str))
>  
> +    # CVEs
> +    td_class = ["centered"]
> +    if len(pkg.cves) == 0:
> +        td_class.append("correct")
> +    else:
> +        td_class.append("wrong")
> +    f.write("  <td class=\"%s\">\n" % " ".join(td_class))
> +    for cve in pkg.cves:
> +        f.write("   <a href=\"https://security-tracker.debian.org/tracker/%s\">%s<br/>\n" % (cve, cve))
> +    f.write("  </td>\n")
> +
>      f.write(" </tr>\n")
>  
>  
> @@ -618,6 +696,7 @@ def dump_html_all_pkgs(f, packages):
>  <td class=\"centered\">Latest version</td>
>  <td class=\"centered\">Warnings</td>
>  <td class=\"centered\">Upstream URL</td>
> +<td class=\"centered\">CVEs</td>
>  </tr>
>  """)
>      for pkg in sorted(packages):
> @@ -656,6 +735,10 @@ def dump_html_stats(f, stats):
>              stats["version-not-uptodate"])
>      f.write("<tr><td>Packages with no known upstream version</td><td>%s</td></tr>\n" %
>              stats["version-unknown"])
> +    f.write("<tr><td>Packages affected by CVEs</td><td>%s</td></tr>\n" %
> +            stats["pkg-cves"])
> +    f.write("<tr><td>Total number of CVEs affecting all packages</td><td>%s</td></tr>\n" %
> +            stats["total-cves"])

 Shouldn't this be conditional on whether we were checking CVEs? Same for the title.

>      f.write("</table>\n")
>  
>  
> @@ -702,6 +785,72 @@ def dump_json(packages, stats, date, commit, output):
>          f.write('\n')
>  
>  
> +def download_nvd_year(nvd_path, year):
> +    metaf = "nvdcve-%s-%s.meta" % (NVD_JSON_VERSION, year)
> +    path_metaf = os.path.join(nvd_path, metaf)
> +
> +    # If the meta file is less than a day old, we assume the NVD data
> +    # locally available is recent enough.
> +    if os.path.exists(path_metaf) and os.stat(path_metaf).st_mtime >= time.time() - 86400:
> +        return
> +
> +    # If not, we download the meta file
> +    url = "%s/%s" % (NVD_BASE_URL, metaf)
> +    print("Getting %s" % url)
> +    r = requests.get(url)
> +    meta_new = dict(x.strip().split(':', 1) for x in r.text.splitlines())
> +    if os.path.exists(path_metaf):
> +        # If the meta file already existed, we compare the existing
> +        # one with the data newly downloaded. If they are different,
> +        # we need to re-download the database.
> +        with open(path_metaf, "r") as f:
> +            meta_old = dict(x.strip().split(':', 1) for x in f)
> +        needs_download = meta_new != meta_old

 Since the meta is only used for comparing new and old, why not simply

meta_old = f.read()
needs_download = r != meta_old

?

> +    else:
> +        # If the meta file does not exist locally, of course we need
> +        # to download the database
> +        needs_download = True
> +
> +    # If we don't need to download the database, bail out
> +    if not needs_download:
> +        return
> +
> +    # Write the meta file, possibly overwriting the existing one
> +    with open(path_metaf, "w") as f:
> +        f.write(r.text)
> +
> +    # Grab the compressed JSON NVD database
> +    jsonf = "nvdcve-%s-%s.json" % (NVD_JSON_VERSION, year)
> +    jsonf_gz = jsonf + ".gz"
> +    path_jsonf = os.path.join(nvd_path, jsonf)
> +    path_jsonf_gz = os.path.join(nvd_path, jsonf_gz)
> +    url = "%s/%s" % (NVD_BASE_URL, jsonf_gz)
> +    print("Getting %s" % url)
> +    r = requests.get(url)
> +    with open(path_jsonf_gz, "wb") as f:
> +        f.write(r.content)
> +
> +    # Uncompress and write it
> +    gz = gzip.GzipFile(path_jsonf_gz)
> +    with open(path_jsonf, "w") as f:
> +        f.write(gz.read())
> +
> +    # Remove the temporary compressed file
> +    os.unlink(path_jsonf_gz)

 Like Peter, I think it's better to store the gzipped file and unzip when
reading the json.

> +
> +
> +def download_nvd(nvd_path):
> +    if nvd_path is None:
> +        print("NVD database path not provided, not checking CVEs")
> +        return
> +
> +    if not os.path.isdir(nvd_path):
> +        raise RuntimeError("NVD database path doesn't exist")
> +
> +    for year in range(NVD_START_YEAR, datetime.date.today().year + 1):
> +        download_nvd_year(nvd_path, year)
> +
> +
>  def parse_args():
>      parser = argparse.ArgumentParser()
>      output = parser.add_argument_group('output', 'Output file(s)')
> @@ -714,6 +863,8 @@ def parse_args():
>                            help='Number of packages')
>      packages.add_argument('-p', dest='packages', action='store',
>                            help='List of packages (comma separated)')
> +    parser.add_argument('--nvd-path', dest='nvd_path',
> +                        help='Path to the local NVD database')
>      args = parser.parse_args()
>      if not args.html and not args.json:
>          parser.error('at least one of --html or --json (or both) is required')
> @@ -729,6 +880,7 @@ def __main__():
>      date = datetime.datetime.utcnow()
>      commit = subprocess.check_output(['git', 'rev-parse',
>                                        'HEAD']).splitlines()[0]
> +    download_nvd(args.nvd_path)
>      print("Build package list ...")
>      packages = get_pkglist(args.npackages, package_list)
>      print("Getting package make info ...")
> @@ -746,6 +898,9 @@ def __main__():
>      check_package_urls(packages)
>      print("Getting latest versions ...")
>      check_package_latest_version(packages)
> +    if args.nvd_path:
> +        print("CVE")
> +        check_package_cves(args.nvd_path, packages)
>      print("Calculate stats")
>      stats = calculate_stats(packages)
>      if args.html:
> 

^ permalink raw reply	[flat|nested] 11+ messages in thread
* [Buildroot] [PATCH 1/2] support/scripts/pkg-stats: add support for CVE reporting
  2020-02-05 10:48   ` Arnout Vandecappelle
@ 2020-02-13 22:03     ` Peter Korsgaard
  0 siblings, 0 replies; 11+ messages in thread
From: Peter Korsgaard @ 2020-02-13 22:03 UTC (permalink / raw)
  To: buildroot

>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

Hi,

 >> +            pkg_version = distutils.version.LooseVersion(pkg.current_version)
 >> +            if not hasattr(pkg_version, "version"):
 >> +                print("Cannot parse package '%s' version '%s'" % (pkg.name, pkg.current_version))
 >> +                continue
 >> +            cve_affected_version = distutils.version.LooseVersion(v["version_value"])
 >> +            if not hasattr(cve_affected_version, "version"):
 >> +                print("Cannot parse CVE affected version '%s'" % v["version_value"])
 >> +                continue
 >> +            return pkg_version < cve_affected_version

 >  Maybe use packaging.version instead? [1]

 > [1]
 > https://stackoverflow.com/questions/11887762/how-do-i-compare-version-numbers-in-python

 >  Hm, but the packaging module is not installed on my system (which does have
 > setuptools), so maybe not...

On Debian atleast, this is available in python{,3}-packaging, but is not
a reverse dependency of setuptools:

apt-cache rdepends python-packaging
python-packaging
Reverse Depends:
  legit
  python-sphinx
  sagemath
  python-pdal
  python-deprecation
  python-nipype

The stackoverflow package alternatively talks about distutils.version,
which we AFAIK also mentioned earlier.

-- 
Bye, Peter Korsgaard

^ permalink raw reply	[flat|nested] 11+ messages in thread
end of thread, other threads:[~2020-02-13 22:03 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-04 21:52 [Buildroot] [PATCH 0/2] Add CVE reporting to pkg-stats Thomas Petazzoni
2020-02-04 21:52 ` [Buildroot] [PATCH 1/2] support/scripts/pkg-stats: add support for CVE reporting Thomas Petazzoni
2020-02-04 22:34   ` Matthew Weber
2020-02-04 22:56   ` Peter Korsgaard
2020-02-05 10:48   ` Arnout Vandecappelle
2020-02-13 22:03     ` Peter Korsgaard
2020-02-04 21:52 ` [Buildroot] [PATCH 2/2] docs/manual: describe the new <pkg>_IGNORE_CVES variable Thomas Petazzoni
2020-02-04 22:59   ` Peter Korsgaard
2020-02-05  5:53   ` Thomas De Schampheleire
2020-02-04 22:32 ` [Buildroot] [PATCH 0/2] Add CVE reporting to pkg-stats Titouan Christophe
2020-02-04 22:36   ` Matthew Weber

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.