* [PATCH RFC CFH][sumo 01/47] cve-update-db: New recipe to update CVE database
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 02/47] cve-check: Remove dependency to cve-check-tool-native Mikko Rapeli
` (48 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
cve-check-tool-native do_populate_cve_db task was using deprecated NVD
xml data feeds, cve-update-db uses NVD json data feeds.
Sqlite database schema was updated to take into account CVSSv3 CVE
scores and operator in affected product versions.
A new META table was added to store the last modification date of the
NVD json data feeds.
(From OE-Core rev: 546d14135c50c6a571dfbf3baf6e9b22ce3d58e0)
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Conflicts:
meta/conf/distro/include/maintainers.inc
---
meta/conf/distro/include/maintainers.inc | 2 +-
meta/recipes-core/meta/cve-update-db.bb | 121 +++++++++++++++++++++++++++++++
2 files changed, 122 insertions(+), 1 deletion(-)
create mode 100644 meta/recipes-core/meta/cve-update-db.bb
diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index 48aff95..a299177 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -122,7 +122,7 @@ RECIPE_MAINTAINER_pn-cryptodev-module = "Robert Yang <liezhi.yang@windriver.com>
RECIPE_MAINTAINER_pn-cryptodev-tests = "Robert Yang <liezhi.yang@windriver.com>"
RECIPE_MAINTAINER_pn-cups = "Chen Qi <Qi.Chen@windriver.com>"
RECIPE_MAINTAINER_pn-curl = "Armin Kuster <akuster@mvista.com>"
-RECIPE_MAINTAINER_pn-cve-check-tool = "Ross Burton <ross.burton@intel.com>"
+RECIPE_MAINTAINER_pn-cve-update-db = "Ross Burton <ross.burton@intel.com>"
RECIPE_MAINTAINER_pn-cwautomacros = "Maxin B. John <maxin.john@intel.com>"
RECIPE_MAINTAINER_pn-damageproto = "Armin Kuster <akuster@mvista.com>"
RECIPE_MAINTAINER_pn-db = "Mark Hatle <mark.hatle@windriver.com>"
diff --git a/meta/recipes-core/meta/cve-update-db.bb b/meta/recipes-core/meta/cve-update-db.bb
new file mode 100644
index 0000000..522fd23
--- /dev/null
+++ b/meta/recipes-core/meta/cve-update-db.bb
@@ -0,0 +1,121 @@
+SUMMARY = "Updates the NVD CVE database"
+LICENSE = "MIT"
+
+INHIBIT_DEFAULT_DEPS = "1"
+PACKAGES = ""
+
+inherit nopackages
+
+deltask do_fetch
+deltask do_unpack
+deltask do_patch
+deltask do_configure
+deltask do_compile
+deltask do_install
+deltask do_populate_sysroot
+
+python do_populate_cve_db() {
+ """
+ Update NVD database with json data feed
+ """
+
+ import sqlite3, urllib3, shutil, gzip, re
+ from datetime import date
+
+ BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-"
+ YEAR_START = 2002
+ JSON_TMPFILE = d.getVar("CVE_CHECK_DB_DIR") + '/nvd.json.gz'
+
+ # Connect to database
+ db_file = d.getVar("CVE_CHECK_DB_FILE")
+ conn = sqlite3.connect(db_file)
+ c = conn.cursor()
+
+ initialize_db(c)
+
+ http = urllib3.PoolManager()
+
+ for year in range(YEAR_START, date.today().year + 1):
+ year_url = BASE_URL + str(year)
+ meta_url = year_url + ".meta"
+ json_url = year_url + ".json.gz"
+
+ # Retrieve meta last modified date
+ with http.request('GET', meta_url, preload_content=False) as r:
+ date_line = str(r.data.splitlines()[0])
+ last_modified = re.search('lastModifiedDate:(.*)', date_line).group(1)
+
+ # Compare with current db last modified date
+ c.execute("select DATE from META where YEAR = '%d'" % year)
+ meta = c.fetchone()
+ if not meta or meta[0] != last_modified:
+ # Update db with current year json file
+ with http.request('GET', json_url, preload_content=False) as r, open(JSON_TMPFILE, 'wb') as tmpfile:
+ shutil.copyfileobj(r, tmpfile)
+ with gzip.open(JSON_TMPFILE, 'rt') as jsonfile:
+ update_db(c, jsonfile)
+ c.execute("insert or replace into META values (?, ?)",
+ [year, last_modified])
+
+ conn.commit()
+ conn.close()
+
+ with open(d.getVar("CVE_CHECK_TMP_FILE"), 'a'):
+ os.utime(d.getVar("CVE_CHECK_TMP_FILE"), None)
+}
+
+# DJB2 hash algorithm
+def hash_djb2(s):
+ hash = 5381
+ for x in s:
+ hash = (( hash << 5) + hash) + ord(x)
+
+ return hash & 0xFFFFFFFF
+
+def initialize_db(c):
+ c.execute("CREATE TABLE IF NOT EXISTS META (YEAR INTEGER UNIQUE, DATE TEXT)")
+ c.execute("CREATE TABLE IF NOT EXISTS NVD (ID TEXT UNIQUE, SUMMARY TEXT, \
+ SCOREV2 TEXT, SCOREV3 TEXT, MODIFIED INTEGER, VECTOR TEXT)")
+ c.execute("CREATE TABLE IF NOT EXISTS PRODUCTS (HASH INTEGER UNIQUE, ID TEXT, \
+ VENDOR TEXT, PRODUCT TEXT, VERSION TEXT, OPERATOR TEXT)")
+ c.execute("CREATE INDEX IF NOT EXISTS PRODUCT_IDX ON PRODUCTS \
+ (PRODUCT, VERSION)")
+
+def update_db(c, json_filename):
+ import json
+ root = json.load(json_filename)
+
+ for elt in root['CVE_Items']:
+ if not elt['impact']:
+ continue
+
+ cveId = elt['cve']['CVE_data_meta']['ID']
+ cveDesc = elt['cve']['description']['description_data'][0]['value']
+ date = elt['lastModifiedDate']
+ accessVector = elt['impact']['baseMetricV2']['cvssV2']['accessVector']
+ cvssv2 = elt['impact']['baseMetricV2']['cvssV2']['baseScore']
+
+ try:
+ cvssv3 = elt['impact']['baseMetricV3']['cvssV3']['baseScore']
+ except:
+ cvssv3 = 0.0
+
+ c.execute("insert or replace into NVD values (?, ?, ?, ?, ?, ?)",
+ [cveId, cveDesc, cvssv2, cvssv3, date, accessVector])
+
+ for vendor in elt['cve']['affects']['vendor']['vendor_data']:
+ for product in vendor['product']['product_data']:
+ for version in product['version']['version_data']:
+ product_str = cveId+vendor['vendor_name']+product['product_name']+version['version_value']
+ hashstr = hash_djb2(product_str)
+ c.execute("insert or replace into PRODUCTS values (?, ?, ?, ?, ?, ?)",
+ [ hashstr, cveId, vendor['vendor_name'],
+ product['product_name'], version['version_value'],
+ version['version_affected']])
+
+
+
+addtask do_populate_cve_db before do_cve_check
+do_populate_cve_db[nostamp] = "1"
+
+EXCLUDE_FROM_WORLD = "1"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 02/47] cve-check: Remove dependency to cve-check-tool-native
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 01/47] cve-update-db: New recipe to update CVE database Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 03/47] cve-check: Manage CVE_PRODUCT with more than one name Mikko Rapeli
` (47 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Use the new update-cve-db recipe to update database.
(From OE-Core rev: bc144b028f6f51252f4359248f6921028bcb6780)
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/cve-check.bbclass | 71 ++++++++++++++++--------------------------
1 file changed, 26 insertions(+), 45 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 743bc08..28619c7 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -26,7 +26,7 @@ CVE_PRODUCT ??= "${BPN}"
CVE_VERSION ??= "${PV}"
CVE_CHECK_DB_DIR ?= "${DL_DIR}/CVE_CHECK"
-CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvd.db"
+CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvd-json.db"
CVE_CHECK_LOG ?= "${T}/cve.log"
CVE_CHECK_TMP_FILE ?= "${TMPDIR}/cve_check"
@@ -62,7 +62,7 @@ python do_cve_check () {
}
addtask cve_check after do_unpack before do_build
-do_cve_check[depends] = "cve-check-tool-native:do_populate_sysroot cve-check-tool-native:do_populate_cve_db"
+do_cve_check[depends] = "cve-update-db:do_populate_cve_db"
do_cve_check[nostamp] = "1"
python cve_check_cleanup () {
@@ -163,61 +163,40 @@ def get_patches_cves(d):
def check_cves(d, patched_cves):
"""
- Run cve-check-tool looking for patched and unpatched CVEs.
+ Connect to the NVD database and find unpatched cves.
"""
-
import ast, csv, tempfile, subprocess, io
- cves_patched = []
cves_unpatched = []
bpn = d.getVar("CVE_PRODUCT")
# If this has been unset then we're not scanning for CVEs here (for example, image recipes)
if not bpn:
return ([], [])
pv = d.getVar("CVE_VERSION").split("+git")[0]
- cves = " ".join(patched_cves)
- cve_db_dir = d.getVar("CVE_CHECK_DB_DIR")
cve_whitelist = ast.literal_eval(d.getVar("CVE_CHECK_CVE_WHITELIST"))
- cve_cmd = "cve-check-tool"
- cmd = [cve_cmd, "--no-html", "--skip-update", "--csv", "--not-affected", "-t", "faux", "-d", cve_db_dir]
# If the recipe has been whitlisted we return empty lists
if d.getVar("PN") in d.getVar("CVE_CHECK_PN_WHITELIST").split():
bb.note("Recipe has been whitelisted, skipping check")
return ([], [])
- try:
- # Write the faux CSV file to be used with cve-check-tool
- fd, faux = tempfile.mkstemp(prefix="cve-faux-")
- with os.fdopen(fd, "w") as f:
- for pn in bpn.split():
- f.write("%s,%s,%s,\n" % (pn, pv, cves))
- cmd.append(faux)
-
- output = subprocess.check_output(cmd).decode("utf-8")
- bb.debug(2, "Output of command %s:\n%s" % ("\n".join(cmd), output))
- except subprocess.CalledProcessError as e:
- bb.warn("Couldn't check for CVEs: %s (output %s)" % (e, e.output))
- finally:
- os.remove(faux)
-
- for row in csv.reader(io.StringIO(output)):
- # Third row has the unpatched CVEs
- if row[2]:
- for cve in row[2].split():
- # Skip if the CVE has been whitlisted for the current version
- if pv in cve_whitelist.get(cve,[]):
- bb.note("%s-%s has been whitelisted for %s" % (bpn, pv, cve))
- else:
- cves_unpatched.append(cve)
- bb.debug(2, "%s-%s is not patched for %s" % (bpn, pv, cve))
- # Fourth row has patched CVEs
- if row[3]:
- for cve in row[3].split():
- cves_patched.append(cve)
- bb.debug(2, "%s-%s is patched for %s" % (bpn, pv, cve))
-
- return (cves_patched, cves_unpatched)
+ import sqlite3
+ db_file = d.getVar("CVE_CHECK_DB_FILE")
+ conn = sqlite3.connect(db_file)
+ c = conn.cursor()
+ query = "SELECT * FROM PRODUCTS WHERE PRODUCT IS '%s' AND VERSION IS '%s';"
+ for row in c.execute(query % (bpn,pv)):
+ cve = row[1]
+ if pv in cve_whitelist.get(cve,[]):
+ bb.note("%s-%s has been whitelisted for %s" % (bpn, pv, cve))
+ elif cve in patched_cves:
+ bb.note("%s has been patched" % (cve))
+ else:
+ cves_unpatched.append(cve)
+ bb.debug(2, "%s-%s is not patched for %s" % (bpn, pv, cve))
+ conn.close()
+
+ return (list(patched_cves), cves_unpatched)
def get_cve_info(d, cves):
"""
@@ -241,9 +220,10 @@ def get_cve_info(d, cves):
for row in cur.execute(query, tuple(cves)):
cve_data[row[0]] = {}
cve_data[row[0]]["summary"] = row[1]
- cve_data[row[0]]["score"] = row[2]
- cve_data[row[0]]["modified"] = row[3]
- cve_data[row[0]]["vector"] = row[4]
+ cve_data[row[0]]["scorev2"] = row[2]
+ cve_data[row[0]]["scorev3"] = row[3]
+ cve_data[row[0]]["modified"] = row[4]
+ cve_data[row[0]]["vector"] = row[5]
conn.close()
return cve_data
@@ -270,7 +250,8 @@ def cve_write_data(d, patched, unpatched, cve_data):
unpatched_cves.append(cve)
write_string += "CVE STATUS: Unpatched\n"
write_string += "CVE SUMMARY: %s\n" % cve_data[cve]["summary"]
- write_string += "CVSS v2 BASE SCORE: %s\n" % cve_data[cve]["score"]
+ write_string += "CVSS v2 BASE SCORE: %s\n" % cve_data[cve]["scorev2"]
+ write_string += "CVSS v3 BASE SCORE: %s\n" % cve_data[cve]["scorev3"]
write_string += "VECTOR: %s\n" % cve_data[cve]["vector"]
write_string += "MORE INFORMATION: %s%s\n\n" % (nvd_link, cve)
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 03/47] cve-check: Manage CVE_PRODUCT with more than one name
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 01/47] cve-update-db: New recipe to update CVE database Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 02/47] cve-check: Remove dependency to cve-check-tool-native Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 04/47] cve-check: Consider CVE that affects versions with less than operator Mikko Rapeli
` (46 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
In some rare cases (eg. curl recipe) the CVE_PRODUCT contains more than
one name.
(From OE-Core rev: 7f62a20b32a3d42f04ec58786a7d0db68ef1bb05)
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/cve-check.bbclass | 25 ++++++++++++++-----------
1 file changed, 14 insertions(+), 11 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 28619c7..e7540b8 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -168,9 +168,10 @@ def check_cves(d, patched_cves):
import ast, csv, tempfile, subprocess, io
cves_unpatched = []
- bpn = d.getVar("CVE_PRODUCT")
+ # CVE_PRODUCT can contain more than one product (eg. curl/libcurl)
+ bpn = d.getVar("CVE_PRODUCT").split()
# If this has been unset then we're not scanning for CVEs here (for example, image recipes)
- if not bpn:
+ if len(bpn) == 0:
return ([], [])
pv = d.getVar("CVE_VERSION").split("+git")[0]
cve_whitelist = ast.literal_eval(d.getVar("CVE_CHECK_CVE_WHITELIST"))
@@ -184,16 +185,18 @@ def check_cves(d, patched_cves):
db_file = d.getVar("CVE_CHECK_DB_FILE")
conn = sqlite3.connect(db_file)
c = conn.cursor()
+
query = "SELECT * FROM PRODUCTS WHERE PRODUCT IS '%s' AND VERSION IS '%s';"
- for row in c.execute(query % (bpn,pv)):
- cve = row[1]
- if pv in cve_whitelist.get(cve,[]):
- bb.note("%s-%s has been whitelisted for %s" % (bpn, pv, cve))
- elif cve in patched_cves:
- bb.note("%s has been patched" % (cve))
- else:
- cves_unpatched.append(cve)
- bb.debug(2, "%s-%s is not patched for %s" % (bpn, pv, cve))
+ for idx in range(len(bpn)):
+ for row in c.execute(query % (bpn[idx],pv)):
+ cve = row[1]
+ if pv in cve_whitelist.get(cve,[]):
+ bb.note("%s-%s has been whitelisted for %s" % (bpn[idx], pv, cve))
+ elif cve in patched_cves:
+ bb.note("%s has been patched" % (cve))
+ else:
+ cves_unpatched.append(cve)
+ bb.debug(2, "%s-%s is not patched for %s" % (bpn[idx], pv, cve))
conn.close()
return (list(patched_cves), cves_unpatched)
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 04/47] cve-check: Consider CVE that affects versions with less than operator
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (2 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 03/47] cve-check: Manage CVE_PRODUCT with more than one name Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 05/47] flac: also add flac to CVE_PRODUCT Mikko Rapeli
` (45 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
In the NVD json CVE feed, affected versions can be strictly matched to a
version, but they can also be matched with the operator '<='.
Add a new condition in the sqlite query to match affected versions that
are defined with the operator '<='. Then use LooseVersion to discard all
versions that are not relevant.
(From OE-Core rev: 3bf63bc60848d91e90c23f6d854d22b78832aa2d)
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/cve-check.bbclass | 16 ++++++++++++++--
1 file changed, 14 insertions(+), 2 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index e7540b8..379f712 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -166,6 +166,7 @@ def check_cves(d, patched_cves):
Connect to the NVD database and find unpatched cves.
"""
import ast, csv, tempfile, subprocess, io
+ from distutils.version import LooseVersion
cves_unpatched = []
# CVE_PRODUCT can contain more than one product (eg. curl/libcurl)
@@ -186,14 +187,25 @@ def check_cves(d, patched_cves):
conn = sqlite3.connect(db_file)
c = conn.cursor()
- query = "SELECT * FROM PRODUCTS WHERE PRODUCT IS '%s' AND VERSION IS '%s';"
+ query = """SELECT * FROM PRODUCTS WHERE
+ (PRODUCT IS '{0}' AND VERSION = '{1}' AND OPERATOR IS '=') OR
+ (PRODUCT IS '{0}' AND OPERATOR IS '<=');"""
for idx in range(len(bpn)):
- for row in c.execute(query % (bpn[idx],pv)):
+ for row in c.execute(query.format(bpn[idx],pv)):
cve = row[1]
+ version = row[4]
+
+ try:
+ discardVersion = LooseVersion(version) < LooseVersion(pv)
+ except:
+ discardVersion = True
+
if pv in cve_whitelist.get(cve,[]):
bb.note("%s-%s has been whitelisted for %s" % (bpn[idx], pv, cve))
elif cve in patched_cves:
bb.note("%s has been patched" % (cve))
+ elif discardVersion:
+ bb.debug(2, "Do not consider version %s " % (version))
else:
cves_unpatched.append(cve)
bb.debug(2, "%s-%s is not patched for %s" % (bpn[idx], pv, cve))
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 05/47] flac: also add flac to CVE_PRODUCT
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (3 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 04/47] cve-check: Consider CVE that affects versions with less than operator Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 06/47] cve-update-db: Use std library instead of urllib3 Mikko Rapeli
` (44 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Chen Qi <Qi.Chen@windriver.com>
flac uses both 'flac' and 'libflac' as cve product.
(From OE-Core rev: 3a043a078f6cc89bcc097823fa37cd1311805ae7)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-multimedia/flac/flac_1.3.2.bb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/recipes-multimedia/flac/flac_1.3.2.bb b/meta/recipes-multimedia/flac/flac_1.3.2.bb
index 028a429..3e67fa2 100644
--- a/meta/recipes-multimedia/flac/flac_1.3.2.bb
+++ b/meta/recipes-multimedia/flac/flac_1.3.2.bb
@@ -20,7 +20,7 @@ SRC_URI = "http://downloads.xiph.org/releases/flac/${BP}.tar.xz \
SRC_URI[md5sum] = "454f1bfa3f93cc708098d7890d0499bd"
SRC_URI[sha256sum] = "91cfc3ed61dc40f47f050a109b08610667d73477af6ef36dcad31c31a4a8d53f"
-CVE_PRODUCT = "libflac"
+CVE_PRODUCT = "libflac flac"
inherit autotools gettext
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 06/47] cve-update-db: Use std library instead of urllib3
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (4 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 05/47] flac: also add flac to CVE_PRODUCT Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 07/47] cve-check: be idiomatic Mikko Rapeli
` (43 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
urllib3 was used in this recipe but it was not set as a
dependency. As it is not specifically needed, rewrite the recipe with
urllib from the standard library.
(From OE-Core rev: c0eabd30d7b9c2517f4ec9229640be421ecc8a5e)
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-core/meta/cve-update-db.bb | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/meta/recipes-core/meta/cve-update-db.bb b/meta/recipes-core/meta/cve-update-db.bb
index 522fd23..1f48820 100644
--- a/meta/recipes-core/meta/cve-update-db.bb
+++ b/meta/recipes-core/meta/cve-update-db.bb
@@ -19,7 +19,7 @@ python do_populate_cve_db() {
Update NVD database with json data feed
"""
- import sqlite3, urllib3, shutil, gzip, re
+ import sqlite3, urllib, shutil, gzip, re
from datetime import date
BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-"
@@ -33,16 +33,14 @@ python do_populate_cve_db() {
initialize_db(c)
- http = urllib3.PoolManager()
-
for year in range(YEAR_START, date.today().year + 1):
year_url = BASE_URL + str(year)
meta_url = year_url + ".meta"
json_url = year_url + ".json.gz"
# Retrieve meta last modified date
- with http.request('GET', meta_url, preload_content=False) as r:
- date_line = str(r.data.splitlines()[0])
+ with urllib.request.urlopen(meta_url) as r:
+ date_line = str(r.read().splitlines()[0])
last_modified = re.search('lastModifiedDate:(.*)', date_line).group(1)
# Compare with current db last modified date
@@ -50,7 +48,7 @@ python do_populate_cve_db() {
meta = c.fetchone()
if not meta or meta[0] != last_modified:
# Update db with current year json file
- with http.request('GET', json_url, preload_content=False) as r, open(JSON_TMPFILE, 'wb') as tmpfile:
+ with urllib.request.urlopen(json_url) as r, open(JSON_TMPFILE, 'wb') as tmpfile:
shutil.copyfileobj(r, tmpfile)
with gzip.open(JSON_TMPFILE, 'rt') as jsonfile:
update_db(c, jsonfile)
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 07/47] cve-check: be idiomatic
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (5 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 06/47] cve-update-db: Use std library instead of urllib3 Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 08/47] cve-update-db: Manage proxy if needed Mikko Rapeli
` (42 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
Instead of generating a series of indexes via range(len(list)), just iterate the
list.
(From OE-Core rev: 27eb839ee651c2d584db42d23bcf5dd764eb33f1)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/cve-check.bbclass | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 379f712..1e7e8dd 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -170,18 +170,19 @@ def check_cves(d, patched_cves):
cves_unpatched = []
# CVE_PRODUCT can contain more than one product (eg. curl/libcurl)
- bpn = d.getVar("CVE_PRODUCT").split()
+ products = d.getVar("CVE_PRODUCT").split()
# If this has been unset then we're not scanning for CVEs here (for example, image recipes)
- if len(bpn) == 0:
+ if not products:
return ([], [])
pv = d.getVar("CVE_VERSION").split("+git")[0]
- cve_whitelist = ast.literal_eval(d.getVar("CVE_CHECK_CVE_WHITELIST"))
# If the recipe has been whitlisted we return empty lists
if d.getVar("PN") in d.getVar("CVE_CHECK_PN_WHITELIST").split():
bb.note("Recipe has been whitelisted, skipping check")
return ([], [])
+ cve_whitelist = ast.literal_eval(d.getVar("CVE_CHECK_CVE_WHITELIST"))
+
import sqlite3
db_file = d.getVar("CVE_CHECK_DB_FILE")
conn = sqlite3.connect(db_file)
@@ -190,8 +191,8 @@ def check_cves(d, patched_cves):
query = """SELECT * FROM PRODUCTS WHERE
(PRODUCT IS '{0}' AND VERSION = '{1}' AND OPERATOR IS '=') OR
(PRODUCT IS '{0}' AND OPERATOR IS '<=');"""
- for idx in range(len(bpn)):
- for row in c.execute(query.format(bpn[idx],pv)):
+ for product in products:
+ for row in c.execute(query.format(product, pv)):
cve = row[1]
version = row[4]
@@ -200,15 +201,15 @@ def check_cves(d, patched_cves):
except:
discardVersion = True
- if pv in cve_whitelist.get(cve,[]):
- bb.note("%s-%s has been whitelisted for %s" % (bpn[idx], pv, cve))
+ if pv in cve_whitelist.get(cve, []):
+ bb.note("%s-%s has been whitelisted for %s" % (product, pv, cve))
elif cve in patched_cves:
bb.note("%s has been patched" % (cve))
elif discardVersion:
bb.debug(2, "Do not consider version %s " % (version))
else:
cves_unpatched.append(cve)
- bb.debug(2, "%s-%s is not patched for %s" % (bpn[idx], pv, cve))
+ bb.debug(2, "%s-%s is not patched for %s" % (product, pv, cve))
conn.close()
return (list(patched_cves), cves_unpatched)
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 08/47] cve-update-db: Manage proxy if needed.
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (6 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 07/47] cve-check: be idiomatic Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 09/47] cve-update-db: do_populate_cve_db depends on do_fetch Mikko Rapeli
` (41 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
If https_proxy environment variable is defined, manage proxy to be able
to download meta and json data feeds from https://nvd.nist.gov
(From OE-Core rev: 09be21f4d1793b1e26e78391f51bfc0a27b76deb)
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-core/meta/cve-update-db.bb | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/meta/recipes-core/meta/cve-update-db.bb b/meta/recipes-core/meta/cve-update-db.bb
index 1f48820..4c896dc 100644
--- a/meta/recipes-core/meta/cve-update-db.bb
+++ b/meta/recipes-core/meta/cve-update-db.bb
@@ -25,6 +25,7 @@ python do_populate_cve_db() {
BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-"
YEAR_START = 2002
JSON_TMPFILE = d.getVar("CVE_CHECK_DB_DIR") + '/nvd.json.gz'
+ proxy = d.getVar("https_proxy")
# Connect to database
db_file = d.getVar("CVE_CHECK_DB_FILE")
@@ -39,7 +40,10 @@ python do_populate_cve_db() {
json_url = year_url + ".json.gz"
# Retrieve meta last modified date
- with urllib.request.urlopen(meta_url) as r:
+ req = urllib.request.Request(meta_url)
+ if proxy:
+ req.set_proxy(proxy, 'https')
+ with urllib.request.urlopen(req) as r:
date_line = str(r.read().splitlines()[0])
last_modified = re.search('lastModifiedDate:(.*)', date_line).group(1)
@@ -48,7 +52,10 @@ python do_populate_cve_db() {
meta = c.fetchone()
if not meta or meta[0] != last_modified:
# Update db with current year json file
- with urllib.request.urlopen(json_url) as r, open(JSON_TMPFILE, 'wb') as tmpfile:
+ req = urllib.request.Request(json_url)
+ if proxy:
+ req.set_proxy(proxy, 'https')
+ with urllib.request.urlopen(req) as r, open(JSON_TMPFILE, 'wb') as tmpfile:
shutil.copyfileobj(r, tmpfile)
with gzip.open(JSON_TMPFILE, 'rt') as jsonfile:
update_db(c, jsonfile)
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 09/47] cve-update-db: do_populate_cve_db depends on do_fetch
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (7 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 08/47] cve-update-db: Manage proxy if needed Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 10/47] cve-update-db: Catch request.urlopen errors Mikko Rapeli
` (40 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
To be able to populate NVD database on a fetchall
(bitbake <image> --run-all=fetch), set the do_populate_cve_db task to be
executed before do_fetch.
Do not get CVE_CHECK_DB_DIR, CVE_CHECK_DB_FILE and CVE_CHECK_TMP_FILE
variable because do_populate_cve_db can be called in a context where
cve-check class is not loaded.
(From OE-Core rev: 975793e3825a2a9ca6dc0e43577f680214cb7993)
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-core/meta/cve-update-db.bb | 21 +++++++++++++--------
1 file changed, 13 insertions(+), 8 deletions(-)
diff --git a/meta/recipes-core/meta/cve-update-db.bb b/meta/recipes-core/meta/cve-update-db.bb
index 4c896dc..3e5bae8 100644
--- a/meta/recipes-core/meta/cve-update-db.bb
+++ b/meta/recipes-core/meta/cve-update-db.bb
@@ -6,7 +6,6 @@ PACKAGES = ""
inherit nopackages
-deltask do_fetch
deltask do_unpack
deltask do_patch
deltask do_configure
@@ -24,11 +23,16 @@ python do_populate_cve_db() {
BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-"
YEAR_START = 2002
- JSON_TMPFILE = d.getVar("CVE_CHECK_DB_DIR") + '/nvd.json.gz'
+
+ db_dir = d.getVar("DL_DIR") + '/CVE_CHECK'
+ db_file = db_dir + '/nvd-json.db'
+ json_tmpfile = db_dir + '/nvd.json.gz'
proxy = d.getVar("https_proxy")
+ if not os.path.isdir(db_dir):
+ os.mkdir(db_dir)
+
# Connect to database
- db_file = d.getVar("CVE_CHECK_DB_FILE")
conn = sqlite3.connect(db_file)
c = conn.cursor()
@@ -55,9 +59,9 @@ python do_populate_cve_db() {
req = urllib.request.Request(json_url)
if proxy:
req.set_proxy(proxy, 'https')
- with urllib.request.urlopen(req) as r, open(JSON_TMPFILE, 'wb') as tmpfile:
+ with urllib.request.urlopen(req) as r, open(json_tmpfile, 'wb') as tmpfile:
shutil.copyfileobj(r, tmpfile)
- with gzip.open(JSON_TMPFILE, 'rt') as jsonfile:
+ with gzip.open(json_tmpfile, 'rt') as jsonfile:
update_db(c, jsonfile)
c.execute("insert or replace into META values (?, ?)",
[year, last_modified])
@@ -65,8 +69,9 @@ python do_populate_cve_db() {
conn.commit()
conn.close()
- with open(d.getVar("CVE_CHECK_TMP_FILE"), 'a'):
- os.utime(d.getVar("CVE_CHECK_TMP_FILE"), None)
+ cve_check_tmp_file = d.getVar("TMPDIR") + '/cve_check'
+ with open(cve_check_tmp_file, 'a'):
+ os.utime(cve_check_tmp_file, None)
}
# DJB2 hash algorithm
@@ -120,7 +125,7 @@ def update_db(c, json_filename):
-addtask do_populate_cve_db before do_cve_check
+addtask do_populate_cve_db before do_fetch
do_populate_cve_db[nostamp] = "1"
EXCLUDE_FROM_WORLD = "1"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 10/47] cve-update-db: Catch request.urlopen errors.
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (8 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 09/47] cve-update-db: do_populate_cve_db depends on do_fetch Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 11/47] cve-check: Depends on cve-update-db-native Mikko Rapeli
` (39 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
If the NVD url is not accessible, print a warning on top of the CVE
report, and continue. The database will not be fully updated, but
cve_check can still run on the previous database.
(From OE-Core rev: 0325dd72714f0b447558084f481b77f0ec850eed)
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/cve-check.bbclass | 5 +++--
meta/recipes-core/meta/cve-update-db.bb | 30 +++++++++++++++++++++---------
2 files changed, 24 insertions(+), 11 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 1e7e8dd..81071e3 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -51,14 +51,15 @@ python do_cve_check () {
Check recipe for patched and unpatched CVEs
"""
- if os.path.exists(d.getVar("CVE_CHECK_TMP_FILE")):
+ if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")):
patched_cves = get_patches_cves(d)
patched, unpatched = check_cves(d, patched_cves)
if patched or unpatched:
cve_data = get_cve_info(d, patched + unpatched)
cve_write_data(d, patched, unpatched, cve_data)
else:
- bb.note("Failed to update CVE database, skipping CVE check")
+ bb.note("No CVE database found, skipping CVE check")
+
}
addtask cve_check after do_unpack before do_build
diff --git a/meta/recipes-core/meta/cve-update-db.bb b/meta/recipes-core/meta/cve-update-db.bb
index 3e5bae8..ae8f1a9 100644
--- a/meta/recipes-core/meta/cve-update-db.bb
+++ b/meta/recipes-core/meta/cve-update-db.bb
@@ -28,6 +28,7 @@ python do_populate_cve_db() {
db_file = db_dir + '/nvd-json.db'
json_tmpfile = db_dir + '/nvd.json.gz'
proxy = d.getVar("https_proxy")
+ cve_f = open(d.getVar("TMPDIR") + '/cve_check', 'a')
if not os.path.isdir(db_dir):
os.mkdir(db_dir)
@@ -47,9 +48,13 @@ python do_populate_cve_db() {
req = urllib.request.Request(meta_url)
if proxy:
req.set_proxy(proxy, 'https')
- with urllib.request.urlopen(req) as r:
- date_line = str(r.read().splitlines()[0])
- last_modified = re.search('lastModifiedDate:(.*)', date_line).group(1)
+ try:
+ with urllib.request.urlopen(req, timeout=1) as r:
+ date_line = str(r.read().splitlines()[0])
+ last_modified = re.search('lastModifiedDate:(.*)', date_line).group(1)
+ except:
+ cve_f.write('Warning: CVE db update error, CVE data is outdated.\n\n')
+ break
# Compare with current db last modified date
c.execute("select DATE from META where YEAR = '%d'" % year)
@@ -59,19 +64,26 @@ python do_populate_cve_db() {
req = urllib.request.Request(json_url)
if proxy:
req.set_proxy(proxy, 'https')
- with urllib.request.urlopen(req) as r, open(json_tmpfile, 'wb') as tmpfile:
- shutil.copyfileobj(r, tmpfile)
+ try:
+ with urllib.request.urlopen(req, timeout=1) as r, \
+ open(json_tmpfile, 'wb') as tmpfile:
+ shutil.copyfileobj(r, tmpfile)
+ except:
+ cve_f.write('Warning: CVE db update error, CVE data is outdated.\n\n')
+ break
+
with gzip.open(json_tmpfile, 'rt') as jsonfile:
update_db(c, jsonfile)
c.execute("insert or replace into META values (?, ?)",
[year, last_modified])
+ # Update success, set the date to cve_check file.
+ if year == date.today().year:
+ cve_f.write('CVE database update : %s\n\n' % date.today())
+
+ cve_f.close()
conn.commit()
conn.close()
-
- cve_check_tmp_file = d.getVar("TMPDIR") + '/cve_check'
- with open(cve_check_tmp_file, 'a'):
- os.utime(cve_check_tmp_file, None)
}
# DJB2 hash algorithm
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 11/47] cve-check: Depends on cve-update-db-native
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (9 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 10/47] cve-update-db: Catch request.urlopen errors Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 12/47] cve-check: Update unpatched CVE matching Mikko Rapeli
` (38 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
do_populate_cve_db is a native task.
(From OE-Core rev: 4078da92b49946848cddebe1735f301af161e162)
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Conflicts:
meta/conf/distro/include/maintainers.inc
---
meta/classes/cve-check.bbclass | 2 +-
meta/conf/distro/include/maintainers.inc | 2 +
meta/recipes-core/meta/cve-update-db-native.bb | 143 +++++++++++++++++++++++++
meta/recipes-core/meta/cve-update-db.bb | 143 -------------------------
4 files changed, 146 insertions(+), 144 deletions(-)
create mode 100644 meta/recipes-core/meta/cve-update-db-native.bb
delete mode 100644 meta/recipes-core/meta/cve-update-db.bb
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 81071e3..6ffa0c4 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -63,7 +63,7 @@ python do_cve_check () {
}
addtask cve_check after do_unpack before do_build
-do_cve_check[depends] = "cve-update-db:do_populate_cve_db"
+do_cve_check[depends] = "cve-update-db-native:do_populate_cve_db"
do_cve_check[nostamp] = "1"
python cve_check_cleanup () {
diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index a299177..ee46a98 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -122,7 +122,9 @@ RECIPE_MAINTAINER_pn-cryptodev-module = "Robert Yang <liezhi.yang@windriver.com>
RECIPE_MAINTAINER_pn-cryptodev-tests = "Robert Yang <liezhi.yang@windriver.com>"
RECIPE_MAINTAINER_pn-cups = "Chen Qi <Qi.Chen@windriver.com>"
RECIPE_MAINTAINER_pn-curl = "Armin Kuster <akuster@mvista.com>"
+RECIPE_MAINTAINER_pn-cve-check-tool = "Ross Burton <ross.burton@intel.com>"
RECIPE_MAINTAINER_pn-cve-update-db = "Ross Burton <ross.burton@intel.com>"
+RECIPE_MAINTAINER_pn-cve-update-db-native = "Ross Burton <ross.burton@intel.com>"
RECIPE_MAINTAINER_pn-cwautomacros = "Maxin B. John <maxin.john@intel.com>"
RECIPE_MAINTAINER_pn-damageproto = "Armin Kuster <akuster@mvista.com>"
RECIPE_MAINTAINER_pn-db = "Mark Hatle <mark.hatle@windriver.com>"
diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb
new file mode 100644
index 0000000..ae8f1a9
--- /dev/null
+++ b/meta/recipes-core/meta/cve-update-db-native.bb
@@ -0,0 +1,143 @@
+SUMMARY = "Updates the NVD CVE database"
+LICENSE = "MIT"
+
+INHIBIT_DEFAULT_DEPS = "1"
+PACKAGES = ""
+
+inherit nopackages
+
+deltask do_unpack
+deltask do_patch
+deltask do_configure
+deltask do_compile
+deltask do_install
+deltask do_populate_sysroot
+
+python do_populate_cve_db() {
+ """
+ Update NVD database with json data feed
+ """
+
+ import sqlite3, urllib, shutil, gzip, re
+ from datetime import date
+
+ BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-"
+ YEAR_START = 2002
+
+ db_dir = d.getVar("DL_DIR") + '/CVE_CHECK'
+ db_file = db_dir + '/nvd-json.db'
+ json_tmpfile = db_dir + '/nvd.json.gz'
+ proxy = d.getVar("https_proxy")
+ cve_f = open(d.getVar("TMPDIR") + '/cve_check', 'a')
+
+ if not os.path.isdir(db_dir):
+ os.mkdir(db_dir)
+
+ # Connect to database
+ conn = sqlite3.connect(db_file)
+ c = conn.cursor()
+
+ initialize_db(c)
+
+ for year in range(YEAR_START, date.today().year + 1):
+ year_url = BASE_URL + str(year)
+ meta_url = year_url + ".meta"
+ json_url = year_url + ".json.gz"
+
+ # Retrieve meta last modified date
+ req = urllib.request.Request(meta_url)
+ if proxy:
+ req.set_proxy(proxy, 'https')
+ try:
+ with urllib.request.urlopen(req, timeout=1) as r:
+ date_line = str(r.read().splitlines()[0])
+ last_modified = re.search('lastModifiedDate:(.*)', date_line).group(1)
+ except:
+ cve_f.write('Warning: CVE db update error, CVE data is outdated.\n\n')
+ break
+
+ # Compare with current db last modified date
+ c.execute("select DATE from META where YEAR = '%d'" % year)
+ meta = c.fetchone()
+ if not meta or meta[0] != last_modified:
+ # Update db with current year json file
+ req = urllib.request.Request(json_url)
+ if proxy:
+ req.set_proxy(proxy, 'https')
+ try:
+ with urllib.request.urlopen(req, timeout=1) as r, \
+ open(json_tmpfile, 'wb') as tmpfile:
+ shutil.copyfileobj(r, tmpfile)
+ except:
+ cve_f.write('Warning: CVE db update error, CVE data is outdated.\n\n')
+ break
+
+ with gzip.open(json_tmpfile, 'rt') as jsonfile:
+ update_db(c, jsonfile)
+ c.execute("insert or replace into META values (?, ?)",
+ [year, last_modified])
+
+ # Update success, set the date to cve_check file.
+ if year == date.today().year:
+ cve_f.write('CVE database update : %s\n\n' % date.today())
+
+ cve_f.close()
+ conn.commit()
+ conn.close()
+}
+
+# DJB2 hash algorithm
+def hash_djb2(s):
+ hash = 5381
+ for x in s:
+ hash = (( hash << 5) + hash) + ord(x)
+
+ return hash & 0xFFFFFFFF
+
+def initialize_db(c):
+ c.execute("CREATE TABLE IF NOT EXISTS META (YEAR INTEGER UNIQUE, DATE TEXT)")
+ c.execute("CREATE TABLE IF NOT EXISTS NVD (ID TEXT UNIQUE, SUMMARY TEXT, \
+ SCOREV2 TEXT, SCOREV3 TEXT, MODIFIED INTEGER, VECTOR TEXT)")
+ c.execute("CREATE TABLE IF NOT EXISTS PRODUCTS (HASH INTEGER UNIQUE, ID TEXT, \
+ VENDOR TEXT, PRODUCT TEXT, VERSION TEXT, OPERATOR TEXT)")
+ c.execute("CREATE INDEX IF NOT EXISTS PRODUCT_IDX ON PRODUCTS \
+ (PRODUCT, VERSION)")
+
+def update_db(c, json_filename):
+ import json
+ root = json.load(json_filename)
+
+ for elt in root['CVE_Items']:
+ if not elt['impact']:
+ continue
+
+ cveId = elt['cve']['CVE_data_meta']['ID']
+ cveDesc = elt['cve']['description']['description_data'][0]['value']
+ date = elt['lastModifiedDate']
+ accessVector = elt['impact']['baseMetricV2']['cvssV2']['accessVector']
+ cvssv2 = elt['impact']['baseMetricV2']['cvssV2']['baseScore']
+
+ try:
+ cvssv3 = elt['impact']['baseMetricV3']['cvssV3']['baseScore']
+ except:
+ cvssv3 = 0.0
+
+ c.execute("insert or replace into NVD values (?, ?, ?, ?, ?, ?)",
+ [cveId, cveDesc, cvssv2, cvssv3, date, accessVector])
+
+ for vendor in elt['cve']['affects']['vendor']['vendor_data']:
+ for product in vendor['product']['product_data']:
+ for version in product['version']['version_data']:
+ product_str = cveId+vendor['vendor_name']+product['product_name']+version['version_value']
+ hashstr = hash_djb2(product_str)
+ c.execute("insert or replace into PRODUCTS values (?, ?, ?, ?, ?, ?)",
+ [ hashstr, cveId, vendor['vendor_name'],
+ product['product_name'], version['version_value'],
+ version['version_affected']])
+
+
+
+addtask do_populate_cve_db before do_fetch
+do_populate_cve_db[nostamp] = "1"
+
+EXCLUDE_FROM_WORLD = "1"
diff --git a/meta/recipes-core/meta/cve-update-db.bb b/meta/recipes-core/meta/cve-update-db.bb
deleted file mode 100644
index ae8f1a9..0000000
--- a/meta/recipes-core/meta/cve-update-db.bb
+++ /dev/null
@@ -1,143 +0,0 @@
-SUMMARY = "Updates the NVD CVE database"
-LICENSE = "MIT"
-
-INHIBIT_DEFAULT_DEPS = "1"
-PACKAGES = ""
-
-inherit nopackages
-
-deltask do_unpack
-deltask do_patch
-deltask do_configure
-deltask do_compile
-deltask do_install
-deltask do_populate_sysroot
-
-python do_populate_cve_db() {
- """
- Update NVD database with json data feed
- """
-
- import sqlite3, urllib, shutil, gzip, re
- from datetime import date
-
- BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-"
- YEAR_START = 2002
-
- db_dir = d.getVar("DL_DIR") + '/CVE_CHECK'
- db_file = db_dir + '/nvd-json.db'
- json_tmpfile = db_dir + '/nvd.json.gz'
- proxy = d.getVar("https_proxy")
- cve_f = open(d.getVar("TMPDIR") + '/cve_check', 'a')
-
- if not os.path.isdir(db_dir):
- os.mkdir(db_dir)
-
- # Connect to database
- conn = sqlite3.connect(db_file)
- c = conn.cursor()
-
- initialize_db(c)
-
- for year in range(YEAR_START, date.today().year + 1):
- year_url = BASE_URL + str(year)
- meta_url = year_url + ".meta"
- json_url = year_url + ".json.gz"
-
- # Retrieve meta last modified date
- req = urllib.request.Request(meta_url)
- if proxy:
- req.set_proxy(proxy, 'https')
- try:
- with urllib.request.urlopen(req, timeout=1) as r:
- date_line = str(r.read().splitlines()[0])
- last_modified = re.search('lastModifiedDate:(.*)', date_line).group(1)
- except:
- cve_f.write('Warning: CVE db update error, CVE data is outdated.\n\n')
- break
-
- # Compare with current db last modified date
- c.execute("select DATE from META where YEAR = '%d'" % year)
- meta = c.fetchone()
- if not meta or meta[0] != last_modified:
- # Update db with current year json file
- req = urllib.request.Request(json_url)
- if proxy:
- req.set_proxy(proxy, 'https')
- try:
- with urllib.request.urlopen(req, timeout=1) as r, \
- open(json_tmpfile, 'wb') as tmpfile:
- shutil.copyfileobj(r, tmpfile)
- except:
- cve_f.write('Warning: CVE db update error, CVE data is outdated.\n\n')
- break
-
- with gzip.open(json_tmpfile, 'rt') as jsonfile:
- update_db(c, jsonfile)
- c.execute("insert or replace into META values (?, ?)",
- [year, last_modified])
-
- # Update success, set the date to cve_check file.
- if year == date.today().year:
- cve_f.write('CVE database update : %s\n\n' % date.today())
-
- cve_f.close()
- conn.commit()
- conn.close()
-}
-
-# DJB2 hash algorithm
-def hash_djb2(s):
- hash = 5381
- for x in s:
- hash = (( hash << 5) + hash) + ord(x)
-
- return hash & 0xFFFFFFFF
-
-def initialize_db(c):
- c.execute("CREATE TABLE IF NOT EXISTS META (YEAR INTEGER UNIQUE, DATE TEXT)")
- c.execute("CREATE TABLE IF NOT EXISTS NVD (ID TEXT UNIQUE, SUMMARY TEXT, \
- SCOREV2 TEXT, SCOREV3 TEXT, MODIFIED INTEGER, VECTOR TEXT)")
- c.execute("CREATE TABLE IF NOT EXISTS PRODUCTS (HASH INTEGER UNIQUE, ID TEXT, \
- VENDOR TEXT, PRODUCT TEXT, VERSION TEXT, OPERATOR TEXT)")
- c.execute("CREATE INDEX IF NOT EXISTS PRODUCT_IDX ON PRODUCTS \
- (PRODUCT, VERSION)")
-
-def update_db(c, json_filename):
- import json
- root = json.load(json_filename)
-
- for elt in root['CVE_Items']:
- if not elt['impact']:
- continue
-
- cveId = elt['cve']['CVE_data_meta']['ID']
- cveDesc = elt['cve']['description']['description_data'][0]['value']
- date = elt['lastModifiedDate']
- accessVector = elt['impact']['baseMetricV2']['cvssV2']['accessVector']
- cvssv2 = elt['impact']['baseMetricV2']['cvssV2']['baseScore']
-
- try:
- cvssv3 = elt['impact']['baseMetricV3']['cvssV3']['baseScore']
- except:
- cvssv3 = 0.0
-
- c.execute("insert or replace into NVD values (?, ?, ?, ?, ?, ?)",
- [cveId, cveDesc, cvssv2, cvssv3, date, accessVector])
-
- for vendor in elt['cve']['affects']['vendor']['vendor_data']:
- for product in vendor['product']['product_data']:
- for version in product['version']['version_data']:
- product_str = cveId+vendor['vendor_name']+product['product_name']+version['version_value']
- hashstr = hash_djb2(product_str)
- c.execute("insert or replace into PRODUCTS values (?, ?, ?, ?, ?, ?)",
- [ hashstr, cveId, vendor['vendor_name'],
- product['product_name'], version['version_value'],
- version['version_affected']])
-
-
-
-addtask do_populate_cve_db before do_fetch
-do_populate_cve_db[nostamp] = "1"
-
-EXCLUDE_FROM_WORLD = "1"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 12/47] cve-check: Update unpatched CVE matching
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (10 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 11/47] cve-check: Depends on cve-update-db-native Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 13/47] cve-check: remove redundant readline CVE whitelisting Mikko Rapeli
` (37 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Now that cve-update-db added CPE information to NVD database. We can
check for unpatched versions with operators '<', '<=', '>', and '>='.
(From OE-Core rev: bc0195be1b15bcffe60127bc5e8b7011a853c2ed)
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/cve-check.bbclass | 54 +++++++++++++++++++++++++++++++-----------
1 file changed, 40 insertions(+), 14 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 6ffa0c4..ffd6243 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -26,7 +26,7 @@ CVE_PRODUCT ??= "${BPN}"
CVE_VERSION ??= "${PV}"
CVE_CHECK_DB_DIR ?= "${DL_DIR}/CVE_CHECK"
-CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvd-json.db"
+CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve.db"
CVE_CHECK_LOG ?= "${T}/cve.log"
CVE_CHECK_TMP_FILE ?= "${TMPDIR}/cve_check"
@@ -189,27 +189,53 @@ def check_cves(d, patched_cves):
conn = sqlite3.connect(db_file)
c = conn.cursor()
- query = """SELECT * FROM PRODUCTS WHERE
- (PRODUCT IS '{0}' AND VERSION = '{1}' AND OPERATOR IS '=') OR
- (PRODUCT IS '{0}' AND OPERATOR IS '<=');"""
+ query = "SELECT * FROM PRODUCTS WHERE PRODUCT IS '{0}';"
+
for product in products:
for row in c.execute(query.format(product, pv)):
cve = row[1]
- version = row[4]
-
- try:
- discardVersion = LooseVersion(version) < LooseVersion(pv)
- except:
- discardVersion = True
+ version_start = row[4]
+ operator_start = row[5]
+ version_end = row[6]
+ operator_end = row[7]
if pv in cve_whitelist.get(cve, []):
bb.note("%s-%s has been whitelisted for %s" % (product, pv, cve))
elif cve in patched_cves:
bb.note("%s has been patched" % (cve))
- elif discardVersion:
- bb.debug(2, "Do not consider version %s " % (version))
else:
- cves_unpatched.append(cve)
+ if (operator_start == '=' and pv == version_start):
+ cves_unpatched.append(cve)
+ else:
+ if operator_start:
+ try:
+ to_append_start = (operator_start == '>=' and LooseVersion(pv) >= LooseVersion(version_start))
+ to_append_start |= (operator_start == '>' and LooseVersion(pv) > LooseVersion(version_start))
+ except:
+ bb.note("%s: Failed to compare %s %s %s for %s" %
+ (product, pv, operator_start, version_start, cve))
+ to_append_start = False
+ else:
+ to_append_start = False
+
+ if operator_end:
+ try:
+ to_append_end = (operator_end == '<=' and LooseVersion(pv) <= LooseVersion(version_end))
+ to_append_end |= (operator_end == '<' and LooseVersion(pv) < LooseVersion(version_end))
+ except:
+ bb.note("%s: Failed to compare %s %s %s for %s" %
+ (product, pv, operator_end, version_end, cve))
+ to_append_end = False
+ else:
+ to_append_end = False
+
+ if operator_start and operator_end:
+ to_append = to_append_start and to_append_end
+ else:
+ to_append = to_append_start or to_append_end
+
+ if to_append:
+ cves_unpatched.append(cve)
bb.debug(2, "%s-%s is not patched for %s" % (product, pv, cve))
conn.close()
@@ -217,7 +243,7 @@ def check_cves(d, patched_cves):
def get_cve_info(d, cves):
"""
- Get CVE information from the database used by cve-check-tool.
+ Get CVE information from the database.
Unfortunately the only way to get CVE info is set the output to
html (hard to parse) or query directly the database.
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 13/47] cve-check: remove redundant readline CVE whitelisting
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (11 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 12/47] cve-check: Update unpatched CVE matching Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 14/47] cve-check-tool: remove Mikko Rapeli
` (36 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
CVE-2014-2524 is a readline CVE that was fixed in 6.3patch3 onwards, but the
tooling wasn't able to detect this version. As we now ship readline 8 we don't
need to manually whitelist it, and if we did then the whitelisting should be in
the readline recipe.
(From OE-Core rev: 07bb8b25e172aa5c8ae96b6e8eb4ac901b835219)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/cve-check.bbclass | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index ffd6243..5979edf 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -41,10 +41,15 @@ CVE_CHECK_PN_WHITELIST = "\
glibc-locale \
"
-# Whitelist for CVE and version of package
-CVE_CHECK_CVE_WHITELIST = "{\
- 'CVE-2014-2524': ('6.3','5.2',), \
-}"
+# Whitelist for CVE and version of package. If a CVE is found then the PV is
+# compared with the version list, and if found the CVE is considered
+# patched.
+#
+# The value should be valid Python in this format:
+# {
+# 'CVE-2014-2524': ('6.3','5.2')
+# }
+CVE_CHECK_CVE_WHITELIST ?= "{}"
python do_cve_check () {
"""
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 14/47] cve-check-tool: remove
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (12 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 13/47] cve-check: remove redundant readline CVE whitelisting Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 15/47] glibc: exclude child recipes from CVE scanning Mikko Rapeli
` (35 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
(From OE-Core rev: 5388ed6d1378d647a65912dbd537f9ef3cb5760a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
.../cve-check-tool/cve-check-tool_5.6.4.bb | 62 ------
...01-Fix-freeing-memory-allocated-by-sqlite.patch | 50 -----
...ow-overriding-default-CA-certificate-file.patch | 215 ---------------------
...ogress-in-percent-when-downloading-CVE-db.patch | 135 -------------
...are-computed-vs-expected-sha256-digit-str.patch | 52 -----
.../check-for-malloc_trim-before-using-it.patch | 51 -----
6 files changed, 565 deletions(-)
delete mode 100644 meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb
delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-Fix-freeing-memory-allocated-by-sqlite.patch
delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-curl-allow-overriding-default-CA-certificate-file.patch
delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-print-progress-in-percent-when-downloading-CVE-db.patch
delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-update-Compare-computed-vs-expected-sha256-digit-str.patch
delete mode 100644 meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch
diff --git a/meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb b/meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb
deleted file mode 100644
index 1c84fb1..0000000
--- a/meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb
+++ /dev/null
@@ -1,62 +0,0 @@
-SUMMARY = "cve-check-tool"
-DESCRIPTION = "cve-check-tool is a tool for checking known (public) CVEs.\
-The tool will identify potentially vunlnerable software packages within Linux distributions through version matching."
-HOMEPAGE = "https://github.com/ikeydoherty/cve-check-tool"
-SECTION = "Development/Tools"
-LICENSE = "GPL-2.0+"
-LIC_FILES_CHKSUM = "file://LICENSE;md5=e8c1458438ead3c34974bc0be3a03ed6"
-
-SRC_URI = "https://github.com/ikeydoherty/${BPN}/releases/download/v${PV}/${BP}.tar.xz \
- file://check-for-malloc_trim-before-using-it.patch \
- file://0001-print-progress-in-percent-when-downloading-CVE-db.patch \
- file://0001-curl-allow-overriding-default-CA-certificate-file.patch \
- file://0001-update-Compare-computed-vs-expected-sha256-digit-str.patch \
- file://0001-Fix-freeing-memory-allocated-by-sqlite.patch \
- "
-
-SRC_URI[md5sum] = "c5f4247140fc9be3bf41491d31a34155"
-SRC_URI[sha256sum] = "b8f283be718af8d31232ac1bfc10a0378fb958aaaa49af39168f8acf501e6a5b"
-
-UPSTREAM_CHECK_URI = "https://github.com/ikeydoherty/cve-check-tool/releases"
-
-DEPENDS = "libcheck glib-2.0 json-glib curl libxml2 sqlite3 openssl ca-certificates"
-
-RDEPENDS_${PN} = "ca-certificates"
-
-inherit pkgconfig autotools
-
-EXTRA_OECONF = "--disable-coverage --enable-relative-plugins"
-CFLAGS_append = " -Wno-error=pedantic"
-
-do_populate_cve_db() {
- if [ "${BB_NO_NETWORK}" = "1" ] ; then
- bbwarn "BB_NO_NETWORK is set; Can't update cve-check-tool database, new CVEs won't be detected"
- return
- fi
-
- # In case we don't inherit cve-check class, use default values defined in the class.
- cve_dir="${CVE_CHECK_DB_DIR}"
- cve_file="${CVE_CHECK_TMP_FILE}"
-
- [ -z "${cve_dir}" ] && cve_dir="${DL_DIR}/CVE_CHECK"
- [ -z "${cve_file}" ] && cve_file="${TMPDIR}/cve_check"
-
- unused="${@bb.utils.export_proxies(d)}"
- bbdebug 2 "Updating cve-check-tool database located in $cve_dir"
- # --cacert works around curl-native not finding the CA bundle
- if cve-check-update --cacert ${sysconfdir}/ssl/certs/ca-certificates.crt -d "$cve_dir" ; then
- printf "CVE database was updated on %s UTC\n\n" "$(LANG=C date --utc +'%F %T')" > "$cve_file"
- else
- bbwarn "Error in executing cve-check-update"
- if [ "${@'1' if bb.data.inherits_class('cve-check', d) else '0'}" -ne 0 ] ; then
- bbwarn "Failed to update cve-check-tool database, CVEs won't be checked"
- fi
- fi
-}
-
-addtask populate_cve_db after do_populate_sysroot
-do_populate_cve_db[depends] = "cve-check-tool-native:do_populate_sysroot"
-do_populate_cve_db[nostamp] = "1"
-do_populate_cve_db[progress] = "percent"
-
-BBCLASSEXTEND = "native nativesdk"
diff --git a/meta/recipes-devtools/cve-check-tool/files/0001-Fix-freeing-memory-allocated-by-sqlite.patch b/meta/recipes-devtools/cve-check-tool/files/0001-Fix-freeing-memory-allocated-by-sqlite.patch
deleted file mode 100644
index 4a82cf2..0000000
--- a/meta/recipes-devtools/cve-check-tool/files/0001-Fix-freeing-memory-allocated-by-sqlite.patch
+++ /dev/null
@@ -1,50 +0,0 @@
-From a3353429652f83bb8b0316500faa88fa2555542d Mon Sep 17 00:00:00 2001
-From: Peter Marko <peter.marko@siemens.com>
-Date: Thu, 13 Apr 2017 23:09:52 +0200
-Subject: [PATCH] Fix freeing memory allocated by sqlite
-
-Upstream-Status: Backport
-Signed-off-by: Peter Marko <peter.marko@siemens.com>
----
- src/core.c | 8 ++++----
- 1 file changed, 4 insertions(+), 4 deletions(-)
-
-diff --git a/src/core.c b/src/core.c
-index 6263031..6788f16 100644
---- a/src/core.c
-+++ b/src/core.c
-@@ -82,7 +82,7 @@ static bool ensure_table(CveDB *self)
- rc = sqlite3_exec(self->db, query, NULL, NULL, &err);
- if (rc != SQLITE_OK) {
- fprintf(stderr, "ensure_table(): %s\n", err);
-- free(err);
-+ sqlite3_free(err);
- return false;
- }
-
-@@ -91,7 +91,7 @@ static bool ensure_table(CveDB *self)
- rc = sqlite3_exec(self->db, query, NULL, NULL, &err);
- if (rc != SQLITE_OK) {
- fprintf(stderr, "ensure_table(): %s\n", err);
-- free(err);
-+ sqlite3_free(err);
- return false;
- }
-
-@@ -99,11 +99,11 @@ static bool ensure_table(CveDB *self)
- rc = sqlite3_exec(self->db, query, NULL, NULL, &err);
- if (rc != SQLITE_OK) {
- fprintf(stderr, "ensure_table(): %s\n", err);
-- free(err);
-+ sqlite3_free(err);
- return false;
- }
- if (err) {
-- free(err);
-+ sqlite3_free(err);
- }
-
- return true;
---
-2.1.4
-
diff --git a/meta/recipes-devtools/cve-check-tool/files/0001-curl-allow-overriding-default-CA-certificate-file.patch b/meta/recipes-devtools/cve-check-tool/files/0001-curl-allow-overriding-default-CA-certificate-file.patch
deleted file mode 100644
index 3d8ebd1..0000000
--- a/meta/recipes-devtools/cve-check-tool/files/0001-curl-allow-overriding-default-CA-certificate-file.patch
+++ /dev/null
@@ -1,215 +0,0 @@
-From 825a9969dea052b02ba868bdf39e676349f10dce Mon Sep 17 00:00:00 2001
-From: Jussi Kukkonen <jussi.kukkonen@intel.com>
-Date: Thu, 9 Feb 2017 14:51:28 +0200
-Subject: [PATCH] curl: allow overriding default CA certificate file
-
-Similar to curl, --cacert can now be used in cve-check-tool and
-cve-check-update to override the default CA certificate file. Useful
-in cases where the system default is unsuitable (for example,
-out-dated) or broken (as in OE's current native libcurl, which embeds
-a path string from one build host and then uses it on another although
-the right path may have become something different).
-
-Upstream-Status: Submitted [https://github.com/ikeydoherty/cve-check-tool/pull/45]
-
-Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
-
-
-Took Patrick Ohlys original patch from meta-security-isafw, rebased
-on top of other patches.
-
-Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
----
- src/library/cve-check-tool.h | 1 +
- src/library/fetch.c | 10 +++++++++-
- src/library/fetch.h | 3 ++-
- src/main.c | 5 ++++-
- src/update-main.c | 4 +++-
- src/update.c | 12 +++++++-----
- src/update.h | 2 +-
- 7 files changed, 27 insertions(+), 10 deletions(-)
-
-diff --git a/src/library/cve-check-tool.h b/src/library/cve-check-tool.h
-index e4bb5b1..f89eade 100644
---- a/src/library/cve-check-tool.h
-+++ b/src/library/cve-check-tool.h
-@@ -43,6 +43,7 @@ typedef struct CveCheckTool {
- bool bugs; /**<Whether bug tracking is enabled */
- GHashTable *mapping; /**<CVE Mapping */
- const char *output_file; /**<Output file, if any */
-+ const char *cacert_file; /**<Non-default SSL certificate file, if any */
- } CveCheckTool;
-
- /**
-diff --git a/src/library/fetch.c b/src/library/fetch.c
-index 0fe6d76..8f998c3 100644
---- a/src/library/fetch.c
-+++ b/src/library/fetch.c
-@@ -60,7 +60,8 @@ static int progress_callback_new(void *ptr, curl_off_t dltotal, curl_off_t dlnow
- }
-
- FetchStatus fetch_uri(const char *uri, const char *target, bool verbose,
-- unsigned int start_percent, unsigned int end_percent)
-+ unsigned int start_percent, unsigned int end_percent,
-+ const char *cacert_file)
- {
- FetchStatus ret = FETCH_STATUS_FAIL;
- CURLcode res;
-@@ -74,6 +75,13 @@ FetchStatus fetch_uri(const char *uri, const char *target, bool verbose,
- return ret;
- }
-
-+ if (cacert_file) {
-+ res = curl_easy_setopt(curl, CURLOPT_CAINFO, cacert_file);
-+ if (res != CURLE_OK) {
-+ goto bail;
-+ }
-+ }
-+
- if (stat(target, &st) == 0) {
- res = curl_easy_setopt(curl, CURLOPT_TIMECONDITION, CURL_TIMECOND_IFMODSINCE);
- if (res != CURLE_OK) {
-diff --git a/src/library/fetch.h b/src/library/fetch.h
-index 4cce5d1..836c7d7 100644
---- a/src/library/fetch.h
-+++ b/src/library/fetch.h
-@@ -29,7 +29,8 @@ typedef enum {
- * @return A FetchStatus, indicating the operation taken
- */
- FetchStatus fetch_uri(const char *uri, const char *target, bool verbose,
-- unsigned int this_percent, unsigned int next_percent);
-+ unsigned int this_percent, unsigned int next_percent,
-+ const char *cacert_file);
-
- /**
- * Attempt to extract the given gzipped file
-diff --git a/src/main.c b/src/main.c
-index 8e6f158..ae69d47 100644
---- a/src/main.c
-+++ b/src/main.c
-@@ -280,6 +280,7 @@ static bool csv_mode = false;
- static char *modified_stamp = NULL;
- static gchar *mapping_file = NULL;
- static gchar *output_file = NULL;
-+static gchar *cacert_file = NULL;
-
- static GOptionEntry _entries[] = {
- { "not-patched", 'n', 0, G_OPTION_ARG_NONE, &hide_patched, "Hide patched/addressed CVEs", NULL },
-@@ -294,6 +295,7 @@ static GOptionEntry _entries[] = {
- { "csv", 'c', 0, G_OPTION_ARG_NONE, &csv_mode, "Output CSV formatted data only", NULL },
- { "mapping", 'M', 0, G_OPTION_ARG_STRING, &mapping_file, "Path to a mapping file", NULL},
- { "output-file", 'o', 0, G_OPTION_ARG_STRING, &output_file, "Path to the output file (output plugin specific)", NULL},
-+ { "cacert", 'C', 0, G_OPTION_ARG_STRING, &cacert_file, "Path to the combined SSL certificates file (system default is used if not set)", NULL},
- { .short_name = 0 }
- };
-
-@@ -492,6 +494,7 @@ int main(int argc, char **argv)
-
- quiet = csv_mode || !no_html;
- self->output_file = output_file;
-+ self->cacert_file = cacert_file;
-
- if (!csv_mode && self->output_file) {
- quiet = false;
-@@ -530,7 +533,7 @@ int main(int argc, char **argv)
- if (status) {
- fprintf(stderr, "Update of db forced\n");
- cve_db_unlock();
-- if (!update_db(quiet, db_path->str)) {
-+ if (!update_db(quiet, db_path->str, self->cacert_file)) {
- fprintf(stderr, "DB update failure\n");
- goto cleanup;
- }
-diff --git a/src/update-main.c b/src/update-main.c
-index 2379cfa..c52d9d0 100644
---- a/src/update-main.c
-+++ b/src/update-main.c
-@@ -43,11 +43,13 @@ the Free Software Foundation; either version 2 of the License, or\n\
- static gchar *nvds = NULL;
- static bool _show_version = false;
- static bool _quiet = false;
-+static const char *_cacert_file = NULL;
-
- static GOptionEntry _entries[] = {
- { "nvd-dir", 'd', 0, G_OPTION_ARG_STRING, &nvds, "NVD directory in filesystem", NULL },
- { "version", 'v', 0, G_OPTION_ARG_NONE, &_show_version, "Show version", NULL },
- { "quiet", 'q', 0, G_OPTION_ARG_NONE, &_quiet, "Run silently", NULL },
-+ { "cacert", 'C', 0, G_OPTION_ARG_STRING, &_cacert_file, "Path to the combined SSL certificates file (system default is used if not set)", NULL},
- { .short_name = 0 }
- };
-
-@@ -88,7 +90,7 @@ int main(int argc, char **argv)
- goto end;
- }
-
-- if (update_db(_quiet, db_path->str)) {
-+ if (update_db(_quiet, db_path->str, _cacert_file)) {
- ret = EXIT_SUCCESS;
- } else {
- fprintf(stderr, "Failed to update database\n");
-diff --git a/src/update.c b/src/update.c
-index 070560a..8cb4a39 100644
---- a/src/update.c
-+++ b/src/update.c
-@@ -267,7 +267,8 @@ static inline void update_end(int fd, const char *update_fname, bool ok)
-
- static int do_fetch_update(int year, const char *db_dir, CveDB *cve_db,
- bool db_exist, bool verbose,
-- unsigned int this_percent, unsigned int next_percent)
-+ unsigned int this_percent, unsigned int next_percent,
-+ const char *cacert_file)
- {
- const char nvd_uri[] = URI_PREFIX;
- autofree(cve_string) *uri_meta = NULL;
-@@ -331,14 +332,14 @@ refetch:
- }
-
- /* Fetch NVD META file */
-- st = fetch_uri(uri_meta->str, nvdcve_meta->str, verbose, this_percent, this_percent);
-+ st = fetch_uri(uri_meta->str, nvdcve_meta->str, verbose, this_percent, this_percent, cacert_file);
- if (st == FETCH_STATUS_FAIL) {
- fprintf(stderr, "Failed to fetch %s\n", uri_meta->str);
- return -1;
- }
-
- /* Fetch NVD XML file */
-- st = fetch_uri(uri_data_gz->str, nvdcve_data_gz->str, verbose, this_percent, next_percent);
-+ st = fetch_uri(uri_data_gz->str, nvdcve_data_gz->str, verbose, this_percent, next_percent, cacert_file);
- switch (st) {
- case FETCH_STATUS_FAIL:
- fprintf(stderr, "Failed to fetch %s\n", uri_data_gz->str);
-@@ -391,7 +392,7 @@ refetch:
- return 0;
- }
-
--bool update_db(bool quiet, const char *db_file)
-+bool update_db(bool quiet, const char *db_file, const char *cacert_file)
- {
- autofree(char) *db_dir = NULL;
- autofree(CveDB) *cve_db = NULL;
-@@ -466,7 +467,8 @@ bool update_db(bool quiet, const char *db_file)
- if (!quiet)
- fprintf(stderr, "completed: %u%%\r", start_percent);
- rc = do_fetch_update(y, db_dir, cve_db, db_exist, !quiet,
-- start_percent, end_percent);
-+ start_percent, end_percent,
-+ cacert_file);
- switch (rc) {
- case 0:
- if (!quiet)
-diff --git a/src/update.h b/src/update.h
-index b8e9911..ceea0c3 100644
---- a/src/update.h
-+++ b/src/update.h
-@@ -15,7 +15,7 @@ cve_string *get_db_path(const char *path);
-
- int update_required(const char *db_file);
-
--bool update_db(bool quiet, const char *db_file);
-+bool update_db(bool quiet, const char *db_file, const char *cacert_file);
-
-
- /*
---
-2.1.4
-
diff --git a/meta/recipes-devtools/cve-check-tool/files/0001-print-progress-in-percent-when-downloading-CVE-db.patch b/meta/recipes-devtools/cve-check-tool/files/0001-print-progress-in-percent-when-downloading-CVE-db.patch
deleted file mode 100644
index 8ea6f68..0000000
--- a/meta/recipes-devtools/cve-check-tool/files/0001-print-progress-in-percent-when-downloading-CVE-db.patch
+++ /dev/null
@@ -1,135 +0,0 @@
-From e9ed26cde63f8ca7607a010a518329339f8c02d3 Mon Sep 17 00:00:00 2001
-From: =?UTF-8?q?Andr=C3=A9=20Draszik?= <git@andred.net>
-Date: Mon, 26 Sep 2016 12:12:41 +0100
-Subject: [PATCH] print progress in percent when downloading CVE db
-MIME-Version: 1.0
-Content-Type: text/plain; charset=UTF-8
-Content-Transfer-Encoding: 8bit
-
-Upstream-Status: Pending
-Signed-off-by: André Draszik <git@andred.net>
----
- src/library/fetch.c | 28 +++++++++++++++++++++++++++-
- src/library/fetch.h | 3 ++-
- src/update.c | 16 ++++++++++++----
- 3 files changed, 41 insertions(+), 6 deletions(-)
-
-diff --git a/src/library/fetch.c b/src/library/fetch.c
-index 06d4b30..0fe6d76 100644
---- a/src/library/fetch.c
-+++ b/src/library/fetch.c
-@@ -37,13 +37,37 @@ static size_t write_func(void *ptr, size_t size, size_t nmemb, struct fetch_t *f
- return fwrite(ptr, size, nmemb, f->f);
- }
-
--FetchStatus fetch_uri(const char *uri, const char *target, bool verbose)
-+struct percent_t {
-+ unsigned int start;
-+ unsigned int end;
-+};
-+
-+static int progress_callback_new(void *ptr, curl_off_t dltotal, curl_off_t dlnow, curl_off_t ultotal, curl_off_t ulnow)
-+{
-+ (void) ultotal;
-+ (void) ulnow;
-+
-+ struct percent_t *percent = (struct percent_t *) ptr;
-+
-+ if (dltotal && percent && percent->end >= percent->start) {
-+ unsigned int diff = percent->end - percent->start;
-+ if (diff) {
-+ fprintf(stderr,"completed: %"CURL_FORMAT_CURL_OFF_T"%%\r", percent->start + (diff * dlnow / dltotal));
-+ }
-+ }
-+
-+ return 0;
-+}
-+
-+FetchStatus fetch_uri(const char *uri, const char *target, bool verbose,
-+ unsigned int start_percent, unsigned int end_percent)
- {
- FetchStatus ret = FETCH_STATUS_FAIL;
- CURLcode res;
- struct stat st;
- CURL *curl = NULL;
- struct fetch_t *f = NULL;
-+ struct percent_t percent = { .start = start_percent, .end = end_percent };
-
- curl = curl_easy_init();
- if (!curl) {
-@@ -67,6 +91,8 @@ FetchStatus fetch_uri(const char *uri, const char *target, bool verbose)
- }
- if (verbose) {
- (void)curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0L);
-+ (void)curl_easy_setopt(curl, CURLOPT_XFERINFODATA, &percent);
-+ (void)curl_easy_setopt(curl, CURLOPT_XFERINFOFUNCTION, progress_callback_new);
- }
- res = curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, (curl_write_callback)write_func);
- if (res != CURLE_OK) {
-diff --git a/src/library/fetch.h b/src/library/fetch.h
-index 70c3779..4cce5d1 100644
---- a/src/library/fetch.h
-+++ b/src/library/fetch.h
-@@ -28,7 +28,8 @@ typedef enum {
- * @param verbose Whether to be verbose
- * @return A FetchStatus, indicating the operation taken
- */
--FetchStatus fetch_uri(const char *uri, const char *target, bool verbose);
-+FetchStatus fetch_uri(const char *uri, const char *target, bool verbose,
-+ unsigned int this_percent, unsigned int next_percent);
-
- /**
- * Attempt to extract the given gzipped file
-diff --git a/src/update.c b/src/update.c
-index 30fbe96..eaeeefd 100644
---- a/src/update.c
-+++ b/src/update.c
-@@ -266,7 +266,8 @@ static inline void update_end(int fd, const char *update_fname, bool ok)
- }
-
- static int do_fetch_update(int year, const char *db_dir, CveDB *cve_db,
-- bool db_exist, bool verbose)
-+ bool db_exist, bool verbose,
-+ unsigned int this_percent, unsigned int next_percent)
- {
- const char nvd_uri[] = URI_PREFIX;
- autofree(cve_string) *uri_meta = NULL;
-@@ -330,14 +331,14 @@ refetch:
- }
-
- /* Fetch NVD META file */
-- st = fetch_uri(uri_meta->str, nvdcve_meta->str, verbose);
-+ st = fetch_uri(uri_meta->str, nvdcve_meta->str, verbose, this_percent, this_percent);
- if (st == FETCH_STATUS_FAIL) {
- fprintf(stderr, "Failed to fetch %s\n", uri_meta->str);
- return -1;
- }
-
- /* Fetch NVD XML file */
-- st = fetch_uri(uri_data_gz->str, nvdcve_data_gz->str, verbose);
-+ st = fetch_uri(uri_data_gz->str, nvdcve_data_gz->str, verbose, this_percent, next_percent);
- switch (st) {
- case FETCH_STATUS_FAIL:
- fprintf(stderr, "Failed to fetch %s\n", uri_data_gz->str);
-@@ -459,10 +460,17 @@ bool update_db(bool quiet, const char *db_file)
- for (int i = YEAR_START; i <= year+1; i++) {
- int y = i > year ? -1 : i;
- int rc;
-+ unsigned int start_percent = ((i+0 - YEAR_START) * 100) / (year+2 - YEAR_START);
-+ unsigned int end_percent = ((i+1 - YEAR_START) * 100) / (year+2 - YEAR_START);
-
-- rc = do_fetch_update(y, db_dir, cve_db, db_exist, !quiet);
-+ if (!quiet)
-+ fprintf(stderr, "completed: %u%%\r", start_percent);
-+ rc = do_fetch_update(y, db_dir, cve_db, db_exist, !quiet,
-+ start_percent, end_percent);
- switch (rc) {
- case 0:
-+ if (!quiet)
-+ fprintf(stderr,"completed: %u%%\r", end_percent);
- continue;
- case ENOMEM:
- goto oom;
---
-2.9.3
-
diff --git a/meta/recipes-devtools/cve-check-tool/files/0001-update-Compare-computed-vs-expected-sha256-digit-str.patch b/meta/recipes-devtools/cve-check-tool/files/0001-update-Compare-computed-vs-expected-sha256-digit-str.patch
deleted file mode 100644
index 458c0cc..0000000
--- a/meta/recipes-devtools/cve-check-tool/files/0001-update-Compare-computed-vs-expected-sha256-digit-str.patch
+++ /dev/null
@@ -1,52 +0,0 @@
-From b0426e63c9ac61657e029f689bcb8dd051e752c6 Mon Sep 17 00:00:00 2001
-From: Sergey Popovich <popovich_sergei@mail.ua>
-Date: Fri, 21 Apr 2017 07:32:23 -0700
-Subject: [PATCH] update: Compare computed vs expected sha256 digit string
- ignoring case
-
-We produce sha256 digest string using %x snprintf()
-qualifier for each byte of digest which uses alphabetic
-characters from "a" to "f" in lower case to represent
-integer values from 10 to 15.
-
-Previously all of the NVD META files supply sha256
-digest string for corresponding XML file in lower case.
-
-However due to some reason this changed recently to
-provide digest digits in upper case causing fetched
-data consistency checks to fail. This prevents database
-from being updated periodically.
-
-While commit c4f6e94 (update: Do not treat sha256 failure
-as fatal if requested) adds useful option to skip
-digest validation at all and thus provides workaround for
-this situation, it might be unacceptable for some
-deployments where we need to ensure that downloaded
-data is consistent before start parsing it and update
-SQLite database.
-
-Use strcasecmp() to compare two digest strings case
-insensitively and addressing this case.
-
-Upstream-Status: Backport
-Signed-off-by: Sergey Popovich <popovich_sergei@mail.ua>
----
- src/update.c | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/src/update.c b/src/update.c
-index 8588f38..3cc6b67 100644
---- a/src/update.c
-+++ b/src/update.c
-@@ -187,7 +187,7 @@ static bool nvdcve_data_ok(const char *meta, const char *data)
- snprintf(&csum_data[idx], len, "%02hhx", digest[i]);
- }
-
-- ret = streq(csum_meta, csum_data);
-+ ret = !strcasecmp(csum_meta, csum_data);
-
- err_unmap:
- munmap(buffer, length);
---
-2.11.0
-
diff --git a/meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch b/meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch
deleted file mode 100644
index 0774ad9..0000000
--- a/meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch
+++ /dev/null
@@ -1,51 +0,0 @@
-From ce64633b9733e962b8d8482244301f614d8b5845 Mon Sep 17 00:00:00 2001
-From: Khem Raj <raj.khem@gmail.com>
-Date: Mon, 22 Aug 2016 22:54:24 -0700
-Subject: [PATCH] Check for malloc_trim before using it
-
-malloc_trim is gnu specific and not all libc
-implement it, threfore write a configure check
-to poke for it first and use the define to
-guard its use.
-
-Helps in compiling on musl based systems
-
-Signed-off-by: Khem Raj <raj.khem@gmail.com>
----
-Upstream-Status: Submitted [https://github.com/ikeydoherty/cve-check-tool/pull/48]
- configure.ac | 2 ++
- src/core.c | 4 ++--
- 2 files changed, 4 insertions(+), 2 deletions(-)
-
-diff --git a/configure.ac b/configure.ac
-index d3b66ce..79c3542 100644
---- a/configure.ac
-+++ b/configure.ac
-@@ -19,6 +19,8 @@ m4_define([json_required_version], [0.16.0])
- m4_define([openssl_required_version],[1.0.0])
- # TODO: Set minimum sqlite
-
-+AC_CHECK_FUNCS_ONCE(malloc_trim)
-+
- PKG_CHECK_MODULES(CVE_CHECK_TOOL,
- [
- glib-2.0 >= glib_required_version,
-diff --git a/src/core.c b/src/core.c
-index 6263031..0d5df29 100644
---- a/src/core.c
-+++ b/src/core.c
-@@ -498,9 +498,9 @@ bool cve_db_load(CveDB *self, const char *fname)
- }
-
- b = true;
--
-+#ifdef HAVE_MALLOC_TRIM
- malloc_trim(0);
--
-+#endif
- xmlFreeTextReader(r);
- if (fd) {
- close(fd);
---
-2.9.3
-
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 15/47] glibc: exclude child recipes from CVE scanning
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (13 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 14/47] cve-check-tool: remove Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 16/47] cve-check.bbclass: initialize to_append Mikko Rapeli
` (34 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
As glibc will be scanned for CVEs, we don't need to scan glibc-locale,
glibc-mtrace, and glibc-scripts which are all separate recipes for technical
reasons.
Exclude the recipes by setting CVE_PRODUCT in the recipe, instead of using the
global whitelist.
(From OE-Core rev: 1f9a963b9ff7ebe052ba54b9fcbdf7d09478dd17)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/cve-check.bbclass | 4 +---
meta/recipes-core/glibc/glibc-locale.inc | 3 +++
meta/recipes-core/glibc/glibc-mtrace.inc | 3 +++
meta/recipes-core/glibc/glibc-scripts.inc | 3 +++
4 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 5979edf..19ac48c 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -37,9 +37,7 @@ CVE_CHECK_COPY_FILES ??= "1"
CVE_CHECK_CREATE_MANIFEST ??= "1"
# Whitelist for packages (PN)
-CVE_CHECK_PN_WHITELIST = "\
- glibc-locale \
-"
+CVE_CHECK_PN_WHITELIST ?= ""
# Whitelist for CVE and version of package. If a CVE is found then the PV is
# compared with the version list, and if found the CVE is considered
diff --git a/meta/recipes-core/glibc/glibc-locale.inc b/meta/recipes-core/glibc/glibc-locale.inc
index e50e5cf..06edcfe 100644
--- a/meta/recipes-core/glibc/glibc-locale.inc
+++ b/meta/recipes-core/glibc/glibc-locale.inc
@@ -95,3 +95,6 @@ do_install () {
inherit libc-package
BBCLASSEXTEND = "nativesdk"
+
+# Don't scan for CVEs as glibc will be scanned
+CVE_PRODUCT = ""
diff --git a/meta/recipes-core/glibc/glibc-mtrace.inc b/meta/recipes-core/glibc/glibc-mtrace.inc
index d703c14..ef9d60e 100644
--- a/meta/recipes-core/glibc/glibc-mtrace.inc
+++ b/meta/recipes-core/glibc/glibc-mtrace.inc
@@ -11,3 +11,6 @@ do_install() {
install -d -m 0755 ${D}${bindir}
install -m 0755 ${SRC}/mtrace ${D}${bindir}/
}
+
+# Don't scan for CVEs as glibc will be scanned
+CVE_PRODUCT = ""
diff --git a/meta/recipes-core/glibc/glibc-scripts.inc b/meta/recipes-core/glibc/glibc-scripts.inc
index 2a2b415..14a14e4 100644
--- a/meta/recipes-core/glibc/glibc-scripts.inc
+++ b/meta/recipes-core/glibc/glibc-scripts.inc
@@ -18,3 +18,6 @@ do_install() {
# sotruss script requires sotruss-lib.so (given by libsotruss package),
# to produce trace of the library calls.
RDEPENDS_${PN} += "libsotruss"
+
+# Don't scan for CVEs as glibc will be scanned
+CVE_PRODUCT = ""
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 16/47] cve-check.bbclass: initialize to_append
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (14 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 15/47] glibc: exclude child recipes from CVE scanning Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 17/47] cve-check: allow comparison of Vendor as well as Product Mikko Rapeli
` (33 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
Fixes build failure with core-image-minimal:
Exception: UnboundLocalError: local variable 'to_append' referenced before assignment
(From OE-Core rev: 270ac00cb43d0614dfe1c95f960c76e9e5fa20d4)
Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/cve-check.bbclass | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 19ac48c..2a13816 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -207,6 +207,7 @@ def check_cves(d, patched_cves):
elif cve in patched_cves:
bb.note("%s has been patched" % (cve))
else:
+ to_append = False
if (operator_start == '=' and pv == version_start):
cves_unpatched.append(cve)
else:
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 17/47] cve-check: allow comparison of Vendor as well as Product
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (15 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 16/47] cve-check.bbclass: initialize to_append Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 18/47] cve-check: Replace CVE_CHECK_CVE_WHITELIST by CVE_CHECK_WHITELIST Mikko Rapeli
` (32 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
Some product names are too vague to be searched without also matching the
vendor, for example Flex could be the parser compiler we ship, or Adobe Flex, or
Apache Flex, or IBM Flex.
If entries in CVE_PRODUCT contain a colon then split it as vendor:product to improve the search.
Also don't use .format() to construct SQL as that can lead to security
issues. Instead, use ? placeholders and lets sqlite3 handle the escaping.
(From OE-Core rev: e6bf90009877d00243417898700d2320fd87b39c)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/cve-check.bbclass | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 2a13816..e8668b2 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -190,12 +190,16 @@ def check_cves(d, patched_cves):
import sqlite3
db_file = d.getVar("CVE_CHECK_DB_FILE")
conn = sqlite3.connect(db_file)
- c = conn.cursor()
-
- query = "SELECT * FROM PRODUCTS WHERE PRODUCT IS '{0}';"
for product in products:
- for row in c.execute(query.format(product, pv)):
+ c = conn.cursor()
+ if ":" in product:
+ vendor, product = product.split(":", 1)
+ c.execute("SELECT * FROM PRODUCTS WHERE PRODUCT IS ? AND VENDOR IS ?", (product, vendor))
+ else:
+ c.execute("SELECT * FROM PRODUCTS WHERE PRODUCT IS ?", (product,))
+
+ for row in c:
cve = row[1]
version_start = row[4]
operator_start = row[5]
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 18/47] cve-check: Replace CVE_CHECK_CVE_WHITELIST by CVE_CHECK_WHITELIST
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (16 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 17/47] cve-check: allow comparison of Vendor as well as Product Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 19/47] cve-update-db-native: use SQL placeholders instead of format strings Mikko Rapeli
` (31 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
CVE_CHECK_WHITELIST does not contain version anymore, as it was not
used. This variable should be set per recipe.
(From OE-Core rev: 7069302a4ccbb5b72e1902f284cf078516fd7294)
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/cve-check.bbclass | 22 +++++++++++-----------
1 file changed, 11 insertions(+), 11 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index e8668b2..512d4c7 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -39,15 +39,12 @@ CVE_CHECK_CREATE_MANIFEST ??= "1"
# Whitelist for packages (PN)
CVE_CHECK_PN_WHITELIST ?= ""
-# Whitelist for CVE and version of package. If a CVE is found then the PV is
-# compared with the version list, and if found the CVE is considered
-# patched.
-#
-# The value should be valid Python in this format:
-# {
-# 'CVE-2014-2524': ('6.3','5.2')
-# }
-CVE_CHECK_CVE_WHITELIST ?= "{}"
+# Whitelist for CVE. If a CVE is found, then it is considered patched.
+# The value is a string containing space separated CVE values:
+#
+# CVE_CHECK_WHITELIST = 'CVE-2014-2524 CVE-2018-1234'
+#
+CVE_CHECK_WHITELIST ?= ""
python do_cve_check () {
"""
@@ -185,7 +182,10 @@ def check_cves(d, patched_cves):
bb.note("Recipe has been whitelisted, skipping check")
return ([], [])
- cve_whitelist = ast.literal_eval(d.getVar("CVE_CHECK_CVE_WHITELIST"))
+ old_cve_whitelist = d.getVar("CVE_CHECK_CVE_WHITELIST")
+ if old_cve_whitelist:
+ bb.warn("CVE_CHECK_CVE_WHITELIST is deprecated, please use CVE_CHECK_WHITELIST.")
+ cve_whitelist = d.getVar("CVE_CHECK_WHITELIST").split()
import sqlite3
db_file = d.getVar("CVE_CHECK_DB_FILE")
@@ -206,7 +206,7 @@ def check_cves(d, patched_cves):
version_end = row[6]
operator_end = row[7]
- if pv in cve_whitelist.get(cve, []):
+ if cve in cve_whitelist:
bb.note("%s-%s has been whitelisted for %s" % (product, pv, cve))
elif cve in patched_cves:
bb.note("%s has been patched" % (cve))
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 19/47] cve-update-db-native: use SQL placeholders instead of format strings
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (17 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 18/47] cve-check: Replace CVE_CHECK_CVE_WHITELIST by CVE_CHECK_WHITELIST Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 20/47] cve-update-db: Use NVD CPE data to populate PRODUCTS table Mikko Rapeli
` (30 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
(From OE-Core rev: 91770338f76ef35f3c4eeac216eb9d2b3188e575)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-core/meta/cve-update-db-native.bb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb
index ae8f1a9..d60159b 100644
--- a/meta/recipes-core/meta/cve-update-db-native.bb
+++ b/meta/recipes-core/meta/cve-update-db-native.bb
@@ -57,7 +57,7 @@ python do_populate_cve_db() {
break
# Compare with current db last modified date
- c.execute("select DATE from META where YEAR = '%d'" % year)
+ c.execute("select DATE from META where YEAR = ?", (year,))
meta = c.fetchone()
if not meta or meta[0] != last_modified:
# Update db with current year json file
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 20/47] cve-update-db: Use NVD CPE data to populate PRODUCTS table
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (18 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 19/47] cve-update-db-native: use SQL placeholders instead of format strings Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 21/47] cve-update-db-native: Remove hash column from database Mikko Rapeli
` (29 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Instead of using expanded list of affected versions that is not
reliable, use the 'cpe_match' node in the 'configurations' json node.
For cve-check to correctly match affected CVE, the sqlite database need to
contain operator_start, operator_end and the corresponding versions fields.
(From OE-Core rev: f7676e9a38d595564922e5f59acbc69c2109a78f)
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-core/meta/cve-update-db-native.bb | 88 ++++++++++++++++++++++----
1 file changed, 74 insertions(+), 14 deletions(-)
diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb
index d60159b..cd27044 100644
--- a/meta/recipes-core/meta/cve-update-db-native.bb
+++ b/meta/recipes-core/meta/cve-update-db-native.bb
@@ -25,7 +25,7 @@ python do_populate_cve_db() {
YEAR_START = 2002
db_dir = d.getVar("DL_DIR") + '/CVE_CHECK'
- db_file = db_dir + '/nvd-json.db'
+ db_file = db_dir + '/nvdcve.db'
json_tmpfile = db_dir + '/nvd.json.gz'
proxy = d.getVar("https_proxy")
cve_f = open(d.getVar("TMPDIR") + '/cve_check', 'a')
@@ -99,9 +99,76 @@ def initialize_db(c):
c.execute("CREATE TABLE IF NOT EXISTS NVD (ID TEXT UNIQUE, SUMMARY TEXT, \
SCOREV2 TEXT, SCOREV3 TEXT, MODIFIED INTEGER, VECTOR TEXT)")
c.execute("CREATE TABLE IF NOT EXISTS PRODUCTS (HASH INTEGER UNIQUE, ID TEXT, \
- VENDOR TEXT, PRODUCT TEXT, VERSION TEXT, OPERATOR TEXT)")
- c.execute("CREATE INDEX IF NOT EXISTS PRODUCT_IDX ON PRODUCTS \
- (PRODUCT, VERSION)")
+ VENDOR TEXT, PRODUCT TEXT, VERSION_START TEXT, OPERATOR_START TEXT, \
+ VERSION_END TEXT, OPERATOR_END TEXT)")
+
+def insert_elt(c, db_values):
+ product_str = db_values[0] + db_values[1] + db_values[2] + db_values[3]
+ hashstr = hash_djb2(product_str)
+ db_values.insert(0, hashstr)
+ query = "insert or replace into PRODUCTS values (?, ?, ?, ?, ?, ?, ?, ?)"
+ c.execute(query, db_values)
+
+def parse_node_and_insert(c, node, cveId):
+ # Parse children node if needed
+ try:
+ for child in node['children']:
+ parse_node_and_insert(c, child, cveId)
+ except:
+ pass
+
+ # Exit if the cpe_match node does not exists
+ try:
+ cpe_match = node['cpe_match']
+ except:
+ return
+
+ for cpe in cpe_match:
+ if not cpe['vulnerable']:
+ return
+ cpe23 = cpe['cpe23Uri'].split(':')
+ vendor = cpe23[3]
+ product = cpe23[4]
+ version = cpe23[5]
+
+ if version != '*':
+ # Version is defined, this is a '=' match
+ db_values = [cveId, vendor, product, version, '=', '', '']
+ insert_elt(c, db_values)
+ else:
+ # Parse start version, end version and operators
+ op_start = ''
+ op_end = ''
+ v_start = ''
+ v_end = ''
+
+ try:
+ if cpe['versionStartIncluding']:
+ op_start = '>='
+ v_start = cpe['versionStartIncluding']
+ except:
+ pass
+ try:
+ if cpe['versionStartExcluding']:
+ op_start = '>'
+ v_start = cpe['versionStartExcluding']
+ except:
+ pass
+ try:
+ if cpe['versionEndIncluding']:
+ op_end = '<='
+ v_end = cpe['versionEndIncluding']
+ except:
+ pass
+ try:
+ if cpe['versionEndExcluding']:
+ op_end = '<'
+ v_end = cpe['versionEndExcluding']
+ except:
+ pass
+
+ db_values = [cveId, vendor, product, v_start, op_start, v_end, op_end]
+ insert_elt(c, db_values)
def update_db(c, json_filename):
import json
@@ -125,16 +192,9 @@ def update_db(c, json_filename):
c.execute("insert or replace into NVD values (?, ?, ?, ?, ?, ?)",
[cveId, cveDesc, cvssv2, cvssv3, date, accessVector])
- for vendor in elt['cve']['affects']['vendor']['vendor_data']:
- for product in vendor['product']['product_data']:
- for version in product['version']['version_data']:
- product_str = cveId+vendor['vendor_name']+product['product_name']+version['version_value']
- hashstr = hash_djb2(product_str)
- c.execute("insert or replace into PRODUCTS values (?, ?, ?, ?, ?, ?)",
- [ hashstr, cveId, vendor['vendor_name'],
- product['product_name'], version['version_value'],
- version['version_affected']])
-
+ configurations = elt['configurations']['nodes']
+ for config in configurations:
+ parse_node_and_insert(c, config, cveId)
addtask do_populate_cve_db before do_fetch
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 21/47] cve-update-db-native: Remove hash column from database.
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (19 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 20/47] cve-update-db: Use NVD CPE data to populate PRODUCTS table Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 22/47] cve-update-db-native: use os.path.join instead of + Mikko Rapeli
` (28 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
djb2 hash algorithm was found to do collisions, so the database was
sometime missing data. Remove this hash mechanism, clear and populate
elements from scratch in PRODUCTS table if the current year needs an
update.
(From OE-Core rev: 78de2cb39d74b030cd4ec811bf6f9a6daa003d19)
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/cve-check.bbclass | 12 ++++++------
meta/recipes-core/meta/cve-update-db-native.bb | 21 +++++++--------------
2 files changed, 13 insertions(+), 20 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 512d4c7..c00d291 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -26,7 +26,7 @@ CVE_PRODUCT ??= "${BPN}"
CVE_VERSION ??= "${PV}"
CVE_CHECK_DB_DIR ?= "${DL_DIR}/CVE_CHECK"
-CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve.db"
+CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve_1.0.db"
CVE_CHECK_LOG ?= "${T}/cve.log"
CVE_CHECK_TMP_FILE ?= "${TMPDIR}/cve_check"
@@ -200,11 +200,11 @@ def check_cves(d, patched_cves):
c.execute("SELECT * FROM PRODUCTS WHERE PRODUCT IS ?", (product,))
for row in c:
- cve = row[1]
- version_start = row[4]
- operator_start = row[5]
- version_end = row[6]
- operator_end = row[7]
+ cve = row[0]
+ version_start = row[3]
+ operator_start = row[4]
+ version_end = row[5]
+ operator_end = row[6]
if cve in cve_whitelist:
bb.note("%s-%s has been whitelisted for %s" % (product, pv, cve))
diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb
index cd27044..af2946b 100644
--- a/meta/recipes-core/meta/cve-update-db-native.bb
+++ b/meta/recipes-core/meta/cve-update-db-native.bb
@@ -25,7 +25,7 @@ python do_populate_cve_db() {
YEAR_START = 2002
db_dir = d.getVar("DL_DIR") + '/CVE_CHECK'
- db_file = db_dir + '/nvdcve.db'
+ db_file = db_dir + '/nvdcve_1.0.db'
json_tmpfile = db_dir + '/nvd.json.gz'
proxy = d.getVar("https_proxy")
cve_f = open(d.getVar("TMPDIR") + '/cve_check', 'a')
@@ -60,6 +60,10 @@ python do_populate_cve_db() {
c.execute("select DATE from META where YEAR = ?", (year,))
meta = c.fetchone()
if not meta or meta[0] != last_modified:
+ # Clear products table entries corresponding to current year
+ cve_year = 'CVE-' + str(year) + '%'
+ c.execute("delete from PRODUCTS where ID like ?", (cve_year,))
+
# Update db with current year json file
req = urllib.request.Request(json_url)
if proxy:
@@ -86,27 +90,16 @@ python do_populate_cve_db() {
conn.close()
}
-# DJB2 hash algorithm
-def hash_djb2(s):
- hash = 5381
- for x in s:
- hash = (( hash << 5) + hash) + ord(x)
-
- return hash & 0xFFFFFFFF
-
def initialize_db(c):
c.execute("CREATE TABLE IF NOT EXISTS META (YEAR INTEGER UNIQUE, DATE TEXT)")
c.execute("CREATE TABLE IF NOT EXISTS NVD (ID TEXT UNIQUE, SUMMARY TEXT, \
SCOREV2 TEXT, SCOREV3 TEXT, MODIFIED INTEGER, VECTOR TEXT)")
- c.execute("CREATE TABLE IF NOT EXISTS PRODUCTS (HASH INTEGER UNIQUE, ID TEXT, \
+ c.execute("CREATE TABLE IF NOT EXISTS PRODUCTS (ID TEXT, \
VENDOR TEXT, PRODUCT TEXT, VERSION_START TEXT, OPERATOR_START TEXT, \
VERSION_END TEXT, OPERATOR_END TEXT)")
def insert_elt(c, db_values):
- product_str = db_values[0] + db_values[1] + db_values[2] + db_values[3]
- hashstr = hash_djb2(product_str)
- db_values.insert(0, hashstr)
- query = "insert or replace into PRODUCTS values (?, ?, ?, ?, ?, ?, ?, ?)"
+ query = "insert into PRODUCTS values (?, ?, ?, ?, ?, ?, ?)"
c.execute(query, db_values)
def parse_node_and_insert(c, node, cveId):
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 22/47] cve-update-db-native: use os.path.join instead of +
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (20 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 21/47] cve-update-db-native: Remove hash column from database Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 23/47] cve-update-db: actually inherit native Mikko Rapeli
` (27 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
(From OE-Core rev: 4b301030cf9cf7a981dcff85a50e915c045e3130)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-core/meta/cve-update-db-native.bb | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb
index af2946b..35f7472 100644
--- a/meta/recipes-core/meta/cve-update-db-native.bb
+++ b/meta/recipes-core/meta/cve-update-db-native.bb
@@ -24,11 +24,11 @@ python do_populate_cve_db() {
BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-"
YEAR_START = 2002
- db_dir = d.getVar("DL_DIR") + '/CVE_CHECK'
- db_file = db_dir + '/nvdcve_1.0.db'
- json_tmpfile = db_dir + '/nvd.json.gz'
+ db_dir = os.path.join(d.getVar("DL_DIR"), 'CVE_CHECK')
+ db_file = os.path.join(db_dir, 'nvdcve_1.0.db')
+ json_tmpfile = os.path.join(db_dir, 'nvd.json.gz')
proxy = d.getVar("https_proxy")
- cve_f = open(d.getVar("TMPDIR") + '/cve_check', 'a')
+ cve_f = open(os.path.join(d.getVar("TMPDIR"), 'cve_check'), 'a')
if not os.path.isdir(db_dir):
os.mkdir(db_dir)
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 23/47] cve-update-db: actually inherit native
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (21 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 22/47] cve-update-db-native: use os.path.join instead of + Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 24/47] cve-update-db-native: use executemany() to optimise CPE insertion Mikko Rapeli
` (26 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
The recipe was called -native but didn't inherit native.
(From OE-Core rev: f0d822fad2a163d1ee32ed3b4c0359245140e19b)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-core/meta/cve-update-db-native.bb | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb
index 35f7472..9470cbe 100644
--- a/meta/recipes-core/meta/cve-update-db-native.bb
+++ b/meta/recipes-core/meta/cve-update-db-native.bb
@@ -2,9 +2,8 @@ SUMMARY = "Updates the NVD CVE database"
LICENSE = "MIT"
INHIBIT_DEFAULT_DEPS = "1"
-PACKAGES = ""
-inherit nopackages
+inherit native
deltask do_unpack
deltask do_patch
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 24/47] cve-update-db-native: use executemany() to optimise CPE insertion
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (22 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 23/47] cve-update-db: actually inherit native Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 25/47] cve-update-db-native: improve metadata parsing Mikko Rapeli
` (25 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
Instead of calling execute() repeatedly, rewrite the function to be a generator
and use executemany() for performance.
(From OE-Core rev: b309840b6aa3423b909a43499356e929c8761318)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-core/meta/cve-update-db-native.bb | 85 ++++++++++----------------
1 file changed, 32 insertions(+), 53 deletions(-)
diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb
index 9470cbe..a5d8e32 100644
--- a/meta/recipes-core/meta/cve-update-db-native.bb
+++ b/meta/recipes-core/meta/cve-update-db-native.bb
@@ -97,70 +97,49 @@ def initialize_db(c):
VENDOR TEXT, PRODUCT TEXT, VERSION_START TEXT, OPERATOR_START TEXT, \
VERSION_END TEXT, OPERATOR_END TEXT)")
-def insert_elt(c, db_values):
- query = "insert into PRODUCTS values (?, ?, ?, ?, ?, ?, ?)"
- c.execute(query, db_values)
-
def parse_node_and_insert(c, node, cveId):
# Parse children node if needed
- try:
- for child in node['children']:
- parse_node_and_insert(c, child, cveId)
- except:
- pass
-
- # Exit if the cpe_match node does not exists
- try:
- cpe_match = node['cpe_match']
- except:
- return
-
- for cpe in cpe_match:
- if not cpe['vulnerable']:
- return
- cpe23 = cpe['cpe23Uri'].split(':')
- vendor = cpe23[3]
- product = cpe23[4]
- version = cpe23[5]
-
- if version != '*':
- # Version is defined, this is a '=' match
- db_values = [cveId, vendor, product, version, '=', '', '']
- insert_elt(c, db_values)
- else:
- # Parse start version, end version and operators
- op_start = ''
- op_end = ''
- v_start = ''
- v_end = ''
-
- try:
- if cpe['versionStartIncluding']:
+ for child in node.get('children', ()):
+ parse_node_and_insert(c, child, cveId)
+
+ def cpe_generator():
+ for cpe in node.get('cpe_match', ()):
+ if not cpe['vulnerable']:
+ return
+ cpe23 = cpe['cpe23Uri'].split(':')
+ vendor = cpe23[3]
+ product = cpe23[4]
+ version = cpe23[5]
+
+ if version != '*':
+ # Version is defined, this is a '=' match
+ yield [cveId, vendor, product, version, '=', '', '']
+ else:
+ # Parse start version, end version and operators
+ op_start = ''
+ op_end = ''
+ v_start = ''
+ v_end = ''
+
+ if 'versionStartIncluding' in cpe:
op_start = '>='
v_start = cpe['versionStartIncluding']
- except:
- pass
- try:
- if cpe['versionStartExcluding']:
+
+ if 'versionStartExcluding' in cpe:
op_start = '>'
v_start = cpe['versionStartExcluding']
- except:
- pass
- try:
- if cpe['versionEndIncluding']:
+
+ if 'versionEndIncluding' in cpe:
op_end = '<='
v_end = cpe['versionEndIncluding']
- except:
- pass
- try:
- if cpe['versionEndExcluding']:
+
+ if 'versionEndExcluding' in cpe:
op_end = '<'
v_end = cpe['versionEndExcluding']
- except:
- pass
- db_values = [cveId, vendor, product, v_start, op_start, v_end, op_end]
- insert_elt(c, db_values)
+ yield [cveId, vendor, product, v_start, op_start, v_end, op_end]
+
+ c.executemany("insert into PRODUCTS values (?, ?, ?, ?, ?, ?, ?)", cpe_generator())
def update_db(c, json_filename):
import json
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 25/47] cve-update-db-native: improve metadata parsing
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (23 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 24/47] cve-update-db-native: use executemany() to optimise CPE insertion Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 26/47] cve-update-db-native: clean up JSON fetching Mikko Rapeli
` (24 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
The metadata parser is fragile: first it coerces a bytes() to a str() (so the
string is b'LastModifiedDate:2019...'), assumes the first line is the date, and
then uses a regex to parse (which then includes the trailing quote as part of
the date).
Clean this up by parsing the bytes as UTF-8 (ASCII is probably fine, but this is
safer), iterate through the lines and split on colons to find the right
key/value pair.
(From OE-Core rev: bb4e53af33d6ca1e9346464adbdc1b39c47530f3)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-core/meta/cve-update-db-native.bb | 18 ++++++++++--------
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb
index a5d8e32..6907197 100644
--- a/meta/recipes-core/meta/cve-update-db-native.bb
+++ b/meta/recipes-core/meta/cve-update-db-native.bb
@@ -17,7 +17,7 @@ python do_populate_cve_db() {
Update NVD database with json data feed
"""
- import sqlite3, urllib, shutil, gzip, re
+ import sqlite3, urllib, shutil, gzip
from datetime import date
BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-"
@@ -47,13 +47,15 @@ python do_populate_cve_db() {
req = urllib.request.Request(meta_url)
if proxy:
req.set_proxy(proxy, 'https')
- try:
- with urllib.request.urlopen(req, timeout=1) as r:
- date_line = str(r.read().splitlines()[0])
- last_modified = re.search('lastModifiedDate:(.*)', date_line).group(1)
- except:
- cve_f.write('Warning: CVE db update error, CVE data is outdated.\n\n')
- break
+ with urllib.request.urlopen(req) as r:
+ for l in r.read().decode("utf-8").splitlines():
+ key, value = l.split(":", 1)
+ if key == "lastModifiedDate":
+ last_modified = value
+ break
+ else:
+ bb.warn("Cannot parse CVE metadata, update failed")
+ return
# Compare with current db last modified date
c.execute("select DATE from META where YEAR = ?", (year,))
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 26/47] cve-update-db-native: clean up JSON fetching
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (24 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 25/47] cve-update-db-native: improve metadata parsing Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 27/47] cve-update-db-native: fix https proxy issues Mikko Rapeli
` (23 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
Currently the code fetches the compressed JSON, writes it to a temporary file,
uncompresses that with gzip and passes the fake file object to update_db().
Instead, uncompress the gzip'd data in memory and pass the JSON directly to
update_db().
(From OE-Core rev: 9422745979256c442f533770203f62ec071c18fb)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-core/meta/cve-update-db-native.bb | 29 +++++++++++---------------
1 file changed, 12 insertions(+), 17 deletions(-)
diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb
index 6907197..a06b74a 100644
--- a/meta/recipes-core/meta/cve-update-db-native.bb
+++ b/meta/recipes-core/meta/cve-update-db-native.bb
@@ -62,25 +62,20 @@ python do_populate_cve_db() {
meta = c.fetchone()
if not meta or meta[0] != last_modified:
# Clear products table entries corresponding to current year
- cve_year = 'CVE-' + str(year) + '%'
- c.execute("delete from PRODUCTS where ID like ?", (cve_year,))
+ c.execute("delete from PRODUCTS where ID like ?", ('CVE-%d%%' % year,))
# Update db with current year json file
- req = urllib.request.Request(json_url)
- if proxy:
- req.set_proxy(proxy, 'https')
try:
- with urllib.request.urlopen(req, timeout=1) as r, \
- open(json_tmpfile, 'wb') as tmpfile:
- shutil.copyfileobj(r, tmpfile)
- except:
+ req = urllib.request.Request(json_url)
+ if proxy:
+ req.set_proxy(proxy, 'https')
+ with urllib.request.urlopen(req) as r:
+ update_db(c, gzip.decompress(r.read()))
+ c.execute("insert or replace into META values (?, ?)", [year, last_modified])
+ except urllib.error.URLError as e:
cve_f.write('Warning: CVE db update error, CVE data is outdated.\n\n')
- break
-
- with gzip.open(json_tmpfile, 'rt') as jsonfile:
- update_db(c, jsonfile)
- c.execute("insert or replace into META values (?, ?)",
- [year, last_modified])
+ bb.warn("Cannot parse CVE data (%s), update failed" % e.reason)
+ return
# Update success, set the date to cve_check file.
if year == date.today().year:
@@ -143,9 +138,9 @@ def parse_node_and_insert(c, node, cveId):
c.executemany("insert into PRODUCTS values (?, ?, ?, ?, ?, ?, ?)", cpe_generator())
-def update_db(c, json_filename):
+def update_db(c, jsondata):
import json
- root = json.load(json_filename)
+ root = json.loads(jsondata)
for elt in root['CVE_Items']:
if not elt['impact']:
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 27/47] cve-update-db-native: fix https proxy issues
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (25 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 26/47] cve-update-db-native: clean up JSON fetching Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 28/47] cve-check: ensure all known CVEs are in the report Mikko Rapeli
` (22 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Chin Huat Ang <chin.huat.ang@intel.com>
When https_proxy is set, use proxy opener to open CVE metadata and
database URLs, otherwise fallback to the urllib.request.urlopen.
Also fix a minor issue where the json database which has been gzip
decompressed as byte object should be decoded as utf-8 string as
expected by update_db.
(From OE-Core rev: 95438d52b732bec217301fbfc2fb019bbc3707c8)
Signed-off-by: Chin Huat Ang <chin.huat.ang@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-core/meta/cve-update-db-native.bb | 41 +++++++++++++++++++-------
1 file changed, 30 insertions(+), 11 deletions(-)
diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb
index a06b74a..9fbe686 100644
--- a/meta/recipes-core/meta/cve-update-db-native.bb
+++ b/meta/recipes-core/meta/cve-update-db-native.bb
@@ -17,7 +17,7 @@ python do_populate_cve_db() {
Update NVD database with json data feed
"""
- import sqlite3, urllib, shutil, gzip
+ import sqlite3, urllib, urllib.parse, shutil, gzip
from datetime import date
BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-"
@@ -27,6 +27,16 @@ python do_populate_cve_db() {
db_file = os.path.join(db_dir, 'nvdcve_1.0.db')
json_tmpfile = os.path.join(db_dir, 'nvd.json.gz')
proxy = d.getVar("https_proxy")
+
+ if proxy:
+ # instantiate an opener but do not install it as the global
+ # opener unless if we're really sure it's applicable for all
+ # urllib requests
+ proxy_handler = urllib.request.ProxyHandler({'https': proxy})
+ proxy_opener = urllib.request.build_opener(proxy_handler)
+ else:
+ proxy_opener = None
+
cve_f = open(os.path.join(d.getVar("TMPDIR"), 'cve_check'), 'a')
if not os.path.isdir(db_dir):
@@ -44,11 +54,17 @@ python do_populate_cve_db() {
json_url = year_url + ".json.gz"
# Retrieve meta last modified date
- req = urllib.request.Request(meta_url)
- if proxy:
- req.set_proxy(proxy, 'https')
- with urllib.request.urlopen(req) as r:
- for l in r.read().decode("utf-8").splitlines():
+
+ response = None
+
+ if proxy_opener:
+ response = proxy_opener.open(meta_url)
+ else:
+ req = urllib.request.Request(meta_url)
+ response = urllib.request.urlopen(req)
+
+ if response:
+ for l in response.read().decode("utf-8").splitlines():
key, value = l.split(":", 1)
if key == "lastModifiedDate":
last_modified = value
@@ -66,11 +82,14 @@ python do_populate_cve_db() {
# Update db with current year json file
try:
- req = urllib.request.Request(json_url)
- if proxy:
- req.set_proxy(proxy, 'https')
- with urllib.request.urlopen(req) as r:
- update_db(c, gzip.decompress(r.read()))
+ if proxy_opener:
+ response = proxy_opener.open(json_url)
+ else:
+ req = urllib.request.Request(json_url)
+ response = urllib.request.urlopen(req)
+
+ if response:
+ update_db(c, gzip.decompress(response.read()).decode('utf-8'))
c.execute("insert or replace into META values (?, ?)", [year, last_modified])
except urllib.error.URLError as e:
cve_f.write('Warning: CVE db update error, CVE data is outdated.\n\n')
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 28/47] cve-check: ensure all known CVEs are in the report
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (26 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 27/47] cve-update-db-native: fix https proxy issues Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 29/47] cve-check: failure to parse versions should be more visible Mikko Rapeli
` (21 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
CVEs that are whitelisted or were not vulnerable when there are version
comparisons were not included in the report, so alter the logic to ensure that
all relevant CVEs are in the report for completeness.
(From OE-Core rev: 98256ff05fcfe9d5ccad360582c36eafb577c264)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/cve-check.bbclass | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index c00d291..f87bcc9 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -208,12 +208,14 @@ def check_cves(d, patched_cves):
if cve in cve_whitelist:
bb.note("%s-%s has been whitelisted for %s" % (product, pv, cve))
+ # TODO: this should be in the report as 'whitelisted'
+ patched_cves.add(cve)
elif cve in patched_cves:
bb.note("%s has been patched" % (cve))
else:
to_append = False
if (operator_start == '=' and pv == version_start):
- cves_unpatched.append(cve)
+ to_append = True
else:
if operator_start:
try:
@@ -243,8 +245,11 @@ def check_cves(d, patched_cves):
to_append = to_append_start or to_append_end
if to_append:
+ bb.note("%s-%s is vulnerable to %s" % (product, pv, cve))
cves_unpatched.append(cve)
- bb.debug(2, "%s-%s is not patched for %s" % (product, pv, cve))
+ else:
+ bb.note("%s-%s is not vulnerable to %s" % (product, pv, cve))
+ patched_cves.add(cve)
conn.close()
return (list(patched_cves), cves_unpatched)
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 29/47] cve-check: failure to parse versions should be more visible
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (27 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 28/47] cve-check: ensure all known CVEs are in the report Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 30/47] xserver-xorg: set CVE_PRODUCT Mikko Rapeli
` (20 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/cve-check.bbclass | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index f87bcc9..1c8b222 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -222,7 +222,7 @@ def check_cves(d, patched_cves):
to_append_start = (operator_start == '>=' and LooseVersion(pv) >= LooseVersion(version_start))
to_append_start |= (operator_start == '>' and LooseVersion(pv) > LooseVersion(version_start))
except:
- bb.note("%s: Failed to compare %s %s %s for %s" %
+ bb.warn("%s: Failed to compare %s %s %s for %s" %
(product, pv, operator_start, version_start, cve))
to_append_start = False
else:
@@ -233,7 +233,7 @@ def check_cves(d, patched_cves):
to_append_end = (operator_end == '<=' and LooseVersion(pv) <= LooseVersion(version_end))
to_append_end |= (operator_end == '<' and LooseVersion(pv) < LooseVersion(version_end))
except:
- bb.note("%s: Failed to compare %s %s %s for %s" %
+ bb.warn("%s: Failed to compare %s %s %s for %s" %
(product, pv, operator_end, version_end, cve))
to_append_end = False
else:
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 30/47] xserver-xorg: set CVE_PRODUCT
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (28 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 29/47] cve-check: failure to parse versions should be more visible Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 31/47] nasm: add CVE_PRODUCT Mikko Rapeli
` (19 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Chen Qi <Qi.Chen@windriver.com>
(From OE-Core rev: 8995f2c7d6f2f6f760811976af77e949d505a5d8)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-graphics/xorg-xserver/xserver-xorg.inc | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc b/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc
index cf2286c..ae024f0 100644
--- a/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc
+++ b/meta/recipes-graphics/xorg-xserver/xserver-xorg.inc
@@ -19,6 +19,8 @@ XORG_PN = "xorg-server"
SRC_URI = "${XORG_MIRROR}/individual/xserver/${XORG_PN}-${PV}.tar.bz2"
SRC_URI += "file://macro_tweak.patch"
+CVE_PRODUCT = "xorg-server"
+
S = "${WORKDIR}/${XORG_PN}-${PV}"
inherit autotools pkgconfig
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 31/47] nasm: add CVE_PRODUCT
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (29 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 30/47] xserver-xorg: set CVE_PRODUCT Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 32/47] dropbear: set CVE_PRODUCT Mikko Rapeli
` (18 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Chen Qi <Qi.Chen@windriver.com>
(From OE-Core rev: e61c42ee49029ae8ffec58128dd083031305d9e5)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Conflicts:
meta/recipes-devtools/nasm/nasm_2.14.02.bb
---
meta/recipes-devtools/nasm/nasm_2.13.03.bb | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-devtools/nasm/nasm_2.13.03.bb b/meta/recipes-devtools/nasm/nasm_2.13.03.bb
index de4c554..cb3745d 100644
--- a/meta/recipes-devtools/nasm/nasm_2.13.03.bb
+++ b/meta/recipes-devtools/nasm/nasm_2.13.03.bb
@@ -33,3 +33,5 @@ do_install() {
BBCLASSEXTEND = "native"
DEPENDS = "groff-native"
+
+CVE_PRODUCT = "netwide_assembler"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 32/47] dropbear: set CVE_PRODUCT
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (30 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 31/47] nasm: add CVE_PRODUCT Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 33/47] libsdl: " Mikko Rapeli
` (17 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Chen Qi <Qi.Chen@windriver.com>
(From OE-Core rev: 3c247a4a166cabf7ddfea403cf272b3fb4e00872)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-core/dropbear/dropbear.inc | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-core/dropbear/dropbear.inc b/meta/recipes-core/dropbear/dropbear.inc
index b6b436c..faf2f0e 100644
--- a/meta/recipes-core/dropbear/dropbear.inc
+++ b/meta/recipes-core/dropbear/dropbear.inc
@@ -37,6 +37,8 @@ RDEPENDS_${PN} += "${@bb.utils.contains('DISTRO_FEATURES', 'pam', '${PAM_PLUGINS
inherit autotools update-rc.d systemd
+CVE_PRODUCT = "dropbear_ssh"
+
INITSCRIPT_NAME = "dropbear"
INITSCRIPT_PARAMS = "defaults 10"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 33/47] libsdl: set CVE_PRODUCT
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (31 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 32/47] dropbear: set CVE_PRODUCT Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 34/47] ghostscript: " Mikko Rapeli
` (16 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Chen Qi <Qi.Chen@windriver.com>
(From OE-Core rev: 1f0cca19014fef24a359d400c96d178463b2760f)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Conflicts:
meta/recipes-graphics/libsdl2/libsdl2_2.0.9.bb
---
meta/recipes-graphics/libsdl/libsdl_1.2.15.bb | 2 ++
meta/recipes-graphics/libsdl2/libsdl2_2.0.8.bb | 2 ++
2 files changed, 4 insertions(+)
diff --git a/meta/recipes-graphics/libsdl/libsdl_1.2.15.bb b/meta/recipes-graphics/libsdl/libsdl_1.2.15.bb
index 3680ea9..7718d11 100644
--- a/meta/recipes-graphics/libsdl/libsdl_1.2.15.bb
+++ b/meta/recipes-graphics/libsdl/libsdl_1.2.15.bb
@@ -31,6 +31,8 @@ BINCONFIG = "${bindir}/sdl-config"
inherit autotools lib_package binconfig-disabled pkgconfig
+CVE_PRODUCT = "simple_directmedia_layer sdl"
+
EXTRA_OECONF = "--disable-static --enable-cdrom --enable-threads --enable-timers \
--enable-file --disable-oss --disable-esd --disable-arts \
--disable-diskaudio --disable-nas \
diff --git a/meta/recipes-graphics/libsdl2/libsdl2_2.0.8.bb b/meta/recipes-graphics/libsdl2/libsdl2_2.0.8.bb
index 8092fad..298d1ff 100644
--- a/meta/recipes-graphics/libsdl2/libsdl2_2.0.8.bb
+++ b/meta/recipes-graphics/libsdl2/libsdl2_2.0.8.bb
@@ -26,6 +26,8 @@ SRC_URI[sha256sum] = "edc77c57308661d576e843344d8638e025a7818bff73f8fbfab09c3c5f
inherit autotools lib_package binconfig pkgconfig
+CVE_PRODUCT = "simple_directmedia_layer sdl"
+
EXTRA_OECONF = "--disable-oss --disable-esd --disable-arts \
--disable-diskaudio --disable-nas --disable-esd-shared --disable-esdtest \
--disable-video-dummy \
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 34/47] ghostscript: set CVE_PRODUCT
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (32 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 33/47] libsdl: " Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 35/47] squashfs-tools: " Mikko Rapeli
` (15 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Chen Qi <Qi.Chen@windriver.com>
(From OE-Core rev: 721e69aa12dd9ee22618ef13f29fb6d28eeab9af)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Conflicts:
meta/recipes-extended/ghostscript/ghostscript_9.26.bb
---
meta/recipes-extended/ghostscript/ghostscript_9.21.bb | 3 +++
1 file changed, 3 insertions(+)
diff --git a/meta/recipes-extended/ghostscript/ghostscript_9.21.bb b/meta/recipes-extended/ghostscript/ghostscript_9.21.bb
index 50ec7e2..2d323c7 100644
--- a/meta/recipes-extended/ghostscript/ghostscript_9.21.bb
+++ b/meta/recipes-extended/ghostscript/ghostscript_9.21.bb
@@ -134,3 +134,6 @@ BBCLASSEXTEND = "native"
# ghostscript does not supports "arc"
COMPATIBLE_HOST = "^(?!arc).*"
+
+# some entries in NVD uses gpl_ghostscript
+CVE_PRODUCT = "ghostscript gpl_ghostscript"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 35/47] squashfs-tools: set CVE_PRODUCT
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (33 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 34/47] ghostscript: " Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 36/47] libxfont2: " Mikko Rapeli
` (14 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Chen Qi <Qi.Chen@windriver.com>
(From OE-Core rev: 8f03a33f61a94e9b8d8232283204588ce18b45a0)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-devtools/squashfs-tools/squashfs-tools_git.bb | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-devtools/squashfs-tools/squashfs-tools_git.bb b/meta/recipes-devtools/squashfs-tools/squashfs-tools_git.bb
index 1eb0154..2fda5a9 100644
--- a/meta/recipes-devtools/squashfs-tools/squashfs-tools_git.bb
+++ b/meta/recipes-devtools/squashfs-tools/squashfs-tools_git.bb
@@ -40,3 +40,5 @@ do_install () {
ARM_INSTRUCTION_SET = "arm"
BBCLASSEXTEND = "native nativesdk"
+
+CVE_PRODUCT = "squashfs"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 36/47] libxfont2: set CVE_PRODUCT
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (34 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 35/47] squashfs-tools: " Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 37/47] flex: set CVE_PRODUCT to include vendor Mikko Rapeli
` (13 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Chen Qi <Qi.Chen@windriver.com>
(From OE-Core rev: 066fa83eeaaa34e5b901dc4b82ad607d0fa78f0b)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-graphics/xorg-lib/libxfont2_2.0.3.bb | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-graphics/xorg-lib/libxfont2_2.0.3.bb b/meta/recipes-graphics/xorg-lib/libxfont2_2.0.3.bb
index 5f27a55..665abba 100644
--- a/meta/recipes-graphics/xorg-lib/libxfont2_2.0.3.bb
+++ b/meta/recipes-graphics/xorg-lib/libxfont2_2.0.3.bb
@@ -20,3 +20,5 @@ SRC_URI[sha256sum] = "0e8ab7fd737ccdfe87e1f02b55f221f0bd4503a1c5f28be4ed6a54586b
PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)}"
PACKAGECONFIG[ipv6] = "--enable-ipv6,--disable-ipv6,"
+
+CVE_PRODUCT = "libxfont libxfont2"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 37/47] flex: set CVE_PRODUCT to include vendor
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (35 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 36/47] libxfont2: " Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 38/47] webkitgtk: set CVE_PRODUCT Mikko Rapeli
` (12 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
There are many projects called Flex and they have CVEs, so also set the vendor
to remove these false positives.
(From OE-Core rev: 0598ccdcb31e16f1d1227197591b10ba441fcfe2)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-devtools/flex/flex_2.6.0.bb | 3 +++
1 file changed, 3 insertions(+)
diff --git a/meta/recipes-devtools/flex/flex_2.6.0.bb b/meta/recipes-devtools/flex/flex_2.6.0.bb
index b89b751..954fcf7 100644
--- a/meta/recipes-devtools/flex/flex_2.6.0.bb
+++ b/meta/recipes-devtools/flex/flex_2.6.0.bb
@@ -68,3 +68,6 @@ do_install_ptest() {
-e 's/^builddir = \(.*\)/builddir = ./' -e 's/^top_builddir = \(.*\)/top_builddir = ./' \
-i ${D}${PTEST_PATH}/Makefile
}
+
+# Not Apache Flex, or Adobe Flex, or IBM Flex.
+CVE_PRODUCT = "flex_project:flex"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 38/47] webkitgtk: set CVE_PRODUCT
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (36 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 37/47] flex: set CVE_PRODUCT to include vendor Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 39/47] libpam: " Mikko Rapeli
` (11 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Chen Qi <Qi.Chen@windriver.com>
(From OE-Core rev: 43aaa117386490c822b824974fb095bd0d3ce1a3)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Conflicts:
meta/recipes-sato/webkit/webkitgtk_2.24.0.bb
---
meta/recipes-sato/webkit/webkitgtk_2.18.6.bb | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-sato/webkit/webkitgtk_2.18.6.bb b/meta/recipes-sato/webkit/webkitgtk_2.18.6.bb
index c29fa7f..b88966b 100644
--- a/meta/recipes-sato/webkit/webkitgtk_2.18.6.bb
+++ b/meta/recipes-sato/webkit/webkitgtk_2.18.6.bb
@@ -32,6 +32,8 @@ inherit cmake pkgconfig gobject-introspection perlnative distro_features_check u
# depends on libxt
REQUIRED_DISTRO_FEATURES = "x11"
+CVE_PRODUCT = "webkitgtk webkitgtk\+"
+
DEPENDS = "zlib libsoup-2.4 curl libxml2 cairo libxslt libxt libidn libgcrypt \
gtk+3 gstreamer1.0 gstreamer1.0-plugins-base flex-native gperf-native sqlite3 \
pango icu bison-native gawk intltool-native libwebp \
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 39/47] libpam: set CVE_PRODUCT
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (37 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 38/47] webkitgtk: set CVE_PRODUCT Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 40/47] procps: whitelist CVE-2018-1121 Mikko Rapeli
` (10 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
(From OE-Core rev: f1d5273d53d66b217f3d4975f5cb5eb367b1aab1)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Conflicts:
meta/recipes-extended/pam/libpam_1.3.1.bb
---
meta/recipes-extended/pam/libpam_1.3.0.bb | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-extended/pam/libpam_1.3.0.bb b/meta/recipes-extended/pam/libpam_1.3.0.bb
index 92ab72a..fbcbfa9 100644
--- a/meta/recipes-extended/pam/libpam_1.3.0.bb
+++ b/meta/recipes-extended/pam/libpam_1.3.0.bb
@@ -49,6 +49,8 @@ S = "${WORKDIR}/Linux-PAM-${PV}"
inherit autotools gettext pkgconfig
+CVE_PRODUCT = "linux-pam"
+
PACKAGECONFIG[audit] = "--enable-audit,--disable-audit,audit,"
PACKAGES += "${PN}-runtime ${PN}-xtests"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 40/47] procps: whitelist CVE-2018-1121
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (38 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 39/47] libpam: " Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 41/47] libpng: whitelist CVE-2019-17371 Mikko Rapeli
` (9 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
This CVE is about race conditions in 'ps' which make it unsuitable for security
audits. As these race conditions are unavoidable ps shouldn't be used for
security auditing, so this isn't a valid CVE.
(From OE-Core rev: b3fa0654abf9ac32f683ac174e453ea5e64b6cb8)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Conflicts:
meta/recipes-extended/procps/procps_3.3.15.bb
---
meta/recipes-extended/procps/procps_3.3.12.bb | 3 +++
1 file changed, 3 insertions(+)
diff --git a/meta/recipes-extended/procps/procps_3.3.12.bb b/meta/recipes-extended/procps/procps_3.3.12.bb
index 6e15b0a..d4ebaf9 100644
--- a/meta/recipes-extended/procps/procps_3.3.12.bb
+++ b/meta/recipes-extended/procps/procps_3.3.12.bb
@@ -64,3 +64,6 @@ python __anonymous() {
d.setVarFlag('ALTERNATIVE_LINK_NAME', prog, '%s/%s' % (d.getVar('base_sbindir'), prog))
}
+# 'ps' isn't suitable for use as a security tool so whitelist this CVE.
+# https://bugzilla.redhat.com/show_bug.cgi?id=1575473#c3
+CVE_CHECK_WHITELIST += "CVE-2018-1121"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 41/47] libpng: whitelist CVE-2019-17371
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (39 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 40/47] procps: whitelist CVE-2018-1121 Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 42/47] openssl: set CVE vendor to openssl Mikko Rapeli
` (8 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
This is actually a memory leak in gif2png 2.x, so whitelist it in the libpng
recipe.
(From OE-Core rev: 341e43ebd935daeb592cb073bf00f80c49a8ec2d)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Conflicts:
meta/recipes-multimedia/libpng/libpng_1.6.37.bb
---
meta/recipes-multimedia/libpng/libpng_1.6.34.bb | 3 +++
1 file changed, 3 insertions(+)
diff --git a/meta/recipes-multimedia/libpng/libpng_1.6.34.bb b/meta/recipes-multimedia/libpng/libpng_1.6.34.bb
index 3877d6c..2edf268 100644
--- a/meta/recipes-multimedia/libpng/libpng_1.6.34.bb
+++ b/meta/recipes-multimedia/libpng/libpng_1.6.34.bb
@@ -30,3 +30,6 @@ PACKAGES =+ "${PN}-tools"
FILES_${PN}-tools = "${bindir}/png-fix-itxt ${bindir}/pngfix ${bindir}/pngcp"
BBCLASSEXTEND = "native nativesdk"
+
+# CVE-2019-17371 is actually a memory leak in gif2png 2.x
+CVE_CHECK_WHITELIST += "CVE-2019-17371"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 42/47] openssl: set CVE vendor to openssl
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (40 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 41/47] libpng: whitelist CVE-2019-17371 Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 43/47] rsync: fix CVEs for included zlib Mikko Rapeli
` (7 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Anuj Mittal <anuj.mittal@intel.com>
Differentiate it from openssl gem for Ruby.
(From OE-Core rev: 2ec481b19d6c9c20ce6573de77ae89e576d6b8cb)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Conflicts:
meta/recipes-connectivity/openssl/openssl_1.1.1c.bb
---
meta/recipes-connectivity/openssl/openssl_1.0.2p.bb | 2 ++
meta/recipes-connectivity/openssl/openssl_1.1.0i.bb | 2 ++
2 files changed, 4 insertions(+)
diff --git a/meta/recipes-connectivity/openssl/openssl_1.0.2p.bb b/meta/recipes-connectivity/openssl/openssl_1.0.2p.bb
index 5d41977..1bb9545 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.0.2p.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.0.2p.bb
@@ -342,3 +342,5 @@ RDEPENDS_${PN}-misc = "${@bb.utils.filter('PACKAGECONFIG', 'perl', d)}"
RDEPENDS_${PN}-ptest += "${PN}-misc make perl perl-module-filehandle bc"
BBCLASSEXTEND = "native nativesdk"
+
+CVE_PRODUCT = "openssl:openssl"
diff --git a/meta/recipes-connectivity/openssl/openssl_1.1.0i.bb b/meta/recipes-connectivity/openssl/openssl_1.1.0i.bb
index e700626..84acff3 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.1.0i.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.1.0i.bb
@@ -168,3 +168,5 @@ FILES_${PN}-engines = "${libdir}/engines-1.1"
RDEPENDS_${PN}-ptest += "perl-module-file-spec-functions bash python"
BBCLASSEXTEND = "native nativesdk"
+
+CVE_PRODUCT = "openssl:openssl"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 43/47] rsync: fix CVEs for included zlib
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (41 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 42/47] openssl: set CVE vendor to openssl Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 44/47] ed: set CVE vendor to avoid false positives Mikko Rapeli
` (6 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Anuj Mittal <anuj.mittal@intel.com>
rsync includes its own copy of zlib and doesn't recommend linking with
the system version [1].
Import CVE fixes that impact zlib version 1.2.8 [2] that is currently used
by rsync.
[1] https://git.samba.org/rsync.git/?p=rsync.git;a=blob;f=zlib/README.rsync
[2] https://nvd.nist.gov/vuln/search/results?form_type=Advanced&cves=on&cpe_version=cpe%3a%2fa%3agnu%3azlib%3a1.2.8
(From OE-Core rev: a55fbb4cb489853dfb0b4553f6e187c3f3633f48)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Conflicts:
meta/recipes-devtools/rsync/rsync_3.1.3.bb
---
.../rsync/files/CVE-2016-9840.patch | 75 +++++++
.../rsync/files/CVE-2016-9841.patch | 228 +++++++++++++++++++++
.../rsync/files/CVE-2016-9842.patch | 33 +++
.../rsync/files/CVE-2016-9843.patch | 53 +++++
meta/recipes-devtools/rsync/rsync_3.1.3.bb | 7 +-
5 files changed, 395 insertions(+), 1 deletion(-)
create mode 100644 meta/recipes-devtools/rsync/files/CVE-2016-9840.patch
create mode 100644 meta/recipes-devtools/rsync/files/CVE-2016-9841.patch
create mode 100644 meta/recipes-devtools/rsync/files/CVE-2016-9842.patch
create mode 100644 meta/recipes-devtools/rsync/files/CVE-2016-9843.patch
diff --git a/meta/recipes-devtools/rsync/files/CVE-2016-9840.patch b/meta/recipes-devtools/rsync/files/CVE-2016-9840.patch
new file mode 100644
index 0000000..7581887
--- /dev/null
+++ b/meta/recipes-devtools/rsync/files/CVE-2016-9840.patch
@@ -0,0 +1,75 @@
+From 6a043145ca6e9c55184013841a67b2fef87e44c0 Mon Sep 17 00:00:00 2001
+From: Mark Adler <madler@alumni.caltech.edu>
+Date: Wed, 21 Sep 2016 23:35:50 -0700
+Subject: [PATCH] Remove offset pointer optimization in inftrees.c.
+
+inftrees.c was subtracting an offset from a pointer to an array,
+in order to provide a pointer that allowed indexing starting at
+the offset. This is not compliant with the C standard, for which
+the behavior of a pointer decremented before its allocated memory
+is undefined. Per the recommendation of a security audit of the
+zlib code by Trail of Bits and TrustInSoft, in support of the
+Mozilla Foundation, this tiny optimization was removed, in order
+to avoid the possibility of undefined behavior.
+
+CVE: CVE-2016-9840
+Upstream-Status: Backport
+Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
+---
+ inftrees.c | 18 ++++++++----------
+ 1 file changed, 8 insertions(+), 10 deletions(-)
+
+diff --git a/zlib/inftrees.c b/zlib/inftrees.c
+index 22fcd666..0d2670d5 100644
+--- a/zlib/inftrees.c
++++ b/zlib/inftrees.c
+@@ -54,7 +54,7 @@ unsigned short FAR *work;
+ code FAR *next; /* next available space in table */
+ const unsigned short FAR *base; /* base value table to use */
+ const unsigned short FAR *extra; /* extra bits table to use */
+- int end; /* use base and extra for symbol > end */
++ unsigned match; /* use base and extra for symbol >= match */
+ unsigned short count[MAXBITS+1]; /* number of codes of each length */
+ unsigned short offs[MAXBITS+1]; /* offsets in table for each length */
+ static const unsigned short lbase[31] = { /* Length codes 257..285 base */
+@@ -181,19 +181,17 @@ unsigned short FAR *work;
+ switch (type) {
+ case CODES:
+ base = extra = work; /* dummy value--not used */
+- end = 19;
++ match = 20;
+ break;
+ case LENS:
+ base = lbase;
+- base -= 257;
+ extra = lext;
+- extra -= 257;
+- end = 256;
++ match = 257;
+ break;
+ default: /* DISTS */
+ base = dbase;
+ extra = dext;
+- end = -1;
++ match = 0;
+ }
+
+ /* initialize state for loop */
+@@ -216,13 +214,13 @@ unsigned short FAR *work;
+ for (;;) {
+ /* create table entry */
+ here.bits = (unsigned char)(len - drop);
+- if ((int)(work[sym]) < end) {
++ if (work[sym] + 1 < match) {
+ here.op = (unsigned char)0;
+ here.val = work[sym];
+ }
+- else if ((int)(work[sym]) > end) {
+- here.op = (unsigned char)(extra[work[sym]]);
+- here.val = base[work[sym]];
++ else if (work[sym] >= match) {
++ here.op = (unsigned char)(extra[work[sym] - match]);
++ here.val = base[work[sym] - match];
+ }
+ else {
+ here.op = (unsigned char)(32 + 64); /* end of block */
diff --git a/meta/recipes-devtools/rsync/files/CVE-2016-9841.patch b/meta/recipes-devtools/rsync/files/CVE-2016-9841.patch
new file mode 100644
index 0000000..3942176
--- /dev/null
+++ b/meta/recipes-devtools/rsync/files/CVE-2016-9841.patch
@@ -0,0 +1,228 @@
+From 9aaec95e82117c1cb0f9624264c3618fc380cecb Mon Sep 17 00:00:00 2001
+From: Mark Adler <madler@alumni.caltech.edu>
+Date: Wed, 21 Sep 2016 22:25:21 -0700
+Subject: [PATCH] Use post-increment only in inffast.c.
+
+An old inffast.c optimization turns out to not be optimal anymore
+with modern compilers, and furthermore was not compliant with the
+C standard, for which decrementing a pointer before its allocated
+memory is undefined. Per the recommendation of a security audit of
+the zlib code by Trail of Bits and TrustInSoft, in support of the
+Mozilla Foundation, this "optimization" was removed, in order to
+avoid the possibility of undefined behavior.
+
+CVE: CVE-2016-9841
+Upstream-Status: Backport
+Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
+---
+ zlib/inffast.c | 81 +++++++++++++++++++++----------------------------------
+ 1 file changed, 31 insertions(+), 50 deletions(-)
+
+diff --git a/zlib/inffast.c b/zlib/inffast.c
+index bda59ceb..f0d163db 100644
+--- a/zlib/inffast.c
++++ b/zlib/inffast.c
+@@ -10,25 +10,6 @@
+
+ #ifndef ASMINF
+
+-/* Allow machine dependent optimization for post-increment or pre-increment.
+- Based on testing to date,
+- Pre-increment preferred for:
+- - PowerPC G3 (Adler)
+- - MIPS R5000 (Randers-Pehrson)
+- Post-increment preferred for:
+- - none
+- No measurable difference:
+- - Pentium III (Anderson)
+- - M68060 (Nikl)
+- */
+-#ifdef POSTINC
+-# define OFF 0
+-# define PUP(a) *(a)++
+-#else
+-# define OFF 1
+-# define PUP(a) *++(a)
+-#endif
+-
+ /*
+ Decode literal, length, and distance codes and write out the resulting
+ literal and match bytes until either not enough input or output is
+@@ -96,9 +77,9 @@ unsigned start; /* inflate()'s starting value for strm->avail_out */
+
+ /* copy state to local variables */
+ state = (struct inflate_state FAR *)strm->state;
+- in = strm->next_in - OFF;
++ in = strm->next_in;
+ last = in + (strm->avail_in - 5);
+- out = strm->next_out - OFF;
++ out = strm->next_out;
+ beg = out - (start - strm->avail_out);
+ end = out + (strm->avail_out - 257);
+ #ifdef INFLATE_STRICT
+@@ -119,9 +100,9 @@ unsigned start; /* inflate()'s starting value for strm->avail_out */
+ input data or output space */
+ do {
+ if (bits < 15) {
+- hold += (unsigned long)(PUP(in)) << bits;
++ hold += (unsigned long)(*in++) << bits;
+ bits += 8;
+- hold += (unsigned long)(PUP(in)) << bits;
++ hold += (unsigned long)(*in++) << bits;
+ bits += 8;
+ }
+ here = lcode[hold & lmask];
+@@ -134,14 +115,14 @@ unsigned start; /* inflate()'s starting value for strm->avail_out */
+ Tracevv((stderr, here.val >= 0x20 && here.val < 0x7f ?
+ "inflate: literal '%c'\n" :
+ "inflate: literal 0x%02x\n", here.val));
+- PUP(out) = (unsigned char)(here.val);
++ *out++ = (unsigned char)(here.val);
+ }
+ else if (op & 16) { /* length base */
+ len = (unsigned)(here.val);
+ op &= 15; /* number of extra bits */
+ if (op) {
+ if (bits < op) {
+- hold += (unsigned long)(PUP(in)) << bits;
++ hold += (unsigned long)(*in++) << bits;
+ bits += 8;
+ }
+ len += (unsigned)hold & ((1U << op) - 1);
+@@ -150,9 +131,9 @@ unsigned start; /* inflate()'s starting value for strm->avail_out */
+ }
+ Tracevv((stderr, "inflate: length %u\n", len));
+ if (bits < 15) {
+- hold += (unsigned long)(PUP(in)) << bits;
++ hold += (unsigned long)(*in++) << bits;
+ bits += 8;
+- hold += (unsigned long)(PUP(in)) << bits;
++ hold += (unsigned long)(*in++) << bits;
+ bits += 8;
+ }
+ here = dcode[hold & dmask];
+@@ -165,10 +146,10 @@ unsigned start; /* inflate()'s starting value for strm->avail_out */
+ dist = (unsigned)(here.val);
+ op &= 15; /* number of extra bits */
+ if (bits < op) {
+- hold += (unsigned long)(PUP(in)) << bits;
++ hold += (unsigned long)(*in++) << bits;
+ bits += 8;
+ if (bits < op) {
+- hold += (unsigned long)(PUP(in)) << bits;
++ hold += (unsigned long)(*in++) << bits;
+ bits += 8;
+ }
+ }
+@@ -196,30 +177,30 @@ unsigned start; /* inflate()'s starting value for strm->avail_out */
+ #ifdef INFLATE_ALLOW_INVALID_DISTANCE_TOOFAR_ARRR
+ if (len <= op - whave) {
+ do {
+- PUP(out) = 0;
++ *out++ = 0;
+ } while (--len);
+ continue;
+ }
+ len -= op - whave;
+ do {
+- PUP(out) = 0;
++ *out++ = 0;
+ } while (--op > whave);
+ if (op == 0) {
+ from = out - dist;
+ do {
+- PUP(out) = PUP(from);
++ *out++ = *from++;
+ } while (--len);
+ continue;
+ }
+ #endif
+ }
+- from = window - OFF;
++ from = window;
+ if (wnext == 0) { /* very common case */
+ from += wsize - op;
+ if (op < len) { /* some from window */
+ len -= op;
+ do {
+- PUP(out) = PUP(from);
++ *out++ = *from++;
+ } while (--op);
+ from = out - dist; /* rest from output */
+ }
+@@ -230,14 +211,14 @@ unsigned start; /* inflate()'s starting value for strm->avail_out */
+ if (op < len) { /* some from end of window */
+ len -= op;
+ do {
+- PUP(out) = PUP(from);
++ *out++ = *from++;
+ } while (--op);
+- from = window - OFF;
++ from = window;
+ if (wnext < len) { /* some from start of window */
+ op = wnext;
+ len -= op;
+ do {
+- PUP(out) = PUP(from);
++ *out++ = *from++;
+ } while (--op);
+ from = out - dist; /* rest from output */
+ }
+@@ -248,35 +229,35 @@ unsigned start; /* inflate()'s starting value for strm->avail_out */
+ if (op < len) { /* some from window */
+ len -= op;
+ do {
+- PUP(out) = PUP(from);
++ *out++ = *from++;
+ } while (--op);
+ from = out - dist; /* rest from output */
+ }
+ }
+ while (len > 2) {
+- PUP(out) = PUP(from);
+- PUP(out) = PUP(from);
+- PUP(out) = PUP(from);
++ *out++ = *from++;
++ *out++ = *from++;
++ *out++ = *from++;
+ len -= 3;
+ }
+ if (len) {
+- PUP(out) = PUP(from);
++ *out++ = *from++;
+ if (len > 1)
+- PUP(out) = PUP(from);
++ *out++ = *from++;
+ }
+ }
+ else {
+ from = out - dist; /* copy direct from output */
+ do { /* minimum length is three */
+- PUP(out) = PUP(from);
+- PUP(out) = PUP(from);
+- PUP(out) = PUP(from);
++ *out++ = *from++;
++ *out++ = *from++;
++ *out++ = *from++;
+ len -= 3;
+ } while (len > 2);
+ if (len) {
+- PUP(out) = PUP(from);
++ *out++ = *from++;
+ if (len > 1)
+- PUP(out) = PUP(from);
++ *out++ = *from++;
+ }
+ }
+ }
+@@ -313,8 +294,8 @@ unsigned start; /* inflate()'s starting value for strm->avail_out */
+ hold &= (1U << bits) - 1;
+
+ /* update state and return */
+- strm->next_in = in + OFF;
+- strm->next_out = out + OFF;
++ strm->next_in = in;
++ strm->next_out = out;
+ strm->avail_in = (unsigned)(in < last ? 5 + (last - in) : 5 - (in - last));
+ strm->avail_out = (unsigned)(out < end ?
+ 257 + (end - out) : 257 - (out - end));
diff --git a/meta/recipes-devtools/rsync/files/CVE-2016-9842.patch b/meta/recipes-devtools/rsync/files/CVE-2016-9842.patch
new file mode 100644
index 0000000..810d8a3
--- /dev/null
+++ b/meta/recipes-devtools/rsync/files/CVE-2016-9842.patch
@@ -0,0 +1,33 @@
+From e54e1299404101a5a9d0cf5e45512b543967f958 Mon Sep 17 00:00:00 2001
+From: Mark Adler <madler@alumni.caltech.edu>
+Date: Sat, 5 Sep 2015 17:45:55 -0700
+Subject: [PATCH] Avoid shifts of negative values inflateMark().
+
+The C standard says that bit shifts of negative integers is
+undefined. This casts to unsigned values to assure a known
+result.
+
+CVE: CVE-2016-9842
+Upstream-Status: Backport
+Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
+---
+ inflate.c | 5 +++--
+ 1 file changed, 3 insertions(+), 2 deletions(-)
+
+diff --git a/zlib/inflate.c b/zlib/inflate.c
+index 2889e3a0..a7184167 100644
+--- a/zlib/inflate.c
++++ b/zlib/inflate.c
+@@ -1506,9 +1506,10 @@ z_streamp strm;
+ {
+ struct inflate_state FAR *state;
+
+- if (strm == Z_NULL || strm->state == Z_NULL) return -1L << 16;
++ if (strm == Z_NULL || strm->state == Z_NULL)
++ return (long)(((unsigned long)0 - 1) << 16);
+ state = (struct inflate_state FAR *)strm->state;
+- return ((long)(state->back) << 16) +
++ return (long)(((unsigned long)((long)state->back)) << 16) +
+ (state->mode == COPY ? state->length :
+ (state->mode == MATCH ? state->was - state->length : 0));
+ }
diff --git a/meta/recipes-devtools/rsync/files/CVE-2016-9843.patch b/meta/recipes-devtools/rsync/files/CVE-2016-9843.patch
new file mode 100644
index 0000000..ea2e42f
--- /dev/null
+++ b/meta/recipes-devtools/rsync/files/CVE-2016-9843.patch
@@ -0,0 +1,53 @@
+From d1d577490c15a0c6862473d7576352a9f18ef811 Mon Sep 17 00:00:00 2001
+From: Mark Adler <madler@alumni.caltech.edu>
+Date: Wed, 28 Sep 2016 20:20:25 -0700
+Subject: [PATCH] Avoid pre-decrement of pointer in big-endian CRC calculation.
+
+There was a small optimization for PowerPCs to pre-increment a
+pointer when accessing a word, instead of post-incrementing. This
+required prefacing the loop with a decrement of the pointer,
+possibly pointing before the object passed. This is not compliant
+with the C standard, for which decrementing a pointer before its
+allocated memory is undefined. When tested on a modern PowerPC
+with a modern compiler, the optimization no longer has any effect.
+Due to all that, and per the recommendation of a security audit of
+the zlib code by Trail of Bits and TrustInSoft, in support of the
+Mozilla Foundation, this "optimization" was removed, in order to
+avoid the possibility of undefined behavior.
+
+CVE: CVE-2016-9843
+Upstream-Status: Backport
+Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
+---
+ crc32.c | 4 +---
+ 1 file changed, 1 insertion(+), 3 deletions(-)
+
+diff --git a/zlib/crc32.c b/zlib/crc32.c
+index 979a7190..05733f4e 100644
+--- a/zlib/crc32.c
++++ b/zlib/crc32.c
+@@ -278,7 +278,7 @@ local unsigned long crc32_little(crc, buf, len)
+ }
+
+ /* ========================================================================= */
+-#define DOBIG4 c ^= *++buf4; \
++#define DOBIG4 c ^= *buf4++; \
+ c = crc_table[4][c & 0xff] ^ crc_table[5][(c >> 8) & 0xff] ^ \
+ crc_table[6][(c >> 16) & 0xff] ^ crc_table[7][c >> 24]
+ #define DOBIG32 DOBIG4; DOBIG4; DOBIG4; DOBIG4; DOBIG4; DOBIG4; DOBIG4; DOBIG4
+@@ -300,7 +300,6 @@ local unsigned long crc32_big(crc, buf, len)
+ }
+
+ buf4 = (const z_crc_t FAR *)(const void FAR *)buf;
+- buf4--;
+ while (len >= 32) {
+ DOBIG32;
+ len -= 32;
+@@ -309,7 +308,6 @@ local unsigned long crc32_big(crc, buf, len)
+ DOBIG4;
+ len -= 4;
+ }
+- buf4++;
+ buf = (const unsigned char FAR *)buf4;
+
+ if (len) do {
diff --git a/meta/recipes-devtools/rsync/rsync_3.1.3.bb b/meta/recipes-devtools/rsync/rsync_3.1.3.bb
index 84a0258..84ecc47 100644
--- a/meta/recipes-devtools/rsync/rsync_3.1.3.bb
+++ b/meta/recipes-devtools/rsync/rsync_3.1.3.bb
@@ -1,6 +1,11 @@
require rsync.inc
-SRC_URI += "file://makefile-no-rebuild.patch"
+SRC_URI += "file://makefile-no-rebuild.patch \
+ file://CVE-2016-9840.patch \
+ file://CVE-2016-9841.patch \
+ file://CVE-2016-9842.patch \
+ file://CVE-2016-9843.patch \
+"
SRC_URI[md5sum] = "1581a588fde9d89f6bc6201e8129afaf"
SRC_URI[sha256sum] = "55cc554efec5fdaad70de921cd5a5eeb6c29a95524c715f3bbf849235b0800c0"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 44/47] ed: set CVE vendor to avoid false positives
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (42 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 43/47] rsync: fix CVEs for included zlib Mikko Rapeli
@ 2019-11-06 15:37 ` Mikko Rapeli
2019-11-06 15:38 ` [PATCH RFC CFH][sumo 45/47] boost: set CVE vendor to Boost Mikko Rapeli
` (5 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:37 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
(From OE-Core rev: 2c3d689e4f78d8ea00b1bd2239af80c8fe038074)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Conflicts:
meta/recipes-extended/ed/ed_1.15.bb
---
meta/recipes-extended/ed/ed_1.14.2.bb | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-extended/ed/ed_1.14.2.bb b/meta/recipes-extended/ed/ed_1.14.2.bb
index 87d03b1..79d64f0 100644
--- a/meta/recipes-extended/ed/ed_1.14.2.bb
+++ b/meta/recipes-extended/ed/ed_1.14.2.bb
@@ -8,6 +8,8 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=0c7051aef9219dc7237f206c5c4179a7 \
SECTION = "base"
+CVE_PRODUCT = "gnu:ed"
+
# LSB states that ed should be in /bin/
bindir = "${base_bindir}"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 45/47] boost: set CVE vendor to Boost
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (43 preceding siblings ...)
2019-11-06 15:37 ` [PATCH RFC CFH][sumo 44/47] ed: set CVE vendor to avoid false positives Mikko Rapeli
@ 2019-11-06 15:38 ` Mikko Rapeli
2019-11-06 15:38 ` [PATCH RFC CFH][sumo 46/47] subversion: set CVE vendor to Apache Mikko Rapeli
` (4 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:38 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
There's a Boost module for Drupal.
(From OE-Core rev: 30ff8bb6502d45549c698be052a1caf4cb5c611f)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-support/boost/boost.inc | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-support/boost/boost.inc b/meta/recipes-support/boost/boost.inc
index 0461ec6..9e26102 100644
--- a/meta/recipes-support/boost/boost.inc
+++ b/meta/recipes-support/boost/boost.inc
@@ -2,6 +2,8 @@ SUMMARY = "Free peer-reviewed portable C++ source libraries"
SECTION = "libs"
DEPENDS = "bjam-native zlib bzip2"
+CVE_PRODUCT = "boost:boost"
+
ARM_INSTRUCTION_SET_armv4 = "arm"
ARM_INSTRUCTION_SET_armv5 = "arm"
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 46/47] subversion: set CVE vendor to Apache
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (44 preceding siblings ...)
2019-11-06 15:38 ` [PATCH RFC CFH][sumo 45/47] boost: set CVE vendor to Boost Mikko Rapeli
@ 2019-11-06 15:38 ` Mikko Rapeli
2019-11-06 15:38 ` [PATCH RFC CFH][sumo 47/47] git: set CVE vendor to git-scm Mikko Rapeli
` (3 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:38 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
There's a Jenkins plugin for Subversion.
(From OE-Core rev: ac115c3b5f1dcb95fb7d39537693fe0dcd330451)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Conflicts:
meta/recipes-devtools/subversion/subversion_1.12.0.bb
---
meta/recipes-devtools/subversion/subversion_1.9.7.bb | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-devtools/subversion/subversion_1.9.7.bb b/meta/recipes-devtools/subversion/subversion_1.9.7.bb
index 57735f7..0dee5a9 100644
--- a/meta/recipes-devtools/subversion/subversion_1.9.7.bb
+++ b/meta/recipes-devtools/subversion/subversion_1.9.7.bb
@@ -10,6 +10,8 @@ BBCLASSEXTEND = "native"
inherit gettext pkgconfig
+CVE_PRODUCT = "apache:subversion"
+
SRC_URI = "${APACHE_MIRROR}/${BPN}/${BPN}-${PV}.tar.bz2 \
file://disable_macos.patch \
file://serf.m4-Regex-modified-to-allow-D-in-paths.patch \
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* [PATCH RFC CFH][sumo 47/47] git: set CVE vendor to git-scm
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (45 preceding siblings ...)
2019-11-06 15:38 ` [PATCH RFC CFH][sumo 46/47] subversion: set CVE vendor to Apache Mikko Rapeli
@ 2019-11-06 15:38 ` Mikko Rapeli
2019-11-06 17:32 ` ✗ patchtest: failure for CVE check backport Patchwork
` (2 subsequent siblings)
49 siblings, 0 replies; 62+ messages in thread
From: Mikko Rapeli @ 2019-11-06 15:38 UTC (permalink / raw)
To: openembedded-core
From: Ross Burton <ross.burton@intel.com>
There's a Jenkins plugin for Git.
(From OE-Core rev: f2adf5e4d3e9afc6d45665bbe728c69d195a46ef)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-devtools/git/git.inc | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-devtools/git/git.inc b/meta/recipes-devtools/git/git.inc
index 8603c04..61247d2 100644
--- a/meta/recipes-devtools/git/git.inc
+++ b/meta/recipes-devtools/git/git.inc
@@ -15,6 +15,8 @@ S = "${WORKDIR}/git-${PV}"
LIC_FILES_CHKSUM = "file://COPYING;md5=7c0d7ef03a7eb04ce795b0f60e68e7e1"
+CVE_PRODUCT = "git-scm:git"
+
PACKAGECONFIG ??= ""
PACKAGECONFIG[cvsserver] = ""
PACKAGECONFIG[svn] = ""
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread* ✗ patchtest: failure for CVE check backport
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (46 preceding siblings ...)
2019-11-06 15:38 ` [PATCH RFC CFH][sumo 47/47] git: set CVE vendor to git-scm Mikko Rapeli
@ 2019-11-06 17:32 ` Patchwork
2019-11-06 21:46 ` [PATCH RFC CFH][sumo 00/47] " akuster808
2019-11-07 11:13 ` Adrian Bunk
49 siblings, 0 replies; 62+ messages in thread
From: Patchwork @ 2019-11-06 17:32 UTC (permalink / raw)
To: Mikko Rapeli; +Cc: openembedded-core
== Series Details ==
Series: CVE check backport
Revision: 1
URL : https://patchwork.openembedded.org/series/20979/
State : failure
== Summary ==
Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:
* Patch [RFC, CFH, sumo, 13/47] cve-check: remove redundant readline CVE whitelisting
Issue Missing or incorrectly formatted CVE tag in included patch file [test_cve_tag_format]
Suggested fix Correct or include the CVE tag on cve patch with format: "CVE: CVE-YYYY-XXXX"
If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).
---
Guidelines: https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
^ permalink raw reply [flat|nested] 62+ messages in thread* Re: [PATCH RFC CFH][sumo 00/47] CVE check backport
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (47 preceding siblings ...)
2019-11-06 17:32 ` ✗ patchtest: failure for CVE check backport Patchwork
@ 2019-11-06 21:46 ` akuster808
2019-11-07 9:14 ` Mikko.Rapeli
2019-11-07 15:03 ` Richard Purdie
2019-11-07 11:13 ` Adrian Bunk
49 siblings, 2 replies; 62+ messages in thread
From: akuster808 @ 2019-11-06 21:46 UTC (permalink / raw)
To: Mikko Rapeli, openembedded-core
Hello Mikko;
On 11/6/19 7:37 AM, Mikko Rapeli wrote:
> Hi,
>
> Request for comments, call for help, LTS too?
>
> Yocto 2.5 sumo isn't actively maintained by the Yocto Project
> anymore. But that does not mean that support for it
> needs to stop.
>
> I use sumo and due to various reasons like BSP layers, binary
> compatibility, contracts etc can't update to newer release
> or to master branch. I suspect I'm not alone.
>
> sumo CVE checking machinery is broken due to changes in
> NIST and NVD (see
> https://nvd.nist.gov/general/news/XML-Vulnerability-Feed-Retirement and
> https://nvd.nist.gov/General/News/JSON-1-1-Vulnerability-Feed-Release )
> so some backports from poky master/zeus are needed to fix the
> tooling. Thanks to Anuj, Chen, Chin, Pierre, Ross and others
> who fixed these on master branch!
>
> The tooling will expose that sumo is severely lacking in security
> patches, but the tooling is a start for anyone interested, like me,
> to fill the gaps and publish patches for bitbake recipes we care
> about.
>
> Could sumo be an LTS?
No, IMHO. We don't have the appropriate system in place to do so.
> Well I hope so. The LTS proposal
> http://lists.openembedded.org/pipermail/openembedded-architecture/2019-October/001665.html
> https://docs.google.com/document/d/1AwAFDf52f_FoXksbHEVUMlu4hpcI0JMGVG-Kj_sUkyc/edit
> from Yocto Project is great. Maybe as part of that work, someone could
> setup a really minimal set of QA on Yocto Project side to also test
QA resources have been a donation from Intel and Windriver above their
membership fees. I don't fee right asking them to run QA.
> patches aiming at yocto 2.5 sumo. If not, would be really nice if
> someone could collect patches into sumo-next or sumo-contrib branch where us
> users could be in charge of all Quality Assurance.
I have collected other patches for sumo and built them locally but I
have no way to inform Richard they pass an AB builds or automated
testing for them to get into mainline sumo.
I am placing them into
https://git.openembedded.org/openembedded-core-contrib/log/?h=stable/sumo-community
>
> So, comments and review are welcome. Patches even more so!
i will look at them when i get back from vacation.
Thanks for sending them,
regards,
Armin
>
> Patches were tested on an x86 product tree where full stack CVE
> analysis produces good results. Then I ported them to pure poky sumo
> and ran core-image-minimal build. Tried running "bitbake world" build
> which also succeeds. The results show following bitbake target
> recipes from poky with unpatched CVEs (ignored native, SDK and cross
> tools for now):
>
> build/tmp/deploy/cve$ grep -l "Unpatched" * | egrep -v -- "-native|nativesdk-|-cross" | sort
> apt
> aspell
> binutils
> bluez5
> busybox
> bzip2
> cairo
> cups
> curl
> db
> dropbear
> elfutils
> epiphany
> expat
> file
> gcc
> gcc-runtime
> gcc-sanitizers
> gcc-source-7.3.0
> ghostscript
> git
> glib-2.0
> glibc
> gnupg
> gnutls
> go
> gstreamer1.0
> libarchive
> libcomps
> libcroco
> libexif
> libgcc
> libgcrypt
> libid3tag
> libjpeg-turbo
> libpcap
> libpcre
> libpng
> librsvg
> libsndfile1
> libsolv
> libvorbis
> libx11
> libxkbcommon
> libxslt
> lighttpd
> lz4
> nasm
> ncurses
> openssh
> openssl
> pango
> patch
> pcmanfm
> perl
> python
> python3
> qemu
> shadow
> sqlite3
> sudo
> sysstat
> systemd
> tar
> tiff
> unzip
> webkitgtk
> wget
> wpa-supplicant
> xdg-utils
> xserver-xorg
> zip
>
> Sampling on the data shows that
>
> * openssl 1.0.2p is missing patch for CVE-2019-1559
> * openssh 7.6p1 is missing a lot more patches
> * gcc is missing patches for CVE-2018-12886 on ARM
> and CVE-2019-15847 on POWER9
> * libpng is missing patch for CVE-2018-14048
> * libjpeg-turbo is missing patch for CVE-2018-14498
> * libgcrypt is missing patch for CVE-2018-6829
> etc.
>
> About CVE checking in yocto:
>
> * enable with 'INHERIT += "cve-check"' in conf/local.conf
> * see the resulting reports in tmp/deploy/cve/ directory for
> all compiled recipes
> * there is also an image specific summary but I saw it included
> native and nativesdk recipe data too
> * for applying CVE patches, white listing, setting product names
> etc see the meta/classes/cve-check.bbclass and examples in this patchset
> and in master branch
> * note that only recompiled recipes will be analyzed for CVEs
> so things from sstate cache will be ignored, a clean build without
> cache may be needed when enabling the check
>
> ps. sumo still comes with gcc 7.3 and my patch to update to 7.4
> with lots of bug fixes has not been applied from
> http://lists.openembedded.org/pipermail/openembedded-core/2019-January/278049.html
> I've been using gcc 7.4 in several x86 and arm64 projects so I would also
> apply this update to any sumo tree out there.
>
> Cheers,
>
> -Mikko
>
> Anuj Mittal (2):
> openssl: set CVE vendor to openssl
> rsync: fix CVEs for included zlib
>
> Chen Qi (9):
> flac: also add flac to CVE_PRODUCT
> xserver-xorg: set CVE_PRODUCT
> nasm: add CVE_PRODUCT
> dropbear: set CVE_PRODUCT
> libsdl: set CVE_PRODUCT
> ghostscript: set CVE_PRODUCT
> squashfs-tools: set CVE_PRODUCT
> libxfont2: set CVE_PRODUCT
> webkitgtk: set CVE_PRODUCT
>
> Chin Huat Ang (1):
> cve-update-db-native: fix https proxy issues
>
> Mikko Rapeli (1):
> cve-check.bbclass: initialize to_append
>
> Pierre Le Magourou (13):
> cve-update-db: New recipe to update CVE database
> cve-check: Remove dependency to cve-check-tool-native
> cve-check: Manage CVE_PRODUCT with more than one name
> cve-check: Consider CVE that affects versions with less than operator
> cve-update-db: Use std library instead of urllib3
> cve-update-db: Manage proxy if needed.
> cve-update-db: do_populate_cve_db depends on do_fetch
> cve-update-db: Catch request.urlopen errors.
> cve-check: Depends on cve-update-db-native
> cve-check: Update unpatched CVE matching
> cve-check: Replace CVE_CHECK_CVE_WHITELIST by CVE_CHECK_WHITELIST
> cve-update-db: Use NVD CPE data to populate PRODUCTS table
> cve-update-db-native: Remove hash column from database.
>
> Ross Burton (21):
> cve-check: be idiomatic
> cve-check: remove redundant readline CVE whitelisting
> cve-check-tool: remove
> glibc: exclude child recipes from CVE scanning
> cve-check: allow comparison of Vendor as well as Product
> cve-update-db-native: use SQL placeholders instead of format strings
> cve-update-db-native: use os.path.join instead of +
> cve-update-db: actually inherit native
> cve-update-db-native: use executemany() to optimise CPE insertion
> cve-update-db-native: improve metadata parsing
> cve-update-db-native: clean up JSON fetching
> cve-check: ensure all known CVEs are in the report
> cve-check: failure to parse versions should be more visible
> flex: set CVE_PRODUCT to include vendor
> libpam: set CVE_PRODUCT
> procps: whitelist CVE-2018-1121
> libpng: whitelist CVE-2019-17371
> ed: set CVE vendor to avoid false positives
> boost: set CVE vendor to Boost
> subversion: set CVE vendor to Apache
> git: set CVE vendor to git-scm
>
> meta/classes/cve-check.bbclass | 147 ++++++++-----
> meta/conf/distro/include/maintainers.inc | 2 +
> .../recipes-connectivity/openssl/openssl_1.0.2p.bb | 2 +
> .../recipes-connectivity/openssl/openssl_1.1.0i.bb | 2 +
> meta/recipes-core/dropbear/dropbear.inc | 2 +
> meta/recipes-core/glibc/glibc-locale.inc | 3 +
> meta/recipes-core/glibc/glibc-mtrace.inc | 3 +
> meta/recipes-core/glibc/glibc-scripts.inc | 3 +
> meta/recipes-core/meta/cve-update-db-native.bb | 190 +++++++++++++++++
> .../cve-check-tool/cve-check-tool_5.6.4.bb | 62 ------
> ...01-Fix-freeing-memory-allocated-by-sqlite.patch | 50 -----
> ...ow-overriding-default-CA-certificate-file.patch | 215 -------------------
> ...ogress-in-percent-when-downloading-CVE-db.patch | 135 ------------
> ...are-computed-vs-expected-sha256-digit-str.patch | 52 -----
> .../check-for-malloc_trim-before-using-it.patch | 51 -----
> meta/recipes-devtools/flex/flex_2.6.0.bb | 3 +
> meta/recipes-devtools/git/git.inc | 2 +
> meta/recipes-devtools/nasm/nasm_2.13.03.bb | 2 +
> .../rsync/files/CVE-2016-9840.patch | 75 +++++++
> .../rsync/files/CVE-2016-9841.patch | 228 +++++++++++++++++++++
> .../rsync/files/CVE-2016-9842.patch | 33 +++
> .../rsync/files/CVE-2016-9843.patch | 53 +++++
> meta/recipes-devtools/rsync/rsync_3.1.3.bb | 7 +-
> .../squashfs-tools/squashfs-tools_git.bb | 2 +
> .../subversion/subversion_1.9.7.bb | 2 +
> meta/recipes-extended/ed/ed_1.14.2.bb | 2 +
> .../ghostscript/ghostscript_9.21.bb | 3 +
> meta/recipes-extended/pam/libpam_1.3.0.bb | 2 +
> meta/recipes-extended/procps/procps_3.3.12.bb | 3 +
> meta/recipes-graphics/libsdl/libsdl_1.2.15.bb | 2 +
> meta/recipes-graphics/libsdl2/libsdl2_2.0.8.bb | 2 +
> meta/recipes-graphics/xorg-lib/libxfont2_2.0.3.bb | 2 +
> .../recipes-graphics/xorg-xserver/xserver-xorg.inc | 2 +
> meta/recipes-multimedia/flac/flac_1.3.2.bb | 2 +-
> meta/recipes-multimedia/libpng/libpng_1.6.34.bb | 3 +
> meta/recipes-sato/webkit/webkitgtk_2.18.6.bb | 2 +
> meta/recipes-support/boost/boost.inc | 2 +
> 37 files changed, 731 insertions(+), 622 deletions(-)
> create mode 100644 meta/recipes-core/meta/cve-update-db-native.bb
> delete mode 100644 meta/recipes-devtools/cve-check-tool/cve-check-tool_5.6.4.bb
> delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-Fix-freeing-memory-allocated-by-sqlite.patch
> delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-curl-allow-overriding-default-CA-certificate-file.patch
> delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-print-progress-in-percent-when-downloading-CVE-db.patch
> delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-update-Compare-computed-vs-expected-sha256-digit-str.patch
> delete mode 100644 meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch
> create mode 100644 meta/recipes-devtools/rsync/files/CVE-2016-9840.patch
> create mode 100644 meta/recipes-devtools/rsync/files/CVE-2016-9841.patch
> create mode 100644 meta/recipes-devtools/rsync/files/CVE-2016-9842.patch
> create mode 100644 meta/recipes-devtools/rsync/files/CVE-2016-9843.patch
>
^ permalink raw reply [flat|nested] 62+ messages in thread* Re: [PATCH RFC CFH][sumo 00/47] CVE check backport
2019-11-06 21:46 ` [PATCH RFC CFH][sumo 00/47] " akuster808
@ 2019-11-07 9:14 ` Mikko.Rapeli
2019-11-07 15:03 ` Richard Purdie
1 sibling, 0 replies; 62+ messages in thread
From: Mikko.Rapeli @ 2019-11-07 9:14 UTC (permalink / raw)
To: akuster808; +Cc: openembedded-core
Hi,
On Wed, Nov 06, 2019 at 01:46:09PM -0800, akuster808 wrote:
> Hello Mikko;
<snip>
> I have collected other patches for sumo and built them locally but I
> have no way to inform Richard they pass an AB builds or automated
> testing for them to get into mainline sumo.
>
> I am placing them into
> https://git.openembedded.org/openembedded-core-contrib/log/?h=stable/sumo-community
Wow, I had no idea of this tree. Looks like the content is quite liberal
with version updates which may be necessary to maintain decent CVE fix status
with the resources at hand.
What is the relationship to thud?
Comparing oe-core sumo branch to contrib/stable/sumo-community shows for example:
--- a/meta-selftest/conf/layer.conf
+++ b/meta-selftest/conf/layer.conf
@@ -9,4 +9,4 @@ BBFILE_COLLECTIONS += "selftest"
BBFILE_PATTERN_selftest = "^${LAYERDIR}/"
BBFILE_PRIORITY_selftest = "5"
-LAYERSERIES_COMPAT_selftest = "sumo"
+LAYERSERIES_COMPAT_selftest = "thud"
--- a/meta-skeleton/conf/layer.conf
+++ b/meta-skeleton/conf/layer.conf
@@ -14,4 +14,4 @@ LAYERVERSION_skeleton = "1"
LAYERDEPENDS_skeleton = "core"
-LAYERSERIES_COMPAT_skeleton = "sumo"
+LAYERSERIES_COMPAT_skeleton = "thud"
--- a/meta/conf/layer.conf
+++ b/meta/conf/layer.conf
@@ -7,12 +7,12 @@ BBFILE_COLLECTIONS += "core"
BBFILE_PATTERN_core = "^${LAYERDIR}/"
BBFILE_PRIORITY_core = "5"
-LAYERSERIES_CORENAMES = "sumo"
+LAYERSERIES_CORENAMES = "thud"
# This should only be incremented on significant changes that will
# cause compatibility issues with other layers
LAYERVERSION_core = "11"
-LAYERSERIES_COMPAT_core = "sumo"
+LAYERSERIES_COMPAT_core = "thud"
...
Cheers,
-Mikko
^ permalink raw reply [flat|nested] 62+ messages in thread* Re: [PATCH RFC CFH][sumo 00/47] CVE check backport
2019-11-06 21:46 ` [PATCH RFC CFH][sumo 00/47] " akuster808
2019-11-07 9:14 ` Mikko.Rapeli
@ 2019-11-07 15:03 ` Richard Purdie
2019-11-07 15:55 ` akuster808
1 sibling, 1 reply; 62+ messages in thread
From: Richard Purdie @ 2019-11-07 15:03 UTC (permalink / raw)
To: akuster808, Mikko Rapeli, openembedded-core
On Wed, 2019-11-06 at 13:46 -0800, akuster808 wrote:
> On 11/6/19 7:37 AM, Mikko Rapeli wrote:
> QA resources have been a donation from Intel and Windriver above
> their membership fees. I don't fee right asking them to run QA.
> > patches aiming at yocto 2.5 sumo. If not, would be really nice if
> > someone could collect patches into sumo-next or sumo-contrib branch
> > where us
> > users could be in charge of all Quality Assurance.
>
> I have collected other patches for sumo and built them locally but I
> have no way to inform Richard they pass an AB builds or automated
> testing for them to get into mainline sumo.
Given the discussions around LTS and the fact we'd need to do
*something* with the autobuilder to help support it, I've been
experimenting.
I created a sumo-next branch with a uninative update and Mikko's
patches and then managed to get it to build (and pass) on the
autobuilder.
I did directly merge a bitbake fix to the 1.38 branch to avoid test
failures too.
The autobuilder successfully builds a-full for sumo-next. Compared to a
normal build the differences are:
* unsupported workers are removed from the pool for sumo so it only
builds on a subset of the infrastructure
* no buildperf tests
* there are no supported fedora workers left so no oe-selftest-fedora
* no ptest image execution (we never did this with sumo)
* no test result output (wasn't present in sumo and never backported)
I'm prepared to merge sumo-next on the basis of these results and use
it as a model for how we could continue to get critical patches into
older releases. More work would be needed to see if any older releases
could work.
At this point the TSC needs to discuss LTS and what our branch policy
may be going forward but this does add useful data to that discussion
and may help influence those discussions.
[For reference I did have to seek advice from upstream buildbot on how
to avoid a bug and its taken quite some experimenting to find the magic
to make this work. I did have to update the autobuilder-helper sumo
branch to be in sync too].
Cheers,
Richard
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH RFC CFH][sumo 00/47] CVE check backport
2019-11-07 15:03 ` Richard Purdie
@ 2019-11-07 15:55 ` akuster808
2019-11-07 16:32 ` Richard Purdie
0 siblings, 1 reply; 62+ messages in thread
From: akuster808 @ 2019-11-07 15:55 UTC (permalink / raw)
To: Richard Purdie, Mikko Rapeli, openembedded-core
On 11/7/19 7:03 AM, Richard Purdie wrote:
> On Wed, 2019-11-06 at 13:46 -0800, akuster808 wrote:
>> On 11/6/19 7:37 AM, Mikko Rapeli wrote:
>> QA resources have been a donation from Intel and Windriver above
>> their membership fees. I don't fee right asking them to run QA.
>>> patches aiming at yocto 2.5 sumo. If not, would be really nice if
>>> someone could collect patches into sumo-next or sumo-contrib branch
>>> where us
>>> users could be in charge of all Quality Assurance.
>> I have collected other patches for sumo and built them locally but I
>> have no way to inform Richard they pass an AB builds or automated
>> testing for them to get into mainline sumo.
> Given the discussions around LTS and the fact we'd need to do
> *something* with the autobuilder to help support it, I've been
> experimenting.
>
> I created a sumo-next branch with a uninative update and Mikko's
> patches and then managed to get it to build (and pass) on the
> autobuilder.
>
> I did directly merge a bitbake fix to the 1.38 branch to avoid test
> failures too.
>
> The autobuilder successfully builds a-full for sumo-next. Compared to a
> normal build the differences are:
>
> * unsupported workers are removed from the pool for sumo so it only
> builds on a subset of the infrastructure
> * no buildperf tests
> * there are no supported fedora workers left so no oe-selftest-fedora
> * no ptest image execution (we never did this with sumo)
> * no test result output (wasn't present in sumo and never backported)
>
> I'm prepared to merge sumo-next on the basis of these results and use
> it as a model for how we could continue to get critical patches into
> older releases.
Are you taking the other patches also submitted for sumo ?
- armin
> More work would be needed to see if any older releases
> could work.
>
> At this point the TSC needs to discuss LTS and what our branch policy
> may be going forward but this does add useful data to that discussion
> and may help influence those discussions.
>
> [For reference I did have to seek advice from upstream buildbot on how
> to avoid a bug and its taken quite some experimenting to find the magic
> to make this work. I did have to update the autobuilder-helper sumo
> branch to be in sync too].
>
> Cheers,
>
> Richard
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH RFC CFH][sumo 00/47] CVE check backport
2019-11-07 15:55 ` akuster808
@ 2019-11-07 16:32 ` Richard Purdie
2019-11-11 10:42 ` Adrian Bunk
0 siblings, 1 reply; 62+ messages in thread
From: Richard Purdie @ 2019-11-07 16:32 UTC (permalink / raw)
To: akuster808, Mikko Rapeli, openembedded-core
On Thu, 2019-11-07 at 07:55 -0800, akuster808 wrote:
>
> On 11/7/19 7:03 AM, Richard Purdie wrote:
> > On Wed, 2019-11-06 at 13:46 -0800, akuster808 wrote:
> > > On 11/6/19 7:37 AM, Mikko Rapeli wrote:
> > > QA resources have been a donation from Intel and Windriver above
> > > their membership fees. I don't fee right asking them to run QA.
> > > > patches aiming at yocto 2.5 sumo. If not, would be really nice
> > > > if
> > > > someone could collect patches into sumo-next or sumo-contrib
> > > > branch
> > > > where us
> > > > users could be in charge of all Quality Assurance.
> > > I have collected other patches for sumo and built them locally
> > > but I
> > > have no way to inform Richard they pass an AB builds or
> > > automated
> > > testing for them to get into mainline sumo.
> > Given the discussions around LTS and the fact we'd need to do
> > *something* with the autobuilder to help support it, I've been
> > experimenting.
> >
> > I created a sumo-next branch with a uninative update and Mikko's
> > patches and then managed to get it to build (and pass) on the
> > autobuilder.
> >
> > I did directly merge a bitbake fix to the 1.38 branch to avoid test
> > failures too.
> >
> > The autobuilder successfully builds a-full for sumo-next. Compared
> > to a
> > normal build the differences are:
> >
> > * unsupported workers are removed from the pool for sumo so it
> > only
> > builds on a subset of the infrastructure
> > * no buildperf tests
> > * there are no supported fedora workers left so no oe-selftest-
> > fedora
> > * no ptest image execution (we never did this with sumo)
> > * no test result output (wasn't present in sumo and never
> > backported)
> >
> > I'm prepared to merge sumo-next on the basis of these results and
> > use
> > it as a model for how we could continue to get critical patches
> > into
> > older releases.
> Are you taking the other patches also submitted for sumo ?
I am worried about what the bigger picture for this looks like but we
could try testing them. I think the TSC needs to discuss this.
Cheers,
Richard
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH RFC CFH][sumo 00/47] CVE check backport
2019-11-07 16:32 ` Richard Purdie
@ 2019-11-11 10:42 ` Adrian Bunk
2019-11-11 13:12 ` Richard Purdie
0 siblings, 1 reply; 62+ messages in thread
From: Adrian Bunk @ 2019-11-11 10:42 UTC (permalink / raw)
To: Richard Purdie; +Cc: openembedded-core
On Thu, Nov 07, 2019 at 04:32:35PM +0000, Richard Purdie wrote:
> On Thu, 2019-11-07 at 07:55 -0800, akuster808 wrote:
>...
> > Are you taking the other patches also submitted for sumo ?
>
> I am worried about what the bigger picture for this looks like but we
> could try testing them. I think the TSC needs to discuss this.
How were community supported branches supposed to work?
All branches from 2.1 (sic) are documented as supported in the wiki.
> Cheers,
>
> Richard
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
^ permalink raw reply [flat|nested] 62+ messages in thread* Re: [PATCH RFC CFH][sumo 00/47] CVE check backport
2019-11-11 10:42 ` Adrian Bunk
@ 2019-11-11 13:12 ` Richard Purdie
2019-11-11 14:14 ` Adrian Bunk
0 siblings, 1 reply; 62+ messages in thread
From: Richard Purdie @ 2019-11-11 13:12 UTC (permalink / raw)
To: Adrian Bunk; +Cc: openembedded-core
On Mon, 2019-11-11 at 12:42 +0200, Adrian Bunk wrote:
> On Thu, Nov 07, 2019 at 04:32:35PM +0000, Richard Purdie wrote:
> > On Thu, 2019-11-07 at 07:55 -0800, akuster808 wrote:
> > ...
> > > Are you taking the other patches also submitted for sumo ?
> >
> > I am worried about what the bigger picture for this looks like but
> > we
> > could try testing them. I think the TSC needs to discuss this.
>
> How were community supported branches supposed to work?
>
> All branches from 2.1 (sic) are documented as supported in the wiki.
That was changed recently and I kind of wish we'd waited until we'd
followed through with a consistent plan which covers the spectrum of
EOL/community/stable/LTS.
As I've said in a few places, the TSC really needs to figure this out
and its complicated by the LTS discussions. Those discussions are
happening but aren't simple.
Cheers,
Richard
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH RFC CFH][sumo 00/47] CVE check backport
2019-11-11 13:12 ` Richard Purdie
@ 2019-11-11 14:14 ` Adrian Bunk
2019-11-11 15:54 ` Khem Raj
0 siblings, 1 reply; 62+ messages in thread
From: Adrian Bunk @ 2019-11-11 14:14 UTC (permalink / raw)
To: Richard Purdie; +Cc: openembedded-core
On Mon, Nov 11, 2019 at 01:12:47PM +0000, Richard Purdie wrote:
> On Mon, 2019-11-11 at 12:42 +0200, Adrian Bunk wrote:
> > On Thu, Nov 07, 2019 at 04:32:35PM +0000, Richard Purdie wrote:
> > > On Thu, 2019-11-07 at 07:55 -0800, akuster808 wrote:
> > > ...
> > > > Are you taking the other patches also submitted for sumo ?
> > >
> > > I am worried about what the bigger picture for this looks like but
> > > we
> > > could try testing them. I think the TSC needs to discuss this.
> >
> > How were community supported branches supposed to work?
> >
> > All branches from 2.1 (sic) are documented as supported in the wiki.
>
> That was changed recently
What was changed recently was that the pre-2.1 branches were marked EOL:
https://wiki.yoctoproject.org/wiki/index.php?title=Releases&diff=62334&oldid=62324
Non-stable branches are documented as community-supported since 2014:
https://wiki.yoctoproject.org/wiki/index.php?title=Releases&diff=12747&oldid=12743
> and I kind of wish we'd waited until we'd
> followed through with a consistent plan which covers the spectrum of
> EOL/community/stable/LTS.
>
> As I've said in a few places, the TSC really needs to figure this out
> and its complicated by the LTS discussions. Those discussions are
> happening but aren't simple.
Past releases and future releases might be separate topics.
Please keep in mind that many people already have to support products
on existing stable branches, working under the assumption that patches
submitted by the community will be accepted.
Closing future non-LTS branches early might be OK if this is part of a
clearly communicated EOL schedule for future LTS and non-LTS releases,
like it is clear when Ubuntu releases will be released and for how long
they are supported.
But this visibility on upstream support is needed before deciding on
a Yocto release for a product. Future LTS releases are irrelevant for
existing products that cannot move to a different stable branch.
> Cheers,
>
> Richard
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
^ permalink raw reply [flat|nested] 62+ messages in thread* Re: [PATCH RFC CFH][sumo 00/47] CVE check backport
2019-11-11 14:14 ` Adrian Bunk
@ 2019-11-11 15:54 ` Khem Raj
2019-11-11 16:13 ` Adrian Bunk
0 siblings, 1 reply; 62+ messages in thread
From: Khem Raj @ 2019-11-11 15:54 UTC (permalink / raw)
To: Adrian Bunk, Richard Purdie; +Cc: openembedded-core
On Mon, 2019-11-11 at 16:14 +0200, Adrian Bunk wrote:
> On Mon, Nov 11, 2019 at 01:12:47PM +0000, Richard Purdie wrote:
> > On Mon, 2019-11-11 at 12:42 +0200, Adrian Bunk wrote:
> > > On Thu, Nov 07, 2019 at 04:32:35PM +0000, Richard Purdie wrote:
> > > > On Thu, 2019-11-07 at 07:55 -0800, akuster808 wrote:
> > > > ...
> > > > > Are you taking the other patches also submitted for sumo ?
> > > >
> > > > I am worried about what the bigger picture for this looks like
> > > > but
> > > > we
> > > > could try testing them. I think the TSC needs to discuss this.
> > >
> > > How were community supported branches supposed to work?
> > >
> > > All branches from 2.1 (sic) are documented as supported in the
> > > wiki.
> >
> > That was changed recently
>
> What was changed recently was that the pre-2.1 branches were marked
> EOL:
> https://wiki.yoctoproject.org/wiki/index.php?title=Releases&diff=62334&oldid=62324
>
> Non-stable branches are documented as community-supported since 2014:
> https://wiki.yoctoproject.org/wiki/index.php?title=Releases&diff=12747&oldid=12743
>
> > and I kind of wish we'd waited until we'd
> > followed through with a consistent plan which covers the spectrum
> > of
> > EOL/community/stable/LTS.
> >
> > As I've said in a few places, the TSC really needs to figure this
> > out
> > and its complicated by the LTS discussions. Those discussions are
> > happening but aren't simple.
>
> Past releases and future releases might be separate topics.
>
> Please keep in mind that many people already have to support products
> on existing stable branches, working under the assumption that
> patches
> submitted by the community will be accepted.
>
> Closing future non-LTS branches early might be OK if this is part of
> a
> clearly communicated EOL schedule for future LTS and non-LTS
> releases,
> like it is clear when Ubuntu releases will be released and for how
> long
> they are supported.
>
> But this visibility on upstream support is needed before deciding on
> a Yocto release for a product. Future LTS releases are irrelevant
> for
> existing products that cannot move to a different stable branch.
>
current and prior two releases are actively maintained. So that should
have been the consideration when selecting a release for production in
past. see
https://wiki.yoctoproject.org/wiki/Stable_branch_maintenance
> > Cheers,
> >
> > Richard
>
> cu
> Adrian
>
> --
>
> "Is there not promise of rain?" Ling Tan asked suddenly out
> of the darkness. There had been need of rain for many days.
> "Only a promise," Lao Er said.
> Pearl S. Buck - Dragon Seed
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH RFC CFH][sumo 00/47] CVE check backport
2019-11-11 15:54 ` Khem Raj
@ 2019-11-11 16:13 ` Adrian Bunk
0 siblings, 0 replies; 62+ messages in thread
From: Adrian Bunk @ 2019-11-11 16:13 UTC (permalink / raw)
To: Khem Raj; +Cc: openembedded-core
On Mon, Nov 11, 2019 at 07:54:34AM -0800, Khem Raj wrote:
> On Mon, 2019-11-11 at 16:14 +0200, Adrian Bunk wrote:
> > On Mon, Nov 11, 2019 at 01:12:47PM +0000, Richard Purdie wrote:
>...
> > > As I've said in a few places, the TSC really needs to figure this
> > > out
> > > and its complicated by the LTS discussions. Those discussions are
> > > happening but aren't simple.
> >
> > Past releases and future releases might be separate topics.
> >
> > Please keep in mind that many people already have to support products
> > on existing stable branches, working under the assumption that
> > patches
> > submitted by the community will be accepted.
> >
> > Closing future non-LTS branches early might be OK if this is part of
> > a
> > clearly communicated EOL schedule for future LTS and non-LTS
> > releases,
> > like it is clear when Ubuntu releases will be released and for how
> > long
> > they are supported.
> >
> > But this visibility on upstream support is needed before deciding on
> > a Yocto release for a product. Future LTS releases are irrelevant
> > for
> > existing products that cannot move to a different stable branch.
>
> current and prior two releases are actively maintained.
"master and the latest 2 stable branches"
(AFAIK thud is no longer actively maintained)
> So that should
> have been the consideration when selecting a release for production in
> past. see
>
> https://wiki.yoctoproject.org/wiki/Stable_branch_maintenance
Which also says "but well-tested patches may still be accepted for them".
The question is exactly how the patch acceptance for the community
maintained stable branches works for people who are maintaining
products on these older branches.
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH RFC CFH][sumo 00/47] CVE check backport
2019-11-06 15:37 [PATCH RFC CFH][sumo 00/47] CVE check backport Mikko Rapeli
` (48 preceding siblings ...)
2019-11-06 21:46 ` [PATCH RFC CFH][sumo 00/47] " akuster808
@ 2019-11-07 11:13 ` Adrian Bunk
2019-11-07 12:13 ` Mikko.Rapeli
49 siblings, 1 reply; 62+ messages in thread
From: Adrian Bunk @ 2019-11-07 11:13 UTC (permalink / raw)
To: Mikko Rapeli; +Cc: openembedded-core
On Wed, Nov 06, 2019 at 05:37:15PM +0200, Mikko Rapeli wrote:
> Hi,
Hi Mikko,
>...
> I use sumo and due to various reasons like BSP layers, binary
> compatibility, contracts etc can't update to newer release
> or to master branch. I suspect I'm not alone.
I might end up with similar reasons, but for warrior.
And might end up doing similar longer term updates for warrior.
(not yet 100% certain)
>...
> The tooling will expose that sumo is severely lacking in security
> patches, but the tooling is a start for anyone interested, like me,
> to fill the gaps and publish patches for bitbake recipes we care
> about.
>...
Thud is officially still community maintained, as long as this is true
the point could be made that everything that gets fixed in sumo should
also get fixed in thud.
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
^ permalink raw reply [flat|nested] 62+ messages in thread* Re: [PATCH RFC CFH][sumo 00/47] CVE check backport
2019-11-07 11:13 ` Adrian Bunk
@ 2019-11-07 12:13 ` Mikko.Rapeli
2019-11-07 14:47 ` Adrian Bunk
0 siblings, 1 reply; 62+ messages in thread
From: Mikko.Rapeli @ 2019-11-07 12:13 UTC (permalink / raw)
To: bunk; +Cc: openembedded-core
Hi,
On Thu, Nov 07, 2019 at 01:13:32PM +0200, Adrian Bunk wrote:
> On Wed, Nov 06, 2019 at 05:37:15PM +0200, Mikko Rapeli wrote:
> > Hi,
>
> Hi Mikko,
>
> >...
> > I use sumo and due to various reasons like BSP layers, binary
> > compatibility, contracts etc can't update to newer release
> > or to master branch. I suspect I'm not alone.
>
> I might end up with similar reasons, but for warrior.
> And might end up doing similar longer term updates for warrior.
> (not yet 100% certain)
I'm skipping warrior but going to zeus in addition to sumo. After
insipiration from Yocto Project Summit I hope to run master branch
in some projects with regular updates, and eventually aligning to
some stable release again. Hopefully an LTS one :)
> >...
> > The tooling will expose that sumo is severely lacking in security
> > patches, but the tooling is a start for anyone interested, like me,
> > to fill the gaps and publish patches for bitbake recipes we care
> > about.
> >...
>
> Thud is officially still community maintained, as long as this is true
> the point could be made that everything that gets fixed in sumo should
> also get fixed in thud.
So to keep sumo alive, we should the also keep zeus, warrior and thud, and
of course master branch first. For some issues this actually works when
the exact same CVE patch applies, but the open question then is testing.
How should a developer test a patch before submitting it, or multiple versions
of it?
I'm testing in project tree with CI and target tests, then compile testing on
master. qemu ptest runs would be nice but not sure how to get a stable or useful
test set for various branches.
To make things more complicated, the project trees sadly contain more backports, fixes
and workarounds which are not suitable for upstreaming into stable or even master
branches.
-Mikko
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH RFC CFH][sumo 00/47] CVE check backport
2019-11-07 12:13 ` Mikko.Rapeli
@ 2019-11-07 14:47 ` Adrian Bunk
0 siblings, 0 replies; 62+ messages in thread
From: Adrian Bunk @ 2019-11-07 14:47 UTC (permalink / raw)
To: Mikko.Rapeli; +Cc: openembedded-core
On Thu, Nov 07, 2019 at 12:13:51PM +0000, Mikko.Rapeli@bmw.de wrote:
> Hi,
Hi Mikko,
> On Thu, Nov 07, 2019 at 01:13:32PM +0200, Adrian Bunk wrote:
> > On Wed, Nov 06, 2019 at 05:37:15PM +0200, Mikko Rapeli wrote:
> > > Hi,
> >
> > Hi Mikko,
> >
> > >...
> > > I use sumo and due to various reasons like BSP layers, binary
> > > compatibility, contracts etc can't update to newer release
> > > or to master branch. I suspect I'm not alone.
> >
> > I might end up with similar reasons, but for warrior.
> > And might end up doing similar longer term updates for warrior.
> > (not yet 100% certain)
>
> I'm skipping warrior but going to zeus in addition to sumo. After
> insipiration from Yocto Project Summit I hope to run master branch
> in some projects with regular updates, and eventually aligning to
> some stable release again. Hopefully an LTS one :)
everyone is currently running projects on different releases.
Let's hope LTS will happen, and that with a properly communicated LTS
schedule most distributions and users will switch to the LTS releases
just like what happened with Ubuntu.
> > >...
> > > The tooling will expose that sumo is severely lacking in security
> > > patches, but the tooling is a start for anyone interested, like me,
> > > to fill the gaps and publish patches for bitbake recipes we care
> > > about.
> > >...
> >
> > Thud is officially still community maintained, as long as this is true
> > the point could be made that everything that gets fixed in sumo should
> > also get fixed in thud.
>
> So to keep sumo alive, we should the also keep zeus, warrior and thud, and
> of course master branch first. For some issues this actually works when
> the exact same CVE patch applies, but the open question then is testing.
>...
When a branch is EOL it is documented to be dead.
But upgrading to a more recent non-EOL branch, e.g. sumo to thud,
should not result in losing (security) fixes.
The root problem is that "community support" for a stable branch in
practice often means "no support".
If sumo is supported but thud is not, this should at least be made
visible to users.
> -Mikko
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
^ permalink raw reply [flat|nested] 62+ messages in thread