All of lore.kernel.org
 help / color / mirror / Atom feed
From: Armin Kuster <akuster808@gmail.com>
To: openembedded-core@openembedded.org
Subject: [warrior 24/32] cve-update-db-native: clean up proxy handling
Date: Thu, 16 Jan 2020 07:20:33 -0800	[thread overview]
Message-ID: <2ddf1c0bc4267d38069f9dbb0f716fdac29a49a9.1579187957.git.akuster808@gmail.com> (raw)
In-Reply-To: <cover.1579187957.git.akuster808@gmail.com>

From: Ross Burton <ross.burton@intel.com>

urllib handles adding proxy handlers if the proxies are set in the environment,
so call bb.utils.export_proxies() to do that and remove the manual setup.

(From OE-Core rev: 6b73004668b3b71c9c38814b79fbb58c893ed434)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 .../recipes-core/meta/cve-update-db-native.bb | 31 +++----------------
 1 file changed, 5 insertions(+), 26 deletions(-)

diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb
index 08b18f064f..db1d69a28e 100644
--- a/meta/recipes-core/meta/cve-update-db-native.bb
+++ b/meta/recipes-core/meta/cve-update-db-native.bb
@@ -21,10 +21,12 @@ python do_populate_cve_db() {
     """
     Update NVD database with json data feed
     """
-
+    import bb.utils
     import sqlite3, urllib, urllib.parse, shutil, gzip
     from datetime import date
 
+    bb.utils.export_proxies(d)
+
     BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-"
     YEAR_START = 2002
 
@@ -40,16 +42,6 @@ python do_populate_cve_db() {
     except OSError:
         pass
 
-    proxy = d.getVar("https_proxy")
-    if proxy:
-        # instantiate an opener but do not install it as the global
-        # opener unless if we're really sure it's applicable for all
-        # urllib requests
-        proxy_handler = urllib.request.ProxyHandler({'https': proxy})
-        proxy_opener = urllib.request.build_opener(proxy_handler)
-    else:
-        proxy_opener = None
-
     cve_f = open(os.path.join(d.getVar("TMPDIR"), 'cve_check'), 'a')
 
     if not os.path.isdir(db_dir):
@@ -67,15 +59,7 @@ python do_populate_cve_db() {
         json_url = year_url + ".json.gz"
 
         # Retrieve meta last modified date
-
-        response = None
-
-        if proxy_opener:
-            response = proxy_opener.open(meta_url)
-        else:
-            req = urllib.request.Request(meta_url)
-            response = urllib.request.urlopen(req)
-
+        response = urllib.request.urlopen(meta_url)
         if response:
             for l in response.read().decode("utf-8").splitlines():
                 key, value = l.split(":", 1)
@@ -95,12 +79,7 @@ python do_populate_cve_db() {
 
             # Update db with current year json file
             try:
-                if proxy_opener:
-                    response = proxy_opener.open(json_url)
-                else:
-                    req = urllib.request.Request(json_url)
-                    response = urllib.request.urlopen(req)
-
+                response = urllib.request.urlopen(json_url)
                 if response:
                     update_db(c, gzip.decompress(response.read()).decode('utf-8'))
                 c.execute("insert or replace into META values (?, ?)", [year, last_modified])
-- 
2.17.1



  parent reply	other threads:[~2020-01-16 15:21 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-16 15:20 [warrior 00/32] Pull request Armin Kuster
2020-01-16 15:20 ` [warrior 01/32] bind: fix CVE-2019-6471 and CVE-2018-5743 Armin Kuster
2020-01-16 15:20 ` [warrior 02/32] python: fix CVE-2019-16935 Armin Kuster
2020-01-16 15:20 ` [warrior 03/32] python: fix CVE-2018-20852 Armin Kuster
2020-01-16 15:20 ` [warrior 04/32] python: update to 2.7.17 Armin Kuster
2020-01-16 15:20 ` [warrior 05/32] openssl: set CVE vendor to openssl Armin Kuster
2020-01-16 15:20 ` [warrior 06/32] flex: set CVE_PRODUCT to include vendor Armin Kuster
2020-01-16 15:20 ` [warrior 07/32] git: set CVE vendor to git-scm Armin Kuster
2020-01-16 15:20 ` [warrior 08/32] subversion: set CVE vendor to Apache Armin Kuster
2020-01-16 15:20 ` [warrior 09/32] ed: set CVE vendor to avoid false positives Armin Kuster
2020-01-16 15:20 ` [warrior 10/32] boost: set CVE vendor to Boost Armin Kuster
2020-01-16 15:20 ` [warrior 11/32] libpam: set CVE_PRODUCT Armin Kuster
2020-01-16 15:20 ` [warrior 12/32] libxfont2: " Armin Kuster
2020-01-16 15:20 ` [warrior 13/32] libsndfile1: whitelist CVE-2018-13419 Armin Kuster
2020-01-16 15:20 ` [warrior 14/32] webkitgtk: set CVE_PRODUCT Armin Kuster
2020-01-16 15:20 ` [warrior 15/32] procps: whitelist CVE-2018-1121 Armin Kuster
2020-01-16 15:20 ` [warrior 16/32] libpcap: upgrade 1.9.0 -> 1.9.1 Armin Kuster
2020-01-16 15:20 ` [warrior 17/32] sdk: Install nativesdk locales for all TCLIBC variants Armin Kuster
2020-01-16 15:20 ` [warrior 18/32] cve-check: ensure all known CVEs are in the report Armin Kuster
2020-01-16 15:20 ` [warrior 19/32] cve-check: failure to parse versions should be more visible Armin Kuster
2020-01-16 15:20 ` [warrior 20/32] cve-check: we don't actually need to unpack to check Armin Kuster
2020-01-16 15:20 ` [warrior 21/32] cve-update-db-native: don't refresh more than once an hour Armin Kuster
2020-01-16 15:20 ` [warrior 22/32] cve-update-db-native: don't hardcode the database name Armin Kuster
2020-01-16 15:20 ` [warrior 23/32] cve-update-db-native: add an index on the CVE ID column Armin Kuster
2020-01-16 15:20 ` Armin Kuster [this message]
2020-01-16 15:20 ` [warrior 25/32] cve-check: rewrite look to fix false negatives Armin Kuster
2020-01-16 15:20 ` [warrior 26/32] cve-check: neaten get_cve_info Armin Kuster
2020-01-16 15:20 ` [warrior 27/32] cve-check: fetch CVE data once at a time instead of in a single call Armin Kuster
2020-01-16 15:20 ` [warrior 28/32] cve-check: Switch to NVD CVE JSON feed version 1.1 Armin Kuster
2020-01-16 15:20 ` [warrior 29/32] popt: fix SRC_URI Armin Kuster
2020-01-16 15:20 ` [warrior 30/32] sudo: correct SRC_URI Armin Kuster
2020-01-16 15:20 ` [warrior 31/32] sudo: Fix fetching sources Armin Kuster
2020-01-16 15:20 ` [warrior 32/32] stress: update SRC_URI Armin Kuster

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2ddf1c0bc4267d38069f9dbb0f716fdac29a49a9.1579187957.git.akuster808@gmail.com \
    --to=akuster808@gmail.com \
    --cc=openembedded-core@openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.