[PATCH v2] fetch2/wget: fetch securely by default

[PATCH v2] fetch2/wget: fetch securely by default

[Ross Burton]

The days of broken certificates are behind us now, so instead of always
passing --no-check-certificate to wget, don't pass it by default and
instead only pass it BB_CHECK_SSL_CERTS = "0".

[ YOCTO #14108 ]

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 .../bitbake-user-manual-fetching.rst            |  4 ++++
 .../bitbake-user-manual-ref-variables.rst       |  4 ++++
 bitbake/lib/bb/fetch2/wget.py                   | 17 ++++++++++++++---
 3 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
index bd1fb4fc74e..5980349ed55 100644
--- a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
@@ -144,6 +144,10 @@ download without a checksum triggers an error message. The
 make any attempted network access a fatal error, which is useful for
 checking that mirrors are complete as well as other things.
 
+If :term:`BB_CHECK_SSL_CERTS` is set to ``0`` then SSL certificate checking will
+be disabled. This variable defaults to ``1`` so SSL certificates are normally
+checked.
+
 .. _bb-the-unpack:
 
 The Unpack
diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
index 2dca52c4a09..4f989b0fcd2 100644
--- a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst
@@ -92,6 +92,10 @@ overview of their function and contents.
       fetcher does not attempt to use the host listed in ``SRC_URI`` after
       a successful fetch from the ``PREMIRRORS`` occurs.
 
+   :term:`BB_CHECK_SSL_CERTS`
+      Specifies if SSL certificates should be checked when fetching. The default
+      value is ``1`` and certificates are not checked if the value is set to ``0``.
+
    :term:`BB_CONSOLELOG`
       Specifies the path to a log file into which BitBake's user interface
       writes output during the build.
diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index 784df70c9f6..cbd88b81026 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -52,13 +52,19 @@ class WgetProgressHandler(bb.progress.LineFilterProgressHandler):
 
 
 class Wget(FetchMethod):
+    """Class to fetch urls via 'wget'"""
 
     # CDNs like CloudFlare may do a 'browser integrity test' which can fail
     # with the standard wget/urllib User-Agent, so pretend to be a modern
     # browser.
     user_agent = "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:84.0) Gecko/20100101 Firefox/84.0"
 
-    """Class to fetch urls via 'wget'"""
+    def check_certs(self, d):
+        """
+        Should certificates be checked?
+        """
+        return (d.getVar("BB_CHECK_SSL_CERTS") or "1") != "0"
+
     def supports(self, ud, d):
         """
         Check to see if a given url can be fetched with wget.
@@ -82,7 +88,10 @@ class Wget(FetchMethod):
         if not ud.localfile:
             ud.localfile = d.expand(urllib.parse.unquote(ud.host + ud.path).replace("/", "."))
 
-        self.basecmd = d.getVar("FETCHCMD_wget") or "/usr/bin/env wget -t 2 -T 30 --passive-ftp --no-check-certificate"
+        self.basecmd = d.getVar("FETCHCMD_wget") or "/usr/bin/env wget -t 2 -T 30 --passive-ftp"
+
+        if not self.check_certs(d):
+            self.basecmd += " --no-check-certificate"
 
     def _runwget(self, ud, d, command, quiet, workdir=None):
 
@@ -288,12 +297,14 @@ class Wget(FetchMethod):
         if exported_proxies:
             handlers.append(urllib.request.ProxyHandler())
         handlers.append(CacheHTTPHandler())
+
         # Since Python 2.7.9 ssl cert validation is enabled by default
         # see PEP-0476, this causes verification errors on some https servers
         # so disable by default.
         import ssl
-        if hasattr(ssl, '_create_unverified_context'):
+        if not self.check_certs(d) and hasattr(ssl, '_create_unverified_context'):
             handlers.append(urllib.request.HTTPSHandler(context=ssl._create_unverified_context()))
+
         opener = urllib.request.build_opener(*handlers)
 
         try:
-- 
2.25.1

Re: [bitbake-devel] [PATCH v2] fetch2/wget: fetch securely by default

[Richard Purdie]

On Mon, 2021-06-28 at 18:12 +0100, Ross Burton wrote:
> The days of broken certificates are behind us now, so instead of always
> passing --no-check-certificate to wget, don't pass it by default and
> instead only pass it BB_CHECK_SSL_CERTS = "0".
> 
> [ YOCTO #14108 ]
> 
> Signed-off-by: Ross Burton <ross.burton@arm.com>
> ---
>  .../bitbake-user-manual-fetching.rst            |  4 ++++
>  .../bitbake-user-manual-ref-variables.rst       |  4 ++++
>  bitbake/lib/bb/fetch2/wget.py                   | 17 ++++++++++++++---
>  3 files changed, 22 insertions(+), 3 deletions(-)
> 
[...]
> @@ -288,12 +297,14 @@ class Wget(FetchMethod):
>          if exported_proxies:
>              handlers.append(urllib.request.ProxyHandler())
>          handlers.append(CacheHTTPHandler())
> +
>          # Since Python 2.7.9 ssl cert validation is enabled by default
>          # see PEP-0476, this causes verification errors on some https servers
>          # so disable by default.
>          import ssl
> -        if hasattr(ssl, '_create_unverified_context'):
> +        if not self.check_certs(d) and hasattr(ssl, '_create_unverified_context'):
>              handlers.append(urllib.request.HTTPSHandler(context=ssl._create_unverified_context()))
> +
>          opener = urllib.request.build_opener(*handlers)

The comment here is now incorrect. I tweaked it in the version I queued in -next.

Cheers,

Richard