[PATCH v2 0/4] Improve CVE check and patching documentation

[PATCH v2 0/4] Improve CVE check and patching documentation

[michael.opdenacker]

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

From: Mikko Rapeli <mikko.rapeli@linaro.org>

I think detecting and fixing CVE security issues in yocto based distros
is quite important so improve the documentation around it. I've been
using cve-check.bbclass for a long time and these details hopefully
make it easier for others to start using it as well.

Changes in V2:
- Misc wording fixes by Michael Opdenacker

Mikko Rapeli (4):
  ref-manual: variables.rst: add documentation for CVE_VERSION
  ref-manual: classes.rst: improve documentation for cve-check.bbclass
  dev-manual: common-tasks.rst: add regular updates and CVE scans to
    security best practices
  dev-manual: common-tasks.rst: refactor and improve "Checking for
    Vulnerabilities" section

 documentation/dev-manual/common-tasks.rst | 183 +++++++++++++++++-----
 documentation/ref-manual/classes.rst      |  52 +++++-
 documentation/ref-manual/variables.rst    |  12 ++
 3 files changed, 204 insertions(+), 43 deletions(-)

-- 
2.34.1

[PATCH v2 1/4] ref-manual: variables.rst: add documentation for CVE_VERSION

[michael.opdenacker]

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

From: Mikko Rapeli <mikko.rapeli@linaro.org>

Related to cve-check.bbclass.

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 documentation/ref-manual/variables.rst | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index 71e8c272a7..0f9df3ac20 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -1508,6 +1508,18 @@ system and gives an overview of their function and contents.
 
          CVE_PRODUCT = "vendor:package"
 
+   :term:`CVE_VERSION`
+      In a recipe, defines the version used to match the recipe version
+      against the version in the `NIST CVE database <https://nvd.nist.gov/>`__
+      when usign :ref:`cve-check <ref-classes-cve-check>`.
+
+      The default is ${:term:`PV`} but if recipes use custom version numbers
+      which do not map to upstream software component release versions and the versions
+      used in the CVE database, then this variable can be used to set the
+      version number for :ref:`cve-check <ref-classes-cve-check>`. Example::
+
+          CVE_VERSION = "2.39"
+
    :term:`CVSDIR`
       The directory in which files checked out under the CVS system are
       stored.
-- 
2.34.1

Re: [docs] [PATCH v2 1/4] ref-manual: variables.rst: add documentation for CVE_VERSION

[Richard Purdie]

On Wed, 2022-10-26 at 18:07 +0200, Michael Opdenacker via
lists.yoctoproject.org wrote:
> From: Michael Opdenacker <michael.opdenacker@bootlin.com>
> 
> From: Mikko Rapeli <mikko.rapeli@linaro.org>

I think your "From:" injection script may need a tweak to avoid two
entries!

Cheers,

Richard

Re: [docs] [PATCH v2 1/4] ref-manual: variables.rst: add documentation for CVE_VERSION

[Michael Opdenacker]

Hi Richard

On 10/26/22 18:32, Richard Purdie wrote:
> On Wed, 2022-10-26 at 18:07 +0200, Michael Opdenacker via
> lists.yoctoproject.org wrote:
>> From: Michael Opdenacker <michael.opdenacker@bootlin.com>
>>
>> From: Mikko Rapeli <mikko.rapeli@linaro.org>
> I think your "From:" injection script may need a tweak to avoid two
> entries!


Oh, this is funny! Actually, I don't have any special script.

Mikko's "From:" was added by "git format-patch"
Mine was added by "git send-email".

I'll keep an eye on this.
Good catch, thanks!

Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH v2 2/4] ref-manual: classes.rst: improve documentation for cve-check.bbclass

[michael.opdenacker]

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

From: Mikko Rapeli <mikko.rapeli@linaro.org>

It is a quite important tool for maintaining yocto based products
so documentation should include the best practices.

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 documentation/ref-manual/classes.rst | 52 ++++++++++++++++++++++++++--
 1 file changed, 50 insertions(+), 2 deletions(-)

diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst
index 1880e44486..cce0269b9a 100644
--- a/documentation/ref-manual/classes.rst
+++ b/documentation/ref-manual/classes.rst
@@ -412,13 +412,61 @@ discussion on these cross-compilation tools.
 =====================
 
 The :ref:`cve-check <ref-classes-cve-check>` class looks for known CVEs (Common Vulnerabilities
-and Exposures) while building an image. This class is meant to be
+and Exposures) while building with BitBake. This class is meant to be
 inherited globally from a configuration file::
 
    INHERIT += "cve-check"
 
+To filter out obsolete CVE database entries which are known not to impact software from Poky and OE-Core,
+add following line to the build configuration file::
+
+   include cve-extra-exclusions.inc
+
 You can also look for vulnerabilities in specific packages by passing
-``-c cve_check`` to BitBake. You will find details in the
+``-c cve_check`` to BitBake.
+
+After building the software with Bitbake, CVE check output reports are available in ``tmp/deploy/cve``
+and image specific summaries in ``tmp/deploy/images/*.cve`` or ``tmp/deploy/images/*.json`` files.
+
+When building, the CVE checker will emit build time warnings for any detected
+issues which are in the state ``Unpatched``, meaning that CVE issue seems to affect the software component
+and version being compiled and no patches to address the issue are applied. Other states
+for detected CVE issues are: ``Patched`` meaning that a patch to address the issue is already
+applied, and ``Ignored`` meaning that the issue can be ignored.
+
+The ``Patched`` state of a CVE issue is detected from patch files with the format
+``CVE-ID.patch``, e.g. ``CVE-2019-20633.patch``, in the :term:`SRC_URI` and using
+CVE metadata of format ``CVE: CVE-ID`` in the commit message of the patch file.
+
+If the recipe lists the ``CVE-ID`` in :term:`CVE_CHECK_IGNORE` variable, then the CVE state is reported
+as ``Ignored``. Multiple CVEs can be listed separated by spaces. Example::
+
+   CVE_CHECK_IGNORE += "CVE-2020-29509 CVE-2020-29511"
+
+If CVE check reports that a recipe contains false positives or false negatives, these may be
+fixed in recipes by adjusting the CVE product name using :term:`CVE_PRODUCT` and :term:`CVE_VERSION` variables.
+:term:`CVE_PRODUCT` defaults to the plain recipe name :term:`BPN` which can be adjusted to one or more CVE
+database vendor and product pairs using the syntax::
+
+   CVE_PRODUCT = "flex_project:flex"
+
+where ``flex_project`` is the CVE database vendor name and ``flex`` is the product name. Similarly
+if the default recipe version :term:`PV` does not match the version numbers of the software component
+in upstream releases or the CVE database, then the :term:`CVE_VERSION` variable can be used to set the
+CVE database compatible version number, for example::
+
+   CVE_VERSION = "2.39"
+
+Any bugs or missing or incomplete information in the CVE database entries should be fixed in the CVE database
+via the `NVD feedback form <https://nvd.nist.gov/info/contact-form>`__.
+
+Users should note that security is a process, not a product, and thus also CVE checking, analyzing results,
+patching and updating the software should be done as a regular process. The data and assumptions
+required for CVE checker to reliably detect issues are frequently broken in various ways.
+These can only be detected by reviewing the details of the issues and iterating over the generated reports,
+and following what happens in other Linux distributions and in the greater open source community.
+
+You will find some more details in the
 ":ref:`dev-manual/common-tasks:checking for vulnerabilities`"
 section in the Development Tasks Manual.
 
-- 
2.34.1

[PATCH v2 3/4] dev-manual: common-tasks.rst: add regular updates and CVE scans to security best practices

[michael.opdenacker]

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

From: Mikko Rapeli <mikko.rapeli@linaro.org>

Regular security scans and updates to fix issues and updates from
upstream maintainers are best practices.

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 documentation/dev-manual/common-tasks.rst | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/documentation/dev-manual/common-tasks.rst b/documentation/dev-manual/common-tasks.rst
index 53e7686633..d435bc8a4c 100644
--- a/documentation/dev-manual/common-tasks.rst
+++ b/documentation/dev-manual/common-tasks.rst
@@ -6231,6 +6231,13 @@ more secure:
    vulnerabilities discovered in the future. This consideration
    especially applies when your device is network-enabled.
 
+-  Regularly scan and apply fixes for CVE security issues affecting
+   all software components in the product, see ":ref:`dev-manual/common-tasks:checking for vulnerabilities`".
+
+-  Regularly update your version of Poky and OE-Core from their upstream
+   developers, e.g. to apply updates and security fixes from stable
+   and LTS branches.
+
 -  Ensure you remove or disable debugging functionality before producing
    the final image. For information on how to do this, see the
    ":ref:`dev-manual/common-tasks:considerations specific to the openembedded build system`"
-- 
2.34.1

[PATCH v2 4/4] dev-manual: common-tasks.rst: refactor and improve "Checking for Vulnerabilities" section

[michael.opdenacker]

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

From: Mikko Rapeli <mikko.rapeli@linaro.org>

Add sub section to how Poky and OE-Core handle CVE security issues. This
is a generic intro chapter. Also add note that this is a process which
needs quite a bit of review and iteration to keep products and SW stack
secure, a process not a product.

Then change "Vulnerabilites in images" chapter to
"Vulnerability check at build time" since the process applies to
anything compiled with bitbake, not just images.

Explain details of how to work with cve-check.bbclass, especially
the states Patched, Unpatched and Ignored in the generated reports.

Rename recipe chapter to "Fixing CVE product name and version mappings"
since CVE check has some default which works for all recipes
but generated reports may be completely broken. Fixes are then done with
CVE_PRODUCT and CVE_VERSION.

Give some hints how to analyze "Unpatched" CVEs by checking what happens
in other Linux distros etc.

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
---
 documentation/dev-manual/common-tasks.rst | 176 +++++++++++++++++-----
 1 file changed, 135 insertions(+), 41 deletions(-)

diff --git a/documentation/dev-manual/common-tasks.rst b/documentation/dev-manual/common-tasks.rst
index d435bc8a4c..ffa85395a9 100644
--- a/documentation/dev-manual/common-tasks.rst
+++ b/documentation/dev-manual/common-tasks.rst
@@ -11502,8 +11502,8 @@ the license from the fetched source::
 Checking for Vulnerabilities
 ============================
 
-Vulnerabilities in images
--------------------------
+Vulnerabilities in Poky and OE-Core
+-----------------------------------
 
 The Yocto Project has an infrastructure to track and address unfixed
 known security vulnerabilities, as tracked by the public
@@ -11516,14 +11516,78 @@ for packages in Poky and OE-Core, tracking the evolution of the number of
 unpatched CVEs and the status of patches. Such information is available for
 the current development version and for each supported release.
 
-To know which packages are vulnerable to known security vulnerabilities
-in the specific image you are building, add the following setting to your
-configuration::
+Security is a process, not a product, and thus at any time, a number of security
+issues may be impacting Poky and OE-Core. It is up to the maintainers, users,
+contributors and anyone interested in the issues to investigate and possibly fix them by
+updating software components to newer versions or by applying patches to address them.
+It is recommended to work with Poky and OE-Core upstream maintainers and submit
+patches to fix them, see ":ref:`dev-manual/common-tasks:submitting a change to the yocto project`" for details.
+
+Vulnerability check at build time
+---------------------------------
+
+To enable a check for CVE security vulnerabilities using :ref:`cve-check <ref-classes-cve-check>` in the specific image
+or target you are building, add the following setting to your configuration::
 
    INHERIT += "cve-check"
 
-This way, at build time, BitBake will warn you about known CVEs
-as in the example below::
+The CVE database contains some old incomplete entries which have been
+deemed not to impact Poky or OE-Core. These CVE entries can be excluded from the
+check using build configuration::
+
+   include conf/distro/include/cve-extra-exclusions.inc
+
+With this CVE check enabled, BitBake build will try to map each compiled software component
+recipe name and version information to the CVE database and generate recipe and
+image specific reports. These reports will contain:
+
+ - meta data about the software component like names and versions
+
+ - metadata about the CVE issue such as description and NVD link
+
+ - for each software component, a list of CVEs which are possibly impacting this version
+
+ - status of each CVE: ``Patched``, ``Unpatched`` or ``Ignored``
+
+The status ``Patched`` means that a patch file to address the security issue has been
+applied. ``Unpatched`` status means that no patches to address the issue have been
+applied and that the issue needs to be investigated. ``Ignored`` means that after
+analysis, it has been deemed to ignore the issue as it for example affects
+the software component on a different operating system platform.
+
+After build with CVE check enabled, reports for each compiled source recipe will be
+found in ``build/tmp/deploy/cve``.
+
+For example the CVE check report for the ``flex-native`` recipe looks like::
+
+   $ cat poky/build/tmp/deploy/cve/flex-native
+   LAYER: meta
+   PACKAGE NAME: flex-native
+   PACKAGE VERSION: 2.6.4
+   CVE: CVE-2016-6354
+   CVE STATUS: Patched
+   CVE SUMMARY: Heap-based buffer overflow in the yy_get_next_buffer function in Flex before 2.6.1 might allow context-dependent attackers to cause a denial of service or possibly execute arbitrary code via vectors involving num_to_read.
+   CVSS v2 BASE SCORE: 7.5
+   CVSS v3 BASE SCORE: 9.8
+   VECTOR: NETWORK
+   MORE INFORMATION: https://nvd.nist.gov/vuln/detail/CVE-2016-6354
+
+   LAYER: meta
+   PACKAGE NAME: flex-native
+   PACKAGE VERSION: 2.6.4
+   CVE: CVE-2019-6293
+   CVE STATUS: Ignored
+   CVE SUMMARY: An issue was discovered in the function mark_beginning_as_normal in nfa.c in flex 2.6.4. There is a stack exhaustion problem caused by the mark_beginning_as_normal function making recursive calls to itself in certain scenarios involving lots of '*' characters. Remote attackers could leverage this vulnerability to cause a denial-of-service.
+   CVSS v2 BASE SCORE: 4.3
+   CVSS v3 BASE SCORE: 5.5
+   VECTOR: NETWORK
+   MORE INFORMATION: https://nvd.nist.gov/vuln/detail/CVE-2019-6293
+
+For images, a summary of all recipes included in the image and their CVEs is also
+generated in textual and JSON formats. These ``.cve`` and ``.json`` reports can be found
+in the ``tmp/deploy/images`` directory for each compiled image.
+
+At build time CVE check will also throw warnings about ``Unpatched`` CVEs::
 
    WARNING: flex-2.6.4-r0 do_cve_check: Found unpatched CVE (CVE-2019-6293), for more information check /poky/build/tmp/work/core2-64-poky-linux/flex/2.6.4-r0/temp/cve.log
    WARNING: libarchive-3.5.1-r0 do_cve_check: Found unpatched CVE (CVE-2021-36976), for more information check /poky/build/tmp/work/core2-64-poky-linux/libarchive/3.5.1-r0/temp/cve.log
@@ -11532,21 +11596,46 @@ It is also possible to check the CVE status of individual packages as follows::
 
    bitbake -c cve_check flex libarchive
 
-Note that OpenEmbedded-Core keeps a list of known unfixed CVE issues which can
-be ignored. You can pass this list to the check as follows::
+Fixing CVE product name and version mappings
+--------------------------------------------
 
-   bitbake -c cve_check libarchive -R conf/distro/include/cve-extra-exclusions.inc
+By default, :ref:`cve-check <ref-classes-cve-check>` uses the recipe name :term:`BPN` as CVE
+product name when querying the CVE database. If this mapping contains false positives, e.g.
+some reported CVEs are not for the software component in question, or false negatives like
+some CVEs are not found to impact the recipe when they should, then the problems can be
+in the recipe name to CVE product mapping. These mapping issues can be fixed by setting
+the :term:`CVE_PRODUCT` variable inside the recipe. This defines the name of software component in the
+upstream `NIST CVE database <https://nvd.nist.gov/>`__.
 
-Enabling vulnerabily tracking in recipes
-----------------------------------------
+The variable supports using vendor and product names like this::
 
-The :term:`CVE_PRODUCT` variable defines the name used to match the recipe name
-against the name in the upstream `NIST CVE database <https://nvd.nist.gov/>`__.
+   CVE_PRODUCT = "flex_project:flex"
 
-Editing recipes to fix vulnerabilities
---------------------------------------
+In this example from the vendor name used in CVE database is ``flex_project`` and
+product is ``flex``. With this setting the ``flex`` recipe only maps to this specific
+product and not products from other vendors with same name ``flex``.
+
+Similary, when the recipe version :term:`PV` is not compatible with software versions used by
+the upstream software component releases and the CVE database, these can be fixed using
+:term:`CVE_VERSION` variable.
+
+Note that if the CVE entries in NVD databse contain bugs or have missing or incomplete
+information, it is recommended to fix the information there directly instead of working
+around the issues for a possibly long time in Poky and OE-Core side recipes. Feedback to
+NVD about CVEs entries can be provided through the `NVD contact form <https://nvd.nist.gov/info/contact-form>`__.
+
+Fixing vulnerabilities in recipes
+---------------------------------
+
+If a CVE security issue impacts a software component, it can be fixed by updating to a newer
+version of the software component or by applying a patch. For Poky and OE-Core master branches, updating
+to newer software component release with fixes is the best option, but patches can be applied
+if releases are not yet available.
+
+For stable branches, it is preferred to apply patches for the issues. For some software
+components minor version updates can also applied if they are backwards compatible.
 
-To fix a given known vulnerability, you need to add a patch file to your recipe. Here's
+Here is an example of fixing CVE security issues with patch files,
 an example from the :oe_layerindex:`ffmpeg recipe</layerindex/recipe/47350>`::
 
    SRC_URI = "https://www.ffmpeg.org/releases/${BP}.tar.xz \
@@ -11558,31 +11647,21 @@ an example from the :oe_layerindex:`ffmpeg recipe</layerindex/recipe/47350>`::
               file://fix-CVE-2020-22033-CVE-2020-22019.patch \
               file://fix-CVE-2021-33815.patch \
 
-The :ref:`cve-check <ref-classes-cve-check>` class defines two ways of
-supplying a patch for a given CVE. The first
-way is to use a patch filename that matches the below pattern::
+A good practice is to include the CVE identifier in both patch file name
+and inside the patch file commit message use the format::
 
-   cve_file_name_match = re.compile(".*([Cc][Vv][Ee]\-\d{4}\-\d+)")
+   CVE: CVE-2020-22033
 
-As shown in the example above, multiple CVE IDs can appear in a patch filename,
-but the :ref:`cve-check <ref-classes-cve-check>` class will only consider
-the last CVE ID in the filename as patched.
+CVE checker will then capture this information and change the CVE status to ``Patched``
+in the generated reports.
 
-The second way to recognize a patched CVE ID is when a line matching the
-below pattern is found in any patch file provided by the recipe::
+If analysis shows that the CVE issue does not impact the recipe due to configuration, platform,
+version or other reasons, the CVE can be marked as ``Ignored`` using :term:`CVE_CHECK_IGNORE` variable.
+As mentioned previously, if data in the CVE database is wrong, it is recommend to fix those
+issues in the CVE database directly.
 
-  cve_match = re.compile("CVE:( CVE\-\d{4}\-\d+)+")
-
-This allows a single patch file to address multiple CVE IDs at the same time.
-
-Of course, another way to fix vulnerabilities is to upgrade to a version
-of the package which is not impacted, typically a more recent one.
-The NIST database knows which versions are vulnerable and which ones
-are not.
-
-Last but not least, you can choose to ignore vulnerabilities through
-the :term:`CVE_CHECK_SKIP_RECIPE` and :term:`CVE_CHECK_IGNORE`
-variables.
+Recipes can be completely skipped by CVE check by including the recipe name in
+the :term:`CVE_CHECK_SKIP_RECIPE` variable.
 
 Implementation details
 ----------------------
@@ -11600,23 +11679,38 @@ Then, the code looks up all the CVE IDs in the NIST database for all the
 products defined in :term:`CVE_PRODUCT`. Then, for each found CVE:
 
  - If the package name (:term:`PN`) is part of
-   :term:`CVE_CHECK_SKIP_RECIPE`, it is considered as patched.
+   :term:`CVE_CHECK_SKIP_RECIPE`, it is considered as ``Patched``.
 
  - If the CVE ID is part of :term:`CVE_CHECK_IGNORE`, it is
-   considered as patched too.
+   set as ``Ignored``.
 
  - If the CVE ID is part of the patched CVE for the recipe, it is
    already considered as patched.
 
  - Otherwise, the code checks whether the recipe version (:term:`PV`)
    is within the range of versions impacted by the CVE. If so, the CVE
-   is considered as unpatched.
+   is considered as ``Unpatched``.
 
 The CVE database is stored in :term:`DL_DIR` and can be inspected using
 ``sqlite3`` command as follows::
 
    sqlite3 downloads/CVE_CHECK/nvdcve_1.1.db .dump | grep CVE-2021-37462
 
+When analyzing CVEs, it is recommended to:
+
+ - study the latest information in `CVE database <https://nvd.nist.gov/vuln/search>`.
+
+ - check how upstream developers of the software component addressed the issue, e.g.
+   what patch was applied, which upstream release contains the fix.
+
+ - check what other Linux distributions like `Debian <https://security-tracker.debian.org/tracker/>`
+   did to analyze and address the issue.
+
+ - follow security notices from other Linux distributions.
+
+ - follow public `open source security mailing lists <https://oss-security.openwall.org/wiki/mailing-lists>` for
+   discussions and advance notifications of CVE bugs and software releases with fixes.
+
 Using the Error Reporting Tool
 ==============================
 
-- 
2.34.1

Re: [PATCH v2 4/4] dev-manual: common-tasks.rst: refactor and improve "Checking for Vulnerabilities" section

[Mikko Rapeli]

Hi,

On Wed, Oct 26, 2022 at 06:07:13PM +0200, michael.opdenacker@bootlin.com wrote:
> From: Michael Opdenacker <michael.opdenacker@bootlin.com>
> 
> From: Mikko Rapeli <mikko.rapeli@linaro.org>
> 
> Add sub section to how Poky and OE-Core handle CVE security issues. This
> is a generic intro chapter. Also add note that this is a process which
> needs quite a bit of review and iteration to keep products and SW stack
> secure, a process not a product.
> 
> Then change "Vulnerabilites in images" chapter to
> "Vulnerability check at build time" since the process applies to
> anything compiled with bitbake, not just images.
> 
> Explain details of how to work with cve-check.bbclass, especially
> the states Patched, Unpatched and Ignored in the generated reports.
> 
> Rename recipe chapter to "Fixing CVE product name and version mappings"
> since CVE check has some default which works for all recipes
> but generated reports may be completely broken. Fixes are then done with
> CVE_PRODUCT and CVE_VERSION.
> 
> Give some hints how to analyze "Unpatched" CVEs by checking what happens
> in other Linux distros etc.
> 
> Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
> Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
> ---
>  documentation/dev-manual/common-tasks.rst | 176 +++++++++++++++++-----
>  1 file changed, 135 insertions(+), 41 deletions(-)
> 
> diff --git a/documentation/dev-manual/common-tasks.rst b/documentation/dev-manual/common-tasks.rst
> index d435bc8a4c..ffa85395a9 100644
> --- a/documentation/dev-manual/common-tasks.rst
> +++ b/documentation/dev-manual/common-tasks.rst
> @@ -11502,8 +11502,8 @@ the license from the fetched source::
>  Checking for Vulnerabilities
>  ============================
>  
> -Vulnerabilities in images
> --------------------------
> +Vulnerabilities in Poky and OE-Core
> +-----------------------------------
>  
>  The Yocto Project has an infrastructure to track and address unfixed
>  known security vulnerabilities, as tracked by the public
> @@ -11516,14 +11516,78 @@ for packages in Poky and OE-Core, tracking the evolution of the number of
>  unpatched CVEs and the status of patches. Such information is available for
>  the current development version and for each supported release.
>  
> -To know which packages are vulnerable to known security vulnerabilities
> -in the specific image you are building, add the following setting to your
> -configuration::
> +Security is a process, not a product, and thus at any time, a number of security
> +issues may be impacting Poky and OE-Core. It is up to the maintainers, users,
> +contributors and anyone interested in the issues to investigate and possibly fix them by
> +updating software components to newer versions or by applying patches to address them.
> +It is recommended to work with Poky and OE-Core upstream maintainers and submit
> +patches to fix them, see ":ref:`dev-manual/common-tasks:submitting a change to the yocto project`" for details.
> +
> +Vulnerability check at build time
> +---------------------------------
> +
> +To enable a check for CVE security vulnerabilities using :ref:`cve-check <ref-classes-cve-check>` in the specific image
> +or target you are building, add the following setting to your configuration::
>  
>     INHERIT += "cve-check"
>  
> -This way, at build time, BitBake will warn you about known CVEs
> -as in the example below::
> +The CVE database contains some old incomplete entries which have been
> +deemed not to impact Poky or OE-Core. These CVE entries can be excluded from the
> +check using build configuration::
> +
> +   include conf/distro/include/cve-extra-exclusions.inc
> +
> +With this CVE check enabled, BitBake build will try to map each compiled software component
> +recipe name and version information to the CVE database and generate recipe and
> +image specific reports. These reports will contain:
> +
> + - meta data about the software component like names and versions
> +
> + - metadata about the CVE issue such as description and NVD link

"meta data" vs. "metadata". Maybe you can decide?

> +
> + - for each software component, a list of CVEs which are possibly impacting this version
> +
> + - status of each CVE: ``Patched``, ``Unpatched`` or ``Ignored``
> +
> +The status ``Patched`` means that a patch file to address the security issue has been
> +applied. ``Unpatched`` status means that no patches to address the issue have been
> +applied and that the issue needs to be investigated. ``Ignored`` means that after
> +analysis, it has been deemed to ignore the issue as it for example affects
> +the software component on a different operating system platform.
> +
> +After build with CVE check enabled, reports for each compiled source recipe will be
> +found in ``build/tmp/deploy/cve``.
> +
> +For example the CVE check report for the ``flex-native`` recipe looks like::
> +
> +   $ cat poky/build/tmp/deploy/cve/flex-native
> +   LAYER: meta
> +   PACKAGE NAME: flex-native
> +   PACKAGE VERSION: 2.6.4
> +   CVE: CVE-2016-6354
> +   CVE STATUS: Patched
> +   CVE SUMMARY: Heap-based buffer overflow in the yy_get_next_buffer function in Flex before 2.6.1 might allow context-dependent attackers to cause a denial of service or possibly execute arbitrary code via vectors involving num_to_read.
> +   CVSS v2 BASE SCORE: 7.5
> +   CVSS v3 BASE SCORE: 9.8
> +   VECTOR: NETWORK
> +   MORE INFORMATION: https://nvd.nist.gov/vuln/detail/CVE-2016-6354
> +
> +   LAYER: meta
> +   PACKAGE NAME: flex-native
> +   PACKAGE VERSION: 2.6.4
> +   CVE: CVE-2019-6293
> +   CVE STATUS: Ignored
> +   CVE SUMMARY: An issue was discovered in the function mark_beginning_as_normal in nfa.c in flex 2.6.4. There is a stack exhaustion problem caused by the mark_beginning_as_normal function making recursive calls to itself in certain scenarios involving lots of '*' characters. Remote attackers could leverage this vulnerability to cause a denial-of-service.
> +   CVSS v2 BASE SCORE: 4.3
> +   CVSS v3 BASE SCORE: 5.5
> +   VECTOR: NETWORK
> +   MORE INFORMATION: https://nvd.nist.gov/vuln/detail/CVE-2019-6293
> +
> +For images, a summary of all recipes included in the image and their CVEs is also
> +generated in textual and JSON formats. These ``.cve`` and ``.json`` reports can be found
> +in the ``tmp/deploy/images`` directory for each compiled image.
> +
> +At build time CVE check will also throw warnings about ``Unpatched`` CVEs::
>  
>     WARNING: flex-2.6.4-r0 do_cve_check: Found unpatched CVE (CVE-2019-6293), for more information check /poky/build/tmp/work/core2-64-poky-linux/flex/2.6.4-r0/temp/cve.log
>     WARNING: libarchive-3.5.1-r0 do_cve_check: Found unpatched CVE (CVE-2021-36976), for more information check /poky/build/tmp/work/core2-64-poky-linux/libarchive/3.5.1-r0/temp/cve.log
> @@ -11532,21 +11596,46 @@ It is also possible to check the CVE status of individual packages as follows::
>  
>     bitbake -c cve_check flex libarchive
>  
> -Note that OpenEmbedded-Core keeps a list of known unfixed CVE issues which can
> -be ignored. You can pass this list to the check as follows::
> +Fixing CVE product name and version mappings
> +--------------------------------------------
>  
> -   bitbake -c cve_check libarchive -R conf/distro/include/cve-extra-exclusions.inc
> +By default, :ref:`cve-check <ref-classes-cve-check>` uses the recipe name :term:`BPN` as CVE
> +product name when querying the CVE database. If this mapping contains false positives, e.g.
> +some reported CVEs are not for the software component in question, or false negatives like
> +some CVEs are not found to impact the recipe when they should, then the problems can be
> +in the recipe name to CVE product mapping. These mapping issues can be fixed by setting
> +the :term:`CVE_PRODUCT` variable inside the recipe. This defines the name of software component in the
> +upstream `NIST CVE database <https://nvd.nist.gov/>`__.
>  
> -Enabling vulnerabily tracking in recipes
> -----------------------------------------
> +The variable supports using vendor and product names like this::
>  
> -The :term:`CVE_PRODUCT` variable defines the name used to match the recipe name
> -against the name in the upstream `NIST CVE database <https://nvd.nist.gov/>`__.
> +   CVE_PRODUCT = "flex_project:flex"
>  
> -Editing recipes to fix vulnerabilities
> ---------------------------------------
> +In this example from the vendor name used in CVE database is ``flex_project`` and
> +product is ``flex``. With this setting the ``flex`` recipe only maps to this specific
> +product and not products from other vendors with same name ``flex``.
> +
> +Similary, when the recipe version :term:`PV` is not compatible with software versions used by
> +the upstream software component releases and the CVE database, these can be fixed using
> +:term:`CVE_VERSION` variable.
> +
> +Note that if the CVE entries in NVD databse contain bugs or have missing or incomplete
> +information, it is recommended to fix the information there directly instead of working
> +around the issues for a possibly long time in Poky and OE-Core side recipes. Feedback to
> +NVD about CVEs entries can be provided through the `NVD contact form <https://nvd.nist.gov/info/contact-form>`__.
> +
> +Fixing vulnerabilities in recipes
> +---------------------------------
> +
> +If a CVE security issue impacts a software component, it can be fixed by updating to a newer
> +version of the software component or by applying a patch. For Poky and OE-Core master branches, updating
> +to newer software component release with fixes is the best option, but patches can be applied
> +if releases are not yet available.
> +
> +For stable branches, it is preferred to apply patches for the issues. For some software
> +components minor version updates can also applied if they are backwards compatible.
>  
> -To fix a given known vulnerability, you need to add a patch file to your recipe. Here's
> +Here is an example of fixing CVE security issues with patch files,
>  an example from the :oe_layerindex:`ffmpeg recipe</layerindex/recipe/47350>`::
>  
>     SRC_URI = "https://www.ffmpeg.org/releases/${BP}.tar.xz \
> @@ -11558,31 +11647,21 @@ an example from the :oe_layerindex:`ffmpeg recipe</layerindex/recipe/47350>`::
>                file://fix-CVE-2020-22033-CVE-2020-22019.patch \
>                file://fix-CVE-2021-33815.patch \
>  
> -The :ref:`cve-check <ref-classes-cve-check>` class defines two ways of
> -supplying a patch for a given CVE. The first
> -way is to use a patch filename that matches the below pattern::
> +A good practice is to include the CVE identifier in both patch file name
> +and inside the patch file commit message use the format::
>  
> -   cve_file_name_match = re.compile(".*([Cc][Vv][Ee]\-\d{4}\-\d+)")
> +   CVE: CVE-2020-22033
>  
> -As shown in the example above, multiple CVE IDs can appear in a patch filename,
> -but the :ref:`cve-check <ref-classes-cve-check>` class will only consider
> -the last CVE ID in the filename as patched.
> +CVE checker will then capture this information and change the CVE status to ``Patched``
> +in the generated reports.
>  
> -The second way to recognize a patched CVE ID is when a line matching the
> -below pattern is found in any patch file provided by the recipe::
> +If analysis shows that the CVE issue does not impact the recipe due to configuration, platform,
> +version or other reasons, the CVE can be marked as ``Ignored`` using :term:`CVE_CHECK_IGNORE` variable.
> +As mentioned previously, if data in the CVE database is wrong, it is recommend to fix those
> +issues in the CVE database directly.
>  
> -  cve_match = re.compile("CVE:( CVE\-\d{4}\-\d+)+")
> -
> -This allows a single patch file to address multiple CVE IDs at the same time.
> -
> -Of course, another way to fix vulnerabilities is to upgrade to a version
> -of the package which is not impacted, typically a more recent one.
> -The NIST database knows which versions are vulnerable and which ones
> -are not.
> -
> -Last but not least, you can choose to ignore vulnerabilities through
> -the :term:`CVE_CHECK_SKIP_RECIPE` and :term:`CVE_CHECK_IGNORE`
> -variables.
> +Recipes can be completely skipped by CVE check by including the recipe name in
> +the :term:`CVE_CHECK_SKIP_RECIPE` variable.
>  
>  Implementation details
>  ----------------------
> @@ -11600,23 +11679,38 @@ Then, the code looks up all the CVE IDs in the NIST database for all the
>  products defined in :term:`CVE_PRODUCT`. Then, for each found CVE:
>  
>   - If the package name (:term:`PN`) is part of
> -   :term:`CVE_CHECK_SKIP_RECIPE`, it is considered as patched.
> +   :term:`CVE_CHECK_SKIP_RECIPE`, it is considered as ``Patched``.
>  
>   - If the CVE ID is part of :term:`CVE_CHECK_IGNORE`, it is
> -   considered as patched too.
> +   set as ``Ignored``.
>  
>   - If the CVE ID is part of the patched CVE for the recipe, it is
>     already considered as patched.

Maybe highlight the ``Patched`` here too?

>   - Otherwise, the code checks whether the recipe version (:term:`PV`)
>     is within the range of versions impacted by the CVE. If so, the CVE
> -   is considered as unpatched.
> +   is considered as ``Unpatched``.
>  
>  The CVE database is stored in :term:`DL_DIR` and can be inspected using
>  ``sqlite3`` command as follows::
>  
>     sqlite3 downloads/CVE_CHECK/nvdcve_1.1.db .dump | grep CVE-2021-37462
>  
> +When analyzing CVEs, it is recommended to:
> +
> + - study the latest information in `CVE database <https://nvd.nist.gov/vuln/search>`.

I think URL syntax is incorrect here, needs the underscores at the end.

> +
> + - check how upstream developers of the software component addressed the issue, e.g.
> +   what patch was applied, which upstream release contains the fix.
> +
> + - check what other Linux distributions like `Debian <https://security-tracker.debian.org/tracker/>`

Here too, URL syntax needs underscores.

> +   did to analyze and address the issue.
> +
> + - follow security notices from other Linux distributions.
> +
> + - follow public `open source security mailing lists <https://oss-security.openwall.org/wiki/mailing-lists>` for

And here.

Now it looks good to me. Thanks for the review and new version!

Cheers,

-Mikko

Re: [docs] [PATCH v2 4/4] dev-manual: common-tasks.rst: refactor and improve "Checking for Vulnerabilities" section

[Michael Opdenacker]

Hi Mikko

On 10/27/22 08:54, Mikko Rapeli wrote:
> Now it looks good to me. Thanks for the review and new version!


Thanks for reviewing your text once again!
I fixed all the new issues you reported.

Cheers and thanks again for these very useful improvements to the 
documentation.
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com