[poky][master][kirkstone][PATCH] cve-check.bbclass: Add anonymous function to get patched CVEs from recipe

[poky][master][kirkstone][PATCH] cve-check.bbclass: Add anonymous function to get patched CVEs from recipe

[Akash Hadke]

Add an anonymous function to get patched CVEs from the recipe
and set the value to 'CVE_PATCHED' variable
This variable later can be used to do CVE data processing
outside of bitbake

Also, introduce a new variable 'CVE_CHECK_WITH_DB' default set
to '0', when it is set to non zero value it avoids CVE scan for
unpatched CVEs from NVD DB.
It will work as the second operational mode for cve-check.bbclass
which only exports the data.

Signed-off-by: Akash Hadke <akash.hadke@kpit.com>
---
 meta/classes/cve-check.bbclass | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index da7f93371c..b7f7ca73e5 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -82,6 +82,12 @@ CVE_CHECK_LAYER_INCLUDELIST ??= ""
 # set to "alphabetical" for version using single alphabetical character as increment release
 CVE_VERSION_SUFFIX ??= ""
 
+# set to "1" for avoiding full scan for unpatched CVEs
+CVE_CHECK_WITH_DB ??= "0"
+
+# Patched CVEs from recipe will be assigned to this variable
+CVE_PATCHED ??= ""
+
 def generate_json_report(d, out_path, link_path):
     if os.path.exists(d.getVar("CVE_CHECK_SUMMARY_INDEX_PATH")):
         import json
@@ -133,13 +139,18 @@ python cve_save_summary_handler () {
 addhandler cve_save_summary_handler
 cve_save_summary_handler[eventmask] = "bb.event.BuildCompleted"
 
+python() {
+    from oe.cve_check import get_patched_cves
+    d.setVar('CVE_PATCHED', " ".join(get_patched_cves(d)))
+}
+
 python do_cve_check () {
     """
     Check recipe for patched and unpatched CVEs
     """
     from oe.cve_check import get_patched_cves
 
-    if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")):
+    if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")) and d.getVar("CVE_CHECK_WITH_DB") == "0":
         try:
             patched_cves = get_patched_cves(d)
         except FileNotFoundError:
@@ -154,7 +165,7 @@ python do_cve_check () {
 }
 
 addtask cve_check before do_build
-do_cve_check[depends] = "cve-update-db-native:do_fetch"
+do_cve_check[depends] = "${@'cve-update-db-native:do_fetch' if d.getVar('CVE_CHECK_WITH_DB') == '0' else ''}"
 do_cve_check[nostamp] = "1"
 
 python cve_check_cleanup () {
-- 
2.17.1

Re: [OE-core] [poky][master][kirkstone][PATCH] cve-check.bbclass: Add anonymous function to get patched CVEs from recipe

[Ross Burton]

On 12 Jul 2022, at 11:27, akash hadke via lists.openembedded.org <akash.hadke=kpit.com@lists.openembedded.org> wrote:
> 
> Add an anonymous function to get patched CVEs from the recipe
> and set the value to 'CVE_PATCHED' variable
> This variable later can be used to do CVE data processing
> outside of bitbake
> 
> Also, introduce a new variable 'CVE_CHECK_WITH_DB' default set
> to '0', when it is set to non zero value it avoids CVE scan for
> unpatched CVEs from NVD DB.
> It will work as the second operational mode for cve-check.bbclass
> which only exports the data.

This is two changes, so please split into two commits.

I’m curious what problem the CVE_CHECK_WITH_DB variable is solving, can you explain the use-case?>

Ross

Re: [poky][master][kirkstone][PATCH] cve-check.bbclass: Add anonymous function to get patched CVEs from recipe

[akash hadke]

[-- Attachment #1: Type: text/plain, Size: 297 bytes --]

Hello Ross,
We can inherit cve-check by default in the configuration to export CVE information like CVE_PRODUCT, CVE_VERSION and the newly added variable CVE_PATCHED. And CVE_CHECK_WITH_DB variable will avoid scanning the CVEs from DB
It will work like 2 operational mode for cve-check.bbclass

[-- Attachment #2: Type: text/html, Size: 398 bytes --]

Re: [OE-core] [poky][master][kirkstone][PATCH] cve-check.bbclass: Add anonymous function to get patched CVEs from recipe

[Ross Burton]

On 12 Jul 2022, at 12:21, akash hadke via lists.openembedded.org <akash.hadke=kpit.com@lists.openembedded.org> wrote:
> We can inherit cve-check by default in the configuration to export CVE information like CVE_PRODUCT, CVE_VERSION and the newly added variable CVE_PATCHED. And CVE_CHECK_WITH_DB variable will avoid scanning the CVEs from DB
> It will work like 2 operational mode for cve-check.bbclass 

Looking more closely, that’s going to be a pretty horrible performance hit because you’re scanning the entire sources tree every time the recipe is parsed.

If you have CVE_CHECK_WITH_DB, why do you need this variable at all?

Ross

Re: [poky][master][kirkstone][PATCH] cve-check.bbclass: Add anonymous function to get patched CVEs from recipe

[akash hadke]

[-- Attachment #1: Type: text/plain, Size: 280 bytes --]

Actually, I wanted to get all patched CVE numbers from a recipe in one variable, and the value of that variable will be used in other tasks. And for this purpose, I used the anonymous function because without it I will not be able to get the value of CVE_PATCHED in other tasks.

[-- Attachment #2: Type: text/html, Size: 280 bytes --]

Re: [OE-core] [poky][master][kirkstone][PATCH] cve-check.bbclass: Add anonymous function to get patched CVEs from recipe

[Marta Rybczynska]

On Tue, Jul 12, 2022 at 12:28 PM akash hadke via
lists.openembedded.org <akash.hadke=kpit.com@lists.openembedded.org>
wrote:
>
> Add an anonymous function to get patched CVEs from the recipe
> and set the value to 'CVE_PATCHED' variable
> This variable later can be used to do CVE data processing
> outside of bitbake
>
> Also, introduce a new variable 'CVE_CHECK_WITH_DB' default set
> to '0', when it is set to non zero value it avoids CVE scan for
> unpatched CVEs from NVD DB.
> It will work as the second operational mode for cve-check.bbclass
> which only exports the data.
>
> Signed-off-by: Akash Hadke <akash.hadke@kpit.com>
> ---
>  meta/classes/cve-check.bbclass | 15 +++++++++++++--
>  1 file changed, 13 insertions(+), 2 deletions(-)
>
> diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
> index da7f93371c..b7f7ca73e5 100644
> --- a/meta/classes/cve-check.bbclass
> +++ b/meta/classes/cve-check.bbclass
> @@ -82,6 +82,12 @@ CVE_CHECK_LAYER_INCLUDELIST ??= ""
>  # set to "alphabetical" for version using single alphabetical character as increment release
>  CVE_VERSION_SUFFIX ??= ""
>
> +# set to "1" for avoiding full scan for unpatched CVEs
> +CVE_CHECK_WITH_DB ??= "0"

The default behavior is now to check with the database, so this should be at "1"
by default.

> +
> +# Patched CVEs from recipe will be assigned to this variable
> +CVE_PATCHED ??= ""
> +
>  def generate_json_report(d, out_path, link_path):
>      if os.path.exists(d.getVar("CVE_CHECK_SUMMARY_INDEX_PATH")):
>          import json
> @@ -133,13 +139,18 @@ python cve_save_summary_handler () {
>  addhandler cve_save_summary_handler
>  cve_save_summary_handler[eventmask] = "bb.event.BuildCompleted"
>
> +python() {
> +    from oe.cve_check import get_patched_cves
> +    d.setVar('CVE_PATCHED', " ".join(get_patched_cves(d)))
> +}
> +
>  python do_cve_check () {
>      """
>      Check recipe for patched and unpatched CVEs
>      """
>      from oe.cve_check import get_patched_cves
>
> -    if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")):
> +    if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")) and d.getVar("CVE_CHECK_WITH_DB") == "0":
>          try:
>              patched_cves = get_patched_cves(d)
>          except FileNotFoundError:

Instead of the anonymous function, you could add a condition here.


Regards,
Marta

Re: [poky][master][kirkstone][PATCH] cve-check.bbclass: Add anonymous function to get patched CVEs from recipe

[akash hadke]

[-- Attachment #1: Type: text/plain, Size: 2913 bytes --]

On Wed, Jul 20, 2022 at 12:19 AM, Marta Rybczynska wrote:

> 
> On Tue, Jul 12, 2022 at 12:28 PM akash hadke via
> lists.openembedded.org <akash.hadke=kpit.com@lists.openembedded.org>
> wrote:
> 
>> Add an anonymous function to get patched CVEs from the recipe
>> and set the value to 'CVE_PATCHED' variable
>> This variable later can be used to do CVE data processing
>> outside of bitbake
>> 
>> Also, introduce a new variable 'CVE_CHECK_WITH_DB' default set
>> to '0', when it is set to non zero value it avoids CVE scan for
>> unpatched CVEs from NVD DB.
>> It will work as the second operational mode for cve-check.bbclass
>> which only exports the data.
>> 
>> Signed-off-by: Akash Hadke <akash.hadke@kpit.com>
>> ---
>> meta/classes/cve-check.bbclass | 15 +++++++++++++--
>> 1 file changed, 13 insertions(+), 2 deletions(-)
>> 
>> diff --git a/meta/classes/cve-check.bbclass
>> b/meta/classes/cve-check.bbclass
>> index da7f93371c..b7f7ca73e5 100644
>> --- a/meta/classes/cve-check.bbclass
>> +++ b/meta/classes/cve-check.bbclass
>> @@ -82,6 +82,12 @@ CVE_CHECK_LAYER_INCLUDELIST ??= ""
>> # set to "alphabetical" for version using single alphabetical character as
>> increment release
>> CVE_VERSION_SUFFIX ??= ""
>> 
>> +# set to "1" for avoiding full scan for unpatched CVEs
>> +CVE_CHECK_WITH_DB ??= "0"
> 
> The default behavior is now to check with the database, so this should be
> at "1"
> by default.

Ok, I will update it once all discussion is completed.

> 
> 
>> +
>> +# Patched CVEs from recipe will be assigned to this variable
>> +CVE_PATCHED ??= ""
>> +
>> def generate_json_report(d, out_path, link_path):
>> if os.path.exists(d.getVar("CVE_CHECK_SUMMARY_INDEX_PATH")):
>> import json
>> @@ -133,13 +139,18 @@ python cve_save_summary_handler () {
>> addhandler cve_save_summary_handler
>> cve_save_summary_handler[eventmask] = "bb.event.BuildCompleted"
>> 
>> +python() {
>> + from oe.cve_check import get_patched_cves
>> + d.setVar('CVE_PATCHED', " ".join(get_patched_cves(d)))
>> +}
>> +
>> python do_cve_check () {
>> """
>> Check recipe for patched and unpatched CVEs
>> """
>> from oe.cve_check import get_patched_cves
>> 
>> - if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")):
>> + if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")) and
>> d.getVar("CVE_CHECK_WITH_DB") == "0":
>> try:
>> patched_cves = get_patched_cves(d)
>> except FileNotFoundError:
> 
> Instead of the anonymous function, you could add a condition here.

If I use a condition instead of an anonymous function, I will not be able to get the value of the CVE_PATCHED variable in other tasks. The value will be accessed only in the cve_check task. Hence I used the anonymous function.
As per my understanding, this is the only way, please let me know if there is any other way to achieve this.

> 
> 
> Regards,
> Marta

BR,
Akash

[-- Attachment #2: Type: text/html, Size: 3088 bytes --]

Re: [OE-core] [poky][master][kirkstone][PATCH] cve-check.bbclass: Add anonymous function to get patched CVEs from recipe

[Ross Burton]

> On 19 Jul 2022, at 08:54, akash hadke via lists.openembedded.org <akash.hadke=kpit.com@lists.openembedded.org> wrote:
> 
> Actually, I wanted to get all patched CVE numbers from a recipe in one variable, and the value of that variable will be used in other tasks. And for this purpose, I used the anonymous function because without it I will not be able to get the value of CVE_PATCHED in other tasks. 

I’ll be blunt: we’re not adding a scan of every file to every recipe in every build to this class.

Why not generate this data on-demand in the task you’re writing to use it?

Ross

Re: [poky][master][kirkstone][PATCH] cve-check.bbclass: Add anonymous function to get patched CVEs from recipe

[akash hadke]

[-- Attachment #1: Type: text/plain, Size: 871 bytes --]

On Wed, Jul 20, 2022 at 05:11 PM, Ross Burton wrote:

> 
> 
>> On 19 Jul 2022, at 08:54, akash hadke via lists.openembedded.org
>> <akash.hadke=kpit.com@lists.openembedded.org> wrote:
>> 
>> Actually, I wanted to get all patched CVE numbers from a recipe in one
>> variable, and the value of that variable will be used in other tasks. And
>> for this purpose, I used the anonymous function because without it I will
>> not be able to get the value of CVE_PATCHED in other tasks.
> 
> I’ll be blunt: we’re not adding a scan of every file to every recipe in
> every build to this class.
> 
> Why not generate this data on-demand in the task you’re writing to use it?

Hello Ross,

I am trying to add this data in buildhistory. And getting required data in variables will help me like other variables but I don't know another way to set the value.

[-- Attachment #2: Type: text/html, Size: 921 bytes --]