All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] cve-check: Fix report generation
@ 2022-05-17  6:01 Marta Rybczynska
  2022-05-17  7:55 ` [OE-core] " Ernst Sjöstrand
  2022-05-18 15:33 ` Marta Rybczynska
  0 siblings, 2 replies; 5+ messages in thread
From: Marta Rybczynska @ 2022-05-17  6:01 UTC (permalink / raw)
  To: openembedded-core; +Cc: alex.kiernan, Marta Rybczynska, Marta Rybczynska

The addition of summary output caused two issues: error when building
an image and the fact that JSON output was generated even when
CVE_CHECK_FORMAT_JSON.

When generating an image it caused an error like:
ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_func_python() autogenerated:

  The stack trace of python calls that resulted in this exception/failure was:
  File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
       0001:
   *** 0002:cve_check_write_rootfs_manifest(d)
       0003:
  File: '/home/alexk/poky/meta/classes/cve-check.bbclass', lineno: 213, function: cve_check_write_rootfs_manifest
       0209:
       0210:        link_path = os.path.join(deploy_dir, "%s.json" % link_name)
       0211:        manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
       0212:        bb.note("Generating JSON CVE manifest")
   *** 0213:        generate_json_report(json_summary_name, json_summary_link_name)
       0214:        bb.plain("Image CVE JSON report stored in: %s" % link_path)
       0215:}
       0216:
       0217:ROOTFS_POSTPROCESS_COMMAND:prepend = "${@'cve_check_write_rootfs_manifest; ' if d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}"
  Exception: NameError: name 'json_summary_name' is not defined

The fix is to pass the d variable to the pure python function generate_json_report
to get correct values of variables and add conditions for the JSON
output where needed.

In addition clarify the message presenting the summary JSON file,
which isn't related to an image.

Uses partial fixes from Alex Kiernan, Ernst Sjöstrand (ernstp),
and Davide Gardenal.

Fixes: f2987891d315 ("cve-check: add JSON format to summary output")

Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
---
 meta/classes/cve-check.bbclass | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index 24ddb865ea..7cd98ae462 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -79,7 +79,7 @@ CVE_CHECK_LAYER_INCLUDELIST ??= ""
 # set to "alphabetical" for version using single alphabetical character as increment release
 CVE_VERSION_SUFFIX ??= ""
 
-def generate_json_report(out_path, link_path):
+def generate_json_report(d, out_path, link_path):
     if os.path.exists(d.getVar("CVE_CHECK_SUMMARY_INDEX_PATH")):
         import json
         from oe.cve_check import cve_check_merge_jsons
@@ -127,10 +127,11 @@ python cve_save_summary_handler () {
                     os.remove(cvefile_link)
                 os.symlink(os.path.basename(cve_summary_file), cvefile_link)
 
+    if d.getVar("CVE_CHECK_FORMAT_JSON") == "1":
         json_summary_link_name = os.path.join(cvelogpath, d.getVar("CVE_CHECK_SUMMARY_FILE_NAME_JSON"))
         json_summary_name = os.path.join(cvelogpath, "%s-%s.json" % (cve_summary_name, timestamp))
-        generate_json_report(json_summary_name, json_summary_link_name)
-        bb.plain("CVE report summary created at: %s" % json_summary_link_name)
+        generate_json_report(d, json_summary_name, json_summary_link_name)
+        bb.plain("Complete CVE JSON report summary created at: %s" % json_summary_link_name)
 }
 
 addhandler cve_save_summary_handler
@@ -207,11 +208,12 @@ python cve_check_write_rootfs_manifest () {
                 os.symlink(os.path.basename(manifest_name), manifest_link)
             bb.plain("Image CVE report stored in: %s" % manifest_name)
 
-        link_path = os.path.join(deploy_dir, "%s.json" % link_name)
-        manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
-        bb.note("Generating JSON CVE manifest")
-        generate_json_report(json_summary_name, json_summary_link_name)
-        bb.plain("Image CVE JSON report stored in: %s" % link_path)
+        if d.getVar("CVE_CHECK_FORMAT_JSON") == "1":
+            link_path = os.path.join(deploy_dir, "%s.json" % link_name)
+            manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
+            bb.note("Generating JSON CVE manifest")
+            generate_json_report(d, manifest_path, link_path)
+            bb.plain("Image CVE JSON report stored in: %s" % link_path)
 }
 
 ROOTFS_POSTPROCESS_COMMAND:prepend = "${@'cve_check_write_rootfs_manifest; ' if d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}"
-- 
2.33.0


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* Re: [OE-core] [PATCH] cve-check: Fix report generation
  2022-05-17  6:01 [PATCH] cve-check: Fix report generation Marta Rybczynska
@ 2022-05-17  7:55 ` Ernst Sjöstrand
  2022-05-17  8:26   ` Alex Kiernan
  2022-05-18 15:33 ` Marta Rybczynska
  1 sibling, 1 reply; 5+ messages in thread
From: Ernst Sjöstrand @ 2022-05-17  7:55 UTC (permalink / raw)
  To: Marta Rybczynska; +Cc: openembedded-core, alex.kiernan, Marta Rybczynska

[-- Attachment #1: Type: text/plain, Size: 5254 bytes --]

Reviewed-by: Ernst Sjöstrand <ernstp@gmail.com>

Den tis 17 maj 2022 kl 08:01 skrev Marta Rybczynska <rybczynska@gmail.com>:

> The addition of summary output caused two issues: error when building
> an image and the fact that JSON output was generated even when
> CVE_CHECK_FORMAT_JSON.
>
> When generating an image it caused an error like:
> ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python
> function in exec_func_python() autogenerated:
>
>   The stack trace of python calls that resulted in this exception/failure
> was:
>   File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
>        0001:
>    *** 0002:cve_check_write_rootfs_manifest(d)
>        0003:
>   File: '/home/alexk/poky/meta/classes/cve-check.bbclass', lineno: 213,
> function: cve_check_write_rootfs_manifest
>        0209:
>        0210:        link_path = os.path.join(deploy_dir, "%s.json" %
> link_name)
>        0211:        manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
>        0212:        bb.note("Generating JSON CVE manifest")
>    *** 0213:        generate_json_report(json_summary_name,
> json_summary_link_name)
>        0214:        bb.plain("Image CVE JSON report stored in: %s" %
> link_path)
>        0215:}
>        0216:
>        0217:ROOTFS_POSTPROCESS_COMMAND:prepend =
> "${@'cve_check_write_rootfs_manifest; ' if
> d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}"
>   Exception: NameError: name 'json_summary_name' is not defined
>
> The fix is to pass the d variable to the pure python function
> generate_json_report
> to get correct values of variables and add conditions for the JSON
> output where needed.
>
> In addition clarify the message presenting the summary JSON file,
> which isn't related to an image.
>
> Uses partial fixes from Alex Kiernan, Ernst Sjöstrand (ernstp),
> and Davide Gardenal.
>
> Fixes: f2987891d315 ("cve-check: add JSON format to summary output")
>
> Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
> ---
>  meta/classes/cve-check.bbclass | 18 ++++++++++--------
>  1 file changed, 10 insertions(+), 8 deletions(-)
>
> diff --git a/meta/classes/cve-check.bbclass
> b/meta/classes/cve-check.bbclass
> index 24ddb865ea..7cd98ae462 100644
> --- a/meta/classes/cve-check.bbclass
> +++ b/meta/classes/cve-check.bbclass
> @@ -79,7 +79,7 @@ CVE_CHECK_LAYER_INCLUDELIST ??= ""
>  # set to "alphabetical" for version using single alphabetical character
> as increment release
>  CVE_VERSION_SUFFIX ??= ""
>
> -def generate_json_report(out_path, link_path):
> +def generate_json_report(d, out_path, link_path):
>      if os.path.exists(d.getVar("CVE_CHECK_SUMMARY_INDEX_PATH")):
>          import json
>          from oe.cve_check import cve_check_merge_jsons
> @@ -127,10 +127,11 @@ python cve_save_summary_handler () {
>                      os.remove(cvefile_link)
>                  os.symlink(os.path.basename(cve_summary_file),
> cvefile_link)
>
> +    if d.getVar("CVE_CHECK_FORMAT_JSON") == "1":
>          json_summary_link_name = os.path.join(cvelogpath,
> d.getVar("CVE_CHECK_SUMMARY_FILE_NAME_JSON"))
>          json_summary_name = os.path.join(cvelogpath, "%s-%s.json" %
> (cve_summary_name, timestamp))
> -        generate_json_report(json_summary_name, json_summary_link_name)
> -        bb.plain("CVE report summary created at: %s" %
> json_summary_link_name)
> +        generate_json_report(d, json_summary_name, json_summary_link_name)
> +        bb.plain("Complete CVE JSON report summary created at: %s" %
> json_summary_link_name)
>  }
>
>  addhandler cve_save_summary_handler
> @@ -207,11 +208,12 @@ python cve_check_write_rootfs_manifest () {
>                  os.symlink(os.path.basename(manifest_name), manifest_link)
>              bb.plain("Image CVE report stored in: %s" % manifest_name)
>
> -        link_path = os.path.join(deploy_dir, "%s.json" % link_name)
> -        manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
> -        bb.note("Generating JSON CVE manifest")
> -        generate_json_report(json_summary_name, json_summary_link_name)
> -        bb.plain("Image CVE JSON report stored in: %s" % link_path)
> +        if d.getVar("CVE_CHECK_FORMAT_JSON") == "1":
> +            link_path = os.path.join(deploy_dir, "%s.json" % link_name)
> +            manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
> +            bb.note("Generating JSON CVE manifest")
> +            generate_json_report(d, manifest_path, link_path)
> +            bb.plain("Image CVE JSON report stored in: %s" % link_path)
>  }
>
>  ROOTFS_POSTPROCESS_COMMAND:prepend =
> "${@'cve_check_write_rootfs_manifest; ' if
> d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}"
> --
> 2.33.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#165707):
> https://lists.openembedded.org/g/openembedded-core/message/165707
> Mute This Topic: https://lists.openembedded.org/mt/91158052/4947266
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> ernstp@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 6974 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [OE-core] [PATCH] cve-check: Fix report generation
  2022-05-17  7:55 ` [OE-core] " Ernst Sjöstrand
@ 2022-05-17  8:26   ` Alex Kiernan
  0 siblings, 0 replies; 5+ messages in thread
From: Alex Kiernan @ 2022-05-17  8:26 UTC (permalink / raw)
  To: Ernst Sjöstrand
  Cc: Marta Rybczynska,
	Patches and discussions about the oe-core layer,
	Marta Rybczynska

Tested-by: Alex Kiernan <alex.kiernan@gmail.com>

On Tue, May 17, 2022 at 8:55 AM Ernst Sjöstrand <ernstp@gmail.com> wrote:
>
> Reviewed-by: Ernst Sjöstrand <ernstp@gmail.com>
>
> Den tis 17 maj 2022 kl 08:01 skrev Marta Rybczynska <rybczynska@gmail.com>:
>>
>> The addition of summary output caused two issues: error when building
>> an image and the fact that JSON output was generated even when
>> CVE_CHECK_FORMAT_JSON.
>>
>> When generating an image it caused an error like:
>> ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_func_python() autogenerated:
>>
>>   The stack trace of python calls that resulted in this exception/failure was:
>>   File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
>>        0001:
>>    *** 0002:cve_check_write_rootfs_manifest(d)
>>        0003:
>>   File: '/home/alexk/poky/meta/classes/cve-check.bbclass', lineno: 213, function: cve_check_write_rootfs_manifest
>>        0209:
>>        0210:        link_path = os.path.join(deploy_dir, "%s.json" % link_name)
>>        0211:        manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
>>        0212:        bb.note("Generating JSON CVE manifest")
>>    *** 0213:        generate_json_report(json_summary_name, json_summary_link_name)
>>        0214:        bb.plain("Image CVE JSON report stored in: %s" % link_path)
>>        0215:}
>>        0216:
>>        0217:ROOTFS_POSTPROCESS_COMMAND:prepend = "${@'cve_check_write_rootfs_manifest; ' if d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}"
>>   Exception: NameError: name 'json_summary_name' is not defined
>>
>> The fix is to pass the d variable to the pure python function generate_json_report
>> to get correct values of variables and add conditions for the JSON
>> output where needed.
>>
>> In addition clarify the message presenting the summary JSON file,
>> which isn't related to an image.
>>
>> Uses partial fixes from Alex Kiernan, Ernst Sjöstrand (ernstp),
>> and Davide Gardenal.
>>
>> Fixes: f2987891d315 ("cve-check: add JSON format to summary output")
>>
>> Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
>> ---
>>  meta/classes/cve-check.bbclass | 18 ++++++++++--------
>>  1 file changed, 10 insertions(+), 8 deletions(-)
>>
>> diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
>> index 24ddb865ea..7cd98ae462 100644
>> --- a/meta/classes/cve-check.bbclass
>> +++ b/meta/classes/cve-check.bbclass
>> @@ -79,7 +79,7 @@ CVE_CHECK_LAYER_INCLUDELIST ??= ""
>>  # set to "alphabetical" for version using single alphabetical character as increment release
>>  CVE_VERSION_SUFFIX ??= ""
>>
>> -def generate_json_report(out_path, link_path):
>> +def generate_json_report(d, out_path, link_path):
>>      if os.path.exists(d.getVar("CVE_CHECK_SUMMARY_INDEX_PATH")):
>>          import json
>>          from oe.cve_check import cve_check_merge_jsons
>> @@ -127,10 +127,11 @@ python cve_save_summary_handler () {
>>                      os.remove(cvefile_link)
>>                  os.symlink(os.path.basename(cve_summary_file), cvefile_link)
>>
>> +    if d.getVar("CVE_CHECK_FORMAT_JSON") == "1":
>>          json_summary_link_name = os.path.join(cvelogpath, d.getVar("CVE_CHECK_SUMMARY_FILE_NAME_JSON"))
>>          json_summary_name = os.path.join(cvelogpath, "%s-%s.json" % (cve_summary_name, timestamp))
>> -        generate_json_report(json_summary_name, json_summary_link_name)
>> -        bb.plain("CVE report summary created at: %s" % json_summary_link_name)
>> +        generate_json_report(d, json_summary_name, json_summary_link_name)
>> +        bb.plain("Complete CVE JSON report summary created at: %s" % json_summary_link_name)
>>  }
>>
>>  addhandler cve_save_summary_handler
>> @@ -207,11 +208,12 @@ python cve_check_write_rootfs_manifest () {
>>                  os.symlink(os.path.basename(manifest_name), manifest_link)
>>              bb.plain("Image CVE report stored in: %s" % manifest_name)
>>
>> -        link_path = os.path.join(deploy_dir, "%s.json" % link_name)
>> -        manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
>> -        bb.note("Generating JSON CVE manifest")
>> -        generate_json_report(json_summary_name, json_summary_link_name)
>> -        bb.plain("Image CVE JSON report stored in: %s" % link_path)
>> +        if d.getVar("CVE_CHECK_FORMAT_JSON") == "1":
>> +            link_path = os.path.join(deploy_dir, "%s.json" % link_name)
>> +            manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
>> +            bb.note("Generating JSON CVE manifest")
>> +            generate_json_report(d, manifest_path, link_path)
>> +            bb.plain("Image CVE JSON report stored in: %s" % link_path)
>>  }
>>
>>  ROOTFS_POSTPROCESS_COMMAND:prepend = "${@'cve_check_write_rootfs_manifest; ' if d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}"
>> --
>> 2.33.0
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#165707): https://lists.openembedded.org/g/openembedded-core/message/165707
>> Mute This Topic: https://lists.openembedded.org/mt/91158052/4947266
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ernstp@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>


-- 
Alex Kiernan


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH] cve-check: Fix report generation
  2022-05-17  6:01 [PATCH] cve-check: Fix report generation Marta Rybczynska
  2022-05-17  7:55 ` [OE-core] " Ernst Sjöstrand
@ 2022-05-18 15:33 ` Marta Rybczynska
  2022-05-18 15:43   ` Steve Sakoman
  1 sibling, 1 reply; 5+ messages in thread
From: Marta Rybczynska @ 2022-05-18 15:33 UTC (permalink / raw)
  To: OE-core, Steve Sakoman; +Cc: Alex Kiernan, Marta Rybczynska

On Tue, May 17, 2022 at 8:01 AM Marta Rybczynska <rybczynska@gmail.com> wrote:
>
> The addition of summary output caused two issues: error when building
> an image and the fact that JSON output was generated even when
> CVE_CHECK_FORMAT_JSON.
>
> When generating an image it caused an error like:
> ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_func_python() autogenerated:

Hello Steve,
Would you pick this one for dunfell too? The bug affects dunfell too
;( Or do you prefer me to do a resend
a well-applying patch?

Kind regards,
Marta


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH] cve-check: Fix report generation
  2022-05-18 15:33 ` Marta Rybczynska
@ 2022-05-18 15:43   ` Steve Sakoman
  0 siblings, 0 replies; 5+ messages in thread
From: Steve Sakoman @ 2022-05-18 15:43 UTC (permalink / raw)
  To: Marta Rybczynska; +Cc: OE-core, Alex Kiernan, Marta Rybczynska

On Wed, May 18, 2022 at 5:33 AM Marta Rybczynska <rybczynska@gmail.com> wrote:
>
> On Tue, May 17, 2022 at 8:01 AM Marta Rybczynska <rybczynska@gmail.com> wrote:
> >
> > The addition of summary output caused two issues: error when building
> > an image and the fact that JSON output was generated even when
> > CVE_CHECK_FORMAT_JSON.
> >
> > When generating an image it caused an error like:
> > ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_func_python() autogenerated:
>
> Hello Steve,
> Would you pick this one for dunfell too? The bug affects dunfell too
> ;( Or do you prefer me to do a resend
> a well-applying patch?

I was able to cherry-pick it with no problem, so no need to send a patch.

Thanks!

Steve


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2022-05-18 15:43 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-17  6:01 [PATCH] cve-check: Fix report generation Marta Rybczynska
2022-05-17  7:55 ` [OE-core] " Ernst Sjöstrand
2022-05-17  8:26   ` Alex Kiernan
2022-05-18 15:33 ` Marta Rybczynska
2022-05-18 15:43   ` Steve Sakoman

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.