openembedded-core.lists.openembedded.org archive mirror
 help / color / mirror / Atom feed
* Re: [OE-core][PATCH] create-spdx: ignore packing control files from ipk and deb
       [not found] <17077D238077F896.4080@lists.openembedded.org>
@ 2022-08-09 16:40 ` Jose Quaresma
  2022-08-10 14:47   ` Steve Sakoman
  0 siblings, 1 reply; 3+ messages in thread
From: Jose Quaresma @ 2022-08-09 16:40 UTC (permalink / raw)
  To: Jose Quaresma, Steve Sakoman
  Cc: OE-core, Ricardo Salveti, daiane.angolini, Jose Quaresma

[-- Attachment #1: Type: text/plain, Size: 4974 bytes --]

Hi Steve,

Can this patch be backported to kirkstone?

Jose

Jose Quaresma via lists.openembedded.org <quaresma.jose=
gmail.com@lists.openembedded.org> escreveu no dia terça, 2/08/2022 à(s)
10:29:

> Otherwise spdx can have references for data that is not packed
> in the package delivered because this contol data is temporarly
> and only exist while the package is been write.
>
> During do_package_write_ipk task in do_package_ipk the control
> files is cleaned up at the end. This can create a race condiction
> when the do_create_spdx task runs the add_package_files function
> and these files is been deleted at same time in the
> task do_package_write_ipk.
>
> ERROR: alsa-topology-conf-1.2.5.1-r0 do_create_spdx: Error executing a
> python function in exec_func_python() autogenerated:
>
> The stack trace of python calls that resulted in this exception/failure
> was:
> File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
>      0001:
>  *** 0002:do_create_spdx(d)
>      0003:
> File:
> '/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass',
> lineno: 567, function: do_create_spdx
>      0563:            package_doc.add_relationship(package_doc,
> "DESCRIBES", spdx_package)
>      0564:
>      0565:            package_archive = deploy_dir_spdx / "packages" / (
> package_doc.name + ".tar.zst")
>      0566:            with optional_tarfile(package_archive,
> archive_packaged) as archive:
>  *** 0567:                package_files = add_package_files(
>      0568:                    d,
>      0569:                    package_doc,
>      0570:                    spdx_package,
>      0571:                    pkgdest / package,
> File:
> '/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass',
> lineno: 234, function: add_package_files
>      0230:                            info.mtime = source_date_epoch
>      0231:
>      0232:                        archive.addfile(info, f)
>      0233:
>  *** 0234:                sha1 = bb.utils.sha1_file(filepath)
>      0235:                sha1s.append(sha1)
>      0236:                spdx_file.checksums.append(oe.spdx.SPDXChecksum(
>      0237:                        algorithm="SHA1",
>      0238:                        checksumValue=sha1,
> File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 559, function: sha1_file
>      0555:    """
>      0556:    Return the hex string representation of the SHA1 checksum of
> the filename
>      0557:    """
>      0558:    import hashlib
>  *** 0559:    return _hasher(hashlib.sha1(), filename)
>      0560:
>      0561:def sha384_file(filename):
>      0562:    """
>      0563:    Return the hex string representation of the SHA384 checksum
> of the filename
> File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 528, function: _hasher
>      0524:
>      0525:def _hasher(method, filename):
>      0526:    import mmap
>      0527:
>  *** 0528:    with open(filename, "rb") as f:
>      0529:        try:
>      0530:            with mmap.mmap(f.fileno(), 0,
> access=mmap.ACCESS_READ) as mm:
>      0531:                for chunk in iter(lambda: mm.read(8192), b''):
>      0532:                    method.update(chunk)
> Exception: FileNotFoundError: [Errno 2] No such file or directory:
> '/srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/packages-split/alsa-topology-conf/CONTROL/control'
>
> ERROR: Logfile of failure stored in:
> /srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/temp/log.do_create_spdx.998864
> INFO: recipe alsa-topology-conf-1.2.5.1-r0: task do_create_spdx: Failed
>
> Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
> ---
>  meta/classes/create-spdx.bbclass | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/meta/classes/create-spdx.bbclass
> b/meta/classes/create-spdx.bbclass
> index 37b6b569a1..08ef7fd409 100644
> --- a/meta/classes/create-spdx.bbclass
> +++ b/meta/classes/create-spdx.bbclass
> @@ -571,6 +571,7 @@ python do_create_spdx() {
>                      pkgdest / package,
>                      lambda file_counter:
> oe.sbom.get_packaged_file_spdxid(pkg_name, file_counter),
>                      lambda filepath: ["BINARY"],
> +                    ignore_top_level_dirs=['CONTROL', 'DEBIAN'],
>                      archive=archive,
>                  )
>
> --
> 2.37.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#168773):
> https://lists.openembedded.org/g/openembedded-core/message/168773
> Mute This Topic: https://lists.openembedded.org/mt/92767075/5052612
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> quaresma.jose@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
Best regards,

José Quaresma

[-- Attachment #2: Type: text/html, Size: 6732 bytes --]

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [OE-core][PATCH] create-spdx: ignore packing control files from ipk and deb
  2022-08-09 16:40 ` [OE-core][PATCH] create-spdx: ignore packing control files from ipk and deb Jose Quaresma
@ 2022-08-10 14:47   ` Steve Sakoman
  0 siblings, 0 replies; 3+ messages in thread
From: Steve Sakoman @ 2022-08-10 14:47 UTC (permalink / raw)
  To: Jose Quaresma; +Cc: OE-core, Ricardo Salveti, daiane.angolini, Jose Quaresma

On Tue, Aug 9, 2022 at 6:41 AM Jose Quaresma <quaresma.jose@gmail.com> wrote:
>
> Hi Steve,
>
> Can this patch be backported to kirkstone?

Yes, will do!

Steve

> Jose Quaresma via lists.openembedded.org <quaresma.jose=gmail.com@lists.openembedded.org> escreveu no dia terça, 2/08/2022 à(s) 10:29:
>>
>> Otherwise spdx can have references for data that is not packed
>> in the package delivered because this contol data is temporarly
>> and only exist while the package is been write.
>>
>> During do_package_write_ipk task in do_package_ipk the control
>> files is cleaned up at the end. This can create a race condiction
>> when the do_create_spdx task runs the add_package_files function
>> and these files is been deleted at same time in the
>> task do_package_write_ipk.
>>
>> ERROR: alsa-topology-conf-1.2.5.1-r0 do_create_spdx: Error executing a python function in exec_func_python() autogenerated:
>>
>> The stack trace of python calls that resulted in this exception/failure was:
>> File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
>>      0001:
>>  *** 0002:do_create_spdx(d)
>>      0003:
>> File: '/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass', lineno: 567, function: do_create_spdx
>>      0563:            package_doc.add_relationship(package_doc, "DESCRIBES", spdx_package)
>>      0564:
>>      0565:            package_archive = deploy_dir_spdx / "packages" / (package_doc.name + ".tar.zst")
>>      0566:            with optional_tarfile(package_archive, archive_packaged) as archive:
>>  *** 0567:                package_files = add_package_files(
>>      0568:                    d,
>>      0569:                    package_doc,
>>      0570:                    spdx_package,
>>      0571:                    pkgdest / package,
>> File: '/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass', lineno: 234, function: add_package_files
>>      0230:                            info.mtime = source_date_epoch
>>      0231:
>>      0232:                        archive.addfile(info, f)
>>      0233:
>>  *** 0234:                sha1 = bb.utils.sha1_file(filepath)
>>      0235:                sha1s.append(sha1)
>>      0236:                spdx_file.checksums.append(oe.spdx.SPDXChecksum(
>>      0237:                        algorithm="SHA1",
>>      0238:                        checksumValue=sha1,
>> File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 559, function: sha1_file
>>      0555:    """
>>      0556:    Return the hex string representation of the SHA1 checksum of the filename
>>      0557:    """
>>      0558:    import hashlib
>>  *** 0559:    return _hasher(hashlib.sha1(), filename)
>>      0560:
>>      0561:def sha384_file(filename):
>>      0562:    """
>>      0563:    Return the hex string representation of the SHA384 checksum of the filename
>> File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 528, function: _hasher
>>      0524:
>>      0525:def _hasher(method, filename):
>>      0526:    import mmap
>>      0527:
>>  *** 0528:    with open(filename, "rb") as f:
>>      0529:        try:
>>      0530:            with mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as mm:
>>      0531:                for chunk in iter(lambda: mm.read(8192), b''):
>>      0532:                    method.update(chunk)
>> Exception: FileNotFoundError: [Errno 2] No such file or directory: '/srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/packages-split/alsa-topology-conf/CONTROL/control'
>>
>> ERROR: Logfile of failure stored in: /srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/temp/log.do_create_spdx.998864
>> INFO: recipe alsa-topology-conf-1.2.5.1-r0: task do_create_spdx: Failed
>>
>> Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
>> ---
>>  meta/classes/create-spdx.bbclass | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/meta/classes/create-spdx.bbclass b/meta/classes/create-spdx.bbclass
>> index 37b6b569a1..08ef7fd409 100644
>> --- a/meta/classes/create-spdx.bbclass
>> +++ b/meta/classes/create-spdx.bbclass
>> @@ -571,6 +571,7 @@ python do_create_spdx() {
>>                      pkgdest / package,
>>                      lambda file_counter: oe.sbom.get_packaged_file_spdxid(pkg_name, file_counter),
>>                      lambda filepath: ["BINARY"],
>> +                    ignore_top_level_dirs=['CONTROL', 'DEBIAN'],
>>                      archive=archive,
>>                  )
>>
>> --
>> 2.37.1
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#168773): https://lists.openembedded.org/g/openembedded-core/message/168773
>> Mute This Topic: https://lists.openembedded.org/mt/92767075/5052612
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [quaresma.jose@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>
>
> --
> Best regards,
>
> José Quaresma


^ permalink raw reply	[flat|nested] 3+ messages in thread

* [OE-core][PATCH] create-spdx: ignore packing control files from ipk and deb
@ 2022-08-02  9:29 Jose Quaresma
  0 siblings, 0 replies; 3+ messages in thread
From: Jose Quaresma @ 2022-08-02  9:29 UTC (permalink / raw)
  To: openembedded-core; +Cc: ricardo, daiane.angolini, Jose Quaresma

Otherwise spdx can have references for data that is not packed
in the package delivered because this contol data is temporarly
and only exist while the package is been write.

During do_package_write_ipk task in do_package_ipk the control
files is cleaned up at the end. This can create a race condiction
when the do_create_spdx task runs the add_package_files function
and these files is been deleted at same time in the
task do_package_write_ipk.

ERROR: alsa-topology-conf-1.2.5.1-r0 do_create_spdx: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_create_spdx(d)
     0003:
File: '/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass', lineno: 567, function: do_create_spdx
     0563:            package_doc.add_relationship(package_doc, "DESCRIBES", spdx_package)
     0564:
     0565:            package_archive = deploy_dir_spdx / "packages" / (package_doc.name + ".tar.zst")
     0566:            with optional_tarfile(package_archive, archive_packaged) as archive:
 *** 0567:                package_files = add_package_files(
     0568:                    d,
     0569:                    package_doc,
     0570:                    spdx_package,
     0571:                    pkgdest / package,
File: '/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass', lineno: 234, function: add_package_files
     0230:                            info.mtime = source_date_epoch
     0231:
     0232:                        archive.addfile(info, f)
     0233:
 *** 0234:                sha1 = bb.utils.sha1_file(filepath)
     0235:                sha1s.append(sha1)
     0236:                spdx_file.checksums.append(oe.spdx.SPDXChecksum(
     0237:                        algorithm="SHA1",
     0238:                        checksumValue=sha1,
File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 559, function: sha1_file
     0555:    """
     0556:    Return the hex string representation of the SHA1 checksum of the filename
     0557:    """
     0558:    import hashlib
 *** 0559:    return _hasher(hashlib.sha1(), filename)
     0560:
     0561:def sha384_file(filename):
     0562:    """
     0563:    Return the hex string representation of the SHA384 checksum of the filename
File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 528, function: _hasher
     0524:
     0525:def _hasher(method, filename):
     0526:    import mmap
     0527:
 *** 0528:    with open(filename, "rb") as f:
     0529:        try:
     0530:            with mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as mm:
     0531:                for chunk in iter(lambda: mm.read(8192), b''):
     0532:                    method.update(chunk)
Exception: FileNotFoundError: [Errno 2] No such file or directory: '/srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/packages-split/alsa-topology-conf/CONTROL/control'

ERROR: Logfile of failure stored in: /srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/temp/log.do_create_spdx.998864
INFO: recipe alsa-topology-conf-1.2.5.1-r0: task do_create_spdx: Failed

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
---
 meta/classes/create-spdx.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/create-spdx.bbclass b/meta/classes/create-spdx.bbclass
index 37b6b569a1..08ef7fd409 100644
--- a/meta/classes/create-spdx.bbclass
+++ b/meta/classes/create-spdx.bbclass
@@ -571,6 +571,7 @@ python do_create_spdx() {
                     pkgdest / package,
                     lambda file_counter: oe.sbom.get_packaged_file_spdxid(pkg_name, file_counter),
                     lambda filepath: ["BINARY"],
+                    ignore_top_level_dirs=['CONTROL', 'DEBIAN'],
                     archive=archive,
                 )
 
-- 
2.37.1



^ permalink raw reply related	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2022-08-10 14:48 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <17077D238077F896.4080@lists.openembedded.org>
2022-08-09 16:40 ` [OE-core][PATCH] create-spdx: ignore packing control files from ipk and deb Jose Quaresma
2022-08-10 14:47   ` Steve Sakoman
2022-08-02  9:29 Jose Quaresma

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).