archive mirror
 help / color / mirror / Atom feed
From: Alberto Pianon <>
To: Mikko Rapeli <>
Subject: Re: [bitbake-devel] [PATCH v3 1/3] fetch2: Add support for upstream source tracing
Date: Mon, 24 Apr 2023 12:41:45 +0200	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <ZEZIwF3cCgdkOuG2@nuoska>

Hi Mikko,

thanks for your feedback.

On 2023-04-24 11:15 Mikko Rapeli wrote:
> Hi,
> On Sun, Apr 23, 2023 at 08:01:42AM +0200, Alberto Pianon wrote:
>> From: Alberto Pianon <>
>> License compliance, SBoM generation and CVE checking require to be 
>> able
>> to trace each source file back to its corresponding upstream source. 
>> The
>> current implementation of bb.fetch2 makes it difficult, especially 
>> when
>> multiple upstream sources are combined together.
> No comment to the patch itself, which seems to create a way to capture
> checksums of recipe source files which can be mapped to SRC_URI 
> entries.

Not only to SRC_URI entries but to actual upstream download locations,
especially for file:// SRC_URIs (which are local, but they usually have
an upstream source such as a git repo) and for npmsw:// and gitsm://
SRC_URIs (a single SRC_URI may map to multiple download locations).

To grasp a better idea of the final result, you may have a look at
(compressed) test data in the last patch, and at the corresponding
test cases in TraceUnpackIntegrationTest:

> Would be nice to have this as an optional feature though, unless the
> performance impact on builds is close to zero. Measurements?

bitbake core-image-full-cmdline on a 16-core 32GB-ram VM, using an 
download cache (to avoid differences due to network performance) took
41m57.043s without the patches, and 42m26.727s with the patches. That's
roughly 30s more; IMHO it seems acceptable.
Keep also in mind that source tracing is done only once and then data 
be stored somehow in sstate-cache (WIP).
BTW the thing would require some more performance testing in an adequate
testing infrastructure, could you (or others) help in this respect?

> But I see no connection toe CVE checking? The problem I have is that
> I've seen SPDX and SBOM things sold as solutions to CVE checking while
> in reality they have done nothing. Yocto has cve-check.bbclass which
> uses PN/CVE_PRODUCT and PV/CVE_VERSION to query data from CVE database
> and to generate reports about affected, patched and unpatched CVEs,
> which then also include info from patch files (CVE number, if any)
> and list of ignored CVEs from recipe metadata.
> Even if SRC_URI can be split to individual entries, and each file in
> soure tree can be mapped to exact SRC_URI entry, then what's the link
> to CVEs?
> CVEs don't map to SRC_URI entries even if they in theory could. CVEs
> don't map the exact source file checksums. Multiple versions of a 
> source
> file can be mapped to be affected by a CVE they contain the same bug,
> which is usually encoded in CVEs as SW component name and upstream
> release version range. The SRC_URI entry SW component name and version 
> are not
> added in this patch, and the CVE metadata about ignored and patched 
> CVEs
> are not exported, so I don't see any links to CVEs.

Actually the commit message could be improved in this respect. Being 
to get upstream download locations (especially with recipes mixing 
upstream sources, and with gitsm and npmsw fetchers) is a pre-condition 
identify the relevant components and therefore do CVE checks, but CVE 
as such are not covered by the patch. I may clarify that.

For example, a possible improvement in this respect could be to 
calculate not
only download locations (SPDX), but also purls 
which may be used to do CVE checks against commercial or, even better, 
databases (such as VulnerableCode



  reply	other threads:[~2023-04-24 10:41 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-04-23  6:01 [PATCH v3 1/3] fetch2: Add support for upstream source tracing alberto
2023-04-23  6:01 ` [PATCH v3 2/3] fetch2: Add metadata collection " alberto
2023-04-24  9:15 ` [bitbake-devel] [PATCH v3 1/3] fetch2: Add support " Mikko Rapeli
2023-04-24 10:41   ` Alberto Pianon [this message]
2023-04-24 11:38     ` Mikko Rapeli

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).