RE: [bitbake-devel] [PATCH v2 1/3] upstream source tracing: base process

[Peter Kjellerstedt <peter.kjellerstedt@axis.com>]

> -----Original Message-----
> From: bitbake-devel@lists.openembedded.org <bitbake-
> devel@lists.openembedded.org> On Behalf Of Alberto Pianon
> Sent: den 21 april 2023 14:53
> To: Luca Ceresoli <luca.ceresoli@bootlin.com>
> Cc: bitbake-devel@lists.openembedded.org
> Subject: Re: [bitbake-devel] [PATCH v2 1/3] upstream source tracing: base
> process
> 
> Il 2023-04-21 09:28 Luca Ceresoli ha scritto:
> > Hello Alberto,
> >
> > On Fri, 21 Apr 2023 09:05:23 +0200
> > "Alberto Pianon" <alberto@pianon.eu> wrote:
> >
> >> From: Alberto Pianon <alberto@pianon.eu>
> >>
> >> License compliance, SBoM generation and CVE checking require to be
> >> able
> >> to trace each source file back to its corresponding upstream source.
> >> The
> >> current implementation of bb.fetch2 makes it difficult, especially
> >> when
> >> multiple sources are combined together.
> >>
> >> This patch provides an interface to solve the issue by implementing a
> >> process that unpacks each SRC_URI element into a temporary directory,
> >> creates an entrypoint to collect relevant provenance metadata on each
> >> source file, moves everything to the recipe rootdir, and saves
> >> metadata
> >> in a JSON file.
> >>
> >> This patch contains required modifications to fetchers' code plus a
> >> TraceUnpackBase class that implements the above described process.
> >> Data
> >> collection logic should be separately implemented by subclassing
> >> TraceUnpackBase, implementing _collect_data() and _process_data()
> >> methods.
> >>
> >> Splitting the above described solution in multiple patches aims
> >> at easing review and merge process, and also at decoupling the
> >> development of the data processing logic from the process that enables
> >> it.
> >>
> >> Signed-off-by: Alberto Pianon <alberto@pianon.eu>
> >
> > Thank you for having fixed the subject! Now I can apply your patches
> > with a clean git commit.
> >
> > Do you think the build failures I reported on the v1 series are fixed
> > by this v2? If you think so, then I can test v2 on the autobuilders.
> 
> No but I'm working on it. Actually, I should have found the problem:
> it's just that in some tests WORKDIR/temp is not created so I added
> a check to create it if it's not present. I'm testing it, but
> oe-selftest is taking forever in my build machine. So far, no error
> in the logs...
> 
> >
> > Also, I see this patch set is not bisectable as well as v1: you define
> > class TraceUnpack in patch 2 and use it in patch 1.
> >
> 
> do you mean that:
> 
> import TraceUnpackBase as TraceUnpack
> 
> in patch 1 is not good, and that I should use just TraceUnpackBase
> as a class name/alias in patch 1?

My guess is that Luca missed the `import ... as` line. I believe you 
should be fine as you are. However, one thing I would point out is the 
commit subjects. For changes to the fetcher, the subject should 
typically start with "fetch2: " so that one can clearly see with, e.g., 
`git log --oneline` where you are making changes. If it was up to me, 
I would change the subjects for the three commits to:

  fetch2: Add support for upstream source tracing
  fetch2: Add metadata collection for upstream source tracing
  fetch2: Add tests for upstream source tracing

//Peter