[Buildroot] [PATCH v3 0/3] Add tainting support to buildroot

[Buildroot] [PATCH v3 0/3] Add tainting support to buildroot

[Angelo Compagnucci]

Packages that need to resolve dependencies internally
and use a package manager would harm the reproducibility
of a build, moreover they escape the legal infrastructure
not giving enough informations on licensing.

This patch adds a tainting mechanism in the form of a
variable FOO_TAINTS that can be used to signal that
a package harms the reproducibility or licensing under
certain conditions.

This opens the door to include per language dependency
managers in buildroot.

Angelo Compagnucci (3):
  Makefile: add tainting support
  docs/manual: adding infos about tainting
  package/nodejs: taint the build on external modules

 Makefile                                | 10 ++++++++++
 docs/manual/adding-packages-generic.txt |  6 ++++++
 docs/manual/legal-notice.txt            | 12 ++++++++++++
 package/nodejs/nodejs.mk                |  1 +
 package/pkg-generic.mk                  |  8 ++++++++
 5 files changed, 37 insertions(+)

-- 
2.7.4

[Buildroot] [PATCH v3 1/3] Makefile: add tainting support

[Angelo Compagnucci]

From: Angelo Compagnucci <angelo.compagnucci@gmail.com>

Packages who harms the build reproducibility or licensing can declare
FOO_TAINTS variable. If a package taints the build it will be added
to a list of tainting packages. The build ends with a warning if the
tainting packages list is not empty. Moreover, legal info will show
a warning in presence of a tainting package.

Angelo Compagnucci <angelo@amarulasolutions.com>

Signed-off-by: Angelo Compagnucci <angelo@amarulasolutions.com>
---
 Makefile               | 10 ++++++++++
 package/pkg-generic.mk |  8 ++++++++
 2 files changed, 18 insertions(+)

diff --git a/Makefile b/Makefile
index 9d66bba..ad61130 100644
--- a/Makefile
+++ b/Makefile
@@ -758,12 +758,21 @@ endif
 
 	touch $(TARGET_DIR)/usr
 
+.PHONY: check-tainted
+check-tainted:
+ifneq ($(BR2_TAINTED_BY),)
+	$(error Your buildroot configuration is tainted by: $(BR2_TAINTED_BY))
+else
+	@echo "Your buildroot configuration is not tainted"
+endif
+
 .PHONY: target-post-image
 target-post-image: $(TARGETS_ROOTFS) target-finalize
 	@rm -f $(ROOTFS_COMMON_TAR)
 	@$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_IMAGE_SCRIPT)), \
 		$(call MESSAGE,"Executing post-image script $(s)"); \
 		$(EXTRA_ENV) $(s) $(BINARIES_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep))
+	$(if $(BR2_TAINTED_BY), at echo "WARNING: Your buildroot configuration is tainted by: $(BR2_TAINTED_BY).")
 
 .PHONY: source
 source: $(foreach p,$(PACKAGES),$(p)-all-source)
@@ -1070,6 +1079,7 @@ help:
 	@echo '  source                 - download all sources needed for offline-build'
 	@echo '  external-deps          - list external packages used'
 	@echo '  legal-info             - generate info about license compliance'
+	@echo '  check-tainted          - check if any selected package harms build reproducibility or licensing'
 	@echo '  printvars              - dump all the internal variables'
 	@echo
 	@echo '  make V=0|1             - 0 => quiet build (default), 1 => verbose build'
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 91b61c6..bcb4acd 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -551,6 +551,10 @@ ifndef $(2)_REDISTRIBUTE
  endif
 endif
 
+ifdef $(2)_TAINTS
+BR2_TAINTED_BY+=$$($(2)_RAWNAME)
+endif
+
 $(2)_REDISTRIBUTE		?= YES
 
 $(2)_REDIST_SOURCES_DIR = $$(REDIST_SOURCES_DIR_$$(call UPPERCASE,$(4)))/$$($(2)_BASENAME_RAW)
@@ -909,6 +913,10 @@ else
 	$(Q)$$(foreach F,$$($(2)_LICENSE_FILES),$$(call legal-license-file,$$($(2)_RAWNAME),$$($(2)_BASENAME_RAW),$$($(2)_PKGDIR),$$(F),$$($(2)_DIR)/$$(F),$$(call UPPERCASE,$(4)))$$(sep))
 endif # license files
 
+ifeq ($$(call qstrip,$$($(2)_TAINTS)),YES)
+	$(Q)$$(call legal-warning-pkg,$$($(2)_RAWNAME),unknown license for additional modules or dependencies)
+endif
+
 ifeq ($$($(2)_SITE_METHOD),local)
 # Packages without a tarball: don't save and warn
 	@$$(call legal-warning-nosource,$$($(2)_RAWNAME),local)
-- 
2.7.4

[Buildroot] [PATCH v3 2/3] docs/manual: adding infos about tainting

[Angelo Compagnucci]

Adding documentation about the usage of LIBFOO_TAINTS and
"make check-tainted".

Signed-off-by: Angelo Compagnucci <angelo@amarulasolutions.com>
---
 docs/manual/adding-packages-generic.txt |  6 ++++++
 docs/manual/legal-notice.txt            | 12 ++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt
index 7be1754..8f159ad 100644
--- a/docs/manual/adding-packages-generic.txt
+++ b/docs/manual/adding-packages-generic.txt
@@ -445,6 +445,12 @@ not and can not work as people would expect it should:
   to let you know, and +not saved+ will appear in the +license files+ field
   of the manifest file for this package.
 
+* +LIBFOO_TAINTS+ shoud be set to YES if a package taints a Buildroot
+  configuration. A Buildroot configuration is tainted when a packages uses
+  external dependencies for which Buildroot cannot clearly recover licensing
+  informations. If a configuration is tainted, it means that the licensing
+  information produced by +make legal-info+ could not be accurate.
+
 * +LIBFOO_ACTUAL_SOURCE_TARBALL+ only applies to packages whose
   +LIBFOO_SITE+ / +LIBTOO_SOURCE+ pair points to an archive that does
   not actually contain source code, but binary code. This a very
diff --git a/docs/manual/legal-notice.txt b/docs/manual/legal-notice.txt
index 6975328..7fde09a 100644
--- a/docs/manual/legal-notice.txt
+++ b/docs/manual/legal-notice.txt
@@ -73,6 +73,18 @@ distribution is required).
 When you run +make legal-info+, Buildroot produces warnings in the +README+
 file to inform you of relevant material that could not be saved.
 
+Furthermore, a Buildroot configuration could be tainted from a package that uses
+some custom external dependencies from the Buildroot tree. An example could be
+a package manager for a software stack that downloads the required dependencies
+during the building of a package. In such cases, Buildroot cannot check the
+licensing of the downloaded software and thus giving accurate licensing
+informations.
+To check if your configuration is tainted, run:
+
+--------------------
+make check-tainted
+--------------------
+
 Finally, keep in mind that the output of +make legal-info+ is based on
 declarative statements in each of the packages recipes. The Buildroot
 developers try to do their best to keep those declarative statements as
-- 
2.7.4

[Buildroot] [PATCH v3 3/3] package/nodejs: taint the build on external modules

[Angelo Compagnucci]

From: Angelo Compagnucci <angelo.compagnucci@gmail.com>

This patch enables the tainting of the build when an
external module is added.

Signed-off-by: Angelo Compagnucci <angelo@amarulasolutions.com>
---
 package/nodejs/nodejs.mk | 1 +
 1 file changed, 1 insertion(+)

diff --git a/package/nodejs/nodejs.mk b/package/nodejs/nodejs.mk
index e2c94ba..322a1ec 100644
--- a/package/nodejs/nodejs.mk
+++ b/package/nodejs/nodejs.mk
@@ -160,6 +160,7 @@ NPM = $(TARGET_CONFIGURE_OPTS) \
 # We can only call NPM if there's something to install.
 #
 ifneq ($(NODEJS_MODULES_LIST),)
+NODEJS_TAINTS = YES
 define NODEJS_INSTALL_MODULES
 	# If you're having trouble with module installation, adding -d to the
 	# npm install call below and setting npm_config_rollback=false can both
-- 
2.7.4

[Buildroot] [PATCH v3 1/3] Makefile: add tainting support

[Thomas Petazzoni]

Hello,

+Yann/Arnout in Cc.

On Wed,  5 Sep 2018 18:06:34 +0200, Angelo Compagnucci wrote:

> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index 91b61c6..bcb4acd 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -551,6 +551,10 @@ ifndef $(2)_REDISTRIBUTE
>   endif
>  endif
>  
> +ifdef $(2)_TAINTS
> +BR2_TAINTED_BY+=$$($(2)_RAWNAME)
> +endif

I was almost going to apply and push this, but then I found a fairly
major problem. With your implementation, as soon as a package sets
<pkg>_TAINTS = YES, it will be part of the BR2_TAINTED_BY list,
regardless of whether the package is enabled or not.

It happens to work OK with nodejs, because the NODEJS_TAINTS = YES is
inside a condition that NODEJS_MODULES_LIST != "". However, if you put
NODEJS_TAINTS = YES outside of that condition in nodejs.mk, you will
see that the build is always tainted by nodejs, regardless of whether
nodejs is enabled or not. Not good.

This can easily be fixed by moving the BR2_TAINTED_BY += line inside
the:

  ifeq ($$($$($(2)_KCONFIG_VAR)),y)

condition, which is true only when the package is really enabled.

However, this only works for target packages. What about host
packages ? Can they taint the build ? If so, how do we handle that ?

When I was about to apply, I had added something like this in
pkg-generic.mk:

ifndef $(2)_TAINTS
 ifdef $(3)_TAINTS
  $(2)_TAINTS = $$($(3)_TAINTS)
 endif
endif

so that HOST_<pkg>_TAINTS is automatically defined to the same value as
<pkg>_TAINTS. However, we have no way to know if a host package is
enabled or not, since most host packages don't have any
corresponding Config.in option.

So: do we care about host packages for "tainting" ? If we do care, how
do we handle this ?

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v3 1/3] Makefile: add tainting support

[Angelo Compagnucci]

Hi Thomas,

On Wed, Sep 5, 2018 at 9:37 PM, Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
> Hello,
>
> +Yann/Arnout in Cc.
>
> On Wed,  5 Sep 2018 18:06:34 +0200, Angelo Compagnucci wrote:
>
>> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
>> index 91b61c6..bcb4acd 100644
>> --- a/package/pkg-generic.mk
>> +++ b/package/pkg-generic.mk
>> @@ -551,6 +551,10 @@ ifndef $(2)_REDISTRIBUTE
>>   endif
>>  endif
>>
>> +ifdef $(2)_TAINTS
>> +BR2_TAINTED_BY+=$$($(2)_RAWNAME)
>> +endif
>
> I was almost going to apply and push this, but then I found a fairly
> major problem. With your implementation, as soon as a package sets
> <pkg>_TAINTS = YES, it will be part of the BR2_TAINTED_BY list,
> regardless of whether the package is enabled or not.
>
> It happens to work OK with nodejs, because the NODEJS_TAINTS = YES is
> inside a condition that NODEJS_MODULES_LIST != "". However, if you put
> NODEJS_TAINTS = YES outside of that condition in nodejs.mk, you will
> see that the build is always tainted by nodejs, regardless of whether
> nodejs is enabled or not. Not good.

Thank you so much for finding this.

> This can easily be fixed by moving the BR2_TAINTED_BY += line inside
> the:
>
>   ifeq ($$($$($(2)_KCONFIG_VAR)),y)
>
> condition, which is true only when the package is really enabled.

Yes, it works like a charm!

> However, this only works for target packages. What about host
> packages ? Can they taint the build ? If so, how do we handle that ?
>
> When I was about to apply, I had added something like this in
> pkg-generic.mk:
>
> ifndef $(2)_TAINTS
>  ifdef $(3)_TAINTS
>   $(2)_TAINTS = $$($(3)_TAINTS)
>  endif
> endif
>
> so that HOST_<pkg>_TAINTS is automatically defined to the same value as
> <pkg>_TAINTS. However, we have no way to know if a host package is
> enabled or not, since most host packages don't have any
> corresponding Config.in option.
>
> So: do we care about host packages for "tainting" ? If we do care, how
> do we handle this ?

I think we can demand the decision to when we will have a host package
that needs tainting support.

Moreover, I don't think the demand for host packages that needs
tainting support will be high in the future.

I'll send an updated patch.

>
> Best regards,
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com

[Buildroot] [PATCH v3 1/3] Makefile: add tainting support

[Thomas Petazzoni]

Hello,

On Wed, 5 Sep 2018 23:45:47 +0200, Angelo Compagnucci wrote:

> > So: do we care about host packages for "tainting" ? If we do care, how
> > do we handle this ?  
> 
> I think we can demand the decision to when we will have a host package
> that needs tainting support.
> 
> Moreover, I don't think the demand for host packages that needs
> tainting support will be high in the future.

Then we need to make that explicit:

ifeq ($$($(2)_TYPE),host)
ifneq ($$($(2)_TAINTS),)
$$(error "Host package $(1) has $(2)_TAINTS set: not supported)
endif
endif

of course, please check that this really works as expected.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v3 1/3] Makefile: add tainting support

[Angelo Compagnucci]

Hi Thomas,

On Thu, Sep 6, 2018 at 12:01 AM, Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
> Hello,
>
> On Wed, 5 Sep 2018 23:45:47 +0200, Angelo Compagnucci wrote:
>
>> > So: do we care about host packages for "tainting" ? If we do care, how
>> > do we handle this ?
>>
>> I think we can demand the decision to when we will have a host package
>> that needs tainting support.
>>
>> Moreover, I don't think the demand for host packages that needs
>> tainting support will be high in the future.
>
> Then we need to make that explicit:
>
> ifeq ($$($(2)_TYPE),host)
> ifneq ($$($(2)_TAINTS),)
> $$(error "Host package $(1) has $(2)_TAINTS set: not supported)
> endif
> endif
>
> of course, please check that this really works as expected.

Done!

>
> Best regards,
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com